Introduce virtual PMD for Hyper-V/Azure platforms
Subsystems (2)
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Virtual machines hosted by Hyper-V/Azure platforms are fitted with simplified virtual network devices named NetVSC that are used for fast communication between VM to VM, VM to hypervisor, and the outside.
They appear as standard system netdevices to user-land applications, the main difference being they are implemented on top of VMBUS [1] instead of emulated PCI devices.
While this reads like a case for a standard DPDK PMD, there is more to it.
To accelerate outside communication, NetVSC devices as they appear in a VM can be paired with physical SR-IOV virtual function (VF) devices owned by that same VM [2]. Both netdevices share the same MAC address in that case.
When paired, egress and most of the ingress traffic flow through the VF device, while part of it (e.g. multicasts, hypervisor control data) still flows through NetVSC. Moreover VF devices are not retained and disappear during VM migration; from a VM standpoint, they can be hot-plugged anytime with NetVSC acting as a fallback.
Running DPDK applications in such a context involves driving VF devices using their dedicated PMDs in a vendor-independent fashion (to benefit from maximum performance without writing dedicated code) while simultaneously listening to NetVSC and handling the related hot-plug events.
This new virtual PMD (referred to as "hyper-v" from this point on) automatically coordinates the Hyper-V/Azure-specific management part described above by relying on vendor-specific, fail-safe and tap PMDs to expose a single consolidated Ethernet device usable directly by existing applications, as summarized by the following diagram:
.-------------.
| DPDK ethdev |
`------+------'
|
.------+------.
| hyper-v PMD |
`------+------'
|
.------------+------------.
| fail-safe PMD |
`--+-------------------+--'
| |
| .........|.........
| : | :
.----+----. : .------+------. :
| tap PMD | : | $vendor PMD | :
`----+----' : `------+------' :--- hot-pluggable
| : | :
.------+-------. : .-----+-----. :
| NetVSC-based | : | SR-IOV VF | :
| netdevice | : | device | :
`--------------' : `-----------' :
:.................:Given this RFC targets DPDK 18.02, this approach has the least impact on applications while work is being performed to enhance public DPDK APIs to improve it (e.g. hot-plug notification, vdev bus scanning and so on).
Some highlights:
- Enables existing applications to run unmodified with maximum performance on Hyper-V/Azure platforms.
- All changes should be restricted to the hyper-v PMD (possibly a few in fail-safe PMD), no API change in DPDK.
- Modular approach with little maintenance overhead (not much code) that will rely on existing PMDs for all the heavy lifting.
[1] http://dpdk.org/ml/archives/dev/2017-January/054165.html
[2] https://docs.microsoft.com/en-us/windows-hardware/drivers/network/overview-of-hyper-v
Signature · 2 lines
Adrien Mazarguil 6WIND
cover Introduce virtual PMD for Hyper-V/Azure platforms
Virtual machines hosted by Hyper-V/Azure platforms are fitted with simplified virtual network devices named NetVSC that are used for fast communication between VM to VM, VM to hypervisor, and the outside.
They appear as standard system netdevices to user-land applications, the main difference being they are implemented on top of VMBUS [1] instead of emulated PCI devices.
While this reads like a case for a standard DPDK PMD, there is more to it.
To accelerate outside communication, NetVSC devices as they appear in a VM can be paired with physical SR-IOV virtual function (VF) devices owned by that same VM [2]. Both netdevices share the same MAC address in that case.
When paired, egress and most of the ingress traffic flow through the VF device, while part of it (e.g. multicasts, hypervisor control data) still flows through NetVSC. Moreover VF devices are not retained and disappear during VM migration; from a VM standpoint, they can be hot-plugged anytime with NetVSC acting as a fallback.
Running DPDK applications in such a context involves driving VF devices using their dedicated PMDs in a vendor-independent fashion (to benefit from maximum performance without writing dedicated code) while simultaneously listening to NetVSC and handling the related hot-plug events.
This new virtual PMD (referred to as "hyperv" from this point on) automatically coordinates the Hyper-V/Azure-specific management part described above by relying on vendor-specific, failsafe and tap PMDs to expose a single consolidated Ethernet device usable directly by existing applications.
.------------------.
| DPDK application |
`--------+---------'
|
.------+------.
| DPDK ethdev |
`------+------' Control
| |
.------------+------------. v .------------.
| failsafe PMD +---------+ hyperv PMD |
`--+-------------------+--' `------------'
| |
| .........|.........
| : | :
.----+----. : .----+----. :
| tap PMD | : | any PMD | :
`----+----' : `----+----' : <-- Hot-pluggable
| : | :
.------+-------. : .-----+-----. :
| NetVSC-based | : | SR-IOV VF | :
| netdevice | : | device | :
`--------------' : `-----------' :
:.................:Note this diagram differs from that of the original RFC [3], with hyperv no longer acting as a data plane layer.
This initial version of the driver only works in whitelist mode. Users have to provide the --vdev net_hyperv EAL option at least once to trigger it.
Subsequent work will add support for blacklist mode based on automatic detection of the host environment.
[1] http://dpdk.org/ml/archives/dev/2017-January/054165.html
[2] https://docs.microsoft.com/en-us/windows-hardware/drivers/network/overview-of-hyper-v
[3] http://dpdk.org/ml/archives/dev/2017-November/082339.html
Adrien Mazarguil (3):
net/hyperv: introduce MS Hyper-V platform driver net/hyperv: implement core functionality net/hyperv: add "force" parameter
MAINTAINERS | 6 + config/common_base | 6 + config/common_linuxapp | 1 + doc/guides/nics/features/hyperv.ini | 12 + doc/guides/nics/hyperv.rst | 119 +++ doc/guides/nics/index.rst | 1 + drivers/net/Makefile | 1 + drivers/net/hyperv/Makefile | 58 ++ drivers/net/hyperv/hyperv.c | 799 +++++++++++++++++++++ drivers/net/hyperv/rte_pmd_hyperv_version.map | 4 + mk/rte.app.mk | 1 + 11 files changed, 1008 insertions(+) create mode 100644 doc/guides/nics/features/hyperv.ini create mode 100644 doc/guides/nics/hyperv.rst create mode 100644 drivers/net/hyperv/Makefile create mode 100644 drivers/net/hyperv/hyperv.c create mode 100644 drivers/net/hyperv/rte_pmd_hyperv_version.map
Signature · 1 line
2.11.0
PATCH 1/3 net/hyperv: introduce MS Hyper-V platform driver
This patch lays the groundwork for this PMD (draft documentation, copyright notices, code base skeleton and build system hooks). While it can be successfully compiled and invoked, it's an empty shell at this stage.
--- MAINTAINERS | 6 + config/common_base | 6 + config/common_linuxapp | 1 + doc/guides/nics/features/hyperv.ini | 12 ++ doc/guides/nics/hyperv.rst | 49 ++++++++ doc/guides/nics/index.rst | 1 + drivers/net/Makefile | 1 + drivers/net/hyperv/Makefile | 54 +++++++++ drivers/net/hyperv/hyperv.c | 135 +++++++++++++++++++++ drivers/net/hyperv/rte_pmd_hyperv_version.map | 4 + mk/rte.app.mk | 1 + 11 files changed, 270 insertions(+)
MAINTAINERS +6 -0
@@ -451,6 +451,12 @@ F: drivers/net/mrvl/ F: doc/guides/nics/mrvl.rst F: doc/guides/nics/features/mrvl.ini +Microsoft hyperv +M: Adrien Mazarguil <[email protected]> +F: drivers/net/hyperv/ +F: doc/guides/nics/hyperv.rst +F: doc/guides/nics/features/hyperv.ini + Netcope szedata2 M: Matej Vido <[email protected]> F: drivers/net/szedata2/
config/common_base +6 -0
@@ -280,6 +280,12 @@ CONFIG_RTE_LIBRTE_NFP_DEBUG=n CONFIG_RTE_LIBRTE_MRVL_PMD=n # +# Compile Microsoft Hyper-V/Azure driver +# +CONFIG_RTE_LIBRTE_HYPERV_PMD=n +CONFIG_RTE_LIBRTE_HYPERV_DEBUG=n + +# # Compile burst-oriented Broadcom BNXT PMD driver # CONFIG_RTE_LIBRTE_BNXT_PMD=y
config/common_linuxapp +1 -0
@@ -47,6 +47,7 @@ CONFIG_RTE_LIBRTE_PMD_VHOST=y CONFIG_RTE_LIBRTE_PMD_AF_PACKET=y CONFIG_RTE_LIBRTE_PMD_TAP=y CONFIG_RTE_LIBRTE_AVP_PMD=y +CONFIG_RTE_LIBRTE_HYPERV_PMD=y CONFIG_RTE_LIBRTE_NFP_PMD=y CONFIG_RTE_LIBRTE_POWER=y CONFIG_RTE_VIRTIO_USER=y
doc/guides/nics/features/hyperv.ini new file +12 -0
@@ -0,0 +1,12 @@ +; +; Supported features of the 'hyperv' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +ARMv7 = Y +ARMv8 = Y +Power8 = Y +x86-32 = Y +x86-64 = Y +Usage doc = Y
doc/guides/nics/hyperv.rst new file +49 -0
49 lines not on this page · raw message
doc/guides/nics/index.rst +1 -0
@@ -49,6 +49,7 @@ Network Interface Controller Drivers ena enic fm10k + hyperv i40e ixgbe intel_vf
drivers/net/Makefile +1 -0
@@ -55,6 +55,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_LIO_PMD) += liquidio DIRS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) += mlx4 DIRS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5 DIRS-$(CONFIG_RTE_LIBRTE_MRVL_PMD) += mrvl +DIRS-$(CONFIG_RTE_LIBRTE_HYPERV_PMD) += hyperv DIRS-$(CONFIG_RTE_LIBRTE_NFP_PMD) += nfp DIRS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += bnxt DIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += null
drivers/net/hyperv/Makefile new file +54 -0
54 lines not on this page · raw message
drivers/net/hyperv/hyperv.c new file +135 -0
135 lines not on this page · raw message
drivers/net/hyperv/rte_pmd_hyperv_version.map new file +4 -0
@@ -0,0 +1,4 @@ +DPDK_18.02 { + + local: *; +};
mk/rte.app.mk +1 -0
@@ -134,6 +134,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_E1000_PMD) += -lrte_pmd_e1000 _LDLIBS-$(CONFIG_RTE_LIBRTE_ENA_PMD) += -lrte_pmd_ena _LDLIBS-$(CONFIG_RTE_LIBRTE_ENIC_PMD) += -lrte_pmd_enic _LDLIBS-$(CONFIG_RTE_LIBRTE_FM10K_PMD) += -lrte_pmd_fm10k +_LDLIBS-$(CONFIG_RTE_LIBRTE_HYPERV_PMD) += -lrte_pmd_hyperv _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_FAILSAFE) += -lrte_pmd_failsafe _LDLIBS-$(CONFIG_RTE_LIBRTE_I40E_PMD) += -lrte_pmd_i40e _LDLIBS-$(CONFIG_RTE_LIBRTE_IXGBE_PMD) += -lrte_pmd_ixgbe
Signature · 1 line
2.11.0
Re: [PATCH v1 1/3] net/hyperv: introduce MS Hyper-V platform driver
Quoted 15 lines · On Mon, 18 Dec 2017 17:46:21 +0100 Adrien Mazarguil…
drivers/net/hyperv/hyperv.c new file +15
-0+#ifdef RTE_LIBRTE_HYPERV_DEBUG + +#define PMD_DRV_LOG (level, ...) \ + RTE_LOG(level, PMD, \ + RTE_FMT("%s:%u: %s(): " RTE_FMT_HEAD(__VA_ARGS__,) "\n", \ + strrchr("/" __FILE__, '/') + 1, \ + __LINE__, \ + __func__, \ + RTE_FMT_TAIL(__VA_ARGS__,))) + +#else /* RTE_LIBRTE_HYPERV_DEBUG */ + +#define PMD_DRV_LOG (level, ...) \ + RTE_LOG(level, PMD, \ + RTE_FMT(RTE_STR(HYPERV_DRIVER) ": " \
drivers/net/hyperv/hyperv.c line 70
Quoted 10 lines
drivers/net/hyperv/hyperv.c new file +10
-0+ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \ + RTE_FMT_TAIL(__VA_ARGS__,))) + +#endif /* RTE_LIBRTE_HYPERV_DEBUG */ + +#define DEBUG (...) PMD_DRV_LOG(DEBUG, __VA_ARGS__) +#define INFO (...) PMD_DRV_LOG(INFO, __VA_ARGS__) +#define WARN (...) PMD_DRV_LOG(WARNING, __VA_ARGS__) +#define ERROR (...) PMD_DRV_LOG(ERR, __VA_ARGS__) +
Please don't use DEBUG() etc macros. It makes it easier for tools that do global updates or scans if all drivers use the same model of PMD_DRV_LOG
Re: [PATCH v1 1/3] net/hyperv: introduce MS Hyper-V platform driver
18/12/2017 19:28, Stephen Hemminger:
Quoted 23 lines
Quoted 20 lines · On Mon, 18 Dec 2017 17:46:21 +0100 Adrien Mazarguil…
drivers/net/hyperv/hyperv.c new file +20
-0+#ifdef RTE_LIBRTE_HYPERV_DEBUG + +#define PMD_DRV_LOG (level, ...) \ + RTE_LOG(level, PMD, \ + RTE_FMT("%s:%u: %s(): " RTE_FMT_HEAD(__VA_ARGS__,) "\n", \ + strrchr("/" __FILE__, '/') + 1, \ + __LINE__, \ + __func__, \ + RTE_FMT_TAIL(__VA_ARGS__,))) + +#else /* RTE_LIBRTE_HYPERV_DEBUG */ + +#define PMD_DRV_LOG (level, ...) \ + RTE_LOG(level, PMD, \ + RTE_FMT(RTE_STR(HYPERV_DRIVER) ": " \ + RTE_FMT_HEAD(__VA_ARGS__,) "\n", \ + RTE_FMT_TAIL(__VA_ARGS__,))) + +#endif /* RTE_LIBRTE_HYPERV_DEBUG */ +
Quoted 8 lines
drivers/net/hyperv/hyperv.c line 70
Quoted 5 lines
drivers/net/hyperv/hyperv.c new file +5
-0+#define DEBUG (...) PMD_DRV_LOG(DEBUG, __VA_ARGS__) +#define INFO (...) PMD_DRV_LOG(INFO, __VA_ARGS__) +#define WARN (...) PMD_DRV_LOG(WARNING, __VA_ARGS__) +#define ERROR (...) PMD_DRV_LOG(ERR, __VA_ARGS__) +Please don't use
DEBUG()etc macros. It makes it easier for tools that do global updates or scans if all drivers use the same model ofPMD_DRV_LOG
The new standard is to use dynamic logtype.
Re: [PATCH v1 1/3] net/hyperv: introduce MS Hyper-V platform driver
Quoted 11 lines · On Mon, 18 Dec 2017 20:54:16 +0100 Thomas Monjalon <[email protected]> wrote:
Quoted 10 lines
drivers/net/hyperv/hyperv.c line 70
Quoted 7 lines
drivers/net/hyperv/hyperv.c new file +7
-0+#endif /* RTE_LIBRTE_HYPERV_DEBUG */ + +#define DEBUG (...) PMD_DRV_LOG(DEBUG, __VA_ARGS__) +#define INFO (...) PMD_DRV_LOG(INFO, __VA_ARGS__) +#define WARN (...) PMD_DRV_LOG(WARNING, __VA_ARGS__) +#define ERROR (...) PMD_DRV_LOG(ERR, __VA_ARGS__) +Please don't use
DEBUG()etc macros. It makes it easier for tools that do global updates or scans if all drivers use the same model ofPMD_DRV_LOG
Quoted 1 line
The new standard is to use dynamic logtype.
Agree, please use dynamic logging, and also don't redefine new macros like DEBUG/INFO/WARN/ERROR. Instead use PMD_DRV_LOG or equivalent macros.
The base rule here is that all drivers should look the same as much
as reasonably possible. This makes reviewers of other subsystems more likely to see problems. It also allows for later changes where some developer does a global improvement across many PMD's.
Drivers should not be snowflakes, each one is not unique.
Re: [PATCH v1 1/3] net/hyperv: introduce MS Hyper-V platform driver
Quoted 14 lines · Stephen Hemminger
Quoted 11 lines · On Mon, 18 Dec 2017 20:54:16 +0100 Thomas Monjalon <[email protected]> wrote:
Quoted 10 lines
drivers/net/hyperv/hyperv.c line 70
Quoted 7 lines
drivers/net/hyperv/hyperv.c new file +7
-0+#endif /* RTE_LIBRTE_HYPERV_DEBUG */ + +#define DEBUG (...) PMD_DRV_LOG(DEBUG, __VA_ARGS__) +#define INFO (...) PMD_DRV_LOG(INFO, __VA_ARGS__) +#define WARN (...) PMD_DRV_LOG(WARNING, __VA_ARGS__) +#define ERROR (...) PMD_DRV_LOG(ERR, __VA_ARGS__) +Please don't use
DEBUG()etc macros. It makes it easier for tools that do global updates or scans if all drivers use the same model ofPMD_DRV_LOG
Quoted 4 lines
Quoted 1 line
The new standard is to use dynamic logtype.
Agree, please use dynamic logging, and also don't redefine new macros like DEBUG/INFO/WARN/ERROR. Instead use
PMD_DRV_LOGor equivalent macros.
Wait, the above definitions are only convenience wrappers to PMD_DRV_LOG(), itself a wrapper to RTE_LOG(), itself a wrapper to rte_log(), their presence is not triggered according to compilation options, did I miss something?
Let me bring back some context from the original patch:
#ifdef RTE_LIBRTE_HYPERV_DEBUG
#define PMD_DRV_LOG(level, ...) \
RTE_LOG(level, PMD, \
RTE_FMT("%s:%u: %s(): " RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
strrchr("/" __FILE__, '/') + 1, \
__LINE__, \
__func__, \
RTE_FMT_TAIL(__VA_ARGS__,)))
#else /* RTE_LIBRTE_HYPERV_DEBUG */
#define PMD_DRV_LOG(level, ...) \
RTE_LOG(level, PMD, \
RTE_FMT(RTE_STR(HYPERV_DRIVER) ": " \
RTE_FMT_HEAD(__VA_ARGS__,) "\n", \
RTE_FMT_TAIL(__VA_ARGS__,)))
#endif /* RTE_LIBRTE_HYPERV_DEBUG */
#define DEBUG(...) PMD_DRV_LOG(DEBUG, __VA_ARGS__) #define INFO(...) PMD_DRV_LOG(INFO, __VA_ARGS__) #define WARN(...) PMD_DRV_LOG(WARNING, __VA_ARGS__) #define ERROR(...) PMD_DRV_LOG(ERR, __VA_ARGS__)
Enabling RTE_LIBRTE_HYPERV_DEBUG adds file and line information to log output, messages are otherwise unaffected by that compilation option. Adding this information required some sort of wrapper to avoid needless clutter.
Nothing against outputting file/line information when compiled in debug mode right?
Quoted 6 lines
The base rule here is that all drivers should look the same as much
as reasonably possible. This makes reviewers of other subsystems more likely to see problems. It also allows for later changes where some developer does a global improvement across many PMD's.Drivers should not be snowflakes, each one is not unique.
Point taken, do you confirm replacing i.e. WARN(...) with PMD_DRV_LOG(WARN, ...) and friends is all that's needed?
Signature · 2 lines
Adrien Mazarguil 6WIND
Re: [PATCH v1 1/3] net/hyperv: introduce MS Hyper-V platform driver
19/12/2017 11:01, Adrien Mazarguil:
Quoted 51 lines · Stephen Hemminger
Quoted 14 lines
Quoted 11 lines · On Mon, 18 Dec 2017 20:54:16 +0100 Thomas Monjalon <[email protected]> wrote:
Quoted 10 lines
drivers/net/hyperv/hyperv.c line 70
Quoted 7 lines
drivers/net/hyperv/hyperv.c new file +7
-0+#endif /* RTE_LIBRTE_HYPERV_DEBUG */ + +#define DEBUG (...) PMD_DRV_LOG(DEBUG, __VA_ARGS__) +#define INFO (...) PMD_DRV_LOG(INFO, __VA_ARGS__) +#define WARN (...) PMD_DRV_LOG(WARNING, __VA_ARGS__) +#define ERROR (...) PMD_DRV_LOG(ERR, __VA_ARGS__) +Please don't use
DEBUG()etc macros. It makes it easier for tools that do global updates or scans if all drivers use the same model ofPMD_DRV_LOGQuoted 4 lines
Quoted 1 line
The new standard is to use dynamic logtype.
Agree, please use dynamic logging, and also don't redefine new macros like DEBUG/INFO/WARN/ERROR. Instead use
PMD_DRV_LOGor equivalent macros.Wait, the above definitions are only convenience wrappers to
PMD_DRV_LOG(), itself a wrapper toRTE_LOG(), itself a wrapper torte_log(), their presence is not triggered according to compilation options, did I miss something?Let me bring back some context from the original patch:
#ifdef
RTE_LIBRTE_HYPERV_DEBUG#define
PMD_DRV_LOG(level, ...)\(quoted diff) +0
-0RTE_LOG(level, PMD, \ RTE_FMT("%s:%u: %s(): " RTE_FMT_HEAD(__VA_ARGS__,) "\n", \ strrchr("/" __FILE__, '/') + 1, \ __LINE__, \ __func__, \ RTE_FMT_TAIL(__VA_ARGS__,))) #else /* RTE_LIBRTE_HYPERV_DEBUG */ #define PMD_DRV_LOG(level, ...) \ RTE_LOG(level, PMD, \ RTE_FMT(RTE_STR(HYPERV_DRIVER) ": " \ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \ RTE_FMT_TAIL(__VA_ARGS__,))) #endif /* RTE_LIBRTE_HYPERV_DEBUG */ #define DEBUG(...) PMD_DRV_LOG(DEBUG, __VA_ARGS__) #define INFO(...) PMD_DRV_LOG(INFO, __VA_ARGS__) #define WARN(...) PMD_DRV_LOG(WARNING, __VA_ARGS__) #define ERROR(...) PMD_DRV_LOG(ERR, __VA_ARGS__)
Quoted 6 lines
Enabling
RTE_LIBRTE_HYPERV_DEBUGadds file and line information to log output, messages are otherwise unaffected by that compilation option. Adding this information required some sort of wrapper to avoid needless clutter.Nothing against outputting file/line information when compiled in debug mode right?
I am not sure __FILE__, __LINE__ and __func__ are so much useful. The log message should be unique enough.
Quoted 9 lines
Quoted 6 lines
The base rule here is that all drivers should look the same as much
as reasonably possible. This makes reviewers of other subsystems more likely to see problems. It also allows for later changes where some developer does a global improvement across many PMD's.Drivers should not be snowflakes, each one is not unique.
Point taken, do you confirm replacing i.e. WARN(...) with
PMD_DRV_LOG(WARN, ...)and friends is all that's needed?
You need to remove the compile-time option for DEBUG, and rely on dynamic log type, thanks to rte_log_register().
Re: [PATCH v1 1/3] net/hyperv: introduce MS Hyper-V platform driver
Quoted 56 lines · Thomas Monjalon
19/12/2017 11:01, Adrien Mazarguil:
Quoted 55 lines · Stephen Hemminger
Quoted 14 lines
Quoted 11 lines · On Mon, 18 Dec 2017 20:54:16 +0100 Thomas Monjalon <[email protected]> wrote:
Quoted 10 lines
drivers/net/hyperv/hyperv.c line 70
Quoted 7 lines
drivers/net/hyperv/hyperv.c new file +7
-0+#endif /* RTE_LIBRTE_HYPERV_DEBUG */ + +#define DEBUG (...) PMD_DRV_LOG(DEBUG, __VA_ARGS__) +#define INFO (...) PMD_DRV_LOG(INFO, __VA_ARGS__) +#define WARN (...) PMD_DRV_LOG(WARNING, __VA_ARGS__) +#define ERROR (...) PMD_DRV_LOG(ERR, __VA_ARGS__) +Please don't use
DEBUG()etc macros. It makes it easier for tools that do global updates or scans if all drivers use the same model ofPMD_DRV_LOGQuoted 4 lines
Quoted 1 line
The new standard is to use dynamic logtype.
Agree, please use dynamic logging, and also don't redefine new macros like DEBUG/INFO/WARN/ERROR. Instead use
PMD_DRV_LOGor equivalent macros.Wait, the above definitions are only convenience wrappers to
PMD_DRV_LOG(), itself a wrapper toRTE_LOG(), itself a wrapper torte_log(), their presence is not triggered according to compilation options, did I miss something?Let me bring back some context from the original patch:
#ifdef
RTE_LIBRTE_HYPERV_DEBUG#define
PMD_DRV_LOG(level, ...)\(quoted diff) +0
-0RTE_LOG(level, PMD, \ RTE_FMT("%s:%u: %s(): " RTE_FMT_HEAD(__VA_ARGS__,) "\n", \ strrchr("/" __FILE__, '/') + 1, \ __LINE__, \ __func__, \ RTE_FMT_TAIL(__VA_ARGS__,))) #else /* RTE_LIBRTE_HYPERV_DEBUG */ #define PMD_DRV_LOG(level, ...) \ RTE_LOG(level, PMD, \ RTE_FMT(RTE_STR(HYPERV_DRIVER) ": " \ RTE_FMT_HEAD(__VA_ARGS__,) "\n", \ RTE_FMT_TAIL(__VA_ARGS__,))) #endif /* RTE_LIBRTE_HYPERV_DEBUG */ #define DEBUG(...) PMD_DRV_LOG(DEBUG, __VA_ARGS__) #define INFO(...) PMD_DRV_LOG(INFO, __VA_ARGS__) #define WARN(...) PMD_DRV_LOG(WARNING, __VA_ARGS__) #define ERROR(...) PMD_DRV_LOG(ERR, __VA_ARGS__)Enabling
RTE_LIBRTE_HYPERV_DEBUGadds file and line information to log output, messages are otherwise unaffected by that compilation option. Adding this information required some sort of wrapper to avoid needless clutter.
Quoted 5 lines
Quoted 2 lines
Nothing against outputting file/line information when compiled in debug mode right?
I am not sure
__FILE__,__LINE__and__func__are so much useful. The log message should be unique enough.
I don't share your opinion. mlx4/mlx5 PMDs output similar information when compiled in debug mode and that proved quite useful during development and when tracking down bugs.
Thing is, mere users are not the target audience, it's a development tool that doesn't need to be part of distributed binaries, hence the compilation option.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
The base rule here is that all drivers should look the same as much
as reasonably possible. This makes reviewers of other subsystems more likely to see problems. It also allows for later changes where some developer does a global improvement across many PMD's.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Drivers should not be snowflakes, each one is not unique.
Point taken, do you confirm replacing i.e. WARN(...) with
PMD_DRV_LOG(WARN, ...)and friends is all that's needed?You need to remove the compile-time option for DEBUG, and rely on dynamic log type, thanks to
rte_log_register().
OK, I didn't know about rte_log_register() which may explain some of the confusion, I'll add it in v2 then.
To summarize what needs to be done for v2:
- Call rte_log_register() during init.
- Use its return value in place of the second argument to RTE_LOG().
- Replace DEBUG/WARN/INFO/ERROR() wrappers with direct calls to PMD_DRV_LOG() for consistency with other PMDs.
- Finally, remove debugging code/information and related compilation option since they're useless to end users.
Signature · 2 lines
Adrien Mazarguil 6WIND
PATCH 2/3 net/hyperv: implement core functionality
In reply to Adrien MazarguilAs described in more details in the attached documentation (see patch contents), this virtual device driver manages NetVSC interfaces in virtual machines hosted by Hyper-V/Azure platforms.
This driver does not manage traffic nor Ethernet devices directly; it acts as a thin configuration layer that automatically instantiates and controls fail-safe PMD instances combining tap and PCI sub-devices, so that each NetVSC interface is exposed as a single consolidated port to DPDK applications.
PCI sub-devices being hot-pluggable (e.g. during VM migration), applications automatically benefit from increased throughput when present and automatic fallback on NetVSC otherwise without interruption thanks to fail-safe's hot-plug handling.
Once initialized, the sole job of the hyperv driver is to regularly scan for PCI devices to associate with NetVSC interfaces and feed their addresses to corresponding fail-safe instances.
--- doc/guides/nics/hyperv.rst | 65 ++++ drivers/net/hyperv/Makefile | 4 + drivers/net/hyperv/hyperv.c | 654 ++++++++++++++++++++++++++++++++++++++- 3 files changed, 722 insertions(+), 1 deletion(-)
doc/guides/nics/hyperv.rst +65 -0
@@ -37,6 +37,50 @@ machines running on Microsoft Hyper-V_ (including Azure) platforms. .. _Hyper-V: https://docs.microsoft.com/en-us/windows-hardware/drivers/network/overview-of-hyper-v +Implementation details +---------------------- + +Each instance of this driver effectively needs to drive two devices: the +NetVSC interface proper and its SR-IOV VF (referred to as "physical" from +this point on) counterpart sharing the same MAC address. + +Physical devices are part of the host system and cannot be maintained during +VM migration. From a VM standpoint they appear as hot-plug devices that come +and go without prior notice. + +When the physical device is present, egress and most of the ingress traffic +flows through it; only multicasts and other hypervisor control still flow +through NetVSC. Otherwise, NetVSC acts as a fallback for all traffic. + +To avoid unnecessary code duplication and ensure maximum performance, +handling of physical devices is left to their original PMDs; this virtual +device driver (also known as *vdev*) manages other PMDs as summarized by the +following block diagram:: + + .------------------. + | DPDK application | + `--------+---------' + | + .------+------. + | DPDK ethdev | + `------+------' Control + | | + .------------+------------. v .------------. + | failsafe PMD +---------+ hyperv PMD | + `--+-------------------+--' `------------' + | | + | .........|......... + | : | : + .----+----. : .----+----. : + | tap PMD | : | any PMD | : + `----+----' : `----+----' : <-- Hot-pluggable + | : | : + .------+-------. : .-----+-----. : + | NetVSC-based | : | SR-IOV VF | : + | netdevice | : | device | : + `--------------' : `-----------' : + :.................: + Build options -------------
@@ -47,3 +91,24 @@ Build options - ``CONFIG_RTE_LIBRTE_HYPERV_DEBUG`` (default ``n``) Toggle additional debugging code. + +Run-time parameters +------------------- + +To invoke this PMD, applications have to explicitly provide the +``--vdev=net_hyperv`` EAL option. + +The following device parameters are supported: + +- ``iface`` [string] + + Provide a specific NetVSC interface (netdevice) name to attach this PMD + to. Can be provided multiple times for additional instances. + +- ``mac`` [string] + + Same as ``iface`` except a suitable NetVSC interface is located using its + MAC address. + +Not specifying either ``iface`` or ``mac`` makes this PMD attach itself to +all NetVSC interfaces found on the system.
drivers/net/hyperv/Makefile +4 -0
@@ -40,6 +40,9 @@ EXPORT_MAP := rte_pmd_hyperv_version.map CFLAGS += -O3 CFLAGS += -g CFLAGS += -std=c11 -pedantic -Wall -Wextra +CFLAGS += -D_XOPEN_SOURCE=600 +CFLAGS += -D_BSD_SOURCE +CFLAGS += -D_DEFAULT_SOURCE CFLAGS += $(WERROR_FLAGS) # Dependencies.
@@ -47,6 +50,7 @@ LDLIBS += -lrte_bus_vdev LDLIBS += -lrte_eal LDLIBS += -lrte_ethdev LDLIBS += -lrte_kvargs +LDLIBS += -lrte_net # Source files. SRCS-$(CONFIG_RTE_LIBRTE_HYPERV_PMD) += hyperv.c
drivers/net/hyperv/hyperv.c +653 -1
707 lines not on this page · raw message
Signature · 1 line
2.11.0
Re: [PATCH v1 2/3] net/hyperv: implement core functionality
Quoted 241 lines · Adrien Mazarguil
As described in more details in the attached documentation (see patch contents), this virtual device driver manages NetVSC interfaces in virtual machines hosted by Hyper-V/Azure platforms.
This driver does not manage traffic nor Ethernet devices directly; it acts as a thin configuration layer that automatically instantiates and controls fail-safe PMD instances combining tap and PCI sub-devices, so that each NetVSC interface is exposed as a single consolidated port to DPDK applications.
PCI sub-devices being hot-pluggable (e.g. during VM migration), applications automatically benefit from increased throughput when present and automatic fallback on NetVSC otherwise without interruption thanks to fail-safe's hot-plug handling.
Once initialized, the sole job of the hyperv driver is to regularly scan for PCI devices to associate with NetVSC interfaces and feed their addresses to corresponding fail-safe instances.
Signed-off-by Adrien Mazarguil <[email protected]>---
doc/guides/nics/hyperv.rst | 65 ++++ drivers/net/hyperv/Makefile | 4 + drivers/net/hyperv/hyperv.c | 654 ++++++++++++++++++++++++++++++++++++++- 3 files changed, 722 insertions(+), 1 deletion(-)doc/guides/nics/hyperv.rst +0
-0@@ -37,6 +37,50 @@ machines running on Microsoft Hyper-V_ (including Azure) platforms... _Hyper-V: https://docs.microsoft.com/en-us/windows-hardware/drivers/network/overview-of-hyper-v
doc/guides/nics/hyperv.rst +44
-0+Implementation details +---------------------- + +Each instance of this driver effectively needs to drive two devices: the +NetVSC interface proper and its SR-IOV VF (referred to as "physical" from +this point on) counterpart sharing the same MAC address. + +Physical devices are part of the host system and cannot be maintained during +VM migration. From a VM standpoint they appear as hot-plug devices that come +and go without prior notice. + +When the physical device is present, egress and most of the ingress traffic +flows through it; only multicasts and other hypervisor control still flow +through NetVSC. Otherwise, NetVSC acts as a fallback for all traffic. + +To avoid unnecessary code duplication and ensure maximum performance, +handling of physical devices is left to their original PMDs; this virtual +device driver (also known as *vdev*) manages other PMDs as summarized by the +following block diagram:: + + .------------------. + | DPDK application | + `--------+---------' + | + .------+------. + | DPDK ethdev | + `------+------' Control + | | + .------------+------------. v .------------. + | failsafe PMD +---------+ hyperv PMD | + `--+-------------------+--' `------------' + | | + | .........|......... + | : | : + .----+----. : .----+----. : + | tap PMD | : | any PMD | : + `----+----' : `----+----' : <-- Hot-pluggable + | : | : + .------+-------. : .-----+-----. : + | NetVSC-based | : | SR-IOV VF | : + | netdevice | : | device | : + `--------------' : `-----------' : + :.................: +Build options
-------------doc/guides/nics/hyperv.rst +21
-1@@ -47,3 +91,24 @@ Build options - ``CONFIG_RTE_LIBRTE_HYPERV_DEBUG`` (default ``n``) Toggle additional debugging code. + +Run-time parameters +------------------- + +To invoke this PMD, applications have to explicitly provide the +``--vdev=net_hyperv`` EAL option. + +The following device parameters are supported: + +- ``iface`` [string] + + Provide a specific NetVSC interface (netdevice) name to attach this PMD + to. Can be provided multiple times for additional instances. + +- ``mac`` [string] + + Same as ``iface`` except a suitable NetVSC interface is located using its + MAC address. + +Not specifying either ``iface`` or ``mac`` makes this PMD attach itself to +all NetVSC interfaces found on the system.drivers/net/hyperv/Makefile +0
-0@@ -40,6 +40,9 @@ EXPORT_MAP := rte_pmd_hyperv_version.mapCFLAGS += -O3
CFLAGS += -g
CFLAGS += -std=c11 -pedantic -Wall -Wextradrivers/net/hyperv/Makefile +3
-0+CFLAGS += -D_XOPEN_SOURCE=600 +CFLAGS += -D_BSD_SOURCE +CFLAGS += -D_DEFAULT_SOURCECFLAGS += $(WERROR_FLAGS)
# Dependencies.
drivers/net/hyperv/Makefile +0
-0@@ -47,6 +50,7 @@ LDLIBS += -lrte_bus_vdevLDLIBS += -lrte_eal
LDLIBS += -lrte_ethdev
LDLIBS += -lrte_kvargs
+LDLIBS += -lrte_net# Source files.
SRCS-$(CONFIG_RTE_LIBRTE_HYPERV_PMD) +=hyperv.cdrivers/net/hyperv/hyperv.c +83
-0@@ -31,17 +31,40 @@* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ +#include <errno.h> +#include <fcntl.h> +#include <linux/sockios.h> +#include <net/if.h> +#include <netinet/ip.h> +#include <stdarg.h> #include <stddef.h> +#include <stdlib.h> +#include <stdint.h> +#include <stdio.h> #include <string.h> +#include <sys/ioctl.h> +#include <sys/queue.h> +#include <sys/socket.h> +#include <unistd.h> +#include <rte_alarm.h> +#include <rte_bus.h> #include <rte_bus_vdev.h> +#include <rte_common.h> #include <rte_config.h> +#include <rte_dev.h> +#include <rte_errno.h> +#include <rte_ethdev.h> +#include <rte_ether.h> #include <rte_kvargs.h> #include <rte_log.h> #define HYPERV_DRIVER net_hyperv#define HYPERV_ARG_IFACE "iface"#define HYPERV_ARG_MAC "mac" +#define HYPERV_PROBE_MS 1000 + +#define NETVSC_CLASS_ID "{f8615163-df3e-46c5-913f-f2d2f965ed0e}"#ifdef RTE_LIBRTE_HYPERV_DEBUG @@ -68,12 +91,603 @@#define WARN (...) PMD_DRV_LOG(WARNING, __VA_ARGS__)#define ERROR (...) PMD_DRV_LOG(ERR, __VA_ARGS__) +/** +* Convert a MAC address string to binary form. +* +* Note: this function should be exposed by rte_ether.h as the reverse of +* ether_format_addr(). +* +* Several MAC string formats are supported on input for convenience: +* +* 1. "12:34:56:78:9a:bc" +* 2. "12-34-56-78-9a-bc" +* 3. "123456789abc" +* 4. Upper/lowercase hexadecimal. +* 5. Any combination of the above, e.g. "12:34-5678-9aBC". +* 6. Partial addresses are allowed, with low-order bytes filled first: +* - "5:6:78c" translates to "00:00:05:06:07:8c", +* - "5678c" translates to "00:00:00:05:67:8c". +* +* Non-hexadecimal characters, unknown separators and strings specifying +* more than 6 bytes are not allowed. +* +* @param[out] eth_addr +* Pointer to conversion result buffer. +* @param[in] str +* MAC address string to convert. +* +* @return +* 0 on success, -EINVAL in case of unsupported format. +*/ +static int +ether_addr_from_str (struct ether_addr *eth_addr,const char *str) +{ +static const uint8_t conv[0x100 ] = { + ['0' ] =0x80 , ['1' ] =0x81 , ['2' ] =0x82 , ['3' ] =0x83 , + ['4' ] =0x84 , ['5' ] =0x85 , ['6' ] =0x86 , ['7' ] =0x87 , + ['8' ] =0x88 , ['9' ] =0x89 , ['a' ] =0x8a , ['b' ] =0x8b , + ['c' ] =0x8c , ['d' ] =0x8d , ['e' ] =0x8e , ['f' ] =0x8f , + ['A' ] =0x8a , ['B' ] =0x8b , ['C' ] =0x8c , ['D' ] =0x8d , + ['E' ] =0x8e , ['F' ] =0x8f , [':' ] =0x40 , ['-' ] =0x40 , + ['\0' ] =0x60 , + }; +uint64_t addr =0 ; +uint64_t buf =0 ; +unsigned int i =0 ; +unsigned int n =0 ; +uint8_t tmp; + + do { + tmp = conv[(int)*(str++)]; + if (!tmp) + return -EINVAL; + if (tmp & 0x40) { + i += (i & 1) + (!i << 1); + addr = (addr << (i << 2)) | buf; + n += i; + buf = 0; + i = 0; + } else { + buf = (buf << 4) | (tmp & 0xf); + ++i; + }
drivers/net/hyperv/hyperv.c line 163
Quoted 10 lines
drivers/net/hyperv/hyperv.c +10
-0+ }while (!(tmp &0x20 )); +if (n >12 ) +return -EINVAL ; + i =RTE_DIM (eth_addr->addr_bytes); +while (i) { + eth_addr->addr_bytes[--i] = addr &0xff ; + addr >>=8 ; + } +return 0 ; +}
You already called this out above, why not just push this into rte_ether.h file. I know I could use it if it were public.
Quoted 153 lines
drivers/net/hyperv/hyperv.c +149
-0+ +/** Context structure for a hyperv instance. */ +struct hyperv_ctx { +LIST_ENTRY (hyperv_ctx) entry;/**< Next entry in list. */ +unsigned int id;/**< ID used to generate unique names. */ +char name[64 ];/**< Unique name for hyperv instance. */ +char devname[64 ];/**< Fail-safe PMD instance name. */ +char devargs[256 ];/**< Fail-safe PMD instance device arguments. */ +char if_name[IF_NAMESIZE ];/**< NetVSC netdevice name. */ +unsigned int if_index;/**< NetVSC netdevice index. */ +struct ether_addr if_addr;/**< NetVSC MAC address. */ +int pipe[2 ];/**< Communication pipe with fail-safe instance. */ +char yield[256 ];/**< Current device string used with fail-safe. */ +}; + +/** Context list is common to all PMD instances. */ +static LIST_HEAD(, hyperv_ctx) hyperv_ctx_list = +LIST_HEAD_INITIALIZER (hyperv_ctx_list); + +/** Number of entries in context list. */ +static unsigned int hyperv_ctx_count; +/** Number of PMD instances relying on context list. */ static unsigned int hyperv_ctx_inst;/** +* Destroy a hyperv context instance. +* +* @param ctx +* Context to destroy. +*/ +static void +hyperv_ctx_destroy (struct hyperv_ctx *ctx) +{ +if (ctx->pipe[0 ] !=-1 ) +close (ctx->pipe[0 ]); +if (ctx->pipe[1 ] !=-1 ) +close (ctx->pipe[1 ]); +/* Poisoning for debugging purposes. */ +memset (ctx,0x22 ,sizeof (*ctx)); +free (ctx); +} + +/** +* Iterate over system network interfaces. +* +* This function runs a given callback function for each netdevice found on +* the system. +* +* @param func +* Callback function pointer. List traversal is aborted when this function +* returns a nonzero value. +* @param ... +* Variable parameter list passed as @p va_list to @p func. +* +* @return +* 0 when the entire list is traversed successfully, a negative error code +* in case or failure, or the nonzero value returned by @p func when list +* traversal is aborted. +*/ +static int +hyperv_foreach_iface (int (*func)(const struct if_nameindex *iface, +const struct ether_addr *eth_addr, +va_list ap), ...) +{ +struct if_nameindex *iface =if_nameindex (); +int s =socket (PF_INET ,SOCK_DGRAM ,IPPROTO_IP ); +unsigned int i; +int ret =0 ; + +if (!iface) { + ret = -ENOBUFS ; +ERROR ("cannot retrieve system network interfaces" ); + goto error; + } +if (s ==-1 ) { + ret = -errno; +ERROR ("cannot open socket: %s" ,rte_strerror (errno)); + goto error; + } +for (i =0 ; iface[i].if_name; ++i) { +struct ifreq req; +struct ether_addr eth_addr; +va_list ap; + +strncpy (req.ifr_name, iface[i].if_name,sizeof (req.ifr_name)); +if (ioctl (s,SIOCGIFHWADDR , &req) ==-1 ) { +WARN ("cannot retrieve information about interface" +" \"%s\": %s" , + req.ifr_name,rte_strerror (errno)); +continue ; + } +memcpy (eth_addr.addr_bytes, req.ifr_hwaddr.sa_data, +RTE_DIM (eth_addr.addr_bytes)); +va_start (ap, func); + ret =func (&iface[i], ð_addr, ap); +va_end (ap); +if (ret) +break ; + } +error: +if (s !=-1 ) +close (s); +if (iface) +if_freenameindex (iface); +return ret; +} + +/** +* Determine if a network interface is NetVSC. +* +* @param[in] iface +* Pointer to netdevice description structure (name and index). +* +* @return +* A nonzero value when interface is detected as NetVSC. In case of error, +* rte_errno is updated and 0 returned. +*/ +static int +hyperv_iface_is_netvsc (const struct if_nameindex *iface) +{ +static const char temp[] ="/sys/class/net/%s/device/class_id" ; +char path[snprintf (NULL ,0 , temp, iface->if_name) +1 ]; +FILE *f; +int ret; +int len =0 ; + +snprintf (path,sizeof (path), temp, iface->if_name); + f =fopen (path,"r" ); +if (!f) { + rte_errno = errno; +return 0 ; + } + ret =fscanf (f,NETVSC_CLASS_ID "%n" , &len); +if (ret ==EOF ) + rte_errno = errno; + ret = len == (int )strlen (NETVSC_CLASS_ID ); +fclose (f); +return ret; +} + +/** +* Retrieve the last component of a path. +* +* This is a simplified basename() that does not modify its input buffer to +* handle trailing backslashes. +* +* @param[in] path +* Path to retrieve the last component from. +* +* @return +* Pointer to the last component. +*/
drivers/net/hyperv/hyperv.c line 326
Quoted 10 lines
drivers/net/hyperv/hyperv.c +10
-0+static const char * +hyperv_basename (const char *path) +{ +const char *tmp = path; + +while (*tmp) +if (*(tmp++) =='/' ) + path = tmp; +return path; +}
Why not just user rindex() to find the last ‘/‘ instead of this routine? I know it is not performance critical.
Quoted 118 lines
drivers/net/hyperv/hyperv.c +118
-0+ +/** +* Retrieve network interface data from sysfs symbolic link. +* +* @param[out] buf +* Output data buffer. +* @param size +* Output buffer size. +* @param[in] if_name +* Netdevice name. +* @param[in] relpath +* Symbolic link path relative to netdevice sysfs entry. +* +* @return +* 0 on success, a negative error code otherwise. +*/ +static int +hyperv_sysfs_readlink (char *buf,size_t size,const char *if_name, +const char *relpath) +{ +int ret; + + ret =snprintf (buf, size,"/sys/class/net/%s/%s" , if_name, relpath); +if (ret ==-1 || (size_t )ret >= size -1 ) +return -ENOBUFS ; + ret =readlink (buf, buf, size); +if (ret ==-1 ) +return -errno; +if ((size_t )ret >= size -1 ) +return -ENOBUFS ; + buf[ret] ='\0' ; +return 0 ; +} + +/** +* Probe a network interface to associate with hyperv context. +* +* This function determines if the network device matches the properties of +* the NetVSC interface associated with the hyperv context and communicates +* its bus address to the fail-safe PMD instance if so. +* +* It is normally used with hyperv_foreach_iface(). +* +* @param[in] iface +* Pointer to netdevice description structure (name and index). +* @param[in] eth_addr +* MAC address associated with @p iface. +* @param ap +* Variable arguments list comprising: +* +* - struct hyperv_ctx *ctx: +* Context to associate network interface with. +* +* @return +* A nonzero value when interface matches, 0 otherwise or in case of +* error. +*/ +static int +hyperv_device_probe (const struct if_nameindex *iface, +const struct ether_addr *eth_addr, +va_list ap) +{ + struct hyperv_ctx *ctx = va_arg(ap, struct hyperv_ctx *); +char buf[RTE_MAX (sizeof (ctx->yield),256u )]; +const char *addr; +size_t len; +int ret; + +/* Skip non-matching or unwanted NetVSC interfaces. */ +if (ctx->if_index == iface->if_index) { +if (!strcmp (ctx->if_name, iface->if_name)) +return 0 ; +DEBUG ("NetVSC interface \"%s\" (index %u) renamed \"%s\"" , + ctx->if_name, ctx->if_index, iface->if_name); +strncpy (ctx->if_name, iface->if_name,sizeof (ctx->if_name)); +return 0 ; + } +if (hyperv_iface_is_netvsc (iface)) +return 0 ; +if (!is_same_ether_addr (eth_addr, &ctx->if_addr)) +return 0 ; +/* Look for associated PCI device. */ + ret =hyperv_sysfs_readlink (buf,sizeof (buf), iface->if_name, +"device/subsystem" ); +if (ret) +return 0 ; +if (strcmp (hyperv_basename (buf),"pci" )) +return 0 ; + ret =hyperv_sysfs_readlink (buf,sizeof (buf), iface->if_name, +"device" ); +if (ret) +return 0 ; + addr =hyperv_basename (buf); + len =strlen (addr); +if (!len) +return 0 ; +/* Send PCI device argument to fail-safe PMD instance if updated. */ +if (!strcmp (addr, ctx->yield)) +return 1 ; +DEBUG ("associating PCI device \"%s\" with NetVSC interface \"%s\"" +" (index %u)" , + addr, ctx->if_name, ctx->if_index); +memmove (buf, addr, len +1 ); + addr = buf; + buf[len] ='\n' ; + ret =write (ctx->pipe[1 ], addr, len +1 ); + buf[len] ='\0' ; +if (ret ==-1 ) { +if (errno ==EINTR || errno ==EAGAIN ) +return 1 ; +WARN ("cannot associate PCI device name \"%s\" with interface" +" \"%s\": %s" , + addr, ctx->if_name,rte_strerror (errno)); +return 1 ; + } +if ((size_t )ret != len +1 ) { +/* +* Attempt to override previous partial write, no need to
drivers/net/hyperv/hyperv.c line 454
Quoted 10 lines
drivers/net/hyperv/hyperv.c +10
-0+* recover if that fails. +*/ + ret =write (ctx->pipe[1 ],"\n" ,1 ); + (void )ret; +return 1 ; + } +fsync (ctx->pipe[1 ]); +memcpy (ctx->yield, addr, len +1 ); +return 1 ; +}
Not to criticize style, but a few blank lines could help in readability for these files IMHO. Unless blank lines are illegal :-)
Quoted 308 lines
drivers/net/hyperv/hyperv.c +260
-0+ +/** +* Alarm callback that regularly probes system network interfaces. +* +* This callback runs at a frequency determined by HYPERV_PROBE_MS as long +* as an hyperv context instance exists. +* +* @param arg +* Ignored. +*/ +static void +hyperv_alarm (void *arg) +{ +struct hyperv_ctx *ctx; +int ret; + + (void )arg; + LIST_FOREACH(ctx, &hyperv_ctx_list, entry) { + ret =hyperv_foreach_iface (hyperv_device_probe, ctx); +if (ret) +break ; + } +if (!hyperv_ctx_count) +return ; + ret =rte_eal_alarm_set (HYPERV_PROBE_MS *1000 , hyperv_alarm,NULL ); +if (ret <0 ) { +ERROR ("unable to reschedule alarm callback: %s" , +rte_strerror (-ret)); + } +} + +/** +* Probe a NetVSC interface to generate a hyperv context from. +* +* This function instantiates hyperv contexts either for all NetVSC devices +* found on the system or only a subset provided as device arguments. +* +* It is normally used with hyperv_foreach_iface(). +* +* @param[in] iface +* Pointer to netdevice description structure (name and index). +* @param[in] eth_addr +* MAC address associated with @p iface. +* @param ap +* Variable arguments list comprising: +* +* - const char *name: +* Name associated with current driver instance. +* +* - struct rte_kvargs *kvargs: +* Device arguments provided to current driver instance. +* +* - unsigned int specified: +* Number of specific netdevices provided as device arguments. +* +* - unsigned int *matched: +* The number of specified netdevices matched by this function. +* +* @return +* A nonzero value when interface matches, 0 otherwise or in case of +* error. +*/ +static int +hyperv_netvsc_probe (const struct if_nameindex *iface, +const struct ether_addr *eth_addr, +va_list ap) +{ + const char *name = va_arg(ap, const char *); + struct rte_kvargs *kvargs = va_arg(ap, struct rte_kvargs *); + unsigned int specified = va_arg(ap, unsigned int); + unsigned int *matched = va_arg(ap, unsigned int *); +unsigned int i; +struct hyperv_ctx *ctx; +uint16_t port_id; +int ret; + +/* Probe all interfaces when none are specified. */ +if (specified) { +for (i =0 ; i != kvargs->count; ++i) { +const struct rte_kvargs_pair *pair = &kvargs->pairs[i]; + +if (!strcmp (pair->key,HYPERV_ARG_IFACE )) { +if (!strcmp (pair->value, iface->if_name)) +break ; + }else if (!strcmp (pair->key,HYPERV_ARG_MAC )) { +struct ether_addr tmp; + +if (ether_addr_from_str (&tmp, pair->value)) { +ERROR ("invalid MAC address format" +" \"%s\"" , + pair->value); +return -EINVAL ; + } +if (!is_same_ether_addr (eth_addr, &tmp)) +break ; + } + } +if (i == kvargs->count) +return 0 ; + ++(*matched); + } +/* Weed out interfaces already handled. */ + LIST_FOREACH(ctx, &hyperv_ctx_list, entry) +if (ctx->if_index == iface->if_index) +break ; +if (ctx) { +if (!specified) +return 0 ; +WARN ("interface \"%s\" (index %u) is already handled, skipping" , + iface->if_name, iface->if_index); +return 0 ; + } +if (!hyperv_iface_is_netvsc (iface)) { +if (!specified) +return 0 ; +WARN ("interface \"%s\" (index %u) is not NetVSC, skipping" , + iface->if_name, iface->if_index); +return 0 ; + } +/* Create interface context. */ + ctx =calloc (1 ,sizeof (*ctx)); +if (!ctx) { + ret = -errno; +ERROR ("cannot allocate context for interface \"%s\": %s" , + iface->if_name,rte_strerror (errno)); + goto error; + } + ctx->id = hyperv_ctx_count; +strncpy (ctx->if_name, iface->if_name,sizeof (ctx->if_name)); + ctx->if_index = iface->if_index; + ctx->if_addr = *eth_addr; + ctx->pipe[0 ] =-1 ; + ctx->pipe[1 ] =-1 ; + ctx->yield[0 ] ='\0' ; +if (pipe (ctx->pipe) ==-1 ) { + ret = -errno; +ERROR ("cannot allocate control pipe for interface \"%s\": %s" , + ctx->if_name,rte_strerror (errno)); + goto error; + } +for (i =0 ; i !=RTE_DIM (ctx->pipe); ++i) { +int flf =fcntl (ctx->pipe[i],F_GETFL ); +int fdf =fcntl (ctx->pipe[i],F_GETFD ); + +if (flf !=-1 && +fcntl (ctx->pipe[i],F_SETFL , flf |O_NONBLOCK ) !=-1 && + fdf !=-1 && +fcntl (ctx->pipe[i],F_SETFD , + i ? fdf |FD_CLOEXEC : fdf & ~FD_CLOEXEC ) !=-1 ) +continue ; + ret = -errno; +ERROR ("cannot toggle non-blocking or close-on-exec flags on" +" control file descriptor #%u (%d): %s" , + i, ctx->pipe[i],rte_strerror (errno)); + goto error; + } +/* Generate virtual device name and arguments. */ + i =0 ; + ret =snprintf (ctx->name,sizeof (ctx->name),"%s_id%u" , + name, ctx->id); +if (ret ==-1 || (size_t )ret >=sizeof (ctx->name) -1 ) + ++i; + ret =snprintf (ctx->devname,sizeof (ctx->devname),"net_failsafe_%s" , + ctx->name); +if (ret ==-1 || (size_t )ret >=sizeof (ctx->devname) -1 ) + ++i; +/* +* Note: bash replaces the default sh interpreter used by popen() +* because as seen with dash, POSIX-compliant shells do not +* necessarily support redirections with file descriptor numbers +* above 9. +*/ + ret =snprintf (ctx->devargs,sizeof (ctx->devargs), +"exec(exec bash -c " +"'while read -r tmp <&%u 2> /dev/null;" +" do dev=$tmp; done;" +" echo $dev" +"'),dev(net_tap_%s,remote=%s)" , + ctx->pipe[0 ], ctx->name, ctx->if_name); +if (ret ==-1 || (size_t )ret >=sizeof (ctx->devargs) -1 ) + ++i; +if (i) { + ret = -ENOBUFS ; +ERROR ("generated virtual device name or argument list too long" +" for interface \"%s\"" , ctx->if_name); + goto error; + } +/* +* Remove any competing rte_eth_dev entries sharing the same MAC +* address, fail-safe instances created by this PMD will handle them +* as sub-devices later. +*/ +RTE_ETH_FOREACH_DEV (port_id) { +struct rte_device *dev = rte_eth_devices[port_id].device; +struct rte_bus *bus =rte_bus_find_by_device (dev); +struct ether_addr tmp; + +rte_eth_macaddr_get (port_id, &tmp); +if (!is_same_ether_addr (eth_addr, &tmp)) +continue ; +WARN ("removing device \"%s\" with identical MAC address to" +" re-create it as a fail-safe sub-device" , + dev->name); +if (!bus) + ret = -EINVAL ; +else + ret =rte_eal_hotplug_remove (bus->name, dev->name); +if (ret <0 ) { +ERROR ("unable to remove device \"%s\": %s" , + dev->name,rte_strerror (-ret)); + goto error; + } + } +/* Request virtual device generation. */ +DEBUG ("generating virtual device \"%s\" with arguments \"%s\"" , + ctx->devname, ctx->devargs); + ret =rte_eal_hotplug_add ("vdev" , ctx->devname, ctx->devargs); +if (ret) + goto error; +LIST_INSERT_HEAD (&hyperv_ctx_list, ctx, entry); + ++hyperv_ctx_count; +DEBUG ("added NetVSC interface \"%s\" to context list" , ctx->if_name); +return 0 ; +error: +if (ctx) +hyperv_ctx_destroy (ctx); +return ret; +} + +/** * Probe NetVSC interfaces. * +* This function probes system netdevices according to the specified device +* arguments and starts a periodic alarm callback to notify the resulting +* fail-safe PMD instances of their sub-devices whereabouts. +* * @param dev * Virtual device context for PMD instance. * @@ -92,12 +706,38 @@ hyperv_vdev_probe(struct rte_vdev_device *dev)const char *args =rte_vdev_device_args (dev);struct rte_kvargs *kvargs =rte_kvargs_parse (args ? args :"" , hyperv_arg); +unsigned int specified =0 ; +unsigned int matched =0 ; +unsigned int i; +int ret;DEBUG ("invoked as \"%s\", using arguments \"%s\"" , name, args);if (!kvargs) {ERROR ("cannot parse arguments list" ); goto error; } +for (i =0 ; i != kvargs->count; ++i) { +const struct rte_kvargs_pair *pair = &kvargs->pairs[i]; + +if (!strcmp (pair->key,HYPERV_ARG_IFACE ) || + !strcmp (pair->key,HYPERV_ARG_MAC )) + ++specified; + } +rte_eal_alarm_cancel (hyperv_alarm,NULL ); +/* Gather interfaces. */ + ret =hyperv_foreach_iface (hyperv_netvsc_probe, name, kvargs, + specified, &matched); +if (ret <0 ) + goto error; +if (matched < specified) +WARN ("some of the specified parameters did not match valid" +" network interfaces" ); + ret =rte_eal_alarm_set (HYPERV_PROBE_MS *1000 , hyperv_alarm,NULL ); +if (ret <0 ) { +ERROR ("unable to schedule alarm callback: %s" , +rte_strerror (-ret)); + goto error; + }error:
if (kvargs) rte_kvargs_free(kvargs);drivers/net/hyperv/hyperv.c +13
-1@@ -108,6 +748,9 @@ hyperv_vdev_probe(struct rte_vdev_device *dev)/** * Remove PMD instance. * +* The alarm callback and underlying hyperv context instances are only +* destroyed after the last PMD instance is removed. +* * @param dev * Virtual device context for PMD instance. * @@ -118,7 +761,16 @@ static int hyperv_vdev_remove(struct rte_vdev_device *dev) { (void )dev; - --hyperv_ctx_inst; +if (--hyperv_ctx_inst) +return 0 ; +rte_eal_alarm_cancel (hyperv_alarm,NULL ); +while (!LIST_EMPTY (&hyperv_ctx_list)) { +struct hyperv_ctx *ctx =LIST_FIRST (&hyperv_ctx_list); + +LIST_REMOVE (ctx, entry); + --hyperv_ctx_count; +hyperv_ctx_destroy (ctx); + }return 0 ; }Signature · 1 line
2.11.0
Regards,
Keith
Re: [PATCH v1 2/3] net/hyperv: implement core functionality
Quoted 27 lines · Keith
Quoted 27 lines · Adrien Mazarguil
As described in more details in the attached documentation (see patch contents), this virtual device driver manages NetVSC interfaces in virtual machines hosted by Hyper-V/Azure platforms.
This driver does not manage traffic nor Ethernet devices directly; it acts as a thin configuration layer that automatically instantiates and controls fail-safe PMD instances combining tap and PCI sub-devices, so that each NetVSC interface is exposed as a single consolidated port to DPDK applications.
PCI sub-devices being hot-pluggable (e.g. during VM migration), applications automatically benefit from increased throughput when present and automatic fallback on NetVSC otherwise without interruption thanks to fail-safe's hot-plug handling.
Once initialized, the sole job of the hyperv driver is to regularly scan for PCI devices to associate with NetVSC interfaces and feed their addresses to corresponding fail-safe instances.
Signed-off-by Adrien Mazarguil <[email protected]>---
doc/guides/nics/hyperv.rst | 65 ++++ drivers/net/hyperv/Makefile | 4 + drivers/net/hyperv/hyperv.c | 654 ++++++++++++++++++++++++++++++++++++++- 3 files changed, 722 insertions(+), 1 deletion(-)
<snip>
Quoted 4 lines
Quoted 4 lines
drivers/net/hyperv/hyperv.c +0
-0
<snip>
Quoted 64 lines
Quoted 64 lines
drivers/net/hyperv/hyperv.c +64
-0+/** +* Convert a MAC address string to binary form. +* +* Note: this function should be exposed by rte_ether.h as the reverse of +* ether_format_addr(). +* +* Several MAC string formats are supported on input for convenience: +* +* 1. "12:34:56:78:9a:bc" +* 2. "12-34-56-78-9a-bc" +* 3. "123456789abc" +* 4. Upper/lowercase hexadecimal. +* 5. Any combination of the above, e.g. "12:34-5678-9aBC". +* 6. Partial addresses are allowed, with low-order bytes filled first: +* - "5:6:78c" translates to "00:00:05:06:07:8c", +* - "5678c" translates to "00:00:00:05:67:8c". +* +* Non-hexadecimal characters, unknown separators and strings specifying +* more than 6 bytes are not allowed. +* +* @param[out] eth_addr +* Pointer to conversion result buffer. +* @param[in] str +* MAC address string to convert. +* +* @return +* 0 on success, -EINVAL in case of unsupported format. +*/ +static int +ether_addr_from_str (struct ether_addr *eth_addr,const char *str) +{ +static const uint8_t conv[0x100 ] = { + ['0' ] =0x80 , ['1' ] =0x81 , ['2' ] =0x82 , ['3' ] =0x83 , + ['4' ] =0x84 , ['5' ] =0x85 , ['6' ] =0x86 , ['7' ] =0x87 , + ['8' ] =0x88 , ['9' ] =0x89 , ['a' ] =0x8a , ['b' ] =0x8b , + ['c' ] =0x8c , ['d' ] =0x8d , ['e' ] =0x8e , ['f' ] =0x8f , + ['A' ] =0x8a , ['B' ] =0x8b , ['C' ] =0x8c , ['D' ] =0x8d , + ['E' ] =0x8e , ['F' ] =0x8f , [':' ] =0x40 , ['-' ] =0x40 , + ['\0' ] =0x60 , + }; +uint64_t addr =0 ; +uint64_t buf =0 ; +unsigned int i =0 ; +unsigned int n =0 ; +uint8_t tmp; + +do { + tmp = conv[(int )*(str++)]; +if (!tmp) +return -EINVAL ; +if (tmp &0x40 ) { + i += (i &1 ) + (!i <<1 ); + addr = (addr << (i <<2 )) | buf; + n += i; + buf =0 ; + i =0 ; + }else { + buf = (buf <<4 ) | (tmp &0xf ); + ++i; + } + }while (!(tmp &0x20 )); +if (n >12 ) +return -EINVAL ; + i =RTE_DIM (eth_addr->addr_bytes);
Quoted 8 lines
drivers/net/hyperv/hyperv.c line 163
Quoted 6 lines
drivers/net/hyperv/hyperv.c +6
-0+while (i) { + eth_addr->addr_bytes[--i] = addr &0xff ; + addr >>=8 ; + } +return 0 ; +}You already called this out above, why not just push this into
rte_ether.hfile. I know I could use it if it were public.
Hehe, that was to highlight how this driver didn't require any modifications in public APIs. I planned to do just that in v2 or in a subsequent patch.
<snip>
Quoted 16 lines
Quoted 16 lines
drivers/net/hyperv/hyperv.c +16
-0+/** +* Retrieve the last component of a path. +* +* This is a simplified basename() that does not modify its input buffer to +* handle trailing backslashes. +* +* @param[in] path +* Path to retrieve the last component from. +* +* @return +* Pointer to the last component. +*/ +static const char * +hyperv_basename (const char *path) +{ +const char *tmp = path;
Quoted 8 lines
drivers/net/hyperv/hyperv.c line 326
Quoted 6 lines
drivers/net/hyperv/hyperv.c +6
-0+ +while (*tmp) +if (*(tmp++) =='/' ) + path = tmp; +return path; +}Why not just user
rindex()to find the last ‘/‘ instead of this routine? I know it is not performance critical.
Right, however both rindex() and strrchr() return NULL when no '/' is present. strchrnul() works but is GNU-specific (i.e. probably not found on BSD), I didn't want to perform an additional check for that, so actually given the size of that function I didn't give it a second thought. I can modify that if needed.
<snip>
Quoted 88 lines
Quoted 88 lines
drivers/net/hyperv/hyperv.c +88
-0+/** +* Probe a network interface to associate with hyperv context. +* +* This function determines if the network device matches the properties of +* the NetVSC interface associated with the hyperv context and communicates +* its bus address to the fail-safe PMD instance if so. +* +* It is normally used with hyperv_foreach_iface(). +* +* @param[in] iface +* Pointer to netdevice description structure (name and index). +* @param[in] eth_addr +* MAC address associated with @p iface. +* @param ap +* Variable arguments list comprising: +* +* - struct hyperv_ctx *ctx: +* Context to associate network interface with. +* +* @return +* A nonzero value when interface matches, 0 otherwise or in case of +* error. +*/ +static int +hyperv_device_probe (const struct if_nameindex *iface, +const struct ether_addr *eth_addr, +va_list ap) +{ + struct hyperv_ctx *ctx = va_arg(ap, struct hyperv_ctx *); +char buf[RTE_MAX (sizeof (ctx->yield),256u )]; +const char *addr; +size_t len; +int ret; + +/* Skip non-matching or unwanted NetVSC interfaces. */ +if (ctx->if_index == iface->if_index) { +if (!strcmp (ctx->if_name, iface->if_name)) +return 0 ; +DEBUG ("NetVSC interface \"%s\" (index %u) renamed \"%s\"" , + ctx->if_name, ctx->if_index, iface->if_name); +strncpy (ctx->if_name, iface->if_name,sizeof (ctx->if_name)); +return 0 ; + } +if (hyperv_iface_is_netvsc (iface)) +return 0 ; +if (!is_same_ether_addr (eth_addr, &ctx->if_addr)) +return 0 ; +/* Look for associated PCI device. */ + ret =hyperv_sysfs_readlink (buf,sizeof (buf), iface->if_name, +"device/subsystem" ); +if (ret) +return 0 ; +if (strcmp (hyperv_basename (buf),"pci" )) +return 0 ; + ret =hyperv_sysfs_readlink (buf,sizeof (buf), iface->if_name, +"device" ); +if (ret) +return 0 ; + addr =hyperv_basename (buf); + len =strlen (addr); +if (!len) +return 0 ; +/* Send PCI device argument to fail-safe PMD instance if updated. */ +if (!strcmp (addr, ctx->yield)) +return 1 ; +DEBUG ("associating PCI device \"%s\" with NetVSC interface \"%s\"" +" (index %u)" , + addr, ctx->if_name, ctx->if_index); +memmove (buf, addr, len +1 ); + addr = buf; + buf[len] ='\n' ; + ret =write (ctx->pipe[1 ], addr, len +1 ); + buf[len] ='\0' ; +if (ret ==-1 ) { +if (errno ==EINTR || errno ==EAGAIN ) +return 1 ; +WARN ("cannot associate PCI device name \"%s\" with interface" +" \"%s\": %s" , + addr, ctx->if_name,rte_strerror (errno)); +return 1 ; + } +if ((size_t )ret != len +1 ) { +/* +* Attempt to override previous partial write, no need to +* recover if that fails. +*/ + ret =write (ctx->pipe[1 ],"\n" ,1 ); + (void )ret;
Quoted 8 lines
drivers/net/hyperv/hyperv.c line 454
Quoted 6 lines
drivers/net/hyperv/hyperv.c +6
-0+return 1 ; + } +fsync (ctx->pipe[1 ]); +memcpy (ctx->yield, addr, len +1 ); +return 1 ; +}Not to criticize style, but a few blank lines could help in readability for these files IMHO. Unless blank lines are illegal :-)
It's a matter of taste, I think people tend to add random blank lines where they think doing so clarifies things for themselves, resulting in inconsistent coding style not much clearer for everyone after several iterations.
As a maintainer I've grown tired of discussions related to blank lines while reviewing patches. That's why except for a few special cases, I now enforce exactly the bare minimum of one blank line between variable declarations and the rest of the code inside each block.
If doing so makes a function unreadable then perhaps it needs to be split :) I'm sure you'll understand!
Regards,
Signature · 2 lines
Adrien Mazarguil 6WIND
Re: [PATCH v1 2/3] net/hyperv: implement core functionality
Quoted 11 lines · Adrien Mazarguil
Quoted 2 lines
Not to criticize style, but a few blank lines could help in readability for these files IMHO. Unless blank lines are illegal :-)
It's a matter of taste, I think people tend to add random blank lines where they think doing so clarifies things for themselves, resulting in inconsistent coding style not much clearer for everyone after several iterations.
Quoted 7 lines
As a maintainer I've grown tired of discussions related to blank lines while reviewing patches. That's why except for a few special cases, I now enforce exactly the bare minimum of one blank line between variable declarations and the rest of the code inside each block.
If doing so makes a function unreadable then perhaps it needs to be split :) I'm sure you'll understand!
I do not really understand the problem as I have not seen any complaints about blank lines unless two or more in a row. I have never seen someone complain about a given blank line in a function, unless a missing one to split up the declared variables and code in a function or block of code.
It is a shame you have decided to take the minimum approach to blank lines, IMO it does not make a lot of sense. I only bring it up to help others with reading your code like our customers.
We do not have rule for this so I can not force anyone to add blank lines for readability, so I have to live with it. :-(
Quoted 6 lines
Regards,
Signature · 2 lines
Adrien Mazarguil 6WIND
Regards,
Keith
Re: [PATCH v1 2/3] net/hyperv: implement core functionality
Hi Keith,
Quoted 18 lines · Keith
Quoted 18 lines · Adrien Mazarguil
Quoted 4 lines
Not to criticize style, but a few blank lines could help in readability for these files IMHO. Unless blank lines are illegal :-)
It's a matter of taste, I think people tend to add random blank lines where they think doing so clarifies things for themselves, resulting in inconsistent coding style not much clearer for everyone after several iterations.
As a maintainer I've grown tired of discussions related to blank lines while reviewing patches. That's why except for a few special cases, I now enforce exactly the bare minimum of one blank line between variable declarations and the rest of the code inside each block.
Quoted 8 lines
Quoted 2 lines
If doing so makes a function unreadable then perhaps it needs to be split :) I'm sure you'll understand!
I do not really understand the problem as I have not seen any complaints about blank lines unless two or more in a row. I have never seen someone complain about a given blank line in a function, unless a missing one to split up the declared variables and code in a function or block of code.
It is true when the amount of blank lines are few and logical, but we generally see patch where in the same file we see random blank lines added without any logic, generally to easily identify where the modification are done.
Quoted 6 lines
It is a shame you have decided to take the minimum approach to blank lines, IMO it does not make a lot of sense. I only bring it up to help others with reading your code like our customers.
We do not have rule for this so I can not force anyone to add blank lines for readability, so I have to live with it. :-(
As there is no clear rules, the best one is limiting this situation to the extreme minimal, otherwise explaining the logic behind it is very difficult as it will differ from one maintainer to another one, it will increase the amount of patches refused due to coding style issues.
Quoted 9 lines
Quoted 5 lines
Regards,
Signature · 2 lines
Adrien Mazarguil 6WINDRegards,
Keith
Regards,
Signature · 2 lines
Nélio Laranjeiro 6WIND
Re: [PATCH v1 2/3] net/hyperv: implement core functionality
In reply to Adrien MazarguilQuoted 33 lines · On Mon, 18 Dec 2017 17:46:23 +0100 Adrien Mazarguil…
drivers/net/hyperv/hyperv.c +33
-0+static int +ether_addr_from_str (struct ether_addr *eth_addr,const char *str) +{ +static const uint8_t conv[0x100 ] = { + ['0' ] =0x80 , ['1' ] =0x81 , ['2' ] =0x82 , ['3' ] =0x83 , + ['4' ] =0x84 , ['5' ] =0x85 , ['6' ] =0x86 , ['7' ] =0x87 , + ['8' ] =0x88 , ['9' ] =0x89 , ['a' ] =0x8a , ['b' ] =0x8b , + ['c' ] =0x8c , ['d' ] =0x8d , ['e' ] =0x8e , ['f' ] =0x8f , + ['A' ] =0x8a , ['B' ] =0x8b , ['C' ] =0x8c , ['D' ] =0x8d , + ['E' ] =0x8e , ['F' ] =0x8f , [':' ] =0x40 , ['-' ] =0x40 , + ['\0' ] =0x60 , + }; +uint64_t addr =0 ; +uint64_t buf =0 ; +unsigned int i =0 ; +unsigned int n =0 ; +uint8_t tmp; + +do { + tmp = conv[(int )*(str++)]; +if (!tmp) +return -EINVAL ; +if (tmp &0x40 ) { + i += (i &1 ) + (!i <<1 ); + addr = (addr << (i <<2 )) | buf; + n += i; + buf =0 ; + i =0 ; + }else { + buf = (buf <<4 ) | (tmp &0xf ); + ++i; + } + }while (!(tmp &0x20 ));
drivers/net/hyperv/hyperv.c line 164
Quoted 10 lines
drivers/net/hyperv/hyperv.c +10
-0+if (n >12 ) +return -EINVAL ; + i =RTE_DIM (eth_addr->addr_bytes); +while (i) { + eth_addr->addr_bytes[--i] = addr &0xff ; + addr >>=8 ; + } +return 0 ; +} +
Why not ether_ntoa?
Re: [PATCH v1 2/3] net/hyperv: implement core functionality
Quoted 41 lines · Stephen Hemminger
Quoted 38 lines · On Mon, 18 Dec 2017 17:46:23 +0100 Adrien Mazarguil…
drivers/net/hyperv/hyperv.c +38
-0+static int +ether_addr_from_str (struct ether_addr *eth_addr,const char *str) +{ +static const uint8_t conv[0x100 ] = { + ['0' ] =0x80 , ['1' ] =0x81 , ['2' ] =0x82 , ['3' ] =0x83 , + ['4' ] =0x84 , ['5' ] =0x85 , ['6' ] =0x86 , ['7' ] =0x87 , + ['8' ] =0x88 , ['9' ] =0x89 , ['a' ] =0x8a , ['b' ] =0x8b , + ['c' ] =0x8c , ['d' ] =0x8d , ['e' ] =0x8e , ['f' ] =0x8f , + ['A' ] =0x8a , ['B' ] =0x8b , ['C' ] =0x8c , ['D' ] =0x8d , + ['E' ] =0x8e , ['F' ] =0x8f , [':' ] =0x40 , ['-' ] =0x40 , + ['\0' ] =0x60 , + }; +uint64_t addr =0 ; +uint64_t buf =0 ; +unsigned int i =0 ; +unsigned int n =0 ; +uint8_t tmp; + +do { + tmp = conv[(int )*(str++)]; +if (!tmp) +return -EINVAL ; +if (tmp &0x40 ) { + i += (i &1 ) + (!i <<1 ); + addr = (addr << (i <<2 )) | buf; + n += i; + buf =0 ; + i =0 ; + }else { + buf = (buf <<4 ) | (tmp &0xf ); + ++i; + } + }while (!(tmp &0x20 )); +if (n >12 ) +return -EINVAL ; + i =RTE_DIM (eth_addr->addr_bytes); +while (i) { + eth_addr->addr_bytes[--i] = addr &0xff ;
Quoted 8 lines
drivers/net/hyperv/hyperv.c line 164
Quoted 5 lines
drivers/net/hyperv/hyperv.c +5
-0+ addr >>=8 ; + } +return 0 ; +} +Why not
ether_ntoa?
Good question. For the following reasons:
- I forgot about the existence of ether_ntoa() and didn't look it up seeing struct ether_addr is (re-)defined by rte_ether.h. What happens when one includes netinet/ether.h together with that file results in various conflicts that trigger a compilation error. This problem should be addressed first.
- ether_ntoa() returns a static buffer and is not reentrant, ether_ntoa_r() is but as a GNU extension, I'm not sure it exists on other OSes. Even if this driver is currently targeted at Linux, this is likely not the case for other DPDK code relying on rte_ether.h.
- I had ether_addr_from_str()'s code already ready and lying around for a future update in testpmd's flow command parser. No other MAC-48 conversion function I know of is as flexible as this version. The ability to omit ":" and entering partial addresses is a big plus IMO.
I think both can coexist on their own merits. Since rte_ether.h needs to be fixed either way, how about I move this function in a separate commit and address the conflict with netinet/ether.h while there?
Signature · 2 lines
Adrien Mazarguil 6WIND
Re: [PATCH v1 2/3] net/hyperv: implement core functionality
18/12/2017 21:21, Adrien Mazarguil:
Quoted 64 lines · Stephen Hemminger
Quoted 41 lines
Quoted 38 lines · On Mon, 18 Dec 2017 17:46:23 +0100 Adrien Mazarguil…
drivers/net/hyperv/hyperv.c +38
-0+static int +ether_addr_from_str (struct ether_addr *eth_addr,const char *str) +{ +static const uint8_t conv[0x100 ] = { + ['0' ] =0x80 , ['1' ] =0x81 , ['2' ] =0x82 , ['3' ] =0x83 , + ['4' ] =0x84 , ['5' ] =0x85 , ['6' ] =0x86 , ['7' ] =0x87 , + ['8' ] =0x88 , ['9' ] =0x89 , ['a' ] =0x8a , ['b' ] =0x8b , + ['c' ] =0x8c , ['d' ] =0x8d , ['e' ] =0x8e , ['f' ] =0x8f , + ['A' ] =0x8a , ['B' ] =0x8b , ['C' ] =0x8c , ['D' ] =0x8d , + ['E' ] =0x8e , ['F' ] =0x8f , [':' ] =0x40 , ['-' ] =0x40 , + ['\0' ] =0x60 , + }; +uint64_t addr =0 ; +uint64_t buf =0 ; +unsigned int i =0 ; +unsigned int n =0 ; +uint8_t tmp; + +do { + tmp = conv[(int )*(str++)]; +if (!tmp) +return -EINVAL ; +if (tmp &0x40 ) { + i += (i &1 ) + (!i <<1 ); + addr = (addr << (i <<2 )) | buf; + n += i; + buf =0 ; + i =0 ; + }else { + buf = (buf <<4 ) | (tmp &0xf ); + ++i; + } + }while (!(tmp &0x20 )); +if (n >12 ) +return -EINVAL ; + i =RTE_DIM (eth_addr->addr_bytes); +while (i) { + eth_addr->addr_bytes[--i] = addr &0xff ;Quoted 8 lines
drivers/net/hyperv/hyperv.c line 164
Quoted 5 lines
drivers/net/hyperv/hyperv.c +5
-0+ addr >>=8 ; + } +return 0 ; +} +Why not
ether_ntoa?Good question. For the following reasons:
- I forgot about the existence of
ether_ntoa()and didn't look it up seeing structether_addris (re-)defined byrte_ether.h. What happens when one includesnetinet/ether.htogether with that file results in various conflicts that trigger a compilation error. This problem should be addressed first.-
ether_ntoa()returns a static buffer and is not reentrant,ether_ntoa_r()is but as a GNU extension, I'm not sure it exists on other OSes. Even if this driver is currently targeted at Linux, this is likely not the case for other DPDK code relying onrte_ether.h.
Quoted 8 lines
- I had ether_addr_from_str()'s code already ready and lying around for a future update in testpmd's flow command parser. No other MAC-48 conversion function I know of is as flexible as this version. The ability to omit ":" and entering partial addresses is a big plus IMO.
I think both can coexist on their own merits. Since
rte_ether.hneeds to be fixed either way, how about I move this function in a separate commit and address the conflict withnetinet/ether.hwhile there?
Looks to be a good plan.
Re: [PATCH v1 2/3] net/hyperv: implement core functionality
Quoted 19 lines · On Mon, 18 Dec 2017 22:03:55 +0100 Thomas Monjalon <[email protected]> wrote:
Quoted 19 lines
Good question. For the following reasons:
- I forgot about the existence of
ether_ntoa()and didn't look it up seeing structether_addris (re-)defined byrte_ether.h. What happens when one includesnetinet/ether.htogether with that file results in various conflicts that trigger a compilation error. This problem should be addressed first.-
ether_ntoa()returns a static buffer and is not reentrant,ether_ntoa_r()is but as a GNU extension, I'm not sure it exists on other OSes. Even if this driver is currently targeted at Linux, this is likely not the case for other DPDK code relying onrte_ether.h.- I had ether_addr_from_str()'s code already ready and lying around for a future update in testpmd's flow command parser. No other MAC-48 conversion function I know of is as flexible as this version. The ability to omit ":" and entering partial addresses is a big plus IMO.
Quoted 5 lines
Quoted 3 lines
I think both can coexist on their own merits. Since
rte_ether.hneeds to be fixed either way, how about I move this function in a separate commit and address the conflict withnetinet/ether.hwhile there?Looks to be a good plan.
Agree, rte_ether is where it should go. Please put functions for parsing there. The name and logic conflict between netinet/ether.h and rte is both a blessing and a curse. Although the definitions of ether_addr overlap, they are equivalent.
Re: [PATCH v1 2/3] net/hyperv: implement core functionality
In reply to Adrien MazarguilQuoted 7 lines · On Mon, 18 Dec 2017 17:46:23 +0100 Adrien Mazarguil…
/**
+ * Destroy a hyperv context instance.
+ *
+ * @param ctx
+ * Context to destroy.
+ */
drivers/net/hyperv/hyperv.c line 203
Quoted 9 lines
drivers/net/hyperv/hyperv.c +9
-0+static void +hyperv_ctx_destroy (struct hyperv_ctx *ctx) +{ +if (ctx->pipe[0 ] !=-1 ) +close (ctx->pipe[0 ]); +if (ctx->pipe[1 ] !=-1 ) +close (ctx->pipe[1 ]); +/* Poisoning for debugging purposes. */ +memset (ctx,0x22 ,sizeof (*ctx));
Don't leave debug code in submitted drivers
Quoted 73 lines
drivers/net/hyperv/hyperv.c +73
-0+free (ctx); +} + +/** +* Iterate over system network interfaces. +* +* This function runs a given callback function for each netdevice found on +* the system. +* +* @param func +* Callback function pointer. List traversal is aborted when this function +* returns a nonzero value. +* @param ... +* Variable parameter list passed as @p va_list to @p func. +* +* @return +* 0 when the entire list is traversed successfully, a negative error code +* in case or failure, or the nonzero value returned by @p func when list +* traversal is aborted. +*/ +static int +hyperv_foreach_iface (int (*func)(const struct if_nameindex *iface, +const struct ether_addr *eth_addr, +va_list ap), ...) +{ +struct if_nameindex *iface =if_nameindex (); +int s =socket (PF_INET ,SOCK_DGRAM ,IPPROTO_IP ); +unsigned int i; +int ret =0 ; + +if (!iface) { + ret = -ENOBUFS ; +ERROR ("cannot retrieve system network interfaces" ); + goto error; + } +if (s ==-1 ) { + ret = -errno; +ERROR ("cannot open socket: %s" ,rte_strerror (errno)); + goto error; + } +for (i =0 ; iface[i].if_name; ++i) { +struct ifreq req; +struct ether_addr eth_addr; +va_list ap; + +strncpy (req.ifr_name, iface[i].if_name,sizeof (req.ifr_name)); +if (ioctl (s,SIOCGIFHWADDR , &req) ==-1 ) { +WARN ("cannot retrieve information about interface" +" \"%s\": %s" , + req.ifr_name,rte_strerror (errno)); +continue ; + } +memcpy (eth_addr.addr_bytes, req.ifr_hwaddr.sa_data, +RTE_DIM (eth_addr.addr_bytes)); +va_start (ap, func); + ret =func (&iface[i], ð_addr, ap); +va_end (ap); +if (ret) +break ; + } +error: +if (s !=-1 ) +close (s); +if (iface) +if_freenameindex (iface); +return ret; +} + +/** +* Determine if a network interface is NetVSC. +* +* @param[in] iface +* Pointer to netdevice description structure (name and index).
drivers/net/hyperv/hyperv.c line 286
Quoted 10 lines
drivers/net/hyperv/hyperv.c +10
-0+* +* @return +* A nonzero value when interface is detected as NetVSC. In case of error, +* rte_errno is updated and 0 returned. +*/ +static int +hyperv_iface_is_netvsc (const struct if_nameindex *iface) +{ +static const char temp[] ="/sys/class/net/%s/device/class_id" ; +char path[snprintf (NULL ,0 , temp, iface->if_name) +1 ];
Doing this snprintf is gross. Either use PATH_MAX or asprintf
Quoted 27 lines
drivers/net/hyperv/hyperv.c +27
-0+FILE *f; +int ret; +int len =0 ; + +snprintf (path,sizeof (path), temp, iface->if_name); + f =fopen (path,"r" ); +if (!f) { + rte_errno = errno; +return 0 ; + } + ret =fscanf (f,NETVSC_CLASS_ID "%n" , &len); +if (ret ==EOF ) + rte_errno = errno; + ret = len == (int )strlen (NETVSC_CLASS_ID ); +fclose (f); +return ret; +} + +/** +* Retrieve the last component of a path. +* +* This is a simplified basename() that does not modify its input buffer to +* handle trailing backslashes. +* +* @param[in] path +* Path to retrieve the last component from. +*
drivers/net/hyperv/hyperv.c line 323
Quoted 10 lines
drivers/net/hyperv/hyperv.c +10
-0+* @return +* Pointer to the last component. +*/ +static const char * +hyperv_basename (const char *path) +{ +const char *tmp = path; + +while (*tmp) +if (*(tmp++) =='/' )
Too may ()
Quoted 138 lines
drivers/net/hyperv/hyperv.c +138
-0+ path = tmp; +return path; +} + +/** +* Retrieve network interface data from sysfs symbolic link. +* +* @param[out] buf +* Output data buffer. +* @param size +* Output buffer size. +* @param[in] if_name +* Netdevice name. +* @param[in] relpath +* Symbolic link path relative to netdevice sysfs entry. +* +* @return +* 0 on success, a negative error code otherwise. +*/ +static int +hyperv_sysfs_readlink (char *buf,size_t size,const char *if_name, +const char *relpath) +{ +int ret; + + ret =snprintf (buf, size,"/sys/class/net/%s/%s" , if_name, relpath); +if (ret ==-1 || (size_t )ret >= size -1 ) +return -ENOBUFS ; + ret =readlink (buf, buf, size); +if (ret ==-1 ) +return -errno; +if ((size_t )ret >= size -1 ) +return -ENOBUFS ; + buf[ret] ='\0' ; +return 0 ; +} + +/** +* Probe a network interface to associate with hyperv context. +* +* This function determines if the network device matches the properties of +* the NetVSC interface associated with the hyperv context and communicates +* its bus address to the fail-safe PMD instance if so. +* +* It is normally used with hyperv_foreach_iface(). +* +* @param[in] iface +* Pointer to netdevice description structure (name and index). +* @param[in] eth_addr +* MAC address associated with @p iface. +* @param ap +* Variable arguments list comprising: +* +* - struct hyperv_ctx *ctx: +* Context to associate network interface with. +* +* @return +* A nonzero value when interface matches, 0 otherwise or in case of +* error. +*/ +static int +hyperv_device_probe (const struct if_nameindex *iface, +const struct ether_addr *eth_addr, +va_list ap) +{ + struct hyperv_ctx *ctx = va_arg(ap, struct hyperv_ctx *); +char buf[RTE_MAX (sizeof (ctx->yield),256u )]; +const char *addr; +size_t len; +int ret; + +/* Skip non-matching or unwanted NetVSC interfaces. */ +if (ctx->if_index == iface->if_index) { +if (!strcmp (ctx->if_name, iface->if_name)) +return 0 ; +DEBUG ("NetVSC interface \"%s\" (index %u) renamed \"%s\"" , + ctx->if_name, ctx->if_index, iface->if_name); +strncpy (ctx->if_name, iface->if_name,sizeof (ctx->if_name)); +return 0 ; + } +if (hyperv_iface_is_netvsc (iface)) +return 0 ; +if (!is_same_ether_addr (eth_addr, &ctx->if_addr)) +return 0 ; +/* Look for associated PCI device. */ + ret =hyperv_sysfs_readlink (buf,sizeof (buf), iface->if_name, +"device/subsystem" ); +if (ret) +return 0 ; +if (strcmp (hyperv_basename (buf),"pci" )) +return 0 ; + ret =hyperv_sysfs_readlink (buf,sizeof (buf), iface->if_name, +"device" ); +if (ret) +return 0 ; + addr =hyperv_basename (buf); + len =strlen (addr); +if (!len) +return 0 ; +/* Send PCI device argument to fail-safe PMD instance if updated. */ +if (!strcmp (addr, ctx->yield)) +return 1 ; +DEBUG ("associating PCI device \"%s\" with NetVSC interface \"%s\"" +" (index %u)" , + addr, ctx->if_name, ctx->if_index); +memmove (buf, addr, len +1 ); + addr = buf; + buf[len] ='\n' ; + ret =write (ctx->pipe[1 ], addr, len +1 ); + buf[len] ='\0' ; +if (ret ==-1 ) { +if (errno ==EINTR || errno ==EAGAIN ) +return 1 ; +WARN ("cannot associate PCI device name \"%s\" with interface" +" \"%s\": %s" , + addr, ctx->if_name,rte_strerror (errno)); +return 1 ; + } +if ((size_t )ret != len +1 ) { +/* +* Attempt to override previous partial write, no need to +* recover if that fails. +*/ + ret =write (ctx->pipe[1 ],"\n" ,1 ); + (void )ret; +return 1 ; + } +fsync (ctx->pipe[1 ]); +memcpy (ctx->yield, addr, len +1 ); +return 1 ; +} + +/** +* Alarm callback that regularly probes system network interfaces. +* +* This callback runs at a frequency determined by HYPERV_PROBE_MS as long +* as an hyperv context instance exists. +*
drivers/net/hyperv/hyperv.c line 471
Quoted 10 lines
drivers/net/hyperv/hyperv.c +10
-0+* @param arg +* Ignored. +*/ +static void +hyperv_alarm (void *arg) +{ +struct hyperv_ctx *ctx; +int ret; + + (void )arg;
I assume you are trying to suppress unused warnings. The DPDK method of doing this __rte_unused
Quoted 152 lines
drivers/net/hyperv/hyperv.c +152
-0+ LIST_FOREACH(ctx, &hyperv_ctx_list, entry) { + ret =hyperv_foreach_iface (hyperv_device_probe, ctx); +if (ret) +break ; + } +if (!hyperv_ctx_count) +return ; + ret =rte_eal_alarm_set (HYPERV_PROBE_MS *1000 , hyperv_alarm,NULL ); +if (ret <0 ) { +ERROR ("unable to reschedule alarm callback: %s" , +rte_strerror (-ret)); + } +} + +/** +* Probe a NetVSC interface to generate a hyperv context from. +* +* This function instantiates hyperv contexts either for all NetVSC devices +* found on the system or only a subset provided as device arguments. +* +* It is normally used with hyperv_foreach_iface(). +* +* @param[in] iface +* Pointer to netdevice description structure (name and index). +* @param[in] eth_addr +* MAC address associated with @p iface. +* @param ap +* Variable arguments list comprising: +* +* - const char *name: +* Name associated with current driver instance. +* +* - struct rte_kvargs *kvargs: +* Device arguments provided to current driver instance. +* +* - unsigned int specified: +* Number of specific netdevices provided as device arguments. +* +* - unsigned int *matched: +* The number of specified netdevices matched by this function. +* +* @return +* A nonzero value when interface matches, 0 otherwise or in case of +* error. +*/ +static int +hyperv_netvsc_probe (const struct if_nameindex *iface, +const struct ether_addr *eth_addr, +va_list ap) +{ + const char *name = va_arg(ap, const char *); + struct rte_kvargs *kvargs = va_arg(ap, struct rte_kvargs *); + unsigned int specified = va_arg(ap, unsigned int); + unsigned int *matched = va_arg(ap, unsigned int *); +unsigned int i; +struct hyperv_ctx *ctx; +uint16_t port_id; +int ret; + +/* Probe all interfaces when none are specified. */ +if (specified) { +for (i =0 ; i != kvargs->count; ++i) { +const struct rte_kvargs_pair *pair = &kvargs->pairs[i]; + +if (!strcmp (pair->key,HYPERV_ARG_IFACE )) { +if (!strcmp (pair->value, iface->if_name)) +break ; + }else if (!strcmp (pair->key,HYPERV_ARG_MAC )) { +struct ether_addr tmp; + +if (ether_addr_from_str (&tmp, pair->value)) { +ERROR ("invalid MAC address format" +" \"%s\"" , + pair->value); +return -EINVAL ; + } +if (!is_same_ether_addr (eth_addr, &tmp)) +break ; + } + } +if (i == kvargs->count) +return 0 ; + ++(*matched); + } +/* Weed out interfaces already handled. */ + LIST_FOREACH(ctx, &hyperv_ctx_list, entry) +if (ctx->if_index == iface->if_index) +break ; +if (ctx) { +if (!specified) +return 0 ; +WARN ("interface \"%s\" (index %u) is already handled, skipping" , + iface->if_name, iface->if_index); +return 0 ; + } +if (!hyperv_iface_is_netvsc (iface)) { +if (!specified) +return 0 ; +WARN ("interface \"%s\" (index %u) is not NetVSC, skipping" , + iface->if_name, iface->if_index); +return 0 ; + } +/* Create interface context. */ + ctx =calloc (1 ,sizeof (*ctx)); +if (!ctx) { + ret = -errno; +ERROR ("cannot allocate context for interface \"%s\": %s" , + iface->if_name,rte_strerror (errno)); + goto error; + } + ctx->id = hyperv_ctx_count; +strncpy (ctx->if_name, iface->if_name,sizeof (ctx->if_name)); + ctx->if_index = iface->if_index; + ctx->if_addr = *eth_addr; + ctx->pipe[0 ] =-1 ; + ctx->pipe[1 ] =-1 ; + ctx->yield[0 ] ='\0' ; +if (pipe (ctx->pipe) ==-1 ) { + ret = -errno; +ERROR ("cannot allocate control pipe for interface \"%s\": %s" , + ctx->if_name,rte_strerror (errno)); + goto error; + } +for (i =0 ; i !=RTE_DIM (ctx->pipe); ++i) { +int flf =fcntl (ctx->pipe[i],F_GETFL ); +int fdf =fcntl (ctx->pipe[i],F_GETFD ); + +if (flf !=-1 && +fcntl (ctx->pipe[i],F_SETFL , flf |O_NONBLOCK ) !=-1 && + fdf !=-1 && +fcntl (ctx->pipe[i],F_SETFD , + i ? fdf |FD_CLOEXEC : fdf & ~FD_CLOEXEC ) !=-1 ) +continue ; + ret = -errno; +ERROR ("cannot toggle non-blocking or close-on-exec flags on" +" control file descriptor #%u (%d): %s" , + i, ctx->pipe[i],rte_strerror (errno)); + goto error; + } +/* Generate virtual device name and arguments. */ + i =0 ; + ret =snprintf (ctx->name,sizeof (ctx->name),"%s_id%u" , + name, ctx->id); +if (ret ==-1 || (size_t )ret >=sizeof (ctx->name) -1 ) + ++i; + ret =snprintf (ctx->devname,sizeof (ctx->devname),"net_failsafe_%s" , + ctx->name); +if (ret ==-1 || (size_t )ret >=sizeof (ctx->devname) -1 ) + ++i; +/* +* Note: bash replaces the default sh interpreter used by popen() +* because as seen with dash, POSIX-compliant shells do not
drivers/net/hyperv/hyperv.c line 633
Quoted 10 lines
drivers/net/hyperv/hyperv.c +10
-0+* necessarily support redirections with file descriptor numbers +* above 9. +*/ + ret =snprintf (ctx->devargs,sizeof (ctx->devargs), +"exec(exec bash -c " +"'while read -r tmp <&%u 2> /dev/null;" +" do dev=$tmp; done;" +" echo $dev" +"'),dev(net_tap_%s,remote=%s)" , + ctx->pipe[0 ], ctx->name, ctx->if_name);
Write real code. Shelling out to bash is messy, error prone and potential security issue.
Quoted 130 lines
drivers/net/hyperv/hyperv.c +94
-1+if (ret ==-1 || (size_t )ret >=sizeof (ctx->devargs) -1 ) + ++i; +if (i) { + ret = -ENOBUFS ; +ERROR ("generated virtual device name or argument list too long" +" for interface \"%s\"" , ctx->if_name); + goto error; + } +/* +* Remove any competing rte_eth_dev entries sharing the same MAC +* address, fail-safe instances created by this PMD will handle them +* as sub-devices later. +*/ +RTE_ETH_FOREACH_DEV (port_id) { +struct rte_device *dev = rte_eth_devices[port_id].device; +struct rte_bus *bus =rte_bus_find_by_device (dev); +struct ether_addr tmp; + +rte_eth_macaddr_get (port_id, &tmp); +if (!is_same_ether_addr (eth_addr, &tmp)) +continue ; +WARN ("removing device \"%s\" with identical MAC address to" +" re-create it as a fail-safe sub-device" , + dev->name); +if (!bus) + ret = -EINVAL ; +else + ret =rte_eal_hotplug_remove (bus->name, dev->name); +if (ret <0 ) { +ERROR ("unable to remove device \"%s\": %s" , + dev->name,rte_strerror (-ret)); + goto error; + } + } +/* Request virtual device generation. */ +DEBUG ("generating virtual device \"%s\" with arguments \"%s\"" , + ctx->devname, ctx->devargs); + ret =rte_eal_hotplug_add ("vdev" , ctx->devname, ctx->devargs); +if (ret) + goto error; +LIST_INSERT_HEAD (&hyperv_ctx_list, ctx, entry); + ++hyperv_ctx_count; +DEBUG ("added NetVSC interface \"%s\" to context list" , ctx->if_name); +return 0 ; +error: +if (ctx) +hyperv_ctx_destroy (ctx); +return ret; +} + +/** * Probe NetVSC interfaces. * +* This function probes system netdevices according to the specified device +* arguments and starts a periodic alarm callback to notify the resulting +* fail-safe PMD instances of their sub-devices whereabouts. +* * @param dev * Virtual device context for PMD instance. * @@ -92,12 +706,38 @@ hyperv_vdev_probe(struct rte_vdev_device *dev)const char *args =rte_vdev_device_args (dev);struct rte_kvargs *kvargs =rte_kvargs_parse (args ? args :"" , hyperv_arg); +unsigned int specified =0 ; +unsigned int matched =0 ; +unsigned int i; +int ret;DEBUG ("invoked as \"%s\", using arguments \"%s\"" , name, args);if (!kvargs) {ERROR ("cannot parse arguments list" ); goto error; } +for (i =0 ; i != kvargs->count; ++i) { +const struct rte_kvargs_pair *pair = &kvargs->pairs[i]; + +if (!strcmp (pair->key,HYPERV_ARG_IFACE ) || + !strcmp (pair->key,HYPERV_ARG_MAC )) + ++specified; + } +rte_eal_alarm_cancel (hyperv_alarm,NULL ); +/* Gather interfaces. */ + ret =hyperv_foreach_iface (hyperv_netvsc_probe, name, kvargs, + specified, &matched); +if (ret <0 ) + goto error; +if (matched < specified) +WARN ("some of the specified parameters did not match valid" +" network interfaces" ); + ret =rte_eal_alarm_set (HYPERV_PROBE_MS *1000 , hyperv_alarm,NULL ); +if (ret <0 ) { +ERROR ("unable to schedule alarm callback: %s" , +rte_strerror (-ret)); + goto error; + } error:if (kvargs)rte_kvargs_free (kvargs);@@ -108,6 +748,9 @@ hyperv_vdev_probe(struct rte_vdev_device *dev)/** * Remove PMD instance. * +* The alarm callback and underlying hyperv context instances are only +* destroyed after the last PMD instance is removed. +* * @param dev * Virtual device context for PMD instance. * @@ -118,7 +761,16 @@ static int hyperv_vdev_remove(struct rte_vdev_device *dev) { (void )dev; - --hyperv_ctx_inst; +if (--hyperv_ctx_inst) +return 0 ; +rte_eal_alarm_cancel (hyperv_alarm,NULL ); +while (!LIST_EMPTY (&hyperv_ctx_list)) { +struct hyperv_ctx *ctx =LIST_FIRST (&hyperv_ctx_list); + +LIST_REMOVE (ctx, entry); + --hyperv_ctx_count; +hyperv_ctx_destroy (ctx); + }return 0 ; }
Re: [PATCH v1 2/3] net/hyperv: implement core functionality
Quoted 13 lines · Stephen Hemminger
Quoted 10 lines · On Mon, 18 Dec 2017 17:46:23 +0100 Adrien Mazarguil…
/**
+ * Destroy a hyperv context instance.
+ *
+ * @param ctx
+ * Context to destroy.
+ */drivers/net/hyperv/hyperv.c +3
-0+static void +hyperv_ctx_destroy (struct hyperv_ctx *ctx) +{
Quoted 8 lines
drivers/net/hyperv/hyperv.c line 203
Quoted 6 lines
drivers/net/hyperv/hyperv.c +6
-0+if (ctx->pipe[0 ] !=-1 ) +close (ctx->pipe[0 ]); +if (ctx->pipe[1 ] !=-1 ) +close (ctx->pipe[1 ]); +/* Poisoning for debugging purposes. */ +memset (ctx,0x22 ,sizeof (*ctx));Don't leave debug code in submitted drivers
Granted this line should be behind #ifdef RTE_LIBRTE_HYPERV_DEBUG.
Surely you don't mean *no* debugging code at all? This memset() allows an application to crash early in case its control path parallelizes things it shouldn't.
Quoted 78 lines
Quoted 78 lines
drivers/net/hyperv/hyperv.c +77
-0+free (ctx); +} + +/** +* Iterate over system network interfaces. +* +* This function runs a given callback function for each netdevice found on +* the system. +* +* @param func +* Callback function pointer. List traversal is aborted when this function +* returns a nonzero value. +* @param ... +* Variable parameter list passed as @p va_list to @p func. +* +* @return +* 0 when the entire list is traversed successfully, a negative error code +* in case or failure, or the nonzero value returned by @p func when list +* traversal is aborted. +*/ +static int +hyperv_foreach_iface (int (*func)(const struct if_nameindex *iface, +const struct ether_addr *eth_addr, +va_list ap), ...) +{ +struct if_nameindex *iface =if_nameindex (); +int s =socket (PF_INET ,SOCK_DGRAM ,IPPROTO_IP ); +unsigned int i; +int ret =0 ; + +if (!iface) { + ret = -ENOBUFS ; +ERROR ("cannot retrieve system network interfaces" ); + goto error; + } +if (s ==-1 ) { + ret = -errno; +ERROR ("cannot open socket: %s" ,rte_strerror (errno)); + goto error; + } +for (i =0 ; iface[i].if_name; ++i) { +struct ifreq req; +struct ether_addr eth_addr; +va_list ap; + +strncpy (req.ifr_name, iface[i].if_name,sizeof (req.ifr_name)); +if (ioctl (s,SIOCGIFHWADDR , &req) ==-1 ) { +WARN ("cannot retrieve information about interface" +" \"%s\": %s" , + req.ifr_name,rte_strerror (errno)); +continue ; + } +memcpy (eth_addr.addr_bytes, req.ifr_hwaddr.sa_data, +RTE_DIM (eth_addr.addr_bytes)); +va_start (ap, func); + ret =func (&iface[i], ð_addr, ap); +va_end (ap); +if (ret) +break ; + } +error: +if (s !=-1 ) +close (s); +if (iface) +if_freenameindex (iface); +return ret; +} + +/** +* Determine if a network interface is NetVSC. +* +* @param[in] iface +* Pointer to netdevice description structure (name and index). +* +* @return +* A nonzero value when interface is detected as NetVSC. In case of error, +* rte_errno is updated and 0 returned.
Quoted 8 lines
drivers/net/hyperv/hyperv.c line 286
Quoted 6 lines
+ */
drivers/net/hyperv/hyperv.c +5
-0+static int +hyperv_iface_is_netvsc (const struct if_nameindex *iface) +{ +static const char temp[] ="/sys/class/net/%s/device/class_id" ; +char path[snprintf (NULL ,0 , temp, iface->if_name) +1 ];Doing this snprintf is gross. Either use
PATH_MAXor asprintf
I don't think allocating more stack space than necessary or on the heap with a possible allocation failure to deal with is any better, sorry.
Prove this snprintf() call can fail and you'll have a point.
Quoted 31 lines
Quoted 31 lines
drivers/net/hyperv/hyperv.c +31
-0+FILE *f; +int ret; +int len =0 ; + +snprintf (path,sizeof (path), temp, iface->if_name); + f =fopen (path,"r" ); +if (!f) { + rte_errno = errno; +return 0 ; + } + ret =fscanf (f,NETVSC_CLASS_ID "%n" , &len); +if (ret ==EOF ) + rte_errno = errno; + ret = len == (int )strlen (NETVSC_CLASS_ID ); +fclose (f); +return ret; +} + +/** +* Retrieve the last component of a path. +* +* This is a simplified basename() that does not modify its input buffer to +* handle trailing backslashes. +* +* @param[in] path +* Path to retrieve the last component from. +* +* @return +* Pointer to the last component. +*/ +static const char *
Quoted 8 lines
drivers/net/hyperv/hyperv.c line 323
Quoted 6 lines
drivers/net/hyperv/hyperv.c +6
-0+hyperv_basename(const char *path) +{ +const char *tmp = path; + +while (*tmp) +if (*(tmp++) =='/' )Too may ()
Will remove it, I'm considering using strrchr() in the caller and remove this function entirely following Keith's comment.
Quoted 144 lines
Quoted 144 lines
drivers/net/hyperv/hyperv.c +143
-0+ path = tmp; +return path; +} + +/** +* Retrieve network interface data from sysfs symbolic link. +* +* @param[out] buf +* Output data buffer. +* @param size +* Output buffer size. +* @param[in] if_name +* Netdevice name. +* @param[in] relpath +* Symbolic link path relative to netdevice sysfs entry. +* +* @return +* 0 on success, a negative error code otherwise. +*/ +static int +hyperv_sysfs_readlink (char *buf,size_t size,const char *if_name, +const char *relpath) +{ +int ret; + + ret =snprintf (buf, size,"/sys/class/net/%s/%s" , if_name, relpath); +if (ret ==-1 || (size_t )ret >= size -1 ) +return -ENOBUFS ; + ret =readlink (buf, buf, size); +if (ret ==-1 ) +return -errno; +if ((size_t )ret >= size -1 ) +return -ENOBUFS ; + buf[ret] ='\0' ; +return 0 ; +} + +/** +* Probe a network interface to associate with hyperv context. +* +* This function determines if the network device matches the properties of +* the NetVSC interface associated with the hyperv context and communicates +* its bus address to the fail-safe PMD instance if so. +* +* It is normally used with hyperv_foreach_iface(). +* +* @param[in] iface +* Pointer to netdevice description structure (name and index). +* @param[in] eth_addr +* MAC address associated with @p iface. +* @param ap +* Variable arguments list comprising: +* +* - struct hyperv_ctx *ctx: +* Context to associate network interface with. +* +* @return +* A nonzero value when interface matches, 0 otherwise or in case of +* error. +*/ +static int +hyperv_device_probe (const struct if_nameindex *iface, +const struct ether_addr *eth_addr, +va_list ap) +{ + struct hyperv_ctx *ctx = va_arg(ap, struct hyperv_ctx *); +char buf[RTE_MAX (sizeof (ctx->yield),256u )]; +const char *addr; +size_t len; +int ret; + +/* Skip non-matching or unwanted NetVSC interfaces. */ +if (ctx->if_index == iface->if_index) { +if (!strcmp (ctx->if_name, iface->if_name)) +return 0 ; +DEBUG ("NetVSC interface \"%s\" (index %u) renamed \"%s\"" , + ctx->if_name, ctx->if_index, iface->if_name); +strncpy (ctx->if_name, iface->if_name,sizeof (ctx->if_name)); +return 0 ; + } +if (hyperv_iface_is_netvsc (iface)) +return 0 ; +if (!is_same_ether_addr (eth_addr, &ctx->if_addr)) +return 0 ; +/* Look for associated PCI device. */ + ret =hyperv_sysfs_readlink (buf,sizeof (buf), iface->if_name, +"device/subsystem" ); +if (ret) +return 0 ; +if (strcmp (hyperv_basename (buf),"pci" )) +return 0 ; + ret =hyperv_sysfs_readlink (buf,sizeof (buf), iface->if_name, +"device" ); +if (ret) +return 0 ; + addr =hyperv_basename (buf); + len =strlen (addr); +if (!len) +return 0 ; +/* Send PCI device argument to fail-safe PMD instance if updated. */ +if (!strcmp (addr, ctx->yield)) +return 1 ; +DEBUG ("associating PCI device \"%s\" with NetVSC interface \"%s\"" +" (index %u)" , + addr, ctx->if_name, ctx->if_index); +memmove (buf, addr, len +1 ); + addr = buf; + buf[len] ='\n' ; + ret =write (ctx->pipe[1 ], addr, len +1 ); + buf[len] ='\0' ; +if (ret ==-1 ) { +if (errno ==EINTR || errno ==EAGAIN ) +return 1 ; +WARN ("cannot associate PCI device name \"%s\" with interface" +" \"%s\": %s" , + addr, ctx->if_name,rte_strerror (errno)); +return 1 ; + } +if ((size_t )ret != len +1 ) { +/* +* Attempt to override previous partial write, no need to +* recover if that fails. +*/ + ret =write (ctx->pipe[1 ],"\n" ,1 ); + (void )ret; +return 1 ; + } +fsync (ctx->pipe[1 ]); +memcpy (ctx->yield, addr, len +1 ); +return 1 ; +} + +/** +* Alarm callback that regularly probes system network interfaces. +* +* This callback runs at a frequency determined by HYPERV_PROBE_MS as long +* as an hyperv context instance exists. +* +* @param arg +* Ignored. +*/ +static void +hyperv_alarm (void *arg)
Quoted 8 lines
drivers/net/hyperv/hyperv.c line 471
Quoted 5 lines
drivers/net/hyperv/hyperv.c +5
-0+{ +struct hyperv_ctx *ctx; +int ret; + + (void )arg;I assume you are trying to suppress unused warnings. The DPDK method of doing this
__rte_unused
This syntax is the standard method for suppressing such warnings, __rte_unused relies on a GNU syntax extension for that, and I usually tend to favor standard forms when they exist.
Given DPDK coding rules don't say anything about this, I don't mind to update it if you really insist.
Quoted 159 lines
Quoted 159 lines
drivers/net/hyperv/hyperv.c +158
-0+ LIST_FOREACH(ctx, &hyperv_ctx_list, entry) { + ret =hyperv_foreach_iface (hyperv_device_probe, ctx); +if (ret) +break ; + } +if (!hyperv_ctx_count) +return ; + ret =rte_eal_alarm_set (HYPERV_PROBE_MS *1000 , hyperv_alarm,NULL ); +if (ret <0 ) { +ERROR ("unable to reschedule alarm callback: %s" , +rte_strerror (-ret)); + } +} + +/** +* Probe a NetVSC interface to generate a hyperv context from. +* +* This function instantiates hyperv contexts either for all NetVSC devices +* found on the system or only a subset provided as device arguments. +* +* It is normally used with hyperv_foreach_iface(). +* +* @param[in] iface +* Pointer to netdevice description structure (name and index). +* @param[in] eth_addr +* MAC address associated with @p iface. +* @param ap +* Variable arguments list comprising: +* +* - const char *name: +* Name associated with current driver instance. +* +* - struct rte_kvargs *kvargs: +* Device arguments provided to current driver instance. +* +* - unsigned int specified: +* Number of specific netdevices provided as device arguments. +* +* - unsigned int *matched: +* The number of specified netdevices matched by this function. +* +* @return +* A nonzero value when interface matches, 0 otherwise or in case of +* error. +*/ +static int +hyperv_netvsc_probe (const struct if_nameindex *iface, +const struct ether_addr *eth_addr, +va_list ap) +{ + const char *name = va_arg(ap, const char *); + struct rte_kvargs *kvargs = va_arg(ap, struct rte_kvargs *); + unsigned int specified = va_arg(ap, unsigned int); + unsigned int *matched = va_arg(ap, unsigned int *); +unsigned int i; +struct hyperv_ctx *ctx; +uint16_t port_id; +int ret; + +/* Probe all interfaces when none are specified. */ +if (specified) { +for (i =0 ; i != kvargs->count; ++i) { +const struct rte_kvargs_pair *pair = &kvargs->pairs[i]; + +if (!strcmp (pair->key,HYPERV_ARG_IFACE )) { +if (!strcmp (pair->value, iface->if_name)) +break ; + }else if (!strcmp (pair->key,HYPERV_ARG_MAC )) { +struct ether_addr tmp; + +if (ether_addr_from_str (&tmp, pair->value)) { +ERROR ("invalid MAC address format" +" \"%s\"" , + pair->value); +return -EINVAL ; + } +if (!is_same_ether_addr (eth_addr, &tmp)) +break ; + } + } +if (i == kvargs->count) +return 0 ; + ++(*matched); + } +/* Weed out interfaces already handled. */ + LIST_FOREACH(ctx, &hyperv_ctx_list, entry) +if (ctx->if_index == iface->if_index) +break ; +if (ctx) { +if (!specified) +return 0 ; +WARN ("interface \"%s\" (index %u) is already handled, skipping" , + iface->if_name, iface->if_index); +return 0 ; + } +if (!hyperv_iface_is_netvsc (iface)) { +if (!specified) +return 0 ; +WARN ("interface \"%s\" (index %u) is not NetVSC, skipping" , + iface->if_name, iface->if_index); +return 0 ; + } +/* Create interface context. */ + ctx =calloc (1 ,sizeof (*ctx)); +if (!ctx) { + ret = -errno; +ERROR ("cannot allocate context for interface \"%s\": %s" , + iface->if_name,rte_strerror (errno)); + goto error; + } + ctx->id = hyperv_ctx_count; +strncpy (ctx->if_name, iface->if_name,sizeof (ctx->if_name)); + ctx->if_index = iface->if_index; + ctx->if_addr = *eth_addr; + ctx->pipe[0 ] =-1 ; + ctx->pipe[1 ] =-1 ; + ctx->yield[0 ] ='\0' ; +if (pipe (ctx->pipe) ==-1 ) { + ret = -errno; +ERROR ("cannot allocate control pipe for interface \"%s\": %s" , + ctx->if_name,rte_strerror (errno)); + goto error; + } +for (i =0 ; i !=RTE_DIM (ctx->pipe); ++i) { +int flf =fcntl (ctx->pipe[i],F_GETFL ); +int fdf =fcntl (ctx->pipe[i],F_GETFD ); + +if (flf !=-1 && +fcntl (ctx->pipe[i],F_SETFL , flf |O_NONBLOCK ) !=-1 && + fdf !=-1 && +fcntl (ctx->pipe[i],F_SETFD , + i ? fdf |FD_CLOEXEC : fdf & ~FD_CLOEXEC ) !=-1 ) +continue ; + ret = -errno; +ERROR ("cannot toggle non-blocking or close-on-exec flags on" +" control file descriptor #%u (%d): %s" , + i, ctx->pipe[i],rte_strerror (errno)); + goto error; + } +/* Generate virtual device name and arguments. */ + i =0 ; + ret =snprintf (ctx->name,sizeof (ctx->name),"%s_id%u" , + name, ctx->id); +if (ret ==-1 || (size_t )ret >=sizeof (ctx->name) -1 ) + ++i; + ret =snprintf (ctx->devname,sizeof (ctx->devname),"net_failsafe_%s" , + ctx->name); +if (ret ==-1 || (size_t )ret >=sizeof (ctx->devname) -1 ) + ++i; +/* +* Note: bash replaces the default sh interpreter used by popen() +* because as seen with dash, POSIX-compliant shells do not +* necessarily support redirections with file descriptor numbers +* above 9. +*/ + ret =snprintf (ctx->devargs,sizeof (ctx->devargs), +"exec(exec bash -c " + "'while read -r tmp <&%u 2> /dev/null;"
Quoted 8 lines
drivers/net/hyperv/hyperv.c line 633
Quoted 4 lines
drivers/net/hyperv/hyperv.c +4
-0+" do dev=$tmp; done;" +" echo $dev" +"'),dev(net_tap_%s,remote=%s)" , + ctx->pipe[0], ctx->name, ctx->if_name);Write real code. Shelling out to bash is messy, error prone and potential security issue.
Right, this code brings the basic idea. I forgot to mention it in the cover letter, I plan a subsequent commit in fail-safe PMD to add file descriptors as a possible control means in addition to its exec() parameter.
Quoted 120 lines
Quoted 120 lines
drivers/net/hyperv/hyperv.c +89
-1+if (ret ==-1 || (size_t )ret >=sizeof (ctx->devargs) -1 ) + ++i; +if (i) { + ret = -ENOBUFS ; +ERROR ("generated virtual device name or argument list too long" +" for interface \"%s\"" , ctx->if_name); + goto error; + } +/* +* Remove any competing rte_eth_dev entries sharing the same MAC +* address, fail-safe instances created by this PMD will handle them +* as sub-devices later. +*/ +RTE_ETH_FOREACH_DEV (port_id) { +struct rte_device *dev = rte_eth_devices[port_id].device; +struct rte_bus *bus =rte_bus_find_by_device (dev); +struct ether_addr tmp; + +rte_eth_macaddr_get (port_id, &tmp); +if (!is_same_ether_addr (eth_addr, &tmp)) +continue ; +WARN ("removing device \"%s\" with identical MAC address to" +" re-create it as a fail-safe sub-device" , + dev->name); +if (!bus) + ret = -EINVAL ; +else + ret =rte_eal_hotplug_remove (bus->name, dev->name); +if (ret <0 ) { +ERROR ("unable to remove device \"%s\": %s" , + dev->name,rte_strerror (-ret)); + goto error; + } + } +/* Request virtual device generation. */ +DEBUG ("generating virtual device \"%s\" with arguments \"%s\"" , + ctx->devname, ctx->devargs); + ret =rte_eal_hotplug_add ("vdev" , ctx->devname, ctx->devargs); +if (ret) + goto error; +LIST_INSERT_HEAD (&hyperv_ctx_list, ctx, entry); + ++hyperv_ctx_count; +DEBUG ("added NetVSC interface \"%s\" to context list" , ctx->if_name); +return 0 ; +error: +if (ctx) +hyperv_ctx_destroy (ctx); +return ret; +} + +/** * Probe NetVSC interfaces. * +* This function probes system netdevices according to the specified device +* arguments and starts a periodic alarm callback to notify the resulting +* fail-safe PMD instances of their sub-devices whereabouts. +* * @param dev * Virtual device context for PMD instance. * @@ -92,12 +706,38 @@ hyperv_vdev_probe(struct rte_vdev_device *dev)const char *args =rte_vdev_device_args (dev);struct rte_kvargs *kvargs =rte_kvargs_parse (args ? args :"" , hyperv_arg); +unsigned int specified =0 ; +unsigned int matched =0 ; +unsigned int i; +int ret;DEBUG ("invoked as \"%s\", using arguments \"%s\"" , name, args);if (!kvargs) {ERROR ("cannot parse arguments list" ); goto error; } +for (i =0 ; i != kvargs->count; ++i) { +const struct rte_kvargs_pair *pair = &kvargs->pairs[i]; + +if (!strcmp (pair->key,HYPERV_ARG_IFACE ) || + !strcmp (pair->key,HYPERV_ARG_MAC )) + ++specified; + } +rte_eal_alarm_cancel (hyperv_alarm,NULL ); +/* Gather interfaces. */ + ret =hyperv_foreach_iface (hyperv_netvsc_probe, name, kvargs, + specified, &matched); +if (ret <0 ) + goto error; +if (matched < specified) +WARN ("some of the specified parameters did not match valid" +" network interfaces" ); + ret =rte_eal_alarm_set (HYPERV_PROBE_MS *1000 , hyperv_alarm,NULL ); +if (ret <0 ) { +ERROR ("unable to schedule alarm callback: %s" , +rte_strerror (-ret)); + goto error; + } error:if (kvargs)rte_kvargs_free (kvargs);@@ -108,6 +748,9 @@ hyperv_vdev_probe(struct rte_vdev_device *dev)/** * Remove PMD instance. * +* The alarm callback and underlying hyperv context instances are only +* destroyed after the last PMD instance is removed. +* * @param dev * Virtual device context for PMD instance. * @@ -118,7 +761,16 @@ static int hyperv_vdev_remove(struct rte_vdev_device *dev) { (void )dev; - --hyperv_ctx_inst; +if (--hyperv_ctx_inst) +return 0 ; +rte_eal_alarm_cancel (hyperv_alarm,NULL ); +while (!LIST_EMPTY (&hyperv_ctx_list)) { +struct hyperv_ctx *ctx =LIST_FIRST (&hyperv_ctx_list);
Quoted 9 lines
drivers/net/hyperv/hyperv.c static int · line 776
Quoted 8 lines
drivers/net/hyperv/hyperv.c +5
-0+ +LIST_REMOVE (ctx, entry); + --hyperv_ctx_count; +hyperv_ctx_destroy (ctx); + }return 0 ; }
In any case, thanks for the quick review!
Signature · 2 lines
Adrien Mazarguil 6WIND
Re: [PATCH v1 2/3] net/hyperv: implement core functionality
Quoted 4 lines · Adrien Mazarguil
Quoted 3 lines · Stephen Hemminger
On Mon, 18 Dec 2017 17:46:23 +0100
Adrien Mazarguil <[email protected]> wrote:
<snip>
Quoted 8 lines
Quoted 7 lines
drivers/net/hyperv/hyperv.c line 286
Quoted 5 lines
drivers/net/hyperv/hyperv.c +5
-0+static int +hyperv_iface_is_netvsc (const struct if_nameindex *iface) +{ +static const char temp[] ="/sys/class/net/%s/device/class_id" ; +char path[snprintf (NULL ,0 , temp, iface->if_name) +1 ];Doing this snprintf is gross. Either use
PATH_MAXor asprintf
Quoted 5 lines
I don't think allocating more stack space than necessary or on the heap with a possible allocation failure to deal with is any better, sorry.
Prove this
snprintf()call can fail and you'll have a point.
While I get your point, I'd tend to go with Stephen's view on this that it's looking a bit "gross". What's the problem with allocating a bit more stack space for it?
/Bruce
Re: [PATCH v1 2/3] net/hyperv: implement core functionality
Quoted 14 lines · Bruce Richardson
Quoted 4 lines · Adrien Mazarguil
Quoted 3 lines · Stephen Hemminger
On Mon, 18 Dec 2017 17:46:23 +0100
Adrien Mazarguil <[email protected]> wrote:<snip>
Quoted 8 lines
Quoted 7 lines
drivers/net/hyperv/hyperv.c line 286
Quoted 5 lines
drivers/net/hyperv/hyperv.c +5
-0+static int +hyperv_iface_is_netvsc (const struct if_nameindex *iface) +{ +static const char temp[] ="/sys/class/net/%s/device/class_id" ; +char path[snprintf (NULL ,0 , temp, iface->if_name) +1 ];Doing this snprintf is gross. Either use
PATH_MAXor asprintf
Quoted 8 lines
Quoted 5 lines
I don't think allocating more stack space than necessary or on the heap with a possible allocation failure to deal with is any better, sorry.
Prove this
snprintf()call can fail and you'll have a point.While I get your point, I'd tend to go with Stephen's view on this that it's looking a bit "gross". What's the problem with allocating a bit more stack space for it?
Well, apart from making a stand, none really. Too "unusual" perhaps, but I don't think "gross" is a valid argument to reject a perfectly valid piece of code that doesn't rely on obscure knowledge nor weird side effects.
I'll update this in v2 to make it look more acceptable in any case.
Signature · 2 lines
Adrien Mazarguil 6WIND
Re: [PATCH v1 2/3] net/hyperv: implement core functionality
Quoted 24 lines · On Tue, 19 Dec 2017 11:15:38 +0100 Adrien Mazarguil…
Quoted 14 lines · Bruce Richardson
Quoted 4 lines · Adrien Mazarguil
Quoted 3 lines · Stephen Hemminger
On Mon, 18 Dec 2017 17:46:23 +0100
Adrien Mazarguil <[email protected]> wrote:<snip>
Quoted 8 lines
Quoted 7 lines
drivers/net/hyperv/hyperv.c line 286
Quoted 5 lines
drivers/net/hyperv/hyperv.c +5
-0+static int +hyperv_iface_is_netvsc (const struct if_nameindex *iface) +{ +static const char temp[] ="/sys/class/net/%s/device/class_id" ; +char path[snprintf (NULL ,0 , temp, iface->if_name) +1 ];Doing this snprintf is gross. Either use
PATH_MAXor asprintfQuoted 8 lines
Quoted 5 lines
I don't think allocating more stack space than necessary or on the heap with a possible allocation failure to deal with is any better, sorry.
Prove this
snprintf()call can fail and you'll have a point.While I get your point, I'd tend to go with Stephen's view on this that it's looking a bit "gross". What's the problem with allocating a bit more stack space for it?
Quoted 6 lines
Well, apart from making a stand, none really. Too "unusual" perhaps, but I don't think "gross" is a valid argument to reject a perfectly valid piece of code that doesn't rely on obscure knowledge nor weird side effects.
I'll update this in v2 to make it look more acceptable in any case.
In this particular case, you can easily show that the maximum length of the string would be less than the format plus maximum length of interface name.
Why not:
char path[sizeof(temp) + IFNAMSIZ];
which keeps the flexibility but also can be evaluated at compile time.
Upleveling. You need to understand that open source software is a collabrative effort. And like doing improvisational theatre, the best answer to any feedback is yes unless there is a technical reason otherwise.
Re: [PATCH v1 2/3] net/hyperv: implement core functionality
In reply to Adrien MazarguilQuoted 10 lines · On Mon, 18 Dec 2017 17:46:23 +0100 Adrien Mazarguil…
drivers/net/hyperv/hyperv.c +10
-0+static int +ether_addr_from_str (struct ether_addr *eth_addr,const char *str) +{ +static const uint8_t conv[0x100 ] = { + ['0' ] =0x80 , ['1' ] =0x81 , ['2' ] =0x82 , ['3' ] =0x83 , + ['4' ] =0x84 , ['5' ] =0x85 , ['6' ] =0x86 , ['7' ] =0x87 , + ['8' ] =0x88 , ['9' ] =0x89 , ['a' ] =0x8a , ['b' ] =0x8b , + ['c' ] =0x8c , ['d' ] =0x8d , ['e' ] =0x8e , ['f' ] =0x8f , + ['A' ] =0x8a , ['B' ] =0x8b , ['C' ] =0x8c , ['D' ] =0x8d , + ['E' ] =0x8e , ['F' ] =0x8f , [':' ] =0x40 , ['-' ] =0x40 ,
drivers/net/hyperv/hyperv.c line 141
Quoted 10 lines
drivers/net/hyperv/hyperv.c +10
-0+ ['\0'] = 0x60, + }; +uint64_t addr =0 ; +uint64_t buf =0 ; +unsigned int i =0 ; +unsigned int n =0 ; +uint8_t tmp; + + do { + tmp = conv[(int)*(str++)];
Cast to int will cause out of bounds reference on non-ascii strings. The parser will get confused by:
001:aa:bb:cc:dd:ee:ff or invalid strings.
Why not use sscanf which would be safer in this case.
/**
* Parse 48bits Ethernet address in pattern xx:xx:xx:xx:xx:xx. *
* @param eth_addr
* A pointer to a ether_addr structure.
* @param str
* A pointer to string contains the formatted MAC address.
* @return
* 0 if the address is valid
* -EINVAL if address is not formatted properly
*/
static inline intether_parse_addr(struct ether_addr *eth_addr, const char *str)
{
int n;
n = sscanf(str,
"%hhx:%hhx:%hhx:%hhx:%hhx:%hhx",
ð_addr->addr_bytes[0],
ð_addr->addr_bytes[1],
ð_addr->addr_bytes[2],
ð_addr->addr_bytes[3],
ð_addr->addr_bytes[4],
ð_addr->addr_bytes[5]);
return (n == ETHER_ADDR_LEN) ? 0 : -EINVAL;
}Quoted 23 lines
drivers/net/hyperv/hyperv.c +23
-0+if (!tmp) +return -EINVAL ; +if (tmp &0x40 ) { + i += (i &1 ) + (!i <<1 ); + addr = (addr << (i <<2 )) | buf; + n += i; + buf =0 ; + i =0 ; + }else { + buf = (buf <<4 ) | (tmp &0xf ); + ++i; + } + }while (!(tmp &0x20 )); +if (n >12 ) +return -EINVAL ; + i =RTE_DIM (eth_addr->addr_bytes); +while (i) { + eth_addr->addr_bytes[--i] = addr &0xff ; + addr >>=8 ; + } +return 0 ; +} +
Re: [PATCH v1 2/3] net/hyperv: implement core functionality
Quoted 19 lines · Stephen Hemminger
Quoted 16 lines · On Mon, 18 Dec 2017 17:46:23 +0100 Adrien Mazarguil…
drivers/net/hyperv/hyperv.c +16
-0+static int +ether_addr_from_str (struct ether_addr *eth_addr,const char *str) +{ +static const uint8_t conv[0x100 ] = { + ['0' ] =0x80 , ['1' ] =0x81 , ['2' ] =0x82 , ['3' ] =0x83 , + ['4' ] =0x84 , ['5' ] =0x85 , ['6' ] =0x86 , ['7' ] =0x87 , + ['8' ] =0x88 , ['9' ] =0x89 , ['a' ] =0x8a , ['b' ] =0x8b , + ['c' ] =0x8c , ['d' ] =0x8d , ['e' ] =0x8e , ['f' ] =0x8f , + ['A' ] =0x8a , ['B' ] =0x8b , ['C' ] =0x8c , ['D' ] =0x8d , + ['E' ] =0x8e , ['F' ] =0x8f , [':' ] =0x40 , ['-' ] =0x40 , + ['\0' ] =0x60 , + }; +uint64_t addr =0 ; +uint64_t buf =0 ; +unsigned int i =0 ; +unsigned int n =0 ;
Quoted 8 lines
drivers/net/hyperv/hyperv.c line 141
Quoted 4 lines
drivers/net/hyperv/hyperv.c +4
-0+uint8_t tmp; + + do { + tmp = conv[(int)*(str++)];Cast to int will cause out of bounds reference on non-ascii strings. The parser will get confused by:
001:aa:bb:cc:dd:ee:ff or invalid strings.
Nice catch! I added the (int) cast to shut up a GCC complaint about using char as index type. The proper fix taking care of integer conversion and array bounds safety check should read:
tmp = conv[*str++ & 0xffu];
Quoted 1 line
Why not use sscanf which would be safer in this case.
Right, this is indeed the obvious implementation, however not only the fixed MAC-48 format is not the most convenient to use for user input (somewhat like forcing them to enter fully expanded IPv6 addresses every time), sscanf() also ignores leading white spaces and successfully parses weird
expressions like " -42: 0x66: 0af: 0: 44:-6", which I think is a
problem.
Quoted 50 lines
/**
* Parse 48bits Ethernet address in pattern xx:xx:xx:xx:xx:xx. *
* @parameth_addr
* A pointer to aether_addrstructure.
* @param str
* A pointer to string contains the formatted MAC address.
* @return
* 0 if the address is valid
*-EINVALif address is not formatted properly
*/
static inline intether_parse_addr(struct ether_addr *eth_addr, const char *str){ int n; n = sscanf(str, "%hhx:%hhx:%hhx:%hhx:%hhx:%hhx", ð_addr->addr_bytes[0], ð_addr->addr_bytes[1], ð_addr->addr_bytes[2], ð_addr->addr_bytes[3], ð_addr->addr_bytes[4], ð_addr->addr_bytes[5]); return (n == ETHER_ADDR_LEN) ? 0 : -EINVAL; }Quoted 23 lines
drivers/net/hyperv/hyperv.c +23
-0+if (!tmp) +return -EINVAL ; +if (tmp &0x40 ) { + i += (i &1 ) + (!i <<1 ); + addr = (addr << (i <<2 )) | buf; + n += i; + buf =0 ; + i =0 ; + }else { + buf = (buf <<4 ) | (tmp &0xf ); + ++i; + } + }while (!(tmp &0x20 )); +if (n >12 ) +return -EINVAL ; + i =RTE_DIM (eth_addr->addr_bytes); +while (i) { + eth_addr->addr_bytes[--i] = addr &0xff ; + addr >>=8 ; + } +return 0 ; +} +
Signature · 2 lines
Adrien Mazarguil 6WIND
Re: [PATCH v1 2/3] net/hyperv: implement core functionality
Quoted 8 lines · On Tue, 19 Dec 2017 11:01:55 +0100 Adrien Mazarguil…
Quoted 1 line
Why not use sscanf which would be safer in this case.
Right, this is indeed the obvious implementation, however not only the fixed MAC-48 format is not the most convenient to use for user input (somewhat like forcing them to enter fully expanded IPv6 addresses every time),
sscanf()also ignores leading white spaces and successfully parses weird
expressions like " -42:0x66: 0af: 0: 44:-6", which I think is a
problem.
There is a standard for ethernet representation, that is all you need to accept. The only simplifications are optional leading zeros 02 vs 2 and upper and lower case a-f.
Don't overthink this. The FreeBSD version of ether_aton_r is:
struct ether_addr *ether_aton_r(const char *a, struct ether_addr *e)
{
int i;
unsigned int o0, o1, o2, o3, o4, o5;
i = sscanf(a, "%x:%x:%x:%x:%x:%x", &o0, &o1, &o2, &o3, &o4, &o5);
if (i != 6)
return (NULL);
e->octet[0]=o0;
e->octet[1]=o1;
e->octet[2]=o2;
e->octet[3]=o3;
e->octet[4]=o4;
e->octet[5]=o5;
return (e);
}Re: [PATCH v1 2/3] net/hyperv: implement core functionality
In reply to Adrien MazarguilQuoted 20 lines · Adrien Mazarguil
As described in more details in the attached documentation (see patch contents), this virtual device driver manages NetVSC interfaces in virtual machines hosted by Hyper-V/Azure platforms.
This driver does not manage traffic nor Ethernet devices directly; it acts as a thin configuration layer that automatically instantiates and controls fail-safe PMD instances combining tap and PCI sub-devices, so that each NetVSC interface is exposed as a single consolidated port to DPDK applications.
PCI sub-devices being hot-pluggable (e.g. during VM migration), applications automatically benefit from increased throughput when present and automatic fallback on NetVSC otherwise without interruption thanks to fail-safe's hot-plug handling.
Once initialized, the sole job of the hyperv driver is to regularly scan for PCI devices to associate with NetVSC interfaces and feed their addresses to corresponding fail-safe instances.
Signed-off-by Adrien Mazarguil <[email protected]>
<...>
Quoted 1 line
drivers/net/hyperv/hyperv.c +1
-0+RTE_ETH_FOREACH_DEV (port_id) {
<..>
Quoted 1 line
drivers/net/hyperv/hyperv.c +1
-0+ ret =rte_eal_hotplug_remove (bus->name, dev->name);
<..>
drivers/net/hyperv/hyperv.c line 671
Quoted 1 line
drivers/net/hyperv/hyperv.c +1
-0+ ret =rte_eal_hotplug_add ("vdev" , ctx->devname, ctx->devargs);
Overall why this logic implemented as network PMD?
Yes technically you can implement *anything* as PMD :), but should we?
This code does eal level work (scans bus, add/remove devices), and for control path, and not a generic solution either (specific to netvsc and failsafe).
Only device argument part of a PMD seems used, rest is unrelated to being a PMD. Scans netvsc changes in background and reflects them into failsafe PMD...
Why this is implemented as PMD, not another entity, like bus driver perhaps? Or indeed why this in DPDK instead of being in application?
<...>
Re: [PATCH v1 2/3] net/hyperv: implement core functionality
Quoted 36 lines · Ferruh Yigit
Quoted 20 lines · Adrien Mazarguil
As described in more details in the attached documentation (see patch contents), this virtual device driver manages NetVSC interfaces in virtual machines hosted by Hyper-V/Azure platforms.
This driver does not manage traffic nor Ethernet devices directly; it acts as a thin configuration layer that automatically instantiates and controls fail-safe PMD instances combining tap and PCI sub-devices, so that each NetVSC interface is exposed as a single consolidated port to DPDK applications.
PCI sub-devices being hot-pluggable (e.g. during VM migration), applications automatically benefit from increased throughput when present and automatic fallback on NetVSC otherwise without interruption thanks to fail-safe's hot-plug handling.
Once initialized, the sole job of the hyperv driver is to regularly scan for PCI devices to associate with NetVSC interfaces and feed their addresses to corresponding fail-safe instances.
Signed-off-by Adrien Mazarguil <[email protected]><...>
Quoted 1 line
drivers/net/hyperv/hyperv.c +1
-0+RTE_ETH_FOREACH_DEV (port_id) {<..>
Quoted 1 line
drivers/net/hyperv/hyperv.c +1
-0+ ret =rte_eal_hotplug_remove (bus->name, dev->name);<..>
drivers/net/hyperv/hyperv.c line 671
Quoted 1 line
drivers/net/hyperv/hyperv.c +1
-0+ ret =rte_eal_hotplug_add ("vdev" , ctx->devname, ctx->devargs);Overall why this logic implemented as network PMD?
Yes technically you can implement *anything* as PMD :), but should we?This code does eal level work (scans bus, add/remove devices), and for control path, and not a generic solution either (specific to netvsc and failsafe).
Quoted 6 lines
Only device argument part of a PMD seems used, rest is unrelated to being a PMD. Scans netvsc changes in background and reflects them into failsafe PMD...
Why this is implemented as PMD, not another entity, like bus driver perhaps?
Or indeed why this in DPDK instead of being in application?
I'll address that last question first: the point of this driver is enabling existing applications to run within a Hyper-V environment unmodified, because they'd otherwise need to manage two driver instances correctly on their own in addition to hot-plug events during VM migration.
Some kind of driver generating a front end to what otherwise appears as two distinct ethdev to applications is therefore necessary.
Currently without it, users have to manually configure failsafe properly for each NetVSC interface on their system. Besides the inconvenience, it's not even a possibility with DPDK applications that don't rely on EAL command-line arguments.
As such it's more correctly defined as a "platform" driver rather than a true PMD. It leaves VF device handling to their respective PMDs while automatically managing the platform-specific part itself. There's no simpler alternative when running in blacklist mode (i.e. not specifying any device parameters on the command line).
Regarding its presence in drivers/net rather than drivers/bus, the end result from an application standpoint is that each instance exposes a single ethdev, even if not its own (failsafe's). Busses don't do that. It also allows passing arguments to individual devices through --vdev if needed.
You're right about putting device detection at the bus level though, and I think there's work in progress to do just that, this driver will be updated to benefit from it once applied. In the meantime, the code as submitted works fine with the current DPDK code base and addresses an existing use case for which there is no solution at this point.
Signature · 2 lines
Adrien Mazarguil 6WIND
Re: [PATCH v1 2/3] net/hyperv: implement core functionality
Quoted 68 lines · Adrien Mazarguil
Quoted 36 lines · Ferruh Yigit
Quoted 20 lines · Adrien Mazarguil
As described in more details in the attached documentation (see patch contents), this virtual device driver manages NetVSC interfaces in virtual machines hosted by Hyper-V/Azure platforms.
This driver does not manage traffic nor Ethernet devices directly; it acts as a thin configuration layer that automatically instantiates and controls fail-safe PMD instances combining tap and PCI sub-devices, so that each NetVSC interface is exposed as a single consolidated port to DPDK applications.
PCI sub-devices being hot-pluggable (e.g. during VM migration), applications automatically benefit from increased throughput when present and automatic fallback on NetVSC otherwise without interruption thanks to fail-safe's hot-plug handling.
Once initialized, the sole job of the hyperv driver is to regularly scan for PCI devices to associate with NetVSC interfaces and feed their addresses to corresponding fail-safe instances.
Signed-off-by Adrien Mazarguil <[email protected]><...>
Quoted 1 line
drivers/net/hyperv/hyperv.c +1
-0+RTE_ETH_FOREACH_DEV (port_id) {<..>
Quoted 1 line
drivers/net/hyperv/hyperv.c +1
-0+ ret =rte_eal_hotplug_remove (bus->name, dev->name);<..>
drivers/net/hyperv/hyperv.c line 671
Quoted 1 line
drivers/net/hyperv/hyperv.c +1
-0+ ret =rte_eal_hotplug_add ("vdev" , ctx->devname, ctx->devargs);Overall why this logic implemented as network PMD?
Yes technically you can implement *anything* as PMD :), but should we?This code does eal level work (scans bus, add/remove devices), and for control path, and not a generic solution either (specific to netvsc and failsafe).
Quoted 6 lines
Only device argument part of a PMD seems used, rest is unrelated to being a PMD. Scans netvsc changes in background and reflects them into failsafe PMD...
Why this is implemented as PMD, not another entity, like bus driver perhaps?
Or indeed why this in DPDK instead of being in application?
I'll address that last question first: the point of this driver is enabling existing applications to run within a Hyper-V environment unmodified, because they'd otherwise need to manage two driver instances correctly on their own in addition to hot-plug events during VM migration.
Some kind of driver generating a front end to what otherwise appears as two distinct ethdev to applications is therefore necessary.
Currently without it, users have to manually configure failsafe properly for each NetVSC interface on their system. Besides the inconvenience, it's not even a possibility with DPDK applications that don't rely on EAL command-line arguments.
As such it's more correctly defined as a "platform" driver rather than a true PMD. It leaves VF device handling to their respective PMDs while automatically managing the platform-specific part itself. There's no simpler alternative when running in blacklist mode (i.e. not specifying any device parameters on the command line).
Regarding its presence in
drivers/netrather thandrivers/bus, the end result from an application standpoint is that each instance exposes a single ethdev, even if not its own (failsafe's). Busses don't do that. It also allows passing arguments to individual devices through --vdev if needed.
Quoted 5 lines
You're right about putting device detection at the bus level though, and I think there's work in progress to do just that, this driver will be updated to benefit from it once applied. In the meantime, the code as submitted works fine with the current DPDK code base and addresses an existing use case for which there is no solution at this point.
This may be working but this looks like a hack to me.
If we need a platform driver why not properly work on it. If we need to improve eal hotplug, this is a good motivation to improve it.
And if this logic needs to be in application let it be, your argument is to not change the existing application but this logic may lead implementing many unrelated things as PMD to not change application, what is the line here.
What is the work in progress, exact list, that will replace this solution? If this hackish solution will prevent that real work, I am against this solution. Is there a way to ensure this will be a temporary solution and that real work will happen?
Re: [PATCH v1 2/3] net/hyperv: implement core functionality
19/12/2017 21:44, Ferruh Yigit:
Quoted 75 lines · Adrien Mazarguil
Quoted 68 lines · Ferruh Yigit
Quoted 36 lines · Adrien Mazarguil
Quoted 20 lines
As described in more details in the attached documentation (see patch contents), this virtual device driver manages NetVSC interfaces in virtual machines hosted by Hyper-V/Azure platforms.
This driver does not manage traffic nor Ethernet devices directly; it acts as a thin configuration layer that automatically instantiates and controls fail-safe PMD instances combining tap and PCI sub-devices, so that each NetVSC interface is exposed as a single consolidated port to DPDK applications.
PCI sub-devices being hot-pluggable (e.g. during VM migration), applications automatically benefit from increased throughput when present and automatic fallback on NetVSC otherwise without interruption thanks to fail-safe's hot-plug handling.
Once initialized, the sole job of the hyperv driver is to regularly scan for PCI devices to associate with NetVSC interfaces and feed their addresses to corresponding fail-safe instances.
Signed-off-by Adrien Mazarguil <[email protected]><...>
Quoted 1 line
drivers/net/hyperv/hyperv.c +1
-0+RTE_ETH_FOREACH_DEV (port_id) {<..>
Quoted 1 line
drivers/net/hyperv/hyperv.c +1
-0+ ret =rte_eal_hotplug_remove (bus->name, dev->name);<..>
drivers/net/hyperv/hyperv.c line 671
Quoted 1 line
drivers/net/hyperv/hyperv.c +1
-0+ ret =rte_eal_hotplug_add ("vdev" , ctx->devname, ctx->devargs);Overall why this logic implemented as network PMD?
Yes technically you can implement *anything* as PMD :), but should we?This code does eal level work (scans bus, add/remove devices), and for control path, and not a generic solution either (specific to netvsc and failsafe).
Quoted 6 lines
Only device argument part of a PMD seems used, rest is unrelated to being a PMD. Scans netvsc changes in background and reflects them into failsafe PMD...
Why this is implemented as PMD, not another entity, like bus driver perhaps?
Or indeed why this in DPDK instead of being in application?
I'll address that last question first: the point of this driver is enabling existing applications to run within a Hyper-V environment unmodified, because they'd otherwise need to manage two driver instances correctly on their own in addition to hot-plug events during VM migration.
Some kind of driver generating a front end to what otherwise appears as two distinct ethdev to applications is therefore necessary.
Currently without it, users have to manually configure failsafe properly for each NetVSC interface on their system. Besides the inconvenience, it's not even a possibility with DPDK applications that don't rely on EAL command-line arguments.
As such it's more correctly defined as a "platform" driver rather than a true PMD. It leaves VF device handling to their respective PMDs while automatically managing the platform-specific part itself. There's no simpler alternative when running in blacklist mode (i.e. not specifying any device parameters on the command line).
Regarding its presence in
drivers/netrather thandrivers/bus, the end result from an application standpoint is that each instance exposes a single ethdev, even if not its own (failsafe's). Busses don't do that. It also allows passing arguments to individual devices through --vdev if needed.Quoted 5 lines
You're right about putting device detection at the bus level though, and I think there's work in progress to do just that, this driver will be updated to benefit from it once applied. In the meantime, the code as submitted works fine with the current DPDK code base and addresses an existing use case for which there is no solution at this point.
Quoted 4 lines
This may be working but this looks like a hack to me.
If we need a platform driver why not properly work on it. If we need to improve eal hotplug, this is a good motivation to improve it.
I agree this code looks to be a platform driver.
It is the first one of this kind.
Usually, things are managed either in a device driver, a bus driver, or in EAL.
I also agree that hotplug should be managed in EAL and bus drivers.
Quoted 3 lines
And if this logic needs to be in application let it be, your argument is to not change the existing application but this logic may lead implementing many unrelated things as PMD to not change application, what is the line here.
The line is hardware management.
The application should not have to implement device-specific or platform-specific code.
The same application should be able to work on any platform.
Quoted 4 lines
What is the work in progress, exact list, that will replace this solution? If this hackish solution will prevent that real work, I am against this solution. Is there a way to ensure this will be a temporary solution and that real work will happen?
I think we should explicitly mark this code as temporary, or use the EXPERIMENTAL tag. It should motivate us to implement what is needed to completely remove this code later.
About the work in progress:
- When hotplug will be fully supported in EAL and bus drivers, the scan part of this platform driver should be removed.
- When ethdev probe notifications will be integrated, it may also clean a part of this code.
- We may also think how the future port ownership can improve the behaviour of this driver.
- NetVSC is currently supported by the TAP PMD, but it may be replaced by a new NetVSC PMD (VMBUS driver is already sent).
- We should also continue the work on the configuration file. Such user configuration may help for platform behaviours.
As a conclusion, there are a lot of improvements in progress, and I am really happy to see Hyper-V supported in DPDK.
I think this driver must be only a step towards a first class support, like KVM/Qemu/vhost/virtio.
As there is no API implied here, I am OK to progress step by step.
Re: [PATCH v1 2/3] net/hyperv: implement core functionality
In reply to Ferruh YigitDisclaimer: I agree with Thomas's suggestions in his reply [1] to your message, I'm replying below as well to provide more details of my own and clarify the motivations behind this approach a bit more.
Quoted 75 lines · Ferruh Yigit
Quoted 68 lines · Adrien Mazarguil
Quoted 36 lines · Ferruh Yigit
Quoted 20 lines · Adrien Mazarguil
As described in more details in the attached documentation (see patch contents), this virtual device driver manages NetVSC interfaces in virtual machines hosted by Hyper-V/Azure platforms.
This driver does not manage traffic nor Ethernet devices directly; it acts as a thin configuration layer that automatically instantiates and controls fail-safe PMD instances combining tap and PCI sub-devices, so that each NetVSC interface is exposed as a single consolidated port to DPDK applications.
PCI sub-devices being hot-pluggable (e.g. during VM migration), applications automatically benefit from increased throughput when present and automatic fallback on NetVSC otherwise without interruption thanks to fail-safe's hot-plug handling.
Once initialized, the sole job of the hyperv driver is to regularly scan for PCI devices to associate with NetVSC interfaces and feed their addresses to corresponding fail-safe instances.
Signed-off-by Adrien Mazarguil <[email protected]><...>
Quoted 1 line
drivers/net/hyperv/hyperv.c +1
-0+RTE_ETH_FOREACH_DEV (port_id) {<..>
Quoted 1 line
drivers/net/hyperv/hyperv.c +1
-0+ ret =rte_eal_hotplug_remove (bus->name, dev->name);<..>
drivers/net/hyperv/hyperv.c line 671
Quoted 1 line
drivers/net/hyperv/hyperv.c +1
-0+ ret =rte_eal_hotplug_add ("vdev" , ctx->devname, ctx->devargs);Overall why this logic implemented as network PMD?
Yes technically you can implement *anything* as PMD :), but should we?This code does eal level work (scans bus, add/remove devices), and for control path, and not a generic solution either (specific to netvsc and failsafe).
Quoted 6 lines
Only device argument part of a PMD seems used, rest is unrelated to being a PMD. Scans netvsc changes in background and reflects them into failsafe PMD...
Why this is implemented as PMD, not another entity, like bus driver perhaps?
Or indeed why this in DPDK instead of being in application?
I'll address that last question first: the point of this driver is enabling existing applications to run within a Hyper-V environment unmodified, because they'd otherwise need to manage two driver instances correctly on their own in addition to hot-plug events during VM migration.
Some kind of driver generating a front end to what otherwise appears as two distinct ethdev to applications is therefore necessary.
Currently without it, users have to manually configure failsafe properly for each NetVSC interface on their system. Besides the inconvenience, it's not even a possibility with DPDK applications that don't rely on EAL command-line arguments.
As such it's more correctly defined as a "platform" driver rather than a true PMD. It leaves VF device handling to their respective PMDs while automatically managing the platform-specific part itself. There's no simpler alternative when running in blacklist mode (i.e. not specifying any device parameters on the command line).
Regarding its presence in
drivers/netrather thandrivers/bus, the end result from an application standpoint is that each instance exposes a single ethdev, even if not its own (failsafe's). Busses don't do that. It also allows passing arguments to individual devices through --vdev if needed.Quoted 5 lines
You're right about putting device detection at the bus level though, and I think there's work in progress to do just that, this driver will be updated to benefit from it once applied. In the meantime, the code as submitted works fine with the current DPDK code base and addresses an existing use case for which there is no solution at this point.
Quoted 4 lines
This may be working but this looks like a hack to me.
If we need a platform driver why not properly work on it. If we need to improve eal hotplug, this is a good motivation to improve it.
Hotplug surely can be improved but I don't think that alone will be enough for what this driver does. Here's how things are sequenced as currently implemented:
1. DPDK application starts.
2. EAL scans for PCI devices, ethdev ports are created for relevant ones.
3. hyperv vdev scans the system for appropriate NetVSC netdevices, instantiates failsafe PMD accordingly to create ethdev ports for each of them.
At this stage, rte_eal_hotplug_remove() is also called on physical devices found in 2. that will be given to failsafe (see 4.), since they're not supposed to be seen or owned by the application (keep in mind this happens on Hyper-V platforms only).
4. From this point on, application can use the remaining ports normally.
5. A PCI device gets plugged in, kernel recognizes it and creates a netdevice for it.
6. hyperv's timer callback detects the new netdevice, if its properties match NetVSC's then it proceeds to tell failsafe its location.
7. failsafe probes the given address on the appropriate bus to instantiate another hidden ethdev out of it and primarily uses that device for TX until it gets unplugged. Meanwhile, RX is still performed on both underlying devices.
Let's now assume hot-plug is perfectly implemented in DPDK along with Gaetan's netdevice bus [2] (or equivalent) with hotplug properties as well:
1. DPDK application starts.
2. EAL scans for PCI devices, ethdev ports are created for relevant ones.
3. EAL scans for net_bus devices, ethdev ports are created for relevant ones.
4. The piece of code formerly known as the hyperv driver looks at detected net_bus devices, finds relevant ones with NetVSC properties and promptly kicks them out through rte_eal_hotplug_remove() (or equivalent) so that the application doesn't get a chance to "see" them.
It then instantiates fail-safe PMD like before, with fail-safe re-discovering devices as its own.
5. From this point on, application can use the remaining ports normally.
6. A PCI device gets plugged in, kernel recognizes it and creates a netdevice for it.
7. EAL's net_bus hotplug handler kicks in, automatically creates a new ethdev port out of it (note: device properties such as MAC addresses are not known before the associated PMD is initialized and an ethdev created).
8. The piece of code formerly known as the hyperv driver that happens to also be listening for hotplug events sees that new ethdev port; if its properties match NetVSC's then it proceeds to hide it before telling failsafe its location.
9. failsafe probes the given address on the appropriate bus to instantiate another hidden ethdev out of it and primarily uses that device for TX until it gets unplugged. Meanwhile, RX is still performed on both underlying devices.
Hotplug basically removes the timer callback and some of the probing code. I agree it's perfectly fine to update this PMD once hotplug is implemented that way. Now what about the rest?
Without a driver there's no way to orchestrate all the above. A separate layer between applications and PMDs is necessary for that; the handover of ethdev ports to failsafe is mandatory.
Quoted 3 lines
And if this logic needs to be in application let it be, your argument is to not change the existing application but this logic may lead implementing many unrelated things as PMD to not change application, what is the line here.
Well, for this particular case I don't think many applications want to retrieve multicast and some other traffic out of one ethdev and the rest from another only when the latter is present. This complexity must be handled by the framework, not by applications, which ideally are not supposed to know much about the environment they're running in.
For this reason, even a specific API is out of the question.
Quoted 4 lines
What is the work in progress, exact list, that will replace this solution? If this hackish solution will prevent that real work, I am against this solution. Is there a way to ensure this will be a temporary solution and that real work will happen?
I think Thomas answers this question [1], I'll just add that the current approach was developed and submitted in a way that doesn't have any impact on public APIs precisely to avoid conflicts with other work on EAL in the meantime.
If the hotplug subsystem evolves, this driver will catch up, particularly since it's small and shouldn't be too complex to adapt. I volunteer for that work once APIs are ready in any case; failing that, the experimental tag (I'll add it for v2) means its pure and simple removal.
I'd like your opinion on the current approach to determine the next steps:
- Do you agree with the fact hotplug and platform-related functionality are two separate problems, that the approach to implement the former doesn't address the latter?
- About implementing the latter in DPDK as a kind of platform driver so that applications don't need to be modified?
- If you had to choose between drivers/bus and drivers/net for it? (keep in mind the ability to provide per-device options would be great)
[1] http://dpdk.org/ml/archives/dev/2017-December/084558.html
[2] http://dpdk.org/ml/archives/dev/2017-June/067546.html
Signature · 2 lines
Adrien Mazarguil 6WIND
PATCH 3/3 net/hyperv: add "force" parameter
In reply to Adrien MazarguilThis parameter allows specifying any non-NetVSC interface to use with tap sub-devices for development purposes.
--- doc/guides/nics/hyperv.rst | 5 +++++ drivers/net/hyperv/hyperv.c | 26 +++++++++++++++++++------- 2 files changed, 24 insertions(+), 7 deletions(-)
doc/guides/nics/hyperv.rst +5 -0
@@ -110,5 +110,10 @@ The following device parameters are supported: Same as ``iface`` except a suitable NetVSC interface is located using its MAC address. +- ``force`` [int] + + If nonzero, forces the use of specified interfaces even if not detected as + NetVSC. + Not specifying either ``iface`` or ``mac`` makes this PMD attach itself to all NetVSC interfaces found on the system.
drivers/net/hyperv/hyperv.c +19 -7
@@ -62,6 +62,7 @@#define HYPERV_DRIVER net_hyperv#define HYPERV_ARG_IFACE "iface"#define HYPERV_ARG_MAC "mac" +#define HYPERV_ARG_FORCE "force"#define HYPERV_PROBE_MS 1000#define NETVSC_CLASS_ID "{f8615163-df3e-46c5-913f-f2d2f965ed0e}"
@@ -504,6 +505,9 @@ hyperv_alarm(void *arg)* - struct rte_kvargs *kvargs: * Device arguments provided to current driver instance. * +* - int force: +* Accept specified interface even if not detected as NetVSC. +* * - unsigned int specified: * Number of specific netdevices provided as device arguments. *
@@ -521,6 +525,7 @@ hyperv_netvsc_probe(const struct if_nameindex *iface, { const char *name = va_arg(ap, const char *); struct rte_kvargs *kvargs = va_arg(ap, struct rte_kvargs *); +int force =va_arg (ap, int); unsigned int specified = va_arg(ap, unsigned int); unsigned int *matched = va_arg(ap, unsigned int *);unsigned int i;
@@ -567,9 +572,11 @@ hyperv_netvsc_probe(const struct if_nameindex *iface,if (!hyperv_iface_is_netvsc (iface)) {if (!specified)return 0 ; -WARN ("interface \"%s\" (index %u) is not NetVSC, skipping" , - iface->if_name, iface->if_index); -return 0 ; +WARN ("interface \"%s\" (index %u) is not NetVSC, %s" , + iface->if_name, iface->if_index, + force ?"using anyway (forced)" :"skipping" ); +if (!force) +return 0 ; }/* Create interface context. */ ctx =calloc (1 ,sizeof (*ctx));
@@ -700,6 +707,7 @@ hyperv_vdev_probe(struct rte_vdev_device *dev)static const char *const hyperv_arg[] = {HYPERV_ARG_IFACE ,HYPERV_ARG_MAC , +HYPERV_ARG_FORCE ,NULL , };const char *name =rte_vdev_device_name (dev);
@@ -708,6 +716,7 @@ hyperv_vdev_probe(struct rte_vdev_device *dev) hyperv_arg);unsigned int specified =0 ;unsigned int matched =0 ; +int force =0 ;unsigned int i;int ret;
@@ -719,13 +728,15 @@ hyperv_vdev_probe(struct rte_vdev_device *dev)for (i =0 ; i != kvargs->count; ++i) {const struct rte_kvargs_pair *pair = &kvargs->pairs[i]; -if (!strcmp (pair->key,HYPERV_ARG_IFACE ) || - !strcmp (pair->key,HYPERV_ARG_MAC )) +if (!strcmp (pair->key,HYPERV_ARG_FORCE )) + force = !!atoi (pair->value); +else if (!strcmp (pair->key,HYPERV_ARG_IFACE ) || + !strcmp (pair->key,HYPERV_ARG_MAC )) ++specified; }rte_eal_alarm_cancel (hyperv_alarm,NULL );/* Gather interfaces. */ - ret =hyperv_foreach_iface (hyperv_netvsc_probe, name, kvargs, + ret =hyperv_foreach_iface (hyperv_netvsc_probe, name, kvargs, force, specified, &matched);if (ret <0 ) goto error;
@@ -784,4 +795,5 @@ RTE_PMD_REGISTER_VDEV(HYPERV_DRIVER, hyperv_vdev);RTE_PMD_REGISTER_ALIAS (HYPERV_DRIVER , eth_hyperv);RTE_PMD_REGISTER_PARAM_STRING (net_hyperv,HYPERV_ARG_IFACE "=<string> " -HYPERV_ARG_MAC "=<string>" ); +HYPERV_ARG_MAC "=<string> " +HYPERV_ARG_FORCE "=<int>" );
Signature · 1 line
2.11.0
Quoted 82 lines · On Mon, 18 Dec 2017 17:46:19 +0100 Adrien Mazarguil…
Virtual machines hosted by Hyper-V/Azure platforms are fitted with simplified virtual network devices named NetVSC that are used for fast communication between VM to VM, VM to hypervisor, and the outside.
They appear as standard system netdevices to user-land applications, the main difference being they are implemented on top of VMBUS [1] instead of emulated PCI devices.
While this reads like a case for a standard DPDK PMD, there is more to it.
To accelerate outside communication, NetVSC devices as they appear in a VM can be paired with physical SR-IOV virtual function (VF) devices owned by that same VM [2]. Both netdevices share the same MAC address in that case.
When paired, egress and most of the ingress traffic flow through the VF device, while part of it (e.g. multicasts, hypervisor control data) still flows through NetVSC. Moreover VF devices are not retained and disappear during VM migration; from a VM standpoint, they can be hot-plugged anytime with NetVSC acting as a fallback.
Running DPDK applications in such a context involves driving VF devices using their dedicated PMDs in a vendor-independent fashion (to benefit from maximum performance without writing dedicated code) while simultaneously listening to NetVSC and handling the related hot-plug events.
This new virtual PMD (referred to as "hyperv" from this point on) automatically coordinates the Hyper-V/Azure-specific management part described above by relying on vendor-specific, failsafe and tap PMDs to expose a single consolidated Ethernet device usable directly by existing applications.
.------------------. | DPDK application | `--------+---------' | .------+------. | DPDK ethdev | `------+------' Control | | .------------+------------. v .------------. | failsafe PMD +---------+ hyperv PMD | `--+-------------------+--' `------------' | | | .........|......... | : | : .----+----. : .----+----. : | tap PMD | : | any PMD | : `----+----' : `----+----' : <-- Hot-pluggable | : | : .------+-------. : .-----+-----. : | NetVSC-based | : | SR-IOV VF | : | netdevice | : | device | : `--------------' : `-----------' : :.................:Note this diagram differs from that of the original RFC [3], with hyperv no longer acting as a data plane layer.
This initial version of the driver only works in whitelist mode. Users have to provide the --vdev
net_hypervEAL option at least once to trigger it.Subsequent work will add support for blacklist mode based on automatic detection of the host environment.
[1] http://dpdk.org/ml/archives/dev/2017-January/054165.html
[2] https://docs.microsoft.com/en-us/windows-hardware/drivers/network/overview-of-hyper-v
[3] http://dpdk.org/ml/archives/dev/2017-November/082339.htmlAdrien Mazarguil (3):
net/hyperv: introduce MS Hyper-V platform driver net/hyperv: implement core functionality net/hyperv: add "force" parameterMAINTAINERS | 6 + config/common_base | 6 + config/common_linuxapp | 1 + doc/guides/nics/features/hyperv.ini | 12 + doc/guides/nics/hyperv.rst | 119 +++ doc/guides/nics/index.rst | 1 + drivers/net/Makefile | 1 + drivers/net/hyperv/Makefile | 58 ++ drivers/net/hyperv/hyperv.c | 799 +++++++++++++++++++++
Quoted 9 lines
drivers/net/hyperv/rte_pmd_hyperv_version.map | 4 + mk/rte.app.mk | 1 + 11 files changed, 1008 insertions(+) create mode 100644 doc/guides/nics/features/hyperv.ini create mode 100644 doc/guides/nics/hyperv.rst create mode 100644 drivers/net/hyperv/Makefile create mode 100644 drivers/net/hyperv/hyperv.c create mode 100644 drivers/net/hyperv/rte_pmd_hyperv_version.map
Please don't call this drivers/net/hyperv/
that name conflicts with the real netvsc PMD that I am working on.
Maybe vdev-netvsc?
18/12/2017 19:23, Stephen Hemminger:
Quoted 4 lines
Please don't call this
drivers/net/hyperv/
that name conflicts with the real netvsc PMD that I am working on.Maybe vdev-netvsc?
I expect your PMD to be in drivers/net/netvsc/
Why is it conflicting with drivers/net/hyperv/ ?
Quoted 12 lines · Thomas Monjalon
18/12/2017 19:23, Stephen Hemminger:
Quoted 4 lines
Please don't call this
drivers/net/hyperv/
that name conflicts with the real netvsc PMD that I am working on.Maybe vdev-netvsc?
I expect your PMD to be in
drivers/net/netvsc/
Why is it conflicting withdrivers/net/hyperv/?
The naming is a bit confusing, and I am willing to change it since not upstream.
The code uses mostly BSD driver which doesn't call itself netvsc. Instead the BSD driver uses hn_ as a prefix for most visible data and functions.
Have been trying to name netvsc to avoid confusion with the kernel driver.
Like any name it is completely irrelevant to functionality.
Quoted 95 lines · Stephen Hemminger
Quoted 82 lines · On Mon, 18 Dec 2017 17:46:19 +0100 Adrien Mazarguil…
Virtual machines hosted by Hyper-V/Azure platforms are fitted with simplified virtual network devices named NetVSC that are used for fast communication between VM to VM, VM to hypervisor, and the outside.
They appear as standard system netdevices to user-land applications, the main difference being they are implemented on top of VMBUS [1] instead of emulated PCI devices.
While this reads like a case for a standard DPDK PMD, there is more to it.
To accelerate outside communication, NetVSC devices as they appear in a VM can be paired with physical SR-IOV virtual function (VF) devices owned by that same VM [2]. Both netdevices share the same MAC address in that case.
When paired, egress and most of the ingress traffic flow through the VF device, while part of it (e.g. multicasts, hypervisor control data) still flows through NetVSC. Moreover VF devices are not retained and disappear during VM migration; from a VM standpoint, they can be hot-plugged anytime with NetVSC acting as a fallback.
Running DPDK applications in such a context involves driving VF devices using their dedicated PMDs in a vendor-independent fashion (to benefit from maximum performance without writing dedicated code) while simultaneously listening to NetVSC and handling the related hot-plug events.
This new virtual PMD (referred to as "hyperv" from this point on) automatically coordinates the Hyper-V/Azure-specific management part described above by relying on vendor-specific, failsafe and tap PMDs to expose a single consolidated Ethernet device usable directly by existing applications.
.------------------. | DPDK application | `--------+---------' | .------+------. | DPDK ethdev | `------+------' Control | | .------------+------------. v .------------. | failsafe PMD +---------+ hyperv PMD | `--+-------------------+--' `------------' | | | .........|......... | : | : .----+----. : .----+----. : | tap PMD | : | any PMD | : `----+----' : `----+----' : <-- Hot-pluggable | : | : .------+-------. : .-----+-----. : | NetVSC-based | : | SR-IOV VF | : | netdevice | : | device | : `--------------' : `-----------' : :.................:Note this diagram differs from that of the original RFC [3], with hyperv no longer acting as a data plane layer.
This initial version of the driver only works in whitelist mode. Users have to provide the --vdev
net_hypervEAL option at least once to trigger it.Subsequent work will add support for blacklist mode based on automatic detection of the host environment.
[1] http://dpdk.org/ml/archives/dev/2017-January/054165.html
[2] https://docs.microsoft.com/en-us/windows-hardware/drivers/network/overview-of-hyper-v
[3] http://dpdk.org/ml/archives/dev/2017-November/082339.htmlAdrien Mazarguil (3):
net/hyperv: introduce MS Hyper-V platform driver net/hyperv: implement core functionality net/hyperv: add "force" parameterMAINTAINERS | 6 + config/common_base | 6 + config/common_linuxapp | 1 + doc/guides/nics/features/hyperv.ini | 12 + doc/guides/nics/hyperv.rst | 119 +++ doc/guides/nics/index.rst | 1 + drivers/net/Makefile | 1 + drivers/net/hyperv/Makefile | 58 ++ drivers/net/hyperv/hyperv.c | 799 +++++++++++++++++++++Quoted 9 lines
drivers/net/hyperv/rte_pmd_hyperv_version.map | 4 + mk/rte.app.mk | 1 + 11 files changed, 1008 insertions(+) create mode 100644 doc/guides/nics/features/hyperv.ini create mode 100644 doc/guides/nics/hyperv.rst create mode 100644 drivers/net/hyperv/Makefile create mode 100644 drivers/net/hyperv/hyperv.c create mode 100644 drivers/net/hyperv/rte_pmd_hyperv_version.map
Quoted 4 lines
Please don't call this
drivers/net/hyperv/
that name conflicts with the real netvsc PMD that I am working on.Maybe vdev-netvsc?
No problem with that, if vdev-netvsc is good for you, I can update it in v2 if needed.
I'm just curious, I was under the impression both drivers would remain kind of complementary pending various API updates, in which case wouldn't it make sense to use "netvsc" as the better name for the NetVSC PMD? ("hyperv" being more a use case than a true PMD)
Otherwise I also don't mind overwriting the current "hyperv" PMD code base with yours as soon as it's ready, this will most likely make it redundant anyway.
Signature · 2 lines
Adrien Mazarguil 6WIND
v2 cover Introduce virtual PMD for Hyper-V/Azure platforms
In reply to Adrien MazarguilVirtual machines hosted by Hyper-V/Azure platforms are fitted with simplified virtual network devices named NetVSC that are used for fast communication between VM to VM, VM to hypervisor, and the outside.
They appear as standard system netdevices to user-land applications, the main difference being they are implemented on top of VMBUS [1] instead of emulated PCI devices.
While this reads like a case for a standard DPDK PMD, there is more to it.
To accelerate outside communication, NetVSC devices as they appear in a VM can be paired with physical SR-IOV virtual function (VF) devices owned by that same VM [2]. Both netdevices share the same MAC address in that case.
When paired, egress and most of the ingress traffic flow through the VF device, while part of it (e.g. multicasts, hypervisor control data) still flows through NetVSC. Moreover VF devices are not retained and disappear during VM migration; from a VM standpoint, they can be hot-plugged anytime with NetVSC acting as a fallback.
Running DPDK applications in such a context involves driving VF devices using their dedicated PMDs in a vendor-independent fashion (to benefit from maximum performance without writing dedicated code) while simultaneously listening to NetVSC and handling the related hot-plug events.
This new virtual PMD (referred to as "vdev_netvsc" from this point on) automatically coordinates the Hyper-V/Azure-specific management part described above by relying on vendor-specific, failsafe and tap PMDs to expose a single consolidated Ethernet device usable directly by existing applications.
.------------------.
| DPDK application |
`--------+---------'
|
.------+------.
| DPDK ethdev |
`------+------' Control
| |
.------------+------------. v .-----------------.
| failsafe PMD +---------+ vdev_netvsc PMD |
`--+-------------------+--' `-----------------'
| |
| .........|.........
| : | :
.----+----. : .----+----. :
| tap PMD | : | any PMD | :
`----+----' : `----+----' : <-- Hot-pluggable
| : | :
.------+-------. : .-----+-----. :
| NetVSC-based | : | SR-IOV VF | :
| netdevice | : | device | :
`--------------' : `-----------' :
:.................:Note this diagram differs from that of the original RFC [3], with vdev_netvsc no longer acting as a data plane layer.
This initial version of the driver only works in whitelist mode. Users have to provide the --vdev net_vdev_netvsc EAL option at least once to trigger it.
Subsequent work will add support for blacklist mode based on automatic detection of the host environment.
[1] http://dpdk.org/ml/archives/dev/2017-January/054165.html
[2] https://docs.microsoft.com/en-us/windows-hardware/drivers/network/overview-of-hyper-v
[3] http://dpdk.org/ml/archives/dev/2017-November/082339.html
v2 changes:
- Renamed driver from "hyperv" to "vdev_netvsc". This change covers documentation and symbols prefix.
- Driver is now tagged EXPERIMENTAL.
- Replaced ether_addr_from_str() with a basic sscanf() call.
- Removed debugging code (memset() poisoning).
- Fixed hyperv_iface_is_netvsc()'s buffer allocation according to comments.
- Removed hyperv_basename().
- Discarded unused variables through __rte_unused.
- Added separate but necessary free() bugfix for failsafe PMD.
- Added file descriptor input support to failsafe PMD.
- Replaced temporary bash execution; failsafe now reads device definitions directly through a pipe without an intermediate bash one-liner.
- Expanded DEBUG/INFO/WARN/ERROR() macros as PMD_DRV_LOG().
- Added dynamic log type (pmd.vdev_netvsc).
- Modified initialization code to probe devices immediately during startup.
- Fixed several snprintf() return value checks ("ret >= sizeof(foo)" is more appropriate than "ret >= sizeof(foo) - 1").
Adrien Mazarguil (5):
net/failsafe: fix invalid free net/failsafe: add "fd" parameter net/vdev_netvsc: introduce Hyper-V platform driver net/vdev_netvsc: implement core functionality net/vdev_netvsc: add "force" parameter
MAINTAINERS | 6 + config/common_base | 5 + config/common_linuxapp | 1 + doc/guides/nics/fail_safe.rst | 9 + doc/guides/nics/features/vdev_netvsc.ini | 12 + doc/guides/nics/index.rst | 1 + doc/guides/nics/vdev_netvsc.rst | 116 +++ drivers/net/Makefile | 1 + drivers/net/failsafe/failsafe_args.c | 88 ++- drivers/net/failsafe/failsafe_private.h | 3 + drivers/net/vdev_netvsc/Makefile | 58 ++ .../vdev_netvsc/rte_pmd_vdev_netvsc_version.map | 4 + drivers/net/vdev_netvsc/vdev_netvsc.c | 722 +++++++++++++++++++ mk/rte.app.mk | 1 + 14 files changed, 1025 insertions(+), 2 deletions(-) create mode 100644 doc/guides/nics/features/vdev_netvsc.ini create mode 100644 doc/guides/nics/vdev_netvsc.rst create mode 100644 drivers/net/vdev_netvsc/Makefile create mode 100644 drivers/net/vdev_netvsc/rte_pmd_vdev_netvsc_version.map create mode 100644 drivers/net/vdev_netvsc/vdev_netvsc.c
Signature · 1 line
2.11.0
PATCH v2 1/5 net/failsafe: fix invalid free
rte_free() is not supposed to work with pointers returned by calloc().
--- drivers/net/failsafe/failsafe_args.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/failsafe/failsafe_args.c +1 -1
@@ -407,7 +407,7 @@ failsafe_args_free(struct rte_eth_dev *dev)uint8_t i; FOREACH_SUBDEV(sdev, i, dev) { -rte_free (sdev->cmdline); +free (sdev->cmdline); sdev->cmdline =NULL ;free (sdev->devargs.args); sdev->devargs.args =NULL ;
Signature · 1 line
2.11.0
PATCH v2 2/5 net/failsafe: add "fd" parameter
In reply to Adrien MazarguilThis parameter enables applications to provide device definitions through an arbitrary file descriptor number.
--- doc/guides/nics/fail_safe.rst | 9 +++ drivers/net/failsafe/failsafe_args.c | 86 +++++++++++++++++++++++++++- drivers/net/failsafe/failsafe_private.h | 3 + 3 files changed, 97 insertions(+), 1 deletion(-)
doc/guides/nics/fail_safe.rst +9 -0
@@ -106,6 +106,15 @@ Fail-safe command line parameters All commas within the ``shell command`` are replaced by spaces before executing the command. This helps using scripts to specify devices. +- **fd(<file descriptor number>)** parameter + + This parameter reads a device definition from an arbitrary file descriptor + number in ``<iface>`` format as described above. + + The file descriptor is read in non-blocking mode and is never closed in + order to take only the last line into account (unlike ``exec()``) at every + probe attempt. + - **mac** parameter [MAC address] This parameter allows the user to set a default MAC address to the fail-safe
drivers/net/failsafe/failsafe_args.c +85 -1
123 lines not on this page · raw message
drivers/net/failsafe/failsafe_private.h +3 -0
@@ -48,6 +48,7 @@#define PMD_FAILSAFE_PARAM_STRING \ "dev(<ifc>)," \ "exec(<shell command>)," \ + "fd(<fd number>)," \ "mac=mac_addr," \ "hotplug_poll=u64" \ ""
@@ -111,6 +112,8 @@ struct sub_device {struct fs_stats stats_snapshot;/* Some device are defined as a command line */ char *cmdline; +/* Others are retrieved through a file descriptor */ +char *fd_str;/* fail-safe device backreference */ struct rte_eth_dev *fs_dev;/* flag calling for recollection */
Signature · 1 line
2.11.0
PATCH v2 3/5 net/vdev_netvsc: introduce Hyper-V platform driver
In reply to Adrien MazarguilThis patch lays the groundwork for this driver (draft documentation, copyright notices, code base skeleton and build system hooks). While it can be successfully compiled and invoked, it's an empty shell at this stage.
--- MAINTAINERS | 6 + config/common_base | 5 + config/common_linuxapp | 1 + doc/guides/nics/features/vdev_netvsc.ini | 12 ++ doc/guides/nics/index.rst | 1 + doc/guides/nics/vdev_netvsc.rst | 46 +++++++ drivers/net/Makefile | 1 + drivers/net/vdev_netvsc/Makefile | 54 ++++++++ .../vdev_netvsc/rte_pmd_vdev_netvsc_version.map | 4 + drivers/net/vdev_netvsc/vdev_netvsc.c | 132 +++++++++++++++++++ mk/rte.app.mk | 1 + 11 files changed, 263 insertions(+)
MAINTAINERS +6 -0
@@ -451,6 +451,12 @@ F: drivers/net/mrvl/ F: doc/guides/nics/mrvl.rst F: doc/guides/nics/features/mrvl.ini +Microsoft vdev-netvsc - EXPERIMENTAL +M: Adrien Mazarguil <[email protected]> +F: drivers/net/vdev-netvsc/ +F: doc/guides/nics/vdev-netvsc.rst +F: doc/guides/nics/features/vdev-netvsc.ini + Netcope szedata2 M: Matej Vido <[email protected]> F: drivers/net/szedata2/
config/common_base +5 -0
@@ -280,6 +280,11 @@ CONFIG_RTE_LIBRTE_NFP_DEBUG=n CONFIG_RTE_LIBRTE_MRVL_PMD=n # +# Compile virtual device driver for NetVSC on Hyper-V/Azure +# +CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD=n + +# # Compile burst-oriented Broadcom BNXT PMD driver # CONFIG_RTE_LIBRTE_BNXT_PMD=y
config/common_linuxapp +1 -0
@@ -47,6 +47,7 @@ CONFIG_RTE_LIBRTE_PMD_VHOST=y CONFIG_RTE_LIBRTE_PMD_AF_PACKET=y CONFIG_RTE_LIBRTE_PMD_TAP=y CONFIG_RTE_LIBRTE_AVP_PMD=y +CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD=y CONFIG_RTE_LIBRTE_NFP_PMD=y CONFIG_RTE_LIBRTE_POWER=y CONFIG_RTE_VIRTIO_USER=y
doc/guides/nics/features/vdev_netvsc.ini new file +12 -0
@@ -0,0 +1,12 @@ +; +; Supported features of the 'vdev_netvsc' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +ARMv7 = Y +ARMv8 = Y +Power8 = Y +x86-32 = Y +x86-64 = Y +Usage doc = Y
doc/guides/nics/index.rst +1 -0
@@ -64,6 +64,7 @@ Network Interface Controller Drivers szedata2 tap thunderx + vdev_netvsc virtio vhost vmxnet3
doc/guides/nics/vdev_netvsc.rst new file +46 -0
46 lines not on this page · raw message
drivers/net/Makefile +1 -0
@@ -66,6 +66,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += sfc DIRS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2) += szedata2 DIRS-$(CONFIG_RTE_LIBRTE_PMD_TAP) += tap DIRS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += thunderx +DIRS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += vdev_netvsc DIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio DIRS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += vmxnet3
drivers/net/vdev_netvsc/Makefile new file +54 -0
54 lines not on this page · raw message
drivers/net/vdev_netvsc/rte_pmd_vdev_netvsc_version.map new file +4 -0
@@ -0,0 +1,4 @@ +DPDK_18.02 { + + local: *; +};
drivers/net/vdev_netvsc/vdev_netvsc.c new file +132 -0
132 lines not on this page · raw message
mk/rte.app.mk +1 -0
@@ -156,6 +156,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += -lrte_pmd_sfc_efx _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2) += -lrte_pmd_szedata2 -lsze2 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_TAP) += -lrte_pmd_tap _LDLIBS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += -lrte_pmd_thunderx_nicvf +_LDLIBS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += -lrte_pmd_vdev_netvsc _LDLIBS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += -lrte_pmd_virtio ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y) _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += -lrte_pmd_vhost
Signature · 1 line
2.11.0
PATCH v2 4/5 net/vdev_netvsc: implement core functionality
In reply to Adrien MazarguilAs described in more details in the attached documentation (see patch contents), this virtual device driver manages NetVSC interfaces in virtual machines hosted by Hyper-V/Azure platforms.
This driver does not manage traffic nor Ethernet devices directly; it acts as a thin configuration layer that automatically instantiates and controls fail-safe PMD instances combining tap and PCI sub-devices, so that each NetVSC interface is exposed as a single consolidated port to DPDK applications.
PCI sub-devices being hot-pluggable (e.g. during VM migration), applications automatically benefit from increased throughput when present and automatic fallback on NetVSC otherwise without interruption thanks to fail-safe's hot-plug handling.
Once initialized, the sole job of the vdev_netvsc driver is to regularly scan for PCI devices to associate with NetVSC interfaces and feed their addresses to corresponding fail-safe instances.
--- doc/guides/nics/vdev_netvsc.rst | 65 ++++ drivers/net/vdev_netvsc/Makefile | 4 + drivers/net/vdev_netvsc/vdev_netvsc.c | 581 ++++++++++++++++++++++++++++- 3 files changed, 649 insertions(+), 1 deletion(-)
doc/guides/nics/vdev_netvsc.rst +65 -0
@@ -38,9 +38,74 @@ platforms. .. _Hyper-V: https://docs.microsoft.com/en-us/windows-hardware/drivers/network/overview-of-hyper-v +Implementation details +---------------------- + +Each instance of this driver effectively needs to drive two devices: the +NetVSC interface proper and its SR-IOV VF (referred to as "physical" from +this point on) counterpart sharing the same MAC address. + +Physical devices are part of the host system and cannot be maintained during +VM migration. From a VM standpoint they appear as hot-plug devices that come +and go without prior notice. + +When the physical device is present, egress and most of the ingress traffic +flows through it; only multicasts and other hypervisor control still flow +through NetVSC. Otherwise, NetVSC acts as a fallback for all traffic. + +To avoid unnecessary code duplication and ensure maximum performance, +handling of physical devices is left to their original PMDs; this virtual +device driver (also known as *vdev*) manages other PMDs as summarized by the +following block diagram:: + + .------------------. + | DPDK application | + `--------+---------' + | + .------+------. + | DPDK ethdev | + `------+------' Control + | | + .------------+------------. v .-----------------. + | failsafe PMD +---------+ vdev_netvsc PMD | + `--+-------------------+--' `-----------------' + | | + | .........|......... + | : | : + .----+----. : .----+----. : + | tap PMD | : | any PMD | : + `----+----' : `----+----' : <-- Hot-pluggable + | : | : + .------+-------. : .-----+-----. : + | NetVSC-based | : | SR-IOV VF | : + | netdevice | : | device | : + `--------------' : `-----------' : + :.................: + Build options ------------- - ``CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD`` (default ``y``) Toggle compilation of this driver. + +Run-time parameters +------------------- + +To invoke this PMD, applications have to explicitly provide the +``--vdev=net_vdev_netvsc`` EAL option. + +The following device parameters are supported: + +- ``iface`` [string] + + Provide a specific NetVSC interface (netdevice) name to attach this PMD + to. Can be provided multiple times for additional instances. + +- ``mac`` [string] + + Same as ``iface`` except a suitable NetVSC interface is located using its + MAC address. + +Not specifying either ``iface`` or ``mac`` makes this PMD attach itself to +all NetVSC interfaces found on the system.
drivers/net/vdev_netvsc/Makefile +4 -0
@@ -40,6 +40,9 @@ EXPORT_MAP := rte_pmd_vdev_netvsc_version.map CFLAGS += -O3 CFLAGS += -g CFLAGS += -std=c11 -pedantic -Wall -Wextra +CFLAGS += -D_XOPEN_SOURCE=600 +CFLAGS += -D_BSD_SOURCE +CFLAGS += -D_DEFAULT_SOURCE CFLAGS += $(WERROR_FLAGS) # Dependencies.
@@ -47,6 +50,7 @@ LDLIBS += -lrte_bus_vdev LDLIBS += -lrte_eal LDLIBS += -lrte_ethdev LDLIBS += -lrte_kvargs +LDLIBS += -lrte_net # Source files. SRCS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += vdev_netvsc.c
drivers/net/vdev_netvsc/vdev_netvsc.c +580 -1
634 lines not on this page · raw message
Signature · 1 line
2.11.0
PATCH v2 5/5 net/vdev_netvsc: add "force" parameter
In reply to Adrien MazarguilThis parameter allows specifying any non-NetVSC interface to use with tap sub-devices for development purposes.
--- doc/guides/nics/vdev_netvsc.rst | 5 +++++ drivers/net/vdev_netvsc/vdev_netvsc.c | 27 +++++++++++++++++++-------- 2 files changed, 24 insertions(+), 8 deletions(-)
doc/guides/nics/vdev_netvsc.rst +5 -0
@@ -107,5 +107,10 @@ The following device parameters are supported: Same as ``iface`` except a suitable NetVSC interface is located using its MAC address. +- ``force`` [int] + + If nonzero, forces the use of specified interfaces even if not detected as + NetVSC. + Not specifying either ``iface`` or ``mac`` makes this PMD attach itself to all NetVSC interfaces found on the system.
drivers/net/vdev_netvsc/vdev_netvsc.c +19 -8
@@ -63,6 +63,7 @@#define VDEV_NETVSC_DRIVER net_vdev_netvsc#define VDEV_NETVSC_ARG_IFACE "iface"#define VDEV_NETVSC_ARG_MAC "mac" +#define VDEV_NETVSC_ARG_FORCE "force"#define VDEV_NETVSC_PROBE_MS 1000#define NETVSC_CLASS_ID "{f8615163-df3e-46c5-913f-f2d2f965ed0e}"
@@ -405,6 +406,9 @@ vdev_netvsc_alarm(__rte_unused void *arg)* - struct rte_kvargs *kvargs: * Device arguments provided to current driver instance. * +* - int force: +* Accept specified interface even if not detected as NetVSC. +* * - unsigned int specified: * Number of specific netdevices provided as device arguments. *
@@ -422,6 +426,7 @@ vdev_netvsc_netvsc_probe(const struct if_nameindex *iface, { const char *name = va_arg(ap, const char *); struct rte_kvargs *kvargs = va_arg(ap, struct rte_kvargs *); +int force =va_arg (ap, int); unsigned int specified = va_arg(ap, unsigned int); unsigned int *matched = va_arg(ap, unsigned int *);unsigned int i;
@@ -480,10 +485,11 @@ vdev_netvsc_netvsc_probe(const struct if_nameindex *iface,if (!specified)return 0 ;PMD_DRV_LOG (WARNING , -"interface \"%s\" (index %u) is not NetVSC," -" skipping" , - iface->if_name, iface->if_index); -return 0 ; +"interface \"%s\" (index %u) is not NetVSC, %s" , + iface->if_name, iface->if_index, + force ?"using anyway (forced)" :"skipping" ); +if (!force) +return 0 ; }/* Create interface context. */ ctx =calloc (1 ,sizeof (*ctx));
@@ -610,6 +616,7 @@ vdev_netvsc_vdev_probe(struct rte_vdev_device *dev)static const char *const vdev_netvsc_arg[] = {VDEV_NETVSC_ARG_IFACE ,VDEV_NETVSC_ARG_MAC , +VDEV_NETVSC_ARG_FORCE ,NULL , };const char *name =rte_vdev_device_name (dev);
@@ -618,6 +625,7 @@ vdev_netvsc_vdev_probe(struct rte_vdev_device *dev) vdev_netvsc_arg);unsigned int specified =0 ;unsigned int matched =0 ; +int force =0 ;unsigned int i;int ret;
@@ -631,14 +639,16 @@ vdev_netvsc_vdev_probe(struct rte_vdev_device *dev)for (i =0 ; i != kvargs->count; ++i) {const struct rte_kvargs_pair *pair = &kvargs->pairs[i]; -if (!strcmp (pair->key,VDEV_NETVSC_ARG_IFACE ) || - !strcmp (pair->key,VDEV_NETVSC_ARG_MAC )) +if (!strcmp (pair->key,VDEV_NETVSC_ARG_FORCE )) + force = !!atoi (pair->value); +else if (!strcmp (pair->key,VDEV_NETVSC_ARG_IFACE ) || + !strcmp (pair->key,VDEV_NETVSC_ARG_MAC )) ++specified; }rte_eal_alarm_cancel (vdev_netvsc_alarm,NULL );/* Gather interfaces. */ ret =vdev_netvsc_foreach_iface (vdev_netvsc_netvsc_probe, name, kvargs, - specified, &matched); + force, specified, &matched);if (ret <0 ) goto error;if (matched < specified)
@@ -697,7 +707,8 @@ RTE_PMD_REGISTER_VDEV(VDEV_NETVSC_DRIVER, vdev_netvsc_vdev);RTE_PMD_REGISTER_ALIAS (VDEV_NETVSC_DRIVER , eth_vdev_netvsc);RTE_PMD_REGISTER_PARAM_STRING (net_vdev_netvsc,VDEV_NETVSC_ARG_IFACE "=<string> " -VDEV_NETVSC_ARG_MAC "=<string>" ); +VDEV_NETVSC_ARG_MAC "=<string> " +VDEV_NETVSC_ARG_FORCE "=<int>" );/** Initialize driver log type. */ static void
Signature · 1 line
2.11.0
Why does this need to be a PMD?
Maybe we need some platform infrastructure?
My definition of PMD is it can send and receive
Quoted 123 lines · Adrien Mazarguil
Virtual machines hosted by Hyper-V/Azure platforms are fitted with simplified virtual network devices named NetVSC that are used for fast communication between VM to VM, VM to hypervisor, and the outside.
They appear as standard system netdevices to user-land applications, the main difference being they are implemented on top of VMBUS [1] instead of emulated PCI devices.
While this reads like a case for a standard DPDK PMD, there is more to it.
To accelerate outside communication, NetVSC devices as they appear in a VM can be paired with physical SR-IOV virtual function (VF) devices owned by that same VM [2]. Both netdevices share the same MAC address in that case.
When paired, egress and most of the ingress traffic flow through the VF device, while part of it (e.g. multicasts, hypervisor control data) still flows through NetVSC. Moreover VF devices are not retained and disappear during VM migration; from a VM standpoint, they can be hot-plugged anytime with NetVSC acting as a fallback.
Running DPDK applications in such a context involves driving VF devices using their dedicated PMDs in a vendor-independent fashion (to benefit from maximum performance without writing dedicated code) while simultaneously listening to NetVSC and handling the related hot-plug events.
This new virtual PMD (referred to as "
vdev_netvsc" from this point on) automatically coordinates the Hyper-V/Azure-specific management part described above by relying on vendor-specific, failsafe and tap PMDs to expose a single consolidated Ethernet device usable directly by existing applications..------------------. | DPDK application | `--------+---------' | .------+------. | DPDK ethdev | `------+------' Control | | .------------+------------. v .-----------------. | failsafe PMD +---------+ vdev_netvsc PMD | `--+-------------------+--' `-----------------' | | | .........|......... | : | : .----+----. : .----+----. : | tap PMD | : | any PMD | : `----+----' : `----+----' : <-- Hot-pluggable | : | : .------+-------. : .-----+-----. : | NetVSC-based | : | SR-IOV VF | : | netdevice | : | device | : `--------------' : `-----------' : :.................:Note this diagram differs from that of the original RFC [3], with
vdev_netvscno longer acting as a data plane layer.This initial version of the driver only works in whitelist mode. Users have to provide the --vdev
net_vdev_netvscEAL option at least once to trigger it.Subsequent work will add support for blacklist mode based on automatic detection of the host environment.
[1] http://dpdk.org/ml/archives/dev/2017-January/054165.html
[2] https://docs.microsoft.com/en-us/windows-hardware/drivers/
network/overview-of-hyper-v
[3] http://dpdk.org/ml/archives/dev/2017-November/082339.htmlv2 changes:
- Renamed driver from "hyperv" to "
vdev_netvsc". This change covers documentation and symbols prefix.
- Driver is now tagged EXPERIMENTAL.
- Replacedether_addr_from_str()with a basicsscanf()call.
- Removed debugging code (memset()poisoning).
- Fixed hyperv_iface_is_netvsc()'s buffer allocation according to comments.
- Removedhyperv_basename().
- Discarded unused variables through__rte_unused.
- Added separate but necessaryfree()bugfix for failsafe PMD.
- Added file descriptor input support to failsafe PMD.
- Replaced temporary bash execution; failsafe now reads device definitions directly through a pipe without an intermediate bash one-liner.
- Expanded DEBUG/INFO/WARN/ERROR() macros asPMD_DRV_LOG().
- Added dynamic log type (pmd.vdev_netvsc).
- Modified initialization code to probe devices immediately during startup.
- Fixed severalsnprintf()return value checks ("ret >= sizeof(foo)" is more
appropriate than "ret >= sizeof(foo) - 1").Adrien Mazarguil (5):
net/failsafe: fix invalid free net/failsafe: add "fd" parameternet/vdev_netvsc: introduce Hyper-V platform drivernet/vdev_netvsc: implement core functionalitynet/vdev_netvsc: add "force" parameterMAINTAINERS | 6 + config/common_base | 5 + config/common_linuxapp | 1 + doc/guides/nics/fail_safe.rst | 9 + doc/guides/nics/features/vdev_netvsc.ini | 12 + doc/guides/nics/index.rst | 1 + doc/guides/nics/vdev_netvsc.rst | 116 +++ drivers/net/Makefile | 1 + drivers/net/failsafe/failsafe_args.c | 88 ++- drivers/net/failsafe/failsafe_private.h | 3 + drivers/net/vdev_netvsc/Makefile | 58 ++ .../vdev_netvsc/rte_pmd_vdev_netvsc_version.map | 4 + drivers/net/vdev_netvsc/vdev_netvsc.c | 722 +++++++++++++++++++ mk/rte.app.mk | 1 + 14 files changed, 1025 insertions(+), 2 deletions(-) create mode 100644 doc/guides/nics/features/vdev_netvsc.ini create mode 100644 doc/guides/nics/vdev_netvsc.rst create mode 100644 drivers/net/vdev_netvsc/Makefile create mode 100644 drivers/net/vdev_netvsc/rte_ pmd_vdev_netvsc_version.map create mode 100644 drivers/net/vdev_netvsc/vdev_netvsc.c--
2.11.0
23/12/2017 03:06, Stephen Hemminger:
Quoted 1 line
Why does this need to be a PMD?
It needs to be a driver on top of buses.
Quoted 1 line
Maybe we need some platform infrastructure?
What would be such infrastructure? A new driver type? Something like drivers/platform/?
I am not sure it is required for this driver given it is most probably only a temporary driver waiting for the NetVSC PMD and a full hotplug support in DPDK internals.
I think we should create such new infrastructure only when we are sure it is needed permanently for some drivers.
Quoted 1 line
My definition of PMD is it can send and receive
It is the definition of an ethdev driver, yes.
v3 cover Introduce virtual driver for Hyper-V/Azure platforms
In reply to Adrien MazarguilVirtual machines hosted by Hyper-V/Azure platforms are fitted with simplified virtual network devices named NetVSC that are used for fast communication between VM to VM, VM to hypervisor, and the outside.
They appear as standard system netdevices to user-land applications, the main difference being they are implemented on top of VMBUS instead of emulated PCI devices.
While this reads like a case for a standard DPDK PMD, there is more to it.
To accelerate outside communication, NetVSC devices as they appear in a VM can be paired with physical SR-IOV virtual function (VF) devices owned by that same VM. Both netdevices share the same MAC address in that case.
When paired, egress and most of the ingress traffic flow through the VF device, while part of it (e.g. multicasts, hypervisor control data) still flows through NetVSC. Moreover VF devices are not retained and disappear during VM migration; from a VM standpoint, they can be hot-plugged anytime with NetVSC acting as a fallback.
Running DPDK applications in such a context involves driving VF devices using their dedicated PMDs in a vendor-independent fashion (to benefit from maximum performance without writing dedicated code) while simultaneously listening to NetVSC and handling the related hot-plug events.
This new virtual driver (referred to as "vdev_netvsc" from this point on) automatically coordinates the Hyper-V/Azure-specific management part described above by relying on vendor-specific, failsafe and tap PMDs to expose a single consolidated Ethernet device usable directly by existing applications.
.------------------.
| DPDK application |
`--------+---------'
|
.------+------.
| DPDK ethdev |
`------+------' Control
| |
.------------+------------. v .--------------------.
| failsafe PMD +---------+ vdev_netvsc driver |
`--+-------------------+--' `--------------------'
| |
| .........|.........
| : | :
.----+----. : .----+----. :
| tap PMD | : | any PMD | :
`----+----' : `----+----' : <-- Hot-pluggable
| : | :
.------+-------. : .-----+-----. :
| NetVSC-based | : | SR-IOV VF | :
| netdevice | : | device | :
`--------------' : `-----------' :
:.................:v2 changes(Adrien):
- Renamed driver from "hyperv" to "vdev_netvsc". This change covers documentation and symbols prefix.
- Driver is now tagged EXPERIMENTAL.
- Replaced ether_addr_from_str() with a basic sscanf() call.
- Removed debugging code (memset() poisoning).
- Fixed hyperv_iface_is_netvsc()'s buffer allocation according to comments.
- Removed hyperv_basename().
- Discarded unused variables through __rte_unused.
- Added separate but necessary free() bugfix for failsafe PMD.
- Added file descriptor input support to failsafe PMD.
- Replaced temporary bash execution; failsafe now reads device definitions directly through a pipe without an intermediate bash one-liner.
- Expanded DEBUG/INFO/WARN/ERROR() macros as PMD_DRV_LOG().
- Added dynamic log type (pmd.vdev_netvsc).
- Modified initialization code to probe devices immediately during startup.
- Fixed several snprintf() return value checks ("ret >= sizeof(foo)" is more appropriate than "ret >= sizeof(foo) - 1").
v3 changes(Matan):
- Fixed clang compilation in V2.
- Removed hotplug remove code from the new driver.
- Supported probed sub-devices getting in fail-safe.
- Added automatic probing for HyperV VM systems.
- Added option to ignore the automatic probing.
- Skiped routed NetVSC devices probing.
- Adjusted documentation and semantics.
- Replaced maintainer.
Adrien Mazarguil (2):
net/failsafe: fix invalid free
net/failsafe: add "fd" parameter
Matan Azrad (6):
net/failsafe: support probed sub-devices getting net/vdev_netvsc: introduce Hyper-V platform driver net/vdev_netvsc: implement core functionality net/vdev_netvsc: skip routed netvsc probing net/vdev_netvsc: add "force" parameter net/vdev_netvsc: add automatic probing
MAINTAINERS | 6 + config/common_base | 5 + config/common_linuxapp | 1 + doc/guides/nics/fail_safe.rst | 14 + doc/guides/nics/features/vdev_netvsc.ini | 12 + doc/guides/nics/index.rst | 1 + doc/guides/nics/vdev_netvsc.rst | 100 +++ drivers/net/Makefile | 1 + drivers/net/failsafe/failsafe_args.c | 88 ++- drivers/net/failsafe/failsafe_eal.c | 60 +- drivers/net/failsafe/failsafe_private.h | 3 + drivers/net/vdev_netvsc/Makefile | 31 + .../vdev_netvsc/rte_pmd_vdev_netvsc_version.map | 4 + drivers/net/vdev_netvsc/vdev_netvsc.c | 746 +++++++++++++++++++++ mk/rte.app.mk | 1 + 15 files changed, 1051 insertions(+), 22 deletions(-) create mode 100644 doc/guides/nics/features/vdev_netvsc.ini create mode 100644 doc/guides/nics/vdev_netvsc.rst create mode 100644 drivers/net/vdev_netvsc/Makefile create mode 100644 drivers/net/vdev_netvsc/rte_pmd_vdev_netvsc_version.map create mode 100644 drivers/net/vdev_netvsc/vdev_netvsc.c
Signature · 1 line
1.8.3.1
PATCH v3 1/8 net/failsafe: fix invalid free
From: Adrien Mazarguil <[email protected]>
rte_free() is not supposed to work with pointers returned by calloc().
--- drivers/net/failsafe/failsafe_args.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/failsafe/failsafe_args.c +1 -1
@@ -407,7 +407,7 @@ typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params,uint8_t i; FOREACH_SUBDEV(sdev, i, dev) { -rte_free (sdev->cmdline); +free (sdev->cmdline); sdev->cmdline =NULL ;free (sdev->devargs.args); sdev->devargs.args =NULL ;
Signature · 1 line
1.8.3.1
Re: [PATCH v3 1/8] net/failsafe: fix invalid free
Hi Matan,
Quoted 8 lines · Matan Azrad
From: Adrien Mazarguil <[email protected]>
rte_free()is not supposed to work with pointers returned bycalloc().Fixes a0194d828100 ("net/failsafe: add flexible device definition")Signed-off-by Adrien Mazarguil <[email protected]>
Quoted 20 lines
--- drivers/net/failsafe/failsafe_args.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/net/failsafe/failsafe_args.c +1
-1@@ -407,7 +407,7 @@ typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params,uint8_t i; FOREACH_SUBDEV(sdev, i, dev) { -rte_free (sdev->cmdline); +free (sdev->cmdline); sdev->cmdline =NULL ;free (sdev->devargs.args); sdev->devargs.args =NULL ;Signature · 1 line
1.8.3.1
Signature · 2 lines
Gaëtan Rivet 6WIND
PATCH v3 2/8 net/failsafe: add "fd" parameter
In reply to Matan AzradFrom: Adrien Mazarguil <[email protected]>
This parameter enables applications to provide device definitions through an arbitrary file descriptor number.
--- doc/guides/nics/fail_safe.rst | 9 ++++ drivers/net/failsafe/failsafe_args.c | 86 ++++++++++++++++++++++++++++++++- drivers/net/failsafe/failsafe_private.h | 3 ++ 3 files changed, 97 insertions(+), 1 deletion(-)
doc/guides/nics/fail_safe.rst +9 -0
@@ -106,6 +106,15 @@ Fail-safe command line parameters All commas within the ``shell command`` are replaced by spaces before executing the command. This helps using scripts to specify devices. +- **fd(<file descriptor number>)** parameter + + This parameter reads a device definition from an arbitrary file descriptor + number in ``<iface>`` format as described above. + + The file descriptor is read in non-blocking mode and is never closed in + order to take only the last line into account (unlike ``exec()``) at every + probe attempt. + - **mac** parameter [MAC address] This parameter allows the user to set a default MAC address to the fail-safe
drivers/net/failsafe/failsafe_args.c +85 -1
123 lines not on this page · raw message
drivers/net/failsafe/failsafe_private.h +3 -0
@@ -48,6 +48,7 @@#define PMD_FAILSAFE_PARAM_STRING \ "dev(<ifc>)," \ "exec(<shell command>)," \ + "fd(<fd number>)," \ "mac=mac_addr," \ "hotplug_poll=u64" \ ""
@@ -111,6 +112,8 @@ struct sub_device {struct fs_stats stats_snapshot;/* Some device are defined as a command line */ char *cmdline; +/* Others are retrieved through a file descriptor */ +char *fd_str;/* fail-safe device backreference */ struct rte_eth_dev *fs_dev;/* flag calling for recollection */
Signature · 1 line
1.8.3.1
Re: [PATCH v3 2/8] net/failsafe: add "fd" parameter
Hi Matam, Adrien,
Quoted 4 lines · Matan Azrad
From: Adrien Mazarguil <[email protected]>
This parameter enables applications to provide device definitions through an arbitrary file descriptor number.
Ok on the principle,
<snip>
drivers/net/failsafe/failsafe_args.c typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, · line 175
Quoted 12 lines
drivers/net/failsafe/failsafe_args.c +8
-0@@ -161,6 +165,73 @@ typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, }static int +fs_read_fd (struct sub_device *sdev,char *fd_str) +{ +FILE *fp =NULL ; +int fd =-1 ; +/* store possible newline as well */ +char output[DEVARGS_MAXLEN +1 ]; +int err = -ENODEV ; +int ret;
ret is used as flag older, line counter and then error reporting. err should be the only variable used for reading errors from function and reporting it.
It would be clearer to use descriptive names, such as "oflags" and "nl" or "lcount". I don't really care about one additional variable in this function, for the sake of expressiveness.
Quoted 10 lines
drivers/net/failsafe/failsafe_args.c +10
-0+ +RTE_ASSERT (fd_str !=NULL || sdev->fd_str !=NULL ); +if (sdev->fd_str ==NULL ) { + sdev->fd_str =strdup (fd_str); +if (sdev->fd_str ==NULL ) { +ERROR ("Command line allocation failed" ); +return -ENOMEM ; + } + } + errno =0 ;
drivers/net/failsafe/failsafe_args.c typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, · line 195
Quoted 10 lines
drivers/net/failsafe/failsafe_args.c +10
-0+ fd =strtol (fd_str, &fd_str,0 ); +if (errno || *fd_str || fd <0 ) { +ERROR ("Parsing FD number failed" ); + goto error; + } +/* Fiddle with copy of file descriptor */ + fd =dup (fd); +if (fd ==-1 ) + goto error; + ret =fcntl (fd,F_GETFL );
oflags = fcntl(...);
drivers/net/failsafe/failsafe_args.c typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, · line 198
Quoted 3 lines
drivers/net/failsafe/failsafe_args.c +3
-0+if (ret ==-1 ) + goto error; + ret =fcntl (fd,F_SETFL , fd |O_NONBLOCK );
err = fcntl(fd, F_SETFL, oflags | O_NONBLOCK);
Using (fd | O_NONBLOCK) is probably a mistake.
drivers/net/failsafe/failsafe_args.c typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, · line 208
Quoted 10 lines
drivers/net/failsafe/failsafe_args.c +10
-0+if (ret ==-1 ) + goto error; + fp =fdopen (fd,"r" ); +if (!fp) + goto error; + fd =-1 ; +/* Only take the last line into account */ + ret =0 ; +while (fgets (output,sizeof (output), fp)) + ++ret;
lcount = 0;
while (fgets(output, sizeof(output), fp))
++lcount;drivers/net/failsafe/failsafe_args.c typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, · line 217
Quoted 9 lines
drivers/net/failsafe/failsafe_args.c +9
-0+if (feof (fp)) { +if (!ret) + goto error; + }else if (ferror (fp)) { +if (errno !=EAGAIN || !ret) + goto error; + }else if (!ret) { + goto error; + }
These branches seems needlessly complicated:
if (lcount == 0)
goto error;
else if (ferror(fp) && errno != EAGAIN)
goto error;drivers/net/failsafe/failsafe_args.c typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, · line 225
Quoted 8 lines
drivers/net/failsafe/failsafe_args.c +8
-0+/* Line must end with a newline character */ +fs_sanitize_cmdline (output); +if (output[0 ] =='\0' ) + goto error; + ret =fs_parse_device (sdev, output); +if (ret) +ERROR ("Parsing device '%s' failed" , output); + err = ret;
no need to use ret instead of err here?
err = fs_parse_device(sdev, output);
if (err)
ERROR("Parsing device '%s' failed", output);Thus allowing to remove the "ret" variable completely.
Quoted 13 lines
drivers/net/failsafe/failsafe_args.c +9
-0+error: +if (fp) +fclose (fp); +if (fd !=-1 ) +close (fd); +return err; +} + +static int fs_parse_device_param (struct rte_eth_dev *dev,const char *param,uint8_t head) {
drivers/net/failsafe/failsafe_args.c typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, · line 276
Quoted 4 lines
drivers/net/failsafe/failsafe_args.c +1
-0@@ -202,6 +273,14 @@ typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, }if (ret) goto free_args; + }else if (strncmp (param,"fd" ,2 ) ==0 ) {
How about strncmp(param, "fd(", 3) == 0 here?
I think I made a mistake for dev and exec device types, no reason at this point to reiterate for fd as well.
Quoted 20 lines
drivers/net/failsafe/failsafe_args.c +9
-0+ ret =fs_read_fd (sdev, args); +if (ret == -ENODEV ) { +DEBUG ("Reading device info from FD failed" ); + ret =0 ; + } +if (ret) + goto free_args; }else {ERROR ("Unrecognized device type: %.*s" , (int )b, param);return -EINVAL ;@@ -409,6 +488,8 @@ typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, FOREACH_SUBDEV(sdev, i, dev) {free (sdev->cmdline); sdev->cmdline =NULL ; +free (sdev->fd_str); + sdev->fd_str =NULL ;free (sdev->devargs.args); sdev->devargs.args =NULL ; }
drivers/net/failsafe/failsafe_args.c typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, · line 509
Quoted 6 lines
drivers/net/failsafe/failsafe_args.c +2
-1@@ -424,7 +505,8 @@ typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, param[b] != '\0') b++;if (strncmp (param,"dev" , b) !=0 && -strncmp (param,"exec" , b) !=0 ) { +strncmp (param,"exec" , b) !=0 && +strncmp (param,"fd" , b) !=0 ) {
If the strncmp above is modified, this one should be as well for consistency.
Signature · 2 lines
Gaëtan Rivet 6WIND
Re: [PATCH v3 2/8] net/failsafe: add "fd" parameter
Hi again,
made a mistake in reviewing, see below.
Quoted 54 lines · Gaëtan Rivet
Hi Matam, Adrien,
Quoted 4 lines · Matan Azrad
From: Adrien Mazarguil <[email protected]>
This parameter enables applications to provide device definitions through an arbitrary file descriptor number.
Ok on the principle,
<snip>
drivers/net/failsafe/failsafe_args.c typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, · line 175
Quoted 12 lines
drivers/net/failsafe/failsafe_args.c +8
-0@@ -161,6 +165,73 @@ typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, }static int +fs_read_fd (struct sub_device *sdev,char *fd_str) +{ +FILE *fp =NULL ; +int fd =-1 ; +/* store possible newline as well */ +char output[DEVARGS_MAXLEN +1 ]; +int err = -ENODEV ; +int ret;ret is used as flag older, line counter and then error reporting. err should be the only variable used for reading errors from function and reporting it.
It would be clearer to use descriptive names, such as "oflags" and "nl" or "lcount". I don't really care about one additional variable in this function, for the sake of expressiveness.
Quoted 10 lines
drivers/net/failsafe/failsafe_args.c +10
-0+ +RTE_ASSERT (fd_str !=NULL || sdev->fd_str !=NULL ); +if (sdev->fd_str ==NULL ) { + sdev->fd_str =strdup (fd_str); +if (sdev->fd_str ==NULL ) { +ERROR ("Command line allocation failed" ); +return -ENOMEM ; + } + } + errno =0 ;drivers/net/failsafe/failsafe_args.c typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, · line 195
Quoted 10 lines
drivers/net/failsafe/failsafe_args.c +10
-0+ fd =strtol (fd_str, &fd_str,0 ); +if (errno || *fd_str || fd <0 ) { +ERROR ("Parsing FD number failed" ); + goto error; + } +/* Fiddle with copy of file descriptor */ + fd =dup (fd); +if (fd ==-1 ) + goto error; + ret =fcntl (fd,F_GETFL );
Quoted 9 lines
oflags = fcntl(...);
drivers/net/failsafe/failsafe_args.c typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, · line 198
Quoted 3 lines
drivers/net/failsafe/failsafe_args.c +3
-0+if (ret ==-1 ) + goto error; + ret =fcntl (fd,F_SETFL , fd |O_NONBLOCK );err =
fcntl(fd, F_SETFL, oflags | O_NONBLOCK);
Using (fd |O_NONBLOCK) is probably a mistake.
This is sneaky. err is -ENODEV and would change to -1 on error, losing some meaning.
Quoted 26 lines
drivers/net/failsafe/failsafe_args.c typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, · line 208
Quoted 10 lines
drivers/net/failsafe/failsafe_args.c +10
-0+if (ret ==-1 ) + goto error; + fp =fdopen (fd,"r" ); +if (!fp) + goto error; + fd =-1 ; +/* Only take the last line into account */ + ret =0 ; +while (fgets (output,sizeof (output), fp)) + ++ret;lcount = 0; while (fgets(output, sizeof(output), fp)) ++lcount;drivers/net/failsafe/failsafe_args.c typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, · line 217
Quoted 9 lines
drivers/net/failsafe/failsafe_args.c +9
-0+if (feof (fp)) { +if (!ret) + goto error; + }else if (ferror (fp)) { +if (errno !=EAGAIN || !ret) + goto error; + }else if (!ret) { + goto error; + }
Quoted 7 lines
These branches seems needlessly complicated:
if (lcount == 0) goto error; else if (ferror(fp) && errno != EAGAIN) goto error;
Here err would have been set to 0 previously with the fcntl call, meaning that jumping to error would return 0 as well.
I know Adrien wanted to avoid the usual ugly
if (error) {
err = -ENODEV;
goto error;
}But this kind of sneakiness is not easy to parse and maintain. If someone adds a new path of error later, this kind of subtlety *will* be lost.
So between ugliness and maintainability, I choose maintainability (being the maintainer, of course).
Signature · 2 lines
Gaëtan Rivet 6WIND
Re: [PATCH v3 2/8] net/failsafe: add "fd" parameter
Hi Gaetan
OK for all, will change it.
From: Gaëtan Rivet, Tuesday, January 16, 2018 1:19 PM
Quoted 124 lines
Hi again,
made a mistake in reviewing, see below.
Quoted 53 lines · Gaëtan Rivet
Hi Matam, Adrien,
Quoted 4 lines · Matan Azrad
From: Adrien Mazarguil <[email protected]>
This parameter enables applications to provide device definitions through an arbitrary file descriptor number.
Ok on the principle,
<snip>
drivers/net/failsafe/failsafe_args.c typedef int (parse_cb)(struct rte_eth_dev *dev, · line 174
Quoted 11 lines
drivers/net/failsafe/failsafe_args.c +7
-0@@ -161,6 +165,73 @@ typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, }static int +fs_read_fd (struct sub_device *sdev,char *fd_str) { +FILE *fp =NULL ; +int fd =-1 ; +/* store possible newline as well */ +char output[DEVARGS_MAXLEN +1 ]; +int err = -ENODEV ; +int ret;ret is used as flag older, line counter and then error reporting. err should be the only variable used for reading errors from function and reporting it.
It would be clearer to use descriptive names, such as "oflags" and "nl" or "lcount". I don't really care about one additional variable in this function, for the sake of expressiveness.
Quoted 10 lines
drivers/net/failsafe/failsafe_args.c +10
-0+ +RTE_ASSERT (fd_str !=NULL || sdev->fd_str !=NULL ); +if (sdev->fd_str ==NULL ) { + sdev->fd_str =strdup (fd_str); +if (sdev->fd_str ==NULL ) { +ERROR ("Command line allocation failed" ); +return -ENOMEM ; + } + } + errno =0 ;drivers/net/failsafe/failsafe_args.c typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, · line 195
Quoted 10 lines
drivers/net/failsafe/failsafe_args.c +10
-0+ fd =strtol (fd_str, &fd_str,0 ); +if (errno || *fd_str || fd <0 ) { +ERROR ("Parsing FD number failed" ); + goto error; + } +/* Fiddle with copy of file descriptor */ + fd =dup (fd); +if (fd ==-1 ) + goto error; + ret =fcntl (fd,F_GETFL );Quoted 9 lines
oflags = fcntl(...);
drivers/net/failsafe/failsafe_args.c typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, · line 198
Quoted 3 lines
drivers/net/failsafe/failsafe_args.c +3
-0+if (ret ==-1 ) + goto error; + ret =fcntl (fd,F_SETFL , fd |O_NONBLOCK );err =
fcntl(fd, F_SETFL, oflags | O_NONBLOCK); Using (fd |O_NONBLOCK) is probably a mistake.This is sneaky. err is
-ENODEVand would change to -1 on error, losing some meaning.Quoted 26 lines
drivers/net/failsafe/failsafe_args.c typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, · line 208
Quoted 10 lines
drivers/net/failsafe/failsafe_args.c +10
-0+if (ret ==-1 ) + goto error; + fp =fdopen (fd,"r" ); +if (!fp) + goto error; + fd =-1 ; +/* Only take the last line into account */ + ret =0 ; +while (fgets (output,sizeof (output), fp)) + ++ret;lcount = 0; while (fgets(output, sizeof(output), fp)) ++lcount;drivers/net/failsafe/failsafe_args.c typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, · line 217
Quoted 9 lines
drivers/net/failsafe/failsafe_args.c +9
-0+if (feof (fp)) { +if (!ret) + goto error; + }else if (ferror (fp)) { +if (errno !=EAGAIN || !ret) + goto error; + }else if (!ret) { + goto error; + }Quoted 7 lines
These branches seems needlessly complicated:
if (lcount == 0) goto error; else if (ferror(fp) && errno != EAGAIN) goto error;Here err would have been set to 0 previously with the fcntl call, meaning that jumping to error would return 0 as well.
I know Adrien wanted to avoid the usual ugly
if (error) { err = -ENODEV; goto error; }But this kind of sneakiness is not easy to parse and maintain. If someone adds a new path of error later, this kind of subtlety *will* be lost.
So between ugliness and maintainability, I choose maintainability (being the maintainer, of course).
--
Gaëtan Rivet
6WIND
PATCH v3 3/8 net/failsafe: support probed sub-devices getting
In reply to Matan AzradPrevious fail-safe code didn't support getting probed sub-devices and failed when it tried to probe them.
Skip fail-safe sub-device probing when it already was probed.
--- doc/guides/nics/fail_safe.rst | 5 ++++ drivers/net/failsafe/failsafe_eal.c | 60 ++++++++++++++++++++++++------------- 2 files changed, 45 insertions(+), 20 deletions(-)
doc/guides/nics/fail_safe.rst +5 -0
@@ -115,6 +115,11 @@ Fail-safe command line parameters order to take only the last line into account (unlike ``exec()``) at every probe attempt. +.. note:: + + In case of whitelist sub-device probed by EAL, fail-safe PMD will take the device + as is, which means that EAL device options are taken in this case. + - **mac** parameter [MAC address] This parameter allows the user to set a default MAC address to the fail-safe
drivers/net/failsafe/failsafe_eal.c +40 -20
79 lines not on this page · raw message
Signature · 1 line
1.8.3.1
Re: [PATCH v3 3/8] net/failsafe: support probed sub-devices getting
Hi Matan,
I'n not fond of the commit title, how about:
[PATCH v3 3/8] net/failsafe: add probed etherdev capture
?
Quoted 33 lines · Matan Azrad
Previous fail-safe code didn't support getting probed sub-devices and failed when it tried to probe them.
Skip fail-safe sub-device probing when it already was probed.
Signed-off-by Matan Azrad <[email protected]>Cc Gaetan Rivet <[email protected]>--- doc/guides/nics/fail_safe.rst | 5 ++++ drivers/net/failsafe/failsafe_eal.c | 60 ++++++++++++++++++++++++------------- 2 files changed, 45 insertions(+), 20 deletions(-)doc/guides/nics/fail_safe.rst +5
-0@@ -115,6 +115,11 @@ Fail-safe command line parameters order to take only the last line into account (unlike ``exec()``) at every probe attempt. +.. note:: + + In case of whitelist sub-device probed by EAL, fail-safe PMD will take the device + as is, which means that EAL device options are taken in this case. + - **mac** parameter [MAC address] This parameter allows the user to set a default MAC address to the fail-safedrivers/net/failsafe/failsafe_eal.c +0
-0
drivers/net/failsafe/failsafe_eal.c line 39
Quoted 4 lines
drivers/net/failsafe/failsafe_eal.c +1
-0@@ -36,39 +36,59 @@#include "failsafe_private.h" static int +fs_get_port_by_device_name (const char *name,uint16_t *port_id)
The naming convention for the failsafe driver is
namespace_object_sub-object_action()
With an ordering of objects by their scope (std, rte, failsafe, file). Also, "get" as an action is not descriptive enough.
static intfs_ethdev_capture(const char *name, uint16_t *port_id);
Quoted 56 lines
drivers/net/failsafe/failsafe_eal.c +29
-16+{ +uint16_t pid; +size_t len; + +if (name ==NULL ) { +DEBUG ("Null pointer is specified\n" ); +return -EINVAL ; + } + len =strlen (name); +RTE_ETH_FOREACH_DEV (pid) { +if (!strncmp (name, rte_eth_devices[pid].device->name, len)) { + *port_id = pid; +return 0 ; + } + } +return -ENODEV ; +} + +static int fs_bus_init (struct rte_eth_dev *dev) {struct sub_device *sdev;struct rte_devargs *da;uint8_t i; -uint16_t j; +uint16_t pid;int ret; FOREACH_SUBDEV(sdev, i, dev) {if (sdev->state !=DEV_PARSED )continue ; da = &sdev->devargs; - ret =rte_eal_hotplug_add (da->bus->name, - da->name, - da->args); -if (ret) { -ERROR ("sub_device %d probe failed %s%s%s" , i, - rte_errno ?"(" :"" , - rte_errno ?strerror (rte_errno) :"" , - rte_errno ?")" :"" ); -continue ; - } -RTE_ETH_FOREACH_DEV (j) { -if (strcmp (rte_eth_devices[j].device->name, - da->name) ==0 ) { -ETH (sdev) = &rte_eth_devices[j]; -break ; +if (fs_get_port_by_device_name (da->name, &pid) !=0 ) { + ret =rte_eal_hotplug_add (da->bus->name, + da->name, + da->args); +if (ret) { +ERROR ("sub_device %d probe failed %s%s%s" , i, + rte_errno ?"(" :"" , + rte_errno ?strerror (rte_errno) :"" , + rte_errno ?")" :"" );
drivers/net/failsafe/failsafe_eal.c line 89
Quoted 10 lines
drivers/net/failsafe/failsafe_eal.c +9
-0+continue ; } +if (fs_get_port_by_device_name (da->name, &pid) !=0 ) { +ERROR ("sub_device %d init went wrong" , i); +return -ENODEV ; + } + }else { +/* Take control of device probed by EAL options. */ +DEBUG ("Taking control of a probed sub device" +" %d named %s" , i, da->name);
In this case, the devargs of the probed device must be copied within the sub-device definition and removed from the EAL using the proper rte_devargs API.
Note that there is no rte_devargs copy function. You can use rte_devargs_parse instead, "parsing" again the original devargs into the sub-device one. It is necessary for complying with internal rte_devargs requirements (da->args being malloc-ed, at the moment, but may evolve).
The rte_eal_devargs_parse function is not easy enough to use right now, you will have to build a devargs string (using snprintf) and submit it. I proposed a change this release for it but it will not make it for 18.02, that would have simplified your implementation.
Signature · 2 lines
Gaëtan Rivet 6WIND
Re: [PATCH v3 3/8] net/failsafe: support probed sub-devices getting
Hi Gaetan
From: Gaëtan Rivet, Tuesday, January 16, 2018 1:09 PM
Quoted 8 lines
Hi Matan,
I'n not fond of the commit title, how about:
[PATCH v3 3/8] net/failsafe: add probed etherdev capture
?
OK, no problem.
Quoted 39 lines · Matan Azrad
Quoted 18 lines
Previous fail-safe code didn't support getting probed sub-devices and failed when it tried to probe them.
Skip fail-safe sub-device probing when it already was probed.
Signed-off-by Matan Azrad <[email protected]>Cc Gaetan Rivet <[email protected]>--- doc/guides/nics/fail_safe.rst | 5 ++++ drivers/net/failsafe/failsafe_eal.c | 60 ++++++++++++++++++++++++------------- 2 files changed, 45 insertions(+), 20 deletions(-)(quoted diff) +0
-0
b/doc/guides/nics/fail_safe.rstindex 5b1b47e..b89e53b 100644doc/guides/nics/fail_safe.rst Fail-safe command line parameters · line 120
Quoted 6 lines
doc/guides/nics/fail_safe.rst +3
-0@@ -115,6 +115,11 @@ Fail-safe command line parameters order to take only the last line into account (unlike ``exec()``) at every probe attempt. +.. note:: + + In case of whitelist sub-device probed by EAL, fail-safe PMD will take thedevice
Quoted 13 lines
doc/guides/nics/fail_safe.rst +2
-0+ as is, which means that EAL device options are taken in this case. + - **mac** parameter [MAC address] This parameter allows the user to set a default MAC address to thefail-safe diff --git
a/drivers/net/failsafe/failsafe_eal.cb/drivers/net/failsafe/failsafe_eal.c
index 19d26f5..7bc7453 100644drivers/net/failsafe/failsafe_eal.c +0
-0@@ -36,39 +36,59 @@#include "failsafe_private.h"
Quoted 7 lines
drivers/net/failsafe/failsafe_eal.c line 39
Quoted 2 lines
drivers/net/failsafe/failsafe_eal.c +1
-0static int +fs_get_port_by_device_name (const char *name,uint16_t *port_id)The naming convention for the failsafe driver is
namespace_object_sub-object_action()
OK.
Quoted 3 lines
With an ordering of objects by their scope (std, rte, failsafe, file). Also, "get" as an action is not descriptive enough.
Isn't "get by device name" descriptive?
Quoted 3 lines
static int
fs_ethdev_capture(const char *name, uint16_t *port_id);
You miss here the main reason why we need this function instead of using rte_eth_dev_get_port_by_name. The reason we need this function is because we want to find the device by the device name and not ethdev name. What's about fs_port_capture_by_device_name?
Maybe comparing it to device->devargs->name is better, What do you think?
Quoted 79 lines
drivers/net/failsafe/failsafe_eal.c
Quoted 11 lines
drivers/net/failsafe/failsafe_eal.c +11
-0+{ + uint16_t pid; + size_t len; + + if (name == NULL) { + DEBUG("Null pointer is specified\n"); + return -EINVAL; + } + len = strlen(name); + RTE_ETH_FOREACH_DEV(pid) { + if (!strncmp(name, rte_eth_devices[pid].device->name,len)) {
Quoted 32 lines
drivers/net/failsafe/failsafe_eal.c +9
-12+ *port_id = pid; +return 0 ; + } + } +return -ENODEV ; +} + +static int fs_bus_init (struct rte_eth_dev *dev) {struct sub_device *sdev;struct rte_devargs *da;uint8_t i; -uint16_t j; +uint16_t pid;int ret; FOREACH_SUBDEV(sdev, i, dev) {if (sdev->state !=DEV_PARSED )continue ; da = &sdev->devargs; - ret =rte_eal_hotplug_add (da->bus->name, - da->name, - da->args); -if (ret) { -ERROR ("sub_device %d probe failed %s%s%s" , i, - rte_errno ?"(" :"" , - rte_errno ?strerror (rte_errno) :"" , - rte_errno ?")" :"" ); -continue ; - } -RTE_ETH_FOREACH_DEV (j) {drivers/net/failsafe/failsafe_eal.c
Quoted 10 lines
drivers/net/failsafe/failsafe_eal.c +6
-4-if (strcmp (rte_eth_devices[j].device->name, - da->name) ==0 ) { -ETH (sdev) = &rte_eth_devices[j]; -break ; +if (fs_get_port_by_device_name (da->name, &pid) !=0 ) { + ret =rte_eal_hotplug_add (da->bus->name, + da->name, + da->args); +if (ret) { + ERROR("sub_device %d probe failed%s%s%s", i,
drivers/net/failsafe/failsafe_eal.c
Quoted 6 lines
drivers/net/failsafe/failsafe_eal.c +5
-0+ rte_errno ?"(" :"" , + rte_errno ?strerror (rte_errno) :"" , + rte_errno ? ")" : ""); +continue ; } + if (fs_get_port_by_device_name(da->name, &pid)!= 0) {
drivers/net/failsafe/failsafe_eal.c line 89
Quoted 7 lines
drivers/net/failsafe/failsafe_eal.c +7
-0+ERROR ("sub_device %d init went wrong" , i); +return -ENODEV ; + } + }else { +/* Take control of device probed by EAL options. */ +DEBUG ("Taking control of a probed sub device" +" %d named %s" , i, da->name);In this case, the devargs of the probed device must be copied within the sub- device definition and removed from the EAL using the proper
rte_devargsAPI.Note that there is no
rte_devargscopy function. You can userte_devargs_parseinstead, "parsing" again the original devargs into the sub- device one. It is necessary for complying with internalrte_devargsrequirements (da->argsbeing malloc-ed, at the moment, but may evolve).
Quoted 5 lines
The
rte_eal_devargs_parsefunction is not easy enough to use right now, you will have to build a devargs string (using snprintf) and submit it. I proposed a change this release for it but it will not make it for 18.02, that would have simplified your implementation.
Got you. You right we need to remove the created devargs in fail-safe parse level. What do you think about checking it in the parse level and avoid the new devargs creation? Also to do the copy in parse level(same method as we are doing in probe level)?
Quoted 3 lines
--
Gaëtan Rivet
6WIND
Re: [PATCH v3 3/8] net/failsafe: support probed sub-devices getting
Quoted 57 lines · Matan Azrad
Hi Gaetan
From: Gaëtan Rivet, Tuesday, January 16, 2018 1:09 PM
Quoted 8 lines
Hi Matan,
I'n not fond of the commit title, how about:
[PATCH v3 3/8] net/failsafe: add probed etherdev capture
?
OK, no problem.
Quoted 44 lines · Matan Azrad
Quoted 18 lines
Previous fail-safe code didn't support getting probed sub-devices and failed when it tried to probe them.
Skip fail-safe sub-device probing when it already was probed.
Signed-off-by Matan Azrad <[email protected]>Cc Gaetan Rivet <[email protected]>--- doc/guides/nics/fail_safe.rst | 5 ++++ drivers/net/failsafe/failsafe_eal.c | 60 ++++++++++++++++++++++++------------- 2 files changed, 45 insertions(+), 20 deletions(-)(quoted diff) +0
-0
b/doc/guides/nics/fail_safe.rstindex 5b1b47e..b89e53b 100644doc/guides/nics/fail_safe.rst Fail-safe command line parameters · line 120
Quoted 6 lines
doc/guides/nics/fail_safe.rst +3
-0@@ -115,6 +115,11 @@ Fail-safe command line parameters order to take only the last line into account (unlike ``exec()``) at every probe attempt. +.. note:: + + In case of whitelist sub-device probed by EAL, fail-safe PMD will take thedevice
Quoted 11 lines
doc/guides/nics/fail_safe.rst +2
-0+ as is, which means that EAL device options are taken in this case. + - **mac** parameter [MAC address] This parameter allows the user to set a default MAC address to thefail-safe diff --git
a/drivers/net/failsafe/failsafe_eal.cb/drivers/net/failsafe/failsafe_eal.c
index 19d26f5..7bc7453 100644drivers/net/failsafe/failsafe_eal.c line 39
Quoted 4 lines
drivers/net/failsafe/failsafe_eal.c +1
-0@@ -36,39 +36,59 @@#include "failsafe_private.h" static int +fs_get_port_by_device_name (const char *name,uint16_t *port_id)The naming convention for the failsafe driver is
Quoted 7 lines
Quoted 2 lines
namespace_object_sub-object_action()
OK.
Quoted 3 lines
With an ordering of objects by their scope (std, rte, failsafe, file). Also, "get" as an action is not descriptive enough.
Isn't "get by device name" descriptive?
The endgame is capturing a device that we know we are interested in. The device name being used for matching is an implementation detail, which should be abstracted by using a sub-function.
Putting this in the name defeat the reason for using another function.
Quoted 6 lines
Quoted 3 lines
static int
fs_ethdev_capture(const char *name, uint16_t *port_id);You miss here the main reason why we need this function instead of using
rte_eth_dev_get_port_by_name. The reason we need this function is because we want to find the device by the device name and not ethdev name. What's aboutfs_port_capture_by_device_name?
You are getting a port_id that is only valid for the rte_eth_devices array, by using the ethdev iterator. You are only looking for an ethdev.
So it doesn't really matter whether you are using the ethdev name or the device name, in the end you are capturing an ethdev
--> fs_ethdev_capture seems good for me.
Now, I guess you will say that the user would need to know that they have to provide a device name that would be written in device->name. The issue here is that you have a leaky abstraction for your function, forcing this kind of consideration on your function user.
So I'd go further and will ask you to change the const char *name to a const rte_devargs *da in the parameters.
Quoted 2 lines
Maybe comparing it to
device->devargs->nameis better, What do you think?
You are touching at a pretty contentious subject here :) .
Identifying devices is not currently a well-defined function in DPDK. Some ports (actually, only one model: ConnectX-3) will have several ports using the same PCI slot. But even ignoring this glaring problem...
As it is, the device->name for PCI will match the name given as a devargs, so functionally this should not change anything.
Furthermore, you will have devices probed without any devargs. The fail-safe would thus be unable to capture non-blacklisted devices when the PCI bus is in blacklist mode.
These not-blacklisted devices actually will have a full-PCI name (DomBDF format), so a simple match with the one passed in your fail-safe devargs will fail, ex:
# A physical port exists at 0000:00:02.0 testpmd --vdev="net_failsafe,dev(00:02.0)" -- -i
Would fail to capture the device 0000:00:02.0, as this is the name that the PCI bus would give to this device, in the absence of a user-given name.
In 18.05, or 18.08 there should be an EAL function that would be able to identify a device given a specific ID string (very close to an rte_devargs). Currently, this API does not exist.
You can hack your way around this for the moment, IF you really, really
want: parse your devargs, get the bus, use the bus->parse() function to get a binary device representation, and compare bytes per bytes the binary representation given by your devargs and by the device->name.
But this is a hack, and a pretty ugly one at that: you have no way of knowing the size taken by this binary representation, so you can restrict yourself to the vdev and PCI bus for the moment and take the larger of an rte_vdev_driver pointer and an rte_pci_addr....
{
union {
rte_vdev_driver *drv;
struct rte_pci_addr pci_addr;
} bindev1, bindev2;
memset(&bindev1, 0, sizeof(bindev1));
memset(&bindev2, 0, sizeof(bindev2));
rte_eal_devargs_parse(device->name, da1);
rte_eal_devargs_parse(your_devstr, da2);
RTE_ASSERT(da1->bus == rte_bus_find_by_name("pci") ||
da1->bus == rte_bus_find_by_name("vdev"));
RTE_ASSERT(da2->bus == rte_bus_find_by_name("pci") ||
da2->bus == rte_bus_find_by_name("vdev"));
da1->bus->parse(da1->name, &bindev1);
da1->bus->parse(da2->name, &bindev2);
if (memcmp(&bindev1, &bindev2, sizeof(bindev1)) == 0) {
/* found the device */
} else {
/* not found */
}
}So, really, really ugly. Anyway.
<snip>
Quoted 19 lines
Quoted 13 lines
drivers/net/failsafe/failsafe_eal.c line 89
Quoted 3 lines
drivers/net/failsafe/failsafe_eal.c +3
-0+/* Take control of device probed by EAL options. */ +DEBUG ("Taking control of a probed sub device" +" %d named %s" , i, da->name);In this case, the devargs of the probed device must be copied within the sub- device definition and removed from the EAL using the proper
rte_devargsAPI.Note that there is no
rte_devargscopy function. You can userte_devargs_parseinstead, "parsing" again the original devargs into the sub- device one. It is necessary for complying with internalrte_devargsrequirements (da->argsbeing malloc-ed, at the moment, but may evolve).Quoted 5 lines
The
rte_eal_devargs_parsefunction is not easy enough to use right now, you will have to build a devargs string (using snprintf) and submit it. I proposed a change this release for it but it will not make it for 18.02, that would have simplified your implementation.
Quoted 4 lines
Got you. You right we need to remove the created devargs in fail-safe parse level. What do you think about checking it in the parse level and avoid the new devargs creation? Also to do the copy in parse level(same method as we are doing in probe level)?
Not sure I follow here, but the new rte_devargs is part of the sub-device (it is not a pointer, but allocated alongside the sub_device).
So keep everything here, it is the right place to deal with these things.
Signature · 2 lines
Gaëtan Rivet 6WIND
Re: [PATCH v3 3/8] net/failsafe: support probed sub-devices getting
Hi Gaetan
From: Gaëtan Rivet, Tuesday, January 16, 2018 4:41 PM
Quoted 80 lines · Matan Azrad
Quoted 58 lines
Hi Gaetan
From: Gaëtan Rivet, Tuesday, January 16, 2018 1:09 PM
Quoted 8 lines
Hi Matan,
I'n not fond of the commit title, how about:
[PATCH v3 3/8] net/failsafe: add probed etherdev capture
?
OK, no problem.
Quoted 45 lines · Matan Azrad
Quoted 18 lines
Previous fail-safe code didn't support getting probed sub-devices and failed when it tried to probe them.
Skip fail-safe sub-device probing when it already was probed.
Signed-off-by Matan Azrad <[email protected]>Cc Gaetan Rivet <[email protected]>--- doc/guides/nics/fail_safe.rst | 5 ++++ drivers/net/failsafe/failsafe_eal.c | 60 ++++++++++++++++++++++++------------- 2 files changed, 45 insertions(+), 20 deletions(-)(quoted diff) +0
-0
b/doc/guides/nics/fail_safe.rstindex 5b1b47e..b89e53b 100644doc/guides/nics/fail_safe.rst Fail-safe command line parameters · line 121
Quoted 7 lines
doc/guides/nics/fail_safe.rst +4
-0@@ -115,6 +115,11 @@ Fail-safe command line parameters order to take only the last line into account (unlike ``exec()``) at every probe attempt. +.. note:: + + In case of whitelist sub-device probed by EAL, fail-safe PMD + will take thedevice
Quoted 11 lines
doc/guides/nics/fail_safe.rst +2
-0+ as is, which means that EAL device options are taken in this case. + - **mac** parameter [MAC address] This parameter allows the user to set a default MAC address tothe fail-safe diff --git
a/drivers/net/failsafe/failsafe_eal.cb/drivers/net/failsafe/failsafe_eal.c
index 19d26f5..7bc7453 100644drivers/net/failsafe/failsafe_eal.c line 39
Quoted 4 lines
drivers/net/failsafe/failsafe_eal.c +1
-0@@ -36,39 +36,59 @@#include "failsafe_private.h" static int +fs_get_port_by_device_name (const char *name,uint16_t *port_id)The naming convention for the failsafe driver is
Quoted 7 lines
Quoted 2 lines
namespace_object_sub-object_action()
OK.
Quoted 3 lines
With an ordering of objects by their scope (std, rte, failsafe, file). Also, "get" as an action is not descriptive enough.
Isn't "get by device name" descriptive?
The endgame is capturing a device that we know we are interested in. The device name being used for matching is an implementation detail, which should be abstracted by using a sub-function.
Putting this in the name defeat the reason for using another function.
Quoted 4 lines
Quoted 3 lines
static int
fs_ethdev_capture(const char *name, uint16_t *port_id);You miss here the main reason why we need this function instead of using
rte_eth_dev_get_port_by_name.Quoted 1 line
The reason we need this function is because we want to find the device by
the device name and not ethdev name.
Quoted 9 lines
Quoted 1 line
What's about
fs_port_capture_by_device_name?You are getting a
port_idthat is only valid for therte_eth_devicesarray, by using the ethdev iterator. You are only looking for an ethdev.So it doesn't really matter whether you are using the ethdev name or the device name, in the end you are capturing an ethdev
-->fs_ethdev_captureseems good for me.
I don't think so, this function doesn't take(capture) the device, just gets its ethdev port id using the device name. The function which actually captures the device is the fs_bus_init.
So maybe even the "capture" name looks problematic here.
The main idea of this function is just to get the port_id.
Quoted 71 lines
Now, I guess you will say that the user would need to know that they have to provide a device name that would be written in
device->name. The issue here is that you have a leaky abstraction for your function, forcing this kind of consideration on your function user.So I'd go further and will ask you to change the
const char *nameto aconst rte_devargs *dain the parameters.Quoted 1 line
Maybe comparing it to
device->devargs->nameis better, What do youthink?
Quoted 1 line
You are touching at a pretty contentious subject here :) .
Identifying devices is not currently a well-defined function in DPDK. Some ports (actually, only one model: ConnectX-3) will have several ports using the same PCI slot. But even ignoring this glaring problem...
As it is, the
device->namefor PCI will match the name given as a devargs, so functionally this should not change anything.Furthermore, you will have devices probed without any devargs. The fail- safe would thus be unable to capture non-blacklisted devices when the PCI bus is in blacklist mode.
These not-blacklisted devices actually will have a full-PCI name (DomBDF format), so a simple match with the one passed in your fail-safe devargs will fail, ex:
# A physical port exists at 0000:00:02.0 testpmd --vdev="net_failsafe,dev(00:02.0)" -- -iWould fail to capture the device 0000:00:02.0, as this is the name that the PCI bus would give to this device, in the absence of a user-given name.
In 18.05, or 18.08 there should be an EAL function that would be able to identify a device given a specific ID string (very close to an
rte_devargs). Currently, this API does not exist.You can hack your way around this for the moment, IF you really, really
want: parse your devargs, get the bus, use thebus->parse()function to get a binary device representation, and compare bytes per bytes the binary representation given by your devargs and by thedevice->name.But this is a hack, and a pretty ugly one at that: you have no way of knowing the size taken by this binary representation, so you can restrict yourself to the vdev and PCI bus for the moment and take the larger of an
rte_vdev_driverpointer and anrte_pci_addr....{ union { rte_vdev_driver *drv; struct rte_pci_addr pci_addr; } bindev1, bindev2; memset(&bindev1, 0, sizeof(bindev1)); memset(&bindev2, 0, sizeof(bindev2)); rte_eal_devargs_parse(device->name, da1); rte_eal_devargs_parse(your_devstr, da2); RTE_ASSERT(da1->bus == rte_bus_find_by_name("pci") || da1->bus == rte_bus_find_by_name("vdev")); RTE_ASSERT(da2->bus == rte_bus_find_by_name("pci") || da2->bus == rte_bus_find_by_name("vdev")); da1->bus->parse(da1->name, &bindev1); da1->bus->parse(da2->name, &bindev2); if (memcmp(&bindev1, &bindev2, sizeof(bindev1)) == 0) { /* found the device */ } else { /* not found */ } }
Quoted 2 lines
So, really, really ugly. Anyway.
Yes, ugly :) Thanks for this update!
Will keep the comparison by device->name.
Quoted 26 lines
<snip>
Quoted 8 lines
Quoted 8 lines
drivers/net/failsafe/failsafe_eal.c line 89
Quoted 3 lines
drivers/net/failsafe/failsafe_eal.c +3
-0+/* Take control of device probed by EAL options. */ +DEBUG ("Taking control of a probed sub device" +" %d named %s" , i, da->name);In this case, the devargs of the probed device must be copied within the sub- device definition and removed from the EAL using the proper
rte_devargsAPI.Quoted 4 lines
Quoted 4 lines
Note that there is no
rte_devargscopy function. You can userte_devargs_parseinstead, "parsing" again the original devargs into the sub- device one. It is necessary for complying with internalrte_devargsrequirements (da->argsbeing malloc-ed, at the moment,but may evolve).
Quoted 8 lines
Quoted 6 lines
The
rte_eal_devargs_parsefunction is not easy enough to use right now, you will have to build a devargs string (using snprintf) and submit it. I proposed a change this release for it but it will not make it for 18.02, that would have simplified your implementation.Got you. You right we need to remove the created devargs in fail-safe
parse level.
Quoted 1 line
What do you think about checking it in the parse level and avoid the new
devargs creation?
Quoted 9 lines
Quoted 1 line
Also to do the copy in parse level(same method as we are doing in probe
level)?
Quoted 1 line
Not sure I follow here, but the new
rte_devargsis part of the sub-device (it is not a pointer, but allocated alongside thesub_device).So keep everything here, it is the right place to deal with these things.
But it will prevent the double parsing and also saves the method:
If the device already parsed - copy its devargs and continue. If the device already probed - copy the device pointer and continue.
I think this is the right dealing, no?
Why to deal with parse level in probe level? Just keep all the parse work to parse level and the probe work to probe level.
Thanks, Matan.
Quoted 3 lines
--
Gaëtan Rivet
6WIND
Re: [PATCH v3 3/8] net/failsafe: support probed sub-devices getting
Quoted 93 lines · Matan Azrad
Hi Gaetan
From: Gaëtan Rivet, Tuesday, January 16, 2018 4:41 PM
Quoted 80 lines · Matan Azrad
Quoted 58 lines
Hi Gaetan
From: Gaëtan Rivet, Tuesday, January 16, 2018 1:09 PM
Quoted 8 lines
Hi Matan,
I'n not fond of the commit title, how about:
[PATCH v3 3/8] net/failsafe: add probed etherdev capture
?
OK, no problem.
Quoted 45 lines · Matan Azrad
Quoted 18 lines
Previous fail-safe code didn't support getting probed sub-devices and failed when it tried to probe them.
Skip fail-safe sub-device probing when it already was probed.
Signed-off-by Matan Azrad <[email protected]>Cc Gaetan Rivet <[email protected]>--- doc/guides/nics/fail_safe.rst | 5 ++++ drivers/net/failsafe/failsafe_eal.c | 60 ++++++++++++++++++++++++------------- 2 files changed, 45 insertions(+), 20 deletions(-)(quoted diff) +0
-0
b/doc/guides/nics/fail_safe.rstindex 5b1b47e..b89e53b 100644doc/guides/nics/fail_safe.rst Fail-safe command line parameters · line 121
Quoted 7 lines
doc/guides/nics/fail_safe.rst +4
-0@@ -115,6 +115,11 @@ Fail-safe command line parameters order to take only the last line into account (unlike ``exec()``) at every probe attempt. +.. note:: + + In case of whitelist sub-device probed by EAL, fail-safe PMD + will take thedevice
Quoted 11 lines
doc/guides/nics/fail_safe.rst +2
-0+ as is, which means that EAL device options are taken in this case. + - **mac** parameter [MAC address] This parameter allows the user to set a default MAC address tothe fail-safe diff --git
a/drivers/net/failsafe/failsafe_eal.cb/drivers/net/failsafe/failsafe_eal.c
index 19d26f5..7bc7453 100644drivers/net/failsafe/failsafe_eal.c line 39
Quoted 4 lines
drivers/net/failsafe/failsafe_eal.c +1
-0@@ -36,39 +36,59 @@#include "failsafe_private.h" static int +fs_get_port_by_device_name (const char *name,uint16_t *port_id)The naming convention for the failsafe driver is
Quoted 7 lines
Quoted 2 lines
namespace_object_sub-object_action()
OK.
Quoted 3 lines
With an ordering of objects by their scope (std, rte, failsafe, file). Also, "get" as an action is not descriptive enough.
Isn't "get by device name" descriptive?
The endgame is capturing a device that we know we are interested in. The device name being used for matching is an implementation detail, which should be abstracted by using a sub-function.
Putting this in the name defeat the reason for using another function.
Quoted 4 lines
Quoted 3 lines
static int
fs_ethdev_capture(const char *name, uint16_t *port_id);You miss here the main reason why we need this function instead of using
rte_eth_dev_get_port_by_name.Quoted 1 line
The reason we need this function is because we want to find the device by
the device name and not ethdev name.
Quoted 9 lines
Quoted 1 line
What's about
fs_port_capture_by_device_name?You are getting a
port_idthat is only valid for therte_eth_devicesarray, by using the ethdev iterator. You are only looking for an ethdev.So it doesn't really matter whether you are using the ethdev name or the device name, in the end you are capturing an ethdev
-->fs_ethdev_captureseems good for me.
Quoted 5 lines
I don't think so, this function doesn't take(capture) the device, just gets its ethdev port id using the device name. The function which actually captures the device is the
fs_bus_init.
So maybe even the "capture" name looks problematic here.
The main idea of this function is just to get theport_id.
Right :) . Call it fs_ethdev_portid_get() or fs_ethdev_find() then.
Quoted 71 lines
Quoted 71 lines
Now, I guess you will say that the user would need to know that they have to provide a device name that would be written in
device->name. The issue here is that you have a leaky abstraction for your function, forcing this kind of consideration on your function user.So I'd go further and will ask you to change the
const char *nameto aconst rte_devargs *dain the parameters.Quoted 1 line
Maybe comparing it to
device->devargs->nameis better, What do youthink?
Quoted 1 line
You are touching at a pretty contentious subject here :) .
Identifying devices is not currently a well-defined function in DPDK. Some ports (actually, only one model: ConnectX-3) will have several ports using the same PCI slot. But even ignoring this glaring problem...
As it is, the
device->namefor PCI will match the name given as a devargs, so functionally this should not change anything.Furthermore, you will have devices probed without any devargs. The fail- safe would thus be unable to capture non-blacklisted devices when the PCI bus is in blacklist mode.
These not-blacklisted devices actually will have a full-PCI name (DomBDF format), so a simple match with the one passed in your fail-safe devargs will fail, ex:
# A physical port exists at 0000:00:02.0 testpmd --vdev="net_failsafe,dev(00:02.0)" -- -iWould fail to capture the device 0000:00:02.0, as this is the name that the PCI bus would give to this device, in the absence of a user-given name.
In 18.05, or 18.08 there should be an EAL function that would be able to identify a device given a specific ID string (very close to an
rte_devargs). Currently, this API does not exist.You can hack your way around this for the moment, IF you really, really
want: parse your devargs, get the bus, use thebus->parse()function to get a binary device representation, and compare bytes per bytes the binary representation given by your devargs and by thedevice->name.But this is a hack, and a pretty ugly one at that: you have no way of knowing the size taken by this binary representation, so you can restrict yourself to the vdev and PCI bus for the moment and take the larger of an
rte_vdev_driverpointer and anrte_pci_addr....{ union { rte_vdev_driver *drv; struct rte_pci_addr pci_addr; } bindev1, bindev2; memset(&bindev1, 0, sizeof(bindev1)); memset(&bindev2, 0, sizeof(bindev2)); rte_eal_devargs_parse(device->name, da1); rte_eal_devargs_parse(your_devstr, da2); RTE_ASSERT(da1->bus == rte_bus_find_by_name("pci") || da1->bus == rte_bus_find_by_name("vdev")); RTE_ASSERT(da2->bus == rte_bus_find_by_name("pci") || da2->bus == rte_bus_find_by_name("vdev")); da1->bus->parse(da1->name, &bindev1); da1->bus->parse(da2->name, &bindev2); if (memcmp(&bindev1, &bindev2, sizeof(bindev1)) == 0) { /* found the device */ } else { /* not found */ } }
Quoted 5 lines
Quoted 2 lines
So, really, really ugly. Anyway.
Yes, ugly :) Thanks for this update!
Will keep the comparison bydevice->name.
Well as explained, above, the comparison by device->name only works with whitelisted devices.
So either implement something broken right now that you will need to update in 18.05, or implement it properly in 18.05 from the get go.
Quoted 33 lines
Quoted 33 lines
<snip>
Quoted 8 lines
Quoted 8 lines
drivers/net/failsafe/failsafe_eal.c line 89
Quoted 3 lines
drivers/net/failsafe/failsafe_eal.c +3
-0+/* Take control of device probed by EAL options. */ +DEBUG ("Taking control of a probed sub device" +" %d named %s" , i, da->name);In this case, the devargs of the probed device must be copied within the sub- device definition and removed from the EAL using the proper
rte_devargsAPI.Quoted 4 lines
Quoted 4 lines
Note that there is no
rte_devargscopy function. You can userte_devargs_parseinstead, "parsing" again the original devargs into the sub- device one. It is necessary for complying with internalrte_devargsrequirements (da->argsbeing malloc-ed, at the moment,but may evolve).
Quoted 8 lines
Quoted 6 lines
The
rte_eal_devargs_parsefunction is not easy enough to use right now, you will have to build a devargs string (using snprintf) and submit it. I proposed a change this release for it but it will not make it for 18.02, that would have simplified your implementation.Got you. You right we need to remove the created devargs in fail-safe
parse level.
Quoted 1 line
What do you think about checking it in the parse level and avoid the new
devargs creation?
Quoted 1 line
Also to do the copy in parse level(same method as we are doing in probe
level)?
Quoted 1 line
Not sure I follow here, but the new
rte_devargsis part of the sub-device (it is not a pointer, but allocated alongside thesub_device).
Quoted 8 lines
Quoted 2 lines
So keep everything here, it is the right place to deal with these things.
But it will prevent the double parsing and also saves the method:
If the device already parsed - copy its devargs and continue. If the device already probed - copy the device pointer and continue.I think this is the right dealing, no?
Why to deal with parse level in probe level? Just keep all the parse work to parse level and the probe work to probe level.
After re-reading, I think we misunderstood each other. You cannot remove the rte_devargs created during parsing: it is allocated alongside the sub_device structure.
You must only remove the rte_devargs allocated by the EAL (using rte_eal_devargs_remove()).
Before removing it, you must copy its content in the local sub_device rte_devargs structure. I only proposed a way to do this copy that would not deal with rte_devargs internals, as it is bound to evolve rather soon.
Otherwise, no, I do not want to complicate the parsing operations, they are already too complicated and too criticals. Better to keep it all here.
Signature · 2 lines
Gaëtan Rivet 6WIND
Re: [PATCH v3 3/8] net/failsafe: support probed sub-devices getting
Hi Gaetan
From: Gaëtan Rivet, Tuesday, January 16, 2018 6:54 PM
Quoted 99 lines · Matan Azrad
Quoted 31 lines
Hi Gaetan
From: Gaëtan Rivet, Tuesday, January 16, 2018 4:41 PM
Quoted 28 lines · Matan Azrad
Quoted 27 lines
Hi Gaetan
From: Gaëtan Rivet, Tuesday, January 16, 2018 1:09 PM
Quoted 8 lines
Hi Matan,
I'n not fond of the commit title, how about:
[PATCH v3 3/8] net/failsafe: add probed etherdev capture
?
OK, no problem.
Quoted 14 lines · Matan Azrad
Quoted 13 lines
Previous fail-safe code didn't support getting probed sub-devices and failed when it tried to probe them.
Skip fail-safe sub-device probing when it already was probed.
Signed-off-by Matan Azrad <[email protected]>Cc Gaetan Rivet <[email protected]>--- doc/guides/nics/fail_safe.rst | 5 ++++ drivers/net/failsafe/failsafe_eal.c | 60 ++++++++++++++++++++++++------------- 2 files changed, 45 insertions(+), 20 deletions(-)Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
doc/guides/nics/fail_safe.rst Fail-safe command line parameters · line 115
Quoted 6 lines
(quoted diff) +0
-0
b/doc/guides/nics/fail_safe.rstindex 5b1b47e..b89e53b 100644doc/guides/nics/fail_safe.rst +0
-0@@ -115,6 +115,11 @@ Fail-safe command line parameters order to take only the last line into account (unlike ``exec()``) atevery
Quoted 60 lines
Quoted 59 lines
Quoted 27 lines
Quoted 27 lines
doc/guides/nics/fail_safe.rst Fail-safe command line parameters
Quoted 6 lines
doc/guides/nics/fail_safe.rst +4
-0probe attempt. +.. note:: + + In case of whitelist sub-device probed by EAL, fail-safe + PMD will take thedevice
Quoted 12 lines
doc/guides/nics/fail_safe.rst +2
-0+ as is, which means that EAL device options are taken in this case. + - **mac** parameter [MAC address] This parameter allows the user to set a default MAC addressto the fail-safe diff --git
a/drivers/net/failsafe/failsafe_eal.cb/drivers/net/failsafe/failsafe_eal.c
index 19d26f5..7bc7453 100644drivers/net/failsafe/failsafe_eal.c line 40
Quoted 5 lines
drivers/net/failsafe/failsafe_eal.c +2
-0@@ -36,39 +36,59 @@#include "failsafe_private.h" static int +fs_get_port_by_device_name (const char *name,uint16_t +*port_id)The naming convention for the failsafe driver is
Quoted 7 lines
Quoted 2 lines
namespace_object_sub-object_action()
OK.
Quoted 3 lines
With an ordering of objects by their scope (std, rte, failsafe, file). Also, "get" as an action is not descriptive enough.
Isn't "get by device name" descriptive?
The endgame is capturing a device that we know we are interested in. The device name being used for matching is an implementation detail, which should be abstracted by using a sub-function.
Putting this in the name defeat the reason for using another function.
Quoted 5 lines
Quoted 3 lines
static int
fs_ethdev_capture(const char *name, uint16_t *port_id);You miss here the main reason why we need this function instead of using
rte_eth_dev_get_port_by_name.Quoted 2 lines
The reason we need this function is because we want to find the device by
the device name and not ethdev name.
Quoted 1 line
What's about
fs_port_capture_by_device_name?You are getting a
port_idthat is only valid for therte_eth_devicesarray, by using the ethdev iterator. You are only looking for an ethdev.So it doesn't really matter whether you are using the ethdev name or the device name, in the end you are capturing an ethdev -->
fs_ethdev_captureseems good for me.
Quoted 9 lines
Quoted 1 line
I don't think so, this function doesn't take(capture) the device, just gets its
ethdev port id using the device name.
Quoted 4 lines
The function which actually captures the device is the
fs_bus_init. So maybe even the "capture" name looks problematic here. The main idea of this function is just to get theport_id.Right :) . Call it
fs_ethdev_portid_get()orfs_ethdev_find()then.
Sure, agree with the first one.
Quoted 81 lines
Quoted 29 lines
Quoted 29 lines
Now, I guess you will say that the user would need to know that they have to provide a device name that would be written in
device->name. The issue here is that you have a leaky abstraction for your function, forcing this kind of consideration on your function user.So I'd go further and will ask you to change the
const char *nameto aconst rte_devargs *dain the parameters.Quoted 1 line
Maybe comparing it to
device->devargs->nameis better, What do youthink?
Quoted 1 line
You are touching at a pretty contentious subject here :) .
Identifying devices is not currently a well-defined function in DPDK. Some ports (actually, only one model: ConnectX-3) will have several ports using the same PCI slot. But even ignoring this glaring problem...
As it is, the
device->namefor PCI will match the name given as a devargs, so functionally this should not change anything.Furthermore, you will have devices probed without any devargs. The fail- safe would thus be unable to capture non-blacklisted devices when the PCI bus is in blacklist mode.
These not-blacklisted devices actually will have a full-PCI name (DomBDF format), so a simple match with the one passed in your fail-safe devargs will fail, ex:
Quoted 5 lines
Quoted 5 lines
# A physical port exists at 0000:00:02.0 testpmd --vdev="net_failsafe,dev(00:02.0)" -- -iWould fail to capture the device 0000:00:02.0, as this is the name that the PCI bus would give to this device, in the absence of a user-given
name.
Quoted 3 lines
Quoted 3 lines
In 18.05, or 18.08 there should be an EAL function that would be able to identify a device given a specific ID string (very close to an
rte_devargs).Quoted 36 lines
Quoted 7 lines
Currently, this API does not exist.
You can hack your way around this for the moment, IF you really, really
want: parse your devargs, get the bus, use thebus->parse()function to get a binary device representation, and compare bytes per bytes the binary representation given by your devargs and by the device-name.
Quoted 28 lines
But this is a hack, and a pretty ugly one at that: you have no way of knowing the size taken by this binary representation, so you can restrict yourself to the vdev and PCI bus for the moment and take the larger of an
rte_vdev_driverpointer and anrte_pci_addr....{ union { rte_vdev_driver *drv; struct rte_pci_addr pci_addr; } bindev1, bindev2; memset(&bindev1, 0, sizeof(bindev1)); memset(&bindev2, 0, sizeof(bindev2)); rte_eal_devargs_parse(device->name, da1); rte_eal_devargs_parse(your_devstr, da2); RTE_ASSERT(da1->bus == rte_bus_find_by_name("pci") || da1->bus == rte_bus_find_by_name("vdev")); RTE_ASSERT(da2->bus == rte_bus_find_by_name("pci") || da2->bus == rte_bus_find_by_name("vdev")); da1->bus->parse(da1->name, &bindev1); da1->bus->parse(da2->name, &bindev2); if (memcmp(&bindev1, &bindev2, sizeof(bindev1)) == 0) { /* found the device */ } else { /* not found */ } }Quoted 5 lines
Quoted 2 lines
So, really, really ugly. Anyway.
Yes, ugly :) Thanks for this update!
Will keep the comparison bydevice->name.
Quoted 7 lines
Well as explained, above, the comparison by
device->nameonly works with whitelisted devices.So either implement something broken right now that you will need to update in 18.05, or implement it properly in 18.05 from the get go.
For the current needs it is enough.
We can also say that it is the user responsibility to pass to failsafe the same names and same args as he passes for EAL(or default EAL names). I think I emphasized it in documentation.
Quoted 48 lines
Quoted 3 lines
Quoted 3 lines
<snip>
Quoted 1 line
Quoted 1 line
drivers/net/failsafe/failsafe_eal.c
Quoted 1 line
drivers/net/failsafe/failsafe_eal.c +1
-0+/* Take control of device probed by EAL options. */
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
drivers/net/failsafe/failsafe_eal.c
Quoted 1 line
drivers/net/failsafe/failsafe_eal.c +1
-0+ DEBUG("Taking control of a probed subdevice"
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
drivers/net/failsafe/failsafe_eal.c line 89
Quoted 1 line
drivers/net/failsafe/failsafe_eal.c +1
-0+ " %d named %s", i, da->name);In this case, the devargs of the probed device must be copied within the sub- device definition and removed from the EAL using the proper
rte_devargsAPI.Note that there is no
rte_devargscopy function. You can userte_devargs_parseinstead, "parsing" again the original devargs into the sub- device one. It is necessary for complying with internalrte_devargsrequirements (da->argsbeing malloc-ed, at the moment,Quoted 4 lines
Quoted 4 lines
but may evolve).
Quoted 3 lines
Quoted 3 lines
The
rte_eal_devargs_parsefunction is not easy enough to use right now, you will have to build a devargs string (using snprintf) andsubmit it.
Quoted 18 lines
Quoted 18 lines
Quoted 6 lines
Quoted 3 lines
I proposed a change this release for it but it will not make it for 18.02, that would have simplified your implementation.
Got you. You right we need to remove the created devargs in fail-safe
parse level.
Quoted 2 lines
What do you think about checking it in the parse level and avoid the new
devargs creation?
Quoted 2 lines
Also to do the copy in parse level(same method as we are doing in probe
level)?
Quoted 1 line
Not sure I follow here, but the new
rte_devargsis part of the sub-device (it is not a pointer, but allocated alongside thesub_device).Quoted 8 lines
Quoted 2 lines
So keep everything here, it is the right place to deal with these things.
But it will prevent the double parsing and also saves the method:
If the device already parsed - copy its devargs and continue. If the device already probed - copy the device pointer and continue.I think this is the right dealing, no?
Why to deal with parse level in probe level? Just keep all the parse work to
Quoted 9 lines
parse level and the probe work to probe level.
After re-reading, I think we misunderstood each other.
You cannot remove therte_devargscreated during parsing: it is allocated alongside thesub_devicestructure.You must only remove the
rte_devargsallocated by the EAL (usingrte_eal_devargs_remove()).
Sure.
Quoted 4 lines
Before removing it, you must copy its content in the local
sub_devicerte_devargsstructure. I only proposed a way to do this copy that would not deal withrte_devargsinternals, as it is bound to evolve rather soon.
Yes.
Quoted 2 lines
Otherwise, no, I do not want to complicate the parsing operations, they are already too complicated and too criticals. Better to keep it all here.
I think fs_parse_device function is not complicated and it is the natural place for devargs games. For me this is the right place for the copy & remove devargs.
Are you insisting to put all in fs_bus_init?
Quoted 3 lines
--
Gaëtan Rivet
6WIND
Re: [PATCH v3 3/8] net/failsafe: support probed sub-devices getting
Hi Matan,
Quoted 2 lines · Matan Azrad
Hi Gaetan
<snip>
Quoted 49 lines
Quoted 48 lines
Quoted 2 lines
Quoted 2 lines
In 18.05, or 18.08 there should be an EAL function that would be able to identify a device given a specific ID string (very close to an
rte_devargs).Quoted 36 lines
Quoted 7 lines
Currently, this API does not exist.
You can hack your way around this for the moment, IF you really, really
want: parse your devargs, get the bus, use thebus->parse()function to get a binary device representation, and compare bytes per bytes the binary representation given by your devargs and by the device-name.
Quoted 28 lines
But this is a hack, and a pretty ugly one at that: you have no way of knowing the size taken by this binary representation, so you can restrict yourself to the vdev and PCI bus for the moment and take the larger of an
rte_vdev_driverpointer and anrte_pci_addr....{ union { rte_vdev_driver *drv; struct rte_pci_addr pci_addr; } bindev1, bindev2; memset(&bindev1, 0, sizeof(bindev1)); memset(&bindev2, 0, sizeof(bindev2)); rte_eal_devargs_parse(device->name, da1); rte_eal_devargs_parse(your_devstr, da2); RTE_ASSERT(da1->bus == rte_bus_find_by_name("pci") || da1->bus == rte_bus_find_by_name("vdev")); RTE_ASSERT(da2->bus == rte_bus_find_by_name("pci") || da2->bus == rte_bus_find_by_name("vdev")); da1->bus->parse(da1->name, &bindev1); da1->bus->parse(da2->name, &bindev2); if (memcmp(&bindev1, &bindev2, sizeof(bindev1)) == 0) { /* found the device */ } else { /* not found */ } }Quoted 5 lines
Quoted 2 lines
So, really, really ugly. Anyway.
Yes, ugly :) Thanks for this update!
Will keep the comparison bydevice->name.Well as explained, above, the comparison by
device->nameonly works with whitelisted devices.
Quoted 7 lines
Quoted 3 lines
So either implement something broken right now that you will need to update in 18.05, or implement it properly in 18.05 from the get go.
For the current needs it is enough.
We can also say that it is the user responsibility to pass to failsafe the same names and same args as he passes for EAL(or default EAL names). I think I emphasized it in documentation.
Okay, as you wish. Just be aware of this limitation.
I think this functionality is good and useful, but it needs to be made clean. The proper function should be available soon, then this implementaion should be cleaned up.
Quoted 64 lines
Quoted 48 lines
Quoted 3 lines
Quoted 3 lines
<snip>
Quoted 1 line
Quoted 1 line
Review
Quoted 1 line
(quoted diff) +1
-0+ /* Take control of device probed by EALoptions. */
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Review
Quoted 1 line
(quoted diff) +1
-0+ DEBUG("Taking control of a probed subdevice"
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Review
Quoted 1 line
(quoted diff) +1
-0+ " %d named %s", i, da->name);In this case, the devargs of the probed device must be copied within the sub- device definition and removed from the EAL using the proper
rte_devargsAPI.Note that there is no
rte_devargscopy function. You can userte_devargs_parseinstead, "parsing" again the original devargs into the sub- device one. It is necessary for complying with internalrte_devargsrequirements (da->argsbeing malloc-ed, at the moment,Quoted 4 lines
Quoted 4 lines
but may evolve).
Quoted 3 lines
Quoted 3 lines
The
rte_eal_devargs_parsefunction is not easy enough to use right now, you will have to build a devargs string (using snprintf) andsubmit it.
Quoted 18 lines
Quoted 18 lines
Quoted 6 lines
Quoted 3 lines
I proposed a change this release for it but it will not make it for 18.02, that would have simplified your implementation.
Got you. You right we need to remove the created devargs in fail-safe
parse level.
Quoted 2 lines
What do you think about checking it in the parse level and avoid the new
devargs creation?
Quoted 2 lines
Also to do the copy in parse level(same method as we are doing in probe
level)?
Quoted 1 line
Not sure I follow here, but the new
rte_devargsis part of the sub-device (it is not a pointer, but allocated alongside thesub_device).Quoted 8 lines
Quoted 2 lines
So keep everything here, it is the right place to deal with these things.
But it will prevent the double parsing and also saves the method:
If the device already parsed - copy its devargs and continue. If the device already probed - copy the device pointer and continue.I think this is the right dealing, no?
Why to deal with parse level in probe level? Just keep all the parse work toQuoted 9 lines
parse level and the probe work to probe level.
After re-reading, I think we misunderstood each other.
You cannot remove therte_devargscreated during parsing: it is allocated alongside thesub_devicestructure.You must only remove the
rte_devargsallocated by the EAL (usingrte_eal_devargs_remove()).Sure.
Quoted 4 lines
Before removing it, you must copy its content in the local
sub_devicerte_devargsstructure. I only proposed a way to do this copy that would not deal withrte_devargsinternals, as it is bound to evolve rather soon.
Quoted 8 lines
Yes.
Quoted 2 lines
Otherwise, no, I do not want to complicate the parsing operations, they are already too complicated and too criticals. Better to keep it all here.
I think
fs_parse_devicefunction is not complicated and it is the natural place for devargs games. For me this is the right place for the copy & remove devargs.
Are you insisting to put all infs_bus_init?
You would have to put fs_ethdev_portid_find in failsafe_args, which is mixing layers. Sorry but yes, please keep all these changes in this file.
Thanks,
Signature · 2 lines
Gaëtan Rivet 6WIND
Re: [PATCH v3 3/8] net/failsafe: support probed sub-devices getting
Hi Gaetan
From: Gaëtan Rivet, Wednesday, January 17, 2018 12:31 AM
Quoted 66 lines
Hi Matan,
Quoted 2 lines · Matan Azrad
Hi Gaetan
<snip>
Quoted 12 lines
Quoted 12 lines
Quoted 3 lines
Quoted 3 lines
In 18.05, or 18.08 there should be an EAL function that would be able to identify a device given a specific ID string (very close to an
rte_devargs).Quoted 8 lines
Quoted 8 lines
Currently, this API does not exist.
You can hack your way around this for the moment, IF you really, really
want: parse your devargs, get the bus, use thebus->parse()function to get a binary device representation, and compare bytes per bytes the binary representation given by your devargs and by the device-Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
name.
Quoted 5 lines
But this is a hack, and a pretty ugly one at that: you have no way of knowing the size taken by this binary representation, so you can restrict yourself to the vdev and PCI bus for the moment and take the larger of an
rte_vdev_driverpointer and an
rte_pci_addr....Quoted 33 lines
Quoted 32 lines
Quoted 23 lines
Quoted 23 lines
{ union { rte_vdev_driver *drv; struct rte_pci_addr pci_addr; } bindev1, bindev2; memset(&bindev1, 0, sizeof(bindev1)); memset(&bindev2, 0, sizeof(bindev2)); rte_eal_devargs_parse(device->name, da1); rte_eal_devargs_parse(your_devstr, da2); RTE_ASSERT(da1->bus == rte_bus_find_by_name("pci") || da1->bus == rte_bus_find_by_name("vdev")); RTE_ASSERT(da2->bus == rte_bus_find_by_name("pci") || da2->bus == rte_bus_find_by_name("vdev")); da1->bus->parse(da1->name, &bindev1); da1->bus->parse(da2->name, &bindev2); if (memcmp(&bindev1, &bindev2, sizeof(bindev1)) == 0) { /* found the device */ } else { /* not found */ } }Quoted 5 lines
Quoted 2 lines
So, really, really ugly. Anyway.
Yes, ugly :) Thanks for this update!
Will keep the comparison bydevice->name.Well as explained, above, the comparison by
device->nameonly works with whitelisted devices.Quoted 5 lines
Quoted 3 lines
So either implement something broken right now that you will need to update in 18.05, or implement it properly in 18.05 from the get go.
For the current needs it is enough.
We can also say that it is the user responsibility to pass to failsafe the samenames and same args as he passes for EAL(or default EAL names).
Quoted 8 lines
Quoted 2 lines
I think I emphasized it in documentation.
Okay, as you wish. Just be aware of this limitation.
I think this functionality is good and useful, but it needs to be made clean. The proper function should be available soon, then this implementaion should be cleaned up.
Sure.
Quoted 75 lines
Quoted 33 lines
Quoted 33 lines
Quoted 4 lines
Quoted 4 lines
<snip>
Quoted 1 line
Quoted 1 line
Review
Quoted 1 line
(quoted diff) +1
-0+ /* Take control of device probed by EALoptions. */
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Review
Quoted 1 line
(quoted diff) +1
-0+ DEBUG("Taking control of a probed subdevice"
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Review
Quoted 1 line
(quoted diff) +1
-0+ " %d named %s", i, da->name);In this case, the devargs of the probed device must be copied within the sub- device definition and removed from the EAL using the proper
rte_devargsAPI.Note that there is no
rte_devargscopy function. You can userte_devargs_parseinstead, "parsing" again the original devargs into the sub- device one. It is necessary for complying with internalrte_devargsrequirements (da->argsbeing malloc-ed, at the moment,Quoted 5 lines
Quoted 5 lines
but may evolve).
Quoted 4 lines
Quoted 4 lines
The
rte_eal_devargs_parsefunction is not easy enough to use right now, you will have to build a devargs string (using snprintf) andsubmit it.
Quoted 9 lines
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
I proposed a change this release for it but it will not make it for 18.02, that would have simplified your implementation.
Got you. You right we need to remove the created devargs in fail-safe
parse level.
Quoted 2 lines
What do you think about checking it in the parse level and avoid the new
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
devargs creation?
Quoted 2 lines
Also to do the copy in parse level(same method as we are doing in probe
level)?
Quoted 1 line
Not sure I follow here, but the new
rte_devargsis part of the sub-device (it is not a pointer, but allocated alongside the
sub_device).Quoted 16 lines
Quoted 16 lines
Quoted 10 lines
Quoted 3 lines
So keep everything here, it is the right place to deal with these things.
But it will prevent the double parsing and also saves the method:
If the device already parsed - copy its devargs and continue. If the device already probed - copy the device pointer and continue.I think this is the right dealing, no?
Why to deal with parse level in probe level? Just keep all the parse work toparse level and the probe work to probe level.
After re-reading, I think we misunderstood each other. You cannot remove the
rte_devargscreated during parsing: it is allocated alongside thesub_devicestructure.Quoted 9 lines
Quoted 3 lines
You must only remove the
rte_devargsallocated by the EAL (usingrte_eal_devargs_remove()).Sure.
Quoted 3 lines
Before removing it, you must copy its content in the local
sub_devicerte_devargsstructure. I only proposed a way to do this copy that would not deal withrte_devargsinternals, as it is bound toevolve rather soon.
Quoted 5 lines
Quoted 1 line
Yes.
Quoted 2 lines
Otherwise, no, I do not want to complicate the parsing operations, they are already too complicated and too criticals. Better to keep it all
here.
Quoted 1 line
Quoted 8 lines
Quoted 1 line
I think
fs_parse_devicefunction is not complicated and it is the naturalplace for devargs games.
Quoted 2 lines
For me this is the right place for the copy & remove devargs. Are you insisting to put all in
fs_bus_init?You would have to put
fs_ethdev_portid_findinfailsafe_args, which is mixing layers. Sorry but yes, please keep all these changes in this file.
OK, Thanks man!
Quoted 4 lines
Thanks,
--
Gaëtan Rivet
6WIND
PATCH v3 4/8 net/vdev_netvsc: introduce Hyper-V platform driver
In reply to Matan AzradThis patch lays the groundwork for this driver (draft documentation, copyright notices, code base skeleton and build system hooks). While it can be successfully compiled and invoked, it's an empty shell at this stage.
--- MAINTAINERS | 6 ++ config/common_base | 5 ++ config/common_linuxapp | 1 + doc/guides/nics/features/vdev_netvsc.ini | 12 +++ doc/guides/nics/index.rst | 1 + doc/guides/nics/vdev_netvsc.rst | 20 +++++ drivers/net/Makefile | 1 + drivers/net/vdev_netvsc/Makefile | 27 ++++++ .../vdev_netvsc/rte_pmd_vdev_netvsc_version.map | 4 + drivers/net/vdev_netvsc/vdev_netvsc.c | 99 ++++++++++++++++++++++ mk/rte.app.mk | 1 + 11 files changed, 177 insertions(+) create mode 100644 doc/guides/nics/features/vdev_netvsc.ini create mode 100644 doc/guides/nics/vdev_netvsc.rst create mode 100644 drivers/net/vdev_netvsc/Makefile create mode 100644 drivers/net/vdev_netvsc/rte_pmd_vdev_netvsc_version.map create mode 100644 drivers/net/vdev_netvsc/vdev_netvsc.c
MAINTAINERS +6 -0
@@ -451,6 +451,12 @@ F: drivers/net/mrvl/ F: doc/guides/nics/mrvl.rst F: doc/guides/nics/features/mrvl.ini +Microsoft vdev-netvsc - EXPERIMENTAL +M: Matan Azrad <[email protected]> +F: drivers/net/vdev-netvsc/ +F: doc/guides/nics/vdev-netvsc.rst +F: doc/guides/nics/features/vdev-netvsc.ini + Netcope szedata2 M: Matej Vido <[email protected]> F: drivers/net/szedata2/
config/common_base +5 -0
@@ -281,6 +281,11 @@ CONFIG_RTE_LIBRTE_NFP_DEBUG=n CONFIG_RTE_LIBRTE_MRVL_PMD=n # +# Compile virtual device driver for NetVSC on Hyper-V/Azure +# +CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD=n + +# # Compile burst-oriented Broadcom BNXT PMD driver # CONFIG_RTE_LIBRTE_BNXT_PMD=y
config/common_linuxapp +1 -0
@@ -47,6 +47,7 @@ CONFIG_RTE_LIBRTE_PMD_VHOST=y CONFIG_RTE_LIBRTE_PMD_AF_PACKET=y CONFIG_RTE_LIBRTE_PMD_TAP=y CONFIG_RTE_LIBRTE_AVP_PMD=y +CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD=y CONFIG_RTE_LIBRTE_NFP_PMD=y CONFIG_RTE_LIBRTE_POWER=y CONFIG_RTE_VIRTIO_USER=y
doc/guides/nics/features/vdev_netvsc.ini new file +12 -0
@@ -0,0 +1,12 @@ +; +; Supported features of the 'vdev_netvsc' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +ARMv7 = Y +ARMv8 = Y +Power8 = Y +x86-32 = Y +x86-64 = Y +Usage doc = Y
doc/guides/nics/index.rst +1 -0
@@ -64,6 +64,7 @@ Network Interface Controller Drivers szedata2 tap thunderx + vdev_netvsc virtio vhost vmxnet3
doc/guides/nics/vdev_netvsc.rst new file +20 -0
@@ -0,0 +1,20 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright 2017 6WIND S.A. + Copyright 2017 Mellanox Technologies, Ltd. + +VDEV_NETVSC driver +================== + +The VDEV_NETVSC driver (librte_pmd_vdev_netvsc) provides support for NetVSC +interfaces and associated SR-IOV virtual function (VF) devices found in +Linux virtual machines running on Microsoft Hyper-V_ (including Azure) +platforms. + +.. _Hyper-V: https://docs.microsoft.com/en-us/windows-hardware/drivers/network/overview-of-hyper-v + +Build options +------------- + +- ``CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD`` (default ``y``) + + Toggle compilation of this driver.
drivers/net/Makefile +1 -0
@@ -66,6 +66,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += sfc DIRS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2) += szedata2 DIRS-$(CONFIG_RTE_LIBRTE_PMD_TAP) += tap DIRS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += thunderx +DIRS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += vdev_netvsc DIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio DIRS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += vmxnet3
drivers/net/vdev_netvsc/Makefile new file +27 -0
@@ -0,0 +1,27 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2017 6WIND S.A. +# Copyright 2017 Mellanox Technologies, Ltd. + +include $(RTE_SDK)/mk/rte.vars.mk + +# Properties of the generated library. +LIB = librte_pmd_vdev_netvsc.a +LIBABIVER := 1 +EXPORT_MAP := rte_pmd_vdev_netvsc_version.map + +# Additional compilation flags. +CFLAGS += -O3 +CFLAGS += -g +CFLAGS += -std=c11 -pedantic -Wall -Wextra +CFLAGS += $(WERROR_FLAGS) + +# Dependencies. +LDLIBS += -lrte_bus_vdev +LDLIBS += -lrte_eal +LDLIBS += -lrte_ethdev +LDLIBS += -lrte_kvargs + +# Source files. +SRCS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += vdev_netvsc.c + +include $(RTE_SDK)/mk/rte.lib.mk
drivers/net/vdev_netvsc/rte_pmd_vdev_netvsc_version.map new file +4 -0
@@ -0,0 +1,4 @@ +DPDK_18.02 { + + local: *; +};
drivers/net/vdev_netvsc/vdev_netvsc.c new file +99 -0
99 lines not on this page · raw message
mk/rte.app.mk +1 -0
@@ -156,6 +156,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += -lrte_pmd_sfc_efx _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2) += -lrte_pmd_szedata2 -lsze2 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_TAP) += -lrte_pmd_tap _LDLIBS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += -lrte_pmd_thunderx_nicvf +_LDLIBS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += -lrte_pmd_vdev_netvsc _LDLIBS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += -lrte_pmd_virtio ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y) _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += -lrte_pmd_vhost
Signature · 1 line
1.8.3.1
PATCH v3 5/8 net/vdev_netvsc: implement core functionality
In reply to Matan AzradAs described in more details in the attached documentation (see patch contents), this virtual device driver manages NetVSC interfaces in virtual machines hosted by Hyper-V/Azure platforms.
This driver does not manage traffic nor Ethernet devices directly; it acts as a thin configuration layer that automatically instantiates and controls fail-safe PMD instances combining tap and PCI sub-devices, so that each NetVSC interface is exposed as a single consolidated port to DPDK applications.
PCI sub-devices being hot-pluggable (e.g. during VM migration), applications automatically benefit from increased throughput when present and automatic fallback on NetVSC otherwise without interruption thanks to fail-safe's hot-plug handling.
Once initialized, the sole job of the vdev_netvsc driver is to regularly scan for PCI devices to associate with NetVSC interfaces and feed their addresses to corresponding fail-safe instances.
--- doc/guides/nics/vdev_netvsc.rst | 70 +++++ drivers/net/vdev_netvsc/Makefile | 4 + drivers/net/vdev_netvsc/vdev_netvsc.c | 544 +++++++++++++++++++++++++++++++++- 3 files changed, 617 insertions(+), 1 deletion(-)
doc/guides/nics/vdev_netvsc.rst +70 -0
@@ -12,9 +12,79 @@ platforms. .. _Hyper-V: https://docs.microsoft.com/en-us/windows-hardware/drivers/network/overview-of-hyper-v +Implementation details +---------------------- + +Each instance of this driver effectively needs to drive two devices: the +NetVSC interface proper and its SR-IOV VF (referred to as "physical" from +this point on) counterpart sharing the same MAC address. + +Physical devices are part of the host system and cannot be maintained during +VM migration. From a VM standpoint they appear as hot-plug devices that come +and go without prior notice. + +When the physical device is present, egress and most of the ingress traffic +flows through it; only multicasts and other hypervisor control still flow +through NetVSC. Otherwise, NetVSC acts as a fallback for all traffic. + +To avoid unnecessary code duplication and ensure maximum performance, +handling of physical devices is left to their original PMDs; this virtual +device driver (also known as *vdev*) manages other PMDs as summarized by the +following block diagram:: + + .------------------. + | DPDK application | + `--------+---------' + | + .------+------. + | DPDK ethdev | + `------+------' Control + | | + .------------+------------. v .--------------------. + | failsafe PMD +---------+ vdev_netvsc driver | + `--+-------------------+--' `--------------------' + | | + | .........|......... + | : | : + .----+----. : .----+----. : + | tap PMD | : | any PMD | : + `----+----' : `----+----' : <-- Hot-pluggable + | : | : + .------+-------. : .-----+-----. : + | NetVSC-based | : | SR-IOV VF | : + | netdevice | : | device | : + `--------------' : `-----------' : + :.................: + + +This driver implementation may be temporary and should be improved or removed +either when hot-plug will be fully supported in EAL and bus drivers or when +a new NetVSC driver will be integrated. + Build options ------------- - ``CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD`` (default ``y``) Toggle compilation of this driver. + +Run-time parameters +------------------- + +To invoke this driver, applications have to explicitly provide the +``--vdev=net_vdev_netvsc`` EAL option. + +The following device parameters are supported: + +- ``iface`` [string] + + Provide a specific NetVSC interface (netdevice) name to attach this driver + to. Can be provided multiple times for additional instances. + +- ``mac`` [string] + + Same as ``iface`` except a suitable NetVSC interface is located using its + MAC address. + +Not specifying either ``iface`` or ``mac`` makes this driver attach itself to +all NetVSC interfaces found on the system.
drivers/net/vdev_netvsc/Makefile +4 -0
@@ -13,6 +13,9 @@ EXPORT_MAP := rte_pmd_vdev_netvsc_version.map CFLAGS += -O3 CFLAGS += -g CFLAGS += -std=c11 -pedantic -Wall -Wextra +CFLAGS += -D_XOPEN_SOURCE=600 +CFLAGS += -D_BSD_SOURCE +CFLAGS += -D_DEFAULT_SOURCE CFLAGS += $(WERROR_FLAGS) # Dependencies.
@@ -20,6 +23,7 @@ LDLIBS += -lrte_bus_vdev LDLIBS += -lrte_eal LDLIBS += -lrte_ethdev LDLIBS += -lrte_kvargs +LDLIBS += -lrte_net # Source files. SRCS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += vdev_netvsc.c
drivers/net/vdev_netvsc/vdev_netvsc.c +543 -1
597 lines not on this page · raw message
Signature · 1 line
1.8.3.1
Re: [PATCH v3 5/8] net/vdev_netvsc: implement core functionality
Quoted 15 lines · On Tue, 9 Jan 2018 14:47:30 +0000 Matan Azrad <[email protected]> wrote:
As described in more details in the attached documentation (see patch contents), this virtual device driver manages NetVSC interfaces in virtual machines hosted by Hyper-V/Azure platforms.
This driver does not manage traffic nor Ethernet devices directly; it acts as a thin configuration layer that automatically instantiates and controls fail-safe PMD instances combining tap and PCI sub-devices, so that each NetVSC interface is exposed as a single consolidated port to DPDK applications.
PCI sub-devices being hot-pluggable (e.g. during VM migration), applications automatically benefit from increased throughput when present and automatic fallback on NetVSC otherwise without interruption thanks to fail-safe's hot-plug handling.
Quoted 6 lines
Once initialized, the sole job of the
vdev_netvscdriver is to regularly scan for PCI devices to associate with NetVSC interfaces and feed their addresses to corresponding fail-safe instances.Signed-off-by Adrien Mazarguil <[email protected]>Signed-off-by Matan Azrad <[email protected]>
There is also the issue of how rescind is handled, but that may be more complex than you want to deal with now. Host may rescind PCI devices for other reasons than migration. For example, if host needs to do live upgrade of PF device driver on host (or firmware); then it will rescind VF device from all guests and then restore it after upgrade.
drivers/net/vdev_netvsc/Makefile EXPORT_MAP := rte_pmd_vdev_netvsc_version.map · line 18
Quoted 11 lines
drivers/net/vdev_netvsc/Makefile +3
-0@@ -13,6 +13,9 @@ EXPORT_MAP := rte_pmd_vdev_netvsc_version.map CFLAGS += -O3 CFLAGS += -g CFLAGS += -std=c11 -pedantic -Wall -Wextra +CFLAGS += -D_XOPEN_SOURCE=600 +CFLAGS += -D_BSD_SOURCE +CFLAGS += -D_DEFAULT_SOURCE
These are kind of a nuisance, can't it just use same CFLAGS as other code?
Quoted 26 lines
drivers/net/vdev_netvsc/Makefile +0
-0# Source files. SRCS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += vdev_netvsc.cdrivers/net/vdev_netvsc/vdev_netvsc.c +6
-0@@ -3,17 +3,41 @@ * Copyright 2017 Mellanox Technologies, Ltd.#define VDEV_NETVSC_DRIVER net_vdev_netvsc#define VDEV_NETVSC_ARG_IFACE "iface"#define VDEV_NETVSC_ARG_MAC "mac" +#define VDEV_NETVSC_PROBE_MS 1000 + +#define NETVSC_CLASS_ID "{f8615163-df3e-46c5-913f-f2d2f965ed0e}"#define DRV_LOG (level, ...) \ rte_log(RTE_LOG_ ## level, \@@ -25,12 +49,490 @@/** Driver-specific log messages type. */ static int vdev_netvsc_logtype; +/** Context structure for a vdev_netvsc instance. */ +struct vdev_netvsc_ctx { +LIST_ENTRY (vdev_netvsc_ctx) entry;/**< Next entry in list. */
drivers/net/vdev_netvsc/vdev_netvsc.c line 64
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+unsigned int id;/**< ID used to generate unique names. */ +char name[64 ];/**< Unique name for vdev_netvsc instance. */ +char devname[64 ];/**< Fail-safe PMD instance name. */ +char devargs[256 ];/**< Fail-safe PMD instance device arguments. */ +char if_name[IF_NAMESIZE ];/**< NetVSC netdevice name. */ +unsigned int if_index;/**< NetVSC netdevice index. */ +struct ether_addr if_addr;/**< NetVSC MAC address. */ +int pipe[2 ];/**< Communication pipe with fail-safe instance. */ +char yield[256 ];/**< Current device string used with fail-safe. */ +};
Please align comments.
Quoted 65 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +61
-0+/** Context list is common to all driver instances. */ +static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list = +LIST_HEAD_INITIALIZER (vdev_netvsc_ctx_list); + +/** Number of entries in context list. */ +static unsigned int vdev_netvsc_ctx_count; +/** Number of driver instances relying on context list. */ static unsigned int vdev_netvsc_ctx_inst;/** +* Destroy a vdev_netvsc context instance. +* +* @param ctx +* Context to destroy. +*/ +static void +vdev_netvsc_ctx_destroy (struct vdev_netvsc_ctx *ctx) +{ +if (ctx->pipe[0 ] !=-1 ) +close (ctx->pipe[0 ]); +if (ctx->pipe[1 ] !=-1 ) +close (ctx->pipe[1 ]); +free (ctx); +} + +/** +* Iterate over system network interfaces. +* +* This function runs a given callback function for each netdevice found on +* the system. +* +* @param func +* Callback function pointer. List traversal is aborted when this function +* returns a nonzero value. +* @param ... +* Variable parameter list passed as @p va_list to @p func. +* +* @return +* 0 when the entire list is traversed successfully, a negative error code +* in case or failure, or the nonzero value returned by @p func when list +* traversal is aborted. +*/ +static int +vdev_netvsc_foreach_iface (int (*func)(const struct if_nameindex *iface, +const struct ether_addr *eth_addr, +va_list ap), ...) +{ +struct if_nameindex *iface =if_nameindex (); +int s =socket (PF_INET ,SOCK_DGRAM ,IPPROTO_IP ); +unsigned int i; +int ret =0 ; + +if (!iface) { + ret = -ENOBUFS ; +DRV_LOG (ERR ,"cannot retrieve system network interfaces" ); + goto error; + } +if (s ==-1 ) { + ret = -errno; +DRV_LOG (ERR ,"cannot open socket: %s" ,rte_strerror (errno)); + goto error; + } +for (i =0 ; iface[i].if_name; ++i) { +struct ifreq req;
drivers/net/vdev_netvsc/vdev_netvsc.c line 140
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+struct ether_addr eth_addr; +va_list ap; + +strncpy (req.ifr_name, iface[i].if_name,sizeof (req.ifr_name)); +if (ioctl (s,SIOCGIFHWADDR , &req) ==-1 ) { +DRV_LOG (WARNING ,"cannot retrieve information about" +" interface \"%s\": %s" , + req.ifr_name,rte_strerror (errno)); +continue ; + }
Skip non-ethernet interfaces where addr length != 6
Quoted 36 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +36
-0+memcpy (eth_addr.addr_bytes, req.ifr_hwaddr.sa_data, +RTE_DIM (eth_addr.addr_bytes)); +va_start (ap, func); + ret =func (&iface[i], ð_addr, ap); +va_end (ap); +if (ret) +break ; + } +error: +if (s !=-1 ) +close (s); +if (iface) +if_freenameindex (iface); +return ret; +} + +/** +* Determine if a network interface is NetVSC. +* +* @param[in] iface +* Pointer to netdevice description structure (name and index). +* +* @return +* A nonzero value when interface is detected as NetVSC. In case of error, +* rte_errno is updated and 0 returned. +*/ +static int +vdev_netvsc_iface_is_netvsc (const struct if_nameindex *iface) +{ +static const char temp[] ="/sys/class/net/%s/device/class_id" ; +char path[sizeof (temp) +IF_NAMESIZE ]; +FILE *f; +int ret; +int len =0 ; + + ret =snprintf (path,sizeof (path), temp, iface->if_name);
drivers/net/vdev_netvsc/vdev_netvsc.c line 186
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+if (ret ==-1 || (size_t )ret >=sizeof (path)) { + rte_errno =ENOBUFS ; +return 0 ; + } + f =fopen (path,"r" ); +if (!f) { + rte_errno = errno; +return 0 ; + } + ret =fscanf (f,NETVSC_CLASS_ID "%n" , &len);
This is different way to compare uuid, maybe use fgets() and uuid_compare?
Quoted 29 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +29
-0+if (ret ==EOF ) + rte_errno = errno; + ret = len == (int )strlen (NETVSC_CLASS_ID ); +fclose (f); +return ret; +} + +/** +* Retrieve network interface data from sysfs symbolic link. +* +* @param[out] buf +* Output data buffer. +* @param size +* Output buffer size. +* @param[in] if_name +* Netdevice name. +* @param[in] relpath +* Symbolic link path relative to netdevice sysfs entry. +* +* @return +* 0 on success, a negative error code otherwise. +*/ +static int +vdev_netvsc_sysfs_readlink (char *buf,size_t size,const char *if_name, +const char *relpath) +{ +int ret; + + ret =snprintf (buf, size,"/sys/class/net/%s/%s" , if_name, relpath);
drivers/net/vdev_netvsc/vdev_netvsc.c line 225
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+if (ret ==-1 || (size_t )ret >= size) +return -ENOBUFS ; + ret =readlink (buf, buf, size); +if (ret ==-1 ) +return -errno; +if ((size_t )ret >= size -1 ) +return -ENOBUFS ; + buf[ret] ='\0' ; +return 0 ; +}
You might find it easier to look at directory.
/sys/bus/vmbus/drivers/hv_netvsc/
Quoted 118 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +118
-0+ +/** +* Probe a network interface to associate with vdev_netvsc context. +* +* This function determines if the network device matches the properties of +* the NetVSC interface associated with the vdev_netvsc context and +* communicates its bus address to the fail-safe PMD instance if so. +* +* It is normally used with vdev_netvsc_foreach_iface(). +* +* @param[in] iface +* Pointer to netdevice description structure (name and index). +* @param[in] eth_addr +* MAC address associated with @p iface. +* @param ap +* Variable arguments list comprising: +* +* - struct vdev_netvsc_ctx *ctx: +* Context to associate network interface with. +* +* @return +* A nonzero value when interface matches, 0 otherwise or in case of +* error. +*/ +static int +vdev_netvsc_device_probe (const struct if_nameindex *iface, +const struct ether_addr *eth_addr, +va_list ap) +{ + struct vdev_netvsc_ctx *ctx = va_arg(ap, struct vdev_netvsc_ctx *); +char buf[RTE_MAX (sizeof (ctx->yield),256u )]; +const char *addr; +size_t len; +int ret; + +/* Skip non-matching or unwanted NetVSC interfaces. */ +if (ctx->if_index == iface->if_index) { +if (!strcmp (ctx->if_name, iface->if_name)) +return 0 ; +DRV_LOG (DEBUG , +"NetVSC interface \"%s\" (index %u) renamed \"%s\"" , + ctx->if_name, ctx->if_index, iface->if_name); +strncpy (ctx->if_name, iface->if_name,sizeof (ctx->if_name)); +return 0 ; + } +if (vdev_netvsc_iface_is_netvsc (iface)) +return 0 ; +if (!is_same_ether_addr (eth_addr, &ctx->if_addr)) +return 0 ; +/* Look for associated PCI device. */ + ret =vdev_netvsc_sysfs_readlink (buf,sizeof (buf), iface->if_name, +"device/subsystem" ); +if (ret) +return 0 ; + addr =strrchr (buf,'/' ); + addr = addr ? addr +1 : buf; +if (strcmp (addr,"pci" )) +return 0 ; + ret =vdev_netvsc_sysfs_readlink (buf,sizeof (buf), iface->if_name, +"device" ); +if (ret) +return 0 ; + addr =strrchr (buf,'/' ); + addr = addr ? addr +1 : buf; + len =strlen (addr); +if (!len) +return 0 ; +/* Send PCI device argument to fail-safe PMD instance. */ +if (strcmp (addr, ctx->yield)) +DRV_LOG (DEBUG ,"associating PCI device \"%s\" with NetVSC" +" interface \"%s\" (index %u)" , addr, ctx->if_name, + ctx->if_index); +memmove (buf, addr, len +1 ); + addr = buf; + buf[len] ='\n' ; + ret =write (ctx->pipe[1 ], addr, len +1 ); + buf[len] ='\0' ; +if (ret ==-1 ) { +if (errno ==EINTR || errno ==EAGAIN ) +return 1 ; +DRV_LOG (WARNING ,"cannot associate PCI device name \"%s\" with" +" interface \"%s\": %s" , addr, ctx->if_name, +rte_strerror (errno)); +return 1 ; + } +if ((size_t )ret != len +1 ) { +/* +* Attempt to override previous partial write, no need to +* recover if that fails. +*/ + ret =write (ctx->pipe[1 ],"\n" ,1 ); + (void )ret; +return 1 ; + } +fsync (ctx->pipe[1 ]); +memcpy (ctx->yield, addr, len +1 ); +return 1 ; +} + +/** +* Alarm callback that regularly probes system network interfaces. +* +* This callback runs at a frequency determined by VDEV_NETVSC_PROBE_MS as +* long as an vdev_netvsc context instance exists. +* +* @param arg +* Ignored. +*/ +static void +vdev_netvsc_alarm(__rte_unused void *arg) +{ +struct vdev_netvsc_ctx *ctx; +int ret; + + LIST_FOREACH(ctx, &vdev_netvsc_ctx_list, entry) { + ret =vdev_netvsc_foreach_iface (vdev_netvsc_device_probe, ctx); +if (ret) +break ;
drivers/net/vdev_netvsc/vdev_netvsc.c line 353
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+ } +if (!vdev_netvsc_ctx_count) +return ; + ret =rte_eal_alarm_set (VDEV_NETVSC_PROBE_MS *1000 , + vdev_netvsc_alarm,NULL ); +if (ret <0 ) { +DRV_LOG (ERR ,"unable to reschedule alarm callback: %s" , +rte_strerror (-ret)); + } +}
Why not use netlink uevent?
Quoted 253 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +220
-1+/** +* Probe a NetVSC interface to generate a vdev_netvsc context from. +* +* This function instantiates vdev_netvsc contexts either for all NetVSC +* devices found on the system or only a subset provided as device +* arguments. +* +* It is normally used with vdev_netvsc_foreach_iface(). +* +* @param[in] iface +* Pointer to netdevice description structure (name and index). +* @param[in] eth_addr +* MAC address associated with @p iface. +* @param ap +* Variable arguments list comprising: +* +* - const char *name: +* Name associated with current driver instance. +* +* - struct rte_kvargs *kvargs: +* Device arguments provided to current driver instance. +* +* - unsigned int specified: +* Number of specific netdevices provided as device arguments. +* +* - unsigned int *matched: +* The number of specified netdevices matched by this function. +* +* @return +* A nonzero value when interface matches, 0 otherwise or in case of +* error. +*/ +static int +vdev_netvsc_netvsc_probe (const struct if_nameindex *iface, +const struct ether_addr *eth_addr, +va_list ap) +{ + const char *name = va_arg(ap, const char *); + struct rte_kvargs *kvargs = va_arg(ap, struct rte_kvargs *); + unsigned int specified = va_arg(ap, unsigned int); + unsigned int *matched = va_arg(ap, unsigned int *); +unsigned int i; +struct vdev_netvsc_ctx *ctx; +int ret; + +/* Probe all interfaces when none are specified. */ +if (specified) { +for (i =0 ; i != kvargs->count; ++i) { +const struct rte_kvargs_pair *pair = &kvargs->pairs[i]; + +if (!strcmp (pair->key,VDEV_NETVSC_ARG_IFACE )) { +if (!strcmp (pair->value, iface->if_name)) +break ; + }else if (!strcmp (pair->key,VDEV_NETVSC_ARG_MAC )) { +struct ether_addr tmp; + +if (sscanf (pair->value, +"%" SCNx8":%" SCNx8":%" SCNx8":" +"%" SCNx8":%" SCNx8":%" SCNx8, + &tmp.addr_bytes[0 ], + &tmp.addr_bytes[1 ], + &tmp.addr_bytes[2 ], + &tmp.addr_bytes[3 ], + &tmp.addr_bytes[4 ], + &tmp.addr_bytes[5 ]) !=6 ) { +DRV_LOG (ERR , +"invalid MAC address format" +" \"%s\"" , + pair->value); +return -EINVAL ; + } +if (is_same_ether_addr (eth_addr, &tmp)) +break ; + } + } +if (i == kvargs->count) +return 0 ; + ++(*matched); + } +/* Weed out interfaces already handled. */ + LIST_FOREACH(ctx, &vdev_netvsc_ctx_list, entry) +if (ctx->if_index == iface->if_index) +break ; +if (ctx) { +if (!specified) +return 0 ; +DRV_LOG (WARNING , +"interface \"%s\" (index %u) is already handled," +" skipping" , + iface->if_name, iface->if_index); +return 0 ; + } +if (!vdev_netvsc_iface_is_netvsc (iface)) { +if (!specified) +return 0 ; +DRV_LOG (WARNING , +"interface \"%s\" (index %u) is not NetVSC," +" skipping" , + iface->if_name, iface->if_index); +return 0 ; + } +/* Create interface context. */ + ctx =calloc (1 ,sizeof (*ctx)); +if (!ctx) { + ret = -errno; +DRV_LOG (ERR ,"cannot allocate context for interface \"%s\": %s" , + iface->if_name,rte_strerror (errno)); + goto error; + } + ctx->id = vdev_netvsc_ctx_count; +strncpy (ctx->if_name, iface->if_name,sizeof (ctx->if_name)); + ctx->if_index = iface->if_index; + ctx->if_addr = *eth_addr; + ctx->pipe[0 ] =-1 ; + ctx->pipe[1 ] =-1 ; + ctx->yield[0 ] ='\0' ; +if (pipe (ctx->pipe) ==-1 ) { + ret = -errno; +DRV_LOG (ERR , +"cannot allocate control pipe for interface \"%s\": %s" , + ctx->if_name,rte_strerror (errno)); + goto error; + } +for (i =0 ; i !=RTE_DIM (ctx->pipe); ++i) { +int flf =fcntl (ctx->pipe[i],F_GETFL ); + +if (flf !=-1 && +fcntl (ctx->pipe[i],F_SETFL , flf |O_NONBLOCK ) !=-1 ) +continue ; + ret = -errno; +DRV_LOG (ERR ,"cannot toggle non-blocking flag on control file" +" descriptor #%u (%d): %s" , i, ctx->pipe[i], +rte_strerror (errno)); + goto error; + } +/* Generate virtual device name and arguments. */ + i =0 ; + ret =snprintf (ctx->name,sizeof (ctx->name),"%s_id%u" , + name, ctx->id); +if (ret ==-1 || (size_t )ret >=sizeof (ctx->name)) + ++i; + ret =snprintf (ctx->devname,sizeof (ctx->devname),"net_failsafe_%s" , + ctx->name); +if (ret ==-1 || (size_t )ret >=sizeof (ctx->devname)) + ++i; + ret =snprintf (ctx->devargs,sizeof (ctx->devargs), +"fd(%d),dev(net_tap_%s,remote=%s)" , + ctx->pipe[0 ], ctx->name, ctx->if_name); +if (ret ==-1 || (size_t )ret >=sizeof (ctx->devargs)) + ++i; +if (i) { + ret = -ENOBUFS ; +DRV_LOG (ERR ,"generated virtual device name or argument list" +" too long for interface \"%s\"" , ctx->if_name); + goto error; + } +/* Request virtual device generation. */ +DRV_LOG (DEBUG ,"generating virtual device \"%s\" with arguments \"%s\"" , + ctx->devname, ctx->devargs); +vdev_netvsc_foreach_iface (vdev_netvsc_device_probe, ctx); + ret =rte_eal_hotplug_add ("vdev" , ctx->devname, ctx->devargs); +if (ret) + goto error; +LIST_INSERT_HEAD (&vdev_netvsc_ctx_list, ctx, entry); + ++vdev_netvsc_ctx_count; +DRV_LOG (DEBUG ,"added NetVSC interface \"%s\" to context list" , + ctx->if_name); +return 0 ; +error: +if (ctx) +vdev_netvsc_ctx_destroy (ctx); +return ret; +} + +/** * Probe NetVSC interfaces. * +* This function probes system netdevices according to the specified device +* arguments and starts a periodic alarm callback to notify the resulting +* fail-safe PMD instances of their sub-devices whereabouts. +* * @param dev * Virtual device context for driver instance. * @@ -49,12 +551,40 @@const char *args =rte_vdev_device_args (dev);struct rte_kvargs *kvargs =rte_kvargs_parse (args ? args :"" , vdev_netvsc_arg); +unsigned int specified =0 ; +unsigned int matched =0 ; +unsigned int i; +int ret;DRV_LOG (DEBUG ,"invoked as \"%s\", using arguments \"%s\"" , name, args);if (!kvargs) {DRV_LOG (ERR ,"cannot parse arguments list" ); goto error; } +for (i =0 ; i != kvargs->count; ++i) { +const struct rte_kvargs_pair *pair = &kvargs->pairs[i]; + +if (!strcmp (pair->key,VDEV_NETVSC_ARG_IFACE ) || + !strcmp (pair->key,VDEV_NETVSC_ARG_MAC )) + ++specified; + } +rte_eal_alarm_cancel (vdev_netvsc_alarm,NULL ); +/* Gather interfaces. */ + ret =vdev_netvsc_foreach_iface (vdev_netvsc_netvsc_probe, name, kvargs, + specified, &matched); +if (ret <0 ) + goto error; +if (matched < specified) +DRV_LOG (WARNING , +"some of the specified parameters did not match" +" recognized network interfaces" ); + ret =rte_eal_alarm_set (VDEV_NETVSC_PROBE_MS *1000 , + vdev_netvsc_alarm,NULL ); +if (ret <0 ) { +DRV_LOG (ERR ,"unable to schedule alarm callback: %s" , +rte_strerror (-ret)); + goto error; + } error:if (kvargs)rte_kvargs_free (kvargs);@@ -65,6 +595,9 @@/** * Remove driver instance. * +* The alarm callback and underlying vdev_netvsc context instances are only +* destroyed after the last PMD instance is removed. +* * @param dev * Virtual device context for driver instance. * @@ -74,7 +607,16 @@static int vdev_netvsc_vdev_remove(__rte_unused struct rte_vdev_device *dev) { - --vdev_netvsc_ctx_inst; +if (--vdev_netvsc_ctx_inst) +return 0 ; +rte_eal_alarm_cancel (vdev_netvsc_alarm,NULL ); +while (!LIST_EMPTY (&vdev_netvsc_ctx_list)) { +struct vdev_netvsc_ctx *ctx =LIST_FIRST (&vdev_netvsc_ctx_list); + +LIST_REMOVE (ctx, entry); + --vdev_netvsc_ctx_count; +vdev_netvsc_ctx_destroy (ctx); + }return 0 ; }
Re: [PATCH v3 5/8] net/vdev_netvsc: implement core functionality
Hi Stephan
Thank you for this quick review, please see some comments.
From: Stephen Hemminger, Tuesday, January 9, 2018 8:49 PM
Quoted 32 lines
Quoted 15 lines · On Tue, 9 Jan 2018 14:47:30 +0000 Matan Azrad <[email protected]> wrote:
As described in more details in the attached documentation (see patch contents), this virtual device driver manages NetVSC interfaces in virtual machines hosted by Hyper-V/Azure platforms.
This driver does not manage traffic nor Ethernet devices directly; it acts as a thin configuration layer that automatically instantiates and controls fail-safe PMD instances combining tap and PCI sub-devices, so that each NetVSC interface is exposed as a single consolidated port to DPDK applications.
PCI sub-devices being hot-pluggable (e.g. during VM migration), applications automatically benefit from increased throughput when present and automatic fallback on NetVSC otherwise without interruption thanks to fail-safe's hot-plug handling.
Quoted 6 lines
Once initialized, the sole job of the
vdev_netvscdriver is to regularly scan for PCI devices to associate with NetVSC interfaces and feed their addresses to corresponding fail-safe instances.Signed-off-by Adrien Mazarguil <[email protected]>Signed-off-by Matan Azrad <[email protected]>There is also the issue of how rescind is handled, but that may be more complex than you want to deal with now. Host may rescind PCI devices for other reasons than migration. For example, if host needs to do live upgrade of PF device driver on host (or firmware); then it will rescind VF device from all guests and then restore it after upgrade.
Quoted 15 lines
drivers/net/vdev_netvsc/Makefile EXPORT_MAP := rte_pmd_vdev_netvsc_version.map · line 18
Quoted 11 lines
drivers/net/vdev_netvsc/Makefile +0
-0@@ -13,6 +13,9 @@ EXPORT_MAP := rte_pmd_vdev_netvsc_version.mapCFLAGS += -O3 CFLAGS += -g CFLAGS += -std=c11 -pedantic -Wall
drivers/net/vdev_netvsc/Makefile +3
-1-Wextra +CFLAGS += -D_XOPEN_SOURCE=600 +CFLAGS += -D_BSD_SOURCE +CFLAGS += -D_DEFAULT_SOURCEThese are kind of a nuisance, can't it just use same CFLAGS as other code?
Will check.
Quoted 25 lines
Quoted 2 lines
# Source files. SRCS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) +=
vdev_netvsc.cdiff--git
Quoted 22 lines
a/drivers/net/vdev_netvsc/vdev_netvsc.cb/drivers/net/vdev_netvsc/vdev_netvsc.c
index e895b32..3d8895b 100644drivers/net/vdev_netvsc/vdev_netvsc.c +0
-0@@ -3,17 +3,41 @@ * Copyright 2017 Mellanox Technologies, Ltd.#define VDEV_NETVSC_DRIVER net_vdev_netvsc #define
VDEV_NETVSC_ARG_IFACE"iface"
#defineVDEV_NETVSC_ARG_MAC"mac"drivers/net/vdev_netvsc/vdev_netvsc.c +3
-0+#define VDEV_NETVSC_PROBE_MS 1000 + +#define NETVSC_CLASS_ID "{f8615163-df3e-46c5-913f-f2d2f965ed0e}"#define DRV_LOG (level, ...) \ rte_log(RTE_LOG_ ## level, \@@ -25,12 +49,490 @@/** Driver-specific log messages type. */ static int vdev_netvsc_logtype;
Quoted 16 lines
drivers/net/vdev_netvsc/vdev_netvsc.c line 64
Quoted 13 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +13
-0+/** Context structure for a vdev_netvsc instance. */ struct +vdev_netvsc_ctx { +LIST_ENTRY (vdev_netvsc_ctx) entry;/**< Next entry in list. */ +unsigned int id;/**< ID used to generate unique names. */ +char name[64 ];/**< Unique name for vdev_netvsc instance. */ +char devname[64 ];/**< Fail-safe PMD instance name. */ +char devargs[256 ];/**< Fail-safe PMD instance device arguments. */ +char if_name[IF_NAMESIZE ];/**< NetVSC netdevice name. */ +unsigned int if_index;/**< NetVSC netdevice index. */ +struct ether_addr if_addr;/**< NetVSC MAC address. */ +int pipe[2 ];/**< Communication pipe with fail-safe instance. */ +char yield[256 ];/**< Current device string used with fail-safe. */ +};Please align comments.
Sure.
Quoted 75 lines
Quoted 46 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +42
-0+/** Context list is common to all driver instances. */ static +LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list = +LIST_HEAD_INITIALIZER (vdev_netvsc_ctx_list); + +/** Number of entries in context list. */ static unsigned int +vdev_netvsc_ctx_count; +/** Number of driver instances relying on context list. */ static unsigned int vdev_netvsc_ctx_inst;/** +* Destroy a vdev_netvsc context instance. +* +* @param ctx +* Context to destroy. +*/ +static void +vdev_netvsc_ctx_destroy (struct vdev_netvsc_ctx *ctx) { +if (ctx->pipe[0 ] !=-1 ) +close (ctx->pipe[0 ]); +if (ctx->pipe[1 ] !=-1 ) +close (ctx->pipe[1 ]); +free (ctx); +} + +/** +* Iterate over system network interfaces. +* +* This function runs a given callback function for each netdevice +found on +* the system. +* +* @param func +* Callback function pointer. List traversal is aborted when this function +* returns a nonzero value. +* @param ... +* Variable parameter list passed as @p va_list to @p func. +* +* @return +* 0 when the entire list is traversed successfully, a negative error code +* in case or failure, or the nonzero value returned by @p func when list +* traversal is aborted. +*/ +static int +vdev_netvsc_foreach_iface(int (*func)(const struct if_nameindex *iface, + const struct ether_addr *eth_addr,drivers/net/vdev_netvsc/vdev_netvsc.c
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+ va_list ap), ...) +{ +struct if_nameindex *iface =if_nameindex (); +int s =socket (PF_INET ,SOCK_DGRAM ,IPPROTO_IP ); +unsigned int i; +int ret =0 ; + +if (!iface) { + ret = -ENOBUFS ; + DRV_LOG(ERR, "cannot retrieve system networkinterfaces");
drivers/net/vdev_netvsc/vdev_netvsc.c
Quoted 5 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +5
-0+ goto error; + } +if (s ==-1 ) { + ret = -errno; + DRV_LOG(ERR, "cannot open socket: %s",
rte_strerror(errno));drivers/net/vdev_netvsc/vdev_netvsc.c
Quoted 8 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +8
-0+ goto error; + } +for (i =0 ; iface[i].if_name; ++i) { +struct ifreq req; +struct ether_addr eth_addr; +va_list ap; + + strncpy(req.ifr_name, iface[i].if_name,
sizeof(req.ifr_name));drivers/net/vdev_netvsc/vdev_netvsc.c line 135
Quoted 2 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +2
-0+if (ioctl (s,SIOCGIFHWADDR , &req) ==-1 ) { + DRV_LOG(WARNING, "cannot retrieve informationabout"
Quoted 7 lines
drivers/net/vdev_netvsc/vdev_netvsc.c line 140
Quoted 4 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +4
-0+" interface \"%s\": %s" , + req.ifr_name, rte_strerror(errno)); +continue ; + }Skip non-ethernet interfaces where addr length != 6
Will check.
Quoted 39 lines
Quoted 14 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +14
-0+memcpy (eth_addr.addr_bytes, req.ifr_hwaddr.sa_data, +RTE_DIM (eth_addr.addr_bytes)); +va_start (ap, func); + ret =func (&iface[i], ð_addr, ap); +va_end (ap); +if (ret) +break ; + } +error: +if (s !=-1 ) +close (s); +if (iface) +if_freenameindex (iface); +return ret;drivers/net/vdev_netvsc/vdev_netvsc.c
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+} + +/** +* Determine if a network interface is NetVSC. +* +* @param[in] iface +* Pointer to netdevice description structure (name and index). +* +* @return +* A nonzero value when interface is detected as NetVSC. In case of error,
Quoted 14 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +14
-0+* rte_errno is updated and 0 returned. +*/ +static int +vdev_netvsc_iface_is_netvsc (const struct if_nameindex *iface) { +static const char temp[] ="/sys/class/net/%s/device/class_id" ; +char path[sizeof (temp) +IF_NAMESIZE ]; +FILE *f; +int ret; +int len =0 ; + + ret =snprintf (path,sizeof (path), temp, iface->if_name); +if (ret ==-1 || (size_t )ret >=sizeof (path)) { + rte_errno =ENOBUFS ; +return 0 ;
Quoted 9 lines
drivers/net/vdev_netvsc/vdev_netvsc.c line 186
Quoted 7 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +7
-0+ } + f =fopen (path,"r" ); +if (!f) { + rte_errno = errno; +return 0 ; + } + ret =fscanf (f,NETVSC_CLASS_ID "%n" , &len);This is different way to compare uuid, maybe use
fgets()anduuid_compare?
Different and nice.
I don't see a reason to replace it.
Quoted 34 lines
Quoted 34 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +34
-0+if (ret ==EOF ) + rte_errno = errno; + ret = len == (int )strlen (NETVSC_CLASS_ID ); +fclose (f); +return ret; +} + +/** +* Retrieve network interface data from sysfs symbolic link. +* +* @param[out] buf +* Output data buffer. +* @param size +* Output buffer size. +* @param[in] if_name +* Netdevice name. +* @param[in] relpath +* Symbolic link path relative to netdevice sysfs entry. +* +* @return +* 0 on success, a negative error code otherwise. +*/ +static int +vdev_netvsc_sysfs_readlink (char *buf,size_t size,const char *if_name, +const char *relpath) +{ +int ret; + + ret =snprintf (buf, size,"/sys/class/net/%s/%s" , if_name, relpath); +if (ret ==-1 || (size_t )ret >= size) +return -ENOBUFS ; + ret =readlink (buf, buf, size); +if (ret ==-1 ) +return -errno;
Quoted 9 lines
drivers/net/vdev_netvsc/vdev_netvsc.c line 225
Quoted 5 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +5
-0+if ((size_t )ret >= size -1 ) +return -ENOBUFS ; + buf[ret] ='\0' ; +return 0 ; +}You might find it easier to look at directory.
/sys/bus/vmbus/drivers/hv_netvsc/
This driver allows to run regular netdevice instead of NetVSC(as described in doc) for debug purpose(even in non-HyperV-VM machine ), So, It doesn't make sense.
Quoted 122 lines
Quoted 32 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +32
-0+ +/** +* Probe a network interface to associate with vdev_netvsc context. +* +* This function determines if the network device matches the +properties of +* the NetVSC interface associated with the vdev_netvsc context and +* communicates its bus address to the fail-safe PMD instance if so. +* +* It is normally used with vdev_netvsc_foreach_iface(). +* +* @param[in] iface +* Pointer to netdevice description structure (name and index). +* @param[in] eth_addr +* MAC address associated with @p iface. +* @param ap +* Variable arguments list comprising: +* +* - struct vdev_netvsc_ctx *ctx: +* Context to associate network interface with. +* +* @return +* A nonzero value when interface matches, 0 otherwise or in case of +* error. +*/ +static int +vdev_netvsc_device_probe (const struct if_nameindex *iface, +const struct ether_addr *eth_addr, +va_list ap) +{ + struct vdev_netvsc_ctx *ctx = va_arg(ap, struct vdev_netvsc_ctx *); +char buf[RTE_MAX (sizeof (ctx->yield),256u )];drivers/net/vdev_netvsc/vdev_netvsc.c
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+const char *addr; +size_t len; +int ret; + +/* Skip non-matching or unwanted NetVSC interfaces. */ +if (ctx->if_index == iface->if_index) { +if (!strcmp (ctx->if_name, iface->if_name)) +return 0 ; + DRV_LOG(DEBUG, + "NetVSC interface \"%s\" (index %u) renamed\"%s\"",
Quoted 20 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +19
-0+ ctx->if_name, ctx->if_index, iface->if_name); +strncpy (ctx->if_name, iface->if_name,sizeof (ctx- if_name)); +return 0 ; + } +if (vdev_netvsc_iface_is_netvsc (iface)) +return 0 ; +if (!is_same_ether_addr (eth_addr, &ctx->if_addr)) +return 0 ; +/* Look for associated PCI device. */ + ret =vdev_netvsc_sysfs_readlink (buf,sizeof (buf), iface->if_name, +"device/subsystem" ); +if (ret) +return 0 ; + addr =strrchr (buf,'/' ); + addr = addr ? addr +1 : buf; +if (strcmp (addr,"pci" )) +return 0 ; + ret =vdev_netvsc_sysfs_readlink (buf,sizeof (buf), iface->if_name, +"device" );drivers/net/vdev_netvsc/vdev_netvsc.c
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+if (ret) +return 0 ; + addr =strrchr (buf,'/' ); + addr = addr ? addr +1 : buf; + len =strlen (addr); +if (!len) +return 0 ; +/* Send PCI device argument to fail-safe PMD instance. */ +if (strcmp (addr, ctx->yield)) + DRV_LOG(DEBUG, "associating PCI device \"%s\" withNetVSC"
drivers/net/vdev_netvsc/vdev_netvsc.c
Quoted 11 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +11
-0+" interface \"%s\" (index %u)" , addr, ctx->if_name, + ctx->if_index); +memmove (buf, addr, len +1 ); + addr = buf; + buf[len] ='\n' ; + ret =write (ctx->pipe[1 ], addr, len +1 ); + buf[len] ='\0' ; +if (ret ==-1 ) { +if (errno ==EINTR || errno ==EAGAIN ) +return 1 ; + DRV_LOG(WARNING, "cannot associate PCI device name\"%s\" with"
Quoted 25 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +25
-0+" interface \"%s\": %s" , addr, ctx->if_name, + rte_strerror(errno)); +return 1 ; + } +if ((size_t )ret != len +1 ) { +/* +* Attempt to override previous partial write, no need to +* recover if that fails. +*/ + ret =write (ctx->pipe[1 ],"\n" ,1 ); + (void )ret; +return 1 ; + } +fsync (ctx->pipe[1 ]); +memcpy (ctx->yield, addr, len +1 ); +return 1 ; +} + +/** +* Alarm callback that regularly probes system network interfaces. +* +* This callback runs at a frequency determined by +VDEV_NETVSC_PROBE_MS as +* long as an vdev_netvsc context instance exists. +* drivers/net/vdev_netvsc/vdev_netvsc.c
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+* @param arg +* Ignored. +*/ +static void +vdev_netvsc_alarm(__rte_unused void *arg) { +struct vdev_netvsc_ctx *ctx; +int ret; + + LIST_FOREACH(ctx, &vdev_netvsc_ctx_list, entry) { + ret =
vdev_netvsc_foreach_iface(vdev_netvsc_device_probe, ctx);
Quoted 14 lines
drivers/net/vdev_netvsc/vdev_netvsc.c line 353
Quoted 12 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +12
-0+if (ret) +break ; + } +if (!vdev_netvsc_ctx_count) +return ; + ret =rte_eal_alarm_set (VDEV_NETVSC_PROBE_MS *1000 , + vdev_netvsc_alarm,NULL ); +if (ret <0 ) { +DRV_LOG (ERR ,"unable to reschedule alarm callback: %s" , +rte_strerror (-ret)); + } +}Why not use netlink uevent?
As described in doc, we can improve the hotplug mechanism(here and in fail-safe) after EAL hotplug work will be done. So, maybe in next release we will change it to use uevent by EAL hotplug.
Quoted 269 lines
Quoted 46 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +45
-0+/** +* Probe a NetVSC interface to generate a vdev_netvsc context from. +* +* This function instantiates vdev_netvsc contexts either for all +NetVSC +* devices found on the system or only a subset provided as device +* arguments. +* +* It is normally used with vdev_netvsc_foreach_iface(). +* +* @param[in] iface +* Pointer to netdevice description structure (name and index). +* @param[in] eth_addr +* MAC address associated with @p iface. +* @param ap +* Variable arguments list comprising: +* +* - const char *name: +* Name associated with current driver instance. +* +* - struct rte_kvargs *kvargs: +* Device arguments provided to current driver instance. +* +* - unsigned int specified: +* Number of specific netdevices provided as device arguments. +* +* - unsigned int *matched: +* The number of specified netdevices matched by this function. +* +* @return +* A nonzero value when interface matches, 0 otherwise or in case of +* error. +*/ +static int +vdev_netvsc_netvsc_probe (const struct if_nameindex *iface, +const struct ether_addr *eth_addr, +va_list ap) +{ + const char *name = va_arg(ap, const char *); + struct rte_kvargs *kvargs = va_arg(ap, struct rte_kvargs *); + unsigned int specified = va_arg(ap, unsigned int); + unsigned int *matched = va_arg(ap, unsigned int *); +unsigned int i; +struct vdev_netvsc_ctx *ctx; +int ret;drivers/net/vdev_netvsc/vdev_netvsc.c
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+ + /* Probe all interfaces when none are specified. */ + if (specified) { + for (i = 0; i != kvargs->count; ++i) { + const struct rte_kvargs_pair *pair = &kvargs->pairs[i]; + + if (!strcmp(pair->key, VDEV_NETVSC_ARG_IFACE)) { + if (!strcmp(pair->value, iface->if_name)) + break; + } else if (!strcmp(pair->key,
VDEV_NETVSC_ARG_MAC)) {drivers/net/vdev_netvsc/vdev_netvsc.c
Quoted 4 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +4
-0+struct ether_addr tmp; + + if (sscanf(pair->value, + "%" SCNx8 ":%" SCNx8 ":%" SCNx8":"
Quoted 38 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +38
-0+ "%" SCNx8 ":%" SCNx8 ":%" SCNx8, + &tmp.addr_bytes[0], + &tmp.addr_bytes[1], + &tmp.addr_bytes[2], + &tmp.addr_bytes[3], + &tmp.addr_bytes[4], + &tmp.addr_bytes[5]) != 6) { +DRV_LOG (ERR , +"invalid MAC address format" +" \"%s\"" , + pair->value); +return -EINVAL ; + } +if (is_same_ether_addr (eth_addr, &tmp)) +break ; + } + } +if (i == kvargs->count) +return 0 ; + ++(*matched); + } +/* Weed out interfaces already handled. */ + LIST_FOREACH(ctx, &vdev_netvsc_ctx_list, entry) +if (ctx->if_index == iface->if_index) +break ; +if (ctx) { +if (!specified) +return 0 ; +DRV_LOG (WARNING , +"interface \"%s\" (index %u) is already handled," +" skipping" , + iface->if_name, iface->if_index); +return 0 ; + } +if (!vdev_netvsc_iface_is_netvsc (iface)) { +if (!specified) +return 0 ; + DRV_LOG(WARNING,drivers/net/vdev_netvsc/vdev_netvsc.c
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+"interface \"%s\" (index %u) is not NetVSC," +" skipping" , + iface->if_name, iface->if_index); +return 0 ; + } +/* Create interface context. */ + ctx =calloc (1 ,sizeof (*ctx)); +if (!ctx) { + ret = -errno; + DRV_LOG(ERR, "cannot allocate context for interface \"%s\":%s",
Quoted 4 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +4
-0+ iface->if_name, rte_strerror(errno)); + goto error; + } + ctx->id = vdev_netvsc_ctx_count;drivers/net/vdev_netvsc/vdev_netvsc.c
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+strncpy (ctx->if_name, iface->if_name,sizeof (ctx->if_name)); + ctx->if_index = iface->if_index; + ctx->if_addr = *eth_addr; + ctx->pipe[0 ] =-1 ; + ctx->pipe[1 ] =-1 ; + ctx->yield[0 ] ='\0' ; +if (pipe (ctx->pipe) ==-1 ) { + ret = -errno; + DRV_LOG(ERR, + "cannot allocate control pipe for interface \"%s\":%s",
drivers/net/vdev_netvsc/vdev_netvsc.c
Quoted 11 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +11
-0+ ctx->if_name, rte_strerror(errno)); + goto error; + } +for (i =0 ; i !=RTE_DIM (ctx->pipe); ++i) { +int flf =fcntl (ctx->pipe[i],F_GETFL ); + +if (flf !=-1 && +fcntl (ctx->pipe[i],F_SETFL , flf |O_NONBLOCK ) !=-1 ) +continue ; + ret = -errno; + DRV_LOG(ERR, "cannot toggle non-blocking flag on controlfile"
drivers/net/vdev_netvsc/vdev_netvsc.c
Quoted 11 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +11
-0+" descriptor #%u (%d): %s" , i, ctx->pipe[i], + rte_strerror(errno)); + goto error; + } +/* Generate virtual device name and arguments. */ + i =0 ; + ret =snprintf (ctx->name,sizeof (ctx->name),"%s_id%u" , + name, ctx->id); +if (ret ==-1 || (size_t )ret >=sizeof (ctx->name)) + ++i; + ret = snprintf(ctx->devname, sizeof(ctx->devname),"net_failsafe_%s",
drivers/net/vdev_netvsc/vdev_netvsc.c
Quoted 11 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +11
-0+ ctx->name); +if (ret ==-1 || (size_t )ret >=sizeof (ctx->devname)) + ++i; + ret =snprintf (ctx->devargs,sizeof (ctx->devargs), +"fd(%d),dev(net_tap_%s,remote=%s)" , + ctx->pipe[0 ], ctx->name, ctx->if_name); +if (ret ==-1 || (size_t )ret >=sizeof (ctx->devargs)) + ++i; +if (i) { + ret = -ENOBUFS ; + DRV_LOG(ERR, "generated virtual device name or argumentlist"
drivers/net/vdev_netvsc/vdev_netvsc.c
Quoted 5 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +5
-0+ " too long for interface \"%s\"", ctx->if_name); + goto error; + } +/* Request virtual device generation. */ + DRV_LOG(DEBUG, "generating virtual device \"%s\" with arguments\"%s\"",
Quoted 29 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +23
-0+ ctx->devname, ctx->devargs); +vdev_netvsc_foreach_iface (vdev_netvsc_device_probe, ctx); + ret =rte_eal_hotplug_add ("vdev" , ctx->devname, ctx->devargs); +if (ret) + goto error; +LIST_INSERT_HEAD (&vdev_netvsc_ctx_list, ctx, entry); + ++vdev_netvsc_ctx_count; +DRV_LOG (DEBUG ,"added NetVSC interface \"%s\" to context list" , + ctx->if_name); +return 0 ; +error: +if (ctx) +vdev_netvsc_ctx_destroy (ctx); +return ret; +} + +/** * Probe NetVSC interfaces. * +* This function probes system netdevices according to the specified +device +* arguments and starts a periodic alarm callback to notify the +resulting +* fail-safe PMD instances of their sub-devices whereabouts. +* * @param dev * Virtual device context for driver instance. * drivers/net/vdev_netvsc/vdev_netvsc.c line 559
Quoted 9 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +4
-0@@ -49,12 +551,40 @@const char *args =rte_vdev_device_args (dev);struct rte_kvargs *kvargs =rte_kvargs_parse (args ? args :"" , vdev_netvsc_arg); +unsigned int specified =0 ; +unsigned int matched =0 ; +unsigned int i; +int ret; DRV_LOG(DEBUG, "invoked as \"%s\", using arguments \"%s\"",name, args);
Quoted 4 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +0
-0if (!kvargs) {DRV_LOG (ERR ,"cannot parse arguments list" ); goto error; }drivers/net/vdev_netvsc/vdev_netvsc.c
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+for (i =0 ; i != kvargs->count; ++i) { +const struct rte_kvargs_pair *pair = &kvargs->pairs[i]; + +if (!strcmp (pair->key,VDEV_NETVSC_ARG_IFACE ) || + !strcmp (pair->key,VDEV_NETVSC_ARG_MAC )) + ++specified; + } +rte_eal_alarm_cancel (vdev_netvsc_alarm,NULL ); +/* Gather interfaces. */ + ret = vdev_netvsc_foreach_iface(vdev_netvsc_netvsc_probe,name, kvargs,
Quoted 29 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +18
-0+ specified, &matched); +if (ret <0 ) + goto error; +if (matched < specified) +DRV_LOG (WARNING , +"some of the specified parameters did not match" +" recognized network interfaces" ); + ret =rte_eal_alarm_set (VDEV_NETVSC_PROBE_MS *1000 , + vdev_netvsc_alarm,NULL ); +if (ret <0 ) { +DRV_LOG (ERR ,"unable to schedule alarm callback: %s" , +rte_strerror (-ret)); + goto error; + } error:if (kvargs)rte_kvargs_free (kvargs);@@ -65,6 +595,9 @@/** * Remove driver instance. * +* The alarm callback and underlying vdev_netvsc context instances +are only +* destroyed after the last PMD instance is removed. +* * @param dev * Virtual device context for driver instance. * drivers/net/vdev_netvsc/vdev_netvsc.c line 608
Quoted 2 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +0
-0@@ -74,7 +607,16 @@static int vdev_netvsc_vdev_remove(__rte_unused struct rte_vdev_device *dev){
drivers/net/vdev_netvsc/vdev_netvsc.c
Quoted 6 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +5
-1- --vdev_netvsc_ctx_inst; +if (--vdev_netvsc_ctx_inst) +return 0 ; +rte_eal_alarm_cancel (vdev_netvsc_alarm,NULL ); +while (!LIST_EMPTY (&vdev_netvsc_ctx_list)) { + struct vdev_netvsc_ctx *ctx =
LIST_FIRST(&vdev_netvsc_ctx_list);Quoted 8 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +5
-0+ +LIST_REMOVE (ctx, entry); + --vdev_netvsc_ctx_count; +vdev_netvsc_ctx_destroy (ctx); + }return 0 ; }
Re: [PATCH v3 5/8] net/vdev_netvsc: implement core functionality
10/01/2018 16:02, Matan Azrad:
Quoted 13 lines
From: Stephen Hemminger, Tuesday, January 9, 2018 8:49 PM
Quoted 11 lines
drivers/net/vdev_netvsc/vdev_netvsc.c line 353
Quoted 7 lines · On Tue, 9 Jan 2018 14:47:30 +0000 Matan Azrad <[email protected]> wrote:
drivers/net/vdev_netvsc/vdev_netvsc.c +7
-0+ ret =rte_eal_alarm_set (VDEV_NETVSC_PROBE_MS *1000 , + vdev_netvsc_alarm,NULL ); +if (ret <0 ) { +DRV_LOG (ERR ,"unable to reschedule alarm callback: %s" , +rte_strerror (-ret)); + } +}Why not use netlink uevent?
Quoted 2 lines
As described in doc, we can improve the hotplug mechanism(here and in fail-safe) after EAL hotplug work will be done. So, maybe in next release we will change it to use uevent by EAL hotplug.
I don't see any progress here for one week.
Yes it is a temporary solution waiting for hotplug event callback in EAL. Hopefully it will be possible to do such improvements in 18.05.
Am I missing something else?
Or can it be applied to next-net?
PATCH v3 6/8 net/vdev_netvsc: skip routed netvsc probing
In reply to Matan AzradNetVSC netdevices which are already routed should not be probed because they are used for management purposes by the HyperV.
prevent routed netvsc devices probing.
--- doc/guides/nics/vdev_netvsc.rst | 2 +- drivers/net/vdev_netvsc/vdev_netvsc.c | 46 +++++++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+), 1 deletion(-)
doc/guides/nics/vdev_netvsc.rst +1 -1
@@ -87,4 +87,4 @@ The following device parameters are supported: MAC address. Not specifying either ``iface`` or ``mac`` makes this driver attach itself to -all NetVSC interfaces found on the system. +all unrouted NetVSC interfaces found on the system.
drivers/net/vdev_netvsc/vdev_netvsc.c +46 -0
64 lines not on this page · raw message
Signature · 1 line
1.8.3.1
Re: [PATCH v3 6/8] net/vdev_netvsc: skip routed netvsc probing
Quoted 64 lines · On Tue, 9 Jan 2018 14:47:31 +0000 Matan Azrad <[email protected]> wrote:
NetVSC netdevices which are already routed should not be probed because they are used for management purposes by the HyperV.
prevent routed netvsc devices probing.
Signed-off-by Raslan Darawsheh <[email protected]>Signed-off-by Matan Azrad <[email protected]>--- doc/guides/nics/vdev_netvsc.rst | 2 +- drivers/net/vdev_netvsc/vdev_netvsc.c | 46 +++++++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+), 1 deletion(-)doc/guides/nics/vdev_netvsc.rst +1
-1@@ -87,4 +87,4 @@ The following device parameters are supported: MAC address. Not specifying either ``iface`` or ``mac`` makes this driver attach itself to -all NetVSC interfaces found on the system. +all unrouted NetVSC interfaces found on the system.drivers/net/vdev_netvsc/vdev_netvsc.c +27
-0@@ -38,6 +38,7 @@#define VDEV_NETVSC_PROBE_MS 1000#define NETVSC_CLASS_ID "{f8615163-df3e-46c5-913f-f2d2f965ed0e}" +#define NETVSC_MAX_ROUTE_LINE_SIZE 300#define DRV_LOG (level, ...) \ rte_log(RTE_LOG_ ## level, \@@ -192,6 +193,44 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list = }/** +* Determine if a network interface has a route. +* +* @param[in] name +* Network device name. +* +* @return +* A nonzero value when interface has an route. In case of error, +* rte_errno is updated and 0 returned. +*/ +static int +vdev_netvsc_has_route (const char *name) +{ +FILE *fp; +int ret =0 ; +char route[NETVSC_MAX_ROUTE_LINE_SIZE ]; +char *netdev; + + fp =fopen ("/proc/net/route" ,"r" ); +if (!fp) { + rte_errno = errno; +return 0 ; + } +while (fgets (route,NETVSC_MAX_ROUTE_LINE_SIZE , fp) !=NULL ) { + netdev =strtok (route,"\t" ); +if (strcmp (netdev, name) ==0 ) { + ret =1 ;
drivers/net/vdev_netvsc/vdev_netvsc.c static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list = · line 231
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+break ; + } +/* Move file pointer to the next line. */ +while (strchr (route,'\n' ) ==NULL && +fgets (route,NETVSC_MAX_ROUTE_LINE_SIZE , fp) !=NULL ) + ; + } +fclose (fp); +return ret; +}
In many ways /proc/net/route is legacy intervace.
And system may have 1 M routes.
Maybe there is faster way to do this with netlink by looking to see if there is an address associated with the interface.
Re: [PATCH v3 6/8] net/vdev_netvsc: skip routed netvsc probing
Hi Stephan
From: Stephen Hemminger, Tuesday, January 9, 2018 8:51 PM
Quoted 89 lines
To: Matan Azrad <[email protected]>
Cc: Ferruh Yigit <[email protected]>; Thomas Monjalon <[email protected]>; [email protected]; Raslan Darawsheh <[email protected]>
Subject: Re: [PATCH v3 6/8]net/vdev_netvsc: skip routed netvsc probingQuoted 60 lines · On Tue, 9 Jan 2018 14:47:31 +0000 Matan Azrad <[email protected]> wrote:
NetVSC netdevices which are already routed should not be probed because they are used for management purposes by the HyperV.
prevent routed netvsc devices probing.
Signed-off-by Raslan Darawsheh <[email protected]>Signed-off-by Matan Azrad <[email protected]>--- doc/guides/nics/vdev_netvsc.rst | 2 +- drivers/net/vdev_netvsc/vdev_netvsc.c | 46 +++++++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+), 1 deletion(-)(quoted diff) +0
-0
b/doc/guides/nics/vdev_netvsc.rstindex fde1fb8..f779862 100644doc/guides/nics/vdev_netvsc.rst +0
-0@@ -87,4 +87,4 @@ The following device parameters are supported: MAC address. Not specifying either ``iface`` or ``mac`` makes this driver attachitself to -all NetVSC interfaces found on the system.
doc/guides/nics/vdev_netvsc.rst +1
-0+all unrouted NetVSC interfaces found on the system.drivers/net/vdev_netvsc/vdev_netvsc.c +21
-0@@ -38,6 +38,7 @@#define VDEV_NETVSC_PROBE_MS 1000#define NETVSC_CLASS_ID "{f8615163-df3e-46c5-913f-f2d2f965ed0e}" +#define NETVSC_MAX_ROUTE_LINE_SIZE 300#define DRV_LOG (level, ...) \ rte_log(RTE_LOG_ ## level, \@@ -192,6 +193,44 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list = }/** +* Determine if a network interface has a route. +* +* @param[in] name +* Network device name. +* +* @return +* A nonzero value when interface has an route. In case of error, +* rte_errno is updated and 0 returned. +*/ +static int +vdev_netvsc_has_route (const char *name) { +FILE *fp; +int ret =0 ; +char route[NETVSC_MAX_ROUTE_LINE_SIZE ]; +char *netdev; + + fp =fopen ("/proc/net/route" ,"r" ); +if (!fp) { + rte_errno = errno; +return 0 ;drivers/net/vdev_netvsc/vdev_netvsc.c static LIST_HEAD(, vdev_netvsc_ctx) · line 225
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+ } + while (fgets(route, NETVSC_MAX_ROUTE_LINE_SIZE, fp) != NULL) { + netdev = strtok(route, "\t"); + if (strcmp(netdev, name) == 0) { + ret = 1; + break; + } + /* Move file pointer to the next line. */ + while (strchr(route, '\n') == NULL && + fgets(route, NETVSC_MAX_ROUTE_LINE_SIZE, fp) !=NULL)
drivers/net/vdev_netvsc/vdev_netvsc.c static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list = · line 231
Quoted 5 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +5
-0+ ; + } +fclose (fp); +return ret; +}In many ways
/proc/net/routeis legacy intervace.
And system may have 1 M routes.
Quoted 2 lines
Maybe there is faster way to do this with netlink by looking to see if there is an address associated with the interface.
Actually this is control path, we don't care about performance very much. But I can get other idea here, Do you have suggestion?
Thanks!
Re: [PATCH v3 6/8] net/vdev_netvsc: skip routed netvsc probing
Quoted 92 lines · On Wed, 10 Jan 2018 15:07:14 +0000 Matan Azrad <[email protected]> wrote:
Hi Stephan
From: Stephen Hemminger, Tuesday, January 9, 2018 8:51 PM
Quoted 89 lines
To: Matan Azrad <[email protected]>
Cc: Ferruh Yigit <[email protected]>; Thomas Monjalon <[email protected]>; [email protected]; Raslan Darawsheh <[email protected]>
Subject: Re: [PATCH v3 6/8]net/vdev_netvsc: skip routed netvsc probingQuoted 60 lines · On Tue, 9 Jan 2018 14:47:31 +0000 Matan Azrad <[email protected]> wrote:
NetVSC netdevices which are already routed should not be probed because they are used for management purposes by the HyperV.
prevent routed netvsc devices probing.
Signed-off-by Raslan Darawsheh <[email protected]>Signed-off-by Matan Azrad <[email protected]>--- doc/guides/nics/vdev_netvsc.rst | 2 +- drivers/net/vdev_netvsc/vdev_netvsc.c | 46 +++++++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+), 1 deletion(-)(quoted diff) +0
-0
b/doc/guides/nics/vdev_netvsc.rstindex fde1fb8..f779862 100644doc/guides/nics/vdev_netvsc.rst +0
-0@@ -87,4 +87,4 @@ The following device parameters are supported: MAC address. Not specifying either ``iface`` or ``mac`` makes this driver attachitself to -all NetVSC interfaces found on the system.
doc/guides/nics/vdev_netvsc.rst +1
-0+all unrouted NetVSC interfaces found on the system.drivers/net/vdev_netvsc/vdev_netvsc.c +21
-0@@ -38,6 +38,7 @@#define VDEV_NETVSC_PROBE_MS 1000#define NETVSC_CLASS_ID "{f8615163-df3e-46c5-913f-f2d2f965ed0e}" +#define NETVSC_MAX_ROUTE_LINE_SIZE 300#define DRV_LOG (level, ...) \ rte_log(RTE_LOG_ ## level, \@@ -192,6 +193,44 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list = }/** +* Determine if a network interface has a route. +* +* @param[in] name +* Network device name. +* +* @return +* A nonzero value when interface has an route. In case of error, +* rte_errno is updated and 0 returned. +*/ +static int +vdev_netvsc_has_route (const char *name) { +FILE *fp; +int ret =0 ; +char route[NETVSC_MAX_ROUTE_LINE_SIZE ]; +char *netdev; + + fp =fopen ("/proc/net/route" ,"r" ); +if (!fp) { + rte_errno = errno; +return 0 ;drivers/net/vdev_netvsc/vdev_netvsc.c static LIST_HEAD(, vdev_netvsc_ctx) · line 225
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+ } + while (fgets(route, NETVSC_MAX_ROUTE_LINE_SIZE, fp) != NULL) { + netdev = strtok(route, "\t"); + if (strcmp(netdev, name) == 0) { + ret = 1; + break; + } + /* Move file pointer to the next line. */ + while (strchr(route, '\n') == NULL && + fgets(route, NETVSC_MAX_ROUTE_LINE_SIZE, fp) !=NULL)
drivers/net/vdev_netvsc/vdev_netvsc.c static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list = · line 231
Quoted 5 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +5
-0+ ; + } +fclose (fp); +return ret; +}In many ways
/proc/net/routeis legacy intervace.
And system may have 1 M routes.
Quoted 8 lines
Quoted 2 lines
Maybe there is faster way to do this with netlink by looking to see if there is an address associated with the interface.
Actually this is control path, we don't care about performance very much. But I can get other idea here, Do you have suggestion?
Thanks!
Use netlink (or ioctl) to get interface address.
If interface has an IPv4 or IPv6 (not link local), then skip it.
Re: [PATCH v3 6/8] net/vdev_netvsc: skip routed netvsc probing
Hi Stephan
From: Stephen Hemminger, Wednesday, January 10, 2018 6:44 PM
Quoted 99 lines
Quoted 96 lines · On Wed, 10 Jan 2018 15:07:14 +0000 Matan Azrad <[email protected]> wrote:
Hi Stephan
From: Stephen Hemminger, Tuesday, January 9, 2018 8:51 PM
Quoted 92 lines
To: Matan Azrad <[email protected]>
Cc: Ferruh Yigit <[email protected]>; Thomas Monjalon <[email protected]>; [email protected]; Raslan Darawsheh <[email protected]>
Subject: Re: [PATCH v3 6/8]net/vdev_netvsc: skip routed netvsc probingQuoted 60 lines · On Tue, 9 Jan 2018 14:47:31 +0000 Matan Azrad <[email protected]> wrote:
NetVSC netdevices which are already routed should not be probed because they are used for management purposes by the HyperV.
prevent routed netvsc devices probing.
Signed-off-by Raslan Darawsheh <[email protected]>Signed-off-by Matan Azrad <[email protected]>--- doc/guides/nics/vdev_netvsc.rst | 2 +- drivers/net/vdev_netvsc/vdev_netvsc.c | 46 +++++++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+), 1 deletion(-)(quoted diff) +0
-0
b/doc/guides/nics/vdev_netvsc.rstindex fde1fb8..f779862 100644doc/guides/nics/vdev_netvsc.rst +0
-0@@ -87,4 +87,4 @@ The following device parameters are supported: MAC address. Not specifying either ``iface`` or ``mac`` makes this driverattach itself to -all NetVSC interfaces found on the system.
doc/guides/nics/vdev_netvsc.rst +1
-0+all unrouted NetVSC interfaces found on the system.drivers/net/vdev_netvsc/vdev_netvsc.c +21
-0@@ -38,6 +38,7 @@#define VDEV_NETVSC_PROBE_MS 1000#define NETVSC_CLASS_ID "{f8615163-df3e-46c5-913f-f2d2f965ed0e}" +#define NETVSC_MAX_ROUTE_LINE_SIZE 300#define DRV_LOG (level, ...) \ rte_log(RTE_LOG_ ## level, \@@ -192,6 +193,44 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list = }/** +* Determine if a network interface has a route. +* +* @param[in] name +* Network device name. +* +* @return +* A nonzero value when interface has an route. In case of error, +* rte_errno is updated and 0 returned. +*/ +static int +vdev_netvsc_has_route (const char *name) { +FILE *fp; +int ret =0 ; +char route[NETVSC_MAX_ROUTE_LINE_SIZE ]; +char *netdev; + + fp =fopen ("/proc/net/route" ,"r" ); +if (!fp) { + rte_errno = errno; +return 0 ;drivers/net/vdev_netvsc/vdev_netvsc.c static LIST_HEAD(, vdev_netvsc_ctx) · line 225
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+ } + while (fgets(route, NETVSC_MAX_ROUTE_LINE_SIZE, fp) != NULL) { + netdev = strtok(route, "\t"); + if (strcmp(netdev, name) == 0) { + ret = 1; + break; + } + /* Move file pointer to the next line. */ + while (strchr(route, '\n') == NULL && + fgets(route, NETVSC_MAX_ROUTE_LINE_SIZE, fp) !=NULL)
drivers/net/vdev_netvsc/vdev_netvsc.c static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list = · line 231
Quoted 5 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +5
-0+ ; + } +fclose (fp); +return ret; +}In many ways
/proc/net/routeis legacy intervace.
And system may have 1 M routes.Maybe there is faster way to do this with netlink by looking to see if there is an address associated with the interface.
Quoted 8 lines
Quoted 5 lines
Actually this is control path, we don't care about performance very much. But I can get other idea here, Do you have suggestion?
Thanks!
Use netlink (or ioctl) to get interface address.
If interface has an IPv4 or IPv6 (not link local), then skip it.
As I a little bit investigated I found that IPv6 getting is problematic by ioctl.
And using nelink for it, really doesn't worth the effort.
So, I suggest to keep this code simple as is in spite of the optional high latency for this function, after all it is a control path.
Re: [PATCH v3 6/8] net/vdev_netvsc: skip routed netvsc probing
11/01/2018 10:00, Matan Azrad:
Quoted 44 lines
From: Stephen Hemminger, Wednesday, January 10, 2018 6:44 PM
Quoted 43 lines
Quoted 42 lines
From: Stephen Hemminger, Tuesday, January 9, 2018 8:51 PM
Quoted 33 lines
Quoted 11 lines · On Tue, 9 Jan 2018 14:47:31 +0000 Matan Azrad <[email protected]> wrote:
drivers/net/vdev_netvsc/vdev_netvsc.c +11
-0+static int +vdev_netvsc_has_route (const char *name) { +FILE *fp; +int ret =0 ; +char route[NETVSC_MAX_ROUTE_LINE_SIZE ]; +char *netdev; + + fp =fopen ("/proc/net/route" ,"r" ); +if (!fp) { + rte_errno = errno; +return 0 ;drivers/net/vdev_netvsc/vdev_netvsc.c static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+ } + while (fgets(route, NETVSC_MAX_ROUTE_LINE_SIZE, fp) != NULL) { + netdev = strtok(route, "\t"); + if (strcmp(netdev, name) == 0) { + ret = 1; + break; + } + /* Move file pointer to the next line. */ + while (strchr(route, '\n') == NULL && + fgets(route, NETVSC_MAX_ROUTE_LINE_SIZE, fp) !=NULL)
drivers/net/vdev_netvsc/vdev_netvsc.c static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list = · line 231
Quoted 5 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +5
-0+ ; + } +fclose (fp); +return ret; +}In many ways
/proc/net/routeis legacy intervace.
And system may have 1 M routes.Quoted 2 lines
Maybe there is faster way to do this with netlink by looking to see if there is an address associated with the interface.
Actually this is control path, we don't care about performance very much. But I can get other idea here, Do you have suggestion?
Thanks!
Quoted 6 lines
Quoted 2 lines
Use netlink (or ioctl) to get interface address.
If interface has an IPv4 or IPv6 (not link local), then skip it.As I a little bit investigated I found that IPv6 getting is problematic by ioctl.
And using nelink for it, really doesn't worth the effort.
So, I suggest to keep this code simple as is in spite of the optional high latency for this function, after all it is a control path.
No more comment?
So we are OK with this solution for now?
If we see real performance issue, I guess it can be fixed later.
PATCH v3 7/8 net/vdev_netvsc: add "force" parameter
In reply to Matan AzradThis parameter allows specifying any non-NetVSC interface or routed NetVSC interfaces to use with tap sub-devices for development purposes.
--- doc/guides/nics/vdev_netvsc.rst | 5 +++++ drivers/net/vdev_netvsc/vdev_netvsc.c | 30 +++++++++++++++++++----------- 2 files changed, 24 insertions(+), 11 deletions(-)
doc/guides/nics/vdev_netvsc.rst +5 -0
@@ -86,5 +86,10 @@ The following device parameters are supported: Same as ``iface`` except a suitable NetVSC interface is located using its MAC address. +- ``force`` [int] + + If nonzero, forces the use of specified interfaces even if not detected as + NetVSC or detected as routed NETVSC. + Not specifying either ``iface`` or ``mac`` makes this driver attach itself to all unrouted NetVSC interfaces found on the system.
drivers/net/vdev_netvsc/vdev_netvsc.c +19 -11
@@ -35,6 +35,7 @@#define VDEV_NETVSC_DRIVER net_vdev_netvsc#define VDEV_NETVSC_ARG_IFACE "iface"#define VDEV_NETVSC_ARG_MAC "mac" +#define VDEV_NETVSC_ARG_FORCE "force"#define VDEV_NETVSC_PROBE_MS 1000#define NETVSC_CLASS_ID "{f8615163-df3e-46c5-913f-f2d2f965ed0e}"
@@ -413,6 +414,9 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =* - struct rte_kvargs *kvargs: * Device arguments provided to current driver instance. * +* - int force: +* Accept specified interface even if not detected as NetVSC. +* * - unsigned int specified: * Number of specific netdevices provided as device arguments. *
@@ -430,6 +434,7 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list = { const char *name = va_arg(ap, const char *); struct rte_kvargs *kvargs = va_arg(ap, struct rte_kvargs *); +int force =va_arg (ap, int); unsigned int specified = va_arg(ap, unsigned int); unsigned int *matched = va_arg(ap, unsigned int *);unsigned int i;
@@ -484,20 +489,18 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =return 0 ; }if (!vdev_netvsc_iface_is_netvsc (iface)) { -if (!specified) +if (!specified || !force)return 0 ;DRV_LOG (WARNING , -"interface \"%s\" (index %u) is not NetVSC," -" skipping" , +"using non-NetVSC interface \"%s\" (index %u)" , iface->if_name, iface->if_index); -return 0 ; }/* Routed NetVSC should not be probed. */ if (vdev_netvsc_has_route (iface->if_name)) { -DRV_LOG (WARNING ,"NetVSC interface \"%s\" (index %u) is routed" , - iface->if_name, iface->if_index); -if (!specified) +if (!specified || !force)return 0 ; +DRV_LOG (WARNING ,"using routed NetVSC interface \"%s\"" +" (index %u)" , iface->if_name, iface->if_index); }/* Create interface context. */ ctx =calloc (1 ,sizeof (*ctx));
@@ -591,6 +594,7 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =static const char *const vdev_netvsc_arg[] = {VDEV_NETVSC_ARG_IFACE ,VDEV_NETVSC_ARG_MAC , +VDEV_NETVSC_ARG_FORCE ,NULL , };const char *name =rte_vdev_device_name (dev);
@@ -599,6 +603,7 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list = vdev_netvsc_arg);unsigned int specified =0 ;unsigned int matched =0 ; +int force =0 ;unsigned int i;int ret;
@@ -610,14 +615,16 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =for (i =0 ; i != kvargs->count; ++i) {const struct rte_kvargs_pair *pair = &kvargs->pairs[i]; -if (!strcmp (pair->key,VDEV_NETVSC_ARG_IFACE ) || - !strcmp (pair->key,VDEV_NETVSC_ARG_MAC )) +if (!strcmp (pair->key,VDEV_NETVSC_ARG_FORCE )) + force = !!atoi (pair->value); +else if (!strcmp (pair->key,VDEV_NETVSC_ARG_IFACE ) || + !strcmp (pair->key,VDEV_NETVSC_ARG_MAC )) ++specified; }rte_eal_alarm_cancel (vdev_netvsc_alarm,NULL );/* Gather interfaces. */ ret =vdev_netvsc_foreach_iface (vdev_netvsc_netvsc_probe, name, kvargs, - specified, &matched); + force, specified, &matched);if (ret <0 ) goto error;if (matched < specified)
@@ -676,7 +683,8 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =RTE_PMD_REGISTER_ALIAS (VDEV_NETVSC_DRIVER , eth_vdev_netvsc);RTE_PMD_REGISTER_PARAM_STRING (net_vdev_netvsc,VDEV_NETVSC_ARG_IFACE "=<string> " -VDEV_NETVSC_ARG_MAC "=<string>" ); +VDEV_NETVSC_ARG_MAC "=<string> " +VDEV_NETVSC_ARG_FORCE "=<int>" );/** Initialize driver log type. */ RTE_INIT (vdev_netvsc_init_log)
Signature · 1 line
1.8.3.1
PATCH v3 8/8 net/vdev_netvsc: add automatic probing
In reply to Matan AzradUsing DPDK in Hyper-V VM systems requires vdev_netvsc driver to pair the NetVSC netdev device with the same MAC address PCI device by fail-safe PMD.
Add vdev_netvsc custom scan in vdev bus to allow automatic probing in Hyper-V VM systems unless it was already specified by command line.
Add "ignore" parameter to disable this auto-detection.
--- doc/guides/nics/vdev_netvsc.rst | 9 ++++-- drivers/net/vdev_netvsc/vdev_netvsc.c | 55 +++++++++++++++++++++++++++++++++-- 2 files changed, 60 insertions(+), 4 deletions(-)
doc/guides/nics/vdev_netvsc.rst +7 -2
@@ -71,8 +71,8 @@ Build options Run-time parameters ------------------- -To invoke this driver, applications have to explicitly provide the -``--vdev=net_vdev_netvsc`` EAL option. +This driver is invoked automatically in Hyper-V VM systems unless the user +invoked it by command line using ``--vdev=net_vdev_netvsc`` EAL option. The following device parameters are supported:
@@ -91,5 +91,10 @@ The following device parameters are supported: If nonzero, forces the use of specified interfaces even if not detected as NetVSC or detected as routed NETVSC. +- ``ignore`` [int] + + If nonzero, ignores the driver runnig (actually used to disable the + auto-detection in Hyper-V VM). + Not specifying either ``iface`` or ``mac`` makes this driver attach itself to all unrouted NetVSC interfaces found on the system.
drivers/net/vdev_netvsc/vdev_netvsc.c +53 -2
105 lines not on this page · raw message
Signature · 1 line
1.8.3.1
v4 cover Introduce virtual driver for Hyper-V/Azure platforms
In reply to Matan AzradVirtual machines hosted by Hyper-V/Azure platforms are fitted with simplified virtual network devices named NetVSC that are used for fast communication between VM to VM, VM to hypervisor, and the outside.
They appear as standard system netdevices to user-land applications, the main difference being they are implemented on top of VMBUS instead of emulated PCI devices.
While this reads like a case for a standard DPDK PMD, there is more to it.
To accelerate outside communication, NetVSC devices as they appear in a VM can be paired with physical SR-IOV virtual function (VF) devices owned by that same VM. Both netdevices share the same MAC address in that case.
When paired, egress and most of the ingress traffic flow through the VF device, while part of it (e.g. multicasts, hypervisor control data) still flows through NetVSC. Moreover VF devices are not retained and disappear during VM migration; from a VM standpoint, they can be hot-plugged anytime with NetVSC acting as a fallback.
Running DPDK applications in such a context involves driving VF devices using their dedicated PMDs in a vendor-independent fashion (to benefit from maximum performance without writing dedicated code) while simultaneously listening to NetVSC and handling the related hot-plug events.
This new virtual driver (referred to as "vdev_netvsc" from this point on) automatically coordinates the Hyper-V/Azure-specific management part described above by relying on vendor-specific, failsafe and tap PMDs to expose a single consolidated Ethernet device usable directly by existing applications.
.------------------.
| DPDK application |
`--------+---------'
|
.------+------.
| DPDK ethdev |
`------+------' Control
| |
.------------+------------. v .--------------------.
| failsafe PMD +---------+ vdev_netvsc driver |
`--+-------------------+--' `--------------------'
| |
| .........|.........
| : | :
.----+----. : .----+----. :
| tap PMD | : | any PMD | :
`----+----' : `----+----' : <-- Hot-pluggable
| : | :
.------+-------. : .-----+-----. :
| NetVSC-based | : | SR-IOV VF | :
| netdevice | : | device | :
`--------------' : `-----------' :
:.................:v2 changes(Adrien):
- Renamed driver from "hyperv" to "vdev_netvsc". This change covers documentation and symbols prefix.
- Driver is now tagged EXPERIMENTAL.
- Replaced ether_addr_from_str() with a basic sscanf() call.
- Removed debugging code (memset() poisoning).
- Fixed hyperv_iface_is_netvsc()'s buffer allocation according to comments.
- Removed hyperv_basename().
- Discarded unused variables through __rte_unused.
- Added separate but necessary free() bugfix for failsafe PMD.
- Added file descriptor input support to failsafe PMD.
- Replaced temporary bash execution; failsafe now reads device definitions directly through a pipe without an intermediate bash one-liner.
- Expanded DEBUG/INFO/WARN/ERROR() macros as PMD_DRV_LOG().
- Added dynamic log type (pmd.vdev_netvsc).
- Modified initialization code to probe devices immediately during startup.
- Fixed several snprintf() return value checks ("ret >= sizeof(foo)" is more appropriate than "ret >= sizeof(foo) - 1").
v3 changes(Matan):
- Fixed clang compilation in V2.
- Removed hotplug remove code from the new driver.
- Supported probed sub-devices getting in fail-safe.
- Added automatic probing for HyperV VM systems.
- Added option to ignore the automatic probing.
- Skiped routed NetVSC devices probing.
- Adjusted documentation and semantics.
- Replaced maintainer.
v4 changes(Matan):
- Align descriptions of context struct(Stephen suggestion).
- Skip non-ethernet devices in netdev loop(Stephen suggestion).
- Use different variable names in "add fd parameter"(Gaetan suggestion).
- Change name of get port id function in "add automatic probing"(Gaetan suggestion).
- Update internal fail-safe devargs in case of probed device(Gaetan suggestion).
- use deferent commit title instead of "support probed sub-devices getting"(Gaetan suggestion).
Adrien Mazarguil (1):
net/failsafe: fix invalid free
Matan Azrad (7):
net/failsafe: add "fd" parameter net/failsafe: add probed etherdev capture net/vdev_netvsc: introduce Hyper-V platform driver net/vdev_netvsc: implement core functionality net/vdev_netvsc: skip routed netvsc probing net/vdev_netvsc: add "force" parameter net/vdev_netvsc: add automatic probing
MAINTAINERS | 6 + config/common_base | 5 + config/common_linuxapp | 1 + doc/guides/nics/fail_safe.rst | 14 + doc/guides/nics/features/vdev_netvsc.ini | 12 + doc/guides/nics/index.rst | 1 + doc/guides/nics/vdev_netvsc.rst | 100 +++ drivers/net/Makefile | 1 + drivers/net/failsafe/failsafe_args.c | 84 ++- drivers/net/failsafe/failsafe_eal.c | 78 ++- drivers/net/failsafe/failsafe_private.h | 5 + drivers/net/vdev_netvsc/Makefile | 31 + .../vdev_netvsc/rte_pmd_vdev_netvsc_version.map | 4 + drivers/net/vdev_netvsc/vdev_netvsc.c | 752 +++++++++++++++++++++ mk/rte.app.mk | 1 + 15 files changed, 1071 insertions(+), 24 deletions(-) create mode 100644 doc/guides/nics/features/vdev_netvsc.ini create mode 100644 doc/guides/nics/vdev_netvsc.rst create mode 100644 drivers/net/vdev_netvsc/Makefile create mode 100644 drivers/net/vdev_netvsc/rte_pmd_vdev_netvsc_version.map create mode 100644 drivers/net/vdev_netvsc/vdev_netvsc.c
Signature · 1 line
1.8.3.1
PATCH v4 1/8 net/failsafe: fix invalid free
From: Adrien Mazarguil <[email protected]>
rte_free() is not supposed to work with pointers returned by calloc().
--- drivers/net/failsafe/failsafe_args.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/failsafe/failsafe_args.c +1 -1
@@ -407,7 +407,7 @@ typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params,uint8_t i; FOREACH_SUBDEV(sdev, i, dev) { -rte_free (sdev->cmdline); +free (sdev->cmdline); sdev->cmdline =NULL ;free (sdev->devargs.args); sdev->devargs.args =NULL ;
Signature · 1 line
1.8.3.1
PATCH v4 2/8 net/failsafe: add "fd" parameter
In reply to Matan AzradThis parameter enables applications to provide device definitions through an arbitrary file descriptor number.
--- doc/guides/nics/fail_safe.rst | 9 ++++ drivers/net/failsafe/failsafe_args.c | 80 ++++++++++++++++++++++++++++++++- drivers/net/failsafe/failsafe_private.h | 3 ++ 3 files changed, 91 insertions(+), 1 deletion(-)
doc/guides/nics/fail_safe.rst +9 -0
@@ -106,6 +106,15 @@ Fail-safe command line parameters All commas within the ``shell command`` are replaced by spaces before executing the command. This helps using scripts to specify devices. +- **fd(<file descriptor number>)** parameter + + This parameter reads a device definition from an arbitrary file descriptor + number in ``<iface>`` format as described above. + + The file descriptor is read in non-blocking mode and is never closed in + order to take only the last line into account (unlike ``exec()``) at every + probe attempt. + - **mac** parameter [MAC address] This parameter allows the user to set a default MAC address to the fail-safe
drivers/net/failsafe/failsafe_args.c +79 -1
117 lines not on this page · raw message
drivers/net/failsafe/failsafe_private.h +3 -0
@@ -48,6 +48,7 @@#define PMD_FAILSAFE_PARAM_STRING \ "dev(<ifc>)," \ "exec(<shell command>)," \ + "fd(<fd number>)," \ "mac=mac_addr," \ "hotplug_poll=u64" \ ""
@@ -112,6 +113,8 @@ struct sub_device {struct fs_stats stats_snapshot;/* Some device are defined as a command line */ char *cmdline; +/* Others are retrieved through a file descriptor */ +char *fd_str;/* fail-safe device backreference */ struct rte_eth_dev *fs_dev;/* flag calling for recollection */
Signature · 1 line
1.8.3.1
Re: [PATCH v4 2/8] net/failsafe: add "fd" parameter
Hi Matan,
You forgot to fix the fcntl call, see below,
Quoted 5 lines · Matan Azrad
This parameter enables applications to provide device definitions through an arbitrary file descriptor number.
Signed-off-by Adrien Mazarguil <[email protected]>Signed-off-by Matan Azrad <[email protected]>
with the relevant fixes:
Quoted 68 lines
--- doc/guides/nics/fail_safe.rst | 9 ++++ drivers/net/failsafe/failsafe_args.c | 80 ++++++++++++++++++++++++++++++++- drivers/net/failsafe/failsafe_private.h | 3 ++ 3 files changed, 91 insertions(+), 1 deletion(-)doc/guides/nics/fail_safe.rst +9
-0@@ -106,6 +106,15 @@ Fail-safe command line parameters All commas within the ``shell command`` are replaced by spaces before executing the command. This helps using scripts to specify devices. +- **fd(<file descriptor number>)** parameter + + This parameter reads a device definition from an arbitrary file descriptor + number in ``<iface>`` format as described above. + + The file descriptor is read in non-blocking mode and is never closed in + order to take only the last line into account (unlike ``exec()``) at every + probe attempt. + - **mac** parameter [MAC address] This parameter allows the user to set a default MAC address to the fail-safedrivers/net/failsafe/failsafe_args.c +26
-0@@ -31,7 +31,11 @@* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ +#include <fcntl.h> +#include <stdio.h> +#include <stdlib.h> #include <string.h> +#include <unistd.h> #include <errno.h> #include <rte_debug.h> @@ -161,6 +165,67 @@ typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, }static int +fs_read_fd (struct sub_device *sdev,char *fd_str) +{ +FILE *fp =NULL ; +int fd =-1 ; +/* store possible newline as well */ +char output[DEVARGS_MAXLEN +1 ]; +int err = -ENODEV ; +int oflags; +int lcount; + +RTE_ASSERT (fd_str !=NULL || sdev->fd_str !=NULL ); +if (sdev->fd_str ==NULL ) { + sdev->fd_str =strdup (fd_str); +if (sdev->fd_str ==NULL ) { +ERROR ("Command line allocation failed" ); +return -ENOMEM ; + } + } + errno =0 ; + fd =strtol (fd_str, &fd_str,0 ); +if (errno || *fd_str || fd <0 ) { +ERROR ("Parsing FD number failed" );
drivers/net/failsafe/failsafe_args.c typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, · line 199
Quoted 10 lines
drivers/net/failsafe/failsafe_args.c +10
-0+ goto error; + } +/* Fiddle with copy of file descriptor */ + fd =dup (fd); +if (fd ==-1 ) + goto error; + oflags =fcntl (fd,F_GETFL ); +if (oflags ==-1 ) + goto error; +if (fcntl (fd,F_SETFL , fd |O_NONBLOCK ) ==-1 )
fcntl(fd, F_SETFL, oflags | O_NONBLOCK); here
drivers/net/failsafe/failsafe_args.c typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params, · line 202
Quoted 3 lines
drivers/net/failsafe/failsafe_args.c +3
-0+ goto error; + fp =fdopen (fd,"r" ); +if (!fp)
While you're at it, here please use
if (fp != NULL)
instead.
Regards,
Signature · 2 lines
Gaëtan Rivet 6WIND
PATCH v4 3/8 net/failsafe: add probed etherdev capture
In reply to Matan AzradPrevious fail-safe code didn't support probed sub-devices capture and failed when it tried to probe them.
Skip fail-safe sub-device probing when it already was probed.
--- doc/guides/nics/fail_safe.rst | 5 +++ drivers/net/failsafe/failsafe_args.c | 2 - drivers/net/failsafe/failsafe_eal.c | 78 ++++++++++++++++++++++++--------- drivers/net/failsafe/failsafe_private.h | 2 + 4 files changed, 65 insertions(+), 22 deletions(-)
doc/guides/nics/fail_safe.rst +5 -0
@@ -115,6 +115,11 @@ Fail-safe command line parameters order to take only the last line into account (unlike ``exec()``) at every probe attempt. +.. note:: + + In case of whitelist sub-device probed by EAL, fail-safe PMD will take the device + as is, which means that EAL device options are taken in this case. + - **mac** parameter [MAC address] This parameter allows the user to set a default MAC address to the fail-safe
drivers/net/failsafe/failsafe_args.c +0 -2
@@ -45,8 +45,6 @@#include "failsafe_private.h" -#define DEVARGS_MAXLEN 4096 -/* Callback used when a new device is found in devargs */ typedef int (parse_cb )(struct rte_eth_dev *dev,const char *params,uint8_t head);
drivers/net/failsafe/failsafe_eal.c +58 -20
97 lines not on this page · raw message
drivers/net/failsafe/failsafe_private.h +2 -0
@@ -58,6 +58,8 @@#define FAILSAFE_MAX_ETHPORTS 2#define FAILSAFE_MAX_ETHADDR 128 +#define DEVARGS_MAXLEN 4096 +/* TYPES */ struct rxq {
Signature · 1 line
1.8.3.1
Re: [PATCH v4 3/8] net/failsafe: add probed etherdev capture
Hi Matan,
Quoted 5 lines · Matan Azrad
Previous fail-safe code didn't support probed sub-devices capture and failed when it tried to probe them.
Skip fail-safe sub-device probing when it already was probed.
What happens when
app --vdev "net_failsafe0,dev(net_failsafe0)" -- -i
? I guess infinite recursion.
Quoted 14 lines
Signed-off-by Matan Azrad <[email protected]>Cc Gaetan Rivet <[email protected]>--- doc/guides/nics/fail_safe.rst | 5 +++ drivers/net/failsafe/failsafe_args.c | 2 - drivers/net/failsafe/failsafe_eal.c | 78 ++++++++++++++++++++++++--------- drivers/net/failsafe/failsafe_private.h | 2 + 4 files changed, 65 insertions(+), 22 deletions(-)doc/guides/nics/fail_safe.rst +0
-0
doc/guides/nics/fail_safe.rst Fail-safe command line parameters · line 122
Quoted 8 lines
doc/guides/nics/fail_safe.rst +5
-0@@ -115,6 +115,11 @@ Fail-safe command line parameters order to take only the last line into account (unlike ``exec()``) at every probe attempt. +.. note:: + + In case of whitelist sub-device probed by EAL, fail-safe PMD will take the device + as is, which means that EAL device options are taken in this case. +
This note should be right under the "dev()" parameter help I think.
If the self-capture is possible and you fix it, you should as well add a line here about the limitation, concerning the PCI blacklist mode and the expected PCI id format?
Something like:
--- 8< ---
When trying to use a PCI device automatically probed in blacklist mode, the syntax for the fail-safe must be with the full PCI id:
Domain:Bus:Device.Function. See the usage example section.
.. ^^^^^^^^^^^^^ Here, an ReST reference
.. Would be nice, I don't recall
.. the exact syntax.
.. In the Usage example section:
#. Start testpmd, automatically probing the device 84:00.0 and using it with
the fail-safe
.. code-block:: console
$RTE_TARGET/build/app/testpmd -c 0xff -n 4 \
--vdev 'net_failsafe0,dev(0000:84:00.0),dev(net_ring0)' \
-- -i--- >8 ---
Ensure that this is working before using this command, I haven't tested it.
Regards,
Signature · 2 lines
Gaëtan Rivet 6WIND
Re: [PATCH v4 3/8] net/failsafe: add probed etherdev capture
Hi Gaetan
From: Gaëtan Rivet, Thursday, January 18, 2018 11:11 AM
Quoted 11 lines
To: Matan Azrad <[email protected]>
Cc: Ferruh Yigit <[email protected]>; Thomas Monjalon <[email protected]>; [email protected]; [email protected]
Subject: Re: [PATCH v4 3/8] net/failsafe: add probed etherdev captureHi Matan,
Quoted 3 lines · Matan Azrad
Previous fail-safe code didn't support probed sub-devices capture and failed when it tried to probe them.
Quoted 9 lines
Quoted 2 lines
Skip fail-safe sub-device probing when it already was probed.
What happens when
app --vdev "net_failsafe0,dev(net_failsafe0)" -- -i
? I guess infinite recursion.
:) interesting
./x86_64-native-linuxapp-gcc/build/app/test-pmd/testpmd -n 4 --vdev="net_failsafe0,dev(net_failsafe0)" --vdev="net_vdev_netvsc,ignore=1" -- --burst=118 --mbcache=512 --portmask 0xf -i --nb-cores=11 --rxq=2 --txq=2 --txd=1024 --rxd=1024
EAL: Detected 12 lcore(s)
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Debug dataplane logs available - lower performance
EAL: Probing VFIO support...
EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !
EAL: PCI device 0002:00:02.0 on NUMA socket 0
EAL: probe driver: 15b3:1004 net_mlx4
PMD: net_mlx4: PCI information matches, using device "mlx4_0" (VF: true)
PMD: net_mlx4: 1 port(s) detected
PMD: net_mlx4: port 1 MAC address is 00:15:5d:44:4b:24
PMD: net_failsafe: Initializing Fail-safe PMD for net_failsafe0
PMD: net_failsafe: Creating fail-safe device on NUMA socket 0
PMD: net_failsafe: Taking control of a probed sub device 0 named net_failsafe0
PMD: net_failsafe: MAC address is 00:00:00:00:00:00
Interactive-mode selected
testpmd: create a new mbuf pool <mbuf_pool_socket_0>: n=327680, size=2176, socket=0
Configuring Port 0 (socket 0)
Port 0: 00:15:5D:44:4B:24
Checking link statuses...
Done
testpmd>
Failsafe0 took control of itself (since it is already probed we don't probe it again).
Quoted 18 lines
Quoted 6 lines
Signed-off-by Matan Azrad <[email protected]>Cc Gaetan Rivet <[email protected]>--- doc/guides/nics/fail_safe.rst | 5 +++ drivers/net/failsafe/failsafe_args.c | 2 - drivers/net/failsafe/failsafe_eal.c | 78 ++++++++++++++++++++++++---------
Quoted 11 lines
drivers/net/failsafe/failsafe_private.h | 2 + 4 files changed, 65 insertions(+), 22 deletions(-)(quoted diff) +0
-0
b/doc/guides/nics/fail_safe.rstindex 5b1b47e..b89e53b 100644doc/guides/nics/fail_safe.rst +0
-0@@ -115,6 +115,11 @@ Fail-safe command line parameters order to take only the last line into account (unlike ``exec()``) at every probe attempt.
Quoted 9 lines
doc/guides/nics/fail_safe.rst Fail-safe command line parameters
Quoted 3 lines
doc/guides/nics/fail_safe.rst +3
-0+.. note:: + + In case of whitelist sub-device probed by EAL, fail-safe PMD will take thedevice
doc/guides/nics/fail_safe.rst Fail-safe command line parameters · line 122
Quoted 2 lines
doc/guides/nics/fail_safe.rst +2
-0+ as is, which means that EAL device options are taken in this case. +This note should be right under the "
dev()" parameter help I think.
OK.
Quoted 20 lines
If the self-capture is possible and you fix it, you should as well add a line here about the limitation, concerning the PCI blacklist mode and the expected PCI id format?
Something like:
--- 8< ---
When trying to use a PCI device automatically probed in blacklist mode, the syntax for the fail-safe must be with the full PCI id:
Domain:Bus:Device.Function. See the usage example section... ^^^^^^^^^^^^^ Here, an ReST reference
.. Would be nice, I don't recall
.. the exact syntax.
.. In theUsage examplesection:#. Start testpmd, automatically probing the device 84:00.0 and using it with
the fail-safe
Quoted 8 lines
.. code-block:: console $RTE_TARGET/build/app/testpmd -c 0xff -n 4 \ --vdev 'net_failsafe0,dev(0000:84:00.0),dev(net_ring0)' \ -- -i--- >8 ---
Ok.
Quoted 2 lines
Ensure that this is working before using this command, I haven't tested it.
Sure.
Quoted 4 lines
Regards,
--
Gaëtan Rivet
6WIND
PATCH v4 4/8 net/vdev_netvsc: introduce Hyper-V platform driver
In reply to Matan AzradThis patch lays the groundwork for this driver (draft documentation, copyright notices, code base skeleton and build system hooks). While it can be successfully compiled and invoked, it's an empty shell at this stage.
--- MAINTAINERS | 6 ++ config/common_base | 5 ++ config/common_linuxapp | 1 + doc/guides/nics/features/vdev_netvsc.ini | 12 +++ doc/guides/nics/index.rst | 1 + doc/guides/nics/vdev_netvsc.rst | 20 +++++ drivers/net/Makefile | 1 + drivers/net/vdev_netvsc/Makefile | 27 ++++++ .../vdev_netvsc/rte_pmd_vdev_netvsc_version.map | 4 + drivers/net/vdev_netvsc/vdev_netvsc.c | 99 ++++++++++++++++++++++ mk/rte.app.mk | 1 + 11 files changed, 177 insertions(+) create mode 100644 doc/guides/nics/features/vdev_netvsc.ini create mode 100644 doc/guides/nics/vdev_netvsc.rst create mode 100644 drivers/net/vdev_netvsc/Makefile create mode 100644 drivers/net/vdev_netvsc/rte_pmd_vdev_netvsc_version.map create mode 100644 drivers/net/vdev_netvsc/vdev_netvsc.c
MAINTAINERS +6 -0
@@ -462,6 +462,12 @@ F: drivers/net/mrvl/ F: doc/guides/nics/mrvl.rst F: doc/guides/nics/features/mrvl.ini +Microsoft vdev-netvsc - EXPERIMENTAL +M: Matan Azrad <[email protected]> +F: drivers/net/vdev-netvsc/ +F: doc/guides/nics/vdev-netvsc.rst +F: doc/guides/nics/features/vdev-netvsc.ini + Netcope szedata2 M: Matej Vido <[email protected]> F: drivers/net/szedata2/
config/common_base +5 -0
@@ -279,6 +279,11 @@ CONFIG_RTE_LIBRTE_NFP_DEBUG_RX=n CONFIG_RTE_LIBRTE_MRVL_PMD=n # +# Compile virtual device driver for NetVSC on Hyper-V/Azure +# +CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD=n + +# # Compile burst-oriented Broadcom BNXT PMD driver # CONFIG_RTE_LIBRTE_BNXT_PMD=y
config/common_linuxapp +1 -0
@@ -47,6 +47,7 @@ CONFIG_RTE_LIBRTE_PMD_VHOST=y CONFIG_RTE_LIBRTE_PMD_AF_PACKET=y CONFIG_RTE_LIBRTE_PMD_TAP=y CONFIG_RTE_LIBRTE_AVP_PMD=y +CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD=y CONFIG_RTE_LIBRTE_NFP_PMD=y CONFIG_RTE_LIBRTE_POWER=y CONFIG_RTE_VIRTIO_USER=y
doc/guides/nics/features/vdev_netvsc.ini new file +12 -0
@@ -0,0 +1,12 @@ +; +; Supported features of the 'vdev_netvsc' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +ARMv7 = Y +ARMv8 = Y +Power8 = Y +x86-32 = Y +x86-64 = Y +Usage doc = Y
doc/guides/nics/index.rst +1 -0
@@ -64,6 +64,7 @@ Network Interface Controller Drivers szedata2 tap thunderx + vdev_netvsc virtio vhost vmxnet3
doc/guides/nics/vdev_netvsc.rst new file +20 -0
@@ -0,0 +1,20 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright 2017 6WIND S.A. + Copyright 2017 Mellanox Technologies, Ltd. + +VDEV_NETVSC driver +================== + +The VDEV_NETVSC driver (librte_pmd_vdev_netvsc) provides support for NetVSC +interfaces and associated SR-IOV virtual function (VF) devices found in +Linux virtual machines running on Microsoft Hyper-V_ (including Azure) +platforms. + +.. _Hyper-V: https://docs.microsoft.com/en-us/windows-hardware/drivers/network/overview-of-hyper-v + +Build options +------------- + +- ``CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD`` (default ``y``) + + Toggle compilation of this driver.
drivers/net/Makefile +1 -0
@@ -39,6 +39,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += sfc DIRS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2) += szedata2 DIRS-$(CONFIG_RTE_LIBRTE_PMD_TAP) += tap DIRS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += thunderx +DIRS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += vdev_netvsc DIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio DIRS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += vmxnet3
drivers/net/vdev_netvsc/Makefile new file +27 -0
@@ -0,0 +1,27 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2017 6WIND S.A. +# Copyright 2017 Mellanox Technologies, Ltd. + +include $(RTE_SDK)/mk/rte.vars.mk + +# Properties of the generated library. +LIB = librte_pmd_vdev_netvsc.a +LIBABIVER := 1 +EXPORT_MAP := rte_pmd_vdev_netvsc_version.map + +# Additional compilation flags. +CFLAGS += -O3 +CFLAGS += -g +CFLAGS += -std=c11 -pedantic -Wall -Wextra +CFLAGS += $(WERROR_FLAGS) + +# Dependencies. +LDLIBS += -lrte_bus_vdev +LDLIBS += -lrte_eal +LDLIBS += -lrte_ethdev +LDLIBS += -lrte_kvargs + +# Source files. +SRCS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += vdev_netvsc.c + +include $(RTE_SDK)/mk/rte.lib.mk
drivers/net/vdev_netvsc/rte_pmd_vdev_netvsc_version.map new file +4 -0
@@ -0,0 +1,4 @@ +DPDK_18.02 { + + local: *; +};
drivers/net/vdev_netvsc/vdev_netvsc.c new file +99 -0
99 lines not on this page · raw message
mk/rte.app.mk +1 -0
@@ -157,6 +157,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += -lrte_pmd_sfc_efx _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2) += -lrte_pmd_szedata2 -lsze2 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_TAP) += -lrte_pmd_tap _LDLIBS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += -lrte_pmd_thunderx_nicvf +_LDLIBS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += -lrte_pmd_vdev_netvsc _LDLIBS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += -lrte_pmd_virtio ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y) _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += -lrte_pmd_vhost
Signature · 1 line
1.8.3.1
PATCH v4 5/8 net/vdev_netvsc: implement core functionality
In reply to Matan AzradAs described in more details in the attached documentation (see patch contents), this virtual device driver manages NetVSC interfaces in virtual machines hosted by Hyper-V/Azure platforms.
This driver does not manage traffic nor Ethernet devices directly; it acts as a thin configuration layer that automatically instantiates and controls fail-safe PMD instances combining tap and PCI sub-devices, so that each NetVSC interface is exposed as a single consolidated port to DPDK applications.
PCI sub-devices being hot-pluggable (e.g. during VM migration), applications automatically benefit from increased throughput when present and automatic fallback on NetVSC otherwise without interruption thanks to fail-safe's hot-plug handling.
Once initialized, the sole job of the vdev_netvsc driver is to regularly scan for PCI devices to associate with NetVSC interfaces and feed their addresses to corresponding fail-safe instances.
--- doc/guides/nics/vdev_netvsc.rst | 70 +++++ drivers/net/vdev_netvsc/Makefile | 4 + drivers/net/vdev_netvsc/vdev_netvsc.c | 550 +++++++++++++++++++++++++++++++++- 3 files changed, 623 insertions(+), 1 deletion(-)
doc/guides/nics/vdev_netvsc.rst +70 -0
@@ -12,9 +12,79 @@ platforms. .. _Hyper-V: https://docs.microsoft.com/en-us/windows-hardware/drivers/network/overview-of-hyper-v +Implementation details +---------------------- + +Each instance of this driver effectively needs to drive two devices: the +NetVSC interface proper and its SR-IOV VF (referred to as "physical" from +this point on) counterpart sharing the same MAC address. + +Physical devices are part of the host system and cannot be maintained during +VM migration. From a VM standpoint they appear as hot-plug devices that come +and go without prior notice. + +When the physical device is present, egress and most of the ingress traffic +flows through it; only multicasts and other hypervisor control still flow +through NetVSC. Otherwise, NetVSC acts as a fallback for all traffic. + +To avoid unnecessary code duplication and ensure maximum performance, +handling of physical devices is left to their original PMDs; this virtual +device driver (also known as *vdev*) manages other PMDs as summarized by the +following block diagram:: + + .------------------. + | DPDK application | + `--------+---------' + | + .------+------. + | DPDK ethdev | + `------+------' Control + | | + .------------+------------. v .--------------------. + | failsafe PMD +---------+ vdev_netvsc driver | + `--+-------------------+--' `--------------------' + | | + | .........|......... + | : | : + .----+----. : .----+----. : + | tap PMD | : | any PMD | : + `----+----' : `----+----' : <-- Hot-pluggable + | : | : + .------+-------. : .-----+-----. : + | NetVSC-based | : | SR-IOV VF | : + | netdevice | : | device | : + `--------------' : `-----------' : + :.................: + + +This driver implementation may be temporary and should be improved or removed +either when hot-plug will be fully supported in EAL and bus drivers or when +a new NetVSC driver will be integrated. + Build options ------------- - ``CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD`` (default ``y``) Toggle compilation of this driver. + +Run-time parameters +------------------- + +To invoke this driver, applications have to explicitly provide the +``--vdev=net_vdev_netvsc`` EAL option. + +The following device parameters are supported: + +- ``iface`` [string] + + Provide a specific NetVSC interface (netdevice) name to attach this driver + to. Can be provided multiple times for additional instances. + +- ``mac`` [string] + + Same as ``iface`` except a suitable NetVSC interface is located using its + MAC address. + +Not specifying either ``iface`` or ``mac`` makes this driver attach itself to +all NetVSC interfaces found on the system.
drivers/net/vdev_netvsc/Makefile +4 -0
@@ -13,6 +13,9 @@ EXPORT_MAP := rte_pmd_vdev_netvsc_version.map CFLAGS += -O3 CFLAGS += -g CFLAGS += -std=c11 -pedantic -Wall -Wextra +CFLAGS += -D_XOPEN_SOURCE=600 +CFLAGS += -D_BSD_SOURCE +CFLAGS += -D_DEFAULT_SOURCE CFLAGS += $(WERROR_FLAGS) # Dependencies.
@@ -20,6 +23,7 @@ LDLIBS += -lrte_bus_vdev LDLIBS += -lrte_eal LDLIBS += -lrte_ethdev LDLIBS += -lrte_kvargs +LDLIBS += -lrte_net # Source files. SRCS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += vdev_netvsc.c
drivers/net/vdev_netvsc/vdev_netvsc.c +549 -1
603 lines not on this page · raw message
Signature · 1 line
1.8.3.1
Re: [PATCH v4 5/8] net/vdev_netvsc: implement core functionality
Quoted 21 lines · On Thu, 18 Jan 2018 08:43:43 +0000 Matan Azrad <[email protected]> wrote:
drivers/net/vdev_netvsc/vdev_netvsc.c +21
-0+ +/** +* Alarm callback that regularly probes system network interfaces. +* +* This callback runs at a frequency determined by VDEV_NETVSC_PROBE_MS as +* long as an vdev_netvsc context instance exists. +* +* @param arg +* Ignored. +*/ +static void +vdev_netvsc_alarm(__rte_unused void *arg) +{ +struct vdev_netvsc_ctx *ctx; +int ret; + + LIST_FOREACH(ctx, &vdev_netvsc_ctx_list, entry) { + ret =vdev_netvsc_foreach_iface (vdev_netvsc_device_probe, ctx); +if (ret) +break ; + }
drivers/net/vdev_netvsc/vdev_netvsc.c line 360
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+if (!vdev_netvsc_ctx_count) +return ; + ret =rte_eal_alarm_set (VDEV_NETVSC_PROBE_MS *1000 , + vdev_netvsc_alarm,NULL ); +if (ret <0 ) { +DRV_LOG (ERR ,"unable to reschedule alarm callback: %s" , +rte_strerror (-ret)); + } +} +
Not a fan of polling for network interface changes. Alarms in core code make life difficult for applications.
Also, at least on current Azure infrastructure hotplug of netvsc devices is not supported.
Can we just wait until proper hotplug API from kernel (ie read netlink uevent) is done?
Re: [PATCH v4 5/8] net/vdev_netvsc: implement core functionality
From: Stephen Hemminger, Thursday, January 18, 2018 8:26 PM
Quoted 22 lines
Quoted 8 lines · On Thu, 18 Jan 2018 08:43:43 +0000 Matan Azrad <[email protected]> wrote:
drivers/net/vdev_netvsc/vdev_netvsc.c +8
-0+ +/** +* Alarm callback that regularly probes system network interfaces. +* +* This callback runs at a frequency determined by +VDEV_NETVSC_PROBE_MS as +* long as an vdev_netvsc context instance exists. +* drivers/net/vdev_netvsc/vdev_netvsc.c
Quoted 10 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +10
-0+* @param arg +* Ignored. +*/ +static void +vdev_netvsc_alarm(__rte_unused void *arg) { +struct vdev_netvsc_ctx *ctx; +int ret; + + LIST_FOREACH(ctx, &vdev_netvsc_ctx_list, entry) { + ret =
vdev_netvsc_foreach_iface(vdev_netvsc_device_probe, ctx);
Quoted 17 lines
drivers/net/vdev_netvsc/vdev_netvsc.c line 360
Quoted 13 lines
drivers/net/vdev_netvsc/vdev_netvsc.c +13
-0+if (ret) +break ; + } +if (!vdev_netvsc_ctx_count) +return ; + ret =rte_eal_alarm_set (VDEV_NETVSC_PROBE_MS *1000 , + vdev_netvsc_alarm,NULL ); +if (ret <0 ) { +DRV_LOG (ERR ,"unable to reschedule alarm callback: %s" , +rte_strerror (-ret)); + } +} +Not a fan of polling for network interface changes. Alarms in core code make life difficult for applications.
What is the connection to application? It is netvsc driver alarm.
Quoted 3 lines
Also, at least on current Azure infrastructure hotplug of netvsc devices is not supported.
It detects the PCI device hotplug, no netvsc device.
Quoted 2 lines
Can we just wait until proper hotplug API from kernel (ie read netlink uevent) is done?
Why?
PATCH v4 6/8 net/vdev_netvsc: skip routed netvsc probing
In reply to Matan AzradNetVSC netdevices which are already routed should not be probed because they are used for management purposes by the HyperV.
prevent routed netvsc devices probing.
--- doc/guides/nics/vdev_netvsc.rst | 2 +- drivers/net/vdev_netvsc/vdev_netvsc.c | 46 +++++++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+), 1 deletion(-)
doc/guides/nics/vdev_netvsc.rst +1 -1
@@ -87,4 +87,4 @@ The following device parameters are supported: MAC address. Not specifying either ``iface`` or ``mac`` makes this driver attach itself to -all NetVSC interfaces found on the system. +all unrouted NetVSC interfaces found on the system.
drivers/net/vdev_netvsc/vdev_netvsc.c +46 -0
64 lines not on this page · raw message
Signature · 1 line
1.8.3.1
Re: [PATCH v4 6/8] net/vdev_netvsc: skip routed netvsc probing
Quoted 7 lines · On Thu, 18 Jan 2018 08:43:44 +0000 Matan Azrad <[email protected]> wrote:
NetVSC netdevices which are already routed should not be probed because they are used for management purposes by the HyperV.
prevent routed netvsc devices probing.
Signed-off-by Raslan Darawsheh <[email protected]>Signed-off-by Matan Azrad <[email protected]>
Just checking for interface IPv4 or IPv6 (non-link local) is enough. If device has a L3 address than skip it.
No need to read route table which maybe huge in some environments.
Re: [PATCH v4 6/8] net/vdev_netvsc: skip routed netvsc probing
18/01/2018 19:26, Stephen Hemminger:
Quoted 8 lines
Quoted 5 lines · On Thu, 18 Jan 2018 08:43:44 +0000 Matan Azrad <[email protected]> wrote:
NetVSC netdevices which are already routed should not be probed because they are used for management purposes by the HyperV.
prevent routed netvsc devices probing.
Quoted 7 lines
Quoted 2 lines
Signed-off-by Raslan Darawsheh <[email protected]>Signed-off-by Matan Azrad <[email protected]>Just checking for interface IPv4 or IPv6 (non-link local) is enough. If device has a L3 address than skip it.
No need to read route table which maybe huge in some environments.
Stephen, I think you are in a better position to do this improvement. Can we accept this patch, so you can send a patch on top of it? Such PMD improvement may be integrated in RC2.
PATCH v4 7/8 net/vdev_netvsc: add "force" parameter
In reply to Matan AzradThis parameter allows specifying any non-NetVSC interface or routed NetVSC interfaces to use with tap sub-devices for development purposes.
--- doc/guides/nics/vdev_netvsc.rst | 5 +++++ drivers/net/vdev_netvsc/vdev_netvsc.c | 30 +++++++++++++++++++----------- 2 files changed, 24 insertions(+), 11 deletions(-)
doc/guides/nics/vdev_netvsc.rst +5 -0
@@ -86,5 +86,10 @@ The following device parameters are supported: Same as ``iface`` except a suitable NetVSC interface is located using its MAC address. +- ``force`` [int] + + If nonzero, forces the use of specified interfaces even if not detected as + NetVSC or detected as routed NETVSC. + Not specifying either ``iface`` or ``mac`` makes this driver attach itself to all unrouted NetVSC interfaces found on the system.
drivers/net/vdev_netvsc/vdev_netvsc.c +19 -11
@@ -36,6 +36,7 @@#define VDEV_NETVSC_DRIVER net_vdev_netvsc#define VDEV_NETVSC_ARG_IFACE "iface"#define VDEV_NETVSC_ARG_MAC "mac" +#define VDEV_NETVSC_ARG_FORCE "force"#define VDEV_NETVSC_PROBE_MS 1000#define NETVSC_CLASS_ID "{f8615163-df3e-46c5-913f-f2d2f965ed0e}"
@@ -419,6 +420,9 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =* - struct rte_kvargs *kvargs: * Device arguments provided to current driver instance. * +* - int force: +* Accept specified interface even if not detected as NetVSC. +* * - unsigned int specified: * Number of specific netdevices provided as device arguments. *
@@ -436,6 +440,7 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list = { const char *name = va_arg(ap, const char *); struct rte_kvargs *kvargs = va_arg(ap, struct rte_kvargs *); +int force =va_arg (ap, int); unsigned int specified = va_arg(ap, unsigned int); unsigned int *matched = va_arg(ap, unsigned int *);unsigned int i;
@@ -490,20 +495,18 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =return 0 ; }if (!vdev_netvsc_iface_is_netvsc (iface)) { -if (!specified) +if (!specified || !force)return 0 ;DRV_LOG (WARNING , -"interface \"%s\" (index %u) is not NetVSC," -" skipping" , +"using non-NetVSC interface \"%s\" (index %u)" , iface->if_name, iface->if_index); -return 0 ; }/* Routed NetVSC should not be probed. */ if (vdev_netvsc_has_route (iface->if_name)) { -DRV_LOG (WARNING ,"NetVSC interface \"%s\" (index %u) is routed" , - iface->if_name, iface->if_index); -if (!specified) +if (!specified || !force)return 0 ; +DRV_LOG (WARNING ,"using routed NetVSC interface \"%s\"" +" (index %u)" , iface->if_name, iface->if_index); }/* Create interface context. */ ctx =calloc (1 ,sizeof (*ctx));
@@ -597,6 +600,7 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =static const char *const vdev_netvsc_arg[] = {VDEV_NETVSC_ARG_IFACE ,VDEV_NETVSC_ARG_MAC , +VDEV_NETVSC_ARG_FORCE ,NULL , };const char *name =rte_vdev_device_name (dev);
@@ -605,6 +609,7 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list = vdev_netvsc_arg);unsigned int specified =0 ;unsigned int matched =0 ; +int force =0 ;unsigned int i;int ret;
@@ -616,14 +621,16 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =for (i =0 ; i != kvargs->count; ++i) {const struct rte_kvargs_pair *pair = &kvargs->pairs[i]; -if (!strcmp (pair->key,VDEV_NETVSC_ARG_IFACE ) || - !strcmp (pair->key,VDEV_NETVSC_ARG_MAC )) +if (!strcmp (pair->key,VDEV_NETVSC_ARG_FORCE )) + force = !!atoi (pair->value); +else if (!strcmp (pair->key,VDEV_NETVSC_ARG_IFACE ) || + !strcmp (pair->key,VDEV_NETVSC_ARG_MAC )) ++specified; }rte_eal_alarm_cancel (vdev_netvsc_alarm,NULL );/* Gather interfaces. */ ret =vdev_netvsc_foreach_iface (vdev_netvsc_netvsc_probe, name, kvargs, - specified, &matched); + force, specified, &matched);if (ret <0 ) goto error;if (matched < specified)
@@ -682,7 +689,8 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =RTE_PMD_REGISTER_ALIAS (VDEV_NETVSC_DRIVER , eth_vdev_netvsc);RTE_PMD_REGISTER_PARAM_STRING (net_vdev_netvsc,VDEV_NETVSC_ARG_IFACE "=<string> " -VDEV_NETVSC_ARG_MAC "=<string>" ); +VDEV_NETVSC_ARG_MAC "=<string> " +VDEV_NETVSC_ARG_FORCE "=<int>" );/** Initialize driver log type. */ RTE_INIT (vdev_netvsc_init_log)
Signature · 1 line
1.8.3.1
Re: [PATCH v4 7/8] net/vdev_netvsc: add "force" parameter
Quoted 5 lines · On Thu, 18 Jan 2018 08:43:45 +0000 Matan Azrad <[email protected]> wrote:
This parameter allows specifying any non-NetVSC interface or routed NetVSC interfaces to use with tap sub-devices for development purposes.
Signed-off-by Adrien Mazarguil <[email protected]>Signed-off-by Matan Azrad <[email protected]>
Might whitelist work for this?
Re: [PATCH v4 7/8] net/vdev_netvsc: add "force" parameter
From: Stephen Hemminger, Thursday, January 18, 2018 8:28 PM
Quoted 10 lines
Quoted 5 lines · On Thu, 18 Jan 2018 08:43:45 +0000 Matan Azrad <[email protected]> wrote:
This parameter allows specifying any non-NetVSC interface or routed NetVSC interfaces to use with tap sub-devices for development purposes.
Signed-off-by Adrien Mazarguil <[email protected]>Signed-off-by Matan Azrad <[email protected]>Might whitelist work for this?
It is an optional parameter, you don't need to configure any parameter for this driver running.
PATCH v4 8/8 net/vdev_netvsc: add automatic probing
In reply to Matan AzradUsing DPDK in Hyper-V VM systems requires vdev_netvsc driver to pair the NetVSC netdev device with the same MAC address PCI device by fail-safe PMD.
Add vdev_netvsc custom scan in vdev bus to allow automatic probing in Hyper-V VM systems unless it was already specified by command line.
Add "ignore" parameter to disable this auto-detection.
--- doc/guides/nics/vdev_netvsc.rst | 9 ++++-- drivers/net/vdev_netvsc/vdev_netvsc.c | 55 +++++++++++++++++++++++++++++++++-- 2 files changed, 60 insertions(+), 4 deletions(-)
doc/guides/nics/vdev_netvsc.rst +7 -2
@@ -71,8 +71,8 @@ Build options Run-time parameters ------------------- -To invoke this driver, applications have to explicitly provide the -``--vdev=net_vdev_netvsc`` EAL option. +This driver is invoked automatically in Hyper-V VM systems unless the user +invoked it by command line using ``--vdev=net_vdev_netvsc`` EAL option. The following device parameters are supported:
@@ -91,5 +91,10 @@ The following device parameters are supported: If nonzero, forces the use of specified interfaces even if not detected as NetVSC or detected as routed NETVSC. +- ``ignore`` [int] + + If nonzero, ignores the driver runnig (actually used to disable the + auto-detection in Hyper-V VM). + Not specifying either ``iface`` or ``mac`` makes this driver attach itself to all unrouted NetVSC interfaces found on the system.
drivers/net/vdev_netvsc/vdev_netvsc.c +53 -2
105 lines not on this page · raw message
Signature · 1 line
1.8.3.1
v5 cover Introduce virtual driver for Hyper-V/Azure platforms
In reply to Matan AzradVirtual machines hosted by Hyper-V/Azure platforms are fitted with simplified virtual network devices named NetVSC that are used for fast communication between VM to VM, VM to hypervisor, and the outside.
They appear as standard system netdevices to user-land applications, the main difference being they are implemented on top of VMBUS instead of emulated PCI devices.
While this reads like a case for a standard DPDK PMD, there is more to it.
To accelerate outside communication, NetVSC devices as they appear in a VM can be paired with physical SR-IOV virtual function (VF) devices owned by that same VM. Both netdevices share the same MAC address in that case.
When paired, egress and most of the ingress traffic flow through the VF device, while part of it (e.g. multicasts, hypervisor control data) still flows through NetVSC. Moreover VF devices are not retained and disappear during VM migration; from a VM standpoint, they can be hot-plugged anytime with NetVSC acting as a fallback.
Running DPDK applications in such a context involves driving VF devices using their dedicated PMDs in a vendor-independent fashion (to benefit from maximum performance without writing dedicated code) while simultaneously listening to NetVSC and handling the related hot-plug events.
This new virtual driver (referred to as "vdev_netvsc" from this point on) automatically coordinates the Hyper-V/Azure-specific management part described above by relying on vendor-specific, failsafe and tap PMDs to expose a single consolidated Ethernet device usable directly by existing applications.
.------------------.
| DPDK application |
`--------+---------'
|
.------+------.
| DPDK ethdev |
`------+------' Control
| |
.------------+------------. v .--------------------.
| failsafe PMD +---------+ vdev_netvsc driver |
`--+-------------------+--' `--------------------'
| |
| .........|.........
| : | :
.----+----. : .----+----. :
| tap PMD | : | any PMD | :
`----+----' : `----+----' : <-- Hot-pluggable
| : | :
.------+-------. : .-----+-----. :
| NetVSC-based | : | SR-IOV VF | :
| netdevice | : | device | :
`--------------' : `-----------' :
:.................:v2 changes(Adrien):
- Renamed driver from "hyperv" to "vdev_netvsc". This change covers documentation and symbols prefix.
- Driver is now tagged EXPERIMENTAL.
- Replaced ether_addr_from_str() with a basic sscanf() call.
- Removed debugging code (memset() poisoning).
- Fixed hyperv_iface_is_netvsc()'s buffer allocation according to comments.
- Removed hyperv_basename().
- Discarded unused variables through __rte_unused.
- Added separate but necessary free() bugfix for failsafe PMD.
- Added file descriptor input support to failsafe PMD.
- Replaced temporary bash execution; failsafe now reads device definitions directly through a pipe without an intermediate bash one-liner.
- Expanded DEBUG/INFO/WARN/ERROR() macros as PMD_DRV_LOG().
- Added dynamic log type (pmd.vdev_netvsc).
- Modified initialization code to probe devices immediately during startup.
- Fixed several snprintf() return value checks ("ret >= sizeof(foo)" is more appropriate than "ret >= sizeof(foo) - 1").
v3 changes(Matan):
- Fixed clang compilation in V2.
- Removed hotplug remove code from the new driver.
- Supported probed sub-devices getting in fail-safe.
- Added automatic probing for HyperV VM systems.
- Added option to ignore the automatic probing.
- Skiped routed NetVSC devices probing.
- Adjusted documentation and semantics.
- Replaced maintainer.
v4 changes(Matan):
- Align descriptions of context struct(Stephen suggestion).
- Skip non-ethernet devices in netdev loop(Stephen suggestion).
- Use different variable names in "add fd parameter"(Gaetan suggestion).
- Change name of get port id function in "add automatic probing"(Gaetan suggestion).
- Update internal fail-safe devargs in case of probed device(Gaetan suggestion).
- use deferent commit title instead of "support probed sub-devices getting"(Gaetan suggestion).
v5 changes(Matan):
- Improve fail-safe documentation as Gaetan suggested.
- Fix fcntl paramenter.
Adrien Mazarguil (1):
net/failsafe: fix invalid free
Matan Azrad (7):
net/failsafe: add "fd" parameter net/failsafe: add probed etherdev capture net/vdev_netvsc: introduce Hyper-V platform driver net/vdev_netvsc: implement core functionality net/vdev_netvsc: skip routed netvsc probing net/vdev_netvsc: add "force" parameter net/vdev_netvsc: add automatic probing
MAINTAINERS | 6 + config/common_base | 5 + config/common_linuxapp | 1 + doc/guides/nics/fail_safe.rst | 26 + doc/guides/nics/features/vdev_netvsc.ini | 12 + doc/guides/nics/index.rst | 1 + doc/guides/nics/vdev_netvsc.rst | 100 +++ drivers/net/Makefile | 1 + drivers/net/failsafe/failsafe_args.c | 84 ++- drivers/net/failsafe/failsafe_eal.c | 78 ++- drivers/net/failsafe/failsafe_private.h | 5 + drivers/net/vdev_netvsc/Makefile | 31 + .../vdev_netvsc/rte_pmd_vdev_netvsc_version.map | 4 + drivers/net/vdev_netvsc/vdev_netvsc.c | 752 +++++++++++++++++++++ mk/rte.app.mk | 1 + 15 files changed, 1083 insertions(+), 24 deletions(-) create mode 100644 doc/guides/nics/features/vdev_netvsc.ini create mode 100644 doc/guides/nics/vdev_netvsc.rst create mode 100644 drivers/net/vdev_netvsc/Makefile create mode 100644 drivers/net/vdev_netvsc/rte_pmd_vdev_netvsc_version.map create mode 100644 drivers/net/vdev_netvsc/vdev_netvsc.c
Signature · 1 line
1.8.3.1
PATCH v5 1/8 net/failsafe: fix invalid free
From: Adrien Mazarguil <[email protected]>
rte_free() is not supposed to work with pointers returned by calloc().
--- drivers/net/failsafe/failsafe_args.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/failsafe/failsafe_args.c +1 -1
@@ -407,7 +407,7 @@ typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params,uint8_t i; FOREACH_SUBDEV(sdev, i, dev) { -rte_free (sdev->cmdline); +free (sdev->cmdline); sdev->cmdline =NULL ;free (sdev->devargs.args); sdev->devargs.args =NULL ;
Signature · 1 line
1.8.3.1
PATCH v5 2/8 net/failsafe: add "fd" parameter
In reply to Matan AzradThis parameter enables applications to provide device definitions through an arbitrary file descriptor number.
--- doc/guides/nics/fail_safe.rst | 9 ++++ drivers/net/failsafe/failsafe_args.c | 80 ++++++++++++++++++++++++++++++++- drivers/net/failsafe/failsafe_private.h | 3 ++ 3 files changed, 91 insertions(+), 1 deletion(-)
doc/guides/nics/fail_safe.rst +9 -0
@@ -106,6 +106,15 @@ Fail-safe command line parameters All commas within the ``shell command`` are replaced by spaces before executing the command. This helps using scripts to specify devices. +- **fd(<file descriptor number>)** parameter + + This parameter reads a device definition from an arbitrary file descriptor + number in ``<iface>`` format as described above. + + The file descriptor is read in non-blocking mode and is never closed in + order to take only the last line into account (unlike ``exec()``) at every + probe attempt. + - **mac** parameter [MAC address] This parameter allows the user to set a default MAC address to the fail-safe
drivers/net/failsafe/failsafe_args.c +79 -1
117 lines not on this page · raw message
drivers/net/failsafe/failsafe_private.h +3 -0
@@ -48,6 +48,7 @@#define PMD_FAILSAFE_PARAM_STRING \ "dev(<ifc>)," \ "exec(<shell command>)," \ + "fd(<fd number>)," \ "mac=mac_addr," \ "hotplug_poll=u64" \ ""
@@ -112,6 +113,8 @@ struct sub_device {struct fs_stats stats_snapshot;/* Some device are defined as a command line */ char *cmdline; +/* Others are retrieved through a file descriptor */ +char *fd_str;/* fail-safe device backreference */ struct rte_eth_dev *fs_dev;/* flag calling for recollection */
Signature · 1 line
1.8.3.1
PATCH v5 3/8 net/failsafe: add probed etherdev capture
In reply to Matan AzradPrevious fail-safe code didn't support probed sub-devices capture and failed when it tried to probe them.
Skip fail-safe sub-device probing when it already was probed.
--- doc/guides/nics/fail_safe.rst | 17 +++++++ drivers/net/failsafe/failsafe_args.c | 2 - drivers/net/failsafe/failsafe_eal.c | 78 ++++++++++++++++++++++++--------- drivers/net/failsafe/failsafe_private.h | 2 + 4 files changed, 77 insertions(+), 22 deletions(-)
doc/guides/nics/fail_safe.rst +17 -0
@@ -93,6 +93,14 @@ Fail-safe command line parameters additional sub-device parameters if need be. They will be passed on to the sub-device. +.. note:: + + In case of whitelist sub-device probed by EAL, fail-safe PMD will take the device + as is, which means that EAL device options are taken in this case. + When trying to use a PCI device automatically probed in blacklist mode, + the syntax for the fail-safe must be with the full PCI id: + Domain:Bus:Device.Function. See the usage example section. + - **exec(<shell command>)** parameter This parameter allows the user to provide a command to the fail-safe PMD to
@@ -169,6 +177,15 @@ This section shows some example of using **testpmd** with a fail-safe PMD. $RTE_TARGET/build/app/testpmd -c 0xff -n 4 --no-pci \ --vdev='net_failsafe0,exec(echo 84:00.0)' -- -i +#. Start testpmd, automatically probing the device 84:00.0 and using it with + the fail-safe. + + .. code-block:: console + + $RTE_TARGET/build/app/testpmd -c 0xff -n 4 \ + --vdev 'net_failsafe0,dev(0000:84:00.0),dev(net_ring0)' -- -i + + Using the Fail-safe PMD from an application -------------------------------------------
drivers/net/failsafe/failsafe_args.c +0 -2
@@ -45,8 +45,6 @@#include "failsafe_private.h" -#define DEVARGS_MAXLEN 4096 -/* Callback used when a new device is found in devargs */ typedef int (parse_cb )(struct rte_eth_dev *dev,const char *params,uint8_t head);
drivers/net/failsafe/failsafe_eal.c +58 -20
97 lines not on this page · raw message
drivers/net/failsafe/failsafe_private.h +2 -0
@@ -58,6 +58,8 @@#define FAILSAFE_MAX_ETHPORTS 2#define FAILSAFE_MAX_ETHADDR 128 +#define DEVARGS_MAXLEN 4096 +/* TYPES */ struct rxq {
Signature · 1 line
1.8.3.1
Re: [PATCH v5 3/8] net/failsafe: add probed etherdev capture
Quoted 7 lines · Matan Azrad
Previous fail-safe code didn't support probed sub-devices capture and failed when it tried to probe them.
Skip fail-safe sub-device probing when it already was probed.
Signed-off-by Matan Azrad <[email protected]>Cc Gaetan Rivet <[email protected]>
Okay, ignoring the recursive probing. It could be dangerous, with the ownership evolutions and unforeseen side-effects, but device matching will be reworked next release, so this new functionality will be fixed anyway at this point.
Signature · 2 lines
Gaëtan Rivet 6WIND
PATCH v5 4/8 net/vdev_netvsc: introduce Hyper-V platform driver
In reply to Matan AzradThis patch lays the groundwork for this driver (draft documentation, copyright notices, code base skeleton and build system hooks). While it can be successfully compiled and invoked, it's an empty shell at this stage.
--- MAINTAINERS | 6 ++ config/common_base | 5 ++ config/common_linuxapp | 1 + doc/guides/nics/features/vdev_netvsc.ini | 12 +++ doc/guides/nics/index.rst | 1 + doc/guides/nics/vdev_netvsc.rst | 20 +++++ drivers/net/Makefile | 1 + drivers/net/vdev_netvsc/Makefile | 27 ++++++ .../vdev_netvsc/rte_pmd_vdev_netvsc_version.map | 4 + drivers/net/vdev_netvsc/vdev_netvsc.c | 99 ++++++++++++++++++++++ mk/rte.app.mk | 1 + 11 files changed, 177 insertions(+) create mode 100644 doc/guides/nics/features/vdev_netvsc.ini create mode 100644 doc/guides/nics/vdev_netvsc.rst create mode 100644 drivers/net/vdev_netvsc/Makefile create mode 100644 drivers/net/vdev_netvsc/rte_pmd_vdev_netvsc_version.map create mode 100644 drivers/net/vdev_netvsc/vdev_netvsc.c
MAINTAINERS +6 -0
@@ -462,6 +462,12 @@ F: drivers/net/mrvl/ F: doc/guides/nics/mrvl.rst F: doc/guides/nics/features/mrvl.ini +Microsoft vdev-netvsc - EXPERIMENTAL +M: Matan Azrad <[email protected]> +F: drivers/net/vdev-netvsc/ +F: doc/guides/nics/vdev-netvsc.rst +F: doc/guides/nics/features/vdev-netvsc.ini + Netcope szedata2 M: Matej Vido <[email protected]> F: drivers/net/szedata2/
config/common_base +5 -0
@@ -279,6 +279,11 @@ CONFIG_RTE_LIBRTE_NFP_DEBUG_RX=n CONFIG_RTE_LIBRTE_MRVL_PMD=n # +# Compile virtual device driver for NetVSC on Hyper-V/Azure +# +CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD=n + +# # Compile burst-oriented Broadcom BNXT PMD driver # CONFIG_RTE_LIBRTE_BNXT_PMD=y
config/common_linuxapp +1 -0
@@ -47,6 +47,7 @@ CONFIG_RTE_LIBRTE_PMD_VHOST=y CONFIG_RTE_LIBRTE_PMD_AF_PACKET=y CONFIG_RTE_LIBRTE_PMD_TAP=y CONFIG_RTE_LIBRTE_AVP_PMD=y +CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD=y CONFIG_RTE_LIBRTE_NFP_PMD=y CONFIG_RTE_LIBRTE_POWER=y CONFIG_RTE_VIRTIO_USER=y
doc/guides/nics/features/vdev_netvsc.ini new file +12 -0
@@ -0,0 +1,12 @@ +; +; Supported features of the 'vdev_netvsc' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +ARMv7 = Y +ARMv8 = Y +Power8 = Y +x86-32 = Y +x86-64 = Y +Usage doc = Y
doc/guides/nics/index.rst +1 -0
@@ -64,6 +64,7 @@ Network Interface Controller Drivers szedata2 tap thunderx + vdev_netvsc virtio vhost vmxnet3
doc/guides/nics/vdev_netvsc.rst new file +20 -0
@@ -0,0 +1,20 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright 2017 6WIND S.A. + Copyright 2017 Mellanox Technologies, Ltd. + +VDEV_NETVSC driver +================== + +The VDEV_NETVSC driver (librte_pmd_vdev_netvsc) provides support for NetVSC +interfaces and associated SR-IOV virtual function (VF) devices found in +Linux virtual machines running on Microsoft Hyper-V_ (including Azure) +platforms. + +.. _Hyper-V: https://docs.microsoft.com/en-us/windows-hardware/drivers/network/overview-of-hyper-v + +Build options +------------- + +- ``CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD`` (default ``y``) + + Toggle compilation of this driver.
drivers/net/Makefile +1 -0
@@ -39,6 +39,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += sfc DIRS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2) += szedata2 DIRS-$(CONFIG_RTE_LIBRTE_PMD_TAP) += tap DIRS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += thunderx +DIRS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += vdev_netvsc DIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio DIRS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += vmxnet3
drivers/net/vdev_netvsc/Makefile new file +27 -0
@@ -0,0 +1,27 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2017 6WIND S.A. +# Copyright 2017 Mellanox Technologies, Ltd. + +include $(RTE_SDK)/mk/rte.vars.mk + +# Properties of the generated library. +LIB = librte_pmd_vdev_netvsc.a +LIBABIVER := 1 +EXPORT_MAP := rte_pmd_vdev_netvsc_version.map + +# Additional compilation flags. +CFLAGS += -O3 +CFLAGS += -g +CFLAGS += -std=c11 -pedantic -Wall -Wextra +CFLAGS += $(WERROR_FLAGS) + +# Dependencies. +LDLIBS += -lrte_bus_vdev +LDLIBS += -lrte_eal +LDLIBS += -lrte_ethdev +LDLIBS += -lrte_kvargs + +# Source files. +SRCS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += vdev_netvsc.c + +include $(RTE_SDK)/mk/rte.lib.mk
drivers/net/vdev_netvsc/rte_pmd_vdev_netvsc_version.map new file +4 -0
@@ -0,0 +1,4 @@ +DPDK_18.02 { + + local: *; +};
drivers/net/vdev_netvsc/vdev_netvsc.c new file +99 -0
99 lines not on this page · raw message
mk/rte.app.mk +1 -0
@@ -157,6 +157,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += -lrte_pmd_sfc_efx _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2) += -lrte_pmd_szedata2 -lsze2 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_TAP) += -lrte_pmd_tap _LDLIBS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += -lrte_pmd_thunderx_nicvf +_LDLIBS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += -lrte_pmd_vdev_netvsc _LDLIBS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += -lrte_pmd_virtio ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y) _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += -lrte_pmd_vhost
Signature · 1 line
1.8.3.1
PATCH v5 5/8 net/vdev_netvsc: implement core functionality
In reply to Matan AzradAs described in more details in the attached documentation (see patch contents), this virtual device driver manages NetVSC interfaces in virtual machines hosted by Hyper-V/Azure platforms.
This driver does not manage traffic nor Ethernet devices directly; it acts as a thin configuration layer that automatically instantiates and controls fail-safe PMD instances combining tap and PCI sub-devices, so that each NetVSC interface is exposed as a single consolidated port to DPDK applications.
PCI sub-devices being hot-pluggable (e.g. during VM migration), applications automatically benefit from increased throughput when present and automatic fallback on NetVSC otherwise without interruption thanks to fail-safe's hot-plug handling.
Once initialized, the sole job of the vdev_netvsc driver is to regularly scan for PCI devices to associate with NetVSC interfaces and feed their addresses to corresponding fail-safe instances.
--- doc/guides/nics/vdev_netvsc.rst | 70 +++++ drivers/net/vdev_netvsc/Makefile | 4 + drivers/net/vdev_netvsc/vdev_netvsc.c | 550 +++++++++++++++++++++++++++++++++- 3 files changed, 623 insertions(+), 1 deletion(-)
doc/guides/nics/vdev_netvsc.rst +70 -0
@@ -12,9 +12,79 @@ platforms. .. _Hyper-V: https://docs.microsoft.com/en-us/windows-hardware/drivers/network/overview-of-hyper-v +Implementation details +---------------------- + +Each instance of this driver effectively needs to drive two devices: the +NetVSC interface proper and its SR-IOV VF (referred to as "physical" from +this point on) counterpart sharing the same MAC address. + +Physical devices are part of the host system and cannot be maintained during +VM migration. From a VM standpoint they appear as hot-plug devices that come +and go without prior notice. + +When the physical device is present, egress and most of the ingress traffic +flows through it; only multicasts and other hypervisor control still flow +through NetVSC. Otherwise, NetVSC acts as a fallback for all traffic. + +To avoid unnecessary code duplication and ensure maximum performance, +handling of physical devices is left to their original PMDs; this virtual +device driver (also known as *vdev*) manages other PMDs as summarized by the +following block diagram:: + + .------------------. + | DPDK application | + `--------+---------' + | + .------+------. + | DPDK ethdev | + `------+------' Control + | | + .------------+------------. v .--------------------. + | failsafe PMD +---------+ vdev_netvsc driver | + `--+-------------------+--' `--------------------' + | | + | .........|......... + | : | : + .----+----. : .----+----. : + | tap PMD | : | any PMD | : + `----+----' : `----+----' : <-- Hot-pluggable + | : | : + .------+-------. : .-----+-----. : + | NetVSC-based | : | SR-IOV VF | : + | netdevice | : | device | : + `--------------' : `-----------' : + :.................: + + +This driver implementation may be temporary and should be improved or removed +either when hot-plug will be fully supported in EAL and bus drivers or when +a new NetVSC driver will be integrated. + Build options ------------- - ``CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD`` (default ``y``) Toggle compilation of this driver. + +Run-time parameters +------------------- + +To invoke this driver, applications have to explicitly provide the +``--vdev=net_vdev_netvsc`` EAL option. + +The following device parameters are supported: + +- ``iface`` [string] + + Provide a specific NetVSC interface (netdevice) name to attach this driver + to. Can be provided multiple times for additional instances. + +- ``mac`` [string] + + Same as ``iface`` except a suitable NetVSC interface is located using its + MAC address. + +Not specifying either ``iface`` or ``mac`` makes this driver attach itself to +all NetVSC interfaces found on the system.
drivers/net/vdev_netvsc/Makefile +4 -0
@@ -13,6 +13,9 @@ EXPORT_MAP := rte_pmd_vdev_netvsc_version.map CFLAGS += -O3 CFLAGS += -g CFLAGS += -std=c11 -pedantic -Wall -Wextra +CFLAGS += -D_XOPEN_SOURCE=600 +CFLAGS += -D_BSD_SOURCE +CFLAGS += -D_DEFAULT_SOURCE CFLAGS += $(WERROR_FLAGS) # Dependencies.
@@ -20,6 +23,7 @@ LDLIBS += -lrte_bus_vdev LDLIBS += -lrte_eal LDLIBS += -lrte_ethdev LDLIBS += -lrte_kvargs +LDLIBS += -lrte_net # Source files. SRCS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += vdev_netvsc.c
drivers/net/vdev_netvsc/vdev_netvsc.c +549 -1
603 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH v5 6/8 net/vdev_netvsc: skip routed netvsc probing
In reply to Matan AzradNetVSC netdevices which are already routed should not be probed because they are used for management purposes by the HyperV.
prevent routed netvsc devices probing.
--- doc/guides/nics/vdev_netvsc.rst | 2 +- drivers/net/vdev_netvsc/vdev_netvsc.c | 46 +++++++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+), 1 deletion(-)
doc/guides/nics/vdev_netvsc.rst +1 -1
@@ -87,4 +87,4 @@ The following device parameters are supported: MAC address. Not specifying either ``iface`` or ``mac`` makes this driver attach itself to -all NetVSC interfaces found on the system. +all unrouted NetVSC interfaces found on the system.
drivers/net/vdev_netvsc/vdev_netvsc.c +46 -0
64 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH v5 7/8 net/vdev_netvsc: add "force" parameter
In reply to Matan AzradThis parameter allows specifying any non-NetVSC interface or routed NetVSC interfaces to use with tap sub-devices for development purposes.
--- doc/guides/nics/vdev_netvsc.rst | 5 +++++ drivers/net/vdev_netvsc/vdev_netvsc.c | 30 +++++++++++++++++++----------- 2 files changed, 24 insertions(+), 11 deletions(-)
doc/guides/nics/vdev_netvsc.rst +5 -0
@@ -86,5 +86,10 @@ The following device parameters are supported: Same as ``iface`` except a suitable NetVSC interface is located using its MAC address. +- ``force`` [int] + + If nonzero, forces the use of specified interfaces even if not detected as + NetVSC or detected as routed NETVSC. + Not specifying either ``iface`` or ``mac`` makes this driver attach itself to all unrouted NetVSC interfaces found on the system.
drivers/net/vdev_netvsc/vdev_netvsc.c +19 -11
@@ -36,6 +36,7 @@#define VDEV_NETVSC_DRIVER net_vdev_netvsc#define VDEV_NETVSC_ARG_IFACE "iface"#define VDEV_NETVSC_ARG_MAC "mac" +#define VDEV_NETVSC_ARG_FORCE "force"#define VDEV_NETVSC_PROBE_MS 1000#define NETVSC_CLASS_ID "{f8615163-df3e-46c5-913f-f2d2f965ed0e}"
@@ -419,6 +420,9 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =* - struct rte_kvargs *kvargs: * Device arguments provided to current driver instance. * +* - int force: +* Accept specified interface even if not detected as NetVSC. +* * - unsigned int specified: * Number of specific netdevices provided as device arguments. *
@@ -436,6 +440,7 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list = { const char *name = va_arg(ap, const char *); struct rte_kvargs *kvargs = va_arg(ap, struct rte_kvargs *); +int force =va_arg (ap, int); unsigned int specified = va_arg(ap, unsigned int); unsigned int *matched = va_arg(ap, unsigned int *);unsigned int i;
@@ -490,20 +495,18 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =return 0 ; }if (!vdev_netvsc_iface_is_netvsc (iface)) { -if (!specified) +if (!specified || !force)return 0 ;DRV_LOG (WARNING , -"interface \"%s\" (index %u) is not NetVSC," -" skipping" , +"using non-NetVSC interface \"%s\" (index %u)" , iface->if_name, iface->if_index); -return 0 ; }/* Routed NetVSC should not be probed. */ if (vdev_netvsc_has_route (iface->if_name)) { -DRV_LOG (WARNING ,"NetVSC interface \"%s\" (index %u) is routed" , - iface->if_name, iface->if_index); -if (!specified) +if (!specified || !force)return 0 ; +DRV_LOG (WARNING ,"using routed NetVSC interface \"%s\"" +" (index %u)" , iface->if_name, iface->if_index); }/* Create interface context. */ ctx =calloc (1 ,sizeof (*ctx));
@@ -597,6 +600,7 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =static const char *const vdev_netvsc_arg[] = {VDEV_NETVSC_ARG_IFACE ,VDEV_NETVSC_ARG_MAC , +VDEV_NETVSC_ARG_FORCE ,NULL , };const char *name =rte_vdev_device_name (dev);
@@ -605,6 +609,7 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list = vdev_netvsc_arg);unsigned int specified =0 ;unsigned int matched =0 ; +int force =0 ;unsigned int i;int ret;
@@ -616,14 +621,16 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =for (i =0 ; i != kvargs->count; ++i) {const struct rte_kvargs_pair *pair = &kvargs->pairs[i]; -if (!strcmp (pair->key,VDEV_NETVSC_ARG_IFACE ) || - !strcmp (pair->key,VDEV_NETVSC_ARG_MAC )) +if (!strcmp (pair->key,VDEV_NETVSC_ARG_FORCE )) + force = !!atoi (pair->value); +else if (!strcmp (pair->key,VDEV_NETVSC_ARG_IFACE ) || + !strcmp (pair->key,VDEV_NETVSC_ARG_MAC )) ++specified; }rte_eal_alarm_cancel (vdev_netvsc_alarm,NULL );/* Gather interfaces. */ ret =vdev_netvsc_foreach_iface (vdev_netvsc_netvsc_probe, name, kvargs, - specified, &matched); + force, specified, &matched);if (ret <0 ) goto error;if (matched < specified)
@@ -682,7 +689,8 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =RTE_PMD_REGISTER_ALIAS (VDEV_NETVSC_DRIVER , eth_vdev_netvsc);RTE_PMD_REGISTER_PARAM_STRING (net_vdev_netvsc,VDEV_NETVSC_ARG_IFACE "=<string> " -VDEV_NETVSC_ARG_MAC "=<string>" ); +VDEV_NETVSC_ARG_MAC "=<string> " +VDEV_NETVSC_ARG_FORCE "=<int>" );/** Initialize driver log type. */ RTE_INIT (vdev_netvsc_init_log)
Signature · 1 line
1.8.3.1
PATCH v5 8/8 net/vdev_netvsc: add automatic probing
In reply to Matan AzradUsing DPDK in Hyper-V VM systems requires vdev_netvsc driver to pair the NetVSC netdev device with the same MAC address PCI device by fail-safe PMD.
Add vdev_netvsc custom scan in vdev bus to allow automatic probing in Hyper-V VM systems unless it was already specified by command line.
Add "ignore" parameter to disable this auto-detection.
--- doc/guides/nics/vdev_netvsc.rst | 9 ++++-- drivers/net/vdev_netvsc/vdev_netvsc.c | 55 +++++++++++++++++++++++++++++++++-- 2 files changed, 60 insertions(+), 4 deletions(-)
doc/guides/nics/vdev_netvsc.rst +7 -2
@@ -71,8 +71,8 @@ Build options Run-time parameters ------------------- -To invoke this driver, applications have to explicitly provide the -``--vdev=net_vdev_netvsc`` EAL option. +This driver is invoked automatically in Hyper-V VM systems unless the user +invoked it by command line using ``--vdev=net_vdev_netvsc`` EAL option. The following device parameters are supported:
@@ -91,5 +91,10 @@ The following device parameters are supported: If nonzero, forces the use of specified interfaces even if not detected as NetVSC or detected as routed NETVSC. +- ``ignore`` [int] + + If nonzero, ignores the driver runnig (actually used to disable the + auto-detection in Hyper-V VM). + Not specifying either ``iface`` or ``mac`` makes this driver attach itself to all unrouted NetVSC interfaces found on the system.
drivers/net/vdev_netvsc/vdev_netvsc.c +53 -2
105 lines not on this page · raw message
Signature · 1 line
1.8.3.1
v6 cover Introduce virtual driver for Hyper-V/Azure platforms
In reply to Matan AzradVirtual machines hosted by Hyper-V/Azure platforms are fitted with simplified virtual network devices named NetVSC that are used for fast communication between VM to VM, VM to hypervisor, and the outside.
They appear as standard system netdevices to user-land applications, the main difference being they are implemented on top of VMBUS instead of emulated PCI devices.
While this reads like a case for a standard DPDK PMD, there is more to it.
To accelerate outside communication, NetVSC devices as they appear in a VM can be paired with physical SR-IOV virtual function (VF) devices owned by that same VM. Both netdevices share the same MAC address in that case.
When paired, egress and most of the ingress traffic flow through the VF device, while part of it (e.g. multicasts, hypervisor control data) still flows through NetVSC. Moreover VF devices are not retained and disappear during VM migration; from a VM standpoint, they can be hot-plugged anytime with NetVSC acting as a fallback.
Running DPDK applications in such a context involves driving VF devices using their dedicated PMDs in a vendor-independent fashion (to benefit from maximum performance without writing dedicated code) while simultaneously listening to NetVSC and handling the related hot-plug events.
This new virtual driver (referred to as "vdev_netvsc" from this point on) automatically coordinates the Hyper-V/Azure-specific management part described above by relying on vendor-specific, failsafe and tap PMDs to expose a single consolidated Ethernet device usable directly by existing applications.
.------------------.
| DPDK application |
`--------+---------'
|
.------+------.
| DPDK ethdev |
`------+------' Control
| |
.------------+------------. v .--------------------.
| failsafe PMD +---------+ vdev_netvsc driver |
`--+-------------------+--' `--------------------'
| |
| .........|.........
| : | :
.----+----. : .----+----. :
| tap PMD | : | any PMD | :
`----+----' : `----+----' : <-- Hot-pluggable
| : | :
.------+-------. : .-----+-----. :
| NetVSC-based | : | SR-IOV VF | :
| netdevice | : | device | :
`--------------' : `-----------' :
:.................:v2 changes(Adrien):
- Renamed driver from "hyperv" to "vdev_netvsc". This change covers documentation and symbols prefix.
- Driver is now tagged EXPERIMENTAL.
- Replaced ether_addr_from_str() with a basic sscanf() call.
- Removed debugging code (memset() poisoning).
- Fixed hyperv_iface_is_netvsc()'s buffer allocation according to comments.
- Removed hyperv_basename().
- Discarded unused variables through __rte_unused.
- Added separate but necessary free() bugfix for failsafe PMD.
- Added file descriptor input support to failsafe PMD.
- Replaced temporary bash execution; failsafe now reads device definitions directly through a pipe without an intermediate bash one-liner.
- Expanded DEBUG/INFO/WARN/ERROR() macros as PMD_DRV_LOG().
- Added dynamic log type (pmd.vdev_netvsc).
- Modified initialization code to probe devices immediately during startup.
- Fixed several snprintf() return value checks ("ret >= sizeof(foo)" is more appropriate than "ret >= sizeof(foo) - 1").
v3 changes(Matan):
- Fixed clang compilation in V2.
- Removed hotplug remove code from the new driver.
- Supported probed sub-devices getting in fail-safe.
- Added automatic probing for HyperV VM systems.
- Added option to ignore the automatic probing.
- Skiped routed NetVSC devices probing.
- Adjusted documentation and semantics.
- Replaced maintainer.
v4 changes(Matan):
- Align descriptions of context struct(Stephen suggestion).
- Skip non-ethernet devices in netdev loop(Stephen suggestion).
- Use different variable names in "add fd parameter"(Gaetan suggestion).
- Change name of get port id function in "add automatic probing"(Gaetan suggestion).
- Update internal fail-safe devargs in case of probed device(Gaetan suggestion).
- use deferent commit title instead of "support probed sub-devices getting"(Gaetan suggestion).
v5 changes(Matan):
- Improve fail-safe documentation as Gaetan suggested.
- Fix fcntl paramenter.
v6 changes:
- fp!=NULL => fp==NULL in "add fd parameter".
Adrien Mazarguil (1):
net/failsafe: fix invalid free
Matan Azrad (7):
net/failsafe: add "fd" parameter net/failsafe: add probed etherdev capture net/vdev_netvsc: introduce Hyper-V platform driver net/vdev_netvsc: implement core functionality net/vdev_netvsc: skip routed netvsc probing net/vdev_netvsc: add "force" parameter net/vdev_netvsc: add automatic probing
MAINTAINERS | 6 + config/common_base | 5 + config/common_linuxapp | 1 + doc/guides/nics/fail_safe.rst | 26 + doc/guides/nics/features/vdev_netvsc.ini | 12 + doc/guides/nics/index.rst | 1 + doc/guides/nics/vdev_netvsc.rst | 100 +++ drivers/net/Makefile | 1 + drivers/net/failsafe/failsafe_args.c | 84 ++- drivers/net/failsafe/failsafe_eal.c | 78 ++- drivers/net/failsafe/failsafe_private.h | 5 + drivers/net/vdev_netvsc/Makefile | 31 + .../vdev_netvsc/rte_pmd_vdev_netvsc_version.map | 4 + drivers/net/vdev_netvsc/vdev_netvsc.c | 752 +++++++++++++++++++++ mk/rte.app.mk | 1 + 15 files changed, 1083 insertions(+), 24 deletions(-) create mode 100644 doc/guides/nics/features/vdev_netvsc.ini create mode 100644 doc/guides/nics/vdev_netvsc.rst create mode 100644 drivers/net/vdev_netvsc/Makefile create mode 100644 drivers/net/vdev_netvsc/rte_pmd_vdev_netvsc_version.map create mode 100644 drivers/net/vdev_netvsc/vdev_netvsc.c
Signature · 1 line
1.8.3.1
PATCH v6 1/8 net/failsafe: fix invalid free
From: Adrien Mazarguil <[email protected]>
rte_free() is not supposed to work with pointers returned by calloc().
--- drivers/net/failsafe/failsafe_args.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/failsafe/failsafe_args.c +1 -1
@@ -407,7 +407,7 @@ typedef int (parse_cb)(struct rte_eth_dev *dev, const char *params,uint8_t i; FOREACH_SUBDEV(sdev, i, dev) { -rte_free (sdev->cmdline); +free (sdev->cmdline); sdev->cmdline =NULL ;free (sdev->devargs.args); sdev->devargs.args =NULL ;
Signature · 1 line
1.8.3.1
PATCH v6 2/8 net/failsafe: add "fd" parameter
In reply to Matan AzradThis parameter enables applications to provide device definitions through an arbitrary file descriptor number.
--- doc/guides/nics/fail_safe.rst | 9 ++++ drivers/net/failsafe/failsafe_args.c | 80 ++++++++++++++++++++++++++++++++- drivers/net/failsafe/failsafe_private.h | 3 ++ 3 files changed, 91 insertions(+), 1 deletion(-)
doc/guides/nics/fail_safe.rst +9 -0
@@ -106,6 +106,15 @@ Fail-safe command line parameters All commas within the ``shell command`` are replaced by spaces before executing the command. This helps using scripts to specify devices. +- **fd(<file descriptor number>)** parameter + + This parameter reads a device definition from an arbitrary file descriptor + number in ``<iface>`` format as described above. + + The file descriptor is read in non-blocking mode and is never closed in + order to take only the last line into account (unlike ``exec()``) at every + probe attempt. + - **mac** parameter [MAC address] This parameter allows the user to set a default MAC address to the fail-safe
drivers/net/failsafe/failsafe_args.c +79 -1
117 lines not on this page · raw message
drivers/net/failsafe/failsafe_private.h +3 -0
@@ -48,6 +48,7 @@#define PMD_FAILSAFE_PARAM_STRING \ "dev(<ifc>)," \ "exec(<shell command>)," \ + "fd(<fd number>)," \ "mac=mac_addr," \ "hotplug_poll=u64" \ ""
@@ -112,6 +113,8 @@ struct sub_device {struct fs_stats stats_snapshot;/* Some device are defined as a command line */ char *cmdline; +/* Others are retrieved through a file descriptor */ +char *fd_str;/* fail-safe device backreference */ struct rte_eth_dev *fs_dev;/* flag calling for recollection */
Signature · 1 line
1.8.3.1
PATCH v6 3/8 net/failsafe: add probed etherdev capture
In reply to Matan AzradPrevious fail-safe code didn't support probed sub-devices capture and failed when it tried to probe them.
Skip fail-safe sub-device probing when it already was probed.
--- doc/guides/nics/fail_safe.rst | 17 +++++++ drivers/net/failsafe/failsafe_args.c | 2 - drivers/net/failsafe/failsafe_eal.c | 78 ++++++++++++++++++++++++--------- drivers/net/failsafe/failsafe_private.h | 2 + 4 files changed, 77 insertions(+), 22 deletions(-)
doc/guides/nics/fail_safe.rst +17 -0
@@ -93,6 +93,14 @@ Fail-safe command line parameters additional sub-device parameters if need be. They will be passed on to the sub-device. +.. note:: + + In case of whitelist sub-device probed by EAL, fail-safe PMD will take the device + as is, which means that EAL device options are taken in this case. + When trying to use a PCI device automatically probed in blacklist mode, + the syntax for the fail-safe must be with the full PCI id: + Domain:Bus:Device.Function. See the usage example section. + - **exec(<shell command>)** parameter This parameter allows the user to provide a command to the fail-safe PMD to
@@ -169,6 +177,15 @@ This section shows some example of using **testpmd** with a fail-safe PMD. $RTE_TARGET/build/app/testpmd -c 0xff -n 4 --no-pci \ --vdev='net_failsafe0,exec(echo 84:00.0)' -- -i +#. Start testpmd, automatically probing the device 84:00.0 and using it with + the fail-safe. + + .. code-block:: console + + $RTE_TARGET/build/app/testpmd -c 0xff -n 4 \ + --vdev 'net_failsafe0,dev(0000:84:00.0),dev(net_ring0)' -- -i + + Using the Fail-safe PMD from an application -------------------------------------------
drivers/net/failsafe/failsafe_args.c +0 -2
@@ -45,8 +45,6 @@#include "failsafe_private.h" -#define DEVARGS_MAXLEN 4096 -/* Callback used when a new device is found in devargs */ typedef int (parse_cb )(struct rte_eth_dev *dev,const char *params,uint8_t head);
drivers/net/failsafe/failsafe_eal.c +58 -20
97 lines not on this page · raw message
drivers/net/failsafe/failsafe_private.h +2 -0
@@ -58,6 +58,8 @@#define FAILSAFE_MAX_ETHPORTS 2#define FAILSAFE_MAX_ETHADDR 128 +#define DEVARGS_MAXLEN 4096 +/* TYPES */ struct rxq {
Signature · 1 line
1.8.3.1
Re: [PATCH v6 3/8] net/failsafe: add probed etherdev capture
18/01/2018 14:51, Matan Azrad:
Quoted 7 lines
Previous fail-safe code didn't support probed sub-devices capture and failed when it tried to probe them.
Skip fail-safe sub-device probing when it already was probed.
Signed-off-by Matan Azrad <[email protected]>Cc Gaetan Rivet <[email protected]>
Was acked by Gaetan in v5.
PATCH v6 4/8 net/vdev_netvsc: introduce Hyper-V platform driver
In reply to Matan AzradThis patch lays the groundwork for this driver (draft documentation, copyright notices, code base skeleton and build system hooks). While it can be successfully compiled and invoked, it's an empty shell at this stage.
--- MAINTAINERS | 6 ++ config/common_base | 5 ++ config/common_linuxapp | 1 + doc/guides/nics/features/vdev_netvsc.ini | 12 +++ doc/guides/nics/index.rst | 1 + doc/guides/nics/vdev_netvsc.rst | 20 +++++ drivers/net/Makefile | 1 + drivers/net/vdev_netvsc/Makefile | 27 ++++++ .../vdev_netvsc/rte_pmd_vdev_netvsc_version.map | 4 + drivers/net/vdev_netvsc/vdev_netvsc.c | 99 ++++++++++++++++++++++ mk/rte.app.mk | 1 + 11 files changed, 177 insertions(+) create mode 100644 doc/guides/nics/features/vdev_netvsc.ini create mode 100644 doc/guides/nics/vdev_netvsc.rst create mode 100644 drivers/net/vdev_netvsc/Makefile create mode 100644 drivers/net/vdev_netvsc/rte_pmd_vdev_netvsc_version.map create mode 100644 drivers/net/vdev_netvsc/vdev_netvsc.c
MAINTAINERS +6 -0
@@ -462,6 +462,12 @@ F: drivers/net/mrvl/ F: doc/guides/nics/mrvl.rst F: doc/guides/nics/features/mrvl.ini +Microsoft vdev-netvsc - EXPERIMENTAL +M: Matan Azrad <[email protected]> +F: drivers/net/vdev-netvsc/ +F: doc/guides/nics/vdev-netvsc.rst +F: doc/guides/nics/features/vdev-netvsc.ini + Netcope szedata2 M: Matej Vido <[email protected]> F: drivers/net/szedata2/
config/common_base +5 -0
@@ -279,6 +279,11 @@ CONFIG_RTE_LIBRTE_NFP_DEBUG_RX=n CONFIG_RTE_LIBRTE_MRVL_PMD=n # +# Compile virtual device driver for NetVSC on Hyper-V/Azure +# +CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD=n + +# # Compile burst-oriented Broadcom BNXT PMD driver # CONFIG_RTE_LIBRTE_BNXT_PMD=y
config/common_linuxapp +1 -0
@@ -47,6 +47,7 @@ CONFIG_RTE_LIBRTE_PMD_VHOST=y CONFIG_RTE_LIBRTE_PMD_AF_PACKET=y CONFIG_RTE_LIBRTE_PMD_TAP=y CONFIG_RTE_LIBRTE_AVP_PMD=y +CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD=y CONFIG_RTE_LIBRTE_NFP_PMD=y CONFIG_RTE_LIBRTE_POWER=y CONFIG_RTE_VIRTIO_USER=y
doc/guides/nics/features/vdev_netvsc.ini new file +12 -0
@@ -0,0 +1,12 @@ +; +; Supported features of the 'vdev_netvsc' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +ARMv7 = Y +ARMv8 = Y +Power8 = Y +x86-32 = Y +x86-64 = Y +Usage doc = Y
doc/guides/nics/index.rst +1 -0
@@ -64,6 +64,7 @@ Network Interface Controller Drivers szedata2 tap thunderx + vdev_netvsc virtio vhost vmxnet3
doc/guides/nics/vdev_netvsc.rst new file +20 -0
@@ -0,0 +1,20 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright 2017 6WIND S.A. + Copyright 2017 Mellanox Technologies, Ltd. + +VDEV_NETVSC driver +================== + +The VDEV_NETVSC driver (librte_pmd_vdev_netvsc) provides support for NetVSC +interfaces and associated SR-IOV virtual function (VF) devices found in +Linux virtual machines running on Microsoft Hyper-V_ (including Azure) +platforms. + +.. _Hyper-V: https://docs.microsoft.com/en-us/windows-hardware/drivers/network/overview-of-hyper-v + +Build options +------------- + +- ``CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD`` (default ``y``) + + Toggle compilation of this driver.
drivers/net/Makefile +1 -0
@@ -39,6 +39,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += sfc DIRS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2) += szedata2 DIRS-$(CONFIG_RTE_LIBRTE_PMD_TAP) += tap DIRS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += thunderx +DIRS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += vdev_netvsc DIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio DIRS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += vmxnet3
drivers/net/vdev_netvsc/Makefile new file +27 -0
@@ -0,0 +1,27 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright 2017 6WIND S.A. +# Copyright 2017 Mellanox Technologies, Ltd. + +include $(RTE_SDK)/mk/rte.vars.mk + +# Properties of the generated library. +LIB = librte_pmd_vdev_netvsc.a +LIBABIVER := 1 +EXPORT_MAP := rte_pmd_vdev_netvsc_version.map + +# Additional compilation flags. +CFLAGS += -O3 +CFLAGS += -g +CFLAGS += -std=c11 -pedantic -Wall -Wextra +CFLAGS += $(WERROR_FLAGS) + +# Dependencies. +LDLIBS += -lrte_bus_vdev +LDLIBS += -lrte_eal +LDLIBS += -lrte_ethdev +LDLIBS += -lrte_kvargs + +# Source files. +SRCS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += vdev_netvsc.c + +include $(RTE_SDK)/mk/rte.lib.mk
drivers/net/vdev_netvsc/rte_pmd_vdev_netvsc_version.map new file +4 -0
@@ -0,0 +1,4 @@ +DPDK_18.02 { + + local: *; +};
drivers/net/vdev_netvsc/vdev_netvsc.c new file +99 -0
99 lines not on this page · raw message
mk/rte.app.mk +1 -0
@@ -157,6 +157,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_SFC_EFX_PMD) += -lrte_pmd_sfc_efx _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2) += -lrte_pmd_szedata2 -lsze2 _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_TAP) += -lrte_pmd_tap _LDLIBS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += -lrte_pmd_thunderx_nicvf +_LDLIBS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += -lrte_pmd_vdev_netvsc _LDLIBS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += -lrte_pmd_virtio ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y) _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += -lrte_pmd_vhost
Signature · 1 line
1.8.3.1
PATCH v6 5/8 net/vdev_netvsc: implement core functionality
In reply to Matan AzradAs described in more details in the attached documentation (see patch contents), this virtual device driver manages NetVSC interfaces in virtual machines hosted by Hyper-V/Azure platforms.
This driver does not manage traffic nor Ethernet devices directly; it acts as a thin configuration layer that automatically instantiates and controls fail-safe PMD instances combining tap and PCI sub-devices, so that each NetVSC interface is exposed as a single consolidated port to DPDK applications.
PCI sub-devices being hot-pluggable (e.g. during VM migration), applications automatically benefit from increased throughput when present and automatic fallback on NetVSC otherwise without interruption thanks to fail-safe's hot-plug handling.
Once initialized, the sole job of the vdev_netvsc driver is to regularly scan for PCI devices to associate with NetVSC interfaces and feed their addresses to corresponding fail-safe instances.
--- doc/guides/nics/vdev_netvsc.rst | 70 +++++ drivers/net/vdev_netvsc/Makefile | 4 + drivers/net/vdev_netvsc/vdev_netvsc.c | 550 +++++++++++++++++++++++++++++++++- 3 files changed, 623 insertions(+), 1 deletion(-)
doc/guides/nics/vdev_netvsc.rst +70 -0
@@ -12,9 +12,79 @@ platforms. .. _Hyper-V: https://docs.microsoft.com/en-us/windows-hardware/drivers/network/overview-of-hyper-v +Implementation details +---------------------- + +Each instance of this driver effectively needs to drive two devices: the +NetVSC interface proper and its SR-IOV VF (referred to as "physical" from +this point on) counterpart sharing the same MAC address. + +Physical devices are part of the host system and cannot be maintained during +VM migration. From a VM standpoint they appear as hot-plug devices that come +and go without prior notice. + +When the physical device is present, egress and most of the ingress traffic +flows through it; only multicasts and other hypervisor control still flow +through NetVSC. Otherwise, NetVSC acts as a fallback for all traffic. + +To avoid unnecessary code duplication and ensure maximum performance, +handling of physical devices is left to their original PMDs; this virtual +device driver (also known as *vdev*) manages other PMDs as summarized by the +following block diagram:: + + .------------------. + | DPDK application | + `--------+---------' + | + .------+------. + | DPDK ethdev | + `------+------' Control + | | + .------------+------------. v .--------------------. + | failsafe PMD +---------+ vdev_netvsc driver | + `--+-------------------+--' `--------------------' + | | + | .........|......... + | : | : + .----+----. : .----+----. : + | tap PMD | : | any PMD | : + `----+----' : `----+----' : <-- Hot-pluggable + | : | : + .------+-------. : .-----+-----. : + | NetVSC-based | : | SR-IOV VF | : + | netdevice | : | device | : + `--------------' : `-----------' : + :.................: + + +This driver implementation may be temporary and should be improved or removed +either when hot-plug will be fully supported in EAL and bus drivers or when +a new NetVSC driver will be integrated. + Build options ------------- - ``CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD`` (default ``y``) Toggle compilation of this driver. + +Run-time parameters +------------------- + +To invoke this driver, applications have to explicitly provide the +``--vdev=net_vdev_netvsc`` EAL option. + +The following device parameters are supported: + +- ``iface`` [string] + + Provide a specific NetVSC interface (netdevice) name to attach this driver + to. Can be provided multiple times for additional instances. + +- ``mac`` [string] + + Same as ``iface`` except a suitable NetVSC interface is located using its + MAC address. + +Not specifying either ``iface`` or ``mac`` makes this driver attach itself to +all NetVSC interfaces found on the system.
drivers/net/vdev_netvsc/Makefile +4 -0
@@ -13,6 +13,9 @@ EXPORT_MAP := rte_pmd_vdev_netvsc_version.map CFLAGS += -O3 CFLAGS += -g CFLAGS += -std=c11 -pedantic -Wall -Wextra +CFLAGS += -D_XOPEN_SOURCE=600 +CFLAGS += -D_BSD_SOURCE +CFLAGS += -D_DEFAULT_SOURCE CFLAGS += $(WERROR_FLAGS) # Dependencies.
@@ -20,6 +23,7 @@ LDLIBS += -lrte_bus_vdev LDLIBS += -lrte_eal LDLIBS += -lrte_ethdev LDLIBS += -lrte_kvargs +LDLIBS += -lrte_net # Source files. SRCS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += vdev_netvsc.c
drivers/net/vdev_netvsc/vdev_netvsc.c +549 -1
603 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH v6 6/8 net/vdev_netvsc: skip routed netvsc probing
In reply to Matan AzradNetVSC netdevices which are already routed should not be probed because they are used for management purposes by the HyperV.
prevent routed netvsc devices probing.
--- doc/guides/nics/vdev_netvsc.rst | 2 +- drivers/net/vdev_netvsc/vdev_netvsc.c | 46 +++++++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+), 1 deletion(-)
doc/guides/nics/vdev_netvsc.rst +1 -1
@@ -87,4 +87,4 @@ The following device parameters are supported: MAC address. Not specifying either ``iface`` or ``mac`` makes this driver attach itself to -all NetVSC interfaces found on the system. +all unrouted NetVSC interfaces found on the system.
drivers/net/vdev_netvsc/vdev_netvsc.c +46 -0
64 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH v6 7/8 net/vdev_netvsc: add "force" parameter
In reply to Matan AzradThis parameter allows specifying any non-NetVSC interface or routed NetVSC interfaces to use with tap sub-devices for development purposes.
--- doc/guides/nics/vdev_netvsc.rst | 5 +++++ drivers/net/vdev_netvsc/vdev_netvsc.c | 30 +++++++++++++++++++----------- 2 files changed, 24 insertions(+), 11 deletions(-)
doc/guides/nics/vdev_netvsc.rst +5 -0
@@ -86,5 +86,10 @@ The following device parameters are supported: Same as ``iface`` except a suitable NetVSC interface is located using its MAC address. +- ``force`` [int] + + If nonzero, forces the use of specified interfaces even if not detected as + NetVSC or detected as routed NETVSC. + Not specifying either ``iface`` or ``mac`` makes this driver attach itself to all unrouted NetVSC interfaces found on the system.
drivers/net/vdev_netvsc/vdev_netvsc.c +19 -11
@@ -36,6 +36,7 @@#define VDEV_NETVSC_DRIVER net_vdev_netvsc#define VDEV_NETVSC_ARG_IFACE "iface"#define VDEV_NETVSC_ARG_MAC "mac" +#define VDEV_NETVSC_ARG_FORCE "force"#define VDEV_NETVSC_PROBE_MS 1000#define NETVSC_CLASS_ID "{f8615163-df3e-46c5-913f-f2d2f965ed0e}"
@@ -419,6 +420,9 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =* - struct rte_kvargs *kvargs: * Device arguments provided to current driver instance. * +* - int force: +* Accept specified interface even if not detected as NetVSC. +* * - unsigned int specified: * Number of specific netdevices provided as device arguments. *
@@ -436,6 +440,7 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list = { const char *name = va_arg(ap, const char *); struct rte_kvargs *kvargs = va_arg(ap, struct rte_kvargs *); +int force =va_arg (ap, int); unsigned int specified = va_arg(ap, unsigned int); unsigned int *matched = va_arg(ap, unsigned int *);unsigned int i;
@@ -490,20 +495,18 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =return 0 ; }if (!vdev_netvsc_iface_is_netvsc (iface)) { -if (!specified) +if (!specified || !force)return 0 ;DRV_LOG (WARNING , -"interface \"%s\" (index %u) is not NetVSC," -" skipping" , +"using non-NetVSC interface \"%s\" (index %u)" , iface->if_name, iface->if_index); -return 0 ; }/* Routed NetVSC should not be probed. */ if (vdev_netvsc_has_route (iface->if_name)) { -DRV_LOG (WARNING ,"NetVSC interface \"%s\" (index %u) is routed" , - iface->if_name, iface->if_index); -if (!specified) +if (!specified || !force)return 0 ; +DRV_LOG (WARNING ,"using routed NetVSC interface \"%s\"" +" (index %u)" , iface->if_name, iface->if_index); }/* Create interface context. */ ctx =calloc (1 ,sizeof (*ctx));
@@ -597,6 +600,7 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =static const char *const vdev_netvsc_arg[] = {VDEV_NETVSC_ARG_IFACE ,VDEV_NETVSC_ARG_MAC , +VDEV_NETVSC_ARG_FORCE ,NULL , };const char *name =rte_vdev_device_name (dev);
@@ -605,6 +609,7 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list = vdev_netvsc_arg);unsigned int specified =0 ;unsigned int matched =0 ; +int force =0 ;unsigned int i;int ret;
@@ -616,14 +621,16 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =for (i =0 ; i != kvargs->count; ++i) {const struct rte_kvargs_pair *pair = &kvargs->pairs[i]; -if (!strcmp (pair->key,VDEV_NETVSC_ARG_IFACE ) || - !strcmp (pair->key,VDEV_NETVSC_ARG_MAC )) +if (!strcmp (pair->key,VDEV_NETVSC_ARG_FORCE )) + force = !!atoi (pair->value); +else if (!strcmp (pair->key,VDEV_NETVSC_ARG_IFACE ) || + !strcmp (pair->key,VDEV_NETVSC_ARG_MAC )) ++specified; }rte_eal_alarm_cancel (vdev_netvsc_alarm,NULL );/* Gather interfaces. */ ret =vdev_netvsc_foreach_iface (vdev_netvsc_netvsc_probe, name, kvargs, - specified, &matched); + force, specified, &matched);if (ret <0 ) goto error;if (matched < specified)
@@ -682,7 +689,8 @@ static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list =RTE_PMD_REGISTER_ALIAS (VDEV_NETVSC_DRIVER , eth_vdev_netvsc);RTE_PMD_REGISTER_PARAM_STRING (net_vdev_netvsc,VDEV_NETVSC_ARG_IFACE "=<string> " -VDEV_NETVSC_ARG_MAC "=<string>" ); +VDEV_NETVSC_ARG_MAC "=<string> " +VDEV_NETVSC_ARG_FORCE "=<int>" );/** Initialize driver log type. */ RTE_INIT (vdev_netvsc_init_log)
Signature · 1 line
1.8.3.1
PATCH v6 8/8 net/vdev_netvsc: add automatic probing
In reply to Matan AzradUsing DPDK in Hyper-V VM systems requires vdev_netvsc driver to pair the NetVSC netdev device with the same MAC address PCI device by fail-safe PMD.
Add vdev_netvsc custom scan in vdev bus to allow automatic probing in Hyper-V VM systems unless it was already specified by command line.
Add "ignore" parameter to disable this auto-detection.
--- doc/guides/nics/vdev_netvsc.rst | 9 ++++-- drivers/net/vdev_netvsc/vdev_netvsc.c | 55 +++++++++++++++++++++++++++++++++-- 2 files changed, 60 insertions(+), 4 deletions(-)
doc/guides/nics/vdev_netvsc.rst +7 -2
@@ -71,8 +71,8 @@ Build options Run-time parameters ------------------- -To invoke this driver, applications have to explicitly provide the -``--vdev=net_vdev_netvsc`` EAL option. +This driver is invoked automatically in Hyper-V VM systems unless the user +invoked it by command line using ``--vdev=net_vdev_netvsc`` EAL option. The following device parameters are supported:
@@ -91,5 +91,10 @@ The following device parameters are supported: If nonzero, forces the use of specified interfaces even if not detected as NetVSC or detected as routed NETVSC. +- ``ignore`` [int] + + If nonzero, ignores the driver runnig (actually used to disable the + auto-detection in Hyper-V VM). + Not specifying either ``iface`` or ``mac`` makes this driver attach itself to all unrouted NetVSC interfaces found on the system.
drivers/net/vdev_netvsc/vdev_netvsc.c +53 -2
105 lines not on this page · raw message
Signature · 1 line
1.8.3.1
Re: [PATCH v6 0/8] Introduce virtual driver for Hyper-V/Azure platforms
In reply to Matan AzradQuoted 88 lines · Matan Azrad
Virtual machines hosted by Hyper-V/Azure platforms are fitted with simplified virtual network devices named NetVSC that are used for fast communication between VM to VM, VM to hypervisor, and the outside.
They appear as standard system netdevices to user-land applications, the main difference being they are implemented on top of VMBUS instead of emulated PCI devices.
While this reads like a case for a standard DPDK PMD, there is more to it.
To accelerate outside communication, NetVSC devices as they appear in a VM can be paired with physical SR-IOV virtual function (VF) devices owned by that same VM. Both netdevices share the same MAC address in that case.
When paired, egress and most of the ingress traffic flow through the VF device, while part of it (e.g. multicasts, hypervisor control data) still flows through NetVSC. Moreover VF devices are not retained and disappear during VM migration; from a VM standpoint, they can be hot-plugged anytime with NetVSC acting as a fallback.
Running DPDK applications in such a context involves driving VF devices using their dedicated PMDs in a vendor-independent fashion (to benefit from maximum performance without writing dedicated code) while simultaneously listening to NetVSC and handling the related hot-plug events.
This new virtual driver (referred to as "
vdev_netvsc" from this point on) automatically coordinates the Hyper-V/Azure-specific management part described above by relying on vendor-specific, failsafe and tap PMDs to expose a single consolidated Ethernet device usable directly by existing applications..------------------. | DPDK application | `--------+---------' | .------+------. | DPDK ethdev | `------+------' Control | | .------------+------------. v .--------------------. | failsafe PMD +---------+ vdev_netvsc driver | `--+-------------------+--' `--------------------' | | | .........|......... | : | : .----+----. : .----+----. : | tap PMD | : | any PMD | : `----+----' : `----+----' : <-- Hot-pluggable | : | : .------+-------. : .-----+-----. : | NetVSC-based | : | SR-IOV VF | : | netdevice | : | device | : `--------------' : `-----------' : :.................:v2 changes(Adrien):
- Renamed driver from "hyperv" to "
vdev_netvsc". This change covers documentation and symbols prefix.
- Driver is now tagged EXPERIMENTAL.
- Replacedether_addr_from_str()with a basicsscanf()call.
- Removed debugging code (memset()poisoning).
- Fixed hyperv_iface_is_netvsc()'s buffer allocation according to comments.
- Removedhyperv_basename().
- Discarded unused variables through__rte_unused.
- Added separate but necessaryfree()bugfix for failsafe PMD.
- Added file descriptor input support to failsafe PMD.
- Replaced temporary bash execution; failsafe now reads device definitions directly through a pipe without an intermediate bash one-liner.
- Expanded DEBUG/INFO/WARN/ERROR() macros asPMD_DRV_LOG().
- Added dynamic log type (pmd.vdev_netvsc).
- Modified initialization code to probe devices immediately during startup.
- Fixed severalsnprintf()return value checks ("ret >= sizeof(foo)" is more appropriate than "ret >= sizeof(foo) - 1").v3 changes(Matan):
- Fixed clang compilation in V2.
- Removed hotplug remove code from the new driver.
- Supported probed sub-devices getting in fail-safe.
- Added automatic probing for HyperV VM systems.
- Added option to ignore the automatic probing.
- Skiped routed NetVSC devices probing.
- Adjusted documentation and semantics.
- Replaced maintainer.v4 changes(Matan):
- Align descriptions of context struct(Stephen suggestion).
- Skip non-ethernet devices in netdev loop(Stephen suggestion).
- Use different variable names in "add fd parameter"(Gaetan suggestion).
- Change name of get port id function in "add automatic probing"(Gaetan suggestion).
- Update internal fail-safe devargs in case of probed device(Gaetan suggestion).
- use deferent commit title instead of "support probed sub-devices getting"(Gaetan suggestion).v5 changes(Matan):
- Improve fail-safe documentation as Gaetan suggested.
- Fix fcntl paramenter.v6 changes:
- fp!=NULL => fp==NULL in "add fd parameter".Adrien Mazarguil (1):
net/failsafe: fix invalid free
Quoted 8 lines
Matan Azrad (7):
net/failsafe: add "fd" parameter net/failsafe: add probed etherdev capturenet/vdev_netvsc: introduce Hyper-V platform drivernet/vdev_netvsc: implement core functionalitynet/vdev_netvsc: skip routed netvsc probingnet/vdev_netvsc: add "force" parameternet/vdev_netvsc: add automatic probing
Series applied to dpdk-next-net/master, thanks.