openvswitch: Introduce per-cpu upcall dispatch
Subsystems (3)
PATCH v2 1/1 openvswitch: Introduce per-cpu upcall dispatch
The Open vSwitch kernel module uses the upcall mechanism to send packets from kernel space to user space when it misses in the kernel space flow table. The upcall sends packets via a Netlink socket. Currently, a Netlink socket is created for every vport. In this way, there is a 1:1 mapping between a vport and a Netlink socket. When a packet is received by a vport, if it needs to be sent to user space, it is sent via the corresponding Netlink socket.
This mechanism, with various iterations of the corresponding user space code, has seen some limitations and issues:
* On systems with a large number of vports, there is a correspondingly large number of Netlink sockets which can limit scaling.
(https://bugzilla.redhat.com/show_bug.cgi?id=1526306)
* Packet reordering on upcalls.
(https://bugzilla.redhat.com/show_bug.cgi?id=1844576)
* A thundering herd issue.
(https://bugzilla.redhat.com/show_bug.cgi?id=1834444)
This patch introduces an alternative, feature-negotiated, upcall mode using a per-cpu dispatch rather than a per-vport dispatch.
In this mode, the Netlink socket to be used for the upcall is selected based on the CPU of the thread that is executing the upcall. In this way, it resolves the issues above as:
a) The number of Netlink sockets scales with the number of CPUs rather than the number of vports.
b) Ordering per-flow is maintained as packets are distributed to CPUs based on mechanisms such as RSS and flows are distributed to a single user space thread.
c) Packets from a flow can only wake up one user space thread.
The corresponding user space code can be found at:
https://mail.openvswitch.org/pipermail/ovs-dev/2021-July/385139.html
Bugzilla: https://bugzilla.redhat.com/1844576
---
Notes:
*******
As OVS 2.16 is about to close, we are looking for reviews/acks to determine if there are any major blockers to this getting accepted.
In this way, we can decide if the feature should be added to OVS 2.16 before it closes. Thanks
*******
v1 - Reworked based on Flavio's comments:
* Fixed handling of userspace action case
* Renamed 'struct dp_portids'
* Fixed handling of return from kmalloc()
* Removed check for dispatch type from ovs_dp_get_upcall_portid()
- Reworked based on Dan's comments:
* Fixed handling of return from kmalloc()
- Reworked based on Pravin's comments:
* Fixed handling of userspace action case
- Added kfree() in destroy_dp_rcu() to cleanup netlink port ids
v2 - Reworked based on Pravin's comments
* Changed ovs_dp_set_upcall_portids() to static
* Changed rcu_dereference_ovsl in ovs_dp_get_upcall_portid()
- Updated link to corresponding user space code
include/uapi/linux/openvswitch.h | 8 ++++ net/openvswitch/actions.c | 6 ++- net/openvswitch/datapath.c | 72 +++++++++++++++++++++++++++++++- net/openvswitch/datapath.h | 20 +++++++++ 4 files changed, 103 insertions(+), 3 deletions(-)
include/uapi/linux/openvswitch.h +8 -0
@@ -70,6 +70,8 @@ enum ovs_datapath_cmd {* set on the datapath port (for OVS_ACTION_ATTR_MISS). Only valid on * %OVS_DP_CMD_NEW requests. A value of zero indicates that upcalls should * not be sent. +* OVS_DP_ATTR_PER_CPU_PIDS: Per-cpu array of PIDs for upcalls when +* OVS_DP_F_DISPATCH_UPCALL_PER_CPU feature is set. * @OVS_DP_ATTR_STATS: Statistics about packets that have passed through the * datapath. Always present in notifications. * @OVS_DP_ATTR_MEGAFLOW_STATS: Statistics about mega flow masks usage for the
@@ -87,6 +89,9 @@ enum ovs_datapath_attr {OVS_DP_ATTR_USER_FEATURES ,/* OVS_DP_F_* */ OVS_DP_ATTR_PAD ,OVS_DP_ATTR_MASKS_CACHE_SIZE , +OVS_DP_ATTR_PER_CPU_PIDS ,/* Netlink PIDS to receive upcalls in per-cpu +* dispatch mode +*/ __OVS_DP_ATTR_MAX };
@@ -127,6 +132,9 @@ struct ovs_vport_stats {/* Allow tc offload recirc sharing */ #define OVS_DP_F_TC_RECIRC_SHARING (1 << 2) +/* Allow per-cpu dispatch of upcalls */ +#define OVS_DP_F_DISPATCH_UPCALL_PER_CPU (1 << 3) +/* Fixed logical ports. */ #define OVSP_LOCAL ((__u32)0)
net/openvswitch/actions.c +5 -1
@@ -924,7 +924,11 @@ static int output_userspace(struct datapath *dp, struct sk_buff *skb,break ;case OVS_USERSPACE_ATTR_PID : - upcall.portid =nla_get_u32 (a); +if (dp->user_features &OVS_DP_F_DISPATCH_UPCALL_PER_CPU ) + upcall.portid = +ovs_dp_get_upcall_portid (dp,smp_processor_id ()); +else + upcall.portid =nla_get_u32 (a);break ;case OVS_USERSPACE_ATTR_EGRESS_TUN_PORT : {
net/openvswitch/datapath.c +70 -2
111 lines not on this page · raw message
net/openvswitch/datapath.h +20 -0
@@ -50,6 +50,21 @@ struct dp_stats_percpu {struct u64_stats_sync syncp; }; +/** +* struct dp_nlsk_pids - array of netlink portids of for a datapath. +* This is used when OVS_DP_F_DISPATCH_UPCALL_PER_CPU +* is enabled and must be protected by rcu. +* @rcu: RCU callback head for deferred destruction. +* @n_pids: Size of @pids array. +* @pids: Array storing the Netlink socket PIDs indexed by CPU ID for packets +* that miss the flow table. +*/ +struct dp_nlsk_pids { +struct rcu_head rcu; +u32 n_pids; +u32 pids[]; +}; +/** * struct datapath - datapath for flow-based packet switching * @rcu: RCU callback head for deferred destruction.
@@ -61,6 +76,7 @@ struct dp_stats_percpu {* @net: Reference to net namespace. * @max_headroom: the maximum headroom of all vports in this datapath; it will * be used by all the internal vports in this dp. +* @upcall_portids: RCU protected 'struct dp_nlsk_pids'. * * Context: See the comment on locking at the top of datapath.c for additional * locking information.
@@ -87,6 +103,8 @@ struct datapath {/* Switch meters. */ struct dp_meter_table meter_tbl; + +struct dp_nlsk_pids __rcu *upcall_portids; };/**
@@ -243,6 +261,8 @@ int ovs_dp_upcall(struct datapath *, struct sk_buff *, const struct sw_flow_key *, const struct dp_upcall_info *, uint32_t cutlen); +u32 ovs_dp_get_upcall_portid (const struct datapath *dp,uint32_t cpu_id); +const char *ovs_dp_name (const struct datapath *dp);struct sk_buff *ovs_vport_cmd_build_info (struct vport *vport,struct net *net,u32 portid,u32 seq,u8 cmd);
Signature · 1 line
2.27.0
Quoted 34 lines · Mark Gray
The Open vSwitch kernel module uses the upcall mechanism to send packets from kernel space to user space when it misses in the kernel space flow table. The upcall sends packets via a Netlink socket. Currently, a Netlink socket is created for every vport. In this way, there is a 1:1 mapping between a vport and a Netlink socket. When a packet is received by a vport, if it needs to be sent to user space, it is sent via the corresponding Netlink socket.
This mechanism, with various iterations of the corresponding user space code, has seen some limitations and issues:
* On systems with a large number of vports, there is a correspondingly large number of Netlink sockets which can limit scaling.
(https://bugzilla.redhat.com/show_bug.cgi?id=1526306)
* Packet reordering on upcalls.
(https://bugzilla.redhat.com/show_bug.cgi?id=1844576)
* A thundering herd issue.
(https://bugzilla.redhat.com/show_bug.cgi?id=1834444)This patch introduces an alternative, feature-negotiated, upcall mode using a per-cpu dispatch rather than a per-vport dispatch.
In this mode, the Netlink socket to be used for the upcall is selected based on the CPU of the thread that is executing the upcall. In this way, it resolves the issues above as:
a) The number of Netlink sockets scales with the number of CPUs rather than the number of vports.
b) Ordering per-flow is maintained as packets are distributed to CPUs based on mechanisms such as RSS and flows are distributed to a single user space thread.
c) Packets from a flow can only wake up one user space thread.
Quoted 6 lines
The corresponding user space code can be found at:
https://mail.openvswitch.org/pipermail/ovs-dev/2021-July/385139.htmlBugzilla: https://bugzilla.redhat.com/1844576
Signed-off-by Mark Gray <[email protected]>Acked-by Flavio Leitner <[email protected]>
Thanks,
Quoted 38 lines · Pravin Shelar
Quoted 37 lines · Mark Gray
The Open vSwitch kernel module uses the upcall mechanism to send packets from kernel space to user space when it misses in the kernel space flow table. The upcall sends packets via a Netlink socket. Currently, a Netlink socket is created for every vport. In this way, there is a 1:1 mapping between a vport and a Netlink socket. When a packet is received by a vport, if it needs to be sent to user space, it is sent via the corresponding Netlink socket.
This mechanism, with various iterations of the corresponding user space code, has seen some limitations and issues:
* On systems with a large number of vports, there is a correspondingly large number of Netlink sockets which can limit scaling.
(https://bugzilla.redhat.com/show_bug.cgi?id=1526306)
* Packet reordering on upcalls.
(https://bugzilla.redhat.com/show_bug.cgi?id=1844576)
* A thundering herd issue.
(https://bugzilla.redhat.com/show_bug.cgi?id=1834444)This patch introduces an alternative, feature-negotiated, upcall mode using a per-cpu dispatch rather than a per-vport dispatch.
In this mode, the Netlink socket to be used for the upcall is selected based on the CPU of the thread that is executing the upcall. In this way, it resolves the issues above as:
a) The number of Netlink sockets scales with the number of CPUs rather than the number of vports.
b) Ordering per-flow is maintained as packets are distributed to CPUs based on mechanisms such as RSS and flows are distributed to a single user space thread.
c) Packets from a flow can only wake up one user space thread.The corresponding user space code can be found at:
https://mail.openvswitch.org/pipermail/ovs-dev/2021-July/385139.html
Quoted 8 lines
Quoted 3 lines
Bugzilla: https://bugzilla.redhat.com/1844576
Signed-off-by Mark Gray <[email protected]>Acked-by Flavio Leitner <[email protected]>Acked-by Pravin B Shelar <[email protected]>Thanks,
Thanks for the review Pravin.
This patch was applied to netdev/net-next.git (refs/heads/master):
Hello:
This patch was applied to netdev/net-next.git (refs/heads/master):
Quoted 9 lines · On Thu, 15 Jul 2021 08:27:54 -0400 you wrote:
The Open vSwitch kernel module uses the upcall mechanism to send packets from kernel space to user space when it misses in the kernel space flow table. The upcall sends packets via a Netlink socket. Currently, a Netlink socket is created for every vport. In this way, there is a 1:1 mapping between a vport and a Netlink socket. When a packet is received by a vport, if it needs to be sent to user space, it is sent via the corresponding Netlink socket.
[...]
Here is the summary with links:
- [net-next,v2] openvswitch: Introduce per-cpu upcall dispatch
https://git.kernel.org/netdev/net-next/c/b83d23a2a38b
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html