Introduce IPPROTO_SMC
Subsystems (5)
D. Wythe
maintainerv8 cover Introduce IPPROTO_SMC
From: "D. Wythe" <[email protected]>
This patch allows to create smc socket via AF_INET, similar to the following code,
/* create v4 smc sock */
v4 = socket(AF_INET, SOCK_STREAM, IPPROTO_SMC);
/* create v6 smc sock */
v6 = socket(AF_INET6, SOCK_STREAM, IPPROTO_SMC);
There are several reasons why we believe it is appropriate here:
1. For smc sockets, it actually use IPv4 (AF-INET) or IPv6 (AF-INET6) address. There is no AF_SMC address at all.
2. Create smc socket in the AF_INET(6) path, which allows us to reuse the infrastructure of AF_INET(6) path, such as common ebpf hooks. Otherwise, smc have to implement it again in AF_SMC path. Such as:
1. Replace IPPROTO_TCP with IPPROTO_SMC in the socket() syscall initiated by the user, without the use of LD-PRELOAD.
2. Select whether immediate fallback is required based on peer's port/ip before connect().
A very significant result is that we can now use eBPF to implement smc_run instead of LD_PRELOAD, who is completely ineffective in scenarios of static linking.
Another potential value is that we are attempting to optimize the performance of fallback socks, where merging socks is an important part, and it relies on the creation of SMC sockets under the AF_INET path. (More information :
https://lore.kernel.org/netdev/[email protected]/T/)
v2 -> v1:
- Code formatting, mainly including alignment and annotation repair.
- move inet_smc proto ops to inet_smc.c, avoiding af_smc.c becoming too bulky.
- Fix the issue where refactoring affects the initialization order.
- Fix compile warning (unused out_inet_prot) while CONFIG_IPV6 was not set.
v3 -> v2:
- Add Alibaba's copyright information to the newfile
v4 -> v3:
- Fix some spelling errors
- Align function naming style with smc_sock_init() to smc_sk_init()
- Reversing the order of the conditional checks on clcsock to make the code more intuitive
v5 -> v4:
- Fix some spelling errors
- Added comment, "/* CONFIG_IPV6 */", after the final #endif directive.
- Rename smc_inet.h and smc_inet.c to smc_inet.h and smc_inet.c
- Encapsulate the initialization and destruction of inet_smc in inet_smc.c, rather than implementing it directly in af_smc.c.
- Remove useless header files in smc_inet.h
- Make smc_inet_prot_xxx and smc_inet_sock_init() to be static, since it's only used in smc_inet.c
v6 -> v5:
- Wrapping lines to not exceed 80 characters
- Combine initialization and error handling of smc_inet6 into the same #if macro block.
v7 -> v6:
- Modify the value of IPPROTO_SMC to 256 so that it does not affect IPPROTO-MAX
v8 -> v7:
- Remove useless declarations.
D. Wythe (3):
net/smc: refactoring initialization of smc sock
net/smc: expose smc proto operations
net/smc: Introduce IPPROTO_SMC
include/uapi/linux/in.h | 2 + net/smc/Makefile | 2 +- net/smc/af_smc.c | 162 +++++++++++++++++++++++++++--------------------- net/smc/smc.h | 38 ++++++++++++ net/smc/smc_inet.c | 159 +++++++++++++++++++++++++++++++++++++++++++++++ net/smc/smc_inet.h | 22 +++++++ 6 files changed, 314 insertions(+), 71 deletions(-) create mode 100644 net/smc/smc_inet.c create mode 100644 net/smc/smc_inet.h
Signature · 1 line
1.8.3.1
D. Wythe
maintainerPATCH v8 1/3 net/smc: refactoring initialization of smc sock
From: "D. Wythe" <[email protected]>
This patch aims to isolate the shared components of SMC socket allocation by introducing smc_sk_init() for sock initialization and __smc_create_clcsk() for the initialization of clcsock.
This is in preparation for the subsequent implementation of the AF_INET version of SMC.
--- net/smc/af_smc.c | 86 +++++++++++++++++++++++++++++++------------------------- net/smc/smc.h | 5 ++++ 2 files changed, 53 insertions(+), 38 deletions(-)
net/smc/af_smc.c +48 -38
@@ -361,25 +361,15 @@ static void smc_destruct(struct sock *sk)return ; } -static struct sock *smc_sock_alloc (struct net *net,struct socket *sock, -int protocol) +void smc_sk_init (struct net *net,struct sock *sk,int protocol) { -struct smc_sock *smc; -struct proto *prot; -struct sock *sk; - - prot = (protocol ==SMCPROTO_SMC6 ) ? &smc_proto6 : &smc_proto; - sk =sk_alloc (net,PF_SMC ,GFP_KERNEL , prot,0 ); -if (!sk) -return NULL ; +struct smc_sock *smc =smc_sk (sk); -sock_init_data (sock, sk);/* sets sk_refcnt to 1 */ sk->sk_state =SMC_INIT ; sk->sk_destruct = smc_destruct; sk->sk_protocol = protocol;WRITE_ONCE (sk->sk_sndbuf,2 *READ_ONCE (net->smc.sysctl_wmem));WRITE_ONCE (sk->sk_rcvbuf,2 *READ_ONCE (net->smc.sysctl_rmem)); - smc =smc_sk (sk);INIT_WORK (&smc->tcp_listen_work, smc_tcp_listen_work);INIT_WORK (&smc->connect_work, smc_connect_work);INIT_DELAYED_WORK (&smc->conn.tx_work, smc_tx_work);
@@ -389,6 +379,24 @@ static struct sock *smc_sock_alloc(struct net *net, struct socket *sock, sk->sk_prot->hash (sk);mutex_init (&smc->clcsock_release_lock);smc_init_saved_callbacks (smc); + smc->limit_smc_hs = net->smc.limit_smc_hs; + smc->use_fallback = false;/* assume rdma capability first */ + smc->fallback_rsn =0 ; +} + +static struct sock *smc_sock_alloc (struct net *net,struct socket *sock, +int protocol) +{ +struct proto *prot; +struct sock *sk; + + prot = (protocol ==SMCPROTO_SMC6 ) ? &smc_proto6 : &smc_proto; + sk =sk_alloc (net,PF_SMC ,GFP_KERNEL , prot,0 ); +if (!sk) +return NULL ; + +sock_init_data (sock, sk);/* sets sk_refcnt to 1 */ +smc_sk_init (net, sk, protocol);return sk; }
@@ -3321,6 +3329,31 @@ static ssize_t smc_splice_read(struct socket *sock, loff_t *ppos, .splice_read = smc_splice_read, }; +int smc_create_clcsk (struct net *net,struct sock *sk,int family) +{ +struct smc_sock *smc =smc_sk (sk); +int rc; + + rc =sock_create_kern (net, family,SOCK_STREAM ,IPPROTO_TCP , + &smc->clcsock); +if (rc) { +sk_common_release (sk); +return rc; + } + +/* smc_clcsock_release() does not wait smc->clcsock->sk's +* destruction; its sk_state might not be TCP_CLOSE after +* smc->sk is close()d, and TCP timers can be fired later, +* which need net ref. +*/ + sk = smc->clcsock->sk; +__netns_tracker_free (net, &sk->ns_tracker, false); + sk->sk_net_refcnt =1 ; +get_net_track (net, &sk->ns_tracker,GFP_KERNEL ); +sock_inuse_add (net,1 ); +return 0 ; +} +static int __smc_create (struct net *net,struct socket *sock,int protocol,int kern,struct socket *clcsock) {
@@ -3346,35 +3379,12 @@ static int __smc_create(struct net *net, struct socket *sock, int protocol,/* create internal TCP socket for CLC handshake and fallback */ smc =smc_sk (sk); - smc->use_fallback = false;/* assume rdma capability first */ - smc->fallback_rsn =0 ; - -/* default behavior from limit_smc_hs in every net namespace */ - smc->limit_smc_hs = net->smc.limit_smc_hs; rc =0 ; -if (!clcsock) { - rc =sock_create_kern (net, family,SOCK_STREAM ,IPPROTO_TCP , - &smc->clcsock); -if (rc) { -sk_common_release (sk); - goto out; - } - -/* smc_clcsock_release() does not wait smc->clcsock->sk's -* destruction; its sk_state might not be TCP_CLOSE after -* smc->sk is close()d, and TCP timers can be fired later, -* which need net ref. -*/ - sk = smc->clcsock->sk; -__netns_tracker_free (net, &sk->ns_tracker, false); - sk->sk_net_refcnt =1 ; -get_net_track (net, &sk->ns_tracker,GFP_KERNEL ); -sock_inuse_add (net,1 ); - }else { +if (clcsock) smc->clcsock = clcsock; - } - +else + rc =smc_create_clcsk (net, sk, family); out:return rc; }
net/smc/smc.h +5 -0
@@ -34,6 +34,11 @@extern struct proto smc_proto;extern struct proto smc_proto6; +/* smc sock initialization */ +void smc_sk_init (struct net *net,struct sock *sk,int protocol); +/* clcsock initialization */ +int smc_create_clcsk (struct net *net,struct sock *sk,int family); +#ifdef ATOMIC64_INIT #define KERNEL_HAS_ATOMIC64 #endif
Signature · 1 line
1.8.3.1
D. Wythe
maintainerPATCH v8 2/3 net/smc: expose smc proto operations
In reply to D. WytheFrom: "D. Wythe" <[email protected]>
Externalize smc proto operations (smc_xxx) to allow access from files other than af_smc.c
This is in preparation for the subsequent implementation of the AF_INET version of SMC.
--- net/smc/af_smc.c | 60 ++++++++++++++++++++++++++++---------------------------- net/smc/smc.h | 33 +++++++++++++++++++++++++++++++ 2 files changed, 63 insertions(+), 30 deletions(-)
net/smc/af_smc.c +30 -30
@@ -170,15 +170,15 @@ static bool smc_hs_congested(const struct sock *sk)return false; } -static struct smc_hashinfo smc_v4_hashinfo = { +struct smc_hashinfo smc_v4_hashinfo = { .lock =__RW_LOCK_UNLOCKED (smc_v4_hashinfo.lock), }; -static struct smc_hashinfo smc_v6_hashinfo = { +struct smc_hashinfo smc_v6_hashinfo = { .lock =__RW_LOCK_UNLOCKED (smc_v6_hashinfo.lock), }; -static int smc_hash_sk (struct sock *sk) +int smc_hash_sk (struct sock *sk) {struct smc_hashinfo *h = sk->sk_prot->h.smc_hash;struct hlist_head *head;
@@ -193,7 +193,7 @@ static int smc_hash_sk(struct sock *sk)return 0 ; } -static void smc_unhash_sk (struct sock *sk) +void smc_unhash_sk (struct sock *sk) {struct smc_hashinfo *h = sk->sk_prot->h.smc_hash;
@@ -207,7 +207,7 @@ static void smc_unhash_sk(struct sock *sk)* work which we didn't do because of user hold the sock_lock in the * BH context */ -static void smc_release_cb (struct sock *sk) +void smc_release_cb (struct sock *sk) {struct smc_sock *smc =smc_sk (sk);
@@ -307,7 +307,7 @@ static int __smc_release(struct smc_sock *smc)return rc; } -static int smc_release (struct socket *sock) +int smc_release (struct socket *sock) {struct sock *sk = sock->sk;struct smc_sock *smc;
@@ -401,8 +401,8 @@ static struct sock *smc_sock_alloc(struct net *net, struct socket *sock,return sk; } -static int smc_bind (struct socket *sock,struct sockaddr *uaddr, -int addr_len) +int smc_bind (struct socket *sock,struct sockaddr *uaddr, +int addr_len) {struct sockaddr_in *addr = (struct sockaddr_in *)uaddr;struct sock *sk = sock->sk;
@@ -1649,8 +1649,8 @@ static void smc_connect_work(struct work_struct *work)release_sock (&smc->sk); } -static int smc_connect (struct socket *sock,struct sockaddr *addr, -int alen,int flags) +int smc_connect (struct socket *sock,struct sockaddr *addr, +int alen,int flags) {struct sock *sk = sock->sk;struct smc_sock *smc;
@@ -2631,7 +2631,7 @@ static void smc_clcsock_data_ready(struct sock *listen_clcsock)read_unlock_bh (&listen_clcsock->sk_callback_lock); } -static int smc_listen (struct socket *sock,int backlog) +int smc_listen (struct socket *sock,int backlog) {struct sock *sk = sock->sk;struct smc_sock *smc;
@@ -2696,8 +2696,8 @@ static int smc_listen(struct socket *sock, int backlog)return rc; } -static int smc_accept (struct socket *sock,struct socket *new_sock, -struct proto_accept_arg *arg) +int smc_accept (struct socket *sock,struct socket *new_sock, +struct proto_accept_arg *arg) {struct sock *sk = sock->sk, *nsk;DECLARE_WAITQUEUE (wait, current);
@@ -2766,8 +2766,8 @@ static int smc_accept(struct socket *sock, struct socket *new_sock,return rc; } -static int smc_getname (struct socket *sock,struct sockaddr *addr, -int peer) +int smc_getname (struct socket *sock,struct sockaddr *addr, +int peer) {struct smc_sock *smc;
@@ -2780,7 +2780,7 @@ static int smc_getname(struct socket *sock, struct sockaddr *addr,return smc->clcsock->ops->getname (smc->clcsock, addr, peer); } -static int smc_sendmsg (struct socket *sock,struct msghdr *msg,size_t len) +int smc_sendmsg (struct socket *sock,struct msghdr *msg,size_t len) {struct sock *sk = sock->sk;struct smc_sock *smc;
@@ -2818,8 +2818,8 @@ static int smc_sendmsg(struct socket *sock, struct msghdr *msg, size_t len)return rc; } -static int smc_recvmsg (struct socket *sock,struct msghdr *msg,size_t len, -int flags) +int smc_recvmsg (struct socket *sock,struct msghdr *msg,size_t len, +int flags) {struct sock *sk = sock->sk;struct smc_sock *smc;
@@ -2868,8 +2868,8 @@ static __poll_t smc_accept_poll(struct sock *parent)return mask; } -static __poll_t smc_poll (struct file *file,struct socket *sock, -poll_table *wait) +__poll_t smc_poll (struct file *file,struct socket *sock, +poll_table *wait) {struct sock *sk = sock->sk;struct smc_sock *smc;
@@ -2921,7 +2921,7 @@ static __poll_t smc_poll(struct file *file, struct socket *sock,return mask; } -static int smc_shutdown (struct socket *sock,int how) +int smc_shutdown (struct socket *sock,int how) {struct sock *sk = sock->sk;bool do_shutdown = true;
@@ -3061,8 +3061,8 @@ static int __smc_setsockopt(struct socket *sock, int level, int optname,return rc; } -static int smc_setsockopt (struct socket *sock,int level,int optname, -sockptr_t optval,unsigned int optlen) +int smc_setsockopt (struct socket *sock,int level,int optname, +sockptr_t optval,unsigned int optlen) {struct sock *sk = sock->sk;struct smc_sock *smc;
@@ -3148,8 +3148,8 @@ static int smc_setsockopt(struct socket *sock, int level, int optname,return rc; } -static int smc_getsockopt (struct socket *sock,int level,int optname, -char __user *optval,int __user *optlen) +int smc_getsockopt (struct socket *sock,int level,int optname, +char __user *optval,int __user *optlen) {struct smc_sock *smc;int rc;
@@ -3174,8 +3174,8 @@ static int smc_getsockopt(struct socket *sock, int level, int optname,return rc; } -static int smc_ioctl (struct socket *sock,unsigned int cmd, -unsigned long arg) +int smc_ioctl (struct socket *sock,unsigned int cmd, +unsigned long arg) {union smc_host_cursor cons, urg;struct smc_connection *conn;
@@ -3261,9 +3261,9 @@ static int smc_ioctl(struct socket *sock, unsigned int cmd,* Note that subsequent recv() calls have to wait till all splice() processing * completed. */ -static ssize_t smc_splice_read (struct socket *sock,loff_t *ppos, -struct pipe_inode_info *pipe,size_t len, -unsigned int flags) +ssize_t smc_splice_read (struct socket *sock,loff_t *ppos, +struct pipe_inode_info *pipe,size_t len, +unsigned int flags) {struct sock *sk = sock->sk;struct smc_sock *smc;
net/smc/smc.h +33 -0
@@ -34,6 +34,39 @@extern struct proto smc_proto;extern struct proto smc_proto6; +extern struct smc_hashinfo smc_v4_hashinfo; +extern struct smc_hashinfo smc_v6_hashinfo; + +int smc_hash_sk (struct sock *sk); +void smc_unhash_sk (struct sock *sk); +void smc_release_cb (struct sock *sk); + +int smc_release (struct socket *sock); +int smc_bind (struct socket *sock,struct sockaddr *uaddr, +int addr_len); +int smc_connect (struct socket *sock,struct sockaddr *addr, +int alen,int flags); +int smc_accept (struct socket *sock,struct socket *new_sock, +struct proto_accept_arg *arg); +int smc_getname (struct socket *sock,struct sockaddr *addr, +int peer); +__poll_t smc_poll (struct file *file,struct socket *sock, +poll_table *wait); +int smc_ioctl (struct socket *sock,unsigned int cmd, +unsigned long arg); +int smc_listen (struct socket *sock,int backlog); +int smc_shutdown (struct socket *sock,int how); +int smc_setsockopt (struct socket *sock,int level,int optname, +sockptr_t optval,unsigned int optlen); +int smc_getsockopt (struct socket *sock,int level,int optname, +char __user *optval,int __user *optlen); +int smc_sendmsg (struct socket *sock,struct msghdr *msg,size_t len); +int smc_recvmsg (struct socket *sock,struct msghdr *msg,size_t len, +int flags); +ssize_t smc_splice_read (struct socket *sock,loff_t *ppos, +struct pipe_inode_info *pipe,size_t len, +unsigned int flags); +/* smc sock initialization */ void smc_sk_init (struct net *net,struct sock *sk,int protocol);/* clcsock initialization */
Signature · 1 line
1.8.3.1
D. Wythe
maintainerPATCH v8 3/3 net/smc: Introduce IPPROTO_SMC
In reply to D. WytheFrom: "D. Wythe" <[email protected]>
This patch allows to create smc socket via AF_INET, similar to the following code,
/* create v4 smc sock */
v4 = socket(AF_INET, SOCK_STREAM, IPPROTO_SMC);
/* create v6 smc sock */
v6 = socket(AF_INET6, SOCK_STREAM, IPPROTO_SMC);
There are several reasons why we believe it is appropriate here:
1. For smc sockets, it actually use IPv4 (AF-INET) or IPv6 (AF-INET6) address. There is no AF_SMC address at all.
2. Create smc socket in the AF_INET(6) path, which allows us to reuse the infrastructure of AF_INET(6) path, such as common ebpf hooks. Otherwise, smc have to implement it again in AF_SMC path.
--- include/uapi/linux/in.h | 2 + net/smc/Makefile | 2 +- net/smc/af_smc.c | 16 ++++- net/smc/smc_inet.c | 159 ++++++++++++++++++++++++++++++++++++++++++++++++ net/smc/smc_inet.h | 22 +++++++ 5 files changed, 198 insertions(+), 3 deletions(-) create mode 100644 net/smc/smc_inet.c create mode 100644 net/smc/smc_inet.h
include/uapi/linux/in.h +2 -0
@@ -81,6 +81,8 @@ enum {#define IPPROTO_ETHERNET IPPROTO_ETHERNET IPPROTO_RAW = 255, /* Raw IP packets */#define IPPROTO_RAW IPPROTO_RAW + IPPROTO_SMC = 256, /* Shared Memory Communications */ +#define IPPROTO_SMC IPPROTO_SMC IPPROTO_MPTCP = 262, /* Multipath TCP connection */#define IPPROTO_MPTCP IPPROTO_MPTCPIPPROTO_MAX
net/smc/Makefile +1 -1
@@ -4,6 +4,6 @@ obj-$(CONFIG_SMC) += smc.o obj-$(CONFIG_SMC_DIAG) += smc_diag.o smc-y := af_smc.o smc_pnet.o smc_ib.o smc_clc.o smc_core.o smc_wr.o smc_llc.o smc-y += smc_cdc.o smc_tx.o smc_rx.o smc_close.o smc_ism.o smc_netlink.o smc_stats.o -smc-y += smc_tracepoint.o +smc-y += smc_tracepoint.o smc_inet.o smc-$(CONFIG_SYSCTL) += smc_sysctl.o smc-$(CONFIG_SMC_LO) += smc_loopback.o
net/smc/af_smc.c +14 -2
@@ -54,6 +54,7 @@#include "smc_tracepoint.h" #include "smc_sysctl.h" #include "smc_loopback.h" +#include "smc_inet.h" static DEFINE_MUTEX (smc_server_lgr_pending);/* serialize link group * creation on server
@@ -3593,10 +3594,15 @@ static int __init smc_init(void)pr_err ("%s: tcp_ulp_register fails with %d\n" , __func__, rc); goto out_lo; } - + rc =smc_inet_init (); +if (rc) { +pr_err ("%s: smc_inet_init fails with %d\n" , __func__, rc); + goto out_ulp; + }static_branch_enable (&tcp_have_smc);return 0 ; - +out_ulp: +tcp_unregister_ulp (&smc_ulp_ops); out_lo:smc_loopback_exit (); out_ib:
@@ -3633,6 +3639,7 @@ static int __init smc_init(void) static void __exit smc_exit(void) {static_branch_disable (&tcp_have_smc); +smc_inet_exit ();tcp_unregister_ulp (&smc_ulp_ops);sock_unregister (PF_SMC );smc_core_exit ();
@@ -3660,4 +3667,9 @@ static void __exit smc_exit(void)MODULE_LICENSE ("GPL" );MODULE_ALIAS_NETPROTO (PF_SMC );MODULE_ALIAS_TCP_ULP ("smc" ); +/* 256 for IPPROTO_SMC and 1 for SOCK_STREAM */ +MODULE_ALIAS_NET_PF_PROTO_TYPE (PF_INET ,256 ,1 ); +#if IS_ENABLED (CONFIG_IPV6 ) +MODULE_ALIAS_NET_PF_PROTO_TYPE (PF_INET6 ,256 ,1 ); +#endif /* CONFIG_IPV6 */ MODULE_ALIAS_GENL_FAMILY (SMC_GENL_FAMILY_NAME );
net/smc/smc_inet.c new file +159 -0
159 lines not on this page · raw message
net/smc/smc_inet.h new file +22 -0
@@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Shared Memory Communications over RDMA (SMC-R) and RoCE +* +* Definitions for the IPPROTO_SMC (socket related) + +* Copyright IBM Corp. 2016 +* Copyright (c) 2024, Alibaba Inc. +* +* Author: D. Wythe <[email protected]> +*/ +#ifndef __INET_SMC +#define __INET_SMC + +/* Initialize protocol registration on IPPROTO_SMC, +* @return 0 on success +*/ +int smc_inet_init (void ); + +void smc_inet_exit (void ); + +#endif /* __INET_SMC */
Signature · 1 line
1.8.3.1
Re: [PATCH net-next v8 3/3] net/smc: Introduce IPPROTO_SMC
Quoted 164 lines · D. Wythe
From: "D. Wythe" <[email protected]>
This patch allows to create smc socket via
AF_INET, similar to the following code,/* create v4 smc sock */
v4 =socket(AF_INET, SOCK_STREAM, IPPROTO_SMC);/* create v6 smc sock */
v6 =socket(AF_INET6, SOCK_STREAM, IPPROTO_SMC);There are several reasons why we believe it is appropriate here:
1. For smc sockets, it actually use IPv4 (AF-INET) or IPv6 (AF-INET6) address. There is no
AF_SMCaddress at all.2. Create smc socket in the
AF_INET(6)path, which allows us to reuse the infrastructure ofAF_INET(6)path, such as common ebpf hooks. Otherwise, smc have to implement it again inAF_SMCpath.Signed-off-by D. Wythe <[email protected]>Reviewed-by Wenjia Zhang <[email protected]>Reviewed-by Dust Li <[email protected]>Tested-by Niklas Schnelle <[email protected]>Tested-by Wenjia Zhang <[email protected]>--- include/uapi/linux/in.h | 2 + net/smc/Makefile | 2 +- net/smc/af_smc.c | 16 ++++- net/smc/smc_inet.c | 159 ++++++++++++++++++++++++++++++++++++++++++++++++ net/smc/smc_inet.h | 22 +++++++ 5 files changed, 198 insertions(+), 3 deletions(-) create mode 100644 net/smc/smc_inet.c create mode 100644 net/smc/smc_inet.hinclude/uapi/linux/in.h +2
-0@@ -81,6 +81,8 @@ enum {#define IPPROTO_ETHERNET IPPROTO_ETHERNET IPPROTO_RAW = 255, /* Raw IP packets */#define IPPROTO_RAW IPPROTO_RAW + IPPROTO_SMC = 256, /* Shared Memory Communications */ +#define IPPROTO_SMC IPPROTO_SMC IPPROTO_MPTCP = 262, /* Multipath TCP connection */#define IPPROTO_MPTCP IPPROTO_MPTCPIPPROTO_MAX net/smc/Makefile +1
-1@@ -4,6 +4,6 @@ obj-$(CONFIG_SMC) += smc.o obj-$(CONFIG_SMC_DIAG) += smc_diag.o smc-y := af_smc.o smc_pnet.o smc_ib.o smc_clc.o smc_core.o smc_wr.o smc_llc.o smc-y += smc_cdc.o smc_tx.o smc_rx.o smc_close.o smc_ism.o smc_netlink.o smc_stats.o -smc-y += smc_tracepoint.o +smc-y += smc_tracepoint.o smc_inet.o smc-$(CONFIG_SYSCTL) += smc_sysctl.o smc-$(CONFIG_SMC_LO) += smc_loopback.onet/smc/af_smc.c +14
-2@@ -54,6 +54,7 @@#include "smc_tracepoint.h" #include "smc_sysctl.h" #include "smc_loopback.h" +#include "smc_inet.h" static DEFINE_MUTEX (smc_server_lgr_pending);/* serialize link group * creation on server @@ -3593,10 +3594,15 @@ static int __init smc_init(void)pr_err ("%s: tcp_ulp_register fails with %d\n" , __func__, rc); goto out_lo; } - + rc =smc_inet_init (); +if (rc) { +pr_err ("%s: smc_inet_init fails with %d\n" , __func__, rc); + goto out_ulp; + }static_branch_enable (&tcp_have_smc);return 0 ; - +out_ulp: +tcp_unregister_ulp (&smc_ulp_ops); out_lo:smc_loopback_exit (); out_ib:@@ -3633,6 +3639,7 @@ static int __init smc_init(void) static void __exit smc_exit(void) {static_branch_disable (&tcp_have_smc); +smc_inet_exit ();tcp_unregister_ulp (&smc_ulp_ops);sock_unregister (PF_SMC );smc_core_exit ();@@ -3660,4 +3667,9 @@ static void __exit smc_exit(void)MODULE_LICENSE ("GPL" );MODULE_ALIAS_NETPROTO (PF_SMC );MODULE_ALIAS_TCP_ULP ("smc" ); +/* 256 for IPPROTO_SMC and 1 for SOCK_STREAM */ +MODULE_ALIAS_NET_PF_PROTO_TYPE (PF_INET ,256 ,1 ); +#if IS_ENABLED (CONFIG_IPV6 ) +MODULE_ALIAS_NET_PF_PROTO_TYPE (PF_INET6 ,256 ,1 ); +#endif /* CONFIG_IPV6 */ MODULE_ALIAS_GENL_FAMILY (SMC_GENL_FAMILY_NAME );net/smc/smc_inet.c new file +49
-0@@ -0,0 +1,159 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Shared Memory Communications over RDMA (SMC-R) and RoCE +* +* Definitions for the IPPROTO_SMC (socket related) +* +* Copyright IBM Corp. 2016, 2018 +* Copyright (c) 2024, Alibaba Inc. +* +* Author: D. Wythe <[email protected]> +*/ + +#include <net/protocol.h> +#include <net/sock.h> + +#include "smc_inet.h" +#include "smc.h" + +static int smc_inet_init_sock (struct sock *sk); + +static struct proto smc_inet_prot = { + .name ="INET_SMC" , + .owner =THIS_MODULE , + .init = smc_inet_init_sock, + .hash = smc_hash_sk, + .unhash = smc_unhash_sk, + .release_cb = smc_release_cb, + .obj_size =sizeof (struct smc_sock ), + .h.smc_hash = &smc_v4_hashinfo, + .slab_flags =SLAB_TYPESAFE_BY_RCU , +}; + +static const struct proto_ops smc_inet_stream_ops = { + .family =PF_INET , + .owner =THIS_MODULE , + .release = smc_release, + .bind = smc_bind, + .connect = smc_connect, + .socketpair = sock_no_socketpair, + .accept = smc_accept, + .getname = smc_getname, + .poll = smc_poll, + .ioctl = smc_ioctl, + .listen = smc_listen, + .shutdown = smc_shutdown, + .setsockopt = smc_setsockopt, + .getsockopt = smc_getsockopt, + .sendmsg = smc_sendmsg, + .recvmsg = smc_recvmsg,
net/smc/smc_inet.c line 59
Quoted 10 lines
net/smc/smc_inet.c new file +10
-0+ .mmap = sock_no_mmap, + .splice_read = smc_splice_read, +}; + +static struct inet_protosw smc_inet_protosw = { + .type =SOCK_STREAM , + .protocol =IPPROTO_SMC , + .prot = &smc_inet_prot, + .ops = &smc_inet_stream_ops, + .flags =INET_PROTOSW_ICSK ,
When this flag is set, icsk->icsk_sync_mss must be set.
Unable to handle kernel NULL pointer dereference at virtual address
0000000000000000
Mem abort info:
ESR = 0x0000000086000005
EC = 0x21: IABT (current EL), IL = 32 bits
SET = 0, FnV = 0
EA = 0, S1PTW = 0
FSC = 0x05: level 1 translation fault
user pgtable: 4k pages, 48-bit VAs, pgdp=00000001195d1000 [0000000000000000] pgd=0800000109c46003, p4d=0800000109c46003, pud=0000000000000000
Internal error: Oops: 0000000086000005 [#1] PREEMPT SMP Modules linked in: CPU: 1 UID: 0 PID: 8037 Comm: syz.3.265 Not tainted 6.11.0-rc7-syzkaller-g5f5673607153 #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/06/2024 pstate: 80400005 (Nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) pc : 0x0 lr : cipso_v4_sock_setattr+0x2a8/0x3c0 net/ipv4/cipso_ipv4.c:1910
sp : ffff80009b887a90
x29: ffff80009b887aa0 x28: ffff80008db94050 x27: 0000000000000000
x26: 1fffe0001aa6f5b3 x25: dfff800000000000 x24: ffff0000db75da00
x23: 0000000000000000 x22: ffff0000d8b78518 x21: 0000000000000000
x20: ffff0000d537ad80 x19: ffff0000d8b78000 x18: 1fffe000366d79ee
x17: ffff8000800614a8 x16: ffff800080569b84 x15: 0000000000000001
x14: 000000008b336894 x13: 00000000cd96feaa x12: 0000000000000003
x11: 0000000000040000 x10: 00000000000020a3 x9 : 1fffe0001b16f0f1 x8 : 0000000000000000 x7 : 0000000000000000 x6 : 000000000000003f x5 : 0000000000000040 x4 : 0000000000000001 x3 : 0000000000000000 x2 : 0000000000000002 x1 : 0000000000000000 x0 : ffff0000d8b78000
Call trace: 0x0 netlbl_sock_setattr+0x2e4/0x338 net/netlabel/netlabel_kapi.c:1000 smack_netlbl_add+0xa4/0x154 security/smack/smack_lsm.c:2593 smack_socket_post_create+0xa8/0x14c security/smack/smack_lsm.c:2973 security_socket_post_create+0x94/0xd4 security/security.c:4425 __sock_create+0x4c8/0x884 net/socket.c:1587 sock_create net/socket.c:1622 [inline] __sys_socket_create net/socket.c:1659 [inline] __sys_socket+0x134/0x340 net/socket.c:1706 __do_sys_socket net/socket.c:1720 [inline] __se_sys_socket net/socket.c:1718 [inline] __arm64_sys_socket+0x7c/0x94 net/socket.c:1718 __invoke_syscall arch/arm64/kernel/syscall.c:35 [inline] invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:49 el0_svc_common+0x130/0x23c arch/arm64/kernel/syscall.c:132 do_el0_svc+0x48/0x58 arch/arm64/kernel/syscall.c:151 el0_svc+0x54/0x168 arch/arm64/kernel/entry-common.c:712 el0t_64_sync_handler+0x84/0xfc arch/arm64/kernel/entry-common.c:730 el0t_64_sync+0x190/0x194 arch/arm64/kernel/entry.S:598 Code: ???????? ???????? ???????? ???????? (????????) ---[ end trace 0000000000000000 ]---
D. Wythe
maintainerRe: [PATCH net-next v8 3/3] net/smc: Introduce IPPROTO_SMC
Quoted 169 lines · Eric Dumazet
Quoted 168 lines · D. Wythe
From: "D. Wythe" <[email protected]>
This patch allows to create smc socket via
AF_INET, similar to the following code,/* create v4 smc sock */
v4 =socket(AF_INET, SOCK_STREAM, IPPROTO_SMC);/* create v6 smc sock */
v6 =socket(AF_INET6, SOCK_STREAM, IPPROTO_SMC);There are several reasons why we believe it is appropriate here:
1. For smc sockets, it actually use IPv4 (AF-INET) or IPv6 (AF-INET6) address. There is no
AF_SMCaddress at all.2. Create smc socket in the
AF_INET(6)path, which allows us to reuse the infrastructure ofAF_INET(6)path, such as common ebpf hooks. Otherwise, smc have to implement it again inAF_SMCpath.Signed-off-by D. Wythe <[email protected]>Reviewed-by Wenjia Zhang <[email protected]>Reviewed-by Dust Li <[email protected]>Tested-by Niklas Schnelle <[email protected]>Tested-by Wenjia Zhang <[email protected]>--- include/uapi/linux/in.h | 2 + net/smc/Makefile | 2 +- net/smc/af_smc.c | 16 ++++- net/smc/smc_inet.c | 159 ++++++++++++++++++++++++++++++++++++++++++++++++ net/smc/smc_inet.h | 22 +++++++ 5 files changed, 198 insertions(+), 3 deletions(-)create mode 100644
net/smc/smc_inet.c
create mode 100644net/smc/smc_inet.hinclude/uapi/linux/in.h +2
-0@@ -81,6 +81,8 @@ enum {#define IPPROTO_ETHERNET IPPROTO_ETHERNET IPPROTO_RAW = 255, /* Raw IP packets */#define IPPROTO_RAW IPPROTO_RAW + IPPROTO_SMC = 256, /* Shared Memory Communications */ +#define IPPROTO_SMC IPPROTO_SMC IPPROTO_MPTCP = 262, /* Multipath TCP connection */#define IPPROTO_MPTCP IPPROTO_MPTCPIPPROTO_MAX net/smc/Makefile +1
-1@@ -4,6 +4,6 @@ obj-$(CONFIG_SMC) += smc.o obj-$(CONFIG_SMC_DIAG) += smc_diag.o smc-y := af_smc.o smc_pnet.o smc_ib.o smc_clc.o smc_core.o smc_wr.o smc_llc.o smc-y += smc_cdc.o smc_tx.o smc_rx.o smc_close.o smc_ism.o smc_netlink.o smc_stats.o -smc-y += smc_tracepoint.o +smc-y += smc_tracepoint.o smc_inet.o smc-$(CONFIG_SYSCTL) += smc_sysctl.o smc-$(CONFIG_SMC_LO) += smc_loopback.onet/smc/af_smc.c +14
-2@@ -54,6 +54,7 @@#include "smc_tracepoint.h" #include "smc_sysctl.h" #include "smc_loopback.h" +#include "smc_inet.h" static DEFINE_MUTEX (smc_server_lgr_pending);/* serialize link group * creation on server @@ -3593,10 +3594,15 @@ static int __init smc_init(void)pr_err ("%s: tcp_ulp_register fails with %d\n" , __func__, rc); goto out_lo; } - + rc =smc_inet_init (); +if (rc) { +pr_err ("%s: smc_inet_init fails with %d\n" , __func__, rc); + goto out_ulp; + }static_branch_enable (&tcp_have_smc);return 0 ; - +out_ulp: +tcp_unregister_ulp (&smc_ulp_ops); out_lo:smc_loopback_exit (); out_ib:@@ -3633,6 +3639,7 @@ static int __init smc_init(void) static void __exit smc_exit(void) {static_branch_disable (&tcp_have_smc); +smc_inet_exit ();tcp_unregister_ulp (&smc_ulp_ops);sock_unregister (PF_SMC );smc_core_exit ();@@ -3660,4 +3667,9 @@ static void __exit smc_exit(void)MODULE_LICENSE ("GPL" );MODULE_ALIAS_NETPROTO (PF_SMC );MODULE_ALIAS_TCP_ULP ("smc" ); +/* 256 for IPPROTO_SMC and 1 for SOCK_STREAM */ +MODULE_ALIAS_NET_PF_PROTO_TYPE (PF_INET ,256 ,1 ); +#if IS_ENABLED (CONFIG_IPV6 ) +MODULE_ALIAS_NET_PF_PROTO_TYPE (PF_INET6 ,256 ,1 ); +#endif /* CONFIG_IPV6 */ MODULE_ALIAS_GENL_FAMILY (SMC_GENL_FAMILY_NAME );net/smc/smc_inet.c new file +53
-0@@ -0,0 +1,159 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Shared Memory Communications over RDMA (SMC-R) and RoCE +* +* Definitions for the IPPROTO_SMC (socket related) +* +* Copyright IBM Corp. 2016, 2018 +* Copyright (c) 2024, Alibaba Inc. +* +* Author: D. Wythe <[email protected]> +*/ + +#include <net/protocol.h> +#include <net/sock.h> + +#include "smc_inet.h" +#include "smc.h" + +static int smc_inet_init_sock (struct sock *sk); + +static struct proto smc_inet_prot = { + .name ="INET_SMC" , + .owner =THIS_MODULE , + .init = smc_inet_init_sock, + .hash = smc_hash_sk, + .unhash = smc_unhash_sk, + .release_cb = smc_release_cb, + .obj_size =sizeof (struct smc_sock ), + .h.smc_hash = &smc_v4_hashinfo, + .slab_flags =SLAB_TYPESAFE_BY_RCU , +}; + +static const struct proto_ops smc_inet_stream_ops = { + .family =PF_INET , + .owner =THIS_MODULE , + .release = smc_release, + .bind = smc_bind, + .connect = smc_connect, + .socketpair = sock_no_socketpair, + .accept = smc_accept, + .getname = smc_getname, + .poll = smc_poll, + .ioctl = smc_ioctl, + .listen = smc_listen, + .shutdown = smc_shutdown, + .setsockopt = smc_setsockopt, + .getsockopt = smc_getsockopt, + .sendmsg = smc_sendmsg, + .recvmsg = smc_recvmsg, + .mmap = sock_no_mmap, + .splice_read = smc_splice_read, +}; +
Quoted 9 lines
net/smc/smc_inet.c line 59
Quoted 6 lines
net/smc/smc_inet.c new file +6
-0+static struct inet_protosw smc_inet_protosw = { + .type =SOCK_STREAM , + .protocol =IPPROTO_SMC , + .prot = &smc_inet_prot, + .ops = &smc_inet_stream_ops, + .flags =INET_PROTOSW_ICSK ,When this flag is set,
icsk->icsk_sync_mssmust be set.
Hi Eric,
Thanks for your report. I will fix this issue ASAP.
Best wishes,
D. Wythe
Quoted 53 lines
Unable to handle kernel NULL pointer dereference at virtual address0000000000000000
Mem abort info:
ESR =0x0000000086000005
EC =0x21: IABT (current EL), IL = 32 bits
SET = 0, FnV = 0
EA = 0, S1PTW = 0
FSC =0x05: level 1 translation fault
user pgtable: 4k pages, 48-bit VAs, pgdp=00000001195d1000 [0000000000000000] pgd=0800000109c46003, p4d=0800000109c46003, pud=0000000000000000Internal error: Oops: 0000000086000005 [#1] PREEMPT SMP Modules linked in: CPU: 1 UID: 0 PID: 8037 Comm: syz.3.265 Not tainted 6.11.0-rc7-syzkaller-g5f5673607153 #0 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/06/2024 pstate: 80400005 (Nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) pc : 0x0 lr : cipso_v4_sock_setattr+0x2a8/0x3c0 net/ipv4/cipso_ipv4.c:1910sp :
ffff80009b887a90
x29:ffff80009b887aa0x28:ffff80008db94050x27: 0000000000000000
x26:1fffe0001aa6f5b3x25:dfff800000000000x24:ffff0000db75da00
x23: 0000000000000000 x22:ffff0000d8b78518x21: 0000000000000000
x20:ffff0000d537ad80x19:ffff0000d8b78000x18:1fffe000366d79ee
x17:ffff8000800614a8x16:ffff800080569b84x15: 0000000000000001
x14:000000008b336894x13:00000000cd96feaax12: 0000000000000003
x11: 0000000000040000 x10:00000000000020a3x9 :1fffe0001b16f0f1x8 : 0000000000000000 x7 : 0000000000000000 x6 :000000000000003fx5 : 0000000000000040 x4 : 0000000000000001 x3 : 0000000000000000 x2 : 0000000000000002 x1 : 0000000000000000 x0 :ffff0000d8b78000Call trace: 0x0 netlbl_sock_setattr+0x2e4/0x338 net/netlabel/netlabel_kapi.c:1000 smack_netlbl_add+0xa4/0x154 security/smack/smack_lsm.c:2593 smack_socket_post_create+0xa8/0x14c security/smack/smack_lsm.c:2973 security_socket_post_create+0x94/0xd4 security/security.c:4425 __sock_create+0x4c8/0x884 net/socket.c:1587 sock_create net/socket.c:1622 [inline] __sys_socket_create net/socket.c:1659 [inline] __sys_socket+0x134/0x340 net/socket.c:1706 __do_sys_socket net/socket.c:1720 [inline] __se_sys_socket net/socket.c:1718 [inline] __arm64_sys_socket+0x7c/0x94 net/socket.c:1718 __invoke_syscall arch/arm64/kernel/syscall.c:35 [inline] invoke_syscall+0x98/0x2b8 arch/arm64/kernel/syscall.c:49 el0_svc_common+0x130/0x23c arch/arm64/kernel/syscall.c:132 do_el0_svc+0x48/0x58 arch/arm64/kernel/syscall.c:151 el0_svc+0x54/0x168 arch/arm64/kernel/entry-common.c:712 el0t_64_sync_handler+0x84/0xfc arch/arm64/kernel/entry-common.c:730 el0t_64_sync+0x190/0x194 arch/arm64/kernel/entry.S:598 Code: ???????? ???????? ???????? ???????? (????????) ---[ end trace 0000000000000000 ]---
This series was applied to netdev/net-next.git (main) by David S. Miller <[email protected]>:
Hello:
This series was applied to netdev/net-next.git (main) by David S. Miller <[email protected]>:
Quoted 9 lines · On Fri, 14 Jun 2024 02:00:27 +0800 you wrote:
From: "D. Wythe" <[email protected]>
This patch allows to create smc socket via
AF_INET, similar to the following code,/* create v4 smc sock */
v4 =socket(AF_INET, SOCK_STREAM, IPPROTO_SMC);[...]
Here is the summary with links:
- [net-next,v8,1/3] net/smc: refactoring initialization of smc sock
https://git.kernel.org/netdev/net-next/c/d0e35656d834 - [net-next,v8,2/3] net/smc: expose smc proto operations https://git.kernel.org/netdev/net-next/c/13543d02c90d - [net-next,v8,3/3] net/smc: Introduce IPPROTO_SMC https://git.kernel.org/netdev/net-next/c/d25a92ccae6b
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html