smb: smbdirect/client/server: moving to common functions and smbdirect.ko
Subsystems (3)
Stefan Metzmacher
reviewerv4 cover smb: smbdirect/client/server: moving to common functions and smbdirect.ko
Hi,
this is the move use common functions in an smbdirect.ko.
It can also be found in my for-6.19/fs-smb-20251125-v4 branch,
at commit d72b76bd33831f43c2b1fef74619eca20236ba6b:
git fetch https://git.samba.org/metze/linux/wip.git for-6.19/fs-smb-20251125-v4
https://git.samba.org/?p=metze/linux/wip.git;a=shortlog;h=refs/heads/for-6.19/fs-smb-20251125-v4
It is based on dhowells-linux-fs/cifs-cleanup at commit 1bc2089694a33afb711779dfcb4bbbe10ccbe466, which is based on smfrench-smb3-kernel/ksmbd-for-next at commit f9fd35675dbd6f6acbb7535f951b6d651b06eb7c.
On top of this I have the 4 patches from:
https://lore.kernel.org/linux-cifs/[email protected]/
smb: smbdirect: introduce SMBDIRECT_DEBUG_ERR_PTR() helper
smb: smbdirect: introduce SMBDIRECT_CHECK_STATUS_{WARN,DISCONNECT}()
smb: server: relax WARN_ON_ONCE(SMBDIRECT_SOCKET_*) checks in
recv_done() and smb_direct_cm_handler()
smb: client: relax WARN_ON_ONCE(SMBDIRECT_SOCKET_*) checks in
recv_done() and smbd_conn_upcall()
Up to commit 78d3a80b0c86e690867d5f8b7ce6920539591aa6 in my for-6.19/fs-smb-20251125-v4.
The end goal is still to go via the socket layer in order to provide smbdirect support to userspace, so that Samba can use if as client and server. But that will be done on top of this patchset.
The patchset starts with the introduction of
a logging infrastructure that allows us to have common code, but still do logging with cifs.ko and ksmbd.ko specific functions.
Then it introduces smbdirect_all_c_files.c, which is only temporary. It is included in the client and server and will itself include more files with common functions in the following patches. This makes it possible to do the transition in small steps. At the end this will be removed again, while removing the file itself will be done in a later patchset.
Then we have a series of patches introducing common functions, most of them are copies
of the existing functions, just with a new name and some reformatting. But the core logic is mostly unchanged.
Then we have new functions related to async
connect and accept handling.
Followed by preparing some functions as public and some as private, so that we have theKbuild logic for smbdirect.ko, which is
still disabled at that point.
Then we have small steps in the client in order to make use of the new functions. At the end is only uses smbdirect.ko and its exported functions.
Then we have small steps in the server in order to make use of the new functions. At the end is only uses smbdirect.ko and its exported functions.
Then there's the move to global workqueues in smbdirect.ko.
The addition of smbdirect_{bind,listen,accept}() function and the transition of the server to use them instead of the custom rdma listener.
Finally some unused stuff is removed.
Every patch compiles on its own and passes
grep -v 'Fixes: 0626e6641f6b ' | \
scripts/checkpatch.pl --quiet --codespell \
--ignore=FILE_PATH_CHANGES,EXPORT_SYMBOL,COMPLEX_MACROI've tested these tests at the end of the
patchset with siw and rxe:
cifs/001
generic/001
generic/002
generic/005
generic/006
generic/007
generic/008
generic/011
generic/024
generic/028
generic/029
generic/030
generic/033
generic/036
generic/069
generic/071
generic/080
generic/084
generic/086
generic/095
generic/098
generic/103
generic/124
generic/130
generic/132
generic/135
generic/141
generic/198
generic/207
generic/210
generic/212
generic/214
generic/215
generic/221
generic/228
generic/236
generic/246
generic/248
generic/249
generic/257
generic/258
generic/308
generic/309
generic/313
generic/315
generic/339
generic/340
generic/344
generic/345
generic/346
generic/354
generic/360
generic/390
generic/391
generic/393
generic/394
generic/406
generic/412
Assuming we want the relax WARN_ON_ONCE patches from:
https://lore.kernel.org/linux-cifs/[email protected]/
in 6.18, can we please get everything up to commit 78d3a80b0c86e690867d5f8b7ce6920539591aa6 into ksmbd-for-next. And once it was in linux-next for one interation, we can put the full set into ksmbd-for-next.
Note that cifs: Use netfs_alloc/free_folioq_buffer() should be removed from for-next, as that will already be in ksmbd-for-next and will cause a merge conflict.
From there I'll work on the changes to introduce IPPROTO_SMBDIRECT, but that needs a bit more work an coordination
with the netdev-next and maybw rdma and io_uring trees, the work in progress can be found here:
https://git.samba.org/?p=metze/linux/wip.git;a=shortlog;h=refs/heads/master-ipproto-smbdirect
v4: some functions are moved to smbdirect_socket.c
and got smbdirect_socket_ as prefix. add and use smbdirect_{bind,listen,accept}() a lot of small fixes here and there...
v3: async connect/accept code was moved to
workqueue functions instead of beeing called in interrupt context via IB_POLL_SOFTIRQ.
Added some includes to smbdirect_socket.h
I also did a few minor changes like using sc->ib.dev consistently.
Stefan Metzmacher (145):
smb: smbdirect: let smbdirect.h include #include <linux/types.h>
smb: smbdirect: introduce smbdirect_socket.logging infrastructure
smb: smbdirect: add some logging to
SMBDIRECT_CHECK_STATUS_{WARN,DISCONNECT}()
smb: smbdirect: introduce smbdirect_recv_io.complex_work
smb: smbdirect: introduce smbdirect_all_c_files.c
smb: smbdirect: introduce smbdirect_internal.h
smb: client: include smbdirect_all_c_files.c
smb: server: include smbdirect_all_c_files.c
smb: smbdirect: introduce smbdirect_socket.c to be filled
smb: smbdirect: introduce smbdirect_socket_prepare_create()
smb: smbdirect: introduce smbdirect_socket_set_logging()
smb: smbdirect: introduce smbdirect_socket_wake_up_all()
smb: smbdirect: introduce smbdirect_socket_cleanup_work()
smb: smbdirect: introduce
smbdirect_socket_schedule_cleanup[{_lvl,_status}]()
smb: smbdirect: introduce smbdirect_connection.c to be filled
smb: smbdirect: introduce smbdirect_connection_{get,put}_recv_io()
smb: smbdirect: introduce
smbdirect_connection_reassembly_{append,first}_recv_io()
smb: smbdirect: introduce smbdirect_connection_idle_timer_work()
smb: smbdirect: set SMBDIRECT_KEEPALIVE_NONE before
disable_delayed_work(&sc->idle.timer_work);
smb: smbdirect: introduce smbdirect_frwr_is_supported()
smb: smbdirect: introduce smbdirect_socket.{send,recv}_io.mem.gfp_mask
smb: smbdirect: introduce smbdirect_connection_{alloc,free}_send_io()
smb: smbdirect: introduce smbdirect_connection_send_io_done()
smb: smbdirect: introduce
smbdirect_connection_{create,destroy}_mem_pools()
smb: smbdirect: introduce smbdirect_map_sges_from_iter() and helper
functions
smb: smbdirect: introduce smbdirect_connection_qp_event_handler()
smb: smbdirect: introduce
smbdirect_connection_negotiate_rdma_resources()
smb: smbdirect: introduce smbdirect_connection_{create,destroy}_qp()
smb: smbdirect: introduce smbdirect_connection_post_recv_io()
smb: smbdirect: introduce smbdirect_connection_recv_io_refill_work()
smb: smbdirect: split out smbdirect_connection_recv_io_refill()
smb: smbdirect: introduce smbdirect_get_buf_page_count()
smb: smbdirect: introduce smbdirect_socket_wait_for_credits()
smb: smbdirect: introduce smbdirect_mr.c with client mr code
smb: smbdirect: introduce smbdirect_rw.c with server rw code
smb: smbdirect: define SMBDIRECT_MIN_{RECEIVE,FRAGMENTED}_SIZE
smb: smbdirect: define SMBDIRECT_RDMA_CM_[RNR_]RETRY
smb: smbdirect: introduce smbdirect_connection_recv_io_done()
smb: smbdirect: introduce smbdirect_socket_destroy[_sync]()
smb: smbdirect: introduce
smbdirect_connection_rdma_{established,event_handler}()
smb: smbdirect: introduce smbdirect_connection_recvmsg()
smb: smbdirect: introduce smbdirect_connection_grant_recv_credits()
smb: smbdirect: introduce smbdirect_connection_request_keep_alive()
smb: smbdirect: introduce smbdirect_connection_send_iter() and related
functions
smb: smbdirect: introduce smbdirect_connection_send_immediate_work()
smb: smbdirect: introduce smbdirect_connection_negotiation_done()
smb: smbdirect: introduce smbdirect_mr_io_fill_buffer_descriptor()
smb: smbdirect: introduce
smbdirect_connection_legacy_debug_proc_show()
smb: smbdirect: introduce smbdirect_connection_wait_for_connected()
smb: smbdirect: introduce smbdirect_connection_is_connected()
smb: smbdirect: introduce smbdirect_socket_shutdown()
smb: smbdirect: introduce smbdirect_socket_init_{new,accepting}() and
helpers
smb: smbdirect: introduce smbdirect_connect[_sync]()
smb: smbdirect: introduce smbdirect_accept_connect_request()
smb: smbdirect: introduce smbdirect_socket_create_{kern,accepting}()
and smbdirect_socket_release()
smb: smbdirect: let smbdirect_socket.h include all headers for used
structures
smb: smbdirect: let smbdirect_internal.h define pr_fmt without
SMBDIRECT_USE_INLINE_C_FILES
smb: smbdirect: introduce smbdirect_public.h with prototypes
smb: smbdirect: provide explicit prototypes for cross .c file
functions
smb: smbdirect: introduce the basic smbdirect.ko
smb: client: make use of smbdirect_socket_prepare_create()
smb: client: make use of smbdirect_socket_set_logging()
smb: client: make use of smbdirect_socket_wake_up_all()
smb: client: let allocate_receive_buffers() initialize complex_work as
disabled
smb: client: make use of smbdirect_socket_cleanup_work()
smb: client: make use of smbdirect_socket_schedule_cleanup()
smb: client: make use of smbdirect_connection_{get,put}_recv_io()
smb: client: make use of
smbdirect_connection_reassembly_{append,first}_recv_io()
smb: client: make use of smbdirect_connection_idle_timer_work()
smb: client: make use of smbdirect_frwr_is_supported()
smb: client: make use of smbdirect_connection_{alloc,free}_send_io()
smb: client: make use of smbdirect_connection_send_io_done()
smb: client: make use of
smbdirect_connection_{create,destroy}_mem_pools()
smb: client: make use of smbdirect_map_sges_from_iter()
smb: client: make use of smbdirect_connection_qp_event_handler()
smb: client: make use of
smbdirect_connection_negotiate_rdma_resources()
smb: client: make use of smbdirect_connection_{create,destroy}_qp()
smb: client: initialize recv_io->cqe.done = recv_done just once
smb: client: make use of smbdirect_connection_post_recv_io()
smb: client: make use of smbdirect_connection_recv_io_refill_work()
smb: client: make use of functions from smbdirect_mr.c
smb: client: make use of smbdirect_socket_destroy_sync()
smb: client: make use of smbdirect_connection_recvmsg()
smb: client: let smbd_post_send() make use of request->wr
smb: client: make use of smbdirect_connection_grant_recv_credits()
smb: client: make use of smbdirect_connection_request_keep_alive()
smb: client: change smbd_post_send_empty() to void return
smb: client: let smbd_post_send_iter() get remaining_length and return
data_length
smb: client: let smbd_post_send_full_iter() get remaining_length and
return data_length
smb: client: make use of
smbdirect_connection_send_{single_iter,immediate_work}()
smb: client: introduce and use smbd_mr_fill_buffer_descriptor()
smb: client: introduce and use smbd_debug_proc_show()
smb: client: make use of smbdirect_socket_init_new() and
smbdirect_connect_sync()
smb: client: make use of
smbdirect_socket_create_kern()/smbdirect_socket_release()
smb: client: only use public smbdirect functions
smb: client: make use of smbdirect.ko
smb: server: make use of smbdirect_socket_prepare_create()
smb: server: make use of smbdirect_socket_set_logging()
smb: server: make use of smbdirect_socket_wake_up_all()
smb: server: let smb_direct_create_pools() initialize complex_work as
disabled
smb: server: make use of smbdirect_socket_cleanup_work()
smb: server: make use of smbdirect_socket_schedule_cleanup()
smb: server: make use of smbdirect_connection_{get,put}_recv_io()
smb: server: make use of
smbdirect_connection_reassembly_{append,first}_recv_io()
smb: server: make use of smbdirect_connection_idle_timer_work()
smb: server: make use of smbdirect_frwr_is_supported()
smb: server: make use of smbdirect_connection_{alloc,free}_send_io()
smb: server: make use of smbdirect_connection_send_io_done()
smb: server: make use of
smbdirect_connection_{create,destroy}_mem_pools()
smb: server: make use of smbdirect_map_sges_from_iter()
smb: server: make use of smbdirect_connection_qp_event_handler()
smb: server: make use of
smbdirect_connection_negotiate_rdma_resources()
smb: server: make use of smbdirect_connection_{create,destroy}_qp()
smb: server: initialize recv_io->cqe.done = recv_done just once
smb: server: make use of smbdirect_connection_post_recv_io()
smb: server: make use of smbdirect_connection_recv_io_refill[_work]()
smb: server: make use of smbdirect_get_buf_page_count()
smb: server: make use of smbdirect_socket_wait_for_credits()
smb: server: make use of functions from smbdirect_rw.c
smb: server: make use of smbdirect_socket_destroy_sync()
smb: server: make use of smbdirect_connection_recvmsg()
smb: server: make use of smbdirect_connection_grant_recv_credits()
smb: server: make use of smbdirect_connection_request_keep_alive()
smb: server: move iov_iter_kvec() out of smb_direct_post_send_data()
smb: server: inline smb_direct_create_header() into
smb_direct_post_send_data()
smb: server: let smbdirect_map_sges_from_iter() truncate the message
boundary
smb: server: split out smb_direct_send_iter() out of
smb_direct_writev()
smb: server: let smb_direct_post_send_data() return data_length
smb: server: make use of smbdirect_connection_send_iter() and related
functions
smb: server: make use of
smbdirect_{socket_init_accepting,connection_wait_for_connected}()
smb: server: make use of
smbdirect_socket_create_accepting()/smbdirect_socket_release()
smb: server: only use public smbdirect functions
smb: server: make use of smbdirect.ko
smb: smbdirect: wrap rdma_disconnect() in rdma_[un]lock_handler()
smb: smbdirect: remove unused
smbdirect_connection_mr_io_recovery_work()
smb: smbdirect: prepare use of dedicated workqueues for different
steps
smb: smbdirect: introduce global workqueues
smb: client: no longer use smbdirect_socket_set_custom_workqueue()
smb: server: no longer use smbdirect_socket_set_custom_workqueue()
smb: smbdirect: split out smbdirect_accept_negotiate_finish()
smb: smbdirect: introduce smbdirect_socket_bind()
smb: smbdirect: introduce smbdirect_socket_{listen,accept}()
smb: server: make use of smbdirect_socket_{listen,accept}()
smb: server: remove unused ksmbd_transport_ops.prepare()
smb: smbdirect: remove unused SMBDIRECT_USE_INLINE_C_FILES logic
fs/smb/Kconfig | 2 + fs/smb/client/Kconfig | 3 +- fs/smb/client/cifs_debug.c | 67 +- fs/smb/client/smb2pdu.c | 9 +- fs/smb/client/smbdirect.c | 2802 +---------------- fs/smb/client/smbdirect.h | 18 +- fs/smb/common/Makefile | 1 + fs/smb/common/smbdirect/Kconfig | 9 + fs/smb/common/smbdirect/Makefile | 17 + fs/smb/common/smbdirect/smbdirect.h | 2 + fs/smb/common/smbdirect/smbdirect_accept.c | 786 +++++ fs/smb/common/smbdirect/smbdirect_connect.c | 842 +++++ .../common/smbdirect/smbdirect_connection.c | 2035 ++++++++++++ fs/smb/common/smbdirect/smbdirect_debug.c | 88 + fs/smb/common/smbdirect/smbdirect_internal.h | 115 + fs/smb/common/smbdirect/smbdirect_listen.c | 284 ++ fs/smb/common/smbdirect/smbdirect_main.c | 112 + fs/smb/common/smbdirect/smbdirect_mr.c | 486 +++ fs/smb/common/smbdirect/smbdirect_pdu.h | 4 + fs/smb/common/smbdirect/smbdirect_public.h | 125 + fs/smb/common/smbdirect/smbdirect_rw.c | 255 ++ fs/smb/common/smbdirect/smbdirect_socket.c | 756 +++++ fs/smb/common/smbdirect/smbdirect_socket.h | 311 +- fs/smb/server/Kconfig | 4 +- fs/smb/server/connection.c | 5 - fs/smb/server/connection.h | 1 - fs/smb/server/smb2pdu.c | 1 - fs/smb/server/transport_rdma.c | 2659 ++-------------- fs/smb/server/transport_rdma.h | 4 +- 29 files changed, 6633 insertions(+), 5170 deletions(-) create mode 100644 fs/smb/common/smbdirect/Kconfig create mode 100644 fs/smb/common/smbdirect/Makefile create mode 100644 fs/smb/common/smbdirect/smbdirect_accept.c create mode 100644 fs/smb/common/smbdirect/smbdirect_connect.c create mode 100644 fs/smb/common/smbdirect/smbdirect_connection.c create mode 100644 fs/smb/common/smbdirect/smbdirect_debug.c create mode 100644 fs/smb/common/smbdirect/smbdirect_internal.h create mode 100644 fs/smb/common/smbdirect/smbdirect_listen.c create mode 100644 fs/smb/common/smbdirect/smbdirect_main.c create mode 100644 fs/smb/common/smbdirect/smbdirect_mr.c create mode 100644 fs/smb/common/smbdirect/smbdirect_public.h create mode 100644 fs/smb/common/smbdirect/smbdirect_rw.c create mode 100644 fs/smb/common/smbdirect/smbdirect_socket.c
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 1/145 smb: smbdirect: let smbdirect.h include #include <linux/types.h>
This will make it easier to use.
--- fs/smb/common/smbdirect/smbdirect.h | 2 ++ 1 file changed, 2 insertions(+)
fs/smb/common/smbdirect/smbdirect.h +2 -0
@@ -7,6 +7,8 @@ #ifndef __FS_SMB_COMMON_SMBDIRECT_SMBDIRECT_H__ #define __FS_SMB_COMMON_SMBDIRECT_SMBDIRECT_H__ +#include <linux/types.h> + /* SMB-DIRECT buffer descriptor V1 structure [MS-SMBD] 2.2.3.1 */ struct smbdirect_buffer_descriptor_v1 { __le64 offset;
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 2/145 smb: smbdirect: introduce smbdirect_socket.logging infrastructure
In reply to Stefan MetzmacherThis will be used by client and server in order to keep controlling the logging when we move to shared functions.
--- fs/smb/common/smbdirect/smbdirect_socket.h | 127 +++++++++++++++++++++ 1 file changed, 127 insertions(+)
fs/smb/common/smbdirect/smbdirect_socket.h +127 -0
@@ -330,6 +330,35 @@ struct smbdirect_socket {u64 dequeue_reassembly_queue;u64 send_empty; } statistics; + +struct { +#define SMBDIRECT_LOG_ERR 0x0 +#define SMBDIRECT_LOG_INFO 0x1 + +#define SMBDIRECT_LOG_OUTGOING 0x1 +#define SMBDIRECT_LOG_INCOMING 0x2 +#define SMBDIRECT_LOG_READ 0x4 +#define SMBDIRECT_LOG_WRITE 0x8 +#define SMBDIRECT_LOG_RDMA_SEND 0x10 +#define SMBDIRECT_LOG_RDMA_RECV 0x20 +#define SMBDIRECT_LOG_KEEP_ALIVE 0x40 +#define SMBDIRECT_LOG_RDMA_EVENT 0x80 +#define SMBDIRECT_LOG_RDMA_MR 0x100 +#define SMBDIRECT_LOG_RDMA_RW 0x200 +#define SMBDIRECT_LOG_NEGOTIATE 0x400 +void *private_ptr; +bool (*needed)(struct smbdirect_socket *sc, +void *private_ptr, +unsigned int lvl, +unsigned int cls); +void (*vaprintf)(struct smbdirect_socket *sc, +const char *func, +unsigned int line, +void *private_ptr, +unsigned int lvl, +unsigned int cls, +struct va_format *vaf); + } logging; };static void __smbdirect_socket_disabled_work (struct work_struct *work)
@@ -340,6 +369,100 @@ static void __smbdirect_socket_disabled_work(struct work_struct *work)WARN_ON_ONCE (1 ); } +static bool __smbdirect_log_needed (struct smbdirect_socket *sc, +void *private_ptr, +unsigned int lvl, +unsigned int cls) +{ +/* +* Should never be called, the caller should +* set it's own functions. +*/ +WARN_ON_ONCE (1 ); +return false; +} + +static void __smbdirect_log_vaprintf (struct smbdirect_socket *sc, +const char *func, +unsigned int line, +void *private_ptr, +unsigned int lvl, +unsigned int cls, +struct va_format *vaf) +{ +/* +* Should never be called, the caller should +* set it's own functions. +*/ +WARN_ON_ONCE (1 ); +} + +__printf(6, 7) +static void __smbdirect_log_printf (struct smbdirect_socket *sc, +const char *func, +unsigned int line, +unsigned int lvl, +unsigned int cls, +const char *fmt, + ...); +__maybe_unused +static void __smbdirect_log_printf (struct smbdirect_socket *sc, +const char *func, +unsigned int line, +unsigned int lvl, +unsigned int cls, +const char *fmt, + ...) +{ +struct va_format vaf; +va_list args; + +va_start (args, fmt); + + vaf.fmt = fmt; + vaf.va = &args; + + sc->logging.vaprintf (sc, + func, + line, + sc->logging.private_ptr, + lvl, + cls, + &vaf); +va_end (args); +} + +#define ___smbdirect_log_generic(sc, func, line, lvl, cls, fmt, args...) do { \ + if (sc->logging.needed(sc, sc->logging.private_ptr, lvl, cls)) { \ + __smbdirect_log_printf(sc, func, line, lvl, cls, fmt, ##args); \ + } \ +} while (0) +#define __smbdirect_log_generic(sc, lvl, cls, fmt, args...) \ + ___smbdirect_log_generic(sc, __func__, __LINE__, lvl, cls, fmt, ##args) + +#define smbdirect_log_outgoing(sc, lvl, fmt, args...) \ + __smbdirect_log_generic(sc, lvl, SMBDIRECT_LOG_OUTGOING, fmt, ##args) +#define smbdirect_log_incoming(sc, lvl, fmt, args...) \ + __smbdirect_log_generic(sc, lvl, SMBDIRECT_LOG_INCOMING, fmt, ##args) +#define smbdirect_log_read(sc, lvl, fmt, args...) \ + __smbdirect_log_generic(sc, lvl, SMBDIRECT_LOG_READ, fmt, ##args) +#define smbdirect_log_write(sc, lvl, fmt, args...) \ + __smbdirect_log_generic(sc, lvl, SMBDIRECT_LOG_WRITE, fmt, ##args) +#define smbdirect_log_rdma_send(sc, lvl, fmt, args...) \ + __smbdirect_log_generic(sc, lvl, SMBDIRECT_LOG_RDMA_SEND, fmt, ##args) +#define smbdirect_log_rdma_recv(sc, lvl, fmt, args...) \ + __smbdirect_log_generic(sc, lvl, SMBDIRECT_LOG_RDMA_RECV, fmt, ##args) +#define smbdirect_log_keep_alive(sc, lvl, fmt, args...) \ + __smbdirect_log_generic(sc, lvl, SMBDIRECT_LOG_KEEP_ALIVE, fmt, ##args) +#define smbdirect_log_rdma_event(sc, lvl, fmt, args...) \ + __smbdirect_log_generic(sc, lvl, SMBDIRECT_LOG_RDMA_EVENT, fmt, ##args) +#define smbdirect_log_rdma_mr(sc, lvl, fmt, args...) \ + __smbdirect_log_generic(sc, lvl, SMBDIRECT_LOG_RDMA_MR, fmt, ##args) +#define smbdirect_log_rdma_rw(sc, lvl, fmt, args...) \ + __smbdirect_log_generic(sc, lvl, SMBDIRECT_LOG_RDMA_RW, fmt, ##args) +#define smbdirect_log_negotiate(sc, lvl, fmt, args...) \ + __smbdirect_log_generic(sc, lvl, SMBDIRECT_LOG_NEGOTIATE, fmt, ##args) + static __always_inline void smbdirect_socket_init(struct smbdirect_socket *sc) {/*
@@ -392,6 +515,10 @@ static __always_inline void smbdirect_socket_init(struct smbdirect_socket *sc)INIT_WORK (&sc->mr_io.recovery_work, __smbdirect_socket_disabled_work);disable_work_sync (&sc->mr_io.recovery_work);init_waitqueue_head (&sc->mr_io.cleanup.wait_queue); + + sc->logging.private_ptr =NULL ; + sc->logging.needed = __smbdirect_log_needed; + sc->logging.vaprintf = __smbdirect_log_vaprintf; }#define __SMBDIRECT_CHECK_STATUS_FAILED (__sc, __expected_status, __error_cmd, __unexpected_cmd) ({ \
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 3/145 smb: smbdirect: add some logging to SMBDIRECT_CHECK_STATUS_{WARN,DISCONNECT}()
In reply to Stefan MetzmacherThis should make it easier to analyze any possible problems.
--- fs/smb/common/smbdirect/smbdirect_socket.h | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-)
fs/smb/common/smbdirect/smbdirect_socket.h +20 -1
@@ -535,7 +535,6 @@ static __always_inline void smbdirect_socket_init(struct smbdirect_socket *sc)#define __SMBDIRECT_CHECK_STATUS_WARN (__sc, __expected_status, __unexpected_cmd) \ __SMBDIRECT_CHECK_STATUS_FAILED(__sc, __expected_status, \ - , \ { \ const struct sockaddr_storage *__src = NULL; \ const struct sockaddr_storage *__dst = NULL; \
@@ -543,6 +542,26 @@ static __always_inline void smbdirect_socket_init(struct smbdirect_socket *sc) __src = &(__sc)->rdma.cm_id->route.addr.src_addr; \ __dst = &(__sc)->rdma.cm_id->route.addr.dst_addr; \ } \ +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_INFO , \ +"expected[%s] != %s first_error=%1pe local=%pISpsfc remote=%pISpsfc\n" , \ +smbdirect_socket_status_string (__expected_status), \ +smbdirect_socket_status_string ((__sc)->status), \ +SMBDIRECT_DEBUG_ERR_PTR ((__sc)->first_error), \ + __src, __dst); \ + }, \ + { \ +const struct sockaddr_storage *__src =NULL ; \ +const struct sockaddr_storage *__dst =NULL ; \ +if ((__sc)->rdma.cm_id) { \ + __src = &(__sc)->rdma.cm_id->route.addr.src_addr; \ + __dst = &(__sc)->rdma.cm_id->route.addr.dst_addr; \ + } \ +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , \ +"expected[%s] != %s first_error=%1pe local=%pISpsfc remote=%pISpsfc\n" , \ +smbdirect_socket_status_string (__expected_status), \ +smbdirect_socket_status_string ((__sc)->status), \ +SMBDIRECT_DEBUG_ERR_PTR ((__sc)->first_error), \ + __src, __dst); \ WARN_ONCE(1, \ "expected[%s] != %s first_error=%1pe local=%pISpsfc remote=%pISpsfc\n", \ smbdirect_socket_status_string(__expected_status), \
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 4/145 smb: smbdirect: introduce smbdirect_recv_io.complex_work
In reply to Stefan MetzmacherThis will be used by client and server in order to implement async connect/accept logic.
The problems is that ib_post_recv() completions can run in an interrupt context, where we are not allowed to sleep.
During the negotiation phase we need to do complex things e.g. allocate memory, which might sleep, so we'll offload this to the workqueue instead.
--- fs/smb/common/smbdirect/smbdirect_socket.h | 8 ++++++++ 1 file changed, 8 insertions(+)
fs/smb/common/smbdirect/smbdirect_socket.h +8 -0
@@ -631,6 +631,14 @@ struct smbdirect_recv_io {#define SMBDIRECT_RECV_IO_MAX_SGE 1struct ib_sge sge; +/* +* We may need to handle complex +* work that might sleep and are +* not allowed to run in an interrupt +* context. +*/ +struct work_struct complex_work; +/* Link to free or reassembly list */ struct list_head list;
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 5/145 smb: smbdirect: introduce smbdirect_all_c_files.c
In reply to Stefan MetzmacherThis is a very basic start in order to introduce common functions, which will be shared by client and server.
As a start smbdirect_all_c_files.c will be included in fs/smb/client/smbdirect.c and fs/smb/server/transport_rdma.c in order to allow tiny steps in the direction of moving to a few exported functions from an smbdirect.ko.
Step by step this will include individual c files
with the real functions.
--- fs/smb/common/smbdirect/smbdirect_all_c_files.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 fs/smb/common/smbdirect/smbdirect_all_c_files.c
fs/smb/common/smbdirect/smbdirect_all_c_files.c new file +17 -0
@@ -0,0 +1,17 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (c) 2025, Stefan Metzmacher +*/ + +/* +* This is a temporary solution in order +* to include the common smbdirect functions +* into .c files in order to make a transformation +* in tiny bisectable steps possible. +* +* It will be replaced by a smbdirect.ko with +* exported public functions at the end. +*/ +#ifndef SMBDIRECT_USE_INLINE_C_FILES +#error SMBDIRECT_USE_INLINE_C_FILES define needed +#endif
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 6/145 smb: smbdirect: introduce smbdirect_internal.h
In reply to Stefan MetzmacherThis will be included by individual .c files as first header.
--- fs/smb/common/smbdirect/smbdirect_internal.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 fs/smb/common/smbdirect/smbdirect_internal.h
fs/smb/common/smbdirect/smbdirect_internal.h new file +14 -0
@@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* Copyright (c) 2025, Stefan Metzmacher +*/ + +#ifndef __FS_SMB_COMMON_SMBDIRECT_INTERNAL_H__ +#define __FS_SMB_COMMON_SMBDIRECT_INTERNAL_H__ + +#include <linux/errname.h> +#include "smbdirect.h" +#include "smbdirect_pdu.h" +#include "smbdirect_socket.h" + +#endif /* __FS_SMB_COMMON_SMBDIRECT_INTERNAL_H__ */
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 7/145 smb: client: include smbdirect_all_c_files.c
In reply to Stefan MetzmacherThis is the first tiny step in order to use common functions in future. Once we have all functions in common we'll move to an smbdirect.ko that exports public functions instead of including the .c file.
--- fs/smb/client/smbdirect.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
fs/smb/client/smbdirect.c +13 -0
@@ -4,6 +4,9 @@* * Author(s): Long Li <[email protected]> */ + +#define SMBDIRECT_USE_INLINE_C_FILES 1 +#include <linux/module.h> #include <linux/highmem.h> #include <linux/folio_queue.h>
@@ -143,6 +146,16 @@ module_param(smbd_logging_level, uint, 0644);MODULE_PARM_DESC (smbd_logging_level,"Logging level for SMBD transport, 0 (default): error, 1: info" ); +static void smbd_disconnect_rdma_connection (struct smbdirect_socket *sc); + +/* +* This is a temporary solution until all code +* is moved to smbdirect_all_c_files.c and we +* have an smbdirect.ko that exports the required +* functions. +*/ +#include "../common/smbdirect/smbdirect_all_c_files.c" + #define log_rdma(level, class, fmt, args...) \ do { \ if (level <= smbd_logging_level || class & smbd_logging_class) \
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 8/145 smb: server: include smbdirect_all_c_files.c
In reply to Stefan MetzmacherThis is the first tiny step in order to use common functions in future.
Once we have all functions in common we'll move to an smbdirect.ko that exports public functions instead of including the .c file.
--- fs/smb/server/transport_rdma.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
fs/smb/server/transport_rdma.c +12 -0
@@ -9,6 +9,8 @@#define SUBMOD_NAME "smb_direct" +#define SMBDIRECT_USE_INLINE_C_FILES 1 +#include <linux/kthread.h> #include <linux/list.h> #include <linux/mempool.h>
@@ -30,6 +32,16 @@#include "../common/smbdirect/smbdirect_socket.h" #include "transport_rdma.h" +static void smb_direct_disconnect_rdma_connection (struct smbdirect_socket *sc); + +/* +* This is a temporary solution until all code +* is moved to smbdirect_all_c_files.c and we +* have an smbdirect.ko that exports the required +* functions. +*/ +#include "../common/smbdirect/smbdirect_all_c_files.c" +#define SMB_DIRECT_PORT_IWARP 5445#define SMB_DIRECT_PORT_INFINIBAND 445
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 9/145 smb: smbdirect: introduce smbdirect_socket.c to be filled
In reply to Stefan MetzmacherOver time smbdirect_socket.c will get more and more functions which will be included in fs/smb/client/smbdirect.c and fs/smb/server/transport_rdma.c via fs/smb/common/smbdirect/smbdirect_all_c_files.c
in order to allow tiny steps in the direction of moving to a few exported functions from an smbdirect.ko. That's why __maybe_unused is added for now it will be removed at the end of the road to common code.
Note the Copyright (C) 2017, Microsoft Corporation is added as a lot of functions from fs/smb/client/smbdirect.c will be moved into this file soon and I don't want to forget about adding it.
--- fs/smb/common/smbdirect/smbdirect_all_c_files.c | 1 + fs/smb/common/smbdirect/smbdirect_socket.c | 7 +++++++ 2 files changed, 8 insertions(+) create mode 100644 fs/smb/common/smbdirect/smbdirect_socket.c
fs/smb/common/smbdirect/smbdirect_all_c_files.c +1 -0
@@ -15,3 +15,4 @@#ifndef SMBDIRECT_USE_INLINE_C_FILES #error SMBDIRECT_USE_INLINE_C_FILES define needed#endif +#include "smbdirect_socket.c"
fs/smb/common/smbdirect/smbdirect_socket.c new file +7 -0
@@ -0,0 +1,7 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2017, Microsoft Corporation. +* Copyright (c) 2025, Stefan Metzmacher +*/ + +#include "smbdirect_internal.h"
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 10/145 smb: smbdirect: introduce smbdirect_socket_prepare_create()
In reply to Stefan MetzmacherThis will be used by client and server until we reach the point where we have only public function from an smbdirect.ko.
--- fs/smb/common/smbdirect/smbdirect_socket.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
fs/smb/common/smbdirect/smbdirect_socket.c +19 -0
@@ -5,3 +5,22 @@*/ #include "smbdirect_internal.h" + +__maybe_unused /* this is temporary while this file is included in others */ +static void smbdirect_socket_prepare_create (struct smbdirect_socket *sc, +const struct smbdirect_socket_parameters *sp, +struct workqueue_struct *workqueue) +{ +smbdirect_socket_init (sc); + +/* +* Make a copy of the callers parameters +* from here we only work on the copy +*/ + sc->parameters = *sp; + +/* +* Remember the callers workqueue +*/ + sc->workqueue = workqueue; +}
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 11/145 smb: smbdirect: introduce smbdirect_socket_set_logging()
In reply to Stefan MetzmacherThis will be used by client and server in order to setup their own logging functions.
--- fs/smb/common/smbdirect/smbdirect_socket.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
fs/smb/common/smbdirect/smbdirect_socket.c +20 -0
@@ -24,3 +24,23 @@ static void smbdirect_socket_prepare_create(struct smbdirect_socket *sc,*/ sc->workqueue = workqueue; } + +__maybe_unused /* this is temporary while this file is included in others */ +static void smbdirect_socket_set_logging (struct smbdirect_socket *sc, +void *private_ptr, +bool (*needed)(struct smbdirect_socket *sc, +void *private_ptr, +unsigned int lvl, +unsigned int cls), +void (*vaprintf)(struct smbdirect_socket *sc, +const char *func, +unsigned int line, +void *private_ptr, +unsigned int lvl, +unsigned int cls, +struct va_format *vaf)) +{ + sc->logging.private_ptr = private_ptr; + sc->logging.needed = needed; + sc->logging.vaprintf = vaprintf; +}
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 12/145 smb: smbdirect: introduce smbdirect_socket_wake_up_all()
In reply to Stefan MetzmacherThis is a superset of smbd_disconnect_wake_up_all() in the client and smb_direct_disconnect_wake_up_all() in the server and will replace them.
--- fs/smb/common/smbdirect/smbdirect_socket.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
fs/smb/common/smbdirect/smbdirect_socket.c +18 -0
@@ -44,3 +44,21 @@ static void smbdirect_socket_set_logging(struct smbdirect_socket *sc, sc->logging.needed = needed; sc->logging.vaprintf = vaprintf; } + +__maybe_unused /* this is temporary while this file is included in others */ +static void smbdirect_socket_wake_up_all (struct smbdirect_socket *sc) +{ +/* +* Wake up all waiters in all wait queues +* in order to notice the broken connection. +*/ +wake_up_all (&sc->status_wait); +wake_up_all (&sc->send_io.lcredits.wait_queue); +wake_up_all (&sc->send_io.credits.wait_queue); +wake_up_all (&sc->send_io.pending.dec_wait_queue); +wake_up_all (&sc->send_io.pending.zero_wait_queue); +wake_up_all (&sc->recv_io.reassembly.wait_queue); +wake_up_all (&sc->rw_io.credits.wait_queue); +wake_up_all (&sc->mr_io.ready.wait_queue); +wake_up_all (&sc->mr_io.cleanup.wait_queue); +}
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 13/145 smb: smbdirect: introduce smbdirect_socket_cleanup_work()
In reply to Stefan MetzmacherThis is basically a copy of smbd_disconnect_rdma_work() and smb_direct_disconnect_rdma_work() and will replace them in the next steps.
Differences is that a message is logged if first error is still 0, which makes it easier to analyze problems.
And also disable any complex work from recv_io objects, currently these are not used and the work is always disabled anyway, but this prepares future changes.
It also makes sure it's never used in an interrupt, which is not expected anyway...
--- fs/smb/common/smbdirect/smbdirect_socket.c | 90 +++++++++++++++++++++- 1 file changed, 89 insertions(+), 1 deletion(-)
fs/smb/common/smbdirect/smbdirect_socket.c +89 -1
@@ -6,6 +6,8 @@#include "smbdirect_internal.h" +static void smbdirect_socket_cleanup_work (struct work_struct *work); +__maybe_unused /* this is temporary while this file is included in others */ static void smbdirect_socket_prepare_create(struct smbdirect_socket *sc, const struct smbdirect_socket_parameters *sp,
@@ -23,6 +25,8 @@ static void smbdirect_socket_prepare_create(struct smbdirect_socket *sc,* Remember the callers workqueue */ sc->workqueue = workqueue; + +INIT_WORK (&sc->disconnect_work, smbdirect_socket_cleanup_work); }__maybe_unused /* this is temporary while this file is included in others */
@@ -45,7 +49,6 @@ static void smbdirect_socket_set_logging(struct smbdirect_socket *sc, sc->logging.vaprintf = vaprintf; } -__maybe_unused /* this is temporary while this file is included in others */ static void smbdirect_socket_wake_up_all (struct smbdirect_socket *sc) {/*
@@ -62,3 +65,88 @@ static void smbdirect_socket_wake_up_all(struct smbdirect_socket *sc)wake_up_all (&sc->mr_io.ready.wait_queue);wake_up_all (&sc->mr_io.cleanup.wait_queue); } + +static void smbdirect_socket_cleanup_work (struct work_struct *work) +{ +struct smbdirect_socket *sc = + container_of(work, struct smbdirect_socket, disconnect_work); +struct smbdirect_recv_io *recv_io, *recv_tmp; +unsigned long flags; + +/* +* This should not never be called in an interrupt! +*/ +WARN_ON_ONCE (in_interrupt ()); + +if (!sc->first_error) { +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"%s called with first_error==0\n" , +smbdirect_socket_status_string (sc->status)); + + sc->first_error = -ECONNABORTED ; + } + +/* +* make sure this and other work is not queued again +* but here we don't block and avoid +* disable[_delayed]_work_sync() +*/ +disable_work (&sc->disconnect_work); +disable_work (&sc->recv_io.posted.refill_work); +disable_work (&sc->mr_io.recovery_work); +disable_work (&sc->idle.immediate_work); +disable_delayed_work (&sc->idle.timer_work); + +/* +* If any complex work was scheduled we +* should disable it (only happens during +* negotiation)... +* +* Note that sc->first_error is set before, +* so any future smbdirect_connection_get_recv_io() +* will see it and return NULL. +*/ +spin_lock_irqsave (&sc->recv_io.free.lock, flags); +list_for_each_entry_safe (recv_io, recv_tmp, &sc->recv_io.free.list, list) +disable_work (&recv_io->complex_work); +spin_unlock_irqrestore (&sc->recv_io.free.lock, flags); + +switch (sc->status) { +case SMBDIRECT_SOCKET_NEGOTIATE_NEEDED : +case SMBDIRECT_SOCKET_NEGOTIATE_RUNNING : +case SMBDIRECT_SOCKET_NEGOTIATE_FAILED : +case SMBDIRECT_SOCKET_CONNECTED : +case SMBDIRECT_SOCKET_ERROR : + sc->status =SMBDIRECT_SOCKET_DISCONNECTING ; +rdma_disconnect (sc->rdma.cm_id); +break ; + +case SMBDIRECT_SOCKET_CREATED : +case SMBDIRECT_SOCKET_RESOLVE_ADDR_NEEDED : +case SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING : +case SMBDIRECT_SOCKET_RESOLVE_ADDR_FAILED : +case SMBDIRECT_SOCKET_RESOLVE_ROUTE_NEEDED : +case SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING : +case SMBDIRECT_SOCKET_RESOLVE_ROUTE_FAILED : +case SMBDIRECT_SOCKET_RDMA_CONNECT_NEEDED : +case SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING : +case SMBDIRECT_SOCKET_RDMA_CONNECT_FAILED : +/* +* rdma_{accept,connect}() never reached +* RDMA_CM_EVENT_ESTABLISHED +*/ + sc->status =SMBDIRECT_SOCKET_DISCONNECTED ; +break ; + +case SMBDIRECT_SOCKET_DISCONNECTING : +case SMBDIRECT_SOCKET_DISCONNECTED : +case SMBDIRECT_SOCKET_DESTROYED : +break ; + } + +/* +* Wake up all waiters in all wait queues +* in order to notice the broken connection. +*/ +smbdirect_socket_wake_up_all (sc); +}
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 14/145 smb: smbdirect: introduce smbdirect_socket_schedule_cleanup[{_lvl,_status}]()
In reply to Stefan Metzmachersmbdirect_socket_schedule_cleanup() is more
or less copy of smbd_disconnect_rdma_connection() and smb_direct_disconnect_rdma_connection(). It will replace them in the next steps.
A difference is that the location of the first error is logged, which makes it easier to analyze problems.
And also disable any complex work from recv_io objects, currently these are not used and the work is always disabled anyway, but this prepares future changes.
It also gets an explicit error passed
in instead of hardcoding -ECONNABORTED.
Beside the main smbdirect_socket_schedule_cleanup() there are some special additions:
- smbdirect_socket_schedule_cleanup_lvl(), will be used for cases where we don't want a log message with SMBDIRECT_LOG_ERR.
- smbdirect_socket_schedule_cleanup_status(),
will be used to specify the log level together with a direct final status, for the RDMA_CM_EVENT_DEVICE_REMOVAL and RDMA_CM_EVENT_DISCONNECTED cases where we need to avoid SMBDIRECT_SOCKET_DISCONNECTING and rdma_disconnect() in smbdirect_socket_cleanup_work().
With this we're also able to define a default for __SMBDIRECT_SOCKET_DISCONNECT() just using:smbdirect_socket_schedule_cleanup(__sc, -ECONNABORTED)
--- fs/smb/common/smbdirect/smbdirect_internal.h | 22 ++++ fs/smb/common/smbdirect/smbdirect_socket.c | 110 +++++++++++++++++++ fs/smb/common/smbdirect/smbdirect_socket.h | 5 + 3 files changed, 137 insertions(+)
fs/smb/common/smbdirect/smbdirect_internal.h +22 -0
@@ -11,4 +11,26 @@#include "smbdirect_pdu.h" #include "smbdirect_socket.h" +static void __smbdirect_socket_schedule_cleanup (struct smbdirect_socket *sc, +const char *macro_name, +unsigned int lvl, +const char *func, +unsigned int line, +int error, +enum smbdirect_socket_status *force_status); +#define smbdirect_socket_schedule_cleanup (__sc, __error) \ + __smbdirect_socket_schedule_cleanup(__sc, \ + "smbdirect_socket_schedule_cleanup", SMBDIRECT_LOG_ERR, \ + __func__, __LINE__, __error, NULL) +#define smbdirect_socket_schedule_cleanup_lvl (__sc, __lvl, __error) \ + __smbdirect_socket_schedule_cleanup(__sc, \ + "smbdirect_socket_schedule_cleanup_lvl", __lvl, \ + __func__, __LINE__, __error, NULL) +#define smbdirect_socket_schedule_cleanup_status (__sc, __lvl, __error, __status) do { \ + enum smbdirect_socket_status __force_status = __status; \ + __smbdirect_socket_schedule_cleanup(__sc, \ + "smbdirect_socket_schedule_cleanup_status", __lvl, \ + __func__, __LINE__, __error, &__force_status); \ +} while (0) +#endif /* __FS_SMB_COMMON_SMBDIRECT_INTERNAL_H__ */
fs/smb/common/smbdirect/smbdirect_socket.c +110 -0
116 lines not on this page · raw message
fs/smb/common/smbdirect/smbdirect_socket.h +5 -0
@@ -574,6 +574,11 @@ static __always_inline void smbdirect_socket_init(struct smbdirect_socket *sc) #define SMBDIRECT_CHECK_STATUS_WARN(__sc, __expected_status) \ __SMBDIRECT_CHECK_STATUS_WARN(__sc, __expected_status, /* nothing */) +#ifndef __SMBDIRECT_SOCKET_DISCONNECT +#define __SMBDIRECT_SOCKET_DISCONNECT (__sc) \ + smbdirect_socket_schedule_cleanup(__sc, -ECONNABORTED) +#endif /* ! __SMBDIRECT_SOCKET_DISCONNECT */ +#define SMBDIRECT_CHECK_STATUS_DISCONNECT (__sc, __expected_status) \ __SMBDIRECT_CHECK_STATUS_WARN(__sc, __expected_status, \ __SMBDIRECT_SOCKET_DISCONNECT(__sc);)
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 15/145 smb: smbdirect: introduce smbdirect_connection.c to be filled
In reply to Stefan MetzmacherOver time smbdirect_connection.c will get more and more functions which will be included in fs/smb/client/smbdirect.c and fs/smb/server/transport_rdma.c via fs/smb/common/smbdirect/smbdirect_all_c_files.c
in order to allow tiny steps in the direction of moving to a few exported functions from an smbdirect.ko.
That's why __maybe_unused is added for now it will
be removed at the end of the road to common code.
Note the Copyright (C) 2017, Microsoft Corporation is added as a lot of functions from fs/smb/client/smbdirect.c will be moved into this file soon and I don't want to forget about adding it.
--- fs/smb/common/smbdirect/smbdirect_all_c_files.c | 1 + fs/smb/common/smbdirect/smbdirect_connection.c | 7 +++++++ 2 files changed, 8 insertions(+) create mode 100644 fs/smb/common/smbdirect/smbdirect_connection.c
fs/smb/common/smbdirect/smbdirect_all_c_files.c +1 -0
@@ -16,3 +16,4 @@#error SMBDIRECT_USE_INLINE_C_FILES define needed#endif #include "smbdirect_socket.c" +#include "smbdirect_connection.c"
fs/smb/common/smbdirect/smbdirect_connection.c new file +7 -0
@@ -0,0 +1,7 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2017, Microsoft Corporation. +* Copyright (c) 2025, Stefan Metzmacher +*/ + +#include "smbdirect_internal.h"
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 16/145 smb: smbdirect: introduce smbdirect_connection_{get,put}_recv_io()
In reply to Stefan MetzmacherThese are basically copies of {get,put}_receive_buffer() in the client and they are very similar to {get_free,put}_recvmsg() in the server.
The only difference to {get_free,put}_recvmsg() are the updating of the sc->statistics.*.
In addition smbdirect_connection_get_recv_io() uses list_first_entry_or_null() in order to simplify the code. We also only use it on a healthy connection.
smbdirect_connection_put_recv_io() uses msg->socket instead of an explicit argument. And it disables any complex_work.
--- .../common/smbdirect/smbdirect_connection.c | 47 +++++++++++++++++++ 1 file changed, 47 insertions(+)
fs/smb/common/smbdirect/smbdirect_connection.c +47 -0
@@ -5,3 +5,50 @@*/ #include "smbdirect_internal.h" + +__maybe_unused /* this is temporary while this file is included in others */ +static struct smbdirect_recv_io *smbdirect_connection_get_recv_io (struct smbdirect_socket *sc) +{ +struct smbdirect_recv_io *msg =NULL ; +unsigned long flags; + +spin_lock_irqsave (&sc->recv_io.free.lock, flags); +if (likely (!sc->first_error)) + msg =list_first_entry_or_null (&sc->recv_io.free.list, + struct smbdirect_recv_io, + list); +if (likely (msg)) { +list_del (&msg->list); + sc->statistics.get_receive_buffer++; + } +spin_unlock_irqrestore (&sc->recv_io.free.lock, flags); + +return msg; +} + +__maybe_unused /* this is temporary while this file is included in others */ +static void smbdirect_connection_put_recv_io (struct smbdirect_recv_io *msg) +{ +struct smbdirect_socket *sc = msg->socket; +unsigned long flags; + +/* +* Should already be disabled anyway. +*/ +disable_work (&msg->complex_work); + +if (likely (msg->sge.length !=0 )) { +ib_dma_unmap_single (sc->ib.dev, + msg->sge.addr, + msg->sge.length, +DMA_FROM_DEVICE ); + msg->sge.length =0 ; + } + +spin_lock_irqsave (&sc->recv_io.free.lock, flags); +list_add_tail (&msg->list, &sc->recv_io.free.list); + sc->statistics.put_receive_buffer++; +spin_unlock_irqrestore (&sc->recv_io.free.lock, flags); + +queue_work (sc->workqueue, &sc->recv_io.posted.refill_work); +}
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 17/145 smb: smbdirect: introduce smbdirect_connection_reassembly_{append,first}_recv_io()
In reply to Stefan MetzmacherThese are basically copies of enqueue_reassembly() and [_]get_first_reassembly() of both client and server. The only difference is that enqueue_reassembly() of the server does not have:
sc->statistics.enqueue_reassembly_queue++
Also smbdirect_connection_reassembly_first_recv_io() makes use of list_first_entry_or_null() in order to simplify the code.
In the next commits they will replace the existing functions.
--- .../common/smbdirect/smbdirect_connection.c | 40 +++++++++++++++++++ 1 file changed, 40 insertions(+)
fs/smb/common/smbdirect/smbdirect_connection.c +40 -0
@@ -52,3 +52,43 @@ static void smbdirect_connection_put_recv_io(struct smbdirect_recv_io *msg)queue_work (sc->workqueue, &sc->recv_io.posted.refill_work); } + +__maybe_unused /* this is temporary while this file is included in others */ +static void smbdirect_connection_reassembly_append_recv_io (struct smbdirect_socket *sc, +struct smbdirect_recv_io *msg, +u32 data_length) +{ +unsigned long flags; + +/* +* The work should already/still be disabled +*/ +disable_work (&msg->complex_work); + +spin_lock_irqsave (&sc->recv_io.reassembly.lock, flags); +list_add_tail (&msg->list, &sc->recv_io.reassembly.list); + sc->recv_io.reassembly.queue_length++; +/* +* Make sure reassembly_data_length is updated after list and +* reassembly_queue_length are updated. On the dequeue side +* reassembly_data_length is checked without a lock to determine +* if reassembly_queue_length and list is up to date +*/ +virt_wmb (); + sc->recv_io.reassembly.data_length += data_length; +spin_unlock_irqrestore (&sc->recv_io.reassembly.lock, flags); + sc->statistics.enqueue_reassembly_queue++; +} + +__maybe_unused /* this is temporary while this file is included in others */ +static struct smbdirect_recv_io * +smbdirect_connection_reassembly_first_recv_io (struct smbdirect_socket *sc) +{ +struct smbdirect_recv_io *msg; + + msg =list_first_entry_or_null (&sc->recv_io.reassembly.list, + struct smbdirect_recv_io, + list); + +return msg; +}
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 18/145 smb: smbdirect: introduce smbdirect_connection_idle_timer_work()
In reply to Stefan MetzmacherThis is basically a copy of idle_connection_timer() in the client and smb_direct_idle_connection_timer() in the server. The only difference is that the server does not have logging.
Currently the callers set their own timer function after smbdirect_socket_prepare_create(), but that will change in the next steps...
--- .../common/smbdirect/smbdirect_connection.c | 31 +++++++++++++++++++ fs/smb/common/smbdirect/smbdirect_internal.h | 2 ++ fs/smb/common/smbdirect/smbdirect_socket.c | 3 +- 3 files changed, 35 insertions(+), 1 deletion(-)
fs/smb/common/smbdirect/smbdirect_connection.c +31 -0
@@ -92,3 +92,34 @@ smbdirect_connection_reassembly_first_recv_io(struct smbdirect_socket *sc)return msg; } + +static void smbdirect_connection_idle_timer_work (struct work_struct *work) +{ +struct smbdirect_socket *sc = + container_of(work, struct smbdirect_socket, idle.timer_work.work); +const struct smbdirect_socket_parameters *sp = &sc->parameters; + +if (sc->idle.keepalive !=SMBDIRECT_KEEPALIVE_NONE ) { +smbdirect_log_keep_alive (sc,SMBDIRECT_LOG_ERR , +"%s => timeout sc->idle.keepalive=%s\n" , +smbdirect_socket_status_string (sc->status), + sc->idle.keepalive ==SMBDIRECT_KEEPALIVE_SENT ? +"SENT" :"PENDING" ); +smbdirect_socket_schedule_cleanup (sc, -ETIMEDOUT ); +return ; + } + +if (sc->status !=SMBDIRECT_SOCKET_CONNECTED ) +return ; + +/* +* Now use the keepalive timeout (instead of keepalive interval) +* in order to wait for a response +*/ + sc->idle.keepalive =SMBDIRECT_KEEPALIVE_PENDING ; +mod_delayed_work (sc->workqueue, &sc->idle.timer_work, +msecs_to_jiffies (sp->keepalive_timeout_msec)); +smbdirect_log_keep_alive (sc,SMBDIRECT_LOG_INFO , +"schedule send of empty idle message\n" ); +queue_work (sc->workqueue, &sc->idle.immediate_work); +}
fs/smb/common/smbdirect/smbdirect_internal.h +2 -0
@@ -33,4 +33,6 @@ static void __smbdirect_socket_schedule_cleanup(struct smbdirect_socket *sc, __func__, __LINE__, __error, &__force_status); \ } while (0) +static void smbdirect_connection_idle_timer_work(struct work_struct *work); +#endif /* __FS_SMB_COMMON_SMBDIRECT_INTERNAL_H__ */
fs/smb/common/smbdirect/smbdirect_socket.c +2 -1
@@ -27,6 +27,8 @@ static void smbdirect_socket_prepare_create(struct smbdirect_socket *sc, sc->workqueue = workqueue;INIT_WORK (&sc->disconnect_work, smbdirect_socket_cleanup_work); + +INIT_DELAYED_WORK (&sc->idle.timer_work, smbdirect_connection_idle_timer_work); }__maybe_unused /* this is temporary while this file is included in others */
@@ -66,7 +68,6 @@ static void smbdirect_socket_wake_up_all(struct smbdirect_socket *sc)wake_up_all (&sc->mr_io.cleanup.wait_queue); } -__maybe_unused /* this is temporary while this file is included in others */ static void __smbdirect_socket_schedule_cleanup(struct smbdirect_socket *sc, const char *macro_name, unsigned int lvl,
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 19/145 smb: smbdirect: set SMBDIRECT_KEEPALIVE_NONE before disable_delayed_work(&sc->idle.timer_work);
In reply to Stefan MetzmacherThis avoids a potential confusing log message from smbdirect_connection_idle_timer_work() if it's already running.
This is a very small race windows and not really needed, but it feels better when reading the code.
--- fs/smb/common/smbdirect/smbdirect_socket.c | 2 ++ 1 file changed, 2 insertions(+)
fs/smb/common/smbdirect/smbdirect_socket.c +2 -0
@@ -106,6 +106,7 @@ static void __smbdirect_socket_schedule_cleanup(struct smbdirect_socket *sc,disable_work (&sc->recv_io.posted.refill_work);disable_work (&sc->mr_io.recovery_work);disable_work (&sc->idle.immediate_work); + sc->idle.keepalive =SMBDIRECT_KEEPALIVE_NONE ;disable_delayed_work (&sc->idle.timer_work);/*
@@ -206,6 +207,7 @@ static void smbdirect_socket_cleanup_work(struct work_struct *work)disable_work (&sc->recv_io.posted.refill_work);disable_work (&sc->mr_io.recovery_work);disable_work (&sc->idle.immediate_work); + sc->idle.keepalive =SMBDIRECT_KEEPALIVE_NONE ;disable_delayed_work (&sc->idle.timer_work);/*
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 20/145 smb: smbdirect: introduce smbdirect_frwr_is_supported()
In reply to Stefan MetzmacherThis will replace frwr_is_supported() on the client and rdma_frwr_is_supported() on the server.
--- fs/smb/common/smbdirect/smbdirect_socket.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
fs/smb/common/smbdirect/smbdirect_socket.c +16 -0
@@ -6,6 +6,22 @@#include "smbdirect_internal.h" +__maybe_unused /* this is temporary while this file is included in others */ +static bool smbdirect_frwr_is_supported (const struct ib_device_attr *attrs) +{ +/* +* Test if FRWR (Fast Registration Work Requests) is supported on the +* device This implementation requires FRWR on RDMA read/write return +* value: true if it is supported +*/ + +if (!(attrs->device_cap_flags &IB_DEVICE_MEM_MGT_EXTENSIONS )) +return false; +if (attrs->max_fast_reg_page_list_len ==0 ) +return false; +return true; +} +static void smbdirect_socket_cleanup_work (struct work_struct *work);__maybe_unused /* this is temporary while this file is included in others */
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 21/145 smb: smbdirect: introduce smbdirect_socket.{send,recv}_io.mem.gfp_mask
In reply to Stefan MetzmacherThis will allow common code to be split out while still using the gfp_mask currently used.
--- fs/smb/common/smbdirect/smbdirect_socket.h | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)
fs/smb/common/smbdirect/smbdirect_socket.h +10 -4
@@ -150,8 +150,9 @@ struct smbdirect_socket {* smbdirect_send_io buffers */ struct { -struct kmem_cache *cache; -mempool_t *pool; +struct kmem_cache *cache; +mempool_t *pool; +gfp_t gfp_mask; } mem;/*
@@ -204,8 +205,9 @@ struct smbdirect_socket {* smbdirect_recv_io buffers */ struct { -struct kmem_cache *cache; -mempool_t *pool; +struct kmem_cache *cache; +mempool_t *pool; +gfp_t gfp_mask; } mem;/*
@@ -481,6 +483,8 @@ static __always_inline void smbdirect_socket_init(struct smbdirect_socket *sc)INIT_DELAYED_WORK (&sc->idle.timer_work, __smbdirect_socket_disabled_work);disable_delayed_work_sync (&sc->idle.timer_work); + sc->send_io.mem.gfp_mask =GFP_KERNEL ; +atomic_set (&sc->send_io.lcredits.count,0 );init_waitqueue_head (&sc->send_io.lcredits.wait_queue);
@@ -491,6 +495,8 @@ static __always_inline void smbdirect_socket_init(struct smbdirect_socket *sc)init_waitqueue_head (&sc->send_io.pending.dec_wait_queue);init_waitqueue_head (&sc->send_io.pending.zero_wait_queue); + sc->recv_io.mem.gfp_mask =GFP_KERNEL ; +INIT_LIST_HEAD (&sc->recv_io.free.list);spin_lock_init (&sc->recv_io.free.lock);
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 22/145 smb: smbdirect: introduce smbdirect_connection_{alloc,free}_send_io()
In reply to Stefan MetzmacherThese are more or less copies of smb_direct_{alloc,free}_sendmsg() in the server.
The only difference is that we use ib_dma_unmap_page() for all sges, this simplifies the logic and doesn't matter as ib_dma_unmap_single() and ib_dma_unmap_page() both operate on dma_addr_t and dma_unmap_single_attrs() is just an alias for dma_unmap_page_attrs().
We already have in inconsistency like that in the client code where we use ib_dma_unmap_single(), while we mapped using ib_dma_map_page().
The new functions will replace the existing once in the next commits and will also be used in the client.
--- .../common/smbdirect/smbdirect_connection.c | 47 +++++++++++++++++++ 1 file changed, 47 insertions(+)
fs/smb/common/smbdirect/smbdirect_connection.c +47 -0
@@ -6,6 +6,53 @@#include "smbdirect_internal.h" +__maybe_unused /* this is temporary while this file is included in others */ +static struct smbdirect_send_io *smbdirect_connection_alloc_send_io (struct smbdirect_socket *sc) +{ +struct smbdirect_send_io *msg; + + msg =mempool_alloc (sc->send_io.mem.pool, sc->send_io.mem.gfp_mask); +if (!msg) +return ERR_PTR (-ENOMEM ); + msg->socket = sc; +INIT_LIST_HEAD (&msg->sibling_list); + msg->num_sge =0 ; + +return msg; +} + +__maybe_unused /* this is temporary while this file is included in others */ +static void smbdirect_connection_free_send_io (struct smbdirect_send_io *msg) +{ +struct smbdirect_socket *sc = msg->socket; +size_t i; + +/* +* The list needs to be empty! +* The caller should take care of it. +*/ +WARN_ON_ONCE (!list_empty (&msg->sibling_list)); + +/* +* Note we call ib_dma_unmap_page(), even if some sges are mapped using +* ib_dma_map_single(). +* +* The difference between _single() and _page() only matters for the +* ib_dma_map_*() case. +* +* For the ib_dma_unmap_*() case it does not matter as both take the +* dma_addr_t and dma_unmap_single_attrs() is just an alias to +* dma_unmap_page_attrs(). +*/ +for (i =0 ; i < msg->num_sge; i++) +ib_dma_unmap_page (sc->ib.dev, + msg->sge[i].addr, + msg->sge[i].length, +DMA_TO_DEVICE ); + +mempool_free (msg, sc->send_io.mem.pool); +} +__maybe_unused /* this is temporary while this file is included in others */ static struct smbdirect_recv_io *smbdirect_connection_get_recv_io (struct smbdirect_socket *sc) {
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 23/145 smb: smbdirect: introduce smbdirect_connection_send_io_done()
In reply to Stefan MetzmacherThis is a combination of send_done() of client and server. It will replace both...
--- .../common/smbdirect/smbdirect_connection.c | 43 +++++++++++++++++++ 1 file changed, 43 insertions(+)
fs/smb/common/smbdirect/smbdirect_connection.c +43 -0
@@ -170,3 +170,46 @@ static void smbdirect_connection_idle_timer_work(struct work_struct *work) "schedule send of empty idle message\n");queue_work (sc->workqueue, &sc->idle.immediate_work); } + +__maybe_unused /* this is temporary while this file is included in others */ +static void smbdirect_connection_send_io_done (struct ib_cq *cq,struct ib_wc *wc) +{ +struct smbdirect_send_io *msg = + container_of(wc->wr_cqe, struct smbdirect_send_io, cqe); +struct smbdirect_socket *sc = msg->socket; +struct smbdirect_send_io *sibling, *next; +int lcredits =0 ; + +smbdirect_log_rdma_send (sc,SMBDIRECT_LOG_INFO , +"smbdirect_send_io completed. status='%s (%d)', opcode=%d\n" , +ib_wc_status_msg (wc->status), wc->status, wc->opcode); + +/* +* Free possible siblings and then the main send_io +*/ +list_for_each_entry_safe (sibling, next, &msg->sibling_list, sibling_list) { +list_del_init (&sibling->sibling_list); +smbdirect_connection_free_send_io (sibling); + lcredits +=1 ; + } +/* Note this frees wc->wr_cqe, but not wc */ +smbdirect_connection_free_send_io (msg); + lcredits +=1 ; + +if (unlikely (wc->status !=IB_WC_SUCCESS ||WARN_ON_ONCE (wc->opcode !=IB_WC_SEND ))) { +if (wc->status !=IB_WC_WR_FLUSH_ERR ) +smbdirect_log_rdma_send (sc,SMBDIRECT_LOG_ERR , +"wc->status=%s (%d) wc->opcode=%d\n" , +ib_wc_status_msg (wc->status), wc->status, wc->opcode); +smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); +return ; + } + +atomic_add (lcredits, &sc->send_io.lcredits.count); +wake_up (&sc->send_io.lcredits.wait_queue); + +if (atomic_dec_and_test (&sc->send_io.pending.count)) +wake_up (&sc->send_io.pending.zero_wait_queue); + +wake_up (&sc->send_io.pending.dec_wait_queue); +}
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 24/145 smb: smbdirect: introduce smbdirect_connection_{create,destroy}_mem_pools()
In reply to Stefan MetzmacherThis is based on smb_direct_{create,destroy}_pools() in the server.
But it doesn't use smbdirect_connection_get_recv_io() on cleanup, instead it uses list_for_each_entry_safe()...
It also keep some logic to allow userspace access to smbdirect_recv_io payload, which is needed for the client code. But it exposes the whole payload including the smbdirect_data_transfer header as documentation says data_offset = 0 and data_length != 0 would be valid, while the existing client code requires data_offset >= 24.
This should replace the related server functions and also be used on the client.
It also abstracts recv_io.mem.gfp_mask in order to allow server to keep using __GFP_RETRY_MAYFAIL.
It also uses struct kmem_cache_args consistently as that's the currently preferred version of kmem_cache_create(). And it makes use of the mempool_create_slab_pool() helper.
And it uses list_add_tail() just to let me feel better when looking at the code...
--- .../common/smbdirect/smbdirect_connection.c | 113 ++++++++++++++++++ 1 file changed, 113 insertions(+)
fs/smb/common/smbdirect/smbdirect_connection.c +113 -0
@@ -6,6 +6,119 @@#include "smbdirect_internal.h" +static void smbdirect_connection_destroy_mem_pools (struct smbdirect_socket *sc); + +__maybe_unused /* this is temporary while this file is included in others */ +static int smbdirect_connection_create_mem_pools (struct smbdirect_socket *sc) +{ +const struct smbdirect_socket_parameters *sp = &sc->parameters; +char name[80 ]; +size_t i; + +/* +* We use sizeof(struct smbdirect_negotiate_resp) for the +* payload size as it is larger as +* sizeof(struct smbdirect_data_transfer). +* +* This will fit client and server usage for now. +*/ +snprintf (name,sizeof (name),"smbdirect_send_io_cache_%p" , sc); +struct kmem_cache_args send_io_args = { + .align = __alignof__(struct smbdirect_send_io ), + }; + sc->send_io.mem.cache =kmem_cache_create (name, +sizeof (struct smbdirect_send_io ) + +sizeof (struct smbdirect_negotiate_resp ), + &send_io_args, +SLAB_HWCACHE_ALIGN ); +if (!sc->send_io.mem.cache) + goto err; + + sc->send_io.mem.pool =mempool_create_slab_pool (sp->send_credit_target, + sc->send_io.mem.cache); +if (!sc->send_io.mem.pool) + goto err; + +/* +* A payload size of sp->max_recv_size should fit +* any message. +* +* For smbdirect_data_transfer messages the whole +* buffer might be exposed to userspace +* (currently on the client side...) +* The documentation says data_offset = 0 would be +* strange but valid. +*/ +snprintf (name,sizeof (name),"smbdirect_recv_io_cache_%p" , sc); +struct kmem_cache_args recv_io_args = { + .align = __alignof__(struct smbdirect_recv_io ), + .useroffset =sizeof (struct smbdirect_recv_io ), + .usersize = sp->max_recv_size, + }; + sc->recv_io.mem.cache =kmem_cache_create (name, +sizeof (struct smbdirect_recv_io ) + + sp->max_recv_size, + &recv_io_args, +SLAB_HWCACHE_ALIGN ); +if (!sc->recv_io.mem.cache) + goto err; + + sc->recv_io.mem.pool =mempool_create_slab_pool (sp->recv_credit_max, + sc->recv_io.mem.cache); +if (!sc->recv_io.mem.pool) + goto err; + +for (i =0 ; i < sp->recv_credit_max; i++) { +struct smbdirect_recv_io *recv_io; + + recv_io =mempool_alloc (sc->recv_io.mem.pool, + sc->recv_io.mem.gfp_mask); +if (!recv_io) + goto err; + recv_io->socket = sc; + recv_io->sge.length =0 ; +INIT_WORK (&recv_io->complex_work, __smbdirect_socket_disabled_work); +disable_work_sync (&recv_io->complex_work); +list_add_tail (&recv_io->list, &sc->recv_io.free.list); + } + +return 0 ; +err: +smbdirect_connection_destroy_mem_pools (sc); +return -ENOMEM ; +} + +static void smbdirect_connection_destroy_mem_pools (struct smbdirect_socket *sc) +{ +struct smbdirect_recv_io *recv_io, *next_io; + +list_for_each_entry_safe (recv_io, next_io, &sc->recv_io.free.list, list) { +/* +* The work should already be disabled +*/ +disable_work_sync (&recv_io->complex_work); +list_del (&recv_io->list); +mempool_free (recv_io, sc->recv_io.mem.pool); + } + +/* +* Note mempool_destroy() and kmem_cache_destroy() +* work fine with a NULL pointer +*/ + +mempool_destroy (sc->recv_io.mem.pool); + sc->recv_io.mem.pool =NULL ; + +kmem_cache_destroy (sc->recv_io.mem.cache); + sc->recv_io.mem.cache =NULL ; + +mempool_destroy (sc->send_io.mem.pool); + sc->send_io.mem.pool =NULL ; + +kmem_cache_destroy (sc->send_io.mem.cache); + sc->send_io.mem.cache =NULL ; +} +__maybe_unused /* this is temporary while this file is included in others */ static struct smbdirect_send_io *smbdirect_connection_alloc_send_io (struct smbdirect_socket *sc) {
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 25/145 smb: smbdirect: introduce smbdirect_map_sges_from_iter() and helper functions
In reply to Stefan MetzmacherThese are basically copies of smb_extract_iter_to_rdma() and its helpers in the client, which will be replaced in the next steps.
The goal is to use them also in the server, which will simplify a lot.
--- .../common/smbdirect/smbdirect_connection.c | 255 ++++++++++++++++++ 1 file changed, 255 insertions(+)
fs/smb/common/smbdirect/smbdirect_connection.c +255 -0
@@ -5,6 +5,19 @@*/ #include "smbdirect_internal.h" +#include <linux/folio_queue.h> + +struct smbdirect_map_sges { +struct ib_sge *sge; +size_t num_sge; +size_t max_sge; +struct ib_device *device; +u32 local_dma_lkey; +enum dma_data_direction direction; +}; + +static ssize_t smbdirect_map_sges_from_iter (struct iov_iter *iter,size_t len, +struct smbdirect_map_sges *state);static void smbdirect_connection_destroy_mem_pools (struct smbdirect_socket *sc);
@@ -326,3 +339,245 @@ static void smbdirect_connection_send_io_done(struct ib_cq *cq, struct ib_wc *wcwake_up (&sc->send_io.pending.dec_wait_queue); } + +static bool smbdirect_map_sges_single_page (struct smbdirect_map_sges *state, +struct page *page,size_t off,size_t len) +{ +struct ib_sge *sge; +u64 addr; + +if (state->num_sge >= state->max_sge) +return false; + + addr =ib_dma_map_page (state->device, page, + off, len, state->direction); +if (ib_dma_mapping_error (state->device, addr)) +return false; + + sge = &state->sge[state->num_sge++]; + sge->addr = addr; + sge->length = len; + sge->lkey = state->local_dma_lkey; + +return true; +} + +/* +* Extract page fragments from a BVEC-class iterator and add them to an ib_sge +* list. The pages are not pinned. +*/ +static ssize_t smbdirect_map_sges_from_bvec (struct iov_iter *iter, +struct smbdirect_map_sges *state, +ssize_t maxsize) +{ +const struct bio_vec *bv = iter->bvec; +unsigned long start = iter->iov_offset; +unsigned int i; +ssize_t ret =0 ; + +for (i =0 ; i < iter->nr_segs; i++) { +size_t off, len; +bool ok; + + len = bv[i].bv_len; +if (start >= len) { + start -= len; +continue ; + } + + len =min_t (size_t, maxsize, len - start); + off = bv[i].bv_offset + start; + + ok =smbdirect_map_sges_single_page (state, + bv[i].bv_page, + off, + len); +if (!ok) +return -EIO ; + + ret += len; + maxsize -= len; +if (state->num_sge >= state->max_sge || maxsize <=0 ) +break ; + start =0 ; + } + +if (ret >0 ) +iov_iter_advance (iter, ret); +return ret; +} + +/* +* Extract fragments from a KVEC-class iterator and add them to an ib_sge list. +* This can deal with vmalloc'd buffers as well as kmalloc'd or static buffers. +* The pages are not pinned. +*/ +static ssize_t smbdirect_map_sges_from_kvec (struct iov_iter *iter, +struct smbdirect_map_sges *state, +ssize_t maxsize) +{ +const struct kvec *kv = iter->kvec; +unsigned long start = iter->iov_offset; +unsigned int i; +ssize_t ret =0 ; + +for (i =0 ; i < iter->nr_segs; i++) { +struct page *page; +unsigned long kaddr; +size_t off, len, seg; + + len = kv[i].iov_len; +if (start >= len) { + start -= len; +continue ; + } + + kaddr = (unsigned long )kv[i].iov_base + start; + off = kaddr & ~PAGE_MASK ; + len =min_t (size_t, maxsize, len - start); + kaddr &=PAGE_MASK ; + + maxsize -= len; +do { +bool ok; + + seg =min_t (size_t, len,PAGE_SIZE - off); + +if (is_vmalloc_or_module_addr ((void *)kaddr)) + page =vmalloc_to_page ((void *)kaddr); +else + page =virt_to_page ((void *)kaddr); + + ok =smbdirect_map_sges_single_page (state, page, off, seg); +if (!ok) +return -EIO ; + + ret += seg; + len -= seg; + kaddr +=PAGE_SIZE ; + off =0 ; + }while (len >0 && state->num_sge < state->max_sge); + +if (state->num_sge >= state->max_sge || maxsize <=0 ) +break ; + start =0 ; + } + +if (ret >0 ) +iov_iter_advance (iter, ret); +return ret; +} + +/* +* Extract folio fragments from a FOLIOQ-class iterator and add them to an +* ib_sge list. The folios are not pinned. +*/ +static ssize_t smbdirect_map_sges_from_folioq (struct iov_iter *iter, +struct smbdirect_map_sges *state, +ssize_t maxsize) +{ +const struct folio_queue *folioq = iter->folioq; +unsigned int slot = iter->folioq_slot; +ssize_t ret =0 ; +size_t offset = iter->iov_offset; + +if (WARN_ON_ONCE (!folioq)) +return -EIO ; + +if (slot >=folioq_nr_slots (folioq)) { + folioq = folioq->next; +if (WARN_ON_ONCE (!folioq)) +return -EIO ; + slot =0 ; + } + +do { +struct folio *folio =folioq_folio (folioq, slot); +size_t fsize =folioq_folio_size (folioq, slot); + +if (offset < fsize) { +size_t part =umin (maxsize, fsize - offset); +bool ok; + + ok =smbdirect_map_sges_single_page (state, +folio_page (folio,0 ), + offset, + part); +if (!ok) +return -EIO ; + + offset += part; + ret += part; + maxsize -= part; + } + +if (offset >= fsize) { + offset =0 ; + slot++; +if (slot >=folioq_nr_slots (folioq)) { +if (!folioq->next) { +WARN_ON_ONCE (ret < iter->count); +break ; + } + folioq = folioq->next; + slot =0 ; + } + } + }while (state->num_sge < state->max_sge && maxsize >0 ); + + iter->folioq = folioq; + iter->folioq_slot = slot; + iter->iov_offset = offset; + iter->count -= ret; +return ret; +} + +/* +* Extract page fragments from up to the given amount of the source iterator +* and build up an ib_sge list that refers to all of those bits. The ib_sge list +* is appended to, up to the maximum number of elements set in the parameter +* block. +* +* The extracted page fragments are not pinned or ref'd in any way; if an +* IOVEC/UBUF-type iterator is to be used, it should be converted to a +* BVEC-type iterator and the pages pinned, ref'd or otherwise held in some +* way. +*/ +__maybe_unused /* this is temporary while this file is included in others */ +static ssize_t smbdirect_map_sges_from_iter (struct iov_iter *iter,size_t len, +struct smbdirect_map_sges *state) +{ +ssize_t ret; +size_t before = state->num_sge; + +if (WARN_ON_ONCE (iov_iter_rw (iter) !=ITER_SOURCE )) +return -EIO ; + +switch (iov_iter_type (iter)) { +case ITER_BVEC : + ret =smbdirect_map_sges_from_bvec (iter, state, len); +break ; +case ITER_KVEC : + ret =smbdirect_map_sges_from_kvec (iter, state, len); +break ; +case ITER_FOLIOQ : + ret =smbdirect_map_sges_from_folioq (iter, state, len); +break ; +default : +WARN_ONCE (1 ,"iov_iter_type[%u]\n" ,iov_iter_type (iter)); +return -EIO ; + } + +if (ret <0 ) { +while (state->num_sge > before) { +struct ib_sge *sge = &state->sge[state->num_sge--]; + +ib_dma_unmap_page (state->device, + sge->addr, + sge->length, + state->direction); + } + } + +return ret; +}
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 26/145 smb: smbdirect: introduce smbdirect_connection_qp_event_handler()
In reply to Stefan MetzmacherThis is basically a copy of smbd_qp_async_error_upcall() in the client and smb_direct_qpair_handler() in the server. They will be replaced by the new common function soon, which will allow more code to be moved as well.
--- .../common/smbdirect/smbdirect_connection.c | 25 +++++++++++++++++++ 1 file changed, 25 insertions(+)
fs/smb/common/smbdirect/smbdirect_connection.c +25 -0
@@ -19,6 +19,31 @@ struct smbdirect_map_sges {static ssize_t smbdirect_map_sges_from_iter (struct iov_iter *iter,size_t len,struct smbdirect_map_sges *state); +__maybe_unused /* this is temporary while this file is included in others */ +static void smbdirect_connection_qp_event_handler (struct ib_event *event,void *context) +{ +struct smbdirect_socket *sc = context; + +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"%s on device %.*s socket %p (cm_id=%p) status %s first_error %1pe\n" , +ib_event_msg (event->event), +IB_DEVICE_NAME_MAX , + event->device->name, + sc, sc->rdma.cm_id, +smbdirect_socket_status_string (sc->status), +SMBDIRECT_DEBUG_ERR_PTR (sc->first_error)); + +switch (event->event) { +case IB_EVENT_CQ_ERR : +case IB_EVENT_QP_FATAL : +smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); +break ; + +default : +break ; + } +} +static void smbdirect_connection_destroy_mem_pools (struct smbdirect_socket *sc);__maybe_unused /* this is temporary while this file is included in others */
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 27/145 smb: smbdirect: introduce smbdirect_connection_negotiate_rdma_resources()
In reply to Stefan MetzmacherThis is a copy of the same logic used in client and server, it's inlined there, but they will use the new helper function soon.
--- .../common/smbdirect/smbdirect_connection.c | 62 +++++++++++++++++++ 1 file changed, 62 insertions(+)
fs/smb/common/smbdirect/smbdirect_connection.c +62 -0
@@ -291,6 +291,68 @@ smbdirect_connection_reassembly_first_recv_io(struct smbdirect_socket *sc)return msg; } +__maybe_unused /* this is temporary while this file is included in others */ +static void smbdirect_connection_negotiate_rdma_resources (struct smbdirect_socket *sc, +u8 peer_initiator_depth, +u8 peer_responder_resources, +const struct rdma_conn_param *param) +{ +struct smbdirect_socket_parameters *sp = &sc->parameters; + +if (rdma_protocol_iwarp (sc->ib.dev, sc->rdma.cm_id->port_num) && + param->private_data_len ==8 ) { +/* +* Legacy clients with only iWarp MPA v1 support +* need a private blob in order to negotiate +* the IRD/ORD values. +*/ +const __be32 *ird_ord_hdr = param->private_data; +u32 ird32 =be32_to_cpu (ird_ord_hdr[0 ]); +u32 ord32 =be32_to_cpu (ird_ord_hdr[1 ]); + +/* +* cifs.ko sends the legacy IRD/ORD negotiation +* event if iWarp MPA v2 was used. +* +* Here we check that the values match and only +* mark the client as legacy if they don't match. +*/ +if ((u32 )param->initiator_depth != ird32 || + (u32 )param->responder_resources != ord32) { +/* +* There are broken clients (old cifs.ko) +* using little endian and also +* struct rdma_conn_param only uses u8 +* for initiator_depth and responder_resources, +* so we truncate the value to U8_MAX. +* +* smb_direct_accept_client() will then +* do the real negotiation in order to +* select the minimum between client and +* server. +*/ + ird32 =min_t (u32, ird32,U8_MAX ); + ord32 =min_t (u32, ord32,U8_MAX ); + + sc->rdma.legacy_iwarp = true; + peer_initiator_depth = (u8 )ird32; + peer_responder_resources = (u8 )ord32; + } + } + +/* +* negotiate the value by using the minimum +* between client and server if the client provided +* non 0 values. +*/ +if (peer_initiator_depth !=0 ) + sp->initiator_depth =min_t (u8, sp->initiator_depth, + peer_initiator_depth); +if (peer_responder_resources !=0 ) + sp->responder_resources =min_t (u8, sp->responder_resources, + peer_responder_resources); +} +static void smbdirect_connection_idle_timer_work (struct work_struct *work) { struct smbdirect_socket *sc =
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 28/145 smb: smbdirect: introduce smbdirect_connection_{create,destroy}_qp()
In reply to Stefan Metzmachersmbdirect_connection_create_qp() is basically a copy of smb_direct_create_qpair() in the server, it just adds extra send_wr space for MR requests.
smbdirect_connection_destroy_qp() is the cleanup code smb_direct_create_qpair() has, plus calling ib_drain_qp(), it be a no-op if no requests are posted.
These additions allow the functions to be used by client and server.
--- .../common/smbdirect/smbdirect_connection.c | 214 ++++++++++++++++++ fs/smb/common/smbdirect/smbdirect_socket.h | 3 + 2 files changed, 217 insertions(+)
fs/smb/common/smbdirect/smbdirect_connection.c +214 -0
@@ -44,6 +44,220 @@ static void smbdirect_connection_qp_event_handler(struct ib_event *event, void * } } +static u32 smbdirect_rdma_rw_send_wrs (struct ib_device *dev, +const struct ib_qp_init_attr *attr) +{ +/* +* This could be split out of rdma_rw_init_qp() +* and be a helper function next to rdma_rw_mr_factor() +* +* We can't check unlikely(rdma_rw_force_mr) here, +* but that is most likely 0 anyway. +*/ +u32 factor; + +WARN_ON_ONCE (attr->port_num ==0 ); + +/* +* Each context needs at least one RDMA READ or WRITE WR. +* +* For some hardware we might need more, eventually we should ask the +* HCA driver for a multiplier here. +*/ + factor =1 ; + +/* +* If the device needs MRs to perform RDMA READ or WRITE operations, +* we'll need two additional MRs for the registrations and the +* invalidation. +*/ +if (rdma_protocol_iwarp (dev, attr->port_num) || dev->attrs.max_sgl_rd) + factor +=2 ;/* inv + reg */ + +return factor * attr->cap.max_rdma_ctxs; +} + +static void smbdirect_connection_destroy_qp (struct smbdirect_socket *sc); + +__maybe_unused /* this is temporary while this file is included in others */ +static int smbdirect_connection_create_qp (struct smbdirect_socket *sc) +{ +const struct smbdirect_socket_parameters *sp = &sc->parameters; +struct ib_qp_init_attr qp_attr; +struct ib_qp_cap qp_cap; +u32 rdma_send_wr; +u32 max_send_wr; +int ret; + +/* +* Note that {rdma,ib}_create_qp() will call +* rdma_rw_init_qp() if max_rdma_ctxs is not 0. +* It will adjust max_send_wr to the required +* number of additional WRs for the RDMA RW operations. +* It will cap max_send_wr to the device limit. +* +* We use allocate sp->responder_resources * 2 MRs +* and each MR needs WRs for REG and INV, so +* we use '* 4'. +* +* +1 for ib_drain_qp() +*/ +memset (&qp_cap,0 ,sizeof (qp_cap)); + qp_cap.max_send_wr = sp->send_credit_target + sp->responder_resources *4 +1 ; + qp_cap.max_recv_wr = sp->recv_credit_max +1 ; + qp_cap.max_send_sge =SMBDIRECT_SEND_IO_MAX_SGE ; + qp_cap.max_recv_sge =SMBDIRECT_RECV_IO_MAX_SGE ; + qp_cap.max_inline_data =0 ; + qp_cap.max_rdma_ctxs = sc->rw_io.credits.max; + +/* +* Find out the number of max_send_wr +* after rdma_rw_init_qp() adjusted it. +* +* We only do it on a temporary variable, +* as rdma_create_qp() will trigger +* rdma_rw_init_qp() again. +*/ +memset (&qp_attr,0 ,sizeof (qp_attr)); + qp_attr.cap = qp_cap; + qp_attr.port_num = sc->rdma.cm_id->port_num; + rdma_send_wr =smbdirect_rdma_rw_send_wrs (sc->ib.dev, &qp_attr); + max_send_wr = qp_cap.max_send_wr + rdma_send_wr; + +if (qp_cap.max_send_wr > sc->ib.dev->attrs.max_cqe || + qp_cap.max_send_wr > sc->ib.dev->attrs.max_qp_wr) { +pr_err ("Possible CQE overrun: max_send_wr %d\n" , + qp_cap.max_send_wr); +pr_err ("device %.*s reporting max_cqe %d max_qp_wr %d\n" , +IB_DEVICE_NAME_MAX , + sc->ib.dev->name, + sc->ib.dev->attrs.max_cqe, + sc->ib.dev->attrs.max_qp_wr); +pr_err ("consider lowering send_credit_target = %d\n" , + sp->send_credit_target); +return -EINVAL ; + } + +if (qp_cap.max_rdma_ctxs && + (max_send_wr >= sc->ib.dev->attrs.max_cqe || + max_send_wr >= sc->ib.dev->attrs.max_qp_wr)) { +pr_err ("Possible CQE overrun: rdma_send_wr %d + max_send_wr %d = %d\n" , + rdma_send_wr, qp_cap.max_send_wr, max_send_wr); +pr_err ("device %.*s reporting max_cqe %d max_qp_wr %d\n" , +IB_DEVICE_NAME_MAX , + sc->ib.dev->name, + sc->ib.dev->attrs.max_cqe, + sc->ib.dev->attrs.max_qp_wr); +pr_err ("consider lowering send_credit_target = %d, max_rdma_ctxs = %d\n" , + sp->send_credit_target, qp_cap.max_rdma_ctxs); +return -EINVAL ; + } + +if (qp_cap.max_recv_wr > sc->ib.dev->attrs.max_cqe || + qp_cap.max_recv_wr > sc->ib.dev->attrs.max_qp_wr) { +pr_err ("Possible CQE overrun: max_recv_wr %d\n" , + qp_cap.max_recv_wr); +pr_err ("device %.*s reporting max_cqe %d max_qp_wr %d\n" , +IB_DEVICE_NAME_MAX , + sc->ib.dev->name, + sc->ib.dev->attrs.max_cqe, + sc->ib.dev->attrs.max_qp_wr); +pr_err ("consider lowering receive_credit_max = %d\n" , + sp->recv_credit_max); +return -EINVAL ; + } + +if (qp_cap.max_send_sge > sc->ib.dev->attrs.max_send_sge || + qp_cap.max_recv_sge > sc->ib.dev->attrs.max_recv_sge) { +pr_err ("device %.*s max_send_sge/max_recv_sge = %d/%d too small\n" , +IB_DEVICE_NAME_MAX , + sc->ib.dev->name, + sc->ib.dev->attrs.max_send_sge, + sc->ib.dev->attrs.max_recv_sge); +return -EINVAL ; + } + + sc->ib.pd =ib_alloc_pd (sc->ib.dev,0 ); +if (IS_ERR (sc->ib.pd)) { +pr_err ("Can't create RDMA PD: %1pe\n" , sc->ib.pd); + ret =PTR_ERR (sc->ib.pd); + sc->ib.pd =NULL ; +return ret; + } + + sc->ib.send_cq =ib_alloc_cq_any (sc->ib.dev, sc, + max_send_wr, + sc->ib.poll_ctx); +if (IS_ERR (sc->ib.send_cq)) { +pr_err ("Can't create RDMA send CQ: %1pe\n" , sc->ib.send_cq); + ret =PTR_ERR (sc->ib.send_cq); + sc->ib.send_cq =NULL ; + goto err; + } + + sc->ib.recv_cq =ib_alloc_cq_any (sc->ib.dev, sc, + qp_cap.max_recv_wr, + sc->ib.poll_ctx); +if (IS_ERR (sc->ib.recv_cq)) { +pr_err ("Can't create RDMA recv CQ: %1pe\n" , sc->ib.recv_cq); + ret =PTR_ERR (sc->ib.recv_cq); + sc->ib.recv_cq =NULL ; + goto err; + } + +/* +* We reset completely here! +* As the above use was just temporary +* to calc max_send_wr and rdma_send_wr. +* +* rdma_create_qp() will trigger rdma_rw_init_qp() +* again if max_rdma_ctxs is not 0. +*/ +memset (&qp_attr,0 ,sizeof (qp_attr)); + qp_attr.event_handler = smbdirect_connection_qp_event_handler; + qp_attr.qp_context = sc; + qp_attr.cap = qp_cap; + qp_attr.sq_sig_type =IB_SIGNAL_REQ_WR ; + qp_attr.qp_type =IB_QPT_RC ; + qp_attr.send_cq = sc->ib.send_cq; + qp_attr.recv_cq = sc->ib.recv_cq; + qp_attr.port_num = ~0 ; + + ret =rdma_create_qp (sc->rdma.cm_id, sc->ib.pd, &qp_attr); +if (ret) { +pr_err ("Can't create RDMA QP: %1pe\n" , +SMBDIRECT_DEBUG_ERR_PTR (ret)); + goto err; + } + sc->ib.qp = sc->rdma.cm_id->qp; + +return 0 ; +err: +smbdirect_connection_destroy_qp (sc); +return ret; +} + +static void smbdirect_connection_destroy_qp (struct smbdirect_socket *sc) +{ +if (sc->ib.qp) { +ib_drain_qp (sc->ib.qp); + sc->ib.qp =NULL ; +rdma_destroy_qp (sc->rdma.cm_id); + } +if (sc->ib.recv_cq) { +ib_destroy_cq (sc->ib.recv_cq); + sc->ib.recv_cq =NULL ; + } +if (sc->ib.send_cq) { +ib_destroy_cq (sc->ib.send_cq); + sc->ib.send_cq =NULL ; + } +if (sc->ib.pd) { +ib_dealloc_pd (sc->ib.pd); + sc->ib.pd =NULL ; + } +} +static void smbdirect_connection_destroy_mem_pools (struct smbdirect_socket *sc);__maybe_unused /* this is temporary while this file is included in others */
fs/smb/common/smbdirect/smbdirect_socket.h +3 -0
@@ -120,6 +120,7 @@ struct smbdirect_socket {/* IB verbs related */ struct {struct ib_pd *pd; +enum ib_poll_context poll_ctx;struct ib_cq *send_cq;struct ib_cq *recv_cq;
@@ -478,6 +479,8 @@ static __always_inline void smbdirect_socket_init(struct smbdirect_socket *sc)INIT_WORK (&sc->disconnect_work, __smbdirect_socket_disabled_work);disable_work_sync (&sc->disconnect_work); + sc->ib.poll_ctx =IB_POLL_UNBOUND_WORKQUEUE ; +INIT_WORK (&sc->idle.immediate_work, __smbdirect_socket_disabled_work);disable_work_sync (&sc->idle.immediate_work);INIT_DELAYED_WORK (&sc->idle.timer_work, __smbdirect_socket_disabled_work);
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 29/145 smb: smbdirect: introduce smbdirect_connection_post_recv_io()
In reply to Stefan MetzmacherThis is basically a copy of smbd_post_recv() in the client and smb_direct_post_recv() in the server.
The only difference is that this returns early if the connection is already broken.
--- .../common/smbdirect/smbdirect_connection.c | 42 +++++++++++++++++++ 1 file changed, 42 insertions(+)
fs/smb/common/smbdirect/smbdirect_connection.c +42 -0
@@ -641,6 +641,48 @@ static void smbdirect_connection_send_io_done(struct ib_cq *cq, struct ib_wc *wcwake_up (&sc->send_io.pending.dec_wait_queue); } +__maybe_unused /* this is temporary while this file is included in others */ +static int smbdirect_connection_post_recv_io (struct smbdirect_recv_io *msg) +{ +struct smbdirect_socket *sc = msg->socket; +const struct smbdirect_socket_parameters *sp = &sc->parameters; +struct ib_recv_wr recv_wr = { + .wr_cqe = &msg->cqe, + .sg_list = &msg->sge, + .num_sge =1 , + }; +int ret; + +if (unlikely (sc->first_error)) +return sc->first_error; + + msg->sge.addr =ib_dma_map_single (sc->ib.dev, + msg->packet, + sp->max_recv_size, +DMA_FROM_DEVICE ); + ret =ib_dma_mapping_error (sc->ib.dev, msg->sge.addr); +if (ret) +return ret; + + msg->sge.length = sp->max_recv_size; + msg->sge.lkey = sc->ib.pd->local_dma_lkey; + + ret =ib_post_recv (sc->ib.qp, &recv_wr,NULL ); +if (ret) { +smbdirect_log_rdma_recv (sc,SMBDIRECT_LOG_ERR , +"ib_post_recv failed ret=%d (%s)\n" , + ret,errname (ret)); +ib_dma_unmap_single (sc->ib.dev, + msg->sge.addr, + msg->sge.length, +DMA_FROM_DEVICE ); + msg->sge.length =0 ; +smbdirect_socket_schedule_cleanup (sc, ret); + } + +return ret; +} +static bool smbdirect_map_sges_single_page (struct smbdirect_map_sges *state,struct page *page,size_t off,size_t len) {
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 30/145 smb: smbdirect: introduce smbdirect_connection_recv_io_refill_work()
In reply to Stefan MetzmacherThis is basically a copy of smbd_post_send_credits() in the client and smb_direct_post_recv_credits() in the server.
There are several improvements compared to the existing functions:
1. We calculate the number of missing posted buffers by getting the difference between recv_io.credits.target and recv_io.posted.count.
Instead of the difference between recv_io.credits.target and recv_io.credits.count, because recv_io.credits.count is only updated once a message is send to the peer.
It was not really a problem before, because we have a fixed number smbdirect_recv_io buffers, so the loop terminated when smbdirect_connection_get_recv_io() returns NULL.
But using recv_io.posted.count makes it easier to understand.
2. In order to tell the peer about the newly posted buffer and grant the credits, we only trigger the send immediate when we're not granting only the last possible credit.
This is mostly a difference relative to the servers smb_direct_post_recv_credits() implementation, which should avoid useless ping pong messages.
--- .../common/smbdirect/smbdirect_connection.c | 92 +++++++++++++++++++ 1 file changed, 92 insertions(+)
fs/smb/common/smbdirect/smbdirect_connection.c +92 -0
@@ -683,6 +683,98 @@ static int smbdirect_connection_post_recv_io(struct smbdirect_recv_io *msg)return ret; } +__maybe_unused /* this is temporary while this file is included in others */ +static void smbdirect_connection_recv_io_refill_work (struct work_struct *work) +{ +struct smbdirect_socket *sc = + container_of(work, struct smbdirect_socket, recv_io.posted.refill_work); +int missing; +int posted =0 ; + +if (unlikely (sc->first_error)) +return ; + +/* +* Find out how much smbdirect_recv_io buffers we should post. +* +* Note that sc->recv_io.credits.target is the value +* from the peer and it can in theory change over time, +* but it is forced to be at least 1 and at max +* sp->recv_credit_max. +* +* So it can happen that missing will be lower than 0, +* which means the peer has recently lowered its desired +* tarted, while be already granted a higher number of credits. +* +* Note 'posted' is the number of smbdirect_recv_io buffers +* posted within this function, while sc->recv_io.posted.count +* is the overall value of posted smbdirect_recv_io buffers. +* +* We try to post as much buffers as missing, but +* this is limited if a lot of smbdirect_recv_io buffers +* are still in the sc->recv_io.reassembly.list instead of +* the sc->recv_io.free.list. +* +*/ + missing = (int )sc->recv_io.credits.target -atomic_read (&sc->recv_io.posted.count); +while (posted < missing) { +struct smbdirect_recv_io *recv_io; +int ret; + +/* +* It's ok if smbdirect_connection_get_recv_io() +* returns NULL, it means smbdirect_recv_io structures +* are still be in the reassembly.list. +*/ + recv_io =smbdirect_connection_get_recv_io (sc); +if (!recv_io) +break ; + + recv_io->first_segment = false; + + ret =smbdirect_connection_post_recv_io (recv_io); +if (ret) { +smbdirect_log_rdma_recv (sc,SMBDIRECT_LOG_ERR , +"smbdirect_connection_post_recv_io failed rc=%d (%s)\n" , + ret,errname (ret)); +smbdirect_connection_put_recv_io (recv_io); +return ; + } + +atomic_inc (&sc->recv_io.posted.count); + posted +=1 ; + } + +/* If nothing was posted we're done */ +if (posted ==0 ) +return ; + +/* +* If we posted at least one smbdirect_recv_io buffer, +* we need to inform the peer about it and grant +* additional credits. +* +* However there is one case where we don't want to +* do that. +* +* If only a single credit was missing before +* reaching the requested target, we should not +* post an immediate send, as that would cause +* endless ping pong once a keep alive exchange +* is started. +* +* However if sc->recv_io.credits.target is only 1, +* the peer has no credit left and we need to +* grant the credit anyway. +*/ +if (missing ==1 && sc->recv_io.credits.target !=1 ) +return ; + +smbdirect_log_keep_alive (sc,SMBDIRECT_LOG_INFO , +"schedule send of an empty message\n" ); +queue_work (sc->workqueue, &sc->idle.immediate_work); +} +static bool smbdirect_map_sges_single_page (struct smbdirect_map_sges *state,struct page *page,size_t off,size_t len) {
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 31/145 smb: smbdirect: split out smbdirect_connection_recv_io_refill()
In reply to Stefan MetzmacherThis will allow us to refill the recv queue in a sync way after negotiation.
--- .../common/smbdirect/smbdirect_connection.c | 36 +++++++++++++------ 1 file changed, 25 insertions(+), 11 deletions(-)
fs/smb/common/smbdirect/smbdirect_connection.c +25 -11
@@ -683,16 +683,13 @@ static int smbdirect_connection_post_recv_io(struct smbdirect_recv_io *msg)return ret; } -__maybe_unused /* this is temporary while this file is included in others */ -static void smbdirect_connection_recv_io_refill_work (struct work_struct *work) +static int smbdirect_connection_recv_io_refill (struct smbdirect_socket *sc) { -struct smbdirect_socket *sc = - container_of(work, struct smbdirect_socket, recv_io.posted.refill_work);int missing;int posted =0 ;if (unlikely (sc->first_error)) -return ; +return sc->first_error;/* * Find out how much smbdirect_recv_io buffers we should post.
@@ -738,7 +735,7 @@ static void smbdirect_connection_recv_io_refill_work(struct work_struct *work)"smbdirect_connection_post_recv_io failed rc=%d (%s)\n" , ret, errname(ret));smbdirect_connection_put_recv_io (recv_io); -return ; +return ret; }atomic_inc (&sc->recv_io.posted.count);
@@ -747,7 +744,7 @@ static void smbdirect_connection_recv_io_refill_work(struct work_struct *work)/* If nothing was posted we're done */ if (posted ==0 ) -return ; +return 0 ;/* * If we posted at least one smbdirect_recv_io buffer,
@@ -768,11 +765,28 @@ static void smbdirect_connection_recv_io_refill_work(struct work_struct *work)* grant the credit anyway. */ if (missing ==1 && sc->recv_io.credits.target !=1 ) -return ; +return 0 ; -smbdirect_log_keep_alive (sc,SMBDIRECT_LOG_INFO , -"schedule send of an empty message\n" ); -queue_work (sc->workqueue, &sc->idle.immediate_work); +return posted; +} + +__maybe_unused /* this is temporary while this file is included in others */ +static void smbdirect_connection_recv_io_refill_work (struct work_struct *work) +{ +struct smbdirect_socket *sc = + container_of(work, struct smbdirect_socket, recv_io.posted.refill_work); +int posted; + + posted =smbdirect_connection_recv_io_refill (sc); +if (unlikely (posted <0 )) { +smbdirect_socket_schedule_cleanup (sc, posted); +return ; + } +if (posted >0 ) { +smbdirect_log_keep_alive (sc,SMBDIRECT_LOG_INFO , +"schedule send of an empty message\n" ); +queue_work (sc->workqueue, &sc->idle.immediate_work); + } } static bool smbdirect_map_sges_single_page(struct smbdirect_map_sges *state,
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 32/145 smb: smbdirect: introduce smbdirect_get_buf_page_count()
In reply to Stefan MetzmacherThis is a copy of get_buf_page_count() in the server and will replace it soon.
The only difference is that we now use size_t instead of int.
--- fs/smb/common/smbdirect/smbdirect_socket.h | 6 ++++++ 1 file changed, 6 insertions(+)
fs/smb/common/smbdirect/smbdirect_socket.h +6 -0
@@ -712,4 +712,10 @@ struct smbdirect_rw_io { struct scatterlist sg_list[]; }; +static inline size_t smbdirect_get_buf_page_count (const void *buf,size_t size) +{ +return DIV_ROUND_UP ((uintptr_t )buf + size,PAGE_SIZE ) - + (uintptr_t )buf /PAGE_SIZE ; +} +#endif /* __FS_SMB_COMMON_SMBDIRECT_SMBDIRECT_SOCKET_H__ */
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 33/145 smb: smbdirect: introduce smbdirect_socket_wait_for_credits()
In reply to Stefan MetzmacherThis is a copy of wait_for_credits() in the server, which will be replaced by this soon.
This will allow us to share more common code between client and server soon.
--- fs/smb/common/smbdirect/smbdirect_internal.h | 7 +++++ fs/smb/common/smbdirect/smbdirect_socket.c | 29 ++++++++++++++++++++ 2 files changed, 36 insertions(+)
fs/smb/common/smbdirect/smbdirect_internal.h +7 -0
@@ -33,6 +33,13 @@ static void __smbdirect_socket_schedule_cleanup(struct smbdirect_socket *sc, __func__, __LINE__, __error, &__force_status); \ } while (0) +static int smbdirect_socket_wait_for_credits(struct smbdirect_socket *sc, +enum smbdirect_socket_status expected_status, +int unexpected_errno, +wait_queue_head_t *waitq, +atomic_t *total_credits, +int needed); +static void smbdirect_connection_idle_timer_work (struct work_struct *work);#endif /* __FS_SMB_COMMON_SMBDIRECT_INTERNAL_H__ */
fs/smb/common/smbdirect/smbdirect_socket.c +29 -0
@@ -279,3 +279,32 @@ static void smbdirect_socket_cleanup_work(struct work_struct *work)*/ smbdirect_socket_wake_up_all (sc); } + +__maybe_unused /* this is temporary while this file is included in others */ +static int smbdirect_socket_wait_for_credits (struct smbdirect_socket *sc, +enum smbdirect_socket_status expected_status, +int unexpected_errno, +wait_queue_head_t *waitq, +atomic_t *total_credits, +int needed) +{ +int ret; + +if (WARN_ON_ONCE (needed <0 )) +return -EINVAL ; + +do { +if (atomic_sub_return (needed, total_credits) >=0 ) +return 0 ; + +atomic_add (needed, total_credits); + ret =wait_event_interruptible (*waitq, +atomic_read (total_credits) >= needed || + sc->status != expected_status); + +if (sc->status != expected_status) +return unexpected_errno; +else if (ret <0 ) +return ret; + }while (true); +}
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 34/145 smb: smbdirect: introduce smbdirect_mr.c with client mr code
In reply to Stefan MetzmacherThis is basically contains the following functions copied from the client: destroy_mr_list, allocate_mr_list, register_mr_done, smbd_mr_recovery_work, get_mr, smbd_iter_to_mr, smbd_register_mr and smbd_deregister_mr.
They got new names, some indentation/formatting changes, some variable names are changed too.
They also only use struct smbdirect_socket instead of struct smbd_connection.
But the logic is still the same. They will be used by the client soon.
--- .../common/smbdirect/smbdirect_all_c_files.c | 1 + fs/smb/common/smbdirect/smbdirect_mr.c | 544 ++++++++++++++++++ 2 files changed, 545 insertions(+) create mode 100644 fs/smb/common/smbdirect/smbdirect_mr.c
fs/smb/common/smbdirect/smbdirect_all_c_files.c +1 -0
@@ -17,3 +17,4 @@#endif #include "smbdirect_socket.c" #include "smbdirect_connection.c" +#include "smbdirect_mr.c"
fs/smb/common/smbdirect/smbdirect_mr.c new file +544 -0
544 lines not on this page · raw message
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 35/145 smb: smbdirect: introduce smbdirect_rw.c with server rw code
In reply to Stefan MetzmacherThis is basically contains the following functions copied from the server: wait_for_rw_credits, calc_rw_credits, get_sg_list, smb_direct_free_rdma_rw_msg, read_write_done, read_done, write_done, smb_direct_rdma_xmit.
They got new names, some indentation/formatting changes, some variable names are changed too.
They also only use struct smbdirect_socket instead of struct smb_direct_transport.
But the logic is still the same. They will be used by the server soon.
--- .../common/smbdirect/smbdirect_all_c_files.c | 1 + fs/smb/common/smbdirect/smbdirect_rw.c | 255 ++++++++++++++++++ fs/smb/common/smbdirect/smbdirect_socket.h | 9 + 3 files changed, 265 insertions(+) create mode 100644 fs/smb/common/smbdirect/smbdirect_rw.c
fs/smb/common/smbdirect/smbdirect_all_c_files.c +1 -0
@@ -18,3 +18,4 @@#include "smbdirect_socket.c" #include "smbdirect_connection.c" #include "smbdirect_mr.c" +#include "smbdirect_rw.c"
fs/smb/common/smbdirect/smbdirect_rw.c new file +255 -0
255 lines not on this page · raw message
fs/smb/common/smbdirect/smbdirect_socket.h +9 -0
@@ -306,6 +306,14 @@ struct smbdirect_socket {* The state for RDMA read/write requests on the server */ struct { +/* +* Memory hints for +* smbdirect_rw_io structs +*/ +struct { +gfp_t gfp_mask; + } mem; +/* * The credit state for the send side */
@@ -513,6 +521,7 @@ static __always_inline void smbdirect_socket_init(struct smbdirect_socket *sc)spin_lock_init (&sc->recv_io.reassembly.lock);init_waitqueue_head (&sc->recv_io.reassembly.wait_queue); + sc->rw_io.mem.gfp_mask =GFP_KERNEL ;atomic_set (&sc->rw_io.credits.count,0 );init_waitqueue_head (&sc->rw_io.credits.wait_queue);
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 36/145 smb: smbdirect: define SMBDIRECT_MIN_{RECEIVE,FRAGMENTED}_SIZE
In reply to Stefan MetzmacherThese are specified in MS-SMBD...
--- fs/smb/common/smbdirect/smbdirect_pdu.h | 4 ++++ 1 file changed, 4 insertions(+)
fs/smb/common/smbdirect/smbdirect_pdu.h +4 -0
@@ -8,6 +8,10 @@#define SMBDIRECT_V1 0x0100 +/* SMBD minimum receive size and fragmented sized defined in [MS-SMBD] */ +#define SMBDIRECT_MIN_RECEIVE_SIZE 128 +#define SMBDIRECT_MIN_FRAGMENTED_SIZE 131072 +/* SMBD negotiation request packet [MS-SMBD] 2.2.1 */ struct smbdirect_negotiate_req {__le16 min_version;
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 37/145 smb: smbdirect: define SMBDIRECT_RDMA_CM_[RNR_]RETRY
In reply to Stefan MetzmacherThese are copies of {SMBD,SMB_DIRECT}_CM_[RNR_]RETRY.
--- fs/smb/common/smbdirect/smbdirect_socket.h | 9 +++++++++ 1 file changed, 9 insertions(+)
fs/smb/common/smbdirect/smbdirect_socket.h +9 -0
@@ -727,4 +727,13 @@ static inline size_t smbdirect_get_buf_page_count(const void *buf, size_t size) (uintptr_t)buf / PAGE_SIZE; } +/* +* Maximum number of retries on data transfer operations +*/ +#define SMBDIRECT_RDMA_CM_RETRY 6 +/* +* No need to retry on Receiver Not Ready since SMB_DIRECT manages credits +*/ +#define SMBDIRECT_RDMA_CM_RNR_RETRY 0 +#endif /* __FS_SMB_COMMON_SMBDIRECT_SMBDIRECT_SOCKET_H__ */
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 38/145 smb: smbdirect: introduce smbdirect_connection_recv_io_done()
In reply to Stefan MetzmacherThis is basically a copy of recv_done() in client and server, with the following additions:
- Only handling the SMBDIRECT_EXPECT_DATA_TRANSFER code path, as we'll have separate functions for the negotiate messages.
- Using more helper variables
- Improved logging
- Add credits_requested == 0 error check
- Add data_offset not 8 bytes aligned error check
- Use disable_work(&sc->recv_io.posted.refill_work) before smbdirect_connection_put_recv_io, when it is followed by smbdirect_socket_schedule_cleanup()
This will be used on common between client and server in future and replace the existing recv_done() functions.
--- .../common/smbdirect/smbdirect_connection.c | 167 ++++++++++++++++++ 1 file changed, 167 insertions(+)
fs/smb/common/smbdirect/smbdirect_connection.c +167 -0
@@ -683,6 +683,173 @@ static int smbdirect_connection_post_recv_io(struct smbdirect_recv_io *msg)return ret; } +__maybe_unused /* this is temporary while this file is included in others */ +static void smbdirect_connection_recv_io_done (struct ib_cq *cq,struct ib_wc *wc) +{ +struct smbdirect_recv_io *recv_io = + container_of(wc->wr_cqe, struct smbdirect_recv_io, cqe); +struct smbdirect_socket *sc = recv_io->socket; +const struct smbdirect_socket_parameters *sp = &sc->parameters; +struct smbdirect_data_transfer *data_transfer; +u16 old_recv_credit_target; +u16 credits_requested; +u16 credits_granted; +u16 flags; +u32 data_offset; +u32 data_length; +u32 remaining_data_length; + +if (unlikely (wc->status !=IB_WC_SUCCESS ||WARN_ON_ONCE (wc->opcode !=IB_WC_RECV ))) { +if (wc->status !=IB_WC_WR_FLUSH_ERR ) +smbdirect_log_rdma_recv (sc,SMBDIRECT_LOG_ERR , +"wc->status=%s (%d) wc->opcode=%d\n" , +ib_wc_status_msg (wc->status), wc->status, wc->opcode); + goto error; + } + +smbdirect_log_rdma_recv (sc,SMBDIRECT_LOG_INFO , +"recv_io=0x%p type=%d wc status=%s wc opcode %d byte_len=%d pkey_index=%u\n" , + recv_io, sc->recv_io.expected, +ib_wc_status_msg (wc->status), wc->opcode, + wc->byte_len, wc->pkey_index); + +/* +* Reset timer to the keepalive interval in +* order to trigger our next keepalive message. +*/ + sc->idle.keepalive =SMBDIRECT_KEEPALIVE_NONE ; +mod_delayed_work (sc->workqueue, &sc->idle.timer_work, +msecs_to_jiffies (sp->keepalive_interval_msec)); + +ib_dma_sync_single_for_cpu (sc->ib.dev, + recv_io->sge.addr, + recv_io->sge.length, +DMA_FROM_DEVICE ); + +if (unlikely (wc->byte_len < + offsetof(struct smbdirect_data_transfer , padding))) { +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"wc->byte_len=%u < %zu\n" , + wc->byte_len, + offsetof(struct smbdirect_data_transfer , padding)); + goto error; + } + + data_transfer = (struct smbdirect_data_transfer *)recv_io->packet; + credits_requested =le16_to_cpu (data_transfer->credits_requested); + credits_granted =le16_to_cpu (data_transfer->credits_granted); + flags =le16_to_cpu (data_transfer->flags); + remaining_data_length =le32_to_cpu (data_transfer->remaining_data_length); + data_offset =le32_to_cpu (data_transfer->data_offset); + data_length =le32_to_cpu (data_transfer->data_length); + +smbdirect_log_incoming (sc,SMBDIRECT_LOG_INFO , +"DataIn: %s=%u, %s=%u, %s=0x%x, %s=%u, %s=%u, %s=%u\n" , +"CreditsRequested" , + credits_requested, +"CreditsGranted" , + credits_granted, +"Flags" , + flags, +"RemainingDataLength" , + remaining_data_length, +"DataOffset" , + data_offset, +"DataLength" , + data_length); + +if (unlikely (credits_requested ==0 )) { +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"invalid: credits_requested == 0\n" ); + goto error; + } + +if (unlikely (data_offset %8 !=0 )) { +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"invalid: data_offset=%u (0x%x) not aligned to 8\n" , + data_offset, data_offset); + goto error; + } + +if (unlikely (wc->byte_len < data_offset || + (u64 )wc->byte_len < (u64 )data_offset + data_length)) { +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"wc->byte_len=%u < date_offset=%u + data_length=%u\n" , + wc->byte_len, data_offset, data_length); + goto error; + } + +if (unlikely (remaining_data_length > sp->max_fragmented_recv_size || + data_length > sp->max_fragmented_recv_size || + (u64 )remaining_data_length + (u64 )data_length > (u64 )sp->max_fragmented_recv_size)) { +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"remaining_data_length=%u + data_length=%u > max_fragmented=%u\n" , + remaining_data_length, data_length, sp->max_fragmented_recv_size); + goto error; + } + +if (data_length) { +if (sc->recv_io.reassembly.full_packet_received) + recv_io->first_segment = true; + +if (remaining_data_length) + sc->recv_io.reassembly.full_packet_received = false; +else + sc->recv_io.reassembly.full_packet_received = true; + } + +atomic_dec (&sc->recv_io.posted.count); +atomic_dec (&sc->recv_io.credits.count); + old_recv_credit_target = sc->recv_io.credits.target; +/* +* We take the value from the peer, which is checked to be higher than 0, +* but we limit it to the max value we support in order to have +* the main logic simpler. +*/ + sc->recv_io.credits.target = credits_requested; + sc->recv_io.credits.target =min_t (u16, sc->recv_io.credits.target, + sp->recv_credit_max); +if (credits_granted) { +atomic_add (credits_granted, &sc->send_io.credits.count); +/* +* We have new send credits granted from remote peer +* If any sender is waiting for credits, unblock it +*/ +wake_up (&sc->send_io.credits.wait_queue); + } + +/* Send an immediate response right away if requested */ +if (flags &SMBDIRECT_FLAG_RESPONSE_REQUESTED ) { +smbdirect_log_keep_alive (sc,SMBDIRECT_LOG_INFO , +"schedule send of immediate response\n" ); +queue_work (sc->workqueue, &sc->idle.immediate_work); + } + +/* +* If this is a packet with data playload place the data in +* reassembly queue and wake up the reading thread +*/ +if (data_length) { +if (sc->recv_io.credits.target > old_recv_credit_target) +queue_work (sc->workqueue, &sc->recv_io.posted.refill_work); + +smbdirect_connection_reassembly_append_recv_io (sc, recv_io, data_length); +wake_up (&sc->recv_io.reassembly.wait_queue); + }else +smbdirect_connection_put_recv_io (recv_io); + +return ; + +error: +/* +* Make sure smbdirect_connection_put_recv_io() does not +* start recv_io.posted.refill_work. +*/ +disable_work (&sc->recv_io.posted.refill_work); +smbdirect_connection_put_recv_io (recv_io); +smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); +} +static int smbdirect_connection_recv_io_refill (struct smbdirect_socket *sc) {int missing;
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 39/145 smb: smbdirect: introduce smbdirect_socket_destroy[_sync]()
In reply to Stefan MetzmacherThis will be used in common between client and server in order to destroy all resources attached to a connection.
--- .../common/smbdirect/smbdirect_connection.c | 4 - fs/smb/common/smbdirect/smbdirect_internal.h | 8 + fs/smb/common/smbdirect/smbdirect_mr.c | 1 - fs/smb/common/smbdirect/smbdirect_socket.c | 151 ++++++++++++++++++ 4 files changed, 159 insertions(+), 5 deletions(-)
fs/smb/common/smbdirect/smbdirect_connection.c +0 -4
@@ -77,8 +77,6 @@ static u32 smbdirect_rdma_rw_send_wrs(struct ib_device *dev,return factor * attr->cap.max_rdma_ctxs; } -static void smbdirect_connection_destroy_qp (struct smbdirect_socket *sc); -__maybe_unused /* this is temporary while this file is included in others */ static int smbdirect_connection_create_qp (struct smbdirect_socket *sc) {
@@ -258,8 +256,6 @@ static void smbdirect_connection_destroy_qp(struct smbdirect_socket *sc) } } -static void smbdirect_connection_destroy_mem_pools (struct smbdirect_socket *sc); -__maybe_unused /* this is temporary while this file is included in others */ static int smbdirect_connection_create_mem_pools (struct smbdirect_socket *sc) {
fs/smb/common/smbdirect/smbdirect_internal.h +8 -0
@@ -40,6 +40,14 @@ static int smbdirect_socket_wait_for_credits(struct smbdirect_socket *sc,atomic_t *total_credits, int needed); +static void smbdirect_connection_destroy_qp (struct smbdirect_socket *sc); + +static void smbdirect_connection_destroy_mem_pools (struct smbdirect_socket *sc); + +static void smbdirect_connection_put_recv_io (struct smbdirect_recv_io *msg); +static void smbdirect_connection_idle_timer_work (struct work_struct *work); +static void smbdirect_connection_destroy_mr_list (struct smbdirect_socket *sc); +#endif /* __FS_SMB_COMMON_SMBDIRECT_INTERNAL_H__ */
fs/smb/common/smbdirect/smbdirect_mr.c +0 -1
@@ -6,7 +6,6 @@#include "smbdirect_internal.h" -static void smbdirect_connection_destroy_mr_list (struct smbdirect_socket *sc);static void smbdirect_connection_mr_io_recovery_work (struct work_struct *work);/*
fs/smb/common/smbdirect/smbdirect_socket.c +151 -0
157 lines not on this page · raw message
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 40/145 smb: smbdirect: introduce smbdirect_connection_rdma_{established,event_handler}()
In reply to Stefan MetzmacherThis will be used by client and server in future, it will be used after the rdma connection is established in order to simplify the events happening on an established connection.
We'll also have smbdirect_{connect,accept}_rdma_event_handler functions which will be used before the rdma connection is established.
--- .../common/smbdirect/smbdirect_connection.c | 110 ++++++++++++++++++ fs/smb/common/smbdirect/smbdirect_socket.h | 8 ++ 2 files changed, 118 insertions(+)
fs/smb/common/smbdirect/smbdirect_connection.c +110 -0
@@ -44,6 +44,116 @@ static void smbdirect_connection_qp_event_handler(struct ib_event *event, void * } } +static int smbdirect_connection_rdma_event_handler (struct rdma_cm_id *id, +struct rdma_cm_event *event) +{ +struct smbdirect_socket *sc = id->context; +int ret = -ECONNRESET ; + +if (event->event ==RDMA_CM_EVENT_DEVICE_REMOVAL ) + ret = -ENETDOWN ; +if (IS_ERR (SMBDIRECT_DEBUG_ERR_PTR (event->status))) + ret = event->status; + +/* +* cma_cm_event_handler() has +* lockdep_assert_held(&id_priv->handler_mutex); +* +* Mutexes are not allowed in interrupts, +* and we rely on not being in an interrupt here. +*/ +WARN_ON_ONCE (in_interrupt ()); + +if (event->event != sc->rdma.expected_event) { +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"%s (first_error=%1pe, expected=%s) => event=%s status=%d => ret=%1pe\n" , +smbdirect_socket_status_string (sc->status), +SMBDIRECT_DEBUG_ERR_PTR (sc->first_error), +rdma_event_msg (sc->rdma.expected_event), +rdma_event_msg (event->event), + event->status, +SMBDIRECT_DEBUG_ERR_PTR (ret)); + +/* +* If we get RDMA_CM_EVENT_DEVICE_REMOVAL, +* we should change to SMBDIRECT_SOCKET_DISCONNECTED, +* so that rdma_disconnect() is avoided later via +* smbdirect_socket_schedule_cleanup[_status]() => +* smbdirect_socket_cleanup_work(). +* +* As otherwise we'd set SMBDIRECT_SOCKET_DISCONNECTING, +* but never ever get RDMA_CM_EVENT_DISCONNECTED and +* never reach SMBDIRECT_SOCKET_DISCONNECTED. +*/ +if (event->event ==RDMA_CM_EVENT_DEVICE_REMOVAL ) +smbdirect_socket_schedule_cleanup_status (sc, +SMBDIRECT_LOG_ERR , + ret, +SMBDIRECT_SOCKET_DISCONNECTED ); +else +smbdirect_socket_schedule_cleanup (sc, ret); +if (sc->ib.qp) +ib_drain_qp (sc->ib.qp); +return 0 ; + } + +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_INFO , +"%s (first_error=%1pe) event=%s\n" , +smbdirect_socket_status_string (sc->status), +SMBDIRECT_DEBUG_ERR_PTR (sc->first_error), +rdma_event_msg (event->event)); + +switch (event->event) { +case RDMA_CM_EVENT_DISCONNECTED : +/* +* We need to change to SMBDIRECT_SOCKET_DISCONNECTED, +* so that rdma_disconnect() is avoided later via +* smbdirect_socket_schedule_cleanup_status() => +* smbdirect_socket_cleanup_work(). +* +* As otherwise we'd set SMBDIRECT_SOCKET_DISCONNECTING, +* but never ever get RDMA_CM_EVENT_DISCONNECTED and +* never reach SMBDIRECT_SOCKET_DISCONNECTED. +* +* This is also a normal disconnect so +* SMBDIRECT_LOG_INFO should be good enough +* and avoids spamming the default logs. +*/ +smbdirect_socket_schedule_cleanup_status (sc, +SMBDIRECT_LOG_INFO , + ret, +SMBDIRECT_SOCKET_DISCONNECTED ); +if (sc->ib.qp) +ib_drain_qp (sc->ib.qp); +return 0 ; + +default : +break ; + } + +/* +* This is an internal error, should be handled above via +* event->event != sc->rdma.expected_event already. +*/ +WARN_ON_ONCE (sc->rdma.expected_event !=RDMA_CM_EVENT_DISCONNECTED ); +smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); +return 0 ; +} + +__maybe_unused /* this is temporary while this file is included in others */ +static void smbdirect_connection_rdma_established (struct smbdirect_socket *sc) +{ +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_INFO , +"rdma established: device: %.*s local: %pISpsfc remote: %pISpsfc\n" , +IB_DEVICE_NAME_MAX , + sc->ib.dev->name, + &sc->rdma.cm_id->route.addr.src_addr, + &sc->rdma.cm_id->route.addr.dst_addr); + + sc->rdma.cm_id->event_handler = smbdirect_connection_rdma_event_handler; + sc->rdma.expected_event =RDMA_CM_EVENT_DISCONNECTED ; +} +static u32 smbdirect_rdma_rw_send_wrs (struct ib_device *dev,const struct ib_qp_init_attr *attr) {
fs/smb/common/smbdirect/smbdirect_socket.h +8 -0
@@ -111,6 +111,12 @@ struct smbdirect_socket {/* RDMA related */ struct {struct rdma_cm_id *cm_id; +/* +* The expected event in our current +* cm_id->event_handler, all other events +* are treated as an error. +*/ +enum rdma_cm_event_type expected_event;/* * This is for iWarp MPA v1 */
@@ -487,6 +493,8 @@ static __always_inline void smbdirect_socket_init(struct smbdirect_socket *sc)INIT_WORK (&sc->disconnect_work, __smbdirect_socket_disabled_work);disable_work_sync (&sc->disconnect_work); + sc->rdma.expected_event =RDMA_CM_EVENT_INTERNAL ; + sc->ib.poll_ctx =IB_POLL_UNBOUND_WORKQUEUE ;INIT_WORK (&sc->idle.immediate_work, __smbdirect_socket_disabled_work);
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 41/145 smb: smbdirect: introduce smbdirect_connection_recvmsg()
In reply to Stefan MetzmacherThis is basically a copy of smbd_recv() in the client.
And it's very similar to smb_direct_read() in the server.
It will replace both in the following commits.
--- .../common/smbdirect/smbdirect_connection.c | 151 ++++++++++++++++++ 1 file changed, 151 insertions(+)
fs/smb/common/smbdirect/smbdirect_connection.c +151 -0
@@ -1062,6 +1062,157 @@ static void smbdirect_connection_recv_io_refill_work(struct work_struct *work) } } +__maybe_unused /* this is temporary while this file is included in others */ +static int smbdirect_connection_recvmsg (struct smbdirect_socket *sc, +struct msghdr *msg, +unsigned int flags) +{ +struct smbdirect_recv_io *response; +struct smbdirect_data_transfer *data_transfer; +size_t size =iov_iter_count (&msg->msg_iter); +int to_copy, to_read, data_read, offset; +u32 data_length, remaining_data_length, data_offset; +int ret; + +if (WARN_ON_ONCE (flags)) +return -EINVAL ;/* no flags support for now */ + +if (WARN_ON_ONCE (iov_iter_rw (&msg->msg_iter) !=ITER_DEST )) +return -EINVAL ;/* It's a bug in upper layer to get there */ + +again: +if (sc->status !=SMBDIRECT_SOCKET_CONNECTED ) { +smbdirect_log_read (sc,SMBDIRECT_LOG_ERR , +"status=%s first_error=%1pe => %s\n" , +smbdirect_socket_status_string (sc->status), +SMBDIRECT_DEBUG_ERR_PTR (sc->first_error), +errname (-ENOTCONN )); +return -ENOTCONN ; + } + +/* +* No need to hold the reassembly queue lock all the time as we are +* the only one reading from the front of the queue. The transport +* may add more entries to the back of the queue at the same time +*/ +smbdirect_log_read (sc,SMBDIRECT_LOG_INFO , +"size=%zd sc->recv_io.reassembly.data_length=%d\n" , + size, sc->recv_io.reassembly.data_length); +if (sc->recv_io.reassembly.data_length >= size) { +int queue_length; +int queue_removed =0 ; +unsigned long flags; + +/* +* Need to make sure reassembly_data_length is read before +* reading reassembly_queue_length and calling +* smbdirect_connection_reassembly_first_recv_io. This call is lock free +* as we never read at the end of the queue which are being +* updated in SOFTIRQ as more data is received +*/ +virt_rmb (); + queue_length = sc->recv_io.reassembly.queue_length; + data_read =0 ; + to_read = size; + offset = sc->recv_io.reassembly.first_entry_offset; +while (data_read < size) { + response =smbdirect_connection_reassembly_first_recv_io (sc); + data_transfer = (void *)response->packet; + data_length =le32_to_cpu (data_transfer->data_length); + remaining_data_length = +le32_to_cpu ( + data_transfer->remaining_data_length); + data_offset =le32_to_cpu (data_transfer->data_offset); + +/* +* The upper layer expects RFC1002 length at the +* beginning of the payload. Return it to indicate +* the total length of the packet. This minimize the +* change to upper layer packet processing logic. This +* will be eventually remove when an intermediate +* transport layer is added +*/ +if (response->first_segment && size ==4 ) { +unsigned int rfc1002_len = + data_length + remaining_data_length; +__be32 rfc1002_hdr =cpu_to_be32 (rfc1002_len); + +if (copy_to_iter (&rfc1002_hdr,sizeof (rfc1002_hdr), + &msg->msg_iter) !=sizeof (rfc1002_hdr)) +return -EFAULT ; + data_read =4 ; + response->first_segment = false; +smbdirect_log_read (sc,SMBDIRECT_LOG_INFO , +"returning rfc1002 length %d\n" , + rfc1002_len); + goto read_rfc1002_done; + } + + to_copy =min_t (int, data_length - offset, to_read); +if (copy_to_iter ((u8 *)data_transfer + data_offset + offset, + to_copy, &msg->msg_iter) != to_copy) +return -EFAULT ; + +/* move on to the next buffer? */ +if (to_copy == data_length - offset) { + queue_length--; +/* +* No need to lock if we are not at the +* end of the queue +*/ +if (queue_length) +list_del (&response->list); +else { +spin_lock_irqsave ( + &sc->recv_io.reassembly.lock, flags); +list_del (&response->list); +spin_unlock_irqrestore ( + &sc->recv_io.reassembly.lock, flags); + } + queue_removed++; + sc->statistics.dequeue_reassembly_queue++; +smbdirect_connection_put_recv_io (response); + offset =0 ; +smbdirect_log_read (sc,SMBDIRECT_LOG_INFO , +"smbdirect_connection_put_recv_io offset=0\n" ); + }else + offset += to_copy; + + to_read -= to_copy; + data_read += to_copy; + +smbdirect_log_read (sc,SMBDIRECT_LOG_INFO , +"memcpy %d bytes len-ofs=%u => todo=%u done=%u ofs=%u\n" , + to_copy, data_length - offset, + to_read, data_read, offset); + } + +spin_lock_irqsave (&sc->recv_io.reassembly.lock, flags); + sc->recv_io.reassembly.data_length -= data_read; + sc->recv_io.reassembly.queue_length -= queue_removed; +spin_unlock_irqrestore (&sc->recv_io.reassembly.lock, flags); + + sc->recv_io.reassembly.first_entry_offset = offset; +smbdirect_log_read (sc,SMBDIRECT_LOG_INFO , +"returning data_read=%d reassembly_length=%d first_ofs=%u\n" , + data_read, sc->recv_io.reassembly.data_length, + sc->recv_io.reassembly.first_entry_offset); +read_rfc1002_done: +return data_read; + } + +smbdirect_log_read (sc,SMBDIRECT_LOG_INFO , +"wait_event on more data\n" ); + ret =wait_event_interruptible (sc->recv_io.reassembly.wait_queue, + sc->recv_io.reassembly.data_length >= size || + sc->status !=SMBDIRECT_SOCKET_CONNECTED ); +/* Don't return any data if interrupted */ +if (ret) +return ret; + + goto again; +} +static bool smbdirect_map_sges_single_page (struct smbdirect_map_sges *state,struct page *page,size_t off,size_t len) {
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 42/145 smb: smbdirect: introduce smbdirect_connection_grant_recv_credits()
In reply to Stefan MetzmacherThis is a copy of manage_credits_prior_sending() in the server.
It is very similar to manage_credits_prior_sending() in the client, the only difference is that atomic_add(new_credits, &sc->recv_io.credits.count) is done in the caller there.
It will replace both versions in future.
--- .../common/smbdirect/smbdirect_connection.c | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+)
fs/smb/common/smbdirect/smbdirect_connection.c +20 -0
@@ -704,6 +704,26 @@ static void smbdirect_connection_idle_timer_work(struct work_struct *work)queue_work (sc->workqueue, &sc->idle.immediate_work); } +__maybe_unused /* this is temporary while this file is included in others */ +static u16 smbdirect_connection_grant_recv_credits (struct smbdirect_socket *sc) +{ +u16 new_credits; + +if (atomic_read (&sc->recv_io.credits.count) >= sc->recv_io.credits.target) +return 0 ; + + new_credits =atomic_read (&sc->recv_io.posted.count); +if (new_credits ==0 ) +return 0 ; + + new_credits -=atomic_read (&sc->recv_io.credits.count); +if (new_credits <=0 ) +return 0 ; + +atomic_add (new_credits, &sc->recv_io.credits.count); +return new_credits; +} +__maybe_unused /* this is temporary while this file is included in others */ static void smbdirect_connection_send_io_done (struct ib_cq *cq,struct ib_wc *wc) {
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 43/145 smb: smbdirect: introduce smbdirect_connection_request_keep_alive()
In reply to Stefan MetzmacherThis a copy of manage_keep_alive_before_sending() in client and server, it will replace these in future.
--- .../common/smbdirect/smbdirect_connection.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
fs/smb/common/smbdirect/smbdirect_connection.c +19 -0
@@ -724,6 +724,25 @@ static u16 smbdirect_connection_grant_recv_credits(struct smbdirect_socket *sc)return new_credits; } +__maybe_unused /* this is temporary while this file is included in others */ +static bool smbdirect_connection_request_keep_alive (struct smbdirect_socket *sc) +{ +const struct smbdirect_socket_parameters *sp = &sc->parameters; + +if (sc->idle.keepalive ==SMBDIRECT_KEEPALIVE_PENDING ) { + sc->idle.keepalive =SMBDIRECT_KEEPALIVE_SENT ; +/* +* Now use the keepalive timeout (instead of keepalive interval) +* in order to wait for a response +*/ +mod_delayed_work (sc->workqueue, &sc->idle.timer_work, +msecs_to_jiffies (sp->keepalive_timeout_msec)); +return true; + } + +return false; +} +__maybe_unused /* this is temporary while this file is included in others */ static void smbdirect_connection_send_io_done (struct ib_cq *cq,struct ib_wc *wc) {
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 44/145 smb: smbdirect: introduce smbdirect_connection_send_iter() and related functions
In reply to Stefan MetzmacherThis is basically a copy of the following functions in the server:
smb_direct_post_send => smbdirect_connection_post_send_wrsmb_direct_send_ctx_init => smbdirect_connection_send_batch_initsmb_direct_flush_send_list => smbdirect_connection_send_batch_flush
wait_for_send_lcredit => smbdirect_connection_wait_for_send_lcredit wait_for_send_credits => smbdirect_connection_wait_for_send_credits post_sendmsg => smbdirect_connection_post_send_io smb_direct_post_send_data => smbdirect_connection_send_single_iter smb_direct_send_iter => smbdirect_connection_send_iter
They will replace the server functions soon and will also be used in the client as smbdirect_connection_send_single_iter() is very similar to smbd_post_send_iter().
There's also a smbdirect_connection_send_wait_zero_pending() helper that can be used together with smbdirect_connection_send_single_iter() in the client until it can use smbdirect_connection_send_iter(), which can happen with David's refactoring.
--- .../common/smbdirect/smbdirect_connection.c | 408 +++++++++++++++++- 1 file changed, 407 insertions(+), 1 deletion(-)
fs/smb/common/smbdirect/smbdirect_connection.c +407 -1
@@ -743,7 +743,414 @@ static bool smbdirect_connection_request_keep_alive(struct smbdirect_socket *sc)return false; } +static int smbdirect_connection_post_send_wr (struct smbdirect_socket *sc, +struct ib_send_wr *wr) +{ +int ret; + +if (unlikely (sc->first_error)) +return sc->first_error; + +atomic_inc (&sc->send_io.pending.count); + ret =ib_post_send (sc->ib.qp, wr,NULL ); +if (ret) { +atomic_dec (&sc->send_io.pending.count); +smbdirect_log_rdma_send (sc,SMBDIRECT_LOG_ERR , +"ib_post_send() failed %1pe\n" , +SMBDIRECT_DEBUG_ERR_PTR (ret)); +smbdirect_socket_schedule_cleanup (sc, ret); + } + +return ret; +} + +static void smbdirect_connection_send_batch_init (struct smbdirect_send_batch *batch, +bool need_invalidate_rkey, +unsigned int remote_key) +{ +INIT_LIST_HEAD (&batch->msg_list); + batch->wr_cnt =0 ; + batch->need_invalidate_rkey = need_invalidate_rkey; + batch->remote_key = remote_key; +} + +static int smbdirect_connection_send_batch_flush (struct smbdirect_socket *sc, +struct smbdirect_send_batch *batch, +bool is_last) +{ +struct smbdirect_send_io *first, *last; +int ret; + +if (list_empty (&batch->msg_list)) +return 0 ; + + first =list_first_entry (&batch->msg_list, + struct smbdirect_send_io, + sibling_list); + last =list_last_entry (&batch->msg_list, + struct smbdirect_send_io, + sibling_list); + +if (batch->need_invalidate_rkey) { + first->wr.opcode =IB_WR_SEND_WITH_INV ; + first->wr.ex.invalidate_rkey = batch->remote_key; + batch->need_invalidate_rkey = false; + batch->remote_key =0 ; + } + + last->wr.send_flags =IB_SEND_SIGNALED ; + last->wr.wr_cqe = &last->cqe; + +/* +* Remove last from send_ctx->msg_list +* and splice the rest of send_ctx->msg_list +* to last->sibling_list. +* +* send_ctx->msg_list is a valid empty list +* at the end. +*/ +list_del_init (&last->sibling_list); +list_splice_tail_init (&batch->msg_list, &last->sibling_list); + batch->wr_cnt =0 ; + + ret =smbdirect_connection_post_send_wr (sc, &first->wr); +if (ret) { +struct smbdirect_send_io *sibling, *next; + +list_for_each_entry_safe (sibling, next, &last->sibling_list, sibling_list) { +list_del_init (&sibling->sibling_list); +smbdirect_connection_free_send_io (sibling); + } +smbdirect_connection_free_send_io (last); + } + +return ret; +} + + +static int smbdirect_connection_wait_for_send_lcredit (struct smbdirect_socket *sc, +struct smbdirect_send_batch *batch) +{ +if (batch &&atomic_read (&sc->send_io.lcredits.count) <=1 ) { +int ret; + + ret =smbdirect_connection_send_batch_flush (sc, batch, false); +if (ret) +return ret; + } + +return smbdirect_socket_wait_for_credits (sc, +SMBDIRECT_SOCKET_CONNECTED , + -ENOTCONN , + &sc->send_io.lcredits.wait_queue, + &sc->send_io.lcredits.count, +1 ); +} + +static int smbdirect_connection_wait_for_send_credits (struct smbdirect_socket *sc, +struct smbdirect_send_batch *batch) +{ +if (batch && (batch->wr_cnt >=16 ||atomic_read (&sc->send_io.credits.count) <=1 )) { +int ret; + + ret =smbdirect_connection_send_batch_flush (sc, batch, false); +if (ret) +return ret; + } + +return smbdirect_socket_wait_for_credits (sc, +SMBDIRECT_SOCKET_CONNECTED , + -ENOTCONN , + &sc->send_io.credits.wait_queue, + &sc->send_io.credits.count, +1 ); +} + +static void smbdirect_connection_send_io_done (struct ib_cq *cq,struct ib_wc *wc); + +static int smbdirect_connection_post_send_io (struct smbdirect_socket *sc, +struct smbdirect_send_batch *batch, +struct smbdirect_send_io *msg) +{ +int i; + +for (i =0 ; i < msg->num_sge; i++) +ib_dma_sync_single_for_device (sc->ib.dev, + msg->sge[i].addr, msg->sge[i].length, +DMA_TO_DEVICE ); + + msg->cqe.done = smbdirect_connection_send_io_done; + msg->wr.wr_cqe = &msg->cqe; + msg->wr.opcode =IB_WR_SEND ; + msg->wr.sg_list = &msg->sge[0 ]; + msg->wr.num_sge = msg->num_sge; + msg->wr.next =NULL ; + +if (batch) { + msg->wr.send_flags =0 ; +if (!list_empty (&batch->msg_list)) { +struct smbdirect_send_io *last; + + last =list_last_entry (&batch->msg_list, + struct smbdirect_send_io, + sibling_list); + last->wr.next = &msg->wr; + } +list_add_tail (&msg->sibling_list, &batch->msg_list); + batch->wr_cnt++; +return 0 ; + } + + msg->wr.send_flags =IB_SEND_SIGNALED ; +return smbdirect_connection_post_send_wr (sc, &msg->wr); +} + +static int smbdirect_connection_send_single_iter (struct smbdirect_socket *sc, +struct smbdirect_send_batch *batch, +struct iov_iter *iter, +unsigned int flags, +u32 remaining_data_length) +{ +const struct smbdirect_socket_parameters *sp = &sc->parameters; +struct smbdirect_send_io *msg; +struct smbdirect_data_transfer *packet; +size_t header_length; +u16 new_credits =0 ; +u32 data_length =0 ; +int ret; + +if (WARN_ON_ONCE (flags)) +return -EINVAL ;/* no flags support for now */ + +if (iter) { +if (WARN_ON_ONCE (iov_iter_rw (iter) !=ITER_SOURCE )) +return -EINVAL ;/* It's a bug in upper layer to get there */ + + header_length =sizeof (struct smbdirect_data_transfer ); +if (WARN_ON_ONCE (remaining_data_length ==0 || +iov_iter_count (iter) > remaining_data_length)) +return -EINVAL ; + }else { +/* If this is a packet without payload, don't send padding */ + header_length = offsetof(struct smbdirect_data_transfer , padding); +if (WARN_ON_ONCE (remaining_data_length)) +return -EINVAL ; + } + +if (sc->status !=SMBDIRECT_SOCKET_CONNECTED ) { +smbdirect_log_write (sc,SMBDIRECT_LOG_ERR , +"status=%s first_error=%1pe => %s\n" , +smbdirect_socket_status_string (sc->status), +SMBDIRECT_DEBUG_ERR_PTR (sc->first_error), +errname (-ENOTCONN )); +return -ENOTCONN ; + } + + ret =smbdirect_connection_wait_for_send_lcredit (sc, batch); +if (ret) + goto lcredit_failed; + + ret =smbdirect_connection_wait_for_send_credits (sc, batch); +if (ret) + goto credit_failed; + + msg =smbdirect_connection_alloc_send_io (sc); +if (IS_ERR (msg)) { + ret =PTR_ERR (msg); + goto alloc_failed; + } + +/* Map the packet to DMA */ + msg->sge[0 ].addr =ib_dma_map_single (sc->ib.dev, + msg->packet, + header_length, +DMA_TO_DEVICE ); + ret =ib_dma_mapping_error (sc->ib.dev, msg->sge[0 ].addr); +if (ret) + goto err; + + msg->sge[0 ].length = header_length; + msg->sge[0 ].lkey = sc->ib.pd->local_dma_lkey; + msg->num_sge =1 ; + +if (iter) { +struct smbdirect_map_sges extract = { + .num_sge = msg->num_sge, + .max_sge =ARRAY_SIZE (msg->sge), + .sge = msg->sge, + .device = sc->ib.dev, + .local_dma_lkey = sc->ib.pd->local_dma_lkey, + .direction =DMA_TO_DEVICE , + }; +size_t payload_len =umin (iov_iter_count (iter), + sp->max_send_size -sizeof (*packet)); + + ret =smbdirect_map_sges_from_iter (iter, payload_len, &extract); +if (ret <0 ) + goto err; + data_length = ret; + remaining_data_length -= data_length; + msg->num_sge = extract.num_sge; + } + +/* Fill in the packet header */ + packet = (struct smbdirect_data_transfer *)msg->packet; + packet->credits_requested =cpu_to_le16 (sp->send_credit_target); + new_credits =smbdirect_connection_grant_recv_credits (sc); + packet->credits_granted =cpu_to_le16 (new_credits); + + packet->flags =0 ; +if (smbdirect_connection_request_keep_alive (sc)) + packet->flags |=cpu_to_le16 (SMBDIRECT_FLAG_RESPONSE_REQUESTED ); + + packet->reserved =0 ; +if (!data_length) + packet->data_offset =0 ; +else + packet->data_offset =cpu_to_le32 (24 ); + packet->data_length =cpu_to_le32 (data_length); + packet->remaining_data_length =cpu_to_le32 (remaining_data_length); + packet->padding =0 ; + +smbdirect_log_outgoing (sc,SMBDIRECT_LOG_INFO , +"DataOut: %s=%u, %s=%u, %s=0x%x, %s=%u, %s=%u, %s=%u\n" , +"CreditsRequested" , +le16_to_cpu (packet->credits_requested), +"CreditsGranted" , +le16_to_cpu (packet->credits_granted), +"Flags" , +le16_to_cpu (packet->flags), +"RemainingDataLength" , +le32_to_cpu (packet->remaining_data_length), +"DataOffset" , +le32_to_cpu (packet->data_offset), +"DataLength" , +le32_to_cpu (packet->data_length)); + + ret =smbdirect_connection_post_send_io (sc, batch, msg); +if (ret) + goto err; +return data_length; +err: +smbdirect_connection_free_send_io (msg); +alloc_failed: +atomic_inc (&sc->send_io.credits.count); +credit_failed: +atomic_inc (&sc->send_io.lcredits.count); +lcredit_failed: +return ret; +} +__maybe_unused /* this is temporary while this file is included in others */ +static int smbdirect_connection_send_wait_zero_pending (struct smbdirect_socket *sc) +{ +/* +* As an optimization, we don't wait for individual I/O to finish +* before sending the next one. +* Send them all and wait for pending send count to get to 0 +* that means all the I/Os have been out and we are good to return +*/ + +wait_event (sc->send_io.pending.zero_wait_queue, +atomic_read (&sc->send_io.pending.count) ==0 || + sc->status !=SMBDIRECT_SOCKET_CONNECTED ); +if (sc->status !=SMBDIRECT_SOCKET_CONNECTED ) { +smbdirect_log_write (sc,SMBDIRECT_LOG_ERR , +"status=%s first_error=%1pe => %s\n" , +smbdirect_socket_status_string (sc->status), +SMBDIRECT_DEBUG_ERR_PTR (sc->first_error), +errname (-ENOTCONN )); +return -ENOTCONN ; + } + +return 0 ; +} + +__maybe_unused /* this is temporary while this file is included in others */ +static int smbdirect_connection_send_iter (struct smbdirect_socket *sc, +struct iov_iter *iter, +unsigned int flags, +bool need_invalidate, +unsigned int remote_key) +{ +const struct smbdirect_socket_parameters *sp = &sc->parameters; +struct smbdirect_send_batch batch; +int total_count =iov_iter_count (iter); +int ret; +int error =0 ; +__be32 hdr; + +if (WARN_ON_ONCE (flags)) +return -EINVAL ;/* no flags support for now */ + +if (WARN_ON_ONCE (iov_iter_rw (iter) !=ITER_SOURCE )) +return -EINVAL ;/* It's a bug in upper layer to get there */ + +if (sc->status !=SMBDIRECT_SOCKET_CONNECTED ) { +smbdirect_log_write (sc,SMBDIRECT_LOG_ERR , +"status=%s first_error=%1pe => %s\n" , +smbdirect_socket_status_string (sc->status), +SMBDIRECT_DEBUG_ERR_PTR (sc->first_error), +errname (-ENOTCONN )); +return -ENOTCONN ; + } + +/* +* For now we expect the iter to have the full +* message, including a 4 byte length header. +*/ +if (iov_iter_count (iter) <=4 ) +return -EINVAL ; +if (!copy_from_iter_full (&hdr,sizeof (hdr), iter)) +return -EFAULT ; +if (iov_iter_count (iter) !=be32_to_cpu (hdr)) +return -EINVAL ; + +/* +* The size must fit into the negotiated +* fragmented send size. +*/ +if (iov_iter_count (iter) > sp->max_fragmented_send_size) +return -EMSGSIZE ; + +smbdirect_log_write (sc,SMBDIRECT_LOG_INFO , +"Sending (RDMA): length=%zu\n" , +iov_iter_count (iter)); + +smbdirect_connection_send_batch_init (&batch, need_invalidate, remote_key); +while (iov_iter_count (iter)) { + ret =smbdirect_connection_send_single_iter (sc, + &batch, + iter, + flags, +iov_iter_count (iter)); +if (unlikely (ret <0 )) { + error = ret; +break ; + } + } + + ret =smbdirect_connection_send_batch_flush (sc, &batch, true); +if (unlikely (ret && !error)) + error = ret; + +/* +* As an optimization, we don't wait for individual I/O to finish +* before sending the next one. +* Send them all and wait for pending send count to get to 0 +* that means all the I/Os have been out and we are good to return +*/ + + ret =smbdirect_connection_send_wait_zero_pending (sc); +if (unlikely (ret && !error)) + error = ret; + +if (unlikely (error)) +return error; + +return total_count; +} +static void smbdirect_connection_send_io_done (struct ib_cq *cq,struct ib_wc *wc) { struct smbdirect_send_io *msg =
@@ -1455,7 +1862,6 @@ static ssize_t smbdirect_map_sges_from_folioq(struct iov_iter *iter,* BVEC-type iterator and the pages pinned, ref'd or otherwise held in some * way. */ -__maybe_unused /* this is temporary while this file is included in others */ static ssize_t smbdirect_map_sges_from_iter (struct iov_iter *iter,size_t len,struct smbdirect_map_sges *state) {
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 45/145 smb: smbdirect: introduce smbdirect_connection_send_immediate_work()
In reply to Stefan MetzmacherThis is a combination of smb_direct_send_immediate_work() in the server as well as send_immediate_empty_message() and smbd_post_send_empty() in the client.
smbdirect_connection_send_immediate_work() replace all of them in client and server.
--- .../common/smbdirect/smbdirect_connection.c | 22 +++++++++++++++++++ 1 file changed, 22 insertions(+)
fs/smb/common/smbdirect/smbdirect_connection.c +22 -0
@@ -1193,6 +1193,28 @@ static void smbdirect_connection_send_io_done(struct ib_cq *cq, struct ib_wc *wcwake_up (&sc->send_io.pending.dec_wait_queue); } +__maybe_unused /* this is temporary while this file is included in others */ +static void smbdirect_connection_send_immediate_work (struct work_struct *work) +{ +struct smbdirect_socket *sc = + container_of(work, struct smbdirect_socket, idle.immediate_work); +int ret; + +if (sc->status !=SMBDIRECT_SOCKET_CONNECTED ) +return ; + +smbdirect_log_keep_alive (sc,SMBDIRECT_LOG_INFO , +"send an empty message\n" ); + sc->statistics.send_empty++; + ret =smbdirect_connection_send_single_iter (sc,NULL ,NULL ,0 ,0 ); +if (ret <0 ) { +smbdirect_log_write (sc,SMBDIRECT_LOG_ERR , +"smbdirect_connection_send_single_iter ret=%1pe\n" , +SMBDIRECT_DEBUG_ERR_PTR (ret)); +smbdirect_socket_schedule_cleanup (sc, ret); + } +} +__maybe_unused /* this is temporary while this file is included in others */ static int smbdirect_connection_post_recv_io (struct smbdirect_recv_io *msg) {
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 46/145 smb: smbdirect: introduce smbdirect_connection_negotiation_done()
In reply to Stefan MetzmacherThis will be used by client and server in order to turn the connection into a usable state.
--- .../common/smbdirect/smbdirect_connection.c | 48 ++++++++++++++++++- 1 file changed, 46 insertions(+), 2 deletions(-)
fs/smb/common/smbdirect/smbdirect_connection.c +46 -2
@@ -19,6 +19,9 @@ struct smbdirect_map_sges {static ssize_t smbdirect_map_sges_from_iter (struct iov_iter *iter,size_t len,struct smbdirect_map_sges *state); +static void smbdirect_connection_recv_io_refill_work (struct work_struct *work); +static void smbdirect_connection_send_immediate_work (struct work_struct *work); +__maybe_unused /* this is temporary while this file is included in others */ static void smbdirect_connection_qp_event_handler (struct ib_event *event,void *context) {
@@ -154,6 +157,49 @@ static void smbdirect_connection_rdma_established(struct smbdirect_socket *sc) sc->rdma.expected_event =RDMA_CM_EVENT_DISCONNECTED ; } +__maybe_unused /* this is temporary while this file is included in others */ +static void smbdirect_connection_negotiation_done (struct smbdirect_socket *sc) +{ +if (unlikely (sc->first_error)) +return ; + +if (sc->status !=SMBDIRECT_SOCKET_NEGOTIATE_RUNNING ) { +/* +* Something went wrong... +*/ +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"status=%s first_error=%1pe local: %pISpsfc remote: %pISpsfc\n" , +smbdirect_socket_status_string (sc->status), +SMBDIRECT_DEBUG_ERR_PTR (sc->first_error), + &sc->rdma.cm_id->route.addr.src_addr, + &sc->rdma.cm_id->route.addr.dst_addr); +return ; + } + +/* +* We are done, so we can wake up the waiter. +*/ +WARN_ONCE (sc->status ==SMBDIRECT_SOCKET_CONNECTED , +"status=%s first_error=%1pe" , +smbdirect_socket_status_string (sc->status), +SMBDIRECT_DEBUG_ERR_PTR (sc->first_error)); + sc->status =SMBDIRECT_SOCKET_CONNECTED ; + +/* +* We need to setup the refill and send immediate work +* in order to get a working connection. +*/ +INIT_WORK (&sc->recv_io.posted.refill_work, smbdirect_connection_recv_io_refill_work); +INIT_WORK (&sc->idle.immediate_work, smbdirect_connection_send_immediate_work); + +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_INFO , +"negotiated: local: %pISpsfc remote: %pISpsfc\n" , + &sc->rdma.cm_id->route.addr.src_addr, + &sc->rdma.cm_id->route.addr.dst_addr); + +wake_up (&sc->status_wait); +} +static u32 smbdirect_rdma_rw_send_wrs (struct ib_device *dev,const struct ib_qp_init_attr *attr) {
@@ -1193,7 +1239,6 @@ static void smbdirect_connection_send_io_done(struct ib_cq *cq, struct ib_wc *wcwake_up (&sc->send_io.pending.dec_wait_queue); } -__maybe_unused /* this is temporary while this file is included in others */ static void smbdirect_connection_send_immediate_work (struct work_struct *work) { struct smbdirect_socket *sc =
@@ -1511,7 +1556,6 @@ static int smbdirect_connection_recv_io_refill(struct smbdirect_socket *sc)return posted; } -__maybe_unused /* this is temporary while this file is included in others */ static void smbdirect_connection_recv_io_refill_work (struct work_struct *work) { struct smbdirect_socket *sc =
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 47/145 smb: smbdirect: introduce smbdirect_mr_io_fill_buffer_descriptor()
In reply to Stefan MetzmacherThis will be used by the client instead of dereferencing struct smbdirect_mr_io internals.
--- fs/smb/common/smbdirect/smbdirect_mr.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
fs/smb/common/smbdirect/smbdirect_mr.c +18 -0
@@ -452,6 +452,24 @@ smbdirect_connection_register_mr_io(struct smbdirect_socket *sc,mutex_unlock (&mr->mutex);return NULL ; } + +__maybe_unused /* this is temporary while this file is included in others */ +static void smbdirect_mr_io_fill_buffer_descriptor (struct smbdirect_mr_io *mr, +struct smbdirect_buffer_descriptor_v1 *v1) +{ +mutex_lock (&mr->mutex); +if (mr->state ==SMBDIRECT_MR_REGISTERED ) { + v1->offset =cpu_to_le64 (mr->mr->iova); + v1->token =cpu_to_le32 (mr->mr->rkey); + v1->length =cpu_to_le32 (mr->mr->length); + }else { + v1->offset =cpu_to_le64 (U64_MAX ); + v1->token =cpu_to_le32 (U32_MAX ); + v1->length =cpu_to_le32 (U32_MAX ); + } +mutex_unlock (&mr->mutex); +} +/* * Deregister a MR after I/O is done * This function may wait if remote invalidation is not used
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 48/145 smb: smbdirect: introduce smbdirect_connection_legacy_debug_proc_show()
In reply to Stefan MetzmacherThis will be used by the client in order to keep the debug output in the current way without the need to access struct smbdirect_socket internals.
--- .../common/smbdirect/smbdirect_all_c_files.c | 1 + fs/smb/common/smbdirect/smbdirect_debug.c | 88 +++++++++++++++++++ 2 files changed, 89 insertions(+) create mode 100644 fs/smb/common/smbdirect/smbdirect_debug.c
fs/smb/common/smbdirect/smbdirect_all_c_files.c +1 -0
@@ -19,3 +19,4 @@#include "smbdirect_connection.c" #include "smbdirect_mr.c" #include "smbdirect_rw.c" +#include "smbdirect_debug.c"
fs/smb/common/smbdirect/smbdirect_debug.c new file +88 -0
88 lines not on this page · raw message
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 49/145 smb: smbdirect: introduce smbdirect_connection_wait_for_connected()
In reply to Stefan MetzmacherThis will be used by client and server in order to wait for the connect/negotiation to finish in order to get a usable connection.
--- .../common/smbdirect/smbdirect_connection.c | 68 +++++++++++++++++++ 1 file changed, 68 insertions(+)
fs/smb/common/smbdirect/smbdirect_connection.c +68 -0
@@ -719,6 +719,74 @@ static void smbdirect_connection_negotiate_rdma_resources(struct smbdirect_socke peer_responder_resources); } +__maybe_unused /* this is temporary while this file is included in others */ +static int smbdirect_connection_wait_for_connected (struct smbdirect_socket *sc) +{ +const struct smbdirect_socket_parameters *sp = &sc->parameters; +union { +struct sockaddr sa; +struct sockaddr_storage ss; + } src_addr, dst_addr; +const struct sockaddr *src =NULL ; +const struct sockaddr *dst =NULL ; +char _devname[IB_DEVICE_NAME_MAX ] = {0 , }; +const char *devname =NULL ; +int ret; + +if (sc->rdma.cm_id) { + src_addr.ss = sc->rdma.cm_id->route.addr.src_addr; +if (src_addr.sa.sa_family !=AF_UNSPEC ) + src = &src_addr.sa; + dst_addr.ss = sc->rdma.cm_id->route.addr.dst_addr; +if (dst_addr.sa.sa_family !=AF_UNSPEC ) + dst = &dst_addr.sa; + +if (sc->ib.dev) { +memcpy (_devname, sc->ib.dev->name,IB_DEVICE_NAME_MAX ); + devname = _devname; + } + } + +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_INFO , +"waiting for connection: device: %.*s local: %pISpsfc remote: %pISpsfc\n" , +IB_DEVICE_NAME_MAX , devname, src, dst); + + ret =wait_event_interruptible_timeout (sc->status_wait, + sc->status ==SMBDIRECT_SOCKET_CONNECTED || + sc->first_error, +msecs_to_jiffies (sp->negotiate_timeout_msec)); +if (sc->rdma.cm_id) { +/* +* Maybe src and dev are updated in the meantime. +*/ + src_addr.ss = sc->rdma.cm_id->route.addr.src_addr; +if (src_addr.sa.sa_family !=AF_UNSPEC ) + src = &src_addr.sa; + dst_addr.ss = sc->rdma.cm_id->route.addr.dst_addr; +if (dst_addr.sa.sa_family !=AF_UNSPEC ) + dst = &dst_addr.sa; + +if (sc->ib.dev) { +memcpy (_devname, sc->ib.dev->name,IB_DEVICE_NAME_MAX ); + devname = _devname; + } + } +if (ret ==0 ) + ret = -ETIMEDOUT ; +if (ret <0 ) +smbdirect_socket_schedule_cleanup (sc, ret); +if (sc->first_error) { + ret = sc->first_error; +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"connection failed %1pe device: %.*s local: %pISpsfc remote: %pISpsfc\n" , +SMBDIRECT_DEBUG_ERR_PTR (ret), +IB_DEVICE_NAME_MAX , devname, src, dst); +return ret; + } + +return 0 ; +} +static void smbdirect_connection_idle_timer_work (struct work_struct *work) { struct smbdirect_socket *sc =
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 50/145 smb: smbdirect: introduce smbdirect_connection_is_connected()
In reply to Stefan MetzmacherThis is a simple way to check is the connection is still ok without the need to know internals of struct smbdirect_socket.
--- fs/smb/common/smbdirect/smbdirect_connection.c | 8 ++++++++ 1 file changed, 8 insertions(+)
fs/smb/common/smbdirect/smbdirect_connection.c +8 -0
@@ -719,6 +719,14 @@ static void smbdirect_connection_negotiate_rdma_resources(struct smbdirect_socke peer_responder_resources); } +__maybe_unused /* this is temporary while this file is included in others */ +static bool smbdirect_connection_is_connected (struct smbdirect_socket *sc) +{ +if (unlikely (!sc || sc->first_error || sc->status !=SMBDIRECT_SOCKET_CONNECTED )) +return false; +return true; +} +__maybe_unused /* this is temporary while this file is included in others */ static int smbdirect_connection_wait_for_connected (struct smbdirect_socket *sc) {
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 51/145 smb: smbdirect: introduce smbdirect_socket_shutdown()
In reply to Stefan MetzmacherThis can be used by client and server to trigger a disconnect of the connection, the idea of to be similar to kernel_sock_shutdown(), but for smbdirect there's no point in shutting down only one direct so there's no 'how' argument.
--- fs/smb/common/smbdirect/smbdirect_socket.c | 6 ++++++ 1 file changed, 6 insertions(+)
fs/smb/common/smbdirect/smbdirect_socket.c +6 -0
@@ -431,6 +431,12 @@ static void smbdirect_socket_destroy_sync(struct smbdirect_socket *sc) SMBDIRECT_DEBUG_ERR_PTR(sc->first_error)); } +__maybe_unused /* this is temporary while this file is included in others */ +static void smbdirect_socket_shutdown (struct smbdirect_socket *sc) +{ +smbdirect_socket_schedule_cleanup (sc, -ESHUTDOWN ); +} +__maybe_unused /* this is temporary while this file is included in others */ static int smbdirect_socket_wait_for_credits(struct smbdirect_socket *sc, enum smbdirect_socket_status expected_status,
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 52/145 smb: smbdirect: introduce smbdirect_socket_init_{new,accepting}() and helpers
In reply to Stefan MetzmacherThese will be used in order to initialize struct smbdirect_socket with rdma.cm_id being valid from the start in order to hold a reference to the correct net namespace, this will allow us to implement async connecting and accepting logic in the next steps.
This comes with some related helper functions in
order to initialize the socket without the need
to access internals of struct smbdirect_socket:
smbdirect_socket_set_initial_parameters
smbdirect_socket_get_current_parameters
smbdirect_socket_set_kernel_settings
smbdirect_socket_set_custom_workqueue
--- fs/smb/common/smbdirect/smbdirect_socket.c | 157 ++++++++++++++++++++- 1 file changed, 155 insertions(+), 2 deletions(-)
fs/smb/common/smbdirect/smbdirect_socket.c +155 -2
@@ -24,6 +24,159 @@ static bool smbdirect_frwr_is_supported(const struct ib_device_attr *attrs)static void smbdirect_socket_cleanup_work (struct work_struct *work); +static int smbdirect_socket_rdma_event_handler (struct rdma_cm_id *id, +struct rdma_cm_event *event) +{ +struct smbdirect_socket *sc = id->context; +int ret = -ESTALE ; + +/* +* This should be replaced before any real work +* starts! So it should never be called! +*/ + +if (event->event ==RDMA_CM_EVENT_DEVICE_REMOVAL ) + ret = -ENETDOWN ; +if (IS_ERR (SMBDIRECT_DEBUG_ERR_PTR (event->status))) + ret = event->status; +pr_err ("%s (first_error=%1pe, expected=%s) => event=%s status=%d => ret=%1pe\n" , +smbdirect_socket_status_string (sc->status), +SMBDIRECT_DEBUG_ERR_PTR (sc->first_error), +rdma_event_msg (sc->rdma.expected_event), +rdma_event_msg (event->event), + event->status, +SMBDIRECT_DEBUG_ERR_PTR (ret)); +WARN_ONCE (1 ,"%s should not be called!\n" , __func__); + sc->rdma.cm_id =NULL ; +return -ESTALE ; +} + +__maybe_unused /* this is temporary while this file is included in others */ +static int smbdirect_socket_init_new (struct net *net,struct smbdirect_socket *sc) +{ +struct rdma_cm_id *id; +int ret; + +smbdirect_socket_init (sc); + + id =rdma_create_id (net, + smbdirect_socket_rdma_event_handler, + sc, +RDMA_PS_TCP , +IB_QPT_RC ); +if (IS_ERR (id)) { +pr_err ("%s: rdma_create_id() failed %1pe\n" , __func__, id); +return PTR_ERR (id); + } + + ret =rdma_set_afonly (id,1 ); +if (ret) { +rdma_destroy_id (id); +pr_err ("%s: rdma_set_afonly() failed %1pe\n" , __func__,errname (ret)); +return ret; + } + + sc->rdma.cm_id = id; + +INIT_WORK (&sc->disconnect_work, smbdirect_socket_cleanup_work); + +return 0 ; +} + +__maybe_unused /* this is temporary while this file is included in others */ +static int smbdirect_socket_init_accepting (struct rdma_cm_id *id,struct smbdirect_socket *sc) +{ +smbdirect_socket_init (sc); + + sc->rdma.cm_id = id; + sc->rdma.cm_id->context = sc; + sc->rdma.cm_id->event_handler = smbdirect_socket_rdma_event_handler; + + sc->ib.dev = sc->rdma.cm_id->device; + +INIT_WORK (&sc->disconnect_work, smbdirect_socket_cleanup_work); + +return 0 ; +} + +__maybe_unused /* this is temporary while this file is included in others */ +static int smbdirect_socket_set_initial_parameters (struct smbdirect_socket *sc, +const struct smbdirect_socket_parameters *sp) +{ +/* +* This is only allowed before connect or accept +*/ +WARN_ONCE (sc->status !=SMBDIRECT_SOCKET_CREATED , +"status=%s first_error=%1pe" , +smbdirect_socket_status_string (sc->status), +SMBDIRECT_DEBUG_ERR_PTR (sc->first_error)); +if (sc->status !=SMBDIRECT_SOCKET_CREATED ) +return -EINVAL ; + +/* +* Make a copy of the callers parameters +* from here we only work on the copy +* +* TODO: do we want consistency checking? +*/ + sc->parameters = *sp; + +return 0 ; +} + +__maybe_unused /* this is temporary while this file is included in others */ +static const struct smbdirect_socket_parameters * +smbdirect_socket_get_current_parameters (struct smbdirect_socket *sc) +{ +return &sc->parameters; +} + +__maybe_unused /* this is temporary while this file is included in others */ +static int smbdirect_socket_set_kernel_settings (struct smbdirect_socket *sc, +enum ib_poll_context poll_ctx, +gfp_t gfp_mask) +{ +/* +* This is only allowed before connect or accept +*/ +WARN_ONCE (sc->status !=SMBDIRECT_SOCKET_CREATED , +"status=%s first_error=%1pe" , +smbdirect_socket_status_string (sc->status), +SMBDIRECT_DEBUG_ERR_PTR (sc->first_error)); +if (sc->status !=SMBDIRECT_SOCKET_CREATED ) +return -EINVAL ; + + sc->ib.poll_ctx = poll_ctx; + + sc->send_io.mem.gfp_mask = gfp_mask; + sc->recv_io.mem.gfp_mask = gfp_mask; + sc->rw_io.mem.gfp_mask = gfp_mask; + +return 0 ; +} + +__maybe_unused /* this is temporary while this file is included in others */ +static int smbdirect_socket_set_custom_workqueue (struct smbdirect_socket *sc, +struct workqueue_struct *workqueue) +{ +/* +* This is only allowed before connect or accept +*/ +WARN_ONCE (sc->status !=SMBDIRECT_SOCKET_CREATED , +"status=%s first_error=%1pe" , +smbdirect_socket_status_string (sc->status), +SMBDIRECT_DEBUG_ERR_PTR (sc->first_error)); +if (sc->status !=SMBDIRECT_SOCKET_CREATED ) +return -EINVAL ; + +/* +* Remember the callers workqueue +*/ + sc->workqueue = workqueue; + +return 0 ; +} +__maybe_unused /* this is temporary while this file is included in others */ static void smbdirect_socket_prepare_create(struct smbdirect_socket *sc, const struct smbdirect_socket_parameters *sp,
@@ -35,12 +188,12 @@ static void smbdirect_socket_prepare_create(struct smbdirect_socket *sc,* Make a copy of the callers parameters * from here we only work on the copy */ - sc->parameters = *sp; +smbdirect_socket_set_initial_parameters (sc, sp);/* * Remember the callers workqueue */ - sc->workqueue = workqueue; +smbdirect_socket_set_custom_workqueue (sc, workqueue);INIT_WORK (&sc->disconnect_work, smbdirect_socket_cleanup_work);
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 53/145 smb: smbdirect: introduce smbdirect_connect[_sync]()
In reply to Stefan MetzmacherThis implements a fully async connect logic over all rdma related operations: rdma_resolve_addr, rdma_resolve_route and rdma_connect_locked until we reach RDMA_CM_EVENT_ESTABLISHED, followed by the smbdirect negotiation request/response handling until we reach SMBDIRECT_SOCKET_CONNECTED.
smbdirect_connect_sync() is just a useful wrapper around the async logic calling smbdirect_connect() followed by smbdirect_connection_wait_for_connected(), which only waits for SMBDIRECT_SOCKET_CONNECTED or an error.
--- .../common/smbdirect/smbdirect_all_c_files.c | 1 + fs/smb/common/smbdirect/smbdirect_connect.c | 843 ++++++++++++++++++ 2 files changed, 844 insertions(+) create mode 100644 fs/smb/common/smbdirect/smbdirect_connect.c
fs/smb/common/smbdirect/smbdirect_all_c_files.c +1 -0
@@ -20,3 +20,4 @@#include "smbdirect_mr.c" #include "smbdirect_rw.c" #include "smbdirect_debug.c" +#include "smbdirect_connect.c"
fs/smb/common/smbdirect/smbdirect_connect.c new file +843 -0
843 lines not on this page · raw message
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 54/145 smb: smbdirect: introduce smbdirect_accept_connect_request()
In reply to Stefan MetzmacherThis will be used by the server to handle new connections. All rdma processing from rdma_accept() to RDMA_CM_EVENT_ESTABLISHED as well as the waiting for the smbdirect negotiation request and sending the negotiation response is done async
until we reach SMBDIRECT_SOCKET_CONNECTED.
Sync behaviour will be done by the server callingsmbdirect_conection_wait_for_connected() in order
to each SMBDIRECT_SOCKET_CONNECTED or an error.
--- fs/smb/common/smbdirect/smbdirect_accept.c | 666 ++++++++++++++++++ .../common/smbdirect/smbdirect_all_c_files.c | 1 + 2 files changed, 667 insertions(+) create mode 100644 fs/smb/common/smbdirect/smbdirect_accept.c
fs/smb/common/smbdirect/smbdirect_accept.c new file +666 -0
@@ -0,0 +1,666 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (C) 2017, Microsoft Corporation. +* Copyright (C) 2018, LG Electronics. +* Copyright (c) 2025, Stefan Metzmacher +*/ + +#include "smbdirect_internal.h" + +static int smbdirect_accept_rdma_event_handler (struct rdma_cm_id *id, +struct rdma_cm_event *event); +static int smbdirect_accept_init_params (struct smbdirect_socket *sc); +static void smbdirect_accept_negotiate_recv_done (struct ib_cq *cq,struct ib_wc *wc); +static void smbdirect_accept_negotiate_send_done (struct ib_cq *cq,struct ib_wc *wc); + +__maybe_unused /* this is temporary while this file is included in others */ +static int smbdirect_accept_connect_request (struct smbdirect_socket *sc, +const struct rdma_conn_param *param) +{ +struct smbdirect_socket_parameters *sp = &sc->parameters; +struct smbdirect_recv_io *recv_io; +u8 peer_initiator_depth; +u8 peer_responder_resources; +struct rdma_conn_param conn_param; +__be32 ird_ord_hdr[2 ]; +int ret; + +if (SMBDIRECT_CHECK_STATUS_WARN (sc,SMBDIRECT_SOCKET_CREATED )) +return -EINVAL ; + +/* +* First set what the we as server are able to support +*/ + sp->initiator_depth =min_t (u8, sp->initiator_depth, + sc->ib.dev->attrs.max_qp_rd_atom); + + peer_initiator_depth = param->initiator_depth; + peer_responder_resources = param->responder_resources; +smbdirect_connection_negotiate_rdma_resources (sc, + peer_initiator_depth, + peer_responder_resources, + param); + + ret =smbdirect_accept_init_params (sc); +if (ret) { +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"smbdirect_accept_init_params() failed %1pe\n" , +SMBDIRECT_DEBUG_ERR_PTR (ret)); + goto init_params_failed; + } + + ret =smbdirect_connection_create_qp (sc); +if (ret) { +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"smbdirect_connection_create_qp() failed %1pe\n" , +SMBDIRECT_DEBUG_ERR_PTR (ret)); + goto create_qp_failed; + } + + ret =smbdirect_connection_create_mem_pools (sc); +if (ret) { +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"smbdirect_connection_create_mem_pools() failed %1pe\n" , +SMBDIRECT_DEBUG_ERR_PTR (ret)); + goto create_mem_failed; + } + + recv_io =smbdirect_connection_get_recv_io (sc); +if (WARN_ON_ONCE (!recv_io)) { + ret = -EINVAL ; +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"smbdirect_connection_get_recv_io() failed %1pe\n" , +SMBDIRECT_DEBUG_ERR_PTR (ret)); + goto get_recv_io_failed; + } + recv_io->cqe.done = smbdirect_accept_negotiate_recv_done; + +/* +* Now post the recv_io buffer in order to get +* the negotiate request +*/ + sc->recv_io.expected =SMBDIRECT_EXPECT_NEGOTIATE_REQ ; + ret =smbdirect_connection_post_recv_io (recv_io); +if (ret) { +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"smbdirect_connection_post_recv_io() failed %1pe\n" , +SMBDIRECT_DEBUG_ERR_PTR (ret)); + goto post_recv_io_failed; + } +/* +* From here recv_io is known to the RDMA QP and needs ib_drain_qp and +* smbdirect_accept_negotiate_recv_done to cleanup... +*/ + recv_io =NULL ; + +/* already checked with SMBDIRECT_CHECK_STATUS_WARN above */ +WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_CREATED ); + sc->status =SMBDIRECT_SOCKET_RDMA_CONNECT_NEEDED ; + +/* +* We already negotiated sp->initiator_depth +* and sp->responder_resources above. +*/ +memset (&conn_param,0 ,sizeof (conn_param)); + conn_param.initiator_depth = sp->initiator_depth; + conn_param.responder_resources = sp->responder_resources; + +if (sc->rdma.legacy_iwarp) { + ird_ord_hdr[0 ] =cpu_to_be32 (conn_param.responder_resources); + ird_ord_hdr[1 ] =cpu_to_be32 (conn_param.initiator_depth); + conn_param.private_data = ird_ord_hdr; + conn_param.private_data_len =sizeof (ird_ord_hdr); + }else { + conn_param.private_data =NULL ; + conn_param.private_data_len =0 ; + } + conn_param.retry_count =SMBDIRECT_RDMA_CM_RETRY ; + conn_param.rnr_retry_count =SMBDIRECT_RDMA_CM_RNR_RETRY ; + conn_param.flow_control =0 ; + +/* explicitly set above */ +WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RDMA_CONNECT_NEEDED ); + sc->status =SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING ; + sc->rdma.expected_event =RDMA_CM_EVENT_ESTABLISHED ; + sc->rdma.cm_id->event_handler = smbdirect_accept_rdma_event_handler; + ret =rdma_accept (sc->rdma.cm_id, &conn_param); +if (ret) { +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"rdma_accept() failed %1pe\n" , +SMBDIRECT_DEBUG_ERR_PTR (ret)); + goto rdma_accept_failed; + } + +/* +* start with the negotiate timeout and SMBDIRECT_KEEPALIVE_PENDING +* so that the timer will cause a disconnect. +*/ +INIT_DELAYED_WORK (&sc->idle.timer_work, smbdirect_connection_idle_timer_work); + sc->idle.keepalive =SMBDIRECT_KEEPALIVE_PENDING ; +mod_delayed_work (sc->workqueue, &sc->idle.timer_work, +msecs_to_jiffies (sp->negotiate_timeout_msec)); + +return 0 ; + +rdma_accept_failed: +/* +* smbdirect_connection_destroy_qp() calls ib_drain_qp(), +* so that smbdirect_accept_negotiate_recv_done() will +* call smbdirect_connection_put_recv_io() +*/ +post_recv_io_failed: +if (recv_io) +smbdirect_connection_put_recv_io (recv_io); +get_recv_io_failed: +smbdirect_connection_destroy_mem_pools (sc); +create_mem_failed: +smbdirect_connection_destroy_qp (sc); +create_qp_failed: +init_params_failed: +return ret; +} + +static int smbdirect_accept_init_params (struct smbdirect_socket *sc) +{ +const struct smbdirect_socket_parameters *sp = &sc->parameters; +int max_send_sges; +unsigned int maxpages; + +/* need 3 more sge. because a SMB_DIRECT header, SMB2 header, +* SMB2 response could be mapped. +*/ + max_send_sges =DIV_ROUND_UP (sp->max_send_size,PAGE_SIZE ) +3 ; +if (max_send_sges >SMBDIRECT_SEND_IO_MAX_SGE ) { +pr_err ("max_send_size %d is too large\n" , sp->max_send_size); +return -EINVAL ; + } + +/* +* Initialize the local credits to post +* IB_WR_SEND[_WITH_INV]. +*/ +atomic_set (&sc->send_io.lcredits.count, sp->send_credit_target); + + maxpages =DIV_ROUND_UP (sp->max_read_write_size,PAGE_SIZE ); + sc->rw_io.credits.max =rdma_rw_mr_factor (sc->ib.dev, + sc->rdma.cm_id->port_num, + maxpages); + sc->rw_io.credits.num_pages =DIV_ROUND_UP (maxpages, sc->rw_io.credits.max); +/* add one extra in order to handle unaligned pages */ + sc->rw_io.credits.max +=1 ; + + sc->recv_io.credits.target =1 ; + +atomic_set (&sc->rw_io.credits.count, sc->rw_io.credits.max); + +return 0 ; +} + +static void smbdirect_accept_negotiate_recv_work (struct work_struct *work); + +static void smbdirect_accept_negotiate_recv_done (struct ib_cq *cq,struct ib_wc *wc) +{ +struct smbdirect_recv_io *recv_io = + container_of(wc->wr_cqe, struct smbdirect_recv_io, cqe); +struct smbdirect_socket *sc = recv_io->socket; +const struct smbdirect_socket_parameters *sp = &sc->parameters; + +if (unlikely (wc->status !=IB_WC_SUCCESS ||WARN_ON_ONCE (wc->opcode !=IB_WC_RECV ))) { +if (wc->status !=IB_WC_WR_FLUSH_ERR ) +smbdirect_log_rdma_recv (sc,SMBDIRECT_LOG_ERR , +"wc->status=%s (%d) wc->opcode=%d\n" , +ib_wc_status_msg (wc->status), wc->status, wc->opcode); + goto error; + } + +smbdirect_log_rdma_recv (sc,SMBDIRECT_LOG_INFO , +"smbdirect_recv_io completed. status='%s (%d)', opcode=%d\n" , +ib_wc_status_msg (wc->status), wc->status, wc->opcode); + +/* +* Some drivers (at least mlx5_ib) might post a +* recv completion before RDMA_CM_EVENT_ESTABLISHED, +* we need to adjust our expectation in that case. +*/ +if (!sc->first_error && sc->status ==SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING ) + sc->status =SMBDIRECT_SOCKET_NEGOTIATE_NEEDED ; +if (SMBDIRECT_CHECK_STATUS_WARN (sc,SMBDIRECT_SOCKET_NEGOTIATE_NEEDED )) + goto error; + sc->status =SMBDIRECT_SOCKET_NEGOTIATE_RUNNING ; + +/* +* Reset timer to the keepalive interval in +* order to trigger our next keepalive message. +*/ + sc->idle.keepalive =SMBDIRECT_KEEPALIVE_NONE ; +mod_delayed_work (sc->workqueue, &sc->idle.timer_work, +msecs_to_jiffies (sp->keepalive_interval_msec)); + +ib_dma_sync_single_for_cpu (sc->ib.dev, + recv_io->sge.addr, + recv_io->sge.length, +DMA_FROM_DEVICE ); + +if (wc->byte_len <sizeof (struct smbdirect_negotiate_req )) { +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"wc->byte_len=%u < %zu\n" , + wc->byte_len,sizeof (struct smbdirect_negotiate_req )); + goto error; + } + +/* +* We continue via the workqueue as we may have +* complex work that might sleep. +* +* The work should already/still be disabled, +* but smbdirect_connection_put_recv_io() disables +* it again. +* +* Note that smbdirect_connection_put_recv_io() +* only moved recv_io into the free list, but +* we didn't call smbdirect_connection_recv_io_refill() +* yet, so it won't be reused, but the cleanup code +* on disconnect is able to find it, disables +* recv_io->complex_work again. +*/ +smbdirect_connection_put_recv_io (recv_io); +INIT_WORK (&recv_io->complex_work, smbdirect_accept_negotiate_recv_work); +queue_work (sc->workqueue, &recv_io->complex_work); +return ; + +error: +/* +* recv_io.posted.refill_work is still disabled, +* so smbdirect_connection_put_recv_io() won't +* start it. +*/ +smbdirect_connection_put_recv_io (recv_io); +smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); +} + +static void smbdirect_accept_negotiate_recv_work (struct work_struct *work) +{ +struct smbdirect_recv_io *recv_io = + container_of(work, struct smbdirect_recv_io, complex_work); +struct smbdirect_socket *sc = recv_io->socket; +struct smbdirect_socket_parameters *sp = &sc->parameters; +struct smbdirect_negotiate_req *nreq; +u16 min_version; +u16 max_version; +u16 credits_requested; +u32 preferred_send_size; +u32 max_receive_size; +u32 max_fragmented_size; +struct smbdirect_send_io *send_io =NULL ; +struct smbdirect_negotiate_resp *nrep; +u32 ntstatus; +int posted; +int ret; + +/* +* make sure we won't start again... +*/ +disable_work (work); + +/* +* Note recv_io is already part of the free list, +* as smbdirect_accept_negotiate_recv_done() called +* smbdirect_connection_put_recv_io(), but +* it won't be reused before we call +* smbdirect_connection_recv_io_refill() below. +*/ + +if (SMBDIRECT_CHECK_STATUS_DISCONNECT (sc,SMBDIRECT_SOCKET_NEGOTIATE_RUNNING )) +return ; + + nreq = (struct smbdirect_negotiate_req *)recv_io->packet; + min_version =le16_to_cpu (nreq->min_version); + max_version =le16_to_cpu (nreq->max_version); + credits_requested =le16_to_cpu (nreq->credits_requested); + preferred_send_size =le32_to_cpu (nreq->preferred_send_size); + max_receive_size =le32_to_cpu (nreq->max_receive_size); + max_fragmented_size =le32_to_cpu (nreq->max_fragmented_size); + +smbdirect_log_negotiate (sc,SMBDIRECT_LOG_INFO , +"ReqIn: %s%x, %s%x, %s%u, %s%u, %s%u, %s%u\n" , +"MinVersion=0x" , +le16_to_cpu (nreq->min_version), +"MaxVersion=0x" , +le16_to_cpu (nreq->max_version), +"CreditsRequested=" , +le16_to_cpu (nreq->credits_requested), +"PreferredSendSize=" , +le32_to_cpu (nreq->preferred_send_size), +"MaxRecvSize=" , +le32_to_cpu (nreq->max_receive_size), +"MaxFragmentedSize=" , +le32_to_cpu (nreq->max_fragmented_size)); + +if (!(min_version <=SMBDIRECT_V1 && max_version >=SMBDIRECT_V1 )) { +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"invalid: min_version=0x%x max_version=0x%x\n" , + min_version, max_version); + ntstatus =0xC00000bb ;/* NT_STATUS_NOT_SUPPORTED */ + goto not_supported; + } + +if (credits_requested ==0 ) { +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"invalid: credits_requested == 0\n" ); +smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); +return ; + } + +if (max_receive_size <SMBDIRECT_MIN_RECEIVE_SIZE ) { +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"invalid: max_receive_size=%u < %u\n" , + max_receive_size, +SMBDIRECT_MIN_RECEIVE_SIZE ); +smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); +return ; + } + +if (max_fragmented_size <SMBDIRECT_MIN_FRAGMENTED_SIZE ) { +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"invalid: max_fragmented_size=%u < %u\n" , + max_fragmented_size, +SMBDIRECT_MIN_FRAGMENTED_SIZE ); +smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); +return ; + } + +/* +* At least the value of SMBDIRECT_MIN_RECEIVE_SIZE is used. +*/ + sp->max_recv_size =min_t (u32, sp->max_recv_size, preferred_send_size); + sp->max_recv_size =max_t (u32, sp->max_recv_size,SMBDIRECT_MIN_RECEIVE_SIZE ); + +/* +* We take the value from the peer, which is checked to be higher than 0, +* but we limit it to the max value we support in order to have +* the main logic simpler. +*/ + sc->recv_io.credits.target = credits_requested; + sc->recv_io.credits.target =min_t (u16, sc->recv_io.credits.target, + sp->recv_credit_max); + +/* +* Note nreq->max_receive_size was already checked against +* SMBDIRECT_MIN_RECEIVE_SIZE above. +*/ + sp->max_send_size =min_t (u32, sp->max_send_size, max_receive_size); + +/* +* Note nreq->max_fragmented_size was already checked against +* SMBDIRECT_MIN_FRAGMENTED_SIZE above. +*/ + sp->max_fragmented_send_size = max_fragmented_size; + +/* +* Prepare for receiving data_transfer messages +*/ + sc->recv_io.reassembly.full_packet_received = true; + sc->recv_io.expected =SMBDIRECT_EXPECT_DATA_TRANSFER ; +list_for_each_entry (recv_io, &sc->recv_io.free.list, list) + recv_io->cqe.done = smbdirect_connection_recv_io_done; + recv_io =NULL ; + +/* +* We should at least post 1 smbdirect_recv_io! +*/ + posted =smbdirect_connection_recv_io_refill (sc); +if (posted <1 ) { +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"smbdirect_connection_recv_io_refill() failed %1pe\n" , +SMBDIRECT_DEBUG_ERR_PTR (posted)); +smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); +return ; + } + +/* +* The response will grant credits for all posted +* smbdirect_recv_io messages. +*/ +atomic_set (&sc->recv_io.credits.count, posted); + + ntstatus =0 ;/* NT_STATUS_OK */ + +not_supported: + send_io =smbdirect_connection_alloc_send_io (sc); +if (IS_ERR (send_io)) { + ret =PTR_ERR (send_io); +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"smbdirect_connection_alloc_send_io() failed %1pe\n" , +SMBDIRECT_DEBUG_ERR_PTR (ret)); +smbdirect_socket_schedule_cleanup (sc, ret); +return ; + } + send_io->cqe.done = smbdirect_accept_negotiate_send_done; + + nrep = (struct smbdirect_negotiate_resp *)send_io->packet; + nrep->min_version =cpu_to_le16 (SMBDIRECT_V1 ); + nrep->max_version =cpu_to_le16 (SMBDIRECT_V1 ); +if (ntstatus ==0 ) { + nrep->negotiated_version =cpu_to_le16 (SMBDIRECT_V1 ); + nrep->reserved =0 ; + nrep->credits_requested =cpu_to_le16 (sp->send_credit_target); + nrep->credits_granted =cpu_to_le16 (posted); + nrep->status =cpu_to_le32 (ntstatus); + nrep->max_readwrite_size =cpu_to_le32 (sp->max_read_write_size); + nrep->preferred_send_size =cpu_to_le32 (sp->max_send_size); + nrep->max_receive_size =cpu_to_le32 (sp->max_recv_size); + nrep->max_fragmented_size =cpu_to_le32 (sp->max_fragmented_recv_size); + }else { + nrep->negotiated_version =0 ; + nrep->reserved =0 ; + nrep->credits_requested =0 ; + nrep->credits_granted =0 ; + nrep->status =cpu_to_le32 (ntstatus); + nrep->max_readwrite_size =0 ; + nrep->preferred_send_size =0 ; + nrep->max_receive_size =0 ; + nrep->max_fragmented_size =0 ; + } + +smbdirect_log_negotiate (sc,SMBDIRECT_LOG_INFO , +"RepOut: %s%x, %s%x, %s%x, %s%u, %s%u, %s%x, %s%u, %s%u, %s%u, %s%u\n" , +"MinVersion=0x" , +le16_to_cpu (nrep->min_version), +"MaxVersion=0x" , +le16_to_cpu (nrep->max_version), +"NegotiatedVersion=0x" , +le16_to_cpu (nrep->negotiated_version), +"CreditsRequested=" , +le16_to_cpu (nrep->credits_requested), +"CreditsGranted=" , +le16_to_cpu (nrep->credits_granted), +"Status=0x" , +le32_to_cpu (nrep->status), +"MaxReadWriteSize=" , +le32_to_cpu (nrep->max_readwrite_size), +"PreferredSendSize=" , +le32_to_cpu (nrep->preferred_send_size), +"MaxRecvSize=" , +le32_to_cpu (nrep->max_receive_size), +"MaxFragmentedSize=" , +le32_to_cpu (nrep->max_fragmented_size)); + + send_io->sge[0 ].addr =ib_dma_map_single (sc->ib.dev, + nrep, +sizeof (*nrep), +DMA_TO_DEVICE ); + ret =ib_dma_mapping_error (sc->ib.dev, send_io->sge[0 ].addr); +if (ret) { +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"ib_dma_mapping_error() failed %1pe\n" , +SMBDIRECT_DEBUG_ERR_PTR (ret)); +smbdirect_connection_free_send_io (send_io); +smbdirect_socket_schedule_cleanup (sc, ret); +return ; + } + + send_io->sge[0 ].length =sizeof (*nrep); + send_io->sge[0 ].lkey = sc->ib.pd->local_dma_lkey; + send_io->num_sge =1 ; + +ib_dma_sync_single_for_device (sc->ib.dev, + send_io->sge[0 ].addr, + send_io->sge[0 ].length, +DMA_TO_DEVICE ); + + send_io->wr.next =NULL ; + send_io->wr.wr_cqe = &send_io->cqe; + send_io->wr.sg_list = send_io->sge; + send_io->wr.num_sge = send_io->num_sge; + send_io->wr.opcode =IB_WR_SEND ; + send_io->wr.send_flags =IB_SEND_SIGNALED ; + + ret =smbdirect_connection_post_send_wr (sc, &send_io->wr); +if (ret) { +/* if we reach here, post send failed */ +smbdirect_log_rdma_send (sc,SMBDIRECT_LOG_ERR , +"smbdirect_connection_post_send_wr() failed %1pe\n" , +SMBDIRECT_DEBUG_ERR_PTR (ret)); +/* +* Note smbdirect_connection_free_send_io() +* does ib_dma_unmap_page() +*/ +smbdirect_connection_free_send_io (send_io); +smbdirect_socket_schedule_cleanup (sc, ret); +return ; + } + +/* +* smbdirect_accept_negotiate_send_done +* will do all remaining work... +*/ +} + +static void smbdirect_accept_negotiate_send_done (struct ib_cq *cq,struct ib_wc *wc) +{ +struct smbdirect_send_io *send_io = + container_of(wc->wr_cqe, struct smbdirect_send_io, cqe); +struct smbdirect_socket *sc = send_io->socket; +struct smbdirect_negotiate_resp *nrep; +u32 ntstatus; + +smbdirect_log_rdma_send (sc,SMBDIRECT_LOG_INFO , +"smbdirect_send_io completed. status='%s (%d)', opcode=%d\n" , +ib_wc_status_msg (wc->status), wc->status, wc->opcode); + + nrep = (struct smbdirect_negotiate_resp *)send_io->packet; + ntstatus =le32_to_cpu (nrep->status); + +/* Note this frees wc->wr_cqe, but not wc */ +smbdirect_connection_free_send_io (send_io); +atomic_dec (&sc->send_io.pending.count); + +if (unlikely (wc->status !=IB_WC_SUCCESS ||WARN_ON_ONCE (wc->opcode !=IB_WC_SEND ))) { +if (wc->status !=IB_WC_WR_FLUSH_ERR ) +smbdirect_log_rdma_send (sc,SMBDIRECT_LOG_ERR , +"wc->status=%s (%d) wc->opcode=%d\n" , +ib_wc_status_msg (wc->status), wc->status, wc->opcode); +smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); +return ; + } + +/* +* If we send a smbdirect_negotiate_resp without NT_STATUS_OK (0) +* we need to disconnect now. +* +* Otherwise smbdirect_connection_negotiation_done() +* will setup all required things and wake up +* the waiter. +*/ +if (ntstatus) +smbdirect_socket_schedule_cleanup (sc, -EOPNOTSUPP ); +else +smbdirect_connection_negotiation_done (sc); +} + +static int smbdirect_accept_rdma_event_handler (struct rdma_cm_id *id, +struct rdma_cm_event *event) +{ +struct smbdirect_socket *sc = id->context; + +/* +* cma_cm_event_handler() has +* lockdep_assert_held(&id_priv->handler_mutex); +* +* Mutexes are not allowed in interrupts, +* and we rely on not being in an interrupt here, +* as we might sleep. +* +* We didn't timeout so we cancel our idle timer, +* it will be scheduled again if needed. +*/ +WARN_ON_ONCE (in_interrupt ()); + +if (event->status || event->event != sc->rdma.expected_event) { +int ret = -ECONNABORTED ; + +if (event->event ==RDMA_CM_EVENT_REJECTED ) + ret = -ECONNREFUSED ; +if (event->event ==RDMA_CM_EVENT_DEVICE_REMOVAL ) + ret = -ENETDOWN ; +if (IS_ERR (SMBDIRECT_DEBUG_ERR_PTR (event->status))) + ret = event->status; + +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_ERR , +"%s (first_error=%1pe, expected=%s) => event=%s status=%d => ret=%1pe\n" , +smbdirect_socket_status_string (sc->status), +SMBDIRECT_DEBUG_ERR_PTR (sc->first_error), +rdma_event_msg (sc->rdma.expected_event), +rdma_event_msg (event->event), + event->status, +SMBDIRECT_DEBUG_ERR_PTR (ret)); + +smbdirect_socket_schedule_cleanup (sc, ret); +return 0 ; + } + +smbdirect_log_rdma_event (sc,SMBDIRECT_LOG_INFO , +"%s (first_error=%1pe) event=%s\n" , +smbdirect_socket_status_string (sc->status), +SMBDIRECT_DEBUG_ERR_PTR (sc->first_error), +rdma_event_msg (event->event)); + +if (sc->first_error) +return 0 ; + +switch (event->event) { +case RDMA_CM_EVENT_ESTABLISHED : +smbdirect_connection_rdma_established (sc); + +/* +* Some drivers (at least mlx5_ib) might post a +* recv completion before RDMA_CM_EVENT_ESTABLISHED, +* we need to adjust our expectation in that case. +* +* As we already started the negotiation, we just +* ignore RDMA_CM_EVENT_ESTABLISHED here. +*/ +if (sc->status >SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING ) +return 0 ; + +if (SMBDIRECT_CHECK_STATUS_DISCONNECT (sc,SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING )) +return 0 ; + sc->status =SMBDIRECT_SOCKET_NEGOTIATE_NEEDED ; + +/* +* wait for smbdirect_accept_negotiate_recv_done() +* to get the negotiate request. +*/ +return 0 ; + +default : +break ; + } + +/* +* This is an internal error +*/ +WARN_ON_ONCE (sc->rdma.expected_event !=RDMA_CM_EVENT_ESTABLISHED ); +smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); +return 0 ; +}
fs/smb/common/smbdirect/smbdirect_all_c_files.c +1 -0
@@ -21,3 +21,4 @@#include "smbdirect_rw.c" #include "smbdirect_debug.c" #include "smbdirect_connect.c" +#include "smbdirect_accept.c"
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 55/145 smb: smbdirect: introduce smbdirect_socket_create_{kern,accepting}() and smbdirect_socket_release()
In reply to Stefan MetzmacherThis provides functions which also allocate and free struct smbdirect_socket.
This allows callers to use the same flow as with
sock_create_kern()/sock_release().
The end goal would be to use sock_create_kern()/sock_release(), but the first step will be to use smbdirect specific functions without any struct socket nor struct sock.
--- fs/smb/common/smbdirect/smbdirect_socket.c | 95 +++++++++++++++++++++- fs/smb/common/smbdirect/smbdirect_socket.h | 33 ++++++++ 2 files changed, 127 insertions(+), 1 deletion(-)
fs/smb/common/smbdirect/smbdirect_socket.c +94 -1
@@ -51,7 +51,6 @@ static int smbdirect_socket_rdma_event_handler(struct rdma_cm_id *id,return -ESTALE ; } -__maybe_unused /* this is temporary while this file is included in others */ static int smbdirect_socket_init_new (struct net *net,struct smbdirect_socket *sc) {struct rdma_cm_id *id;
@@ -84,6 +83,31 @@ static int smbdirect_socket_init_new(struct net *net, struct smbdirect_socket *s }__maybe_unused /* this is temporary while this file is included in others */ +static int smbdirect_socket_create_kern (struct net *net,struct smbdirect_socket **_sc) +{ +struct smbdirect_socket *sc; +int ret; + + ret = -ENOMEM ; + sc =kzalloc (sizeof (*sc),GFP_KERNEL ); +if (!sc) + goto alloc_failed; + + ret =smbdirect_socket_init_new (net, sc); +if (ret) + goto init_failed; + +kref_init (&sc->refs.destroy); + + *_sc = sc; +return 0 ; + +init_failed: +kfree (sc); +alloc_failed: +return ret; +} +static int smbdirect_socket_init_accepting (struct rdma_cm_id *id,struct smbdirect_socket *sc) {smbdirect_socket_init (sc);
@@ -99,6 +123,32 @@ static int smbdirect_socket_init_accepting(struct rdma_cm_id *id, struct smbdirereturn 0 ; } +__maybe_unused /* this is temporary while this file is included in others */ +static int smbdirect_socket_create_accepting (struct rdma_cm_id *id,struct smbdirect_socket **_sc) +{ +struct smbdirect_socket *sc; +int ret; + + ret = -ENOMEM ; + sc =kzalloc (sizeof (*sc),GFP_KERNEL ); +if (!sc) + goto alloc_failed; + + ret =smbdirect_socket_init_accepting (id, sc); +if (ret) + goto init_failed; + +kref_init (&sc->refs.destroy); + + *_sc = sc; +return 0 ; + +init_failed: +kfree (sc); +alloc_failed: +return ret; +} +__maybe_unused /* this is temporary while this file is included in others */ static int smbdirect_socket_set_initial_parameters (struct smbdirect_socket *sc,const struct smbdirect_socket_parameters *sp)
@@ -590,6 +640,49 @@ static void smbdirect_socket_shutdown(struct smbdirect_socket *sc)smbdirect_socket_schedule_cleanup (sc, -ESHUTDOWN ); } +static void smbdirect_socket_release_disconnect (struct kref *kref) +{ +struct smbdirect_socket *sc = + container_of(kref, struct smbdirect_socket, refs.disconnect); + +/* +* For now do a sync disconnect/destroy +*/ +smbdirect_socket_destroy_sync (sc); +} + +static void smbdirect_socket_release_destroy (struct kref *kref) +{ +struct smbdirect_socket *sc = + container_of(kref, struct smbdirect_socket, refs.destroy); + +/* +* Do a sync disconnect/destroy... +* hopefully a no-op, as it should be already +* in DESTROYED state, before we free the memory. +*/ +smbdirect_socket_destroy_sync (sc); +kfree (sc); +} + +__maybe_unused /* this is temporary while this file is included in others */ +static void smbdirect_socket_release (struct smbdirect_socket *sc) +{ +/* +* We expect only 1 disconnect reference +* and if it is already 0, it's a use after free! +*/ +WARN_ON_ONCE (kref_read (&sc->refs.disconnect) !=1 ); +WARN_ON (!kref_put (&sc->refs.disconnect, smbdirect_socket_release_disconnect)); + +/* +* This may not trigger smbdirect_socket_release_destroy(), +* if struct smbdirect_socket is embedded in another structure +* indicated by REFCOUNT_MAX. +*/ +kref_put (&sc->refs.destroy, smbdirect_socket_release_destroy); +} +__maybe_unused /* this is temporary while this file is included in others */ static int smbdirect_socket_wait_for_credits(struct smbdirect_socket *sc, enum smbdirect_socket_status expected_status,
fs/smb/common/smbdirect/smbdirect_socket.h +33 -0
@@ -108,6 +108,36 @@ struct smbdirect_socket {struct work_struct disconnect_work; +/* +* The reference counts. +*/ +struct { +/* +* This holds the references by the +* frontend, typically the smb layer. +* +* It is typically 1 and a disconnect +* will happen if it reaches 0. +*/ +struct kref disconnect; + +/* +* This holds the reference by the +* backend, the code that manages +* the lifetime of the whole +* struct smbdirect_socket, +* if this reaches 0 it can will +* be freed. +* +* Can be REFCOUNT_MAX is part +* of another structure. +* +* This is equal or higher than +* the disconnect refcount. +*/ +struct kref destroy; + } refs; +/* RDMA related */ struct {struct rdma_cm_id *cm_id;
@@ -493,6 +523,9 @@ static __always_inline void smbdirect_socket_init(struct smbdirect_socket *sc)INIT_WORK (&sc->disconnect_work, __smbdirect_socket_disabled_work);disable_work_sync (&sc->disconnect_work); +kref_init (&sc->refs.disconnect); + sc->refs.destroy = (struct kref )KREF_INIT (REFCOUNT_MAX ); + sc->rdma.expected_event =RDMA_CM_EVENT_INTERNAL ; sc->ib.poll_ctx =IB_POLL_UNBOUND_WORKQUEUE ;
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 56/145 smb: smbdirect: let smbdirect_socket.h include all headers for used structures
In reply to Stefan MetzmacherCurrently they are implicitly included via client and server code, but this is needed when we move to an smbdirect.ko.
--- fs/smb/common/smbdirect/smbdirect_socket.h | 7 +++++++ 1 file changed, 7 insertions(+)
fs/smb/common/smbdirect/smbdirect_socket.h +7 -0
@@ -6,6 +6,13 @@#ifndef __FS_SMB_COMMON_SMBDIRECT_SMBDIRECT_SOCKET_H__#define __FS_SMB_COMMON_SMBDIRECT_SMBDIRECT_SOCKET_H__ +#include <linux/wait.h> +#include <linux/workqueue.h> +#include <linux/kref.h> +#include <linux/mempool.h> +#include <linux/spinlock.h> +#include <linux/mutex.h> +#include <linux/completion.h> #include <rdma/rw.h> enum smbdirect_socket_status {
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 57/145 smb: smbdirect: let smbdirect_internal.h define pr_fmt without SMBDIRECT_USE_INLINE_C_FILES
In reply to Stefan MetzmacherWhen we move to smbdirect.ko we want log message prefixed with the module name.
Note callers are still using smbdirect_socket_set_logging() in order to redirect the per connection logging to their own log functions.
--- fs/smb/common/smbdirect/smbdirect_internal.h | 4 ++++ 1 file changed, 4 insertions(+)
fs/smb/common/smbdirect/smbdirect_internal.h +4 -0
@@ -6,6 +6,10 @@#ifndef __FS_SMB_COMMON_SMBDIRECT_INTERNAL_H__#define __FS_SMB_COMMON_SMBDIRECT_INTERNAL_H__ +#ifndef SMBDIRECT_USE_INLINE_C_FILES +#define pr_fmt (fmt) KBUILD_MODNAME ": " fmt +#endif /* ! SMBDIRECT_USE_INLINE_C_FILES */ +#include <linux/errname.h> #include "smbdirect.h" #include "smbdirect_pdu.h"
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 58/145 smb: smbdirect: introduce smbdirect_public.h with prototypes
In reply to Stefan Metzmachersmbdirect_public.h contains functions which will be still be eported when we move to an smbdirect.ko.
For now this uses the SMBDIRECT_USE_INLINE_C_FILES code path and marks all function as '__maybe_unused static', but this will make further changes easier.
Note this generates the following things from checkpatch.pl, so I passed --ignore=FILE_PATH_CHANGES,EXPORT_SYMBOL,COMPLEX_MACRO
ERROR: Macros with complex values should be enclosed in parentheses #514: FILE: fs/smb/common/smbdirect/smbdirect_public.h:18:
+#define __SMBDIRECT_PUBLIC__ __maybe_unused static
WARNING: EXPORT_SYMBOL(foo); should immediately follow its function/variable #515: FILE: fs/smb/common/smbdirect/smbdirect_public.h:19:
+#define __SMBDIRECT_EXPORT_SYMBOL__(__sym)
WARNING: EXPORT_SYMBOL(foo); should immediately follow its function/variable #518: FILE: fs/smb/common/smbdirect/smbdirect_public.h:22:
+#define __SMBDIRECT_EXPORT_SYMBOL__(__sym) EXPORT_SYMBOL_FOR_MODULES(__sym, "cifs,ksmbd")
This is exactly what we want here, so we should ignore the checkpatch.pl problems.
--- fs/smb/common/smbdirect/smbdirect_accept.c | 7 +- fs/smb/common/smbdirect/smbdirect_connect.c | 13 +- .../common/smbdirect/smbdirect_connection.c | 51 +++--- fs/smb/common/smbdirect/smbdirect_debug.c | 9 +- fs/smb/common/smbdirect/smbdirect_internal.h | 1 + fs/smb/common/smbdirect/smbdirect_mr.c | 17 +- fs/smb/common/smbdirect/smbdirect_public.h | 154 ++++++++++++++++++ fs/smb/common/smbdirect/smbdirect_rw.c | 13 +- fs/smb/common/smbdirect/smbdirect_socket.c | 82 ++++++---- fs/smb/common/smbdirect/smbdirect_socket.h | 14 -- 10 files changed, 263 insertions(+), 98 deletions(-) create mode 100644 fs/smb/common/smbdirect/smbdirect_public.h
fs/smb/common/smbdirect/smbdirect_accept.c +4 -3
@@ -13,9 +13,9 @@ static int smbdirect_accept_init_params(struct smbdirect_socket *sc);static void smbdirect_accept_negotiate_recv_done (struct ib_cq *cq,struct ib_wc *wc);static void smbdirect_accept_negotiate_send_done (struct ib_cq *cq,struct ib_wc *wc); -__maybe_unused /* this is temporary while this file is included in others */ -static int smbdirect_accept_connect_request (struct smbdirect_socket *sc, -const struct rdma_conn_param *param) +__SMBDIRECT_PUBLIC__ +int smbdirect_accept_connect_request(struct smbdirect_socket *sc, +const struct rdma_conn_param *param) {struct smbdirect_socket_parameters *sp = &sc->parameters;struct smbdirect_recv_io *recv_io;
@@ -159,6 +159,7 @@ static int smbdirect_accept_connect_request(struct smbdirect_socket *sc, init_params_failed:return ret; } +__SMBDIRECT_EXPORT_SYMBOL__ (smbdirect_accept_connect_request);static int smbdirect_accept_init_params (struct smbdirect_socket *sc) {
fs/smb/common/smbdirect/smbdirect_connect.c +7 -6
@@ -15,9 +15,8 @@ static int smbdirect_connect_negotiate_start(struct smbdirect_socket *sc);static void smbdirect_connect_negotiate_send_done (struct ib_cq *cq,struct ib_wc *wc);static void smbdirect_connect_negotiate_recv_done (struct ib_cq *cq,struct ib_wc *wc); -__maybe_unused /* this is temporary while this file is included in others */ -static int smbdirect_connect (struct smbdirect_socket *sc, -const struct sockaddr *dst) +__SMBDIRECT_PUBLIC__ +int smbdirect_connect(struct smbdirect_socket *sc, const struct sockaddr *dst) {const struct sockaddr *src =NULL ;union {
@@ -61,6 +60,7 @@ static int smbdirect_connect(struct smbdirect_socket *sc,*/ return 0 ; } +__SMBDIRECT_EXPORT_SYMBOL__ (smbdirect_connect);static int smbdirect_connect_setup_connection (struct smbdirect_socket *sc) {
@@ -817,9 +817,9 @@ static void smbdirect_connect_negotiate_recv_work(struct work_struct *work)smbdirect_connection_negotiation_done (sc); } -__maybe_unused /* this is temporary while this file is included in others */ -static int smbdirect_connect_sync (struct smbdirect_socket *sc, -const struct sockaddr *dst) +__SMBDIRECT_PUBLIC__ +int smbdirect_connect_sync(struct smbdirect_socket *sc, +const struct sockaddr *dst) {int ret;
@@ -841,3 +841,4 @@ static int smbdirect_connect_sync(struct smbdirect_socket *sc,return 0 ; } +__SMBDIRECT_EXPORT_SYMBOL__ (smbdirect_connect_sync);
fs/smb/common/smbdirect/smbdirect_connection.c +29 -22
106 lines not on this page · raw message
fs/smb/common/smbdirect/smbdirect_debug.c +5 -4
@@ -7,10 +7,10 @@#include "smbdirect_internal.h" #include <linux/seq_file.h> -__maybe_unused /* this is temporary while this file is included in others */ -static void smbdirect_connection_legacy_debug_proc_show (struct smbdirect_socket *sc, -unsigned int rdma_readwrite_threshold, -struct seq_file *m) +__SMBDIRECT_PUBLIC__ +void smbdirect_connection_legacy_debug_proc_show(struct smbdirect_socket *sc, +unsigned int rdma_readwrite_threshold, +struct seq_file *m) {const struct smbdirect_socket_parameters *sp;
@@ -86,3 +86,4 @@ static void smbdirect_connection_legacy_debug_proc_show(struct smbdirect_socketatomic_read (&sc->mr_io.ready.count), atomic_read(&sc->mr_io.used.count)); } +__SMBDIRECT_EXPORT_SYMBOL__ (smbdirect_connection_legacy_debug_proc_show);
fs/smb/common/smbdirect/smbdirect_internal.h +1 -0
@@ -13,6 +13,7 @@#include <linux/errname.h> #include "smbdirect.h" #include "smbdirect_pdu.h" +#include "smbdirect_public.h" #include "smbdirect_socket.h" static void __smbdirect_socket_schedule_cleanup(struct smbdirect_socket *sc,
fs/smb/common/smbdirect/smbdirect_mr.c +10 -7
@@ -330,8 +330,8 @@ static int smbdirect_iter_to_sgt(struct iov_iter *iter,* need_invalidate: true if this MR needs to be locally invalidated after I/O * return value: the MR registered, NULL if failed. */ -__maybe_unused /* this is temporary while this file is included in others */ -static struct smbdirect_mr_io * +__SMBDIRECT_PUBLIC__ +struct smbdirect_mr_io * smbdirect_connection_register_mr_io(struct smbdirect_socket *sc, struct iov_iter *iter, bool writing,
@@ -452,10 +452,11 @@ smbdirect_connection_register_mr_io(struct smbdirect_socket *sc,mutex_unlock (&mr->mutex);return NULL ; } +__SMBDIRECT_EXPORT_SYMBOL__ (smbdirect_connection_register_mr_io); -__maybe_unused /* this is temporary while this file is included in others */ -static void smbdirect_mr_io_fill_buffer_descriptor (struct smbdirect_mr_io *mr, -struct smbdirect_buffer_descriptor_v1 *v1) +__SMBDIRECT_PUBLIC__ +void smbdirect_mr_io_fill_buffer_descriptor(struct smbdirect_mr_io *mr, +struct smbdirect_buffer_descriptor_v1 *v1) {mutex_lock (&mr->mutex);if (mr->state ==SMBDIRECT_MR_REGISTERED ) {
@@ -469,6 +470,7 @@ static void smbdirect_mr_io_fill_buffer_descriptor(struct smbdirect_mr_io *mr, }mutex_unlock (&mr->mutex); } +__SMBDIRECT_EXPORT_SYMBOL__ (smbdirect_mr_io_fill_buffer_descriptor);/* * Deregister a MR after I/O is done
@@ -476,8 +478,8 @@ static void smbdirect_mr_io_fill_buffer_descriptor(struct smbdirect_mr_io *mr,* and we have to locally invalidate the buffer to prevent data is being * modified by remote peer after upper layer consumes it */ -__maybe_unused /* this is temporary while this file is included in others */ -static void smbdirect_connection_deregister_mr_io (struct smbdirect_mr_io *mr) +__SMBDIRECT_PUBLIC__ +void smbdirect_connection_deregister_mr_io(struct smbdirect_mr_io *mr) {struct smbdirect_socket *sc = mr->socket;int ret =0 ;
@@ -559,3 +561,4 @@ static void smbdirect_connection_deregister_mr_io(struct smbdirect_mr_io *mr)if (!kref_put (&mr->kref, smbdirect_mr_io_free_locked))mutex_unlock (&mr->mutex); } +__SMBDIRECT_EXPORT_SYMBOL__ (smbdirect_connection_deregister_mr_io);
fs/smb/common/smbdirect/smbdirect_public.h new file +154 -0
154 lines not on this page · raw message
fs/smb/common/smbdirect/smbdirect_rw.c +7 -6
@@ -105,12 +105,12 @@ static void smbdirect_connection_rdma_write_done(struct ib_cq *cq, struct ib_wcsmbdirect_connection_rdma_rw_done (cq, wc,DMA_TO_DEVICE ); } -__maybe_unused /* this is temporary while this file is included in others */ -static int smbdirect_connection_rdma_xmit (struct smbdirect_socket *sc, -void *buf,size_t buf_len, -struct smbdirect_buffer_descriptor_v1 *desc, -size_t desc_len, -bool is_read) +__SMBDIRECT_PUBLIC__ +int smbdirect_connection_rdma_xmit(struct smbdirect_socket *sc, +void *buf,size_t buf_len, +struct smbdirect_buffer_descriptor_v1 *desc, +size_t desc_len, +bool is_read) {const struct smbdirect_socket_parameters *sp = &sc->parameters;enum dma_data_direction direction = is_read ?DMA_FROM_DEVICE :DMA_TO_DEVICE ;
@@ -253,3 +253,4 @@ static int smbdirect_connection_rdma_xmit(struct smbdirect_socket *sc,kfree (msg); goto out; } +__SMBDIRECT_EXPORT_SYMBOL__ (smbdirect_connection_rdma_xmit);
fs/smb/common/smbdirect/smbdirect_socket.c +46 -36
176 lines not on this page · raw message
fs/smb/common/smbdirect/smbdirect_socket.h +0 -14
@@ -386,20 +386,6 @@ struct smbdirect_socket { } statistics;struct { -#define SMBDIRECT_LOG_ERR 0x0 -#define SMBDIRECT_LOG_INFO 0x1 - -#define SMBDIRECT_LOG_OUTGOING 0x1 -#define SMBDIRECT_LOG_INCOMING 0x2 -#define SMBDIRECT_LOG_READ 0x4 -#define SMBDIRECT_LOG_WRITE 0x8 -#define SMBDIRECT_LOG_RDMA_SEND 0x10 -#define SMBDIRECT_LOG_RDMA_RECV 0x20 -#define SMBDIRECT_LOG_KEEP_ALIVE 0x40 -#define SMBDIRECT_LOG_RDMA_EVENT 0x80 -#define SMBDIRECT_LOG_RDMA_MR 0x100 -#define SMBDIRECT_LOG_RDMA_RW 0x200 -#define SMBDIRECT_LOG_NEGOTIATE 0x400void *private_ptr; bool (*needed)(struct smbdirect_socket *sc, void *private_ptr,
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 59/145 smb: smbdirect: provide explicit prototypes for cross .c file functions
In reply to Stefan MetzmacherThese prototypes are used between private .c files, when they will be compiled alone into smbdirect.ko.
For now this uses the SMBDIRECT_USE_INLINE_C_FILES code path and marks all function as '__maybe_unused static', but this will make further changes easier.
--- .../common/smbdirect/smbdirect_connection.c | 67 ++++++------ fs/smb/common/smbdirect/smbdirect_internal.h | 102 ++++++++++++++---- fs/smb/common/smbdirect/smbdirect_mr.c | 7 +- fs/smb/common/smbdirect/smbdirect_socket.c | 39 +++---- 4 files changed, 145 insertions(+), 70 deletions(-)
fs/smb/common/smbdirect/smbdirect_connection.c +36 -31
@@ -143,8 +143,8 @@ static int smbdirect_connection_rdma_event_handler(struct rdma_cm_id *id,return 0 ; } -__maybe_unused /* this is temporary while this file is included in others */ -static void smbdirect_connection_rdma_established (struct smbdirect_socket *sc) +__SMBDIRECT_PRIVATE__ +void smbdirect_connection_rdma_established(struct smbdirect_socket *sc) { smbdirect_log_rdma_event(sc, SMBDIRECT_LOG_INFO, "rdma established: device: %.*s local: %pISpsfc remote: %pISpsfc\n",
@@ -157,8 +157,8 @@ static void smbdirect_connection_rdma_established(struct smbdirect_socket *sc) sc->rdma.expected_event =RDMA_CM_EVENT_DISCONNECTED ; } -__maybe_unused /* this is temporary while this file is included in others */ -static void smbdirect_connection_negotiation_done (struct smbdirect_socket *sc) +__SMBDIRECT_PRIVATE__ +void smbdirect_connection_negotiation_done(struct smbdirect_socket *sc) {if (unlikely (sc->first_error))return ;
@@ -233,8 +233,8 @@ static u32 smbdirect_rdma_rw_send_wrs(struct ib_device *dev,return factor * attr->cap.max_rdma_ctxs; } -__maybe_unused /* this is temporary while this file is included in others */ -static int smbdirect_connection_create_qp (struct smbdirect_socket *sc) +__SMBDIRECT_PRIVATE__ +int smbdirect_connection_create_qp(struct smbdirect_socket *sc) {const struct smbdirect_socket_parameters *sp = &sc->parameters;struct ib_qp_init_attr qp_attr;
@@ -391,7 +391,8 @@ static int smbdirect_connection_create_qp(struct smbdirect_socket *sc)return ret; } -static void smbdirect_connection_destroy_qp (struct smbdirect_socket *sc) +__SMBDIRECT_PRIVATE__ +void smbdirect_connection_destroy_qp(struct smbdirect_socket *sc) {if (sc->ib.qp) {ib_drain_qp (sc->ib.qp);
@@ -412,8 +413,8 @@ static void smbdirect_connection_destroy_qp(struct smbdirect_socket *sc) } } -__maybe_unused /* this is temporary while this file is included in others */ -static int smbdirect_connection_create_mem_pools (struct smbdirect_socket *sc) +__SMBDIRECT_PRIVATE__ +int smbdirect_connection_create_mem_pools(struct smbdirect_socket *sc) {const struct smbdirect_socket_parameters *sp = &sc->parameters;char name[80 ];
@@ -492,7 +493,8 @@ static int smbdirect_connection_create_mem_pools(struct smbdirect_socket *sc)return -ENOMEM ; } -static void smbdirect_connection_destroy_mem_pools (struct smbdirect_socket *sc) +__SMBDIRECT_PRIVATE__ +void smbdirect_connection_destroy_mem_pools(struct smbdirect_socket *sc) {struct smbdirect_recv_io *recv_io, *next_io;
@@ -523,8 +525,8 @@ static void smbdirect_connection_destroy_mem_pools(struct smbdirect_socket *sc) sc->send_io.mem.cache =NULL ; } -__maybe_unused /* this is temporary while this file is included in others */ -static struct smbdirect_send_io *smbdirect_connection_alloc_send_io (struct smbdirect_socket *sc) +__SMBDIRECT_PRIVATE__ +struct smbdirect_send_io *smbdirect_connection_alloc_send_io(struct smbdirect_socket *sc) {struct smbdirect_send_io *msg;
@@ -538,8 +540,8 @@ static struct smbdirect_send_io *smbdirect_connection_alloc_send_io(struct smbdireturn msg; } -__maybe_unused /* this is temporary while this file is included in others */ -static void smbdirect_connection_free_send_io (struct smbdirect_send_io *msg) +__SMBDIRECT_PRIVATE__ +void smbdirect_connection_free_send_io(struct smbdirect_send_io *msg) {struct smbdirect_socket *sc = msg->socket;size_t i;
@@ -570,8 +572,8 @@ static void smbdirect_connection_free_send_io(struct smbdirect_send_io *msg)mempool_free (msg, sc->send_io.mem.pool); } -__maybe_unused /* this is temporary while this file is included in others */ -static struct smbdirect_recv_io *smbdirect_connection_get_recv_io (struct smbdirect_socket *sc) +__SMBDIRECT_PRIVATE__ +struct smbdirect_recv_io *smbdirect_connection_get_recv_io(struct smbdirect_socket *sc) {struct smbdirect_recv_io *msg =NULL ;unsigned long flags;
@@ -590,8 +592,8 @@ static struct smbdirect_recv_io *smbdirect_connection_get_recv_io(struct smbdirereturn msg; } -__maybe_unused /* this is temporary while this file is included in others */ -static void smbdirect_connection_put_recv_io (struct smbdirect_recv_io *msg) +__SMBDIRECT_PRIVATE__ +void smbdirect_connection_put_recv_io(struct smbdirect_recv_io *msg) {struct smbdirect_socket *sc = msg->socket;unsigned long flags;
@@ -657,11 +659,11 @@ smbdirect_connection_reassembly_first_recv_io(struct smbdirect_socket *sc)return msg; } -__maybe_unused /* this is temporary while this file is included in others */ -static void smbdirect_connection_negotiate_rdma_resources (struct smbdirect_socket *sc, -u8 peer_initiator_depth, -u8 peer_responder_resources, -const struct rdma_conn_param *param) +__SMBDIRECT_PRIVATE__ +void smbdirect_connection_negotiate_rdma_resources(struct smbdirect_socket *sc, +u8 peer_initiator_depth, +u8 peer_responder_resources, +const struct rdma_conn_param *param) {struct smbdirect_socket_parameters *sp = &sc->parameters;
@@ -797,7 +799,8 @@ int smbdirect_connection_wait_for_connected(struct smbdirect_socket *sc) }__SMBDIRECT_EXPORT_SYMBOL__ (smbdirect_connection_wait_for_connected); -static void smbdirect_connection_idle_timer_work (struct work_struct *work) +__SMBDIRECT_PRIVATE__ +void smbdirect_connection_idle_timer_work(struct work_struct *work) {struct smbdirect_socket *sc = container_of(work, struct smbdirect_socket, idle.timer_work.work);
@@ -867,8 +870,9 @@ static bool smbdirect_connection_request_keep_alive(struct smbdirect_socket *sc)return false; } -static int smbdirect_connection_post_send_wr (struct smbdirect_socket *sc, -struct ib_send_wr *wr) +__SMBDIRECT_PRIVATE__ +int smbdirect_connection_post_send_wr(struct smbdirect_socket *sc, +struct ib_send_wr *wr) {int ret;
@@ -1342,8 +1346,8 @@ static void smbdirect_connection_send_immediate_work(struct work_struct *work) } } -__maybe_unused /* this is temporary while this file is included in others */ -static int smbdirect_connection_post_recv_io (struct smbdirect_recv_io *msg) +__SMBDIRECT_PRIVATE__ +int smbdirect_connection_post_recv_io(struct smbdirect_recv_io *msg) {struct smbdirect_socket *sc = msg->socket;const struct smbdirect_socket_parameters *sp = &sc->parameters;
@@ -1384,8 +1388,8 @@ static int smbdirect_connection_post_recv_io(struct smbdirect_recv_io *msg)return ret; } -__maybe_unused /* this is temporary while this file is included in others */ -static void smbdirect_connection_recv_io_done (struct ib_cq *cq,struct ib_wc *wc) +__SMBDIRECT_PRIVATE__ +void smbdirect_connection_recv_io_done(struct ib_cq *cq, struct ib_wc *wc) {struct smbdirect_recv_io *recv_io = container_of(wc->wr_cqe, struct smbdirect_recv_io, cqe);
@@ -1551,7 +1555,8 @@ static void smbdirect_connection_recv_io_done(struct ib_cq *cq, struct ib_wc *wcsmbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); } -static int smbdirect_connection_recv_io_refill (struct smbdirect_socket *sc) +__SMBDIRECT_PRIVATE__ +int smbdirect_connection_recv_io_refill(struct smbdirect_socket *sc) {int missing;int posted =0 ;
fs/smb/common/smbdirect/smbdirect_internal.h +84 -18
118 lines not on this page · raw message
fs/smb/common/smbdirect/smbdirect_mr.c +4 -3
@@ -15,8 +15,8 @@ static void smbdirect_connection_mr_io_recovery_work(struct work_struct *work);* Recovery is done in smbd_mr_recovery_work. The content of list entry changes * as MRs are used and recovered for I/O, but the list links will not change */ -__maybe_unused /* this is temporary while this file is included in others */ -static int smbdirect_connection_create_mr_list (struct smbdirect_socket *sc) +__SMBDIRECT_PRIVATE__ +int smbdirect_connection_create_mr_list(struct smbdirect_socket *sc) {const struct smbdirect_socket_parameters *sp = &sc->parameters;struct smbdirect_mr_io *mr;
@@ -121,7 +121,8 @@ static void smbdirect_mr_io_free_locked(struct kref *kref)kfree (mr); } -static void smbdirect_connection_destroy_mr_list (struct smbdirect_socket *sc) +__SMBDIRECT_PRIVATE__ +void smbdirect_connection_destroy_mr_list(struct smbdirect_socket *sc) {struct smbdirect_mr_io *mr, *tmp;LIST_HEAD (all_list);
fs/smb/common/smbdirect/smbdirect_socket.c +21 -18
@@ -52,7 +52,8 @@ static int smbdirect_socket_rdma_event_handler(struct rdma_cm_id *id,return -ESTALE ; } -static int smbdirect_socket_init_new (struct net *net,struct smbdirect_socket *sc) +__SMBDIRECT_PRIVATE__ +int smbdirect_socket_init_new(struct net *net, struct smbdirect_socket *sc) {struct rdma_cm_id *id;int ret;
@@ -110,7 +111,8 @@ int smbdirect_socket_create_kern(struct net *net, struct smbdirect_socket **_sc) }__SMBDIRECT_EXPORT_SYMBOL__ (smbdirect_socket_create_kern); -static int smbdirect_socket_init_accepting (struct rdma_cm_id *id,struct smbdirect_socket *sc) +__SMBDIRECT_PRIVATE__ +int smbdirect_socket_init_accepting(struct rdma_cm_id *id, struct smbdirect_socket *sc) {smbdirect_socket_init (sc);
@@ -295,13 +297,14 @@ static void smbdirect_socket_wake_up_all(struct smbdirect_socket *sc)wake_up_all (&sc->mr_io.cleanup.wait_queue); } -static void __smbdirect_socket_schedule_cleanup (struct smbdirect_socket *sc, -const char *macro_name, -unsigned int lvl, -const char *func, -unsigned int line, -int error, -enum smbdirect_socket_status *force_status) +__SMBDIRECT_PRIVATE__ +void __smbdirect_socket_schedule_cleanup(struct smbdirect_socket *sc, +const char *macro_name, +unsigned int lvl, +const char *func, +unsigned int line, +int error, +enum smbdirect_socket_status *force_status) {struct smbdirect_recv_io *recv_io, *recv_tmp;unsigned long flags;
@@ -593,8 +596,8 @@ static void smbdirect_socket_destroy(struct smbdirect_socket *sc) "rdma session destroyed\n"); } -__maybe_unused /* this is temporary while this file is included in others */ -static void smbdirect_socket_destroy_sync (struct smbdirect_socket *sc) +__SMBDIRECT_PRIVATE__ +void smbdirect_socket_destroy_sync(struct smbdirect_socket *sc) { smbdirect_log_rdma_event(sc, SMBDIRECT_LOG_INFO, "status=%s first_error=%1pe",
@@ -693,13 +696,13 @@ void smbdirect_socket_release(struct smbdirect_socket *sc) }__SMBDIRECT_EXPORT_SYMBOL__ (smbdirect_socket_release); -__maybe_unused /* this is temporary while this file is included in others */ -static int smbdirect_socket_wait_for_credits (struct smbdirect_socket *sc, -enum smbdirect_socket_status expected_status, -int unexpected_errno, -wait_queue_head_t *waitq, -atomic_t *total_credits, -int needed) +__SMBDIRECT_PRIVATE__ +int smbdirect_socket_wait_for_credits(struct smbdirect_socket *sc, +enum smbdirect_socket_status expected_status, +int unexpected_errno, +wait_queue_head_t *waitq, +atomic_t *total_credits, +int needed) {int ret;
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 60/145 smb: smbdirect: introduce the basic smbdirect.ko
In reply to Stefan MetzmacherThis exports the functions needed by cifs.ko and ksmbd.ko.
It doesn't yet provide a generic socket layer, but it is a good start to introduce that on top.
It will be much easier after Davids refactoring using MSG_SPLICE_PAGES, will make it easier to use the socket layer without an additional copy.
--- fs/smb/Kconfig | 2 + fs/smb/common/Makefile | 1 + fs/smb/common/smbdirect/Kconfig | 9 +++++ fs/smb/common/smbdirect/Makefile | 16 ++++++++ fs/smb/common/smbdirect/smbdirect_internal.h | 9 +++++ fs/smb/common/smbdirect/smbdirect_main.c | 40 ++++++++++++++++++++ 6 files changed, 77 insertions(+) create mode 100644 fs/smb/common/smbdirect/Kconfig create mode 100644 fs/smb/common/smbdirect/Makefile create mode 100644 fs/smb/common/smbdirect/smbdirect_main.c
fs/smb/Kconfig +2 -0
@@ -9,3 +9,5 @@ config SMBFS tristate default y if CIFS=y || SMB_SERVER=y default m if CIFS=m || SMB_SERVER=m + +source "fs/smb/common/smbdirect/Kconfig"
fs/smb/common/Makefile +1 -0
@@ -4,3 +4,4 @@ # obj-$(CONFIG_SMBFS) += cifs_md4.o +obj-$(CONFIG_SMB_COMMON_SMBDIRECT) += smbdirect/
fs/smb/common/smbdirect/Kconfig new file +9 -0
@@ -0,0 +1,9 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# +# smbdirect configuration + +config SMB_COMMON_SMBDIRECT + def_tristate n + depends on INFINIBAND && INFINIBAND_ADDR_TRANS + depends on m || INFINIBAND != m + select SG_POOL
fs/smb/common/smbdirect/Makefile new file +16 -0
@@ -0,0 +1,16 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Makefile for smbdirect support +# + +obj-$(CONFIG_SMB_COMMON_SMBDIRECT) += smbdirect.o + +smbdirect-y := \ + smbdirect_socket.o \ + smbdirect_connection.o \ + smbdirect_mr.o \ + smbdirect_rw.o \ + smbdirect_debug.o \ + smbdirect_connect.o \ + smbdirect_accept.o \ + smbdirect_main.o
fs/smb/common/smbdirect/smbdirect_internal.h +9 -0
@@ -14,6 +14,15 @@#include "smbdirect.h" #include "smbdirect_pdu.h" #include "smbdirect_public.h" + +#include <linux/mutex.h> + +struct smbdirect_module_state { +struct mutex mutex; +}; + +extern struct smbdirect_module_state smbdirect_globals; +#include "smbdirect_socket.h" #ifdef SMBDIRECT_USE_INLINE_C_FILES
fs/smb/common/smbdirect/smbdirect_main.c new file +40 -0
@@ -0,0 +1,40 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* +* Copyright (c) 2025, Stefan Metzmacher +*/ + +#include "smbdirect_internal.h" +#include <linux/module.h> + +struct smbdirect_module_state smbdirect_globals = { + .mutex =__MUTEX_INITIALIZER (smbdirect_globals.mutex), +}; + +static __init int smbdirect_module_init (void ) +{ +pr_notice ("subsystem loading...\n" ); +mutex_lock (&smbdirect_globals.mutex); + +/* TODO... */ + +mutex_unlock (&smbdirect_globals.mutex); +pr_notice ("subsystem loaded\n" ); +return 0 ; +} + +static __exit void smbdirect_module_exit(void) +{ +pr_notice ("subsystem unloading...\n" ); +mutex_lock (&smbdirect_globals.mutex); + +/* TODO... */ + +mutex_unlock (&smbdirect_globals.mutex); +pr_notice ("subsystem unloaded\n" ); +} + +module_init (smbdirect_module_init); +module_exit (smbdirect_module_exit); + +MODULE_DESCRIPTION ("smbdirect subsystem" ); +MODULE_LICENSE ("GPL" );
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 61/145 smb: client: make use of smbdirect_socket_prepare_create()
In reply to Stefan MetzmacherThis prepares the use of functions from smbdirect_connection.c.
--- fs/smb/client/smbdirect.c | 35 +++++++++++++++++++++-------------- 1 file changed, 21 insertions(+), 14 deletions(-)
fs/smb/client/smbdirect.c +21 -14
@@ -1800,6 +1800,7 @@ static struct smbd_connection *_smbd_get_connection(int rc;struct smbd_connection *info;struct smbdirect_socket *sc; +struct smbdirect_socket_parameters init_params = {};struct smbdirect_socket_parameters *sp;struct rdma_conn_param conn_param;struct ib_qp_cap qp_cap;
@@ -1810,20 +1811,10 @@ static struct smbd_connection *_smbd_get_connection(char wq_name[80 ];struct workqueue_struct *workqueue; - info =kzalloc (sizeof (struct smbd_connection ),GFP_KERNEL ); -if (!info) -return NULL ; - sc = &info->socket; -scnprintf (wq_name,ARRAY_SIZE (wq_name),"smbd_%p" , sc); - workqueue =create_workqueue (wq_name); -if (!workqueue) - goto create_wq_failed; -smbdirect_socket_init (sc); - sc->workqueue = workqueue; - sp = &sc->parameters; - -INIT_WORK (&sc->disconnect_work, smbd_disconnect_rdma_work); - +/* +* Create the initial parameters +*/ + sp = &init_params; sp->resolve_addr_timeout_msec =RDMA_RESOLVE_TIMEOUT ; sp->resolve_route_timeout_msec =RDMA_RESOLVE_TIMEOUT ; sp->rdma_connect_timeout_msec =RDMA_RESOLVE_TIMEOUT ;
@@ -1839,6 +1830,22 @@ static struct smbd_connection *_smbd_get_connection( sp->keepalive_interval_msec = smbd_keep_alive_interval *1000 ; sp->keepalive_timeout_msec =KEEPALIVE_RECV_TIMEOUT *1000 ; + info =kzalloc (sizeof (struct smbd_connection ),GFP_KERNEL ); +if (!info) +return NULL ; + sc = &info->socket; +scnprintf (wq_name,ARRAY_SIZE (wq_name),"smbd_%p" , sc); + workqueue =create_workqueue (wq_name); +if (!workqueue) + goto create_wq_failed; +smbdirect_socket_prepare_create (sc, sp, workqueue); +/* +* from here we operate on the copy. +*/ + sp = &sc->parameters; + +INIT_WORK (&sc->disconnect_work, smbd_disconnect_rdma_work); + rc =smbd_ia_open (sc, dstaddr, port);if (rc) {log_rdma_event (INFO ,"smbd_ia_open rc=%d\n" , rc);
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 62/145 smb: client: make use of smbdirect_socket_set_logging()
In reply to Stefan MetzmacherThis will allow the logging to keep working as before, when we move to common functions in the next commits.
--- fs/smb/client/smbdirect.c | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+)
fs/smb/client/smbdirect.c +38 -0
@@ -156,6 +156,43 @@ static void smbd_disconnect_rdma_connection(struct smbdirect_socket *sc);*/ #include "../common/smbdirect/smbdirect_all_c_files.c" +static bool smbd_logging_needed (struct smbdirect_socket *sc, +void *private_ptr, +unsigned int lvl, +unsigned int cls) +{ +#define BUILD_BUG_SAME (x) BUILD_BUG_ON(x != SMBDIRECT_LOG_ ##x) +BUILD_BUG_SAME (ERR ); +BUILD_BUG_SAME (INFO ); +#undef BUILD_BUG_SAME +#define BUILD_BUG_SAME (x) BUILD_BUG_ON(x != SMBDIRECT_ ##x) +BUILD_BUG_SAME (LOG_OUTGOING ); +BUILD_BUG_SAME (LOG_INCOMING ); +BUILD_BUG_SAME (LOG_READ ); +BUILD_BUG_SAME (LOG_WRITE ); +BUILD_BUG_SAME (LOG_RDMA_SEND ); +BUILD_BUG_SAME (LOG_RDMA_RECV ); +BUILD_BUG_SAME (LOG_KEEP_ALIVE ); +BUILD_BUG_SAME (LOG_RDMA_EVENT ); +BUILD_BUG_SAME (LOG_RDMA_MR ); +#undef BUILD_BUG_SAME + +if (lvl <= smbd_logging_level || cls & smbd_logging_class) +return true; +return false; +} + +static void smbd_logging_vaprintf (struct smbdirect_socket *sc, +const char *func, +unsigned int line, +void *private_ptr, +unsigned int lvl, +unsigned int cls, +struct va_format *vaf) +{ +cifs_dbg (VFS ,"%s:%u %pV" , func, line, vaf); +} + #define log_rdma(level, class, fmt, args...) \ do { \ if (level <= smbd_logging_level || class & smbd_logging_class) \
@@ -1839,6 +1876,7 @@ static struct smbd_connection *_smbd_get_connection(if (!workqueue) goto create_wq_failed;smbdirect_socket_prepare_create (sc, sp, workqueue); +smbdirect_socket_set_logging (sc,NULL , smbd_logging_needed, smbd_logging_vaprintf);/* * from here we operate on the copy. */
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 63/145 smb: client: make use of smbdirect_socket_wake_up_all()
In reply to Stefan MetzmacherThis is a superset of smbd_disconnect_wake_up_all() and calling wake_up_all(&sc->rw_io.credits.wait_queue); in addition should not matter as it's not used on the client anyway.
--- fs/smb/client/smbdirect.c | 22 +++------------------- 1 file changed, 3 insertions(+), 19 deletions(-)
fs/smb/client/smbdirect.c +3 -19
@@ -216,22 +216,6 @@ do { \ #define log_rdma_mr(level, fmt, args...) \ log_rdma(level, LOG_RDMA_MR, fmt, ##args) -static void smbd_disconnect_wake_up_all (struct smbdirect_socket *sc) -{ -/* -* Wake up all waiters in all wait queues -* in order to notice the broken connection. -*/ -wake_up_all (&sc->status_wait); -wake_up_all (&sc->send_io.lcredits.wait_queue); -wake_up_all (&sc->send_io.credits.wait_queue); -wake_up_all (&sc->send_io.pending.dec_wait_queue); -wake_up_all (&sc->send_io.pending.zero_wait_queue); -wake_up_all (&sc->recv_io.reassembly.wait_queue); -wake_up_all (&sc->mr_io.ready.wait_queue); -wake_up_all (&sc->mr_io.cleanup.wait_queue); -} -static void smbd_disconnect_rdma_work (struct work_struct *work) { struct smbdirect_socket *sc =
@@ -288,7 +272,7 @@ static void smbd_disconnect_rdma_work(struct work_struct *work)* Wake up all waiters in all wait queues * in order to notice the broken connection. */ -smbd_disconnect_wake_up_all (sc); +smbdirect_socket_wake_up_all (sc); }static void smbd_disconnect_rdma_connection (struct smbdirect_socket *sc)
@@ -353,7 +337,7 @@ static void smbd_disconnect_rdma_connection(struct smbdirect_socket *sc)* Wake up all waiters in all wait queues * in order to notice the broken connection. */ -smbd_disconnect_wake_up_all (sc); +smbdirect_socket_wake_up_all (sc);queue_work (sc->workqueue, &sc->disconnect_work); }
@@ -1662,7 +1646,7 @@ void smbd_destroy(struct TCP_Server_Info *server)* Most likely this was already called via * smbd_disconnect_rdma_work(), but call it again... */ -smbd_disconnect_wake_up_all (sc); +smbdirect_socket_wake_up_all (sc);log_rdma_event (INFO ,"cancelling recv_io.posted.refill_work\n" );disable_work_sync (&sc->recv_io.posted.refill_work);
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 64/145 smb: client: let allocate_receive_buffers() initialize complex_work as disabled
In reply to Stefan Metzmachersmbdirect_recv_io.complex_work won't be used in client until the end of the move to common functions, but common cleanup code will call disable_work[_sync]() before that, so we better initialize it now.
--- fs/smb/client/smbdirect.c | 2 ++ 1 file changed, 2 insertions(+)
fs/smb/client/smbdirect.c +2 -0
@@ -1543,6 +1543,8 @@ static int allocate_receive_buffers(struct smbdirect_socket *sc, int num_buf) response->socket = sc; response->sge.length =0 ; +INIT_WORK (&response->complex_work, __smbdirect_socket_disabled_work); +disable_work_sync (&response->complex_work);list_add_tail (&response->list, &sc->recv_io.free.list); }
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 65/145 smb: client: make use of smbdirect_socket_cleanup_work()
In reply to Stefan MetzmacherNote smbdirect_socket_prepare_create() already calls INIT_WORK() with smbdirect_socket_cleanup_work.
--- fs/smb/client/smbdirect.c | 73 ++++----------------------------------- 1 file changed, 6 insertions(+), 67 deletions(-)
fs/smb/client/smbdirect.c +6 -67
@@ -216,65 +216,6 @@ do { \ #define log_rdma_mr(level, fmt, args...) \ log_rdma(level, LOG_RDMA_MR, fmt, ##args) -static void smbd_disconnect_rdma_work (struct work_struct *work) -{ -struct smbdirect_socket *sc = - container_of(work, struct smbdirect_socket, disconnect_work); - -if (sc->first_error ==0 ) - sc->first_error = -ECONNABORTED ; - -/* -* make sure this and other work is not queued again -* but here we don't block and avoid -* disable[_delayed]_work_sync() -*/ -disable_work (&sc->disconnect_work); -disable_work (&sc->recv_io.posted.refill_work); -disable_work (&sc->mr_io.recovery_work); -disable_work (&sc->idle.immediate_work); -disable_delayed_work (&sc->idle.timer_work); - -switch (sc->status) { -case SMBDIRECT_SOCKET_NEGOTIATE_NEEDED : -case SMBDIRECT_SOCKET_NEGOTIATE_RUNNING : -case SMBDIRECT_SOCKET_NEGOTIATE_FAILED : -case SMBDIRECT_SOCKET_CONNECTED : -case SMBDIRECT_SOCKET_ERROR : - sc->status =SMBDIRECT_SOCKET_DISCONNECTING ; -rdma_disconnect (sc->rdma.cm_id); -break ; - -case SMBDIRECT_SOCKET_CREATED : -case SMBDIRECT_SOCKET_RESOLVE_ADDR_NEEDED : -case SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING : -case SMBDIRECT_SOCKET_RESOLVE_ADDR_FAILED : -case SMBDIRECT_SOCKET_RESOLVE_ROUTE_NEEDED : -case SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING : -case SMBDIRECT_SOCKET_RESOLVE_ROUTE_FAILED : -case SMBDIRECT_SOCKET_RDMA_CONNECT_NEEDED : -case SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING : -case SMBDIRECT_SOCKET_RDMA_CONNECT_FAILED : -/* -* rdma_connect() never reached -* RDMA_CM_EVENT_ESTABLISHED -*/ - sc->status =SMBDIRECT_SOCKET_DISCONNECTED ; -break ; - -case SMBDIRECT_SOCKET_DISCONNECTING : -case SMBDIRECT_SOCKET_DISCONNECTED : -case SMBDIRECT_SOCKET_DESTROYED : -break ; - } - -/* -* Wake up all waiters in all wait queues -* in order to notice the broken connection. -*/ -smbdirect_socket_wake_up_all (sc); -} -static void smbd_disconnect_rdma_connection (struct smbdirect_socket *sc) {if (sc->first_error ==0 )
@@ -373,13 +314,13 @@ static int smbd_conn_upcall(case RDMA_CM_EVENT_ADDR_ERROR :log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name); sc->status =SMBDIRECT_SOCKET_RESOLVE_ADDR_FAILED ; -smbd_disconnect_rdma_work (&sc->disconnect_work); +smbdirect_socket_cleanup_work (&sc->disconnect_work);break ;case RDMA_CM_EVENT_ROUTE_ERROR :log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name); sc->status =SMBDIRECT_SOCKET_RESOLVE_ROUTE_FAILED ; -smbd_disconnect_rdma_work (&sc->disconnect_work); +smbdirect_socket_cleanup_work (&sc->disconnect_work);break ;case RDMA_CM_EVENT_ESTABLISHED :
@@ -474,7 +415,7 @@ static int smbd_conn_upcall(case RDMA_CM_EVENT_REJECTED :log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name); sc->status =SMBDIRECT_SOCKET_RDMA_CONNECT_FAILED ; -smbd_disconnect_rdma_work (&sc->disconnect_work); +smbdirect_socket_cleanup_work (&sc->disconnect_work);break ;case RDMA_CM_EVENT_DEVICE_REMOVAL :
@@ -485,7 +426,7 @@ static int smbd_conn_upcall( } sc->status =SMBDIRECT_SOCKET_DISCONNECTED ; -smbd_disconnect_rdma_work (&sc->disconnect_work); +smbdirect_socket_cleanup_work (&sc->disconnect_work);break ;default :
@@ -1634,7 +1575,7 @@ void smbd_destroy(struct TCP_Server_Info *server)log_rdma_event (INFO ,"destroying rdma session\n" );if (sc->status <SMBDIRECT_SOCKET_DISCONNECTING ) -smbd_disconnect_rdma_work (&sc->disconnect_work); +smbdirect_socket_cleanup_work (&sc->disconnect_work);if (sc->status <SMBDIRECT_SOCKET_DISCONNECTED ) {log_rdma_event (INFO ,"wait for transport being disconnected\n" );wait_event (sc->status_wait, sc->status ==SMBDIRECT_SOCKET_DISCONNECTED );
@@ -1646,7 +1587,7 @@ void smbd_destroy(struct TCP_Server_Info *server)* in order to notice the broken connection. * * Most likely this was already called via -* smbd_disconnect_rdma_work(), but call it again... +* smbdirect_socket_cleanup_work(), but call it again... */ smbdirect_socket_wake_up_all (sc);
@@ -1868,8 +1809,6 @@ static struct smbd_connection *_smbd_get_connection(*/ sp = &sc->parameters; -INIT_WORK (&sc->disconnect_work, smbd_disconnect_rdma_work); - rc =smbd_ia_open (sc, dstaddr, port);if (rc) {log_rdma_event (INFO ,"smbd_ia_open rc=%d\n" , rc);
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 66/145 smb: client: make use of smbdirect_socket_schedule_cleanup()
In reply to Stefan MetzmacherThis removes smbd_disconnect_rdma_connection() which is basically the same as smbdirect_socket_schedule_cleanup().
And we pass more useful errors than -ECONNABORTED if we have them.
--- fs/smb/client/smbdirect.c | 97 ++++++--------------------------------- 1 file changed, 14 insertions(+), 83 deletions(-)
fs/smb/client/smbdirect.c +14 -83
@@ -10,7 +10,6 @@#include <linux/module.h> #include <linux/highmem.h> #include <linux/folio_queue.h> -#define __SMBDIRECT_SOCKET_DISCONNECT (__sc) smbd_disconnect_rdma_connection(__sc)#include "../common/smbdirect/smbdirect_pdu.h" #include "smbdirect.h" #include "cifs_debug.h"
@@ -146,8 +145,6 @@ module_param(smbd_logging_level, uint, 0644);MODULE_PARM_DESC (smbd_logging_level,"Logging level for SMBD transport, 0 (default): error, 1: info" ); -static void smbd_disconnect_rdma_connection (struct smbdirect_socket *sc); -/* * This is a temporary solution until all code * is moved to smbdirect_all_c_files.c and we
@@ -216,73 +213,6 @@ do { \ #define log_rdma_mr(level, fmt, args...) \ log_rdma(level, LOG_RDMA_MR, fmt, ##args) -static void smbd_disconnect_rdma_connection (struct smbdirect_socket *sc) -{ -if (sc->first_error ==0 ) - sc->first_error = -ECONNABORTED ; - -/* -* make sure other work (than disconnect_work) is -* not queued again but here we don't block and avoid -* disable[_delayed]_work_sync() -*/ -disable_work (&sc->recv_io.posted.refill_work); -disable_work (&sc->mr_io.recovery_work); -disable_work (&sc->idle.immediate_work); -disable_delayed_work (&sc->idle.timer_work); - -switch (sc->status) { -case SMBDIRECT_SOCKET_RESOLVE_ADDR_FAILED : -case SMBDIRECT_SOCKET_RESOLVE_ROUTE_FAILED : -case SMBDIRECT_SOCKET_RDMA_CONNECT_FAILED : -case SMBDIRECT_SOCKET_NEGOTIATE_FAILED : -case SMBDIRECT_SOCKET_ERROR : -case SMBDIRECT_SOCKET_DISCONNECTING : -case SMBDIRECT_SOCKET_DISCONNECTED : -case SMBDIRECT_SOCKET_DESTROYED : -/* -* Keep the current error status -*/ -break ; - -case SMBDIRECT_SOCKET_RESOLVE_ADDR_NEEDED : -case SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING : - sc->status =SMBDIRECT_SOCKET_RESOLVE_ADDR_FAILED ; -break ; - -case SMBDIRECT_SOCKET_RESOLVE_ROUTE_NEEDED : -case SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING : - sc->status =SMBDIRECT_SOCKET_RESOLVE_ROUTE_FAILED ; -break ; - -case SMBDIRECT_SOCKET_RDMA_CONNECT_NEEDED : -case SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING : - sc->status =SMBDIRECT_SOCKET_RDMA_CONNECT_FAILED ; -break ; - -case SMBDIRECT_SOCKET_NEGOTIATE_NEEDED : -case SMBDIRECT_SOCKET_NEGOTIATE_RUNNING : - sc->status =SMBDIRECT_SOCKET_NEGOTIATE_FAILED ; -break ; - -case SMBDIRECT_SOCKET_CREATED : - sc->status =SMBDIRECT_SOCKET_DISCONNECTED ; -break ; - -case SMBDIRECT_SOCKET_CONNECTED : - sc->status =SMBDIRECT_SOCKET_ERROR ; -break ; - } - -/* -* Wake up all waiters in all wait queues -* in order to notice the broken connection. -*/ -smbdirect_socket_wake_up_all (sc); - -queue_work (sc->workqueue, &sc->disconnect_work); -} -/* Upcall from RDMA CM */ static int smbd_conn_upcall (struct rdma_cm_id *id,struct rdma_cm_event *event)
@@ -450,7 +380,7 @@ smbd_qp_async_error_upcall(struct ib_event *event, void *context)switch (event->event) {case IB_EVENT_CQ_ERR :case IB_EVENT_QP_FATAL : -smbd_disconnect_rdma_connection (sc); +smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED );break ;default :
@@ -492,7 +422,7 @@ static void send_done(struct ib_cq *cq, struct ib_wc *wc)if (wc->status !=IB_WC_WR_FLUSH_ERR )log_rdma_send (ERR ,"wc->status=%s wc->opcode=%d\n" ,ib_wc_status_msg (wc->status), wc->opcode); -smbd_disconnect_rdma_connection (sc); +smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED );return ; }
@@ -679,7 +609,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc) negotiate_done = false;if (!negotiate_done) { sc->status =SMBDIRECT_SOCKET_NEGOTIATE_FAILED ; -smbd_disconnect_rdma_connection (sc); +smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); }else { sc->status =SMBDIRECT_SOCKET_CONNECTED ;wake_up (&sc->status_wait);
@@ -776,7 +706,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)WARN_ON_ONCE (sc->recv_io.expected !=SMBDIRECT_EXPECT_DATA_TRANSFER ); error:put_receive_buffer (sc, response); -smbd_disconnect_rdma_connection (sc); +smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); } static struct rdma_cm_id *smbd_create_id(
@@ -991,7 +921,7 @@ static int smbd_post_send_negotiate_req(struct smbdirect_socket *sc)ib_dma_unmap_single (sc->ib.dev, request->sge[0 ].addr, request->sge[0 ].length,DMA_TO_DEVICE ); -smbd_disconnect_rdma_connection (sc); +smbdirect_socket_schedule_cleanup (sc, rc); dma_mapping_failed:mempool_free (request, sc->send_io.mem.pool);
@@ -1080,7 +1010,7 @@ static int smbd_post_send(struct smbdirect_socket *sc, rc =ib_post_send (sc->ib.qp, &send_wr,NULL );if (rc) {log_rdma_send (ERR ,"ib_post_send failed rc=%d\n" , rc); -smbd_disconnect_rdma_connection (sc); +smbdirect_socket_schedule_cleanup (sc, rc); rc = -EAGAIN ; }
@@ -1321,7 +1251,7 @@ static int smbd_post_recv(ib_dma_unmap_single (sc->ib.dev, response->sge.addr, response->sge.length,DMA_FROM_DEVICE ); response->sge.length =0 ; -smbd_disconnect_rdma_connection (sc); +smbdirect_socket_schedule_cleanup (sc, rc);log_rdma_recv (ERR ,"ib_post_recv failed rc=%d\n" , rc); }
@@ -1534,7 +1464,7 @@ static void idle_connection_timer(struct work_struct *work)log_keep_alive (ERR ,"error status sc->idle.keepalive=%d\n" , sc->idle.keepalive); -smbd_disconnect_rdma_connection (sc); +smbdirect_socket_schedule_cleanup (sc, -ETIMEDOUT );return ; }
@@ -2293,7 +2223,7 @@ static void register_mr_done(struct ib_cq *cq, struct ib_wc *wc)if (wc->status) {log_rdma_mr (ERR ,"status=%d\n" , wc->status); -smbd_disconnect_rdma_connection (sc); +smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); } }
@@ -2323,7 +2253,7 @@ static void smbd_mr_recovery_work(struct work_struct *work)log_rdma_mr (ERR ,"ib_dereg_mr failed rc=%x\n" , rc); -smbd_disconnect_rdma_connection (sc); +smbdirect_socket_schedule_cleanup (sc, rc);continue ; }
@@ -2331,10 +2261,11 @@ static void smbd_mr_recovery_work(struct work_struct *work) sc->ib.pd, sc->mr_io.type, sp->max_frmr_depth);if (IS_ERR (smbdirect_mr->mr)) { + rc =PTR_ERR (smbdirect_mr->mr);log_rdma_mr (ERR ,"ib_alloc_mr failed mr_type=%x max_frmr_depth=%x\n" , sc->mr_io.type, sp->max_frmr_depth); -smbd_disconnect_rdma_connection (sc); +smbdirect_socket_schedule_cleanup (sc, rc);continue ; } }else
@@ -2670,7 +2601,7 @@ struct smbdirect_mr_io *smbd_register_mr(struct smbd_connection *info,if (atomic_dec_and_test (&sc->mr_io.used.count))wake_up (&sc->mr_io.cleanup.wait_queue); -smbd_disconnect_rdma_connection (sc); +smbdirect_socket_schedule_cleanup (sc, rc);/* * get_mr() gave us a reference
@@ -2745,7 +2676,7 @@ void smbd_deregister_mr(struct smbdirect_mr_io *mr)if (rc) {log_rdma_mr (ERR ,"ib_post_send failed rc=%x\n" , rc);smbd_mr_disable_locked (mr); -smbd_disconnect_rdma_connection (sc); +smbdirect_socket_schedule_cleanup (sc, rc); goto done; }wait_for_completion (&mr->invalidate_done);
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 67/145 smb: client: make use of smbdirect_connection_{get,put}_recv_io()
In reply to Stefan MetzmacherThese are basically copies of {get,put}_receive_buffer().
--- fs/smb/client/smbdirect.c | 78 ++++++--------------------------------- 1 file changed, 11 insertions(+), 67 deletions(-)
fs/smb/client/smbdirect.c +11 -67
@@ -23,11 +23,6 @@ const struct smbdirect_socket_parameters *smbd_get_parameters(struct smbd_connecreturn &sc->parameters; } -static struct smbdirect_recv_io *get_receive_buffer ( -struct smbdirect_socket *sc); -static void put_receive_buffer ( -struct smbdirect_socket *sc, -struct smbdirect_recv_io *response);static int allocate_receive_buffers (struct smbdirect_socket *sc,int num_buf);static void destroy_receive_buffers (struct smbdirect_socket *sc);
@@ -531,7 +526,7 @@ static void smbd_post_send_credits(struct work_struct *work)if (sc->recv_io.credits.target >atomic_read (&sc->recv_io.credits.count)) {while (true) { - response =get_receive_buffer (sc); + response =smbdirect_connection_get_recv_io (sc);if (!response)break ;
@@ -540,7 +535,7 @@ static void smbd_post_send_credits(struct work_struct *work)if (rc) {log_rdma_recv (ERR ,"post_recv failed rc=%d\n" , rc); -put_receive_buffer (sc, response); +smbdirect_connection_put_recv_io (response);break ; }
@@ -604,7 +599,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc) sc->recv_io.reassembly.full_packet_received = true; negotiate_done =process_negotiation_response (response, wc->byte_len); -put_receive_buffer (sc, response); +smbdirect_connection_put_recv_io (response);if (SMBDIRECT_CHECK_STATUS_WARN (sc,SMBDIRECT_SOCKET_NEGOTIATE_RUNNING )) negotiate_done = false;if (!negotiate_done) {
@@ -690,7 +685,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)enqueue_reassembly (sc, response, data_length);wake_up (&sc->recv_io.reassembly.wait_queue); }else -put_receive_buffer (sc, response); +smbdirect_connection_put_recv_io (response);return ;
@@ -705,7 +700,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)log_rdma_recv (ERR ,"unexpected response type=%d\n" , sc->recv_io.expected);WARN_ON_ONCE (sc->recv_io.expected !=SMBDIRECT_EXPECT_DATA_TRANSFER ); error: -put_receive_buffer (sc, response); +smbdirect_connection_put_recv_io (response);smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); }
@@ -1263,7 +1258,7 @@ static int smbd_negotiate(struct smbdirect_socket *sc) {struct smbdirect_socket_parameters *sp = &sc->parameters;int rc; -struct smbdirect_recv_io *response =get_receive_buffer (sc); +struct smbdirect_recv_io *response =smbdirect_connection_get_recv_io (sc);WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_NEGOTIATE_NEEDED ); sc->status =SMBDIRECT_SOCKET_NEGOTIATE_RUNNING ;
@@ -1274,7 +1269,7 @@ static int smbd_negotiate(struct smbdirect_socket *sc) rc, response->sge.addr, response->sge.length, response->sge.lkey);if (rc) { -put_receive_buffer (sc, response); +smbdirect_connection_put_recv_io (response);return rc; }
@@ -1350,57 +1345,6 @@ static struct smbdirect_recv_io *_get_first_reassembly(struct smbdirect_socket *return ret; } -/* -* Get a receive buffer -* For each remote send, we need to post a receive. The receive buffers are -* pre-allocated in advance. -* return value: the receive buffer, NULL if none is available -*/ -static struct smbdirect_recv_io *get_receive_buffer (struct smbdirect_socket *sc) -{ -struct smbdirect_recv_io *ret =NULL ; -unsigned long flags; - -spin_lock_irqsave (&sc->recv_io.free.lock, flags); -if (!list_empty (&sc->recv_io.free.list)) { - ret =list_first_entry ( - &sc->recv_io.free.list, - struct smbdirect_recv_io, list); -list_del (&ret->list); - sc->statistics.get_receive_buffer++; - } -spin_unlock_irqrestore (&sc->recv_io.free.lock, flags); - -return ret; -} - -/* -* Return a receive buffer -* Upon returning of a receive buffer, we can post new receive and extend -* more receive credits to remote peer. This is done immediately after a -* receive buffer is returned. -*/ -static void put_receive_buffer ( -struct smbdirect_socket *sc,struct smbdirect_recv_io *response) -{ -unsigned long flags; - -if (likely (response->sge.length !=0 )) { -ib_dma_unmap_single (sc->ib.dev, - response->sge.addr, - response->sge.length, -DMA_FROM_DEVICE ); - response->sge.length =0 ; - } - -spin_lock_irqsave (&sc->recv_io.free.lock, flags); -list_add_tail (&response->list, &sc->recv_io.free.list); - sc->statistics.put_receive_buffer++; -spin_unlock_irqrestore (&sc->recv_io.free.lock, flags); - -queue_work (sc->workqueue, &sc->recv_io.posted.refill_work); -} -/* Preallocate all receive buffer on transport establishment */ static int allocate_receive_buffers (struct smbdirect_socket *sc,int num_buf) {
@@ -1437,7 +1381,7 @@ static void destroy_receive_buffers(struct smbdirect_socket *sc) {struct smbdirect_recv_io *response; -while ((response =get_receive_buffer (sc))) +while ((response =smbdirect_connection_get_recv_io (sc)))mempool_free (response, sc->recv_io.mem.pool); }
@@ -1543,7 +1487,7 @@ void smbd_destroy(struct TCP_Server_Info *server)list_del (&response->list);spin_unlock_irqrestore ( &sc->recv_io.reassembly.lock, flags); -put_receive_buffer (sc, response); +smbdirect_connection_put_recv_io (response); }else spin_unlock_irqrestore ( &sc->recv_io.reassembly.lock, flags);
@@ -2082,9 +2026,9 @@ int smbd_recv(struct smbd_connection *info, struct msghdr *msg) } queue_removed++; sc->statistics.dequeue_reassembly_queue++; -put_receive_buffer (sc, response); +smbdirect_connection_put_recv_io (response); offset =0 ; -log_read (INFO ,"put_receive_buffer offset=0\n" ); +log_read (INFO ,"smbdirect_connection_put_recv_io offset=0\n" ); }else offset += to_copy;
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 68/145 smb: client: make use of smbdirect_connection_reassembly_{append,first}_recv_io()
In reply to Stefan MetzmacherThese are exact copies of enqueue_reassembly() and _get_first_reassembly().
--- fs/smb/client/smbdirect.c | 63 +++------------------------------------ 1 file changed, 4 insertions(+), 59 deletions(-)
fs/smb/client/smbdirect.c +4 -59
@@ -26,12 +26,6 @@ const struct smbdirect_socket_parameters *smbd_get_parameters(struct smbd_connecstatic int allocate_receive_buffers (struct smbdirect_socket *sc,int num_buf);static void destroy_receive_buffers (struct smbdirect_socket *sc); -static void enqueue_reassembly ( -struct smbdirect_socket *sc, -struct smbdirect_recv_io *response,int data_length); -static struct smbdirect_recv_io *_get_first_reassembly ( -struct smbdirect_socket *sc); -static int smbd_post_recv (struct smbdirect_socket *sc,struct smbdirect_recv_io *response);
@@ -682,7 +676,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)if (sc->recv_io.credits.target > old_recv_credit_target)queue_work (sc->workqueue, &sc->recv_io.posted.refill_work); -enqueue_reassembly (sc, response, data_length); +smbdirect_connection_reassembly_append_recv_io (sc, response, data_length);wake_up (&sc->recv_io.reassembly.wait_queue); }else smbdirect_connection_put_recv_io (response);
@@ -1296,55 +1290,6 @@ static int smbd_negotiate(struct smbdirect_socket *sc)return rc; } -/* -* Implement Connection.FragmentReassemblyBuffer defined in [MS-SMBD] 3.1.1.1 -* This is a queue for reassembling upper layer payload and present to upper -* layer. All the inncoming payload go to the reassembly queue, regardless of -* if reassembly is required. The uuper layer code reads from the queue for all -* incoming payloads. -* Put a received packet to the reassembly queue -* response: the packet received -* data_length: the size of payload in this packet -*/ -static void enqueue_reassembly ( -struct smbdirect_socket *sc, -struct smbdirect_recv_io *response, -int data_length) -{ -unsigned long flags; - -spin_lock_irqsave (&sc->recv_io.reassembly.lock, flags); -list_add_tail (&response->list, &sc->recv_io.reassembly.list); - sc->recv_io.reassembly.queue_length++; -/* -* Make sure reassembly_data_length is updated after list and -* reassembly_queue_length are updated. On the dequeue side -* reassembly_data_length is checked without a lock to determine -* if reassembly_queue_length and list is up to date -*/ -virt_wmb (); - sc->recv_io.reassembly.data_length += data_length; -spin_unlock_irqrestore (&sc->recv_io.reassembly.lock, flags); - sc->statistics.enqueue_reassembly_queue++; -} - -/* -* Get the first entry at the front of reassembly queue -* Caller is responsible for locking -* return value: the first entry if any, NULL if queue is empty -*/ -static struct smbdirect_recv_io *_get_first_reassembly (struct smbdirect_socket *sc) -{ -struct smbdirect_recv_io *ret =NULL ; - -if (!list_empty (&sc->recv_io.reassembly.list)) { - ret =list_first_entry ( - &sc->recv_io.reassembly.list, - struct smbdirect_recv_io, list); - } -return ret; -} -/* Preallocate all receive buffer on transport establishment */ static int allocate_receive_buffers (struct smbdirect_socket *sc,int num_buf) {
@@ -1482,7 +1427,7 @@ void smbd_destroy(struct TCP_Server_Info *server)log_rdma_event (INFO ,"drain the reassembly queue\n" ); do { spin_lock_irqsave(&sc->recv_io.reassembly.lock, flags); - response = _get_first_reassembly(sc); + response = smbdirect_connection_reassembly_first_recv_io(sc); if (response) { list_del(&response->list); spin_unlock_irqrestore(
@@ -1963,7 +1908,7 @@ int smbd_recv(struct smbd_connection *info, struct msghdr *msg)/* * Need to make sure reassembly_data_length is read before * reading reassembly_queue_length and calling -* _get_first_reassembly. This call is lock free +* smbdirect_connection_reassembly_first_recv_io. This call is lock free * as we never read at the end of the queue which are being * updated in SOFTIRQ as more data is received */
@@ -1973,7 +1918,7 @@ int smbd_recv(struct smbd_connection *info, struct msghdr *msg) to_read = size; offset = sc->recv_io.reassembly.first_entry_offset;while (data_read < size) { - response =_get_first_reassembly (sc); + response =smbdirect_connection_reassembly_first_recv_io (sc); data_transfer =smbdirect_recv_io_payload (response); data_length =le32_to_cpu (data_transfer->data_length); remaining_data_length =
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 69/145 smb: client: make use of smbdirect_connection_idle_timer_work()
In reply to Stefan MetzmacherThis is basically a copy of idle_connection_timer().
Note smbdirect_socket_prepare_create() already calls INIT_DELAYED_WORK().
--- fs/smb/client/smbdirect.c | 30 ------------------------------ 1 file changed, 30 deletions(-)
fs/smb/client/smbdirect.c +0 -30
@@ -1342,35 +1342,6 @@ static void send_immediate_empty_message(struct work_struct *work)smbd_post_send_empty (sc); } -/* Implement idle connection timer [MS-SMBD] 3.1.6.2 */ -static void idle_connection_timer (struct work_struct *work) -{ -struct smbdirect_socket *sc = - container_of(work, struct smbdirect_socket, idle.timer_work.work); -struct smbdirect_socket_parameters *sp = &sc->parameters; - -if (sc->idle.keepalive !=SMBDIRECT_KEEPALIVE_NONE ) { -log_keep_alive (ERR , -"error status sc->idle.keepalive=%d\n" , - sc->idle.keepalive); -smbdirect_socket_schedule_cleanup (sc, -ETIMEDOUT ); -return ; - } - -if (sc->status !=SMBDIRECT_SOCKET_CONNECTED ) -return ; - -/* -* Now use the keepalive timeout (instead of keepalive interval) -* in order to wait for a response -*/ - sc->idle.keepalive =SMBDIRECT_KEEPALIVE_PENDING ; -mod_delayed_work (sc->workqueue, &sc->idle.timer_work, -msecs_to_jiffies (sp->keepalive_timeout_msec)); -log_keep_alive (INFO ,"schedule send of empty idle message\n" ); -queue_work (sc->workqueue, &sc->idle.immediate_work); -} -/* * Destroy the transport and related RDMA and memory resources * Need to go through all the pending counters and make sure on one is using
@@ -1774,7 +1745,6 @@ static struct smbd_connection *_smbd_get_connection( }INIT_WORK (&sc->idle.immediate_work, send_immediate_empty_message); -INIT_DELAYED_WORK (&sc->idle.timer_work, idle_connection_timer);/* * start with the negotiate timeout and SMBDIRECT_KEEPALIVE_PENDING * so that the timer will cause a disconnect.
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 70/145 smb: client: make use of smbdirect_frwr_is_supported()
In reply to Stefan MetzmacherThis an exact copy of frwr_is_supported().
--- fs/smb/client/smbdirect.c | 16 +--------------- 1 file changed, 1 insertion(+), 15 deletions(-)
fs/smb/client/smbdirect.c +1 -15
@@ -784,20 +784,6 @@ static struct rdma_cm_id *smbd_create_id(return ERR_PTR (rc); } -/* -* Test if FRWR (Fast Registration Work Requests) is supported on the device -* This implementation requires FRWR on RDMA read/write -* return value: true if it is supported -*/ -static bool frwr_is_supported (struct ib_device_attr *attrs) -{ -if (!(attrs->device_cap_flags &IB_DEVICE_MEM_MGT_EXTENSIONS )) -return false; -if (attrs->max_fast_reg_page_list_len ==0 ) -return false; -return true; -} -static int smbd_ia_open (struct smbdirect_socket *sc,struct sockaddr *dstaddr,int port)
@@ -815,7 +801,7 @@ static int smbd_ia_open( } sc->ib.dev = sc->rdma.cm_id->device; -if (!frwr_is_supported (&sc->ib.dev->attrs)) { +if (!smbdirect_frwr_is_supported (&sc->ib.dev->attrs)) {log_rdma_event (ERR ,"Fast Registration Work Requests (FRWR) is not supported\n" ); log_rdma_event(ERR, "Device capability flags = %llx max_fast_reg_page_list_len = %u\n", sc->ib.dev->attrs.device_cap_flags,
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 71/145 smb: client: make use of smbdirect_connection_{alloc,free}_send_io()
In reply to Stefan MetzmacherThis simplifies the code and allows us to share more code in common with the server.
The only difference is that we use ib_dma_unmap_page() for all sges, this simplifies the logic and doesn't matter as ib_dma_unmap_single() and ib_dma_unmap_page() both operate on dma_addr_t and dma_unmap_single_attrs() is just an alias for dma_unmap_page_attrs().
We already had such an inconsistency before as we called ib_dma_unmap_single(), while we mapped using ib_dma_map_page() in smb_set_sge().
--- fs/smb/client/smbdirect.c | 42 ++++++++++++--------------------------- 1 file changed, 13 insertions(+), 29 deletions(-)
fs/smb/client/smbdirect.c +13 -29
@@ -390,7 +390,6 @@ static inline void *smbdirect_recv_io_payload(struct smbdirect_recv_io *response/* Called when a RDMA send is done */ static void send_done (struct ib_cq *cq,struct ib_wc *wc) { -int i;struct smbdirect_send_io *request = container_of(wc->wr_cqe, struct smbdirect_send_io, cqe);struct smbdirect_socket *sc = request->socket;
@@ -399,12 +398,8 @@ static void send_done(struct ib_cq *cq, struct ib_wc *wc)log_rdma_send (INFO ,"smbdirect_send_io 0x%p completed wc->status=%s\n" , request,ib_wc_status_msg (wc->status)); -for (i =0 ; i < request->num_sge; i++) -ib_dma_unmap_single (sc->ib.dev, - request->sge[i].addr, - request->sge[i].length, -DMA_TO_DEVICE ); -mempool_free (request, sc->send_io.mem.pool); +/* Note this frees wc->wr_cqe, but not wc */ +smbdirect_connection_free_send_io (request); lcredits +=1 ;if (wc->status !=IB_WC_SUCCESS || wc->opcode !=IB_WC_SEND ) {
@@ -836,15 +831,13 @@ static int smbd_post_send_negotiate_req(struct smbdirect_socket *sc) {struct smbdirect_socket_parameters *sp = &sc->parameters;struct ib_send_wr send_wr; -int rc = -ENOMEM ; +int rc;struct smbdirect_send_io *request;struct smbdirect_negotiate_req *packet; - request =mempool_alloc (sc->send_io.mem.pool,GFP_KERNEL ); -if (!request) -return rc; - - request->socket = sc; + request =smbdirect_connection_alloc_send_io (sc); +if (IS_ERR (request)) +return PTR_ERR (request); packet =smbdirect_send_io_payload (request); packet->min_version =cpu_to_le16 (SMBDIRECT_V1 );
@@ -856,7 +849,6 @@ static int smbd_post_send_negotiate_req(struct smbdirect_socket *sc) packet->max_fragmented_size =cpu_to_le32 (sp->max_fragmented_recv_size); - request->num_sge =1 ; request->sge[0 ].addr =ib_dma_map_single ( sc->ib.dev, (void *)packet,sizeof (*packet),DMA_TO_DEVICE );
@@ -867,6 +859,7 @@ static int smbd_post_send_negotiate_req(struct smbdirect_socket *sc) request->sge[0 ].length =sizeof (*packet); request->sge[0 ].lkey = sc->ib.pd->local_dma_lkey; + request->num_sge =1 ; ib_dma_sync_single_for_device( sc->ib.dev, request->sge[0].addr,
@@ -893,13 +886,11 @@ static int smbd_post_send_negotiate_req(struct smbdirect_socket *sc)/* if we reach here, post send failed */ log_rdma_send (ERR ,"ib_post_send failed rc=%d\n" , rc);atomic_dec (&sc->send_io.pending.count); -ib_dma_unmap_single (sc->ib.dev, request->sge[0 ].addr, - request->sge[0 ].length,DMA_TO_DEVICE );smbdirect_socket_schedule_cleanup (sc, rc); dma_mapping_failed: -mempool_free (request, sc->send_io.mem.pool); +smbdirect_connection_free_send_io (request);return rc; }
@@ -997,7 +988,7 @@ static int smbd_post_send_iter(struct smbdirect_socket *sc, int *_remaining_data_length) {struct smbdirect_socket_parameters *sp = &sc->parameters; -int i, rc; +int rc;int header_length;int data_length;struct smbdirect_send_io *request;
@@ -1040,13 +1031,12 @@ static int smbd_post_send_iter(struct smbdirect_socket *sc, goto wait_credit; } - request =mempool_alloc (sc->send_io.mem.pool,GFP_KERNEL ); -if (!request) { - rc = -ENOMEM ; + request =smbdirect_connection_alloc_send_io (sc); +if (IS_ERR (request)) { + rc =PTR_ERR (request); goto err_alloc; } - request->socket = sc;memset (request->sge,0 ,sizeof (request->sge));/* Map the packet to DMA */
@@ -1136,13 +1126,7 @@ static int smbd_post_send_iter(struct smbdirect_socket *sc,wake_up (&sc->send_io.pending.dec_wait_queue); err_dma: -for (i =0 ; i < request->num_sge; i++) -if (request->sge[i].addr) -ib_dma_unmap_single (sc->ib.dev, - request->sge[i].addr, - request->sge[i].length, -DMA_TO_DEVICE ); -mempool_free (request, sc->send_io.mem.pool); +smbdirect_connection_free_send_io (request);/* roll back the granted receive credits */ atomic_sub (new_credits, &sc->recv_io.credits.count);
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 72/145 smb: client: make use of smbdirect_connection_send_io_done()
In reply to Stefan MetzmacherThis handles freeing of siblings too, which is used on the client yet, but that might follow later.
--- fs/smb/client/smbdirect.c | 36 ++---------------------------------- 1 file changed, 2 insertions(+), 34 deletions(-)
fs/smb/client/smbdirect.c +2 -34
@@ -387,38 +387,6 @@ static inline void *smbdirect_recv_io_payload(struct smbdirect_recv_io *responsereturn (void *)response->packet; } -/* Called when a RDMA send is done */ -static void send_done (struct ib_cq *cq,struct ib_wc *wc) -{ -struct smbdirect_send_io *request = - container_of(wc->wr_cqe, struct smbdirect_send_io, cqe); -struct smbdirect_socket *sc = request->socket; -int lcredits =0 ; - -log_rdma_send (INFO ,"smbdirect_send_io 0x%p completed wc->status=%s\n" , - request,ib_wc_status_msg (wc->status)); - -/* Note this frees wc->wr_cqe, but not wc */ -smbdirect_connection_free_send_io (request); - lcredits +=1 ; - -if (wc->status !=IB_WC_SUCCESS || wc->opcode !=IB_WC_SEND ) { -if (wc->status !=IB_WC_WR_FLUSH_ERR ) -log_rdma_send (ERR ,"wc->status=%s wc->opcode=%d\n" , -ib_wc_status_msg (wc->status), wc->opcode); -smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); -return ; - } - -atomic_add (lcredits, &sc->send_io.lcredits.count); -wake_up (&sc->send_io.lcredits.wait_queue); - -if (atomic_dec_and_test (&sc->send_io.pending.count)) -wake_up (&sc->send_io.pending.zero_wait_queue); - -wake_up (&sc->send_io.pending.dec_wait_queue); -} -static void dump_smbdirect_negotiate_resp (struct smbdirect_negotiate_resp *resp) { log_rdma_event(INFO, "resp message min_version %u max_version %u negotiated_version %u credits_requested %u credits_granted %u status %u max_readwrite_size %u preferred_send_size %u max_receive_size %u max_fragmented_size %u\n",
@@ -865,7 +833,7 @@ static int smbd_post_send_negotiate_req(struct smbdirect_socket *sc) sc->ib.dev, request->sge[0 ].addr, request->sge[0].length, DMA_TO_DEVICE); - request->cqe.done = send_done; + request->cqe.done = smbdirect_connection_send_io_done; send_wr.next =NULL ; send_wr.wr_cqe = &request->cqe;
@@ -964,7 +932,7 @@ static int smbd_post_send(struct smbdirect_socket *sc, DMA_TO_DEVICE); } - request->cqe.done = send_done; + request->cqe.done = smbdirect_connection_send_io_done; send_wr.next =NULL ; send_wr.wr_cqe = &request->cqe;
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 73/145 smb: client: make use of smbdirect_connection_{create,destroy}_mem_pools()
In reply to Stefan MetzmacherThe main logical differences are the following:
We now don't use smbdirect_connection_get_recv_io() on cleanup, instead it uses list_for_each_entry_safe()...
For the smbdirect_recv_io payload we expose the whole payload including the smbdirect_data_transfer header as documentation says data_offset = 0 and data_length != 0 would be valid, while the existing client code requires data_offset >= 24.
The smbdirect_send_io cache includes header space for sizeof(struct smbdirect_negotiate_resp) = 32 bytes instead of sizeof(struct smbdirect_data_transfer) = 24 bytes. If this ever becomes a problem, we can allocate separate space for the smbdirect_negotiate_resp in the server.
--- fs/smb/client/smbdirect.c | 131 +------------------------------------- 1 file changed, 3 insertions(+), 128 deletions(-)
fs/smb/client/smbdirect.c +3 -128
@@ -23,9 +23,6 @@ const struct smbdirect_socket_parameters *smbd_get_parameters(struct smbd_connecreturn &sc->parameters; } -static int allocate_receive_buffers (struct smbdirect_socket *sc,int num_buf); -static void destroy_receive_buffers (struct smbdirect_socket *sc); -static int smbd_post_recv (struct smbdirect_socket *sc,struct smbdirect_recv_io *response);
@@ -1228,46 +1225,6 @@ static int smbd_negotiate(struct smbdirect_socket *sc)return rc; } -/* Preallocate all receive buffer on transport establishment */ -static int allocate_receive_buffers (struct smbdirect_socket *sc,int num_buf) -{ -struct smbdirect_recv_io *response; -int i; - -for (i =0 ; i < num_buf; i++) { - response =mempool_alloc (sc->recv_io.mem.pool,GFP_KERNEL ); -if (!response) - goto allocate_failed; - - response->socket = sc; - response->sge.length =0 ; -INIT_WORK (&response->complex_work, __smbdirect_socket_disabled_work); -disable_work_sync (&response->complex_work); -list_add_tail (&response->list, &sc->recv_io.free.list); - } - -return 0 ; - -allocate_failed: -while (!list_empty (&sc->recv_io.free.list)) { - response =list_first_entry ( - &sc->recv_io.free.list, - struct smbdirect_recv_io, list); -list_del (&response->list); - -mempool_free (response, sc->recv_io.mem.pool); - } -return -ENOMEM ; -} - -static void destroy_receive_buffers (struct smbdirect_socket *sc) -{ -struct smbdirect_recv_io *response; - -while ((response =smbdirect_connection_get_recv_io (sc))) -mempool_free (response, sc->recv_io.mem.pool); -} -static void send_immediate_empty_message (struct work_struct *work) { struct smbdirect_socket *sc =
@@ -1348,9 +1305,6 @@ void smbd_destroy(struct TCP_Server_Info *server) }while (response); sc->recv_io.reassembly.data_length =0 ; -log_rdma_event (INFO ,"free receive buffers\n" ); -destroy_receive_buffers (sc); -log_rdma_event (INFO ,"freeing mr list\n" );destroy_mr_list (sc);
@@ -1360,11 +1314,7 @@ void smbd_destroy(struct TCP_Server_Info *server)rdma_destroy_id (sc->rdma.cm_id);/* free mempools */ -mempool_destroy (sc->send_io.mem.pool); -kmem_cache_destroy (sc->send_io.mem.cache); - -mempool_destroy (sc->recv_io.mem.pool); -kmem_cache_destroy (sc->recv_io.mem.cache); +smbdirect_connection_destroy_mem_pools (sc); sc->status =SMBDIRECT_SOCKET_DESTROYED ;
@@ -1410,81 +1360,6 @@ int smbd_reconnect(struct TCP_Server_Info *server)return -ENOENT ; } -static void destroy_caches (struct smbdirect_socket *sc) -{ -destroy_receive_buffers (sc); -mempool_destroy (sc->recv_io.mem.pool); -kmem_cache_destroy (sc->recv_io.mem.cache); -mempool_destroy (sc->send_io.mem.pool); -kmem_cache_destroy (sc->send_io.mem.cache); -} - -#define MAX_NAME_LEN 80 -static int allocate_caches (struct smbdirect_socket *sc) -{ -struct smbdirect_socket_parameters *sp = &sc->parameters; -char name[MAX_NAME_LEN ]; -int rc; - -if (WARN_ON_ONCE (sp->max_recv_size <sizeof (struct smbdirect_data_transfer ))) -return -ENOMEM ; - -scnprintf (name,MAX_NAME_LEN ,"smbdirect_send_io_%p" , sc); - sc->send_io.mem.cache = -kmem_cache_create ( - name, -sizeof (struct smbdirect_send_io ) + -sizeof (struct smbdirect_data_transfer ), -0 ,SLAB_HWCACHE_ALIGN ,NULL ); -if (!sc->send_io.mem.cache) -return -ENOMEM ; - - sc->send_io.mem.pool = -mempool_create (sp->send_credit_target, mempool_alloc_slab, - mempool_free_slab, sc->send_io.mem.cache); -if (!sc->send_io.mem.pool) - goto out1; - -scnprintf (name,MAX_NAME_LEN ,"smbdirect_recv_io_%p" , sc); - -struct kmem_cache_args response_args = { - .align = __alignof__(struct smbdirect_recv_io ), - .useroffset = (offsetof(struct smbdirect_recv_io , packet) + -sizeof (struct smbdirect_data_transfer )), - .usersize = sp->max_recv_size -sizeof (struct smbdirect_data_transfer ), - }; - sc->recv_io.mem.cache = -kmem_cache_create (name, -sizeof (struct smbdirect_recv_io ) + sp->max_recv_size, - &response_args,SLAB_HWCACHE_ALIGN ); -if (!sc->recv_io.mem.cache) - goto out2; - - sc->recv_io.mem.pool = -mempool_create (sp->recv_credit_max, mempool_alloc_slab, - mempool_free_slab, sc->recv_io.mem.cache); -if (!sc->recv_io.mem.pool) - goto out3; - - rc =allocate_receive_buffers (sc, sp->recv_credit_max); -if (rc) { -log_rdma_event (ERR ,"failed to allocate receive buffers\n" ); - goto out4; - } - -return 0 ; - -out4: -mempool_destroy (sc->recv_io.mem.pool); -out3: -kmem_cache_destroy (sc->recv_io.mem.cache); -out2: -mempool_destroy (sc->send_io.mem.pool); -out1: -kmem_cache_destroy (sc->send_io.mem.cache); -return -ENOMEM ; -} -/* Create a SMBD connection, called by upper layer */ static struct smbd_connection *_smbd_get_connection (struct TCP_Server_Info *server,struct sockaddr *dstaddr,int port)
@@ -1676,7 +1551,7 @@ static struct smbd_connection *_smbd_get_connection(log_rdma_event (INFO ,"rdma_connect connected\n" ); - rc =allocate_caches (sc); + rc =smbdirect_connection_create_mem_pools (sc);if (rc) {log_rdma_event (ERR ,"cache allocation failed\n" ); goto allocate_cache_failed;
@@ -1715,7 +1590,7 @@ static struct smbd_connection *_smbd_get_connection( negotiation_failed:disable_delayed_work_sync (&sc->idle.timer_work); -destroy_caches (sc); +smbdirect_connection_destroy_mem_pools (sc); sc->status =SMBDIRECT_SOCKET_NEGOTIATE_FAILED ;rdma_disconnect (sc->rdma.cm_id); wait_event(sc->status_wait,
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 74/145 smb: client: make use of smbdirect_map_sges_from_iter()
In reply to Stefan MetzmacherThis is basically a copy of smb_extract_iter_to_rdma() moved to common code.
Before we had the inconsistency we called ib_dma_unmap_single(), while we mapped using ib_dma_map_page() in smb_set_sge().
Now ib_dma_unmap_page() is used for consistency.
It doesn't really matter as ib_dma_unmap_single() and ib_dma_unmap_page() both operate
on dma_addr_t and dma_unmap_single_attrs() is just an alias for dma_unmap_page_attrs().
--- fs/smb/client/smbdirect.c | 241 +------------------------------------- 1 file changed, 5 insertions(+), 236 deletions(-)
fs/smb/client/smbdirect.c +5 -236
@@ -32,17 +32,6 @@ static int smbd_post_send_empty(struct smbdirect_socket *sc);static void destroy_mr_list (struct smbdirect_socket *sc);static int allocate_mr_list (struct smbdirect_socket *sc); -struct smb_extract_to_rdma { -struct ib_sge *sge; -unsigned int nr_sge; -unsigned int max_sge; -struct ib_device *device; -u32 local_dma_lkey; -enum dma_data_direction direction; -}; -static ssize_t smb_extract_iter_to_rdma (struct iov_iter *iter,size_t len, -struct smb_extract_to_rdma *rdma); -/* Port numbers for SMBD transport */ #define SMB_PORT 445#define SMBD_PORT 5445
@@ -1026,9 +1015,9 @@ static int smbd_post_send_iter(struct smbdirect_socket *sc,/* Fill in the data payload to find out how much data we can add */ if (iter) { -struct smb_extract_to_rdma extract = { - .nr_sge = request->num_sge, - .max_sge =SMBDIRECT_SEND_IO_MAX_SGE , +struct smbdirect_map_sges extract = { + .num_sge = request->num_sge, + .max_sge =ARRAY_SIZE (request->sge), .sge = request->sge, .device = sc->ib.dev, .local_dma_lkey = sc->ib.pd->local_dma_lkey,
@@ -1037,12 +1026,11 @@ static int smbd_post_send_iter(struct smbdirect_socket *sc,size_t payload_len =umin (*_remaining_data_length, sp->max_send_size -sizeof (*packet)); - rc =smb_extract_iter_to_rdma (iter, payload_len, - &extract); + rc =smbdirect_map_sges_from_iter (iter, payload_len, &extract);if (rc <0 ) goto err_dma; data_length = rc; - request->num_sge = extract.nr_sge; + request->num_sge = extract.num_sge; *_remaining_data_length -= data_length; }else { data_length =0 ;
@@ -2398,222 +2386,3 @@ void smbd_deregister_mr(struct smbdirect_mr_io *mr)if (!kref_put (&mr->kref, smbd_mr_free_locked))mutex_unlock (&mr->mutex); } - -static bool smb_set_sge (struct smb_extract_to_rdma *rdma, -struct page *lowest_page,size_t off,size_t len) -{ -struct ib_sge *sge = &rdma->sge[rdma->nr_sge]; -u64 addr; - - addr =ib_dma_map_page (rdma->device, lowest_page, - off, len, rdma->direction); -if (ib_dma_mapping_error (rdma->device, addr)) -return false; - - sge->addr = addr; - sge->length = len; - sge->lkey = rdma->local_dma_lkey; - rdma->nr_sge++; -return true; -} - -/* -* Extract page fragments from a BVEC-class iterator and add them to an RDMA -* element list. The pages are not pinned. -*/ -static ssize_t smb_extract_bvec_to_rdma (struct iov_iter *iter, -struct smb_extract_to_rdma *rdma, -ssize_t maxsize) -{ -const struct bio_vec *bv = iter->bvec; -unsigned long start = iter->iov_offset; -unsigned int i; -ssize_t ret =0 ; - -for (i =0 ; i < iter->nr_segs; i++) { -size_t off, len; - - len = bv[i].bv_len; -if (start >= len) { - start -= len; -continue ; - } - - len =min_t (size_t, maxsize, len - start); - off = bv[i].bv_offset + start; - -if (!smb_set_sge (rdma, bv[i].bv_page, off, len)) -return -EIO ; - - ret += len; - maxsize -= len; -if (rdma->nr_sge >= rdma->max_sge || maxsize <=0 ) -break ; - start =0 ; - } - -if (ret >0 ) -iov_iter_advance (iter, ret); -return ret; -} - -/* -* Extract fragments from a KVEC-class iterator and add them to an RDMA list. -* This can deal with vmalloc'd buffers as well as kmalloc'd or static buffers. -* The pages are not pinned. -*/ -static ssize_t smb_extract_kvec_to_rdma (struct iov_iter *iter, -struct smb_extract_to_rdma *rdma, -ssize_t maxsize) -{ -const struct kvec *kv = iter->kvec; -unsigned long start = iter->iov_offset; -unsigned int i; -ssize_t ret =0 ; - -for (i =0 ; i < iter->nr_segs; i++) { -struct page *page; -unsigned long kaddr; -size_t off, len, seg; - - len = kv[i].iov_len; -if (start >= len) { - start -= len; -continue ; - } - - kaddr = (unsigned long )kv[i].iov_base + start; - off = kaddr & ~PAGE_MASK ; - len =min_t (size_t, maxsize, len - start); - kaddr &=PAGE_MASK ; - - maxsize -= len; -do { - seg =min_t (size_t, len,PAGE_SIZE - off); - -if (is_vmalloc_or_module_addr ((void *)kaddr)) - page =vmalloc_to_page ((void *)kaddr); -else - page =virt_to_page ((void *)kaddr); - -if (!smb_set_sge (rdma, page, off, seg)) -return -EIO ; - - ret += seg; - len -= seg; - kaddr +=PAGE_SIZE ; - off =0 ; - }while (len >0 && rdma->nr_sge < rdma->max_sge); - -if (rdma->nr_sge >= rdma->max_sge || maxsize <=0 ) -break ; - start =0 ; - } - -if (ret >0 ) -iov_iter_advance (iter, ret); -return ret; -} - -/* -* Extract folio fragments from a FOLIOQ-class iterator and add them to an RDMA -* list. The folios are not pinned. -*/ -static ssize_t smb_extract_folioq_to_rdma (struct iov_iter *iter, -struct smb_extract_to_rdma *rdma, -ssize_t maxsize) -{ -const struct folio_queue *folioq = iter->folioq; -unsigned int slot = iter->folioq_slot; -ssize_t ret =0 ; -size_t offset = iter->iov_offset; - -BUG_ON (!folioq); - -if (slot >=folioq_nr_slots (folioq)) { - folioq = folioq->next; -if (WARN_ON_ONCE (!folioq)) -return -EIO ; - slot =0 ; - } - -do { -struct folio *folio =folioq_folio (folioq, slot); -size_t fsize =folioq_folio_size (folioq, slot); - -if (offset < fsize) { -size_t part =umin (maxsize, fsize - offset); - -if (!smb_set_sge (rdma,folio_page (folio,0 ), offset, part)) -return -EIO ; - - offset += part; - ret += part; - maxsize -= part; - } - -if (offset >= fsize) { - offset =0 ; - slot++; -if (slot >=folioq_nr_slots (folioq)) { -if (!folioq->next) { -WARN_ON_ONCE (ret < iter->count); -break ; - } - folioq = folioq->next; - slot =0 ; - } - } - }while (rdma->nr_sge < rdma->max_sge && maxsize >0 ); - - iter->folioq = folioq; - iter->folioq_slot = slot; - iter->iov_offset = offset; - iter->count -= ret; -return ret; -} - -/* -* Extract page fragments from up to the given amount of the source iterator -* and build up an RDMA list that refers to all of those bits. The RDMA list -* is appended to, up to the maximum number of elements set in the parameter -* block. -* -* The extracted page fragments are not pinned or ref'd in any way; if an -* IOVEC/UBUF-type iterator is to be used, it should be converted to a -* BVEC-type iterator and the pages pinned, ref'd or otherwise held in some -* way. -*/ -static ssize_t smb_extract_iter_to_rdma (struct iov_iter *iter,size_t len, -struct smb_extract_to_rdma *rdma) -{ -ssize_t ret; -int before = rdma->nr_sge; - -switch (iov_iter_type (iter)) { -case ITER_BVEC : - ret =smb_extract_bvec_to_rdma (iter, rdma, len); -break ; -case ITER_KVEC : - ret =smb_extract_kvec_to_rdma (iter, rdma, len); -break ; -case ITER_FOLIOQ : - ret =smb_extract_folioq_to_rdma (iter, rdma, len); -break ; -default : -WARN_ON_ONCE (1 ); -return -EIO ; - } - -if (ret <0 ) { -while (rdma->nr_sge > before) { -struct ib_sge *sge = &rdma->sge[rdma->nr_sge--]; - -ib_dma_unmap_single (rdma->device, sge->addr, sge->length, - rdma->direction); - sge->addr =0 ; - } - } - -return ret; -}
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 75/145 smb: client: make use of smbdirect_connection_qp_event_handler()
In reply to Stefan MetzmacherThis is a copy of smbd_qp_async_error_upcall()...
It will allow more code to be moved to common functions soon.
--- fs/smb/client/smbdirect.c | 22 +--------------------- 1 file changed, 1 insertion(+), 21 deletions(-)
fs/smb/client/smbdirect.c +1 -21
@@ -343,26 +343,6 @@ static int smbd_conn_upcall(return 0 ; } -/* Upcall from RDMA QP */ -static void -smbd_qp_async_error_upcall (struct ib_event *event,void *context) -{ -struct smbdirect_socket *sc = context; - -log_rdma_event (ERR ,"%s on device %s socket %p\n" , -ib_event_msg (event->event), event->device->name, sc); - -switch (event->event) { -case IB_EVENT_CQ_ERR : -case IB_EVENT_QP_FATAL : -smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); -break ; - -default : -break ; - } -} -static inline void *smbdirect_send_io_payload (struct smbdirect_send_io *request) {return (void *)request->packet;
@@ -1479,7 +1459,7 @@ static struct smbd_connection *_smbd_get_connection( }memset (&qp_attr,0 ,sizeof (qp_attr)); - qp_attr.event_handler = smbd_qp_async_error_upcall; + qp_attr.event_handler = smbdirect_connection_qp_event_handler; qp_attr.qp_context = sc; qp_attr.cap = qp_cap; qp_attr.sq_sig_type =IB_SIGNAL_REQ_WR ;
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 76/145 smb: client: make use of smbdirect_connection_negotiate_rdma_resources()
In reply to Stefan MetzmacherIt's good to have this logic in a central place, it will allow us share more code soon.
--- fs/smb/client/smbdirect.c | 59 +++------------------------------------ 1 file changed, 4 insertions(+), 55 deletions(-)
fs/smb/client/smbdirect.c +4 -55
@@ -193,7 +193,6 @@ static int smbd_conn_upcall( struct rdma_cm_id *id, struct rdma_cm_event *event) { struct smbdirect_socket *sc = id->context; -struct smbdirect_socket_parameters *sp = &sc->parameters;const char *event_name =rdma_event_msg (event->event);u8 peer_initiator_depth;u8 peer_responder_resources;
@@ -254,60 +253,10 @@ static int smbd_conn_upcall( peer_initiator_depth = event->param.conn.initiator_depth; peer_responder_resources = event->param.conn.responder_resources; } -if (rdma_protocol_iwarp (id->device, id->port_num) && - event->param.conn.private_data_len ==8 ) { -/* -* Legacy clients with only iWarp MPA v1 support -* need a private blob in order to negotiate -* the IRD/ORD values. -*/ -const __be32 *ird_ord_hdr = event->param.conn.private_data; -u32 ird32 =be32_to_cpu (ird_ord_hdr[0 ]); -u32 ord32 =be32_to_cpu (ird_ord_hdr[1 ]); - -/* -* cifs.ko sends the legacy IRD/ORD negotiation -* event if iWarp MPA v2 was used. -* -* Here we check that the values match and only -* mark the client as legacy if they don't match. -*/ -if ((u32 )event->param.conn.initiator_depth != ird32 || - (u32 )event->param.conn.responder_resources != ord32) { -/* -* There are broken clients (old cifs.ko) -* using little endian and also -* struct rdma_conn_param only uses u8 -* for initiator_depth and responder_resources, -* so we truncate the value to U8_MAX. -* -* smb_direct_accept_client() will then -* do the real negotiation in order to -* select the minimum between client and -* server. -*/ - ird32 =min_t (u32, ird32,U8_MAX ); - ord32 =min_t (u32, ord32,U8_MAX ); - - sc->rdma.legacy_iwarp = true; - peer_initiator_depth = (u8 )ird32; - peer_responder_resources = (u8 )ord32; - } - } - -/* -* negotiate the value by using the minimum -* between client and server if the client provided -* non 0 values. -*/ -if (peer_initiator_depth !=0 ) - sp->initiator_depth = -min_t (u8, sp->initiator_depth, - peer_initiator_depth); -if (peer_responder_resources !=0 ) - sp->responder_resources = -min_t (u8, sp->responder_resources, - peer_responder_resources); +smbdirect_connection_negotiate_rdma_resources (sc, + peer_initiator_depth, + peer_responder_resources, + &event->param.conn);if (SMBDIRECT_CHECK_STATUS_DISCONNECT (sc,SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING ))break ;
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 77/145 smb: client: make use of smbdirect_connection_{create,destroy}_qp()
In reply to Stefan MetzmacherIt's good a use common code for this and it will allow us to share more code in the next steps.
Calling ib_drain_qp() twice is ok.
--- fs/smb/client/smbdirect.c | 105 +++----------------------------------- 1 file changed, 7 insertions(+), 98 deletions(-)
fs/smb/client/smbdirect.c +7 -98
@@ -1196,10 +1196,8 @@ void smbd_destroy(struct TCP_Server_Info *server)log_rdma_event (INFO ,"cancelling recv_io.posted.refill_work\n" );disable_work_sync (&sc->recv_io.posted.refill_work); -log_rdma_event (INFO ,"destroying qp\n" ); +log_rdma_event (INFO ,"drain qp\n" );ib_drain_qp (sc->ib.qp); -rdma_destroy_qp (sc->rdma.cm_id); - sc->ib.qp =NULL ;log_rdma_event (INFO ,"cancelling idle timer\n" );disable_delayed_work_sync (&sc->idle.timer_work);
@@ -1225,9 +1223,8 @@ void smbd_destroy(struct TCP_Server_Info *server)log_rdma_event (INFO ,"freeing mr list\n" );destroy_mr_list (sc); -ib_free_cq (sc->ib.send_cq); -ib_free_cq (sc->ib.recv_cq); -ib_dealloc_pd (sc->ib.pd); +log_rdma_event (INFO ,"destroying qp\n" ); +smbdirect_connection_destroy_qp (sc);rdma_destroy_id (sc->rdma.cm_id);/* free mempools */
@@ -1287,8 +1284,6 @@ static struct smbd_connection *_smbd_get_connection(struct smbdirect_socket_parameters init_params = {};struct smbdirect_socket_parameters *sp;struct rdma_conn_param conn_param; -struct ib_qp_cap qp_cap; -struct ib_qp_init_attr qp_attr;struct sockaddr_in *addr_in = (struct sockaddr_in *) dstaddr;struct ib_port_immutable port_immutable;__be32 ird_ord_hdr[2 ];
@@ -1324,6 +1319,7 @@ static struct smbd_connection *_smbd_get_connection( goto create_wq_failed;smbdirect_socket_prepare_create (sc, sp, workqueue);smbdirect_socket_set_logging (sc,NULL , smbd_logging_needed, smbd_logging_vaprintf); + sc->ib.poll_ctx =IB_POLL_SOFTIRQ ;/* * from here we operate on the copy. */
@@ -1335,94 +1331,17 @@ static struct smbd_connection *_smbd_get_connection( goto create_id_failed; } -if (sp->send_credit_target > sc->ib.dev->attrs.max_cqe || - sp->send_credit_target > sc->ib.dev->attrs.max_qp_wr) { -log_rdma_event (ERR ,"consider lowering send_credit_target = %d. Possible CQE overrun, device reporting max_cqe %d max_qp_wr %d\n" , - sp->send_credit_target, - sc->ib.dev->attrs.max_cqe, - sc->ib.dev->attrs.max_qp_wr); - goto config_failed; - } - -if (sp->recv_credit_max > sc->ib.dev->attrs.max_cqe || - sp->recv_credit_max > sc->ib.dev->attrs.max_qp_wr) { -log_rdma_event (ERR ,"consider lowering receive_credit_max = %d. Possible CQE overrun, device reporting max_cqe %d max_qp_wr %d\n" , - sp->recv_credit_max, - sc->ib.dev->attrs.max_cqe, - sc->ib.dev->attrs.max_qp_wr); - goto config_failed; - } - -if (sc->ib.dev->attrs.max_send_sge <SMBDIRECT_SEND_IO_MAX_SGE || - sc->ib.dev->attrs.max_recv_sge <SMBDIRECT_RECV_IO_MAX_SGE ) { -log_rdma_event (ERR , -"device %.*s max_send_sge/max_recv_sge = %d/%d too small\n" , -IB_DEVICE_NAME_MAX , - sc->ib.dev->name, - sc->ib.dev->attrs.max_send_sge, - sc->ib.dev->attrs.max_recv_sge); - goto config_failed; - } - sp->responder_resources =min_t (u8, sp->responder_resources, sc->ib.dev->attrs.max_qp_rd_atom);log_rdma_mr (INFO ,"responder_resources=%d\n" , sp->responder_resources); -/* -* We use allocate sp->responder_resources * 2 MRs -* and each MR needs WRs for REG and INV, so -* we use '* 4'. -* -* +1 for ib_drain_qp() -*/ -memset (&qp_cap,0 ,sizeof (qp_cap)); - qp_cap.max_send_wr = sp->send_credit_target + sp->responder_resources *4 +1 ; - qp_cap.max_recv_wr = sp->recv_credit_max +1 ; - qp_cap.max_send_sge =SMBDIRECT_SEND_IO_MAX_SGE ; - qp_cap.max_recv_sge =SMBDIRECT_RECV_IO_MAX_SGE ; - - sc->ib.pd =ib_alloc_pd (sc->ib.dev,0 ); -if (IS_ERR (sc->ib.pd)) { - rc =PTR_ERR (sc->ib.pd); - sc->ib.pd =NULL ; -log_rdma_event (ERR ,"ib_alloc_pd() returned %d\n" , rc); - goto alloc_pd_failed; - } - - sc->ib.send_cq = -ib_alloc_cq_any (sc->ib.dev, sc, - qp_cap.max_send_wr,IB_POLL_SOFTIRQ ); -if (IS_ERR (sc->ib.send_cq)) { - sc->ib.send_cq =NULL ; - goto alloc_cq_failed; - } - - sc->ib.recv_cq = -ib_alloc_cq_any (sc->ib.dev, sc, - qp_cap.max_recv_wr,IB_POLL_SOFTIRQ ); -if (IS_ERR (sc->ib.recv_cq)) { - sc->ib.recv_cq =NULL ; - goto alloc_cq_failed; - } - -memset (&qp_attr,0 ,sizeof (qp_attr)); - qp_attr.event_handler = smbdirect_connection_qp_event_handler; - qp_attr.qp_context = sc; - qp_attr.cap = qp_cap; - qp_attr.sq_sig_type =IB_SIGNAL_REQ_WR ; - qp_attr.qp_type =IB_QPT_RC ; - qp_attr.send_cq = sc->ib.send_cq; - qp_attr.recv_cq = sc->ib.recv_cq; - qp_attr.port_num = ~0 ; - - rc =rdma_create_qp (sc->rdma.cm_id, sc->ib.pd, &qp_attr); + rc =smbdirect_connection_create_qp (sc);if (rc) { -log_rdma_event (ERR ,"rdma_create_qp failed %i\n" , rc); +log_rdma_event (ERR ,"smbdirect_connection_create_qp failed %i\n" , rc); goto create_qp_failed; } - sc->ib.qp = sc->rdma.cm_id->qp;memset (&conn_param,0 ,sizeof (conn_param)); conn_param.initiator_depth = sp->initiator_depth;
@@ -1515,19 +1434,9 @@ static struct smbd_connection *_smbd_get_connection( allocate_cache_failed: rdma_connect_failed: -rdma_destroy_qp (sc->rdma.cm_id); +smbdirect_connection_destroy_qp (sc); create_qp_failed: -alloc_cq_failed: -if (sc->ib.send_cq) -ib_free_cq (sc->ib.send_cq); -if (sc->ib.recv_cq) -ib_free_cq (sc->ib.recv_cq); - -ib_dealloc_pd (sc->ib.pd); - -alloc_pd_failed: -config_failed:rdma_destroy_id (sc->rdma.cm_id); create_id_failed:
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 78/145 smb: client: initialize recv_io->cqe.done = recv_done just once
In reply to Stefan Metzmachersmbdirect_recv_io structures are pre-allocated so we can set the callback function just once.
This will make it easy to move smbd_post_recv() to common code soon.
--- fs/smb/client/smbdirect.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
fs/smb/client/smbdirect.c +4 -2
@@ -1080,8 +1080,6 @@ static int smbd_post_recv( response->sge.length = sp->max_recv_size; response->sge.lkey = sc->ib.pd->local_dma_lkey; - response->cqe.done = recv_done; - recv_wr.wr_cqe = &response->cqe; recv_wr.next =NULL ; recv_wr.sg_list = &response->sge;
@@ -1289,6 +1287,7 @@ static struct smbd_connection *_smbd_get_connection(__be32 ird_ord_hdr[2 ];char wq_name[80 ];struct workqueue_struct *workqueue; +struct smbdirect_recv_io *recv_io;/* * Create the initial parameters
@@ -1393,6 +1392,9 @@ static struct smbd_connection *_smbd_get_connection( goto allocate_cache_failed; } +list_for_each_entry (recv_io, &sc->recv_io.free.list, list) + recv_io->cqe.done = recv_done; +INIT_WORK (&sc->idle.immediate_work, send_immediate_empty_message);/* * start with the negotiate timeout and SMBDIRECT_KEEPALIVE_PENDING
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 79/145 smb: client: make use of smbdirect_connection_post_recv_io()
In reply to Stefan MetzmacherThe only difference is that smbdirect_connection_post_recv_io() returns early if the connection is already broken.
And that the error code from ib_dma_mapping_error() (currently only -ENOMEM is possible) is returned instead of -EIO.
--- fs/smb/client/smbdirect.c | 46 ++------------------------------------- 1 file changed, 2 insertions(+), 44 deletions(-)
fs/smb/client/smbdirect.c +2 -44
@@ -23,10 +23,6 @@ const struct smbdirect_socket_parameters *smbd_get_parameters(struct smbd_connecreturn &sc->parameters; } -static int smbd_post_recv ( -struct smbdirect_socket *sc, -struct smbdirect_recv_io *response); -static int smbd_post_send_empty (struct smbdirect_socket *sc);static void destroy_mr_list (struct smbdirect_socket *sc);
@@ -403,7 +399,7 @@ static void smbd_post_send_credits(struct work_struct *work)break ; response->first_segment = false; - rc =smbd_post_recv (sc, response); + rc =smbdirect_connection_post_recv_io (response);if (rc) {log_rdma_recv (ERR ,"post_recv failed rc=%d\n" , rc);
@@ -1059,44 +1055,6 @@ static int smbd_post_send_full_iter(struct smbdirect_socket *sc,return rc; } -/* -* Post a receive request to the transport -* The remote peer can only send data when a receive request is posted -* The interaction is controlled by send/receive credit system -*/ -static int smbd_post_recv ( -struct smbdirect_socket *sc,struct smbdirect_recv_io *response) -{ -struct smbdirect_socket_parameters *sp = &sc->parameters; -struct ib_recv_wr recv_wr; -int rc = -EIO ; - - response->sge.addr =ib_dma_map_single ( - sc->ib.dev, response->packet, - sp->max_recv_size,DMA_FROM_DEVICE ); -if (ib_dma_mapping_error (sc->ib.dev, response->sge.addr)) -return rc; - - response->sge.length = sp->max_recv_size; - response->sge.lkey = sc->ib.pd->local_dma_lkey; - - recv_wr.wr_cqe = &response->cqe; - recv_wr.next =NULL ; - recv_wr.sg_list = &response->sge; - recv_wr.num_sge =1 ; - - rc =ib_post_recv (sc->ib.qp, &recv_wr,NULL ); -if (rc) { -ib_dma_unmap_single (sc->ib.dev, response->sge.addr, - response->sge.length,DMA_FROM_DEVICE ); - response->sge.length =0 ; -smbdirect_socket_schedule_cleanup (sc, rc); -log_rdma_recv (ERR ,"ib_post_recv failed rc=%d\n" , rc); - } - -return rc; -} -/* Perform SMBD negotiate according to [MS-SMBD] 3.1.5.2 */ static int smbd_negotiate (struct smbdirect_socket *sc) {
@@ -1108,7 +1066,7 @@ static int smbd_negotiate(struct smbdirect_socket *sc) sc->status =SMBDIRECT_SOCKET_NEGOTIATE_RUNNING ; sc->recv_io.expected =SMBDIRECT_EXPECT_NEGOTIATE_REP ; - rc =smbd_post_recv (sc, response); + rc =smbdirect_connection_post_recv_io (response);log_rdma_event (INFO ,"smbd_post_recv rc=%d iov.addr=0x%llx iov.length=%u iov.lkey=0x%x\n" , rc, response->sge.addr, response->sge.length, response->sge.lkey);
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 80/145 smb: client: make use of smbdirect_connection_recv_io_refill_work()
In reply to Stefan MetzmacherThis is basically a copy of smbd_post_send_credits(), but there are several improvements compared to the existing function:
We calculate the number of missing posted buffers by getting the difference between recv_io.credits.target and recv_io.posted.count.
Instead of the difference between recv_io.credits.target and recv_io.credits.count, because recv_io.credits.count is only updated once a message is send to the peer.
It was not really a problem before, because we have a fixed number smbdirect_recv_io buffers, so the loop terminated when smbdirect_connection_get_recv_io() returns NULL.
But using recv_io.posted.count makes it easier to understand.
--- fs/smb/client/smbdirect.c | 41 +-------------------------------------- 1 file changed, 1 insertion(+), 40 deletions(-)
fs/smb/client/smbdirect.c +1 -40
@@ -380,45 +380,6 @@ static bool process_negotiation_response(return true; } -static void smbd_post_send_credits (struct work_struct *work) -{ -int rc; -struct smbdirect_recv_io *response; -struct smbdirect_socket *sc = - container_of(work, struct smbdirect_socket, recv_io.posted.refill_work); - -if (sc->status !=SMBDIRECT_SOCKET_CONNECTED ) { -return ; - } - -if (sc->recv_io.credits.target > -atomic_read (&sc->recv_io.credits.count)) { -while (true) { - response =smbdirect_connection_get_recv_io (sc); -if (!response) -break ; - - response->first_segment = false; - rc =smbdirect_connection_post_recv_io (response); -if (rc) { -log_rdma_recv (ERR , -"post_recv failed rc=%d\n" , rc); -smbdirect_connection_put_recv_io (response); -break ; - } - -atomic_inc (&sc->recv_io.posted.count); - } - } - -/* Promptly send an immediate packet as defined in [MS-SMBD] 3.1.1.1 */ -if (atomic_read (&sc->recv_io.credits.count) < - sc->recv_io.credits.target -1 ) { -log_keep_alive (INFO ,"schedule send of an empty message\n" ); -queue_work (sc->workqueue, &sc->idle.immediate_work); - } -} -/* Called from softirq, when recv is done */ static void recv_done (struct ib_cq *cq,struct ib_wc *wc) {
@@ -1362,7 +1323,7 @@ static struct smbd_connection *_smbd_get_connection(mod_delayed_work (sc->workqueue, &sc->idle.timer_work,msecs_to_jiffies (sp->negotiate_timeout_msec)); -INIT_WORK (&sc->recv_io.posted.refill_work, smbd_post_send_credits); +INIT_WORK (&sc->recv_io.posted.refill_work, smbdirect_connection_recv_io_refill_work); rc =smbd_negotiate (sc);if (rc) {
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 81/145 smb: client: make use of functions from smbdirect_mr.c
In reply to Stefan MetzmacherThe copied code only got new names, some indentation/formatting changes, some variable names are changed too.
They also only use struct smbdirect_socket instead of struct smbd_connection.
But the logic is still the same.
--- fs/smb/client/smbdirect.c | 496 +------------------------------------- 1 file changed, 4 insertions(+), 492 deletions(-)
fs/smb/client/smbdirect.c +4 -492
@@ -25,9 +25,6 @@ const struct smbdirect_socket_parameters *smbd_get_parameters(struct smbd_connecstatic int smbd_post_send_empty (struct smbdirect_socket *sc); -static void destroy_mr_list (struct smbdirect_socket *sc); -static int allocate_mr_list (struct smbdirect_socket *sc); -/* Port numbers for SMBD transport */ #define SMB_PORT 445#define SMBD_PORT 5445
@@ -1138,7 +1135,7 @@ void smbd_destroy(struct TCP_Server_Info *server) sc->recv_io.reassembly.data_length =0 ;log_rdma_event (INFO ,"freeing mr list\n" ); -destroy_mr_list (sc); +smbdirect_connection_destroy_mr_list (sc);log_rdma_event (INFO ,"destroying qp\n" );smbdirect_connection_destroy_qp (sc);
@@ -1331,7 +1328,7 @@ static struct smbd_connection *_smbd_get_connection( goto negotiation_failed; } - rc =allocate_mr_list (sc); + rc =smbdirect_connection_create_mr_list (sc);if (rc) {log_rdma_mr (ERR ,"memory registration allocation failed\n" ); goto allocate_mr_failed;
@@ -1634,292 +1631,6 @@ int smbd_send(struct TCP_Server_Info *server,return rc; } -static void register_mr_done (struct ib_cq *cq,struct ib_wc *wc) -{ -struct smbdirect_mr_io *mr = - container_of(wc->wr_cqe, struct smbdirect_mr_io, cqe); -struct smbdirect_socket *sc = mr->socket; - -if (wc->status) { -log_rdma_mr (ERR ,"status=%d\n" , wc->status); -smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); - } -} - -/* -* The work queue function that recovers MRs -* We need to call ib_dereg_mr() and ib_alloc_mr() before this MR can be used -* again. Both calls are slow, so finish them in a workqueue. This will not -* block I/O path. -* There is one workqueue that recovers MRs, there is no need to lock as the -* I/O requests calling smbd_register_mr will never update the links in the -* mr_list. -*/ -static void smbd_mr_recovery_work (struct work_struct *work) -{ -struct smbdirect_socket *sc = - container_of(work, struct smbdirect_socket, mr_io.recovery_work); -struct smbdirect_socket_parameters *sp = &sc->parameters; -struct smbdirect_mr_io *smbdirect_mr; -int rc; - -list_for_each_entry (smbdirect_mr, &sc->mr_io.all.list, list) { -if (smbdirect_mr->state ==SMBDIRECT_MR_ERROR ) { - -/* recover this MR entry */ - rc =ib_dereg_mr (smbdirect_mr->mr); -if (rc) { -log_rdma_mr (ERR , -"ib_dereg_mr failed rc=%x\n" , - rc); -smbdirect_socket_schedule_cleanup (sc, rc); -continue ; - } - - smbdirect_mr->mr =ib_alloc_mr ( - sc->ib.pd, sc->mr_io.type, - sp->max_frmr_depth); -if (IS_ERR (smbdirect_mr->mr)) { - rc =PTR_ERR (smbdirect_mr->mr); -log_rdma_mr (ERR ,"ib_alloc_mr failed mr_type=%x max_frmr_depth=%x\n" , - sc->mr_io.type, - sp->max_frmr_depth); -smbdirect_socket_schedule_cleanup (sc, rc); -continue ; - } - }else -/* This MR is being used, don't recover it */ -continue ; - - smbdirect_mr->state =SMBDIRECT_MR_READY ; - -/* smbdirect_mr->state is updated by this function -* and is read and updated by I/O issuing CPUs trying -* to get a MR, the call to atomic_inc_return -* implicates a memory barrier and guarantees this -* value is updated before waking up any calls to -* get_mr() from the I/O issuing CPUs -*/ -if (atomic_inc_return (&sc->mr_io.ready.count) ==1 ) -wake_up (&sc->mr_io.ready.wait_queue); - } -} - -static void smbd_mr_disable_locked (struct smbdirect_mr_io *mr) -{ -struct smbdirect_socket *sc = mr->socket; - -lockdep_assert_held (&mr->mutex); - -if (mr->state ==SMBDIRECT_MR_DISABLED ) -return ; - -if (mr->mr) -ib_dereg_mr (mr->mr); -if (mr->sgt.nents) -ib_dma_unmap_sg (sc->ib.dev, mr->sgt.sgl, mr->sgt.nents, mr->dir); -kfree (mr->sgt.sgl); - - mr->mr =NULL ; - mr->sgt.sgl =NULL ; - mr->sgt.nents =0 ; - - mr->state =SMBDIRECT_MR_DISABLED ; -} - -static void smbd_mr_free_locked (struct kref *kref) -{ -struct smbdirect_mr_io *mr = - container_of(kref, struct smbdirect_mr_io, kref); - -lockdep_assert_held (&mr->mutex); - -/* -* smbd_mr_disable_locked() should already be called! -*/ -if (WARN_ON_ONCE (mr->state !=SMBDIRECT_MR_DISABLED )) -smbd_mr_disable_locked (mr); - -mutex_unlock (&mr->mutex); -mutex_destroy (&mr->mutex); -kfree (mr); -} - -static void destroy_mr_list (struct smbdirect_socket *sc) -{ -struct smbdirect_mr_io *mr, *tmp; -LIST_HEAD (all_list); -unsigned long flags; - -disable_work_sync (&sc->mr_io.recovery_work); - -spin_lock_irqsave (&sc->mr_io.all.lock, flags); -list_splice_tail_init (&sc->mr_io.all.list, &all_list); -spin_unlock_irqrestore (&sc->mr_io.all.lock, flags); - -list_for_each_entry_safe (mr, tmp, &all_list, list) { -mutex_lock (&mr->mutex); - -smbd_mr_disable_locked (mr); -list_del (&mr->list); - mr->socket =NULL ; - -/* -* No kref_put_mutex() as it's already locked. -* -* If smbd_mr_free_locked() is called -* and the mutex is unlocked and mr is gone, -* in that case kref_put() returned 1. -* -* If kref_put() returned 0 we know that -* smbd_mr_free_locked() didn't -* run. Not by us nor by anyone else, as we -* still hold the mutex, so we need to unlock. -* -* If the mr is still registered it will -* be dangling (detached from the connection -* waiting for smbd_deregister_mr() to be -* called in order to free the memory. -*/ -if (!kref_put (&mr->kref, smbd_mr_free_locked)) -mutex_unlock (&mr->mutex); - } -} - -/* -* Allocate MRs used for RDMA read/write -* The number of MRs will not exceed hardware capability in responder_resources -* All MRs are kept in mr_list. The MR can be recovered after it's used -* Recovery is done in smbd_mr_recovery_work. The content of list entry changes -* as MRs are used and recovered for I/O, but the list links will not change -*/ -static int allocate_mr_list (struct smbdirect_socket *sc) -{ -struct smbdirect_socket_parameters *sp = &sc->parameters; -struct smbdirect_mr_io *mr; -int ret; -u32 i; - -if (sp->responder_resources ==0 ) { -log_rdma_mr (ERR ,"responder_resources negotiated as 0\n" ); -return -EINVAL ; - } - -/* Allocate more MRs (2x) than hardware responder_resources */ -for (i =0 ; i < sp->responder_resources *2 ; i++) { - mr =kzalloc (sizeof (*mr),GFP_KERNEL ); -if (!mr) { - ret = -ENOMEM ; - goto kzalloc_mr_failed; - } - -kref_init (&mr->kref); -mutex_init (&mr->mutex); - - mr->mr =ib_alloc_mr (sc->ib.pd, - sc->mr_io.type, - sp->max_frmr_depth); -if (IS_ERR (mr->mr)) { - ret =PTR_ERR (mr->mr); -log_rdma_mr (ERR ,"ib_alloc_mr failed mr_type=%x max_frmr_depth=%x\n" , - sc->mr_io.type, sp->max_frmr_depth); - goto ib_alloc_mr_failed; - } - - mr->sgt.sgl =kcalloc (sp->max_frmr_depth, -sizeof (struct scatterlist ), -GFP_KERNEL ); -if (!mr->sgt.sgl) { - ret = -ENOMEM ; -log_rdma_mr (ERR ,"failed to allocate sgl\n" ); - goto kcalloc_sgl_failed; - } - mr->state =SMBDIRECT_MR_READY ; - mr->socket = sc; - -list_add_tail (&mr->list, &sc->mr_io.all.list); -atomic_inc (&sc->mr_io.ready.count); - } - -INIT_WORK (&sc->mr_io.recovery_work, smbd_mr_recovery_work); - -return 0 ; - -kcalloc_sgl_failed: -ib_dereg_mr (mr->mr); -ib_alloc_mr_failed: -mutex_destroy (&mr->mutex); -kfree (mr); -kzalloc_mr_failed: -destroy_mr_list (sc); -return ret; -} - -/* -* Get a MR from mr_list. This function waits until there is at least one -* MR available in the list. It may access the list while the -* smbd_mr_recovery_work is recovering the MR list. This doesn't need a lock -* as they never modify the same places. However, there may be several CPUs -* issuing I/O trying to get MR at the same time, mr_list_lock is used to -* protect this situation. -*/ -static struct smbdirect_mr_io *get_mr (struct smbdirect_socket *sc) -{ -struct smbdirect_mr_io *ret; -unsigned long flags; -int rc; -again: - rc =wait_event_interruptible (sc->mr_io.ready.wait_queue, -atomic_read (&sc->mr_io.ready.count) || - sc->status !=SMBDIRECT_SOCKET_CONNECTED ); -if (rc) { -log_rdma_mr (ERR ,"wait_event_interruptible rc=%x\n" , rc); -return NULL ; - } - -if (sc->status !=SMBDIRECT_SOCKET_CONNECTED ) { -log_rdma_mr (ERR ,"sc->status=%x\n" , sc->status); -return NULL ; - } - -spin_lock_irqsave (&sc->mr_io.all.lock, flags); -list_for_each_entry (ret, &sc->mr_io.all.list, list) { -if (ret->state ==SMBDIRECT_MR_READY ) { - ret->state =SMBDIRECT_MR_REGISTERED ; -kref_get (&ret->kref); -spin_unlock_irqrestore (&sc->mr_io.all.lock, flags); -atomic_dec (&sc->mr_io.ready.count); -atomic_inc (&sc->mr_io.used.count); -return ret; - } - } - -spin_unlock_irqrestore (&sc->mr_io.all.lock, flags); -/* -* It is possible that we could fail to get MR because other processes may -* try to acquire a MR at the same time. If this is the case, retry it. -*/ - goto again; -} - -/* -* Transcribe the pages from an iterator into an MR scatterlist. -*/ -static int smbd_iter_to_mr (struct iov_iter *iter, -struct sg_table *sgt, -unsigned int max_sg) -{ -int ret; - -memset (sgt->sgl,0 , max_sg *sizeof (struct scatterlist )); - - ret =extract_iter_to_sg (iter,iov_iter_count (iter), sgt, max_sg,0 ); -WARN_ON (ret <0 ); -if (sgt->nents >0 ) -sg_mark_end (&sgt->sgl[sgt->nents -1 ]); -return ret; -} -/* * Register memory for RDMA read/write * iter: the buffer to register memory with
@@ -1932,131 +1643,8 @@ struct smbdirect_mr_io *smbd_register_mr(struct smbd_connection *info, bool writing, bool need_invalidate) {struct smbdirect_socket *sc = &info->socket; -struct smbdirect_socket_parameters *sp = &sc->parameters; -struct smbdirect_mr_io *mr; -int rc, num_pages; -struct ib_reg_wr *reg_wr; - - num_pages =iov_iter_npages (iter, sp->max_frmr_depth +1 ); -if (num_pages > sp->max_frmr_depth) { -log_rdma_mr (ERR ,"num_pages=%d max_frmr_depth=%d\n" , - num_pages, sp->max_frmr_depth); -WARN_ON_ONCE (1 ); -return NULL ; - } - - mr =get_mr (sc); -if (!mr) { -log_rdma_mr (ERR ,"get_mr returning NULL\n" ); -return NULL ; - } - -mutex_lock (&mr->mutex); - - mr->dir = writing ?DMA_FROM_DEVICE :DMA_TO_DEVICE ; - mr->need_invalidate = need_invalidate; - mr->sgt.nents =0 ; - mr->sgt.orig_nents =0 ; - -log_rdma_mr (INFO ,"num_pages=0x%x count=0x%zx depth=%u\n" , - num_pages,iov_iter_count (iter), sp->max_frmr_depth); -smbd_iter_to_mr (iter, &mr->sgt, sp->max_frmr_depth); - - rc =ib_dma_map_sg (sc->ib.dev, mr->sgt.sgl, mr->sgt.nents, mr->dir); -if (!rc) { -log_rdma_mr (ERR ,"ib_dma_map_sg num_pages=%x dir=%x rc=%x\n" , - num_pages, mr->dir, rc); - goto dma_map_error; - } - - rc =ib_map_mr_sg (mr->mr, mr->sgt.sgl, mr->sgt.nents,NULL ,PAGE_SIZE ); -if (rc != mr->sgt.nents) { -log_rdma_mr (ERR , -"ib_map_mr_sg failed rc = %d nents = %x\n" , - rc, mr->sgt.nents); - goto map_mr_error; - } - -ib_update_fast_reg_key (mr->mr,ib_inc_rkey (mr->mr->rkey)); - reg_wr = &mr->wr; - reg_wr->wr.opcode =IB_WR_REG_MR ; - mr->cqe.done = register_mr_done; - reg_wr->wr.wr_cqe = &mr->cqe; - reg_wr->wr.num_sge =0 ; - reg_wr->wr.send_flags =IB_SEND_SIGNALED ; - reg_wr->mr = mr->mr; - reg_wr->key = mr->mr->rkey; - reg_wr->access = writing ? -IB_ACCESS_REMOTE_WRITE |IB_ACCESS_LOCAL_WRITE : -IB_ACCESS_REMOTE_READ ; - -/* -* There is no need for waiting for complemtion on ib_post_send -* on IB_WR_REG_MR. Hardware enforces a barrier and order of execution -* on the next ib_post_send when we actually send I/O to remote peer -*/ - rc =ib_post_send (sc->ib.qp, ®_wr->wr,NULL ); -if (!rc) { -/* -* get_mr() gave us a reference -* via kref_get(&mr->kref), we keep that and let -* the caller use smbd_deregister_mr() -* to remove it again. -*/ -mutex_unlock (&mr->mutex); -return mr; - } - -log_rdma_mr (ERR ,"ib_post_send failed rc=%x reg_wr->key=%x\n" , - rc, reg_wr->key); - -/* If all failed, attempt to recover this MR by setting it SMBDIRECT_MR_ERROR*/ -map_mr_error: -ib_dma_unmap_sg (sc->ib.dev, mr->sgt.sgl, mr->sgt.nents, mr->dir); - -dma_map_error: - mr->sgt.nents =0 ; - mr->state =SMBDIRECT_MR_ERROR ; -if (atomic_dec_and_test (&sc->mr_io.used.count)) -wake_up (&sc->mr_io.cleanup.wait_queue); - -smbdirect_socket_schedule_cleanup (sc, rc); -/* -* get_mr() gave us a reference -* via kref_get(&mr->kref), we need to remove it again -* on error. -* -* No kref_put_mutex() as it's already locked. -* -* If smbd_mr_free_locked() is called -* and the mutex is unlocked and mr is gone, -* in that case kref_put() returned 1. -* -* If kref_put() returned 0 we know that -* smbd_mr_free_locked() didn't -* run. Not by us nor by anyone else, as we -* still hold the mutex, so we need to unlock. -*/ -if (!kref_put (&mr->kref, smbd_mr_free_locked)) -mutex_unlock (&mr->mutex); - -return NULL ; -} - -static void local_inv_done (struct ib_cq *cq,struct ib_wc *wc) -{ -struct smbdirect_mr_io *smbdirect_mr; -struct ib_cqe *cqe; - - cqe = wc->wr_cqe; - smbdirect_mr = container_of(cqe, struct smbdirect_mr_io, cqe); - smbdirect_mr->state =SMBDIRECT_MR_INVALIDATED ; -if (wc->status !=IB_WC_SUCCESS ) { -log_rdma_mr (ERR ,"invalidate failed status=%x\n" , wc->status); - smbdirect_mr->state =SMBDIRECT_MR_ERROR ; - } -complete (&smbdirect_mr->invalidate_done); +return smbdirect_connection_register_mr_io (sc, iter, writing, need_invalidate); }/*
@@ -2067,81 +1655,5 @@ static void local_inv_done(struct ib_cq *cq, struct ib_wc *wc)*/ void smbd_deregister_mr (struct smbdirect_mr_io *mr) { -struct smbdirect_socket *sc = mr->socket; - -mutex_lock (&mr->mutex); -if (mr->state ==SMBDIRECT_MR_DISABLED ) - goto put_kref; - -if (sc->status !=SMBDIRECT_SOCKET_CONNECTED ) { -smbd_mr_disable_locked (mr); - goto put_kref; - } - -if (mr->need_invalidate) { -struct ib_send_wr *wr = &mr->inv_wr; -int rc; - -/* Need to finish local invalidation before returning */ - wr->opcode =IB_WR_LOCAL_INV ; - mr->cqe.done = local_inv_done; - wr->wr_cqe = &mr->cqe; - wr->num_sge =0 ; - wr->ex.invalidate_rkey = mr->mr->rkey; - wr->send_flags =IB_SEND_SIGNALED ; - -init_completion (&mr->invalidate_done); - rc =ib_post_send (sc->ib.qp, wr,NULL ); -if (rc) { -log_rdma_mr (ERR ,"ib_post_send failed rc=%x\n" , rc); -smbd_mr_disable_locked (mr); -smbdirect_socket_schedule_cleanup (sc, rc); - goto done; - } -wait_for_completion (&mr->invalidate_done); - mr->need_invalidate = false; - }else -/* -* For remote invalidation, just set it to SMBDIRECT_MR_INVALIDATED -* and defer to mr_recovery_work to recover the MR for next use -*/ - mr->state =SMBDIRECT_MR_INVALIDATED ; - -if (mr->sgt.nents) { -ib_dma_unmap_sg (sc->ib.dev, mr->sgt.sgl, mr->sgt.nents, mr->dir); - mr->sgt.nents =0 ; - } - -if (mr->state ==SMBDIRECT_MR_INVALIDATED ) { - mr->state =SMBDIRECT_MR_READY ; -if (atomic_inc_return (&sc->mr_io.ready.count) ==1 ) -wake_up (&sc->mr_io.ready.wait_queue); - }else -/* -* Schedule the work to do MR recovery for future I/Os MR -* recovery is slow and don't want it to block current I/O -*/ -queue_work (sc->workqueue, &sc->mr_io.recovery_work); - -done: -if (atomic_dec_and_test (&sc->mr_io.used.count)) -wake_up (&sc->mr_io.cleanup.wait_queue); - -put_kref: -/* -* No kref_put_mutex() as it's already locked. -* -* If smbd_mr_free_locked() is called -* and the mutex is unlocked and mr is gone, -* in that case kref_put() returned 1. -* -* If kref_put() returned 0 we know that -* smbd_mr_free_locked() didn't -* run. Not by us nor by anyone else, as we -* still hold the mutex, so we need to unlock -* and keep the mr in SMBDIRECT_MR_READY or -* SMBDIRECT_MR_ERROR state. -*/ -if (!kref_put (&mr->kref, smbd_mr_free_locked)) -mutex_unlock (&mr->mutex); +smbdirect_connection_deregister_mr_io (mr); }
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 82/145 smb: client: make use of smbdirect_socket_destroy_sync()
In reply to Stefan MetzmacherThis is basically the same logic as before, but we now use common code, which will also be used by the server soon.
--- fs/smb/client/smbdirect.c | 63 +-------------------------------------- 1 file changed, 1 insertion(+), 62 deletions(-)
fs/smb/client/smbdirect.c +1 -62
@@ -1077,8 +1077,6 @@ void smbd_destroy(struct TCP_Server_Info *server) {struct smbd_connection *info = server->smbd_conn;struct smbdirect_socket *sc; -struct smbdirect_recv_io *response; -unsigned long flags;if (!info) {log_rdma_event (INFO ,"rdma session already destroyed\n" );
@@ -1086,68 +1084,9 @@ void smbd_destroy(struct TCP_Server_Info *server) } sc = &info->socket; -log_rdma_event (INFO ,"cancelling and disable disconnect_work\n" ); -disable_work_sync (&sc->disconnect_work); - -log_rdma_event (INFO ,"destroying rdma session\n" ); -if (sc->status <SMBDIRECT_SOCKET_DISCONNECTING ) -smbdirect_socket_cleanup_work (&sc->disconnect_work); -if (sc->status <SMBDIRECT_SOCKET_DISCONNECTED ) { -log_rdma_event (INFO ,"wait for transport being disconnected\n" ); -wait_event (sc->status_wait, sc->status ==SMBDIRECT_SOCKET_DISCONNECTED ); -log_rdma_event (INFO ,"waited for transport being disconnected\n" ); - } - -/* -* Wake up all waiters in all wait queues -* in order to notice the broken connection. -* -* Most likely this was already called via -* smbdirect_socket_cleanup_work(), but call it again... -*/ -smbdirect_socket_wake_up_all (sc); - -log_rdma_event (INFO ,"cancelling recv_io.posted.refill_work\n" ); -disable_work_sync (&sc->recv_io.posted.refill_work); - -log_rdma_event (INFO ,"drain qp\n" ); -ib_drain_qp (sc->ib.qp); - -log_rdma_event (INFO ,"cancelling idle timer\n" ); -disable_delayed_work_sync (&sc->idle.timer_work); -log_rdma_event (INFO ,"cancelling send immediate work\n" ); -disable_work_sync (&sc->idle.immediate_work); - -/* It's not possible for upper layer to get to reassembly */ -log_rdma_event (INFO ,"drain the reassembly queue\n" ); -do { -spin_lock_irqsave (&sc->recv_io.reassembly.lock, flags); - response =smbdirect_connection_reassembly_first_recv_io (sc); -if (response) { -list_del (&response->list); -spin_unlock_irqrestore ( - &sc->recv_io.reassembly.lock, flags); -smbdirect_connection_put_recv_io (response); - }else -spin_unlock_irqrestore ( - &sc->recv_io.reassembly.lock, flags); - }while (response); - sc->recv_io.reassembly.data_length =0 ; - -log_rdma_event (INFO ,"freeing mr list\n" ); -smbdirect_connection_destroy_mr_list (sc); - -log_rdma_event (INFO ,"destroying qp\n" ); -smbdirect_connection_destroy_qp (sc); -rdma_destroy_id (sc->rdma.cm_id); - -/* free mempools */ -smbdirect_connection_destroy_mem_pools (sc); - - sc->status =SMBDIRECT_SOCKET_DESTROYED ; +smbdirect_socket_destroy_sync (sc);destroy_workqueue (sc->workqueue); -log_rdma_event (INFO ,"rdma session destroyed\n" );kfree (info); server->smbd_conn =NULL ; }
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 83/145 smb: client: make use of smbdirect_connection_recvmsg()
In reply to Stefan MetzmacherThis is basically the same as it was copied before...
--- fs/smb/client/smbdirect.c | 131 +------------------------------------- 1 file changed, 1 insertion(+), 130 deletions(-)
fs/smb/client/smbdirect.c +1 -130
@@ -1348,137 +1348,8 @@ struct smbd_connection *smbd_get_connection(int smbd_recv (struct smbd_connection *info,struct msghdr *msg) {struct smbdirect_socket *sc = &info->socket; -struct smbdirect_recv_io *response; -struct smbdirect_data_transfer *data_transfer; -size_t size =iov_iter_count (&msg->msg_iter); -int to_copy, to_read, data_read, offset; -u32 data_length, remaining_data_length, data_offset; -int rc; - -if (WARN_ON_ONCE (iov_iter_rw (&msg->msg_iter) ==WRITE )) -return -EINVAL ;/* It's a bug in upper layer to get there */ - -again: -/* -* No need to hold the reassembly queue lock all the time as we are -* the only one reading from the front of the queue. The transport -* may add more entries to the back of the queue at the same time -*/ -log_read (INFO ,"size=%zd sc->recv_io.reassembly.data_length=%d\n" , size, - sc->recv_io.reassembly.data_length); -if (sc->recv_io.reassembly.data_length >= size) { -int queue_length; -int queue_removed =0 ; -unsigned long flags; - -/* -* Need to make sure reassembly_data_length is read before -* reading reassembly_queue_length and calling -* smbdirect_connection_reassembly_first_recv_io. This call is lock free -* as we never read at the end of the queue which are being -* updated in SOFTIRQ as more data is received -*/ -virt_rmb (); - queue_length = sc->recv_io.reassembly.queue_length; - data_read =0 ; - to_read = size; - offset = sc->recv_io.reassembly.first_entry_offset; -while (data_read < size) { - response =smbdirect_connection_reassembly_first_recv_io (sc); - data_transfer =smbdirect_recv_io_payload (response); - data_length =le32_to_cpu (data_transfer->data_length); - remaining_data_length = -le32_to_cpu ( - data_transfer->remaining_data_length); - data_offset =le32_to_cpu (data_transfer->data_offset); - -/* -* The upper layer expects RFC1002 length at the -* beginning of the payload. Return it to indicate -* the total length of the packet. This minimize the -* change to upper layer packet processing logic. This -* will be eventually remove when an intermediate -* transport layer is added -*/ -if (response->first_segment && size ==4 ) { -unsigned int rfc1002_len = - data_length + remaining_data_length; -__be32 rfc1002_hdr =cpu_to_be32 (rfc1002_len); -if (copy_to_iter (&rfc1002_hdr,sizeof (rfc1002_hdr), - &msg->msg_iter) !=sizeof (rfc1002_hdr)) -return -EFAULT ; - data_read =4 ; - response->first_segment = false; -log_read (INFO ,"returning rfc1002 length %d\n" , - rfc1002_len); - goto read_rfc1002_done; - } - - to_copy =min_t (int, data_length - offset, to_read); -if (copy_to_iter ((char *)data_transfer + data_offset + offset, - to_copy, &msg->msg_iter) != to_copy) -return -EFAULT ; - -/* move on to the next buffer? */ -if (to_copy == data_length - offset) { - queue_length--; -/* -* No need to lock if we are not at the -* end of the queue -*/ -if (queue_length) -list_del (&response->list); -else { -spin_lock_irqsave ( - &sc->recv_io.reassembly.lock, flags); -list_del (&response->list); -spin_unlock_irqrestore ( - &sc->recv_io.reassembly.lock, flags); - } - queue_removed++; - sc->statistics.dequeue_reassembly_queue++; -smbdirect_connection_put_recv_io (response); - offset =0 ; -log_read (INFO ,"smbdirect_connection_put_recv_io offset=0\n" ); - }else - offset += to_copy; - - to_read -= to_copy; - data_read += to_copy; - -log_read (INFO ,"_get_first_reassembly memcpy %d bytes data_transfer_length-offset=%d after that to_read=%d data_read=%d offset=%d\n" , - to_copy, data_length - offset, - to_read, data_read, offset); - } - -spin_lock_irqsave (&sc->recv_io.reassembly.lock, flags); - sc->recv_io.reassembly.data_length -= data_read; - sc->recv_io.reassembly.queue_length -= queue_removed; -spin_unlock_irqrestore (&sc->recv_io.reassembly.lock, flags); - - sc->recv_io.reassembly.first_entry_offset = offset; -log_read (INFO ,"returning to thread data_read=%d reassembly_data_length=%d first_entry_offset=%d\n" , - data_read, sc->recv_io.reassembly.data_length, - sc->recv_io.reassembly.first_entry_offset); -read_rfc1002_done: -return data_read; - } - -log_read (INFO ,"wait_event on more data\n" ); - rc =wait_event_interruptible ( - sc->recv_io.reassembly.wait_queue, - sc->recv_io.reassembly.data_length >= size || - sc->status !=SMBDIRECT_SOCKET_CONNECTED ); -/* Don't return any data if interrupted */ -if (rc) -return rc; - -if (sc->status !=SMBDIRECT_SOCKET_CONNECTED ) { -log_read (ERR ,"disconnected\n" ); -return -ECONNABORTED ; - } - goto again; +return smbdirect_connection_recvmsg (sc, msg,0 ); }/*
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 84/145 smb: client: let smbd_post_send() make use of request->wr
In reply to Stefan MetzmacherWe don't need a stack variable in addition.
--- fs/smb/client/smbdirect.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-)
fs/smb/client/smbdirect.c +7 -8
@@ -787,7 +787,6 @@ static int manage_keep_alive_before_sending(struct smbdirect_socket *sc)static int smbd_post_send (struct smbdirect_socket *sc,struct smbdirect_send_io *request) { -struct ib_send_wr send_wr;int rc, i;for (i =0 ; i < request->num_sge; i++) {
@@ -803,14 +802,14 @@ static int smbd_post_send(struct smbdirect_socket *sc, request->cqe.done = smbdirect_connection_send_io_done; - send_wr.next =NULL ; - send_wr.wr_cqe = &request->cqe; - send_wr.sg_list = request->sge; - send_wr.num_sge = request->num_sge; - send_wr.opcode =IB_WR_SEND ; - send_wr.send_flags =IB_SEND_SIGNALED ; + request->wr.next =NULL ; + request->wr.wr_cqe = &request->cqe; + request->wr.sg_list = request->sge; + request->wr.num_sge = request->num_sge; + request->wr.opcode =IB_WR_SEND ; + request->wr.send_flags =IB_SEND_SIGNALED ; - rc =ib_post_send (sc->ib.qp, &send_wr,NULL ); + rc =ib_post_send (sc->ib.qp, &request->wr,NULL );if (rc) {log_rdma_send (ERR ,"ib_post_send failed rc=%d\n" , rc);smbdirect_socket_schedule_cleanup (sc, rc);
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 85/145 smb: client: make use of smbdirect_connection_grant_recv_credits()
In reply to Stefan MetzmacherThis already calls atomic_add(new_credits, &sc->recv_io.credits.count), so there's no need to do it in the caller anymore.
--- fs/smb/client/smbdirect.c | 31 ++----------------------------- 1 file changed, 2 insertions(+), 29 deletions(-)
fs/smb/client/smbdirect.c +2 -29
@@ -731,32 +731,6 @@ static int smbd_post_send_negotiate_req(struct smbdirect_socket *sc)return rc; } -/* -* Extend the credits to remote peer -* This implements [MS-SMBD] 3.1.5.9 -* The idea is that we should extend credits to remote peer as quickly as -* it's allowed, to maintain data flow. We allocate as much receive -* buffer as possible, and extend the receive credits to remote peer -* return value: the new credtis being granted. -*/ -static int manage_credits_prior_sending (struct smbdirect_socket *sc) -{ -int new_credits; - -if (atomic_read (&sc->recv_io.credits.count) >= sc->recv_io.credits.target) -return 0 ; - - new_credits =atomic_read (&sc->recv_io.posted.count); -if (new_credits ==0 ) -return 0 ; - - new_credits -=atomic_read (&sc->recv_io.credits.count); -if (new_credits <=0 ) -return 0 ; - -return new_credits; -} -/* * Check if we need to send a KEEP_ALIVE message * The idle connection timer triggers a KEEP_ALIVE message when expires
@@ -829,7 +803,7 @@ static int smbd_post_send_iter(struct smbdirect_socket *sc,int data_length;struct smbdirect_send_io *request;struct smbdirect_data_transfer *packet; -int new_credits =0 ; +u16 new_credits =0 ; wait_lcredit:/* Wait for local send credits */
@@ -921,8 +895,7 @@ static int smbd_post_send_iter(struct smbdirect_socket *sc,/* Fill in the packet header */ packet->credits_requested =cpu_to_le16 (sp->send_credit_target); - new_credits =manage_credits_prior_sending (sc); -atomic_add (new_credits, &sc->recv_io.credits.count); + new_credits =smbdirect_connection_grant_recv_credits (sc); packet->credits_granted =cpu_to_le16 (new_credits); packet->flags =0 ;
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 86/145 smb: client: make use of smbdirect_connection_request_keep_alive()
In reply to Stefan MetzmacherThis will help to share more common code soon.
--- fs/smb/client/smbdirect.c | 28 +--------------------------- 1 file changed, 1 insertion(+), 27 deletions(-)
fs/smb/client/smbdirect.c +1 -27
@@ -731,32 +731,6 @@ static int smbd_post_send_negotiate_req(struct smbdirect_socket *sc)return rc; } -/* -* Check if we need to send a KEEP_ALIVE message -* The idle connection timer triggers a KEEP_ALIVE message when expires -* SMBDIRECT_FLAG_RESPONSE_REQUESTED is set in the message flag to have peer send -* back a response. -* return value: -* 1 if SMBDIRECT_FLAG_RESPONSE_REQUESTED needs to be set -* 0: otherwise -*/ -static int manage_keep_alive_before_sending (struct smbdirect_socket *sc) -{ -struct smbdirect_socket_parameters *sp = &sc->parameters; - -if (sc->idle.keepalive ==SMBDIRECT_KEEPALIVE_PENDING ) { - sc->idle.keepalive =SMBDIRECT_KEEPALIVE_SENT ; -/* -* Now use the keepalive timeout (instead of keepalive interval) -* in order to wait for a response -*/ -mod_delayed_work (sc->workqueue, &sc->idle.timer_work, -msecs_to_jiffies (sp->keepalive_timeout_msec)); -return 1 ; - } -return 0 ; -} -/* Post the send request */ static int smbd_post_send (struct smbdirect_socket *sc,struct smbdirect_send_io *request)
@@ -899,7 +873,7 @@ static int smbd_post_send_iter(struct smbdirect_socket *sc, packet->credits_granted =cpu_to_le16 (new_credits); packet->flags =0 ; -if (manage_keep_alive_before_sending (sc)) +if (smbdirect_connection_request_keep_alive (sc)) packet->flags |=cpu_to_le16 (SMBDIRECT_FLAG_RESPONSE_REQUESTED ); packet->reserved =0 ;
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 87/145 smb: client: change smbd_post_send_empty() to void return
In reply to Stefan MetzmacherThe caller doesn't check, so we better call smbdirect_socket_schedule_cleanup() to handle the error.
--- fs/smb/client/smbdirect.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
fs/smb/client/smbdirect.c +7 -4
@@ -23,8 +23,6 @@ const struct smbdirect_socket_parameters *smbd_get_parameters(struct smbd_connecreturn &sc->parameters; } -static int smbd_post_send_empty (struct smbdirect_socket *sc); -/* Port numbers for SMBD transport */ #define SMB_PORT 445#define SMBD_PORT 5445
@@ -930,12 +928,17 @@ static int smbd_post_send_iter(struct smbdirect_socket *sc,* Empty message is used to extend credits to peer to for keep live * while there is no upper layer payload to send at the time */ -static int smbd_post_send_empty (struct smbdirect_socket *sc) +static void smbd_post_send_empty (struct smbdirect_socket *sc) {int remaining_data_length =0 ; +int ret; sc->statistics.send_empty++; -return smbd_post_send_iter (sc,NULL , &remaining_data_length); + ret =smbd_post_send_iter (sc,NULL , &remaining_data_length); +if (ret <0 ) { +log_rdma_send (ERR ,"smbd_post_send_iter failed ret=%d\n" , ret); +smbdirect_socket_schedule_cleanup (sc, ret); + } } static int smbd_post_send_full_iter(struct smbdirect_socket *sc,
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 88/145 smb: client: let smbd_post_send_iter() get remaining_length and return data_length
In reply to Stefan MetzmacherThis lets the logic be like smb_direct_post_send_data(), so we can share common code in the next steps.
--- fs/smb/client/smbdirect.c | 35 +++++++++++++++++++++-------------- 1 file changed, 21 insertions(+), 14 deletions(-)
fs/smb/client/smbdirect.c +21 -14
@@ -767,7 +767,7 @@ static int smbd_post_send(struct smbdirect_socket *sc,static int smbd_post_send_iter (struct smbdirect_socket *sc,struct iov_iter *iter, -int *_remaining_data_length) +u32 remaining_data_length) {struct smbdirect_socket_parameters *sp = &sc->parameters;int rc;
@@ -777,6 +777,18 @@ static int smbd_post_send_iter(struct smbdirect_socket *sc,struct smbdirect_data_transfer *packet;u16 new_credits =0 ; +if (iter) { + header_length =sizeof (struct smbdirect_data_transfer ); +if (WARN_ON_ONCE (remaining_data_length ==0 || +iov_iter_count (iter) > remaining_data_length)) +return -EINVAL ; + }else { +/* If this is a packet without payload, don't send padding */ + header_length = offsetof(struct smbdirect_data_transfer , padding); +if (WARN_ON_ONCE (remaining_data_length)) +return -EINVAL ; + } + wait_lcredit:/* Wait for local send credits */ rc = wait_event_interruptible(sc->send_io.lcredits.wait_queue,
@@ -821,12 +833,6 @@ static int smbd_post_send_iter(struct smbdirect_socket *sc,memset (request->sge,0 ,sizeof (request->sge)); -/* Map the packet to DMA */ - header_length =sizeof (struct smbdirect_data_transfer ); -/* If this is a packet without payload, don't send padding */ -if (!iter) - header_length = offsetof(struct smbdirect_data_transfer , padding); - packet =smbdirect_send_io_payload (request); request->sge[0].addr = ib_dma_map_single(sc->ib.dev, (void *)packet,
@@ -851,7 +857,7 @@ static int smbd_post_send_iter(struct smbdirect_socket *sc, .local_dma_lkey = sc->ib.pd->local_dma_lkey, .direction = DMA_TO_DEVICE, }; -size_t payload_len =umin (*_remaining_data_length, +size_t payload_len =umin (iov_iter_count (iter), sp->max_send_size -sizeof (*packet)); rc =smbdirect_map_sges_from_iter (iter, payload_len, &extract);
@@ -859,7 +865,7 @@ static int smbd_post_send_iter(struct smbdirect_socket *sc, goto err_dma; data_length = rc; request->num_sge = extract.num_sge; - *_remaining_data_length -= data_length; + remaining_data_length -= data_length; }else { data_length =0 ; }
@@ -880,7 +886,7 @@ static int smbd_post_send_iter(struct smbdirect_socket *sc,else packet->data_offset =cpu_to_le32 (24 ); packet->data_length =cpu_to_le32 (data_length); - packet->remaining_data_length =cpu_to_le32 (*_remaining_data_length); + packet->remaining_data_length =cpu_to_le32 (remaining_data_length); packet->padding =0 ; log_outgoing(INFO, "credits_requested=%d credits_granted=%d data_offset=%d data_length=%d remaining_data_length=%d\n",
@@ -898,7 +904,7 @@ static int smbd_post_send_iter(struct smbdirect_socket *sc, rc =smbd_post_send (sc, request);if (!rc) -return 0 ; +return data_length;if (atomic_dec_and_test (&sc->send_io.pending.count))wake_up (&sc->send_io.pending.zero_wait_queue);
@@ -930,11 +936,10 @@ static int smbd_post_send_iter(struct smbdirect_socket *sc,*/ static void smbd_post_send_empty (struct smbdirect_socket *sc) { -int remaining_data_length =0 ;int ret; sc->statistics.send_empty++; - ret =smbd_post_send_iter (sc,NULL , &remaining_data_length); + ret =smbd_post_send_iter (sc,NULL ,0 );if (ret <0 ) {log_rdma_send (ERR ,"smbd_post_send_iter failed ret=%d\n" , ret);smbdirect_socket_schedule_cleanup (sc, ret);
@@ -954,9 +959,11 @@ static int smbd_post_send_full_iter(struct smbdirect_socket *sc,*/ while (iov_iter_count (iter) >0 ) { - rc =smbd_post_send_iter (sc, iter, _remaining_data_length); + rc =smbd_post_send_iter (sc, iter, *_remaining_data_length);if (rc <0 )break ; + *_remaining_data_length -= rc; + rc =0 ; }return rc;
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 89/145 smb: client: let smbd_post_send_full_iter() get remaining_length and return data_length
In reply to Stefan MetzmacherThis will simplify further changes in order to share more common code in future.
--- fs/smb/client/smbdirect.c | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-)
fs/smb/client/smbdirect.c +15 -9
@@ -948,9 +948,9 @@ static void smbd_post_send_empty(struct smbdirect_socket *sc)static int smbd_post_send_full_iter (struct smbdirect_socket *sc,struct iov_iter *iter, -int *_remaining_data_length) +u32 remaining_data_length) { -int rc =0 ; +int bytes =0 ;/* * smbd_post_send_iter() respects the
@@ -959,14 +959,16 @@ static int smbd_post_send_full_iter(struct smbdirect_socket *sc,*/ while (iov_iter_count (iter) >0 ) { - rc =smbd_post_send_iter (sc, iter, *_remaining_data_length); +int rc; + + rc =smbd_post_send_iter (sc, iter, remaining_data_length);if (rc <0 ) -break ; - *_remaining_data_length -= rc; - rc =0 ; +return rc; + remaining_data_length -= rc; + bytes += rc; } -return rc; +return bytes; }/* Perform SMBD negotiate according to [MS-SMBD] 3.1.5.2 */
@@ -1366,16 +1368,20 @@ int smbd_send(struct TCP_Server_Info *server, klen += rqst->rq_iov[i].iov_len;iov_iter_kvec (&iter,ITER_SOURCE , rqst->rq_iov, rqst->rq_nvec, klen); - rc =smbd_post_send_full_iter (sc, &iter, &remaining_data_length); + rc =smbd_post_send_full_iter (sc, &iter, remaining_data_length);if (rc <0 )break ; + remaining_data_length -= rc; + rc =0 ;if (iov_iter_count (&rqst->rq_iter) >0 ) {/* And then the data pages if there are any */ rc =smbd_post_send_full_iter (sc, &rqst->rq_iter, - &remaining_data_length); + remaining_data_length);if (rc <0 )break ; + remaining_data_length -= rc; + rc =0 ; } }while (++rqst_idx < num_rqst);
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 90/145 smb: client: make use of smbdirect_connection_send_{single_iter,immediate_work}()
In reply to Stefan MetzmacherWith this the low level send functions are in common, we'll have to do some more changes in generic smb code in order to use smbdirect_connection_send_iter() instead of looping around smbdirect_connection_send_single_iter().
David's cleanups will allow us to usesmbdirect_connection_send_iter().
--- fs/smb/client/smbdirect.c | 253 +++----------------------------------- 1 file changed, 15 insertions(+), 238 deletions(-)
fs/smb/client/smbdirect.c +15 -238
@@ -729,223 +729,6 @@ static int smbd_post_send_negotiate_req(struct smbdirect_socket *sc)return rc; } -/* Post the send request */ -static int smbd_post_send (struct smbdirect_socket *sc, -struct smbdirect_send_io *request) -{ -int rc, i; - -for (i =0 ; i < request->num_sge; i++) { -log_rdma_send (INFO , -"rdma_request sge[%d] addr=0x%llx length=%u\n" , - i, request->sge[i].addr, request->sge[i].length); -ib_dma_sync_single_for_device ( - sc->ib.dev, - request->sge[i].addr, - request->sge[i].length, -DMA_TO_DEVICE ); - } - - request->cqe.done = smbdirect_connection_send_io_done; - - request->wr.next =NULL ; - request->wr.wr_cqe = &request->cqe; - request->wr.sg_list = request->sge; - request->wr.num_sge = request->num_sge; - request->wr.opcode =IB_WR_SEND ; - request->wr.send_flags =IB_SEND_SIGNALED ; - - rc =ib_post_send (sc->ib.qp, &request->wr,NULL ); -if (rc) { -log_rdma_send (ERR ,"ib_post_send failed rc=%d\n" , rc); -smbdirect_socket_schedule_cleanup (sc, rc); - rc = -EAGAIN ; - } - -return rc; -} - -static int smbd_post_send_iter (struct smbdirect_socket *sc, -struct iov_iter *iter, -u32 remaining_data_length) -{ -struct smbdirect_socket_parameters *sp = &sc->parameters; -int rc; -int header_length; -int data_length; -struct smbdirect_send_io *request; -struct smbdirect_data_transfer *packet; -u16 new_credits =0 ; - -if (iter) { - header_length =sizeof (struct smbdirect_data_transfer ); -if (WARN_ON_ONCE (remaining_data_length ==0 || -iov_iter_count (iter) > remaining_data_length)) -return -EINVAL ; - }else { -/* If this is a packet without payload, don't send padding */ - header_length = offsetof(struct smbdirect_data_transfer , padding); -if (WARN_ON_ONCE (remaining_data_length)) -return -EINVAL ; - } - -wait_lcredit: -/* Wait for local send credits */ - rc =wait_event_interruptible (sc->send_io.lcredits.wait_queue, -atomic_read (&sc->send_io.lcredits.count) >0 || - sc->status !=SMBDIRECT_SOCKET_CONNECTED ); -if (rc) - goto err_wait_lcredit; - -if (sc->status !=SMBDIRECT_SOCKET_CONNECTED ) { -log_outgoing (ERR ,"disconnected not sending on wait_credit\n" ); - rc = -EAGAIN ; - goto err_wait_lcredit; - } -if (unlikely (atomic_dec_return (&sc->send_io.lcredits.count) <0 )) { -atomic_inc (&sc->send_io.lcredits.count); - goto wait_lcredit; - } - -wait_credit: -/* Wait for send credits. A SMBD packet needs one credit */ - rc =wait_event_interruptible (sc->send_io.credits.wait_queue, -atomic_read (&sc->send_io.credits.count) >0 || - sc->status !=SMBDIRECT_SOCKET_CONNECTED ); -if (rc) - goto err_wait_credit; - -if (sc->status !=SMBDIRECT_SOCKET_CONNECTED ) { -log_outgoing (ERR ,"disconnected not sending on wait_credit\n" ); - rc = -EAGAIN ; - goto err_wait_credit; - } -if (unlikely (atomic_dec_return (&sc->send_io.credits.count) <0 )) { -atomic_inc (&sc->send_io.credits.count); - goto wait_credit; - } - - request =smbdirect_connection_alloc_send_io (sc); -if (IS_ERR (request)) { - rc =PTR_ERR (request); - goto err_alloc; - } - -memset (request->sge,0 ,sizeof (request->sge)); - - packet =smbdirect_send_io_payload (request); - request->sge[0 ].addr =ib_dma_map_single (sc->ib.dev, - (void *)packet, - header_length, -DMA_TO_DEVICE ); -if (ib_dma_mapping_error (sc->ib.dev, request->sge[0 ].addr)) { - rc = -EIO ; - goto err_dma; - } - - request->sge[0 ].length = header_length; - request->sge[0 ].lkey = sc->ib.pd->local_dma_lkey; - request->num_sge =1 ; - -/* Fill in the data payload to find out how much data we can add */ -if (iter) { -struct smbdirect_map_sges extract = { - .num_sge = request->num_sge, - .max_sge =ARRAY_SIZE (request->sge), - .sge = request->sge, - .device = sc->ib.dev, - .local_dma_lkey = sc->ib.pd->local_dma_lkey, - .direction =DMA_TO_DEVICE , - }; -size_t payload_len =umin (iov_iter_count (iter), - sp->max_send_size -sizeof (*packet)); - - rc =smbdirect_map_sges_from_iter (iter, payload_len, &extract); -if (rc <0 ) - goto err_dma; - data_length = rc; - request->num_sge = extract.num_sge; - remaining_data_length -= data_length; - }else { - data_length =0 ; - } - -/* Fill in the packet header */ - packet->credits_requested =cpu_to_le16 (sp->send_credit_target); - - new_credits =smbdirect_connection_grant_recv_credits (sc); - packet->credits_granted =cpu_to_le16 (new_credits); - - packet->flags =0 ; -if (smbdirect_connection_request_keep_alive (sc)) - packet->flags |=cpu_to_le16 (SMBDIRECT_FLAG_RESPONSE_REQUESTED ); - - packet->reserved =0 ; -if (!data_length) - packet->data_offset =0 ; -else - packet->data_offset =cpu_to_le32 (24 ); - packet->data_length =cpu_to_le32 (data_length); - packet->remaining_data_length =cpu_to_le32 (remaining_data_length); - packet->padding =0 ; - -log_outgoing (INFO ,"credits_requested=%d credits_granted=%d data_offset=%d data_length=%d remaining_data_length=%d\n" , -le16_to_cpu (packet->credits_requested), -le16_to_cpu (packet->credits_granted), -le32_to_cpu (packet->data_offset), -le32_to_cpu (packet->data_length), -le32_to_cpu (packet->remaining_data_length)); - -/* -* Now that we got a local and a remote credit -* we add us as pending -*/ -atomic_inc (&sc->send_io.pending.count); - - rc =smbd_post_send (sc, request); -if (!rc) -return data_length; - -if (atomic_dec_and_test (&sc->send_io.pending.count)) -wake_up (&sc->send_io.pending.zero_wait_queue); - -wake_up (&sc->send_io.pending.dec_wait_queue); - -err_dma: -smbdirect_connection_free_send_io (request); - -/* roll back the granted receive credits */ -atomic_sub (new_credits, &sc->recv_io.credits.count); - -err_alloc: -atomic_inc (&sc->send_io.credits.count); -wake_up (&sc->send_io.credits.wait_queue); - -err_wait_credit: -atomic_inc (&sc->send_io.lcredits.count); -wake_up (&sc->send_io.lcredits.wait_queue); - -err_wait_lcredit: -return rc; -} - -/* -* Send an empty message -* Empty message is used to extend credits to peer to for keep live -* while there is no upper layer payload to send at the time -*/ -static void smbd_post_send_empty (struct smbdirect_socket *sc) -{ -int ret; - - sc->statistics.send_empty++; - ret =smbd_post_send_iter (sc,NULL ,0 ); -if (ret <0 ) { -log_rdma_send (ERR ,"smbd_post_send_iter failed ret=%d\n" , ret); -smbdirect_socket_schedule_cleanup (sc, ret); - } -} -static int smbd_post_send_full_iter (struct smbdirect_socket *sc,struct iov_iter *iter,u32 remaining_data_length)
@@ -953,7 +736,7 @@ static int smbd_post_send_full_iter(struct smbdirect_socket *sc,int bytes =0 ;/* -* smbd_post_send_iter() respects the +* smbdirect_connection_send_single_iter() respects the * negotiated max_send_size, so we need to * loop until the full iter is posted */
@@ -961,7 +744,11 @@ static int smbd_post_send_full_iter(struct smbdirect_socket *sc,while (iov_iter_count (iter) >0 ) {int rc; - rc =smbd_post_send_iter (sc, iter, remaining_data_length); + rc =smbdirect_connection_send_single_iter (sc, +NULL ,/* batch */ + iter, +0 ,/* flags */ + remaining_data_length);if (rc <0 )return rc; remaining_data_length -= rc;
@@ -1014,18 +801,6 @@ static int smbd_negotiate(struct smbdirect_socket *sc)return rc; } -static void send_immediate_empty_message (struct work_struct *work) -{ -struct smbdirect_socket *sc = - container_of(work, struct smbdirect_socket, idle.immediate_work); - -if (sc->status !=SMBDIRECT_SOCKET_CONNECTED ) -return ; - -log_keep_alive (INFO ,"send an empty message\n" ); -smbd_post_send_empty (sc); -} -/* * Destroy the transport and related RDMA and memory resources * Need to go through all the pending counters and make sure on one is using
@@ -1208,7 +983,7 @@ static struct smbd_connection *_smbd_get_connection(list_for_each_entry (recv_io, &sc->recv_io.free.list, list) recv_io->cqe.done = recv_done; -INIT_WORK (&sc->idle.immediate_work, send_immediate_empty_message); +INIT_WORK (&sc->idle.immediate_work, smbdirect_connection_send_immediate_work);/* * start with the negotiate timeout and SMBDIRECT_KEEPALIVE_PENDING * so that the timer will cause a disconnect.
@@ -1326,6 +1101,7 @@ int smbd_send(struct TCP_Server_Info *server,struct iov_iter iter;unsigned int remaining_data_length, klen;int rc, i, rqst_idx; +int error =0 ;if (sc->status !=SMBDIRECT_SOCKET_CONNECTED )return -EAGAIN ;
@@ -1393,14 +1169,15 @@ int smbd_send(struct TCP_Server_Info *server,* that means all the I/Os have been out and we are good to return */ -wait_event (sc->send_io.pending.zero_wait_queue, -atomic_read (&sc->send_io.pending.count) ==0 || - sc->status !=SMBDIRECT_SOCKET_CONNECTED ); + error = rc; + rc =smbdirect_connection_send_wait_zero_pending (sc); +if (unlikely (rc && !error)) + error = -EAGAIN ; -if (sc->status !=SMBDIRECT_SOCKET_CONNECTED && rc ==0 ) - rc = -EAGAIN ; +if (unlikely (error)) +return error; -return rc; +return 0 ; }/*
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 91/145 smb: client: introduce and use smbd_mr_fill_buffer_descriptor()
In reply to Stefan MetzmacherThis will allow us to make struct smbdirect_mr_io private in future.
--- fs/smb/client/smb2pdu.c | 8 ++------ fs/smb/client/smbdirect.c | 6 ++++++ fs/smb/client/smbdirect.h | 2 ++ 3 files changed, 10 insertions(+), 6 deletions(-)
fs/smb/client/smb2pdu.c +2 -6
@@ -4518,9 +4518,7 @@ smb2_new_read_req(void **buf, unsigned int *total_len, req->ReadChannelInfoLength =cpu_to_le16 (sizeof (struct smbdirect_buffer_descriptor_v1 )); v1 = (struct smbdirect_buffer_descriptor_v1 *) &req->Buffer[0 ]; - v1->offset =cpu_to_le64 (rdata->mr->mr->iova); - v1->token =cpu_to_le32 (rdata->mr->mr->rkey); - v1->length =cpu_to_le32 (rdata->mr->mr->length); +smbd_mr_fill_buffer_descriptor (rdata->mr, v1); *total_len +=sizeof (*v1) -1 ; }
@@ -5068,9 +5066,7 @@ smb2_async_writev(struct cifs_io_subrequest *wdata) req->WriteChannelInfoLength =cpu_to_le16 (sizeof (struct smbdirect_buffer_descriptor_v1 )); v1 = (struct smbdirect_buffer_descriptor_v1 *) &req->Buffer[0 ]; - v1->offset =cpu_to_le64 (wdata->mr->mr->iova); - v1->token =cpu_to_le32 (wdata->mr->mr->rkey); - v1->length =cpu_to_le32 (wdata->mr->mr->length); +smbd_mr_fill_buffer_descriptor (wdata->mr, v1); rqst.rq_iov[0 ].iov_len +=sizeof (*v1);
fs/smb/client/smbdirect.c +6 -0
@@ -1196,6 +1196,12 @@ struct smbdirect_mr_io *smbd_register_mr(struct smbd_connection *info,return smbdirect_connection_register_mr_io (sc, iter, writing, need_invalidate); } +void smbd_mr_fill_buffer_descriptor (struct smbdirect_mr_io *mr, +struct smbdirect_buffer_descriptor_v1 *v1) +{ +smbdirect_mr_io_fill_buffer_descriptor (mr, v1); +} +/* * Deregister a MR after I/O is done * This function may wait if remote invalidation is not used
fs/smb/client/smbdirect.h +2 -0
@@ -60,6 +60,8 @@ int smbd_send(struct TCP_Server_Info *server,struct smbdirect_mr_io *smbd_register_mr (struct smbd_connection *info,struct iov_iter *iter,bool writing,bool need_invalidate); +void smbd_mr_fill_buffer_descriptor (struct smbdirect_mr_io *mr, +struct smbdirect_buffer_descriptor_v1 *v1);void smbd_deregister_mr (struct smbdirect_mr_io *mr);
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 92/145 smb: client: introduce and use smbd_debug_proc_show()
In reply to Stefan MetzmacherThis will allow us to make struct smbdirect_socket private in future.
--- fs/smb/client/cifs_debug.c | 67 +------------------------------------- fs/smb/client/smbdirect.c | 15 +++++++++ fs/smb/client/smbdirect.h | 1 + 3 files changed, 17 insertions(+), 66 deletions(-)
fs/smb/client/cifs_debug.c +1 -66
@@ -24,7 +24,6 @@#endif #ifdef CONFIG_CIFS_SMB_DIRECT #include "smbdirect.h" -#include "../common/smbdirect/smbdirect_pdu.h" #endif #include "cifs_swn.h" #include "cached_dir.h"
@@ -456,11 +455,6 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v) c =0 ;spin_lock (&cifs_tcp_ses_lock);list_for_each_entry (server, &cifs_tcp_ses_list, tcp_ses_list) { -#ifdef CONFIG_CIFS_SMB_DIRECT -struct smbdirect_socket *sc; -struct smbdirect_socket_parameters *sp; -#endif -/* channel info will be printed as a part of sessions below */ if (SERVER_IS_CHAN (server))continue ;
@@ -475,66 +469,7 @@ static int cifs_debug_data_proc_show(struct seq_file *m, void *v)seq_printf (m,"\nClientGUID: %pUL" , server->client_guid);spin_unlock (&server->srv_lock);#ifdef CONFIG_CIFS_SMB_DIRECT -if (!server->rdma) - goto skip_rdma; - -if (!server->smbd_conn) { -seq_printf (m,"\nSMBDirect transport not available" ); - goto skip_rdma; - } - sc = &server->smbd_conn->socket; - sp = &sc->parameters; - -seq_printf (m,"\nSMBDirect protocol version: 0x%x " -"transport status: %s (%u)" , -SMBDIRECT_V1 , -smbdirect_socket_status_string (sc->status), - sc->status); -seq_printf (m,"\nConn receive_credit_max: %u " -"send_credit_target: %u max_send_size: %u" , - sp->recv_credit_max, - sp->send_credit_target, - sp->max_send_size); -seq_printf (m,"\nConn max_fragmented_recv_size: %u " -"max_fragmented_send_size: %u max_receive_size:%u" , - sp->max_fragmented_recv_size, - sp->max_fragmented_send_size, - sp->max_recv_size); -seq_printf (m,"\nConn keep_alive_interval: %u " -"max_readwrite_size: %u rdma_readwrite_threshold: %u" , - sp->keepalive_interval_msec *1000 , - sp->max_read_write_size, - server->rdma_readwrite_threshold); -seq_printf (m,"\nDebug count_get_receive_buffer: %llu " -"count_put_receive_buffer: %llu count_send_empty: %llu" , - sc->statistics.get_receive_buffer, - sc->statistics.put_receive_buffer, - sc->statistics.send_empty); -seq_printf (m,"\nRead Queue " -"count_enqueue_reassembly_queue: %llu " -"count_dequeue_reassembly_queue: %llu " -"reassembly_data_length: %u " -"reassembly_queue_length: %u" , - sc->statistics.enqueue_reassembly_queue, - sc->statistics.dequeue_reassembly_queue, - sc->recv_io.reassembly.data_length, - sc->recv_io.reassembly.queue_length); -seq_printf (m,"\nCurrent Credits send_credits: %u " -"receive_credits: %u receive_credit_target: %u" , -atomic_read (&sc->send_io.credits.count), -atomic_read (&sc->recv_io.credits.count), - sc->recv_io.credits.target); -seq_printf (m,"\nPending send_pending: %u " , -atomic_read (&sc->send_io.pending.count)); -seq_printf (m,"\nMR responder_resources: %u " -"max_frmr_depth: %u mr_type: 0x%x" , - sp->responder_resources, - sp->max_frmr_depth, - sc->mr_io.type); -seq_printf (m,"\nMR mr_ready_count: %u mr_used_count: %u" , -atomic_read (&sc->mr_io.ready.count), -atomic_read (&sc->mr_io.used.count)); -skip_rdma: +smbd_debug_proc_show (server, m);#endif seq_printf(m, "\nNumber of credits: %d,%d,%d Dialect 0x%x", server->credits,
fs/smb/client/smbdirect.c +15 -0
@@ -1212,3 +1212,18 @@ void smbd_deregister_mr(struct smbdirect_mr_io *mr) {smbdirect_connection_deregister_mr_io (mr); } + +void smbd_debug_proc_show (struct TCP_Server_Info *server,struct seq_file *m) +{ +if (!server->rdma) +return ; + +if (!server->smbd_conn) { +seq_puts (m,"\nSMBDirect transport not available" ); +return ; + } + +smbdirect_connection_legacy_debug_proc_show (&server->smbd_conn->socket, + server->rdma_readwrite_threshold, + m); +}
fs/smb/client/smbdirect.h +1 -0
@@ -64,6 +64,7 @@ void smbd_mr_fill_buffer_descriptor(struct smbdirect_mr_io *mr, struct smbdirect_buffer_descriptor_v1 *v1);void smbd_deregister_mr (struct smbdirect_mr_io *mr); +void smbd_debug_proc_show (struct TCP_Server_Info *server,struct seq_file *m);#else #define cifs_rdma_enabled (server) 0
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 93/145 smb: client: make use of smbdirect_socket_init_new() and smbdirect_connect_sync()
In reply to Stefan MetzmacherThis means we finally only use common functions in the client.
We still use the embedded struct smbdirect_socket and are able to access internals, but the will be removed in the next commits as well.
--- fs/smb/client/smbdirect.c | 798 +++----------------------------------- fs/smb/client/smbdirect.h | 1 + 2 files changed, 56 insertions(+), 743 deletions(-)
fs/smb/client/smbdirect.c +55 -743
@@ -10,19 +10,12 @@#include <linux/module.h> #include <linux/highmem.h> #include <linux/folio_queue.h> -#include "../common/smbdirect/smbdirect_pdu.h" +#include <linux/errname.h> #include "smbdirect.h" #include "cifs_debug.h" #include "cifsproto.h" #include "smb2proto.h" -const struct smbdirect_socket_parameters *smbd_get_parameters (struct smbd_connection *conn) -{ -struct smbdirect_socket *sc = &conn->socket; - -return &sc->parameters; -} -/* Port numbers for SMBD transport */ #define SMB_PORT 445#define SMBD_PORT 5445
@@ -36,21 +29,12 @@ const struct smbdirect_socket_parameters *smbd_get_parameters(struct smbd_connec/* The timeout to wait for a keepalive message from peer in seconds */ #define KEEPALIVE_RECV_TIMEOUT 5 -/* SMBD minimum receive size and fragmented sized defined in [MS-SMBD] */ -#define SMBD_MIN_RECEIVE_SIZE 128 -#define SMBD_MIN_FRAGMENTED_SIZE 131072 -/* * Default maximum number of RDMA read/write outstanding on this connection * This value is possibly decreased during QP creation on hardware limit */ #define SMBD_CM_RESPONDER_RESOURCES 32 -/* Maximum number of retries on data transfer operations */ -#define SMBD_CM_RETRY 6 -/* No need to retry on Receiver Not Ready since SMBD manages credits */ -#define SMBD_CM_RNR_RETRY 0 -/* * User configurable initial values per SMBD transport connection * as defined in [MS-SMBD] 3.1.1.1
@@ -179,556 +163,6 @@ do { \ #define log_rdma_mr(level, fmt, args...) \ log_rdma(level, LOG_RDMA_MR, fmt, ##args) -/* Upcall from RDMA CM */ -static int smbd_conn_upcall ( -struct rdma_cm_id *id,struct rdma_cm_event *event) -{ -struct smbdirect_socket *sc = id->context; -const char *event_name =rdma_event_msg (event->event); -u8 peer_initiator_depth; -u8 peer_responder_resources; - -log_rdma_event (INFO ,"event=%s status=%d\n" , - event_name, event->status); - -switch (event->event) { -case RDMA_CM_EVENT_ADDR_RESOLVED : -if (SMBDIRECT_CHECK_STATUS_DISCONNECT (sc,SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING )) -break ; - sc->status =SMBDIRECT_SOCKET_RESOLVE_ROUTE_NEEDED ; -wake_up (&sc->status_wait); -break ; - -case RDMA_CM_EVENT_ROUTE_RESOLVED : -if (SMBDIRECT_CHECK_STATUS_DISCONNECT (sc,SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING )) -break ; - sc->status =SMBDIRECT_SOCKET_RDMA_CONNECT_NEEDED ; -wake_up (&sc->status_wait); -break ; - -case RDMA_CM_EVENT_ADDR_ERROR : -log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name); - sc->status =SMBDIRECT_SOCKET_RESOLVE_ADDR_FAILED ; -smbdirect_socket_cleanup_work (&sc->disconnect_work); -break ; - -case RDMA_CM_EVENT_ROUTE_ERROR : -log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name); - sc->status =SMBDIRECT_SOCKET_RESOLVE_ROUTE_FAILED ; -smbdirect_socket_cleanup_work (&sc->disconnect_work); -break ; - -case RDMA_CM_EVENT_ESTABLISHED : -log_rdma_event (INFO ,"connected event=%s\n" , event_name); - -/* -* Here we work around an inconsistency between -* iWarp and other devices (at least rxe and irdma using RoCEv2) -*/ -if (rdma_protocol_iwarp (id->device, id->port_num)) { -/* -* iWarp devices report the peer's values -* with the perspective of the peer here. -* Tested with siw and irdma (in iwarp mode) -* We need to change to our perspective here, -* so we need to switch the values. -*/ - peer_initiator_depth = event->param.conn.responder_resources; - peer_responder_resources = event->param.conn.initiator_depth; - }else { -/* -* Non iWarp devices report the peer's values -* already changed to our perspective here. -* Tested with rxe and irdma (in roce mode). -*/ - peer_initiator_depth = event->param.conn.initiator_depth; - peer_responder_resources = event->param.conn.responder_resources; - } -smbdirect_connection_negotiate_rdma_resources (sc, - peer_initiator_depth, - peer_responder_resources, - &event->param.conn); - -if (SMBDIRECT_CHECK_STATUS_DISCONNECT (sc,SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING )) -break ; - sc->status =SMBDIRECT_SOCKET_NEGOTIATE_NEEDED ; -wake_up (&sc->status_wait); -break ; - -case RDMA_CM_EVENT_CONNECT_ERROR : -case RDMA_CM_EVENT_UNREACHABLE : -case RDMA_CM_EVENT_REJECTED : -log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name); - sc->status =SMBDIRECT_SOCKET_RDMA_CONNECT_FAILED ; -smbdirect_socket_cleanup_work (&sc->disconnect_work); -break ; - -case RDMA_CM_EVENT_DEVICE_REMOVAL : -case RDMA_CM_EVENT_DISCONNECTED : -/* This happens when we fail the negotiation */ -if (sc->status ==SMBDIRECT_SOCKET_NEGOTIATE_FAILED ) { -log_rdma_event (ERR ,"event=%s during negotiation\n" , event_name); - } - - sc->status =SMBDIRECT_SOCKET_DISCONNECTED ; -smbdirect_socket_cleanup_work (&sc->disconnect_work); -break ; - -default : -log_rdma_event (ERR ,"unexpected event=%s status=%d\n" , - event_name, event->status); -break ; - } - -return 0 ; -} - -static inline void *smbdirect_send_io_payload (struct smbdirect_send_io *request) -{ -return (void *)request->packet; -} - -static inline void *smbdirect_recv_io_payload (struct smbdirect_recv_io *response) -{ -return (void *)response->packet; -} - -static void dump_smbdirect_negotiate_resp (struct smbdirect_negotiate_resp *resp) -{ -log_rdma_event (INFO ,"resp message min_version %u max_version %u negotiated_version %u credits_requested %u credits_granted %u status %u max_readwrite_size %u preferred_send_size %u max_receive_size %u max_fragmented_size %u\n" , - resp->min_version, resp->max_version, - resp->negotiated_version, resp->credits_requested, - resp->credits_granted, resp->status, - resp->max_readwrite_size, resp->preferred_send_size, - resp->max_receive_size, resp->max_fragmented_size); -} - -/* -* Process a negotiation response message, according to [MS-SMBD]3.1.5.7 -* response, packet_length: the negotiation response message -* return value: true if negotiation is a success, false if failed -*/ -static bool process_negotiation_response ( -struct smbdirect_recv_io *response,int packet_length) -{ -struct smbdirect_socket *sc = response->socket; -struct smbdirect_socket_parameters *sp = &sc->parameters; -struct smbdirect_negotiate_resp *packet =smbdirect_recv_io_payload (response); - -if (packet_length <sizeof (struct smbdirect_negotiate_resp )) { -log_rdma_event (ERR , -"error: packet_length=%d\n" , packet_length); -return false; - } - -if (le16_to_cpu (packet->negotiated_version) !=SMBDIRECT_V1 ) { -log_rdma_event (ERR ,"error: negotiated_version=%x\n" , -le16_to_cpu (packet->negotiated_version)); -return false; - } - -if (packet->credits_requested ==0 ) { -log_rdma_event (ERR ,"error: credits_requested==0\n" ); -return false; - } - sc->recv_io.credits.target =le16_to_cpu (packet->credits_requested); - sc->recv_io.credits.target =min_t (u16, sc->recv_io.credits.target, sp->recv_credit_max); - -if (packet->credits_granted ==0 ) { -log_rdma_event (ERR ,"error: credits_granted==0\n" ); -return false; - } -atomic_set (&sc->send_io.lcredits.count, sp->send_credit_target); -atomic_set (&sc->send_io.credits.count,le16_to_cpu (packet->credits_granted)); - -if (le32_to_cpu (packet->preferred_send_size) > sp->max_recv_size) { -log_rdma_event (ERR ,"error: preferred_send_size=%d\n" , -le32_to_cpu (packet->preferred_send_size)); -return false; - } - sp->max_recv_size =le32_to_cpu (packet->preferred_send_size); - -if (le32_to_cpu (packet->max_receive_size) <SMBD_MIN_RECEIVE_SIZE ) { -log_rdma_event (ERR ,"error: max_receive_size=%d\n" , -le32_to_cpu (packet->max_receive_size)); -return false; - } - sp->max_send_size =min_t (u32, sp->max_send_size, -le32_to_cpu (packet->max_receive_size)); - -if (le32_to_cpu (packet->max_fragmented_size) < -SMBD_MIN_FRAGMENTED_SIZE ) { -log_rdma_event (ERR ,"error: max_fragmented_size=%d\n" , -le32_to_cpu (packet->max_fragmented_size)); -return false; - } - sp->max_fragmented_send_size = -le32_to_cpu (packet->max_fragmented_size); - - - sp->max_read_write_size =min_t (u32, -le32_to_cpu (packet->max_readwrite_size), - sp->max_frmr_depth *PAGE_SIZE ); - sp->max_frmr_depth = sp->max_read_write_size /PAGE_SIZE ; - - sc->recv_io.expected =SMBDIRECT_EXPECT_DATA_TRANSFER ; -return true; -} - -/* Called from softirq, when recv is done */ -static void recv_done (struct ib_cq *cq,struct ib_wc *wc) -{ -struct smbdirect_data_transfer *data_transfer; -struct smbdirect_recv_io *response = - container_of(wc->wr_cqe, struct smbdirect_recv_io, cqe); -struct smbdirect_socket *sc = response->socket; -struct smbdirect_socket_parameters *sp = &sc->parameters; -u16 old_recv_credit_target; -u32 data_offset =0 ; -u32 data_length =0 ; -u32 remaining_data_length =0 ; -bool negotiate_done = false; - -log_rdma_recv (INFO , -"response=0x%p type=%d wc status=%s wc opcode %d byte_len=%d pkey_index=%u\n" , - response, sc->recv_io.expected, -ib_wc_status_msg (wc->status), wc->opcode, - wc->byte_len, wc->pkey_index); - -if (wc->status !=IB_WC_SUCCESS || wc->opcode !=IB_WC_RECV ) { -if (wc->status !=IB_WC_WR_FLUSH_ERR ) -log_rdma_recv (ERR ,"wc->status=%s opcode=%d\n" , -ib_wc_status_msg (wc->status), wc->opcode); - goto error; - } - -ib_dma_sync_single_for_cpu ( - wc->qp->device, - response->sge.addr, - response->sge.length, -DMA_FROM_DEVICE ); - -/* -* Reset timer to the keepalive interval in -* order to trigger our next keepalive message. -*/ - sc->idle.keepalive =SMBDIRECT_KEEPALIVE_NONE ; -mod_delayed_work (sc->workqueue, &sc->idle.timer_work, -msecs_to_jiffies (sp->keepalive_interval_msec)); - -switch (sc->recv_io.expected) { -/* SMBD negotiation response */ -case SMBDIRECT_EXPECT_NEGOTIATE_REP : -dump_smbdirect_negotiate_resp (smbdirect_recv_io_payload (response)); - sc->recv_io.reassembly.full_packet_received = true; - negotiate_done = -process_negotiation_response (response, wc->byte_len); -smbdirect_connection_put_recv_io (response); -if (SMBDIRECT_CHECK_STATUS_WARN (sc,SMBDIRECT_SOCKET_NEGOTIATE_RUNNING )) - negotiate_done = false; -if (!negotiate_done) { - sc->status =SMBDIRECT_SOCKET_NEGOTIATE_FAILED ; -smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); - }else { - sc->status =SMBDIRECT_SOCKET_CONNECTED ; -wake_up (&sc->status_wait); - } - -return ; - -/* SMBD data transfer packet */ -case SMBDIRECT_EXPECT_DATA_TRANSFER : - data_transfer =smbdirect_recv_io_payload (response); - -if (wc->byte_len < - offsetof(struct smbdirect_data_transfer , padding)) - goto error; - - remaining_data_length =le32_to_cpu (data_transfer->remaining_data_length); - data_offset =le32_to_cpu (data_transfer->data_offset); - data_length =le32_to_cpu (data_transfer->data_length); -if (wc->byte_len < data_offset || - (u64 )wc->byte_len < (u64 )data_offset + data_length) - goto error; - -if (remaining_data_length > sp->max_fragmented_recv_size || - data_length > sp->max_fragmented_recv_size || - (u64 )remaining_data_length + (u64 )data_length > (u64 )sp->max_fragmented_recv_size) - goto error; - -if (data_length) { -if (sc->recv_io.reassembly.full_packet_received) - response->first_segment = true; - -if (le32_to_cpu (data_transfer->remaining_data_length)) - sc->recv_io.reassembly.full_packet_received = false; -else - sc->recv_io.reassembly.full_packet_received = true; - } - -atomic_dec (&sc->recv_io.posted.count); -atomic_dec (&sc->recv_io.credits.count); - old_recv_credit_target = sc->recv_io.credits.target; - sc->recv_io.credits.target = -le16_to_cpu (data_transfer->credits_requested); - sc->recv_io.credits.target = -min_t (u16, sc->recv_io.credits.target, sp->recv_credit_max); - sc->recv_io.credits.target = -max_t (u16, sc->recv_io.credits.target,1 ); -if (le16_to_cpu (data_transfer->credits_granted)) { -atomic_add (le16_to_cpu (data_transfer->credits_granted), - &sc->send_io.credits.count); -/* -* We have new send credits granted from remote peer -* If any sender is waiting for credits, unblock it -*/ -wake_up (&sc->send_io.credits.wait_queue); - } - -log_incoming (INFO ,"data flags %d data_offset %d data_length %d remaining_data_length %d\n" , -le16_to_cpu (data_transfer->flags), -le32_to_cpu (data_transfer->data_offset), -le32_to_cpu (data_transfer->data_length), -le32_to_cpu (data_transfer->remaining_data_length)); - -/* Send an immediate response right away if requested */ -if (le16_to_cpu (data_transfer->flags) & -SMBDIRECT_FLAG_RESPONSE_REQUESTED ) { -log_keep_alive (INFO ,"schedule send of immediate response\n" ); -queue_work (sc->workqueue, &sc->idle.immediate_work); - } - -/* -* If this is a packet with data playload place the data in -* reassembly queue and wake up the reading thread -*/ -if (data_length) { -if (sc->recv_io.credits.target > old_recv_credit_target) -queue_work (sc->workqueue, &sc->recv_io.posted.refill_work); - -smbdirect_connection_reassembly_append_recv_io (sc, response, data_length); -wake_up (&sc->recv_io.reassembly.wait_queue); - }else -smbdirect_connection_put_recv_io (response); - -return ; - -case SMBDIRECT_EXPECT_NEGOTIATE_REQ : -/* Only server... */ -break ; - } - -/* -* This is an internal error! -*/ -log_rdma_recv (ERR ,"unexpected response type=%d\n" , sc->recv_io.expected); -WARN_ON_ONCE (sc->recv_io.expected !=SMBDIRECT_EXPECT_DATA_TRANSFER ); -error: -smbdirect_connection_put_recv_io (response); -smbdirect_socket_schedule_cleanup (sc, -ECONNABORTED ); -} - -static struct rdma_cm_id *smbd_create_id ( -struct smbdirect_socket *sc, -struct sockaddr *dstaddr,int port) -{ -struct smbdirect_socket_parameters *sp = &sc->parameters; -struct rdma_cm_id *id; -int rc; -__be16 *sport; - - id =rdma_create_id (&init_net, smbd_conn_upcall, sc, -RDMA_PS_TCP ,IB_QPT_RC ); -if (IS_ERR (id)) { - rc =PTR_ERR (id); -log_rdma_event (ERR ,"rdma_create_id() failed %i\n" , rc); -return id; - } - -if (dstaddr->sa_family ==AF_INET6 ) - sport = &((struct sockaddr_in6 *)dstaddr)->sin6_port; -else - sport = &((struct sockaddr_in *)dstaddr)->sin_port; - - *sport =htons (port); - -WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ADDR_NEEDED ); - sc->status =SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING ; - rc =rdma_resolve_addr (id,NULL , (struct sockaddr *)dstaddr, - sp->resolve_addr_timeout_msec); -if (rc) { -log_rdma_event (ERR ,"rdma_resolve_addr() failed %i\n" , rc); - goto out; - } - rc =wait_event_interruptible_timeout ( - sc->status_wait, - sc->status !=SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING , -msecs_to_jiffies (sp->resolve_addr_timeout_msec)); -/* e.g. if interrupted returns -ERESTARTSYS */ -if (rc <0 ) { -log_rdma_event (ERR ,"rdma_resolve_addr timeout rc: %i\n" , rc); - goto out; - } -if (sc->status ==SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING ) { - rc = -ETIMEDOUT ; -log_rdma_event (ERR ,"rdma_resolve_addr() completed %i\n" , rc); - goto out; - } -if (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ROUTE_NEEDED ) { - rc = -EHOSTUNREACH ; -log_rdma_event (ERR ,"rdma_resolve_addr() completed %i\n" , rc); - goto out; - } - -WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ROUTE_NEEDED ); - sc->status =SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING ; - rc =rdma_resolve_route (id, sp->resolve_route_timeout_msec); -if (rc) { -log_rdma_event (ERR ,"rdma_resolve_route() failed %i\n" , rc); - goto out; - } - rc =wait_event_interruptible_timeout ( - sc->status_wait, - sc->status !=SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING , -msecs_to_jiffies (sp->resolve_route_timeout_msec)); -/* e.g. if interrupted returns -ERESTARTSYS */ -if (rc <0 ) { -log_rdma_event (ERR ,"rdma_resolve_addr timeout rc: %i\n" , rc); - goto out; - } -if (sc->status ==SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING ) { - rc = -ETIMEDOUT ; -log_rdma_event (ERR ,"rdma_resolve_route() completed %i\n" , rc); - goto out; - } -if (sc->status !=SMBDIRECT_SOCKET_RDMA_CONNECT_NEEDED ) { - rc = -ENETUNREACH ; -log_rdma_event (ERR ,"rdma_resolve_route() completed %i\n" , rc); - goto out; - } - -return id; - -out: -rdma_destroy_id (id); -return ERR_PTR (rc); -} - -static int smbd_ia_open ( -struct smbdirect_socket *sc, -struct sockaddr *dstaddr,int port) -{ -struct smbdirect_socket_parameters *sp = &sc->parameters; -int rc; - -WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_CREATED ); - sc->status =SMBDIRECT_SOCKET_RESOLVE_ADDR_NEEDED ; - - sc->rdma.cm_id =smbd_create_id (sc, dstaddr, port); -if (IS_ERR (sc->rdma.cm_id)) { - rc =PTR_ERR (sc->rdma.cm_id); - goto out1; - } - sc->ib.dev = sc->rdma.cm_id->device; - -if (!smbdirect_frwr_is_supported (&sc->ib.dev->attrs)) { -log_rdma_event (ERR ,"Fast Registration Work Requests (FRWR) is not supported\n" ); -log_rdma_event (ERR ,"Device capability flags = %llx max_fast_reg_page_list_len = %u\n" , - sc->ib.dev->attrs.device_cap_flags, - sc->ib.dev->attrs.max_fast_reg_page_list_len); - rc = -EPROTONOSUPPORT ; - goto out2; - } - sp->max_frmr_depth =min_t (u32, - sp->max_frmr_depth, - sc->ib.dev->attrs.max_fast_reg_page_list_len); - sc->mr_io.type =IB_MR_TYPE_MEM_REG ; -if (sc->ib.dev->attrs.kernel_cap_flags &IBK_SG_GAPS_REG ) - sc->mr_io.type =IB_MR_TYPE_SG_GAPS ; - -return 0 ; - -out2: -rdma_destroy_id (sc->rdma.cm_id); - sc->rdma.cm_id =NULL ; - -out1: -return rc; -} - -/* -* Send a negotiation request message to the peer -* The negotiation procedure is in [MS-SMBD] 3.1.5.2 and 3.1.5.3 -* After negotiation, the transport is connected and ready for -* carrying upper layer SMB payload -*/ -static int smbd_post_send_negotiate_req (struct smbdirect_socket *sc) -{ -struct smbdirect_socket_parameters *sp = &sc->parameters; -struct ib_send_wr send_wr; -int rc; -struct smbdirect_send_io *request; -struct smbdirect_negotiate_req *packet; - - request =smbdirect_connection_alloc_send_io (sc); -if (IS_ERR (request)) -return PTR_ERR (request); - - packet =smbdirect_send_io_payload (request); - packet->min_version =cpu_to_le16 (SMBDIRECT_V1 ); - packet->max_version =cpu_to_le16 (SMBDIRECT_V1 ); - packet->reserved =0 ; - packet->credits_requested =cpu_to_le16 (sp->send_credit_target); - packet->preferred_send_size =cpu_to_le32 (sp->max_send_size); - packet->max_receive_size =cpu_to_le32 (sp->max_recv_size); - packet->max_fragmented_size = -cpu_to_le32 (sp->max_fragmented_recv_size); - - request->sge[0 ].addr =ib_dma_map_single ( - sc->ib.dev, (void *)packet, -sizeof (*packet),DMA_TO_DEVICE ); -if (ib_dma_mapping_error (sc->ib.dev, request->sge[0 ].addr)) { - rc = -EIO ; - goto dma_mapping_failed; - } - - request->sge[0 ].length =sizeof (*packet); - request->sge[0 ].lkey = sc->ib.pd->local_dma_lkey; - request->num_sge =1 ; - -ib_dma_sync_single_for_device ( - sc->ib.dev, request->sge[0 ].addr, - request->sge[0 ].length,DMA_TO_DEVICE ); - - request->cqe.done = smbdirect_connection_send_io_done; - - send_wr.next =NULL ; - send_wr.wr_cqe = &request->cqe; - send_wr.sg_list = request->sge; - send_wr.num_sge = request->num_sge; - send_wr.opcode =IB_WR_SEND ; - send_wr.send_flags =IB_SEND_SIGNALED ; - -log_rdma_send (INFO ,"sge addr=0x%llx length=%u lkey=0x%x\n" , - request->sge[0 ].addr, - request->sge[0 ].length, request->sge[0 ].lkey); - -atomic_inc (&sc->send_io.pending.count); - rc =ib_post_send (sc->ib.qp, &send_wr,NULL ); -if (!rc) -return 0 ; - -/* if we reach here, post send failed */ -log_rdma_send (ERR ,"ib_post_send failed rc=%d\n" , rc); -atomic_dec (&sc->send_io.pending.count); - -smbdirect_socket_schedule_cleanup (sc, rc); - -dma_mapping_failed: -smbdirect_connection_free_send_io (request); -return rc; -} -static int smbd_post_send_full_iter (struct smbdirect_socket *sc,struct iov_iter *iter,u32 remaining_data_length)
@@ -758,49 +192,6 @@ static int smbd_post_send_full_iter(struct smbdirect_socket *sc,return bytes; } -/* Perform SMBD negotiate according to [MS-SMBD] 3.1.5.2 */ -static int smbd_negotiate (struct smbdirect_socket *sc) -{ -struct smbdirect_socket_parameters *sp = &sc->parameters; -int rc; -struct smbdirect_recv_io *response =smbdirect_connection_get_recv_io (sc); - -WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_NEGOTIATE_NEEDED ); - sc->status =SMBDIRECT_SOCKET_NEGOTIATE_RUNNING ; - - sc->recv_io.expected =SMBDIRECT_EXPECT_NEGOTIATE_REP ; - rc =smbdirect_connection_post_recv_io (response); -log_rdma_event (INFO ,"smbd_post_recv rc=%d iov.addr=0x%llx iov.length=%u iov.lkey=0x%x\n" , - rc, response->sge.addr, - response->sge.length, response->sge.lkey); -if (rc) { -smbdirect_connection_put_recv_io (response); -return rc; - } - - rc =smbd_post_send_negotiate_req (sc); -if (rc) -return rc; - - rc =wait_event_interruptible_timeout ( - sc->status_wait, - sc->status !=SMBDIRECT_SOCKET_NEGOTIATE_RUNNING , -msecs_to_jiffies (sp->negotiate_timeout_msec)); -log_rdma_event (INFO ,"wait_event_interruptible_timeout rc=%d\n" , rc); - -if (sc->status ==SMBDIRECT_SOCKET_CONNECTED ) -return 0 ; - -if (rc ==0 ) - rc = -ETIMEDOUT ; -else if (rc == -ERESTARTSYS ) - rc = -EINTR ; -else - rc = -ENOTCONN ; - -return rc; -} -/* * Destroy the transport and related RDMA and memory resources * Need to go through all the pending counters and make sure on one is using
@@ -819,7 +210,7 @@ void smbd_destroy(struct TCP_Server_Info *server)smbdirect_socket_destroy_sync (sc); -destroy_workqueue (sc->workqueue); +destroy_workqueue (info->workqueue);kfree (info); server->smbd_conn =NULL ; }
@@ -841,10 +232,8 @@ int smbd_reconnect(struct TCP_Server_Info *server)* This is possible if transport is disconnected and we haven't received * notification from RDMA, but upper layer has detected timeout */ -if (server->smbd_conn->socket.status ==SMBDIRECT_SOCKET_CONNECTED ) { -log_rdma_event (INFO ,"disconnecting transport\n" ); -smbd_destroy (server); - } +log_rdma_event (INFO ,"disconnecting transport\n" ); +smbd_destroy (server); create_conn:log_rdma_event (INFO ,"creating rdma session\n" );
@@ -864,18 +253,14 @@ int smbd_reconnect(struct TCP_Server_Info *server)static struct smbd_connection *_smbd_get_connection (struct TCP_Server_Info *server,struct sockaddr *dstaddr,int port) { -int rc; +struct net *net =cifs_net_ns (server);struct smbd_connection *info;struct smbdirect_socket *sc;struct smbdirect_socket_parameters init_params = {};struct smbdirect_socket_parameters *sp; -struct rdma_conn_param conn_param; -struct sockaddr_in *addr_in = (struct sockaddr_in *) dstaddr; -struct ib_port_immutable port_immutable; -__be32 ird_ord_hdr[2 ]; +__be16 *sport;char wq_name[80 ]; -struct workqueue_struct *workqueue; -struct smbdirect_recv_io *recv_io; +int ret;/* * Create the initial parameters
@@ -899,143 +284,64 @@ static struct smbd_connection *_smbd_get_connection( info =kzalloc (sizeof (struct smbd_connection ),GFP_KERNEL );if (!info)return NULL ; - sc = &info->socket; -scnprintf (wq_name,ARRAY_SIZE (wq_name),"smbd_%p" , sc); - workqueue =create_workqueue (wq_name); -if (!workqueue) +scnprintf (wq_name,ARRAY_SIZE (wq_name),"smbd_%p" , info); + info->workqueue =create_workqueue (wq_name); +if (!info->workqueue) goto create_wq_failed; -smbdirect_socket_prepare_create (sc, sp, workqueue); + sc = &info->socket; + ret =smbdirect_socket_init_new (net, sc); +if (ret) + goto socket_init_failed;smbdirect_socket_set_logging (sc,NULL , smbd_logging_needed, smbd_logging_vaprintf); - sc->ib.poll_ctx =IB_POLL_SOFTIRQ ; -/* -* from here we operate on the copy. -*/ - sp = &sc->parameters; - - rc =smbd_ia_open (sc, dstaddr, port); -if (rc) { -log_rdma_event (INFO ,"smbd_ia_open rc=%d\n" , rc); - goto create_id_failed; - } - - sp->responder_resources = -min_t (u8, sp->responder_resources, - sc->ib.dev->attrs.max_qp_rd_atom); -log_rdma_mr (INFO ,"responder_resources=%d\n" , - sp->responder_resources); - - rc =smbdirect_connection_create_qp (sc); -if (rc) { -log_rdma_event (ERR ,"smbdirect_connection_create_qp failed %i\n" , rc); - goto create_qp_failed; - } - -memset (&conn_param,0 ,sizeof (conn_param)); - conn_param.initiator_depth = sp->initiator_depth; - conn_param.responder_resources = sp->responder_resources; - -/* Need to send IRD/ORD in private data for iWARP */ - sc->ib.dev->ops.get_port_immutable ( - sc->ib.dev, sc->rdma.cm_id->port_num, &port_immutable); -if (port_immutable.core_cap_flags &RDMA_CORE_PORT_IWARP ) { - ird_ord_hdr[0 ] =cpu_to_be32 (conn_param.responder_resources); - ird_ord_hdr[1 ] =cpu_to_be32 (conn_param.initiator_depth); - conn_param.private_data = ird_ord_hdr; - conn_param.private_data_len =sizeof (ird_ord_hdr); - }else { - conn_param.private_data =NULL ; - conn_param.private_data_len =0 ; - } - - conn_param.retry_count =SMBD_CM_RETRY ; - conn_param.rnr_retry_count =SMBD_CM_RNR_RETRY ; - conn_param.flow_control =0 ; - -log_rdma_event (INFO ,"connecting to IP %pI4 port %d\n" , - &addr_in->sin_addr, port); - -WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RDMA_CONNECT_NEEDED ); - sc->status =SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING ; - rc =rdma_connect (sc->rdma.cm_id, &conn_param); -if (rc) { -log_rdma_event (ERR ,"rdma_connect() failed with %i\n" , rc); - goto rdma_connect_failed; - } - -wait_event_interruptible_timeout ( - sc->status_wait, - sc->status !=SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING , -msecs_to_jiffies (sp->rdma_connect_timeout_msec)); - -if (sc->status !=SMBDIRECT_SOCKET_NEGOTIATE_NEEDED ) { -log_rdma_event (ERR ,"rdma_connect failed port=%d\n" , port); - goto rdma_connect_failed; - } - -log_rdma_event (INFO ,"rdma_connect connected\n" ); + ret =smbdirect_socket_set_initial_parameters (sc, sp); +if (ret) + goto set_params_failed; + ret =smbdirect_socket_set_kernel_settings (sc,IB_POLL_SOFTIRQ ,GFP_KERNEL ); +if (ret) + goto set_settings_failed; + ret =smbdirect_socket_set_custom_workqueue (sc, info->workqueue); +if (ret) + goto set_workqueue_failed; - rc =smbdirect_connection_create_mem_pools (sc); -if (rc) { -log_rdma_event (ERR ,"cache allocation failed\n" ); - goto allocate_cache_failed; - } - -list_for_each_entry (recv_io, &sc->recv_io.free.list, list) - recv_io->cqe.done = recv_done; - -INIT_WORK (&sc->idle.immediate_work, smbdirect_connection_send_immediate_work); -/* -* start with the negotiate timeout and SMBDIRECT_KEEPALIVE_PENDING -* so that the timer will cause a disconnect. -*/ - sc->idle.keepalive =SMBDIRECT_KEEPALIVE_PENDING ; -mod_delayed_work (sc->workqueue, &sc->idle.timer_work, -msecs_to_jiffies (sp->negotiate_timeout_msec)); - -INIT_WORK (&sc->recv_io.posted.refill_work, smbdirect_connection_recv_io_refill_work); +if (dstaddr->sa_family ==AF_INET6 ) + sport = &((struct sockaddr_in6 *)dstaddr)->sin6_port; +else + sport = &((struct sockaddr_in *)dstaddr)->sin_port; - rc =smbd_negotiate (sc); -if (rc) { -log_rdma_event (ERR ,"smbd_negotiate rc=%d\n" , rc); - goto negotiation_failed; - } + *sport =htons (port); - rc =smbdirect_connection_create_mr_list (sc); -if (rc) { -log_rdma_mr (ERR ,"memory registration allocation failed\n" ); - goto allocate_mr_failed; + ret =smbdirect_connect_sync (sc, dstaddr); +if (ret) { +log_rdma_event (ERR ,"smbdirect_connect_sync(%pISpsfc) failed with %d %s\n" , + dstaddr, ret,errname (ret)); + goto connect_failed; }return info; -allocate_mr_failed: +connect_failed: +set_workqueue_failed: +set_settings_failed: +set_params_failed:/* At this point, need to a full transport shutdown */ server->smbd_conn = info;smbd_destroy (server);return NULL ; -negotiation_failed: -disable_delayed_work_sync (&sc->idle.timer_work); -smbdirect_connection_destroy_mem_pools (sc); - sc->status =SMBDIRECT_SOCKET_NEGOTIATE_FAILED ; -rdma_disconnect (sc->rdma.cm_id); -wait_event (sc->status_wait, - sc->status ==SMBDIRECT_SOCKET_DISCONNECTED ); - -allocate_cache_failed: -rdma_connect_failed: -smbdirect_connection_destroy_qp (sc); - -create_qp_failed: -rdma_destroy_id (sc->rdma.cm_id); - -create_id_failed: -destroy_workqueue (sc->workqueue); +socket_init_failed: +destroy_workqueue (info->workqueue); create_wq_failed:kfree (info);return NULL ; } +const struct smbdirect_socket_parameters *smbd_get_parameters (struct smbd_connection *conn) +{ +struct smbdirect_socket *sc = &conn->socket; + +return smbdirect_socket_get_current_parameters (sc); +} +struct smbd_connection *smbd_get_connection (struct TCP_Server_Info *server,struct sockaddr *dstaddr) {
@@ -1054,7 +360,7 @@ struct smbd_connection *smbd_get_connection(if (!ret)return NULL ; - sp = &ret->socket.parameters; + sp =smbd_get_parameters (ret); server->rdma_readwrite_threshold = rdma_readwrite_threshold > sp->max_fragmented_send_size ?
@@ -1082,6 +388,9 @@ int smbd_recv(struct smbd_connection *info, struct msghdr *msg) {struct smbdirect_socket *sc = &info->socket; +if (!smbdirect_connection_is_connected (sc)) +return -ENOTCONN ; +return smbdirect_connection_recvmsg (sc, msg,0 ); }
@@ -1096,14 +405,14 @@ int smbd_send(struct TCP_Server_Info *server, {struct smbd_connection *info = server->smbd_conn;struct smbdirect_socket *sc = &info->socket; -struct smbdirect_socket_parameters *sp = &sc->parameters; +const struct smbdirect_socket_parameters *sp =smbd_get_parameters (info);struct smb_rqst *rqst;struct iov_iter iter;unsigned int remaining_data_length, klen;int rc, i, rqst_idx;int error =0 ; -if (sc->status !=SMBDIRECT_SOCKET_CONNECTED ) +if (!smbdirect_connection_is_connected (sc))return -EAGAIN ;/*
@@ -1193,6 +502,9 @@ struct smbdirect_mr_io *smbd_register_mr(struct smbd_connection *info, {struct smbdirect_socket *sc = &info->socket; +if (!smbdirect_connection_is_connected (sc)) +return NULL ; +return smbdirect_connection_register_mr_io (sc, iter, writing, need_invalidate); }
fs/smb/client/smbdirect.h +1 -0
@@ -38,6 +38,7 @@ extern int smbd_receive_credit_max;*/ struct smbd_connection {struct smbdirect_socket socket; +struct workqueue_struct *workqueue; };/* Create a SMBDirect session */
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 94/145 smb: client: make use of smbdirect_socket_create_kern()/smbdirect_socket_release()
In reply to Stefan MetzmacherWith this we no longer embed struct smbdirect_socket, which will allow us to make it private in the following commits.
--- fs/smb/client/smbdirect.c | 30 ++++++++++++++---------------- fs/smb/client/smbdirect.h | 2 +- 2 files changed, 15 insertions(+), 17 deletions(-)
fs/smb/client/smbdirect.c +14 -16
@@ -200,15 +200,13 @@ static int smbd_post_send_full_iter(struct smbdirect_socket *sc,void smbd_destroy (struct TCP_Server_Info *server) {struct smbd_connection *info = server->smbd_conn; -struct smbdirect_socket *sc;if (!info) {log_rdma_event (INFO ,"rdma session already destroyed\n" );return ; } - sc = &info->socket; -smbdirect_socket_destroy_sync (sc); +smbdirect_socket_release (info->socket);destroy_workqueue (info->workqueue);kfree (info);
@@ -288,8 +286,7 @@ static struct smbd_connection *_smbd_get_connection( info->workqueue =create_workqueue (wq_name);if (!info->workqueue) goto create_wq_failed; - sc = &info->socket; - ret =smbdirect_socket_init_new (net, sc); + ret =smbdirect_socket_create_kern (net, &sc);if (ret) goto socket_init_failed;smbdirect_socket_set_logging (sc,NULL , smbd_logging_needed, smbd_logging_vaprintf);
@@ -317,17 +314,14 @@ static struct smbd_connection *_smbd_get_connection( goto connect_failed; } + info->socket = sc;return info; connect_failed: set_workqueue_failed: set_settings_failed: set_params_failed: -/* At this point, need to a full transport shutdown */ - server->smbd_conn = info; -smbd_destroy (server); -return NULL ; - +smbdirect_socket_release (sc); socket_init_failed:destroy_workqueue (info->workqueue); create_wq_failed:
@@ -337,9 +331,13 @@ static struct smbd_connection *_smbd_get_connection(const struct smbdirect_socket_parameters *smbd_get_parameters (struct smbd_connection *conn) { -struct smbdirect_socket *sc = &conn->socket; +if (unlikely (!conn->socket)) { +static const struct smbdirect_socket_parameters zero_params; + +return &zero_params; + } -return smbdirect_socket_get_current_parameters (sc); +return smbdirect_socket_get_current_parameters (conn->socket); } struct smbd_connection *smbd_get_connection(
@@ -386,7 +384,7 @@ struct smbd_connection *smbd_get_connection(*/ int smbd_recv (struct smbd_connection *info,struct msghdr *msg) { -struct smbdirect_socket *sc = &info->socket; +struct smbdirect_socket *sc = info->socket;if (!smbdirect_connection_is_connected (sc))return -ENOTCONN ;
@@ -404,7 +402,7 @@ int smbd_send(struct TCP_Server_Info *server, int num_rqst, struct smb_rqst *rqst_array) { struct smbd_connection *info = server->smbd_conn; -struct smbdirect_socket *sc = &info->socket; +struct smbdirect_socket *sc = info->socket;const struct smbdirect_socket_parameters *sp =smbd_get_parameters (info);struct smb_rqst *rqst;struct iov_iter iter;
@@ -500,7 +498,7 @@ struct smbdirect_mr_io *smbd_register_mr(struct smbd_connection *info,struct iov_iter *iter, bool writing, bool need_invalidate) { - struct smbdirect_socket *sc = &info->socket; + struct smbdirect_socket *sc = info->socket;if (!smbdirect_connection_is_connected (sc))return NULL ;
@@ -535,7 +533,7 @@ void smbd_debug_proc_show(struct TCP_Server_Info *server, struct seq_file *m)return ; } -smbdirect_connection_legacy_debug_proc_show (&server->smbd_conn->socket, +smbdirect_connection_legacy_debug_proc_show (server->smbd_conn->socket, server->rdma_readwrite_threshold, m); }
fs/smb/client/smbdirect.h +1 -1
@@ -37,7 +37,7 @@ extern int smbd_receive_credit_max;* 5. mempools for allocating packets */ struct smbd_connection { -struct smbdirect_socket socket; +struct smbdirect_socket *socket;struct workqueue_struct *workqueue; };
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 95/145 smb: client: only use public smbdirect functions
In reply to Stefan MetzmacherAlso remove a lot of unused includes...
--- fs/smb/client/smb2pdu.c | 1 - fs/smb/client/smbdirect.c | 20 +++++++++----------- fs/smb/client/smbdirect.h | 13 ------------- 3 files changed, 9 insertions(+), 25 deletions(-)
fs/smb/client/smb2pdu.c +0 -1
@@ -36,7 +36,6 @@#include "smb2glob.h" #include "cifspdu.h" #include "cifs_spnego.h" -#include "../common/smbdirect/smbdirect.h" #include "smbdirect.h" #include "trace.h" #ifdef CONFIG_CIFS_DFS_UPCALL
fs/smb/client/smbdirect.c +9 -11
@@ -7,14 +7,12 @@#define SMBDIRECT_USE_INLINE_C_FILES 1 -#include <linux/module.h> -#include <linux/highmem.h> -#include <linux/folio_queue.h> #include <linux/errname.h> #include "smbdirect.h" #include "cifs_debug.h" #include "cifsproto.h" #include "smb2proto.h" +#include "../common/smbdirect/smbdirect_public.h" /* Port numbers for SMBD transport */ #define SMB_PORT 445
@@ -95,14 +93,6 @@ module_param(smbd_logging_level, uint, 0644);MODULE_PARM_DESC (smbd_logging_level,"Logging level for SMBD transport, 0 (default): error, 1: info" ); -/* -* This is a temporary solution until all code -* is moved to smbdirect_all_c_files.c and we -* have an smbdirect.ko that exports the required -* functions. -*/ -#include "../common/smbdirect/smbdirect_all_c_files.c" - static bool smbd_logging_needed(struct smbdirect_socket *sc, void *private_ptr, unsigned int lvl,
@@ -537,3 +527,11 @@ void smbd_debug_proc_show(struct TCP_Server_Info *server, struct seq_file *m) server->rdma_readwrite_threshold, m); } + +/* +* This is a temporary solution until all code +* is moved to smbdirect_all_c_files.c and we +* have an smbdirect.ko that exports the required +* functions. +*/ +#include "../common/smbdirect/smbdirect_all_c_files.c"
fs/smb/client/smbdirect.h +0 -13
@@ -11,12 +11,8 @@#define cifs_rdma_enabled (server) ((server)->rdma)#include "cifsglob.h" -#include <rdma/ib_verbs.h> -#include <rdma/rdma_cm.h> -#include <linux/mempool.h> #include "../common/smbdirect/smbdirect.h" -#include "../common/smbdirect/smbdirect_socket.h" extern int rdma_readwrite_threshold;extern int smbd_max_frmr_depth;
@@ -27,15 +23,6 @@ extern int smbd_max_send_size;extern int smbd_send_credit_target;extern int smbd_receive_credit_max; -/* -* The context for the SMBDirect transport -* Everything related to the transport is here. It has several logical parts -* 1. RDMA related structures -* 2. SMBDirect connection parameters -* 3. Memory registrations -* 4. Receive and reassembly queues for data receive path -* 5. mempools for allocating packets -*/ struct smbd_connection {struct smbdirect_socket *socket;struct workqueue_struct *workqueue;
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 96/145 smb: client: make use of smbdirect.ko
In reply to Stefan MetzmacherThis means we no longer inline the common smbdirect .c files and use the exported functions from the module instead.
Note the connection specific logging is still
redirect to cifs.ko functions viasmbdirect_socket_set_logging().
We still don't use real socket layer,
but we're very close...
--- fs/smb/client/Kconfig | 3 ++- fs/smb/client/smbdirect.c | 10 ---------- 2 files changed, 2 insertions(+), 11 deletions(-)
fs/smb/client/Kconfig +2 -1
@@ -181,7 +181,8 @@ if CIFS config CIFS_SMB_DIRECT bool "SMB Direct support" - depends on CIFS=m && INFINIBAND && INFINIBAND_ADDR_TRANS || CIFS=y && INFINIBAND=y && INFINIBAND_ADDR_TRANS=y + depends on CIFS && INFINIBAND && INFINIBAND_ADDR_TRANS + select SMB_COMMON_SMBDIRECT help Enables SMB Direct support for SMB 3.0, 3.02 and 3.1.1. SMB Direct allows transferring SMB packets over RDMA. If unsure,
fs/smb/client/smbdirect.c +0 -10
@@ -5,8 +5,6 @@* Author(s): Long Li <[email protected]> */ -#define SMBDIRECT_USE_INLINE_C_FILES 1 -#include <linux/errname.h> #include "smbdirect.h" #include "cifs_debug.h"
@@ -527,11 +525,3 @@ void smbd_debug_proc_show(struct TCP_Server_Info *server, struct seq_file *m) server->rdma_readwrite_threshold, m); } - -/* -* This is a temporary solution until all code -* is moved to smbdirect_all_c_files.c and we -* have an smbdirect.ko that exports the required -* functions. -*/ -#include "../common/smbdirect/smbdirect_all_c_files.c"
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 97/145 smb: server: make use of smbdirect_socket_prepare_create()
In reply to Stefan MetzmacherThis prepares the use of functions from smbdirect_connection.c.
--- fs/smb/server/transport_rdma.c | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-)
fs/smb/server/transport_rdma.c +17 -11
@@ -404,20 +404,14 @@ static struct smb_direct_transport *alloc_transport(struct rdma_cm_id *cm_id) {struct smb_direct_transport *t;struct smbdirect_socket *sc; +struct smbdirect_socket_parameters init_params = {};struct smbdirect_socket_parameters *sp;struct ksmbd_conn *conn; - t =kzalloc (sizeof (*t),KSMBD_DEFAULT_GFP ); -if (!t) -return NULL ; - sc = &t->socket; -smbdirect_socket_init (sc); - sp = &sc->parameters; - - sc->workqueue = smb_direct_wq; - -INIT_WORK (&sc->disconnect_work, smb_direct_disconnect_rdma_work); - +/* +* Create the initial parameters +*/ + sp = &init_params; sp->negotiate_timeout_msec =SMB_DIRECT_NEGOTIATE_TIMEOUT *1000 ; sp->initiator_depth =SMB_DIRECT_CM_INITIATOR_DEPTH ; sp->responder_resources =1 ;
@@ -430,6 +424,18 @@ static struct smb_direct_transport *alloc_transport(struct rdma_cm_id *cm_id) sp->keepalive_interval_msec =SMB_DIRECT_KEEPALIVE_SEND_INTERVAL *1000 ; sp->keepalive_timeout_msec =SMB_DIRECT_KEEPALIVE_RECV_TIMEOUT *1000 ; + t =kzalloc (sizeof (*t),KSMBD_DEFAULT_GFP ); +if (!t) +return NULL ; + sc = &t->socket; +smbdirect_socket_prepare_create (sc, sp, smb_direct_wq); +/* +* from here we operate on the copy. +*/ + sp = &sc->parameters; + +INIT_WORK (&sc->disconnect_work, smb_direct_disconnect_rdma_work); + sc->rdma.cm_id = cm_id; cm_id->context = sc;
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH v4 98/145 smb: server: make use of smbdirect_socket_set_logging()
In reply to Stefan MetzmacherThis will allow the logging to keep working as before, when we move to common functions in the next commits.
--- fs/smb/server/transport_rdma.c | 62 ++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+)
fs/smb/server/transport_rdma.c +62 -0
@@ -113,6 +113,65 @@ struct smb_direct_transport {struct smbdirect_socket socket; }; +static bool smb_direct_logging_needed (struct smbdirect_socket *sc, +void *private_ptr, +unsigned int lvl, +unsigned int cls) +{ +if (lvl <=SMBDIRECT_LOG_ERR ) +return true; + +if (lvl >SMBDIRECT_LOG_INFO ) +return false; + +switch (cls) { +/* +* These were more or less also logged before +* the move to common code. +* +* SMBDIRECT_LOG_RDMA_MR was not used, but +* that's client only code and we should +* notice if it's used on the server... +*/ +case SMBDIRECT_LOG_RDMA_EVENT : +case SMBDIRECT_LOG_RDMA_SEND : +case SMBDIRECT_LOG_RDMA_RECV : +case SMBDIRECT_LOG_WRITE : +case SMBDIRECT_LOG_READ : +case SMBDIRECT_LOG_NEGOTIATE : +case SMBDIRECT_LOG_OUTGOING : +case SMBDIRECT_LOG_RDMA_RW : +case SMBDIRECT_LOG_RDMA_MR : +return true; +/* +* These were not logged before the move +* to common code. +*/ +case SMBDIRECT_LOG_KEEP_ALIVE : +case SMBDIRECT_LOG_INCOMING : +return false; + } + +/* +* Log all unknown messages +*/ +return true; +} + +static void smb_direct_logging_vaprintf (struct smbdirect_socket *sc, +const char *func, +unsigned int line, +void *private_ptr, +unsigned int lvl, +unsigned int cls, +struct va_format *vaf) +{ +if (lvl <=SMBDIRECT_LOG_ERR ) +pr_err ("%pV" , vaf); +else +ksmbd_debug (RDMA ,"%pV" , vaf); +} +#define KSMBD_TRANS (t) (&(t)->transport)#define SMBD_TRANS (t) (container_of(t, \ struct smb_direct_transport, transport))
@@ -429,6 +488,9 @@ static struct smb_direct_transport *alloc_transport(struct rdma_cm_id *cm_id)return NULL ; sc = &t->socket;smbdirect_socket_prepare_create (sc, sp, smb_direct_wq); +smbdirect_socket_set_logging (sc,NULL , + smb_direct_logging_needed, + smb_direct_logging_vaprintf);/* * from here we operate on the copy. */
Signature · 1 line
2.43.0