smbdirect: don't hang on netdev reconfiguration
Status from this message: Applied, thanks!
Get series with b4
b4 am [email protected]Versions newest first
Patches in v1 Applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/2 | infiniband: force disconnect if DREP and DREQ faildrivers/ | +35 | 0 replies | |
| 2/2 | infiniband: trace force disconnectionsdrivers/ drivers/ | +3 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.
Cover letter [PATCH 0/2] smbdirect: don't hang on netdev reconfiguration
When I mount a CIFS share using SMB Direct over RoCEv2, I observe a kernel thread hang if I "configure" its slave network device by taking its link down and bringing it back up. Attempting to just ls the mount point in this state gives EHOSTDOWN.
I believe the following is the root-cause of the hang: When the link is taken down, the corresponding GID Table Entry is marked as pending deletion and has its slave ndev set to NULL. All sends on RDMA connections still using that GID Table Entry fail at MAD creation. SMB Direct eventually detects this and tries to disconnect / reconnect to recover. Unfortunately, disconnecting requires successfully sending either a DREQ or a DREP. Since neither of them even post, the connection remains in the RDMA_CM_CONNECT state, and no callback moves it out. The SMB Direct layer never gets the RDMA_CM_EVENT_DISCONNECTED it's waiting for, and hangs.
Fix this in the CMA. If we call rdma_disconnect on a connected connection and we fail to send both a DREP and a DREQ; disconnect, and thereby send the RDMA_CM_EVENT_DISCONNECTED event to SMB Direct.
I tested this change in QEMU using RXE. I ran Ubuntu 26.04.1 with a mainline kernel. On commit 9f0346dcbea3 ("Merge tag 'driver-core-7.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/driver-core/driver-core"),
I reproduce the hang. With this patch applied, SMB Direct immediately disconnects and reconnects when the slave device is brought down then up. Listing and reading files from the mount point also work afterwards.
Artifacts for reproducing the hang and testing my fix are available at
https://github.com/ammrat13/linux-cifs
There, you can find: logs from the "bad" and "good" kernels, the kernel configuration, the VMs' configuration, and a playbook to trigger the hang.
Ammar Ratnani (2):
infiniband: force disconnect if DREP and DREQ fail
infiniband: trace force disconnections
drivers/infiniband/core/cma.c | 41 ++++++++++++++++++++++++++--- drivers/infiniband/core/cma_trace.h | 1 + 2 files changed, 38 insertions(+), 4 deletions(-)
Signature · 1 line
2.55.0