RDMA/cm: Read the remote_id_table root under cm.lock
PATCH 1/1 RDMA/cm: Read the remote_id_table root under cm.lock
cm_find_remote_id() reads the root of cm.remote_id_table before it takes cm.lock. While it waits for the lock, cm_destroy_id() or cm_reset_to_idle() on another CPU can remove that cm_timewait_info from the tree and free it, and the lookup then starts from freed memory. This lookup is done for a REJ with reason IB_CM_REJ_TIMEOUT.
Read the root after taking cm.lock.
---
Found with a static analysis tool we are developing.
Reproduced in QEMU (x86_64, KASAN) on rdma_rxe over virtio-net, with a test module that connects to a listener on the same device through rdma_cm and destroys each active cm_id while it is in IB_CM_REQ_SENT, so a REJ with IB_CM_REJ_TIMEOUT is sent. Another kthread destroys the passive cm_ids. The window was widened for the test with an msleep(20) between reading the root and taking cm.lock. Without this patch, KASAN reported 46 to 48 slab-use-after-free reads in cm_work_handler() per 20 second run in three runs, on a cm_timewait_info allocated by cm_create_timewait_info() and freed by cm_destroy_id() through cm_reset_to_idle(). With it, there was no KASAN report in three runs.
drivers/infiniband/core/cm.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/infiniband/core/cm.c +2 -1
@@ -734,11 +734,12 @@ cm_insert_remote_id(struct cm_timewait_info *timewait_info)static struct cm_id_private *cm_find_remote_id (__be64 remote_ca_guid,__be32 remote_id) { -struct rb_node *node = cm.remote_id_table.rb_node; +struct rb_node *node;struct cm_timewait_info *timewait_info;struct cm_id_private *res =NULL ;spin_lock_irq (&cm.lock); + node = cm.remote_id_table.rb_node;while (node) { timewait_info = rb_entry(node, struct cm_timewait_info, remote_id_node);
base-commit: 7475ea8b58616b33c04afb976cf0fbaad9dd63fb
Signature · 1 line
2.43.0
[email protected]
AI reviewSashiko has reviewed this patch and found no issues.
Quoted 10 lines
cm_find_remote_id()reads the root of cm.remote_id_table before it takes cm.lock. While it waits for the lock,cm_destroy_id()orcm_reset_to_idle()on another CPU can remove thatcm_timewait_infofrom the tree and free it, and the lookup then starts from freed memory. This lookup is done for a REJ with reasonIB_CM_REJ_TIMEOUT.Read the root after taking cm.lock.
Fixes cfa68b0d0440 ("RDMA/cm: Make find_remote_id() return a cm_id_private")Signed-off-by Changyul Lee <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Quoted 13 lines · Changyul Lee
cm_find_remote_id()reads the root of cm.remote_id_table before it takes cm.lock. While it waits for the lock,cm_destroy_id()orcm_reset_to_idle()on another CPU can remove thatcm_timewait_infofrom the tree and free it, and the lookup then starts from freed memory. This lookup is done for a REJ with reasonIB_CM_REJ_TIMEOUT.Read the root after taking cm.lock.
Fixes cfa68b0d0440 ("RDMA/cm: Make find_remote_id() return a cm_id_private")Signed-off-by Changyul Lee <[email protected]>---
Found with a static analysis tool we are developing.
Quoted 10 lines
Reproduced in QEMU (
x86_64, KASAN) onrdma_rxeover virtio-net, with a test module that connects to a listener on the same device throughrdma_cmand destroys each activecm_idwhile it is inIB_CM_REQ_SENT, so a REJ withIB_CM_REJ_TIMEOUTis sent. Another kthread destroys the passivecm_ids. The window was widened for the test with an msleep(20) between reading the root and taking cm.lock. Without this patch, KASAN reported 46 to 48 slab-use-after-free reads incm_work_handler()per 20 second run in three runs, on acm_timewait_infoallocated bycm_create_timewait_info()and freed bycm_destroy_id()throughcm_reset_to_idle(). With it, there was no KASAN report in three runs.
I'll take this patch, since it's simple enough. However, using a test module that misuses an in-kernel API is not the right approach.
Thanks
Quoted 27 lines
drivers/infiniband/core/cm.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/infiniband/core/cm.c +2
-1@@ -734,11 +734,12 @@ cm_insert_remote_id(struct cm_timewait_info *timewait_info)static struct cm_id_private *cm_find_remote_id (__be64 remote_ca_guid,__be32 remote_id) { -struct rb_node *node = cm.remote_id_table.rb_node; +struct rb_node *node;struct cm_timewait_info *timewait_info;struct cm_id_private *res =NULL ;spin_lock_irq (&cm.lock); + node = cm.remote_id_table.rb_node;while (node) { timewait_info = rb_entry(node, struct cm_timewait_info, remote_id_node);base-commit:
7475ea8b58616b33c04afb976cf0fbaad9dd63fbSignature · 1 line
2.43.0
Leon Romanovsky
AppliedQuoted 9 lines · Changyul Lee
cm_find_remote_id()reads the root of cm.remote_id_table before it takes cm.lock. While it waits for the lock,cm_destroy_id()orcm_reset_to_idle()on another CPU can remove thatcm_timewait_infofrom the tree and free it, and the lookup then starts from freed memory. This lookup is done for a REJ with reasonIB_CM_REJ_TIMEOUT.Read the root after taking cm.lock.
[...]
Applied, thanks!
[1/1] RDMA/cm: Read the remote_id_table root under cm.lock
https://git.kernel.org/rdma/rdma/c/a6b0e712b0fccc
Best regards,
Signature · 1 line
Leon Romanovsky <[email protected]>