rdma_cm resource over use
Hi,
the rdma_cm library opens all availabe devices and keeps them open even after binding to a specific device/port. This eats resources from other devices that would otherwise be available to other applications. Is there a way to avoid this? If not, maybe we should close all other devices after binfing?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 5 lines
the
rdma_cmlibrary opens all availabe devices and keeps them open even after binding to a specific device/port. This eats resources from other devices that would otherwise be available to other applications. Is there a way to avoid this? If not, maybe we should close all other devices after binfing?
The librdmacm documents that all RDMA devices remain open while the librdmacm is loaded (see rdma_get_devices). Also, a listen doesn't need to be bound to any specific device, but the connection request will be. Personally, I really don't see this as a major use of resources.
- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A message that is not in the archive (sent off-list or not yet received): CF9C39F99A89134C9CF9C4CCB68B8DDF25B8C1F52B-osO9UTpF0USkrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org
For ConnectX, mlx4 will consume a UAR page for each open of libibverbs device file. On a 4K page architectures we usually have 1024 UAR pages. On a 64K page size the situation becomes worth. Suppose you have a system with more than one device. One of the devices is used by rdam_cm and the other is not. You can easily exhaust all the resources. We actually have this situation at a customer.
Quoted 8 lines · Sean
Quoted 5 lines
the
rdma_cmlibrary opens all availabe devices and keeps them open even after binding to a specific device/port. This eats resources from other devices that would otherwise be available to other applications. Is there a way to avoid this? If not, maybe we should close all other devices after binfing?The librdmacm documents that all RDMA devices remain open while the librdmacm is loaded (see
rdma_get_devices). Also, a listen doesn't need to be bound to any specific device, but the connection request will be. Personally, I really don't see this as a major use of resources.
Quoted 8 lines
- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html