IB selinux related fixes
Subsystems (2)
cover IB selinux related fixes
From: Leon Romanovsky <[email protected]>
Hi,
This is second version of previously sent patch "IB/core: Don't register MAD agents for LSM notifications", due to the fact that we found extra things to fix during the review, I grouped that patch together with other fixes in that area.
Thanks
Daniel Jurgens (4):
IB/core: Unregister notifier before freeing MAD security IB/core: Fix potential memory leak while creating MAD agents IB/core: Eliminate a hole in MAD agent struct IB/core: Don't register each MAD agent for LSM notifier
drivers/infiniband/core/core_priv.h | 5 +++ drivers/infiniband/core/device.c | 1 + drivers/infiniband/core/security.c | 57 +++++++++++++++++------------ include/rdma/ib_mad.h | 5 +-- 4 files changed, 42 insertions(+), 26 deletions(-)
--
2.19.1
PATCH 1/4 IB/core: Unregister notifier before freeing MAD security
From: Daniel Jurgens <[email protected]>
If the notifier runs after the security context is freed an access of freed memory can occur.
--- drivers/infiniband/core/security.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/infiniband/core/security.c +2 -1
@@ -727,9 +727,10 @@ void ib_mad_agent_security_cleanup(struct ib_mad_agent *agent)if (!rdma_protocol_ib (agent->device, agent->port_num))return ; -security_ib_free_security (agent->security);if (agent->lsm_nb_reg)unregister_lsm_notifier (&agent->lsm_nb); + +security_ib_free_security (agent->security); }int ib_mad_enforce_security (struct ib_mad_agent_private *map,u16 pkey_index)
--
2.19.1
PATCH 2/4 IB/core: Fix potential memory leak while creating MAD agents
In reply to Leon RomanovskyFrom: Daniel Jurgens <[email protected]>
If the MAD agents isn't allowed to manage the subnet, or fails to register for the LSM notifier, the security context is leaked. Free the context in these cases.
--- drivers/infiniband/core/security.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/infiniband/core/security.c +6 -2
@@ -710,16 +710,20 @@ int ib_mad_agent_security_setup(struct ib_mad_agent *agent,dev_name (&agent->device->dev), agent->port_num);if (ret) -return ret; + goto free_security; agent->lsm_nb.notifier_call = ib_mad_agent_security_change; ret =register_lsm_notifier (&agent->lsm_nb);if (ret) -return ret; + goto free_security; agent->smp_allowed = true; agent->lsm_nb_reg = true;return 0 ; + +free_security: +security_ib_free_security (agent->security); +return ret; }void ib_mad_agent_security_cleanup (struct ib_mad_agent *agent)
--
2.19.1
PATCH 3/4 IB/core: Eliminate a hole in MAD agent struct
In reply to Leon RomanovskyFrom: Daniel Jurgens <[email protected]>
Move the security related fields above the u8s to eliminate a hole in the struct.
pahole before:
struct ib_mad_agent {
...
u32 hi_tid; /* 48 4 */ u32 flags; /* 52 4 */ u8 port_num; /* 56 1 */ u8 rmpp_version; /* 57 1 */
/* XXX 6 bytes hole, try to pack */
/* --- cacheline 1 boundary (64 bytes) --- */
void * security; /* 64 8 */ bool smp_allowed; /* 72 1 */ bool lsm_nb_reg; /* 73 1 */
/* XXX 6 bytes hole, try to pack */
struct notifier_block lsm_nb; /* 80 24 */
/* XXX last struct has 4 bytes of padding */
/* size: 104, cachelines: 2, members: 14 */
...
};
pahole after:
struct ib_mad_agent {
...
u32 hi_tid; /* 48 4 */ u32 flags; /* 52 4 */ void * security; /* 56 8 */
/* --- cacheline 1 boundary (64 bytes) --- */
struct notifier_block lsm_nb; /* 64 24 */
/* XXX last struct has 4 bytes of padding */
u8 port_num; /* 88 1 */ u8 rmpp_version; /* 89 1 */ bool smp_allowed; /* 90 1 */ bool lsm_nb_reg; /* 91 1 */
/* size: 96, cachelines: 2, members: 14 */
...
};
--- include/rdma/ib_mad.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
include/rdma/ib_mad.h +2 -2
@@ -616,12 +616,12 @@ struct ib_mad_agent {void *context;u32 hi_tid;u32 flags; +void *security; +struct notifier_block lsm_nb;u8 port_num;u8 rmpp_version; -void *security;bool smp_allowed;bool lsm_nb_reg; -struct notifier_block lsm_nb; };/**
--
2.19.1
PATCH 4/4 IB/core: Don't register each MAD agent for LSM notifier
In reply to Leon RomanovskyFrom: Daniel Jurgens <[email protected]>
When creating many MAD agents in a short period of time, receive packet processing can be delayed long enough to cause timeouts while new agents are being added to the atomic notifier chain with IRQs disabled. Notifier chain registration and unregstration is an O(n) operation. With large numbers of MAD agents being created and destroyed simultaneously the CPUs spend too much time with interrupts disabled.
Instead of each MAD agent registering for it's own LSM notification, maintain a list of agents internally and register once, this registration already existed for handling the PKeys. This list is write mostly, so a normal spin lock is used vs a read/write lock. All MAD agents must be checked, so a single list is used instead of breaking them down per device.
Notifier calls are done under rcu_read_lock, so there isn't a risk of similar packet timeouts while checking the MAD agents security settings when notified.
--- drivers/infiniband/core/core_priv.h | 5 +++ drivers/infiniband/core/device.c | 1 + drivers/infiniband/core/security.c | 50 ++++++++++++++++------------- include/rdma/ib_mad.h | 3 +- 4 files changed, 35 insertions(+), 24 deletions(-)
drivers/infiniband/core/core_priv.h +5 -0
@@ -199,6 +199,7 @@ int ib_mad_agent_security_setup(struct ib_mad_agent *agent, enum ib_qp_type qp_type);void ib_mad_agent_security_cleanup (struct ib_mad_agent *agent);int ib_mad_enforce_security (struct ib_mad_agent_private *map,u16 pkey_index); +void ib_mad_agent_security_change (void );#else static inline void ib_security_destroy_port_pkey_list (struct ib_device *device) {
@@ -264,6 +265,10 @@ static inline int ib_mad_enforce_security(struct ib_mad_agent_private *map, {return 0 ; } + +static inline void ib_mad_agent_security_change (void ) +{ +}#endif struct ib_device *ib_device_get_by_index (u32 ifindex);
drivers/infiniband/core/device.c +1 -0
@@ -455,6 +455,7 @@ static int ib_security_change(struct notifier_block *nb, unsigned long event,return NOTIFY_DONE ;schedule_work (&ib_policy_change_work); +ib_mad_agent_security_change ();return NOTIFY_OK ; }
drivers/infiniband/core/security.c +28 -22
96 lines not on this page · raw message
include/rdma/ib_mad.h +1 -2
@@ -617,11 +617,10 @@ struct ib_mad_agent {u32 hi_tid;u32 flags;void *security; -struct notifier_block lsm_nb; +struct list_head mad_agent_sec_list;u8 port_num;u8 rmpp_version;bool smp_allowed; -bool lsm_nb_reg; };/**
--
2.19.1
Re: [PATCH rdma-rc v1 4/4] IB/core: Don't register each MAD agent for LSM notifier
Quoted 26 lines · Leon Romanovsky
From: Daniel Jurgens <[email protected]>
When creating many MAD agents in a short period of time, receive packet processing can be delayed long enough to cause timeouts while new agents are being added to the atomic notifier chain with IRQs disabled. Notifier chain registration and unregstration is an O(n) operation. With large numbers of MAD agents being created and destroyed simultaneously the CPUs spend too much time with interrupts disabled.
Instead of each MAD agent registering for it's own LSM notification, maintain a list of agents internally and register once, this registration already existed for handling the PKeys. This list is write mostly, so a normal spin lock is used vs a read/write lock. All MAD agents must be checked, so a single list is used instead of breaking them down per device.
Notifier calls are done under
rcu_read_lock, so there isn't a risk of similar packet timeouts while checking the MAD agents security settings when notified.Signed-off-by Daniel Jurgens <[email protected]>Reviewed-by Parav Pandit <[email protected]>Signed-off-by Leon Romanovsky <[email protected]>
Quoted 6 lines
--- drivers/infiniband/core/core_priv.h | 5 +++ drivers/infiniband/core/device.c | 1 + drivers/infiniband/core/security.c | 50 ++++++++++++++++------------- include/rdma/ib_mad.h | 3 +- 4 files changed, 35 insertions(+), 24 deletions(-)
This looks reasonable from a SELinux perspective. I would still be curious to see how this would compare to calling security_ib_endport_manage_subnet() directly in ip_mad_enforce_security(), but at least this patch should fix the problem you're seeing.
Quoted 161 lines
drivers/infiniband/core/core_priv.h +5
-0@@ -199,6 +199,7 @@ int ib_mad_agent_security_setup(struct ib_mad_agent *agent, enum ib_qp_type qp_type);void ib_mad_agent_security_cleanup (struct ib_mad_agent *agent);int ib_mad_enforce_security (struct ib_mad_agent_private *map,u16 pkey_index); +void ib_mad_agent_security_change (void );#else static inline void ib_security_destroy_port_pkey_list (struct ib_device *device) {@@ -264,6 +265,10 @@ static inline int ib_mad_enforce_security(struct ib_mad_agent_private *map, {return 0 ; } + +static inline void ib_mad_agent_security_change (void ) +{ +}#endif struct ib_device *ib_device_get_by_index (u32 ifindex);drivers/infiniband/core/device.c +1
-0@@ -455,6 +455,7 @@ static int ib_security_change(struct notifier_block *nb, unsigned long event,return NOTIFY_DONE ;schedule_work (&ib_policy_change_work); +ib_mad_agent_security_change ();return NOTIFY_OK ; }drivers/infiniband/core/security.c +28
-22@@ -39,6 +39,10 @@#include "core_priv.h" #include "mad_priv.h" +static LIST_HEAD (mad_agent_list); +/* Lock to protect mad_agent_list */ +static DEFINE_SPINLOCK (mad_agent_list_lock); +static struct pkey_index_qp_list *get_pkey_idx_qp_list (struct ib_port_pkey *pp) {struct pkey_index_qp_list *pkey =NULL ;@@ -676,19 +680,18 @@ static int ib_security_pkey_access(struct ib_device *dev,return security_ib_pkey_access (sec, subnet_prefix, pkey); } -static int ib_mad_agent_security_change (struct notifier_block *nb, -unsigned long event, -void *data) +void ib_mad_agent_security_change (void ) { - struct ib_mad_agent *ag = container_of(nb, struct ib_mad_agent, lsm_nb); - -if (event !=LSM_POLICY_CHANGE ) -return NOTIFY_DONE ; - - ag->smp_allowed = !security_ib_endport_manage_subnet ( - ag->security,dev_name (&ag->device->dev), ag->port_num); - -return NOTIFY_OK ; +struct ib_mad_agent *ag; + +spin_lock (&mad_agent_list_lock); +list_for_each_entry (ag, + &mad_agent_list, + mad_agent_sec_list) +WRITE_ONCE (ag->smp_allowed, + !security_ib_endport_manage_subnet (ag->security, +dev_name (&ag->device->dev), ag->port_num)); +spin_unlock (&mad_agent_list_lock); } int ib_mad_agent_security_setup(struct ib_mad_agent *agent,@@ -699,6 +702,8 @@ int ib_mad_agent_security_setup(struct ib_mad_agent *agent,if (!rdma_protocol_ib (agent->device, agent->port_num))return 0 ; +INIT_LIST_HEAD (&agent->mad_agent_sec_list); + ret =security_ib_alloc_security (&agent->security);if (ret)return ret;@@ -706,22 +711,20 @@ int ib_mad_agent_security_setup(struct ib_mad_agent *agent,if (qp_type !=IB_QPT_SMI )return 0 ; +spin_lock (&mad_agent_list_lock); ret =security_ib_endport_manage_subnet (agent->security,dev_name (&agent->device->dev), agent->port_num);if (ret) goto free_security; - agent->lsm_nb.notifier_call = ib_mad_agent_security_change; - ret =register_lsm_notifier (&agent->lsm_nb); -if (ret) - goto free_security; - - agent->smp_allowed = true; - agent->lsm_nb_reg = true; +WRITE_ONCE (agent->smp_allowed, true); +list_add (&agent->mad_agent_sec_list, &mad_agent_list); +spin_unlock (&mad_agent_list_lock);return 0 ; free_security: +spin_unlock (&mad_agent_list_lock);security_ib_free_security (agent->security);return ret; }@@ -731,8 +734,11 @@ void ib_mad_agent_security_cleanup(struct ib_mad_agent *agent)if (!rdma_protocol_ib (agent->device, agent->port_num))return ; -if (agent->lsm_nb_reg) -unregister_lsm_notifier (&agent->lsm_nb); +if (agent->qp->qp_type ==IB_QPT_SMI ) { +spin_lock (&mad_agent_list_lock); +list_del (&agent->mad_agent_sec_list); +spin_unlock (&mad_agent_list_lock); + }security_ib_free_security (agent->security); }@@ -743,7 +749,7 @@ int ib_mad_enforce_security(struct ib_mad_agent_private *map, u16 pkey_index)return 0 ;if (map->agent.qp->qp_type ==IB_QPT_SMI ) { -if (!map->agent.smp_allowed) +if (!READ_ONCE (map->agent.smp_allowed))return -EACCES ;return 0 ; }include/rdma/ib_mad.h +1
-3@@ -617,11 +617,10 @@ struct ib_mad_agent {u32 hi_tid;u32 flags;void *security; -struct notifier_block lsm_nb; +struct list_head mad_agent_sec_list;u8 port_num;u8 rmpp_version;bool smp_allowed; -bool lsm_nb_reg; };/** -- 2.19.1
Signature · 2 lines
paul moore www.paul-moore.com
Jason Gunthorpe
AppliedQuoted 9 lines · Leon Romanovsky
From: Leon Romanovsky <[email protected]>
Hi,
This is second version of previously sent patch "IB/core: Don't register MAD agents for LSM notifications", due to the fact that we found extra things to fix during the review, I grouped that patch together with other fixes in that area.
Quoted 7 lines
Thanks
Daniel Jurgens (4):
IB/core: Unregister notifier before freeing MAD security IB/core: Fix potential memory leak while creating MAD agents IB/core: Eliminate a hole in MAD agent struct IB/core: Don't register each MAD agent for LSM notifier
Applied to for-next
Thanks,
Jason