Skip to contentSkip to conversation

ipath/qib deadlock

PATCH New Mike Marciniszyn 2 participants 4 messages Oct 4, 2013 2 messages not in the archive InfiniBand

lore
Share & tools
lore
#0 lore raw

cover ipath/qib deadlock

These two patches fix flaws that have been
observed in the wild with qib.

The issue has also been code inspected by
Jan Kara in his get_user_pages_unlocked()
patch series.

By using get_user_pages_fast() the dependency
on Jan's patch set has been removed. I can't
find any issue with losing the force in these
use cases.

Jan Kara (2):
IB/ipath: Convert ipath_user_sdma_pin_pages() to use get_user_pages_fast() IB/qib: Convert qib_user_sdma_pin_pages() to use get_user_pages_fast()

 drivers/infiniband/hw/ipath/ipath_user_sdma.c |    7 +------
 drivers/infiniband/hw/qib/qib_user_sdma.c     |    6 +-----
 2 files changed, 2 insertions(+), 11 deletions(-)
Signature · 5 lines
Mike
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20131004132821.29162.46169.stgit-K+u1se/DcYrLESAwzcoQNrvm/[email protected]

#2 lore raw

PATCH 1/2 IB/ipath: Convert ipath_user_sdma_pin_pages() to use get_user_pages_fast()

From: Jan Kara <[email protected]>

Function ipath_user_sdma_queue_pkts() gets called with mmap_sem held for writing. Except for get_user_pages() deep down in ipath_user_sdma_pin_pages() we don't seem to need mmap_sem at all. Even more interestingly the function ipath_user_sdma_queue_pkts() (and also ipath_user_sdma_coalesce() called somewhat later) call copy_from_user() which can hit a page fault and we deadlock on trying to get mmap_sem when handling that fault. So just make ipath_user_sdma_pin_pages() use get_user_pages_fast() and leave mmap_sem locking for mm.

This deadlock has actually been observed in the wild when the node is under memory pressure.

---
 drivers/infiniband/hw/ipath/ipath_user_sdma.c |    7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
drivers/infiniband/hw/ipath/ipath_user_sdma.c +1 -6
index f5cb13b..06cbfd4 100644
@@ -280,9 +280,7 @@ static int ipath_user_sdma_pin_pages(const struct ipath_devdata *dd,
 	int j;
 	int ret;
 
-	ret = get_user_pages(current, current->mm, addr,
-			     npages, 0, 1, pages, NULL);
-
+	ret = get_user_pages_fast(addr, j, 0, pages);
 	if (ret != npages) {
 		int i;
 
@@ -811,10 +809,7 @@ int ipath_user_sdma_writev(struct ipath_devdata *dd,
 	while (dim) {
 		const int mxp = 8;
 
-		down_write(&current->mm->mmap_sem);
 		ret = ipath_user_sdma_queue_pkts(dd, pq, &list, iov, dim, mxp);
-		up_write(&current->mm->mmap_sem);
-
 		if (ret <= 0)
 			goto done_unlock;
 		else {

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20131004132906.29162.1038.stgit-K+u1se/DcYrLESAwzcoQNrvm/[email protected]

#4 lore raw

RE: [PATCH 1/2] IB/ipath: Convert ipath_user_sdma_pin_pages() to use get_user_pages_fast()

PiBFdmVuIG1vcmUgaW50ZXJlc3RpbmdseSB0aGUgZnVuY3Rpb24gaXBhdGhfdXNlcl9zZG1hX3F1 ZXVlX3BrdHMoKSAoYW5kDQo+IGFsc28NCj4gaXBhdGhfdXNlcl9zZG1hX2NvYWxlc2NlKCkgY2Fs bGVkIHNvbWV3aGF0IGxhdGVyKSBjYWxsIGNvcHlfZnJvbV91c2VyKCkNCj4gd2hpY2ggY2FuIGhp dCBhIHBhZ2UgZmF1bHQgYW5kIHdlIGRlYWRsb2NrIG9uIHRyeWluZyB0byBnZXQgbW1hcF9zZW0N Cj4gd2hlbiBoYW5kbGluZyB0aGF0IGZhdWx0LiBTbyBqdXN0IG1ha2UgaXBhdGhfdXNlcl9zZG1h X3Bpbl9wYWdlcygpIHVzZQ0KPiBnZXRfdXNlcl9wYWdlc19mYXN0KCkgYW5kIGxlYXZlIG1tYXBf c2VtIGxvY2tpbmcgZm9yIG1tLg0KPiANCj4gVGhpcyBkZWFkbG9jayBoYXMgYWN0dWFsbHkgYmVl biBvYnNlcnZlZCBpbiB0aGUgd2lsZCB3aGVuIHRoZSBub2RlIGlzIHVuZGVyDQo+IG1lbW9yeSBw cmVzc3VyZS4NCg0KVGhpcyBwYXRjaCBoYXMgYmVlbiByZS1zdWJtaXRlZCBpbiBodHRwOi8vbWFy Yy5pbmZvLz9sPWxpbnV4LXJkbWEmbT0xMzgwODkzMzUxMDYzNTMmdz0yLg0KDQpNaWtlDQo= --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#5 lore raw

PATCH 2/2 IB/qib: Convert qib_user_sdma_pin_pages() to use get_user_pages_fast()

In reply to a message not in the archive

From: Jan Kara <[email protected]>

Function qib_user_sdma_queue_pkts() gets called with mmap_sem held for writing. Except for get_user_pages() deep down in qib_user_sdma_pin_pages() we don't seem to need mmap_sem at all. Even more interestingly the function qib_user_sdma_queue_pkts() (and also qib_user_sdma_coalesce() called somewhat later) call copy_from_user() which can hit a page fault and we deadlock on trying to get mmap_sem when handling that fault. So just make qib_user_sdma_pin_pages() use get_user_pages_fast() and leave mmap_sem locking for mm.

This deadlock has actually been observed in the wild when the node is under memory pressure.

---
 drivers/infiniband/hw/qib/qib_user_sdma.c |    6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)
drivers/infiniband/hw/qib/qib_user_sdma.c +1 -5
index d0a0ea0..165aee2 100644
@@ -594,8 +594,7 @@ static int qib_user_sdma_pin_pages(const struct qib_devdata *dd,
 		else
 			j = npages;
 
-		ret = get_user_pages(current, current->mm, addr,
-			     j, 0, 1, pages, NULL);
+		ret = get_user_pages_fast(addr, j, 0, pages);
 		if (ret != j) {
 			i = 0;
 			j = ret;
@@ -1294,11 +1293,8 @@ int qib_user_sdma_writev(struct qib_ctxtdata *rcd,
 		int mxp = 8;
 		int ndesc = 0;
 
-		down_write(&current->mm->mmap_sem);
 		ret = qib_user_sdma_queue_pkts(dd, ppd, pq,
 				iov, dim, &list, &mxp, &ndesc);
-		up_write(&current->mm->mmap_sem);
-
 		if (ret < 0)
 			goto done_unlock;
 		else {

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html