rapidio: Fix error handling path
Subsystems (3)
PATCH 1/1 rapidio: Fix error handling path
There is an error when pin_user_pages_fast() returns -ERRNO and inside error handling path driver end up calling unpin_user_pages() with -ERRNO which is not correct.
This patch will fix the problem.
--- drivers/rapidio/devices/rio_mport_cdev.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)
drivers/rapidio/devices/rio_mport_cdev.c +7 -6
@@ -871,15 +871,16 @@ static int do_dma_request(struct mport_dma_req *req,rmcd_error ("pin_user_pages_fast err=%ld" , pinned); nr_pages =0 ; - }else + }else {rmcd_error ("pinned %ld out of %ld pages" , pinned, nr_pages); +/* +* Set nr_pages up to mean "how many pages to unpin, in +* the error handler: +*/ + nr_pages = pinned; + } ret = -EFAULT ; -/* -* Set nr_pages up to mean "how many pages to unpin, in -* the error handler: -*/ - nr_pages = pinned; goto err_pg; }
Signature · 1 line
1.9.1
Quoted 4 lines · Souptick Joarder
There is an error when
pin_user_pages_fast()returns-ERRNOand inside error handling path driver end up callingunpin_user_pages()with-ERRNOwhich is not correct.Quoted 1 line
This patch will fix the problem.
How about:
rio_dma_transfer() attempts to clamp the return value of pin_user_pages_fast() to be >= 0. However, the attempt fails because nr_pages is overridden a few lines later, and restored to the undesirable -ERRNO value.
The return value is ultimately stored in nr_pages, which in turn is passed to unpin_user_pages(), which expects nr_pages >= 0, else, disaster.
Fix this by fixing the nesting of the assignment to nr_pages: nr_pages should be clamped to zero if pin_user_pages_fast() returns -ERRNO, or set to the return value of pin_user_pages_fast(), otherwise.
Quoted 28 lines
Fixes e8de370188d09 ("rapidio: add mport char device driver")Signed-off-by Souptick Joarder <[email protected]>Cc Ira Weiny <[email protected]>Cc John Hubbard <[email protected]>Cc Matthew Wilcox <[email protected]>--- drivers/rapidio/devices/rio_mport_cdev.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)drivers/rapidio/devices/rio_mport_cdev.c +6
-1@@ -871,15 +871,16 @@ static int do_dma_request(struct mport_dma_req *req,rmcd_error ("pin_user_pages_fast err=%ld" , pinned); nr_pages =0 ; - }else + }else {rmcd_error ("pinned %ld out of %ld pages" , pinned, nr_pages); +/* +* Set nr_pages up to mean "how many pages to unpin, in +* the error handler: +*/ + nr_pages = pinned;
drivers/rapidio/devices/rio_mport_cdev.c static int do_dma_request(struct mport_dma_req *req, · line 886
Quoted 10 lines
drivers/rapidio/devices/rio_mport_cdev.c +1
-5+ } ret = -EFAULT ; -/* -* Set nr_pages up to mean "how many pages to unpin, in -* the error handler: -*/ - nr_pages = pinned; goto err_pg; }
thanks,
Signature · 2 lines
John Hubbard NVIDIA
Quoted 5 lines · Souptick Joarder
There is an error when
pin_user_pages_fast()returns-ERRNOand inside error handling path driver end up callingunpin_user_pages()with-ERRNOwhich is not correct.This patch will fix the problem.
There are a few ways we could prevent bug in the future.
1) This could have been caught with existing static analysis tools which warn about when a value is set but not used.
2) I've created a Smatch check which warngs about:
drivers/rapidio/devices/rio_mport_cdev.c:955 rio_dma_transfer() warn: unpinning negative pages 'nr_pages'
I'll test it out tonight and see how well it works. I don't immediately see any other bugs allthough Smatch doesn't like the code in siw_umem_release(). It uses "min_t(int" which suggests that negative pages are okay.
int to_free = min_t(int, PAGES_PER_CHUNK, num_pages);
3) We could add a check in unpin_user_pages().
if (WARN_ON(IS_ERR_VALUE(npages))) return;
It's not possible to pin more than "ULONG_MAX - 4095" because otherwise returning error pointers wouldn't work. So this check can't break anything and it could prevent a crash.
regards,
dan carpenter
Quoted 23 lines · Dan Carpenter
Quoted 5 lines · Souptick Joarder
There is an error when
pin_user_pages_fast()returns-ERRNOand inside error handling path driver end up callingunpin_user_pages()with-ERRNOwhich is not correct.This patch will fix the problem.
There are a few ways we could prevent bug in the future.
1) This could have been caught with existing static analysis tools which warn about when a value is set but not used.
2) I've created a Smatch check which warngs about:
drivers/rapidio/devices/rio_mport_cdev.c:955rio_dma_transfer()warn: unpinning negative pages 'nr_pages'I'll test it out tonight and see how well it works. I don't immediately see any other bugs allthough Smatch doesn't like the code in
siw_umem_release(). It uses "min_t(int" which suggests that negative pages are okay.int to_free = min_t(int, PAGES_PER_CHUNK, num_pages);
Quoted 8 lines
3) We could add a check in
unpin_user_pages().if (WARN_ON(IS_ERR_VALUE(npages))) return;It's not possible to pin more than "
ULONG_MAX- 4095" because otherwise returning error pointers wouldn't work. So this check can't break anything and it could prevent a crash.
Actually pin_user_pages_fast() returns an int. It's not possible to pin more than INT_MAX.
regards,
dan carpenter
Quoted 21 lines · Dan Carpenter
Quoted 5 lines · Souptick Joarder
There is an error when
pin_user_pages_fast()returns-ERRNOand inside error handling path driver end up callingunpin_user_pages()with-ERRNOwhich is not correct.This patch will fix the problem.
There are a few ways we could prevent bug in the future.
1) This could have been caught with existing static analysis tools which warn about when a value is set but not used.
2) I've created a Smatch check which warngs about:
drivers/rapidio/devices/rio_mport_cdev.c:955rio_dma_transfer()warn: unpinning negative pages 'nr_pages'I'll test it out tonight and see how well it works. I don't immediately see any other bugs allthough Smatch doesn't like the code in
siw_umem_release(). It uses "min_t(int" which suggests that negative pages are okay.
Quoted 6 lines
int to_free = min_t(int, PAGES_PER_CHUNK, num_pages);3) We could add a check in
unpin_user_pages().if (WARN_ON(IS_ERR_VALUE(npages))) return;
Does IS_ERR_VALUE() work on an unsigned variable? The issue with adding a check in unpin_user_pages() is that npages is unsigned long.
Ira
Quoted 8 lines
It's not possible to pin more than "
ULONG_MAX- 4095" because otherwise returning error pointers wouldn't work. So this check can't break anything and it could prevent a crash.regards,
dan carpenter
Quoted 24 lines · Ira Weiny
Quoted 23 lines · Dan Carpenter
Quoted 5 lines · Souptick Joarder
There is an error when
pin_user_pages_fast()returns-ERRNOand inside error handling path driver end up callingunpin_user_pages()with-ERRNOwhich is not correct.This patch will fix the problem.
There are a few ways we could prevent bug in the future.
1) This could have been caught with existing static analysis tools which warn about when a value is set but not used.
2) I've created a Smatch check which warngs about:
drivers/rapidio/devices/rio_mport_cdev.c:955rio_dma_transfer()warn: unpinning negative pages 'nr_pages'I'll test it out tonight and see how well it works. I don't immediately see any other bugs allthough Smatch doesn't like the code in
siw_umem_release(). It uses "min_t(int" which suggests that negative pages are okay.int to_free = min_t(int, PAGES_PER_CHUNK, num_pages);
Quoted 7 lines
Quoted 4 lines
3) We could add a check in
unpin_user_pages().if (WARN_ON(IS_ERR_VALUE(npages))) return;Does
IS_ERR_VALUE()work on an unsigned variable? The issue with adding a check inunpin_user_pages()is that npages is unsigned long.
Yeah, it does. It only works on long and unsigned long.
(Or long long types if they have the same number of bytes as a long).
regards,
dan carpenter
John Hubbard
reviewerPATCH 1/1 mm/gup: protect unpin_user_pages() against npages==-ERRNO
In reply to Dan CarpenterAs suggested by Dan Carpenter, fortify unpin_user_pages() just a bit, against a typical caller mistake: check if the npages arg is really a -ERRNO value, which would blow up the unpinning loop: WARN and return.
If this new WARN_ON() fires, then the system *might* be leaking pages (by leaving them pinned), but probably not. More likely, gup/pup returned a hard -ERRNO error to the caller, who erroneously passed it here.
---
Hi Dan,
Is is OK to use your signed-off-by here? Since you came up with this.
thanks,
John Hubbard
NVIDIA
mm/gup.c | 7 +++++++ 1 file changed, 7 insertions(+)
mm/gup.c +7 -0
@@ -328,6 +328,13 @@ void unpin_user_pages(struct page **pages, unsigned long npages) {unsigned long index; +/* +* If this WARN_ON() fires, then the system *might* be leaking pages (by +* leaving them pinned), but probably not. More likely, gup/pup returned +* a hard -ERRNO error to the caller, who erroneously passed it here. +*/ +if (WARN_ON (IS_ERR_VALUE (npages))) +return ;/* * TODO: this can be optimized for huge pages: if a series of pages is * physically contiguous and part of the same compound page, then a
Signature · 1 line
2.28.0
Re: [PATCH] mm/gup: protect unpin_user_pages() against npages==-ERRNO
Quoted 13 lines · John Hubbard
As suggested by Dan Carpenter, fortify
unpin_user_pages()just a bit, against a typical caller mistake: check if the npages arg is really a-ERRNOvalue, which would blow up the unpinning loop: WARN and return.If this new
WARN_ON()fires, then the system *might* be leaking pages (by leaving them pinned), but probably not. More likely, gup/pup returned a hard-ERRNOerror to the caller, who erroneously passed it here.Cc Ira Weiny <[email protected]>Cc Souptick Joarder <[email protected]>Signed-off-by Dan Carpenter <[email protected]>Signed-off-by John Hubbard <[email protected]>
Quoted 6 lines
---
Hi Dan,
Is is OK to use your signed-off-by here? Since you came up with this.
Yeah. That's fine.
regards,
dan carpenter
Re: [PATCH] mm/gup: protect unpin_user_pages() against npages==-ERRNO
Quoted 17 lines · Dan Carpenter
Quoted 15 lines · John Hubbard
As suggested by Dan Carpenter, fortify
unpin_user_pages()just a bit, against a typical caller mistake: check if the npages arg is really a-ERRNOvalue, which would blow up the unpinning loop: WARN and return.If this new
WARN_ON()fires, then the system *might* be leaking pages (by leaving them pinned), but probably not. More likely, gup/pup returned a hard-ERRNOerror to the caller, who erroneously passed it here.Cc Ira Weiny <[email protected]>Cc Souptick Joarder <[email protected]>Signed-off-by Dan Carpenter <[email protected]>Signed-off-by John Hubbard <[email protected]>---
Quoted 6 lines
Quoted 4 lines
Hi Dan,
Is is OK to use your signed-off-by here? Since you came up with this.
Yeah. That's fine.
Do we need a similar check inside unpin_user_pages_dirty_lock(), when make_dirty set to false ?
John Hubbard
reviewerRe: [PATCH] mm/gup: protect unpin_user_pages() against npages==-ERRNO
Quoted 10 lines · Souptick Joarder
Quoted 9 lines · Dan Carpenter
Quoted 8 lines · John Hubbard
As suggested by Dan Carpenter, fortify
unpin_user_pages()just a bit, against a typical caller mistake: check if the npages arg is really a-ERRNOvalue, which would blow up the unpinning loop: WARN and return.If this new
WARN_ON()fires, then the system *might* be leaking pages (by leaving them pinned), but probably not. More likely, gup/pup returned a hard-ERRNOerror to the caller, who erroneously passed it here.
...
Quoted 3 lines
Do we need a similar check inside
unpin_user_pages_dirty_lock(), whenmake_dirtyset to false ?
Maybe not. This call is rarely if ever used for error handling, but rather, for finishing up a successful use of the pages.
There is a balance between protecting against buggy callers and just fixing any buggy callers. There is also a limit to how much code one can write in hopes of avoiding bugs in...code that one writes. :) Which is why static analysis, unit and regression tests, code reviews are important too.
Here, I submit that that we're about to cross the line and go too far. But if you have any examples of buggy callers for unpin_user_pages_dirty_lock(), that might shift the line.
Or maybe others feel that we haven't gone far enough yet after all?
thanks,
Signature · 2 lines
John Hubbard NVIDIA
Re: [PATCH] mm/gup: protect unpin_user_pages() against npages==-ERRNO
Quoted 26 lines · John Hubbard
Quoted 10 lines · Souptick Joarder
Quoted 9 lines · Dan Carpenter
Quoted 8 lines · John Hubbard
As suggested by Dan Carpenter, fortify
unpin_user_pages()just a bit, against a typical caller mistake: check if the npages arg is really a-ERRNOvalue, which would blow up the unpinning loop: WARN and return.If this new
WARN_ON()fires, then the system *might* be leaking pages (by leaving them pinned), but probably not. More likely, gup/pup returned a hard-ERRNOerror to the caller, who erroneously passed it here....
Quoted 3 lines
Do we need a similar check inside
unpin_user_pages_dirty_lock(), whenmake_dirtyset to false ?Maybe not. This call is rarely if ever used for error handling, but rather, for finishing up a successful use of the pages.
There is a balance between protecting against buggy callers and just fixing any buggy callers. There is also a limit to how much code one can write in hopes of avoiding bugs in...code that one writes. :) Which is why static analysis, unit and regression tests, code reviews are important too.
Quoted 3 lines
Here, I submit that that we're about to cross the line and go too far. But if you have any examples of buggy callers for
unpin_user_pages_dirty_lock(), that might shift the line.
I checked for buggy uses of unpin_user_pages_dirty_lock() using Smatch and didn't find anything. (Which doesn't mean that there aren't any).
regards,
dan carpenter
Quoted 14 lines · Dan Carpenter
Quoted 5 lines · Souptick Joarder
There is an error when
pin_user_pages_fast()returns-ERRNOand inside error handling path driver end up callingunpin_user_pages()with-ERRNOwhich is not correct.This patch will fix the problem.
There are a few ways we could prevent bug in the future.
1) This could have been caught with existing static analysis tools which warn about when a value is set but not used.
2) I've created a Smatch check which warngs about:
Quoted 9 lines
drivers/rapidio/devices/rio_mport_cdev.c:955rio_dma_transfer()warn: unpinning negative pages 'nr_pages'I'll test it out tonight and see how well it works. I don't immediately see any other bugs allthough Smatch doesn't like the code in
siw_umem_release(). It uses "min_t(int" which suggests that negative pages are okay.int to_free = min_t(int, PAGES_PER_CHUNK, num_pages);
I only found one bug but I'm going to add unpin_user_pages_dirty_lock() to the mix a retest. There were a few other false positives. In reviewing the code, I noticed that orangefs_bufmap_map() is also buggy.
I sort of feel like returning partial successes is not working. We could easily make a wrapper which either pins everything or it returns an error code.
drivers/misc/mic/scif/scif_rma.c:1399 __scif_pin_pages() warn: unpinning negative pages 'pinned_pages->nr_pages'
drivers/misc/mic/scif/scif_rma.c
1355 vmalloc_addr = true;
1356
1357 for (i = 0; i < nr_pages; i++) {
1358 if (vmalloc_addr)
1359 pinned_pages->pages[i] =
1360 vmalloc_to_page(addr + (i * PAGE_SIZE));
1361 else
1362 pinned_pages->pages[i] =
1363 virt_to_page(addr + (i * PAGE_SIZE));
1364 }
1365 pinned_pages->nr_pages = nr_pages;
1366 pinned_pages->map_flags = SCIF_MAP_KERNEL;
1367 } else {
1368 /*
1369 * SCIF supports registration caching. If a registration has
1370 * been requested with read only permissions, then we try
1371 * to pin the pages with RW permissions so that a subsequent
1372 * transfer with RW permission can hit the cache instead of
1373 * invalidating it. If the upgrade fails with RW then we
1374 * revert back to R permission and retry
1375 */
1376 if (prot == SCIF_PROT_READ)
1377 try_upgrade = true;
1378 prot |= SCIF_PROT_WRITE;
1379 retry:
1380 mm = current->mm;
1381 if (ulimit) {
1382 err = __scif_check_inc_pinned_vm(mm, nr_pages);
1383 if (err) {
1384 pinned_pages->nr_pages = 0;
1385 goto error_unmap;
1386 }
1387 }
1388
1389 pinned_pages->nr_pages = pin_user_pages_fast(
1390 (u64)addr,
1391 nr_pages,
1392 (prot & SCIF_PROT_WRITE) ? FOLL_WRITE : 0,
1393 pinned_pages->pages);
1394 if (nr_pages != pinned_pages->nr_pages) {
1395 if (try_upgrade) {
1396 if (ulimit)
1397 __scif_dec_pinned_vm_lock(mm, nr_pages);
1398 /* Roll back any pinned pages */
1399 unpin_user_pages(pinned_pages->pages,
1400 pinned_pages->nr_pages);
^^^^^^^^^^^^^^^^^^^^^^Negative.
1401 prot &= ~SCIF_PROT_WRITE;
1402 try_upgrade = false;
1403 goto retry;
1404 }
1405 }
1406 pinned_pages->map_flags = 0;
1407 }
1408
1409 if (pinned_pages->nr_pages < nr_pages) {
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^These are both signed so it negative ->nr_pages are less than nr_pages.
1410 err = -EFAULT;
1411 pinned_pages->nr_pages = nr_pages;
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^This sets it to "everything was pinned".
1412 goto dec_pinned;
1413 }
1414
1415 *out_prot = prot;
1416 atomic_set(&pinned_pages->ref_count, 1);
1417 *pages = pinned_pages;
1418 return err;
1419 dec_pinned:
1420 if (ulimit)
1421 __scif_dec_pinned_vm_lock(mm, nr_pages);
1422 /* Something went wrong! Rollback */
1423 error_unmap:
1424 pinned_pages->nr_pages = nr_pages;
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^This assumes everything was pinned successfully.
1425 scif_destroy_pinned_pages(pinned_pages);
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^We absolutely don't want to pass negative ->nr_pages to this function either.
1426 *pages = NULL; 1427 dev_dbg(scif_info.mdev.this_device, 1428 "%s %d err %d len 0x%lx\n", __func__, __LINE__, err, len); 1429 return err; 1430 }
regards,
dan carpenter
Quoted 25 lines · Dan Carpenter
Quoted 14 lines · Dan Carpenter
Quoted 5 lines · Souptick Joarder
There is an error when
pin_user_pages_fast()returns-ERRNOand inside error handling path driver end up callingunpin_user_pages()with-ERRNOwhich is not correct.This patch will fix the problem.
There are a few ways we could prevent bug in the future.
1) This could have been caught with existing static analysis tools which warn about when a value is set but not used.
2) I've created a Smatch check which warngs about:
Quoted 9 lines
drivers/rapidio/devices/rio_mport_cdev.c:955rio_dma_transfer()warn: unpinning negative pages 'nr_pages'I'll test it out tonight and see how well it works. I don't immediately see any other bugs allthough Smatch doesn't like the code in
siw_umem_release(). It uses "min_t(int" which suggests that negative pages are okay.int to_free = min_t(int, PAGES_PER_CHUNK, num_pages);
Quoted 7 lines
I only found one bug but I'm going to add
unpin_user_pages_dirty_lock()to the mix a retest. There were a few other false positives. In reviewing the code, I noticed thatorangefs_bufmap_map()is also buggy.I sort of feel like returning partial successes is not working. We could easily make a wrapper which either pins everything or it returns an error code.
I guess the question is are there drivers which will keep working (or limp along?) on partial pins? A quick search of a driver I thought did this does not apparently any more... So it sounds good to me from 30,000 feet! :-D
Ira
Quoted 100 lines
drivers/misc/mic/scif/scif_rma.c:1399__scif_pin_pages()warn: unpinning negative pages 'pinned_pages->nr_pages'
drivers/misc/mic/scif/scif_rma.c
1355vmalloc_addr= true;
1356
1357 for (i = 0; i <nr_pages; i++) {
1358 if (vmalloc_addr)
1359 pinned_pages->pages[i] =
1360vmalloc_to_page(addr + (i * PAGE_SIZE));
1361 else
1362 pinned_pages->pages[i] =
1363virt_to_page(addr + (i * PAGE_SIZE));1364 } 1365 pinned_pages->nr_pages = nr_pages; 1366 pinned_pages->map_flags = SCIF_MAP_KERNEL; 1367 } else { 1368 /* 1369 * SCIF supports registration caching. If a registration has 1370 * been requested with read only permissions, then we try 1371 * to pin the pages with RW permissions so that a subsequent 1372 * transfer with RW permission can hit the cache instead of 1373 * invalidating it. If the upgrade fails with RW then we 1374 * revert back to R permission and retry 1375 */ 1376 if (prot == SCIF_PROT_READ) 1377 try_upgrade = true; 1378 prot |= SCIF_PROT_WRITE; 1379 retry: 1380 mm = current->mm; 1381 if (ulimit) { 1382 err = __scif_check_inc_pinned_vm(mm, nr_pages); 1383 if (err) { 1384 pinned_pages->nr_pages = 0; 1385 goto error_unmap; 1386 } 1387 } 1388 1389 pinned_pages->nr_pages = pin_user_pages_fast( 1390 (u64)addr, 1391 nr_pages, 1392 (prot & SCIF_PROT_WRITE) ? FOLL_WRITE : 0, 1393 pinned_pages->pages); 1394 if (nr_pages != pinned_pages->nr_pages) { 1395 if (try_upgrade) { 1396 if (ulimit) 1397 __scif_dec_pinned_vm_lock(mm, nr_pages); 1398 /* Roll back any pinned pages */ 1399 unpin_user_pages(pinned_pages->pages, 1400 pinned_pages->nr_pages); ^^^^^^^^^^^^^^^^^^^^^^Negative.
1401 prot &= ~SCIF_PROT_WRITE; 1402 try_upgrade = false; 1403 goto retry; 1404 } 1405 } 1406 pinned_pages->map_flags = 0; 1407 } 1408 1409 if (pinned_pages->nr_pages < nr_pages) { ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^These are both signed so it negative ->nr_pages are less than
nr_pages.1410 err =
-EFAULT;1411 pinned_pages->nr_pages = nr_pages; ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^This sets it to "everything was pinned".
1412 goto
dec_pinned;
1413 }
14141415 *out_prot = prot; 1416 atomic_set(&pinned_pages->ref_count, 1); 1417 *pages = pinned_pages; 1418 return err; 1419 dec_pinned: 1420 if (ulimit) 1421 __scif_dec_pinned_vm_lock(mm, nr_pages); 1422 /* Something went wrong! Rollback */ 1423 error_unmap: 1424 pinned_pages->nr_pages = nr_pages; ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^This assumes everything was pinned successfully.
1425 scif_destroy_pinned_pages(pinned_pages); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^We absolutely don't want to pass negative ->nr_pages to this function either.
1426 *pages = NULL; 1427 dev_dbg(scif_info.mdev.this_device, 1428 "%s %d err %d len 0x%lx\n", __func__, __LINE__, err, len); 1429 return err; 1430 }regards,
dan carpenter
John Hubbard
reviewerQuoted 26 lines · Ira Weiny
Quoted 25 lines · Dan Carpenter
Quoted 23 lines · Dan Carpenter
Quoted 5 lines · Souptick Joarder
There is an error when
pin_user_pages_fast()returns-ERRNOand inside error handling path driver end up callingunpin_user_pages()with-ERRNOwhich is not correct.This patch will fix the problem.
There are a few ways we could prevent bug in the future.
1) This could have been caught with existing static analysis tools which warn about when a value is set but not used.
2) I've created a Smatch check which warngs about:
drivers/rapidio/devices/rio_mport_cdev.c:955rio_dma_transfer()warn: unpinning negative pages 'nr_pages'I'll test it out tonight and see how well it works. I don't immediately see any other bugs allthough Smatch doesn't like the code in
siw_umem_release(). It uses "min_t(int" which suggests that negative pages are okay.int
to_free=min_t(int, PAGES_PER_CHUNK, num_pages);
Quoted 7 lines
Quoted 7 lines
I only found one bug but I'm going to add
unpin_user_pages_dirty_lock()to the mix a retest. There were a few other false positives. In reviewing the code, I noticed thatorangefs_bufmap_map()is also buggy.I sort of feel like returning partial successes is not working. We could easily make a wrapper which either pins everything or it returns an error code.
Yes we could. And I have the same feeling about this API. It's generated a remarkable amount of bug fixes, several of which ended up being partial or wrong in themselves. And mostly this is due to the complicated tristate return code: instead of 0 or -ERRNO, it also can return "N pages that is less than what you requested", and there are no standard helpers in the kernel to make that easier to deal with.
Quoted 4 lines
I guess the question is are there drivers which will keep working (or limp along?) on partial pins? A quick search of a driver I thought did this does not apparently any more... So it sounds good to me from 30,000 feet! :-D
It sounds good to me too--and from just a *few hundred feet* (having touched most of the call sites at some point)! haha :)
I think the wrapper should be short-term, though, just until all the callers are converted to the simpler API. Then change the core gup/pup calls to the simpler API. There are more than enough gup/pup API entry points as it is, that's for sure.
thanks,
Signature · 2 lines
John Hubbard NVIDIA
Quoted 36 lines · John Hubbard
Quoted 26 lines · Ira Weiny
Quoted 25 lines · Dan Carpenter
Quoted 23 lines · Dan Carpenter
Quoted 5 lines · Souptick Joarder
There is an error when
pin_user_pages_fast()returns-ERRNOand inside error handling path driver end up callingunpin_user_pages()with-ERRNOwhich is not correct.This patch will fix the problem.
There are a few ways we could prevent bug in the future.
1) This could have been caught with existing static analysis tools which warn about when a value is set but not used.
2) I've created a Smatch check which warngs about:
drivers/rapidio/devices/rio_mport_cdev.c:955rio_dma_transfer()warn: unpinning negative pages 'nr_pages'I'll test it out tonight and see how well it works. I don't immediately see any other bugs allthough Smatch doesn't like the code in
siw_umem_release(). It uses "min_t(int" which suggests that negative pages are okay.int
to_free=min_t(int, PAGES_PER_CHUNK, num_pages);Quoted 7 lines
Quoted 7 lines
I only found one bug but I'm going to add
unpin_user_pages_dirty_lock()to the mix a retest. There were a few other false positives. In reviewing the code, I noticed thatorangefs_bufmap_map()is also buggy.I sort of feel like returning partial successes is not working. We could easily make a wrapper which either pins everything or it returns an error code.
Quoted 6 lines
Yes we could. And I have the same feeling about this API. It's generated a remarkable amount of bug fixes, several of which ended up being partial or wrong in themselves. And mostly this is due to the complicated tristate return code: instead of 0 or
-ERRNO, it also can return "N pages that is less than what you requested", and there are no standard helpers in the kernel to make that easier to deal with
There was some discussion on removing return value 0 from one of the gup variants [1].
I think it might be partially relevant to the current discussion.
[1] https://patchwork.kernel.org/patch/11529795/
Quoted 18 lines
Quoted 5 lines
I guess the question is are there drivers which will keep working (or limp along?) on partial pins? A quick search of a driver I thought did this does not apparently any more... So it sounds good to me from 30,000 feet! :-D
It sounds good to me too--and from just a *few hundred feet* (having touched most of the call sites at some point)! haha :)
I think the wrapper should be short-term, though, just until all the callers are converted to the simpler API. Then change the core gup/pup calls to the simpler API. There are more than enough gup/pup API entry points as it is, that's for sure.
thanks,
--
John Hubbard
NVIDIA
John Hubbard
reviewerQuoted 1 line · Souptick Joarder
On Thu, Sep 17, 2020 at 11:17 PM John Hubbard <[email protected]> wrote:
...
Quoted 11 lines
Quoted 10 lines
Quoted 3 lines
Quoted 3 lines
I sort of feel like returning partial successes is not working. We could easily make a wrapper which either pins everything or it returns an error code.
Yes we could. And I have the same feeling about this API. It's generated a remarkable amount of bug fixes, several of which ended up being partial or wrong in themselves. And mostly this is due to the complicated tristate return code: instead of 0 or
-ERRNO, it also can return "N pages that is less than what you requested", and there are no standard helpers in the kernel to make that easier to deal with
Quoted 6 lines
There was some discussion on removing return value 0 from one of the gup variants [1].
I think it might be partially relevant to the current discussion.
Yes, although as I mentioned above, I'm thinking of a 0 or -ERRNO return value, and not even return nr_pages at all.
But in any case, as a practical matter, I'm not sure if it's a good idea to actually change all the callsites, or not. If we just fix the remaining buggy callers, maybe that's better than the churn associated with another API change.
On the other-other hand, there does seem to be more churn coming anyway, with talk of actually doing a [get|pin]_user_bvec(), for example. So maybe it's better to head off the coming mess.
This is something that should be discussed on linux-mm.
Quoted 14 lines
Quoted 14 lines
Quoted 5 lines
I guess the question is are there drivers which will keep working (or limp along?) on partial pins? A quick search of a driver I thought did this does not apparently any more... So it sounds good to me from 30,000 feet! :-D
It sounds good to me too--and from just a *few hundred feet* (having touched most of the call sites at some point)! haha :)
I think the wrapper should be short-term, though, just until all the callers are converted to the simpler API. Then change the core gup/pup calls to the simpler API. There are more than enough gup/pup API entry points as it is, that's for sure.
Quoted 4 lines
Quoted 4 lines
thanks,
--
John Hubbard
NVIDIA
thanks,
Signature · 2 lines
John Hubbard NVIDIA
Hi Dan,
Quoted 124 lines · Dan Carpenter
Quoted 14 lines · Dan Carpenter
Quoted 5 lines · Souptick Joarder
There is an error when
pin_user_pages_fast()returns-ERRNOand inside error handling path driver end up callingunpin_user_pages()with-ERRNOwhich is not correct.This patch will fix the problem.
There are a few ways we could prevent bug in the future.
1) This could have been caught with existing static analysis tools which warn about when a value is set but not used.
2) I've created a Smatch check which warngs about:
Quoted 9 lines
drivers/rapidio/devices/rio_mport_cdev.c:955rio_dma_transfer()warn: unpinning negative pages 'nr_pages'I'll test it out tonight and see how well it works. I don't immediately see any other bugs allthough Smatch doesn't like the code in
siw_umem_release(). It uses "min_t(int" which suggests that negative pages are okay.int to_free = min_t(int, PAGES_PER_CHUNK, num_pages);I only found one bug but I'm going to add
unpin_user_pages_dirty_lock()to the mix a retest. There were a few other false positives. In reviewing the code, I noticed thatorangefs_bufmap_map()is also buggy.I sort of feel like returning partial successes is not working. We could easily make a wrapper which either pins everything or it returns an error code.
drivers/misc/mic/scif/scif_rma.c:1399__scif_pin_pages()warn: unpinning negative pages 'pinned_pages->nr_pages'
drivers/misc/mic/scif/scif_rma.c
1355vmalloc_addr= true;
1356
1357 for (i = 0; i <nr_pages; i++) {
1358 if (vmalloc_addr)
1359 pinned_pages->pages[i] =
1360vmalloc_to_page(addr + (i * PAGE_SIZE));
1361 else
1362 pinned_pages->pages[i] =
1363virt_to_page(addr + (i * PAGE_SIZE));1364 } 1365 pinned_pages->nr_pages = nr_pages; 1366 pinned_pages->map_flags = SCIF_MAP_KERNEL; 1367 } else { 1368 /* 1369 * SCIF supports registration caching. If a registration has 1370 * been requested with read only permissions, then we try 1371 * to pin the pages with RW permissions so that a subsequent 1372 * transfer with RW permission can hit the cache instead of 1373 * invalidating it. If the upgrade fails with RW then we 1374 * revert back to R permission and retry 1375 */ 1376 if (prot == SCIF_PROT_READ) 1377 try_upgrade = true; 1378 prot |= SCIF_PROT_WRITE; 1379 retry: 1380 mm = current->mm; 1381 if (ulimit) { 1382 err = __scif_check_inc_pinned_vm(mm, nr_pages); 1383 if (err) { 1384 pinned_pages->nr_pages = 0; 1385 goto error_unmap; 1386 } 1387 } 1388 1389 pinned_pages->nr_pages = pin_user_pages_fast( 1390 (u64)addr, 1391 nr_pages, 1392 (prot & SCIF_PROT_WRITE) ? FOLL_WRITE : 0, 1393 pinned_pages->pages); 1394 if (nr_pages != pinned_pages->nr_pages) { 1395 if (try_upgrade) { 1396 if (ulimit) 1397 __scif_dec_pinned_vm_lock(mm, nr_pages); 1398 /* Roll back any pinned pages */ 1399 unpin_user_pages(pinned_pages->pages, 1400 pinned_pages->nr_pages); ^^^^^^^^^^^^^^^^^^^^^^Negative.
1401 prot &= ~SCIF_PROT_WRITE; 1402 try_upgrade = false; 1403 goto retry; 1404 } 1405 } 1406 pinned_pages->map_flags = 0; 1407 } 1408 1409 if (pinned_pages->nr_pages < nr_pages) { ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^These are both signed so it negative ->nr_pages are less than
nr_pages.1410 err =
-EFAULT;1411 pinned_pages->nr_pages = nr_pages; ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^This sets it to "everything was pinned".
1412 goto
dec_pinned;
1413 }
14141415 *out_prot = prot; 1416 atomic_set(&pinned_pages->ref_count, 1); 1417 *pages = pinned_pages; 1418 return err; 1419 dec_pinned: 1420 if (ulimit) 1421 __scif_dec_pinned_vm_lock(mm, nr_pages); 1422 /* Something went wrong! Rollback */ 1423 error_unmap: 1424 pinned_pages->nr_pages = nr_pages; ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^This assumes everything was pinned successfully.
1425 scif_destroy_pinned_pages(pinned_pages); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^We absolutely don't want to pass negative ->nr_pages to this function either.
Quoted 8 lines
1426 *pages = NULL; 1427 dev_dbg(scif_info.mdev.this_device, 1428 "%s %d err %d len 0x%lx\n", __func__, __LINE__, err, len); 1429 return err; 1430 }regards,
dan carpenter
I have drafted a patch for this bug. Shall I post it ?
Quoted 133 lines · Souptick Joarder
Hi Dan,
Quoted 130 lines · Dan Carpenter
Quoted 14 lines · Dan Carpenter
Quoted 5 lines · Souptick Joarder
There is an error when
pin_user_pages_fast()returns-ERRNOand inside error handling path driver end up callingunpin_user_pages()with-ERRNOwhich is not correct.This patch will fix the problem.
There are a few ways we could prevent bug in the future.
1) This could have been caught with existing static analysis tools which warn about when a value is set but not used.
2) I've created a Smatch check which warngs about:
Quoted 9 lines
drivers/rapidio/devices/rio_mport_cdev.c:955rio_dma_transfer()warn: unpinning negative pages 'nr_pages'I'll test it out tonight and see how well it works. I don't immediately see any other bugs allthough Smatch doesn't like the code in
siw_umem_release(). It uses "min_t(int" which suggests that negative pages are okay.int to_free = min_t(int, PAGES_PER_CHUNK, num_pages);I only found one bug but I'm going to add
unpin_user_pages_dirty_lock()to the mix a retest. There were a few other false positives. In reviewing the code, I noticed thatorangefs_bufmap_map()is also buggy.I sort of feel like returning partial successes is not working. We could easily make a wrapper which either pins everything or it returns an error code.
drivers/misc/mic/scif/scif_rma.c:1399__scif_pin_pages()warn: unpinning negative pages 'pinned_pages->nr_pages'
drivers/misc/mic/scif/scif_rma.c
1355vmalloc_addr= true;
1356
1357 for (i = 0; i <nr_pages; i++) {
1358 if (vmalloc_addr)
1359 pinned_pages->pages[i] =
1360vmalloc_to_page(addr + (i * PAGE_SIZE));
1361 else
1362 pinned_pages->pages[i] =
1363virt_to_page(addr + (i * PAGE_SIZE));1364 } 1365 pinned_pages->nr_pages = nr_pages; 1366 pinned_pages->map_flags = SCIF_MAP_KERNEL; 1367 } else { 1368 /* 1369 * SCIF supports registration caching. If a registration has 1370 * been requested with read only permissions, then we try 1371 * to pin the pages with RW permissions so that a subsequent 1372 * transfer with RW permission can hit the cache instead of 1373 * invalidating it. If the upgrade fails with RW then we 1374 * revert back to R permission and retry 1375 */ 1376 if (prot == SCIF_PROT_READ) 1377 try_upgrade = true; 1378 prot |= SCIF_PROT_WRITE; 1379 retry: 1380 mm = current->mm; 1381 if (ulimit) { 1382 err = __scif_check_inc_pinned_vm(mm, nr_pages); 1383 if (err) { 1384 pinned_pages->nr_pages = 0; 1385 goto error_unmap; 1386 } 1387 } 1388 1389 pinned_pages->nr_pages = pin_user_pages_fast( 1390 (u64)addr, 1391 nr_pages, 1392 (prot & SCIF_PROT_WRITE) ? FOLL_WRITE : 0, 1393 pinned_pages->pages); 1394 if (nr_pages != pinned_pages->nr_pages) { 1395 if (try_upgrade) { 1396 if (ulimit) 1397 __scif_dec_pinned_vm_lock(mm, nr_pages); 1398 /* Roll back any pinned pages */ 1399 unpin_user_pages(pinned_pages->pages, 1400 pinned_pages->nr_pages); ^^^^^^^^^^^^^^^^^^^^^^Negative.
1401 prot &= ~SCIF_PROT_WRITE; 1402 try_upgrade = false; 1403 goto retry; 1404 } 1405 } 1406 pinned_pages->map_flags = 0; 1407 } 1408 1409 if (pinned_pages->nr_pages < nr_pages) { ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^These are both signed so it negative ->nr_pages are less than
nr_pages.1410 err =
-EFAULT;1411 pinned_pages->nr_pages = nr_pages; ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^This sets it to "everything was pinned".
1412 goto
dec_pinned;
1413 }
14141415 *out_prot = prot; 1416 atomic_set(&pinned_pages->ref_count, 1); 1417 *pages = pinned_pages; 1418 return err; 1419 dec_pinned: 1420 if (ulimit) 1421 __scif_dec_pinned_vm_lock(mm, nr_pages); 1422 /* Something went wrong! Rollback */ 1423 error_unmap: 1424 pinned_pages->nr_pages = nr_pages; ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^This assumes everything was pinned successfully.
1425 scif_destroy_pinned_pages(pinned_pages); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^We absolutely don't want to pass negative ->nr_pages to this function either.
1426 *pages = NULL; 1427 dev_dbg(scif_info.mdev.this_device, 1428 "%s %d err %d len 0x%lx\n", __func__, __LINE__, err, len); 1429 return err; 1430 }
Quoted 5 lines
Quoted 2 lines
regards,
dan carpenterI have drafted a patch for this bug. Shall I post it ?
Absolutely, please. :)
regards,
dan carpenter
Quoted 9 lines · Souptick Joarder
There is an error when
pin_user_pages_fast()returns-ERRNOand inside error handling path driver end up callingunpin_user_pages()with-ERRNOwhich is not correct.This patch will fix the problem.
Fixes e8de370188d09 ("rapidio: add mport char device driver")Signed-off-by Souptick Joarder <[email protected]>Cc Ira Weiny <[email protected]>
Quoted 36 lines
Cc John Hubbard <[email protected]>Cc Matthew Wilcox <[email protected]>--- drivers/rapidio/devices/rio_mport_cdev.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)drivers/rapidio/devices/rio_mport_cdev.c +7
-6@@ -871,15 +871,16 @@ static int do_dma_request(struct mport_dma_req *req,rmcd_error ("pin_user_pages_fast err=%ld" , pinned); nr_pages =0 ; - }else + }else {rmcd_error ("pinned %ld out of %ld pages" , pinned, nr_pages); +/* +* Set nr_pages up to mean "how many pages to unpin, in +* the error handler: +*/ + nr_pages = pinned; + } ret = -EFAULT ; -/* -* Set nr_pages up to mean "how many pages to unpin, in -* the error handler: -*/ - nr_pages = pinned; goto err_pg; }Signature · 1 line
1.9.1