OOM and USB, latest Linux 2.6
Hello,
Is it possible that OOM isn't handled very well if say, my entire file system structure is on a USB storage device?
I'm not an expert on this particular matter but I'm pretty sure that I noticed GFP_KERNEL allocation being done on the write-out path in the usb-storage kernel thread, leading to a deadlock during OOM.
Suggestions are welcomed...
Signature · 3 lines
Dan Aloni XIV LTD, http://www.xivstorage.com da-x (at) monatomic.org, dan (at) xiv.co.il
Quoted 9 lines · Dan Aloni
Hello,
Is it possible that OOM isn't handled very well if say, my entire file system structure is on a USB storage device?
I'm not an expert on this particular matter but I'm pretty sure that I noticed
GFP_KERNELallocation being done on the write-out path in the usb-storage kernel thread, leading to a deadlock during OOM.
Can you be any more specific than that? usb-storage should use only GFP_NOIO in its I/O paths.
Alan Stern
Quoted 5 lines · Alan Stern
Quoted 2 lines · Dan Aloni
Hello,
Quoted 10 lines
Quoted 5 lines
Is it possible that OOM isn't handled very well if say, my entire file system structure is on a USB storage device?
I'm not an expert on this particular matter but I'm pretty sure that I noticed
GFP_KERNELallocation being done on the write-out path in the usb-storage kernel thread, leading to a deadlock during OOM.Can you be any more specific than that? usb-storage should use only
GFP_NOIOin its I/O paths.
You are right, I looked over this state with kdb, and usb-storage waited in usb_stor_bulk_transfer_sg, which does pass GFP_NOIO at this scenario.
It looked suspicious though, because OOM handling was invoked from many processes, and it didn't print about any process being killed and it didn't complain about no processes to kill either.
(I'll look more into this, perhaps there's an OOM handling bug)
BTW, soft-rebooting the machine in that state made the USB storage device (LEXAR, JD LIGHTNING II) inaccessible to the BIOS. I had to do a complete power cycle in order or the BIOS to see it again.
Signature · 3 lines
Dan Aloni XIV LTD, http://www.xivstorage.com da-x (at) monatomic.org, dan (at) xiv.co.il
Quoted 1 line · Dan Aloni
Alan Stern wrote:
[...]
Quoted 11 lines
Quoted 4 lines
Can you be any more specific than that? usb-storage should use only
GFP_NOIOin its I/O paths.You are right, I looked over this state with kdb, and usb-storage waited in
usb_stor_bulk_transfer_sg, which does passGFP_NOIOat this scenario.It looked suspicious though, because OOM handling was invoked from many processes, and it didn't print about any process being killed and it didn't complain about no processes to kill either.
Hmm, I'm pretty sure I stomped over this (from select_bad_process()):
/*
* This task already has access to memory reserves and is
* being killed. Don't allow any other task access to the
* memory reserve.
*
* Note: this may have a chance of deadlock if it gets
* blocked waiting for another task which itself is waiting
* for memory. Is there a better alternative? */
if (test_tsk_thread_flag(p, TIF_MEMDIE))
return ERR_PTR(-1UL);
Which might explains why the OOM handling was behaving like it did.
It would have been nice if it at least printed "OOM: I'm in a deadlock, please FIXME...".
Signature · 3 lines
Dan Aloni XIV LTD, http://www.xivstorage.com da-x (at) monatomic.org, dan (at) xiv.co.il
Quoted 3 lines · Dan Aloni
You are right, I looked over this state with kdb, and usb-storage waited in
usb_stor_bulk_transfer_sg, which does passGFP_NOIOat this scenario.
...
Quoted 4 lines
BTW, soft-rebooting the machine in that state made the USB storage device (LEXAR, JD LIGHTNING II) inaccessible to the BIOS. I had to do a complete power cycle in order or the BIOS to see it again.
That's the fault of the BIOS and/or the Lexar device. The BIOS ought to do a complete reset of the USB controller and a reset of the device. If the device remains unusable after that, there isn't anything we can do about it.
Alan Stern