2.4.1-pre10 deadlock (Re: ps hang in 241-pre10)
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 8 lines · Jens Axboe
Quoted 4 lines
So what happens is that somebody takes a page fault (and gets the mm lock), tries to read something in, and never gets anything back, thus leaving the MM locked.
What was the trace of this? Just curious, the below case outlined by Linus should be pretty generic, but I'd still like to know what can lead to this condition.
It was posted on linux-kernel - I don't save the dang things because I have too much in my "archives" as is ;)
Quoted 6 lines
Quoted 6 lines
Lorenzo, does the problem go away for you if you remove the
if (!list_empty(&q->request_freelist[rw])) { ... }
Quoted 7 lines
Quoted 1 line
code from
blkdev_release_request()indrivers/block/ll_rw_block.c?Good spotting. Actually I see one more problem with it too. If we've started batching (under heavy I/O of course), we could splice the pending list and wake up X number of sleepers, but there's a) no guarentee that these sleepers will actually get the requests if new ones keep flooding in
(a) is ok. They'll go back to sleep - it's a loop waiting for requests..
Quoted 2 lines
and b) no guarentee
that X sleepers require X request slots.
Well, IF they are sleeping (and thus, if the wake_up_nr() will trigger on them), they _will_ use a request. I don't think we have to worry about that. At most we will wake up "too many" - we'll wake up processes even though they end up not being able to get a request anyway because somebody else got to it first. And that's ok. It's the "wake up too few" that causes trouble, and I think that will be fixed by my suggestion.
Now, I'd worred if somebody wants several requests at the same time, and doesn't feed them to the IO layer until it has gotten all of them. In that case, you can get starvation with many people having "reserved" their requests, and there not be enough free requests around to actually ever wake anybody up again. But the regular IO paths do not do this: they will all allocate a request and just submit it immediately, no "reservation".
(Obviously, _submitting_ the request doesn't mean that we'd actually start processing it, but if somebody ends up waiting for requests they'll do the unplug that does start it all going, so effectively we can think of it as a logical "start this request now" thing even if it gets delayed in order to coalesce IO).
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
Quoted 6 lines · Linus Torvalds
Quoted 3 lines
What was the trace of this? Just curious, the below case outlined by Linus should be pretty generic, but I'd still like to know what can lead to this condition.
It was posted on linux-kernel - I don't save the dang things because I have too much in my "archives" as is ;)
Ok I see it now, confused wrt the different threads...
Quoted 7 lines
Quoted 5 lines
Good spotting. Actually I see one more problem with it too. If we've started batching (under heavy I/O of course), we could splice the pending list and wake up X number of sleepers, but there's a) no guarentee that these sleepers will actually get the requests if new ones keep flooding in
(a) is ok. They'll go back to sleep - it's a loop waiting for requests..
My point is not that it's broken, but it will favor new comers instead of tasks having blocked on a free slot already. So it would still be nice to get right.
Quoted 9 lines
Quoted 2 lines
and b) no guarentee
that X sleepers require X request slots.Well, IF they are sleeping (and thus, if the
wake_up_nr()will trigger on them), they _will_ use a request. I don't think we have to worry about that. At most we will wake up "too many" - we'll wake up processes even though they end up not being able to get a request anyway because somebody else got to it first. And that's ok. It's the "wake up too few" that causes trouble, and I think that will be fixed by my suggestion.
Yes they may end up sleeing right away again as per the above a) case for instance. The logic now is 'we have X free slots now, wake up x sleepers' where it instead should be 'we have X free slots now, wake up people until the free list is exhausted'.
Quoted 6 lines
Now, I'd worred if somebody wants several requests at the same time, and doesn't feed them to the IO layer until it has gotten all of them. In that case, you can get starvation with many people having "reserved" their requests, and there not be enough free requests around to actually ever wake anybody up again. But the regular IO paths do not do this: they will all allocate a request and just submit it immediately, no "reservation".
Right, the I/O path doesn't do this and it would seem more appropriate to have such users use their own requests instead of eating from the internal pool.
Signature · 6 lines
* Jens Axboe <[email protected]> * SuSE Labs - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] Please read the FAQ at http://www.tux.org/lkml/
Quoted 44 lines · Jens Axboe
Quoted 6 lines · Linus Torvalds
Quoted 3 lines
What was the trace of this? Just curious, the below case outlined by Linus should be pretty generic, but I'd still like to know what can lead to this condition.
It was posted on linux-kernel - I don't save the dang things because I have too much in my "archives" as is ;)
Ok I see it now, confused wrt the different threads...
Quoted 7 lines
Quoted 5 lines
Good spotting. Actually I see one more problem with it too. If we've started batching (under heavy I/O of course), we could splice the pending list and wake up X number of sleepers, but there's a) no guarentee that these sleepers will actually get the requests if new ones keep flooding in
(a) is ok. They'll go back to sleep - it's a loop waiting for requests..
My point is not that it's broken, but it will favor new comers instead of tasks having blocked on a free slot already. So it would still be nice to get right.
Quoted 9 lines
Quoted 2 lines
and b) no guarentee
that X sleepers require X request slots.Well, IF they are sleeping (and thus, if the
wake_up_nr()will trigger on them), they _will_ use a request. I don't think we have to worry about that. At most we will wake up "too many" - we'll wake up processes even though they end up not being able to get a request anyway because somebody else got to it first. And that's ok. It's the "wake up too few" that causes trouble, and I think that will be fixed by my suggestion.Yes they may end up sleeing right away again as per the above a) case for instance. The logic now is 'we have X free slots now, wake up x sleepers' where it instead should be 'we have X free slots now, wake up people until the free list is exhausted'.
Quoted 6 lines
Now, I'd worred if somebody wants several requests at the same time, and doesn't feed them to the IO layer until it has gotten all of them. In that case, you can get starvation with many people having "reserved" their requests, and there not be enough free requests around to actually ever wake anybody up again. But the regular IO paths do not do this: they will all allocate a request and just submit it immediately, no "reservation".
Quoted 12 lines
Right, the I/O path doesn't do this and it would seem more appropriate to have such users use their own requests instead of eating from the internal pool.
Signature · 6 lines
* Jens Axboe <[email protected]> * SuSE Labs - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] Please read the FAQ at http://www.tux.org/lkml/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/