Skip to contentSkip to conversation

More 2.2.17pre9 VM issues

DISCUSSION Stephen C. Tweedie 2 participants 3 messages Jul 3, 2000 2 messages not in the archive Memory management

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#2 lore raw

Hi,

Quoted 1 line · Andrea Arcangeli

Stephen, are we really sure we still need kpiod?i

Yes.

Quoted 2 lines

Isn't GFP_IO meant to be
clear if anybody is helding any filesystem lock (like superblock lock)?

That has never been a requirement, and I'd think it would be dangerous to make such a new rule so close to 2.4.

Certainly, in 2.2 we would do all sorts of stuff while holding filesystem locks (in particular the inode and superblock locks). Any file write, including mm page writes, would take the inode lock.

Right now, in 2.4, the mm locks less but write(2) still takes the inode lock. That means that we _must_ be able to allocate with GFP_IO while holding the inode lock, since (a) write()s go through the page cache, and (b) touching the user's buffer during the write() can cause pages to be swapped in, invoking parts of the VM which assume they are able to use GFP_IO safely.

Sure, you could audit every single path through every filesystem to make sure that there are no possible deadlocks here, but the whole point of kpiod is to separate out the pageout from the process doing the write() in such situations to make deadlock impossible.

Cheers,
Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

#3 lore raw
Quoted 2 lines · Stephen C. Tweedie

That has never been a requirement, and I'd think it would be dangerous to make such a new rule so close to 2.4.

Note that 2.4.0-test* doesn't have kpiod just now (it's been dropped in the 2.3.x cycle). Here I was wondering only about 2.2.x.

Quoted 8 lines

Certainly, in 2.2 we would do all sorts of stuff while holding filesystem locks (in particular the inode and superblock locks). Any file write, including mm page writes, would take the inode lock.

Right now, in 2.4, the mm locks less but write(2) still takes the inode lock. That means that we _must_ be able to allocate with GFP_IO while holding the inode lock, since (a) write()s go through the page cache, and (b) touching the user's buffer during the write() can cause

writes in 2.2.x goes through the buffer cache that is for this reason allocated with GFP_BUFFER.

Quoted 2 lines

pages to be swapped in, invoking parts of the VM which assume they are able to use GFP_IO safely.

arghh b is a problem. I could workaround that with per per-process bitflag set before down(&inode->i_sem) that reminds me not to write on any fs because I would risk to recurse on the inode->i_sem.

The main problem I have with kpiod is that while it obviously avoids any kind of deadlocks on the fs since make_pio_request is completly asynchronous, it also introduces a problem in the swap_out code where we have no way to know if we did some progress or not and if we should wait some buffer to be written to disk. Waiting in shrink_mmap isn't even enough since kpiod could have not yet been started writing and so there may be none locked or dirty buffer in first place... We can't even wait on kpiod ala wakeup_bdflush(1) since it would re-introduce the same deadlock problem making kpiod useless for its anti-deadlock purpose.

So the only sane solution looked to me to skip the filemap_swapout when we can't do it, and to do it ourselfs when we can. The problem is the "when we can". Do you think it would be insane to implement the workaround I mentioned above around the write semaphore?

Quoted 4 lines

Sure, you could audit every single path through every filesystem to make sure that there are no possible deadlocks here, but the whole point of kpiod is to separate out the pageout from the process doing the write() in such situations to make deadlock impossible.

I'll think some more on it.

Andrea

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

#4 lore raw

Hi,

Quoted 7 lines · Andrea Arcangeli
Quoted 3 lines

pages to be swapped in, invoking parts of the VM which assume they are able to use GFP_IO safely.

arghh b is a problem. I could workaround that with per per-process bitflag set before down(&inode->i_sem) that reminds me not to write on any fs because I would risk to recurse on the inode->i_sem.

It's not necessarily a problem, as the file paging routines don't take the inode semaphore any more (at least on ext2). But unless we want to explicitly ban the read/writepage routines from invoking that semaphore, we have to be prepared for this to happen.

Given that a write() syscall takes the semaphore for its whole duration, that's an *awefully* long time to be preventing paging on that inode. So this is in fact probably the way forward --- document that only write() can use the semaphore, but VM-invoked functions like *writepage must not.

Quoted 5 lines

The main problem I have with kpiod is that while it obviously avoids any kind of deadlocks on the fs since make_pio_request is completly asynchronous, it also introduces a problem in the swap_out code where we have no way to know if we did some progress or not and if we should wait some buffer to be written to disk.

Sure, but I've already said that I think we need multiple separate paging queues, with the process of aging and cleaning pages made separate from the process of evicting pages. If you do that, then you can always tell, from the length of the queues, whether or not you still have work to do.

But I still agree that getting rid of kpiod is probably a good thing. We just can't do it in 2.2. In 2.4, keep it away by all means, but we have to be aware of the implications when you do write() to an mmaped file.

Cheers,
Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/