dirty pages in memory & co.

O.k. I've dug through all of my obvious bugs and I a working set of kernel patches.
Currently they are against 2.2.5.
Location:
http://ebiederm/files/patches9.tar.gz

I consider this set of patches alpha (as I haven't had a chance to stress test it yet). But if you want to see the direction I'm going it's a good thing to look at.

Documentaion, porting shmfs, and stress testing are still to come.

The patches included are:
eb1 -- Allow reuse of page->buffers if you aren't the buffer cache eb2 -- Allow old old a.out binaries to run even if we can't mmap them
properly because their data isn't page aligned.
eb3 -- Much with page offset.
eb4 -- Allow registration and unregistration for functions needed by
swap off.
eb5 -- Large file support, basically this removes unused bits from all
of the relevant interfaces.
eb6 -- Introduction of struct vm_store, and associated cleanups.
In particular get_inode_page.
eb7 -- Actuall patch for dirty buffers in the page cache.
I'm fairly well satisfied except for generic_file_write. It looks like I need 2 variations on generic_file_write at the moment.
1) for network filesystems that can get away without filling the page on a partial write.
2) for block based filesystems that must fill the page on a partial write because they can't write arbitrary chunks of data.

TODO:
1) document the new interfaces
2) porting shmfs
3) stress testing.
4) Experimenting with heuristics so that programs that are writing data faster than the disk can handle are put to sleep (think floppies).
5) Playing with mapped memory, and removing the need for kpiod. This will either require either reverse page tables, or something equally effecting at finding page mappings from a struct page.
6) Removing the need for struct vm_operations in the vm_area_struct. A struct vm_store can probably handle everything.
7) Removing the swap lock map, by modify ipc/shm to use the page cache and vm_stores.

I'm going to visit my parents this weekend so I don't expect to get much farther for a while.

Eric

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

I just killed the swap lock map and I just use the swap cache for ipc shm memory.

Now I was thinking at the reverse lookup from pagemap to pagetable that you mentioned. It would be easy to that at least for the page/swap cache mappings with the interface I added in my tree.

But to support dynamic relocation/defrag of memory on the whole VM we should do that for _all_ pages. And to do the relocation we should run with the GFP pages mapped in a separate pte (not in the 4mbyte page table with the kernel). So I don't know if it would be better to just move all kernel memory (the one available through GFP) to virtual memory and to support the reverse lookup for all pages in the system, or if I should only do the quite-easy backdoor for the page/swap cache. The point is that supporting the reverse lookup for all kernel memory and having all kernel memory in virtual memory, will be a _major_ performance hit for all operations according to me.

Right now i would need the reverse lookup only for the mapped cache because I would like to avoid to run swap_out to know if the pte is been accessed or not and in the case it's an old pte I could unmap the mmapped-page directly from shrink_mmap. But I am not convinced this will be an improvement too because I just run swap_out at the right time...

Comments?

Andrea Arcangeli

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

AA> I just killed the swap lock map and I just use the swap cache for ipc shm AA> memory.

Cool. I'll have to take a look. It should save some work.

AA> Now I was thinking at the reverse lookup from pagemap to pagetable that AA> you mentioned. It would be easy to that at least for the page/swap cache AA> mappings with the interface I added in my tree.

AA> But to support dynamic relocation/defrag of memory on the whole VM we AA> should do that for _all_ pages. And to do the relocation we should run AA> with the GFP pages mapped in a separate pte (not in the 4mbyte page table AA> with the kernel). So I don't know if it would be better to just move all AA> kernel memory (the one available through GFP) to virtual memory and to AA> support the reverse lookup for all pages in the system, or if I should AA> only do the quite-easy backdoor for the page/swap cache. The point is that AA> supporting the reverse lookup for all kernel memory and having all kernel AA> memory in virtual memory, will be a _major_ performance hit for all AA> operations according to me.

Right, and it doesn't buy you very much.
We have some constants.
A) Defragging doesn't need to happen often.
B) We don't have much kernel memory tied up in locked down memory (that we refer to with pointers).
C) For locked down memory there is also internal fragmentation to worry about.
D) The biggest gain is handling memory that isn't locked down.

So I think I would first concentrate on the common case, pages mappable in user space, the page cache and anonymous memory.

For locked down memory a good solution looks like an incremental copy collector. Not that we need the garbage collecting properties, but it is the only incremental memory packing algorithm I currently know. And we could take advantage of fine grained smp locks to implement the needed write barrier.

AA> Right now i would need the reverse lookup only for the mapped cache AA> because I would like to avoid to run swap_out to know if the pte is been AA> accessed or not and in the case it's an old pte I could unmap the AA> mmapped-page directly from shrink_mmap. But I am not convinced this will AA> be an improvement too because I just run swap_out at the right time...

AA> Comments?

The reason I am looking at reverse page entries, is I would like to handle dirty mapped pages better.
My thought is basically to trap the fault that dirties the page and mark it dirty. Then after it has aged long enough I unmap or at least clear the write allow bits of the pte or ptes.

This does buy an improvement, in when things get written out. But beyond that I don't know.

It's certainly something to think about for your other algorithms. The current scheme seems good enough for the most part however.

Eric
--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Having the reverse lookup from pagemap to ptes would also make life a bit easier in my update_shared_mappings ;). So in general I see your point. Think when you'll clear the dirty bit from the pagemap, then you'll want to mark clean also the pte in the tasks. Right?

But I am worried by page faults. The page fault that allow us to know where there is an uptodate swap-entry on disk just hurt performances more than not having such information (I did benchmarks).

I am not sure if it's worthwhile, but I think it worth testing ;).

Andrea Arcangeli

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Hi,

It obviously depends on whether you are swap-bound or CPU-bound. Have you tried both?

One thing I definitely agree with is that it may sometimes be preferable to drop the swap cache to avoid fragmentation. If we have a new dirty page requiring writing to swap, and its VA neighbours are already in the swap cache, it makes sense to eliminate the swap cache and write all the pages to the new location to keep them contiguous on disk.

The real aim here is to allow us to keep dirty pages in the swap cache
too: this will allow us to keep good, unfragmented swap allocations by persistently assigning a contiguous range of swap to a contiguous range of process data pages, even if the process is only dirtying some of those pages.

--Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Hi,

I'd be very worried by (1): sounds like a partial write followed by a read of the full page could show up garbage in the page cache if you do this. If NFS skips the page clearing for partial writes, how does it avoid returning garbage later?

--Stephen

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Hmmm, it shouldn't be a problem if the write blocks the reading of the page and PG_uptodate isn't set. This conflicts with the current assumption in generic_file_read that a locked page becoming unlocked without PG_uptodate being set indicates an error -- the best thing here is probably to add a PG_error flag and do away with the overloading. Everything else should be checking PG_uptodate, right?

-ben

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Hi,

I'm not convinced: doing an explicit read-page and waking up to find it not uptodate sure sounds like an error to me. If we find a page which isn't uptodate, then the first thing we do is try to read it, we don't generate the error immediately. Why do we need a new flag?

--Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

ST> Hi,
ST> On 07 May 1999 09:56:00 -0500, [email protected] (Eric W. Biederman)

ST> I'd be very worried by (1): sounds like a partial write followed by a ST> read of the full page could show up garbage in the page cache if you do ST> this. If NFS skips the page clearing for partial writes, how does it ST> avoid returning garbage later?

Actually (1) is current behaviour. I really don't like it but I can see how it can potentially improve performance. Partial writes are handled by not setting PG_uptodate.

Reads are handled by always flushing the per page dirty data before reading.

I don't especially like it but it's what we have now.

Eric
--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/