Why don't we make mmap MAP_SHARED with /dev/zero possible?
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 5 lines · Christoph Rohland
This lines up with some remarks from Eric Biederman about his shmfs, which is BTW a feature I would _love_ to have in Linux to do posix shm and perhaps redo sysv shm. He said that he would like to make the pagecache highmem-capable and AFAIK the main work for shmfs was makeing the pagecache working with writeable pages.
hm, i've got the pagecache in high memory already on my box, patch under cleanup right now. It was the next natural step after doing all the hard work to get 64GB RAM support. Eric, is there any conflicting work here?
-- mingo
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/
Quoted 11 lines · Ingo Molnar
Quoted 5 lines · Christoph Rohland
This lines up with some remarks from Eric Biederman about his shmfs, which is BTW a feature I would _love_ to have in Linux to do posix shm and perhaps redo sysv shm. He said that he would like to make the pagecache highmem-capable and AFAIK the main work for shmfs was makeing the pagecache working with writeable pages.
hm, i've got the pagecache in high memory already on my box, patch under cleanup right now. It was the next natural step after doing all the hard work to get 64GB RAM support. Eric, is there any conflicting work here?
Not really. I played with the idea, and the only really tricky aspect I saw was how to write a version of copy_to/from_user that would handle the bigmem
case. Because kmap ... copy .. kunmap isn't safe as you can sleep due
to a page fault.
I got about half way to a solution by having the page fault handler basically act like an exception handler, and switch the return address for this one specific case. So eventualy when the page fault would return area would magically rekmap itself, and continue with life. Duing it this was is important as it only penalizes the uncommon case. So within a clock or two highmem_copy_to/from_user should be as fast as copy_to/from_user.
And I played with putting a wrapper around ll_rw_block calls in buffer.c that would allocate bounce buffers from the buffer cache as needed.
I've been a little busy so keeping up with the kernel changes has been too much just lately. I wound up hacking on dosemu instead where I can out a six month old patch and finish it up. . .
I'll probably get back to shmfs in a kernel version or two.
Quoted 1 line
From the last pre-2.3.25-3 it looks like everything I have proposed,
except moving bdflush to the page cache level is finding it's way into the kernel. And that last isn't critical for 2.4+
So when I get back to hacking it. I'm going to concentrate on the practical things needed to get shmfs working on 2.3.25+ And let some of the rest of you work on the generic mechanisms, you are doing fine right now. . .
If you'd like to compare mechanisms or whatever I'd be happy too.
Eric
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/
Quoted 4 lines · Eric W. Biederman
Not really. I played with the idea, and the only really tricky aspect I saw was how to write a version of copy_to/from_user that would handle the bigmem
case. Because kmap ... copy .. kunmap isn't safe as you can sleep due
to a page fault.
yes, i implemented a new 'kaddr = kmap_permanent(page)' 'kunmap_permanent(kaddr)' interface which is schedulable. This is now getting used in exec.c (argument pages can be significantly big) and the page cache.
Quoted 3 lines
And I played with putting a wrapper around
ll_rw_blockcalls inbuffer.cthat would allocate bounce buffers from the buffer cache as needed.
that is a much more problematic issue, especially if you consider future 64-bit PCI DMAing. What i did was to change bh->b_data to bh->b_page, which b_page is a 32-bit value describing the physical address of the buffer, in 512-byte units. This also ment changing bazillion places where b_data was used (lowlevel fs, buffer-cache and block layer, device drivers) ... But it's working just fine on my box:
moon:~> cat /proc/meminfo
MemTotal: 8249708 kB
MemFree: 7760256 kB
MemShared: 0 kB
Buffers: 20292 kB
Cached: 432052 kB <=== 432M pagecache
HighTotal: 7471104 kB
HighFree: 7035928 kB <=== 444M high memory allocated
LowTotal: 778604 kB
LowFree: 724328 kB <=== 50M normal memory allocated
SwapTotal: 0 kB
SwapFree: 0 kBQuoted 1 line
I'll probably get back to shmfs in a kernel version or two.
looking forward to test it, i believe we could get some spectacular benchmark numbers with that thing and 2.4 ...
-- mingo
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/
Quoted 11 lines · Ingo Molnar
Quoted 4 lines · Eric W. Biederman
Not really. I played with the idea, and the only really tricky aspect I saw was how to write a version of copy_to/from_user that would handle the bigmem
case. Because kmap ... copy .. kunmap isn't safe as you can sleep due
to a page fault.yes, i implemented a new 'kaddr =
kmap_permanent(page)' 'kunmap_permanent(kaddr)' interface which is schedulable. This is now getting used inexec.c(argument pages can be significantly big) and the page cache.
Do you have a patch around that the rest of us can look at?
Quoted 6 lines
that is a much more problematic issue, especially if you consider future 64-bit PCI DMAing. What i did was to change
bh->b_datatobh->b_page, whichb_pageis a 32-bit value describing the physical address of the buffer, in 512-byte units. This also ment changing bazillion places whereb_datawas used (lowlevel fs, buffer-cache and block layer, device drivers) ... But it's working just fine on my box:
Click.
Which lets up access up to 2Terabytes of ram, on a 32 bit machine. And you have to do something like that or you can't put buffers on those high pages even temporarily. I missed that trick.
Quoted 4 lines
Quoted 1 line
I'll probably get back to shmfs in a kernel version or two.
looking forward to test it, i believe we could get some spectacular benchmark numbers with that thing and 2.4 ...
We'll see. I just want to get it functional first. There are no binary compatibility constrainsts so after it works any optimizations are easy :)
Eric
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/
[snip]
Q: Highmem allocation for the page cache.
Note: page_cache_alloc currently doesn't take any parameters.
It should take __GFP_BIGMEM or whatever so we can yes high memory is ok. Or no.
I'm going to put metadata in this page and high memory is not o.k, too inconvienint.
Eric
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/
Quoted 4 lines · Eric W. Biederman
Q: Highmem allocation for the page cache.
Note:
page_cache_alloccurrently doesn't take any parameters. It should take__GFP_BIGMEMor whatever so we can yes high memory is ok.
it's now an unconditional __GFP_HIGHMEM in my tree. HIGHMEM gfp() allocation automatically falls back to allocate in lowmem, if highmem lists are empty.
Quoted 3 lines
Or no.
I'm going to put metadata in this page and high memory is not o.k, too inconvienint.
hm, i see, this makes sense. permanent mappings are not inconvenient at all (you can hold a number of them, can sleep inbetween), but maybe we still want to allocate in low memory in some cases, for performance reasons. It's not a problem at all and completely legal, as low memory pages can happen anyway, so everything is completely symmetric. Any 'high memory enabled code' automatically works with low memory (or exclusive low memory) as well.
-- mingo
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/
Quoted 3 lines · Ingo Molnar
it's now an unconditional
__GFP_HIGHMEMin my tree. HIGHMEMgfp()allocation automatically falls back to allocate in lowmem, if highmem lists are empty.
I'd like to look through the patches to see how you're doing things before making any comments. Specifically, I want to look at the buffer head address thing that was mentioned -- given that the devices that support addressing memory above 4G will expect a 64 bit address, I don't think the shift is the right way to go.
-ben
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/
Quoted 6 lines · Benjamin C.R. LaHaise
Quoted 3 lines
it's now an unconditional
__GFP_HIGHMEMin my tree. HIGHMEMgfp()allocation automatically falls back to allocate in lowmem, if highmem lists are empty.I'd like to look through the patches to see how you're doing things before making any comments. Specifically, I want to look at the buffer head
(soon, probably tomorrow)
Quoted 3 lines
address thing that was mentioned -- given that the devices that support addressing memory above 4G will expect a 64 bit address, I don't think the shift is the right way to go.
i introduced bh->b_page _specifically_ to support 64-bit addresses. Below <4GB RAM DMA32 can be supported with no IO-layer changes. Right now it supports up to 2TB DMA64 target space (on 32-bit boxes), which is more than enough.
-- mingo
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/