why my systems never cache more than ~900 MB?
On my (32 bit) systems with more than 1 GB memory it is impossible to cache more than about 900 MB. Why?
Caching never goes beyond ~900 MB (i.e. when I read a mounted drive with dd):
# free
total used free shared buffers cached
Mem: 2076164 966788 1109376 0 855132 68932
-/+ buffers/cache: 42724 2033440
Swap: 2097144 0 2097144Same behaviour on 32 bit machines with 4 GB RAM.
No problems on 64 bit machines.
I have one 32 bit machine that caches beyond ~900 MB without problems.
Is it some kernel/proc/sys setting that I'm missing?
Signature · 8 lines
Tomasz Chmielewski http://wpkg.org -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 5 lines · On Tuesday 24 March 2009 19:42:08 Tomasz Chmielewski wrote:
On my (32 bit) systems with more than 1 GB memory it is impossible to cache more than about 900 MB. Why?
Caching never goes beyond ~900 MB (i.e. when I read a mounted drive with dd):
Because blockdev mappings are limited to lowmem due to sharing their cache with filesystem metadata cache, which needs kernel mapped memory. It will >900MB of pagecache data OK (data from regular files)
Quoted 11 lines
# free total used free shared buffers cached Mem: 2076164 966788 1109376 0 855132 68932 -/+ buffers/cache: 42724 2033440 Swap: 2097144 0 2097144Same behaviour on 32 bit machines with 4 GB RAM.
No problems on 64 bit machines.
I have one 32 bit machine that caches beyond ~900 MB without problems.
Does it have a different user/kernel split?
Quoted 1 line
Is it some
kernel/proc/syssetting that I'm missing?
No, it just can't be done without changing code.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 10 lines · Nick Piggin
Quoted 5 lines · On Tuesday 24 March 2009 19:42:08 Tomasz Chmielewski wrote:
On my (32 bit) systems with more than 1 GB memory it is impossible to cache more than about 900 MB. Why?
Caching never goes beyond ~900 MB (i.e. when I read a mounted drive with dd):
Because blockdev mappings are limited to lowmem due to sharing their cache with filesystem metadata cache, which needs kernel mapped memory. It will >900MB of pagecache data OK (data from regular files)
Does not help me, as what interests me here on these machines is mainly caching block device data; they are iSCSI targets and access block devices directly.
(...)
Quoted 6 lines
Quoted 4 lines
Same behaviour on 32 bit machines with 4 GB RAM.
No problems on 64 bit machines.
I have one 32 bit machine that caches beyond ~900 MB without problems.Does it have a different user/kernel split?
Yes it does:
CONFIG_VMSPLIT_2G_OPT=y
What split should I choose to enable blockdev mapping on the whole memory on 32 bit system with 3 or 4 GB RAM? Is it possible with 4 GB RAM at all?
Quoted 3 lines
Quoted 1 line
Is it some
kernel/proc/syssetting that I'm missing?No, it just can't be done without changing code.
Signature · 8 lines
Tomasz Chmielewski http://wpkg.org -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 8 lines · On Wednesday 25 March 2009 02:35:48 Tomasz Chmielewski wrote:
Quoted 7 lines · Nick Piggin
Quoted 5 lines · On Tuesday 24 March 2009 19:42:08 Tomasz Chmielewski wrote:
On my (32 bit) systems with more than 1 GB memory it is impossible to cache more than about 900 MB. Why?
Caching never goes beyond ~900 MB (i.e. when I read a mounted drive with dd):
Quoted 7 lines
Quoted 3 lines
Because blockdev mappings are limited to lowmem due to sharing their cache with filesystem metadata cache, which needs kernel mapped memory. It will >900MB of pagecache data OK (data from regular files)
Does not help me, as what interests me here on these machines is mainly caching block device data; they are iSCSI targets and access block devices directly.
OK. Yeah, it's just due to crufty old code... I'm slowly working to remove this limitation (can be done in 1 of 2 ways: either disconnect filesystem metadata cache from blockdev data cache, or rework filesystems to be able to handle highmem for metadata cache).
Actually the quickest way for you might be just to create a new type of block device that can use highmem but not support filesystems.
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Same behaviour on 32 bit machines with 4 GB RAM.
No problems on 64 bit machines.
I have one 32 bit machine that caches beyond ~900 MB without problems.Does it have a different user/kernel split?
Quoted 8 lines
Yes it does:
CONFIG_VMSPLIT_2G_OPT=yWhat split should I choose to enable blockdev mapping on the whole memory on 32 bit system with 3 or 4 GB RAM? Is it possible with 4 GB RAM at all?
It's not possible to use 4GB RAM with VMSPLIT. You could use VMSPLIT_1G to give the kernel the most amount of virtual memory. And even reduce vmalloc space to get a few more MB with vmalloc= boot parameter.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 4 lines · Tomasz Chmielewski
Nick Piggin schrieb:
Does not help me, as what interests me here on these machines is mainly caching block device data; they are iSCSI targets and access block devices directly.
You can run a 64 bit kernel on those machines. 64 bit kernels can use 32 bit userspace without a problem. Just install an additional kernel and try booting your existing setup with it.
Quoted 2 lines
What split should I choose to enable blockdev mapping on the whole memory on 32 bit system with 3 or 4 GB RAM? Is it possible with 4 GB RAM at all?
A 64 bit kernel will do the trick.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 7 lines · Christoph Lameter
Quoted 4 lines · Tomasz Chmielewski
Nick Piggin schrieb:
Does not help me, as what interests me here on these machines is mainly caching block device data; they are iSCSI targets and access block devices directly.
Quoted 8 lines
You can run a 64 bit kernel on those machines. 64 bit kernels can use 32 bit userspace without a problem. Just install an additional kernel and try booting your existing setup with it.
Quoted 2 lines
What split should I choose to enable blockdev mapping on the whole memory on 32 bit system with 3 or 4 GB RAM? Is it possible with 4 GB RAM at all?
A 64 bit kernel will do the trick.
This hardware has problems booting 64 bit kernels (read: CPUs come from the 32-bit land).
Signature · 8 lines
Tomasz Chmielewski http://wpkg.org -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 12 lines · Tomasz Chmielewski
Quoted 11 lines · Christoph Lameter
Quoted 4 lines · Tomasz Chmielewski
Nick Piggin schrieb:
Does not help me, as what interests me here on these machines is mainly caching block device data; they are iSCSI targets and access block devices directly.You can run a 64 bit kernel on those machines. 64 bit kernels can use 32 bit userspace without a problem. Just install an additional kernel and try booting your existing setup with it.
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
What split should I choose to enable blockdev mapping on the whole memory on
32 bit system with 3 or 4 GB RAM? Is it possible with 4 GB RAM at all?A 64 bit kernel will do the trick.
This hardware has problems booting 64 bit kernels (read: CPUs come from the 32-bit land).
Then the 1G/3G separation (VMSPLIT_1G) is the best you can do. Will use up to 3G for low mem. Be aware that your I/O device must support DMA to full 32 bit addresses. If this is an old 32 bit machine with limitations on the use of bit 31 then the box may have issues.
Put as much as you can into Highmem. Set HIGHMEM4G, HIGHPTE
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>