Skip to contentSkip to conversation

2.5.48-mm1

DISCUSSION Andrew Morton 5 participants 5 messages Nov 19, 2002 – Nov 22, 2002 Memory management

lore
Share & tools
lore
#0 lore raw

url: http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.48/2.5.48-mm1/

Lots of little bits and pieces here. Most notably I've started to look at the scheduling latency of the uniprocessor preemptible kernel. It was actually pretty awful. Most everything in the MM/VFS area has been fixed here, and it is now achieving 500 microseconds max latency at 500MHz.

With the notable exception of the case where a task exits while holding a large amount of mmapped memory. 300 milliseconds for a 700 megabyte mapping, and a couple of milliseconds while just running cvs. This will take quite some fixing.

Only ext2 has been done. Other filesystems will need attention.

Since 2.5.47-mm3:

linus.patch

Latest from Linus.

+axboe-scsi-fix.patch

Fix a scsi plugging bug.

+kgdb-nmi-signal.patch
+kgdb-nr-cpus.patch

Some work against George's patch

+misc.patch

Little fixes

+radix-tree-height-reinit.patch

Fix a radix-tree bug

+radix-tree-overflow-fix.patch

And another (maybe - needs specific testing yet)

-mbcache-cleanup.patch
-rmap-flush-cache-page.patch
-swap-get_page-page-unlock.patch
-swap-writepages-swizzled.patch
-misc.patch
-htlb-combined-2.patch
-htlb-fixes.patch
-slab-no-BUG.patch
-remove-inode-buffers.patch
-mpage-kmap.patch
-inode-reclaim-balancing.patch

Merged

+loop-balance-pages.patch

Small optimisation to loop

+page-state-messages.patch

Handle bad pages in the page allocator without going BUG

+remove-lame-test.patch

Don't allow NULL pointers to be passed into wake_up any more.

+plugbug.patch

Fix an SMP race in the disk queue plugging code.

+mmapped-blockdev-warning.patch

Kill a warning which comes out with MAP_SHARED mappings of blockdevs

+s_dir_count-fix.patch

Fix ext2/3 directory counting (for the Orlov allocator heuristics; not a serious bug)

+rz-bootmem-fix.patch

Don't assume that physical memory starts at physical address 0

+dio-counting.patch

Direct-IO cleanups

+dio-reduce-context-switch-rate.patch

Reduce the context switch rate during direct-IO

+writeback-reduced-context-switches.patch

Reduced context switch rate during writeback

+scheduling-points.patch

Fix some long-held locks at the pagecache layer

+swap-accounting.patch

And in the swap accounting

+swapoff-cleanup.patch

Stuf from Hugh.

+page-reclaim-scheduling-points.patch

And in page reclaim

+sync_blockdev-lock-kernel.patch

And in the blockdev driver.

+incremental-slab-shrink.patch

And in the slab cache pruner.

+np-deadline.patch

Deadline scheduler work from Nick Piggin

+less-requests.patch

I reduced the size of the disk queues. A quarter-gig per disk
is excessive.

-dcache_rcu.patch
+dcache_rcu-2-2.5.48.patch
+dcache_rcu-3-2.5.48.patch

Split up

+shpte-protection-fix.patch

A shared pagetable fix.

All patches

linus.patch
cset-1.842-to-1.897.txt.gz

axboe-scsi-fix.patch

kgdb-ga.patch
kgdb stub for ia32 (George Anzinger's one)

kgdb-nmi-signal.patch

kgdb-nr-cpus.patch

misc.patch
misc fixes

rcu-stats.patch
RCU statistics reporting

genksyms-fix.patch
modversions fix for exporting per-cpu data

radix-tree-height-reinit.patch
Reinitialise radix tree height

radix-tree-overflow-fix.patch
handle overflows in radix_tree_gang_lookup()

buffer-debug.patch
buffer.c debugging

loop-balance-pages.patch
Use balance_dirty_pages_ratelimited() in loop.c

page-state-messages.patch
Expanded bad page handling

remove-lame-test.patch
remove a strange test in __wake_up()

congestion-wait.patch
Fix busy-wait with writeback to large queues

plugbug.patch
plug a plugging bug

mmapped-blockdev-warning.patch
remove a warning from __block_write_full_page()

s_dir_count-fix.patch
ext2/ext3 Orlov directory accounting fix

rz-bootmem-fix.patch
bootmem crash fix

back-to-writepage.patch
Remove mapping->vm_writeback

aio-direct-io-infrastructure.patch
AIO support for raw/O_DIRECT

aio-direct-io.patch
AIO support for raw/O_DIRECT

dio-counting.patch

dio-reduce-context-switch-rate.patch
Reduced wakeup rate in direct-io code

inlines-net.patch

reiserfs-readpages.patch
reiserfs v3 readpages support

reiserfs-readpages-fix.patch

resurrect-incremental-min.patch
strengthen the `incremental min' logic in the page allocator

unfreeable-zones.patch
VM: handle zones which are full of unreclaimable pages

nobh.patch
no-buffer-head ext2 option

writeback-reduced-context-switches.patch
reduced context switch rate in writeback

scheduling-points.patch
Add some low-latency scheduling points

swap-accounting.patch
realtime swapspace accounting

swapoff-cleanup.patch
swapoff accounting cleanup

page-reclaim-scheduling-points.patch
Add a scheduling point to page reclaim

sync_blockdev-lock-kernel.patch
Don't hold BKL across sync_blockdev() in blkdev_put()

simplified-vm-throttling.patch
Remove the final per-page throttling site in the VM

auto-unplug.patch
self-unplugging request queues

less-unplugging.patch
Remove most of the blk_run_queues() calls

page-reclaim-motion.patch
Move reclaimable pages to the tail ofthe inactive list on IO completion

handle-fail-writepage.patch
Special-case fail_writepage() in page reclaim

activate-unreleaseable-pages.patch
Move unreleasable pages onto the active list

incremental-slab-shrink.patch
reduced latency in dentry and inode cache shrinking

rbtree-iosched.patch
rbtree-based IO scheduler

np-deadline.patch
deadline IO scheduler tweaks

less-requests.patch
Go back to 128 read and 128 write requests per queue

page-reservation.patch
Page reservation API

wli-show_free_areas.patch
show_free_areas extensions

kmap-atomic-nfs.patch
Subject: Re: [RFC] use kmap_atomic in the NFS client

dcache_rcu-2-2.5.48.patch

dcache_rcu-3-2.5.48.patch

shpte-ng.patch
pagetable sharing for ia32

shpte-protection-fix.patch
shared pagetable protection fix

#1 lore raw
Quoted 2 lines · Andrew Morton

url: http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.48/2.5.48-mm1/

snip

Quoted 4 lines

Since 2.5.47-mm3:

snip

Quoted 4 lines

+np-deadline.patch

Deadline scheduler work from Nick Piggin

snip

This may degrade IO read latency a little bit and will still be suboptimal when there are both read and write requests outstanding for some io patterns (no worse than 47-mm3). Jens and I have been making progress here however.

Nick

#2 lore raw
In reply to Andrew Morton
Quoted 4 lines · Andrew Morton

+loop-balance-pages.patch

Small optimisation to loop

I disagree with this one (changing balance_dirty_pages to _ratelimited when loop_thread writes to file): it's a step in the right direction, but I think you should remove that balance_dirty_pages call completely.

I'm experimenting with what's needed to prevent deadoralivelock in loop over tmpfs under heavy memory pressure (thank you for eliminating wait_on_page_bit from shrink_list!). One element of that is to ignore balance_dirty_pages below loop (I hadn't noticed the explicit call, offhand I'm unsure whether that's the only possible instance).

The loop_thread is working towards undirtying memory (completing writeback): a loop of blk_congestion_waits is appropriate at the upper level where the user task generating dirt needs to be throttled, but I don't believe it's appropriate at this level - we wouldn't want to throttle the disk, no more should we throttle the loop_thread.

Hugh

#3 lore raw
Quoted 10 lines · Hugh Dickins

I disagree with this one (changing balance_dirty_pages to _ratelimited when loop_thread writes to file): it's a step in the right direction, but I think you should remove that balance_dirty_pages call completely.

I'm experimenting with what's needed to prevent deadoralivelock in loop over tmpfs under heavy memory pressure (thank you for eliminating wait_on_page_bit from shrink_list!). One element of that is to ignore balance_dirty_pages below loop (I hadn't noticed the explicit call, offhand I'm unsure whether that's the only possible instance).

Quoted 5 lines

The loop_thread is working towards undirtying memory (completing writeback): a loop of blk_congestion_waits is appropriate at the upper level where the user task generating dirt needs to be throttled, but I don't believe it's appropriate at this level - we wouldn't want to throttle the disk, no more should we throttle the loop_thread.

This is purely a performance decision. If you want to avoid bad latency on reads then you have to throttle writes. The loop_thread will make the system just as slow as a user application writing the same number of pages.

If you want io scheduling you will deliberately slow writes to let reads happen in reasonable time. And vice-versa I imagine, although I don't think I've seen that case.

Signature · 3 lines
bill davidsen <[email protected]>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
#4 lore raw
Quoted 7 lines · Bill Davidsen

This is purely a performance decision. If you want to avoid bad latency on reads then you have to throttle writes. The loop_thread will make the system just as slow as a user application writing the same number of pages.
If you want io scheduling you will deliberately slow writes to let reads happen in reasonable time. And vice-versa I imagine, although I don't think I've seen that case.

Not entirely so. This is just a scheduling decision that has to discriminate between blocking and nonblocking requests and prevent starvation of the blocking requests. Write throttling is an oversimplification that functions poorly.

Bill