2.5.50-mm2

url: http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.50/2.5.50-mm2/

-> 2.5.50-mm2.gz Full patch
-> 2.5.50-mm2-shpte.gz Up to `shpte-always-on', for Dave to work against

Lots of fixes. Some things dropped for various reasons. A decent amount of rework against the high-level writeback code.

Since 2.5.50-mm1:

-fbcon-timer-fix.patch
-hugepage-fixes.patch
-ext3-oldalloc.patch
-simplified-vm-throttling.patch
-page-reclaim-motion.patch
-handle-fail-writepage.patch
-activate-unreleaseable-pages.patch
-ipc_barriers.patch
-signal-speedup.patch
-pf_memdie.patch
-suppress-write-error-warnings.patch
-truncate-speedup.patch
-spill-lru-lists.patch
-readdir-speedup.patch (Linus fixed all the bugs in it)

Merged

-epoll-bits-0.57.patch
+epoll.patch

Davide's latest

-aio-dio-really-submit.patch
-aio-dio-deferred-dirtying.patch
-dio-counting.patch

Folded into other patches.

-reiserfs-readpages.patch

Dropped - it was causing fsx-linux failures.

+cputimes_stat.patch

Put the per-cpu times back into /proc/pid/stat, CONFIGurably

-poll-1-wqalloc.patch
-poll-2-selectalloc.patch
-poll-3-alloc.patch
-poll-4-fast-select.patch
-poll-5-fast-poll.patch
-poll-6-merge.patch

Dropped. Increased stack utilisation was a concern, and the performance benefits are patchy. Manfred is taking another look at it.

+deadsched-fix.patch

Deadline scheduler fix

+shpte-nonlinear.patch

Fix shared pagetables for nonlinear mappings

+pmd-allocation-fix.patch

Always allocate PMD's out to the top of the virtual address space.

-dcache_rcu-2-2.5.48.patch
-dcache_rcu-3-2.5.48.patch

Dropped. Other changes broke it, and Dipankar is redoing a few things anyway.

+filldir-checks.patch

Some copy_*_user checks.

+vmstats-fixes.patch

Small vm stats fix and enhancement.

+hugetlb-fixes.patch

Fixes from Rohit.

+writeback-interaction-fix.patch

I had a few oddities and bogons in the fs/fs-writeback.c code. This patch is a fair-sized revamp, cleanup and general sort-things-out in there. It works better.

+scalable-zone-protection.patch

Allow runtime tuning of the lower-zone protection ratio. Needs work.

+page-wait-table-min-size.patch

Don't allow a one-slot hashed wait table for wait_on_page()

+ext3-transaction-reserved-blocks.patch

Fix an ext3 assertion failure which is triggerable with corrupt disk contents.

+remove-PF_SYNC.patch

remove the current->flags:PF_SYNC abomination. Adds a `sync' arg to all writepage implementations to tell them whether they are being called for memory cleansing or for data integrity.

+dont-inherit-mlockall.patch

Don't inherit mlockall(MCL_FUTURE) across forks

+bootmem-alloc-alignment.patch

Improved coalescing of bootmem regions

+ext23_free_blocks-check.patch

Additional sanity checks in the block allocator.

+blkdev-rlimit.patch

Don't apply rlimits to blockdev writes.

+readahead-pinned-memory.patch

Don't allow gargantuan amounts of memory to be pinned in readahead

-page-walk-api-improvements.patch
-page-walk-api-bugfix.patch

Folded into page-walk-api.patch

+page-walk-api-update.patch

New stuff from Ingo (needs splitting up and changelogging...)

All 55 patches:

linus.patch
cset-1.797.133.2-to-1.857.txt.gz

epoll.patch
epoll bits 0.59 ...

kgdb.patch

dio-return-partial-result.patch

aio-direct-io-infrastructure.patch
AIO support for raw/O_DIRECT

deferred-bio-dirtying.patch
bio dirtying infrastructure

aio-direct-io.patch
AIO support for raw/O_DIRECT

aio-dio-debug.patch

dio-reduce-context-switch-rate.patch
Reduced wakeup rate in direct-io code

cputimes_stat.patch
Retore per-cpu time accounting, with a config option

deprecate-bdflush.patch
deprecate use of bdflush()

reduce-random-context-switch-rate.patch
Reduce context switch rate due to the random driver

bcrl-printk.patch

read_zero-speedup.patch
speed up read_zero() for !CONFIG_MMU

nommu-rmap-locking.patch
Fix rmap locking for CONFIG_SWAP=n

semtimedop.patch
semtimedop - semop() with a timeout

writeback-handle-memory-backed.patch
skip memory-backed filesystems in writeback

remove-fail_writepage.patch
Remove fail_writepage()

page-reservation.patch
Page reservation API

wli-show_free_areas.patch
show_free_areas extensions

inlines-net.patch

rbtree-iosched.patch
rbtree-based IO scheduler

deadsched-fix.patch
deadline scheduler fix

quota-smp-locks.patch
Subject: [PATCH] Quota SMP locks

shpte-ng.patch
pagetable sharing for ia32

shpte-nonlinear.patch
shpte: support nonlinear mappings and clean up clear_share_range()

shpte-always-on.patch
Force CONFIG_SHAREPTE=y for ia32

pmd-allocation-fix.patch
make sure all PMDs are allocated under PAE mode

ptrace-flush.patch
Subject: [PATCH] ptrace on 2.5.44

buffer-debug.patch
buffer.c debugging

warn-null-wakeup.patch

pentium-II.patch
Pentium-II support bits

radix-tree-overflow-fix.patch
handle overflows in radix_tree_gang_lookup()

rcu-stats.patch
RCU statistics reporting

auto-unplug.patch
self-unplugging request queues

less-unplugging.patch
Remove most of the blk_run_queues() calls

sync_fs.patch
Add a sync_fs super_block operation

ext3_sync_fs.patch
implement ext3_sync_fs

ext3-fsync-speedup.patch
Clean up ext3_sync_file()

filldir-checks.patch
copy_user checks in filldir()

vmstats-fixes.patch
vm accounting fixes and addition

hugetlb-fixes.patch
hugetlb fixes

writeback-interaction-fix.patch
fs-writeback rework.

scalable-zone-protection.patch
Add /proc/sys/vm/lower_zone_protection

page-wait-table-min-size.patch
Set a minimum hash table size for wait_on_page()

ext3-transaction-reserved-blocks.patch
Reserve an additional transaction block in ext3_dirty_inode

remove-PF_SYNC.patch

dont-inherit-mlockall.patch
Don't inherit mm->def_flags across forks

bootmem-alloc-alignment.patch
bootmem allocator merging fix

ext23_free_blocks-check.patch
ext2/ext3_free_blocks() extra check

blkdev-rlimit.patch
don't allpy file size rlimits to blockdevs

readahead-pinned-memory.patch
limit pinned memory due to readahead

page-walk-api.patch

page-walk-scsi.patch

page-walk-api-update.patch
pagewalk API update

Any chance you could pass down a struct writeback_control instead of just the sync flag? XFS always used ->writepage similar to the ->vm_writeback in older kernel releases because writing out more than one page of delalloc space is really needed to be efficient and this would allow us to get a few more hints about the VM's intentions.

Yup, no probs.

It would be good to measure how often that codepath actually gets invoked during testing and use. It's typically quite rare. It should be just MAP_SHARED stuff, although there are probably some highmem-related scenarii in which it will happen.

I'll add a writeback_control.for_reclaim boolean so we don't have to play games with PF_MEMALLOC to reverse engineer the calling context.

If XFS is going to writearound extra pages in ->writepage() then it would be best to set PG_reclaim (if wbc->for_reclaim) so end_page_writeback() will rotate them.

Hi,

Without this QTDIR needs to be set to /usr/share/qt3 with the latest qt 3.1.1 in debian unstable. This fixes it.

Ed Tomlinson

# This is a BitKeeper generated patch for the following project:
# Project Name: Linux kernel tree
# This patch format is intended for GNU patch command version 2.5 or higher. # This patch includes the following deltas:
# ChangeSet 1.961 -> 1.962
# scripts/kconfig/Makefile 1.5 -> 1.6
#
# The following is the BitKeeper ChangeSet Log
# --------------------------------------------
# 03/02/04 [email protected] 1.962

# find debian sid's 'libqt3-headers' without setting QTDIR manually
# --------------------------------------------
#
diff -Nru a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
--- a/scripts/kconfig/Makefile	Tue Feb  4 23:40:13 2003
+++ b/scripts/kconfig/Makefile	Tue Feb  4 23:40:13 2003
@@ -38,7 +38,7 @@

# QT needs some extra effort...
$(obj)/.tmp_qtcheck:
- @set -e; for d in $$QTDIR /usr/share/qt /usr/lib/qt*3*; do \
+ @set -e; for d in $$QTDIR /usr/share/qt /usr/lib/qt*3* /usr/share/qt*3*; do \
if [ -f $$d/include/qconfig.h ]; then DIR=$$d; break; fi; \
done; \
if [ -z "$$DIR" ]; then \

This has been around for a while... Its becoming a bit if a pain since debian unstable flipped to gcc3.2 as its default compiler. This works with gcc2.95. Is gcc3.2 detecting an error 2.95 misses?

ld -m elf_i386 -r -o drivers/usb/input/hid.o drivers/usb/input/hid-core.o drivers/usb/input/hid-input.o ld -m elf_i386 -r -o drivers/usb/input/hid.ko drivers/usb/input/hid.o init/vermagic.o
make -f scripts/Makefile.build obj=drivers/usb/serial
rm -f drivers/usb/serial/built-in.o; ar rcs drivers/usb/serial/built-in.o
gcc -Wp,-MD,drivers/usb/serial/.usb-serial.o.d -D__KERNEL__ -Iinclude -Wall -Wstrict-prototypes -Wno-trigraphs -O2 -fno-strict-aliasing -fno-common -pipe -mpreferred-stack-boundary=2 -march=k6 -Iinclude/asm-i386/mach-default -nostdinc -iwithprefix include -DMODULE -DKBUILD_BASENAME=usb_serial -DKBUILD_MODNAME=usbserial -c -o drivers/usb/serial/usb-serial.o drivers/usb/serial/usb-serial.c
{standard input}: Assembler messages:
{standard input}:2603: Error: value of -129 too large for field of 1 bytes at 5959 make[3]: *** [drivers/usb/serial/usb-serial.o] Error 1
make[2]: *** [drivers/usb/serial] Error 2
make[1]: *** [drivers/usb] Error 2
make: *** [drivers] Error 2

I have a pl2303 based device.

Ed Tomlinson

I dug into this a bit more. It would seem to be a compiler bug, where it tries to branch back 129 bytes... I will report it using debian channels.

The following change will let things compile until gcc is fixed.

diff -Nru a/drivers/usb/serial/usb-serial.c b/drivers/usb/serial/usb-serial.c
@@ -1114,6 +1114,7 @@
 		port->magic = USB_SERIAL_PORT_MAGIC;
 		INIT_WORK(&port->work, usb_serial_port_softint, port);
 		init_MUTEX (&port->sem);
+		dev_info(&interface->dev, "endpoints\n");
 	}
 
        /* if this device type has an attach function, call it */

This is will the following gcc:

oscar% gcc --version
gcc (GCC) 3.2.2

Ed Tomlinson

Thanks for finding this, but I don't think that work around is ok as it's printing out something that isn't necessary :)

thanks,

greg k-h

Agreed. I posted it just so anyone with the problem would know one way to fix it - I do not propose this for inclusion.

Ed

Hi

Here is an update of my thread group and user governor patch. This patch reduces timeslices of process belonging to groups with too many active processes. This, in effect, lowers their priority and makes it more likely that other tasks can run preventing tasks in a governed group from using too large a share of the the cpu.

There is a governor for thread groups, which are defined as processes sharing both mm and files or processes with CLONE_THREAD set. The default governor is set to 1.5 threads (15). This means that when there are 2 or more threads active the timeslices will be reduced as follows:

slice = (slice * 1.5) / 2

A second governor is defined for user processes. This one defaults to 30 active task (300). A process has only one governor. If its a member of a thread group that governor applies and it is ignored by the user governor process.

A couple of notes about the patch. There is debug code in sched.c that can be enabled if you want to see what is happening. I would not recommend this code be used on NUMA systems. I am looking into a version of the patch that would impose the governors on a per node basis. Earlier versions of this patch counted user processes incorrectly which caused the user governor to trigger limiting users when it should not have. This is now fixed.

As it stands now a modified schedule-tunables patch based off the one found in the mm kernels can optionally be applied. I think that using the rlimit infrastructure probably makes more sense - comments?

The patches should apply to 60-mk and probably to -mm1 after reversing the schedule-tunables included in it.

The patch as been tested on UP.

As always comments/suggestion appriecated

Ed Tomlinson

The numbers from the second message are nice - especially considering this is only the second iteration...

A question about io priorities. I wonder if they could not be implemented via a per pid cfq_quantum? If I am not missunderstanding things, a bigger value here for a given process should mean that it gets a larger share of the io bandwidth...

Comments?
Ed Tomlinson

That's exactly right, and yes that would be one way of doing it.

Jens Axboe