Skip to contentSkip to conversation

2.5.41-mm2

DISCUSSION Andrew Morton 5 participants 7 messages Oct 10, 2002 – Oct 11, 2002

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

url: http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.41/2.5.41-mm2/

. The per-cpu pages patches continue to disgrace themselves. Improvements
in some microbenchmarks range from moderate (2%) to stunning (60%) but we're seeing a consistent few-percent regression in tests which perform networking to localhost. Heads continue to be scratched. This is irritating because that code is supposed to be the foundation for a page reservation API which fixes the radix-tree and pte_chain allocation failure problems.

Ingo's original per-cpu-pages patch was said to be mainly beneficial for web-serving type things, but no specweb testing has been possible for a week or two due to oopses in the timer code.

. David McCracken's shared pagetable implementation is included - it is
configurable on or off under the "Processor type and features" menu.

. The swappiness control code seems to be working well.

-lbd1.patch
-lbd2.patch
-lbd3.patch
-lbd4.patch
-lbd5.patch
-lbd6.patch
-get_bios_geometry.patch
-64-bit-sector_t.patch
-alloc_pages_node-cleanup.patch
-discontig-no-contig_page_data.patch
-discontig-setup-fix.patch
-remove-get_free_page.patch
-ext3-dxdir.patch
-free_area_init-cleanup.patch
-per-node-mem_map.patch
-wli-libfs.patch

Merged

+ext3-yield.patch

Remove a sched_yield() from ext3 which causes fsync() to enormously suck when the machine is under CPU load.

+guruhugh.patch

Fix a use-after-free bug in the mremap code.

+pte-highmem-warning.patch

Fix an "illegal sleep" warning in the mremap code

+raid0-fix.patch

Peter Chubb's "make raid0 work again" patch.

+ramfs-prepare-write-speedup.patch

Some fiddling with the fs/libfs address_space ops implementation.

+readv-writev-check-fix.patch

Make readv/writev return zero for zero segments

+shpte.patch

Shared pagetables for ia32

+slab-split-10-list_for_each_fix.patch

Fix a slab bug

+timer-tricks.patch

Tries to fix the timer problems, but doesn't.

linus.patch
cset-1.573.96.12-to-1.746.txt.gz

timer-tricks.patch

guruhugh.patch
Fix use-after-free bug in move_page_tables

pte-highmem-warning.patch
Fix an atomicity warning with pte-highmem

raw-use-o_direct.patch
Fix the raw driver

remove-radix_tree_reserve.patch
remove radix_tree_reserve()

ext3-yield.patch
Speed up ext3 fsyncs

readv-writev-check-fix.patch
readv/writev: return zero for zero segments

misc.patch
misc

swsusp-feature.patch
add shrink_all_memory() for swsusp

large-queue-throttle.patch
Improve writer throttling for small machines

exit-page-referenced.patch
Propagate pte referenced bit into pagecache during unmap

swappiness.patch
swappiness control

mapped-start-active.patch
start anonymous pages on the active list

rename-dirty_async_ratio.patch
rename dirty_async_ratio to dirty_ratio

auto-dirty-memory.patch
adaptive dirty-memory thresholding

batched-slab-asap.patch
batched slab shrinking and shrinker callback API

dio-fine-alignment.patch
Allow O_DIRECT to use 512-byte alignment

orlov-allocator.patch

lseek-ext2_readdir.patch
remove lock_kernel() from ext2_readdir()

write-deadlock.patch
Fix the generic_file_write-from-same-mmapped-page deadlock

rd-cleanup.patch
Cleanup and fix the ramdisk driver (doesn't work right yet)

spin-lock-check.patch
spinlock/rwlock checking infrastructure

hugetlb-prefault.patch
hugetlbpages: factor out some code for hugetlbfs

ramfs-aops.patch
Move ramfs address_space ops into libfs

hugetlb-header-split.patch
Move hugetlb declarations into their own header

hugetlbfs.patch
hugetlbfs file system

hugetlb-shm.patch
hugetlbfs backing for SYSV shared memory

ramfs-prepare-write-speedup.patch
correctness fixes in libfs address_space ops

akpm-deadline.patch
deadline scheduler tweaks

intel-user-copy.patch
Faster copt_*_user for Intel ia32 CPUs

raid0-fix.patch
RAID0 fix

rmqueue_bulk.patch
bulk page allocator

free_pages_bulk.patch
Bulk page freeing function

hot_cold_pages.patch
Hot/Cold pages and zone->lock amortisation

readahead-cold-pages.patch
Use cache-cold pages for pagecache reads.

pagevec-hot-cold-hint.patch
hot/cold hints for truncate and page reclaim

page-reservation.patch
Page reservation API

slab-split-01-rename.patch
slab cleanup: rename static functions

slab-split-02-SMP.patch
slab: enable the cpu arrays on uniprocessor

slab-split-03-tail.patch
slab: reduced internal fragmentation

slab-split-04-drain.patch
slab: take the spinlock in the drain function.

slab-split-05-name.patch
slab: remove spaces from /proc identifiers

slab-split-06-mand-cpuarray.patch
slab: cleanups and speedups

slab-split-07-inline.patch
slab: uninline poisoning checks

slab-split-08-reap.patch
slab: reap timers

cpucache_init-fix.patch
cpucache_init fix

slab-split-10-list_for_each_fix.patch
slab: for a list walking bug

shpte.patch

read_barrier_depends.patch
extended barrier primitives

rcu_ltimer.patch
RCU core

dcache_rcu.patch
Use RCU for dcache

#1 lore raw

but we're seeing a consistent few-percent regression in tests which perform networking to localhost.

There's debugging code in loopback that is helping us stress test the TCP segmentation offload, you might want to disable that to get more reliable numbers in 2.5.x.

Try this:

net/loopback.c renamed from net/loopback.c.~1~ +2 -2
@@ -190,12 +190,12 @@
 	dev->rebuild_header	= eth_rebuild_header;
 	dev->flags		= IFF_LOOPBACK;
 	dev->features		= NETIF_F_SG|NETIF_F_FRAGLIST|NETIF_F_NO_CSUM|NETIF_F_HIGHDMA;
-
+#if 0
 	/* Current netfilter will die with oom linearizing large skbs,
 	 * however this will be cured before 2.5.x is done.
 	 */
 	dev->features	       |= NETIF_F_TSO;
-
+#endif
 	dev->priv = kmalloc(sizeof(struct net_device_stats), GFP_KERNEL);
 	if (dev->priv == NULL)
 			return -ENOMEM;

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/

#2 lore raw
In reply to Andrew Morton
Quoted 3 lines · Andrew Morton

Ingo's original per-cpu-pages patch was said to be mainly beneficial for web-serving type things, but no specweb testing has been possible for a week or two due to oopses in the timer code.

i sent my latest timer patch to Dave Hansen but have not heard back since. I've attached the latest patch, this kernel also printks a bit more when it sees invalid timer usage.

in any case, the oops Dave was seeing i believe was fixed by Linus (the PgUp fix), and it was in the keyboard code. If there's anything else still going on then the attached patch should either fix it or provide further clues.

Ingo

kernel/timer.c renamed from kernel/timer.c.orig +22 -24
@@ -266,29 +266,31 @@
 int del_timer_sync(timer_t *timer)
 {
 	tvec_base_t *base = tvec_bases;
-	int i, ret;
+	int i, ret = 0;
 
-	ret = del_timer(timer);
+del_again:
+	ret += del_timer(timer);
 
-	for (i = 0; i < NR_CPUS; i++) {
+	for (i = 0; i < NR_CPUS; i++, base++) {
 		if (!cpu_online(i))
 			continue;
 		if (base->running_timer == timer) {
 			while (base->running_timer == timer) {
 				cpu_relax();
-				preempt_disable();
-				preempt_enable();
+				preempt_check_resched();
 			}
 			break;
 		}
-		base++;
 	}
+	if (timer_pending(timer))
+		goto del_again;
+
 	return ret;
 }
 #endif
 
 
-static void cascade(tvec_base_t *base, tvec_t *tv)
+static int cascade(tvec_base_t *base, tvec_t *tv)
 {
 	/* cascade all the timers from tv up one level */
 	struct list_head *head, *curr, *next;
@@ -310,7 +312,8 @@
 		curr = next;
 	}
 	INIT_LIST_HEAD(head);
-	tv->index = (tv->index + 1) & TVN_MASK;
+
+	return tv->index = (tv->index + 1) & TVN_MASK;
 }
 
 /***
@@ -322,26 +325,18 @@
  */
 static inline void __run_timers(tvec_base_t *base)
 {
-	unsigned long flags;
-
-	spin_lock_irqsave(&base->lock, flags);
+	spin_lock_irq(&base->lock);
 	while ((long)(jiffies - base->timer_jiffies) >= 0) {
 		struct list_head *head, *curr;
 
 		/*
 		 * Cascade timers:
 		 */
-		if (!base->tv1.index) {
-			cascade(base, &base->tv2);
-			if (base->tv2.index == 1) {
-				cascade(base, &base->tv3);
-				if (base->tv3.index == 1) {
-					cascade(base, &base->tv4);
-					if (base->tv4.index == 1)
-						cascade(base, &base->tv5);
-				}
-			}
-		}
+		if (!base->tv1.index &&
+			(cascade(base, &base->tv2) == 1) &&
+				(cascade(base, &base->tv3) == 1) &&
+					cascade(base, &base->tv4) == 1)
+			cascade(base, &base->tv5);
 repeat:
 		head = base->tv1.vec + base->tv1.index;
 		curr = head->next;
@@ -360,7 +355,10 @@
 			base->running_timer = timer;
 #endif
 			spin_unlock_irq(&base->lock);
-			fn(data);
+			if (!fn)
+				printk("Bad: timer %p has NULL fn. (data: %08lx)\n", timer, data);
+			else
+				fn(data);
 			spin_lock_irq(&base->lock);
 			goto repeat;
 		}
@@ -370,7 +368,7 @@
 #if CONFIG_SMP
 	base->running_timer = NULL;
 #endif
-	spin_unlock_irqrestore(&base->lock, flags);
+	spin_unlock_irq(&base->lock);
 }
 
 /******************************************************************/
#3 lore raw
Quoted 6 lines · Ingo Molnar
Quoted 3 lines · Andrew Morton

Ingo's original per-cpu-pages patch was said to be mainly beneficial for web-serving type things, but no specweb testing has been possible for a week or two due to oopses in the timer code.

Quoted 8 lines

i sent my latest timer patch to Dave Hansen but have not heard back since. I've attached the latest patch, this kernel also printks a bit more when it sees invalid timer usage.

in any case, the oops Dave was seeing i believe was fixed by Linus (the PgUp fix), and it was in the keyboard code. If there's anything else still going on then the attached patch should either fix it or provide further clues.

Sorry, I haven't had a chance to test it yet. The Specweb setup likes to eat ethernet cards and I haven't put in replacements yet. I'll try and get some time in on it tomorrow.

Signature · 2 lines
Dave Hansen
[email protected]
#6 lore raw
In reply to Andrew Morton
Quoted 1 line · Andrew Morton

url: http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.41/2.5.41-mm2/

hugetlbfs update:

CAP_IPC_LOCK is required to utilize hugetlb shm segments, memory allocation, and other facilities. The following patch does three things:

(1) check capable(CAP_IPC_LOCK) in ->f_ops->mmap
This may be redundant but it errors out with less state to clean up and at least clarifies the fact that checks are being performed at the relevant entry points.

(2) check capable(CAP_IPC_LOCK) in hugetlbfs_zero_setup() This is called at shmget() time and is an actual potential security hole. hugetlb_prefault() does not perform this check itself, so it must be done here.

fs/hugetlbfs/inode.c +6 -0
@@ -56,6 +56,9 @@
 	struct address_space *mapping = inode->i_mapping;
 	int ret;
 
+	if (!capable(CAP_IPC_LOCK))
+		return -EPERM;
+
 	if (vma->vm_start & ~HPAGE_MASK)
 		return -EINVAL;
 
@@ -259,6 +262,9 @@
 	struct qstr quick_string;
 	char buf[16];
 
+	if (!capable(CAP_IPC_LOCK))
+		return ERR_PTR(-EPERM);
+
 	n = atomic_read(&hugetlbfs_counter);
 	atomic_inc(&hugetlbfs_counter);
 
#4 lore raw
In reply to Ingo Molnar
Quoted 5 lines · Ingo Molnar
Quoted 3 lines · Andrew Morton

Ingo's original per-cpu-pages patch was said to be mainly beneficial for web-serving type things, but no specweb testing has been possible for a week or two due to oopses in the timer code.

Quoted 8 lines

i sent my latest timer patch to Dave Hansen but have not heard back since. I've attached the latest patch, this kernel also printks a bit more when it sees invalid timer usage.

in any case, the oops Dave was seeing i believe was fixed by Linus (the PgUp fix), and it was in the keyboard code. If there's anything else still going on then the attached patch should either fix it or provide further clues.

Ingo,
I'm running the current Bitkeeper tree with your patch, with no
oopses in sight. The oops happens in seconds without the patch, but I haven't seen anything in an hour of running Specweb. It looks pretty good. Please feed that patch to Linus if you haven't already.

Signature · 7 lines
Dave Hansen
[email protected]

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/
#5 lore raw
Quoted 4 lines · Dave Hansen

I'm running the current Bitkeeper tree with your patch, with no
oopses in sight. The oops happens in seconds without the patch, but I haven't seen anything in an hour of running Specweb. It looks pretty good. Please feed that patch to Linus if you haven't already.

great. Linus has applied the patch already.

Ingo

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/