Skip to contentSkip to conversation

per-process shared information

DISCUSSION Andrea Arcangeli 6 participants 35 messages Oct 13, 2004 – Oct 19, 2004

Subsystems (7)
lore
Share & tools
lore
#0 lore raw

The 2.6 kernel dropped the shared information for each task, the third field in /proc/<pid>/statem is a fake largely overstimating the number of shared pages (it gets even bigger than the rss, that's why people noticed, because the shared information was even bigger than the rss of the task, so clearly those pages couldn't be shared if they didn't exist).

Nothing important goes wrong in practice, but some statistics gets screwed (gets underflows since they're rightfully used to compute rss - shared to find the "private" not shared anonymous data).

I guess I can simply resurrect the O(N) pagetable scanning and check for page_mapcount > 1 out of order in task_mmu.c? I want to add a new file "statm_phys_shared" instead of doing it in statm, to avoid slowing down ps xav. So special apps can still get the information out of the kernel.

At first I considered providing the "shared" information in O(log(N)) but I believe it would waste quite some cpu in the fast paths (it would require walking the prio trees or the anon-vmas in each page fault, and it would need to keep track of each vma->mm that we already updated during the prio-tree walking), so I'd prefer the statistics to be slow in O(N) instead of hurting the fast paths. If I allow rescheduling and I trap signals there should be no DoS involved even in the very huge boxes.

Comments? (can you suggest a better name?)

Ps. if somebody like Hugh volunteers implementing it, you're very welcome, just let me know (I'll eventually want to work on the oom handling too, which is pretty screwed right now, I've plenty of bugs open on that area and the lowmem zone protection needs a rewrite too to be set to a sane default value no matter the pages_lows etc..).

#1 lore raw
Quoted 6 lines · Andrea Arcangeli

The 2.6 kernel dropped the shared information for each task, the third field in /proc/<pid>/statem is a fake largely overstimating the number of shared pages (it gets even bigger than the rss, that's why people noticed, because the shared information was even bigger than the rss of the task, so clearly those pages couldn't be shared if they didn't exist).

Yes, in 2.4 they were pte counts (seventh always 0), but in 2.6 all but the second field (rss) are just extents (fifth and seventh now always 0). Very wasteful to support the old, often meaningless, counts.

Quoted 3 lines

Nothing important goes wrong in practice, but some statistics gets screwed (gets underflows since they're rightfully used to compute rss - shared to find the "private" not shared anonymous data).

Is "shared" generally expected to pair with rss? Would it make sense to revert shared to a pte count, but leave size, text, and data as extents? I don't know.

Quoted 5 lines

I guess I can simply resurrect the O(N) pagetable scanning and check for page_mapcount > 1 out of order in task_mmu.c? I want to add a new file "statm_phys_shared" instead of doing it in statm, to avoid slowing down ps xav. So special apps can still get the information out of the kernel.

Quoted 8 lines

At first I considered providing the "shared" information in O(log(N)) but I believe it would waste quite some cpu in the fast paths (it would require walking the prio trees or the anon-vmas in each page fault, and it would need to keep track of each vma->mm that we already updated during the prio-tree walking), so I'd prefer the statistics to be slow in O(N) instead of hurting the fast paths. If I allow rescheduling and I trap signals there should be no DoS involved even in the very huge boxes.

Sounds horrid to me! I'm not inclined to volunteer for that: plus this is very much Bill's territory, though he might be glad to surrender it!

But how about the patch below? Really a mixture of four patches...

One, support anon_rss as a subset of rss, "shared" being (rss - anon_rss). Yes, that's a slight change in meaning of "shared" from in 2.4, but easy to support and I think very reasonable. On the one hand, yes, of course we know an anon page may actually be shared between several mms of the fork group, whereas it won't be counted in "shared" with this patch. But the old definition of "shared" was considerably more stupid, wasn't it? for example, a private page in pte and swap cache got counted as shared. Would this new meaning of "shared" suit your purposes well enough?

Two, more dubious, report this new "shared" as the third field in /proc/pid/statm, instead of the current "shared_vm". Maybe we shouldn't change that now, but add your statm_phys_shared; whatever, this patch at least allows you to see if it makes sense.

Three, I see negative "data" sometimes: since "text" is (always?) a subset of "shared_vm", we're subtracting that twice from total_vm. Fix that, just say data is total_vm-shared_vm: same as VmData+VmStk from /proc/pid/status.

Four, the __vm_stat_accounting in do_mmap_pgoff didn't work quite right in the test I happened to try, didn't balance with munmap - shared anon starts off with NULL file, then gets given a vm_file at the end, __vm_stat_account needs to use that. And a driver is likely to change vm_flags too (adding VM_IO or VM_RESERVED for example), so better pick that up. (I've also thrown in updating pgoff for the vma_merge there: noticed that in an earlier audit, though in fact all the drivers which adjust vm_pgoff happen to set one of the VM_SPECIAL flags which prevent merging.)

Hugh

fs/proc/task_mmu.c +2 -2
@@ -35,9 +35,9 @@ unsigned long task_vsize(struct mm_struc
 int task_statm(struct mm_struct *mm, int *shared, int *text,
 	       int *data, int *resident)
 {
-	*shared = mm->shared_vm;
+	*shared = mm->rss - mm->anon_rss;
 	*text = (mm->end_code - mm->start_code) >> PAGE_SHIFT;
-	*data = mm->total_vm - mm->shared_vm - *text;
+	*data = mm->total_vm - mm->shared_vm;
 	*resident = mm->rss;
 	return mm->total_vm;
 }
include/linux/sched.h +1 -1
@@ -225,7 +225,7 @@ struct mm_struct {
 	unsigned long start_code, end_code, start_data, end_data;
 	unsigned long start_brk, brk, start_stack;
 	unsigned long arg_start, arg_end, env_start, env_end;
-	unsigned long rss, total_vm, locked_vm, shared_vm;
+	unsigned long rss, anon_rss, total_vm, locked_vm, shared_vm;
 	unsigned long exec_vm, stack_vm, reserved_vm, def_flags;
 
 	unsigned long saved_auxv[42]; /* for /proc/PID/auxv */
kernel/fork.c +1 -0
@@ -297,6 +297,7 @@ static inline int dup_mmap(struct mm_str
 	mm->free_area_cache = oldmm->mmap_base;
 	mm->map_count = 0;
 	mm->rss = 0;
+	mm->anon_rss = 0;
 	cpus_clear(mm->cpu_vm_mask);
 	mm->mm_rb = RB_ROOT;
 	rb_link = &mm->mm_rb.rb_node;
mm/memory.c +7 -1
@@ -326,6 +326,8 @@ skip_copy_pte_range:
 				pte = pte_mkold(pte);
 				get_page(page);
 				dst->rss++;
+				if (PageAnon(page))
+					dst->anon_rss++;
 				set_pte(dst_pte, pte);
 				page_dup_rmap(page);
 cont_copy_pte_range_noset:
@@ -416,7 +418,9 @@ static void zap_pte_range(struct mmu_gat
 				set_pte(ptep, pgoff_to_pte(page->index));
 			if (pte_dirty(pte))
 				set_page_dirty(page);
-			if (pte_young(pte) && !PageAnon(page))
+			if (PageAnon(page))
+				tlb->mm->anon_rss--;
+			else if (pte_young(pte))
 				mark_page_accessed(page);
 			tlb->freed++;
 			page_remove_rmap(page);
@@ -1095,6 +1099,8 @@ static int do_wp_page(struct mm_struct *
 	spin_lock(&mm->page_table_lock);
 	page_table = pte_offset_map(pmd, address);
 	if (likely(pte_same(*page_table, pte))) {
+		if (PageAnon(old_page))
+			mm->anon_rss--;
 		if (PageReserved(old_page))
 			++mm->rss;
 		else
mm/mmap.c +4 -1
@@ -988,9 +988,12 @@ munmap_back:
 	 *         f_op->mmap method. -DaveM
 	 */
 	addr = vma->vm_start;
+	pgoff = vma->vm_pgoff;
+	vm_flags = vma->vm_flags;
 
 	if (!file || !vma_merge(mm, prev, addr, vma->vm_end,
 			vma->vm_flags, NULL, file, pgoff, vma_policy(vma))) {
+		file = vma->vm_file;
 		vma_link(mm, vma, prev, rb_link, rb_parent);
 		if (correct_wcount)
 			atomic_inc(&inode->i_writecount);
@@ -1005,6 +1008,7 @@ munmap_back:
 	}
 out:	
 	mm->total_vm += len >> PAGE_SHIFT;
+	__vm_stat_account(mm, vm_flags, file, len >> PAGE_SHIFT);
 	if (vm_flags & VM_LOCKED) {
 		mm->locked_vm += len >> PAGE_SHIFT;
 		make_pages_present(addr, addr + len);
@@ -1015,7 +1019,6 @@ out:	
 					pgoff, flags & MAP_NONBLOCK);
 		down_write(&mm->mmap_sem);
 	}
-	__vm_stat_account(mm, vm_flags, file, len >> PAGE_SHIFT);
 	return addr;
 
 unmap_and_free_vma:
mm/rmap.c +3 -0
@@ -431,6 +431,8 @@ void page_add_anon_rmap(struct page *pag
 	BUG_ON(PageReserved(page));
 	BUG_ON(!anon_vma);
 
+	vma->vm_mm->anon_rss++;
+
 	anon_vma = (void *) anon_vma + PAGE_MAPPING_ANON;
 	index = (address - vma->vm_start) >> PAGE_SHIFT;
 	index += vma->vm_pgoff;
@@ -578,6 +580,7 @@ static int try_to_unmap_one(struct page 
 		swap_duplicate(entry);
 		set_pte(pte, swp_entry_to_pte(entry));
 		BUG_ON(pte_file(*pte));
+		mm->anon_rss--;
 	}
 
 	mm->rss--;
#2 lore raw
Quoted 10 lines · Hugh Dickins

Sounds horrid to me! I'm not inclined to volunteer for that: plus this is very much Bill's territory, though he might be glad to surrender it! But how about the patch below? Really a mixture of four patches... One, support anon_rss as a subset of rss, "shared" being (rss - anon_rss). Yes, that's a slight change in meaning of "shared" from in 2.4, but easy to support and I think very reasonable. On the one hand, yes, of course we know an anon page may actually be shared between several mms of the fork group, whereas it won't be counted in "shared" with this patch. But the old definition of "shared" was considerably more stupid, wasn't it? for example, a private page in pte and swap cache got counted as shared.

This is all very reasonable.

-- wli

#3 lore raw
In reply to Hugh Dickins
Quoted 1 line · Hugh Dickins

Is "shared" generally expected to pair with rss? Would it make

shared is expected to work like in linux 2.4 (and apparently solaris), which means _physical_ pages mapped in more than one task.

Quoted 1 line

Sounds horrid to me! I'm not inclined to volunteer for that: plus this

what's horrid? would you add a O(log(N)) slowdown in the fast paths to provide the stat in O(1)? I much prefer an O(N) loop in the stats as far as it catches signals and reschedules as soon as need_resched is set.

if you can suggest a not-horrid approach to avoid breaking binary compatibility to 2.4 you're welcome ;)

Quoted 3 lines

One, support anon_rss as a subset of rss, "shared" being (rss - anon_rss). Yes, that's a slight change in meaning of "shared" from in 2.4, but easy to support and I think very reasonable. On the one hand, yes, of course

that's certainly much better than what we have right now, it's much closer to the old semantics, but I'm not sure if it's enough to be compliant with the other OS (including 2.4). I will ask.

I also guess the app will stop breaking since rss - shared will not wrap anymore.

Quoted 4 lines

we know an anon page may actually be shared between several mms of the fork group, whereas it won't be counted in "shared" with this patch. But the old definition of "shared" was considerably more stupid, wasn't it? for example, a private page in pte and swap cache got counted as shared.

just checking mapcount > 1 would do it right in 2.6.

Quoted 1 line

Would this new meaning of "shared" suit your purposes well enough?

It'd be fine for me, but I'm no the one how's having troubles.

Quoted 1 line

shouldn't change that now, but add your statm_phys_shared; whatever,

the only reason to add statm_phys_shared was to keep ps xav fast, if you don't slowdown pa xav you can add another field at the end of statm.

Ideally shared should have been set to 0 and it should have been moved to statm_phys_shared. It's slow but not so horrid thanks to the mapcount which makes it really strightforward (it's just a vma + pgtable walk, the only tricky bit is the signal catch and need_resched).

#4 lore raw
Quoted 1 line · Hugh Dickins
Quoted 1 line

shouldn't change that now, but add your statm_phys_shared; whatever,

Quoted 6 lines · Andrea Arcangeli

the only reason to add statm_phys_shared was to keep ps xav fast, if you don't slowdown pa xav you can add another field at the end of statm. Ideally shared should have been set to 0 and it should have been moved to statm_phys_shared. It's slow but not so horrid thanks to the mapcount which makes it really strightforward (it's just a vma + pgtable walk, the only tricky bit is the signal catch and need_resched).

I'd be rather loath to endorse reintroducing the problematic pagetable and vma walks I spent so much effort eliminating. Will Hugh's patch suffice for you? If not, what else is needed?

-- wli

#5 lore raw
In reply to Andrea Arcangeli
Quoted 5 lines · Andrea Arcangeli
Quoted 1 line · Hugh Dickins

Is "shared" generally expected to pair with rss? Would it make

shared is expected to work like in linux 2.4 (and apparently solaris), which means _physical_ pages mapped in more than one task.

Well, it didn't quite mean that in 2.4: since any pagecache (including swapcache) page mapped into a single task would have page_count 2 and so be counted as shared.

I think 2.4 was already trying to come up with a plausible simulacrum of numbers that made sense to gather in 2.2, but the numbers had lost their point, and it only had page_count to play with. Or maybe 2.2 was already trying to make up numbers to fit with 2.0...

Quoted 5 lines
Quoted 1 line

Sounds horrid to me! I'm not inclined to volunteer for that: plus this

what's horrid? would you add a O(log(N)) slowdown in the fast paths to provide the stat in O(1)? I much prefer an O(N) loop in the stats as far as it catches signals and reschedules as soon as need_resched is set.

Of course I prefer to keep significant slowdown out of the fast paths ("significant" inserted there because I don't mind the fastish path incrementing just one count in an already dirty cacheline).

But I don't want to give myself unnecessary work, and I don't want to give the cpu unnecessary work, particularly if the stats gathering is in danger of dominating some profiled load. Bill had good reason to remove even the vma walk; but I accept you're being careful to propose that we keep the overhead out of existing /proc/pid uses - right if we have to go that way, but I just prefer to avoid the work myself.

Quoted 2 lines

if you can suggest a not-horrid approach to avoid breaking binary compatibility to 2.4 you're welcome ;)

I hope that's what my patch would be sufficient to achieve.

It would be unfair to say 2.4's numbers were actually a bug, but certainly peculiar: I'm about as interested in exactly reproducing their oddities as in building a replica of some antique furniture.

Quoted 7 lines
Quoted 3 lines

One, support anon_rss as a subset of rss, "shared" being (rss - anon_rss). Yes, that's a slight change in meaning of "shared" from in 2.4, but easy to support and I think very reasonable. On the one hand, yes, of course

that's certainly much better than what we have right now, it's much closer to the old semantics, but I'm not sure if it's enough to be compliant with the other OS (including 2.4). I will ask.

Thanks, please do.

Quoted 2 lines

I also guess the app will stop breaking since rss - shared will not wrap anymore.

Oh, if that's all we need, I can do a simpler patch ;)

Quoted 6 lines
Quoted 4 lines

we know an anon page may actually be shared between several mms of the fork group, whereas it won't be counted in "shared" with this patch. But the old definition of "shared" was considerably more stupid, wasn't it? for example, a private page in pte and swap cache got counted as shared.

just checking mapcount > 1 would do it right in 2.6.

Interesting idea, and now (well, 2.6.9-mm heading to 2.6.10) we have atomic_inc_return and atomic_dec_return supported on all architectures, it should be possible to adjust an mm->shared_rss each time mapcount goes up from 1 or down to 1, as well as adjusting nr_mapped count as we do when it goes up from 0 or down to 0.

Though I think I prefer the anon_rss count in yesterday's patch, which is at least well-defined. And will usually give you numbers much closer to 2.4's than shared_rss (since, as noted above, 2.4 counted a page shared between pagetable and pagecache as shared, which mapcount 1 would not).

Quoted 3 lines
Quoted 1 line

Would this new meaning of "shared" suit your purposes well enough?

It'd be fine for me, but I'm no the one how's having troubles.

Let's wait and see how (rss - anon_rss) works out for your customer.

Quoted 4 lines
Quoted 1 line

shouldn't change that now, but add your statm_phys_shared; whatever,

the only reason to add statm_phys_shared was to keep ps xav fast, if you don't slowdown pa xav you can add another field at the end of statm.

We should ask Albert which he prefers: /proc/pid/statm "shared" field revert to an rss-like count as in 2.4, subset of "resident", while size, text and data fields remain extents; or leave that third field as in earlier 2.6 and add a shared-rss field on the end?

Hugh

#6 lore raw
Quoted 11 lines · Hugh Dickins
Quoted 4 lines · Andrea Arcangeli

On Thu, Oct 14, 2004 at 10:49:28PM +0100, Hugh Dickins wrote:

if you can suggest a not-horrid approach to avoid breaking binary compatibility to 2.4 you're welcome ;)

I hope that's what my patch would be sufficient to achieve.

It would be unfair to say 2.4's numbers were actually a bug, but certainly peculiar: I'm about as interested in exactly reproducing their oddities as in building a replica of some antique furniture.

Most other people believe that Linux should have a stable ABI so that apps don't break left and right. Even if people have source code, they lose it. Even if they don't lose it, they don't want to retest. The developers may have gone on to better things.

People actually rely on Linux to work. Didn't OSDL just add app+ABI testing? This is why.

Quoted 13 lines
Quoted 6 lines
Quoted 4 lines

we know an anon page may actually be shared between several mms of the fork group, whereas it won't be counted in "shared" with this patch. But the old definition of "shared" was considerably more stupid, wasn't it? for example, a private page in pte and swap cache got counted as shared.

just checking mapcount > 1 would do it right in 2.6.

Interesting idea, and now (well, 2.6.9-mm heading to 2.6.10) we have atomic_inc_return and atomic_dec_return supported on all architectures, it should be possible to adjust an mm->shared_rss each time mapcount goes up from 1 or down to 1, as well as adjusting nr_mapped count as we do when it goes up from 0 or down to 0.

Quoted 5 lines

Though I think I prefer the anon_rss count in yesterday's patch, which is at least well-defined. And will usually give you numbers much closer to 2.4's than shared_rss (since, as noted above, 2.4 counted a page shared between pagetable and pagecache as shared, which mapcount 1 would not).

I don't see why it is such trouble to provide the old data. If new data is useful, provide that too.

Quoted 9 lines
Quoted 4 lines
Quoted 1 line

shouldn't change that now, but add your statm_phys_shared; whatever,

the only reason to add statm_phys_shared was to keep ps xav fast, if you don't slowdown pa xav you can add another field at the end of statm.

We should ask Albert which he prefers: /proc/pid/statm "shared" field revert to an rss-like count as in 2.4, subset of "resident", while size, text and data fields remain extents; or leave that third field as in earlier 2.6 and add a shared-rss field on the end?

I display the data as a column in "top". Docomentation is much easier to deal with if it doesn't have lots of special cases for different kernel versions.

I guess I'd prefer that the fields of Linux 2.4 be restored, and that any new fields be added on the end. Note that the text and data fields are supposed to be rss-like as well. Except for the size, they're all supposed to be that way. This data was created to match what BSD provides.

If adding a new file to /proc, please pick a short name that is friendly toward tab completion. "phymem" is OK.

#7 lore raw
Quoted 10 lines · Albert Cahalan

I display the data as a column in "top". Docomentation is much easier to deal with if it doesn't have lots of special cases for different kernel versions.
I guess I'd prefer that the fields of Linux 2.4 be restored, and that any new fields be added on the end. Note that the text and data fields are supposed to be rss-like as well. Except for the size, they're all supposed to be that way. This data was created to match what BSD provides. If adding a new file to /proc, please pick a short name that is friendly toward tab completion. "phymem" is OK.

The overhead is too catastrophic to tolerate. Please work with us to find a sufficient approximation to whatever statistics you want opposed to reverting to 2.4 algorithms or ones of similar expense.

-- wli

#8 lore raw
Quoted 12 lines · William Lee Irwin III
Quoted 10 lines · Albert Cahalan

I display the data as a column in "top". Docomentation is much easier to deal with if it doesn't have lots of special cases for different kernel versions.
I guess I'd prefer that the fields of Linux 2.4 be restored, and that any new fields be added on the end. Note that the text and data fields are supposed to be rss-like as well. Except for the size, they're all supposed to be that way. This data was created to match what BSD provides. If adding a new file to /proc, please pick a short name that is friendly toward tab completion. "phymem" is OK.

Quoted 3 lines

The overhead is too catastrophic to tolerate. Please work with us to find a sufficient approximation to whatever statistics you want opposed to reverting to 2.4 algorithms or ones of similar expense.

Why not get rid of rss too then? It's overhead.

#9 lore raw
Quoted 3 lines · William Lee Irwin III
Quoted 3 lines

The overhead is too catastrophic to tolerate. Please work with us to find a sufficient approximation to whatever statistics you want opposed to reverting to 2.4 algorithms or ones of similar expense.

Quoted 1 line · Albert Cahalan

Why not get rid of rss too then? It's overhead.

If you can't distinguish catastrophic from non-catastrophic you are beyond my ability to help you.

-- wli

#10 lore raw
In reply to Albert Cahalan
Quoted 1 line · Albert Cahalan

Why not get rid of rss too then? It's overhead.

"rss" is O(1), old "shared" is O(N). With N being a few terabytes of data divided by PAGE_SIZE...

on any desktop machine you couldn't notice the difference, it only matters on the high end.

But I believe Hugh's proposal will make everyone happy. Anything that we stuff in statm must have O(1) complexity (or at least O(log(N)), otherwise ps v will nearly stop running in the big boxes.

#11 lore raw
In reply to Albert Cahalan
Quoted 1 line · Albert Cahalan

I don't see why it is such trouble to provide the old data.

I agree with you w.r.t. binary compatibility, here it's even a "source compatibility" matter, a recompile wouldn't fix it.

However I wasn't exactly advocating to keep it 100% backwards compatible in this case: somebody already broke it from 2.5.x to 2.6.9-rc, and since there was a very good reason for that, we should probably declare it broken. Here there has been a very strong technical reason to break statm, but they didn't break binary and source compatibility gratuitously like some solaris kernel developer seems to think in some blog.

the problem is that when ps xav wants to know the RSS it reads statm, so we just cannot hurt ps xav to show the "old shared" information that would be extremely slow to collect.

I was only not happy about dropping the old feature completely instead of providing it with a different new API. Now I think the solution Hugh just proposed with the anon_rss should mimic the old behaviour well enough and it's probably the right way to go, it's still not literally the same, but I doubt most people from userspace could notice the difference, and most important it provides useful information, which is the number of _physical_ pages mapped that aren't anonymous memory, this is very valuable info and it's basically the same info that people was getting from the old "shared". So I like it.

#12 lore raw
Quoted 3 lines · Andrea Arcangeli

the problem is that when ps xav wants to know the RSS it reads statm, so we just cannot hurt ps xav to show the "old shared" information that would be extremely slow to collect.

Currently, ps uses /proc/*/stat for that. The /proc/*/statm file is read to determine TRS and DRS, which are broken now. That it, unless you count "ps -o OL_m" format.

The top program uses /proc/*/statm for many more fields:

%MEM  Memory usage (RES)
VIRT  Virtual Image (kb)
SWAP  Swapped size (kb)
RES   Resident size (kb)
CODE  Code size (kb)
DATA  Data+Stack size (kb)
SHR   Shared Mem size (kb)
nDRT  Dirty Pages count
Quoted 9 lines

I was only not happy about dropping the old feature completely instead of providing it with a different new API. Now I think the solution Hugh just proposed with the anon_rss should mimic the old behaviour well enough and it's probably the right way to go, it's still not literally the same, but I doubt most people from userspace could notice the difference, and most important it provides useful information, which is the number of _physical_ pages mapped that aren't anonymous memory, this is very valuable info and it's basically the same info that people was getting from the old "shared". So I like it.

What exactly would be the difference, and when might users see it?

#13 lore raw
Quoted 4 lines · Albert Cahalan

Currently, ps uses /proc/*/stat for that. The /proc/*/statm file is read to determine TRS and DRS, which are broken now. That it, unless you count "ps -o OL_m" format.
The top program uses /proc/*/statm for many more fields:

And here I refute every last field with a description of what 2.4.x actually implemented and how its implementation renders the field gibberish.

Quoted 2 lines · Albert Cahalan

%MEM Memory usage (RES)
RES Resident size (kb)

These would be the valid pages installed into pagetable entries, which differs from RSS by racing with pagetable updates while counting up pages one-by-one, where it could otherwise just use the RSS count in the mm.

Quoted 1 line · Albert Cahalan

VIRT Virtual Image (kb)

This one isn't even close to VSZ in 2.4. This is the number of allocated non-pte_none() ptes under vmas, which has no bearing on any meaningful statistics, as not even pagetable space consumption may be inferred from it due to alignment issues.

Quoted 1 line · Albert Cahalan

SWAP Swapped size (kb)

This isn't actually calculated by 2.4, and can't be inferred from it either, as swapped pages are indistinguishable from reserved pages according to /proc/, most prominently the zero page.

Quoted 1 line · Albert Cahalan

CODE Code size (kb)

This is the subset of the miscalculated RSS under VM_EXECUTABLE vmas, hence it's a miscalculated subset of the RSS.

Quoted 1 line · Albert Cahalan

DATA Data+Stack size (kb)

This is the subset of the miscalculated RSS under VM_GROWSDOWN vmas, which is incorrect for stack-grows-upward architectures, or are entirely below 0x60000000, which is pure gibberish.

Quoted 1 line · Albert Cahalan

SHR Shared Mem size (kb)

page_count(pte_page(pte)) > 1; this is gibberish on numerous levels, which Hugh pointed out, that is, owing to the references held by pagecache, swapcache, and the like.

Quoted 1 line · Albert Cahalan

nDRT Dirty Pages count

A count of dirty ptes, not dirty pages. This is meaningless for most purposes except perhaps mmap() IO, and it's rather dubious even then. These are not the dirty physical pages, or anything else remotely meaningful to common scenarios. For instance, when userspace uses a single buffer page for numerous write(2) calls, it actually dirties large amounts of pagecache, but only dirties 1 page according to this metric. When read/write sharing occurs, there is no trace of other processes' having dirtied the page. Furthermore, ptes are dirtied in numerous cases when the page has not been modified. For instance, when the region has been mapped read/write but only a read fault was taken. So our last field is meaningless.

Thus we are left with exactly zero fields which are not gibberish in 2.4, and 2.4.x semantics have no leg left to stand upon.

-- wli

#14 lore raw
Quoted 9 lines · William Lee Irwin III
Quoted 4 lines · Albert Cahalan

Currently, ps uses /proc/*/stat for that. The /proc/*/statm file is read to determine TRS and DRS, which are broken now. That it, unless you count "ps -o OL_m" format.
The top program uses /proc/*/statm for many more fields:

And here I refute every last field with a description of what 2.4.x actually implemented and how its implementation renders the field gibberish.

...

Quoted 2 lines

Thus we are left with exactly zero fields which are not gibberish in 2.4, and 2.4.x semantics have no leg left to stand upon.

Many are reasonable.

Jim developed "top" partly with a 2.2.xx kernel. He had avoided the statm values at first, for performance, but went back to using them when he found that the numbers made more sense than the status and stat numbers did.

It is only recently that Debian stopped defaulting to the 2.2.xx kernel. This isn't ancient history anywhere beyond the linux-kernel mailing list.

Changing the columns offered may screw people up. Believe it or not, people actually use top in scripts. (everybody together now: "Eeeeeeew!")

Let /proc/*/statm be as slow as it needs to be. It'll work right on normal systems then, and the Altix users can simply configure top to display columns that don't involve the statm files.

#15 lore raw
In reply to Albert Cahalan
Quoted 1 line · Albert Cahalan

Currently, ps uses /proc/*/stat for that. The /proc/*/statm

maybe you mean /proc/pid/status, status has the RSS and most virtual info too, I doubt stat has it too (a trivial grep doesn't reveal it at least).

Anyways my ps definitely reads /proc/*/statm with the 'v' option (confirmed by strace).

open("/proc/1/stat", O_RDONLY) = 6
read(6, "1 (init) S 0 0 0 0 -1 256 207 16"..., 1023) = 199
close(6) = 0
open("/proc/1/statm", O_RDONLY) = 6
read(6, "147 17 111 111 0 36 0\n", 1023) = 22
close(6) = 0
open("/proc/1/status", O_RDONLY) = 6
read(6, "Name:\tinit\nState:\tS (sleeping)\nS"..., 1023) = 473

close(6)                                = 0
open("/proc/1/cmdline", O_RDONLY)       = 6
read(6, "init [5]\0\0\0", 2047)         = 11
close(6)                                = 0

fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(3, 5), ...}) = 0 ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x401cd000

write(1, "  PID TTY      STAT   TIME  MAJF"..., 64) = 64
write(1, "    1 ?        S      0:02     2"..., 67) = 67

stat64("/proc/2", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
open("/proc/2/stat", O_RDONLY) = 6
read(6, "2 (migration/0) S 1 0 0 0 -1 328"..., 1023) = 135
close(6) = 0

peraphs we use different procps version. I heard there is more than one version, and I know you're maintaining one but I never followed the details, you sure know better than me why the above is happening on my machine. But the point is that there are widely used apps opening statm (top as you mentioned), and those apps normally don't care about "shared" (or at least they can't underflow), and those must run on the big boxes too, and statm was basically unfixable.

Quoted 1 line

What exactly would be the difference, and when might users see it?

one difference for example is that it cannot take into account for shared anonymous pages generated by fork(). Or other corner cases would be posisble, but I believe it's a minor issue... so it's looking good. Frankly I was thinking shared as a page_mapcount > 1, while all pagecache is considered "shared", so Hugh's algorithm is a lot closer to the old shared than what I thought originally. Peraphs solaris also implements it as rss - anon_rss (I mean, they must be facing similar issues, and the report states the "shared" info is being reported with the same semantics on linux 2.4 and solaris and some other unix... so the theory Hugh's matching other unix even better than 2.4 sounds reasonable).

#16 lore raw
Quoted 6 lines · Andrea Arcangeli
Quoted 1 line · Albert Cahalan

Currently, ps uses /proc/*/stat for that. The /proc/*/statm

maybe you mean /proc/pid/status, status has the RSS and most virtual info too, I doubt stat has it too (a trivial grep doesn't reveal it at least).

It's a popular value.

stat: in bytes, between vsize and rss_rlim
statm: in pages, between size and share
status: in KiB as VmRSS

Quoted 2 lines

Anyways my ps definitely reads /proc/*/statm with the 'v' option (confirmed by strace).

Sure. That's not because of RSS. It's for TRS and DRS, which are supposed to be RSS-like values specific to text (code) and data.

The VM size of text is TSIZ, and of data is DSIZ. These numbers, while useful, are not the same thing.

Quoted 7 lines

peraphs we use different procps version. I heard there is more than one version, and I know you're maintaining one but I never followed the details, you sure know better than me why the above is happening on my machine. But the point is that there are widely used apps opening statm (top as you mentioned), and those apps normally don't care about "shared" (or at least they can't underflow), and those must run on the big boxes too, and statm was basically unfixable.

A user can configure top to display other columns if he has a box that can't handle /proc/*/statm well. The file will not be read if it is not needed. Start top, then do:

f     enters field modification screen
o     disable VIRT
q     disable RES
t     disable SHR
n     disable %MEM

enter exits field modification screen
W writes a ~/.toprc file

So, what is the problem again? :-)

Quoted 13 lines
Quoted 1 line

What exactly would be the difference, and when might users see it?

one difference for example is that it cannot take into account for shared anonymous pages generated by fork(). Or other corner cases would be posisble, but I believe it's a minor issue... so it's looking good. Frankly I was thinking shared as a page_mapcount > 1, while all pagecache is considered "shared", so Hugh's algorithm is a lot closer to the old shared than what I thought originally. Peraphs solaris also implements it as rss - anon_rss (I mean, they must be facing similar issues, and the report states the "shared" info is being reported with the same semantics on linux 2.4 and solaris and some other unix... so the theory Hugh's matching other unix even better than 2.4 sounds reasonable).

Well, as long as it makes the users happy... I don't personally care, except to say that I don't care to document all sorts of kernel-specific variations. It gets hopelessly messy.

#17 lore raw
Quoted 3 lines · Albert Cahalan

Sure. That's not because of RSS. It's for TRS and DRS, which are supposed to be RSS-like values specific to text (code) and data.

And they're not RSS-like right now if you pick them from statm, the only RSS-like variable is rss itself in 2.6 ;).

	*data = mm->total_vm - mm->shared_vm

all those are virtual, not physical. dunno about 2.4, but I doubt 2.4 would be much different, rss + shared where the only physical driven things in 2.4 IIRC. (now only rss is left, and Hugh's patch adds anon_rss back)

to me TRS and DRS have always been _virtual_ when I read the ps output, obviously since DRS tends to be orders of magnitude bigger than RSS itself ;).

Quoted 2 lines

The VM size of text is TSIZ, and of data is DSIZ. These numbers, while useful, are not the same thing.

those should come out of statm pretty nicely.

Quoted 10 lines

A user can configure top to display other columns if he has a box that can't handle /proc/*/statm well. The file will not be read if it is not needed. Start top, then do:

f     enters field modification screen
o     disable VIRT
q     disable RES
t     disable SHR
n     disable %MEM
Quoted 4 lines

enter exits field modification screen
W writes a ~/.toprc file

So, what is the problem again? :-)

that you can't get those values efficiently. Even assuming you're ok to drop shared by disabling SHR, it wouldn't help, without a kernel API change.

Quoted 3 lines

Well, as long as it makes the users happy... I don't personally care, except to say that I don't care to document all sorts of kernel-specific variations. It gets hopelessly messy.

Yep, I believe users could be happy with Hugh's rss-anon_rss variant.

#18 lore raw
Quoted 3 lines · Andrea Arcangeli

that you can't get those values efficiently. Even assuming you're ok to drop shared by disabling SHR, it wouldn't help, without a kernel API change.

Quoted 3 lines · Albert Cahalan
Quoted 3 lines

Well, as long as it makes the users happy... I don't personally care, except to say that I don't care to document all sorts of kernel-specific variations. It gets hopelessly messy.

Quoted 1 line · Andrea Arcangeli

Yep, I believe users could be happy with Hugh's rss-anon_rss variant.

I just checked in with some Oracle people and the primary concern is splitting up RSS into shared and private. Given either shared or private the other is calculable.

-- wli

#19 lore raw
Quoted 3 lines · William Lee Irwin III

I just checked in with some Oracle people and the primary concern is splitting up RSS into shared and private. Given either shared or private the other is calculable.

can you define private a bit more? Is private the page_count == 1 like 2.4? Or is "private" == anonymous? that's the only question.

In Hugh's patch private == "anonymous". With 2.4 private == "page_count == 1" (which is a subset of anonymous).

#20 lore raw
Quoted 3 lines · William Lee Irwin III
Quoted 3 lines

I just checked in with some Oracle people and the primary concern is splitting up RSS into shared and private. Given either shared or private the other is calculable.

Quoted 4 lines · Andrea Arcangeli

can you define private a bit more? Is private the page_count == 1 like 2.4? Or is "private" == anonymous? that's the only question. In Hugh's patch private == "anonymous". With 2.4 private == "page_count == 1" (which is a subset of anonymous).

Private should be "anonymous" as far as I can tell. What's actually going on is that they're trying to estimate per-process user memory footprints so that the amount of client load that should be distributed to a given box may be estimated from that. They at least used to believe (I've since debunked this) that 2.4.x reported this information. Their task (and hence our reporting) is not providing the complete information to determine per-process memory footprints for general workloads, rather it's known up-front that no fork()-based COW sharing is going on in Oracle's case, so in this case, "anonymous" very happily corresponds to "process-private". In fact, the /proc/ changes to report threads only under the directory hierarchy of some distinguished thread assists in this estimation effort.

-- wli

#21 lore raw
Quoted 3 lines · William Lee Irwin III

workloads, rather it's known up-front that no fork()-based COW sharing is going on in Oracle's case, so in this case, "anonymous" very happily corresponds to "process-private". [..]

Ok fine.

Quoted 3 lines

[..] In fact, the /proc/ changes to report
threads only under the directory hierarchy of some distinguished thread assists in this estimation effort.

do you use threads now?

#22 lore raw
Quoted 3 lines · William Lee Irwin III
Quoted 3 lines

workloads, rather it's known up-front that no fork()-based COW sharing is going on in Oracle's case, so in this case, "anonymous" very happily corresponds to "process-private". [..]

Quoted 1 line · Andrea Arcangeli

Ok fine.

Quoted 3 lines · William Lee Irwin III
Quoted 3 lines

[..] In fact, the /proc/ changes to report
threads only under the directory hierarchy of some distinguished thread assists in this estimation effort.

Quoted 1 line · Andrea Arcangeli

do you use threads now?

I believe using threads to some extent has been an option for some time, though not a commonly used one on Linux.

-- wli

#23 lore raw
Quoted 1 line · William Lee Irwin III

though not a commonly used one on Linux.

I guess this is also because it's a no-way on x86.

#24 lore raw
In reply to William Lee Irwin III
Quoted 4 lines · William Lee Irwin III

Private should be "anonymous" as far as I can tell. What's actually going on is that they're trying to estimate per-process user memory footprints so that the amount of client load that should be distributed to a given box may be estimated from that. They at least used to

Quoted 8 lines

believe (I've since debunked this) that 2.4.x reported this information. Their task (and hence our reporting) is not providing the complete information to determine per-process memory footprints for general workloads, rather it's known up-front that no fork()-based COW sharing is going on in Oracle's case, so in this case, "anonymous" very happily corresponds to "process-private". In fact, the /proc/ changes to report threads only under the directory hierarchy of some distinguished thread assists in this estimation effort.

Okay, I reached the very original source(s) of these requirements inside Oracle, and they are more than satisfied with Hugh's patch, particularly as I explained to them how it was actually more accurate than 2.4.x; they're only waiting for ports to the vendor kernel(s) now.

-- wli

#25 lore raw
Quoted 1 line · William Lee Irwin III

they're only waiting for ports to the vendor kernel(s) now.

Ok fine. But first it has to be included into mainline, then of course we'll merge it. Fixing Oracle at the expense of being incompatible with the user-ABI with future 2.6 is a no-way.

#26 lore raw
Quoted 1 line · William Lee Irwin III
Quoted 1 line

they're only waiting for ports to the vendor kernel(s) now.

Quoted 3 lines · Andrea Arcangeli

Ok fine. But first it has to be included into mainline, then of course we'll merge it. Fixing Oracle at the expense of being incompatible with the user-ABI with future 2.6 is a no-way.

I've come to expect this as a requirement.

The thing I wanted to convey most was that I got an acknowledgment from the original sources of Oracle's requirement, including the project lead for the team that maintains statistics collection kit that uses the statistics to estimate the client capacity of a system and not just whoever got the bug assigned to them inside Oracle, that Hugh's specific implementation we want to go with also satisfies the user requirements. They've even committed to runtime testing the patches to verify the patch does everything they want it to.

-- wli

#27 lore raw
Quoted 5 lines · William Lee Irwin III
Quoted 3 lines · Andrea Arcangeli

Ok fine. But first it has to be included into mainline, then of course we'll merge it. Fixing Oracle at the expense of being incompatible with the user-ABI with future 2.6 is a no-way.

Quoted 8 lines

The thing I wanted to convey most was that I got an acknowledgment from the original sources of Oracle's requirement, including the project lead for the team that maintains statistics collection kit that uses the statistics to estimate the client capacity of a system and not just whoever got the bug assigned to them inside Oracle, that Hugh's specific implementation we want to go with also satisfies the user requirements. They've even committed to runtime testing the patches to verify the patch does everything they want it to.

Andrea, Bill, great, thanks a lot for doing all the fieldwork on this.

After going through the discussions, I'm inclined to stick with the patch as is i.e. "correct the 2.6 bug" in the "shared" third field of /proc/pid/statm, rather than adding this as another field on the end.

Once 2.6.9 is out and we're open for patches again, I'll break it into the four parts originally outlined, and send to Andrew.

Hugh

#28 lore raw
In reply to Andrea Arcangeli
Quoted 20 lines · Andrea Arcangeli
Quoted 1 line · Albert Cahalan

I don't see why it is such trouble to provide the old data.

I agree with you w.r.t. binary compatibility, here it's even a "source compatibility" matter, a recompile wouldn't fix it.

However I wasn't exactly advocating to keep it 100% backwards compatible in this case: somebody already broke it from 2.5.x to 2.6.9-rc, and since there was a very good reason for that, we should probably declare it broken. Here there has been a very strong technical reason to break statm, but they didn't break binary and source compatibility gratuitously like some solaris kernel developer seems to think in some blog.

the problem is that when ps xav wants to know the RSS it reads statm, so we just cannot hurt ps xav to show the "old shared" information that would be extremely slow to collect.

Quoted 9 lines

I was only not happy about dropping the old feature completely instead of providing it with a different new API. Now I think the solution Hugh just proposed with the anon_rss should mimic the old behaviour well enough and it's probably the right way to go, it's still not literally the same, but I doubt most people from userspace could notice the difference, and most important it provides useful information, which is the number of _physical_ pages mapped that aren't anonymous memory, this is very valuable info and it's basically the same info that people was getting from the old "shared". So I like it.

I think that's clearly the right solution. Going to significant effort to produce compatible but incorrect values and/or formats is not desirable. I've seen this with users and applications, too, complaining that the new output doesn't match the old, even when the old was clearly wrong.

Signature · 3 lines
   -bill davidsen ([email protected])
"The secret to procrastination is to put things off until the
 last possible moment - but no longer"  -me
#29 lore raw
In reply to Hugh Dickins
Quoted 3 lines · Hugh Dickins

Well, it didn't quite mean that in 2.4: since any pagecache (including swapcache) page mapped into a single task would have page_count 2 and so be counted as shared.

Well, doing page_mapcount + !PageAnon should do the trick. My point is that the confusion of an anon page going in swapcache (the one you mentioned) is easily fixable.

Quoted 4 lines

I think 2.4 was already trying to come up with a plausible simulacrum of numbers that made sense to gather in 2.2, but the numbers had lost their point, and it only had page_count to play with. Or maybe 2.2 was already trying to make up numbers to fit with 2.0...

;)

Quoted 2 lines

[..] and I don't want to
give the cpu unnecessary work, [..]

this doesn't make much sense to me, then you should as well forbid the user to run main () { for(;;) }.

Of course doing a sort of for(;;) in each pid of ps xav is overkill, but on demand easily killable and reschedule friendly would be no different than allowing an user to execute main () { for(;;) }. All you can do is to renice it or use CKRM to lower its cpu availability from the scheduler, or kill -9.

Quoted 3 lines

remove even the vma walk; but I accept you're being careful to propose that we keep the overhead out of existing /proc/pid uses - right if we have to go that way, but I just prefer to avoid the work myself.

correct. I'd also prefer to avoid this work myself.

Quoted 1 line

I hope that's what my patch would be sufficient to achieve.

I hope too.

Quoted 3 lines

It would be unfair to say 2.4's numbers were actually a bug, but certainly peculiar: I'm about as interested in exactly reproducing their oddities as in building a replica of some antique furniture.

sure the swapcache bit would need fixing ;)

Quoted 1 line

Oh, if that's all we need, I can do a simpler patch ;)

yep ;) though such simpler patch would return no-sensical results, it would provide no-information at all in some case.

Quoted 5 lines

Interesting idea, and now (well, 2.6.9-mm heading to 2.6.10) we have atomic_inc_return and atomic_dec_return supported on all architectures, it should be possible to adjust an mm->shared_rss each time mapcount goes up from 1 or down to 1, as well as adjusting nr_mapped count as we do when it goes up from 0 or down to 0.

I believe your approach will work fine, and it's much closer to the 2.4 physical-driven semantics. It looks like "shared" really means not-anonymous memory, but accounted on a physical basis.

However I wouldn't mind if you want to add a new field and to provide both the "virtual" shared like 2.6 right now, along the "physical" shared miming the semantics of 2.4 (they could be both in statm since they're O(1) to collect).

#30 lore raw
Quoted 71 lines · Andrea Arcangeli
Quoted 3 lines · Hugh Dickins

Well, it didn't quite mean that in 2.4: since any pagecache (including swapcache) page mapped into a single task would have page_count 2 and so be counted as shared.

Well, doing page_mapcount + !PageAnon should do the trick. My point is that the confusion of an anon page going in swapcache (the one you mentioned) is easily fixable.

Quoted 4 lines

I think 2.4 was already trying to come up with a plausible simulacrum of numbers that made sense to gather in 2.2, but the numbers had lost their point, and it only had page_count to play with. Or maybe 2.2 was already trying to make up numbers to fit with 2.0...

;)

Quoted 2 lines

[..] and I don't want to
give the cpu unnecessary work, [..]

this doesn't make much sense to me, then you should as well forbid the user to run main () { for(;;) }.

Of course doing a sort of for(;;) in each pid of ps xav is overkill, but on demand easily killable and reschedule friendly would be no different than allowing an user to execute main () { for(;;) }. All you can do is to renice it or use CKRM to lower its cpu availability from the scheduler, or kill -9.

Quoted 3 lines

remove even the vma walk; but I accept you're being careful to propose that we keep the overhead out of existing /proc/pid uses - right if we have to go that way, but I just prefer to avoid the work myself.

correct. I'd also prefer to avoid this work myself.

Quoted 1 line

I hope that's what my patch would be sufficient to achieve.

I hope too.

Quoted 3 lines

It would be unfair to say 2.4's numbers were actually a bug, but certainly peculiar: I'm about as interested in exactly reproducing their oddities as in building a replica of some antique furniture.

sure the swapcache bit would need fixing ;)

Quoted 1 line

Oh, if that's all we need, I can do a simpler patch ;)

yep ;) though such simpler patch would return no-sensical results, it would provide no-information at all in some case.

Quoted 5 lines

Interesting idea, and now (well, 2.6.9-mm heading to 2.6.10) we have atomic_inc_return and atomic_dec_return supported on all architectures, it should be possible to adjust an mm->shared_rss each time mapcount goes up from 1 or down to 1, as well as adjusting nr_mapped count as we do when it goes up from 0 or down to 0.

Quoted 8 lines

I believe your approach will work fine, and it's much closer to the 2.4 physical-driven semantics. It looks like "shared" really means not-anonymous memory, but accounted on a physical basis.

However I wouldn't mind if you want to add a new field and to provide both the "virtual" shared like 2.6 right now, along the "physical" shared miming the semantics of 2.4 (they could be both in statm since they're O(1) to collect).

If it's cheap I'd like to see that. I don't know until I see the numbers if it will be useful information for system tuning, but many servers (web, news, mail) offer fork vs. thread operation and sometimes non-intuitive performance results, so I'd like to have more data.

Signature · 3 lines
   -bill davidsen ([email protected])
"The secret to procrastination is to put things off until the
 last possible moment - but no longer"  -me
#31 lore raw
In reply to Andrea Arcangeli

Hi Andrea!

No useful comments on the statm reporting issue.

Quoted 3 lines

Ps. if somebody like Hugh volunteers implementing it, you're very welcome, just let me know (I'll eventually want to work on the oom handling too, which is pretty screwed right now,

Yes, we've got reports of bad OOM killing behaviour (is that what you're talking about?)

One thing is the removal of "if (nr_swap_pages > 0) goto out" from oom_kill() causes problems (spurious oom kill).

We need to throttle more, on page reclaiming progress I think.

Take a look at

http://marc.theaimsgroup.com/?l=linux-mm&m=109587921204602&w=2

What else you're seeing?

Quoted 3 lines

I've plenty of bugs
open on that area and the lowmem zone protection needs a rewrite too to be set to a sane default value no matter the pages_lows etc..).

Nick has been working on that lately I think. What is the problem?

#32 lore raw
Quoted 21 lines · Marcelo Tosatti

Hi Andrea!

No useful comments on the statm reporting issue.

Quoted 3 lines

Ps. if somebody like Hugh volunteers implementing it, you're very welcome, just let me know (I'll eventually want to work on the oom handling too, which is pretty screwed right now,

Yes, we've got reports of bad OOM killing behaviour (is that what you're talking about?)

One thing is the removal of "if (nr_swap_pages > 0) goto out" from oom_kill() causes problems (spurious oom kill).

We need to throttle more, on page reclaiming progress I think.

Take a look at

http://marc.theaimsgroup.com/?l=linux-mm&m=109587921204602&w=2

Quoted 7 lines

What else you're seeing?

Quoted 3 lines

I've plenty of bugs
open on that area and the lowmem zone protection needs a rewrite too to be set to a sane default value no matter the pages_lows etc..).

Nick has been working on that lately I think. What is the problem?

things went worse with the switch from 2.6.8 to 2.6.9-rc, so that's not the nr_swap_pages > 0, likely the latest changes introduced regressions instead of fixing them.

I'm seeing both hard deadlocks and suprious oom kills, and that all makes sense, I can see the bugs, it's just I need to fix them, my plan is to forward port some code from 2.4 which works fine, objrmap will make it even better.

#33 lore raw
Quoted 28 lines · Andrea Arcangeli
Quoted 27 lines · Marcelo Tosatti

Hi Andrea!

No useful comments on the statm reporting issue.

Quoted 3 lines

Ps. if somebody like Hugh volunteers implementing it, you're very welcome, just let me know (I'll eventually want to work on the oom handling too, which is pretty screwed right now,

Yes, we've got reports of bad OOM killing behaviour (is that what you're talking about?)

One thing is the removal of "if (nr_swap_pages > 0) goto out" from oom_kill() causes problems (spurious oom kill).

We need to throttle more, on page reclaiming progress I think.

Take a look at

http://marc.theaimsgroup.com/?l=linux-mm&m=109587921204602&w=2

What else you're seeing?

Quoted 3 lines

I've plenty of bugs
open on that area and the lowmem zone protection needs a rewrite too to be set to a sane default value no matter the pages_lows etc..).

Quoted 5 lines
Quoted 1 line

Nick has been working on that lately I think. What is the problem?

things went worse with the switch from 2.6.8 to 2.6.9-rc, so that's not the nr_swap_pages > 0, likely the latest changes introduced regressions instead of fixing them.

Just FYI - removing the "nr_swap_pages > 0" fixes the problem at the URL I posted above.

But having it creates hard locks on Oracle workloads (wli removed that line) due to pinned memory.

Quoted 4 lines

I'm seeing both hard deadlocks and suprious oom kills, and that all makes sense, I can see the bugs, it's just I need to fix them, my plan is to forward port some code from 2.4 which works fine, objrmap will make it even better.

Ok, very nice!

#34 lore raw
In reply to Andrea Arcangeli
Quoted 1 line · Marcelo Tosatti
Quoted 1 line

Nick has been working on that lately I think. What is the problem?

Quoted 7 lines · Andrea Arcangeli

things went worse with the switch from 2.6.8 to 2.6.9-rc, so that's not the nr_swap_pages > 0, likely the latest changes introduced regressions instead of fixing them.
I'm seeing both hard deadlocks and suprious oom kills, and that all makes sense, I can see the bugs, it's just I need to fix them, my plan is to forward port some code from 2.4 which works fine, objrmap will make it even better.

I'm not aware of these. If you could relay some of that information to me (SuSE bugzilla numbers and similar are fine) I'd be much obliged.

-- wli