add vmalloc stats to meminfo

PATCH In review Dave Hansen 6 participants 9 messages Sep 15, 2002 – Sep 16, 2002

Subsystems (2)

Memory management Proc filesystem

lore

Share & tools

lore

Dave Hansen

author

2002-09-15 05:59 UTC #0 lore raw

PATCH 1/1 add vmalloc stats to meminfo

Some workloads like to eat up a lot of vmalloc space. It is often hard to tell
whether this is because the area is too small, or just too fragmented. This makes it easy to determine.

Signature · 2 lines

Dave Hansen
[email protected]

fs/proc/proc_misc.c +50 -2

diff -ur linux-2.5.34-mm4/fs/proc/proc_misc.c linux-2.5.34-mm4-vmalloc-stats/fs/proc/proc_misc.c
@@ -38,6 +38,7 @@
 #include <linux/smp_lock.h>
 #include <linux/seq_file.h>
 #include <linux/times.h>
+#include <linux/vmalloc.h>
 
 #include <asm/uaccess.h>
 #include <asm/pgtable.h>

@@ -128,6 +129,40 @@
 	return proc_calc_metrics(page, start, off, count, eof, len);
 }
 
+struct vmalloc_info {
+	unsigned long used;
+	unsigned long largest_chunk;
+};
+
+static struct vmalloc_info get_vmalloc_info(void)
+{
+	unsigned long prev_end = VMALLOC_START;
+	struct vm_struct* vma;
+	struct vmalloc_info vmi;
+	vmi.used = 0;
+
+	read_lock(&vmlist_lock);
+	
+	if(!vmlist) 
+		vmi.largest_chunk = (VMALLOC_END-VMALLOC_START);
+	else 
+		vmi.largest_chunk = 0;
+	
+	for (vma = vmlist; vma; vma = vma->next) {
+		unsigned long free_area_size = 
+			(unsigned long)vma->addr - prev_end;
+		vmi.used += vma->size;
+		if (vmi.largest_chunk < free_area_size ) 
+			vmi.largest_chunk = free_area_size;
+		prev_end = vma->size + (unsigned long)vma->addr;
+	}
+	if(VMALLOC_END-prev_end > vmi.largest_chunk)
+		vmi.largest_chunk = VMALLOC_END-prev_end;
+	
+	read_unlock(&vmlist_lock);
+	return vmi;
+}
+
 extern atomic_t vm_committed_space;
 
 static int meminfo_read_proc(char *page, char **start, off_t off,

@@ -138,6 +173,8 @@
 	struct page_state ps;
 	unsigned long inactive;
 	unsigned long active;
+	unsigned long vmtot;
+	struct vmalloc_info vmi;
 
 	get_page_state(&ps);
 	get_zone_counts(&active, &inactive);

@@ -150,6 +187,11 @@
 	si_swapinfo(&i);
 	committed = atomic_read(&vm_committed_space);
 
+	vmtot = (VMALLOC_END-VMALLOC_START)>>10;
+	vmi = get_vmalloc_info();
+	vmi.used >>= 10;
+	vmi.largest_chunk >>= 10;
+
 	/*
 	 * Tagged format, for easy grepping and expansion.
 	 */

@@ -174,7 +216,10 @@
 		"Slab:         %8lu kB\n"
 		"Committed_AS: %8u kB\n"
 		"PageTables:   %8lu kB\n"
-		"ReverseMaps:  %8lu\n",
+		"ReverseMaps:  %8lu\n"
+		"VmalTotal:    %8lu kB\n"
+		"VmalUsed:     %8lu kB\n"
+		"VmalChunk:    %8lu kB\n",
 		K(i.totalram),
 		K(i.freeram),
 		K(i.sharedram),

@@ -195,7 +240,10 @@
 		K(ps.nr_slab),
 		K(committed),
 		K(ps.nr_page_table_pages),
-		ps.nr_reverse_maps
+		ps.nr_reverse_maps,
+		vmtot,
+		vmi.used,
+		vmi.largest_chunk
 		);
 
 #ifdef CONFIG_HUGETLB_PAGE

Andrew Morton

2002-09-15 06:56 UTC #1 lore raw

Quoted 2 lines · Dave Hansen

Some workloads like to eat up a lot of vmalloc space.

Which workloads are those?

Quoted 3 lines

It is often hard to tell
whether this is because the area is too small, or just too fragmented. This makes it easy to determine.

I do not recall ever having seen any bug/problem reports which this patch would have helped to solve. Could you explain in more detai why is it useful?

William Lee Irwin III

2002-09-15 07:10 UTC #2 lore raw

Quoted 3 lines · Dave Hansen

Quoted 3 lines
It is often hard to tell
whether this is because the area is too small, or just too fragmented. This makes it easy to determine.

Quoted 2 lines · Andrew Morton

I do not recall ever having seen any bug/problem reports which this patch would have helped to solve. Could you explain in more detai why is it useful?

LDT's were formerly allocated in vmallocspace. This presented difficulties with many simultaneous threaded applications. Also, given that there is zero vmallocspace OOM recovery now present in the kernel some method of monitoring this aspect of system behavior up until the point of failure is useful for detecting further problem areas (LDT's were addressed by using non-vmalloc allocations).

Also, dynamic vmalloc allocations may very well be starved by boot-time allocations on systems where much vmallocspace is required for IO memory. The failure mode of such is effectively deadlock, since they block indefinitely waiting for permanent boot-time allocations to be freed up.

Cheers,
Bill

William Lee Irwin III

2002-09-15 07:12 UTC #3 lore raw

Quoted 4 lines · William Lee Irwin III

Also, dynamic vmalloc allocations may very well be starved by boot-time allocations on systems where much vmallocspace is required for IO memory. The failure mode of such is effectively deadlock, since they block indefinitely waiting for permanent boot-time allocations to be freed up.

This is dead wrong. NFI wtf I was thinking. Ignore that one.

Bill

M. Edward (Ed) Borasky

2002-09-15 17:18 UTC #4 lore raw

In reply to William Lee Irwin III

Quoted 10 lines · William Lee Irwin III

Quoted 2 lines · Dave Hansen
Quoted 2 lines
It is often hard to tell
whether this is because the area is too small, or just too fragmented. This
LDT's were formerly allocated in vmallocspace. This presented difficulties with many simultaneous threaded applications. Also, given that there is zero vmallocspace OOM recovery now present in the kernel some method of monitoring this aspect of system behavior up until the point of failure is useful for detecting further problem areas (LDT's were addressed by using non-vmalloc allocations).

Quoted 4 lines

Also, dynamic vmalloc allocations may very well be starved by boot-time allocations on systems where much vmallocspace is required for IO memory. The failure mode of such is effectively deadlock, since they block indefinitely waiting for permanent boot-time allocations to be freed up.

Thank you!! How difficult would it be to back-port this to 2.4.18?

--
Take Your Trading to the Next Level!
M. Edward Borasky, Meta-Trading Coach

[email protected]
http://www.borasky-research.net/Meta-Trading-Coach.htm
http://groups.yahoo.com/group/meta-trading-coach

ransacked: participated in a sack race.

William Lee Irwin III

2002-09-15 17:22 UTC #5 lore raw

Quoted 4 lines · William Lee Irwin III

Quoted 4 lines
Also, dynamic vmalloc allocations may very well be starved by boot-time allocations on systems where much vmallocspace is required for IO memory. The failure mode of such is effectively deadlock, since they block indefinitely waiting for permanent boot-time allocations to be freed up.

Quoted 1 line · M. Edward (Ed) Borasky

Thank you!! How difficult would it be to back-port this to 2.4.18?

Note the follow-up to this saying that the above paragraph was incorrect. It doesn't sleep except to allocate backing memmory.

Bill

M. Edward Borasky

2002-09-15 17:39 UTC #6 lore raw

I'd still like a backport to the (Red Hat 7.3) 2.4.18 kernel if it is possible. I'm a big fan of statistics and logging them.

M. Edward (Ed) Borasky
mailto: [email protected]
http://www.pdxneurosemantics.com
http://www.meta-trading-coach.com
http://www.borasky-research.net

Coaching: It's Not Just for Athletes and Executives Any More!

-----Original Message-----
From: William Lee Irwin III [mailto:[email protected]]
Sent: Sunday, September 15, 2002 10:26 AM
To: M. Edward (Ed) Borasky
Cc: Andrew Morton; Dave Hansen; Martin J. Bligh; [email protected]; [email protected]
Subject: Re: [PATCH] add vmalloc stats to meminfo

Quoted 4 lines · William Lee Irwin III

Quoted 4 lines
Also, dynamic vmalloc allocations may very well be starved by boot-time allocations on systems where much vmallocspace is required for IO memory. The failure mode of such is effectively deadlock, since they block indefinitely waiting for permanent boot-time allocations to be freed up.

Quoted 1 line · M. Edward (Ed) Borasky

Thank you!! How difficult would it be to back-port this to 2.4.18?

Note the follow-up to this saying that the above paragraph was incorrect. It doesn't sleep except to allocate backing memmory.

Bill

Martin J. Bligh

2002-09-15 14:47 UTC #7 lore raw

In reply to Andrew Morton

Quoted 10 lines

Quoted 1 line
Some workloads like to eat up a lot of vmalloc space.
Which workloads are those?
Quoted 3 lines
It is often hard to tell
whether this is because the area is too small, or just too fragmented. This makes it easy to determine.
I do not recall ever having seen any bug/problem reports which this patch would have helped to solve. Could you explain in more detai why is it useful?

Seen on specweb - doubling the size of the vmalloc space made it go away, but without any counters, it was all really just guesswork.

I am also going to implement per-node slabcache on top of vmalloc, as slabs have to be in permanently mapped KVA, but all ZONE_NORMAL is on node 0. That's going to put a lot of pressure on the vmalloc area.

Matt Porter

2002-09-16 05:08 UTC #8 lore raw

In reply to Dave Hansen

Quoted 3 lines · Dave Hansen

Some workloads like to eat up a lot of vmalloc space. It is often hard to tell whether this is because the area is too small, or just too fragmented. This makes it easy to determine.

Great, I was going to do something nearly the same to help out with debugging high-end embedded applications. It is quite common for us to see multiple PCI masters with PCI memory windows in sizes ranging from 256MB-1GB that are being ioremapped and consuming vmalloc space (along with all the other consumers). I'd love to see this in the kernel since it would make it much easier to debug some folks' custom board ports when they show symptoms of running out of vmalloc space (i.e. modules not loading).

Regards,

Signature · 3 lines

Matt Porter
[email protected]
This is Linux Country. On a quiet night, you can hear Windows reboot.