Skip to contentSkip to conversation

kmemtrace.txt question: kernel parameter(s)?

PATCH In review Randy Dunlap 4 participants 6 messages Dec 17, 2009 – Mar 28, 2010

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi,

Documentation/trace/kmemtrace.txt says:

Q: 'cat /sys/kernel/debug/kmemtrace/total_overruns' is non-zero, how do I fix this? Should I worry?
A: If it's non-zero, this affects kmemtrace's accuracy, depending on how large the number is. You can fix it by supplying a higher 'kmemtrace.subbufs=N' kernel parameter.

Where is this kernel parameter implemented and where is it documented? or should this Answer just be fixed?

thanks,
---
~Randy

#1 lore raw
Quoted 11 lines · Randy Dunlap

Hi,

Documentation/trace/kmemtrace.txt says:

Q: 'cat /sys/kernel/debug/kmemtrace/total_overruns' is non-zero, how do I fix this? Should I worry?
A: If it's non-zero, this affects kmemtrace's accuracy, depending on how large the number is. You can fix it by supplying a higher 'kmemtrace.subbufs=N' kernel parameter.

Quoted 3 lines

Where is this kernel parameter implemented and where is it documented? or should this Answer just be fixed?

The whole documentation is out-dated. The original kmemtrace was based on relay, and then it moved to ftrace, without updating the documentation.

#2 lore raw
Quoted 12 lines · On Fri, 18 Dec 2009 08:57:49 +0800 Li Zefan wrote:
Quoted 11 lines · Randy Dunlap

Hi,

Documentation/trace/kmemtrace.txt says:

Q: 'cat /sys/kernel/debug/kmemtrace/total_overruns' is non-zero, how do I fix this? Should I worry?
A: If it's non-zero, this affects kmemtrace's accuracy, depending on how large the number is. You can fix it by supplying a higher 'kmemtrace.subbufs=N' kernel parameter.

Quoted 6 lines
Quoted 3 lines

Where is this kernel parameter implemented and where is it documented? or should this Answer just be fixed?

The whole documentation is out-dated. The original kmemtrace was based on relay, and then it moved to ftrace, without updating the documentation.

Thank you. I guess that explains why kmemtraced (from kmemtrace-user.git) fails with:
Could not open() file /sys/kernel/debug/kmemtrace/cpu0: No such file or directory

Eduard, can we get Documentation/trace/kmemtrace.txt updated, please?

---
~Randy

#3 lore raw
In reply to Li Zefan
Quoted 12 lines · On Fri, 18 Dec 2009 08:57:49 +0800 Li Zefan wrote:
Quoted 11 lines · Randy Dunlap

Hi,

Documentation/trace/kmemtrace.txt says:

Q: 'cat /sys/kernel/debug/kmemtrace/total_overruns' is non-zero, how do I fix this? Should I worry?
A: If it's non-zero, this affects kmemtrace's accuracy, depending on how large the number is. You can fix it by supplying a higher 'kmemtrace.subbufs=N' kernel parameter.

Quoted 6 lines
Quoted 3 lines

Where is this kernel parameter implemented and where is it documented? or should this Answer just be fixed?

The whole documentation is out-dated. The original kmemtrace was based on relay, and then it moved to ftrace, without updating the documentation.

Sorry for the delay.

Shouldn't Documentation/trace/kmemtrace.txt just be completely deleted?

It's confusing and all relevant kmemtrace documentation is elsewhere.

---
~Randy

#4 lore raw

PATCH 1/1 Remove Documentation/trace/kmemtrace.txt

The current implementation of kmemtrace has been based on ftrace for a while. Documentation/trace/kmemtrace.txt concerns the old, relay-based implementation, so it's outdated. This removes it to avoid confusion.

---
 Documentation/trace/kmemtrace.txt |  126 -------------------------------------
 1 files changed, 0 insertions(+), 126 deletions(-)
 delete mode 100644 Documentation/trace/kmemtrace.txt
Documentation/trace/kmemtrace.txt deleted +0 -126
deleted file mode 100644
index 6308735..0000000
@@ -1,126 +0,0 @@
-			kmemtrace - Kernel Memory Tracer
-
-			  by Eduard - Gabriel Munteanu
-			     <[email protected]>
-
-I. Introduction
-===============
-
-kmemtrace helps kernel developers figure out two things:
-1) how different allocators (SLAB, SLUB etc.) perform
-2) how kernel code allocates memory and how much
-
-To do this, we trace every allocation and export information to the userspace
-through the relay interface. We export things such as the number of requested
-bytes, the number of bytes actually allocated (i.e. including internal
-fragmentation), whether this is a slab allocation or a plain kmalloc() and so
-on.
-
-The actual analysis is performed by a userspace tool (see section III for
-details on where to get it from). It logs the data exported by the kernel,
-processes it and (as of writing this) can provide the following information:
-- the total amount of memory allocated and fragmentation per call-site
-- the amount of memory allocated and fragmentation per allocation
-- total memory allocated and fragmentation in the collected dataset
-- number of cross-CPU allocation and frees (makes sense in NUMA environments)
-
-Moreover, it can potentially find inconsistent and erroneous behavior in
-kernel code, such as using slab free functions on kmalloc'ed memory or
-allocating less memory than requested (but not truly failed allocations).
-
-kmemtrace also makes provisions for tracing on some arch and analysing the
-data on another.
-
-II. Design and goals
-====================
-
-kmemtrace was designed to handle rather large amounts of data. Thus, it uses
-the relay interface to export whatever is logged to userspace, which then
-stores it. Analysis and reporting is done asynchronously, that is, after the
-data is collected and stored. By design, it allows one to log and analyse
-on different machines and different arches.
-
-As of writing this, the ABI is not considered stable, though it might not
-change much. However, no guarantees are made about compatibility yet. When
-deemed stable, the ABI should still allow easy extension while maintaining
-backward compatibility. This is described further in Documentation/ABI.
-
-Summary of design goals:
-	- allow logging and analysis to be done across different machines
-	- be fast and anticipate usage in high-load environments (*)
-	- be reasonably extensible
-	- make it possible for GNU/Linux distributions to have kmemtrace
-	included in their repositories
-
-(*) - one of the reasons Pekka Enberg's original userspace data analysis
-    tool's code was rewritten from Perl to C (although this is more than a
-    simple conversion)
-
-
-III. Quick usage guide
-======================
-
-1) Get a kernel that supports kmemtrace and build it accordingly (i.e. enable
-CONFIG_KMEMTRACE).
-
-2) Get the userspace tool and build it:
-$ git clone git://repo.or.cz/kmemtrace-user.git		# current repository
-$ cd kmemtrace-user/
-$ ./autogen.sh
-$ ./configure
-$ make
-
-3) Boot the kmemtrace-enabled kernel if you haven't, preferably in the
-'single' runlevel (so that relay buffers don't fill up easily), and run
-kmemtrace:
-# '$' does not mean user, but root here.
-$ mount -t debugfs none /sys/kernel/debug
-$ mount -t proc none /proc
-$ cd path/to/kmemtrace-user/
-$ ./kmemtraced
-Wait a bit, then stop it with CTRL+C.
-$ cat /sys/kernel/debug/kmemtrace/total_overruns	# Check if we didn't
-							# overrun, should
-							# be zero.
-$ (Optionally) [Run kmemtrace_check separately on each cpu[0-9]*.out file to
-		check its correctness]
-$ ./kmemtrace-report
-
-Now you should have a nice and short summary of how the allocator performs.
-
-IV. FAQ and known issues
-========================
-
-Q: 'cat /sys/kernel/debug/kmemtrace/total_overruns' is non-zero, how do I fix
-this? Should I worry?
-A: If it's non-zero, this affects kmemtrace's accuracy, depending on how
-large the number is. You can fix it by supplying a higher
-'kmemtrace.subbufs=N' kernel parameter.
----
-
-Q: kmemtrace_check reports errors, how do I fix this? Should I worry?
-A: This is a bug and should be reported. It can occur for a variety of
-reasons:
-	- possible bugs in relay code
-	- possible misuse of relay by kmemtrace
-	- timestamps being collected unorderly
-Or you may fix it yourself and send us a patch.
----
-
-Q: kmemtrace_report shows many errors, how do I fix this? Should I worry?
-A: This is a known issue and I'm working on it. These might be true errors
-in kernel code, which may have inconsistent behavior (e.g. allocating memory
-with kmem_cache_alloc() and freeing it with kfree()). Pekka Enberg pointed
-out this behavior may work with SLAB, but may fail with other allocators.
-
-It may also be due to lack of tracing in some unusual allocator functions.
-
-We don't want bug reports regarding this issue yet.
----
-
-V. See also
-===========
-
-Documentation/kernel-parameters.txt
-Documentation/ABI/testing/debugfs-kmemtrace
-
Signature · 1 line
1.6.4.4
Acked-by on v1 1/1
#5 lore raw

Re: [PATCH] Remove Documentation/trace/kmemtrace.txt

Quoted 5 lines · Eduard - Gabriel Munteanu

The current implementation of kmemtrace has been based on ftrace for a while. Documentation/trace/kmemtrace.txt concerns the old, relay-based implementation, so it's outdated. This removes it to avoid confusion.

Quoted 137 lines
---
 Documentation/trace/kmemtrace.txt |  126 -------------------------------------
 1 files changed, 0 insertions(+), 126 deletions(-)
 delete mode 100644 Documentation/trace/kmemtrace.txt
Documentation/trace/kmemtrace.txt deleted +0 -126
deleted file mode 100644
index 6308735..0000000
@@ -1,126 +0,0 @@
-			kmemtrace - Kernel Memory Tracer
-
-			  by Eduard - Gabriel Munteanu
-			     <[email protected]>
-
-I. Introduction
-===============
-
-kmemtrace helps kernel developers figure out two things:
-1) how different allocators (SLAB, SLUB etc.) perform
-2) how kernel code allocates memory and how much
-
-To do this, we trace every allocation and export information to the userspace
-through the relay interface. We export things such as the number of requested
-bytes, the number of bytes actually allocated (i.e. including internal
-fragmentation), whether this is a slab allocation or a plain kmalloc() and so
-on.
-
-The actual analysis is performed by a userspace tool (see section III for
-details on where to get it from). It logs the data exported by the kernel,
-processes it and (as of writing this) can provide the following information:
-- the total amount of memory allocated and fragmentation per call-site
-- the amount of memory allocated and fragmentation per allocation
-- total memory allocated and fragmentation in the collected dataset
-- number of cross-CPU allocation and frees (makes sense in NUMA environments)
-
-Moreover, it can potentially find inconsistent and erroneous behavior in
-kernel code, such as using slab free functions on kmalloc'ed memory or
-allocating less memory than requested (but not truly failed allocations).
-
-kmemtrace also makes provisions for tracing on some arch and analysing the
-data on another.
-
-II. Design and goals
-====================
-
-kmemtrace was designed to handle rather large amounts of data. Thus, it uses
-the relay interface to export whatever is logged to userspace, which then
-stores it. Analysis and reporting is done asynchronously, that is, after the
-data is collected and stored. By design, it allows one to log and analyse
-on different machines and different arches.
-
-As of writing this, the ABI is not considered stable, though it might not
-change much. However, no guarantees are made about compatibility yet. When
-deemed stable, the ABI should still allow easy extension while maintaining
-backward compatibility. This is described further in Documentation/ABI.
-
-Summary of design goals:
-	- allow logging and analysis to be done across different machines
-	- be fast and anticipate usage in high-load environments (*)
-	- be reasonably extensible
-	- make it possible for GNU/Linux distributions to have kmemtrace
-	included in their repositories
-
-(*) - one of the reasons Pekka Enberg's original userspace data analysis
-    tool's code was rewritten from Perl to C (although this is more than a
-    simple conversion)
-
-
-III. Quick usage guide
-======================
-
-1) Get a kernel that supports kmemtrace and build it accordingly (i.e. enable
-CONFIG_KMEMTRACE).
-
-2) Get the userspace tool and build it:
-$ git clone git://repo.or.cz/kmemtrace-user.git		# current repository
-$ cd kmemtrace-user/
-$ ./autogen.sh
-$ ./configure
-$ make
-
-3) Boot the kmemtrace-enabled kernel if you haven't, preferably in the
-'single' runlevel (so that relay buffers don't fill up easily), and run
-kmemtrace:
-# '$' does not mean user, but root here.
-$ mount -t debugfs none /sys/kernel/debug
-$ mount -t proc none /proc
-$ cd path/to/kmemtrace-user/
-$ ./kmemtraced
-Wait a bit, then stop it with CTRL+C.
-$ cat /sys/kernel/debug/kmemtrace/total_overruns	# Check if we didn't
-							# overrun, should
-							# be zero.
-$ (Optionally) [Run kmemtrace_check separately on each cpu[0-9]*.out file to
-		check its correctness]
-$ ./kmemtrace-report
-
-Now you should have a nice and short summary of how the allocator performs.
-
-IV. FAQ and known issues
-========================
-
-Q: 'cat /sys/kernel/debug/kmemtrace/total_overruns' is non-zero, how do I fix
-this? Should I worry?
-A: If it's non-zero, this affects kmemtrace's accuracy, depending on how
-large the number is. You can fix it by supplying a higher
-'kmemtrace.subbufs=N' kernel parameter.
----
-
-Q: kmemtrace_check reports errors, how do I fix this? Should I worry?
-A: This is a bug and should be reported. It can occur for a variety of
-reasons:
-	- possible bugs in relay code
-	- possible misuse of relay by kmemtrace
-	- timestamps being collected unorderly
-Or you may fix it yourself and send us a patch.
----
-
-Q: kmemtrace_report shows many errors, how do I fix this? Should I worry?
-A: This is a known issue and I'm working on it. These might be true errors
-in kernel code, which may have inconsistent behavior (e.g. allocating memory
-with kmem_cache_alloc() and freeing it with kfree()). Pekka Enberg pointed
-out this behavior may work with SLAB, but may fail with other allocators.
-
-It may also be due to lack of tracing in some unusual allocator functions.
-
-We don't want bug reports regarding this issue yet.
----
-
-V. See also
-===========
-
-Documentation/kernel-parameters.txt
-Documentation/ABI/testing/debugfs-kmemtrace
-