Kernel memory leak detector 0.10
Subsystems (16)
cover Kernel memory leak detector 0.10
This is a new version (0.10) of the kernel memory leak detector. See the Documentation/kmemleak.txt file for a more detailed description. The patches are downloadable from (the whole patch or the broken-out series):
http://homepage.ntlworld.com/cmarinas/kmemleak/patch-2.6.18-rc6-kmemleak-0.10.bz2
http://homepage.ntlworld.com/cmarinas/kmemleak/broken-out/patches-kmemleak-0.10.tar.bz2
What's new in this version:
- replaced the pointers radix tree with a hash table to avoid the locking dependencies caused by the radix tree memory allocations
- fixed locking dependency problems by no longer holding the memleak_lock when allocating/freeing memory from kmemleak and also using RCU
- changed the naming of tracked memory blocks from "pointer" to "object"
- code clean-up
To do:
- testing on a wider range of platforms and configurations
- support for ioremap tracking (once the generic ioremap patches are merged)
- eliminate the task stacks scanning (if possible, by marking the allocated blocks as temporary until the return to user-space - Ingo's suggestion)
- precise type identification (after first assessing the efficiency of the current method as it requires changes to the kernel API)
Signature · 1 line
Catalin
PATCH 1/10 Base support for kmemleak
From: Catalin Marinas <[email protected]>
This patch adds the base support for the kernel memory leak detector. It traces the memory allocation/freeing in a way similar to the Boehm's conservative garbage collector, the difference being that the orphan pointers are not freed but only shown in /proc/memleak. Enabling this feature would introduce an overhead to memory allocations.
---
include/linux/kernel.h | 7 include/linux/memleak.h | 111 +++ init/main.c | 3 lib/Kconfig.debug | 90 +++ mm/Makefile | 2 mm/memleak.c | 1518 +++++++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 1728 insertions(+), 3 deletions(-)
include/linux/kernel.h +6 -1
@@ -13,6 +13,7 @@ #include <linux/stddef.h>#include <linux/types.h> #include <linux/compiler.h> #include <linux/bitops.h> +#include <linux/memleak.h> #include <asm/byteorder.h> #include <asm/bug.h>
@@ -292,9 +293,13 @@ #define max_t(type,x,y) \* @member: the name of the member within the struct. * */ -#define container_of (ptr, type, member) ({ \ +#define __container_of (ptr, type, member) ({ \ const typeof( ((type *)0)->member ) *__mptr = (ptr); \ (type *)( (char *)__mptr - offsetof(type,member) );}) +#define container_of (ptr, type, member) ({ \ + DECLARE_MEMLEAK_OFFSET(container_of, type, member); \ + __container_of(ptr, type, member); \ +})/* * Check at compile time that something is of a particular type.
include/linux/memleak.h new file +111 -0
111 lines not on this page · raw message
init/main.c +2 -1
@@ -548,6 +548,8 @@ #endifcpuset_init_early ();mem_init ();kmem_cache_init (); +radix_tree_init (); +memleak_init ();setup_per_cpu_pageset ();numa_policy_init ();if (late_time_init)
@@ -568,7 +570,6 @@ #endifkey_init ();security_init ();vfs_caches_init (num_physpages); -radix_tree_init ();signals_init ();/* rootfs populating might need page-writeback */ page_writeback_init ();
lib/Kconfig.debug +90 -0
96 lines not on this page · raw message
mm/Makefile +1 -1
@@ -23,4 +23,4 @@ obj-$(CONFIG_SLAB) += slab.o obj-$(CONFIG_MEMORY_HOTPLUG) += memory_hotplug.o obj-$(CONFIG_FS_XIP) += filemap_xip.o obj-$(CONFIG_MIGRATION) += migrate.o - +obj-$(CONFIG_DEBUG_MEMLEAK) += memleak.o
mm/memleak.c new file +1518 -0
1,518 lines not on this page · raw message
PATCH 2/10 Some documentation for kmemleak
In reply to Catalin MarinasFrom: Catalin Marinas <[email protected]>
---
Documentation/kmemleak.txt | 157 ++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 157 insertions(+), 0 deletions(-)
Documentation/kmemleak.txt new file +157 -0
@@ -0,0 +1,157 @@ +Kernel Memory Leak Detector +=========================== + + +Introduction +------------ + +Kmemleak provides a way of detecting possible kernel memory leaks in a +way similar to a tracing garbage collector +(http://en.wikipedia.org/wiki/Garbage_collection_%28computer_science%29#Tracing_garbage_collectors), +with the difference that the orphan pointers are not freed but only +reported via /sys/kernel/debug/memleak. A similar method is used by +the Valgrind tool (memcheck --leak-check) to detect the memory leaks +in user-space applications. + + +Usage +----- + +CONFIG_DEBUG_MEMLEAK has to be enabled. For additional config options, +look in: + + -> Kernel hacking + -> Kernel debugging + -> Debug slab memory allocations + -> Kernel memory leak detector + +To display the possible memory leaks: + + # mount -t debugfs nodev /sys/kernel/debug/ + # cat /sys/kernel/debug/memleak + +In order to reduce the run-time overhead, memory scanning is only +performed when reading the /sys/kernel/debug/memleak file. + + +Basic Algorithm +--------------- + +The memory allocations via kmalloc, vmalloc, kmem_cache_alloc and +friends are tracked and the pointers, together with additional +information like size and stack trace, are stored in a hash table. The +corresponding freeing function calls are tracked and the pointers +removed from the hash table. + +An allocated block of memory is considered orphan if a pointer to its +start address or to an alias (pointer aliases are explained later) +cannot be found by scanning the memory (including saved +registers). This means that there might be no way for the kernel to +pass the address of the allocated block to a freeing function and +therefore the block is considered a leak. + +The scanning algorithm steps: + + 1. mark all pointers as white (remaining white pointers will later + be considered orphan) + 2. scan the memory starting with the data section and stacks, + checking the values against the addresses stored in the hash + table. If a white pointer is found, it is added to the grey list + 3. scan the grey pointers for matching addresses (some white + pointers can become grey and added at the end of the grey list) + until the grey set is finished + 4. the remaining white pointers are considered orphan and reported + via /sys/kernel/debug/memleak + + +Improvements +------------ + +Because the Linux kernel calculates many pointers at run-time via the +container_of macro (see the lists implementation), a lot of false +positives would be reported. This tool re-writes the container_of +macro so that the offset and type information is stored in the +.init.memleak_offsets section. The memleak_init() function creates a +radix tree with corresponding offsets for every encountered block +type. The memory allocations hook stores the pointer address together +with its aliases based on the type of the allocated block. + +While one level of offsets should be enough for most cases, a second +level, i.e. container_of(container_of(...)), can be enabled via the +configuration options (one false positive is the "struct socket_alloc" +allocation in the sock_alloc_inode() function). + +Some allocated memory blocks have pointers stored in the kernel's +internal data structures and they cannot be detected as orphans. To +avoid this, kmemleak can also store the number of values equal to the +pointer (or aliases) that need to be found so that the block is not +considered a leak. One example is __vmalloc(). + + +Limitations and Drawbacks +------------------------- + +The biggest drawback is the reduced performance of memory allocation +and freeing. To avoid other penalties, the memory scanning is only +performed when the /sys/kernel/debug/memleak file is read. Anyway, +this tool is intended for debugging purposes where the performance +might not be the most important requirement. + +Kmemleak currently approximates the type id using the sizeof() +compiler built-in function. This is not accurate and can lead to false +negatives. The aim is to gradually change the kernel and kmemleak to +do more precise type identification. + +Another source of false negatives is the data stored in non-pointer +values. Together with the more precise type identification, kmemleak +could only scan the pointer members in the allocated structures. + +The tool can report false positives. These are cases where an +allocated block doesn't need to be freed (some cases in the init_call +functions), the pointer is calculated by other methods than the +container_of macro or the pointer is stored in a location not scanned +by kmemleak. If the "member" argument in the offsetof(type, member) +call is not constant, kmemleak considers the offset as zero since it +cannot be determined at compilation time. + +Page allocations and ioremap are not tracked. Only the ARM and i386 +architectures are currently supported. + + +Kmemleak API +------------ + +See the include/linux/memleak.h header for the functions prototype. + +memleak_init - initialize kmemleak +memleak_alloc - notify of a memory block allocation +memleak_free - notify of a memory block freeing +memleak_padding - mark the boundaries of the data inside the block +memleak_not_leak - mark a pointer as not a leak +memleak_ignore - do not scan or report a pointer as leak +memleak_scan_area - add scan areas inside a memory block +memleak_insert_aliases - add aliases for a given type +memleak_erase - erase an old value in a pointer variable +memleak_typeid_raw - set the typeid for an allocated block +memleak_container - statically declare a pointer alias +memleak_typeid - set the typeid for an allocated block (takes + a type rather than typeid as argument) + + +Dealing with false positives/negatives +-------------------------------------- + +To reduce the false negatives, kmemleak provides the memleak_ignore, +memleak_scan_area and memleak_erase functions. The task stacks also +increase the amount of false negatives and their scanning is not +enabled by default. + +To eliminate the false positives caused by code allocating a different +size from the object one (either for alignment or for extra memory +after the end of the structure), kmemleak provides the memleak_padding +and memleak_typeid functions. + +For pointers known not to be leaks, kmemleak provides the +memleak_not_leak function. The memleak_ignore could also be used if +the memory block is known not to contain other pointers as it will no +longer be scanned.
PATCH 3/10 Add the memory allocation/freeing hooks for kmemleak
In reply to Catalin MarinasFrom: Catalin Marinas <[email protected]>
This patch adds the callbacks to memleak_(alloc|free) functions from kmalloc/kfree, kmem_cache_(alloc|free), vmalloc/vfree etc.
---
include/linux/slab.h | 6 ++++++ mm/page_alloc.c | 2 ++ mm/slab.c | 19 +++++++++++++++++-- mm/vmalloc.c | 22 ++++++++++++++++++++-- 4 files changed, 45 insertions(+), 4 deletions(-)
include/linux/slab.h +6 -0
@@ -133,6 +133,8 @@ #endif*/ static inline void *kmalloc (size_t size,gfp_t flags) { +#ifndef CONFIG_DEBUG_MEMLEAK +/* this block removes the size information needed by kmemleak */ if (__builtin_constant_p (size)) {int i =0 ;#define CACHE (x) \
@@ -151,6 +153,7 @@ found: malloc_sizes[i].cs_dmacachep : malloc_sizes[i].cs_cachep, flags); } +#endif return __kmalloc (size, flags); }
@@ -163,6 +166,8 @@ extern void *__kzalloc(size_t, gfp_t);*/ static inline void *kzalloc (size_t size,gfp_t flags) { +#ifndef CONFIG_DEBUG_MEMLEAK +/* this block removes the size information needed by kmemleak */ if (__builtin_constant_p (size)) {int i =0 ;#define CACHE (x) \
@@ -181,6 +186,7 @@ found: malloc_sizes[i].cs_dmacachep : malloc_sizes[i].cs_cachep, flags); } +#endif return __kzalloc (size, flags); }
mm/page_alloc.c +2 -0
@@ -2450,6 +2450,8 @@ void *__init alloc_large_system_hash(conif (_hash_mask) *_hash_mask = (1 << log2qty) -1 ; +memleak_alloc (table, size,1 ); +return table; }
mm/slab.c +17 -2
@@ -2446,6 +2446,9 @@ static struct slab *alloc_slabmgmt(struc/* Slab management obj is off-slab. */ slabp =kmem_cache_alloc_node (cachep->slabp_cache, local_flags, nodeid); +/* only scan the list member to avoid false negatives */ +memleak_scan_area (slabp, offsetof(struct slab , list), +sizeof (struct list_head ));if (!slabp)return NULL ; }else {
@@ -2986,6 +2989,8 @@ #endifSTATS_INC_ALLOCMISS (cachep); objp =cache_alloc_refill (cachep, flags); } +/* avoid false negatives */ +memleak_erase (&ac->entry[ac->avail]);return objp; }
@@ -3002,6 +3007,7 @@ static __always_inline void *__cache_alllocal_irq_restore (save_flags); objp =cache_alloc_debugcheck_after (cachep, flags, objp, caller); +memleak_alloc (objp,obj_size (cachep),1 );prefetchw (objp);return objp; }
@@ -3193,6 +3199,7 @@ static inline void __cache_free(struct kstruct array_cache *ac =cpu_cache_get (cachep);check_irq_off (); +memleak_free (objp); objp =cache_free_debugcheck (cachep, objp,__builtin_return_address (0 ));if (cache_free_alien (cachep, objp))
@@ -3312,6 +3319,7 @@ void *kmem_cache_alloc_node(struct kmem_ ptr =cache_alloc_debugcheck_after (cachep, flags, ptr,__builtin_return_address (0 )); +memleak_alloc (ptr,obj_size (cachep),1 );return ptr; }
@@ -3320,11 +3328,14 @@ EXPORT_SYMBOL(kmem_cache_alloc_node);void *kmalloc_node (size_t size,gfp_t flags,int node) {struct kmem_cache *cachep; +void *ptr; cachep =kmem_find_general_cachep (size, flags);if (unlikely (cachep ==NULL ))return NULL ; -return kmem_cache_alloc_node (cachep, flags, node); + ptr =kmem_cache_alloc_node (cachep, flags, node); +memleak_padding (ptr,0 , size); +return ptr; }EXPORT_SYMBOL (kmalloc_node);#endif
@@ -3339,6 +3350,7 @@ static __always_inline void *__do_kmallo void *caller) {struct kmem_cache *cachep; +void *ptr;/* If you want to save a few bytes .text space: replace * __ with kmem_.
@@ -3348,7 +3360,10 @@ static __always_inline void *__do_kmallo cachep =__find_general_cachep (size, flags);if (unlikely (cachep ==NULL ))return NULL ; -return __cache_alloc (cachep, flags, caller); + ptr =__cache_alloc (cachep, flags, caller); +memleak_padding (ptr,0 , size); + +return ptr; }
mm/vmalloc.c +20 -2
@@ -364,6 +364,9 @@ void __vunmap(void *addr, int deallocatevoid vfree (void *addr) {BUG_ON (in_interrupt ()); + +memleak_free (addr); +__vunmap (addr,1 ); }EXPORT_SYMBOL (vfree);
@@ -463,7 +466,14 @@ fail:void *__vmalloc_area (struct vm_struct *area,gfp_t gfp_mask,pgprot_t prot) { -return __vmalloc_area_node (area, gfp_mask, prot,-1 ); +void *addr =__vmalloc_area_node (area, gfp_mask, prot,-1 ); + +/* this needs ref_count = 2 since vm_struct also contains a +* pointer to this address. The guard page is also subtracted +* from the size */ +memleak_alloc (addr, area->size -PAGE_SIZE ,2 ); + +return addr; }/**
@@ -482,6 +492,8 @@ void *__vmalloc_node(unsigned long size, int node) {struct vm_struct *area; +void *addr; +unsigned long real_size = size; size =PAGE_ALIGN (size);if (!size || (size >>PAGE_SHIFT ) > num_physpages)
@@ -491,7 +503,13 @@ void *__vmalloc_node(unsigned long size,if (!area)return NULL ; -return __vmalloc_area_node (area, gfp_mask, prot, node); + addr =__vmalloc_area_node (area, gfp_mask, prot, node); + +/* this needs ref_count = 2 since the vm_struct also contains +a pointer to this address */ +memleak_alloc (addr, real_size,2 ); + +return addr; }EXPORT_SYMBOL (__vmalloc_node);
PATCH 4/10 Modules support for kmemleak
In reply to Catalin MarinasFrom: Catalin Marinas <[email protected]>
This patch handles the kmemleak operations needed for modules loading so that memory allocations from inside a module are properly tracked.
---
kernel/module.c | 41 +++++++++++++++++++++++++++++++++++++++++ 1 files changed, 41 insertions(+), 0 deletions(-)
kernel/module.c +41 -0
@@ -1481,6 +1481,11 @@ static struct module *load_module(void _unsigned int unusedcrcindex;unsigned int unusedgplindex;unsigned int unusedgplcrcindex; +#ifdef CONFIG_DEBUG_MEMLEAK +unsigned int dataindex; +unsigned int bssindex; +unsigned int mloffindex; +#endif struct module *mod;long err =0 ;void *percpu =NULL , *ptr =NULL ;/* Stops spurious gcc warning */
@@ -1577,6 +1582,11 @@ #endif#ifdef ARCH_UNWIND_SECTION_NAME unwindex =find_sec (hdr, sechdrs, secstrings,ARCH_UNWIND_SECTION_NAME );#endif +#ifdef CONFIG_DEBUG_MEMLEAK + dataindex =find_sec (hdr, sechdrs, secstrings,".data" ); + bssindex =find_sec (hdr, sechdrs, secstrings,".bss" ); + mloffindex =find_sec (hdr, sechdrs, secstrings,".init.memleak_offsets" ); +#endif /* Don't keep modinfo section */ sechdrs[infoindex].sh_flags &= ~(unsigned long )SHF_ALLOC ;
@@ -1646,6 +1656,10 @@ #endif/* Do the allocs. */ ptr =module_alloc (mod->core_size); +/* the pointer to this block is stored in the module structure +* which is inside the block. Just mark it as not being a +* leak */ +memleak_not_leak (ptr);if (!ptr) { err = -ENOMEM ; goto free_percpu;
@@ -1654,6 +1668,11 @@ #endif mod->module_core = ptr; ptr =module_alloc (mod->init_size); +/* the pointer to this block is stored in the module structure +* which is inside the block. This block doesn't need to be +* scanned as it contains data and code that will be freed +* after the module is initialized */ +memleak_ignore (ptr);if (!ptr && mod->init_size) { err = -ENOMEM ; goto free_core;
@@ -1685,6 +1704,28 @@ #endif/* Module has been moved. */ mod = (void *)sechdrs[modindex].sh_addr; +#ifdef CONFIG_DEBUG_MEMLEAK +if (mloffindex) +memleak_insert_aliases ((void *)sechdrs[mloffindex].sh_addr, + (void *)sechdrs[mloffindex].sh_addr + + sechdrs[mloffindex].sh_size); + +/* only scan the sections containing data */ +memleak_scan_area (mod->module_core, + (unsigned long )mod - (unsigned long )mod->module_core, +sizeof (struct module )); +if (dataindex) +memleak_scan_area (mod->module_core, + sechdrs[dataindex].sh_addr + - (unsigned long )mod->module_core, + sechdrs[dataindex].sh_size); +if (bssindex) +memleak_scan_area (mod->module_core, + sechdrs[bssindex].sh_addr + - (unsigned long )mod->module_core, + sechdrs[bssindex].sh_size); +#endif +/* Now we've moved module, initialize linked lists, etc. */ module_unload_init (mod);
PATCH 5/10 Add kmemleak support for i386
In reply to Catalin MarinasFrom: Catalin Marinas <[email protected]>
This patch modifies the vmlinux.lds.S script and adds the backtrace support for i386 to be used with kmemleak.
---
arch/i386/kernel/vmlinux.lds.S | 4 ++++ include/asm-i386/processor.h | 12 ++++++++++++ include/asm-i386/thread_info.h | 10 +++++++++- 3 files changed, 25 insertions(+), 1 deletions(-)
arch/i386/kernel/vmlinux.lds.S +4 -0
@@ -45,6 +45,7 @@ SECTIONS __tracedata_end = .; /* writeable */ + _sdata = .; /* Start of data section */ .data : AT(ADDR(.data) - LOAD_OFFSET) { /* Data */ *(.data) CONSTRUCTORS
@@ -156,6 +157,9 @@ #endif __per_cpu_start = .; .data.percpu : AT(ADDR(.data.percpu) - LOAD_OFFSET) { *(.data.percpu) } __per_cpu_end = .; + __memleak_offsets_start = .; + .init.memleak_offsets : AT(ADDR(.init.memleak_offsets) - LOAD_OFFSET) { *(.init.memleak_offsets) } + __memleak_offsets_end = .; . = ALIGN(4096); __init_end = .; /* freed after init ends here */
include/asm-i386/processor.h +12 -0
@@ -731,4 +731,16 @@ extern unsigned long boot_option_idle_ovextern void enable_sep_cpu (void );extern int sysenter_setup (void ); +#ifdef CONFIG_FRAME_POINTER +static inline unsigned long arch_call_address (void *frame) +{ +return *(unsigned long *)(frame +4 ); +} + +static inline void *arch_prev_frame (void *frame) +{ +return *(void **)frame; +} +#endif +#endif /* __ASM_I386_PROCESSOR_H */
include/asm-i386/thread_info.h +9 -1
@@ -100,12 +100,20 @@ #define alloc_thread_info(tsk) \struct thread_info *ret; \ \ ret =kmalloc (THREAD_SIZE ,GFP_KERNEL ); \ +memleak_ignore (ret); \if (ret) \memset (ret,0 ,THREAD_SIZE ); \ ret; \ })#else -#define alloc_thread_info (tsk) kmalloc(THREAD_SIZE, GFP_KERNEL) +#define alloc_thread_info (tsk) \ + ({ \ + struct thread_info *ret; \ + \ + ret = kmalloc(THREAD_SIZE, GFP_KERNEL); \ + memleak_ignore(ret); \ + ret; \ + })#endif #define free_thread_info (info) kfree(info)
PATCH 6/10 Add kmemleak support for ARM
In reply to Catalin MarinasFrom: Catalin Marinas <[email protected]>
This patch modifies the vmlinux.lds.S script and adds the backtrace support for ARM to be used with kmemleak.
---
arch/arm/kernel/vmlinux.lds.S | 7 +++++++ include/asm-arm/processor.h | 12 ++++++++++++ 2 files changed, 19 insertions(+), 0 deletions(-)
arch/arm/kernel/vmlinux.lds.S +7 -0
@@ -67,6 +67,11 @@ #endif __per_cpu_start = .; *(.data.percpu) __per_cpu_end = .; +#ifdef CONFIG_DEBUG_MEMLEAK + __memleak_offsets_start = .; + *(.init.memleak_offsets) + __memleak_offsets_end = .; +#endif #ifndef CONFIG_XIP_KERNEL __init_begin = _stext; *(.init.data)
@@ -115,6 +120,7 @@ #endif .data : AT(__data_loc) { __data_start = .; /* address in memory */ + _sdata = .; /* * first, the init task union, aligned
@@ -165,6 +171,7 @@ #endif __bss_start = .; /* BSS */ *(.bss) *(COMMON) + __bss_stop = .; _end = .; } /* Stabs debugging sections. */
include/asm-arm/processor.h +12 -0
@@ -121,6 +121,18 @@ #define spin_lock_prefetch(x) do { } whi#endif +#ifdef CONFIG_FRAME_POINTER +static inline unsigned long arch_call_address (void *frame) +{ +return *(unsigned long *)(frame -4 ) -4 ; +} + +static inline void *arch_prev_frame (void *frame) +{ +return *(void **)(frame -12 ); +} +#endif +#endif #endif /* __ASM_ARM_PROCESSOR_H */
PATCH 7/10 Remove some of the kmemleak false positives
In reply to Catalin MarinasFrom: Catalin Marinas <[email protected]>
There are allocations for which the main pointer cannot be found but they are not memory leaks. This patch fixes some of them.
---
drivers/base/platform.c | 3 +++ drivers/char/vt.c | 4 ++++ drivers/hwmon/w83627hf.c | 4 ++++ drivers/scsi/hosts.c | 3 +++ drivers/video/console/fbcon.c | 3 +++ fs/ext3/dir.c | 3 +++ ipc/util.c | 6 ++++++ kernel/params.c | 8 +++++++- mm/slab.c | 4 ++++ net/core/dev.c | 6 ++++++ net/core/skbuff.c | 3 +++ net/ipv4/netfilter/ip_conntrack_core.c | 5 +++++ net/sched/sch_generic.c | 5 +++++ 13 files changed, 56 insertions(+), 1 deletions(-)
drivers/base/platform.c +3 -0
@@ -166,6 +166,9 @@ struct platform_device *platform_device_struct platform_object *pa; pa =kzalloc (sizeof (struct platform_object ) +strlen (name),GFP_KERNEL ); +/* kmemleak cannot guess the object type because the block +* size is different from the object size */ + memleak_typeid(pa, struct platform_object);if (pa) {strcpy (pa->name, name); pa->pdev.name = pa->name;
drivers/char/vt.c +4 -0
@@ -2611,6 +2611,10 @@ static int __init con_init(void)*/ for (currcons =0 ; currcons <MIN_NR_CONSOLES ; currcons++) { vc_cons[currcons].d = vc =alloc_bootmem (sizeof (struct vc_data )); +/* kmemleak does not track the memory allocated via +* alloc_bootmem() but this block contains pointers to +* other blocks allocated via kmalloc */ +memleak_alloc (vc,sizeof (struct vc_data ),1 );visual_init (vc, currcons,1 ); vc->vc_screenbuf = (unsigned short *)alloc_bootmem (vc->vc_screenbuf_size); vc->vc_kmalloced =0 ;
drivers/hwmon/w83627hf.c +4 -0
@@ -1065,6 +1065,10 @@ static int w83627hf_detect(struct i2c_ad err = -ENOMEM ; goto ERROR1; } +/* the pointer to member is stored but the code doesn't use +* container_of for access and the alias need to be +* explicitely declared here */ + memleak_container(struct w83627hf_data, client); new_client = &data->client;i2c_set_clientdata (new_client, data);
drivers/scsi/hosts.c +3 -0
@@ -297,6 +297,9 @@ struct Scsi_Host *scsi_host_alloc(struct shost =kzalloc (sizeof (struct Scsi_Host ) + privsize, gfp_mask);if (!shost)return NULL ; +/* kmemleak cannot guess the object type because the block +* size is different from the object size */ + memleak_typeid(shost, struct Scsi_Host);spin_lock_init (&shost->default_lock);scsi_assign_lock (shost, &shost->default_lock);
drivers/video/console/fbcon.c +3 -0
@@ -2479,6 +2479,9 @@ static int fbcon_set_font(struct vc_data size = h * pitch * charcount; new_data =kmalloc (FONT_EXTRA_WORDS *sizeof (int ) + size,GFP_USER ); +/* the stored pointer is different from the address of the +* allocated block because of padding */ +memleak_padding (new_data,FONT_EXTRA_WORDS *sizeof (int ), size);if (!new_data)return -ENOMEM ;
fs/ext3/dir.c +3 -0
@@ -346,6 +346,9 @@ int ext3_htree_store_dirent(struct file new_fn =kmalloc (len,GFP_KERNEL );if (!new_fn)return -ENOMEM ; +/* kmemleak cannot guess the object type because the block +* size is different from the object size */ + memleak_typeid(new_fn, struct fname);memset (new_fn,0 , len); new_fn->hash = hash; new_fn->minor_hash = minor_hash;
ipc/util.c +6 -0
@@ -388,6 +388,9 @@ void* ipc_rcu_alloc(int size)*/ if (rcu_use_vmalloc (size)) { out =vmalloc (HDRLEN_VMALLOC + size); +/* the stored pointer is different from the address of +* the allocated block because of padding */ +memleak_padding (out,HDRLEN_VMALLOC , size);if (out) { out +=HDRLEN_VMALLOC ; container_of(out, struct ipc_rcu_hdr, data)->is_vmalloc = 1;
@@ -395,6 +398,9 @@ void* ipc_rcu_alloc(int size) } }else { out =kmalloc (HDRLEN_KMALLOC + size,GFP_KERNEL ); +/* the stored pointer is different from the address of +* the allocated block because of padding */ +memleak_padding (out,HDRLEN_KMALLOC , size);if (out) { out +=HDRLEN_KMALLOC ; container_of(out, struct ipc_rcu_hdr, data)->is_vmalloc = 0;
kernel/params.c +7 -1
@@ -547,6 +547,7 @@ static void __init kernel_param_sysfs_se unsigned int name_skip) {struct module_kobject *mk; +struct module_param_attrs *mp; mk =kzalloc (sizeof (struct module_kobject ),GFP_KERNEL );BUG_ON (!mk);
@@ -556,8 +557,13 @@ static void __init kernel_param_sysfs_sekobject_set_name (&mk->kobj, name);kobject_register (&mk->kobj); + mp =param_sysfs_setup (mk, kparam, num_params, name_skip); +/* this structure is not freed but the pointer is +* lost. However, there are other pointers to its members and +* the object has to be kept */ +memleak_not_leak (mp);/* no need to keep the kobject if no parameter is exported */ -if (!param_sysfs_setup (mk, kparam, num_params, name_skip)) { +if (!mp) {kobject_unregister (&mk->kobj);kfree (mk); }
mm/slab.c +4 -0
@@ -3419,6 +3419,10 @@ void *__alloc_percpu(size_t size)memset (pdata->ptrs[i],0 , size); } +/* the code below changes the value of the returned pointer +* and kmemleak cannot find the original value during +* scanning. It is marked as not being a leak */ +memleak_not_leak (pdata);/* Catch derefs w/o wrappers */ return (void *)(~(unsigned long )pdata);
net/core/dev.c +6 -0
@@ -3204,6 +3204,12 @@ struct net_device *alloc_netdev(int size dev = (struct net_device *) (((long )p +NETDEV_ALIGN_CONST ) & ~NETDEV_ALIGN_CONST ); dev->padded = (char *)dev - (char *)p; +/* kmemleak cannot guess the object type because the block +* size is different from the object size. The stored pointer +* is also different from the address of the allocated block +* because of padding */ +memleak_padding (p, dev->padded, alloc_size - dev->padded); + memleak_typeid(p, struct net_device);if (sizeof_priv) dev->priv =netdev_priv (dev);
net/core/skbuff.c +3 -0
@@ -151,6 +151,9 @@ struct sk_buff *__alloc_skb(unsigned int/* Get the HEAD */ skb =kmem_cache_alloc (cache, gfp_mask & ~__GFP_DMA); +/* the skbuff_fclone_cache contains objects larger than +* "struct sk_buff" and kmemleak cannot guess the type */ + memleak_typeid(skb, struct sk_buff);if (!skb) goto out;
net/ipv4/netfilter/ip_conntrack_core.c +5 -0
@@ -654,6 +654,11 @@ struct ip_conntrack *ip_conntrack_alloc( } conntrack =kmem_cache_alloc (ip_conntrack_cachep,GFP_ATOMIC ); +/* tuplehash_to_ctrack doesn't pass a constant argument to +* container_of and therefore the conntrack->tuplehash[].list +* aliases are ignored */ + memleak_container(struct ip_conntrack, tuplehash[IP_CT_DIR_ORIGINAL]); + memleak_container(struct ip_conntrack, tuplehash[IP_CT_DIR_REPLY]);if (!conntrack) {DEBUGP ("Can't allocate conntrack.\n" );return ERR_PTR (-ENOMEM );
net/sched/sch_generic.c +5 -0
@@ -437,6 +437,11 @@ struct Qdisc *qdisc_alloc(struct net_dev goto errout; sch = (struct Qdisc *)QDISC_ALIGN ((unsigned long ) p); sch->padded = (char *) sch - (char *) p; +/* kmemleak cannot guess the object type because the block +* size is different from the object size. The stored pointer +* is also different from the address of the allocated block +* because of padding */ +memleak_padding (p, sch->padded,sizeof (struct Qdisc ));INIT_LIST_HEAD (&sch->list);skb_queue_head_init (&sch->q);
PATCH 8/10 Keep the __init functions after initialization
In reply to Catalin MarinasFrom: Catalin Marinas <[email protected]>
This patch adds the CONFIG_DEBUG_KEEP_INIT option which preserves the .init.text section after initialization. Memory leaks happening during this phase can be more easily tracked.
---
include/linux/init.h | 5 +++++ lib/Kconfig.debug | 12 ++++++++++++ 2 files changed, 17 insertions(+), 0 deletions(-)
include/linux/init.h +5 -0
@@ -40,8 +40,13 @@ #include <linux/compiler.h>/* These are for everybody (although not all archs will actually discard it in modules) */ +#ifdef CONFIG_DEBUG_KEEP_INIT +#define __init +#define __initdata +#else #define __init __attribute__ ((__section__ (".init.text")))#define __initdata __attribute__ ((__section__ (".init.data"))) +#endif #define __exitdata __attribute__ ((__section__(".exit.data")))#define __exit_call __attribute_used__ __attribute__ ((__section__ (".exitcall.exit")))
lib/Kconfig.debug +12 -0
@@ -195,6 +195,18 @@ config DEBUG_MEMLEAK_REPORTS_NR reading the /sys/kernel/debug/memleak file could lead to some soft-locks. +config DEBUG_KEEP_INIT + bool "Do not free the __init code/data" + default n + depends on DEBUG_MEMLEAK + help + This option moves the __init code/data out of the + .init.text/.init.data sections. It is useful for identifying + memory leaks happening during the kernel or modules + initialization. + + If unsure, say N. + config DEBUG_PREEMPT bool "Debug preemptible kernel" depends on DEBUG_KERNEL && PREEMPT && TRACE_IRQFLAGS_SUPPORT
PATCH 9/10 Simple testing for kmemleak
In reply to Catalin MarinasFrom: Catalin Marinas <[email protected]>
This patch only contains some very simple testing at the moment. Proper testing will be needed.
---
lib/Kconfig.debug | 11 +++++++ mm/Makefile | 1 + mm/memleak-test.c | 89 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 101 insertions(+), 0 deletions(-)
lib/Kconfig.debug +11 -0
@@ -207,6 +207,17 @@ config DEBUG_KEEP_INIT If unsure, say N. +config DEBUG_MEMLEAK_TEST + tristate "Test the kernel memory leak detector" + default n + depends on DEBUG_MEMLEAK + help + Say Y or M here to build the test harness for the kernel + memory leak detector. At the moment, this option enables a + module that explicitly leaks memory. + + If unsure, say N. + config DEBUG_PREEMPT bool "Debug preemptible kernel" depends on DEBUG_KERNEL && PREEMPT && TRACE_IRQFLAGS_SUPPORT
mm/Makefile +1 -0
@@ -24,3 +24,4 @@ obj-$(CONFIG_MEMORY_HOTPLUG) += memory_h obj-$(CONFIG_FS_XIP) += filemap_xip.o obj-$(CONFIG_MIGRATION) += migrate.o obj-$(CONFIG_DEBUG_MEMLEAK) += memleak.o +obj-$(CONFIG_DEBUG_MEMLEAK_TEST) += memleak-test.o
mm/memleak-test.c new file +89 -0
89 lines not on this page · raw message
PATCH 10/10 Update the MAINTAINERS file for kmemleak
In reply to Catalin MarinasFrom: Catalin Marinas <[email protected]>
---
MAINTAINERS | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-)
MAINTAINERS +6 -0
@@ -1669,6 +1669,12 @@ L: [email protected] W: http://www.kerneljanitors.org/ S: Maintained +KERNEL MEMORY LEAK DETECTOR +P: Catalin Marinas +M: [email protected] +W: http://www.procode.org/ +S: Maintained + KERNEL NFSD P: Neil Brown M: [email protected]
Hi Catalin,
Quoted 6 lines · Catalin Marinas
This is a new version (0.10) of the kernel memory leak detector. See the
Documentation/kmemleak.txtfile for a more detailed description. The patches are downloadable from (the whole patch or the broken-out series):http://homepage.ntlworld.com/cmarinas/kmemleak/patch-2.6.18-rc6-kmemleak-0.10.bz2
I get a kernel panic
http://www.stardust.webpages.pl/files/o_bugs/kmemleak-0.10/panic.jpg
http://www.stardust.webpages.pl/files/o_bugs/kmemleak-0.10/kml-config
Regards,
Michal
Signature · 3 lines
Michal K. K. Piotrowski LTG - Linux Testers Group (http://www.stardust.webpages.pl/ltg/)
Quoted 4 lines · Michal Piotrowski
I get a kernel panic
http://www.stardust.webpages.pl/files/o_bugs/kmemleak-0.10/panic.jpghttp://www.stardust.webpages.pl/files/o_bugs/kmemleak-0.10/kml-config
Well, you set CONFIG_DEBUG_MEMLEAK_HASH_BITS to 32, this means that kmemleak needs to allocate (sizeof(void*) * 2^32) which is 16GB of RAM. I think a maximum of 20 should be enough (I got acceptable results with 16 hash bits, the default value, and it seemed to do better with 18).
Signature · 1 line
Catalin
Quoted 11 lines · Catalin Marinas
Quoted 4 lines · Michal Piotrowski
I get a kernel panic
http://www.stardust.webpages.pl/files/o_bugs/kmemleak-0.10/panic.jpghttp://www.stardust.webpages.pl/files/o_bugs/kmemleak-0.10/kml-config
Well, you set
CONFIG_DEBUG_MEMLEAK_HASH_BITSto 32, this means that kmemleak needs to allocate (sizeof(void*)* 2^32) which is 16GB of RAM. I think a maximum of 20 should be enough (I got acceptable results with 16 hash bits, the default value, and it seemed to do better with 18).
CONFIG_DEBUG_MEMLEAK=y CONFIG_DEBUG_MEMLEAK_HASH_BITS=8 CONFIG_DEBUG_MEMLEAK_TRACE_LENGTH=12 CONFIG_DEBUG_MEMLEAK_PREINIT_OBJECTS=512
# CONFIG_DEBUG_MEMLEAK_SECONDARY_ALIASES is not set # CONFIG_DEBUG_MEMLEAK_TASK_STACKS is not set # CONFIG_DEBUG_MEMLEAK_ORPHAN_FREEING is not set CONFIG_DEBUG_MEMLEAK_REPORTS_NR=100
# CONFIG_DEBUG_KEEP_INIT is not set
# CONFIG_DEBUG_MEMLEAK_TEST is not set
I still get a panic
Quoted 4 lines
--
Catalin
Regards,
Michal
Signature · 3 lines
Michal K. K. Piotrowski LTG - Linux Testers Group (http://www.stardust.webpages.pl/ltg/)
Quoted 24 lines · Michal Piotrowski
Quoted 11 lines · Catalin Marinas
Quoted 4 lines · Michal Piotrowski
I get a kernel panic
http://www.stardust.webpages.pl/files/o_bugs/kmemleak-0.10/panic.jpghttp://www.stardust.webpages.pl/files/o_bugs/kmemleak-0.10/kml-config
Well, you set
CONFIG_DEBUG_MEMLEAK_HASH_BITSto 32, this means that kmemleak needs to allocate (sizeof(void*)* 2^32) which is 16GB of RAM. I think a maximum of 20 should be enough (I got acceptable results with 16 hash bits, the default value, and it seemed to do better with 18).
CONFIG_DEBUG_MEMLEAK=yCONFIG_DEBUG_MEMLEAK_HASH_BITS=8CONFIG_DEBUG_MEMLEAK_TRACE_LENGTH=12CONFIG_DEBUG_MEMLEAK_PREINIT_OBJECTS=512
#CONFIG_DEBUG_MEMLEAK_SECONDARY_ALIASESis not set #CONFIG_DEBUG_MEMLEAK_TASK_STACKSis not set #CONFIG_DEBUG_MEMLEAK_ORPHAN_FREEINGis not setCONFIG_DEBUG_MEMLEAK_REPORTS_NR=100
#CONFIG_DEBUG_KEEP_INITis not set
#CONFIG_DEBUG_MEMLEAK_TESTis not set
Quoted 1 line
I still get a panic
Is it in the same place? What does the panic text say?
BTW, you should probably enable SECONDARY_ALIASES and TASK_STACKS above to reduce the number of false positives.
Signature · 1 line
Catalin
Quoted 25 lines · Catalin Marinas
Quoted 24 lines · Michal Piotrowski
Quoted 11 lines · Catalin Marinas
Quoted 4 lines · Michal Piotrowski
I get a kernel panic
http://www.stardust.webpages.pl/files/o_bugs/kmemleak-0.10/panic.jpghttp://www.stardust.webpages.pl/files/o_bugs/kmemleak-0.10/kml-config
Well, you set
CONFIG_DEBUG_MEMLEAK_HASH_BITSto 32, this means that kmemleak needs to allocate (sizeof(void*)* 2^32) which is 16GB of RAM. I think a maximum of 20 should be enough (I got acceptable results with 16 hash bits, the default value, and it seemed to do better with 18).
CONFIG_DEBUG_MEMLEAK=yCONFIG_DEBUG_MEMLEAK_HASH_BITS=8CONFIG_DEBUG_MEMLEAK_TRACE_LENGTH=12CONFIG_DEBUG_MEMLEAK_PREINIT_OBJECTS=512
#CONFIG_DEBUG_MEMLEAK_SECONDARY_ALIASESis not set #CONFIG_DEBUG_MEMLEAK_TASK_STACKSis not set #CONFIG_DEBUG_MEMLEAK_ORPHAN_FREEINGis not setCONFIG_DEBUG_MEMLEAK_REPORTS_NR=100
#CONFIG_DEBUG_KEEP_INITis not set
#CONFIG_DEBUG_MEMLEAK_TESTis not set
Quoted 3 lines
Quoted 1 line
I still get a panic
Is it in the same place? What does the panic text say?
It's the same "kmemleak: cannot allocate the pointer hash" panic.
Quoted 7 lines
BTW, you should probably enable
SECONDARY_ALIASESandTASK_STACKSabove to reduce the number of false positives.--
Catalin
Regards,
Michal
Signature · 3 lines
Michal K. K. Piotrowski LTG - Linux Testers Group (http://www.stardust.webpages.pl/ltg/)
Quoted 4 lines · Michal Piotrowski
Quoted 3 lines · Catalin Marinas
Quoted 2 lines · Michal Piotrowski
CONFIG_DEBUG_MEMLEAK=yCONFIG_DEBUG_MEMLEAK_HASH_BITS=8
Have you tried 16? With 8, a negative argument is passed as order to __get_free_pages(). I'll modify kmemleak to cope with this.
Signature · 1 line
Catalin
Quoted 7 lines · Catalin Marinas
Quoted 4 lines · Michal Piotrowski
Quoted 3 lines · Catalin Marinas
Quoted 2 lines · Michal Piotrowski
CONFIG_DEBUG_MEMLEAK=yCONFIG_DEBUG_MEMLEAK_HASH_BITS=8Have you tried 16?
No, I haven't.
Quoted 6 lines
With 8, a negative argument is passed as order to
__get_free_pages(). I'll modify kmemleak to cope with this.--
Catalin
Regards,
Michal
Signature · 3 lines
Michal K. K. Piotrowski LTG - Linux Testers Group (http://www.stardust.webpages.pl/ltg/)
Quoted 10 lines · Michal Piotrowski
Quoted 7 lines · Catalin Marinas
Quoted 4 lines · Michal Piotrowski
Quoted 3 lines · Catalin Marinas
Quoted 2 lines · Michal Piotrowski
CONFIG_DEBUG_MEMLEAK=yCONFIG_DEBUG_MEMLEAK_HASH_BITS=8Have you tried 16?
No, I haven't.
8 hash bits would lead to a really slow hash table lookup since you would only have 256 entries and it uses linked lists to deal with collisions (you may have tens of thousands of pointers to be stored in the hash). Anyway, I attach a patch which allows you to set small values but it is highly unrecommended.
Signature · 1 line
Catalin
Allow small values of HASH_BITS to be configured
From: Catalin Marinas <[email protected]>
The original implementation was assuming that the HASH_BITS value is small big enough so that the page order is at least 0. This patch corrects this.
---
mm/memleak.c | 9 ++++++--- 1 files changed, 6 insertions(+), 3 deletions(-)
mm/memleak.c +6 -3
@@ -180,12 +180,15 @@ #define OBJECT_TYPE_GUESSED 0x2/* Hash functions */ static void hash_init (void ) { -int i; -int hash_size =sizeof (*pointer_hash) * (1 <<HASH_BITS ); -int hash_order =fls (hash_size) -1 ; +unsigned int i; +unsigned int hash_size =sizeof (*pointer_hash) * (1 <<HASH_BITS ); +unsigned int hash_order =fls (hash_size) -1 ; +/* hash_size not a power of 2 */ if (hash_size & ((1 << hash_order) -1 )) hash_order +=1 ; +if (hash_order <PAGE_SHIFT ) + hash_order =PAGE_SHIFT ; pointer_hash = (struct hlist_head *)__get_free_pages (GFP_ATOMIC , hash_order -PAGE_SHIFT );
Quoted 5 lines · Catalin Marinas
Quoted 4 lines · Michal Piotrowski
Quoted 1 line · Catalin Marinas
Have you tried 16?
No, I haven't.
With DEBUG_MEMLEAK_HASH_BITS=16 system woks fine.
Quoted 6 lines
8 hash bits would lead to a really slow hash table lookup since you would only have 256 entries and it uses linked lists to deal with collisions (you may have tens of thousands of pointers to be stored in the hash). Anyway, I attach a patch which allows you to set small values but it is highly unrecommended.
How about documeting maxial "safe" values?
Something like
"If you have X MB/GB of RAM memory you can set Y
X | Y
256 MB | 25
512 MB | 26
1GB | 27 2GB | 28 4GB | 29"
Quoted 4 lines
--
Catalin
Regards,
Michal
Signature · 3 lines
Michal K. K. Piotrowski LTG - Linux Testers Group (http://www.stardust.webpages.pl/ltg/)