Skip to contentSkip to conversation

Export return addresses etc. for better diagnostics

PATCH Applied Paul E. McKenney 3 participants 17 messages Jan 6, 2021 – Jan 8, 2021

Subsystems (7)
lore
Share & tools
lore

Paul E. McKenney

author maintainer
#0 lore raw

v4 cover Export return addresses etc. for better diagnostics

Hello!

This is v4 of the series the improves diagnostics by providing access to additional information including the return addresses, slab names, offsets, and sizes collected by the sl*b allocators and by vmalloc(). If the allocator is not configured to collect this information, the diagnostics fall back to a reasonable approximation of their earlier state.

One use case is the queue_rcu_work() function, which might be used by any number of kernel subsystems. If the caller does back-to-back invocations of queue_rcu_work(), this constitutes a double-free bug, and (if so configured) the debug-objects system will flag this, printing the callback function. In most cases, printing this function suffices. However, for double-free bugs involving queue_rcu_work(), the RCU callback function will always be rcu_work_rcufn(), which provides almost no help to the poor person trying to find this double-free bug. The return address from the allocator of the memory containing the rcu_work structure can provide an additional valuable clue.

Another use case is the percpu_ref_switch_to_atomic_rcu() function, which detects percpu_ref reference-count underflow. Unfortunately, the only data that this function has access to doesn't have much in the way of identifying characteristics. Yes, it might be possible to gain more information from a crash dump, but it is more convenient for the needed hints to be in the console log.

Unfortunately, printing the return address in this case is of little help because this object is allocated from percpu_ref_init(), regardless of what part of the kernel is responsible for the reference-count underflow (though perhaps the slab and offsets might help in some cases). However, CONFIG_STACKTRACE=y kernels (such as those enabling ftrace) using slub with debugging enabled also collect stack traces. This series therefore also provides a way of extracting these stack traces to provide additional information to those debugging percpu_ref reference-count underflows.

The patches are as follows:

1. Add mem_dump_obj() to print source of memory block.

2. Make mem_dump_obj() handle NULL and zero-sized pointers.

3. Make mem_dump_obj() handle vmalloc() memory.

4. Make mem_obj_dump() vmalloc() dumps include start and length.

5. Make call_rcu() print mem_dump_obj() info for double-freed
callback.

6. percpu_ref: Dump mem_dump_obj() info upon reference-count
underflow.

Thanx, Paul

Changes since v3 (https://lore.kernel.org/lkml/20201211011907.GA16110@paulmck-ThinkPad-P72/):

o Extract more information from CONFIG_SLUB_DEBUG=n builds.

o Add Joonsoo Kim's Acked-by to 1/6 above.

o Rebased onto v5.11-rc1.

Changes since v2:

o Apply more feedback from Joonsoo Kim on naming and code structure.

o Based on discussions with Vlastimil Babka, added code to print
offsets and sizes where available. This can help identify which structure is involved.

Changes since v1:

o Apply feedback from Joonsoo Kim, mostly around naming and
code structure.

o Apply fix suggested by Stephen Rothwell for a bug that was
also located by kbuild test robot.

o Add support for vmalloc().

o Add support for special pointers.

o Additional rework simplifying use of mem_dump_obj(), which
simplifies both the RCU and the percpu_ref uses.

------------------------------------------------------------------------

 include/linux/mm.h      |    2 +
 include/linux/slab.h    |    2 +
 include/linux/vmalloc.h |    6 +++
 kernel/rcu/tree.c       |    7 +++-
 lib/percpu-refcount.c   |   12 +++++--
 mm/slab.c               |   20 ++++++++++++
 mm/slab.h               |   12 +++++++
 mm/slab_common.c        |   74 ++++++++++++++++++++++++++++++++++++++++++++++++
 mm/slob.c               |    6 +++
 mm/slub.c               |   40 +++++++++++++++++++++++++
 mm/util.c               |   45 ++++++++++++++++++++++++-----
 mm/vmalloc.c            |   15 +++++++++
 12 files changed, 228 insertions(+), 13 deletions(-)
author maintainer
#1 lore raw

PATCH v4 1/6 mm: Add mem_dump_obj() to print source of memory block

From: "Paul E. McKenney" <[email protected]>

There are kernel facilities such as per-CPU reference counts that give error messages in generic handlers or callbacks, whose messages are unenlightening. In the case of per-CPU reference-count underflow, this is not a problem when creating a new use of this facility because in that case the bug is almost certainly in the code implementing that new use. However, trouble arises when deploying across many systems, which might exercise corner cases that were not seen during development and testing. Here, it would be really nice to get some kind of hint as to which of several uses the underflow was caused by.

This commit therefore exposes a mem_dump_obj() function that takes a pointer to memory (which must still be allocated if it has been dynamically allocated) and prints available information on where that memory came from. This pointer can reference the middle of the block as well as the beginning of the block, as needed by things like RCU callback functions and timer handlers that might not know where the beginning of the memory block is. These functions and handlers can use mem_dump_obj() to print out better hints as to where the problem might lie.

The information printed can depend on kernel configuration. For example, the allocation return address can be printed only for slab and slub, and even then only when the necessary debug has been enabled. For slab, build with CONFIG_DEBUG_SLAB=y, and either use sizes with ample space to the next power of two or use the SLAB_STORE_USER when creating the kmem_cache structure. For slub, build with CONFIG_SLUB_DEBUG=y and boot with slub_debug=U, or pass SLAB_STORE_USER to kmem_cache_create() if more focused use is desired. Also for slub, use CONFIG_STACKTRACE to enable printing of the allocation-time stack trace.

Cc Christoph Lameter <[email protected]>
Cc Pekka Enberg <[email protected]>
Cc David Rientjes <[email protected]>
Cc Joonsoo Kim <[email protected]>
Cc Andrew Morton <[email protected]>
[ paulmck: Convert to printing and change names per Joonsoo Kim. ]
[ paulmck: Move slab definition per Stephen Rothwell and kbuild test robot. ]
[ paulmck: Handle CONFIG_MMU=n case where vmalloc() is kmalloc(). ]
[ paulmck: Apply Vlastimil Babka feedback on slab.c kmem_provenance(). ]
[ paulmck: Extract more info from !SLUB_DEBUG per Joonsoo Kim. ]
---
 include/linux/mm.h   |  2 ++
 include/linux/slab.h |  2 ++
 mm/slab.c            | 20 ++++++++++++++
 mm/slab.h            | 12 +++++++++
 mm/slab_common.c     | 74 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 mm/slob.c            |  6 +++++
 mm/slub.c            | 40 ++++++++++++++++++++++++++++
 mm/util.c            | 24 +++++++++++++++++
 8 files changed, 180 insertions(+)
include/linux/mm.h +2 -0
index 5299b90a..af7d050 100644
@@ -3169,5 +3169,7 @@ unsigned long wp_shared_mapping_range(struct address_space *mapping,
 
 extern int sysctl_nr_trim_pages;
 
+void mem_dump_obj(void *object);
+
 #endif /* __KERNEL__ */
 #endif /* _LINUX_MM_H */
include/linux/slab.h +2 -0
index be4ba58..7ae6040 100644
@@ -186,6 +186,8 @@ void kfree(const void *);
 void kfree_sensitive(const void *);
 size_t __ksize(const void *);
 size_t ksize(const void *);
+bool kmem_valid_obj(void *object);
+void kmem_dump_obj(void *object);
 
 #ifdef CONFIG_HAVE_HARDENED_USERCOPY_ALLOCATOR
 void __check_heap_object(const void *ptr, unsigned long n, struct page *page,
mm/slab.c +20 -0
index d7c8da9..dcc55e7 100644
@@ -3635,6 +3635,26 @@ void *__kmalloc_node_track_caller(size_t size, gfp_t flags,
 EXPORT_SYMBOL(__kmalloc_node_track_caller);
 #endif /* CONFIG_NUMA */
 
+void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
+{
+	struct kmem_cache *cachep;
+	unsigned int objnr;
+	void *objp;
+
+	kpp->kp_ptr = object;
+	kpp->kp_page = page;
+	cachep = page->slab_cache;
+	kpp->kp_slab_cache = cachep;
+	objp = object - obj_offset(cachep);
+	kpp->kp_data_offset = obj_offset(cachep);
+	page = virt_to_head_page(objp);
+	objnr = obj_to_index(cachep, page, objp);
+	objp = index_to_obj(cachep, page, objnr);
+	kpp->kp_objp = objp;
+	if (DEBUG && cachep->flags & SLAB_STORE_USER)
+		kpp->kp_ret = *dbg_userword(cachep, objp);
+}
+
 /**
  * __do_kmalloc - allocate memory
  * @size: how many bytes of memory are required.
mm/slab.h +12 -0
index 1a756a3..ecad9b5 100644
@@ -615,4 +615,16 @@ static inline bool slab_want_init_on_free(struct kmem_cache *c)
 	return false;
 }
 
+#define KS_ADDRS_COUNT 16
+struct kmem_obj_info {
+	void *kp_ptr;
+	struct page *kp_page;
+	void *kp_objp;
+	unsigned long kp_data_offset;
+	struct kmem_cache *kp_slab_cache;
+	void *kp_ret;
+	void *kp_stack[KS_ADDRS_COUNT];
+};
+void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page);
+
 #endif /* MM_SLAB_H */
mm/slab_common.c +74 -0

80 lines not on this page · raw message

mm/slob.c +6 -0
index 8d4bfa4..ef87ada 100644
@@ -461,6 +461,12 @@ static void slob_free(void *block, int size)
 	spin_unlock_irqrestore(&slob_lock, flags);
 }
 
+void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
+{
+	kpp->kp_ptr = object;
+	kpp->kp_page = page;
+}
+
 /*
  * End of slob allocator proper. Begin kmem_cache_alloc and kmalloc frontend.
  */
mm/slub.c +40 -0
index 0c8b43a..3c1a843 100644
@@ -3919,6 +3919,46 @@ int __kmem_cache_shutdown(struct kmem_cache *s)
 	return 0;
 }
 
+void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
+{
+	void *base;
+	int __maybe_unused i;
+	unsigned int objnr;
+	void *objp;
+	void *objp0;
+	struct kmem_cache *s = page->slab_cache;
+	struct track __maybe_unused *trackp;
+
+	kpp->kp_ptr = object;
+	kpp->kp_page = page;
+	kpp->kp_slab_cache = s;
+	base = page_address(page);
+	objp0 = kasan_reset_tag(object);
+#ifdef CONFIG_SLUB_DEBUG
+	objp = restore_red_left(s, objp0);
+#else
+	objp = objp0;
+#endif
+	objnr = obj_to_index(s, page, objp);
+	kpp->kp_data_offset = (unsigned long)((char *)objp0 - (char *)objp);
+	objp = base + s->size * objnr;
+	kpp->kp_objp = objp;
+	if (WARN_ON_ONCE(objp < base || objp >= base + page->objects * s->size || (objp - base) % s->size) ||
+	    !(s->flags & SLAB_STORE_USER))
+		return;
+#ifdef CONFIG_SLUB_DEBUG
+	trackp = get_track(s, objp, TRACK_ALLOC);
+	kpp->kp_ret = (void *)trackp->addr;
+#ifdef CONFIG_STACKTRACE
+	for (i = 0; i < KS_ADDRS_COUNT && i < TRACK_ADDRS_COUNT; i++) {
+		kpp->kp_stack[i] = (void *)trackp->addrs[i];
+		if (!kpp->kp_stack[i])
+			break;
+	}
+#endif
+#endif
+}
+
 /********************************************************************
  *		Kmalloc subsystem
  *******************************************************************/
mm/util.c +24 -0
index 8c9b7d1..da46f9d 100644
@@ -982,3 +982,27 @@ int __weak memcmp_pages(struct page *page1, struct page *page2)
 	kunmap_atomic(addr1);
 	return ret;
 }
+
+/**
+ * mem_dump_obj - Print available provenance information
+ * @object: object for which to find provenance information.
+ *
+ * This function uses pr_cont(), so that the caller is expected to have
+ * printed out whatever preamble is appropriate.  The provenance information
+ * depends on the type of object and on how much debugging is enabled.
+ * For example, for a slab-cache object, the slab name is printed, and,
+ * if available, the return address and stack trace from the allocation
+ * of that object.
+ */
+void mem_dump_obj(void *object)
+{
+	if (!virt_addr_valid(object)) {
+		pr_cont(" non-paged (local) memory.\n");
+		return;
+	}
+	if (kmem_valid_obj(object)) {
+		kmem_dump_obj(object);
+		return;
+	}
+	pr_cont(" non-slab memory.\n");
+}
Signature · 1 line
2.9.5
Acked-by on v4 1/6
#2 lore raw

Re: [PATCH mm,percpu_ref,rcu 1/6] mm: Add mem_dump_obj() to print source of memory block

Quoted 31 lines · On 1/6/21 2:17 AM, [email protected] wrote:

From: "Paul E. McKenney" <[email protected]>

There are kernel facilities such as per-CPU reference counts that give error messages in generic handlers or callbacks, whose messages are unenlightening. In the case of per-CPU reference-count underflow, this is not a problem when creating a new use of this facility because in that case the bug is almost certainly in the code implementing that new use. However, trouble arises when deploying across many systems, which might exercise corner cases that were not seen during development and testing. Here, it would be really nice to get some kind of hint as to which of several uses the underflow was caused by.

This commit therefore exposes a mem_dump_obj() function that takes a pointer to memory (which must still be allocated if it has been dynamically allocated) and prints available information on where that memory came from. This pointer can reference the middle of the block as well as the beginning of the block, as needed by things like RCU callback functions and timer handlers that might not know where the beginning of the memory block is. These functions and handlers can use mem_dump_obj() to print out better hints as to where the problem might lie.

The information printed can depend on kernel configuration. For example, the allocation return address can be printed only for slab and slub, and even then only when the necessary debug has been enabled. For slab, build with CONFIG_DEBUG_SLAB=y, and either use sizes with ample space to the next power of two or use the SLAB_STORE_USER when creating the kmem_cache structure. For slub, build with CONFIG_SLUB_DEBUG=y and boot with slub_debug=U, or pass SLAB_STORE_USER to kmem_cache_create() if more focused use is desired. Also for slub, use CONFIG_STACKTRACE to enable printing of the allocation-time stack trace.

Quoted 13 lines
Cc Christoph Lameter <[email protected]>
Cc Pekka Enberg <[email protected]>
Cc David Rientjes <[email protected]>
Cc Joonsoo Kim <[email protected]>
Cc Andrew Morton <[email protected]>
[ paulmck: Convert to printing and change names per Joonsoo Kim. ]
[ paulmck: Move slab definition per Stephen Rothwell and kbuild test robot. ]
[ paulmck: Handle CONFIG_MMU=n case where vmalloc() is kmalloc(). ]
[ paulmck: Apply Vlastimil Babka feedback on slab.c kmem_provenance(). ]
[ paulmck: Extract more info from !SLUB_DEBUG per Joonsoo Kim. ]

Some nits below:

Quoted 9 lines
mm/slab.c +3 -0
@@ -3635,6 +3635,26 @@ void *__kmalloc_node_track_caller(size_t size, gfp_t flags,
 EXPORT_SYMBOL(__kmalloc_node_track_caller);
 #endif /* CONFIG_NUMA */
 
+void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
+{
+	struct kmem_cache *cachep;

mm/slab.c void *__kmalloc_node_track_caller(size_t size, gfp_t flags, · line 3650

Quoted 10 lines
mm/slab.c +10 -0
+	unsigned int objnr;
+	void *objp;
+
+	kpp->kp_ptr = object;
+	kpp->kp_page = page;
+	cachep = page->slab_cache;
+	kpp->kp_slab_cache = cachep;
+	objp = object - obj_offset(cachep);
+	kpp->kp_data_offset = obj_offset(cachep);
+	page = virt_to_head_page(objp);

Hm when can this page be different from the "page" we got as function parameter? I guess only if "object" was so close to the beginning of page that "object - obj_offset(cachep)" underflowed it. So either "object" pointed to the padding/redzone, or even below page->s_mem. Both situations sounds like we should handle them differently than continuing with an unrelated page that's below our slab page?

mm/slab.c void *__kmalloc_node_track_caller(size_t size, gfp_t flags, · line 3651

Quoted 1 line
mm/slab.c +1 -0
+	objnr = obj_to_index(cachep, page, objp);

Related, this will return bogus value for objp below page->s_mem. And if our "object" pointer pointed beyond last valid object, this will give us too large index.

mm/slab.c void *__kmalloc_node_track_caller(size_t size, gfp_t flags, · line 3652

Quoted 1 line
mm/slab.c +1 -0
+	objp = index_to_obj(cachep, page, objnr);

Too large index can cause dbg_userword to be beyond our page.
In SLUB version you have the WARN_ON_ONCE that catches such invalid pointers (before first valid object or after last valid object) and skips getting the backtrace for those, so analogical thing should probably be done here?

Quoted 24 lines
mm/slab.c +5 -0
+	kpp->kp_objp = objp;
+	if (DEBUG && cachep->flags & SLAB_STORE_USER)
+		kpp->kp_ret = *dbg_userword(cachep, objp);
+}
+
mm/slub.c +11 -0
index 0c8b43a..3c1a843 100644
@@ -3919,6 +3919,46 @@ int __kmem_cache_shutdown(struct kmem_cache *s)
 	return 0;
 }
 
+void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
+{
+	void *base;
+	int __maybe_unused i;
+	unsigned int objnr;
+	void *objp;
+	void *objp0;
+	struct kmem_cache *s = page->slab_cache;
+	struct track __maybe_unused *trackp;
+
+	kpp->kp_ptr = object;

mm/slub.c int __kmem_cache_shutdown(struct kmem_cache *s) · line 3942

Quoted 10 lines
mm/slub.c +10 -0
+	kpp->kp_page = page;
+	kpp->kp_slab_cache = s;
+	base = page_address(page);
+	objp0 = kasan_reset_tag(object);
+#ifdef CONFIG_SLUB_DEBUG
+	objp = restore_red_left(s, objp0);
+#else
+	objp = objp0;
+#endif
+	objnr = obj_to_index(s, page, objp);

It would be safer to use objp0 instead of objp here I think. In case "object" was pointer to the first object's left red zone, then we would not have "objp" underflow "base" and get a bogus objnr. The WARN_ON_ONCE below could then be less paranoid? Basically just the "objp >= base + page->objects * s->size" should be possible if "object" points beyond the last valid object. But otherwise we should get valid index and thus valid "objp = base + s->size * objnr;" below, and "objp < base" and "(objp - base) % s->size)" should be impossible?

Hmm but since it would then be possible to get a negative pointer offset (into the left padding/redzone), kmem_dump_obj() should calculate and print it as signed? But it's not obvious if a pointer to left red zone is a pointer that was an overflow of object N-1 or underflow of object N, and which one to report (unless it's the very first object). AFAICS your current code will report all as overflows of object N-1, which is problematic with N=0 (as described above) so changing it to report underflows of object N would make more sense?

Thanks,
Vlastimil

author maintainer
#3 lore raw

Re: [PATCH mm,percpu_ref,rcu 1/6] mm: Add mem_dump_obj() to print source of memory block

Quoted 46 lines · Vlastimil Babka
Quoted 31 lines · On 1/6/21 2:17 AM, [email protected] wrote:

From: "Paul E. McKenney" <[email protected]>

There are kernel facilities such as per-CPU reference counts that give error messages in generic handlers or callbacks, whose messages are unenlightening. In the case of per-CPU reference-count underflow, this is not a problem when creating a new use of this facility because in that case the bug is almost certainly in the code implementing that new use. However, trouble arises when deploying across many systems, which might exercise corner cases that were not seen during development and testing. Here, it would be really nice to get some kind of hint as to which of several uses the underflow was caused by.

This commit therefore exposes a mem_dump_obj() function that takes a pointer to memory (which must still be allocated if it has been dynamically allocated) and prints available information on where that memory came from. This pointer can reference the middle of the block as well as the beginning of the block, as needed by things like RCU callback functions and timer handlers that might not know where the beginning of the memory block is. These functions and handlers can use mem_dump_obj() to print out better hints as to where the problem might lie.

The information printed can depend on kernel configuration. For example, the allocation return address can be printed only for slab and slub, and even then only when the necessary debug has been enabled. For slab, build with CONFIG_DEBUG_SLAB=y, and either use sizes with ample space to the next power of two or use the SLAB_STORE_USER when creating the kmem_cache structure. For slub, build with CONFIG_SLUB_DEBUG=y and boot with slub_debug=U, or pass SLAB_STORE_USER to kmem_cache_create() if more focused use is desired. Also for slub, use CONFIG_STACKTRACE to enable printing of the allocation-time stack trace.

Quoted 13 lines
Cc Christoph Lameter <[email protected]>
Cc Pekka Enberg <[email protected]>
Cc David Rientjes <[email protected]>
Cc Joonsoo Kim <[email protected]>
Cc Andrew Morton <[email protected]>
[ paulmck: Convert to printing and change names per Joonsoo Kim. ]
[ paulmck: Move slab definition per Stephen Rothwell and kbuild test robot. ]
[ paulmck: Handle CONFIG_MMU=n case where vmalloc() is kmalloc(). ]
[ paulmck: Apply Vlastimil Babka feedback on slab.c kmem_provenance(). ]
[ paulmck: Extract more info from !SLUB_DEBUG per Joonsoo Kim. ]
Quoted 1 line

Thank you for the review and comments!

Quoted 1 line

Some nits below:

Andrew pushed this to an upstream maintainer, but I have not seen these patches appear anywhere. So if that upstream maintainer was Linus, I can send a follow-up patch once we converge. If the upstream maintainer was in fact me, I can of course update the commit directly. If the upstream maintainer was someone else, please let me know who it is. ;-)

(Linus does not appear to have pushed anything out since before Andrew's email, hence my uncertainty.)

Quoted 18 lines
Quoted 18 lines
mm/slab.c +12 -0
@@ -3635,6 +3635,26 @@ void *__kmalloc_node_track_caller(size_t size, gfp_t flags,
 EXPORT_SYMBOL(__kmalloc_node_track_caller);
 #endif /* CONFIG_NUMA */
 
+void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
+{
+	struct kmem_cache *cachep;
+	unsigned int objnr;
+	void *objp;
+
+	kpp->kp_ptr = object;
+	kpp->kp_page = page;
+	cachep = page->slab_cache;
+	kpp->kp_slab_cache = cachep;
+	objp = object - obj_offset(cachep);
+	kpp->kp_data_offset = obj_offset(cachep);
Quoted 8 lines

mm/slab.c void *__kmalloc_node_track_caller(size_t size, gfp_t flags, · line 3650

Quoted 1 line
mm/slab.c +1 -0
+	page = virt_to_head_page(objp);

Hm when can this page be different from the "page" we got as function parameter? I guess only if "object" was so close to the beginning of page that "object - obj_offset(cachep)" underflowed it. So either "object" pointed to the padding/redzone, or even below page->s_mem. Both situations sounds like we should handle them differently than continuing with an unrelated page that's below our slab page?

I examined other code to obtain this. I have been assuming that the point was to be able to handle multipage slabs, but I freely confess to having no idea. But I am reluctant to change this sequence unless the other code translating from pointer to in-slab object is also changed.

Quoted 7 lines

mm/slab.c void *__kmalloc_node_track_caller(size_t size, gfp_t flags, · line 3651

Quoted 1 line
mm/slab.c +1 -0
+	objnr = obj_to_index(cachep, page, objp);

Related, this will return bogus value for objp below page->s_mem. And if our "object" pointer pointed beyond last valid object, this will give us too large index.

Quoted 6 lines

mm/slab.c void *__kmalloc_node_track_caller(size_t size, gfp_t flags, · line 3652

Quoted 1 line
mm/slab.c +1 -0
+	objp = index_to_obj(cachep, page, objnr);

Too large index can cause dbg_userword to be beyond our page.
In SLUB version you have the WARN_ON_ONCE that catches such invalid pointers (before first valid object or after last valid object) and skips getting the backtrace for those, so analogical thing should probably be done here?

Like this, just before the "objp =" statement?

	WARN_ON_ONCE(objnr >= cachep->num);
Quoted 44 lines
Quoted 24 lines
mm/slab.c +5 -0
+	kpp->kp_objp = objp;
+	if (DEBUG && cachep->flags & SLAB_STORE_USER)
+		kpp->kp_ret = *dbg_userword(cachep, objp);
+}
+
mm/slub.c +11 -0
index 0c8b43a..3c1a843 100644
@@ -3919,6 +3919,46 @@ int __kmem_cache_shutdown(struct kmem_cache *s)
 	return 0;
 }
 
+void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
+{
+	void *base;
+	int __maybe_unused i;
+	unsigned int objnr;
+	void *objp;
+	void *objp0;
+	struct kmem_cache *s = page->slab_cache;
+	struct track __maybe_unused *trackp;
+
+	kpp->kp_ptr = object;

mm/slub.c int __kmem_cache_shutdown(struct kmem_cache *s) · line 3942

Quoted 10 lines
mm/slub.c +10 -0
+	kpp->kp_page = page;
+	kpp->kp_slab_cache = s;
+	base = page_address(page);
+	objp0 = kasan_reset_tag(object);
+#ifdef CONFIG_SLUB_DEBUG
+	objp = restore_red_left(s, objp0);
+#else
+	objp = objp0;
+#endif
+	objnr = obj_to_index(s, page, objp);

It would be safer to use objp0 instead of objp here I think. In case "object" was pointer to the first object's left red zone, then we would not have "objp" underflow "base" and get a bogus objnr. The WARN_ON_ONCE below could then be less paranoid? Basically just the "objp >= base + page->objects * s->size" should be possible if "object" points beyond the last valid object. But otherwise we should get valid index and thus valid "objp = base + s->size * objnr;" below, and "objp < base" and "(objp - base) % s->size)" should be impossible?

Quoted 7 lines

Hmm but since it would then be possible to get a negative pointer offset (into the left padding/redzone), kmem_dump_obj() should calculate and print it as signed? But it's not obvious if a pointer to left red zone is a pointer that was an overflow of object N-1 or underflow of object N, and which one to report (unless it's the very first object). AFAICS your current code will report all as overflows of object N-1, which is problematic with N=0 (as described above) so changing it to report underflows of object N would make more sense?

Doesn't the "WARN_ON_ONCE(objp < base" further down report underflows? Or am I missing something subtle here?

Thanx, Paul

#4 lore raw

Re: [PATCH mm,percpu_ref,rcu 1/6] mm: Add mem_dump_obj() to print source of memory block

Quoted 9 lines · Paul E. McKenney

Andrew pushed this to an upstream maintainer, but I have not seen these patches appear anywhere. So if that upstream maintainer was Linus, I can send a follow-up patch once we converge. If the upstream maintainer was in fact me, I can of course update the commit directly. If the upstream maintainer was someone else, please let me know who it is. ;-)

(Linus does not appear to have pushed anything out since before Andrew's email, hence my uncertainty.)

I've wondered about the mm-commits messages too, and concluded that the most probable explanation is that Andrew tried to add your series to mmotm and then tried mmotm merge to linux-next and found out the series is already there via your rcu tree :)

Quoted 27 lines
Quoted 18 lines
Quoted 18 lines
mm/slab.c +12 -0
@@ -3635,6 +3635,26 @@ void *__kmalloc_node_track_caller(size_t size, gfp_t flags,
 EXPORT_SYMBOL(__kmalloc_node_track_caller);
 #endif /* CONFIG_NUMA */
 
+void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
+{
+	struct kmem_cache *cachep;
+	unsigned int objnr;
+	void *objp;
+
+	kpp->kp_ptr = object;
+	kpp->kp_page = page;
+	cachep = page->slab_cache;
+	kpp->kp_slab_cache = cachep;
+	objp = object - obj_offset(cachep);
+	kpp->kp_data_offset = obj_offset(cachep);
Quoted 8 lines

mm/slab.c void *__kmalloc_node_track_caller(size_t size, gfp_t flags, · line 3650

Quoted 1 line
mm/slab.c +1 -0
+	page = virt_to_head_page(objp);

Hm when can this page be different from the "page" we got as function parameter? I guess only if "object" was so close to the beginning of page that "object - obj_offset(cachep)" underflowed it. So either "object" pointed to the padding/redzone, or even below page->s_mem. Both situations sounds like we should handle them differently than continuing with an unrelated page that's below our slab page?

Quoted 4 lines

I examined other code to obtain this. I have been assuming that the point was to be able to handle multipage slabs, but I freely confess to having no idea. But I am reluctant to change this sequence unless the other code translating from pointer to in-slab object is also changed.

OK, I will check the other code.

Quoted 14 lines
Quoted 7 lines

mm/slab.c void *__kmalloc_node_track_caller(size_t size, gfp_t flags, · line 3651

Quoted 1 line
mm/slab.c +1 -0
+	objnr = obj_to_index(cachep, page, objp);

Related, this will return bogus value for objp below page->s_mem. And if our "object" pointer pointed beyond last valid object, this will give us too large index.

Quoted 6 lines

mm/slab.c void *__kmalloc_node_track_caller(size_t size, gfp_t flags, · line 3652

Quoted 1 line
mm/slab.c +1 -0
+	objp = index_to_obj(cachep, page, objnr);

Too large index can cause dbg_userword to be beyond our page.
In SLUB version you have the WARN_ON_ONCE that catches such invalid pointers (before first valid object or after last valid object) and skips getting the backtrace for those, so analogical thing should probably be done here?

Quoted 3 lines

Like this, just before the "objp =" statement?

	WARN_ON_ONCE(objnr >= cachep->num);

Yeah, that should do the trick to prevent accessing garbage dbg_userword.

But I wrote the comments about SLAB first and only in the SLUB part I realized about the larger picture. So now I would consider something like below, which should find the closest valid object index and resulting pointer offset in kmem_dump_obj() might become negative. Pointers to padding, below page->s_mem or beyond last object just become respectively large negative or positive pointer offsets and we probably don't need to warn for them specially unless we warn also for all other pointers that are not within the "data area" of the object.

void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)

{
        struct kmem_cache *cachep;
        unsigned int objnr;
        void *objp;

        kpp->kp_ptr = object;
        kpp->kp_page = page;
        cachep = page->slab_cache;
        kpp->kp_slab_cache = cachep;
        kpp->kp_data_offset = obj_offset(cachep);
	if (object < page->s_mem)
		objnr = 0;
	else
        	objnr = obj_to_index(cachep, page, object);
	if (objnr >= cachep->num)
		objnr = cachep->num - 1;
        objp = index_to_obj(cachep, page, objnr);
        kpp->kp_objp = objp;
        if (DEBUG && cachep->flags & SLAB_STORE_USER)
                kpp->kp_ret = *dbg_userword(cachep, objp);
}
Quoted 52 lines
Quoted 44 lines
Quoted 24 lines
mm/slab.c +5 -0
+	kpp->kp_objp = objp;
+	if (DEBUG && cachep->flags & SLAB_STORE_USER)
+		kpp->kp_ret = *dbg_userword(cachep, objp);
+}
+
mm/slub.c +11 -0
index 0c8b43a..3c1a843 100644
@@ -3919,6 +3919,46 @@ int __kmem_cache_shutdown(struct kmem_cache *s)
 	return 0;
 }
 
+void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
+{
+	void *base;
+	int __maybe_unused i;
+	unsigned int objnr;
+	void *objp;
+	void *objp0;
+	struct kmem_cache *s = page->slab_cache;
+	struct track __maybe_unused *trackp;
+
+	kpp->kp_ptr = object;

mm/slub.c int __kmem_cache_shutdown(struct kmem_cache *s) · line 3942

Quoted 10 lines
mm/slub.c +10 -0
+	kpp->kp_page = page;
+	kpp->kp_slab_cache = s;
+	base = page_address(page);
+	objp0 = kasan_reset_tag(object);
+#ifdef CONFIG_SLUB_DEBUG
+	objp = restore_red_left(s, objp0);
+#else
+	objp = objp0;
+#endif
+	objnr = obj_to_index(s, page, objp);

It would be safer to use objp0 instead of objp here I think. In case "object" was pointer to the first object's left red zone, then we would not have "objp" underflow "base" and get a bogus objnr. The WARN_ON_ONCE below could then be less paranoid? Basically just the "objp >= base + page->objects * s->size" should be possible if "object" points beyond the last valid object. But otherwise we should get valid index and thus valid "objp = base + s->size * objnr;" below, and "objp < base" and "(objp - base) % s->size)" should be impossible?

Quoted 7 lines

Hmm but since it would then be possible to get a negative pointer offset (into the left padding/redzone), kmem_dump_obj() should calculate and print it as signed? But it's not obvious if a pointer to left red zone is a pointer that was an overflow of object N-1 or underflow of object N, and which one to report (unless it's the very first object). AFAICS your current code will report all as overflows of object N-1, which is problematic with N=0 (as described above) so changing it to report underflows of object N would make more sense?

Quoted 1 line

Doesn't the "WARN_ON_ONCE(objp < base" further down report underflows?

I don't think it could be possible, could you describe the conditions?

Quoted 1 line

Or am I missing something subtle here?

A version analogical to the SLAB one above could AFAICS look like this:

...

        kpp->kp_ptr = object;
        kpp->kp_page = page;
        kpp->kp_slab_cache = s;
        base = page_address(page);
        objp0 = kasan_reset_tag(object);
#ifdef CONFIG_SLUB_DEBUG
        objp = restore_red_left(s, objp0);
#else
        objp = objp0;
#endif
        kpp->kp_data_offset = (unsigned long)((char *)objp0 - (char *)objp);
        objnr = obj_to_index(s, page, objp0); // unlike SLAB this can't underflow
	if (objnr >= page->objects)
		objnr = page->objects - 1;
        objp = base + s->size * objnr;
        kpp->kp_objp = objp;
	// no WARN_ON_ONCE() needed, objp has to be valid, we just might have negative
	// offset to it, or a larger than s->size positive offset
#ifdef CONFIG_SLUB_DEBUG
	// etc, no changes below
author maintainer
#5 lore raw

PATCH v4 2/6 mm: Make mem_dump_obj() handle NULL and zero-sized pointers

In reply to Paul E. McKenney

From: "Paul E. McKenney" <[email protected]>

This commit makes mem_dump_obj() call out NULL and zero-sized pointers specially instead of classifying them as non-paged memory.

---
 mm/util.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
mm/util.c +6 -1
index da46f9d..92f23d2 100644
@@ -997,7 +997,12 @@ int __weak memcmp_pages(struct page *page1, struct page *page2)
 void mem_dump_obj(void *object)
 {
 	if (!virt_addr_valid(object)) {
-		pr_cont(" non-paged (local) memory.\n");
+		if (object == NULL)
+			pr_cont(" NULL pointer.\n");
+		else if (object == ZERO_SIZE_PTR)
+			pr_cont(" zero-size pointer.\n");
+		else
+			pr_cont(" non-paged (local) memory.\n");
 		return;
 	}
 	if (kmem_valid_obj(object)) {
Signature · 1 line
2.9.5
author maintainer
#6 lore raw

PATCH v4 3/6 mm: Make mem_dump_obj() handle vmalloc() memory

In reply to Paul E. McKenney

From: "Paul E. McKenney" <[email protected]>

This commit adds vmalloc() support to mem_dump_obj(). Note that the vmalloc_dump_obj() function combines the checking and dumping, in contrast with the split between kmem_valid_obj() and kmem_dump_obj(). The reason for the difference is that the checking in the vmalloc() case involves acquiring a global lock, and redundant acquisitions of global locks should be avoided, even on not-so-fast paths.

Note that this change causes on-stack variables to be reported as vmalloc() storage from kernel_clone() or similar, depending on the degree of inlining that your compiler does. This is likely more helpful than the earlier "non-paged (local) memory".

---
 include/linux/vmalloc.h |  6 ++++++
 mm/util.c               | 14 ++++++++------
 mm/vmalloc.c            | 12 ++++++++++++
 3 files changed, 26 insertions(+), 6 deletions(-)
include/linux/vmalloc.h +6 -0
index 80c0181..c18f475 100644
@@ -246,4 +246,10 @@ pcpu_free_vm_areas(struct vm_struct **vms, int nr_vms)
 int register_vmap_purge_notifier(struct notifier_block *nb);
 int unregister_vmap_purge_notifier(struct notifier_block *nb);
 
+#ifdef CONFIG_MMU
+bool vmalloc_dump_obj(void *object);
+#else
+static inline bool vmalloc_dump_obj(void *object) { return false; }
+#endif
+
 #endif /* _LINUX_VMALLOC_H */
mm/util.c +8 -6
index 92f23d2..5487022 100644
@@ -996,18 +996,20 @@ int __weak memcmp_pages(struct page *page1, struct page *page2)
  */
 void mem_dump_obj(void *object)
 {
+	if (kmem_valid_obj(object)) {
+		kmem_dump_obj(object);
+		return;
+	}
+	if (vmalloc_dump_obj(object))
+		return;
 	if (!virt_addr_valid(object)) {
 		if (object == NULL)
 			pr_cont(" NULL pointer.\n");
 		else if (object == ZERO_SIZE_PTR)
 			pr_cont(" zero-size pointer.\n");
 		else
-			pr_cont(" non-paged (local) memory.\n");
-		return;
-	}
-	if (kmem_valid_obj(object)) {
-		kmem_dump_obj(object);
+			pr_cont(" non-paged memory.\n");
 		return;
 	}
-	pr_cont(" non-slab memory.\n");
+	pr_cont(" non-slab/vmalloc memory.\n");
 }
mm/vmalloc.c +12 -0
index 4d88fe5..c274ea4 100644
@@ -3448,6 +3448,18 @@ void pcpu_free_vm_areas(struct vm_struct **vms, int nr_vms)
 }
 #endif	/* CONFIG_SMP */
 
+bool vmalloc_dump_obj(void *object)
+{
+	struct vm_struct *vm;
+	void *objp = (void *)PAGE_ALIGN((unsigned long)object);
+
+	vm = find_vm_area(objp);
+	if (!vm)
+		return false;
+	pr_cont(" vmalloc allocated at %pS\n", vm->caller);
+	return true;
+}
+
 #ifdef CONFIG_PROC_FS
 static void *s_start(struct seq_file *m, loff_t *pos)
 	__acquires(&vmap_purge_lock)
Signature · 1 line
2.9.5
Acked-by on v4 3/6
#7 lore raw

Re: [PATCH mm,percpu_ref,rcu 3/6] mm: Make mem_dump_obj() handle vmalloc() memory

Quoted 14 lines · On 1/6/21 2:17 AM, [email protected] wrote:

From: "Paul E. McKenney" <[email protected]>

This commit adds vmalloc() support to mem_dump_obj(). Note that the vmalloc_dump_obj() function combines the checking and dumping, in contrast with the split between kmem_valid_obj() and kmem_dump_obj(). The reason for the difference is that the checking in the vmalloc() case involves acquiring a global lock, and redundant acquisitions of global locks should be avoided, even on not-so-fast paths.

Note that this change causes on-stack variables to be reported as vmalloc() storage from kernel_clone() or similar, depending on the degree of inlining that your compiler does. This is likely more helpful than the earlier "non-paged (local) memory".

Quoted 5 lines
author maintainer
#8 lore raw

PATCH v4 4/6 mm: Make mem_obj_dump() vmalloc() dumps include start and length

In reply to Paul E. McKenney

From: "Paul E. McKenney" <[email protected]>

This commit adds the starting address and number of pages to the vmalloc() information dumped by way of vmalloc_dump_obj().

---
 mm/vmalloc.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
mm/vmalloc.c +2 -1
index c274ea4..e3229ff 100644
@@ -3456,7 +3456,8 @@ bool vmalloc_dump_obj(void *object)
 	vm = find_vm_area(objp);
 	if (!vm)
 		return false;
-	pr_cont(" vmalloc allocated at %pS\n", vm->caller);
+	pr_cont(" %u-page vmalloc region starting at %#lx allocated at %pS\n",
+		vm->nr_pages, (unsigned long)vm->addr, vm->caller);
 	return true;
 }
 
Signature · 1 line
2.9.5
Acked-by on v4 4/6
#9 lore raw

Re: [PATCH mm,percpu_ref,rcu 4/6] mm: Make mem_obj_dump() vmalloc() dumps include start and length

Quoted 11 lines · On 1/6/21 2:17 AM, [email protected] wrote:

From: "Paul E. McKenney" <[email protected]>

This commit adds the starting address and number of pages to the vmalloc() information dumped by way of vmalloc_dump_obj().

Quoted 19 lines
---
 mm/vmalloc.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
mm/vmalloc.c +2 -1
index c274ea4..e3229ff 100644
@@ -3456,7 +3456,8 @@ bool vmalloc_dump_obj(void *object)
 	vm = find_vm_area(objp);
 	if (!vm)
 		return false;
-	pr_cont(" vmalloc allocated at %pS\n", vm->caller);
+	pr_cont(" %u-page vmalloc region starting at %#lx allocated at %pS\n",
+		vm->nr_pages, (unsigned long)vm->addr, vm->caller);
 	return true;
 }
 
author maintainer
#10 lore raw

PATCH v4 5/6 rcu: Make call_rcu() print mem_dump_obj() info for double-freed callback

In reply to Paul E. McKenney

From: "Paul E. McKenney" <[email protected]>

The debug-object double-free checks in __call_rcu() print out the RCU callback function, which is usually sufficient to track down the double free. However, all uses of things like queue_rcu_work() will have the same RCU callback function (rcu_work_rcufn() in this case), so a diagnostic message for a double queue_rcu_work() needs more than just the callback function.

This commit therefore calls mem_dump_obj() to dump out any additional available information on the double-freed callback.

Cc Christoph Lameter <[email protected]>
Cc Pekka Enberg <[email protected]>
Cc David Rientjes <[email protected]>
Cc Joonsoo Kim <[email protected]>
Cc Andrew Morton <[email protected]>
---
 kernel/rcu/tree.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
kernel/rcu/tree.c +5 -2
index 40e5e3d..84513c5 100644
@@ -2941,6 +2941,7 @@ static void check_cb_ovld(struct rcu_data *rdp)
 static void
 __call_rcu(struct rcu_head *head, rcu_callback_t func)
 {
+	static atomic_t doublefrees;
 	unsigned long flags;
 	struct rcu_data *rdp;
 	bool was_alldone;
@@ -2954,8 +2955,10 @@ __call_rcu(struct rcu_head *head, rcu_callback_t func)
 		 * Use rcu:rcu_callback trace event to find the previous
 		 * time callback was passed to __call_rcu().
 		 */
-		WARN_ONCE(1, "__call_rcu(): Double-freed CB %p->%pS()!!!\n",
-			  head, head->func);
+		if (atomic_inc_return(&doublefrees) < 4) {
+			pr_err("%s(): Double-freed CB %p->%pS()!!!  ", __func__, head, head->func);
+			mem_dump_obj(head);
+		}
 		WRITE_ONCE(head->func, rcu_leak_callback);
 		return;
 	}
Signature · 1 line
2.9.5
author maintainer
#11 lore raw

PATCH v4 6/6 percpu_ref: Dump mem_dump_obj() info upon reference-count underflow

In reply to Paul E. McKenney

From: "Paul E. McKenney" <[email protected]>

Reference-count underflow for percpu_ref is detected in the RCU callback percpu_ref_switch_to_atomic_rcu(), and the resulting warning does not print anything allowing easy identification of which percpu_ref use case is underflowing. This is of course not normally a problem when developing a new percpu_ref use case because it is most likely that the problem resides in this new use case. However, when deploying a new kernel to a large set of servers, the underflow might well be a new corner case in any of the old percpu_ref use cases.

This commit therefore calls mem_dump_obj() to dump out any additional available information on the underflowing percpu_ref instance.

---
 lib/percpu-refcount.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
lib/percpu-refcount.c +9 -3
index e59eda0..a1071cd 100644
@@ -5,6 +5,7 @@
 #include <linux/sched.h>
 #include <linux/wait.h>
 #include <linux/slab.h>
+#include <linux/mm.h>
 #include <linux/percpu-refcount.h>
 
 /*
@@ -168,6 +169,7 @@ static void percpu_ref_switch_to_atomic_rcu(struct rcu_head *rcu)
 			struct percpu_ref_data, rcu);
 	struct percpu_ref *ref = data->ref;
 	unsigned long __percpu *percpu_count = percpu_count_ptr(ref);
+	static atomic_t underflows;
 	unsigned long count = 0;
 	int cpu;
 
@@ -191,9 +193,13 @@ static void percpu_ref_switch_to_atomic_rcu(struct rcu_head *rcu)
 	 */
 	atomic_long_add((long)count - PERCPU_COUNT_BIAS, &data->count);
 
-	WARN_ONCE(atomic_long_read(&data->count) <= 0,
-		  "percpu ref (%ps) <= 0 (%ld) after switching to atomic",
-		  data->release, atomic_long_read(&data->count));
+	if (WARN_ONCE(atomic_long_read(&data->count) <= 0,
+		      "percpu ref (%ps) <= 0 (%ld) after switching to atomic",
+		      data->release, atomic_long_read(&data->count)) &&
+	    atomic_inc_return(&underflows) < 4) {
+		pr_err("%s(): percpu_ref underflow", __func__);
+		mem_dump_obj(data);
+	}
 
 	/* @ref is viewed as dead on all CPUs, send out switch confirmation */
 	percpu_ref_call_confirm_rcu(rcu);
Signature · 1 line
2.9.5
maintainer
#12 lore raw
In reply to Paul E. McKenney
Quoted 3 lines · On Tue, 5 Jan 2021 17:16:03 -0800 "Paul E. McKenney" <[email protected]> wrote:

This is v4 of the series the improves diagnostics by providing access to additional information including the return addresses, slab names, offsets, and sizes collected by the sl*b allocators and by vmalloc().

Looks reasonable. And not as bloaty as I feared, but it does add ~300 bytes to my allnoconfig build. Is the CONFIG_ coverage as tight as it could be?

author maintainer
#13 lore raw
Quoted 9 lines · Andrew Morton
Quoted 3 lines · On Tue, 5 Jan 2021 17:16:03 -0800 "Paul E. McKenney" <[email protected]> wrote:

This is v4 of the series the improves diagnostics by providing access to additional information including the return addresses, slab names, offsets, and sizes collected by the sl*b allocators and by vmalloc().

Looks reasonable. And not as bloaty as I feared, but it does add ~300 bytes to my allnoconfig build. Is the CONFIG_ coverage as tight as it could be?

Glad I managed to exceed your expectations. ;-)

Let's see... When I do an allnoconfig build, it has CONFIG_PRINTK=n. Given that this facility is based on printk(), I could create an additional patch that #ifdef'ed everything out in CONFIG_PRINTK=n kernels. This would uglify things a bit, but it would save ~300 bytes.

If I don't hear otherwise from you, I will put something together, test it, and send it along.

Thanx, Paul

author maintainer
#14 lore raw
Quoted 13 lines · Paul E. McKenney
Quoted 9 lines · Andrew Morton
Quoted 3 lines · On Tue, 5 Jan 2021 17:16:03 -0800 "Paul E. McKenney" <[email protected]> wrote:

This is v4 of the series the improves diagnostics by providing access to additional information including the return addresses, slab names, offsets, and sizes collected by the sl*b allocators and by vmalloc().

Looks reasonable. And not as bloaty as I feared, but it does add ~300 bytes to my allnoconfig build. Is the CONFIG_ coverage as tight as it could be?

Glad I managed to exceed your expectations. ;-)

Quoted 7 lines

Let's see... When I do an allnoconfig build, it has CONFIG_PRINTK=n. Given that this facility is based on printk(), I could create an additional patch that #ifdef'ed everything out in CONFIG_PRINTK=n kernels. This would uglify things a bit, but it would save ~300 bytes.

If I don't hear otherwise from you, I will put something together, test it, and send it along.

And is a first draft, very lightly tested. Thoughts?

Thanx, Paul

------------------------------------------------------------------------

commit 4164efdca255093a423b55f44bd788b46d9c648f

Author: Paul E. McKenney <[email protected]>
Date:   Thu Jan 7 13:46:11 2021 -0800

mm: Don't build mm_dump_obj() on CONFIG_PRINTK=n kernels

The mem_dump_obj() functionality adds a few hundred bytes, which is a small price to pay. Except on kernels built with CONFIG_PRINTK=n, in which mem_dump_obj() messages will be suppressed. This commit therefore makes mem_dump_obj() be a static inline empty function on kernels built with CONFIG_PRINTK=n and excludes all of its support functions as well. This avoids kernel bloat on systems that cannot use mem_dump_obj().

Cc Christoph Lameter <[email protected]>
Cc Pekka Enberg <[email protected]>
Cc David Rientjes <[email protected]>
Cc Joonsoo Kim <[email protected]>
include/linux/mm.h +4 -0
index af7d050..ed75a3a 100644
@@ -3169,7 +3169,11 @@ unsigned long wp_shared_mapping_range(struct address_space *mapping,
 
 extern int sysctl_nr_trim_pages;
 
+#ifdef CONFIG_PRINTK
 void mem_dump_obj(void *object);
+#else
+static inline void mem_dump_obj(void *object) {}
+#endif
 
 #endif /* __KERNEL__ */
 #endif /* _LINUX_MM_H */
include/linux/slab.h +2 -0
index 7ae6040..0c97d78 100644
@@ -186,8 +186,10 @@ void kfree(const void *);
 void kfree_sensitive(const void *);
 size_t __ksize(const void *);
 size_t ksize(const void *);
+#ifdef CONFIG_PRINTK
 bool kmem_valid_obj(void *object);
 void kmem_dump_obj(void *object);
+#endif
 
 #ifdef CONFIG_HAVE_HARDENED_USERCOPY_ALLOCATOR
 void __check_heap_object(const void *ptr, unsigned long n, struct page *page,
include/linux/vmalloc.h +1 -1
index c18f475..3c8a765 100644
@@ -246,7 +246,7 @@ pcpu_free_vm_areas(struct vm_struct **vms, int nr_vms)
 int register_vmap_purge_notifier(struct notifier_block *nb);
 int unregister_vmap_purge_notifier(struct notifier_block *nb);
 
-#ifdef CONFIG_MMU
+#if defined(CONFIG_MMU) && defined(CONFIG_PRINTK)
 bool vmalloc_dump_obj(void *object);
 #else
 static inline bool vmalloc_dump_obj(void *object) { return false; }
mm/slab.c +2 -0
index dcc55e7..965d277 100644
@@ -3635,6 +3635,7 @@ void *__kmalloc_node_track_caller(size_t size, gfp_t flags,
 EXPORT_SYMBOL(__kmalloc_node_track_caller);
 #endif /* CONFIG_NUMA */
 
+#ifdef CONFIG_PRINTK
 void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
 {
 	struct kmem_cache *cachep;
@@ -3654,6 +3655,7 @@ void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
 	if (DEBUG && cachep->flags & SLAB_STORE_USER)
 		kpp->kp_ret = *dbg_userword(cachep, objp);
 }
+#endif
 
 /**
  * __do_kmalloc - allocate memory
mm/slab.h +2 -0
index ecad9b5..d2e39ab 100644
@@ -615,6 +615,7 @@ static inline bool slab_want_init_on_free(struct kmem_cache *c)
 	return false;
 }
 
+#ifdef CONFIG_PRINTK
 #define KS_ADDRS_COUNT 16
 struct kmem_obj_info {
 	void *kp_ptr;
@@ -626,5 +627,6 @@ struct kmem_obj_info {
 	void *kp_stack[KS_ADDRS_COUNT];
 };
 void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page);
+#endif
 
 #endif /* MM_SLAB_H */
mm/slab_common.c +2 -0
index b594413..20b2cc6 100644
@@ -537,6 +537,7 @@ bool slab_is_available(void)
 	return slab_state >= UP;
 }
 
+#ifdef CONFIG_PRINTK
 /**
  * kmem_valid_obj - does the pointer reference a valid slab object?
  * @object: pointer to query.
@@ -610,6 +611,7 @@ void kmem_dump_obj(void *object)
 		pr_info("    %pS\n", kp.kp_stack[i]);
 	}
 }
+#endif
 
 #ifndef CONFIG_SLOB
 /* Create a cache during boot when no slab services are available yet */
mm/slob.c +2 -0
index ef87ada..8726931 100644
@@ -461,11 +461,13 @@ static void slob_free(void *block, int size)
 	spin_unlock_irqrestore(&slob_lock, flags);
 }
 
+#ifdef CONFIG_PRINTK
 void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
 {
 	kpp->kp_ptr = object;
 	kpp->kp_page = page;
 }
+#endif
 
 /*
  * End of slob allocator proper. Begin kmem_cache_alloc and kmalloc frontend.
mm/slub.c +2 -0
index 3c1a843..9e205e4 100644
@@ -3919,6 +3919,7 @@ int __kmem_cache_shutdown(struct kmem_cache *s)
 	return 0;
 }
 
+#ifdef CONFIG_PRINTK
 void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
 {
 	void *base;
@@ -3958,6 +3959,7 @@ void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
 #endif
 #endif
 }
+#endif
 
 /********************************************************************
  *		Kmalloc subsystem
mm/util.c +2 -0
index 5487022..2d497fe 100644
@@ -983,6 +983,7 @@ int __weak memcmp_pages(struct page *page1, struct page *page2)
 	return ret;
 }
 
+#ifdef CONFIG_PRINTK
 /**
  * mem_dump_obj - Print available provenance information
  * @object: object for which to find provenance information.
@@ -1013,3 +1014,4 @@ void mem_dump_obj(void *object)
 	}
 	pr_cont(" non-slab/vmalloc memory.\n");
 }
+#endif
mm/vmalloc.c +2 -0
index e3229ff..5002fd6 100644
@@ -3448,6 +3448,7 @@ void pcpu_free_vm_areas(struct vm_struct **vms, int nr_vms)
 }
 #endif	/* CONFIG_SMP */
 
+#ifdef CONFIG_PRINTK
 bool vmalloc_dump_obj(void *object)
 {
 	struct vm_struct *vm;
@@ -3460,6 +3461,7 @@ bool vmalloc_dump_obj(void *object)
 		vm->nr_pages, (unsigned long)vm->addr, vm->caller);
 	return true;
 }
+#endif
 
 #ifdef CONFIG_PROC_FS
 static void *s_start(struct seq_file *m, loff_t *pos)
#15 lore raw
Quoted 19 lines · Paul E. McKenney
Quoted 18 lines · Paul E. McKenney
Quoted 9 lines · Andrew Morton
Quoted 3 lines · On Tue, 5 Jan 2021 17:16:03 -0800 "Paul E. McKenney" <[email protected]> wrote:

This is v4 of the series the improves diagnostics by providing access to additional information including the return addresses, slab names, offsets, and sizes collected by the sl*b allocators and by vmalloc().

Looks reasonable. And not as bloaty as I feared, but it does add ~300 bytes to my allnoconfig build. Is the CONFIG_ coverage as tight as it could be?

Glad I managed to exceed your expectations. ;-)

Let's see... When I do an allnoconfig build, it has CONFIG_PRINTK=n. Given that this facility is based on printk(), I could create an additional patch that #ifdef'ed everything out in CONFIG_PRINTK=n kernels. This would uglify things a bit, but it would save ~300 bytes.

Quoted 4 lines
Quoted 2 lines

If I don't hear otherwise from you, I will put something together, test it, and send it along.

And is a first draft, very lightly tested. Thoughts?

Looks ok, but I'm not sure it's worth the trouble, there's probably tons of other code that could be treated like this and nobody is doing that systematically (at least I haven't heard about kernel tinyfication effort for years)... Up to Andrew I guess.

Thanks,
Vlastimil

Quoted 204 lines

Thanx, Paul

------------------------------------------------------------------------

commit 4164efdca255093a423b55f44bd788b46d9c648f

Author: Paul E. McKenney <[email protected]>
Date:   Thu Jan 7 13:46:11 2021 -0800

mm: Don't build mm_dump_obj() on CONFIG_PRINTK=n kernels

The mem_dump_obj() functionality adds a few hundred bytes, which is a small price to pay. Except on kernels built with CONFIG_PRINTK=n, in which mem_dump_obj() messages will be suppressed. This commit therefore makes mem_dump_obj() be a static inline empty function on kernels built with CONFIG_PRINTK=n and excludes all of its support functions as well. This avoids kernel bloat on systems that cannot use mem_dump_obj().

Cc Christoph Lameter <[email protected]>
Cc Pekka Enberg <[email protected]>
Cc David Rientjes <[email protected]>
Cc Joonsoo Kim <[email protected]>
include/linux/mm.h +4 -0
index af7d050..ed75a3a 100644
@@ -3169,7 +3169,11 @@ unsigned long wp_shared_mapping_range(struct address_space *mapping,
 
 extern int sysctl_nr_trim_pages;
 
+#ifdef CONFIG_PRINTK
 void mem_dump_obj(void *object);
+#else
+static inline void mem_dump_obj(void *object) {}
+#endif
 
 #endif /* __KERNEL__ */
 #endif /* _LINUX_MM_H */
include/linux/slab.h +2 -0
index 7ae6040..0c97d78 100644
@@ -186,8 +186,10 @@ void kfree(const void *);
 void kfree_sensitive(const void *);
 size_t __ksize(const void *);
 size_t ksize(const void *);
+#ifdef CONFIG_PRINTK
 bool kmem_valid_obj(void *object);
 void kmem_dump_obj(void *object);
+#endif
 
 #ifdef CONFIG_HAVE_HARDENED_USERCOPY_ALLOCATOR
 void __check_heap_object(const void *ptr, unsigned long n, struct page *page,
include/linux/vmalloc.h +1 -1
index c18f475..3c8a765 100644
@@ -246,7 +246,7 @@ pcpu_free_vm_areas(struct vm_struct **vms, int nr_vms)
 int register_vmap_purge_notifier(struct notifier_block *nb);
 int unregister_vmap_purge_notifier(struct notifier_block *nb);
 
-#ifdef CONFIG_MMU
+#if defined(CONFIG_MMU) && defined(CONFIG_PRINTK)
 bool vmalloc_dump_obj(void *object);
 #else
 static inline bool vmalloc_dump_obj(void *object) { return false; }
mm/slab.c +2 -0
index dcc55e7..965d277 100644
@@ -3635,6 +3635,7 @@ void *__kmalloc_node_track_caller(size_t size, gfp_t flags,
 EXPORT_SYMBOL(__kmalloc_node_track_caller);
 #endif /* CONFIG_NUMA */
 
+#ifdef CONFIG_PRINTK
 void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
 {
 	struct kmem_cache *cachep;
@@ -3654,6 +3655,7 @@ void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
 	if (DEBUG && cachep->flags & SLAB_STORE_USER)
 		kpp->kp_ret = *dbg_userword(cachep, objp);
 }
+#endif
 
 /**
  * __do_kmalloc - allocate memory
mm/slab.h +2 -0
index ecad9b5..d2e39ab 100644
@@ -615,6 +615,7 @@ static inline bool slab_want_init_on_free(struct kmem_cache *c)
 	return false;
 }
 
+#ifdef CONFIG_PRINTK
 #define KS_ADDRS_COUNT 16
 struct kmem_obj_info {
 	void *kp_ptr;
@@ -626,5 +627,6 @@ struct kmem_obj_info {
 	void *kp_stack[KS_ADDRS_COUNT];
 };
 void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page);
+#endif
 
 #endif /* MM_SLAB_H */
mm/slab_common.c +2 -0
index b594413..20b2cc6 100644
@@ -537,6 +537,7 @@ bool slab_is_available(void)
 	return slab_state >= UP;
 }
 
+#ifdef CONFIG_PRINTK
 /**
  * kmem_valid_obj - does the pointer reference a valid slab object?
  * @object: pointer to query.
@@ -610,6 +611,7 @@ void kmem_dump_obj(void *object)
 		pr_info("    %pS\n", kp.kp_stack[i]);
 	}
 }
+#endif
 
 #ifndef CONFIG_SLOB
 /* Create a cache during boot when no slab services are available yet */
mm/slob.c +2 -0
index ef87ada..8726931 100644
@@ -461,11 +461,13 @@ static void slob_free(void *block, int size)
 	spin_unlock_irqrestore(&slob_lock, flags);
 }
 
+#ifdef CONFIG_PRINTK
 void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
 {
 	kpp->kp_ptr = object;
 	kpp->kp_page = page;
 }
+#endif
 
 /*
  * End of slob allocator proper. Begin kmem_cache_alloc and kmalloc frontend.
mm/slub.c +2 -0
index 3c1a843..9e205e4 100644
@@ -3919,6 +3919,7 @@ int __kmem_cache_shutdown(struct kmem_cache *s)
 	return 0;
 }
 
+#ifdef CONFIG_PRINTK
 void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
 {
 	void *base;
@@ -3958,6 +3959,7 @@ void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
 #endif
 #endif
 }
+#endif
 
 /********************************************************************
  *		Kmalloc subsystem
mm/util.c +2 -0
index 5487022..2d497fe 100644
@@ -983,6 +983,7 @@ int __weak memcmp_pages(struct page *page1, struct page *page2)
 	return ret;
 }
 
+#ifdef CONFIG_PRINTK
 /**
  * mem_dump_obj - Print available provenance information
  * @object: object for which to find provenance information.
@@ -1013,3 +1014,4 @@ void mem_dump_obj(void *object)
 	}
 	pr_cont(" non-slab/vmalloc memory.\n");
 }
+#endif
mm/vmalloc.c +2 -0
index e3229ff..5002fd6 100644
@@ -3448,6 +3448,7 @@ void pcpu_free_vm_areas(struct vm_struct **vms, int nr_vms)
 }
 #endif	/* CONFIG_SMP */
 
+#ifdef CONFIG_PRINTK
 bool vmalloc_dump_obj(void *object)
 {
 	struct vm_struct *vm;
@@ -3460,6 +3461,7 @@ bool vmalloc_dump_obj(void *object)
 		vm->nr_pages, (unsigned long)vm->addr, vm->caller);
 	return true;
 }
+#endif
 
 #ifdef CONFIG_PROC_FS
 static void *s_start(struct seq_file *m, loff_t *pos)
author maintainer
#16 lore raw
Quoted 23 lines · Vlastimil Babka
Quoted 22 lines · Paul E. McKenney
Quoted 20 lines · Paul E. McKenney
Quoted 9 lines · Andrew Morton
Quoted 3 lines · On Tue, 5 Jan 2021 17:16:03 -0800 "Paul E. McKenney" <[email protected]> wrote:

This is v4 of the series the improves diagnostics by providing access to additional information including the return addresses, slab names, offsets, and sizes collected by the sl*b allocators and by vmalloc().

Looks reasonable. And not as bloaty as I feared, but it does add ~300 bytes to my allnoconfig build. Is the CONFIG_ coverage as tight as it could be?

Glad I managed to exceed your expectations. ;-)

Let's see... When I do an allnoconfig build, it has CONFIG_PRINTK=n. Given that this facility is based on printk(), I could create an additional patch that #ifdef'ed everything out in CONFIG_PRINTK=n kernels. This would uglify things a bit, but it would save ~300 bytes.

If I don't hear otherwise from you, I will put something together, test it, and send it along.

Quoted 6 lines
Quoted 1 line

And is a first draft, very lightly tested. Thoughts?

Looks ok, but I'm not sure it's worth the trouble, there's probably tons of other code that could be treated like this and nobody is doing that systematically (at least I haven't heard about kernel tinyfication effort for years)... Up to Andrew I guess.

I am good either way. ;-)

Thanx, Paul

Quoted 208 lines

Thanks,
Vlastimil

Quoted 204 lines

Thanx, Paul

------------------------------------------------------------------------

commit 4164efdca255093a423b55f44bd788b46d9c648f

Author: Paul E. McKenney <[email protected]>
Date:   Thu Jan 7 13:46:11 2021 -0800

mm: Don't build mm_dump_obj() on CONFIG_PRINTK=n kernels

The mem_dump_obj() functionality adds a few hundred bytes, which is a small price to pay. Except on kernels built with CONFIG_PRINTK=n, in which mem_dump_obj() messages will be suppressed. This commit therefore makes mem_dump_obj() be a static inline empty function on kernels built with CONFIG_PRINTK=n and excludes all of its support functions as well. This avoids kernel bloat on systems that cannot use mem_dump_obj().

Cc Christoph Lameter <[email protected]>
Cc Pekka Enberg <[email protected]>
Cc David Rientjes <[email protected]>
Cc Joonsoo Kim <[email protected]>
include/linux/mm.h +4 -0
index af7d050..ed75a3a 100644
@@ -3169,7 +3169,11 @@ unsigned long wp_shared_mapping_range(struct address_space *mapping,
 
 extern int sysctl_nr_trim_pages;
 
+#ifdef CONFIG_PRINTK
 void mem_dump_obj(void *object);
+#else
+static inline void mem_dump_obj(void *object) {}
+#endif
 
 #endif /* __KERNEL__ */
 #endif /* _LINUX_MM_H */
include/linux/slab.h +2 -0
index 7ae6040..0c97d78 100644
@@ -186,8 +186,10 @@ void kfree(const void *);
 void kfree_sensitive(const void *);
 size_t __ksize(const void *);
 size_t ksize(const void *);
+#ifdef CONFIG_PRINTK
 bool kmem_valid_obj(void *object);
 void kmem_dump_obj(void *object);
+#endif
 
 #ifdef CONFIG_HAVE_HARDENED_USERCOPY_ALLOCATOR
 void __check_heap_object(const void *ptr, unsigned long n, struct page *page,
include/linux/vmalloc.h +1 -1
index c18f475..3c8a765 100644
@@ -246,7 +246,7 @@ pcpu_free_vm_areas(struct vm_struct **vms, int nr_vms)
 int register_vmap_purge_notifier(struct notifier_block *nb);
 int unregister_vmap_purge_notifier(struct notifier_block *nb);
 
-#ifdef CONFIG_MMU
+#if defined(CONFIG_MMU) && defined(CONFIG_PRINTK)
 bool vmalloc_dump_obj(void *object);
 #else
 static inline bool vmalloc_dump_obj(void *object) { return false; }
mm/slab.c +2 -0
index dcc55e7..965d277 100644
@@ -3635,6 +3635,7 @@ void *__kmalloc_node_track_caller(size_t size, gfp_t flags,
 EXPORT_SYMBOL(__kmalloc_node_track_caller);
 #endif /* CONFIG_NUMA */
 
+#ifdef CONFIG_PRINTK
 void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
 {
 	struct kmem_cache *cachep;
@@ -3654,6 +3655,7 @@ void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
 	if (DEBUG && cachep->flags & SLAB_STORE_USER)
 		kpp->kp_ret = *dbg_userword(cachep, objp);
 }
+#endif
 
 /**
  * __do_kmalloc - allocate memory
mm/slab.h +2 -0
index ecad9b5..d2e39ab 100644
@@ -615,6 +615,7 @@ static inline bool slab_want_init_on_free(struct kmem_cache *c)
 	return false;
 }
 
+#ifdef CONFIG_PRINTK
 #define KS_ADDRS_COUNT 16
 struct kmem_obj_info {
 	void *kp_ptr;
@@ -626,5 +627,6 @@ struct kmem_obj_info {
 	void *kp_stack[KS_ADDRS_COUNT];
 };
 void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page);
+#endif
 
 #endif /* MM_SLAB_H */
mm/slab_common.c +2 -0
index b594413..20b2cc6 100644
@@ -537,6 +537,7 @@ bool slab_is_available(void)
 	return slab_state >= UP;
 }
 
+#ifdef CONFIG_PRINTK
 /**
  * kmem_valid_obj - does the pointer reference a valid slab object?
  * @object: pointer to query.
@@ -610,6 +611,7 @@ void kmem_dump_obj(void *object)
 		pr_info("    %pS\n", kp.kp_stack[i]);
 	}
 }
+#endif
 
 #ifndef CONFIG_SLOB
 /* Create a cache during boot when no slab services are available yet */
mm/slob.c +2 -0
index ef87ada..8726931 100644
@@ -461,11 +461,13 @@ static void slob_free(void *block, int size)
 	spin_unlock_irqrestore(&slob_lock, flags);
 }
 
+#ifdef CONFIG_PRINTK
 void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
 {
 	kpp->kp_ptr = object;
 	kpp->kp_page = page;
 }
+#endif
 
 /*
  * End of slob allocator proper. Begin kmem_cache_alloc and kmalloc frontend.
mm/slub.c +2 -0
index 3c1a843..9e205e4 100644
@@ -3919,6 +3919,7 @@ int __kmem_cache_shutdown(struct kmem_cache *s)
 	return 0;
 }
 
+#ifdef CONFIG_PRINTK
 void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
 {
 	void *base;
@@ -3958,6 +3959,7 @@ void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct page *page)
 #endif
 #endif
 }
+#endif
 
 /********************************************************************
  *		Kmalloc subsystem
mm/util.c +2 -0
index 5487022..2d497fe 100644
@@ -983,6 +983,7 @@ int __weak memcmp_pages(struct page *page1, struct page *page2)
 	return ret;
 }
 
+#ifdef CONFIG_PRINTK
 /**
  * mem_dump_obj - Print available provenance information
  * @object: object for which to find provenance information.
@@ -1013,3 +1014,4 @@ void mem_dump_obj(void *object)
 	}
 	pr_cont(" non-slab/vmalloc memory.\n");
 }
+#endif
mm/vmalloc.c +2 -0
index e3229ff..5002fd6 100644
@@ -3448,6 +3448,7 @@ void pcpu_free_vm_areas(struct vm_struct **vms, int nr_vms)
 }
 #endif	/* CONFIG_SMP */
 
+#ifdef CONFIG_PRINTK
 bool vmalloc_dump_obj(void *object)
 {
 	struct vm_struct *vm;
@@ -3460,6 +3461,7 @@ bool vmalloc_dump_obj(void *object)
 		vm->nr_pages, (unsigned long)vm->addr, vm->caller);
 	return true;
 }
+#endif
 
 #ifdef CONFIG_PROC_FS
 static void *s_start(struct seq_file *m, loff_t *pos)