Skip to contentSkip to conversation

"slab: Fix missing DEBUG_SLAB last user" breaks ARM

DISCUSSION Russell King 3 participants 11 messages Feb 7, 2011 – Feb 14, 2011 1 message not in the archive Memory management

lore
Share & tools
lore
#0 lore raw

commit 5c5e3b33 appears to break ARM thusly:

| Mount-cache hash table entries: 512
| slab error in verify_redzone_free(): cache `idr_layer_cache': memory outside object was overwritten
| Backtrace:               
| [<c0227088>] (dump_backtrace+0x0/0x110) from [<c0431afc>] (dump_stack+0x18/0x1c)
| [<c0431ae4>] (dump_stack+0x0/0x1c) from [<c0293304>] (__slab_error+0x28/0x30)
| [<c02932dc>] (__slab_error+0x0/0x30) from [<c0293a74>] (cache_free_debugcheck+0x1c0/0x2b8)               
| [<c02938b4>] (cache_free_debugcheck+0x0/0x2b8) from [<c0293f78>] (kmem_cache_free+0x3c/0xc0)             
| [<c0293f3c>] (kmem_cache_free+0x0/0xc0) from [<c032b1c8>] (ida_get_new_above+0x19c/0x1c0)                
| [<c032b02c>] (ida_get_new_above+0x0/0x1c0) from [<c02af7ec>] (alloc_vfsmnt+0x54/0x144)                   
| [<c02af798>] (alloc_vfsmnt+0x0/0x144) from [<c0299830>] (vfs_kern_mount+0x30/0xec)                       
| [<c0299800>] (vfs_kern_mount+0x0/0xec) from [<c0299908>] (kern_mount_data+0x1c/0x20)                     
| [<c02998ec>] (kern_mount_data+0x0/0x20) from [<c02146c4>] (sysfs_init+0x68/0xc8)
| [<c021465c>] (sysfs_init+0x0/0xc8) from [<c02137d4>] (mnt_init+0x90/0x1b0)
| [<c0213744>] (mnt_init+0x0/0x1b0) from [<c0213388>] (vfs_caches_init+0x100/0x140)
| [<c0213288>] (vfs_caches_init+0x0/0x140) from [<c0208c0c>] (start_kernel+0x2e8/0x368)                    
| [<c0208924>] (start_kernel+0x0/0x368) from [<c0208034>] (__enable_mmu+0x0/0x2c)
| c0113268: redzone 1:0xd84156c5c032b3ac, redzone 2:0xd84156c5635688c0.
| slab error in cache_alloc_debugcheck_after(): cache `idr_layer_cache': double free, or memory outside object was overwritten
| ...
| c011307c: redzone 1:0x9f91102ffffffff, redzone 2:0x9f911029d74e35b
| slab: Internal list corruption detected in cache 'idr_layer_cache'(24), slabp c0113000(16). Hexdump:     
|                          
| 000: 20 4f 10 c0 20 4f 10 c0 7c 00 00 00 7c 30 11 c0
| 010: 10 00 00 00 10 00 00 00 00 00 c9 17 fe ff ff ff
| 020: fe ff ff ff fe ff ff ff fe ff ff ff fe ff ff ff
| 030: fe ff ff ff fe ff ff ff fe ff ff ff fe ff ff ff
| 040: fe ff ff ff fe ff ff ff fe ff ff ff fe ff ff ff
| 050: fe ff ff ff fe ff ff ff fe ff ff ff 11 00 00 00
| 060: 12 00 00 00 13 00 00 00 14 00 00 00 15 00 00 00
| 070: 16 00 00 00 17 00 00 00 c0 88 56 63
| kernel BUG at /home/rmk/git/linux-2.6-rmk/mm/slab.c:2928!

This hasn't been noticed as I guess not many people use SLAB on ARM anymore, and even less people probably have SLAB debugging enabled. With SLAB debugging disabled, the system appears to behave correctly - or maybe the problem is just hidden.

--
Russell King
Linux kernel 2.6 ARM Linux - http://www.arm.linux.org.uk/
maintainer of:

#1 lore raw

Hi!

Quoted 34 lines · Russell King

commit 5c5e3b33 appears to break ARM thusly:

| Mount-cache hash table entries: 512
| slab error in verify_redzone_free(): cache `idr_layer_cache': memory outside object was overwritten
| Backtrace:
| [<c0227088>] (dump_backtrace+0x0/0x110) from [<c0431afc>] (dump_stack+0x18/0x1c)
| [<c0431ae4>] (dump_stack+0x0/0x1c) from [<c0293304>] (__slab_error+0x28/0x30)
| [<c02932dc>] (__slab_error+0x0/0x30) from [<c0293a74>] (cache_free_debugcheck+0x1c0/0x2b8)
| [<c02938b4>] (cache_free_debugcheck+0x0/0x2b8) from [<c0293f78>] (kmem_cache_free+0x3c/0xc0)
| [<c0293f3c>] (kmem_cache_free+0x0/0xc0) from [<c032b1c8>] (ida_get_new_above+0x19c/0x1c0)
| [<c032b02c>] (ida_get_new_above+0x0/0x1c0) from [<c02af7ec>] (alloc_vfsmnt+0x54/0x144)
| [<c02af798>] (alloc_vfsmnt+0x0/0x144) from [<c0299830>] (vfs_kern_mount+0x30/0xec)
| [<c0299800>] (vfs_kern_mount+0x0/0xec) from [<c0299908>] (kern_mount_data+0x1c/0x20)
| [<c02998ec>] (kern_mount_data+0x0/0x20) from [<c02146c4>] (sysfs_init+0x68/0xc8)
| [<c021465c>] (sysfs_init+0x0/0xc8) from [<c02137d4>] (mnt_init+0x90/0x1b0)
| [<c0213744>] (mnt_init+0x0/0x1b0) from [<c0213388>] (vfs_caches_init+0x100/0x140)
| [<c0213288>] (vfs_caches_init+0x0/0x140) from [<c0208c0c>] (start_kernel+0x2e8/0x368)
| [<c0208924>] (start_kernel+0x0/0x368) from [<c0208034>] (__enable_mmu+0x0/0x2c)
| c0113268: redzone 1:0xd84156c5c032b3ac, redzone 2:0xd84156c5635688c0.
| slab error in cache_alloc_debugcheck_after(): cache `idr_layer_cache': double free, or memory outside object was overwritten
| ...
| c011307c: redzone 1:0x9f91102ffffffff, redzone 2:0x9f911029d74e35b
| slab: Internal list corruption detected in cache 'idr_layer_cache'(24), slabp c0113000(16). Hexdump:
|
| 000: 20 4f 10 c0 20 4f 10 c0 7c 00 00 00 7c 30 11 c0
| 010: 10 00 00 00 10 00 00 00 00 00 c9 17 fe ff ff ff
| 020: fe ff ff ff fe ff ff ff fe ff ff ff fe ff ff ff
| 030: fe ff ff ff fe ff ff ff fe ff ff ff fe ff ff ff
| 040: fe ff ff ff fe ff ff ff fe ff ff ff fe ff ff ff
| 050: fe ff ff ff fe ff ff ff fe ff ff ff 11 00 00 00
| 060: 12 00 00 00 13 00 00 00 14 00 00 00 15 00 00 00
| 070: 16 00 00 00 17 00 00 00 c0 88 56 63
| kernel BUG at /home/rmk/git/linux-2.6-rmk/mm/slab.c:2928!
Quoted 4 lines

This hasn't been noticed as I guess not many people use SLAB on ARM anymore, and even less people probably have SLAB debugging enabled. With SLAB debugging disabled, the system appears to behave correctly - or maybe the problem is just hidden.

I thought the fix was for ARM, actually, but unfortunately the changelog is somewhat terse (shame on me). Hmm?

#2 lore raw
Quoted 37 lines · Pekka Enberg

Hi!

Quoted 34 lines · Russell King

commit 5c5e3b33 appears to break ARM thusly:

| Mount-cache hash table entries: 512
| slab error in verify_redzone_free(): cache `idr_layer_cache': memory outside object was overwritten
| Backtrace:
| [<c0227088>] (dump_backtrace+0x0/0x110) from [<c0431afc>] (dump_stack+0x18/0x1c)
| [<c0431ae4>] (dump_stack+0x0/0x1c) from [<c0293304>] (__slab_error+0x28/0x30)
| [<c02932dc>] (__slab_error+0x0/0x30) from [<c0293a74>] (cache_free_debugcheck+0x1c0/0x2b8)
| [<c02938b4>] (cache_free_debugcheck+0x0/0x2b8) from [<c0293f78>] (kmem_cache_free+0x3c/0xc0)
| [<c0293f3c>] (kmem_cache_free+0x0/0xc0) from [<c032b1c8>] (ida_get_new_above+0x19c/0x1c0)
| [<c032b02c>] (ida_get_new_above+0x0/0x1c0) from [<c02af7ec>] (alloc_vfsmnt+0x54/0x144)
| [<c02af798>] (alloc_vfsmnt+0x0/0x144) from [<c0299830>] (vfs_kern_mount+0x30/0xec)
| [<c0299800>] (vfs_kern_mount+0x0/0xec) from [<c0299908>] (kern_mount_data+0x1c/0x20)
| [<c02998ec>] (kern_mount_data+0x0/0x20) from [<c02146c4>] (sysfs_init+0x68/0xc8)
| [<c021465c>] (sysfs_init+0x0/0xc8) from [<c02137d4>] (mnt_init+0x90/0x1b0)
| [<c0213744>] (mnt_init+0x0/0x1b0) from [<c0213388>] (vfs_caches_init+0x100/0x140)
| [<c0213288>] (vfs_caches_init+0x0/0x140) from [<c0208c0c>] (start_kernel+0x2e8/0x368)
| [<c0208924>] (start_kernel+0x0/0x368) from [<c0208034>] (__enable_mmu+0x0/0x2c)
| c0113268: redzone 1:0xd84156c5c032b3ac, redzone 2:0xd84156c5635688c0.
| slab error in cache_alloc_debugcheck_after(): cache `idr_layer_cache': double free, or memory outside object was overwritten
| ...
| c011307c: redzone 1:0x9f91102ffffffff, redzone 2:0x9f911029d74e35b
| slab: Internal list corruption detected in cache 'idr_layer_cache'(24), slabp c0113000(16). Hexdump:
|
| 000: 20 4f 10 c0 20 4f 10 c0 7c 00 00 00 7c 30 11 c0
| 010: 10 00 00 00 10 00 00 00 00 00 c9 17 fe ff ff ff
| 020: fe ff ff ff fe ff ff ff fe ff ff ff fe ff ff ff
| 030: fe ff ff ff fe ff ff ff fe ff ff ff fe ff ff ff
| 040: fe ff ff ff fe ff ff ff fe ff ff ff fe ff ff ff
| 050: fe ff ff ff fe ff ff ff fe ff ff ff 11 00 00 00
| 060: 12 00 00 00 13 00 00 00 14 00 00 00 15 00 00 00
| 070: 16 00 00 00 17 00 00 00 c0 88 56 63
| kernel BUG at /home/rmk/git/linux-2.6-rmk/mm/slab.c:2928!
Quoted 7 lines
Quoted 4 lines

This hasn't been noticed as I guess not many people use SLAB on ARM anymore, and even less people probably have SLAB debugging enabled. With SLAB debugging disabled, the system appears to behave correctly - or maybe the problem is just hidden.

I thought the fix was for ARM, actually, but unfortunately the changelog is somewhat terse (shame on me). Hmm?

As you say, the changelog is soo terse that we now have no idea why this commit was created.

So, what's the fix here? It looks to me like a revert is in order.

I suggest that if it does get reverted, a copy of my original bug report is included in the revert commit so that all the details are available if it needs revisiting.

Signature · 3 lines
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:
#3 lore raw
Quoted 11 lines · Russell King
Quoted 2 lines

I thought the fix was for ARM, actually, but unfortunately the changelog is somewhat terse (shame on me). Hmm?

As you say, the changelog is soo terse that we now have no idea why this commit was created.

So, what's the fix here? It looks to me like a revert is in order.

I suggest that if it does get reverted, a copy of my original bug report is included in the revert commit so that all the details are available if it needs revisiting.

Uhh.. That first hunk in commit 5c5e3b33b7cb959a401f823707bee006caadd76e looks wrong to me. ralign is usually a power of two and greater than alignof(unsigned long long). The & operation will result in 0 and never exempt any caches.

mm/slab.c +0 -0
index bac0f4f..525c664 100644
@@ -2220,8 +2220,8 @@ kmem_cache_create (const char *name, size_t size,

size_t align,

        if (ralign < align) {
                ralign = align;
        }
-       /* disable debug if necessary */
-       if (ralign > __alignof__(unsigned long long))
+       /* disable debug if not aligning with REDZONE_ALIGN */
+       if (ralign & (__alignof__(unsigned long long) - 1))
                flags &= ~(SLAB_RED_ZONE | SLAB_STORE_USER);
        /*
         * 4) Store it.

Did you mean

if (ralign > REDZONE_ALIGN) ...

?

#4 lore raw
Quoted 37 lines · Christoph Lameter
Quoted 11 lines · Russell King
Quoted 2 lines

I thought the fix was for ARM, actually, but unfortunately the changelog is somewhat terse (shame on me). Hmm?

As you say, the changelog is soo terse that we now have no idea why this commit was created.

So, what's the fix here?  It looks to me like a revert is in order.

I suggest that if it does get reverted, a copy of my original bug report is included in the revert commit so that all the details are available if it needs revisiting.

Uhh.. That first hunk in commit 5c5e3b33b7cb959a401f823707bee006caadd76e looks wrong to me. ralign is usually a power of two and greater than alignof(unsigned long long). The & operation will result in 0 and never exempt any caches.

mm/slab.c +2 -2
index bac0f4f..525c664 100644
@@ -2220,8 +2220,8 @@ kmem_cache_create (const char *name, size_t size,
 size_t align,
        if (ralign < align) {
                ralign = align;
        }
-       /* disable debug if necessary */
-       if (ralign > __alignof__(unsigned long long))
+       /* disable debug if not aligning with REDZONE_ALIGN */
+       if (ralign & (__alignof__(unsigned long long) - 1))
                flags &= ~(SLAB_RED_ZONE | SLAB_STORE_USER);
        /*

        * 4) Store it.

Quoted 5 lines

Did you mean

       if (ralign > REDZONE_ALIGN) ...

?

Ping. Any arm-capable tester out there that can verify this fix? OTOH, since Shiyong Li hasn't showed up, I'm inclined to just revert the commit...

#5 lore raw
Quoted 26 lines · Pekka Enberg
Quoted 25 lines · Christoph Lameter

Uhh.. That first hunk in commit 5c5e3b33b7cb959a401f823707bee006caadd76e looks wrong to me. ralign is usually a power of two and greater than alignof(unsigned long long). The & operation will result in 0 and never exempt any caches.

mm/slab.c +2 -2
index bac0f4f..525c664 100644
@@ -2220,8 +2220,8 @@ kmem_cache_create (const char *name, size_t size,
 size_t align,
        if (ralign < align) {
                ralign = align;
        }
-       /* disable debug if necessary */
-       if (ralign > __alignof__(unsigned long long))
+       /* disable debug if not aligning with REDZONE_ALIGN */
+       if (ralign & (__alignof__(unsigned long long) - 1))
                flags &= ~(SLAB_RED_ZONE | SLAB_STORE_USER);
        /*

        * 4) Store it.

Did you mean

Quoted 7 lines
Quoted 3 lines

       if (ralign > REDZONE_ALIGN) ...

?

Ping. Any arm-capable tester out there that can verify this fix? OTOH, since Shiyong Li hasn't showed up, I'm inclined to just revert the commit...

What fix? The above is a quote of the patch in 5c5e3b33.

Signature · 3 lines
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:
#6 lore raw
Quoted 29 lines · Russell King
Quoted 28 lines · Pekka Enberg
Quoted 27 lines · Christoph Lameter

Uhh.. That first hunk in commit 5c5e3b33b7cb959a401f823707bee006caadd76e looks wrong to me. ralign is usually a power of two and greater than alignof(unsigned long long). The & operation will result in 0 and never exempt any caches.

mm/slab.c +2 -2
index bac0f4f..525c664 100644
@@ -2220,8 +2220,8 @@ kmem_cache_create (const char *name, size_t size,
 size_t align,
        if (ralign < align) {
                ralign = align;
        }
-       /* disable debug if necessary */
-       if (ralign > __alignof__(unsigned long long))
+       /* disable debug if not aligning with REDZONE_ALIGN */
+       if (ralign & (__alignof__(unsigned long long) - 1))
                flags &= ~(SLAB_RED_ZONE | SLAB_STORE_USER);
        /*

        * 4) Store it.

Did you mean

       if (ralign > REDZONE_ALIGN) ...

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

?

Ping. Any arm-capable tester out there that can verify this fix? OTOH, since Shiyong Li hasn't showed up, I'm inclined to just revert the commit...

What fix? The above is a quote of the patch in 5c5e3b33.

He was referring to the change of the if statement.

#7 lore raw
Quoted 37 lines · Christoph Lameter
Quoted 35 lines · Russell King
Quoted 33 lines · Pekka Enberg
Quoted 23 lines · Christoph Lameter

Uhh.. That first hunk in commit 5c5e3b33b7cb959a401f823707bee006caadd76e looks wrong to me. ralign is usually a power of two and greater than alignof(unsigned long long). The & operation will result in 0 and never exempt any caches.

mm/slab.c +2 -2
index bac0f4f..525c664 100644
@@ -2220,8 +2220,8 @@ kmem_cache_create (const char *name, size_t size,
 size_t align,
        if (ralign < align) {
                ralign = align;
        }
-       /* disable debug if necessary */
-       if (ralign > __alignof__(unsigned long long))
+       /* disable debug if not aligning with REDZONE_ALIGN */
+       if (ralign & (__alignof__(unsigned long long) - 1))
                flags &= ~(SLAB_RED_ZONE | SLAB_STORE_USER);
        /*

        * 4) Store it.

Quoted 5 lines

Did you mean

       if (ralign > REDZONE_ALIGN) ...

?

Ping. Any arm-capable tester out there that can verify this fix? OTOH, since Shiyong Li hasn't showed up, I'm inclined to just revert the commit...

Quoted 3 lines
Quoted 1 line

What fix? The above is a quote of the patch in 5c5e3b33.

He was referring to the change of the if statement.

Replacing what with what?

	if (ralign & (__alignof__(unsigned long long) - 1))

with
if (ralign > __alignof__(unsigned long long))
or
if (ralign > REDZONE_ALIGN)

? A patch is a wonderful way to describe the change to be tested.

Signature · 3 lines
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:

A message that is not in the archive (sent off-list or not yet received): [email protected]

#9 lore raw
Quoted 7 lines · ShiYong LI

Hi all,

Sorry for my late reply.

My commit is created to enable ARM with 128 byte cache line support last information dump while slab corruption occurs. I've verified it works fine on ARMv7.

Mainline only supports 32 byte and 64 byte cache lines on ARM. It does not support 128 byte cache lines, so maybe your attempts are the wrong approach. Can you please post your 128 byte cache line support patches to the ARM kernel mailing list for review.

In the mean time, because your patch causes an oopsing regression, I strongly suggest that it is reverted until we start supporting 128 byte cache lines on ARM in mainline.

Quoted 4 lines

Russell,

Can you provide more information for the problem? What kind of ARM are you using ?

ARM926, which has a cache line size of 32.

Signature · 3 lines
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:
#10 lore raw
Quoted 8 lines · Russell King

Mainline only supports 32 byte and 64 byte cache lines on ARM.  It does not support 128 byte cache lines, so maybe your attempts are the wrong approach.  Can you please post your 128 byte cache line support patches to the ARM kernel mailing list for review.

In the mean time, because your patch causes an oopsing regression, I strongly suggest that it is reverted until we start supporting 128 byte cache lines on ARM in mainline.

Agreed. I'll queue a revert. Thanks, Russell!

#11 lore raw
Quoted 11 lines · Pekka Enberg
Quoted 8 lines · Russell King

Mainline only supports 32 byte and 64 byte cache lines on ARM.  It does not support 128 byte cache lines, so maybe your attempts are the wrong approach.  Can you please post your 128 byte cache line support patches to the ARM kernel mailing list for review.

In the mean time, because your patch causes an oopsing regression, I strongly suggest that it is reverted until we start supporting 128 byte cache lines on ARM in mainline.

Agreed. I'll queue a revert. Thanks, Russell!

It's this commit queued for linux-next:

http://git.kernel.org/?p=linux/kernel/git/penberg/slab-2.6.git;a=commit;h=3ff84a7f36554b257cd57325b1a7c1fa4b49fbe3