Skip to contentSkip to conversation

kmem_cache_destroy() badness with SLUB

DISCUSSION Benjamin Herrenschmidt 4 participants 5 messages Jun 28, 2010 – Jul 6, 2010 Memory management

lore
Share & tools
lore
#0 lore raw

Hi folks !

Internally, I'm hitting a little "nit"...

sysfs_slab_add() has this check:

	if (slab_state < SYSFS)
		/* Defer until later */
		return 0;

But sysfs_slab_remove() doesn't.

So if the slab is created -and- destroyed at, for example, arch_initcall time, then we hit a WARN in the kobject code, trying to dispose of a non-existing kobject.

Now, at first sight, just adding the same test to sysfs_slab_remove() would do the job... but it all seems very racy to me.

I don't understand in fact how this slab_state deals with races at all.

What prevents us from hitting slab_sysfs_init() at the same time as another CPU deos sysfs_slab_add() ? How do that deal with collisions trying to register the same kobject twice ? Similar race with remove...

Shouldn't we have a mutex around those guys ?

Cheers,
Ben.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#1 lore raw
Quoted 10 lines · Benjamin Herrenschmidt

Hi folks !

Internally, I'm hitting a little "nit"...

sysfs_slab_add() has this check:

	if (slab_state < SYSFS)
		/* Defer until later */
		return 0;
Quoted 6 lines

But sysfs_slab_remove() doesn't.

So if the slab is created -and- destroyed at, for example, arch_initcall time, then we hit a WARN in the kobject code, trying to dispose of a non-existing kobject.

Indeed, but shouldn't we be appropriately handling the return value of sysfs_slab_add() so that it fails cache creation? We wouldn't be calling sysfs_slab_remove() on a cache that was never created.

Quoted 5 lines

Now, at first sight, just adding the same test to sysfs_slab_remove() would do the job... but it all seems very racy to me.

I don't understand in fact how this slab_state deals with races at all.

All modifiers of slab_state are intended to be run only on the boot cpu so the only concern is the ordering. We need slab_state to indicate how far slab has been initialized since we can't otherwise enforce how code uses slab in between things like kmem_cache_init(), kmem_cache_init_late(), and initcalls on the boot cpu.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#2 lore raw
Quoted 14 lines · David Rientjes
Quoted 12 lines · Benjamin Herrenschmidt

Hi folks !

Internally, I'm hitting a little "nit"...

sysfs_slab_add() has this check:

	if (slab_state < SYSFS)
		/* Defer until later */
		return 0;

But sysfs_slab_remove() doesn't.

Quoted 7 lines
Quoted 4 lines

So if the slab is created -and- destroyed at, for example, arch_initcall time, then we hit a WARN in the kobject code, trying to dispose of a non-existing kobject.

Indeed, but shouldn't we be appropriately handling the return value of sysfs_slab_add() so that it fails cache creation? We wouldn't be calling sysfs_slab_remove() on a cache that was never created.

It's eventually created, but yes, we should probably store a state, unless we have a clean way to know the kobject in there is uninitialized and test for that.

Quoted 10 lines
Quoted 5 lines

Now, at first sight, just adding the same test to sysfs_slab_remove() would do the job... but it all seems very racy to me.

I don't understand in fact how this slab_state deals with races at all.

All modifiers of slab_state are intended to be run only on the boot cpu so the only concern is the ordering. We need slab_state to indicate how far slab has been initialized since we can't otherwise enforce how code uses slab in between things like kmem_cache_init(), kmem_cache_init_late(), and initcalls on the boot cpu.

But initcalls aren't pinned to the boot CPU... IE. I don't see how the sysfs creation avoids racing with SLAB creation, or am I missing something ?

Cheers,
Ben.

Quoted 6 lines

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#3 lore raw
In reply to Benjamin Herrenschmidt
Quoted 3 lines · Benjamin Herrenschmidt

So if the slab is created -and- destroyed at, for example, arch_initcall time, then we hit a WARN in the kobject code, trying to dispose of a non-existing kobject.

Yes dont do that.

Quoted 2 lines

Now, at first sight, just adding the same test to sysfs_slab_remove() would do the job... but it all seems very racy to me.

Yes lets leave as is. Dont remove slabs during boot.

Quoted 1 line

Shouldn't we have a mutex around those guys ?

At boot time?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#4 lore raw
In reply to Benjamin Herrenschmidt

Hi folks !
Internally, I'm hitting a little "nit"...

sysfs_slab_add() has this check:

	if (slab_state < SYSFS)
		/* Defer until later */
		return 0;

But sysfs_slab_remove() doesn't.

So if the slab is created -and- destroyed at, for example, arch_initcall time, then we hit a WARN in the kobject code, trying to dispose of a non-existing kobject.

Now, at first sight, just adding the same test to sysfs_slab_remove() would do the job... but it all seems very racy to me.

I don't understand in fact how this slab_state deals with races at all.

What prevents us from hitting slab_sysfs_init() at the same time as another CPU deos sysfs_slab_add() ? How do that deal with collisions trying to register the same kobject twice ? Similar race with remove...

Shouldn't we have a mutex around those guys ?

Cheers,
Ben.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Signature · 8 lines
Roland Dreier <[email protected]> || For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/index.html

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>