mm: Rename SLAB_DESTROY_BY_RCU to SLAB_TYPESAFE_BY_RCU
Subsystems (20)
RFC PATCH 1/1 mm: Rename SLAB_DESTROY_BY_RCU to SLAB_TYPESAFE_BY_RCU
A group of Linux kernel hackers reported chasing a bug that resulted from their assumption that SLAB_DESTROY_BY_RCU provided an existence guarantee, that is, that no block from such a slab would be reallocated during an RCU read-side critical section. Of course, that is not the case. Instead, SLAB_DESTROY_BY_RCU only prevents freeing of an entire slab of blocks.
However, there is a phrase for this, namely "type safety". This commit therefore renames SLAB_DESTROY_BY_RCU to SLAB_TYPESAFE_BY_RCU in order to avoid future instances of this sort of confusion.
Documentation/RCU/00-INDEX +1 -1
@@ -17,7 +17,7 @@ rcu_dereference.txt rcubarrier.txt - RCU and Unloadable Modules rculist_nulls.txt - - RCU list primitives for use with SLAB_DESTROY_BY_RCU + - RCU list primitives for use with SLAB_TYPESAFE_BY_RCU rcuref.txt - Reference-count design for elements of lists/arrays protected by RCU rcu.txt
Documentation/RCU/rculist_nulls.txt +3 -3
@@ -1,5 +1,5 @@ Using hlist_nulls to protect read-mostly linked lists and -objects using SLAB_DESTROY_BY_RCU allocations. +objects using SLAB_TYPESAFE_BY_RCU allocations. Please read the basics in Documentation/RCU/listRCU.txt
@@ -7,7 +7,7 @@ Using special makers (called 'nulls') is a convenient way to solve following problem : A typical RCU linked list managing objects which are -allocated with SLAB_DESTROY_BY_RCU kmem_cache can +allocated with SLAB_TYPESAFE_BY_RCU kmem_cache can use following algos : 1) Lookup algo
@@ -96,7 +96,7 @@ unlock_chain(); // typically a spin_unlock() 3) Remove algo -------------- Nothing special here, we can use a standard RCU hlist deletion. -But thanks to SLAB_DESTROY_BY_RCU, beware a deleted object can be reused +But thanks to SLAB_TYPESAFE_BY_RCU, beware a deleted object can be reused very very fast (before the end of RCU grace period) if (put_last_reference_on(obj) {
Documentation/RCU/whatisRCU.txt +1 -1
@@ -925,7 +925,7 @@ d. Do you need RCU grace periods to complete even in the face e. Is your workload too update-intensive for normal use of RCU, but inappropriate for other synchronization mechanisms? - If so, consider SLAB_DESTROY_BY_RCU. But please be careful! + If so, consider SLAB_TYPESAFE_BY_RCU. But please be careful! f. Do you need read-side critical sections that are respected even though they are in the middle of the idle loop, during
drivers/gpu/drm/i915/i915_gem.c +1 -1
@@ -4541,7 +4541,7 @@ i915_gem_load_init(struct drm_device *dev) dev_priv->requests =KMEM_CACHE (drm_i915_gem_request,SLAB_HWCACHE_ALIGN |SLAB_RECLAIM_ACCOUNT | -SLAB_DESTROY_BY_RCU ); +SLAB_TYPESAFE_BY_RCU );if (!dev_priv->requests) goto err_vmas;
drivers/gpu/drm/i915/i915_gem_request.h +1 -1
@@ -504,7 +504,7 @@ static inline struct drm_i915_gem_request * __i915_gem_active_get_rcu(const struct i915_gem_active *active) {/* Performing a lockless retrieval of the active request is super -* tricky. SLAB_DESTROY_BY_RCU merely guarantees that the backing +* tricky. SLAB_TYPESAFE_BY_RCU merely guarantees that the backing * slab of request objects will not be freed whilst we hold the * RCU read lock. It does not guarantee that the request itself * will not be freed and then *reused*. Viz,
drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c +1 -1
@@ -1071,7 +1071,7 @@ int ldlm_init(void) ldlm_lock_slab =kmem_cache_create ("ldlm_locks" ,sizeof (struct ldlm_lock ),0 ,SLAB_HWCACHE_ALIGN | -SLAB_DESTROY_BY_RCU ,NULL ); +SLAB_TYPESAFE_BY_RCU ,NULL );if (!ldlm_lock_slab) {kmem_cache_destroy (ldlm_resource_slab);return -ENOMEM ;
fs/jbd2/journal.c +1 -1
@@ -2340,7 +2340,7 @@ static int jbd2_journal_init_journal_head_cache(void) jbd2_journal_head_cache =kmem_cache_create ("jbd2_journal_head" ,sizeof (struct journal_head ),0 ,/* offset */ -SLAB_TEMPORARY |SLAB_DESTROY_BY_RCU , +SLAB_TEMPORARY |SLAB_TYPESAFE_BY_RCU ,NULL );/* ctor */ retval =0 ;if (!jbd2_journal_head_cache) {
fs/signalfd.c +1 -1
@@ -38,7 +38,7 @@ void signalfd_cleanup(struct sighand_struct *sighand)/* * The lockless check can race with remove_wait_queue() in progress, * but in this case its caller should run under rcu_read_lock() and -* sighand_cachep is SLAB_DESTROY_BY_RCU, we can safely return. +* sighand_cachep is SLAB_TYPESAFE_BY_RCU, we can safely return. */ if (likely (!waitqueue_active (wqh)))return ;
include/linux/dma-fence.h +2 -2
@@ -229,7 +229,7 @@ static inline struct dma_fence *dma_fence_get_rcu(struct dma_fence *fence)* * Function returns NULL if no refcount could be obtained, or the fence. * This function handles acquiring a reference to a fence that may be -* reallocated within the RCU grace period (such as with SLAB_DESTROY_BY_RCU), +* reallocated within the RCU grace period (such as with SLAB_TYPESAFE_BY_RCU), * so long as the caller is using RCU on the pointer to the fence. * * An alternative mechanism is to employ a seqlock to protect a bunch of
@@ -257,7 +257,7 @@ dma_fence_get_rcu_safe(struct dma_fence * __rcu *fencep)* have successfully acquire a reference to it. If it no * longer matches, we are holding a reference to some other * reallocated pointer. This is possible if the allocator -* is using a freelist like SLAB_DESTROY_BY_RCU where the +* is using a freelist like SLAB_TYPESAFE_BY_RCU where the * fence remains valid for the RCU grace period, but it * may be reallocated. When using such allocators, we are * responsible for ensuring the reference we get is to
include/linux/slab.h +2 -2
@@ -28,7 +28,7 @@#define SLAB_STORE_USER 0x00010000UL/* DEBUG: Store the last owner for bug hunting */ #define SLAB_PANIC 0x00040000UL/* Panic if kmem_cache_create() fails */ /* -* SLAB_DESTROY_BY_RCU - **WARNING** READ THIS! +* SLAB_TYPESAFE_BY_RCU - **WARNING** READ THIS! * * This delays freeing the SLAB page by a grace period, it does _NOT_ * delay object freeing. This means that if you do kmem_cache_free()
@@ -62,7 +62,7 @@* rcu_read_lock before reading the address, then rcu_read_unlock after * taking the spinlock within the structure expected at that address. */ -#define SLAB_DESTROY_BY_RCU 0x00080000UL/* Defer freeing slabs to RCU */ +#define SLAB_TYPESAFE_BY_RCU 0x00080000UL/* Defer freeing slabs to RCU */ #define SLAB_MEM_SPREAD 0x00100000UL/* Spread some memory over cpuset */ #define SLAB_TRACE 0x00200000UL/* Trace allocations and frees */
include/net/sock.h +1 -1
@@ -989,7 +989,7 @@ struct raw_hashinfo;struct module ;/* -* caches using SLAB_DESTROY_BY_RCU should let .next pointer from nulls nodes +* caches using SLAB_TYPESAFE_BY_RCU should let .next pointer from nulls nodes * un-modified. Special care is taken when initializing object to zero. */ static inline void sk_prot_clear_nulls (struct sock *sk,int size)
kernel/fork.c +2 -2
@@ -1297,7 +1297,7 @@ void __cleanup_sighand(struct sighand_struct *sighand)if (atomic_dec_and_test (&sighand->count)) {signalfd_cleanup (sighand);/* -* sighand_cachep is SLAB_DESTROY_BY_RCU so we can free it +* sighand_cachep is SLAB_TYPESAFE_BY_RCU so we can free it * without an RCU grace period, see __lock_task_sighand(). */ kmem_cache_free (sighand_cachep, sighand);
@@ -2069,7 +2069,7 @@ void __init proc_caches_init(void) { sighand_cachep =kmem_cache_create ("sighand_cache" ,sizeof (struct sighand_struct ),0 , -SLAB_HWCACHE_ALIGN |SLAB_PANIC |SLAB_DESTROY_BY_RCU | +SLAB_HWCACHE_ALIGN |SLAB_PANIC |SLAB_TYPESAFE_BY_RCU |SLAB_NOTRACK |SLAB_ACCOUNT , sighand_ctor); signal_cachep = kmem_cache_create("signal_cache", sizeof(struct signal_struct), 0,
kernel/signal.c +1 -1
@@ -1232,7 +1232,7 @@ struct sighand_struct *__lock_task_sighand(struct task_struct *tsk, }/* * This sighand can be already freed and even reused, but -* we rely on SLAB_DESTROY_BY_RCU and sighand_ctor() which +* we rely on SLAB_TYPESAFE_BY_RCU and sighand_ctor() which * initializes ->siglock: this slab can't go away, it has * the same object type, ->siglock can't be reinitialized. *
mm/kasan/kasan.c +3 -3
@@ -402,7 +402,7 @@ void kasan_cache_create(struct kmem_cache *cache, size_t *size, *size +=sizeof (struct kasan_alloc_meta );/* Add free meta. */ -if (cache->flags &SLAB_DESTROY_BY_RCU || cache->ctor || +if (cache->flags &SLAB_TYPESAFE_BY_RCU || cache->ctor || cache->object_size <sizeof (struct kasan_free_meta )) { cache->kasan_info.free_meta_offset = *size; *size +=sizeof (struct kasan_free_meta );
@@ -550,7 +550,7 @@ static void kasan_poison_slab_free(struct kmem_cache *cache, void *object)unsigned long rounded_up_size =round_up (size,KASAN_SHADOW_SCALE_SIZE );/* RCU slabs could be legally used after free within the RCU period */ -if (unlikely (cache->flags &SLAB_DESTROY_BY_RCU )) +if (unlikely (cache->flags &SLAB_TYPESAFE_BY_RCU ))return ;kasan_poison_shadow (object, rounded_up_size,KASAN_KMALLOC_FREE );
@@ -561,7 +561,7 @@ bool kasan_slab_free(struct kmem_cache *cache, void *object)s8 shadow_byte;/* RCU slabs could be legally used after free within the RCU period */ -if (unlikely (cache->flags &SLAB_DESTROY_BY_RCU )) +if (unlikely (cache->flags &SLAB_TYPESAFE_BY_RCU ))return false; shadow_byte =READ_ONCE (*(s8 *)kasan_mem_to_shadow (object));
mm/kmemcheck.c +1 -1
@@ -95,7 +95,7 @@ void kmemcheck_slab_alloc(struct kmem_cache *s, gfp_t gfpflags, void *object,void kmemcheck_slab_free (struct kmem_cache *s,void *object,size_t size) {/* TODO: RCU freeing is unsupported for now; hide false positives. */ -if (!s->ctor && !(s->flags &SLAB_DESTROY_BY_RCU )) +if (!s->ctor && !(s->flags &SLAB_TYPESAFE_BY_RCU ))kmemcheck_mark_freed (object, size); }
mm/rmap.c +2 -2
@@ -428,7 +428,7 @@ static void anon_vma_ctor(void *data)void __init anon_vma_init (void ) { anon_vma_cachep =kmem_cache_create ("anon_vma" ,sizeof (struct anon_vma ), -0 ,SLAB_DESTROY_BY_RCU |SLAB_PANIC |SLAB_ACCOUNT , +0 ,SLAB_TYPESAFE_BY_RCU |SLAB_PANIC |SLAB_ACCOUNT , anon_vma_ctor); anon_vma_chain_cachep =KMEM_CACHE (anon_vma_chain,SLAB_PANIC |SLAB_ACCOUNT );
@@ -479,7 +479,7 @@ struct anon_vma *page_get_anon_vma(struct page *page)* If this page is still mapped, then its anon_vma cannot have been * freed. But if it has been unmapped, we have no security against the * anon_vma structure being freed and reused (for another anon_vma: -* SLAB_DESTROY_BY_RCU guarantees that - so the atomic_inc_not_zero() +* SLAB_TYPESAFE_BY_RCU guarantees that - so the atomic_inc_not_zero() * above cannot corrupt). */ if (!page_mapped (page)) {
mm/slab.c +3 -3
@@ -1726,7 +1726,7 @@ static void slab_destroy(struct kmem_cache *cachep, struct page *page) freelist = page->freelist;slab_destroy_debugcheck (cachep, page); -if (unlikely (cachep->flags &SLAB_DESTROY_BY_RCU )) +if (unlikely (cachep->flags &SLAB_TYPESAFE_BY_RCU ))call_rcu (&page->rcu_head, kmem_rcu_free);else kmem_freepages (cachep, page);
@@ -1922,7 +1922,7 @@ static bool set_objfreelist_slab_cache(struct kmem_cache *cachep, cachep->num =0 ; -if (cachep->ctor || flags &SLAB_DESTROY_BY_RCU ) +if (cachep->ctor || flags &SLAB_TYPESAFE_BY_RCU )return false; left = calculate_slab_order(cachep, size,
@@ -2028,7 +2028,7 @@ __kmem_cache_create (struct kmem_cache *cachep, unsigned long flags)if (size <4096 ||fls (size -1 ) ==fls (size-1 +REDZONE_ALIGN +2 *sizeof (unsigned long long ))) flags |=SLAB_RED_ZONE |SLAB_STORE_USER ; -if (!(flags &SLAB_DESTROY_BY_RCU )) +if (!(flags &SLAB_TYPESAFE_BY_RCU )) flags |=SLAB_POISON ;#endif #endif
mm/slab.h +2 -2
@@ -120,7 +120,7 @@ static inline unsigned long kmem_cache_flags(unsigned long object_size,/* Legal flag mask for kmem_cache_create(), for various configurations */ #define SLAB_CORE_FLAGS (SLAB_HWCACHE_ALIGN | SLAB_CACHE_DMA | SLAB_PANIC | \ - SLAB_DESTROY_BY_RCU | SLAB_DEBUG_OBJECTS ) + SLAB_TYPESAFE_BY_RCU | SLAB_DEBUG_OBJECTS )#if defined(CONFIG_DEBUG_SLAB )#define SLAB_DEBUG_FLAGS (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER)
@@ -391,7 +391,7 @@ static inline size_t slab_ksize(const struct kmem_cache *s)* back there or track user information then we can * only use the space before that information. */ -if (s->flags & (SLAB_DESTROY_BY_RCU |SLAB_STORE_USER )) +if (s->flags & (SLAB_TYPESAFE_BY_RCU |SLAB_STORE_USER ))return s->inuse;/* * Else we can use all the padding etc for the allocation
mm/slab_common.c +2 -2
@@ -34,7 +34,7 @@ struct kmem_cache *kmem_cache;* Set of flags that will prevent slab merging */ #define SLAB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \ - SLAB_TRACE | SLAB_DESTROY_BY_RCU | SLAB_NOLEAKTRACE | \ + SLAB_TRACE | SLAB_TYPESAFE_BY_RCU | SLAB_NOLEAKTRACE | \ SLAB_FAILSLAB | SLAB_KASAN)#define SLAB_MERGE_SAME (SLAB_RECLAIM_ACCOUNT | SLAB_CACHE_DMA | \
@@ -464,7 +464,7 @@ static int shutdown_cache(struct kmem_cache *s,if (__kmem_cache_shutdown (s) !=0 )return -EBUSY ; -if (s->flags &SLAB_DESTROY_BY_RCU ) +if (s->flags &SLAB_TYPESAFE_BY_RCU ) *need_rcu_barrier = true;list_move (&s->list, release);
mm/slob.c +3 -3
@@ -126,7 +126,7 @@ static inline void clear_slob_page_free(struct page *sp)/* * struct slob_rcu is inserted at the tail of allocated slob blocks, which -* were created with a SLAB_DESTROY_BY_RCU slab. slob_rcu is used to free +* were created with a SLAB_TYPESAFE_BY_RCU slab. slob_rcu is used to free * the block using call_rcu. */ struct slob_rcu {
@@ -524,7 +524,7 @@ EXPORT_SYMBOL(ksize);int __kmem_cache_create (struct kmem_cache *c,unsigned long flags) { -if (flags &SLAB_DESTROY_BY_RCU ) { +if (flags &SLAB_TYPESAFE_BY_RCU ) {/* leave room for rcu footer at the end of object */ c->size +=sizeof (struct slob_rcu ); }
@@ -598,7 +598,7 @@ static void kmem_rcu_free(struct rcu_head *head)void kmem_cache_free (struct kmem_cache *c,void *b) {kmemleak_free_recursive (b, c->flags); -if (unlikely (c->flags &SLAB_DESTROY_BY_RCU )) { +if (unlikely (c->flags &SLAB_TYPESAFE_BY_RCU )) {struct slob_rcu *slob_rcu; slob_rcu = b + (c->size -sizeof (struct slob_rcu )); slob_rcu->size = c->size;
mm/slub.c +6 -6
@@ -1677,7 +1677,7 @@ static void rcu_free_slab(struct rcu_head *h)static void free_slab (struct kmem_cache *s,struct page *page) { -if (unlikely (s->flags &SLAB_DESTROY_BY_RCU )) { +if (unlikely (s->flags &SLAB_TYPESAFE_BY_RCU )) {struct rcu_head *head;if (need_reserve_slab_rcu) {
@@ -2953,7 +2953,7 @@ static __always_inline void slab_free(struct kmem_cache *s, struct page *page,* slab_free_freelist_hook() could have put the items into quarantine. * If so, no need to free them. */ -if (s->flags &SLAB_KASAN && !(s->flags &SLAB_DESTROY_BY_RCU )) +if (s->flags &SLAB_KASAN && !(s->flags &SLAB_TYPESAFE_BY_RCU ))return ;do_slab_free (s, page, head, tail, cnt, addr); }
@@ -3423,7 +3423,7 @@ static int calculate_sizes(struct kmem_cache *s, int forced_order)* the slab may touch the object after free or before allocation * then we should never poison the object itself. */ -if ((flags &SLAB_POISON ) && !(flags &SLAB_DESTROY_BY_RCU ) && +if ((flags &SLAB_POISON ) && !(flags &SLAB_TYPESAFE_BY_RCU ) && !s->ctor) s->flags |= __OBJECT_POISON;else
@@ -3445,7 +3445,7 @@ static int calculate_sizes(struct kmem_cache *s, int forced_order)*/ s->inuse = size; -if (((flags & (SLAB_DESTROY_BY_RCU |SLAB_POISON )) || +if (((flags & (SLAB_TYPESAFE_BY_RCU |SLAB_POISON )) || s->ctor)) {/* * Relocate free pointer after the object if it is not
@@ -3527,7 +3527,7 @@ static int kmem_cache_open(struct kmem_cache *s, unsigned long flags) s->flags =kmem_cache_flags (s->size, flags, s->name, s->ctor); s->reserved =0 ; -if (need_reserve_slab_rcu && (s->flags &SLAB_DESTROY_BY_RCU )) +if (need_reserve_slab_rcu && (s->flags &SLAB_TYPESAFE_BY_RCU )) s->reserved =sizeof (struct rcu_head );if (!calculate_sizes (s,-1 ))
@@ -4978,7 +4978,7 @@ SLAB_ATTR_RO(cache_dma);static ssize_t destroy_by_rcu_show (struct kmem_cache *s,char *buf) { -return sprintf (buf,"%d\n" , !!(s->flags &SLAB_DESTROY_BY_RCU )); +return sprintf (buf,"%d\n" , !!(s->flags &SLAB_TYPESAFE_BY_RCU )); }SLAB_ATTR_RO (destroy_by_rcu);
net/dccp/ipv4.c +1 -1
@@ -951,7 +951,7 @@ static struct proto dccp_v4_prot = { .orphan_count = &dccp_orphan_count, .max_header = MAX_DCCP_HEADER, .obj_size = sizeof(struct dccp_sock), - .slab_flags = SLAB_DESTROY_BY_RCU, + .slab_flags = SLAB_TYPESAFE_BY_RCU, .rsk_prot = &dccp_request_sock_ops, .twsk_prot = &dccp_timewait_sock_ops, .h.hashinfo = &dccp_hashinfo,
net/dccp/ipv6.c +1 -1
@@ -1014,7 +1014,7 @@ static struct proto dccp_v6_prot = { .orphan_count = &dccp_orphan_count, .max_header = MAX_DCCP_HEADER, .obj_size = sizeof(struct dccp6_sock), - .slab_flags = SLAB_DESTROY_BY_RCU, + .slab_flags = SLAB_TYPESAFE_BY_RCU, .rsk_prot = &dccp6_request_sock_ops, .twsk_prot = &dccp6_timewait_sock_ops, .h.hashinfo = &dccp_hashinfo,
net/ipv4/tcp_ipv4.c +1 -1
@@ -2394,7 +2394,7 @@ struct proto tcp_prot = { .sysctl_rmem = sysctl_tcp_rmem, .max_header = MAX_TCP_HEADER, .obj_size = sizeof(struct tcp_sock), - .slab_flags = SLAB_DESTROY_BY_RCU, + .slab_flags = SLAB_TYPESAFE_BY_RCU, .twsk_prot = &tcp_timewait_sock_ops, .rsk_prot = &tcp_request_sock_ops, .h.hashinfo = &tcp_hashinfo,
net/ipv6/tcp_ipv6.c +1 -1
@@ -1907,7 +1907,7 @@ struct proto tcpv6_prot = { .sysctl_rmem = sysctl_tcp_rmem, .max_header = MAX_TCP_HEADER, .obj_size = sizeof(struct tcp6_sock), - .slab_flags = SLAB_DESTROY_BY_RCU, + .slab_flags = SLAB_TYPESAFE_BY_RCU, .twsk_prot = &tcp6_timewait_sock_ops, .rsk_prot = &tcp6_request_sock_ops, .h.hashinfo = &tcp_hashinfo,
net/llc/af_llc.c +1 -1
@@ -140,7 +140,7 @@ static struct proto llc_proto = { .name = "LLC", .owner = THIS_MODULE, .obj_size = sizeof(struct llc_sock), - .slab_flags = SLAB_DESTROY_BY_RCU, + .slab_flags = SLAB_TYPESAFE_BY_RCU, };/**
net/llc/llc_conn.c +2 -2
@@ -506,7 +506,7 @@ static struct sock *__llc_lookup_established(struct llc_sap *sap, again:sk_nulls_for_each_rcu (rc, node, laddr_hb) {if (llc_estab_match (sap, daddr, laddr, rc)) { -/* Extra checks required by SLAB_DESTROY_BY_RCU */ +/* Extra checks required by SLAB_TYPESAFE_BY_RCU */ if (unlikely (!atomic_inc_not_zero (&rc->sk_refcnt))) goto again; if (unlikely(llc_sk(rc)->sap != sap ||
@@ -565,7 +565,7 @@ static struct sock *__llc_lookup_listener(struct llc_sap *sap, again:sk_nulls_for_each_rcu (rc, node, laddr_hb) {if (llc_listener_match (sap, laddr, rc)) { -/* Extra checks required by SLAB_DESTROY_BY_RCU */ +/* Extra checks required by SLAB_TYPESAFE_BY_RCU */ if (unlikely (!atomic_inc_not_zero (&rc->sk_refcnt))) goto again; if (unlikely(llc_sk(rc)->sap != sap ||
net/llc/llc_sap.c +1 -1
@@ -325,7 +325,7 @@ static struct sock *llc_lookup_dgram(struct llc_sap *sap, again:sk_nulls_for_each_rcu (rc, node, laddr_hb) {if (llc_dgram_match (sap, laddr, rc)) { -/* Extra checks required by SLAB_DESTROY_BY_RCU */ +/* Extra checks required by SLAB_TYPESAFE_BY_RCU */ if (unlikely (!atomic_inc_not_zero (&rc->sk_refcnt))) goto again; if (unlikely(llc_sk(rc)->sap != sap ||
net/netfilter/nf_conntrack_core.c +4 -4
@@ -895,7 +895,7 @@ static unsigned int early_drop_list(struct net *net,continue ;/* kill only if still in same netns -- might have moved due to -* SLAB_DESTROY_BY_RCU rules. +* SLAB_TYPESAFE_BY_RCU rules. * * We steal the timer reference. If that fails timer has * already fired or someone else deleted it. Just drop ref
@@ -1052,7 +1052,7 @@ __nf_conntrack_alloc(struct net *net,/* * Do not use kmem_cache_zalloc(), as this cache uses -* SLAB_DESTROY_BY_RCU. +* SLAB_TYPESAFE_BY_RCU. */ ct =kmem_cache_alloc (nf_conntrack_cachep, gfp);if (ct ==NULL )
@@ -1097,7 +1097,7 @@ void nf_conntrack_free(struct nf_conn *ct)struct net *net =nf_ct_net (ct);/* A freed object has refcnt == 0, that's -* the golden rule for SLAB_DESTROY_BY_RCU +* the golden rule for SLAB_TYPESAFE_BY_RCU */ NF_CT_ASSERT (atomic_read (&ct->ct_general.use) ==0 );
@@ -1863,7 +1863,7 @@ int nf_conntrack_init_start(void) nf_conntrack_cachep =kmem_cache_create ("nf_conntrack" ,sizeof (struct nf_conn ),0 , -SLAB_DESTROY_BY_RCU |SLAB_HWCACHE_ALIGN ,NULL ); +SLAB_TYPESAFE_BY_RCU |SLAB_HWCACHE_ALIGN ,NULL );if (!nf_conntrack_cachep) goto err_cachep;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 10 lines · Paul E. McKenney
A group of Linux kernel hackers reported chasing a bug that resulted from their assumption that
SLAB_DESTROY_BY_RCUprovided an existence guarantee, that is, that no block from such a slab would be reallocated during an RCU read-side critical section. Of course, that is not the case. Instead,SLAB_DESTROY_BY_RCUonly prevents freeing of an entire slab of blocks.However, there is a phrase for this, namely "type safety". This commit therefore renames
SLAB_DESTROY_BY_RCUtoSLAB_TYPESAFE_BY_RCUin order to avoid future instances of this sort of confusion.
This is probably the ultimate in bikeshedding, but RCU is not the thing which is providing the typesafety. Slab is providing the typesafety in order to help RCU. So would a better name not be 'SLAB_TYPESAFETY_FOR_RCU', or more succinctly 'SLAB_RCU_TYPESAFE'?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 8 lines · On Wed, Jan 18, 2017 at 03:12:01AM -0800, [email protected] wrote:
Quoted 7 lines · Paul E. McKenney
A group of Linux kernel hackers reported chasing a bug that resulted from their assumption that
SLAB_DESTROY_BY_RCUprovided an existence guarantee, that is, that no block from such a slab would be reallocated during an RCU read-side critical section. Of course, that is not the case. Instead,SLAB_DESTROY_BY_RCUonly prevents freeing of an entire slab of blocks.
Quoted 8 lines
Quoted 3 lines
However, there is a phrase for this, namely "type safety". This commit therefore renames
SLAB_DESTROY_BY_RCUtoSLAB_TYPESAFE_BY_RCUin order to avoid future instances of this sort of confusion.This is probably the ultimate in bikeshedding, but RCU is not the thing which is providing the typesafety. Slab is providing the typesafety in order to help RCU. So would a better name not be '
SLAB_TYPESAFETY_FOR_RCU', or more succinctly 'SLAB_RCU_TYPESAFE'?
Actually, slab is using RCU to provide type safety to those slab users who request it.
Thanx, Paul
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 2 lines · Paul E. McKenney
Actually, slab is using RCU to provide type safety to those slab users who request it.
Typesafety is a side effect. The main idea here is that the object can still be accessed in RCU sections after another processor frees the object. We guarantee that the object is not freed but it may be reused for another object within the RCU period.
Can we have a name that expresses all of that properly?
Quoted 11 lines · Christoph Lameter
Quoted 2 lines · Paul E. McKenney
Actually, slab is using RCU to provide type safety to those slab users who request it.
Typesafety is a side effect. The main idea here is that the object can still be accessed in RCU sections after another processor frees the object. We guarantee that the object is not freed but it may be reused for another object within the RCU period.
Can we have a name that expresses all of that properly?
But of course!!! "Type safety". http://wiki.c2.com/?TypeSafe
Thanx, Paul
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 6 lines · Paul E. McKenney
Quoted 5 lines · Christoph Lameter
Quoted 2 lines · Paul E. McKenney
Actually, slab is using RCU to provide type safety to those slab users who request it.
Quoted 8 lines
Quoted 6 lines
Typesafety is a side effect. The main idea here is that the object can still be accessed in RCU sections after another processor frees the object. We guarantee that the object is not freed but it may be reused for another object within the RCU period.
Can we have a name that expresses all of that properly?
But of course!!! "Type safety". http://wiki.c2.com/?TypeSafe
Well that does not convey the idea that RCU is involved here.
SLAB_DESTROY_RCU_TYPESAFE
?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 13 lines · Christoph Lameter
Quoted 11 lines · Paul E. McKenney
Quoted 10 lines · Christoph Lameter
Quoted 2 lines · Paul E. McKenney
Actually, slab is using RCU to provide type safety to those slab users who request it.
Typesafety is a side effect. The main idea here is that the object can still be accessed in RCU sections after another processor frees the object. We guarantee that the object is not freed but it may be reused for another object within the RCU period.
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
Can we have a name that expresses all of that properly?
But of course!!! "Type safety". http://wiki.c2.com/?TypeSafe
Well that does not convey the idea that RCU is involved here.
SLAB_DESTROY_RCU_TYPESAFE
Not clear why we need to change this very fine name ?
SLAB_DESTROY_BY_RCU was only used by few of us, we know damn well what it means.
Consider we wont be able to change it in various web pages / archives / changelogs.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 22 lines · Eric Dumazet
Quoted 13 lines · Christoph Lameter
Quoted 11 lines · Paul E. McKenney
Quoted 10 lines · Christoph Lameter
Quoted 2 lines · Paul E. McKenney
Actually, slab is using RCU to provide type safety to those slab users who request it.
Typesafety is a side effect. The main idea here is that the object can still be accessed in RCU sections after another processor frees the object. We guarantee that the object is not freed but it may be reused for another object within the RCU period.
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
Can we have a name that expresses all of that properly?
But of course!!! "Type safety". http://wiki.c2.com/?TypeSafe
Well that does not convey the idea that RCU is involved here.
SLAB_DESTROY_RCU_TYPESAFE
Quoted 7 lines
Not clear why we need to change this very fine name ?
SLAB_DESTROY_BY_RCUwas only used by few of us, we know damn well what it means.Consider we wont be able to change it in various web pages / archives / changelogs.
The reason I proposed this change is that I ran into some people last week who had burned some months learning that this very fine flag provides only type safety, not identity safety.
Other proposals?
Thanx, Paul
Quoted 8 lines · Paul E. McKenney
Quoted 6 lines · Eric Dumazet
Not clear why we need to change this very fine name ?
SLAB_DESTROY_BY_RCUwas only used by few of us, we know damn well what it means.
Quoted 8 lines
Quoted 2 lines
Consider we wont be able to change it in various web pages / archives / changelogs.
The reason I proposed this change is that I ran into some people last week who had burned some months learning that this very fine flag provides only type safety, not identity safety.
Other proposals?
Bung hunting requires scrapping git log, in particular in areas touching RCU (not exactly trivial stuff :) :) :) )
git log | grep SLAB_DESTROY_BY_RCU
A Google search on SLAB_DESTROY_BY_RCU finds ~4600 results.
If we rename SLAB_DESTROY_BY_RCU by XXXXXXXX, someone trying to see prior patches or articles about XXXXXXXX mistakes will find nothing.
So please leave comments giving the old name ( SLAB_DESTROY_BY_RCU ) and save time for future hackers.
Thanks.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 23 lines · Eric Dumazet
Quoted 8 lines · Paul E. McKenney
Quoted 6 lines · Eric Dumazet
Not clear why we need to change this very fine name ?
SLAB_DESTROY_BY_RCUwas only used by few of us, we know damn well what it means.Quoted 8 lines
Quoted 2 lines
Consider we wont be able to change it in various web pages / archives / changelogs.
The reason I proposed this change is that I ran into some people last week who had burned some months learning that this very fine flag provides only type safety, not identity safety.
Other proposals?
Bung hunting requires scrapping git log, in particular in areas touching RCU (not exactly trivial stuff :) :) :) )
git log | grep
SLAB_DESTROY_BY_RCU
Quoted 7 lines
A Google search on
SLAB_DESTROY_BY_RCUfinds ~4600 results.If we rename
SLAB_DESTROY_BY_RCUby XXXXXXXX, someone trying to see prior patches or articles about XXXXXXXX mistakes will find nothing.So please leave comments giving the old name (
SLAB_DESTROY_BY_RCU) and save time for future hackers.
Good point! How about the following diff against the old patch?
------------------------------------------------------------------------
Documentation/RCU/whatisRCU.txt +2 -1
@@ -925,7 +925,8 @@ d. Do you need RCU grace periods to complete even in the face e. Is your workload too update-intensive for normal use of RCU, but inappropriate for other synchronization mechanisms? - If so, consider SLAB_TYPESAFE_BY_RCU. But please be careful! + If so, consider SLAB_TYPESAFE_BY_RCU (which was originally + named SLAB_DESTROY_BY_RCU). But please be careful! f. Do you need read-side critical sections that are respected even though they are in the middle of the idle loop, during
include/linux/slab.h +2 -0
@@ -61,6 +61,8 @@* * rcu_read_lock before reading the address, then rcu_read_unlock after * taking the spinlock within the structure expected at that address. +* +* Note that SLAB_TYPESAFE_BY_RCU was originally named SLAB_DESTROY_BY_RCU. */ #define SLAB_TYPESAFE_BY_RCU 0x00080000UL/* Defer freeing slabs to RCU */ #define SLAB_MEM_SPREAD 0x00100000UL/* Spread some memory over cpuset */
------------------------------------------------------------------------
Any other "tombstone" comments needed?
The full patch is shown below for completeness.
Thanx, Paul
------------------------------------------------------------------------
commit 55f619e5c322171698f0d5aecf5009391ac7e2db
Author: Paul E. McKenney <[email protected]> Date: Wed Jan 18 02:53:44 2017 -0800
mm: Rename SLAB_DESTROY_BY_RCU to SLAB_TYPESAFE_BY_RCU
A group of Linux kernel hackers reported chasing a bug that resulted from their assumption that SLAB_DESTROY_BY_RCU provided an existence guarantee, that is, that no block from such a slab would be reallocated during an RCU read-side critical section. Of course, that is not the case. Instead, SLAB_DESTROY_BY_RCU only prevents freeing of an entire slab of blocks.
However, there is a phrase for this, namely "type safety". This commit therefore renames SLAB_DESTROY_BY_RCU to SLAB_TYPESAFE_BY_RCU in order to avoid future instances of this sort of confusion.
Documentation/RCU/00-INDEX +1 -1
@@ -17,7 +17,7 @@ rcu_dereference.txt rcubarrier.txt - RCU and Unloadable Modules rculist_nulls.txt - - RCU list primitives for use with SLAB_DESTROY_BY_RCU + - RCU list primitives for use with SLAB_TYPESAFE_BY_RCU rcuref.txt - Reference-count design for elements of lists/arrays protected by RCU rcu.txt
Documentation/RCU/rculist_nulls.txt +3 -3
@@ -1,5 +1,5 @@ Using hlist_nulls to protect read-mostly linked lists and -objects using SLAB_DESTROY_BY_RCU allocations. +objects using SLAB_TYPESAFE_BY_RCU allocations. Please read the basics in Documentation/RCU/listRCU.txt
@@ -7,7 +7,7 @@ Using special makers (called 'nulls') is a convenient way to solve following problem : A typical RCU linked list managing objects which are -allocated with SLAB_DESTROY_BY_RCU kmem_cache can +allocated with SLAB_TYPESAFE_BY_RCU kmem_cache can use following algos : 1) Lookup algo
@@ -96,7 +96,7 @@ unlock_chain(); // typically a spin_unlock() 3) Remove algo -------------- Nothing special here, we can use a standard RCU hlist deletion. -But thanks to SLAB_DESTROY_BY_RCU, beware a deleted object can be reused +But thanks to SLAB_TYPESAFE_BY_RCU, beware a deleted object can be reused very very fast (before the end of RCU grace period) if (put_last_reference_on(obj) {
Documentation/RCU/whatisRCU.txt +2 -1
@@ -925,7 +925,8 @@ d. Do you need RCU grace periods to complete even in the face e. Is your workload too update-intensive for normal use of RCU, but inappropriate for other synchronization mechanisms? - If so, consider SLAB_DESTROY_BY_RCU. But please be careful! + If so, consider SLAB_TYPESAFE_BY_RCU (which was originally + named SLAB_DESTROY_BY_RCU). But please be careful! f. Do you need read-side critical sections that are respected even though they are in the middle of the idle loop, during
drivers/gpu/drm/i915/i915_gem.c +1 -1
@@ -4541,7 +4541,7 @@ i915_gem_load_init(struct drm_device *dev) dev_priv->requests =KMEM_CACHE (drm_i915_gem_request,SLAB_HWCACHE_ALIGN |SLAB_RECLAIM_ACCOUNT | -SLAB_DESTROY_BY_RCU ); +SLAB_TYPESAFE_BY_RCU );if (!dev_priv->requests) goto err_vmas;
drivers/gpu/drm/i915/i915_gem_request.h +1 -1
@@ -504,7 +504,7 @@ static inline struct drm_i915_gem_request * __i915_gem_active_get_rcu(const struct i915_gem_active *active) {/* Performing a lockless retrieval of the active request is super -* tricky. SLAB_DESTROY_BY_RCU merely guarantees that the backing +* tricky. SLAB_TYPESAFE_BY_RCU merely guarantees that the backing * slab of request objects will not be freed whilst we hold the * RCU read lock. It does not guarantee that the request itself * will not be freed and then *reused*. Viz,
drivers/staging/lustre/lustre/ldlm/ldlm_lockd.c +1 -1
@@ -1071,7 +1071,7 @@ int ldlm_init(void) ldlm_lock_slab =kmem_cache_create ("ldlm_locks" ,sizeof (struct ldlm_lock ),0 ,SLAB_HWCACHE_ALIGN | -SLAB_DESTROY_BY_RCU ,NULL ); +SLAB_TYPESAFE_BY_RCU ,NULL );if (!ldlm_lock_slab) {kmem_cache_destroy (ldlm_resource_slab);return -ENOMEM ;
fs/jbd2/journal.c +1 -1
@@ -2340,7 +2340,7 @@ static int jbd2_journal_init_journal_head_cache(void) jbd2_journal_head_cache =kmem_cache_create ("jbd2_journal_head" ,sizeof (struct journal_head ),0 ,/* offset */ -SLAB_TEMPORARY |SLAB_DESTROY_BY_RCU , +SLAB_TEMPORARY |SLAB_TYPESAFE_BY_RCU ,NULL );/* ctor */ retval =0 ;if (!jbd2_journal_head_cache) {
fs/signalfd.c +1 -1
@@ -38,7 +38,7 @@ void signalfd_cleanup(struct sighand_struct *sighand)/* * The lockless check can race with remove_wait_queue() in progress, * but in this case its caller should run under rcu_read_lock() and -* sighand_cachep is SLAB_DESTROY_BY_RCU, we can safely return. +* sighand_cachep is SLAB_TYPESAFE_BY_RCU, we can safely return. */ if (likely (!waitqueue_active (wqh)))return ;
include/linux/dma-fence.h +2 -2
@@ -229,7 +229,7 @@ static inline struct dma_fence *dma_fence_get_rcu(struct dma_fence *fence)* * Function returns NULL if no refcount could be obtained, or the fence. * This function handles acquiring a reference to a fence that may be -* reallocated within the RCU grace period (such as with SLAB_DESTROY_BY_RCU), +* reallocated within the RCU grace period (such as with SLAB_TYPESAFE_BY_RCU), * so long as the caller is using RCU on the pointer to the fence. * * An alternative mechanism is to employ a seqlock to protect a bunch of
@@ -257,7 +257,7 @@ dma_fence_get_rcu_safe(struct dma_fence * __rcu *fencep)* have successfully acquire a reference to it. If it no * longer matches, we are holding a reference to some other * reallocated pointer. This is possible if the allocator -* is using a freelist like SLAB_DESTROY_BY_RCU where the +* is using a freelist like SLAB_TYPESAFE_BY_RCU where the * fence remains valid for the RCU grace period, but it * may be reallocated. When using such allocators, we are * responsible for ensuring the reference we get is to
include/linux/slab.h +4 -2
@@ -28,7 +28,7 @@#define SLAB_STORE_USER 0x00010000UL/* DEBUG: Store the last owner for bug hunting */ #define SLAB_PANIC 0x00040000UL/* Panic if kmem_cache_create() fails */ /* -* SLAB_DESTROY_BY_RCU - **WARNING** READ THIS! +* SLAB_TYPESAFE_BY_RCU - **WARNING** READ THIS! * * This delays freeing the SLAB page by a grace period, it does _NOT_ * delay object freeing. This means that if you do kmem_cache_free()
@@ -61,8 +61,10 @@* * rcu_read_lock before reading the address, then rcu_read_unlock after * taking the spinlock within the structure expected at that address. +* +* Note that SLAB_TYPESAFE_BY_RCU was originally named SLAB_DESTROY_BY_RCU. */ -#define SLAB_DESTROY_BY_RCU 0x00080000UL/* Defer freeing slabs to RCU */ +#define SLAB_TYPESAFE_BY_RCU 0x00080000UL/* Defer freeing slabs to RCU */ #define SLAB_MEM_SPREAD 0x00100000UL/* Spread some memory over cpuset */ #define SLAB_TRACE 0x00200000UL/* Trace allocations and frees */
include/net/sock.h +1 -1
@@ -989,7 +989,7 @@ struct raw_hashinfo;struct module ;/* -* caches using SLAB_DESTROY_BY_RCU should let .next pointer from nulls nodes +* caches using SLAB_TYPESAFE_BY_RCU should let .next pointer from nulls nodes * un-modified. Special care is taken when initializing object to zero. */ static inline void sk_prot_clear_nulls (struct sock *sk,int size)
kernel/fork.c +2 -2
@@ -1297,7 +1297,7 @@ void __cleanup_sighand(struct sighand_struct *sighand)if (atomic_dec_and_test (&sighand->count)) {signalfd_cleanup (sighand);/* -* sighand_cachep is SLAB_DESTROY_BY_RCU so we can free it +* sighand_cachep is SLAB_TYPESAFE_BY_RCU so we can free it * without an RCU grace period, see __lock_task_sighand(). */ kmem_cache_free (sighand_cachep, sighand);
@@ -2069,7 +2069,7 @@ void __init proc_caches_init(void) { sighand_cachep =kmem_cache_create ("sighand_cache" ,sizeof (struct sighand_struct ),0 , -SLAB_HWCACHE_ALIGN |SLAB_PANIC |SLAB_DESTROY_BY_RCU | +SLAB_HWCACHE_ALIGN |SLAB_PANIC |SLAB_TYPESAFE_BY_RCU |SLAB_NOTRACK |SLAB_ACCOUNT , sighand_ctor); signal_cachep = kmem_cache_create("signal_cache", sizeof(struct signal_struct), 0,
kernel/signal.c +1 -1
@@ -1232,7 +1232,7 @@ struct sighand_struct *__lock_task_sighand(struct task_struct *tsk, }/* * This sighand can be already freed and even reused, but -* we rely on SLAB_DESTROY_BY_RCU and sighand_ctor() which +* we rely on SLAB_TYPESAFE_BY_RCU and sighand_ctor() which * initializes ->siglock: this slab can't go away, it has * the same object type, ->siglock can't be reinitialized. *
mm/kasan/kasan.c +3 -3
@@ -402,7 +402,7 @@ void kasan_cache_create(struct kmem_cache *cache, size_t *size, *size +=sizeof (struct kasan_alloc_meta );/* Add free meta. */ -if (cache->flags &SLAB_DESTROY_BY_RCU || cache->ctor || +if (cache->flags &SLAB_TYPESAFE_BY_RCU || cache->ctor || cache->object_size <sizeof (struct kasan_free_meta )) { cache->kasan_info.free_meta_offset = *size; *size +=sizeof (struct kasan_free_meta );
@@ -550,7 +550,7 @@ static void kasan_poison_slab_free(struct kmem_cache *cache, void *object)unsigned long rounded_up_size =round_up (size,KASAN_SHADOW_SCALE_SIZE );/* RCU slabs could be legally used after free within the RCU period */ -if (unlikely (cache->flags &SLAB_DESTROY_BY_RCU )) +if (unlikely (cache->flags &SLAB_TYPESAFE_BY_RCU ))return ;kasan_poison_shadow (object, rounded_up_size,KASAN_KMALLOC_FREE );
@@ -561,7 +561,7 @@ bool kasan_slab_free(struct kmem_cache *cache, void *object)s8 shadow_byte;/* RCU slabs could be legally used after free within the RCU period */ -if (unlikely (cache->flags &SLAB_DESTROY_BY_RCU )) +if (unlikely (cache->flags &SLAB_TYPESAFE_BY_RCU ))return false; shadow_byte =READ_ONCE (*(s8 *)kasan_mem_to_shadow (object));
mm/kmemcheck.c +1 -1
@@ -95,7 +95,7 @@ void kmemcheck_slab_alloc(struct kmem_cache *s, gfp_t gfpflags, void *object,void kmemcheck_slab_free (struct kmem_cache *s,void *object,size_t size) {/* TODO: RCU freeing is unsupported for now; hide false positives. */ -if (!s->ctor && !(s->flags &SLAB_DESTROY_BY_RCU )) +if (!s->ctor && !(s->flags &SLAB_TYPESAFE_BY_RCU ))kmemcheck_mark_freed (object, size); }
mm/rmap.c +2 -2
@@ -428,7 +428,7 @@ static void anon_vma_ctor(void *data)void __init anon_vma_init (void ) { anon_vma_cachep =kmem_cache_create ("anon_vma" ,sizeof (struct anon_vma ), -0 ,SLAB_DESTROY_BY_RCU |SLAB_PANIC |SLAB_ACCOUNT , +0 ,SLAB_TYPESAFE_BY_RCU |SLAB_PANIC |SLAB_ACCOUNT , anon_vma_ctor); anon_vma_chain_cachep =KMEM_CACHE (anon_vma_chain,SLAB_PANIC |SLAB_ACCOUNT );
@@ -479,7 +479,7 @@ struct anon_vma *page_get_anon_vma(struct page *page)* If this page is still mapped, then its anon_vma cannot have been * freed. But if it has been unmapped, we have no security against the * anon_vma structure being freed and reused (for another anon_vma: -* SLAB_DESTROY_BY_RCU guarantees that - so the atomic_inc_not_zero() +* SLAB_TYPESAFE_BY_RCU guarantees that - so the atomic_inc_not_zero() * above cannot corrupt). */ if (!page_mapped (page)) {
mm/slab.c +3 -3
@@ -1726,7 +1726,7 @@ static void slab_destroy(struct kmem_cache *cachep, struct page *page) freelist = page->freelist;slab_destroy_debugcheck (cachep, page); -if (unlikely (cachep->flags &SLAB_DESTROY_BY_RCU )) +if (unlikely (cachep->flags &SLAB_TYPESAFE_BY_RCU ))call_rcu (&page->rcu_head, kmem_rcu_free);else kmem_freepages (cachep, page);
@@ -1922,7 +1922,7 @@ static bool set_objfreelist_slab_cache(struct kmem_cache *cachep, cachep->num =0 ; -if (cachep->ctor || flags &SLAB_DESTROY_BY_RCU ) +if (cachep->ctor || flags &SLAB_TYPESAFE_BY_RCU )return false; left = calculate_slab_order(cachep, size,
@@ -2028,7 +2028,7 @@ __kmem_cache_create (struct kmem_cache *cachep, unsigned long flags)if (size <4096 ||fls (size -1 ) ==fls (size-1 +REDZONE_ALIGN +2 *sizeof (unsigned long long ))) flags |=SLAB_RED_ZONE |SLAB_STORE_USER ; -if (!(flags &SLAB_DESTROY_BY_RCU )) +if (!(flags &SLAB_TYPESAFE_BY_RCU )) flags |=SLAB_POISON ;#endif #endif
mm/slab.h +2 -2
@@ -120,7 +120,7 @@ static inline unsigned long kmem_cache_flags(unsigned long object_size,/* Legal flag mask for kmem_cache_create(), for various configurations */ #define SLAB_CORE_FLAGS (SLAB_HWCACHE_ALIGN | SLAB_CACHE_DMA | SLAB_PANIC | \ - SLAB_DESTROY_BY_RCU | SLAB_DEBUG_OBJECTS ) + SLAB_TYPESAFE_BY_RCU | SLAB_DEBUG_OBJECTS )#if defined(CONFIG_DEBUG_SLAB )#define SLAB_DEBUG_FLAGS (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER)
@@ -391,7 +391,7 @@ static inline size_t slab_ksize(const struct kmem_cache *s)* back there or track user information then we can * only use the space before that information. */ -if (s->flags & (SLAB_DESTROY_BY_RCU |SLAB_STORE_USER )) +if (s->flags & (SLAB_TYPESAFE_BY_RCU |SLAB_STORE_USER ))return s->inuse;/* * Else we can use all the padding etc for the allocation
mm/slab_common.c +2 -2
@@ -34,7 +34,7 @@ struct kmem_cache *kmem_cache;* Set of flags that will prevent slab merging */ #define SLAB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \ - SLAB_TRACE | SLAB_DESTROY_BY_RCU | SLAB_NOLEAKTRACE | \ + SLAB_TRACE | SLAB_TYPESAFE_BY_RCU | SLAB_NOLEAKTRACE | \ SLAB_FAILSLAB | SLAB_KASAN)#define SLAB_MERGE_SAME (SLAB_RECLAIM_ACCOUNT | SLAB_CACHE_DMA | \
@@ -464,7 +464,7 @@ static int shutdown_cache(struct kmem_cache *s,if (__kmem_cache_shutdown (s) !=0 )return -EBUSY ; -if (s->flags &SLAB_DESTROY_BY_RCU ) +if (s->flags &SLAB_TYPESAFE_BY_RCU ) *need_rcu_barrier = true;list_move (&s->list, release);
mm/slob.c +3 -3
@@ -126,7 +126,7 @@ static inline void clear_slob_page_free(struct page *sp)/* * struct slob_rcu is inserted at the tail of allocated slob blocks, which -* were created with a SLAB_DESTROY_BY_RCU slab. slob_rcu is used to free +* were created with a SLAB_TYPESAFE_BY_RCU slab. slob_rcu is used to free * the block using call_rcu. */ struct slob_rcu {
@@ -524,7 +524,7 @@ EXPORT_SYMBOL(ksize);int __kmem_cache_create (struct kmem_cache *c,unsigned long flags) { -if (flags &SLAB_DESTROY_BY_RCU ) { +if (flags &SLAB_TYPESAFE_BY_RCU ) {/* leave room for rcu footer at the end of object */ c->size +=sizeof (struct slob_rcu ); }
@@ -598,7 +598,7 @@ static void kmem_rcu_free(struct rcu_head *head)void kmem_cache_free (struct kmem_cache *c,void *b) {kmemleak_free_recursive (b, c->flags); -if (unlikely (c->flags &SLAB_DESTROY_BY_RCU )) { +if (unlikely (c->flags &SLAB_TYPESAFE_BY_RCU )) {struct slob_rcu *slob_rcu; slob_rcu = b + (c->size -sizeof (struct slob_rcu )); slob_rcu->size = c->size;
mm/slub.c +6 -6
@@ -1677,7 +1677,7 @@ static void rcu_free_slab(struct rcu_head *h)static void free_slab (struct kmem_cache *s,struct page *page) { -if (unlikely (s->flags &SLAB_DESTROY_BY_RCU )) { +if (unlikely (s->flags &SLAB_TYPESAFE_BY_RCU )) {struct rcu_head *head;if (need_reserve_slab_rcu) {
@@ -2953,7 +2953,7 @@ static __always_inline void slab_free(struct kmem_cache *s, struct page *page,* slab_free_freelist_hook() could have put the items into quarantine. * If so, no need to free them. */ -if (s->flags &SLAB_KASAN && !(s->flags &SLAB_DESTROY_BY_RCU )) +if (s->flags &SLAB_KASAN && !(s->flags &SLAB_TYPESAFE_BY_RCU ))return ;do_slab_free (s, page, head, tail, cnt, addr); }
@@ -3423,7 +3423,7 @@ static int calculate_sizes(struct kmem_cache *s, int forced_order)* the slab may touch the object after free or before allocation * then we should never poison the object itself. */ -if ((flags &SLAB_POISON ) && !(flags &SLAB_DESTROY_BY_RCU ) && +if ((flags &SLAB_POISON ) && !(flags &SLAB_TYPESAFE_BY_RCU ) && !s->ctor) s->flags |= __OBJECT_POISON;else
@@ -3445,7 +3445,7 @@ static int calculate_sizes(struct kmem_cache *s, int forced_order)*/ s->inuse = size; -if (((flags & (SLAB_DESTROY_BY_RCU |SLAB_POISON )) || +if (((flags & (SLAB_TYPESAFE_BY_RCU |SLAB_POISON )) || s->ctor)) {/* * Relocate free pointer after the object if it is not
@@ -3527,7 +3527,7 @@ static int kmem_cache_open(struct kmem_cache *s, unsigned long flags) s->flags =kmem_cache_flags (s->size, flags, s->name, s->ctor); s->reserved =0 ; -if (need_reserve_slab_rcu && (s->flags &SLAB_DESTROY_BY_RCU )) +if (need_reserve_slab_rcu && (s->flags &SLAB_TYPESAFE_BY_RCU )) s->reserved =sizeof (struct rcu_head );if (!calculate_sizes (s,-1 ))
@@ -4978,7 +4978,7 @@ SLAB_ATTR_RO(cache_dma);static ssize_t destroy_by_rcu_show (struct kmem_cache *s,char *buf) { -return sprintf (buf,"%d\n" , !!(s->flags &SLAB_DESTROY_BY_RCU )); +return sprintf (buf,"%d\n" , !!(s->flags &SLAB_TYPESAFE_BY_RCU )); }SLAB_ATTR_RO (destroy_by_rcu);
net/dccp/ipv4.c +1 -1
@@ -951,7 +951,7 @@ static struct proto dccp_v4_prot = { .orphan_count = &dccp_orphan_count, .max_header = MAX_DCCP_HEADER, .obj_size = sizeof(struct dccp_sock), - .slab_flags = SLAB_DESTROY_BY_RCU, + .slab_flags = SLAB_TYPESAFE_BY_RCU, .rsk_prot = &dccp_request_sock_ops, .twsk_prot = &dccp_timewait_sock_ops, .h.hashinfo = &dccp_hashinfo,
net/dccp/ipv6.c +1 -1
@@ -1014,7 +1014,7 @@ static struct proto dccp_v6_prot = { .orphan_count = &dccp_orphan_count, .max_header = MAX_DCCP_HEADER, .obj_size = sizeof(struct dccp6_sock), - .slab_flags = SLAB_DESTROY_BY_RCU, + .slab_flags = SLAB_TYPESAFE_BY_RCU, .rsk_prot = &dccp6_request_sock_ops, .twsk_prot = &dccp6_timewait_sock_ops, .h.hashinfo = &dccp_hashinfo,
net/ipv4/tcp_ipv4.c +1 -1
@@ -2394,7 +2394,7 @@ struct proto tcp_prot = { .sysctl_rmem = sysctl_tcp_rmem, .max_header = MAX_TCP_HEADER, .obj_size = sizeof(struct tcp_sock), - .slab_flags = SLAB_DESTROY_BY_RCU, + .slab_flags = SLAB_TYPESAFE_BY_RCU, .twsk_prot = &tcp_timewait_sock_ops, .rsk_prot = &tcp_request_sock_ops, .h.hashinfo = &tcp_hashinfo,
net/ipv6/tcp_ipv6.c +1 -1
@@ -1907,7 +1907,7 @@ struct proto tcpv6_prot = { .sysctl_rmem = sysctl_tcp_rmem, .max_header = MAX_TCP_HEADER, .obj_size = sizeof(struct tcp6_sock), - .slab_flags = SLAB_DESTROY_BY_RCU, + .slab_flags = SLAB_TYPESAFE_BY_RCU, .twsk_prot = &tcp6_timewait_sock_ops, .rsk_prot = &tcp6_request_sock_ops, .h.hashinfo = &tcp_hashinfo,
net/llc/af_llc.c +1 -1
@@ -140,7 +140,7 @@ static struct proto llc_proto = { .name = "LLC", .owner = THIS_MODULE, .obj_size = sizeof(struct llc_sock), - .slab_flags = SLAB_DESTROY_BY_RCU, + .slab_flags = SLAB_TYPESAFE_BY_RCU, };/**
net/llc/llc_conn.c +2 -2
@@ -506,7 +506,7 @@ static struct sock *__llc_lookup_established(struct llc_sap *sap, again:sk_nulls_for_each_rcu (rc, node, laddr_hb) {if (llc_estab_match (sap, daddr, laddr, rc)) { -/* Extra checks required by SLAB_DESTROY_BY_RCU */ +/* Extra checks required by SLAB_TYPESAFE_BY_RCU */ if (unlikely (!atomic_inc_not_zero (&rc->sk_refcnt))) goto again; if (unlikely(llc_sk(rc)->sap != sap ||
@@ -565,7 +565,7 @@ static struct sock *__llc_lookup_listener(struct llc_sap *sap, again:sk_nulls_for_each_rcu (rc, node, laddr_hb) {if (llc_listener_match (sap, laddr, rc)) { -/* Extra checks required by SLAB_DESTROY_BY_RCU */ +/* Extra checks required by SLAB_TYPESAFE_BY_RCU */ if (unlikely (!atomic_inc_not_zero (&rc->sk_refcnt))) goto again; if (unlikely(llc_sk(rc)->sap != sap ||
net/llc/llc_sap.c +1 -1
@@ -325,7 +325,7 @@ static struct sock *llc_lookup_dgram(struct llc_sap *sap, again:sk_nulls_for_each_rcu (rc, node, laddr_hb) {if (llc_dgram_match (sap, laddr, rc)) { -/* Extra checks required by SLAB_DESTROY_BY_RCU */ +/* Extra checks required by SLAB_TYPESAFE_BY_RCU */ if (unlikely (!atomic_inc_not_zero (&rc->sk_refcnt))) goto again; if (unlikely(llc_sk(rc)->sap != sap ||
net/netfilter/nf_conntrack_core.c +4 -4
@@ -895,7 +895,7 @@ static unsigned int early_drop_list(struct net *net,continue ;/* kill only if still in same netns -- might have moved due to -* SLAB_DESTROY_BY_RCU rules. +* SLAB_TYPESAFE_BY_RCU rules. * * We steal the timer reference. If that fails timer has * already fired or someone else deleted it. Just drop ref
@@ -1052,7 +1052,7 @@ __nf_conntrack_alloc(struct net *net,/* * Do not use kmem_cache_zalloc(), as this cache uses -* SLAB_DESTROY_BY_RCU. +* SLAB_TYPESAFE_BY_RCU. */ ct =kmem_cache_alloc (nf_conntrack_cachep, gfp);if (ct ==NULL )
@@ -1097,7 +1097,7 @@ void nf_conntrack_free(struct nf_conn *ct)struct net *net =nf_ct_net (ct);/* A freed object has refcnt == 0, that's -* the golden rule for SLAB_DESTROY_BY_RCU +* the golden rule for SLAB_TYPESAFE_BY_RCU */ NF_CT_ASSERT (atomic_read (&ct->ct_general.use) ==0 );
@@ -1863,7 +1863,7 @@ int nf_conntrack_init_start(void) nf_conntrack_cachep =kmem_cache_create ("nf_conntrack" ,sizeof (struct nf_conn ),0 , -SLAB_DESTROY_BY_RCU |SLAB_HWCACHE_ALIGN ,NULL ); +SLAB_TYPESAFE_BY_RCU |SLAB_HWCACHE_ALIGN ,NULL );if (!nf_conntrack_cachep) goto err_cachep;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 7 lines · Paul E. McKenney
A group of Linux kernel hackers reported chasing a bug that resulted from their assumption that
SLAB_DESTROY_BY_RCUprovided an existence guarantee, that is, that no block from such a slab would be reallocated during an RCU read-side critical section. Of course, that is not the case. Instead,SLAB_DESTROY_BY_RCUonly prevents freeing of an entire slab of blocks.
Quoted 5 lines
However, there is a phrase for this, namely "type safety". This commit therefore renames
SLAB_DESTROY_BY_RCUtoSLAB_TYPESAFE_BY_RCUin order to avoid future instances of this sort of confusion.Signed-off-by Paul E. McKenney <[email protected]>
This has come up in the past, and it always proved hard to agree on a better name for it. But I like SLAB_TYPESAFE_BY_RCU the best out of all proposals, and it's much more poignant than the current name.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Vlastimil Babka
Acked-byQuoted 12 lines · Johannes Weiner
Quoted 11 lines · Paul E. McKenney
A group of Linux kernel hackers reported chasing a bug that resulted from their assumption that
SLAB_DESTROY_BY_RCUprovided an existence guarantee, that is, that no block from such a slab would be reallocated during an RCU read-side critical section. Of course, that is not the case. Instead,SLAB_DESTROY_BY_RCUonly prevents freeing of an entire slab of blocks.However, there is a phrase for this, namely "type safety". This commit therefore renames
SLAB_DESTROY_BY_RCUtoSLAB_TYPESAFE_BY_RCUin order to avoid future instances of this sort of confusion.
Quoted 5 lines
Quoted 1 line
Signed-off-by Paul E. McKenney <[email protected]>This has come up in the past, and it always proved hard to agree on a better name for it. But I like
SLAB_TYPESAFE_BY_RCUthe best out of all proposals, and it's much more poignant than the current name.
Heh, until I've seen this thread I had the same wrong assumption about the flag, so it suprised me. Good thing I didn't have a chance to use it wrongly so far :)
"Type safety" in this context seems quite counter-intuitive for me, as I've only heard it to describe programming languages. But that's fine when the name sounds so exotic that one has to look up what it does. Much safer than when the meaning seems obvious, but in fact it's misleading.
Quoted 8 lines
Acked-by Johannes Weiner <[email protected]>--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 23 lines · Vlastimil Babka
Quoted 12 lines · Johannes Weiner
Quoted 11 lines · Paul E. McKenney
A group of Linux kernel hackers reported chasing a bug that resulted from their assumption that
SLAB_DESTROY_BY_RCUprovided an existence guarantee, that is, that no block from such a slab would be reallocated during an RCU read-side critical section. Of course, that is not the case. Instead,SLAB_DESTROY_BY_RCUonly prevents freeing of an entire slab of blocks.However, there is a phrase for this, namely "type safety". This commit therefore renames
SLAB_DESTROY_BY_RCUtoSLAB_TYPESAFE_BY_RCUin order to avoid future instances of this sort of confusion.Quoted 5 lines
Quoted 1 line
Signed-off-by Paul E. McKenney <[email protected]>This has come up in the past, and it always proved hard to agree on a better name for it. But I like
SLAB_TYPESAFE_BY_RCUthe best out of all proposals, and it's much more poignant than the current name.Heh, until I've seen this thread I had the same wrong assumption about the flag, so it suprised me. Good thing I didn't have a chance to use it wrongly so far :)
Quoted 9 lines
"Type safety" in this context seems quite counter-intuitive for me, as I've only heard it to describe programming languages. But that's fine when the name sounds so exotic that one has to look up what it does. Much safer than when the meaning seems obvious, but in fact it's misleading.
Acked-by Vlastimil Babka <[email protected]>Quoted 1 line
Acked-by Johannes Weiner <[email protected]>
Thank you both! I have added these, as well as a Not-acked-by for Eric. ;-)
Thanx, Paul
Quoted 7 lines
Quoted 6 lines
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>