Skip to contentSkip to conversation

slub: introduce __kfree_rcu

PATCH In review Lai Jiangshan 4 participants 6 messages Mar 3, 2009 – Mar 23, 2009

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 3/6 slub: introduce __kfree_rcu

Introduce __kfree_rcu() for kfree_rcu()

It like __kfree_rcu() in slab.c.
We can calculate the object poiter from a poiter inside this object in slub.c, so we can use it for rcu callback and free the object.

---

mm/slub.c +24 -0
index 0280eee..b438ec5 100644
@@ -2756,6 +2756,30 @@ void kfree(const void *x)
 }
 EXPORT_SYMBOL(kfree);
 
+static void kfree_rcu_callback(struct rcu_head *rcu)
+{
+	void *portion = rcu;
+	struct page *page = virt_to_head_page(portion);
+
+	if (unlikely(!PageSlab(page))) {
+		BUG_ON(!PageCompound(page));
+		put_page(page);
+	} else {
+		struct kmem_cache *s = page->slab;
+		void *start = page_address(page);
+		unsigned int index = (portion - start) / s->size;
+		void *object = start + s->size * index;
+
+		slab_free(s, page, object, _RET_IP_);
+	}
+}
+
+void __kfree_rcu(const void *x, struct rcu_head *rcu)
+{
+	call_rcu(rcu, kfree_rcu_callback);
+}
+EXPORT_SYMBOL(__kfree_rcu);
+
 /*
  * kmem_cache_shrink removes empty slabs from the partial lists and sorts
  * the remaining slabs by the number of items in use. The slabs with the
#1 lore raw

mm/slub.c void kfree(const void *x) · line 2782

Quoted 7 lines · Lai Jiangshan
mm/slub.c +7 -0
+
+void __kfree_rcu(const void *x, struct rcu_head *rcu)
+{
+	call_rcu(rcu, kfree_rcu_callback);
+}
+EXPORT_SYMBOL(__kfree_rcu);
+

The parameter x is unused why require it? I see that SLOB needs it. Cant you do a similar trick as in SLUB just calculating the start address of the object from the rcu address?

#2 lore raw
Quoted 14 lines · Christoph Lameter

mm/slub.c void kfree(const void *x) · line 2782

Quoted 7 lines · Lai Jiangshan
mm/slub.c +7 -0
+
+void __kfree_rcu(const void *x, struct rcu_head *rcu)
+{
+	call_rcu(rcu, kfree_rcu_callback);
+}
+EXPORT_SYMBOL(__kfree_rcu);
+

The parameter x is unused why require it? I see that SLOB needs it. Cant you do a similar trick as in SLUB just calculating the start address of the object from the rcu address?

The first parameter of __kfree_rcu() is unused in slab,slub,slqb. (I used it before, for CONFIG_SLUB_DEBUG=y ...etc. But I found these debugging code is needless)

Object sizes of the memory objects in slob's kmalloc-slab are various. We hardly calculate the start address of the object from the rcu address.

Lai

#3 lore raw
Quoted 6 lines · Lai Jiangshan

The first parameter of __kfree_rcu() is unused in slab,slub,slqb. (I used it before, for CONFIG_SLUB_DEBUG=y ...etc. But I found these debugging code is needless)

Object sizes of the memory objects in slob's kmalloc-slab are various. We hardly calculate the start address of the object from the rcu address.

But you can start with the descriptor at the beginning of the page and scan until you find the object.

#4 lore raw
Quoted 11 lines · Christoph Lameter
Quoted 6 lines · Lai Jiangshan

The first parameter of __kfree_rcu() is unused in slab,slub,slqb. (I used it before, for CONFIG_SLUB_DEBUG=y ...etc. But I found these debugging code is needless)

Object sizes of the memory objects in slob's kmalloc-slab are various. We hardly calculate the start address of the object from the rcu address.

But you can start with the descriptor at the beginning of the page and scan until you find the object.

Another approach would be to keep a per-page bitmap of the objects that are awaiting a grace period.

Not sure whether or not it is worthwhile, but it does reduce the per-object overhead to two bits (one for the objects awaiting the current grace period, the other for objects that will await the next grace period).

Thanx, Paul

#5 lore raw
In reply to Lai Jiangshan
Quoted 14 lines · Lai Jiangshan

Introduce __kfree_rcu() for kfree_rcu()

It like __kfree_rcu() in slab.c.
We can calculate the object poiter from a poiter inside this object in slub.c, so we can use it for rcu callback and free the object.

---

mm/slub.c +0 -0
index 0280eee..b438ec5 100644

mm/slub.c void kfree(const void *x) · line 2765

Quoted 10 lines
mm/slub.c +7 -0
@@ -2756,6 +2756,30 @@ void kfree(const void *x)
 }
 EXPORT_SYMBOL(kfree);
 
+static void kfree_rcu_callback(struct rcu_head *rcu)
+{
+	void *portion = rcu;
+	struct page *page = virt_to_head_page(portion);
+
+	if (unlikely(!PageSlab(page))) {
+		BUG_ON(!PageCompound(page));

I'd prefer you did a WARN_ON() to increase the likelihood that people will be able to copy-paste the oops for a bug report.

mm/slub.c void kfree(const void *x) · line 2772

Quoted 7 lines
mm/slub.c +7 -0
+		put_page(page);
+	} else {
+		struct kmem_cache *s = page->slab;
+		void *start = page_address(page);
+		unsigned int index = (portion - start) / s->size;
+		void *object = start + s->size * index;
+

Again, I think the above would be more readable if you split the variable initialization from the declaration.

Quoted 23 lines
mm/slub.c +10 -0
+		slab_free(s, page, object, _RET_IP_);
+	}
+}
+
+void __kfree_rcu(const void *x, struct rcu_head *rcu)
+{
+	call_rcu(rcu, kfree_rcu_callback);
+}
+EXPORT_SYMBOL(__kfree_rcu);
+
 /*
  * kmem_cache_shrink removes empty slabs from the partial lists and sorts
  * the remaining slabs by the number of items in use. The slabs with the