Skip to contentSkip to conversation

slub: per cpu partial statistics change

RFC PATCH Applied Alex,Shi 3 participants 11 messages Feb 3, 2012 – Feb 20, 2012

Subsystems (2)
Memory management Slab allocator
lore
Share & tools
lore

Alex,Shi

author
#0 lore raw

RFC PATCH 1/1 slub: per cpu partial statistics change

This patch split the cpu_partial_free into 2 parts: cpu_partial_node, PCP refilling times from node partial; and same name cpu_partial_free, PCP refilling times in slab_free slow path. A new statistic 'release_cpu_partial' is added to get PCP release times. These info are useful when do PCP tunning.

The slabinfo.c code is unchanged, since cpu_partial_node is not on slow path.

Signed-off-by Alex Shi <[email protected]>
---
 include/linux/slub_def.h |    6 ++++--
 mm/slub.c                |   12 +++++++++---
 2 files changed, 13 insertions(+), 5 deletions(-)
include/linux/slub_def.h +4 -2
index a32bcfd..57ea943 100644
@@ -21,7 +21,7 @@ enum stat_item {
 	FREE_FROZEN,		/* Freeing to frozen slab */
 	FREE_ADD_PARTIAL,	/* Freeing moves slab to partial list */
 	FREE_REMOVE_PARTIAL,	/* Freeing removes last object */
-	ALLOC_FROM_PARTIAL,	/* Cpu slab acquired from partial list */
+	ALLOC_FROM_PARTIAL,	/* Cpu slab acquired from node partial list */
 	ALLOC_SLAB,		/* Cpu slab acquired from page allocator */
 	ALLOC_REFILL,		/* Refill cpu slab from slab freelist */
 	ALLOC_NODE_MISMATCH,	/* Switching cpu slab */

@@ -37,7 +37,9 @@ enum stat_item {
 	CMPXCHG_DOUBLE_CPU_FAIL,/* Failure of this_cpu_cmpxchg_double */
 	CMPXCHG_DOUBLE_FAIL,	/* Number of times that cmpxchg double did not match */
 	CPU_PARTIAL_ALLOC,	/* Used cpu partial on alloc */
-	CPU_PARTIAL_FREE,	/* USed cpu partial on free */
+	CPU_PARTIAL_FREE,	/* Refill cpu partial on free */
+	CPU_PARTIAL_NODE,	/* Refill cpu partial from node partial */
+	RELEASE_CPU_PARTIAL,	/* Release per cpu partial */
 	NR_SLUB_STAT_ITEMS };
 
 struct kmem_cache_cpu {
mm/slub.c +9 -3
index 4907563..5dd299c 100644
@@ -1560,6 +1560,7 @@ static void *get_partial_node(struct kmem_cache *s,
 		} else {
 			page->freelist = t;
 			available = put_cpu_partial(s, page, 0);
+			stat(s, CPU_PARTIAL_NODE);
 		}
 		if (kmem_cache_debug(s) || available > s->cpu_partial / 2)
 			break;

@@ -1973,6 +1974,7 @@ int put_cpu_partial(struct kmem_cache *s, struct page *page, int drain)
 				local_irq_restore(flags);
 				pobjects = 0;
 				pages = 0;
+				stat(s, RELEASE_CPU_PARTIAL);
 			}
 		}
 

@@ -1984,7 +1986,6 @@ int put_cpu_partial(struct kmem_cache *s, struct page *page, int drain)
 		page->next = oldpage;
 
 	} while (this_cpu_cmpxchg(s->cpu_slab->partial, oldpage, page) != oldpage);
-	stat(s, CPU_PARTIAL_FREE);
 	return pobjects;
 }
 

@@ -2465,9 +2466,10 @@ static void __slab_free(struct kmem_cache *s, struct page *page,
 		 * If we just froze the page then put it onto the
 		 * per cpu partial list.
 		 */
-		if (new.frozen && !was_frozen)
+		if (new.frozen && !was_frozen) {
 			put_cpu_partial(s, page, 1);
-
+			stat(s, CPU_PARTIAL_FREE);
+		}
 		/*
 		 * The list lock was not taken therefore no list
 		 * activity can be necessary.

@@ -5059,6 +5061,8 @@ STAT_ATTR(CMPXCHG_DOUBLE_CPU_FAIL, cmpxchg_double_cpu_fail);
 STAT_ATTR(CMPXCHG_DOUBLE_FAIL, cmpxchg_double_fail);
 STAT_ATTR(CPU_PARTIAL_ALLOC, cpu_partial_alloc);
 STAT_ATTR(CPU_PARTIAL_FREE, cpu_partial_free);
+STAT_ATTR(CPU_PARTIAL_NODE, cpu_partial_node);
+STAT_ATTR(RELEASE_CPU_PARTIAL, release_cpu_partial);
 #endif
 
 static struct attribute *slab_attrs[] = {

@@ -5124,6 +5128,8 @@ static struct attribute *slab_attrs[] = {
 	&cmpxchg_double_cpu_fail_attr.attr,
 	&cpu_partial_alloc_attr.attr,
 	&cpu_partial_free_attr.attr,
+	&cpu_partial_node_attr.attr,
+	&release_cpu_partial_attr.attr,
 #endif
 #ifdef CONFIG_FAILSLAB
 	&failslab_attr.attr,
Signature · 10 lines
1.7.5.1



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#1 lore raw
Quoted 4 lines · Alex,Shi

This patch split the cpu_partial_free into 2 parts: cpu_partial_node, PCP refilling times from node partial; and same name cpu_partial_free, PCP refilling times in slab_free slow path. A new statistic 'release_cpu_partial' is added to get PCP release times. These info are useful when do PCP tunning.

Releasing? The code where you inserted the new statistics counts the pages put on the cpu partial list when refilling from the node partial list.

See more below.

mm/slub.c static void *get_partial_node(struct kmem_cache *s, · line 1563

Quoted 10 lines

struct kmem_cache_cpu {

mm/slub.c +1 -0
index 4907563..5dd299c 100644
@@ -1560,6 +1560,7 @@ static void *get_partial_node(struct kmem_cache *s,
 		} else {
 			page->freelist = t;
 			available = put_cpu_partial(s, page, 0);
+			stat(s, CPU_PARTIAL_NODE);

This is refilling the per cpu partial list from the node list.

mm/slub.c int put_cpu_partial(struct kmem_cache *s, struct page *page, int drain) · line 1977

Quoted 8 lines
mm/slub.c +1 -0
 		}
 		if (kmem_cache_debug(s) || available > s->cpu_partial / 2)
 			break;

@@ -1973,6 +1974,7 @@ int put_cpu_partial(struct kmem_cache *s, struct page *page, int drain)
 				local_irq_restore(flags);
 				pobjects = 0;
 				pages = 0;
+				stat(s, RELEASE_CPU_PARTIAL);

The callers count the cpu partial operations. Why is there now one in put_cpu_partial? It is moving a page to the cpu partial list. Not releasing it from the cpu partial list.

mm/slub.c static void __slab_free(struct kmem_cache *s, struct page *page, · line 2471

Quoted 10 lines
mm/slub.c +2 -2
@@ -2465,9 +2466,10 @@ static void __slab_free(struct kmem_cache *s, struct page *page,
 		 * If we just froze the page then put it onto the
 		 * per cpu partial list.
 		 */
-		if (new.frozen && !was_frozen)
+		if (new.frozen && !was_frozen) {
 			put_cpu_partial(s, page, 1);
-
+			stat(s, CPU_PARTIAL_FREE);

cpu partial list filled with a partial page created from a fully allocated slab (which therefore was not on any list before).

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#2 lore raw
Quoted 9 lines · Christoph Lameter
Quoted 4 lines · Alex,Shi

This patch split the cpu_partial_free into 2 parts: cpu_partial_node, PCP refilling times from node partial; and same name cpu_partial_free, PCP refilling times in slab_free slow path. A new statistic 'release_cpu_partial' is added to get PCP release times. These info are useful when do PCP tunning.

Releasing? The code where you inserted the new statistics counts the pages put on the cpu partial list when refilling from the node partial list.

Ops, are we talking the same base kernel: Linus' tree? :) Here the Releasing code only be called in slow free path and the PCP is full at the same time, not in PCP refilling from node partial.

explanations more below.

Quoted 4 lines

See more below.

Quoted 12 lines

mm/slub.c static void *get_partial_node(struct kmem_cache *s, · line 1563

Quoted 10 lines

struct kmem_cache_cpu {

mm/slub.c +1 -0
index 4907563..5dd299c 100644
@@ -1560,6 +1560,7 @@ static void *get_partial_node(struct kmem_cache *s,
 		} else {
 			page->freelist = t;
 			available = put_cpu_partial(s, page, 0);
+			stat(s, CPU_PARTIAL_NODE);

This is refilling the per cpu partial list from the node list.

Yes. and same as my explanation in patch:

-       CPU_PARTIAL_FREE,       /* USed cpu partial on free */
+       CPU_PARTIAL_FREE,       /* Refill cpu partial on free */
+       CPU_PARTIAL_NODE,       /* Refill cpu partial from node partial */
Quoted 13 lines

mm/slub.c int put_cpu_partial(struct kmem_cache *s, struct page *page, int drain) · line 1977

Quoted 9 lines
mm/slub.c +1 -0
 		}
 		if (kmem_cache_debug(s) || available > s->cpu_partial / 2)
 			break;

@@ -1973,6 +1974,7 @@ int put_cpu_partial(struct kmem_cache *s, struct page *page, int drain)
 				local_irq_restore(flags);
 				pobjects = 0;
 				pages = 0;
+				stat(s, RELEASE_CPU_PARTIAL);

The callers count the cpu partial operations. Why is there now one in put_cpu_partial? It is moving a page to the cpu partial list. Not releasing it from the cpu partial list.

All old PCP will drain out on running CPU by unfreeze_partials() even it is not accurate here. The new one is not lost counting. It still be counted as CPU_PARTIAL_FREE in the following change as before.

If release is right, maybe named as drain_cpu_partial or unfreeze_cpu_partial?

Quoted 14 lines

mm/slub.c static void __slab_free(struct kmem_cache *s, struct page *page, · line 2471

Quoted 11 lines
mm/slub.c +2 -2
@@ -2465,9 +2466,10 @@ static void __slab_free(struct kmem_cache *s, struct page *page,
 		 * If we just froze the page then put it onto the
 		 * per cpu partial list.
 		 */
-		if (new.frozen && !was_frozen)
+		if (new.frozen && !was_frozen) {
 			put_cpu_partial(s, page, 1);
-
+			stat(s, CPU_PARTIAL_FREE);

cpu partial list filled with a partial page created from a fully allocated slab (which therefore was not on any list before).

Yes, but the counting is not new here. It just moved out of put_cpu_partial().

Quoted 2 lines

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#3 lore raw
Quoted 9 lines · Alex Shi
Quoted 7 lines · Christoph Lameter
Quoted 4 lines · Alex,Shi

This patch split the cpu_partial_free into 2 parts: cpu_partial_node, PCP refilling times from node partial; and same name cpu_partial_free, PCP refilling times in slab_free slow path. A new statistic 'release_cpu_partial' is added to get PCP release times. These info are useful when do PCP tunning.

Quoted 7 lines
Quoted 2 lines

Releasing? The code where you inserted the new statistics counts the pages put on the cpu partial list when refilling from the node partial list.

Ops, are we talking the same base kernel: Linus' tree? :) Here the Releasing code only be called in slow free path and the PCP is full at the same time, not in PCP refilling from node partial.

Well the term releasing is unfortunate. per cpu partial pages can migrate to and from the per node partial list and become per cpu slabs under allocation.

Quoted 14 lines
Quoted 4 lines
Quoted 4 lines
mm/slub.c +0 -0
@@ -2465,9 +2466,10 @@ static void __slab_free(struct kmem_cache *s, struct page *page,
 		 * If we just froze the page then put it onto the
 		 * per cpu partial list.
 		 */
Quoted 8 lines

mm/slub.c static void __slab_free(struct kmem_cache *s, struct page *page, · line 2471

Quoted 5 lines
mm/slub.c +2 -2
-		if (new.frozen && !was_frozen)
+		if (new.frozen && !was_frozen) {
 			put_cpu_partial(s, page, 1);
-
+			stat(s, CPU_PARTIAL_FREE);

cpu partial list filled with a partial page created from a fully allocated slab (which therefore was not on any list before).

Quoted 2 lines

Yes, but the counting is not new here. It just moved out of put_cpu_partial().

Ok but then you also added different accounting in put_cpu_partial.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#4 lore raw
Quoted 4 lines

Well the term releasing is unfortunate. per cpu partial pages can migrate to and from the per node partial list and become per cpu slabs under allocation.

Yes, The word is really not good here. How about of CPU_PARTIAL_UNFREEZE since a unfreeze_cpu_partial() just called before ?

Quoted 15 lines
Quoted 15 lines
Quoted 13 lines

Review static void __slab_free(struct kmem_cache *s, struct page *page, · line 2471

Quoted 10 lines
(quoted diff) +2 -2
@@ -2465,9 +2466,10 @@ static void __slab_free(struct kmem_cache *s, struct page *page,
 		 * If we just froze the page then put it onto the
 		 * per cpu partial list.
 		 */
-		if (new.frozen && !was_frozen)
+		if (new.frozen && !was_frozen) {
 			put_cpu_partial(s, page, 1);
-
+			stat(s, CPU_PARTIAL_FREE);

cpu partial list filled with a partial page created from a fully allocated slab (which therefore was not on any list before).

Quoted 4 lines
Quoted 2 lines

Yes, but the counting is not new here. It just moved out of put_cpu_partial().

Ok but then you also added different accounting in put_cpu_partial.

Yes, I want to account the unfreeze_partials（） actions in put_cpu_partial（). The unfreezing accounting isn't conflict or repeat with the cpu_partial_free accounting, since they are different actions for the PCP.

According your above comments, how about the new patch with new accounting name?
------------

Quoted 1 line

From bd2b79297b4550035b0b0ec16dd0f3008a3a76dc Mon Sep 17 00:00:00 2001

From: Alex Shi <[email protected]>
Date: Fri, 3 Feb 2012 23:34:56 +0800
Subject: [PATCH] slub: per cpu partial statistics change

This patch split the cpu_partial_free into 2 parts: cpu_partial_node, PCP refilling times from node partial; and same name cpu_partial_free, PCP refilling times in slab_free slow path. A new statistic 'cpu_partial_unfreeze' is added to get PCP unfreeze times. These info are useful when do PCP tunning.

The slabinfo.c code is unchanged, since cpu_partial_node is not on slow path.

Signed-off-by Alex Shi <[email protected]>
---
 include/linux/slub_def.h |    6 ++++--
 mm/slub.c                |   12 +++++++++---
 2 files changed, 13 insertions(+), 5 deletions(-)
include/linux/slub_def.h +4 -2
index a32bcfd..2549483 100644
@@ -21,7 +21,7 @@ enum stat_item {
 	FREE_FROZEN,		/* Freeing to frozen slab */
 	FREE_ADD_PARTIAL,	/* Freeing moves slab to partial list */
 	FREE_REMOVE_PARTIAL,	/* Freeing removes last object */
-	ALLOC_FROM_PARTIAL,	/* Cpu slab acquired from partial list */
+	ALLOC_FROM_PARTIAL,	/* Cpu slab acquired from node partial list */
 	ALLOC_SLAB,		/* Cpu slab acquired from page allocator */
 	ALLOC_REFILL,		/* Refill cpu slab from slab freelist */
 	ALLOC_NODE_MISMATCH,	/* Switching cpu slab */

@@ -37,7 +37,9 @@ enum stat_item {
 	CMPXCHG_DOUBLE_CPU_FAIL,/* Failure of this_cpu_cmpxchg_double */
 	CMPXCHG_DOUBLE_FAIL,	/* Number of times that cmpxchg double did not match */
 	CPU_PARTIAL_ALLOC,	/* Used cpu partial on alloc */
-	CPU_PARTIAL_FREE,	/* USed cpu partial on free */
+	CPU_PARTIAL_FREE,	/* Refill cpu partial on free */
+	CPU_PARTIAL_NODE,	/* Refill cpu partial from node partial */
+	CPU_PARTIAL_UNFREEZE,	/* Unfreeze cpu partial */
 	NR_SLUB_STAT_ITEMS };
 
 struct kmem_cache_cpu {
mm/slub.c +9 -3
index 4907563..6ededd7 100644
@@ -1560,6 +1560,7 @@ static void *get_partial_node(struct kmem_cache *s,
 		} else {
 			page->freelist = t;
 			available = put_cpu_partial(s, page, 0);
+			stat(s, CPU_PARTIAL_NODE);
 		}
 		if (kmem_cache_debug(s) || available > s->cpu_partial / 2)
 			break;

@@ -1973,6 +1974,7 @@ int put_cpu_partial(struct kmem_cache *s, struct page *page, int drain)
 				local_irq_restore(flags);
 				pobjects = 0;
 				pages = 0;
+				stat(s, CPU_PARTIAL_UNFREEZE);
 			}
 		}
 

@@ -1984,7 +1986,6 @@ int put_cpu_partial(struct kmem_cache *s, struct page *page, int drain)
 		page->next = oldpage;
 
 	} while (this_cpu_cmpxchg(s->cpu_slab->partial, oldpage, page) != oldpage);
-	stat(s, CPU_PARTIAL_FREE);
 	return pobjects;
 }
 

@@ -2465,9 +2466,10 @@ static void __slab_free(struct kmem_cache *s, struct page *page,
 		 * If we just froze the page then put it onto the
 		 * per cpu partial list.
 		 */
-		if (new.frozen && !was_frozen)
+		if (new.frozen && !was_frozen) {
 			put_cpu_partial(s, page, 1);
-
+			stat(s, CPU_PARTIAL_FREE);
+		}
 		/*
 		 * The list lock was not taken therefore no list
 		 * activity can be necessary.

@@ -5059,6 +5061,8 @@ STAT_ATTR(CMPXCHG_DOUBLE_CPU_FAIL, cmpxchg_double_cpu_fail);
 STAT_ATTR(CMPXCHG_DOUBLE_FAIL, cmpxchg_double_fail);
 STAT_ATTR(CPU_PARTIAL_ALLOC, cpu_partial_alloc);
 STAT_ATTR(CPU_PARTIAL_FREE, cpu_partial_free);
+STAT_ATTR(CPU_PARTIAL_NODE, cpu_partial_node);
+STAT_ATTR(CPU_PARTIAL_UNFREEZE, cpu_partial_unfreeze);
 #endif
 
 static struct attribute *slab_attrs[] = {

@@ -5124,6 +5128,8 @@ static struct attribute *slab_attrs[] = {
 	&cmpxchg_double_cpu_fail_attr.attr,
 	&cpu_partial_alloc_attr.attr,
 	&cpu_partial_free_attr.attr,
+	&cpu_partial_node_attr.attr,
+	&cpu_partial_unfreeze_attr.attr,
 #endif
 #ifdef CONFIG_FAILSLAB
 	&failslab_attr.attr,
Signature · 1 line
1.6.3.3
#5 lore raw
Quoted 4 lines · Alex,Shi

Yes, I want to account the unfreeze_partials（） actions in put_cpu_partial（). The unfreezing accounting isn't conflict or repeat with the cpu_partial_free accounting, since they are different actions for the PCP.

Well what is happening here is that the whole per cpu partial list is moved back to the per node partial list.

CPU_PARTIAL_DRAIN_TO_NODE_PARTIAL ?

A bit long I think. CPU_PARTIAL_DRAIN?

UNFREEZE does not truly reflect what is going on here.

author
#6 lore raw
Quoted 7 lines · Christoph Lameter
Quoted 4 lines · Alex,Shi

Yes, I want to account the unfreeze_partialsi 1/4 ?i 1/4 ? actions in put_cpu_partiali 1/4 ?). The unfreezing accounting isn't conflict or repeat with the cpu_partial_free accounting, since they are different actions for the PCP.

Quoted 6 lines

Well what is happening here is that the whole per cpu partial list is moved back to the per node partial list.

CPU_PARTIAL_DRAIN_TO_NODE_PARTIAL ?

A bit long I think. CPU_PARTIAL_DRAIN?

Yes. it is more meaningful. :)
Patch change here.

----------------

Quoted 1 line

From af88a7b0134d3eea82a4cf9985026852e50f5343 Mon Sep 17 00:00:00 2001

From: Alex Shi <[email protected]>
Date: Fri, 3 Feb 2012 23:34:56 +0800
Subject: [PATCH] slub: per cpu partial statistics change

This patch split the cpu_partial_free into 2 parts: cpu_partial_node, PCP refilling times from node partial; and same name cpu_partial_free, PCP refilling times in slab_free slow path. A new statistic 'cpu_partial_drain' is added to get PCP drain to node partial times. These info are useful when do PCP tunning.

The slabinfo.c code is unchanged, since cpu_partial_node is not on slow path.

Signed-off-by Alex Shi <[email protected]>
---
 include/linux/slub_def.h |    6 ++++--
 mm/slub.c                |   12 +++++++++---
 2 files changed, 13 insertions(+), 5 deletions(-)
include/linux/slub_def.h +4 -2
index a32bcfd..6388a66 100644
@@ -21,7 +21,7 @@ enum stat_item {
 	FREE_FROZEN,		/* Freeing to frozen slab */
 	FREE_ADD_PARTIAL,	/* Freeing moves slab to partial list */
 	FREE_REMOVE_PARTIAL,	/* Freeing removes last object */
-	ALLOC_FROM_PARTIAL,	/* Cpu slab acquired from partial list */
+	ALLOC_FROM_PARTIAL,	/* Cpu slab acquired from node partial list */
 	ALLOC_SLAB,		/* Cpu slab acquired from page allocator */
 	ALLOC_REFILL,		/* Refill cpu slab from slab freelist */
 	ALLOC_NODE_MISMATCH,	/* Switching cpu slab */

@@ -37,7 +37,9 @@ enum stat_item {
 	CMPXCHG_DOUBLE_CPU_FAIL,/* Failure of this_cpu_cmpxchg_double */
 	CMPXCHG_DOUBLE_FAIL,	/* Number of times that cmpxchg double did not match */
 	CPU_PARTIAL_ALLOC,	/* Used cpu partial on alloc */
-	CPU_PARTIAL_FREE,	/* USed cpu partial on free */
+	CPU_PARTIAL_FREE,	/* Refill cpu partial on free */
+	CPU_PARTIAL_NODE,	/* Refill cpu partial from node partial */
+	CPU_PARTIAL_DRAIN,	/* Drain cpu partial to node partial */
 	NR_SLUB_STAT_ITEMS };
 
 struct kmem_cache_cpu {
mm/slub.c +9 -3
index 4907563..4e71a0a 100644
@@ -1560,6 +1560,7 @@ static void *get_partial_node(struct kmem_cache *s,
 		} else {
 			page->freelist = t;
 			available = put_cpu_partial(s, page, 0);
+			stat(s, CPU_PARTIAL_NODE);
 		}
 		if (kmem_cache_debug(s) || available > s->cpu_partial / 2)
 			break;

@@ -1973,6 +1974,7 @@ int put_cpu_partial(struct kmem_cache *s, struct page *page, int drain)
 				local_irq_restore(flags);
 				pobjects = 0;
 				pages = 0;
+				stat(s, CPU_PARTIAL_DRAIN);
 			}
 		}
 

@@ -1984,7 +1986,6 @@ int put_cpu_partial(struct kmem_cache *s, struct page *page, int drain)
 		page->next = oldpage;
 
 	} while (this_cpu_cmpxchg(s->cpu_slab->partial, oldpage, page) != oldpage);
-	stat(s, CPU_PARTIAL_FREE);
 	return pobjects;
 }
 

@@ -2465,9 +2466,10 @@ static void __slab_free(struct kmem_cache *s, struct page *page,
 		 * If we just froze the page then put it onto the
 		 * per cpu partial list.
 		 */
-		if (new.frozen && !was_frozen)
+		if (new.frozen && !was_frozen) {
 			put_cpu_partial(s, page, 1);
-
+			stat(s, CPU_PARTIAL_FREE);
+		}
 		/*
 		 * The list lock was not taken therefore no list
 		 * activity can be necessary.

@@ -5059,6 +5061,8 @@ STAT_ATTR(CMPXCHG_DOUBLE_CPU_FAIL, cmpxchg_double_cpu_fail);
 STAT_ATTR(CMPXCHG_DOUBLE_FAIL, cmpxchg_double_fail);
 STAT_ATTR(CPU_PARTIAL_ALLOC, cpu_partial_alloc);
 STAT_ATTR(CPU_PARTIAL_FREE, cpu_partial_free);
+STAT_ATTR(CPU_PARTIAL_NODE, cpu_partial_node);
+STAT_ATTR(CPU_PARTIAL_DRAIN, cpu_partial_drain);
 #endif
 
 static struct attribute *slab_attrs[] = {

@@ -5124,6 +5128,8 @@ static struct attribute *slab_attrs[] = {
 	&cmpxchg_double_cpu_fail_attr.attr,
 	&cpu_partial_alloc_attr.attr,
 	&cpu_partial_free_attr.attr,
+	&cpu_partial_node_attr.attr,
+	&cpu_partial_drain_attr.attr,
 #endif
 #ifdef CONFIG_FAILSLAB
 	&failslab_attr.attr,
Signature · 10 lines
1.6.3.3



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Acked-by
#7 lore raw
Quoted 4 lines · Alex,Shi
Quoted 1 line

A bit long I think. CPU_PARTIAL_DRAIN?

Yes. it is more meaningful. :)
Patch change here.

Acked-by Christoph Lameter <[email protected]>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author Acked-by Christoph Lameter
#8 lore raw
Quoted 8 lines · Christoph Lameter
Quoted 4 lines · Alex,Shi
Quoted 1 line

A bit long I think. CPU_PARTIAL_DRAIN?

Yes. it is more meaningful. :)
Patch change here.

Acked-by Christoph Lameter <[email protected]>

Pakka:
Would you like to pick up this patch? It works on latest Linus' tree.

Thanks!
========

Quoted 1 line

From af88a7b0134d3eea82a4cf9985026852e50f5343 Mon Sep 17 00:00:00 2001

From: Alex Shi <[email protected]>
Date: Fri, 3 Feb 2012 23:34:56 +0800
Subject: [PATCH] slub: per cpu partial statistics change

This patch split the cpu_partial_free into 2 parts: cpu_partial_node, PCP refilling times from node partial; and same name cpu_partial_free, PCP refilling times in slab_free slow path. A new statistic 'cpu_partial_drain' is added to get PCP drain to node partial times. These info are useful when do PCP tunning.

The slabinfo.c code is unchanged, since cpu_partial_node is not on slow path.

Signed-off-by Alex Shi <[email protected]>
Acked-by Christoph Lameter <[email protected]>
---
 include/linux/slub_def.h |    6 ++++--
 mm/slub.c                |   12 +++++++++---
 2 files changed, 13 insertions(+), 5 deletions(-)
include/linux/slub_def.h +4 -2
index a32bcfd..6388a66 100644
@@ -21,7 +21,7 @@ enum stat_item {
 	FREE_FROZEN,		/* Freeing to frozen slab */
 	FREE_ADD_PARTIAL,	/* Freeing moves slab to partial list */
 	FREE_REMOVE_PARTIAL,	/* Freeing removes last object */
-	ALLOC_FROM_PARTIAL,	/* Cpu slab acquired from partial list */
+	ALLOC_FROM_PARTIAL,	/* Cpu slab acquired from node partial list */
 	ALLOC_SLAB,		/* Cpu slab acquired from page allocator */
 	ALLOC_REFILL,		/* Refill cpu slab from slab freelist */
 	ALLOC_NODE_MISMATCH,	/* Switching cpu slab */

@@ -37,7 +37,9 @@ enum stat_item {
 	CMPXCHG_DOUBLE_CPU_FAIL,/* Failure of this_cpu_cmpxchg_double */
 	CMPXCHG_DOUBLE_FAIL,	/* Number of times that cmpxchg double did not match */
 	CPU_PARTIAL_ALLOC,	/* Used cpu partial on alloc */
-	CPU_PARTIAL_FREE,	/* USed cpu partial on free */
+	CPU_PARTIAL_FREE,	/* Refill cpu partial on free */
+	CPU_PARTIAL_NODE,	/* Refill cpu partial from node partial */
+	CPU_PARTIAL_DRAIN,	/* Drain cpu partial to node partial */
 	NR_SLUB_STAT_ITEMS };
 
 struct kmem_cache_cpu {
mm/slub.c +9 -3
index 4907563..4e71a0a 100644
@@ -1560,6 +1560,7 @@ static void *get_partial_node(struct kmem_cache *s,
 		} else {
 			page->freelist = t;
 			available = put_cpu_partial(s, page, 0);
+			stat(s, CPU_PARTIAL_NODE);
 		}
 		if (kmem_cache_debug(s) || available > s->cpu_partial / 2)
 			break;

@@ -1973,6 +1974,7 @@ int put_cpu_partial(struct kmem_cache *s, struct page *page, int drain)
 				local_irq_restore(flags);
 				pobjects = 0;
 				pages = 0;
+				stat(s, CPU_PARTIAL_DRAIN);
 			}
 		}
 

@@ -1984,7 +1986,6 @@ int put_cpu_partial(struct kmem_cache *s, struct page *page, int drain)
 		page->next = oldpage;
 
 	} while (this_cpu_cmpxchg(s->cpu_slab->partial, oldpage, page) != oldpage);
-	stat(s, CPU_PARTIAL_FREE);
 	return pobjects;
 }
 

@@ -2465,9 +2466,10 @@ static void __slab_free(struct kmem_cache *s, struct page *page,
 		 * If we just froze the page then put it onto the
 		 * per cpu partial list.
 		 */
-		if (new.frozen && !was_frozen)
+		if (new.frozen && !was_frozen) {
 			put_cpu_partial(s, page, 1);
-
+			stat(s, CPU_PARTIAL_FREE);
+		}
 		/*
 		 * The list lock was not taken therefore no list
 		 * activity can be necessary.

@@ -5059,6 +5061,8 @@ STAT_ATTR(CMPXCHG_DOUBLE_CPU_FAIL, cmpxchg_double_cpu_fail);
 STAT_ATTR(CMPXCHG_DOUBLE_FAIL, cmpxchg_double_fail);
 STAT_ATTR(CPU_PARTIAL_ALLOC, cpu_partial_alloc);
 STAT_ATTR(CPU_PARTIAL_FREE, cpu_partial_free);
+STAT_ATTR(CPU_PARTIAL_NODE, cpu_partial_node);
+STAT_ATTR(CPU_PARTIAL_DRAIN, cpu_partial_drain);
 #endif
 
 static struct attribute *slab_attrs[] = {

@@ -5124,6 +5128,8 @@ static struct attribute *slab_attrs[] = {
 	&cmpxchg_double_cpu_fail_attr.attr,
 	&cpu_partial_alloc_attr.attr,
 	&cpu_partial_free_attr.attr,
+	&cpu_partial_node_attr.attr,
+	&cpu_partial_drain_attr.attr,
 #endif
 #ifdef CONFIG_FAILSLAB
 	&failslab_attr.attr,
Signature · 10 lines
1.6.3.3



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Applied
#9 lore raw
Quoted 2 lines · Alex,Shi

Pakka:
Would you like to pick up this patch? It works on latest Linus' tree.

Applied, thanks! Can you please use my @kernel.org email address in the future? I don't really follow this account as often.

Pekka

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#10 lore raw
Quoted 6 lines · Pekka Enberg
Quoted 2 lines · Alex,Shi

Pakka:
Would you like to pick up this patch? It works on latest Linus' tree.

Applied, thanks! Can you please use my @kernel.org email address in the future? I don't really follow this account as often.

Thanks! Pekka, I will use your kernel.org address. :)

Quoted 2 lines

Pekka

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>