Skip to contentSkip to conversation

+ mempool-fix-first-round-failure-behavior.patch added to -mm tree

DISCUSSION Oleg Nesterov 2 participants 2 messages Dec 22, 2011

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hello,

Quoted 10 lines

For the initial allocation, mempool passes modified gfp mask to the backing allocator so that it doesn't try too hard when there are reserved elements waiting in the pool; however, when that allocation fails and pool is empty too, it either waits for the pool to be replenished before retrying or fails if !__GFP_WAIT.

* If the caller was calling in with GFP_ATOMIC, it never gets to try emergency reserve. Allocations which would have succeeded without mempool may fail, which is just wrong.

Quoted 7 lines

* Allocation which could have succeeded after a bit of reclaim now has to wait on the reserved items and it's not like mempool doesn't retry with the original gfp mask. It just does that *after* someone returns an element, pointlessly delaying things.

Fix it by retrying immediately with the gfp mask requested by the caller if the first round of allocation attempts fails with modified mask.

I can't even explain why this (simple!) logic looks confusing to me, with or without the patch. A couple of questions:

1. Why do we remove __GFP_WAIT unconditionally before the the very 1st allocation?

2. Why do we always restore it after io_schedule(), even if we have the reserved items?

IOW, what do you think about the patch below instead?

Oleg.

mm/mempool.c +8 -4
@@ -212,10 +212,12 @@ void * mempool_alloc(mempool_t *pool, gf
 	gfp_mask |= __GFP_NORETRY;	/* don't loop in __alloc_pages */
 	gfp_mask |= __GFP_NOWARN;	/* failures are OK */
 
-	gfp_temp = gfp_mask & ~(__GFP_WAIT|__GFP_IO);
-
 repeat_alloc:
 
+	gfp_temp = gfp_mask;
+	if (pool->curr_nr)
+		gfp_temp &= ~(__GFP_WAIT|__GFP_IO);
+
 	element = pool->alloc(gfp_temp, pool->pool_data);
 	if (likely(element != NULL))
 		return element;
@@ -229,13 +231,15 @@ repeat_alloc:
 	}
 
 	/* We must not sleep in the GFP_ATOMIC case */
-	if (!(gfp_mask & __GFP_WAIT)) {
+	if (!(gfp_temp & __GFP_WAIT)) {
 		spin_unlock_irqrestore(&pool->lock, flags);
+		/* raced with another mempool_alloc? */
+		if (gfp_mask & __GFP_WAIT)
+			goto repeat_alloc;
 		return NULL;
 	}
 
 	/* Let's wait for someone else to return an element to @pool */
-	gfp_temp = gfp_mask;
 	init_wait(&wait);
 	prepare_to_wait(&pool->wait, &wait, TASK_UNINTERRUPTIBLE);
 
#1 lore raw

Hello, Oleg.

Quoted 1 line · Oleg Nesterov

I can't even explain why this (simple!) logic looks confusing to me,

Yeah, gfp_mask and temp confused me pretty good too.

Quoted 4 lines

with or without the patch. A couple of questions:

1. Why do we remove __GFP_WAIT unconditionally before the the very 1st allocation?

To avoid blocking when there's pool sitting around.

Quoted 2 lines

2. Why do we always restore it after io_schedule(), even if we have the reserved items?

No idea.

Quoted 22 lines
mm/mempool.c +5 -3
@@ -212,10 +212,12 @@ void * mempool_alloc(mempool_t *pool, gf
 	gfp_mask |= __GFP_NORETRY;	/* don't loop in __alloc_pages */
 	gfp_mask |= __GFP_NOWARN;	/* failures are OK */
 
-	gfp_temp = gfp_mask & ~(__GFP_WAIT|__GFP_IO);
-
 repeat_alloc:
 
+	gfp_temp = gfp_mask;
+	if (pool->curr_nr)
+		gfp_temp &= ~(__GFP_WAIT|__GFP_IO);
+
 	element = pool->alloc(gfp_temp, pool->pool_data);
 	if (likely(element != NULL))
 		return element;
@@ -229,13 +231,15 @@ repeat_alloc:
 	}
 
 	/* We must not sleep in the GFP_ATOMIC case */
-	if (!(gfp_mask & __GFP_WAIT)) {
+	if (!(gfp_temp & __GFP_WAIT)) {
 		spin_unlock_irqrestore(&pool->lock, flags);

mm/mempool.c repeat_alloc: · line 244

Quoted 10 lines
mm/mempool.c +3 -1
+		/* raced with another mempool_alloc? */
+		if (gfp_mask & __GFP_WAIT)
+			goto repeat_alloc;
 		return NULL;
 	}
 
 	/* Let's wait for someone else to return an element to @pool */
-	gfp_temp = gfp_mask;
 	init_wait(&wait);
 	prepare_to_wait(&pool->wait, &wait, TASK_UNINTERRUPTIBLE);

Yeah, this one definitely looks better & makes more sense. Andrew, please feel free to drop mine and take this one.

Thanks.

Signature · 1 line
tejun