aio: fix io_setup/io_destroy race

PATCH In review Al Viro 3 participants 6 messages Mar 7, 2012 – Mar 10, 2012

Subsystems (2)

VFS Aio

lore

Share & tools

lore

Al Viro

author maintainer

2012-03-07 05:16 UTC #0 lore raw

PATCH 1/1 aio: fix io_setup/io_destroy race

Have ioctx_alloc() return an extra reference, so that caller would drop it on success and not bother with re-grabbing it on failure exit. The current code is obviously broken - io_destroy() from another thread that managed to guess the address io_setup() would've returned would free ioctx right under us; gets especially interesting if aio_context_t * we pass to io_setup() points to PROT_READ mapping, so put_user() fails and we end up doing io_destroy() on kioctx another thread has just got freed...

Cc [email protected]

Signed-off-by Al Viro <[email protected]>

---
[there's other pending aio stuff, but this part is really obvious and self-contained]

fs/aio.c +4 -4

index 67e4b90..f6578cb 100644
@@ -273,7 +273,7 @@ static struct kioctx *ioctx_alloc(unsigned nr_events)
 	mm = ctx->mm = current->mm;
 	atomic_inc(&mm->mm_count);
 
-	atomic_set(&ctx->users, 1);
+	atomic_set(&ctx->users, 2);
 	spin_lock_init(&ctx->ctx_lock);
 	spin_lock_init(&ctx->ring_info.ring_lock);
 	init_waitqueue_head(&ctx->wait);

@@ -1338,10 +1338,10 @@ SYSCALL_DEFINE2(io_setup, unsigned, nr_events, aio_context_t __user *, ctxp)
 	ret = PTR_ERR(ioctx);
 	if (!IS_ERR(ioctx)) {
 		ret = put_user(ioctx->user_id, ctxp);
-		if (!ret)
+		if (!ret) {
+			put_ioctx(ioctx);
 			return 0;
-
-		get_ioctx(ioctx); /* io_destroy() expects us to hold a ref */
+		}
 		io_destroy(ioctx);
 	}

Benjamin LaHaise

maintainer

2012-03-07 17:13 UTC #1 lore raw

Quoted 10 lines · Al Viro

Have ioctx_alloc() return an extra reference, so that caller would drop it on success and not bother with re-grabbing it on failure exit. The current code is obviously broken - io_destroy() from another thread that managed to guess the address io_setup() would've returned would free ioctx right under us; gets especially interesting if aio_context_t * we pass to io_setup() points to PROT_READ mapping, so put_user() fails and we end up doing io_destroy() on kioctx another thread has just got freed...
Cc [email protected]
Signed-off-by Al Viro <[email protected]>

Signed-off-by Benjamin LaHaise <[email protected]>

The fix looks good, and works with a quick test program checking the two cases involved (ioctx create success + failure due to PROT_READ).

-ben

Jeff Moyer

Reviewed-by on v1 1/1

2012-03-07 18:16 UTC #2 lore raw

In reply to Al Viro

Quoted 7 lines · Al Viro

Have ioctx_alloc() return an extra reference, so that caller would drop it on success and not bother with re-grabbing it on failure exit. The current code is obviously broken - io_destroy() from another thread that managed to guess the address io_setup() would've returned would free ioctx right under us; gets especially interesting if aio_context_t * we pass to io_setup() points to PROT_READ mapping, so put_user() fails and we end up doing io_destroy() on kioctx another thread has just got freed...

Al, you certainly are creative. ;-) I agree with the problem and the fix. It would be nice, though if you had added comments.

I ran xfstests ./check -g aio, and there were no problems.

Reviewed-by Jeff Moyer <[email protected]>

Al Viro

author maintainer

2012-03-08 17:51 UTC #3 lore raw

PATCH 1/1 aio: fix the "too late munmap()" race

In reply to Al Viro

Current code has put_ioctx() called asynchronously from aio_fput_routine(); that's done *after* we have killed the request that used to pin ioctx, so there's nothing to stop io_destroy() waiting in wait_for_all_aios() from progressing. As the result, we can end up with async call of put_ioctx() being the last one and possibly happening during exit_mmap() or elf_core_dump(), neither of which expects stray munmap() being done to them...

We do need to prevent _freeing_ ioctx until aio_fput_routine() is done with that, but that's all we care about - neither io_destroy() nor exit_aio() will progress past wait_for_all_aios() until aio_fput_routine() does really_put_req(), so the ioctx teardown won't be done until then and we don't care about the contents of ioctx past that point.

Since actual freeing of these suckers is RCU-delayed, we don't need to bump ioctx refcount when request goes into list for async removal. All we need is rcu_read_lock held just over the ->ctx_lock-protected area in aio_fput_routine().

Cc [email protected]

Signed-off-by Al Viro <[email protected]>

---
[on top of the previous patch, applies to anything past 2008 when freeing via call_rcu() went in; earlier versions just need kmem_cache_free() done via call_rcu() as commit abf137dd7712132ee56d5b3143c2ff61a72a5faa does, except that we don't really need to delay playing with aio_nr - mainline also will be better off by not bothering with that]

fs/aio.c +6 -8

index f6578cb..8cf3796 100644
@@ -228,12 +228,6 @@ static void __put_ioctx(struct kioctx *ctx)
 	call_rcu(&ctx->rcu_head, ctx_rcu_free);
 }
 
-static inline void get_ioctx(struct kioctx *kioctx)
-{
-	BUG_ON(atomic_read(&kioctx->users) <= 0);
-	atomic_inc(&kioctx->users);
-}
-
 static inline int try_get_ioctx(struct kioctx *kioctx)
 {
 	return atomic_inc_not_zero(&kioctx->users);

@@ -609,11 +603,16 @@ static void aio_fput_routine(struct work_struct *data)
 			fput(req->ki_filp);
 
 		/* Link the iocb into the context's free list */
+		rcu_read_lock();
 		spin_lock_irq(&ctx->ctx_lock);
 		really_put_req(ctx, req);
+		/*
+		 * at that point ctx might've been killed, but actual
+		 * freeing is RCU'd
+		 */
 		spin_unlock_irq(&ctx->ctx_lock);
+		rcu_read_unlock();
 
-		put_ioctx(ctx);
 		spin_lock_irq(&fput_lock);
 	}
 	spin_unlock_irq(&fput_lock);

@@ -644,7 +643,6 @@ static int __aio_put_req(struct kioctx *ctx, struct kiocb *req)
 	 * this function will be executed w/out any aio kthread wakeup.
 	 */
 	if (unlikely(!fput_atomic(req->ki_filp))) {
-		get_ioctx(ctx);
 		spin_lock(&fput_lock);
 		list_add(&req->ki_list, &fput_head);
 		spin_unlock(&fput_lock);

Jeff Moyer

Reviewed-by on v1 1/1

2012-03-08 18:15 UTC #4 lore raw

Re: [PATCH] aio: fix the "too late munmap()" race

Quoted 14 lines · Al Viro

Current code has put_ioctx() called asynchronously from aio_fput_routine(); that's done *after* we have killed the request that used to pin ioctx, so there's nothing to stop io_destroy() waiting in wait_for_all_aios() from progressing. As the result, we can end up with async call of put_ioctx() being the last one and possibly happening during exit_mmap() or elf_core_dump(), neither of which expects stray munmap() being done to them...
We do need to prevent _freeing_ ioctx until aio_fput_routine() is done with that, but that's all we care about - neither io_destroy() nor exit_aio() will progress past wait_for_all_aios() until aio_fput_routine() does really_put_req(), so the ioctx teardown won't be done until then and we don't care about the contents of ioctx past that point.

Quoted 4 lines

Since actual freeing of these suckers is RCU-delayed, we don't need to bump ioctx refcount when request goes into list for async removal. All we need is rcu_read_lock held just over the ->ctx_lock-protected area in aio_fput_routine().

Looks good to me.

Reviewed-by Jeff Moyer <[email protected]>

Quoted 57 lines

Cc [email protected]
Signed-off-by Al Viro <[email protected]>
---
[on top of the previous patch, applies to anything past 2008 when freeing via call_rcu() went in; earlier versions just need kmem_cache_free() done via call_rcu() as commit abf137dd7712132ee56d5b3143c2ff61a72a5faa does, except that we don't really need to delay playing with aio_nr - mainline also will be better off by not bothering with that]
fs/aio.c +6 -9
index f6578cb..8cf3796 100644
@@ -228,12 +228,6 @@ static void __put_ioctx(struct kioctx *ctx)
 	call_rcu(&ctx->rcu_head, ctx_rcu_free);
 }
 
-static inline void get_ioctx(struct kioctx *kioctx)
-{
-	BUG_ON(atomic_read(&kioctx->users) <= 0);
-	atomic_inc(&kioctx->users);
-}
-
 static inline int try_get_ioctx(struct kioctx *kioctx)
 {
 	return atomic_inc_not_zero(&kioctx->users);
@@ -609,11 +603,16 @@ static void aio_fput_routine(struct work_struct *data)
 			fput(req->ki_filp);
 
 		/* Link the iocb into the context's free list */
+		rcu_read_lock();
 		spin_lock_irq(&ctx->ctx_lock);
 		really_put_req(ctx, req);
+		/*
+		 * at that point ctx might've been killed, but actual
+		 * freeing is RCU'd
+		 */
 		spin_unlock_irq(&ctx->ctx_lock);
+		rcu_read_unlock();
 
-		put_ioctx(ctx);
 		spin_lock_irq(&fput_lock);
 	}
 	spin_unlock_irq(&fput_lock);
@@ -644,7 +643,6 @@ static int __aio_put_req(struct kioctx *ctx, struct kiocb *req)
 	 * this function will be executed w/out any aio kthread wakeup.
 	 */
 	if (unlikely(!fput_atomic(req->ki_filp))) {
-		get_ioctx(ctx);
 		spin_lock(&fput_lock);
 		list_add(&req->ki_list, &fput_head);
 		spin_unlock(&fput_lock);
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Benjamin LaHaise

maintainer

2012-03-10 02:13 UTC #5 lore raw

Re: [PATCH] aio: fix the "too late munmap()" race

In reply to Al Viro

Quoted 7 lines · Al Viro

Since actual freeing of these suckers is RCU-delayed, we don't need to bump ioctx refcount when request goes into list for async removal. All we need is rcu_read_lock held just over the ->ctx_lock-protected area in aio_fput_routine().
Cc [email protected]
Signed-off-by Al Viro <[email protected]>

Signed-off-by Benjamin LaHaise <[email protected]>

Looks good. As with the previous patch, I instrumented and tested the code to trigger fput_work() with no issues encountered.

-ben