Skip to contentSkip to conversation

aio-2.6.13-rc6-B1

AIO Benjamin LaHaise 3 participants 7 messages Aug 17, 2005 – Sep 21, 2005

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

The bugfix followup to the last aio rollup is now available at:

	http://www.kvack.org/~bcrl/patches/aio-2.6.13-rc6-B1-all.diff

with the split up in:

	http://www.kvack.org/~bcrl/patches/aio-2.6.13-rc6-B1/

This fixes the bugs noticed in the -B0 variant. Major changes in this patchset are:

- added aio semaphore ops
- aio thread based fallbacks
- vectored aio file_operations
- aio sendmsg/recvmsg via thread fallbacks
- retry based aio pipe operations

Comments?

-ben

 arch/i386/Kconfig              |    4 
 arch/i386/kernel/semaphore.c   |  185 +-----------
 arch/um/Kconfig_i386           |    4 
 arch/um/Kconfig_x86_64         |    4 
 arch/x86_64/Kconfig            |    4 
 arch/x86_64/kernel/Makefile    |    2 
 arch/x86_64/kernel/semaphore.c |  180 ------------
 arch/x86_64/lib/thunk.S        |    1 
 description                    |    9 
 drivers/usb/gadget/inode.c     |    7 
 fs/aio.c                       |  610 +++++++++++++++++++++++++++++++++++------
 fs/bad_inode.c                 |    2 
 fs/block_dev.c                 |    9 
 fs/buffer.c                    |    2 
 fs/ext2/file.c                 |    2 
 fs/ext3/file.c                 |   16 -
 fs/inode.c                     |    2 
 fs/jfs/file.c                  |    2 
 fs/ntfs/file.c                 |    2 
 fs/pipe.c                      |  194 ++++++++++---
 fs/read_write.c                |  182 ++++++++----
 fs/reiserfs/file.c             |   10 
 include/asm-i386/semaphore.h   |   42 ++
 include/asm-x86_64/semaphore.h |   44 ++
 include/linux/aio.h            |   38 ++
 include/linux/aio_abi.h        |   13 
 include/linux/fs.h             |    9 
 include/linux/net.h            |    4 
 include/linux/pagemap.h        |   29 +
 include/linux/sched.h          |   13 
 include/linux/wait.h           |   42 ++
 include/linux/writeback.h      |    2 
 kernel/exit.c                  |    2 
 kernel/fork.c                  |    7 
 kernel/sched.c                 |   14 
 kernel/wait.c                  |   40 +-
 lib/Makefile                   |    1 
 lib/semaphore-sleepers.c       |  253 +++++++++++++++++
 mm/filemap.c                   |  164 ++++++++---
 net/socket.c                   |   97 +++++-
 40 files changed, 1593 insertions(+), 654 deletions(-)
#1 lore raw
Quoted 11 lines · Benjamin LaHaise

The bugfix followup to the last aio rollup is now available at:

	http://www.kvack.org/~bcrl/patches/aio-2.6.13-rc6-B1-all.diff

with the split up in:

	http://www.kvack.org/~bcrl/patches/aio-2.6.13-rc6-B1/

This fixes the bugs noticed in the -B0 variant. Major changes in this patchset are:

Quoted 7 lines

- added aio semaphore ops
- aio thread based fallbacks
- vectored aio file_operations
- aio sendmsg/recvmsg via thread fallbacks
- retry based aio pipe operations

Comments?

Tried http://www.kvack.org/~bcrl/patches/aio-2.6.13-rc6-all.diff (since there is no aio-2.6.13-rc6-B1-all.diff). It hangs during bootup :(

...
Initializing IPsec netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 17
NET: Registered protocol family 15
Starting balanced_irq
Using IPI Shortcut mode
VFS: Cannot open root device "sda6" or unknown-block(8,6) Please append a correct "root=" boot option

Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(8,6)

Regards
Suparna

Quoted 49 lines

-ben

 arch/i386/Kconfig              |    4 
 arch/i386/kernel/semaphore.c   |  185 +-----------
 arch/um/Kconfig_i386           |    4 
 arch/um/Kconfig_x86_64         |    4 
 arch/x86_64/Kconfig            |    4 
 arch/x86_64/kernel/Makefile    |    2 
 arch/x86_64/kernel/semaphore.c |  180 ------------
 arch/x86_64/lib/thunk.S        |    1 
 description                    |    9 
 drivers/usb/gadget/inode.c     |    7 
 fs/aio.c                       |  610 +++++++++++++++++++++++++++++++++++------
 fs/bad_inode.c                 |    2 
 fs/block_dev.c                 |    9 
 fs/buffer.c                    |    2 
 fs/ext2/file.c                 |    2 
 fs/ext3/file.c                 |   16 -
 fs/inode.c                     |    2 
 fs/jfs/file.c                  |    2 
 fs/ntfs/file.c                 |    2 
 fs/pipe.c                      |  194 ++++++++++---
 fs/read_write.c                |  182 ++++++++----
 fs/reiserfs/file.c             |   10 
 include/asm-i386/semaphore.h   |   42 ++
 include/asm-x86_64/semaphore.h |   44 ++
 include/linux/aio.h            |   38 ++
 include/linux/aio_abi.h        |   13 
 include/linux/fs.h             |    9 
 include/linux/net.h            |    4 
 include/linux/pagemap.h        |   29 +
 include/linux/sched.h          |   13 
 include/linux/wait.h           |   42 ++
 include/linux/writeback.h      |    2 
 kernel/exit.c                  |    2 
 kernel/fork.c                  |    7 
 kernel/sched.c                 |   14 
 kernel/wait.c                  |   40 +-
 lib/Makefile                   |    1 
 lib/semaphore-sleepers.c       |  253 +++++++++++++++++
 mm/filemap.c                   |  164 ++++++++---
 net/socket.c                   |   97 +++++-
 40 files changed, 1593 insertions(+), 654 deletions(-)

--
To unsubscribe, send a message with 'unsubscribe linux-aio' in the body to [email protected]. For more info on Linux AIO,
see: http://www.kvack.org/aio/
Don't email: <a href=mailto:"[email protected]">[email protected]</a>

Signature · 3 lines
Suparna Bhattacharya ([email protected])
Linux Technology Center
IBM Software Lab, India
#2 lore raw
Quoted 4 lines · Suparna Bhattacharya

Using IPI Shortcut mode
VFS: Cannot open root device "sda6" or unknown-block(8,6) Please append a correct "root=" boot option

Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(8,6)

Are you sure the scsi driver is configured in? This doesn't look related to any of the changes in the patch. I'm going to be away from email for the next week, so you'll have to figure this out.

-ben

#3 lore raw
Quoted 9 lines · Benjamin LaHaise
Quoted 4 lines · Suparna Bhattacharya

Using IPI Shortcut mode
VFS: Cannot open root device "sda6" or unknown-block(8,6) Please append a correct "root=" boot option

Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(8,6)

Are you sure the scsi driver is configured in? This doesn't look related to any of the changes in the patch. I'm going to be away from email for the next week, so you'll have to figure this out.

You are right, it does look like a different problem - happens even without the patchset, just with vanilla 2.6.13-rc6. I'll refresh my build and try again to see if it is a real problem..

Regards
Suparna

Quoted 7 lines

-ben
--
To unsubscribe, send a message with 'unsubscribe linux-aio' in the body to [email protected]. For more info on Linux AIO,
see: http://www.kvack.org/aio/
Don't email: <a href=mailto:"[email protected]">[email protected]</a>

Signature · 3 lines
Suparna Bhattacharya ([email protected])
Linux Technology Center
IBM Software Lab, India
#4 lore raw
In reply to Benjamin LaHaise
Quoted 17 lines · Benjamin LaHaise

The bugfix followup to the last aio rollup is now available at:

	http://www.kvack.org/~bcrl/patches/aio-2.6.13-rc6-B1-all.diff

with the split up in:

	http://www.kvack.org/~bcrl/patches/aio-2.6.13-rc6-B1/

This fixes the bugs noticed in the -B0 variant. Major changes in this patchset are:

- added aio semaphore ops
- aio thread based fallbacks
- vectored aio file_operations
- aio sendmsg/recvmsg via thread fallbacks
- retry based aio pipe operations

Quoted 4 lines

Comments?

-ben

Hi Ben,

what's the point of calling wake_up_locked(&sem->wait) in
aio_down_wait? We're already in a wakeup path and end up calling __wake_up_common recursively.

I think it may be one of the cause of my kernel hanging at the
very beginning.

When I remove this call things go further but at some point a
semaphore wait queue gets thrashed and __wake_up_common tries to call an invalid callback function.

Any input appreciated.

Sébastien.

Signature · 10 lines
------------------------------------------------------

  Sébastien Dugué                BULL/FREC:B1-247
  phone: (+33) 476 29 77 70      Bullcom: 229-7770

  mailto:[email protected]

  Linux POSIX AIO: http://www.bullopensource.org/posix
  
------------------------------------------------------
#5 lore raw
Quoted 3 lines · Sébastien Dugué

what's the point of calling wake_up_locked(&sem->wait) in
aio_down_wait? We're already in a wakeup path and end up calling __wake_up_common recursively.

That's necessary to kick the next semaphore op in the list. The list_del_init() right above that makes sure that we don't recurse and run the routine again.

Quoted 6 lines

I think it may be one of the cause of my kernel hanging at the
very beginning.

When I remove this call things go further but at some point a
semaphore wait queue gets thrashed and __wake_up_common tries to call an invalid callback function.

This patch from Zach might make a difference. Let me know if it changes the symptoms at all. Sorry if it doesn't apply cleanly, as it is against a base kernel. Basically, we could sleep while holding ctx_lock, which does Bad Things(tm) on SMP systems.

-ben

fs/aio.c +4 -4
Index: 2.6.13-git12-lock-kiocb/fs/aio.c
===================================================================
@@ -398,7 +398,7 @@ static struct kiocb fastcall *__aio_get_
 	if (unlikely(!req))
 		return NULL;
 
-	req->ki_flags = 1 << KIF_LOCKED;
+	req->ki_flags = 0;
 	req->ki_users = 2;
 	req->ki_key = 0;
 	req->ki_ctx = ctx;
@@ -717,6 +717,8 @@ static ssize_t aio_run_iocb(struct kiocb
 	iocb->ki_run_list.next = iocb->ki_run_list.prev = NULL;
 	spin_unlock_irq(&ctx->ctx_lock);
 
+	lock_kiocb(iocb);
+
 	/* Quit retrying if the i/o has been cancelled */
 	if (kiocbIsCancelled(iocb)) {
 		ret = -EINTR;
@@ -781,6 +783,7 @@ out:
 			aio_queue_work(ctx);
 		}
 	}
+	unlock_kiocb(iocb);
 	return ret;
 }
 
@@ -805,9 +808,7 @@ static int __aio_run_iocbs(struct kioctx
 		 * Hold an extra reference while retrying i/o.
 		 */
 		iocb->ki_users++;       /* grab extra reference */
-		lock_kiocb(iocb);
 		aio_run_iocb(iocb);
-		unlock_kiocb(iocb);
 		if (__aio_put_req(ctx, iocb))  /* drop extra ref */
 			put_ioctx(ctx);
  	}
@@ -1549,7 +1550,6 @@ int fastcall io_submit_one(struct kioctx
 
 	spin_lock_irq(&ctx->ctx_lock);
 	aio_run_iocb(req);
-	unlock_kiocb(req);
 	if (!list_empty(&ctx->run_list)) {
 		/* drain the run list */
 		while (__aio_run_iocbs(ctx))
#6 lore raw
Quoted 8 lines · Benjamin LaHaise
Quoted 3 lines · Sébastien Dugué

what's the point of calling wake_up_locked(&sem->wait) in
aio_down_wait? We're already in a wakeup path and end up calling __wake_up_common recursively.

That's necessary to kick the next semaphore op in the list. The list_del_init() right above that makes sure that we don't recurse and run the routine again.

OK, understood.

Quoted 4 lines
Quoted 4 lines

I think it may be one of the cause of my kernel hanging at the
very beginning.

Quoted 9 lines
Quoted 3 lines

When I remove this call things go further but at some point a
semaphore wait queue gets thrashed and __wake_up_common tries to call an invalid callback function.

This patch from Zach might make a difference. Let me know if it changes the symptoms at all. Sorry if it doesn't apply cleanly, as it is against a base kernel. Basically, we could sleep while holding ctx_lock, which does Bad Things(tm) on SMP systems.

Well, it does not change a thing (I was not expecting it to). I think
the problem rather lies in the async semaphores (aio_down/aio_up) mechanism and not in the fs aio.

What leads me to this is that the crash occurs only when there is
contention on an inode semaphore, which seems to happen frequently with pipes and not so frequently with regular file IO.

And as the pipes are the only users (aside from xxx_aio_writev) of
this mecanism it shows right after the kernel is booted. Without the 90_pipe_aio.diff patch, the kernel boots normally but Oopses during shutdown involving again a semaphore operation.

Any ideas?

Sébastien.

Signature · 10 lines
------------------------------------------------------

  Sébastien Dugué                BULL/FREC:B1-247
  phone: (+33) 476 29 77 70      Bullcom: 229-7770

  mailto:[email protected]

  Linux POSIX AIO: http://www.bullopensource.org/posix
  
------------------------------------------------------