xfs: Non-blocking inode locking in IO completion
PATCH 1/1 xfs: Non-blocking inode locking in IO completion
The introduction of barriers to DM loop devices (e.g. dm-crypt) has created a new IO order completion dependency that XFS does not handle. That is, the completion of log IOs (which have barriers) in the loop filesystem are now dependent on completion of data IO in the backing filesystem.
This can cause deadlocks when a flush daemon issues a log force with an inode locked because the IO completion of IO on the inode is blocked by the inode lock. This in turn prevents further data IO completion from occuring, which prevents the log IO from completing because DM is waiting for data Io completion as well.
The fix for this new completion order dependency issue is to make the IO completion inode locking non-blocking. If the inode lock can't be grabbed, simply requeue the IO completion back to the work queue so that it can be processed later. This prevents the completion queue from being blocked and allows data IO completion on other inodes to proceed, hence avoiding completion order dependent deadlocks.
--- fs/xfs/linux-2.6/xfs_aops.c | 93 ++++++++++++++++++++++++++----------------- 1 files changed, 56 insertions(+), 37 deletions(-)
fs/xfs/linux-2.6/xfs_aops.c +56 -37
@@ -163,14 +163,17 @@ xfs_ioend_new_eof( }/* -* Update on-disk file size now that data has been written to disk. -* The current in-memory file size is i_size. If a write is beyond -* eof i_new_size will be the intended file size until i_size is -* updated. If this write does not extend all the way to the valid -* file size then restrict this update to the end of the write. +* Update on-disk file size now that data has been written to disk. The +* current in-memory file size is i_size. If a write is beyond eof i_new_size +* will be the intended file size until i_size is updated. If this write does +* not extend all the way to the valid file size then restrict this update to +* the end of the write. +* +* This function does not block as blocking on the inode lock in IO completion +* can lead to IO completion order dependency deadlocks.. If it can't get the +* inode ilock it will return EAGAIN. Callers must handle this. */ - -STATIC void +STATIC intxfs_setfilesize (xfs_ioend_t *ioend) {
@@ -181,9 +184,11 @@ xfs_setfilesize(ASSERT (ioend->io_type !=IOMAP_READ );if (unlikely (ioend->io_error)) -return ; +return 0 ; + +if (!xfs_ilock_nowait (ip,XFS_ILOCK_EXCL )) +return EAGAIN ; -xfs_ilock (ip,XFS_ILOCK_EXCL ); isize =xfs_ioend_new_eof (ioend);if (isize) { ip->i_d.di_size = isize;
@@ -191,6 +196,28 @@ xfs_setfilesize( }xfs_iunlock (ip,XFS_ILOCK_EXCL ); +return 0 ; +} + +/* +* Schedule IO completion handling on a xfsdatad if this was +* the final hold on this ioend. If we are asked to wait, +* flush the workqueue. +*/ +STATIC void +xfs_finish_ioend ( +xfs_ioend_t *ioend, +int wait) +{ +if (atomic_dec_and_test (&ioend->io_remaining)) { +struct workqueue_struct *wq; + + wq = (ioend->io_type ==IOMAP_UNWRITTEN ) ? + xfsconvertd_workqueue : xfsdatad_workqueue; +queue_work (wq, &ioend->io_work); +if (wait) +flush_workqueue (wq); + } }/*
@@ -198,11 +225,11 @@ xfs_setfilesize(*/ STATIC voidxfs_end_io ( -struct work_struct *work) +struct work_struct *work) { -xfs_ioend_t *ioend = -container_of (work, xfs_ioend_t, io_work); -struct xfs_inode *ip =XFS_I (ioend->io_inode); +xfs_ioend_t *ioend =container_of (work, xfs_ioend_t, io_work); +struct xfs_inode *ip =XFS_I (ioend->io_inode); +int error;/* * For unwritten extents we need to issue transactions to convert a
@@ -210,7 +237,6 @@ xfs_end_io(*/ if (ioend->io_type ==IOMAP_UNWRITTEN &&likely (!ioend->io_error && !XFS_FORCED_SHUTDOWN (ip->i_mount))) { -int error; error =xfs_iomap_write_unwritten (ip, ioend->io_offset, ioend->io_size);
@@ -222,30 +248,23 @@ xfs_end_io(* We might have to update the on-disk file size after extending * writes. */ -if (ioend->io_type !=IOMAP_READ ) -xfs_setfilesize (ioend); -xfs_destroy_ioend (ioend); -} - -/* -* Schedule IO completion handling on a xfsdatad if this was -* the final hold on this ioend. If we are asked to wait, -* flush the workqueue. -*/ -STATIC void -xfs_finish_ioend ( -xfs_ioend_t *ioend, -int wait) -{ -if (atomic_dec_and_test (&ioend->io_remaining)) { -struct workqueue_struct *wq; - - wq = (ioend->io_type ==IOMAP_UNWRITTEN ) ? - xfsconvertd_workqueue : xfsdatad_workqueue; -queue_work (wq, &ioend->io_work); -if (wait) -flush_workqueue (wq); +if (ioend->io_type !=IOMAP_READ ) { + error =xfs_setfilesize (ioend); +ASSERT (!error || error ==EAGAIN ); } + +/* +* If we didn't complete processing of the ioend, requeue it to the +* tail of the workqueue for another attempt later. Otherwise destroy +* it. +*/ +if (error ==EAGAIN ) { +atomic_inc (&ioend->io_remaining); +xfs_finish_ioend (ioend,0 ); +/* ensure we don't spin on blocked ioends */ +delay (1 ); + }else +xfs_destroy_ioend (ioend); }/*
Signature · 6 lines
1.6.5 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 5 lines · Dave Chinner
The introduction of barriers to DM loop devices (e.g. dm-crypt) has created a new IO order completion dependency that XFS does not handle. That is, the completion of log IOs (which have barriers) in the loop filesystem are now dependent on completion of data IO in the backing filesystem.
I don't think dm belongs into the picture here at all. The problem is simply with the loop device, which sits below dm-crypt in the bugzilla reports. The loop device in SuSE (and for a short time in mainline until we saw unexplainable XFS lockups) implements barriers using fsync. Now that fsync turns a log I/O that issues a barrier in the XFS filesystem inside the loop device into a data I/O the backing filesystem. With this the rest of your description applies again.
The patch looks good to me - while I hate introducing random delay() calls I don't really see a way around this.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 7 lines · Christoph Hellwig
Quoted 5 lines · Dave Chinner
The introduction of barriers to DM loop devices (e.g. dm-crypt) has created a new IO order completion dependency that XFS does not handle. That is, the completion of log IOs (which have barriers) in the loop filesystem are now dependent on completion of data IO in the backing filesystem.
Quoted 8 lines
I don't think dm belongs into the picture here at all. The problem is simply with the loop device, which sits below dm-crypt in the bugzilla reports. The loop device in SuSE (and for a short time in mainline until we saw unexplainable XFS lockups) implements barriers using fsync. Now that fsync turns a log I/O that issues a barrier in the XFS filesystem inside the loop device into a data I/O the backing filesystem. With this the rest of your description applies again.
Fair point. I'll change the description to be more accurate.
Quoted 2 lines
The patch looks good to me - while I hate introducing random
delay()calls I don't really see a way around this.
I thought about using queue_delayed_work(), but then the change became much bigger and has other side effects like increasing the size of the ioend structure.
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 6 lines · Dave Chinner
Quoted 2 lines
The patch looks good to me - while I hate introducing random
delay()calls I don't really see a way around this.I thought about using
queue_delayed_work(), but then the change became much bigger and has other side effects like increasing the size of the ioend structure.
Yes, now that the normal work struct and the delayed work struct are different it would be a pain, agreed.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH v2 1/1 xfs: Non-blocking inode locking in IO completion
Quoted 10 lines · Christoph Hellwig
Quoted 6 lines · Dave Chinner
Quoted 2 lines
The patch looks good to me - while I hate introducing random
delay()calls I don't really see a way around this.I thought about using
queue_delayed_work(), but then the change became much bigger and has other side effects like increasing the size of the ioend structure.Yes, now that the normal work struct and the delayed work struct are different it would be a pain, agreed.
Version with updated commit message below.
Cheers,
Dave.
Signature · 33 lines
Dave Chinner [email protected] xfs: Non-blocking inode locking in IO completion The introduction of barriers to loop devices has created a new IO order completion dependency that XFS does not handle. The loop device implements barriers using fsync and so turns a log IO in the XFS filesystem on the loop device into a data IO in the backing filesystem. That is, the completion of log IOs in the loop filesystem are now dependent on completion of data IO in the backing filesystem. This can cause deadlocks when a flush daemon issues a log force with an inode locked because the IO completion of IO on the inode is blocked by the inode lock. This in turn prevents further data IO completion from occuring on all XFS filesystems on that CPU (due to the shared nature of the completion queues). This then prevents the log IO from completing because the log is waiting for data IO completion as well. The fix for this new completion order dependency issue is to make the IO completion inode locking non-blocking. If the inode lock can't be grabbed, simply requeue the IO completion back to the work queue so that it can be processed later. This prevents the completion queue from being blocked and allows data IO completion on other inodes to proceed, hence avoiding completion order dependent deadlocks. Signed-off-by: Dave Chinner <[email protected]> --- fs/xfs/linux-2.6/xfs_aops.c | 93 ++++++++++++++++++++++++++----------------- 1 files changed, 56 insertions(+), 37 deletions(-)
fs/xfs/linux-2.6/xfs_aops.c +56 -37
@@ -163,14 +163,17 @@ xfs_ioend_new_eof( }/* -* Update on-disk file size now that data has been written to disk. -* The current in-memory file size is i_size. If a write is beyond -* eof i_new_size will be the intended file size until i_size is -* updated. If this write does not extend all the way to the valid -* file size then restrict this update to the end of the write. +* Update on-disk file size now that data has been written to disk. The +* current in-memory file size is i_size. If a write is beyond eof i_new_size +* will be the intended file size until i_size is updated. If this write does +* not extend all the way to the valid file size then restrict this update to +* the end of the write. +* +* This function does not block as blocking on the inode lock in IO completion +* can lead to IO completion order dependency deadlocks.. If it can't get the +* inode ilock it will return EAGAIN. Callers must handle this. */ - -STATIC void +STATIC intxfs_setfilesize (xfs_ioend_t *ioend) {
@@ -181,9 +184,11 @@ xfs_setfilesize(ASSERT (ioend->io_type !=IOMAP_READ );if (unlikely (ioend->io_error)) -return ; +return 0 ; + +if (!xfs_ilock_nowait (ip,XFS_ILOCK_EXCL )) +return EAGAIN ; -xfs_ilock (ip,XFS_ILOCK_EXCL ); isize =xfs_ioend_new_eof (ioend);if (isize) { ip->i_d.di_size = isize;
@@ -191,6 +196,28 @@ xfs_setfilesize( }xfs_iunlock (ip,XFS_ILOCK_EXCL ); +return 0 ; +} + +/* +* Schedule IO completion handling on a xfsdatad if this was +* the final hold on this ioend. If we are asked to wait, +* flush the workqueue. +*/ +STATIC void +xfs_finish_ioend ( +xfs_ioend_t *ioend, +int wait) +{ +if (atomic_dec_and_test (&ioend->io_remaining)) { +struct workqueue_struct *wq; + + wq = (ioend->io_type ==IOMAP_UNWRITTEN ) ? + xfsconvertd_workqueue : xfsdatad_workqueue; +queue_work (wq, &ioend->io_work); +if (wait) +flush_workqueue (wq); + } }/*
@@ -198,11 +225,11 @@ xfs_setfilesize(*/ STATIC voidxfs_end_io ( -struct work_struct *work) +struct work_struct *work) { -xfs_ioend_t *ioend = -container_of (work, xfs_ioend_t, io_work); -struct xfs_inode *ip =XFS_I (ioend->io_inode); +xfs_ioend_t *ioend =container_of (work, xfs_ioend_t, io_work); +struct xfs_inode *ip =XFS_I (ioend->io_inode); +int error =0 ;/* * For unwritten extents we need to issue transactions to convert a
@@ -210,7 +237,6 @@ xfs_end_io(*/ if (ioend->io_type ==IOMAP_UNWRITTEN &&likely (!ioend->io_error && !XFS_FORCED_SHUTDOWN (ip->i_mount))) { -int error; error =xfs_iomap_write_unwritten (ip, ioend->io_offset, ioend->io_size);
@@ -222,30 +248,23 @@ xfs_end_io(* We might have to update the on-disk file size after extending * writes. */ -if (ioend->io_type !=IOMAP_READ ) -xfs_setfilesize (ioend); -xfs_destroy_ioend (ioend); -} - -/* -* Schedule IO completion handling on a xfsdatad if this was -* the final hold on this ioend. If we are asked to wait, -* flush the workqueue. -*/ -STATIC void -xfs_finish_ioend ( -xfs_ioend_t *ioend, -int wait) -{ -if (atomic_dec_and_test (&ioend->io_remaining)) { -struct workqueue_struct *wq; - - wq = (ioend->io_type ==IOMAP_UNWRITTEN ) ? - xfsconvertd_workqueue : xfsdatad_workqueue; -queue_work (wq, &ioend->io_work); -if (wait) -flush_workqueue (wq); +if (ioend->io_type !=IOMAP_READ ) { + error =xfs_setfilesize (ioend); +ASSERT (!error || error ==EAGAIN ); } + +/* +* If we didn't complete processing of the ioend, requeue it to the +* tail of the workqueue for another attempt later. Otherwise destroy +* it. +*/ +if (error ==EAGAIN ) { +atomic_inc (&ioend->io_remaining); +xfs_finish_ioend (ioend,0 ); +/* ensure we don't spin on blocked ioends */ +delay (1 ); + }else +xfs_destroy_ioend (ioend); }/*
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 5 lines · Dave Chinner
Quoted 4 lines · Christoph Hellwig
Quoted 3 lines · Dave Chinner
Quoted 2 lines
The patch looks good to me - while I hate introducing random
delay()calls I don't really see a way around this.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
I thought about using
queue_delayed_work(), but then the change became much bigger and has other side effects like increasing the size of the ioend structure.Yes, now that the normal work struct and the delayed work struct are different it would be a pain, agreed.
Version with updated commit message below.
This looks good and I'll be incorporating this version. Thanks for updating it.
-Alex
Quoted 16 lines
Cheers,
Dave.
Signature · 12 lines
Dave Chinner [email protected] xfs: Non-blocking inode locking in IO completion The introduction of barriers to loop devices has created a new IO order completion dependency that XFS does not handle. The loop device implements barriers using fsync and so turns a log IO in the XFS filesystem on the loop device into a data IO in the backing filesystem. That is, the completion of log IOs in the loop filesystem are now dependent on completion of data IO in the backing filesystem.
. . .
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 5 lines · Dave Chinner
The introduction of barriers to DM loop devices (e.g. dm-crypt) has created a new IO order completion dependency that XFS does not handle. That is, the completion of log IOs (which have barriers) in the loop filesystem are now dependent on completion of data IO in the backing filesystem.
One comment, below
. . .
fs/xfs/linux-2.6/xfs_aops.c xfs_end_io( · line 261
Quoted 11 lines
fs/xfs/linux-2.6/xfs_aops.c +10
-0+if (ioend->io_type !=IOMAP_READ ) { + error =xfs_setfilesize (ioend); +ASSERT (!error || error ==EAGAIN ); } + +/* +* If we didn't complete processing of the ioend, requeue it to the +* tail of the workqueue for another attempt later. Otherwise destroy +* it. +*/ +if (error ==EAGAIN ) {
It's not a problem now (and may never be), but it's conceivable error could have been set to the return value from xfs_iomap_write_unwritten() to have value EAGAIN. It might have been better to include this block inside the if (ioend->io_type != IOMAP_READ) { block, above.
(I'll take it as is, however...)
-Alex
fs/xfs/linux-2.6/xfs_aops.c xfs_end_io( · line 270
Quoted 9 lines
fs/xfs/linux-2.6/xfs_aops.c +6
-0+atomic_inc (&ioend->io_remaining); +xfs_finish_ioend (ioend,0 ); +/* ensure we don't spin on blocked ioends */ +delay (1 ); + }else +xfs_destroy_ioend (ioend); }/*
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs