Skip to contentSkip to conversation

xfs: flush all log buffers in xlog_dealloc_log

PATCH In review Christoph Hellwig 2 participants 6 messages Feb 1, 2010 – Feb 5, 2010 XFS filesystem

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xfs: flush all log buffers in xlog_dealloc_log

Make sure all log buffers have made it to disk before we free them. Otherwise we might reference freed buffers or a NULL mp->m_log from the I/O completion handlers.

This fixes kernel.org bz #15150.

fs/xfs/xfs_log.c +2 -0
Index: linux-2.6/fs/xfs/xfs_log.c
===================================================================
@@ -1602,6 +1602,8 @@ xlog_dealloc_log(xlog_t *log)
 	xlog_in_core_t	*iclog, *next_iclog;
 	int		i;
 
+	xfs_flush_buftarg(log->l_mp->m_logdev_targp, 1);
+
 	iclog = log->l_iclog;
 	for (i=0; i<log->l_iclog_bufs; i++) {
 		sv_destroy(&iclog->ic_force_wait);

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#2 lore raw
Quoted 4 lines · Christoph Hellwig

Actually I'll take this one back - the log buffers aren't delwri so xfs_flush_buftarg probably only helped by timing or the xfs_buf_runall_queues on xfslogd_workqueue. Need to think about this a bit more.

Can you point me at the bug report? IIRC I've seen this in the past where we freed the log before we've done all the correct shutdown processing and they got fixed by correcting the order of shutdown to ensure the log is idle before freeing it...

Cheers,

Dave.

Signature · 7 lines
Dave Chinner
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#3 lore raw
Quoted 11 lines · Dave Chinner
Quoted 4 lines · Christoph Hellwig

Actually I'll take this one back - the log buffers aren't delwri so xfs_flush_buftarg probably only helped by timing or the xfs_buf_runall_queues on xfslogd_workqueue. Need to think about this a bit more.

Can you point me at the bug report? IIRC I've seen this in the past where we freed the log before we've done all the correct shutdown processing and they got fixed by correcting the order of shutdown to ensure the log is idle before freeing it...

Yes, that I was going to look into next. The recent report is http://bugzilla.kernel.org/show_bug.cgi?id=15150 and there's also a very similar one from Ed Cashin <[email protected]> in mail archives from November.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#4 lore raw
Quoted 7 lines · Christoph Hellwig
Quoted 6 lines · Dave Chinner
Quoted 4 lines · Christoph Hellwig

Actually I'll take this one back - the log buffers aren't delwri so xfs_flush_buftarg probably only helped by timing or the xfs_buf_runall_queues on xfslogd_workqueue. Need to think about this a bit more.

Quoted 8 lines
Quoted 5 lines

Can you point me at the bug report? IIRC I've seen this in the past where we freed the log before we've done all the correct shutdown processing and they got fixed by correcting the order of shutdown to ensure the log is idle before freeing it...

Yes, that I was going to look into next. The recent report is http://bugzilla.kernel.org/show_bug.cgi?id=15150

That was from 2.6.27. I didn't think that we finished the mount/unmount path cleanup that fixed all those problems until 2.6.28 or .29...

Quoted 3 lines

and there's
also a very similar one from Ed Cashin <[email protected]> in mail archives from November.

Yeah, that's clearly IO completion after the call to xfs_unmountfs_wait(). That does a xfs_wait_buftarg() call which waits for all hashed buffers on the buftarg except for those that are XBF_FS_MANAGED - the superblock.

Looking at xfs_fs_put_super(), the superblock buffer is freed after xfs_unmountfs() frees the log, so if the superblock was logged and is undergoing IO then xfs_unmountfs_wait() would not have blocked on it and it could leak past the "no more IO should bein progress" barrier that xfs_unmountfs_wait() is supposed to provide.

Does this look like a plausible cause of the problem?

Is so, given that the superblock is the only buffer that is marked XBF_FS_MANAGED now and it seems to me that xfs_wait_buftarg() should be waiting on it, maybe the fix needed is to xfs_wait_buftarg()....

Cheers,

Dave.

Signature · 7 lines
Dave Chinner
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#5 lore raw
Quoted 9 lines · Dave Chinner
Quoted 3 lines

and there's
also a very similar one from Ed Cashin <[email protected]> in mail archives from November.

Yeah, that's clearly IO completion after the call to xfs_unmountfs_wait(). That does a xfs_wait_buftarg() call which waits for all hashed buffers on the buftarg except for those that are XBF_FS_MANAGED - the superblock.

Quoted 8 lines

Looking at xfs_fs_put_super(), the superblock buffer is freed after xfs_unmountfs() frees the log, so if the superblock was logged and is undergoing IO then xfs_unmountfs_wait() would not have blocked on it and it could leak past the "no more IO should bein progress" barrier that xfs_unmountfs_wait() is supposed to provide.

Does this look like a plausible cause of the problem?

We just did a xfs_unmountfs_writesb before the call to xfs_unmountfs_wait, so the superblock buffer should be a sync buffer at that point, so something more fishy is going on here. syncd is also stopped before that call, so that gets rid of another possible case of an async buffer pending.

Independent of that I think xfs_wait_buftarg waiting for the SB buffer seems correct to me, but I fear something else is going on here.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs