XFS doesn't correctly account for IO-Wait for directory reading

Hi

Some days ago Mr. Chinner(?, don't have the e-mail anymore) said that XFS fakes ( :-) ) it's way around IO-wait accounting for file-deletion by deferring it to the log.

Today i thought again about the initial 'rm -rf'-isn't-accounted-properly "problem", and the bigger part of "rm -rf" is the directory-traversal(IOW read) and not the actual "unlink"-part.

So what better test than a simple 'find'.

Situation: Cache is cold:
find /<wherever> >/dev/null
While running (which takes some time) it shows exactly 0.0%wa in top on an otherwise completely idle system, where there should be a near 50%wa (Dual-Core system) or 100% on a UP system.

For plain old file-reading the IO-Wait appears to show correctly (AFAICT). In a short test:
cat <somelargefiles> > /dev/null
peaked at 48%wa (said Dual-Core) with an average around 45%wa.

SO how das XFS fake around IO-wait accounting this time?

Unfortunatly i don't have any sufficiently large non-XFS-filesystems to do a good(tm) comparison, but a test on a small ext3-fs appeared to correctly(tm) account IO-Wait in directory traversal.

Tested Kernels: 2.6.23 & 2.6.24rc6

Bis denn

Real Programmers consider "what you see is what you get" to be just as 
bad a concept in Text Editors as it is in women. No, the Real Programmer
wants a "you asked for it, you got it" text editor -- complicated, 
cryptic, powerful, unforgiving, dangerous.

Try this one-liner patch which should give you much better I/O wait reporting. There's some more I/O waits hidden in the log code, but to fix this we'd need to dig into the sv_t abstraction. Given that it only has four users left I'm probably going to simply remove it and fix the I/O wait accounting while I'm at it.

@@ -976,7 +976,7 @@ xfs_buf_wait_unpin(
 			break;
 		if (atomic_read(&bp->b_io_remaining))
 			blk_run_address_space(bp->b_target->bt_mapping);
-		schedule();
+		io_schedule();
 	}
 	remove_wait_queue(&bp->b_waiters, &wait);
 	set_current_state(TASK_RUNNING);

Either with 2.6.23.12 & 2.6.24-git-HEAD, and the shown line changed, the %wa reported by top is still at 0.0

Bis denn

Real Programmers consider "what you see is what you get" to be just as 
bad a concept in Text Editors as it is in women. No, the Real Programmer
wants a "you asked for it, you got it" text editor -- complicated, 
cryptic, powerful, unforgiving, dangerous.

XFS issues readahead on directories, so when the I/O has not completed, it waits on semaphores in the xfs_buf layer, not on the I/O itself. Hence we cannot account for the wait time as iowait as we cannot tell the semaphore to call io_schedule() instead of schedule().

Cheers,

Dave.

Dave Chinner
Principal Engineer
SGI Australian Software Group