Skip to content

mkfs.xfs IO scalability

Dave Chinner linux-xfs 2 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest4 patches 9 repliesIn review
  2. v1 RFC2 patches 0 repliesSuperseded

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/4mkfs: stop zeroing old superblocks excessivelymkfs/xfs_mkfs.c+59 -53 replies
2/4mkfs: rework AG header initialisation orderingmkfs/xfs_mkfs.c+25 -152 replies
3/4mkfs: introduce new delayed write buffer listinclude/xfs_trans.h libxfs/libxfs_io.h libxfs/rdwr.c +2 more+131 -262 replies
4/4mkfs: Use AIO for batched writebacklibxfs/libxfs_io.h libxfs/rdwr.c mkfs/Makefile +1 more+169 -62 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v2 yet.

Read the v2 thread

Cover letter [RFCRAP PATCH 0/4 v2] mkfs.xfs IO scalability

Hi folks,

More on getting mkfs to be usable to testing unrealistically large filesystems. The first two patches of this series are unchanged from yesterday - the second two are new and build on them.

The second two patches hack a delayed write buffer submission list in the mkfs and libxfs. It's a bit nasty, because I've chosen to ignore the fact that the libxfs has no concept of async IO or background write and instead hacked around it. You can see the result in passing a buffer list to xfs_trans_commit() to get it to add buffers to the delwri list rather than write them synchronously.

Fast, loose and stupidly dangerous, all in one. Yeehaw!

Better yet, it doesn't even make any difference to performance - it's just an enabling patch.

The last patch is the performance improvement - it hacks a grotty, non-re-entrant AIO submission/completion ring to turn the single threaded sync write batching into a single threaded concurrent IO loop using AIO. This can drive really deep IO queues as long as it's got enough queued IO to work with, so mkfs is hacked to only submit IO every few hundred AGs it initialises.

This sustains queue depths of around 100 IOs and SSD utilisation at around 80% using about half a CPU, and so the time to make an 8EB filesystem drops to around 15 minutes.

This is most definitely not production code. This is a load of crap hacked together in a few hours as a proof of concept. But it's a successful proof of concept, so what we now need is someone who is looking around for a substantial project to volunteer to rewrite the libxfs buffer cache around an AIO submission/completion core and implement all this in a "proper" fashion. If you're interested, let me know...

Cheers,

Dave.