Skip to content

RFC xfs: new icreate transaction

Dave Chinner linux-xfs 1 version New

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest7 patches 0 repliesNew

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/7xfs: don't do IO when creating an new inodefs/xfs/xfs_inode.c+28 -80 replies
2/7xfs: xfs_ifree doesn't need to modify the inode bufferfs/xfs/xfs_inode.c+4 -280 replies
3/7xfs: Introduce an ordered buffer itemfs/xfs/xfs_buf_item.c fs/xfs/xfs_buf_item.h fs/xfs/xfs_trace.h +2 more+84 -310 replies
4/7xfs: Inode create log itemsfs/xfs/Makefile fs/xfs/xfs_icreate_item.c fs/xfs/xfs_icreate_item.h +3 more+260 -20 replies
5/7xfs: Inode create transaction reservationsfs/xfs/xfs_trans.c+77 -410 replies
6/7xfs: Use inode create transactionfs/xfs/xfs_ialloc.c fs/xfs/xfs_ialloc.h+37 -90 replies
7/7xfs: Inode create item recoveryfs/xfs/xfs_ialloc.c fs/xfs/xfs_ialloc.h fs/xfs/xfs_log_recover.c+113 -100 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [RFC, PATCH 0/7] xfs: new icreate transaction

Hi folks,

As ben and I discussed during the review of the initial CRC series, inode allocation needs to log the entire inode to ensure the replayed create transaction results in an inode with the correct CRC. This means that the logging overhead of inode create doubled for 256 byte inodes, and is close to 5x higher for 512 byte inodes.

Ben suggested that having a transaction to initialise buffers to zero without needing to log them physically might be a way to solve the problem. It would solve the problem, but I already have a patchset from a few years back that introduces a new inode create transaction that doesn't require any physical logging on inodes at all.

This patch series is a forward port of my original work from 2009 (hence the SOBs being from [email protected]) with a couple of more recent patches that will also help reduce inode buffer lookups and hence improve performance.

The first two patches are for reducing he number of inode buffer lookups. When we are allocating a new inode, the only reason we look up the inode buffer is to read the generation number so we can increment it. This patch replaces the inode buffer read with radomly calculating a new generation number, resulting in an inode allocation being a purely in-memory operation requiring no IO. There is a caveat to that - for people using noikeep, we still need to ensure the generation number increments monotonically so we only take the new path if that mount option is not set. This reduces buffer lookups under create heavy workloads by roughly 10%.

The second patch removes a buffer lookup and modification on unlink that was added for coherency with bulkstat back when bulkstat did non-cohernet inode lookups. bulkstat is using coherent lookups again, so the code in unlink is not necessary any more.

The remaining 5 patches are the new icreate transaction series. The first patch introduced ordered buffers. These are buffers that are modified in transactions but are not logged by the transaction. They have an identical lifecycle to a normal buffer, and so pin the tail ofthe log until they are written back. This enables us to do log a logical change and have all the physical changes behave as though physical logging had been performed. This is used for the inode buffers by the new icreate transaction.

The rest of the patches are simply mechanical - introducing the inode create log item, the changes to transaction reservations (uses less space in the log), converting the code to selectively use the new logging method and adding recovery support to it.

Right now the code will use this transaction if the filesystem is CRC enabled. Given that CRC enabled filesystems are experimental at this point, adding a new log item type should not be a major problem for anyone using them - just make sure the log is clean before downgrading to an older kernel...

The patchset passes xfstests on non-CRC filesystems without new regressions and the initial two patches are resulting in a ~10% improvement in 8-way create speed and a ~15% improvement in 8-way unlink speed. I don't have any numbers on CRC enabled filesystems as I've been working on the userspace CRC patchset and getting that into shape rather than tesing and benchmarking kernel CRC code...

Comments, thoughts, flames?

-Dave.

PS. I'm working on an equivalent patchset for unlink that logs the the unlinked list as part of the inode core for CRC enabled filesystems. That's a little bit away from working yet, though...

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs