xfs: new icreate transaction
cover xfs: new icreate transaction
Hi folks,
As ben and I discussed during the review of the initial CRC series, inode allocation needs to log the entire inode to ensure the replayed create transaction results in an inode with the correct CRC. This means that the logging overhead of inode create doubled for 256 byte inodes, and is close to 5x higher for 512 byte inodes.
Ben suggested that having a transaction to initialise buffers to zero without needing to log them physically might be a way to solve the problem. It would solve the problem, but I already have a patchset from a few years back that introduces a new inode create transaction that doesn't require any physical logging on inodes at all.
This patch series is a forward port of my original work from 2009 (hence the SOBs being from [email protected]) with a couple of more recent patches that will also help reduce inode buffer lookups and hence improve performance.
The first two patches are for reducing he number of inode buffer lookups. When we are allocating a new inode, the only reason we look up the inode buffer is to read the generation number so we can increment it. This patch replaces the inode buffer read with radomly calculating a new generation number, resulting in an inode allocation being a purely in-memory operation requiring no IO. There is a caveat to that - for people using noikeep, we still need to ensure the generation number increments monotonically so we only take the new path if that mount option is not set. This reduces buffer lookups under create heavy workloads by roughly 10%.
The second patch removes a buffer lookup and modification on unlink that was added for coherency with bulkstat back when bulkstat did non-cohernet inode lookups. bulkstat is using coherent lookups again, so the code in unlink is not necessary any more.
The remaining 5 patches are the new icreate transaction series. The first patch introduced ordered buffers. These are buffers that are modified in transactions but are not logged by the transaction. They have an identical lifecycle to a normal buffer, and so pin the tail ofthe log until they are written back. This enables us to do log a logical change and have all the physical changes behave as though physical logging had been performed. This is used for the inode buffers by the new icreate transaction.
The rest of the patches are simply mechanical - introducing the inode create log item, the changes to transaction reservations (uses less space in the log), converting the code to selectively use the new logging method and adding recovery support to it.
Right now the code will use this transaction if the filesystem is CRC enabled. Given that CRC enabled filesystems are experimental at this point, adding a new log item type should not be a major problem for anyone using them - just make sure the log is clean before downgrading to an older kernel...
The patchset passes xfstests on non-CRC filesystems without new regressions and the initial two patches are resulting in a ~10% improvement in 8-way create speed and a ~15% improvement in 8-way unlink speed. I don't have any numbers on CRC enabled filesystems as I've been working on the userspace CRC patchset and getting that into shape rather than tesing and benchmarking kernel CRC code...
Comments, thoughts, flames?
-Dave.
PS. I'm working on an equivalent patchset for unlink that logs the the unlinked list as part of the inode core for CRC enabled filesystems. That's a little bit away from working yet, though...
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH 1/7 xfs: don't do IO when creating an new inode
From: Dave Chinner <[email protected]>
When we are allocating a new inode, we read the inode cluster off disk to increment the generation number. We are already using a random generation number for newly allocated inodes, so if we are not using the ikeep mode, we can just generate a new generation number when we initialise the newly allocated inode.
This avoids the need for reading the inode buffer during inode creation. This will speed up allocation of inodes in cold, partially allocated clusters as they will no longer need to be read from disk during allocation. It will also reduce the CPU overhead of inode allocation by not having the process the buffer read, even on cache hits.
--- fs/xfs/xfs_inode.c | 36 ++++++++++++++++++++++++++++-------- 1 file changed, 28 insertions(+), 8 deletions(-)
fs/xfs/xfs_inode.c +28 -8
@@ -1028,6 +1028,11 @@ xfs_dinode_calc_crc(/* * Read the disk inode attributes into the in-core inode structure. +* +* If we are initialising a new inode and we are not utilising the +* XFS_MOUNT_IKEEP inode cluster mode, we can simple build the new inode core +* with a random generation number. If we are keeping inodes around, we need to +* read the inode cluster to get the existing generation number off disk. */ int xfs_iread(
@@ -1047,6 +1052,22 @@ xfs_iread(if (error)return error; +/* shortcut IO on inode allocation if possible */ +if ((iget_flags &XFS_IGET_CREATE ) && + !(mp->m_flags &XFS_MOUNT_IKEEP )) { +/* initialise the on-disk inode core */ +memset (&ip->i_d,0 ,sizeof (ip->i_d)); + ip->i_d.di_magic =XFS_DINODE_MAGIC ; + ip->i_d.di_gen =prandom_u32 (); +if (xfs_sb_version_hascrc (&mp->m_sb)) { + ip->i_d.di_version =3 ; + ip->i_d.di_ino = ip->i_ino; +uuid_copy (&ip->i_d.di_uuid, &mp->m_sb.sb_uuid); + }else + ip->i_d.di_version =2 ; +return 0 ; + } +/* * Get pointers to the on-disk inode and the buffer containing it. */
@@ -1133,17 +1154,16 @@ xfs_iread(xfs_buf_set_ref (bp,XFS_INO_REF );/* -* Use xfs_trans_brelse() to release the buffer containing the -* on-disk inode, because it was acquired with xfs_trans_read_buf() -* in xfs_imap_to_bp() above. If tp is NULL, this is just a normal +* Use xfs_trans_brelse() to release the buffer containing the on-disk +* inode, because it was acquired with xfs_trans_read_buf() in +* xfs_imap_to_bp() above. If tp is NULL, this is just a normal * brelse(). If we're within a transaction, then xfs_trans_brelse() * will only release the buffer if it is not dirty within the * transaction. It will be OK to release the buffer in this case, -* because inodes on disk are never destroyed and we will be -* locking the new in-core inode before putting it in the hash -* table where other processes can find it. Thus we don't have -* to worry about the inode being changed just because we released -* the buffer. +* because inodes on disk are never destroyed and we will be locking the +* new in-core inode before putting it in the cache where other +* processes can find it. Thus we don't have to worry about the inode +* being changed just because we released the buffer. */ out_brelse:xfs_trans_brelse (tp, bp);
Signature · 6 lines
1.7.10.4 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
PATCH 2/7 xfs: xfs_ifree doesn't need to modify the inode buffer
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Long ago, bulkstat used to read inodes directly fromteh backing buffer for speed. This had the unfortunate problem of being cache incoherent with unlinks, and so xfs_ifree() had to mark the inode as free directly in the backing buffer. bulkstat was changed some time ago to use inode cache coherent lookups, and so will never see unlinked inodes in it's lookups. Hence xfs_ifree() does not need to touch the inode backing buffer anymore.
--- fs/xfs/xfs_inode.c | 32 ++++---------------------------- 1 file changed, 4 insertions(+), 28 deletions(-)
fs/xfs/xfs_inode.c +4 -28
@@ -2032,8 +2032,6 @@ xfs_ifree(int error;int delete;xfs_ino_t first_ino; -xfs_dinode_t *dip; -xfs_buf_t *ibp;ASSERT (xfs_isilocked (ip,XFS_ILOCK_EXCL ));ASSERT (ip->i_d.di_nlink ==0 );
@@ -2046,14 +2044,13 @@ xfs_ifree(* Pull the on-disk inode from the AGI unlinked list. */ error =xfs_iunlink_remove (tp, ip); -if (error !=0 ) { +if (error)return error; - } error =xfs_difree (tp, ip->i_ino, flist, &delete, &first_ino); -if (error !=0 ) { +if (error)return error; - } + ip->i_d.di_mode =0 ;/* mark incore inode as free */ ip->i_d.di_flags =0 ; ip->i_d.di_dmevmask =0 ;
@@ -2065,31 +2062,10 @@ xfs_ifree(* by reincarnations of this inode. */ ip->i_d.di_gen++; -xfs_trans_log_inode (tp, ip,XFS_ILOG_CORE ); - error =xfs_imap_to_bp (ip->i_mount, tp, &ip->i_imap, &dip, &ibp, -0 ,0 ); -if (error) -return error; - -/* -* Clear the on-disk di_mode. This is to prevent xfs_bulkstat -* from picking up this inode when it is reclaimed (its incore state -* initialzed but not flushed to disk yet). The in-core di_mode is -* already cleared and a corresponding transaction logged. -* The hack here just synchronizes the in-core to on-disk -* di_mode value in advance before the actual inode sync to disk. -* This is OK because the inode is already unlinked and would never -* change its di_mode again for this inode generation. -* This is a temporary hack that would require a proper fix -* in the future. -*/ - dip->di_mode =0 ; - -if (delete) { +if (delete) error =xfs_ifree_cluster (ip, tp, first_ino); - }return error; }
Signature · 6 lines
1.7.10.4 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
PATCH 3/7 xfs: Introduce an ordered buffer item
In reply to Dave ChinnerIf we have a buffer that we have modified but we do not wish to physically log in a transaction (e.g. we've logged a logical change), we still need to ensure that transactional integrity is maintained. Hence we must not move the tail of the log past the transaction that the buffer is associated with before the buffer is written to disk.
This means these special buffers still need to be included in the transaction and added to the AIL just like a normal buffer, but we do not want the modifications to the buffer written into the transaction. IOWs, what we want is an "ordered buffer" that maintains the same transactional life cycle as a physically logged buffer, just without the transcribing of the modifications to the log.
Hence we need to flag the buffer as an "ordered buffer" to avoid including it in vector size calculations or formatting during the transaction. Once the transaction is committed, the buffer appears for all intents to be the same as a physically logged buffer as it transitions through the log and AIL.
Relogging will also work just fine for such an ordered buffer - the logical transaction will be replayed before the subsequent modifications that relog the buffer, so everything will be reconstructed correctly by recovery.
--- fs/xfs/xfs_buf_item.c | 75 +++++++++++++++++++++++++++++++----------------- fs/xfs/xfs_buf_item.h | 4 ++- fs/xfs/xfs_trace.h | 2 ++ fs/xfs/xfs_trans.h | 1 + fs/xfs/xfs_trans_buf.c | 33 +++++++++++++++++++-- 5 files changed, 84 insertions(+), 31 deletions(-)
fs/xfs/xfs_buf_item.c +48 -27
@@ -140,6 +140,16 @@ xfs_buf_item_size(ASSERT (bip->bli_flags &XFS_BLI_LOGGED ); +if (bip->bli_flags &XFS_BLI_ORDERED ) { +/* +* The buffer has been logged just to order it. +* It is not being included in the transaction +* commit, so no vectors are used at all. +*/ +trace_xfs_buf_item_size_ordered (bip); +return 0 ; + } +/* * the vector count is based on the number of buffer vectors we have * dirty bits in. This will only be greater than one when we have a
@@ -212,6 +222,7 @@ xfs_buf_item_format_segment( goto out; } +/* * Fill in an iovec for each set of contiguous chunks. */
@@ -316,6 +327,16 @@ xfs_buf_item_format( bip->bli_flags &= ~XFS_BLI_INODE_BUF ; } +if ((bip->bli_flags & (XFS_BLI_ORDERED |XFS_BLI_STALE )) == +XFS_BLI_ORDERED ) { +/* +* The buffer has been logged just to order it. It is not being +* included in the transaction commit, so don't format it. +*/ +trace_xfs_buf_item_format_ordered (bip); +return ; + } +for (i =0 ; i < bip->bli_format_count; i++) { vecp =xfs_buf_item_format_segment (bip, vecp, offset, &bip->bli_formats[i]);
@@ -345,6 +366,7 @@ xfs_buf_item_pin(ASSERT (atomic_read (&bip->bli_refcount) >0 );ASSERT ((bip->bli_flags &XFS_BLI_LOGGED ) || + (bip->bli_flags &XFS_BLI_ORDERED ) || (bip->bli_flags &XFS_BLI_STALE ));trace_xfs_buf_item_pin (bip);
@@ -517,8 +539,9 @@ xfs_buf_item_unlock( {struct xfs_buf_log_item *bip =BUF_ITEM (lip);struct xfs_buf *bp = bip->bli_buf; -int aborted, clean, i; -uint hold; +bool clean; +bool aborted; +int flags;/* Clear the buffer's association with this transaction. */ bp->b_transp =NULL ;
@@ -529,23 +552,21 @@ xfs_buf_item_unlock(* (cancelled) buffers at unpin time, but we'll never go through the * pin/unpin cycle if we abort inside commit. */ - aborted = (lip->li_flags &XFS_LI_ABORTED ) !=0 ; - + aborted = (lip->li_flags &XFS_LI_ABORTED ) ? true : false;/* -* Before possibly freeing the buf item, determine if we should -* release the buffer at the end of this routine. +* Before possibly freeing the buf item, copy the per-transaction state +* so we can reference it safely later after clearing it from the +* buffer log item. */ - hold = bip->bli_flags &XFS_BLI_HOLD ; - -/* Clear the per transaction state. */ - bip->bli_flags &= ~(XFS_BLI_LOGGED |XFS_BLI_HOLD ); + flags = bip->bli_flags; + bip->bli_flags &= ~(XFS_BLI_LOGGED |XFS_BLI_HOLD |XFS_BLI_ORDERED );/* * If the buf item is marked stale, then don't do anything. We'll * unlock the buffer and free the buf item when the buffer is unpinned * for the last time. */ -if (bip->bli_flags &XFS_BLI_STALE ) { +if (flags &XFS_BLI_STALE ) {trace_xfs_buf_item_unlock_stale (bip);ASSERT (bip->__bli_format.blf_flags &XFS_BLF_CANCEL );if (!aborted) {
@@ -562,13 +583,19 @@ xfs_buf_item_unlock(* be the only reference to the buf item, so we free it anyway * regardless of whether it is dirty or not. A dirty abort implies a * shutdown, anyway. +* +* Ordered buffers are dirty but may have no recorded changes, so ensure +* we only release clean items here. */ - clean =1 ; -for (i =0 ; i < bip->bli_format_count; i++) { -if (!xfs_bitmap_empty (bip->bli_formats[i].blf_data_map, - bip->bli_formats[i].blf_map_size)) { - clean =0 ; -break ; + clean = (flags &XFS_BLI_DIRTY ) ? false : true; +if (clean) { +int i; +for (i =0 ; i < bip->bli_format_count; i++) { +if (!xfs_bitmap_empty (bip->bli_formats[i].blf_data_map, + bip->bli_formats[i].blf_map_size)) { + clean = false; +break ; + } } }if (clean)
@@ -581,7 +608,7 @@ xfs_buf_item_unlock( }else atomic_dec (&bip->bli_refcount); -if (!hold) +if (!(flags &XFS_BLI_HOLD ))xfs_buf_relse (bp); }
@@ -847,12 +874,6 @@ xfs_buf_item_log(struct xfs_buf *bp = bip->bli_buf;/* -* Mark the item as having some dirty data for -* quick reference in xfs_buf_item_dirty. -*/ - bip->bli_flags |=XFS_BLI_DIRTY ; - -/* * walk each buffer segment and mark them dirty appropriately. */ start =0 ;
@@ -878,7 +899,7 @@ xfs_buf_item_log(/* -* Return 1 if the buffer has some data that has been logged (at any +* Return 1 if the buffer has been logged or ordered in a transaction (at any * point, not just the current transaction) and 0 if not. */ uint
@@ -912,11 +933,11 @@ voidxfs_buf_item_relse ( xfs_buf_t *bp) { -xfs_buf_log_item_t *bip; +xfs_buf_log_item_t *bip = bp->b_fspriv;trace_xfs_buf_item_relse (bp, _RET_IP_); +ASSERT (!(bip->bli_item.li_flags &XFS_LI_IN_AIL )); - bip = bp->b_fspriv; bp->b_fspriv = bip->bli_item.li_bio_list;if (bp->b_fspriv ==NULL ) bp->b_iodone =NULL ;
fs/xfs/xfs_buf_item.h +3 -1
@@ -120,6 +120,7 @@ xfs_blft_from_flags(struct xfs_buf_log_format *blf)#define XFS_BLI_INODE_ALLOC_BUF 0x10#define XFS_BLI_STALE_INODE 0x20#define XFS_BLI_INODE_BUF 0x40 +#define XFS_BLI_ORDERED 0x80#define XFS_BLI_FLAGS \ { XFS_BLI_HOLD, "HOLD" }, \
@@ -128,7 +129,8 @@ xfs_blft_from_flags(struct xfs_buf_log_format *blf) { XFS_BLI_LOGGED, "LOGGED" }, \ { XFS_BLI_INODE_ALLOC_BUF, "INODE_ALLOC" }, \ { XFS_BLI_STALE_INODE, "STALE_INODE" }, \ - { XFS_BLI_INODE_BUF, "INODE_BUF" } + { XFS_BLI_INODE_BUF, "INODE_BUF" }, \ + { XFS_BLI_ORDERED, "ORDERED" } #ifdef __KERNEL__
fs/xfs/xfs_trace.h +2 -0
@@ -486,8 +486,10 @@ DEFINE_EVENT(xfs_buf_item_class, name, \ TP_PROTO(struct xfs_buf_log_item *bip), \ TP_ARGS(bip)) DEFINE_BUF_ITEM_EVENT(xfs_buf_item_size); +DEFINE_BUF_ITEM_EVENT(xfs_buf_item_size_ordered); DEFINE_BUF_ITEM_EVENT(xfs_buf_item_size_stale); DEFINE_BUF_ITEM_EVENT(xfs_buf_item_format); +DEFINE_BUF_ITEM_EVENT(xfs_buf_item_format_ordered); DEFINE_BUF_ITEM_EVENT(xfs_buf_item_format_stale); DEFINE_BUF_ITEM_EVENT(xfs_buf_item_pin); DEFINE_BUF_ITEM_EVENT(xfs_buf_item_unpin);
fs/xfs/xfs_trans.h +1 -0
@@ -503,6 +503,7 @@ void xfs_trans_bhold_release(xfs_trans_t *, struct xfs_buf *);void xfs_trans_binval (xfs_trans_t *,struct xfs_buf *);void xfs_trans_inode_buf (xfs_trans_t *,struct xfs_buf *);void xfs_trans_stale_inode_buf (xfs_trans_t *,struct xfs_buf *); +void xfs_trans_ordered_buf (xfs_trans_t *,struct xfs_buf *);void xfs_trans_dquot_buf (xfs_trans_t *,struct xfs_buf *,uint );void xfs_trans_inode_alloc_buf (xfs_trans_t *,struct xfs_buf *);void xfs_trans_ichgtime (struct xfs_trans *,struct xfs_inode *,int );
fs/xfs/xfs_trans_buf.c +30 -3
@@ -397,7 +397,6 @@ shutdown_abort:return XFS_ERROR (EIO ); } -/* * Release the buffer bp which was previously acquired with one of the * xfs_trans_... buffer allocation routines if the buffer has not
@@ -603,8 +602,14 @@ xfs_trans_log_buf(xfs_trans_t *tp, tp->t_flags |=XFS_TRANS_DIRTY ; bip->bli_item.li_desc->lid_flags |=XFS_LID_DIRTY ; - bip->bli_flags |=XFS_BLI_LOGGED ; -xfs_buf_item_log (bip, first, last); + +/* +* If we have an ordered buffer we are not logging any dirty range but +* it still needs to be marked dirty and that it has been logged. +*/ + bip->bli_flags |=XFS_BLI_DIRTY |XFS_BLI_LOGGED ; +if (!(bip->bli_flags &XFS_BLI_ORDERED )) +xfs_buf_item_log (bip, first, last); }
@@ -757,6 +762,28 @@ xfs_trans_inode_alloc_buf( }/* +* Mark the buffer as ordered for this transaction. This means +* that the contents of the buffer are not recorded in the transaction +* but it is tracked in the AIL as though it was. This allows us +* to record logical changes in transactions rather than the physical +* changes we make to the buffer without changing writeback ordering +* constraints of metadata buffers. +*/ +void +xfs_trans_ordered_buf ( +struct xfs_trans *tp, +struct xfs_buf *bp) +{ +struct xfs_buf_log_item *bip = bp->b_fspriv; + +ASSERT (bp->b_transp == tp); +ASSERT (bip !=NULL ); +ASSERT (atomic_read (&bip->bli_refcount) >0 ); + + bip->bli_flags |=XFS_BLI_ORDERED ; +} + +/* * Set the type of the buffer for log recovery so that it can correctly identify * and hence attach the correct buffer ops to the buffer after replay. */
Signature · 6 lines
1.7.10.4 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
PATCH 4/7 xfs: Inode create log items
In reply to Dave ChinnerIntroduce the inode create log item type for logical inode create logging. Instead of logging the changes in buffers, pass the range to be initialised through the log by a new transaction type. This reduces the amount of log space required to record initialisation during allocation from about 128 bytes per inode to a small fixed amount per inode extent to be initialised.
This requires a new log item type to track it through the log and the AIL. This is a relatively simple item - most callbacks are noops as this item has the same life cycle as the transaction.
--- fs/xfs/Makefile | 1 + fs/xfs/xfs_icreate_item.c | 195 +++++++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_icreate_item.h | 51 ++++++++++++ fs/xfs/xfs_log.h | 3 +- fs/xfs/xfs_super.c | 8 ++ fs/xfs/xfs_trans.h | 4 +- 6 files changed, 260 insertions(+), 2 deletions(-) create mode 100644 fs/xfs/xfs_icreate_item.c create mode 100644 fs/xfs/xfs_icreate_item.h
fs/xfs/Makefile +1 -0
@@ -71,6 +71,7 @@ xfs-y += xfs_alloc.o \ xfs_dir2_sf.o \ xfs_ialloc.o \ xfs_ialloc_btree.o \ + xfs_icreate_item.o \ xfs_inode.o \ xfs_log_recover.o \ xfs_mount.o \
fs/xfs/xfs_icreate_item.c new file +195 -0
195 lines not on this page · raw message
fs/xfs/xfs_icreate_item.h new file +51 -0
51 lines not on this page · raw message
fs/xfs/xfs_log.h +2 -1
@@ -88,7 +88,8 @@ static inline xfs_lsn_t _lsn_cmp(xfs_lsn_t lsn1, xfs_lsn_t lsn2)#define XLOG_REG_TYPE_UNMOUNT 17#define XLOG_REG_TYPE_COMMIT 18#define XLOG_REG_TYPE_TRANSHDR 19 -#define XLOG_REG_TYPE_MAX 19 +#define XLOG_REG_TYPE_ICREATE 20 +#define XLOG_REG_TYPE_MAX 20 typedef struct xfs_log_iovec { void *i_addr; /* beginning address of region */
fs/xfs/xfs_super.c +8 -0
@@ -51,6 +51,7 @@#include "xfs_inode_item.h" #include "xfs_icache.h" #include "xfs_trace.h" +#include "xfs_icreate_item.h" #include <linux/namei.h> #include <linux/init.h>
@@ -1644,9 +1645,15 @@ xfs_init_zones(void) KM_ZONE_SPREAD, NULL);if (!xfs_ili_zone) goto out_destroy_inode_zone; + xfs_icreate_zone =kmem_zone_init (sizeof (struct xfs_icreate_item ), +"xfs_icr" ); +if (!xfs_icreate_zone) + goto out_destroy_ili_zone;return 0 ; + out_destroy_ili_zone: +kmem_zone_destroy (xfs_ili_zone); out_destroy_inode_zone:kmem_zone_destroy (xfs_inode_zone); out_destroy_efi_zone:
@@ -1685,6 +1692,7 @@ xfs_destroy_zones(void)* destroy caches. */ rcu_barrier (); +kmem_zone_destroy (xfs_icreate_zone);kmem_zone_destroy (xfs_ili_zone);kmem_zone_destroy (xfs_inode_zone);kmem_zone_destroy (xfs_efi_zone);
fs/xfs/xfs_trans.h +3 -1
@@ -48,6 +48,7 @@ typedef struct xfs_trans_header {#define XFS_LI_BUF 0x123c/* v2 bufs, variable sized inode bufs */ #define XFS_LI_DQUOT 0x123d#define XFS_LI_QUOTAOFF 0x123e +#define XFS_LI_ICREATE 0x123f#define XFS_LI_TYPE_DESC \ { XFS_LI_EFI, "XFS_LI_EFI" }, \
@@ -107,7 +108,8 @@ typedef struct xfs_trans_header {#define XFS_TRANS_SWAPEXT 40#define XFS_TRANS_SB_COUNT 41#define XFS_TRANS_CHECKPOINT 42 -#define XFS_TRANS_TYPE_MAX 42 +#define XFS_TRANS_ICREATE 43 +#define XFS_TRANS_TYPE_MAX 43/* new transaction types need to be reflected in xfs_logprint(8) */ #define XFS_TRANS_TYPES \
Signature · 6 lines
1.7.10.4 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
PATCH 5/7 xfs: Inode create transaction reservations
In reply to Dave ChinnerDefine the log and space transaction sizes. Factor the current create log reservation macro into the two logical halves and reuse one half for the new icreate transactions. The icreate transaction is transparent to all the high level create code - the pre-calculated reservations will correctly set the reservations dependent on whether the filesystem supports the icreate transaction.
--- fs/xfs/xfs_trans.c | 118 ++++++++++++++++++++++++++++++++++------------------ 1 file changed, 77 insertions(+), 41 deletions(-)
fs/xfs/xfs_trans.c +77 -41
@@ -234,71 +234,93 @@ xfs_calc_remove_reservation( }/* -* For symlink we can modify: +* For create, break it in to the two cases that the transaction +* covers. We start with the modify case - allocation done by modification +* of the state of existing inodes - and the allocation case. +*/ + +/* +* For create we can modify: * the parent directory inode: inode size * the new inode: inode size -* the inode btree entry: 1 block +* the inode btree entry: block size +* the superblock for the nlink flag: sector size * the directory btree: (max depth + v2) * dir block size * the directory inode's bmap btree: (max depth + v2) * block size -* the blocks for the symlink: 1 kB -* Or in the first xact we allocate some inodes giving: +*/ +STATIC uint +xfs_calc_create_resv_modify ( +struct xfs_mount *mp) +{ +return xfs_calc_buf_res (2 , mp->m_sb.sb_inodesize) + +xfs_calc_buf_res (1 , mp->m_sb.sb_sectsize) + + (uint )XFS_FSB_TO_B (mp,1 ) + +xfs_calc_buf_res (XFS_DIROP_LOG_COUNT (mp),XFS_FSB_TO_B (mp,1 )); +} + +/* +* For create we can allocate some inodes giving: * the agi and agf of the ag getting the new inodes: 2 * sectorsize +* the superblock for the nlink flag: sector size * the inode blocks allocated: XFS_IALLOC_BLOCKS * blocksize * the inode btree: max depth * blocksize -* the allocation btrees: 2 trees * (2 * max depth - 1) * block size +* the allocation btrees: 2 trees * (max depth - 1) * block size */ STATIC uint -xfs_calc_symlink_reservation ( +xfs_calc_create_resv_alloc ( +struct xfs_mount *mp) +{ +return xfs_calc_buf_res (2 , mp->m_sb.sb_sectsize) + + mp->m_sb.sb_sectsize + +xfs_calc_buf_res (XFS_IALLOC_BLOCKS (mp),XFS_FSB_TO_B (mp,1 )) + +xfs_calc_buf_res (mp->m_in_maxlevels,XFS_FSB_TO_B (mp,1 )) + +xfs_calc_buf_res (XFS_ALLOCFREE_LOG_COUNT (mp,1 ), +XFS_FSB_TO_B (mp,1 )); +} + +STATIC uint +__xfs_calc_create_reservation (struct xfs_mount *mp) {return XFS_DQUOT_LOGRES (mp) + -MAX ((xfs_calc_buf_res (2 , mp->m_sb.sb_inodesize) + -xfs_calc_buf_res (1 ,XFS_FSB_TO_B (mp,1 )) + -xfs_calc_buf_res (XFS_DIROP_LOG_COUNT (mp), -XFS_FSB_TO_B (mp,1 )) + -xfs_calc_buf_res (1 ,1024 )), - (xfs_calc_buf_res (2 , mp->m_sb.sb_sectsize) + -xfs_calc_buf_res (XFS_IALLOC_BLOCKS (mp), -XFS_FSB_TO_B (mp,1 )) + -xfs_calc_buf_res (mp->m_in_maxlevels, -XFS_FSB_TO_B (mp,1 )) + -xfs_calc_buf_res (XFS_ALLOCFREE_LOG_COUNT (mp,1 ), -XFS_FSB_TO_B (mp,1 )))); +MAX (xfs_calc_create_resv_alloc (mp), +xfs_calc_create_resv_modify (mp)); }/* -* For create we can modify: -* the parent directory inode: inode size -* the new inode: inode size -* the inode btree entry: block size -* the superblock for the nlink flag: sector size -* the directory btree: (max depth + v2) * dir block size -* the directory inode's bmap btree: (max depth + v2) * block size -* Or in the first xact we allocate some inodes giving: +* For icreate we can allocate some inodes giving: * the agi and agf of the ag getting the new inodes: 2 * sectorsize * the superblock for the nlink flag: sector size -* the inode blocks allocated: XFS_IALLOC_BLOCKS * blocksize * the inode btree: max depth * blocksize * the allocation btrees: 2 trees * (max depth - 1) * block size */ STATIC uint -xfs_calc_create_reservation ( +xfs_calc_icreate_resv_alloc (struct xfs_mount *mp) { +return xfs_calc_buf_res (2 , mp->m_sb.sb_sectsize) + + mp->m_sb.sb_sectsize + +xfs_calc_buf_res (mp->m_in_maxlevels,XFS_FSB_TO_B (mp,1 )) + +xfs_calc_buf_res (XFS_ALLOCFREE_LOG_COUNT (mp,1 ), +XFS_FSB_TO_B (mp,1 )); +} + +STATIC uint +xfs_calc_icreate_reservation (xfs_mount_t *mp) +{return XFS_DQUOT_LOGRES (mp) + -MAX ((xfs_calc_buf_res (2 , mp->m_sb.sb_inodesize) + -xfs_calc_buf_res (1 , mp->m_sb.sb_sectsize) + - (uint )XFS_FSB_TO_B (mp,1 ) + -xfs_calc_buf_res (XFS_DIROP_LOG_COUNT (mp), -XFS_FSB_TO_B (mp,1 ))), - (xfs_calc_buf_res (2 , mp->m_sb.sb_sectsize) + - mp->m_sb.sb_sectsize + -xfs_calc_buf_res (XFS_IALLOC_BLOCKS (mp), -XFS_FSB_TO_B (mp,1 )) + -xfs_calc_buf_res (mp->m_in_maxlevels, -XFS_FSB_TO_B (mp,1 )) + -xfs_calc_buf_res (XFS_ALLOCFREE_LOG_COUNT (mp,1 ), -XFS_FSB_TO_B (mp,1 )))); +MAX (xfs_calc_icreate_resv_alloc (mp), +xfs_calc_create_resv_modify (mp)); +} + +STATIC uint +xfs_calc_create_reservation ( +struct xfs_mount *mp) +{ +if (xfs_sb_version_hascrc (&mp->m_sb)) +return xfs_calc_icreate_reservation (mp); +return __xfs_calc_create_reservation (mp); + }/*
@@ -311,6 +333,20 @@ xfs_calc_mkdir_reservation(return xfs_calc_create_reservation (mp); } + +/* +* Making a new symplink is the same as creating a new file, but +* with the added blocks for remote symlink data which can be up to 1kB in +* length (MAXPATHLEN). +*/ +STATIC uint +xfs_calc_symlink_reservation ( +struct xfs_mount *mp) +{ +return xfs_calc_create_reservation (mp) + +xfs_calc_buf_res (1 ,MAXPATHLEN ); +} +/* * In freeing an inode we can modify: * the inode being freed: inode size
Signature · 6 lines
1.7.10.4 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
PATCH 6/7 xfs: Use inode create transaction
In reply to Dave ChinnerReplace the use of buffer based logging of inode initialisation, uses the new logical form to describe the range to be initialised in recovery. We continue to "log" the inode buffers to push them into the AIL and ensure that the inode create transaction is not removed from the log before the inode buffers are written to disk.
Update the transaction identifier and reservations to match the changed implementation.
--- fs/xfs/xfs_ialloc.c | 39 ++++++++++++++++++++++++++++++--------- fs/xfs/xfs_ialloc.h | 7 +++++++ 2 files changed, 37 insertions(+), 9 deletions(-)
fs/xfs/xfs_ialloc.c +30 -9
@@ -38,6 +38,7 @@#include "xfs_bmap.h" #include "xfs_cksum.h" #include "xfs_buf_item.h" +#include "xfs_icreate_item.h" /*
@@ -152,7 +153,7 @@ xfs_check_agi_freecount(/* * Initialise a new set of inodes. */ -STATIC int +int xfs_ialloc_inode_init( struct xfs_mount *mp, struct xfs_trans *tp,
@@ -208,6 +209,17 @@ xfs_ialloc_inode_init( version =3 ; ino =XFS_AGINO_TO_INO (mp, agno,XFS_OFFBNO_TO_AGINO (mp, agbno,0 )); + +/* +* log the initialisation that is about to take place as an +* logical operation. This means the transaction does not +* need to log the physical changes to the inode buffers as log +* recovery will know what initialisation is actually needed. +* Hence we only need to log the buffers as "ordered" buffers so +* they track in the AIL as if they were physically logged. +*/ +xfs_icreate_log (tp, agno, agbno,XFS_IALLOC_INODES (mp), + mp->m_sb.sb_inodesize, length, gen); }else if (xfs_sb_version_hasnlink (&mp->m_sb)) version =2 ;else
@@ -223,13 +235,8 @@ xfs_ialloc_inode_init( XBF_UNMAPPED);if (!fbuf)return ENOMEM ; -/* -* Initialize all inodes in this buffer and then log them. -* -* XXX: It would be much better if we had just one transaction -* to log a whole cluster of inodes instead of all the -* individual transactions causing a lot of log traffic. -*/ + +/* Initialize the inode buffers and log them appropriately. */ fbuf->b_ops = &xfs_inode_buf_ops;xfs_buf_zero (fbuf,0 ,BBTOB (fbuf->b_length));for (i =0 ; i < ninodes; i++) {
@@ -253,11 +260,25 @@ xfs_ialloc_inode_init( ioffset + isize - 1); } } +if (version ==3 ) { -/* need to log the entire buffer */ +/* +* Mark the buffer as ordered so that it is not logged +* in the transaction but still tracked in the AIL as +* part of the transaction. +*/ +xfs_trans_ordered_buf (tp, fbuf);xfs_trans_log_buf (tp, fbuf,0 ,BBTOB (fbuf->b_length) -1 ); } + +/* +* Mark the buffer as an inode allocation buffer so it sticks in +* AIL at the point of this allocation transaction. This ensures +* the they are on disk before the tail of the log can be moved +* past this transaction (i.e. by preventing relogging from +* moving it forward in the log). +*/ xfs_trans_inode_alloc_buf (tp, fbuf); }return 0 ;
fs/xfs/xfs_ialloc.h +7 -0
@@ -152,4 +152,11 @@ int xfs_inobt_get_rec(struct xfs_btree_cur *cur,extern const struct xfs_buf_ops xfs_agi_buf_ops; +/* +* Inode buffer initialisation routine +*/ +int xfs_ialloc_inode_init (struct xfs_mount *mp,struct xfs_trans *tp, +xfs_agnumber_t agno,xfs_agblock_t agbno, +xfs_agblock_t length,unsigned int gen); +#endif /* __XFS_IALLOC_H__ */
Signature · 6 lines
1.7.10.4 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
PATCH 7/7 xfs: Inode create item recovery
In reply to Dave ChinnerWhen we find a icreate transaction, we need to get and initialise the buffers in the range that has been passed. Extract and verify the information in the item record, then loop over the range initialising and issuing the buffer writes delayed.
Support an arbitrary size range to initialise so that in future when we allocate inodes in much larger chunks all kernels that understand this transaction can still recover them.
--- fs/xfs/xfs_ialloc.c | 24 ++++++++---- fs/xfs/xfs_ialloc.h | 5 ++- fs/xfs/xfs_log_recover.c | 94 ++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 113 insertions(+), 10 deletions(-)
fs/xfs/xfs_ialloc.c +16 -8
@@ -151,12 +151,16 @@ xfs_check_agi_freecount(#endif /* -* Initialise a new set of inodes. +* Initialise a new set of inodes. When called without a transaction context +* (e.g. from recovery) we initiate a delayed write of the inode buffers rather +* than logging them (which in a transaction context puts them into the AIL +* for writeback rather than the xfsbufd queue). */ int xfs_ialloc_inode_init( struct xfs_mount *mp, struct xfs_trans *tp, + struct list_head *buffer_list, xfs_agnumber_t agno, xfs_agblock_t agbno, xfs_agblock_t length,
@@ -218,8 +222,9 @@ xfs_ialloc_inode_init(* Hence we only need to log the buffers as "ordered" buffers so * they track in the AIL as if they were physically logged. */ -xfs_icreate_log (tp, agno, agbno,XFS_IALLOC_INODES (mp), - mp->m_sb.sb_inodesize, length, gen); +if (tp) +xfs_icreate_log (tp, agno, agbno,XFS_IALLOC_INODES (mp), + mp->m_sb.sb_inodesize, length, gen); }else if (xfs_sb_version_hasnlink (&mp->m_sb)) version =2 ;else
@@ -254,14 +259,14 @@ xfs_ialloc_inode_init( ino++;uuid_copy (&free->di_uuid, &mp->m_sb.sb_uuid);xfs_dinode_calc_crc (mp, free); - }else { + }else if (tp) {/* just log the inode core */ xfs_trans_log_buf (tp, fbuf, ioffset, ioffset + isize -1 ); } } -if (version ==3 ) { +if (version ==3 && tp) {/* * Mark the buffer as ordered so that it is not logged * in the transaction but still tracked in the AIL as
@@ -279,7 +284,10 @@ xfs_ialloc_inode_init(* past this transaction (i.e. by preventing relogging from * moving it forward in the log). */ -xfs_trans_inode_alloc_buf (tp, fbuf); +if (tp) +xfs_trans_inode_alloc_buf (tp, fbuf); +else +xfs_buf_delwri_queue (fbuf, buffer_list); }return 0 ; }
@@ -324,7 +332,7 @@ xfs_ialloc_ag_alloc(* First try to allocate inodes contiguous with the last-allocated * chunk of inodes. If the filesystem is striped, this will fill * an entire stripe unit with inodes. -*/ +*/ agi =XFS_BUF_TO_AGI (agbp); newino =be32_to_cpu (agi->agi_newino); agno =be32_to_cpu (agi->agi_seqno);
@@ -423,7 +431,7 @@ xfs_ialloc_ag_alloc(* rather than a linear progression to prevent the next generation * number from being easily guessable. */ - error =xfs_ialloc_inode_init (args.mp, tp, agno, args.agbno, + error =xfs_ialloc_inode_init (args.mp, tp,NULL , agno, args.agbno, args.len,prandom_u32 ());if (error)
fs/xfs/xfs_ialloc.h +3 -2
@@ -156,7 +156,8 @@ extern const struct xfs_buf_ops xfs_agi_buf_ops;* Inode buffer initialisation routine */ int xfs_ialloc_inode_init (struct xfs_mount *mp,struct xfs_trans *tp, -xfs_agnumber_t agno,xfs_agblock_t agbno, -xfs_agblock_t length,unsigned int gen); +struct list_head *buffer_list, +xfs_agnumber_t agno,xfs_agblock_t agbno, +xfs_agblock_t length,unsigned int gen);#endif /* __XFS_IALLOC_H__ */
fs/xfs/xfs_log_recover.c +94 -0
124 lines not on this page · raw message
Signature · 6 lines
1.7.10.4 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs