Skip to contentSkip to conversation

xfs: various log stuff...

PATCH New Dave Chinner 6 participants 59 messages Feb 23, 2021 – Mar 5, 2021 XFS filesystem

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 xfs: various log stuff...

HI folks,

Version 2 of this set of changes to the log code. First version was posted here:

https://lore.kernel.org/linux-xfs/[email protected]/

"Quick patch dump for y'all. A couple of minor cleanups to the log behaviour, a fix for the CIL throttle hang and a couple of patches to rework the cache flushing that journal IO does to reduce the number of cache flushes by a couple of orders of magnitude."

Version 2:
- fix ticket reservation roundoff to include 2 roundoffs
- removed stale copied comment from roundoff initialisation.
- clarified "separation" to mean "separation for ordering purposes" in commit message.
- added comment that newly activated, clean, empty iclogs have a LSN of 0 so are captured by the "iclog lsn < start_lsn" case that avoids needing to wait before releasing the commit iclog to be written.
- added async cache flush infrastructure
- convert CIL checkpoint push work it issue an unconditional metadata device cache flush rather than asking the first iclog write to issue it via REQ_PREFLUSH.
- cleaned up xlog_write() to remove a redundant parameter and prepare the logic for setting flags on the iclog based on the type of operational data is being written to the log.
- added XLOG_ICL_NEED_FUA flag to complement the NEED_FLUSH flag, allowing callers to issue explicit flushes and clear the NEED_FLUSH flag before the iclog is written without dropping the REQ_FUA requirement in /dev/null...
- added CIL commit-in-start-iclog optimisation that clears the NEED_FLUSH flag to avoid an unnecessary cache flush when issuing the iclog.
- fixed typo in CIL throttle bugfix comment.
- fixed trailing whitespace in commit message.

#1 lore raw

PATCH 1/8 xfs: log stripe roundoff is a property of the log

From: Dave Chinner <[email protected]>

We don't need to look at the xfs_mount and superblock every time we need to do an iclog roundoff calculation. The property is fixed for the life of the log, so store the roundoff in the log at mount time and use that everywhere.

On a debug build:

$ size fs/xfs/xfs_log.o.*
   text	   data	    bss	    dec	    hex	filename
  27360	    560	      8	  27928	   6d18	fs/xfs/xfs_log.o.orig
  27219	    560	      8	  27787	   6c8b	fs/xfs/xfs_log.o.patched
---
 fs/xfs/libxfs/xfs_log_format.h |  3 --
 fs/xfs/xfs_log.c               | 59 ++++++++++++++--------------------
 fs/xfs/xfs_log_priv.h          |  2 ++
 3 files changed, 27 insertions(+), 37 deletions(-)
fs/xfs/libxfs/xfs_log_format.h +0 -3
index 8bd00da6d2a4..16587219549c 100644
@@ -34,9 +34,6 @@ typedef uint32_t xlog_tid_t;
 #define XLOG_MIN_RECORD_BSHIFT	14		/* 16384 == 1 << 14 */
 #define XLOG_BIG_RECORD_BSHIFT	15		/* 32k == 1 << 15 */
 #define XLOG_MAX_RECORD_BSHIFT	18		/* 256k == 1 << 18 */
-#define XLOG_BTOLSUNIT(log, b)  (((b)+(log)->l_mp->m_sb.sb_logsunit-1) / \
-                                 (log)->l_mp->m_sb.sb_logsunit)
-#define XLOG_LSUNITTOB(log, su) ((su) * (log)->l_mp->m_sb.sb_logsunit)
 
 #define XLOG_HEADER_SIZE	512
 
fs/xfs/xfs_log.c +25 -34

107 lines not on this page · raw message

fs/xfs/xfs_log_priv.h +2 -0
index 1c6fdbf3d506..037950cf1061 100644
@@ -436,6 +436,8 @@ struct xlog {
 #endif
 	/* log recovery lsn tracking (for buffer submission */
 	xfs_lsn_t		l_recovery_lsn;
+
+	uint32_t		l_iclog_roundoff;/* padding roundoff */
 };
 
 #define XLOG_BUF_CANCEL_BUCKET(log, blkno) \
Signature · 1 line
2.28.0
Reviewed-by on v1 1/8
#2 lore raw

Re: [PATCH 1/8] xfs: log stripe roundoff is a property of the log

Quoted 7 lines · Dave Chinner

From: Dave Chinner <[email protected]>

We don't need to look at the xfs_mount and superblock every time we need to do an iclog roundoff calculation. The property is fixed for the life of the log, so store the roundoff in the log at mount time and use that everywhere.

Quoted 7 lines

On a debug build:

$ size fs/xfs/xfs_log.o.*
   text	   data	    bss	    dec	    hex	filename
  27360	    560	      8	  27928	   6d18	fs/xfs/xfs_log.o.orig
  27219	    560	      8	  27787	   6c8b	fs/xfs/xfs_log.o.patched

The changes look good to me.

Quoted 151 lines
---
 fs/xfs/libxfs/xfs_log_format.h |  3 --
 fs/xfs/xfs_log.c               | 59 ++++++++++++++--------------------
 fs/xfs/xfs_log_priv.h          |  2 ++
 3 files changed, 27 insertions(+), 37 deletions(-)
fs/xfs/libxfs/xfs_log_format.h +0 -3
index 8bd00da6d2a4..16587219549c 100644
@@ -34,9 +34,6 @@ typedef uint32_t xlog_tid_t;
 #define XLOG_MIN_RECORD_BSHIFT	14		/* 16384 == 1 << 14 */
 #define XLOG_BIG_RECORD_BSHIFT	15		/* 32k == 1 << 15 */
 #define XLOG_MAX_RECORD_BSHIFT	18		/* 256k == 1 << 18 */
-#define XLOG_BTOLSUNIT(log, b)  (((b)+(log)->l_mp->m_sb.sb_logsunit-1) / \
-                                 (log)->l_mp->m_sb.sb_logsunit)
-#define XLOG_LSUNITTOB(log, su) ((su) * (log)->l_mp->m_sb.sb_logsunit)
 
 #define XLOG_HEADER_SIZE	512
 
fs/xfs/xfs_log.c +25 -34
index 06041834daa3..fa284f26d10e 100644
@@ -1399,6 +1399,11 @@ xlog_alloc_log(
 	xlog_assign_atomic_lsn(&log->l_last_sync_lsn, 1, 0);
 	log->l_curr_cycle  = 1;	    /* 0 is bad since this is initial value */
 
+	if (xfs_sb_version_haslogv2(&mp->m_sb) && mp->m_sb.sb_logsunit > 1)
+		log->l_iclog_roundoff = mp->m_sb.sb_logsunit;
+	else
+		log->l_iclog_roundoff = BBSIZE;
+
 	xlog_grant_head_init(&log->l_reserve_head);
 	xlog_grant_head_init(&log->l_write_head);
 
@@ -1852,29 +1857,15 @@ xlog_calc_iclog_size(
 	uint32_t		*roundoff)
 {
 	uint32_t		count_init, count;
-	bool			use_lsunit;
-
-	use_lsunit = xfs_sb_version_haslogv2(&log->l_mp->m_sb) &&
-			log->l_mp->m_sb.sb_logsunit > 1;
 
 	/* Add for LR header */
 	count_init = log->l_iclog_hsize + iclog->ic_offset;
+	count = roundup(count_init, log->l_iclog_roundoff);
 
-	/* Round out the log write size */
-	if (use_lsunit) {
-		/* we have a v2 stripe unit to use */
-		count = XLOG_LSUNITTOB(log, XLOG_BTOLSUNIT(log, count_init));
-	} else {
-		count = BBTOB(BTOBB(count_init));
-	}
-
-	ASSERT(count >= count_init);
 	*roundoff = count - count_init;
 
-	if (use_lsunit)
-		ASSERT(*roundoff < log->l_mp->m_sb.sb_logsunit);
-	else
-		ASSERT(*roundoff < BBTOB(1));
+	ASSERT(count >= count_init);
+	ASSERT(*roundoff < log->l_iclog_roundoff);
 	return count;
 }
 
@@ -3149,10 +3140,9 @@ xlog_state_switch_iclogs(
 	log->l_curr_block += BTOBB(eventual_size)+BTOBB(log->l_iclog_hsize);
 
 	/* Round up to next log-sunit */
-	if (xfs_sb_version_haslogv2(&log->l_mp->m_sb) &&
-	    log->l_mp->m_sb.sb_logsunit > 1) {
-		uint32_t sunit_bb = BTOBB(log->l_mp->m_sb.sb_logsunit);
-		log->l_curr_block = roundup(log->l_curr_block, sunit_bb);
+	if (log->l_iclog_roundoff > BBSIZE) {
+		log->l_curr_block = roundup(log->l_curr_block,
+						BTOBB(log->l_iclog_roundoff));
 	}
 
 	if (log->l_curr_block >= log->l_logBBsize) {
@@ -3404,12 +3394,11 @@ xfs_log_ticket_get(
  * Figure out the total log space unit (in bytes) that would be
  * required for a log ticket.
  */
-int
-xfs_log_calc_unit_res(
-	struct xfs_mount	*mp,
+static int
+xlog_calc_unit_res(
+	struct xlog		*log,
 	int			unit_bytes)
 {
-	struct xlog		*log = mp->m_log;
 	int			iclog_space;
 	uint			num_headers;
 
@@ -3485,18 +3474,20 @@ xfs_log_calc_unit_res(
 	/* for commit-rec LR header - note: padding will subsume the ophdr */
 	unit_bytes += log->l_iclog_hsize;
 
-	/* for roundoff padding for transaction data and one for commit record */
-	if (xfs_sb_version_haslogv2(&mp->m_sb) && mp->m_sb.sb_logsunit > 1) {
-		/* log su roundoff */
-		unit_bytes += 2 * mp->m_sb.sb_logsunit;
-	} else {
-		/* BB roundoff */
-		unit_bytes += 2 * BBSIZE;
-        }
+	/* roundoff padding for transaction data and one for commit record */
+	unit_bytes += 2 * log->l_iclog_roundoff;
 
 	return unit_bytes;
 }
 
+int
+xfs_log_calc_unit_res(
+	struct xfs_mount	*mp,
+	int			unit_bytes)
+{
+	return xlog_calc_unit_res(mp->m_log, unit_bytes);
+}
+
 /*
  * Allocate and initialise a new log ticket.
  */
@@ -3513,7 +3504,7 @@ xlog_ticket_alloc(
 
 	tic = kmem_cache_zalloc(xfs_log_ticket_zone, GFP_NOFS | __GFP_NOFAIL);
 
-	unit_res = xfs_log_calc_unit_res(log->l_mp, unit_bytes);
+	unit_res = xlog_calc_unit_res(log, unit_bytes);
 
 	atomic_set(&tic->t_ref, 1);
 	tic->t_task		= current;
fs/xfs/xfs_log_priv.h +2 -0
index 1c6fdbf3d506..037950cf1061 100644
@@ -436,6 +436,8 @@ struct xlog {
 #endif
 	/* log recovery lsn tracking (for buffer submission */
 	xfs_lsn_t		l_recovery_lsn;
+
+	uint32_t		l_iclog_roundoff;/* padding roundoff */
 };
 
 #define XLOG_BUF_CANCEL_BUCKET(log, blkno) \
Signature · 1 line
chandan
Reviewed-by on v1 1/8
#3 lore raw

Re: [PATCH 1/8] xfs: log stripe roundoff is a property of the log

In reply to Dave Chinner
Quoted 7 lines · Dave Chinner

From: Dave Chinner <[email protected]>

We don't need to look at the xfs_mount and superblock every time we need to do an iclog roundoff calculation. The property is fixed for the life of the log, so store the roundoff in the log at mount time and use that everywhere.

Quoted 8 lines

On a debug build:

$ size fs/xfs/xfs_log.o.*
   text	   data	    bss	    dec	    hex	filename
  27360	    560	      8	  27928	   6d18	fs/xfs/xfs_log.o.orig
  27219	    560	      8	  27787	   6c8b	fs/xfs/xfs_log.o.patched

Seems ok to me...

--D

Quoted 153 lines
---
 fs/xfs/libxfs/xfs_log_format.h |  3 --
 fs/xfs/xfs_log.c               | 59 ++++++++++++++--------------------
 fs/xfs/xfs_log_priv.h          |  2 ++
 3 files changed, 27 insertions(+), 37 deletions(-)
fs/xfs/libxfs/xfs_log_format.h +0 -3
index 8bd00da6d2a4..16587219549c 100644
@@ -34,9 +34,6 @@ typedef uint32_t xlog_tid_t;
 #define XLOG_MIN_RECORD_BSHIFT	14		/* 16384 == 1 << 14 */
 #define XLOG_BIG_RECORD_BSHIFT	15		/* 32k == 1 << 15 */
 #define XLOG_MAX_RECORD_BSHIFT	18		/* 256k == 1 << 18 */
-#define XLOG_BTOLSUNIT(log, b)  (((b)+(log)->l_mp->m_sb.sb_logsunit-1) / \
-                                 (log)->l_mp->m_sb.sb_logsunit)
-#define XLOG_LSUNITTOB(log, su) ((su) * (log)->l_mp->m_sb.sb_logsunit)
 
 #define XLOG_HEADER_SIZE	512
 
fs/xfs/xfs_log.c +25 -34
index 06041834daa3..fa284f26d10e 100644
@@ -1399,6 +1399,11 @@ xlog_alloc_log(
 	xlog_assign_atomic_lsn(&log->l_last_sync_lsn, 1, 0);
 	log->l_curr_cycle  = 1;	    /* 0 is bad since this is initial value */
 
+	if (xfs_sb_version_haslogv2(&mp->m_sb) && mp->m_sb.sb_logsunit > 1)
+		log->l_iclog_roundoff = mp->m_sb.sb_logsunit;
+	else
+		log->l_iclog_roundoff = BBSIZE;
+
 	xlog_grant_head_init(&log->l_reserve_head);
 	xlog_grant_head_init(&log->l_write_head);
 
@@ -1852,29 +1857,15 @@ xlog_calc_iclog_size(
 	uint32_t		*roundoff)
 {
 	uint32_t		count_init, count;
-	bool			use_lsunit;
-
-	use_lsunit = xfs_sb_version_haslogv2(&log->l_mp->m_sb) &&
-			log->l_mp->m_sb.sb_logsunit > 1;
 
 	/* Add for LR header */
 	count_init = log->l_iclog_hsize + iclog->ic_offset;
+	count = roundup(count_init, log->l_iclog_roundoff);
 
-	/* Round out the log write size */
-	if (use_lsunit) {
-		/* we have a v2 stripe unit to use */
-		count = XLOG_LSUNITTOB(log, XLOG_BTOLSUNIT(log, count_init));
-	} else {
-		count = BBTOB(BTOBB(count_init));
-	}
-
-	ASSERT(count >= count_init);
 	*roundoff = count - count_init;
 
-	if (use_lsunit)
-		ASSERT(*roundoff < log->l_mp->m_sb.sb_logsunit);
-	else
-		ASSERT(*roundoff < BBTOB(1));
+	ASSERT(count >= count_init);
+	ASSERT(*roundoff < log->l_iclog_roundoff);
 	return count;
 }
 
@@ -3149,10 +3140,9 @@ xlog_state_switch_iclogs(
 	log->l_curr_block += BTOBB(eventual_size)+BTOBB(log->l_iclog_hsize);
 
 	/* Round up to next log-sunit */
-	if (xfs_sb_version_haslogv2(&log->l_mp->m_sb) &&
-	    log->l_mp->m_sb.sb_logsunit > 1) {
-		uint32_t sunit_bb = BTOBB(log->l_mp->m_sb.sb_logsunit);
-		log->l_curr_block = roundup(log->l_curr_block, sunit_bb);
+	if (log->l_iclog_roundoff > BBSIZE) {
+		log->l_curr_block = roundup(log->l_curr_block,
+						BTOBB(log->l_iclog_roundoff));
 	}
 
 	if (log->l_curr_block >= log->l_logBBsize) {
@@ -3404,12 +3394,11 @@ xfs_log_ticket_get(
  * Figure out the total log space unit (in bytes) that would be
  * required for a log ticket.
  */
-int
-xfs_log_calc_unit_res(
-	struct xfs_mount	*mp,
+static int
+xlog_calc_unit_res(
+	struct xlog		*log,
 	int			unit_bytes)
 {
-	struct xlog		*log = mp->m_log;
 	int			iclog_space;
 	uint			num_headers;
 
@@ -3485,18 +3474,20 @@ xfs_log_calc_unit_res(
 	/* for commit-rec LR header - note: padding will subsume the ophdr */
 	unit_bytes += log->l_iclog_hsize;
 
-	/* for roundoff padding for transaction data and one for commit record */
-	if (xfs_sb_version_haslogv2(&mp->m_sb) && mp->m_sb.sb_logsunit > 1) {
-		/* log su roundoff */
-		unit_bytes += 2 * mp->m_sb.sb_logsunit;
-	} else {
-		/* BB roundoff */
-		unit_bytes += 2 * BBSIZE;
-        }
+	/* roundoff padding for transaction data and one for commit record */
+	unit_bytes += 2 * log->l_iclog_roundoff;
 
 	return unit_bytes;
 }
 
+int
+xfs_log_calc_unit_res(
+	struct xfs_mount	*mp,
+	int			unit_bytes)
+{
+	return xlog_calc_unit_res(mp->m_log, unit_bytes);
+}
+
 /*
  * Allocate and initialise a new log ticket.
  */
@@ -3513,7 +3504,7 @@ xlog_ticket_alloc(
 
 	tic = kmem_cache_zalloc(xfs_log_ticket_zone, GFP_NOFS | __GFP_NOFAIL);
 
-	unit_res = xfs_log_calc_unit_res(log->l_mp, unit_bytes);
+	unit_res = xlog_calc_unit_res(log, unit_bytes);
 
 	atomic_set(&tic->t_ref, 1);
 	tic->t_task		= current;
fs/xfs/xfs_log_priv.h +2 -0
index 1c6fdbf3d506..037950cf1061 100644
@@ -436,6 +436,8 @@ struct xlog {
 #endif
 	/* log recovery lsn tracking (for buffer submission */
 	xfs_lsn_t		l_recovery_lsn;
+
+	uint32_t		l_iclog_roundoff;/* padding roundoff */
 };
 
 #define XLOG_BUF_CANCEL_BUCKET(log, blkno) \
Signature · 1 line
2.28.0
Reviewed-by on v1 1/8
#5 lore raw

Re: [PATCH 1/8] xfs: log stripe roundoff is a property of the log

In reply to Dave Chinner
Quoted 9 lines · Dave Chinner

From: Dave Chinner <[email protected]>

We don't need to look at the xfs_mount and superblock every time we need to do an iclog roundoff calculation. The property is fixed for the life of the log, so store the roundoff in the log at mount time and use that everywhere.

On a debug build:

Quoted 7 lines
$ size fs/xfs/xfs_log.o.*
   text	   data	    bss	    dec	    hex	filename
  27360	    560	      8	  27928	   6d18	fs/xfs/xfs_log.o.orig
  27219	    560	      8	  27787	   6c8b	fs/xfs/xfs_log.o.patched

---

Quoted 152 lines
 fs/xfs/libxfs/xfs_log_format.h |  3 --
 fs/xfs/xfs_log.c               | 59 ++++++++++++++--------------------
 fs/xfs/xfs_log_priv.h          |  2 ++
 3 files changed, 27 insertions(+), 37 deletions(-)
fs/xfs/libxfs/xfs_log_format.h +0 -3
index 8bd00da6d2a4..16587219549c 100644
@@ -34,9 +34,6 @@ typedef uint32_t xlog_tid_t;
 #define XLOG_MIN_RECORD_BSHIFT	14		/* 16384 == 1 << 14 */
 #define XLOG_BIG_RECORD_BSHIFT	15		/* 32k == 1 << 15 */
 #define XLOG_MAX_RECORD_BSHIFT	18		/* 256k == 1 << 18 */
-#define XLOG_BTOLSUNIT(log, b)  (((b)+(log)->l_mp->m_sb.sb_logsunit-1) / \
-                                 (log)->l_mp->m_sb.sb_logsunit)
-#define XLOG_LSUNITTOB(log, su) ((su) * (log)->l_mp->m_sb.sb_logsunit)
 
 #define XLOG_HEADER_SIZE	512
 
fs/xfs/xfs_log.c +25 -34
index 06041834daa3..fa284f26d10e 100644
@@ -1399,6 +1399,11 @@ xlog_alloc_log(
 	xlog_assign_atomic_lsn(&log->l_last_sync_lsn, 1, 0);
 	log->l_curr_cycle  = 1;	    /* 0 is bad since this is initial value */
 
+	if (xfs_sb_version_haslogv2(&mp->m_sb) && mp->m_sb.sb_logsunit > 1)
+		log->l_iclog_roundoff = mp->m_sb.sb_logsunit;
+	else
+		log->l_iclog_roundoff = BBSIZE;
+
 	xlog_grant_head_init(&log->l_reserve_head);
 	xlog_grant_head_init(&log->l_write_head);
 
@@ -1852,29 +1857,15 @@ xlog_calc_iclog_size(
 	uint32_t		*roundoff)
 {
 	uint32_t		count_init, count;
-	bool			use_lsunit;
-
-	use_lsunit = xfs_sb_version_haslogv2(&log->l_mp->m_sb) &&
-			log->l_mp->m_sb.sb_logsunit > 1;
 
 	/* Add for LR header */
 	count_init = log->l_iclog_hsize + iclog->ic_offset;
+	count = roundup(count_init, log->l_iclog_roundoff);
 
-	/* Round out the log write size */
-	if (use_lsunit) {
-		/* we have a v2 stripe unit to use */
-		count = XLOG_LSUNITTOB(log, XLOG_BTOLSUNIT(log, count_init));
-	} else {
-		count = BBTOB(BTOBB(count_init));
-	}
-
-	ASSERT(count >= count_init);
 	*roundoff = count - count_init;
 
-	if (use_lsunit)
-		ASSERT(*roundoff < log->l_mp->m_sb.sb_logsunit);
-	else
-		ASSERT(*roundoff < BBTOB(1));
+	ASSERT(count >= count_init);
+	ASSERT(*roundoff < log->l_iclog_roundoff);
 	return count;
 }
 
@@ -3149,10 +3140,9 @@ xlog_state_switch_iclogs(
 	log->l_curr_block += BTOBB(eventual_size)+BTOBB(log->l_iclog_hsize);
 
 	/* Round up to next log-sunit */
-	if (xfs_sb_version_haslogv2(&log->l_mp->m_sb) &&
-	    log->l_mp->m_sb.sb_logsunit > 1) {
-		uint32_t sunit_bb = BTOBB(log->l_mp->m_sb.sb_logsunit);
-		log->l_curr_block = roundup(log->l_curr_block, sunit_bb);
+	if (log->l_iclog_roundoff > BBSIZE) {
+		log->l_curr_block = roundup(log->l_curr_block,
+						BTOBB(log->l_iclog_roundoff));
 	}
 
 	if (log->l_curr_block >= log->l_logBBsize) {
@@ -3404,12 +3394,11 @@ xfs_log_ticket_get(
  * Figure out the total log space unit (in bytes) that would be
  * required for a log ticket.
  */
-int
-xfs_log_calc_unit_res(
-	struct xfs_mount	*mp,
+static int
+xlog_calc_unit_res(
+	struct xlog		*log,
 	int			unit_bytes)
 {
-	struct xlog		*log = mp->m_log;
 	int			iclog_space;
 	uint			num_headers;
 
@@ -3485,18 +3474,20 @@ xfs_log_calc_unit_res(
 	/* for commit-rec LR header - note: padding will subsume the ophdr */
 	unit_bytes += log->l_iclog_hsize;
 
-	/* for roundoff padding for transaction data and one for commit record */
-	if (xfs_sb_version_haslogv2(&mp->m_sb) && mp->m_sb.sb_logsunit > 1) {
-		/* log su roundoff */
-		unit_bytes += 2 * mp->m_sb.sb_logsunit;
-	} else {
-		/* BB roundoff */
-		unit_bytes += 2 * BBSIZE;
-        }
+	/* roundoff padding for transaction data and one for commit record */
+	unit_bytes += 2 * log->l_iclog_roundoff;
 
 	return unit_bytes;
 }
 
+int
+xfs_log_calc_unit_res(
+	struct xfs_mount	*mp,
+	int			unit_bytes)
+{
+	return xlog_calc_unit_res(mp->m_log, unit_bytes);
+}
+
 /*
  * Allocate and initialise a new log ticket.
  */
@@ -3513,7 +3504,7 @@ xlog_ticket_alloc(
 
 	tic = kmem_cache_zalloc(xfs_log_ticket_zone, GFP_NOFS | __GFP_NOFAIL);
 
-	unit_res = xfs_log_calc_unit_res(log->l_mp, unit_bytes);
+	unit_res = xlog_calc_unit_res(log, unit_bytes);
 
 	atomic_set(&tic->t_ref, 1);
 	tic->t_task		= current;
fs/xfs/xfs_log_priv.h +2 -0
index 1c6fdbf3d506..037950cf1061 100644
@@ -436,6 +436,8 @@ struct xlog {
 #endif
 	/* log recovery lsn tracking (for buffer submission */
 	xfs_lsn_t		l_recovery_lsn;
+
+	uint32_t		l_iclog_roundoff;/* padding roundoff */
 };
 
 #define XLOG_BUF_CANCEL_BUCKET(log, blkno) \
Signature · 1 line
2.28.0
#6 lore raw

PATCH 2/8 xfs: separate CIL commit record IO

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

To allow for iclog IO device cache flush behaviour to be optimised, we first need to separate out the commit record iclog IO from the rest of the checkpoint so we can wait for the checkpoint IO to complete before we issue the commit record.

This separation is only necessary if the commit record is being written into a different iclog to the start of the checkpoint as the upcoming cache flushing changes requires completion ordering against the other iclogs submitted by the checkpoint.

If the entire checkpoint and commit is in the one iclog, then they are both covered by the one set of cache flush primitives on the iclog and hence there is no need to separate them for ordering.

Otherwise, we need to wait for all the previous iclogs to complete so they are ordered correctly and made stable by the REQ_PREFLUSH that the commit record iclog IO issues. This guarantees that if a reader sees the commit record in the journal, they will also see the entire checkpoint that commit record closes off.

This also provides the guarantee that when the commit record IO completes, we can safely unpin all the log items in the checkpoint so they can be written back because the entire checkpoint is stable in the journal.

---
 fs/xfs/xfs_log.c      | 55 +++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_log_cil.c  |  7 ++++++
 fs/xfs/xfs_log_priv.h |  2 ++
 3 files changed, 64 insertions(+)
fs/xfs/xfs_log.c +55 -0
index fa284f26d10e..ff26fb46d70f 100644
@@ -808,6 +808,61 @@ xlog_wait_on_iclog(
 	return 0;
 }
 
+/*
+ * Wait on any iclogs that are still flushing in the range of start_lsn to the
+ * current iclog's lsn. The caller holds a reference to the iclog, but otherwise
+ * holds no log locks.
+ *
+ * We walk backwards through the iclogs to find the iclog with the highest lsn
+ * in the range that we need to wait for and then wait for it to complete.
+ * Completion ordering of iclog IOs ensures that all prior iclogs to the
+ * candidate iclog we need to sleep on have been complete by the time our
+ * candidate has completed it's IO.
+ *
+ * Therefore we only need to find the first iclog that isn't clean within the
+ * span of our flush range. If we come across a clean, newly activated iclog
+ * with a lsn of 0, it means IO has completed on this iclog and all previous
+ * iclogs will be have been completed prior to this one. Hence finding a newly
+ * activated iclog indicates that there are no iclogs in the range we need to
+ * wait on and we are done searching.
+ */
+int
+xlog_wait_on_iclog_lsn(
+	struct xlog_in_core	*iclog,
+	xfs_lsn_t		start_lsn)
+{
+	struct xlog		*log = iclog->ic_log;
+	struct xlog_in_core	*prev;
+	int			error = -EIO;
+
+	spin_lock(&log->l_icloglock);
+	if (XLOG_FORCED_SHUTDOWN(log))
+		goto out_unlock;
+
+	error = 0;
+	for (prev = iclog->ic_prev; prev != iclog; prev = prev->ic_prev) {
+
+		/* Done if the lsn is before our start lsn */
+		if (XFS_LSN_CMP(be64_to_cpu(prev->ic_header.h_lsn),
+				start_lsn) < 0)
+			break;
+
+		/* Don't need to wait on completed, clean iclogs */
+		if (prev->ic_state == XLOG_STATE_DIRTY ||
+		    prev->ic_state == XLOG_STATE_ACTIVE) {
+			continue;
+		}
+
+		/* wait for completion on this iclog */
+		xlog_wait(&prev->ic_force_wait, &log->l_icloglock);
+		return 0;
+	}
+
+out_unlock:
+	spin_unlock(&log->l_icloglock);
+	return error;
+}
+
 /*
  * Write out an unmount record using the ticket provided. We have to account for
  * the data space used in the unmount ticket as this write is not done from a
fs/xfs/xfs_log_cil.c +7 -0
index b0ef071b3cb5..c5cc1b7ad25e 100644
@@ -870,6 +870,13 @@ xlog_cil_push_work(
 	wake_up_all(&cil->xc_commit_wait);
 	spin_unlock(&cil->xc_push_lock);
 
+	/*
+	 * If the checkpoint spans multiple iclogs, wait for all previous
+	 * iclogs to complete before we submit the commit_iclog.
+	 */
+	if (ctx->start_lsn != commit_lsn)
+		xlog_wait_on_iclog_lsn(commit_iclog, ctx->start_lsn);
+
 	/* release the hounds! */
 	xfs_log_release_iclog(commit_iclog);
 	return;
fs/xfs/xfs_log_priv.h +2 -0
index 037950cf1061..a7ac85aaff4e 100644
@@ -584,6 +584,8 @@ xlog_wait(
 	remove_wait_queue(wq, &wait);
 }
 
+int xlog_wait_on_iclog_lsn(struct xlog_in_core *iclog, xfs_lsn_t start_lsn);
+
 /*
  * The LSN is valid so long as it is behind the current LSN. If it isn't, this
  * means that the next log record that includes this metadata could have a
Signature · 1 line
2.28.0
Reviewed-by on v1 2/8
#7 lore raw

Re: [PATCH 2/8] xfs: separate CIL commit record IO

Quoted 22 lines · Dave Chinner

From: Dave Chinner <[email protected]>

To allow for iclog IO device cache flush behaviour to be optimised, we first need to separate out the commit record iclog IO from the rest of the checkpoint so we can wait for the checkpoint IO to complete before we issue the commit record.

This separation is only necessary if the commit record is being written into a different iclog to the start of the checkpoint as the upcoming cache flushing changes requires completion ordering against the other iclogs submitted by the checkpoint.

If the entire checkpoint and commit is in the one iclog, then they are both covered by the one set of cache flush primitives on the iclog and hence there is no need to separate them for ordering.

Otherwise, we need to wait for all the previous iclogs to complete so they are ordered correctly and made stable by the REQ_PREFLUSH that the commit record iclog IO issues. This guarantees that if a reader sees the commit record in the journal, they will also see the entire checkpoint that commit record closes off.

Quoted 5 lines

This also provides the guarantee that when the commit record IO completes, we can safely unpin all the log items in the checkpoint so they can be written back because the entire checkpoint is stable in the journal.

The changes seem to be logically correct.

Quoted 104 lines
---
 fs/xfs/xfs_log.c      | 55 +++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_log_cil.c  |  7 ++++++
 fs/xfs/xfs_log_priv.h |  2 ++
 3 files changed, 64 insertions(+)
fs/xfs/xfs_log.c +55 -0
index fa284f26d10e..ff26fb46d70f 100644
@@ -808,6 +808,61 @@ xlog_wait_on_iclog(
 	return 0;
 }
 
+/*
+ * Wait on any iclogs that are still flushing in the range of start_lsn to the
+ * current iclog's lsn. The caller holds a reference to the iclog, but otherwise
+ * holds no log locks.
+ *
+ * We walk backwards through the iclogs to find the iclog with the highest lsn
+ * in the range that we need to wait for and then wait for it to complete.
+ * Completion ordering of iclog IOs ensures that all prior iclogs to the
+ * candidate iclog we need to sleep on have been complete by the time our
+ * candidate has completed it's IO.
+ *
+ * Therefore we only need to find the first iclog that isn't clean within the
+ * span of our flush range. If we come across a clean, newly activated iclog
+ * with a lsn of 0, it means IO has completed on this iclog and all previous
+ * iclogs will be have been completed prior to this one. Hence finding a newly
+ * activated iclog indicates that there are no iclogs in the range we need to
+ * wait on and we are done searching.
+ */
+int
+xlog_wait_on_iclog_lsn(
+	struct xlog_in_core	*iclog,
+	xfs_lsn_t		start_lsn)
+{
+	struct xlog		*log = iclog->ic_log;
+	struct xlog_in_core	*prev;
+	int			error = -EIO;
+
+	spin_lock(&log->l_icloglock);
+	if (XLOG_FORCED_SHUTDOWN(log))
+		goto out_unlock;
+
+	error = 0;
+	for (prev = iclog->ic_prev; prev != iclog; prev = prev->ic_prev) {
+
+		/* Done if the lsn is before our start lsn */
+		if (XFS_LSN_CMP(be64_to_cpu(prev->ic_header.h_lsn),
+				start_lsn) < 0)
+			break;
+
+		/* Don't need to wait on completed, clean iclogs */
+		if (prev->ic_state == XLOG_STATE_DIRTY ||
+		    prev->ic_state == XLOG_STATE_ACTIVE) {
+			continue;
+		}
+
+		/* wait for completion on this iclog */
+		xlog_wait(&prev->ic_force_wait, &log->l_icloglock);
+		return 0;
+	}
+
+out_unlock:
+	spin_unlock(&log->l_icloglock);
+	return error;
+}
+
 /*
  * Write out an unmount record using the ticket provided. We have to account for
  * the data space used in the unmount ticket as this write is not done from a
fs/xfs/xfs_log_cil.c +7 -0
index b0ef071b3cb5..c5cc1b7ad25e 100644
@@ -870,6 +870,13 @@ xlog_cil_push_work(
 	wake_up_all(&cil->xc_commit_wait);
 	spin_unlock(&cil->xc_push_lock);
 
+	/*
+	 * If the checkpoint spans multiple iclogs, wait for all previous
+	 * iclogs to complete before we submit the commit_iclog.
+	 */
+	if (ctx->start_lsn != commit_lsn)
+		xlog_wait_on_iclog_lsn(commit_iclog, ctx->start_lsn);
+
 	/* release the hounds! */
 	xfs_log_release_iclog(commit_iclog);
 	return;
fs/xfs/xfs_log_priv.h +2 -0
index 037950cf1061..a7ac85aaff4e 100644
@@ -584,6 +584,8 @@ xlog_wait(
 	remove_wait_queue(wq, &wait);
 }
 
+int xlog_wait_on_iclog_lsn(struct xlog_in_core *iclog, xfs_lsn_t start_lsn);
+
 /*
  * The LSN is valid so long as it is behind the current LSN. If it isn't, this
  * means that the next log record that includes this metadata could have a
Signature · 1 line
chandan
#8 lore raw

Re: [PATCH 2/8] xfs: separate CIL commit record IO

In reply to Dave Chinner
Quoted 42 lines · Dave Chinner

From: Dave Chinner <[email protected]>

To allow for iclog IO device cache flush behaviour to be optimised, we first need to separate out the commit record iclog IO from the rest of the checkpoint so we can wait for the checkpoint IO to complete before we issue the commit record.

This separation is only necessary if the commit record is being written into a different iclog to the start of the checkpoint as the upcoming cache flushing changes requires completion ordering against the other iclogs submitted by the checkpoint.

If the entire checkpoint and commit is in the one iclog, then they are both covered by the one set of cache flush primitives on the iclog and hence there is no need to separate them for ordering.

Otherwise, we need to wait for all the previous iclogs to complete so they are ordered correctly and made stable by the REQ_PREFLUSH that the commit record iclog IO issues. This guarantees that if a reader sees the commit record in the journal, they will also see the entire checkpoint that commit record closes off.

This also provides the guarantee that when the commit record IO completes, we can safely unpin all the log items in the checkpoint so they can be written back because the entire checkpoint is stable in the journal.

---
 fs/xfs/xfs_log.c      | 55 +++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_log_cil.c  |  7 ++++++
 fs/xfs/xfs_log_priv.h |  2 ++
 3 files changed, 64 insertions(+)
fs/xfs/xfs_log.c +0 -0
index fa284f26d10e..ff26fb46d70f 100644
@@ -808,6 +808,61 @@ xlog_wait_on_iclog(
 	return 0;
 }
 

fs/xfs/xfs_log.c xlog_wait_on_iclog( · line 820

Quoted 10 lines
fs/xfs/xfs_log.c +10 -0
+/*
+ * Wait on any iclogs that are still flushing in the range of start_lsn to the
+ * current iclog's lsn. The caller holds a reference to the iclog, but otherwise
+ * holds no log locks.
+ *
+ * We walk backwards through the iclogs to find the iclog with the highest lsn
+ * in the range that we need to wait for and then wait for it to complete.
+ * Completion ordering of iclog IOs ensures that all prior iclogs to the
+ * candidate iclog we need to sleep on have been complete by the time our
+ * candidate has completed it's IO.

Hmm, I guess this means that iclog header lsns are supposed to increase as one walks forwards through the list?

fs/xfs/xfs_log.c xlog_wait_on_iclog( · line 827

Quoted 7 lines
fs/xfs/xfs_log.c +7 -0
+ *
+ * Therefore we only need to find the first iclog that isn't clean within the
+ * span of our flush range. If we come across a clean, newly activated iclog
+ * with a lsn of 0, it means IO has completed on this iclog and all previous
+ * iclogs will be have been completed prior to this one. Hence finding a newly
+ * activated iclog indicates that there are no iclogs in the range we need to
+ * wait on and we are done searching.

I don't see an explicit check for an iclog with a zero lsn? Is that implied by XLOG_STATE_ACTIVE?

Also, do you have any idea what was Christoph talking about wrt devices with no-op flushes the last time this patch was posted? This change seems straightforward to me (assuming the answers to my two question are 'yes') but I didn't grok what subtlety he was alluding to...?

--D

Quoted 75 lines
fs/xfs/xfs_log.c +38 -0
+ */
+int
+xlog_wait_on_iclog_lsn(
+	struct xlog_in_core	*iclog,
+	xfs_lsn_t		start_lsn)
+{
+	struct xlog		*log = iclog->ic_log;
+	struct xlog_in_core	*prev;
+	int			error = -EIO;
+
+	spin_lock(&log->l_icloglock);
+	if (XLOG_FORCED_SHUTDOWN(log))
+		goto out_unlock;
+
+	error = 0;
+	for (prev = iclog->ic_prev; prev != iclog; prev = prev->ic_prev) {
+
+		/* Done if the lsn is before our start lsn */
+		if (XFS_LSN_CMP(be64_to_cpu(prev->ic_header.h_lsn),
+				start_lsn) < 0)
+			break;
+
+		/* Don't need to wait on completed, clean iclogs */
+		if (prev->ic_state == XLOG_STATE_DIRTY ||
+		    prev->ic_state == XLOG_STATE_ACTIVE) {
+			continue;
+		}
+
+		/* wait for completion on this iclog */
+		xlog_wait(&prev->ic_force_wait, &log->l_icloglock);
+		return 0;
+	}
+
+out_unlock:
+	spin_unlock(&log->l_icloglock);
+	return error;
+}
+
 /*
  * Write out an unmount record using the ticket provided. We have to account for
  * the data space used in the unmount ticket as this write is not done from a
fs/xfs/xfs_log_cil.c +7 -0
index b0ef071b3cb5..c5cc1b7ad25e 100644
@@ -870,6 +870,13 @@ xlog_cil_push_work(
 	wake_up_all(&cil->xc_commit_wait);
 	spin_unlock(&cil->xc_push_lock);
 
+	/*
+	 * If the checkpoint spans multiple iclogs, wait for all previous
+	 * iclogs to complete before we submit the commit_iclog.
+	 */
+	if (ctx->start_lsn != commit_lsn)
+		xlog_wait_on_iclog_lsn(commit_iclog, ctx->start_lsn);
+
 	/* release the hounds! */
 	xfs_log_release_iclog(commit_iclog);
 	return;
fs/xfs/xfs_log_priv.h +2 -0
index 037950cf1061..a7ac85aaff4e 100644
@@ -584,6 +584,8 @@ xlog_wait(
 	remove_wait_queue(wq, &wait);
 }
 
+int xlog_wait_on_iclog_lsn(struct xlog_in_core *iclog, xfs_lsn_t start_lsn);
+
 /*
  * The LSN is valid so long as it is behind the current LSN. If it isn't, this
  * means that the next log record that includes this metadata could have a
Signature · 1 line
2.28.0
#9 lore raw

Re: [PATCH 2/8] xfs: separate CIL commit record IO

Quoted 43 lines · Darrick J. Wong
Quoted 42 lines · Dave Chinner

From: Dave Chinner <[email protected]>

To allow for iclog IO device cache flush behaviour to be optimised, we first need to separate out the commit record iclog IO from the rest of the checkpoint so we can wait for the checkpoint IO to complete before we issue the commit record.

This separation is only necessary if the commit record is being written into a different iclog to the start of the checkpoint as the upcoming cache flushing changes requires completion ordering against the other iclogs submitted by the checkpoint.

If the entire checkpoint and commit is in the one iclog, then they are both covered by the one set of cache flush primitives on the iclog and hence there is no need to separate them for ordering.

Otherwise, we need to wait for all the previous iclogs to complete so they are ordered correctly and made stable by the REQ_PREFLUSH that the commit record iclog IO issues. This guarantees that if a reader sees the commit record in the journal, they will also see the entire checkpoint that commit record closes off.

This also provides the guarantee that when the commit record IO completes, we can safely unpin all the log items in the checkpoint so they can be written back because the entire checkpoint is stable in the journal.

---
 fs/xfs/xfs_log.c      | 55 +++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_log_cil.c  |  7 ++++++
 fs/xfs/xfs_log_priv.h |  2 ++
 3 files changed, 64 insertions(+)
fs/xfs/xfs_log.c +0 -0
index fa284f26d10e..ff26fb46d70f 100644
@@ -808,6 +808,61 @@ xlog_wait_on_iclog(
 	return 0;
 }
 
Quoted 13 lines

fs/xfs/xfs_log.c xlog_wait_on_iclog( · line 820

Quoted 10 lines
fs/xfs/xfs_log.c +10 -0
+/*
+ * Wait on any iclogs that are still flushing in the range of start_lsn to the
+ * current iclog's lsn. The caller holds a reference to the iclog, but otherwise
+ * holds no log locks.
+ *
+ * We walk backwards through the iclogs to find the iclog with the highest lsn
+ * in the range that we need to wait for and then wait for it to complete.
+ * Completion ordering of iclog IOs ensures that all prior iclogs to the
+ * candidate iclog we need to sleep on have been complete by the time our
+ * candidate has completed it's IO.

Hmm, I guess this means that iclog header lsns are supposed to increase as one walks forwards through the list?

yes, the iclogs are written sequentially to the log - we don't switch the log->l_iclog pointer to the current active iclog until we switch it out, and then the next iclog in the loop is physically located at a higher lsn to the one we just switched out.

Quoted 10 lines

fs/xfs/xfs_log.c xlog_wait_on_iclog( · line 827

Quoted 7 lines
fs/xfs/xfs_log.c +7 -0
+ *
+ * Therefore we only need to find the first iclog that isn't clean within the
+ * span of our flush range. If we come across a clean, newly activated iclog
+ * with a lsn of 0, it means IO has completed on this iclog and all previous
+ * iclogs will be have been completed prior to this one. Hence finding a newly
+ * activated iclog indicates that there are no iclogs in the range we need to
+ * wait on and we are done searching.

I don't see an explicit check for an iclog with a zero lsn? Is that implied by XLOG_STATE_ACTIVE?

It's handled by the XFS_LSN_CMP(prev_lsn, start_lsn) < 0 check. if the prev_lsn is zero because the iclog is clean, then this check will always be true.

Quoted 4 lines

Also, do you have any idea what was Christoph talking about wrt devices with no-op flushes the last time this patch was posted? This change seems straightforward to me (assuming the answers to my two question are 'yes') but I didn't grok what subtlety he was alluding to...?

He was wondering what devices benefited from this. It has no impact on highspeed devices that do not require flushes/FUA (e.g. high end intel optane SSDs) but those are not the devices this change is aimed at. There are no regressions on these high end devices, either, so they are largely irrelevant to the patch and what it targets...

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
Reviewed-by
#10 lore raw

Re: [PATCH 2/8] xfs: separate CIL commit record IO

Quoted 83 lines · Dave Chinner
Quoted 43 lines · Darrick J. Wong
Quoted 42 lines · Dave Chinner

From: Dave Chinner <[email protected]>

To allow for iclog IO device cache flush behaviour to be optimised, we first need to separate out the commit record iclog IO from the rest of the checkpoint so we can wait for the checkpoint IO to complete before we issue the commit record.

This separation is only necessary if the commit record is being written into a different iclog to the start of the checkpoint as the upcoming cache flushing changes requires completion ordering against the other iclogs submitted by the checkpoint.

If the entire checkpoint and commit is in the one iclog, then they are both covered by the one set of cache flush primitives on the iclog and hence there is no need to separate them for ordering.

Otherwise, we need to wait for all the previous iclogs to complete so they are ordered correctly and made stable by the REQ_PREFLUSH that the commit record iclog IO issues. This guarantees that if a reader sees the commit record in the journal, they will also see the entire checkpoint that commit record closes off.

This also provides the guarantee that when the commit record IO completes, we can safely unpin all the log items in the checkpoint so they can be written back because the entire checkpoint is stable in the journal.

---
 fs/xfs/xfs_log.c      | 55 +++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_log_cil.c  |  7 ++++++
 fs/xfs/xfs_log_priv.h |  2 ++
 3 files changed, 64 insertions(+)
fs/xfs/xfs_log.c +0 -0
index fa284f26d10e..ff26fb46d70f 100644
@@ -808,6 +808,61 @@ xlog_wait_on_iclog(
 	return 0;
 }
 
Quoted 13 lines

fs/xfs/xfs_log.c xlog_wait_on_iclog( · line 820

Quoted 10 lines
fs/xfs/xfs_log.c +10 -0
+/*
+ * Wait on any iclogs that are still flushing in the range of start_lsn to the
+ * current iclog's lsn. The caller holds a reference to the iclog, but otherwise
+ * holds no log locks.
+ *
+ * We walk backwards through the iclogs to find the iclog with the highest lsn
+ * in the range that we need to wait for and then wait for it to complete.
+ * Completion ordering of iclog IOs ensures that all prior iclogs to the
+ * candidate iclog we need to sleep on have been complete by the time our
+ * candidate has completed it's IO.

Hmm, I guess this means that iclog header lsns are supposed to increase as one walks forwards through the list?

yes, the iclogs are written sequentially to the log - we don't switch the log->l_iclog pointer to the current active iclog until we switch it out, and then the next iclog in the loop is physically located at a higher lsn to the one we just switched out.

Quoted 10 lines

fs/xfs/xfs_log.c xlog_wait_on_iclog( · line 827

Quoted 7 lines
fs/xfs/xfs_log.c +7 -0
+ *
+ * Therefore we only need to find the first iclog that isn't clean within the
+ * span of our flush range. If we come across a clean, newly activated iclog
+ * with a lsn of 0, it means IO has completed on this iclog and all previous
+ * iclogs will be have been completed prior to this one. Hence finding a newly
+ * activated iclog indicates that there are no iclogs in the range we need to
+ * wait on and we are done searching.

I don't see an explicit check for an iclog with a zero lsn? Is that implied by XLOG_STATE_ACTIVE?

It's handled by the XFS_LSN_CMP(prev_lsn, start_lsn) < 0 check. if the prev_lsn is zero because the iclog is clean, then this check will always be true.

Quoted 4 lines

Also, do you have any idea what was Christoph talking about wrt devices with no-op flushes the last time this patch was posted? This change seems straightforward to me (assuming the answers to my two question are 'yes') but I didn't grok what subtlety he was alluding to...?

Quoted 6 lines

He was wondering what devices benefited from this. It has no impact on highspeed devices that do not require flushes/FUA (e.g. high end intel optane SSDs) but those are not the devices this change is aimed at. There are no regressions on these high end devices, either, so they are largely irrelevant to the patch and what it targets...

Ok, that's what I thought. It seemed fairly self-evident to me that high speed devices wouldn't care.

--D

Quoted 6 lines

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#11 lore raw

Re: [PATCH 2/8] xfs: separate CIL commit record IO

In reply to Dave Chinner
Quoted 11 lines · Dave Chinner
Quoted 4 lines

Also, do you have any idea what was Christoph talking about wrt devices with no-op flushes the last time this patch was posted? This change seems straightforward to me (assuming the answers to my two question are 'yes') but I didn't grok what subtlety he was alluding to...?

He was wondering what devices benefited from this. It has no impact on highspeed devices that do not require flushes/FUA (e.g. high end intel optane SSDs) but those are not the devices this change is aimed at. There are no regressions on these high end devices, either, so they are largely irrelevant to the patch and what it targets...

I don't think it is that simple. Pretty much every device aimed at enterprise use does not enable a volatile write cache by default. That also includes hard drives, arrays and NAND based SSDs.

Especially for hard drives (or slower arrays) the actual I/O wait might matter. What is the argument against making this conditional?

#12 lore raw

Re: [PATCH 2/8] xfs: separate CIL commit record IO

Quoted 13 lines · Christoph Hellwig
Quoted 11 lines · Dave Chinner
Quoted 4 lines

Also, do you have any idea what was Christoph talking about wrt devices with no-op flushes the last time this patch was posted? This change seems straightforward to me (assuming the answers to my two question are 'yes') but I didn't grok what subtlety he was alluding to...?

He was wondering what devices benefited from this. It has no impact on highspeed devices that do not require flushes/FUA (e.g. high end intel optane SSDs) but those are not the devices this change is aimed at. There are no regressions on these high end devices, either, so they are largely irrelevant to the patch and what it targets...

Quoted 6 lines

I don't think it is that simple. Pretty much every device aimed at enterprise use does not enable a volatile write cache by default. That also includes hard drives, arrays and NAND based SSDs.

Especially for hard drives (or slower arrays) the actual I/O wait might matter.

Sorry, I/O wait might matter for what?

I'm really not sure what you're objecting to - you've hand-waved about hardware that doesn't need cache flushes twice now and inferred that they'd be adversely affected by removing cache flushes. That just doesn't make any sense at all, and I have numbers to back it up.

You also asked what storage it improved performance on and I told you and then also pointed out all the software layers that it massively helps, too, regardless of the physical storage characteristics.

https://lore.kernel.org/linux-xfs/[email protected]/

I have numbers to back it up. You did not reply to me, so I'm not going to waste time repeating myself here.

Quoted 1 line

What is the argument against making this conditional?

There is no argument for making this conditional. You've created an undefined strawman and are demanding that I prove it wrong. If you've got anything concrete, then tell us about it directly and provide numbers.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#13 lore raw

Re: [PATCH 2/8] xfs: separate CIL commit record IO

Quoted 1 line · Dave Chinner

Sorry, I/O wait might matter for what?

Think you have a SAS hard drive, WCE=0, typical queue depth of a few dozend commands.

Before that we'd submit a bunch of iclogs, which are generally sequential except of course for the log wrap around case. The drive can now easily take all the iclogs and write them in one rotation.

Now if we wait for the previous iclogs before submitting the commit_iclog we need at least one more additional full roundtrip.

#14 lore raw

Re: [PATCH 2/8] xfs: separate CIL commit record IO

Quoted 5 lines · Christoph Hellwig
Quoted 1 line · Dave Chinner

Sorry, I/O wait might matter for what?

Think you have a SAS hard drive, WCE=0, typical queue depth of a few dozend commands.

Yup, so typical IO latency of 2-3ms, assuming 10krpm, assuming we aren't doing metadata writeback which will blow this out.

I've tested on slower iSCSI devices than this (7-8ms typical av. seek time), and it didn't show up any performance anomalies.

Quoted 3 lines

Before that we'd submit a bunch of iclogs, which are generally sequential except of course for the log wrap around case. The drive can now easily take all the iclogs and write them in one rotation.

Even if we take the best case for your example, this still means we block on every 8 iclogs waiting 2-3ms for the spindle to rotate and complete the IOs. Hence for a checkpoint of 32MB with 256kB iclogs, we're blocking for 2-3ms at least 16 times before we get to the commit iclog. With default iclog size of 32kB, we'll block a couple of hundred times waiting on iclog IO...

IOWs, we're already talking about a best case checkpoint commit latency of 30-50ms here.

[ And this isn't even considering media bandwidth there - 32MB on a drive that can do maybe 200MB/s in the middle of the spindle where the log is. That's another 150ms of data transfer time to physical media. So if the drive is actually writing to physical media because WCE=0, then we're taking *at least* 200ms per 32MB checkpoint. ]

Quoted 2 lines

Now if we wait for the previous iclogs before submitting the commit_iclog we need at least one more additional full roundtrip.

So we add an average of 2-3ms to what is already taking, in the best case, 30-50ms.

And these are mostly async commits this overhead is added to, so there's rarely anything waiting on it and hence the extra small latency is almost always lost in the noise. Even if the extra delay is larger, there is rarely anything waiting on it so it's still noise...

I just don't see anything relevant that stands out from the noise on my systems.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#15 lore raw

Re: [PATCH 2/8] xfs: separate CIL commit record IO

In reply to Christoph Hellwig
Quoted 13 lines · Christoph Hellwig
Quoted 11 lines · Dave Chinner
Quoted 4 lines

Also, do you have any idea what was Christoph talking about wrt devices with no-op flushes the last time this patch was posted? This change seems straightforward to me (assuming the answers to my two question are 'yes') but I didn't grok what subtlety he was alluding to...?

He was wondering what devices benefited from this. It has no impact on highspeed devices that do not require flushes/FUA (e.g. high end intel optane SSDs) but those are not the devices this change is aimed at. There are no regressions on these high end devices, either, so they are largely irrelevant to the patch and what it targets...

Quoted 6 lines

I don't think it is that simple. Pretty much every device aimed at enterprise use does not enable a volatile write cache by default. That also includes hard drives, arrays and NAND based SSDs.

Especially for hard drives (or slower arrays) the actual I/O wait might matter. What is the argument against making this conditional?

I still don't understand what you're asking about here --

AFAICT the net effect of this patchset is that it reduces the number of preflushes and FUA log writes. To my knowledge, on a high end device with no volatile write cache, flushes are a no-op (because all writes are persisted somewhere immediately) and a FUA write should be the exact same thing as a non-FUA write. Because XFS will now issue fewer no-op persistence commands to the device, there should be no effect at all.

In contrast, a dumb stone tablet with a write cache hooked up to SATA will have agonizingly slow cache flushes. XFS will issue fewer persistence commands to the rock, which in turn makes things faster because we're calling the engravers less often.

What am I missing here? Are you saying that the cost of a cache flush goes up much faster than the amount of data that has to be flushed?

--D

#16 lore raw

Re: [PATCH 2/8] xfs: separate CIL commit record IO

Quoted 21 lines · Darrick J. Wong
Quoted 13 lines · Christoph Hellwig
Quoted 11 lines · Dave Chinner
Quoted 4 lines

Also, do you have any idea what was Christoph talking about wrt devices with no-op flushes the last time this patch was posted? This change seems straightforward to me (assuming the answers to my two question are 'yes') but I didn't grok what subtlety he was alluding to...?

He was wondering what devices benefited from this. It has no impact on highspeed devices that do not require flushes/FUA (e.g. high end intel optane SSDs) but those are not the devices this change is aimed at. There are no regressions on these high end devices, either, so they are largely irrelevant to the patch and what it targets...

Quoted 6 lines

I don't think it is that simple. Pretty much every device aimed at enterprise use does not enable a volatile write cache by default. That also includes hard drives, arrays and NAND based SSDs.

Especially for hard drives (or slower arrays) the actual I/O wait might matter. What is the argument against making this conditional?

Quoted 9 lines

I still don't understand what you're asking about here --

AFAICT the net effect of this patchset is that it reduces the number of preflushes and FUA log writes. To my knowledge, on a high end device with no volatile write cache, flushes are a no-op (because all writes are persisted somewhere immediately) and a FUA write should be the exact same thing as a non-FUA write. Because XFS will now issue fewer no-op persistence commands to the device, there should be no effect at all.

Except the cost of the new iowaits used to implement iclog ordering... which I think is what Christoph has been asking about..?

IOW, considering the storage configuration noted above where the impact of the flush/fua optimizations is neutral, the net effect of this change is whatever impact is introduced by intra-checkpoint iowaits and iclog ordering. What is that impact?

Note that it's not clear enough to me to suggest whether that impact might be significant or not. Hopefully it's neutral (?), but that seems like best case scenario so I do think it's a reasonable question.

Brian

Quoted 10 lines

In contrast, a dumb stone tablet with a write cache hooked up to SATA will have agonizingly slow cache flushes. XFS will issue fewer persistence commands to the rock, which in turn makes things faster because we're calling the engravers less often.

What am I missing here? Are you saying that the cost of a cache flush goes up much faster than the amount of data that has to be flushed?

--D

#17 lore raw

Re: [PATCH 2/8] xfs: separate CIL commit record IO

Quoted 32 lines · Brian Foster
Quoted 21 lines · Darrick J. Wong
Quoted 13 lines · Christoph Hellwig
Quoted 11 lines · Dave Chinner
Quoted 4 lines

Also, do you have any idea what was Christoph talking about wrt devices with no-op flushes the last time this patch was posted? This change seems straightforward to me (assuming the answers to my two question are 'yes') but I didn't grok what subtlety he was alluding to...?

He was wondering what devices benefited from this. It has no impact on highspeed devices that do not require flushes/FUA (e.g. high end intel optane SSDs) but those are not the devices this change is aimed at. There are no regressions on these high end devices, either, so they are largely irrelevant to the patch and what it targets...

Quoted 6 lines

I don't think it is that simple. Pretty much every device aimed at enterprise use does not enable a volatile write cache by default. That also includes hard drives, arrays and NAND based SSDs.

Especially for hard drives (or slower arrays) the actual I/O wait might matter. What is the argument against making this conditional?

Quoted 9 lines

I still don't understand what you're asking about here --

AFAICT the net effect of this patchset is that it reduces the number of preflushes and FUA log writes. To my knowledge, on a high end device with no volatile write cache, flushes are a no-op (because all writes are persisted somewhere immediately) and a FUA write should be the exact same thing as a non-FUA write. Because XFS will now issue fewer no-op persistence commands to the device, there should be no effect at all.

Quoted 2 lines

Except the cost of the new iowaits used to implement iclog ordering... which I think is what Christoph has been asking about..?

And I've already answered - it is largely just noise.

Quoted 4 lines

IOW, considering the storage configuration noted above where the impact of the flush/fua optimizations is neutral, the net effect of this change is whatever impact is introduced by intra-checkpoint iowaits and iclog ordering. What is that impact?

All I've really noticed is that long tail latencies on operations go down a bit. That seems to correlate with spending less time waiting for log space when the log is full, but it's a marginal improvement at best.

Otherwise I cannot measure any significant difference in performance or behaviour across any of the metrics I monitor during performance testing.

Quoted 3 lines

Note that it's not clear enough to me to suggest whether that impact might be significant or not. Hopefully it's neutral (?), but that seems like best case scenario so I do think it's a reasonable question.

Yes, It's a reasonable question, but I answered it entirely and in great detail the first time. Repeating the same question multiple times just with slightly different phrasing does not change the answer, nor explain to me what the undocumented concern might be...

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#18 lore raw

Re: [PATCH 2/8] xfs: separate CIL commit record IO

Quoted 43 lines · Dave Chinner
Quoted 32 lines · Brian Foster
Quoted 21 lines · Darrick J. Wong
Quoted 13 lines · Christoph Hellwig
Quoted 11 lines · Dave Chinner
Quoted 4 lines

Also, do you have any idea what was Christoph talking about wrt devices with no-op flushes the last time this patch was posted? This change seems straightforward to me (assuming the answers to my two question are 'yes') but I didn't grok what subtlety he was alluding to...?

He was wondering what devices benefited from this. It has no impact on highspeed devices that do not require flushes/FUA (e.g. high end intel optane SSDs) but those are not the devices this change is aimed at. There are no regressions on these high end devices, either, so they are largely irrelevant to the patch and what it targets...

Quoted 6 lines

I don't think it is that simple. Pretty much every device aimed at enterprise use does not enable a volatile write cache by default. That also includes hard drives, arrays and NAND based SSDs.

Especially for hard drives (or slower arrays) the actual I/O wait might matter. What is the argument against making this conditional?

Quoted 9 lines

I still don't understand what you're asking about here --

AFAICT the net effect of this patchset is that it reduces the number of preflushes and FUA log writes. To my knowledge, on a high end device with no volatile write cache, flushes are a no-op (because all writes are persisted somewhere immediately) and a FUA write should be the exact same thing as a non-FUA write. Because XFS will now issue fewer no-op persistence commands to the device, there should be no effect at all.

Quoted 2 lines

Except the cost of the new iowaits used to implement iclog ordering... which I think is what Christoph has been asking about..?

And I've already answered - it is largely just noise.

Quoted 4 lines

IOW, considering the storage configuration noted above where the impact of the flush/fua optimizations is neutral, the net effect of this change is whatever impact is introduced by intra-checkpoint iowaits and iclog ordering. What is that impact?

Quoted 9 lines

All I've really noticed is that long tail latencies on operations go down a bit. That seems to correlate with spending less time waiting for log space when the log is full, but it's a marginal improvement at best.

Otherwise I cannot measure any significant difference in performance or behaviour across any of the metrics I monitor during performance testing.

Ok.

Quoted 9 lines
Quoted 3 lines

Note that it's not clear enough to me to suggest whether that impact might be significant or not. Hopefully it's neutral (?), but that seems like best case scenario so I do think it's a reasonable question.

Yes, It's a reasonable question, but I answered it entirely and in great detail the first time. Repeating the same question multiple times just with slightly different phrasing does not change the answer, nor explain to me what the undocumented concern might be...

Darrick noted he wasn't clear on the question being asked. I rephrased it to hopefully add some clarity, not change the answer (?).

(FWIW, the response in the previous version of this series didn't clearly answer the question from my perspective either, so perhaps that is why you're seeing it repeated by multiple reviewers. Regardless, Christoph already replied with more detail so I'll just follow along in that sub-thread..)

Brian

Quoted 7 lines

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#19 lore raw

Re: [PATCH 2/8] xfs: separate CIL commit record IO

In reply to Dave Chinner
Quoted 79 lines · Dave Chinner

From: Dave Chinner <[email protected]>

To allow for iclog IO device cache flush behaviour to be optimised, we first need to separate out the commit record iclog IO from the rest of the checkpoint so we can wait for the checkpoint IO to complete before we issue the commit record.

This separation is only necessary if the commit record is being written into a different iclog to the start of the checkpoint as the upcoming cache flushing changes requires completion ordering against the other iclogs submitted by the checkpoint.

If the entire checkpoint and commit is in the one iclog, then they are both covered by the one set of cache flush primitives on the iclog and hence there is no need to separate them for ordering.

Otherwise, we need to wait for all the previous iclogs to complete so they are ordered correctly and made stable by the REQ_PREFLUSH that the commit record iclog IO issues. This guarantees that if a reader sees the commit record in the journal, they will also see the entire checkpoint that commit record closes off.

This also provides the guarantee that when the commit record IO completes, we can safely unpin all the log items in the checkpoint so they can be written back because the entire checkpoint is stable in the journal.

---
 fs/xfs/xfs_log.c      | 55 +++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_log_cil.c  |  7 ++++++
 fs/xfs/xfs_log_priv.h |  2 ++
 3 files changed, 64 insertions(+)
fs/xfs/xfs_log.c +37 -0
index fa284f26d10e..ff26fb46d70f 100644
@@ -808,6 +808,61 @@ xlog_wait_on_iclog(
 	return 0;
 }
 
+/*
+ * Wait on any iclogs that are still flushing in the range of start_lsn to the
+ * current iclog's lsn. The caller holds a reference to the iclog, but otherwise
+ * holds no log locks.
+ *
+ * We walk backwards through the iclogs to find the iclog with the highest lsn
+ * in the range that we need to wait for and then wait for it to complete.
+ * Completion ordering of iclog IOs ensures that all prior iclogs to the
+ * candidate iclog we need to sleep on have been complete by the time our
+ * candidate has completed it's IO.
+ *
+ * Therefore we only need to find the first iclog that isn't clean within the
+ * span of our flush range. If we come across a clean, newly activated iclog
+ * with a lsn of 0, it means IO has completed on this iclog and all previous
+ * iclogs will be have been completed prior to this one. Hence finding a newly
+ * activated iclog indicates that there are no iclogs in the range we need to
+ * wait on and we are done searching.
+ */
+int
+xlog_wait_on_iclog_lsn(
+	struct xlog_in_core	*iclog,
+	xfs_lsn_t		start_lsn)
+{
+	struct xlog		*log = iclog->ic_log;
+	struct xlog_in_core	*prev;
+	int			error = -EIO;
+
+	spin_lock(&log->l_icloglock);
+	if (XLOG_FORCED_SHUTDOWN(log))
+		goto out_unlock;
+
+	error = 0;
+	for (prev = iclog->ic_prev; prev != iclog; prev = prev->ic_prev) {
+
+		/* Done if the lsn is before our start lsn */
+		if (XFS_LSN_CMP(be64_to_cpu(prev->ic_header.h_lsn),
+				start_lsn) < 0)

fs/xfs/xfs_log.c xlog_wait_on_iclog( · line 857

Quoted 10 lines
fs/xfs/xfs_log.c +10 -0
+			break;
+
+		/* Don't need to wait on completed, clean iclogs */
+		if (prev->ic_state == XLOG_STATE_DIRTY ||
+		    prev->ic_state == XLOG_STATE_ACTIVE) {
+			continue;
+		}
+
+		/* wait for completion on this iclog */
+		xlog_wait(&prev->ic_force_wait, &log->l_icloglock);

You haven't addressed my feedback from the previous version. In particular the bit about whether it is safe to block on ->ic_force_wait from here considering some of our more quirky buffer locking behavior.

That aside, this iteration logic all seems a bit overengineered to me. We have the commit record iclog of the current checkpoint and thus the immediately previous iclog in the ring. We know that previous record isn't earlier than start_lsn because the caller confirmed that start_lsn != commit_lsn. We also know that iclog can't become dirty -> active until it and all previous iclog writes have completed because the callback ordering implemented by xlog_state_do_callback() won't clean the iclog until that point. Given that, can't this whole thing be replaced with a check of iclog->prev to either see if it's been cleaned or to otherwise xlog_wait() for that condition and return?

Brian

Quoted 45 lines
fs/xfs/xfs_log.c +8 -0
+		return 0;
+	}
+
+out_unlock:
+	spin_unlock(&log->l_icloglock);
+	return error;
+}
+
 /*
  * Write out an unmount record using the ticket provided. We have to account for
  * the data space used in the unmount ticket as this write is not done from a
fs/xfs/xfs_log_cil.c +7 -0
index b0ef071b3cb5..c5cc1b7ad25e 100644
@@ -870,6 +870,13 @@ xlog_cil_push_work(
 	wake_up_all(&cil->xc_commit_wait);
 	spin_unlock(&cil->xc_push_lock);
 
+	/*
+	 * If the checkpoint spans multiple iclogs, wait for all previous
+	 * iclogs to complete before we submit the commit_iclog.
+	 */
+	if (ctx->start_lsn != commit_lsn)
+		xlog_wait_on_iclog_lsn(commit_iclog, ctx->start_lsn);
+
 	/* release the hounds! */
 	xfs_log_release_iclog(commit_iclog);
 	return;
fs/xfs/xfs_log_priv.h +2 -0
index 037950cf1061..a7ac85aaff4e 100644
@@ -584,6 +584,8 @@ xlog_wait(
 	remove_wait_queue(wq, &wait);
 }
 
+int xlog_wait_on_iclog_lsn(struct xlog_in_core *iclog, xfs_lsn_t start_lsn);
+
 /*
  * The LSN is valid so long as it is behind the current LSN. If it isn't, this
  * means that the next log record that includes this metadata could have a
Signature · 1 line
2.28.0
#20 lore raw

Re: [PATCH 2/8] xfs: separate CIL commit record IO

Quoted 86 lines · Brian Foster
Quoted 85 lines · Dave Chinner

From: Dave Chinner <[email protected]>

To allow for iclog IO device cache flush behaviour to be optimised, we first need to separate out the commit record iclog IO from the rest of the checkpoint so we can wait for the checkpoint IO to complete before we issue the commit record.

This separation is only necessary if the commit record is being written into a different iclog to the start of the checkpoint as the upcoming cache flushing changes requires completion ordering against the other iclogs submitted by the checkpoint.

If the entire checkpoint and commit is in the one iclog, then they are both covered by the one set of cache flush primitives on the iclog and hence there is no need to separate them for ordering.

Otherwise, we need to wait for all the previous iclogs to complete so they are ordered correctly and made stable by the REQ_PREFLUSH that the commit record iclog IO issues. This guarantees that if a reader sees the commit record in the journal, they will also see the entire checkpoint that commit record closes off.

This also provides the guarantee that when the commit record IO completes, we can safely unpin all the log items in the checkpoint so they can be written back because the entire checkpoint is stable in the journal.

---
 fs/xfs/xfs_log.c      | 55 +++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_log_cil.c  |  7 ++++++
 fs/xfs/xfs_log_priv.h |  2 ++
 3 files changed, 64 insertions(+)
fs/xfs/xfs_log.c +43 -0
index fa284f26d10e..ff26fb46d70f 100644
@@ -808,6 +808,61 @@ xlog_wait_on_iclog(
 	return 0;
 }
 
+/*
+ * Wait on any iclogs that are still flushing in the range of start_lsn to the
+ * current iclog's lsn. The caller holds a reference to the iclog, but otherwise
+ * holds no log locks.
+ *
+ * We walk backwards through the iclogs to find the iclog with the highest lsn
+ * in the range that we need to wait for and then wait for it to complete.
+ * Completion ordering of iclog IOs ensures that all prior iclogs to the
+ * candidate iclog we need to sleep on have been complete by the time our
+ * candidate has completed it's IO.
+ *
+ * Therefore we only need to find the first iclog that isn't clean within the
+ * span of our flush range. If we come across a clean, newly activated iclog
+ * with a lsn of 0, it means IO has completed on this iclog and all previous
+ * iclogs will be have been completed prior to this one. Hence finding a newly
+ * activated iclog indicates that there are no iclogs in the range we need to
+ * wait on and we are done searching.
+ */
+int
+xlog_wait_on_iclog_lsn(
+	struct xlog_in_core	*iclog,
+	xfs_lsn_t		start_lsn)
+{
+	struct xlog		*log = iclog->ic_log;
+	struct xlog_in_core	*prev;
+	int			error = -EIO;
+
+	spin_lock(&log->l_icloglock);
+	if (XLOG_FORCED_SHUTDOWN(log))
+		goto out_unlock;
+
+	error = 0;
+	for (prev = iclog->ic_prev; prev != iclog; prev = prev->ic_prev) {
+
+		/* Done if the lsn is before our start lsn */
+		if (XFS_LSN_CMP(be64_to_cpu(prev->ic_header.h_lsn),
+				start_lsn) < 0)
+			break;
+
+		/* Don't need to wait on completed, clean iclogs */
+		if (prev->ic_state == XLOG_STATE_DIRTY ||
+		    prev->ic_state == XLOG_STATE_ACTIVE) {
+			continue;
Quoted 8 lines

fs/xfs/xfs_log.c xlog_wait_on_iclog( · line 857

Quoted 4 lines
fs/xfs/xfs_log.c +4 -0
+		}
+
+		/* wait for completion on this iclog */
+		xlog_wait(&prev->ic_force_wait, &log->l_icloglock);

You haven't addressed my feedback from the previous version. In particular the bit about whether it is safe to block on ->ic_force_wait from here considering some of our more quirky buffer locking behavior.

Sorry, first I've heard about this. I don't have any such email in my inbox.

I don't know what waiting on an iclog in the middle of a checkpoint has to do with buffer locking behaviour, because iclogs don't use buffers and we block waiting on iclog IO completion all the time in xlog_state_get_iclog_space(). If it's not safe to block on iclog IO completion here, then it's not safe to block on an iclog in xlog_state_get_iclog_space(). That's obviously not true, so I'm really not sure what the concern here is...

Quoted 10 lines

That aside, this iteration logic all seems a bit overengineered to me. We have the commit record iclog of the current checkpoint and thus the immediately previous iclog in the ring. We know that previous record isn't earlier than start_lsn because the caller confirmed that start_lsn != commit_lsn. We also know that iclog can't become dirty -> active until it and all previous iclog writes have completed because the callback ordering implemented by xlog_state_do_callback() won't clean the iclog until that point. Given that, can't this whole thing be replaced with a check of iclog->prev to either see if it's been cleaned or to otherwise xlog_wait() for that condition and return?

Maybe. I was more concerned about ensuring that it did the right thing so I checked all the things that came to mind. There was more than enough compexity in other parts of this patchset to fill my brain that minimal implementation were not a concern. I'll go take another look at it.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#21 lore raw

Re: [PATCH 2/8] xfs: separate CIL commit record IO

Quoted 96 lines · Dave Chinner
Quoted 86 lines · Brian Foster
Quoted 85 lines · Dave Chinner

From: Dave Chinner <[email protected]>

To allow for iclog IO device cache flush behaviour to be optimised, we first need to separate out the commit record iclog IO from the rest of the checkpoint so we can wait for the checkpoint IO to complete before we issue the commit record.

This separation is only necessary if the commit record is being written into a different iclog to the start of the checkpoint as the upcoming cache flushing changes requires completion ordering against the other iclogs submitted by the checkpoint.

If the entire checkpoint and commit is in the one iclog, then they are both covered by the one set of cache flush primitives on the iclog and hence there is no need to separate them for ordering.

Otherwise, we need to wait for all the previous iclogs to complete so they are ordered correctly and made stable by the REQ_PREFLUSH that the commit record iclog IO issues. This guarantees that if a reader sees the commit record in the journal, they will also see the entire checkpoint that commit record closes off.

This also provides the guarantee that when the commit record IO completes, we can safely unpin all the log items in the checkpoint so they can be written back because the entire checkpoint is stable in the journal.

---
 fs/xfs/xfs_log.c      | 55 +++++++++++++++++++++++++++++++++++++++++++
 fs/xfs/xfs_log_cil.c  |  7 ++++++
 fs/xfs/xfs_log_priv.h |  2 ++
 3 files changed, 64 insertions(+)
fs/xfs/xfs_log.c +43 -0
index fa284f26d10e..ff26fb46d70f 100644
@@ -808,6 +808,61 @@ xlog_wait_on_iclog(
 	return 0;
 }
 
+/*
+ * Wait on any iclogs that are still flushing in the range of start_lsn to the
+ * current iclog's lsn. The caller holds a reference to the iclog, but otherwise
+ * holds no log locks.
+ *
+ * We walk backwards through the iclogs to find the iclog with the highest lsn
+ * in the range that we need to wait for and then wait for it to complete.
+ * Completion ordering of iclog IOs ensures that all prior iclogs to the
+ * candidate iclog we need to sleep on have been complete by the time our
+ * candidate has completed it's IO.
+ *
+ * Therefore we only need to find the first iclog that isn't clean within the
+ * span of our flush range. If we come across a clean, newly activated iclog
+ * with a lsn of 0, it means IO has completed on this iclog and all previous
+ * iclogs will be have been completed prior to this one. Hence finding a newly
+ * activated iclog indicates that there are no iclogs in the range we need to
+ * wait on and we are done searching.
+ */
+int
+xlog_wait_on_iclog_lsn(
+	struct xlog_in_core	*iclog,
+	xfs_lsn_t		start_lsn)
+{
+	struct xlog		*log = iclog->ic_log;
+	struct xlog_in_core	*prev;
+	int			error = -EIO;
+
+	spin_lock(&log->l_icloglock);
+	if (XLOG_FORCED_SHUTDOWN(log))
+		goto out_unlock;
+
+	error = 0;
+	for (prev = iclog->ic_prev; prev != iclog; prev = prev->ic_prev) {
+
+		/* Done if the lsn is before our start lsn */
+		if (XFS_LSN_CMP(be64_to_cpu(prev->ic_header.h_lsn),
+				start_lsn) < 0)
+			break;
+
+		/* Don't need to wait on completed, clean iclogs */
+		if (prev->ic_state == XLOG_STATE_DIRTY ||
+		    prev->ic_state == XLOG_STATE_ACTIVE) {
+			continue;
Quoted 8 lines

fs/xfs/xfs_log.c xlog_wait_on_iclog( · line 857

Quoted 4 lines
fs/xfs/xfs_log.c +4 -0
+		}
+
+		/* wait for completion on this iclog */
+		xlog_wait(&prev->ic_force_wait, &log->l_icloglock);

You haven't addressed my feedback from the previous version. In particular the bit about whether it is safe to block on ->ic_force_wait from here considering some of our more quirky buffer locking behavior.

Quoted 3 lines

Sorry, first I've heard about this. I don't have any such email in my inbox.

For reference, the last bit of this mail:

https://lore.kernel.org/linux-xfs/20210201160737.GA3252048@bfoster/

Quoted 8 lines

I don't know what waiting on an iclog in the middle of a checkpoint has to do with buffer locking behaviour, because iclogs don't use buffers and we block waiting on iclog IO completion all the time in xlog_state_get_iclog_space(). If it's not safe to block on iclog IO completion here, then it's not safe to block on an iclog in xlog_state_get_iclog_space(). That's obviously not true, so I'm really not sure what the concern here is...

I think the broader question is not so much whether it's safe to block here or not, but whether our current use of async log forces might have a deadlock vector (which may or may not also include the _get_iclog_space() scenario, I'd need to stare at that one a bit). I referred to buffer locking because the buffer ->iop_unpin() handler can attempt to acquire a buffer lock.

Looking again, that is the only place I see that blocks in iclog completion callbacks and it's actually an abort scenario, which means shutdown. I am slightly concerned that introducing more regular blocking in the CIL push might lead to more frequent async log forces that block on callback iclogs and thus exacerbate that issue (i.e. somebody might be able to now reproduce yet another shutdown deadlock scenario to track down that might not have been reproducible before, for whatever reason), but that's probably not a serious enough problem to block this patch and the advantages of the series overall.

Brian

Quoted 24 lines
Quoted 10 lines

That aside, this iteration logic all seems a bit overengineered to me. We have the commit record iclog of the current checkpoint and thus the immediately previous iclog in the ring. We know that previous record isn't earlier than start_lsn because the caller confirmed that start_lsn != commit_lsn. We also know that iclog can't become dirty -> active until it and all previous iclog writes have completed because the callback ordering implemented by xlog_state_do_callback() won't clean the iclog until that point. Given that, can't this whole thing be replaced with a check of iclog->prev to either see if it's been cleaned or to otherwise xlog_wait() for that condition and return?

Maybe. I was more concerned about ensuring that it did the right thing so I checked all the things that came to mind. There was more than enough compexity in other parts of this patchset to fill my brain that minimal implementation were not a concern. I'll go take another look at it.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#22 lore raw

Re: [PATCH 2/8] xfs: separate CIL commit record IO

Quoted 24 lines · Brian Foster
Quoted 9 lines · Dave Chinner
Quoted 4 lines · Brian Foster

On Tue, Feb 23, 2021 at 02:34:36PM +1100, Dave Chinner wrote:
You haven't addressed my feedback from the previous version. In particular the bit about whether it is safe to block on ->ic_force_wait from here considering some of our more quirky buffer locking behavior.

Sorry, first I've heard about this. I don't have any such email in my inbox.

For reference, the last bit of this mail:

https://lore.kernel.org/linux-xfs/20210201160737.GA3252048@bfoster/

Quoted 8 lines

I don't know what waiting on an iclog in the middle of a checkpoint has to do with buffer locking behaviour, because iclogs don't use buffers and we block waiting on iclog IO completion all the time in xlog_state_get_iclog_space(). If it's not safe to block on iclog IO completion here, then it's not safe to block on an iclog in xlog_state_get_iclog_space(). That's obviously not true, so I'm really not sure what the concern here is...

Quoted 6 lines

I think the broader question is not so much whether it's safe to block here or not, but whether our current use of async log forces might have a deadlock vector (which may or may not also include the _get_iclog_space() scenario, I'd need to stare at that one a bit). I referred to buffer locking because the buffer ->iop_unpin() handler can attempt to acquire a buffer lock.

There are none that I know of, and I'm not changing any of the log write blocking rules. Hence if there is a problem, it's a zero-day that we have never triggered nor have any awareness about at all. Hence for the purposes of development and review, we can assume such unknown design problems don't actually exist because there's absolutely zero evidence to indicate there is problem here...

Quoted 3 lines

Looking again, that is the only place I see that blocks in iclog completion callbacks and it's actually an abort scenario, which means shutdown.

Yup. The AIL simply needs to abort writeback of such locked, pinned buffers and then everything works just fine.

Quoted 8 lines

I am slightly concerned that introducing more regular blocking in the CIL push might lead to more frequent async log forces that block on callback iclogs and thus exacerbate that issue (i.e. somebody might be able to now reproduce yet another shutdown deadlock scenario to track down that might not have been reproducible before, for whatever reason), but that's probably not a serious enough problem to block this patch and the advantages of the series overall.

And that's why I updated the log force stats accounting to capture the async log forces and how we account log forces that block. That gives me direct visibility into the blocking behaviour while I'm running tests. And even with this new visibility, I can't see any change in the metrics that are above the noise floor...

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#23 lore raw

Re: [PATCH 2/8] xfs: separate CIL commit record IO

In reply to Dave Chinner
Quoted 13 lines · Dave Chinner
Quoted 11 lines · Brian Foster

On Tue, Feb 23, 2021 at 02:34:36PM +1100, Dave Chinner wrote:
That aside, this iteration logic all seems a bit overengineered to me. We have the commit record iclog of the current checkpoint and thus the immediately previous iclog in the ring. We know that previous record isn't earlier than start_lsn because the caller confirmed that start_lsn != commit_lsn. We also know that iclog can't become dirty -> active until it and all previous iclog writes have completed because the callback ordering implemented by xlog_state_do_callback() won't clean the iclog until that point. Given that, can't this whole thing be replaced with a check of iclog->prev to either see if it's been cleaned or to otherwise xlog_wait() for that condition and return?

Quoted 5 lines

Maybe. I was more concerned about ensuring that it did the right thing so I checked all the things that came to mind. There was more than enough compexity in other parts of this patchset to fill my brain that minimal implementation were not a concern. I'll go take another look at it.

Ok, so we can just use xlog_wait_on_iclog() here. I didn't look too closely at the implementation of that function, just took the comment above it at face value that it only waited for an iclog to hit the disk.

We actually have two different iclog IO completion wait points - one to wait for an iclog to hit the disk, and one to wait for hit the disk and run completion callbacks. i.e. one is not ordered against other iclogs and the other is strictly ordered.

The ordered version runs completion callbacks before waking waiters thereby guaranteeing all previous iclog have been completed before completing the current iclog and waking waiters.

The CIL code needs the latter, so yes, this can be simplified down to a single xlog_wait_on_iclog(commit_iclog->ic_prev); call from the CIL.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#24 lore raw

PATCH 3/8 xfs: move and rename xfs_blkdev_issue_flush

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

Move it to xfs_bio_io.c as we are about to add new async cache flush functionality that uses bios directly, so all this stuff should be in the same place. Rename the function to xfs_flush_bdev() to match the xfs_rw_bdev() function that already exists in this file.

---
 fs/xfs/xfs_bio_io.c | 8 ++++++++
 fs/xfs/xfs_buf.c    | 2 +-
 fs/xfs/xfs_file.c   | 6 +++---
 fs/xfs/xfs_linux.h  | 1 +
 fs/xfs/xfs_log.c    | 2 +-
 fs/xfs/xfs_super.c  | 7 -------
 fs/xfs/xfs_super.h  | 1 -
 7 files changed, 14 insertions(+), 13 deletions(-)
fs/xfs/xfs_bio_io.c +8 -0
index e2148f2d5d6b..5abf653a45d4 100644
@@ -59,3 +59,11 @@ xfs_rw_bdev(
 		invalidate_kernel_vmap_range(data, count);
 	return error;
 }
+
+void
+xfs_flush_bdev(
+	struct block_device	*bdev)
+{
+	blkdev_issue_flush(bdev, GFP_NOFS);
+}
+
fs/xfs/xfs_buf.c +1 -1
index f6e5235df7c9..b1d6c530c693 100644
@@ -1958,7 +1958,7 @@ xfs_free_buftarg(
 	percpu_counter_destroy(&btp->bt_io_count);
 	list_lru_destroy(&btp->bt_lru);
 
-	xfs_blkdev_issue_flush(btp);
+	xfs_flush_bdev(btp->bt_bdev);
 
 	kmem_free(btp);
 }
fs/xfs/xfs_file.c +3 -3
index 38528e59030e..dd33ef2d0e20 100644
@@ -196,9 +196,9 @@ xfs_file_fsync(
 	 * inode size in case of an extending write.
 	 */
 	if (XFS_IS_REALTIME_INODE(ip))
-		xfs_blkdev_issue_flush(mp->m_rtdev_targp);
+		xfs_flush_bdev(mp->m_rtdev_targp->bt_bdev);
 	else if (mp->m_logdev_targp != mp->m_ddev_targp)
-		xfs_blkdev_issue_flush(mp->m_ddev_targp);
+		xfs_flush_bdev(mp->m_ddev_targp->bt_bdev);
 
 	/*
 	 * Any inode that has dirty modifications in the log is pinned.  The
@@ -218,7 +218,7 @@ xfs_file_fsync(
 	 */
 	if (!log_flushed && !XFS_IS_REALTIME_INODE(ip) &&
 	    mp->m_logdev_targp == mp->m_ddev_targp)
-		xfs_blkdev_issue_flush(mp->m_ddev_targp);
+		xfs_flush_bdev(mp->m_ddev_targp->bt_bdev);
 
 	return error;
 }
fs/xfs/xfs_linux.h +1 -0
index af6be9b9ccdf..e94a2aeefee8 100644
@@ -196,6 +196,7 @@ static inline uint64_t howmany_64(uint64_t x, uint32_t y)
 
 int xfs_rw_bdev(struct block_device *bdev, sector_t sector, unsigned int count,
 		char *data, unsigned int op);
+void xfs_flush_bdev(struct block_device *bdev);
 
 #define ASSERT_ALWAYS(expr)	\
 	(likely(expr) ? (void)0 : assfail(NULL, #expr, __FILE__, __LINE__))
fs/xfs/xfs_log.c +1 -1
index ff26fb46d70f..493454c98c6f 100644
@@ -2015,7 +2015,7 @@ xlog_sync(
 	 * layer state machine for preflushes.
 	 */
 	if (log->l_targ != log->l_mp->m_ddev_targp || split) {
-		xfs_blkdev_issue_flush(log->l_mp->m_ddev_targp);
+		xfs_flush_bdev(log->l_mp->m_ddev_targp->bt_bdev);
 		need_flush = false;
 	}
 
fs/xfs/xfs_super.c +0 -7
index 21b1d034aca3..85dd9593b40b 100644
@@ -339,13 +339,6 @@ xfs_blkdev_put(
 		blkdev_put(bdev, FMODE_READ|FMODE_WRITE|FMODE_EXCL);
 }
 
-void
-xfs_blkdev_issue_flush(
-	xfs_buftarg_t		*buftarg)
-{
-	blkdev_issue_flush(buftarg->bt_bdev, GFP_NOFS);
-}
-
 STATIC void
 xfs_close_devices(
 	struct xfs_mount	*mp)
fs/xfs/xfs_super.h +0 -1
index 1ca484b8357f..79cb2dece811 100644
@@ -88,7 +88,6 @@ struct block_device;
 
 extern void xfs_quiesce_attr(struct xfs_mount *mp);
 extern void xfs_flush_inodes(struct xfs_mount *mp);
-extern void xfs_blkdev_issue_flush(struct xfs_buftarg *);
 extern xfs_agnumber_t xfs_set_inode_alloc(struct xfs_mount *,
 					   xfs_agnumber_t agcount);
 
Signature · 1 line
2.28.0
Reviewed-by on v1 3/8
#25 lore raw

Re: [PATCH 3/8] xfs: move and rename xfs_blkdev_issue_flush

Quoted 7 lines · Dave Chinner

From: Dave Chinner <[email protected]>

Move it to xfs_bio_io.c as we are about to add new async cache flush functionality that uses bios directly, so all this stuff should be in the same place. Rename the function to xfs_flush_bdev() to match the xfs_rw_bdev() function that already exists in this file.

Looks good to me.

Quoted 119 lines
---
 fs/xfs/xfs_bio_io.c | 8 ++++++++
 fs/xfs/xfs_buf.c    | 2 +-
 fs/xfs/xfs_file.c   | 6 +++---
 fs/xfs/xfs_linux.h  | 1 +
 fs/xfs/xfs_log.c    | 2 +-
 fs/xfs/xfs_super.c  | 7 -------
 fs/xfs/xfs_super.h  | 1 -
 7 files changed, 14 insertions(+), 13 deletions(-)
fs/xfs/xfs_bio_io.c +8 -0
index e2148f2d5d6b..5abf653a45d4 100644
@@ -59,3 +59,11 @@ xfs_rw_bdev(
 		invalidate_kernel_vmap_range(data, count);
 	return error;
 }
+
+void
+xfs_flush_bdev(
+	struct block_device	*bdev)
+{
+	blkdev_issue_flush(bdev, GFP_NOFS);
+}
+
fs/xfs/xfs_buf.c +1 -1
index f6e5235df7c9..b1d6c530c693 100644
@@ -1958,7 +1958,7 @@ xfs_free_buftarg(
 	percpu_counter_destroy(&btp->bt_io_count);
 	list_lru_destroy(&btp->bt_lru);
 
-	xfs_blkdev_issue_flush(btp);
+	xfs_flush_bdev(btp->bt_bdev);
 
 	kmem_free(btp);
 }
fs/xfs/xfs_file.c +3 -3
index 38528e59030e..dd33ef2d0e20 100644
@@ -196,9 +196,9 @@ xfs_file_fsync(
 	 * inode size in case of an extending write.
 	 */
 	if (XFS_IS_REALTIME_INODE(ip))
-		xfs_blkdev_issue_flush(mp->m_rtdev_targp);
+		xfs_flush_bdev(mp->m_rtdev_targp->bt_bdev);
 	else if (mp->m_logdev_targp != mp->m_ddev_targp)
-		xfs_blkdev_issue_flush(mp->m_ddev_targp);
+		xfs_flush_bdev(mp->m_ddev_targp->bt_bdev);
 
 	/*
 	 * Any inode that has dirty modifications in the log is pinned.  The
@@ -218,7 +218,7 @@ xfs_file_fsync(
 	 */
 	if (!log_flushed && !XFS_IS_REALTIME_INODE(ip) &&
 	    mp->m_logdev_targp == mp->m_ddev_targp)
-		xfs_blkdev_issue_flush(mp->m_ddev_targp);
+		xfs_flush_bdev(mp->m_ddev_targp->bt_bdev);
 
 	return error;
 }
fs/xfs/xfs_linux.h +1 -0
index af6be9b9ccdf..e94a2aeefee8 100644
@@ -196,6 +196,7 @@ static inline uint64_t howmany_64(uint64_t x, uint32_t y)
 
 int xfs_rw_bdev(struct block_device *bdev, sector_t sector, unsigned int count,
 		char *data, unsigned int op);
+void xfs_flush_bdev(struct block_device *bdev);
 
 #define ASSERT_ALWAYS(expr)	\
 	(likely(expr) ? (void)0 : assfail(NULL, #expr, __FILE__, __LINE__))
fs/xfs/xfs_log.c +1 -1
index ff26fb46d70f..493454c98c6f 100644
@@ -2015,7 +2015,7 @@ xlog_sync(
 	 * layer state machine for preflushes.
 	 */
 	if (log->l_targ != log->l_mp->m_ddev_targp || split) {
-		xfs_blkdev_issue_flush(log->l_mp->m_ddev_targp);
+		xfs_flush_bdev(log->l_mp->m_ddev_targp->bt_bdev);
 		need_flush = false;
 	}
 
fs/xfs/xfs_super.c +0 -7
index 21b1d034aca3..85dd9593b40b 100644
@@ -339,13 +339,6 @@ xfs_blkdev_put(
 		blkdev_put(bdev, FMODE_READ|FMODE_WRITE|FMODE_EXCL);
 }
 
-void
-xfs_blkdev_issue_flush(
-	xfs_buftarg_t		*buftarg)
-{
-	blkdev_issue_flush(buftarg->bt_bdev, GFP_NOFS);
-}
-
 STATIC void
 xfs_close_devices(
 	struct xfs_mount	*mp)
fs/xfs/xfs_super.h +0 -1
index 1ca484b8357f..79cb2dece811 100644
@@ -88,7 +88,6 @@ struct block_device;
 
 extern void xfs_quiesce_attr(struct xfs_mount *mp);
 extern void xfs_flush_inodes(struct xfs_mount *mp);
-extern void xfs_blkdev_issue_flush(struct xfs_buftarg *);
 extern xfs_agnumber_t xfs_set_inode_alloc(struct xfs_mount *,
 					   xfs_agnumber_t agcount);
Signature · 1 line
chandan
#26 lore raw

Re: [PATCH 3/8] xfs: move and rename xfs_blkdev_issue_flush

In reply to Dave Chinner
Quoted 8 lines · Dave Chinner

From: Dave Chinner <[email protected]>

Move it to xfs_bio_io.c as we are about to add new async cache flush functionality that uses bios directly, so all this stuff should be in the same place. Rename the function to xfs_flush_bdev() to match the xfs_rw_bdev() function that already exists in this file.

I don't get why it's necessary to consolidate the synchronous flush function with the (future) async flush, since all the sync flush callers go through a buftarg, including the log. All this seems to do is shift pointer dereferencing into callers.

Why not make the async flush function take a buftarg?

--D

Quoted 122 lines
---
 fs/xfs/xfs_bio_io.c | 8 ++++++++
 fs/xfs/xfs_buf.c    | 2 +-
 fs/xfs/xfs_file.c   | 6 +++---
 fs/xfs/xfs_linux.h  | 1 +
 fs/xfs/xfs_log.c    | 2 +-
 fs/xfs/xfs_super.c  | 7 -------
 fs/xfs/xfs_super.h  | 1 -
 7 files changed, 14 insertions(+), 13 deletions(-)
fs/xfs/xfs_bio_io.c +8 -0
index e2148f2d5d6b..5abf653a45d4 100644
@@ -59,3 +59,11 @@ xfs_rw_bdev(
 		invalidate_kernel_vmap_range(data, count);
 	return error;
 }
+
+void
+xfs_flush_bdev(
+	struct block_device	*bdev)
+{
+	blkdev_issue_flush(bdev, GFP_NOFS);
+}
+
fs/xfs/xfs_buf.c +1 -1
index f6e5235df7c9..b1d6c530c693 100644
@@ -1958,7 +1958,7 @@ xfs_free_buftarg(
 	percpu_counter_destroy(&btp->bt_io_count);
 	list_lru_destroy(&btp->bt_lru);
 
-	xfs_blkdev_issue_flush(btp);
+	xfs_flush_bdev(btp->bt_bdev);
 
 	kmem_free(btp);
 }
fs/xfs/xfs_file.c +3 -3
index 38528e59030e..dd33ef2d0e20 100644
@@ -196,9 +196,9 @@ xfs_file_fsync(
 	 * inode size in case of an extending write.
 	 */
 	if (XFS_IS_REALTIME_INODE(ip))
-		xfs_blkdev_issue_flush(mp->m_rtdev_targp);
+		xfs_flush_bdev(mp->m_rtdev_targp->bt_bdev);
 	else if (mp->m_logdev_targp != mp->m_ddev_targp)
-		xfs_blkdev_issue_flush(mp->m_ddev_targp);
+		xfs_flush_bdev(mp->m_ddev_targp->bt_bdev);
 
 	/*
 	 * Any inode that has dirty modifications in the log is pinned.  The
@@ -218,7 +218,7 @@ xfs_file_fsync(
 	 */
 	if (!log_flushed && !XFS_IS_REALTIME_INODE(ip) &&
 	    mp->m_logdev_targp == mp->m_ddev_targp)
-		xfs_blkdev_issue_flush(mp->m_ddev_targp);
+		xfs_flush_bdev(mp->m_ddev_targp->bt_bdev);
 
 	return error;
 }
fs/xfs/xfs_linux.h +1 -0
index af6be9b9ccdf..e94a2aeefee8 100644
@@ -196,6 +196,7 @@ static inline uint64_t howmany_64(uint64_t x, uint32_t y)
 
 int xfs_rw_bdev(struct block_device *bdev, sector_t sector, unsigned int count,
 		char *data, unsigned int op);
+void xfs_flush_bdev(struct block_device *bdev);
 
 #define ASSERT_ALWAYS(expr)	\
 	(likely(expr) ? (void)0 : assfail(NULL, #expr, __FILE__, __LINE__))
fs/xfs/xfs_log.c +1 -1
index ff26fb46d70f..493454c98c6f 100644
@@ -2015,7 +2015,7 @@ xlog_sync(
 	 * layer state machine for preflushes.
 	 */
 	if (log->l_targ != log->l_mp->m_ddev_targp || split) {
-		xfs_blkdev_issue_flush(log->l_mp->m_ddev_targp);
+		xfs_flush_bdev(log->l_mp->m_ddev_targp->bt_bdev);
 		need_flush = false;
 	}
 
fs/xfs/xfs_super.c +0 -7
index 21b1d034aca3..85dd9593b40b 100644
@@ -339,13 +339,6 @@ xfs_blkdev_put(
 		blkdev_put(bdev, FMODE_READ|FMODE_WRITE|FMODE_EXCL);
 }
 
-void
-xfs_blkdev_issue_flush(
-	xfs_buftarg_t		*buftarg)
-{
-	blkdev_issue_flush(buftarg->bt_bdev, GFP_NOFS);
-}
-
 STATIC void
 xfs_close_devices(
 	struct xfs_mount	*mp)
fs/xfs/xfs_super.h +0 -1
index 1ca484b8357f..79cb2dece811 100644
@@ -88,7 +88,6 @@ struct block_device;
 
 extern void xfs_quiesce_attr(struct xfs_mount *mp);
 extern void xfs_flush_inodes(struct xfs_mount *mp);
-extern void xfs_blkdev_issue_flush(struct xfs_buftarg *);
 extern xfs_agnumber_t xfs_set_inode_alloc(struct xfs_mount *,
 					   xfs_agnumber_t agcount);
 
Signature · 1 line
2.28.0
#27 lore raw

Re: [PATCH 3/8] xfs: move and rename xfs_blkdev_issue_flush

Quoted 8 lines · Darrick J. Wong
Quoted 7 lines · Dave Chinner

From: Dave Chinner <[email protected]>

Move it to xfs_bio_io.c as we are about to add new async cache flush functionality that uses bios directly, so all this stuff should be in the same place. Rename the function to xfs_flush_bdev() to match the xfs_rw_bdev() function that already exists in this file.

Quoted 8 lines
Quoted 1 line

I don't get why it's necessary to consolidate the synchronous flush function with the (future) async flush, since all the sync flush callers go through a buftarg, including the log. All this seems to do is shift pointer dereferencing into callers.

Why not make the async flush function take a buftarg?

Because we pretty much got rid of the buffer/buftarg abstraction from the log IO code completely by going direct to bios. The async flush goes direct to bios like the rest of the log code does. And given that xfs_blkdev_issue_flush() is just a one line wrapper around the blkdev interface, it just seems totally weird to wrap it differently to other interfaces that go direct to the bios and block devices.

Part of the problem here is that we've completely screwed up the separation/abstraction of the log from the xfs_mount and buftargs. The log just doesn't use buftargs anymore except as a holder of the log bdev, and the only other interaction it has with them is when the metadata device cache needs to be invalidated after log recovery and during unmount.

It just doesn't make sense to me to have bdev flush interfaces that the log uses hidden behind an abstraction that the rest of the log subsystem doesn't use...

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#28 lore raw

Re: [PATCH 3/8] xfs: move and rename xfs_blkdev_issue_flush

In reply to Dave Chinner
Quoted 6 lines · Dave Chinner

From: Dave Chinner <[email protected]>

Move it to xfs_bio_io.c as we are about to add new async cache flush functionality that uses bios directly, so all this stuff should be in the same place. Rename the function to xfs_flush_bdev() to match the xfs_rw_bdev() function that already exists in this file.

I'd rather kill it off. None that as of the latest Linus tree, blkdev_issue_flush has also lost the gfp_mask argument.

#29 lore raw

PATCH 4/8 xfs: async blkdev cache flush

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

The new checkpoint caceh flush mechanism requires us to issue an unconditional cache flush before we start a new checkpoint. We don't want to block for this if we can help it, and we have a fair chunk of CPU work to do between starting the checkpoint and issuing the first journal IO.

Hence it makes sense to amortise the latency cost of the cache flush by issuing it asynchronously and then waiting for it only when we need to issue the first IO in the transaction.

TO do this, we need async cache flush primitives to submit the cache flush bio and to wait on it. THe block layer has no such primitives for filesystems, so roll our own for the moment.

---
 fs/xfs/xfs_bio_io.c | 30 ++++++++++++++++++++++++++++++
 fs/xfs/xfs_linux.h  |  1 +
 2 files changed, 31 insertions(+)
fs/xfs/xfs_bio_io.c +30 -0
index 5abf653a45d4..d55420bc72b5 100644
@@ -67,3 +67,33 @@ xfs_flush_bdev(
 	blkdev_issue_flush(bdev, GFP_NOFS);
 }
 
+void
+xfs_flush_bdev_async_endio(
+	struct bio	*bio)
+{
+	if (bio->bi_private)
+		complete(bio->bi_private);
+	bio_put(bio);
+}
+
+/*
+ * Submit a request for an async cache flush to run. If the caller needs to wait
+ * for the flush completion at a later point in time, they must supply a
+ * valid completion. This will be signalled when the flush completes.
+ * The caller never sees the bio that is issued here.
+ */
+void
+xfs_flush_bdev_async(
+	struct block_device	*bdev,
+	struct completion	*done)
+{
+	struct bio *bio;
+
+	bio = bio_alloc(GFP_NOFS, 0);
+	bio_set_dev(bio, bdev);
+	bio->bi_opf = REQ_OP_WRITE | REQ_PREFLUSH | REQ_SYNC;
+	bio->bi_private = done;
+        bio->bi_end_io = xfs_flush_bdev_async_endio;
+
+	submit_bio(bio);
+}
fs/xfs/xfs_linux.h +1 -0
index e94a2aeefee8..293ff2355e80 100644
@@ -197,6 +197,7 @@ static inline uint64_t howmany_64(uint64_t x, uint32_t y)
 int xfs_rw_bdev(struct block_device *bdev, sector_t sector, unsigned int count,
 		char *data, unsigned int op);
 void xfs_flush_bdev(struct block_device *bdev);
+void xfs_flush_bdev_async(struct block_device *bdev, struct completion *done);
 
 #define ASSERT_ALWAYS(expr)	\
 	(likely(expr) ? (void)0 : assfail(NULL, #expr, __FILE__, __LINE__))
Signature · 1 line
2.28.0
Reviewed-by on v1 4/8
#30 lore raw

Re: [PATCH 4/8] xfs: async blkdev cache flush

Quoted 48 lines · Dave Chinner

From: Dave Chinner <[email protected]>

The new checkpoint caceh flush mechanism requires us to issue an unconditional cache flush before we start a new checkpoint. We don't want to block for this if we can help it, and we have a fair chunk of CPU work to do between starting the checkpoint and issuing the first journal IO.

Hence it makes sense to amortise the latency cost of the cache flush by issuing it asynchronously and then waiting for it only when we need to issue the first IO in the transaction.

TO do this, we need async cache flush primitives to submit the cache flush bio and to wait on it. THe block layer has no such primitives for filesystems, so roll our own for the moment.

---
 fs/xfs/xfs_bio_io.c | 30 ++++++++++++++++++++++++++++++
 fs/xfs/xfs_linux.h  |  1 +
 2 files changed, 31 insertions(+)
fs/xfs/xfs_bio_io.c +18 -0
index 5abf653a45d4..d55420bc72b5 100644
@@ -67,3 +67,33 @@ xfs_flush_bdev(
 	blkdev_issue_flush(bdev, GFP_NOFS);
 }
 
+void
+xfs_flush_bdev_async_endio(
+	struct bio	*bio)
+{
+	if (bio->bi_private)
+		complete(bio->bi_private);
+	bio_put(bio);
+}
+
+/*
+ * Submit a request for an async cache flush to run. If the caller needs to wait
+ * for the flush completion at a later point in time, they must supply a
+ * valid completion. This will be signalled when the flush completes.
+ * The caller never sees the bio that is issued here.
+ */
+void
+xfs_flush_bdev_async(
+	struct block_device	*bdev,

fs/xfs/xfs_bio_io.c xfs_flush_bdev( · line 97

Quoted 10 lines
fs/xfs/xfs_bio_io.c +10 -0
+	struct completion	*done)
+{
+	struct bio *bio;
+
+	bio = bio_alloc(GFP_NOFS, 0);
+	bio_set_dev(bio, bdev);
+	bio->bi_opf = REQ_OP_WRITE | REQ_PREFLUSH | REQ_SYNC;
+	bio->bi_private = done;
+        bio->bi_end_io = xfs_flush_bdev_async_endio;
+

nit: need to align above line with the rest of the code ? can be done at the time of applying the patch.

Quoted 7 lines
fs/xfs/xfs_bio_io.c +2 -0
+	submit_bio(bio);
+}
fs/xfs/xfs_linux.h +0 -0
index e94a2aeefee8..293ff2355e80 100644

fs/xfs/xfs_linux.h static inline uint64_t howmany_64(uint64_t x, uint32_t y) · line 203

Quoted 7 lines
fs/xfs/xfs_linux.h +1 -0
@@ -197,6 +197,7 @@ static inline uint64_t howmany_64(uint64_t x, uint32_t y)
 int xfs_rw_bdev(struct block_device *bdev, sector_t sector, unsigned int count,
 		char *data, unsigned int op);
 void xfs_flush_bdev(struct block_device *bdev);
+void xfs_flush_bdev_async(struct block_device *bdev, struct completion *done);
 
 #define ASSERT_ALWAYS(expr)	\
 	(likely(expr) ? (void)0 : assfail(NULL, #expr, __FILE__, __LINE__))

Looks good.

Reviewed-by on v1 4/8
#31 lore raw

Re: [PATCH 4/8] xfs: async blkdev cache flush

In reply to Dave Chinner
Quoted 8 lines · Dave Chinner

From: Dave Chinner <[email protected]>

The new checkpoint caceh flush mechanism requires us to issue an unconditional cache flush before we start a new checkpoint. We don't want to block for this if we can help it, and we have a fair chunk of CPU work to do between starting the checkpoint and issuing the first journal IO.

Quoted 8 lines

Hence it makes sense to amortise the latency cost of the cache flush by issuing it asynchronously and then waiting for it only when we need to issue the first IO in the transaction.

TO do this, we need async cache flush primitives to submit the cache flush bio and to wait on it. THe block layer has no such primitives for filesystems, so roll our own for the moment.

Thanks for the detailed commit message explaining the reasoning behind the requirement for an async cache flush primitive.

Quoted 49 lines
---
 fs/xfs/xfs_bio_io.c | 30 ++++++++++++++++++++++++++++++
 fs/xfs/xfs_linux.h  |  1 +
 2 files changed, 31 insertions(+)
fs/xfs/xfs_bio_io.c +30 -0
index 5abf653a45d4..d55420bc72b5 100644
@@ -67,3 +67,33 @@ xfs_flush_bdev(
 	blkdev_issue_flush(bdev, GFP_NOFS);
 }
 
+void
+xfs_flush_bdev_async_endio(
+	struct bio	*bio)
+{
+	if (bio->bi_private)
+		complete(bio->bi_private);
+	bio_put(bio);
+}
+
+/*
+ * Submit a request for an async cache flush to run. If the caller needs to wait
+ * for the flush completion at a later point in time, they must supply a
+ * valid completion. This will be signalled when the flush completes.
+ * The caller never sees the bio that is issued here.
+ */
+void
+xfs_flush_bdev_async(
+	struct block_device	*bdev,
+	struct completion	*done)
+{
+	struct bio *bio;
+
+	bio = bio_alloc(GFP_NOFS, 0);
+	bio_set_dev(bio, bdev);
+	bio->bi_opf = REQ_OP_WRITE | REQ_PREFLUSH | REQ_SYNC;
+	bio->bi_private = done;
+        bio->bi_end_io = xfs_flush_bdev_async_endio;
+
+	submit_bio(bio);
+}
fs/xfs/xfs_linux.h +0 -0
index e94a2aeefee8..293ff2355e80 100644

fs/xfs/xfs_linux.h static inline uint64_t howmany_64(uint64_t x, uint32_t y) · line 203

Quoted 7 lines
fs/xfs/xfs_linux.h +1 -0
@@ -197,6 +197,7 @@ static inline uint64_t howmany_64(uint64_t x, uint32_t y)
 int xfs_rw_bdev(struct block_device *bdev, sector_t sector, unsigned int count,
 		char *data, unsigned int op);
 void xfs_flush_bdev(struct block_device *bdev);
+void xfs_flush_bdev_async(struct block_device *bdev, struct completion *done);
 
 #define ASSERT_ALWAYS(expr)	\
 	(likely(expr) ? (void)0 : assfail(NULL, #expr, __FILE__, __LINE__))

--
chandan

#32 lore raw

Re: [PATCH 4/8] xfs: async blkdev cache flush

In reply to Dave Chinner
Quoted 38 lines · Dave Chinner

From: Dave Chinner <[email protected]>

The new checkpoint caceh flush mechanism requires us to issue an unconditional cache flush before we start a new checkpoint. We don't want to block for this if we can help it, and we have a fair chunk of CPU work to do between starting the checkpoint and issuing the first journal IO.

Hence it makes sense to amortise the latency cost of the cache flush by issuing it asynchronously and then waiting for it only when we need to issue the first IO in the transaction.

TO do this, we need async cache flush primitives to submit the cache flush bio and to wait on it. THe block layer has no such primitives for filesystems, so roll our own for the moment.

---
 fs/xfs/xfs_bio_io.c | 30 ++++++++++++++++++++++++++++++
 fs/xfs/xfs_linux.h  |  1 +
 2 files changed, 31 insertions(+)
fs/xfs/xfs_bio_io.c +8 -0
index 5abf653a45d4..d55420bc72b5 100644
@@ -67,3 +67,33 @@ xfs_flush_bdev(
 	blkdev_issue_flush(bdev, GFP_NOFS);
 }
 
+void
+xfs_flush_bdev_async_endio(
+	struct bio	*bio)
+{
+	if (bio->bi_private)
+		complete(bio->bi_private);
+	bio_put(bio);
+}

fs/xfs/xfs_bio_io.c xfs_flush_bdev( · line 87

Quoted 10 lines
fs/xfs/xfs_bio_io.c +10 -0
+
+/*
+ * Submit a request for an async cache flush to run. If the caller needs to wait
+ * for the flush completion at a later point in time, they must supply a
+ * valid completion. This will be signalled when the flush completes.
+ * The caller never sees the bio that is issued here.
+ */
+void
+xfs_flush_bdev_async(
+	struct block_device	*bdev,

Not sure why this isn't a buftarg function, since (AFAICT) this is the only caller in the ~30 patches you've sent to the list. Is there something else coming down the pipeline such that you only have a raw block_device pointer?

fs/xfs/xfs_bio_io.c xfs_flush_bdev( · line 96

Quoted 9 lines
fs/xfs/xfs_bio_io.c +9 -0
+	struct completion	*done)
+{
+	struct bio *bio;
+
+	bio = bio_alloc(GFP_NOFS, 0);
+	bio_set_dev(bio, bdev);
+	bio->bi_opf = REQ_OP_WRITE | REQ_PREFLUSH | REQ_SYNC;
+	bio->bi_private = done;
+        bio->bi_end_io = xfs_flush_bdev_async_endio;

Weird indent here.

--D

Quoted 18 lines
fs/xfs/xfs_bio_io.c +3 -0
+
+	submit_bio(bio);
+}
fs/xfs/xfs_linux.h +1 -0
index e94a2aeefee8..293ff2355e80 100644
@@ -197,6 +197,7 @@ static inline uint64_t howmany_64(uint64_t x, uint32_t y)
 int xfs_rw_bdev(struct block_device *bdev, sector_t sector, unsigned int count,
 		char *data, unsigned int op);
 void xfs_flush_bdev(struct block_device *bdev);
+void xfs_flush_bdev_async(struct block_device *bdev, struct completion *done);
 
 #define ASSERT_ALWAYS(expr)	\
 	(likely(expr) ? (void)0 : assfail(NULL, #expr, __FILE__, __LINE__))
Signature · 1 line
2.28.0
#33 lore raw

PATCH 5/8 xfs: CIL checkpoint flushes caches unconditionally

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

Currently every journal IO is issued as REQ_PREFLUSH | REQ_FUA to guarantee the ordering requirements the journal has w.r.t. metadata writeback. THe two ordering constraints are:

1. we cannot overwrite metadata in the journal until we guarantee that the dirty metadata has been written back in place and is stable.

2. we cannot write back dirty metadata until it has been written to the journal and guaranteed to be stable (and hence recoverable) in the journal.

These rules apply to the atomic transactions recorded in the journal, not to the journal IO itself. Hence we need to ensure metadata is stable before we start writing a new transaction to the journal (guarantee #1), and we need to ensure the entire transaction is stable in the journal before we start metadata writeback (guarantee #2).

The ordering guarantees of #1 are currently provided by REQ_PREFLUSH being added to every iclog IO. This causes the journal IO to issue a cache flush and wait for it to complete before issuing the write IO to the journal. Hence all completed metadata IO is guaranteed to be stable before the journal overwrites the old metadata.

However, for long running CIL checkpoints that might do a thousand journal IOs, we don't need every single one of these iclog IOs to issue a cache flush - the cache flush done before the first iclog is submitted is sufficient to cover the entire range in the log that the checkpoint will overwrite because the CIL space reservation guarantees the tail of the log (completed metadata) is already beyond the range of the checkpoint write.

Hence we only need a full cache flush between closing off the CIL checkpoint context (i.e. when the push switches it out) and issuing the first journal IO. Rather than plumbing this through to the journal IO, we can start this cache flush the moment the CIL context is owned exclusively by the push worker. The cache flush can be in progress while we process the CIL ready for writing, hence reducing the latency of the initial iclog write. This is especially true for large checkpoints, where we might have to process hundreds of thousands of log vectors before we issue the first iclog write. In these cases, it is likely the cache flush has already been completed by the time we have built the CIL log vector chain.

---
 fs/xfs/xfs_log_cil.c | 29 +++++++++++++++++++++++++----
 1 file changed, 25 insertions(+), 4 deletions(-)
fs/xfs/xfs_log_cil.c +25 -4
index c5cc1b7ad25e..8bcacd463f06 100644
@@ -656,6 +656,7 @@ xlog_cil_push_work(
 	struct xfs_log_vec	lvhdr = { NULL };
 	xfs_lsn_t		commit_lsn;
 	xfs_lsn_t		push_seq;
+	DECLARE_COMPLETION_ONSTACK(bdev_flush);
 
 	new_ctx = kmem_zalloc(sizeof(*new_ctx), KM_NOFS);
 	new_ctx->ticket = xlog_cil_ticket_alloc(log);
@@ -719,10 +720,24 @@ xlog_cil_push_work(
 	spin_unlock(&cil->xc_push_lock);
 
 	/*
-	 * pull all the log vectors off the items in the CIL, and
-	 * remove the items from the CIL. We don't need the CIL lock
-	 * here because it's only needed on the transaction commit
-	 * side which is currently locked out by the flush lock.
+	 * The CIL is stable at this point - nothing new will be added to it
+	 * because we hold the flush lock exclusively. Hence we can now issue
+	 * a cache flush to ensure all the completed metadata in the journal we
+	 * are about to overwrite is on stable storage.
+	 *
+	 * This avoids the need to have the iclogs issue REQ_PREFLUSH based
+	 * cache flushes to provide this ordering guarantee, and hence for CIL
+	 * checkpoints that require hundreds or thousands of log writes no
+	 * longer need to issue device cache flushes to provide metadata
+	 * writeback ordering.
+	 */
+	xfs_flush_bdev_async(log->l_mp->m_ddev_targp->bt_bdev, &bdev_flush);
+
+	/*
+	 * Pull all the log vectors off the items in the CIL, and remove the
+	 * items from the CIL. We don't need the CIL lock here because it's only
+	 * needed on the transaction commit side which is currently locked out
+	 * by the flush lock.
 	 */
 	lv = NULL;
 	num_iovecs = 0;
@@ -806,6 +821,12 @@ xlog_cil_push_work(
 	lvhdr.lv_iovecp = &lhdr;
 	lvhdr.lv_next = ctx->lv_chain;
 
+	/*
+	 * Before we format and submit the first iclog, we have to ensure that
+	 * the metadata writeback ordering cache flush is complete.
+	 */
+	wait_for_completion(&bdev_flush);
+
 	error = xlog_write(log, &lvhdr, tic, &ctx->start_lsn, NULL, 0, true);
 	if (error)
 		goto out_abort_free_ticket;
Signature · 1 line
2.28.0
Reviewed-by on v1 5/8
#34 lore raw

Re: [PATCH 5/8] xfs: CIL checkpoint flushes caches unconditionally

Quoted 35 lines · Dave Chinner

From: Dave Chinner <[email protected]>

Currently every journal IO is issued as REQ_PREFLUSH | REQ_FUA to guarantee the ordering requirements the journal has w.r.t. metadata writeback. THe two ordering constraints are:

1. we cannot overwrite metadata in the journal until we guarantee that the dirty metadata has been written back in place and is stable.

2. we cannot write back dirty metadata until it has been written to the journal and guaranteed to be stable (and hence recoverable) in the journal.

These rules apply to the atomic transactions recorded in the journal, not to the journal IO itself. Hence we need to ensure metadata is stable before we start writing a new transaction to the journal (guarantee #1), and we need to ensure the entire transaction is stable in the journal before we start metadata writeback (guarantee #2).

The ordering guarantees of #1 are currently provided by REQ_PREFLUSH being added to every iclog IO. This causes the journal IO to issue a cache flush and wait for it to complete before issuing the write IO to the journal. Hence all completed metadata IO is guaranteed to be stable before the journal overwrites the old metadata.

However, for long running CIL checkpoints that might do a thousand journal IOs, we don't need every single one of these iclog IOs to issue a cache flush - the cache flush done before the first iclog is submitted is sufficient to cover the entire range in the log that the checkpoint will overwrite because the CIL space reservation guarantees the tail of the log (completed metadata) is already beyond the range of the checkpoint write.

Quoted 12 lines

Hence we only need a full cache flush between closing off the CIL checkpoint context (i.e. when the push switches it out) and issuing the first journal IO. Rather than plumbing this through to the journal IO, we can start this cache flush the moment the CIL context is owned exclusively by the push worker. The cache flush can be in progress while we process the CIL ready for writing, hence reducing the latency of the initial iclog write. This is especially true for large checkpoints, where we might have to process hundreds of thousands of log vectors before we issue the first iclog write. In these cases, it is likely the cache flush has already been completed by the time we have built the CIL log vector chain.

Indeed, a single cache flush of the "data device" that is issued before writing the first iclog of a CIL context is sufficient to make sure that the metadata has really reached non-volatile storage.

Quoted 49 lines
---
 fs/xfs/xfs_log_cil.c | 29 +++++++++++++++++++++++++----
 1 file changed, 25 insertions(+), 4 deletions(-)
fs/xfs/xfs_log_cil.c +19 -4
index c5cc1b7ad25e..8bcacd463f06 100644
@@ -656,6 +656,7 @@ xlog_cil_push_work(
 	struct xfs_log_vec	lvhdr = { NULL };
 	xfs_lsn_t		commit_lsn;
 	xfs_lsn_t		push_seq;
+	DECLARE_COMPLETION_ONSTACK(bdev_flush);
 
 	new_ctx = kmem_zalloc(sizeof(*new_ctx), KM_NOFS);
 	new_ctx->ticket = xlog_cil_ticket_alloc(log);
@@ -719,10 +720,24 @@ xlog_cil_push_work(
 	spin_unlock(&cil->xc_push_lock);
 
 	/*
-	 * pull all the log vectors off the items in the CIL, and
-	 * remove the items from the CIL. We don't need the CIL lock
-	 * here because it's only needed on the transaction commit
-	 * side which is currently locked out by the flush lock.
+	 * The CIL is stable at this point - nothing new will be added to it
+	 * because we hold the flush lock exclusively. Hence we can now issue
+	 * a cache flush to ensure all the completed metadata in the journal we
+	 * are about to overwrite is on stable storage.
+	 *
+	 * This avoids the need to have the iclogs issue REQ_PREFLUSH based
+	 * cache flushes to provide this ordering guarantee, and hence for CIL
+	 * checkpoints that require hundreds or thousands of log writes no
+	 * longer need to issue device cache flushes to provide metadata
+	 * writeback ordering.
+	 */
+	xfs_flush_bdev_async(log->l_mp->m_ddev_targp->bt_bdev, &bdev_flush);
+
+	/*
+	 * Pull all the log vectors off the items in the CIL, and remove the
+	 * items from the CIL. We don't need the CIL lock here because it's only
+	 * needed on the transaction commit side which is currently locked out
+	 * by the flush lock.
 	 */
 	lv = NULL;
 	num_iovecs = 0;
@@ -806,6 +821,12 @@ xlog_cil_push_work(
 	lvhdr.lv_iovecp = &lhdr;
 	lvhdr.lv_next = ctx->lv_chain;

fs/xfs/xfs_log_cil.c xlog_cil_push_work( · line 832

Quoted 10 lines
fs/xfs/xfs_log_cil.c +6 -0
 
+	/*
+	 * Before we format and submit the first iclog, we have to ensure that
+	 * the metadata writeback ordering cache flush is complete.
+	 */
+	wait_for_completion(&bdev_flush);
+
 	error = xlog_write(log, &lvhdr, tic, &ctx->start_lsn, NULL, 0, true);
 	if (error)
 		goto out_abort_free_ticket;

--
chandan

Reviewed-by on v1 5/8
#35 lore raw

Re: [PATCH 5/8] xfs: CIL checkpoint flushes caches unconditionally

In reply to Dave Chinner
Quoted 47 lines · Dave Chinner

From: Dave Chinner <[email protected]>

Currently every journal IO is issued as REQ_PREFLUSH | REQ_FUA to guarantee the ordering requirements the journal has w.r.t. metadata writeback. THe two ordering constraints are:

1. we cannot overwrite metadata in the journal until we guarantee that the dirty metadata has been written back in place and is stable.

2. we cannot write back dirty metadata until it has been written to the journal and guaranteed to be stable (and hence recoverable) in the journal.

These rules apply to the atomic transactions recorded in the journal, not to the journal IO itself. Hence we need to ensure metadata is stable before we start writing a new transaction to the journal (guarantee #1), and we need to ensure the entire transaction is stable in the journal before we start metadata writeback (guarantee #2).

The ordering guarantees of #1 are currently provided by REQ_PREFLUSH being added to every iclog IO. This causes the journal IO to issue a cache flush and wait for it to complete before issuing the write IO to the journal. Hence all completed metadata IO is guaranteed to be stable before the journal overwrites the old metadata.

However, for long running CIL checkpoints that might do a thousand journal IOs, we don't need every single one of these iclog IOs to issue a cache flush - the cache flush done before the first iclog is submitted is sufficient to cover the entire range in the log that the checkpoint will overwrite because the CIL space reservation guarantees the tail of the log (completed metadata) is already beyond the range of the checkpoint write.

Hence we only need a full cache flush between closing off the CIL checkpoint context (i.e. when the push switches it out) and issuing the first journal IO. Rather than plumbing this through to the journal IO, we can start this cache flush the moment the CIL context is owned exclusively by the push worker. The cache flush can be in progress while we process the CIL ready for writing, hence reducing the latency of the initial iclog write. This is especially true for large checkpoints, where we might have to process hundreds of thousands of log vectors before we issue the first iclog write. In these cases, it is likely the cache flush has already been completed by the time we have built the CIL log vector chain.

Quoted 1 line

Looks ok,

--D

Quoted 61 lines
---
 fs/xfs/xfs_log_cil.c | 29 +++++++++++++++++++++++++----
 1 file changed, 25 insertions(+), 4 deletions(-)
fs/xfs/xfs_log_cil.c +25 -4
index c5cc1b7ad25e..8bcacd463f06 100644
@@ -656,6 +656,7 @@ xlog_cil_push_work(
 	struct xfs_log_vec	lvhdr = { NULL };
 	xfs_lsn_t		commit_lsn;
 	xfs_lsn_t		push_seq;
+	DECLARE_COMPLETION_ONSTACK(bdev_flush);
 
 	new_ctx = kmem_zalloc(sizeof(*new_ctx), KM_NOFS);
 	new_ctx->ticket = xlog_cil_ticket_alloc(log);
@@ -719,10 +720,24 @@ xlog_cil_push_work(
 	spin_unlock(&cil->xc_push_lock);
 
 	/*
-	 * pull all the log vectors off the items in the CIL, and
-	 * remove the items from the CIL. We don't need the CIL lock
-	 * here because it's only needed on the transaction commit
-	 * side which is currently locked out by the flush lock.
+	 * The CIL is stable at this point - nothing new will be added to it
+	 * because we hold the flush lock exclusively. Hence we can now issue
+	 * a cache flush to ensure all the completed metadata in the journal we
+	 * are about to overwrite is on stable storage.
+	 *
+	 * This avoids the need to have the iclogs issue REQ_PREFLUSH based
+	 * cache flushes to provide this ordering guarantee, and hence for CIL
+	 * checkpoints that require hundreds or thousands of log writes no
+	 * longer need to issue device cache flushes to provide metadata
+	 * writeback ordering.
+	 */
+	xfs_flush_bdev_async(log->l_mp->m_ddev_targp->bt_bdev, &bdev_flush);
+
+	/*
+	 * Pull all the log vectors off the items in the CIL, and remove the
+	 * items from the CIL. We don't need the CIL lock here because it's only
+	 * needed on the transaction commit side which is currently locked out
+	 * by the flush lock.
 	 */
 	lv = NULL;
 	num_iovecs = 0;
@@ -806,6 +821,12 @@ xlog_cil_push_work(
 	lvhdr.lv_iovecp = &lhdr;
 	lvhdr.lv_next = ctx->lv_chain;
 
+	/*
+	 * Before we format and submit the first iclog, we have to ensure that
+	 * the metadata writeback ordering cache flush is complete.
+	 */
+	wait_for_completion(&bdev_flush);
+
 	error = xlog_write(log, &lvhdr, tic, &ctx->start_lsn, NULL, 0, true);
 	if (error)
 		goto out_abort_free_ticket;
Signature · 1 line
2.28.0
#36 lore raw

Re: [PATCH 5/8] xfs: CIL checkpoint flushes caches unconditionally

In reply to Dave Chinner

This looks ok, but please make add two trivial checks that the device actually supports/needs flushes. All that magic of allocating a bio

Quoted 12 lines · Dave Chinner
fs/xfs/xfs_log_cil.c +2 -4
 	new_ctx = kmem_zalloc(sizeof(*new_ctx), KM_NOFS);
 	new_ctx->ticket = xlog_cil_ticket_alloc(log);
@@ -719,10 +720,24 @@ xlog_cil_push_work(
 	spin_unlock(&cil->xc_push_lock);
 
 	/*
-	 * pull all the log vectors off the items in the CIL, and
-	 * remove the items from the CIL. We don't need the CIL lock
-	 * here because it's only needed on the transaction commit
-	 * side which is currently locked out by the flush lock.
+	 * The CIL is stable at this point - nothing new will be added to it
+	 * because we hold the flush lock exclusively. Hence we can now issue

fs/xfs/xfs_log_cil.c xlog_cil_push_work( · line 734

Quoted 10 lines
fs/xfs/xfs_log_cil.c +10 -0
+	 * a cache flush to ensure all the completed metadata in the journal we
+	 * are about to overwrite is on stable storage.
+	 *
+	 * This avoids the need to have the iclogs issue REQ_PREFLUSH based
+	 * cache flushes to provide this ordering guarantee, and hence for CIL
+	 * checkpoints that require hundreds or thousands of log writes no
+	 * longer need to issue device cache flushes to provide metadata
+	 * writeback ordering.
+	 */
+	xfs_flush_bdev_async(log->l_mp->m_ddev_targp->bt_bdev, &bdev_flush);

This still causes a bio allocation, also even if the device does not need flush. Please also use bio_init on a bio passed into xfs_flush_bdev_async to avoid that, and make the whole code conditional to only run if we actually need to flush caches.

#37 lore raw

Re: [PATCH 5/8] xfs: CIL checkpoint flushes caches unconditionally

Quoted 2 lines · Christoph Hellwig

This looks ok, but please make add two trivial checks that the device actually supports/needs flushes. All that magic of allocating a bio

Ok, should be easy enough to do.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#38 lore raw

PATCH 6/8 xfs: remove need_start_rec parameter from xlog_write()

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

The CIL push is the only call to xlog_write that sets this variable to true. The other callers don't need a start rec, and they tell xlog_write what to do by passing the type of ophdr they need written in the flags field. The need_start_rec parameter essentially tells xlog_write to to write an extra ophdr with a XLOG_START_TRANS type, so get rid of the variable to do this and pass XLOG_START_TRANS as the flag value into xlog_write() from the CIL push.

$ size fs/xfs/xfs_log.o*
  text	   data	    bss	    dec	    hex	filename
 27595	    560	      8	  28163	   6e03	fs/xfs/xfs_log.o.orig
 27454	    560	      8	  28022	   6d76	fs/xfs/xfs_log.o.patched
---
 fs/xfs/xfs_log.c      | 44 +++++++++++++++++++++----------------------
 fs/xfs/xfs_log_cil.c  |  3 ++-
 fs/xfs/xfs_log_priv.h |  3 +--
 3 files changed, 25 insertions(+), 25 deletions(-)
fs/xfs/xfs_log.c +22 -22
index 493454c98c6f..6c3fb6dcb505 100644
@@ -871,9 +871,7 @@ xlog_wait_on_iclog_lsn(
 static int
 xlog_write_unmount_record(
 	struct xlog		*log,
-	struct xlog_ticket	*ticket,
-	xfs_lsn_t		*lsn,
-	uint			flags)
+	struct xlog_ticket	*ticket)
 {
 	struct xfs_unmount_log_format ulf = {
 		.magic = XLOG_UNMOUNT_TYPE,
@@ -890,7 +888,7 @@ xlog_write_unmount_record(
 
 	/* account for space used by record data */
 	ticket->t_curr_res -= sizeof(ulf);
-	return xlog_write(log, &vec, ticket, lsn, NULL, flags, false);
+	return xlog_write(log, &vec, ticket, NULL, NULL, XLOG_UNMOUNT_TRANS);
 }
 
 /*
@@ -904,15 +902,13 @@ xlog_unmount_write(
 	struct xfs_mount	*mp = log->l_mp;
 	struct xlog_in_core	*iclog;
 	struct xlog_ticket	*tic = NULL;
-	xfs_lsn_t		lsn;
-	uint			flags = XLOG_UNMOUNT_TRANS;
 	int			error;
 
 	error = xfs_log_reserve(mp, 600, 1, &tic, XFS_LOG, 0);
 	if (error)
 		goto out_err;
 
-	error = xlog_write_unmount_record(log, tic, &lsn, flags);
+	error = xlog_write_unmount_record(log, tic);
 	/*
 	 * At this point, we're umounting anyway, so there's no point in
 	 * transitioning log state to IOERROR. Just continue...
@@ -1604,8 +1600,7 @@ xlog_commit_record(
 	if (XLOG_FORCED_SHUTDOWN(log))
 		return -EIO;
 
-	error = xlog_write(log, &vec, ticket, lsn, iclog, XLOG_COMMIT_TRANS,
-			   false);
+	error = xlog_write(log, &vec, ticket, lsn, iclog, XLOG_COMMIT_TRANS);
 	if (error)
 		xfs_force_shutdown(log->l_mp, SHUTDOWN_LOG_IO_ERROR);
 	return error;
@@ -2202,13 +2197,16 @@ static int
 xlog_write_calc_vec_length(
 	struct xlog_ticket	*ticket,
 	struct xfs_log_vec	*log_vector,
-	bool			need_start_rec)
+	uint			optype)
 {
 	struct xfs_log_vec	*lv;
-	int			headers = need_start_rec ? 1 : 0;
+	int			headers = 0;
 	int			len = 0;
 	int			i;
 
+	if (optype & XLOG_START_TRANS)
+		headers++;
+
 	for (lv = log_vector; lv; lv = lv->lv_next) {
 		/* we don't write ordered log vectors */
 		if (lv->lv_buf_len == XFS_LOG_VEC_ORDERED)
@@ -2428,8 +2426,7 @@ xlog_write(
 	struct xlog_ticket	*ticket,
 	xfs_lsn_t		*start_lsn,
 	struct xlog_in_core	**commit_iclog,
-	uint			flags,
-	bool			need_start_rec)
+	uint			optype)
 {
 	struct xlog_in_core	*iclog = NULL;
 	struct xfs_log_vec	*lv = log_vector;
@@ -2457,8 +2454,9 @@ xlog_write(
 		xfs_force_shutdown(log->l_mp, SHUTDOWN_LOG_IO_ERROR);
 	}
 
-	len = xlog_write_calc_vec_length(ticket, log_vector, need_start_rec);
-	*start_lsn = 0;
+	len = xlog_write_calc_vec_length(ticket, log_vector, optype);
+	if (start_lsn)
+		*start_lsn = 0;
 	while (lv && (!lv->lv_niovecs || index < lv->lv_niovecs)) {
 		void		*ptr;
 		int		log_offset;
@@ -2472,7 +2470,7 @@ xlog_write(
 		ptr = iclog->ic_datap + log_offset;
 
 		/* start_lsn is the first lsn written to. That's all we need. */
-		if (!*start_lsn)
+		if (start_lsn && !*start_lsn)
 			*start_lsn = be64_to_cpu(iclog->ic_header.h_lsn);
 
 		/*
@@ -2485,6 +2483,7 @@ xlog_write(
 			int			copy_len;
 			int			copy_off;
 			bool			ordered = false;
+			bool			wrote_start_rec = false;
 
 			/* ordered log vectors have no regions to write */
 			if (lv->lv_buf_len == XFS_LOG_VEC_ORDERED) {
@@ -2502,13 +2501,15 @@ xlog_write(
 			 * write a start record. Only do this for the first
 			 * iclog we write to.
 			 */
-			if (need_start_rec) {
+			if (optype & XLOG_START_TRANS) {
 				xlog_write_start_rec(ptr, ticket);
 				xlog_write_adv_cnt(&ptr, &len, &log_offset,
 						sizeof(struct xlog_op_header));
+				optype &= ~XLOG_START_TRANS;
+				wrote_start_rec = true;
 			}
 
-			ophdr = xlog_write_setup_ophdr(log, ptr, ticket, flags);
+			ophdr = xlog_write_setup_ophdr(log, ptr, ticket, optype);
 			if (!ophdr)
 				return -EIO;
 
@@ -2539,14 +2540,13 @@ xlog_write(
 			}
 			copy_len += sizeof(struct xlog_op_header);
 			record_cnt++;
-			if (need_start_rec) {
+			if (wrote_start_rec) {
 				copy_len += sizeof(struct xlog_op_header);
 				record_cnt++;
-				need_start_rec = false;
 			}
 			data_cnt += contwr ? copy_len : 0;
 
-			error = xlog_write_copy_finish(log, iclog, flags,
+			error = xlog_write_copy_finish(log, iclog, optype,
 						       &record_cnt, &data_cnt,
 						       &partial_copy,
 						       &partial_copy_len,
@@ -2590,7 +2590,7 @@ xlog_write(
 	spin_lock(&log->l_icloglock);
 	xlog_state_finish_copy(log, iclog, record_cnt, data_cnt);
 	if (commit_iclog) {
-		ASSERT(flags & XLOG_COMMIT_TRANS);
+		ASSERT(optype & XLOG_COMMIT_TRANS);
 		*commit_iclog = iclog;
 	} else {
 		error = xlog_state_release_iclog(log, iclog);
fs/xfs/xfs_log_cil.c +2 -1
index 8bcacd463f06..4093d2d0db7c 100644
@@ -827,7 +827,8 @@ xlog_cil_push_work(
 	 */
 	wait_for_completion(&bdev_flush);
 
-	error = xlog_write(log, &lvhdr, tic, &ctx->start_lsn, NULL, 0, true);
+	error = xlog_write(log, &lvhdr, tic, &ctx->start_lsn, NULL,
+				XLOG_START_TRANS);
 	if (error)
 		goto out_abort_free_ticket;
 
fs/xfs/xfs_log_priv.h +1 -2
index a7ac85aaff4e..10a41b1dd895 100644
@@ -480,8 +480,7 @@ void	xlog_print_tic_res(struct xfs_mount *mp, struct xlog_ticket *ticket);
 void	xlog_print_trans(struct xfs_trans *);
 int	xlog_write(struct xlog *log, struct xfs_log_vec *log_vector,
 		struct xlog_ticket *tic, xfs_lsn_t *start_lsn,
-		struct xlog_in_core **commit_iclog, uint flags,
-		bool need_start_rec);
+		struct xlog_in_core **commit_iclog, uint optype);
 int	xlog_commit_record(struct xlog *log, struct xlog_ticket *ticket,
 		struct xlog_in_core **iclog, xfs_lsn_t *lsn);
 void	xfs_log_ticket_ungrant(struct xlog *log, struct xlog_ticket *ticket);
Signature · 1 line
2.28.0
Reviewed-by on v1 6/8
#39 lore raw

Re: [PATCH 6/8] xfs: remove need_start_rec parameter from xlog_write()

Quoted 10 lines · Dave Chinner

From: Dave Chinner <[email protected]>

The CIL push is the only call to xlog_write that sets this variable to true. The other callers don't need a start rec, and they tell xlog_write what to do by passing the type of ophdr they need written in the flags field. The need_start_rec parameter essentially tells xlog_write to to write an extra ophdr with a XLOG_START_TRANS type, so get rid of the variable to do this and pass XLOG_START_TRANS as the flag value into xlog_write() from the CIL push.

Quoted 5 lines
$ size fs/xfs/xfs_log.o*
  text	   data	    bss	    dec	    hex	filename
 27595	    560	      8	  28163	   6e03	fs/xfs/xfs_log.o.orig
 27454	    560	      8	  28022	   6d76	fs/xfs/xfs_log.o.patched

Looks good to me.

Quoted 188 lines
---
 fs/xfs/xfs_log.c      | 44 +++++++++++++++++++++----------------------
 fs/xfs/xfs_log_cil.c  |  3 ++-
 fs/xfs/xfs_log_priv.h |  3 +--
 3 files changed, 25 insertions(+), 25 deletions(-)
fs/xfs/xfs_log.c +22 -22
index 493454c98c6f..6c3fb6dcb505 100644
@@ -871,9 +871,7 @@ xlog_wait_on_iclog_lsn(
 static int
 xlog_write_unmount_record(
 	struct xlog		*log,
-	struct xlog_ticket	*ticket,
-	xfs_lsn_t		*lsn,
-	uint			flags)
+	struct xlog_ticket	*ticket)
 {
 	struct xfs_unmount_log_format ulf = {
 		.magic = XLOG_UNMOUNT_TYPE,
@@ -890,7 +888,7 @@ xlog_write_unmount_record(
 
 	/* account for space used by record data */
 	ticket->t_curr_res -= sizeof(ulf);
-	return xlog_write(log, &vec, ticket, lsn, NULL, flags, false);
+	return xlog_write(log, &vec, ticket, NULL, NULL, XLOG_UNMOUNT_TRANS);
 }
 
 /*
@@ -904,15 +902,13 @@ xlog_unmount_write(
 	struct xfs_mount	*mp = log->l_mp;
 	struct xlog_in_core	*iclog;
 	struct xlog_ticket	*tic = NULL;
-	xfs_lsn_t		lsn;
-	uint			flags = XLOG_UNMOUNT_TRANS;
 	int			error;
 
 	error = xfs_log_reserve(mp, 600, 1, &tic, XFS_LOG, 0);
 	if (error)
 		goto out_err;
 
-	error = xlog_write_unmount_record(log, tic, &lsn, flags);
+	error = xlog_write_unmount_record(log, tic);
 	/*
 	 * At this point, we're umounting anyway, so there's no point in
 	 * transitioning log state to IOERROR. Just continue...
@@ -1604,8 +1600,7 @@ xlog_commit_record(
 	if (XLOG_FORCED_SHUTDOWN(log))
 		return -EIO;
 
-	error = xlog_write(log, &vec, ticket, lsn, iclog, XLOG_COMMIT_TRANS,
-			   false);
+	error = xlog_write(log, &vec, ticket, lsn, iclog, XLOG_COMMIT_TRANS);
 	if (error)
 		xfs_force_shutdown(log->l_mp, SHUTDOWN_LOG_IO_ERROR);
 	return error;
@@ -2202,13 +2197,16 @@ static int
 xlog_write_calc_vec_length(
 	struct xlog_ticket	*ticket,
 	struct xfs_log_vec	*log_vector,
-	bool			need_start_rec)
+	uint			optype)
 {
 	struct xfs_log_vec	*lv;
-	int			headers = need_start_rec ? 1 : 0;
+	int			headers = 0;
 	int			len = 0;
 	int			i;
 
+	if (optype & XLOG_START_TRANS)
+		headers++;
+
 	for (lv = log_vector; lv; lv = lv->lv_next) {
 		/* we don't write ordered log vectors */
 		if (lv->lv_buf_len == XFS_LOG_VEC_ORDERED)
@@ -2428,8 +2426,7 @@ xlog_write(
 	struct xlog_ticket	*ticket,
 	xfs_lsn_t		*start_lsn,
 	struct xlog_in_core	**commit_iclog,
-	uint			flags,
-	bool			need_start_rec)
+	uint			optype)
 {
 	struct xlog_in_core	*iclog = NULL;
 	struct xfs_log_vec	*lv = log_vector;
@@ -2457,8 +2454,9 @@ xlog_write(
 		xfs_force_shutdown(log->l_mp, SHUTDOWN_LOG_IO_ERROR);
 	}
 
-	len = xlog_write_calc_vec_length(ticket, log_vector, need_start_rec);
-	*start_lsn = 0;
+	len = xlog_write_calc_vec_length(ticket, log_vector, optype);
+	if (start_lsn)
+		*start_lsn = 0;
 	while (lv && (!lv->lv_niovecs || index < lv->lv_niovecs)) {
 		void		*ptr;
 		int		log_offset;
@@ -2472,7 +2470,7 @@ xlog_write(
 		ptr = iclog->ic_datap + log_offset;
 
 		/* start_lsn is the first lsn written to. That's all we need. */
-		if (!*start_lsn)
+		if (start_lsn && !*start_lsn)
 			*start_lsn = be64_to_cpu(iclog->ic_header.h_lsn);
 
 		/*
@@ -2485,6 +2483,7 @@ xlog_write(
 			int			copy_len;
 			int			copy_off;
 			bool			ordered = false;
+			bool			wrote_start_rec = false;
 
 			/* ordered log vectors have no regions to write */
 			if (lv->lv_buf_len == XFS_LOG_VEC_ORDERED) {
@@ -2502,13 +2501,15 @@ xlog_write(
 			 * write a start record. Only do this for the first
 			 * iclog we write to.
 			 */
-			if (need_start_rec) {
+			if (optype & XLOG_START_TRANS) {
 				xlog_write_start_rec(ptr, ticket);
 				xlog_write_adv_cnt(&ptr, &len, &log_offset,
 						sizeof(struct xlog_op_header));
+				optype &= ~XLOG_START_TRANS;
+				wrote_start_rec = true;
 			}
 
-			ophdr = xlog_write_setup_ophdr(log, ptr, ticket, flags);
+			ophdr = xlog_write_setup_ophdr(log, ptr, ticket, optype);
 			if (!ophdr)
 				return -EIO;
 
@@ -2539,14 +2540,13 @@ xlog_write(
 			}
 			copy_len += sizeof(struct xlog_op_header);
 			record_cnt++;
-			if (need_start_rec) {
+			if (wrote_start_rec) {
 				copy_len += sizeof(struct xlog_op_header);
 				record_cnt++;
-				need_start_rec = false;
 			}
 			data_cnt += contwr ? copy_len : 0;
 
-			error = xlog_write_copy_finish(log, iclog, flags,
+			error = xlog_write_copy_finish(log, iclog, optype,
 						       &record_cnt, &data_cnt,
 						       &partial_copy,
 						       &partial_copy_len,
@@ -2590,7 +2590,7 @@ xlog_write(
 	spin_lock(&log->l_icloglock);
 	xlog_state_finish_copy(log, iclog, record_cnt, data_cnt);
 	if (commit_iclog) {
-		ASSERT(flags & XLOG_COMMIT_TRANS);
+		ASSERT(optype & XLOG_COMMIT_TRANS);
 		*commit_iclog = iclog;
 	} else {
 		error = xlog_state_release_iclog(log, iclog);
fs/xfs/xfs_log_cil.c +2 -1
index 8bcacd463f06..4093d2d0db7c 100644
@@ -827,7 +827,8 @@ xlog_cil_push_work(
 	 */
 	wait_for_completion(&bdev_flush);
 
-	error = xlog_write(log, &lvhdr, tic, &ctx->start_lsn, NULL, 0, true);
+	error = xlog_write(log, &lvhdr, tic, &ctx->start_lsn, NULL,
+				XLOG_START_TRANS);
 	if (error)
 		goto out_abort_free_ticket;
 
fs/xfs/xfs_log_priv.h +1 -2
index a7ac85aaff4e..10a41b1dd895 100644
@@ -480,8 +480,7 @@ void	xlog_print_tic_res(struct xfs_mount *mp, struct xlog_ticket *ticket);
 void	xlog_print_trans(struct xfs_trans *);
 int	xlog_write(struct xlog *log, struct xfs_log_vec *log_vector,
 		struct xlog_ticket *tic, xfs_lsn_t *start_lsn,
-		struct xlog_in_core **commit_iclog, uint flags,
-		bool need_start_rec);
+		struct xlog_in_core **commit_iclog, uint optype);
 int	xlog_commit_record(struct xlog *log, struct xlog_ticket *ticket,
 		struct xlog_in_core **iclog, xfs_lsn_t *lsn);
 void	xfs_log_ticket_ungrant(struct xlog *log, struct xlog_ticket *ticket);
Signature · 1 line
chandan
Reviewed-by on v1 6/8
#40 lore raw

Re: [PATCH 6/8] xfs: remove need_start_rec parameter from xlog_write()

In reply to Dave Chinner
Quoted 10 lines · Dave Chinner

From: Dave Chinner <[email protected]>

The CIL push is the only call to xlog_write that sets this variable to true. The other callers don't need a start rec, and they tell xlog_write what to do by passing the type of ophdr they need written in the flags field. The need_start_rec parameter essentially tells xlog_write to to write an extra ophdr with a XLOG_START_TRANS type, so get rid of the variable to do this and pass XLOG_START_TRANS as the flag value into xlog_write() from the CIL push.

Quoted 6 lines
$ size fs/xfs/xfs_log.o*
  text	   data	    bss	    dec	    hex	filename
 27595	    560	      8	  28163	   6e03	fs/xfs/xfs_log.o.orig
 27454	    560	      8	  28022	   6d76	fs/xfs/xfs_log.o.patched

Looks good,

--D

Quoted 190 lines
---
 fs/xfs/xfs_log.c      | 44 +++++++++++++++++++++----------------------
 fs/xfs/xfs_log_cil.c  |  3 ++-
 fs/xfs/xfs_log_priv.h |  3 +--
 3 files changed, 25 insertions(+), 25 deletions(-)
fs/xfs/xfs_log.c +22 -22
index 493454c98c6f..6c3fb6dcb505 100644
@@ -871,9 +871,7 @@ xlog_wait_on_iclog_lsn(
 static int
 xlog_write_unmount_record(
 	struct xlog		*log,
-	struct xlog_ticket	*ticket,
-	xfs_lsn_t		*lsn,
-	uint			flags)
+	struct xlog_ticket	*ticket)
 {
 	struct xfs_unmount_log_format ulf = {
 		.magic = XLOG_UNMOUNT_TYPE,
@@ -890,7 +888,7 @@ xlog_write_unmount_record(
 
 	/* account for space used by record data */
 	ticket->t_curr_res -= sizeof(ulf);
-	return xlog_write(log, &vec, ticket, lsn, NULL, flags, false);
+	return xlog_write(log, &vec, ticket, NULL, NULL, XLOG_UNMOUNT_TRANS);
 }
 
 /*
@@ -904,15 +902,13 @@ xlog_unmount_write(
 	struct xfs_mount	*mp = log->l_mp;
 	struct xlog_in_core	*iclog;
 	struct xlog_ticket	*tic = NULL;
-	xfs_lsn_t		lsn;
-	uint			flags = XLOG_UNMOUNT_TRANS;
 	int			error;
 
 	error = xfs_log_reserve(mp, 600, 1, &tic, XFS_LOG, 0);
 	if (error)
 		goto out_err;
 
-	error = xlog_write_unmount_record(log, tic, &lsn, flags);
+	error = xlog_write_unmount_record(log, tic);
 	/*
 	 * At this point, we're umounting anyway, so there's no point in
 	 * transitioning log state to IOERROR. Just continue...
@@ -1604,8 +1600,7 @@ xlog_commit_record(
 	if (XLOG_FORCED_SHUTDOWN(log))
 		return -EIO;
 
-	error = xlog_write(log, &vec, ticket, lsn, iclog, XLOG_COMMIT_TRANS,
-			   false);
+	error = xlog_write(log, &vec, ticket, lsn, iclog, XLOG_COMMIT_TRANS);
 	if (error)
 		xfs_force_shutdown(log->l_mp, SHUTDOWN_LOG_IO_ERROR);
 	return error;
@@ -2202,13 +2197,16 @@ static int
 xlog_write_calc_vec_length(
 	struct xlog_ticket	*ticket,
 	struct xfs_log_vec	*log_vector,
-	bool			need_start_rec)
+	uint			optype)
 {
 	struct xfs_log_vec	*lv;
-	int			headers = need_start_rec ? 1 : 0;
+	int			headers = 0;
 	int			len = 0;
 	int			i;
 
+	if (optype & XLOG_START_TRANS)
+		headers++;
+
 	for (lv = log_vector; lv; lv = lv->lv_next) {
 		/* we don't write ordered log vectors */
 		if (lv->lv_buf_len == XFS_LOG_VEC_ORDERED)
@@ -2428,8 +2426,7 @@ xlog_write(
 	struct xlog_ticket	*ticket,
 	xfs_lsn_t		*start_lsn,
 	struct xlog_in_core	**commit_iclog,
-	uint			flags,
-	bool			need_start_rec)
+	uint			optype)
 {
 	struct xlog_in_core	*iclog = NULL;
 	struct xfs_log_vec	*lv = log_vector;
@@ -2457,8 +2454,9 @@ xlog_write(
 		xfs_force_shutdown(log->l_mp, SHUTDOWN_LOG_IO_ERROR);
 	}
 
-	len = xlog_write_calc_vec_length(ticket, log_vector, need_start_rec);
-	*start_lsn = 0;
+	len = xlog_write_calc_vec_length(ticket, log_vector, optype);
+	if (start_lsn)
+		*start_lsn = 0;
 	while (lv && (!lv->lv_niovecs || index < lv->lv_niovecs)) {
 		void		*ptr;
 		int		log_offset;
@@ -2472,7 +2470,7 @@ xlog_write(
 		ptr = iclog->ic_datap + log_offset;
 
 		/* start_lsn is the first lsn written to. That's all we need. */
-		if (!*start_lsn)
+		if (start_lsn && !*start_lsn)
 			*start_lsn = be64_to_cpu(iclog->ic_header.h_lsn);
 
 		/*
@@ -2485,6 +2483,7 @@ xlog_write(
 			int			copy_len;
 			int			copy_off;
 			bool			ordered = false;
+			bool			wrote_start_rec = false;
 
 			/* ordered log vectors have no regions to write */
 			if (lv->lv_buf_len == XFS_LOG_VEC_ORDERED) {
@@ -2502,13 +2501,15 @@ xlog_write(
 			 * write a start record. Only do this for the first
 			 * iclog we write to.
 			 */
-			if (need_start_rec) {
+			if (optype & XLOG_START_TRANS) {
 				xlog_write_start_rec(ptr, ticket);
 				xlog_write_adv_cnt(&ptr, &len, &log_offset,
 						sizeof(struct xlog_op_header));
+				optype &= ~XLOG_START_TRANS;
+				wrote_start_rec = true;
 			}
 
-			ophdr = xlog_write_setup_ophdr(log, ptr, ticket, flags);
+			ophdr = xlog_write_setup_ophdr(log, ptr, ticket, optype);
 			if (!ophdr)
 				return -EIO;
 
@@ -2539,14 +2540,13 @@ xlog_write(
 			}
 			copy_len += sizeof(struct xlog_op_header);
 			record_cnt++;
-			if (need_start_rec) {
+			if (wrote_start_rec) {
 				copy_len += sizeof(struct xlog_op_header);
 				record_cnt++;
-				need_start_rec = false;
 			}
 			data_cnt += contwr ? copy_len : 0;
 
-			error = xlog_write_copy_finish(log, iclog, flags,
+			error = xlog_write_copy_finish(log, iclog, optype,
 						       &record_cnt, &data_cnt,
 						       &partial_copy,
 						       &partial_copy_len,
@@ -2590,7 +2590,7 @@ xlog_write(
 	spin_lock(&log->l_icloglock);
 	xlog_state_finish_copy(log, iclog, record_cnt, data_cnt);
 	if (commit_iclog) {
-		ASSERT(flags & XLOG_COMMIT_TRANS);
+		ASSERT(optype & XLOG_COMMIT_TRANS);
 		*commit_iclog = iclog;
 	} else {
 		error = xlog_state_release_iclog(log, iclog);
fs/xfs/xfs_log_cil.c +2 -1
index 8bcacd463f06..4093d2d0db7c 100644
@@ -827,7 +827,8 @@ xlog_cil_push_work(
 	 */
 	wait_for_completion(&bdev_flush);
 
-	error = xlog_write(log, &lvhdr, tic, &ctx->start_lsn, NULL, 0, true);
+	error = xlog_write(log, &lvhdr, tic, &ctx->start_lsn, NULL,
+				XLOG_START_TRANS);
 	if (error)
 		goto out_abort_free_ticket;
 
fs/xfs/xfs_log_priv.h +1 -2
index a7ac85aaff4e..10a41b1dd895 100644
@@ -480,8 +480,7 @@ void	xlog_print_tic_res(struct xfs_mount *mp, struct xlog_ticket *ticket);
 void	xlog_print_trans(struct xfs_trans *);
 int	xlog_write(struct xlog *log, struct xfs_log_vec *log_vector,
 		struct xlog_ticket *tic, xfs_lsn_t *start_lsn,
-		struct xlog_in_core **commit_iclog, uint flags,
-		bool need_start_rec);
+		struct xlog_in_core **commit_iclog, uint optype);
 int	xlog_commit_record(struct xlog *log, struct xlog_ticket *ticket,
 		struct xlog_in_core **iclog, xfs_lsn_t *lsn);
 void	xfs_log_ticket_ungrant(struct xlog *log, struct xlog_ticket *ticket);
Signature · 1 line
2.28.0
#41 lore raw

Re: [PATCH 6/8] xfs: remove need_start_rec parameter from xlog_write()

In reply to Dave Chinner

fs/xfs/xfs_log.c static int · line 2208

Quoted 2 lines
fs/xfs/xfs_log.c +2 -0
+	if (optype & XLOG_START_TRANS)
+		headers++;

This deserves a comment.

fs/xfs/xfs_log.c xlog_write( · line 2459

Quoted 3 lines
fs/xfs/xfs_log.c +3 -0
+	len = xlog_write_calc_vec_length(ticket, log_vector, optype);
+	if (start_lsn)
+		*start_lsn = 0;

I'd slightly prefer that allowing a NULL start_lsn was a separate prep patch. As-is it really clutters the patch and detracts from the real change.

fs/xfs/xfs_log.c xlog_write( · line 2504

Quoted 13 lines
fs/xfs/xfs_log.c +2 -1
 			int			copy_len;
 			int			copy_off;
 			bool			ordered = false;
+			bool			wrote_start_rec = false;
 
 			/* ordered log vectors have no regions to write */
 			if (lv->lv_buf_len == XFS_LOG_VEC_ORDERED) {
@@ -2502,13 +2501,15 @@ xlog_write(
 			 * write a start record. Only do this for the first
 			 * iclog we write to.
 			 */
-			if (need_start_rec) {
+			if (optype & XLOG_START_TRANS) {

So this relies on the fact that the only callers that passes an optype of XLOG_START_TRANS only writes a single lv. I think we want an assert for that somewhere to avoid a bad surprise later.

#42 lore raw

Re: [PATCH 6/8] xfs: remove need_start_rec parameter from xlog_write()

Quoted 4 lines · Christoph Hellwig

fs/xfs/xfs_log.c static int · line 2208

Quoted 2 lines
fs/xfs/xfs_log.c +2 -0
+	if (optype & XLOG_START_TRANS)
+		headers++;

This deserves a comment.

It gets killed off later, so it's a waste of time to prettify this.

Quoted 7 lines

fs/xfs/xfs_log.c xlog_write( · line 2459

Quoted 3 lines
fs/xfs/xfs_log.c +3 -0
+	len = xlog_write_calc_vec_length(ticket, log_vector, optype);
+	if (start_lsn)
+		*start_lsn = 0;

I'd slightly prefer that allowing a NULL start_lsn was a separate prep patch. As-is it really clutters the patch and detracts from the real change.

No, I've already got enough patches in this whole series to deal with. I'm not splitting out simple, obvious changes into tiny two line patches that require me to do more work for zero gain.

Quoted 5 lines
Quoted 5 lines
fs/xfs/xfs_log.c +1 -0
 			int			copy_len;
 			int			copy_off;
 			bool			ordered = false;
+			bool			wrote_start_rec = false;
Quoted 12 lines

fs/xfs/xfs_log.c xlog_write( · line 2504

Quoted 8 lines
fs/xfs/xfs_log.c +1 -1
 			/* ordered log vectors have no regions to write */
 			if (lv->lv_buf_len == XFS_LOG_VEC_ORDERED) {
@@ -2502,13 +2501,15 @@ xlog_write(
 			 * write a start record. Only do this for the first
 			 * iclog we write to.
 			 */
-			if (need_start_rec) {
+			if (optype & XLOG_START_TRANS) {

So this relies on the fact that the only callers that passes an optype of XLOG_START_TRANS only writes a single lv. I think we want an assert for that somewhere to avoid a bad surprise later.

This also gets killed off later, so again such things are largely a waste of my time as all it does is cause rebase conflicts in multiple patches and doesn't actually change the end result. So, again, no.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#43 lore raw

PATCH 7/8 xfs: journal IO cache flush reductions

In reply to Dave Chinner

From: Steve Lord <[email protected]>

Currently every journal IO is issued as REQ_PREFLUSH | REQ_FUA to guarantee the ordering requirements the journal has w.r.t. metadata writeback. THe two ordering constraints are:

1. we cannot overwrite metadata in the journal until we guarantee that the dirty metadata has been written back in place and is stable.

2. we cannot write back dirty metadata until it has been written to the journal and guaranteed to be stable (and hence recoverable) in the journal.

The ordering guarantees of #1 are provided by REQ_PREFLUSH. This causes the journal IO to issue a cache flush and wait for it to complete before issuing the write IO to the journal. Hence all completed metadata IO is guaranteed to be stable before the journal overwrites the old metadata.

The ordering guarantees of #2 are provided by the REQ_FUA, which ensures the journal writes do not complete until they are on stable storage. Hence by the time the last journal IO in a checkpoint completes, we know that the entire checkpoint is on stable storage and we can unpin the dirty metadata and allow it to be written back.

This is the mechanism by which ordering was first implemented in XFS way back in 2002 by this commit:

commit 95d97c36e5155075ba2eb22b17562cfcc53fcf96

Author: Steve Lord <[email protected]>
Date:   Fri May 24 14:30:21 2002 +0000

Add support for drive write cache flushing - should the kernel have the infrastructure

A lot has changed since then, most notably we now use delayed logging to checkpoint the filesystem to the journal rather than write each individual transaction to the journal. Cache flushes on journal IO are necessary when individual transactions are wholly contained within a single iclog. However, CIL checkpoints are single transactions that typically span hundreds to thousands of individual journal writes, and so the requirements for device cache flushing have changed.

That is, the ordering rules I state above apply to ordering of atomic transactions recorded in the journal, not to the journal IO itself. Hence we need to ensure metadata is stable before we start writing a new transaction to the journal (guarantee #1), and we need to ensure the entire transaction is stable in the journal before we start metadata writeback (guarantee #2).

Hence we only need a REQ_PREFLUSH on the journal IO that starts a new journal transaction to provide #1, and it is not on any other journal IO done within the context of that journal transaction.

The CIL checkpoint already issues a cache flush before it starts writing to the log, so we no longer need the iclog IO to issue a REQ_REFLUSH for us. Hence if XLOG_START_TRANS is passed to xlog_write(), we no longer need to mark the first iclog in the log write with REQ_PREFLUSH for this case.

Given the new ordering semantics of commit records for the CIL, we need iclogs containing commit to issue a REQ_PREFLUSH. We also require unmount records to do this. Hence for both XLOG_COMMIT_TRANS and XLOG_UNMOUNT_TRANS xlog_write() calls we need to mark the first iclog being written with REQ_PREFLUSH.

For both commit records and unmount records, we also want them immediately on stable storage, so we want to also mark the iclogs that contain these records to be marked REQ_FUA. That means if a record is split across multiple iclogs, they are all marked REQ_FUA and not just the last one so that when the transaction is completed all the parts of the record are on stable storage.

As an optimisation, when the commit record lands in the same iclog as the journal transaction starts, we don't need to wait for anything and can simply use REQ_FUA to provide guarantee #2. This means that for fsync() heavy workloads, the cache flush behaviour is completely unchanged and there is no degradation in performance as a result of optimise the multi-IO transaction case.

The most notable sign that there is less IO latency on my test machine (nvme SSDs) is that the "noiclogs" rate has dropped substantially. This metric indicates that the CIL push is blocking in xlog_get_iclog_space() waiting for iclog IO completion to occur. With 8 iclogs of 256kB, the rate is appoximately 1 noiclog event to every 4 iclog writes. IOWs, every 4th call to xlog_get_iclog_space() is blocking waiting for log IO. With the changes in this patch, this drops to 1 noiclog event for every 100 iclog writes. Hence it is clear that log IO is completing much faster than it was previously, but it is also clear that for large iclog sizes, this isn't the performance limiting factor on this hardware.

With smaller iclogs (32kB), however, there is a sustantial difference. With the cache flush modifications, the journal is now running at over 4000 write IOPS, and the journal throughput is largely identical to the 256kB iclogs and the noiclog event rate stays low at about 1:50 iclog writes. The existing code tops out at about 2500 IOPS as the number of cache flushes dominate performance and latency. The noiclog event rate is about 1:4, and the performance variance is quite large as the journal throughput can fall to less than half the peak sustained rate when the cache flush rate prevents metadata writeback from keeping up and the log runs out of space and throttles reservations.

As a result:

logbsize fsmark create rate rm -rf
before 32kb 152851+/-5.3e+04 5m28s
patched 32kb 221533+/-1.1e+04 5m24s

before 256kb 220239+/-6.2e+03 4m58s
patched 256kb 228286+/-9.2e+03 5m06s

The rm -rf times are included because I ran them, but the differences are largely noise. This workload is largely metadata read IO latency bound and the changes to the journal cache flushing doesn't really make any noticable difference to behaviour apart from a reduction in noiclog events from background CIL pushing.

---
 fs/xfs/xfs_log.c      | 33 +++++++++++++++++++++++----------
 fs/xfs/xfs_log_cil.c  |  7 ++++++-
 fs/xfs/xfs_log_priv.h |  4 ++++
 3 files changed, 33 insertions(+), 11 deletions(-)
fs/xfs/xfs_log.c +23 -10
index 6c3fb6dcb505..08d68a6161ae 100644
@@ -1806,8 +1806,7 @@ xlog_write_iclog(
 	struct xlog		*log,
 	struct xlog_in_core	*iclog,
 	uint64_t		bno,
-	unsigned int		count,
-	bool			need_flush)
+	unsigned int		count)
 {
 	ASSERT(bno < log->l_logBBsize);
 
@@ -1845,10 +1844,12 @@ xlog_write_iclog(
 	 * writeback throttle from throttling log writes behind background
 	 * metadata writeback and causing priority inversions.
 	 */
-	iclog->ic_bio.bi_opf = REQ_OP_WRITE | REQ_META | REQ_SYNC |
-				REQ_IDLE | REQ_FUA;
-	if (need_flush)
+	iclog->ic_bio.bi_opf = REQ_OP_WRITE | REQ_META | REQ_SYNC | REQ_IDLE;
+	if (iclog->ic_flags & XLOG_ICL_NEED_FLUSH)
 		iclog->ic_bio.bi_opf |= REQ_PREFLUSH;
+	if (iclog->ic_flags & XLOG_ICL_NEED_FUA)
+		iclog->ic_bio.bi_opf |= REQ_FUA;
+	iclog->ic_flags &= ~(XLOG_ICL_NEED_FLUSH | XLOG_ICL_NEED_FUA);
 
 	if (xlog_map_iclog_data(&iclog->ic_bio, iclog->ic_data, count)) {
 		xfs_force_shutdown(log->l_mp, SHUTDOWN_LOG_IO_ERROR);
@@ -1951,7 +1952,7 @@ xlog_sync(
 	unsigned int		roundoff;       /* roundoff to BB or stripe */
 	uint64_t		bno;
 	unsigned int		size;
-	bool			need_flush = true, split = false;
+	bool			split = false;
 
 	ASSERT(atomic_read(&iclog->ic_refcnt) == 0);
 
@@ -2009,13 +2010,14 @@ xlog_sync(
 	 * synchronously here; for an internal log we can simply use the block
 	 * layer state machine for preflushes.
 	 */
-	if (log->l_targ != log->l_mp->m_ddev_targp || split) {
+	if (log->l_targ != log->l_mp->m_ddev_targp ||
+	    (split && (iclog->ic_flags & XLOG_ICL_NEED_FLUSH))) {
 		xfs_flush_bdev(log->l_mp->m_ddev_targp->bt_bdev);
-		need_flush = false;
+		iclog->ic_flags &= ~XLOG_ICL_NEED_FLUSH;
 	}
 
 	xlog_verify_iclog(log, iclog, count);
-	xlog_write_iclog(log, iclog, bno, count, need_flush);
+	xlog_write_iclog(log, iclog, bno, count);
 }
 
 /*
@@ -2469,10 +2471,21 @@ xlog_write(
 		ASSERT(log_offset <= iclog->ic_size - 1);
 		ptr = iclog->ic_datap + log_offset;
 
-		/* start_lsn is the first lsn written to. That's all we need. */
+		/* Start_lsn is the first lsn written to. */
 		if (start_lsn && !*start_lsn)
 			*start_lsn = be64_to_cpu(iclog->ic_header.h_lsn);
 
+		/*
+		 * iclogs containing commit records or unmount records need
+		 * to issue ordering cache flushes and commit immediately
+		 * to stable storage to guarantee journal vs metadata ordering
+		 * is correctly maintained in the storage media.
+		 */
+		if (optype & (XLOG_COMMIT_TRANS | XLOG_UNMOUNT_TRANS)) {
+			iclog->ic_flags |= (XLOG_ICL_NEED_FLUSH |
+						XLOG_ICL_NEED_FUA);
+		}
+
 		/*
 		 * This loop writes out as many regions as can fit in the amount
 		 * of space which was allocated by xlog_state_get_iclog_space().
fs/xfs/xfs_log_cil.c +6 -1
index 4093d2d0db7c..370da7c2bfc8 100644
@@ -894,10 +894,15 @@ xlog_cil_push_work(
 
 	/*
 	 * If the checkpoint spans multiple iclogs, wait for all previous
-	 * iclogs to complete before we submit the commit_iclog.
+	 * iclogs to complete before we submit the commit_iclog. If it is in the
+	 * same iclog as the start of the checkpoint, then we can skip the iclog
+	 * cache flush because there are no other iclogs we need to order
+	 * against.
 	 */
 	if (ctx->start_lsn != commit_lsn)
 		xlog_wait_on_iclog_lsn(commit_iclog, ctx->start_lsn);
+	else
+		commit_iclog->ic_flags &= ~XLOG_ICL_NEED_FLUSH;
 
 	/* release the hounds! */
 	xfs_log_release_iclog(commit_iclog);
fs/xfs/xfs_log_priv.h +4 -0
index 10a41b1dd895..24acdc54e44e 100644
@@ -133,6 +133,9 @@ enum xlog_iclog_state {
 
 #define XLOG_COVER_OPS		5
 
+#define XLOG_ICL_NEED_FLUSH	(1 << 0)	/* iclog needs REQ_PREFLUSH */
+#define XLOG_ICL_NEED_FUA	(1 << 0)	/* iclog needs REQ_FUA */
+
 /* Ticket reservation region accounting */ 
 #define XLOG_TIC_LEN_MAX	15
 
@@ -201,6 +204,7 @@ typedef struct xlog_in_core {
 	u32			ic_size;
 	u32			ic_offset;
 	enum xlog_iclog_state	ic_state;
+	unsigned int		ic_flags;
 	char			*ic_datap;	/* pointer to iclog data */
 
 	/* Callback structures need their own cacheline */
Signature · 1 line
2.28.0
#44 lore raw

PATCH v2 7/8 xfs: journal IO cache flush reductions

From: Dave Chinner <[email protected]>

Currently every journal IO is issued as REQ_PREFLUSH | REQ_FUA to guarantee the ordering requirements the journal has w.r.t. metadata writeback. THe two ordering constraints are:

1. we cannot overwrite metadata in the journal until we guarantee that the dirty metadata has been written back in place and is stable.

2. we cannot write back dirty metadata until it has been written to the journal and guaranteed to be stable (and hence recoverable) in the journal.

The ordering guarantees of #1 are provided by REQ_PREFLUSH. This causes the journal IO to issue a cache flush and wait for it to complete before issuing the write IO to the journal. Hence all completed metadata IO is guaranteed to be stable before the journal overwrites the old metadata.

The ordering guarantees of #2 are provided by the REQ_FUA, which ensures the journal writes do not complete until they are on stable storage. Hence by the time the last journal IO in a checkpoint completes, we know that the entire checkpoint is on stable storage and we can unpin the dirty metadata and allow it to be written back.

This is the mechanism by which ordering was first implemented in XFS way back in 2002 by this commit:

commit 95d97c36e5155075ba2eb22b17562cfcc53fcf96

Author: Steve Lord <[email protected]>
Date:   Fri May 24 14:30:21 2002 +0000

Add support for drive write cache flushing - should the kernel have the infrastructure

A lot has changed since then, most notably we now use delayed logging to checkpoint the filesystem to the journal rather than write each individual transaction to the journal. Cache flushes on journal IO are necessary when individual transactions are wholly contained within a single iclog. However, CIL checkpoints are single transactions that typically span hundreds to thousands of individual journal writes, and so the requirements for device cache flushing have changed.

That is, the ordering rules I state above apply to ordering of atomic transactions recorded in the journal, not to the journal IO itself. Hence we need to ensure metadata is stable before we start writing a new transaction to the journal (guarantee #1), and we need to ensure the entire transaction is stable in the journal before we start metadata writeback (guarantee #2).

Hence we only need a REQ_PREFLUSH on the journal IO that starts a new journal transaction to provide #1, and it is not on any other journal IO done within the context of that journal transaction.

The CIL checkpoint already issues a cache flush before it starts writing to the log, so we no longer need the iclog IO to issue a REQ_REFLUSH for us. Hence if XLOG_START_TRANS is passed to xlog_write(), we no longer need to mark the first iclog in the log write with REQ_PREFLUSH for this case.

Given the new ordering semantics of commit records for the CIL, we need iclogs containing commit to issue a REQ_PREFLUSH. We also require unmount records to do this. Hence for both XLOG_COMMIT_TRANS and XLOG_UNMOUNT_TRANS xlog_write() calls we need to mark the first iclog being written with REQ_PREFLUSH.

For both commit records and unmount records, we also want them immediately on stable storage, so we want to also mark the iclogs that contain these records to be marked REQ_FUA. That means if a record is split across multiple iclogs, they are all marked REQ_FUA and not just the last one so that when the transaction is completed all the parts of the record are on stable storage.

As an optimisation, when the commit record lands in the same iclog as the journal transaction starts, we don't need to wait for anything and can simply use REQ_FUA to provide guarantee #2. This means that for fsync() heavy workloads, the cache flush behaviour is completely unchanged and there is no degradation in performance as a result of optimise the multi-IO transaction case.

The most notable sign that there is less IO latency on my test machine (nvme SSDs) is that the "noiclogs" rate has dropped substantially. This metric indicates that the CIL push is blocking in xlog_get_iclog_space() waiting for iclog IO completion to occur. With 8 iclogs of 256kB, the rate is appoximately 1 noiclog event to every 4 iclog writes. IOWs, every 4th call to xlog_get_iclog_space() is blocking waiting for log IO. With the changes in this patch, this drops to 1 noiclog event for every 100 iclog writes. Hence it is clear that log IO is completing much faster than it was previously, but it is also clear that for large iclog sizes, this isn't the performance limiting factor on this hardware.

With smaller iclogs (32kB), however, there is a sustantial difference. With the cache flush modifications, the journal is now running at over 4000 write IOPS, and the journal throughput is largely identical to the 256kB iclogs and the noiclog event rate stays low at about 1:50 iclog writes. The existing code tops out at about 2500 IOPS as the number of cache flushes dominate performance and latency. The noiclog event rate is about 1:4, and the performance variance is quite large as the journal throughput can fall to less than half the peak sustained rate when the cache flush rate prevents metadata writeback from keeping up and the log runs out of space and throttles reservations.

As a result:

logbsize fsmark create rate rm -rf
before 32kb 152851+/-5.3e+04 5m28s
patched 32kb 221533+/-1.1e+04 5m24s

before 256kb 220239+/-6.2e+03 4m58s
patched 256kb 228286+/-9.2e+03 5m06s

The rm -rf times are included because I ran them, but the differences are largely noise. This workload is largely metadata read IO latency bound and the changes to the journal cache flushing doesn't really make any noticable difference to behaviour apart from a reduction in noiclog events from background CIL pushing.

---
Version 2:
- repost manually without git/guilt mangling the patch author
- fix bug in XLOG_ICL_NEED_FUA definition that didn't manifest as an ordering bug in generic/45[57] until testing the CIL pipelining changes much later in the series.

 fs/xfs/xfs_log.c      | 33 +++++++++++++++++++++++----------
 fs/xfs/xfs_log_cil.c  |  7 ++++++-
 fs/xfs/xfs_log_priv.h |  4 ++++
 3 files changed, 33 insertions(+), 11 deletions(-)
fs/xfs/xfs_log.c +23 -10
index 6c3fb6dcb505..08d68a6161ae 100644
@@ -1806,8 +1806,7 @@ xlog_write_iclog(
 	struct xlog		*log,
 	struct xlog_in_core	*iclog,
 	uint64_t		bno,
-	unsigned int		count,
-	bool			need_flush)
+	unsigned int		count)
 {
 	ASSERT(bno < log->l_logBBsize);
 
@@ -1845,10 +1844,12 @@ xlog_write_iclog(
 	 * writeback throttle from throttling log writes behind background
 	 * metadata writeback and causing priority inversions.
 	 */
-	iclog->ic_bio.bi_opf = REQ_OP_WRITE | REQ_META | REQ_SYNC |
-				REQ_IDLE | REQ_FUA;
-	if (need_flush)
+	iclog->ic_bio.bi_opf = REQ_OP_WRITE | REQ_META | REQ_SYNC | REQ_IDLE;
+	if (iclog->ic_flags & XLOG_ICL_NEED_FLUSH)
 		iclog->ic_bio.bi_opf |= REQ_PREFLUSH;
+	if (iclog->ic_flags & XLOG_ICL_NEED_FUA)
+		iclog->ic_bio.bi_opf |= REQ_FUA;
+	iclog->ic_flags &= ~(XLOG_ICL_NEED_FLUSH | XLOG_ICL_NEED_FUA);
 
 	if (xlog_map_iclog_data(&iclog->ic_bio, iclog->ic_data, count)) {
 		xfs_force_shutdown(log->l_mp, SHUTDOWN_LOG_IO_ERROR);
@@ -1951,7 +1952,7 @@ xlog_sync(
 	unsigned int		roundoff;       /* roundoff to BB or stripe */
 	uint64_t		bno;
 	unsigned int		size;
-	bool			need_flush = true, split = false;
+	bool			split = false;
 
 	ASSERT(atomic_read(&iclog->ic_refcnt) == 0);
 
@@ -2009,13 +2010,14 @@ xlog_sync(
 	 * synchronously here; for an internal log we can simply use the block
 	 * layer state machine for preflushes.
 	 */
-	if (log->l_targ != log->l_mp->m_ddev_targp || split) {
+	if (log->l_targ != log->l_mp->m_ddev_targp ||
+	    (split && (iclog->ic_flags & XLOG_ICL_NEED_FLUSH))) {
 		xfs_flush_bdev(log->l_mp->m_ddev_targp->bt_bdev);
-		need_flush = false;
+		iclog->ic_flags &= ~XLOG_ICL_NEED_FLUSH;
 	}
 
 	xlog_verify_iclog(log, iclog, count);
-	xlog_write_iclog(log, iclog, bno, count, need_flush);
+	xlog_write_iclog(log, iclog, bno, count);
 }
 
 /*
@@ -2469,10 +2471,21 @@ xlog_write(
 		ASSERT(log_offset <= iclog->ic_size - 1);
 		ptr = iclog->ic_datap + log_offset;
 
-		/* start_lsn is the first lsn written to. That's all we need. */
+		/* Start_lsn is the first lsn written to. */
 		if (start_lsn && !*start_lsn)
 			*start_lsn = be64_to_cpu(iclog->ic_header.h_lsn);
 
+		/*
+		 * iclogs containing commit records or unmount records need
+		 * to issue ordering cache flushes and commit immediately
+		 * to stable storage to guarantee journal vs metadata ordering
+		 * is correctly maintained in the storage media.
+		 */
+		if (optype & (XLOG_COMMIT_TRANS | XLOG_UNMOUNT_TRANS)) {
+			iclog->ic_flags |= (XLOG_ICL_NEED_FLUSH |
+						XLOG_ICL_NEED_FUA);
+		}
+
 		/*
 		 * This loop writes out as many regions as can fit in the amount
 		 * of space which was allocated by xlog_state_get_iclog_space().
fs/xfs/xfs_log_cil.c +6 -1
index 4093d2d0db7c..370da7c2bfc8 100644
@@ -894,10 +894,15 @@ xlog_cil_push_work(
 
 	/*
 	 * If the checkpoint spans multiple iclogs, wait for all previous
-	 * iclogs to complete before we submit the commit_iclog.
+	 * iclogs to complete before we submit the commit_iclog. If it is in the
+	 * same iclog as the start of the checkpoint, then we can skip the iclog
+	 * cache flush because there are no other iclogs we need to order
+	 * against.
 	 */
 	if (ctx->start_lsn != commit_lsn)
 		xlog_wait_on_iclog_lsn(commit_iclog, ctx->start_lsn);
+	else
+		commit_iclog->ic_flags &= ~XLOG_ICL_NEED_FLUSH;
 
 	/* release the hounds! */
 	xfs_log_release_iclog(commit_iclog);
fs/xfs/xfs_log_priv.h +4 -0
index 10a41b1dd895..a77e00b7789a 100644
@@ -133,6 +133,9 @@ enum xlog_iclog_state {
 
 #define XLOG_COVER_OPS		5
 
+#define XLOG_ICL_NEED_FLUSH	(1 << 0)	/* iclog needs REQ_PREFLUSH */
+#define XLOG_ICL_NEED_FUA	(1 << 1)	/* iclog needs REQ_FUA */
+
 /* Ticket reservation region accounting */ 
 #define XLOG_TIC_LEN_MAX	15
 
@@ -201,6 +204,7 @@ typedef struct xlog_in_core {
 	u32			ic_size;
 	u32			ic_offset;
 	enum xlog_iclog_state	ic_state;
+	unsigned int		ic_flags;
 	char			*ic_datap;	/* pointer to iclog data */
 
 	/* Callback structures need their own cacheline */
#45 lore raw

Re: [PATCH 7/8 v2] xfs: journal IO cache flush reductions

Quoted 56 lines · Dave Chinner

From: Dave Chinner <[email protected]>

Currently every journal IO is issued as REQ_PREFLUSH | REQ_FUA to guarantee the ordering requirements the journal has w.r.t. metadata writeback. THe two ordering constraints are:

1. we cannot overwrite metadata in the journal until we guarantee that the dirty metadata has been written back in place and is stable.

2. we cannot write back dirty metadata until it has been written to the journal and guaranteed to be stable (and hence recoverable) in the journal.

The ordering guarantees of #1 are provided by REQ_PREFLUSH. This causes the journal IO to issue a cache flush and wait for it to complete before issuing the write IO to the journal. Hence all completed metadata IO is guaranteed to be stable before the journal overwrites the old metadata.

The ordering guarantees of #2 are provided by the REQ_FUA, which ensures the journal writes do not complete until they are on stable storage. Hence by the time the last journal IO in a checkpoint completes, we know that the entire checkpoint is on stable storage and we can unpin the dirty metadata and allow it to be written back.

This is the mechanism by which ordering was first implemented in XFS way back in 2002 by this commit:

commit 95d97c36e5155075ba2eb22b17562cfcc53fcf96

Author: Steve Lord <[email protected]>
Date:   Fri May 24 14:30:21 2002 +0000

Add support for drive write cache flushing - should the kernel have the infrastructure

A lot has changed since then, most notably we now use delayed logging to checkpoint the filesystem to the journal rather than write each individual transaction to the journal. Cache flushes on journal IO are necessary when individual transactions are wholly contained within a single iclog. However, CIL checkpoints are single transactions that typically span hundreds to thousands of individual journal writes, and so the requirements for device cache flushing have changed.

That is, the ordering rules I state above apply to ordering of atomic transactions recorded in the journal, not to the journal IO itself. Hence we need to ensure metadata is stable before we start writing a new transaction to the journal (guarantee #1), and we need to ensure the entire transaction is stable in the journal before we start metadata writeback (guarantee #2).

Hence we only need a REQ_PREFLUSH on the journal IO that starts a new journal transaction to provide #1, and it is not on any other journal IO done within the context of that journal transaction.

Quoted 8 lines

The CIL checkpoint already issues a cache flush before it starts writing to the log, so we no longer need the iclog IO to issue a REQ_REFLUSH for us. Hence if XLOG_START_TRANS is passed to xlog_write(), we no longer need to mark the first iclog in the log write with REQ_PREFLUSH for this case.

Given the new ordering semantics of commit records for the CIL, we need iclogs containing commit to issue a REQ_PREFLUSH. We also

We flush the data device before writing the first iclog (containing XLOG_START_TRANS) to the disk. This satisfies the first ordering constraint listed above. Why is it required to have another REQ_PREFLUSH when writing the iclog containing XLOG_COMMIT_TRANS? I am guessing that it is required to make sure that the previous iclogs (belonging to the same checkpoint transaction) have indeed been written to the disk.

Quoted 186 lines

require unmount records to do this. Hence for both XLOG_COMMIT_TRANS and XLOG_UNMOUNT_TRANS xlog_write() calls we need to mark the first iclog being written with REQ_PREFLUSH.

For both commit records and unmount records, we also want them immediately on stable storage, so we want to also mark the iclogs that contain these records to be marked REQ_FUA. That means if a record is split across multiple iclogs, they are all marked REQ_FUA and not just the last one so that when the transaction is completed all the parts of the record are on stable storage.

As an optimisation, when the commit record lands in the same iclog as the journal transaction starts, we don't need to wait for anything and can simply use REQ_FUA to provide guarantee #2. This means that for fsync() heavy workloads, the cache flush behaviour is completely unchanged and there is no degradation in performance as a result of optimise the multi-IO transaction case.

The most notable sign that there is less IO latency on my test machine (nvme SSDs) is that the "noiclogs" rate has dropped substantially. This metric indicates that the CIL push is blocking in xlog_get_iclog_space() waiting for iclog IO completion to occur. With 8 iclogs of 256kB, the rate is appoximately 1 noiclog event to every 4 iclog writes. IOWs, every 4th call to xlog_get_iclog_space() is blocking waiting for log IO. With the changes in this patch, this drops to 1 noiclog event for every 100 iclog writes. Hence it is clear that log IO is completing much faster than it was previously, but it is also clear that for large iclog sizes, this isn't the performance limiting factor on this hardware.

With smaller iclogs (32kB), however, there is a sustantial difference. With the cache flush modifications, the journal is now running at over 4000 write IOPS, and the journal throughput is largely identical to the 256kB iclogs and the noiclog event rate stays low at about 1:50 iclog writes. The existing code tops out at about 2500 IOPS as the number of cache flushes dominate performance and latency. The noiclog event rate is about 1:4, and the performance variance is quite large as the journal throughput can fall to less than half the peak sustained rate when the cache flush rate prevents metadata writeback from keeping up and the log runs out of space and throttles reservations.

As a result:

logbsize fsmark create rate rm -rf
before 32kb 152851+/-5.3e+04 5m28s
patched 32kb 221533+/-1.1e+04 5m24s

before 256kb 220239+/-6.2e+03 4m58s
patched 256kb 228286+/-9.2e+03 5m06s

The rm -rf times are included because I ran them, but the differences are largely noise. This workload is largely metadata read IO latency bound and the changes to the journal cache flushing doesn't really make any noticable difference to behaviour apart from a reduction in noiclog events from background CIL pushing.

---
Version 2:
- repost manually without git/guilt mangling the patch author
- fix bug in XLOG_ICL_NEED_FUA definition that didn't manifest as an ordering bug in generic/45[57] until testing the CIL pipelining changes much later in the series.

 fs/xfs/xfs_log.c      | 33 +++++++++++++++++++++++----------
 fs/xfs/xfs_log_cil.c  |  7 ++++++-
 fs/xfs/xfs_log_priv.h |  4 ++++
 3 files changed, 33 insertions(+), 11 deletions(-)
fs/xfs/xfs_log.c +23 -10
index 6c3fb6dcb505..08d68a6161ae 100644
@@ -1806,8 +1806,7 @@ xlog_write_iclog(
 	struct xlog		*log,
 	struct xlog_in_core	*iclog,
 	uint64_t		bno,
-	unsigned int		count,
-	bool			need_flush)
+	unsigned int		count)
 {
 	ASSERT(bno < log->l_logBBsize);
 
@@ -1845,10 +1844,12 @@ xlog_write_iclog(
 	 * writeback throttle from throttling log writes behind background
 	 * metadata writeback and causing priority inversions.
 	 */
-	iclog->ic_bio.bi_opf = REQ_OP_WRITE | REQ_META | REQ_SYNC |
-				REQ_IDLE | REQ_FUA;
-	if (need_flush)
+	iclog->ic_bio.bi_opf = REQ_OP_WRITE | REQ_META | REQ_SYNC | REQ_IDLE;
+	if (iclog->ic_flags & XLOG_ICL_NEED_FLUSH)
 		iclog->ic_bio.bi_opf |= REQ_PREFLUSH;
+	if (iclog->ic_flags & XLOG_ICL_NEED_FUA)
+		iclog->ic_bio.bi_opf |= REQ_FUA;
+	iclog->ic_flags &= ~(XLOG_ICL_NEED_FLUSH | XLOG_ICL_NEED_FUA);
 
 	if (xlog_map_iclog_data(&iclog->ic_bio, iclog->ic_data, count)) {
 		xfs_force_shutdown(log->l_mp, SHUTDOWN_LOG_IO_ERROR);
@@ -1951,7 +1952,7 @@ xlog_sync(
 	unsigned int		roundoff;       /* roundoff to BB or stripe */
 	uint64_t		bno;
 	unsigned int		size;
-	bool			need_flush = true, split = false;
+	bool			split = false;
 
 	ASSERT(atomic_read(&iclog->ic_refcnt) == 0);
 
@@ -2009,13 +2010,14 @@ xlog_sync(
 	 * synchronously here; for an internal log we can simply use the block
 	 * layer state machine for preflushes.
 	 */
-	if (log->l_targ != log->l_mp->m_ddev_targp || split) {
+	if (log->l_targ != log->l_mp->m_ddev_targp ||
+	    (split && (iclog->ic_flags & XLOG_ICL_NEED_FLUSH))) {
 		xfs_flush_bdev(log->l_mp->m_ddev_targp->bt_bdev);
-		need_flush = false;
+		iclog->ic_flags &= ~XLOG_ICL_NEED_FLUSH;
 	}
 
 	xlog_verify_iclog(log, iclog, count);
-	xlog_write_iclog(log, iclog, bno, count, need_flush);
+	xlog_write_iclog(log, iclog, bno, count);
 }
 
 /*
@@ -2469,10 +2471,21 @@ xlog_write(
 		ASSERT(log_offset <= iclog->ic_size - 1);
 		ptr = iclog->ic_datap + log_offset;
 
-		/* start_lsn is the first lsn written to. That's all we need. */
+		/* Start_lsn is the first lsn written to. */
 		if (start_lsn && !*start_lsn)
 			*start_lsn = be64_to_cpu(iclog->ic_header.h_lsn);
 
+		/*
+		 * iclogs containing commit records or unmount records need
+		 * to issue ordering cache flushes and commit immediately
+		 * to stable storage to guarantee journal vs metadata ordering
+		 * is correctly maintained in the storage media.
+		 */
+		if (optype & (XLOG_COMMIT_TRANS | XLOG_UNMOUNT_TRANS)) {
+			iclog->ic_flags |= (XLOG_ICL_NEED_FLUSH |
+						XLOG_ICL_NEED_FUA);
+		}
+
 		/*
 		 * This loop writes out as many regions as can fit in the amount
 		 * of space which was allocated by xlog_state_get_iclog_space().
fs/xfs/xfs_log_cil.c +6 -1
index 4093d2d0db7c..370da7c2bfc8 100644
@@ -894,10 +894,15 @@ xlog_cil_push_work(
 
 	/*
 	 * If the checkpoint spans multiple iclogs, wait for all previous
-	 * iclogs to complete before we submit the commit_iclog.
+	 * iclogs to complete before we submit the commit_iclog. If it is in the
+	 * same iclog as the start of the checkpoint, then we can skip the iclog
+	 * cache flush because there are no other iclogs we need to order
+	 * against.
 	 */
 	if (ctx->start_lsn != commit_lsn)
 		xlog_wait_on_iclog_lsn(commit_iclog, ctx->start_lsn);
+	else
+		commit_iclog->ic_flags &= ~XLOG_ICL_NEED_FLUSH;
 
 	/* release the hounds! */
 	xfs_log_release_iclog(commit_iclog);
fs/xfs/xfs_log_priv.h +3 -0
index 10a41b1dd895..a77e00b7789a 100644
@@ -133,6 +133,9 @@ enum xlog_iclog_state {
 
 #define XLOG_COVER_OPS		5
 
+#define XLOG_ICL_NEED_FLUSH	(1 << 0)	/* iclog needs REQ_PREFLUSH */
+#define XLOG_ICL_NEED_FUA	(1 << 1)	/* iclog needs REQ_FUA */
+
 /* Ticket reservation region accounting */
 #define XLOG_TIC_LEN_MAX	15
 

fs/xfs/xfs_log_priv.h typedef struct xlog_in_core { · line 210

Quoted 7 lines
fs/xfs/xfs_log_priv.h +1 -0
@@ -201,6 +204,7 @@ typedef struct xlog_in_core {
 	u32			ic_size;
 	u32			ic_offset;
 	enum xlog_iclog_state	ic_state;
+	unsigned int		ic_flags;
 	char			*ic_datap;	/* pointer to iclog data */
 
 	/* Callback structures need their own cacheline */

--
chandan

#46 lore raw

Re: [PATCH 7/8 v2] xfs: journal IO cache flush reductions

Quoted 66 lines · Chandan Babu R
Quoted 56 lines · Dave Chinner

From: Dave Chinner <[email protected]>

Currently every journal IO is issued as REQ_PREFLUSH | REQ_FUA to guarantee the ordering requirements the journal has w.r.t. metadata writeback. THe two ordering constraints are:

1. we cannot overwrite metadata in the journal until we guarantee that the dirty metadata has been written back in place and is stable.

2. we cannot write back dirty metadata until it has been written to the journal and guaranteed to be stable (and hence recoverable) in the journal.

The ordering guarantees of #1 are provided by REQ_PREFLUSH. This causes the journal IO to issue a cache flush and wait for it to complete before issuing the write IO to the journal. Hence all completed metadata IO is guaranteed to be stable before the journal overwrites the old metadata.

The ordering guarantees of #2 are provided by the REQ_FUA, which ensures the journal writes do not complete until they are on stable storage. Hence by the time the last journal IO in a checkpoint completes, we know that the entire checkpoint is on stable storage and we can unpin the dirty metadata and allow it to be written back.

This is the mechanism by which ordering was first implemented in XFS way back in 2002 by this commit:

commit 95d97c36e5155075ba2eb22b17562cfcc53fcf96

Author: Steve Lord <[email protected]>
Date:   Fri May 24 14:30:21 2002 +0000

Add support for drive write cache flushing - should the kernel have the infrastructure

A lot has changed since then, most notably we now use delayed logging to checkpoint the filesystem to the journal rather than write each individual transaction to the journal. Cache flushes on journal IO are necessary when individual transactions are wholly contained within a single iclog. However, CIL checkpoints are single transactions that typically span hundreds to thousands of individual journal writes, and so the requirements for device cache flushing have changed.

That is, the ordering rules I state above apply to ordering of atomic transactions recorded in the journal, not to the journal IO itself. Hence we need to ensure metadata is stable before we start writing a new transaction to the journal (guarantee #1), and we need to ensure the entire transaction is stable in the journal before we start metadata writeback (guarantee #2).

Hence we only need a REQ_PREFLUSH on the journal IO that starts a new journal transaction to provide #1, and it is not on any other journal IO done within the context of that journal transaction.

Quoted 8 lines

The CIL checkpoint already issues a cache flush before it starts writing to the log, so we no longer need the iclog IO to issue a REQ_REFLUSH for us. Hence if XLOG_START_TRANS is passed to xlog_write(), we no longer need to mark the first iclog in the log write with REQ_PREFLUSH for this case.

Given the new ordering semantics of commit records for the CIL, we need iclogs containing commit to issue a REQ_PREFLUSH. We also

Quoted 6 lines

We flush the data device before writing the first iclog (containing XLOG_START_TRANS) to the disk. This satisfies the first ordering constraint listed above. Why is it required to have another REQ_PREFLUSH when writing the iclog containing XLOG_COMMIT_TRANS? I am guessing that it is required to make sure that the previous iclogs (belonging to the same checkpoint transaction) have indeed been written to the disk.

Yes, that is correct.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
Reviewed-by on v2 7/8
#47 lore raw

Re: [PATCH 7/8 v2] xfs: journal IO cache flush reductions

In reply to Dave Chinner
Quoted 115 lines · Dave Chinner

From: Dave Chinner <[email protected]>

Currently every journal IO is issued as REQ_PREFLUSH | REQ_FUA to guarantee the ordering requirements the journal has w.r.t. metadata writeback. THe two ordering constraints are:

1. we cannot overwrite metadata in the journal until we guarantee that the dirty metadata has been written back in place and is stable.

2. we cannot write back dirty metadata until it has been written to the journal and guaranteed to be stable (and hence recoverable) in the journal.

The ordering guarantees of #1 are provided by REQ_PREFLUSH. This causes the journal IO to issue a cache flush and wait for it to complete before issuing the write IO to the journal. Hence all completed metadata IO is guaranteed to be stable before the journal overwrites the old metadata.

The ordering guarantees of #2 are provided by the REQ_FUA, which ensures the journal writes do not complete until they are on stable storage. Hence by the time the last journal IO in a checkpoint completes, we know that the entire checkpoint is on stable storage and we can unpin the dirty metadata and allow it to be written back.

This is the mechanism by which ordering was first implemented in XFS way back in 2002 by this commit:

commit 95d97c36e5155075ba2eb22b17562cfcc53fcf96

Author: Steve Lord <[email protected]>
Date:   Fri May 24 14:30:21 2002 +0000

Add support for drive write cache flushing - should the kernel have the infrastructure

A lot has changed since then, most notably we now use delayed logging to checkpoint the filesystem to the journal rather than write each individual transaction to the journal. Cache flushes on journal IO are necessary when individual transactions are wholly contained within a single iclog. However, CIL checkpoints are single transactions that typically span hundreds to thousands of individual journal writes, and so the requirements for device cache flushing have changed.

That is, the ordering rules I state above apply to ordering of atomic transactions recorded in the journal, not to the journal IO itself. Hence we need to ensure metadata is stable before we start writing a new transaction to the journal (guarantee #1), and we need to ensure the entire transaction is stable in the journal before we start metadata writeback (guarantee #2).

Hence we only need a REQ_PREFLUSH on the journal IO that starts a new journal transaction to provide #1, and it is not on any other journal IO done within the context of that journal transaction.

The CIL checkpoint already issues a cache flush before it starts writing to the log, so we no longer need the iclog IO to issue a REQ_REFLUSH for us. Hence if XLOG_START_TRANS is passed to xlog_write(), we no longer need to mark the first iclog in the log write with REQ_PREFLUSH for this case.

Given the new ordering semantics of commit records for the CIL, we need iclogs containing commit to issue a REQ_PREFLUSH. We also require unmount records to do this. Hence for both XLOG_COMMIT_TRANS and XLOG_UNMOUNT_TRANS xlog_write() calls we need to mark the first iclog being written with REQ_PREFLUSH.

For both commit records and unmount records, we also want them immediately on stable storage, so we want to also mark the iclogs that contain these records to be marked REQ_FUA. That means if a record is split across multiple iclogs, they are all marked REQ_FUA and not just the last one so that when the transaction is completed all the parts of the record are on stable storage.

As an optimisation, when the commit record lands in the same iclog as the journal transaction starts, we don't need to wait for anything and can simply use REQ_FUA to provide guarantee #2. This means that for fsync() heavy workloads, the cache flush behaviour is completely unchanged and there is no degradation in performance as a result of optimise the multi-IO transaction case.

The most notable sign that there is less IO latency on my test machine (nvme SSDs) is that the "noiclogs" rate has dropped substantially. This metric indicates that the CIL push is blocking in xlog_get_iclog_space() waiting for iclog IO completion to occur. With 8 iclogs of 256kB, the rate is appoximately 1 noiclog event to every 4 iclog writes. IOWs, every 4th call to xlog_get_iclog_space() is blocking waiting for log IO. With the changes in this patch, this drops to 1 noiclog event for every 100 iclog writes. Hence it is clear that log IO is completing much faster than it was previously, but it is also clear that for large iclog sizes, this isn't the performance limiting factor on this hardware.

With smaller iclogs (32kB), however, there is a sustantial difference. With the cache flush modifications, the journal is now running at over 4000 write IOPS, and the journal throughput is largely identical to the 256kB iclogs and the noiclog event rate stays low at about 1:50 iclog writes. The existing code tops out at about 2500 IOPS as the number of cache flushes dominate performance and latency. The noiclog event rate is about 1:4, and the performance variance is quite large as the journal throughput can fall to less than half the peak sustained rate when the cache flush rate prevents metadata writeback from keeping up and the log runs out of space and throttles reservations.

As a result:

logbsize fsmark create rate rm -rf
before 32kb 152851+/-5.3e+04 5m28s
patched 32kb 221533+/-1.1e+04 5m24s

before 256kb 220239+/-6.2e+03 4m58s
patched 256kb 228286+/-9.2e+03 5m06s

Quoted 7 lines

The rm -rf times are included because I ran them, but the differences are largely noise. This workload is largely metadata read IO latency bound and the changes to the journal cache flushing doesn't really make any noticable difference to behaviour apart from a reduction in noiclog events from background CIL pushing.

If I've gotten this right so far, patch 4 introduces the ability to flush the data device while we get ready to overwrite parts of the log. This patch makes it so that writing a commit (or unmount) record to an iclog causes that iclog write to be issued (to the log device) with at least FUA set, and possibly PREFLUSH if we've written out more than one iclog since the last commit?

IOWs, the data device flush is now asynchronous with CIL processing and we've ripped out all the cache flushes and FUA writes for iclogs except for the last one in a chain, which should maintain data integrity requirements 1 and 2?

If that's correct,

--D

Quoted 135 lines

---
Version 2:
- repost manually without git/guilt mangling the patch author
- fix bug in XLOG_ICL_NEED_FUA definition that didn't manifest as an ordering bug in generic/45[57] until testing the CIL pipelining changes much later in the series.

 fs/xfs/xfs_log.c      | 33 +++++++++++++++++++++++----------
 fs/xfs/xfs_log_cil.c  |  7 ++++++-
 fs/xfs/xfs_log_priv.h |  4 ++++
 3 files changed, 33 insertions(+), 11 deletions(-)
fs/xfs/xfs_log.c +23 -10
index 6c3fb6dcb505..08d68a6161ae 100644
@@ -1806,8 +1806,7 @@ xlog_write_iclog(
 	struct xlog		*log,
 	struct xlog_in_core	*iclog,
 	uint64_t		bno,
-	unsigned int		count,
-	bool			need_flush)
+	unsigned int		count)
 {
 	ASSERT(bno < log->l_logBBsize);
 
@@ -1845,10 +1844,12 @@ xlog_write_iclog(
 	 * writeback throttle from throttling log writes behind background
 	 * metadata writeback and causing priority inversions.
 	 */
-	iclog->ic_bio.bi_opf = REQ_OP_WRITE | REQ_META | REQ_SYNC |
-				REQ_IDLE | REQ_FUA;
-	if (need_flush)
+	iclog->ic_bio.bi_opf = REQ_OP_WRITE | REQ_META | REQ_SYNC | REQ_IDLE;
+	if (iclog->ic_flags & XLOG_ICL_NEED_FLUSH)
 		iclog->ic_bio.bi_opf |= REQ_PREFLUSH;
+	if (iclog->ic_flags & XLOG_ICL_NEED_FUA)
+		iclog->ic_bio.bi_opf |= REQ_FUA;
+	iclog->ic_flags &= ~(XLOG_ICL_NEED_FLUSH | XLOG_ICL_NEED_FUA);
 
 	if (xlog_map_iclog_data(&iclog->ic_bio, iclog->ic_data, count)) {
 		xfs_force_shutdown(log->l_mp, SHUTDOWN_LOG_IO_ERROR);
@@ -1951,7 +1952,7 @@ xlog_sync(
 	unsigned int		roundoff;       /* roundoff to BB or stripe */
 	uint64_t		bno;
 	unsigned int		size;
-	bool			need_flush = true, split = false;
+	bool			split = false;
 
 	ASSERT(atomic_read(&iclog->ic_refcnt) == 0);
 
@@ -2009,13 +2010,14 @@ xlog_sync(
 	 * synchronously here; for an internal log we can simply use the block
 	 * layer state machine for preflushes.
 	 */
-	if (log->l_targ != log->l_mp->m_ddev_targp || split) {
+	if (log->l_targ != log->l_mp->m_ddev_targp ||
+	    (split && (iclog->ic_flags & XLOG_ICL_NEED_FLUSH))) {
 		xfs_flush_bdev(log->l_mp->m_ddev_targp->bt_bdev);
-		need_flush = false;
+		iclog->ic_flags &= ~XLOG_ICL_NEED_FLUSH;
 	}
 
 	xlog_verify_iclog(log, iclog, count);
-	xlog_write_iclog(log, iclog, bno, count, need_flush);
+	xlog_write_iclog(log, iclog, bno, count);
 }
 
 /*
@@ -2469,10 +2471,21 @@ xlog_write(
 		ASSERT(log_offset <= iclog->ic_size - 1);
 		ptr = iclog->ic_datap + log_offset;
 
-		/* start_lsn is the first lsn written to. That's all we need. */
+		/* Start_lsn is the first lsn written to. */
 		if (start_lsn && !*start_lsn)
 			*start_lsn = be64_to_cpu(iclog->ic_header.h_lsn);
 
+		/*
+		 * iclogs containing commit records or unmount records need
+		 * to issue ordering cache flushes and commit immediately
+		 * to stable storage to guarantee journal vs metadata ordering
+		 * is correctly maintained in the storage media.
+		 */
+		if (optype & (XLOG_COMMIT_TRANS | XLOG_UNMOUNT_TRANS)) {
+			iclog->ic_flags |= (XLOG_ICL_NEED_FLUSH |
+						XLOG_ICL_NEED_FUA);
+		}
+
 		/*
 		 * This loop writes out as many regions as can fit in the amount
 		 * of space which was allocated by xlog_state_get_iclog_space().
fs/xfs/xfs_log_cil.c +6 -1
index 4093d2d0db7c..370da7c2bfc8 100644
@@ -894,10 +894,15 @@ xlog_cil_push_work(
 
 	/*
 	 * If the checkpoint spans multiple iclogs, wait for all previous
-	 * iclogs to complete before we submit the commit_iclog.
+	 * iclogs to complete before we submit the commit_iclog. If it is in the
+	 * same iclog as the start of the checkpoint, then we can skip the iclog
+	 * cache flush because there are no other iclogs we need to order
+	 * against.
 	 */
 	if (ctx->start_lsn != commit_lsn)
 		xlog_wait_on_iclog_lsn(commit_iclog, ctx->start_lsn);
+	else
+		commit_iclog->ic_flags &= ~XLOG_ICL_NEED_FLUSH;
 
 	/* release the hounds! */
 	xfs_log_release_iclog(commit_iclog);
fs/xfs/xfs_log_priv.h +4 -0
index 10a41b1dd895..a77e00b7789a 100644
@@ -133,6 +133,9 @@ enum xlog_iclog_state {
 
 #define XLOG_COVER_OPS		5
 
+#define XLOG_ICL_NEED_FLUSH	(1 << 0)	/* iclog needs REQ_PREFLUSH */
+#define XLOG_ICL_NEED_FUA	(1 << 1)	/* iclog needs REQ_FUA */
+
 /* Ticket reservation region accounting */ 
 #define XLOG_TIC_LEN_MAX	15
 
@@ -201,6 +204,7 @@ typedef struct xlog_in_core {
 	u32			ic_size;
 	u32			ic_offset;
 	enum xlog_iclog_state	ic_state;
+	unsigned int		ic_flags;
 	char			*ic_datap;	/* pointer to iclog data */
 
 	/* Callback structures need their own cacheline */
#48 lore raw

Re: [PATCH 7/8 v2] xfs: journal IO cache flush reductions

Quoted 26 lines · Darrick J. Wong
Quoted 25 lines · Dave Chinner

From: Dave Chinner <[email protected]>

Currently every journal IO is issued as REQ_PREFLUSH | REQ_FUA to guarantee the ordering requirements the journal has w.r.t. metadata writeback. THe two ordering constraints are:

1. we cannot overwrite metadata in the journal until we guarantee that the dirty metadata has been written back in place and is stable.

2. we cannot write back dirty metadata until it has been written to the journal and guaranteed to be stable (and hence recoverable) in the journal.

The ordering guarantees of #1 are provided by REQ_PREFLUSH. This causes the journal IO to issue a cache flush and wait for it to complete before issuing the write IO to the journal. Hence all completed metadata IO is guaranteed to be stable before the journal overwrites the old metadata.

The ordering guarantees of #2 are provided by the REQ_FUA, which ensures the journal writes do not complete until they are on stable storage. Hence by the time the last journal IO in a checkpoint completes, we know that the entire checkpoint is on stable storage and we can unpin the dirty metadata and allow it to be written back.

....

Quoted 2 lines

If I've gotten this right so far, patch 4 introduces the ability to flush the data device while we get ready to overwrite parts of the log.

Yes.

Quoted 4 lines

This patch makes it so that writing a commit (or unmount) record to an iclog causes that iclog write to be issued (to the log device) with at least FUA set, and possibly PREFLUSH if we've written out more than one iclog since the last commit?

Yes.

Quoted 4 lines

IOWs, the data device flush is now asynchronous with CIL processing and we've ripped out all the cache flushes and FUA writes for iclogs except for the last one in a chain, which should maintain data integrity requirements 1 and 2?

Yes.

Quoted 2 lines

If that's correct,

Thanks!

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#49 lore raw

Re: [PATCH 7/8 v2] xfs: journal IO cache flush reductions

In reply to Dave Chinner
Quoted 174 lines · Dave Chinner

From: Dave Chinner <[email protected]>

Currently every journal IO is issued as REQ_PREFLUSH | REQ_FUA to guarantee the ordering requirements the journal has w.r.t. metadata writeback. THe two ordering constraints are:

1. we cannot overwrite metadata in the journal until we guarantee that the dirty metadata has been written back in place and is stable.

2. we cannot write back dirty metadata until it has been written to the journal and guaranteed to be stable (and hence recoverable) in the journal.

The ordering guarantees of #1 are provided by REQ_PREFLUSH. This causes the journal IO to issue a cache flush and wait for it to complete before issuing the write IO to the journal. Hence all completed metadata IO is guaranteed to be stable before the journal overwrites the old metadata.

The ordering guarantees of #2 are provided by the REQ_FUA, which ensures the journal writes do not complete until they are on stable storage. Hence by the time the last journal IO in a checkpoint completes, we know that the entire checkpoint is on stable storage and we can unpin the dirty metadata and allow it to be written back.

This is the mechanism by which ordering was first implemented in XFS way back in 2002 by this commit:

commit 95d97c36e5155075ba2eb22b17562cfcc53fcf96

Author: Steve Lord <[email protected]>
Date:   Fri May 24 14:30:21 2002 +0000

Add support for drive write cache flushing - should the kernel have the infrastructure

A lot has changed since then, most notably we now use delayed logging to checkpoint the filesystem to the journal rather than write each individual transaction to the journal. Cache flushes on journal IO are necessary when individual transactions are wholly contained within a single iclog. However, CIL checkpoints are single transactions that typically span hundreds to thousands of individual journal writes, and so the requirements for device cache flushing have changed.

That is, the ordering rules I state above apply to ordering of atomic transactions recorded in the journal, not to the journal IO itself. Hence we need to ensure metadata is stable before we start writing a new transaction to the journal (guarantee #1), and we need to ensure the entire transaction is stable in the journal before we start metadata writeback (guarantee #2).

Hence we only need a REQ_PREFLUSH on the journal IO that starts a new journal transaction to provide #1, and it is not on any other journal IO done within the context of that journal transaction.

The CIL checkpoint already issues a cache flush before it starts writing to the log, so we no longer need the iclog IO to issue a REQ_REFLUSH for us. Hence if XLOG_START_TRANS is passed to xlog_write(), we no longer need to mark the first iclog in the log write with REQ_PREFLUSH for this case.

Given the new ordering semantics of commit records for the CIL, we need iclogs containing commit to issue a REQ_PREFLUSH. We also require unmount records to do this. Hence for both XLOG_COMMIT_TRANS and XLOG_UNMOUNT_TRANS xlog_write() calls we need to mark the first iclog being written with REQ_PREFLUSH.

For both commit records and unmount records, we also want them immediately on stable storage, so we want to also mark the iclogs that contain these records to be marked REQ_FUA. That means if a record is split across multiple iclogs, they are all marked REQ_FUA and not just the last one so that when the transaction is completed all the parts of the record are on stable storage.

As an optimisation, when the commit record lands in the same iclog as the journal transaction starts, we don't need to wait for anything and can simply use REQ_FUA to provide guarantee #2. This means that for fsync() heavy workloads, the cache flush behaviour is completely unchanged and there is no degradation in performance as a result of optimise the multi-IO transaction case.

The most notable sign that there is less IO latency on my test machine (nvme SSDs) is that the "noiclogs" rate has dropped substantially. This metric indicates that the CIL push is blocking in xlog_get_iclog_space() waiting for iclog IO completion to occur. With 8 iclogs of 256kB, the rate is appoximately 1 noiclog event to every 4 iclog writes. IOWs, every 4th call to xlog_get_iclog_space() is blocking waiting for log IO. With the changes in this patch, this drops to 1 noiclog event for every 100 iclog writes. Hence it is clear that log IO is completing much faster than it was previously, but it is also clear that for large iclog sizes, this isn't the performance limiting factor on this hardware.

With smaller iclogs (32kB), however, there is a sustantial difference. With the cache flush modifications, the journal is now running at over 4000 write IOPS, and the journal throughput is largely identical to the 256kB iclogs and the noiclog event rate stays low at about 1:50 iclog writes. The existing code tops out at about 2500 IOPS as the number of cache flushes dominate performance and latency. The noiclog event rate is about 1:4, and the performance variance is quite large as the journal throughput can fall to less than half the peak sustained rate when the cache flush rate prevents metadata writeback from keeping up and the log runs out of space and throttles reservations.

As a result:

logbsize fsmark create rate rm -rf
before 32kb 152851+/-5.3e+04 5m28s
patched 32kb 221533+/-1.1e+04 5m24s

before 256kb 220239+/-6.2e+03 4m58s
patched 256kb 228286+/-9.2e+03 5m06s

The rm -rf times are included because I ran them, but the differences are largely noise. This workload is largely metadata read IO latency bound and the changes to the journal cache flushing doesn't really make any noticable difference to behaviour apart from a reduction in noiclog events from background CIL pushing.

---
Version 2:
- repost manually without git/guilt mangling the patch author
- fix bug in XLOG_ICL_NEED_FUA definition that didn't manifest as an ordering bug in generic/45[57] until testing the CIL pipelining changes much later in the series.

 fs/xfs/xfs_log.c      | 33 +++++++++++++++++++++++----------
 fs/xfs/xfs_log_cil.c  |  7 ++++++-
 fs/xfs/xfs_log_priv.h |  4 ++++
 3 files changed, 33 insertions(+), 11 deletions(-)
fs/xfs/xfs_log.c +7 -6
index 6c3fb6dcb505..08d68a6161ae 100644
@@ -1806,8 +1806,7 @@ xlog_write_iclog(
 	struct xlog		*log,
 	struct xlog_in_core	*iclog,
 	uint64_t		bno,
-	unsigned int		count,
-	bool			need_flush)
+	unsigned int		count)
 {
 	ASSERT(bno < log->l_logBBsize);
 
@@ -1845,10 +1844,12 @@ xlog_write_iclog(
 	 * writeback throttle from throttling log writes behind background
 	 * metadata writeback and causing priority inversions.
 	 */
-	iclog->ic_bio.bi_opf = REQ_OP_WRITE | REQ_META | REQ_SYNC |
-				REQ_IDLE | REQ_FUA;
-	if (need_flush)
+	iclog->ic_bio.bi_opf = REQ_OP_WRITE | REQ_META | REQ_SYNC | REQ_IDLE;
+	if (iclog->ic_flags & XLOG_ICL_NEED_FLUSH)
 		iclog->ic_bio.bi_opf |= REQ_PREFLUSH;
+	if (iclog->ic_flags & XLOG_ICL_NEED_FUA)
+		iclog->ic_bio.bi_opf |= REQ_FUA;
+	iclog->ic_flags &= ~(XLOG_ICL_NEED_FLUSH | XLOG_ICL_NEED_FUA);
 
 	if (xlog_map_iclog_data(&iclog->ic_bio, iclog->ic_data, count)) {
 		xfs_force_shutdown(log->l_mp, SHUTDOWN_LOG_IO_ERROR);
@@ -1951,7 +1952,7 @@ xlog_sync(
 	unsigned int		roundoff;       /* roundoff to BB or stripe */
 	uint64_t		bno;
 	unsigned int		size;
-	bool			need_flush = true, split = false;
+	bool			split = false;
 
 	ASSERT(atomic_read(&iclog->ic_refcnt) == 0);
 

fs/xfs/xfs_log.c xlog_sync( · line 2017

Quoted 10 lines
fs/xfs/xfs_log.c +3 -2
@@ -2009,13 +2010,14 @@ xlog_sync(
 	 * synchronously here; for an internal log we can simply use the block
 	 * layer state machine for preflushes.
 	 */
-	if (log->l_targ != log->l_mp->m_ddev_targp || split) {
+	if (log->l_targ != log->l_mp->m_ddev_targp ||
+	    (split && (iclog->ic_flags & XLOG_ICL_NEED_FLUSH))) {
 		xfs_flush_bdev(log->l_mp->m_ddev_targp->bt_bdev);
-		need_flush = false;
+		iclog->ic_flags &= ~XLOG_ICL_NEED_FLUSH;
 	}

If a checkpoint transaction spans across 2 or more iclogs and the log is stored on an external device, then the above would remove XLOG_ICL_NEED_FLUSH flag from iclog->ic_flags causing xlog_write_iclog() to include only REQ_FUA flag in the corresponding bio.

Documentation/block/writeback_cache_control.rst seems to suggest that REQ_FUA guarantees only that the data associated with the bio is stable on disk before I/O completion is signalled. So looks like REQ_PREFLUSH is required in this scenario.

--
chandan

#50 lore raw

Re: [PATCH 7/8 v2] xfs: journal IO cache flush reductions

Quoted 174 lines · Chandan Babu R
Quoted 173 lines · Dave Chinner

From: Dave Chinner <[email protected]>

Currently every journal IO is issued as REQ_PREFLUSH | REQ_FUA to guarantee the ordering requirements the journal has w.r.t. metadata writeback. THe two ordering constraints are:

1. we cannot overwrite metadata in the journal until we guarantee that the dirty metadata has been written back in place and is stable.

2. we cannot write back dirty metadata until it has been written to the journal and guaranteed to be stable (and hence recoverable) in the journal.

The ordering guarantees of #1 are provided by REQ_PREFLUSH. This causes the journal IO to issue a cache flush and wait for it to complete before issuing the write IO to the journal. Hence all completed metadata IO is guaranteed to be stable before the journal overwrites the old metadata.

The ordering guarantees of #2 are provided by the REQ_FUA, which ensures the journal writes do not complete until they are on stable storage. Hence by the time the last journal IO in a checkpoint completes, we know that the entire checkpoint is on stable storage and we can unpin the dirty metadata and allow it to be written back.

This is the mechanism by which ordering was first implemented in XFS way back in 2002 by this commit:

commit 95d97c36e5155075ba2eb22b17562cfcc53fcf96

Author: Steve Lord <[email protected]>
Date:   Fri May 24 14:30:21 2002 +0000

Add support for drive write cache flushing - should the kernel have the infrastructure

A lot has changed since then, most notably we now use delayed logging to checkpoint the filesystem to the journal rather than write each individual transaction to the journal. Cache flushes on journal IO are necessary when individual transactions are wholly contained within a single iclog. However, CIL checkpoints are single transactions that typically span hundreds to thousands of individual journal writes, and so the requirements for device cache flushing have changed.

That is, the ordering rules I state above apply to ordering of atomic transactions recorded in the journal, not to the journal IO itself. Hence we need to ensure metadata is stable before we start writing a new transaction to the journal (guarantee #1), and we need to ensure the entire transaction is stable in the journal before we start metadata writeback (guarantee #2).

Hence we only need a REQ_PREFLUSH on the journal IO that starts a new journal transaction to provide #1, and it is not on any other journal IO done within the context of that journal transaction.

The CIL checkpoint already issues a cache flush before it starts writing to the log, so we no longer need the iclog IO to issue a REQ_REFLUSH for us. Hence if XLOG_START_TRANS is passed to xlog_write(), we no longer need to mark the first iclog in the log write with REQ_PREFLUSH for this case.

Given the new ordering semantics of commit records for the CIL, we need iclogs containing commit to issue a REQ_PREFLUSH. We also require unmount records to do this. Hence for both XLOG_COMMIT_TRANS and XLOG_UNMOUNT_TRANS xlog_write() calls we need to mark the first iclog being written with REQ_PREFLUSH.

For both commit records and unmount records, we also want them immediately on stable storage, so we want to also mark the iclogs that contain these records to be marked REQ_FUA. That means if a record is split across multiple iclogs, they are all marked REQ_FUA and not just the last one so that when the transaction is completed all the parts of the record are on stable storage.

As an optimisation, when the commit record lands in the same iclog as the journal transaction starts, we don't need to wait for anything and can simply use REQ_FUA to provide guarantee #2. This means that for fsync() heavy workloads, the cache flush behaviour is completely unchanged and there is no degradation in performance as a result of optimise the multi-IO transaction case.

The most notable sign that there is less IO latency on my test machine (nvme SSDs) is that the "noiclogs" rate has dropped substantially. This metric indicates that the CIL push is blocking in xlog_get_iclog_space() waiting for iclog IO completion to occur. With 8 iclogs of 256kB, the rate is appoximately 1 noiclog event to every 4 iclog writes. IOWs, every 4th call to xlog_get_iclog_space() is blocking waiting for log IO. With the changes in this patch, this drops to 1 noiclog event for every 100 iclog writes. Hence it is clear that log IO is completing much faster than it was previously, but it is also clear that for large iclog sizes, this isn't the performance limiting factor on this hardware.

With smaller iclogs (32kB), however, there is a sustantial difference. With the cache flush modifications, the journal is now running at over 4000 write IOPS, and the journal throughput is largely identical to the 256kB iclogs and the noiclog event rate stays low at about 1:50 iclog writes. The existing code tops out at about 2500 IOPS as the number of cache flushes dominate performance and latency. The noiclog event rate is about 1:4, and the performance variance is quite large as the journal throughput can fall to less than half the peak sustained rate when the cache flush rate prevents metadata writeback from keeping up and the log runs out of space and throttles reservations.

As a result:

logbsize fsmark create rate rm -rf
before 32kb 152851+/-5.3e+04 5m28s
patched 32kb 221533+/-1.1e+04 5m24s

before 256kb 220239+/-6.2e+03 4m58s
patched 256kb 228286+/-9.2e+03 5m06s

The rm -rf times are included because I ran them, but the differences are largely noise. This workload is largely metadata read IO latency bound and the changes to the journal cache flushing doesn't really make any noticable difference to behaviour apart from a reduction in noiclog events from background CIL pushing.

---
Version 2:
- repost manually without git/guilt mangling the patch author
- fix bug in XLOG_ICL_NEED_FUA definition that didn't manifest as an ordering bug in generic/45[57] until testing the CIL pipelining changes much later in the series.

 fs/xfs/xfs_log.c      | 33 +++++++++++++++++++++++----------
 fs/xfs/xfs_log_cil.c  |  7 ++++++-
 fs/xfs/xfs_log_priv.h |  4 ++++
 3 files changed, 33 insertions(+), 11 deletions(-)
fs/xfs/xfs_log.c +7 -6
index 6c3fb6dcb505..08d68a6161ae 100644
@@ -1806,8 +1806,7 @@ xlog_write_iclog(
 	struct xlog		*log,
 	struct xlog_in_core	*iclog,
 	uint64_t		bno,
-	unsigned int		count,
-	bool			need_flush)
+	unsigned int		count)
 {
 	ASSERT(bno < log->l_logBBsize);
 
@@ -1845,10 +1844,12 @@ xlog_write_iclog(
 	 * writeback throttle from throttling log writes behind background
 	 * metadata writeback and causing priority inversions.
 	 */
-	iclog->ic_bio.bi_opf = REQ_OP_WRITE | REQ_META | REQ_SYNC |
-				REQ_IDLE | REQ_FUA;
-	if (need_flush)
+	iclog->ic_bio.bi_opf = REQ_OP_WRITE | REQ_META | REQ_SYNC | REQ_IDLE;
+	if (iclog->ic_flags & XLOG_ICL_NEED_FLUSH)
 		iclog->ic_bio.bi_opf |= REQ_PREFLUSH;
+	if (iclog->ic_flags & XLOG_ICL_NEED_FUA)
+		iclog->ic_bio.bi_opf |= REQ_FUA;
+	iclog->ic_flags &= ~(XLOG_ICL_NEED_FLUSH | XLOG_ICL_NEED_FUA);
 
 	if (xlog_map_iclog_data(&iclog->ic_bio, iclog->ic_data, count)) {
 		xfs_force_shutdown(log->l_mp, SHUTDOWN_LOG_IO_ERROR);
@@ -1951,7 +1952,7 @@ xlog_sync(
 	unsigned int		roundoff;       /* roundoff to BB or stripe */
 	uint64_t		bno;
 	unsigned int		size;
-	bool			need_flush = true, split = false;
+	bool			split = false;
 
 	ASSERT(atomic_read(&iclog->ic_refcnt) == 0);
 
Quoted 15 lines

fs/xfs/xfs_log.c xlog_sync( · line 2017

Quoted 11 lines
fs/xfs/xfs_log.c +3 -2
@@ -2009,13 +2010,14 @@ xlog_sync(
 	 * synchronously here; for an internal log we can simply use the block
 	 * layer state machine for preflushes.
 	 */
-	if (log->l_targ != log->l_mp->m_ddev_targp || split) {
+	if (log->l_targ != log->l_mp->m_ddev_targp ||
+	    (split && (iclog->ic_flags & XLOG_ICL_NEED_FLUSH))) {
 		xfs_flush_bdev(log->l_mp->m_ddev_targp->bt_bdev);
-		need_flush = false;
+		iclog->ic_flags &= ~XLOG_ICL_NEED_FLUSH;
 	}

If a checkpoint transaction spans across 2 or more iclogs and the log is stored on an external device, then the above would remove XLOG_ICL_NEED_FLUSH flag from iclog->ic_flags causing xlog_write_iclog() to include only REQ_FUA

... would remove XLOG_ICL_NEED_FLUSH flag from *commit iclog's* ic_flags

Quoted 6 lines

flag in the corresponding bio.

Documentation/block/writeback_cache_control.rst seems to suggest that REQ_FUA guarantees only that the data associated with the bio is stable on disk before I/O completion is signalled. So looks like REQ_PREFLUSH is required in this scenario.

Signature · 1 line
chandan
#51 lore raw

Re: [PATCH 7/8 v2] xfs: journal IO cache flush reductions

In reply to Chandan Babu R
Quoted 17 lines · Chandan Babu R

fs/xfs/xfs_log.c xlog_sync( · line 2017

Quoted 11 lines · Dave Chinner
fs/xfs/xfs_log.c +3 -2
@@ -2009,13 +2010,14 @@ xlog_sync(
 	 * synchronously here; for an internal log we can simply use the block
 	 * layer state machine for preflushes.
 	 */
-	if (log->l_targ != log->l_mp->m_ddev_targp || split) {
+	if (log->l_targ != log->l_mp->m_ddev_targp ||
+	    (split && (iclog->ic_flags & XLOG_ICL_NEED_FLUSH))) {
 		xfs_flush_bdev(log->l_mp->m_ddev_targp->bt_bdev);
-		need_flush = false;
+		iclog->ic_flags &= ~XLOG_ICL_NEED_FLUSH;
 	}

If a checkpoint transaction spans across 2 or more iclogs and the log is stored on an external device, then the above would remove XLOG_ICL_NEED_FLUSH flag from iclog->ic_flags causing xlog_write_iclog() to include only REQ_FUA flag in the corresponding bio.

Yup, good catch, this is a subtle change of behaviour only for external logs and only for the commit iclog that needs to flush the previous log writes to stable storage. I'll rework the logic here.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#52 lore raw

Re: [PATCH 7/8 v2] xfs: journal IO cache flush reductions

Quoted 19 lines · Dave Chinner
Quoted 17 lines · Chandan Babu R

fs/xfs/xfs_log.c xlog_sync( · line 2017

Quoted 11 lines · Dave Chinner
fs/xfs/xfs_log.c +3 -2
@@ -2009,13 +2010,14 @@ xlog_sync(
 	 * synchronously here; for an internal log we can simply use the block
 	 * layer state machine for preflushes.
 	 */
-	if (log->l_targ != log->l_mp->m_ddev_targp || split) {
+	if (log->l_targ != log->l_mp->m_ddev_targp ||
+	    (split && (iclog->ic_flags & XLOG_ICL_NEED_FLUSH))) {
 		xfs_flush_bdev(log->l_mp->m_ddev_targp->bt_bdev);
-		need_flush = false;
+		iclog->ic_flags &= ~XLOG_ICL_NEED_FLUSH;
 	}

If a checkpoint transaction spans across 2 or more iclogs and the log is stored on an external device, then the above would remove XLOG_ICL_NEED_FLUSH flag from iclog->ic_flags causing xlog_write_iclog() to include only REQ_FUA flag in the corresponding bio.

Quoted 3 lines

Yup, good catch, this is a subtle change of behaviour only for external logs and only for the commit iclog that needs to flush the previous log writes to stable storage. I'll rework the logic here.

And now that I think about it, we can simply remove this code if we put an explicit data device cache flush in the unmount recrod write code. The CIL has already guaranteed metadata vs journal ordering before we start writing the checkpoint, meaning the XLOG_ICL_NEED_FLUSH flag only has meaning for internal iclog write ordering, not external metadata ordering. ANd for the split, we can simply clear the REQ_PREFLUSH flag from the split bio before submitting it....

Much simpler and faster for external logs, too.

CHeers,

Dav.e

Signature · 2 lines
Dave Chinner
[email protected]
#53 lore raw

Re: [PATCH 7/8 v2] xfs: journal IO cache flush reductions

In reply to Dave Chinner
Quoted 6 lines

As a result:

logbsize fsmark create rate rm -rf
before 32kb 152851+/-5.3e+04 5m28s
patched 32kb 221533+/-1.1e+04 5m24s

Quoted 8 lines

before 256kb 220239+/-6.2e+03 4m58s
patched 256kb 228286+/-9.2e+03 5m06s

The rm -rf times are included because I ran them, but the differences are largely noise. This workload is largely metadata read IO latency bound and the changes to the journal cache flushing doesn't really make any noticable difference to behaviour apart from a reduction in noiclog events from background CIL pushing.

The 256b rm -rf case actually seems like a regression not in the noise here. Does this reproduce over multiple runs?

fs/xfs/xfs_log.c xlog_sync( · line 2016

Quoted 10 lines
fs/xfs/xfs_log.c +3 -2
@@ -2009,13 +2010,14 @@ xlog_sync(
 	 * synchronously here; for an internal log we can simply use the block
 	 * layer state machine for preflushes.
 	 */
-	if (log->l_targ != log->l_mp->m_ddev_targp || split) {
+	if (log->l_targ != log->l_mp->m_ddev_targp ||
+	    (split && (iclog->ic_flags & XLOG_ICL_NEED_FLUSH))) {
 		xfs_flush_bdev(log->l_mp->m_ddev_targp->bt_bdev);
-		need_flush = false;
+		iclog->ic_flags &= ~XLOG_ICL_NEED_FLUSH;

Once you touch all the buffer flags anyway we should optimize the log wraparound case here - insteaad of th synchronous flush we just need to set REQ_PREFLUSH on the first log bio, which should be nicely doable with your infrastruture.

Quoted 19 lines
fs/xfs/xfs_log.c +11 -0
+		/*
+		 * iclogs containing commit records or unmount records need
+		 * to issue ordering cache flushes and commit immediately
+		 * to stable storage to guarantee journal vs metadata ordering
+		 * is correctly maintained in the storage media.
+		 */
+		if (optype & (XLOG_COMMIT_TRANS | XLOG_UNMOUNT_TRANS)) {
+			iclog->ic_flags |= (XLOG_ICL_NEED_FLUSH |
+						XLOG_ICL_NEED_FUA);
+		}
+
 		/*
 		 * This loop writes out as many regions as can fit in the amount
 		 * of space which was allocated by xlog_state_get_iclog_space().
fs/xfs/xfs_log_cil.c +0 -0
index 4093d2d0db7c..370da7c2bfc8 100644

fs/xfs/xfs_log_cil.c xlog_cil_push_work( · line 900

Quoted 8 lines
fs/xfs/xfs_log_cil.c +4 -1
@@ -894,10 +894,15 @@ xlog_cil_push_work(
 
 	/*
 	 * If the checkpoint spans multiple iclogs, wait for all previous
-	 * iclogs to complete before we submit the commit_iclog.
+	 * iclogs to complete before we submit the commit_iclog. If it is in the
+	 * same iclog as the start of the checkpoint, then we can skip the iclog
+	 * cache flush because there are no other iclogs we need to order
+	 * against.

Nit: the iclogs in the first changed line would easily fit onto the previous line.

#54 lore raw

Re: [PATCH 7/8 v2] xfs: journal IO cache flush reductions

Quoted 9 lines · Christoph Hellwig
Quoted 9 lines

As a result:

logbsize fsmark create rate rm -rf
before 32kb 152851+/-5.3e+04 5m28s
patched 32kb 221533+/-1.1e+04 5m24s

before 256kb 220239+/-6.2e+03 4m58s
patched 256kb 228286+/-9.2e+03 5m06s

Quoted 8 lines
Quoted 5 lines

The rm -rf times are included because I ran them, but the differences are largely noise. This workload is largely metadata read IO latency bound and the changes to the journal cache flushing doesn't really make any noticable difference to behaviour apart from a reduction in noiclog events from background CIL pushing.

The 256b rm -rf case actually seems like a regression not in the noise here. Does this reproduce over multiple runs?

It's noise. The unlink repeat times on this machine at 16 threads are at least +/-15s because the removals are not synchronised in groups like the creates are.

These are CPU bound workloads when the log is not limiting the transaction rate (only the {before, 32kB} numbers in this test are log IO bound) so there's always some variation in performance due to non-deterministic factors like memory reclaim, AG lock-stepping between threads, etc.

Hence there's a bit of unfairness between the threads and often the first thread finishes 30s before the last thread. The times are for the last thread completing and there can be significant variation on that.

Quoted 7 lines
Quoted 7 lines
fs/xfs/xfs_log.c +2 -1
@@ -2009,13 +2010,14 @@ xlog_sync(
 	 * synchronously here; for an internal log we can simply use the block
 	 * layer state machine for preflushes.
 	 */
-	if (log->l_targ != log->l_mp->m_ddev_targp || split) {
+	if (log->l_targ != log->l_mp->m_ddev_targp ||
+	    (split && (iclog->ic_flags & XLOG_ICL_NEED_FLUSH))) {
Quoted 8 lines

fs/xfs/xfs_log.c xlog_sync( · line 2016

Quoted 3 lines
fs/xfs/xfs_log.c +1 -1
 		xfs_flush_bdev(log->l_mp->m_ddev_targp->bt_bdev);
-		need_flush = false;
+		iclog->ic_flags &= ~XLOG_ICL_NEED_FLUSH;

Once you touch all the buffer flags anyway we should optimize the log wraparound case here - insteaad of th synchronous flush we just need to set REQ_PREFLUSH on the first log bio, which should be nicely doable with your infrastruture.

That sounds like another patch because it's a change of behaviour.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#55 lore raw

Re: [PATCH 7/8 v2] xfs: journal IO cache flush reductions

In reply to Dave Chinner
Quoted 2 lines · Dave Chinner

From: Dave Chinner <[email protected]>

...

Quoted 18 lines

---
Version 2:
- repost manually without git/guilt mangling the patch author
- fix bug in XLOG_ICL_NEED_FUA definition that didn't manifest as an ordering bug in generic/45[57] until testing the CIL pipelining changes much later in the series.

 fs/xfs/xfs_log.c      | 33 +++++++++++++++++++++++----------
 fs/xfs/xfs_log_cil.c  |  7 ++++++-
 fs/xfs/xfs_log_priv.h |  4 ++++
 3 files changed, 33 insertions(+), 11 deletions(-)
fs/xfs/xfs_log.c +0 -0
index 6c3fb6dcb505..08d68a6161ae 100644

...

fs/xfs/xfs_log.c xlog_sync( · line 1955

Quoted 6 lines
fs/xfs/xfs_log.c +1 -1
@@ -1951,7 +1952,7 @@ xlog_sync(
 	unsigned int		roundoff;       /* roundoff to BB or stripe */
 	uint64_t		bno;
 	unsigned int		size;
-	bool			need_flush = true, split = false;
+	bool			split = false;

I find the mash of the logic change (i.e. when to flush) and rework of the bool to ->ic_flags changes in this patch unnecessarily convoluted. IMO, this should be two patches.

Quoted 52 lines
fs/xfs/xfs_log.c +16 -4
 	ASSERT(atomic_read(&iclog->ic_refcnt) == 0);
 
@@ -2009,13 +2010,14 @@ xlog_sync(
 	 * synchronously here; for an internal log we can simply use the block
 	 * layer state machine for preflushes.
 	 */
-	if (log->l_targ != log->l_mp->m_ddev_targp || split) {
+	if (log->l_targ != log->l_mp->m_ddev_targp ||
+	    (split && (iclog->ic_flags & XLOG_ICL_NEED_FLUSH))) {
 		xfs_flush_bdev(log->l_mp->m_ddev_targp->bt_bdev);
-		need_flush = false;
+		iclog->ic_flags &= ~XLOG_ICL_NEED_FLUSH;
 	}
 
 	xlog_verify_iclog(log, iclog, count);
-	xlog_write_iclog(log, iclog, bno, count, need_flush);
+	xlog_write_iclog(log, iclog, bno, count);
 }
 
 /*
@@ -2469,10 +2471,21 @@ xlog_write(
 		ASSERT(log_offset <= iclog->ic_size - 1);
 		ptr = iclog->ic_datap + log_offset;
 
-		/* start_lsn is the first lsn written to. That's all we need. */
+		/* Start_lsn is the first lsn written to. */
 		if (start_lsn && !*start_lsn)
 			*start_lsn = be64_to_cpu(iclog->ic_header.h_lsn);
 
+		/*
+		 * iclogs containing commit records or unmount records need
+		 * to issue ordering cache flushes and commit immediately
+		 * to stable storage to guarantee journal vs metadata ordering
+		 * is correctly maintained in the storage media.
+		 */
+		if (optype & (XLOG_COMMIT_TRANS | XLOG_UNMOUNT_TRANS)) {
+			iclog->ic_flags |= (XLOG_ICL_NEED_FLUSH |
+						XLOG_ICL_NEED_FUA);
+		}
+
 		/*
 		 * This loop writes out as many regions as can fit in the amount
 		 * of space which was allocated by xlog_state_get_iclog_space().
fs/xfs/xfs_log_cil.c +0 -0
index 4093d2d0db7c..370da7c2bfc8 100644
@@ -894,10 +894,15 @@ xlog_cil_push_work(
 
 	/*
 	 * If the checkpoint spans multiple iclogs, wait for all previous

fs/xfs/xfs_log_cil.c xlog_cil_push_work( · line 905

Quoted 10 lines
fs/xfs/xfs_log_cil.c +6 -1
-	 * iclogs to complete before we submit the commit_iclog.
+	 * iclogs to complete before we submit the commit_iclog. If it is in the
+	 * same iclog as the start of the checkpoint, then we can skip the iclog
+	 * cache flush because there are no other iclogs we need to order
+	 * against.
 	 */
 	if (ctx->start_lsn != commit_lsn)
 		xlog_wait_on_iclog_lsn(commit_iclog, ctx->start_lsn);
+	else
+		commit_iclog->ic_flags &= ~XLOG_ICL_NEED_FLUSH;

Is there a reason we're making these iclog changes (here and in xlog_write(), at least) outside of ->l_icloglock? I suspect this is safe atm due to the fact that the CIL push is single threaded via the workqueue, but the rest of the iclog management code is written with proper internal exclusion in mind. This seems like a bit of a landmine if the execution context is the only thing protecting us here... hm?

Brian

Quoted 26 lines
fs/xfs/xfs_log_cil.c +0 -0
 	/* release the hounds! */
 	xfs_log_release_iclog(commit_iclog);
fs/xfs/xfs_log_priv.h +4 -0
index 10a41b1dd895..a77e00b7789a 100644
@@ -133,6 +133,9 @@ enum xlog_iclog_state {
 
 #define XLOG_COVER_OPS		5
 
+#define XLOG_ICL_NEED_FLUSH	(1 << 0)	/* iclog needs REQ_PREFLUSH */
+#define XLOG_ICL_NEED_FUA	(1 << 1)	/* iclog needs REQ_FUA */
+
 /* Ticket reservation region accounting */ 
 #define XLOG_TIC_LEN_MAX	15
 
@@ -201,6 +204,7 @@ typedef struct xlog_in_core {
 	u32			ic_size;
 	u32			ic_offset;
 	enum xlog_iclog_state	ic_state;
+	unsigned int		ic_flags;
 	char			*ic_datap;	/* pointer to iclog data */
 
 	/* Callback structures need their own cacheline */
#56 lore raw

PATCH 8/8 xfs: Fix CIL throttle hang when CIL space used going backwards

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

A hang with tasks stuck on the CIL hard throttle was reported and largely diagnosed by Donald Buczek, who discovered that it was a result of the CIL context space usage decrementing in committed transactions once the hard throttle limit had been hit and processes were already blocked. This resulted in the CIL push not waking up those waiters because the CIL context was no longer over the hard throttle limit.

The surprising aspect of this was the CIL space usage going backwards regularly enough to trigger this situation. Assumptions had been made in design that the relogging process would only increase the size of the objects in the CIL, and so that space would only increase.

This change and commit message fixes the issue and documents the result of an audit of the triggers that can cause the CIL space to go backwards, how large the backwards steps tend to be, the frequency in which they occur, and what the impact on the CIL accounting code is.

Even though the CIL ctx->space_used can go backwards, it will only do so if the log item is already logged to the CIL and contains a space reservation for it's entire logged state. This is tracked by the shadow buffer state on the log item. If the item is not previously logged in the CIL it has no shadow buffer nor log vector, and hence the entire size of the logged item copied to the log vector is accounted to the CIL space usage. i.e. it will always go up in this case.

If the item has a log vector (i.e. already in the CIL) and the size decreases, then the existing log vector will be overwritten and the space usage will go down. This is the only condition where the space usage reduces, and it can only occur when an item is already tracked in the CIL. Hence we are safe from CIL space usage underruns as a result of log items decreasing in size when they are relogged.

Typically this reduction in CIL usage occurs from metadta blocks being free, such as when a btree block merge
occurs or a directory enter/xattr entry is removed and the da-tree is reduced in size. This generally results in a reduction in size of around a single block in the CIL, but also tends to increase the number of log vectors because the parent and sibling nodes in the tree needs to be updated when a btree block is removed. If a multi-level merge occurs, then we see reduction in size of 2+ blocks, but again the log vector count goes up.

The other vector is inode fork size changes, which only log the current size of the fork and ignore the previously logged size when the fork is relogged. Hence if we are removing items from the inode fork (dir/xattr removal in shortform, extent record removal in extent form, etc) the relogged size of the inode for can decrease.

No other log items can decrease in size either because they are a fixed size (e.g. dquots) or they cannot be relogged (e.g. relogging an intent actually creates a new intent log item and doesn't relog the old item at all.) Hence the only two vectors for CIL context size reduction are relogging inode forks and marking buffers active in the CIL as stale.

Long story short: the majority of the code does the right thing and handles the reduction in log item size correctly, and only the CIL hard throttle implementation is problematic and needs fixing. This patch makes that fix, as well as adds comments in the log item code that result in items shrinking in size when they are relogged as a clear reminder that this can and does happen frequently.

The throttle fix is based upon the change Donald proposed, though it goes further to ensure that once the throttle is activated, it captures all tasks until the CIL push issues a wakeup, regardless of whether the CIL space used has gone back under the throttle threshold.

This ensures that we prevent tasks reducing the CIL slightly under the throttle threshold and then making more changes that push it well over the throttle limit. This is acheived by checking if the throttle wait queue is already active as a condition of throttling. Hence once we start throttling, we continue to apply the throttle until the CIL context push wakes everything on the wait queue.

We can use waitqueue_active() for the waitqueue manipulations and checks as they are all done under the ctx->xc_push_lock. Hence the waitqueue has external serialisation and we can safely peek inside the wait queue without holding the internal waitqueue locks.

Many thanks to Donald for his diagnostic and analysis work to isolate the cause of this hang.

---
 fs/xfs/xfs_buf_item.c   | 37 ++++++++++++++++++-------------------
 fs/xfs/xfs_inode_item.c | 14 ++++++++++++++
 fs/xfs/xfs_log_cil.c    | 22 +++++++++++++++++-----
 3 files changed, 49 insertions(+), 24 deletions(-)
fs/xfs/xfs_buf_item.c +18 -19
index dc0be2a639cc..17960b1ce5ef 100644
@@ -56,14 +56,12 @@ xfs_buf_log_format_size(
 }
 
 /*
- * This returns the number of log iovecs needed to log the
- * given buf log item.
+ * Return the number of log iovecs and space needed to log the given buf log
+ * item segment.
  *
- * It calculates this as 1 iovec for the buf log format structure
- * and 1 for each stretch of non-contiguous chunks to be logged.
- * Contiguous chunks are logged in a single iovec.
- *
- * If the XFS_BLI_STALE flag has been set, then log nothing.
+ * It calculates this as 1 iovec for the buf log format structure and 1 for each
+ * stretch of non-contiguous chunks to be logged.  Contiguous chunks are logged
+ * in a single iovec.
  */
 STATIC void
 xfs_buf_item_size_segment(
@@ -119,11 +117,8 @@ xfs_buf_item_size_segment(
 }
 
 /*
- * This returns the number of log iovecs needed to log the given buf log item.
- *
- * It calculates this as 1 iovec for the buf log format structure and 1 for each
- * stretch of non-contiguous chunks to be logged.  Contiguous chunks are logged
- * in a single iovec.
+ * Return the number of log iovecs and space needed to log the given buf log
+ * item.
  *
  * Discontiguous buffers need a format structure per region that is being
  * logged. This makes the changes in the buffer appear to log recovery as though
@@ -133,7 +128,11 @@ xfs_buf_item_size_segment(
  * what ends up on disk.
  *
  * If the XFS_BLI_STALE flag has been set, then log nothing but the buf log
- * format structures.
+ * format structures. If the item has previously been logged and has dirty
+ * regions, we do not relog them in stale buffers. This has the effect of
+ * reducing the size of the relogged item by the amount of dirty data tracked
+ * by the log item. This can result in the committing transaction reducing the
+ * amount of space being consumed by the CIL.
  */
 STATIC void
 xfs_buf_item_size(
@@ -147,9 +146,9 @@ xfs_buf_item_size(
 	ASSERT(atomic_read(&bip->bli_refcount) > 0);
 	if (bip->bli_flags & XFS_BLI_STALE) {
 		/*
-		 * The buffer is stale, so all we need to log
-		 * is the buf log format structure with the
-		 * cancel flag in it.
+		 * The buffer is stale, so all we need to log is the buf log
+		 * format structure with the cancel flag in it as we are never
+		 * going to replay the changes tracked in the log item.
 		 */
 		trace_xfs_buf_item_size_stale(bip);
 		ASSERT(bip->__bli_format.blf_flags & XFS_BLF_CANCEL);
@@ -164,9 +163,9 @@ xfs_buf_item_size(
 
 	if (bip->bli_flags & XFS_BLI_ORDERED) {
 		/*
-		 * The buffer has been logged just to order it.
-		 * It is not being included in the transaction
-		 * commit, so no vectors are used at all.
+		 * The buffer has been logged just to order it. It is not being
+		 * included in the transaction commit, so no vectors are used at
+		 * all.
 		 */
 		trace_xfs_buf_item_size_ordered(bip);
 		*nvecs = XFS_LOG_VEC_ORDERED;
fs/xfs/xfs_inode_item.c +14 -0
index 17e20a6d8b4e..6ff91e5bf3cd 100644
@@ -28,6 +28,20 @@ static inline struct xfs_inode_log_item *INODE_ITEM(struct xfs_log_item *lip)
 	return container_of(lip, struct xfs_inode_log_item, ili_item);
 }
 
+/*
+ * The logged size of an inode fork is always the current size of the inode
+ * fork. This means that when an inode fork is relogged, the size of the logged
+ * region is determined by the current state, not the combination of the
+ * previously logged state + the current state. This is different relogging
+ * behaviour to most other log items which will retain the size of the
+ * previously logged changes when smaller regions are relogged.
+ *
+ * Hence operations that remove data from the inode fork (e.g. shortform
+ * dir/attr remove, extent form extent removal, etc), the size of the relogged
+ * inode gets -smaller- rather than stays the same size as the previously logged
+ * size and this can result in the committing transaction reducing the amount of
+ * space being consumed by the CIL.
+ */
 STATIC void
 xfs_inode_item_data_fork_size(
 	struct xfs_inode_log_item *iip,
fs/xfs/xfs_log_cil.c +17 -5
index 370da7c2bfc8..0a00c3c9610c 100644
@@ -669,9 +669,14 @@ xlog_cil_push_work(
 	ASSERT(push_seq <= ctx->sequence);
 
 	/*
-	 * Wake up any background push waiters now this context is being pushed.
+	 * As we are about to switch to a new, empty CIL context, we no longer
+	 * need to throttle tasks on CIL space overruns. Wake any waiters that
+	 * the hard push throttle may have caught so they can start committing
+	 * to the new context. The ctx->xc_push_lock provides the serialisation
+	 * necessary for safely using the lockless waitqueue_active() check in
+	 * this context.
 	 */
-	if (ctx->space_used >= XLOG_CIL_BLOCKING_SPACE_LIMIT(log))
+	if (waitqueue_active(&cil->xc_push_wait))
 		wake_up_all(&cil->xc_push_wait);
 
 	/*
@@ -941,7 +946,7 @@ xlog_cil_push_background(
 	ASSERT(!list_empty(&cil->xc_cil));
 
 	/*
-	 * don't do a background push if we haven't used up all the
+	 * Don't do a background push if we haven't used up all the
 	 * space available yet.
 	 */
 	if (cil->xc_ctx->space_used < XLOG_CIL_SPACE_LIMIT(log)) {
@@ -965,9 +970,16 @@ xlog_cil_push_background(
 
 	/*
 	 * If we are well over the space limit, throttle the work that is being
-	 * done until the push work on this context has begun.
+	 * done until the push work on this context has begun. Enforce the hard
+	 * throttle on all transaction commits once it has been activated, even
+	 * if the committing transactions have resulted in the space usage
+	 * dipping back down under the hard limit.
+	 *
+	 * The ctx->xc_push_lock provides the serialisation necessary for safely
+	 * using the lockless waitqueue_active() check in this context.
 	 */
-	if (cil->xc_ctx->space_used >= XLOG_CIL_BLOCKING_SPACE_LIMIT(log)) {
+	if (cil->xc_ctx->space_used >= XLOG_CIL_BLOCKING_SPACE_LIMIT(log) ||
+	    waitqueue_active(&cil->xc_push_wait)) {
 		trace_xfs_log_cil_wait(log, cil->xc_ctx->ticket);
 		ASSERT(cil->xc_ctx->space_used < log->l_logsize);
 		xlog_wait(&cil->xc_push_wait, &cil->xc_push_lock);
Signature · 1 line
2.28.0
Reviewed-by on v1 8/8
#57 lore raw

Re: [PATCH 8/8] xfs: Fix CIL throttle hang when CIL space used going backwards

Quoted 31 lines · Dave Chinner

From: Dave Chinner <[email protected]>

A hang with tasks stuck on the CIL hard throttle was reported and largely diagnosed by Donald Buczek, who discovered that it was a result of the CIL context space usage decrementing in committed transactions once the hard throttle limit had been hit and processes were already blocked. This resulted in the CIL push not waking up those waiters because the CIL context was no longer over the hard throttle limit.

The surprising aspect of this was the CIL space usage going backwards regularly enough to trigger this situation. Assumptions had been made in design that the relogging process would only increase the size of the objects in the CIL, and so that space would only increase.

This change and commit message fixes the issue and documents the result of an audit of the triggers that can cause the CIL space to go backwards, how large the backwards steps tend to be, the frequency in which they occur, and what the impact on the CIL accounting code is.

Even though the CIL ctx->space_used can go backwards, it will only do so if the log item is already logged to the CIL and contains a space reservation for it's entire logged state. This is tracked by the shadow buffer state on the log item. If the item is not previously logged in the CIL it has no shadow buffer nor log vector, and hence the entire size of the logged item copied to the log vector is accounted to the CIL space usage. i.e. it will always go up in this case.

Quoted 8 lines

If the item has a log vector (i.e. already in the CIL) and the size decreases, then the existing log vector will be overwritten and the space usage will go down. This is the only condition where the space usage reduces, and it can only occur when an item is already tracked in the CIL. Hence we are safe from CIL space usage underruns as a result of log items decreasing in size when they are relogged.

Typically this reduction in CIL usage occurs from metadta blocks

"metadata"...

Quoted 47 lines

being free, such as when a btree block merge
occurs or a directory enter/xattr entry is removed and the da-tree is reduced in size. This generally results in a reduction in size of around a single block in the CIL, but also tends to increase the number of log vectors because the parent and sibling nodes in the tree needs to be updated when a btree block is removed. If a multi-level merge occurs, then we see reduction in size of 2+ blocks, but again the log vector count goes up.

The other vector is inode fork size changes, which only log the current size of the fork and ignore the previously logged size when the fork is relogged. Hence if we are removing items from the inode fork (dir/xattr removal in shortform, extent record removal in extent form, etc) the relogged size of the inode for can decrease.

No other log items can decrease in size either because they are a fixed size (e.g. dquots) or they cannot be relogged (e.g. relogging an intent actually creates a new intent log item and doesn't relog the old item at all.) Hence the only two vectors for CIL context size reduction are relogging inode forks and marking buffers active in the CIL as stale.

Long story short: the majority of the code does the right thing and handles the reduction in log item size correctly, and only the CIL hard throttle implementation is problematic and needs fixing. This patch makes that fix, as well as adds comments in the log item code that result in items shrinking in size when they are relogged as a clear reminder that this can and does happen frequently.

The throttle fix is based upon the change Donald proposed, though it goes further to ensure that once the throttle is activated, it captures all tasks until the CIL push issues a wakeup, regardless of whether the CIL space used has gone back under the throttle threshold.

This ensures that we prevent tasks reducing the CIL slightly under the throttle threshold and then making more changes that push it well over the throttle limit. This is acheived by checking if the throttle wait queue is already active as a condition of throttling. Hence once we start throttling, we continue to apply the throttle until the CIL context push wakes everything on the wait queue.

We can use waitqueue_active() for the waitqueue manipulations and checks as they are all done under the ctx->xc_push_lock. Hence the waitqueue has external serialisation and we can safely peek inside the wait queue without holding the internal waitqueue locks.

Quoted 4 lines

Many thanks to Donald for his diagnostic and analysis work to isolate the cause of this hang.

Does this whole series fix the Donald's problem?

Quoted 3 lines

Looks ok to me,

--D

Quoted 160 lines
---
 fs/xfs/xfs_buf_item.c   | 37 ++++++++++++++++++-------------------
 fs/xfs/xfs_inode_item.c | 14 ++++++++++++++
 fs/xfs/xfs_log_cil.c    | 22 +++++++++++++++++-----
 3 files changed, 49 insertions(+), 24 deletions(-)
fs/xfs/xfs_buf_item.c +18 -19
index dc0be2a639cc..17960b1ce5ef 100644
@@ -56,14 +56,12 @@ xfs_buf_log_format_size(
 }
 
 /*
- * This returns the number of log iovecs needed to log the
- * given buf log item.
+ * Return the number of log iovecs and space needed to log the given buf log
+ * item segment.
  *
- * It calculates this as 1 iovec for the buf log format structure
- * and 1 for each stretch of non-contiguous chunks to be logged.
- * Contiguous chunks are logged in a single iovec.
- *
- * If the XFS_BLI_STALE flag has been set, then log nothing.
+ * It calculates this as 1 iovec for the buf log format structure and 1 for each
+ * stretch of non-contiguous chunks to be logged.  Contiguous chunks are logged
+ * in a single iovec.
  */
 STATIC void
 xfs_buf_item_size_segment(
@@ -119,11 +117,8 @@ xfs_buf_item_size_segment(
 }
 
 /*
- * This returns the number of log iovecs needed to log the given buf log item.
- *
- * It calculates this as 1 iovec for the buf log format structure and 1 for each
- * stretch of non-contiguous chunks to be logged.  Contiguous chunks are logged
- * in a single iovec.
+ * Return the number of log iovecs and space needed to log the given buf log
+ * item.
  *
  * Discontiguous buffers need a format structure per region that is being
  * logged. This makes the changes in the buffer appear to log recovery as though
@@ -133,7 +128,11 @@ xfs_buf_item_size_segment(
  * what ends up on disk.
  *
  * If the XFS_BLI_STALE flag has been set, then log nothing but the buf log
- * format structures.
+ * format structures. If the item has previously been logged and has dirty
+ * regions, we do not relog them in stale buffers. This has the effect of
+ * reducing the size of the relogged item by the amount of dirty data tracked
+ * by the log item. This can result in the committing transaction reducing the
+ * amount of space being consumed by the CIL.
  */
 STATIC void
 xfs_buf_item_size(
@@ -147,9 +146,9 @@ xfs_buf_item_size(
 	ASSERT(atomic_read(&bip->bli_refcount) > 0);
 	if (bip->bli_flags & XFS_BLI_STALE) {
 		/*
-		 * The buffer is stale, so all we need to log
-		 * is the buf log format structure with the
-		 * cancel flag in it.
+		 * The buffer is stale, so all we need to log is the buf log
+		 * format structure with the cancel flag in it as we are never
+		 * going to replay the changes tracked in the log item.
 		 */
 		trace_xfs_buf_item_size_stale(bip);
 		ASSERT(bip->__bli_format.blf_flags & XFS_BLF_CANCEL);
@@ -164,9 +163,9 @@ xfs_buf_item_size(
 
 	if (bip->bli_flags & XFS_BLI_ORDERED) {
 		/*
-		 * The buffer has been logged just to order it.
-		 * It is not being included in the transaction
-		 * commit, so no vectors are used at all.
+		 * The buffer has been logged just to order it. It is not being
+		 * included in the transaction commit, so no vectors are used at
+		 * all.
 		 */
 		trace_xfs_buf_item_size_ordered(bip);
 		*nvecs = XFS_LOG_VEC_ORDERED;
fs/xfs/xfs_inode_item.c +14 -0
index 17e20a6d8b4e..6ff91e5bf3cd 100644
@@ -28,6 +28,20 @@ static inline struct xfs_inode_log_item *INODE_ITEM(struct xfs_log_item *lip)
 	return container_of(lip, struct xfs_inode_log_item, ili_item);
 }
 
+/*
+ * The logged size of an inode fork is always the current size of the inode
+ * fork. This means that when an inode fork is relogged, the size of the logged
+ * region is determined by the current state, not the combination of the
+ * previously logged state + the current state. This is different relogging
+ * behaviour to most other log items which will retain the size of the
+ * previously logged changes when smaller regions are relogged.
+ *
+ * Hence operations that remove data from the inode fork (e.g. shortform
+ * dir/attr remove, extent form extent removal, etc), the size of the relogged
+ * inode gets -smaller- rather than stays the same size as the previously logged
+ * size and this can result in the committing transaction reducing the amount of
+ * space being consumed by the CIL.
+ */
 STATIC void
 xfs_inode_item_data_fork_size(
 	struct xfs_inode_log_item *iip,
fs/xfs/xfs_log_cil.c +17 -5
index 370da7c2bfc8..0a00c3c9610c 100644
@@ -669,9 +669,14 @@ xlog_cil_push_work(
 	ASSERT(push_seq <= ctx->sequence);
 
 	/*
-	 * Wake up any background push waiters now this context is being pushed.
+	 * As we are about to switch to a new, empty CIL context, we no longer
+	 * need to throttle tasks on CIL space overruns. Wake any waiters that
+	 * the hard push throttle may have caught so they can start committing
+	 * to the new context. The ctx->xc_push_lock provides the serialisation
+	 * necessary for safely using the lockless waitqueue_active() check in
+	 * this context.
 	 */
-	if (ctx->space_used >= XLOG_CIL_BLOCKING_SPACE_LIMIT(log))
+	if (waitqueue_active(&cil->xc_push_wait))
 		wake_up_all(&cil->xc_push_wait);
 
 	/*
@@ -941,7 +946,7 @@ xlog_cil_push_background(
 	ASSERT(!list_empty(&cil->xc_cil));
 
 	/*
-	 * don't do a background push if we haven't used up all the
+	 * Don't do a background push if we haven't used up all the
 	 * space available yet.
 	 */
 	if (cil->xc_ctx->space_used < XLOG_CIL_SPACE_LIMIT(log)) {
@@ -965,9 +970,16 @@ xlog_cil_push_background(
 
 	/*
 	 * If we are well over the space limit, throttle the work that is being
-	 * done until the push work on this context has begun.
+	 * done until the push work on this context has begun. Enforce the hard
+	 * throttle on all transaction commits once it has been activated, even
+	 * if the committing transactions have resulted in the space usage
+	 * dipping back down under the hard limit.
+	 *
+	 * The ctx->xc_push_lock provides the serialisation necessary for safely
+	 * using the lockless waitqueue_active() check in this context.
 	 */
-	if (cil->xc_ctx->space_used >= XLOG_CIL_BLOCKING_SPACE_LIMIT(log)) {
+	if (cil->xc_ctx->space_used >= XLOG_CIL_BLOCKING_SPACE_LIMIT(log) ||
+	    waitqueue_active(&cil->xc_push_wait)) {
 		trace_xfs_log_cil_wait(log, cil->xc_ctx->ticket);
 		ASSERT(cil->xc_ctx->space_used < log->l_logsize);
 		xlog_wait(&cil->xc_push_wait, &cil->xc_push_lock);
Signature · 1 line
2.28.0
#58 lore raw

Re: [PATCH 8/8] xfs: Fix CIL throttle hang when CIL space used going backwards

Quoted 22 lines · Darrick J. Wong
Quoted 21 lines · Dave Chinner

From: Dave Chinner <[email protected]>

A hang with tasks stuck on the CIL hard throttle was reported and largely diagnosed by Donald Buczek, who discovered that it was a result of the CIL context space usage decrementing in committed transactions once the hard throttle limit had been hit and processes were already blocked. This resulted in the CIL push not waking up those waiters because the CIL context was no longer over the hard throttle limit.

The surprising aspect of this was the CIL space usage going backwards regularly enough to trigger this situation. Assumptions had been made in design that the relogging process would only increase the size of the objects in the CIL, and so that space would only increase.

This change and commit message fixes the issue and documents the result of an audit of the triggers that can cause the CIL space to go backwards, how large the backwards steps tend to be, the frequency in which they occur, and what the impact on the CIL accounting code is.

....

Quoted 1 line

Does this whole series fix the Donald's problem?

No, just this patch is needed to fix that problem.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]