Skip to contentSkip to conversation

speculative preallocation quota throttling

PATCH Superseded Brian Foster 3 participants 16 messages Feb 20, 2013 – Feb 25, 2013 1 message not in the archive XFS filesystem

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v4 cover speculative preallocation quota throttling

Hi all,

Here is v4 of the speculative preallocation quota throttling set. Patch 3 of v3 is dropped due to a redundancy and a few changes are made to the threshold management code (patch 4/6). The remaining patches are unchanged from v3.

Brian

v4:
- Drop patch 3/7 from v3 (xfs: cap prealloc size to free space before shift).
- Several updates to patch 4/6:
- Rename xfs_dquot_init_prealloc() to xfs_dquot_set_prealloc_limits().
- Unroll the prealloc threshold loop and remove the increment def.
- Fix up some comments.
v3:
- Rebased on top of updated speculative preallocation algorithm. v2:
- Fix up xfs_iomap_prealloc_size() rounding (patch 2).
- Add pre-calculated fields to xfs_dquot to support throttling.
- Move to logarithmic (shift) throttler and finer tuned trigger/throttle logic.

Brian Foster (6):
xfs: reorganize xfs_iomap_prealloc_size to remove indentation
xfs: push rounddown_pow_of_two() to after prealloc throttle
xfs: pass xfs_dquot to xfs_qm_adjust_dqlimits() instead of
xfs_disk_dquot_t
xfs: xfs_dquot prealloc throttling watermarks and low free space
xfs: add quota-driven speculative preallocation throttling
xfs: xfs_iomap_prealloc_size() tracepoint

 fs/xfs/xfs_dquot.c       |   44 ++++++++++++-
 fs/xfs/xfs_dquot.h       |   14 ++++-
 fs/xfs/xfs_iomap.c       |  161 +++++++++++++++++++++++++++++++++++++---------
 fs/xfs/xfs_qm.c          |    2 +-
 fs/xfs/xfs_qm_syscalls.c |    1 +
 fs/xfs/xfs_trace.h       |   24 +++++++
 fs/xfs/xfs_trans_dquot.c |    2 +-
 7 files changed, 212 insertions(+), 36 deletions(-)
Signature · 6 lines
1.7.7.6

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#1 lore raw

PATCH v4 1/6 xfs: reorganize xfs_iomap_prealloc_size to remove indentation

The majority of xfs_iomap_prealloc_size() executes within the check for lack of default I/O size. Reverse the logic to remove the extra indentation.

---
 fs/xfs/xfs_iomap.c |   63 ++++++++++++++++++++++++++-------------------------
 1 files changed, 32 insertions(+), 31 deletions(-)
fs/xfs/xfs_iomap.c +32 -31
index 912d83d..d914419 100644
@@ -381,42 +381,43 @@ xfs_iomap_prealloc_size(
 	int			nimaps)
 {
 	xfs_fsblock_t		alloc_blocks = 0;
+	int			shift = 0;
+	int64_t			freesp;
 
 	alloc_blocks = xfs_iomap_eof_prealloc_initial_size(mp, ip, offset,
 							   imap, nimaps);
-	if (alloc_blocks > 0) {
-		int shift = 0;
-		int64_t freesp;
-
-		alloc_blocks = XFS_FILEOFF_MIN(MAXEXTLEN,
-					rounddown_pow_of_two(alloc_blocks));
-
-		xfs_icsb_sync_counters(mp, XFS_ICSB_LAZY_COUNT);
-		freesp = mp->m_sb.sb_fdblocks;
-		if (freesp < mp->m_low_space[XFS_LOWSP_5_PCNT]) {
-			shift = 2;
-			if (freesp < mp->m_low_space[XFS_LOWSP_4_PCNT])
-				shift++;
-			if (freesp < mp->m_low_space[XFS_LOWSP_3_PCNT])
-				shift++;
-			if (freesp < mp->m_low_space[XFS_LOWSP_2_PCNT])
-				shift++;
-			if (freesp < mp->m_low_space[XFS_LOWSP_1_PCNT])
-				shift++;
-		}
-		if (shift)
-			alloc_blocks >>= shift;
-
-		/*
-		 * If we are still trying to allocate more space than is
-		 * available, squash the prealloc hard. This can happen if we
-		 * have a large file on a small filesystem and the above
-		 * lowspace thresholds are smaller than MAXEXTLEN.
-		 */
-		while (alloc_blocks >= freesp)
-			alloc_blocks >>= 4;
+	if (!alloc_blocks)
+		goto check_writeio;
+
+	alloc_blocks = XFS_FILEOFF_MIN(MAXEXTLEN,
+				rounddown_pow_of_two(alloc_blocks));
+
+	xfs_icsb_sync_counters(mp, XFS_ICSB_LAZY_COUNT);
+	freesp = mp->m_sb.sb_fdblocks;
+	if (freesp < mp->m_low_space[XFS_LOWSP_5_PCNT]) {
+		shift = 2;
+		if (freesp < mp->m_low_space[XFS_LOWSP_4_PCNT])
+			shift++;
+		if (freesp < mp->m_low_space[XFS_LOWSP_3_PCNT])
+			shift++;
+		if (freesp < mp->m_low_space[XFS_LOWSP_2_PCNT])
+			shift++;
+		if (freesp < mp->m_low_space[XFS_LOWSP_1_PCNT])
+			shift++;
 	}
+	if (shift)
+		alloc_blocks >>= shift;
+
+	/*
+	 * If we are still trying to allocate more space than is
+	 * available, squash the prealloc hard. This can happen if we
+	 * have a large file on a small filesystem and the above
+	 * lowspace thresholds are smaller than MAXEXTLEN.
+	 */
+	while (alloc_blocks >= freesp)
+		alloc_blocks >>= 4;
 
+check_writeio:
 	if (alloc_blocks < mp->m_writeio_blocks)
 		alloc_blocks = mp->m_writeio_blocks;
 
Signature · 6 lines
1.7.7.6

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#2 lore raw

Re: [PATCH v4 1/6] xfs: reorganize xfs_iomap_prealloc_size to remove indentation

Quoted 21 lines · Brian Foster

The majority of xfs_iomap_prealloc_size() executes within the check for lack of default I/O size. Reverse the logic to remove the extra indentation.

---
  fs/xfs/xfs_iomap.c |   63 ++++++++++++++++++++++++++-------------------------
  1 files changed, 32 insertions(+), 31 deletions(-)
fs/xfs/xfs_iomap.c +2 -0
index 912d83d..d914419 100644
@@ -381,42 +381,43 @@ xfs_iomap_prealloc_size(
  	int			nimaps)
  {
  	xfs_fsblock_t		alloc_blocks = 0;
+	int			shift = 0;
+	int64_t			freesp;

...

Quoted 1 line
fs/xfs/xfs_iomap.c +1 -0
+	freesp = mp->m_sb.sb_fdblocks;

...

fs/xfs/xfs_iomap.c xfs_iomap_prealloc_size(

Quoted 8 lines
fs/xfs/xfs_iomap.c +8 -0
+	/*
+	 * If we are still trying to allocate more space than is
+	 * available, squash the prealloc hard. This can happen if we
+	 * have a large file on a small filesystem and the above
+	 * lowspace thresholds are smaller than MAXEXTLEN.
+	 */
+	while (alloc_blocks>= freesp)
+		alloc_blocks>>= 4;

Hi Brian, I am looking at your speculative preallocation quota throttling series.

I know this code is from commit 4d559a3b. would this not be bad of freesp == 0?

--Mark.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#3 lore raw

Re: [PATCH v4 1/6] xfs: reorganize xfs_iomap_prealloc_size to remove indentation

Quoted 40 lines · Mark Tinguely
Quoted 22 lines · Brian Foster

The majority of xfs_iomap_prealloc_size() executes within the check for lack of default I/O size. Reverse the logic to remove the extra indentation.

---
  fs/xfs/xfs_iomap.c |   63
++++++++++++++++++++++++++-------------------------
  1 files changed, 32 insertions(+), 31 deletions(-)
fs/xfs/xfs_iomap.c +2 -0
index 912d83d..d914419 100644
@@ -381,42 +381,43 @@ xfs_iomap_prealloc_size(
      int            nimaps)
  {
      xfs_fsblock_t        alloc_blocks = 0;
+    int            shift = 0;
+    int64_t            freesp;

...

Quoted 1 line

+ freesp = mp->m_sb.sb_fdblocks;

...

fs/xfs/xfs_iomap.c xfs_iomap_prealloc_size(

Quoted 8 lines
fs/xfs/xfs_iomap.c +8 -0
+    /*
+     * If we are still trying to allocate more space than is
+     * available, squash the prealloc hard. This can happen if we
+     * have a large file on a small filesystem and the above
+     * lowspace thresholds are smaller than MAXEXTLEN.
+     */
+    while (alloc_blocks>= freesp)
+        alloc_blocks>>= 4;
Quoted 6 lines

Hi Brian, I am looking at your speculative preallocation quota throttling series.

I know this code is from commit 4d559a3b. would this not be bad of freesp == 0?

Thanks. Hmm, I guess if freesp is 0 we'd hit an infinite loop (irrespective of this patchset). We could change the comparison to >, but I think the following would be more clear:

	while (alloc_blocks && alloc_blocks >= freesp)
		alloc_blocks >>= 4;

Thoughts? I'll send out a one-liner to bat around if that looks reasonable. Good catch.

Brian

Quoted 2 lines

--Mark.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#4 lore raw

Re: [PATCH v4 1/6] xfs: reorganize xfs_iomap_prealloc_size to remove indentation

Quoted 52 lines · Brian Foster
Quoted 40 lines · Mark Tinguely
Quoted 22 lines · Brian Foster

The majority of xfs_iomap_prealloc_size() executes within the check for lack of default I/O size. Reverse the logic to remove the extra indentation.

---
   fs/xfs/xfs_iomap.c |   63
++++++++++++++++++++++++++-------------------------
   1 files changed, 32 insertions(+), 31 deletions(-)
fs/xfs/xfs_iomap.c +2 -0
index 912d83d..d914419 100644
@@ -381,42 +381,43 @@ xfs_iomap_prealloc_size(
       int            nimaps)
   {
       xfs_fsblock_t        alloc_blocks = 0;
+    int            shift = 0;
+    int64_t            freesp;

...

Quoted 1 line

+ freesp = mp->m_sb.sb_fdblocks;

...

fs/xfs/xfs_iomap.c xfs_iomap_prealloc_size(

Quoted 8 lines
fs/xfs/xfs_iomap.c +8 -0
+    /*
+     * If we are still trying to allocate more space than is
+     * available, squash the prealloc hard. This can happen if we
+     * have a large file on a small filesystem and the above
+     * lowspace thresholds are smaller than MAXEXTLEN.
+     */
+    while (alloc_blocks>= freesp)
+        alloc_blocks>>= 4;
Quoted 6 lines

Hi Brian, I am looking at your speculative preallocation quota throttling series.

I know this code is from commit 4d559a3b. would this not be bad of freesp == 0?

Thanks. Hmm, I guess if freesp is 0 we'd hit an infinite loop (irrespective of this patchset). We could change the comparison to>, but I think the following would be more clear:

Quoted 9 lines
	while (alloc_blocks&&  alloc_blocks>= freesp)
		alloc_blocks>>= 4;

Thoughts? I'll send out a one-liner to bat around if that looks reasonable. Good catch.

Brian

Quoted 1 line

--Mark.

That looks reasonable to me.

Thanks,

--Mark.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#5 lore raw

PATCH v4 2/6 xfs: push rounddown_pow_of_two() to after prealloc throttle

In reply to Brian Foster

The round down occurs towards the beginning of the function. Push it down after throttling has occurred. This is to support adding further transformations to 'alloc_blocks' that might not preserve power-of-two alignment (and thus could lead to rounding down multiple times).

---
 fs/xfs/xfs_iomap.c |   19 +++++++++++++++++--
 1 files changed, 17 insertions(+), 2 deletions(-)
fs/xfs/xfs_iomap.c +17 -2
index d914419..daa08f6 100644
@@ -389,8 +389,15 @@ xfs_iomap_prealloc_size(
 	if (!alloc_blocks)
 		goto check_writeio;
 
-	alloc_blocks = XFS_FILEOFF_MIN(MAXEXTLEN,
-				rounddown_pow_of_two(alloc_blocks));
+	/*
+	 * MAXEXTLEN is not a power of two value but we round the prealloc down
+	 * to the nearest power of two value after throttling. To prevent the
+	 * round down from unconditionally reducing the maximum supported prealloc
+	 * size, we round up first, apply appropriate throttling, round down and
+	 * cap the value to MAXEXTLEN.
+	 */
+	alloc_blocks = XFS_FILEOFF_MIN(roundup_pow_of_two(MAXEXTLEN),
+				       alloc_blocks);
 
 	xfs_icsb_sync_counters(mp, XFS_ICSB_LAZY_COUNT);
 	freesp = mp->m_sb.sb_fdblocks;
@@ -407,6 +414,14 @@ xfs_iomap_prealloc_size(
 	}
 	if (shift)
 		alloc_blocks >>= shift;
+	/*
+	 * rounddown_pow_of_two() returns an undefined result if we pass in
+	 * alloc_blocks = 0.
+	 */
+	if (alloc_blocks)
+		alloc_blocks = rounddown_pow_of_two(alloc_blocks);
+	if (alloc_blocks > MAXEXTLEN)
+		alloc_blocks = MAXEXTLEN;
 
 	/*
 	 * If we are still trying to allocate more space than is
Signature · 6 lines
1.7.7.6

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#6 lore raw

PATCH v4 3/6 xfs: pass xfs_dquot to xfs_qm_adjust_dqlimits() instead of xfs_disk_dquot_t

In reply to Brian Foster

Modify xfs_qm_adjust_dqlimits() to take the xfs_dquot as a parameter instead of just the xfs_disk_dquot_t so we can update in-memory fields if necessary.

---
 fs/xfs/xfs_dquot.c       |    3 ++-
 fs/xfs/xfs_dquot.h       |    2 +-
 fs/xfs/xfs_qm.c          |    2 +-
 fs/xfs/xfs_trans_dquot.c |    2 +-
 4 files changed, 5 insertions(+), 4 deletions(-)
fs/xfs/xfs_dquot.c +2 -1
index 8025eb2..e1833b9 100644
@@ -86,9 +86,10 @@ xfs_qm_dqdestroy(
 void
 xfs_qm_adjust_dqlimits(
 	xfs_mount_t		*mp,
-	xfs_disk_dquot_t	*d)
+	struct xfs_dquot	*dq)
 {
 	xfs_quotainfo_t		*q = mp->m_quotainfo;
+	xfs_disk_dquot_t	*d = &dq->q_core;
 
 	ASSERT(d->d_id);
 
fs/xfs/xfs_dquot.h +1 -1
index c694a84..69491b6 100644
@@ -146,7 +146,7 @@ extern void		xfs_qm_dqunpin_wait(xfs_dquot_t *);
 extern void		xfs_qm_adjust_dqtimers(xfs_mount_t *,
 					xfs_disk_dquot_t *);
 extern void		xfs_qm_adjust_dqlimits(xfs_mount_t *,
-					xfs_disk_dquot_t *);
+					       struct xfs_dquot *);
 extern int		xfs_qm_dqget(xfs_mount_t *, xfs_inode_t *,
 					xfs_dqid_t, uint, uint, xfs_dquot_t **);
 extern void		xfs_qm_dqput(xfs_dquot_t *);
fs/xfs/xfs_qm.c +1 -1
index e5b5cf9..d0acb4e 100644
@@ -1057,7 +1057,7 @@ xfs_qm_quotacheck_dqadjust(
 	 * There are no timers for the default values set in the root dquot.
 	 */
 	if (dqp->q_core.d_id) {
-		xfs_qm_adjust_dqlimits(mp, &dqp->q_core);
+		xfs_qm_adjust_dqlimits(mp, dqp);
 		xfs_qm_adjust_dqtimers(mp, &dqp->q_core);
 	}
 
fs/xfs/xfs_trans_dquot.c +1 -1
index 642c2d6..22d3da3 100644
@@ -412,7 +412,7 @@ xfs_trans_apply_dquot_deltas(
 			 * Start/reset the timer(s) if needed.
 			 */
 			if (d->d_id) {
-				xfs_qm_adjust_dqlimits(tp->t_mountp, d);
+				xfs_qm_adjust_dqlimits(tp->t_mountp, dqp);
 				xfs_qm_adjust_dqtimers(tp->t_mountp, d);
 			}
 
Signature · 6 lines
1.7.7.6

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#7 lore raw

PATCH v4 4/6 xfs: xfs_dquot prealloc throttling watermarks and low free space

In reply to Brian Foster

Enable tracking of high and low watermarks for preallocation throttling of files under quota restrictions. These values are calculated when the quota limit is read from disk or modified and cached for later use by the throttling algorithm.

The high watermark specifies when preallocation is disabled, the low watermark specifies when throttling is enabled and the low free space data structure contains precalculated low free space limits to serve as input to determine the level of throttling required.

Note that the low free space data structure is based on the existing global low free space data structure with the exception of using three stages (5%, 3% and 1%) rather than five to reduce the impact of xfs_dquot memory overhead.

---
 fs/xfs/xfs_dquot.c       |   41 +++++++++++++++++++++++++++++++++++++++--
 fs/xfs/xfs_dquot.h       |   12 ++++++++++++
 fs/xfs/xfs_qm_syscalls.c |    1 +
 3 files changed, 52 insertions(+), 2 deletions(-)
fs/xfs/xfs_dquot.c +39 -2
index e1833b9..e3aa801 100644
@@ -90,13 +90,18 @@ xfs_qm_adjust_dqlimits(
 {
 	xfs_quotainfo_t		*q = mp->m_quotainfo;
 	xfs_disk_dquot_t	*d = &dq->q_core;
+	int			prealloc = 0;
 
 	ASSERT(d->d_id);
 
-	if (q->qi_bsoftlimit && !d->d_blk_softlimit)
+	if (q->qi_bsoftlimit && !d->d_blk_softlimit) {
 		d->d_blk_softlimit = cpu_to_be64(q->qi_bsoftlimit);
-	if (q->qi_bhardlimit && !d->d_blk_hardlimit)
+		prealloc = 1;
+	}
+	if (q->qi_bhardlimit && !d->d_blk_hardlimit) {
 		d->d_blk_hardlimit = cpu_to_be64(q->qi_bhardlimit);
+		prealloc = 1;
+	}
 	if (q->qi_isoftlimit && !d->d_ino_softlimit)
 		d->d_ino_softlimit = cpu_to_be64(q->qi_isoftlimit);
 	if (q->qi_ihardlimit && !d->d_ino_hardlimit)
@@ -105,6 +110,9 @@ xfs_qm_adjust_dqlimits(
 		d->d_rtb_softlimit = cpu_to_be64(q->qi_rtbsoftlimit);
 	if (q->qi_rtbhardlimit && !d->d_rtb_hardlimit)
 		d->d_rtb_hardlimit = cpu_to_be64(q->qi_rtbhardlimit);
+
+	if (prealloc)
+		xfs_dquot_set_prealloc_limits(dq);
 }
 
 /*
@@ -249,6 +257,32 @@ xfs_qm_init_dquot_blk(
 	xfs_trans_log_buf(tp, bp, 0, BBTOB(q->qi_dqchunklen) - 1);
 }
 
+/*
+ * Initialize the dynamic speculative preallocation thresholds. The lo/hi
+ * watermarks correspond to the soft and hard limits by default. If a soft limit
+ * is not specified, we use 95% of the hard limit.
+ */
+void
+xfs_dquot_set_prealloc_limits(struct xfs_dquot *dqp)
+{
+	__uint64_t space;
+
+	dqp->q_prealloc_hi_wmark = be64_to_cpu(dqp->q_core.d_blk_hardlimit);
+	dqp->q_prealloc_lo_wmark = be64_to_cpu(dqp->q_core.d_blk_softlimit);
+	if (!dqp->q_prealloc_lo_wmark) {
+		dqp->q_prealloc_lo_wmark = dqp->q_prealloc_hi_wmark;
+		do_div(dqp->q_prealloc_lo_wmark, 100);
+		dqp->q_prealloc_lo_wmark *= 95;
+	}
+
+	space = dqp->q_prealloc_hi_wmark;
+
+	do_div(space, 100);
+	dqp->q_low_space[XFS_QLOWSP_1_PCNT] = space;
+	dqp->q_low_space[XFS_QLOWSP_3_PCNT] = space * 3;
+	dqp->q_low_space[XFS_QLOWSP_5_PCNT] = space * 5;
+}
+
 static void
 xfs_dquot_buf_verify(
 	struct xfs_buf		*bp)
@@ -649,6 +683,9 @@ xfs_qm_dqread(
 	dqp->q_res_icount = be64_to_cpu(ddqp->d_icount);
 	dqp->q_res_rtbcount = be64_to_cpu(ddqp->d_rtbcount);
 
+	/* initialize the dquot speculative prealloc thresholds */
+	xfs_dquot_set_prealloc_limits(dqp);
+
 	/* Mark the buf so that this will stay incore a little longer */
 	xfs_buf_set_ref(bp, XFS_DQUOT_REF);
 
fs/xfs/xfs_dquot.h +12 -0
index 69491b6..4263f33 100644
@@ -32,6 +32,13 @@
 struct xfs_mount;
 struct xfs_trans;
 
+enum {
+	XFS_QLOWSP_1_PCNT = 0,
+	XFS_QLOWSP_3_PCNT,
+	XFS_QLOWSP_5_PCNT,
+	XFS_QLOWSP_MAX
+};
+
 /*
  * The incore dquot structure
  */
@@ -51,6 +58,9 @@ typedef struct xfs_dquot {
 	xfs_qcnt_t	 q_res_bcount;	/* total regular nblks used+reserved */
 	xfs_qcnt_t	 q_res_icount;	/* total inos allocd+reserved */
 	xfs_qcnt_t	 q_res_rtbcount;/* total realtime blks used+reserved */
+	xfs_qcnt_t	 q_prealloc_lo_wmark;/* prealloc throttle wmark */
+	xfs_qcnt_t	 q_prealloc_hi_wmark;/* prealloc disabled wmark */
+	int64_t		 q_low_space[XFS_QLOWSP_MAX];
 	struct mutex	 q_qlock;	/* quota lock */
 	struct completion q_flush;	/* flush completion queue */
 	atomic_t          q_pincount;	/* dquot pin count */
@@ -153,6 +163,8 @@ extern void		xfs_qm_dqput(xfs_dquot_t *);
 
 extern void		xfs_dqlock2(struct xfs_dquot *, struct xfs_dquot *);
 
+extern void		xfs_dquot_set_prealloc_limits(struct xfs_dquot *);
+
 static inline struct xfs_dquot *xfs_qm_dqhold(struct xfs_dquot *dqp)
 {
 	xfs_dqlock(dqp);
fs/xfs/xfs_qm_syscalls.c +1 -0
index cf9a340..6096eed 100644
@@ -529,6 +529,7 @@ xfs_qm_scall_setqlim(
 	if (hard == 0 || hard >= soft) {
 		ddq->d_blk_hardlimit = cpu_to_be64(hard);
 		ddq->d_blk_softlimit = cpu_to_be64(soft);
+		xfs_dquot_set_prealloc_limits(dqp);
 		if (id == 0) {
 			q->qi_bhardlimit = hard;
 			q->qi_bsoftlimit = soft;
Signature · 6 lines
1.7.7.6

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#8 lore raw

PATCH v4 5/6 xfs: add quota-driven speculative preallocation throttling

In reply to Brian Foster

Introduce the need_throttle() and calc_throttle() functions to independently check whether throttling is required for a particular dquot and if so, calculate the associated throttling metrics based on the state of the quota. We use the same general algorithm to calculate the throttle shift as for global free space with the exception of using three stages rather than five.

Update xfs_iomap_prealloc_size() to use the smallest available prealloc size based on each of the constraints and apply the maximum shift to obtain the throttled preallocation size.

---
 fs/xfs/xfs_iomap.c |   82 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 82 insertions(+), 0 deletions(-)
fs/xfs/xfs_iomap.c +82 -0
index daa08f6..34640a3f 100644
@@ -42,6 +42,8 @@
 #include "xfs_iomap.h"
 #include "xfs_trace.h"
 #include "xfs_icache.h"
+#include "xfs_dquot_item.h"
+#include "xfs_dquot.h"
 
 
 #define XFS_WRITEIO_ALIGN(mp,off)	(((off) >> mp->m_writeio_log) \
@@ -366,6 +368,61 @@ xfs_iomap_eof_prealloc_initial_size(
 	return XFS_B_TO_FSB(mp, offset);
 }
 
+STATIC bool
+xfs_quota_need_throttle(
+	struct xfs_inode *ip,
+	int type,
+	xfs_fsblock_t alloc_blocks)
+{
+	struct xfs_dquot *dq = xfs_inode_dquot(ip, type);
+
+	if (!dq || !xfs_this_quota_on(ip->i_mount, type))
+		return false;
+
+	/* no hi watermark, no throttle */
+	if (!dq->q_prealloc_hi_wmark)
+		return false;
+
+	/* under the lo watermark, no throttle */
+	if (dq->q_res_bcount + alloc_blocks < dq->q_prealloc_lo_wmark)
+		return false;
+
+	return true;
+}
+
+STATIC void
+xfs_quota_calc_throttle(
+	struct xfs_inode *ip,
+	int type,
+	xfs_fsblock_t *qblocks,
+	int *qshift)
+{
+	int64_t freesp;
+	int shift = 0;
+	struct xfs_dquot *dq = xfs_inode_dquot(ip, type);
+
+	/* over hi wmark, squash the prealloc completely */
+	if (dq->q_res_bcount >= dq->q_prealloc_hi_wmark) {
+		*qblocks = 0;
+		return;
+	}
+
+	freesp = dq->q_prealloc_hi_wmark - dq->q_res_bcount;
+	if (freesp < dq->q_low_space[XFS_QLOWSP_5_PCNT]) {
+		shift = 2;
+		if (freesp < dq->q_low_space[XFS_QLOWSP_3_PCNT])
+			shift += 2;
+		if (freesp < dq->q_low_space[XFS_QLOWSP_1_PCNT])
+			shift += 2;
+	}
+
+	/* only overwrite the throttle values if we are more aggressive */
+	if ((freesp >> shift) < (*qblocks >> *qshift)) {
+		*qblocks = freesp;
+		*qshift = shift;
+	}
+}
+
 /*
  * If we don't have a user specified preallocation size, dynamically increase
  * the preallocation size as the size of the file grows. Cap the maximum size
@@ -383,11 +440,14 @@ xfs_iomap_prealloc_size(
 	xfs_fsblock_t		alloc_blocks = 0;
 	int			shift = 0;
 	int64_t			freesp;
+	xfs_fsblock_t		qblocks;
+	int			qshift = 0;
 
 	alloc_blocks = xfs_iomap_eof_prealloc_initial_size(mp, ip, offset,
 							   imap, nimaps);
 	if (!alloc_blocks)
 		goto check_writeio;
+	qblocks = alloc_blocks;
 
 	/*
 	 * MAXEXTLEN is not a power of two value but we round the prealloc down
@@ -412,6 +472,28 @@ xfs_iomap_prealloc_size(
 		if (freesp < mp->m_low_space[XFS_LOWSP_1_PCNT])
 			shift++;
 	}
+
+	/*
+	 * Check each quota to cap the prealloc size and provide a shift
+	 * value to throttle with.
+	 */
+	if (xfs_quota_need_throttle(ip, XFS_DQ_USER, alloc_blocks))
+		xfs_quota_calc_throttle(ip, XFS_DQ_USER, &qblocks, &qshift);
+	if (xfs_quota_need_throttle(ip, XFS_DQ_GROUP, alloc_blocks))
+		xfs_quota_calc_throttle(ip, XFS_DQ_GROUP, &qblocks, &qshift);
+	if (xfs_quota_need_throttle(ip, XFS_DQ_PROJ, alloc_blocks))
+		xfs_quota_calc_throttle(ip, XFS_DQ_PROJ, &qblocks, &qshift);
+
+	/*
+	 * The final prealloc size is set to the minimum of free space available
+	 * in each of the quotas and the overall filesystem.
+	 *
+	 * The shift throttle value is set to the maximum value as determined by
+	 * the global low free space values and per-quota low free space values.
+	 */
+	alloc_blocks = MIN(alloc_blocks, qblocks);
+	shift = MAX(shift, qshift);
+
 	if (shift)
 		alloc_blocks >>= shift;
 	/*
Signature · 6 lines
1.7.7.6

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#9 lore raw

Re: [PATCH v4 5/6] xfs: add quota-driven speculative preallocation throttling

Quoted 35 lines · Brian Foster

Introduce the need_throttle() and calc_throttle() functions to independently check whether throttling is required for a particular dquot and if so, calculate the associated throttling metrics based on the state of the quota. We use the same general algorithm to calculate the throttle shift as for global free space with the exception of using three stages rather than five.

Update xfs_iomap_prealloc_size() to use the smallest available prealloc size based on each of the constraints and apply the maximum shift to obtain the throttled preallocation size.

---

/*
* MAXEXTLEN is not a power of two value but we round the prealloc down

fs/xfs/xfs_iomap.c +15 -0
@@ -412,6 +472,28 @@ xfs_iomap_prealloc_size(
  		if (freesp<  mp->m_low_space[XFS_LOWSP_1_PCNT])
  			shift++;
  	}
+
+	/*
+	 * Check each quota to cap the prealloc size and provide a shift
+	 * value to throttle with.
+	 */
+	if (xfs_quota_need_throttle(ip, XFS_DQ_USER, alloc_blocks))
+		xfs_quota_calc_throttle(ip, XFS_DQ_USER,&qblocks,&qshift);
+	if (xfs_quota_need_throttle(ip, XFS_DQ_GROUP, alloc_blocks))
+		xfs_quota_calc_throttle(ip, XFS_DQ_GROUP,&qblocks,&qshift);
+	if (xfs_quota_need_throttle(ip, XFS_DQ_PROJ, alloc_blocks))
+		xfs_quota_calc_throttle(ip, XFS_DQ_PROJ,&qblocks,&qshift);
+
+	/*
+	 * The final prealloc size is set to the minimum of free space available
+	 * in each of the quotas and the overall filesystem.

fs/xfs/xfs_iomap.c xfs_iomap_prealloc_size( · line 499

Quoted 10 lines
fs/xfs/xfs_iomap.c +7 -0
+	 *
+	 * The shift throttle value is set to the maximum value as determined by
+	 * the global low free space values and per-quota low free space values.
+	 */
+	alloc_blocks = MIN(alloc_blocks, qblocks);
+	shift = MAX(shift, qshift);
+
  	if (shift)
  		alloc_blocks>>= shift;
  	/*

All the limits are applied from previous extents, quota and then the code from commit 055388a3 is applied:

	if (alloc_blocks < mp->m_writeio_blocks)
		alloc_blocks = mp->m_writeio_blocks;

                       ^^^^

we may not have mp->m_writeio_blocks left in the filesytem. Doesn't the following make more sense?:

	if (alloc_blocks < mp->m_writeio_blocks)
		alloc_blocks = 0;

Thank-you,

--Mark

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#10 lore raw

Re: [PATCH v4 5/6] xfs: add quota-driven speculative preallocation throttling

Quoted 53 lines · Mark Tinguely
Quoted 35 lines · Brian Foster

Introduce the need_throttle() and calc_throttle() functions to independently check whether throttling is required for a particular dquot and if so, calculate the associated throttling metrics based on the state of the quota. We use the same general algorithm to calculate the throttle shift as for global free space with the exception of using three stages rather than five.

Update xfs_iomap_prealloc_size() to use the smallest available prealloc size based on each of the constraints and apply the maximum shift to obtain the throttled preallocation size.

---

fs/xfs/xfs_iomap.c +15 -0
 	/*
 	 * MAXEXTLEN is not a power of two value but we round the prealloc down
@@ -412,6 +472,28 @@ xfs_iomap_prealloc_size(
 		if (freesp<  mp->m_low_space[XFS_LOWSP_1_PCNT])
 			shift++;
 	}
+
+	/*
+	 * Check each quota to cap the prealloc size and provide a shift
+	 * value to throttle with.
+	 */
+	if (xfs_quota_need_throttle(ip, XFS_DQ_USER, alloc_blocks))
+		xfs_quota_calc_throttle(ip, XFS_DQ_USER,&qblocks,&qshift);
+	if (xfs_quota_need_throttle(ip, XFS_DQ_GROUP, alloc_blocks))
+		xfs_quota_calc_throttle(ip, XFS_DQ_GROUP,&qblocks,&qshift);
+	if (xfs_quota_need_throttle(ip, XFS_DQ_PROJ, alloc_blocks))
+		xfs_quota_calc_throttle(ip, XFS_DQ_PROJ,&qblocks,&qshift);
+
+	/*
+	 * The final prealloc size is set to the minimum of free space available
+	 * in each of the quotas and the overall filesystem.

fs/xfs/xfs_iomap.c xfs_iomap_prealloc_size( · line 499

Quoted 10 lines
fs/xfs/xfs_iomap.c +7 -0
+	 *
+	 * The shift throttle value is set to the maximum value as determined by
+	 * the global low free space values and per-quota low free space values.
+	 */
+	alloc_blocks = MIN(alloc_blocks, qblocks);
+	shift = MAX(shift, qshift);
+
 	if (shift)
 		alloc_blocks>>= shift;
 	/*

All the limits are applied from previous extents, quota and then the code from commit 055388a3 is applied:

	if (alloc_blocks < mp->m_writeio_blocks)
		alloc_blocks = mp->m_writeio_blocks;

                       ^^^^
Quoted 5 lines

we may not have mp->m_writeio_blocks left in the filesytem. Doesn't the following make more sense?:

	if (alloc_blocks < mp->m_writeio_blocks)
		alloc_blocks = 0;

No. This is for prealloc, and we alwys try to prealloc the minimum configured. If we can't prealloc that amount, then preallocation fails and we'll try a single block.

Remember, the freesp calculation is done unlocked and hence freesp is only an estimate. It may change between the reading of it and the actual allocation attempt, and so the only real determination of ENOSPC is the failure ot the allocation attempt.....

Cheers,

Dave.

Signature · 7 lines
Dave Chinner
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#11 lore raw

Re: [PATCH v4 5/6] xfs: add quota-driven speculative preallocation throttling

Quoted 64 lines · Dave Chinner
Quoted 53 lines · Mark Tinguely
Quoted 35 lines · Brian Foster

Introduce the need_throttle() and calc_throttle() functions to independently check whether throttling is required for a particular dquot and if so, calculate the associated throttling metrics based on the state of the quota. We use the same general algorithm to calculate the throttle shift as for global free space with the exception of using three stages rather than five.

Update xfs_iomap_prealloc_size() to use the smallest available prealloc size based on each of the constraints and apply the maximum shift to obtain the throttled preallocation size.

---

/*
* MAXEXTLEN is not a power of two value but we round the prealloc down

fs/xfs/xfs_iomap.c +15 -0
@@ -412,6 +472,28 @@ xfs_iomap_prealloc_size(
  		if (freesp<   mp->m_low_space[XFS_LOWSP_1_PCNT])
  			shift++;
  	}
+
+	/*
+	 * Check each quota to cap the prealloc size and provide a shift
+	 * value to throttle with.
+	 */
+	if (xfs_quota_need_throttle(ip, XFS_DQ_USER, alloc_blocks))
+		xfs_quota_calc_throttle(ip, XFS_DQ_USER,&qblocks,&qshift);
+	if (xfs_quota_need_throttle(ip, XFS_DQ_GROUP, alloc_blocks))
+		xfs_quota_calc_throttle(ip, XFS_DQ_GROUP,&qblocks,&qshift);
+	if (xfs_quota_need_throttle(ip, XFS_DQ_PROJ, alloc_blocks))
+		xfs_quota_calc_throttle(ip, XFS_DQ_PROJ,&qblocks,&qshift);
+
+	/*
+	 * The final prealloc size is set to the minimum of free space available
+	 * in each of the quotas and the overall filesystem.

fs/xfs/xfs_iomap.c xfs_iomap_prealloc_size( · line 499

Quoted 10 lines
fs/xfs/xfs_iomap.c +7 -0
+	 *
+	 * The shift throttle value is set to the maximum value as determined by
+	 * the global low free space values and per-quota low free space values.
+	 */
+	alloc_blocks = MIN(alloc_blocks, qblocks);
+	shift = MAX(shift, qshift);
+
  	if (shift)
  		alloc_blocks>>= shift;
  	/*

All the limits are applied from previous extents, quota and then the code from commit 055388a3 is applied:

	if (alloc_blocks<  mp->m_writeio_blocks)
		alloc_blocks = mp->m_writeio_blocks;

                        ^^^^
Quoted 5 lines

we may not have mp->m_writeio_blocks left in the filesytem. Doesn't the following make more sense?:

	if (alloc_blocks<  mp->m_writeio_blocks)
		alloc_blocks = 0;

No. This is for prealloc, and we alwys try to prealloc the minimum configured. If we can't prealloc that amount, then preallocation fails and we'll try a single block.

Quoted 8 lines

Remember, the freesp calculation is done unlocked and hence freesp is only an estimate. It may change between the reading of it and the actual allocation attempt, and so the only real determination of ENOSPC is the failure ot the allocation attempt.....

Cheers,

Dave.

Thank-you for the information.

And without the minimum, it will immediately trip the assert:

Assertion failed: last_fsb > offset_fsb, file: /root/xfs/fs/xfs/xfs_iomap.c, line: 590

Sorry for the noise.

--Mark.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#12 lore raw

PATCH v4 6/6 xfs: xfs_iomap_prealloc_size() tracepoint

In reply to Brian Foster

Add a tracepoint to provide some feedback on preallocation size calculation.

---
 fs/xfs/xfs_iomap.c |    3 +++
 fs/xfs/xfs_trace.h |   24 ++++++++++++++++++++++++
 2 files changed, 27 insertions(+), 0 deletions(-)
fs/xfs/xfs_iomap.c +3 -0
index 34640a3f..d5f800c 100644
@@ -518,6 +518,9 @@ check_writeio:
 	if (alloc_blocks < mp->m_writeio_blocks)
 		alloc_blocks = mp->m_writeio_blocks;
 
+	trace_xfs_iomap_prealloc_size(ip, alloc_blocks, shift,
+				      mp->m_writeio_blocks);
+
 	return alloc_blocks;
 }
 
fs/xfs/xfs_trace.h +24 -0
index 16a8129..aa4db33 100644
@@ -619,6 +619,30 @@ DECLARE_EVENT_CLASS(xfs_iref_class,
 		  (char *)__entry->caller_ip)
 )
 
+TRACE_EVENT(xfs_iomap_prealloc_size,
+	TP_PROTO(struct xfs_inode *ip, xfs_fsblock_t blocks, int shift,
+		 unsigned int writeio_blocks),
+	TP_ARGS(ip, blocks, shift, writeio_blocks),
+	TP_STRUCT__entry(
+		__field(dev_t, dev)
+		__field(xfs_ino_t, ino)
+		__field(xfs_fsblock_t, blocks)
+		__field(int, shift)
+		__field(unsigned int, writeio_blocks)
+	),
+	TP_fast_assign(
+		__entry->dev = VFS_I(ip)->i_sb->s_dev;
+		__entry->ino = ip->i_ino;
+		__entry->blocks = blocks;
+		__entry->shift = shift;
+		__entry->writeio_blocks = writeio_blocks;
+	),
+	TP_printk("dev %d:%d ino 0x%llx prealloc blocks %llu shift %d "
+		  "m_writeio_blocks %u",
+		  MAJOR(__entry->dev), MINOR(__entry->dev), __entry->ino,
+		  __entry->blocks, __entry->shift, __entry->writeio_blocks)
+)
+
 #define DEFINE_IREF_EVENT(name) \
 DEFINE_EVENT(xfs_iref_class, name, \
 	TP_PROTO(struct xfs_inode *ip, unsigned long caller_ip), \
Signature · 6 lines
1.7.7.6

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#13 lore raw
In reply to Brian Foster
Quoted 30 lines · Brian Foster

Hi all,

Here is v4 of the speculative preallocation quota throttling set. Patch 3 of v3 is dropped due to a redundancy and a few changes are made to the threshold management code (patch 4/6). The remaining patches are unchanged from v3.

Brian

v4:
- Drop patch 3/7 from v3 (xfs: cap prealloc size to free space before shift).
- Several updates to patch 4/6:
- Rename xfs_dquot_init_prealloc() to xfs_dquot_set_prealloc_limits().
- Unroll the prealloc threshold loop and remove the increment def.
- Fix up some comments.
v3:
- Rebased on top of updated speculative preallocation algorithm. v2:
- Fix up xfs_iomap_prealloc_size() rounding (patch 2).
- Add pre-calculated fields to xfs_dquot to support throttling.
- Move to logarithmic (shift) throttler and finer tuned trigger/throttle logic.

Brian Foster (6):
xfs: reorganize xfs_iomap_prealloc_size to remove indentation
xfs: push rounddown_pow_of_two() to after prealloc throttle
xfs: pass xfs_dquot to xfs_qm_adjust_dqlimits() instead of
xfs_disk_dquot_t
xfs: xfs_dquot prealloc throttling watermarks and low free space
xfs: add quota-driven speculative preallocation throttling
xfs: xfs_iomap_prealloc_size() tracepoint

Quoted 9 lines
  fs/xfs/xfs_dquot.c       |   44 ++++++++++++-
  fs/xfs/xfs_dquot.h       |   14 ++++-
  fs/xfs/xfs_iomap.c       |  161 +++++++++++++++++++++++++++++++++++++---------
  fs/xfs/xfs_qm.c          |    2 +-
  fs/xfs/xfs_qm_syscalls.c |    1 +
  fs/xfs/xfs_trace.h       |   24 +++++++
  fs/xfs/xfs_trans_dquot.c |    2 +-
  7 files changed, 212 insertions(+), 36 deletions(-)

The rest of the series looks good to me. Consider it Review-by: me.

The tracing in patch 6 says how much, but not why, BUT I think the why would require multiple tracepoints, so I okay with it.

-- since I am being a grouch ... ---
I wish I had caught in the review for commit a1e16c26 that xfs_iomap_eof_prealloc_initial_size() is returning an int rather than xfs_fsblock_t for consistency.

--Mark.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#14 lore raw
Quoted 5 lines · Mark Tinguely

-- since I am being a grouch ... ---
I wish I had caught in the review for commit a1e16c26 that xfs_iomap_eof_prealloc_initial_size() is returning an int rather than xfs_fsblock_t for consistency.

Which is a bug, not an inconsistency. It means that alloc_blocks may not return the right thing at the 2^31/2^32 FSB boundary.

Don't grouch like you can't possibly fix it, just send a patch. :P The code isn't upstream yet, so we've got the entire 3.9-rc cycle to fix it... :)

Cheers,

Dave.

Signature · 7 lines
Dave Chinner
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

A message that is not in the archive (sent off-list or not yet received): [email protected]

#16 lore raw

[CC list]

Quoted 11 lines · Mark Tinguely
Quoted 10 lines · Brian Foster

Hi all,

Here is v4 of the speculative preallocation quota throttling set. Patch 3 of v3
is dropped due to a redundancy and a few changes are made to the threshold
management code (patch 4/6). The remaining patches are unchanged from v3.

Brian

...

Quoted 5 lines
Quoted 1 line

Since you are doing a v5 repost, you can clean up the typedefs in the routines that you touch:

Sure, I'll make a pass through the set for such clean ups.

Brian

Quoted 12 lines

For example in patch 3:

        xfs_quotainfo_t         *q = mp->m_quotainfo;
+       xfs_disk_dquot_t        *d = &dq->q_core;
        ^^^^^^^^^
    struct xfs_disk_dquot    d = &dq->q_core;

---
patch 4:

xfs_disk_dquot_t

Quoted 8 lines
        xfs_quotainfo_t         *q = mp->m_quotainfo;
        xfs_disk_dquot_t        *d = &dq->q_core;
+       int                     prealloc = 0;

clean up the xfs_quotainfo_t and xfs_disk_dquot_t

--Mark.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs