Skip to contentSkip to conversation

ceph: switch to 4KB block size if quota size is not aligned to 4MB

PATCH Superseded [email protected] 3 participants 9 messages Jun 27, 2022 – Jun 30, 2022 Ceph client

lore
Share & tools
lore
#0 lore raw

v2 cover ceph: switch to 4KB block size if quota size is not aligned to 4MB

From: Xiubo Li <[email protected]>

V2:
- Switched to IS_ALIGNED() macro
- Added CEPH_4K_BLOCK_SIZE macro
- Rename CEPH_BLOCK to CEPH_BLOCK_SIZE

Xiubo Li (3):
ceph: make f_bsize always equal to f_frsize
ceph: switch to use CEPH_4K_BLOCK_SHIFT macro
ceph: switch to 4KB block size if quota size is not aligned to 4MB

 fs/ceph/quota.c | 32 ++++++++++++++++++++------------
 fs/ceph/super.c | 28 ++++++++++++++--------------
 fs/ceph/super.h |  5 +++--
 3 files changed, 37 insertions(+), 28 deletions(-)
Signature · 1 line
2.36.0.rc1
#1 lore raw

PATCH v2 1/3 ceph: make f_bsize always equal to f_frsize

From: Xiubo Li <[email protected]>

The f_frsize maybe changed in the quota size is less than the defualt 4MB.

---
 fs/ceph/super.c | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)
fs/ceph/super.c +11 -11
index 74f9cc5f37e9..88d7e67130b8 100644
@@ -70,17 +70,6 @@ static int ceph_statfs(struct dentry *dentry, struct kstatfs *buf)
 
 	/* fill in kstatfs */
 	buf->f_type = CEPH_SUPER_MAGIC;  /* ?? */
-
-	/*
-	 * express utilization in terms of large blocks to avoid
-	 * overflow on 32-bit machines.
-	 *
-	 * NOTE: for the time being, we make bsize == frsize to humor
-	 * not-yet-ancient versions of glibc that are broken.
-	 * Someday, we will probably want to report a real block
-	 * size...  whatever that may mean for a network file system!
-	 */
-	buf->f_bsize = 1 << CEPH_BLOCK_SHIFT;
 	buf->f_frsize = 1 << CEPH_BLOCK_SHIFT;
 
 	/*
@@ -95,6 +84,17 @@ static int ceph_statfs(struct dentry *dentry, struct kstatfs *buf)
 		buf->f_bavail = le64_to_cpu(st.kb_avail) >> (CEPH_BLOCK_SHIFT-10);
 	}
 
+	/*
+	 * express utilization in terms of large blocks to avoid
+	 * overflow on 32-bit machines.
+	 *
+	 * NOTE: for the time being, we make bsize == frsize to humor
+	 * not-yet-ancient versions of glibc that are broken.
+	 * Someday, we will probably want to report a real block
+	 * size...  whatever that may mean for a network file system!
+	 */
+	buf->f_bsize = buf->f_frsize;
+
 	buf->f_files = le64_to_cpu(st.num_objects);
 	buf->f_ffree = -1;
 	buf->f_namelen = NAME_MAX;
Signature · 1 line
2.36.0.rc1
#2 lore raw

PATCH v2 2/3 ceph: switch to use CEPH_4K_BLOCK_SHIFT macro

In reply to [email protected]

From: Xiubo Li <[email protected]>

At the same time rename CEPH_BLOCK to CEPH_BLOCK_SIZE.

---
 fs/ceph/super.c | 8 ++++----
 fs/ceph/super.h | 5 +++--
 2 files changed, 7 insertions(+), 6 deletions(-)
fs/ceph/super.c +4 -4
index 88d7e67130b8..ba835db374a3 100644
@@ -70,7 +70,7 @@ static int ceph_statfs(struct dentry *dentry, struct kstatfs *buf)
 
 	/* fill in kstatfs */
 	buf->f_type = CEPH_SUPER_MAGIC;  /* ?? */
-	buf->f_frsize = 1 << CEPH_BLOCK_SHIFT;
+	buf->f_frsize = CEPH_BLOCK_SIZE;
 
 	/*
 	 * By default use root quota for stats; fallback to overall filesystem
@@ -79,9 +79,9 @@ static int ceph_statfs(struct dentry *dentry, struct kstatfs *buf)
 	 */
 	if (ceph_test_mount_opt(fsc, NOQUOTADF) ||
 	    !ceph_quota_update_statfs(fsc, buf)) {
-		buf->f_blocks = le64_to_cpu(st.kb) >> (CEPH_BLOCK_SHIFT-10);
-		buf->f_bfree = le64_to_cpu(st.kb_avail) >> (CEPH_BLOCK_SHIFT-10);
-		buf->f_bavail = le64_to_cpu(st.kb_avail) >> (CEPH_BLOCK_SHIFT-10);
+		buf->f_blocks = le64_to_cpu(st.kb) >> CEPH_4K_BLOCK_SHIFT;
+		buf->f_bfree = le64_to_cpu(st.kb_avail) >> CEPH_4K_BLOCK_SHIFT;
+		buf->f_bavail = le64_to_cpu(st.kb_avail) >> CEPH_4K_BLOCK_SHIFT;
 	}
 
 	/*
fs/ceph/super.h +3 -2
index 9bc34c31831b..02115ed59ff1 100644
@@ -25,9 +25,10 @@
 
 /* large granularity for statfs utilization stats to facilitate
  * large volume sizes on 32-bit machines. */
-#define CEPH_BLOCK_SHIFT   22  /* 4 MB */
-#define CEPH_BLOCK         (1 << CEPH_BLOCK_SHIFT)
+#define CEPH_BLOCK_SHIFT    22  /* 4 MB */
+#define CEPH_BLOCK_SIZE     (1 << CEPH_BLOCK_SHIFT)
 #define CEPH_4K_BLOCK_SHIFT 12  /* 4 KB */
+#define CEPH_4K_BLOCK_SIZE  (1 << CEPH_4K_BLOCK_SHIFT)
 
 #define CEPH_MOUNT_OPT_CLEANRECOVER    (1<<1) /* auto reonnect (clean mode) after blocklisted */
 #define CEPH_MOUNT_OPT_DIRSTAT         (1<<4) /* `cat dirname` for stats */
Signature · 1 line
2.36.0.rc1
#3 lore raw

PATCH v2 3/3 ceph: switch to 4KB block size if quota size is not aligned to 4MB

In reply to [email protected]

From: Xiubo Li <[email protected]>

If the quota size is larger than but not aligned to 4MB, the statfs will always set the block size to 4MB and round down the fragment size. For exmaple if the quota size is 6MB, the df will always show 4MB capacity.

Make the block size to 4KB as default if quota size is set unless the quota size is larger than or equals to 4MB and at the same time it aligns to 4MB.

---
 fs/ceph/quota.c | 32 ++++++++++++++++++++------------
 1 file changed, 20 insertions(+), 12 deletions(-)
fs/ceph/quota.c +20 -12
index 64592adfe48f..37fe86bfb919 100644
@@ -483,6 +483,7 @@ bool ceph_quota_update_statfs(struct ceph_fs_client *fsc, struct kstatfs *buf)
 	struct inode *in;
 	u64 total = 0, used, free;
 	bool is_updated = false;
+	u32 block_shift = CEPH_4K_BLOCK_SHIFT;
 
 	down_read(&mdsc->snap_rwsem);
 	realm = get_quota_realm(mdsc, d_inode(fsc->sb->s_root),
@@ -498,25 +499,32 @@ bool ceph_quota_update_statfs(struct ceph_fs_client *fsc, struct kstatfs *buf)
 		ci = ceph_inode(in);
 		spin_lock(&ci->i_ceph_lock);
 		if (ci->i_max_bytes) {
-			total = ci->i_max_bytes >> CEPH_BLOCK_SHIFT;
-			used = ci->i_rbytes >> CEPH_BLOCK_SHIFT;
-			/* For quota size less than 4MB, use 4KB block size */
-			if (!total) {
-				total = ci->i_max_bytes >> CEPH_4K_BLOCK_SHIFT;
-				used = ci->i_rbytes >> CEPH_4K_BLOCK_SHIFT;
-	                        buf->f_frsize = 1 << CEPH_4K_BLOCK_SHIFT;
-			}
-			/* It is possible for a quota to be exceeded.
+			/*
+			 * Switch to 4MB block size if quota size is
+			 * larger than or equals to 4MB and at the
+			 * same time is aligned to 4MB.
+			 */
+			if (IS_ALIGNED(ci->i_max_bytes, CEPH_BLOCK_SIZE))
+				block_shift = CEPH_BLOCK_SHIFT;
+
+			total = ci->i_max_bytes >> block_shift;
+			used = ci->i_rbytes >> block_shift;
+			buf->f_frsize = 1 << block_shift;
+
+			/*
+			 * It is possible for a quota to be exceeded.
 			 * Report 'zero' in that case
 			 */
 			free = total > used ? total - used : 0;
-			/* For quota size less than 4KB, report the
+			/*
+			 * For quota size less than 4KB, report the
 			 * total=used=4KB,free=0 when quota is full
-			 * and total=free=4KB, used=0 otherwise */
+			 * and total=free=4KB, used=0 otherwise
+			 */
 			if (!total) {
 				total = 1;
 				free = ci->i_max_bytes > ci->i_rbytes ? 1 : 0;
-	                        buf->f_frsize = 1 << CEPH_4K_BLOCK_SHIFT;
+	                        buf->f_frsize = CEPH_4K_BLOCK_SIZE;
 			}
 		}
 		spin_unlock(&ci->i_ceph_lock);
Signature · 1 line
2.36.0.rc1
Reviewed-by on the v2 series
#4 lore raw
In reply to [email protected]
Quoted 12 lines · On Mon, 2022-06-27 at 10:02 +0800, [email protected] wrote:

From: Xiubo Li <[email protected]>

V2:
- Switched to IS_ALIGNED() macro
- Added CEPH_4K_BLOCK_SIZE macro
- Rename CEPH_BLOCK to CEPH_BLOCK_SIZE

Xiubo Li (3):
ceph: make f_bsize always equal to f_frsize
ceph: switch to use CEPH_4K_BLOCK_SHIFT macro
ceph: switch to 4KB block size if quota size is not aligned to 4MB

Quoted 5 lines
 fs/ceph/quota.c | 32 ++++++++++++++++++++------------
 fs/ceph/super.c | 28 ++++++++++++++--------------
 fs/ceph/super.h |  5 +++--
 3 files changed, 37 insertions(+), 28 deletions(-)
#5 lore raw
In reply to [email protected]

NACK, this change will significantly increase mds load. Inaccuracy is inherent in current quota design.

Quoted 21 lines · On Mon, Jun 27, 2022 at 10:06 AM <[email protected]> wrote:

From: Xiubo Li <[email protected]>

V2:
- Switched to IS_ALIGNED() macro
- Added CEPH_4K_BLOCK_SIZE macro
- Rename CEPH_BLOCK to CEPH_BLOCK_SIZE

Xiubo Li (3):
ceph: make f_bsize always equal to f_frsize
ceph: switch to use CEPH_4K_BLOCK_SHIFT macro
ceph: switch to 4KB block size if quota size is not aligned to 4MB

 fs/ceph/quota.c | 32 ++++++++++++++++++++------------
 fs/ceph/super.c | 28 ++++++++++++++--------------
 fs/ceph/super.h |  5 +++--
 3 files changed, 37 insertions(+), 28 deletions(-)

--
2.36.0.rc1

author
#6 lore raw
Quoted 2 lines · Zheng

NACK, this change will significantly increase mds load. Inaccuracy is inherent in current quota design.

Yeah, I was also thinking could we just allow the quota size to be aligned to 4KB if it < 4MB, or must be aligned to 4MB ?

Any idea ?

- Xiubo

Quoted 21 lines · On Mon, Jun 27, 2022 at 10:06 AM <[email protected]> wrote:
Quoted 20 lines

From: Xiubo Li <[email protected]>

V2:
- Switched to IS_ALIGNED() macro
- Added CEPH_4K_BLOCK_SIZE macro
- Rename CEPH_BLOCK to CEPH_BLOCK_SIZE

Xiubo Li (3):
ceph: make f_bsize always equal to f_frsize
ceph: switch to use CEPH_4K_BLOCK_SHIFT macro
ceph: switch to 4KB block size if quota size is not aligned to 4MB

  fs/ceph/quota.c | 32 ++++++++++++++++++++------------
  fs/ceph/super.c | 28 ++++++++++++++--------------
  fs/ceph/super.h |  5 +++--
  3 files changed, 37 insertions(+), 28 deletions(-)

--
2.36.0.rc1

#7 lore raw
Quoted 10 lines · Xiubo Li
Quoted 2 lines · Zheng

NACK, this change will significantly increase mds load. Inaccuracy is inherent in current quota design.

Yeah, I was also thinking could we just allow the quota size to be aligned to 4KB if it < 4MB, or must be aligned to 4MB ?

Any idea ?

make sense

Quoted 26 lines

- Xiubo

Quoted 21 lines · On Mon, Jun 27, 2022 at 10:06 AM <[email protected]> wrote:
Quoted 20 lines

From: Xiubo Li <[email protected]>

V2:
- Switched to IS_ALIGNED() macro
- Added CEPH_4K_BLOCK_SIZE macro
- Rename CEPH_BLOCK to CEPH_BLOCK_SIZE

Xiubo Li (3):
ceph: make f_bsize always equal to f_frsize
ceph: switch to use CEPH_4K_BLOCK_SHIFT macro
ceph: switch to 4KB block size if quota size is not aligned to 4MB

  fs/ceph/quota.c | 32 ++++++++++++++++++++------------
  fs/ceph/super.c | 28 ++++++++++++++--------------
  fs/ceph/super.h |  5 +++--
  3 files changed, 37 insertions(+), 28 deletions(-)

--
2.36.0.rc1

author
#8 lore raw
Quoted 10 lines · Zheng
Quoted 8 lines · Xiubo Li
Quoted 2 lines · Zheng

NACK, this change will significantly increase mds load. Inaccuracy is inherent in current quota design.

Yeah, I was also thinking could we just allow the quota size to be aligned to 4KB if it < 4MB, or must be aligned to 4MB ?

Any idea ?

make sense

Cool, Thanks Zheng.

I will work on that.

-- Xiubo

Quoted 24 lines
Quoted 24 lines

- Xiubo

Quoted 21 lines · On Mon, Jun 27, 2022 at 10:06 AM <[email protected]> wrote:
Quoted 20 lines

From: Xiubo Li <[email protected]>

V2:
- Switched to IS_ALIGNED() macro
- Added CEPH_4K_BLOCK_SIZE macro
- Rename CEPH_BLOCK to CEPH_BLOCK_SIZE

Xiubo Li (3):
ceph: make f_bsize always equal to f_frsize
ceph: switch to use CEPH_4K_BLOCK_SHIFT macro
ceph: switch to 4KB block size if quota size is not aligned to 4MB

   fs/ceph/quota.c | 32 ++++++++++++++++++++------------
   fs/ceph/super.c | 28 ++++++++++++++--------------
   fs/ceph/super.h |  5 +++--
   3 files changed, 37 insertions(+), 28 deletions(-)

--
2.36.0.rc1