ceph: switch to 4KB block size if quota size is not aligned to 4MB
v2 cover ceph: switch to 4KB block size if quota size is not aligned to 4MB
From: Xiubo Li <[email protected]>
V2:
- Switched to IS_ALIGNED() macro
- Added CEPH_4K_BLOCK_SIZE macro
- Rename CEPH_BLOCK to CEPH_BLOCK_SIZE
Xiubo Li (3):
ceph: make f_bsize always equal to f_frsize
ceph: switch to use CEPH_4K_BLOCK_SHIFT macro
ceph: switch to 4KB block size if quota size is not aligned to 4MB
fs/ceph/quota.c | 32 ++++++++++++++++++++------------ fs/ceph/super.c | 28 ++++++++++++++-------------- fs/ceph/super.h | 5 +++-- 3 files changed, 37 insertions(+), 28 deletions(-)
Signature · 1 line
2.36.0.rc1
PATCH v2 1/3 ceph: make f_bsize always equal to f_frsize
From: Xiubo Li <[email protected]>
The f_frsize maybe changed in the quota size is less than the defualt 4MB.
--- fs/ceph/super.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-)
fs/ceph/super.c +11 -11
@@ -70,17 +70,6 @@ static int ceph_statfs(struct dentry *dentry, struct kstatfs *buf)/* fill in kstatfs */ buf->f_type =CEPH_SUPER_MAGIC ;/* ?? */ - -/* -* express utilization in terms of large blocks to avoid -* overflow on 32-bit machines. -* -* NOTE: for the time being, we make bsize == frsize to humor -* not-yet-ancient versions of glibc that are broken. -* Someday, we will probably want to report a real block -* size... whatever that may mean for a network file system! -*/ - buf->f_bsize =1 <<CEPH_BLOCK_SHIFT ; buf->f_frsize =1 <<CEPH_BLOCK_SHIFT ;/*
@@ -95,6 +84,17 @@ static int ceph_statfs(struct dentry *dentry, struct kstatfs *buf) buf->f_bavail =le64_to_cpu (st.kb_avail) >> (CEPH_BLOCK_SHIFT -10 ); } +/* +* express utilization in terms of large blocks to avoid +* overflow on 32-bit machines. +* +* NOTE: for the time being, we make bsize == frsize to humor +* not-yet-ancient versions of glibc that are broken. +* Someday, we will probably want to report a real block +* size... whatever that may mean for a network file system! +*/ + buf->f_bsize = buf->f_frsize; + buf->f_files =le64_to_cpu (st.num_objects); buf->f_ffree =-1 ; buf->f_namelen =NAME_MAX ;
Signature · 1 line
2.36.0.rc1
PATCH v2 2/3 ceph: switch to use CEPH_4K_BLOCK_SHIFT macro
In reply to [email protected]From: Xiubo Li <[email protected]>
At the same time rename CEPH_BLOCK to CEPH_BLOCK_SIZE.
--- fs/ceph/super.c | 8 ++++---- fs/ceph/super.h | 5 +++-- 2 files changed, 7 insertions(+), 6 deletions(-)
fs/ceph/super.c +4 -4
@@ -70,7 +70,7 @@ static int ceph_statfs(struct dentry *dentry, struct kstatfs *buf)/* fill in kstatfs */ buf->f_type =CEPH_SUPER_MAGIC ;/* ?? */ - buf->f_frsize =1 <<CEPH_BLOCK_SHIFT ; + buf->f_frsize =CEPH_BLOCK_SIZE ;/* * By default use root quota for stats; fallback to overall filesystem
@@ -79,9 +79,9 @@ static int ceph_statfs(struct dentry *dentry, struct kstatfs *buf)*/ if (ceph_test_mount_opt (fsc,NOQUOTADF ) || !ceph_quota_update_statfs (fsc, buf)) { - buf->f_blocks =le64_to_cpu (st.kb) >> (CEPH_BLOCK_SHIFT -10 ); - buf->f_bfree =le64_to_cpu (st.kb_avail) >> (CEPH_BLOCK_SHIFT -10 ); - buf->f_bavail =le64_to_cpu (st.kb_avail) >> (CEPH_BLOCK_SHIFT -10 ); + buf->f_blocks =le64_to_cpu (st.kb) >>CEPH_4K_BLOCK_SHIFT ; + buf->f_bfree =le64_to_cpu (st.kb_avail) >>CEPH_4K_BLOCK_SHIFT ; + buf->f_bavail =le64_to_cpu (st.kb_avail) >>CEPH_4K_BLOCK_SHIFT ; }/*
fs/ceph/super.h +3 -2
@@ -25,9 +25,10 @@/* large granularity for statfs utilization stats to facilitate * large volume sizes on 32-bit machines. */ -#define CEPH_BLOCK_SHIFT 22/* 4 MB */ -#define CEPH_BLOCK (1 << CEPH_BLOCK_SHIFT) +#define CEPH_BLOCK_SHIFT 22/* 4 MB */ +#define CEPH_BLOCK_SIZE (1 << CEPH_BLOCK_SHIFT)#define CEPH_4K_BLOCK_SHIFT 12/* 4 KB */ +#define CEPH_4K_BLOCK_SIZE (1 << CEPH_4K_BLOCK_SHIFT)#define CEPH_MOUNT_OPT_CLEANRECOVER (1<<1)/* auto reonnect (clean mode) after blocklisted */ #define CEPH_MOUNT_OPT_DIRSTAT (1<<4)/* `cat dirname` for stats */
Signature · 1 line
2.36.0.rc1
PATCH v2 3/3 ceph: switch to 4KB block size if quota size is not aligned to 4MB
In reply to [email protected]From: Xiubo Li <[email protected]>
If the quota size is larger than but not aligned to 4MB, the statfs will always set the block size to 4MB and round down the fragment size. For exmaple if the quota size is 6MB, the df will always show 4MB capacity.
Make the block size to 4KB as default if quota size is set unless the quota size is larger than or equals to 4MB and at the same time it aligns to 4MB.
--- fs/ceph/quota.c | 32 ++++++++++++++++++++------------ 1 file changed, 20 insertions(+), 12 deletions(-)
fs/ceph/quota.c +20 -12
@@ -483,6 +483,7 @@ bool ceph_quota_update_statfs(struct ceph_fs_client *fsc, struct kstatfs *buf)struct inode *in;u64 total =0 , used, free;bool is_updated = false; +u32 block_shift =CEPH_4K_BLOCK_SHIFT ;down_read (&mdsc->snap_rwsem); realm = get_quota_realm(mdsc, d_inode(fsc->sb->s_root),
@@ -498,25 +499,32 @@ bool ceph_quota_update_statfs(struct ceph_fs_client *fsc, struct kstatfs *buf) ci =ceph_inode (in);spin_lock (&ci->i_ceph_lock);if (ci->i_max_bytes) { - total = ci->i_max_bytes >>CEPH_BLOCK_SHIFT ; - used = ci->i_rbytes >>CEPH_BLOCK_SHIFT ; -/* For quota size less than 4MB, use 4KB block size */ -if (!total) { - total = ci->i_max_bytes >>CEPH_4K_BLOCK_SHIFT ; - used = ci->i_rbytes >>CEPH_4K_BLOCK_SHIFT ; - buf->f_frsize =1 <<CEPH_4K_BLOCK_SHIFT ; - } -/* It is possible for a quota to be exceeded. +/* +* Switch to 4MB block size if quota size is +* larger than or equals to 4MB and at the +* same time is aligned to 4MB. +*/ +if (IS_ALIGNED (ci->i_max_bytes,CEPH_BLOCK_SIZE )) + block_shift =CEPH_BLOCK_SHIFT ; + + total = ci->i_max_bytes >> block_shift; + used = ci->i_rbytes >> block_shift; + buf->f_frsize =1 << block_shift; + +/* +* It is possible for a quota to be exceeded. * Report 'zero' in that case */ free = total > used ? total - used :0 ; -/* For quota size less than 4KB, report the +/* +* For quota size less than 4KB, report the * total=used=4KB,free=0 when quota is full -* and total=free=4KB, used=0 otherwise */ +* and total=free=4KB, used=0 otherwise +*/ if (!total) { total =1 ; free = ci->i_max_bytes > ci->i_rbytes ?1 :0 ; - buf->f_frsize =1 <<CEPH_4K_BLOCK_SHIFT ; + buf->f_frsize =CEPH_4K_BLOCK_SIZE ; } }spin_unlock (&ci->i_ceph_lock);
Signature · 1 line
2.36.0.rc1
Quoted 12 lines · On Mon, 2022-06-27 at 10:02 +0800, [email protected] wrote:
From: Xiubo Li <[email protected]>
V2:
- Switched toIS_ALIGNED()macro
- AddedCEPH_4K_BLOCK_SIZEmacro
- RenameCEPH_BLOCKtoCEPH_BLOCK_SIZEXiubo Li (3):
ceph: makef_bsizealways equal tof_frsize
ceph: switch to useCEPH_4K_BLOCK_SHIFTmacro
ceph: switch to 4KB block size if quota size is not aligned to 4MB
Quoted 5 lines
fs/ceph/quota.c | 32 ++++++++++++++++++++------------ fs/ceph/super.c | 28 ++++++++++++++-------------- fs/ceph/super.h | 5 +++-- 3 files changed, 37 insertions(+), 28 deletions(-)
Yan, Zheng
NAKNACK, this change will significantly increase mds load. Inaccuracy is inherent in current quota design.
Quoted 21 lines · On Mon, Jun 27, 2022 at 10:06 AM <[email protected]> wrote:
From: Xiubo Li <[email protected]>
V2:
- Switched toIS_ALIGNED()macro
- AddedCEPH_4K_BLOCK_SIZEmacro
- RenameCEPH_BLOCKtoCEPH_BLOCK_SIZEXiubo Li (3):
ceph: makef_bsizealways equal tof_frsize
ceph: switch to useCEPH_4K_BLOCK_SHIFTmacro
ceph: switch to 4KB block size if quota size is not aligned to 4MBfs/ceph/quota.c | 32 ++++++++++++++++++++------------ fs/ceph/super.c | 28 ++++++++++++++-------------- fs/ceph/super.h | 5 +++-- 3 files changed, 37 insertions(+), 28 deletions(-)--
2.36.0.rc1
Quoted 2 lines · Zheng
NACK, this change will significantly increase mds load. Inaccuracy is inherent in current quota design.
Yeah, I was also thinking could we just allow the quota size to be aligned to 4KB if it < 4MB, or must be aligned to 4MB ?
Any idea ?
- Xiubo
Quoted 21 lines · On Mon, Jun 27, 2022 at 10:06 AM <[email protected]> wrote:
Quoted 20 lines
From: Xiubo Li <[email protected]>
V2:
- Switched toIS_ALIGNED()macro
- AddedCEPH_4K_BLOCK_SIZEmacro
- RenameCEPH_BLOCKtoCEPH_BLOCK_SIZEXiubo Li (3):
ceph: makef_bsizealways equal tof_frsize
ceph: switch to useCEPH_4K_BLOCK_SHIFTmacro
ceph: switch to 4KB block size if quota size is not aligned to 4MBfs/ceph/quota.c | 32 ++++++++++++++++++++------------ fs/ceph/super.c | 28 ++++++++++++++-------------- fs/ceph/super.h | 5 +++-- 3 files changed, 37 insertions(+), 28 deletions(-)--
2.36.0.rc1
Quoted 10 lines · Xiubo Li
Quoted 2 lines · Zheng
NACK, this change will significantly increase mds load. Inaccuracy is inherent in current quota design.
Yeah, I was also thinking could we just allow the quota size to be aligned to 4KB if it < 4MB, or must be aligned to 4MB ?
Any idea ?
make sense
Quoted 26 lines
- Xiubo
Quoted 21 lines · On Mon, Jun 27, 2022 at 10:06 AM <[email protected]> wrote:
Quoted 20 lines
From: Xiubo Li <[email protected]>
V2:
- Switched toIS_ALIGNED()macro
- AddedCEPH_4K_BLOCK_SIZEmacro
- RenameCEPH_BLOCKtoCEPH_BLOCK_SIZEXiubo Li (3):
ceph: makef_bsizealways equal tof_frsize
ceph: switch to useCEPH_4K_BLOCK_SHIFTmacro
ceph: switch to 4KB block size if quota size is not aligned to 4MBfs/ceph/quota.c | 32 ++++++++++++++++++++------------ fs/ceph/super.c | 28 ++++++++++++++-------------- fs/ceph/super.h | 5 +++-- 3 files changed, 37 insertions(+), 28 deletions(-)--
2.36.0.rc1
Quoted 10 lines · Zheng
Quoted 8 lines · Xiubo Li
Quoted 2 lines · Zheng
NACK, this change will significantly increase mds load. Inaccuracy is inherent in current quota design.
Yeah, I was also thinking could we just allow the quota size to be aligned to 4KB if it < 4MB, or must be aligned to 4MB ?
Any idea ?
make sense
Cool, Thanks Zheng.
I will work on that.
-- Xiubo
Quoted 24 lines
Quoted 24 lines
- Xiubo
Quoted 21 lines · On Mon, Jun 27, 2022 at 10:06 AM <[email protected]> wrote:
Quoted 20 lines
From: Xiubo Li <[email protected]>
V2:
- Switched toIS_ALIGNED()macro
- AddedCEPH_4K_BLOCK_SIZEmacro
- RenameCEPH_BLOCKtoCEPH_BLOCK_SIZEXiubo Li (3):
ceph: makef_bsizealways equal tof_frsize
ceph: switch to useCEPH_4K_BLOCK_SHIFTmacro
ceph: switch to 4KB block size if quota size is not aligned to 4MBfs/ceph/quota.c | 32 ++++++++++++++++++++------------ fs/ceph/super.c | 28 ++++++++++++++-------------- fs/ceph/super.h | 5 +++-- 3 files changed, 37 insertions(+), 28 deletions(-)--
2.36.0.rc1