ceph: fix blindly expanding the readahead windows
Subsystems (2)
PATCH v3 1/1 ceph: fix blindly expanding the readahead windows
From: Xiubo Li <[email protected]>
Blindly expanding the readahead windows will cause unneccessary pagecache thrashing and also will introdue the network workload. We should disable expanding the windows if the readahead is disabled and also shouldn't expand the windows too much.
Expanding forward firstly instead of expanding backward for possible sequential reads.
Bound rreq->len to the actual file size to restore the previous page cache usage.
URL: https://lore.kernel.org/ceph-devel/[email protected]
URL: https://www.spinics.net/lists/ceph-users/msg76183.html
---
V3:
- Folded Hu Weiwen's fix and bound rreq->len to the actual file size.
fs/ceph/addr.c | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-)
fs/ceph/addr.c +22 -6
@@ -188,16 +188,32 @@ static void ceph_netfs_expand_readahead(struct netfs_io_request *rreq)struct inode *inode = rreq->inode;struct ceph_inode_info *ci =ceph_inode (inode);struct ceph_file_layout *lo = &ci->i_layout; +unsigned long max_pages = inode->i_sb->s_bdi->ra_pages; +unsigned long max_len = max_pages <<PAGE_SHIFT ; +loff_t end = rreq->start + rreq->len, new_end;u32 blockoff;u64 blockno; -/* Expand the start downward */ - blockno =div_u64_rem (rreq->start, lo->stripe_unit, &blockoff); - rreq->start = blockno * lo->stripe_unit; - rreq->len += blockoff; +/* Readahead is disabled */ +if (!max_pages) +return ; -/* Now, round up the length to the next block */ - rreq->len =roundup (rreq->len, lo->stripe_unit); +/* +* Try to expand the length forward by rounding up it to the next +* block, but do not exceed the file size, unless the original +* request already exceeds it. +*/ + new_end =round_up (end, lo->stripe_unit); + new_end =min (new_end, rreq->i_size); +if (new_end > end && new_end <= rreq->start + max_len) + rreq->len = new_end - rreq->start; + +/* Try to expand the start downward */ + blockno =div_u64_rem (rreq->start, lo->stripe_unit, &blockoff); +if (rreq->len + blockoff <= max_len) { + rreq->start = blockno * lo->stripe_unit; + rreq->len += blockoff; + } }static bool ceph_netfs_clamp_length (struct netfs_io_subrequest *subreq)
Signature · 1 line
2.40.0
Ilya Dryomov
maintainerQuoted 16 lines · On Thu, May 4, 2023 at 1:11 PM <[email protected]> wrote:
From: Xiubo Li <[email protected]>
Blindly expanding the readahead windows will cause unneccessary pagecache thrashing and also will introdue the network workload. We should disable expanding the windows if the readahead is disabled and also shouldn't expand the windows too much.
Expanding forward firstly instead of expanding backward for possible sequential reads.
Bound
rreq->lento the actual file size to restore the previous page cache usage.Fixes 49870056005c ("ceph: convert ceph_readpages to ceph_readahead")
Quoted 8 lines
URL: https://lore.kernel.org/ceph-devel/[email protected]
URL: https://www.spinics.net/lists/ceph-users/msg76183.htmlCc Hu Weiwen <[email protected]>Signed-off-by Xiubo Li <[email protected]>---
V3:
- Folded Hu Weiwen's fix and boundrreq->lento the actual file size.
Hi Xiubo,
This looks much better! Just a couple of nits:
Quoted 27 lines
fs/ceph/addr.c | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-)fs/ceph/addr.c +6
-4@@ -188,16 +188,32 @@ static void ceph_netfs_expand_readahead(struct netfs_io_request *rreq)struct inode *inode = rreq->inode;struct ceph_inode_info *ci =ceph_inode (inode);struct ceph_file_layout *lo = &ci->i_layout; +unsigned long max_pages = inode->i_sb->s_bdi->ra_pages; +unsigned long max_len = max_pages <<PAGE_SHIFT ; +loff_t end = rreq->start + rreq->len, new_end;u32 blockoff;u64 blockno; -/* Expand the start downward */ - blockno =div_u64_rem (rreq->start, lo->stripe_unit, &blockoff); - rreq->start = blockno * lo->stripe_unit; - rreq->len += blockoff; +/* Readahead is disabled */ +if (!max_pages) +return ;
fs/ceph/addr.c static void ceph_netfs_expand_readahead(struct netfs_io_request *rreq) · line 207
Quoted 10 lines
fs/ceph/addr.c +7
-2-/* Now, round up the length to the next block */ - rreq->len =roundup (rreq->len, lo->stripe_unit); +/* +* Try to expand the length forward by rounding up it to the next +* block, but do not exceed the file size, unless the original +* request already exceeds it. +*/ + new_end =round_up (end, lo->stripe_unit); + new_end =min (new_end, rreq->i_size);
This can be done on single line:
new_end = min(round_up(end, lo->stripe_unit), rreq->i_size);
fs/ceph/addr.c static void ceph_netfs_expand_readahead(struct netfs_io_request *rreq) · line 214
Quoted 7 lines
fs/ceph/addr.c +7
-0+if (new_end > end && new_end <= rreq->start + max_len) + rreq->len = new_end - rreq->start; + +/* Try to expand the start downward */ + blockno =div_u64_rem (rreq->start, lo->stripe_unit, &blockoff); +if (rreq->len + blockoff <= max_len) { + rreq->start = blockno * lo->stripe_unit;
Can this be written as:
rreq->start -= blockoff;
It seems like it would be easier to read and probably cheaper to compute too.
Thanks,
Ilya
Quoted 66 lines · Ilya Dryomov
Quoted 15 lines · On Thu, May 4, 2023 at 1:11 PM <[email protected]> wrote:
From: Xiubo Li <[email protected]>
Blindly expanding the readahead windows will cause unneccessary pagecache thrashing and also will introdue the network workload. We should disable expanding the windows if the readahead is disabled and also shouldn't expand the windows too much.
Expanding forward firstly instead of expanding backward for possible sequential reads.
Bound
rreq->lento the actual file size to restore the previous page cache usage.Fixes 49870056005c ("ceph: convert ceph_readpages to ceph_readahead")Quoted 8 lines
URL: https://lore.kernel.org/ceph-devel/[email protected]
URL: https://www.spinics.net/lists/ceph-users/msg76183.htmlCc Hu Weiwen <[email protected]>Signed-off-by Xiubo Li <[email protected]>---
V3:
- Folded Hu Weiwen's fix and boundrreq->lento the actual file size.Hi Xiubo,
This looks much better! Just a couple of nits:
Quoted 26 lines
fs/ceph/addr.c | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-)fs/ceph/addr.c +6
-4@@ -188,16 +188,32 @@ static void ceph_netfs_expand_readahead(struct netfs_io_request *rreq)struct inode *inode = rreq->inode;struct ceph_inode_info *ci =ceph_inode (inode);struct ceph_file_layout *lo = &ci->i_layout; +unsigned long max_pages = inode->i_sb->s_bdi->ra_pages; +unsigned long max_len = max_pages <<PAGE_SHIFT ; +loff_t end = rreq->start + rreq->len, new_end;u32 blockoff;u64 blockno; -/* Expand the start downward */ - blockno =div_u64_rem (rreq->start, lo->stripe_unit, &blockoff); - rreq->start = blockno * lo->stripe_unit; - rreq->len += blockoff; +/* Readahead is disabled */ +if (!max_pages) +return ;fs/ceph/addr.c static void ceph_netfs_expand_readahead(struct netfs_io_request *rreq) · line 207
Quoted 10 lines
fs/ceph/addr.c +7
-2-/* Now, round up the length to the next block */ - rreq->len =roundup (rreq->len, lo->stripe_unit); +/* +* Try to expand the length forward by rounding up it to the next +* block, but do not exceed the file size, unless the original +* request already exceeds it. +*/ + new_end =round_up (end, lo->stripe_unit); + new_end =min (new_end, rreq->i_size);This can be done on single line:
Quoted 1 line
new_end = min(round_up(end, lo->stripe_unit), rreq->i_size);
Sure, will fix it.
Quoted 7 lines
fs/ceph/addr.c static void ceph_netfs_expand_readahead(struct netfs_io_request *rreq) · line 214
Quoted 7 lines
fs/ceph/addr.c +7
-0+if (new_end > end && new_end <= rreq->start + max_len) + rreq->len = new_end - rreq->start; + +/* Try to expand the start downward */ + blockno =div_u64_rem (rreq->start, lo->stripe_unit, &blockoff); +if (rreq->len + blockoff <= max_len) { + rreq->start = blockno * lo->stripe_unit;
Quoted 6 lines
Can this be written as:
rreq->start -= blockoff;It seems like it would be easier to read and probably cheaper to compute too.
Yeah, this looks much easier to understand.
Thanks
- Xiubo
Quoted 4 lines
Thanks,
Ilya