Skip to contentSkip to conversation

ceph: fix blindly expanding the readahead windows

PATCH Superseded [email protected] 2 participants 3 messages Mar 23, 2023 – May 4, 2023 Ceph client

v1 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 ceph: fix blindly expanding the readahead windows

From: Xiubo Li <[email protected]>

Blindly expanding the readahead windows will cause unneccessary pagecache thrashing and also will introdue the network workload. We should disable expanding the windows if the readahead is disabled and also shouldn't expand the windows too much.

Expanding forward firstly instead of expanding backward for possible sequential reads.

URL: https://www.spinics.net/lists/ceph-users/msg76183.html

---
 fs/ceph/addr.c | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)
fs/ceph/addr.c +17 -6
index ca4dc6450887..01d997f6c66c 100644
@@ -188,16 +188,27 @@ static void ceph_netfs_expand_readahead(struct netfs_io_request *rreq)
 	struct inode *inode = rreq->inode;
 	struct ceph_inode_info *ci = ceph_inode(inode);
 	struct ceph_file_layout *lo = &ci->i_layout;
+	unsigned long max_pages = inode->i_sb->s_bdi->ra_pages;
+	unsigned long max_len = max_pages << PAGE_SHIFT;
+	unsigned long len;
 	u32 blockoff;
 	u64 blockno;
 
-	/* Expand the start downward */
-	blockno = div_u64_rem(rreq->start, lo->stripe_unit, &blockoff);
-	rreq->start = blockno * lo->stripe_unit;
-	rreq->len += blockoff;
+	/* Readahead is disabled */
+	if (!max_pages)
+		return;
+
+	/* Expand the length forward by rounding up it to the next block */
+	len = roundup(rreq->len, lo->stripe_unit);
+	if (len <= max_len)
+		rreq->len = len;
 
-	/* Now, round up the length to the next block */
-	rreq->len = roundup(rreq->len, lo->stripe_unit);
+	/* Now try to expand the start downward */
+	blockno = div_u64_rem(rreq->start, lo->stripe_unit, &blockoff);
+	if (rreq->len + blockoff <= max_len) {
+		rreq->start = blockno * lo->stripe_unit;
+		rreq->len += blockoff;
+	}
 }
 
 static bool ceph_netfs_clamp_length(struct netfs_io_subrequest *subreq)
Signature · 1 line
2.31.1
maintainer
#1 lore raw
Quoted 33 lines · On Thu, Mar 23, 2023 at 8:01 AM <[email protected]> wrote:

From: Xiubo Li <[email protected]>

Blindly expanding the readahead windows will cause unneccessary pagecache thrashing and also will introdue the network workload. We should disable expanding the windows if the readahead is disabled and also shouldn't expand the windows too much.

Expanding forward firstly instead of expanding backward for possible sequential reads.

URL: https://www.spinics.net/lists/ceph-users/msg76183.html

---
 fs/ceph/addr.c | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)
fs/ceph/addr.c +3 -2
index ca4dc6450887..01d997f6c66c 100644
@@ -188,16 +188,27 @@ static void ceph_netfs_expand_readahead(struct netfs_io_request *rreq)
        struct inode *inode = rreq->inode;
        struct ceph_inode_info *ci = ceph_inode(inode);
        struct ceph_file_layout *lo = &ci->i_layout;
+       unsigned long max_pages = inode->i_sb->s_bdi->ra_pages;
+       unsigned long max_len = max_pages << PAGE_SHIFT;
+       unsigned long len;
        u32 blockoff;
        u64 blockno;
 
-       /* Expand the start downward */
-       blockno = div_u64_rem(rreq->start, lo->stripe_unit, &blockoff);

fs/ceph/addr.c static void ceph_netfs_expand_readahead(struct netfs_io_request *rreq) · line 204

Quoted 10 lines
fs/ceph/addr.c +8 -2
-       rreq->start = blockno * lo->stripe_unit;
-       rreq->len += blockoff;
+       /* Readahead is disabled */
+       if (!max_pages)
+               return;
+
+       /* Expand the length forward by rounding up it to the next block */
+       len = roundup(rreq->len, lo->stripe_unit);
+       if (len <= max_len)
+               rreq->len = len;

Hi Xiubo,

This change makes it possible for the request to be expanded into the next block (i.e. it's not rounded _up_ to the next block as the comment says) because rreq->len is no longer guaranteed to be based off of the start of the block here. Previously that was ensured by the preceding downward expansion.

Thanks,

Ilya

author
#2 lore raw
Quoted 45 lines · Ilya Dryomov
Quoted 32 lines · On Thu, Mar 23, 2023 at 8:01 AM <[email protected]> wrote:

From: Xiubo Li <[email protected]>

Blindly expanding the readahead windows will cause unneccessary pagecache thrashing and also will introdue the network workload. We should disable expanding the windows if the readahead is disabled and also shouldn't expand the windows too much.

Expanding forward firstly instead of expanding backward for possible sequential reads.

URL: https://www.spinics.net/lists/ceph-users/msg76183.html

---
  fs/ceph/addr.c | 23 +++++++++++++++++------
  1 file changed, 17 insertions(+), 6 deletions(-)
fs/ceph/addr.c +3 -2
index ca4dc6450887..01d997f6c66c 100644
@@ -188,16 +188,27 @@ static void ceph_netfs_expand_readahead(struct netfs_io_request *rreq)
         struct inode *inode = rreq->inode;
         struct ceph_inode_info *ci = ceph_inode(inode);
         struct ceph_file_layout *lo = &ci->i_layout;
+       unsigned long max_pages = inode->i_sb->s_bdi->ra_pages;
+       unsigned long max_len = max_pages << PAGE_SHIFT;
+       unsigned long len;
         u32 blockoff;
         u64 blockno;
 
-       /* Expand the start downward */
-       blockno = div_u64_rem(rreq->start, lo->stripe_unit, &blockoff);

fs/ceph/addr.c static void ceph_netfs_expand_readahead(struct netfs_io_request *rreq) · line 204

Quoted 10 lines
fs/ceph/addr.c +8 -2
-       rreq->start = blockno * lo->stripe_unit;
-       rreq->len += blockoff;
+       /* Readahead is disabled */
+       if (!max_pages)
+               return;
+
+       /* Expand the length forward by rounding up it to the next block */
+       len = roundup(rreq->len, lo->stripe_unit);
+       if (len <= max_len)
+               rreq->len = len;

Hi Xiubo,

Quoted 5 lines

This change makes it possible for the request to be expanded into the next block (i.e. it's not rounded _up_ to the next block as the comment says) because rreq->len is no longer guaranteed to be based off of the start of the block here. Previously that was ensured by the preceding downward expansion.

Correct. I will fix it.

Thanks Ilya.

- Xiubo

Quoted 4 lines

Thanks,

Ilya