zram: Add a huge_idle writeback mode
Subsystems (5)
PATCH 1/1 zram: Add a huge_idle writeback mode
Today it's only possible to write back as a page, idle, or huge. A user might want to writeback pages which are huge and idle first as these idle pages do not require decompression and make a good first pass for writeback.
--- Documentation/admin-guide/blockdev/zram.rst | 6 ++++++ drivers/block/zram/zram_drv.c | 10 ++++++---- 2 files changed, 12 insertions(+), 4 deletions(-)
Documentation/admin-guide/blockdev/zram.rst +6 -0
@@ -343,6 +343,12 @@ Admin can request writeback of those idle pages at right timing via:: With the command, zram writeback idle pages from memory to the storage. +Additionally, if a user choose to writeback only huge and idle pages +this can be accomplished with:: + + echo huge_idle > /sys/block/zramX/writeback + + If admin want to write a specific page in zram device to backing device, they could write a page index into the interface.
drivers/block/zram/zram_drv.c +6 -4
@@ -643,8 +643,8 @@ static int read_from_bdev_async(struct zram *zram, struct bio_vec *bvec,#define PAGE_WB_SIG "page_index="#define PAGE_WRITEBACK 0 -#define HUGE_WRITEBACK 1 -#define IDLE_WRITEBACK 2 +#define HUGE_WRITEBACK (1<<0) +#define IDLE_WRITEBACK (1<<1) static ssize_t writeback_store(struct device *dev,
@@ -664,6 +664,8 @@ static ssize_t writeback_store(struct device *dev, mode =IDLE_WRITEBACK ; else if (sysfs_streq(buf, "huge")) mode =HUGE_WRITEBACK ; + else if (sysfs_streq(buf, "huge_idle")) + mode =IDLE_WRITEBACK |HUGE_WRITEBACK ; else {if (strncmp (buf,PAGE_WB_SIG ,sizeof (PAGE_WB_SIG ) -1 ))return -EINVAL ;
@@ -725,10 +727,10 @@ static ssize_t writeback_store(struct device *dev, zram_test_flag(zram, index, ZRAM_UNDER_WB)) goto next; -if (mode ==IDLE_WRITEBACK && +if (mode &IDLE_WRITEBACK && !zram_test_flag (zram, index,ZRAM_IDLE )) goto next; -if (mode ==HUGE_WRITEBACK && +if (mode &HUGE_WRITEBACK && !zram_test_flag (zram, index,ZRAM_HUGE )) goto next;/*
Signature · 1 line
2.35.1.723.g4982287a31-goog
Quoted 4 lines · Brian Geffon
Today it's only possible to write back as a page, idle, or huge. A user might want to writeback pages which are huge and idle first as these idle pages do not require decompression and make a good first pass for writeback.
We're moving towards having many different sizes of page in play, not just PMD and PTE sizes. Is this patch actually a good idea in a case where we have, eg, a 32kB anonymous page on a system with 4kB pages? How should zram handle this case? What's our cut-off for declaring a page to be "huge"?
Quoted 7 lines · Matthew Wilcox
Quoted 4 lines · Brian Geffon
Today it's only possible to write back as a page, idle, or huge. A user might want to writeback pages which are huge and idle first as these idle pages do not require decompression and make a good first pass for writeback.
Quoted 6 lines
We're moving towards having many different sizes of page in play, not just PMD and PTE sizes. Is this patch actually a good idea in a case where we have, eg, a 32kB anonymous page on a system with 4kB pages? How should zram handle this case? What's our cut-off for declaring a page to be "huge"?
Huge isn't a great term IMO, but it is what it is. ZRAM_HUGE is used to identify pages which are incompressible. Since zram is a block device which presents PAGE_SIZED blocks, do these new changes which involve many different page sizes matter as that seems orthogonal to the block subsystem. Correct me if I'm misunderstanding.
Thanks
Brian
Quoted 15 lines · Brian Geffon
Quoted 7 lines · Matthew Wilcox
Quoted 4 lines · Brian Geffon
Today it's only possible to write back as a page, idle, or huge. A user might want to writeback pages which are huge and idle first as these idle pages do not require decompression and make a good first pass for writeback.
Quoted 6 lines
We're moving towards having many different sizes of page in play, not just PMD and PTE sizes. Is this patch actually a good idea in a case where we have, eg, a 32kB anonymous page on a system with 4kB pages? How should zram handle this case? What's our cut-off for declaring a page to be "huge"?
Quoted 5 lines
Huge isn't a great term IMO, but it is what it is.
ZRAM_HUGEis used to identify pages which are incompressible. Since zram is a block device which presentsPAGE_SIZEDblocks, do these new changes which involve many different page sizes matter as that seems orthogonal to the block subsystem. Correct me if I'm misunderstanding.
Oh, so ZRAM's concept of huge is not the same as the "huge" in "hugetlbfs" or "THP"? That's not at all confusing ...
Quoted 17 lines · Matthew Wilcox
Quoted 15 lines · Brian Geffon
Quoted 13 lines · Matthew Wilcox
Quoted 4 lines · Brian Geffon
Today it's only possible to write back as a page, idle, or huge. A user might want to writeback pages which are huge and idle first as these idle pages do not require decompression and make a good first pass for writeback.
We're moving towards having many different sizes of page in play, not just PMD and PTE sizes. Is this patch actually a good idea in a case where we have, eg, a 32kB anonymous page on a system with 4kB pages? How should zram handle this case? What's our cut-off for declaring a page to be "huge"?
Quoted 8 lines
Quoted 5 lines
Huge isn't a great term IMO, but it is what it is.
ZRAM_HUGEis used to identify pages which are incompressible. Since zram is a block device which presentsPAGE_SIZEDblocks, do these new changes which involve many different page sizes matter as that seems orthogonal to the block subsystem. Correct me if I'm misunderstanding.Oh, so ZRAM's concept of huge is not the same as the "huge" in "hugetlbfs" or "THP"? That's not at all confusing ...
I do not disagree, but there isn't much that can be done about it at this point given the sysfs file takes an argument called "huge"
Minchan Kim
maintainerQuoted 4 lines · Brian Geffon
Today it's only possible to write back as a page, idle, or huge. A user might want to writeback pages which are huge and idle first as these idle pages do not require decompression and make a good first pass for writeback.
Hi Brian,
I am not sure how much the decompression overhead matter for idle pages writeback since it's already *very slow* path in zram but I agree that it would be a good first pass since the memory saving for huge writing would be cost efficient.
Just out of curiosity. Do you have real usecase?
Quoted 64 lines
Signed-off-by Brian Geffon <[email protected]>--- Documentation/admin-guide/blockdev/zram.rst | 6 ++++++ drivers/block/zram/zram_drv.c | 10 ++++++---- 2 files changed, 12 insertions(+), 4 deletions(-)Documentation/admin-guide/blockdev/zram.rst +6
-0@@ -343,6 +343,12 @@ Admin can request writeback of those idle pages at right timing via:: With the command, zram writeback idle pages from memory to the storage. +Additionally, if a user choose to writeback only huge and idle pages +this can be accomplished with:: + + echo huge_idle > /sys/block/zramX/writeback + + If admin want to write a specific page in zram device to backing device, they could write a page index into the interface.drivers/block/zram/zram_drv.c +6
-4@@ -643,8 +643,8 @@ static int read_from_bdev_async(struct zram *zram, struct bio_vec *bvec,#define PAGE_WB_SIG "page_index="#define PAGE_WRITEBACK 0 -#define HUGE_WRITEBACK 1 -#define IDLE_WRITEBACK 2 +#define HUGE_WRITEBACK (1<<0) +#define IDLE_WRITEBACK (1<<1) static ssize_t writeback_store(struct device *dev,@@ -664,6 +664,8 @@ static ssize_t writeback_store(struct device *dev, mode =IDLE_WRITEBACK ; else if (sysfs_streq(buf, "huge")) mode =HUGE_WRITEBACK ; + else if (sysfs_streq(buf, "huge_idle")) + mode =IDLE_WRITEBACK |HUGE_WRITEBACK ; else {if (strncmp (buf,PAGE_WB_SIG ,sizeof (PAGE_WB_SIG ) -1 ))return -EINVAL ;@@ -725,10 +727,10 @@ static ssize_t writeback_store(struct device *dev, zram_test_flag(zram, index, ZRAM_UNDER_WB)) goto next; -if (mode ==IDLE_WRITEBACK && +if (mode &IDLE_WRITEBACK && !zram_test_flag (zram, index,ZRAM_IDLE )) goto next; -if (mode ==HUGE_WRITEBACK && +if (mode &HUGE_WRITEBACK && !zram_test_flag (zram, index,ZRAM_HUGE )) goto next;/* Signature · 1 line
2.35.1.723.g4982287a31-goog
Quoted 7 lines · Minchan Kim
Quoted 4 lines · Brian Geffon
Today it's only possible to write back as a page, idle, or huge. A user might want to writeback pages which are huge and idle first as these idle pages do not require decompression and make a good first pass for writeback.
Quoted 8 lines
Hi Brian,
I am not sure how much the decompression overhead matter for idle pages writeback since it's already *very slow* path in zram but I agree that it would be a good first pass since the memory saving for huge writing would be cost efficient.
Just out of curiosity. Do you have real usecase?
Hi Minchan,
Thank you for taking a look. When we are thinking about writeback we're trying to be very sensitive to our devices storage endurance, for this reason we will have a fairly conservative writeback limit. Given that, we want to make sure we're maximizing what lands on disk while still minimizing the refault time. We could take the approach where we always writeback huge pages but then we may result in very quick refaults which would be a huge waste of time. So idle writeback is a must for us and being able to writeback the pages which have maximum value (huge) would be very useful.
Brian
Quoted 65 lines
Quoted 65 lines
Signed-off-by Brian Geffon <[email protected]>--- Documentation/admin-guide/blockdev/zram.rst | 6 ++++++ drivers/block/zram/zram_drv.c | 10 ++++++---- 2 files changed, 12 insertions(+), 4 deletions(-)Documentation/admin-guide/blockdev/zram.rst +6
-0@@ -343,6 +343,12 @@ Admin can request writeback of those idle pages at right timing via:: With the command, zram writeback idle pages from memory to the storage. +Additionally, if a user choose to writeback only huge and idle pages +this can be accomplished with:: + + echo huge_idle > /sys/block/zramX/writeback + + If admin want to write a specific page in zram device to backing device, they could write a page index into the interface.drivers/block/zram/zram_drv.c +6
-5@@ -643,8 +643,8 @@ static int read_from_bdev_async(struct zram *zram, struct bio_vec *bvec,#define PAGE_WB_SIG "page_index="#define PAGE_WRITEBACK 0 -#define HUGE_WRITEBACK 1 -#define IDLE_WRITEBACK 2 +#define HUGE_WRITEBACK (1<<0) +#define IDLE_WRITEBACK (1<<1) static ssize_t writeback_store(struct device *dev,@@ -664,6 +664,8 @@ static ssize_t writeback_store(struct device *dev, mode =IDLE_WRITEBACK ; else if (sysfs_streq(buf, "huge")) mode =HUGE_WRITEBACK ; + else if (sysfs_streq(buf, "huge_idle")) + mode =IDLE_WRITEBACK |HUGE_WRITEBACK ; else {if (strncmp (buf,PAGE_WB_SIG ,sizeof (PAGE_WB_SIG ) -1 ))return -EINVAL ;@@ -725,10 +727,10 @@ static ssize_t writeback_store(struct device *dev, zram_test_flag(zram, index, ZRAM_UNDER_WB)) goto next; -if (mode ==IDLE_WRITEBACK && +if (mode &IDLE_WRITEBACK && !zram_test_flag (zram, index,ZRAM_IDLE )) goto next; -if (mode ==HUGE_WRITEBACK && +if (mode &HUGE_WRITEBACK && !zram_test_flag (zram, index,ZRAM_HUGE )) goto next;/* -- 2.35.1.723.g4982287a31-goog
Minchan Kim
maintainerQuoted 17 lines · Brian Geffon
Quoted 7 lines · Minchan Kim
Quoted 4 lines · Brian Geffon
Today it's only possible to write back as a page, idle, or huge. A user might want to writeback pages which are huge and idle first as these idle pages do not require decompression and make a good first pass for writeback.
Quoted 8 lines
Hi Brian,
I am not sure how much the decompression overhead matter for idle pages writeback since it's already *very slow* path in zram but I agree that it would be a good first pass since the memory saving for huge writing would be cost efficient.
Just out of curiosity. Do you have real usecase?
Quoted 10 lines
Hi Minchan,
Thank you for taking a look. When we are thinking about writeback we're trying to be very sensitive to our devices storage endurance, for this reason we will have a fairly conservative writeback limit. Given that, we want to make sure we're maximizing what lands on disk while still minimizing the refault time. We could take the approach where we always writeback huge pages but then we may result in very quick refaults which would be a huge waste of time. So idle writeback is a must for us and being able to writeback the pages which have maximum value (huge) would be very useful.
Thanks for sharing the thought. It really make sense to me and would be great if it goes on the description.
PATCH 1/1 zram: Add a huge_idle writeback mode
In reply to Brian GeffonToday it's only possible to write back as a page, idle, or huge. A user might want to writeback pages which are huge and idle first as these idle pages do not require decompression and make a good first pass for writeback.
Idle writeback specifically has the advantage that a refault is unlikely given that the page has been swapped for some amount of time without being refaulted.
Huge writeback has the advantage that you're guaranteed to get the maximum benefit from a single page writeback, that is, you're reclaiming one full page of memory. Pages which are compressed in zram being written back result in some benefit which is always less than a page size because of the fact that it was compressed.
This change allows for users to write back huge pages which are also idle.
--- Documentation/admin-guide/blockdev/zram.rst | 6 ++++++ drivers/block/zram/zram_drv.c | 10 ++++++---- 2 files changed, 12 insertions(+), 4 deletions(-)
Documentation/admin-guide/blockdev/zram.rst +6 -0
@@ -343,6 +343,12 @@ Admin can request writeback of those idle pages at right timing via:: With the command, zram writeback idle pages from memory to the storage. +Additionally, if a user choose to writeback only huge and idle pages +this can be accomplished with:: + + echo huge_idle > /sys/block/zramX/writeback + + If admin want to write a specific page in zram device to backing device, they could write a page index into the interface.
drivers/block/zram/zram_drv.c +6 -4
@@ -643,8 +643,8 @@ static int read_from_bdev_async(struct zram *zram, struct bio_vec *bvec,#define PAGE_WB_SIG "page_index="#define PAGE_WRITEBACK 0 -#define HUGE_WRITEBACK 1 -#define IDLE_WRITEBACK 2 +#define HUGE_WRITEBACK (1<<0) +#define IDLE_WRITEBACK (1<<1) static ssize_t writeback_store(struct device *dev,
@@ -664,6 +664,8 @@ static ssize_t writeback_store(struct device *dev, mode =IDLE_WRITEBACK ; else if (sysfs_streq(buf, "huge")) mode =HUGE_WRITEBACK ; + else if (sysfs_streq(buf, "huge_idle")) + mode =IDLE_WRITEBACK |HUGE_WRITEBACK ; else {if (strncmp (buf,PAGE_WB_SIG ,sizeof (PAGE_WB_SIG ) -1 ))return -EINVAL ;
@@ -725,10 +727,10 @@ static ssize_t writeback_store(struct device *dev, zram_test_flag(zram, index, ZRAM_UNDER_WB)) goto next; -if (mode ==IDLE_WRITEBACK && +if (mode &IDLE_WRITEBACK && !zram_test_flag (zram, index,ZRAM_IDLE )) goto next; -if (mode ==HUGE_WRITEBACK && +if (mode &HUGE_WRITEBACK && !zram_test_flag (zram, index,ZRAM_HUGE )) goto next;/*
Signature · 1 line
2.35.1.894.gb6a874cedc-goog
Quoted 9 lines · Brian Geffon
Today it's only possible to write back as a page, idle, or huge. A user might want to writeback pages which are huge and idle first as these idle pages do not require decompression and make a good first pass for writeback.
Idle writeback specifically has the advantage that a refault is unlikely given that the page has been swapped for some amount of time without being refaulted.
Quoted 8 lines
Huge writeback has the advantage that you're guaranteed to get the maximum benefit from a single page writeback, that is, you're reclaiming one full page of memory. Pages which are compressed in zram being written back result in some benefit which is always less than a page size because of the fact that it was compressed.
This change allows for users to write back huge pages which are also idle.
Hey Brian,
I really want to add your explanation about the storage endurance because it's real issue.
So, could't you add up below in the description?
From your previous reply
"
we're trying to be very sensitive to our devices storage endurance, for this reason we will have a fairly conservative writeback limit. Given that, we want to make sure we're maximizing what lands on disk while still minimizing the refault time. We could take the approach where we always writeback huge pages but then we may result in very quick refaults which would be a huge waste of time. So idle writeback is a must for us and being able to writeback the pages which have maximum value (huge) would be very useful
"
Quoted 2 lines
Signed-off-by Brian Geffon <[email protected]>
Other than that, feel free to add my
Thanks.
Quoted 20 lines · Minchan Kim
Quoted 9 lines · Brian Geffon
Today it's only possible to write back as a page, idle, or huge. A user might want to writeback pages which are huge and idle first as these idle pages do not require decompression and make a good first pass for writeback.
Idle writeback specifically has the advantage that a refault is unlikely given that the page has been swapped for some amount of time without being refaulted.
Quoted 8 lines
Huge writeback has the advantage that you're guaranteed to get the maximum benefit from a single page writeback, that is, you're reclaiming one full page of memory. Pages which are compressed in zram being written back result in some benefit which is always less than a page size because of the fact that it was compressed.
This change allows for users to write back huge pages which are also idle.
Quoted 6 lines
Hey Brian,
I really want to add your explanation about the storage endurance because it's real issue.
So, could't you add up below in the description?
Sure thing.
Quoted 14 lines
From your previous reply
"
we're trying to be very sensitive to our devices storage endurance, for this reason we will have a fairly conservative writeback limit. Given that, we want to make sure we're maximizing what lands on disk while still minimizing the refault time. We could take the approach where we always writeback huge pages but then we may result in very quick refaults which would be a huge waste of time. So idle writeback is a must for us and being able to writeback the pages which have maximum value (huge) would be very useful
"Quoted 1 line
Quoted 4 lines
Quoted 1 line
Signed-off-by Brian Geffon <[email protected]>Other than that, feel free to add my
Acked-by Minchan Kim <[email protected]>
Thanks Minchan.
Quoted 2 lines
Thanks.
PATCH v2 1/1 zram: Add a huge_idle writeback mode
In reply to Brian GeffonToday it's only possible to write back as a page, idle, or huge. A user might want to writeback pages which are huge and idle first as these idle pages do not require decompression and make a good first pass for writeback.
Idle writeback specifically has the advantage that a refault is unlikely given that the page has been swapped for some amount of time without being refaulted.
Huge writeback has the advantage that you're guaranteed to get the maximum benefit from a single page writeback, that is, you're reclaiming one full page of memory. Pages which are compressed in zram being written back result in some benefit which is always less than a page size because of the fact that it was compressed.
The primary use of this is for minimizing refaults in situations where the device has to be sensitive to storage endurance. On ChromeOS we have devices with slow eMMC and repeated writes and refaults can negatively affect performance and endurance.
--- Documentation/admin-guide/blockdev/zram.rst | 5 +++++ drivers/block/zram/zram_drv.c | 10 ++++++---- 2 files changed, 11 insertions(+), 4 deletions(-)
Documentation/admin-guide/blockdev/zram.rst +5 -0
@@ -343,6 +343,11 @@ Admin can request writeback of those idle pages at right timing via:: With the command, zram will writeback idle pages from memory to the storage. +Additionally, if a user choose to writeback only huge and idle pages +this can be accomplished with:: + + echo huge_idle > /sys/block/zramX/writeback + If an admin wants to write a specific page in zram device to the backing device, they could write a page index into the interface.
drivers/block/zram/zram_drv.c +6 -4
@@ -639,8 +639,8 @@ static int read_from_bdev_async(struct zram *zram, struct bio_vec *bvec,#define PAGE_WB_SIG "page_index="#define PAGE_WRITEBACK 0 -#define HUGE_WRITEBACK 1 -#define IDLE_WRITEBACK 2 +#define HUGE_WRITEBACK (1<<0) +#define IDLE_WRITEBACK (1<<1) static ssize_t writeback_store(struct device *dev,
@@ -660,6 +660,8 @@ static ssize_t writeback_store(struct device *dev, mode =IDLE_WRITEBACK ; else if (sysfs_streq(buf, "huge")) mode =HUGE_WRITEBACK ; + else if (sysfs_streq(buf, "huge_idle")) + mode =IDLE_WRITEBACK |HUGE_WRITEBACK ; else {if (strncmp (buf,PAGE_WB_SIG ,sizeof (PAGE_WB_SIG ) -1 ))return -EINVAL ;
@@ -721,10 +723,10 @@ static ssize_t writeback_store(struct device *dev, zram_test_flag(zram, index, ZRAM_UNDER_WB)) goto next; -if (mode ==IDLE_WRITEBACK && +if (mode &IDLE_WRITEBACK && !zram_test_flag (zram, index,ZRAM_IDLE )) goto next; -if (mode ==HUGE_WRITEBACK && +if (mode &HUGE_WRITEBACK && !zram_test_flag (zram, index,ZRAM_HUGE )) goto next;/*
Signature · 1 line
2.35.1.894.gb6a874cedc-goog