FAILED: patch "[PATCH] erofs: fix unsafe pagevec reuse of hooked pclusters" failed to apply to 5.10-stable tree
Subsystems (2)
[email protected]
maintainerThe patch below does not apply to the 5.10-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.
thanks,
greg k-h
------------------ original commit in Linus's tree ------------------
From 86432a6dca9bed79111990851df5756d3eb5f57c Mon Sep 17 00:00:00 2001
From: Gao Xiang <[email protected]>
Date: Thu, 4 Nov 2021 02:20:06 +0800
Subject: [PATCH] erofs: fix unsafe pagevec reuse of hooked pclusters
There are pclusters in runtime marked with Z_EROFS_PCLUSTER_TAIL before actual I/O submission. Thus, the decompression chain can be extended if the following pcluster chain hooks such tail pcluster.
As the related comment mentioned, if some page is made of a hooked pcluster and another followed pcluster, it can be reused for in-place I/O (since I/O should be submitted anyway):
_______________________________________________________________
| tail (partial) page | head (partial) page | |_____PRIMARY_HOOKED___|____________PRIMARY_FOLLOWED____________|
However, it's by no means safe to reuse as pagevec since if such PRIMARY_HOOKED pclusters finally move into bypass chain without I/O submission. It's somewhat hard to reproduce with LZ4 and I just found it (general protection fault) by ro_fsstressing a LZMA image for long time.
I'm going to actively clean up related code together with multi-page folio adaption in the next few months. Let's address it directly for easier backporting for now.
Call trace for reference:
z_erofs_decompress_pcluster+0x10a/0x8a0 [erofs] z_erofs_decompress_queue.isra.36+0x3c/0x60 [erofs] z_erofs_runqueue+0x5f3/0x840 [erofs] z_erofs_readahead+0x1e8/0x320 [erofs] read_pages+0x91/0x270 page_cache_ra_unbounded+0x18b/0x240 filemap_get_pages+0x10a/0x5f0 filemap_read+0xa9/0x330 new_sync_read+0x11b/0x1a0 vfs_read+0xf1/0x190
fs/erofs/zdata.c +7 -6
@@ -373,8 +373,8 @@ static bool z_erofs_try_inplace_io(struct z_erofs_collector *clt,/* callers must be with collection lock held */ static int z_erofs_attach_page (struct z_erofs_collector *clt, -struct page *page, -enum z_erofs_page_type type) +struct page *page,enum z_erofs_page_type type, +bool pvec_safereuse) {int ret;
@@ -384,9 +384,9 @@ static int z_erofs_attach_page(struct z_erofs_collector *clt, z_erofs_try_inplace_io(clt, page))return 0 ; - ret =z_erofs_pagevec_enqueue (&clt->vector, page, type); + ret =z_erofs_pagevec_enqueue (&clt->vector, page, type, + pvec_safereuse); clt->cl->vcnt += (unsigned int )ret; -return ret ?0 : -EAGAIN ; }
@@ -729,7 +729,8 @@ static int z_erofs_do_read_page(struct z_erofs_decompress_frontend *fe, tight &= (clt->mode >=COLLECT_PRIMARY_FOLLOWED ); retry: - err =z_erofs_attach_page (clt, page, page_type); + err =z_erofs_attach_page (clt, page, page_type, + clt->mode >=COLLECT_PRIMARY_FOLLOWED );/* should allocate an additional short-lived page for pagevec */ if (err == -EAGAIN ) { struct page *const newpage =
@@ -737,7 +738,7 @@ static int z_erofs_do_read_page(struct z_erofs_decompress_frontend *fe,set_page_private (newpage,Z_EROFS_SHORTLIVED_PAGE ); err =z_erofs_attach_page (clt, newpage, -Z_EROFS_PAGE_TYPE_EXCLUSIVE ); +Z_EROFS_PAGE_TYPE_EXCLUSIVE , true);if (!err) goto retry; }
fs/erofs/zpvec.h +10 -3
@@ -106,11 +106,18 @@ static inline void z_erofs_pagevec_ctor_init(struct z_erofs_pagevec_ctor *ctor,static inline bool z_erofs_pagevec_enqueue (struct z_erofs_pagevec_ctor *ctor,struct page *page, -enum z_erofs_page_type type) +enum z_erofs_page_type type, +bool pvec_safereuse) { -if (!ctor->next && type) -if (ctor->index +1 == ctor->nr) +if (!ctor->next) { +/* some pages cannot be reused as pvec safely without I/O */ +if (type ==Z_EROFS_PAGE_TYPE_EXCLUSIVE && !pvec_safereuse) + type =Z_EROFS_VLE_PAGE_TYPE_TAIL_SHARED ; + +if (type !=Z_EROFS_PAGE_TYPE_EXCLUSIVE && + ctor->index +1 == ctor->nr)return false; + }if (ctor->index >= ctor->nr)z_erofs_pagevec_ctor_pagedown (ctor, false);
PATCH 1/2 erofs: remove the occupied parameter from z_erofs_pagevec_enqueue()
From: Yue Hu <[email protected]>
commit 7dea3de7d384f4c8156e8bd93112ba6db1eb276c upstream.
No any behavior to variable occupied in z_erofs_attach_page() which is only caller to z_erofs_pagevec_enqueue().
---
Gao Xiang: Apply this trivial cleanup (no behavior change) for easier
backporting.
fs/erofs/zdata.c | 4 +--- fs/erofs/zpvec.h | 5 +---- 2 files changed, 2 insertions(+), 7 deletions(-)
fs/erofs/zdata.c +1 -3
@@ -282,7 +282,6 @@ static int z_erofs_attach_page(struct z_erofs_collector *clt, enum z_erofs_page_type type) {int ret; -bool occupied;/* give priority for inplaceio */ if (clt->mode >= COLLECT_PRIMARY &&
@@ -290,8 +289,7 @@ static int z_erofs_attach_page(struct z_erofs_collector *clt, z_erofs_try_inplace_io(clt, page))return 0 ; - ret =z_erofs_pagevec_enqueue (&clt->vector, - page, type, &occupied); + ret =z_erofs_pagevec_enqueue (&clt->vector, page, type); clt->cl->vcnt += (unsigned int )ret;return ret ?0 : -EAGAIN ;
fs/erofs/zpvec.h +1 -4
@@ -107,10 +107,8 @@ static inline void z_erofs_pagevec_ctor_init(struct z_erofs_pagevec_ctor *ctor,static inline bool z_erofs_pagevec_enqueue (struct z_erofs_pagevec_ctor *ctor,struct page *page, -enum z_erofs_page_type type, -bool *occupied) +enum z_erofs_page_type type) { - *occupied = false;if (!ctor->next && type)if (ctor->index +1 == ctor->nr)return false;
@@ -125,7 +123,6 @@ static inline bool z_erofs_pagevec_enqueue(struct z_erofs_pagevec_ctor *ctor,/* should remind that collector->next never equal to 1, 2 */ if (type == (uintptr_t )ctor->next) { ctor->next = page; - *occupied = true; } ctor->pages[ctor->index++] =tagptr_fold (erofs_vtptr_t, page, type);return true;
Signature · 1 line
2.24.4
PATCH 2/2 erofs: fix unsafe pagevec reuse of hooked pclusters
commit 86432a6dca9bed79111990851df5756d3eb5f57c upstream.
There are pclusters in runtime marked with Z_EROFS_PCLUSTER_TAIL before actual I/O submission. Thus, the decompression chain can be extended if the following pcluster chain hooks such tail pcluster.
As the related comment mentioned, if some page is made of a hooked pcluster and another followed pcluster, it can be reused for in-place I/O (since I/O should be submitted anyway):
_______________________________________________________________
| tail (partial) page | head (partial) page | |_____PRIMARY_HOOKED___|____________PRIMARY_FOLLOWED____________|
However, it's by no means safe to reuse as pagevec since if such PRIMARY_HOOKED pclusters finally move into bypass chain without I/O submission. It's somewhat hard to reproduce with LZ4 and I just found it (general protection fault) by ro_fsstressing a LZMA image for long time.
I'm going to actively clean up related code together with multi-page folio adaption in the next few months. Let's address it directly for easier backporting for now.
Call trace for reference:
z_erofs_decompress_pcluster+0x10a/0x8a0 [erofs] z_erofs_decompress_queue.isra.36+0x3c/0x60 [erofs] z_erofs_runqueue+0x5f3/0x840 [erofs] z_erofs_readahead+0x1e8/0x320 [erofs] read_pages+0x91/0x270 page_cache_ra_unbounded+0x18b/0x240 filemap_get_pages+0x10a/0x5f0 filemap_read+0xa9/0x330 new_sync_read+0x11b/0x1a0 vfs_read+0xf1/0x190
--- fs/erofs/zdata.c | 13 +++++++------ fs/erofs/zpvec.h | 13 ++++++++++--- 2 files changed, 17 insertions(+), 9 deletions(-)
fs/erofs/zdata.c +7 -6
@@ -278,8 +278,8 @@ static inline bool z_erofs_try_inplace_io(struct z_erofs_collector *clt,/* callers must be with collection lock held */ static int z_erofs_attach_page (struct z_erofs_collector *clt, -struct page *page, -enum z_erofs_page_type type) +struct page *page,enum z_erofs_page_type type, +bool pvec_safereuse) {int ret;
@@ -289,9 +289,9 @@ static int z_erofs_attach_page(struct z_erofs_collector *clt, z_erofs_try_inplace_io(clt, page))return 0 ; - ret =z_erofs_pagevec_enqueue (&clt->vector, page, type); + ret =z_erofs_pagevec_enqueue (&clt->vector, page, type, + pvec_safereuse); clt->cl->vcnt += (unsigned int )ret; -return ret ?0 : -EAGAIN ; }
@@ -645,7 +645,8 @@ static int z_erofs_do_read_page(struct z_erofs_decompress_frontend *fe, tight &= (clt->mode >=COLLECT_PRIMARY_FOLLOWED ); retry: - err =z_erofs_attach_page (clt, page, page_type); + err =z_erofs_attach_page (clt, page, page_type, + clt->mode >=COLLECT_PRIMARY_FOLLOWED );/* should allocate an additional staging page for pagevec */ if (err == -EAGAIN ) { struct page *const newpage =
@@ -653,7 +654,7 @@ static int z_erofs_do_read_page(struct z_erofs_decompress_frontend *fe, newpage->mapping =Z_EROFS_MAPPING_STAGING ; err =z_erofs_attach_page (clt, newpage, -Z_EROFS_PAGE_TYPE_EXCLUSIVE ); +Z_EROFS_PAGE_TYPE_EXCLUSIVE , true);if (!err) goto retry; }
fs/erofs/zpvec.h +10 -3
@@ -107,11 +107,18 @@ static inline void z_erofs_pagevec_ctor_init(struct z_erofs_pagevec_ctor *ctor,static inline bool z_erofs_pagevec_enqueue (struct z_erofs_pagevec_ctor *ctor,struct page *page, -enum z_erofs_page_type type) +enum z_erofs_page_type type, +bool pvec_safereuse) { -if (!ctor->next && type) -if (ctor->index +1 == ctor->nr) +if (!ctor->next) { +/* some pages cannot be reused as pvec safely without I/O */ +if (type ==Z_EROFS_PAGE_TYPE_EXCLUSIVE && !pvec_safereuse) + type =Z_EROFS_VLE_PAGE_TYPE_TAIL_SHARED ; + +if (type !=Z_EROFS_PAGE_TYPE_EXCLUSIVE && + ctor->index +1 == ctor->nr)return false; + }if (ctor->index >= ctor->nr)z_erofs_pagevec_ctor_pagedown (ctor, false);
Signature · 1 line
2.24.4
Patch "erofs: fix unsafe pagevec reuse of hooked pclusters" has been added to the 5.10-stable tree
This is a note to let you know that I've just added the patch titled
erofs: fix unsafe pagevec reuse of hooked pclusters
to the 5.10-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary
The filename of the patch is:
erofs-fix-unsafe-pagevec-reuse-of-hooked-pclusters.patch
and it can be found in the queue-5.10 subdirectory.
If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.
From foo@baz Fri Nov 19 03:21:54 PM CET 2021
From: Gao Xiang <[email protected]>
Date: Tue, 16 Nov 2021 09:08:19 +0800
Subject: erofs: fix unsafe pagevec reuse of hooked pclusters
To: Greg Kroah-Hartman <[email protected]>, [email protected]
Cc: [email protected], Gao Xiang <[email protected]>, Chao Yu <[email protected]>
Message-ID: <[email protected]>
From: Gao Xiang <[email protected]>
commit 86432a6dca9bed79111990851df5756d3eb5f57c upstream.
There are pclusters in runtime marked with Z_EROFS_PCLUSTER_TAIL before actual I/O submission. Thus, the decompression chain can be extended if the following pcluster chain hooks such tail pcluster.
As the related comment mentioned, if some page is made of a hooked pcluster and another followed pcluster, it can be reused for in-place I/O (since I/O should be submitted anyway):
_______________________________________________________________
| tail (partial) page | head (partial) page | |_____PRIMARY_HOOKED___|____________PRIMARY_FOLLOWED____________|
However, it's by no means safe to reuse as pagevec since if such PRIMARY_HOOKED pclusters finally move into bypass chain without I/O submission. It's somewhat hard to reproduce with LZ4 and I just found it (general protection fault) by ro_fsstressing a LZMA image for long time.
I'm going to actively clean up related code together with multi-page folio adaption in the next few months. Let's address it directly for easier backporting for now.
Call trace for reference:
z_erofs_decompress_pcluster+0x10a/0x8a0 [erofs] z_erofs_decompress_queue.isra.36+0x3c/0x60 [erofs] z_erofs_runqueue+0x5f3/0x840 [erofs] z_erofs_readahead+0x1e8/0x320 [erofs] read_pages+0x91/0x270 page_cache_ra_unbounded+0x18b/0x240 filemap_get_pages+0x10a/0x5f0 filemap_read+0xa9/0x330 new_sync_read+0x11b/0x1a0 vfs_read+0xf1/0x190
--- fs/erofs/zdata.c | 13 +++++++------ fs/erofs/zpvec.h | 13 ++++++++++--- 2 files changed, 17 insertions(+), 9 deletions(-)
fs/erofs/zdata.c +7 -6
@@ -278,8 +278,8 @@ static inline bool z_erofs_try_inplace_i/* callers must be with collection lock held */ static int z_erofs_attach_page (struct z_erofs_collector *clt, -struct page *page, -enum z_erofs_page_type type) +struct page *page,enum z_erofs_page_type type, +bool pvec_safereuse) {int ret;
@@ -289,9 +289,9 @@ static int z_erofs_attach_page(struct z_ z_erofs_try_inplace_io(clt, page))return 0 ; - ret =z_erofs_pagevec_enqueue (&clt->vector, page, type); + ret =z_erofs_pagevec_enqueue (&clt->vector, page, type, + pvec_safereuse); clt->cl->vcnt += (unsigned int )ret; -return ret ?0 : -EAGAIN ; }
@@ -645,7 +645,8 @@ hitted: tight &= (clt->mode >=COLLECT_PRIMARY_FOLLOWED ); retry: - err =z_erofs_attach_page (clt, page, page_type); + err =z_erofs_attach_page (clt, page, page_type, + clt->mode >=COLLECT_PRIMARY_FOLLOWED );/* should allocate an additional staging page for pagevec */ if (err == -EAGAIN ) { struct page *const newpage =
@@ -653,7 +654,7 @@ retry: newpage->mapping =Z_EROFS_MAPPING_STAGING ; err =z_erofs_attach_page (clt, newpage, -Z_EROFS_PAGE_TYPE_EXCLUSIVE ); +Z_EROFS_PAGE_TYPE_EXCLUSIVE , true);if (!err) goto retry; }
fs/erofs/zpvec.h +10 -3
@@ -107,11 +107,18 @@ static inline void z_erofs_pagevec_ctor_static inline bool z_erofs_pagevec_enqueue (struct z_erofs_pagevec_ctor *ctor,struct page *page, -enum z_erofs_page_type type) +enum z_erofs_page_type type, +bool pvec_safereuse) { -if (!ctor->next && type) -if (ctor->index +1 == ctor->nr) +if (!ctor->next) { +/* some pages cannot be reused as pvec safely without I/O */ +if (type ==Z_EROFS_PAGE_TYPE_EXCLUSIVE && !pvec_safereuse) + type =Z_EROFS_VLE_PAGE_TYPE_TAIL_SHARED ; + +if (type !=Z_EROFS_PAGE_TYPE_EXCLUSIVE && + ctor->index +1 == ctor->nr)return false; + }if (ctor->index >= ctor->nr)z_erofs_pagevec_ctor_pagedown (ctor, false);
Patches currently in stable-queue which might be from [email protected] are
queue-5.10/erofs-fix-unsafe-pagevec-reuse-of-hooked-pclusters.patch queue-5.10/erofs-remove-the-occupied-parameter-from-z_erofs_pagevec_enqueue.patch
Greg Kroah-Hartman
maintainerRe: [PATCH 5.10.y 1/2] erofs: remove the occupied parameter from z_erofs_pagevec_enqueue()
In reply to Gao XiangQuoted 69 lines · Gao Xiang
From: Yue Hu <[email protected]>
commit
7dea3de7d384f4c8156e8bd93112ba6db1eb276cupstream.No any behavior to variable occupied in
z_erofs_attach_page()which is only caller toz_erofs_pagevec_enqueue().Signed-off-by Yue Hu <[email protected]>Reviewed-by Gao Xiang <[email protected]>Signed-off-by Gao Xiang <[email protected]>Signed-off-by Gao Xiang <[email protected]>---
Gao Xiang: Apply this trivial cleanup (no behavior change) for easier
backporting.fs/erofs/zdata.c | 4 +--- fs/erofs/zpvec.h | 5 +---- 2 files changed, 2 insertions(+), 7 deletions(-)fs/erofs/zdata.c +1
-3@@ -282,7 +282,6 @@ static int z_erofs_attach_page(struct z_erofs_collector *clt, enum z_erofs_page_type type) {int ret; -bool occupied;/* give priority for inplaceio */ if (clt->mode >= COLLECT_PRIMARY &&@@ -290,8 +289,7 @@ static int z_erofs_attach_page(struct z_erofs_collector *clt, z_erofs_try_inplace_io(clt, page))return 0 ; - ret =z_erofs_pagevec_enqueue (&clt->vector, - page, type, &occupied); + ret =z_erofs_pagevec_enqueue (&clt->vector, page, type); clt->cl->vcnt += (unsigned int )ret;return ret ?0 : -EAGAIN ;fs/erofs/zpvec.h +1
-4@@ -107,10 +107,8 @@ static inline void z_erofs_pagevec_ctor_init(struct z_erofs_pagevec_ctor *ctor,static inline bool z_erofs_pagevec_enqueue (struct z_erofs_pagevec_ctor *ctor,struct page *page, -enum z_erofs_page_type type, -bool *occupied) +enum z_erofs_page_type type) { - *occupied = false;if (!ctor->next && type)if (ctor->index +1 == ctor->nr)return false;@@ -125,7 +123,6 @@ static inline bool z_erofs_pagevec_enqueue(struct z_erofs_pagevec_ctor *ctor,/* should remind that collector->next never equal to 1, 2 */ if (type == (uintptr_t )ctor->next) { ctor->next = page; - *occupied = true; } ctor->pages[ctor->index++] =tagptr_fold (erofs_vtptr_t, page, type);return true;Signature · 1 line
2.24.4
All now queued up, thanks.
greg k-h
Patch "erofs: remove the occupied parameter from z_erofs_pagevec_enqueue()" has been added to the 5.10-stable tree
In reply to Gao XiangThis is a note to let you know that I've just added the patch titled
erofs: remove the occupied parameter from z_erofs_pagevec_enqueue()
to the 5.10-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary
The filename of the patch is:
erofs-remove-the-occupied-parameter-from-z_erofs_pagevec_enqueue.patch
and it can be found in the queue-5.10 subdirectory.
If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.
From foo@baz Fri Nov 19 03:21:54 PM CET 2021
From: Gao Xiang <[email protected]>
Date: Tue, 16 Nov 2021 09:08:18 +0800
Subject: erofs: remove the occupied parameter from z_erofs_pagevec_enqueue()
To: Greg Kroah-Hartman <[email protected]>, [email protected]
Cc: [email protected], Yue Hu <[email protected]>, Gao Xiang <[email protected]>, Gao Xiang <[email protected]>
Message-ID: <[email protected]>
From: Yue Hu <[email protected]>
commit 7dea3de7d384f4c8156e8bd93112ba6db1eb276c upstream.
No any behavior to variable occupied in z_erofs_attach_page() which is only caller to z_erofs_pagevec_enqueue().
--- fs/erofs/zdata.c | 4 +--- fs/erofs/zpvec.h | 5 +---- 2 files changed, 2 insertions(+), 7 deletions(-)
fs/erofs/zdata.c +1 -3
@@ -282,7 +282,6 @@ static int z_erofs_attach_page(struct z_ enum z_erofs_page_type type) {int ret; -bool occupied;/* give priority for inplaceio */ if (clt->mode >= COLLECT_PRIMARY &&
@@ -290,8 +289,7 @@ static int z_erofs_attach_page(struct z_ z_erofs_try_inplace_io(clt, page))return 0 ; - ret =z_erofs_pagevec_enqueue (&clt->vector, - page, type, &occupied); + ret =z_erofs_pagevec_enqueue (&clt->vector, page, type); clt->cl->vcnt += (unsigned int )ret;return ret ?0 : -EAGAIN ;
fs/erofs/zpvec.h +1 -4
@@ -107,10 +107,8 @@ static inline void z_erofs_pagevec_ctor_static inline bool z_erofs_pagevec_enqueue (struct z_erofs_pagevec_ctor *ctor,struct page *page, -enum z_erofs_page_type type, -bool *occupied) +enum z_erofs_page_type type) { - *occupied = false;if (!ctor->next && type)if (ctor->index +1 == ctor->nr)return false;
@@ -125,7 +123,6 @@ static inline bool z_erofs_pagevec_enque/* should remind that collector->next never equal to 1, 2 */ if (type == (uintptr_t )ctor->next) { ctor->next = page; - *occupied = true; } ctor->pages[ctor->index++] =tagptr_fold (erofs_vtptr_t, page, type);return true;
Patches currently in stable-queue which might be from [email protected] are
queue-5.10/erofs-fix-unsafe-pagevec-reuse-of-hooked-pclusters.patch queue-5.10/erofs-remove-the-occupied-parameter-from-z_erofs_pagevec_enqueue.patch