kasan: call clear_page with a match-all tag instead of changing page tag
Subsystems (2)
PATCH 1/1 kasan: call clear_page with a match-all tag instead of changing page tag
Instead of changing the page's tag solely in order to obtain a pointer with a match-all tag and then changing it back again, just convert the pointer that we get from kmap_atomic() into one with a match-all tag before passing it to clear_page().
On a certain microarchitecture, this has been observed to cause a measurable improvement in microbenchmark performance, presumably as a result of being able to avoid the atomic operations on the page tag.
--- include/linux/highmem.h | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)
include/linux/highmem.h +3 -5
@@ -245,12 +245,10 @@ static inline void clear_highpage(struct page *page)static inline void clear_highpage_kasan_tagged (struct page *page) { -u8 tag; +void *kaddr =kmap_atomic (page); - tag =page_kasan_tag (page); -page_kasan_tag_reset (page); -clear_highpage (page); -page_kasan_tag_set (page, tag); +clear_page (kasan_reset_tag (kaddr)); +kunmap_atomic (kaddr); }#ifndef __HAVE_ARCH_TAG_CLEAR_HIGHPAGE
Signature · 1 line
2.39.1.581.gbfd45094c4-goog
Quoted 8 lines · Peter Collingbourne
Instead of changing the page's tag solely in order to obtain a pointer with a match-all tag and then changing it back again, just convert the pointer that we get from
kmap_atomic()into one with a match-all tag before passing it toclear_page().On a certain microarchitecture, this has been observed to cause a measurable improvement in microbenchmark performance, presumably as a result of being able to avoid the atomic operations on the page tag.
Yeah, this would likely break the write streaming mode on some ARM CPUs.
Quoted 14 lines
Signed-off-by Peter Collingbourne <[email protected]>--- include/linux/highmem.h | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)include/linux/highmem.h +0
-0@@ -245,12 +245,10 @@ static inline void clear_highpage(struct page *page)static inline void clear_highpage_kasan_tagged (struct page *page) {
include/linux/highmem.h static inline void clear_highpage(struct page *page) · line 252
Quoted 10 lines
include/linux/highmem.h +3
-5-u8 tag; +void *kaddr =kmap_atomic (page); - tag =page_kasan_tag (page); -page_kasan_tag_reset (page); -clear_highpage (page); -page_kasan_tag_set (page, tag); +clear_page (kasan_reset_tag (kaddr)); +kunmap_atomic (kaddr); }
Please don't add kmap_atomic() back. See commit d2c20e51e396 ("mm/highmem: remove deprecated kmap_atomic"). I'd duplicate the clear_highpage() logic in here and call clear_page() directly on the address with the kasan tag reset.
Signature · 1 line
Catalin
Quoted 30 lines · Catalin Marinas
Quoted 8 lines · Peter Collingbourne
Instead of changing the page's tag solely in order to obtain a pointer with a match-all tag and then changing it back again, just convert the pointer that we get from
kmap_atomic()into one with a match-all tag before passing it toclear_page().On a certain microarchitecture, this has been observed to cause a measurable improvement in microbenchmark performance, presumably as a result of being able to avoid the atomic operations on the page tag.
Yeah, this would likely break the write streaming mode on some ARM CPUs.
Quoted 17 lines
Signed-off-by Peter Collingbourne <[email protected]>--- include/linux/highmem.h | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)include/linux/highmem.h +1
-1@@ -245,12 +245,10 @@ static inline void clear_highpage(struct page *page)static inline void clear_highpage_kasan_tagged (struct page *page) { -u8 tag; +void *kaddr =kmap_atomic (page);
Quoted 12 lines
include/linux/highmem.h static inline void clear_highpage(struct page *page) · line 252
Quoted 7 lines
include/linux/highmem.h +2
-4- tag =page_kasan_tag (page); -page_kasan_tag_reset (page); -clear_highpage (page); -page_kasan_tag_set (page, tag); +clear_page (kasan_reset_tag (kaddr)); +kunmap_atomic (kaddr); }Please don't add
kmap_atomic()back. See commitd2c20e51e396("mm/highmem: remove deprecatedkmap_atomic"). I'd duplicate theclear_highpage()logic in here and callclear_page()directly on the address with the kasan tag reset.
Right, that's how I originally developed this patch. As you might have guessed, I was developing against a stable kernel, so I was copying the old version of clear_highpage(). Done in v2.
Peter