Implement MTE tag compression for swapped pages
Subsystems (2)
v9 cover Implement MTE tag compression for swapped pages
Currently, when MTE pages are swapped out, the tags are kept in the memory, occupying PAGE_SIZE/32 bytes per page. This is especially problematic for devices that use zram-backed in-memory swap, because tags stored uncompressed in the heap effectively reduce the available amount of swap memory.
The RLE-based algorithm suggested by Evgenii Stepanov and implemented in this patch series is able to efficiently compress fixed-size tag buffers, resulting in practical compression ratio of 2x. In many cases it is possible to store the compressed data in 63-bit Xarray values, resulting in no extra memory allocations.
This patch series depends on "lib/bitmap: add bitmap_{read,write}()"
(https://lore.kernel.org/linux-arm-kernel/[email protected]/T/)
that is mailed separately.
v9:
- split off the stats collection code into a separate patch in the series (as suggested by Yury Norov)
v8:
- split off the bitmap_read()/bitmap_write() series
- simplified the compression logic (only compress data if it fits into a pointer)
v7:
- fixed comments by Yury Norov, Andy Shevchenko, Rasmus Villemoes
- added perf tests for bitmap_read()/bitmap_write()
- more efficient bitmap_write() implementation (meant to be sent in v5)
v6:
- fixed comments by Yury Norov
- fixed handling of sizes divisible by MTE_GRANULES_PER_PAGE / 2 (caught while testing on a real device)
v5:
- fixed comments by Andy Shevchenko, Catalin Marinas, and Yury Norov
- added support for 16K- and 64K pages
- more efficient bitmap_write() implementation
v4:
- fixed a bunch of comments by Andy Shevchenko and Yury Norov
- added Documentation/arch/arm64/mte-tag-compression.rst
v3:
- as suggested by Andy Shevchenko, use bitmap_get_value()/bitmap_set_value() written by Syed Nayyar Waris
- switched to unsigned long to reduce typecasts
- simplified the compression code
v2:
- as suggested by Yuri Norov, replace the poorly implemented struct bitq with <linux/bitmap.h>
Alexander Potapenko (4):
arm64: mte: implement CONFIG_ARM64_MTE_COMP
arm64: mte: add a test for MTE tags compression
arm64: mte: add compression support to mteswap.c
arm64: mte: implement CONFIG_ARM64_MTE_SWAP_STATS
Documentation/arch/arm64/index.rst | 1 + .../arch/arm64/mte-tag-compression.rst | 166 ++++++++ arch/arm64/Kconfig | 37 ++ arch/arm64/include/asm/mtecomp.h | 39 ++ arch/arm64/mm/Makefile | 2 + arch/arm64/mm/mtecomp.c | 257 +++++++++++++ arch/arm64/mm/mtecomp.h | 12 + arch/arm64/mm/mteswap.c | 110 +++++- arch/arm64/mm/test_mtecomp.c | 364 ++++++++++++++++++ 9 files changed, 985 insertions(+), 3 deletions(-) create mode 100644 Documentation/arch/arm64/mte-tag-compression.rst create mode 100644 arch/arm64/include/asm/mtecomp.h create mode 100644 arch/arm64/mm/mtecomp.c create mode 100644 arch/arm64/mm/mtecomp.h create mode 100644 arch/arm64/mm/test_mtecomp.c
Signature · 1 line
2.42.0.869.gea05f2083d-goog
PATCH v9 1/4 arm64: mte: implement CONFIG_ARM64_MTE_COMP
The config implements the algorithm compressing memory tags for ARM MTE during swapping.
The algorithm is based on RLE and specifically targets buffers of tags corresponding to a single page. In many cases a buffer can be compressed into 63 bits, making it possible to store it without additional memory allocation.
---
v8:
- As suggested by Catalin Marinas, only compress tags if they can be stored inline. This simplifies the code drastically.
- Update the documentation.
- Split off patches introducing bitmap_read()/bitmap_write().
v6:
- shuffle bits in inline handles so that they can't be confused with
canonical pointers;
- use kmem_cache_zalloc() to allocate compressed storage
- correctly handle range size overflow
- minor documentation fixes, clarify the special cases
v5:
- make code PAGE_SIZE-agnostic, remove hardcoded constants, updated the docs
- implement debugfs interface
- Address comments by Andy Shevchenko:
- update description of mtecomp.c
- remove redundant assignments, simplify mte_tags_to_ranges()
- various code simplifications
- introduce mtecomp.h
- add :export: to Documentation/arch/arm64/mte-tag-compression.rst
v4:
- Addressed comments by Andy Shevchenko:
- expanded "MTE" to "Memory Tagging Extension" in Kconfig
- fixed kernel-doc comments, moved them to C source
- changed variables to unsigned where applicable
- some code simplifications, fewer unnecessary assignments
- added the mte_largest_idx_bits() helper
- added namespace prefixes to all functions
- added missing headers (but removed bits.h)
- Addressed comments by Yury Norov:
- removed test-only functions from mtecomp.h
- dropped the algoritm name (all functions are now prefixed with "mte")
- added more comments
- got rid of MTE_RANGES_INLINE
- renamed bitmap_{get,set}_value() to bitmap_{read,write}()
- moved the big comment explaining the algorithm to Documentation/arch/arm64/mte-tag-compression.rst, expanded it, add a link to it from Documentation/arch/arm64/index.rst
- removed hardcoded ranges from mte_alloc_size()/mte_size_to_ranges()
v3:
- Addressed comments by Andy Shevchenko:
- use bitmap_{set,get}_value() writte by Syed Nayyar Waris
- switched to unsigned long everywhere (fewer casts)
- simplified the code, removed redundant checks
- dropped ea0_compress_inline()
- added bit size constants and helpers to access the bitmap
- explicitly initialize all compressed sizes in ea0_compress_to_buf()
- initialize all handle bits
v2:
- as suggested by Yury Norov, switched from struct bitq (which is not needed anymore) to <linux/bitmap.h>
- add missing symbol exports
--- Documentation/arch/arm64/index.rst | 1 + .../arch/arm64/mte-tag-compression.rst | 154 +++++++++++ arch/arm64/Kconfig | 11 + arch/arm64/include/asm/mtecomp.h | 39 +++ arch/arm64/mm/Makefile | 1 + arch/arm64/mm/mtecomp.c | 257 ++++++++++++++++++ arch/arm64/mm/mtecomp.h | 12 + 7 files changed, 475 insertions(+) create mode 100644 Documentation/arch/arm64/mte-tag-compression.rst create mode 100644 arch/arm64/include/asm/mtecomp.h create mode 100644 arch/arm64/mm/mtecomp.c create mode 100644 arch/arm64/mm/mtecomp.h
Documentation/arch/arm64/index.rst +1 -0
@@ -19,6 +19,7 @@ ARM64 Architecture legacy_instructions memory memory-tagging-extension + mte-tag-compression perf pointer-authentication ptdump
Documentation/arch/arm64/mte-tag-compression.rst new file +154 -0
154 lines not on this page · raw message
arch/arm64/Kconfig +11 -0
@@ -2073,6 +2073,17 @@ config ARM64_EPAN if the cpu does not implement the feature. endmenu # "ARMv8.7 architectural features" +config ARM64_MTE_COMP + bool "Tag compression for ARM64 Memory Tagging Extension" + default y + depends on ARM64_MTE + help + Enable tag compression support for ARM64 Memory Tagging Extension. + + Tag buffers corresponding to swapped RAM pages are compressed using + RLE to conserve heap memory. In the common case compressed tags + occupy 2.5x less memory. + config ARM64_SVE bool "ARM Scalable Vector Extension support" default y
arch/arm64/include/asm/mtecomp.h new file +39 -0
@@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __ASM_MTECOMP_H +#define __ASM_MTECOMP_H + +#include <linux/types.h> + +/** +* mte_is_compressed() - check if the supplied pointer contains compressed tags. +* @ptr: pointer returned by kmalloc() or mte_compress(). +* +* Returns: true iff bit 0 of @ptr is 1, which is only possible if @ptr was +* returned by mte_is_compressed(). +*/ +static inline bool mte_is_compressed (void *ptr) +{ +return ((unsigned long )ptr &1 ); +} + +#if defined(CONFIG_ARM64_MTE_COMP ) + +void *mte_compress (u8 *tags); +bool mte_decompress (void *handle,u8 *tags); + +#else + +static inline void *mte_compress (u8 *tags) +{ +return NULL ; +} + +static inline bool mte_decompress (void *data,u8 *tags) +{ +return false; +} + +#endif // CONFIG_ARM64_MTE_COMP + +#endif // __ASM_MTECOMP_H
arch/arm64/mm/Makefile +1 -0
@@ -10,6 +10,7 @@ obj-$(CONFIG_TRANS_TABLE) += trans_pgd.o obj-$(CONFIG_TRANS_TABLE) += trans_pgd-asm.o obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o obj-$(CONFIG_ARM64_MTE) += mteswap.o +obj-$(CONFIG_ARM64_MTE_COMP) += mtecomp.o KASAN_SANITIZE_physaddr.o += n obj-$(CONFIG_KASAN) += kasan_init.o
arch/arm64/mm/mtecomp.c new file +257 -0
257 lines not on this page · raw message
arch/arm64/mm/mtecomp.h new file +12 -0
@@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef ARCH_ARM64_MM_MTECOMP_H_ +#define ARCH_ARM64_MM_MTECOMP_H_ + +/* Functions exported from mtecomp.c for test_mtecomp.c. */ +void mte_tags_to_ranges (u8 *tags,u8 *out_tags,unsigned short *out_sizes, +size_t *out_len); +void mte_ranges_to_tags (u8 *r_tags,unsigned short *r_sizes,size_t r_len, +u8 *tags); + +#endif // ARCH_ARM64_MM_TEST_MTECOMP_H_
Signature · 1 line
2.42.0.869.gea05f2083d-goog
PATCH v9 2/4 arm64: mte: add a test for MTE tags compression
In reply to Alexander PotapenkoEnsure that tag sequences containing alternating values are compressed to buffers of expected size and correctly decompressed afterwards.
---
v9:
- minor changes to Kconfig description
v8:
- adapt to the simplified compression algorithm
v6:
- add test_decompress_invalid() to ensure invalid handles are ignored;
- add test_upper_bits(), which is a regression test for a case where
an inline handle looked like an out-of-line one;
- add test_compress_nonzero() to ensure a full nonzero tag array is
compressed correctly; - add test_two_ranges() to test cases when the input buffer is divided into two ranges.
v5:
- remove hardcoded constants, added test setup/teardown;
- support 16- and 64K pages;
- replace nested if-clauses with expected_size_from_ranges();
- call mte_release_handle() after tests that perform
compression/decompression;
- address comments by Andy Shevchenko:
- fix include order;
- use mtecomp.h instead of function prototypes.
v4:
- addressed comments by Andy Shevchenko:
- expanded MTE to "Memory Tagging Extension" in Kconfig
- changed signed variables to unsigned where applicable
- added missing header dependencies
- addressed comments by Yury Norov:
- moved test-only declarations from mtecomp.h into this test
- switched to the new "mte"-prefixed function names, dropped the mentions of "EA0"
- added test_tag_to_ranges_n()
v3:
- addressed comments by Andy Shevchenko in another patch:
- switched from u64 to unsigned long
- added MODULE_IMPORT_NS(MTECOMP)
- fixed includes order
--- arch/arm64/Kconfig | 11 ++ arch/arm64/mm/Makefile | 1 + arch/arm64/mm/test_mtecomp.c | 364 +++++++++++++++++++++++++++++++++++ 3 files changed, 376 insertions(+) create mode 100644 arch/arm64/mm/test_mtecomp.c
arch/arm64/Kconfig +11 -0
@@ -2084,6 +2084,17 @@ config ARM64_MTE_COMP RLE to conserve heap memory. In the common case compressed tags occupy 2.5x less memory. +config ARM64_MTE_COMP_KUNIT_TEST + tristate "Test tag compression for ARM64 Memory Tagging Extension" if !KUNIT_ALL_TESTS + default KUNIT_ALL_TESTS + depends on KUNIT && ARM64_MTE_COMP + help + Test MTE compression algorithm enabled by CONFIG_ARM64_MTE_COMP. + + Ensure that certain tag sequences containing alternating values can + be compressed into pointer-size values and correctly decompressed + afterwards. + config ARM64_SVE bool "ARM Scalable Vector Extension support" default y
arch/arm64/mm/Makefile +1 -0
@@ -11,6 +11,7 @@ obj-$(CONFIG_TRANS_TABLE) += trans_pgd-asm.o obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o obj-$(CONFIG_ARM64_MTE) += mteswap.o obj-$(CONFIG_ARM64_MTE_COMP) += mtecomp.o +obj-$(CONFIG_ARM64_MTE_COMP_KUNIT_TEST) += test_mtecomp.o KASAN_SANITIZE_physaddr.o += n obj-$(CONFIG_KASAN) += kasan_init.o
arch/arm64/mm/test_mtecomp.c new file +364 -0
364 lines not on this page · raw message
Signature · 1 line
2.42.0.869.gea05f2083d-goog
PATCH v9 4/4 arm64: mte: implement CONFIG_ARM64_MTE_SWAP_STATS
In reply to Alexander PotapenkoProvide a config to collect the usage statistics for ARM MTE tag compression. This patch introduces allocation/deallocation counters for buffers that were stored uncompressed (and thus occupy 128 bytes of heap plus the Xarray overhead to store a pointer) and those that were compressed into 8-byte pointers (effectively using 0 bytes of heap in addition to the Xarray overhead).
The counters are exposed to the userspace via/sys/kernel/debug/mteswap/stats:
# cat /sys/kernel/debug/mteswap/stats 8 bytes: 102496 allocations, 67302 deallocations 128 bytes: 212234 allocations, 178278 deallocations uncompressed tag storage size: 8851200 compressed tag storage size: 4346368
---
This patch was split off from the "arm64: mte: add compression support to mteswap.c" patch
(https://lore.kernel.org/linux-arm-kernel/ZUVulBKVYK7cq2rJ@yury-ThinkPad/T/#m819ec30beb9de53d5c442f7e3247456f8966d88a)
v9:
- add this patch, put the stats behind a separate config, mention /sys/kernel/debug/mteswap/stats in the documentation
--- .../arch/arm64/mte-tag-compression.rst | 12 +++ arch/arm64/Kconfig | 15 +++ arch/arm64/mm/mteswap.c | 93 ++++++++++++++++++- 3 files changed, 118 insertions(+), 2 deletions(-)
Documentation/arch/arm64/mte-tag-compression.rst +12 -0
@@ -145,6 +145,18 @@ Tag compression and decompression implicitly rely on the fixed MTE tag size (4 bits) and number of tags per page. Should these values change, the algorithm may need to be revised. +Stats +===== + +When `CONFIG_ARM64_MTE_SWAP_STATS` is enabled, `arch/arm64/mm/mteswap.c` exports +usage statistics for tag compression used when swapping tagged pages. The data +can be accessed via debugfs:: + + # cat /sys/kernel/debug/mteswap/stats + 8 bytes: 10438 allocations, 10417 deallocations + 128 bytes: 26180 allocations, 26179 deallocations + uncompressed tag storage size: 2816 + compressed tag storage size: 128 Programming Interface =====================
arch/arm64/Kconfig +15 -0
@@ -2095,6 +2095,21 @@ config ARM64_MTE_COMP_KUNIT_TEST be compressed into pointer-size values and correctly decompressed afterwards. +config ARM64_MTE_SWAP_STATS + bool "Collect usage statistics of tag compression for swapped MTE tags" + default y + depends on ARM64_MTE && ARM64_MTE_COMP + help + Collect usage statistics for ARM64 MTE tag compression during swapping. + + Adds allocation/deallocation counters for buffers that were stored + uncompressed (and thus occupy 128 bytes of heap plus the Xarray + overhead to store a pointer) and those that were compressed into + 8-byte pointers (effectively using 0 bytes of heap in addition to + the Xarray overhead). + The counters are exposed to the userspace via + /sys/kernel/debug/mteswap/stats. + config ARM64_SVE bool "ARM Scalable Vector Extension support" default y
arch/arm64/mm/mteswap.c +91 -2
121 lines not on this page · raw message
Signature · 1 line
2.42.0.869.gea05f2083d-goog
PATCH v9 3/4 arm64: mte: add compression support to mteswap.c
In reply to Alexander PotapenkoUpdate mteswap.c to perform inline compression of memory tags when possible.
If CONFIG_ARM64_MTE_COMP is enabled, mteswap.c will attempt to compress saved tags for a struct page and store them directly in Xarray entry instead of wasting heap space.
Soon after booting Android, tag compression saves ~2x memory previously spent by mteswap.c on tag allocations. On a moderately loaded device with ~20% tagged pages, this leads to saving several megabytes of kernel heap:
# cat /sys/kernel/debug/mteswap/stats 8 bytes: 102496 allocations, 67302 deallocations 128 bytes: 212234 allocations, 178278 deallocations uncompressed tag storage size: 8851200 compressed tag storage size: 4346368
(statistics collection is introduced in the following patch)
---
v9:
- as requested by Yury Norov, split off statistics collection into a separate patch
- minor fixes
v8:
- adapt to the new compression API, abandon mteswap_{no,}comp.c
- move stats collection to mteswap.c
v5:
- drop a dead variable from _mte_free_saved_tags() in mteswap_comp.c
- ensure MTE compression works with arbitrary page sizes
- update patch description
v4:
- minor code simplifications suggested by Andy Shevchenko, added missing header dependencies
- changed compression API names to reflect modifications made to memcomp.h (as suggested by Yury Norov)
v3:
- Addressed comments by Andy Shevchenko in another patch:
- fixed includes order
- replaced u64 with unsigned long
- added MODULE_IMPORT_NS(MTECOMP)
--- arch/arm64/mm/mteswap.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-)
arch/arm64/mm/mteswap.c +17 -2
@@ -6,6 +6,8 @@#include <linux/swap.h> #include <linux/swapops.h> #include <asm/mte.h> +#include <asm/mtecomp.h> +#include "mtecomp.h" static DEFINE_XARRAY (mte_pages);
@@ -17,12 +19,13 @@ void *mte_allocate_tag_storage(void)void mte_free_tag_storage (char *storage) { -kfree (storage); +if (!mte_is_compressed (storage)) +kfree (storage); }int mte_save_tags (struct page *page) { -void *tag_storage, *ret; +void *tag_storage, *compressed_storage, *ret;if (!page_mte_tagged (page))return 0 ;
@@ -32,6 +35,11 @@ int mte_save_tags(struct page *page)return -ENOMEM ;mte_save_page_tags (page_address (page), tag_storage); + compressed_storage =mte_compress (tag_storage); +if (compressed_storage) { +mte_free_tag_storage (tag_storage); + tag_storage = compressed_storage; + }/* lookup the swap entry.val from the page */ ret = xa_store(&mte_pages, page_swap_entry(page).val, tag_storage,
@@ -50,13 +58,20 @@ int mte_save_tags(struct page *page)void mte_restore_tags (swp_entry_t entry,struct page *page) {void *tags =xa_load (&mte_pages, entry.val); +void *tag_storage =NULL ;if (!tags)return ;if (try_page_mte_tagging (page)) { +if (mte_is_compressed (tags)) { + tag_storage =mte_allocate_tag_storage (); +mte_decompress (tags, tag_storage); + tags = tag_storage; + }mte_restore_page_tags (page_address (page), tags);set_page_mte_tagged (page); +mte_free_tag_storage (tag_storage); } }
Signature · 1 line
2.42.0.869.gea05f2083d-goog
Re: [PATCH v9 1/4] arm64: mte: implement CONFIG_ARM64_MTE_COMP
In reply to Alexander PotapenkoQuoted 10 lines · Alexander Potapenko
The config implements the algorithm compressing memory tags for ARM MTE during swapping.
The algorithm is based on RLE and specifically targets buffers of tags corresponding to a single page. In many cases a buffer can be compressed into 63 bits, making it possible to store it without additional memory allocation.
Suggested-by Evgenii Stepanov <[email protected]>Signed-off-by Alexander Potapenko <[email protected]>
Quoted 3 lines
arch/arm64/mm/mtecomp.c new file +3
-0+void mte_tags_to_ranges (u8 *tags,u8 *out_tags,unsigned short *out_sizes, +size_t *out_len) +{
[...]
Review
Quoted 2 lines
(quoted diff) +2
-0+} +EXPORT_SYMBOL_NS(mte_tags_to_ranges, MTECOMP);
I may have asked before, what the reason for EXPORT_SYMBOL_NS? Is it for the kunit tests? Otherwise we don't expect those to be accessed from modules.
Signature · 1 line
Catalin
Re: [PATCH v9 2/4] arm64: mte: add a test for MTE tags compression
In reply to Alexander PotapenkoQuoted 4 lines · Alexander Potapenko
Ensure that tag sequences containing alternating values are compressed to buffers of expected size and correctly decompressed afterwards.
Signed-off-by Alexander Potapenko <[email protected]>
Re: [PATCH v9 3/4] arm64: mte: add compression support to mteswap.c
In reply to Alexander PotapenkoQuoted 17 lines · Alexander Potapenko
Update
mteswap.cto perform inline compression of memory tags when possible.If
CONFIG_ARM64_MTE_COMPis enabled,mteswap.cwill attempt to compress saved tags for a struct page and store them directly in Xarray entry instead of wasting heap space.Soon after booting Android, tag compression saves ~2x memory previously spent by
mteswap.con tag allocations. On a moderately loaded device with ~20% tagged pages, this leads to saving several megabytes of kernel heap:# cat /sys/kernel/debug/mteswap/stats 8 bytes: 102496 allocations, 67302 deallocations 128 bytes: 212234 allocations, 178278 deallocations uncompressed tag storage size: 8851200 compressed tag storage size: 4346368
Quoted 3 lines
(statistics collection is introduced in the following patch)
Signed-off-by Alexander Potapenko <[email protected]>
Re: [PATCH v9 4/4] arm64: mte: implement CONFIG_ARM64_MTE_SWAP_STATS
In reply to Alexander PotapenkoQuoted 10 lines · Alexander Potapenko
Provide a config to collect the usage statistics for ARM MTE tag compression. This patch introduces allocation/deallocation counters for buffers that were stored uncompressed (and thus occupy 128 bytes of heap plus the Xarray overhead to store a pointer) and those that were compressed into 8-byte pointers (effectively using 0 bytes of heap in addition to the Xarray overhead).
The counters are exposed to the userspace via
/sys/kernel/debug/mteswap/stats:
Quoted 8 lines
# cat /sys/kernel/debug/mteswap/stats 8 bytes: 102496 allocations, 67302 deallocations 128 bytes: 212234 allocations, 178278 deallocations uncompressed tag storage size: 8851200 compressed tag storage size: 4346368Suggested-by Yury Norov <[email protected]>Signed-off-by Alexander Potapenko <[email protected]>
Will Deacon
maintainerRe: [PATCH v9 1/4] arm64: mte: implement CONFIG_ARM64_MTE_COMP
In reply to Catalin MarinasQuoted 18 lines · Catalin Marinas
Quoted 10 lines · Alexander Potapenko
The config implements the algorithm compressing memory tags for ARM MTE during swapping.
The algorithm is based on RLE and specifically targets buffers of tags corresponding to a single page. In many cases a buffer can be compressed into 63 bits, making it possible to store it without additional memory allocation.
Suggested-by Evgenii Stepanov <[email protected]>Signed-off-by Alexander Potapenko <[email protected]>Acked-by Catalin Marinas <[email protected]>Quoted 3 lines
arch/arm64/mm/mtecomp.c new file +3
-0+void mte_tags_to_ranges (u8 *tags,u8 *out_tags,unsigned short *out_sizes, +size_t *out_len) +{[...]
Quoted 6 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+} +EXPORT_SYMBOL_NS(mte_tags_to_ranges, MTECOMP);I may have asked before, what the reason for
EXPORT_SYMBOL_NS? Is it for the kunit tests? Otherwise we don't expect those to be accessed from modules.
It looks like it. The Kconfig option for the test is tristate and it calls this symbol directly.
Will
Will Deacon
maintainerQuoted 12 lines · Alexander Potapenko
Currently, when MTE pages are swapped out, the tags are kept in the memory, occupying PAGE_SIZE/32 bytes per page. This is especially problematic for devices that use zram-backed in-memory swap, because tags stored uncompressed in the heap effectively reduce the available amount of swap memory.
The RLE-based algorithm suggested by Evgenii Stepanov and implemented in this patch series is able to efficiently compress fixed-size tag buffers, resulting in practical compression ratio of 2x. In many cases it is possible to store the compressed data in 63-bit Xarray values, resulting in no extra memory allocations.
Quoted 3 lines
This patch series depends on "lib/bitmap: add bitmap_{read,write}()"
(https://lore.kernel.org/linux-arm-kernel/[email protected]/T/)
that is mailed separately.
That's a shame, because it means I can't apply the series as-is:
arch/arm64/mm/mtecomp.c: In function ‘mte_bitmap_write’:arch/arm64/mm/mtecomp.c:105:2: error: implicit declaration of function ‘bitmap_write’; did you mean ‘bitmap_free’? [-Werror=implicit-function-declaration]
105 | bitmap_write(bitmap, value, *pos, bits);
| ^~~~~~~~~~~~
| bitmap_free
arch/arm64/mm/mtecomp.c: In function ‘mte_bitmap_read’:
arch/arm64/mm/mtecomp.c:198:9: error: implicit declaration of function ‘bitmap_read’; did you mean ‘bitmap_remap’? [-Werror=implicit-function-declaration]
198 | return bitmap_read(bitmap, start, bits);
| ^~~~~~~~~~~
| bitmap_remapcc1: some warnings being treated as errors
make[5]: *** [scripts/Makefile.build:243: arch/arm64/mm/mtecomp.o] Error 1
Do you really have such a hard dependency on those new bitmap ops?
Will
Quoted 14 lines · Will Deacon
Quoted 12 lines · Alexander Potapenko
Currently, when MTE pages are swapped out, the tags are kept in the memory, occupying PAGE_SIZE/32 bytes per page. This is especially problematic for devices that use zram-backed in-memory swap, because tags stored uncompressed in the heap effectively reduce the available amount of swap memory.
The RLE-based algorithm suggested by Evgenii Stepanov and implemented in this patch series is able to efficiently compress fixed-size tag buffers, resulting in practical compression ratio of 2x. In many cases it is possible to store the compressed data in 63-bit Xarray values, resulting in no extra memory allocations.
Quoted 6 lines
Quoted 3 lines
This patch series depends on "lib/bitmap: add bitmap_{read,write}()"
(https://lore.kernel.org/linux-arm-kernel/[email protected]/T/)
that is mailed separately.That's a shame, because it means I can't apply the series as-is:
Uh-oh, sorry about that. There was another series depending on the bitmap_read/bitmap_write API, and I thought mailing those patches separately would speed things up.
But in fact Yury requested them to have at least one user, so now that the MTE series is also acked I'd better bring everything back together.
Let me send out another version.
Quoted 11 lines
arch/arm64/mm/mtecomp.c: In function ‘mte_bitmap_write’:arch/arm64/mm/mtecomp.c:105:2: error: implicit declaration of function ‘bitmap_write’; did you mean ‘bitmap_free’? [-Werror=implicit-function-declaration]105 | bitmap_write(bitmap, value, *pos, bits); | ^~~~~~~~~~~~ | bitmap_free arch/arm64/mm/mtecomp.c: In function ‘mte_bitmap_read’: arch/arm64/mm/mtecomp.c:198:9: error: implicit declaration of function ‘bitmap_read’; did you mean ‘bitmap_remap’? [-Werror=implicit-function-declaration] 198 | return bitmap_read(bitmap, start, bits); | ^~~~~~~~~~~ | bitmap_remap
Quoted 5 lines
cc1: some warnings being treated as errors
make[5]: *** [scripts/Makefile.build:243:arch/arm64/mm/mtecomp.o] Error 1Do you really have such a hard dependency on those new bitmap ops?
Having them in bitmap.h seems natural - this way they can be reused by other people.
Re: [PATCH v9 1/4] arm64: mte: implement CONFIG_ARM64_MTE_COMP
In reply to Will DeaconQuoted 27 lines · Will Deacon
Quoted 18 lines · Catalin Marinas
Quoted 10 lines · Alexander Potapenko
The config implements the algorithm compressing memory tags for ARM MTE during swapping.
The algorithm is based on RLE and specifically targets buffers of tags corresponding to a single page. In many cases a buffer can be compressed into 63 bits, making it possible to store it without additional memory allocation.
Suggested-by Evgenii Stepanov <[email protected]>Signed-off-by Alexander Potapenko <[email protected]>Acked-by Catalin Marinas <[email protected]>Quoted 3 lines
arch/arm64/mm/mtecomp.c new file +3
-0+void mte_tags_to_ranges (u8 *tags,u8 *out_tags,unsigned short *out_sizes, +size_t *out_len) +{[...]
Quoted 6 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+} +EXPORT_SYMBOL_NS(mte_tags_to_ranges, MTECOMP);I may have asked before, what the reason for
EXPORT_SYMBOL_NS? Is it for the kunit tests? Otherwise we don't expect those to be accessed from modules.
Quoted 4 lines
It looks like it. The
Kconfigoption for the test is tristate and it calls this symbol directly.Will
What Will said.