S390 hardware compression support for kernel zlib
Subsystems (4)
cover S390 hardware compression support for kernel zlib
With IBM z15 mainframe the new DFLTCC instruction is available. It implements deflate algorithm in hardware (Nest Acceleration Unit - NXU) with estimated compression and decompression performance orders of magnitude faster than the current zlib.
This patch-set adds s390 hardware compression support to kernel zlib. The code is based on the userspace zlib implementation:
https://github.com/madler/zlib/pull/410
The coding style is also preserved for future maintainability. There is only limited set of userspace zlib functions represented in kernel. Apart from that, all the memory allocation should be performed in advance. Thus, the workarea structures are extended with the parameter lists required for the DEFLATE CONVENTION CALL instruction. Since kernel zlib itself does not support gzip headers, only Adler-32 checksum is processed (also can be produced by DFLTCC facility). Like it was implemented for userspace, kernel zlib will compress in hardware on level 1, and in software on all other levels. Decompression will always happen in hardware (when enabled).
Two DFLTCC compression calls produce the same results only when they both are made on machines of the same generation, and when the respective buffers have the same offset relative to the start of the page. Therefore care should be taken when using hardware compression when reproducible results are desired. However it does always produce the standard conform output which can be inflated anyway.
The new kernel command line parameter 'dfltcc' is introduced to configure s390 zlib hardware support:
Format: { on | off | def_only | inf_only | always }
on: s390 zlib hardware support for compression on
level 1 and decompression (default)
off: No s390 zlib hardware support
def_only: s390 zlib hardware support for deflate
only (compression on level 1)
inf_only: s390 zlib hardware support for inflate
only (decompression)
always: Same as 'on' but ignores the selected compression
level always using hardware support (used for debugging)
The main purpose of the integration of the NXU support into the kernel zlib is the use of hardware deflate in btrfs filesystem with on-the-fly compression enabled. Apart from that, hardware support can also be used during boot for decompressing the kernel or the ramdisk image
With the patch for btrfs expanding zlib buffer from 1 to 4 pages (patch 5) the following performance results have been achieved using the ramdisk with btrfs. These are relative numbers based on throughput rate and compression ratio for zlib level 1:
Input data Deflate rate Inflate rate Compression ratio
NXU/Software NXU/Software NXU/Software
stream of zeroes 1.46 1.02 1.00
random ASCII data 10.44 3.00 0.96
ASCII text (dickens) 6,21 3.33 0.94
binary data (vmlinux) 8,37 3.90 1.02This means that s390 hardware deflate can provide up to 10 times faster compression (on level 1) and up to 4 times faster decompression (refers to all compression levels) for btrfs zlib.
Disclaimer: Performance results are based on IBM internal tests using DD command-line utility on btrfs on a Fedora 30 based internal driver in native LPAR on a z15 system. Results may vary based on individual workload, configuration and software levels.
Mikhail Zaslonko (5):
lib/zlib: Add s390 hardware support for kernel zlib_deflate s390/boot: Rename HEAP_SIZE due to name collision lib/zlib: Add s390 hardware support for kernel zlib_inflate s390/boot: Add dfltcc= kernel command line parameter
btrfs: Increase buffer size for zlib functions
.../admin-guide/kernel-parameters.txt | 12 + arch/s390/boot/compressed/decompressor.c | 8 +- arch/s390/boot/ipl_parm.c | 14 + arch/s390/include/asm/setup.h | 7 + arch/s390/kernel/setup.c | 1 + fs/btrfs/compression.c | 2 +- fs/btrfs/zlib.c | 109 ++++--- lib/Kconfig | 22 ++ lib/Makefile | 1 + lib/decompress_inflate.c | 13 + lib/zlib_deflate/deflate.c | 79 ++--- lib/zlib_deflate/deftree.c | 54 ---- lib/zlib_deflate/defutil.h | 134 ++++++++- lib/zlib_dfltcc/Makefile | 11 + lib/zlib_dfltcc/dfltcc.c | 55 ++++ lib/zlib_dfltcc/dfltcc.h | 144 +++++++++ lib/zlib_dfltcc/dfltcc_deflate.c | 280 ++++++++++++++++++ lib/zlib_dfltcc/dfltcc_inflate.c | 149 ++++++++++ lib/zlib_dfltcc/dfltcc_syms.c | 17 ++ lib/zlib_dfltcc/dfltcc_util.h | 124 ++++++++ lib/zlib_inflate/inflate.c | 32 +- lib/zlib_inflate/inflate.h | 8 + lib/zlib_inflate/infutil.h | 18 +- 23 files changed, 1137 insertions(+), 157 deletions(-) create mode 100644 lib/zlib_dfltcc/Makefile create mode 100644 lib/zlib_dfltcc/dfltcc.c create mode 100644 lib/zlib_dfltcc/dfltcc.h create mode 100644 lib/zlib_dfltcc/dfltcc_deflate.c create mode 100644 lib/zlib_dfltcc/dfltcc_inflate.c create mode 100644 lib/zlib_dfltcc/dfltcc_syms.c create mode 100644 lib/zlib_dfltcc/dfltcc_util.h
Signature · 1 line
2.17.1
PATCH 1/5 lib/zlib: Add s390 hardware support for kernel zlib_deflate
Create zlib_dfltcc library with the s390 DEFLATE CONVERSION CALL implementation and related compression functions.
Update zlib_deflate functions with the hooks for s390 hardware support and adjust workspace structures with extra parameter lists required for hardware deflate.
--- lib/Kconfig | 14 ++ lib/Makefile | 1 + lib/zlib_deflate/deflate.c | 79 ++++----- lib/zlib_deflate/deftree.c | 54 ------ lib/zlib_deflate/defutil.h | 134 +++++++++++++-- lib/zlib_dfltcc/Makefile | 11 ++ lib/zlib_dfltcc/dfltcc.c | 52 ++++++ lib/zlib_dfltcc/dfltcc.h | 115 +++++++++++++ lib/zlib_dfltcc/dfltcc_deflate.c | 274 +++++++++++++++++++++++++++++++ lib/zlib_dfltcc/dfltcc_syms.c | 17 ++ lib/zlib_dfltcc/dfltcc_util.h | 119 ++++++++++++++ 11 files changed, 768 insertions(+), 102 deletions(-) create mode 100644 lib/zlib_dfltcc/Makefile create mode 100644 lib/zlib_dfltcc/dfltcc.c create mode 100644 lib/zlib_dfltcc/dfltcc.h create mode 100644 lib/zlib_dfltcc/dfltcc_deflate.c create mode 100644 lib/zlib_dfltcc/dfltcc_syms.c create mode 100644 lib/zlib_dfltcc/dfltcc_util.h
lib/Kconfig +14 -0
@@ -278,6 +278,20 @@ config ZLIB_DEFLATE tristate select BITREVERSE +config ZLIB_DFLTCC + def_bool y + depends on S390 + prompt "Enable s390x DEFLATE CONVERSION CALL support for zlib" + help + Enable s390x hardware support for zlib in the kernel. + +config ZLIB_DFLTCC_DEFLATE + def_bool y + depends on ZLIB_DFLTCC && ZLIB_DEFLATE + prompt "Enable s390x hardware support for zlib deflate functions" + help + Enable s390x hardware support for zlib deflate in the kernel. + config LZO_COMPRESS tristate
lib/Makefile +1 -0
@@ -139,6 +139,7 @@ obj-$(CONFIG_842_COMPRESS) += 842/ obj-$(CONFIG_842_DECOMPRESS) += 842/ obj-$(CONFIG_ZLIB_INFLATE) += zlib_inflate/ obj-$(CONFIG_ZLIB_DEFLATE) += zlib_deflate/ +obj-$(CONFIG_ZLIB_DFLTCC) += zlib_dfltcc/ obj-$(CONFIG_REED_SOLOMON) += reed_solomon/ obj-$(CONFIG_BCH) += bch.o obj-$(CONFIG_LZO_COMPRESS) += lzo/
lib/zlib_deflate/deflate.c +41 -38
132 lines not on this page · raw message
lib/zlib_deflate/deftree.c +0 -54
72 lines not on this page · raw message
lib/zlib_deflate/defutil.h +124 -10
155 lines not on this page · raw message
lib/zlib_dfltcc/Makefile new file +11 -0
@@ -0,0 +1,11 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# This is a modified version of zlib, which does all memory +# allocation ahead of time. +# +# This is the code for s390 zlib hardware support. +# + +obj-$(CONFIG_ZLIB_DFLTCC) += zlib_dfltcc.o + +zlib_dfltcc-objs := dfltcc.o dfltcc_deflate.o dfltcc_syms.o
lib/zlib_dfltcc/dfltcc.c new file +52 -0
52 lines not on this page · raw message
lib/zlib_dfltcc/dfltcc.h new file +115 -0
115 lines not on this page · raw message
lib/zlib_dfltcc/dfltcc_deflate.c new file +274 -0
274 lines not on this page · raw message
lib/zlib_dfltcc/dfltcc_syms.c new file +17 -0
@@ -0,0 +1,17 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* linux/lib/zlib_dfltcc/dfltcc_syms.c +* +* Exported symbols for the s390 zlib dfltcc support. +* +*/ + +#include <linux/init.h> +#include <linux/module.h> +#include <linux/zlib.h> +#include "dfltcc.h" + +EXPORT_SYMBOL (dfltcc_can_deflate); +EXPORT_SYMBOL (dfltcc_deflate); +EXPORT_SYMBOL (dfltcc_reset); +MODULE_LICENSE ("GPL" );
lib/zlib_dfltcc/dfltcc_util.h new file +119 -0
119 lines not on this page · raw message
Signature · 1 line
2.17.1
PATCH 2/5 s390/boot: Rename HEAP_SIZE due to name collision
In reply to Mikhail ZaslonkoChange the conflicting macro name in preparation for zlib_inflate hardware support.
--- arch/s390/boot/compressed/decompressor.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
arch/s390/boot/compressed/decompressor.c +4 -4
@@ -30,13 +30,13 @@ extern unsigned char _compressed_start[];extern unsigned char _compressed_end[];#ifdef CONFIG_HAVE_KERNEL_BZIP2 -#define HEAP_SIZE 0x400000 +#define BOOT_HEAP_SIZE 0x400000#else -#define HEAP_SIZE 0x10000 +#define BOOT_HEAP_SIZE 0x10000#endif static unsigned long free_mem_ptr = (unsigned long ) _end; -static unsigned long free_mem_end_ptr = (unsigned long ) _end +HEAP_SIZE ; +static unsigned long free_mem_end_ptr = (unsigned long ) _end +BOOT_HEAP_SIZE ;#ifdef CONFIG_KERNEL_GZIP #include "../../../../lib/decompress_inflate.c"
@@ -62,7 +62,7 @@ static unsigned long free_mem_end_ptr = (unsigned long) _end + HEAP_SIZE;#include "../../../../lib/decompress_unxz.c" #endif -#define decompress_offset ALIGN((unsigned long)_end + HEAP_SIZE, PAGE_SIZE) +#define decompress_offset ALIGN((unsigned long)_end + BOOT_HEAP_SIZE, PAGE_SIZE)unsigned long mem_safe_offset (void ) {
Signature · 1 line
2.17.1
PATCH 3/5 lib/zlib: Add s390 hardware support for kernel zlib_inflate
In reply to Mikhail ZaslonkoAdd decompression functions to zlib_dfltcc library.
Update zlib_inflate functions with the hooks for s390 hardware support and adjust workspace structures with extra parameter lists required for hardware inflate decompression.
--- lib/Kconfig | 8 ++ lib/decompress_inflate.c | 13 +++ lib/zlib_dfltcc/Makefile | 2 +- lib/zlib_dfltcc/dfltcc.h | 28 ++++++ lib/zlib_dfltcc/dfltcc_inflate.c | 143 +++++++++++++++++++++++++++++++ lib/zlib_inflate/inflate.c | 32 +++++-- lib/zlib_inflate/inflate.h | 8 ++ lib/zlib_inflate/infutil.h | 18 +++- 8 files changed, 241 insertions(+), 11 deletions(-) create mode 100644 lib/zlib_dfltcc/dfltcc_inflate.c
lib/Kconfig +8 -0
@@ -292,6 +292,14 @@ config ZLIB_DFLTCC_DEFLATE help Enable s390x hardware support for zlib deflate in the kernel. +config ZLIB_DFLTCC_INFLATE + def_bool y + depends on ZLIB_DFLTCC && ZLIB_INFLATE + prompt "Enable s390x hardware support for zlib inflate functions" + help + Enable s390x hardware support for zlib inflate in the kernel. + + config LZO_COMPRESS tristate
lib/decompress_inflate.c +13 -0
@@ -10,6 +10,10 @@#include "zlib_inflate/inftrees.c" #include "zlib_inflate/inffast.c" #include "zlib_inflate/inflate.c" +#ifdef CONFIG_ZLIB_DFLTCC_INFLATE +#include "zlib_dfltcc/dfltcc.c" +#include "zlib_dfltcc/dfltcc_inflate.c" +#endif #else /* STATIC */ /* initramfs et al: linked */
@@ -76,7 +80,12 @@ STATIC int INIT __gunzip(unsigned char *buf, long len, } strm->workspace =malloc (flush ?zlib_inflate_workspacesize () : +#ifdef CONFIG_ZLIB_DFLTCC_INFLATE +/* Always allocate the full workspace for DFLTCC */ +zlib_inflate_workspacesize ()); +#else sizeof(struct inflate_state)); +#endif if (strm->workspace ==NULL ) {error ("Out of memory while allocating workspace" ); goto gunzip_nomem4;
@@ -123,10 +132,14 @@ STATIC int INIT __gunzip(unsigned char *buf, long len, rc =zlib_inflateInit2 (strm, -MAX_WBITS ); +#ifdef CONFIG_ZLIB_DFLTCC_INFLATE +/* Always keep the window for DFLTCC */ +#else if (!flush) {WS (strm)->inflate_state.wsize =0 ;WS (strm)->inflate_state.window =NULL ; } +#endif while (rc ==Z_OK ) {if (strm->avail_in ==0 ) {
lib/zlib_dfltcc/Makefile +1 -1
@@ -8,4 +8,4 @@ obj-$(CONFIG_ZLIB_DFLTCC) += zlib_dfltcc.o -zlib_dfltcc-objs := dfltcc.o dfltcc_deflate.o dfltcc_syms.o +zlib_dfltcc-objs := dfltcc.o dfltcc_deflate.o dfltcc_inflate.o dfltcc_syms.o
lib/zlib_dfltcc/dfltcc.h +28 -0
@@ -104,6 +104,14 @@ int dfltcc_deflate(z_streamp strm,int flush, block_state *result);void dfltcc_reset (z_streamp strm,uInt size); +int dfltcc_can_inflate (z_streamp strm); +typedef enum { +DFLTCC_INFLATE_CONTINUE , +DFLTCC_INFLATE_BREAK , +DFLTCC_INFLATE_SOFTWARE , +}dfltcc_inflate_action ; +dfltcc_inflate_action dfltcc_inflate (z_streamp strm, +int flush,int *ret);#define DEFLATE_RESET_HOOK (strm) \ dfltcc_reset((strm), sizeof(deflate_state))
@@ -112,4 +120,24 @@ void dfltcc_reset(z_streamp strm, uInt size);#define DEFLATE_NEED_CHECKSUM (strm) (!dfltcc_can_deflate((strm))) +#define INFLATE_RESET_HOOK (strm) \ + dfltcc_reset((strm), sizeof(struct inflate_state)) + +#define INFLATE_TYPEDO_HOOK (strm, flush) \ + if (dfltcc_can_inflate((strm))) { \ + dfltcc_inflate_action action; \ +\ + RESTORE(); \ + action = dfltcc_inflate((strm), (flush), &ret); \ + LOAD(); \ + if (action == DFLTCC_INFLATE_CONTINUE) \ + break; \ + else if (action == DFLTCC_INFLATE_BREAK) \ + goto inf_leave; \ + } + +#define INFLATE_NEED_CHECKSUM (strm) (!dfltcc_can_inflate((strm))) + +#define INFLATE_NEED_UPDATEWINDOW (strm) (!dfltcc_can_inflate((strm))) +#endif /* DFLTCC_H */
lib/zlib_dfltcc/dfltcc_inflate.c new file +143 -0
143 lines not on this page · raw message
lib/zlib_inflate/inflate.c +24 -8
@@ -15,6 +15,16 @@#include "inffast.h" #include "infutil.h" +/* architecture-specific bits */ +#ifdef CONFIG_ZLIB_DFLTCC_INFLATE +# include "../zlib_dfltcc/dfltcc.h" +#else +#define INFLATE_RESET_HOOK (strm) do {} while (0) +#define INFLATE_TYPEDO_HOOK (strm, flush) do {} while (0) +#define INFLATE_NEED_UPDATEWINDOW (strm) 1 +#define INFLATE_NEED_CHECKSUM (strm) 1 +#endif +int zlib_inflate_workspacesize (void ) {return sizeof (struct inflate_workspace );
@@ -42,6 +52,7 @@ int zlib_inflateReset(z_streamp strm) state->write =0 ; state->whave =0 ; +INFLATE_RESET_HOOK (strm);return Z_OK ; }
@@ -66,7 +77,15 @@ int zlib_inflateInit2(z_streamp strm, int windowBits)return Z_STREAM_ERROR ; } state->wbits = (unsigned )windowBits; +#ifdef CONFIG_ZLIB_DFLTCC_INFLATE +/* +* DFLTCC requires the window to be page aligned. +* Thus, we overallocate and take the aligned portion of the buffer. +*/ + state->window =PTR_ALIGN (&WS (strm)->working_window[0 ],PAGE_SIZE ); +#else state->window = &WS (strm)->working_window[0 ]; +#endif return zlib_inflateReset (strm); }
@@ -227,11 +246,6 @@ static int zlib_inflateSyncPacket(z_streamp strm) bits -= bits &7 ; \ }while (0 ) -/* Reverse the bytes in a 32-bit value */ -#define REVERSE(q) \ - ((((q) >>24 ) &0xff ) + (((q) >>8 ) &0xff00 ) + \ - (((q) &0xff00 ) <<8 ) + (((q) &0xff ) <<24 )) -/* inflate() uses a state machine to process as much input data and generate as much output data as possible before returning. The state machine is
@@ -395,6 +409,7 @@ int zlib_inflate(z_streamp strm, int flush)if (flush ==Z_BLOCK ) goto inf_leave;/* fall through */ case TYPEDO : +INFLATE_TYPEDO_HOOK (strm, flush);if (state->last) {BYTEBITS (); state->mode =CHECK ;
@@ -692,7 +707,7 @@ int zlib_inflate(z_streamp strm, int flush) out -= left; strm->total_out += out; state->total += out; -if (out) +if (INFLATE_NEED_CHECKSUM (strm) && out) strm->adler = state->check =UPDATE (state->check, put - out, out); out = left;
@@ -726,7 +741,8 @@ int zlib_inflate(z_streamp strm, int flush)*/ inf_leave:RESTORE (); -if (state->wsize || (state->mode <CHECK && out != strm->avail_out)) +if (INFLATE_NEED_UPDATEWINDOW (strm) && + (state->wsize || (state->mode <CHECK && out != strm->avail_out)))zlib_updatewindow (strm, out); in -= strm->avail_in;
@@ -734,7 +750,7 @@ int zlib_inflate(z_streamp strm, int flush) strm->total_in += in; strm->total_out += out; state->total += out; -if (state->wrap && out) +if (INFLATE_NEED_CHECKSUM (strm) && state->wrap && out) strm->adler = state->check =UPDATE (state->check, strm->next_out - out, out);
lib/zlib_inflate/inflate.h +8 -0
@@ -11,6 +11,8 @@subject to change. Applications should only use zlib.h. */ +#include "inftrees.h" +/* Possible inflate modes between inflate() calls */ typedef enum { HEAD, /* i: waiting for magic header */
@@ -108,4 +110,10 @@ struct inflate_state {unsigned short work[288 ];/* work area for code table building */ code codes[ENOUGH]; /* space for code tables */ }; + +/* Reverse the bytes in a 32-bit value */ +#define REVERSE (q) \ + ((((q) >> 24) & 0xff) + (((q) >> 8) & 0xff00) + \ + (((q) & 0xff00) << 8) + (((q) & 0xff) << 24)) +#endif
lib/zlib_inflate/infutil.h +16 -2
@@ -12,14 +12,28 @@#define _INFUTIL_H#include <linux/zlib.h> +#ifdef CONFIG_ZLIB_DFLTCC_INFLATE +#include "../zlib_dfltcc/dfltcc.h" +#include <asm/page.h> +#endif /* memory allocation for inflation */ struct inflate_workspace {struct inflate_state inflate_state; -unsigned char working_window[1 <<MAX_WBITS ]; +#ifdef CONFIG_ZLIB_DFLTCC_INFLATE +struct dfltcc_state dfltcc_state; +unsigned char working_window[(1 <<MAX_WBITS ) +PAGE_SIZE ]; +#else +unsigned char working_window[(1 <<MAX_WBITS )]; +#endif }; -#define WS (z) ((struct inflate_workspace *)(z->workspace)) +#ifdef CONFIG_ZLIB_DFLTCC_INFLATE +/* dfltcc_state must be doubleword aligned for DFLTCC call */ +static_assert (offsetof(struct inflate_workspace , dfltcc_state) %8 ==0 ); +#endif + +#define WS (strm) ((struct inflate_workspace *)(strm->workspace))#endif
Signature · 1 line
2.17.1
PATCH 4/5 s390/boot: Add dfltcc= kernel command line parameter
In reply to Mikhail ZaslonkoAdd the new kernel command line parameter 'dfltcc=' to configure s390 zlib hardware support.
Format: { on | off | def_only | inf_only | always }
on: s390 zlib hardware support for compression on
level 1 and decompression (default)
off: No s390 zlib hardware support
def_only: s390 zlib hardware support for deflate
only (compression on level 1)
inf_only: s390 zlib hardware support for inflate
only (decompression)
always: Same as 'on' but ignores the selected compression
level always using hardware support (used for debugging)
--- Documentation/admin-guide/kernel-parameters.txt | 12 ++++++++++++ arch/s390/boot/ipl_parm.c | 14 ++++++++++++++ arch/s390/include/asm/setup.h | 7 +++++++ arch/s390/kernel/setup.c | 1 + lib/zlib_dfltcc/dfltcc.c | 5 ++++- lib/zlib_dfltcc/dfltcc.h | 1 + lib/zlib_dfltcc/dfltcc_deflate.c | 6 ++++++ lib/zlib_dfltcc/dfltcc_inflate.c | 6 ++++++ lib/zlib_dfltcc/dfltcc_util.h | 5 +++++ 9 files changed, 56 insertions(+), 1 deletion(-)
Documentation/admin-guide/kernel-parameters.txt +12 -0
@@ -836,6 +836,18 @@ dump out devices still on the deferred probe list after retrying. + dfltcc= [HW,S390] + Format: { on | off | def_only | inf_only | always } + on: s390 zlib hardware support for compression on + level 1 and decompression (default) + off: No s390 zlib hardware support + def_only: s390 zlib hardware support for deflate + only (compression on level 1) + inf_only: s390 zlib hardware support for inflate + only (decompression) + always: Same as 'on' but ignores the selected compression + level always using hardware support (used for debugging) + dhash_entries= [KNL] Set number of hash buckets for dentry cache.
arch/s390/boot/ipl_parm.c +14 -0
@@ -14,6 +14,7 @@char __bootdata (early_command_line )[COMMAND_LINE_SIZE ];struct ipl_parameter_block __bootdata_preserved (ipl_block );int __bootdata_preserved (ipl_block_valid ); +unsigned int __bootdata_preserved (zlib_dfltcc_support ) =ZLIB_DFLTCC_FULL ;unsigned long __bootdata (vmalloc_size ) =VMALLOC_DEFAULT_SIZE ;unsigned long __bootdata (memory_end );
@@ -229,6 +230,19 @@ void parse_boot_command_line(void)if (!strcmp (param,"vmalloc" ) && val) vmalloc_size =round_up (memparse (val,NULL ),PAGE_SIZE ); +if (!strcmp (param,"dfltcc" )) { +if (!strcmp (val,"off" )) + zlib_dfltcc_support =ZLIB_DFLTCC_DISABLED ; +else if (!strcmp (val,"on" )) + zlib_dfltcc_support =ZLIB_DFLTCC_FULL ; +else if (!strcmp (val,"def_only" )) + zlib_dfltcc_support =ZLIB_DFLTCC_DEFLATE_ONLY ; +else if (!strcmp (val,"inf_only" )) + zlib_dfltcc_support =ZLIB_DFLTCC_INFLATE_ONLY ; +else if (!strcmp (val,"always" )) + zlib_dfltcc_support =ZLIB_DFLTCC_FULL_DEBUG ; + } +if (!strcmp (param,"noexec" )) { rc =kstrtobool (val, &enabled);if (!rc && !enabled)
arch/s390/include/asm/setup.h +7 -0
@@ -80,6 +80,13 @@ struct parmarea {char command_line[ARCH_COMMAND_LINE_SIZE ];/* 0x10480 */ }; +extern unsigned int zlib_dfltcc_support; +#define ZLIB_DFLTCC_DISABLED 0 +#define ZLIB_DFLTCC_FULL 1 +#define ZLIB_DFLTCC_DEFLATE_ONLY 2 +#define ZLIB_DFLTCC_INFLATE_ONLY 3 +#define ZLIB_DFLTCC_FULL_DEBUG 4 +extern int noexec_disabled;extern int memory_end_set;extern unsigned long memory_end;
arch/s390/kernel/setup.c +1 -0
@@ -111,6 +111,7 @@ unsigned long __bootdata_preserved(__etext_dma);unsigned long __bootdata_preserved (__sdma );unsigned long __bootdata_preserved (__edma );unsigned long __bootdata_preserved (__kaslr_offset ); +unsigned int __bootdata_preserved (zlib_dfltcc_support );unsigned long VMALLOC_START ;EXPORT_SYMBOL (VMALLOC_START );
lib/zlib_dfltcc/dfltcc.c +4 -1
@@ -44,7 +44,10 @@ void dfltcc_reset( dfltcc_state->param.nt =1 ;/* Initialize tuning parameters */ - dfltcc_state->level_mask =DFLTCC_LEVEL_MASK ; +if (zlib_dfltcc_support ==ZLIB_DFLTCC_FULL_DEBUG ) + dfltcc_state->level_mask =DFLTCC_LEVEL_MASK_DEBUG ; +else + dfltcc_state->level_mask =DFLTCC_LEVEL_MASK ; dfltcc_state->block_size =DFLTCC_BLOCK_SIZE ; dfltcc_state->block_threshold =DFLTCC_FIRST_FHT_BLOCK_SIZE ; dfltcc_state->dht_threshold =DFLTCC_DHT_MIN_SAMPLE_SIZE ;
lib/zlib_dfltcc/dfltcc.h +1 -0
@@ -8,6 +8,7 @@* Tuning parameters. */ #define DFLTCC_LEVEL_MASK 0x2/* DFLTCC compression for level 1 only */ +#define DFLTCC_LEVEL_MASK_DEBUG 0x3fe/* DFLTCC compression for all levels */ #define DFLTCC_BLOCK_SIZE 1048576#define DFLTCC_FIRST_FHT_BLOCK_SIZE 4096#define DFLTCC_DHT_MIN_SAMPLE_SIZE 4096
lib/zlib_dfltcc/dfltcc_deflate.c +6 -0
@@ -3,6 +3,7 @@#include "../zlib_deflate/defutil.h" #include "dfltcc_util.h" #include "dfltcc.h" +#include <asm/setup.h> #include <linux/zutil.h> /*
@@ -15,6 +16,11 @@ int dfltcc_can_deflate(deflate_state *state = (deflate_state *)strm->state;struct dfltcc_state *dfltcc_state =GET_DFLTCC_STATE (state); +/* Check for kernel dfltcc command line parameter */ +if (zlib_dfltcc_support ==ZLIB_DFLTCC_DISABLED || + zlib_dfltcc_support ==ZLIB_DFLTCC_INFLATE_ONLY ) +return 0 ; +/* Unsupported compression settings */ if (!dfltcc_are_params_ok (state->level, state->w_bits, state->strategy, dfltcc_state->level_mask))
lib/zlib_dfltcc/dfltcc_inflate.c +6 -0
@@ -3,6 +3,7 @@#include "../zlib_inflate/inflate.h" #include "dfltcc_util.h" #include "dfltcc.h" +#include <asm/setup.h> #include <linux/zutil.h> /*
@@ -15,6 +16,11 @@ int dfltcc_can_inflate(struct inflate_state *state = (struct inflate_state *)strm->state;struct dfltcc_state *dfltcc_state =GET_DFLTCC_STATE (state); +/* Check for kernel dfltcc command line parameter */ +if (zlib_dfltcc_support ==ZLIB_DFLTCC_DISABLED || + zlib_dfltcc_support ==ZLIB_DFLTCC_DEFLATE_ONLY ) +return 0 ; +/* Unsupported compression settings */ if (state->wbits !=HB_BITS )return 0 ;
lib/zlib_dfltcc/dfltcc_util.h +5 -0
@@ -3,6 +3,7 @@#define DFLTCC_UTIL_H #include <linux/zutil.h> +#include <asm/setup.h> /* * C wrapper for the DEFLATE CONVERSION CALL instruction.
@@ -104,6 +105,10 @@ static inline int is_dfltcc_enabled(void)uint64_t facilities[(DFLTCC_FACILITY /64 ) +1 ]; registerchar r0 __asm__("r0" ); +/* Check for kernel dfltcc command line parameter */ +if (zlib_dfltcc_support ==ZLIB_DFLTCC_DISABLED ) +return 0 ; +memset (facilities,0 ,sizeof (facilities)); r0 =sizeof (facilities) /sizeof (facilities[0 ]) -1 ; __asm__ volatile("stfle %[facilities]\n"
Signature · 1 line
2.17.1
PATCH 5/5 btrfs: Increase buffer size for zlib functions
In reply to Mikhail ZaslonkoDue to the small size of zlib buffer (1 page) set in btrfs code, s390 hardware compression is rather limited in terms of performance. Increasing the buffer size to 4 pages would bring significant benefit for s390 hardware compression (up to 60% better performance compared to the PAGE_SIZE buffer) and should not bring much overhead in terms of memory consumption due to order 2 allocations.
--- fs/btrfs/compression.c | 2 +- fs/btrfs/zlib.c | 109 ++++++++++++++++++++++++++--------------- 2 files changed, 70 insertions(+), 41 deletions(-)
fs/btrfs/compression.c +1 -1
@@ -1158,7 +1158,7 @@ int btrfs_decompress_buf2page(const char *buf, unsigned long buf_start,/* copy bytes from the working buffer into the pages */ while (working_bytes >0 ) { bytes = min_t(unsigned long, bvec.bv_len, -PAGE_SIZE - buf_offset); +PAGE_SIZE - (buf_offset %PAGE_SIZE )); bytes =min (bytes, working_bytes); kaddr =kmap_atomic (bvec.bv_page);
fs/btrfs/zlib.c +69 -40
205 lines not on this page · raw message
Signature · 1 line
2.17.1
Re: [PATCH 5/5] btrfs: Increase buffer size for zlib functions
Quoted 8 lines · Mikhail Zaslonko
Due to the small size of zlib buffer (1 page) set in btrfs code, s390 hardware compression is rather limited in terms of performance. Increasing the buffer size to 4 pages would bring significant benefit for s390 hardware compression (up to 60% better performance compared to the
PAGE_SIZEbuffer) and should not bring much overhead in terms of memory consumption due to order 2 allocations.Signed-off-by Mikhail Zaslonko <[email protected]>
We may have to make these allocations under memory pressure in the IO context, order 2 allocations here is going to be not awesome. If you really want it then you need to at least be able to fall back to single page if you fail to get the allocation. Thanks,
josef
Re: [PATCH 5/5] btrfs: Increase buffer size for zlib functions
Quoted 8 lines · Josef Bacik
Quoted 7 lines · Mikhail Zaslonko
Due to the small size of zlib buffer (1 page) set in btrfs code, s390 hardware compression is rather limited in terms of performance. Increasing the buffer size to 4 pages would bring significant benefit for s390 hardware compression (up to 60% better performance compared to the
PAGE_SIZEbuffer) and should not bring much overhead in terms of memory consumption due to order 2 allocations.
Quoted 6 lines
Quoted 1 line
Signed-off-by Mikhail Zaslonko <[email protected]>We may have to make these allocations under memory pressure in the IO context, order 2 allocations here is going to be not awesome. If you really want it then you need to at least be able to fall back to single page if you fail to get the allocation. Thanks,
The allocation is only for the workspace and it does not happen on the IO path for each call. There's the pool and if
btrfs_get_workspace
alloc_workspace
fails, then there's fallback path to wait for an existing workspace to be free.
The order 2 allocation can put more pressure on the allocator though so it's possible to have effects in some corner cases, but not in general. I don't think the single page fallback code is needed.
And of course evaluation of the effects of the larger zlib buffer should be done, it could improve compression but probably at the cost of cpu time. Also decompression of blocks created on new code (4 pages) must work on the old code (1 page).
Re: [PATCH 5/5] btrfs: Increase buffer size for zlib functions
Quoted 16 lines · David Sterba
Quoted 8 lines · Josef Bacik
Quoted 7 lines · Mikhail Zaslonko
Due to the small size of zlib buffer (1 page) set in btrfs code, s390 hardware compression is rather limited in terms of performance. Increasing the buffer size to 4 pages would bring significant benefit for s390 hardware compression (up to 60% better performance compared to the
PAGE_SIZEbuffer) and should not bring much overhead in terms of memory consumption due to order 2 allocations.Quoted 6 lines
Quoted 1 line
Signed-off-by Mikhail Zaslonko <[email protected]>We may have to make these allocations under memory pressure in the IO context, order 2 allocations here is going to be not awesome. If you really want it then you need to at least be able to fall back to single page if you fail to get the allocation. Thanks,
Quoted 8 lines
The allocation is only for the workspace and it does not happen on the IO path for each call. There's the pool and if
btrfs_get_workspace
alloc_workspacefails, then there's fallback path to wait for an existing workspace to be free.
Only if we are maxed out, otherwise it tries to allocate. If it can happen it will happen, and I'll be the guy swearing in the middle of the night trying to deal with this making boxes fall over in production. I'm ok if we pre-allocate them first and only do 1 page on-demand, but having it always this way will bite us in the ass in production. Thanks,
Josef
Re: [PATCH 5/5] btrfs: Increase buffer size for zlib functions
In reply to David SterbaHi David,
Quoted 25 lines · David Sterba
Quoted 8 lines · Josef Bacik
Quoted 7 lines · Mikhail Zaslonko
Due to the small size of zlib buffer (1 page) set in btrfs code, s390 hardware compression is rather limited in terms of performance. Increasing the buffer size to 4 pages would bring significant benefit for s390 hardware compression (up to 60% better performance compared to the
PAGE_SIZEbuffer) and should not bring much overhead in terms of memory consumption due to order 2 allocations.Quoted 6 lines
Quoted 1 line
Signed-off-by Mikhail Zaslonko <[email protected]>We may have to make these allocations under memory pressure in the IO context, order 2 allocations here is going to be not awesome. If you really want it then you need to at least be able to fall back to single page if you fail to get the allocation. Thanks,
The allocation is only for the workspace and it does not happen on the IO path for each call. There's the pool and if
btrfs_get_workspace
alloc_workspacefails, then there's fallback path to wait for an existing workspace to be free.
Quoted 8 lines
The order 2 allocation can put more pressure on the allocator though so it's possible to have effects in some corner cases, but not in general. I don't think the single page fallback code is needed.
And of course evaluation of the effects of the larger zlib buffer should be done, it could improve compression but probably at the cost of cpu time. Also decompression of blocks created on new code (4 pages) must work on the old code (1 page).
Regarding 'improve compression but probably at the cost of cpu' ... what would be the proper way to evaluate this effect?
As for backward compatibility, I do not see side effects of using larger buffers. Data in the compressed state might differ indeed, but it will sill conform to zlib standard and thus can be decompressed.
BTW, I have sent around V2 of the patch set. I would appreciate if you take a look as well.
Quoted 1 line
Thanks,
Mikhail
Re: [PATCH 5/5] btrfs: Increase buffer size for zlib functions
In reply to Josef BacikHello,
Quoted 8 lines · Josef Bacik
Quoted 7 lines · Mikhail Zaslonko
Due to the small size of zlib buffer (1 page) set in btrfs code, s390 hardware compression is rather limited in terms of performance. Increasing the buffer size to 4 pages would bring significant benefit for s390 hardware compression (up to 60% better performance compared to the
PAGE_SIZEbuffer) and should not bring much overhead in terms of memory consumption due to order 2 allocations.
Quoted 7 lines
Quoted 1 line
Signed-off-by Mikhail Zaslonko <[email protected]>We may have to make these allocations under memory pressure in the IO context, order 2 allocations here is going to be not awesome. If you really want it then you need to at least be able to fall back to single page if you fail to get the allocation. Thanks,
As far as I understand GFP_KERNEL allocations would never fail for the order <= PAGE_ALLOC_COSTLY_ORDER. How else can the memory pressure condition be identified here?
Quoted 2 lines
josef
Re: [PATCH 5/5] btrfs: Increase buffer size for zlib functions
Quoted 19 lines · Zaslonko Mikhail
Hello,
Quoted 8 lines · Josef Bacik
Quoted 7 lines · Mikhail Zaslonko
Due to the small size of zlib buffer (1 page) set in btrfs code, s390 hardware compression is rather limited in terms of performance. Increasing the buffer size to 4 pages would bring significant benefit for s390 hardware compression (up to 60% better performance compared to the
PAGE_SIZEbuffer) and should not bring much overhead in terms of memory consumption due to order 2 allocations.Quoted 7 lines
Quoted 1 line
Signed-off-by Mikhail Zaslonko <[email protected]>We may have to make these allocations under memory pressure in the IO context, order 2 allocations here is going to be not awesome. If you really want it then you need to at least be able to fall back to single page if you fail to get the allocation. Thanks,
Quoted 4 lines
As far as I understand
GFP_KERNELallocations would never fail for the order <=PAGE_ALLOC_COSTLY_ORDER. How else can the memory pressure condition be identified here?
Except these can be done under NOFS, and just because GFP_KERNEL probably won't fail doesn't mean it won't cause problems accross the system at alloc time. Half of our rebase time at Facebook is spent finding all the fun ways people have abused memory allocations not thinking about how the system behaves under memory pressure. Thanks,
Josef
Re: [PATCH 5/5] btrfs: Increase buffer size for zlib functions
In reply to Zaslonko MikhailQuoted 13 lines · Zaslonko Mikhail
Hello,
Quoted 10 lines · Josef Bacik
Quoted 8 lines · Mikhail Zaslonko
Due to the small size of zlib buffer (1 page) set in btrfs code, s390 hardware compression is rather limited in terms of performance. Increasing the buffer size to 4 pages would bring significant benefit for s390 hardware compression (up to 60% better performance compared to the
PAGE_SIZEbuffer) and should not bring much overhead in terms of memory consumption due to order 2 allocations.Signed-off-by Mikhail Zaslonko <[email protected]>
Quoted 7 lines
Quoted 4 lines
We may have to make these allocations under memory pressure in the IO context, order 2 allocations here is going to be not awesome. If you really want it then you need to at least be able to fall back to single page if you fail to get the allocation. Thanks,
As far as I understand
GFP_KERNELallocations would never fail for the order <=PAGE_ALLOC_COSTLY_ORDER.
There's no guaranteed no-fail semantics for GFP flags (obviously besides __GFP_NOFAIL), GFP_KERNEL can fail and GFP_NOFS is unlikely to fail for order below costly allocations. This depends on the allocator internals and has never been an API-level guarantee AFAIK. There's ongoing to work to relax the allocator constraints and allow to fail in more cases (like for GFP_NOFS).
Quoted 2 lines
How else can the memory pressure condition be identified here?
All data write paths must consider what happens under memory pressure, because the reason to write the data could be started by an allocation that can get free memory by writing dirty data. So it's kind of implied here.