x86, crypto: remove always-defined CONFIG_AS_* and cosolidate Kconfig/Makefiles
Subsystems (9)
v2 cover x86, crypto: remove always-defined CONFIG_AS_* and cosolidate Kconfig/Makefiles
This series of cleanups was prompted by Linus:
https://lkml.org/lkml/2020/3/12/726
First, this series drop always-on CONFIG_AS_* options. Some of those options were introduced in old days. For example, the check for CONFIG_AS_CFI dates back to 2006.
We raise the minimal tool versions from time to time. Currently, we require binutils 2.21
(and we even plan to bump it to 2.23).
After cleaning away the old checks, as-instr calls are moved to Kconfig from Makefiles, then more Kconfig / Makefile code is cleaned up.
I folded all relevanet patches into this series,
as suggested by Jason A. Donenfeld.
The update for v2 is quite small.
I just swapped the patch order of patch 8 and 11
instead of moving comments around files,
which was addressed by Nick Desaulniers.
Borislav Petkov (1):
Documentation/changes: Raise minimum supported binutils version to
2.23
Jason A. Donenfeld (4):
x86: probe assembler capabilities via kconfig instead of makefile
crypto: x86 - rework configuration based on Kconfig
crypto: curve25519 - do not pollute dispatcher based on assembler
x86: update AS_* macros to binutils >=2.23, supporting ADX and AVX2
Masahiro Yamada (11):
lib/raid6/test: fix build on distros whose /bin/sh is not bash
x86: remove unneeded defined(__ASSEMBLY__) check from asm/dwarf2.h
x86: remove always-defined CONFIG_AS_CFI
x86: remove unneeded (CONFIG_AS_)CFI_SIGNAL_FRAME
x86: remove always-defined CONFIG_AS_CFI_SECTIONS
x86: remove always-defined CONFIG_AS_SSSE3
x86: remove always-defined CONFIG_AS_AVX
x86: replace arch macros from compiler with CONFIG_X86_{32,64} drm/i915: remove always-defined CONFIG_AS_MOVNTDQA
x86: add comments about the binutils version to support code in
as-instr
crypto: x86 - clean up poly1305-x86_64-cryptogams.S by 'make clean'
Documentation/process/changes.rst | 4 +- arch/x86/Kconfig | 2 + arch/x86/Kconfig.assembler | 17 ++ arch/x86/Makefile | 22 --- arch/x86/crypto/Makefile | 162 +++++++----------- arch/x86/crypto/aesni-intel_avx-x86_64.S | 6 - arch/x86/crypto/aesni-intel_glue.c | 21 +-- arch/x86/crypto/blake2s-core.S | 2 - arch/x86/crypto/chacha_glue.c | 6 +- arch/x86/crypto/poly1305-x86_64-cryptogams.pl | 16 -- arch/x86/crypto/poly1305_glue.c | 11 +- arch/x86/crypto/sha1_ssse3_asm.S | 4 - arch/x86/crypto/sha1_ssse3_glue.c | 13 -- arch/x86/crypto/sha256-avx-asm.S | 3 - arch/x86/crypto/sha256-avx2-asm.S | 3 - arch/x86/crypto/sha256_ssse3_glue.c | 12 -- arch/x86/crypto/sha512-avx-asm.S | 2 - arch/x86/crypto/sha512-avx2-asm.S | 3 - arch/x86/crypto/sha512_ssse3_glue.c | 10 -- arch/x86/include/asm/dwarf2.h | 44 ----- arch/x86/include/asm/xor_avx.h | 9 - drivers/gpu/drm/i915/Makefile | 3 - drivers/gpu/drm/i915/i915_memcpy.c | 5 - include/crypto/curve25519.h | 6 +- kernel/signal.c | 2 +- lib/raid6/algos.c | 12 +- lib/raid6/avx2.c | 4 - lib/raid6/recov_avx2.c | 6 - lib/raid6/recov_ssse3.c | 6 - lib/raid6/test/Makefile | 9 +- 30 files changed, 101 insertions(+), 324 deletions(-) create mode 100644 arch/x86/Kconfig.assembler
Signature · 6 lines
2.17.1 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
PATCH v2 9/16 drm/i915: remove always-defined CONFIG_AS_MOVNTDQA
CONFIG_AS_MOVNTDQA was introduced by commit 0b1de5d58e19 ("drm/i915:
Use SSE4.1 movntdqa to accelerate reads from WC memory").
We raise the minimal supported binutils version from time to time. The last bump was commit 1fb12b35e5ff ("kbuild: Raise the minimum required binutils version to 2.21").
I confirmed the code in $(call as-instr,...) can be assembled by the binutils 2.21 assembler and also by LLVM integrated assembler.
Remove CONFIG_AS_MOVNTDQA, which is always defined.
---
Changes in v2: None
drivers/gpu/drm/i915/Makefile | 3 --- drivers/gpu/drm/i915/i915_memcpy.c | 5 ----- 2 files changed, 8 deletions(-)
drivers/gpu/drm/i915/Makefile +0 -3
@@ -28,9 +28,6 @@ subdir-ccflags-$(CONFIG_DRM_I915_WERROR) += -Werror CFLAGS_i915_pci.o = $(call cc-disable-warning, override-init) CFLAGS_display/intel_fbdev.o = $(call cc-disable-warning, override-init) -subdir-ccflags-y += \ - $(call as-instr,movntdqa (%eax)$(comma)%xmm0,-DCONFIG_AS_MOVNTDQA) - subdir-ccflags-y += -I$(srctree)/$(src) # Please keep these build lists sorted!
drivers/gpu/drm/i915/i915_memcpy.c +0 -5
@@ -35,7 +35,6 @@static DEFINE_STATIC_KEY_FALSE (has_movntdqa); -#ifdef CONFIG_AS_MOVNTDQA static void __memcpy_ntdqa (void *dst,const void *src,unsigned long len) {kernel_fpu_begin ();
@@ -93,10 +92,6 @@ static void __memcpy_ntdqu(void *dst, const void *src, unsigned long len)kernel_fpu_end (); } -#else -static void __memcpy_ntdqa (void *dst,const void *src,unsigned long len) {} -static void __memcpy_ntdqu (void *dst,const void *src,unsigned long len) {} -#endif /** * i915_memcpy_from_wc: perform an accelerated *aligned* read from WC
Signature · 6 lines
2.17.1 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
PATCH v2 8/16 x86: replace arch macros from compiler with CONFIG_X86_{32,64}
In reply to Masahiro YamadaIf the intention is to check i386/x86_64 excluding UML, testing CONFIG_X86_{32,64} is simpler.
The reason for checking __i386__ / __x86_64__ was perhaps because lib/raid6/algos.c is built not only for the kernel but also for testing the library code from userspace.
However, lib/raid6/test/Makefile passes -DCONFIG_X86_{32,64} for this case. So, I do not see a reason to not use CONFIG option here.
---
Changes in v2: None
kernel/signal.c | 2 +- lib/raid6/algos.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
kernel/signal.c +1 -1
@@ -1246,7 +1246,7 @@ static void print_fatal_signal(int signr)struct pt_regs *regs =signal_pt_regs ();pr_info ("potentially unexpected fatal signal %d.\n" , signr); -#if defined(__i386__) && !defined(__arch_um__) +#ifdef CONFIG_X86_32 pr_info ("code at %08lx: " , regs->ip); {int i;
lib/raid6/algos.c +2 -2
@@ -29,7 +29,7 @@ struct raid6_calls raid6_call;EXPORT_SYMBOL_GPL (raid6_call); const struct raid6_calls * const raid6_algos[] = { -#if defined(__i386__) && !defined(__arch_um__) +#ifdef CONFIG_X86_32 #ifdef CONFIG_AS_AVX512 &raid6_avx512x2, &raid6_avx512x1,
@@ -45,7 +45,7 @@ const struct raid6_calls * const raid6_algos[] = { &raid6_mmxx2, &raid6_mmxx1,#endif -#if defined(__x86_64__) && !defined(__arch_um__) +#ifdef CONFIG_X86_64 #ifdef CONFIG_AS_AVX512 &raid6_avx512x4, &raid6_avx512x2,
Signature · 1 line
2.17.1
PATCH v2 10/16 x86: probe assembler capabilities via kconfig instead of makefile
In reply to Masahiro YamadaFrom: "Jason A. Donenfeld" <[email protected]>
Doing this probing inside of the Makefiles means we have a maze of ifdefs inside the source code and child Makefiles that need to make proper decisions on this too. Instead, we do it at Kconfig time, like many other compiler and assembler options, which allows us to set up the dependencies normally for full compilation units. In the process, the ADX test changes to use %eax instead of %r10 so that it's valid in both 32-bit and 64-bit mode.
---
Changes in v2: None
arch/x86/Kconfig | 2 ++ arch/x86/Kconfig.assembler | 17 +++++++++++++++++ arch/x86/Makefile | 10 ---------- 3 files changed, 19 insertions(+), 10 deletions(-) create mode 100644 arch/x86/Kconfig.assembler
arch/x86/Kconfig +2 -0
@@ -2935,3 +2935,5 @@ config HAVE_ATOMIC_IOMAP source "drivers/firmware/Kconfig" source "arch/x86/kvm/Kconfig" + +source "arch/x86/Kconfig.assembler"
arch/x86/Kconfig.assembler new file +17 -0
@@ -0,0 +1,17 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2020 Jason A. Donenfeld <[email protected]>. All Rights Reserved. + +config AS_AVX2 + def_bool $(as-instr,vpbroadcastb %xmm0$(comma)%ymm1) + +config AS_AVX512 + def_bool $(as-instr,vpmovm2b %k1$(comma)%zmm5) + +config AS_SHA1_NI + def_bool $(as-instr,sha1msg1 %xmm0$(comma)%xmm1) + +config AS_SHA256_NI + def_bool $(as-instr,sha256msg1 %xmm0$(comma)%xmm1) + +config AS_ADX + def_bool $(as-instr,adox %eax$(comma)%eax)
arch/x86/Makefile +0 -10
@@ -177,16 +177,6 @@ ifeq ($(ACCUMULATE_OUTGOING_ARGS), 1) KBUILD_CFLAGS += $(call cc-option,-maccumulate-outgoing-args,) endif -# does binutils support specific instructions? -avx2_instr :=$(call as-instr,vpbroadcastb %xmm0$(comma)%ymm1,-DCONFIG_AS_AVX2=1) -avx512_instr :=$(call as-instr,vpmovm2b %k1$(comma)%zmm5,-DCONFIG_AS_AVX512=1) -sha1_ni_instr :=$(call as-instr,sha1msg1 %xmm0$(comma)%xmm1,-DCONFIG_AS_SHA1_NI=1) -sha256_ni_instr :=$(call as-instr,sha256msg1 %xmm0$(comma)%xmm1,-DCONFIG_AS_SHA256_NI=1) -adx_instr := $(call as-instr,adox %r10$(comma)%r10,-DCONFIG_AS_ADX=1) - -KBUILD_AFLAGS += $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) -KBUILD_CFLAGS += $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) - KBUILD_LDFLAGS := -m elf_$(UTS_MACHINE) #
Signature · 1 line
2.17.1
PATCH v2 3/16 x86: remove always-defined CONFIG_AS_CFI
In reply to Masahiro YamadaCONFIG_AS_CFI was introduced by commit e2414910f212 ("[PATCH] x86:
Detect CFI support in the assembler at runtime"), and extended by commit f0f12d85af85 ("x86_64: Check for .cfi_rel_offset in CFI probe").
We raise the minimal supported binutils version from time to time. The last bump was commit 1fb12b35e5ff ("kbuild: Raise the minimum required binutils version to 2.21").
I confirmed the code in $(call as-instr,...) can be assembled by the binutils 2.21 assembler and also by LLVM integrated assembler.
Remove CONFIG_AS_CFI, which is always defined.
---
If this series is applied,
I can hard-code the assembler code, and delete CFI_* macros entirely.
Changes in v2: None
arch/x86/Makefile | 10 ++-------- arch/x86/include/asm/dwarf2.h | 36 ----------------------------------- 2 files changed, 2 insertions(+), 44 deletions(-)
arch/x86/Makefile +2 -8
@@ -177,12 +177,6 @@ ifeq ($(ACCUMULATE_OUTGOING_ARGS), 1) KBUILD_CFLAGS += $(call cc-option,-maccumulate-outgoing-args,) endif -# Stackpointer is addressed different for 32 bit and 64 bit x86 -sp-$(CONFIG_X86_32) := esp -sp-$(CONFIG_X86_64) := rsp - -# do binutils support CFI? -cfi := $(call as-instr,.cfi_startproc\n.cfi_rel_offset $(sp-y)$(comma)0\n.cfi_endproc,-DCONFIG_AS_CFI=1) # is .cfi_signal_frame supported too? cfi-sigframe := $(call as-instr,.cfi_startproc\n.cfi_signal_frame\n.cfi_endproc,-DCONFIG_AS_CFI_SIGNAL_FRAME=1) cfi-sections := $(call as-instr,.cfi_sections .debug_frame,-DCONFIG_AS_CFI_SECTIONS=1)
@@ -196,8 +190,8 @@ sha1_ni_instr :=$(call as-instr,sha1msg1 %xmm0$(comma)%xmm1,-DCONFIG_AS_SHA1_NI= sha256_ni_instr :=$(call as-instr,sha256msg1 %xmm0$(comma)%xmm1,-DCONFIG_AS_SHA256_NI=1) adx_instr := $(call as-instr,adox %r10$(comma)%r10,-DCONFIG_AS_ADX=1) -KBUILD_AFLAGS += $(cfi) $(cfi-sigframe) $(cfi-sections) $(asinstr) $(avx_instr) $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) -KBUILD_CFLAGS += $(cfi) $(cfi-sigframe) $(cfi-sections) $(asinstr) $(avx_instr) $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) +KBUILD_AFLAGS += $(cfi-sigframe) $(cfi-sections) $(asinstr) $(avx_instr) $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) +KBUILD_CFLAGS += $(cfi-sigframe) $(cfi-sections) $(asinstr) $(avx_instr) $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) KBUILD_LDFLAGS := -m elf_$(UTS_MACHINE)
arch/x86/include/asm/dwarf2.h +0 -36
@@ -6,15 +6,6 @@#warning "asm/dwarf2.h should be only included in pure assembly files"#endif -/* -* Macros for dwarf2 CFI unwind table entries. -* See "as.info" for details on these pseudo ops. Unfortunately -* they are only supported in very new binutils, so define them -* away for older version. -*/ - -#ifdef CONFIG_AS_CFI -#define CFI_STARTPROC .cfi_startproc#define CFI_ENDPROC .cfi_endproc#define CFI_DEF_CFA .cfi_def_cfa
@@ -55,31 +46,4 @@#endif #endif -#else - -/* -* Due to the structure of pre-exisiting code, don't use assembler line -* comment character # to ignore the arguments. Instead, use a dummy macro. -*/ -.macro cfi_ignore a=0, b=0, c=0, d=0 -.endm - -#define CFI_STARTPROC cfi_ignore -#define CFI_ENDPROC cfi_ignore -#define CFI_DEF_CFA cfi_ignore -#define CFI_DEF_CFA_REGISTER cfi_ignore -#define CFI_DEF_CFA_OFFSET cfi_ignore -#define CFI_ADJUST_CFA_OFFSET cfi_ignore -#define CFI_OFFSET cfi_ignore -#define CFI_REL_OFFSET cfi_ignore -#define CFI_REGISTER cfi_ignore -#define CFI_RESTORE cfi_ignore -#define CFI_REMEMBER_STATE cfi_ignore -#define CFI_RESTORE_STATE cfi_ignore -#define CFI_UNDEFINED cfi_ignore -#define CFI_ESCAPE cfi_ignore -#define CFI_SIGNAL_FRAME cfi_ignore - -#endif -#endif /* _ASM_X86_DWARF2_H */
Signature · 1 line
2.17.1
PATCH v2 4/16 x86: remove unneeded (CONFIG_AS_)CFI_SIGNAL_FRAME
In reply to Masahiro YamadaCommit 131484c8da97 ("x86/debug: Remove perpetually broken, unmaintainable dwarf annotations") removes all the users of CFI_SIGNAL_FRAME.
Remove the CFI_SIGNAL_FRAME and CONFIG_AS_CFI_SIGNAL_FRAME.
---
Changes in v2: None
arch/x86/Makefile | 6 ++---- arch/x86/include/asm/dwarf2.h | 6 ------ 2 files changed, 2 insertions(+), 10 deletions(-)
arch/x86/Makefile +2 -4
@@ -177,8 +177,6 @@ ifeq ($(ACCUMULATE_OUTGOING_ARGS), 1) KBUILD_CFLAGS += $(call cc-option,-maccumulate-outgoing-args,) endif -# is .cfi_signal_frame supported too? -cfi-sigframe := $(call as-instr,.cfi_startproc\n.cfi_signal_frame\n.cfi_endproc,-DCONFIG_AS_CFI_SIGNAL_FRAME=1) cfi-sections := $(call as-instr,.cfi_sections .debug_frame,-DCONFIG_AS_CFI_SECTIONS=1) # does binutils support specific instructions?
@@ -190,8 +188,8 @@ sha1_ni_instr :=$(call as-instr,sha1msg1 %xmm0$(comma)%xmm1,-DCONFIG_AS_SHA1_NI= sha256_ni_instr :=$(call as-instr,sha256msg1 %xmm0$(comma)%xmm1,-DCONFIG_AS_SHA256_NI=1) adx_instr := $(call as-instr,adox %r10$(comma)%r10,-DCONFIG_AS_ADX=1) -KBUILD_AFLAGS += $(cfi-sigframe) $(cfi-sections) $(asinstr) $(avx_instr) $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) -KBUILD_CFLAGS += $(cfi-sigframe) $(cfi-sections) $(asinstr) $(avx_instr) $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) +KBUILD_AFLAGS += $(cfi-sections) $(asinstr) $(avx_instr) $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) +KBUILD_CFLAGS += $(cfi-sections) $(asinstr) $(avx_instr) $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) KBUILD_LDFLAGS := -m elf_$(UTS_MACHINE)
arch/x86/include/asm/dwarf2.h +0 -6
@@ -21,12 +21,6 @@#define CFI_UNDEFINED .cfi_undefined#define CFI_ESCAPE .cfi_escape -#ifdef CONFIG_AS_CFI_SIGNAL_FRAME -#define CFI_SIGNAL_FRAME .cfi_signal_frame -#else -#define CFI_SIGNAL_FRAME -#endif -#if defined(CONFIG_AS_CFI_SECTIONS )#ifndef BUILD_VDSO /*
Signature · 1 line
2.17.1
PATCH v2 5/16 x86: remove always-defined CONFIG_AS_CFI_SECTIONS
In reply to Masahiro YamadaCONFIG_AS_CFI_SECTIONS was introduced by commit 9e565292270a ("x86:
Use .cfi_sections for assembly code").
We raise the minimal supported binutils version from time to time. The last bump was commit 1fb12b35e5ff ("kbuild: Raise the minimum required binutils version to 2.21").
I confirmed the code in $(call as-instr,...) can be assembled by the binutils 2.21 assembler and also by LLVM integrated assembler.
Remove CONFIG_AS_CFI_SECTIONS, which is always defined.
---
Changes in v2: None
arch/x86/Makefile | 6 ++---- arch/x86/include/asm/dwarf2.h | 2 -- 2 files changed, 2 insertions(+), 6 deletions(-)
arch/x86/Makefile +2 -4
@@ -177,8 +177,6 @@ ifeq ($(ACCUMULATE_OUTGOING_ARGS), 1) KBUILD_CFLAGS += $(call cc-option,-maccumulate-outgoing-args,) endif -cfi-sections := $(call as-instr,.cfi_sections .debug_frame,-DCONFIG_AS_CFI_SECTIONS=1) - # does binutils support specific instructions? asinstr += $(call as-instr,pshufb %xmm0$(comma)%xmm0,-DCONFIG_AS_SSSE3=1) avx_instr := $(call as-instr,vxorps %ymm0$(comma)%ymm1$(comma)%ymm2,-DCONFIG_AS_AVX=1)
@@ -188,8 +186,8 @@ sha1_ni_instr :=$(call as-instr,sha1msg1 %xmm0$(comma)%xmm1,-DCONFIG_AS_SHA1_NI= sha256_ni_instr :=$(call as-instr,sha256msg1 %xmm0$(comma)%xmm1,-DCONFIG_AS_SHA256_NI=1) adx_instr := $(call as-instr,adox %r10$(comma)%r10,-DCONFIG_AS_ADX=1) -KBUILD_AFLAGS += $(cfi-sections) $(asinstr) $(avx_instr) $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) -KBUILD_CFLAGS += $(cfi-sections) $(asinstr) $(avx_instr) $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) +KBUILD_AFLAGS += $(asinstr) $(avx_instr) $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) +KBUILD_CFLAGS += $(asinstr) $(avx_instr) $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) KBUILD_LDFLAGS := -m elf_$(UTS_MACHINE)
arch/x86/include/asm/dwarf2.h +0 -2
@@ -21,7 +21,6 @@#define CFI_UNDEFINED .cfi_undefined#define CFI_ESCAPE .cfi_escape -#if defined(CONFIG_AS_CFI_SECTIONS )#ifndef BUILD_VDSO /* * Emit CFI data in .debug_frame sections, not .eh_frame sections.
@@ -38,6 +37,5 @@*/ .cfi_sections .eh_frame, .debug_frame#endif -#endif #endif /* _ASM_X86_DWARF2_H */
Signature · 1 line
2.17.1
PATCH v2 2/16 x86: remove unneeded defined(__ASSEMBLY__) check from asm/dwarf2.h
In reply to Masahiro YamadaThis header file has the following check at the top:
#ifndef __ASSEMBLY__ #warning "asm/dwarf2.h should be only included in pure assembly files" #endif
So, we expect defined(__ASSEMBLY__) is always true.
---
Changes in v2: None
arch/x86/include/asm/dwarf2.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/include/asm/dwarf2.h +1 -1
@@ -36,7 +36,7 @@#define CFI_SIGNAL_FRAME #endif -#if defined(CONFIG_AS_CFI_SECTIONS ) && defined(__ASSEMBLY__) +#if defined(CONFIG_AS_CFI_SECTIONS )#ifndef BUILD_VDSO /* * Emit CFI data in .debug_frame sections, not .eh_frame sections.
Signature · 1 line
2.17.1
PATCH v2 11/16 x86: add comments about the binutils version to support code in as-instr
In reply to Masahiro YamadaWe raise the minimal supported binutils version from time to time. The last bump was commit 1fb12b35e5ff ("kbuild: Raise the minimum required binutils version to 2.21").
We need to keep these as-instr checks because binutils 2.21 does not support them.
I hope this will be a good hint which one can be dropped when we bump the minimal binutils version next time.
As for the Clang/LLVM builds, we require very new LLVM version, so the LLVM integrated assembler supports all of them.
---
Changes in v2:
- Change the patch order and rebase
arch/x86/Kconfig.assembler | 10 ++++++++++ 1 file changed, 10 insertions(+)
arch/x86/Kconfig.assembler +10 -0
@@ -3,15 +3,25 @@ config AS_AVX2 def_bool $(as-instr,vpbroadcastb %xmm0$(comma)%ymm1) + help + Supported by binutils >= 2.22 and LLVM integrated assembler config AS_AVX512 def_bool $(as-instr,vpmovm2b %k1$(comma)%zmm5) + help + Supported by binutils >= 2.25 and LLVM integrated assembler config AS_SHA1_NI def_bool $(as-instr,sha1msg1 %xmm0$(comma)%xmm1) + help + Supported by binutils >= 2.24 and LLVM integrated assembler config AS_SHA256_NI def_bool $(as-instr,sha256msg1 %xmm0$(comma)%xmm1) + help + Supported by binutils >= 2.24 and LLVM integrated assembler config AS_ADX def_bool $(as-instr,adox %eax$(comma)%eax) + help + Supported by binutils >= 2.23 and LLVM integrated assembler
Signature · 1 line
2.17.1
PATCH v2 1/16 lib/raid6/test: fix build on distros whose /bin/sh is not bash
In reply to Masahiro YamadaYou can build a test program to test raid6 library code from user-space, like this:
$ cd lib/raid6/test $ make
The command in $(shell ...) function is evaluated by /bin/sh by default. (or, you can specify the shell by passing SHELL=<shell> from command line)
Currently '>&/dev/null' is used to sink both stdout and stderr. Because this code is bash-ism, it only works when /bin/sh is a symbolic link to bash (this is the case on RHEL etc.)
This does not work on Ubuntu where /bin/sh is a symbolic link to dash.
I see lots of
/bin/sh: 1: Syntax error: Bad fd number
and
warning "your version of binutils lacks ... support"
Replace it with portable '>/dev/null 2>&1'.
---
Changes in v2: None
lib/raid6/test/Makefile | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
lib/raid6/test/Makefile +3 -3
@@ -35,13 +35,13 @@ endif ifeq ($(IS_X86),yes) OBJS += mmx.o sse1.o sse2.o avx2.o recov_ssse3.o recov_avx2.o avx512.o recov_avx512.o CFLAGS += $(shell echo "pshufb %xmm0, %xmm0" | \ - gcc -c -x assembler - >&/dev/null && \ + gcc -c -x assembler - >/dev/null 2>&1 && \ rm ./-.o && echo -DCONFIG_AS_SSSE3=1) CFLAGS += $(shell echo "vpbroadcastb %xmm0, %ymm1" | \ - gcc -c -x assembler - >&/dev/null && \ + gcc -c -x assembler - >/dev/null 2>&1 && \ rm ./-.o && echo -DCONFIG_AS_AVX2=1) CFLAGS += $(shell echo "vpmovm2b %k1, %zmm5" | \ - gcc -c -x assembler - >&/dev/null && \ + gcc -c -x assembler - >/dev/null 2>&1 && \ rm ./-.o && echo -DCONFIG_AS_AVX512=1) else ifeq ($(HAS_NEON),yes) OBJS += neon.o neon1.o neon2.o neon4.o neon8.o recov_neon.o recov_neon_inner.o
Signature · 1 line
2.17.1
PATCH v2 14/16 Documentation/changes: Raise minimum supported binutils version to 2.23
In reply to Masahiro YamadaFrom: Borislav Petkov <[email protected]>
The currently minimum-supported binutils version 2.21 has the problem of promoting symbols which are defined outside of a section into absolute. According to Arvind:
binutils-2.21 and -2.22. An x86-64 defconfig will fail with
Invalid absolute R_X86_64_32S relocation: _etext
and after fixing that one, with
Invalid absolute R_X86_64_32S relocation: __end_of_kernel_reserve
Those two versions of binutils have a bug when it comes to handling symbols defined outside of a section and binutils 2.23 has the proper fix, see: https://sourceware.org/legacy-ml/binutils/2012-06/msg00155.html
Therefore, up to the fixed version directly, skipping the broken ones.
Currently shipping distros already have the fixed binutils version so there should be no breakage resulting from this.
For more details about the whole thing, see the thread in Link.
---
Changes in v2: None
Documentation/process/changes.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Documentation/process/changes.rst +2 -2
@@ -31,7 +31,7 @@ you probably needn't concern yourself with pcmciautils. ====================== =============== ======================================== GNU C 4.6 gcc --version GNU make 3.81 make --version -binutils 2.21 ld -v +binutils 2.23 ld -v flex 2.5.35 flex --version bison 2.0 bison --version util-linux 2.10o fdformat --version
@@ -76,7 +76,7 @@ You will need GNU make 3.81 or later to build the kernel. Binutils -------- -Binutils 2.21 or newer is needed to build the kernel. +Binutils 2.23 or newer is needed to build the kernel. pkg-config ----------
Signature · 1 line
2.17.1
PATCH v2 13/16 crypto: curve25519 - do not pollute dispatcher based on assembler
In reply to Masahiro YamadaFrom: "Jason A. Donenfeld" <[email protected]>
Since we're doing a static inline dispatch here, we normally branch based on whether or not there's an arch implementation. That would have been fine in general, except the crypto Makefile prior used to turn things off -- despite the Kconfig -- resulting in us needing to also hard code various assembler things into the dispatcher too. The horror! Now that the assembler config options are done by Kconfig, we can get rid of the inconsistency.
---
Changes in v2: None
include/crypto/curve25519.h | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
include/crypto/curve25519.h +2 -4
@@ -33,8 +33,7 @@ bool __must_check curve25519(u8 mypublic[CURVE25519_KEY_SIZE],const u8 secret[CURVE25519_KEY_SIZE ], const u8 basepoint[CURVE25519_KEY_SIZE]) { - if (IS_ENABLED(CONFIG_CRYPTO_ARCH_HAVE_LIB_CURVE25519) && - (!IS_ENABLED(CONFIG_CRYPTO_CURVE25519_X86) || IS_ENABLED(CONFIG_AS_ADX))) + if (IS_ENABLED(CONFIG_CRYPTO_ARCH_HAVE_LIB_CURVE25519)) curve25519_arch(mypublic, secret, basepoint); elsecurve25519_generic (mypublic ,secret ,basepoint );
@@ -50,8 +49,7 @@ __must_check curve25519_generate_public(u8 pub[CURVE25519_KEY_SIZE], CURVE25519_KEY_SIZE)))return false; -if (IS_ENABLED (CONFIG_CRYPTO_ARCH_HAVE_LIB_CURVE25519 ) && - (!IS_ENABLED (CONFIG_CRYPTO_CURVE25519_X86 ) ||IS_ENABLED (CONFIG_AS_ADX ))) +if (IS_ENABLED (CONFIG_CRYPTO_ARCH_HAVE_LIB_CURVE25519 ))curve25519_base_arch (pub, secret);else curve25519_generic (pub, secret, curve25519_base_point);
Signature · 1 line
2.17.1
PATCH v2 16/16 crypto: x86 - clean up poly1305-x86_64-cryptogams.S by 'make clean'
In reply to Masahiro Yamadapoly1305-x86_64-cryptogams.S is a generated file, so it should be cleaned up by 'make clean'.
Assigning it to the variable 'targets' teaches Kbuild that it is a generated file. However, this line is not evaluated while cleaning because scripts/Makefile.clean does not include include/config/auto.conf.
Removing the ifneq-conditional, so this file is correctly cleaned up.
---
Changes in v2: None
arch/x86/crypto/Makefile | 2 -- 1 file changed, 2 deletions(-)
arch/x86/crypto/Makefile +0 -2
@@ -83,9 +83,7 @@ crct10dif-pclmul-y := crct10dif-pcl-asm_64.o crct10dif-pclmul_glue.o obj-$(CONFIG_CRYPTO_POLY1305_X86_64) += poly1305-x86_64.o poly1305-x86_64-y := poly1305-x86_64-cryptogams.o poly1305_glue.o -ifneq ($(CONFIG_CRYPTO_POLY1305_X86_64),) targets += poly1305-x86_64-cryptogams.S -endif obj-$(CONFIG_CRYPTO_NHPOLY1305_SSE2) += nhpoly1305-sse2.o nhpoly1305-sse2-y := nh-sse2-x86_64.o nhpoly1305-sse2-glue.o
Signature · 1 line
2.17.1
PATCH v2 6/16 x86: remove always-defined CONFIG_AS_SSSE3
In reply to Masahiro YamadaCONFIG_AS_SSSE3 was introduced by commit 75aaf4c3e6a4 ("x86/raid6:
correctly check for assembler capabilities").
We raise the minimal supported binutils version from time to time. The last bump was commit 1fb12b35e5ff ("kbuild: Raise the minimum required binutils version to 2.21").
I confirmed the code in $(call as-instr,...) can be assembled by the binutils 2.21 assembler and also by LLVM integrated assembler.
Remove CONFIG_AS_SSSE3, which is always defined.
I added ifdef CONFIG_X86 to lib/raid6/algos.c to avoid link errors on non-x86 architectures.
lib/raid6/algos.c is built not only for the kernel but also for testing the library code from userspace. I added -DCONFIG_X86 to lib/raid6/test/Makefile to cator to this usecase.
---
Changes in v2: None
arch/x86/Makefile | 5 ++--- arch/x86/crypto/blake2s-core.S | 2 -- lib/raid6/algos.c | 2 +- lib/raid6/recov_ssse3.c | 6 ------ lib/raid6/test/Makefile | 4 +--- 5 files changed, 4 insertions(+), 15 deletions(-)
arch/x86/Makefile +2 -3
@@ -178,7 +178,6 @@ ifeq ($(ACCUMULATE_OUTGOING_ARGS), 1) endif # does binutils support specific instructions? -asinstr += $(call as-instr,pshufb %xmm0$(comma)%xmm0,-DCONFIG_AS_SSSE3=1) avx_instr := $(call as-instr,vxorps %ymm0$(comma)%ymm1$(comma)%ymm2,-DCONFIG_AS_AVX=1) avx2_instr :=$(call as-instr,vpbroadcastb %xmm0$(comma)%ymm1,-DCONFIG_AS_AVX2=1) avx512_instr :=$(call as-instr,vpmovm2b %k1$(comma)%zmm5,-DCONFIG_AS_AVX512=1)
@@ -186,8 +185,8 @@ sha1_ni_instr :=$(call as-instr,sha1msg1 %xmm0$(comma)%xmm1,-DCONFIG_AS_SHA1_NI= sha256_ni_instr :=$(call as-instr,sha256msg1 %xmm0$(comma)%xmm1,-DCONFIG_AS_SHA256_NI=1) adx_instr := $(call as-instr,adox %r10$(comma)%r10,-DCONFIG_AS_ADX=1) -KBUILD_AFLAGS += $(asinstr) $(avx_instr) $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) -KBUILD_CFLAGS += $(asinstr) $(avx_instr) $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) +KBUILD_AFLAGS += $(avx_instr) $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) +KBUILD_CFLAGS += $(avx_instr) $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) KBUILD_LDFLAGS := -m elf_$(UTS_MACHINE)
arch/x86/crypto/blake2s-core.S +0 -2
@@ -46,7 +46,6 @@ SIGMA2: #endif /* CONFIG_AS_AVX512 */ .text -#ifdef CONFIG_AS_SSSE3 SYM_FUNC_START(blake2s_compress_ssse3) testq %rdx,%rdx je .Lendofloop
@@ -174,7 +173,6 @@ SYM_FUNC_START(blake2s_compress_ssse3) .Lendofloop: ret SYM_FUNC_END(blake2s_compress_ssse3) -#endif /* CONFIG_AS_SSSE3 */ #ifdef CONFIG_AS_AVX512 SYM_FUNC_START(blake2s_compress_avx512)
lib/raid6/algos.c +1 -1
@@ -97,13 +97,13 @@ void (*raid6_datap_recov)(int, size_t, int, void **);EXPORT_SYMBOL_GPL (raid6_datap_recov);const struct raid6_recov_calls *const raid6_recov_algos[] = { +#ifdef CONFIG_X86 #ifdef CONFIG_AS_AVX512 &raid6_recov_avx512, #endif #ifdef CONFIG_AS_AVX2 &raid6_recov_avx2, #endif -#ifdef CONFIG_AS_SSSE3 &raid6_recov_ssse3, #endif #ifdef CONFIG_S390
lib/raid6/recov_ssse3.c +0 -6
@@ -3,8 +3,6 @@* Copyright (C) 2012 Intel Corporation */ -#ifdef CONFIG_AS_SSSE3 -#include <linux/raid/pq.h> #include "x86.h"
@@ -328,7 +326,3 @@ const struct raid6_recov_calls raid6_recov_ssse3 = {#endif .priority = 1, }; - -#else -#warning "your version of binutils lacks SSSE3 support" -#endif
lib/raid6/test/Makefile +1 -3
@@ -34,9 +34,7 @@ endif ifeq ($(IS_X86),yes) OBJS += mmx.o sse1.o sse2.o avx2.o recov_ssse3.o recov_avx2.o avx512.o recov_avx512.o - CFLAGS += $(shell echo "pshufb %xmm0, %xmm0" | \ - gcc -c -x assembler - >/dev/null 2>&1 && \ - rm ./-.o && echo -DCONFIG_AS_SSSE3=1) + CFLAGS += -DCONFIG_X86 CFLAGS += $(shell echo "vpbroadcastb %xmm0, %ymm1" | \ gcc -c -x assembler - >/dev/null 2>&1 && \ rm ./-.o && echo -DCONFIG_AS_AVX2=1)
Signature · 1 line
2.17.1
PATCH v2 7/16 x86: remove always-defined CONFIG_AS_AVX
In reply to Masahiro YamadaCONFIG_AS_AVX was introduced by commit ea4d26ae24e5 ("raid5: add AVX optimized RAID5 checksumming").
We raise the minimal supported binutils version from time to time. The last bump was commit 1fb12b35e5ff ("kbuild: Raise the minimum required binutils version to 2.21").
I confirmed the code in $(call as-instr,...) can be assembled by the binutils 2.21 assembler and also by LLVM integrated assembler.
Remove CONFIG_AS_AVX, which is always defined.
---
Changes in v2: None
arch/x86/Makefile | 5 ++- arch/x86/crypto/Makefile | 32 +++++++------------ arch/x86/crypto/aesni-intel_avx-x86_64.S | 3 -- arch/x86/crypto/aesni-intel_glue.c | 14 +------- arch/x86/crypto/poly1305-x86_64-cryptogams.pl | 8 ----- arch/x86/crypto/poly1305_glue.c | 6 ++-- arch/x86/crypto/sha1_ssse3_asm.S | 4 --- arch/x86/crypto/sha1_ssse3_glue.c | 9 +----- arch/x86/crypto/sha256-avx-asm.S | 3 -- arch/x86/crypto/sha256_ssse3_glue.c | 8 +---- arch/x86/crypto/sha512-avx-asm.S | 2 -- arch/x86/crypto/sha512_ssse3_glue.c | 7 +--- arch/x86/include/asm/xor_avx.h | 9 ------ 13 files changed, 21 insertions(+), 89 deletions(-)
arch/x86/Makefile +2 -3
@@ -178,15 +178,14 @@ ifeq ($(ACCUMULATE_OUTGOING_ARGS), 1) endif # does binutils support specific instructions? -avx_instr := $(call as-instr,vxorps %ymm0$(comma)%ymm1$(comma)%ymm2,-DCONFIG_AS_AVX=1) avx2_instr :=$(call as-instr,vpbroadcastb %xmm0$(comma)%ymm1,-DCONFIG_AS_AVX2=1) avx512_instr :=$(call as-instr,vpmovm2b %k1$(comma)%zmm5,-DCONFIG_AS_AVX512=1) sha1_ni_instr :=$(call as-instr,sha1msg1 %xmm0$(comma)%xmm1,-DCONFIG_AS_SHA1_NI=1) sha256_ni_instr :=$(call as-instr,sha256msg1 %xmm0$(comma)%xmm1,-DCONFIG_AS_SHA256_NI=1) adx_instr := $(call as-instr,adox %r10$(comma)%r10,-DCONFIG_AS_ADX=1) -KBUILD_AFLAGS += $(avx_instr) $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) -KBUILD_CFLAGS += $(avx_instr) $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) +KBUILD_AFLAGS += $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) +KBUILD_CFLAGS += $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) KBUILD_LDFLAGS := -m elf_$(UTS_MACHINE)
arch/x86/crypto/Makefile +12 -20
@@ -5,7 +5,6 @@ OBJECT_FILES_NON_STANDARD := y -avx_supported := $(call as-instr,vpxor %xmm0$(comma)%xmm0$(comma)%xmm0,yes,no) avx2_supported := $(call as-instr,vpgatherdd %ymm0$(comma)(%eax$(comma)%ymm1\ $(comma)4)$(comma)%ymm2,yes,no) avx512_supported :=$(call as-instr,vpmovm2b %k1$(comma)%zmm5,yes,no)
@@ -47,15 +46,12 @@ ifeq ($(adx_supported),yes) endif # These modules require assembler to support AVX. -ifeq ($(avx_supported),yes) - obj-$(CONFIG_CRYPTO_CAMELLIA_AESNI_AVX_X86_64) += \ - camellia-aesni-avx-x86_64.o - obj-$(CONFIG_CRYPTO_CAST5_AVX_X86_64) += cast5-avx-x86_64.o - obj-$(CONFIG_CRYPTO_CAST6_AVX_X86_64) += cast6-avx-x86_64.o - obj-$(CONFIG_CRYPTO_TWOFISH_AVX_X86_64) += twofish-avx-x86_64.o - obj-$(CONFIG_CRYPTO_SERPENT_AVX_X86_64) += serpent-avx-x86_64.o - obj-$(CONFIG_CRYPTO_BLAKE2S_X86) += blake2s-x86_64.o -endif +obj-$(CONFIG_CRYPTO_CAMELLIA_AESNI_AVX_X86_64) += camellia-aesni-avx-x86_64.o +obj-$(CONFIG_CRYPTO_CAST5_AVX_X86_64) += cast5-avx-x86_64.o +obj-$(CONFIG_CRYPTO_CAST6_AVX_X86_64) += cast6-avx-x86_64.o +obj-$(CONFIG_CRYPTO_TWOFISH_AVX_X86_64) += twofish-avx-x86_64.o +obj-$(CONFIG_CRYPTO_SERPENT_AVX_X86_64) += serpent-avx-x86_64.o +obj-$(CONFIG_CRYPTO_BLAKE2S_X86) += blake2s-x86_64.o # These modules require assembler to support AVX2. ifeq ($(avx2_supported),yes)
@@ -83,16 +79,12 @@ ifneq ($(CONFIG_CRYPTO_POLY1305_X86_64),) targets += poly1305-x86_64-cryptogams.S endif -ifeq ($(avx_supported),yes) - camellia-aesni-avx-x86_64-y := camellia-aesni-avx-asm_64.o \ - camellia_aesni_avx_glue.o - cast5-avx-x86_64-y := cast5-avx-x86_64-asm_64.o cast5_avx_glue.o - cast6-avx-x86_64-y := cast6-avx-x86_64-asm_64.o cast6_avx_glue.o - twofish-avx-x86_64-y := twofish-avx-x86_64-asm_64.o \ - twofish_avx_glue.o - serpent-avx-x86_64-y := serpent-avx-x86_64-asm_64.o \ - serpent_avx_glue.o -endif +camellia-aesni-avx-x86_64-y := camellia-aesni-avx-asm_64.o \ + camellia_aesni_avx_glue.o +cast5-avx-x86_64-y := cast5-avx-x86_64-asm_64.o cast5_avx_glue.o +cast6-avx-x86_64-y := cast6-avx-x86_64-asm_64.o cast6_avx_glue.o +twofish-avx-x86_64-y := twofish-avx-x86_64-asm_64.o twofish_avx_glue.o +serpent-avx-x86_64-y := serpent-avx-x86_64-asm_64.o serpent_avx_glue.o ifeq ($(avx2_supported),yes) camellia-aesni-avx2-y := camellia-aesni-avx2-asm_64.o camellia_aesni_avx2_glue.o
arch/x86/crypto/aesni-intel_avx-x86_64.S +0 -3
@@ -886,7 +886,6 @@ _less_than_8_bytes_left_\@: _partial_block_done_\@: .endm # PARTIAL_BLOCK -#ifdef CONFIG_AS_AVX ############################################################################### # GHASH_MUL MACRO to implement: Data*HashKey mod (128,127,126,121,0) # Input: A and B (128-bits each, bit-reflected)
@@ -1869,8 +1868,6 @@ key_256_finalize: ret SYM_FUNC_END(aesni_gcm_finalize_avx_gen2) -#endif /* CONFIG_AS_AVX */ - #ifdef CONFIG_AS_AVX2 ############################################################################### # GHASH_MUL MACRO to implement: Data*HashKey mod (128,127,126,121,0)
arch/x86/crypto/aesni-intel_glue.c +1 -13
@@ -185,7 +185,6 @@ static const struct aesni_gcm_tfm_s aesni_gcm_tfm_sse = { .finalize = &aesni_gcm_finalize, }; -#ifdef CONFIG_AS_AVX asmlinkage void aes_ctr_enc_128_avx_by8(const u8 *in, u8 *iv,void *keys,u8 *out,unsigned int num_bytes); asmlinkage void aes_ctr_enc_192_avx_by8(const u8 *in, u8 *iv,
@@ -234,8 +233,6 @@ static const struct aesni_gcm_tfm_s aesni_gcm_tfm_avx_gen2 = { .finalize = &aesni_gcm_finalize_avx_gen2, }; -#endif -#ifdef CONFIG_AS_AVX2 /* * asmlinkage void aesni_gcm_init_avx_gen4()
@@ -476,7 +473,6 @@ static void ctr_crypt_final(struct crypto_aes_ctx *ctx,crypto_inc (ctrblk,AES_BLOCK_SIZE ); } -#ifdef CONFIG_AS_AVX static void aesni_ctr_enc_avx_tfm (struct crypto_aes_ctx *ctx,u8 *out,const u8 *in,unsigned int len,u8 *iv) {
@@ -493,7 +489,6 @@ static void aesni_ctr_enc_avx_tfm(struct crypto_aes_ctx *ctx, u8 *out,else aes_ctr_enc_256_avx_by8 (in, iv, (void *)ctx, out, len); } -#endif static int ctr_crypt (struct skcipher_request *req) {
@@ -715,10 +710,8 @@ static int gcmaes_crypt_by_sg(bool enc, struct aead_request *req,if (left <AVX_GEN4_OPTSIZE && gcm_tfm == &aesni_gcm_tfm_avx_gen4) gcm_tfm = &aesni_gcm_tfm_avx_gen2;#endif -#ifdef CONFIG_AS_AVX if (left <AVX_GEN2_OPTSIZE && gcm_tfm == &aesni_gcm_tfm_avx_gen2) gcm_tfm = &aesni_gcm_tfm_sse; -#endif /* Linearize assoc, if not already linear */ if (req->src->length >= assoclen && req->src->length &&
@@ -1082,24 +1075,19 @@ static int __init aesni_init(void) aesni_gcm_tfm = &aesni_gcm_tfm_avx_gen4; } else#endif -#ifdef CONFIG_AS_AVX if (boot_cpu_has (X86_FEATURE_AVX )) {pr_info ("AVX version of gcm_enc/dec engaged.\n" ); aesni_gcm_tfm = &aesni_gcm_tfm_avx_gen2; - } else -#endif - { + }else {pr_info ("SSE version of gcm_enc/dec engaged.\n" ); aesni_gcm_tfm = &aesni_gcm_tfm_sse; } aesni_ctr_enc_tfm = aesni_ctr_enc; -#ifdef CONFIG_AS_AVX if (boot_cpu_has (X86_FEATURE_AVX )) {/* optimize performance of ctr mode encryption transform */ aesni_ctr_enc_tfm = aesni_ctr_enc_avx_tfm;pr_info ("AES CTR mode by8 optimization enabled\n" ); } -#endif #endif err =crypto_register_alg (&aesni_cipher_alg);
arch/x86/crypto/poly1305-x86_64-cryptogams.pl +0 -8
@@ -404,10 +404,6 @@ ___ &end_function("poly1305_emit_x86_64"); if ($avx) { -if($kernel) { - $code .= "#ifdef CONFIG_AS_AVX\n"; -} - ######################################################################## # Layout of opaque area is following. #
@@ -1516,10 +1512,6 @@ $code.=<<___; ___ &end_function("poly1305_emit_avx"); -if ($kernel) { - $code .= "#endif\n"; -} - if ($avx>1) { if ($kernel) {
arch/x86/crypto/poly1305_glue.c +3 -3
@@ -94,7 +94,7 @@ static void poly1305_simd_blocks(void *ctx, const u8 *inp, size_t len,BUILD_BUG_ON (PAGE_SIZE <POLY1305_BLOCK_SIZE ||PAGE_SIZE %POLY1305_BLOCK_SIZE ); -if (!IS_ENABLED (CONFIG_AS_AVX ) || !static_branch_likely (&poly1305_use_avx) || +if (!static_branch_likely (&poly1305_use_avx) || (len < (POLY1305_BLOCK_SIZE *18 ) && !state->is_base2_26) || !crypto_simd_usable ()) {convert_to_base2_64 (ctx);
@@ -123,7 +123,7 @@ static void poly1305_simd_blocks(void *ctx, const u8 *inp, size_t len,static void poly1305_simd_emit (void *ctx,u8 mac[POLY1305_DIGEST_SIZE ],const u32 nonce[4 ]) { -if (!IS_ENABLED (CONFIG_AS_AVX ) || !static_branch_likely (&poly1305_use_avx)) +if (!static_branch_likely (&poly1305_use_avx))poly1305_emit_x86_64 (ctx, mac, nonce);else poly1305_emit_avx (ctx, mac, nonce);
@@ -261,7 +261,7 @@ static struct shash_alg alg = {static int __init poly1305_simd_mod_init (void ) { -if (IS_ENABLED (CONFIG_AS_AVX ) &&boot_cpu_has (X86_FEATURE_AVX ) && +if (boot_cpu_has (X86_FEATURE_AVX ) &&cpu_has_xfeatures (XFEATURE_MASK_SSE |XFEATURE_MASK_YMM ,NULL ))static_branch_enable (&poly1305_use_avx); if (IS_ENABLED(CONFIG_AS_AVX2) && boot_cpu_has(X86_FEATURE_AVX) &&
arch/x86/crypto/sha1_ssse3_asm.S +0 -4
@@ -467,8 +467,6 @@ W_PRECALC_SSSE3 */ SHA1_VECTOR_ASM sha1_transform_ssse3 -#ifdef CONFIG_AS_AVX - .macro W_PRECALC_AVX .purgem W_PRECALC_00_15
@@ -553,5 +551,3 @@ W_PRECALC_AVX * const u8 *data, int blocks); */ SHA1_VECTOR_ASM sha1_transform_avx - -#endif
arch/x86/crypto/sha1_ssse3_glue.c +1 -8
@@ -114,7 +114,6 @@ static void unregister_sha1_ssse3(void)crypto_unregister_shash (&sha1_ssse3_alg); } -#ifdef CONFIG_AS_AVX asmlinkage void sha1_transform_avx(struct sha1_state *state,const u8 *data,int blocks);
@@ -175,13 +174,7 @@ static void unregister_sha1_avx(void)crypto_unregister_shash (&sha1_avx_alg); } -#else /* CONFIG_AS_AVX */ -static inline int register_sha1_avx (void ) {return 0 ; } -static inline void unregister_sha1_avx (void ) { } -#endif /* CONFIG_AS_AVX */ - - -#if defined(CONFIG_AS_AVX2 ) && (CONFIG_AS_AVX ) +#if defined(CONFIG_AS_AVX2 )#define SHA1_AVX2_BLOCK_OPTSIZE 4/* optimal 4*64 bytes of SHA1 blocks */ asmlinkage void sha1_transform_avx2(struct sha1_state *state,
arch/x86/crypto/sha256-avx-asm.S +0 -3
@@ -47,7 +47,6 @@ # This code schedules 1 block at a time, with 4 lanes per block ######################################################################## -#ifdef CONFIG_AS_AVX #include <linux/linkage.h> ## assume buffers not aligned
@@ -498,5 +497,3 @@ _SHUF_00BA: # shuffle xDxC -> DC00 _SHUF_DC00: .octa 0x0b0a090803020100FFFFFFFFFFFFFFFF - -#endif
arch/x86/crypto/sha256_ssse3_glue.c +1 -7
@@ -144,7 +144,6 @@ static void unregister_sha256_ssse3(void) ARRAY_SIZE(sha256_ssse3_algs)); } -#ifdef CONFIG_AS_AVX asmlinkage void sha256_transform_avx(struct sha256_state *state,const u8 *data,int blocks);
@@ -221,12 +220,7 @@ static void unregister_sha256_avx(void) ARRAY_SIZE(sha256_avx_algs)); } -#else -static inline int register_sha256_avx (void ) {return 0 ; } -static inline void unregister_sha256_avx (void ) { } -#endif - -#if defined(CONFIG_AS_AVX2 ) && defined(CONFIG_AS_AVX ) +#if defined(CONFIG_AS_AVX2 ) asmlinkage void sha256_transform_rorx(struct sha256_state *state,const u8 *data,int blocks);
arch/x86/crypto/sha512-avx-asm.S +0 -2
@@ -47,7 +47,6 @@ # ######################################################################## -#ifdef CONFIG_AS_AVX #include <linux/linkage.h> .text
@@ -424,4 +423,3 @@ K512: .quad 0x3c9ebe0a15c9bebc,0x431d67c49c100d4c .quad 0x4cc5d4becb3e42b6,0x597f299cfc657e2a .quad 0x5fcb6fab3ad6faec,0x6c44198c4a475817 -#endif
arch/x86/crypto/sha512_ssse3_glue.c +1 -6
@@ -142,7 +142,6 @@ static void unregister_sha512_ssse3(void) ARRAY_SIZE(sha512_ssse3_algs)); } -#ifdef CONFIG_AS_AVX asmlinkage void sha512_transform_avx(struct sha512_state *state,const u8 *data,int blocks);static bool avx_usable (void )
@@ -218,12 +217,8 @@ static void unregister_sha512_avx(void)crypto_unregister_shashes (sha512_avx_algs,ARRAY_SIZE (sha512_avx_algs)); } -#else -static inline int register_sha512_avx (void ) {return 0 ; } -static inline void unregister_sha512_avx (void ) { } -#endif -#if defined(CONFIG_AS_AVX2 ) && defined(CONFIG_AS_AVX ) +#if defined(CONFIG_AS_AVX2 ) asmlinkage void sha512_transform_rorx(struct sha512_state *state,const u8 *data,int blocks);
arch/x86/include/asm/xor_avx.h +0 -9
@@ -11,8 +11,6 @@* Based on Ingo Molnar and Zach Brown's respective MMX and SSE routines */ -#ifdef CONFIG_AS_AVX -#include <linux/compiler.h> #include <asm/fpu/api.h>
@@ -170,11 +168,4 @@ do { \#define AVX_SELECT (FASTEST ) \ (boot_cpu_has(X86_FEATURE_AVX) && boot_cpu_has(X86_FEATURE_OSXSAVE) ? &xor_block_avx : FASTEST) -#else - -#define AVX_XOR_SPEED {} - -#define AVX_SELECT (FASTEST ) (FASTEST) - -#endif #endif
Signature · 1 line
2.17.1
PATCH v2 12/16 crypto: x86 - rework configuration based on Kconfig
In reply to Masahiro YamadaFrom: "Jason A. Donenfeld" <[email protected]>
Now that assembler capabilities are probed inside of Kconfig, we can set up proper Kconfig-based dependencies. We also take this opportunity to reorder the Makefile, so that items are grouped logically by primitive.
---
Changes in v2: None
arch/x86/crypto/Makefile | 152 +++++++++++++++++---------------------- crypto/Kconfig | 8 +-- 2 files changed, 69 insertions(+), 91 deletions(-)
arch/x86/crypto/Makefile +65 -87
@@ -1,122 +1,100 @@ # SPDX-License-Identifier: GPL-2.0 # -# Arch-specific CryptoAPI modules. -# +# x86 crypto algorithms OBJECT_FILES_NON_STANDARD := y -avx2_supported := $(call as-instr,vpgatherdd %ymm0$(comma)(%eax$(comma)%ymm1\ - $(comma)4)$(comma)%ymm2,yes,no) -avx512_supported :=$(call as-instr,vpmovm2b %k1$(comma)%zmm5,yes,no) -sha1_ni_supported :=$(call as-instr,sha1msg1 %xmm0$(comma)%xmm1,yes,no) -sha256_ni_supported :=$(call as-instr,sha256msg1 %xmm0$(comma)%xmm1,yes,no) -adx_supported := $(call as-instr,adox %r10$(comma)%r10,yes,no) - obj-$(CONFIG_CRYPTO_GLUE_HELPER_X86) += glue_helper.o obj-$(CONFIG_CRYPTO_TWOFISH_586) += twofish-i586.o +twofish-i586-y := twofish-i586-asm_32.o twofish_glue.o +obj-$(CONFIG_CRYPTO_TWOFISH_X86_64) += twofish-x86_64.o +twofish-x86_64-y := twofish-x86_64-asm_64.o twofish_glue.o +obj-$(CONFIG_CRYPTO_TWOFISH_X86_64_3WAY) += twofish-x86_64-3way.o +twofish-x86_64-3way-y := twofish-x86_64-asm_64-3way.o twofish_glue_3way.o +obj-$(CONFIG_CRYPTO_TWOFISH_AVX_X86_64) += twofish-avx-x86_64.o +twofish-avx-x86_64-y := twofish-avx-x86_64-asm_64.o twofish_avx_glue.o + obj-$(CONFIG_CRYPTO_SERPENT_SSE2_586) += serpent-sse2-i586.o +serpent-sse2-i586-y := serpent-sse2-i586-asm_32.o serpent_sse2_glue.o +obj-$(CONFIG_CRYPTO_SERPENT_SSE2_X86_64) += serpent-sse2-x86_64.o +serpent-sse2-x86_64-y := serpent-sse2-x86_64-asm_64.o serpent_sse2_glue.o +obj-$(CONFIG_CRYPTO_SERPENT_AVX_X86_64) += serpent-avx-x86_64.o +serpent-avx-x86_64-y := serpent-avx-x86_64-asm_64.o serpent_avx_glue.o +obj-$(CONFIG_CRYPTO_SERPENT_AVX2_X86_64) += serpent-avx2.o +serpent-avx2-y := serpent-avx2-asm_64.o serpent_avx2_glue.o obj-$(CONFIG_CRYPTO_DES3_EDE_X86_64) += des3_ede-x86_64.o +des3_ede-x86_64-y := des3_ede-asm_64.o des3_ede_glue.o + obj-$(CONFIG_CRYPTO_CAMELLIA_X86_64) += camellia-x86_64.o +camellia-x86_64-y := camellia-x86_64-asm_64.o camellia_glue.o +obj-$(CONFIG_CRYPTO_CAMELLIA_AESNI_AVX_X86_64) += camellia-aesni-avx-x86_64.o +camellia-aesni-avx-x86_64-y := camellia-aesni-avx-asm_64.o camellia_aesni_avx_glue.o +obj-$(CONFIG_CRYPTO_CAMELLIA_AESNI_AVX2_X86_64) += camellia-aesni-avx2.o +camellia-aesni-avx2-y := camellia-aesni-avx2-asm_64.o camellia_aesni_avx2_glue.o + obj-$(CONFIG_CRYPTO_BLOWFISH_X86_64) += blowfish-x86_64.o -obj-$(CONFIG_CRYPTO_TWOFISH_X86_64) += twofish-x86_64.o -obj-$(CONFIG_CRYPTO_TWOFISH_X86_64_3WAY) += twofish-x86_64-3way.o +blowfish-x86_64-y := blowfish-x86_64-asm_64.o blowfish_glue.o + +obj-$(CONFIG_CRYPTO_CAST5_AVX_X86_64) += cast5-avx-x86_64.o +cast5-avx-x86_64-y := cast5-avx-x86_64-asm_64.o cast5_avx_glue.o + +obj-$(CONFIG_CRYPTO_CAST6_AVX_X86_64) += cast6-avx-x86_64.o +cast6-avx-x86_64-y := cast6-avx-x86_64-asm_64.o cast6_avx_glue.o + +obj-$(CONFIG_CRYPTO_AEGIS128_AESNI_SSE2) += aegis128-aesni.o +aegis128-aesni-y := aegis128-aesni-asm.o aegis128-aesni-glue.o + obj-$(CONFIG_CRYPTO_CHACHA20_X86_64) += chacha-x86_64.o -obj-$(CONFIG_CRYPTO_SERPENT_SSE2_X86_64) += serpent-sse2-x86_64.o +chacha-x86_64-y := chacha-ssse3-x86_64.o chacha_glue.o +chacha-x86_64-$(CONFIG_AS_AVX2) += chacha-avx2-x86_64.o +chacha-x86_64-$(CONFIG_AS_AVX512) += chacha-avx512vl-x86_64.o + obj-$(CONFIG_CRYPTO_AES_NI_INTEL) += aesni-intel.o -obj-$(CONFIG_CRYPTO_GHASH_CLMUL_NI_INTEL) += ghash-clmulni-intel.o +aesni-intel-y := aesni-intel_asm.o aesni-intel_glue.o +aesni-intel-$(CONFIG_64BIT) += aesni-intel_avx-x86_64.o aes_ctrby8_avx-x86_64.o -obj-$(CONFIG_CRYPTO_CRC32C_INTEL) += crc32c-intel.o obj-$(CONFIG_CRYPTO_SHA1_SSSE3) += sha1-ssse3.o -obj-$(CONFIG_CRYPTO_CRC32_PCLMUL) += crc32-pclmul.o -obj-$(CONFIG_CRYPTO_SHA256_SSSE3) += sha256-ssse3.o -obj-$(CONFIG_CRYPTO_SHA512_SSSE3) += sha512-ssse3.o -obj-$(CONFIG_CRYPTO_CRCT10DIF_PCLMUL) += crct10dif-pclmul.o -obj-$(CONFIG_CRYPTO_POLY1305_X86_64) += poly1305-x86_64.o - -obj-$(CONFIG_CRYPTO_AEGIS128_AESNI_SSE2) += aegis128-aesni.o +sha1-ssse3-y := sha1_ssse3_asm.o sha1_ssse3_glue.o +sha1-ssse3-$(CONFIG_AS_AVX2) += sha1_avx2_x86_64_asm.o +sha1-ssse3-$(CONFIG_AS_SHA1_NI) += sha1_ni_asm.o -obj-$(CONFIG_CRYPTO_NHPOLY1305_SSE2) += nhpoly1305-sse2.o -obj-$(CONFIG_CRYPTO_NHPOLY1305_AVX2) += nhpoly1305-avx2.o +obj-$(CONFIG_CRYPTO_SHA256_SSSE3) += sha256-ssse3.o +sha256-ssse3-y := sha256-ssse3-asm.o sha256-avx-asm.o sha256-avx2-asm.o sha256_ssse3_glue.o +sha256-ssse3-$(CONFIG_AS_SHA256_NI) += sha256_ni_asm.o -# These modules require the assembler to support ADX. -ifeq ($(adx_supported),yes) - obj-$(CONFIG_CRYPTO_CURVE25519_X86) += curve25519-x86_64.o -endif +obj-$(CONFIG_CRYPTO_SHA512_SSSE3) += sha512-ssse3.o +sha512-ssse3-y := sha512-ssse3-asm.o sha512-avx-asm.o sha512-avx2-asm.o sha512_ssse3_glue.o -# These modules require assembler to support AVX. -obj-$(CONFIG_CRYPTO_CAMELLIA_AESNI_AVX_X86_64) += camellia-aesni-avx-x86_64.o -obj-$(CONFIG_CRYPTO_CAST5_AVX_X86_64) += cast5-avx-x86_64.o -obj-$(CONFIG_CRYPTO_CAST6_AVX_X86_64) += cast6-avx-x86_64.o -obj-$(CONFIG_CRYPTO_TWOFISH_AVX_X86_64) += twofish-avx-x86_64.o -obj-$(CONFIG_CRYPTO_SERPENT_AVX_X86_64) += serpent-avx-x86_64.o obj-$(CONFIG_CRYPTO_BLAKE2S_X86) += blake2s-x86_64.o +blake2s-x86_64-y := blake2s-core.o blake2s-glue.o -# These modules require assembler to support AVX2. -ifeq ($(avx2_supported),yes) - obj-$(CONFIG_CRYPTO_CAMELLIA_AESNI_AVX2_X86_64) += camellia-aesni-avx2.o - obj-$(CONFIG_CRYPTO_SERPENT_AVX2_X86_64) += serpent-avx2.o -endif +obj-$(CONFIG_CRYPTO_GHASH_CLMUL_NI_INTEL) += ghash-clmulni-intel.o +ghash-clmulni-intel-y := ghash-clmulni-intel_asm.o ghash-clmulni-intel_glue.o -twofish-i586-y := twofish-i586-asm_32.o twofish_glue.o -serpent-sse2-i586-y := serpent-sse2-i586-asm_32.o serpent_sse2_glue.o +obj-$(CONFIG_CRYPTO_CRC32C_INTEL) += crc32c-intel.o +crc32c-intel-y := crc32c-intel_glue.o +crc32c-intel-$(CONFIG_64BIT) += crc32c-pcl-intel-asm_64.o -des3_ede-x86_64-y := des3_ede-asm_64.o des3_ede_glue.o -camellia-x86_64-y := camellia-x86_64-asm_64.o camellia_glue.o -blowfish-x86_64-y := blowfish-x86_64-asm_64.o blowfish_glue.o -twofish-x86_64-y := twofish-x86_64-asm_64.o twofish_glue.o -twofish-x86_64-3way-y := twofish-x86_64-asm_64-3way.o twofish_glue_3way.o -chacha-x86_64-y := chacha-ssse3-x86_64.o chacha_glue.o -serpent-sse2-x86_64-y := serpent-sse2-x86_64-asm_64.o serpent_sse2_glue.o +obj-$(CONFIG_CRYPTO_CRC32_PCLMUL) += crc32-pclmul.o +crc32-pclmul-y := crc32-pclmul_asm.o crc32-pclmul_glue.o -aegis128-aesni-y := aegis128-aesni-asm.o aegis128-aesni-glue.o +obj-$(CONFIG_CRYPTO_CRCT10DIF_PCLMUL) += crct10dif-pclmul.o +crct10dif-pclmul-y := crct10dif-pcl-asm_64.o crct10dif-pclmul_glue.o -nhpoly1305-sse2-y := nh-sse2-x86_64.o nhpoly1305-sse2-glue.o -blake2s-x86_64-y := blake2s-core.o blake2s-glue.o +obj-$(CONFIG_CRYPTO_POLY1305_X86_64) += poly1305-x86_64.o poly1305-x86_64-y := poly1305-x86_64-cryptogams.o poly1305_glue.o ifneq ($(CONFIG_CRYPTO_POLY1305_X86_64),) targets += poly1305-x86_64-cryptogams.S endif -camellia-aesni-avx-x86_64-y := camellia-aesni-avx-asm_64.o \ - camellia_aesni_avx_glue.o -cast5-avx-x86_64-y := cast5-avx-x86_64-asm_64.o cast5_avx_glue.o -cast6-avx-x86_64-y := cast6-avx-x86_64-asm_64.o cast6_avx_glue.o -twofish-avx-x86_64-y := twofish-avx-x86_64-asm_64.o twofish_avx_glue.o -serpent-avx-x86_64-y := serpent-avx-x86_64-asm_64.o serpent_avx_glue.o - -ifeq ($(avx2_supported),yes) - camellia-aesni-avx2-y := camellia-aesni-avx2-asm_64.o camellia_aesni_avx2_glue.o - chacha-x86_64-y += chacha-avx2-x86_64.o - serpent-avx2-y := serpent-avx2-asm_64.o serpent_avx2_glue.o - - nhpoly1305-avx2-y := nh-avx2-x86_64.o nhpoly1305-avx2-glue.o -endif - -ifeq ($(avx512_supported),yes) - chacha-x86_64-y += chacha-avx512vl-x86_64.o -endif +obj-$(CONFIG_CRYPTO_NHPOLY1305_SSE2) += nhpoly1305-sse2.o +nhpoly1305-sse2-y := nh-sse2-x86_64.o nhpoly1305-sse2-glue.o +obj-$(CONFIG_CRYPTO_NHPOLY1305_AVX2) += nhpoly1305-avx2.o +nhpoly1305-avx2-y := nh-avx2-x86_64.o nhpoly1305-avx2-glue.o -aesni-intel-y := aesni-intel_asm.o aesni-intel_glue.o -aesni-intel-$(CONFIG_64BIT) += aesni-intel_avx-x86_64.o aes_ctrby8_avx-x86_64.o -ghash-clmulni-intel-y := ghash-clmulni-intel_asm.o ghash-clmulni-intel_glue.o -sha1-ssse3-y := sha1_ssse3_asm.o sha1_ssse3_glue.o -ifeq ($(avx2_supported),yes) -sha1-ssse3-y += sha1_avx2_x86_64_asm.o -endif -ifeq ($(sha1_ni_supported),yes) -sha1-ssse3-y += sha1_ni_asm.o -endif -crc32c-intel-y := crc32c-intel_glue.o -crc32c-intel-$(CONFIG_64BIT) += crc32c-pcl-intel-asm_64.o -crc32-pclmul-y := crc32-pclmul_asm.o crc32-pclmul_glue.o -sha256-ssse3-y := sha256-ssse3-asm.o sha256-avx-asm.o sha256-avx2-asm.o sha256_ssse3_glue.o -ifeq ($(sha256_ni_supported),yes) -sha256-ssse3-y += sha256_ni_asm.o -endif -sha512-ssse3-y := sha512-ssse3-asm.o sha512-avx-asm.o sha512-avx2-asm.o sha512_ssse3_glue.o -crct10dif-pclmul-y := crct10dif-pcl-asm_64.o crct10dif-pclmul_glue.o +obj-$(CONFIG_CRYPTO_CURVE25519_X86) += curve25519-x86_64.o quiet_cmd_perlasm = PERLASM $@ cmd_perlasm = $(PERL) $< > $@
crypto/Kconfig +4 -4
@@ -267,7 +267,7 @@ config CRYPTO_CURVE25519 config CRYPTO_CURVE25519_X86 tristate "x86_64 accelerated Curve25519 scalar multiplication library" - depends on X86 && 64BIT + depends on X86 && 64BIT && AS_ADX select CRYPTO_LIB_CURVE25519_GENERIC select CRYPTO_ARCH_HAVE_LIB_CURVE25519
@@ -465,7 +465,7 @@ config CRYPTO_NHPOLY1305_SSE2 config CRYPTO_NHPOLY1305_AVX2 tristate "NHPoly1305 hash function (x86_64 AVX2 implementation)" - depends on X86 && 64BIT + depends on X86 && 64BIT && AS_AVX2 select CRYPTO_NHPOLY1305 help AVX2 optimized implementation of the hash function used by the
@@ -1303,7 +1303,7 @@ config CRYPTO_CAMELLIA_AESNI_AVX_X86_64 config CRYPTO_CAMELLIA_AESNI_AVX2_X86_64 tristate "Camellia cipher algorithm (x86_64/AES-NI/AVX2)" - depends on X86 && 64BIT + depends on X86 && 64BIT && AS_AVX2 depends on CRYPTO select CRYPTO_CAMELLIA_AESNI_AVX_X86_64 help
@@ -1573,7 +1573,7 @@ config CRYPTO_SERPENT_AVX_X86_64 config CRYPTO_SERPENT_AVX2_X86_64 tristate "Serpent cipher algorithm (x86_64/AVX2)" - depends on X86 && 64BIT + depends on X86 && 64BIT && AS_AVX2 select CRYPTO_SERPENT_AVX_X86_64 help Serpent cipher algorithm, by Anderson, Biham & Knudsen.
Signature · 1 line
2.17.1
PATCH v2 15/16 x86: update AS_* macros to binutils >=2.23, supporting ADX and AVX2
In reply to Masahiro YamadaFrom: "Jason A. Donenfeld" <[email protected]>
Now that the kernel specifies binutils 2.23 as the minimum version, we can remove ifdefs for AVX2 and ADX throughout.
---
Changes in v2: None
arch/x86/Kconfig.assembler | 10 ---------- arch/x86/crypto/Makefile | 6 ++---- arch/x86/crypto/aesni-intel_avx-x86_64.S | 3 --- arch/x86/crypto/aesni-intel_glue.c | 7 ------- arch/x86/crypto/chacha_glue.c | 6 ++---- arch/x86/crypto/poly1305-x86_64-cryptogams.pl | 8 -------- arch/x86/crypto/poly1305_glue.c | 5 ++--- arch/x86/crypto/sha1_ssse3_glue.c | 6 ------ arch/x86/crypto/sha256-avx2-asm.S | 3 --- arch/x86/crypto/sha256_ssse3_glue.c | 6 ------ arch/x86/crypto/sha512-avx2-asm.S | 3 --- arch/x86/crypto/sha512_ssse3_glue.c | 5 ----- crypto/Kconfig | 8 ++++---- lib/raid6/algos.c | 6 ------ lib/raid6/avx2.c | 4 ---- lib/raid6/recov_avx2.c | 6 ------ lib/raid6/test/Makefile | 3 --- 17 files changed, 10 insertions(+), 85 deletions(-)
arch/x86/Kconfig.assembler +0 -10
@@ -1,11 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 # Copyright (C) 2020 Jason A. Donenfeld <[email protected]>. All Rights Reserved. -config AS_AVX2 - def_bool $(as-instr,vpbroadcastb %xmm0$(comma)%ymm1) - help - Supported by binutils >= 2.22 and LLVM integrated assembler - config AS_AVX512 def_bool $(as-instr,vpmovm2b %k1$(comma)%zmm5) help
@@ -20,8 +15,3 @@ config AS_SHA256_NI def_bool $(as-instr,sha256msg1 %xmm0$(comma)%xmm1) help Supported by binutils >= 2.24 and LLVM integrated assembler - -config AS_ADX - def_bool $(as-instr,adox %eax$(comma)%eax) - help - Supported by binutils >= 2.23 and LLVM integrated assembler
arch/x86/crypto/Makefile +2 -4
@@ -47,8 +47,7 @@ obj-$(CONFIG_CRYPTO_AEGIS128_AESNI_SSE2) += aegis128-aesni.o aegis128-aesni-y := aegis128-aesni-asm.o aegis128-aesni-glue.o obj-$(CONFIG_CRYPTO_CHACHA20_X86_64) += chacha-x86_64.o -chacha-x86_64-y := chacha-ssse3-x86_64.o chacha_glue.o -chacha-x86_64-$(CONFIG_AS_AVX2) += chacha-avx2-x86_64.o +chacha-x86_64-y := chacha-avx2-x86_64.o chacha-ssse3-x86_64.o chacha_glue.o chacha-x86_64-$(CONFIG_AS_AVX512) += chacha-avx512vl-x86_64.o obj-$(CONFIG_CRYPTO_AES_NI_INTEL) += aesni-intel.o
@@ -56,8 +55,7 @@ aesni-intel-y := aesni-intel_asm.o aesni-intel_glue.o aesni-intel-$(CONFIG_64BIT) += aesni-intel_avx-x86_64.o aes_ctrby8_avx-x86_64.o obj-$(CONFIG_CRYPTO_SHA1_SSSE3) += sha1-ssse3.o -sha1-ssse3-y := sha1_ssse3_asm.o sha1_ssse3_glue.o -sha1-ssse3-$(CONFIG_AS_AVX2) += sha1_avx2_x86_64_asm.o +sha1-ssse3-y := sha1_avx2_x86_64_asm.o sha1_ssse3_asm.o sha1_ssse3_glue.o sha1-ssse3-$(CONFIG_AS_SHA1_NI) += sha1_ni_asm.o obj-$(CONFIG_CRYPTO_SHA256_SSSE3) += sha256-ssse3.o
arch/x86/crypto/aesni-intel_avx-x86_64.S +0 -3
@@ -1868,7 +1868,6 @@ key_256_finalize: ret SYM_FUNC_END(aesni_gcm_finalize_avx_gen2) -#ifdef CONFIG_AS_AVX2 ############################################################################### # GHASH_MUL MACRO to implement: Data*HashKey mod (128,127,126,121,0) # Input: A and B (128-bits each, bit-reflected)
@@ -2836,5 +2835,3 @@ key_256_finalize4: FUNC_RESTORE ret SYM_FUNC_END(aesni_gcm_finalize_avx_gen4) - -#endif /* CONFIG_AS_AVX2 */
arch/x86/crypto/aesni-intel_glue.c +0 -7
@@ -233,7 +233,6 @@ static const struct aesni_gcm_tfm_s aesni_gcm_tfm_avx_gen2 = { .finalize = &aesni_gcm_finalize_avx_gen2, }; -#ifdef CONFIG_AS_AVX2 /* * asmlinkage void aesni_gcm_init_avx_gen4() * gcm_data *my_ctx_data, context data
@@ -276,8 +275,6 @@ static const struct aesni_gcm_tfm_s aesni_gcm_tfm_avx_gen4 = { .finalize = &aesni_gcm_finalize_avx_gen4, }; -#endif -static inline struct aesni_rfc4106_gcm_ctx *aesni_rfc4106_gcm_ctx_get (struct crypto_aead *tfm) {
@@ -706,10 +703,8 @@ static int gcmaes_crypt_by_sg(bool enc, struct aead_request *req,if (!enc) left -= auth_tag_len; -#ifdef CONFIG_AS_AVX2 if (left <AVX_GEN4_OPTSIZE && gcm_tfm == &aesni_gcm_tfm_avx_gen4) gcm_tfm = &aesni_gcm_tfm_avx_gen2; -#endif if (left <AVX_GEN2_OPTSIZE && gcm_tfm == &aesni_gcm_tfm_avx_gen2) gcm_tfm = &aesni_gcm_tfm_sse;
@@ -1069,12 +1064,10 @@ static int __init aesni_init(void)if (!x86_match_cpu (aesni_cpu_id))return -ENODEV ;#ifdef CONFIG_X86_64 -#ifdef CONFIG_AS_AVX2 if (boot_cpu_has (X86_FEATURE_AVX2 )) {pr_info ("AVX2 version of gcm_enc/dec engaged.\n" ); aesni_gcm_tfm = &aesni_gcm_tfm_avx_gen4; }else -#endif if (boot_cpu_has (X86_FEATURE_AVX )) {pr_info ("AVX version of gcm_enc/dec engaged.\n" ); aesni_gcm_tfm = &aesni_gcm_tfm_avx_gen2;
arch/x86/crypto/chacha_glue.c +2 -4
@@ -79,8 +79,7 @@ static void chacha_dosimd(u32 *state, u8 *dst, const u8 *src, } } -if (IS_ENABLED (CONFIG_AS_AVX2 ) && -static_branch_likely (&chacha_use_avx2)) { +if (static_branch_likely (&chacha_use_avx2)) {while (bytes >=CHACHA_BLOCK_SIZE *8 ) {chacha_8block_xor_avx2 (state, dst, src, bytes, nrounds); bytes -=CHACHA_BLOCK_SIZE *8 ;
@@ -288,8 +287,7 @@ static int __init chacha_simd_mod_init(void)static_branch_enable (&chacha_use_simd); -if (IS_ENABLED (CONFIG_AS_AVX2 ) && -boot_cpu_has (X86_FEATURE_AVX ) && +if (boot_cpu_has (X86_FEATURE_AVX ) &&boot_cpu_has (X86_FEATURE_AVX2 ) &&cpu_has_xfeatures (XFEATURE_MASK_SSE |XFEATURE_MASK_YMM ,NULL )) {static_branch_enable (&chacha_use_avx2);
arch/x86/crypto/poly1305-x86_64-cryptogams.pl +0 -8
@@ -1514,10 +1514,6 @@ ___ if ($avx>1) { -if ($kernel) { - $code .= "#ifdef CONFIG_AS_AVX2\n"; -} - my ($H0,$H1,$H2,$H3,$H4, $MASK, $T4,$T0,$T1,$T2,$T3, $D0,$D1,$D2,$D3,$D4) = map("%ymm$_",(0..15)); my $S4=$MASK;
@@ -2808,10 +2804,6 @@ ___ poly1305_blocks_avxN(0); &end_function("poly1305_blocks_avx2"); -if($kernel) { - $code .= "#endif\n"; -} - ####################################################################### if ($avx>2) { # On entry we have input length divisible by 64. But since inner loop
arch/x86/crypto/poly1305_glue.c +2 -3
@@ -108,7 +108,7 @@ static void poly1305_simd_blocks(void *ctx, const u8 *inp, size_t len,kernel_fpu_begin ();if (IS_ENABLED (CONFIG_AS_AVX512 ) &&static_branch_likely (&poly1305_use_avx512))poly1305_blocks_avx512 (ctx, inp, bytes, padbit); -else if (IS_ENABLED (CONFIG_AS_AVX2 ) &&static_branch_likely (&poly1305_use_avx2)) +else if (static_branch_likely (&poly1305_use_avx2))poly1305_blocks_avx2 (ctx, inp, bytes, padbit);else poly1305_blocks_avx (ctx, inp, bytes, padbit);
@@ -264,8 +264,7 @@ static int __init poly1305_simd_mod_init(void)if (boot_cpu_has (X86_FEATURE_AVX ) &&cpu_has_xfeatures (XFEATURE_MASK_SSE |XFEATURE_MASK_YMM ,NULL ))static_branch_enable (&poly1305_use_avx); -if (IS_ENABLED (CONFIG_AS_AVX2 ) &&boot_cpu_has (X86_FEATURE_AVX ) && -boot_cpu_has (X86_FEATURE_AVX2 ) && +if (boot_cpu_has (X86_FEATURE_AVX ) &&boot_cpu_has (X86_FEATURE_AVX2 ) &&cpu_has_xfeatures (XFEATURE_MASK_SSE |XFEATURE_MASK_YMM ,NULL ))static_branch_enable (&poly1305_use_avx2); if (IS_ENABLED(CONFIG_AS_AVX512) && boot_cpu_has(X86_FEATURE_AVX) &&
arch/x86/crypto/sha1_ssse3_glue.c +0 -6
@@ -174,7 +174,6 @@ static void unregister_sha1_avx(void)crypto_unregister_shash (&sha1_avx_alg); } -#if defined(CONFIG_AS_AVX2 )#define SHA1_AVX2_BLOCK_OPTSIZE 4/* optimal 4*64 bytes of SHA1 blocks */ asmlinkage void sha1_transform_avx2(struct sha1_state *state,
@@ -246,11 +245,6 @@ static void unregister_sha1_avx2(void)crypto_unregister_shash (&sha1_avx2_alg); } -#else -static inline int register_sha1_avx2 (void ) {return 0 ; } -static inline void unregister_sha1_avx2 (void ) { } -#endif -#ifdef CONFIG_AS_SHA1_NI asmlinkage void sha1_ni_transform(struct sha1_state *digest, const u8 *data,int rounds);
arch/x86/crypto/sha256-avx2-asm.S +0 -3
@@ -48,7 +48,6 @@ # This code schedules 2 blocks at a time, with 4 lanes per block ######################################################################## -#ifdef CONFIG_AS_AVX2 #include <linux/linkage.h> ## assume buffers not aligned
@@ -767,5 +766,3 @@ _SHUF_00BA: .align 32 _SHUF_DC00: .octa 0x0b0a090803020100FFFFFFFFFFFFFFFF,0x0b0a090803020100FFFFFFFFFFFFFFFF - -#endif
arch/x86/crypto/sha256_ssse3_glue.c +0 -6
@@ -220,7 +220,6 @@ static void unregister_sha256_avx(void) ARRAY_SIZE(sha256_avx_algs)); } -#if defined(CONFIG_AS_AVX2 ) asmlinkage void sha256_transform_rorx(struct sha256_state *state,const u8 *data,int blocks);
@@ -295,11 +294,6 @@ static void unregister_sha256_avx2(void) ARRAY_SIZE(sha256_avx2_algs)); } -#else -static inline int register_sha256_avx2 (void ) {return 0 ; } -static inline void unregister_sha256_avx2 (void ) { } -#endif -#ifdef CONFIG_AS_SHA256_NI asmlinkage void sha256_ni_transform(struct sha256_state *digest,const u8 *data,int rounds);
arch/x86/crypto/sha512-avx2-asm.S +0 -3
@@ -49,7 +49,6 @@ # This code schedules 1 blocks at a time, with 4 lanes per block ######################################################################## -#ifdef CONFIG_AS_AVX2 #include <linux/linkage.h> .text
@@ -749,5 +748,3 @@ PSHUFFLE_BYTE_FLIP_MASK: MASK_YMM_LO: .octa 0x00000000000000000000000000000000 .octa 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF - -#endif
arch/x86/crypto/sha512_ssse3_glue.c +0 -5
@@ -218,7 +218,6 @@ static void unregister_sha512_avx(void) ARRAY_SIZE(sha512_avx_algs)); } -#if defined(CONFIG_AS_AVX2 ) asmlinkage void sha512_transform_rorx(struct sha512_state *state,const u8 *data,int blocks);
@@ -293,10 +292,6 @@ static void unregister_sha512_avx2(void)crypto_unregister_shashes (sha512_avx2_algs,ARRAY_SIZE (sha512_avx2_algs)); } -#else -static inline int register_sha512_avx2 (void ) {return 0 ; } -static inline void unregister_sha512_avx2 (void ) { } -#endif static int __init sha512_ssse3_mod_init (void ) {
crypto/Kconfig +4 -4
@@ -267,7 +267,7 @@ config CRYPTO_CURVE25519 config CRYPTO_CURVE25519_X86 tristate "x86_64 accelerated Curve25519 scalar multiplication library" - depends on X86 && 64BIT && AS_ADX + depends on X86 && 64BIT select CRYPTO_LIB_CURVE25519_GENERIC select CRYPTO_ARCH_HAVE_LIB_CURVE25519
@@ -465,7 +465,7 @@ config CRYPTO_NHPOLY1305_SSE2 config CRYPTO_NHPOLY1305_AVX2 tristate "NHPoly1305 hash function (x86_64 AVX2 implementation)" - depends on X86 && 64BIT && AS_AVX2 + depends on X86 && 64BIT select CRYPTO_NHPOLY1305 help AVX2 optimized implementation of the hash function used by the
@@ -1303,7 +1303,7 @@ config CRYPTO_CAMELLIA_AESNI_AVX_X86_64 config CRYPTO_CAMELLIA_AESNI_AVX2_X86_64 tristate "Camellia cipher algorithm (x86_64/AES-NI/AVX2)" - depends on X86 && 64BIT && AS_AVX2 + depends on X86 && 64BIT depends on CRYPTO select CRYPTO_CAMELLIA_AESNI_AVX_X86_64 help
@@ -1573,7 +1573,7 @@ config CRYPTO_SERPENT_AVX_X86_64 config CRYPTO_SERPENT_AVX2_X86_64 tristate "Serpent cipher algorithm (x86_64/AVX2)" - depends on X86 && 64BIT && AS_AVX2 + depends on X86 && 64BIT select CRYPTO_SERPENT_AVX_X86_64 help Serpent cipher algorithm, by Anderson, Biham & Knudsen.
lib/raid6/algos.c +0 -6
@@ -34,10 +34,8 @@ const struct raid6_calls * const raid6_algos[] = { &raid6_avx512x2, &raid6_avx512x1, #endif -#ifdef CONFIG_AS_AVX2 &raid6_avx2x2, &raid6_avx2x1, -#endif &raid6_sse2x2, &raid6_sse2x1, &raid6_sse1x2,
@@ -51,11 +49,9 @@ const struct raid6_calls * const raid6_algos[] = { &raid6_avx512x2, &raid6_avx512x1, #endif -#ifdef CONFIG_AS_AVX2 &raid6_avx2x4, &raid6_avx2x2, &raid6_avx2x1, -#endif &raid6_sse2x4, &raid6_sse2x2, &raid6_sse2x1,
@@ -101,9 +97,7 @@ const struct raid6_recov_calls *const raid6_recov_algos[] = {#ifdef CONFIG_AS_AVX512 &raid6_recov_avx512, #endif -#ifdef CONFIG_AS_AVX2 &raid6_recov_avx2, -#endif &raid6_recov_ssse3, #endif #ifdef CONFIG_S390
lib/raid6/avx2.c +0 -4
@@ -13,8 +13,6 @@* */ -#ifdef CONFIG_AS_AVX2 -#include <linux/raid/pq.h> #include "x86.h"
@@ -470,5 +468,3 @@ const struct raid6_calls raid6_avx2x4 = {1 /* Has cache hints */ };#endif - -#endif /* CONFIG_AS_AVX2 */
lib/raid6/recov_avx2.c +0 -6
@@ -4,8 +4,6 @@* Author: Jim Kukunas <[email protected]> */ -#ifdef CONFIG_AS_AVX2 -#include <linux/raid/pq.h> #include "x86.h"
@@ -313,7 +311,3 @@ const struct raid6_recov_calls raid6_recov_avx2 = {#endif .priority = 2, }; - -#else -#warning "your version of binutils lacks AVX2 support" -#endif
lib/raid6/test/Makefile +0 -3
@@ -35,9 +35,6 @@ endif ifeq ($(IS_X86),yes) OBJS += mmx.o sse1.o sse2.o avx2.o recov_ssse3.o recov_avx2.o avx512.o recov_avx512.o CFLAGS += -DCONFIG_X86 - CFLAGS += $(shell echo "vpbroadcastb %xmm0, %ymm1" | \ - gcc -c -x assembler - >/dev/null 2>&1 && \ - rm ./-.o && echo -DCONFIG_AS_AVX2=1) CFLAGS += $(shell echo "vpmovm2b %k1, %zmm5" | \ gcc -c -x assembler - >/dev/null 2>&1 && \ rm ./-.o && echo -DCONFIG_AS_AVX512=1)
Signature · 1 line
2.17.1
Very little has changed from last time, and this whole series still looks good to me. I think I already ack'd most packages, but in case it helps:
Since this touches a lot of stuff, it might be best to get it in as early as possible during the merge window, as I imagine new code being added is going to want to be touching those makefiles too.
Jani Nikula
maintainerRe: [PATCH v2 09/16] drm/i915: remove always-defined CONFIG_AS_MOVNTDQA
In reply to Masahiro YamadaQuoted 7 lines · Masahiro Yamada
CONFIG_AS_MOVNTDQAwas introduced by commit0b1de5d58e19("drm/i915:
Use SSE4.1 movntdqa to accelerate reads from WC memory").We raise the minimal supported binutils version from time to time. The last bump was commit
1fb12b35e5ff("kbuild: Raise the minimum required binutils version to 2.21").
Quoted 7 lines
I confirmed the code in $(call as-instr,...) can be assembled by the binutils 2.21 assembler and also by LLVM integrated assembler.
Remove
CONFIG_AS_MOVNTDQA, which is always defined.Signed-off-by Masahiro Yamada <[email protected]>Reviewed-by Nick Desaulniers <[email protected]>
Ack for merging this via whichever tree you see fit; please let me know if you want me to pick this up via drm-intel.
BR,
Jani.
Quoted 45 lines
---
Changes in v2: None
drivers/gpu/drm/i915/Makefile | 3 --- drivers/gpu/drm/i915/i915_memcpy.c | 5 ----- 2 files changed, 8 deletions(-)drivers/gpu/drm/i915/Makefile +0
-3@@ -28,9 +28,6 @@ subdir-ccflags-$(CONFIG_DRM_I915_WERROR) += -Werror CFLAGS_i915_pci.o = $(call cc-disable-warning, override-init) CFLAGS_display/intel_fbdev.o = $(call cc-disable-warning, override-init) -subdir-ccflags-y += \ - $(call as-instr,movntdqa (%eax)$(comma)%xmm0,-DCONFIG_AS_MOVNTDQA) - subdir-ccflags-y += -I$(srctree)/$(src) # Please keep these build lists sorted!drivers/gpu/drm/i915/i915_memcpy.c +0
-5@@ -35,7 +35,6 @@static DEFINE_STATIC_KEY_FALSE (has_movntdqa); -#ifdef CONFIG_AS_MOVNTDQA static void __memcpy_ntdqa (void *dst,const void *src,unsigned long len) {kernel_fpu_begin ();@@ -93,10 +92,6 @@ static void __memcpy_ntdqu(void *dst, const void *src, unsigned long len)kernel_fpu_end (); } -#else -static void __memcpy_ntdqa (void *dst,const void *src,unsigned long len) {} -static void __memcpy_ntdqu (void *dst,const void *src,unsigned long len) {} -#endif /** * i915_memcpy_from_wc: perform an accelerated *aligned* read from WC
Signature · 1 line
Jani Nikula, Intel Open Source Graphics Center
Ingo Molnar
Acked-byQuoted 5 lines · Jason A. Donenfeld
Very little has changed from last time, and this whole series still looks good to me. I think I already ack'd most packages, but in case it helps:
Reviewed-by Jason A. Donenfeld <[email protected]>
Quoted 3 lines
Since this touches a lot of stuff, it might be best to get it in as early as possible during the merge window, as I imagine new code being added is going to want to be touching those makefiles too.
I'd argue the opposite: please merge this later in the merge window, to not disrupt the vast body of other stuff that has already been lined up and has been tested, and to give time for these new bits to get tested some more.
Also, please get it into -next ASAP, today would be ideal for test coverage ...
Thanks,
Ingo
Re: [PATCH v2 10/16] x86: probe assembler capabilities via kconfig instead of makefile
In reply to Masahiro YamadaQuoted 10 lines · Masahiro Yamada
From: "Jason A. Donenfeld" <[email protected]>
Doing this probing inside of the Makefiles means we have a maze of ifdefs inside the source code and child Makefiles that need to make proper decisions on this too. Instead, we do it at
Kconfigtime, like many other compiler and assembler options, which allows us to set up the dependencies normally for full compilation units. In the process, the ADX test changes to use %eax instead of %r10 so that it's valid in both 32-bit and 64-bit mode.
Does KConfig generate -D<foo> flags for KBUILD_CFLAGS and KBUILD_AFLAGS? Looks like lib/raid6/test/Makefile also generates some of these?
Quoted 70 lines
Signed-off-by Jason A. Donenfeld <[email protected]>Signed-off-by Masahiro Yamada <[email protected]>---
Changes in v2: None
arch/x86/Kconfig | 2 ++ arch/x86/Kconfig.assembler | 17 +++++++++++++++++ arch/x86/Makefile | 10 ---------- 3 files changed, 19 insertions(+), 10 deletions(-) create mode 100644 arch/x86/Kconfig.assemblerarch/x86/Kconfig +2
-0@@ -2935,3 +2935,5 @@ config HAVE_ATOMIC_IOMAP source "drivers/firmware/Kconfig" source "arch/x86/kvm/Kconfig" + +source "arch/x86/Kconfig.assembler"arch/x86/Kconfig.assembler new file +17
-0@@ -0,0 +1,17 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2020 Jason A. Donenfeld <[email protected]>. All Rights Reserved. + +config AS_AVX2 + def_bool $(as-instr,vpbroadcastb %xmm0$(comma)%ymm1) + +config AS_AVX512 + def_bool $(as-instr,vpmovm2b %k1$(comma)%zmm5) + +config AS_SHA1_NI + def_bool $(as-instr,sha1msg1 %xmm0$(comma)%xmm1) + +config AS_SHA256_NI + def_bool $(as-instr,sha256msg1 %xmm0$(comma)%xmm1) + +config AS_ADX + def_bool $(as-instr,adox %eax$(comma)%eax)arch/x86/Makefile +0
-11@@ -177,16 +177,6 @@ ifeq ($(ACCUMULATE_OUTGOING_ARGS), 1) KBUILD_CFLAGS += $(call cc-option,-maccumulate-outgoing-args,) endif -# does binutils support specific instructions? -avx2_instr :=$(call as-instr,vpbroadcastb %xmm0$(comma)%ymm1,-DCONFIG_AS_AVX2=1) -avx512_instr :=$(call as-instr,vpmovm2b %k1$(comma)%zmm5,-DCONFIG_AS_AVX512=1) -sha1_ni_instr :=$(call as-instr,sha1msg1 %xmm0$(comma)%xmm1,-DCONFIG_AS_SHA1_NI=1) -sha256_ni_instr :=$(call as-instr,sha256msg1 %xmm0$(comma)%xmm1,-DCONFIG_AS_SHA256_NI=1) -adx_instr := $(call as-instr,adox %r10$(comma)%r10,-DCONFIG_AS_ADX=1) - -KBUILD_AFLAGS += $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) -KBUILD_CFLAGS += $(avx2_instr) $(avx512_instr) $(sha1_ni_instr) $(sha256_ni_instr) $(adx_instr) - KBUILD_LDFLAGS := -m elf_$(UTS_MACHINE) # --2.17.1
Signature · 2 lines
Thanks, ~Nick Desaulniers
Re: [PATCH v2 11/16] x86: add comments about the binutils version to support code in as-instr
In reply to Masahiro YamadaQuoted 8 lines · Masahiro Yamada
We raise the minimal supported binutils version from time to time. The last bump was commit
1fb12b35e5ff("kbuild: Raise the minimum required binutils version to 2.21").We need to keep these as-instr checks because binutils 2.21 does not support them.
Quoted 8 lines
I hope this will be a good hint which one can be dropped when we bump the minimal binutils version next time.
As for the Clang/LLVM builds, we require very new LLVM version, so the LLVM integrated assembler supports all of them.
Signed-off-by Masahiro Yamada <[email protected]>
Quoted 41 lines
---
Changes in v2:
- Change the patch order and rebasearch/x86/Kconfig.assembler | 10 ++++++++++ 1 file changed, 10 insertions(+)arch/x86/Kconfig.assembler +10
-1@@ -3,15 +3,25 @@ config AS_AVX2 def_bool $(as-instr,vpbroadcastb %xmm0$(comma)%ymm1) + help + Supported by binutils >= 2.22 and LLVM integrated assembler config AS_AVX512 def_bool $(as-instr,vpmovm2b %k1$(comma)%zmm5) + help + Supported by binutils >= 2.25 and LLVM integrated assembler config AS_SHA1_NI def_bool $(as-instr,sha1msg1 %xmm0$(comma)%xmm1) + help + Supported by binutils >= 2.24 and LLVM integrated assembler config AS_SHA256_NI def_bool $(as-instr,sha256msg1 %xmm0$(comma)%xmm1) + help + Supported by binutils >= 2.24 and LLVM integrated assembler config AS_ADX def_bool $(as-instr,adox %eax$(comma)%eax) + help + Supported by binutils >= 2.23 and LLVM integrated assembler --2.17.1
Signature · 2 lines
Thanks, ~Nick Desaulniers
Re: [PATCH v2 15/16] x86: update AS_* macros to binutils >=2.23, supporting ADX and AVX2
In reply to Masahiro YamadaQuoted 21 lines · Masahiro Yamada
From: "Jason A. Donenfeld" <[email protected]>
Now that the kernel specifies binutils 2.23 as the minimum version, we can remove ifdefs for AVX2 and ADX throughout.
Signed-off-by Jason A. Donenfeld <[email protected]>Signed-off-by Masahiro Yamada <[email protected]>---
Changes in v2: None
arch/x86/Kconfig.assembler | 10 ---------- arch/x86/crypto/Makefile | 6 ++---- arch/x86/crypto/aesni-intel_avx-x86_64.S | 3 --- arch/x86/crypto/aesni-intel_glue.c | 7 ------- arch/x86/crypto/chacha_glue.c | 6 ++---- arch/x86/crypto/poly1305-x86_64-cryptogams.pl | 8 -------- arch/x86/crypto/poly1305_glue.c | 5 ++--- arch/x86/crypto/sha1_ssse3_glue.c | 6 ------ arch/x86/crypto/sha256-avx2-asm.S | 3 ---
Quoted 8 lines
arch/x86/crypto/sha256_ssse3_glue.c | 6 ------ arch/x86/crypto/sha512-avx2-asm.S | 3 --- arch/x86/crypto/sha512_ssse3_glue.c | 5 ----- crypto/Kconfig | 8 ++++---- lib/raid6/algos.c | 6 ------ lib/raid6/avx2.c | 4 ---- lib/raid6/recov_avx2.c | 6 ------ lib/raid6/test/Makefile | 3 ---
I see four checks for CONFIG_AS_AVX2 in:net/netfilter/nft_set_pipapo.cnet/netfilter/nf_tables_api.cnet/netfilter/Makefilenet/netfilter/nft_set_pipapo_avx2.h
Can changes for those be added to this?
Quoted 428 lines
17 files changed, 10 insertions(+), 85 deletions(-)arch/x86/Kconfig.assembler +0
-10@@ -1,11 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 # Copyright (C) 2020 Jason A. Donenfeld <[email protected]>. All Rights Reserved. -config AS_AVX2 - def_bool $(as-instr,vpbroadcastb %xmm0$(comma)%ymm1) - help - Supported by binutils >= 2.22 and LLVM integrated assembler - config AS_AVX512 def_bool $(as-instr,vpmovm2b %k1$(comma)%zmm5) help@@ -20,8 +15,3 @@ config AS_SHA256_NI def_bool $(as-instr,sha256msg1 %xmm0$(comma)%xmm1) help Supported by binutils >= 2.24 and LLVM integrated assembler - -config AS_ADX - def_bool $(as-instr,adox %eax$(comma)%eax) - help - Supported by binutils >= 2.23 and LLVM integrated assemblerarch/x86/crypto/Makefile +2
-4@@ -47,8 +47,7 @@ obj-$(CONFIG_CRYPTO_AEGIS128_AESNI_SSE2) += aegis128-aesni.o aegis128-aesni-y := aegis128-aesni-asm.o aegis128-aesni-glue.o obj-$(CONFIG_CRYPTO_CHACHA20_X86_64) += chacha-x86_64.o -chacha-x86_64-y := chacha-ssse3-x86_64.o chacha_glue.o -chacha-x86_64-$(CONFIG_AS_AVX2) += chacha-avx2-x86_64.o +chacha-x86_64-y := chacha-avx2-x86_64.o chacha-ssse3-x86_64.o chacha_glue.o chacha-x86_64-$(CONFIG_AS_AVX512) += chacha-avx512vl-x86_64.o obj-$(CONFIG_CRYPTO_AES_NI_INTEL) += aesni-intel.o@@ -56,8 +55,7 @@ aesni-intel-y := aesni-intel_asm.o aesni-intel_glue.o aesni-intel-$(CONFIG_64BIT) += aesni-intel_avx-x86_64.o aes_ctrby8_avx-x86_64.o obj-$(CONFIG_CRYPTO_SHA1_SSSE3) += sha1-ssse3.o -sha1-ssse3-y := sha1_ssse3_asm.o sha1_ssse3_glue.o -sha1-ssse3-$(CONFIG_AS_AVX2) += sha1_avx2_x86_64_asm.o +sha1-ssse3-y := sha1_avx2_x86_64_asm.o sha1_ssse3_asm.o sha1_ssse3_glue.o sha1-ssse3-$(CONFIG_AS_SHA1_NI) += sha1_ni_asm.o obj-$(CONFIG_CRYPTO_SHA256_SSSE3) += sha256-ssse3.oarch/x86/crypto/aesni-intel_avx-x86_64.S +0
-3@@ -1868,7 +1868,6 @@ key_256_finalize: ret SYM_FUNC_END(aesni_gcm_finalize_avx_gen2) -#ifdef CONFIG_AS_AVX2 ############################################################################### # GHASH_MUL MACRO to implement: Data*HashKey mod (128,127,126,121,0) # Input: A and B (128-bits each, bit-reflected)@@ -2836,5 +2835,3 @@ key_256_finalize4: FUNC_RESTORE ret SYM_FUNC_END(aesni_gcm_finalize_avx_gen4) - -#endif /* CONFIG_AS_AVX2 */arch/x86/crypto/aesni-intel_glue.c +0
-7@@ -233,7 +233,6 @@ static const struct aesni_gcm_tfm_s aesni_gcm_tfm_avx_gen2 = { .finalize = &aesni_gcm_finalize_avx_gen2, }; -#ifdef CONFIG_AS_AVX2 /* * asmlinkage void aesni_gcm_init_avx_gen4() * gcm_data *my_ctx_data, context data @@ -276,8 +275,6 @@ static const struct aesni_gcm_tfm_s aesni_gcm_tfm_avx_gen4 = { .finalize = &aesni_gcm_finalize_avx_gen4, }; -#endif -static inline struct aesni_rfc4106_gcm_ctx *aesni_rfc4106_gcm_ctx_get (struct crypto_aead *tfm) {@@ -706,10 +703,8 @@ static int gcmaes_crypt_by_sg(bool enc, struct aead_request *req,if (!enc) left -= auth_tag_len; -#ifdef CONFIG_AS_AVX2 if (left <AVX_GEN4_OPTSIZE && gcm_tfm == &aesni_gcm_tfm_avx_gen4) gcm_tfm = &aesni_gcm_tfm_avx_gen2; -#endif if (left <AVX_GEN2_OPTSIZE && gcm_tfm == &aesni_gcm_tfm_avx_gen2) gcm_tfm = &aesni_gcm_tfm_sse;@@ -1069,12 +1064,10 @@ static int __init aesni_init(void)if (!x86_match_cpu (aesni_cpu_id))return -ENODEV ;#ifdef CONFIG_X86_64 -#ifdef CONFIG_AS_AVX2 if (boot_cpu_has (X86_FEATURE_AVX2 )) {pr_info ("AVX2 version of gcm_enc/dec engaged.\n" ); aesni_gcm_tfm = &aesni_gcm_tfm_avx_gen4; }else -#endif if (boot_cpu_has (X86_FEATURE_AVX )) {pr_info ("AVX version of gcm_enc/dec engaged.\n" ); aesni_gcm_tfm = &aesni_gcm_tfm_avx_gen2;arch/x86/crypto/chacha_glue.c +2
-4@@ -79,8 +79,7 @@ static void chacha_dosimd(u32 *state, u8 *dst, const u8 *src, } } -if (IS_ENABLED (CONFIG_AS_AVX2 ) && -static_branch_likely (&chacha_use_avx2)) { +if (static_branch_likely (&chacha_use_avx2)) {while (bytes >=CHACHA_BLOCK_SIZE *8 ) {chacha_8block_xor_avx2 (state, dst, src, bytes, nrounds); bytes -=CHACHA_BLOCK_SIZE *8 ;@@ -288,8 +287,7 @@ static int __init chacha_simd_mod_init(void)static_branch_enable (&chacha_use_simd); -if (IS_ENABLED (CONFIG_AS_AVX2 ) && -boot_cpu_has (X86_FEATURE_AVX ) && +if (boot_cpu_has (X86_FEATURE_AVX ) &&boot_cpu_has (X86_FEATURE_AVX2 ) &&cpu_has_xfeatures (XFEATURE_MASK_SSE |XFEATURE_MASK_YMM ,NULL )) {static_branch_enable (&chacha_use_avx2);arch/x86/crypto/poly1305-x86_64-cryptogams.pl +0
-8@@ -1514,10 +1514,6 @@ ___ if ($avx>1) { -if ($kernel) { - $code .= "#ifdef CONFIG_AS_AVX2\n"; -} - my ($H0,$H1,$H2,$H3,$H4, $MASK, $T4,$T0,$T1,$T2,$T3, $D0,$D1,$D2,$D3,$D4) = map("%ymm$_",(0..15)); my $S4=$MASK;@@ -2808,10 +2804,6 @@ ___ poly1305_blocks_avxN(0); &end_function("poly1305_blocks_avx2"); -if($kernel) { - $code .= "#endif\n"; -} - ####################################################################### if ($avx>2) { # On entry we have input length divisible by 64. But since inner looparch/x86/crypto/poly1305_glue.c +2
-3@@ -108,7 +108,7 @@ static void poly1305_simd_blocks(void *ctx, const u8 *inp, size_t len,kernel_fpu_begin ();if (IS_ENABLED (CONFIG_AS_AVX512 ) &&static_branch_likely (&poly1305_use_avx512))poly1305_blocks_avx512 (ctx, inp, bytes, padbit); -else if (IS_ENABLED (CONFIG_AS_AVX2 ) &&static_branch_likely (&poly1305_use_avx2)) +else if (static_branch_likely (&poly1305_use_avx2))poly1305_blocks_avx2 (ctx, inp, bytes, padbit);else poly1305_blocks_avx (ctx, inp, bytes, padbit);@@ -264,8 +264,7 @@ static int __init poly1305_simd_mod_init(void)if (boot_cpu_has (X86_FEATURE_AVX ) &&cpu_has_xfeatures (XFEATURE_MASK_SSE |XFEATURE_MASK_YMM ,NULL ))static_branch_enable (&poly1305_use_avx); -if (IS_ENABLED (CONFIG_AS_AVX2 ) &&boot_cpu_has (X86_FEATURE_AVX ) && -boot_cpu_has (X86_FEATURE_AVX2 ) && +if (boot_cpu_has (X86_FEATURE_AVX ) &&boot_cpu_has (X86_FEATURE_AVX2 ) &&cpu_has_xfeatures (XFEATURE_MASK_SSE |XFEATURE_MASK_YMM ,NULL ))static_branch_enable (&poly1305_use_avx2); if (IS_ENABLED(CONFIG_AS_AVX512) && boot_cpu_has(X86_FEATURE_AVX) &&arch/x86/crypto/sha1_ssse3_glue.c +0
-6@@ -174,7 +174,6 @@ static void unregister_sha1_avx(void)crypto_unregister_shash (&sha1_avx_alg); } -#if defined(CONFIG_AS_AVX2 )#define SHA1_AVX2_BLOCK_OPTSIZE 4/* optimal 4*64 bytes of SHA1 blocks */ asmlinkage void sha1_transform_avx2(struct sha1_state *state,@@ -246,11 +245,6 @@ static void unregister_sha1_avx2(void)crypto_unregister_shash (&sha1_avx2_alg); } -#else -static inline int register_sha1_avx2 (void ) {return 0 ; } -static inline void unregister_sha1_avx2 (void ) { } -#endif -#ifdef CONFIG_AS_SHA1_NI asmlinkage void sha1_ni_transform(struct sha1_state *digest, const u8 *data,int rounds);arch/x86/crypto/sha256-avx2-asm.S +0
-3@@ -48,7 +48,6 @@ # This code schedules 2 blocks at a time, with 4 lanes per block ######################################################################## -#ifdef CONFIG_AS_AVX2 #include <linux/linkage.h> ## assume buffers not aligned@@ -767,5 +766,3 @@ _SHUF_00BA: .align 32 _SHUF_DC00: .octa 0x0b0a090803020100FFFFFFFFFFFFFFFF,0x0b0a090803020100FFFFFFFFFFFFFFFF - -#endifarch/x86/crypto/sha256_ssse3_glue.c +0
-6@@ -220,7 +220,6 @@ static void unregister_sha256_avx(void) ARRAY_SIZE(sha256_avx_algs)); } -#if defined(CONFIG_AS_AVX2 ) asmlinkage void sha256_transform_rorx(struct sha256_state *state,const u8 *data,int blocks);@@ -295,11 +294,6 @@ static void unregister_sha256_avx2(void) ARRAY_SIZE(sha256_avx2_algs)); } -#else -static inline int register_sha256_avx2 (void ) {return 0 ; } -static inline void unregister_sha256_avx2 (void ) { } -#endif -#ifdef CONFIG_AS_SHA256_NI asmlinkage void sha256_ni_transform(struct sha256_state *digest,const u8 *data,int rounds);arch/x86/crypto/sha512-avx2-asm.S +0
-3@@ -49,7 +49,6 @@ # This code schedules 1 blocks at a time, with 4 lanes per block ######################################################################## -#ifdef CONFIG_AS_AVX2 #include <linux/linkage.h> .text@@ -749,5 +748,3 @@ PSHUFFLE_BYTE_FLIP_MASK: MASK_YMM_LO: .octa 0x00000000000000000000000000000000 .octa 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF - -#endifarch/x86/crypto/sha512_ssse3_glue.c +0
-5@@ -218,7 +218,6 @@ static void unregister_sha512_avx(void) ARRAY_SIZE(sha512_avx_algs)); } -#if defined(CONFIG_AS_AVX2 ) asmlinkage void sha512_transform_rorx(struct sha512_state *state,const u8 *data,int blocks);@@ -293,10 +292,6 @@ static void unregister_sha512_avx2(void)crypto_unregister_shashes (sha512_avx2_algs,ARRAY_SIZE (sha512_avx2_algs)); } -#else -static inline int register_sha512_avx2 (void ) {return 0 ; } -static inline void unregister_sha512_avx2 (void ) { } -#endif static int __init sha512_ssse3_mod_init (void ) {crypto/Kconfig +4
-4@@ -267,7 +267,7 @@ config CRYPTO_CURVE25519 config CRYPTO_CURVE25519_X86 tristate "x86_64 accelerated Curve25519 scalar multiplication library" - depends on X86 && 64BIT && AS_ADX + depends on X86 && 64BIT select CRYPTO_LIB_CURVE25519_GENERIC select CRYPTO_ARCH_HAVE_LIB_CURVE25519@@ -465,7 +465,7 @@ config CRYPTO_NHPOLY1305_SSE2 config CRYPTO_NHPOLY1305_AVX2 tristate "NHPoly1305 hash function (x86_64 AVX2 implementation)" - depends on X86 && 64BIT && AS_AVX2 + depends on X86 && 64BIT select CRYPTO_NHPOLY1305 help AVX2 optimized implementation of the hash function used by the@@ -1303,7 +1303,7 @@ config CRYPTO_CAMELLIA_AESNI_AVX_X86_64 config CRYPTO_CAMELLIA_AESNI_AVX2_X86_64 tristate "Camellia cipher algorithm (x86_64/AES-NI/AVX2)" - depends on X86 && 64BIT && AS_AVX2 + depends on X86 && 64BIT depends on CRYPTO select CRYPTO_CAMELLIA_AESNI_AVX_X86_64 help@@ -1573,7 +1573,7 @@ config CRYPTO_SERPENT_AVX_X86_64 config CRYPTO_SERPENT_AVX2_X86_64 tristate "Serpent cipher algorithm (x86_64/AVX2)" - depends on X86 && 64BIT && AS_AVX2 + depends on X86 && 64BIT select CRYPTO_SERPENT_AVX_X86_64 help Serpent cipher algorithm, by Anderson, Biham & Knudsen.lib/raid6/algos.c +0
-6@@ -34,10 +34,8 @@ const struct raid6_calls * const raid6_algos[] = { &raid6_avx512x2, &raid6_avx512x1, #endif -#ifdef CONFIG_AS_AVX2 &raid6_avx2x2, &raid6_avx2x1, -#endif &raid6_sse2x2, &raid6_sse2x1, &raid6_sse1x2,@@ -51,11 +49,9 @@ const struct raid6_calls * const raid6_algos[] = { &raid6_avx512x2, &raid6_avx512x1, #endif -#ifdef CONFIG_AS_AVX2 &raid6_avx2x4, &raid6_avx2x2, &raid6_avx2x1, -#endif &raid6_sse2x4, &raid6_sse2x2, &raid6_sse2x1,@@ -101,9 +97,7 @@ const struct raid6_recov_calls *const raid6_recov_algos[] = {#ifdef CONFIG_AS_AVX512 &raid6_recov_avx512, #endif -#ifdef CONFIG_AS_AVX2 &raid6_recov_avx2, -#endif &raid6_recov_ssse3, #endif #ifdef CONFIG_S390lib/raid6/avx2.c +0
-4@@ -13,8 +13,6 @@* */ -#ifdef CONFIG_AS_AVX2 -#include <linux/raid/pq.h> #include "x86.h" @@ -470,5 +468,3 @@ const struct raid6_calls raid6_avx2x4 = {1 /* Has cache hints */ };#endif - -#endif /* CONFIG_AS_AVX2 */ lib/raid6/recov_avx2.c +0
-6@@ -4,8 +4,6 @@* Author: Jim Kukunas <[email protected]> */ -#ifdef CONFIG_AS_AVX2 -#include <linux/raid/pq.h> #include "x86.h" @@ -313,7 +311,3 @@ const struct raid6_recov_calls raid6_recov_avx2 = {#endif .priority = 2, }; - -#else -#warning "your version of binutils lacks AVX2 support" -#endif lib/raid6/test/Makefile +0
-4@@ -35,9 +35,6 @@ endif ifeq ($(IS_X86),yes) OBJS += mmx.o sse1.o sse2.o avx2.o recov_ssse3.o recov_avx2.o avx512.o recov_avx512.o CFLAGS += -DCONFIG_X86 - CFLAGS += $(shell echo "vpbroadcastb %xmm0, %ymm1" | \ - gcc -c -x assembler - >/dev/null 2>&1 && \ - rm ./-.o && echo -DCONFIG_AS_AVX2=1) CFLAGS += $(shell echo "vpmovm2b %k1, %zmm5" | \ gcc -c -x assembler - >/dev/null 2>&1 && \ rm ./-.o && echo -DCONFIG_AS_AVX512=1) --2.17.1
Signature · 2 lines
Thanks, ~Nick Desaulniers
Re: [PATCH v2 15/16] x86: update AS_* macros to binutils >=2.23, supporting ADX and AVX2
Quoted 5 lines · Nick Desaulniers
I see four checks for
CONFIG_AS_AVX2in:net/netfilter/nft_set_pipapo.cnet/netfilter/nf_tables_api.cnet/netfilter/Makefilenet/netfilter/nft_set_pipapo_avx2.h
That code isn't in Linus' tree right now is it? Does it make sense for us to see which subsystem trees (crypto, netfilter, raid, etc) are submitted to 5.7? Or would you rather this patchset be rebased now on next?
Re: [PATCH v2 10/16] x86: probe assembler capabilities via kconfig instead of makefile
In reply to Nick DesaulniersQuoted 13 lines · Nick Desaulniers
Quoted 10 lines · Masahiro Yamada
From: "Jason A. Donenfeld" <[email protected]>
Doing this probing inside of the Makefiles means we have a maze of ifdefs inside the source code and child Makefiles that need to make proper decisions on this too. Instead, we do it at
Kconfigtime, like many other compiler and assembler options, which allows us to set up the dependencies normally for full compilation units. In the process, the ADX test changes to use %eax instead of %r10 so that it's valid in both 32-bit and 64-bit mode.
Quoted 1 line
Does KConfig generate -D<foo> flags for
KBUILD_CFLAGSandKBUILD_AFLAGS?
kconfig sticks everything it's got into include/generated/autoconf.h. That's how you're able to use all those #ifdef CONFIG_* macros already. This change moves things from a command line -D to the autoconf.h file.
Quoted 1 line
Looks like
lib/raid6/test/Makefilealso generates some of these?
raid6 has its own crazy thing going on. The test directory compiles that code for use in userspace. You might argue that its whole situation is non-standard and weird and should be reworked differently, but that seems like fodder for a different patchset on the linux-raid list.
Re: [PATCH v2 15/16] x86: update AS_* macros to binutils >=2.23, supporting ADX and AVX2
In reply to Jason A. DonenfeldQuoted 10 lines · Jason A. Donenfeld
Quoted 5 lines · Nick Desaulniers
I see four checks for
CONFIG_AS_AVX2in:net/netfilter/nft_set_pipapo.cnet/netfilter/nf_tables_api.cnet/netfilter/Makefilenet/netfilter/nft_set_pipapo_avx2.hThat code isn't in Linus' tree right now is it? Does it make sense for
Indeed, it seems I was grepping in my checkout of -next.
Quoted 3 lines
us to see which subsystem trees (crypto, netfilter, raid, etc) are submitted to 5.7? Or would you rather this patchset be rebased now on next?
I think rebasing on -next is the way to go. I usually generate my patches off of that, though some trees that don't feed into -next are technically further ahead.
Signature · 2 lines
Thanks, ~Nick Desaulniers
Nick Desaulniers
Reviewed-byRe: [PATCH v2 10/16] x86: probe assembler capabilities via kconfig instead of makefile
In reply to Jason A. DonenfeldQuoted 16 lines · Jason A. Donenfeld
Quoted 13 lines · Nick Desaulniers
Quoted 10 lines · Masahiro Yamada
From: "Jason A. Donenfeld" <[email protected]>
Doing this probing inside of the Makefiles means we have a maze of ifdefs inside the source code and child Makefiles that need to make proper decisions on this too. Instead, we do it at
Kconfigtime, like many other compiler and assembler options, which allows us to set up the dependencies normally for full compilation units. In the process, the ADX test changes to use %eax instead of %r10 so that it's valid in both 32-bit and 64-bit mode.
Quoted 6 lines
Quoted 1 line
Does KConfig generate -D<foo> flags for
KBUILD_CFLAGSandKBUILD_AFLAGS?kconfig sticks everything it's got into
include/generated/autoconf.h. That's how you're able to use all those #ifdef CONFIG_* macros already. This change moves things from a command line -D to theautoconf.hfile.
Cool, I wondered how that was wired up. Though seeing the word "autoconf" brings on PTSD for me.
Patch LGTM.
Quoted 8 lines
Quoted 2 lines
Looks like
lib/raid6/test/Makefilealso generates some of these?raid6 has its own crazy thing going on. The test directory compiles that code for use in userspace. You might argue that its whole situation is non-standard and weird and should be reworked differently, but that seems like fodder for a different patchset on the linux-raid list.
Signature · 2 lines
Thanks, ~Nick Desaulniers
PATCH 1/1 x86: update AS_* macros to binutils >=2.23, supporting ADX and AVX2
In reply to Nick DesaulniersNow that the kernel specifies binutils 2.23 as the minimum version, we can remove ifdefs for AVX2 and ADX throughout.
---
Masahiro - here's an updated patch based on -next, as Nick requested. Whether or not you'd prefer this one to the other one based on Linus' tree is up to you, but here it is in case it's helpful.
arch/x86/Kconfig.assembler | 10 ---------- arch/x86/crypto/Makefile | 6 ++---- arch/x86/crypto/aesni-intel_avx-x86_64.S | 3 --- arch/x86/crypto/aesni-intel_glue.c | 7 ------- arch/x86/crypto/chacha_glue.c | 6 ++---- arch/x86/crypto/poly1305-x86_64-cryptogams.pl | 8 -------- arch/x86/crypto/poly1305_glue.c | 5 ++--- arch/x86/crypto/sha1_ssse3_glue.c | 6 ------ arch/x86/crypto/sha256-avx2-asm.S | 3 --- arch/x86/crypto/sha256_ssse3_glue.c | 6 ------ arch/x86/crypto/sha512-avx2-asm.S | 3 --- arch/x86/crypto/sha512_ssse3_glue.c | 5 ----- crypto/Kconfig | 8 ++++---- lib/raid6/algos.c | 6 ------ lib/raid6/avx2.c | 4 ---- lib/raid6/recov_avx2.c | 6 ------ lib/raid6/test/Makefile | 3 --- net/netfilter/Makefile | 2 -- net/netfilter/nf_tables_api.c | 2 +- net/netfilter/nft_set_pipapo.c | 2 +- net/netfilter/nft_set_pipapo_avx2.h | 3 --- 21 files changed, 12 insertions(+), 92 deletions(-)
arch/x86/Kconfig.assembler +0 -10
@@ -1,11 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 # Copyright (C) 2020 Jason A. Donenfeld <[email protected]>. All Rights Reserved. -config AS_AVX2 - def_bool $(as-instr,vpbroadcastb %xmm0$(comma)%ymm1) - help - Supported by binutils >= 2.22 and LLVM integrated assembler - config AS_AVX512 def_bool $(as-instr,vpmovm2b %k1$(comma)%zmm5) help
@@ -20,8 +15,3 @@ config AS_SHA256_NI def_bool $(as-instr,sha256msg1 %xmm0$(comma)%xmm1) help Supported by binutils >= 2.24 and LLVM integrated assembler - -config AS_ADX - def_bool $(as-instr,adox %eax$(comma)%eax) - help - Supported by binutils >= 2.23 and LLVM integrated assembler
arch/x86/crypto/Makefile +2 -4
@@ -47,8 +47,7 @@ obj-$(CONFIG_CRYPTO_AEGIS128_AESNI_SSE2) += aegis128-aesni.o aegis128-aesni-y := aegis128-aesni-asm.o aegis128-aesni-glue.o obj-$(CONFIG_CRYPTO_CHACHA20_X86_64) += chacha-x86_64.o -chacha-x86_64-y := chacha-ssse3-x86_64.o chacha_glue.o -chacha-x86_64-$(CONFIG_AS_AVX2) += chacha-avx2-x86_64.o +chacha-x86_64-y := chacha-avx2-x86_64.o chacha-ssse3-x86_64.o chacha_glue.o chacha-x86_64-$(CONFIG_AS_AVX512) += chacha-avx512vl-x86_64.o obj-$(CONFIG_CRYPTO_AES_NI_INTEL) += aesni-intel.o
@@ -56,8 +55,7 @@ aesni-intel-y := aesni-intel_asm.o aesni-intel_glue.o aesni-intel-$(CONFIG_64BIT) += aesni-intel_avx-x86_64.o aes_ctrby8_avx-x86_64.o obj-$(CONFIG_CRYPTO_SHA1_SSSE3) += sha1-ssse3.o -sha1-ssse3-y := sha1_ssse3_asm.o sha1_ssse3_glue.o -sha1-ssse3-$(CONFIG_AS_AVX2) += sha1_avx2_x86_64_asm.o +sha1-ssse3-y := sha1_avx2_x86_64_asm.o sha1_ssse3_asm.o sha1_ssse3_glue.o sha1-ssse3-$(CONFIG_AS_SHA1_NI) += sha1_ni_asm.o obj-$(CONFIG_CRYPTO_SHA256_SSSE3) += sha256-ssse3.o
arch/x86/crypto/aesni-intel_avx-x86_64.S +0 -3
@@ -1868,7 +1868,6 @@ key_256_finalize: ret SYM_FUNC_END(aesni_gcm_finalize_avx_gen2) -#ifdef CONFIG_AS_AVX2 ############################################################################### # GHASH_MUL MACRO to implement: Data*HashKey mod (128,127,126,121,0) # Input: A and B (128-bits each, bit-reflected)
@@ -2836,5 +2835,3 @@ key_256_finalize4: FUNC_RESTORE ret SYM_FUNC_END(aesni_gcm_finalize_avx_gen4) - -#endif /* CONFIG_AS_AVX2 */
arch/x86/crypto/aesni-intel_glue.c +0 -7
@@ -233,7 +233,6 @@ static const struct aesni_gcm_tfm_s aesni_gcm_tfm_avx_gen2 = { .finalize = &aesni_gcm_finalize_avx_gen2, }; -#ifdef CONFIG_AS_AVX2 /* * asmlinkage void aesni_gcm_init_avx_gen4() * gcm_data *my_ctx_data, context data
@@ -276,8 +275,6 @@ static const struct aesni_gcm_tfm_s aesni_gcm_tfm_avx_gen4 = { .finalize = &aesni_gcm_finalize_avx_gen4, }; -#endif -static inline struct aesni_rfc4106_gcm_ctx *aesni_rfc4106_gcm_ctx_get (struct crypto_aead *tfm) {
@@ -706,10 +703,8 @@ static int gcmaes_crypt_by_sg(bool enc, struct aead_request *req,if (!enc) left -= auth_tag_len; -#ifdef CONFIG_AS_AVX2 if (left <AVX_GEN4_OPTSIZE && gcm_tfm == &aesni_gcm_tfm_avx_gen4) gcm_tfm = &aesni_gcm_tfm_avx_gen2; -#endif if (left <AVX_GEN2_OPTSIZE && gcm_tfm == &aesni_gcm_tfm_avx_gen2) gcm_tfm = &aesni_gcm_tfm_sse;
@@ -1069,12 +1064,10 @@ static int __init aesni_init(void)if (!x86_match_cpu (aesni_cpu_id))return -ENODEV ;#ifdef CONFIG_X86_64 -#ifdef CONFIG_AS_AVX2 if (boot_cpu_has (X86_FEATURE_AVX2 )) {pr_info ("AVX2 version of gcm_enc/dec engaged.\n" ); aesni_gcm_tfm = &aesni_gcm_tfm_avx_gen4; }else -#endif if (boot_cpu_has (X86_FEATURE_AVX )) {pr_info ("AVX version of gcm_enc/dec engaged.\n" ); aesni_gcm_tfm = &aesni_gcm_tfm_avx_gen2;
arch/x86/crypto/chacha_glue.c +2 -4
@@ -79,8 +79,7 @@ static void chacha_dosimd(u32 *state, u8 *dst, const u8 *src, } } -if (IS_ENABLED (CONFIG_AS_AVX2 ) && -static_branch_likely (&chacha_use_avx2)) { +if (static_branch_likely (&chacha_use_avx2)) {while (bytes >=CHACHA_BLOCK_SIZE *8 ) {chacha_8block_xor_avx2 (state, dst, src, bytes, nrounds); bytes -=CHACHA_BLOCK_SIZE *8 ;
@@ -288,8 +287,7 @@ static int __init chacha_simd_mod_init(void)static_branch_enable (&chacha_use_simd); -if (IS_ENABLED (CONFIG_AS_AVX2 ) && -boot_cpu_has (X86_FEATURE_AVX ) && +if (boot_cpu_has (X86_FEATURE_AVX ) &&boot_cpu_has (X86_FEATURE_AVX2 ) &&cpu_has_xfeatures (XFEATURE_MASK_SSE |XFEATURE_MASK_YMM ,NULL )) {static_branch_enable (&chacha_use_avx2);
arch/x86/crypto/poly1305-x86_64-cryptogams.pl +0 -8
@@ -1514,10 +1514,6 @@ ___ if ($avx>1) { -if ($kernel) { - $code .= "#ifdef CONFIG_AS_AVX2\n"; -} - my ($H0,$H1,$H2,$H3,$H4, $MASK, $T4,$T0,$T1,$T2,$T3, $D0,$D1,$D2,$D3,$D4) = map("%ymm$_",(0..15)); my $S4=$MASK;
@@ -2808,10 +2804,6 @@ ___ poly1305_blocks_avxN(0); &end_function("poly1305_blocks_avx2"); -if($kernel) { - $code .= "#endif\n"; -} - ####################################################################### if ($avx>2) { # On entry we have input length divisible by 64. But since inner loop
arch/x86/crypto/poly1305_glue.c +2 -3
@@ -108,7 +108,7 @@ static void poly1305_simd_blocks(void *ctx, const u8 *inp, size_t len,kernel_fpu_begin ();if (IS_ENABLED (CONFIG_AS_AVX512 ) &&static_branch_likely (&poly1305_use_avx512))poly1305_blocks_avx512 (ctx, inp, bytes, padbit); -else if (IS_ENABLED (CONFIG_AS_AVX2 ) &&static_branch_likely (&poly1305_use_avx2)) +else if (static_branch_likely (&poly1305_use_avx2))poly1305_blocks_avx2 (ctx, inp, bytes, padbit);else poly1305_blocks_avx (ctx, inp, bytes, padbit);
@@ -264,8 +264,7 @@ static int __init poly1305_simd_mod_init(void)if (boot_cpu_has (X86_FEATURE_AVX ) &&cpu_has_xfeatures (XFEATURE_MASK_SSE |XFEATURE_MASK_YMM ,NULL ))static_branch_enable (&poly1305_use_avx); -if (IS_ENABLED (CONFIG_AS_AVX2 ) &&boot_cpu_has (X86_FEATURE_AVX ) && -boot_cpu_has (X86_FEATURE_AVX2 ) && +if (boot_cpu_has (X86_FEATURE_AVX ) &&boot_cpu_has (X86_FEATURE_AVX2 ) &&cpu_has_xfeatures (XFEATURE_MASK_SSE |XFEATURE_MASK_YMM ,NULL ))static_branch_enable (&poly1305_use_avx2); if (IS_ENABLED(CONFIG_AS_AVX512) && boot_cpu_has(X86_FEATURE_AVX) &&
arch/x86/crypto/sha1_ssse3_glue.c +0 -6
@@ -174,7 +174,6 @@ static void unregister_sha1_avx(void)crypto_unregister_shash (&sha1_avx_alg); } -#if defined(CONFIG_AS_AVX2 )#define SHA1_AVX2_BLOCK_OPTSIZE 4/* optimal 4*64 bytes of SHA1 blocks */ asmlinkage void sha1_transform_avx2(struct sha1_state *state,
@@ -246,11 +245,6 @@ static void unregister_sha1_avx2(void)crypto_unregister_shash (&sha1_avx2_alg); } -#else -static inline int register_sha1_avx2 (void ) {return 0 ; } -static inline void unregister_sha1_avx2 (void ) { } -#endif -#ifdef CONFIG_AS_SHA1_NI asmlinkage void sha1_ni_transform(struct sha1_state *digest, const u8 *data,int rounds);
arch/x86/crypto/sha256-avx2-asm.S +0 -3
@@ -48,7 +48,6 @@ # This code schedules 2 blocks at a time, with 4 lanes per block ######################################################################## -#ifdef CONFIG_AS_AVX2 #include <linux/linkage.h> ## assume buffers not aligned
@@ -767,5 +766,3 @@ _SHUF_00BA: .align 32 _SHUF_DC00: .octa 0x0b0a090803020100FFFFFFFFFFFFFFFF,0x0b0a090803020100FFFFFFFFFFFFFFFF - -#endif
arch/x86/crypto/sha256_ssse3_glue.c +0 -6
@@ -220,7 +220,6 @@ static void unregister_sha256_avx(void) ARRAY_SIZE(sha256_avx_algs)); } -#if defined(CONFIG_AS_AVX2 ) asmlinkage void sha256_transform_rorx(struct sha256_state *state,const u8 *data,int blocks);
@@ -295,11 +294,6 @@ static void unregister_sha256_avx2(void) ARRAY_SIZE(sha256_avx2_algs)); } -#else -static inline int register_sha256_avx2 (void ) {return 0 ; } -static inline void unregister_sha256_avx2 (void ) { } -#endif -#ifdef CONFIG_AS_SHA256_NI asmlinkage void sha256_ni_transform(struct sha256_state *digest,const u8 *data,int rounds);
arch/x86/crypto/sha512-avx2-asm.S +0 -3
@@ -49,7 +49,6 @@ # This code schedules 1 blocks at a time, with 4 lanes per block ######################################################################## -#ifdef CONFIG_AS_AVX2 #include <linux/linkage.h> .text
@@ -749,5 +748,3 @@ PSHUFFLE_BYTE_FLIP_MASK: MASK_YMM_LO: .octa 0x00000000000000000000000000000000 .octa 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF - -#endif
arch/x86/crypto/sha512_ssse3_glue.c +0 -5
@@ -218,7 +218,6 @@ static void unregister_sha512_avx(void) ARRAY_SIZE(sha512_avx_algs)); } -#if defined(CONFIG_AS_AVX2 ) asmlinkage void sha512_transform_rorx(struct sha512_state *state,const u8 *data,int blocks);
@@ -293,10 +292,6 @@ static void unregister_sha512_avx2(void)crypto_unregister_shashes (sha512_avx2_algs,ARRAY_SIZE (sha512_avx2_algs)); } -#else -static inline int register_sha512_avx2 (void ) {return 0 ; } -static inline void unregister_sha512_avx2 (void ) { } -#endif static int __init sha512_ssse3_mod_init (void ) {
crypto/Kconfig +4 -4
@@ -267,7 +267,7 @@ config CRYPTO_CURVE25519 config CRYPTO_CURVE25519_X86 tristate "x86_64 accelerated Curve25519 scalar multiplication library" - depends on X86 && 64BIT && AS_ADX + depends on X86 && 64BIT select CRYPTO_LIB_CURVE25519_GENERIC select CRYPTO_ARCH_HAVE_LIB_CURVE25519
@@ -465,7 +465,7 @@ config CRYPTO_NHPOLY1305_SSE2 config CRYPTO_NHPOLY1305_AVX2 tristate "NHPoly1305 hash function (x86_64 AVX2 implementation)" - depends on X86 && 64BIT && AS_AVX2 + depends on X86 && 64BIT select CRYPTO_NHPOLY1305 help AVX2 optimized implementation of the hash function used by the
@@ -1303,7 +1303,7 @@ config CRYPTO_CAMELLIA_AESNI_AVX_X86_64 config CRYPTO_CAMELLIA_AESNI_AVX2_X86_64 tristate "Camellia cipher algorithm (x86_64/AES-NI/AVX2)" - depends on X86 && 64BIT && AS_AVX2 + depends on X86 && 64BIT depends on CRYPTO select CRYPTO_CAMELLIA_AESNI_AVX_X86_64 help
@@ -1573,7 +1573,7 @@ config CRYPTO_SERPENT_AVX_X86_64 config CRYPTO_SERPENT_AVX2_X86_64 tristate "Serpent cipher algorithm (x86_64/AVX2)" - depends on X86 && 64BIT && AS_AVX2 + depends on X86 && 64BIT select CRYPTO_SERPENT_AVX_X86_64 help Serpent cipher algorithm, by Anderson, Biham & Knudsen.
lib/raid6/algos.c +0 -6
@@ -34,10 +34,8 @@ const struct raid6_calls * const raid6_algos[] = { &raid6_avx512x2, &raid6_avx512x1, #endif -#ifdef CONFIG_AS_AVX2 &raid6_avx2x2, &raid6_avx2x1, -#endif &raid6_sse2x2, &raid6_sse2x1, &raid6_sse1x2,
@@ -51,11 +49,9 @@ const struct raid6_calls * const raid6_algos[] = { &raid6_avx512x2, &raid6_avx512x1, #endif -#ifdef CONFIG_AS_AVX2 &raid6_avx2x4, &raid6_avx2x2, &raid6_avx2x1, -#endif &raid6_sse2x4, &raid6_sse2x2, &raid6_sse2x1,
@@ -101,9 +97,7 @@ const struct raid6_recov_calls *const raid6_recov_algos[] = {#ifdef CONFIG_AS_AVX512 &raid6_recov_avx512, #endif -#ifdef CONFIG_AS_AVX2 &raid6_recov_avx2, -#endif &raid6_recov_ssse3, #endif #ifdef CONFIG_S390
lib/raid6/avx2.c +0 -4
@@ -13,8 +13,6 @@* */ -#ifdef CONFIG_AS_AVX2 -#include <linux/raid/pq.h> #include "x86.h"
@@ -470,5 +468,3 @@ const struct raid6_calls raid6_avx2x4 = {1 /* Has cache hints */ };#endif - -#endif /* CONFIG_AS_AVX2 */
lib/raid6/recov_avx2.c +0 -6
@@ -4,8 +4,6 @@* Author: Jim Kukunas <[email protected]> */ -#ifdef CONFIG_AS_AVX2 -#include <linux/raid/pq.h> #include "x86.h"
@@ -313,7 +311,3 @@ const struct raid6_recov_calls raid6_recov_avx2 = {#endif .priority = 2, }; - -#else -#warning "your version of binutils lacks AVX2 support" -#endif
lib/raid6/test/Makefile +0 -3
@@ -35,9 +35,6 @@ endif ifeq ($(IS_X86),yes) OBJS += mmx.o sse1.o sse2.o avx2.o recov_ssse3.o recov_avx2.o avx512.o recov_avx512.o CFLAGS += -DCONFIG_X86 - CFLAGS += $(shell echo "vpbroadcastb %xmm0, %ymm1" | \ - gcc -c -x assembler - >/dev/null 2>&1 && \ - rm ./-.o && echo -DCONFIG_AS_AVX2=1) CFLAGS += $(shell echo "vpmovm2b %k1, %zmm5" | \ gcc -c -x assembler - >/dev/null 2>&1 && \ rm ./-.o && echo -DCONFIG_AS_AVX512=1)
net/netfilter/Makefile +0 -2
@@ -83,10 +83,8 @@ nf_tables-objs := nf_tables_core.o nf_tables_api.o nft_chain_filter.o \ nft_set_pipapo.o ifdef CONFIG_X86_64 -ifneq (,$(findstring -DCONFIG_AS_AVX2=1,$(KBUILD_CFLAGS))) nf_tables-objs += nft_set_pipapo_avx2.o endif -endif obj-$(CONFIG_NF_TABLES) += nf_tables.o obj-$(CONFIG_NFT_COMPAT) += nft_compat.o
net/netfilter/nf_tables_api.c +1 -1
@@ -3290,7 +3290,7 @@ static const struct nft_set_type *nft_set_types[] = { &nft_set_rhash_type, &nft_set_bitmap_type, &nft_set_rbtree_type, -#if defined(CONFIG_X86_64) && defined(CONFIG_AS_AVX2) +#if defined(CONFIG_X86_64) &nft_set_pipapo_avx2_type,#endif &nft_set_pipapo_type,
net/netfilter/nft_set_pipapo.c +1 -1
@@ -2201,7 +2201,7 @@ const struct nft_set_type nft_set_pipapo_type = { }, }; -#if defined(CONFIG_X86_64 ) && defined(CONFIG_AS_AVX2 ) +#if defined(CONFIG_X86_64 )const struct nft_set_type nft_set_pipapo_avx2_type = { .features =NFT_SET_INTERVAL |NFT_SET_MAP |NFT_SET_OBJECT | NFT_SET_TIMEOUT,
net/netfilter/nft_set_pipapo_avx2.h +0 -3
@@ -1,7 +1,6 @@/* SPDX-License-Identifier: GPL-2.0-only */ #ifndef _NFT_SET_PIPAPO_AVX2_H -#ifdef CONFIG_AS_AVX2 #include <asm/fpu/xstate.h> #define NFT_PIPAPO_ALIGN (XSAVE_YMM_SIZE / BITS_PER_BYTE)
@@ -9,6 +8,4 @@ bool nft_pipapo_avx2_lookup(const struct net *net, const struct nft_set *set, const u32 *key, const struct nft_set_ext **ext);bool nft_pipapo_avx2_estimate (const struct nft_set_desc *desc,u32 features,struct nft_set_estimate *est); -#endif /* CONFIG_AS_AVX2 */ -#endif /* _NFT_SET_PIPAPO_AVX2_H */
Signature · 1 line
2.26.0
Re: [PATCH] x86: update AS_* macros to binutils >=2.23, supporting ADX and AVX2
Quoted 5 lines · Jason A. Donenfeld
Now that the kernel specifies binutils 2.23 as the minimum version, we can remove ifdefs for AVX2 and ADX throughout.
Signed-off-by Jason A. Donenfeld <[email protected]>
That's the version I'd sign off on. Better to fix potential issues we know are on the horizon, before they become breakages. Especially with the merge window fast approaching. I appreciate you following up on a fix.
Quoted 517 lines
---
Masahiro - here's an updated patch based on -next, as Nick requested. Whether or not you'd prefer this one to the other one based on Linus' tree is up to you, but here it is in case it's helpful.
arch/x86/Kconfig.assembler | 10 ---------- arch/x86/crypto/Makefile | 6 ++---- arch/x86/crypto/aesni-intel_avx-x86_64.S | 3 --- arch/x86/crypto/aesni-intel_glue.c | 7 ------- arch/x86/crypto/chacha_glue.c | 6 ++---- arch/x86/crypto/poly1305-x86_64-cryptogams.pl | 8 -------- arch/x86/crypto/poly1305_glue.c | 5 ++--- arch/x86/crypto/sha1_ssse3_glue.c | 6 ------ arch/x86/crypto/sha256-avx2-asm.S | 3 --- arch/x86/crypto/sha256_ssse3_glue.c | 6 ------ arch/x86/crypto/sha512-avx2-asm.S | 3 --- arch/x86/crypto/sha512_ssse3_glue.c | 5 ----- crypto/Kconfig | 8 ++++---- lib/raid6/algos.c | 6 ------ lib/raid6/avx2.c | 4 ---- lib/raid6/recov_avx2.c | 6 ------ lib/raid6/test/Makefile | 3 --- net/netfilter/Makefile | 2 -- net/netfilter/nf_tables_api.c | 2 +- net/netfilter/nft_set_pipapo.c | 2 +- net/netfilter/nft_set_pipapo_avx2.h | 3 --- 21 files changed, 12 insertions(+), 92 deletions(-)arch/x86/Kconfig.assembler +0
-10@@ -1,11 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 # Copyright (C) 2020 Jason A. Donenfeld <[email protected]>. All Rights Reserved. -config AS_AVX2 - def_bool $(as-instr,vpbroadcastb %xmm0$(comma)%ymm1) - help - Supported by binutils >= 2.22 and LLVM integrated assembler - config AS_AVX512 def_bool $(as-instr,vpmovm2b %k1$(comma)%zmm5) help@@ -20,8 +15,3 @@ config AS_SHA256_NI def_bool $(as-instr,sha256msg1 %xmm0$(comma)%xmm1) help Supported by binutils >= 2.24 and LLVM integrated assembler - -config AS_ADX - def_bool $(as-instr,adox %eax$(comma)%eax) - help - Supported by binutils >= 2.23 and LLVM integrated assemblerarch/x86/crypto/Makefile +2
-4@@ -47,8 +47,7 @@ obj-$(CONFIG_CRYPTO_AEGIS128_AESNI_SSE2) += aegis128-aesni.o aegis128-aesni-y := aegis128-aesni-asm.o aegis128-aesni-glue.o obj-$(CONFIG_CRYPTO_CHACHA20_X86_64) += chacha-x86_64.o -chacha-x86_64-y := chacha-ssse3-x86_64.o chacha_glue.o -chacha-x86_64-$(CONFIG_AS_AVX2) += chacha-avx2-x86_64.o +chacha-x86_64-y := chacha-avx2-x86_64.o chacha-ssse3-x86_64.o chacha_glue.o chacha-x86_64-$(CONFIG_AS_AVX512) += chacha-avx512vl-x86_64.o obj-$(CONFIG_CRYPTO_AES_NI_INTEL) += aesni-intel.o@@ -56,8 +55,7 @@ aesni-intel-y := aesni-intel_asm.o aesni-intel_glue.o aesni-intel-$(CONFIG_64BIT) += aesni-intel_avx-x86_64.o aes_ctrby8_avx-x86_64.o obj-$(CONFIG_CRYPTO_SHA1_SSSE3) += sha1-ssse3.o -sha1-ssse3-y := sha1_ssse3_asm.o sha1_ssse3_glue.o -sha1-ssse3-$(CONFIG_AS_AVX2) += sha1_avx2_x86_64_asm.o +sha1-ssse3-y := sha1_avx2_x86_64_asm.o sha1_ssse3_asm.o sha1_ssse3_glue.o sha1-ssse3-$(CONFIG_AS_SHA1_NI) += sha1_ni_asm.o obj-$(CONFIG_CRYPTO_SHA256_SSSE3) += sha256-ssse3.oarch/x86/crypto/aesni-intel_avx-x86_64.S +0
-3@@ -1868,7 +1868,6 @@ key_256_finalize: ret SYM_FUNC_END(aesni_gcm_finalize_avx_gen2) -#ifdef CONFIG_AS_AVX2 ############################################################################### # GHASH_MUL MACRO to implement: Data*HashKey mod (128,127,126,121,0) # Input: A and B (128-bits each, bit-reflected)@@ -2836,5 +2835,3 @@ key_256_finalize4: FUNC_RESTORE ret SYM_FUNC_END(aesni_gcm_finalize_avx_gen4) - -#endif /* CONFIG_AS_AVX2 */arch/x86/crypto/aesni-intel_glue.c +0
-7@@ -233,7 +233,6 @@ static const struct aesni_gcm_tfm_s aesni_gcm_tfm_avx_gen2 = { .finalize = &aesni_gcm_finalize_avx_gen2, }; -#ifdef CONFIG_AS_AVX2 /* * asmlinkage void aesni_gcm_init_avx_gen4() * gcm_data *my_ctx_data, context data @@ -276,8 +275,6 @@ static const struct aesni_gcm_tfm_s aesni_gcm_tfm_avx_gen4 = { .finalize = &aesni_gcm_finalize_avx_gen4, }; -#endif -static inline struct aesni_rfc4106_gcm_ctx *aesni_rfc4106_gcm_ctx_get (struct crypto_aead *tfm) {@@ -706,10 +703,8 @@ static int gcmaes_crypt_by_sg(bool enc, struct aead_request *req,if (!enc) left -= auth_tag_len; -#ifdef CONFIG_AS_AVX2 if (left <AVX_GEN4_OPTSIZE && gcm_tfm == &aesni_gcm_tfm_avx_gen4) gcm_tfm = &aesni_gcm_tfm_avx_gen2; -#endif if (left <AVX_GEN2_OPTSIZE && gcm_tfm == &aesni_gcm_tfm_avx_gen2) gcm_tfm = &aesni_gcm_tfm_sse;@@ -1069,12 +1064,10 @@ static int __init aesni_init(void)if (!x86_match_cpu (aesni_cpu_id))return -ENODEV ;#ifdef CONFIG_X86_64 -#ifdef CONFIG_AS_AVX2 if (boot_cpu_has (X86_FEATURE_AVX2 )) {pr_info ("AVX2 version of gcm_enc/dec engaged.\n" ); aesni_gcm_tfm = &aesni_gcm_tfm_avx_gen4; }else -#endif if (boot_cpu_has (X86_FEATURE_AVX )) {pr_info ("AVX version of gcm_enc/dec engaged.\n" ); aesni_gcm_tfm = &aesni_gcm_tfm_avx_gen2;arch/x86/crypto/chacha_glue.c +2
-4@@ -79,8 +79,7 @@ static void chacha_dosimd(u32 *state, u8 *dst, const u8 *src, } } -if (IS_ENABLED (CONFIG_AS_AVX2 ) && -static_branch_likely (&chacha_use_avx2)) { +if (static_branch_likely (&chacha_use_avx2)) {while (bytes >=CHACHA_BLOCK_SIZE *8 ) {chacha_8block_xor_avx2 (state, dst, src, bytes, nrounds); bytes -=CHACHA_BLOCK_SIZE *8 ;@@ -288,8 +287,7 @@ static int __init chacha_simd_mod_init(void)static_branch_enable (&chacha_use_simd); -if (IS_ENABLED (CONFIG_AS_AVX2 ) && -boot_cpu_has (X86_FEATURE_AVX ) && +if (boot_cpu_has (X86_FEATURE_AVX ) &&boot_cpu_has (X86_FEATURE_AVX2 ) &&cpu_has_xfeatures (XFEATURE_MASK_SSE |XFEATURE_MASK_YMM ,NULL )) {static_branch_enable (&chacha_use_avx2);arch/x86/crypto/poly1305-x86_64-cryptogams.pl +0
-8@@ -1514,10 +1514,6 @@ ___ if ($avx>1) { -if ($kernel) { - $code .= "#ifdef CONFIG_AS_AVX2\n"; -} - my ($H0,$H1,$H2,$H3,$H4, $MASK, $T4,$T0,$T1,$T2,$T3, $D0,$D1,$D2,$D3,$D4) = map("%ymm$_",(0..15)); my $S4=$MASK;@@ -2808,10 +2804,6 @@ ___ poly1305_blocks_avxN(0); &end_function("poly1305_blocks_avx2"); -if($kernel) { - $code .= "#endif\n"; -} - ####################################################################### if ($avx>2) { # On entry we have input length divisible by 64. But since inner looparch/x86/crypto/poly1305_glue.c +2
-3@@ -108,7 +108,7 @@ static void poly1305_simd_blocks(void *ctx, const u8 *inp, size_t len,kernel_fpu_begin ();if (IS_ENABLED (CONFIG_AS_AVX512 ) &&static_branch_likely (&poly1305_use_avx512))poly1305_blocks_avx512 (ctx, inp, bytes, padbit); -else if (IS_ENABLED (CONFIG_AS_AVX2 ) &&static_branch_likely (&poly1305_use_avx2)) +else if (static_branch_likely (&poly1305_use_avx2))poly1305_blocks_avx2 (ctx, inp, bytes, padbit);else poly1305_blocks_avx (ctx, inp, bytes, padbit);@@ -264,8 +264,7 @@ static int __init poly1305_simd_mod_init(void)if (boot_cpu_has (X86_FEATURE_AVX ) &&cpu_has_xfeatures (XFEATURE_MASK_SSE |XFEATURE_MASK_YMM ,NULL ))static_branch_enable (&poly1305_use_avx); -if (IS_ENABLED (CONFIG_AS_AVX2 ) &&boot_cpu_has (X86_FEATURE_AVX ) && -boot_cpu_has (X86_FEATURE_AVX2 ) && +if (boot_cpu_has (X86_FEATURE_AVX ) &&boot_cpu_has (X86_FEATURE_AVX2 ) &&cpu_has_xfeatures (XFEATURE_MASK_SSE |XFEATURE_MASK_YMM ,NULL ))static_branch_enable (&poly1305_use_avx2); if (IS_ENABLED(CONFIG_AS_AVX512) && boot_cpu_has(X86_FEATURE_AVX) &&arch/x86/crypto/sha1_ssse3_glue.c +0
-6@@ -174,7 +174,6 @@ static void unregister_sha1_avx(void)crypto_unregister_shash (&sha1_avx_alg); } -#if defined(CONFIG_AS_AVX2 )#define SHA1_AVX2_BLOCK_OPTSIZE 4/* optimal 4*64 bytes of SHA1 blocks */ asmlinkage void sha1_transform_avx2(struct sha1_state *state,@@ -246,11 +245,6 @@ static void unregister_sha1_avx2(void)crypto_unregister_shash (&sha1_avx2_alg); } -#else -static inline int register_sha1_avx2 (void ) {return 0 ; } -static inline void unregister_sha1_avx2 (void ) { } -#endif -#ifdef CONFIG_AS_SHA1_NI asmlinkage void sha1_ni_transform(struct sha1_state *digest, const u8 *data,int rounds);arch/x86/crypto/sha256-avx2-asm.S +0
-3@@ -48,7 +48,6 @@ # This code schedules 2 blocks at a time, with 4 lanes per block ######################################################################## -#ifdef CONFIG_AS_AVX2 #include <linux/linkage.h> ## assume buffers not aligned@@ -767,5 +766,3 @@ _SHUF_00BA: .align 32 _SHUF_DC00: .octa 0x0b0a090803020100FFFFFFFFFFFFFFFF,0x0b0a090803020100FFFFFFFFFFFFFFFF - -#endifarch/x86/crypto/sha256_ssse3_glue.c +0
-6@@ -220,7 +220,6 @@ static void unregister_sha256_avx(void) ARRAY_SIZE(sha256_avx_algs)); } -#if defined(CONFIG_AS_AVX2 ) asmlinkage void sha256_transform_rorx(struct sha256_state *state,const u8 *data,int blocks);@@ -295,11 +294,6 @@ static void unregister_sha256_avx2(void) ARRAY_SIZE(sha256_avx2_algs)); } -#else -static inline int register_sha256_avx2 (void ) {return 0 ; } -static inline void unregister_sha256_avx2 (void ) { } -#endif -#ifdef CONFIG_AS_SHA256_NI asmlinkage void sha256_ni_transform(struct sha256_state *digest,const u8 *data,int rounds);arch/x86/crypto/sha512-avx2-asm.S +0
-3@@ -49,7 +49,6 @@ # This code schedules 1 blocks at a time, with 4 lanes per block ######################################################################## -#ifdef CONFIG_AS_AVX2 #include <linux/linkage.h> .text@@ -749,5 +748,3 @@ PSHUFFLE_BYTE_FLIP_MASK: MASK_YMM_LO: .octa 0x00000000000000000000000000000000 .octa 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF - -#endifarch/x86/crypto/sha512_ssse3_glue.c +0
-5@@ -218,7 +218,6 @@ static void unregister_sha512_avx(void) ARRAY_SIZE(sha512_avx_algs)); } -#if defined(CONFIG_AS_AVX2 ) asmlinkage void sha512_transform_rorx(struct sha512_state *state,const u8 *data,int blocks);@@ -293,10 +292,6 @@ static void unregister_sha512_avx2(void)crypto_unregister_shashes (sha512_avx2_algs,ARRAY_SIZE (sha512_avx2_algs)); } -#else -static inline int register_sha512_avx2 (void ) {return 0 ; } -static inline void unregister_sha512_avx2 (void ) { } -#endif static int __init sha512_ssse3_mod_init (void ) {crypto/Kconfig +4
-4@@ -267,7 +267,7 @@ config CRYPTO_CURVE25519 config CRYPTO_CURVE25519_X86 tristate "x86_64 accelerated Curve25519 scalar multiplication library" - depends on X86 && 64BIT && AS_ADX + depends on X86 && 64BIT select CRYPTO_LIB_CURVE25519_GENERIC select CRYPTO_ARCH_HAVE_LIB_CURVE25519@@ -465,7 +465,7 @@ config CRYPTO_NHPOLY1305_SSE2 config CRYPTO_NHPOLY1305_AVX2 tristate "NHPoly1305 hash function (x86_64 AVX2 implementation)" - depends on X86 && 64BIT && AS_AVX2 + depends on X86 && 64BIT select CRYPTO_NHPOLY1305 help AVX2 optimized implementation of the hash function used by the@@ -1303,7 +1303,7 @@ config CRYPTO_CAMELLIA_AESNI_AVX_X86_64 config CRYPTO_CAMELLIA_AESNI_AVX2_X86_64 tristate "Camellia cipher algorithm (x86_64/AES-NI/AVX2)" - depends on X86 && 64BIT && AS_AVX2 + depends on X86 && 64BIT depends on CRYPTO select CRYPTO_CAMELLIA_AESNI_AVX_X86_64 help@@ -1573,7 +1573,7 @@ config CRYPTO_SERPENT_AVX_X86_64 config CRYPTO_SERPENT_AVX2_X86_64 tristate "Serpent cipher algorithm (x86_64/AVX2)" - depends on X86 && 64BIT && AS_AVX2 + depends on X86 && 64BIT select CRYPTO_SERPENT_AVX_X86_64 help Serpent cipher algorithm, by Anderson, Biham & Knudsen.lib/raid6/algos.c +0
-6@@ -34,10 +34,8 @@ const struct raid6_calls * const raid6_algos[] = { &raid6_avx512x2, &raid6_avx512x1, #endif -#ifdef CONFIG_AS_AVX2 &raid6_avx2x2, &raid6_avx2x1, -#endif &raid6_sse2x2, &raid6_sse2x1, &raid6_sse1x2,@@ -51,11 +49,9 @@ const struct raid6_calls * const raid6_algos[] = { &raid6_avx512x2, &raid6_avx512x1, #endif -#ifdef CONFIG_AS_AVX2 &raid6_avx2x4, &raid6_avx2x2, &raid6_avx2x1, -#endif &raid6_sse2x4, &raid6_sse2x2, &raid6_sse2x1,@@ -101,9 +97,7 @@ const struct raid6_recov_calls *const raid6_recov_algos[] = {#ifdef CONFIG_AS_AVX512 &raid6_recov_avx512, #endif -#ifdef CONFIG_AS_AVX2 &raid6_recov_avx2, -#endif &raid6_recov_ssse3, #endif #ifdef CONFIG_S390lib/raid6/avx2.c +0
-4@@ -13,8 +13,6 @@* */ -#ifdef CONFIG_AS_AVX2 -#include <linux/raid/pq.h> #include "x86.h" @@ -470,5 +468,3 @@ const struct raid6_calls raid6_avx2x4 = {1 /* Has cache hints */ };#endif - -#endif /* CONFIG_AS_AVX2 */ lib/raid6/recov_avx2.c +0
-6@@ -4,8 +4,6 @@* Author: Jim Kukunas <[email protected]> */ -#ifdef CONFIG_AS_AVX2 -#include <linux/raid/pq.h> #include "x86.h" @@ -313,7 +311,3 @@ const struct raid6_recov_calls raid6_recov_avx2 = {#endif .priority = 2, }; - -#else -#warning "your version of binutils lacks AVX2 support" -#endif lib/raid6/test/Makefile +0
-3@@ -35,9 +35,6 @@ endif ifeq ($(IS_X86),yes) OBJS += mmx.o sse1.o sse2.o avx2.o recov_ssse3.o recov_avx2.o avx512.o recov_avx512.o CFLAGS += -DCONFIG_X86 - CFLAGS += $(shell echo "vpbroadcastb %xmm0, %ymm1" | \ - gcc -c -x assembler - >/dev/null 2>&1 && \ - rm ./-.o && echo -DCONFIG_AS_AVX2=1) CFLAGS += $(shell echo "vpmovm2b %k1, %zmm5" | \ gcc -c -x assembler - >/dev/null 2>&1 && \ rm ./-.o && echo -DCONFIG_AS_AVX512=1)net/netfilter/Makefile +0
-2@@ -83,10 +83,8 @@ nf_tables-objs := nf_tables_core.o nf_tables_api.o nft_chain_filter.o \ nft_set_pipapo.o ifdef CONFIG_X86_64 -ifneq (,$(findstring -DCONFIG_AS_AVX2=1,$(KBUILD_CFLAGS))) nf_tables-objs += nft_set_pipapo_avx2.o endif -endif obj-$(CONFIG_NF_TABLES) += nf_tables.o obj-$(CONFIG_NFT_COMPAT) += nft_compat.onet/netfilter/nf_tables_api.c +1
-1@@ -3290,7 +3290,7 @@ static const struct nft_set_type *nft_set_types[] = { &nft_set_rhash_type, &nft_set_bitmap_type, &nft_set_rbtree_type, -#if defined(CONFIG_X86_64) && defined(CONFIG_AS_AVX2) +#if defined(CONFIG_X86_64) &nft_set_pipapo_avx2_type,#endif &nft_set_pipapo_type,net/netfilter/nft_set_pipapo.c +1
-1@@ -2201,7 +2201,7 @@ const struct nft_set_type nft_set_pipapo_type = { }, }; -#if defined(CONFIG_X86_64 ) && defined(CONFIG_AS_AVX2 ) +#if defined(CONFIG_X86_64 )const struct nft_set_type nft_set_pipapo_avx2_type = { .features =NFT_SET_INTERVAL |NFT_SET_MAP |NFT_SET_OBJECT | NFT_SET_TIMEOUT,net/netfilter/nft_set_pipapo_avx2.h +0
-4@@ -1,7 +1,6 @@/* SPDX-License-Identifier: GPL-2.0-only */ #ifndef _NFT_SET_PIPAPO_AVX2_H -#ifdef CONFIG_AS_AVX2 #include <asm/fpu/xstate.h> #define NFT_PIPAPO_ALIGN (XSAVE_YMM_SIZE / BITS_PER_BYTE)@@ -9,6 +8,4 @@ bool nft_pipapo_avx2_lookup(const struct net *net, const struct nft_set *set, const u32 *key, const struct nft_set_ext **ext);bool nft_pipapo_avx2_estimate (const struct nft_set_desc *desc,u32 features,struct nft_set_estimate *est); -#endif /* CONFIG_AS_AVX2 */ -#endif /* _NFT_SET_PIPAPO_AVX2_H */ --2.26.0
Signature · 2 lines
Thanks, ~Nick Desaulniers
Hi all,
Quoted 12 lines · Ingo Molnar
Quoted 5 lines · Jason A. Donenfeld
Very little has changed from last time, and this whole series still looks good to me. I think I already ack'd most packages, but in case it helps:
Reviewed-by Jason A. Donenfeld <[email protected]>Acked-by Ingo Molnar <[email protected]>
Quoted 8 lines
Quoted 3 lines
Since this touches a lot of stuff, it might be best to get it in as early as possible during the merge window, as I imagine new code being added is going to want to be touching those makefiles too.
I'd argue the opposite: please merge this later in the merge window, to not disrupt the vast body of other stuff that has already been lined up and has been tested, and to give time for these new bits to get tested some more.
I agree.
Quoted 2 lines
Also, please get it into -next ASAP, today would be ideal for test coverage ...
I collected more Reviewed-by and Acked-by,
then pushed this series to
git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
kbuild-asinstr
It will show up in -next soon.
Signature · 2 lines
Best Regards Masahiro Yamada
Quoted 5 lines · Masahiro Yamada
I collected more Reviewed-by and Acked-by,
then pushed this series to
git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
kbuild-asinstr
But not the version of the penultimate patch that Nick ack'd....
Re: [PATCH] x86: update AS_* macros to binutils >=2.23, supporting ADX and AVX2
In reply to Jason A. DonenfeldQuoted 11 lines · Jason A. Donenfeld
Now that the kernel specifies binutils 2.23 as the minimum version, we can remove ifdefs for AVX2 and ADX throughout.
Signed-off-by Jason A. Donenfeld <[email protected]>---
Masahiro - here's an updated patch based on -next, as Nick requested. Whether or not you'd prefer this one to the other one based on Linus' tree is up to you, but here it is in case it's helpful.
Thanks.
When the netfilter PR is pulled,
I will rebase on top of Linus' tree.
net/netfilter/Makefile nf_tables-objs := nf_tables_core.o nf_tables_api.o nft_chain_filter.o \ · line 89
Quoted 12 lines
net/netfilter/Makefile +0
-2@@ -83,10 +83,8 @@ nf_tables-objs := nf_tables_core.o nf_tables_api.o nft_chain_filter.o \ nft_set_pipapo.o ifdef CONFIG_X86_64 -ifneq (,$(findstring -DCONFIG_AS_AVX2=1,$(KBUILD_CFLAGS))) nf_tables-objs += nft_set_pipapo_avx2.o endif -endif
This hunk should go to:
[v2,10/16] x86: probe assembler capabilities via kconfig instead of makefile
I will try my best to remember this.
--
Best Regards
Masahiro Yamada
Quoted 9 lines · Jason A. Donenfeld
Quoted 5 lines · Masahiro Yamada
I collected more Reviewed-by and Acked-by,
then pushed this series to
git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git
kbuild-asinstrBut not the version of the penultimate patch that Nick ack'd....
Dropped Nick's Reviewed-by.
Signature · 2 lines
Best Regards Masahiro Yamada
Re: [PATCH v2 08/16] x86: replace arch macros from compiler with CONFIG_X86_{32,64}
In reply to Masahiro YamadaQuoted 8 lines · Masahiro Yamada
If the intention is to check i386/x86_64 excluding UML, testing CONFIG_X86_{32,64} is simpler.
The reason for checking
__i386__/__x86_64__was perhaps becauselib/raid6/algos.cis built not only for the kernel but also for testing the library code from userspace.
Quoted 6 lines
However,
lib/raid6/test/Makefilepasses -DCONFIG_X86_{32,64} for this case. So, I do not see a reason to not use CONFIG option here.Signed-off-by Masahiro Yamada <[email protected]>---
Sorry, this patch broke UML builds.
https://lkml.org/lkml/2020/3/27/618
I will drop this patch for tomorrow's linux-next.
CONFIG_X86 is not defined for ARCH=um.
So, I still think it is OK to replace one complicated line in lib/raid6/x86.h
#if (defined(__i386__) || defined(__x86_64__)) && !defined(__arch_um__)
with:
#ifdef CONFIG_X86
Also, I slightly prefer
#if defined(CONFIG_X86_32) && !defined(CONFIG_UML)
to
#if defined(__i386__) && !defined(__arch_um__)
But, we do not need to rush this now.
This patch does not affect the rest of this series.
Thanks.
Quoted 44 lines
Changes in v2: None
kernel/signal.c | 2 +- lib/raid6/algos.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)kernel/signal.c +1
-1@@ -1246,7 +1246,7 @@ static void print_fatal_signal(int signr)struct pt_regs *regs =signal_pt_regs ();pr_info ("potentially unexpected fatal signal %d.\n" , signr); -#if defined(__i386__) && !defined(__arch_um__) +#ifdef CONFIG_X86_32 pr_info ("code at %08lx: " , regs->ip); {int i;lib/raid6/algos.c +2
-3@@ -29,7 +29,7 @@ struct raid6_calls raid6_call;EXPORT_SYMBOL_GPL (raid6_call); const struct raid6_calls * const raid6_algos[] = { -#if defined(__i386__) && !defined(__arch_um__) +#ifdef CONFIG_X86_32 #ifdef CONFIG_AS_AVX512 &raid6_avx512x2, &raid6_avx512x1,@@ -45,7 +45,7 @@ const struct raid6_calls * const raid6_algos[] = { &raid6_mmxx2, &raid6_mmxx1,#endif -#if defined(__x86_64__) && !defined(__arch_um__) +#ifdef CONFIG_X86_64 #ifdef CONFIG_AS_AVX512 &raid6_avx512x4, &raid6_avx512x2, --2.17.1
Signature · 2 lines
Best Regards Masahiro Yamada
Re: [PATCH v2 08/16] x86: replace arch macros from compiler with CONFIG_X86_{32,64}
Quoted 8 lines · Masahiro Yamada
Also, I slightly prefer
#if defined(CONFIG_X86_32) && !defined(CONFIG_UML)to
#if defined(__i386__) && !defined(__arch_um__)
I would much prefer that too.
At some point it'd be nice to see this whole mess cleaned up with UML. I've seen this a few places -- UML thinks its x86, except where it's not, and then there are problems. For example, a lot of SIMD related code has this.
Jason