Skip to contentSkip to conversation

crypto: serpent - add x86_64/avx assembler implementation

PATCH In review Johannes Goetzfried 4 participants 23 messages May 27, 2012 – Jun 20, 2012

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 crypto: serpent - add x86_64/avx assembler implementation

This patch adds a x86_64/avx assembler implementation of the Serpent block cipher. The implementation is very similar to the sse2 implementation and processes eight blocks in parallel. Because of the new non-destructive three operand syntax all move-instructions can be removed and therefore a little performance increase is provided.

Patch has been tested with tcrypt and automated filesystem tests.

Tcrypt benchmark results:

Intel Core i5-2500 CPU (fam:6, model:42, step:7)

serpent-avx-x86_64 vs. serpent-sse2-x86_64
128bit key: (lrw:256bit) (xts:256bit)

size    ecb-enc ecb-dec cbc-enc cbc-dec ctr-enc ctr-dec lrw-enc lrw-dec xts-enc xts-dec
16B     1.03x   1.01x	1.01x   1.01x   1.00x   1.00x   1.00x   1.00x   1.00x   1.01x
64B     1.00x   1.00x   1.00x   1.00x   1.00x   0.99x   1.00x   1.01x   1.00x   1.00x
256B    1.05x   1.03x   1.00x   1.02x   1.05x   1.06x   1.05x   1.02x   1.05x   1.02x
1024B   1.05x   1.02x   1.00x   1.02x   1.05x   1.06x   1.05x   1.03x   1.05x   1.02x
8192B   1.05x   1.02x   1.00x   1.02x   1.06x   1.06x   1.04x   1.03x   1.04x   1.02x

256bit key: (lrw:384bit) (xts:512bit)

size    ecb-enc ecb-dec cbc-enc cbc-dec ctr-enc ctr-dec lrw-enc lrw-dec xts-enc xts-dec
16B     1.01x   1.00x   1.01x   1.01x   1.00x   1.00x   0.99x   1.03x   1.01x   1.01x
64B     1.00x   1.00x   1.00x   1.00x   1.00x   1.00x   1.00x   1.01x   1.00x   1.02x
256B    1.05x   1.02x   1.00x   1.02x   1.05x   1.02x   1.04x   1.05x   1.05x   1.02x
1024B   1.06x   1.02x   1.00x   1.02x   1.07x   1.06x   1.05x   1.04x   1.05x   1.02x
8192B   1.05x   1.02x   1.00x   1.02x   1.06x   1.06x   1.04x   1.05x   1.05x   1.02x

serpent-avx-x86_64 vs aes-asm (8kB block):

         128bit  256bit
ecb-enc  1.26x   1.73x
ecb-dec  1.20x   1.64x
cbc-enc  0.33x   0.45x
cbc-dec  1.24x   1.67x
ctr-enc  1.32x   1.76x
ctr-dec  1.32x   1.76x
lrw-enc  1.20x   1.60x
lrw-dec  1.15x   1.54x
xts-enc  1.22x   1.64x
xts-dec  1.17x   1.57x
---
 arch/x86/crypto/Makefile                    |    2 +
 arch/x86/crypto/serpent-avx-x86_64-asm_64.S |  704 ++++++++++++++++++++
 arch/x86/crypto/serpent_avx_glue.c          |  949 +++++++++++++++++++++++++++
 crypto/Kconfig                              |   20 +
 crypto/testmgr.c                            |   60 ++
 5 files changed, 1735 insertions(+), 0 deletions(-)
 create mode 100644 arch/x86/crypto/serpent-avx-x86_64-asm_64.S
 create mode 100644 arch/x86/crypto/serpent_avx_glue.c
arch/x86/crypto/Makefile +2 -0
index e191ac0..8ef7ba5 100644
@@ -14,6 +14,7 @@ obj-$(CONFIG_CRYPTO_TWOFISH_X86_64) += twofish-x86_64.o
 obj-$(CONFIG_CRYPTO_TWOFISH_X86_64_3WAY) += twofish-x86_64-3way.o
 obj-$(CONFIG_CRYPTO_SALSA20_X86_64) += salsa20-x86_64.o
 obj-$(CONFIG_CRYPTO_SERPENT_SSE2_X86_64) += serpent-sse2-x86_64.o
+obj-$(CONFIG_CRYPTO_SERPENT_AVX_X86_64) += serpent-avx-x86_64.o
 obj-$(CONFIG_CRYPTO_AES_NI_INTEL) += aesni-intel.o
 obj-$(CONFIG_CRYPTO_GHASH_CLMUL_NI_INTEL) += ghash-clmulni-intel.o
 
@@ -32,6 +33,7 @@ twofish-x86_64-y := twofish-x86_64-asm_64.o twofish_glue.o
 twofish-x86_64-3way-y := twofish-x86_64-asm_64-3way.o twofish_glue_3way.o
 salsa20-x86_64-y := salsa20-x86_64-asm_64.o salsa20_glue.o
 serpent-sse2-x86_64-y := serpent-sse2-x86_64-asm_64.o serpent_sse2_glue.o
+serpent-avx-x86_64-y := serpent-avx-x86_64-asm_64.o serpent_avx_glue.o
 
 aesni-intel-y := aesni-intel_asm.o aesni-intel_glue.o fpu.o
 
arch/x86/crypto/serpent-avx-x86_64-asm_64.S new file +704 -0

704 lines not on this page · raw message

arch/x86/crypto/serpent_avx_glue.c new file +949 -0

949 lines not on this page · raw message

crypto/Kconfig +20 -0
index 8e84225..3f18b97 100644
@@ -821,6 +821,26 @@ config CRYPTO_SERPENT_SSE2_586
 	  See also:
 	  <http://www.cl.cam.ac.uk/~rja14/serpent.html>
 
+config CRYPTO_SERPENT_AVX_X86_64
+	tristate "Serpent cipher algorithm (x86_64/AVX)"
+	depends on X86 && 64BIT
+	select CRYPTO_ALGAPI
+	select CRYPTO_CRYPTD
+	select CRYPTO_SERPENT
+	select CRYPTO_LRW
+	select CRYPTO_XTS
+	help
+	  Serpent cipher algorithm, by Anderson, Biham & Knudsen.
+
+	  Keys are allowed to be from 0 to 256 bits in length, in steps
+	  of 8 bits.
+
+	  This module provides the Serpent cipher algorithm that processes
+	  eight blocks parallel using the AVX instruction set.
+
+	  See also:
+	  <http://www.cl.cam.ac.uk/~rja14/serpent.html>
+
 config CRYPTO_TEA
 	tristate "TEA, XTEA and XETA cipher algorithms"
 	select CRYPTO_ALGAPI
crypto/testmgr.c +60 -0

84 lines not on this page · raw message

Signature · 1 line
1.7.2.5
#1 lore raw

Quoting Johannes Goetzfried <[email protected]>:

Quoted 5 lines

This patch adds a x86_64/avx assembler implementation of the Serpent block cipher. The implementation is very similar to the sse2 implementation and processes eight blocks in parallel. Because of the new non-destructive three operand syntax all move-instructions can be removed and therefore a little performance increase is provided.

/* /me adds CPU with AVX to wishlist. */

<snip>

arch/x86/crypto/serpent_avx_glue.c line 7

Quoted 13 lines
arch/x86/crypto/serpent_avx_glue.c new file +7 -0
new file mode 100644
index 0000000..85ef6e7
@@ -0,0 +1,949 @@
+/*
+ * Glue Code for AVX assembler versions of Serpent Cipher
+ *
+ * Copyright (C) 2012 Johannes Goetzfried
+ *     <[email protected]>
+ *
+ * Glue code based on twofish_avx_glue.c by:

Should be serpent_sse2_glue.c?

Quoted 2 lines
arch/x86/crypto/serpent_avx_glue.c new file +2 -0
+ *  Copyright (C) 2011 Jussi Kivilinna <[email protected]>
+ *

<snip>

arch/x86/crypto/serpent_avx_glue.c line 805

Quoted 4 lines
arch/x86/crypto/serpent_avx_glue.c new file +4 -0
+}, {
+	.cra_name		= "ecb(serpent)",
+	.cra_driver_name	= "ecb-serpent-avx",
+	.cra_priority		= 400,

serpent_sse2_glue.c has priority 400 too, so you should increase priority here to 500.

...

Actually about duplicating glue code.. is it really needed? On x86_64, both avx and sse2 versions process 8-blocks parallel and therefore glue code could be easily shared (as is done in SHA1 SSSE3/AVX).

-Jussi

#2 lore raw

Hello,

Quoted 1 line

Should be serpent_sse2_glue.c?

Yeah, same error as in the other patch :-)

Quoted 7 lines

arch/x86/crypto/serpent_avx_glue.c line 805

Quoted 4 lines
arch/x86/crypto/serpent_avx_glue.c new file +4 -0
+}, {
+	.cra_name		= "ecb(serpent)",
+	.cra_driver_name	= "ecb-serpent-avx",
+	.cra_priority		= 400,

serpent_sse2_glue.c has priority 400 too, so you should increase priority here to 500.

You are right. Actually it is useless to load both modules at the same time and so this shouldn't be a problem. But better be safe. I'll resend this patch as well.

Quoted 4 lines

Actually about duplicating glue code.. is it really needed? On x86_64, both avx and sse2 versions process 8-blocks parallel and therefore glue code could be easily shared (as is done in SHA1 SSSE3/AVX).

I thought about doing it the way as it is done in the SHA1-Module. But I don't think that's a good idea, because then I had to compile both implementations in the same module and decide at runtime which one to use depending on the processor capabilities. This would result in an increasing object size and on a specific processor only one implementation is used. I think it's better to decide statically at compile-time which implementation to use.
I agree, the code duplication is ugly, and have a different approach. All the glue code shared by both implementations can be moved to a serpent_common_glue.c file and only the module init stuff needs to be seperate. This should reduce the code size and keeps the possibility of compiling only one implementation. I will send a second patch which should do the job.

- Johannes

#3 lore raw

Quoting Johannes Goetzfried <[email protected]>:

Quoted 14 lines

Hello,

Quoted 1 line

Should be serpent_sse2_glue.c?

Yeah, same error as in the other patch :-)

Quoted 7 lines

arch/x86/crypto/serpent_avx_glue.c line 805

Quoted 4 lines
arch/x86/crypto/serpent_avx_glue.c new file +4 -0
+}, {
+	.cra_name		= "ecb(serpent)",
+	.cra_driver_name	= "ecb-serpent-avx",
+	.cra_priority		= 400,

serpent_sse2_glue.c has priority 400 too, so you should increase priority here to 500.

Quoted 4 lines

You are right. Actually it is useless to load both modules at the same time and so this shouldn't be a problem. But better be safe. I'll resend this patch as well.

Thing is that since both have MODULE_ALIAS(serpent), they both get loaded with 'modprobe serpent' etc.

Quoted 5 lines
Quoted 4 lines

Actually about duplicating glue code.. is it really needed? On x86_64, both avx and sse2 versions process 8-blocks parallel and therefore glue code could be easily shared (as is done in SHA1 SSSE3/AVX).

Quoted 12 lines

I thought about doing it the way as it is done in the SHA1-Module. But I don't think that's a good idea, because then I had to compile both implementations in the same module and decide at runtime which one to use depending on the processor capabilities. This would result in an increasing object size and on a specific processor only one implementation is used. I think it's better to decide statically at compile-time which implementation to use.
I agree, the code duplication is ugly, and have a different approach. All the glue code shared by both implementations can be moved to a serpent_common_glue.c file and only the module init stuff needs to be seperate. This should reduce the code size and keeps the possibility of compiling only one implementation. I will send a second patch which should do the job.

Looks better.

Maybe later when I have time, I'll check if all glue code for 16-byte block-ciphers found in arch/x86/crypto could be merged in to single helper module.

-Jussi

#4 lore raw
In reply to Johannes Goetzfried

Johannes Goetzfried

Quoted 20 lines · <[email protected]> writes:
arch/x86/crypto/serpent_avx_glue.c new file +20 -0
+
+static int __init serpent_init(void)
+{
+	u64 xcr0;
+
+	if (!cpu_has_avx || !cpu_has_osxsave) {
+		printk(KERN_INFO "AVX instructions are not detected.\n");
+		return -ENODEV;
+	}
+
+	xcr0 = xgetbv(XCR_XFEATURE_ENABLED_MASK);
+	if ((xcr0 & (XSTATE_SSE | XSTATE_YMM)) != (XSTATE_SSE | XSTATE_YMM)) {
+		printk(KERN_INFO "AVX detected but unusable.\n");
+		return -ENODEV;
+	}
+
+	return crypto_register_algs(serpent_algs, ARRAY_SIZE(serpent_algs));
+}
+
+static void __exit serpent_exit(void)

arch/x86/crypto/serpent_avx_glue.c line 949

Quoted 10 lines
arch/x86/crypto/serpent_avx_glue.c new file +10 -0
+{
+	crypto_unregister_algs(serpent_algs, ARRAY_SIZE(serpent_algs));
+}
+
+module_init(serpent_init);
+module_exit(serpent_exit);
+
+MODULE_DESCRIPTION("Serpent Cipher Algorithm, AVX optimized");
+MODULE_LICENSE("GPL");
+MODULE_ALIAS("serpent");

The driver needs CPUID annotations now (since 3.3), so that it can be autoloaded. Something like:

#include <asm/cpu_device_id.h>

static const struct x86_cpu_id avx_cpu_id[] = {
        X86_FEATURE_MATCH(X86_FEATURE_AVX),
        {}
};

MODULE_DEVICE_TABLE(x86cpu, avx_cpu_id);

Replace the manual check in init with

            if (!x86_match_cpu(avx_cpu_id))
                return -ENODEV;

Also drivers should never print anything when they cannot find hardware. Remove that printk.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only
#5 lore raw

Quoting Andi Kleen <[email protected]>:

Quoted 4 lines

The driver needs CPUID annotations now (since 3.3), so that it can be autoloaded. Something like:

Is it really good idea to autoload crypto modules? Currently loading different cipher implementations is handled via module-aliases and cipher priorities. With autoloading by CPUID, crypto-modules would be loaded and be mostly unused. For example, serpent-sse2 would be autoloaded on all x86-64 kernels (well, on those that have it compiled as module => probably most of distros).

IMHO these should be loaded only when needed, as is done with generic i586/x86-64 assembler cipher implementations.

-Jussi

#6 lore raw
Quoted 14 lines · Jussi Kivilinna

Quoting Andi Kleen <[email protected]>:

Quoted 4 lines

The driver needs CPUID annotations now (since 3.3), so that it can be autoloaded. Something like:

Is it really good idea to autoload crypto modules? Currently loading different cipher implementations is handled via module-aliases and cipher priorities. With autoloading by CPUID, crypto-modules would be loaded and be mostly unused. For example, serpent-sse2 would be autoloaded on all x86-64 kernels (well, on those that have it compiled as module => probably most of distros).

Quoted 2 lines

IMHO these should be loaded only when needed, as is done with generic i586/x86-64 assembler cipher implementations.

I agree with that. Currently when I boot my PC with a new 3.4 kernel all the ciphers from the intel-aesni module get loaded whether I need them or not. As Jussi stated most people using distros probably won't need the serpent-avx-x86_64 module get loaded automatically, so it's probably better to leave it that way.

#20 lore raw
In reply to Andi Kleen
Quoted 2 lines · Andi Kleen

Also drivers should never print anything when they cannot find hardware. Remove that printk.

I tried to be consistent with the existing ciphers in arch/x86/crypto. In serpent_sse2_glue.c and sha1_ssse3_glue.c it is done exactly that way, so if it will be decided to remove the printk in this patch it probably should also be removed in the other modules.

- Johannes

#7 lore raw
In reply to Johannes Goetzfried
Quoted 5 lines

I agree with that. Currently when I boot my PC with a new 3.4 kernel all the ciphers from the intel-aesni module get loaded whether I need them or not. As Jussi stated most people using distros probably won't need the serpent-avx-x86_64 module get loaded automatically, so it's probably better to leave it that way.

That means you got a 50% chance to use the wrong serpent.

This was a continuous problem with AESNI and the accelerated CRC, that is why the cpuid probing was implemented.

Without some form of auto probing you may as well not bother with the optimization.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.
#21 lore raw
In reply to Johannes Goetzfried
Quoted 8 lines · Johannes Goetzfried
Quoted 2 lines · Andi Kleen

Also drivers should never print anything when they cannot find hardware. Remove that printk.

I tried to be consistent with the existing ciphers in arch/x86/crypto. In serpent_sse2_glue.c and sha1_ssse3_glue.c it is done exactly that way, so if it will be decided to remove the printk in this patch it probably should also be removed in the other modules.

They are wrong and need to be fixed.

Drivers can be loaded for all kinds of reasons, but they are not supposed to spam your kernel log when they do nothing.

In fact I fixed all the existing ones with the cpuid changes, unfortunately somehow snuck in wrong code :-(

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.
#8 lore raw
In reply to Andi Kleen
Quoted 6 lines · Andi Kleen
Quoted 5 lines

I agree with that. Currently when I boot my PC with a new 3.4 kernel all the ciphers from the intel-aesni module get loaded whether I need them or not. As Jussi stated most people using distros probably won't need the serpent-avx-x86_64 module get loaded automatically, so it's probably better to leave it that way.

Quoted 7 lines

That means you got a 50% chance to use the wrong serpent.

This was a continuous problem with AESNI and the accelerated CRC, that is why the cpuid probing was implemented.

Without some form of auto probing you may as well not bother with the optimization.

Seems like I missunderstood the priority concept of the crypto API. I thought both algorithms have module alias "serpent", so they get both loaded when they are needed, for example by the device mapper, and after that the algorithm with higher priority is selected. Where do you get the 50% chance of using the wrong serpent?

If there is for some reason no possibility to make it work this way then cpuid probing might be the better choice.

- Johannes

#22 lore raw
In reply to Andi Kleen
Quoted 6 lines · Andi Kleen
Quoted 4 lines

I tried to be consistent with the existing ciphers in arch/x86/crypto. In serpent_sse2_glue.c and sha1_ssse3_glue.c it is done exactly that way, so if it will be decided to remove the printk in this patch it probably should also be removed in the other modules.

Quoted 7 lines

They are wrong and need to be fixed.

Drivers can be loaded for all kinds of reasons, but they are not supposed to spam your kernel log when they do nothing.

In fact I fixed all the existing ones with the cpuid changes, unfortunately somehow snuck in wrong code :-(

Hm, ok, removing the printk stuff is no big deal. I would fix this together with the cpuid probing, when I'm sure that this is the preferred "new" way :-)

- Johannes

#9 lore raw
In reply to Andi Kleen

Quoting Andi Kleen <[email protected]>:

Quoted 8 lines
Quoted 5 lines

I agree with that. Currently when I boot my PC with a new 3.4 kernel all the ciphers from the intel-aesni module get loaded whether I need them or not. As Jussi stated most people using distros probably won't need the serpent-avx-x86_64 module get loaded automatically, so it's probably better to leave it that way.

That means you got a 50% chance to use the wrong serpent.

Quoted 7 lines

This was a continuous problem with AESNI and the accelerated CRC, that is why the cpuid probing was implemented.

Without some form of auto probing you may as well not bother with the optimization.

-Andi

Hm.. should not the i586/x86_64 assembler implementations of AES etc be autoloaded then too?

-Jussi

Quoted 9 lines

--
[email protected] -- Speaking for myself only.
--
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#15 lore raw
In reply to Andi Kleen
Quoted 7 lines · Andi Kleen
Quoted 5 lines

I agree with that. Currently when I boot my PC with a new 3.4 kernel all the ciphers from the intel-aesni module get loaded whether I need them or not. As Jussi stated most people using distros probably won't need the serpent-avx-x86_64 module get loaded automatically, so it's probably better to leave it that way.

That means you got a 50% chance to use the wrong serpent.

You should always get the best one available. For example, when you request for "aes" all implementations of it will be loaded by modprobe.

Quoted 2 lines

This was a continuous problem with AESNI and the accelerated CRC, that is why the cpuid probing was implemented.

The only case where it doesn't work is if you have some variants built-in and the faster ones built as modules. But then the answer is to either build all of them as modules or all of them built-in so that priority-based selection can work.

Can you provide an example where it doesn't work as intended?

What we could do is to use the cpuid-based probing when an algorithm is needed to selectively load the relevant implementations instead of all of them. However, for most algorithms it won't make that big a difference since all the available ones will be loaded anyway.

Cheers,

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#16 lore raw
Quoted 1 line

Can you provide an example where it doesn't work as intended?

It works for the current algorithms except serpent.

Quoted 4 lines

What we could do is to use the cpuid-based probing when an algorithm is needed to selectively load the relevant implementations instead of all of them. However, for most algorithms it won't make that big a difference since all the available ones will be loaded anyway.

Yes cpuid probing does that.

The only problem is just that serpent does not use cpuid probing. -Andi

Signature · 1 line
[email protected] -- Speaking for myself only.
#17 lore raw
Quoted 7 lines · Andi Kleen
Quoted 5 lines

What we could do is to use the cpuid-based probing when an algorithm is needed to selectively load the relevant implementations instead of all of them. However, for most algorithms it won't make that big a difference since all the available ones will be loaded anyway.

Yes cpuid probing does that.

I was under the impression that using cpuid meant that the algorithm would be loaded regardless of whether it has been requested, as long as the cpuid matches.

If that's not the case then there is certainly no reason to not use cpuid-based loading.

Thanks,

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#18 lore raw
Quoted 7 lines · Herbert Xu
Quoted 6 lines · Andi Kleen
Quoted 5 lines

What we could do is to use the cpuid-based probing when an algorithm is needed to selectively load the relevant implementations instead of all of them. However, for most algorithms it won't make that big a difference since all the available ones will be loaded anyway.

Quoted 5 lines
Quoted 1 line

Yes cpuid probing does that.

I was under the impression that using cpuid meant that the algorithm would be loaded regardless of whether it has been requested, as long as the cpuid matches.

It loads it always correct.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.
#19 lore raw
Quoted 8 lines · Andi Kleen
Quoted 7 lines · Herbert Xu
Quoted 6 lines · Andi Kleen
Quoted 5 lines

What we could do is to use the cpuid-based probing when an algorithm is needed to selectively load the relevant implementations instead of all of them. However, for most algorithms it won't make that big a difference since all the available ones will be loaded anyway.

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

Yes cpuid probing does that.

I was under the impression that using cpuid meant that the algorithm would be loaded regardless of whether it has been requested, as long as the cpuid matches.

It loads it always correct.

You mean with cpuid it'll always load the algorithm if it is available even if noone is using it? Then it's probably not a good fit for serpent.

Cheers,

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#10 lore raw

PATCH 1/1 crypto: aes - make assembler implementation default for i386 and x86-64

In reply to Jussi Kivilinna

AES is often built-in and therefore implementations other than aes-generic don't get autoloaded by module-aliases. This is solved for aes-ni by cpuid autoloading, but for x86-64/i586 assembler implementation cpuid matching does not make much sense (aes-generic built-in but overridden by x86-64/i586 assembler implementation module). Instead change Kconfig so that assembler implementation is default on x86-64 and i386.

This patch also splits common parts exported to assembler implementations from aes-generic to new module aes-common.

Patch tested on x86-64, compile tested on i386 and arm.

---
 crypto/Kconfig       |   16 +
 crypto/Makefile      |    3 
 crypto/aes_common.c  | 1301 ++++++++++++++++++++++++++++++++++++++++++++++++++
 crypto/aes_generic.c | 1235 -----------------------------------------------
 4 files changed, 1317 insertions(+), 1238 deletions(-)
 create mode 100644 crypto/aes_common.c
crypto/Kconfig +14 -2
index 21ff9d0..f17c66a 100644
@@ -483,8 +483,20 @@ config CRYPTO_GHASH_CLMUL_NI_INTEL
 comment "Ciphers"
 
 config CRYPTO_AES
+	tristate
+	select CRYPTO_ALGAPI
+	select CRYPTO_AES_GENERIC if !X86
+	select CRYPTO_AES_586 if X86 && !64BIT
+	select CRYPTO_AES_X86_64 if X86 && 64BIT
+
+config CRYPTO_AES_COMMON
+	tristate
+	select CRYPTO_ALGAPI
+
+config CRYPTO_AES_GENERIC
 	tristate "AES cipher algorithms"
 	select CRYPTO_ALGAPI
+	select CRYPTO_AES_COMMON
 	help
 	  AES cipher algorithms (FIPS-197). AES uses the Rijndael
 	  algorithm.
@@ -506,7 +518,7 @@ config CRYPTO_AES_586
 	tristate "AES cipher algorithms (i586)"
 	depends on (X86 || UML_X86) && !64BIT
 	select CRYPTO_ALGAPI
-	select CRYPTO_AES
+	select CRYPTO_AES_COMMON
 	help
 	  AES cipher algorithms (FIPS-197). AES uses the Rijndael
 	  algorithm.
@@ -528,7 +540,7 @@ config CRYPTO_AES_X86_64
 	tristate "AES cipher algorithms (x86_64)"
 	depends on (X86 || UML_X86) && 64BIT
 	select CRYPTO_ALGAPI
-	select CRYPTO_AES
+	select CRYPTO_AES_COMMON
 	help
 	  AES cipher algorithms (FIPS-197). AES uses the Rijndael
 	  algorithm.
crypto/Makefile +2 -1
index 30f33d6..2786b76 100644
@@ -66,7 +66,8 @@ obj-$(CONFIG_CRYPTO_BLOWFISH_COMMON) += blowfish_common.o
 obj-$(CONFIG_CRYPTO_TWOFISH) += twofish_generic.o
 obj-$(CONFIG_CRYPTO_TWOFISH_COMMON) += twofish_common.o
 obj-$(CONFIG_CRYPTO_SERPENT) += serpent_generic.o
-obj-$(CONFIG_CRYPTO_AES) += aes_generic.o
+obj-$(CONFIG_CRYPTO_AES_COMMON) += aes_common.o
+obj-$(CONFIG_CRYPTO_AES_GENERIC) += aes_generic.o
 obj-$(CONFIG_CRYPTO_CAMELLIA) += camellia_generic.o
 obj-$(CONFIG_CRYPTO_CAST5) += cast5.o
 obj-$(CONFIG_CRYPTO_CAST6) += cast6.o
crypto/aes_common.c new file +1301 -0

1,301 lines not on this page · raw message

crypto/aes_generic.c +0 -1235

1,241 lines not on this page · raw message

#11 lore raw

Re: [PATCH] crypto: aes - make assembler implementation default for i386 and x86-64

Quoted 7 lines · Jussi Kivilinna

AES is often built-in and therefore implementations other than aes-generic don't get autoloaded by module-aliases. This is solved for aes-ni by cpuid autoloading, but for x86-64/i586 assembler implementation cpuid matching does not make much sense (aes-generic built-in but overridden by x86-64/i586 assembler implementation module). Instead change Kconfig so that assembler implementation is default on x86-64 and i386.

Quoted 6 lines

This patch also splits common parts exported to assembler implementations from aes-generic to new module aes-common.

Patch tested on x86-64, compile tested on i386 and arm.

I like the idea but I'm not sure about where we draw the line. E.g., why not enable padlock, or for that matter every single hardware implementation of AES?

This can get unwieldy very quickly.

Cheers,

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#12 lore raw

Re: [PATCH] crypto: aes - make assembler implementation default for i386 and x86-64

Quoting Herbert Xu <[email protected]>:

Quoted 13 lines · Jussi Kivilinna
Quoted 12 lines

AES is often built-in and therefore implementations other than aes-generic don't get autoloaded by module-aliases. This is solved for aes-ni by cpuid autoloading, but for x86-64/i586 assembler implementation cpuid matching does not make much sense (aes-generic built-in but overridden by x86-64/i586 assembler implementation module). Instead change Kconfig so that assembler implementation is default on x86-64 and i386.

This patch also splits common parts exported to assembler implementations from aes-generic to new module aes-common.

Patch tested on x86-64, compile tested on i386 and arm.

Quoted 8 lines
Quoted 1 line

I like the idea but I'm not sure about where we draw the line. E.g., why not enable padlock, or for that matter every single hardware implementation of AES?

This can get unwieldy very quickly.

Well, how about letting arch specific assembler implementations replace aes-generic completely.. in this case add "depends on !X86" on CRYPTO_AES_GENERIC. Hardware modules get autoloaded (cpuid/pci/platform/etc) but generic assembler implementation might as well be replacement as it cannot autoload depending on hardware support (or it can, but will be always loaded).

-Jussi

#13 lore raw

Re: [PATCH] crypto: aes - make assembler implementation default for i386 and x86-64

Quoted 10 lines · Jussi Kivilinna

Well, how about letting arch specific assembler implementations replace aes-generic
completely.. in this case add "depends on !X86" on CRYPTO_AES_GENERIC. Hardware
modules get autoloaded (cpuid/pci/platform/etc) but generic assembler implementation
might as well be replacement as it cannot autoload depending on hardware support (or
it can, but will be always loaded).

Perhaps we can just force a modprobe of a given algorithm at least once for each boot (when requested for the first time). I think that should be easy to implement and should solve your problem scenario.

What do you think?

Thanks,

Signature · 3 lines
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
#14 lore raw

Re: [PATCH] crypto: aes - make assembler implementation default for i386 and x86-64

Quoting Herbert Xu <[email protected]>:

Quoted 12 lines · Jussi Kivilinna
Quoted 10 lines

Well, how about letting arch specific assembler implementations replace aes-generic
completely.. in this case add "depends on !X86" on CRYPTO_AES_GENERIC. Hardware
modules get autoloaded (cpuid/pci/platform/etc) but generic assembler implementation
might as well be replacement as it cannot autoload depending on hardware support (or
it can, but will be always loaded).

Quoted 6 lines

Perhaps we can just force a modprobe of a given algorithm at least once for each boot (when requested for the first time). I think that should be easy to implement and should solve your problem scenario.

What do you think?

Well, that could work.

-Jussi

Quoted 8 lines

Thanks,
--
Email: Herbert Xu <[email protected]> Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt