Parallel IPsec
Subsystems (5)
Steffen Klassert
maintainerv2 cover Parallel IPsec
This patchset adds the 'pcrypt' parallel crypto template. With this template it is possible to process the crypto requests of a transform in parallel without getting request reorder. This is in particular interesting for IPsec.
The parallel crypto template is based on a generic parallelization/serialization method. This method uses the remote softirq invocation infrastructure for parallelization and serialization. With this method data objects can be processed in parallel, starting at some given point.
After doing some expensive operations in parallel, it is possible to serialize again. The parallelized data objects return after serialization in the order as they were before the parallelization. In the case of IPsec, this makes it possible to run the expensive parts in parallel without getting packet reordering.
Changes from v1:
- cpu_chainiv is dropped, pcrypt uses eseqiv as it's IV generator now.
- Add a xfrm netlink message to be able to choose for pcrypt from userspace.
- Use pcrypt just if it is selected from userspace.
- Patchset applies to 2.6.30-rc3
Steffen
Steffen Klassert
maintainerRFC PATCH v2 1/4 padata: generic interface for parallel processing
This patch introduces an interface to process data objects in parallel. On request it is possible to serialize again. The parallelized objects return after serialization in the same order as they were before the parallelization.
--- include/linux/interrupt.h | 3 +- include/linux/padata.h | 116 +++++++++++ kernel/Makefile | 2 +- kernel/padata.c | 490 +++++++++++++++++++++++++++++++++++++++++++++ kernel/softirq.c | 2 +- 5 files changed, 610 insertions(+), 3 deletions(-) create mode 100644 include/linux/padata.h create mode 100644 kernel/padata.c
include/linux/interrupt.h +2 -1
@@ -338,7 +338,8 @@ enumTASKLET_SOFTIRQ ,SCHED_SOFTIRQ ,HRTIMER_SOFTIRQ , -RCU_SOFTIRQ ,/* Preferable RCU should always be the last softirq */ +PADATA_SOFTIRQ , +RCU_SOFTIRQ ,/* Preferable RCU should always be the last softirq */ NR_SOFTIRQS };
include/linux/padata.h new file +116 -0
116 lines not on this page · raw message
kernel/Makefile +1 -1
@@ -40,7 +40,7 @@ obj-$(CONFIG_RT_MUTEXES) += rtmutex.o obj-$(CONFIG_DEBUG_RT_MUTEXES) += rtmutex-debug.o obj-$(CONFIG_RT_MUTEX_TESTER) += rtmutex-tester.o obj-$(CONFIG_GENERIC_ISA_DMA) += dma.o -obj-$(CONFIG_USE_GENERIC_SMP_HELPERS) += smp.o +obj-$(CONFIG_USE_GENERIC_SMP_HELPERS) += smp.o padata.o ifneq ($(CONFIG_SMP),y) obj-y += up.o endif
kernel/padata.c new file +490 -0
490 lines not on this page · raw message
kernel/softirq.c +1 -1
@@ -56,7 +56,7 @@ static DEFINE_PER_CPU(struct task_struct *, ksoftirqd);char *softirq_to_name[NR_SOFTIRQS ] = {"HI" ,"TIMER" ,"NET_TX" ,"NET_RX" ,"BLOCK" , -"TASKLET" ,"SCHED" ,"HRTIMER" ,"RCU" +"TASKLET" ,"SCHED" ,"HRTIMER" ,"PADATA" ,"RCU" };/*
Signature · 1 line
1.5.4.2
Herbert Xu
maintainerRe: [RFC] [PATCH v2 1/4] padata: generic interface for parallel processing
Quoted 6 lines · Steffen Klassert
This patch introduces an interface to process data objects in parallel. On request it is possible to serialize again. The parallelized objects return after serialization in the same order as they were before the parallelization.
Signed-off-by Steffen Klassert <[email protected]>
This part is ready to go in as far as I'm concerned.
Cheers,
Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txtSteffen Klassert
maintainerRFC PATCH v2 2/4 pcrypt: Add pcrypt crypto parallelization engine
In reply to Steffen KlassertThis patch adds a parallel crypto template that takes a crypto algorithm and converts it to process the crypto transforms in parallel. For the moment only aead is supported.
--- crypto/Kconfig | 13 ++ crypto/Makefile | 2 + crypto/pcrypt.c | 415 +++++++++++++++++++++++++++++++++++++++++++++ crypto/pcrypt_core.c | 106 ++++++++++++ include/crypto/pcrypt.h | 56 ++++++ include/linux/crypto.h | 2 + include/linux/interrupt.h | 2 + kernel/softirq.c | 3 +- 8 files changed, 598 insertions(+), 1 deletions(-) create mode 100644 crypto/pcrypt.c create mode 100644 crypto/pcrypt_core.c create mode 100644 include/crypto/pcrypt.h
crypto/Kconfig +13 -0
@@ -112,6 +112,19 @@ config CRYPTO_NULL help These are 'Null' algorithms, used by IPsec, which do nothing. +config CRYPTO_PCRYPT_CORE + bool + select CRYPTO_AEAD + +config CRYPTO_PCRYPT + tristate "Parallel crypto engine (EXPERIMENTAL)" + depends on USE_GENERIC_SMP_HELPERS && EXPERIMENTAL + select CRYPTO_MANAGER + select CRYPTO_PCRYPT_CORE + help + This converts an arbitrary crypto algorithm into a parallel + algorithm that is executed in a softirq. + config CRYPTO_WORKQUEUE tristate
crypto/Makefile +2 -0
@@ -56,6 +56,8 @@ obj-$(CONFIG_CRYPTO_XTS) += xts.o obj-$(CONFIG_CRYPTO_CTR) += ctr.o obj-$(CONFIG_CRYPTO_GCM) += gcm.o obj-$(CONFIG_CRYPTO_CCM) += ccm.o +obj-$(CONFIG_CRYPTO_PCRYPT_CORE) += pcrypt_core.o +obj-$(CONFIG_CRYPTO_PCRYPT) += pcrypt.o obj-$(CONFIG_CRYPTO_CRYPTD) += cryptd.o obj-$(CONFIG_CRYPTO_DES) += des_generic.o obj-$(CONFIG_CRYPTO_FCRYPT) += fcrypt.o
crypto/pcrypt.c new file +415 -0
415 lines not on this page · raw message
crypto/pcrypt_core.c new file +106 -0
106 lines not on this page · raw message
include/crypto/pcrypt.h new file +56 -0
56 lines not on this page · raw message
include/linux/crypto.h +2 -0
@@ -71,6 +71,8 @@#define CRYPTO_ALG_TESTED 0x00000400 +#define CRYPTO_ALG_PCRYPT 0x00000800 +/* * Transform masks and values (for crt_flags). */
include/linux/interrupt.h +2 -0
@@ -338,6 +338,8 @@ enumTASKLET_SOFTIRQ ,SCHED_SOFTIRQ ,HRTIMER_SOFTIRQ , +AEAD_ENC_SOFTIRQ , +AEAD_DEC_SOFTIRQ ,PADATA_SOFTIRQ , RCU_SOFTIRQ, /* Preferable RCU should always be the last softirq */
kernel/softirq.c +2 -1
@@ -56,7 +56,8 @@ static DEFINE_PER_CPU(struct task_struct *, ksoftirqd);char *softirq_to_name[NR_SOFTIRQS ] = {"HI" ,"TIMER" ,"NET_TX" ,"NET_RX" ,"BLOCK" , -"TASKLET" ,"SCHED" ,"HRTIMER" ,"PADATA" ,"RCU" +"TASKLET" ,"SCHED" ,"HRTIMER" ,"AEAD_ENC" , +"AEAD_DEC" ,"PADATA" ,"RCU" };/*
Signature · 1 line
1.5.4.2
Herbert Xu
maintainerRe: [RFC] [PATCH v2 2/4] pcrypt: Add pcrypt crypto parallelization engine
crypto/pcrypt.c line 332
Quoted 4 lines · Steffen Klassert
crypto/pcrypt.c new file +3
-0+ alg =crypto_get_attr_alg (tb, algt->type, + (algt->mask &CRYPTO_ALG_TYPE_MASK ) + |CRYPTO_ALG_PCRYPT );
Do you really need to exclude pcrypt algorithms here? If the user wants to added another layer of pcrypt then we should obey.
Otherwise this looks pretty good to me.
Cheers,
Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txtSteffen Klassert
maintainerRe: [RFC] [PATCH v2 2/4] pcrypt: Add pcrypt crypto parallelization engine
Quoted 9 lines · Herbert Xu
crypto/pcrypt.c line 332
Quoted 4 lines · Steffen Klassert
crypto/pcrypt.c new file +3
-0+ alg =crypto_get_attr_alg (tb, algt->type, + (algt->mask &CRYPTO_ALG_TYPE_MASK ) + |CRYPTO_ALG_PCRYPT );Do you really need to exclude pcrypt algorithms here? If the user wants to added another layer of pcrypt then we should obey.
I don't need to exclude it. I just wanted to avoid another layer of pcrypt here because it does not make too much sense to parallelize a second time. If the aead algorithm is already parallel it would just add overhead if we parallelize an underlying algorithm too. Anyway, I would not mind to allow more than one pcrypt layer.
Steffen
Herbert Xu
maintainerRe: [RFC] [PATCH v2 2/4] pcrypt: Add pcrypt crypto parallelization engine
Quoted 6 lines · Steffen Klassert
I don't need to exclude it. I just wanted to avoid another layer of pcrypt here because it does not make too much sense to parallelize a second time. If the aead algorithm is already parallel it would just add overhead if we parallelize an underlying algorithm too. Anyway, I would not mind to allow more than one pcrypt layer.
I thought so. The system is certainly not going to automatically construct a second pcrypt algorithm once one exists. So I think we can just remove this bit.
Cheers,
Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txtSteffen Klassert
maintainerRe: [RFC] [PATCH v2 2/4] pcrypt: Add pcrypt crypto parallelization engine
Quoted 5 lines · Herbert Xu
I thought so. The system is certainly not going to automatically construct a second pcrypt algorithm once one exists. So I think we can just remove this bit.
Ok, so I will remove it in the next version.
Steffen
Steffen Klassert
maintainerRFC PATCH v2 3/4 xfrm: Add a netlink attribute for software crypto accelerators
In reply to Steffen KlassertThis patch adds a netlink attribute for software crypto accelerators like pcrypt. This makes it possible to choose a crypto accelerator template by it's name from the userspace.
--- include/linux/xfrm.h | 10 ++++++++++ include/net/xfrm.h | 3 +++ net/xfrm/xfrm_user.c | 8 ++++++++ 3 files changed, 21 insertions(+), 0 deletions(-)
include/linux/xfrm.h +10 -0
@@ -240,6 +240,15 @@ struct xfrm_encap_tmpl {xfrm_address_t encap_oa; }; +struct xfrm_accl { +char name[64 ]; +__u8 type; +#define XFRM_ACCL_ENC 1 +#define XFRM_ACCL_AUTH 2 +#define XFRM_ACCL_COMP 4 +#define XFRM_ACCL_AEAD 8 +}; +/* AEVENT flags */ enum xfrm_ae_ftype_t {XFRM_AE_UNSPEC ,
@@ -283,6 +292,7 @@ enum xfrm_attr_type_t {XFRMA_MIGRATE ,XFRMA_ALG_AEAD ,/* struct xfrm_algo_aead */ XFRMA_KMADDRESS ,/* struct xfrm_user_kmaddress */ +XFRMA_CRYPTO_ACCL ,/* struct xfrm_accl */ __XFRMA_MAX#define XFRMA_MAX (__XFRMA_MAX - 1)
include/net/xfrm.h +3 -0
@@ -171,6 +171,9 @@ struct xfrm_state/* Data for care-of address */ xfrm_address_t *coaddr; +/* Data for crypto accelerator */ +struct xfrm_accl *accl; +/* IPComp needs an IPIP tunnel for handling uncompressed packets */ struct xfrm_state *tunnel;
net/xfrm/xfrm_user.c +8 -0
@@ -359,6 +359,13 @@ static struct xfrm_state *xfrm_state_construct(struct net *net, goto error; } +if (attrs[XFRMA_CRYPTO_ACCL ]) { + x->accl =kmemdup (nla_data (attrs[XFRMA_CRYPTO_ACCL ]), +sizeof (*x->accl),GFP_KERNEL ); +if (x->accl ==NULL ) + goto error; + } + err =xfrm_init_state (x);if (err) goto error;
@@ -1958,6 +1965,7 @@ static const struct nla_policy xfrma_policy[XFRMA_MAX+1] = { [XFRMA_POLICY_TYPE] = { .len = sizeof(struct xfrm_userpolicy_type)}, [XFRMA_MIGRATE] = { .len = sizeof(struct xfrm_user_migrate) }, [XFRMA_KMADDRESS] = { .len = sizeof(struct xfrm_user_kmaddress) }, + [XFRMA_CRYPTO_ACCL] = { .len = sizeof(struct xfrm_accl) }, }; static struct xfrm_link {
Signature · 1 line
1.5.4.2
Herbert Xu
maintainerRe: [RFC] [PATCH v2 3/4] xfrm: Add a netlink attribute for software crypto accelerators
include/linux/xfrm.h struct xfrm_encap_tmpl { · line 250
Quoted 9 lines · Steffen Klassert
include/linux/xfrm.h +8
-0+struct xfrm_accl { +char name[64 ]; +__u8 type; +#define XFRM_ACCL_ENC 1 +#define XFRM_ACCL_AUTH 2 +#define XFRM_ACCL_COMP 4 +#define XFRM_ACCL_AEAD 8 +};
While this should work for pcrypt, I'd like this to be solved more generally. The crux of the issue is that we can't specify an arbitrary implementation of a given algorithm. So the obvious solution is to specify the driver name along with the algorithm name.
This is in fact pretty much what you've done, but I'd just like it to be generalised. In particular, instead of having just a single name per SA, we should allow one to be set for each algorithm type.
On another note, I don't expect this to be the primary mechanism for activating parallel processing. Doing it manually on each SA is just painful. This should be used for testing or when you want to specify it for a subset of SAs only.
When the admin wants to turn the entire system over to pcrypt, it should be done at the crypto layer, by simply registering the pcrypt version of the algorithm in question, and having it as the default implementation of that algorithm.
In fact, this mechanism should then be able to allow specific SAs to not use parallel processing, which means that it should definitely not be called accl :)
Thanks,
Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txtSteffen Klassert
maintainerRe: [RFC] [PATCH v2 3/4] xfrm: Add a netlink attribute for software crypto accelerators
Quoted 6 lines · Herbert Xu
While this should work for pcrypt, I'd like this to be solved more generally. The crux of the issue is that we can't specify an arbitrary implementation of a given algorithm. So the obvious solution is to specify the driver name along with the algorithm name.
So how general should it be? For the moment I would see pcrypt and maybe cryptd as possible candidates to use this mechanism. I'm just wondering if it is worth to set up a list of crypto templates that can be choosen from userspace, similar to the xfrm_algo_list.
Quoted 5 lines
This is in fact pretty much what you've done, but I'd just like it to be generalised. In particular, instead of having just a single name per SA, we should allow one to be set for each algorithm type.
Just to get you right, do you think about adding a netlink attribute for each algorithm type?
Quoted 10 lines
On another note, I don't expect this to be the primary mechanism for activating parallel processing. Doing it manually on each SA is just painful. This should be used for testing or when you want to specify it for a subset of SAs only.
When the admin wants to turn the entire system over to pcrypt, it should be done at the crypto layer, by simply registering the pcrypt version of the algorithm in question, and having it as the default implementation of that algorithm.
That's not really clear to me how to let the user register the pcrypt version of the algorithm, so what's the desired way do this.
Quoted 5 lines
In fact, this mechanism should then be able to allow specific SAs to not use parallel processing, which means that it should definitely not be called accl :)
Yes, I think I'll find a better name :)
Steffen
Steffen Klassert
maintainerRFC PATCH v2 4/4 esp: Use pcrypt if it is selected as software crypto accelerator
In reply to Steffen KlassertThis patch adds a function that sets up the format string for authenc algorithms. If pcrypt is coosen as accelerator, the format string is set up to use pcrypt. If no accelerator is choosen, the default format string is set up.
--- net/ipv4/esp4.c | 40 ++++++++++++++++++++++++++++++++++++---- net/ipv6/esp6.c | 40 ++++++++++++++++++++++++++++++++++++---- 2 files changed, 72 insertions(+), 8 deletions(-)
net/ipv4/esp4.c +36 -4
@@ -467,6 +467,40 @@ error:return err; } +static int esp_authenc_name (struct xfrm_state *x,char *authenc_name) +{ +struct xfrm_accl *accl = x->accl; +int err =0 ; + +if (!accl) { +if (snprintf (authenc_name,CRYPTO_MAX_ALG_NAME , +"authenc(%s,%s)" , + x->aalg ? x->aalg->alg_name :"digest_null" , + x->ealg->alg_name) >=CRYPTO_MAX_ALG_NAME ) + err = -ENAMETOOLONG ; + + goto out; + } + +/* Set aead as the default accl type if type is unspecified */ +if (!accl->type) + accl->type |=XFRM_ACCL_AEAD ; + +if (!strcmp (accl->name,"pcrypt" ) && (accl->type &XFRM_ACCL_AEAD )) { +if (snprintf (authenc_name,CRYPTO_MAX_ALG_NAME , +"pcrypt(authenc(%s,eseqiv(%s)))" , + x->aalg ? x->aalg->alg_name :"digest_null" , + x->ealg->alg_name) >=CRYPTO_MAX_ALG_NAME ) + err = -ENAMETOOLONG ; + + goto out; + } + + err = -EINVAL ; +out: +return err; +} +static int esp_init_authenc (struct xfrm_state *x) {struct esp_data *esp = x->data;
@@ -483,10 +517,8 @@ static int esp_init_authenc(struct xfrm_state *x)if (x->ealg ==NULL ) goto error; - err = -ENAMETOOLONG ; -if (snprintf (authenc_name,CRYPTO_MAX_ALG_NAME ,"authenc(%s,%s)" , - x->aalg ? x->aalg->alg_name :"digest_null" , - x->ealg->alg_name) >=CRYPTO_MAX_ALG_NAME ) + err =esp_authenc_name (x, authenc_name); +if (err) goto error; aead =crypto_alloc_aead (authenc_name,0 ,0 );
net/ipv6/esp6.c +36 -4
@@ -410,6 +410,40 @@ error:return err; } +static int esp_authenc_name (struct xfrm_state *x,char *authenc_name) +{ +struct xfrm_accl *accl = x->accl; +int err =0 ; + +if (!accl) { +if (snprintf (authenc_name,CRYPTO_MAX_ALG_NAME , +"authenc(%s,%s)" , + x->aalg ? x->aalg->alg_name :"digest_null" , + x->ealg->alg_name) >=CRYPTO_MAX_ALG_NAME ) + err = -ENAMETOOLONG ; + + goto out; + } + +/* Set aead as the default accl type if type is unspecified */ +if (!accl->type) + accl->type |=XFRM_ACCL_AEAD ; + +if (!strcmp (accl->name,"pcrypt" ) && (accl->type &XFRM_ACCL_AEAD )) { +if (snprintf (authenc_name,CRYPTO_MAX_ALG_NAME , +"pcrypt(authenc(%s,eseqiv(%s)))" , + x->aalg ? x->aalg->alg_name :"digest_null" , + x->ealg->alg_name) >=CRYPTO_MAX_ALG_NAME ) + err = -ENAMETOOLONG ; + + goto out; + } + + err = -EINVAL ; +out: +return err; +} +static int esp_init_authenc (struct xfrm_state *x) {struct esp_data *esp = x->data;
@@ -426,10 +460,8 @@ static int esp_init_authenc(struct xfrm_state *x)if (x->ealg ==NULL ) goto error; - err = -ENAMETOOLONG ; -if (snprintf (authenc_name,CRYPTO_MAX_ALG_NAME ,"authenc(%s,%s)" , - x->aalg ? x->aalg->alg_name :"digest_null" , - x->ealg->alg_name) >=CRYPTO_MAX_ALG_NAME ) + err =esp_authenc_name (x, authenc_name); +if (err) goto error; aead =crypto_alloc_aead (authenc_name,0 ,0 );
Signature · 1 line
1.5.4.2
Hi.
Quoted 3 lines · Steffen Klassert
This patchset adds the 'pcrypt' parallel crypto template. With this template it is possible to process the crypto requests of a transform in parallel without getting request reorder. This is in particular interesting for IPsec.
Why can't it be used by default for all crypto operations instead of synchronous one?
Signature · 1 line
Evgeniy Polyakov
Herbert Xu
maintainerQuoted 9 lines · Evgeniy Polyakov
Hi.
Quoted 3 lines · Steffen Klassert
This patchset adds the 'pcrypt' parallel crypto template. With this template it is possible to process the crypto requests of a transform in parallel without getting request reorder. This is in particular interesting for IPsec.
Why can't it be used by default for all crypto operations instead of synchronous one?
PCI-based drivers will not benefit from spreading the requests across CPUs. If anything they will suffer from the synchronisation.
Cheers,
Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txtQuoted 5 lines · Herbert Xu
Quoted 2 lines
Why can't it be used by default for all crypto operations instead of synchronous one?
PCI-based drivers will not benefit from spreading the requests across CPUs. If anything they will suffer from the synchronisation.
What's the deal PCI drivers have with CPUs data comes from/to? They do not touch cache, just run DMA transfer and complete the request.
Signature · 1 line
Evgeniy Polyakov
Herbert Xu
maintainerQuoted 3 lines · Evgeniy Polyakov
What's the deal PCI drivers have with CPUs data comes from/to? They do not touch cache, just run DMA transfer and complete the request.
It's the same as NICs. To set things up and to reap the result you need to synchronise.
Cheers,
Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt