Skip to contentSkip to conversation

security: add software synchronous crypto process

RFC PATCH In review Fan Zhang 5 participants 87 messages Sep 3, 2019 – Nov 1, 2019 Page 2 of 2

Subsystems (2)
lore
Share & tools
lore
#40 lore raw

Re: [dpdk-dev] [RFC PATCH 1/9] security: introduce CPU Crypto action type and API

In reply to Ananyev, Konstantin

Hi Konstantin,

Quoted 313 lines

Hi Akhil,

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Added my comments inline with your draft.
[snip]..

Quoted 3 lines

Ok, then my suggestion:
Let's at least write down all points about crypto-dev approach where

we

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

disagree and then probably try to resolve them one by one.... If we fail to make an agreement/progress in next week or so, (and no more reviews from the community)
will have bring that subject to TB meeting to decide. Sounds fair to you?

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Agreed

Quoted 5 lines

List is below.
Please add/correct me, if I missed something.

Konstantin

Before going into comparison, we should define the requirement as

well.

Quoted 37 lines
Quoted 37 lines
Quoted 5 lines
Quoted 5 lines

Good point.

Quoted 2 lines

What I understood from the patchset,
"You need a synchronous API to perform crypto operations on raw data

using

Quoted 9 lines
Quoted 9 lines

SW PMDs"

Quoted 7 lines

So,
- no crypto-ops,
- no separate enq-deq, only single process API for data path
- Do not need any value addition to the session parameters. (You would need some parameters from the crypto-op which Are constant per session and since you wont use crypto-op, You need some place to store that)

Quoted 7 lines
Quoted 7 lines

Yes, this is correct, I think.

Quoted 5 lines

Now as per your mail, the comparison
1. extra input parameters to create/init rte_(cpu)_sym_session.

Will leverage existing 6B gap inside rte_crypto_*_xform between 'algo'

and

Quoted 2 lines
Quoted 2 lines

'key' fields.

Quoted 1 line

New fields will be optional and would be used by PMD only when cpu-

crypto

Quoted 11 lines
Quoted 11 lines

session is requested.

Quoted 8 lines

For lksd-crypto session PMD is free to ignore these fields. No ABI breakage is required.

[Akhil] Agreed, no issues.

2. cpu-crypto create/init.
a) Our suggestion - introduce new API for that:
- rte_crypto_cpu_sym_init() that would init completely opaque

rte_crypto_cpu_sym_session.

Quoted 1 line

- struct rte_crypto_cpu_sym_session_ops {(*process)(...); (*clear);

Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines

/*whatever else we'll need *'};

Quoted 1 line

- rte_crypto_cpu_sym_get_ops(const struct rte_crypto_sym_xform

*xforms)

Quoted 1 line

that would return const struct rte_crypto_cpu_sym_session_ops

*based

Quoted 3 lines
Quoted 3 lines

on input xforms.

Quoted 2 lines

Advantages:
1) totally opaque data structure (no ABI breakages in future),

PMD

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

writer is totally free

Quoted 3 lines

with it format and contents.

[Akhil] It will have breakage at some point till we don't hit the union

size.

Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
Quoted 2 lines

Not sure, what union you are talking about?

Union of xforms in rte_security_session_conf

Quoted 7 lines
Quoted 7 lines

Hmm, how does it relates here?
I thought we discussing pure rte_cryptodev_sym_session, no?

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Rather I don't suspect there will be more parameters added. Or do we really care about the ABI breakage when the argument is

about

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

the correct place to add a piece of code or do we really agree to add

code

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line

anywhere just to avoid that breakage.

I am talking about maintaining it in future.
if your struct is not seen externally, no chances to introduce ABI

breakage.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

2) each session entity is self-contained, user doesn't need to

bring along

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

dev_id etc.

Quoted 1 line

dev_id is needed only at init stage, after that user will use

session ops

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines

to perform

Quoted 3 lines

all operations on that session (process(), clear(), etc.).

[Akhil] There is nothing called as session ops in current DPDK.

True, but it doesn't mean we can't/shouldn't have it.

We can have it if it is not adding complexity for the user. Creating 2

different

Quoted 6 lines
Quoted 6 lines

code

Quoted 5 lines

Paths for user is not desirable for the stack developers.

Quoted 3 lines
Quoted 3 lines

What you are proposing
is a new concept which doesn't have any extra benefit, rather it is

adding

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

complexity

Quoted 4 lines

to have two different code paths for session create.

3) User can decide does he wants to store ops[] pointer on a per

session

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

basis,

Quoted 3 lines

or on a per group of same sessions, or...

[Akhil] Will the user really care which process API should be called from

the

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

PMD.

Quoted 1 line

Rather it should be driver's responsibility to store that in the session

private

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

data

Quoted 1 line

which would be opaque to the user. As per my suggestion same process

function can

Quoted 1 line

be added to multiple sessions or a single session can be managed inside

the

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

PMD.

In that case we either need to have a function per session (stored

internally),

Quoted 10 lines
Quoted 10 lines
Quoted 7 lines
Quoted 7 lines

or make decision (branches) at run-time.
But as I said in other mail - I am ok to add small shim structure here:
either rte_crypto_cpu_sym_session { void *ses; struct rte_crypto_cpu_sym_session_ops ops; }
or rte_crypto_cpu_sym_session { void *ses; struct rte_crypto_cpu_sym_session_ops *ops; }
And merge rte_crypto_cpu_sym_init() and rte_crypto_cpu_sym_get_ops()

into

Quoted 2 lines
Quoted 1 line

one (init).

Quoted 7 lines
Quoted 7 lines
Quoted 1 line

Again that will be a separate API call from the user perspective which is not

good.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

4) No mandatory mempools for private sessions. User can

allocate

Quoted 15 lines
Quoted 15 lines
Quoted 11 lines
Quoted 11 lines

memory for cpu-crypto

Quoted 3 lines

session whenever he likes.

[Akhil] you mean session private data?

Yes.

Quoted 2 lines

You would need that memory anyways, user will be allocating that already. You do not need to manage that.

What I am saying - right now user has no choice but to allocate it via

mempool.

Quoted 3 lines
Quoted 3 lines

Which is probably not the best options for all cases.

Quoted 1 line
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Disadvantages:
5) Extra changes in control path
6) User has to store session_ops pointer explicitly.

[Akhil] More disadvantages:
- All supporting PMDs will need to maintain TWO types of session for

the

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

same crypto processing. Suppose a fix or a new feature(or algo) is

added,

Quoted 7 lines
Quoted 7 lines

PMD

Quoted 6 lines
Quoted 6 lines

owner

Quoted 1 line

will need to add code in both the session create APIs. Hence more

maintenance and

Quoted 1 line

error prone.

I think majority of code for both paths will be common, plus even we'll

reuse

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

current sym_session_init() -
changes in PMD session_init() code will be unavoidable.
But yes, it will be new entry in devops, that PMD will have to support. Ok to add it as 7) to the list.

Quoted 3 lines

- Stacks which will be using these new APIs also need to maintain two code path for the same processing while doing session initialization for sync and async

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

That's the same as #5 above, I think.

Quoted 3 lines

b) Your suggestion - reuse existing rte_cryptodev_sym_session_init()

and

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

existing rte_cryptodev_sym_session

Quoted 6 lines

structure.
Advantages:
1) allows to reuse same struct and init/create/clear() functions. Probably less changes in control path.
Disadvantages:
2) rte_cryptodev_sym_session. sess_data[] is indexed by

driver_id,

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

which means that

Quoted 1 line

we can't use the same rte_cryptodev_sym_session to hold

private

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

sessions pointers

Quoted 4 lines

for both sync and async mode for the same device.
So the only option we have - make PMD devops-
sym_session_configure()
always create a session that can work in both cpu and lksd

modes.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

For some implementations that would probably mean that

under the

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

hood PMD would create

Quoted 1 line

2 different session structs (sync/async) and then use one or

another

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

depending on from what API been called.

Quoted 3 lines

Seems doable, but ...:
- will contradict with statement from 1:
" New fields will be optional and would be used by PMD only

when

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

cpu-crypto session is requested."

Quoted 1 line

Now it becomes mandatory for all apps to specify cpu-

crypto

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

related parameters too,

Quoted 1 line

even if they don't plan to use that mode - i.e. behavior

change,

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

existing app change.

Quoted 3 lines

- might cause extra space overhead.

[Akhil] It will not contradict with #1, you will only have few checks in

the

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

session init PMD

Quoted 1 line

Which support this mode, find appropriate values and set the

appropriate

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

process() in it.

Quoted 1 line

User should be able to call, legacy enq-deq as well as the new process()

without any issue.

Quoted 2 lines

User would be at runtime will be able to change the datapath. So this is not a disadvantage, it would be additional flexibility for the

user.

Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
Quoted 4 lines

Ok, but that's what I am saying - if PMD would *always* have to create a session that can handle
both modes (sync/async), then user would *always* have to provide

parameters

Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines

for both modes too.
Otherwise if let say user didn't setup sync specific parameters at all, what

PMD

Quoted 5 lines
Quoted 5 lines

should do?
- return with error?
- init session that can be used with async path only?
My current assumption is #1.
If #2, then how user will be able to distinguish is that session valid for

both

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line

modes, or only for one?

I would say a 3rd option, do nothing if sync params are not set. Probably have a debug print in the PMD(which support sync mode) to

specify

Quoted 4 lines
Quoted 4 lines

that

Quoted 1 line

session is not configured properly for sync mode.

So, just print warning and proceed with init session that can be used with

async

Quoted 8 lines
Quoted 8 lines

path only?
Then it sounds the same as #2 above.
Which actually means that sync mode parameters for sym_session_init() becomes optional.
Then we need an API to provide to the user information what modes (sync+async/async only) is supported by that session for given dev_id. And user would have to query/retain this information at control-path, and store it somewhere in user-space together with session pointer and

dev_ids

Quoted 3 lines
Quoted 3 lines

to use later at data-path (same as we do now for session type). That definitely requires changes in control-path to start using it. Plus the fact that this value can differ for different dev_ids for the same

session -

Quoted 3 lines
Quoted 1 line

doesn't make things easier here.

API wont be required to specify that. Feature flag will be sufficient, not a big

change

Quoted 3 lines

From the application perspective.

Here is some pseudo code just to elaborate my understanding. This will need

some

Quoted 6 lines

From application,

If(dev_info->feature_flags & RTE_CRYPTODEV_FF_SYNC) {
	/* set additional params in crypto xform */
}
Quoted 7 lines

Now in the driver,

pmd_sym_session_configure(dev,xform,sess,mempool) {
	...
	If(dev_info->feature_flags & RTE_CRYPTODEV_FF_SYNC
		&& xform->/*sync params are set*/) {
		/*Assign process function pointer in sess->priv_data*/
	} /* It may return error if FF_SYNC is set and params are not correct.
Quoted 5 lines

Then all apps will always *have to* setup sync parameters in xform. What you suggest is *mandatory* sync mode: user always has to setup sync mode params if PMD does support it (no matter does he plan to use sync mode or not).
Which means behavior change in existing apps.

We are adding new params in xform, and user may not fill those params and defaults To 0 for all the params. Or we can pack a flag in xform when all sync params are set. It can be dealt with when we do the code.

I don't say, user will always have to set the params when sync mode is supported. It will be a warning from the PMD and user may ignore it if he doesn't want to use sync mode.

Quoted 32 lines
Quoted 2 lines

It would be upto the driver whether it support both SYNC and

ASYNC.*/

Quoted 12 lines

}

Now the new sync API

pmd_process(...) {
	If(dev_info->feature_flags & RTE_CRYPTODEV_FF_SYNC
			 && sess_priv->process != NULL)
		sess_priv->process(...);
	else
		ASSERT("sync mode not configured properly or not supported");
}
Quoted 9 lines

In the data path, there is no extra processing happening.
Even in case of your suggestion, you should have these type of error checks, You cannot blindly trust on the application that the pointers are correct.

Quoted 5 lines
Quoted 3 lines

Internally the PMD will not store the process() API in the session priv data And while calling the first packet, devops->process will give an assert that

session

Quoted 1 line

Is not configured for sync mode. The session validation would be done in

any

Quoted 7 lines
Quoted 7 lines

case

Quoted 1 line

your suggestion or mine. So no extra overhead at runtime.

I believe that after session_init() user should get either an error or valid session handler that he can use at runtime.
Pushing session validation to runtime doesn't seem like a good idea.

Quoted 6 lines
Quoted 2 lines

It may get a warning from the PMD, that FF_SYNC is set but params are not Correct/available. See above.

I think warning is not enough.
There should be a clear way (API) for developer to realize is the created session can be used by sync API data-path or not.

Warning is a clear notification to the user, that SYNC mode can be supported by the device But user does not want to use that.
Moreover, when first packet is sent, sync API will throw error. So what is the issue.

Quoted 204 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

3) not possible to store device (not driver) specific data within

the

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

session, but I think it is not really needed right now.

Quoted 3 lines

So probably minor compared to 2.b.2.

[Akhil] So lets omit this for current discussion. And I hope we can find

some

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

way to deal with it.

I don't think there is an easy way to fix that with existing API.

Quoted 6 lines

Actually #3 follows from #2, but decided to have them separated.

3. process() parameters/behavior
a) Our suggestion: user stores ptr to session ops (or to (*process)

itself)

Quoted 10 lines
Quoted 10 lines

and

Quoted 7 lines
Quoted 7 lines

just does:

Quoted 6 lines

session_ops->process(sess, ...); Advantages:
1) fastest possible execution path
2) no need to carry on dev_id for data-path

[Akhil] I don't see any overhead of carrying dev id, at least it would be

inline

Quoted 1 line
Quoted 1 line

with the

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
Quoted 7 lines
Quoted 1 line

current DPDK methodology.

If we'll add process() into rte_cryptodev itself (same as we have enqueue_burst/dequeue_burst),
then it will be an ABI breakage.
Also there are discussions to get rid of that approach completely:

https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmails.dpd

Quoted 2 lines
Quoted 2 lines

k.org%2Farchives%2Fdev%2F2019-

September%2F144674.html&data=02%7C01%7Cakhil.goyal%40nxp.com%7

Quoted 1 line
Quoted 1 line

C1859dc1d29cd45a51e9908d7571784bb%7C686ea1d3bc2b4c6fa92cd99c5c301

Quoted 1 line
Quoted 1 line

635%7C0%7C0%7C637073630835415165&sdata=Bz9jgisyVzRJNt1BijtvSlurh

Quoted 7 lines
Quoted 7 lines

JU1vXBbynNwlMDjaco%3D&reserved=0

Quoted 6 lines
Quoted 1 line

So I am not sure this is a recommended way these days.

We can either have it in rte_cryptodev or in rte_cryptodev_ops whichever is good for you.

Whether it is ABI breakage or not, as per your requirements, this is the

correct

Quoted 21 lines
Quoted 6 lines
Quoted 1 line

approach. Do you agree with this or not?

I think it is possible approach, but not the best one:
it looks quite flakey to me (see all these uncertainty with sym_session_init above),
plus introduces extra overhead at data-path.

Uncertainties can be handled appropriately using a feature flag

Quoted 12 lines
Quoted 12 lines

Now handling the API/ABI breakage is a separate story. In 19.11 release we Are not much concerned about the ABI breakages, this was discussed in community. So adding a new dev_ops wouldn't have been an issue. Now since we are so close to RC1 deadline, we should come up with some other solution for next release. May be having a pmd API in 20.02 and converting it into formal one in 20.11

Quoted 2 lines
Quoted 2 lines

What you are suggesting is a new way to get the things done without

much

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

benefit.

Would help with ABI stability plus better performance, isn't it enough?

Quoted 1 line

Also I don't see any performance difference as crypto workload is

heavier

Quoted 6 lines
Quoted 6 lines

than

Quoted 5 lines
Quoted 5 lines
Quoted 1 line

Code cycles, so that wont matter.

It depends.
Suppose function call costs you ~30 cycles.
If you have burst of big packets (let say crypto for each will take ~2K

cycles)

Quoted 3 lines
Quoted 3 lines

that

Quoted 2 lines
Quoted 2 lines

belong
to the same session, then yes you wouldn't notice these extra 30 cycles

at all.

Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line

If you have burst of small packets (let say crypto for each will take ~300

cycles)

Quoted 4 lines
Quoted 2 lines

each
belongs to different session, then it will cost you ~10% extra.

Let us do some profiling on openssl with both the approaches and find out

the

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

difference.

Quoted 7 lines
Quoted 7 lines

So IMO, there is no advantage in your suggestion as well.

Disadvantages:
3) user has to carry on session_ops pointer explicitly
b) Your suggestion: add (*cpu_process) inside rte_cryptodev_ops

and

Quoted 8 lines
Quoted 8 lines

then:

Quoted 1 line
Quoted 1 line
Quoted 1 line

rte_crypto_cpu_sym_process(uint8_t dev_id,

rte_cryptodev_sym_session

Quoted 5 lines
Quoted 4 lines

*sess, /*data parameters*/) {...

Quoted 3 lines
                     rte_cryptodevs[dev_id].dev_ops->cpu_process(ses, ...);
                      /*and then inside PMD specifc process: */
                     pmd_private_session = sess-

sess_data[this_pmd_driver_id].data;

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

/* and then most likely either */ pmd_private_session->process(pmd_private_session, ...); /* or jump based on session/input data */
Advantages:
1) don't see any...
Disadvantages:
2) User has to carry on dev_id inside data-path
3) Extra level of indirection (plus data dependency) - both for

data and

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

instructions.

Quoted 3 lines

Possible slowdown compared to a) (not measured).

Having said all this, if the disagreements cannot be resolved, you can

go

Quoted 6 lines
Quoted 6 lines

for a

Quoted 5 lines
Quoted 5 lines

pmd API specific

Quoted 1 line

to your PMDs,

I don't think it is good idea.
PMD specific API is sort of deprecated path, also there is no clean way to

use

Quoted 4 lines
Quoted 4 lines

it

Quoted 3 lines
Quoted 1 line

within the libraries.

I know that this is a deprecated path, we can use it until we are not

allowed

Quoted 7 lines
Quoted 7 lines
Quoted 4 lines

to break ABI/API

Quoted 2 lines
Quoted 2 lines

because as per my understanding the solution doesn't look scalable to

other

Quoted 2 lines
Quoted 2 lines

PMDs.

Quoted 1 line

Your approach is aligned only to Intel , will not benefit others like

openssl

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

which is used by all

Quoted 1 line

vendors.

I feel quite opposite, from my perspective majority of SW backed PMDs

will

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

benefit from it.
And I don't see anything Intel specific in my proposals above. About openssl PMD: I am not an expert here, but looking at the code, I

think

Quoted 5 lines
Quoted 5 lines

it

Quoted 4 lines
Quoted 4 lines

will fit really well.
Look yourself at its internal functions:
process_openssl_auth_op/process_openssl_crypto_op,
I think they doing exactly the same - they use sync API underneath, and

they

Quoted 7 lines
Quoted 7 lines

are

Quoted 6 lines
Quoted 4 lines

session based
(AFAIK you don't need any device/queue data, everything that needed for crypto/auth is stored inside session).

By vendor specific, I mean,
- no PMD would like to have 2 different variants of session Init APIs for

doing

Quoted 2 lines
Quoted 2 lines

the same stuff.

Quoted 1 line

- stacks will become vendor specific while using 2 separate session create

APIs.

Quoted 2 lines
Quoted 2 lines

No stack would

Quoted 1 line

Like to support 2 variants of session create- one for HW PMDs and one for

SW

Quoted 18 lines
Quoted 17 lines

PMDs.

I think what you refer on has nothing to do with 'vendor specific'. I would name it 'extra overhead for PMD and stack writers'. Yes, for sure there is extra overhead (as always with new API) - for both producer (PMD writer) and consumer (stack writer):
New function(s) to support, probably more tests to create/run, etc. Though this API is optional - if PMD/stack maintainer doesn't see value in it, they are free not to support it.
From other side, re-using rte_cryptodev_sym_session_init() wouldn't help anyway - both data-path and control-path would differ from async mode anyway.
BTW, right now to support different HW flavors
we do have 4 different control and data-paths for both ipsec-secgw and librte_ipsec:
lkds-none/lksd-proto/inline-crypto/inline-proto.
And that is considered to be ok.

Quoted 5 lines
Quoted 1 line

No that is not ok. We cannot add new paths for every other case.

What I am saying: if let-say lookaside-proto/inline-crypto/inline-proto deserves its own case in rte_security/rte_crypto API,
I don't understand why cpu-crypto doesn't.

Because cpu-crypto is done by a crypto device and for that we have lookaside none. SW PMDs are registered as crypto device and we should leverage crypto framework. I would suggest in future may be 20.11, we can have a security wrapper over cryptodev API For lookaside none use case. So that we will have a single API set for all cases.

Quoted 19 lines
Quoted 2 lines

Those 4 are controlled using 2 set of APIs.

Yes there are 2 API sets (rte_cryptodev/rte_security),
but in fact if you look at ipsec-secgw and librte_ipsec we have 4 different code paths.
For both create_session() and ipsec_enqueue() we have a big switch() with 4 different cases.
Nearly the same for librte_ipsec - we have different prepare/process function pointers for each security type.

Quoted 2 lines

We should try our best to
Have minimum overhead to the application writer. This pain was also

discussed

Quoted 5 lines

In the one of DPDK conference as well.
DPDK is not a standalone entity, there are stacks running over it always. We should not add API for every other use case when we have an alternative Approach with the existing API set.

Quoted 7 lines
Quoted 2 lines

Now introducing another one would add to that pain and a lot of work for Both producer and consumer.

If I would see a clean approach to implement desired functionality without introducing new API - I would definitely support it.
The problem is that from my perspective,
what you suggesting with existing API will bring more drawbacks then positives.

From my perspective I see more benefits than the negatives.
- less changes in driver/app
- no major performance gap
- easier migration for the stack from one SOC to other.

The main argument from my side is that:
You need synchronous processing for SW PMDs which is data path.
Why do you need a special session control path to do that. You should have some extra Params packed in the same control API.

Quoted 3 lines

BTW, our first approach (via rte_security) does reuse existing API, so if adding new API is the main concern - let's reconsider that path.

That will be there only if we have security wrapper on cryptodev session create For lookaside none use case. But the issue would still remain the same. No special session create for supporting sync mode.

Quoted 16 lines
Quoted 1 line

It would be interesting to see how much performance difference will be there

in the

Quoted 1 line

Two approaches. As per my understanding it wont be much as compared to

the

Quoted 12 lines

Extra work that you will be inducing.

-Akhil

Quoted 8 lines

Honestly, I don't understand why SW backed implementations can't have their own path that would suite them most. Konstantin

author
#41 lore raw

RFC PATCH 2/9 crypto/aesni_gcm: add rte_security handler

In reply to Fan Zhang

This patch add rte_security support support to AESNI-GCM PMD. The PMD now initialize security context instance, create/delete PMD specific security sessions, and process crypto workloads in synchronous mode with scatter-gather list buffer supported.

---
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c         | 91 ++++++++++++++++++++++-
 drivers/crypto/aesni_gcm/aesni_gcm_pmd_ops.c     | 95 ++++++++++++++++++++++++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h | 23 ++++++
 3 files changed, 208 insertions(+), 1 deletion(-)
drivers/crypto/aesni_gcm/aesni_gcm_pmd.c +90 -1
index 1006a5c4d..0a346eddd 100644
@@ -6,6 +6,7 @@
 #include <rte_hexdump.h>
 #include <rte_cryptodev.h>
 #include <rte_cryptodev_pmd.h>
+#include <rte_security_driver.h>
 #include <rte_bus_vdev.h>
 #include <rte_malloc.h>
 #include <rte_cpuflags.h>
@@ -174,6 +175,56 @@ aesni_gcm_get_session(struct aesni_gcm_qp *qp, struct rte_crypto_op *op)
 	return sess;
 }
 
+static __rte_always_inline int
+process_gcm_security_sgl_buf(struct aesni_gcm_security_session *sess,
+		struct rte_security_vec *buf, uint8_t *iv,
+		uint8_t *aad, uint8_t *digest)
+{
+	struct aesni_gcm_session *session = &sess->sess;
+	uint8_t *tag;
+	uint32_t i;
+
+	sess->init(&session->gdata_key, &sess->gdata_ctx, iv, aad,
+			(uint64_t)session->aad_length);
+
+	for (i = 0; i < buf->num; i++) {
+		struct iovec *vec = &buf->vec[i];
+
+		sess->update(&session->gdata_key, &sess->gdata_ctx,
+				vec->iov_base, vec->iov_base, vec->iov_len);
+	}
+
+	switch (session->op) {
+	case AESNI_GCM_OP_AUTHENTICATED_ENCRYPTION:
+		if (session->req_digest_length != session->gen_digest_length)
+			tag = sess->temp_digest;
+		else
+			tag = digest;
+
+		sess->finalize(&session->gdata_key, &sess->gdata_ctx, tag,
+				session->gen_digest_length);
+
+		if (session->req_digest_length != session->gen_digest_length)
+			memcpy(digest, sess->temp_digest,
+					session->req_digest_length);
+		break;
+
+	case AESNI_GCM_OP_AUTHENTICATED_DECRYPTION:
+		tag = sess->temp_digest;
+
+		sess->finalize(&session->gdata_key, &sess->gdata_ctx, tag,
+				session->gen_digest_length);
+
+		if (memcmp(tag, digest,	session->req_digest_length) != 0)
+			return -1;
+		break;
+	default:
+		return -1;
+	}
+
+	return 0;
+}
+
 /**
  * Process a crypto operation, calling
  * the GCM API from the multi buffer library.
@@ -488,8 +539,10 @@ aesni_gcm_create(const char *name,
 {
 	struct rte_cryptodev *dev;
 	struct aesni_gcm_private *internals;
+	struct rte_security_ctx *sec_ctx;
 	enum aesni_gcm_vector_mode vector_mode;
 	MB_MGR *mb_mgr;
+	char sec_name[RTE_DEV_NAME_MAX_LEN];
 
 	/* Check CPU for support for AES instruction set */
 	if (!rte_cpu_get_flag_enabled(RTE_CPUFLAG_AES)) {
@@ -524,7 +577,8 @@ aesni_gcm_create(const char *name,
 			RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
 			RTE_CRYPTODEV_FF_CPU_AESNI |
 			RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT |
-			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT;
+			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
+			RTE_CRYPTODEV_FF_SECURITY;
 
 	mb_mgr = alloc_mb_mgr(0);
 	if (mb_mgr == NULL)
@@ -587,6 +641,21 @@ aesni_gcm_create(const char *name,
 
 	internals->max_nb_queue_pairs = init_params->max_nb_queue_pairs;
 
+	/* setup security operations */
+	snprintf(sec_name, sizeof(sec_name) - 1, "aes_gcm_sec_%u",
+			dev->driver_id);
+	sec_ctx = rte_zmalloc_socket(sec_name,
+			sizeof(struct rte_security_ctx),
+			RTE_CACHE_LINE_SIZE, init_params->socket_id);
+	if (sec_ctx == NULL) {
+		AESNI_GCM_LOG(ERR, "memory allocation failed\n");
+		goto error_exit;
+	}
+
+	sec_ctx->device = (void *)dev;
+	sec_ctx->ops = rte_aesni_gcm_pmd_security_ops;
+	dev->security_ctx = sec_ctx;
+
 #if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
 	AESNI_GCM_LOG(INFO, "IPSec Multi-buffer library version used: %s\n",
 			imb_get_version_str());
@@ -641,6 +710,8 @@ aesni_gcm_remove(struct rte_vdev_device *vdev)
 	if (cryptodev == NULL)
 		return -ENODEV;
 
+	rte_free(cryptodev->security_ctx);
+
 	internals = cryptodev->data->dev_private;
 
 	free_mb_mgr(internals->mb_mgr);
@@ -648,6 +719,24 @@ aesni_gcm_remove(struct rte_vdev_device *vdev)
 	return rte_cryptodev_pmd_destroy(cryptodev);
 }
 
+void
+aesni_gcm_sec_crypto_process_bulk(struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num)
+{
+	struct aesni_gcm_security_session *session =
+			get_sec_session_private_data(sess);
+	uint32_t i;
+
+	if (unlikely(!session))
+		return;
+
+	for (i = 0; i < num; i++)
+		status[i] = process_gcm_security_sgl_buf(session, &buf[i],
+				(uint8_t *)iv[i], (uint8_t *)aad[i],
+				(uint8_t *)digest[i]);
+}
+
 static struct rte_vdev_driver aesni_gcm_pmd_drv = {
 	.probe = aesni_gcm_probe,
 	.remove = aesni_gcm_remove
drivers/crypto/aesni_gcm/aesni_gcm_pmd_ops.c +95 -0

111 lines not on this page · raw message

drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h +23 -0
index 56b29e013..8e490b6ce 100644
@@ -114,5 +114,28 @@ aesni_gcm_set_session_parameters(const struct aesni_gcm_ops *ops,
  * Device specific operations function pointer structure */
 extern struct rte_cryptodev_ops *rte_aesni_gcm_pmd_ops;
 
+/**
+ * Security session structure.
+ */
+struct aesni_gcm_security_session {
+	/** Temp digest for decryption */
+	uint8_t temp_digest[DIGEST_LENGTH_MAX];
+	/** GCM operations */
+	aesni_gcm_pre_t pre;
+	aesni_gcm_init_t init;
+	aesni_gcm_update_t update;
+	aesni_gcm_finalize_t finalize;
+	/** AESNI-GCM session */
+	struct aesni_gcm_session sess;
+	/** AESNI-GCM context */
+	struct gcm_context_data gdata_ctx;
+};
+
+extern void
+aesni_gcm_sec_crypto_process_bulk(struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+
+extern struct rte_security_ops *rte_aesni_gcm_pmd_security_ops;
 
 #endif /* _RTE_AESNI_GCM_PMD_PRIVATE_H_ */
Signature · 1 line
2.14.5
author
#42 lore raw

RFC PATCH 3/9 app/test: add security cpu crypto autotest

In reply to Fan Zhang

This patch adds cpu crypto unit test for AESNI_GCM PMD.

---
 app/test/Makefile                   |   1 +
 app/test/meson.build                |   1 +
 app/test/test_security_cpu_crypto.c | 564 ++++++++++++++++++++++++++++++++++++
 3 files changed, 566 insertions(+)
 create mode 100644 app/test/test_security_cpu_crypto.c
app/test/Makefile +1 -0
index 26ba6fe2b..090c55746 100644
@@ -196,6 +196,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_RING) += test_pmd_ring_perf.c
 SRCS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += test_cryptodev_blockcipher.c
 SRCS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += test_cryptodev.c
 SRCS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += test_cryptodev_asym.c
+SRCS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += test_security_cpu_crypto.c
 
 SRCS-$(CONFIG_RTE_LIBRTE_METRICS) += test_metrics.c
 
app/test/meson.build +1 -0
index ec40943bd..b7834ff21 100644
@@ -103,6 +103,7 @@ test_sources = files('commands.c',
 	'test_ring_perf.c',
 	'test_rwlock.c',
 	'test_sched.c',
+	'test_security_cpu_crypto.c',
 	'test_service_cores.c',
 	'test_spinlock.c',
 	'test_stack.c',
app/test/test_security_cpu_crypto.c new file +564 -0

564 lines not on this page · raw message

Signature · 1 line
2.14.5
author
#43 lore raw

RFC PATCH 4/9 app/test: add security cpu crypto perftest

In reply to Fan Zhang

Since crypto perf application does not support rte_security, this patch adds a simple GCM CPU crypto performance test to crypto unittest application. The test includes different key and data sizes test with single buffer and SGL buffer test items and will display the throughput as well as cycle count performance information.

---
 app/test/test_security_cpu_crypto.c | 201 ++++++++++++++++++++++++++++++++++++
 1 file changed, 201 insertions(+)
app/test/test_security_cpu_crypto.c +201 -0
index d345922b2..ca9a8dae6 100644
@@ -23,6 +23,7 @@
 
 #define CPU_CRYPTO_TEST_MAX_AAD_LENGTH	16
 #define MAX_NB_SIGMENTS			4
+#define CACHE_WARM_ITER			2048
 
 enum buffer_assemble_option {
 	SGL_MAX_SEG,
@@ -560,5 +561,205 @@ test_security_cpu_crypto_aesni_gcm(void)
 	return unit_test_suite_runner(&security_cpu_crypto_aesgcm_testsuite);
 }
 
+
+static inline void
+gen_rand(uint8_t *data, uint32_t len)
+{
+	uint32_t i;
+
+	for (i = 0; i < len; i++)
+		data[i] = (uint8_t)rte_rand();
+}
+
+static inline void
+switch_aead_enc_to_dec(struct aead_test_data *tdata,
+		struct cpu_crypto_test_case *tcase,
+		enum buffer_assemble_option sgl_option)
+{
+	uint32_t i;
+	uint8_t *dst = tdata->ciphertext.data;
+
+	switch (sgl_option) {
+	case SGL_ONE_SEG:
+		memcpy(dst, tcase->seg_buf[0].seg, tcase->seg_buf[0].seg_len);
+		tdata->ciphertext.len = tcase->seg_buf[0].seg_len;
+		break;
+	case SGL_MAX_SEG:
+		tdata->ciphertext.len = 0;
+		for (i = 0; i < MAX_NB_SIGMENTS; i++) {
+			memcpy(dst, tcase->seg_buf[i].seg,
+					tcase->seg_buf[i].seg_len);
+			tdata->ciphertext.len += tcase->seg_buf[i].seg_len;
+		}
+		break;
+	}
+
+	memcpy(tdata->auth_tag.data, tcase->digest, tdata->auth_tag.len);
+}
+
+static int
+cpu_crypto_test_aead_perf(enum buffer_assemble_option sgl_option,
+		uint32_t key_sz)
+{
+	struct aead_test_data tdata = {0};
+	struct cpu_crypto_testsuite_params *ts_params = &testsuite_params;
+	struct cpu_crypto_unittest_params *ut_params = &unittest_params;
+	struct cpu_crypto_test_obj *obj = &ut_params->test_obj;
+	struct cpu_crypto_test_case *tcase;
+	uint64_t hz = rte_get_tsc_hz(), time_start, time_now;
+	double rate, cycles_per_buf;
+	uint32_t test_data_szs[] = {64, 128, 256, 512, 1024, 2048};
+	uint32_t i, j;
+	uint8_t aad[16];
+	int ret;
+
+	tdata.key.len = key_sz;
+	gen_rand(tdata.key.data, tdata.key.len);
+	tdata.algo = RTE_CRYPTO_AEAD_AES_GCM;
+	tdata.aad.data = aad;
+
+	ut_params->sess = create_aead_session(ts_params->ctx,
+			ts_params->session_priv_mpool,
+			RTE_CRYPTO_AEAD_OP_DECRYPT,
+			&tdata,
+			0);
+	if (!ut_params->sess)
+		return -1;
+
+	ret = allocate_buf(MAX_NUM_OPS_INFLIGHT);
+	if (ret)
+		return ret;
+
+	for (i = 0; i < RTE_DIM(test_data_szs); i++) {
+		for (j = 0; j < MAX_NUM_OPS_INFLIGHT; j++) {
+			tdata.plaintext.len = test_data_szs[i];
+			gen_rand(tdata.plaintext.data,
+					tdata.plaintext.len);
+
+			tdata.aad.len = 12;
+			gen_rand(tdata.aad.data, tdata.aad.len);
+
+			tdata.auth_tag.len = 16;
+
+			tdata.iv.len = 16;
+			gen_rand(tdata.iv.data, tdata.iv.len);
+
+			tcase = ut_params->test_datas[j];
+			ret = assemble_aead_buf(tcase, obj, j,
+					RTE_CRYPTO_AEAD_OP_ENCRYPT,
+					&tdata, sgl_option, 0);
+			if (ret < 0) {
+				printf("Test is not supported by the driver\n");
+				return ret;
+			}
+		}
+
+		/* warm up cache */
+		for (j = 0; j < CACHE_WARM_ITER; j++)
+			run_test(ts_params->ctx, ut_params->sess, obj,
+					MAX_NUM_OPS_INFLIGHT);
+
+		time_start = rte_rdtsc();
+
+		run_test(ts_params->ctx, ut_params->sess, obj,
+				MAX_NUM_OPS_INFLIGHT);
+
+		time_now = rte_rdtsc();
+
+		rate = time_now - time_start;
+		cycles_per_buf = rate / MAX_NUM_OPS_INFLIGHT;
+
+		rate = ((hz / cycles_per_buf)) / 1000000;
+
+		printf("AES-GCM-%u(%4uB) Enc %03.3fMpps (%03.3fGbps) ",
+				key_sz * 8, test_data_szs[i], rate,
+				rate  * test_data_szs[i] * 8 / 1000);
+		printf("cycles per buf %03.3f per byte %03.3f\n",
+				cycles_per_buf,
+				cycles_per_buf / test_data_szs[i]);
+
+		for (j = 0; j < MAX_NUM_OPS_INFLIGHT; j++) {
+			tcase = ut_params->test_datas[j];
+
+			switch_aead_enc_to_dec(&tdata, tcase, sgl_option);
+			ret = assemble_aead_buf(tcase, obj, j,
+					RTE_CRYPTO_AEAD_OP_DECRYPT,
+					&tdata, sgl_option, 0);
+			if (ret < 0) {
+				printf("Test is not supported by the driver\n");
+				return ret;
+			}
+		}
+
+		time_start = rte_get_timer_cycles();
+
+		run_test(ts_params->ctx, ut_params->sess, obj,
+				MAX_NUM_OPS_INFLIGHT);
+
+		time_now = rte_get_timer_cycles();
+
+		rate = time_now - time_start;
+		cycles_per_buf = rate / MAX_NUM_OPS_INFLIGHT;
+
+		rate = ((hz / cycles_per_buf)) / 1000000;
+
+		printf("AES-GCM-%u(%4uB) Dec %03.3fMpps (%03.3fGbps) ",
+				key_sz * 8, test_data_szs[i], rate,
+				rate  * test_data_szs[i] * 8 / 1000);
+		printf("cycles per buf %03.3f per byte %03.3f\n",
+				cycles_per_buf,
+				cycles_per_buf / test_data_szs[i]);
+	}
+
+	return 0;
+}
+
+/* test-perfix/key-size/sgl-type */
+#define all_gcm_perf_test_cases(type)					\
+	TEST_EXPAND(_128, 16, type)					\
+	TEST_EXPAND(_192, 24, type)					\
+	TEST_EXPAND(_256, 32, type)
+
+#define TEST_EXPAND(a, b, c)						\
+static int								\
+cpu_crypto_gcm_perf##a##_##c(void)					\
+{									\
+	return cpu_crypto_test_aead_perf(c, b);				\
+}									\
+
+all_gcm_perf_test_cases(SGL_ONE_SEG)
+all_gcm_perf_test_cases(SGL_MAX_SEG)
+#undef TEST_EXPAND
+
+static struct unit_test_suite security_cpu_crypto_aesgcm_perf_testsuite  = {
+		.suite_name = "Security CPU Crypto AESNI-GCM Perf Test Suite",
+		.setup = testsuite_setup,
+		.teardown = testsuite_teardown,
+		.unit_test_cases = {
+#define TEST_EXPAND(a, b, c)						\
+		TEST_CASE_ST(ut_setup, ut_teardown,			\
+				cpu_crypto_gcm_perf##a##_##c),		\
+
+		all_gcm_perf_test_cases(SGL_ONE_SEG)
+		all_gcm_perf_test_cases(SGL_MAX_SEG)
+#undef TEST_EXPAND
+
+		TEST_CASES_END() /**< NULL terminate unit test array */
+		},
+};
+
+static int
+test_security_cpu_crypto_aesni_gcm_perf(void)
+{
+	gbl_driver_id =	rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD));
+
+	return unit_test_suite_runner(
+			&security_cpu_crypto_aesgcm_perf_testsuite);
+}
+
 REGISTER_TEST_COMMAND(security_aesni_gcm_autotest,
 		test_security_cpu_crypto_aesni_gcm);
+
+REGISTER_TEST_COMMAND(security_aesni_gcm_perftest,
+		test_security_cpu_crypto_aesni_gcm_perf);
Signature · 1 line
2.14.5
author
#44 lore raw

RFC PATCH 5/9 crypto/aesni_mb: add rte_security handler

In reply to Fan Zhang

This patch add rte_security support support to AESNI-MB PMD. The PMD now initialize security context instance, create/delete PMD specific security sessions, and process crypto workloads in synchronous mode.

---
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c         | 291 ++++++++++++++++++++-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c     |  91 ++++++-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h |  21 +-
 3 files changed, 398 insertions(+), 5 deletions(-)
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +290 -1
index b495a9679..68767c04e 100644
@@ -8,6 +8,8 @@
 #include <rte_hexdump.h>
 #include <rte_cryptodev.h>
 #include <rte_cryptodev_pmd.h>
+#include <rte_security.h>
+#include <rte_security_driver.h>
 #include <rte_bus_vdev.h>
 #include <rte_malloc.h>
 #include <rte_cpuflags.h>
@@ -789,6 +791,167 @@ auth_start_offset(struct rte_crypto_op *op, struct aesni_mb_session *session,
 			(UINT64_MAX - u_src + u_dst + 1);
 }
 
+union sec_userdata_field {
+	int status;
+	struct {
+		uint16_t is_gen_digest;
+		uint16_t digest_len;
+	};
+};
+
+struct sec_udata_digest_field {
+	uint32_t is_digest_gen;
+	uint32_t digest_len;
+};
+
+static inline int
+set_mb_job_params_sec(JOB_AES_HMAC *job, struct aesni_mb_sec_session *sec_sess,
+		void *buf, uint32_t buf_len, void *iv, void *aad, void *digest,
+		int *status, uint8_t *digest_idx)
+{
+	struct aesni_mb_session *session = &sec_sess->sess;
+	uint32_t cipher_offset = sec_sess->cipher_offset;
+	void *user_digest = NULL;
+	union sec_userdata_field udata;
+
+	if (unlikely(cipher_offset > buf_len))
+		return -EINVAL;
+
+	/* Set crypto operation */
+	job->chain_order = session->chain_order;
+
+	/* Set cipher parameters */
+	job->cipher_direction = session->cipher.direction;
+	job->cipher_mode = session->cipher.mode;
+
+	job->aes_key_len_in_bytes = session->cipher.key_length_in_bytes;
+
+	/* Set authentication parameters */
+	job->hash_alg = session->auth.algo;
+	job->iv = iv;
+
+	switch (job->hash_alg) {
+	case AES_XCBC:
+		job->u.XCBC._k1_expanded = session->auth.xcbc.k1_expanded;
+		job->u.XCBC._k2 = session->auth.xcbc.k2;
+		job->u.XCBC._k3 = session->auth.xcbc.k3;
+
+		job->aes_enc_key_expanded =
+				session->cipher.expanded_aes_keys.encode;
+		job->aes_dec_key_expanded =
+				session->cipher.expanded_aes_keys.decode;
+		break;
+
+	case AES_CCM:
+		job->u.CCM.aad = (uint8_t *)aad + 18;
+		job->u.CCM.aad_len_in_bytes = session->aead.aad_len;
+		job->aes_enc_key_expanded =
+				session->cipher.expanded_aes_keys.encode;
+		job->aes_dec_key_expanded =
+				session->cipher.expanded_aes_keys.decode;
+		job->iv++;
+		break;
+
+	case AES_CMAC:
+		job->u.CMAC._key_expanded = session->auth.cmac.expkey;
+		job->u.CMAC._skey1 = session->auth.cmac.skey1;
+		job->u.CMAC._skey2 = session->auth.cmac.skey2;
+		job->aes_enc_key_expanded =
+				session->cipher.expanded_aes_keys.encode;
+		job->aes_dec_key_expanded =
+				session->cipher.expanded_aes_keys.decode;
+		break;
+
+	case AES_GMAC:
+		if (session->cipher.mode == GCM) {
+			job->u.GCM.aad = aad;
+			job->u.GCM.aad_len_in_bytes = session->aead.aad_len;
+		} else {
+			/* For GMAC */
+			job->u.GCM.aad = aad;
+			job->u.GCM.aad_len_in_bytes = buf_len;
+			job->cipher_mode = GCM;
+		}
+		job->aes_enc_key_expanded = &session->cipher.gcm_key;
+		job->aes_dec_key_expanded = &session->cipher.gcm_key;
+		break;
+
+	default:
+		job->u.HMAC._hashed_auth_key_xor_ipad =
+				session->auth.pads.inner;
+		job->u.HMAC._hashed_auth_key_xor_opad =
+				session->auth.pads.outer;
+
+		if (job->cipher_mode == DES3) {
+			job->aes_enc_key_expanded =
+				session->cipher.exp_3des_keys.ks_ptr;
+			job->aes_dec_key_expanded =
+				session->cipher.exp_3des_keys.ks_ptr;
+		} else {
+			job->aes_enc_key_expanded =
+				session->cipher.expanded_aes_keys.encode;
+			job->aes_dec_key_expanded =
+				session->cipher.expanded_aes_keys.decode;
+		}
+	}
+
+	/* Set digest output location */
+	if (job->hash_alg != NULL_HASH &&
+			session->auth.operation == RTE_CRYPTO_AUTH_OP_VERIFY) {
+		job->auth_tag_output = sec_sess->temp_digests[*digest_idx];
+		*digest_idx = (*digest_idx + 1) % MAX_JOBS;
+
+		udata.is_gen_digest = 0;
+		udata.digest_len = session->auth.req_digest_len;
+		user_digest = (void *)digest;
+	} else {
+		udata.is_gen_digest = 1;
+		udata.digest_len = session->auth.req_digest_len;
+
+		if (session->auth.req_digest_len !=
+				session->auth.gen_digest_len) {
+			job->auth_tag_output =
+					sec_sess->temp_digests[*digest_idx];
+			*digest_idx = (*digest_idx + 1) % MAX_JOBS;
+
+			user_digest = (void *)digest;
+		} else
+			job->auth_tag_output = digest;
+
+		/* A bit of hack here, since job structure only supports
+		 * 2 user data fields and we need 4 params to be passed
+		 * (status, direction, digest for verify, and length of
+		 * digest), we set the status value as digest length +
+		 * direction here temporarily to avoid creating longer
+		 * buffer to store all 4 params.
+		 */
+		*status = udata.status;
+	}
+	/*
+	 * Multi-buffer library current only support returning a truncated
+	 * digest length as specified in the relevant IPsec RFCs
+	 */
+
+	/* Set digest length */
+	job->auth_tag_output_len_in_bytes = session->auth.gen_digest_len;
+
+	/* Set IV parameters */
+	job->iv_len_in_bytes = session->iv.length;
+
+	/* Data Parameters */
+	job->src = buf;
+	job->dst = buf;
+	job->cipher_start_src_offset_in_bytes = cipher_offset;
+	job->msg_len_to_cipher_in_bytes = buf_len - cipher_offset;
+	job->hash_start_src_offset_in_bytes = 0;
+	job->msg_len_to_hash_in_bytes = buf_len;
+
+	job->user_data = (void *)status;
+	job->user_data2 = user_digest;
+
+	return 0;
+}
+
 /**
  * Process a crypto operation and complete a JOB_AES_HMAC job structure for
  * submission to the multi buffer library for processing.
@@ -1081,6 +1244,37 @@ post_process_mb_job(struct aesni_mb_qp *qp, JOB_AES_HMAC *job)
 	return op;
 }
 
+static inline void
+post_process_mb_sec_job(JOB_AES_HMAC *job)
+{
+	void *user_digest = job->user_data2;
+	int *status = job->user_data;
+	union sec_userdata_field udata;
+
+	switch (job->status) {
+	case STS_COMPLETED:
+		if (user_digest) {
+			udata.status = *status;
+
+			if (udata.is_gen_digest) {
+				*status = RTE_CRYPTO_OP_STATUS_SUCCESS;
+				memcpy(user_digest, job->auth_tag_output,
+						udata.digest_len);
+			} else {
+				verify_digest(job, user_digest,
+					udata.digest_len, (uint8_t *)status);
+
+				if (*status == RTE_CRYPTO_OP_STATUS_AUTH_FAILED)
+					*status = -1;
+			}
+		} else
+			*status = RTE_CRYPTO_OP_STATUS_SUCCESS;
+		break;
+	default:
+		*status = RTE_CRYPTO_OP_STATUS_ERROR;
+	}
+}
+
 /**
  * Process a completed JOB_AES_HMAC job and keep processing jobs until
  * get_completed_job return NULL
@@ -1117,6 +1311,32 @@ handle_completed_jobs(struct aesni_mb_qp *qp, JOB_AES_HMAC *job,
 	return processed_jobs;
 }
 
+static inline uint32_t
+handle_completed_sec_jobs(JOB_AES_HMAC *job, MB_MGR *mb_mgr)
+{
+	uint32_t processed = 0;
+
+	while (job != NULL) {
+		post_process_mb_sec_job(job);
+		job = IMB_GET_COMPLETED_JOB(mb_mgr);
+		processed++;
+	}
+
+	return processed;
+}
+
+static inline uint32_t
+flush_mb_sec_mgr(MB_MGR *mb_mgr)
+{
+	JOB_AES_HMAC *job = IMB_FLUSH_JOB(mb_mgr);
+	uint32_t processed = 0;
+
+	if (job)
+		processed = handle_completed_sec_jobs(job, mb_mgr);
+
+	return processed;
+}
+
 static inline uint16_t
 flush_mb_mgr(struct aesni_mb_qp *qp, struct rte_crypto_op **ops,
 		uint16_t nb_ops)
@@ -1220,6 +1440,55 @@ aesni_mb_pmd_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops,
 	return processed_jobs;
 }
 
+void
+aesni_mb_sec_crypto_process_bulk(struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num)
+{
+	struct aesni_mb_sec_session *sec_sess = sess->sess_private_data;
+	JOB_AES_HMAC *job;
+	uint8_t digest_idx = sec_sess->digest_idx;
+	uint32_t i, processed = 0;
+	int ret;
+
+	for (i = 0; i < num; i++) {
+		void *seg_buf = buf[i].vec[0].iov_base;
+		uint32_t buf_len = buf[i].vec[0].iov_len;
+
+		job = IMB_GET_NEXT_JOB(sec_sess->mb_mgr);
+		if (unlikely(job == NULL)) {
+			processed += flush_mb_sec_mgr(sec_sess->mb_mgr);
+
+			job = IMB_GET_NEXT_JOB(sec_sess->mb_mgr);
+			if (!job)
+				return;
+		}
+
+		ret = set_mb_job_params_sec(job, sec_sess, seg_buf, buf_len,
+				iv[i], aad[i], digest[i], &status[i],
+				&digest_idx);
+				/* Submit job to multi-buffer for processing */
+		if (ret) {
+			processed++;
+			status[i] = ret;
+			continue;
+		}
+
+#ifdef RTE_LIBRTE_PMD_AESNI_MB_DEBUG
+		job = IMB_SUBMIT_JOB(sec_sess->mb_mgr);
+#else
+		job = IMB_SUBMIT_JOB_NOCHECK(sec_sess->mb_mgr);
+#endif
+
+		if (job)
+			processed += handle_completed_sec_jobs(job,
+					sec_sess->mb_mgr);
+	}
+
+	while (processed < num)
+		processed += flush_mb_sec_mgr(sec_sess->mb_mgr);
+}
+
 static int cryptodev_aesni_mb_remove(struct rte_vdev_device *vdev);
 
 static int
@@ -1229,8 +1498,10 @@ cryptodev_aesni_mb_create(const char *name,
 {
 	struct rte_cryptodev *dev;
 	struct aesni_mb_private *internals;
+	struct rte_security_ctx *sec_ctx;
 	enum aesni_mb_vector_mode vector_mode;
 	MB_MGR *mb_mgr;
+	char sec_name[RTE_DEV_NAME_MAX_LEN];
 
 	/* Check CPU for support for AES instruction set */
 	if (!rte_cpu_get_flag_enabled(RTE_CPUFLAG_AES)) {
@@ -1264,7 +1535,8 @@ cryptodev_aesni_mb_create(const char *name,
 	dev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO |
 			RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
 			RTE_CRYPTODEV_FF_CPU_AESNI |
-			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT;
+			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
+			RTE_CRYPTODEV_FF_SECURITY;
 
 
 	mb_mgr = alloc_mb_mgr(0);
@@ -1303,11 +1575,28 @@ cryptodev_aesni_mb_create(const char *name,
 	AESNI_MB_LOG(INFO, "IPSec Multi-buffer library version used: %s\n",
 			imb_get_version_str());
 
+	/* setup security operations */
+	snprintf(sec_name, sizeof(sec_name) - 1, "aes_mb_sec_%u",
+			dev->driver_id);
+	sec_ctx = rte_zmalloc_socket(sec_name,
+			sizeof(struct rte_security_ctx),
+			RTE_CACHE_LINE_SIZE, init_params->socket_id);
+	if (sec_ctx == NULL) {
+		AESNI_MB_LOG(ERR, "memory allocation failed\n");
+		goto error_exit;
+	}
+
+	sec_ctx->device = (void *)dev;
+	sec_ctx->ops = rte_aesni_mb_pmd_security_ops;
+	dev->security_ctx = sec_ctx;
+
 	return 0;
 
 error_exit:
 	if (mb_mgr)
 		free_mb_mgr(mb_mgr);
+	if (sec_ctx)
+		rte_free(sec_ctx);
 
 	rte_cryptodev_pmd_destroy(dev);
 
drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c +90 -1

107 lines not on this page · raw message

drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h +18 -3
index b794d4bc1..d1cf416ab 100644
@@ -176,7 +176,6 @@ struct aesni_mb_qp {
 	 */
 } __rte_cache_aligned;
 
-/** AES-NI multi-buffer private session structure */
 struct aesni_mb_session {
 	JOB_CHAIN_ORDER chain_order;
 	struct {
@@ -265,16 +264,32 @@ struct aesni_mb_session {
 		/** AAD data length */
 		uint16_t aad_len;
 	} aead;
-} __rte_cache_aligned;
+};
+
+/** AES-NI multi-buffer private security session structure */
+struct aesni_mb_sec_session {
+	/**< Unique Queue Pair Name */
+	struct aesni_mb_session sess;
+	uint8_t temp_digests[MAX_JOBS][DIGEST_LENGTH_MAX];
+	uint16_t digest_idx;
+	uint32_t cipher_offset;
+	MB_MGR *mb_mgr;
+};
 
 extern int
 aesni_mb_set_session_parameters(const MB_MGR *mb_mgr,
 		struct aesni_mb_session *sess,
 		const struct rte_crypto_sym_xform *xform);
 
+extern void
+aesni_mb_sec_crypto_process_bulk(struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+
 /** device specific operations function pointer structure */
 extern struct rte_cryptodev_ops *rte_aesni_mb_pmd_ops;
 
-
+/** device specific operations function pointer structure for rte_security */
+extern struct rte_security_ops *rte_aesni_mb_pmd_security_ops;
 
 #endif /* _RTE_AESNI_MB_PMD_PRIVATE_H_ */
Signature · 1 line
2.14.5
author
#45 lore raw

RFC PATCH 6/9 app/test: add aesni_mb security cpu crypto autotest

In reply to Fan Zhang

This patch adds cpu crypto unit test for AESNI_MB PMD.

---
 app/test/test_security_cpu_crypto.c | 367 ++++++++++++++++++++++++++++++++++++
 1 file changed, 367 insertions(+)
app/test/test_security_cpu_crypto.c +367 -0
index ca9a8dae6..0ea406390 100644
@@ -19,12 +19,23 @@
 
 #include "test.h"
 #include "test_cryptodev.h"
+#include "test_cryptodev_blockcipher.h"
+#include "test_cryptodev_aes_test_vectors.h"
 #include "test_cryptodev_aead_test_vectors.h"
+#include "test_cryptodev_des_test_vectors.h"
+#include "test_cryptodev_hash_test_vectors.h"
 
 #define CPU_CRYPTO_TEST_MAX_AAD_LENGTH	16
 #define MAX_NB_SIGMENTS			4
 #define CACHE_WARM_ITER			2048
 
+#define TOP_ENC		BLOCKCIPHER_TEST_OP_ENCRYPT
+#define TOP_DEC		BLOCKCIPHER_TEST_OP_DECRYPT
+#define TOP_AUTH_GEN	BLOCKCIPHER_TEST_OP_AUTH_GEN
+#define TOP_AUTH_VER	BLOCKCIPHER_TEST_OP_AUTH_VERIFY
+#define TOP_ENC_AUTH	BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN
+#define TOP_AUTH_DEC	BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC
+
 enum buffer_assemble_option {
 	SGL_MAX_SEG,
 	SGL_ONE_SEG,
@@ -516,6 +527,11 @@ cpu_crypto_test_aead(const struct aead_test_data *tdata,
 	TEST_EXPAND(gcm_test_case_256_6, type)	\
 	TEST_EXPAND(gcm_test_case_256_7, type)
 
+/* test-vector/sgl-option */
+#define all_ccm_unit_test_cases \
+	TEST_EXPAND(ccm_test_case_128_1, SGL_ONE_SEG) \
+	TEST_EXPAND(ccm_test_case_128_2, SGL_ONE_SEG) \
+	TEST_EXPAND(ccm_test_case_128_3, SGL_ONE_SEG)
 
 #define TEST_EXPAND(t, o)						\
 static int								\
@@ -531,6 +547,7 @@ cpu_crypto_aead_dec_test_##t##_##o(void)				\
 
 all_gcm_unit_test_cases(SGL_ONE_SEG)
 all_gcm_unit_test_cases(SGL_MAX_SEG)
+all_ccm_unit_test_cases
 #undef TEST_EXPAND
 
 static struct unit_test_suite security_cpu_crypto_aesgcm_testsuite  = {
@@ -758,8 +775,358 @@ test_security_cpu_crypto_aesni_gcm_perf(void)
 			&security_cpu_crypto_aesgcm_perf_testsuite);
 }
 
+static struct rte_security_session *
+create_blockcipher_session(struct rte_security_ctx *ctx,
+		struct rte_mempool *sess_mp,
+		uint32_t op_mask,
+		const struct blockcipher_test_data *test_data,
+		uint32_t is_unit_test)
+{
+	struct rte_security_session_conf sess_conf = {0};
+	struct rte_crypto_sym_xform xforms[2] = { {0} };
+	struct rte_crypto_sym_xform *cipher_xform = NULL;
+	struct rte_crypto_sym_xform *auth_xform = NULL;
+	struct rte_crypto_sym_xform *xform;
+
+	if (op_mask & BLOCKCIPHER_TEST_OP_CIPHER) {
+		cipher_xform = &xforms[0];
+		cipher_xform->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
+
+		if (op_mask & TOP_ENC)
+			cipher_xform->cipher.op =
+				RTE_CRYPTO_CIPHER_OP_ENCRYPT;
+		else
+			cipher_xform->cipher.op =
+				RTE_CRYPTO_CIPHER_OP_DECRYPT;
+
+		cipher_xform->cipher.algo = test_data->crypto_algo;
+		cipher_xform->cipher.key.data = test_data->cipher_key.data;
+		cipher_xform->cipher.key.length = test_data->cipher_key.len;
+		cipher_xform->cipher.iv.offset = 0;
+		cipher_xform->cipher.iv.length = test_data->iv.len;
+
+		if (is_unit_test)
+			debug_hexdump(stdout, "cipher key:",
+					test_data->cipher_key.data,
+					test_data->cipher_key.len);
+	}
+
+	if (op_mask & BLOCKCIPHER_TEST_OP_AUTH) {
+		auth_xform = &xforms[1];
+		auth_xform->type = RTE_CRYPTO_SYM_XFORM_AUTH;
+
+		if (op_mask & TOP_AUTH_GEN)
+			auth_xform->auth.op = RTE_CRYPTO_AUTH_OP_GENERATE;
+		else
+			auth_xform->auth.op = RTE_CRYPTO_AUTH_OP_VERIFY;
+
+		auth_xform->auth.algo = test_data->auth_algo;
+		auth_xform->auth.key.length = test_data->auth_key.len;
+		auth_xform->auth.key.data = test_data->auth_key.data;
+		auth_xform->auth.digest_length = test_data->digest.len;
+
+		if (is_unit_test)
+			debug_hexdump(stdout, "auth key:",
+					test_data->auth_key.data,
+					test_data->auth_key.len);
+	}
+
+	if (op_mask == TOP_ENC ||
+			op_mask == TOP_DEC)
+		xform = cipher_xform;
+	else if (op_mask == TOP_AUTH_GEN ||
+			op_mask == TOP_AUTH_VER)
+		xform = auth_xform;
+	else if (op_mask == TOP_ENC_AUTH) {
+		xform = cipher_xform;
+		xform->next = auth_xform;
+	} else if (op_mask == TOP_AUTH_DEC) {
+		xform = auth_xform;
+		xform->next = cipher_xform;
+	} else
+		return NULL;
+
+	if (test_data->cipher_offset < test_data->auth_offset)
+		return NULL;
+
+	sess_conf.action_type = RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO;
+	sess_conf.crypto_xform = xform;
+	sess_conf.cpucrypto.cipher_offset = test_data->cipher_offset -
+			test_data->auth_offset;
+
+	return rte_security_session_create(ctx, &sess_conf, sess_mp);
+}
+
+static inline int
+assemble_blockcipher_buf(struct cpu_crypto_test_case *data,
+		struct cpu_crypto_test_obj *obj,
+		uint32_t obj_idx,
+		uint32_t op_mask,
+		const struct blockcipher_test_data *test_data,
+		uint32_t is_unit_test)
+{
+	const uint8_t *src;
+	uint32_t src_len;
+	uint32_t offset;
+
+	if (op_mask == TOP_ENC_AUTH ||
+			op_mask == TOP_AUTH_GEN ||
+			op_mask == BLOCKCIPHER_TEST_OP_AUTH_VERIFY)
+		offset = test_data->auth_offset;
+	else
+		offset = test_data->cipher_offset;
+
+	if (op_mask & TOP_ENC_AUTH) {
+		src = test_data->plaintext.data;
+		src_len = test_data->plaintext.len;
+		if (is_unit_test)
+			debug_hexdump(stdout, "plaintext:", src, src_len);
+	} else {
+		src = test_data->ciphertext.data;
+		src_len = test_data->ciphertext.len;
+		memcpy(data->digest, test_data->digest.data,
+				test_data->digest.len);
+		if (is_unit_test) {
+			debug_hexdump(stdout, "ciphertext:", src, src_len);
+			debug_hexdump(stdout, "digest:", test_data->digest.data,
+					test_data->digest.len);
+		}
+	}
+
+	if (src_len > MBUF_DATAPAYLOAD_SIZE)
+		return -ENOMEM;
+
+	memcpy(data->seg_buf[0].seg, src, src_len);
+	data->seg_buf[0].seg_len = src_len;
+	obj->vec[obj_idx][0].iov_base =
+			(void *)(data->seg_buf[0].seg + offset);
+	obj->vec[obj_idx][0].iov_len = src_len - offset;
+
+	obj->sec_buf[obj_idx].vec = obj->vec[obj_idx];
+	obj->sec_buf[obj_idx].num = 1;
+
+	memcpy(data->iv, test_data->iv.data, test_data->iv.len);
+	if (is_unit_test)
+		debug_hexdump(stdout, "iv:", test_data->iv.data,
+				test_data->iv.len);
+
+	obj->iv[obj_idx] = (void *)data->iv;
+	obj->digest[obj_idx] = (void *)data->digest;
+
+	return 0;
+}
+
+static int
+check_blockcipher_result(struct cpu_crypto_test_case *tcase,
+		uint32_t op_mask,
+		const struct blockcipher_test_data *test_data)
+{
+	int ret;
+
+	if (op_mask & BLOCKCIPHER_TEST_OP_CIPHER) {
+		const char *err_msg1, *err_msg2;
+		const uint8_t *src_pt_ct;
+		uint32_t src_len;
+
+		if (op_mask & TOP_ENC) {
+			src_pt_ct = test_data->ciphertext.data;
+			src_len = test_data->ciphertext.len;
+			err_msg1 = CPU_CRYPTO_ERR_EXP_CT;
+			err_msg2 = CPU_CRYPTO_ERR_GEN_CT;
+		} else {
+			src_pt_ct = test_data->plaintext.data;
+			src_len = test_data->plaintext.len;
+			err_msg1 = CPU_CRYPTO_ERR_EXP_PT;
+			err_msg2 = CPU_CRYPTO_ERR_GEN_PT;
+		}
+
+		ret = memcmp(tcase->seg_buf[0].seg, src_pt_ct, src_len);
+		if (ret != 0) {
+			debug_hexdump(stdout, err_msg1, src_pt_ct, src_len);
+			debug_hexdump(stdout, err_msg2,
+					tcase->seg_buf[0].seg,
+					test_data->ciphertext.len);
+			return -1;
+		}
+	}
+
+	if (op_mask & TOP_AUTH_GEN) {
+		ret = memcmp(tcase->digest, test_data->digest.data,
+				test_data->digest.len);
+		if (ret != 0) {
+			debug_hexdump(stdout, "expect digest:",
+					test_data->digest.data,
+					test_data->digest.len);
+			debug_hexdump(stdout, "gen digest:",
+					tcase->digest,
+					test_data->digest.len);
+			return -1;
+		}
+	}
+
+	return 0;
+}
+
+static int
+cpu_crypto_test_blockcipher(const struct blockcipher_test_data *tdata,
+		uint32_t op_mask)
+{
+	struct cpu_crypto_testsuite_params *ts_params = &testsuite_params;
+	struct cpu_crypto_unittest_params *ut_params = &unittest_params;
+	struct cpu_crypto_test_obj *obj = &ut_params->test_obj;
+	struct cpu_crypto_test_case *tcase;
+	int ret;
+
+	ut_params->sess = create_blockcipher_session(ts_params->ctx,
+			ts_params->session_priv_mpool,
+			op_mask,
+			tdata,
+			1);
+	if (!ut_params->sess)
+		return -1;
+
+	ret = allocate_buf(1);
+	if (ret)
+		return ret;
+
+	tcase = ut_params->test_datas[0];
+	ret = assemble_blockcipher_buf(tcase, obj, 0, op_mask, tdata, 1);
+	if (ret < 0) {
+		printf("Test is not supported by the driver\n");
+		return ret;
+	}
+
+	run_test(ts_params->ctx, ut_params->sess, obj, 1);
+
+	ret = check_status(obj, 1);
+	if (ret < 0)
+		return ret;
+
+	ret = check_blockcipher_result(tcase, op_mask, tdata);
+	if (ret < 0)
+		return ret;
+
+	return 0;
+}
+
+/* Macro to save code for defining BlockCipher test cases */
+/* test-vector-name/op */
+#define all_blockcipher_test_cases \
+	TEST_EXPAND(aes_test_data_1, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_1, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_1, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_1, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_2, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_2, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_2, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_2, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_3, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_3, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_3, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_3, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_4, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_4, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_4, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_4, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_5, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_5, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_5, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_5, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_6, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_6, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_6, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_6, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_7, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_7, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_7, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_7, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_8, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_8, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_8, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_8, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_9, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_9, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_9, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_9, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_10, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_10, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_11, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_11, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_12, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_12, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_12, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_12, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_13, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_13, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_13, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_13, TOP_AUTH_DEC) \
+	TEST_EXPAND(des_test_data_1, TOP_ENC) \
+	TEST_EXPAND(des_test_data_1, TOP_DEC) \
+	TEST_EXPAND(des_test_data_2, TOP_ENC) \
+	TEST_EXPAND(des_test_data_2, TOP_DEC) \
+	TEST_EXPAND(des_test_data_3, TOP_ENC) \
+	TEST_EXPAND(des_test_data_3, TOP_DEC) \
+	TEST_EXPAND(triple_des128cbc_hmac_sha1_test_vector, TOP_ENC) \
+	TEST_EXPAND(triple_des128cbc_hmac_sha1_test_vector, TOP_DEC) \
+	TEST_EXPAND(triple_des128cbc_hmac_sha1_test_vector, TOP_ENC_AUTH) \
+	TEST_EXPAND(triple_des128cbc_hmac_sha1_test_vector, TOP_AUTH_DEC) \
+	TEST_EXPAND(triple_des64cbc_test_vector, TOP_ENC) \
+	TEST_EXPAND(triple_des64cbc_test_vector, TOP_DEC) \
+	TEST_EXPAND(triple_des128cbc_test_vector, TOP_ENC) \
+	TEST_EXPAND(triple_des128cbc_test_vector, TOP_DEC) \
+	TEST_EXPAND(triple_des192cbc_test_vector, TOP_ENC) \
+	TEST_EXPAND(triple_des192cbc_test_vector, TOP_DEC) \
+
+#define TEST_EXPAND(t, o)						\
+static int								\
+cpu_crypto_blockcipher_test_##t##_##o(void)				\
+{									\
+	return cpu_crypto_test_blockcipher(&t, o);			\
+}
+
+all_blockcipher_test_cases
+#undef TEST_EXPAND
+
+static struct unit_test_suite security_cpu_crypto_aesni_mb_testsuite  = {
+	.suite_name = "Security CPU Crypto AESNI-MB Unit Test Suite",
+	.setup = testsuite_setup,
+	.teardown = testsuite_teardown,
+	.unit_test_cases = {
+#define TEST_EXPAND(t, o)						\
+	TEST_CASE_ST(ut_setup, ut_teardown,				\
+			cpu_crypto_aead_enc_test_##t##_##o),		\
+	TEST_CASE_ST(ut_setup, ut_teardown,				\
+			cpu_crypto_aead_dec_test_##t##_##o),		\
+
+	all_gcm_unit_test_cases(SGL_ONE_SEG)
+	all_ccm_unit_test_cases
+#undef TEST_EXPAND
+
+#define TEST_EXPAND(t, o)						\
+	TEST_CASE_ST(ut_setup, ut_teardown,				\
+			cpu_crypto_blockcipher_test_##t##_##o),		\
+
+	all_blockcipher_test_cases
+#undef TEST_EXPAND
+
+	TEST_CASES_END() /**< NULL terminate unit test array */
+	},
+};
+
+static int
+test_security_cpu_crypto_aesni_mb(void)
+{
+	gbl_driver_id =	rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
+
+	return unit_test_suite_runner(&security_cpu_crypto_aesni_mb_testsuite);
+}
+
 REGISTER_TEST_COMMAND(security_aesni_gcm_autotest,
 		test_security_cpu_crypto_aesni_gcm);
 
 REGISTER_TEST_COMMAND(security_aesni_gcm_perftest,
 		test_security_cpu_crypto_aesni_gcm_perf);
+
+REGISTER_TEST_COMMAND(security_aesni_mb_autotest,
+		test_security_cpu_crypto_aesni_mb);
Signature · 1 line
2.14.5
author
#46 lore raw

RFC PATCH 7/9 app/test: add aesni_mb security cpu crypto perftest

In reply to Fan Zhang

Since crypto perf application does not support rte_security, this patch adds a simple AES-CBC-SHA1-HMAC CPU crypto performance test to crypto unittest application. The test includes different key and data sizes test with single buffer test items and will display the throughput as well as cycle count performance information.

---
 app/test/test_security_cpu_crypto.c | 194 ++++++++++++++++++++++++++++++++++++
 1 file changed, 194 insertions(+)
app/test/test_security_cpu_crypto.c +194 -0
index 0ea406390..6e012672e 100644
@@ -1122,6 +1122,197 @@ test_security_cpu_crypto_aesni_mb(void)
 	return unit_test_suite_runner(&security_cpu_crypto_aesni_mb_testsuite);
 }
 
+static inline void
+switch_blockcipher_enc_to_dec(struct blockcipher_test_data *tdata,
+		struct cpu_crypto_test_case *tcase, uint8_t *dst)
+{
+	memcpy(dst, tcase->seg_buf[0].seg, tcase->seg_buf[0].seg_len);
+	tdata->ciphertext.len = tcase->seg_buf[0].seg_len;
+	memcpy(tdata->digest.data, tcase->digest, tdata->digest.len);
+}
+
+static int
+cpu_crypto_test_blockcipher_perf(
+		const enum rte_crypto_cipher_algorithm cipher_algo,
+		uint32_t cipher_key_sz,
+		const enum rte_crypto_auth_algorithm auth_algo,
+		uint32_t auth_key_sz, uint32_t digest_sz,
+		uint32_t op_mask)
+{
+	struct blockcipher_test_data tdata = {0};
+	uint8_t plaintext[3000], ciphertext[3000];
+	struct cpu_crypto_testsuite_params *ts_params = &testsuite_params;
+	struct cpu_crypto_unittest_params *ut_params = &unittest_params;
+	struct cpu_crypto_test_obj *obj = &ut_params->test_obj;
+	struct cpu_crypto_test_case *tcase;
+	uint64_t hz = rte_get_tsc_hz(), time_start, time_now;
+	double rate, cycles_per_buf;
+	uint32_t test_data_szs[] = {64, 128, 256, 512, 1024, 2048};
+	uint32_t i, j;
+	uint32_t op_mask_opp = 0;
+	int ret;
+
+	if (op_mask & BLOCKCIPHER_TEST_OP_CIPHER)
+		op_mask_opp |= (~op_mask & BLOCKCIPHER_TEST_OP_CIPHER);
+	if (op_mask & BLOCKCIPHER_TEST_OP_AUTH)
+		op_mask_opp |= (~op_mask & BLOCKCIPHER_TEST_OP_AUTH);
+
+	tdata.plaintext.data = plaintext;
+	tdata.ciphertext.data = ciphertext;
+
+	tdata.cipher_key.len = cipher_key_sz;
+	tdata.auth_key.len = auth_key_sz;
+
+	gen_rand(tdata.cipher_key.data, cipher_key_sz / 8);
+	gen_rand(tdata.auth_key.data, auth_key_sz / 8);
+
+	tdata.crypto_algo = cipher_algo;
+	tdata.auth_algo = auth_algo;
+
+	tdata.digest.len = digest_sz;
+
+	ut_params->sess = create_blockcipher_session(ts_params->ctx,
+			ts_params->session_priv_mpool,
+			op_mask,
+			&tdata,
+			0);
+	if (!ut_params->sess)
+		return -1;
+
+	ret = allocate_buf(MAX_NUM_OPS_INFLIGHT);
+	if (ret)
+		return ret;
+
+	for (i = 0; i < RTE_DIM(test_data_szs); i++) {
+		for (j = 0; j < MAX_NUM_OPS_INFLIGHT; j++) {
+			tdata.plaintext.len = test_data_szs[i];
+			gen_rand(plaintext, tdata.plaintext.len);
+
+			tdata.iv.len = 16;
+			gen_rand(tdata.iv.data, tdata.iv.len);
+
+			tcase = ut_params->test_datas[j];
+			ret = assemble_blockcipher_buf(tcase, obj, j,
+					op_mask,
+					&tdata,
+					0);
+			if (ret < 0) {
+				printf("Test is not supported by the driver\n");
+				return ret;
+			}
+		}
+
+		/* warm up cache */
+		for (j = 0; j < CACHE_WARM_ITER; j++)
+			run_test(ts_params->ctx, ut_params->sess, obj,
+					MAX_NUM_OPS_INFLIGHT);
+
+		time_start = rte_rdtsc();
+
+		run_test(ts_params->ctx, ut_params->sess, obj,
+				MAX_NUM_OPS_INFLIGHT);
+
+		time_now = rte_rdtsc();
+
+		rate = time_now - time_start;
+		cycles_per_buf = rate / MAX_NUM_OPS_INFLIGHT;
+
+		rate = ((hz / cycles_per_buf)) / 1000000;
+
+		printf("%s-%u-%s(%4uB) Enc %03.3fMpps (%03.3fGbps) ",
+			rte_crypto_cipher_algorithm_strings[cipher_algo],
+			cipher_key_sz * 8,
+			rte_crypto_auth_algorithm_strings[auth_algo],
+			test_data_szs[i],
+			rate, rate  * test_data_szs[i] * 8 / 1000);
+		printf("cycles per buf %03.3f per byte %03.3f\n",
+			cycles_per_buf, cycles_per_buf / test_data_szs[i]);
+
+		for (j = 0; j < MAX_NUM_OPS_INFLIGHT; j++) {
+			tcase = ut_params->test_datas[j];
+
+			switch_blockcipher_enc_to_dec(&tdata, tcase,
+					ciphertext);
+			ret = assemble_blockcipher_buf(tcase, obj, j,
+					op_mask_opp,
+					&tdata,
+					0);
+			if (ret < 0) {
+				printf("Test is not supported by the driver\n");
+				return ret;
+			}
+		}
+
+		time_start = rte_get_timer_cycles();
+
+		run_test(ts_params->ctx, ut_params->sess, obj,
+				MAX_NUM_OPS_INFLIGHT);
+
+		time_now = rte_get_timer_cycles();
+
+		rate = time_now - time_start;
+		cycles_per_buf = rate / MAX_NUM_OPS_INFLIGHT;
+
+		rate = ((hz / cycles_per_buf)) / 1000000;
+
+		printf("%s-%u-%s(%4uB) Dec %03.3fMpps (%03.3fGbps) ",
+			rte_crypto_cipher_algorithm_strings[cipher_algo],
+			cipher_key_sz * 8,
+			rte_crypto_auth_algorithm_strings[auth_algo],
+			test_data_szs[i],
+			rate, rate  * test_data_szs[i] * 8 / 1000);
+		printf("cycles per buf %03.3f per byte %03.3f\n",
+				cycles_per_buf,
+				cycles_per_buf / test_data_szs[i]);
+	}
+
+	return 0;
+}
+
+/* cipher-algo/cipher-key-len/auth-algo/auth-key-len/digest-len/op */
+#define all_block_cipher_perf_test_cases				\
+	TEST_EXPAND(_AES_CBC, 128, _NULL, 0, 0, TOP_ENC)		\
+	TEST_EXPAND(_NULL, 0, _SHA1_HMAC, 160, 20, TOP_AUTH_GEN)	\
+	TEST_EXPAND(_AES_CBC, 128, _SHA1_HMAC, 160, 20, TOP_ENC_AUTH)
+
+#define TEST_EXPAND(a, b, c, d, e, f)					\
+static int								\
+cpu_crypto_blockcipher_perf##a##_##b##c##_##f(void)			\
+{									\
+	return cpu_crypto_test_blockcipher_perf(RTE_CRYPTO_CIPHER##a,	\
+			b / 8, RTE_CRYPTO_AUTH##c, d / 8, e, f);	\
+}									\
+
+all_block_cipher_perf_test_cases
+#undef TEST_EXPAND
+
+static struct unit_test_suite security_cpu_crypto_aesni_mb_perf_testsuite  = {
+	.suite_name = "Security CPU Crypto AESNI-MB Perf Test Suite",
+	.setup = testsuite_setup,
+	.teardown = testsuite_teardown,
+	.unit_test_cases = {
+#define TEST_EXPAND(a, b, c, d, e, f)					\
+	TEST_CASE_ST(ut_setup, ut_teardown,				\
+		cpu_crypto_blockcipher_perf##a##_##b##c##_##f),	\
+
+	all_block_cipher_perf_test_cases
+#undef TEST_EXPAND
+
+	TEST_CASES_END() /**< NULL terminate unit test array */
+	},
+};
+
+static int
+test_security_cpu_crypto_aesni_mb_perf(void)
+{
+	gbl_driver_id =	rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
+
+	return unit_test_suite_runner(
+			&security_cpu_crypto_aesni_mb_perf_testsuite);
+}
+
+
 REGISTER_TEST_COMMAND(security_aesni_gcm_autotest,
 		test_security_cpu_crypto_aesni_gcm);
 
@@ -1130,3 +1321,6 @@ REGISTER_TEST_COMMAND(security_aesni_gcm_perftest,
 
 REGISTER_TEST_COMMAND(security_aesni_mb_autotest,
 		test_security_cpu_crypto_aesni_mb);
+
+REGISTER_TEST_COMMAND(security_aesni_mb_perftest,
+		test_security_cpu_crypto_aesni_mb_perf);
Signature · 1 line
2.14.5
author
#47 lore raw

RFC PATCH 8/9 ipsec: add rte_security cpu_crypto action support

In reply to Fan Zhang

This patch updates the ipsec library to handle the newly introduced RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO action.

---
 lib/librte_ipsec/esp_inb.c  | 174 +++++++++++++++++++++++++-
 lib/librte_ipsec/esp_outb.c | 290 +++++++++++++++++++++++++++++++++++++++++++-
 lib/librte_ipsec/sa.c       |  53 ++++++--
 lib/librte_ipsec/sa.h       |  29 +++++
 lib/librte_ipsec/ses.c      |   4 +-
 5 files changed, 539 insertions(+), 11 deletions(-)
lib/librte_ipsec/esp_inb.c +173 -1
index 8e3ecbc64..2220df0f6 100644
@@ -105,6 +105,73 @@ inb_cop_prepare(struct rte_crypto_op *cop,
 	}
 }
 
+static inline int
+inb_sync_crypto_proc_prepare(const struct rte_ipsec_sa *sa, struct rte_mbuf *mb,
+	const union sym_op_data *icv, uint32_t pofs, uint32_t plen,
+	struct rte_security_vec *buf, struct iovec *cur_vec,
+	void *iv, void **aad, void **digest)
+{
+	struct rte_mbuf *ms;
+	struct iovec *vec = cur_vec;
+	struct aead_gcm_iv *gcm;
+	struct aesctr_cnt_blk *ctr;
+	uint64_t *ivp;
+	uint32_t algo, left, off = 0, n_seg = 0;
+
+	ivp = rte_pktmbuf_mtod_offset(mb, uint64_t *,
+		pofs + sizeof(struct rte_esp_hdr));
+	algo = sa->algo_type;
+
+	switch (algo) {
+	case ALGO_TYPE_AES_GCM:
+		gcm = (struct aead_gcm_iv *)iv;
+		aead_gcm_iv_fill(gcm, ivp[0], sa->salt);
+		*aad = icv->va + sa->icv_len;
+		off = sa->ctp.cipher.offset + pofs;
+		break;
+	case ALGO_TYPE_AES_CBC:
+	case ALGO_TYPE_3DES_CBC:
+		off = sa->ctp.auth.offset + pofs;
+		break;
+	case ALGO_TYPE_AES_CTR:
+		off = sa->ctp.auth.offset + pofs;
+		ctr = (struct aesctr_cnt_blk *)iv;
+		aes_ctr_cnt_blk_fill(ctr, ivp[0], sa->salt);
+		break;
+	case ALGO_TYPE_NULL:
+		break;
+	}
+
+	*digest = icv->va;
+
+	left = plen - sa->ctp.cipher.length;
+
+	ms = mbuf_get_seg_ofs(mb, &off);
+	if (!ms)
+		return -1;
+
+	while (n_seg < RTE_LIBRTE_IP_FRAG_MAX_FRAG && left && ms) {
+		uint32_t len = RTE_MIN(left, ms->data_len - off);
+
+		vec->iov_base = rte_pktmbuf_mtod_offset(ms, void *, off);
+		vec->iov_len = len;
+
+		left -= len;
+		vec++;
+		n_seg++;
+		ms = ms->next;
+		off = 0;
+	}
+
+	if (left)
+		return -1;
+
+	buf->vec = cur_vec;
+	buf->num = n_seg;
+
+	return n_seg;
+}
+
 /*
  * Helper function for prepare() to deal with situation when
  * ICV is spread by two segments. Tries to move ICV completely into the
@@ -512,7 +579,6 @@ tun_process(const struct rte_ipsec_sa *sa, struct rte_mbuf *mb[],
 	return k;
 }
 
-
 /*
  * *process* function for tunnel packets
  */
@@ -625,6 +691,112 @@ esp_inb_pkt_process(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[],
 	return n;
 }
 
+/*
+ * process packets using sync crypto engine
+ */
+static uint16_t
+esp_inb_sync_crypto_pkt_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num, uint8_t sqh_len,
+		esp_inb_process_t process)
+{
+	int32_t rc;
+	uint32_t i, k, hl, n, p;
+	struct rte_ipsec_sa *sa;
+	struct replay_sqn *rsn;
+	union sym_op_data icv;
+	uint32_t sqn[num];
+	uint32_t dr[num];
+	struct rte_security_vec buf[num];
+	struct iovec vec[RTE_LIBRTE_IP_FRAG_MAX_FRAG * num];
+	uint32_t vec_idx = 0;
+	uint8_t ivs[num][IPSEC_MAX_IV_SIZE];
+	void *iv[num];
+	void *aad[num];
+	void *digest[num];
+	int status[num];
+
+	sa = ss->sa;
+	rsn = rsn_acquire(sa);
+
+	k = 0;
+	for (i = 0; i != num; i++) {
+		hl = mb[i]->l2_len + mb[i]->l3_len;
+		rc = inb_pkt_prepare(sa, rsn, mb[i], hl, &icv);
+		if (rc >= 0) {
+			iv[k] = (void *)ivs[k];
+			rc = inb_sync_crypto_proc_prepare(sa, mb[i], &icv, hl,
+					rc, buf + k, vec + vec_idx, iv + k,
+					&aad[k], &digest[k]);
+			if (rc < 0) {
+				dr[i - k] = i;
+				continue;
+			}
+
+			vec_idx += rc;
+			k++;
+		} else
+			dr[i - k] = i;
+	}
+
+	/* copy not prepared mbufs beyond good ones */
+	if (k != num) {
+		rte_errno = EBADMSG;
+
+		if (unlikely(k == 0))
+			return 0;
+
+		move_bad_mbufs(mb, dr, num, num - k);
+	}
+
+	/* process the packets */
+	n = 0;
+	rte_security_process_cpu_crypto_bulk(ss->security.ctx,
+			ss->security.ses, buf, iv, aad, digest, status,
+			k);
+	/* move failed process packets to dr */
+	for (i = 0; i < k; i++) {
+		if (status[i]) {
+			dr[n++] = i;
+			rte_errno = EBADMSG;
+		}
+	}
+
+	/* move bad packets to the back */
+	if (n)
+		move_bad_mbufs(mb, dr, k, n);
+
+	/* process packets */
+	p = process(sa, mb, sqn, dr, k - n, sqh_len);
+
+	if (p != k - n && p != 0)
+		move_bad_mbufs(mb, dr, k - n, k - n - p);
+
+	if (p != num)
+		rte_errno = EBADMSG;
+
+	return p;
+}
+
+uint16_t
+esp_inb_tun_sync_crypto_pkt_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num)
+{
+	struct rte_ipsec_sa *sa = ss->sa;
+
+	return esp_inb_sync_crypto_pkt_process(ss, mb, num, sa->sqh_len,
+			tun_process);
+}
+
+uint16_t
+esp_inb_trs_sync_crypto_pkt_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num)
+{
+	struct rte_ipsec_sa *sa = ss->sa;
+
+	return esp_inb_sync_crypto_pkt_process(ss, mb, num, sa->sqh_len,
+			trs_process);
+}
+
 /*
  * process group of ESP inbound tunnel packets.
  */
lib/librte_ipsec/esp_outb.c +288 -2

302 lines not on this page · raw message

lib/librte_ipsec/sa.c +46 -7

80 lines not on this page · raw message

lib/librte_ipsec/sa.h +29 -0
index 51e69ad05..02c7abc60 100644
@@ -156,6 +156,14 @@ uint16_t
 inline_inb_trs_pkt_process(const struct rte_ipsec_session *ss,
 	struct rte_mbuf *mb[], uint16_t num);
 
+uint16_t
+esp_inb_tun_sync_crypto_pkt_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
+uint16_t
+esp_inb_trs_sync_crypto_pkt_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
 /* outbound processing */
 
 uint16_t
@@ -170,6 +178,10 @@ uint16_t
 esp_outb_sqh_process(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
 	uint16_t num);
 
+uint16_t
+esp_outb_pkt_flag_process(const struct rte_ipsec_session *ss,
+	struct rte_mbuf *mb[], uint16_t num);
+
 uint16_t
 inline_outb_tun_pkt_process(const struct rte_ipsec_session *ss,
 	struct rte_mbuf *mb[], uint16_t num);
@@ -182,4 +194,21 @@ uint16_t
 inline_proto_outb_pkt_process(const struct rte_ipsec_session *ss,
 	struct rte_mbuf *mb[], uint16_t num);
 
+uint16_t
+esp_outb_tun_sync_crpyto_sqh_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
+uint16_t
+esp_outb_tun_sync_crpyto_flag_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
+uint16_t
+esp_outb_trs_sync_crpyto_sqh_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
+uint16_t
+esp_outb_trs_sync_crpyto_flag_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
+
 #endif /* _SA_H_ */
lib/librte_ipsec/ses.c +3 -1
index 82c765a33..eaa8c17b7 100644
@@ -19,7 +19,9 @@ session_check(struct rte_ipsec_session *ss)
 			return -EINVAL;
 		if ((ss->type == RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO ||
 				ss->type ==
-				RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL) &&
+				RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL ||
+				ss->type ==
+				RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) &&
 				ss->security.ctx == NULL)
 			return -EINVAL;
 	}
Signature · 1 line
2.14.5
author
#48 lore raw

RFC PATCH 9/9 examples/ipsec-secgw: add security cpu_crypto action support

In reply to Fan Zhang

Since ipsec library is added cpu_crypto security action type support, this patch updates ipsec-secgw sample application with added action type "cpu-crypto". The patch also includes a number of test scripts to prove the correctness of the implementation.

---
 examples/ipsec-secgw/ipsec.c                       | 22 ++++++++++++++++++++++
 examples/ipsec-secgw/ipsec_process.c               |  4 ++--
 examples/ipsec-secgw/sa.c                          | 13 +++++++++++--
 examples/ipsec-secgw/test/run_test.sh              | 10 ++++++++++
 .../test/trs_3descbc_sha1_cpu_crypto_defs.sh       |  5 +++++
 .../test/trs_aescbc_sha1_cpu_crypto_defs.sh        |  5 +++++
 .../test/trs_aesctr_sha1_cpu_crypto_defs.sh        |  5 +++++
 .../ipsec-secgw/test/trs_aesgcm_cpu_crypto_defs.sh |  5 +++++
 .../test/trs_aesgcm_mb_cpu_crypto_defs.sh          |  7 +++++++
 .../test/tun_3descbc_sha1_cpu_crypto_defs.sh       |  5 +++++
 .../test/tun_aescbc_sha1_cpu_crypto_defs.sh        |  5 +++++
 .../test/tun_aesctr_sha1_cpu_crypto_defs.sh        |  5 +++++
 .../ipsec-secgw/test/tun_aesgcm_cpu_crypto_defs.sh |  5 +++++
 .../test/tun_aesgcm_mb_cpu_crypto_defs.sh          |  7 +++++++
 14 files changed, 99 insertions(+), 4 deletions(-)
 create mode 100644 examples/ipsec-secgw/test/trs_3descbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aescbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aesctr_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_mb_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_3descbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aescbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aesctr_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_mb_cpu_crypto_defs.sh
examples/ipsec-secgw/ipsec.c +22 -0
index dc85adfe5..4c39a7de6 100644
@@ -10,6 +10,7 @@
 #include <rte_crypto.h>
 #include <rte_security.h>
 #include <rte_cryptodev.h>
+#include <rte_ipsec.h>
 #include <rte_ethdev.h>
 #include <rte_mbuf.h>
 #include <rte_hash.h>
@@ -105,6 +106,26 @@ create_lookaside_session(struct ipsec_ctx *ipsec_ctx, struct ipsec_sa *sa)
 				"SEC Session init failed: err: %d\n", ret);
 				return -1;
 			}
+		} else if (sa->type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) {
+			struct rte_security_ctx *ctx =
+				(struct rte_security_ctx *)
+				rte_cryptodev_get_sec_ctx(
+					ipsec_ctx->tbl[cdev_id_qp].id);
+			int32_t offset = sizeof(struct rte_esp_hdr) +
+					sa->iv_len;
+
+			/* Set IPsec parameters in conf */
+			sess_conf.cpucrypto.cipher_offset = offset;
+
+			set_ipsec_conf(sa, &(sess_conf.ipsec));
+			sa->security_ctx = ctx;
+			sa->sec_session = rte_security_session_create(ctx,
+				&sess_conf, ipsec_ctx->session_priv_pool);
+			if (sa->sec_session == NULL) {
+				RTE_LOG(ERR, IPSEC,
+				"SEC Session init failed: err: %d\n", ret);
+				return -1;
+			}
 		} else {
 			RTE_LOG(ERR, IPSEC, "Inline not supported\n");
 			return -1;
@@ -473,6 +494,7 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec_ctx *ipsec_ctx,
 						sa->sec_session, pkts[i], NULL);
 			continue;
 		case RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO:
+		case RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO:
 			RTE_ASSERT(sa->sec_session != NULL);
 			priv->cop.type = RTE_CRYPTO_OP_TYPE_SYMMETRIC;
 			priv->cop.status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED;
examples/ipsec-secgw/ipsec_process.c +2 -2
index 868f1a28d..73bfb314e 100644
@@ -227,8 +227,8 @@ ipsec_process(struct ipsec_ctx *ctx, struct ipsec_traffic *trf)
 
 		/* process packets inline */
 		else if (sa->type == RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO ||
-				sa->type ==
-				RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL) {
+			sa->type == RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL ||
+			sa->type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) {
 
 			satp = rte_ipsec_sa_type(ips->sa);
 
examples/ipsec-secgw/sa.c +11 -2
index c3cf3bd1f..ba773346f 100644
@@ -570,6 +570,9 @@ parse_sa_tokens(char **tokens, uint32_t n_tokens,
 				RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL;
 			else if (strcmp(tokens[ti], "no-offload") == 0)
 				rule->type = RTE_SECURITY_ACTION_TYPE_NONE;
+			else if (strcmp(tokens[ti], "cpu-crypto") == 0)
+				rule->type =
+					RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO;
 			else {
 				APP_CHECK(0, status, "Invalid input \"%s\"",
 						tokens[ti]);
@@ -624,10 +627,13 @@ parse_sa_tokens(char **tokens, uint32_t n_tokens,
 	if (status->status < 0)
 		return;
 
-	if ((rule->type != RTE_SECURITY_ACTION_TYPE_NONE) && (portid_p == 0))
+	if ((rule->type != RTE_SECURITY_ACTION_TYPE_NONE && rule->type !=
+			RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) &&
+			(portid_p == 0))
 		printf("Missing portid option, falling back to non-offload\n");
 
-	if (!type_p || !portid_p) {
+	if (!type_p || (!portid_p && rule->type !=
+			RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)) {
 		rule->type = RTE_SECURITY_ACTION_TYPE_NONE;
 		rule->portid = -1;
 	}
@@ -709,6 +715,9 @@ print_one_sa_rule(const struct ipsec_sa *sa, int inbound)
 	case RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL:
 		printf("lookaside-protocol-offload ");
 		break;
+	case RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO:
+		printf("cpu-crypto-accelerated");
+		break;
 	}
 	printf("\n");
 }
examples/ipsec-secgw/test/run_test.sh +10 -0
index 8055a4c04..f322aa785 100755
@@ -32,15 +32,21 @@ usage()
 }
 
 LINUX_TEST="tun_aescbc_sha1 \
+tun_aescbc_sha1_cpu_crypto \
 tun_aescbc_sha1_esn \
 tun_aescbc_sha1_esn_atom \
 tun_aesgcm \
+tun_aesgcm_cpu_crypto \
+tun_aesgcm_mb_cpu_crypto \
 tun_aesgcm_esn \
 tun_aesgcm_esn_atom \
 trs_aescbc_sha1 \
+trs_aescbc_sha1_cpu_crypto \
 trs_aescbc_sha1_esn \
 trs_aescbc_sha1_esn_atom \
 trs_aesgcm \
+trs_aesgcm_cpu_crypto \
+trs_aesgcm_mb_cpu_crypto \
 trs_aesgcm_esn \
 trs_aesgcm_esn_atom \
 tun_aescbc_sha1_old \
@@ -49,17 +55,21 @@ trs_aescbc_sha1_old \
 trs_aesgcm_old \
 tun_aesctr_sha1 \
 tun_aesctr_sha1_old \
+tun_aesctr_cpu_crypto \
 tun_aesctr_sha1_esn \
 tun_aesctr_sha1_esn_atom \
 trs_aesctr_sha1 \
+trs_aesctr_sha1_cpu_crypto \
 trs_aesctr_sha1_old \
 trs_aesctr_sha1_esn \
 trs_aesctr_sha1_esn_atom \
 tun_3descbc_sha1 \
+tun_3descbc_sha1_cpu_crypto \
 tun_3descbc_sha1_old \
 tun_3descbc_sha1_esn \
 tun_3descbc_sha1_esn_atom \
 trs_3descbc_sha1 \
+trs_3descbc_sha1 \
 trs_3descbc_sha1_old \
 trs_3descbc_sha1_esn \
 trs_3descbc_sha1_esn_atom"
examples/ipsec-secgw/test/trs_3descbc_sha1_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..a864a8886
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/trs_3descbc_sha1_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/trs_aescbc_sha1_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..a4d83e9c4
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/trs_aescbc_sha1_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/trs_aesctr_sha1_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..745a2a02b
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/trs_aesctr_sha1_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/trs_aesgcm_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..8917122da
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/trs_aesgcm_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/trs_aesgcm_mb_cpu_crypto_defs.sh new file +7 -0
new file mode 100644
index 000000000..26943321f
@@ -0,0 +1,7 @@
+#! /bin/bash
+
+. ${DIR}/trs_aesgcm_defs.sh
+
+CRYPTO_DEV=${CRYPTO_DEV:-'--vdev="crypto_aesni_mb0"'}
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/tun_3descbc_sha1_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..747141f62
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/tun_3descbc_sha1_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/tun_aescbc_sha1_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..56076fa50
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/tun_aescbc_sha1_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/tun_aesctr_sha1_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..3af680533
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/tun_aesctr_sha1_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/tun_aesgcm_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..5bf1c0ae5
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/tun_aesgcm_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/tun_aesgcm_mb_cpu_crypto_defs.sh new file +7 -0
new file mode 100644
index 000000000..039b8095e
@@ -0,0 +1,7 @@
+#! /bin/bash
+
+. ${DIR}/tun_aesgcm_defs.sh
+
+CRYPTO_DEV=${CRYPTO_DEV:-'--vdev="crypto_aesni_mb0"'}
+
+SGW_CFG_XPRM='type cpu-crypto'
Signature · 1 line
2.14.5
author
#49 lore raw

cover security: add software synchronous crypto process

In reply to Fan Zhang

This RFC patch adds a way to rte_security to process symmetric crypto workload in bulk synchronously for SW crypto devices.

Originally both SW and HW crypto PMDs works under rte_cryptodev to process the crypto workload asynchronously. This way provides uniformity to both PMD types but also introduce unnecessary performance penalty to SW PMDs such as extra SW ring enqueue/dequeue steps to "simulate" asynchronous working manner and unnecessary HW addresses computation.

We introduce a new way for SW crypto devices that perform crypto operation synchronously with only fields required for the computation as input.

In rte_security, a new action type "RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO" is introduced. This action type allows the burst of symmetric crypto workload using the same algorithm, key, and direction being processed by CPU cycles synchronously. This flexible action type does not require external hardware involvement.

This patch also includes the announcement of a new API "rte_security_process_cpu_crypto_bulk". With this API the packet is sent to the crypto device for symmetric crypto processing. The device will encrypt or decrypt the buffer based on the session data specified and preprocessed in the security session. Different than the inline or lookaside modes, when the function exits, the user will expect the buffers are either processed successfully, or having the error number assigned to the appropriate index of the status array.

The proof-of-concept AESNI-GCM and AESNI-MB SW PMDs are updated with the support of this new method. To demonstrate the performance gain with this method 2 simple performance evaluation apps under unit-test are added "app/test: security_aesni_gcm_perftest/security_aesni_mb_perftest". The users can freely compare their results against crypto perf application results.

In the end, the ipsec library and ipsec-secgw sample application are also updated to support this feature. Several test scripts are added to the ipsec-secgw test-suite to prove the correctness of the implementation.

Fan Zhang (10):
security: introduce CPU Crypto action type and API crypto/aesni_gcm: add rte_security handler app/test: add security cpu crypto autotest app/test: add security cpu crypto perftest crypto/aesni_mb: add rte_security handler app/test: add aesni_mb security cpu crypto autotest app/test: add aesni_mb security cpu crypto perftest
ipsec: add rte_security cpu_crypto action support examples/ipsec-secgw: add security cpu_crypto action support
doc: update security cpu process description

 app/test/Makefile                                  |    1 +
 app/test/meson.build                               |    1 +
 app/test/test_security_cpu_crypto.c                | 1326 ++++++++++++++++++++
 doc/guides/cryptodevs/aesni_gcm.rst                |    6 +
 doc/guides/cryptodevs/aesni_mb.rst                 |    7 +
 doc/guides/prog_guide/rte_security.rst             |  112 +-
 doc/guides/rel_notes/release_19_11.rst             |    7 +
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c           |   91 +-
 drivers/crypto/aesni_gcm/aesni_gcm_pmd_ops.c       |   95 ++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h   |   23 +
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c         |  291 ++++-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c     |   91 +-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h |   21 +-
 examples/ipsec-secgw/ipsec.c                       |   22 +
 examples/ipsec-secgw/ipsec_process.c               |    7 +-
 examples/ipsec-secgw/sa.c                          |   13 +-
 examples/ipsec-secgw/test/run_test.sh              |   10 +
 .../test/trs_3descbc_sha1_cpu_crypto_defs.sh       |    5 +
 .../test/trs_aescbc_sha1_cpu_crypto_defs.sh        |    5 +
 .../test/trs_aesctr_sha1_cpu_crypto_defs.sh        |    5 +
 .../ipsec-secgw/test/trs_aesgcm_cpu_crypto_defs.sh |    5 +
 .../test/trs_aesgcm_mb_cpu_crypto_defs.sh          |    7 +
 .../test/tun_3descbc_sha1_cpu_crypto_defs.sh       |    5 +
 .../test/tun_aescbc_sha1_cpu_crypto_defs.sh        |    5 +
 .../test/tun_aesctr_sha1_cpu_crypto_defs.sh        |    5 +
 .../ipsec-secgw/test/tun_aesgcm_cpu_crypto_defs.sh |    5 +
 .../test/tun_aesgcm_mb_cpu_crypto_defs.sh          |    7 +
 lib/librte_ipsec/esp_inb.c                         |  174 ++-
 lib/librte_ipsec/esp_outb.c                        |  290 ++++-
 lib/librte_ipsec/sa.c                              |   53 +-
 lib/librte_ipsec/sa.h                              |   29 +
 lib/librte_ipsec/ses.c                             |    4 +-
 lib/librte_security/rte_security.c                 |   16 +
 lib/librte_security/rte_security.h                 |   51 +-
 lib/librte_security/rte_security_driver.h          |   19 +
 lib/librte_security/rte_security_version.map       |    1 +
 36 files changed, 2791 insertions(+), 24 deletions(-)
 create mode 100644 app/test/test_security_cpu_crypto.c
 create mode 100644 examples/ipsec-secgw/test/trs_3descbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aescbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aesctr_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_mb_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_3descbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aescbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aesctr_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_mb_cpu_crypto_defs.sh
Signature · 1 line
2.14.5
author
#50 lore raw

PATCH 1/10 security: introduce CPU Crypto action type and API

This patch introduce new RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO action type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for PMDs.

---
 lib/librte_security/rte_security.c           | 16 +++++++++
 lib/librte_security/rte_security.h           | 51 +++++++++++++++++++++++++++-
 lib/librte_security/rte_security_driver.h    | 19 +++++++++++
 lib/librte_security/rte_security_version.map |  1 +
 4 files changed, 86 insertions(+), 1 deletion(-)
lib/librte_security/rte_security.c +16 -0
index bc81ce15d..0f85c1b59 100644
@@ -141,3 +141,19 @@ rte_security_capability_get(struct rte_security_ctx *instance,
 
 	return NULL;
 }
+
+void
+rte_security_process_cpu_crypto_bulk(struct rte_security_ctx *instance,
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num)
+{
+	uint32_t i;
+
+	for (i = 0; i < num; i++)
+		status[i] = -1;
+
+	RTE_FUNC_PTR_OR_RET(*instance->ops->process_cpu_crypto_bulk);
+	instance->ops->process_cpu_crypto_bulk(sess, buf, iv,
+			aad, digest, status, num);
+}
lib/librte_security/rte_security.h +50 -1

84 lines not on this page · raw message

lib/librte_security/rte_security_driver.h +19 -0
index 1b561f852..70fcb0c26 100644
@@ -132,6 +132,23 @@ typedef int (*security_get_userdata_t)(void *device,
 typedef const struct rte_security_capability *(*security_capabilities_get_t)(
 		void *device);
 
+/**
+ * Process security operations in bulk using CPU accelerated method.
+ *
+ * @param	sess		Security session structure.
+ * @param	buf		Buffer to the vectors to be processed.
+ * @param	iv		IV pointers.
+ * @param	aad		AAD pointers.
+ * @param	digest		Digest pointers.
+ * @param	status		Array of status value.
+ * @param	num		Number of elements in each array.
+ */
+
+typedef void (*security_process_cpu_crypto_bulk_t)(
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+
 /** Security operations function pointer table */
 struct rte_security_ops {
 	security_session_create_t session_create;
@@ -150,6 +167,8 @@ struct rte_security_ops {
 	/**< Get userdata associated with session which processed the packet. */
 	security_capabilities_get_t capabilities_get;
 	/**< Get security capabilities. */
+	security_process_cpu_crypto_bulk_t process_cpu_crypto_bulk;
+	/**< Process data in bulk. */
 };
 
 #ifdef __cplusplus
lib/librte_security/rte_security_version.map +1 -0
index 53267bf3c..2132e7a00 100644
@@ -18,4 +18,5 @@ EXPERIMENTAL {
 	rte_security_get_userdata;
 	rte_security_session_stats_get;
 	rte_security_session_update;
+	rte_security_process_cpu_crypto_bulk;
 };
Signature · 1 line
2.14.5
#51 lore raw

Re: [dpdk-dev] [PATCH 01/10] security: introduce CPU Crypto action type and API

Quoted 9 lines
lib/librte_security/rte_security.h +9 -0
+/**
+ * Security vector structure, contains pointer to vector array and the length
+ * of the array
+ */
+struct rte_security_vec {
+	struct iovec *vec;
+	uint32_t num;
+};
+

lib/librte_security/rte_security.h const struct rte_security_capability * · line 680

Quoted 10 lines
lib/librte_security/rte_security.h +10 -0
+/**
+ * Processing bulk crypto workload with CPU
+ *
+ * @param	instance	security instance.
+ * @param	sess		security session
+ * @param	buf		array of buffer SGL vectors
+ * @param	iv		array of IV pointers
+ * @param	aad		array of AAD pointers
+ * @param	digest		array of digest pointers
+ * @param	status		array of status for the function to return

Need to specify what are expected status values.
I suppose zero for success, negative errno for some error happens?

Quoted 13 lines

+ * @param num number of elements in each array
+ *
+ */

lib/librte_security/rte_security.h +7 -0
+__rte_experimental
+void
+rte_security_process_cpu_crypto_bulk(struct rte_security_ctx *instance,
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+
 #ifdef __cplusplus
 }
 #endif
#52 lore raw

Re: [dpdk-dev] [PATCH 01/10] security: introduce CPU Crypto action type and API

In reply to Fan Zhang

Some comments inline.

Quoted 73 lines · Fan Zhang

This patch introduce new RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO action type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for PMDs.

---
  lib/librte_security/rte_security.c           | 16 +++++++++
  lib/librte_security/rte_security.h           | 51 +++++++++++++++++++++++++++-
  lib/librte_security/rte_security_driver.h    | 19 +++++++++++
  lib/librte_security/rte_security_version.map |  1 +
  4 files changed, 86 insertions(+), 1 deletion(-)
lib/librte_security/rte_security.c +16 -0
index bc81ce15d..0f85c1b59 100644
@@ -141,3 +141,19 @@ rte_security_capability_get(struct rte_security_ctx *instance,
 
  	return NULL;
  }
+
+void
+rte_security_process_cpu_crypto_bulk(struct rte_security_ctx *instance,
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num)
+{
+	uint32_t i;
+
+	for (i = 0; i < num; i++)
+		status[i] = -1;
+
+	RTE_FUNC_PTR_OR_RET(*instance->ops->process_cpu_crypto_bulk);
+	instance->ops->process_cpu_crypto_bulk(sess, buf, iv,
+			aad, digest, status, num);
+}
lib/librte_security/rte_security.h +15 -0
index 96806e3a2..5a0f8901b 100644
@@ -18,6 +18,7 @@ extern "C" {
  #endif
 
  #include <sys/types.h>
+#include <sys/uio.h>
 
  #include <netinet/in.h>
  #include <netinet/ip.h>
@@ -272,6 +273,20 @@ struct rte_security_pdcp_xform {
  	uint32_t hfn_threshold;
  };
 
+struct rte_security_cpu_crypto_xform {
+	/** For cipher/authentication crypto operation the authentication may
+	 * cover more content then the cipher. E.g., for IPSec ESP encryption
+	 * with AES-CBC and SHA1-HMAC, the encryption happens after the ESP
+	 * header but whole packet (apart from MAC header) is authenticated.
+	 * The cipher_offset field is used to deduct the cipher data pointer
+	 * from the buffer to be processed.
+	 *
+	 * NOTE this parameter shall be ignored by AEAD algorithms, since it
+	 * uses the same offset for cipher and authentication.
+	 */
+	int32_t cipher_offset;
+};
+
  /**
   * Security session action type.
   */
@@ -286,10 +301,14 @@ enum rte_security_session_action_type {
  	/**< All security protocol processing is performed inline during
  	 * transmission

lib/librte_security/rte_security.h enum rte_security_session_action_type { · line 311

Quoted 10 lines
lib/librte_security/rte_security.h +5 -1
  	 */
-	RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL
+	RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL,
  	/**< All security protocol processing including crypto is performed
  	 * on a lookaside accelerator
  	 */
+	RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO
+	/**< Crypto processing for security protocol is processed by CPU
+	 * synchronously
+	 */

though you are naming it cpu crypto, but it is more like raw packet crypto, where you want to skip mbuf/crypto ops and directly wants to work on raw buffer.

Quoted 14 lines
lib/librte_security/rte_security.h +1 -0
  };
 
  /** Security session protocol definition */
@@ -315,6 +334,7 @@ struct rte_security_session_conf {
  		struct rte_security_ipsec_xform ipsec;
  		struct rte_security_macsec_xform macsec;
  		struct rte_security_pdcp_xform pdcp;
+		struct rte_security_cpu_crypto_xform cpucrypto;
  	};
  	/**< Configuration parameters for security session */
  	struct rte_crypto_sym_xform *crypto_xform;
@@ -639,6 +659,35 @@ const struct rte_security_capability *
  rte_security_capability_get(struct rte_security_ctx *instance,
  			    struct rte_security_capability_idx *idx);

lib/librte_security/rte_security.h const struct rte_security_capability * · line 670

Quoted 10 lines
lib/librte_security/rte_security.h +9 -0
 
+/**
+ * Security vector structure, contains pointer to vector array and the length
+ * of the array
+ */
+struct rte_security_vec {
+	struct iovec *vec;
+	uint32_t num;
+};
+

Just wondering if you want to change it to *in_vec and *out_vec, that will be helpful in future, if the out-of-place processing is required for CPU usecase as well?

Quoted 10 lines
lib/librte_security/rte_security.h +10 -0
+/**
+ * Processing bulk crypto workload with CPU
+ *
+ * @param	instance	security instance.
+ * @param	sess		security session
+ * @param	buf		array of buffer SGL vectors
+ * @param	iv		array of IV pointers
+ * @param	aad		array of AAD pointers
+ * @param	digest		array of digest pointers
+ * @param	status		array of status for the function to return

lib/librte_security/rte_security.h const struct rte_security_capability * · line 690

Quoted 10 lines
lib/librte_security/rte_security.h +10 -0
+ * @param	num		number of elements in each array
+ *
+ */
+__rte_experimental
+void
+rte_security_process_cpu_crypto_bulk(struct rte_security_ctx *instance,
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+

Why not make the return as int, to indicate whether this API completely failed or processed or have some valid status to look into?

Quoted 50 lines
lib/librte_security/rte_security.h +0 -0
  #ifdef __cplusplus
  }
  #endif
lib/librte_security/rte_security_driver.h +19 -0
index 1b561f852..70fcb0c26 100644
@@ -132,6 +132,23 @@ typedef int (*security_get_userdata_t)(void *device,
  typedef const struct rte_security_capability *(*security_capabilities_get_t)(
  		void *device);
 
+/**
+ * Process security operations in bulk using CPU accelerated method.
+ *
+ * @param	sess		Security session structure.
+ * @param	buf		Buffer to the vectors to be processed.
+ * @param	iv		IV pointers.
+ * @param	aad		AAD pointers.
+ * @param	digest		Digest pointers.
+ * @param	status		Array of status value.
+ * @param	num		Number of elements in each array.
+ */
+
+typedef void (*security_process_cpu_crypto_bulk_t)(
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+
  /** Security operations function pointer table */
  struct rte_security_ops {
  	security_session_create_t session_create;
@@ -150,6 +167,8 @@ struct rte_security_ops {
  	/**< Get userdata associated with session which processed the packet. */
  	security_capabilities_get_t capabilities_get;
  	/**< Get security capabilities. */
+	security_process_cpu_crypto_bulk_t process_cpu_crypto_bulk;
+	/**< Process data in bulk. */
  };
 
  #ifdef __cplusplus
lib/librte_security/rte_security_version.map +1 -0
index 53267bf3c..2132e7a00 100644
@@ -18,4 +18,5 @@ EXPERIMENTAL {
  	rte_security_get_userdata;
  	rte_security_session_stats_get;
  	rte_security_session_update;
+	rte_security_process_cpu_crypto_bulk;
  };
#53 lore raw

Re: [dpdk-dev] [PATCH 01/10] security: introduce CPU Crypto action type and API

Hi Hemant,

Quoted 80 lines · Fan Zhang
Quoted 78 lines

This patch introduce new RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO action type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for PMDs.

---
  lib/librte_security/rte_security.c           | 16 +++++++++
  lib/librte_security/rte_security.h           | 51 +++++++++++++++++++++++++++-
  lib/librte_security/rte_security_driver.h    | 19 +++++++++++
  lib/librte_security/rte_security_version.map |  1 +
  4 files changed, 86 insertions(+), 1 deletion(-)
lib/librte_security/rte_security.c +16 -0
index bc81ce15d..0f85c1b59 100644
@@ -141,3 +141,19 @@ rte_security_capability_get(struct rte_security_ctx *instance,
 
  	return NULL;
  }
+
+void
+rte_security_process_cpu_crypto_bulk(struct rte_security_ctx *instance,
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num)
+{
+	uint32_t i;
+
+	for (i = 0; i < num; i++)
+		status[i] = -1;
+
+	RTE_FUNC_PTR_OR_RET(*instance->ops->process_cpu_crypto_bulk);
+	instance->ops->process_cpu_crypto_bulk(sess, buf, iv,
+			aad, digest, status, num);
+}
lib/librte_security/rte_security.h +16 -1
index 96806e3a2..5a0f8901b 100644
@@ -18,6 +18,7 @@ extern "C" {
  #endif
 
  #include <sys/types.h>
+#include <sys/uio.h>
 
  #include <netinet/in.h>
  #include <netinet/ip.h>
@@ -272,6 +273,20 @@ struct rte_security_pdcp_xform {
  	uint32_t hfn_threshold;
  };
 
+struct rte_security_cpu_crypto_xform {
+	/** For cipher/authentication crypto operation the authentication may
+	 * cover more content then the cipher. E.g., for IPSec ESP encryption
+	 * with AES-CBC and SHA1-HMAC, the encryption happens after the ESP
+	 * header but whole packet (apart from MAC header) is authenticated.
+	 * The cipher_offset field is used to deduct the cipher data pointer
+	 * from the buffer to be processed.
+	 *
+	 * NOTE this parameter shall be ignored by AEAD algorithms, since it
+	 * uses the same offset for cipher and authentication.
+	 */
+	int32_t cipher_offset;
+};
+
  /**
   * Security session action type.
   */
@@ -286,10 +301,14 @@ enum rte_security_session_action_type {
  	/**< All security protocol processing is performed inline during
  	 * transmission
  	 */
-	RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL
+	RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL,
  	/**< All security protocol processing including crypto is performed
  	 * on a lookaside accelerator
Quoted 8 lines

lib/librte_security/rte_security.h enum rte_security_session_action_type { · line 311

Quoted 5 lines
lib/librte_security/rte_security.h +4 -0
  	 */
+	RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO
+	/**< Crypto processing for security protocol is processed by CPU
+	 * synchronously
+	 */

though you are naming it cpu crypto, but it is more like raw packet crypto, where you want to skip mbuf/crypto ops and directly wants to work on raw buffer.

Yes, but we do wat to do that (skip mbuf/crypto ops and use raw buffer), because this API is destined for SW backed implementation.
For that case crypto-ops , mbuf, enqueue/dequeue are just unnecessary overhead.

Quoted 15 lines
Quoted 15 lines
lib/librte_security/rte_security.h +1 -0
  };
 
  /** Security session protocol definition */
@@ -315,6 +334,7 @@ struct rte_security_session_conf {
  		struct rte_security_ipsec_xform ipsec;
  		struct rte_security_macsec_xform macsec;
  		struct rte_security_pdcp_xform pdcp;
+		struct rte_security_cpu_crypto_xform cpucrypto;
  	};
  	/**< Configuration parameters for security session */
  	struct rte_crypto_sym_xform *crypto_xform;
@@ -639,6 +659,35 @@ const struct rte_security_capability *
  rte_security_capability_get(struct rte_security_ctx *instance,
  			    struct rte_security_capability_idx *idx);
 
Quoted 13 lines

lib/librte_security/rte_security.h const struct rte_security_capability * · line 670

Quoted 9 lines
lib/librte_security/rte_security.h +9 -0
+/**
+ * Security vector structure, contains pointer to vector array and the length
+ * of the array
+ */
+struct rte_security_vec {
+	struct iovec *vec;
+	uint32_t num;
+};
+

Just wondering if you want to change it to *in_vec and *out_vec, that will be helpful in future, if the out-of-place processing is required for CPU usecase as well?

I suppose this is doable, though right now we don't plan to support such model.

Quoted 16 lines
Quoted 16 lines
lib/librte_security/rte_security.h +15 -0
+/**
+ * Processing bulk crypto workload with CPU
+ *
+ * @param	instance	security instance.
+ * @param	sess		security session
+ * @param	buf		array of buffer SGL vectors
+ * @param	iv		array of IV pointers
+ * @param	aad		array of AAD pointers
+ * @param	digest		array of digest pointers
+ * @param	status		array of status for the function to return
+ * @param	num		number of elements in each array
+ *
+ */
+__rte_experimental
+void
Quoted 8 lines

lib/librte_security/rte_security.h const struct rte_security_capability * · line 690

Quoted 5 lines
lib/librte_security/rte_security.h +5 -0
+rte_security_process_cpu_crypto_bulk(struct rte_security_ctx *instance,
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+

Why not make the return as int, to indicate whether this API completely failed or processed or have some valid status to look into?

Good point, will change as suggested.

Quoted 52 lines
Quoted 52 lines
lib/librte_security/rte_security.h +0 -0
  #ifdef __cplusplus
  }
  #endif
lib/librte_security/rte_security_driver.h +19 -0
index 1b561f852..70fcb0c26 100644
@@ -132,6 +132,23 @@ typedef int (*security_get_userdata_t)(void *device,
  typedef const struct rte_security_capability *(*security_capabilities_get_t)(
  		void *device);
 
+/**
+ * Process security operations in bulk using CPU accelerated method.
+ *
+ * @param	sess		Security session structure.
+ * @param	buf		Buffer to the vectors to be processed.
+ * @param	iv		IV pointers.
+ * @param	aad		AAD pointers.
+ * @param	digest		Digest pointers.
+ * @param	status		Array of status value.
+ * @param	num		Number of elements in each array.
+ */
+
+typedef void (*security_process_cpu_crypto_bulk_t)(
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+
  /** Security operations function pointer table */
  struct rte_security_ops {
  	security_session_create_t session_create;
@@ -150,6 +167,8 @@ struct rte_security_ops {
  	/**< Get userdata associated with session which processed the packet. */
  	security_capabilities_get_t capabilities_get;
  	/**< Get security capabilities. */
+	security_process_cpu_crypto_bulk_t process_cpu_crypto_bulk;
+	/**< Process data in bulk. */
  };
 
  #ifdef __cplusplus
lib/librte_security/rte_security_version.map +1 -0
index 53267bf3c..2132e7a00 100644
@@ -18,4 +18,5 @@ EXPERIMENTAL {
  	rte_security_get_userdata;
  	rte_security_session_stats_get;
  	rte_security_session_update;
+	rte_security_process_cpu_crypto_bulk;
  };
#54 lore raw

Re: [dpdk-dev] [PATCH 01/10] security: introduce CPU Crypto action type and API

Hi Konstantin,

Quoted 86 lines · Fan Zhang
Quoted 78 lines
Quoted 78 lines

This patch introduce new RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO action type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for PMDs.

---
   lib/librte_security/rte_security.c           | 16 +++++++++
   lib/librte_security/rte_security.h           | 51 +++++++++++++++++++++++++++-
   lib/librte_security/rte_security_driver.h    | 19 +++++++++++
   lib/librte_security/rte_security_version.map |  1 +
   4 files changed, 86 insertions(+), 1 deletion(-)
lib/librte_security/rte_security.c +16 -0
index bc81ce15d..0f85c1b59 100644
@@ -141,3 +141,19 @@ rte_security_capability_get(struct rte_security_ctx *instance,
 
   	return NULL;
   }
+
+void
+rte_security_process_cpu_crypto_bulk(struct rte_security_ctx *instance,
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num)
+{
+	uint32_t i;
+
+	for (i = 0; i < num; i++)
+		status[i] = -1;
+
+	RTE_FUNC_PTR_OR_RET(*instance->ops->process_cpu_crypto_bulk);
+	instance->ops->process_cpu_crypto_bulk(sess, buf, iv,
+			aad, digest, status, num);
+}
lib/librte_security/rte_security.h +16 -1
index 96806e3a2..5a0f8901b 100644
@@ -18,6 +18,7 @@ extern "C" {
   #endif
 
   #include <sys/types.h>
+#include <sys/uio.h>
 
   #include <netinet/in.h>
   #include <netinet/ip.h>
@@ -272,6 +273,20 @@ struct rte_security_pdcp_xform {
   	uint32_t hfn_threshold;
   };
 
+struct rte_security_cpu_crypto_xform {
+	/** For cipher/authentication crypto operation the authentication may
+	 * cover more content then the cipher. E.g., for IPSec ESP encryption
+	 * with AES-CBC and SHA1-HMAC, the encryption happens after the ESP
+	 * header but whole packet (apart from MAC header) is authenticated.
+	 * The cipher_offset field is used to deduct the cipher data pointer
+	 * from the buffer to be processed.
+	 *
+	 * NOTE this parameter shall be ignored by AEAD algorithms, since it
+	 * uses the same offset for cipher and authentication.
+	 */
+	int32_t cipher_offset;
+};
+
   /**
    * Security session action type.
    */
@@ -286,10 +301,14 @@ enum rte_security_session_action_type {
   	/**< All security protocol processing is performed inline during
   	 * transmission
   	 */
-	RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL
+	RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL,
   	/**< All security protocol processing including crypto is performed
   	 * on a lookaside accelerator
Quoted 8 lines

lib/librte_security/rte_security.h enum rte_security_session_action_type { · line 311

Quoted 5 lines
lib/librte_security/rte_security.h +4 -0
   	 */
+	RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO
+	/**< Crypto processing for security protocol is processed by CPU
+	 * synchronously
+	 */

though you are naming it cpu crypto, but it is more like raw packet crypto, where you want to skip mbuf/crypto ops and directly wants to work on raw buffer.

Quoted 3 lines

Yes, but we do wat to do that (skip mbuf/crypto ops and use raw buffer), because this API is destined for SW backed implementation.
For that case crypto-ops , mbuf, enqueue/dequeue are just unnecessary overhead.

I agree, we are also planning to take advantage of it for some specific use-cases in future.

Quoted 27 lines
Quoted 15 lines
Quoted 15 lines
lib/librte_security/rte_security.h +1 -0
   };
 
   /** Security session protocol definition */
@@ -315,6 +334,7 @@ struct rte_security_session_conf {
   		struct rte_security_ipsec_xform ipsec;
   		struct rte_security_macsec_xform macsec;
   		struct rte_security_pdcp_xform pdcp;
+		struct rte_security_cpu_crypto_xform cpucrypto;
   	};
   	/**< Configuration parameters for security session */
   	struct rte_crypto_sym_xform *crypto_xform;
@@ -639,6 +659,35 @@ const struct rte_security_capability *
   rte_security_capability_get(struct rte_security_ctx *instance,
   			    struct rte_security_capability_idx *idx);
 
Quoted 12 lines

lib/librte_security/rte_security.h const struct rte_security_capability * · line 670

Quoted 9 lines
lib/librte_security/rte_security.h +9 -0
+/**
+ * Security vector structure, contains pointer to vector array and the length
+ * of the array
+ */
+struct rte_security_vec {
+	struct iovec *vec;
+	uint32_t num;
+};
+

Just wondering if you want to change it to *in_vec and *out_vec, that will be helpful in future, if the out-of-place processing is required for CPU usecase as well?

Quoted 1 line

I suppose this is doable, though right now we don't plan to support such model.

They will come handy in future. I plan to use it in future and we can skip the API/ABI breakage, if the placeholder are present

Quoted 23 lines
Quoted 15 lines
Quoted 15 lines
lib/librte_security/rte_security.h +14 -0
+/**
+ * Processing bulk crypto workload with CPU
+ *
+ * @param	instance	security instance.
+ * @param	sess		security session
+ * @param	buf		array of buffer SGL vectors
+ * @param	iv		array of IV pointers
+ * @param	aad		array of AAD pointers
+ * @param	digest		array of digest pointers
+ * @param	status		array of status for the function to return
+ * @param	num		number of elements in each array
+ *
+ */
+__rte_experimental
Quoted 8 lines

lib/librte_security/rte_security.h const struct rte_security_capability * · line 690

Quoted 6 lines
lib/librte_security/rte_security.h +6 -0
+void
+rte_security_process_cpu_crypto_bulk(struct rte_security_ctx *instance,
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+

Why not make the return as int, to indicate whether this API completely failed or processed or have some valid status to look into?

Quoted 1 line

Good point, will change as suggested.

I have another suggestions w.r.t iv, aad, digest etc. Why not put them in a structure, so that you will

be able to add/remove the variable without breaking the API prototype.

Quoted 52 lines
Quoted 52 lines
Quoted 52 lines
lib/librte_security/rte_security.h +0 -0
   #ifdef __cplusplus
   }
   #endif
lib/librte_security/rte_security_driver.h +19 -0
index 1b561f852..70fcb0c26 100644
@@ -132,6 +132,23 @@ typedef int (*security_get_userdata_t)(void *device,
   typedef const struct rte_security_capability *(*security_capabilities_get_t)(
   		void *device);
 
+/**
+ * Process security operations in bulk using CPU accelerated method.
+ *
+ * @param	sess		Security session structure.
+ * @param	buf		Buffer to the vectors to be processed.
+ * @param	iv		IV pointers.
+ * @param	aad		AAD pointers.
+ * @param	digest		Digest pointers.
+ * @param	status		Array of status value.
+ * @param	num		Number of elements in each array.
+ */
+
+typedef void (*security_process_cpu_crypto_bulk_t)(
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+
   /** Security operations function pointer table */
   struct rte_security_ops {
   	security_session_create_t session_create;
@@ -150,6 +167,8 @@ struct rte_security_ops {
   	/**< Get userdata associated with session which processed the packet. */
   	security_capabilities_get_t capabilities_get;
   	/**< Get security capabilities. */
+	security_process_cpu_crypto_bulk_t process_cpu_crypto_bulk;
+	/**< Process data in bulk. */
   };
 
   #ifdef __cplusplus
lib/librte_security/rte_security_version.map +1 -0
index 53267bf3c..2132e7a00 100644
@@ -18,4 +18,5 @@ EXPERIMENTAL {
   	rte_security_get_userdata;
   	rte_security_session_stats_get;
   	rte_security_session_update;
+	rte_security_process_cpu_crypto_bulk;
   };
#55 lore raw

Re: [dpdk-dev] [PATCH 01/10] security: introduce CPU Crypto action type and API

Hi Hemant,

Quoted 144 lines
Quoted 86 lines
Quoted 78 lines
Quoted 78 lines

This patch introduce new RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO action type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for PMDs.

---
   lib/librte_security/rte_security.c           | 16 +++++++++
   lib/librte_security/rte_security.h           | 51 +++++++++++++++++++++++++++-
   lib/librte_security/rte_security_driver.h    | 19 +++++++++++
   lib/librte_security/rte_security_version.map |  1 +
   4 files changed, 86 insertions(+), 1 deletion(-)
lib/librte_security/rte_security.c +16 -0
index bc81ce15d..0f85c1b59 100644
@@ -141,3 +141,19 @@ rte_security_capability_get(struct rte_security_ctx *instance,
 
   	return NULL;
   }
+
+void
+rte_security_process_cpu_crypto_bulk(struct rte_security_ctx *instance,
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num)
+{
+	uint32_t i;
+
+	for (i = 0; i < num; i++)
+		status[i] = -1;
+
+	RTE_FUNC_PTR_OR_RET(*instance->ops->process_cpu_crypto_bulk);
+	instance->ops->process_cpu_crypto_bulk(sess, buf, iv,
+			aad, digest, status, num);
+}
lib/librte_security/rte_security.h +16 -1
index 96806e3a2..5a0f8901b 100644
@@ -18,6 +18,7 @@ extern "C" {
   #endif
 
   #include <sys/types.h>
+#include <sys/uio.h>
 
   #include <netinet/in.h>
   #include <netinet/ip.h>
@@ -272,6 +273,20 @@ struct rte_security_pdcp_xform {
   	uint32_t hfn_threshold;
   };
 
+struct rte_security_cpu_crypto_xform {
+	/** For cipher/authentication crypto operation the authentication may
+	 * cover more content then the cipher. E.g., for IPSec ESP encryption
+	 * with AES-CBC and SHA1-HMAC, the encryption happens after the ESP
+	 * header but whole packet (apart from MAC header) is authenticated.
+	 * The cipher_offset field is used to deduct the cipher data pointer
+	 * from the buffer to be processed.
+	 *
+	 * NOTE this parameter shall be ignored by AEAD algorithms, since it
+	 * uses the same offset for cipher and authentication.
+	 */
+	int32_t cipher_offset;
+};
+
   /**
    * Security session action type.
    */
@@ -286,10 +301,14 @@ enum rte_security_session_action_type {
   	/**< All security protocol processing is performed inline during
   	 * transmission
   	 */
-	RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL
+	RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL,
   	/**< All security protocol processing including crypto is performed
   	 * on a lookaside accelerator
Quoted 8 lines

lib/librte_security/rte_security.h enum rte_security_session_action_type { · line 311

Quoted 5 lines
lib/librte_security/rte_security.h +4 -0
   	 */
+	RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO
+	/**< Crypto processing for security protocol is processed by CPU
+	 * synchronously
+	 */

though you are naming it cpu crypto, but it is more like raw packet crypto, where you want to skip mbuf/crypto ops and directly wants to work on raw buffer.

Quoted 3 lines

Yes, but we do wat to do that (skip mbuf/crypto ops and use raw buffer), because this API is destined for SW backed implementation.
For that case crypto-ops , mbuf, enqueue/dequeue are just unnecessary overhead.

I agree, we are also planning to take advantage of it for some specific use-cases in future.

Quoted 27 lines
Quoted 15 lines
Quoted 15 lines
lib/librte_security/rte_security.h +1 -0
   };
 
   /** Security session protocol definition */
@@ -315,6 +334,7 @@ struct rte_security_session_conf {
   		struct rte_security_ipsec_xform ipsec;
   		struct rte_security_macsec_xform macsec;
   		struct rte_security_pdcp_xform pdcp;
+		struct rte_security_cpu_crypto_xform cpucrypto;
   	};
   	/**< Configuration parameters for security session */
   	struct rte_crypto_sym_xform *crypto_xform;
@@ -639,6 +659,35 @@ const struct rte_security_capability *
   rte_security_capability_get(struct rte_security_ctx *instance,
   			    struct rte_security_capability_idx *idx);
 
Quoted 12 lines

lib/librte_security/rte_security.h const struct rte_security_capability * · line 670

Quoted 9 lines
lib/librte_security/rte_security.h +9 -0
+/**
+ * Security vector structure, contains pointer to vector array and the length
+ * of the array
+ */
+struct rte_security_vec {
+	struct iovec *vec;
+	uint32_t num;
+};
+

Just wondering if you want to change it to *in_vec and *out_vec, that will be helpful in future, if the out-of-place processing is required for CPU usecase as well?

Quoted 1 line

I suppose this is doable, though right now we don't plan to support such model.

They will come handy in future. I plan to use it in future and we can skip the API/ABI breakage, if the placeholder are present

Quoted 23 lines
Quoted 23 lines
Quoted 11 lines
lib/librte_security/rte_security.h +10 -0
+/**
+ * Processing bulk crypto workload with CPU
+ *
+ * @param	instance	security instance.
+ * @param	sess		security session
+ * @param	buf		array of buffer SGL vectors
+ * @param	iv		array of IV pointers
+ * @param	aad		array of AAD pointers
+ * @param	digest		array of digest pointers
+ * @param	status		array of status for the function to return

lib/librte_security/rte_security.h const struct rte_security_capability * · line 690

Quoted 10 lines
lib/librte_security/rte_security.h +10 -0
+ * @param	num		number of elements in each array
+ *
+ */
+__rte_experimental
+void
+rte_security_process_cpu_crypto_bulk(struct rte_security_ctx *instance,
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+

Why not make the return as int, to indicate whether this API completely failed or processed or have some valid status to look into?

Quoted 6 lines
Quoted 1 line

Good point, will change as suggested.

I have another suggestions w.r.t iv, aad, digest etc. Why not put them in a structure, so that you will

be able to add/remove the variable without breaking the API prototype.

Just to confirm, you are talking about something like:

struct rte_security_vec {
struct iovec *vec;
uint32_t num;
};

struct rte_security_sym_vec {
      struct rte_security_vec buf;
      void *iv;
      void *aad;
      void *digest;
};

rte_security_process_cpu_crypto_bulk(struct rte_security_ctx *instance,

	struct rte_security_session *sess, struct rte_security_sym_vec buf[],
               int status[], uint32_t num);

?
We thought about such way, though for PMD it would be more plausible to have same type of params grouped together, i.e. void *in[], void *out[], void *digest[], ... Another thing - above grouping wouldn't help to avoid ABI breakage, in case we'll need to add new field into rte_security_sym_vec (though it might help to avoid API breakage).

In theory other way is also possible:

struct rte_security_sym_vec {
      struct rte_security_vec *buf;
      void **iv;
      void **aad;
      void **digest;
};

rte_security_process_cpu_crypto_bulk(struct rte_security_ctx *instance,

	struct rte_security_session *sess, struct rte_security_sym_vec *buf,
               int status[], uint32_t num);

And that might help for both ABI and API stability,
but it looks really weird that way (at least to me).
Also this API is experimental and I suppose needs to stay experimental for few releases before we are sure nothing important is missing, so probably API/ABI stability is not that high concern for it right now.

Konstantin

#56 lore raw

Re: [dpdk-dev] [PATCH 01/10] security: introduce CPU Crypto action type and API

Hi Konstantin,

Quoted 165 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

This patch introduce new RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO action type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for

PMDs.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
---
   lib/librte_security/rte_security.c           | 16 +++++++++
   lib/librte_security/rte_security.h           | 51

+++++++++++++++++++++++++++-

Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
   lib/librte_security/rte_security_driver.h    | 19 +++++++++++
   lib/librte_security/rte_security_version.map |  1 +
   4 files changed, 86 insertions(+), 1 deletion(-)
lib/librte_security/rte_security.c +0 -0
index bc81ce15d..0f85c1b59 100644
@@ -141,3 +141,19 @@ rte_security_capability_get(struct
 rte_security_ctx *instance,
 
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

lib/librte_security/rte_security.c rte_security_capability_get(struct rte_security_ctx *instance,

Quoted 5 lines
lib/librte_security/rte_security.c +3 -0
   	return NULL;
   }
+
+void
+rte_security_process_cpu_crypto_bulk(struct rte_security_ctx

*instance,

Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
lib/librte_security/rte_security.c +12 -0
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num) {
+	uint32_t i;
+
+	for (i = 0; i < num; i++)
+		status[i] = -1;
+
+	RTE_FUNC_PTR_OR_RET(*instance->ops->process_cpu_crypto_bulk);
+	instance->ops->process_cpu_crypto_bulk(sess, buf, iv,
+			aad, digest, status, num);
+}
lib/librte_security/rte_security.h +1 -0
index 96806e3a2..5a0f8901b 100644
@@ -18,6 +18,7 @@ extern "C" {
   #endif
 
   #include <sys/types.h>
+#include <sys/uio.h>
 
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

lib/librte_security/rte_security.h struct rte_security_pdcp_xform { · line 277

Quoted 8 lines
lib/librte_security/rte_security.h +2 -0
   #include <netinet/in.h>
   #include <netinet/ip.h>
@@ -272,6 +273,20 @@ struct rte_security_pdcp_xform {
   	uint32_t hfn_threshold;
   };
 
+struct rte_security_cpu_crypto_xform {
+	/** For cipher/authentication crypto operation the authentication

may

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

lib/librte_security/rte_security.h struct rte_security_pdcp_xform { · line 279

Quoted 2 lines
lib/librte_security/rte_security.h +2 -0
+	 * cover more content then the cipher. E.g., for IPSec ESP encryption
+	 * with AES-CBC and SHA1-HMAC, the encryption happens after the

ESP

Quoted 1 line
Quoted 1 line
Quoted 1 line

lib/librte_security/rte_security.h struct rte_security_pdcp_xform { · line 280

Quoted 1 line
lib/librte_security/rte_security.h +1 -0
+	 * header but whole packet (apart from MAC header) is

authenticated.

Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Quoted 15 lines
lib/librte_security/rte_security.h +9 -0
+	 * The cipher_offset field is used to deduct the cipher data pointer
+	 * from the buffer to be processed.
+	 *
+	 * NOTE this parameter shall be ignored by AEAD algorithms, since it
+	 * uses the same offset for cipher and authentication.
+	 */
+	int32_t cipher_offset;
+};
+
   /**
    * Security session action type.
    */
@@ -286,10 +301,14 @@ enum rte_security_session_action_type {
   	/**< All security protocol processing is performed inline during
   	 * transmission

lib/librte_security/rte_security.h enum rte_security_session_action_type { · line 311

Quoted 10 lines
lib/librte_security/rte_security.h +5 -1
   	 */
-	RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL
+	RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL,
   	/**< All security protocol processing including crypto is performed
   	 * on a lookaside accelerator
   	 */
+	RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO
+	/**< Crypto processing for security protocol is processed by CPU
+	 * synchronously
+	 */

though you are naming it cpu crypto, but it is more like raw packet crypto, where you want to skip mbuf/crypto ops and directly wants to work on raw buffer.

Quoted 3 lines
Quoted 3 lines

Yes, but we do wat to do that (skip mbuf/crypto ops and use raw buffer), because this API is destined for SW backed implementation. For that case crypto-ops , mbuf, enqueue/dequeue are just unnecessary

overhead.

Quoted 31 lines

I agree, we are also planning to take advantage of it for some specific use-cases in future.

Quoted 29 lines
Quoted 15 lines
Quoted 15 lines
lib/librte_security/rte_security.h +0 -0
   };
 
   /** Security session protocol definition */ @@ -315,6 +334,7 @@
struct rte_security_session_conf {
   		struct rte_security_ipsec_xform ipsec;
   		struct rte_security_macsec_xform macsec;
   		struct rte_security_pdcp_xform pdcp;
lib/librte_security/rte_security.h +1 -0
+		struct rte_security_cpu_crypto_xform cpucrypto;
   	};
   	/**< Configuration parameters for security session */
   	struct rte_crypto_sym_xform *crypto_xform; @@ -639,6 +659,35

@@ const struct rte_security_capability *
rte_security_capability_get(struct rte_security_ctx *instance,
struct rte_security_capability_idx *idx);

Quoted 13 lines

lib/librte_security/rte_security.h const struct rte_security_capability * · line 670

Quoted 10 lines
lib/librte_security/rte_security.h +10 -0
+/**
+ * Security vector structure, contains pointer to vector array
+and the length
+ * of the array
+ */
+struct rte_security_vec {
+	struct iovec *vec;
+	uint32_t num;
+};
+

Just wondering if you want to change it to *in_vec and *out_vec, that will be helpful in future, if the out-of-place processing is required for CPU usecase as well?

I suppose this is doable, though right now we don't plan to support such

model.

Quoted 13 lines

They will come handy in future. I plan to use it in future and we can skip the API/ABI breakage, if the placeholder are present

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

+/**
+ * Processing bulk crypto workload with CPU
+ *
+ * @param instance security instance.
+ * @param sess security session
+ * @param buf array of buffer SGL vectors
+ * @param iv array of IV pointers
+ * @param aad array of AAD pointers
+ * @param digest array of digest pointers
+ * @param status array of status for the function to

return

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

lib/librte_security/rte_security.h const struct rte_security_capability *

Quoted 6 lines

+ * @param num number of elements in each array
+ *
+ */

lib/librte_security/rte_security.h +3 -0
+__rte_experimental
+void
+rte_security_process_cpu_crypto_bulk(struct rte_security_ctx

*instance,

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

lib/librte_security/rte_security.h const struct rte_security_capability * · line 690

Quoted 4 lines
lib/librte_security/rte_security.h +4 -0
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+

Why not make the return as int, to indicate whether this API completely failed or processed or have some valid status to look into?

Quoted 6 lines
Quoted 1 line

Good point, will change as suggested.

I have another suggestions w.r.t iv, aad, digest etc. Why not put them in a structure, so that you will

be able to add/remove the variable without breaking the API prototype.

Quoted 6 lines

Just to confirm, you are talking about something like:

struct rte_security_vec {
   struct iovec *vec;
   uint32_t num;
};

[Hemant] My idea is:

 struct rte_security_vec {
    struct iovec *vec;
    struct iovec *out_vec;
    uint32_t num_in;
    uint32_t num_out; 
};
Quoted 8 lines
struct rte_security_sym_vec {
      struct rte_security_vec buf;
      void *iv;
      void *aad;
      void *digest;
};

[Hemant] or leave the rte_security_vec altogether and make it part of rte_security_sym_vec itself.

Quoted 20 lines

rte_security_process_cpu_crypto_bulk(struct rte_security_ctx *instance,

	struct rte_security_session *sess, struct rte_security_sym_vec buf[],
               int status[], uint32_t num);

?
We thought about such way, though for PMD it would be more plausible to have same type of params grouped together, i.e. void *in[], void *out[], void *digest[], ...
Another thing - above grouping wouldn't help to avoid ABI breakage, in case we'll need to add new field into rte_security_sym_vec (though it might help to avoid API breakage).

In theory other way is also possible:

struct rte_security_sym_vec {
      struct rte_security_vec *buf;
      void **iv;
      void **aad;
      void **digest;
};
Quoted 6 lines

rte_security_process_cpu_crypto_bulk(struct rte_security_ctx *instance,

	struct rte_security_session *sess, struct rte_security_sym_vec *buf,
               int status[], uint32_t num);

And that might help for both ABI and API stability, but it looks really weird that way (at least to me).

[Hemant] I am fine either way.

Quoted 7 lines

Also this API is experimental and I suppose needs to stay experimental for few releases before we are sure nothing important is missing, so probably API/ABI stability is not that high concern for it right now.

Konstantin

author
#57 lore raw

PATCH 2/10 crypto/aesni_gcm: add rte_security handler

In reply to Fan Zhang

This patch add rte_security support support to AESNI-GCM PMD. The PMD now initialize security context instance, create/delete PMD specific security sessions, and process crypto workloads in synchronous mode with scatter-gather list buffer supported.

---
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c         | 91 ++++++++++++++++++++++-
 drivers/crypto/aesni_gcm/aesni_gcm_pmd_ops.c     | 95 ++++++++++++++++++++++++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h | 23 ++++++
 3 files changed, 208 insertions(+), 1 deletion(-)
drivers/crypto/aesni_gcm/aesni_gcm_pmd.c +90 -1
index 1006a5c4d..0a346eddd 100644
@@ -6,6 +6,7 @@
 #include <rte_hexdump.h>
 #include <rte_cryptodev.h>
 #include <rte_cryptodev_pmd.h>
+#include <rte_security_driver.h>
 #include <rte_bus_vdev.h>
 #include <rte_malloc.h>
 #include <rte_cpuflags.h>
@@ -174,6 +175,56 @@ aesni_gcm_get_session(struct aesni_gcm_qp *qp, struct rte_crypto_op *op)
 	return sess;
 }
 
+static __rte_always_inline int
+process_gcm_security_sgl_buf(struct aesni_gcm_security_session *sess,
+		struct rte_security_vec *buf, uint8_t *iv,
+		uint8_t *aad, uint8_t *digest)
+{
+	struct aesni_gcm_session *session = &sess->sess;
+	uint8_t *tag;
+	uint32_t i;
+
+	sess->init(&session->gdata_key, &sess->gdata_ctx, iv, aad,
+			(uint64_t)session->aad_length);
+
+	for (i = 0; i < buf->num; i++) {
+		struct iovec *vec = &buf->vec[i];
+
+		sess->update(&session->gdata_key, &sess->gdata_ctx,
+				vec->iov_base, vec->iov_base, vec->iov_len);
+	}
+
+	switch (session->op) {
+	case AESNI_GCM_OP_AUTHENTICATED_ENCRYPTION:
+		if (session->req_digest_length != session->gen_digest_length)
+			tag = sess->temp_digest;
+		else
+			tag = digest;
+
+		sess->finalize(&session->gdata_key, &sess->gdata_ctx, tag,
+				session->gen_digest_length);
+
+		if (session->req_digest_length != session->gen_digest_length)
+			memcpy(digest, sess->temp_digest,
+					session->req_digest_length);
+		break;
+
+	case AESNI_GCM_OP_AUTHENTICATED_DECRYPTION:
+		tag = sess->temp_digest;
+
+		sess->finalize(&session->gdata_key, &sess->gdata_ctx, tag,
+				session->gen_digest_length);
+
+		if (memcmp(tag, digest,	session->req_digest_length) != 0)
+			return -1;
+		break;
+	default:
+		return -1;
+	}
+
+	return 0;
+}
+
 /**
  * Process a crypto operation, calling
  * the GCM API from the multi buffer library.
@@ -488,8 +539,10 @@ aesni_gcm_create(const char *name,
 {
 	struct rte_cryptodev *dev;
 	struct aesni_gcm_private *internals;
+	struct rte_security_ctx *sec_ctx;
 	enum aesni_gcm_vector_mode vector_mode;
 	MB_MGR *mb_mgr;
+	char sec_name[RTE_DEV_NAME_MAX_LEN];
 
 	/* Check CPU for support for AES instruction set */
 	if (!rte_cpu_get_flag_enabled(RTE_CPUFLAG_AES)) {
@@ -524,7 +577,8 @@ aesni_gcm_create(const char *name,
 			RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
 			RTE_CRYPTODEV_FF_CPU_AESNI |
 			RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT |
-			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT;
+			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
+			RTE_CRYPTODEV_FF_SECURITY;
 
 	mb_mgr = alloc_mb_mgr(0);
 	if (mb_mgr == NULL)
@@ -587,6 +641,21 @@ aesni_gcm_create(const char *name,
 
 	internals->max_nb_queue_pairs = init_params->max_nb_queue_pairs;
 
+	/* setup security operations */
+	snprintf(sec_name, sizeof(sec_name) - 1, "aes_gcm_sec_%u",
+			dev->driver_id);
+	sec_ctx = rte_zmalloc_socket(sec_name,
+			sizeof(struct rte_security_ctx),
+			RTE_CACHE_LINE_SIZE, init_params->socket_id);
+	if (sec_ctx == NULL) {
+		AESNI_GCM_LOG(ERR, "memory allocation failed\n");
+		goto error_exit;
+	}
+
+	sec_ctx->device = (void *)dev;
+	sec_ctx->ops = rte_aesni_gcm_pmd_security_ops;
+	dev->security_ctx = sec_ctx;
+
 #if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
 	AESNI_GCM_LOG(INFO, "IPSec Multi-buffer library version used: %s\n",
 			imb_get_version_str());
@@ -641,6 +710,8 @@ aesni_gcm_remove(struct rte_vdev_device *vdev)
 	if (cryptodev == NULL)
 		return -ENODEV;
 
+	rte_free(cryptodev->security_ctx);
+
 	internals = cryptodev->data->dev_private;
 
 	free_mb_mgr(internals->mb_mgr);
@@ -648,6 +719,24 @@ aesni_gcm_remove(struct rte_vdev_device *vdev)
 	return rte_cryptodev_pmd_destroy(cryptodev);
 }
 
+void
+aesni_gcm_sec_crypto_process_bulk(struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num)
+{
+	struct aesni_gcm_security_session *session =
+			get_sec_session_private_data(sess);
+	uint32_t i;
+
+	if (unlikely(!session))
+		return;
+
+	for (i = 0; i < num; i++)
+		status[i] = process_gcm_security_sgl_buf(session, &buf[i],
+				(uint8_t *)iv[i], (uint8_t *)aad[i],
+				(uint8_t *)digest[i]);
+}
+
 static struct rte_vdev_driver aesni_gcm_pmd_drv = {
 	.probe = aesni_gcm_probe,
 	.remove = aesni_gcm_remove
drivers/crypto/aesni_gcm/aesni_gcm_pmd_ops.c +95 -0

111 lines not on this page · raw message

drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h +23 -0
index 56b29e013..8e490b6ce 100644
@@ -114,5 +114,28 @@ aesni_gcm_set_session_parameters(const struct aesni_gcm_ops *ops,
  * Device specific operations function pointer structure */
 extern struct rte_cryptodev_ops *rte_aesni_gcm_pmd_ops;
 
+/**
+ * Security session structure.
+ */
+struct aesni_gcm_security_session {
+	/** Temp digest for decryption */
+	uint8_t temp_digest[DIGEST_LENGTH_MAX];
+	/** GCM operations */
+	aesni_gcm_pre_t pre;
+	aesni_gcm_init_t init;
+	aesni_gcm_update_t update;
+	aesni_gcm_finalize_t finalize;
+	/** AESNI-GCM session */
+	struct aesni_gcm_session sess;
+	/** AESNI-GCM context */
+	struct gcm_context_data gdata_ctx;
+};
+
+extern void
+aesni_gcm_sec_crypto_process_bulk(struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+
+extern struct rte_security_ops *rte_aesni_gcm_pmd_security_ops;
 
 #endif /* _RTE_AESNI_GCM_PMD_PRIVATE_H_ */
Signature · 1 line
2.14.5
#58 lore raw

Re: [dpdk-dev] [PATCH 02/10] crypto/aesni_gcm: add rte_security handler

Hi Fan,

Quoted 52 lines

This patch add rte_security support support to AESNI-GCM PMD. The PMD now initialize security context instance, create/delete PMD specific security sessions, and process crypto workloads in synchronous mode with scatter-gather list buffer supported.Hi

---
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c         | 91 ++++++++++++++++++++++-
 drivers/crypto/aesni_gcm/aesni_gcm_pmd_ops.c     | 95 ++++++++++++++++++++++++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h | 23 ++++++
 3 files changed, 208 insertions(+), 1 deletion(-)
drivers/crypto/aesni_gcm/aesni_gcm_pmd.c +24 -0
index 1006a5c4d..0a346eddd 100644
@@ -6,6 +6,7 @@
 #include <rte_hexdump.h>
 #include <rte_cryptodev.h>
 #include <rte_cryptodev_pmd.h>
+#include <rte_security_driver.h>
 #include <rte_bus_vdev.h>
 #include <rte_malloc.h>
 #include <rte_cpuflags.h>
@@ -174,6 +175,56 @@ aesni_gcm_get_session(struct aesni_gcm_qp *qp, struct rte_crypto_op *op)
 	return sess;
 }
 
+static __rte_always_inline int
+process_gcm_security_sgl_buf(struct aesni_gcm_security_session *sess,
+		struct rte_security_vec *buf, uint8_t *iv,
+		uint8_t *aad, uint8_t *digest)
+{
+	struct aesni_gcm_session *session = &sess->sess;
+	uint8_t *tag;
+	uint32_t i;
+
+	sess->init(&session->gdata_key, &sess->gdata_ctx, iv, aad,
+			(uint64_t)session->aad_length);
+
+	for (i = 0; i < buf->num; i++) {
+		struct iovec *vec = &buf->vec[i];
+
+		sess->update(&session->gdata_key, &sess->gdata_ctx,
+				vec->iov_base, vec->iov_base, vec->iov_len);
+	}
+
+	switch (session->op) {
+	case AESNI_GCM_OP_AUTHENTICATED_ENCRYPTION:
+		if (session->req_digest_length != session->gen_digest_length)
+			tag = sess->temp_digest;

drivers/crypto/aesni_gcm/aesni_gcm_pmd.c aesni_gcm_get_session(struct aesni_gcm_qp *qp, struct rte_crypto_op *op) · line 210

Quoted 10 lines
drivers/crypto/aesni_gcm/aesni_gcm_pmd.c +10 -0
+		else
+			tag = digest;
+
+		sess->finalize(&session->gdata_key, &sess->gdata_ctx, tag,
+				session->gen_digest_length);
+
+		if (session->req_digest_length != session->gen_digest_length)
+			memcpy(digest, sess->temp_digest,
+					session->req_digest_length);
+		break;

Wonder can we move all these cases and ifs into session_create() time - so instead of one process() function with a lot of branches, we'll have several process functions with minimal/none branches. I think it should help us to save extra cycles.

Quoted 77 lines
drivers/crypto/aesni_gcm/aesni_gcm_pmd.c +39 -1
+
+	case AESNI_GCM_OP_AUTHENTICATED_DECRYPTION:
+		tag = sess->temp_digest;
+
+		sess->finalize(&session->gdata_key, &sess->gdata_ctx, tag,
+				session->gen_digest_length);
+
+		if (memcmp(tag, digest,	session->req_digest_length) != 0)
+			return -1;
+		break;
+	default:
+		return -1;
+	}
+
+	return 0;
+}
+
 /**
  * Process a crypto operation, calling
  * the GCM API from the multi buffer library.
@@ -488,8 +539,10 @@ aesni_gcm_create(const char *name,
 {
 	struct rte_cryptodev *dev;
 	struct aesni_gcm_private *internals;
+	struct rte_security_ctx *sec_ctx;
 	enum aesni_gcm_vector_mode vector_mode;
 	MB_MGR *mb_mgr;
+	char sec_name[RTE_DEV_NAME_MAX_LEN];
 
 	/* Check CPU for support for AES instruction set */
 	if (!rte_cpu_get_flag_enabled(RTE_CPUFLAG_AES)) {
@@ -524,7 +577,8 @@ aesni_gcm_create(const char *name,
 			RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
 			RTE_CRYPTODEV_FF_CPU_AESNI |
 			RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT |
-			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT;
+			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
+			RTE_CRYPTODEV_FF_SECURITY;
 
 	mb_mgr = alloc_mb_mgr(0);
 	if (mb_mgr == NULL)
@@ -587,6 +641,21 @@ aesni_gcm_create(const char *name,
 
 	internals->max_nb_queue_pairs = init_params->max_nb_queue_pairs;
 
+	/* setup security operations */
+	snprintf(sec_name, sizeof(sec_name) - 1, "aes_gcm_sec_%u",
+			dev->driver_id);
+	sec_ctx = rte_zmalloc_socket(sec_name,
+			sizeof(struct rte_security_ctx),
+			RTE_CACHE_LINE_SIZE, init_params->socket_id);
+	if (sec_ctx == NULL) {
+		AESNI_GCM_LOG(ERR, "memory allocation failed\n");
+		goto error_exit;
+	}
+
+	sec_ctx->device = (void *)dev;
+	sec_ctx->ops = rte_aesni_gcm_pmd_security_ops;
+	dev->security_ctx = sec_ctx;
+
 #if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
 	AESNI_GCM_LOG(INFO, "IPSec Multi-buffer library version used: %s\n",
 			imb_get_version_str());
@@ -641,6 +710,8 @@ aesni_gcm_remove(struct rte_vdev_device *vdev)
 	if (cryptodev == NULL)
 		return -ENODEV;
 
+	rte_free(cryptodev->security_ctx);
+
 	internals = cryptodev->data->dev_private;
 
 	free_mb_mgr(internals->mb_mgr);
@@ -648,6 +719,24 @@ aesni_gcm_remove(struct rte_vdev_device *vdev)
 	return rte_cryptodev_pmd_destroy(cryptodev);
 }
 
+void

drivers/crypto/aesni_gcm/aesni_gcm_pmd.c aesni_gcm_remove(struct rte_vdev_device *vdev) · line 732

Quoted 10 lines
drivers/crypto/aesni_gcm/aesni_gcm_pmd.c +10 -0
+aesni_gcm_sec_crypto_process_bulk(struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num)
+{
+	struct aesni_gcm_security_session *session =
+			get_sec_session_private_data(sess);
+	uint32_t i;
+
+	if (unlikely(!session))
+		return;

I think you can't just return here, you need to
set all status[] entries to some -errno value.

Quoted 163 lines
drivers/crypto/aesni_gcm/aesni_gcm_pmd.c +7 -0
+
+	for (i = 0; i < num; i++)
+		status[i] = process_gcm_security_sgl_buf(session, &buf[i],
+				(uint8_t *)iv[i], (uint8_t *)aad[i],
+				(uint8_t *)digest[i]);
+}
+
 static struct rte_vdev_driver aesni_gcm_pmd_drv = {
 	.probe = aesni_gcm_probe,
 	.remove = aesni_gcm_remove
drivers/crypto/aesni_gcm/aesni_gcm_pmd_ops.c +95 -0
index 2f66c7c58..cc71dbd60 100644
@@ -7,6 +7,7 @@
 #include <rte_common.h>
 #include <rte_malloc.h>
 #include <rte_cryptodev_pmd.h>
+#include <rte_security_driver.h>
 
 #include "aesni_gcm_pmd_private.h"
 
@@ -316,6 +317,85 @@ aesni_gcm_pmd_sym_session_clear(struct rte_cryptodev *dev,
 	}
 }
 
+static int
+aesni_gcm_security_session_create(void *dev,
+		struct rte_security_session_conf *conf,
+		struct rte_security_session *sess,
+		struct rte_mempool *mempool)
+{
+	struct rte_cryptodev *cdev = dev;
+	struct aesni_gcm_private *internals = cdev->data->dev_private;
+	struct aesni_gcm_security_session *sess_priv;
+	int ret;
+
+	if (!conf->crypto_xform) {
+		AESNI_GCM_LOG(ERR, "Invalid security session conf");
+		return -EINVAL;
+	}
+
+	if (conf->crypto_xform->type == RTE_CRYPTO_SYM_XFORM_AUTH) {
+		AESNI_GCM_LOG(ERR, "GMAC is not supported in security session");
+		return -EINVAL;
+	}
+
+
+	if (rte_mempool_get(mempool, (void **)(&sess_priv))) {
+		AESNI_GCM_LOG(ERR,
+				"Couldn't get object from session mempool");
+		return -ENOMEM;
+	}
+
+	ret = aesni_gcm_set_session_parameters(internals->ops,
+				&sess_priv->sess, conf->crypto_xform);
+	if (ret != 0) {
+		AESNI_GCM_LOG(ERR, "Failed configure session parameters");
+
+		/* Return session to mempool */
+		rte_mempool_put(mempool, (void *)sess_priv);
+		return ret;
+	}
+
+	sess_priv->pre = internals->ops[sess_priv->sess.key].pre;
+	sess_priv->init = internals->ops[sess_priv->sess.key].init;
+	if (sess_priv->sess.op == AESNI_GCM_OP_AUTHENTICATED_ENCRYPTION) {
+		sess_priv->update =
+			internals->ops[sess_priv->sess.key].update_enc;
+		sess_priv->finalize =
+			internals->ops[sess_priv->sess.key].finalize_enc;
+	} else {
+		sess_priv->update =
+			internals->ops[sess_priv->sess.key].update_dec;
+		sess_priv->finalize =
+			internals->ops[sess_priv->sess.key].finalize_dec;
+	}
+
+	sess->sess_private_data = sess_priv;
+
+	return 0;
+}
+
+static int
+aesni_gcm_security_session_destroy(void *dev __rte_unused,
+		struct rte_security_session *sess)
+{
+	void *sess_priv = get_sec_session_private_data(sess);
+
+	if (sess_priv) {
+		struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv);
+
+		memset(sess, 0, sizeof(struct aesni_gcm_security_session));
+		set_sec_session_private_data(sess, NULL);
+		rte_mempool_put(sess_mp, sess_priv);
+	}
+	return 0;
+}
+
+static unsigned int
+aesni_gcm_sec_session_get_size(__rte_unused void *device)
+{
+	return sizeof(struct aesni_gcm_security_session);
+}
+
 struct rte_cryptodev_ops aesni_gcm_pmd_ops = {
 		.dev_configure		= aesni_gcm_pmd_config,
 		.dev_start		= aesni_gcm_pmd_start,
@@ -336,4 +416,19 @@ struct rte_cryptodev_ops aesni_gcm_pmd_ops = {
 		.sym_session_clear	= aesni_gcm_pmd_sym_session_clear
 };
 
+static struct rte_security_ops aesni_gcm_security_ops = {
+		.session_create = aesni_gcm_security_session_create,
+		.session_get_size = aesni_gcm_sec_session_get_size,
+		.session_update = NULL,
+		.session_stats_get = NULL,
+		.session_destroy = aesni_gcm_security_session_destroy,
+		.set_pkt_metadata = NULL,
+		.capabilities_get = NULL,
+		.process_cpu_crypto_bulk =
+				aesni_gcm_sec_crypto_process_bulk,
+};
+
 struct rte_cryptodev_ops *rte_aesni_gcm_pmd_ops = &aesni_gcm_pmd_ops;
+
+struct rte_security_ops *rte_aesni_gcm_pmd_security_ops =
+		&aesni_gcm_security_ops;
drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h +23 -1
index 56b29e013..8e490b6ce 100644
@@ -114,5 +114,28 @@ aesni_gcm_set_session_parameters(const struct aesni_gcm_ops *ops,
  * Device specific operations function pointer structure */
 extern struct rte_cryptodev_ops *rte_aesni_gcm_pmd_ops;
 
+/**
+ * Security session structure.
+ */
+struct aesni_gcm_security_session {
+	/** Temp digest for decryption */
+	uint8_t temp_digest[DIGEST_LENGTH_MAX];
+	/** GCM operations */
+	aesni_gcm_pre_t pre;
+	aesni_gcm_init_t init;
+	aesni_gcm_update_t update;
+	aesni_gcm_finalize_t finalize;
+	/** AESNI-GCM session */
+	struct aesni_gcm_session sess;
+	/** AESNI-GCM context */
+	struct gcm_context_data gdata_ctx;
+};
+
+extern void
+aesni_gcm_sec_crypto_process_bulk(struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+
+extern struct rte_security_ops *rte_aesni_gcm_pmd_security_ops;
 
 #endif /* _RTE_AESNI_GCM_PMD_PRIVATE_H_ */
--

2.14.5

author
#59 lore raw

PATCH 3/10 app/test: add security cpu crypto autotest

In reply to Fan Zhang

This patch adds cpu crypto unit test for AESNI_GCM PMD.

---
 app/test/Makefile                   |   1 +
 app/test/meson.build                |   1 +
 app/test/test_security_cpu_crypto.c | 564 ++++++++++++++++++++++++++++++++++++
 3 files changed, 566 insertions(+)
 create mode 100644 app/test/test_security_cpu_crypto.c
app/test/Makefile +1 -0
index 26ba6fe2b..090c55746 100644
@@ -196,6 +196,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_RING) += test_pmd_ring_perf.c
 SRCS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += test_cryptodev_blockcipher.c
 SRCS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += test_cryptodev.c
 SRCS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += test_cryptodev_asym.c
+SRCS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += test_security_cpu_crypto.c
 
 SRCS-$(CONFIG_RTE_LIBRTE_METRICS) += test_metrics.c
 
app/test/meson.build +1 -0
index ec40943bd..b7834ff21 100644
@@ -103,6 +103,7 @@ test_sources = files('commands.c',
 	'test_ring_perf.c',
 	'test_rwlock.c',
 	'test_sched.c',
+	'test_security_cpu_crypto.c',
 	'test_service_cores.c',
 	'test_spinlock.c',
 	'test_stack.c',
app/test/test_security_cpu_crypto.c new file +564 -0

564 lines not on this page · raw message

Signature · 1 line
2.14.5
author
#60 lore raw

PATCH 4/10 app/test: add security cpu crypto perftest

In reply to Fan Zhang

Since crypto perf application does not support rte_security, this patch adds a simple GCM CPU crypto performance test to crypto unittest application. The test includes different key and data sizes test with single buffer and SGL buffer test items and will display the throughput as well as cycle count performance information.

---
 app/test/test_security_cpu_crypto.c | 201 ++++++++++++++++++++++++++++++++++++
 1 file changed, 201 insertions(+)
app/test/test_security_cpu_crypto.c +201 -0
index d345922b2..ca9a8dae6 100644
@@ -23,6 +23,7 @@
 
 #define CPU_CRYPTO_TEST_MAX_AAD_LENGTH	16
 #define MAX_NB_SIGMENTS			4
+#define CACHE_WARM_ITER			2048
 
 enum buffer_assemble_option {
 	SGL_MAX_SEG,
@@ -560,5 +561,205 @@ test_security_cpu_crypto_aesni_gcm(void)
 	return unit_test_suite_runner(&security_cpu_crypto_aesgcm_testsuite);
 }
 
+
+static inline void
+gen_rand(uint8_t *data, uint32_t len)
+{
+	uint32_t i;
+
+	for (i = 0; i < len; i++)
+		data[i] = (uint8_t)rte_rand();
+}
+
+static inline void
+switch_aead_enc_to_dec(struct aead_test_data *tdata,
+		struct cpu_crypto_test_case *tcase,
+		enum buffer_assemble_option sgl_option)
+{
+	uint32_t i;
+	uint8_t *dst = tdata->ciphertext.data;
+
+	switch (sgl_option) {
+	case SGL_ONE_SEG:
+		memcpy(dst, tcase->seg_buf[0].seg, tcase->seg_buf[0].seg_len);
+		tdata->ciphertext.len = tcase->seg_buf[0].seg_len;
+		break;
+	case SGL_MAX_SEG:
+		tdata->ciphertext.len = 0;
+		for (i = 0; i < MAX_NB_SIGMENTS; i++) {
+			memcpy(dst, tcase->seg_buf[i].seg,
+					tcase->seg_buf[i].seg_len);
+			tdata->ciphertext.len += tcase->seg_buf[i].seg_len;
+		}
+		break;
+	}
+
+	memcpy(tdata->auth_tag.data, tcase->digest, tdata->auth_tag.len);
+}
+
+static int
+cpu_crypto_test_aead_perf(enum buffer_assemble_option sgl_option,
+		uint32_t key_sz)
+{
+	struct aead_test_data tdata = {0};
+	struct cpu_crypto_testsuite_params *ts_params = &testsuite_params;
+	struct cpu_crypto_unittest_params *ut_params = &unittest_params;
+	struct cpu_crypto_test_obj *obj = &ut_params->test_obj;
+	struct cpu_crypto_test_case *tcase;
+	uint64_t hz = rte_get_tsc_hz(), time_start, time_now;
+	double rate, cycles_per_buf;
+	uint32_t test_data_szs[] = {64, 128, 256, 512, 1024, 2048};
+	uint32_t i, j;
+	uint8_t aad[16];
+	int ret;
+
+	tdata.key.len = key_sz;
+	gen_rand(tdata.key.data, tdata.key.len);
+	tdata.algo = RTE_CRYPTO_AEAD_AES_GCM;
+	tdata.aad.data = aad;
+
+	ut_params->sess = create_aead_session(ts_params->ctx,
+			ts_params->session_priv_mpool,
+			RTE_CRYPTO_AEAD_OP_DECRYPT,
+			&tdata,
+			0);
+	if (!ut_params->sess)
+		return -1;
+
+	ret = allocate_buf(MAX_NUM_OPS_INFLIGHT);
+	if (ret)
+		return ret;
+
+	for (i = 0; i < RTE_DIM(test_data_szs); i++) {
+		for (j = 0; j < MAX_NUM_OPS_INFLIGHT; j++) {
+			tdata.plaintext.len = test_data_szs[i];
+			gen_rand(tdata.plaintext.data,
+					tdata.plaintext.len);
+
+			tdata.aad.len = 12;
+			gen_rand(tdata.aad.data, tdata.aad.len);
+
+			tdata.auth_tag.len = 16;
+
+			tdata.iv.len = 16;
+			gen_rand(tdata.iv.data, tdata.iv.len);
+
+			tcase = ut_params->test_datas[j];
+			ret = assemble_aead_buf(tcase, obj, j,
+					RTE_CRYPTO_AEAD_OP_ENCRYPT,
+					&tdata, sgl_option, 0);
+			if (ret < 0) {
+				printf("Test is not supported by the driver\n");
+				return ret;
+			}
+		}
+
+		/* warm up cache */
+		for (j = 0; j < CACHE_WARM_ITER; j++)
+			run_test(ts_params->ctx, ut_params->sess, obj,
+					MAX_NUM_OPS_INFLIGHT);
+
+		time_start = rte_rdtsc();
+
+		run_test(ts_params->ctx, ut_params->sess, obj,
+				MAX_NUM_OPS_INFLIGHT);
+
+		time_now = rte_rdtsc();
+
+		rate = time_now - time_start;
+		cycles_per_buf = rate / MAX_NUM_OPS_INFLIGHT;
+
+		rate = ((hz / cycles_per_buf)) / 1000000;
+
+		printf("AES-GCM-%u(%4uB) Enc %03.3fMpps (%03.3fGbps) ",
+				key_sz * 8, test_data_szs[i], rate,
+				rate  * test_data_szs[i] * 8 / 1000);
+		printf("cycles per buf %03.3f per byte %03.3f\n",
+				cycles_per_buf,
+				cycles_per_buf / test_data_szs[i]);
+
+		for (j = 0; j < MAX_NUM_OPS_INFLIGHT; j++) {
+			tcase = ut_params->test_datas[j];
+
+			switch_aead_enc_to_dec(&tdata, tcase, sgl_option);
+			ret = assemble_aead_buf(tcase, obj, j,
+					RTE_CRYPTO_AEAD_OP_DECRYPT,
+					&tdata, sgl_option, 0);
+			if (ret < 0) {
+				printf("Test is not supported by the driver\n");
+				return ret;
+			}
+		}
+
+		time_start = rte_get_timer_cycles();
+
+		run_test(ts_params->ctx, ut_params->sess, obj,
+				MAX_NUM_OPS_INFLIGHT);
+
+		time_now = rte_get_timer_cycles();
+
+		rate = time_now - time_start;
+		cycles_per_buf = rate / MAX_NUM_OPS_INFLIGHT;
+
+		rate = ((hz / cycles_per_buf)) / 1000000;
+
+		printf("AES-GCM-%u(%4uB) Dec %03.3fMpps (%03.3fGbps) ",
+				key_sz * 8, test_data_szs[i], rate,
+				rate  * test_data_szs[i] * 8 / 1000);
+		printf("cycles per buf %03.3f per byte %03.3f\n",
+				cycles_per_buf,
+				cycles_per_buf / test_data_szs[i]);
+	}
+
+	return 0;
+}
+
+/* test-perfix/key-size/sgl-type */
+#define all_gcm_perf_test_cases(type)					\
+	TEST_EXPAND(_128, 16, type)					\
+	TEST_EXPAND(_192, 24, type)					\
+	TEST_EXPAND(_256, 32, type)
+
+#define TEST_EXPAND(a, b, c)						\
+static int								\
+cpu_crypto_gcm_perf##a##_##c(void)					\
+{									\
+	return cpu_crypto_test_aead_perf(c, b);				\
+}									\
+
+all_gcm_perf_test_cases(SGL_ONE_SEG)
+all_gcm_perf_test_cases(SGL_MAX_SEG)
+#undef TEST_EXPAND
+
+static struct unit_test_suite security_cpu_crypto_aesgcm_perf_testsuite  = {
+		.suite_name = "Security CPU Crypto AESNI-GCM Perf Test Suite",
+		.setup = testsuite_setup,
+		.teardown = testsuite_teardown,
+		.unit_test_cases = {
+#define TEST_EXPAND(a, b, c)						\
+		TEST_CASE_ST(ut_setup, ut_teardown,			\
+				cpu_crypto_gcm_perf##a##_##c),		\
+
+		all_gcm_perf_test_cases(SGL_ONE_SEG)
+		all_gcm_perf_test_cases(SGL_MAX_SEG)
+#undef TEST_EXPAND
+
+		TEST_CASES_END() /**< NULL terminate unit test array */
+		},
+};
+
+static int
+test_security_cpu_crypto_aesni_gcm_perf(void)
+{
+	gbl_driver_id =	rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD));
+
+	return unit_test_suite_runner(
+			&security_cpu_crypto_aesgcm_perf_testsuite);
+}
+
 REGISTER_TEST_COMMAND(security_aesni_gcm_autotest,
 		test_security_cpu_crypto_aesni_gcm);
+
+REGISTER_TEST_COMMAND(security_aesni_gcm_perftest,
+		test_security_cpu_crypto_aesni_gcm_perf);
Signature · 1 line
2.14.5
author
#61 lore raw

PATCH 5/10 crypto/aesni_mb: add rte_security handler

In reply to Fan Zhang

This patch add rte_security support support to AESNI-MB PMD. The PMD now initialize security context instance, create/delete PMD specific security sessions, and process crypto workloads in synchronous mode.

---
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c         | 291 ++++++++++++++++++++-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c     |  91 ++++++-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h |  21 +-
 3 files changed, 398 insertions(+), 5 deletions(-)
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +290 -1
index b495a9679..68767c04e 100644
@@ -8,6 +8,8 @@
 #include <rte_hexdump.h>
 #include <rte_cryptodev.h>
 #include <rte_cryptodev_pmd.h>
+#include <rte_security.h>
+#include <rte_security_driver.h>
 #include <rte_bus_vdev.h>
 #include <rte_malloc.h>
 #include <rte_cpuflags.h>
@@ -789,6 +791,167 @@ auth_start_offset(struct rte_crypto_op *op, struct aesni_mb_session *session,
 			(UINT64_MAX - u_src + u_dst + 1);
 }
 
+union sec_userdata_field {
+	int status;
+	struct {
+		uint16_t is_gen_digest;
+		uint16_t digest_len;
+	};
+};
+
+struct sec_udata_digest_field {
+	uint32_t is_digest_gen;
+	uint32_t digest_len;
+};
+
+static inline int
+set_mb_job_params_sec(JOB_AES_HMAC *job, struct aesni_mb_sec_session *sec_sess,
+		void *buf, uint32_t buf_len, void *iv, void *aad, void *digest,
+		int *status, uint8_t *digest_idx)
+{
+	struct aesni_mb_session *session = &sec_sess->sess;
+	uint32_t cipher_offset = sec_sess->cipher_offset;
+	void *user_digest = NULL;
+	union sec_userdata_field udata;
+
+	if (unlikely(cipher_offset > buf_len))
+		return -EINVAL;
+
+	/* Set crypto operation */
+	job->chain_order = session->chain_order;
+
+	/* Set cipher parameters */
+	job->cipher_direction = session->cipher.direction;
+	job->cipher_mode = session->cipher.mode;
+
+	job->aes_key_len_in_bytes = session->cipher.key_length_in_bytes;
+
+	/* Set authentication parameters */
+	job->hash_alg = session->auth.algo;
+	job->iv = iv;
+
+	switch (job->hash_alg) {
+	case AES_XCBC:
+		job->u.XCBC._k1_expanded = session->auth.xcbc.k1_expanded;
+		job->u.XCBC._k2 = session->auth.xcbc.k2;
+		job->u.XCBC._k3 = session->auth.xcbc.k3;
+
+		job->aes_enc_key_expanded =
+				session->cipher.expanded_aes_keys.encode;
+		job->aes_dec_key_expanded =
+				session->cipher.expanded_aes_keys.decode;
+		break;
+
+	case AES_CCM:
+		job->u.CCM.aad = (uint8_t *)aad + 18;
+		job->u.CCM.aad_len_in_bytes = session->aead.aad_len;
+		job->aes_enc_key_expanded =
+				session->cipher.expanded_aes_keys.encode;
+		job->aes_dec_key_expanded =
+				session->cipher.expanded_aes_keys.decode;
+		job->iv++;
+		break;
+
+	case AES_CMAC:
+		job->u.CMAC._key_expanded = session->auth.cmac.expkey;
+		job->u.CMAC._skey1 = session->auth.cmac.skey1;
+		job->u.CMAC._skey2 = session->auth.cmac.skey2;
+		job->aes_enc_key_expanded =
+				session->cipher.expanded_aes_keys.encode;
+		job->aes_dec_key_expanded =
+				session->cipher.expanded_aes_keys.decode;
+		break;
+
+	case AES_GMAC:
+		if (session->cipher.mode == GCM) {
+			job->u.GCM.aad = aad;
+			job->u.GCM.aad_len_in_bytes = session->aead.aad_len;
+		} else {
+			/* For GMAC */
+			job->u.GCM.aad = aad;
+			job->u.GCM.aad_len_in_bytes = buf_len;
+			job->cipher_mode = GCM;
+		}
+		job->aes_enc_key_expanded = &session->cipher.gcm_key;
+		job->aes_dec_key_expanded = &session->cipher.gcm_key;
+		break;
+
+	default:
+		job->u.HMAC._hashed_auth_key_xor_ipad =
+				session->auth.pads.inner;
+		job->u.HMAC._hashed_auth_key_xor_opad =
+				session->auth.pads.outer;
+
+		if (job->cipher_mode == DES3) {
+			job->aes_enc_key_expanded =
+				session->cipher.exp_3des_keys.ks_ptr;
+			job->aes_dec_key_expanded =
+				session->cipher.exp_3des_keys.ks_ptr;
+		} else {
+			job->aes_enc_key_expanded =
+				session->cipher.expanded_aes_keys.encode;
+			job->aes_dec_key_expanded =
+				session->cipher.expanded_aes_keys.decode;
+		}
+	}
+
+	/* Set digest output location */
+	if (job->hash_alg != NULL_HASH &&
+			session->auth.operation == RTE_CRYPTO_AUTH_OP_VERIFY) {
+		job->auth_tag_output = sec_sess->temp_digests[*digest_idx];
+		*digest_idx = (*digest_idx + 1) % MAX_JOBS;
+
+		udata.is_gen_digest = 0;
+		udata.digest_len = session->auth.req_digest_len;
+		user_digest = (void *)digest;
+	} else {
+		udata.is_gen_digest = 1;
+		udata.digest_len = session->auth.req_digest_len;
+
+		if (session->auth.req_digest_len !=
+				session->auth.gen_digest_len) {
+			job->auth_tag_output =
+					sec_sess->temp_digests[*digest_idx];
+			*digest_idx = (*digest_idx + 1) % MAX_JOBS;
+
+			user_digest = (void *)digest;
+		} else
+			job->auth_tag_output = digest;
+
+		/* A bit of hack here, since job structure only supports
+		 * 2 user data fields and we need 4 params to be passed
+		 * (status, direction, digest for verify, and length of
+		 * digest), we set the status value as digest length +
+		 * direction here temporarily to avoid creating longer
+		 * buffer to store all 4 params.
+		 */
+		*status = udata.status;
+	}
+	/*
+	 * Multi-buffer library current only support returning a truncated
+	 * digest length as specified in the relevant IPsec RFCs
+	 */
+
+	/* Set digest length */
+	job->auth_tag_output_len_in_bytes = session->auth.gen_digest_len;
+
+	/* Set IV parameters */
+	job->iv_len_in_bytes = session->iv.length;
+
+	/* Data Parameters */
+	job->src = buf;
+	job->dst = buf;
+	job->cipher_start_src_offset_in_bytes = cipher_offset;
+	job->msg_len_to_cipher_in_bytes = buf_len - cipher_offset;
+	job->hash_start_src_offset_in_bytes = 0;
+	job->msg_len_to_hash_in_bytes = buf_len;
+
+	job->user_data = (void *)status;
+	job->user_data2 = user_digest;
+
+	return 0;
+}
+
 /**
  * Process a crypto operation and complete a JOB_AES_HMAC job structure for
  * submission to the multi buffer library for processing.
@@ -1081,6 +1244,37 @@ post_process_mb_job(struct aesni_mb_qp *qp, JOB_AES_HMAC *job)
 	return op;
 }
 
+static inline void
+post_process_mb_sec_job(JOB_AES_HMAC *job)
+{
+	void *user_digest = job->user_data2;
+	int *status = job->user_data;
+	union sec_userdata_field udata;
+
+	switch (job->status) {
+	case STS_COMPLETED:
+		if (user_digest) {
+			udata.status = *status;
+
+			if (udata.is_gen_digest) {
+				*status = RTE_CRYPTO_OP_STATUS_SUCCESS;
+				memcpy(user_digest, job->auth_tag_output,
+						udata.digest_len);
+			} else {
+				verify_digest(job, user_digest,
+					udata.digest_len, (uint8_t *)status);
+
+				if (*status == RTE_CRYPTO_OP_STATUS_AUTH_FAILED)
+					*status = -1;
+			}
+		} else
+			*status = RTE_CRYPTO_OP_STATUS_SUCCESS;
+		break;
+	default:
+		*status = RTE_CRYPTO_OP_STATUS_ERROR;
+	}
+}
+
 /**
  * Process a completed JOB_AES_HMAC job and keep processing jobs until
  * get_completed_job return NULL
@@ -1117,6 +1311,32 @@ handle_completed_jobs(struct aesni_mb_qp *qp, JOB_AES_HMAC *job,
 	return processed_jobs;
 }
 
+static inline uint32_t
+handle_completed_sec_jobs(JOB_AES_HMAC *job, MB_MGR *mb_mgr)
+{
+	uint32_t processed = 0;
+
+	while (job != NULL) {
+		post_process_mb_sec_job(job);
+		job = IMB_GET_COMPLETED_JOB(mb_mgr);
+		processed++;
+	}
+
+	return processed;
+}
+
+static inline uint32_t
+flush_mb_sec_mgr(MB_MGR *mb_mgr)
+{
+	JOB_AES_HMAC *job = IMB_FLUSH_JOB(mb_mgr);
+	uint32_t processed = 0;
+
+	if (job)
+		processed = handle_completed_sec_jobs(job, mb_mgr);
+
+	return processed;
+}
+
 static inline uint16_t
 flush_mb_mgr(struct aesni_mb_qp *qp, struct rte_crypto_op **ops,
 		uint16_t nb_ops)
@@ -1220,6 +1440,55 @@ aesni_mb_pmd_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops,
 	return processed_jobs;
 }
 
+void
+aesni_mb_sec_crypto_process_bulk(struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num)
+{
+	struct aesni_mb_sec_session *sec_sess = sess->sess_private_data;
+	JOB_AES_HMAC *job;
+	uint8_t digest_idx = sec_sess->digest_idx;
+	uint32_t i, processed = 0;
+	int ret;
+
+	for (i = 0; i < num; i++) {
+		void *seg_buf = buf[i].vec[0].iov_base;
+		uint32_t buf_len = buf[i].vec[0].iov_len;
+
+		job = IMB_GET_NEXT_JOB(sec_sess->mb_mgr);
+		if (unlikely(job == NULL)) {
+			processed += flush_mb_sec_mgr(sec_sess->mb_mgr);
+
+			job = IMB_GET_NEXT_JOB(sec_sess->mb_mgr);
+			if (!job)
+				return;
+		}
+
+		ret = set_mb_job_params_sec(job, sec_sess, seg_buf, buf_len,
+				iv[i], aad[i], digest[i], &status[i],
+				&digest_idx);
+				/* Submit job to multi-buffer for processing */
+		if (ret) {
+			processed++;
+			status[i] = ret;
+			continue;
+		}
+
+#ifdef RTE_LIBRTE_PMD_AESNI_MB_DEBUG
+		job = IMB_SUBMIT_JOB(sec_sess->mb_mgr);
+#else
+		job = IMB_SUBMIT_JOB_NOCHECK(sec_sess->mb_mgr);
+#endif
+
+		if (job)
+			processed += handle_completed_sec_jobs(job,
+					sec_sess->mb_mgr);
+	}
+
+	while (processed < num)
+		processed += flush_mb_sec_mgr(sec_sess->mb_mgr);
+}
+
 static int cryptodev_aesni_mb_remove(struct rte_vdev_device *vdev);
 
 static int
@@ -1229,8 +1498,10 @@ cryptodev_aesni_mb_create(const char *name,
 {
 	struct rte_cryptodev *dev;
 	struct aesni_mb_private *internals;
+	struct rte_security_ctx *sec_ctx;
 	enum aesni_mb_vector_mode vector_mode;
 	MB_MGR *mb_mgr;
+	char sec_name[RTE_DEV_NAME_MAX_LEN];
 
 	/* Check CPU for support for AES instruction set */
 	if (!rte_cpu_get_flag_enabled(RTE_CPUFLAG_AES)) {
@@ -1264,7 +1535,8 @@ cryptodev_aesni_mb_create(const char *name,
 	dev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO |
 			RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
 			RTE_CRYPTODEV_FF_CPU_AESNI |
-			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT;
+			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
+			RTE_CRYPTODEV_FF_SECURITY;
 
 
 	mb_mgr = alloc_mb_mgr(0);
@@ -1303,11 +1575,28 @@ cryptodev_aesni_mb_create(const char *name,
 	AESNI_MB_LOG(INFO, "IPSec Multi-buffer library version used: %s\n",
 			imb_get_version_str());
 
+	/* setup security operations */
+	snprintf(sec_name, sizeof(sec_name) - 1, "aes_mb_sec_%u",
+			dev->driver_id);
+	sec_ctx = rte_zmalloc_socket(sec_name,
+			sizeof(struct rte_security_ctx),
+			RTE_CACHE_LINE_SIZE, init_params->socket_id);
+	if (sec_ctx == NULL) {
+		AESNI_MB_LOG(ERR, "memory allocation failed\n");
+		goto error_exit;
+	}
+
+	sec_ctx->device = (void *)dev;
+	sec_ctx->ops = rte_aesni_mb_pmd_security_ops;
+	dev->security_ctx = sec_ctx;
+
 	return 0;
 
 error_exit:
 	if (mb_mgr)
 		free_mb_mgr(mb_mgr);
+	if (sec_ctx)
+		rte_free(sec_ctx);
 
 	rte_cryptodev_pmd_destroy(dev);
 
drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c +90 -1

107 lines not on this page · raw message

drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h +18 -3
index b794d4bc1..d1cf416ab 100644
@@ -176,7 +176,6 @@ struct aesni_mb_qp {
 	 */
 } __rte_cache_aligned;
 
-/** AES-NI multi-buffer private session structure */
 struct aesni_mb_session {
 	JOB_CHAIN_ORDER chain_order;
 	struct {
@@ -265,16 +264,32 @@ struct aesni_mb_session {
 		/** AAD data length */
 		uint16_t aad_len;
 	} aead;
-} __rte_cache_aligned;
+};
+
+/** AES-NI multi-buffer private security session structure */
+struct aesni_mb_sec_session {
+	/**< Unique Queue Pair Name */
+	struct aesni_mb_session sess;
+	uint8_t temp_digests[MAX_JOBS][DIGEST_LENGTH_MAX];
+	uint16_t digest_idx;
+	uint32_t cipher_offset;
+	MB_MGR *mb_mgr;
+};
 
 extern int
 aesni_mb_set_session_parameters(const MB_MGR *mb_mgr,
 		struct aesni_mb_session *sess,
 		const struct rte_crypto_sym_xform *xform);
 
+extern void
+aesni_mb_sec_crypto_process_bulk(struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+
 /** device specific operations function pointer structure */
 extern struct rte_cryptodev_ops *rte_aesni_mb_pmd_ops;
 
-
+/** device specific operations function pointer structure for rte_security */
+extern struct rte_security_ops *rte_aesni_mb_pmd_security_ops;
 
 #endif /* _RTE_AESNI_MB_PMD_PRIVATE_H_ */
Signature · 1 line
2.14.5
#62 lore raw

Re: [dpdk-dev] [PATCH 05/10] crypto/aesni_mb: add rte_security handler

Quoted 123 lines

This patch add rte_security support support to AESNI-MB PMD. The PMD now initialize security context instance, create/delete PMD specific security sessions, and process crypto workloads in synchronous mode.

---
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c         | 291 ++++++++++++++++++++-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c     |  91 ++++++-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h |  21 +-
 3 files changed, 398 insertions(+), 5 deletions(-)
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +95 -0
index b495a9679..68767c04e 100644
@@ -8,6 +8,8 @@
 #include <rte_hexdump.h>
 #include <rte_cryptodev.h>
 #include <rte_cryptodev_pmd.h>
+#include <rte_security.h>
+#include <rte_security_driver.h>
 #include <rte_bus_vdev.h>
 #include <rte_malloc.h>
 #include <rte_cpuflags.h>
@@ -789,6 +791,167 @@ auth_start_offset(struct rte_crypto_op *op, struct aesni_mb_session *session,
 			(UINT64_MAX - u_src + u_dst + 1);
 }
 
+union sec_userdata_field {
+	int status;
+	struct {
+		uint16_t is_gen_digest;
+		uint16_t digest_len;
+	};
+};
+
+struct sec_udata_digest_field {
+	uint32_t is_digest_gen;
+	uint32_t digest_len;
+};
+
+static inline int
+set_mb_job_params_sec(JOB_AES_HMAC *job, struct aesni_mb_sec_session *sec_sess,
+		void *buf, uint32_t buf_len, void *iv, void *aad, void *digest,
+		int *status, uint8_t *digest_idx)
+{
+	struct aesni_mb_session *session = &sec_sess->sess;
+	uint32_t cipher_offset = sec_sess->cipher_offset;
+	void *user_digest = NULL;
+	union sec_userdata_field udata;
+
+	if (unlikely(cipher_offset > buf_len))
+		return -EINVAL;
+
+	/* Set crypto operation */
+	job->chain_order = session->chain_order;
+
+	/* Set cipher parameters */
+	job->cipher_direction = session->cipher.direction;
+	job->cipher_mode = session->cipher.mode;
+
+	job->aes_key_len_in_bytes = session->cipher.key_length_in_bytes;
+
+	/* Set authentication parameters */
+	job->hash_alg = session->auth.algo;
+	job->iv = iv;
+
+	switch (job->hash_alg) {
+	case AES_XCBC:
+		job->u.XCBC._k1_expanded = session->auth.xcbc.k1_expanded;
+		job->u.XCBC._k2 = session->auth.xcbc.k2;
+		job->u.XCBC._k3 = session->auth.xcbc.k3;
+
+		job->aes_enc_key_expanded =
+				session->cipher.expanded_aes_keys.encode;
+		job->aes_dec_key_expanded =
+				session->cipher.expanded_aes_keys.decode;
+		break;
+
+	case AES_CCM:
+		job->u.CCM.aad = (uint8_t *)aad + 18;
+		job->u.CCM.aad_len_in_bytes = session->aead.aad_len;
+		job->aes_enc_key_expanded =
+				session->cipher.expanded_aes_keys.encode;
+		job->aes_dec_key_expanded =
+				session->cipher.expanded_aes_keys.decode;
+		job->iv++;
+		break;
+
+	case AES_CMAC:
+		job->u.CMAC._key_expanded = session->auth.cmac.expkey;
+		job->u.CMAC._skey1 = session->auth.cmac.skey1;
+		job->u.CMAC._skey2 = session->auth.cmac.skey2;
+		job->aes_enc_key_expanded =
+				session->cipher.expanded_aes_keys.encode;
+		job->aes_dec_key_expanded =
+				session->cipher.expanded_aes_keys.decode;
+		break;
+
+	case AES_GMAC:
+		if (session->cipher.mode == GCM) {
+			job->u.GCM.aad = aad;
+			job->u.GCM.aad_len_in_bytes = session->aead.aad_len;
+		} else {
+			/* For GMAC */
+			job->u.GCM.aad = aad;
+			job->u.GCM.aad_len_in_bytes = buf_len;
+			job->cipher_mode = GCM;
+		}
+		job->aes_enc_key_expanded = &session->cipher.gcm_key;
+		job->aes_dec_key_expanded = &session->cipher.gcm_key;
+		break;
+
+	default:
+		job->u.HMAC._hashed_auth_key_xor_ipad =
+				session->auth.pads.inner;
+		job->u.HMAC._hashed_auth_key_xor_opad =
+				session->auth.pads.outer;
+
+		if (job->cipher_mode == DES3) {
+			job->aes_enc_key_expanded =

drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c auth_start_offset(struct rte_crypto_op *op, struct aesni_mb_session *session, · line 896

Quoted 10 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +10 -0
+				session->cipher.exp_3des_keys.ks_ptr;
+			job->aes_dec_key_expanded =
+				session->cipher.exp_3des_keys.ks_ptr;
+		} else {
+			job->aes_enc_key_expanded =
+				session->cipher.expanded_aes_keys.encode;
+			job->aes_dec_key_expanded =
+				session->cipher.expanded_aes_keys.decode;
+		}
+	}

Seems like too many branches at data-path.
We'll have only one job-type(alg) per session.
So we can have prefilled job struct template with all common fields already setuped, and then at process() just copy it over and update few fields that has to be different (like msg_len_to_cipher_in_bytes).

Quoted 78 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +71 -0
+
+	/* Set digest output location */
+	if (job->hash_alg != NULL_HASH &&
+			session->auth.operation == RTE_CRYPTO_AUTH_OP_VERIFY) {
+		job->auth_tag_output = sec_sess->temp_digests[*digest_idx];
+		*digest_idx = (*digest_idx + 1) % MAX_JOBS;
+
+		udata.is_gen_digest = 0;
+		udata.digest_len = session->auth.req_digest_len;
+		user_digest = (void *)digest;
+	} else {
+		udata.is_gen_digest = 1;
+		udata.digest_len = session->auth.req_digest_len;
+
+		if (session->auth.req_digest_len !=
+				session->auth.gen_digest_len) {
+			job->auth_tag_output =
+					sec_sess->temp_digests[*digest_idx];
+			*digest_idx = (*digest_idx + 1) % MAX_JOBS;
+
+			user_digest = (void *)digest;
+		} else
+			job->auth_tag_output = digest;
+
+		/* A bit of hack here, since job structure only supports
+		 * 2 user data fields and we need 4 params to be passed
+		 * (status, direction, digest for verify, and length of
+		 * digest), we set the status value as digest length +
+		 * direction here temporarily to avoid creating longer
+		 * buffer to store all 4 params.
+		 */
+		*status = udata.status;
+	}
+	/*
+	 * Multi-buffer library current only support returning a truncated
+	 * digest length as specified in the relevant IPsec RFCs
+	 */
+
+	/* Set digest length */
+	job->auth_tag_output_len_in_bytes = session->auth.gen_digest_len;
+
+	/* Set IV parameters */
+	job->iv_len_in_bytes = session->iv.length;
+
+	/* Data Parameters */
+	job->src = buf;
+	job->dst = buf;
+	job->cipher_start_src_offset_in_bytes = cipher_offset;
+	job->msg_len_to_cipher_in_bytes = buf_len - cipher_offset;
+	job->hash_start_src_offset_in_bytes = 0;
+	job->msg_len_to_hash_in_bytes = buf_len;
+
+	job->user_data = (void *)status;
+	job->user_data2 = user_digest;
+
+	return 0;
+}
+
 /**
  * Process a crypto operation and complete a JOB_AES_HMAC job structure for
  * submission to the multi buffer library for processing.
@@ -1081,6 +1244,37 @@ post_process_mb_job(struct aesni_mb_qp *qp, JOB_AES_HMAC *job)
 	return op;
 }
 
+static inline void
+post_process_mb_sec_job(JOB_AES_HMAC *job)
+{
+	void *user_digest = job->user_data2;
+	int *status = job->user_data;
+	union sec_userdata_field udata;
+
+	switch (job->status) {
+	case STS_COMPLETED:
+		if (user_digest) {
+			udata.status = *status;
+
+			if (udata.is_gen_digest) {

drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c post_process_mb_job(struct aesni_mb_qp *qp, JOB_AES_HMAC *job) · line 1269

Quoted 10 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +10 -0
+				*status = RTE_CRYPTO_OP_STATUS_SUCCESS;
+				memcpy(user_digest, job->auth_tag_output,
+						udata.digest_len);
+			} else {
+				verify_digest(job, user_digest,
+					udata.digest_len, (uint8_t *)status);
+
+				if (*status == RTE_CRYPTO_OP_STATUS_AUTH_FAILED)
+					*status = -1;
+			}

Again - multiple process() functions instead of branches at data-path?

Quoted 60 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +46 -0
+		} else
+			*status = RTE_CRYPTO_OP_STATUS_SUCCESS;
+		break;
+	default:
+		*status = RTE_CRYPTO_OP_STATUS_ERROR;
+	}
+}
+
 /**
  * Process a completed JOB_AES_HMAC job and keep processing jobs until
  * get_completed_job return NULL
@@ -1117,6 +1311,32 @@ handle_completed_jobs(struct aesni_mb_qp *qp, JOB_AES_HMAC *job,
 	return processed_jobs;
 }
 
+static inline uint32_t
+handle_completed_sec_jobs(JOB_AES_HMAC *job, MB_MGR *mb_mgr)
+{
+	uint32_t processed = 0;
+
+	while (job != NULL) {
+		post_process_mb_sec_job(job);
+		job = IMB_GET_COMPLETED_JOB(mb_mgr);
+		processed++;
+	}
+
+	return processed;
+}
+
+static inline uint32_t
+flush_mb_sec_mgr(MB_MGR *mb_mgr)
+{
+	JOB_AES_HMAC *job = IMB_FLUSH_JOB(mb_mgr);
+	uint32_t processed = 0;
+
+	if (job)
+		processed = handle_completed_sec_jobs(job, mb_mgr);
+
+	return processed;
+}
+
 static inline uint16_t
 flush_mb_mgr(struct aesni_mb_qp *qp, struct rte_crypto_op **ops,
 		uint16_t nb_ops)
@@ -1220,6 +1440,55 @@ aesni_mb_pmd_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops,
 	return processed_jobs;
 }
 
+void
+aesni_mb_sec_crypto_process_bulk(struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num)
+{
+	struct aesni_mb_sec_session *sec_sess = sess->sess_private_data;
+	JOB_AES_HMAC *job;
+	uint8_t digest_idx = sec_sess->digest_idx;
+	uint32_t i, processed = 0;
+	int ret;
+
+	for (i = 0; i < num; i++) {

drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c aesni_mb_pmd_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops, · line 1464

Quoted 10 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +10 -0
+		void *seg_buf = buf[i].vec[0].iov_base;
+		uint32_t buf_len = buf[i].vec[0].iov_len;
+
+		job = IMB_GET_NEXT_JOB(sec_sess->mb_mgr);
+		if (unlikely(job == NULL)) {
+			processed += flush_mb_sec_mgr(sec_sess->mb_mgr);
+
+			job = IMB_GET_NEXT_JOB(sec_sess->mb_mgr);
+			if (!job)
+				return;

You can't just return here.
Need to fill remaining statsu[] with some meaningfull error value.
As alternative make proceee_bulk() to return number of processed buffers instead of void.

drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c aesni_mb_pmd_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops, · line 1469

Quoted 5 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +5 -0
+		}
+
+		ret = set_mb_job_params_sec(job, sec_sess, seg_buf, buf_len,
+				iv[i], aad[i], digest[i], &status[i],
+				&digest_idx);

That doesn't look right:
digest_idx is a temporary valiable, you pass it's address to set_mb_job_params_sec(), where it will be updated, but then you never write you back.
So do we really need digest_idx inside the session?
Overall, the whole construction with having status and idx stored inside job struct seems overcomplicated and probably error prone.
AFAIK, aesni-mb job-manager guarantees FIFO order jobs submitted.
So just having idx counter inside that function seems enough, no?

Quoted 116 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +43 -1
+				/* Submit job to multi-buffer for processing */
+		if (ret) {
+			processed++;
+			status[i] = ret;
+			continue;
+		}
+
+#ifdef RTE_LIBRTE_PMD_AESNI_MB_DEBUG
+		job = IMB_SUBMIT_JOB(sec_sess->mb_mgr);
+#else
+		job = IMB_SUBMIT_JOB_NOCHECK(sec_sess->mb_mgr);
+#endif
+
+		if (job)
+			processed += handle_completed_sec_jobs(job,
+					sec_sess->mb_mgr);
+	}
+
+	while (processed < num)
+		processed += flush_mb_sec_mgr(sec_sess->mb_mgr);
+}
+
 static int cryptodev_aesni_mb_remove(struct rte_vdev_device *vdev);
 
 static int
@@ -1229,8 +1498,10 @@ cryptodev_aesni_mb_create(const char *name,
 {
 	struct rte_cryptodev *dev;
 	struct aesni_mb_private *internals;
+	struct rte_security_ctx *sec_ctx;
 	enum aesni_mb_vector_mode vector_mode;
 	MB_MGR *mb_mgr;
+	char sec_name[RTE_DEV_NAME_MAX_LEN];
 
 	/* Check CPU for support for AES instruction set */
 	if (!rte_cpu_get_flag_enabled(RTE_CPUFLAG_AES)) {
@@ -1264,7 +1535,8 @@ cryptodev_aesni_mb_create(const char *name,
 	dev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO |
 			RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
 			RTE_CRYPTODEV_FF_CPU_AESNI |
-			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT;
+			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
+			RTE_CRYPTODEV_FF_SECURITY;
 
 
 	mb_mgr = alloc_mb_mgr(0);
@@ -1303,11 +1575,28 @@ cryptodev_aesni_mb_create(const char *name,
 	AESNI_MB_LOG(INFO, "IPSec Multi-buffer library version used: %s\n",
 			imb_get_version_str());
 
+	/* setup security operations */
+	snprintf(sec_name, sizeof(sec_name) - 1, "aes_mb_sec_%u",
+			dev->driver_id);
+	sec_ctx = rte_zmalloc_socket(sec_name,
+			sizeof(struct rte_security_ctx),
+			RTE_CACHE_LINE_SIZE, init_params->socket_id);
+	if (sec_ctx == NULL) {
+		AESNI_MB_LOG(ERR, "memory allocation failed\n");
+		goto error_exit;
+	}
+
+	sec_ctx->device = (void *)dev;
+	sec_ctx->ops = rte_aesni_mb_pmd_security_ops;
+	dev->security_ctx = sec_ctx;
+
 	return 0;
 
 error_exit:
 	if (mb_mgr)
 		free_mb_mgr(mb_mgr);
+	if (sec_ctx)
+		rte_free(sec_ctx);
 
 	rte_cryptodev_pmd_destroy(dev);
 
drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c +18 -1
index 8d15b99d4..ca6cea775 100644
@@ -8,6 +8,7 @@
 #include <rte_common.h>
 #include <rte_malloc.h>
 #include <rte_cryptodev_pmd.h>
+#include <rte_security_driver.h>
 
 #include "rte_aesni_mb_pmd_private.h"
 
@@ -732,7 +733,8 @@ aesni_mb_pmd_qp_count(struct rte_cryptodev *dev)
 static unsigned
 aesni_mb_pmd_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)
 {
-	return sizeof(struct aesni_mb_session);
+	return RTE_ALIGN_CEIL(sizeof(struct aesni_mb_session),
+			RTE_CACHE_LINE_SIZE);
 }
 
 /** Configure a aesni multi-buffer session from a crypto xform chain */
@@ -810,4 +812,91 @@ struct rte_cryptodev_ops aesni_mb_pmd_ops = {
 		.sym_session_clear	= aesni_mb_pmd_sym_session_clear
 };
 
+/** Set session authentication parameters */
+
+static int
+aesni_mb_security_session_create(void *dev,
+		struct rte_security_session_conf *conf,
+		struct rte_security_session *sess,
+		struct rte_mempool *mempool)
+{
+	struct rte_cryptodev *cdev = dev;
+	struct aesni_mb_private *internals = cdev->data->dev_private;
+	struct aesni_mb_sec_session *sess_priv;
+	int ret;
+
+	if (!conf->crypto_xform) {
+		AESNI_MB_LOG(ERR, "Invalid security session conf");

drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c struct rte_cryptodev_ops aesni_mb_pmd_ops = { · line 839

Quoted 10 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c +10 -0
+		return -EINVAL;
+	}
+
+	if (rte_mempool_get(mempool, (void **)(&sess_priv))) {
+		AESNI_MB_LOG(ERR,
+				"Couldn't get object from session mempool");
+		return -ENOMEM;
+	}
+
+	sess_priv->mb_mgr = internals->mb_mgr;

After another thoughts - I don't think it is ok to use the same job-manager across all sessions. Different sessions can be used by different threads, etc. I think we need a separate instance of job-manager for every session.

Quoted 76 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c +62 -0
+	if (sess_priv->mb_mgr == NULL)
+		return -ENOMEM;
+
+	sess_priv->cipher_offset = conf->cpucrypto.cipher_offset;
+
+	ret = aesni_mb_set_session_parameters(sess_priv->mb_mgr,
+			&sess_priv->sess, conf->crypto_xform);
+	if (ret != 0) {
+		AESNI_MB_LOG(ERR, "failed configure session parameters");
+
+		rte_mempool_put(mempool, sess_priv);
+	}
+
+	sess->sess_private_data = (void *)sess_priv;
+
+	return ret;
+}
+
+static int
+aesni_mb_security_session_destroy(void *dev __rte_unused,
+		struct rte_security_session *sess)
+{
+	struct aesni_mb_sec_session *sess_priv =
+			get_sec_session_private_data(sess);
+
+	if (sess_priv) {
+		struct rte_mempool *sess_mp = rte_mempool_from_obj(
+				(void *)sess_priv);
+
+		memset(sess, 0, sizeof(struct aesni_mb_sec_session));
+		set_sec_session_private_data(sess, NULL);
+
+		if (sess_mp == NULL) {
+			AESNI_MB_LOG(ERR, "failed fetch session mempool");
+			return -EINVAL;
+		}
+
+		rte_mempool_put(sess_mp, sess_priv);
+	}
+
+	return 0;
+}
+
+static unsigned int
+aesni_mb_sec_session_get_size(__rte_unused void *device)
+{
+	return RTE_ALIGN_CEIL(sizeof(struct aesni_mb_sec_session),
+			RTE_CACHE_LINE_SIZE);
+}
+
+static struct rte_security_ops aesni_mb_security_ops = {
+		.session_create = aesni_mb_security_session_create,
+		.session_get_size = aesni_mb_sec_session_get_size,
+		.session_update = NULL,
+		.session_stats_get = NULL,
+		.session_destroy = aesni_mb_security_session_destroy,
+		.set_pkt_metadata = NULL,
+		.capabilities_get = NULL,
+		.process_cpu_crypto_bulk = aesni_mb_sec_crypto_process_bulk,
+};
+
 struct rte_cryptodev_ops *rte_aesni_mb_pmd_ops = &aesni_mb_pmd_ops;
+struct rte_security_ops *rte_aesni_mb_pmd_security_ops = &aesni_mb_security_ops;
drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h +0 -1
index b794d4bc1..d1cf416ab 100644
@@ -176,7 +176,6 @@ struct aesni_mb_qp {
 	 */
 } __rte_cache_aligned;
 
-/** AES-NI multi-buffer private session structure */
 struct aesni_mb_session {
 	JOB_CHAIN_ORDER chain_order;
 	struct {

drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h struct aesni_mb_session { · line 268

Quoted 4 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h +0 -1
@@ -265,16 +264,32 @@ struct aesni_mb_session {
 		/** AAD data length */
 		uint16_t aad_len;
 	} aead;
-} __rte_cache_aligned;

Didn't look through all the code

drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h struct aesni_mb_session { · line 273

Quoted 7 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h +7 -0
+};
+
+/** AES-NI multi-buffer private security session structure */
+struct aesni_mb_sec_session {
+	/**< Unique Queue Pair Name */
+	struct aesni_mb_session sess;
+	uint8_t temp_digests[MAX_JOBS][DIGEST_LENGTH_MAX];

Probably better to move these temp_digest[][] at the very end? To have all read-only data grouped together?

Quoted 25 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h +11 -2
+	uint16_t digest_idx;
+	uint32_t cipher_offset;
+	MB_MGR *mb_mgr;
+};
 
 extern int
 aesni_mb_set_session_parameters(const MB_MGR *mb_mgr,
 		struct aesni_mb_session *sess,
 		const struct rte_crypto_sym_xform *xform);
 
+extern void
+aesni_mb_sec_crypto_process_bulk(struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+
 /** device specific operations function pointer structure */
 extern struct rte_cryptodev_ops *rte_aesni_mb_pmd_ops;
 
-
+/** device specific operations function pointer structure for rte_security */
+extern struct rte_security_ops *rte_aesni_mb_pmd_security_ops;
 
 #endif /* _RTE_AESNI_MB_PMD_PRIVATE_H_ */
--

2.14.5

author
#63 lore raw

PATCH 6/10 app/test: add aesni_mb security cpu crypto autotest

In reply to Fan Zhang

This patch adds cpu crypto unit test for AESNI_MB PMD.

---
 app/test/test_security_cpu_crypto.c | 367 ++++++++++++++++++++++++++++++++++++
 1 file changed, 367 insertions(+)
app/test/test_security_cpu_crypto.c +367 -0
index ca9a8dae6..0ea406390 100644
@@ -19,12 +19,23 @@
 
 #include "test.h"
 #include "test_cryptodev.h"
+#include "test_cryptodev_blockcipher.h"
+#include "test_cryptodev_aes_test_vectors.h"
 #include "test_cryptodev_aead_test_vectors.h"
+#include "test_cryptodev_des_test_vectors.h"
+#include "test_cryptodev_hash_test_vectors.h"
 
 #define CPU_CRYPTO_TEST_MAX_AAD_LENGTH	16
 #define MAX_NB_SIGMENTS			4
 #define CACHE_WARM_ITER			2048
 
+#define TOP_ENC		BLOCKCIPHER_TEST_OP_ENCRYPT
+#define TOP_DEC		BLOCKCIPHER_TEST_OP_DECRYPT
+#define TOP_AUTH_GEN	BLOCKCIPHER_TEST_OP_AUTH_GEN
+#define TOP_AUTH_VER	BLOCKCIPHER_TEST_OP_AUTH_VERIFY
+#define TOP_ENC_AUTH	BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN
+#define TOP_AUTH_DEC	BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC
+
 enum buffer_assemble_option {
 	SGL_MAX_SEG,
 	SGL_ONE_SEG,
@@ -516,6 +527,11 @@ cpu_crypto_test_aead(const struct aead_test_data *tdata,
 	TEST_EXPAND(gcm_test_case_256_6, type)	\
 	TEST_EXPAND(gcm_test_case_256_7, type)
 
+/* test-vector/sgl-option */
+#define all_ccm_unit_test_cases \
+	TEST_EXPAND(ccm_test_case_128_1, SGL_ONE_SEG) \
+	TEST_EXPAND(ccm_test_case_128_2, SGL_ONE_SEG) \
+	TEST_EXPAND(ccm_test_case_128_3, SGL_ONE_SEG)
 
 #define TEST_EXPAND(t, o)						\
 static int								\
@@ -531,6 +547,7 @@ cpu_crypto_aead_dec_test_##t##_##o(void)				\
 
 all_gcm_unit_test_cases(SGL_ONE_SEG)
 all_gcm_unit_test_cases(SGL_MAX_SEG)
+all_ccm_unit_test_cases
 #undef TEST_EXPAND
 
 static struct unit_test_suite security_cpu_crypto_aesgcm_testsuite  = {
@@ -758,8 +775,358 @@ test_security_cpu_crypto_aesni_gcm_perf(void)
 			&security_cpu_crypto_aesgcm_perf_testsuite);
 }
 
+static struct rte_security_session *
+create_blockcipher_session(struct rte_security_ctx *ctx,
+		struct rte_mempool *sess_mp,
+		uint32_t op_mask,
+		const struct blockcipher_test_data *test_data,
+		uint32_t is_unit_test)
+{
+	struct rte_security_session_conf sess_conf = {0};
+	struct rte_crypto_sym_xform xforms[2] = { {0} };
+	struct rte_crypto_sym_xform *cipher_xform = NULL;
+	struct rte_crypto_sym_xform *auth_xform = NULL;
+	struct rte_crypto_sym_xform *xform;
+
+	if (op_mask & BLOCKCIPHER_TEST_OP_CIPHER) {
+		cipher_xform = &xforms[0];
+		cipher_xform->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
+
+		if (op_mask & TOP_ENC)
+			cipher_xform->cipher.op =
+				RTE_CRYPTO_CIPHER_OP_ENCRYPT;
+		else
+			cipher_xform->cipher.op =
+				RTE_CRYPTO_CIPHER_OP_DECRYPT;
+
+		cipher_xform->cipher.algo = test_data->crypto_algo;
+		cipher_xform->cipher.key.data = test_data->cipher_key.data;
+		cipher_xform->cipher.key.length = test_data->cipher_key.len;
+		cipher_xform->cipher.iv.offset = 0;
+		cipher_xform->cipher.iv.length = test_data->iv.len;
+
+		if (is_unit_test)
+			debug_hexdump(stdout, "cipher key:",
+					test_data->cipher_key.data,
+					test_data->cipher_key.len);
+	}
+
+	if (op_mask & BLOCKCIPHER_TEST_OP_AUTH) {
+		auth_xform = &xforms[1];
+		auth_xform->type = RTE_CRYPTO_SYM_XFORM_AUTH;
+
+		if (op_mask & TOP_AUTH_GEN)
+			auth_xform->auth.op = RTE_CRYPTO_AUTH_OP_GENERATE;
+		else
+			auth_xform->auth.op = RTE_CRYPTO_AUTH_OP_VERIFY;
+
+		auth_xform->auth.algo = test_data->auth_algo;
+		auth_xform->auth.key.length = test_data->auth_key.len;
+		auth_xform->auth.key.data = test_data->auth_key.data;
+		auth_xform->auth.digest_length = test_data->digest.len;
+
+		if (is_unit_test)
+			debug_hexdump(stdout, "auth key:",
+					test_data->auth_key.data,
+					test_data->auth_key.len);
+	}
+
+	if (op_mask == TOP_ENC ||
+			op_mask == TOP_DEC)
+		xform = cipher_xform;
+	else if (op_mask == TOP_AUTH_GEN ||
+			op_mask == TOP_AUTH_VER)
+		xform = auth_xform;
+	else if (op_mask == TOP_ENC_AUTH) {
+		xform = cipher_xform;
+		xform->next = auth_xform;
+	} else if (op_mask == TOP_AUTH_DEC) {
+		xform = auth_xform;
+		xform->next = cipher_xform;
+	} else
+		return NULL;
+
+	if (test_data->cipher_offset < test_data->auth_offset)
+		return NULL;
+
+	sess_conf.action_type = RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO;
+	sess_conf.crypto_xform = xform;
+	sess_conf.cpucrypto.cipher_offset = test_data->cipher_offset -
+			test_data->auth_offset;
+
+	return rte_security_session_create(ctx, &sess_conf, sess_mp);
+}
+
+static inline int
+assemble_blockcipher_buf(struct cpu_crypto_test_case *data,
+		struct cpu_crypto_test_obj *obj,
+		uint32_t obj_idx,
+		uint32_t op_mask,
+		const struct blockcipher_test_data *test_data,
+		uint32_t is_unit_test)
+{
+	const uint8_t *src;
+	uint32_t src_len;
+	uint32_t offset;
+
+	if (op_mask == TOP_ENC_AUTH ||
+			op_mask == TOP_AUTH_GEN ||
+			op_mask == BLOCKCIPHER_TEST_OP_AUTH_VERIFY)
+		offset = test_data->auth_offset;
+	else
+		offset = test_data->cipher_offset;
+
+	if (op_mask & TOP_ENC_AUTH) {
+		src = test_data->plaintext.data;
+		src_len = test_data->plaintext.len;
+		if (is_unit_test)
+			debug_hexdump(stdout, "plaintext:", src, src_len);
+	} else {
+		src = test_data->ciphertext.data;
+		src_len = test_data->ciphertext.len;
+		memcpy(data->digest, test_data->digest.data,
+				test_data->digest.len);
+		if (is_unit_test) {
+			debug_hexdump(stdout, "ciphertext:", src, src_len);
+			debug_hexdump(stdout, "digest:", test_data->digest.data,
+					test_data->digest.len);
+		}
+	}
+
+	if (src_len > MBUF_DATAPAYLOAD_SIZE)
+		return -ENOMEM;
+
+	memcpy(data->seg_buf[0].seg, src, src_len);
+	data->seg_buf[0].seg_len = src_len;
+	obj->vec[obj_idx][0].iov_base =
+			(void *)(data->seg_buf[0].seg + offset);
+	obj->vec[obj_idx][0].iov_len = src_len - offset;
+
+	obj->sec_buf[obj_idx].vec = obj->vec[obj_idx];
+	obj->sec_buf[obj_idx].num = 1;
+
+	memcpy(data->iv, test_data->iv.data, test_data->iv.len);
+	if (is_unit_test)
+		debug_hexdump(stdout, "iv:", test_data->iv.data,
+				test_data->iv.len);
+
+	obj->iv[obj_idx] = (void *)data->iv;
+	obj->digest[obj_idx] = (void *)data->digest;
+
+	return 0;
+}
+
+static int
+check_blockcipher_result(struct cpu_crypto_test_case *tcase,
+		uint32_t op_mask,
+		const struct blockcipher_test_data *test_data)
+{
+	int ret;
+
+	if (op_mask & BLOCKCIPHER_TEST_OP_CIPHER) {
+		const char *err_msg1, *err_msg2;
+		const uint8_t *src_pt_ct;
+		uint32_t src_len;
+
+		if (op_mask & TOP_ENC) {
+			src_pt_ct = test_data->ciphertext.data;
+			src_len = test_data->ciphertext.len;
+			err_msg1 = CPU_CRYPTO_ERR_EXP_CT;
+			err_msg2 = CPU_CRYPTO_ERR_GEN_CT;
+		} else {
+			src_pt_ct = test_data->plaintext.data;
+			src_len = test_data->plaintext.len;
+			err_msg1 = CPU_CRYPTO_ERR_EXP_PT;
+			err_msg2 = CPU_CRYPTO_ERR_GEN_PT;
+		}
+
+		ret = memcmp(tcase->seg_buf[0].seg, src_pt_ct, src_len);
+		if (ret != 0) {
+			debug_hexdump(stdout, err_msg1, src_pt_ct, src_len);
+			debug_hexdump(stdout, err_msg2,
+					tcase->seg_buf[0].seg,
+					test_data->ciphertext.len);
+			return -1;
+		}
+	}
+
+	if (op_mask & TOP_AUTH_GEN) {
+		ret = memcmp(tcase->digest, test_data->digest.data,
+				test_data->digest.len);
+		if (ret != 0) {
+			debug_hexdump(stdout, "expect digest:",
+					test_data->digest.data,
+					test_data->digest.len);
+			debug_hexdump(stdout, "gen digest:",
+					tcase->digest,
+					test_data->digest.len);
+			return -1;
+		}
+	}
+
+	return 0;
+}
+
+static int
+cpu_crypto_test_blockcipher(const struct blockcipher_test_data *tdata,
+		uint32_t op_mask)
+{
+	struct cpu_crypto_testsuite_params *ts_params = &testsuite_params;
+	struct cpu_crypto_unittest_params *ut_params = &unittest_params;
+	struct cpu_crypto_test_obj *obj = &ut_params->test_obj;
+	struct cpu_crypto_test_case *tcase;
+	int ret;
+
+	ut_params->sess = create_blockcipher_session(ts_params->ctx,
+			ts_params->session_priv_mpool,
+			op_mask,
+			tdata,
+			1);
+	if (!ut_params->sess)
+		return -1;
+
+	ret = allocate_buf(1);
+	if (ret)
+		return ret;
+
+	tcase = ut_params->test_datas[0];
+	ret = assemble_blockcipher_buf(tcase, obj, 0, op_mask, tdata, 1);
+	if (ret < 0) {
+		printf("Test is not supported by the driver\n");
+		return ret;
+	}
+
+	run_test(ts_params->ctx, ut_params->sess, obj, 1);
+
+	ret = check_status(obj, 1);
+	if (ret < 0)
+		return ret;
+
+	ret = check_blockcipher_result(tcase, op_mask, tdata);
+	if (ret < 0)
+		return ret;
+
+	return 0;
+}
+
+/* Macro to save code for defining BlockCipher test cases */
+/* test-vector-name/op */
+#define all_blockcipher_test_cases \
+	TEST_EXPAND(aes_test_data_1, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_1, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_1, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_1, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_2, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_2, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_2, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_2, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_3, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_3, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_3, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_3, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_4, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_4, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_4, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_4, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_5, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_5, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_5, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_5, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_6, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_6, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_6, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_6, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_7, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_7, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_7, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_7, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_8, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_8, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_8, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_8, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_9, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_9, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_9, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_9, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_10, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_10, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_11, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_11, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_12, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_12, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_12, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_12, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_13, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_13, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_13, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_13, TOP_AUTH_DEC) \
+	TEST_EXPAND(des_test_data_1, TOP_ENC) \
+	TEST_EXPAND(des_test_data_1, TOP_DEC) \
+	TEST_EXPAND(des_test_data_2, TOP_ENC) \
+	TEST_EXPAND(des_test_data_2, TOP_DEC) \
+	TEST_EXPAND(des_test_data_3, TOP_ENC) \
+	TEST_EXPAND(des_test_data_3, TOP_DEC) \
+	TEST_EXPAND(triple_des128cbc_hmac_sha1_test_vector, TOP_ENC) \
+	TEST_EXPAND(triple_des128cbc_hmac_sha1_test_vector, TOP_DEC) \
+	TEST_EXPAND(triple_des128cbc_hmac_sha1_test_vector, TOP_ENC_AUTH) \
+	TEST_EXPAND(triple_des128cbc_hmac_sha1_test_vector, TOP_AUTH_DEC) \
+	TEST_EXPAND(triple_des64cbc_test_vector, TOP_ENC) \
+	TEST_EXPAND(triple_des64cbc_test_vector, TOP_DEC) \
+	TEST_EXPAND(triple_des128cbc_test_vector, TOP_ENC) \
+	TEST_EXPAND(triple_des128cbc_test_vector, TOP_DEC) \
+	TEST_EXPAND(triple_des192cbc_test_vector, TOP_ENC) \
+	TEST_EXPAND(triple_des192cbc_test_vector, TOP_DEC) \
+
+#define TEST_EXPAND(t, o)						\
+static int								\
+cpu_crypto_blockcipher_test_##t##_##o(void)				\
+{									\
+	return cpu_crypto_test_blockcipher(&t, o);			\
+}
+
+all_blockcipher_test_cases
+#undef TEST_EXPAND
+
+static struct unit_test_suite security_cpu_crypto_aesni_mb_testsuite  = {
+	.suite_name = "Security CPU Crypto AESNI-MB Unit Test Suite",
+	.setup = testsuite_setup,
+	.teardown = testsuite_teardown,
+	.unit_test_cases = {
+#define TEST_EXPAND(t, o)						\
+	TEST_CASE_ST(ut_setup, ut_teardown,				\
+			cpu_crypto_aead_enc_test_##t##_##o),		\
+	TEST_CASE_ST(ut_setup, ut_teardown,				\
+			cpu_crypto_aead_dec_test_##t##_##o),		\
+
+	all_gcm_unit_test_cases(SGL_ONE_SEG)
+	all_ccm_unit_test_cases
+#undef TEST_EXPAND
+
+#define TEST_EXPAND(t, o)						\
+	TEST_CASE_ST(ut_setup, ut_teardown,				\
+			cpu_crypto_blockcipher_test_##t##_##o),		\
+
+	all_blockcipher_test_cases
+#undef TEST_EXPAND
+
+	TEST_CASES_END() /**< NULL terminate unit test array */
+	},
+};
+
+static int
+test_security_cpu_crypto_aesni_mb(void)
+{
+	gbl_driver_id =	rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
+
+	return unit_test_suite_runner(&security_cpu_crypto_aesni_mb_testsuite);
+}
+
 REGISTER_TEST_COMMAND(security_aesni_gcm_autotest,
 		test_security_cpu_crypto_aesni_gcm);
 
 REGISTER_TEST_COMMAND(security_aesni_gcm_perftest,
 		test_security_cpu_crypto_aesni_gcm_perf);
+
+REGISTER_TEST_COMMAND(security_aesni_mb_autotest,
+		test_security_cpu_crypto_aesni_mb);
Signature · 1 line
2.14.5
author
#64 lore raw

PATCH 7/10 app/test: add aesni_mb security cpu crypto perftest

In reply to Fan Zhang

Since crypto perf application does not support rte_security, this patch adds a simple AES-CBC-SHA1-HMAC CPU crypto performance test to crypto unittest application. The test includes different key and data sizes test with single buffer test items and will display the throughput as well as cycle count performance information.

---
 app/test/test_security_cpu_crypto.c | 194 ++++++++++++++++++++++++++++++++++++
 1 file changed, 194 insertions(+)
app/test/test_security_cpu_crypto.c +194 -0
index 0ea406390..6e012672e 100644
@@ -1122,6 +1122,197 @@ test_security_cpu_crypto_aesni_mb(void)
 	return unit_test_suite_runner(&security_cpu_crypto_aesni_mb_testsuite);
 }
 
+static inline void
+switch_blockcipher_enc_to_dec(struct blockcipher_test_data *tdata,
+		struct cpu_crypto_test_case *tcase, uint8_t *dst)
+{
+	memcpy(dst, tcase->seg_buf[0].seg, tcase->seg_buf[0].seg_len);
+	tdata->ciphertext.len = tcase->seg_buf[0].seg_len;
+	memcpy(tdata->digest.data, tcase->digest, tdata->digest.len);
+}
+
+static int
+cpu_crypto_test_blockcipher_perf(
+		const enum rte_crypto_cipher_algorithm cipher_algo,
+		uint32_t cipher_key_sz,
+		const enum rte_crypto_auth_algorithm auth_algo,
+		uint32_t auth_key_sz, uint32_t digest_sz,
+		uint32_t op_mask)
+{
+	struct blockcipher_test_data tdata = {0};
+	uint8_t plaintext[3000], ciphertext[3000];
+	struct cpu_crypto_testsuite_params *ts_params = &testsuite_params;
+	struct cpu_crypto_unittest_params *ut_params = &unittest_params;
+	struct cpu_crypto_test_obj *obj = &ut_params->test_obj;
+	struct cpu_crypto_test_case *tcase;
+	uint64_t hz = rte_get_tsc_hz(), time_start, time_now;
+	double rate, cycles_per_buf;
+	uint32_t test_data_szs[] = {64, 128, 256, 512, 1024, 2048};
+	uint32_t i, j;
+	uint32_t op_mask_opp = 0;
+	int ret;
+
+	if (op_mask & BLOCKCIPHER_TEST_OP_CIPHER)
+		op_mask_opp |= (~op_mask & BLOCKCIPHER_TEST_OP_CIPHER);
+	if (op_mask & BLOCKCIPHER_TEST_OP_AUTH)
+		op_mask_opp |= (~op_mask & BLOCKCIPHER_TEST_OP_AUTH);
+
+	tdata.plaintext.data = plaintext;
+	tdata.ciphertext.data = ciphertext;
+
+	tdata.cipher_key.len = cipher_key_sz;
+	tdata.auth_key.len = auth_key_sz;
+
+	gen_rand(tdata.cipher_key.data, cipher_key_sz / 8);
+	gen_rand(tdata.auth_key.data, auth_key_sz / 8);
+
+	tdata.crypto_algo = cipher_algo;
+	tdata.auth_algo = auth_algo;
+
+	tdata.digest.len = digest_sz;
+
+	ut_params->sess = create_blockcipher_session(ts_params->ctx,
+			ts_params->session_priv_mpool,
+			op_mask,
+			&tdata,
+			0);
+	if (!ut_params->sess)
+		return -1;
+
+	ret = allocate_buf(MAX_NUM_OPS_INFLIGHT);
+	if (ret)
+		return ret;
+
+	for (i = 0; i < RTE_DIM(test_data_szs); i++) {
+		for (j = 0; j < MAX_NUM_OPS_INFLIGHT; j++) {
+			tdata.plaintext.len = test_data_szs[i];
+			gen_rand(plaintext, tdata.plaintext.len);
+
+			tdata.iv.len = 16;
+			gen_rand(tdata.iv.data, tdata.iv.len);
+
+			tcase = ut_params->test_datas[j];
+			ret = assemble_blockcipher_buf(tcase, obj, j,
+					op_mask,
+					&tdata,
+					0);
+			if (ret < 0) {
+				printf("Test is not supported by the driver\n");
+				return ret;
+			}
+		}
+
+		/* warm up cache */
+		for (j = 0; j < CACHE_WARM_ITER; j++)
+			run_test(ts_params->ctx, ut_params->sess, obj,
+					MAX_NUM_OPS_INFLIGHT);
+
+		time_start = rte_rdtsc();
+
+		run_test(ts_params->ctx, ut_params->sess, obj,
+				MAX_NUM_OPS_INFLIGHT);
+
+		time_now = rte_rdtsc();
+
+		rate = time_now - time_start;
+		cycles_per_buf = rate / MAX_NUM_OPS_INFLIGHT;
+
+		rate = ((hz / cycles_per_buf)) / 1000000;
+
+		printf("%s-%u-%s(%4uB) Enc %03.3fMpps (%03.3fGbps) ",
+			rte_crypto_cipher_algorithm_strings[cipher_algo],
+			cipher_key_sz * 8,
+			rte_crypto_auth_algorithm_strings[auth_algo],
+			test_data_szs[i],
+			rate, rate  * test_data_szs[i] * 8 / 1000);
+		printf("cycles per buf %03.3f per byte %03.3f\n",
+			cycles_per_buf, cycles_per_buf / test_data_szs[i]);
+
+		for (j = 0; j < MAX_NUM_OPS_INFLIGHT; j++) {
+			tcase = ut_params->test_datas[j];
+
+			switch_blockcipher_enc_to_dec(&tdata, tcase,
+					ciphertext);
+			ret = assemble_blockcipher_buf(tcase, obj, j,
+					op_mask_opp,
+					&tdata,
+					0);
+			if (ret < 0) {
+				printf("Test is not supported by the driver\n");
+				return ret;
+			}
+		}
+
+		time_start = rte_get_timer_cycles();
+
+		run_test(ts_params->ctx, ut_params->sess, obj,
+				MAX_NUM_OPS_INFLIGHT);
+
+		time_now = rte_get_timer_cycles();
+
+		rate = time_now - time_start;
+		cycles_per_buf = rate / MAX_NUM_OPS_INFLIGHT;
+
+		rate = ((hz / cycles_per_buf)) / 1000000;
+
+		printf("%s-%u-%s(%4uB) Dec %03.3fMpps (%03.3fGbps) ",
+			rte_crypto_cipher_algorithm_strings[cipher_algo],
+			cipher_key_sz * 8,
+			rte_crypto_auth_algorithm_strings[auth_algo],
+			test_data_szs[i],
+			rate, rate  * test_data_szs[i] * 8 / 1000);
+		printf("cycles per buf %03.3f per byte %03.3f\n",
+				cycles_per_buf,
+				cycles_per_buf / test_data_szs[i]);
+	}
+
+	return 0;
+}
+
+/* cipher-algo/cipher-key-len/auth-algo/auth-key-len/digest-len/op */
+#define all_block_cipher_perf_test_cases				\
+	TEST_EXPAND(_AES_CBC, 128, _NULL, 0, 0, TOP_ENC)		\
+	TEST_EXPAND(_NULL, 0, _SHA1_HMAC, 160, 20, TOP_AUTH_GEN)	\
+	TEST_EXPAND(_AES_CBC, 128, _SHA1_HMAC, 160, 20, TOP_ENC_AUTH)
+
+#define TEST_EXPAND(a, b, c, d, e, f)					\
+static int								\
+cpu_crypto_blockcipher_perf##a##_##b##c##_##f(void)			\
+{									\
+	return cpu_crypto_test_blockcipher_perf(RTE_CRYPTO_CIPHER##a,	\
+			b / 8, RTE_CRYPTO_AUTH##c, d / 8, e, f);	\
+}									\
+
+all_block_cipher_perf_test_cases
+#undef TEST_EXPAND
+
+static struct unit_test_suite security_cpu_crypto_aesni_mb_perf_testsuite  = {
+	.suite_name = "Security CPU Crypto AESNI-MB Perf Test Suite",
+	.setup = testsuite_setup,
+	.teardown = testsuite_teardown,
+	.unit_test_cases = {
+#define TEST_EXPAND(a, b, c, d, e, f)					\
+	TEST_CASE_ST(ut_setup, ut_teardown,				\
+		cpu_crypto_blockcipher_perf##a##_##b##c##_##f),	\
+
+	all_block_cipher_perf_test_cases
+#undef TEST_EXPAND
+
+	TEST_CASES_END() /**< NULL terminate unit test array */
+	},
+};
+
+static int
+test_security_cpu_crypto_aesni_mb_perf(void)
+{
+	gbl_driver_id =	rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
+
+	return unit_test_suite_runner(
+			&security_cpu_crypto_aesni_mb_perf_testsuite);
+}
+
+
 REGISTER_TEST_COMMAND(security_aesni_gcm_autotest,
 		test_security_cpu_crypto_aesni_gcm);
 
@@ -1130,3 +1321,6 @@ REGISTER_TEST_COMMAND(security_aesni_gcm_perftest,
 
 REGISTER_TEST_COMMAND(security_aesni_mb_autotest,
 		test_security_cpu_crypto_aesni_mb);
+
+REGISTER_TEST_COMMAND(security_aesni_mb_perftest,
+		test_security_cpu_crypto_aesni_mb_perf);
Signature · 1 line
2.14.5
author
#65 lore raw

PATCH 8/10 ipsec: add rte_security cpu_crypto action support

In reply to Fan Zhang

This patch updates the ipsec library to handle the newly introduced RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO action.

---
 lib/librte_ipsec/esp_inb.c  | 174 +++++++++++++++++++++++++-
 lib/librte_ipsec/esp_outb.c | 290 +++++++++++++++++++++++++++++++++++++++++++-
 lib/librte_ipsec/sa.c       |  53 ++++++--
 lib/librte_ipsec/sa.h       |  29 +++++
 lib/librte_ipsec/ses.c      |   4 +-
 5 files changed, 539 insertions(+), 11 deletions(-)
lib/librte_ipsec/esp_inb.c +173 -1
index 8e3ecbc64..6077dcb1e 100644
@@ -105,6 +105,73 @@ inb_cop_prepare(struct rte_crypto_op *cop,
 	}
 }
 
+static inline int
+inb_sync_crypto_proc_prepare(const struct rte_ipsec_sa *sa, struct rte_mbuf *mb,
+	const union sym_op_data *icv, uint32_t pofs, uint32_t plen,
+	struct rte_security_vec *buf, struct iovec *cur_vec,
+	void *iv, void **aad, void **digest)
+{
+	struct rte_mbuf *ms;
+	struct iovec *vec = cur_vec;
+	struct aead_gcm_iv *gcm;
+	struct aesctr_cnt_blk *ctr;
+	uint64_t *ivp;
+	uint32_t algo, left, off = 0, n_seg = 0;
+
+	ivp = rte_pktmbuf_mtod_offset(mb, uint64_t *,
+		pofs + sizeof(struct rte_esp_hdr));
+	algo = sa->algo_type;
+
+	switch (algo) {
+	case ALGO_TYPE_AES_GCM:
+		gcm = (struct aead_gcm_iv *)iv;
+		aead_gcm_iv_fill(gcm, ivp[0], sa->salt);
+		*aad = icv->va + sa->icv_len;
+		off = sa->ctp.cipher.offset + pofs;
+		break;
+	case ALGO_TYPE_AES_CBC:
+	case ALGO_TYPE_3DES_CBC:
+		off = sa->ctp.auth.offset + pofs;
+		break;
+	case ALGO_TYPE_AES_CTR:
+		off = sa->ctp.auth.offset + pofs;
+		ctr = (struct aesctr_cnt_blk *)iv;
+		aes_ctr_cnt_blk_fill(ctr, ivp[0], sa->salt);
+		break;
+	case ALGO_TYPE_NULL:
+		break;
+	}
+
+	*digest = icv->va;
+
+	left = plen - sa->ctp.cipher.length;
+
+	ms = mbuf_get_seg_ofs(mb, &off);
+	if (!ms)
+		return -1;
+
+	while (n_seg < RTE_LIBRTE_IP_FRAG_MAX_FRAG && left && ms) {
+		uint32_t len = RTE_MIN(left, ms->data_len - off);
+
+		vec->iov_base = rte_pktmbuf_mtod_offset(ms, void *, off);
+		vec->iov_len = len;
+
+		left -= len;
+		vec++;
+		n_seg++;
+		ms = ms->next;
+		off = 0;
+	}
+
+	if (left)
+		return -1;
+
+	buf->vec = cur_vec;
+	buf->num = n_seg;
+
+	return n_seg;
+}
+
 /*
  * Helper function for prepare() to deal with situation when
  * ICV is spread by two segments. Tries to move ICV completely into the
@@ -512,7 +579,6 @@ tun_process(const struct rte_ipsec_sa *sa, struct rte_mbuf *mb[],
 	return k;
 }
 
-
 /*
  * *process* function for tunnel packets
  */
@@ -625,6 +691,112 @@ esp_inb_pkt_process(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[],
 	return n;
 }
 
+/*
+ * process packets using sync crypto engine
+ */
+static uint16_t
+esp_inb_sync_crypto_pkt_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num, uint8_t sqh_len,
+		esp_inb_process_t process)
+{
+	int32_t rc;
+	uint32_t i, k, hl, n, p;
+	struct rte_ipsec_sa *sa;
+	struct replay_sqn *rsn;
+	union sym_op_data icv;
+	uint32_t sqn[num];
+	uint32_t dr[num];
+	struct rte_security_vec buf[num];
+	struct iovec vec[RTE_LIBRTE_IP_FRAG_MAX_FRAG * num];
+	uint32_t vec_idx = 0;
+	uint8_t ivs[num][IPSEC_MAX_IV_SIZE];
+	void *iv[num];
+	void *aad[num];
+	void *digest[num];
+	int status[num];
+
+	sa = ss->sa;
+	rsn = rsn_acquire(sa);
+
+	k = 0;
+	for (i = 0; i != num; i++) {
+		hl = mb[i]->l2_len + mb[i]->l3_len;
+		rc = inb_pkt_prepare(sa, rsn, mb[i], hl, &icv);
+		if (rc >= 0) {
+			iv[k] = (void *)ivs[k];
+			rc = inb_sync_crypto_proc_prepare(sa, mb[i], &icv, hl,
+					rc, &buf[k], &vec[vec_idx], iv[k],
+					&aad[k], &digest[k]);
+			if (rc < 0) {
+				dr[i - k] = i;
+				continue;
+			}
+
+			vec_idx += rc;
+			k++;
+		} else
+			dr[i - k] = i;
+	}
+
+	/* copy not prepared mbufs beyond good ones */
+	if (k != num) {
+		rte_errno = EBADMSG;
+
+		if (unlikely(k == 0))
+			return 0;
+
+		move_bad_mbufs(mb, dr, num, num - k);
+	}
+
+	/* process the packets */
+	n = 0;
+	rte_security_process_cpu_crypto_bulk(ss->security.ctx,
+			ss->security.ses, buf, iv, aad, digest, status,
+			k);
+	/* move failed process packets to dr */
+	for (i = 0; i < k; i++) {
+		if (status[i]) {
+			dr[n++] = i;
+			rte_errno = EBADMSG;
+		}
+	}
+
+	/* move bad packets to the back */
+	if (n)
+		move_bad_mbufs(mb, dr, k, n);
+
+	/* process packets */
+	p = process(sa, mb, sqn, dr, k - n, sqh_len);
+
+	if (p != k - n && p != 0)
+		move_bad_mbufs(mb, dr, k - n, k - n - p);
+
+	if (p != num)
+		rte_errno = EBADMSG;
+
+	return p;
+}
+
+uint16_t
+esp_inb_tun_sync_crypto_pkt_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num)
+{
+	struct rte_ipsec_sa *sa = ss->sa;
+
+	return esp_inb_sync_crypto_pkt_process(ss, mb, num, sa->sqh_len,
+			tun_process);
+}
+
+uint16_t
+esp_inb_trs_sync_crypto_pkt_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num)
+{
+	struct rte_ipsec_sa *sa = ss->sa;
+
+	return esp_inb_sync_crypto_pkt_process(ss, mb, num, sa->sqh_len,
+			trs_process);
+}
+
 /*
  * process group of ESP inbound tunnel packets.
  */
lib/librte_ipsec/esp_outb.c +288 -2

302 lines not on this page · raw message

lib/librte_ipsec/sa.c +46 -7

80 lines not on this page · raw message

lib/librte_ipsec/sa.h +29 -0
index 51e69ad05..02c7abc60 100644
@@ -156,6 +156,14 @@ uint16_t
 inline_inb_trs_pkt_process(const struct rte_ipsec_session *ss,
 	struct rte_mbuf *mb[], uint16_t num);
 
+uint16_t
+esp_inb_tun_sync_crypto_pkt_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
+uint16_t
+esp_inb_trs_sync_crypto_pkt_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
 /* outbound processing */
 
 uint16_t
@@ -170,6 +178,10 @@ uint16_t
 esp_outb_sqh_process(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
 	uint16_t num);
 
+uint16_t
+esp_outb_pkt_flag_process(const struct rte_ipsec_session *ss,
+	struct rte_mbuf *mb[], uint16_t num);
+
 uint16_t
 inline_outb_tun_pkt_process(const struct rte_ipsec_session *ss,
 	struct rte_mbuf *mb[], uint16_t num);
@@ -182,4 +194,21 @@ uint16_t
 inline_proto_outb_pkt_process(const struct rte_ipsec_session *ss,
 	struct rte_mbuf *mb[], uint16_t num);
 
+uint16_t
+esp_outb_tun_sync_crpyto_sqh_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
+uint16_t
+esp_outb_tun_sync_crpyto_flag_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
+uint16_t
+esp_outb_trs_sync_crpyto_sqh_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
+uint16_t
+esp_outb_trs_sync_crpyto_flag_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
+
 #endif /* _SA_H_ */
lib/librte_ipsec/ses.c +3 -1
index 82c765a33..eaa8c17b7 100644
@@ -19,7 +19,9 @@ session_check(struct rte_ipsec_session *ss)
 			return -EINVAL;
 		if ((ss->type == RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO ||
 				ss->type ==
-				RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL) &&
+				RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL ||
+				ss->type ==
+				RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) &&
 				ss->security.ctx == NULL)
 			return -EINVAL;
 	}
Signature · 1 line
2.14.5
#66 lore raw

Re: [dpdk-dev] [PATCH 08/10] ipsec: add rte_security cpu_crypto action support

Hi Fan,

...

Quoted 11 lines
lib/librte_ipsec/esp_outb.c +3 -0
index 55799a867..097cb663f 100644
@@ -403,6 +403,292 @@ esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
 	return k;
 }
 
+
+static inline int
+outb_sync_crypto_proc_prepare(struct rte_mbuf *m, const struct rte_ipsec_sa *sa,

lib/librte_ipsec/esp_outb.c esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], · line 418

Quoted 10 lines
lib/librte_ipsec/esp_outb.c +10 -0
+		const uint64_t ivp[IPSEC_MAX_IV_QWORD],
+		const union sym_op_data *icv, uint32_t hlen, uint32_t plen,
+		struct rte_security_vec *buf, struct iovec *cur_vec, void *iv,
+		void **aad, void **digest)
+{
+	struct rte_mbuf *ms;
+	struct aead_gcm_iv *gcm;
+	struct aesctr_cnt_blk *ctr;
+	struct iovec *vec = cur_vec;
+	uint32_t left, off = 0, n_seg = 0;

Please separate variable definition and value assignment.
It makes it hard to read, plus we don't do that in the rest of the library, so better to follow rest of the code style.

lib/librte_ipsec/esp_outb.c esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], · line 427

Quoted 9 lines
lib/librte_ipsec/esp_outb.c +9 -0
+	uint32_t algo;
+
+	algo = sa->algo_type;
+
+	switch (algo) {
+	case ALGO_TYPE_AES_GCM:
+		gcm = iv;
+		aead_gcm_iv_fill(gcm, ivp[0], sa->salt);
+		*aad = (void *)(icv->va + sa->icv_len);

Why do we want to allocate aad inside the packet at all?
Why not just to do that on the stack instead?
In that case you probably wouldn't need this icv stuff at all to be passed to that function.

lib/librte_ipsec/esp_outb.c esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], · line 439

Quoted 12 lines
lib/librte_ipsec/esp_outb.c +12 -0
+		off = sa->ctp.cipher.offset + hlen;
+		break;
+	case ALGO_TYPE_AES_CBC:
+	case ALGO_TYPE_3DES_CBC:
+		off = sa->ctp.auth.offset + hlen;
+		break;
+	case ALGO_TYPE_AES_CTR:
+		ctr = iv;
+		aes_ctr_cnt_blk_fill(ctr, ivp[0], sa->salt);
+		break;
+	case ALGO_TYPE_NULL:
+		break;

For latest two, why off is zero?
Shouldn't it at least be 'hlen'?
In fact, I think it needs to be: sa->ctp.auth.offset + hlen;

lib/librte_ipsec/esp_outb.c esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], · line 442

Quoted 3 lines
lib/librte_ipsec/esp_outb.c +3 -0
+	}
+
+	*digest = (void *)icv->va;

Could be done in the upper layer function, together with aad assignment, I think.

Looking at this function, it seems to consist of 2 separate parts:
1. calculates offset and generates iv
2. setup iovec[].
Probably worth to split it into 2 separate functions like that. Would be much easier to read/understand.

lib/librte_ipsec/esp_outb.c esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], · line 448

Quoted 6 lines
lib/librte_ipsec/esp_outb.c +6 -0
+
+	left = sa->ctp.cipher.length + plen;
+
+	ms = mbuf_get_seg_ofs(m, &off);
+	if (!ms)
+		return -1;

outb_tun_pkt_prepare() should already check that we have a valid packet. I don't think there is a need to check for any failure here. Another thing, our esp header will be in the first segment for sure, so do we need get_seg_ofs() here at all?

lib/librte_ipsec/esp_outb.c esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], · line 450

Quoted 2 lines
lib/librte_ipsec/esp_outb.c +2 -0
+
+	while (n_seg < RTE_LIBRTE_IP_FRAG_MAX_FRAG && left && ms) {

I don't think this is right, we shouldn't impose additional limitations to the number of segments in the packet.

lib/librte_ipsec/esp_outb.c esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], · line 460

Quoted 10 lines
lib/librte_ipsec/esp_outb.c +10 -0
+		uint32_t len = RTE_MIN(left, ms->data_len - off);
+
+		vec->iov_base = rte_pktmbuf_mtod_offset(ms, void *, off);
+		vec->iov_len = len;
+
+		left -= len;
+		vec++;
+		n_seg++;
+		ms = ms->next;
+		off = 0;

Whole construction seems a bit over-complicated here...
Why just not have a separate function that would dill iovec[] from mbuf And return an error if there is not enough iovec[] entries? Something like:

static inline int
mbuf_to_iovec(const struct rte_mbuf *mb, uint32_t ofs, uint32_t len, struct iovec vec[], uint32_t num)

{
     uint32_t i;
     if (mb->nb_seg > num)
        return - mb->nb_seg;

    vec[0].iov_base =  rte_pktmbuf_mtod_offset(mb, void *, off);
    vec[0].iov_len = mb->data_len - off;

    for (i = 1, ms = mb->next; mb != NULL; ms = ms->next, i++) {
        vec[i].iov_base = rte_pktmbuf_mtod(ms);
        vec[i].iov_len = ms->data_len;
    }

vec[i].iov_len -= mb->pkt_len - len;
return i;
}

Then we can use that function to fill our iovec[] in a loop.

Quoted 8 lines
lib/librte_ipsec/esp_outb.c +8 -0
+	}
+
+	if (left)
+		return -1;
+
+	buf->vec = cur_vec;
+	buf->num = n_seg;
+

lib/librte_ipsec/esp_outb.c esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], · line 478

Quoted 10 lines
lib/librte_ipsec/esp_outb.c +10 -0
+	return n_seg;
+}
+
+/**
+ * Local post process function prototype that same as process function prototype
+ * as rte_ipsec_sa_pkt_func's process().
+ */
+typedef uint16_t (*sync_crypto_post_process)(const struct rte_ipsec_session *ss,
+				struct rte_mbuf *mb[],
+				uint16_t num);

Stylish thing: typdef newtype_t ....

Quoted 9 lines
lib/librte_ipsec/esp_outb.c +9 -0
+static uint16_t
+esp_outb_tun_sync_crypto_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num,
+		sync_crypto_post_process post_process)
+{
+	uint64_t sqn;
+	rte_be64_t sqc;
+	struct rte_ipsec_sa *sa;
+	struct rte_security_ctx *ctx;

lib/librte_ipsec/esp_outb.c esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], · line 497

Quoted 10 lines
lib/librte_ipsec/esp_outb.c +10 -0
+	struct rte_security_session *rss;
+	union sym_op_data icv;
+	struct rte_security_vec buf[num];
+	struct iovec vec[RTE_LIBRTE_IP_FRAG_MAX_FRAG * num];
+	uint32_t vec_idx = 0;
+	void *aad[num];
+	void *digest[num];
+	void *iv[num];
+	uint8_t ivs[num][IPSEC_MAX_IV_SIZE];
+	uint64_t ivp[IPSEC_MAX_IV_QWORD];

Why do we need both ivs and ivp?

Quoted 16 lines
lib/librte_ipsec/esp_outb.c +16 -0
+	int status[num];
+	uint32_t dr[num];
+	uint32_t i, n, k;
+	int32_t rc;
+
+	sa = ss->sa;
+	ctx = ss->security.ctx;
+	rss = ss->security.ses;
+
+	k = 0;
+	n = num;
+	sqn = esn_outb_update_sqn(sa, &n);
+	if (n != num)
+		rte_errno = EOVERFLOW;
+
+	for (i = 0; i != n; i++) {

lib/librte_ipsec/esp_outb.c esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], · line 523

Quoted 10 lines
lib/librte_ipsec/esp_outb.c +10 -0
+		sqc = rte_cpu_to_be_64(sqn + i);
+		gen_iv(ivp, sqc);
+
+		/* try to update the packet itself */
+		rc = outb_tun_pkt_prepare(sa, sqc, ivp, mb[i], &icv,
+				sa->sqh_len);
+
+		/* success, setup crypto op */
+		if (rc >= 0) {
+			outb_pkt_xprepare(sa, sqc, &icv);

We probably need something like outb_pkt_sync_xprepare(sa, sqc, &aad[i]); here. To avoid using space in the packet for aad.

lib/librte_ipsec/esp_outb.c esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], · line 525

Quoted 2 lines
lib/librte_ipsec/esp_outb.c +2 -0
+
+			iv[k] = (void *)ivs[k];

Do we really need type conversion here?

Quoted 16 lines
lib/librte_ipsec/esp_outb.c +13 -0
+			rc = outb_sync_crypto_proc_prepare(mb[i], sa, ivp, &icv,
+					0, rc, &buf[k], &vec[vec_idx], iv[k],
+					&aad[k], &digest[k]);
 
 
 
+			if (rc < 0) {
+				dr[i - k] = i;
+				rte_errno = -rc;
+				continue;
+			}
+
+			vec_idx += rc;
+			k++;
+		/* failure, put packet into the death-row */
+		} else {

lib/librte_ipsec/esp_outb.c esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], · line 548

Quoted 10 lines
lib/librte_ipsec/esp_outb.c +10 -0
+			dr[i - k] = i;
+			rte_errno = -rc;
+		}
+	}
+
+	 /* copy not prepared mbufs beyond good ones */
+	if (k != n && k != 0)
+		move_bad_mbufs(mb, dr, n, n - k);
+
+	if (unlikely(k == 0)) {

I don't think 'unlikely' will make any difference here here.

lib/librte_ipsec/esp_outb.c esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], · line 556

Quoted 8 lines
lib/librte_ipsec/esp_outb.c +8 -0
+		rte_errno = EBADMSG;
+		return 0;
+	}
+
+	/* process the packets */
+	n = 0;
+	rte_security_process_cpu_crypto_bulk(ctx, rss, buf, iv, aad, digest,
+			status, k);

Looking at the code below, I think it will be plausible to make rte_security_process_cpu_crypto_bulk() to return number of failures (or number of succese).

lib/librte_ipsec/esp_outb.c esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], · line 558

Quoted 2 lines
lib/librte_ipsec/esp_outb.c +2 -0
+	/* move failed process packets to dr */
+	for (i = 0; i < n; i++) {

That loop will never be executed.
Should be i < k.

lib/librte_ipsec/esp_outb.c esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], · line 563

Quoted 2 lines
lib/librte_ipsec/esp_outb.c +2 -0
+		if (status[i])
+			dr[n++] = i;

Forgot to set rte_errno.

lib/librte_ipsec/esp_outb.c esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], · line 566

Quoted 3 lines
lib/librte_ipsec/esp_outb.c +3 -0
+	}
+
+	if (n)

if (n != 0 && n != k)

Quoted 69 lines
lib/librte_ipsec/esp_outb.c +69 -0
+		move_bad_mbufs(mb, dr, k, n);
+
+	return post_process(ss, mb, k - n);
+}
+
+static uint16_t
+esp_outb_trs_sync_crypto_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num,
+		sync_crypto_post_process post_process)
+
+{
+	uint64_t sqn;
+	rte_be64_t sqc;
+	struct rte_ipsec_sa *sa;
+	struct rte_security_ctx *ctx;
+	struct rte_security_session *rss;
+	union sym_op_data icv;
+	struct rte_security_vec buf[num];
+	struct iovec vec[RTE_LIBRTE_IP_FRAG_MAX_FRAG * num];
+	uint32_t vec_idx = 0;
+	void *aad[num];
+	void *digest[num];
+	uint8_t ivs[num][IPSEC_MAX_IV_SIZE];
+	void *iv[num];
+	int status[num];
+	uint64_t ivp[IPSEC_MAX_IV_QWORD];
+	uint32_t dr[num];
+	uint32_t i, n, k;
+	uint32_t l2, l3;
+	int32_t rc;
+
+	sa = ss->sa;
+	ctx = ss->security.ctx;
+	rss = ss->security.ses;
+
+	k = 0;
+	n = num;
+	sqn = esn_outb_update_sqn(sa, &n);
+	if (n != num)
+		rte_errno = EOVERFLOW;
+
+	for (i = 0; i != n; i++) {
+		l2 = mb[i]->l2_len;
+		l3 = mb[i]->l3_len;
+
+		sqc = rte_cpu_to_be_64(sqn + i);
+		gen_iv(ivp, sqc);
+
+		/* try to update the packet itself */
+		rc = outb_trs_pkt_prepare(sa, sqc, ivp, mb[i], l2, l3, &icv,
+				sa->sqh_len);
+
+		/* success, setup crypto op */
+		if (rc >= 0) {
+			outb_pkt_xprepare(sa, sqc, &icv);
+
+			iv[k] = (void *)ivs[k];
+
+			rc = outb_sync_crypto_proc_prepare(mb[i], sa, ivp, &icv,
+					l2 + l3, rc, &buf[k], &vec[vec_idx],
+					iv[k], &aad[k], &digest[k]);
+			if (rc < 0) {
+				dr[i - k] = i;
+				rte_errno = -rc;
+				continue;
+			}
+
+			vec_idx += rc;
+			k++;

lib/librte_ipsec/esp_outb.c esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], · line 642

Quoted 10 lines
lib/librte_ipsec/esp_outb.c +10 -0
+		/* failure, put packet into the death-row */
+		} else {
+			dr[i - k] = i;
+			rte_errno = -rc;
+		}
+	}
+
+	 /* copy not prepared mbufs beyond good ones */
+	if (k != n && k != 0)
+		move_bad_mbufs(mb, dr, n, n - k);

You don't really need to do it here.
Just one such thing at the very end should be enough.

Quoted 13 lines
lib/librte_ipsec/esp_outb.c +13 -0
+
+	/* process the packets */
+	n = 0;
+	rte_security_process_cpu_crypto_bulk(ctx, rss, buf, iv, aad, digest,
+			status, k);
+	/* move failed process packets to dr */
+	for (i = 0; i < k; i++) {
+		if (status[i])
+			dr[n++] = i;
+	}
+
+	if (n)
+		move_bad_mbufs(mb, dr, k, n);

lib/librte_ipsec/esp_outb.c esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], · line 665

Quoted 10 lines
lib/librte_ipsec/esp_outb.c +10 -0
+
+	return post_process(ss, mb, k - n);
+}
+
+uint16_t
+esp_outb_tun_sync_crpyto_sqh_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num)
+{
+	return esp_outb_tun_sync_crypto_process(ss, mb, num,
+			esp_outb_sqh_process);

esp_outb_sqh_process() relies on PKT_RX_SEC_OFFLOAD_FAILED been set in mb->ol_flags for failed packets.
At first for _sync_ case no-one will set it for you.
Second - for _sync_ you don't really need that it is just an extra overhead here. So I think you can't reuse this function without some modifications here. Probably easier to make a new one (and extract some common code into another helper function that esp_out_sqh_process and new one can call).

lib/librte_ipsec/esp_outb.c esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], · line 673

Quoted 8 lines
lib/librte_ipsec/esp_outb.c +8 -0
+}
+
+uint16_t
+esp_outb_tun_sync_crpyto_flag_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num)
+{
+	return esp_outb_tun_sync_crypto_process(ss, mb, num,
+			esp_outb_pkt_flag_process);

Same as above, plus the fact that you made esp_outb_pkt_flag_process() is not a static one, so compiler wouldn't be able to inline it.

Quoted 22 lines
lib/librte_ipsec/esp_outb.c +18 -0
+}
+
+uint16_t
+esp_outb_trs_sync_crpyto_sqh_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num)
+{
+	return esp_outb_trs_sync_crypto_process(ss, mb, num,
+			esp_outb_sqh_process);
+}
+
+uint16_t
+esp_outb_trs_sync_crpyto_flag_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num)
+{
+	return esp_outb_trs_sync_crypto_process(ss, mb, num,
+			esp_outb_pkt_flag_process);
+}
+

/*
* process outbound packets for SA with ESN support,
* for algorithms that require SQN.hibits to be implictly included

lib/librte_ipsec/esp_outb.c esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], · line 700

Quoted 7 lines
lib/librte_ipsec/esp_outb.c +2 -2
@@ -410,8 +696,8 @@ esp_outb_trs_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
  * In that case we have to move ICV bytes back to their proper place.
  */
 uint16_t
-esp_outb_sqh_process(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
-	uint16_t num)
+esp_outb_sqh_process(const struct rte_ipsec_session *ss,
+	struct rte_mbuf *mb[], uint16_t num)

Any purpose for that change?

Quoted 8 lines
lib/librte_ipsec/esp_outb.c +0 -0
 {
 	uint32_t i, k, icv_len, *icv;
 	struct rte_mbuf *ml;
lib/librte_ipsec/sa.c +0 -0
index 23d394b46..31ffbce2c 100644

lib/librte_ipsec/sa.c lksd_proto_prepare(const struct rte_ipsec_session *ss, · line 549

Quoted 9 lines
lib/librte_ipsec/sa.c +3 -3
@@ -544,9 +544,9 @@ lksd_proto_prepare(const struct rte_ipsec_session *ss,
  * - inbound/outbound for RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL
  * - outbound for RTE_SECURITY_ACTION_TYPE_NONE when ESN is disabled
  */
-static uint16_t
-pkt_flag_process(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
-	uint16_t num)
+uint16_t
+esp_outb_pkt_flag_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num)

Why to rename this function?
As comment above it states, the function is used for both inbound and outbound code path.
Such renaming seems misleading to me.

Quoted 16 lines
lib/librte_ipsec/sa.c +3 -2
 {
 	uint32_t i, k;
 	uint32_t dr[num];
@@ -599,12 +599,48 @@ lksd_none_pkt_func_select(const struct rte_ipsec_sa *sa,
 	case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TUNLV6):
 		pf->prepare = esp_outb_tun_prepare;
 		pf->process = (sa->sqh_len != 0) ?
-			esp_outb_sqh_process : pkt_flag_process;
+			esp_outb_sqh_process : esp_outb_pkt_flag_process;
 		break;
 	case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TRANS):
 		pf->prepare = esp_outb_trs_prepare;
 		pf->process = (sa->sqh_len != 0) ?
-			esp_outb_sqh_process : pkt_flag_process;
+			esp_outb_sqh_process : esp_outb_pkt_flag_process;
+		break;

lib/librte_ipsec/sa.c lksd_none_pkt_func_select(const struct rte_ipsec_sa *sa, · line 618

Quoted 10 lines
lib/librte_ipsec/sa.c +10 -0
+	default:
+		rc = -ENOTSUP;
+	}
+
+	return rc;
+}
+
+static int
+lksd_sync_crypto_pkt_func_select(const struct rte_ipsec_sa *sa,
+		struct rte_ipsec_sa_pkt_func *pf)

As a nit: probably no point to have lksd_prefix for _sync_ functions.

Quoted 99 lines
lib/librte_ipsec/sa.c +30 -2
+{
+	int32_t rc;
+
+	static const uint64_t msk = RTE_IPSEC_SATP_DIR_MASK |
+			RTE_IPSEC_SATP_MODE_MASK;
+
+	rc = 0;
+	switch (sa->type & msk) {
+	case (RTE_IPSEC_SATP_DIR_IB | RTE_IPSEC_SATP_MODE_TUNLV4):
+	case (RTE_IPSEC_SATP_DIR_IB | RTE_IPSEC_SATP_MODE_TUNLV6):
+		pf->process = esp_inb_tun_sync_crypto_pkt_process;
+		break;
+	case (RTE_IPSEC_SATP_DIR_IB | RTE_IPSEC_SATP_MODE_TRANS):
+		pf->process = esp_inb_trs_sync_crypto_pkt_process;
+		break;
+	case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TUNLV4):
+	case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TUNLV6):
+		pf->process = (sa->sqh_len != 0) ?
+			esp_outb_tun_sync_crpyto_sqh_process :
+			esp_outb_tun_sync_crpyto_flag_process;
+		break;
+	case (RTE_IPSEC_SATP_DIR_OB | RTE_IPSEC_SATP_MODE_TRANS):
+		pf->process = (sa->sqh_len != 0) ?
+			esp_outb_trs_sync_crpyto_sqh_process :
+			esp_outb_trs_sync_crpyto_flag_process;
 		break;
 	default:
 		rc = -ENOTSUP;
@@ -672,13 +708,16 @@ ipsec_sa_pkt_func_select(const struct rte_ipsec_session *ss,
 	case RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL:
 		if ((sa->type & RTE_IPSEC_SATP_DIR_MASK) ==
 				RTE_IPSEC_SATP_DIR_IB)
-			pf->process = pkt_flag_process;
+			pf->process = esp_outb_pkt_flag_process;
 		else
 			pf->process = inline_proto_outb_pkt_process;
 		break;
 	case RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL:
 		pf->prepare = lksd_proto_prepare;
-		pf->process = pkt_flag_process;
+		pf->process = esp_outb_pkt_flag_process;
+		break;
+	case RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO:
+		rc = lksd_sync_crypto_pkt_func_select(sa, pf);
 		break;
 	default:
 		rc = -ENOTSUP;
lib/librte_ipsec/sa.h +29 -0
index 51e69ad05..02c7abc60 100644
@@ -156,6 +156,14 @@ uint16_t
 inline_inb_trs_pkt_process(const struct rte_ipsec_session *ss,
 	struct rte_mbuf *mb[], uint16_t num);
 
+uint16_t
+esp_inb_tun_sync_crypto_pkt_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
+uint16_t
+esp_inb_trs_sync_crypto_pkt_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
 /* outbound processing */
 
 uint16_t
@@ -170,6 +178,10 @@ uint16_t
 esp_outb_sqh_process(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
 	uint16_t num);
 
+uint16_t
+esp_outb_pkt_flag_process(const struct rte_ipsec_session *ss,
+	struct rte_mbuf *mb[], uint16_t num);
+
 uint16_t
 inline_outb_tun_pkt_process(const struct rte_ipsec_session *ss,
 	struct rte_mbuf *mb[], uint16_t num);
@@ -182,4 +194,21 @@ uint16_t
 inline_proto_outb_pkt_process(const struct rte_ipsec_session *ss,
 	struct rte_mbuf *mb[], uint16_t num);
 
+uint16_t
+esp_outb_tun_sync_crpyto_sqh_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
+uint16_t
+esp_outb_tun_sync_crpyto_flag_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
+uint16_t
+esp_outb_trs_sync_crpyto_sqh_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
+uint16_t
+esp_outb_trs_sync_crpyto_flag_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
+
 #endif /* _SA_H_ */
#67 lore raw

Re: [dpdk-dev] [PATCH 08/10] ipsec: add rte_security cpu_crypto action support

In reply to Fan Zhang

Hi Fan,

Quoted 23 lines

This patch updates the ipsec library to handle the newly introduced RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO action.

---
 lib/librte_ipsec/esp_inb.c  | 174 +++++++++++++++++++++++++-
 lib/librte_ipsec/esp_outb.c | 290 +++++++++++++++++++++++++++++++++++++++++++-
 lib/librte_ipsec/sa.c       |  53 ++++++--
 lib/librte_ipsec/sa.h       |  29 +++++
 lib/librte_ipsec/ses.c      |   4 +-
 5 files changed, 539 insertions(+), 11 deletions(-)
lib/librte_ipsec/esp_inb.c +2 -0
index 8e3ecbc64..6077dcb1e 100644
@@ -105,6 +105,73 @@ inb_cop_prepare(struct rte_crypto_op *cop,
 	}
 }
 
+static inline int
+inb_sync_crypto_proc_prepare(const struct rte_ipsec_sa *sa, struct rte_mbuf *mb,

lib/librte_ipsec/esp_inb.c inb_cop_prepare(struct rte_crypto_op *cop, · line 119

Quoted 10 lines
lib/librte_ipsec/esp_inb.c +10 -0
+	const union sym_op_data *icv, uint32_t pofs, uint32_t plen,
+	struct rte_security_vec *buf, struct iovec *cur_vec,
+	void *iv, void **aad, void **digest)
+{
+	struct rte_mbuf *ms;
+	struct iovec *vec = cur_vec;
+	struct aead_gcm_iv *gcm;
+	struct aesctr_cnt_blk *ctr;
+	uint64_t *ivp;
+	uint32_t algo, left, off = 0, n_seg = 0;

Same thing as for outbound pls keep definitions and assignments separated.

Quoted 22 lines
lib/librte_ipsec/esp_inb.c +22 -0
+
+	ivp = rte_pktmbuf_mtod_offset(mb, uint64_t *,
+		pofs + sizeof(struct rte_esp_hdr));
+	algo = sa->algo_type;
+
+	switch (algo) {
+	case ALGO_TYPE_AES_GCM:
+		gcm = (struct aead_gcm_iv *)iv;
+		aead_gcm_iv_fill(gcm, ivp[0], sa->salt);
+		*aad = icv->va + sa->icv_len;
+		off = sa->ctp.cipher.offset + pofs;
+		break;
+	case ALGO_TYPE_AES_CBC:
+	case ALGO_TYPE_3DES_CBC:
+		off = sa->ctp.auth.offset + pofs;
+		break;
+	case ALGO_TYPE_AES_CTR:
+		off = sa->ctp.auth.offset + pofs;
+		ctr = (struct aesctr_cnt_blk *)iv;
+		aes_ctr_cnt_blk_fill(ctr, ivp[0], sa->salt);
+		break;
+	case ALGO_TYPE_NULL:

lib/librte_ipsec/esp_inb.c inb_cop_prepare(struct rte_crypto_op *cop, · line 151

Quoted 10 lines
lib/librte_ipsec/esp_inb.c +10 -0
+		break;
+	}
+
+	*digest = icv->va;
+
+	left = plen - sa->ctp.cipher.length;
+
+	ms = mbuf_get_seg_ofs(mb, &off);
+	if (!ms)
+		return -1;

Same as for outbound: I think no need to check/return failure. This function could be split into two.

lib/librte_ipsec/esp_inb.c inb_cop_prepare(struct rte_crypto_op *cop, · line 153

Quoted 2 lines
lib/librte_ipsec/esp_inb.c +2 -0
+
+	while (n_seg < RTE_LIBRTE_IP_FRAG_MAX_FRAG && left && ms) {

Same thing - we shouldn't limt ourselves to 5 segs per packet. Pretty much same comments about code restructuring as for outbound case.

Quoted 99 lines
lib/librte_ipsec/esp_inb.c +84 -1
+		uint32_t len = RTE_MIN(left, ms->data_len - off);
+
+		vec->iov_base = rte_pktmbuf_mtod_offset(ms, void *, off);
+		vec->iov_len = len;
+
+		left -= len;
+		vec++;
+		n_seg++;
+		ms = ms->next;
+		off = 0;
+	}
+
+	if (left)
+		return -1;
+
+	buf->vec = cur_vec;
+	buf->num = n_seg;
+
+	return n_seg;
+}
+
 /*
  * Helper function for prepare() to deal with situation when
  * ICV is spread by two segments. Tries to move ICV completely into the
@@ -512,7 +579,6 @@ tun_process(const struct rte_ipsec_sa *sa, struct rte_mbuf *mb[],
 	return k;
 }
 
-
 /*
  * *process* function for tunnel packets
  */
@@ -625,6 +691,112 @@ esp_inb_pkt_process(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[],
 	return n;
 }
 
+/*
+ * process packets using sync crypto engine
+ */
+static uint16_t
+esp_inb_sync_crypto_pkt_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num, uint8_t sqh_len,
+		esp_inb_process_t process)
+{
+	int32_t rc;
+	uint32_t i, k, hl, n, p;
+	struct rte_ipsec_sa *sa;
+	struct replay_sqn *rsn;
+	union sym_op_data icv;
+	uint32_t sqn[num];
+	uint32_t dr[num];
+	struct rte_security_vec buf[num];
+	struct iovec vec[RTE_LIBRTE_IP_FRAG_MAX_FRAG * num];
+	uint32_t vec_idx = 0;
+	uint8_t ivs[num][IPSEC_MAX_IV_SIZE];
+	void *iv[num];
+	void *aad[num];
+	void *digest[num];
+	int status[num];
+
+	sa = ss->sa;
+	rsn = rsn_acquire(sa);
+
+	k = 0;
+	for (i = 0; i != num; i++) {
+		hl = mb[i]->l2_len + mb[i]->l3_len;
+		rc = inb_pkt_prepare(sa, rsn, mb[i], hl, &icv);
+		if (rc >= 0) {
+			iv[k] = (void *)ivs[k];
+			rc = inb_sync_crypto_proc_prepare(sa, mb[i], &icv, hl,
+					rc, &buf[k], &vec[vec_idx], iv[k],
+					&aad[k], &digest[k]);
+			if (rc < 0) {
+				dr[i - k] = i;
+				continue;
+			}
+
+			vec_idx += rc;
+			k++;
+		} else
+			dr[i - k] = i;
+	}
+
+	/* copy not prepared mbufs beyond good ones */
+	if (k != num) {
+		rte_errno = EBADMSG;
+
+		if (unlikely(k == 0))
+			return 0;
+
+		move_bad_mbufs(mb, dr, num, num - k);
+	}
+
+	/* process the packets */
+	n = 0;
+	rte_security_process_cpu_crypto_bulk(ss->security.ctx,
+			ss->security.ses, buf, iv, aad, digest, status,
+			k);
+	/* move failed process packets to dr */

lib/librte_ipsec/esp_inb.c esp_inb_pkt_process(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], · line 766

Quoted 10 lines
lib/librte_ipsec/esp_inb.c +10 -0
+	for (i = 0; i < k; i++) {
+		if (status[i]) {
+			dr[n++] = i;
+			rte_errno = EBADMSG;
+		}
+	}
+
+	/* move bad packets to the back */
+	if (n)
+		move_bad_mbufs(mb, dr, k, n);

I don't think you need to set dr[] here and call that function, see below.

lib/librte_ipsec/esp_inb.c esp_inb_pkt_process(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], · line 769

Quoted 3 lines
lib/librte_ipsec/esp_inb.c +3 -0
+
+	/* process packets */
+	p = process(sa, mb, sqn, dr, k - n, sqh_len);

tun_process(), etc. expects PKT_RX_SEC_OFFLOAD_FAILED to be set in mb->ol_flags for failed packets.
So you either need to set this value in ol_flags based on status, or tweak existing process functions, or introduce new ones.

Quoted 33 lines
lib/librte_ipsec/esp_inb.c +30 -0
+
+	if (p != k - n && p != 0)
+		move_bad_mbufs(mb, dr, k - n, k - n - p);
+
+	if (p != num)
+		rte_errno = EBADMSG;
+
+	return p;
+}
+
+uint16_t
+esp_inb_tun_sync_crypto_pkt_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num)
+{
+	struct rte_ipsec_sa *sa = ss->sa;
+
+	return esp_inb_sync_crypto_pkt_process(ss, mb, num, sa->sqh_len,
+			tun_process);
+}
+
+uint16_t
+esp_inb_trs_sync_crypto_pkt_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num)
+{
+	struct rte_ipsec_sa *sa = ss->sa;
+
+	return esp_inb_sync_crypto_pkt_process(ss, mb, num, sa->sqh_len,
+			trs_process);
+}
+
 /*
  * process group of ESP inbound tunnel packets.
  */
author
#68 lore raw

PATCH 9/10 examples/ipsec-secgw: add security cpu_crypto action support

In reply to Fan Zhang

Since ipsec library is added cpu_crypto security action type support, this patch updates ipsec-secgw sample application with added action type "cpu-crypto". The patch also includes a number of test scripts to prove the correctness of the implementation.

---
 examples/ipsec-secgw/ipsec.c                       | 22 ++++++++++++++++++++++
 examples/ipsec-secgw/ipsec_process.c               |  7 ++++---
 examples/ipsec-secgw/sa.c                          | 13 +++++++++++--
 examples/ipsec-secgw/test/run_test.sh              | 10 ++++++++++
 .../test/trs_3descbc_sha1_cpu_crypto_defs.sh       |  5 +++++
 .../test/trs_aescbc_sha1_cpu_crypto_defs.sh        |  5 +++++
 .../test/trs_aesctr_sha1_cpu_crypto_defs.sh        |  5 +++++
 .../ipsec-secgw/test/trs_aesgcm_cpu_crypto_defs.sh |  5 +++++
 .../test/trs_aesgcm_mb_cpu_crypto_defs.sh          |  7 +++++++
 .../test/tun_3descbc_sha1_cpu_crypto_defs.sh       |  5 +++++
 .../test/tun_aescbc_sha1_cpu_crypto_defs.sh        |  5 +++++
 .../test/tun_aesctr_sha1_cpu_crypto_defs.sh        |  5 +++++
 .../ipsec-secgw/test/tun_aesgcm_cpu_crypto_defs.sh |  5 +++++
 .../test/tun_aesgcm_mb_cpu_crypto_defs.sh          |  7 +++++++
 14 files changed, 101 insertions(+), 5 deletions(-)
 create mode 100644 examples/ipsec-secgw/test/trs_3descbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aescbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aesctr_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_mb_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_3descbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aescbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aesctr_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_mb_cpu_crypto_defs.sh
examples/ipsec-secgw/ipsec.c +22 -0
index dc85adfe5..4c39a7de6 100644
@@ -10,6 +10,7 @@
 #include <rte_crypto.h>
 #include <rte_security.h>
 #include <rte_cryptodev.h>
+#include <rte_ipsec.h>
 #include <rte_ethdev.h>
 #include <rte_mbuf.h>
 #include <rte_hash.h>
@@ -105,6 +106,26 @@ create_lookaside_session(struct ipsec_ctx *ipsec_ctx, struct ipsec_sa *sa)
 				"SEC Session init failed: err: %d\n", ret);
 				return -1;
 			}
+		} else if (sa->type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) {
+			struct rte_security_ctx *ctx =
+				(struct rte_security_ctx *)
+				rte_cryptodev_get_sec_ctx(
+					ipsec_ctx->tbl[cdev_id_qp].id);
+			int32_t offset = sizeof(struct rte_esp_hdr) +
+					sa->iv_len;
+
+			/* Set IPsec parameters in conf */
+			sess_conf.cpucrypto.cipher_offset = offset;
+
+			set_ipsec_conf(sa, &(sess_conf.ipsec));
+			sa->security_ctx = ctx;
+			sa->sec_session = rte_security_session_create(ctx,
+				&sess_conf, ipsec_ctx->session_priv_pool);
+			if (sa->sec_session == NULL) {
+				RTE_LOG(ERR, IPSEC,
+				"SEC Session init failed: err: %d\n", ret);
+				return -1;
+			}
 		} else {
 			RTE_LOG(ERR, IPSEC, "Inline not supported\n");
 			return -1;
@@ -473,6 +494,7 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec_ctx *ipsec_ctx,
 						sa->sec_session, pkts[i], NULL);
 			continue;
 		case RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO:
+		case RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO:
 			RTE_ASSERT(sa->sec_session != NULL);
 			priv->cop.type = RTE_CRYPTO_OP_TYPE_SYMMETRIC;
 			priv->cop.status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED;
examples/ipsec-secgw/ipsec_process.c +4 -3
index 868f1a28d..1932b631f 100644
@@ -101,7 +101,8 @@ fill_ipsec_session(struct rte_ipsec_session *ss, struct ipsec_ctx *ctx,
 		}
 		ss->crypto.ses = sa->crypto_session;
 	/* setup session action type */
-	} else if (sa->type == RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL) {
+	} else if (sa->type == RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL ||
+			sa->type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) {
 		if (sa->sec_session == NULL) {
 			rc = create_lookaside_session(ctx, sa);
 			if (rc != 0)
@@ -227,8 +228,8 @@ ipsec_process(struct ipsec_ctx *ctx, struct ipsec_traffic *trf)
 
 		/* process packets inline */
 		else if (sa->type == RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO ||
-				sa->type ==
-				RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL) {
+			sa->type == RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL ||
+			sa->type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) {
 
 			satp = rte_ipsec_sa_type(ips->sa);
 
examples/ipsec-secgw/sa.c +11 -2
index c3cf3bd1f..ba773346f 100644
@@ -570,6 +570,9 @@ parse_sa_tokens(char **tokens, uint32_t n_tokens,
 				RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL;
 			else if (strcmp(tokens[ti], "no-offload") == 0)
 				rule->type = RTE_SECURITY_ACTION_TYPE_NONE;
+			else if (strcmp(tokens[ti], "cpu-crypto") == 0)
+				rule->type =
+					RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO;
 			else {
 				APP_CHECK(0, status, "Invalid input \"%s\"",
 						tokens[ti]);
@@ -624,10 +627,13 @@ parse_sa_tokens(char **tokens, uint32_t n_tokens,
 	if (status->status < 0)
 		return;
 
-	if ((rule->type != RTE_SECURITY_ACTION_TYPE_NONE) && (portid_p == 0))
+	if ((rule->type != RTE_SECURITY_ACTION_TYPE_NONE && rule->type !=
+			RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) &&
+			(portid_p == 0))
 		printf("Missing portid option, falling back to non-offload\n");
 
-	if (!type_p || !portid_p) {
+	if (!type_p || (!portid_p && rule->type !=
+			RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)) {
 		rule->type = RTE_SECURITY_ACTION_TYPE_NONE;
 		rule->portid = -1;
 	}
@@ -709,6 +715,9 @@ print_one_sa_rule(const struct ipsec_sa *sa, int inbound)
 	case RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL:
 		printf("lookaside-protocol-offload ");
 		break;
+	case RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO:
+		printf("cpu-crypto-accelerated");
+		break;
 	}
 	printf("\n");
 }
examples/ipsec-secgw/test/run_test.sh +10 -0
index 8055a4c04..f322aa785 100755
@@ -32,15 +32,21 @@ usage()
 }
 
 LINUX_TEST="tun_aescbc_sha1 \
+tun_aescbc_sha1_cpu_crypto \
 tun_aescbc_sha1_esn \
 tun_aescbc_sha1_esn_atom \
 tun_aesgcm \
+tun_aesgcm_cpu_crypto \
+tun_aesgcm_mb_cpu_crypto \
 tun_aesgcm_esn \
 tun_aesgcm_esn_atom \
 trs_aescbc_sha1 \
+trs_aescbc_sha1_cpu_crypto \
 trs_aescbc_sha1_esn \
 trs_aescbc_sha1_esn_atom \
 trs_aesgcm \
+trs_aesgcm_cpu_crypto \
+trs_aesgcm_mb_cpu_crypto \
 trs_aesgcm_esn \
 trs_aesgcm_esn_atom \
 tun_aescbc_sha1_old \
@@ -49,17 +55,21 @@ trs_aescbc_sha1_old \
 trs_aesgcm_old \
 tun_aesctr_sha1 \
 tun_aesctr_sha1_old \
+tun_aesctr_cpu_crypto \
 tun_aesctr_sha1_esn \
 tun_aesctr_sha1_esn_atom \
 trs_aesctr_sha1 \
+trs_aesctr_sha1_cpu_crypto \
 trs_aesctr_sha1_old \
 trs_aesctr_sha1_esn \
 trs_aesctr_sha1_esn_atom \
 tun_3descbc_sha1 \
+tun_3descbc_sha1_cpu_crypto \
 tun_3descbc_sha1_old \
 tun_3descbc_sha1_esn \
 tun_3descbc_sha1_esn_atom \
 trs_3descbc_sha1 \
+trs_3descbc_sha1 \
 trs_3descbc_sha1_old \
 trs_3descbc_sha1_esn \
 trs_3descbc_sha1_esn_atom"
examples/ipsec-secgw/test/trs_3descbc_sha1_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..a864a8886
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/trs_3descbc_sha1_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/trs_aescbc_sha1_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..b515cd9f8
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/trs_aescbc_sha1_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/trs_aesctr_sha1_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..745a2a02b
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/trs_aesctr_sha1_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/trs_aesgcm_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..8917122da
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/trs_aesgcm_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/trs_aesgcm_mb_cpu_crypto_defs.sh new file +7 -0
new file mode 100644
index 000000000..26943321f
@@ -0,0 +1,7 @@
+#! /bin/bash
+
+. ${DIR}/trs_aesgcm_defs.sh
+
+CRYPTO_DEV=${CRYPTO_DEV:-'--vdev="crypto_aesni_mb0"'}
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/tun_3descbc_sha1_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..747141f62
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/tun_3descbc_sha1_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/tun_aescbc_sha1_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..56076fa50
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/tun_aescbc_sha1_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/tun_aesctr_sha1_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..3af680533
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/tun_aesctr_sha1_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/tun_aesgcm_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..5bf1c0ae5
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/tun_aesgcm_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/tun_aesgcm_mb_cpu_crypto_defs.sh new file +7 -0
new file mode 100644
index 000000000..039b8095e
@@ -0,0 +1,7 @@
+#! /bin/bash
+
+. ${DIR}/tun_aesgcm_defs.sh
+
+CRYPTO_DEV=${CRYPTO_DEV:-'--vdev="crypto_aesni_mb0"'}
+
+SGW_CFG_XPRM='type cpu-crypto'
Signature · 1 line
2.14.5
author
#69 lore raw

PATCH 10/10 doc: update security cpu process description

In reply to Fan Zhang

This patch updates programmer's guide and release note for newly added security cpu process description.

---
 doc/guides/cryptodevs/aesni_gcm.rst    |   6 ++
 doc/guides/cryptodevs/aesni_mb.rst     |   7 +++
 doc/guides/prog_guide/rte_security.rst | 112 ++++++++++++++++++++++++++++++++-
 doc/guides/rel_notes/release_19_11.rst |   7 +++
 4 files changed, 131 insertions(+), 1 deletion(-)
doc/guides/cryptodevs/aesni_gcm.rst +6 -0
index 9a8bc9323..31297fabd 100644
@@ -9,6 +9,12 @@ The AES-NI GCM PMD (**librte_pmd_aesni_gcm**) provides poll mode crypto driver
 support for utilizing Intel multi buffer library (see AES-NI Multi-buffer PMD documentation
 to learn more about it, including installation).
 
+The AES-NI GCM PMD also supports rte_security with security session create
+and ``rte_security_process_cpu_crypto_bulk`` function call to process
+symmetric crypto synchronously with all algorithms specified below. With this
+way it supports scather-gather buffers (``rte_security_vec`` can be greater than
+``1``. Please refer to ``rte_security`` programmer's guide for more detail.
+
 Features
 --------
 
doc/guides/cryptodevs/aesni_mb.rst +7 -0
index 1eff2b073..1a3ddd850 100644
@@ -12,6 +12,13 @@ support for utilizing Intel multi buffer library, see the white paper
 
 The AES-NI MB PMD has current only been tested on Fedora 21 64-bit with gcc.
 
+The AES-NI MB PMD also supports rte_security with security session create
+and ``rte_security_process_cpu_crypto_bulk`` function call to process
+symmetric crypto synchronously with all algorithms specified below. However
+it does not support scather-gather buffer so the ``num`` value in
+``rte_security_vec`` can only be ``1``. Please refer to ``rte_security``
+programmer's guide for more detail.
+
 Features
 --------
 
doc/guides/prog_guide/rte_security.rst +111 -1

141 lines not on this page · raw message

doc/guides/rel_notes/release_19_11.rst +7 -0
index 8490d897c..6cd21704f 100644
@@ -56,6 +56,13 @@ New Features
      Also, make sure to start the actual text at the margin.
      =========================================================
 
+* **RTE_SECURITY is added new synchronous Crypto burst API with CPU**
+
+  A new API rte_security_process_cpu_crypto_bulk is introduced in security
+  library to process crypto workload in bulk using CPU instructions. AESNI_MB
+  and AESNI_GCM PMD, as well as unit-test and ipsec-secgw sample applications
+  are updated to support this feature.
+
 
 Removed Items
 -------------
Signature · 1 line
2.14.5
#70 lore raw
In reply to Fan Zhang
Quoted 38 lines · Fan Zhang

This RFC patch adds a way to rte_security to process symmetric crypto workload in bulk synchronously for SW crypto devices.

Originally both SW and HW crypto PMDs works under rte_cryptodev to process the crypto workload asynchronously. This way provides uniformity to both PMD types but also introduce unnecessary performance penalty to SW PMDs such as extra SW ring enqueue/dequeue steps to "simulate" asynchronous working manner and unnecessary HW addresses computation.

We introduce a new way for SW crypto devices that perform crypto operation synchronously with only fields required for the computation as input.

In rte_security, a new action type "RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO" is introduced. This action type allows the burst of symmetric crypto workload using the same algorithm, key, and direction being processed by CPU cycles synchronously. This flexible action type does not require external hardware involvement.

This patch also includes the announcement of a new API "rte_security_process_cpu_crypto_bulk". With this API the packet is sent to the crypto device for symmetric crypto processing. The device will encrypt or decrypt the buffer based on the session data specified and preprocessed in the security session. Different than the inline or lookaside modes, when the function exits, the user will expect the buffers are either processed successfully, or having the error number assigned to the appropriate index of the status array.

The proof-of-concept AESNI-GCM and AESNI-MB SW PMDs are updated with the support of this new method. To demonstrate the performance gain with this method 2 simple performance evaluation apps under unit-test are added "app/test: security_aesni_gcm_perftest/security_aesni_mb_perftest". The users can freely compare their results against crypto perf application results.

In the end, the ipsec library and ipsec-secgw sample application are also updated to support this feature. Several test scripts are added to the ipsec-secgw test-suite to prove the correctness of the implementation.

Quoted 12 lines

Fan Zhang (10):
security: introduce CPU Crypto action type and API crypto/aesni_gcm: add rte_security handler app/test: add security cpu crypto autotest app/test: add security cpu crypto perftest crypto/aesni_mb: add rte_security handler app/test: add aesni_mb security cpu crypto autotest app/test: add aesni_mb security cpu crypto perftest
ipsec: add rte_security cpu_crypto action support examples/ipsec-secgw: add security cpu_crypto action support
doc: update security cpu process description

Hi Fan,

This series has problem on aarch64:

../app/test/test_security_cpu_crypto.c:626:16: error: implicit declaration of function ‘rte_get_tsc_hz’ [-Werror=implicit-function-declaration]

     uint64_t hz = rte_get_tsc_hz(), time_start, time_now;
                   ^
   ../app/test/test_security_cpu_crypto.c:679:16: error: implicit declaration of function ‘rte_rdtsc’ [-Werror=implicit-function-declaration]
      time_start = rte_rdtsc();
                   ^
   ../app/test/test_security_cpu_crypto.c:711:16: error: implicit declaration of function ‘rte_get_timer_cycles’ [-Werror=implicit-function-declaration]
      time_start = rte_get_timer_cycles();
                   ^

I'm not sure best way to address this in the test - maybe there's a better API to use for getting the cycles?

author
#71 lore raw

v2 cover security: add software synchronous crypto process

In reply to Fan Zhang

This RFC patch adds a way to rte_security to process symmetric crypto workload in bulk synchronously for SW crypto devices.

Originally both SW and HW crypto PMDs works under rte_cryptodev to process the crypto workload asynchronously. This way provides uniformity to both PMD types but also introduce unnecessary performance penalty to SW PMDs such as extra SW ring enqueue/dequeue steps to "simulate" asynchronous working manner and unnecessary HW addresses computation.

We introduce a new way for SW crypto devices that perform crypto operation synchronously with only fields required for the computation as input.

In rte_security, a new action type "RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO" is introduced. This action type allows the burst of symmetric crypto workload using the same algorithm, key, and direction being processed by CPU cycles synchronously. This flexible action type does not require external hardware involvement.

This patch also includes the announcement of a new API "rte_security_process_cpu_crypto_bulk". With this API the packet is sent to the crypto device for symmetric crypto processing. The device will encrypt or decrypt the buffer based on the session data specified and preprocessed in the security session. Different than the inline or lookaside modes, when the function exits, the user will expect the buffers are either processed successfully, or having the error number assigned to the appropriate index of the status array.

The proof-of-concept AESNI-GCM and AESNI-MB SW PMDs are updated with the support of this new method. To demonstrate the performance gain with this method 2 simple performance evaluation apps under unit-test are added "app/test: security_aesni_gcm_perftest/security_aesni_mb_perftest". The users can freely compare their results against crypto perf application results.

In the end, the ipsec library and ipsec-secgw sample application are also updated to support this feature. Several test scripts are added to the ipsec-secgw test-suite to prove the correctness of the implementation.

v2:
- changed API return from "void" to "int"
- rework on ipsec library implementation.
- fixed bugs in aesni-mb PMD.
- fixed bugs in ipsec-secgw application.

Fan Zhang (10):
security: introduce CPU Crypto action type and API crypto/aesni_gcm: add rte_security handler app/test: add security cpu crypto autotest app/test: add security cpu crypto perftest crypto/aesni_mb: add rte_security handler app/test: add aesni_mb security cpu crypto autotest app/test: add aesni_mb security cpu crypto perftest
ipsec: add rte_security cpu_crypto action support examples/ipsec-secgw: add security cpu_crypto action support
doc: update security cpu process description

 app/test/Makefile                                  |    1 +
 app/test/meson.build                               |    1 +
 app/test/test_security_cpu_crypto.c                | 1326 ++++++++++++++++++++
 doc/guides/cryptodevs/aesni_gcm.rst                |    6 +
 doc/guides/cryptodevs/aesni_mb.rst                 |    7 +
 doc/guides/prog_guide/rte_security.rst             |  112 +-
 doc/guides/rel_notes/release_19_11.rst             |    7 +
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c           |   97 +-
 drivers/crypto/aesni_gcm/aesni_gcm_pmd_ops.c       |   95 ++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h   |   23 +
 drivers/crypto/aesni_gcm/meson.build               |    2 +-
 drivers/crypto/aesni_mb/meson.build                |    2 +-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c         |  368 +++++-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c     |   92 +-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h |   21 +-
 examples/ipsec-secgw/ipsec.c                       |   35 +
 examples/ipsec-secgw/ipsec_process.c               |    7 +-
 examples/ipsec-secgw/sa.c                          |   13 +-
 examples/ipsec-secgw/test/run_test.sh              |   10 +
 .../test/trs_3descbc_sha1_common_defs.sh           |    8 +-
 .../test/trs_3descbc_sha1_cpu_crypto_defs.sh       |    5 +
 .../test/trs_aescbc_sha1_common_defs.sh            |    8 +-
 .../test/trs_aescbc_sha1_cpu_crypto_defs.sh        |    5 +
 .../test/trs_aesctr_sha1_common_defs.sh            |    8 +-
 .../test/trs_aesctr_sha1_cpu_crypto_defs.sh        |    5 +
 .../ipsec-secgw/test/trs_aesgcm_cpu_crypto_defs.sh |    5 +
 .../test/trs_aesgcm_mb_cpu_crypto_defs.sh          |    7 +
 .../test/tun_3descbc_sha1_common_defs.sh           |    8 +-
 .../test/tun_3descbc_sha1_cpu_crypto_defs.sh       |    5 +
 .../test/tun_aescbc_sha1_common_defs.sh            |    8 +-
 .../test/tun_aescbc_sha1_cpu_crypto_defs.sh        |    5 +
 .../test/tun_aesctr_sha1_common_defs.sh            |    8 +-
 .../test/tun_aesctr_sha1_cpu_crypto_defs.sh        |    5 +
 .../ipsec-secgw/test/tun_aesgcm_cpu_crypto_defs.sh |    5 +
 .../test/tun_aesgcm_mb_cpu_crypto_defs.sh          |    7 +
 lib/librte_ipsec/crypto.h                          |   24 +
 lib/librte_ipsec/esp_inb.c                         |  200 ++-
 lib/librte_ipsec/esp_outb.c                        |  369 +++++-
 lib/librte_ipsec/sa.c                              |   53 +-
 lib/librte_ipsec/sa.h                              |   29 +
 lib/librte_ipsec/ses.c                             |    4 +-
 lib/librte_security/rte_security.c                 |   11 +
 lib/librte_security/rte_security.h                 |   53 +-
 lib/librte_security/rte_security_driver.h          |   22 +
 lib/librte_security/rte_security_version.map       |    1 +
 45 files changed, 2994 insertions(+), 99 deletions(-)
 create mode 100644 app/test/test_security_cpu_crypto.c
 create mode 100644 examples/ipsec-secgw/test/trs_3descbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aescbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aesctr_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_mb_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_3descbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aescbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aesctr_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_mb_cpu_crypto_defs.sh
Signature · 1 line
2.14.5
author
#72 lore raw

PATCH v2 1/10 security: introduce CPU Crypto action type and API

This patch introduce new RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO action type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for PMDs.

---
 lib/librte_security/rte_security.c           | 11 ++++++
 lib/librte_security/rte_security.h           | 53 +++++++++++++++++++++++++++-
 lib/librte_security/rte_security_driver.h    | 22 ++++++++++++
 lib/librte_security/rte_security_version.map |  1 +
 4 files changed, 86 insertions(+), 1 deletion(-)
lib/librte_security/rte_security.c +11 -0
index bc81ce15d..cdd1ee6af 100644
@@ -141,3 +141,14 @@ rte_security_capability_get(struct rte_security_ctx *instance,
 
 	return NULL;
 }
+
+int
+rte_security_process_cpu_crypto_bulk(struct rte_security_ctx *instance,
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num)
+{
+	RTE_FUNC_PTR_OR_ERR_RET(*instance->ops->process_cpu_crypto_bulk, -1);
+	return instance->ops->process_cpu_crypto_bulk(sess, buf, iv,
+			aad, digest, status, num);
+}
lib/librte_security/rte_security.h +52 -1

86 lines not on this page · raw message

lib/librte_security/rte_security_driver.h +22 -0
index 1b561f852..fe940fffa 100644
@@ -132,6 +132,26 @@ typedef int (*security_get_userdata_t)(void *device,
 typedef const struct rte_security_capability *(*security_capabilities_get_t)(
 		void *device);
 
+/**
+ * Process security operations in bulk using CPU accelerated method.
+ *
+ * @param	sess		Security session structure.
+ * @param	buf		Buffer to the vectors to be processed.
+ * @param	iv		IV pointers.
+ * @param	aad		AAD pointers.
+ * @param	digest		Digest pointers.
+ * @param	status		Array of status value.
+ * @param	num		Number of elements in each array.
+ * @return
+ *  - On success, 0
+ *  - On any failure, -1
+ */
+
+typedef int (*security_process_cpu_crypto_bulk_t)(
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+
 /** Security operations function pointer table */
 struct rte_security_ops {
 	security_session_create_t session_create;
@@ -150,6 +170,8 @@ struct rte_security_ops {
 	/**< Get userdata associated with session which processed the packet. */
 	security_capabilities_get_t capabilities_get;
 	/**< Get security capabilities. */
+	security_process_cpu_crypto_bulk_t process_cpu_crypto_bulk;
+	/**< Process data in bulk. */
 };
 
 #ifdef __cplusplus
lib/librte_security/rte_security_version.map +1 -0
index 53267bf3c..2132e7a00 100644
@@ -18,4 +18,5 @@ EXPERIMENTAL {
 	rte_security_get_userdata;
 	rte_security_session_stats_get;
 	rte_security_session_update;
+	rte_security_process_cpu_crypto_bulk;
 };
Signature · 1 line
2.14.5
#73 lore raw

Re: [dpdk-dev] [PATCH v2 01/10] security: introduce CPU Crypto action type and API

Hi Fan,

Quoted 107 lines

This patch introduce new RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO action type to security library. The type represents performing crypto operation with CPU cycles. The patch also includes a new API to process crypto operations in bulk and the function pointers for PMDs.

---
 lib/librte_security/rte_security.c           | 11 ++++++
 lib/librte_security/rte_security.h           | 53 +++++++++++++++++++++++++++-
 lib/librte_security/rte_security_driver.h    | 22 ++++++++++++
 lib/librte_security/rte_security_version.map |  1 +
 4 files changed, 86 insertions(+), 1 deletion(-)
lib/librte_security/rte_security.c +11 -0
index bc81ce15d..cdd1ee6af 100644
@@ -141,3 +141,14 @@ rte_security_capability_get(struct rte_security_ctx *instance,
 
 	return NULL;
 }
+
+int
+rte_security_process_cpu_crypto_bulk(struct rte_security_ctx *instance,
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num)
+{
+	RTE_FUNC_PTR_OR_ERR_RET(*instance->ops->process_cpu_crypto_bulk, -1);
+	return instance->ops->process_cpu_crypto_bulk(sess, buf, iv,
+			aad, digest, status, num);
+}
lib/librte_security/rte_security.h +34 -1
index aaafdfcd7..0caf5d697 100644
@@ -18,6 +18,7 @@ extern "C" {
 #endif
 
 #include <sys/types.h>
+#include <sys/uio.h>
 
 #include <netinet/in.h>
 #include <netinet/ip.h>
@@ -289,6 +290,20 @@ struct rte_security_pdcp_xform {
 	uint32_t hfn_ovrd;
 };
 
+struct rte_security_cpu_crypto_xform {
+	/** For cipher/authentication crypto operation the authentication may
+	 * cover more content then the cipher. E.g., for IPSec ESP encryption
+	 * with AES-CBC and SHA1-HMAC, the encryption happens after the ESP
+	 * header but whole packet (apart from MAC header) is authenticated.
+	 * The cipher_offset field is used to deduct the cipher data pointer
+	 * from the buffer to be processed.
+	 *
+	 * NOTE this parameter shall be ignored by AEAD algorithms, since it
+	 * uses the same offset for cipher and authentication.
+	 */
+	int32_t cipher_offset;
+};
+
 /**
  * Security session action type.
  */
@@ -303,10 +318,14 @@ enum rte_security_session_action_type {
 	/**< All security protocol processing is performed inline during
 	 * transmission
 	 */
-	RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL
+	RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL,
 	/**< All security protocol processing including crypto is performed
 	 * on a lookaside accelerator
 	 */
+	RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO
+	/**< Crypto processing for security protocol is processed by CPU
+	 * synchronously
+	 */
 };
 
 /** Security session protocol definition */
@@ -332,6 +351,7 @@ struct rte_security_session_conf {
 		struct rte_security_ipsec_xform ipsec;
 		struct rte_security_macsec_xform macsec;
 		struct rte_security_pdcp_xform pdcp;
+		struct rte_security_cpu_crypto_xform cpucrypto;
 	};
 	/**< Configuration parameters for security session */
 	struct rte_crypto_sym_xform *crypto_xform;
@@ -665,6 +685,37 @@ const struct rte_security_capability *
 rte_security_capability_get(struct rte_security_ctx *instance,
 			    struct rte_security_capability_idx *idx);
 
+/**
+ * Security vector structure, contains pointer to vector array and the length
+ * of the array
+ */
+struct rte_security_vec {
+	struct iovec *vec;
+	uint32_t num;
+};
+
+/**
+ * Processing bulk crypto workload with CPU
+ *
+ * @param	instance	security instance.

lib/librte_security/rte_security.h const struct rte_security_capability * · line 710

Quoted 10 lines
lib/librte_security/rte_security.h +10 -0
+ * @param	sess		security session
+ * @param	buf		array of buffer SGL vectors
+ * @param	iv		array of IV pointers
+ * @param	aad		array of AAD pointers
+ * @param	digest		array of digest pointers
+ * @param	status		array of status for the function to return
+ * @param	num		number of elements in each array
+ * @return
+ *  - On success, 0
+ *  - On any failure, -1

I think it is much better to retrun number of successfully process entries (or number of failed entries - whatever is your preference). Then user can easily determine does he need to walk through status (and if yes till what point) or not at all.
Sorry if I wasn't clear in my previous comment.

Quoted 63 lines
lib/librte_security/rte_security.h +8 -0
+ */
+__rte_experimental
+int
+rte_security_process_cpu_crypto_bulk(struct rte_security_ctx *instance,
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+
 #ifdef __cplusplus
 }
 #endif
lib/librte_security/rte_security_driver.h +22 -0
index 1b561f852..fe940fffa 100644
@@ -132,6 +132,26 @@ typedef int (*security_get_userdata_t)(void *device,
 typedef const struct rte_security_capability *(*security_capabilities_get_t)(
 		void *device);
 
+/**
+ * Process security operations in bulk using CPU accelerated method.
+ *
+ * @param	sess		Security session structure.
+ * @param	buf		Buffer to the vectors to be processed.
+ * @param	iv		IV pointers.
+ * @param	aad		AAD pointers.
+ * @param	digest		Digest pointers.
+ * @param	status		Array of status value.
+ * @param	num		Number of elements in each array.
+ * @return
+ *  - On success, 0
+ *  - On any failure, -1
+ */
+
+typedef int (*security_process_cpu_crypto_bulk_t)(
+		struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+
 /** Security operations function pointer table */
 struct rte_security_ops {
 	security_session_create_t session_create;
@@ -150,6 +170,8 @@ struct rte_security_ops {
 	/**< Get userdata associated with session which processed the packet. */
 	security_capabilities_get_t capabilities_get;
 	/**< Get security capabilities. */
+	security_process_cpu_crypto_bulk_t process_cpu_crypto_bulk;
+	/**< Process data in bulk. */
 };
 
 #ifdef __cplusplus
lib/librte_security/rte_security_version.map +1 -1
index 53267bf3c..2132e7a00 100644
@@ -18,4 +18,5 @@ EXPERIMENTAL {
 	rte_security_get_userdata;
 	rte_security_session_stats_get;
 	rte_security_session_update;
+	rte_security_process_cpu_crypto_bulk;
 };
--

2.14.5

author
#74 lore raw

PATCH v2 2/10 crypto/aesni_gcm: add rte_security handler

In reply to Fan Zhang

This patch add rte_security support support to AESNI-GCM PMD. The PMD now initialize security context instance, create/delete PMD specific security sessions, and process crypto workloads in synchronous mode with scatter-gather list buffer supported.

---
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c         | 97 +++++++++++++++++++++++-
 drivers/crypto/aesni_gcm/aesni_gcm_pmd_ops.c     | 95 +++++++++++++++++++++++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h | 23 ++++++
 drivers/crypto/aesni_gcm/meson.build             |  2 +-
 4 files changed, 215 insertions(+), 2 deletions(-)
drivers/crypto/aesni_gcm/aesni_gcm_pmd.c +96 -1
index 1006a5c4d..2e91bf149 100644
@@ -6,6 +6,7 @@
 #include <rte_hexdump.h>
 #include <rte_cryptodev.h>
 #include <rte_cryptodev_pmd.h>
+#include <rte_security_driver.h>
 #include <rte_bus_vdev.h>
 #include <rte_malloc.h>
 #include <rte_cpuflags.h>
@@ -174,6 +175,56 @@ aesni_gcm_get_session(struct aesni_gcm_qp *qp, struct rte_crypto_op *op)
 	return sess;
 }
 
+static __rte_always_inline int
+process_gcm_security_sgl_buf(struct aesni_gcm_security_session *sess,
+		struct rte_security_vec *buf, uint8_t *iv,
+		uint8_t *aad, uint8_t *digest)
+{
+	struct aesni_gcm_session *session = &sess->sess;
+	uint8_t *tag;
+	uint32_t i;
+
+	sess->init(&session->gdata_key, &sess->gdata_ctx, iv, aad,
+			(uint64_t)session->aad_length);
+
+	for (i = 0; i < buf->num; i++) {
+		struct iovec *vec = &buf->vec[i];
+
+		sess->update(&session->gdata_key, &sess->gdata_ctx,
+				vec->iov_base, vec->iov_base, vec->iov_len);
+	}
+
+	switch (session->op) {
+	case AESNI_GCM_OP_AUTHENTICATED_ENCRYPTION:
+		if (session->req_digest_length != session->gen_digest_length)
+			tag = sess->temp_digest;
+		else
+			tag = digest;
+
+		sess->finalize(&session->gdata_key, &sess->gdata_ctx, tag,
+				session->gen_digest_length);
+
+		if (session->req_digest_length != session->gen_digest_length)
+			memcpy(digest, sess->temp_digest,
+					session->req_digest_length);
+		break;
+
+	case AESNI_GCM_OP_AUTHENTICATED_DECRYPTION:
+		tag = sess->temp_digest;
+
+		sess->finalize(&session->gdata_key, &sess->gdata_ctx, tag,
+				session->gen_digest_length);
+
+		if (memcmp(tag, digest,	session->req_digest_length) != 0)
+			return -1;
+		break;
+	default:
+		return -1;
+	}
+
+	return 0;
+}
+
 /**
  * Process a crypto operation, calling
  * the GCM API from the multi buffer library.
@@ -488,8 +539,10 @@ aesni_gcm_create(const char *name,
 {
 	struct rte_cryptodev *dev;
 	struct aesni_gcm_private *internals;
+	struct rte_security_ctx *sec_ctx;
 	enum aesni_gcm_vector_mode vector_mode;
 	MB_MGR *mb_mgr;
+	char sec_name[RTE_DEV_NAME_MAX_LEN];
 
 	/* Check CPU for support for AES instruction set */
 	if (!rte_cpu_get_flag_enabled(RTE_CPUFLAG_AES)) {
@@ -524,7 +577,8 @@ aesni_gcm_create(const char *name,
 			RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
 			RTE_CRYPTODEV_FF_CPU_AESNI |
 			RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT |
-			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT;
+			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
+			RTE_CRYPTODEV_FF_SECURITY;
 
 	mb_mgr = alloc_mb_mgr(0);
 	if (mb_mgr == NULL)
@@ -587,6 +641,21 @@ aesni_gcm_create(const char *name,
 
 	internals->max_nb_queue_pairs = init_params->max_nb_queue_pairs;
 
+	/* setup security operations */
+	snprintf(sec_name, sizeof(sec_name) - 1, "aes_gcm_sec_%u",
+			dev->driver_id);
+	sec_ctx = rte_zmalloc_socket(sec_name,
+			sizeof(struct rte_security_ctx),
+			RTE_CACHE_LINE_SIZE, init_params->socket_id);
+	if (sec_ctx == NULL) {
+		AESNI_GCM_LOG(ERR, "memory allocation failed\n");
+		goto error_exit;
+	}
+
+	sec_ctx->device = (void *)dev;
+	sec_ctx->ops = rte_aesni_gcm_pmd_security_ops;
+	dev->security_ctx = sec_ctx;
+
 #if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
 	AESNI_GCM_LOG(INFO, "IPSec Multi-buffer library version used: %s\n",
 			imb_get_version_str());
@@ -641,6 +710,8 @@ aesni_gcm_remove(struct rte_vdev_device *vdev)
 	if (cryptodev == NULL)
 		return -ENODEV;
 
+	rte_free(cryptodev->security_ctx);
+
 	internals = cryptodev->data->dev_private;
 
 	free_mb_mgr(internals->mb_mgr);
@@ -648,6 +719,30 @@ aesni_gcm_remove(struct rte_vdev_device *vdev)
 	return rte_cryptodev_pmd_destroy(cryptodev);
 }
 
+int
+aesni_gcm_sec_crypto_process_bulk(struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num)
+{
+	struct aesni_gcm_security_session *session =
+			get_sec_session_private_data(sess);
+	uint32_t i;
+	int errcnt = 0;
+
+	if (unlikely(!session))
+		return -num;
+
+	for (i = 0; i < num; i++) {
+		status[i] = process_gcm_security_sgl_buf(session, &buf[i],
+				(uint8_t *)iv[i], (uint8_t *)aad[i],
+				(uint8_t *)digest[i]);
+		if (unlikely(status[i]))
+			errcnt -= 1;
+	}
+
+	return errcnt;
+}
+
 static struct rte_vdev_driver aesni_gcm_pmd_drv = {
 	.probe = aesni_gcm_probe,
 	.remove = aesni_gcm_remove
drivers/crypto/aesni_gcm/aesni_gcm_pmd_ops.c +95 -0

111 lines not on this page · raw message

drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h +23 -0
index 56b29e013..ed3f6eb2e 100644
@@ -114,5 +114,28 @@ aesni_gcm_set_session_parameters(const struct aesni_gcm_ops *ops,
  * Device specific operations function pointer structure */
 extern struct rte_cryptodev_ops *rte_aesni_gcm_pmd_ops;
 
+/**
+ * Security session structure.
+ */
+struct aesni_gcm_security_session {
+	/** Temp digest for decryption */
+	uint8_t temp_digest[DIGEST_LENGTH_MAX];
+	/** GCM operations */
+	aesni_gcm_pre_t pre;
+	aesni_gcm_init_t init;
+	aesni_gcm_update_t update;
+	aesni_gcm_finalize_t finalize;
+	/** AESNI-GCM session */
+	struct aesni_gcm_session sess;
+	/** AESNI-GCM context */
+	struct gcm_context_data gdata_ctx;
+};
+
+extern int
+aesni_gcm_sec_crypto_process_bulk(struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+
+extern struct rte_security_ops *rte_aesni_gcm_pmd_security_ops;
 
 #endif /* _RTE_AESNI_GCM_PMD_PRIVATE_H_ */
drivers/crypto/aesni_gcm/meson.build +1 -1
index 3a6e332dc..f6e160bb3 100644
@@ -22,4 +22,4 @@ endif
 
 allow_experimental_apis = true
 sources = files('aesni_gcm_pmd.c', 'aesni_gcm_pmd_ops.c')
-deps += ['bus_vdev']
+deps += ['bus_vdev', 'security']
Signature · 1 line
2.14.5
#75 lore raw

Re: [dpdk-dev] [PATCH v2 02/10] crypto/aesni_gcm: add rte_security handler

Quoted 143 lines

This patch add rte_security support support to AESNI-GCM PMD. The PMD now initialize security context instance, create/delete PMD specific security sessions, and process crypto workloads in synchronous mode with scatter-gather list buffer supported.

---
 drivers/crypto/aesni_gcm/aesni_gcm_pmd.c         | 97 +++++++++++++++++++++++-
 drivers/crypto/aesni_gcm/aesni_gcm_pmd_ops.c     | 95 +++++++++++++++++++++++
 drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h | 23 ++++++
 drivers/crypto/aesni_gcm/meson.build             |  2 +-
 4 files changed, 215 insertions(+), 2 deletions(-)
drivers/crypto/aesni_gcm/aesni_gcm_pmd.c +74 -1
index 1006a5c4d..2e91bf149 100644
@@ -6,6 +6,7 @@
 #include <rte_hexdump.h>
 #include <rte_cryptodev.h>
 #include <rte_cryptodev_pmd.h>
+#include <rte_security_driver.h>
 #include <rte_bus_vdev.h>
 #include <rte_malloc.h>
 #include <rte_cpuflags.h>
@@ -174,6 +175,56 @@ aesni_gcm_get_session(struct aesni_gcm_qp *qp, struct rte_crypto_op *op)
 	return sess;
 }
 
+static __rte_always_inline int
+process_gcm_security_sgl_buf(struct aesni_gcm_security_session *sess,
+		struct rte_security_vec *buf, uint8_t *iv,
+		uint8_t *aad, uint8_t *digest)
+{
+	struct aesni_gcm_session *session = &sess->sess;
+	uint8_t *tag;
+	uint32_t i;
+
+	sess->init(&session->gdata_key, &sess->gdata_ctx, iv, aad,
+			(uint64_t)session->aad_length);
+
+	for (i = 0; i < buf->num; i++) {
+		struct iovec *vec = &buf->vec[i];
+
+		sess->update(&session->gdata_key, &sess->gdata_ctx,
+				vec->iov_base, vec->iov_base, vec->iov_len);
+	}
+
+	switch (session->op) {
+	case AESNI_GCM_OP_AUTHENTICATED_ENCRYPTION:
+		if (session->req_digest_length != session->gen_digest_length)
+			tag = sess->temp_digest;
+		else
+			tag = digest;
+
+		sess->finalize(&session->gdata_key, &sess->gdata_ctx, tag,
+				session->gen_digest_length);
+
+		if (session->req_digest_length != session->gen_digest_length)
+			memcpy(digest, sess->temp_digest,
+					session->req_digest_length);
+		break;
+
+	case AESNI_GCM_OP_AUTHENTICATED_DECRYPTION:
+		tag = sess->temp_digest;
+
+		sess->finalize(&session->gdata_key, &sess->gdata_ctx, tag,
+				session->gen_digest_length);
+
+		if (memcmp(tag, digest,	session->req_digest_length) != 0)
+			return -1;
+		break;
+	default:
+		return -1;
+	}
+
+	return 0;
+}
+
 /**
  * Process a crypto operation, calling
  * the GCM API from the multi buffer library.
@@ -488,8 +539,10 @@ aesni_gcm_create(const char *name,
 {
 	struct rte_cryptodev *dev;
 	struct aesni_gcm_private *internals;
+	struct rte_security_ctx *sec_ctx;
 	enum aesni_gcm_vector_mode vector_mode;
 	MB_MGR *mb_mgr;
+	char sec_name[RTE_DEV_NAME_MAX_LEN];
 
 	/* Check CPU for support for AES instruction set */
 	if (!rte_cpu_get_flag_enabled(RTE_CPUFLAG_AES)) {
@@ -524,7 +577,8 @@ aesni_gcm_create(const char *name,
 			RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
 			RTE_CRYPTODEV_FF_CPU_AESNI |
 			RTE_CRYPTODEV_FF_OOP_SGL_IN_LB_OUT |
-			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT;
+			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
+			RTE_CRYPTODEV_FF_SECURITY;
 
 	mb_mgr = alloc_mb_mgr(0);
 	if (mb_mgr == NULL)
@@ -587,6 +641,21 @@ aesni_gcm_create(const char *name,
 
 	internals->max_nb_queue_pairs = init_params->max_nb_queue_pairs;
 
+	/* setup security operations */
+	snprintf(sec_name, sizeof(sec_name) - 1, "aes_gcm_sec_%u",
+			dev->driver_id);
+	sec_ctx = rte_zmalloc_socket(sec_name,
+			sizeof(struct rte_security_ctx),
+			RTE_CACHE_LINE_SIZE, init_params->socket_id);
+	if (sec_ctx == NULL) {
+		AESNI_GCM_LOG(ERR, "memory allocation failed\n");
+		goto error_exit;
+	}
+
+	sec_ctx->device = (void *)dev;
+	sec_ctx->ops = rte_aesni_gcm_pmd_security_ops;
+	dev->security_ctx = sec_ctx;
+
 #if IMB_VERSION_NUM >= IMB_VERSION(0, 50, 0)
 	AESNI_GCM_LOG(INFO, "IPSec Multi-buffer library version used: %s\n",
 			imb_get_version_str());
@@ -641,6 +710,8 @@ aesni_gcm_remove(struct rte_vdev_device *vdev)
 	if (cryptodev == NULL)
 		return -ENODEV;
 
+	rte_free(cryptodev->security_ctx);
+
 	internals = cryptodev->data->dev_private;
 
 	free_mb_mgr(internals->mb_mgr);
@@ -648,6 +719,30 @@ aesni_gcm_remove(struct rte_vdev_device *vdev)
 	return rte_cryptodev_pmd_destroy(cryptodev);
 }
 
+int
+aesni_gcm_sec_crypto_process_bulk(struct rte_security_session *sess,

drivers/crypto/aesni_gcm/aesni_gcm_pmd.c aesni_gcm_remove(struct rte_vdev_device *vdev) · line 733

Quoted 10 lines
drivers/crypto/aesni_gcm/aesni_gcm_pmd.c +10 -0
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num)
+{
+	struct aesni_gcm_security_session *session =
+			get_sec_session_private_data(sess);
+	uint32_t i;
+	int errcnt = 0;
+
+	if (unlikely(!session))
+		return -num;

You return negative status (error), but don't send each status[] value.

Quoted 178 lines
drivers/crypto/aesni_gcm/aesni_gcm_pmd.c +12 -0
+
+	for (i = 0; i < num; i++) {
+		status[i] = process_gcm_security_sgl_buf(session, &buf[i],
+				(uint8_t *)iv[i], (uint8_t *)aad[i],
+				(uint8_t *)digest[i]);
+		if (unlikely(status[i]))
+			errcnt -= 1;
+	}
+
+	return errcnt;
+}
+
 static struct rte_vdev_driver aesni_gcm_pmd_drv = {
 	.probe = aesni_gcm_probe,
 	.remove = aesni_gcm_remove
drivers/crypto/aesni_gcm/aesni_gcm_pmd_ops.c +95 -0
index 2f66c7c58..cc71dbd60 100644
@@ -7,6 +7,7 @@
 #include <rte_common.h>
 #include <rte_malloc.h>
 #include <rte_cryptodev_pmd.h>
+#include <rte_security_driver.h>
 
 #include "aesni_gcm_pmd_private.h"
 
@@ -316,6 +317,85 @@ aesni_gcm_pmd_sym_session_clear(struct rte_cryptodev *dev,
 	}
 }
 
+static int
+aesni_gcm_security_session_create(void *dev,
+		struct rte_security_session_conf *conf,
+		struct rte_security_session *sess,
+		struct rte_mempool *mempool)
+{
+	struct rte_cryptodev *cdev = dev;
+	struct aesni_gcm_private *internals = cdev->data->dev_private;
+	struct aesni_gcm_security_session *sess_priv;
+	int ret;
+
+	if (!conf->crypto_xform) {
+		AESNI_GCM_LOG(ERR, "Invalid security session conf");
+		return -EINVAL;
+	}
+
+	if (conf->crypto_xform->type == RTE_CRYPTO_SYM_XFORM_AUTH) {
+		AESNI_GCM_LOG(ERR, "GMAC is not supported in security session");
+		return -EINVAL;
+	}
+
+
+	if (rte_mempool_get(mempool, (void **)(&sess_priv))) {
+		AESNI_GCM_LOG(ERR,
+				"Couldn't get object from session mempool");
+		return -ENOMEM;
+	}
+
+	ret = aesni_gcm_set_session_parameters(internals->ops,
+				&sess_priv->sess, conf->crypto_xform);
+	if (ret != 0) {
+		AESNI_GCM_LOG(ERR, "Failed configure session parameters");
+
+		/* Return session to mempool */
+		rte_mempool_put(mempool, (void *)sess_priv);
+		return ret;
+	}
+
+	sess_priv->pre = internals->ops[sess_priv->sess.key].pre;
+	sess_priv->init = internals->ops[sess_priv->sess.key].init;
+	if (sess_priv->sess.op == AESNI_GCM_OP_AUTHENTICATED_ENCRYPTION) {
+		sess_priv->update =
+			internals->ops[sess_priv->sess.key].update_enc;
+		sess_priv->finalize =
+			internals->ops[sess_priv->sess.key].finalize_enc;
+	} else {
+		sess_priv->update =
+			internals->ops[sess_priv->sess.key].update_dec;
+		sess_priv->finalize =
+			internals->ops[sess_priv->sess.key].finalize_dec;
+	}
+
+	sess->sess_private_data = sess_priv;
+
+	return 0;
+}
+
+static int
+aesni_gcm_security_session_destroy(void *dev __rte_unused,
+		struct rte_security_session *sess)
+{
+	void *sess_priv = get_sec_session_private_data(sess);
+
+	if (sess_priv) {
+		struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv);
+
+		memset(sess, 0, sizeof(struct aesni_gcm_security_session));
+		set_sec_session_private_data(sess, NULL);
+		rte_mempool_put(sess_mp, sess_priv);
+	}
+	return 0;
+}
+
+static unsigned int
+aesni_gcm_sec_session_get_size(__rte_unused void *device)
+{
+	return sizeof(struct aesni_gcm_security_session);
+}
+
 struct rte_cryptodev_ops aesni_gcm_pmd_ops = {
 		.dev_configure		= aesni_gcm_pmd_config,
 		.dev_start		= aesni_gcm_pmd_start,
@@ -336,4 +416,19 @@ struct rte_cryptodev_ops aesni_gcm_pmd_ops = {
 		.sym_session_clear	= aesni_gcm_pmd_sym_session_clear
 };
 
+static struct rte_security_ops aesni_gcm_security_ops = {
+		.session_create = aesni_gcm_security_session_create,
+		.session_get_size = aesni_gcm_sec_session_get_size,
+		.session_update = NULL,
+		.session_stats_get = NULL,
+		.session_destroy = aesni_gcm_security_session_destroy,
+		.set_pkt_metadata = NULL,
+		.capabilities_get = NULL,
+		.process_cpu_crypto_bulk =
+				aesni_gcm_sec_crypto_process_bulk,
+};
+
 struct rte_cryptodev_ops *rte_aesni_gcm_pmd_ops = &aesni_gcm_pmd_ops;
+
+struct rte_security_ops *rte_aesni_gcm_pmd_security_ops =
+		&aesni_gcm_security_ops;
drivers/crypto/aesni_gcm/aesni_gcm_pmd_private.h +23 -0
index 56b29e013..ed3f6eb2e 100644
@@ -114,5 +114,28 @@ aesni_gcm_set_session_parameters(const struct aesni_gcm_ops *ops,
  * Device specific operations function pointer structure */
 extern struct rte_cryptodev_ops *rte_aesni_gcm_pmd_ops;
 
+/**
+ * Security session structure.
+ */
+struct aesni_gcm_security_session {
+	/** Temp digest for decryption */
+	uint8_t temp_digest[DIGEST_LENGTH_MAX];
+	/** GCM operations */
+	aesni_gcm_pre_t pre;
+	aesni_gcm_init_t init;
+	aesni_gcm_update_t update;
+	aesni_gcm_finalize_t finalize;
+	/** AESNI-GCM session */
+	struct aesni_gcm_session sess;
+	/** AESNI-GCM context */
+	struct gcm_context_data gdata_ctx;
+};
+
+extern int
+aesni_gcm_sec_crypto_process_bulk(struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+
+extern struct rte_security_ops *rte_aesni_gcm_pmd_security_ops;
 
 #endif /* _RTE_AESNI_GCM_PMD_PRIVATE_H_ */
drivers/crypto/aesni_gcm/meson.build +1 -2
index 3a6e332dc..f6e160bb3 100644
@@ -22,4 +22,4 @@ endif
 
 allow_experimental_apis = true
 sources = files('aesni_gcm_pmd.c', 'aesni_gcm_pmd_ops.c')
-deps += ['bus_vdev']
+deps += ['bus_vdev', 'security']
--

2.14.5

author
#76 lore raw

PATCH v2 3/10 app/test: add security cpu crypto autotest

In reply to Fan Zhang

This patch adds cpu crypto unit test for AESNI_GCM PMD.

---
 app/test/Makefile                   |   1 +
 app/test/meson.build                |   1 +
 app/test/test_security_cpu_crypto.c | 564 ++++++++++++++++++++++++++++++++++++
 3 files changed, 566 insertions(+)
 create mode 100644 app/test/test_security_cpu_crypto.c
app/test/Makefile +1 -0
index df7f77f44..0caff561c 100644
@@ -197,6 +197,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += test_cryptodev_blockcipher.c
 SRCS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += test_cryptodev.c
 SRCS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += test_cryptodev_asym.c
 SRCS-$(CONFIG_RTE_LIBRTE_SECURITY) += test_cryptodev_security_pdcp.c
+SRCS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += test_security_cpu_crypto.c
 
 SRCS-$(CONFIG_RTE_LIBRTE_METRICS) += test_metrics.c
 
app/test/meson.build +1 -0
index 2c23c6347..0d096c564 100644
@@ -104,6 +104,7 @@ test_sources = files('commands.c',
 	'test_ring_perf.c',
 	'test_rwlock.c',
 	'test_sched.c',
+	'test_security_cpu_crypto.c',
 	'test_service_cores.c',
 	'test_spinlock.c',
 	'test_stack.c',
app/test/test_security_cpu_crypto.c new file +564 -0

564 lines not on this page · raw message

Signature · 1 line
2.14.5
author
#77 lore raw

PATCH v2 4/10 app/test: add security cpu crypto perftest

In reply to Fan Zhang

Since crypto perf application does not support rte_security, this patch adds a simple GCM CPU crypto performance test to crypto unittest application. The test includes different key and data sizes test with single buffer and SGL buffer test items and will display the throughput as well as cycle count performance information.

---
 app/test/test_security_cpu_crypto.c | 201 ++++++++++++++++++++++++++++++++++++
 1 file changed, 201 insertions(+)
app/test/test_security_cpu_crypto.c +201 -0
index d345922b2..ca9a8dae6 100644
@@ -23,6 +23,7 @@
 
 #define CPU_CRYPTO_TEST_MAX_AAD_LENGTH	16
 #define MAX_NB_SIGMENTS			4
+#define CACHE_WARM_ITER			2048
 
 enum buffer_assemble_option {
 	SGL_MAX_SEG,
@@ -560,5 +561,205 @@ test_security_cpu_crypto_aesni_gcm(void)
 	return unit_test_suite_runner(&security_cpu_crypto_aesgcm_testsuite);
 }
 
+
+static inline void
+gen_rand(uint8_t *data, uint32_t len)
+{
+	uint32_t i;
+
+	for (i = 0; i < len; i++)
+		data[i] = (uint8_t)rte_rand();
+}
+
+static inline void
+switch_aead_enc_to_dec(struct aead_test_data *tdata,
+		struct cpu_crypto_test_case *tcase,
+		enum buffer_assemble_option sgl_option)
+{
+	uint32_t i;
+	uint8_t *dst = tdata->ciphertext.data;
+
+	switch (sgl_option) {
+	case SGL_ONE_SEG:
+		memcpy(dst, tcase->seg_buf[0].seg, tcase->seg_buf[0].seg_len);
+		tdata->ciphertext.len = tcase->seg_buf[0].seg_len;
+		break;
+	case SGL_MAX_SEG:
+		tdata->ciphertext.len = 0;
+		for (i = 0; i < MAX_NB_SIGMENTS; i++) {
+			memcpy(dst, tcase->seg_buf[i].seg,
+					tcase->seg_buf[i].seg_len);
+			tdata->ciphertext.len += tcase->seg_buf[i].seg_len;
+		}
+		break;
+	}
+
+	memcpy(tdata->auth_tag.data, tcase->digest, tdata->auth_tag.len);
+}
+
+static int
+cpu_crypto_test_aead_perf(enum buffer_assemble_option sgl_option,
+		uint32_t key_sz)
+{
+	struct aead_test_data tdata = {0};
+	struct cpu_crypto_testsuite_params *ts_params = &testsuite_params;
+	struct cpu_crypto_unittest_params *ut_params = &unittest_params;
+	struct cpu_crypto_test_obj *obj = &ut_params->test_obj;
+	struct cpu_crypto_test_case *tcase;
+	uint64_t hz = rte_get_tsc_hz(), time_start, time_now;
+	double rate, cycles_per_buf;
+	uint32_t test_data_szs[] = {64, 128, 256, 512, 1024, 2048};
+	uint32_t i, j;
+	uint8_t aad[16];
+	int ret;
+
+	tdata.key.len = key_sz;
+	gen_rand(tdata.key.data, tdata.key.len);
+	tdata.algo = RTE_CRYPTO_AEAD_AES_GCM;
+	tdata.aad.data = aad;
+
+	ut_params->sess = create_aead_session(ts_params->ctx,
+			ts_params->session_priv_mpool,
+			RTE_CRYPTO_AEAD_OP_DECRYPT,
+			&tdata,
+			0);
+	if (!ut_params->sess)
+		return -1;
+
+	ret = allocate_buf(MAX_NUM_OPS_INFLIGHT);
+	if (ret)
+		return ret;
+
+	for (i = 0; i < RTE_DIM(test_data_szs); i++) {
+		for (j = 0; j < MAX_NUM_OPS_INFLIGHT; j++) {
+			tdata.plaintext.len = test_data_szs[i];
+			gen_rand(tdata.plaintext.data,
+					tdata.plaintext.len);
+
+			tdata.aad.len = 12;
+			gen_rand(tdata.aad.data, tdata.aad.len);
+
+			tdata.auth_tag.len = 16;
+
+			tdata.iv.len = 16;
+			gen_rand(tdata.iv.data, tdata.iv.len);
+
+			tcase = ut_params->test_datas[j];
+			ret = assemble_aead_buf(tcase, obj, j,
+					RTE_CRYPTO_AEAD_OP_ENCRYPT,
+					&tdata, sgl_option, 0);
+			if (ret < 0) {
+				printf("Test is not supported by the driver\n");
+				return ret;
+			}
+		}
+
+		/* warm up cache */
+		for (j = 0; j < CACHE_WARM_ITER; j++)
+			run_test(ts_params->ctx, ut_params->sess, obj,
+					MAX_NUM_OPS_INFLIGHT);
+
+		time_start = rte_rdtsc();
+
+		run_test(ts_params->ctx, ut_params->sess, obj,
+				MAX_NUM_OPS_INFLIGHT);
+
+		time_now = rte_rdtsc();
+
+		rate = time_now - time_start;
+		cycles_per_buf = rate / MAX_NUM_OPS_INFLIGHT;
+
+		rate = ((hz / cycles_per_buf)) / 1000000;
+
+		printf("AES-GCM-%u(%4uB) Enc %03.3fMpps (%03.3fGbps) ",
+				key_sz * 8, test_data_szs[i], rate,
+				rate  * test_data_szs[i] * 8 / 1000);
+		printf("cycles per buf %03.3f per byte %03.3f\n",
+				cycles_per_buf,
+				cycles_per_buf / test_data_szs[i]);
+
+		for (j = 0; j < MAX_NUM_OPS_INFLIGHT; j++) {
+			tcase = ut_params->test_datas[j];
+
+			switch_aead_enc_to_dec(&tdata, tcase, sgl_option);
+			ret = assemble_aead_buf(tcase, obj, j,
+					RTE_CRYPTO_AEAD_OP_DECRYPT,
+					&tdata, sgl_option, 0);
+			if (ret < 0) {
+				printf("Test is not supported by the driver\n");
+				return ret;
+			}
+		}
+
+		time_start = rte_get_timer_cycles();
+
+		run_test(ts_params->ctx, ut_params->sess, obj,
+				MAX_NUM_OPS_INFLIGHT);
+
+		time_now = rte_get_timer_cycles();
+
+		rate = time_now - time_start;
+		cycles_per_buf = rate / MAX_NUM_OPS_INFLIGHT;
+
+		rate = ((hz / cycles_per_buf)) / 1000000;
+
+		printf("AES-GCM-%u(%4uB) Dec %03.3fMpps (%03.3fGbps) ",
+				key_sz * 8, test_data_szs[i], rate,
+				rate  * test_data_szs[i] * 8 / 1000);
+		printf("cycles per buf %03.3f per byte %03.3f\n",
+				cycles_per_buf,
+				cycles_per_buf / test_data_szs[i]);
+	}
+
+	return 0;
+}
+
+/* test-perfix/key-size/sgl-type */
+#define all_gcm_perf_test_cases(type)					\
+	TEST_EXPAND(_128, 16, type)					\
+	TEST_EXPAND(_192, 24, type)					\
+	TEST_EXPAND(_256, 32, type)
+
+#define TEST_EXPAND(a, b, c)						\
+static int								\
+cpu_crypto_gcm_perf##a##_##c(void)					\
+{									\
+	return cpu_crypto_test_aead_perf(c, b);				\
+}									\
+
+all_gcm_perf_test_cases(SGL_ONE_SEG)
+all_gcm_perf_test_cases(SGL_MAX_SEG)
+#undef TEST_EXPAND
+
+static struct unit_test_suite security_cpu_crypto_aesgcm_perf_testsuite  = {
+		.suite_name = "Security CPU Crypto AESNI-GCM Perf Test Suite",
+		.setup = testsuite_setup,
+		.teardown = testsuite_teardown,
+		.unit_test_cases = {
+#define TEST_EXPAND(a, b, c)						\
+		TEST_CASE_ST(ut_setup, ut_teardown,			\
+				cpu_crypto_gcm_perf##a##_##c),		\
+
+		all_gcm_perf_test_cases(SGL_ONE_SEG)
+		all_gcm_perf_test_cases(SGL_MAX_SEG)
+#undef TEST_EXPAND
+
+		TEST_CASES_END() /**< NULL terminate unit test array */
+		},
+};
+
+static int
+test_security_cpu_crypto_aesni_gcm_perf(void)
+{
+	gbl_driver_id =	rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_AESNI_GCM_PMD));
+
+	return unit_test_suite_runner(
+			&security_cpu_crypto_aesgcm_perf_testsuite);
+}
+
 REGISTER_TEST_COMMAND(security_aesni_gcm_autotest,
 		test_security_cpu_crypto_aesni_gcm);
+
+REGISTER_TEST_COMMAND(security_aesni_gcm_perftest,
+		test_security_cpu_crypto_aesni_gcm_perf);
Signature · 1 line
2.14.5
author
#78 lore raw

PATCH v2 5/10 crypto/aesni_mb: add rte_security handler

In reply to Fan Zhang

This patch add rte_security support support to AESNI-MB PMD. The PMD now initialize security context instance, create/delete PMD specific security sessions, and process crypto workloads in synchronous mode.

---
 drivers/crypto/aesni_mb/meson.build                |   2 +-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c         | 368 +++++++++++++++++++--
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c     |  92 +++++-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h |  21 +-
 4 files changed, 453 insertions(+), 30 deletions(-)
drivers/crypto/aesni_mb/meson.build +1 -1
index 3e1687416..e7b585168 100644
@@ -23,4 +23,4 @@ endif
 
 sources = files('rte_aesni_mb_pmd.c', 'rte_aesni_mb_pmd_ops.c')
 allow_experimental_apis = true
-deps += ['bus_vdev']
+deps += ['bus_vdev', 'security']
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +343 -25

445 lines not on this page · raw message

drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c +91 -1

108 lines not on this page · raw message

drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h +18 -3
index b794d4bc1..64b58ca8e 100644
@@ -176,7 +176,6 @@ struct aesni_mb_qp {
 	 */
 } __rte_cache_aligned;
 
-/** AES-NI multi-buffer private session structure */
 struct aesni_mb_session {
 	JOB_CHAIN_ORDER chain_order;
 	struct {
@@ -265,16 +264,32 @@ struct aesni_mb_session {
 		/** AAD data length */
 		uint16_t aad_len;
 	} aead;
-} __rte_cache_aligned;
+};
+
+/** AES-NI multi-buffer private security session structure */
+struct aesni_mb_sec_session {
+	/**< Unique Queue Pair Name */
+	struct aesni_mb_session sess;
+	uint8_t temp_digests[MAX_JOBS][DIGEST_LENGTH_MAX];
+	uint16_t digest_idx;
+	uint32_t cipher_offset;
+	MB_MGR *mb_mgr;
+};
 
 extern int
 aesni_mb_set_session_parameters(const MB_MGR *mb_mgr,
 		struct aesni_mb_session *sess,
 		const struct rte_crypto_sym_xform *xform);
 
+extern int
+aesni_mb_sec_crypto_process_bulk(struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+
 /** device specific operations function pointer structure */
 extern struct rte_cryptodev_ops *rte_aesni_mb_pmd_ops;
 
-
+/** device specific operations function pointer structure for rte_security */
+extern struct rte_security_ops *rte_aesni_mb_pmd_security_ops;
 
 #endif /* _RTE_AESNI_MB_PMD_PRIVATE_H_ */
Signature · 1 line
2.14.5
#79 lore raw

Re: [dpdk-dev] [PATCH v2 05/10] crypto/aesni_mb: add rte_security handler

Hi Fan,

Quoted 128 lines

This patch add rte_security support support to AESNI-MB PMD. The PMD now initialize security context instance, create/delete PMD specific security sessions, and process crypto workloads in synchronous mode.

---
 drivers/crypto/aesni_mb/meson.build                |   2 +-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c         | 368 +++++++++++++++++++--
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c     |  92 +++++-
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h |  21 +-
 4 files changed, 453 insertions(+), 30 deletions(-)
drivers/crypto/aesni_mb/meson.build +1 -1
index 3e1687416..e7b585168 100644
@@ -23,4 +23,4 @@ endif
 
 sources = files('rte_aesni_mb_pmd.c', 'rte_aesni_mb_pmd_ops.c')
 allow_experimental_apis = true
-deps += ['bus_vdev']
+deps += ['bus_vdev', 'security']
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +84 -0
index ce1144b95..a4cd518b7 100644
@@ -8,6 +8,8 @@
 #include <rte_hexdump.h>
 #include <rte_cryptodev.h>
 #include <rte_cryptodev_pmd.h>
+#include <rte_security.h>
+#include <rte_security_driver.h>
 #include <rte_bus_vdev.h>
 #include <rte_malloc.h>
 #include <rte_cpuflags.h>
@@ -19,6 +21,9 @@
 #define HMAC_MAX_BLOCK_SIZE 128
 static uint8_t cryptodev_driver_id;
 
+static enum aesni_mb_vector_mode vector_mode;
+/**< CPU vector instruction set mode */
+
 typedef void (*hash_one_block_t)(const void *data, void *digest);
 typedef void (*aes_keyexp_t)(const void *key, void *enc_exp_keys, void *dec_exp_keys);
 
@@ -808,6 +813,164 @@ auth_start_offset(struct rte_crypto_op *op, struct aesni_mb_session *session,
 			(UINT64_MAX - u_src + u_dst + 1);
 }
 
+union sec_userdata_field {
+	int status;
+	struct {
+		uint16_t is_gen_digest;
+		uint16_t digest_len;
+	};
+};
+
+struct sec_udata_digest_field {
+	uint32_t is_digest_gen;
+	uint32_t digest_len;
+};
+
+static inline int
+set_mb_job_params_sec(JOB_AES_HMAC *job, struct aesni_mb_sec_session *sec_sess,
+		void *buf, uint32_t buf_len, void *iv, void *aad, void *digest,
+		int *status, uint8_t *digest_idx)
+{
+	struct aesni_mb_session *session = &sec_sess->sess;
+	uint32_t cipher_offset = sec_sess->cipher_offset;
+	union sec_userdata_field udata;
+
+	if (unlikely(cipher_offset > buf_len))
+		return -EINVAL;
+
+	/* Set crypto operation */
+	job->chain_order = session->chain_order;
+
+	/* Set cipher parameters */
+	job->cipher_direction = session->cipher.direction;
+	job->cipher_mode = session->cipher.mode;
+
+	job->aes_key_len_in_bytes = session->cipher.key_length_in_bytes;
+
+	/* Set authentication parameters */
+	job->hash_alg = session->auth.algo;
+	job->iv = iv;
+
+	switch (job->hash_alg) {
+	case AES_XCBC:
+		job->u.XCBC._k1_expanded = session->auth.xcbc.k1_expanded;
+		job->u.XCBC._k2 = session->auth.xcbc.k2;
+		job->u.XCBC._k3 = session->auth.xcbc.k3;
+
+		job->aes_enc_key_expanded =
+				session->cipher.expanded_aes_keys.encode;
+		job->aes_dec_key_expanded =
+				session->cipher.expanded_aes_keys.decode;
+		break;
+
+	case AES_CCM:
+		job->u.CCM.aad = (uint8_t *)aad + 18;
+		job->u.CCM.aad_len_in_bytes = session->aead.aad_len;
+		job->aes_enc_key_expanded =
+				session->cipher.expanded_aes_keys.encode;
+		job->aes_dec_key_expanded =
+				session->cipher.expanded_aes_keys.decode;
+		job->iv++;
+		break;
+
+	case AES_CMAC:
+		job->u.CMAC._key_expanded = session->auth.cmac.expkey;
+		job->u.CMAC._skey1 = session->auth.cmac.skey1;
+		job->u.CMAC._skey2 = session->auth.cmac.skey2;
+		job->aes_enc_key_expanded =
+				session->cipher.expanded_aes_keys.encode;
+		job->aes_dec_key_expanded =
+				session->cipher.expanded_aes_keys.decode;
+		break;
+
+	case AES_GMAC:
+		if (session->cipher.mode == GCM) {
+			job->u.GCM.aad = aad;
+			job->u.GCM.aad_len_in_bytes = session->aead.aad_len;
+		} else {
+			/* For GMAC */
+			job->u.GCM.aad = aad;
+			job->u.GCM.aad_len_in_bytes = buf_len;
+			job->cipher_mode = GCM;

drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c auth_start_offset(struct rte_crypto_op *op, struct aesni_mb_session *session, · line 904

Quoted 10 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +10 -0
+		}
+		job->aes_enc_key_expanded = &session->cipher.gcm_key;
+		job->aes_dec_key_expanded = &session->cipher.gcm_key;
+		break;
+
+	default:
+		job->u.HMAC._hashed_auth_key_xor_ipad =
+				session->auth.pads.inner;
+		job->u.HMAC._hashed_auth_key_xor_opad =
+				session->auth.pads.outer;

Same question as from v1:
Seems like too many branches at data-path.
We'll have only one job-type(alg) per session.
Can we have prefilled job struct template with all common fields already setuped, and then at process() just copy it over and update few fields that has to be different (like msg_len_to_cipher_in_bytes)?
If the whole job struct is big enough (184B), we at least can copy contents of u (24B) in one go, can't we?

Quoted 89 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +82 -0
+
+		if (job->cipher_mode == DES3) {
+			job->aes_enc_key_expanded =
+				session->cipher.exp_3des_keys.ks_ptr;
+			job->aes_dec_key_expanded =
+				session->cipher.exp_3des_keys.ks_ptr;
+		} else {
+			job->aes_enc_key_expanded =
+				session->cipher.expanded_aes_keys.encode;
+			job->aes_dec_key_expanded =
+				session->cipher.expanded_aes_keys.decode;
+		}
+	}
+
+	/* Set digest output location */
+	if (job->hash_alg != NULL_HASH &&
+			session->auth.operation == RTE_CRYPTO_AUTH_OP_VERIFY) {
+		job->auth_tag_output = sec_sess->temp_digests[*digest_idx];
+		*digest_idx = (*digest_idx + 1) % MAX_JOBS;
+
+		udata.is_gen_digest = 0;
+		udata.digest_len = session->auth.req_digest_len;
+	} else {
+		udata.is_gen_digest = 1;
+		udata.digest_len = session->auth.req_digest_len;
+
+		if (session->auth.req_digest_len !=
+				session->auth.gen_digest_len) {
+			job->auth_tag_output =
+					sec_sess->temp_digests[*digest_idx];
+			*digest_idx = (*digest_idx + 1) % MAX_JOBS;
+		} else
+			job->auth_tag_output = digest;
+	}
+
+	/* A bit of hack here, since job structure only supports
+	 * 2 user data fields and we need 4 params to be passed
+	 * (status, direction, digest for verify, and length of
+	 * digest), we set the status value as digest length +
+	 * direction here temporarily to avoid creating longer
+	 * buffer to store all 4 params.
+	 */
+	*status = udata.status;
+
+	/*
+	 * Multi-buffer library current only support returning a truncated
+	 * digest length as specified in the relevant IPsec RFCs
+	 */
+
+	/* Set digest length */
+	job->auth_tag_output_len_in_bytes = session->auth.gen_digest_len;
+
+	/* Set IV parameters */
+	job->iv_len_in_bytes = session->iv.length;
+
+	/* Data Parameters */
+	job->src = buf;
+	job->dst = (uint8_t *)buf + cipher_offset;
+	job->cipher_start_src_offset_in_bytes = cipher_offset;
+	job->msg_len_to_cipher_in_bytes = buf_len - cipher_offset;
+	job->hash_start_src_offset_in_bytes = 0;
+	job->msg_len_to_hash_in_bytes = buf_len;
+
+	job->user_data = (void *)status;
+	job->user_data2 = digest;
+
+	return 0;
+}
+
 /**
  * Process a crypto operation and complete a JOB_AES_HMAC job structure for
  * submission to the multi buffer library for processing.
@@ -1100,6 +1263,35 @@ post_process_mb_job(struct aesni_mb_qp *qp, JOB_AES_HMAC *job)
 	return op;
 }
 
+static inline void
+post_process_mb_sec_job(JOB_AES_HMAC *job)
+{
+	void *user_digest = job->user_data2;
+	int *status = job->user_data;
+
+	switch (job->status) {
+	case STS_COMPLETED:
+		if (user_digest) {
+			union sec_userdata_field udata;
+
+			udata.status = *status;
+			if (udata.is_gen_digest) {

drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c post_process_mb_job(struct aesni_mb_qp *qp, JOB_AES_HMAC *job) · line 1288

Quoted 10 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +10 -0
+				*status = RTE_CRYPTO_OP_STATUS_SUCCESS;
+				memcpy(user_digest, job->auth_tag_output,
+						udata.digest_len);
+			} else {
+				*status = (memcmp(job->auth_tag_output,
+					user_digest, udata.digest_len) != 0) ?
+						-1 : 0;
+			}
+		} else
+			*status = RTE_CRYPTO_OP_STATUS_SUCCESS;

Same question as for v1:
multiple process() functions instead of branches at data-path?

Quoted 81 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +67 -0
+		break;
+	default:
+		*status = RTE_CRYPTO_OP_STATUS_ERROR;
+	}
+}
+
 /**
  * Process a completed JOB_AES_HMAC job and keep processing jobs until
  * get_completed_job return NULL
@@ -1136,6 +1328,32 @@ handle_completed_jobs(struct aesni_mb_qp *qp, JOB_AES_HMAC *job,
 	return processed_jobs;
 }
 
+static inline uint32_t
+handle_completed_sec_jobs(JOB_AES_HMAC *job, MB_MGR *mb_mgr)
+{
+	uint32_t processed = 0;
+
+	while (job != NULL) {
+		post_process_mb_sec_job(job);
+		job = IMB_GET_COMPLETED_JOB(mb_mgr);
+		processed++;
+	}
+
+	return processed;
+}
+
+static inline uint32_t
+flush_mb_sec_mgr(MB_MGR *mb_mgr)
+{
+	JOB_AES_HMAC *job = IMB_FLUSH_JOB(mb_mgr);
+	uint32_t processed = 0;
+
+	if (job)
+		processed = handle_completed_sec_jobs(job, mb_mgr);
+
+	return processed;
+}
+
 static inline uint16_t
 flush_mb_mgr(struct aesni_mb_qp *qp, struct rte_crypto_op **ops,
 		uint16_t nb_ops)
@@ -1239,6 +1457,105 @@ aesni_mb_pmd_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops,
 	return processed_jobs;
 }
 
+static MB_MGR *
+alloc_init_mb_mgr(void)
+{
+	MB_MGR *mb_mgr = alloc_mb_mgr(0);
+	if (mb_mgr == NULL)
+		return NULL;
+
+	switch (vector_mode) {
+	case RTE_AESNI_MB_SSE:
+		init_mb_mgr_sse(mb_mgr);
+		break;
+	case RTE_AESNI_MB_AVX:
+		init_mb_mgr_avx(mb_mgr);
+		break;
+	case RTE_AESNI_MB_AVX2:
+		init_mb_mgr_avx2(mb_mgr);
+		break;
+	case RTE_AESNI_MB_AVX512:
+		init_mb_mgr_avx512(mb_mgr);
+		break;
+	default:
+		AESNI_MB_LOG(ERR, "Unsupported vector mode %u\n", vector_mode);
+		free_mb_mgr(mb_mgr);
+		return NULL;
+	}
+
+	return mb_mgr;
+}
+
+static MB_MGR *sec_mb_mgrs[RTE_MAX_LCORE];
+
+int
+aesni_mb_sec_crypto_process_bulk(struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num)

drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c aesni_mb_pmd_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops, · line 1504

Quoted 10 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +10 -0
+{
+	struct aesni_mb_sec_session *sec_sess = sess->sess_private_data;
+	JOB_AES_HMAC *job;
+	static MB_MGR *mb_mgr;
+	uint32_t lcore_id = rte_lcore_id();
+	uint8_t digest_idx = sec_sess->digest_idx;
+	uint32_t i, processed = 0;
+	int ret = 0, errcnt = 0;
+
+	if (unlikely(sec_mb_mgrs[lcore_id] == NULL)) {

I don't think it is completely safe.
For non-EAL threads rte_lcore_id() == -1.
So at least need to check for lcore_id < RTE_MAX_LCORE.

Quoted 20 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +20 -0
+		sec_mb_mgrs[lcore_id] = alloc_init_mb_mgr();
+
+		if (sec_mb_mgrs[lcore_id] == NULL) {
+			for (i = 0; i < num; i++)
+				status[i] = -ENOMEM;
+
+			return -num;
+		}
+	}
+
+	mb_mgr = sec_mb_mgrs[lcore_id];
+
+	for (i = 0; i < num; i++) {
+		void *seg_buf = buf[i].vec[0].iov_base;
+		uint32_t buf_len = buf[i].vec[0].iov_len;
+
+		job = IMB_GET_NEXT_JOB(mb_mgr);
+		if (unlikely(job == NULL)) {
+			processed += flush_mb_sec_mgr(mb_mgr);
+

drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c aesni_mb_pmd_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops, · line 1534

Quoted 10 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +10 -0
+			job = IMB_GET_NEXT_JOB(mb_mgr);
+			if (!job) {
+				errcnt -= 1;
+				status[i] = -ENOMEM;
+			}
+		}
+
+		ret = set_mb_job_params_sec(job, sec_sess, seg_buf, buf_len,
+				iv[i], aad[i], digest[i], &status[i],
+				&digest_idx);

I still don't understand the purpose of passing digest_idx pointer here... Why not to just:

ret = set_mb_job_params_sec(job, sec_sess, seg_buf, buf_len,

				iv[i], aad[i], digest[i], &status[i],
				digest_idx);

digest_idx = (digest_idx + 1) % MAX_JOBS;
Second thing, I am not sure what is the purpose to store digest_idx inside the session (sess->digest_idx) at all?
As I can see you never update it, and it seems just digest_idx = 0
at the start of that function is enough?

Quoted 253 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +50 -25
+				/* Submit job to multi-buffer for processing */
+		if (ret) {
+			processed++;
+			status[i] = ret;
+			errcnt -= 1;
+			continue;
+		}
+
+#ifdef RTE_LIBRTE_PMD_AESNI_MB_DEBUG
+		job = IMB_SUBMIT_JOB(mb_mgr);
+#else
+		job = IMB_SUBMIT_JOB_NOCHECK(mb_mgr);
+#endif
+
+		if (job)
+			processed += handle_completed_sec_jobs(job, mb_mgr);
+	}
+
+	while (processed < num)
+		processed += flush_mb_sec_mgr(mb_mgr);
+
+	return errcnt;
+}
+
 static int cryptodev_aesni_mb_remove(struct rte_vdev_device *vdev);
 
 static int
@@ -1248,8 +1565,9 @@ cryptodev_aesni_mb_create(const char *name,
 {
 	struct rte_cryptodev *dev;
 	struct aesni_mb_private *internals;
-	enum aesni_mb_vector_mode vector_mode;
+	struct rte_security_ctx *sec_ctx;
 	MB_MGR *mb_mgr;
+	char sec_name[RTE_DEV_NAME_MAX_LEN];
 
 	/* Check CPU for support for AES instruction set */
 	if (!rte_cpu_get_flag_enabled(RTE_CPUFLAG_AES)) {
@@ -1283,35 +1601,14 @@ cryptodev_aesni_mb_create(const char *name,
 	dev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO |
 			RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING |
 			RTE_CRYPTODEV_FF_CPU_AESNI |
-			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT;
+			RTE_CRYPTODEV_FF_OOP_LB_IN_LB_OUT |
+			RTE_CRYPTODEV_FF_SECURITY;
 
 
-	mb_mgr = alloc_mb_mgr(0);
+	mb_mgr = alloc_init_mb_mgr();
 	if (mb_mgr == NULL)
 		return -ENOMEM;
 
-	switch (vector_mode) {
-	case RTE_AESNI_MB_SSE:
-		dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_SSE;
-		init_mb_mgr_sse(mb_mgr);
-		break;
-	case RTE_AESNI_MB_AVX:
-		dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_AVX;
-		init_mb_mgr_avx(mb_mgr);
-		break;
-	case RTE_AESNI_MB_AVX2:
-		dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_AVX2;
-		init_mb_mgr_avx2(mb_mgr);
-		break;
-	case RTE_AESNI_MB_AVX512:
-		dev->feature_flags |= RTE_CRYPTODEV_FF_CPU_AVX512;
-		init_mb_mgr_avx512(mb_mgr);
-		break;
-	default:
-		AESNI_MB_LOG(ERR, "Unsupported vector mode %u\n", vector_mode);
-		goto error_exit;
-	}
-
 	/* Set vector instructions mode supported */
 	internals = dev->data->dev_private;
 
@@ -1322,11 +1619,28 @@ cryptodev_aesni_mb_create(const char *name,
 	AESNI_MB_LOG(INFO, "IPSec Multi-buffer library version used: %s\n",
 			imb_get_version_str());
 
+	/* setup security operations */
+	snprintf(sec_name, sizeof(sec_name) - 1, "aes_mb_sec_%u",
+			dev->driver_id);
+	sec_ctx = rte_zmalloc_socket(sec_name,
+			sizeof(struct rte_security_ctx),
+			RTE_CACHE_LINE_SIZE, init_params->socket_id);
+	if (sec_ctx == NULL) {
+		AESNI_MB_LOG(ERR, "memory allocation failed\n");
+		goto error_exit;
+	}
+
+	sec_ctx->device = (void *)dev;
+	sec_ctx->ops = rte_aesni_mb_pmd_security_ops;
+	dev->security_ctx = sec_ctx;
+
 	return 0;
 
 error_exit:
 	if (mb_mgr)
 		free_mb_mgr(mb_mgr);
+	if (sec_ctx)
+		rte_free(sec_ctx);
 
 	rte_cryptodev_pmd_destroy(dev);
 
@@ -1367,6 +1681,7 @@ cryptodev_aesni_mb_remove(struct rte_vdev_device *vdev)
 	struct rte_cryptodev *cryptodev;
 	struct aesni_mb_private *internals;
 	const char *name;
+	uint32_t i;
 
 	name = rte_vdev_device_name(vdev);
 	if (name == NULL)
@@ -1379,6 +1694,9 @@ cryptodev_aesni_mb_remove(struct rte_vdev_device *vdev)
 	internals = cryptodev->data->dev_private;
 
 	free_mb_mgr(internals->mb_mgr);
+	for (i = 0; i < RTE_MAX_LCORE; i++)
+		if (sec_mb_mgrs[i])
+			free_mb_mgr(sec_mb_mgrs[i]);
 
 	return rte_cryptodev_pmd_destroy(cryptodev);
 }
drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c +91 -1
index 8d15b99d4..f47df2d57 100644
@@ -8,6 +8,7 @@
 #include <rte_common.h>
 #include <rte_malloc.h>
 #include <rte_cryptodev_pmd.h>
+#include <rte_security_driver.h>
 
 #include "rte_aesni_mb_pmd_private.h"
 
@@ -732,7 +733,8 @@ aesni_mb_pmd_qp_count(struct rte_cryptodev *dev)
 static unsigned
 aesni_mb_pmd_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)
 {
-	return sizeof(struct aesni_mb_session);
+	return RTE_ALIGN_CEIL(sizeof(struct aesni_mb_session),
+			RTE_CACHE_LINE_SIZE);
 }
 
 /** Configure a aesni multi-buffer session from a crypto xform chain */
@@ -810,4 +812,92 @@ struct rte_cryptodev_ops aesni_mb_pmd_ops = {
 		.sym_session_clear	= aesni_mb_pmd_sym_session_clear
 };
 
+/** Set session authentication parameters */
+
+static int
+aesni_mb_security_session_create(void *dev,
+		struct rte_security_session_conf *conf,
+		struct rte_security_session *sess,
+		struct rte_mempool *mempool)
+{
+	struct rte_cryptodev *cdev = dev;
+	struct aesni_mb_private *internals = cdev->data->dev_private;
+	struct aesni_mb_sec_session *sess_priv;
+	int ret;
+
+	if (!conf->crypto_xform) {
+		AESNI_MB_LOG(ERR, "Invalid security session conf");
+		return -EINVAL;
+	}
+
+	if (conf->cpucrypto.cipher_offset < 0) {
+		AESNI_MB_LOG(ERR, "Invalid security session conf");
+		return -EINVAL;
+	}
+
+	if (rte_mempool_get(mempool, (void **)(&sess_priv))) {
+		AESNI_MB_LOG(ERR,
+				"Couldn't get object from session mempool");
+		return -ENOMEM;
+	}
+
+	sess_priv->cipher_offset = conf->cpucrypto.cipher_offset;
+
+	ret = aesni_mb_set_session_parameters(internals->mb_mgr,
+			&sess_priv->sess, conf->crypto_xform);
+	if (ret != 0) {
+		AESNI_MB_LOG(ERR, "failed configure session parameters");
+
+		rte_mempool_put(mempool, sess_priv);
+	}
+
+	sess->sess_private_data = (void *)sess_priv;
+
+	return ret;
+}
+
+static int
+aesni_mb_security_session_destroy(void *dev __rte_unused,
+		struct rte_security_session *sess)
+{
+	struct aesni_mb_sec_session *sess_priv =
+			get_sec_session_private_data(sess);
+
+	if (sess_priv) {
+		struct rte_mempool *sess_mp = rte_mempool_from_obj(
+				(void *)sess_priv);
+
+		memset(sess, 0, sizeof(struct aesni_mb_sec_session));
+		set_sec_session_private_data(sess, NULL);
+
+		if (sess_mp == NULL) {
+			AESNI_MB_LOG(ERR, "failed fetch session mempool");
+			return -EINVAL;
+		}
+
+		rte_mempool_put(sess_mp, sess_priv);
+	}
+
+	return 0;
+}
+
+static unsigned int
+aesni_mb_sec_session_get_size(__rte_unused void *device)
+{
+	return RTE_ALIGN_CEIL(sizeof(struct aesni_mb_sec_session),
+			RTE_CACHE_LINE_SIZE);
+}
+
+static struct rte_security_ops aesni_mb_security_ops = {
+		.session_create = aesni_mb_security_session_create,
+		.session_get_size = aesni_mb_sec_session_get_size,
+		.session_update = NULL,
+		.session_stats_get = NULL,
+		.session_destroy = aesni_mb_security_session_destroy,
+		.set_pkt_metadata = NULL,
+		.capabilities_get = NULL,
+		.process_cpu_crypto_bulk = aesni_mb_sec_crypto_process_bulk,
+};
+
 struct rte_cryptodev_ops *rte_aesni_mb_pmd_ops = &aesni_mb_pmd_ops;
+struct rte_security_ops *rte_aesni_mb_pmd_security_ops = &aesni_mb_security_ops;
drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h +0 -1
index b794d4bc1..64b58ca8e 100644
@@ -176,7 +176,6 @@ struct aesni_mb_qp {
 	 */
 } __rte_cache_aligned;
 
-/** AES-NI multi-buffer private session structure */
 struct aesni_mb_session {
 	JOB_CHAIN_ORDER chain_order;
 	struct {
@@ -265,16 +264,32 @@ struct aesni_mb_session {
 		/** AAD data length */

drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h struct aesni_mb_session { · line 273

Quoted 10 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h +7 -1
 		uint16_t aad_len;
 	} aead;
-} __rte_cache_aligned;
+};
+
+/** AES-NI multi-buffer private security session structure */
+struct aesni_mb_sec_session {
+	/**< Unique Queue Pair Name */
+	struct aesni_mb_session sess;
+	uint8_t temp_digests[MAX_JOBS][DIGEST_LENGTH_MAX];

Same question as for v1:
Probably better to move these temp_digest[][] at the very end? To have all read-only data grouped together?
Another thought - do you need it here at all?
Can't we just allocate temp_digests[MAX_JOBS][DIGEST_LENGTH_MAX];
on the stack inside process() function?

Quoted 25 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h +11 -2
+	uint16_t digest_idx;
+	uint32_t cipher_offset;
+	MB_MGR *mb_mgr;
+};
 
 extern int
 aesni_mb_set_session_parameters(const MB_MGR *mb_mgr,
 		struct aesni_mb_session *sess,
 		const struct rte_crypto_sym_xform *xform);
 
+extern int
+aesni_mb_sec_crypto_process_bulk(struct rte_security_session *sess,
+		struct rte_security_vec buf[], void *iv[], void *aad[],
+		void *digest[], int status[], uint32_t num);
+
 /** device specific operations function pointer structure */
 extern struct rte_cryptodev_ops *rte_aesni_mb_pmd_ops;
 
-
+/** device specific operations function pointer structure for rte_security */
+extern struct rte_security_ops *rte_aesni_mb_pmd_security_ops;
 
 #endif /* _RTE_AESNI_MB_PMD_PRIVATE_H_ */
--

2.14.5

#80 lore raw

Re: [dpdk-dev] [PATCH v2 05/10] crypto/aesni_mb: add rte_security handler

In reply to Fan Zhang

drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c cryptodev_aesni_mb_create(const char *name, · line 1624

Quoted 6 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +3 -0
+	/* setup security operations */
+	snprintf(sec_name, sizeof(sec_name) - 1, "aes_mb_sec_%u",
+			dev->driver_id);

Just a nit here and in aesni_gcm code:
this is useless actually, rte_malloc ignores name argument. You can safely pass NULL here.

Quoted 22 lines
drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +14 -0
+	sec_ctx = rte_zmalloc_socket(sec_name,
+			sizeof(struct rte_security_ctx),
+			RTE_CACHE_LINE_SIZE, init_params->socket_id);
+	if (sec_ctx == NULL) {
+		AESNI_MB_LOG(ERR, "memory allocation failed\n");
+		goto error_exit;
+	}
+
+	sec_ctx->device = (void *)dev;
+	sec_ctx->ops = rte_aesni_mb_pmd_security_ops;
+	dev->security_ctx = sec_ctx;
+
 	return 0;
 
 error_exit:
 	if (mb_mgr)
 		free_mb_mgr(mb_mgr);
+	if (sec_ctx)
+		rte_free(sec_ctx);
 
 	rte_cryptodev_pmd_destroy(dev);
author
#81 lore raw

PATCH v2 6/10 app/test: add aesni_mb security cpu crypto autotest

In reply to Fan Zhang

This patch adds cpu crypto unit test for AESNI_MB PMD.

---
 app/test/test_security_cpu_crypto.c | 371 +++++++++++++++++++++++++++++++++++-
 1 file changed, 369 insertions(+), 2 deletions(-)
app/test/test_security_cpu_crypto.c +369 -2
index ca9a8dae6..a9853a0c0 100644
@@ -19,12 +19,23 @@
 
 #include "test.h"
 #include "test_cryptodev.h"
+#include "test_cryptodev_blockcipher.h"
+#include "test_cryptodev_aes_test_vectors.h"
 #include "test_cryptodev_aead_test_vectors.h"
+#include "test_cryptodev_des_test_vectors.h"
+#include "test_cryptodev_hash_test_vectors.h"
 
 #define CPU_CRYPTO_TEST_MAX_AAD_LENGTH	16
 #define MAX_NB_SIGMENTS			4
 #define CACHE_WARM_ITER			2048
 
+#define TOP_ENC		BLOCKCIPHER_TEST_OP_ENCRYPT
+#define TOP_DEC		BLOCKCIPHER_TEST_OP_DECRYPT
+#define TOP_AUTH_GEN	BLOCKCIPHER_TEST_OP_AUTH_GEN
+#define TOP_AUTH_VER	BLOCKCIPHER_TEST_OP_AUTH_VERIFY
+#define TOP_ENC_AUTH	BLOCKCIPHER_TEST_OP_ENC_AUTH_GEN
+#define TOP_AUTH_DEC	BLOCKCIPHER_TEST_OP_AUTH_VERIFY_DEC
+
 enum buffer_assemble_option {
 	SGL_MAX_SEG,
 	SGL_ONE_SEG,
@@ -35,8 +46,8 @@ struct cpu_crypto_test_case {
 		uint8_t seg[MBUF_DATAPAYLOAD_SIZE];
 		uint32_t seg_len;
 	} seg_buf[MAX_NB_SIGMENTS];
-	uint8_t iv[MAXIMUM_IV_LENGTH];
-	uint8_t aad[CPU_CRYPTO_TEST_MAX_AAD_LENGTH];
+	uint8_t iv[MAXIMUM_IV_LENGTH * 2];
+	uint8_t aad[CPU_CRYPTO_TEST_MAX_AAD_LENGTH * 4];
 	uint8_t digest[DIGEST_BYTE_LENGTH_SHA512];
 } __rte_cache_aligned;
 
@@ -516,6 +527,11 @@ cpu_crypto_test_aead(const struct aead_test_data *tdata,
 	TEST_EXPAND(gcm_test_case_256_6, type)	\
 	TEST_EXPAND(gcm_test_case_256_7, type)
 
+/* test-vector/sgl-option */
+#define all_ccm_unit_test_cases \
+	TEST_EXPAND(ccm_test_case_128_1, SGL_ONE_SEG) \
+	TEST_EXPAND(ccm_test_case_128_2, SGL_ONE_SEG) \
+	TEST_EXPAND(ccm_test_case_128_3, SGL_ONE_SEG)
 
 #define TEST_EXPAND(t, o)						\
 static int								\
@@ -531,6 +547,7 @@ cpu_crypto_aead_dec_test_##t##_##o(void)				\
 
 all_gcm_unit_test_cases(SGL_ONE_SEG)
 all_gcm_unit_test_cases(SGL_MAX_SEG)
+all_ccm_unit_test_cases
 #undef TEST_EXPAND
 
 static struct unit_test_suite security_cpu_crypto_aesgcm_testsuite  = {
@@ -758,8 +775,358 @@ test_security_cpu_crypto_aesni_gcm_perf(void)
 			&security_cpu_crypto_aesgcm_perf_testsuite);
 }
 
+static struct rte_security_session *
+create_blockcipher_session(struct rte_security_ctx *ctx,
+		struct rte_mempool *sess_mp,
+		uint32_t op_mask,
+		const struct blockcipher_test_data *test_data,
+		uint32_t is_unit_test)
+{
+	struct rte_security_session_conf sess_conf = {0};
+	struct rte_crypto_sym_xform xforms[2] = { {0} };
+	struct rte_crypto_sym_xform *cipher_xform = NULL;
+	struct rte_crypto_sym_xform *auth_xform = NULL;
+	struct rte_crypto_sym_xform *xform;
+
+	if (op_mask & BLOCKCIPHER_TEST_OP_CIPHER) {
+		cipher_xform = &xforms[0];
+		cipher_xform->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
+
+		if (op_mask & TOP_ENC)
+			cipher_xform->cipher.op =
+				RTE_CRYPTO_CIPHER_OP_ENCRYPT;
+		else
+			cipher_xform->cipher.op =
+				RTE_CRYPTO_CIPHER_OP_DECRYPT;
+
+		cipher_xform->cipher.algo = test_data->crypto_algo;
+		cipher_xform->cipher.key.data = test_data->cipher_key.data;
+		cipher_xform->cipher.key.length = test_data->cipher_key.len;
+		cipher_xform->cipher.iv.offset = 0;
+		cipher_xform->cipher.iv.length = test_data->iv.len;
+
+		if (is_unit_test)
+			debug_hexdump(stdout, "cipher key:",
+					test_data->cipher_key.data,
+					test_data->cipher_key.len);
+	}
+
+	if (op_mask & BLOCKCIPHER_TEST_OP_AUTH) {
+		auth_xform = &xforms[1];
+		auth_xform->type = RTE_CRYPTO_SYM_XFORM_AUTH;
+
+		if (op_mask & TOP_AUTH_GEN)
+			auth_xform->auth.op = RTE_CRYPTO_AUTH_OP_GENERATE;
+		else
+			auth_xform->auth.op = RTE_CRYPTO_AUTH_OP_VERIFY;
+
+		auth_xform->auth.algo = test_data->auth_algo;
+		auth_xform->auth.key.length = test_data->auth_key.len;
+		auth_xform->auth.key.data = test_data->auth_key.data;
+		auth_xform->auth.digest_length = test_data->digest.len;
+
+		if (is_unit_test)
+			debug_hexdump(stdout, "auth key:",
+					test_data->auth_key.data,
+					test_data->auth_key.len);
+	}
+
+	if (op_mask == TOP_ENC ||
+			op_mask == TOP_DEC)
+		xform = cipher_xform;
+	else if (op_mask == TOP_AUTH_GEN ||
+			op_mask == TOP_AUTH_VER)
+		xform = auth_xform;
+	else if (op_mask == TOP_ENC_AUTH) {
+		xform = cipher_xform;
+		xform->next = auth_xform;
+	} else if (op_mask == TOP_AUTH_DEC) {
+		xform = auth_xform;
+		xform->next = cipher_xform;
+	} else
+		return NULL;
+
+	if (test_data->cipher_offset < test_data->auth_offset)
+		return NULL;
+
+	sess_conf.action_type = RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO;
+	sess_conf.crypto_xform = xform;
+	sess_conf.cpucrypto.cipher_offset = test_data->cipher_offset -
+			test_data->auth_offset;
+
+	return rte_security_session_create(ctx, &sess_conf, sess_mp);
+}
+
+static inline int
+assemble_blockcipher_buf(struct cpu_crypto_test_case *data,
+		struct cpu_crypto_test_obj *obj,
+		uint32_t obj_idx,
+		uint32_t op_mask,
+		const struct blockcipher_test_data *test_data,
+		uint32_t is_unit_test)
+{
+	const uint8_t *src;
+	uint32_t src_len;
+	uint32_t offset;
+
+	if (op_mask == TOP_ENC_AUTH ||
+			op_mask == TOP_AUTH_GEN ||
+			op_mask == BLOCKCIPHER_TEST_OP_AUTH_VERIFY)
+		offset = test_data->auth_offset;
+	else
+		offset = test_data->cipher_offset;
+
+	if (op_mask & TOP_ENC_AUTH) {
+		src = test_data->plaintext.data;
+		src_len = test_data->plaintext.len;
+		if (is_unit_test)
+			debug_hexdump(stdout, "plaintext:", src, src_len);
+	} else {
+		src = test_data->ciphertext.data;
+		src_len = test_data->ciphertext.len;
+		memcpy(data->digest, test_data->digest.data,
+				test_data->digest.len);
+		if (is_unit_test) {
+			debug_hexdump(stdout, "ciphertext:", src, src_len);
+			debug_hexdump(stdout, "digest:", test_data->digest.data,
+					test_data->digest.len);
+		}
+	}
+
+	if (src_len > MBUF_DATAPAYLOAD_SIZE)
+		return -ENOMEM;
+
+	memcpy(data->seg_buf[0].seg, src, src_len);
+	data->seg_buf[0].seg_len = src_len;
+	obj->vec[obj_idx][0].iov_base =
+			(void *)(data->seg_buf[0].seg + offset);
+	obj->vec[obj_idx][0].iov_len = src_len - offset;
+
+	obj->sec_buf[obj_idx].vec = obj->vec[obj_idx];
+	obj->sec_buf[obj_idx].num = 1;
+
+	memcpy(data->iv, test_data->iv.data, test_data->iv.len);
+	if (is_unit_test)
+		debug_hexdump(stdout, "iv:", test_data->iv.data,
+				test_data->iv.len);
+
+	obj->iv[obj_idx] = (void *)data->iv;
+	obj->digest[obj_idx] = (void *)data->digest;
+
+	return 0;
+}
+
+static int
+check_blockcipher_result(struct cpu_crypto_test_case *tcase,
+		uint32_t op_mask,
+		const struct blockcipher_test_data *test_data)
+{
+	int ret;
+
+	if (op_mask & BLOCKCIPHER_TEST_OP_CIPHER) {
+		const char *err_msg1, *err_msg2;
+		const uint8_t *src_pt_ct;
+		uint32_t src_len;
+
+		if (op_mask & TOP_ENC) {
+			src_pt_ct = test_data->ciphertext.data;
+			src_len = test_data->ciphertext.len;
+			err_msg1 = CPU_CRYPTO_ERR_EXP_CT;
+			err_msg2 = CPU_CRYPTO_ERR_GEN_CT;
+		} else {
+			src_pt_ct = test_data->plaintext.data;
+			src_len = test_data->plaintext.len;
+			err_msg1 = CPU_CRYPTO_ERR_EXP_PT;
+			err_msg2 = CPU_CRYPTO_ERR_GEN_PT;
+		}
+
+		ret = memcmp(tcase->seg_buf[0].seg, src_pt_ct, src_len);
+		if (ret != 0) {
+			debug_hexdump(stdout, err_msg1, src_pt_ct, src_len);
+			debug_hexdump(stdout, err_msg2,
+					tcase->seg_buf[0].seg,
+					test_data->ciphertext.len);
+			return -1;
+		}
+	}
+
+	if (op_mask & TOP_AUTH_GEN) {
+		ret = memcmp(tcase->digest, test_data->digest.data,
+				test_data->digest.len);
+		if (ret != 0) {
+			debug_hexdump(stdout, "expect digest:",
+					test_data->digest.data,
+					test_data->digest.len);
+			debug_hexdump(stdout, "gen digest:",
+					tcase->digest,
+					test_data->digest.len);
+			return -1;
+		}
+	}
+
+	return 0;
+}
+
+static int
+cpu_crypto_test_blockcipher(const struct blockcipher_test_data *tdata,
+		uint32_t op_mask)
+{
+	struct cpu_crypto_testsuite_params *ts_params = &testsuite_params;
+	struct cpu_crypto_unittest_params *ut_params = &unittest_params;
+	struct cpu_crypto_test_obj *obj = &ut_params->test_obj;
+	struct cpu_crypto_test_case *tcase;
+	int ret;
+
+	ut_params->sess = create_blockcipher_session(ts_params->ctx,
+			ts_params->session_priv_mpool,
+			op_mask,
+			tdata,
+			1);
+	if (!ut_params->sess)
+		return -1;
+
+	ret = allocate_buf(1);
+	if (ret)
+		return ret;
+
+	tcase = ut_params->test_datas[0];
+	ret = assemble_blockcipher_buf(tcase, obj, 0, op_mask, tdata, 1);
+	if (ret < 0) {
+		printf("Test is not supported by the driver\n");
+		return ret;
+	}
+
+	run_test(ts_params->ctx, ut_params->sess, obj, 1);
+
+	ret = check_status(obj, 1);
+	if (ret < 0)
+		return ret;
+
+	ret = check_blockcipher_result(tcase, op_mask, tdata);
+	if (ret < 0)
+		return ret;
+
+	return 0;
+}
+
+/* Macro to save code for defining BlockCipher test cases */
+/* test-vector-name/op */
+#define all_blockcipher_test_cases \
+	TEST_EXPAND(aes_test_data_1, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_1, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_1, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_1, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_2, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_2, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_2, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_2, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_3, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_3, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_3, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_3, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_4, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_4, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_4, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_4, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_5, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_5, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_5, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_5, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_6, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_6, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_6, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_6, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_7, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_7, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_7, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_7, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_8, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_8, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_8, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_8, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_9, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_9, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_9, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_9, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_10, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_10, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_11, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_11, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_12, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_12, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_12, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_12, TOP_AUTH_DEC) \
+	TEST_EXPAND(aes_test_data_13, TOP_ENC) \
+	TEST_EXPAND(aes_test_data_13, TOP_DEC) \
+	TEST_EXPAND(aes_test_data_13, TOP_ENC_AUTH) \
+	TEST_EXPAND(aes_test_data_13, TOP_AUTH_DEC) \
+	TEST_EXPAND(des_test_data_1, TOP_ENC) \
+	TEST_EXPAND(des_test_data_1, TOP_DEC) \
+	TEST_EXPAND(des_test_data_2, TOP_ENC) \
+	TEST_EXPAND(des_test_data_2, TOP_DEC) \
+	TEST_EXPAND(des_test_data_3, TOP_ENC) \
+	TEST_EXPAND(des_test_data_3, TOP_DEC) \
+	TEST_EXPAND(triple_des128cbc_hmac_sha1_test_vector, TOP_ENC) \
+	TEST_EXPAND(triple_des128cbc_hmac_sha1_test_vector, TOP_DEC) \
+	TEST_EXPAND(triple_des128cbc_hmac_sha1_test_vector, TOP_ENC_AUTH) \
+	TEST_EXPAND(triple_des128cbc_hmac_sha1_test_vector, TOP_AUTH_DEC) \
+	TEST_EXPAND(triple_des64cbc_test_vector, TOP_ENC) \
+	TEST_EXPAND(triple_des64cbc_test_vector, TOP_DEC) \
+	TEST_EXPAND(triple_des128cbc_test_vector, TOP_ENC) \
+	TEST_EXPAND(triple_des128cbc_test_vector, TOP_DEC) \
+	TEST_EXPAND(triple_des192cbc_test_vector, TOP_ENC) \
+	TEST_EXPAND(triple_des192cbc_test_vector, TOP_DEC) \
+
+#define TEST_EXPAND(t, o)						\
+static int								\
+cpu_crypto_blockcipher_test_##t##_##o(void)				\
+{									\
+	return cpu_crypto_test_blockcipher(&t, o);			\
+}
+
+all_blockcipher_test_cases
+#undef TEST_EXPAND
+
+static struct unit_test_suite security_cpu_crypto_aesni_mb_testsuite  = {
+	.suite_name = "Security CPU Crypto AESNI-MB Unit Test Suite",
+	.setup = testsuite_setup,
+	.teardown = testsuite_teardown,
+	.unit_test_cases = {
+#define TEST_EXPAND(t, o)						\
+	TEST_CASE_ST(ut_setup, ut_teardown,				\
+			cpu_crypto_aead_enc_test_##t##_##o),		\
+	TEST_CASE_ST(ut_setup, ut_teardown,				\
+			cpu_crypto_aead_dec_test_##t##_##o),		\
+
+	all_gcm_unit_test_cases(SGL_ONE_SEG)
+	all_ccm_unit_test_cases
+#undef TEST_EXPAND
+
+#define TEST_EXPAND(t, o)						\
+	TEST_CASE_ST(ut_setup, ut_teardown,				\
+			cpu_crypto_blockcipher_test_##t##_##o),		\
+
+	all_blockcipher_test_cases
+#undef TEST_EXPAND
+
+	TEST_CASES_END() /**< NULL terminate unit test array */
+	},
+};
+
+static int
+test_security_cpu_crypto_aesni_mb(void)
+{
+	gbl_driver_id =	rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
+
+	return unit_test_suite_runner(&security_cpu_crypto_aesni_mb_testsuite);
+}
+
 REGISTER_TEST_COMMAND(security_aesni_gcm_autotest,
 		test_security_cpu_crypto_aesni_gcm);
 
 REGISTER_TEST_COMMAND(security_aesni_gcm_perftest,
 		test_security_cpu_crypto_aesni_gcm_perf);
+
+REGISTER_TEST_COMMAND(security_aesni_mb_autotest,
+		test_security_cpu_crypto_aesni_mb);
Signature · 1 line
2.14.5
author
#82 lore raw

PATCH v2 7/10 app/test: add aesni_mb security cpu crypto perftest

In reply to Fan Zhang

Since crypto perf application does not support rte_security, this patch adds a simple AES-CBC-SHA1-HMAC CPU crypto performance test to crypto unittest application. The test includes different key and data sizes test with single buffer test items and will display the throughput as well as cycle count performance information.

---
 app/test/test_security_cpu_crypto.c | 194 ++++++++++++++++++++++++++++++++++++
 1 file changed, 194 insertions(+)
app/test/test_security_cpu_crypto.c +194 -0
index a9853a0c0..c3689d138 100644
@@ -1122,6 +1122,197 @@ test_security_cpu_crypto_aesni_mb(void)
 	return unit_test_suite_runner(&security_cpu_crypto_aesni_mb_testsuite);
 }
 
+static inline void
+switch_blockcipher_enc_to_dec(struct blockcipher_test_data *tdata,
+		struct cpu_crypto_test_case *tcase, uint8_t *dst)
+{
+	memcpy(dst, tcase->seg_buf[0].seg, tcase->seg_buf[0].seg_len);
+	tdata->ciphertext.len = tcase->seg_buf[0].seg_len;
+	memcpy(tdata->digest.data, tcase->digest, tdata->digest.len);
+}
+
+static int
+cpu_crypto_test_blockcipher_perf(
+		const enum rte_crypto_cipher_algorithm cipher_algo,
+		uint32_t cipher_key_sz,
+		const enum rte_crypto_auth_algorithm auth_algo,
+		uint32_t auth_key_sz, uint32_t digest_sz,
+		uint32_t op_mask)
+{
+	struct blockcipher_test_data tdata = {0};
+	uint8_t plaintext[3000], ciphertext[3000];
+	struct cpu_crypto_testsuite_params *ts_params = &testsuite_params;
+	struct cpu_crypto_unittest_params *ut_params = &unittest_params;
+	struct cpu_crypto_test_obj *obj = &ut_params->test_obj;
+	struct cpu_crypto_test_case *tcase;
+	uint64_t hz = rte_get_tsc_hz(), time_start, time_now;
+	double rate, cycles_per_buf;
+	uint32_t test_data_szs[] = {64, 128, 256, 512, 1024, 2048};
+	uint32_t i, j;
+	uint32_t op_mask_opp = 0;
+	int ret;
+
+	if (op_mask & BLOCKCIPHER_TEST_OP_CIPHER)
+		op_mask_opp |= (~op_mask & BLOCKCIPHER_TEST_OP_CIPHER);
+	if (op_mask & BLOCKCIPHER_TEST_OP_AUTH)
+		op_mask_opp |= (~op_mask & BLOCKCIPHER_TEST_OP_AUTH);
+
+	tdata.plaintext.data = plaintext;
+	tdata.ciphertext.data = ciphertext;
+
+	tdata.cipher_key.len = cipher_key_sz;
+	tdata.auth_key.len = auth_key_sz;
+
+	gen_rand(tdata.cipher_key.data, cipher_key_sz / 8);
+	gen_rand(tdata.auth_key.data, auth_key_sz / 8);
+
+	tdata.crypto_algo = cipher_algo;
+	tdata.auth_algo = auth_algo;
+
+	tdata.digest.len = digest_sz;
+
+	ut_params->sess = create_blockcipher_session(ts_params->ctx,
+			ts_params->session_priv_mpool,
+			op_mask,
+			&tdata,
+			0);
+	if (!ut_params->sess)
+		return -1;
+
+	ret = allocate_buf(MAX_NUM_OPS_INFLIGHT);
+	if (ret)
+		return ret;
+
+	for (i = 0; i < RTE_DIM(test_data_szs); i++) {
+		for (j = 0; j < MAX_NUM_OPS_INFLIGHT; j++) {
+			tdata.plaintext.len = test_data_szs[i];
+			gen_rand(plaintext, tdata.plaintext.len);
+
+			tdata.iv.len = 16;
+			gen_rand(tdata.iv.data, tdata.iv.len);
+
+			tcase = ut_params->test_datas[j];
+			ret = assemble_blockcipher_buf(tcase, obj, j,
+					op_mask,
+					&tdata,
+					0);
+			if (ret < 0) {
+				printf("Test is not supported by the driver\n");
+				return ret;
+			}
+		}
+
+		/* warm up cache */
+		for (j = 0; j < CACHE_WARM_ITER; j++)
+			run_test(ts_params->ctx, ut_params->sess, obj,
+					MAX_NUM_OPS_INFLIGHT);
+
+		time_start = rte_rdtsc();
+
+		run_test(ts_params->ctx, ut_params->sess, obj,
+				MAX_NUM_OPS_INFLIGHT);
+
+		time_now = rte_rdtsc();
+
+		rate = time_now - time_start;
+		cycles_per_buf = rate / MAX_NUM_OPS_INFLIGHT;
+
+		rate = ((hz / cycles_per_buf)) / 1000000;
+
+		printf("%s-%u-%s(%4uB) Enc %03.3fMpps (%03.3fGbps) ",
+			rte_crypto_cipher_algorithm_strings[cipher_algo],
+			cipher_key_sz * 8,
+			rte_crypto_auth_algorithm_strings[auth_algo],
+			test_data_szs[i],
+			rate, rate  * test_data_szs[i] * 8 / 1000);
+		printf("cycles per buf %03.3f per byte %03.3f\n",
+			cycles_per_buf, cycles_per_buf / test_data_szs[i]);
+
+		for (j = 0; j < MAX_NUM_OPS_INFLIGHT; j++) {
+			tcase = ut_params->test_datas[j];
+
+			switch_blockcipher_enc_to_dec(&tdata, tcase,
+					ciphertext);
+			ret = assemble_blockcipher_buf(tcase, obj, j,
+					op_mask_opp,
+					&tdata,
+					0);
+			if (ret < 0) {
+				printf("Test is not supported by the driver\n");
+				return ret;
+			}
+		}
+
+		time_start = rte_get_timer_cycles();
+
+		run_test(ts_params->ctx, ut_params->sess, obj,
+				MAX_NUM_OPS_INFLIGHT);
+
+		time_now = rte_get_timer_cycles();
+
+		rate = time_now - time_start;
+		cycles_per_buf = rate / MAX_NUM_OPS_INFLIGHT;
+
+		rate = ((hz / cycles_per_buf)) / 1000000;
+
+		printf("%s-%u-%s(%4uB) Dec %03.3fMpps (%03.3fGbps) ",
+			rte_crypto_cipher_algorithm_strings[cipher_algo],
+			cipher_key_sz * 8,
+			rte_crypto_auth_algorithm_strings[auth_algo],
+			test_data_szs[i],
+			rate, rate  * test_data_szs[i] * 8 / 1000);
+		printf("cycles per buf %03.3f per byte %03.3f\n",
+				cycles_per_buf,
+				cycles_per_buf / test_data_szs[i]);
+	}
+
+	return 0;
+}
+
+/* cipher-algo/cipher-key-len/auth-algo/auth-key-len/digest-len/op */
+#define all_block_cipher_perf_test_cases				\
+	TEST_EXPAND(_AES_CBC, 128, _NULL, 0, 0, TOP_ENC)		\
+	TEST_EXPAND(_NULL, 0, _SHA1_HMAC, 160, 20, TOP_AUTH_GEN)	\
+	TEST_EXPAND(_AES_CBC, 128, _SHA1_HMAC, 160, 20, TOP_ENC_AUTH)
+
+#define TEST_EXPAND(a, b, c, d, e, f)					\
+static int								\
+cpu_crypto_blockcipher_perf##a##_##b##c##_##f(void)			\
+{									\
+	return cpu_crypto_test_blockcipher_perf(RTE_CRYPTO_CIPHER##a,	\
+			b / 8, RTE_CRYPTO_AUTH##c, d / 8, e, f);	\
+}									\
+
+all_block_cipher_perf_test_cases
+#undef TEST_EXPAND
+
+static struct unit_test_suite security_cpu_crypto_aesni_mb_perf_testsuite  = {
+	.suite_name = "Security CPU Crypto AESNI-MB Perf Test Suite",
+	.setup = testsuite_setup,
+	.teardown = testsuite_teardown,
+	.unit_test_cases = {
+#define TEST_EXPAND(a, b, c, d, e, f)					\
+	TEST_CASE_ST(ut_setup, ut_teardown,				\
+		cpu_crypto_blockcipher_perf##a##_##b##c##_##f),	\
+
+	all_block_cipher_perf_test_cases
+#undef TEST_EXPAND
+
+	TEST_CASES_END() /**< NULL terminate unit test array */
+	},
+};
+
+static int
+test_security_cpu_crypto_aesni_mb_perf(void)
+{
+	gbl_driver_id =	rte_cryptodev_driver_id_get(
+			RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD));
+
+	return unit_test_suite_runner(
+			&security_cpu_crypto_aesni_mb_perf_testsuite);
+}
+
+
 REGISTER_TEST_COMMAND(security_aesni_gcm_autotest,
 		test_security_cpu_crypto_aesni_gcm);
 
@@ -1130,3 +1321,6 @@ REGISTER_TEST_COMMAND(security_aesni_gcm_perftest,
 
 REGISTER_TEST_COMMAND(security_aesni_mb_autotest,
 		test_security_cpu_crypto_aesni_mb);
+
+REGISTER_TEST_COMMAND(security_aesni_mb_perftest,
+		test_security_cpu_crypto_aesni_mb_perf);
Signature · 1 line
2.14.5
author
#83 lore raw

PATCH v2 8/10 ipsec: add rte_security cpu_crypto action support

In reply to Fan Zhang

This patch updates the ipsec library to handle the newly introduced RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO action.

---
 lib/librte_ipsec/crypto.h   |  24 +++
 lib/librte_ipsec/esp_inb.c  | 200 ++++++++++++++++++++++--
 lib/librte_ipsec/esp_outb.c | 369 +++++++++++++++++++++++++++++++++++++++++---
 lib/librte_ipsec/sa.c       |  53 ++++++-
 lib/librte_ipsec/sa.h       |  29 ++++
 lib/librte_ipsec/ses.c      |   4 +-
 6 files changed, 643 insertions(+), 36 deletions(-)
lib/librte_ipsec/crypto.h +24 -0
index f8fbf8d4f..901c8c7de 100644
@@ -179,4 +179,28 @@ lksd_none_cop_prepare(struct rte_crypto_op *cop,
 	__rte_crypto_sym_op_attach_sym_session(sop, cs);
 }
 
+typedef void* (*_set_icv_f)(void *val, struct rte_mbuf *ml, uint32_t icv_off);
+
+static inline void *
+set_icv_va_pa(void *val, struct rte_mbuf *ml, uint32_t icv_off)
+{
+	union sym_op_data *icv = val;
+
+	icv->va = rte_pktmbuf_mtod_offset(ml, void *, icv_off);
+	icv->pa = rte_pktmbuf_iova_offset(ml, icv_off);
+
+	return icv->va;
+}
+
+static inline void *
+set_icv_va(__rte_unused void *val, __rte_unused struct rte_mbuf *ml,
+		__rte_unused uint32_t icv_off)
+{
+	void **icv_va = val;
+
+	*icv_va = rte_pktmbuf_mtod_offset(ml, void *, icv_off);
+
+	return *icv_va;
+}
+
 #endif /* _CRYPTO_H_ */
lib/librte_ipsec/esp_inb.c +191 -9

265 lines not on this page · raw message

lib/librte_ipsec/esp_outb.c +350 -19

451 lines not on this page · raw message

lib/librte_ipsec/sa.c +46 -7

80 lines not on this page · raw message

lib/librte_ipsec/sa.h +29 -0
index 51e69ad05..770d36b8b 100644
@@ -156,6 +156,14 @@ uint16_t
 inline_inb_trs_pkt_process(const struct rte_ipsec_session *ss,
 	struct rte_mbuf *mb[], uint16_t num);
 
+uint16_t
+esp_inb_tun_cpu_crypto_pkt_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
+uint16_t
+esp_inb_trs_cpu_crypto_pkt_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
 /* outbound processing */
 
 uint16_t
@@ -170,6 +178,10 @@ uint16_t
 esp_outb_sqh_process(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
 	uint16_t num);
 
+uint16_t
+esp_outb_pkt_flag_process(const struct rte_ipsec_session *ss,
+	struct rte_mbuf *mb[], uint16_t num);
+
 uint16_t
 inline_outb_tun_pkt_process(const struct rte_ipsec_session *ss,
 	struct rte_mbuf *mb[], uint16_t num);
@@ -182,4 +194,21 @@ uint16_t
 inline_proto_outb_pkt_process(const struct rte_ipsec_session *ss,
 	struct rte_mbuf *mb[], uint16_t num);
 
+uint16_t
+esp_outb_tun_cpu_crypto_sqh_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
+uint16_t
+esp_outb_tun_cpu_crypto_flag_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
+uint16_t
+esp_outb_trs_cpu_crypto_sqh_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
+uint16_t
+esp_outb_trs_cpu_crypto_flag_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num);
+
+
 #endif /* _SA_H_ */
lib/librte_ipsec/ses.c +3 -1
index 82c765a33..eaa8c17b7 100644
@@ -19,7 +19,9 @@ session_check(struct rte_ipsec_session *ss)
 			return -EINVAL;
 		if ((ss->type == RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO ||
 				ss->type ==
-				RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL) &&
+				RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL ||
+				ss->type ==
+				RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) &&
 				ss->security.ctx == NULL)
 			return -EINVAL;
 	}
Signature · 1 line
2.14.5
#84 lore raw

Re: [dpdk-dev] [PATCH v2 08/10] ipsec: add rte_security cpu_crypto action support

Hi Fan,
Comments for inbound part inline.
As I can see majority of my v1 comments still are not addressed. Please check.
Konstantin

Quoted 59 lines

This patch updates the ipsec library to handle the newly introduced RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO action.

---
 lib/librte_ipsec/crypto.h   |  24 +++
 lib/librte_ipsec/esp_inb.c  | 200 ++++++++++++++++++++++--
 lib/librte_ipsec/esp_outb.c | 369 +++++++++++++++++++++++++++++++++++++++++---
 lib/librte_ipsec/sa.c       |  53 ++++++-
 lib/librte_ipsec/sa.h       |  29 ++++
 lib/librte_ipsec/ses.c      |   4 +-
 6 files changed, 643 insertions(+), 36 deletions(-)
lib/librte_ipsec/crypto.h +24 -0
index f8fbf8d4f..901c8c7de 100644
@@ -179,4 +179,28 @@ lksd_none_cop_prepare(struct rte_crypto_op *cop,
 	__rte_crypto_sym_op_attach_sym_session(sop, cs);
 }
 
+typedef void* (*_set_icv_f)(void *val, struct rte_mbuf *ml, uint32_t icv_off);
+
+static inline void *
+set_icv_va_pa(void *val, struct rte_mbuf *ml, uint32_t icv_off)
+{
+	union sym_op_data *icv = val;
+
+	icv->va = rte_pktmbuf_mtod_offset(ml, void *, icv_off);
+	icv->pa = rte_pktmbuf_iova_offset(ml, icv_off);
+
+	return icv->va;
+}
+
+static inline void *
+set_icv_va(__rte_unused void *val, __rte_unused struct rte_mbuf *ml,
+		__rte_unused uint32_t icv_off)
+{
+	void **icv_va = val;
+
+	*icv_va = rte_pktmbuf_mtod_offset(ml, void *, icv_off);
+
+	return *icv_va;
+}
+
 #endif /* _CRYPTO_H_ */
lib/librte_ipsec/esp_inb.c +4 -0
index 8e3ecbc64..c4476e819 100644
@@ -105,6 +105,78 @@ inb_cop_prepare(struct rte_crypto_op *cop,
 	}
 }
 
+static inline int
+inb_cpu_crypto_proc_prepare(const struct rte_ipsec_sa *sa, struct rte_mbuf *mb,
+	uint32_t pofs, uint32_t plen,
+	struct rte_security_vec *buf, struct iovec *cur_vec,

lib/librte_ipsec/esp_inb.c inb_cop_prepare(struct rte_crypto_op *cop, · line 121

Quoted 10 lines
lib/librte_ipsec/esp_inb.c +10 -0
+	void *iv)
+{
+	struct rte_mbuf *ms;
+	struct iovec *vec = cur_vec;
+	struct aead_gcm_iv *gcm;
+	struct aesctr_cnt_blk *ctr;
+	uint64_t *ivp;
+	uint32_t algo;
+	uint32_t left;
+	uint32_t off = 0, n_seg = 0;

Same comment as for v1:
Please separate variable definition and value assignment.
It makes it hard to read, plus we don't do that in the rest of the library, so better to follow rest of the code style.

Quoted 20 lines
lib/librte_ipsec/esp_inb.c +20 -0
+
+	ivp = rte_pktmbuf_mtod_offset(mb, uint64_t *,
+		pofs + sizeof(struct rte_esp_hdr));
+	algo = sa->algo_type;
+
+	switch (algo) {
+	case ALGO_TYPE_AES_GCM:
+		gcm = (struct aead_gcm_iv *)iv;
+		aead_gcm_iv_fill(gcm, ivp[0], sa->salt);
+		off = sa->ctp.cipher.offset + pofs;
+		left = plen - sa->ctp.cipher.length;
+		break;
+	case ALGO_TYPE_AES_CBC:
+	case ALGO_TYPE_3DES_CBC:
+		copy_iv(iv, ivp, sa->iv_len);
+		off = sa->ctp.auth.offset + pofs;
+		left = plen - sa->ctp.auth.length;
+		break;
+	case ALGO_TYPE_AES_CTR:
+		copy_iv(iv, ivp, sa->iv_len);

lib/librte_ipsec/esp_inb.c inb_cop_prepare(struct rte_crypto_op *cop, · line 151

Quoted 10 lines
lib/librte_ipsec/esp_inb.c +10 -0
+		off = sa->ctp.auth.offset + pofs;
+		left = plen - sa->ctp.auth.length;
+		ctr = (struct aesctr_cnt_blk *)iv;
+		aes_ctr_cnt_blk_fill(ctr, ivp[0], sa->salt);
+		break;
+	case ALGO_TYPE_NULL:
+		left = plen - sa->ctp.cipher.length;
+		break;
+	default:
+		return -EINVAL;

How we can endup here?
If we have an unknown algorithm, shouldn't we fail at init stage?

lib/librte_ipsec/esp_inb.c inb_cop_prepare(struct rte_crypto_op *cop, · line 156

Quoted 5 lines
lib/librte_ipsec/esp_inb.c +5 -0
+	}
+
+	ms = mbuf_get_seg_ofs(mb, &off);
+	if (!ms)
+		return -1;

Same comment as for v1:
inb_pkt_prepare() should already check that we have a valid packet. I don't think there is a need to check for any failure here. Another thing, our esp header will be in the first segment for sure, so do we need get_seg_ofs() here at all?

lib/librte_ipsec/esp_inb.c inb_cop_prepare(struct rte_crypto_op *cop, · line 159

Quoted 3 lines
lib/librte_ipsec/esp_inb.c +3 -0
+
+	while (n_seg < RTE_LIBRTE_IP_FRAG_MAX_FRAG && left && ms) {
+		uint32_t len = RTE_MIN(left, ms->data_len - off);

Again, same comments as for v1:

- I don't think this is right, we shouldn't impose additional limitations to the number of segments in the packet.

- Whole construction seems a bit over-complicated here... Why just not have a separate function that would dill iovec[] from mbuf And return an error if there is not enough iovec[] entries? Something like:

static inline int
mbuf_to_iovec(const struct rte_mbuf *mb, uint32_t ofs, uint32_t len, struct iovec vec[], uint32_t num)

{
     uint32_t i;
     if (mb->nb_seg > num)
        return - mb->nb_seg;

    vec[0].iov_base =  rte_pktmbuf_mtod_offset(mb, void *, off);
    vec[0].iov_len = mb->data_len - off;

    for (i = 1, ms = mb->next; mb != NULL; ms = ms->next, i++) {
        vec[i].iov_base = rte_pktmbuf_mtod(ms);
        vec[i].iov_len = ms->data_len;
    }

vec[i].iov_len -= mb->pkt_len - len;
return i;
}

Then we can use that function to fill our iovec[] in a loop.

L- ooking at this function, it seems to consist of 2 separate parts:
1. calculates offset and generates iv
2. setup iovec[].
Probably worth to split it into 2 separate functions like that. Would be much easier to read/understand.

Quoted 49 lines
lib/librte_ipsec/esp_inb.c +24 -3
+
+		vec->iov_base = rte_pktmbuf_mtod_offset(ms, void *, off);
+		vec->iov_len = len;
+
+		left -= len;
+		vec++;
+		n_seg++;
+		ms = ms->next;
+		off = 0;
+	}
+
+	if (left)
+		return -1;
+
+	buf->vec = cur_vec;
+	buf->num = n_seg;
+
+	return n_seg;
+}
+
 /*
  * Helper function for prepare() to deal with situation when
  * ICV is spread by two segments. Tries to move ICV completely into the
@@ -139,20 +211,21 @@ move_icv(struct rte_mbuf *ml, uint32_t ofs)
  */
 static inline void
 inb_pkt_xprepare(const struct rte_ipsec_sa *sa, rte_be64_t sqc,
-	const union sym_op_data *icv)
+	uint8_t *icv_va, void *aad_buf, uint32_t aad_off)
 {
 	struct aead_gcm_aad *aad;
 
 	/* insert SQN.hi between ESP trailer and ICV */
 	if (sa->sqh_len != 0)
-		insert_sqh(sqn_hi32(sqc), icv->va, sa->icv_len);
+		insert_sqh(sqn_hi32(sqc), icv_va, sa->icv_len);
 
 	/*
 	 * fill AAD fields, if any (aad fields are placed after icv),
 	 * right now we support only one AEAD algorithm: AES-GCM.
 	 */
 	if (sa->aad_len != 0) {
-		aad = (struct aead_gcm_aad *)(icv->va + sa->icv_len);
+		aad = aad_buf ? aad_buf :
+				(struct aead_gcm_aad *)(icv_va + aad_off);
 		aead_gcm_aad_fill(aad, sa->spi, sqc, IS_ESN(sa));
 	}
 }

lib/librte_ipsec/esp_inb.c inb_pkt_xprepare(const struct rte_ipsec_sa *sa, rte_be64_t sqc, · line 239

Quoted 6 lines
lib/librte_ipsec/esp_inb.c +2 -1
@@ -162,13 +235,15 @@ inb_pkt_xprepare(const struct rte_ipsec_sa *sa, rte_be64_t sqc,
  */
 static inline int32_t
 inb_pkt_prepare(const struct rte_ipsec_sa *sa, const struct replay_sqn *rsn,
-	struct rte_mbuf *mb, uint32_t hlen, union sym_op_data *icv)
+	struct rte_mbuf *mb, uint32_t hlen, _set_icv_f set_icv, void *icv_val,
+	void *aad_buf)

This whole construct with another function pointer , overloaded arguments, etc., looks a bit clumsy and overcomplicated.
I think it would be much cleaner and easier to re-arrange the code like that:

1. update inb_pkt_xprepare to take aad buffer pointer and aad len as a parameters:

static inline void
inb_pkt_xprepare(const struct rte_ipsec_sa *sa, rte_be64_t sqc,
const union sym_op_data *icv, void *aad, uint32_t aad_len)

{
        /* insert SQN.hi between ESP trailer and ICV */
        if (sa->sqh_len != 0)
                insert_sqh(sqn_hi32(sqc), icv->va, sa->icv_len);

        /*
         * fill AAD fields, if any (aad fields are placed after icv),
         * right now we support only one AEAD algorithm: AES-GCM.
         */
        if (aad_len != 0)
                aead_gcm_aad_fill(aad, sa->spi, sqc, IS_ESN(sa));
}

2. split inb_pkt_prepare() into 2 reusable helper functions:

*
* retrieve and reconstruct SQN, then check it, then
* convert it back into network byte order.
*/
static inline int
inb_get_sqn(const struct rte_ipsec_sa *sa, const struct replay_sqn *rsn,
struct rte_mbuf *mb, uint32_t hlen, rte_be64_t *sqc)

{
        int32_t rc;
        uint64_t sqn;
        struct rte_esp_hdr *esph;

        esph = rte_pktmbuf_mtod_offset(mb, struct rte_esp_hdr *, hlen);

        /*
         * retrieve and reconstruct SQN, then check it, then
         * convert it back into network byte order.
         */
        sqn = rte_be_to_cpu_32(esph->seq);
        if (IS_ESN(sa))
                sqn = reconstruct_esn(rsn->sqn, sqn, sa->replay.win_sz);

        rc = esn_inb_check_sqn(rsn, sa, sqn);
        if (rc == 0)
                *sqc = rte_cpu_to_be_64(sqn);

        return rc;
}

static inline int32_t
inb_prepare(const struct rte_ipsec_sa *sa, struct rte_mbuf *mb,
uint32_t hlen, uint32_t aad_len, union sym_op_data *icv)

{
        uint32_t clen, icv_len, icv_ofs, plen;
        struct rte_mbuf *ml;

        /* start packet manipulation */
        plen = mb->pkt_len;
        plen = plen - hlen;

        /* check that packet has a valid length */
        clen = plen - sa->ctp.cipher.length;
        if ((int32_t)clen < 0 || (clen & (sa->pad_align - 1)) != 0)
                return -EBADMSG;

        /* find ICV location */
        icv_len = sa->icv_len;
        icv_ofs = mb->pkt_len - icv_len;

        ml = mbuf_get_seg_ofs(mb, &icv_ofs);

        /*
         * if ICV is spread by two segments, then try to
         * move ICV completely into the last segment.
         */
         if (ml->data_len < icv_ofs + icv_len) {

                ml = move_icv(ml, icv_ofs);
                if (ml == NULL)
                        return -ENOSPC;

                /* new ICV location */
                icv_ofs = 0;
        }

        icv_ofs += sa->sqh_len;

        /* we have to allocate space for AAD somewhere,
         * right now - just use free trailing space at the last segment.
         * Would probably be more convenient to reserve space for AAD
         * inside rte_crypto_op itself
         * (again for IV space is already reserved inside cop).
         */
        if (aad_len + sa->sqh_len > rte_pktmbuf_tailroom(ml))
                return -ENOSPC;

        icv->va = rte_pktmbuf_mtod_offset(ml, void *, icv_ofs);
        icv->pa = rte_pktmbuf_iova_offset(ml, icv_ofs);

        /*
         * if esn is used then high-order 32 bits are also used in ICV
         * calculation but are not transmitted, update packet length
         * to be consistent with auth data length and offset, this will
         * be subtracted from packet length in post crypto processing
         */
        mb->pkt_len += sa->sqh_len;
        ml->data_len += sa->sqh_len;

        return plen;
}      

3. Now inb_pkt_prepare() becomes a simple sequential invocation of these 3 sub-functions with right parameters:

static inline int32_t
inb_pkt_prepare(const struct rte_ipsec_sa *sa, const struct replay_sqn *rsn,
struct rte_mbuf *mb, uint32_t hlen, union sym_op_data *icv)

{
        int32_t rc;
        uint64_t sqn;
        
        rc = inb_get_sqn(sa, rsn, mb, hlen, &sqn);
        if (rc != 0)
                return rc;

        rc = inb_prepare(sa, mb, hlen, sa->aad_len, icv);
        if (rc < 0)
                return rc;

        inb_pkt_xprepare(sa, sqn, icv, icv->va + sa->icv_len, sa->aad_len);
        return rc;
}

4. And that would be version of inb_pkt_prepare for cpu_cypto:

static inline int32_t
inb_cpu_prepare(const struct rte_ipsec_sa *sa, const struct replay_sqn *rsn,
struct rte_mbuf *mb, uint32_t hlen, union sym_op_data *icv, void *aad)

{
        int32_t rc;
        uint64_t sqn;
        
        rc = inb_get_sqn(sa, rsn, mb, hlen, &sqn);
        if (rc != 0)
                return rc;

        rc = inb_prepare(sa, mb, hlen, 0, icv);
        if (rc < 0)
                return rc;

        inb_pkt_xprepare(sa, sqn, icv, aad, sa->aad_len);
        return rc;
}
Quoted 61 lines
lib/librte_ipsec/esp_inb.c +15 -5
 {
 	int32_t rc;
 	uint64_t sqn;
 	uint32_t clen, icv_len, icv_ofs, plen;
 	struct rte_mbuf *ml;
 	struct rte_esp_hdr *esph;
+	void *icv_va;
 
 	esph = rte_pktmbuf_mtod_offset(mb, struct rte_esp_hdr *, hlen);
 
@@ -226,8 +301,8 @@ inb_pkt_prepare(const struct rte_ipsec_sa *sa, const struct replay_sqn *rsn,
 	if (sa->aad_len + sa->sqh_len > rte_pktmbuf_tailroom(ml))
 		return -ENOSPC;
 
-	icv->va = rte_pktmbuf_mtod_offset(ml, void *, icv_ofs);
-	icv->pa = rte_pktmbuf_iova_offset(ml, icv_ofs);
+	icv_va = set_icv(icv_val, ml, icv_ofs);
+	inb_pkt_xprepare(sa, sqn, icv_va, aad_buf, sa->icv_len);
 
 	/*
 	 * if esn is used then high-order 32 bits are also used in ICV
@@ -238,7 +313,6 @@ inb_pkt_prepare(const struct rte_ipsec_sa *sa, const struct replay_sqn *rsn,
 	mb->pkt_len += sa->sqh_len;
 	ml->data_len += sa->sqh_len;
 
-	inb_pkt_xprepare(sa, sqn, icv);
 	return plen;
 }
 
@@ -265,7 +339,8 @@ esp_inb_pkt_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[],
 	for (i = 0; i != num; i++) {
 
 		hl = mb[i]->l2_len + mb[i]->l3_len;
-		rc = inb_pkt_prepare(sa, rsn, mb[i], hl, &icv);
+		rc = inb_pkt_prepare(sa, rsn, mb[i], hl, set_icv_va_pa,
+				(void *)&icv, NULL);
 		if (rc >= 0) {
 			lksd_none_cop_prepare(cop[k], cs, mb[i]);
 			inb_cop_prepare(cop[k], sa, mb[i], &icv, hl, rc);
@@ -512,7 +587,6 @@ tun_process(const struct rte_ipsec_sa *sa, struct rte_mbuf *mb[],
 	return k;
 }
 
-
 /*
  * *process* function for tunnel packets
  */
@@ -625,6 +699,114 @@ esp_inb_pkt_process(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[],
 	return n;
 }
 
+/*
+ * process packets using sync crypto engine
+ */
+static uint16_t
+esp_inb_cpu_crypto_pkt_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num,
+		esp_inb_process_t process)
+{
+	int32_t rc;
+	uint32_t i, hl, n, p;

lib/librte_ipsec/esp_inb.c esp_inb_pkt_process(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], · line 721

Quoted 10 lines
lib/librte_ipsec/esp_inb.c +10 -0
+	struct rte_ipsec_sa *sa;
+	struct replay_sqn *rsn;
+	void *icv_va;
+	uint32_t sqn[num];
+	uint32_t dr[num];
+	uint8_t sqh_len;
+
+	/* cpu crypto specific variables */
+	struct rte_security_vec buf[num];
+	struct iovec vec[RTE_LIBRTE_IP_FRAG_MAX_FRAG * num];

Same comment as for v1:
I don't think this is right, we shouldn't impose additional limitations to the number of segments in the packet.

Quoted 8 lines
lib/librte_ipsec/esp_inb.c +8 -0
+	uint32_t vec_idx = 0;
+	uint64_t iv_buf[num][IPSEC_MAX_IV_QWORD];
+	void *iv[num];
+	int status[num];
+	uint8_t *aad_buf[num][sizeof(struct aead_gcm_aad)];
+	void *aad[num];
+	void *digest[num];
+	uint32_t k;

lib/librte_ipsec/esp_inb.c esp_inb_pkt_process(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], · line 739

Quoted 10 lines
lib/librte_ipsec/esp_inb.c +10 -0
+
+	sa = ss->sa;
+	rsn = rsn_acquire(sa);
+	sqh_len = sa->sqh_len;
+
+	k = 0;
+	for (i = 0; i != num; i++) {
+		hl = mb[i]->l2_len + mb[i]->l3_len;
+		rc = inb_pkt_prepare(sa, rsn, mb[i], hl, set_icv_va,
+				(void *)&icv_va, (void *)aad_buf[k]);

Better to have separate function similar to inb_pkt_prepare(), see above.

lib/librte_ipsec/esp_inb.c esp_inb_pkt_process(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], · line 748

Quoted 9 lines
lib/librte_ipsec/esp_inb.c +9 -0
+		if (rc >= 0) {
+			iv[k] = (void *)iv_buf[k];
+			aad[k] = (void *)aad_buf[k];
+			digest[k] = (void *)icv_va;
+
+			rc = inb_cpu_crypto_proc_prepare(sa, mb[i], hl,
+					rc, &buf[k], &vec[vec_idx], iv[k]);
+			if (rc < 0) {
+				dr[i - k] = i;

I think with you current aproach you can't do like that.
As your vec[] still can contain some entries from fail mbuf. So in theoryyou need to cleanup these entries which is quite complicated, better to avoid such case at all.
I think you need to reorder the code, as described above.

Quoted 14 lines
lib/librte_ipsec/esp_inb.c +14 -0
+				continue;
+			}
+
+			vec_idx += rc;
+			k++;
+		} else
+			dr[i - k] = i;
+	}
+
+	/* copy not prepared mbufs beyond good ones */
+	if (k != num) {
+		rte_errno = EBADMSG;
+
+		if (unlikely(k == 0))

lib/librte_ipsec/esp_inb.c esp_inb_pkt_process(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], · line 772

Quoted 10 lines
lib/librte_ipsec/esp_inb.c +10 -0
+			return 0;
+
+		move_bad_mbufs(mb, dr, num, num - k);
+	}
+
+	/* process the packets */
+	n = 0;
+	rc = rte_security_process_cpu_crypto_bulk(ss->security.ctx,
+			ss->security.ses, buf, iv, aad, digest, status, k);
+	/* move failed process packets to dr */

Same comment as for v1:
That just doesn't look right, instead of updating dr[] and move_bad_mbufs(), you need to:
if (rc != 0) {walk through status[] and for failed ones set PKT_RX_SEC_OFFLOAD_FAILED in appropriate mbuf->ol_flags}. tun_process(), etc. expects PKT_RX_SEC_OFFLOAD_FAILED to be set in mb->ol_flags
for failed packets.

Quoted 40 lines
lib/librte_ipsec/esp_inb.c +37 -0
+	for (i = 0; i < k; i++) {
+		if (status[i]) {
+			dr[n++] = i;
+			rte_errno = EBADMSG;
+		}
+	}
+
+	/* move bad packets to the back */
+	if (n)
+		move_bad_mbufs(mb, dr, k, n);
+
+	/* process packets */
+	p = process(sa, mb, sqn, dr, k - n, sqh_len);
+
 
 
 
+	if (p != k - n && p != 0)
+		move_bad_mbufs(mb, dr, k - n, k - n - p);
+
+	if (p != num)
+		rte_errno = EBADMSG;
+
+	return p;
+}
+
+uint16_t
+esp_inb_tun_cpu_crypto_pkt_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num)
+{
+	return esp_inb_cpu_crypto_pkt_process(ss, mb, num, tun_process);
+}
+
+uint16_t
+esp_inb_trs_cpu_crypto_pkt_process(const struct rte_ipsec_session *ss,
+		struct rte_mbuf *mb[], uint16_t num)
+{
+	return esp_inb_cpu_crypto_pkt_process(ss, mb, num, trs_process);
+}
+
author
#85 lore raw

PATCH v2 9/10 examples/ipsec-secgw: add security cpu_crypto action support

In reply to Fan Zhang

Since ipsec library is added cpu_crypto security action type support, this patch updates ipsec-secgw sample application with added action type "cpu-crypto". The patch also includes a number of test scripts to prove the correctness of the implementation.

---
 examples/ipsec-secgw/ipsec.c                       | 35 ++++++++++++++++++++++
 examples/ipsec-secgw/ipsec_process.c               |  7 +++--
 examples/ipsec-secgw/sa.c                          | 13 ++++++--
 examples/ipsec-secgw/test/run_test.sh              | 10 +++++++
 .../test/trs_3descbc_sha1_common_defs.sh           |  8 ++---
 .../test/trs_3descbc_sha1_cpu_crypto_defs.sh       |  5 ++++
 .../test/trs_aescbc_sha1_common_defs.sh            |  8 ++---
 .../test/trs_aescbc_sha1_cpu_crypto_defs.sh        |  5 ++++
 .../test/trs_aesctr_sha1_common_defs.sh            |  8 ++---
 .../test/trs_aesctr_sha1_cpu_crypto_defs.sh        |  5 ++++
 .../ipsec-secgw/test/trs_aesgcm_cpu_crypto_defs.sh |  5 ++++
 .../test/trs_aesgcm_mb_cpu_crypto_defs.sh          |  7 +++++
 .../test/tun_3descbc_sha1_common_defs.sh           |  8 ++---
 .../test/tun_3descbc_sha1_cpu_crypto_defs.sh       |  5 ++++
 .../test/tun_aescbc_sha1_common_defs.sh            |  8 ++---
 .../test/tun_aescbc_sha1_cpu_crypto_defs.sh        |  5 ++++
 .../test/tun_aesctr_sha1_common_defs.sh            |  8 ++---
 .../test/tun_aesctr_sha1_cpu_crypto_defs.sh        |  5 ++++
 .../ipsec-secgw/test/tun_aesgcm_cpu_crypto_defs.sh |  5 ++++
 .../test/tun_aesgcm_mb_cpu_crypto_defs.sh          |  7 +++++
 20 files changed, 138 insertions(+), 29 deletions(-)
 create mode 100644 examples/ipsec-secgw/test/trs_3descbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aescbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aesctr_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_mb_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_3descbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aescbc_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aesctr_sha1_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_cpu_crypto_defs.sh
 create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_mb_cpu_crypto_defs.sh
examples/ipsec-secgw/ipsec.c +35 -0
index 1145ca1c0..02b9443a8 100644
@@ -10,6 +10,7 @@
 #include <rte_crypto.h>
 #include <rte_security.h>
 #include <rte_cryptodev.h>
+#include <rte_ipsec.h>
 #include <rte_ethdev.h>
 #include <rte_mbuf.h>
 #include <rte_hash.h>
@@ -51,6 +52,19 @@ set_ipsec_conf(struct ipsec_sa *sa, struct rte_security_ipsec_xform *ipsec)
 	ipsec->esn_soft_limit = IPSEC_OFFLOAD_ESN_SOFTLIMIT;
 }
 
+static int32_t
+compute_cipher_offset(struct ipsec_sa *sa)
+{
+	int32_t offset;
+
+	if (sa->aead_algo == RTE_CRYPTO_AEAD_AES_GCM)
+		return 0;
+
+	offset = (sa->iv_len + sizeof(struct rte_esp_hdr));
+
+	return offset;
+}
+
 int
 create_lookaside_session(struct ipsec_ctx *ipsec_ctx, struct ipsec_sa *sa)
 {
@@ -117,6 +131,25 @@ create_lookaside_session(struct ipsec_ctx *ipsec_ctx, struct ipsec_sa *sa)
 				"SEC Session init failed: err: %d\n", ret);
 				return -1;
 			}
+		} else if (sa->type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) {
+			struct rte_security_ctx *ctx =
+				(struct rte_security_ctx *)
+				rte_cryptodev_get_sec_ctx(
+					ipsec_ctx->tbl[cdev_id_qp].id);
+
+			/* Set IPsec parameters in conf */
+			sess_conf.cpucrypto.cipher_offset =
+					compute_cipher_offset(sa);
+
+			set_ipsec_conf(sa, &(sess_conf.ipsec));
+			sa->security_ctx = ctx;
+			sa->sec_session = rte_security_session_create(ctx,
+				&sess_conf, ipsec_ctx->session_priv_pool);
+			if (sa->sec_session == NULL) {
+				RTE_LOG(ERR, IPSEC,
+				"SEC Session init failed: err: %d\n", ret);
+				return -1;
+			}
 		} else {
 			RTE_LOG(ERR, IPSEC, "Inline not supported\n");
 			return -1;
@@ -512,6 +545,8 @@ ipsec_enqueue(ipsec_xform_fn xform_func, struct ipsec_ctx *ipsec_ctx,
 						sa->security_ctx,
 						sa->sec_session, pkts[i], NULL);
 			continue;
+		default:
+			continue;
 		}
 
 		RTE_ASSERT(sa->cdev_id_qp < ipsec_ctx->nb_qps);
examples/ipsec-secgw/ipsec_process.c +4 -3
index 868f1a28d..1932b631f 100644
@@ -101,7 +101,8 @@ fill_ipsec_session(struct rte_ipsec_session *ss, struct ipsec_ctx *ctx,
 		}
 		ss->crypto.ses = sa->crypto_session;
 	/* setup session action type */
-	} else if (sa->type == RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL) {
+	} else if (sa->type == RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL ||
+			sa->type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) {
 		if (sa->sec_session == NULL) {
 			rc = create_lookaside_session(ctx, sa);
 			if (rc != 0)
@@ -227,8 +228,8 @@ ipsec_process(struct ipsec_ctx *ctx, struct ipsec_traffic *trf)
 
 		/* process packets inline */
 		else if (sa->type == RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO ||
-				sa->type ==
-				RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL) {
+			sa->type == RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL ||
+			sa->type == RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) {
 
 			satp = rte_ipsec_sa_type(ips->sa);
 
examples/ipsec-secgw/sa.c +11 -2
index c3cf3bd1f..ba773346f 100644
@@ -570,6 +570,9 @@ parse_sa_tokens(char **tokens, uint32_t n_tokens,
 				RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL;
 			else if (strcmp(tokens[ti], "no-offload") == 0)
 				rule->type = RTE_SECURITY_ACTION_TYPE_NONE;
+			else if (strcmp(tokens[ti], "cpu-crypto") == 0)
+				rule->type =
+					RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO;
 			else {
 				APP_CHECK(0, status, "Invalid input \"%s\"",
 						tokens[ti]);
@@ -624,10 +627,13 @@ parse_sa_tokens(char **tokens, uint32_t n_tokens,
 	if (status->status < 0)
 		return;
 
-	if ((rule->type != RTE_SECURITY_ACTION_TYPE_NONE) && (portid_p == 0))
+	if ((rule->type != RTE_SECURITY_ACTION_TYPE_NONE && rule->type !=
+			RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO) &&
+			(portid_p == 0))
 		printf("Missing portid option, falling back to non-offload\n");
 
-	if (!type_p || !portid_p) {
+	if (!type_p || (!portid_p && rule->type !=
+			RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO)) {
 		rule->type = RTE_SECURITY_ACTION_TYPE_NONE;
 		rule->portid = -1;
 	}
@@ -709,6 +715,9 @@ print_one_sa_rule(const struct ipsec_sa *sa, int inbound)
 	case RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL:
 		printf("lookaside-protocol-offload ");
 		break;
+	case RTE_SECURITY_ACTION_TYPE_CPU_CRYPTO:
+		printf("cpu-crypto-accelerated");
+		break;
 	}
 	printf("\n");
 }
examples/ipsec-secgw/test/run_test.sh +10 -0
index 8055a4c04..bcaf91715 100755
@@ -32,15 +32,21 @@ usage()
 }
 
 LINUX_TEST="tun_aescbc_sha1 \
+tun_aescbc_sha1_cpu_crypto \
 tun_aescbc_sha1_esn \
 tun_aescbc_sha1_esn_atom \
 tun_aesgcm \
+tun_aesgcm_cpu_crypto \
+tun_aesgcm_mb_cpu_crypto \
 tun_aesgcm_esn \
 tun_aesgcm_esn_atom \
 trs_aescbc_sha1 \
+trs_aescbc_sha1_cpu_crypto \
 trs_aescbc_sha1_esn \
 trs_aescbc_sha1_esn_atom \
 trs_aesgcm \
+trs_aesgcm_cpu_crypto \
+trs_aesgcm_mb_cpu_crypto \
 trs_aesgcm_esn \
 trs_aesgcm_esn_atom \
 tun_aescbc_sha1_old \
@@ -49,17 +55,21 @@ trs_aescbc_sha1_old \
 trs_aesgcm_old \
 tun_aesctr_sha1 \
 tun_aesctr_sha1_old \
+tun_aesctr_sha1_cpu_crypto \
 tun_aesctr_sha1_esn \
 tun_aesctr_sha1_esn_atom \
 trs_aesctr_sha1 \
+trs_aesctr_sha1_cpu_crypto \
 trs_aesctr_sha1_old \
 trs_aesctr_sha1_esn \
 trs_aesctr_sha1_esn_atom \
 tun_3descbc_sha1 \
+tun_3descbc_sha1_cpu_crypto \
 tun_3descbc_sha1_old \
 tun_3descbc_sha1_esn \
 tun_3descbc_sha1_esn_atom \
 trs_3descbc_sha1 \
+trs_3descbc_sha1_cpu_crypto \
 trs_3descbc_sha1_old \
 trs_3descbc_sha1_esn \
 trs_3descbc_sha1_esn_atom"
examples/ipsec-secgw/test/trs_3descbc_sha1_common_defs.sh +4 -4
index bb4cef6a9..eda2ddf0c 100644
@@ -32,14 +32,14 @@ cipher_key \
 de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode transport
+mode transport ${SGW_CFG_XPRM}
 
 sa in 9 cipher_algo 3des-cbc \
 cipher_key \
 de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode transport
+mode transport ${SGW_CFG_XPRM}
 
 #SA out rules
 sa out 7 cipher_algo 3des-cbc \
@@ -47,7 +47,7 @@ cipher_key \
 de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode transport
+mode transport ${SGW_CFG_XPRM}
 
 #SA out rules
 sa out 9 cipher_algo 3des-cbc \
@@ -55,7 +55,7 @@ cipher_key \
 de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode transport
+mode transport ${SGW_CFG_XPRM}
 
 #Routing rules
 rt ipv4 dst ${REMOTE_IPV4}/32 port 0
examples/ipsec-secgw/test/trs_3descbc_sha1_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..a864a8886
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/trs_3descbc_sha1_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/trs_aescbc_sha1_common_defs.sh +4 -4
index e2621e0df..49b7b0713 100644
@@ -31,27 +31,27 @@ sa in 7 cipher_algo aes-128-cbc \
 cipher_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode transport
+mode transport ${SGW_CFG_XPRM}
 
 sa in 9 cipher_algo aes-128-cbc \
 cipher_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode transport
+mode transport ${SGW_CFG_XPRM}
 
 #SA out rules
 sa out 7 cipher_algo aes-128-cbc \
 cipher_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode transport
+mode transport ${SGW_CFG_XPRM}
 
 #SA out rules
 sa out 9 cipher_algo aes-128-cbc \
 cipher_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode transport
+mode transport ${SGW_CFG_XPRM}
 
 #Routing rules
 rt ipv4 dst ${REMOTE_IPV4}/32 port 0
examples/ipsec-secgw/test/trs_aescbc_sha1_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..b515cd9f8
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/trs_aescbc_sha1_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/trs_aesctr_sha1_common_defs.sh +4 -4
index 9c213e3cc..428322307 100644
@@ -31,27 +31,27 @@ sa in 7 cipher_algo aes-128-ctr \
 cipher_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode transport
+mode transport ${SGW_CFG_XPRM}
 
 sa in 9 cipher_algo aes-128-ctr \
 cipher_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode transport
+mode transport ${SGW_CFG_XPRM}
 
 #SA out rules
 sa out 7 cipher_algo aes-128-ctr \
 cipher_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode transport
+mode transport ${SGW_CFG_XPRM}
 
 #SA out rules
 sa out 9 cipher_algo aes-128-ctr \
 cipher_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode transport
+mode transport ${SGW_CFG_XPRM}
 
 #Routing rules
 rt ipv4 dst ${REMOTE_IPV4}/32 port 0
examples/ipsec-secgw/test/trs_aesctr_sha1_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..745a2a02b
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/trs_aesctr_sha1_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/trs_aesgcm_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..8917122da
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/trs_aesgcm_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/trs_aesgcm_mb_cpu_crypto_defs.sh new file +7 -0
new file mode 100644
index 000000000..26943321f
@@ -0,0 +1,7 @@
+#! /bin/bash
+
+. ${DIR}/trs_aesgcm_defs.sh
+
+CRYPTO_DEV=${CRYPTO_DEV:-'--vdev="crypto_aesni_mb0"'}
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/tun_3descbc_sha1_common_defs.sh +4 -4
index dd802d6be..a583ef605 100644
@@ -32,14 +32,14 @@ cipher_key \
 de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode ipv4-tunnel src ${REMOTE_IPV4} dst ${LOCAL_IPV4}
+mode ipv4-tunnel src ${REMOTE_IPV4} dst ${LOCAL_IPV4} ${SGW_CFG_XPRM}
 
 sa in 9 cipher_algo 3des-cbc \
 cipher_key \
 de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode ipv6-tunnel src ${REMOTE_IPV6} dst ${LOCAL_IPV6}
+mode ipv6-tunnel src ${REMOTE_IPV6} dst ${LOCAL_IPV6} ${SGW_CFG_XPRM}
 
 #SA out rules
 sa out 7 cipher_algo 3des-cbc \
@@ -47,14 +47,14 @@ cipher_key \
 de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode ipv4-tunnel src ${LOCAL_IPV4} dst ${REMOTE_IPV4}
+mode ipv4-tunnel src ${LOCAL_IPV4} dst ${REMOTE_IPV4} ${SGW_CFG_XPRM}
 
 sa out 9 cipher_algo 3des-cbc \
 cipher_key \
 de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode ipv6-tunnel src ${LOCAL_IPV6} dst ${REMOTE_IPV6}
+mode ipv6-tunnel src ${LOCAL_IPV6} dst ${REMOTE_IPV6} ${SGW_CFG_XPRM}
 
 #Routing rules
 rt ipv4 dst ${REMOTE_IPV4}/32 port 0
examples/ipsec-secgw/test/tun_3descbc_sha1_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..747141f62
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/tun_3descbc_sha1_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/tun_aescbc_sha1_common_defs.sh +4 -4
index 4025da232..ac0232d2c 100644
@@ -31,26 +31,26 @@ sa in 7 cipher_algo aes-128-cbc \
 cipher_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode ipv4-tunnel src ${REMOTE_IPV4} dst ${LOCAL_IPV4}
+mode ipv4-tunnel src ${REMOTE_IPV4} dst ${LOCAL_IPV4} ${SGW_CFG_XPRM}
 
 sa in 9 cipher_algo aes-128-cbc \
 cipher_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode ipv6-tunnel src ${REMOTE_IPV6} dst ${LOCAL_IPV6}
+mode ipv6-tunnel src ${REMOTE_IPV6} dst ${LOCAL_IPV6} ${SGW_CFG_XPRM}
 
 #SA out rules
 sa out 7 cipher_algo aes-128-cbc \
 cipher_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode ipv4-tunnel src ${LOCAL_IPV4} dst ${REMOTE_IPV4}
+mode ipv4-tunnel src ${LOCAL_IPV4} dst ${REMOTE_IPV4} ${SGW_CFG_XPRM}
 
 sa out 9 cipher_algo aes-128-cbc \
 cipher_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode ipv6-tunnel src ${LOCAL_IPV6} dst ${REMOTE_IPV6}
+mode ipv6-tunnel src ${LOCAL_IPV6} dst ${REMOTE_IPV6} ${SGW_CFG_XPRM}
 
 #Routing rules
 rt ipv4 dst ${REMOTE_IPV4}/32 port 0
examples/ipsec-secgw/test/tun_aescbc_sha1_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..56076fa50
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/tun_aescbc_sha1_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/tun_aesctr_sha1_common_defs.sh +4 -4
index a3ac3a698..523c396c9 100644
@@ -31,26 +31,26 @@ sa in 7 cipher_algo aes-128-ctr \
 cipher_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode ipv4-tunnel src ${REMOTE_IPV4} dst ${LOCAL_IPV4}
+mode ipv4-tunnel src ${REMOTE_IPV4} dst ${LOCAL_IPV4} ${SGW_CFG_XPRM}
 
 sa in 9 cipher_algo aes-128-ctr \
 cipher_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode ipv6-tunnel src ${REMOTE_IPV6} dst ${LOCAL_IPV6}
+mode ipv6-tunnel src ${REMOTE_IPV6} dst ${LOCAL_IPV6} ${SGW_CFG_XPRM}
 
 #SA out rules
 sa out 7 cipher_algo aes-128-ctr \
 cipher_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode ipv4-tunnel src ${LOCAL_IPV4} dst ${REMOTE_IPV4}
+mode ipv4-tunnel src ${LOCAL_IPV4} dst ${REMOTE_IPV4} ${SGW_CFG_XPRM}
 
 sa out 9 cipher_algo aes-128-ctr \
 cipher_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
 auth_algo sha1-hmac \
 auth_key de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef:de:ad:be:ef \
-mode ipv6-tunnel src ${LOCAL_IPV6} dst ${REMOTE_IPV6}
+mode ipv6-tunnel src ${LOCAL_IPV6} dst ${REMOTE_IPV6} ${SGW_CFG_XPRM}
 
 #Routing rules
 rt ipv4 dst ${REMOTE_IPV4}/32 port 0
examples/ipsec-secgw/test/tun_aesctr_sha1_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..3af680533
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/tun_aesctr_sha1_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/tun_aesgcm_cpu_crypto_defs.sh new file +5 -0
new file mode 100644
index 000000000..5bf1c0ae5
@@ -0,0 +1,5 @@
+#! /bin/bash
+
+. ${DIR}/tun_aesgcm_defs.sh
+
+SGW_CFG_XPRM='type cpu-crypto'
examples/ipsec-secgw/test/tun_aesgcm_mb_cpu_crypto_defs.sh new file +7 -0
new file mode 100644
index 000000000..039b8095e
@@ -0,0 +1,7 @@
+#! /bin/bash
+
+. ${DIR}/tun_aesgcm_defs.sh
+
+CRYPTO_DEV=${CRYPTO_DEV:-'--vdev="crypto_aesni_mb0"'}
+
+SGW_CFG_XPRM='type cpu-crypto'
Signature · 1 line
2.14.5
author
#86 lore raw

PATCH v2 10/10 doc: update security cpu process description

In reply to Fan Zhang

This patch updates programmer's guide and release note for newly added security cpu process description.

---
 doc/guides/cryptodevs/aesni_gcm.rst    |   6 ++
 doc/guides/cryptodevs/aesni_mb.rst     |   7 +++
 doc/guides/prog_guide/rte_security.rst | 112 ++++++++++++++++++++++++++++++++-
 doc/guides/rel_notes/release_19_11.rst |   7 +++
 4 files changed, 131 insertions(+), 1 deletion(-)
doc/guides/cryptodevs/aesni_gcm.rst +6 -0
index 15002aba7..e1c4f9d24 100644
@@ -9,6 +9,12 @@ The AES-NI GCM PMD (**librte_pmd_aesni_gcm**) provides poll mode crypto driver
 support for utilizing Intel multi buffer library (see AES-NI Multi-buffer PMD documentation
 to learn more about it, including installation).
 
+The AES-NI GCM PMD also supports rte_security with security session create
+and ``rte_security_process_cpu_crypto_bulk`` function call to process
+symmetric crypto synchronously with all algorithms specified below. With this
+way it supports scather-gather buffers (``rte_security_vec`` can be greater than
+``1``. Please refer to ``rte_security`` programmer's guide for more detail.
+
 Features
 --------
 
doc/guides/cryptodevs/aesni_mb.rst +7 -0
index 1eff2b073..1a3ddd850 100644
@@ -12,6 +12,13 @@ support for utilizing Intel multi buffer library, see the white paper
 
 The AES-NI MB PMD has current only been tested on Fedora 21 64-bit with gcc.
 
+The AES-NI MB PMD also supports rte_security with security session create
+and ``rte_security_process_cpu_crypto_bulk`` function call to process
+symmetric crypto synchronously with all algorithms specified below. However
+it does not support scather-gather buffer so the ``num`` value in
+``rte_security_vec`` can only be ``1``. Please refer to ``rte_security``
+programmer's guide for more detail.
+
 Features
 --------
 
doc/guides/prog_guide/rte_security.rst +111 -1

141 lines not on this page · raw message

doc/guides/rel_notes/release_19_11.rst +7 -0
index f971b3f77..3d89ab643 100644
@@ -72,6 +72,13 @@ New Features
   Added a symmetric crypto PMD for Marvell NITROX V security processor.
   See the :doc:`../cryptodevs/nitrox` guide for more details on this new
 
+* **Added synchronous Crypto burst API with CPU for RTE_SECURITY.**
+
+  A new API rte_security_process_cpu_crypto_bulk is introduced in security
+  library to process crypto workload in bulk using CPU instructions. AESNI_MB
+  and AESNI_GCM PMD, as well as unit-test and ipsec-secgw sample applications
+  are updated to support this feature.
+
 
 Removed Items
 -------------
Signature · 1 line
2.14.5