Skip to contentSkip to conversation

BPF crypto API framework

PATCH Applied Vadim Fedorenko 5 participants 10 messages Apr 22, 2024 – Apr 24, 2024

Subsystems (8)
lore
Share & tools
lore
#0 lore raw

v10 cover BPF crypto API framework

This series introduces crypto kfuncs to make BPF programs able to utilize kernel crypto subsystem. Crypto operations made pluggable to avoid extensive growth of kernel when it's not needed. Only skcipher is added within this series, but it can be easily extended to other types of operations. No hardware offload supported as it needs sleepable context which is not available for TX or XDP programs. At the same time crypto context initialization kfunc can only run in sleepable context, that's why it should be run separately and store the result in the map.

Selftests show the common way to implement crypto actions in BPF programs. Benchmark is also added to have a baseline.

Vadim Fedorenko (4):
bpf: make common crypto API for TC/XDP programs
bpf: crypto: add skcipher to bpf crypto
selftests: bpf: crypto skcipher algo selftests
selftests: bpf: crypto: add benchmark for crypto functions

 MAINTAINERS                                   |   8 +
 crypto/Makefile                               |   3 +
 crypto/bpf_crypto_skcipher.c                  |  82 ++++
 include/linux/bpf.h                           |   1 +
 include/linux/bpf_crypto.h                    |  24 ++
 kernel/bpf/Makefile                           |   3 +
 kernel/bpf/crypto.c                           | 382 ++++++++++++++++++
 kernel/bpf/helpers.c                          |   2 +-
 kernel/bpf/verifier.c                         |   1 +
 tools/testing/selftests/bpf/Makefile          |   2 +
 tools/testing/selftests/bpf/bench.c           |   6 +
 .../selftests/bpf/benchs/bench_bpf_crypto.c   | 185 +++++++++
 tools/testing/selftests/bpf/config            |   5 +
 .../selftests/bpf/prog_tests/crypto_sanity.c  | 197 +++++++++
 .../selftests/bpf/progs/crypto_basic.c        |  68 ++++
 .../selftests/bpf/progs/crypto_bench.c        | 109 +++++
 .../selftests/bpf/progs/crypto_common.h       |  66 +++
 .../selftests/bpf/progs/crypto_sanity.c       | 169 ++++++++
 18 files changed, 1312 insertions(+), 1 deletion(-)
 create mode 100644 crypto/bpf_crypto_skcipher.c
 create mode 100644 include/linux/bpf_crypto.h
 create mode 100644 kernel/bpf/crypto.c
 create mode 100644 tools/testing/selftests/bpf/benchs/bench_bpf_crypto.c
 create mode 100644 tools/testing/selftests/bpf/prog_tests/crypto_sanity.c
 create mode 100644 tools/testing/selftests/bpf/progs/crypto_basic.c
 create mode 100644 tools/testing/selftests/bpf/progs/crypto_bench.c
 create mode 100644 tools/testing/selftests/bpf/progs/crypto_common.h
 create mode 100644 tools/testing/selftests/bpf/progs/crypto_sanity.c
Signature · 1 line
2.43.0
#1 lore raw

PATCH v10 1/4 bpf: make common crypto API for TC/XDP programs

Add crypto API support to BPF to be able to decrypt or encrypt packets in TC/XDP BPF programs. Special care should be taken for initialization part of crypto algo because crypto alloc) doesn't work with preemtion disabled, it can be run only in sleepable BPF program. Also async crypto is not supported because of the very same issue - TC/XDP BPF programs are not sleepable.

---
v9 -> v10:
- add params__sz parameter to check the size of parameters structure.
- add reserved[2] into a hole in the structure and check this fields to be zero to avoid any confusion in case new version of structure
- improved documentation
- improved sanity checks of parameters (suggested by Martin) v8 -> v9:
- improve initialization API to provide more data and make one call to have crypto context fully initialized and ready to do crypto actions
- improve fast path to avoid indirect calls
- make bpf_crypto_create runnable from syscall type programs v7 -> v8:
- add statesize ops to bpf crypto type as some ciphers are now stateful
- improve error path in bpf_crypto_create
v6 -> v7:
- style fixes
v5 -> v6:
- replace lskcipher with infrastructure to provide pluggable cipher types
- add BPF skcipher as plug-in module in a separate patch
v4 -> v5:
- replace crypto API to use lskcipher (suggested by Herbert Xu)
- remove SG list usage and provide raw buffers
v3 -> v4:
- reuse __bpf_dynptr_data and remove own implementation
- use const __str to provide algorithm name
- use kfunc macroses to avoid compilator warnings
v2 -> v3:
- fix kdoc issues
v1 -> v2:
- use kmalloc in sleepable func, suggested by Alexei
- use __bpf_dynptr_is_rdonly() to check destination, suggested by Jakub
- use __bpf_dynptr_data_ptr() for all dynptr accesses

---
 include/linux/bpf.h        |   1 +
 include/linux/bpf_crypto.h |  24 +++
 kernel/bpf/Makefile        |   3 +
 kernel/bpf/crypto.c        | 382 +++++++++++++++++++++++++++++++++++++
 kernel/bpf/helpers.c       |   2 +-
 kernel/bpf/verifier.c      |   1 +
 6 files changed, 412 insertions(+), 1 deletion(-)
 create mode 100644 include/linux/bpf_crypto.h
 create mode 100644 kernel/bpf/crypto.c
include/linux/bpf.h +1 -0
index 5034c1b4ded7..acc479c13f52 100644
@@ -1265,6 +1265,7 @@ int bpf_dynptr_check_size(u32 size);
 u32 __bpf_dynptr_size(const struct bpf_dynptr_kern *ptr);
 const void *__bpf_dynptr_data(const struct bpf_dynptr_kern *ptr, u32 len);
 void *__bpf_dynptr_data_rw(const struct bpf_dynptr_kern *ptr, u32 len);
+bool __bpf_dynptr_is_rdonly(const struct bpf_dynptr_kern *ptr);
 
 #ifdef CONFIG_BPF_JIT
 int bpf_trampoline_link_prog(struct bpf_tramp_link *link, struct bpf_trampoline *tr);
include/linux/bpf_crypto.h new file +24 -0
new file mode 100644
index 000000000000..a41e71d4e2d9
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */
+#ifndef _BPF_CRYPTO_H
+#define _BPF_CRYPTO_H
+
+struct bpf_crypto_type {
+	void *(*alloc_tfm)(const char *algo);
+	void (*free_tfm)(void *tfm);
+	int (*has_algo)(const char *algo);
+	int (*setkey)(void *tfm, const u8 *key, unsigned int keylen);
+	int (*setauthsize)(void *tfm, unsigned int authsize);
+	int (*encrypt)(void *tfm, const u8 *src, u8 *dst, unsigned int len, u8 *iv);
+	int (*decrypt)(void *tfm, const u8 *src, u8 *dst, unsigned int len, u8 *iv);
+	unsigned int (*ivsize)(void *tfm);
+	unsigned int (*statesize)(void *tfm);
+	u32 (*get_flags)(void *tfm);
+	struct module *owner;
+	char name[14];
+};
+
+int bpf_crypto_register_type(const struct bpf_crypto_type *type);
+int bpf_crypto_unregister_type(const struct bpf_crypto_type *type);
+
+#endif /* _BPF_CRYPTO_H */
kernel/bpf/Makefile +3 -0
index 368c5d86b5b7..736bd22e5ce0 100644
@@ -44,6 +44,9 @@ obj-$(CONFIG_BPF_SYSCALL) += bpf_struct_ops.o
 obj-$(CONFIG_BPF_SYSCALL) += cpumask.o
 obj-${CONFIG_BPF_LSM} += bpf_lsm.o
 endif
+ifeq ($(CONFIG_CRYPTO),y)
+obj-$(CONFIG_BPF_SYSCALL) += crypto.o
+endif
 obj-$(CONFIG_BPF_PRELOAD) += preload/
 
 obj-$(CONFIG_BPF_SYSCALL) += relo_core.o
kernel/bpf/crypto.c new file +382 -0

382 lines not on this page · raw message

kernel/bpf/helpers.c +1 -1
index 8cde717137bd..a67fa076b844 100644
@@ -1443,7 +1443,7 @@ static const struct bpf_func_proto bpf_kptr_xchg_proto = {
 #define DYNPTR_SIZE_MASK	0xFFFFFF
 #define DYNPTR_RDONLY_BIT	BIT(31)
 
-static bool __bpf_dynptr_is_rdonly(const struct bpf_dynptr_kern *ptr)
+bool __bpf_dynptr_is_rdonly(const struct bpf_dynptr_kern *ptr)
 {
 	return ptr->size & DYNPTR_RDONLY_BIT;
 }
kernel/bpf/verifier.c +1 -0
index 2aad6d90550f..f83f537af60f 100644
@@ -5295,6 +5295,7 @@ BTF_ID(struct, cgroup)
 BTF_ID(struct, bpf_cpumask)
 #endif
 BTF_ID(struct, task_struct)
+BTF_ID(struct, bpf_crypto_ctx)
 BTF_SET_END(rcu_protected_types)
 
 static bool rcu_protected_object(const struct btf *btf, u32 btf_id)
Signature · 1 line
2.43.0
#2 lore raw

Re: [PATCH bpf-next v10 1/4] bpf: make common crypto API for TC/XDP programs

kernel test robot noticed the following build warnings:

Hi Vadim,

kernel test robot noticed the following build warnings:

[auto build test WARNING on bpf-next/master]

url:    https://github.com/intel-lab-lkp/linux/commits/Vadim-Fedorenko/bpf-make-common-crypto-API-for-TC-XDP-programs/20240423-070416
base:   https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
patch link:    https://lore.kernel.org/r/20240422225024.2847039-2-vadfed%40meta.com
patch subject: [PATCH bpf-next v10 1/4] bpf: make common crypto API for TC/XDP programs
config: riscv-defconfig (https://download.01.org/0day-ci/archive/20240423/[email protected]/config)
compiler: clang version 19.0.0git (https://github.com/llvm/llvm-project 5ef5eb66fb428aaf61fb51b709f065c069c11242)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240423/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All warnings (new ones prefixed by >>):

>> kernel/bpf/crypto.c:34: warning: Function parameter or struct member 'reserved' not described in 'bpf_crypto_params'

kernel/bpf/crypto.c:55: warning: Function parameter or struct member 'siv_len' not described in 'bpf_crypto_ctx'

vim +34 kernel/bpf/crypto.c

    17	
    18	/* BPF crypto initialization parameters struct */
    19	/**
    20	 * struct bpf_crypto_params - BPF crypto initialization parameters structure
    21	 * @type:	The string of crypto operation type.
    22	 * @algo:	The string of algorithm to initialize.
    23	 * @key:	The cipher key used to init crypto algorithm.
    24	 * @key_len:	The length of cipher key.
    25	 * @authsize:	The length of authentication tag used by algorithm.
    26	 */
    27	struct bpf_crypto_params {
    28		char type[14];
    29		u8 reserved[2];
    30		char algo[128];
    31		u8 key[256];
    32		u32 key_len;
    33		u32 authsize;
  > 34	};
    35	
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
reviewer Applied
#3 lore raw

Re: [PATCH bpf-next v10 1/4] bpf: make common crypto API for TC/XDP programs

In reply to Vadim Fedorenko
Quoted 31 lines · Vadim Fedorenko
kernel/bpf/Makefile +3 -0
index 368c5d86b5b7..736bd22e5ce0 100644
@@ -44,6 +44,9 @@ obj-$(CONFIG_BPF_SYSCALL) += bpf_struct_ops.o
  obj-$(CONFIG_BPF_SYSCALL) += cpumask.o
  obj-${CONFIG_BPF_LSM} += bpf_lsm.o
  endif
+ifeq ($(CONFIG_CRYPTO),y)
+obj-$(CONFIG_BPF_SYSCALL) += crypto.o
+endif
  obj-$(CONFIG_BPF_PRELOAD) += preload/
 
  obj-$(CONFIG_BPF_SYSCALL) += relo_core.o
kernel/bpf/crypto.c new file +11 -0
new file mode 100644
index 000000000000..fe5e60a2f6ca
@@ -0,0 +1,382 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright (c) 2024 Meta, Inc */
+#include <linux/bpf.h>
+#include <linux/bpf_crypto.h>
+#include <linux/bpf_mem_alloc.h>
+#include <linux/btf.h>
+#include <linux/btf_ids.h>
+#include <linux/filter.h>
+#include <linux/scatterlist.h>
+#include <linux/skbuff.h>
+#include <crypto/skcipher.h>

kernel/bpf/crypto.c line 21

Quoted 10 lines
kernel/bpf/crypto.c new file +10 -0
+
+struct bpf_crypto_type_list {
+	const struct bpf_crypto_type *type;
+	struct list_head list;
+};
+
+/* BPF crypto initialization parameters struct */
+/**
+ * struct bpf_crypto_params - BPF crypto initialization parameters structure
+ * @type:	The string of crypto operation type.

I added kdoc for the "reserved" field.

Quoted 12 lines
kernel/bpf/crypto.c new file +12 -0
+ * @algo:	The string of algorithm to initialize.
+ * @key:	The cipher key used to init crypto algorithm.
+ * @key_len:	The length of cipher key.
+ * @authsize:	The length of authentication tag used by algorithm.
+ */
+struct bpf_crypto_params {
+	char type[14];
+	u8 reserved[2];
+	char algo[128];
+	u8 key[256];
+	u32 key_len;
+	u32 authsize;

kernel/bpf/crypto.c line 43

Quoted 10 lines
kernel/bpf/crypto.c new file +10 -0
+};
+
+static LIST_HEAD(bpf_crypto_types);
+static DECLARE_RWSEM(bpf_crypto_types_sem);
+
+/**
+ * struct bpf_crypto_ctx - refcounted BPF crypto context structure
+ * @type:	The pointer to bpf crypto type
+ * @tfm:	The pointer to instance of crypto API struct.
+ * @siv_len     Size of IV and state storage for cipher

Fixed this missing ":" that the testbot is complaining.

Quoted 184 lines
kernel/bpf/crypto.c new file +184 -0
+ * @rcu:	The RCU head used to free the crypto context with RCU safety.
+ * @usage:	Object reference counter. When the refcount goes to 0, the
+ *		memory is released back to the BPF allocator, which provides
+ *		RCU safety.
+ */
+struct bpf_crypto_ctx {
+	const struct bpf_crypto_type *type;
+	void *tfm;
+	u32 siv_len;
+	struct rcu_head rcu;
+	refcount_t usage;
+};
+
+int bpf_crypto_register_type(const struct bpf_crypto_type *type)
+{
+	struct bpf_crypto_type_list *node;
+	int err = -EEXIST;
+
+	down_write(&bpf_crypto_types_sem);
+	list_for_each_entry(node, &bpf_crypto_types, list) {
+		if (!strcmp(node->type->name, type->name))
+			goto unlock;
+	}
+
+	node = kmalloc(sizeof(*node), GFP_KERNEL);
+	err = -ENOMEM;
+	if (!node)
+		goto unlock;
+
+	node->type = type;
+	list_add(&node->list, &bpf_crypto_types);
+	err = 0;
+
+unlock:
+	up_write(&bpf_crypto_types_sem);
+
+	return err;
+}
+EXPORT_SYMBOL_GPL(bpf_crypto_register_type);
+
+int bpf_crypto_unregister_type(const struct bpf_crypto_type *type)
+{
+	struct bpf_crypto_type_list *node;
+	int err = -ENOENT;
+
+	down_write(&bpf_crypto_types_sem);
+	list_for_each_entry(node, &bpf_crypto_types, list) {
+		if (strcmp(node->type->name, type->name))
+			continue;
+
+		list_del(&node->list);
+		kfree(node);
+		err = 0;
+		break;
+	}
+	up_write(&bpf_crypto_types_sem);
+
+	return err;
+}
+EXPORT_SYMBOL_GPL(bpf_crypto_unregister_type);
+
+static const struct bpf_crypto_type *bpf_crypto_get_type(const char *name)
+{
+	const struct bpf_crypto_type *type = ERR_PTR(-ENOENT);
+	struct bpf_crypto_type_list *node;
+
+	down_read(&bpf_crypto_types_sem);
+	list_for_each_entry(node, &bpf_crypto_types, list) {
+		if (strcmp(node->type->name, name))
+			continue;
+
+		if (try_module_get(node->type->owner))
+			type = node->type;
+		break;
+	}
+	up_read(&bpf_crypto_types_sem);
+
+	return type;
+}
+
+__bpf_kfunc_start_defs();
+
+/**
+ * bpf_crypto_ctx_create() - Create a mutable BPF crypto context.
+ *
+ * Allocates a crypto context that can be used, acquired, and released by
+ * a BPF program. The crypto context returned by this function must either
+ * be embedded in a map as a kptr, or freed with bpf_crypto_ctx_release().
+ * As crypto API functions use GFP_KERNEL allocations, this function can
+ * only be used in sleepable BPF programs.
+ *
+ * bpf_crypto_ctx_create() allocates memory for crypto context.
+ * It may return NULL if no memory is available.
+ * @params:	pointer to struct bpf_crypto_params which contains all the
+ *		details needed to initialise crypto context.
+ * @params__sz:	size of steuct bpf_crypto_params usef by bpf program
+ * @err:	integer to store error code when NULL is returned.
+ */
+__bpf_kfunc struct bpf_crypto_ctx *
+bpf_crypto_ctx_create(const struct bpf_crypto_params *params, u32 params__sz,
+		      int *err)
+{
+	const struct bpf_crypto_type *type;
+	struct bpf_crypto_ctx *ctx;
+
+	if (!params || params->reserved[0] || params->reserved[1] ||
+	    params__sz != sizeof(struct bpf_crypto_params)) {
+		*err = -EINVAL;
+		return NULL;
+	}
+
+	type = bpf_crypto_get_type(params->type);
+	if (IS_ERR(type)) {
+		*err = PTR_ERR(type);
+		return NULL;
+	}
+
+	if (!type->has_algo(params->algo)) {
+		*err = -EOPNOTSUPP;
+		goto err_module_put;
+	}
+
+	if (!!params->authsize ^ !!type->setauthsize) {
+		*err = -EOPNOTSUPP;
+		goto err_module_put;
+	}
+
+	if (!params->key_len || params->key_len > sizeof(params->key)) {
+		*err = -EINVAL;
+		goto err_module_put;
+	}
+
+	ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+	if (!ctx) {
+		*err = -ENOMEM;
+		goto err_module_put;
+	}
+
+	ctx->type = type;
+	ctx->tfm = type->alloc_tfm(params->algo);
+	if (IS_ERR(ctx->tfm)) {
+		*err = PTR_ERR(ctx->tfm);
+		goto err_free_ctx;
+	}
+
+	if (params->authsize) {
+		*err = type->setauthsize(ctx->tfm, params->authsize);
+		if (*err)
+			goto err_free_tfm;
+	}
+
+	*err = type->setkey(ctx->tfm, params->key, params->key_len);
+	if (*err)
+		goto err_free_tfm;
+
+	if (type->get_flags(ctx->tfm) & CRYPTO_TFM_NEED_KEY) {
+		*err = -EINVAL;
+		goto err_free_tfm;
+	}
+
+	ctx->siv_len = type->ivsize(ctx->tfm) + type->statesize(ctx->tfm);
+
+	refcount_set(&ctx->usage, 1);
+
+	return ctx;
+
+err_free_tfm:
+	type->free_tfm(ctx->tfm);
+err_free_ctx:
+	kfree(ctx);
+err_module_put:
+	module_put(type->owner);
+
+	return NULL;
+}
+
+static void crypto_free_cb(struct rcu_head *head)
+{
+	struct bpf_crypto_ctx *ctx;
+
+	ctx = container_of(head, struct bpf_crypto_ctx, rcu);
+	ctx->type->free_tfm(ctx->tfm);
+	module_put(ctx->type->owner);
+	kfree(ctx);

kernel/bpf/crypto.c line 237

Quoted 10 lines
kernel/bpf/crypto.c new file +10 -0
+}
+
+/**
+ * bpf_crypto_ctx_acquire() - Acquire a reference to a BPF crypto context.
+ * @ctx: The BPF crypto context being acquired. The ctx must be a trusted
+ *	     pointer.
+ *
+ * Acquires a reference to a BPF crypto context. The context returned by this function
+ * must either be embedded in a map as a kptr, or freed with
+ * bpf_crypto_skcipher_ctx_release().

The release function name is incorrect. It should be bpf_crypto_ctx_release().

Quoted 6 lines
kernel/bpf/crypto.c new file +6 -0
+ */
+__bpf_kfunc struct bpf_crypto_ctx *
+bpf_crypto_ctx_acquire(struct bpf_crypto_ctx *ctx)
+{
+	if (!refcount_inc_not_zero(&ctx->usage))
+		return NULL;

kernel/bpf/crypto.c line 253

Quoted 10 lines
kernel/bpf/crypto.c new file +10 -0
+	return ctx;
+}
+
+/**
+ * bpf_crypto_ctx_release() - Release a previously acquired BPF crypto context.
+ * @ctx: The crypto context being released.
+ *
+ * Releases a previously acquired reference to a BPF crypto context. When the final
+ * reference of the BPF crypto context has been released, it is subsequently freed in
+ * an RCU callback in the BPF memory allocator.

The BPF memory allocator part is incorrect.

I don't think the bpf prog writer needs to worry about the RCU callback details also.

Quoted 50 lines
kernel/bpf/crypto.c new file +50 -0
+ */
+__bpf_kfunc void bpf_crypto_ctx_release(struct bpf_crypto_ctx *ctx)
+{
+	if (refcount_dec_and_test(&ctx->usage))
+		call_rcu(&ctx->rcu, crypto_free_cb);
+}
+
+static int bpf_crypto_crypt(const struct bpf_crypto_ctx *ctx,
+			    const struct bpf_dynptr_kern *src,
+			    struct bpf_dynptr_kern *dst,
+			    const struct bpf_dynptr_kern *siv,
+			    bool decrypt)
+{
+	u32 src_len, dst_len, siv_len;
+	const u8 *psrc;
+	u8 *pdst, *piv;
+	int err;
+
+	if (__bpf_dynptr_is_rdonly(dst))
+		return -EINVAL;
+
+	siv_len = __bpf_dynptr_size(siv);
+	src_len = __bpf_dynptr_size(src);
+	dst_len = __bpf_dynptr_size(dst);
+	if (!src_len || !dst_len)
+		return -EINVAL;
+
+	if (siv_len != ctx->siv_len)
+		return -EINVAL;
+
+	psrc = __bpf_dynptr_data(src, src_len);
+	if (!psrc)
+		return -EINVAL;
+	pdst = __bpf_dynptr_data_rw(dst, dst_len);
+	if (!pdst)
+		return -EINVAL;
+
+	piv = siv_len ? __bpf_dynptr_data_rw(siv, siv_len) : NULL;
+	if (siv_len && !piv)
+		return -EINVAL;
+
+	err = decrypt ? ctx->type->decrypt(ctx->tfm, psrc, pdst, src_len, piv)
+		      : ctx->type->encrypt(ctx->tfm, psrc, pdst, src_len, piv);
+
+	return err;
+}
+
+/**
+ * bpf_crypto_decrypt() - Decrypt buffer using configured context and IV provided.
+ * @ctx:	The crypto context being used. The ctx must be a trusted pointer.

kernel/bpf/crypto.c line 313

Quoted 10 lines
kernel/bpf/crypto.c new file +10 -0
+ * @src:	bpf_dynptr to the encrypted data. Must be a trusted pointer.
+ * @dst:	bpf_dynptr to the buffer where to store the result. Must be a trusted pointer.
+ * @siv:	bpf_dynptr to IV data and state data to be used by decryptor.
+ *
+ * Decrypts provided buffer using IV data and the crypto context. Crypto context must be configured.
+ */
+__bpf_kfunc int bpf_crypto_decrypt(struct bpf_crypto_ctx *ctx,
+				   const struct bpf_dynptr_kern *src,
+				   struct bpf_dynptr_kern *dst,
+				   struct bpf_dynptr_kern *siv)

nit. Added const to all "struct bpf_dynptr_kern *". Mostly to follow "bpf_dynptr_slice_rdwr(const struct bpf_dynptr_kern *ptr, ...)" which the dynptr itself does not change but the data it pointing to could change.

kernel/bpf/crypto.c line 323

Quoted 10 lines
kernel/bpf/crypto.c new file +10 -0
+{
+	return bpf_crypto_crypt(ctx, src, dst, siv, true);
+}
+
+/**
+ * bpf_crypto_encrypt() - Encrypt buffer using configured context and IV provided.
+ * @ctx:	The crypto context being used. The ctx must be a trusted pointer.
+ * @src:	bpf_dynptr to the plain data. Must be a trusted pointer.
+ * @dst:	bpf_dynptr to buffer where to store the result. Must be a trusted pointer.
+ * @siv:		bpf_dynptr to IV data and state data to be used by decryptor.

Alignment is off. Fixed.

kernel/bpf/crypto.c line 330

Quoted 7 lines
kernel/bpf/crypto.c new file +7 -0
+ *
+ * Encrypts provided buffer using IV data and the crypto context. Crypto context must be configured.
+ */
+__bpf_kfunc int bpf_crypto_encrypt(struct bpf_crypto_ctx *ctx,
+				   const struct bpf_dynptr_kern *src,
+				   struct bpf_dynptr_kern *dst,
+				   struct bpf_dynptr_kern *siv)

Same here. "const struct bpf_dynptr_kern *".

I also fixed the bench in patch 4. Made some adjustment to the percpu map comment in patch 3.

Applied. Thanks.

#4 lore raw

PATCH v10 2/4 bpf: crypto: add skcipher to bpf crypto

In reply to Vadim Fedorenko

Implement skcipher crypto in BPF crypto framework.

---
v9 -> 10:
- no changes
v8 -> v9:
- add Herbert's Ack
v7 -> v8:
- Move bpf_crypto_skcipher.c to crypto and make it part of skcipher module. This way looks more natural and makes bpf crypto proper modular. MAINTAINERS files is adjusted to make bpf part belong to BPF maintainers.
v6 - v7:
- style issues
v6:
- introduce new file

---
 MAINTAINERS                  |  8 ++++
 crypto/Makefile              |  3 ++
 crypto/bpf_crypto_skcipher.c | 82 ++++++++++++++++++++++++++++++++++++
 3 files changed, 93 insertions(+)
 create mode 100644 crypto/bpf_crypto_skcipher.c
MAINTAINERS +8 -0
index 6a233e1a3cf2..c9f887fbb477 100644
@@ -3822,6 +3822,14 @@ F:	kernel/bpf/tnum.c
 F:	kernel/bpf/trampoline.c
 F:	kernel/bpf/verifier.c
 
+BPF [CRYPTO]
+M:	Vadim Fedorenko <[email protected]>
+L:	[email protected]
+S:	Maintained
+F:	crypto/bpf_crypto_skcipher.c
+F:	include/linux/bpf_crypto.h
+F:	kernel/bpf/crypto.c
+
 BPF [DOCUMENTATION] (Related to Standardization)
 R:	David Vernet <[email protected]>
 L:	[email protected]
crypto/Makefile +3 -0
index 408f0a1f9ab9..538124f8bf8a 100644
@@ -20,6 +20,9 @@ crypto_skcipher-y += lskcipher.o
 crypto_skcipher-y += skcipher.o
 
 obj-$(CONFIG_CRYPTO_SKCIPHER2) += crypto_skcipher.o
+ifeq ($(CONFIG_BPF_SYSCALL),y)
+obj-$(CONFIG_CRYPTO_SKCIPHER2) += bpf_crypto_skcipher.o
+endif
 
 obj-$(CONFIG_CRYPTO_SEQIV) += seqiv.o
 obj-$(CONFIG_CRYPTO_ECHAINIV) += echainiv.o
crypto/bpf_crypto_skcipher.c new file +82 -0

82 lines not on this page · raw message

Signature · 1 line
2.43.0
#5 lore raw

PATCH v10 3/4 selftests: bpf: crypto skcipher algo selftests

In reply to Vadim Fedorenko

Add simple tc hook selftests to show the way to work with new crypto BPF API. Some tricky dynptr initialization is used to provide empty iv dynptr. Simple AES-ECB algo is used to demonstrate encryption and decryption of fixed size buffers.

---
v9 -> v10:
- reorder calls in tests to have proper error path
- change test programs to use global vars to fill bpf_crypto_params structure
- removed crypto_shared.h as there is no need for it anymore
- improved bpf_tc_detach part with error check
- improved comments about using bpf_dynptr in production
v8 -> v9:
- adjust tests to use new bpf_crypto_create API
v7 -> v8:
- use sizeof for all constant buffer operations
- make local functions static
- initialize crypto_key value via access to bss data
- add bpf_skb_pull_data to be sure that data is linear
- some comments around tricky dynptr initialization
v6 -> v7:
- style issues
v5 -> v6:
- use AF_ALG socket to confirm proper algorithm test
- adjust test kernel config to include AF_ALG
v4 -> v5:
- adjust selftests to use new naming
- restore tests on aarch64 and s390 as no sg lists are used v3 -> v4:
- adjust selftests to use new syntax of helpers
- add tests for acquire and release
v2 -> v3:
- disable tests on s390 and aarch64 because of unknown Fatal exception in sg_init_one
v1 -> v2:
- add CONFIG_CRYPTO_AES and CONFIG_CRYPTO_ECB to selftest build config suggested by Daniel

---
 tools/testing/selftests/bpf/config            |   5 +
 .../selftests/bpf/prog_tests/crypto_sanity.c  | 197 ++++++++++++++++++
 .../selftests/bpf/progs/crypto_basic.c        |  68 ++++++
 .../selftests/bpf/progs/crypto_common.h       |  66 ++++++
 .../selftests/bpf/progs/crypto_sanity.c       | 169 +++++++++++++++
 5 files changed, 505 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/crypto_sanity.c
 create mode 100644 tools/testing/selftests/bpf/progs/crypto_basic.c
 create mode 100644 tools/testing/selftests/bpf/progs/crypto_common.h
 create mode 100644 tools/testing/selftests/bpf/progs/crypto_sanity.c
tools/testing/selftests/bpf/config +5 -0
index afd675b1bf80..eeabd798bc3a 100644
@@ -13,7 +13,12 @@ CONFIG_BPF_SYSCALL=y
 CONFIG_CGROUP_BPF=y
 CONFIG_CRYPTO_HMAC=y
 CONFIG_CRYPTO_SHA256=y
+CONFIG_CRYPTO_USER_API=y
 CONFIG_CRYPTO_USER_API_HASH=y
+CONFIG_CRYPTO_USER_API_SKCIPHER=y
+CONFIG_CRYPTO_SKCIPHER=y
+CONFIG_CRYPTO_ECB=y
+CONFIG_CRYPTO_AES=y
 CONFIG_DEBUG_INFO=y
 CONFIG_DEBUG_INFO_BTF=y
 CONFIG_DEBUG_INFO_DWARF4=y
tools/testing/selftests/bpf/prog_tests/crypto_sanity.c new file +197 -0

197 lines not on this page · raw message

tools/testing/selftests/bpf/progs/crypto_basic.c new file +68 -0

68 lines not on this page · raw message

tools/testing/selftests/bpf/progs/crypto_common.h new file +66 -0

66 lines not on this page · raw message

tools/testing/selftests/bpf/progs/crypto_sanity.c new file +169 -0

169 lines not on this page · raw message

Signature · 1 line
2.43.0
#6 lore raw

PATCH v10 4/4 selftests: bpf: crypto: add benchmark for crypto functions

In reply to Vadim Fedorenko

Some simple benchmarks are added to understand the baseline of performance.

---
v9 -> v10:
- use global variables to init bpf_crypto_params structure
- fix spelling issues
- fix usage of C23 extension
v9:
- initial submission

---
 tools/testing/selftests/bpf/Makefile          |   2 +
 tools/testing/selftests/bpf/bench.c           |   6 +
 .../selftests/bpf/benchs/bench_bpf_crypto.c   | 185 ++++++++++++++++++
 .../selftests/bpf/progs/crypto_bench.c        | 109 +++++++++++
 4 files changed, 302 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/benchs/bench_bpf_crypto.c
 create mode 100644 tools/testing/selftests/bpf/progs/crypto_bench.c
tools/testing/selftests/bpf/Makefile +2 -0
index edc73f8f5aef..be8567337480 100644
@@ -729,6 +729,7 @@ $(OUTPUT)/bench_local_storage_rcu_tasks_trace.o: $(OUTPUT)/local_storage_rcu_tas
 $(OUTPUT)/bench_local_storage_create.o: $(OUTPUT)/bench_local_storage_create.skel.h
 $(OUTPUT)/bench_bpf_hashmap_lookup.o: $(OUTPUT)/bpf_hashmap_lookup.skel.h
 $(OUTPUT)/bench_htab_mem.o: $(OUTPUT)/htab_mem_bench.skel.h
+$(OUTPUT)/bench_bpf_crypto.o: $(OUTPUT)/crypto_bench.skel.h
 $(OUTPUT)/bench.o: bench.h testing_helpers.h $(BPFOBJ)
 $(OUTPUT)/bench: LDLIBS += -lm
 $(OUTPUT)/bench: $(OUTPUT)/bench.o \
@@ -748,6 +749,7 @@ $(OUTPUT)/bench: $(OUTPUT)/bench.o \
 		 $(OUTPUT)/bench_bpf_hashmap_lookup.o \
 		 $(OUTPUT)/bench_local_storage_create.o \
 		 $(OUTPUT)/bench_htab_mem.o \
+		 $(OUTPUT)/bench_bpf_crypto.o \
 		 #
 	$(call msg,BINARY,,$@)
 	$(Q)$(CC) $(CFLAGS) $(LDFLAGS) $(filter %.a %.o,$^) $(LDLIBS) -o $@
tools/testing/selftests/bpf/bench.c +6 -0
index 82de56c8162e..627b74ae041b 100644
@@ -281,6 +281,7 @@ extern struct argp bench_hashmap_lookup_argp;
 extern struct argp bench_local_storage_create_argp;
 extern struct argp bench_htab_mem_argp;
 extern struct argp bench_trigger_batch_argp;
+extern struct argp bench_crypto_argp;
 
 static const struct argp_child bench_parsers[] = {
 	{ &bench_ringbufs_argp, 0, "Ring buffers benchmark", 0 },
@@ -294,6 +295,7 @@ static const struct argp_child bench_parsers[] = {
 	{ &bench_local_storage_create_argp, 0, "local-storage-create benchmark", 0 },
 	{ &bench_htab_mem_argp, 0, "hash map memory benchmark", 0 },
 	{ &bench_trigger_batch_argp, 0, "BPF triggering benchmark", 0 },
+	{ &bench_crypto_argp, 0, "bpf crypto benchmark", 0 },
 	{},
 };
 
@@ -538,6 +540,8 @@ extern const struct bench bench_local_storage_tasks_trace;
 extern const struct bench bench_bpf_hashmap_lookup;
 extern const struct bench bench_local_storage_create;
 extern const struct bench bench_htab_mem;
+extern const struct bench bench_crypto_encrypt;
+extern const struct bench bench_crypto_decrypt;
 
 static const struct bench *benchs[] = {
 	&bench_count_global,
@@ -590,6 +594,8 @@ static const struct bench *benchs[] = {
 	&bench_bpf_hashmap_lookup,
 	&bench_local_storage_create,
 	&bench_htab_mem,
+	&bench_crypto_encrypt,
+	&bench_crypto_decrypt,
 };
 
 static void find_benchmark(void)
tools/testing/selftests/bpf/benchs/bench_bpf_crypto.c new file +185 -0

185 lines not on this page · raw message

tools/testing/selftests/bpf/progs/crypto_bench.c new file +109 -0

109 lines not on this page · raw message

Signature · 1 line
2.43.0
#7 lore raw

Re: [PATCH bpf-next v10 4/4] selftests: bpf: crypto: add benchmark for crypto functions

Quoted 15 lines · Vadim Fedorenko
tools/testing/selftests/bpf/benchs/bench_bpf_crypto.c new file +9 -0
new file mode 100644
index 000000000000..0b8c1f2fe7e6
@@ -0,0 +1,185 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */
+
+#include <argp.h>
+#include "bench.h"
+#include "crypto_bench.skel.h"
+
+#define MAX_CIPHER_LEN 32
+static char *input;

[ ... ]

Quoted 1 line

+static void *crypto_producer(void *input)

The bench result has all 0s in the output:

$> ./bench -p 4 crypto-decrypt
Setting up benchmark 'crypto-decrypt'...
Benchmark 'crypto-decrypt' started.
Iter 0 (209.082us): hits 0.000M/s ( 0.000M/prod), drops 0.000M/s, total operations 0.000M/s
Iter 1 (154.618us): hits 0.000M/s ( 0.000M/prod), drops 0.000M/s, total operations 0.000M/s
Iter 2 (-36.658us): hits 0.000M/s ( 0.000M/prod), drops 0.000M/s, total operations 0.000M/s

This "void *input" arg shadowed the global variable.

Quoted 11 lines
tools/testing/selftests/bpf/benchs/bench_bpf_crypto.c new file +11 -0
+{
+	LIBBPF_OPTS(bpf_test_run_opts, opts,
+		.repeat = 64,
+		.data_in = input,
+		.data_size_in = args.crypto_len,
+	);
+
+	while (true)
+		(void)bpf_prog_test_run_opts(ctx.pfd, &opts);
+	return NULL;
+}
maintainer
#8 lore raw

Re: [PATCH bpf-next v10 4/4] selftests: bpf: crypto: add benchmark for crypto functions

Quoted 23 lines · Martin KaFai Lau
Quoted 15 lines · Vadim Fedorenko
tools/testing/selftests/bpf/benchs/bench_bpf_crypto.c new file +9 -0
new file mode 100644
index 000000000000..0b8c1f2fe7e6
@@ -0,0 +1,185 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */
+
+#include <argp.h>
+#include "bench.h"
+#include "crypto_bench.skel.h"
+
+#define MAX_CIPHER_LEN 32
+static char *input;

[ ... ]

Quoted 1 line

+static void *crypto_producer(void *input)

The bench result has all 0s in the output:

Quoted 9 lines

$> ./bench -p 4 crypto-decrypt
Setting up benchmark 'crypto-decrypt'...
Benchmark 'crypto-decrypt' started.

Iter   0 (209.082us): hits    0.000M/s (  0.000M/prod), drops    0.000M/s, total operations    0.000M/s
Iter   1 (154.618us): hits    0.000M/s (  0.000M/prod), drops    0.000M/s, total operations    0.000M/s
Iter   2 (-36.658us): hits    0.000M/s (  0.000M/prod), drops    0.000M/s, total operations    0.000M/s

This "void *input" arg shadowed the global variable.

Got it. Will do re-spin then...

Quoted 12 lines
Quoted 11 lines
tools/testing/selftests/bpf/benchs/bench_bpf_crypto.c new file +11 -0
+{
+    LIBBPF_OPTS(bpf_test_run_opts, opts,
+        .repeat = 64,
+        .data_in = input,
+        .data_size_in = args.crypto_len,
+    );
+
+    while (true)
+        (void)bpf_prog_test_run_opts(ctx.pfd, &opts);
+    return NULL;
+}
#9 lore raw
In reply to Vadim Fedorenko
This series was applied to bpf/bpf-next.git (master) by Martin KaFai Lau <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Martin KaFai Lau <[email protected]>:

Quoted 10 lines · On Mon, 22 Apr 2024 15:50:20 -0700 you wrote:

This series introduces crypto kfuncs to make BPF programs able to utilize kernel crypto subsystem. Crypto operations made pluggable to avoid extensive growth of kernel when it's not needed. Only skcipher is added within this series, but it can be easily extended to other types of operations. No hardware offload supported as it needs sleepable context which is not available for TX or XDP programs. At the same time crypto context initialization kfunc can only run in sleepable context, that's why it should be run separately and store the result in the map.

[...]

Here is the summary with links:
- [bpf-next,v10,1/4] bpf: make common crypto API for TC/XDP programs

    https://git.kernel.org/bpf/bpf-next/c/3e1c6f35409f
  - [bpf-next,v10,2/4] bpf: crypto: add skcipher to bpf crypto
    https://git.kernel.org/bpf/bpf-next/c/fda4f71282b2
  - [bpf-next,v10,3/4] selftests: bpf: crypto skcipher algo selftests
    https://git.kernel.org/bpf/bpf-next/c/91541ab192fc
  - [bpf-next,v10,4/4] selftests: bpf: crypto: add benchmark for crypto functions
    https://git.kernel.org/bpf/bpf-next/c/8000e627dc98

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html