Skip to contentSkip to conversation

bpf: add signature

PATCH In review Matteo Croce 6 participants 20 messages Dec 3, 2021 – Dec 9, 2021

Subsystems (7)
lore
Share & tools
lore
#0 lore raw

cover bpf: add signature

From: Matteo Croce <[email protected]>

This series add signature verification for BPF files. The first patch implements the signature validation in the kernel, the second patch optionally makes the signature mandatory, the third adds signature generation to bpftool.

This only works with CO-RE programs.

Matteo Croce (3):
bpf: add signature to eBPF instructions
bpf: add option to require BPF signature
bpftool: add signature in skeleton

 crypto/asymmetric_keys/asymmetric_type.c |   1 +
 crypto/asymmetric_keys/pkcs7_verify.c    |   7 +-
 include/linux/verification.h             |   1 +
 include/uapi/linux/bpf.h                 |   2 +
 kernel/bpf/Kconfig                       |  14 ++
 kernel/bpf/syscall.c                     |  51 +++++-
 tools/bpf/bpftool/Makefile               |  14 +-
 tools/bpf/bpftool/gen.c                  |  33 ++++
 tools/bpf/bpftool/main.c                 |  28 +++
 tools/bpf/bpftool/main.h                 |   7 +
 tools/bpf/bpftool/sign.c                 | 218 +++++++++++++++++++++++
 tools/include/uapi/linux/bpf.h           |   2 +
 tools/lib/bpf/skel_internal.h            |   4 +
 13 files changed, 372 insertions(+), 10 deletions(-)
 create mode 100644 tools/bpf/bpftool/sign.c
Signature · 1 line
2.33.1
#1 lore raw

PATCH 1/3 bpf: add signature to eBPF instructions

From: Matteo Croce <[email protected]>

When loading a BPF program, pass a signature which is used to validate the instructions.
The signature type is the same used to validate the kernel modules.

This happens when loading a program with, respectively, an invalid and a valid signature:

    # ./core-bad
    [ 8524.417567] Invalid BPF signature for '__loader.prog': -EKEYREJECTED
    failed to open and/or load BPF object
    # ./core-ok
---
 crypto/asymmetric_keys/asymmetric_type.c |  1 +
 crypto/asymmetric_keys/pkcs7_verify.c    |  7 +++-
 include/linux/verification.h             |  1 +
 include/uapi/linux/bpf.h                 |  2 +
 kernel/bpf/Kconfig                       |  8 ++++
 kernel/bpf/syscall.c                     | 47 +++++++++++++++++++++---
 6 files changed, 59 insertions(+), 7 deletions(-)
crypto/asymmetric_keys/asymmetric_type.c +1 -0
index ad8af3d70ac0..e4f2fee19c5f 100644
@@ -26,6 +26,7 @@ const char *const key_being_used_for[NR__KEY_BEING_USED_FOR] = {
 	[VERIFYING_KEY_SIGNATURE]		= "key sig",
 	[VERIFYING_KEY_SELF_SIGNATURE]		= "key self sig",
 	[VERIFYING_UNSPECIFIED_SIGNATURE]	= "unspec sig",
+	[VERIFYING_BPF_SIGNATURE]		= "bpf sig",
 };
 EXPORT_SYMBOL_GPL(key_being_used_for);
 
crypto/asymmetric_keys/pkcs7_verify.c +5 -2
index 0b4d07aa8811..ab645f23c021 100644
@@ -411,12 +411,15 @@ int pkcs7_verify(struct pkcs7_message *pkcs7,
 
 	switch (usage) {
 	case VERIFYING_MODULE_SIGNATURE:
+	case VERIFYING_BPF_SIGNATURE:
 		if (pkcs7->data_type != OID_data) {
-			pr_warn("Invalid module sig (not pkcs7-data)\n");
+			pr_warn("Invalid %s (not pkcs7-data)\n",
+				key_being_used_for[usage]);
 			return -EKEYREJECTED;
 		}
 		if (pkcs7->have_authattrs) {
-			pr_warn("Invalid module sig (has authattrs)\n");
+			pr_warn("Invalid %s (has authattrs)\n",
+				key_being_used_for[usage]);
 			return -EKEYREJECTED;
 		}
 		break;
include/linux/verification.h +1 -0
index a655923335ae..71482644eea0 100644
@@ -27,6 +27,7 @@ enum key_being_used_for {
 	VERIFYING_KEY_SIGNATURE,
 	VERIFYING_KEY_SELF_SIGNATURE,
 	VERIFYING_UNSPECIFIED_SIGNATURE,
+	VERIFYING_BPF_SIGNATURE,
 	NR__KEY_BEING_USED_FOR
 };
 extern const char *const key_being_used_for[NR__KEY_BEING_USED_FOR];
include/uapi/linux/bpf.h +2 -0
index c26871263f1f..bbb4435c7586 100644
@@ -1346,6 +1346,8 @@ union bpf_attr {
 		__aligned_u64	fd_array;	/* array of FDs */
 		__aligned_u64	core_relos;
 		__u32		core_relo_rec_size; /* sizeof(struct bpf_core_relo) */
+		__aligned_u64	signature;	/* instruction's signature */
+		__u32		sig_len;	/* signature size */
 	};
 
 	struct { /* anonymous struct used by BPF_OBJ_* commands */
kernel/bpf/Kconfig +8 -0
index d24d518ddd63..735979bb8672 100644
@@ -79,6 +79,14 @@ config BPF_UNPRIV_DEFAULT_OFF
 
 	  If you are unsure how to answer this question, answer Y.
 
+config BPF_SIG
+	bool "BPF signature verification"
+	select SYSTEM_DATA_VERIFICATION
+	depends on BPF_SYSCALL
+	help
+	  Check BPF programs for valid signatures upon load: the signature
+	  is passed via the bpf() syscall together with the instructions.
+
 source "kernel/bpf/preload/Kconfig"
 
 config BPF_LSM
kernel/bpf/syscall.c +42 -5

71 lines not on this page · raw message

Signature · 1 line
2.33.1
#2 lore raw

Re: [PATCH bpf-next 1/3] bpf: add signature to eBPF instructions

Thank you for the patch! Perhaps something to improve:

Hi Matteo,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on bpf-next/master]

url:    https://github.com/0day-ci/linux/commits/Matteo-Croce/bpf-add-signature/20211204-032018
base:   https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
config: nds32-allyesconfig (https://download.01.org/0day-ci/archive/20211204/202112040507.siNkODlN-lkp(a)intel.com/config)
compiler: nds32le-linux-gcc (GCC) 11.2.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/fdfe32b9e64c6a208965002215d467ec383b6f57
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Matteo-Croce/bpf-add-signature/20211204-032018
git checkout fdfe32b9e64c6a208965002215d467ec383b6f57
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=nds32 SHELL=/bin/bash kernel/bpf/

If you fix the issue, kindly add following tag as appropriate

All warnings (new ones prefixed by >>):

kernel/bpf/syscall.c: In function 'bpf_prog_load':

>> kernel/bpf/syscall.c:2324:47: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
    2324 |                 if (copy_from_user(signature, (char *)attr->signature, attr->sig_len)) {
         |                                               ^

vim +2324 kernel/bpf/syscall.c

2192
2193 static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr)
2194 {
2195 enum bpf_prog_type type = attr->prog_type;
2196 struct bpf_prog *prog, *dst_prog = NULL;
2197 struct btf *attach_btf = NULL;
2198 int err;
2199 char license[128];
2200 bool is_gpl;
2201
2202 if (CHECK_ATTR(BPF_PROG_LOAD))
2203 return -EINVAL;
2204
2205 if (attr->prog_flags & ~(BPF_F_STRICT_ALIGNMENT |
2206 BPF_F_ANY_ALIGNMENT |
2207 BPF_F_TEST_STATE_FREQ |
2208 BPF_F_SLEEPABLE |
2209 BPF_F_TEST_RND_HI32))
2210 return -EINVAL;
2211
2212 if (!IS_ENABLED(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS) &&
2213 (attr->prog_flags & BPF_F_ANY_ALIGNMENT) &&
2214 !bpf_capable())
2215 return -EPERM;
2216
2217 /* copy eBPF program license from user space */
2218 if (strncpy_from_bpfptr(license,
2219 make_bpfptr(attr->license, uattr.is_kernel),
2220 sizeof(license) - 1) < 0)
2221 return -EFAULT;
2222 license[sizeof(license) - 1] = 0;
2223
2224 /* eBPF programs must be GPL compatible to use GPL-ed functions */
2225 is_gpl = license_is_gpl_compatible(license);
2226
2227 if (attr->insn_cnt == 0 ||
2228 attr->insn_cnt > (bpf_capable() ? BPF_COMPLEXITY_LIMIT_INSNS : BPF_MAXINSNS))
2229 return -E2BIG;
2230 if (type != BPF_PROG_TYPE_SOCKET_FILTER &&
2231 type != BPF_PROG_TYPE_CGROUP_SKB &&
2232 !bpf_capable())
2233 return -EPERM;
2234
2235 if (is_net_admin_prog_type(type) && !capable(CAP_NET_ADMIN) && !capable(CAP_SYS_ADMIN))
2236 return -EPERM;
2237 if (is_perfmon_prog_type(type) && !perfmon_capable())
2238 return -EPERM;
2239
2240 /* attach_prog_fd/attach_btf_obj_fd can specify fd of either bpf_prog
2241 * or btf, we need to check which one it is
2242 */
2243 if (attr->attach_prog_fd) {
2244 dst_prog = bpf_prog_get(attr->attach_prog_fd);
2245 if (IS_ERR(dst_prog)) {
2246 dst_prog = NULL;
2247 attach_btf = btf_get_by_fd(attr->attach_btf_obj_fd);
2248 if (IS_ERR(attach_btf))
2249 return -EINVAL;
2250 if (!btf_is_kernel(attach_btf)) {
2251 /* attaching through specifying bpf_prog's BTF
2252 * objects directly might be supported eventually
2253 */
2254 btf_put(attach_btf);
2255 return -ENOTSUPP;
2256 }
2257 }
2258 } else if (attr->attach_btf_id) {
2259 /* fall back to vmlinux BTF, if BTF type ID is specified */
2260 attach_btf = bpf_get_btf_vmlinux();
2261 if (IS_ERR(attach_btf))
2262 return PTR_ERR(attach_btf);
2263 if (!attach_btf)
2264 return -EINVAL;
2265 btf_get(attach_btf);
2266 }
2267
2268 bpf_prog_load_fixup_attach_type(attr);
2269 if (bpf_prog_load_check_attach(type, attr->expected_attach_type,
2270 attach_btf, attr->attach_btf_id,
2271 dst_prog)) {
2272 if (dst_prog)
2273 bpf_prog_put(dst_prog);
2274 if (attach_btf)
2275 btf_put(attach_btf);
2276 return -EINVAL;
2277 }
2278
2279 /* plain bpf_prog allocation */
2280 prog = bpf_prog_alloc(bpf_prog_size(attr->insn_cnt), GFP_USER);
2281 if (!prog) {
2282 if (dst_prog)
2283 bpf_prog_put(dst_prog);
2284 if (attach_btf)
2285 btf_put(attach_btf);
2286 return -ENOMEM;
2287 }
2288
2289 prog->expected_attach_type = attr->expected_attach_type;
2290 prog->aux->attach_btf = attach_btf;
2291 prog->aux->attach_btf_id = attr->attach_btf_id;
2292 prog->aux->dst_prog = dst_prog;
2293 prog->aux->offload_requested = !!attr->prog_ifindex;
2294 prog->aux->sleepable = attr->prog_flags & BPF_F_SLEEPABLE;
2295
2296 err = security_bpf_prog_alloc(prog->aux);
2297 if (err)
2298 goto free_prog;
2299
2300 prog->aux->user = get_current_user();
2301 prog->len = attr->insn_cnt;
2302
2303 err = -EFAULT;
2304 if (copy_from_bpfptr(prog->insns,
2305 make_bpfptr(attr->insns, uattr.is_kernel),
2306 bpf_prog_insn_size(prog)) != 0)
2307 goto free_prog_sec;
2308
2309 err = bpf_obj_name_cpy(prog->aux->name, attr->prog_name,
2310 sizeof(attr->prog_name));
2311 if (err < 0)
2312 goto free_prog_sec;
2313
2314 #ifdef CONFIG_BPF_SIG
2315 if (attr->sig_len) {
2316 char *signature;
2317
2318 signature = kmalloc(attr->sig_len, GFP_USER);
2319 if (!signature) {
2320 err = -ENOMEM;
2321 goto free_prog_sec;
2322 }
2323

> 2324			if (copy_from_user(signature, (char *)attr->signature, attr->sig_len)) {
  2325				err = -EFAULT;
  2326				kfree(signature);
  2327				goto free_prog_sec;
  2328			}
  2329	
  2330			err = verify_pkcs7_signature(prog->insns,
  2331						     prog->len * sizeof(struct bpf_insn),
  2332						     signature, attr->sig_len,
  2333						     VERIFY_USE_SECONDARY_KEYRING,
  2334						     VERIFYING_BPF_SIGNATURE,
  2335						     NULL, NULL);

2336 kfree(signature);
2337
2338 if (err) {
2339 pr_warn("Invalid BPF signature for '%s': %pe\n",
2340 prog->aux->name, ERR_PTR(err));
2341 goto free_prog_sec;
2342 }
2343 }
2344 #endif
2345
2346 prog->orig_prog = NULL;
2347 prog->jited = 0;
2348
2349 atomic64_set(&prog->aux->refcnt, 1);
2350 prog->gpl_compatible = is_gpl ? 1 : 0;
2351
2352 if (bpf_prog_is_dev_bound(prog->aux)) {
2353 err = bpf_prog_offload_init(prog, attr);
2354 if (err)
2355 goto free_prog_sec;
2356 }
2357
2358 /* find program type: socket_filter vs tracing_filter */
2359 err = find_prog_type(type, prog);
2360 if (err < 0)
2361 goto free_prog_sec;
2362
2363 prog->aux->load_time = ktime_get_boottime_ns();
2364
2365 /* run eBPF verifier */
2366 err = bpf_check(&prog, attr, uattr);
2367 if (err < 0)
2368 goto free_used_maps;
2369
2370 prog = bpf_prog_select_runtime(prog, &err);
2371 if (err < 0)
2372 goto free_used_maps;
2373
2374 err = bpf_prog_alloc_id(prog);
2375 if (err)
2376 goto free_used_maps;
2377
2378 /* Upon success of bpf_prog_alloc_id(), the BPF prog is
2379 * effectively publicly exposed. However, retrieving via
2380 * bpf_prog_get_fd_by_id() will take another reference,
2381 * therefore it cannot be gone underneath us.
2382 *
2383 * Only for the time /after/ successful bpf_prog_new_fd()
2384 * and before returning to userspace, we might just hold
2385 * one reference and any parallel close on that fd could
2386 * rip everything out. Hence, below notifications must
2387 * happen before bpf_prog_new_fd().
2388 *
2389 * Also, any failure handling from this point onwards must
2390 * be using bpf_prog_put() given the program is exposed.
2391 */
2392 bpf_prog_kallsyms_add(prog);
2393 perf_event_bpf_event(prog, PERF_BPF_EVENT_PROG_LOAD, 0);
2394 bpf_audit_prog(prog, BPF_AUDIT_LOAD);
2395
2396 err = bpf_prog_new_fd(prog);
2397 if (err < 0)
2398 bpf_prog_put(prog);
2399 return err;
2400
2401 free_used_maps:
2402 /* In case we have subprogs, we need to wait for a grace
2403 * period before we can tear down JIT memory since symbols
2404 * are already exposed under kallsyms.
2405 */
2406 __bpf_prog_put_noref(prog, prog->aux->func_cnt);
2407 return err;
2408 free_prog_sec:
2409 free_uid(prog->aux->user);
2410 security_bpf_prog_free(prog->aux);
2411 free_prog:
2412 if (prog->aux->attach_btf)
2413 btf_put(prog->aux->attach_btf);
2414 bpf_prog_free(prog);
2415 return err;
2416 }
2417

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#3 lore raw

PATCH 2/3 bpf: add option to require BPF signature

In reply to Matteo Croce

From: Matteo Croce <[email protected]>

Add a compile time option which makes the BPF signature mandatory, i.e. all programs without signature or with an invalid one are rejected.

CO-RE programs load a program of type BPF_PROG_TYPE_SYSCALL, which then uses the bpf() syscall to load the final program. This one won't have any signature, so never enforce signature for programs coming from the kernel.

This happens when loading a program with a missing signature:

    # ip link set lo xdp object xdp.o
    [ 8677.652546] Rejecting BPF '' with no signature
---
 kernel/bpf/Kconfig   | 6 ++++++
 kernel/bpf/syscall.c | 4 ++++
 2 files changed, 10 insertions(+)
kernel/bpf/Kconfig +6 -0
index 735979bb8672..fe6e84abe84c 100644
@@ -87,6 +87,12 @@ config BPF_SIG
 	  Check BPF programs for valid signatures upon load: the signature
 	  is passed via the bpf() syscall together with the instructions.
 
+config BPF_SIG_FORCE
+	bool "Require BPF to be validly signed"
+	depends on BPF_SIG
+	help
+	  Reject unsigned BPF or signed BPF for which we don't have a key.
+
 source "kernel/bpf/preload/Kconfig"
 
 config BPF_LSM
kernel/bpf/syscall.c +4 -0
index 5aaa74a72b46..9e36614719fd 100644
@@ -2340,6 +2340,10 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr)
 				prog->aux->name, ERR_PTR(err));
 			goto free_prog_sec;
 		}
+	} else if (IS_ENABLED(CONFIG_BPF_SIG_FORCE) && !uattr.is_kernel) {
+		pr_warn("Rejecting BPF '%s' with no signature\n", prog->aux->name);
+		err = -EKEYREJECTED;
+		goto free_prog_sec;
 	}
 #endif
 
Signature · 1 line
2.33.1
#4 lore raw

PATCH 3/3 bpftool: add signature in skeleton

In reply to Matteo Croce

From: Matteo Croce <[email protected]>

When generating the skeleton, allow to add a signature.
The signature will be passed to the kernel in the newly added field. As in sign-file, allow specifying "pkcs11:..." as key file, to use the openssl engine.
Still as in sign-file, read the environment variable KBUILD_SIGN_PIN.

---
 tools/bpf/bpftool/Makefile     |  14 ++-
 tools/bpf/bpftool/gen.c        |  33 +++++
 tools/bpf/bpftool/main.c       |  28 +++++
 tools/bpf/bpftool/main.h       |   7 ++
 tools/bpf/bpftool/sign.c       | 218 +++++++++++++++++++++++++++++++++
 tools/include/uapi/linux/bpf.h |   2 +
 tools/lib/bpf/skel_internal.h  |   4 +
 7 files changed, 303 insertions(+), 3 deletions(-)
 create mode 100644 tools/bpf/bpftool/sign.c
tools/bpf/bpftool/Makefile +11 -3
index 42eb8eee3d89..d2645c2f4bc9 100644
@@ -96,9 +96,9 @@ RM ?= rm -f
 
 FEATURE_USER = .bpftool
 FEATURE_TESTS = libbfd disassembler-four-args reallocarray zlib libcap \
-	clang-bpf-co-re
+	clang-bpf-co-re libcrypto
 FEATURE_DISPLAY = libbfd disassembler-four-args zlib libcap \
-	clang-bpf-co-re
+	clang-bpf-co-re libcrypto
 
 check_feat := 1
 NON_CHECK_FEAT_TARGETS := clean uninstall doc doc-clean doc-install doc-uninstall
@@ -131,6 +131,11 @@ CFLAGS += -DUSE_LIBCAP
 LIBS += -lcap
 endif
 
+ifeq ($(feature-libcrypto), 1)
+CFLAGS_CRYPTO := -DUSE_SIGN
+LIBS += -lcrypto
+endif
+
 include $(wildcard $(OUTPUT)*.d)
 
 all: $(OUTPUT)bpftool
@@ -138,6 +143,9 @@ all: $(OUTPUT)bpftool
 BFD_SRCS = jit_disasm.c
 
 SRCS = $(filter-out $(BFD_SRCS),$(wildcard *.c))
+ifneq ($(feature-libcrypto), 1)
+SRCS := $(filter-out sign.c,$(SRCS))
+endif
 
 ifeq ($(feature-libbfd),1)
   LIBS += -lbfd -ldl -lopcodes
@@ -224,7 +232,7 @@ $(BOOTSTRAP_OUTPUT)%.o: %.c $(LIBBPF_BOOTSTRAP_INTERNAL_HDRS) | $(BOOTSTRAP_OUTP
 		-c -MMD $< -o $@
 
 $(OUTPUT)%.o: %.c
-	$(QUIET_CC)$(CC) $(CFLAGS) -c -MMD $< -o $@
+	$(QUIET_CC)$(CC) $(CFLAGS) $(CFLAGS_CRYPTO) -c -MMD $< -o $@
 
 feature-detect-clean:
 	$(call QUIET_CLEAN, feature-detect)
tools/bpf/bpftool/gen.c +33 -0
index 997a2865e04a..c9f09b222986 100644
@@ -491,6 +491,10 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h
 	struct bpf_map *map;
 	char ident[256];
 	int err = 0;
+#ifdef USE_SIGN
+	char *signature = NULL;
+	int sig_len = 0;
+#endif
 
 	err = bpf_object__gen_loader(obj, &opts);
 	if (err)
@@ -510,6 +514,19 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h
 	 * are populated with the loader program.
 	 */
 
+#ifdef USE_SIGN
+	if (sign_bpf) {
+		sig_len = sign(sign_hash, sign_key, sign_cert,
+			       opts.insns, opts.insns_sz,
+			       (unsigned char **)&signature);
+		if (sig_len <= 0) {
+			p_err("failed to sign instructions");
+			err = -EINVAL;
+			goto out;
+		}
+	}
+#endif
+
 	/* finish generating 'struct skel' */
 	codegen("\
 		\n\
@@ -592,6 +609,18 @@ static int gen_trace(struct bpf_object *obj, const char *obj_name, const char *h
 		",
 		opts.insns_sz);
 	print_hex(opts.insns, opts.insns_sz);
+#ifdef USE_SIGN
+	if (sign_bpf) {
+		codegen("\
+			\n\
+			\";						    \n\
+				opts.sig_sz = %d;			    \n\
+				opts.signature = (void *)\"\\		    \n\
+			",
+			sig_len);
+		print_hex(signature, sig_len);
+	}
+#endif
 	codegen("\
 		\n\
 		\";							    \n\
@@ -1090,6 +1119,10 @@ static int do_help(int argc, char **argv)
 		"       %1$s %2$s help\n"
 		"\n"
 		"       " HELP_SPEC_OPTIONS " |\n"
+#ifdef USE_SIGN
+		"                    {-s|--sign} | {-H|--hash} |\n"
+		"                    {-c|--cert} | {-k|--key} |\n"
+#endif
 		"                    {-L|--use-loader} }\n"
 		"",
 		bin_name, "gen");
tools/bpf/bpftool/main.c +28 -0
index 8b71500e7cb2..cea3d07e98e0 100644
@@ -31,6 +31,10 @@ bool block_mount;
 bool verifier_logs;
 bool relaxed_maps;
 bool use_loader;
+bool sign_bpf;
+const char *sign_hash;
+const char *sign_cert;
+const char *sign_key;
 bool legacy_libbpf;
 struct btf *base_btf;
 struct hashmap *refs_table;
@@ -403,6 +407,12 @@ int main(int argc, char **argv)
 		{ "use-loader",	no_argument,	NULL,	'L' },
 		{ "base-btf",	required_argument, NULL, 'B' },
 		{ "legacy",	no_argument,	NULL,	'l' },
+#ifdef USE_SIGN
+		{ "sign",	no_argument,	NULL,	's' },
+		{ "hash",	required_argument, NULL, 'H' },
+		{ "cert",	required_argument, NULL, 'c' },
+		{ "key",	required_argument, NULL, 'k' },
+#endif
 		{ 0 }
 	};
 	bool version_requested = false;
@@ -416,7 +426,11 @@ int main(int argc, char **argv)
 	bin_name = argv[0];
 
 	opterr = 0;
+#ifdef USE_SIGN
+	while ((opt = getopt_long(argc, argv, "VhpjfLmndB:lsH:c:k:",
+#else
 	while ((opt = getopt_long(argc, argv, "VhpjfLmndB:l",
+#endif
 				  options, NULL)) >= 0) {
 		switch (opt) {
 		case 'V':
@@ -466,6 +480,20 @@ int main(int argc, char **argv)
 		case 'l':
 			legacy_libbpf = true;
 			break;
+#ifdef USE_SIGN
+		case 's':
+			sign_bpf = true;
+			break;
+		case 'H':
+			sign_hash = optarg;
+			break;
+		case 'c':
+			sign_cert = optarg;
+			break;
+		case 'k':
+			sign_key = optarg;
+			break;
+#endif
 		default:
 			p_err("unrecognized option '%s'", argv[optind - 1]);
 			if (json_output)
tools/bpf/bpftool/main.h +7 -0
index 8d76d937a62b..ef82219d3f52 100644
@@ -91,6 +91,10 @@ extern bool verifier_logs;
 extern bool relaxed_maps;
 extern bool use_loader;
 extern bool legacy_libbpf;
+extern bool sign_bpf;
+extern const char *sign_hash;
+extern const char *sign_cert;
+extern const char *sign_key;
 extern struct btf *base_btf;
 extern struct hashmap *refs_table;
 
@@ -260,4 +264,7 @@ static inline bool hashmap__empty(struct hashmap *map)
 	return map ? hashmap__size(map) == 0 : true;
 }
 
+int sign(const char *hash_algo, const char *key_path, const char *x509_path,
+	 const char *indata, int indatalen, unsigned char **outdata);
+
 #endif
tools/bpf/bpftool/sign.c new file +218 -0

218 lines not on this page · raw message

tools/include/uapi/linux/bpf.h +2 -0
index c26871263f1f..bbb4435c7586 100644
@@ -1346,6 +1346,8 @@ union bpf_attr {
 		__aligned_u64	fd_array;	/* array of FDs */
 		__aligned_u64	core_relos;
 		__u32		core_relo_rec_size; /* sizeof(struct bpf_core_relo) */
+		__aligned_u64	signature;	/* instruction's signature */
+		__u32		sig_len;	/* signature size */
 	};
 
 	struct { /* anonymous struct used by BPF_OBJ_* commands */
tools/lib/bpf/skel_internal.h +4 -0
index 0b84d8e6b72a..7c987b79568b 100644
@@ -52,8 +52,10 @@ struct bpf_load_and_run_opts {
 	struct bpf_loader_ctx *ctx;
 	const void *data;
 	const void *insns;
+	const void *signature;
 	__u32 data_sz;
 	__u32 insns_sz;
+	__u32 sig_sz;
 	const char *errstr;
 };
 
@@ -93,6 +95,8 @@ static inline int bpf_load_and_run(struct bpf_load_and_run_opts *opts)
 	attr.prog_type = BPF_PROG_TYPE_SYSCALL;
 	attr.insns = (long) opts->insns;
 	attr.insn_cnt = opts->insns_sz / sizeof(struct bpf_insn);
+	attr.signature = (long) opts->signature;
+	attr.sig_len = opts->sig_sz;
 	attr.license = (long) "Dual BSD/GPL";
 	memcpy(attr.prog_name, "__loader.prog", sizeof("__loader.prog"));
 	attr.fd_array = (long) &map_fd;
Signature · 1 line
2.33.1
#5 lore raw
In reply to Matteo Croce
Quoted 7 lines · Matteo Croce

From: Matteo Croce <[email protected]>

This series add signature verification for BPF files. The first patch implements the signature validation in the kernel, the second patch optionally makes the signature mandatory, the third adds signature generation to bpftool.

Matteo,

I think I already mentioned that it's no-go as-is. We've agreed to go with John's suggestion.

#6 lore raw
Quoted 10 lines · Alexei Starovoitov
Quoted 7 lines · Matteo Croce

From: Matteo Croce <[email protected]>

This series add signature verification for BPF files. The first patch implements the signature validation in the kernel, the second patch optionally makes the signature mandatory, the third adds signature generation to bpftool.

Quoted 4 lines

Matteo,

I think I already mentioned that it's no-go as-is. We've agreed to go with John's suggestion.

Hi,

my previous attempt was loading a whole ELF file and parsing it in kernel. In this series I just validate the instructions against a signature, as with kernel CO-RE libbpf doesn't need to mangle it.

Which suggestion? I think I missed this one..

Signature · 1 line
per aspera ad upstream
#7 lore raw
Quoted 18 lines · Matteo Croce
Quoted 10 lines · Alexei Starovoitov
Quoted 7 lines · Matteo Croce

From: Matteo Croce <[email protected]>

This series add signature verification for BPF files. The first patch implements the signature validation in the kernel, the second patch optionally makes the signature mandatory, the third adds signature generation to bpftool.

Quoted 4 lines

Matteo,

I think I already mentioned that it's no-go as-is. We've agreed to go with John's suggestion.

Quoted 7 lines

Hi,

my previous attempt was loading a whole ELF file and parsing it in kernel. In this series I just validate the instructions against a signature, as with kernel CO-RE libbpf doesn't need to mangle it.

Which suggestion? I think I missed this one..

This talk and discussion:
https://linuxplumbersconf.org/event/11/contributions/947/

#8 lore raw
Quoted 30 lines · Alexei Starovoitov
Quoted 28 lines · Matteo Croce
Quoted 12 lines · Alexei Starovoitov
Quoted 8 lines · Matteo Croce

From: Matteo Croce <[email protected]>

This series add signature verification for BPF files. The first patch implements the signature validation in the kernel,
the second patch optionally makes the signature mandatory, the third adds signature generation to bpftool.

Quoted 4 lines

Matteo,

I think I already mentioned that it's no-go as-is. We've agreed to go with John's suggestion.

Hi,

my previous attempt was loading a whole ELF file and parsing it in kernel.
In this series I just validate the instructions against a signature,
as with kernel CO-RE libbpf doesn't need to mangle it.

Quoted 4 lines
Quoted 1 line

Which suggestion? I think I missed this one..

This talk and discussion:
https://linuxplumbersconf.org/event/11/contributions/947/

Thanks for the link - but for those of us who don't have ~5 hours to watch a video recording, would you mind sharing a one line summary, please? Is there an alternative patch series implementing BPF signing that you can link us so that we can look at it? Just a link or googlable reference would be more than enough.

Thank you!

Signature · 2 lines
Kind regards,
Luca Boccassi
#9 lore raw
Quoted 34 lines · Luca Boccassi
Quoted 32 lines · Alexei Starovoitov
Quoted 29 lines · Matteo Croce
Quoted 12 lines · Alexei Starovoitov
Quoted 8 lines · Matteo Croce

From: Matteo Croce <[email protected]>

This series add signature verification for BPF files. The first patch implements the signature validation in the kernel,
the second patch optionally makes the signature mandatory, the third adds signature generation to bpftool.

Quoted 4 lines

Matteo,

I think I already mentioned that it's no-go as-is. We've agreed to go with John's suggestion.

Hi,

my previous attempt was loading a whole ELF file and parsing it in kernel.
In this series I just validate the instructions against a signature,
as with kernel CO-RE libbpf doesn't need to mangle it.

Which suggestion? I think I missed this one..

Quoted 8 lines
Quoted 2 lines

This talk and discussion:
https://linuxplumbersconf.org/event/11/contributions/947/

Thanks for the link - but for those of us who don't have ~5 hours to watch a video recording, would you mind sharing a one line summary, please? Is there an alternative patch series implementing BPF signing that you can link us so that we can look at it? Just a link or googlable reference would be more than enough.

It's not 5 hours and you have to read slides and watch John's presentation to follow the conversation.

#10 lore raw
Quoted 39 lines · Alexei Starovoitov
Quoted 37 lines · Luca Boccassi
Quoted 34 lines · Alexei Starovoitov
Quoted 22 lines · Matteo Croce
Quoted 12 lines · Alexei Starovoitov
Quoted 8 lines · Matteo Croce

From: Matteo Croce <[email protected]>

This series add signature verification for BPF files. The first patch implements the signature validation in the kernel,
the second patch optionally makes the signature mandatory, the third adds signature generation to bpftool.

Quoted 4 lines

Matteo,

I think I already mentioned that it's no-go as-is. We've agreed to go with John's suggestion.

Hi,

Quoted 7 lines

my previous attempt was loading a whole ELF file and parsing it in kernel.
In this series I just validate the instructions against a signature,
as with kernel CO-RE libbpf doesn't need to mangle it.

Which suggestion? I think I missed this one..

This talk and discussion:
https://linuxplumbersconf.org/event/11/contributions/947/

Quoted 8 lines
Quoted 5 lines

Thanks for the link - but for those of us who don't have ~5 hours to watch a video recording, would you mind sharing a one line summary, please? Is there an alternative patch series implementing BPF signing that you can link us so that we can look at it? Just a link or googlable reference would be more than enough.

It's not 5 hours and you have to read slides and watch John's presentation to follow the conversation.

So, If I have understood correctly, the proposal is to validate the tools which loads the BPF (e.g. perf, ip) with fs-verity, and only allow BPF loading from those validated binaries?
That's nice, but I think that this could be complementary to the instructions signature.
Imagine a validated binary being exploited somehow at runtime, that could be vector of malicious BPF program load.
Can't we have both available, and use one or other, or even both together depending on the use case?

Regards,

Signature · 1 line
per aspera ad upstream
#11 lore raw
Quoted 51 lines · Matteo Croce
Quoted 39 lines · Alexei Starovoitov
Quoted 37 lines · Luca Boccassi
Quoted 34 lines · Alexei Starovoitov
Quoted 22 lines · Matteo Croce
Quoted 12 lines · Alexei Starovoitov
Quoted 8 lines · Matteo Croce

From: Matteo Croce <[email protected]>

This series add signature verification for BPF files. The first patch implements the signature validation in the kernel,
the second patch optionally makes the signature mandatory, the third adds signature generation to bpftool.

Quoted 4 lines

Matteo,

I think I already mentioned that it's no-go as-is. We've agreed to go with John's suggestion.

Hi,

Quoted 7 lines

my previous attempt was loading a whole ELF file and parsing it in kernel.
In this series I just validate the instructions against a signature,
as with kernel CO-RE libbpf doesn't need to mangle it.

Which suggestion? I think I missed this one..

This talk and discussion:
https://linuxplumbersconf.org/event/11/contributions/947/

Quoted 8 lines
Quoted 5 lines

Thanks for the link - but for those of us who don't have ~5 hours to watch a video recording, would you mind sharing a one line summary, please? Is there an alternative patch series implementing BPF signing that you can link us so that we can look at it? Just a link or googlable reference would be more than enough.

It's not 5 hours and you have to read slides and watch John's presentation to follow the conversation.

Quoted 9 lines

So, If I have understood correctly, the proposal is to validate the tools which loads the BPF (e.g. perf, ip) with fs-verity, and only allow BPF loading from those validated binaries?
That's nice, but I think that this could be complementary to the instructions signature.
Imagine a validated binary being exploited somehow at runtime, that could be vector of malicious BPF program load.
Can't we have both available, and use one or other, or even both together depending on the use case?

I'll let John comment.

#12 lore raw
Quoted 62 lines · Alexei Starovoitov
Quoted 51 lines · Matteo Croce
Quoted 39 lines · Alexei Starovoitov
Quoted 37 lines · Luca Boccassi
Quoted 34 lines · Alexei Starovoitov
Quoted 22 lines · Matteo Croce
Quoted 12 lines · Alexei Starovoitov
Quoted 8 lines · Matteo Croce

From: Matteo Croce <[email protected]>

This series add signature verification for BPF files. The first patch implements the signature validation in the kernel,
the second patch optionally makes the signature mandatory, the third adds signature generation to bpftool.

Quoted 4 lines

Matteo,

I think I already mentioned that it's no-go as-is. We've agreed to go with John's suggestion.

Hi,

Quoted 7 lines

my previous attempt was loading a whole ELF file and parsing it in kernel.
In this series I just validate the instructions against a signature,
as with kernel CO-RE libbpf doesn't need to mangle it.

Which suggestion? I think I missed this one..

This talk and discussion:
https://linuxplumbersconf.org/event/11/contributions/947/

Quoted 8 lines
Quoted 5 lines

Thanks for the link - but for those of us who don't have ~5 hours to watch a video recording, would you mind sharing a one line summary, please? Is there an alternative patch series implementing BPF signing that you can link us so that we can look at it? Just a link or googlable reference would be more than enough.

It's not 5 hours and you have to read slides and watch John's presentation to follow the conversation.

Quoted 9 lines

So, If I have understood correctly, the proposal is to validate the tools which loads the BPF (e.g. perf, ip) with fs-verity, and only allow BPF loading from those validated binaries?
That's nice, but I think that this could be complementary to the instructions signature.
Imagine a validated binary being exploited somehow at runtime, that could be vector of malicious BPF program load.
Can't we have both available, and use one or other, or even both together depending on the use case?

Quoted 1 line

I'll let John comment.

I'll give the outline of the argument here.

I do not believe signing BPF instructions for real programs provides much additional security. Given most real programs if the application or loader is exploited at runtime we have all sorts of trouble. First simply verifying the program doesn't prevent malicious use of the program. If its in the network program this means DDOS, data exfiltration, mitm attacks, many other possibilities. If its enforcement program most enforcement actions are programmed from this application so system security is lost already. If its observability application simply drops/manipulates observations that it wants. I don't know of any useful programs that exist in isolation without user space input and output as a critical component. If its not a privileged user, well it better not be doing anything critical anyways or disabled outright for the security focused.

Many critical programs can't be signed by the nature of the program. Optimizing network app generates optimized code at runtime. Observability tools JIT the code on the fly, similarly enforcement tools will do the same. I think the power of being able to optimize JIT the code in application and give to the kernel is something we will see more and more of. Saying I'm only going to accept signed programs, for a distribution or something other than niche use case, is non starter IMO because it breaks so many real use cases. We should encourage these optimizing use cases as I see it as critical to performance and keeping overhead low.

From a purely security standpoint I believe you are better off defining characteristics an application is allowed to have. For example allowed to probe kernel memory, make these helpers calls, have this many instructions, use this much memory, this much cpu, etc. This lets you sandbox a BPF application (both user space and kernel side) much nicer than any signing will allow.

If we want to 'sign' programs we should do that from a BPF program directly where other metadata can be included in the policy. For example having a hash of the program loaded along with the calls made and process allows for rich policy decisions. I have other use cases that need a hash/signature for data blobs, so its on my todo list but not at the top yet. But, being able to verify arbitrary blob of data from BPF feels like a useful operation to me in general. The fact in your case its a set of eBPF insns and in my case its some key in a network header shouldn't matter.

The series as is, scanned commit descriptions, is going to break lots of in-use-today programs if it was ever enabled. And is not as flexible (can't support bpftrace, etc.) or powerful (can't consider fine grained policy decisions) as above.

Add a function we can hook after verify (or before up for debate) and helpers to verify signatures and/or generate hashes and we get a better more general solution. And it can also solve your use case even if I believe its not useful and may break many BPF users running bpftrace, libbpf, etc.

Thanks,
John

#13 lore raw
Quoted 130 lines · John Fastabend
Quoted 74 lines · Alexei Starovoitov
Quoted 59 lines · Matteo Croce
Quoted 53 lines · Alexei Starovoitov
Quoted 40 lines · Luca Boccassi
Quoted 33 lines · Alexei Starovoitov
Quoted 31 lines · Matteo Croce
Quoted 14 lines · Alexei Starovoitov
Quoted 10 lines · Matteo Croce

From: Matteo Croce <[email protected]>

This series add signature verification for BPF files. The first patch implements the signature validation in the
kernel,
the second patch optionally makes the signature mandatory,
the third adds signature generation to bpftool.

Quoted 4 lines

Matteo,

I think I already mentioned that it's no-go as-is. We've agreed to go with John's suggestion.

Hi,

my previous attempt was loading a whole ELF file and parsing it in
kernel.
In this series I just validate the instructions against a signature,
as with kernel CO-RE libbpf doesn't need to mangle it.

Quoted 4 lines
Quoted 1 line

Which suggestion? I think I missed this one..

This talk and discussion:
https://linuxplumbersconf.org/event/11/contributions/947/

Quoted 9 lines

Thanks for the link - but for those of us who don't have ~5 hours to
watch a video recording, would you mind sharing a one line summary,
please? Is there an alternative patch series implementing BPF signing
that you can link us so that we can look at it? Just a link or
googlable reference would be more than enough.

Quoted 2 lines

It's not 5 hours and you have to read slides and watch John's presentation to follow the conversation.

Quoted 12 lines

So, If I have understood correctly, the proposal is to validate the
tools which loads the BPF (e.g. perf, ip) with fs-verity, and only
allow BPF loading from those validated binaries? That's nice, but I think that this could be complementary to the instructions signature.
Imagine a validated binary being exploited somehow at runtime, that
could be vector of malicious BPF program load.
Can't we have both available, and use one or other, or even both together depending on the use case?

Quoted 1 line

I'll let John comment.

I'll give the outline of the argument here.

I do not believe signing BPF instructions for real programs provides much additional security. Given most real programs if the application or loader is exploited at runtime we have all sorts of trouble. First simply verifying the program doesn't prevent malicious use of the program. If its in the network program this means DDOS, data exfiltration,
mitm attacks, many other possibilities. If its enforcement program most enforcement actions are programmed from this application so system
security is lost already.  If its observability application simply drops/manipulates observations that it wants. I don't know of any useful programs that exist in isolation without user space input and output as a critical component. If its not a privileged user, well it better not be doing anything critical anyways or disabled outright for the security focused.

Many critical programs can't be signed by the nature of the program. Optimizing network app generates optimized code at runtime. Observability
tools JIT the code on the fly, similarly enforcement tools will do the
same. I think the power of being able to optimize JIT the code in application and give to the kernel is something we will see more and more of. Saying I'm only going to accept signed programs, for a distribution or something other than niche use case, is non starter IMO because it breaks so many real use cases. We should encourage these optimizing use cases as I see it as critical to performance and keeping overhead low.

From a purely security standpoint I believe you are better off defining characteristics an application is allowed to have. For example allowed to probe kernel memory, make these helpers calls, have this many instructions, use this much memory, this much cpu, etc. This lets you sandbox a BPF application (both user space and kernel side) much nicer than any signing will allow.

If we want to 'sign' programs we should do that from a BPF program directly where other metadata can be included in the policy. For example having a hash of the program loaded along with the calls made and process allows for rich policy decisions. I have other use cases that need a hash/signature for data blobs, so its on my todo list but not at the top yet.  But, being able to verify arbitrary blob of data from BPF feels like a useful operation to me in general. The fact in your case its a set of eBPF insns and in my case its some key in a network header shouldn't matter.

The series as is, scanned commit descriptions, is going to break lots of in-use-today programs if it was ever enabled. And is not as flexible (can't support bpftrace, etc.) or powerful (can't consider fine grained policy decisions) as above.

Quoted 8 lines

Add a function we can hook after verify (or before up for debate) and helpers to verify signatures and/or generate hashes and we get a better more general solution. And it can also solve your use case even if I believe its not useful and may break many BPF users running bpftrace, libbpf, etc.

Thanks,
John

Hello John,

Thank you for the summary, this is much clearer.

First of all, I think there's some misunderstanding: this series does not enable optional signatures by default, and does not enable mandatory signatures by default either. So I don't see how it would break existing use cases as you are saying? Unless I'm missing something?

There's a kconfig to enable optional signatures - if they are there, they are verified, if they are not present then nothing different happens. Unless I am missing something, this should be backward compatible. This kconfig would likely be enabled in most use cases, just like optionally signed kernel modules are.

Then there's a kconfig on top of that which makes signatures mandatory. I would not imagine this to be enabled in may cases, just in custom builds that have more stringent requirements. It certainly would not be enabled in generalist distros. Perhaps a more flexible way would be to introduce a sysctl, like fsverity has with 'fs.verity.require_signatures'? That would be just fine for our use case. Matteo can we do that instead in the next revision?

Secondly, I understand that for your use case signing programs would not be the best approach. That's fine, and I'm glad you are working on an alternative that better fits your model, it will be very interesting to see how it looks like once implemented. But that model doesn't fit all cases. In our case at Microsoft, we absolutely want to be able to pre-define at build time a list of BPF programs that are allowed to be loaded, and reject anything else. Userspace processes in our case are mostly old and crufty c++ programs that can most likely be pwned by looking at them sideways, so they get locked down hard with multiple redundant layers and so on and so forth. But right now for BPF you only have a "can load BPF" or "cannot load BPF" knob, and that's it. This is not good enough: we need to be able to define a list of allowed payloads, and be able to enforce it, so when (not if) said processes do get tricked into loading something else, it will fail, despite having the capability of calling bpf(). Trying to define heuristics is also not good enough for us - creative malicious actors have a tendency to come up with ways to chain things that individually are allowed and benign, but combined in a way that you just couldn't foresee. It would certainly cover a lot of cases, but not all. A strictly pre-defined list of what is allowed to run and what is not is what we need for our case, so that we always know exactly what is going to run and what is not, and can deal with the consequences accordingly, without nasty surprises waiting around the corner. Now in my naive view the best way to achieve this is via signatures and certs, as it's a well-understood system, with processes already in place to revoke/rotate/etc, and it's already used for kmods. An alternative would be hard-coding hashes I guess, but that would be terribly inflexible.

Now in terms of _how_ the signatures are done and validated, I'm sure there are multiple ways, and if some are better than what this series implements, then that's not an issue, it can be reworked. But the core requirement for us is: offline pre-defined list of what is allowed to run and what is not, with ability for hard enforcement that cannot be bypassed. Yes, you lose some features like JIT and so on: we don't care, we don't need those for our use cases. If others have different needs that's fine, this is all intended to be optional, not mandatory. There are obviously trade-offs, as always when security is involved, and each user can decide what's best for them.

Hope this makes sense. Thanks!

Signature · 2 lines
Kind regards,
Luca Boccassi
#14 lore raw

Luca Boccassi wrote:

cutting to just the relevant pieces here.

[...]

Quoted 73 lines
Quoted 54 lines

I'll give the outline of the argument here.

I do not believe signing BPF instructions for real programs provides much additional security. Given most real programs if the application or loader is exploited at runtime we have all sorts of trouble. First simply verifying the program doesn't prevent malicious use of the program. If its in the network program this means DDOS, data exfiltration,
mitm attacks, many other possibilities. If its enforcement program most enforcement actions are programmed from this application so system
security is lost already.  If its observability application simply drops/manipulates observations that it wants. I don't know of any useful programs that exist in isolation without user space input and output as a critical component. If its not a privileged user, well it better not be doing anything critical anyways or disabled outright for the security focused.

Many critical programs can't be signed by the nature of the program. Optimizing network app generates optimized code at runtime. Observability
tools JIT the code on the fly, similarly enforcement tools will do the
same. I think the power of being able to optimize JIT the code in application and give to the kernel is something we will see more and more of. Saying I'm only going to accept signed programs, for a distribution or something other than niche use case, is non starter IMO because it breaks so many real use cases. We should encourage these optimizing use cases as I see it as critical to performance and keeping overhead low.

From a purely security standpoint I believe you are better off defining characteristics an application is allowed to have. For example allowed to probe kernel memory, make these helpers calls, have this many instructions, use this much memory, this much cpu, etc. This lets you sandbox a BPF application (both user space and kernel side) much nicer than any signing will allow.

If we want to 'sign' programs we should do that from a BPF program directly where other metadata can be included in the policy. For example having a hash of the program loaded along with the calls made and process allows for rich policy decisions. I have other use cases that need a hash/signature for data blobs, so its on my todo list but not at the top yet.  But, being able to verify arbitrary blob of data from BPF feels like a useful operation to me in general. The fact in your case its a set of eBPF insns and in my case its some key in a network header shouldn't matter.

The series as is, scanned commit descriptions, is going to break lots of in-use-today programs if it was ever enabled. And is not as flexible (can't support bpftrace, etc.) or powerful (can't consider fine grained policy decisions) as above.

Quoted 8 lines

Add a function we can hook after verify (or before up for debate) and helpers to verify signatures and/or generate hashes and we get a better more general solution. And it can also solve your use case even if I believe its not useful and may break many BPF users running bpftrace, libbpf, etc.

Thanks,
John

Hello John,

Thank you for the summary, this is much clearer.

First of all, I think there's some misunderstanding: this series does not enable optional signatures by default, and does not enable mandatory signatures by default either. So I don't see how it would break existing use cases as you are saying? Unless I'm missing something?

Quoted 5 lines

There's a kconfig to enable optional signatures - if they are there, they are verified, if they are not present then nothing different happens. Unless I am missing something, this should be backward compatible. This kconfig would likely be enabled in most use cases, just like optionally signed kernel modules are.

Agree, without enforcement things should continue to work.

Quoted 8 lines

Then there's a kconfig on top of that which makes signatures mandatory. I would not imagine this to be enabled in may cases, just in custom builds that have more stringent requirements. It certainly would not be enabled in generalist distros. Perhaps a more flexible way would be to introduce a sysctl, like fsverity has with 'fs.verity.require_signatures'? That would be just fine for our use case. Matteo can we do that instead in the next revision?

We want to manage this from BPF side directly. It looks like policy decision and we have use cases that are not as simple as yes/no with global switch. For example, in k8s world this might be enabled via labels which are user specific per container policy. e.g. lockdown some containers more strictly than others.

Quoted 8 lines

Secondly, I understand that for your use case signing programs would not be the best approach. That's fine, and I'm glad you are working on an alternative that better fits your model, it will be very interesting to see how it looks like once implemented. But that model doesn't fit all cases. In our case at Microsoft, we absolutely want to be able to pre-define at build time a list of BPF programs that are allowed to be loaded, and reject anything else. Userspace processes in our case are

By building this into BPF you can get the 'reject anything else' policy and I get the metadata + reject/accept from the same hook. Its just your program can be very simple.

Quoted 7 lines

mostly old and crufty c++ programs that can most likely be pwned by looking at them sideways, so they get locked down hard with multiple redundant layers and so on and so forth. But right now for BPF you only have a "can load BPF" or "cannot load BPF" knob, and that's it. This is not good enough: we need to be able to define a list of allowed payloads, and be able to enforce it, so when (not if) said processes do get tricked into loading something else, it will fail, despite having

Yikes, this is a bit scary from a sec point of view right? Are those programs read-only maps or can the C++ program also write into the maps and control plane. Assuming they do some critical functions then you really shouldn't be trusting them to not do all sorts of other horrible things. Anyways not too important to this discussion.

I'll just reiterate (I think you get it though) that simply signing enforcement doesn't mean now BPF is safe. Further these programs have very high privileges and can do all sorts of things to the system. But, sure sig enforcement locks down one avenue of loading bogus program.

Quoted 4 lines

the capability of calling bpf(). Trying to define heuristics is also not good enough for us - creative malicious actors have a tendency to come up with ways to chain things that individually are allowed and benign, but combined in a way that you just couldn't foresee. It would

Sure, but I would argue some things can be very restrictive and generally useful. For example, never allow kernel memory read could be enforced from BPF side directly. Never allow pkt redirect, etc.

Quoted 9 lines

certainly cover a lot of cases, but not all. A strictly pre-defined list of what is allowed to run and what is not is what we need for our case, so that we always know exactly what is going to run and what is not, and can deal with the consequences accordingly, without nasty surprises waiting around the corner. Now in my naive view the best way to achieve this is via signatures and certs, as it's a well-understood system, with processes already in place to revoke/rotate/etc, and it's already used for kmods. An alternative would be hard-coding hashes I guess, but that would be terribly inflexible.

Another option would be to load your programs at boot time, presumably with trusted boot enabled and then lock down BPF completely. Then ensure all your BPF 'programs' are read-only from user<->kernel interface and this should start looking fairly close to what you want and all programs are correct by root of trust back to trusted boot. Would assume you know what programs to load at boot though. May or may not be a big assumption depending on your env.

Quoted 12 lines

Now in terms of _how_ the signatures are done and validated, I'm sure there are multiple ways, and if some are better than what this series implements, then that's not an issue, it can be reworked. But the core requirement for us is: offline pre-defined list of what is allowed to run and what is not, with ability for hard enforcement that cannot be bypassed. Yes, you lose some features like JIT and so on: we don't care, we don't need those for our use cases. If others have different needs that's fine, this is all intended to be optional, not mandatory. There are obviously trade-offs, as always when security is involved, and each user can decide what's best for them.

Quoted 1 line

Hope this makes sense. Thanks!

I think I understand your use case. When done as BPF helper you can get the behavior you want with a one line BPF program loaded at boot.

int verify_all(struct bpf_prog **prog) {
	return verify_signature(prog->insn,
				prog->len * sizeof(struct bpf_insn),
			        signature, KEYRING, BPF_SIGTYPE);
}

And I can write some more specific things as,

int verify_blobs(void data) {
int reject = verify_signature(data, data_len, sig, KEYRING, TYPE); struct policy_key *key = map_get_key();

return policy(key, reject);
}

map_get_key() looks into some datastor with the policy likely using 'current' to dig something up. It doesn't just apply to BPF progs we can use it on other executables more generally. And I get more interesting use cases like, allowing 'tc' programs unsigned, but requiring kernel memory reads to require signatures or any N other policies that may have value. Or only allowing my dbg user to run read-only programs, because the dbg maybe shouldn't ever be writing into packets, etc. Driving least privilege use cases in fine detail.

By making it a BPF program we side step the debate where the kernel tries to get the 'right' policy for you, me, everyone now and in the future. The only way I can see to do this without getting N policies baked into the kernel and at M different hook points is via a BPF helper.

Thanks,
John

#15 lore raw

Em Mon, Dec 06, 2021 at 12:40:40PM -0800, John Fastabend escreveu:

Quoted 2 lines

I'll just reiterate (I think you get it though) that simply signing enforcement doesn't mean now BPF is safe. Further these programs

I think this was clear from the get go, at most this would help with fingerpointing :-) I.e. BPF signing is not about making things safer, its just an attempt to know who messed up.

Quoted 4 lines

have very high privileges and can do all sorts of things to the system. But, sure sig enforcement locks down one avenue of loading bogus program.

Quoted 8 lines
Quoted 4 lines

the capability of calling bpf(). Trying to define heuristics is also not good enough for us - creative malicious actors have a tendency to come up with ways to chain things that individually are allowed and benign, but combined in a way that you just couldn't foresee. It would

Sure, but I would argue some things can be very restrictive and generally useful. For example, never allow kernel memory read could be enforced from BPF side directly. Never allow pkt redirect, etc.

But this is something unrelated to BPF signing, right? Its something desirable, I'd say this will be at some point required, i.e. one more step in having BPF programs to be more like userspace apps, where you can limit all sorts of things it can do, programmatically, a BPF ulimit, hey, blimit?

- Arnaldo

#16 lore raw
In reply to John Fastabend
Quoted 159 lines · John Fastabend

Luca Boccassi wrote:

cutting to just the relevant pieces here.

[...]

Quoted 85 lines
Quoted 65 lines

I'll give the outline of the argument here.

I do not believe signing BPF instructions for real programs provides
much additional security. Given most real programs if the application
or loader is exploited at runtime we have all sorts of trouble. First
simply verifying the program doesn't prevent malicious use of the program. If its in the network program this means DDOS, data exfiltration,
mitm attacks, many other possibilities. If its enforcement program
most enforcement actions are programmed from this application so system
security is lost already.  If its observability application simply
drops/manipulates observations that it wants. I don't know of any useful programs that exist in isolation without user space input and output as a critical component. If its not a privileged user, well it better not be doing anything critical anyways or disabled outright for the security focused.

Many critical programs can't be signed by the nature of the program.
Optimizing network app generates optimized code at runtime. Observability
tools JIT the code on the fly, similarly enforcement tools will do
the
same. I think the power of being able to optimize JIT the code in application and give to the kernel is something we will see more and
more of. Saying I'm only going to accept signed programs, for a distribution or something other than niche use case, is non starter
IMO because it breaks so many real use cases. We should encourage these optimizing use cases as I see it as critical to performance and keeping overhead low.

From a purely security standpoint I believe you are better off defining characteristics an application is allowed to have. For example allowed to probe kernel memory, make these helpers calls, have this many instructions, use this much memory, this much cpu, etc. This lets you sandbox a BPF application (both user space and kernel side) much nicer than any signing will allow.

If we want to 'sign' programs we should do that from a BPF program
directly where other metadata can be included in the policy. For example having a hash of the program loaded along with the calls made and process allows for rich policy decisions. I have other use cases that need a hash/signature for data blobs, so its on my todo list but not at the top yet.  But, being able to verify arbitrary blob of data from BPF feels like a useful operation to me
in general. The fact in your case its a set of eBPF insns and in my case its some key in a network header shouldn't matter.

The series as is, scanned commit descriptions, is going to break lots of in-use-today programs if it was ever enabled. And is not as flexible (can't support bpftrace, etc.) or powerful (can't consider fine grained policy decisions) as above.

Quoted 8 lines

Add a function we can hook after verify (or before up for debate) and helpers to verify signatures and/or generate hashes and we get a better more general solution. And it can also solve your use case even if I believe its not useful and may break many BPF users running bpftrace, libbpf, etc.

Thanks,
John

Hello John,

Thank you for the summary, this is much clearer.

First of all, I think there's some misunderstanding: this series does
not enable optional signatures by default, and does not enable mandatory signatures by default either. So I don't see how it would break existing use cases as you are saying? Unless I'm missing something?

Quoted 6 lines

There's a kconfig to enable optional signatures - if they are there,
they are verified, if they are not present then nothing different happens. Unless I am missing something, this should be backward compatible. This kconfig would likely be enabled in most use cases, just like optionally signed kernel modules are.

Agree, without enforcement things should continue to work.

Quoted 11 lines

Then there's a kconfig on top of that which makes signatures mandatory.
I would not imagine this to be enabled in may cases, just in custom builds that have more stringent requirements. It certainly would not be
enabled in generalist distros. Perhaps a more flexible way would be to
introduce a sysctl, like fsverity has with 'fs.verity.require_signatures'? That would be just fine for our use case. Matteo can we do that instead in the next revision?

We want to manage this from BPF side directly. It looks like policy decision and we have use cases that are not as simple as yes/no with global switch. For example, in k8s world this might be enabled via labels which are user specific per container
policy. e.g. lockdown some containers more strictly than others.

Quoted 15 lines

Secondly, I understand that for your use case signing programs would
not be the best approach. That's fine, and I'm glad you are working on
an alternative that better fits your model, it will be very interesting
to see how it looks like once implemented. But that model doesn't fit
all cases. In our case at Microsoft, we absolutely want to be able to
pre-define at build time a list of BPF programs that are allowed to be
loaded, and reject anything else. Userspace processes in our case are

By building this into BPF you can get the 'reject anything else' policy
and I get the metadata + reject/accept from the same hook. Its just your program can be very simple.

Quoted 4 lines

mostly old and crufty c++ programs that can most likely be pwned by looking at them sideways, so they get locked down hard with multiple
redundant layers and so on and so forth. But right now for BPF you

Quoted 8 lines

only
have a "can load BPF" or "cannot load BPF" knob, and that's it. This is
not good enough: we need to be able to define a list of allowed payloads, and be able to enforce it, so when (not if) said processes do
get tricked into loading something else, it will fail, despite having

Yikes, this is a bit scary from a sec point of view right? Are those programs read-only maps or can the C++ program also write into the maps and control plane. Assuming they do some critical functions then you really shouldn't be trusting them to not do all sorts of other horrible things. Anyways not too important to this discussion.

Quoted 5 lines

I'll just reiterate (I think you get it though) that simply signing enforcement doesn't mean now BPF is safe. Further these programs have very high privileges and can do all sorts of things to the system. But, sure sig enforcement locks down one avenue of loading bogus program.

Oh it's terrifying - but business needs and all that.
But Arnaldo is spot on - it's not strictly about what is more secure, but more about making it a known quantity. If we can prove what is allowed to run and what not before any machine has even booted (barring bugs in sig verification, of course) then the $org_security_team is satisfied and can sign off on enabling bpf. Otherwise we can keep dreaming.

Quoted 29 lines
Quoted 7 lines

the capability of calling bpf(). Trying to define heuristics is also
not good enough for us - creative malicious actors have a tendency to
come up with ways to chain things that individually are allowed and benign, but combined in a way that you just couldn't foresee. It would

Sure, but I would argue some things can be very restrictive and generally useful. For example, never allow kernel memory read could be
enforced from BPF side directly. Never allow pkt redirect, etc.

Quoted 7 lines

certainly cover a lot of cases, but not all. A strictly pre-defined list of what is allowed to run and what is not is what we need for our
case, so that we always know exactly what is going to run and what is
not, and can deal with the consequences accordingly, without nasty surprises waiting around the corner. Now in my naive view the best

Quoted 8 lines

way
to achieve this is via signatures and certs, as it's a well- understood
system, with processes already in place to revoke/rotate/etc, and it's
already used for kmods. An alternative would be hard-coding hashes I
guess, but that would be terribly inflexible.

Quoted 8 lines

Another option would be to load your programs at boot time, presumably
with trusted boot enabled and then lock down BPF completely. Then ensure all your BPF 'programs' are read-only from user<->kernel interface and this should start looking fairly close to what you want and all programs are correct by root of trust back to trusted boot. Would assume you know what programs to load at boot though. May or may not be a big assumption depending on your env.

One of the use cases we have for BPF is on-demand diagnostics, so loading at boot and blocking afterwards would not work, I think. Environment is constrained in terms of resources, so don't want to load anything that is not needed.

Quoted 51 lines
Quoted 20 lines

Now in terms of _how_ the signatures are done and validated, I'm sure
there are multiple ways, and if some are better than what this series
implements, then that's not an issue, it can be reworked. But the core
requirement for us is: offline pre-defined list of what is allowed to
run and what is not, with ability for hard enforcement that cannot be
bypassed. Yes, you lose some features like JIT and so on: we don't care, we don't need those for our use cases. If others have different
needs that's fine, this is all intended to be optional, not mandatory.
There are obviously trade-offs, as always when security is involved,
and each user can decide what's best for them.

Quoted 1 line

Hope this makes sense. Thanks!

I think I understand your use case. When done as BPF helper you can get the behavior you want with a one line BPF program loaded at boot.

int verify_all(struct bpf_prog **prog) {
        return verify_signature(prog->insn,
                                prog->len * sizeof(struct bpf_insn), signature, KEYRING, BPF_SIGTYPE); }

And I can write some more specific things as,

int verify_blobs(void data) {
  int reject = verify_signature(data, data_len, sig, KEYRING, TYPE); struct policy_key *key = map_get_key();

  return policy(key, reject);
}

map_get_key() looks into some datastor with the policy likely using 'current' to dig something up. It doesn't just apply to BPF progs we can use it on other executables more generally. And I get more interesting use cases like, allowing 'tc' programs unsigned, but requiring kernel memory reads to require signatures or any N other policies that may have value. Or only allowing my dbg user to run read-only programs, because the dbg maybe shouldn't ever be writing into packets, etc. Driving least privilege use cases in fine detail.

Quoted 8 lines

By making it a BPF program we side step the debate where the kernel tries to get the 'right' policy for you, me, everyone now and in the future. The only way I can see to do this without getting N policies baked into the kernel and at M different hook points is via a BPF helper.

Thanks,
John

Now this sounds like something that could work - we can prove that this could be loaded before any writable fs comes up anywhere, so in principle I think it would be acceptable and free of races. Matteo, we should talk about this tomorrow.
And this requires some infrastructure work right? Is there a WIP git tree somewhere that we can test out?

Thank you!

Signature · 2 lines
Kind regards,
Luca Boccassi
#17 lore raw
Quoted 277 lines · Luca Boccassi
Quoted 159 lines · John Fastabend

Luca Boccassi wrote:

cutting to just the relevant pieces here.

[...]

Quoted 85 lines
Quoted 65 lines

I'll give the outline of the argument here.

I do not believe signing BPF instructions for real programs provides
much additional security. Given most real programs if the application
or loader is exploited at runtime we have all sorts of trouble. First
simply verifying the program doesn't prevent malicious use of the program. If its in the network program this means DDOS, data exfiltration,
mitm attacks, many other possibilities. If its enforcement program
most enforcement actions are programmed from this application so system
security is lost already.  If its observability application simply
drops/manipulates observations that it wants. I don't know of any useful programs that exist in isolation without user space input and output as a critical component. If its not a privileged user, well it better not be doing anything critical anyways or disabled outright for the security focused.

Many critical programs can't be signed by the nature of the program.
Optimizing network app generates optimized code at runtime. Observability
tools JIT the code on the fly, similarly enforcement tools will do
the
same. I think the power of being able to optimize JIT the code in application and give to the kernel is something we will see more and
more of. Saying I'm only going to accept signed programs, for a distribution or something other than niche use case, is non starter
IMO because it breaks so many real use cases. We should encourage these optimizing use cases as I see it as critical to performance and keeping overhead low.

From a purely security standpoint I believe you are better off defining characteristics an application is allowed to have. For example allowed to probe kernel memory, make these helpers calls, have this many instructions, use this much memory, this much cpu, etc. This lets you sandbox a BPF application (both user space and kernel side) much nicer than any signing will allow.

If we want to 'sign' programs we should do that from a BPF program
directly where other metadata can be included in the policy. For example having a hash of the program loaded along with the calls made and process allows for rich policy decisions. I have other use cases that need a hash/signature for data blobs, so its on my todo list but not at the top yet.  But, being able to verify arbitrary blob of data from BPF feels like a useful operation to me
in general. The fact in your case its a set of eBPF insns and in my case its some key in a network header shouldn't matter.

The series as is, scanned commit descriptions, is going to break lots of in-use-today programs if it was ever enabled. And is not as flexible (can't support bpftrace, etc.) or powerful (can't consider fine grained policy decisions) as above.

Quoted 8 lines

Add a function we can hook after verify (or before up for debate) and helpers to verify signatures and/or generate hashes and we get a better more general solution. And it can also solve your use case even if I believe its not useful and may break many BPF users running bpftrace, libbpf, etc.

Thanks,
John

Hello John,

Thank you for the summary, this is much clearer.

First of all, I think there's some misunderstanding: this series does
not enable optional signatures by default, and does not enable mandatory signatures by default either. So I don't see how it would break existing use cases as you are saying? Unless I'm missing something?

Quoted 6 lines

There's a kconfig to enable optional signatures - if they are there,
they are verified, if they are not present then nothing different happens. Unless I am missing something, this should be backward compatible. This kconfig would likely be enabled in most use cases, just like optionally signed kernel modules are.

Agree, without enforcement things should continue to work.

Quoted 11 lines

Then there's a kconfig on top of that which makes signatures mandatory.
I would not imagine this to be enabled in may cases, just in custom builds that have more stringent requirements. It certainly would not be
enabled in generalist distros. Perhaps a more flexible way would be to
introduce a sysctl, like fsverity has with 'fs.verity.require_signatures'? That would be just fine for our use case. Matteo can we do that instead in the next revision?

We want to manage this from BPF side directly. It looks like policy decision and we have use cases that are not as simple as yes/no with global switch. For example, in k8s world this might be enabled via labels which are user specific per container
policy. e.g. lockdown some containers more strictly than others.

Quoted 15 lines

Secondly, I understand that for your use case signing programs would
not be the best approach. That's fine, and I'm glad you are working on
an alternative that better fits your model, it will be very interesting
to see how it looks like once implemented. But that model doesn't fit
all cases. In our case at Microsoft, we absolutely want to be able to
pre-define at build time a list of BPF programs that are allowed to be
loaded, and reject anything else. Userspace processes in our case are

By building this into BPF you can get the 'reject anything else' policy
and I get the metadata + reject/accept from the same hook. Its just your program can be very simple.

Quoted 4 lines

mostly old and crufty c++ programs that can most likely be pwned by looking at them sideways, so they get locked down hard with multiple
redundant layers and so on and so forth. But right now for BPF you

Quoted 8 lines

only
have a "can load BPF" or "cannot load BPF" knob, and that's it. This is
not good enough: we need to be able to define a list of allowed payloads, and be able to enforce it, so when (not if) said processes do
get tricked into loading something else, it will fail, despite having

Yikes, this is a bit scary from a sec point of view right? Are those programs read-only maps or can the C++ program also write into the maps and control plane. Assuming they do some critical functions then you really shouldn't be trusting them to not do all sorts of other horrible things. Anyways not too important to this discussion.

Quoted 5 lines

I'll just reiterate (I think you get it though) that simply signing enforcement doesn't mean now BPF is safe. Further these programs have very high privileges and can do all sorts of things to the system. But, sure sig enforcement locks down one avenue of loading bogus program.

Oh it's terrifying - but business needs and all that.
But Arnaldo is spot on - it's not strictly about what is more secure, but more about making it a known quantity. If we can prove what is allowed to run and what not before any machine has even booted (barring bugs in sig verification, of course) then the $org_security_team is satisfied and can sign off on enabling bpf. Otherwise we can keep dreaming.

Quoted 29 lines
Quoted 7 lines

the capability of calling bpf(). Trying to define heuristics is also
not good enough for us - creative malicious actors have a tendency to
come up with ways to chain things that individually are allowed and benign, but combined in a way that you just couldn't foresee. It would

Sure, but I would argue some things can be very restrictive and generally useful. For example, never allow kernel memory read could be
enforced from BPF side directly. Never allow pkt redirect, etc.

Quoted 7 lines

certainly cover a lot of cases, but not all. A strictly pre-defined list of what is allowed to run and what is not is what we need for our
case, so that we always know exactly what is going to run and what is
not, and can deal with the consequences accordingly, without nasty surprises waiting around the corner. Now in my naive view the best

Quoted 8 lines

way
to achieve this is via signatures and certs, as it's a well- understood
system, with processes already in place to revoke/rotate/etc, and it's
already used for kmods. An alternative would be hard-coding hashes I
guess, but that would be terribly inflexible.

Quoted 8 lines

Another option would be to load your programs at boot time, presumably
with trusted boot enabled and then lock down BPF completely. Then ensure all your BPF 'programs' are read-only from user<->kernel interface and this should start looking fairly close to what you want and all programs are correct by root of trust back to trusted boot. Would assume you know what programs to load at boot though. May or may not be a big assumption depending on your env.

One of the use cases we have for BPF is on-demand diagnostics, so loading at boot and blocking afterwards would not work, I think. Environment is constrained in terms of resources, so don't want to load anything that is not needed.

Quoted 51 lines
Quoted 20 lines

Now in terms of _how_ the signatures are done and validated, I'm sure
there are multiple ways, and if some are better than what this series
implements, then that's not an issue, it can be reworked. But the core
requirement for us is: offline pre-defined list of what is allowed to
run and what is not, with ability for hard enforcement that cannot be
bypassed. Yes, you lose some features like JIT and so on: we don't care, we don't need those for our use cases. If others have different
needs that's fine, this is all intended to be optional, not mandatory.
There are obviously trade-offs, as always when security is involved,
and each user can decide what's best for them.

Quoted 1 line

Hope this makes sense. Thanks!

I think I understand your use case. When done as BPF helper you can get the behavior you want with a one line BPF program loaded at boot.

int verify_all(struct bpf_prog **prog) {
        return verify_signature(prog->insn,
                                prog->len * sizeof(struct bpf_insn), signature, KEYRING, BPF_SIGTYPE); }

And I can write some more specific things as,

int verify_blobs(void data) {
  int reject = verify_signature(data, data_len, sig, KEYRING, TYPE); struct policy_key *key = map_get_key();

  return policy(key, reject);
}

map_get_key() looks into some datastor with the policy likely using 'current' to dig something up. It doesn't just apply to BPF progs we can use it on other executables more generally. And I get more interesting use cases like, allowing 'tc' programs unsigned, but requiring kernel memory reads to require signatures or any N other policies that may have value. Or only allowing my dbg user to run read-only programs, because the dbg maybe shouldn't ever be writing into packets, etc. Driving least privilege use cases in fine detail.

Quoted 8 lines

By making it a BPF program we side step the debate where the kernel tries to get the 'right' policy for you, me, everyone now and in the future. The only way I can see to do this without getting N policies baked into the kernel and at M different hook points is via a BPF helper.

Thanks,
John

Quoted 8 lines

Now this sounds like something that could work - we can prove that this could be loaded before any writable fs comes up anywhere, so in principle I think it would be acceptable and free of races. Matteo, we should talk about this tomorrow.
And this requires some infrastructure work right? Is there a WIP git tree somewhere that we can test out?

Thank you!

One question more question: with the signature + kconfig approach, nothing can disable the signature check. But if the signature checker is itself a bpf program, is there/can there be anything stopping root from unloading it?

Signature · 2 lines
Kind regards,
Luca Boccassi
#18 lore raw

[...]

Quoted 40 lines
Quoted 40 lines
Quoted 39 lines
Quoted 1 line

Hope this makes sense. Thanks!

I think I understand your use case. When done as BPF helper you can get the behavior you want with a one line BPF program loaded at boot.

int verify_all(struct bpf_prog **prog) {
        return verify_signature(prog->insn,
                                prog->len * sizeof(struct bpf_insn), signature, KEYRING, BPF_SIGTYPE); }

And I can write some more specific things as,

int verify_blobs(void data) {
  int reject = verify_signature(data, data_len, sig, KEYRING, TYPE); struct policy_key *key = map_get_key();

  return policy(key, reject);
}

map_get_key() looks into some datastor with the policy likely using 'current' to dig something up. It doesn't just apply to BPF progs we can use it on other executables more generally. And I get more interesting use cases like, allowing 'tc' programs unsigned, but requiring kernel memory reads to require signatures or any N other policies that may have value. Or only allowing my dbg user to run read-only programs, because the dbg maybe shouldn't ever be writing into packets, etc. Driving least privilege use cases in fine detail.

By making it a BPF program we side step the debate where the kernel tries to get the 'right' policy for you, me, everyone now and in the future. The only way I can see to do this without getting N policies baked into the kernel and at M different hook points is via a BPF helper.

Thanks,
John

Quoted 9 lines
Quoted 8 lines

Now this sounds like something that could work - we can prove that this could be loaded before any writable fs comes up anywhere, so in principle I think it would be acceptable and free of races. Matteo, we should talk about this tomorrow.
And this requires some infrastructure work right? Is there a WIP git tree somewhere that we can test out?

Thank you!

I don't have a WIP tree, but I believe it should be fairly easy. First I would add a wrapper BPF helper for verify_signature() so we can call it from fentry/freturn context. That can be done on its own IMO as its a generally useful operation.

Then I would stub a hook point into the BPF load path. The exact place to put this is going to have some debate I think, but I would place it immediately after the check_bpf call.

With above two you have enough to do sig verification iiuc.

Early boot loading I would have to check its current status. But I know folks have been working on it. Maybe its done?

Quoted 4 lines

One question more question: with the signature + kconfig approach, nothing can disable the signature check. But if the signature checker is itself a bpf program, is there/can there be anything stopping root from unloading it?

Interesting. Not that I'm aware of. Currently something with sufficient privileges could unload the program. Maybe we should have a flag so early boot programs can signal they shouldn't be unloaded ever. I would be OK with this and also seems generally useful. I have a case where I want to always set the socket cookie and we leave it running all the time. It would be nice if it came up and was pinned at boot.

Maybe slightly better than a flag would be to have a new CAP support that only early boot has like CAP_BPF_EARLY. From my point of view this both seems doable with just some smallish changes on BPF side.

Thanks,
John

#19 lore raw
Quoted 78 lines · John Fastabend

[...]

Quoted 40 lines
Quoted 40 lines
Quoted 39 lines
Quoted 1 line

Hope this makes sense. Thanks!

I think I understand your use case. When done as BPF helper you can get the behavior you want with a one line BPF program loaded at boot.

int verify_all(struct bpf_prog **prog) {
        return verify_signature(prog->insn,
                                prog->len * sizeof(struct bpf_insn), signature, KEYRING, BPF_SIGTYPE); }

And I can write some more specific things as,

int verify_blobs(void data) {
  int reject = verify_signature(data, data_len, sig, KEYRING, TYPE); struct policy_key *key = map_get_key();

  return policy(key, reject);
}

map_get_key() looks into some datastor with the policy likely using 'current' to dig something up. It doesn't just apply to BPF progs we can use it on other executables more generally. And I get more interesting use cases like, allowing 'tc' programs unsigned, but requiring kernel memory reads to require signatures or any N other policies that may have value. Or only allowing my dbg user to run read-only programs, because the dbg maybe shouldn't ever be writing into packets, etc. Driving least privilege use cases in fine detail.

By making it a BPF program we side step the debate where the kernel tries to get the 'right' policy for you, me, everyone now and in the future. The only way I can see to do this without getting N policies baked into the kernel and at M different hook points is via a BPF helper.

Thanks,
John

Quoted 9 lines
Quoted 8 lines

Now this sounds like something that could work - we can prove that this could be loaded before any writable fs comes up anywhere, so in principle I think it would be acceptable and free of races. Matteo, we should talk about this tomorrow.
And this requires some infrastructure work right? Is there a WIP git tree somewhere that we can test out?

Thank you!

I don't have a WIP tree, but I believe it should be fairly easy. First I would add a wrapper BPF helper for verify_signature() so we can call it from fentry/freturn context. That can be done on its own IMO as its a generally useful operation.

Then I would stub a hook point into the BPF load path. The exact place to put this is going to have some debate I think, but I would place it immediately after the check_bpf call.

With above two you have enough to do sig verification iiuc.

Early boot loading I would have to check its current status. But I know folks have been working on it. Maybe its done?

Quoted 4 lines

One question more question: with the signature + kconfig approach, nothing can disable the signature check. But if the signature checker is itself a bpf program, is there/can there be anything stopping root from unloading it?

Interesting. Not that I'm aware of. Currently something with sufficient privileges could unload the program. Maybe we should have a flag so early boot programs can signal they shouldn't be unloaded ever. I would be OK with this and also seems generally useful. I have a case where I want to always set the socket cookie and we leave it running all the time. It would be nice if it came up and was pinned at boot.

Quoted 6 lines

Maybe slightly better than a flag would be to have a new CAP support that only early boot has like CAP_BPF_EARLY. From my point of view this both seems doable with just some smallish changes on BPF side.

Thanks,
John

Thanks - again the means of enforcing this are not too important for our use case, as long as there is something that works reliably and can be attested.

Signature · 2 lines
Kind regards,
Luca Boccassi