Reduce overhead of LSMs with static calls
Subsystems (4)
KP Singh
maintainerv13 cover Reduce overhead of LSMs with static calls
# Background
LSM hooks (callbacks) are currently invoked as indirect function calls. These callbacks are registered into a linked list at boot time as the order of the LSMs can be configured on the kernel command line with the "lsm=" command line parameter.
Indirect function calls have a high overhead due to retpoline mitigation for various speculative execution attacks.
Retpolines remain relevant even with newer generation CPUs as recently discovered speculative attacks, like Spectre BHB need Retpolines to mitigate against branch history injection and still need to be used in combination with newer mitigation features like eIBRS.
This overhead is especially significant for the "bpf" LSM which allows the user to implement LSM functionality with eBPF program. In order to facilitate this the "bpf" LSM provides a default callback for all LSM hooks. When enabled, the "bpf" LSM incurs an unnecessary / avoidable indirect call. This is especially bad in OS hot paths (e.g. in the networking stack). This overhead prevents the adoption of bpf LSM on performance critical systems, and also, in general, slows down all LSMs.
Since we know the address of the enabled LSM callbacks at compile time and only the order is determined at boot time, the LSM framework can allocate static calls for each of the possible LSM callbacks and these calls can be updated once the order is determined at boot.
This series is a respin of the RFC proposed by Paul Renauld ([email protected]) and Brendan Jackman ([email protected]) [1]
# Performance improvement
With this patch-set some syscalls with lots of LSM hooks in their path benefitted at an average of ~3% and I/O and Pipe based system calls benefitting the most.
Here are the results of the relevant Unixbench system benchmarks with BPF LSM and SELinux enabled with default policies enabled with and without these patches.
Benchmark Delta(%): (+ is better)
===============================================================================
Execl Throughput +1.9356 File Write 1024 bufsize 2000 maxblocks +6.5953 Pipe Throughput +9.5499 Pipe-based Context Switching +3.0209 Process Creation +2.3246 Shell Scripts (1 concurrent) +1.4975 System Call Overhead +2.7815 System Benchmarks Index Score (Partial Only): +3.4859
In the best case, some syscalls like eventfd_create benefitted to about ~10%. The full analysis can be viewed at https://kpsingh.ch/lsm-perf
[1] https://lore.kernel.org/linux-security-module/[email protected]/
# BPF LSM Side effects
Patch 4 of the series also addresses the issues with the side effects of the default value return values of the BPF LSM callbacks and also removes the overheads associated with them making it deployable at hyperscale.
# v12 to v13
* Fixed the GCC note when struct randomization is enabled
* Added missing link to BPF LSM side effects discussion in the commit message
# v11 to v12
* Casey's feedback on readability of patch 4
* Tetsuo's catch on the bug in Patch 5
* Changed LSM_HOOK_TOGGLEABLE to LSM_HOOK_RUNTIME, the bikeshed is blue now.
* Also, as security_toggle_hook relies on iterating over a
flat lsm_static_calls_table, I added the __packed attribute.
# v10 to v11
* bpf_lsm_toggle_hook to security_toggle_hook with LSM_HOOK_TOGGLEABLE limiting the hooks as the ones that can be toggled.
# v9 to v10
* Addressed Paul's comments for Patch 3. I did not remove the acks from this one as changes were minor.
* Moved BPF LSM specific hook toggling logic bpf_lsm_toggle_hook to s security_toggle_hook as a generic API. I removed the Ack's from this patch as it's worth another look.
* Refactored the non-standard hooks to use static calls.
# v8 to v9
Paul, I removed the 5th patch about CONFIG_SECURITY_HOOK_LIKELY and went through all the feedback. I believe it all should be addressed now.
But, please let me know if I missed anything.
The patches are based on https://git.kernel.org/pub/scm/linux/kernel/git/pcmoore/lsm.git
(next branch as of 2024-02-07) and resolved a bunch of conflicts.
I also added Andrii's series ack to indidividual patches.
# v7 to v8
* Addressed Andrii's feedback
* Rebased (this seems to have removed the syscall changes). v7 has the required conflict resolution incase the conflicts need to be resolved again.
# v6 -> v7
* Rebased with latest LSM id changes merged
NOTE: The warning shown by the kernel test bot is spurious, there is no flex array and it seems to come from an older tool chain.
https://lore.kernel.org/bpf/[email protected]/
# v5 -> v6
* Fix a bug in BPF LSM hook toggle logic.
# v4 -> v5
* Rebase to linux-next/master
* Fixed the case where MAX_LSM_COUNT comes to zero when just CONFIG_SECURITY is compiled in without any other LSM enabled as reported here:
https://lore.kernel.org/bpf/[email protected]
# v3 -> v4
* Refactor LSM count macros to use COUNT_ARGS
* Change CONFIG_SECURITY_HOOK_LIKELY likely's default value to be based on the LSM enabled and have it depend on CONFIG_EXPERT. There are a lot of subtle options behind CONFIG_EXPERT and this should, hopefully alleviate concerns about yet another knob.
* __randomize_layout for struct lsm_static_call and, in addition to the cover letter add performance numbers to 3rd patch and some minor commit message updates.
* Rebase to linux-next.
# v2 -> v3
* Fixed a build issue on archs which don't have static calls and enable CONFIG_SECURITY.
* Updated the LSM_COUNT macros based on Andrii's suggestions.
* Changed the security_ prefix to lsm_prefix based on Casey's suggestion.
* Inlined static_branch_maybe into lsm_for_each_hook on Kees' feedback.
# v1 -> v2 (based on linux-next, next-20230614)
* Incorporated suggestions from Kees
* Changed the way MAX_LSMs are counted from a binary based generator to a clever header.
* Add CONFIG_SECURITY_HOOK_LIKELY to configure the likelihood of LSM hooks.
KP Singh (5):
kernel: Add helper macros for loop unrolling
security: Count the LSMs enabled at compile time
security: Replace indirect LSM hook calls with static calls
security: Update non standard hooks to use static calls
bpf: Only enable BPF LSM hooks when an LSM program is attached
include/linux/args.h | 6 +- include/linux/lsm_count.h | 128 ++++++++++++ include/linux/lsm_hooks.h | 100 +++++++++- include/linux/unroll.h | 36 ++++ kernel/bpf/trampoline.c | 40 +++- security/bpf/hooks.c | 2 +- security/security.c | 404 ++++++++++++++++++++++++++------------ 7 files changed, 569 insertions(+), 147 deletions(-) create mode 100644 include/linux/lsm_count.h create mode 100644 include/linux/unroll.h
Signature · 1 line
2.45.2.803.g4e1b14247a-goog
KP Singh
maintainerPATCH v13 1/5 kernel: Add helper macros for loop unrolling
This helps in easily initializing blocks of code (e.g. static calls and keys).
UNROLL(N, MACRO, __VA_ARGS__) calls MACRO N times with the first argument as the index of the iteration. This allows string pasting to create unique tokens for variable names, function calls etc.
As an example:
#include <linux/unroll.h>
#define MACRO(N, a, b) \
int add_##N(int a, int b) \
{ \
return a + b + N; \
}
UNROLL(2, MACRO, x, y)expands to:
int add_0(int x, int y)
{
return x + y + 0;
}
int add_1(int x, int y)
{
return x + y + 1;
}--- include/linux/unroll.h | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 include/linux/unroll.h
include/linux/unroll.h new file +36 -0
@@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* +* Copyright (C) 2023 Google LLC. +*/ + +#ifndef __UNROLL_H +#define __UNROLL_H + +#include <linux/args.h> + +#define UNROLL(N, MACRO, args...) CONCATENATE(__UNROLL_, N)(MACRO, args) + +#define __UNROLL_0(MACRO, args...) +#define __UNROLL_1(MACRO, args...) __UNROLL_0(MACRO, args) MACRO(0, args) +#define __UNROLL_2(MACRO, args...) __UNROLL_1(MACRO, args) MACRO(1, args) +#define __UNROLL_3(MACRO, args...) __UNROLL_2(MACRO, args) MACRO(2, args) +#define __UNROLL_4(MACRO, args...) __UNROLL_3(MACRO, args) MACRO(3, args) +#define __UNROLL_5(MACRO, args...) __UNROLL_4(MACRO, args) MACRO(4, args) +#define __UNROLL_6(MACRO, args...) __UNROLL_5(MACRO, args) MACRO(5, args) +#define __UNROLL_7(MACRO, args...) __UNROLL_6(MACRO, args) MACRO(6, args) +#define __UNROLL_8(MACRO, args...) __UNROLL_7(MACRO, args) MACRO(7, args) +#define __UNROLL_9(MACRO, args...) __UNROLL_8(MACRO, args) MACRO(8, args) +#define __UNROLL_10(MACRO, args...) __UNROLL_9(MACRO, args) MACRO(9, args) +#define __UNROLL_11(MACRO, args...) __UNROLL_10(MACRO, args) MACRO(10, args) +#define __UNROLL_12(MACRO, args...) __UNROLL_11(MACRO, args) MACRO(11, args) +#define __UNROLL_13(MACRO, args...) __UNROLL_12(MACRO, args) MACRO(12, args) +#define __UNROLL_14(MACRO, args...) __UNROLL_13(MACRO, args) MACRO(13, args) +#define __UNROLL_15(MACRO, args...) __UNROLL_14(MACRO, args) MACRO(14, args) +#define __UNROLL_16(MACRO, args...) __UNROLL_15(MACRO, args) MACRO(15, args) +#define __UNROLL_17(MACRO, args...) __UNROLL_16(MACRO, args) MACRO(16, args) +#define __UNROLL_18(MACRO, args...) __UNROLL_17(MACRO, args) MACRO(17, args) +#define __UNROLL_19(MACRO, args...) __UNROLL_18(MACRO, args) MACRO(18, args) +#define __UNROLL_20(MACRO, args...) __UNROLL_19(MACRO, args) MACRO(19, args) + +#endif /* __UNROLL_H */
Signature · 1 line
2.45.2.803.g4e1b14247a-goog
KP Singh
maintainerPATCH v13 2/5 security: Count the LSMs enabled at compile time
In reply to KP SinghThese macros are a clever trick to determine a count of the number of LSMs that are enabled in the config to ascertain the maximum number of static calls that need to be configured per LSM hook.
Without this one would need to generate static calls for the total number of LSMs in the kernel (even if they are not compiled) times the number of LSM hooks which ends up being quite wasteful.
--- include/linux/args.h | 6 +- include/linux/lsm_count.h | 128 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 131 insertions(+), 3 deletions(-) create mode 100644 include/linux/lsm_count.h
include/linux/args.h +3 -3
@@ -17,9 +17,9 @@* that as _n. */ -/* This counts to 12. Any more, it will return 13th argument. */ -#define __COUNT_ARGS(_0, _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, _11, _12, _n, X...) _n -#define COUNT_ARGS(X...) __COUNT_ARGS(, ##X, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0) +/* This counts to 15. Any more, it will return 16th argument. */ +#define __COUNT_ARGS(_0, _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, _11, _12, _13, _14, _15, _n, X...) _n +#define COUNT_ARGS(X...) __COUNT_ARGS(, ##X, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0)/* Concatenate two parameters, but allow them to be expanded beforehand. */ #define __CONCAT (a, b) a ## b
include/linux/lsm_count.h new file +128 -0
128 lines not on this page · raw message
Signature · 1 line
2.45.2.803.g4e1b14247a-goog
KP Singh
maintainerPATCH v13 3/5 security: Replace indirect LSM hook calls with static calls
In reply to KP SinghLSM hooks are currently invoked from a linked list as indirect calls which are invoked using retpolines as a mitigation for speculative attacks (Branch History / Target injection) and add extra overhead which is especially bad in kernel hot paths:
security_file_ioctl:
0xff...0320 <+0>: endbr64
0xff...0324 <+4>: push %rbp 0xff...0325 <+5>: push %r15 0xff...0327 <+7>: push %r14 0xff...0329 <+9>: push %rbx 0xff...032a <+10>: mov %rdx,%rbx 0xff...032d <+13>: mov %esi,%ebp 0xff...032f <+15>: mov %rdi,%r14 0xff...0332 <+18>: mov $0xff...7030,%r15 0xff...0339 <+25>: mov (%r15),%r15 0xff...033c <+28>: test %r15,%r15 0xff...033f <+31>: je 0xff...0358 <security_file_ioctl+56> 0xff...0341 <+33>: mov 0x18(%r15),%r11 0xff...0345 <+37>: mov %r14,%rdi 0xff...0348 <+40>: mov %ebp,%esi 0xff...034a <+42>: mov %rbx,%rdx
0xff...034d <+45>: call 0xff...2e0 <__x86_indirect_thunk_array+352>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Indirect calls that use retpolines leading to overhead, not just due to extra instruction but also branch misses.
0xff...0352 <+50>: test %eax,%eax 0xff...0354 <+52>: je 0xff...0339 <security_file_ioctl+25> 0xff...0356 <+54>: jmp 0xff...035a <security_file_ioctl+58> 0xff...0358 <+56>: xor %eax,%eax 0xff...035a <+58>: pop %rbx 0xff...035b <+59>: pop %r14 0xff...035d <+61>: pop %r15 0xff...035f <+63>: pop %rbp 0xff...0360 <+64>: jmp 0xff...47c4 <__x86_return_thunk>
The indirect calls are not really needed as one knows the addresses of enabled LSM callbacks at boot time and only the order can possibly change at boot time with the lsm= kernel command line parameter.
An array of static calls is defined per LSM hook and the static calls are updated at boot time once the order has been determined.
A static key guards whether an LSM static call is enabled or not, without this static key, for LSM hooks that return an int, the presence of the hook that returns a default value can create side-effects which has resulted in bugs [1].
With the hook now exposed as a static call, one can see that the retpolines are no longer there and the LSM callbacks are invoked directly:
security_file_ioctl:
0xff...0ca0 <+0>: endbr64
0xff...0ca4 <+4>: nopl 0x0(%rax,%rax,1)
0xff...0ca9 <+9>: push %rbp
0xff...0caa <+10>: push %r14 0xff...0cac <+12>: push %rbx 0xff...0cad <+13>: mov %rdx,%rbx 0xff...0cb0 <+16>: mov %esi,%ebp 0xff...0cb2 <+18>: mov %rdi,%r14 0xff...0cb5 <+21>: jmp 0xff...0cc7 <security_file_ioctl+39>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Static key enabled for SELinux
0xffffffff818f0cb7 <+23>: jmp 0xff...0cde <security_file_ioctl+62>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Static key enabled for BPF LSM. This is something that is changed to default to false to avoid the existing side effect issues of BPF LSM
[1] in a subsequent patch.
0xff...0cb9 <+25>: xor %eax,%eax 0xff...0cbb <+27>: xchg %ax,%ax 0xff...0cbd <+29>: pop %rbx 0xff...0cbe <+30>: pop %r14 0xff...0cc0 <+32>: pop %rbp 0xff...0cc1 <+33>: cs jmp 0xff...0000 <__x86_return_thunk> 0xff...0cc7 <+39>: endbr64 0xff...0ccb <+43>: mov %r14,%rdi 0xff...0cce <+46>: mov %ebp,%esi 0xff...0cd0 <+48>: mov %rbx,%rdx 0xff...0cd3 <+51>: call 0xff...3230 <selinux_file_ioctl>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Direct call to SELinux.
0xff...0cd8 <+56>: test %eax,%eax 0xff...0cda <+58>: jne 0xff...0cbd <security_file_ioctl+29> 0xff...0cdc <+60>: jmp 0xff...0cb7 <security_file_ioctl+23> 0xff...0cde <+62>: endbr64 0xff...0ce2 <+66>: mov %r14,%rdi 0xff...0ce5 <+69>: mov %ebp,%esi 0xff...0ce7 <+71>: mov %rbx,%rdx 0xff...0cea <+74>: call 0xff...e220 <bpf_lsm_file_ioctl>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Direct call to BPF LSM.
0xff...0cef <+79>: test %eax,%eax 0xff...0cf1 <+81>: jne 0xff...0cbd <security_file_ioctl+29> 0xff...0cf3 <+83>: jmp 0xff...0cb9 <security_file_ioctl+25> 0xff...0cf5 <+85>: endbr64 0xff...0cf9 <+89>: mov %r14,%rdi 0xff...0cfc <+92>: mov %ebp,%esi 0xff...0cfe <+94>: mov %rbx,%rdx 0xff...0d01 <+97>: pop %rbx 0xff...0d02 <+98>: pop %r14
0xff...0d04 <+100>: pop %rbp
0xff...0d05 <+101>: ret
0xff...0d06 <+102>: int3
0xff...0d07 <+103>: int3
0xff...0d08 <+104>: int3
0xff...0d09 <+105>: int3
While this patch uses static_branch_unlikely indicating that an LSM hook is likely to be not present. In most cases this is still a better choice as even when an LSM with one hook is added, empty slots are created for all LSM hooks (especially when many LSMs that do not initialize most hooks are present on the system).
There are some hooks that don't use the call_int_hook and call_void_hook. These hooks are updated to use a new macro called lsm_for_each_hook where the lsm_callback is directly invoked as an indirect call. These are updated in a subsequent patch to also use static calls.
Below are results of the relevant Unixbench system benchmarks with BPF LSM and SELinux enabled with default policies enabled with and without these patches.
Benchmark Delta(%): (+ is better)
===============================================================================
Execl Throughput +1.9356 File Write 1024 bufsize 2000 maxblocks +6.5953 Pipe Throughput +9.5499 Pipe-based Context Switching +3.0209 Process Creation +2.3246 Shell Scripts (1 concurrent) +1.4975 System Call Overhead +2.7815 System Benchmarks Index Score (Partial Only): +3.4859
In the best case, some syscalls like eventfd_create benefitted to about ~10%.
[1] https://lore.kernel.org/linux-security-module/[email protected]/
--- include/linux/lsm_hooks.h | 72 ++++++++++++-- security/security.c | 198 ++++++++++++++++++++++++++------------ 2 files changed, 197 insertions(+), 73 deletions(-)
include/linux/lsm_hooks.h +63 -9
@@ -30,19 +30,66 @@#include <linux/init.h> #include <linux/rculist.h> #include <linux/xattr.h> +#include <linux/static_call.h> +#include <linux/unroll.h> +#include <linux/jump_label.h> +#include <linux/lsm_count.h> + +#define SECURITY_HOOK_ACTIVE_KEY (HOOK ,IDX ) security_hook_active_##HOOK##_##IDX + +/* +* Identifier for the LSM static calls. +* HOOK is an LSM hook as defined in linux/lsm_hookdefs.h +* IDX is the index of the static call. 0 <= NUM < MAX_LSM_COUNT +*/ +#define LSM_STATIC_CALL (HOOK ,IDX ) lsm_static_call_##HOOK##_##IDX + +/* +* Call the macro M for each LSM hook MAX_LSM_COUNT times. +*/ +#define LSM_LOOP_UNROLL (M , ...) \ +do { \ + UNROLL(MAX_LSM_COUNT, M, __VA_ARGS__) \ +} while (0) + +#define LSM_DEFINE_UNROLL (M , ...) UNROLL(MAX_LSM_COUNT, M, __VA_ARGS__)union security_list_options {#define LSM_HOOK (RET ,DEFAULT ,NAME , ...) RET (*NAME)(__VA_ARGS__);#include "lsm_hook_defs.h"#undef LSM_HOOK +void *lsm_func_addr; }; -struct security_hook_heads { -#define LSM_HOOK (RET ,DEFAULT ,NAME , ...) struct hlist_head NAME; -#include "lsm_hook_defs.h" -#undef LSM_HOOK +/* +* @key: static call key as defined by STATIC_CALL_KEY +* @trampoline: static call trampoline as defined by STATIC_CALL_TRAMP +* @hl: The security_hook_list as initialized by the owning LSM. +* @active: Enabled when the static call has an LSM hook associated. +*/ +struct lsm_static_call { +struct static_call_key *key; +void *trampoline; +struct security_hook_list *hl; +/* this needs to be true or false based on what the key defaults to */ +struct static_key_false *active; } __randomize_layout; +/* +* Table of the static calls for each LSM hook. +* Once the LSMs are initialized, their callbacks will be copied to these +* tables such that the calls are filled backwards (from last to first). +* This way, we can jump directly to the first used static call, and execute +* all of them after. This essentially makes the entry point +* dynamic to adapt the number of static calls to the number of callbacks. +*/ +struct lsm_static_calls_table { +#define LSM_HOOK (RET ,DEFAULT ,NAME , ...) \ + struct lsm_static_call NAME[MAX_LSM_COUNT]; +#include <linux/lsm_hook_defs.h> +#undef LSM_HOOK +} __packed __randomize_layout; +/** * struct lsm_id - Identify a Linux Security Module. * @lsm: name of the LSM, must be approved by the LSM maintainers
@@ -58,10 +105,14 @@ struct lsm_id {/* * Security module hook list structure. * For use with generic list macros for common operations. +* +* struct security_hook_list - Contents of a cacheable, mappable object. +* @scalls: The beginning of the array of static calls assigned to this hook. +* @hook: The callback for the hook. +* @lsm: The name of the lsm that owns this hook. */ struct security_hook_list { -struct hlist_node list; -struct hlist_head *head; +struct lsm_static_call *scalls;union security_list_options hook;const struct lsm_id *lsmid; } __randomize_layout;
@@ -111,10 +162,12 @@ static inline struct xattr *lsm_get_xattr_slot(struct xattr *xattrs,* care of the common case and reduces the amount of * text involved. */ -#define LSM_HOOK_INIT (HEAD ,HOOK ) \ - { .head = &security_hook_heads.HEAD, .hook = { .HEAD = HOOK } } +#define LSM_HOOK_INIT (NAME ,HOOK ) \ + { \ + .scalls = static_calls_table.NAME, \ + .hook = { .NAME = HOOK } \ + } -extern struct security_hook_heads security_hook_heads;extern char *lsm_names; extern void security_add_hooks(struct security_hook_list *hooks, int count,
@@ -152,5 +205,6 @@ extern struct lsm_info __start_early_lsm_info[], __end_early_lsm_info[];__aligned(sizeof(unsigned long)) extern int lsm_inode_alloc (struct inode *inode); +extern struct lsm_static_calls_table static_calls_table__ro_after_init ;#endif /* ! __LINUX_LSM_HOOKS_H */
security/security.c +134 -64
355 lines not on this page · raw message
Signature · 1 line
2.45.2.803.g4e1b14247a-goog
KP Singh
maintainerPATCH v13 4/5 security: Update non standard hooks to use static calls
In reply to KP SinghThere are some LSM hooks which do not use the common pattern followed by other LSM hooks and thus cannot use call_{int, void}_hook macros and instead use lsm_for_each_hook macro which still results in indirect call.
There is one additional generalizable pattern where a hook matching an lsmid is called and the indirect calls for these are addressed with the newly added call_hook_with_lsmid macro which internally uses an implementation similar to call_int_hook but has an additional check that matches the lsmid.
For the generic case the lsm_for_each_hook macro is updated to accept logic before and after the invocation of the LSM hook (static call) in the unrolled loop.
--- security/security.c | 248 +++++++++++++++++++++++++------------------- 1 file changed, 144 insertions(+), 104 deletions(-)
security/security.c +144 -104
@@ -948,10 +948,48 @@ out: \RC ; \ }) -#define lsm_for_each_hook (scall,NAME ) \ - for (scall = static_calls_table.NAME; \ - scall - static_calls_table.NAME < MAX_LSM_COUNT; scall++) \ - if (static_key_enabled(&scall->active->key)) +/* +* Can be used in the context passed to lsm_for_each_hook to get the lsmid of the +* current hook +*/ +#define current_lsmid () _hook_lsmid + +#define __CALL_HOOK (NUM ,HOOK ,RC ,BLOCK_BEFORE ,BLOCK_AFTER , ...) \ +do { \ + int __maybe_unused _hook_lsmid; \ + \ + if (static_branch_unlikely(&SECURITY_HOOK_ACTIVE_KEY(HOOK, NUM))) { \ + _hook_lsmid = static_calls_table.HOOK[NUM].hl->lsmid->id; \ + BLOCK_BEFORE \ + RC = static_call(LSM_STATIC_CALL(HOOK, NUM))(__VA_ARGS__); \ + BLOCK_AFTER \ + } \ +} while (0); + +#define lsm_for_each_hook (HOOK ,RC ,BLOCK_AFTER , ...) \ + LSM_LOOP_UNROLL(__CALL_HOOK, HOOK, RC, ;, BLOCK_AFTER, __VA_ARGS__) + +#define call_hook_with_lsmid (HOOK ,LSMID , ...) \ +({ \ + __label__ out; \ + int RC = LSM_RET_DEFAULT(HOOK); \ + \ + LSM_LOOP_UNROLL(__CALL_HOOK, HOOK, RC, \ +/* BLOCK BEFORE INVOCATION */ \ + { \ +if (current_lsmid () !=LSMID ) \ +continue ; \ + }, \ +/* END BLOCK BEFORE INVOCATION */ \ +/* BLOCK AFTER INVOCATION */ \ + { \ + goto out; \ + }, \ +/* END BLOCK AFTER INVOCATION */ \ + __VA_ARGS__); \ +out: \ +RC ; \ +})/* Security operations */
@@ -1187,7 +1225,6 @@ int security_settime64(const struct timespec64 *ts, const struct timezone *tz)*/ int security_vm_enough_memory_mm (struct mm_struct *mm,long pages) { -struct lsm_static_call *scall;int cap_sys_admin =1 ;int rc;
@@ -1198,13 +1235,20 @@ int security_vm_enough_memory_mm(struct mm_struct *mm, long pages)* agree that it should be set it will. If any module * thinks it should not be set it won't. */ -lsm_for_each_hook (scall, vm_enough_memory) { - rc = scall->hl->hook.vm_enough_memory (mm, pages); -if (rc <=0 ) { - cap_sys_admin =0 ; -break ; - } - } + +lsm_for_each_hook ( + vm_enough_memory, rc, +/* BLOCK AFTER INVOCATION */ + { + if (rc <= 0) { + cap_sys_admin =0 ; + goto out; + } + }, +/* END BLOCK AFTER INVOCATION */ + mm, pages); + +out:return __vm_enough_memory (mm, pages, cap_sys_admin); }
@@ -1346,17 +1390,21 @@ int security_fs_context_dup(struct fs_context *fc, struct fs_context *src_fc)int security_fs_context_parse_param (struct fs_context *fc,struct fs_parameter *param) { -struct lsm_static_call *scall; -int trc; +int trc =LSM_RET_DEFAULT (fs_context_parse_param);int rc = -ENOPARAM ; -lsm_for_each_hook (scall, fs_context_parse_param) { - trc = scall->hl->hook.fs_context_parse_param (fc, param); -if (trc ==0 ) - rc =0 ; -else if (trc != -ENOPARAM ) -return trc; - } +lsm_for_each_hook ( + fs_context_parse_param, trc, +/* BLOCK AFTER INVOCATION */ + { + if (trc == 0) + rc = 0; + else if (trc != -ENOPARAM) + return trc; + }, +/* END BLOCK AFTER INVOCATION */ + fc, param); +return rc; }
@@ -1581,15 +1629,19 @@ int security_sb_set_mnt_opts(struct super_block *sb,unsigned long kern_flags, unsigned long *set_kern_flags) { - struct lsm_static_call *scall; int rc = mnt_opts ? -EOPNOTSUPP : LSM_RET_DEFAULT(sb_set_mnt_opts); -lsm_for_each_hook (scall, sb_set_mnt_opts) { - rc = scall->hl->hook.sb_set_mnt_opts (sb, mnt_opts, kern_flags, - set_kern_flags); -if (rc !=LSM_RET_DEFAULT (sb_set_mnt_opts)) -break ; - } +lsm_for_each_hook ( + sb_set_mnt_opts, rc, +/* BLOCK AFTER INVOCATION */ + { + if (rc != LSM_RET_DEFAULT(sb_set_mnt_opts)) + goto out; + }, +/* END BLOCK AFTER INVOCATION */ + sb, mnt_opts, kern_flags, set_kern_flags); + +out:return rc; }EXPORT_SYMBOL (security_sb_set_mnt_opts);
@@ -1780,7 +1832,6 @@ int security_inode_init_security(struct inode *inode, struct inode *dir,const struct qstr *qstr, const initxattrs initxattrs, void *fs_data) { - struct lsm_static_call *scall; struct xattr *new_xattrs = NULL;int ret = -EOPNOTSUPP , xattr_count =0 ;
@@ -1798,18 +1849,21 @@ int security_inode_init_security(struct inode *inode, struct inode *dir,return -ENOMEM ; } -lsm_for_each_hook (scall, inode_init_security) { - ret = scall->hl->hook.inode_init_security (inode, dir, qstr, new_xattrs, - &xattr_count); -if (ret && ret != -EOPNOTSUPP ) - goto out; +lsm_for_each_hook ( + inode_init_security, ret, +/* BLOCK AFTER INVOCATION */ + { /* * As documented in lsm_hooks.h, -EOPNOTSUPP in this context * means that the LSM is not willing to provide an xattr, not * that it wants to signal an error. Thus, continue to invoke * the remaining LSMs. */ - } + if (ret && ret != -EOPNOTSUPP) + goto out; + }, +/* END BLOCK AFTER INVOCATION */ + inode, dir, qstr, new_xattrs, &xattr_count);/* If initxattrs() is NULL, xattr_count is zero, skip the call. */ if (!xattr_count)
@@ -3631,16 +3685,21 @@ int security_task_prctl(int option, unsigned long arg2, unsigned long arg3, {int thisrc;int rc =LSM_RET_DEFAULT (task_prctl); -struct lsm_static_call *scall; - -lsm_for_each_hook (scall, task_prctl) { - thisrc = scall->hl->hook.task_prctl (option, arg2, arg3, arg4, arg5); -if (thisrc !=LSM_RET_DEFAULT (task_prctl)) { - rc = thisrc; -if (thisrc !=0 ) -break ; - } - } + +lsm_for_each_hook ( + task_prctl, thisrc, +/* BLOCK AFTER INVOCATION */ + { + if (thisrc != LSM_RET_DEFAULT(task_prctl)) { + rc = thisrc; +if (thisrc !=0 ) + goto out; + } + }, +/* END BLOCK AFTER INVOCATION */ + option, arg2, arg3, arg4, arg5); + +out:return rc; }
@@ -4040,7 +4099,6 @@ EXPORT_SYMBOL(security_d_instantiate);int security_getselfattr (unsigned int attr,struct lsm_ctx __user *uctx,u32 __user *size,u32 flags) { -struct lsm_static_call *scall;struct lsm_ctx lctx = { .id =LSM_ID_UNDEF , };u8 __user *base = (u8 __user *)uctx;u32 entrysize;
@@ -4078,31 +4136,42 @@ int security_getselfattr(unsigned int attr, struct lsm_ctx __user *uctx,* In the usual case gather all the data from the LSMs. * In the single case only get the data from the LSM specified. */ -lsm_for_each_hook (scall, getselfattr) { -if (single && lctx.id != scall->hl->lsmid->id) -continue ; - entrysize = left; -if (base) - uctx = (struct lsm_ctx __user *)(base + total); - rc = scall->hl->hook.getselfattr (attr, uctx, &entrysize, flags); -if (rc == -EOPNOTSUPP ) { - rc =0 ; -continue ; - } -if (rc == -E2BIG ) { - rc =0 ; - left =0 ; - toobig = true; - }else if (rc <0 ) -return rc; -else - left -= entrysize; +LSM_LOOP_UNROLL ( + __CALL_HOOK, getselfattr, rc, +/* BLOCK BEFORE INVOCATION */ + { +if (single && lctx.id !=current_lsmid ()) +continue ; + entrysize = left; +if (base) + uctx = (struct lsm_ctx __user *)(base + total); + }, +/* END BLOCK BEFORE INVOCATION */ +/* BLOCK AFTER INVOCATION */ + { +if (rc == -EOPNOTSUPP ) { + rc =0 ; + }else { +if (rc == -E2BIG ) { + rc =0 ; + left =0 ; + toobig = true; + }else if (rc <0 ) +return rc; +else + left -= entrysize; + + total += entrysize; + count += rc; +if (single) + goto out; + } + }, +/* END BLOCK AFTER INVOCATION */ + attr, uctx, &entrysize, flags); + +out: - total += entrysize; - count += rc; -if (single) -break ; - }if (put_user (total, size))return -EFAULT ;if (toobig)
@@ -4133,9 +4202,8 @@ int security_getselfattr(unsigned int attr, struct lsm_ctx __user *uctx,int security_setselfattr (unsigned int attr,struct lsm_ctx __user *uctx,u32 size,u32 flags) { -struct lsm_static_call *scall;struct lsm_ctx *lctx; -int rc =LSM_RET_DEFAULT (setselfattr); +int rc;u64 required_len;if (flags)
@@ -4156,11 +4224,7 @@ int security_setselfattr(unsigned int attr, struct lsm_ctx __user *uctx, goto free_out; } -lsm_for_each_hook (scall, setselfattr) -if ((scall->hl->lsmid->id) == lctx->id) { - rc = scall->hl->hook.setselfattr (attr, lctx, size, flags); -break ; - } + rc =call_hook_with_lsmid (setselfattr, lctx->id, attr, lctx, size, flags); free_out:kfree (lctx);
@@ -4181,14 +4245,7 @@ int security_setselfattr(unsigned int attr, struct lsm_ctx __user *uctx,int security_getprocattr (struct task_struct *p,int lsmid,const char *name,char **value) { -struct lsm_static_call *scall; - -lsm_for_each_hook (scall, getprocattr) { -if (lsmid !=0 && lsmid != scall->hl->lsmid->id) -continue ; -return scall->hl->hook.getprocattr (p, name, value); - } -return LSM_RET_DEFAULT (getprocattr); +return call_hook_with_lsmid (getprocattr, lsmid, p, name, value); }/**
@@ -4205,14 +4262,7 @@ int security_getprocattr(struct task_struct *p, int lsmid, const char *name,*/ int security_setprocattr (int lsmid,const char *name,void *value,size_t size) { -struct lsm_static_call *scall; - -lsm_for_each_hook (scall, setprocattr) { -if (lsmid !=0 && lsmid != scall->hl->lsmid->id) -continue ; -return scall->hl->hook.setprocattr (name, value, size); - } -return LSM_RET_DEFAULT (setprocattr); +return call_hook_with_lsmid (setprocattr, lsmid, name, value, size); }/**
@@ -5328,23 +5378,13 @@ int security_xfrm_state_pol_flow_match(struct xfrm_state *x, struct xfrm_policy *xp, const struct flowi_common *flic) { - struct lsm_static_call *scall; -int rc =LSM_RET_DEFAULT (xfrm_state_pol_flow_match); -/* * Since this function is expected to return 0 or 1, the judgment * becomes difficult if multiple LSMs supply this call. Fortunately, * we can use the first LSM's judgment because currently only SELinux * supplies this call. -* -* For speed optimization, we explicitly break the loop rather than -* using the macro */ -lsm_for_each_hook (scall, xfrm_state_pol_flow_match) { - rc = scall->hl->hook.xfrm_state_pol_flow_match (x, xp, flic); -break ; - } -return rc; +return call_int_hook (xfrm_state_pol_flow_match, x, xp, flic); }/**
Signature · 1 line
2.45.2.803.g4e1b14247a-goog
KP Singh
maintainerPATCH v13 5/5 bpf: Only enable BPF LSM hooks when an LSM program is attached
In reply to KP SinghBPF LSM hooks have side-effects (even when a default value's returned) as some hooks end up behaving differently due to the very presence of the hook.
The static keys guarding the BPF LSM hooks are disabled by default and enabled only when a BPF program is attached implementing the hook logic. This avoids the issue of the side-effects and also the minor overhead associated with the empty callback.
security_file_ioctl:
0xff...0e30 <+0>: endbr64
0xff...0e34 <+4>: nopl 0x0(%rax,%rax,1)
0xff...0e39 <+9>: push %rbp
0xff...0e3a <+10>: push %r14 0xff...0e3c <+12>: push %rbx 0xff...0e3d <+13>: mov %rdx,%rbx 0xff...0e40 <+16>: mov %esi,%ebp 0xff...0e42 <+18>: mov %rdi,%r14 0xff...0e45 <+21>: jmp 0xff...0e57 <security_file_ioctl+39>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Static key enabled for SELinux
0xff...0e47 <+23>: xchg %ax,%ax
^^^^^^^^^^^^^^
Static key disabled for BPF. This gets patched when a BPF LSM program is attached
0xff...0e49 <+25>: xor %eax,%eax 0xff...0e4b <+27>: xchg %ax,%ax 0xff...0e4d <+29>: pop %rbx 0xff...0e4e <+30>: pop %r14 0xff...0e50 <+32>: pop %rbp 0xff...0e51 <+33>: cs jmp 0xff...0000 <__x86_return_thunk> 0xff...0e57 <+39>: endbr64 0xff...0e5b <+43>: mov %r14,%rdi 0xff...0e5e <+46>: mov %ebp,%esi 0xff...0e60 <+48>: mov %rbx,%rdx 0xff...0e63 <+51>: call 0xff...33c0 <selinux_file_ioctl> 0xff...0e68 <+56>: test %eax,%eax 0xff...0e6a <+58>: jne 0xff...0e4d <security_file_ioctl+29> 0xff...0e6c <+60>: jmp 0xff...0e47 <security_file_ioctl+23> 0xff...0e6e <+62>: endbr64 0xff...0e72 <+66>: mov %r14,%rdi 0xff...0e75 <+69>: mov %ebp,%esi 0xff...0e77 <+71>: mov %rbx,%rdx 0xff...0e7a <+74>: call 0xff...e3b0 <bpf_lsm_file_ioctl> 0xff...0e7f <+79>: test %eax,%eax 0xff...0e81 <+81>: jne 0xff...0e4d <security_file_ioctl+29> 0xff...0e83 <+83>: jmp 0xff...0e49 <security_file_ioctl+25> 0xff...0e85 <+85>: endbr64 0xff...0e89 <+89>: mov %r14,%rdi 0xff...0e8c <+92>: mov %ebp,%esi 0xff...0e8e <+94>: mov %rbx,%rdx 0xff...0e91 <+97>: pop %rbx 0xff...0e92 <+98>: pop %r14
0xff...0e94 <+100>: pop %rbp
0xff...0e95 <+101>: ret
This patch enables this by providing a LSM_HOOK_INIT_RUNTIME variant that allows the LSMs to opt-in to hooks which can be toggled at runtime which with security_toogle_hook.
--- include/linux/lsm_hooks.h | 30 ++++++++++++++++++++++++++++- kernel/bpf/trampoline.c | 40 +++++++++++++++++++++++++++++++++++---- security/bpf/hooks.c | 2 +- security/security.c | 36 ++++++++++++++++++++++++++++++++++- 4 files changed, 101 insertions(+), 7 deletions(-)
include/linux/lsm_hooks.h +29 -1
@@ -110,11 +110,14 @@ struct lsm_id {* @scalls: The beginning of the array of static calls assigned to this hook. * @hook: The callback for the hook. * @lsm: The name of the lsm that owns this hook. +* @default_state: The state of the LSM hook when initialized. If set to false, +* the static key guarding the hook will be set to disabled. */ struct security_hook_list {struct lsm_static_call *scalls;union security_list_options hook;const struct lsm_id *lsmid; +bool runtime; } __randomize_layout;/*
@@ -165,7 +168,19 @@ static inline struct xattr *lsm_get_xattr_slot(struct xattr *xattrs,#define LSM_HOOK_INIT (NAME ,HOOK ) \ { \ .scalls = static_calls_table.NAME, \ - .hook = { .NAME = HOOK } \ + .hook = { .NAME = HOOK }, \ + .runtime = false \ + } + +/* +* Initialize hooks that are inactive by default and +* enabled at runtime with security_toggle_hook. +*/ +#define LSM_HOOK_INIT_RUNTIME (NAME ,HOOK ) \ + { \ + .scalls = static_calls_table.NAME, \ + .hook = { .NAME = HOOK }, \ + .runtime = true \ }extern char *lsm_names;
@@ -207,4 +222,17 @@ extern struct lsm_info __start_early_lsm_info[], __end_early_lsm_info[];extern int lsm_inode_alloc (struct inode *inode);extern struct lsm_static_calls_table static_calls_table__ro_after_init ; +#ifdef CONFIG_SECURITY + +int security_toggle_hook (void *addr,bool value); + +#else + +static inline int security_toggle_hook (void *addr,bool value) +{ +return -EINVAL ; +} + +#endif /* CONFIG_SECURITY */ +#endif /* ! __LINUX_LSM_HOOKS_H */
kernel/bpf/trampoline.c +36 -4
@@ -523,6 +523,21 @@ static enum bpf_tramp_prog_type bpf_attach_type_to_tramp(struct bpf_prog *prog) } } +static int bpf_trampoline_toggle_lsm (struct bpf_trampoline *tr, +enum bpf_tramp_prog_type kind) +{ +struct bpf_tramp_link *link; +bool found = false; + +hlist_for_each_entry (link, &tr->progs_hlist[kind], tramp_hlist) { +if (link->link.prog->type ==BPF_PROG_TYPE_LSM ) { + found = true; +break ; + } + } +return security_toggle_hook (tr->func.addr, found); +} +static int __bpf_trampoline_link_prog (struct bpf_tramp_link *link,struct bpf_trampoline *tr) {enum bpf_tramp_prog_type kind;
@@ -562,11 +577,22 @@ static int __bpf_trampoline_link_prog(struct bpf_tramp_link *link, struct bpf_trhlist_add_head (&link->tramp_hlist, &tr->progs_hlist[kind]); tr->progs_cnt[kind]++; - err =bpf_trampoline_update (tr, true/* lock_direct_mutex */ ); -if (err) { -hlist_del_init (&link->tramp_hlist); - tr->progs_cnt[kind]--; + +if (link->link.prog->type ==BPF_PROG_TYPE_LSM ) { + err =bpf_trampoline_toggle_lsm (tr, kind); +if (err) + goto cleanup; } + + err =bpf_trampoline_update (tr, true/* lock_direct_mutex */ ); +if (err) + goto cleanup; + +return 0 ; + +cleanup: +hlist_del_init (&link->tramp_hlist); + tr->progs_cnt[kind]--;return err; }
@@ -595,6 +621,12 @@ static int __bpf_trampoline_unlink_prog(struct bpf_tramp_link *link, struct bpf_ }hlist_del_init (&link->tramp_hlist); tr->progs_cnt[kind]--; + +if (link->link.prog->type ==BPF_PROG_TYPE_LSM ) { + err =bpf_trampoline_toggle_lsm (tr, kind); +WARN (err,"BUG: unable to toggle BPF LSM hook" ); + } +return bpf_trampoline_update (tr, true/* lock_direct_mutex */ ); }
security/bpf/hooks.c +1 -1
@@ -9,7 +9,7 @@static struct security_hook_list bpf_lsm_hooks[]__ro_after_init = { #define LSM_HOOK(RET, DEFAULT, NAME, ...) \ - LSM_HOOK_INIT(NAME, bpf_lsm_##NAME), + LSM_HOOK_INIT_RUNTIME(NAME, bpf_lsm_##NAME), #include <linux/lsm_hook_defs.h> #undef LSM_HOOK LSM_HOOK_INIT(inode_free_security, bpf_inode_storage_free),
security/security.c +35 -1
@@ -409,7 +409,9 @@ static void __init lsm_static_call_init(struct security_hook_list *hl)__static_call_update (scall->key, scall->trampoline, hl->hook.lsm_func_addr); scall->hl = hl; -static_branch_enable (scall->active); +/* Runtime hooks are inactive by default */ +if (!hl->runtime) +static_branch_enable (scall->active);return ; } scall++;
@@ -888,6 +890,38 @@ int lsm_fill_user_ctx(struct lsm_ctx __user *uctx, u32 *uctx_len,return rc; } +/** +* security_toggle_hook - Toggle the state of the LSM hook. +* @hook_addr: The address of the hook to be toggled. +* @state: Whether to enable for disable the hook. +* +* Returns 0 on success, -EINVAL if the address is not found. +*/ +int security_toggle_hook (void *hook_addr,bool state) +{ +unsigned long num_entries = + (sizeof (static_calls_table) /sizeof (struct lsm_static_call )); +void *scalls_table = ((void *)&static_calls_table); +struct lsm_static_call *scall; +int i; + +for (i =0 ; i < num_entries; i++) { + scall = scalls_table + (i *sizeof (struct lsm_static_call )); +if (!scall->hl || !scall->hl->runtime) +continue ; + +if (scall->hl->hook.lsm_func_addr != hook_addr) +continue ; + +if (state) +static_branch_enable (scall->active); +else +static_branch_disable (scall->active); +return 0 ; + } +return -EINVAL ; +} +/* * The default value of the LSM hook is defined in linux/lsm_hook_defs.h and * can be accessed with:
Signature · 1 line
2.45.2.803.g4e1b14247a-goog
Paul Moore
maintainerRe: [PATCH v13 3/5] security: Replace indirect LSM hook calls with static calls
In reply to KP SinghQuoted 44 lines · On Jun 29, 2024 KP Singh <[email protected]> wrote:
LSM hooks are currently invoked from a linked list as indirect calls which are invoked using retpolines as a mitigation for speculative attacks (Branch History / Target injection) and add extra overhead which is especially bad in kernel hot paths:
security_file_ioctl:0xff...0320 <+0>: endbr64 0xff...0324 <+4>: push %rbp 0xff...0325 <+5>: push %r15 0xff...0327 <+7>: push %r14 0xff...0329 <+9>: push %rbx 0xff...032a <+10>: mov %rdx,%rbx 0xff...032d <+13>: mov %esi,%ebp 0xff...032f <+15>: mov %rdi,%r14 0xff...0332 <+18>: mov $0xff...7030,%r15 0xff...0339 <+25>: mov (%r15),%r15 0xff...033c <+28>: test %r15,%r15 0xff...033f <+31>: je 0xff...0358 <security_file_ioctl+56> 0xff...0341 <+33>: mov 0x18(%r15),%r11 0xff...0345 <+37>: mov %r14,%rdi 0xff...0348 <+40>: mov %ebp,%esi 0xff...034a <+42>: mov %rbx,%rdx 0xff...034d <+45>: call 0xff...2e0 <__x86_indirect_thunk_array+352> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Indirect calls that use retpolines leading to overhead, not just due to extra instruction but also branch misses. 0xff...0352 <+50>: test %eax,%eax 0xff...0354 <+52>: je 0xff...0339 <security_file_ioctl+25> 0xff...0356 <+54>: jmp 0xff...035a <security_file_ioctl+58> 0xff...0358 <+56>: xor %eax,%eax 0xff...035a <+58>: pop %rbx 0xff...035b <+59>: pop %r14 0xff...035d <+61>: pop %r15 0xff...035f <+63>: pop %rbp 0xff...0360 <+64>: jmp 0xff...47c4 <__x86_return_thunk>The indirect calls are not really needed as one knows the addresses of enabled LSM callbacks at boot time and only the order can possibly change at boot time with the lsm= kernel command line parameter.
Quoted 7 lines
An array of static calls is defined per LSM hook and the static calls are updated at boot time once the order has been determined.
A static key guards whether an LSM static call is enabled or not, without this static key, for LSM hooks that return an int, the presence of the hook that returns a default value can create side-effects which has resulted in bugs [1].
I believe there have been improvements in the LSM framework since the original problems were reported and I'm not entirely sure if this is still a problem. In particular I believe that commit 260017f31a8c ("lsm: use default hook return value in call_int_hook()") and 61df7b828204 ("lsm: fixup the inode xattr capability handling") should fix the more obvious problems. I'd like to know if you are aware of any others? If not, the text above should be adjusted and we should reconsider patch 5/5. If there are other problems I'd like to better understand them as there may be an independent solution for those particular problems.
Quoted 65 lines
With the hook now exposed as a static call, one can see that the retpolines are no longer there and the LSM callbacks are invoked directly:
security_file_ioctl:0xff...0ca0 <+0>: endbr64 0xff...0ca4 <+4>: nopl 0x0(%rax,%rax,1) 0xff...0ca9 <+9>: push %rbp 0xff...0caa <+10>: push %r14 0xff...0cac <+12>: push %rbx 0xff...0cad <+13>: mov %rdx,%rbx 0xff...0cb0 <+16>: mov %esi,%ebp 0xff...0cb2 <+18>: mov %rdi,%r14 0xff...0cb5 <+21>: jmp 0xff...0cc7 <security_file_ioctl+39>^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Static key enabled for SELinux0xffffffff818f0cb7 <+23>: jmp 0xff...0cde <security_file_ioctl+62> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Static key enabled for BPF LSM. This is something that is changed to default to false to avoid the existing side effect issues of BPF LSM [1] in a subsequent patch. 0xff...0cb9 <+25>: xor %eax,%eax 0xff...0cbb <+27>: xchg %ax,%ax 0xff...0cbd <+29>: pop %rbx 0xff...0cbe <+30>: pop %r14 0xff...0cc0 <+32>: pop %rbp 0xff...0cc1 <+33>: cs jmp 0xff...0000 <__x86_return_thunk> 0xff...0cc7 <+39>: endbr64 0xff...0ccb <+43>: mov %r14,%rdi 0xff...0cce <+46>: mov %ebp,%esi 0xff...0cd0 <+48>: mov %rbx,%rdx 0xff...0cd3 <+51>: call 0xff...3230 <selinux_file_ioctl> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Direct call to SELinux. 0xff...0cd8 <+56>: test %eax,%eax 0xff...0cda <+58>: jne 0xff...0cbd <security_file_ioctl+29> 0xff...0cdc <+60>: jmp 0xff...0cb7 <security_file_ioctl+23> 0xff...0cde <+62>: endbr64 0xff...0ce2 <+66>: mov %r14,%rdi 0xff...0ce5 <+69>: mov %ebp,%esi 0xff...0ce7 <+71>: mov %rbx,%rdx 0xff...0cea <+74>: call 0xff...e220 <bpf_lsm_file_ioctl> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Direct call to BPF LSM. 0xff...0cef <+79>: test %eax,%eax 0xff...0cf1 <+81>: jne 0xff...0cbd <security_file_ioctl+29> 0xff...0cf3 <+83>: jmp 0xff...0cb9 <security_file_ioctl+25> 0xff...0cf5 <+85>: endbr64 0xff...0cf9 <+89>: mov %r14,%rdi 0xff...0cfc <+92>: mov %ebp,%esi 0xff...0cfe <+94>: mov %rbx,%rdx 0xff...0d01 <+97>: pop %rbx 0xff...0d02 <+98>: pop %r14 0xff...0d04 <+100>: pop %rbp 0xff...0d05 <+101>: ret 0xff...0d06 <+102>: int3 0xff...0d07 <+103>: int3 0xff...0d08 <+104>: int3 0xff...0d09 <+105>: int3
Quoted 8 lines
While this patch uses
static_branch_unlikelyindicating that an LSM hook is likely to be not present. In most cases this is still a better choice as even when an LSM with one hook is added, empty slots are created for all LSM hooks (especially when many LSMs that do not initialize most hooks are present on the system).There are some hooks that don't use the
call_int_hookandcall_void_hook.
I think you mean "or" and not "and", yes?
Quoted 33 lines
These hooks are updated to use a new macro called
lsm_for_each_hookwhere thelsm_callbackis directly invoked as an indirect call. These are updated in a subsequent patch to also use static calls.Below are results of the relevant Unixbench system benchmarks with BPF LSM and SELinux enabled with default policies enabled with and without these patches.
Benchmark Delta(%): (+ is better)
===============================================================================Execl Throughput +1.9356 File Write 1024 bufsize 2000 maxblocks +6.5953 Pipe Throughput +9.5499 Pipe-based Context Switching +3.0209 Process Creation +2.3246 Shell Scripts (1 concurrent) +1.4975 System Call Overhead +2.7815 System Benchmarks Index Score (Partial Only): +3.4859In the best case, some syscalls like
eventfd_createbenefitted to about ~10%.[1] https://lore.kernel.org/linux-security-module/[email protected]/
Reviewed-by Casey Schaufler <[email protected]>Reviewed-by Kees Cook <[email protected]>Acked-by Song Liu <[email protected]>Acked-by Andrii Nakryiko <[email protected]>Signed-off-by KP Singh <[email protected]>--- include/linux/lsm_hooks.h | 72 ++++++++++++-- security/security.c | 198 ++++++++++++++++++++++++++------------ 2 files changed, 197 insertions(+), 73 deletions(-)
...
Quoted 21 lines
include/linux/lsm_hooks.h +13
-0@@ -30,19 +30,66 @@#include <linux/init.h> #include <linux/rculist.h> #include <linux/xattr.h> +#include <linux/static_call.h> +#include <linux/unroll.h> +#include <linux/jump_label.h> +#include <linux/lsm_count.h> + +#define SECURITY_HOOK_ACTIVE_KEY (HOOK ,IDX ) security_hook_active_##HOOK##_##IDX + +/* +* Identifier for the LSM static calls. +* HOOK is an LSM hook as defined in linux/lsm_hookdefs.h +* IDX is the index of the static call. 0 <= NUM < MAX_LSM_COUNT +*/ +#define LSM_STATIC_CALL (HOOK ,IDX ) lsm_static_call_##HOOK##_##IDX
include/linux/lsm_hooks.h line 55
Quoted 10 lines
include/linux/lsm_hooks.h +10
-0+ +/* +* Call the macro M for each LSM hook MAX_LSM_COUNT times. +*/ +#define LSM_LOOP_UNROLL (M , ...) \ +do { \ + UNROLL(MAX_LSM_COUNT, M, __VA_ARGS__) \ +} while (0) + +#define LSM_DEFINE_UNROLL (M , ...) UNROLL(MAX_LSM_COUNT, M, __VA_ARGS__)
All of the macros above (SECURITY_HOOK_ACTIVE_KEY, LSM_STATIC_CALL, LSM_LOOP_UNROLL, and LSM_DEFINE_UNROLL) are only used in security/security.c, yes? If so, is there a reason why they are defined here and not in security/security.c?
Quoted 20 lines
security/security.c +12
-0@@ -112,6 +113,51 @@ static __initconst const char *const builtin_lsm_order = CONFIG_LSM;static __initdata struct lsm_info **ordered_lsms;static __initdata struct lsm_info *exclusive; + +#ifdef CONFIG_HAVE_STATIC_CALL +#define LSM_HOOK_TRAMP (NAME ,NUM ) \ + &STATIC_CALL_TRAMP(LSM_STATIC_CALL(NAME, NUM)) +#else +#define LSM_HOOK_TRAMP (NAME ,NUM ) NULL +#endif + +/* +* Define static calls and static keys for each LSM hook. +*/ +
security/security.c static __initconst const char *const builtin_lsm_order = CONFIG_LSM; · line 137
Quoted 10 lines
security/security.c +10
-0+#define DEFINE_LSM_STATIC_CALL (NUM ,NAME ,RET , ...) \ + DEFINE_STATIC_CALL_NULL(LSM_STATIC_CALL(NAME, NUM), \ + *((RET(*)(__VA_ARGS__))NULL)); \ + DEFINE_STATIC_KEY_FALSE(SECURITY_HOOK_ACTIVE_KEY(NAME, NUM)); + +#define LSM_HOOK (RET ,DEFAULT ,NAME , ...) \ + LSM_DEFINE_UNROLL(DEFINE_LSM_STATIC_CALL, NAME, RET, __VA_ARGS__) +#include <linux/lsm_hook_defs.h> +#undef LSM_HOOK +#undef DEFINE_LSM_STATIC_CALL
If you end up respinning the patchset, drop the two blank lines before the DEFINE_LSM_STATIC_CALL and LSM_HOOK definitions.
Quoted 37 lines
security/security.c +12
-18@@ -856,29 +916,43 @@ int lsm_fill_user_ctx(struct lsm_ctx __user *uctx, u32 *uctx_len,* call_int_hook: * This is a hook that returns a value. */ +#define __CALL_STATIC_VOID (NUM ,HOOK , ...) \ +do { \ + if (static_branch_unlikely(&SECURITY_HOOK_ACTIVE_KEY(HOOK, NUM))) { \ + static_call(LSM_STATIC_CALL(HOOK, NUM))(__VA_ARGS__); \ + } \ +} while (0); -#define call_void_hook (FUNC , ...) \ - do { \ - struct security_hook_list *P; \ - \ - hlist_for_each_entry(P, &security_hook_heads.FUNC, list) \ - P->hook.FUNC(__VA_ARGS__); \ +#define call_void_hook (HOOK , ...) \ + do { \ + LSM_LOOP_UNROLL(__CALL_STATIC_VOID, HOOK, __VA_ARGS__); \ } while (0) -#define call_int_hook (FUNC , ...) ({ \ - int RC = LSM_RET_DEFAULT(FUNC); \ - do { \ - struct security_hook_list *P; \ - \ - hlist_for_each_entry(P, &security_hook_heads.FUNC, list) { \ - RC = P->hook.FUNC(__VA_ARGS__); \ - if (RC != LSM_RET_DEFAULT(FUNC)) \ - break; \ - } \ - } while (0); \ - RC; \ + +#define __CALL_STATIC_INT (NUM ,R ,HOOK ,LABEL , ...) \ +do { \
security/security.c int lsm_fill_user_ctx(struct lsm_ctx __user *uctx, u32 *uctx_len, · line 943
Quoted 10 lines
security/security.c +10
-0+if (static_branch_unlikely (&SECURITY_HOOK_ACTIVE_KEY (HOOK ,NUM ))) { \ +R =static_call (LSM_STATIC_CALL (HOOK ,NUM ))(__VA_ARGS__); \ +if (R !=LSM_RET_DEFAULT (HOOK )) \ + goto LABEL; \ + } \ +}while (0 ); + +#define call_int_hook (HOOK , ...) \ +({ \ + __label__ out; \
This is another only-if-you-respin, consider /out/OUT/ so it is more consistent.
security/security.c int lsm_fill_user_ctx(struct lsm_ctx __user *uctx, u32 *uctx_len, · line 954
Quoted 11 lines
security/security.c +9
-0+int RC =LSM_RET_DEFAULT (HOOK ); \ + \ +LSM_LOOP_UNROLL (__CALL_STATIC_INT,RC ,HOOK , out, __VA_ARGS__); \ +out: \ +RC ; \ }) +#define lsm_for_each_hook (scall,NAME ) \ + for (scall = static_calls_table.NAME; \ + scall - static_calls_table.NAME < MAX_LSM_COUNT; scall++) \ + if (static_key_enabled(&scall->active->key))
This is probably a stupid question, but why use static_key_enabled() here instead of static_branch_unlikely() as in the call_XXX_macros?
--
paul-moore.com
Paul Moore
maintainerRe: [PATCH v13 4/5] security: Update non standard hooks to use static calls
In reply to KP SinghQuoted 31 lines · On Jun 29, 2024 KP Singh <[email protected]> wrote:
There are some LSM hooks which do not use the common pattern followed by other LSM hooks and thus cannot use call_{int, void}_hook macros and instead use
lsm_for_each_hookmacro which still results in indirect call.There is one additional generalizable pattern where a hook matching an lsmid is called and the indirect calls for these are addressed with the newly added
call_hook_with_lsmidmacro which internally uses an implementation similar tocall_int_hookbut has an additional check that matches the lsmid.For the generic case the
lsm_for_each_hookmacro is updated to accept logic before and after the invocation of the LSM hook (static call) in the unrolled loop.Reviewed-by Casey Schaufler <[email protected]>Reviewed-by Kees Cook <[email protected]>Signed-off-by KP Singh <[email protected]>Reviewed-by John Johansen <[email protected]>--- security/security.c | 248 +++++++++++++++++++++++++------------------- 1 file changed, 144 insertions(+), 104 deletions(-)security/security.c +0
-0@@ -948,10 +948,48 @@ out: \RC ; \ })
security/security.c out: \ · line 955
Quoted 10 lines
security/security.c +5
-4-#define lsm_for_each_hook (scall,NAME ) \ - for (scall = static_calls_table.NAME; \ - scall - static_calls_table.NAME < MAX_LSM_COUNT; scall++) \ - if (static_key_enabled(&scall->active->key)) +/* +* Can be used in the context passed to lsm_for_each_hook to get the lsmid of the +* current hook +*/ +#define current_lsmid () _hook_lsmid
See my comments below about security_getselfattr(), I think we can drop the current_lsmid() macro. If we really must keep it, we need to rename it to something else as it clashes too much with the other current_XXX() macros/functions which are useful outside of our wacky macros.
Quoted 38 lines
security/security.c +36
-0+#define __CALL_HOOK (NUM ,HOOK ,RC ,BLOCK_BEFORE ,BLOCK_AFTER , ...) \ +do { \ + int __maybe_unused _hook_lsmid; \ + \ + if (static_branch_unlikely(&SECURITY_HOOK_ACTIVE_KEY(HOOK, NUM))) { \ + _hook_lsmid = static_calls_table.HOOK[NUM].hl->lsmid->id; \ + BLOCK_BEFORE \ + RC = static_call(LSM_STATIC_CALL(HOOK, NUM))(__VA_ARGS__); \ + BLOCK_AFTER \ + } \ +} while (0); + +#define lsm_for_each_hook (HOOK ,RC ,BLOCK_AFTER , ...) \ + LSM_LOOP_UNROLL(__CALL_HOOK, HOOK, RC, ;, BLOCK_AFTER, __VA_ARGS__) + +#define call_hook_with_lsmid (HOOK ,LSMID , ...) \ +({ \ + __label__ out; \ + int RC = LSM_RET_DEFAULT(HOOK); \ + \ + LSM_LOOP_UNROLL(__CALL_HOOK, HOOK, RC, \ +/* BLOCK BEFORE INVOCATION */ \ + { \ +if (current_lsmid () !=LSMID ) \ +continue ; \ + }, \ +/* END BLOCK BEFORE INVOCATION */ \ +/* BLOCK AFTER INVOCATION */ \ + { \ + goto out; \ + }, \ +/* END BLOCK AFTER INVOCATION */ \ + __VA_ARGS__); \ +out: \ +RC ; \ +})/* Security operations */
...
Quoted 17 lines
security/security.c +4
-7@@ -1581,15 +1629,19 @@ int security_sb_set_mnt_opts(struct super_block *sb,unsigned long kern_flags, unsigned long *set_kern_flags) { - struct lsm_static_call *scall; int rc = mnt_opts ? -EOPNOTSUPP : LSM_RET_DEFAULT(sb_set_mnt_opts); -lsm_for_each_hook (scall, sb_set_mnt_opts) { - rc = scall->hl->hook.sb_set_mnt_opts (sb, mnt_opts, kern_flags, - set_kern_flags); -if (rc !=LSM_RET_DEFAULT (sb_set_mnt_opts)) -break ; - } + lsm_for_each_hook( + sb_set_mnt_opts, rc, + /* BLOCK AFTER INVOCATION */ + {
security/security.c int security_sb_set_mnt_opts(struct super_block *sb, · line 1647
Quoted 10 lines
security/security.c +7
-0+if (rc !=LSM_RET_DEFAULT (sb_set_mnt_opts)) + goto out; + }, +/* END BLOCK AFTER INVOCATION */ + sb, mnt_opts, kern_flags, set_kern_flags); + +out:return rc; }EXPORT_SYMBOL (security_sb_set_mnt_opts);
I know I was the one who asked to implement the static_calls for *all* of the LSM functions - thank you for doing that - but I think we can all agree that some of the resulting code is pretty awful. I'm probably missing something important, but would an apporach similar to the pseudo code below work?
#define call_int_hook_special(HOOK, RC, LABEL, ...) \
LSM_LOOP_UNROLL(HOOK##_SPECIAL, RC, HOOK, LABEL, __VA_ARGS__)
int security_sb_set_mnt_opts(...)
{
int rc = LSM_RET_DEFAULT(sb_set_mnt_opts);
#define sb_set_mnt_opts_SPECIAL \
do { \
if (rc != LSM_RET_DEFAULT(sb_set_mnt_opts)) \
goto out; \
} while (0)rc = call_int_hook_special(sb_set_mnt_opts, rc, out, ...);
out:
return rc;
}Quoted 65 lines
security/security.c +34
-20@@ -4040,7 +4099,6 @@ EXPORT_SYMBOL(security_d_instantiate);int security_getselfattr (unsigned int attr,struct lsm_ctx __user *uctx,u32 __user *size,u32 flags) { -struct lsm_static_call *scall;struct lsm_ctx lctx = { .id =LSM_ID_UNDEF , };u8 __user *base = (u8 __user *)uctx;u32 entrysize;@@ -4078,31 +4136,42 @@ int security_getselfattr(unsigned int attr, struct lsm_ctx __user *uctx,* In the usual case gather all the data from the LSMs. * In the single case only get the data from the LSM specified. */ -lsm_for_each_hook (scall, getselfattr) { -if (single && lctx.id != scall->hl->lsmid->id) -continue ; - entrysize = left; -if (base) - uctx = (struct lsm_ctx __user *)(base + total); - rc = scall->hl->hook.getselfattr (attr, uctx, &entrysize, flags); -if (rc == -EOPNOTSUPP ) { - rc =0 ; -continue ; - } -if (rc == -E2BIG ) { - rc =0 ; - left =0 ; - toobig = true; - }else if (rc <0 ) -return rc; -else - left -= entrysize; +LSM_LOOP_UNROLL ( + __CALL_HOOK, getselfattr, rc, +/* BLOCK BEFORE INVOCATION */ + { +if (single && lctx.id !=current_lsmid ()) +continue ; + entrysize = left; +if (base) + uctx = (struct lsm_ctx __user *)(base + total); + }, +/* END BLOCK BEFORE INVOCATION */ +/* BLOCK AFTER INVOCATION */ + { +if (rc == -EOPNOTSUPP ) { + rc =0 ; + }else { +if (rc == -E2BIG ) { + rc =0 ; + left =0 ; + toobig = true; + }else if (rc <0 ) +return rc; +else + left -= entrysize; + + total += entrysize; + count += rc; +if (single) + goto out; + } + }, +/* END BLOCK AFTER INVOCATION */ + attr, uctx, &entrysize, flags); +
security/security.c int security_getselfattr(unsigned int attr, struct lsm_ctx __user *uctx, · line 4177
Quoted 10 lines
security/security.c +1
-5+out: - total += entrysize; - count += rc; -if (single) -break ; - }if (put_user (total, size))return -EFAULT ;if (toobig)
I think we may need to admit defeat with security_getselfattr() and leave it as-is, the above is just too ugly to live. I'd suggest adding a comment explaining that it wasn't converted due to complexity and the resulting awfulness.
--
paul-moore.com
Paul Moore
maintainerRe: [PATCH v13 5/5] bpf: Only enable BPF LSM hooks when an LSM program is attached
In reply to KP SinghQuoted 68 lines · On Jun 29, 2024 KP Singh <[email protected]> wrote:
BPF LSM hooks have side-effects (even when a default value's returned) as some hooks end up behaving differently due to the very presence of the hook.
The static keys guarding the BPF LSM hooks are disabled by default and enabled only when a BPF program is attached implementing the hook logic. This avoids the issue of the side-effects and also the minor overhead associated with the empty callback.
security_file_ioctl:0xff...0e30 <+0>: endbr64 0xff...0e34 <+4>: nopl 0x0(%rax,%rax,1) 0xff...0e39 <+9>: push %rbp 0xff...0e3a <+10>: push %r14 0xff...0e3c <+12>: push %rbx 0xff...0e3d <+13>: mov %rdx,%rbx 0xff...0e40 <+16>: mov %esi,%ebp 0xff...0e42 <+18>: mov %rdi,%r14 0xff...0e45 <+21>: jmp 0xff...0e57 <security_file_ioctl+39> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Static key enabled for SELinux 0xff...0e47 <+23>: xchg %ax,%ax ^^^^^^^^^^^^^^ Static key disabled for BPF. This gets patched when a BPF LSM program is attached 0xff...0e49 <+25>: xor %eax,%eax 0xff...0e4b <+27>: xchg %ax,%ax 0xff...0e4d <+29>: pop %rbx 0xff...0e4e <+30>: pop %r14 0xff...0e50 <+32>: pop %rbp 0xff...0e51 <+33>: cs jmp 0xff...0000 <__x86_return_thunk> 0xff...0e57 <+39>: endbr64 0xff...0e5b <+43>: mov %r14,%rdi 0xff...0e5e <+46>: mov %ebp,%esi 0xff...0e60 <+48>: mov %rbx,%rdx 0xff...0e63 <+51>: call 0xff...33c0 <selinux_file_ioctl> 0xff...0e68 <+56>: test %eax,%eax 0xff...0e6a <+58>: jne 0xff...0e4d <security_file_ioctl+29> 0xff...0e6c <+60>: jmp 0xff...0e47 <security_file_ioctl+23> 0xff...0e6e <+62>: endbr64 0xff...0e72 <+66>: mov %r14,%rdi 0xff...0e75 <+69>: mov %ebp,%esi 0xff...0e77 <+71>: mov %rbx,%rdx 0xff...0e7a <+74>: call 0xff...e3b0 <bpf_lsm_file_ioctl> 0xff...0e7f <+79>: test %eax,%eax 0xff...0e81 <+81>: jne 0xff...0e4d <security_file_ioctl+29> 0xff...0e83 <+83>: jmp 0xff...0e49 <security_file_ioctl+25> 0xff...0e85 <+85>: endbr64 0xff...0e89 <+89>: mov %r14,%rdi 0xff...0e8c <+92>: mov %ebp,%esi 0xff...0e8e <+94>: mov %rbx,%rdx 0xff...0e91 <+97>: pop %rbx 0xff...0e92 <+98>: pop %r14 0xff...0e94 <+100>: pop %rbp 0xff...0e95 <+101>: retThis patch enables this by providing a
LSM_HOOK_INIT_RUNTIMEvariant that allows the LSMs to opt-in to hooks which can be toggled at runtime which withsecurity_toogle_hook.Reviewed-by Kees Cook <[email protected]>Acked-by Casey Schaufler <[email protected]>Signed-off-by KP Singh <[email protected]>
Quoted 6 lines
--- include/linux/lsm_hooks.h | 30 ++++++++++++++++++++++++++++- kernel/bpf/trampoline.c | 40 +++++++++++++++++++++++++++++++++++---- security/bpf/hooks.c | 2 +- security/security.c | 36 ++++++++++++++++++++++++++++++++++- 4 files changed, 101 insertions(+), 7 deletions(-)
I didn't look at this one too closely, see my previous comments in patch 3/5, but I did catch one typo, see below ...
Quoted 7 lines
include/linux/lsm_hooks.h +0
-0@@ -110,11 +110,14 @@ struct lsm_id {* @scalls: The beginning of the array of static calls assigned to this hook. * @hook: The callback for the hook.
include/linux/lsm_hooks.h struct lsm_id { · line 121
Quoted 10 lines
include/linux/lsm_hooks.h +3
-0* @lsm: The name of the lsm that owns this hook. +* @default_state: The state of the LSM hook when initialized. If set to false, +* the static key guarding the hook will be set to disabled. */ struct security_hook_list {struct lsm_static_call *scalls;union security_list_options hook;const struct lsm_id *lsmid; +bool runtime; } __randomize_layout;
The comment header doesn't match the struct fields, "default_state" vs "runtime".
--
paul-moore.com
Re: [PATCH v13 2/5] security: Count the LSMs enabled at compile time
In reply to KP SinghQuoted 7 lines · KP Singh
These macros are a clever trick to determine a count of the number of LSMs that are enabled in the config to ascertain the maximum number of static calls that need to be configured per LSM hook.
Without this one would need to generate static calls for the total number of LSMs in the kernel (even if they are not compiled) times the number of LSM hooks which ends up being quite wasteful.
[snip]
Quoted 38 lines
include/linux/lsm_count.h new file +32
-0@@ -0,0 +1,128 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* +* Copyright (C) 2023 Google LLC. +*/ + +#ifndef __LINUX_LSM_COUNT_H +#define __LINUX_LSM_COUNT_H + +#include <linux/args.h> + +#ifdef CONFIG_SECURITY + +/* +* Macros to count the number of LSMs enabled in the kernel at compile time. +*/ + +/* +* Capabilities is enabled when CONFIG_SECURITY is enabled. +*/ +#if IS_ENABLED (CONFIG_SECURITY ) +#define CAPABILITIES_ENABLED 1, +#else +#define CAPABILITIES_ENABLED +#endif + +#if IS_ENABLED (CONFIG_SECURITY_SELINUX ) +#define SELINUX_ENABLED 1, +#else +#define SELINUX_ENABLED +#endif +
[snip]
Quoted 27 lines
include/linux/lsm_count.h new file +27
-0+ +#if IS_ENABLED (CONFIG_EVM ) +#define EVM_ENABLED 1, +#else +#define EVM_ENABLED +#endif + +/* +* There is a trailing comma that we need to be accounted for. This is done by +* using a skipped argument in __COUNT_LSMS +*/ +#define __COUNT_LSMS(skipped_arg, args...) COUNT_ARGS(args...) +#define COUNT_LSMS(args...) __COUNT_LSMS(args) + +#define MAX_LSM_COUNT \ + COUNT_LSMS( \ + CAPABILITIES_ENABLED \ + SELINUX_ENABLED \ + SMACK_ENABLED \ + APPARMOR_ENABLED \ + TOMOYO_ENABLED \ + YAMA_ENABLED \ + LOADPIN_ENABLED \ + LOCKDOWN_ENABLED \ + SAFESETID_ENABLED \ + BPF_LSM_ENABLED \ + LANDLOCK_ENABLED \
include/linux/lsm_count.h line 128
Quoted 10 lines
include/linux/lsm_count.h new file +10
-0+ IMA_ENABLED \ + EVM_ENABLED) + +#else + +#define MAX_LSM_COUNT 0 + +#endif /* CONFIG_SECURITY */ + +#endif /* __LINUX_LSM_COUNT_H */
OK, so I can tell from the other patches that this isn't just about getting MAX_LSM_COUNT to be a compile-time constant, it really has to be a single preprocessor token representing the right decimal value. That information could have been in some comment or the commit log. So
#define MAX_LSM_COUNT (IS_ENABLED(CONFIG_SECURITY) + IS_ENABLED(CONFIG_SECURITY_SELINUX) + ...)
doesn't work immediately. But this does provide not just a compile-time constant, but a preprocessor constant, so:
Instead of all this trickery with defining temporary, never used again, macros expanding to something with trailing comma or not, what about this simpler (at least in terms of LOC, but IMO also readability) approach:
/*
* The sum of the IS_ENABLED() values provides the right value, but we
* need MAX_LSM_COUNT to be a single preprocessor token representing
* that value, because it will be passed to the UNROLL macro which
* does token concatenation.
*/
#define __MAX_LSM_COUNT (\
IS_ENABLED(CONFIG_SECURITY) /* capabilities */ + \
IS_ENABLED(CONFIG_SECURITY_SELINUX) + \
... \
IS_ENABLED(CONFIG_EVM) \
)
#if __MAX_LSM_COUNT == 0
#define MAX_LSM_COUNT 0
#elif __MAX_LSM_COUNT == 1
#define MAX_LSM_COUNT 1
#elif
...
#elif __MAX_LSM_COUNT == 15
#define MAX_LSM_COUNT 15
#else
#error "Too many LSMs, add an #elif case" #endif
Rasmus
KP Singh
maintainerRe: [PATCH v13 2/5] security: Count the LSMs enabled at compile time
Quoted 108 lines · Rasmus Villemoes
Quoted 7 lines · KP Singh
These macros are a clever trick to determine a count of the number of LSMs that are enabled in the config to ascertain the maximum number of static calls that need to be configured per LSM hook.
Without this one would need to generate static calls for the total number of LSMs in the kernel (even if they are not compiled) times the number of LSM hooks which ends up being quite wasteful.
[snip]
Quoted 38 lines
include/linux/lsm_count.h new file +32
-0@@ -0,0 +1,128 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* +* Copyright (C) 2023 Google LLC. +*/ + +#ifndef __LINUX_LSM_COUNT_H +#define __LINUX_LSM_COUNT_H + +#include <linux/args.h> + +#ifdef CONFIG_SECURITY + +/* +* Macros to count the number of LSMs enabled in the kernel at compile time. +*/ + +/* +* Capabilities is enabled when CONFIG_SECURITY is enabled. +*/ +#if IS_ENABLED (CONFIG_SECURITY ) +#define CAPABILITIES_ENABLED 1, +#else +#define CAPABILITIES_ENABLED +#endif + +#if IS_ENABLED (CONFIG_SECURITY_SELINUX ) +#define SELINUX_ENABLED 1, +#else +#define SELINUX_ENABLED +#endif +[snip]
Quoted 27 lines
include/linux/lsm_count.h new file +27
-0+ +#if IS_ENABLED (CONFIG_EVM ) +#define EVM_ENABLED 1, +#else +#define EVM_ENABLED +#endif + +/* +* There is a trailing comma that we need to be accounted for. This is done by +* using a skipped argument in __COUNT_LSMS +*/ +#define __COUNT_LSMS(skipped_arg, args...) COUNT_ARGS(args...) +#define COUNT_LSMS(args...) __COUNT_LSMS(args) + +#define MAX_LSM_COUNT \ + COUNT_LSMS( \ + CAPABILITIES_ENABLED \ + SELINUX_ENABLED \ + SMACK_ENABLED \ + APPARMOR_ENABLED \ + TOMOYO_ENABLED \ + YAMA_ENABLED \ + LOADPIN_ENABLED \ + LOCKDOWN_ENABLED \ + SAFESETID_ENABLED \ + BPF_LSM_ENABLED \ + LANDLOCK_ENABLED \include/linux/lsm_count.h line 128
Quoted 10 lines
include/linux/lsm_count.h new file +10
-0+ IMA_ENABLED \ + EVM_ENABLED) + +#else + +#define MAX_LSM_COUNT 0 + +#endif /* CONFIG_SECURITY */ + +#endif /* __LINUX_LSM_COUNT_H */ OK, so I can tell from the other patches that this isn't just about getting
MAX_LSM_COUNTto be a compile-time constant, it really has to be a single preprocessor token representing the right decimal value. That information could have been in some comment or the commit log. So#define
MAX_LSM_COUNT(IS_ENABLED(CONFIG_SECURITY)+IS_ENABLED(CONFIG_SECURITY_SELINUX)+ ...)doesn't work immediately. But this does provide not just a compile-time constant, but a preprocessor constant, so:
Instead of all this trickery with defining temporary, never used again, macros expanding to something with trailing comma or not, what about this simpler (at least in terms of LOC, but IMO also readability) approach:
Quoted 7 lines
/*
* The sum of theIS_ENABLED()values provides the right value, but we
* needMAX_LSM_COUNTto be a single preprocessor token representing
* that value, because it will be passed to the UNROLL macro which
* does token concatenation.
*/
I actually prefer the version we have now from a readability perspective, it makes it more explicit (the check about the CONFIG_* being enabled and counting them). let's keep this as an incremental change that you can propose :) once the patches are merged.
- KP
Quoted 19 lines
#define
__MAX_LSM_COUNT(\
IS_ENABLED(CONFIG_SECURITY)/* capabilities */ + \
IS_ENABLED(CONFIG_SECURITY_SELINUX)+ \
... \
IS_ENABLED(CONFIG_EVM)\
)
#if__MAX_LSM_COUNT== 0
#defineMAX_LSM_COUNT0
#elif__MAX_LSM_COUNT== 1
#defineMAX_LSM_COUNT1
#elif
...
#elif__MAX_LSM_COUNT== 15
#defineMAX_LSM_COUNT15
#else
#error "Too many LSMs, add an #elif case" #endifRasmus
Paul Moore
maintainerRe: [PATCH v13 2/5] security: Count the LSMs enabled at compile time
Quoted 6 lines · KP Singh
Quoted 6 lines · Rasmus Villemoes
Quoted 3 lines · KP Singh
These macros are a clever trick to determine a count of the number of LSMs that are enabled in the config to ascertain the maximum number of static calls that need to be configured per LSM hook.
...
Quoted 4 lines
Quoted 4 lines
Instead of all this trickery with defining temporary, never used again, macros expanding to something with trailing comma or not, what about this simpler (at least in terms of LOC, but IMO also readability) approach:
...
Quoted 1 line
I actually prefer the version we have now from a readability perspective, it makes it more explicit (the check about the CONFIG_* being enabled and counting them). let's keep this as an incremental change that you can propose :) once the patches are merged.
I prefer the original approach by KP as well, let's leave it as-is. IMO, it's far from the worst of the macro shenanigans in this patchset (or existing LSM code for that matter).
Signature · 1 line
paul-moore.com
KP Singh
maintainerRe: [PATCH v13 3/5] security: Replace indirect LSM hook calls with static calls
In reply to Paul MooreQuoted 7 lines · Paul Moore
Quoted 5 lines · On Jun 29, 2024 KP Singh <[email protected]> wrote:
LSM hooks are currently invoked from a linked list as indirect calls which are invoked using retpolines as a mitigation for speculative attacks (Branch History / Target injection) and add extra overhead which is especially bad in kernel hot paths:
[...]
Quoted 5 lines
should fix the more obvious problems. I'd like to know if you are aware of any others? If not, the text above should be adjusted and we should reconsider patch 5/5. If there are other problems I'd like to better understand them as there may be an independent solution for those particular problems.
We did have problems with some other hooks but I was unable to dig up specific examples though, it's been a while. More broadly speaking, a default decision is still a decision. Whereas the intent from the BPF LSM is not to make a default decision unless a BPF program is loaded. I am quite worried about the holes this leaves open, subtle bugs (security or crashes) we have not caught yet and PATCH 5/5 engineers away
the problem of the "default decision".
Quoted 72 lines
Quoted 72 lines
With the hook now exposed as a static call, one can see that the retpolines are no longer there and the LSM callbacks are invoked directly:
security_file_ioctl:0xff...0ca0 <+0>: endbr64 0xff...0ca4 <+4>: nopl 0x0(%rax,%rax,1) 0xff...0ca9 <+9>: push %rbp 0xff...0caa <+10>: push %r14 0xff...0cac <+12>: push %rbx 0xff...0cad <+13>: mov %rdx,%rbx 0xff...0cb0 <+16>: mov %esi,%ebp 0xff...0cb2 <+18>: mov %rdi,%r14 0xff...0cb5 <+21>: jmp 0xff...0cc7 <security_file_ioctl+39> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^Static key enabled for SELinux
0xffffffff818f0cb7 <+23>: jmp 0xff...0cde <security_file_ioctl+62> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Static key enabled for BPF LSM. This is something that is changed to default to false to avoid the existing side effect issues of BPF LSM [1] in a subsequent patch. 0xff...0cb9 <+25>: xor %eax,%eax 0xff...0cbb <+27>: xchg %ax,%ax 0xff...0cbd <+29>: pop %rbx 0xff...0cbe <+30>: pop %r14 0xff...0cc0 <+32>: pop %rbp 0xff...0cc1 <+33>: cs jmp 0xff...0000 <__x86_return_thunk> 0xff...0cc7 <+39>: endbr64 0xff...0ccb <+43>: mov %r14,%rdi 0xff...0cce <+46>: mov %ebp,%esi 0xff...0cd0 <+48>: mov %rbx,%rdx 0xff...0cd3 <+51>: call 0xff...3230 <selinux_file_ioctl> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Direct call to SELinux. 0xff...0cd8 <+56>: test %eax,%eax 0xff...0cda <+58>: jne 0xff...0cbd <security_file_ioctl+29> 0xff...0cdc <+60>: jmp 0xff...0cb7 <security_file_ioctl+23> 0xff...0cde <+62>: endbr64 0xff...0ce2 <+66>: mov %r14,%rdi 0xff...0ce5 <+69>: mov %ebp,%esi 0xff...0ce7 <+71>: mov %rbx,%rdx 0xff...0cea <+74>: call 0xff...e220 <bpf_lsm_file_ioctl> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Direct call to BPF LSM. 0xff...0cef <+79>: test %eax,%eax 0xff...0cf1 <+81>: jne 0xff...0cbd <security_file_ioctl+29> 0xff...0cf3 <+83>: jmp 0xff...0cb9 <security_file_ioctl+25> 0xff...0cf5 <+85>: endbr64 0xff...0cf9 <+89>: mov %r14,%rdi 0xff...0cfc <+92>: mov %ebp,%esi 0xff...0cfe <+94>: mov %rbx,%rdx 0xff...0d01 <+97>: pop %rbx 0xff...0d02 <+98>: pop %r14 0xff...0d04 <+100>: pop %rbp 0xff...0d05 <+101>: ret 0xff...0d06 <+102>: int3 0xff...0d07 <+103>: int3 0xff...0d08 <+104>: int3 0xff...0d09 <+105>: int3While this patch uses
static_branch_unlikelyindicating that an LSM hook is likely to be not present. In most cases this is still a better choice as even when an LSM with one hook is added, empty slots are created for all LSM hooks (especially when many LSMs that do not initialize most hooks are present on the system).
Quoted 4 lines
Quoted 2 lines
There are some hooks that don't use the
call_int_hookandcall_void_hook.I think you mean "or" and not "and", yes?
Yep, thanks, fixed.
Quoted 65 lines
Quoted 34 lines
These hooks are updated to use a new macro called
lsm_for_each_hookwhere thelsm_callbackis directly invoked as an indirect call. These are updated in a subsequent patch to also use static calls.Below are results of the relevant Unixbench system benchmarks with BPF LSM and SELinux enabled with default policies enabled with and without these patches.
Benchmark Delta(%): (+ is better)
===============================================================================Execl Throughput +1.9356 File Write 1024 bufsize 2000 maxblocks +6.5953 Pipe Throughput +9.5499 Pipe-based Context Switching +3.0209 Process Creation +2.3246 Shell Scripts (1 concurrent) +1.4975 System Call Overhead +2.7815 System Benchmarks Index Score (Partial Only): +3.4859In the best case, some syscalls like
eventfd_createbenefitted to about ~10%.[1] https://lore.kernel.org/linux-security-module/[email protected]/
Reviewed-by Casey Schaufler <[email protected]>Reviewed-by Kees Cook <[email protected]>Acked-by Song Liu <[email protected]>Acked-by Andrii Nakryiko <[email protected]>Signed-off-by KP Singh <[email protected]>--- include/linux/lsm_hooks.h | 72 ++++++++++++-- security/security.c | 198 ++++++++++++++++++++++++++------------ 2 files changed, 197 insertions(+), 73 deletions(-)...
Quoted 28 lines
include/linux/lsm_hooks.h +20
-0@@ -30,19 +30,66 @@#include <linux/init.h> #include <linux/rculist.h> #include <linux/xattr.h> +#include <linux/static_call.h> +#include <linux/unroll.h> +#include <linux/jump_label.h> +#include <linux/lsm_count.h> + +#define SECURITY_HOOK_ACTIVE_KEY (HOOK ,IDX ) security_hook_active_##HOOK##_##IDX + +/* +* Identifier for the LSM static calls. +* HOOK is an LSM hook as defined in linux/lsm_hookdefs.h +* IDX is the index of the static call. 0 <= NUM < MAX_LSM_COUNT +*/ +#define LSM_STATIC_CALL (HOOK ,IDX ) lsm_static_call_##HOOK##_##IDX + +/* +* Call the macro M for each LSM hook MAX_LSM_COUNT times. +*/ +#define LSM_LOOP_UNROLL (M , ...) \ +do { \ + UNROLL(MAX_LSM_COUNT, M, __VA_ARGS__) \
Quoted 8 lines
include/linux/lsm_hooks.h line 55
Quoted 3 lines
include/linux/lsm_hooks.h +3
-0+} while (0) + +#define LSM_DEFINE_UNROLL (M , ...) UNROLL(MAX_LSM_COUNT, M, __VA_ARGS__)All of the macros above (
SECURITY_HOOK_ACTIVE_KEY,LSM_STATIC_CALL,LSM_LOOP_UNROLL, andLSM_DEFINE_UNROLL) are only used insecurity/security.c, yes? If so, is there a reason why they are defined here and not insecurity/security.c?
Fair point, fixed.
Quoted 26 lines
Quoted 26 lines
security/security.c +17
-0@@ -112,6 +113,51 @@ static __initconst const char *const builtin_lsm_order = CONFIG_LSM;static __initdata struct lsm_info **ordered_lsms;static __initdata struct lsm_info *exclusive; + +#ifdef CONFIG_HAVE_STATIC_CALL +#define LSM_HOOK_TRAMP (NAME ,NUM ) \ + &STATIC_CALL_TRAMP(LSM_STATIC_CALL(NAME, NUM)) +#else +#define LSM_HOOK_TRAMP (NAME ,NUM ) NULL +#endif + +/* +* Define static calls and static keys for each LSM hook. +*/ + +#define DEFINE_LSM_STATIC_CALL (NUM ,NAME ,RET , ...) \ + DEFINE_STATIC_CALL_NULL(LSM_STATIC_CALL(NAME, NUM), \ + *((RET(*)(__VA_ARGS__))NULL)); \ + DEFINE_STATIC_KEY_FALSE(SECURITY_HOOK_ACTIVE_KEY(NAME, NUM)); +
Quoted 8 lines
security/security.c static __initconst const char *const builtin_lsm_order = CONFIG_LSM; · line 137
Quoted 5 lines
security/security.c +5
-0+#define LSM_HOOK (RET ,DEFAULT ,NAME , ...) \ + LSM_DEFINE_UNROLL(DEFINE_LSM_STATIC_CALL, NAME, RET, __VA_ARGS__) +#include <linux/lsm_hook_defs.h> +#undef LSM_HOOK +#undef DEFINE_LSM_STATIC_CALLIf you end up respinning the patchset, drop the two blank lines before the
DEFINE_LSM_STATIC_CALLandLSM_HOOKdefinitions.
Done.
Quoted 43 lines
Quoted 43 lines
security/security.c +17
-18@@ -856,29 +916,43 @@ int lsm_fill_user_ctx(struct lsm_ctx __user *uctx, u32 *uctx_len,* call_int_hook: * This is a hook that returns a value. */ +#define __CALL_STATIC_VOID (NUM ,HOOK , ...) \ +do { \ + if (static_branch_unlikely(&SECURITY_HOOK_ACTIVE_KEY(HOOK, NUM))) { \ + static_call(LSM_STATIC_CALL(HOOK, NUM))(__VA_ARGS__); \ + } \ +} while (0); -#define call_void_hook (FUNC , ...) \ - do { \ - struct security_hook_list *P; \ - \ - hlist_for_each_entry(P, &security_hook_heads.FUNC, list) \ - P->hook.FUNC(__VA_ARGS__); \ +#define call_void_hook (HOOK , ...) \ + do { \ + LSM_LOOP_UNROLL(__CALL_STATIC_VOID, HOOK, __VA_ARGS__); \ } while (0) -#define call_int_hook (FUNC , ...) ({ \ - int RC = LSM_RET_DEFAULT(FUNC); \ - do { \ - struct security_hook_list *P; \ - \ - hlist_for_each_entry(P, &security_hook_heads.FUNC, list) { \ - RC = P->hook.FUNC(__VA_ARGS__); \ - if (RC != LSM_RET_DEFAULT(FUNC)) \ - break; \ - } \ - } while (0); \ - RC; \ + +#define __CALL_STATIC_INT (NUM ,R ,HOOK ,LABEL , ...) \ +do { \ + if (static_branch_unlikely(&SECURITY_HOOK_ACTIVE_KEY(HOOK, NUM))) { \ + R = static_call(LSM_STATIC_CALL(HOOK, NUM))(__VA_ARGS__); \ + if (R != LSM_RET_DEFAULT(HOOK)) \ + goto LABEL; \ + } \
Quoted 8 lines
security/security.c int lsm_fill_user_ctx(struct lsm_ctx __user *uctx, u32 *uctx_len, · line 943
Quoted 5 lines
security/security.c +5
-0+}while (0 ); + +#define call_int_hook (HOOK , ...) \ +({ \ + __label__ out; \This is another only-if-you-respin, consider /out/OUT/ so it is more consistent.
Done. I will do a re-spin as we do have some fixes.
Quoted 8 lines
Quoted 8 lines
security/security.c +5
-0+int RC =LSM_RET_DEFAULT (HOOK ); \ + \ +LSM_LOOP_UNROLL (__CALL_STATIC_INT,RC ,HOOK , out, __VA_ARGS__); \ +out: \ +RC ; \ })
Quoted 7 lines
security/security.c int lsm_fill_user_ctx(struct lsm_ctx __user *uctx, u32 *uctx_len, · line 954
Quoted 4 lines
security/security.c +4
-0+#define lsm_for_each_hook (scall,NAME ) \ + for (scall = static_calls_table.NAME; \ + scall - static_calls_table.NAME < MAX_LSM_COUNT; scall++) \ + if (static_key_enabled(&scall->active->key))This is probably a stupid question, but why use
static_key_enabled()here instead ofstatic_branch_unlikely()as in thecall_XXX_macros?
The static_key_enabled is a check for the key being enabled, whereas the static_branch_likely is for laying out the right assembly code (jump tables etc.) and based on the value of the static key, here we are not using the static calls or even keys, rather we are following back from direct calls to indirect calls and don't really need any jump tables in the slow path.
We also get rid of this macro in the
subsequent patch, but we might need to keep it around if we leave security_getselfattr alone.
- KP
Quoted 3 lines
--
paul-moore.com
KP Singh
maintainerRe: [PATCH v13 5/5] bpf: Only enable BPF LSM hooks when an LSM program is attached
In reply to Paul MooreQuoted 92 lines · Paul Moore
Quoted 68 lines · On Jun 29, 2024 KP Singh <[email protected]> wrote:
BPF LSM hooks have side-effects (even when a default value's returned) as some hooks end up behaving differently due to the very presence of the hook.
The static keys guarding the BPF LSM hooks are disabled by default and enabled only when a BPF program is attached implementing the hook logic. This avoids the issue of the side-effects and also the minor overhead associated with the empty callback.
security_file_ioctl:0xff...0e30 <+0>: endbr64 0xff...0e34 <+4>: nopl 0x0(%rax,%rax,1) 0xff...0e39 <+9>: push %rbp 0xff...0e3a <+10>: push %r14 0xff...0e3c <+12>: push %rbx 0xff...0e3d <+13>: mov %rdx,%rbx 0xff...0e40 <+16>: mov %esi,%ebp 0xff...0e42 <+18>: mov %rdi,%r14 0xff...0e45 <+21>: jmp 0xff...0e57 <security_file_ioctl+39> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^Static key enabled for SELinux 0xff...0e47 <+23>: xchg %ax,%ax ^^^^^^^^^^^^^^ Static key disabled for BPF. This gets patched when a BPF LSM program is attached 0xff...0e49 <+25>: xor %eax,%eax 0xff...0e4b <+27>: xchg %ax,%ax 0xff...0e4d <+29>: pop %rbx 0xff...0e4e <+30>: pop %r14 0xff...0e50 <+32>: pop %rbp 0xff...0e51 <+33>: cs jmp 0xff...0000 <__x86_return_thunk> 0xff...0e57 <+39>: endbr64 0xff...0e5b <+43>: mov %r14,%rdi 0xff...0e5e <+46>: mov %ebp,%esi 0xff...0e60 <+48>: mov %rbx,%rdx 0xff...0e63 <+51>: call 0xff...33c0 <selinux_file_ioctl> 0xff...0e68 <+56>: test %eax,%eax 0xff...0e6a <+58>: jne 0xff...0e4d <security_file_ioctl+29> 0xff...0e6c <+60>: jmp 0xff...0e47 <security_file_ioctl+23> 0xff...0e6e <+62>: endbr64 0xff...0e72 <+66>: mov %r14,%rdi 0xff...0e75 <+69>: mov %ebp,%esi 0xff...0e77 <+71>: mov %rbx,%rdx 0xff...0e7a <+74>: call 0xff...e3b0 <bpf_lsm_file_ioctl> 0xff...0e7f <+79>: test %eax,%eax 0xff...0e81 <+81>: jne 0xff...0e4d <security_file_ioctl+29> 0xff...0e83 <+83>: jmp 0xff...0e49 <security_file_ioctl+25> 0xff...0e85 <+85>: endbr64 0xff...0e89 <+89>: mov %r14,%rdi 0xff...0e8c <+92>: mov %ebp,%esi 0xff...0e8e <+94>: mov %rbx,%rdx 0xff...0e91 <+97>: pop %rbx 0xff...0e92 <+98>: pop %r14 0xff...0e94 <+100>: pop %rbp 0xff...0e95 <+101>: retThis patch enables this by providing a
LSM_HOOK_INIT_RUNTIMEvariant that allows the LSMs to opt-in to hooks which can be toggled at runtime which withsecurity_toogle_hook.Reviewed-by Kees Cook <[email protected]>Acked-by Casey Schaufler <[email protected]>Signed-off-by KP Singh <[email protected]>Quoted 6 lines
--- include/linux/lsm_hooks.h | 30 ++++++++++++++++++++++++++++- kernel/bpf/trampoline.c | 40 +++++++++++++++++++++++++++++++++++---- security/bpf/hooks.c | 2 +- security/security.c | 36 ++++++++++++++++++++++++++++++++++- 4 files changed, 101 insertions(+), 7 deletions(-)I didn't look at this one too closely, see my previous comments in patch 3/5, but I did catch one typo, see below ...
Quoted 12 lines
include/linux/lsm_hooks.h +2
-0@@ -110,11 +110,14 @@ struct lsm_id {* @scalls: The beginning of the array of static calls assigned to this hook. * @hook: The callback for the hook. * @lsm: The name of the lsm that owns this hook. +* @default_state: The state of the LSM hook when initialized. If set to false, +* the static key guarding the hook will be set to disabled. */ struct security_hook_list {
Quoted 8 lines
include/linux/lsm_hooks.h struct lsm_id { · line 121
Quoted 5 lines
include/linux/lsm_hooks.h +1
-0struct lsm_static_call *scalls;union security_list_options hook;const struct lsm_id *lsmid; +bool runtime; } __randomize_layout;The comment header doesn't match the struct fields, "
default_state" vs "runtime".
Good catch, apologies for the omission.
Quoted 3 lines
--
paul-moore.com
Paul Moore
maintainerRe: [PATCH v13 3/5] security: Replace indirect LSM hook calls with static calls
In reply to KP SinghQuoted 17 lines · KP Singh
Quoted 7 lines · Paul Moore
Quoted 5 lines · On Jun 29, 2024 KP Singh <[email protected]> wrote:
LSM hooks are currently invoked from a linked list as indirect calls which are invoked using retpolines as a mitigation for speculative attacks (Branch History / Target injection) and add extra overhead which is especially bad in kernel hot paths:
[...]
Quoted 5 lines
should fix the more obvious problems. I'd like to know if you are aware of any others? If not, the text above should be adjusted and we should reconsider patch 5/5. If there are other problems I'd like to better understand them as there may be an independent solution for those particular problems.
Quoted 7 lines
We did have problems with some other hooks but I was unable to dig up specific examples though, it's been a while. More broadly speaking, a default decision is still a decision. Whereas the intent from the BPF LSM is not to make a default decision unless a BPF program is loaded. I am quite worried about the holes this leaves open, subtle bugs (security or crashes) we have not caught yet and PATCH 5/5 engineers away
the problem of the "default decision".
The inode/xattr problem you originally mentioned wasn't really rooted in a "bad" default return value, it was really an issue with how the LSM hook was structured due to some legacy design assumptions made well before the initial stacking patches were merged. That should be fixed now[1] and given that the inode/xattr set/remove hooks were unique in this regard (individual LSMs were responsible for performing the capabilities checks) I don't expect this to be a general problem.
There were also some issues caused by the fact that we were defining the default return value in multiple places and these values had gone out of sync in a number of hooks. We've also fixed this problem by only defining the default return value once for each hook, solving all of those problems.
I'm not aware of any other existing problems relating to the LSM hook default values, if there are any, we need to fix them independent of this patchset. The LSM framework should function properly if the "default" values are used.
[1] I just realized that commit 61df7b828204 doesn't properly update the removexattr implementations for SELinux and Smack, expect a patch for that soon. The current code is okay, it just does some unnecessary work (see the setxattr changes to get an idea of the changes needed).
Quoted 8 lines
Quoted 7 lines
security/security.c int lsm_fill_user_ctx(struct lsm_ctx __user *uctx, u32 *uctx_len, · line 954
Quoted 4 lines
security/security.c +4
-0+#define lsm_for_each_hook (scall,NAME ) \ + for (scall = static_calls_table.NAME; \ + scall - static_calls_table.NAME < MAX_LSM_COUNT; scall++) \ + if (static_key_enabled(&scall->active->key))This is probably a stupid question, but why use
static_key_enabled()here instead ofstatic_branch_unlikely()as in thecall_XXX_macros?
Quoted 6 lines
The
static_key_enabledis a check for the key being enabled, whereas thestatic_branch_likelyis for laying out the right assembly code (jump tables etc.) and based on the value of the static key, here we are not using the static calls or even keys, rather we are following back from direct calls to indirect calls and don't really need any jump tables in the slow path.
Gotcha, thanks for the explanation.
Signature · 1 line
paul-moore.com
KP Singh
maintainerRe: [PATCH v13 3/5] security: Replace indirect LSM hook calls with static calls
Quoted 35 lines · Paul Moore
Quoted 17 lines · KP Singh
Quoted 7 lines · Paul Moore
Quoted 5 lines · On Jun 29, 2024 KP Singh <[email protected]> wrote:
LSM hooks are currently invoked from a linked list as indirect calls which are invoked using retpolines as a mitigation for speculative attacks (Branch History / Target injection) and add extra overhead which is especially bad in kernel hot paths:
[...]
Quoted 5 lines
should fix the more obvious problems. I'd like to know if you are aware of any others? If not, the text above should be adjusted and we should reconsider patch 5/5. If there are other problems I'd like to better understand them as there may be an independent solution for those particular problems.
Quoted 7 lines
We did have problems with some other hooks but I was unable to dig up specific examples though, it's been a while. More broadly speaking, a default decision is still a decision. Whereas the intent from the BPF LSM is not to make a default decision unless a BPF program is loaded. I am quite worried about the holes this leaves open, subtle bugs (security or crashes) we have not caught yet and PATCH 5/5 engineers away
the problem of the "default decision".The inode/xattr problem you originally mentioned wasn't really rooted in a "bad" default return value, it was really an issue with how the LSM hook was structured due to some legacy design assumptions made well before the initial stacking patches were merged. That should be fixed now[1] and given that the inode/xattr set/remove hooks were unique in this regard (individual LSMs were responsible for performing the capabilities checks) I don't expect this to be a general problem.
Quoted 5 lines
There were also some issues caused by the fact that we were defining the default return value in multiple places and these values had gone out of sync in a number of hooks. We've also fixed this problem by only defining the default return value once for each hook, solving all of those problems.
I don't see how this solves problems or prevents any future problems with side-effects. I have always been uncomfortable with an extraneous function being called with a side effect ever since we merged BPF LSM with default callback. We have found one bug due to this, not all the bugs.
Quoted 5 lines
I'm not aware of any other existing problems relating to the LSM hook default values, if there are any, we need to fix them independent of this patchset. The LSM framework should function properly if the "default" values are used.
Patch 5 eliminates the possibilities of errors and subtle bugs all together. The problem with subtle bugs is, well, they are subtle, if you and I knew of the bugs, we would fix all of them, but we don't. I really feel we ought to eliminate the class of issues and not just whack-a-mole when we see the bugs.
The LSM framework ought to function with default values is a nice goal, but a weaker position than what we have where we make these subtle bugs impossible. If you feel this should not be a part of the framework, I would still like to revert back to implementation where we kept the toggling logic contained to the BPF LSM.
- KP
Quoted 26 lines
[1] I just realized that commit
61df7b828204doesn't properly update the removexattr implementations for SELinux and Smack, expect a patch for that soon. The current code is okay, it just does some unnecessary work (see the setxattr changes to get an idea of the changes needed).Quoted 8 lines
Quoted 7 lines
security/security.c int lsm_fill_user_ctx(struct lsm_ctx __user *uctx, u32 *uctx_len, · line 954
Quoted 4 lines
security/security.c +4
-0+#define lsm_for_each_hook (scall,NAME ) \ + for (scall = static_calls_table.NAME; \ + scall - static_calls_table.NAME < MAX_LSM_COUNT; scall++) \ + if (static_key_enabled(&scall->active->key))This is probably a stupid question, but why use
static_key_enabled()here instead ofstatic_branch_unlikely()as in thecall_XXX_macros?Quoted 6 lines
The
static_key_enabledis a check for the key being enabled, whereas thestatic_branch_likelyis for laying out the right assembly code (jump tables etc.) and based on the value of the static key, here we are not using the static calls or even keys, rather we are following back from direct calls to indirect calls and don't really need any jump tables in the slow path.Gotcha, thanks for the explanation.
--
paul-moore.com
Paul Moore
maintainerRe: [PATCH v13 3/5] security: Replace indirect LSM hook calls with static calls
Quoted 41 lines · KP Singh
Quoted 34 lines · Paul Moore
Quoted 17 lines · KP Singh
Quoted 7 lines · Paul Moore
Quoted 5 lines · On Jun 29, 2024 KP Singh <[email protected]> wrote:
LSM hooks are currently invoked from a linked list as indirect calls which are invoked using retpolines as a mitigation for speculative attacks (Branch History / Target injection) and add extra overhead which is especially bad in kernel hot paths:
[...]
Quoted 5 lines
should fix the more obvious problems. I'd like to know if you are aware of any others? If not, the text above should be adjusted and we should reconsider patch 5/5. If there are other problems I'd like to better understand them as there may be an independent solution for those particular problems.
Quoted 7 lines
We did have problems with some other hooks but I was unable to dig up specific examples though, it's been a while. More broadly speaking, a default decision is still a decision. Whereas the intent from the BPF LSM is not to make a default decision unless a BPF program is loaded. I am quite worried about the holes this leaves open, subtle bugs (security or crashes) we have not caught yet and PATCH 5/5 engineers away
the problem of the "default decision".The inode/xattr problem you originally mentioned wasn't really rooted in a "bad" default return value, it was really an issue with how the LSM hook was structured due to some legacy design assumptions made well before the initial stacking patches were merged. That should be fixed now[1] and given that the inode/xattr set/remove hooks were unique in this regard (individual LSMs were responsible for performing the capabilities checks) I don't expect this to be a general problem.
Quoted 5 lines
There were also some issues caused by the fact that we were defining the default return value in multiple places and these values had gone out of sync in a number of hooks. We've also fixed this problem by only defining the default return value once for each hook, solving all of those problems.
Quoted 5 lines
I don't see how this solves problems or prevents any future problems with side-effects. I have always been uncomfortable with an extraneous function being called with a side effect ever since we merged BPF LSM with default callback. We have found one bug due to this, not all the bugs.
You've got to give me something more concrete than that. If you can't provide any concrete examples, start with providing a basic concept with far more detail than just "side-effects".
Quoted 10 lines
Quoted 4 lines
I'm not aware of any other existing problems relating to the LSM hook default values, if there are any, we need to fix them independent of this patchset. The LSM framework should function properly if the "default" values are used.
Patch 5 eliminates the possibilities of errors and subtle bugs all together. The problem with subtle bugs is, well, they are subtle, if you and I knew of the bugs, we would fix all of them, but we don't. I really feel we ought to eliminate the class of issues and not just whack-a-mole when we see the bugs.
Here's the thing, I don't really like patch 5/5. To be honest, I don't really like a lot of this patchset. From my perspective, the complexity of the code is likely going to mean more maintenance headaches down the road, but Linus hath spoken so we're doing this (although "this" is still a bit undefined as far as I'm concerned). If you want me to merge patch 5/5 you've got to give me something real and convincing that can't be fixed by any other means. My current opinion is that you're trying to use a previously fixed bug to scare and/or coerce the merging of some changes I don't really want to merge. If you want me to take patch 5/5, you've got to give me a reason that is far more compelling that what you've written thus far.
Signature · 1 line
paul-moore.com
KP Singh
maintainerRe: [PATCH v13 3/5] security: Replace indirect LSM hook calls with static calls
Quoted 64 lines · Paul Moore
Quoted 41 lines · KP Singh
Quoted 34 lines · Paul Moore
Quoted 17 lines · KP Singh
Quoted 7 lines · Paul Moore
Quoted 5 lines · On Jun 29, 2024 KP Singh <[email protected]> wrote:
LSM hooks are currently invoked from a linked list as indirect calls which are invoked using retpolines as a mitigation for speculative attacks (Branch History / Target injection) and add extra overhead which is especially bad in kernel hot paths:
[...]
Quoted 5 lines
should fix the more obvious problems. I'd like to know if you are aware of any others? If not, the text above should be adjusted and we should reconsider patch 5/5. If there are other problems I'd like to better understand them as there may be an independent solution for those particular problems.
Quoted 7 lines
We did have problems with some other hooks but I was unable to dig up specific examples though, it's been a while. More broadly speaking, a default decision is still a decision. Whereas the intent from the BPF LSM is not to make a default decision unless a BPF program is loaded. I am quite worried about the holes this leaves open, subtle bugs (security or crashes) we have not caught yet and PATCH 5/5 engineers away
the problem of the "default decision".The inode/xattr problem you originally mentioned wasn't really rooted in a "bad" default return value, it was really an issue with how the LSM hook was structured due to some legacy design assumptions made well before the initial stacking patches were merged. That should be fixed now[1] and given that the inode/xattr set/remove hooks were unique in this regard (individual LSMs were responsible for performing the capabilities checks) I don't expect this to be a general problem.
Quoted 5 lines
There were also some issues caused by the fact that we were defining the default return value in multiple places and these values had gone out of sync in a number of hooks. We've also fixed this problem by only defining the default return value once for each hook, solving all of those problems.
Quoted 5 lines
I don't see how this solves problems or prevents any future problems with side-effects. I have always been uncomfortable with an extraneous function being called with a side effect ever since we merged BPF LSM with default callback. We have found one bug due to this, not all the bugs.
You've got to give me something more concrete than that. If you can't provide any concrete examples, start with providing a basic concept with far more detail than just "side-effects".
Quoted 10 lines
Quoted 4 lines
I'm not aware of any other existing problems relating to the LSM hook default values, if there are any, we need to fix them independent of this patchset. The LSM framework should function properly if the "default" values are used.
Patch 5 eliminates the possibilities of errors and subtle bugs all together. The problem with subtle bugs is, well, they are subtle, if you and I knew of the bugs, we would fix all of them, but we don't. I really feel we ought to eliminate the class of issues and not just whack-a-mole when we see the bugs.
Quoted 11 lines
Here's the thing, I don't really like patch 5/5. To be honest, I don't really like a lot of this patchset. From my perspective, the complexity of the code is likely going to mean more maintenance headaches down the road, but Linus hath spoken so we're doing this (although "this" is still a bit undefined as far as I'm concerned). If you want me to merge patch 5/5 you've got to give me something real and convincing that can't be fixed by any other means. My current opinion is that you're trying to use a previously fixed bug to scare and/or coerce the merging of some changes I don't really want to merge. If you want me to take patch 5/5, you've got to give me a reason that is far more compelling that what you've written thus far.
Paul, I am not scaring you, I am providing a solution that saves us from headaches with side-effects and bugs in the future. It's safer by design.
You say you have not reviewed it carefully, but you did ask me to move the function from the BPF LSM layer to an LSM API, and we had a bunch of discussion around naming in the subsequent revisions.
https://lore.kernel.org/bpf/[email protected]/
My reasons are:
1. It's safer, no side effects, guaranteed to be not buggy. Neither you, nor me, can guarantee that a default value will be safe in the LSM layer. I request others (Casey, Kees) for their opinion here too.
2. Performance, no extra function call.
If you still don't like it, it's your call, I would still like to keep most of the logic local to the BPF LSM as proposed in
https://lore.kernel.org/bpf/[email protected]/
- KP
Quoted 3 lines
--
paul-moore.com
Re: [PATCH v13 3/5] security: Replace indirect LSM hook calls with static calls
Quoted 74 lines · KP Singh
Quoted 54 lines · Paul Moore
Quoted 35 lines · KP Singh
Quoted 29 lines · Paul Moore
Quoted 13 lines · KP Singh
Quoted 5 lines · Paul Moore
Quoted 4 lines · On Jun 29, 2024 KP Singh <[email protected]> wrote:
LSM hooks are currently invoked from a linked list as indirect calls which are invoked using retpolines as a mitigation for speculative attacks (Branch History / Target injection) and add extra overhead which is especially bad in kernel hot paths:
[...]
Quoted 5 lines
should fix the more obvious problems. I'd like to know if you are aware of any others? If not, the text above should be adjusted and we should reconsider patch 5/5. If there are other problems I'd like to better understand them as there may be an independent solution for those particular problems.
Quoted 7 lines
We did have problems with some other hooks but I was unable to dig up specific examples though, it's been a while. More broadly speaking, a default decision is still a decision. Whereas the intent from the BPF LSM is not to make a default decision unless a BPF program is loaded. I am quite worried about the holes this leaves open, subtle bugs (security or crashes) we have not caught yet and PATCH 5/5 engineers away
the problem of the "default decision".The inode/xattr problem you originally mentioned wasn't really rooted in a "bad" default return value, it was really an issue with how the LSM hook was structured due to some legacy design assumptions made well before the initial stacking patches were merged. That should be fixed now[1] and given that the inode/xattr set/remove hooks were unique in this regard (individual LSMs were responsible for performing the capabilities checks) I don't expect this to be a general problem.
Quoted 5 lines
There were also some issues caused by the fact that we were defining the default return value in multiple places and these values had gone out of sync in a number of hooks. We've also fixed this problem by only defining the default return value once for each hook, solving all of those problems.
Quoted 5 lines
I don't see how this solves problems or prevents any future problems with side-effects. I have always been uncomfortable with an extraneous function being called with a side effect ever since we merged BPF LSM with default callback. We have found one bug due to this, not all the bugs.
You've got to give me something more concrete than that. If you can't provide any concrete examples, start with providing a basic concept with far more detail than just "side-effects".
Quoted 9 lines
Quoted 4 lines
I'm not aware of any other existing problems relating to the LSM hook default values, if there are any, we need to fix them independent of this patchset. The LSM framework should function properly if the "default" values are used.
Patch 5 eliminates the possibilities of errors and subtle bugs all together. The problem with subtle bugs is, well, they are subtle, if you and I knew of the bugs, we would fix all of them, but we don't. I really feel we ought to eliminate the class of issues and not just whack-a-mole when we see the bugs.
Quoted 11 lines
Here's the thing, I don't really like patch 5/5. To be honest, I don't really like a lot of this patchset. From my perspective, the complexity of the code is likely going to mean more maintenance headaches down the road, but Linus hath spoken so we're doing this (although "this" is still a bit undefined as far as I'm concerned). If you want me to merge patch 5/5 you've got to give me something real and convincing that can't be fixed by any other means. My current opinion is that you're trying to use a previously fixed bug to scare and/or coerce the merging of some changes I don't really want to merge. If you want me to take patch 5/5, you've got to give me a reason that is far more compelling that what you've written thus far.
Paul, I am not scaring you, I am providing a solution that saves us from headaches with side-effects and bugs in the future. It's safer by design.
You say you have not reviewed it carefully, but you did ask me to move the function from the BPF LSM layer to an LSM API, and we had a bunch of discussion around naming in the subsequent revisions.
Quoted 8 lines
https://lore.kernel.org/bpf/[email protected]/
My reasons are:
1. It's safer, no side effects, guaranteed to be not buggy. Neither you, nor me, can guarantee that a default value will be safe in the LSM layer. I request others (Casey, Kees) for their opinion here too.
2. Performance, no extra function call.
I want to be very careful about the comments I make on this patch set. I can't say that I trust any fix for the BPF LSM layer. My natural inclination is to isolate the fix to the area that has the problem, that is, BPF. I have a hard time accepting the notion that the implementation will really fix all possible bugs in the future. The pace at which eBPF is evolving gives me the heebee geebees when I think of it as a mechanism for implementing security modules.
My biggest concern is that we may be trying too hard for perfection. I see a situation where we're not moving forward because there are two reasonable solutions and rather than running with either we're desperately looking for a compelling reason to pick one over the other.
Quoted 9 lines
If you still don't like it, it's your call, I would still like to keep most of the logic local to the BPF LSM as proposed in
https://lore.kernel.org/bpf/[email protected]/- KP
Quoted 2 lines
--
paul-moore.com
KP Singh
maintainerRe: [PATCH v13 3/5] security: Replace indirect LSM hook calls with static calls
Quoted 93 lines · Casey Schaufler
Quoted 74 lines · KP Singh
Quoted 54 lines · Paul Moore
Quoted 35 lines · KP Singh
Quoted 29 lines · Paul Moore
Quoted 13 lines · KP Singh
Quoted 5 lines · Paul Moore
Quoted 4 lines · On Jun 29, 2024 KP Singh <[email protected]> wrote:
LSM hooks are currently invoked from a linked list as indirect calls which are invoked using retpolines as a mitigation for speculative attacks (Branch History / Target injection) and add extra overhead which is especially bad in kernel hot paths:
[...]
Quoted 5 lines
should fix the more obvious problems. I'd like to know if you are aware of any others? If not, the text above should be adjusted and we should reconsider patch 5/5. If there are other problems I'd like to better understand them as there may be an independent solution for those particular problems.
Quoted 7 lines
We did have problems with some other hooks but I was unable to dig up specific examples though, it's been a while. More broadly speaking, a default decision is still a decision. Whereas the intent from the BPF LSM is not to make a default decision unless a BPF program is loaded. I am quite worried about the holes this leaves open, subtle bugs (security or crashes) we have not caught yet and PATCH 5/5 engineers away
the problem of the "default decision".The inode/xattr problem you originally mentioned wasn't really rooted in a "bad" default return value, it was really an issue with how the LSM hook was structured due to some legacy design assumptions made well before the initial stacking patches were merged. That should be fixed now[1] and given that the inode/xattr set/remove hooks were unique in this regard (individual LSMs were responsible for performing the capabilities checks) I don't expect this to be a general problem.
Quoted 5 lines
There were also some issues caused by the fact that we were defining the default return value in multiple places and these values had gone out of sync in a number of hooks. We've also fixed this problem by only defining the default return value once for each hook, solving all of those problems.
Quoted 5 lines
I don't see how this solves problems or prevents any future problems with side-effects. I have always been uncomfortable with an extraneous function being called with a side effect ever since we merged BPF LSM with default callback. We have found one bug due to this, not all the bugs.
You've got to give me something more concrete than that. If you can't provide any concrete examples, start with providing a basic concept with far more detail than just "side-effects".
Quoted 9 lines
Quoted 4 lines
I'm not aware of any other existing problems relating to the LSM hook default values, if there are any, we need to fix them independent of this patchset. The LSM framework should function properly if the "default" values are used.
Patch 5 eliminates the possibilities of errors and subtle bugs all together. The problem with subtle bugs is, well, they are subtle, if you and I knew of the bugs, we would fix all of them, but we don't. I really feel we ought to eliminate the class of issues and not just whack-a-mole when we see the bugs.
Quoted 11 lines
Here's the thing, I don't really like patch 5/5. To be honest, I don't really like a lot of this patchset. From my perspective, the complexity of the code is likely going to mean more maintenance headaches down the road, but Linus hath spoken so we're doing this (although "this" is still a bit undefined as far as I'm concerned). If you want me to merge patch 5/5 you've got to give me something real and convincing that can't be fixed by any other means. My current opinion is that you're trying to use a previously fixed bug to scare and/or coerce the merging of some changes I don't really want to merge. If you want me to take patch 5/5, you've got to give me a reason that is far more compelling that what you've written thus far.
Paul, I am not scaring you, I am providing a solution that saves us from headaches with side-effects and bugs in the future. It's safer by design.
You say you have not reviewed it carefully, but you did ask me to move the function from the BPF LSM layer to an LSM API, and we had a bunch of discussion around naming in the subsequent revisions.
Quoted 8 lines
https://lore.kernel.org/bpf/[email protected]/
My reasons are:
1. It's safer, no side effects, guaranteed to be not buggy. Neither you, nor me, can guarantee that a default value will be safe in the LSM layer. I request others (Casey, Kees) for their opinion here too.
2. Performance, no extra function call.I want to be very careful about the comments I make on this patch set. I can't say that I trust any fix for the BPF LSM layer. My natural inclination is to isolate the fix to the area that has the problem, that is, BPF. I have a hard time accepting the notion that the implementation will really fix all possible bugs in the future. The pace at which eBPF is evolving gives me the heebee geebees when I think of it as a mechanism for implementing security modules.
Quoted 4 lines
My biggest concern is that we may be trying too hard for perfection. I see a situation where we're not moving forward because there are two reasonable solutions and rather than running with either we're desperately looking for a compelling reason to pick one over the other.
I am fine with either, if you folks prefer security_toggle_hook to be in BPF only / limited to BPF. I can revert back to what we had in v9, the changes to the LSM layer are then very minimal.
- KP
Quoted 10 lines
Quoted 10 lines
If you still don't like it, it's your call, I would still like to keep most of the logic local to the BPF LSM as proposed in
https://lore.kernel.org/bpf/[email protected]/- KP
Quoted 2 lines
--
paul-moore.com
KP Singh
maintainerRe: [PATCH v13 3/5] security: Replace indirect LSM hook calls with static calls
Quoted 95 lines · KP Singh
Quoted 93 lines · Casey Schaufler
Quoted 74 lines · KP Singh
Quoted 54 lines · Paul Moore
Quoted 35 lines · KP Singh
Quoted 29 lines · Paul Moore
Quoted 13 lines · KP Singh
Quoted 5 lines · Paul Moore
Quoted 4 lines · On Jun 29, 2024 KP Singh <[email protected]> wrote:
LSM hooks are currently invoked from a linked list as indirect calls which are invoked using retpolines as a mitigation for speculative attacks (Branch History / Target injection) and add extra overhead which is especially bad in kernel hot paths:
[...]
Quoted 5 lines
should fix the more obvious problems. I'd like to know if you are aware of any others? If not, the text above should be adjusted and we should reconsider patch 5/5. If there are other problems I'd like to better understand them as there may be an independent solution for those particular problems.
Quoted 7 lines
We did have problems with some other hooks but I was unable to dig up specific examples though, it's been a while. More broadly speaking, a default decision is still a decision. Whereas the intent from the BPF LSM is not to make a default decision unless a BPF program is loaded. I am quite worried about the holes this leaves open, subtle bugs (security or crashes) we have not caught yet and PATCH 5/5 engineers away
the problem of the "default decision".The inode/xattr problem you originally mentioned wasn't really rooted in a "bad" default return value, it was really an issue with how the LSM hook was structured due to some legacy design assumptions made well before the initial stacking patches were merged. That should be fixed now[1] and given that the inode/xattr set/remove hooks were unique in this regard (individual LSMs were responsible for performing the capabilities checks) I don't expect this to be a general problem.
Quoted 5 lines
There were also some issues caused by the fact that we were defining the default return value in multiple places and these values had gone out of sync in a number of hooks. We've also fixed this problem by only defining the default return value once for each hook, solving all of those problems.
Quoted 5 lines
I don't see how this solves problems or prevents any future problems with side-effects. I have always been uncomfortable with an extraneous function being called with a side effect ever since we merged BPF LSM with default callback. We have found one bug due to this, not all the bugs.
You've got to give me something more concrete than that. If you can't provide any concrete examples, start with providing a basic concept with far more detail than just "side-effects".
Quoted 9 lines
Quoted 4 lines
I'm not aware of any other existing problems relating to the LSM hook default values, if there are any, we need to fix them independent of this patchset. The LSM framework should function properly if the "default" values are used.
Patch 5 eliminates the possibilities of errors and subtle bugs all together. The problem with subtle bugs is, well, they are subtle, if you and I knew of the bugs, we would fix all of them, but we don't. I really feel we ought to eliminate the class of issues and not just whack-a-mole when we see the bugs.
Quoted 11 lines
Here's the thing, I don't really like patch 5/5. To be honest, I don't really like a lot of this patchset. From my perspective, the complexity of the code is likely going to mean more maintenance headaches down the road, but Linus hath spoken so we're doing this (although "this" is still a bit undefined as far as I'm concerned). If you want me to merge patch 5/5 you've got to give me something real and convincing that can't be fixed by any other means. My current opinion is that you're trying to use a previously fixed bug to scare and/or coerce the merging of some changes I don't really want to merge. If you want me to take patch 5/5, you've got to give me a reason that is far more compelling that what you've written thus far.
Paul, I am not scaring you, I am providing a solution that saves us from headaches with side-effects and bugs in the future. It's safer by design.
You say you have not reviewed it carefully, but you did ask me to move the function from the BPF LSM layer to an LSM API, and we had a bunch of discussion around naming in the subsequent revisions.
Quoted 8 lines
https://lore.kernel.org/bpf/[email protected]/
My reasons are:
1. It's safer, no side effects, guaranteed to be not buggy. Neither you, nor me, can guarantee that a default value will be safe in the LSM layer. I request others (Casey, Kees) for their opinion here too.
2. Performance, no extra function call.I want to be very careful about the comments I make on this patch set. I can't say that I trust any fix for the BPF LSM layer. My natural inclination is to isolate the fix to the area that has the problem, that is, BPF. I have a hard time accepting the notion that the implementation will really fix all possible bugs in the future. The pace at which eBPF is evolving gives me the heebee geebees when I think of it as a mechanism for implementing security modules.
Quoted 8 lines
Quoted 4 lines
My biggest concern is that we may be trying too hard for perfection. I see a situation where we're not moving forward because there are two reasonable solutions and rather than running with either we're desperately looking for a compelling reason to pick one over the other.
I am fine with either, if you folks prefer
security_toggle_hookto be in BPF only / limited to BPF. I can revert back to what we had in v9, the changes to the LSM layer are then very minimal.
Now if you folks really don't want any changes to the core LSM layer, that too is doable, the patch below accomplishes that.
So, here's where we are at, while the LSM framework is comfortable with saying that default values and empty callbacks are fine, that's not what BPF LSM is comfortable doing. Your concerns around changes to the LSM layer should be addressed by the propose patch below:
include/linux/bpf_lsm.h +5 -0
@@ -29,6 +29,7 @@ int bpf_lsm_verify_prog(struct bpf_verifier_log *vlog,bool bpf_lsm_is_sleepable_hook (u32 btf_id);bool bpf_lsm_is_trusted (const struct bpf_prog *prog); +void bpf_lsm_toggle_hook (void *addr,bool value);static inline struct bpf_storage_blob *bpf_inode (const struct inode *inode)
@@ -52,6 +53,10 @@ static inline bool bpf_lsm_is_sleepable_hook(u32 btf_id)return false; } +static inline void bpf_lsm_toggle_hook (void *addr,bool value) +{ +} +static inline bool bpf_lsm_is_trusted (const struct bpf_prog *prog) {return false;
kernel/bpf/trampoline.c +0 -0
@@ -523,6 +523,22 @@ static enum bpf_tramp_prog_type
bpf_attach_type_to_tramp(struct bpf_prog *prog)
}
}
+static void bpf_trampoline_toggle_lsm(struct bpf_trampoline *tr,
+ enum bpf_tramp_prog_type kind)
+{
+ struct bpf_tramp_link *link;
+ bool found = false;
+
+ hlist_for_each_entry(link, &tr->progs_hlist[kind], tramp_hlist) {
+ if (link->link.prog->type == BPF_PROG_TYPE_LSM) {
+ found = true;
+ break;
+ }
+ }
+
+ bpf_lsm_toggle_hook(tr->func.addr, found);
+}
+
static int __bpf_trampoline_link_prog(struct bpf_tramp_link *link,
struct bpf_trampoline *tr)
{
enum bpf_tramp_prog_type kind;@@ -562,6 +578,10 @@ static int __bpf_trampoline_link_prog(struct bpf_tramp_link *link, struct bpf_tr
hlist_add_head(&link->tramp_hlist, &tr->progs_hlist[kind]);
tr->progs_cnt[kind]++;+
+ if (link->link.prog->type == BPF_PROG_TYPE_LSM)
+ bpf_trampoline_toggle_lsm(tr, kind);
+
err = bpf_trampoline_update(tr, true /* lock_direct_mutex */);
if (err) {
hlist_del_init(&link->tramp_hlist);
@@ -595,6 +615,10 @@ static int __bpf_trampoline_unlink_prog(struct
bpf_tramp_link *link, struct bpf_
}
hlist_del_init(&link->tramp_hlist);
tr->progs_cnt[kind]--;+
+ if (link->link.prog->type == BPF_PROG_TYPE_LSM)
+ bpf_trampoline_toggle_lsm(tr, kind);
+
return bpf_trampoline_update(tr, true /* lock_direct_mutex */);
}
security/bpf/hooks.c +0 -0
@@ -16,6 +16,29 @@ static struct security_hook_list bpf_lsm_hooks[]
__ro_after_init = {
LSM_HOOK_INIT(task_free, bpf_task_storage_free),
};
+void bpf_lsm_toggle_hook(void *addr, bool enable) +{
+ struct lsm_static_call *scalls; + struct security_hook_list *h; + int i, j;
+
+ for (i = 0; i < ARRAY_SIZE(bpf_lsm_hooks); i++) {
+ h = &bpf_lsm_hooks[i];
+ if (h->hook.lsm_func_addr != addr)
+ continue;
+
+ for (j = 0; j < MAX_LSM_COUNT; j++) {
+ scalls = &h->scalls[j];
+ if (scalls->hl != &bpf_lsm_hooks[i])
+ continue;
+ if (enable)
+ static_branch_enable(scalls->active);
+ else
+ static_branch_disable(scalls->active);
+ }
+ }
+}
+
static const struct lsm_id bpf_lsmid = {
.name = "bpf",
.id = LSM_ID_BPF,@@ -23,8 +46,14 @@ static const struct lsm_id bpf_lsmid = {
static int __init bpf_lsm_init(void)
{
+ int i;
+
security_add_hooks(bpf_lsm_hooks, ARRAY_SIZE(bpf_lsm_hooks),
&bpf_lsmid);+
+ for (i = 0; i < ARRAY_SIZE(bpf_lsm_hooks); i++)
+ bpf_lsm_toggle_hook(bpf_lsm_hooks[i].hook.lsm_func_addr, false);
+
pr_info("LSM support for eBPF active\n");
return 0;}
- KP
Quoted 13 lines
- KP
Quoted 10 lines
Quoted 10 lines
If you still don't like it, it's your call, I would still like to keep most of the logic local to the BPF LSM as proposed in
https://lore.kernel.org/bpf/[email protected]/- KP
Quoted 2 lines
--
paul-moore.com
Paul Moore
maintainerRe: [PATCH v13 3/5] security: Replace indirect LSM hook calls with static calls
In reply to KP SinghQuoted 11 lines · KP Singh
Quoted 10 lines · Paul Moore
Quoted 9 lines · KP Singh
Quoted 8 lines · Paul Moore
Quoted 7 lines · KP Singh
Quoted 6 lines · Paul Moore
Quoted 5 lines · On Jun 29, 2024 KP Singh <[email protected]> wrote:
LSM hooks are currently invoked from a linked list as indirect calls which are invoked using retpolines as a mitigation for speculative attacks (Branch History / Target injection) and add extra overhead which is especially bad in kernel hot paths:
...
Quoted 23 lines
Quoted 11 lines
Quoted 10 lines
Quoted 4 lines
I'm not aware of any other existing problems relating to the LSM hook default values, if there are any, we need to fix them independent of this patchset. The LSM framework should function properly if the "default" values are used.
Patch 5 eliminates the possibilities of errors and subtle bugs all together. The problem with subtle bugs is, well, they are subtle, if you and I knew of the bugs, we would fix all of them, but we don't. I really feel we ought to eliminate the class of issues and not just whack-a-mole when we see the bugs.
Quoted 11 lines
Here's the thing, I don't really like patch 5/5. To be honest, I don't really like a lot of this patchset. From my perspective, the complexity of the code is likely going to mean more maintenance headaches down the road, but Linus hath spoken so we're doing this (although "this" is still a bit undefined as far as I'm concerned). If you want me to merge patch 5/5 you've got to give me something real and convincing that can't be fixed by any other means. My current opinion is that you're trying to use a previously fixed bug to scare and/or coerce the merging of some changes I don't really want to merge. If you want me to take patch 5/5, you've got to give me a reason that is far more compelling that what you've written thus far.
Quoted 3 lines
Paul, I am not scaring you, I am providing a solution that saves us from headaches with side-effects and bugs in the future. It's safer by design.
Perhaps I wasn't clear enough in my previous emails; instead of trying to convince me that your solution is literally the best possible thing to ever touch the kernel, convince me that there is a problem we need to fix. Right now, I'm not convinced there is a bug that requires all of the extra code in patch 5/5 (all of which have the potential to introduce new bugs). As mentioned previously, the bugs that typically have been used as examples of unwanted side effects with the LSM hooks have been resolved, both in the specific and general case. If you want me to add more code/functionality to fix a bug, you must first demonstrate the bug exists and the risk is real; you have not done that as far as I'm concerned.
Quoted 1 line
You say you have not reviewed it carefully ...
That may have been true of previous versions of this patchset, but I did not say that about this current patchset.
Quoted 5 lines
... but you did ask me to move
the function from the BPF LSM layer to an LSM API, and we had a bunch of discussion around naming in the subsequent revisions.
That discussion predates commit 61df7b828204 ("lsm: fixup the inode xattr capability handling") which is currently in the lsm/dev branch, marked for stable, and will go up to Linus during the upcoming merge window.
Quoted 5 lines
My reasons are:
1. It's safer, no side effects, guaranteed to be not buggy. Neither you, nor me, can guarantee that a default value will be safe in the LSM layer.
In the first sentence above you "guarantee" that your code is not buggy and then follow that up with a second sentence discussing how no one can guarantee source code safety. Regardless of whatever point you were trying to make here, I maintain that *all* patches have the potential for bugs, even those that are attempting to fix bugs. With that in mind, if you want me to merge more code to fix a bug (class), a bug that I've mentioned several times now that I believe we've already fixed, you first MUST convince me that the bug (class) still exists. You have not done that.
Quoted 1 line
2. Performance, no extra function call.
Convince me the bug still exists first and then we can discuss the merits of whatever solutions are proposed.
Signature · 1 line
paul-moore.com
KP Singh
maintainerRe: [PATCH v13 3/5] security: Replace indirect LSM hook calls with static calls
Quoted 77 lines · Paul Moore
Quoted 11 lines · KP Singh
Quoted 10 lines · Paul Moore
Quoted 9 lines · KP Singh
Quoted 8 lines · Paul Moore
Quoted 7 lines · KP Singh
Quoted 6 lines · Paul Moore
Quoted 5 lines · On Jun 29, 2024 KP Singh <[email protected]> wrote:
LSM hooks are currently invoked from a linked list as indirect calls which are invoked using retpolines as a mitigation for speculative attacks (Branch History / Target injection) and add extra overhead which is especially bad in kernel hot paths:
...
Quoted 23 lines
Quoted 11 lines
Quoted 10 lines
Quoted 4 lines
I'm not aware of any other existing problems relating to the LSM hook default values, if there are any, we need to fix them independent of this patchset. The LSM framework should function properly if the "default" values are used.
Patch 5 eliminates the possibilities of errors and subtle bugs all together. The problem with subtle bugs is, well, they are subtle, if you and I knew of the bugs, we would fix all of them, but we don't. I really feel we ought to eliminate the class of issues and not just whack-a-mole when we see the bugs.
Quoted 11 lines
Here's the thing, I don't really like patch 5/5. To be honest, I don't really like a lot of this patchset. From my perspective, the complexity of the code is likely going to mean more maintenance headaches down the road, but Linus hath spoken so we're doing this (although "this" is still a bit undefined as far as I'm concerned). If you want me to merge patch 5/5 you've got to give me something real and convincing that can't be fixed by any other means. My current opinion is that you're trying to use a previously fixed bug to scare and/or coerce the merging of some changes I don't really want to merge. If you want me to take patch 5/5, you've got to give me a reason that is far more compelling that what you've written thus far.
Quoted 3 lines
Paul, I am not scaring you, I am providing a solution that saves us from headaches with side-effects and bugs in the future. It's safer by design.
Perhaps I wasn't clear enough in my previous emails; instead of trying to convince me that your solution is literally the best possible thing to ever touch the kernel, convince me that there is a problem we need to fix. Right now, I'm not convinced there is a bug that requires all of the extra code in patch 5/5 (all of which have the potential to introduce new bugs). As mentioned previously, the bugs that typically have been used as examples of unwanted side effects with the LSM hooks have been resolved, both in the specific and general case. If you want me to add more code/functionality to fix a bug, you must first demonstrate the bug exists and the risk is real; you have not done that as far as I'm concerned.
Quoted 1 line
You say you have not reviewed it carefully ...
That may have been true of previous versions of this patchset, but I did not say that about this current patchset.
Quoted 5 lines
... but you did ask me to move
the function from the BPF LSM layer to an LSM API, and we had a bunch of discussion around naming in the subsequent revisions.That discussion predates commit
61df7b828204("lsm: fixup the inode xattr capability handling") which is currently in the lsm/dev branch, marked for stable, and will go up to Linus during the upcoming merge window.Quoted 5 lines
My reasons are:
1. It's safer, no side effects, guaranteed to be not buggy. Neither you, nor me, can guarantee that a default value will be safe in the LSM layer.
Quoted 10 lines
In the first sentence above you "guarantee" that your code is not buggy and then follow that up with a second sentence discussing how no one can guarantee source code safety. Regardless of whatever point you were trying to make here, I maintain that *all* patches have the potential for bugs, even those that are attempting to fix bugs. WithD that in mind, if you want me to merge more code to fix a bug (class), a bug that I've mentioned several times now that I believe we've already fixed, you first MUST convince me that the bug (class) still exists. You have not done that.
Paul, I am talking about eliminating a class of bugs, but you don't seem to get the point and you are fixated on the very instance of this bug class.
Quoted 4 lines
Quoted 1 line
2. Performance, no extra function call.
Convince me the bug still exists first and then we can discuss the merits of whatever solutions are proposed.
This is independent of the bug!
The extra function calls have performance overhead and as the BPF LSM maintainer I am not okay with these extraneous calls when I have a clear way of solving it.
As I said, If you don't want to modify the core LSM layer, it's okay, I still want to go with changes local to the BPF LSM, If you really don't agree with the changes local to the BPF LSM, we can have it go via the BPF tree and seek Linus' help to resolve the conflict.
include/linux/bpf_lsm.h +5 -0
@@ -29,6 +29,7 @@ int bpf_lsm_verify_prog(struct bpf_verifier_log *vlog,bool bpf_lsm_is_sleepable_hook (u32 btf_id);bool bpf_lsm_is_trusted (const struct bpf_prog *prog); +void bpf_lsm_toggle_hook (void *addr,bool value);static inline struct bpf_storage_blob *bpf_inode (const struct inode *inode)
@@ -52,6 +53,10 @@ static inline bool bpf_lsm_is_sleepable_hook(u32 btf_id)return false; } +static inline void bpf_lsm_toggle_hook (void *addr,bool value) +{ +} +static inline bool bpf_lsm_is_trusted (const struct bpf_prog *prog) {return false;
kernel/bpf/trampoline.c +0 -0
@@ -523,6 +523,22 @@ static enum bpf_tramp_prog_type
bpf_attach_type_to_tramp(struct bpf_prog *prog)
}
}
+static void bpf_trampoline_toggle_lsm(struct bpf_trampoline *tr,
+ enum bpf_tramp_prog_type kind)
+{
+ struct bpf_tramp_link *link;
+ bool found = false;
+
+ hlist_for_each_entry(link, &tr->progs_hlist[kind], tramp_hlist) {
+ if (link->link.prog->type == BPF_PROG_TYPE_LSM) {
+ found = true;
+ break;
+ }
+ }
+
+ bpf_lsm_toggle_hook(tr->func.addr, found);
+}
+
static int __bpf_trampoline_link_prog(struct bpf_tramp_link *link,
struct bpf_trampoline *tr)
{
enum bpf_tramp_prog_type kind;@@ -562,6 +578,10 @@ static int __bpf_trampoline_link_prog(struct bpf_tramp_link *link, struct bpf_tr
hlist_add_head(&link->tramp_hlist, &tr->progs_hlist[kind]);
tr->progs_cnt[kind]++;+
+ if (link->link.prog->type == BPF_PROG_TYPE_LSM)
+ bpf_trampoline_toggle_lsm(tr, kind);
+
err = bpf_trampoline_update(tr, true /* lock_direct_mutex */);
if (err) {
hlist_del_init(&link->tramp_hlist);
@@ -595,6 +615,10 @@ static int __bpf_trampoline_unlink_prog(struct
bpf_tramp_link *link, struct bpf_
}
hlist_del_init(&link->tramp_hlist);
tr->progs_cnt[kind]--;+
+ if (link->link.prog->type == BPF_PROG_TYPE_LSM)
+ bpf_trampoline_toggle_lsm(tr, kind);
+
return bpf_trampoline_update(tr, true /* lock_direct_mutex */);
}
security/bpf/hooks.c +0 -0
@@ -16,6 +16,29 @@ static struct security_hook_list bpf_lsm_hooks[]
__ro_after_init = {
LSM_HOOK_INIT(task_free, bpf_task_storage_free),
};
+void bpf_lsm_toggle_hook(void *addr, bool enable) +{
+ struct lsm_static_call *scalls; + struct security_hook_list *h; + int i, j;
+
+ for (i = 0; i < ARRAY_SIZE(bpf_lsm_hooks); i++) {
+ h = &bpf_lsm_hooks[i];
+ if (h->hook.lsm_func_addr != addr)
+ continue;
+
+ for (j = 0; j < MAX_LSM_COUNT; j++) {
+ scalls = &h->scalls[j];
+ if (scalls->hl != &bpf_lsm_hooks[i])
+ continue;
+ if (enable)
+ static_branch_enable(scalls->active);
+ else
+ static_branch_disable(scalls->active);
+ }
+ }
+}
+
static const struct lsm_id bpf_lsmid = {
.name = "bpf",
.id = LSM_ID_BPF,@@ -23,8 +46,14 @@ static const struct lsm_id bpf_lsmid = {
static int __init bpf_lsm_init(void)
{
+ int i;
+
security_add_hooks(bpf_lsm_hooks, ARRAY_SIZE(bpf_lsm_hooks),
&bpf_lsmid);+
+ for (i = 0; i < ARRAY_SIZE(bpf_lsm_hooks); i++)
+ bpf_lsm_toggle_hook(bpf_lsm_hooks[i].hook.lsm_func_addr, false);
+
pr_info("LSM support for eBPF active\n");
return 0;}
Quoted 3 lines
--
paul-moore.com
Re: [PATCH v13 3/5] security: Replace indirect LSM hook calls with static calls
Quoted 89 lines · KP Singh
Quoted 77 lines · Paul Moore
Quoted 11 lines · KP Singh
Quoted 10 lines · Paul Moore
Quoted 9 lines · KP Singh
Quoted 8 lines · Paul Moore
Quoted 7 lines · KP Singh
Quoted 6 lines · Paul Moore
Quoted 5 lines · On Jun 29, 2024 KP Singh <[email protected]> wrote:
LSM hooks are currently invoked from a linked list as indirect calls which are invoked using retpolines as a mitigation for speculative attacks (Branch History / Target injection) and add extra overhead which is especially bad in kernel hot paths:
...
Quoted 23 lines
Quoted 11 lines
Quoted 10 lines
Quoted 4 lines
I'm not aware of any other existing problems relating to the LSM hook default values, if there are any, we need to fix them independent of this patchset. The LSM framework should function properly if the "default" values are used.
Patch 5 eliminates the possibilities of errors and subtle bugs all together. The problem with subtle bugs is, well, they are subtle, if you and I knew of the bugs, we would fix all of them, but we don't. I really feel we ought to eliminate the class of issues and not just whack-a-mole when we see the bugs.
Quoted 11 lines
Here's the thing, I don't really like patch 5/5. To be honest, I don't really like a lot of this patchset. From my perspective, the complexity of the code is likely going to mean more maintenance headaches down the road, but Linus hath spoken so we're doing this (although "this" is still a bit undefined as far as I'm concerned). If you want me to merge patch 5/5 you've got to give me something real and convincing that can't be fixed by any other means. My current opinion is that you're trying to use a previously fixed bug to scare and/or coerce the merging of some changes I don't really want to merge. If you want me to take patch 5/5, you've got to give me a reason that is far more compelling that what you've written thus far.
Quoted 3 lines
Paul, I am not scaring you, I am providing a solution that saves us from headaches with side-effects and bugs in the future. It's safer by design.
Perhaps I wasn't clear enough in my previous emails; instead of trying to convince me that your solution is literally the best possible thing to ever touch the kernel, convince me that there is a problem we need to fix. Right now, I'm not convinced there is a bug that requires all of the extra code in patch 5/5 (all of which have the potential to introduce new bugs). As mentioned previously, the bugs that typically have been used as examples of unwanted side effects with the LSM hooks have been resolved, both in the specific and general case. If you want me to add more code/functionality to fix a bug, you must first demonstrate the bug exists and the risk is real; you have not done that as far as I'm concerned.
Quoted 1 line
You say you have not reviewed it carefully ...
That may have been true of previous versions of this patchset, but I did not say that about this current patchset.
Quoted 5 lines
... but you did ask me to move
the function from the BPF LSM layer to an LSM API, and we had a bunch of discussion around naming in the subsequent revisions.That discussion predates commit
61df7b828204("lsm: fixup the inode xattr capability handling") which is currently in the lsm/dev branch, marked for stable, and will go up to Linus during the upcoming merge window.Quoted 5 lines
My reasons are:
1. It's safer, no side effects, guaranteed to be not buggy. Neither you, nor me, can guarantee that a default value will be safe in the LSM layer.
Quoted 10 lines
In the first sentence above you "guarantee" that your code is not buggy and then follow that up with a second sentence discussing how no one can guarantee source code safety. Regardless of whatever point you were trying to make here, I maintain that *all* patches have the potential for bugs, even those that are attempting to fix bugs. WithD that in mind, if you want me to merge more code to fix a bug (class), a bug that I've mentioned several times now that I believe we've already fixed, you first MUST convince me that the bug (class) still exists. You have not done that.
Quoted 3 lines
Paul, I am talking about eliminating a class of bugs, but you don't seem to get the point and you are fixated on the very instance of this bug class.
Let's take this one step at a time. I think patches 1-4 are fine and stand alone and solve a specific problem without creating any new immediate problems.
After 1-4 is accepted, we can come back around to what patch 5 is trying to do, and work on whatever issues may remain at that time.
Signature · 1 line
Kees Cook
Paul Moore
maintainerRe: [PATCH v13 3/5] security: Replace indirect LSM hook calls with static calls
In reply to KP SinghQuoted 87 lines · KP Singh
Quoted 76 lines · Paul Moore
Quoted 11 lines · KP Singh
Quoted 10 lines · Paul Moore
Quoted 9 lines · KP Singh
Quoted 8 lines · Paul Moore
Quoted 7 lines · KP Singh
Quoted 6 lines · Paul Moore
Quoted 5 lines · On Jun 29, 2024 KP Singh <[email protected]> wrote:
LSM hooks are currently invoked from a linked list as indirect calls which are invoked using retpolines as a mitigation for speculative attacks (Branch History / Target injection) and add extra overhead which is especially bad in kernel hot paths:
...
Quoted 23 lines
Quoted 11 lines
Quoted 10 lines
Quoted 4 lines
I'm not aware of any other existing problems relating to the LSM hook default values, if there are any, we need to fix them independent of this patchset. The LSM framework should function properly if the "default" values are used.
Patch 5 eliminates the possibilities of errors and subtle bugs all together. The problem with subtle bugs is, well, they are subtle, if you and I knew of the bugs, we would fix all of them, but we don't. I really feel we ought to eliminate the class of issues and not just whack-a-mole when we see the bugs.
Quoted 11 lines
Here's the thing, I don't really like patch 5/5. To be honest, I don't really like a lot of this patchset. From my perspective, the complexity of the code is likely going to mean more maintenance headaches down the road, but Linus hath spoken so we're doing this (although "this" is still a bit undefined as far as I'm concerned). If you want me to merge patch 5/5 you've got to give me something real and convincing that can't be fixed by any other means. My current opinion is that you're trying to use a previously fixed bug to scare and/or coerce the merging of some changes I don't really want to merge. If you want me to take patch 5/5, you've got to give me a reason that is far more compelling that what you've written thus far.
Quoted 3 lines
Paul, I am not scaring you, I am providing a solution that saves us from headaches with side-effects and bugs in the future. It's safer by design.
Perhaps I wasn't clear enough in my previous emails; instead of trying to convince me that your solution is literally the best possible thing to ever touch the kernel, convince me that there is a problem we need to fix. Right now, I'm not convinced there is a bug that requires all of the extra code in patch 5/5 (all of which have the potential to introduce new bugs). As mentioned previously, the bugs that typically have been used as examples of unwanted side effects with the LSM hooks have been resolved, both in the specific and general case. If you want me to add more code/functionality to fix a bug, you must first demonstrate the bug exists and the risk is real; you have not done that as far as I'm concerned.
Quoted 1 line
You say you have not reviewed it carefully ...
That may have been true of previous versions of this patchset, but I did not say that about this current patchset.
Quoted 5 lines
... but you did ask me to move
the function from the BPF LSM layer to an LSM API, and we had a bunch of discussion around naming in the subsequent revisions.That discussion predates commit
61df7b828204("lsm: fixup the inode xattr capability handling") which is currently in the lsm/dev branch, marked for stable, and will go up to Linus during the upcoming merge window.Quoted 5 lines
My reasons are:
1. It's safer, no side effects, guaranteed to be not buggy. Neither you, nor me, can guarantee that a default value will be safe in the LSM layer.
Quoted 9 lines
In the first sentence above you "guarantee" that your code is not buggy and then follow that up with a second sentence discussing how no one can guarantee source code safety. Regardless of whatever point you were trying to make here, I maintain that *all* patches have the potential for bugs, even those that are attempting to fix bugs. WithD that in mind, if you want me to merge more code to fix a bug (class), a bug that I've mentioned several times now that I believe we've already fixed, you first MUST convince me that the bug (class) still exists. You have not done that.
Quoted 3 lines
Paul, I am talking about eliminating a class of bugs, but you don't seem to get the point and you are fixated on the very instance of this bug class.
I do understand that you are trying to eliminate a class of bugs, the point I'm trying to make is that I believe we have addressed that already with the patches I've previously cited.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
2. Performance, no extra function call.
Convince me the bug still exists first and then we can discuss the merits of whatever solutions are proposed.
This is independent of the bug!
Correctness first, maintainability second, performance third. That's my current priority and I feel the maintainability hit doesn't justify the performance win at this point in time. Besides, we're already expecting a big performance boost simply by moving to static_calls.
Quoted 4 lines
As I said, If you don't want to modify the core LSM layer, it's okay, I still want to go with changes local to the BPF LSM, If you really don't agree with the changes local to the BPF LSM, we can have it go via the BPF tree and seek Linus' help to resolve the conflict.
As the BPF maintainer you are always free to do whatever you like within the scope of the LSM you maintain so long as it does not touch or otherwise impact any of the other LSMs or the LSM framework. If you do affect the other LSMs, or the LSM framework, you need to get an ACK from the associated maintainer. That's pretty much how Linux kernel development works.
Signature · 1 line
paul-moore.com
Paul Moore
maintainerRe: [PATCH v13 3/5] security: Replace indirect LSM hook calls with static calls
In reply to Kees CookQuoted 7 lines · Kees Cook
Let's take this one step at a time. I think patches 1-4 are fine and stand alone and solve a specific problem without creating any new immediate problems.
After 1-4 is accepted, we can come back around to what patch 5 is trying to do, and work on whatever issues may remain at that time.
Kees, whatever technical review you may have for any proposed patches is always welcome, and if you want to provide process advice, solicited or otherwise, to people posting patches that is up to you. However, after your last comments regarding the management of the LSM tree, I want to make it clear that I'm not interested in your opinions on what should be merged into the LSM tree at this point in time.
Signature · 1 line
paul-moore.com
KP Singh
maintainerRe: [PATCH v13 3/5] security: Replace indirect LSM hook calls with static calls
In reply to Paul MooreQuoted 4 lines · Paul Moore
Quoted 2 lines · KP Singh
Quoted 1 line · Paul Moore
On Wed, Jul 3, 2024 at 7:08 PM KP Singh <[email protected]> wrote:
[...]
Quoted 8 lines
Quoted 4 lines
Paul, I am talking about eliminating a class of bugs, but you don't seem to get the point and you are fixated on the very instance of this bug class.
I do understand that you are trying to eliminate a class of bugs, the point I'm trying to make is that I believe we have addressed that already with the patches I've previously cited.
The class I am referring to is useless hooks returning a default value and imposing a denial / enforcement when they are not supposed to. If you think this class of issues is not relevant to the overall LSM, sure. I would still like BPF LSM to not add default callbacks as I have always maintained since day 1:
https://lwn.net/ml/linux-kernel/[email protected]/
BPF LSM does not want to provide a default decision until a BPF LSM policy program is loaded,
Quoted 18 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
2. Performance, no extra function call.
Convince me the bug still exists first and then we can discuss the merits of whatever solutions are proposed.
This is independent of the bug!
Correctness first, maintainability second, performance third. That's my current priority and I feel the maintainability hit doesn't justify the performance win at this point in time. Besides, we're already expecting a big performance boost simply by moving to
static_calls.Quoted 4 lines
As I said, If you don't want to modify the core LSM layer, it's okay, I still want to go with changes local to the BPF LSM, If you really don't agree with the changes local to the BPF LSM, we can have it go via the BPF tree and seek Linus' help to resolve the conflict.
Quoted 6 lines
As the BPF maintainer you are always free to do whatever you like within the scope of the LSM you maintain so long as it does not touch or otherwise impact any of the other LSMs or the LSM framework. If you do affect the other LSMs, or the LSM framework, you need to get an ACK from the associated maintainer. That's pretty much how Linux kernel development works.
Okay, then let's not make an LSM API, I will handle it within the BPF LSM.
The patch I proposed should not affect any other LSMs and is self contained within BPF LSM:
https://lore.kernel.org/bpf/CACYkzJ6jADoGNuPP3-1wkk-kV7NOQh+eFkU5KEDEZgq9qNNEfg@mail.gmail.com/
Quoted 3 lines
--
paul-moore.com
Paul Moore
maintainerRe: [PATCH v13 3/5] security: Replace indirect LSM hook calls with static calls
Quoted 12 lines · KP Singh
Quoted 3 lines · Paul Moore
Quoted 2 lines · KP Singh
Quoted 1 line · Paul Moore
On Wed, Jul 3, 2024 at 7:08 PM KP Singh <[email protected]> wrote:
[...]
Quoted 5 lines
Quoted 4 lines
Paul, I am talking about eliminating a class of bugs, but you don't seem to get the point and you are fixated on the very instance of this bug class.
Quoted 6 lines
Quoted 3 lines
I do understand that you are trying to eliminate a class of bugs, the point I'm trying to make is that I believe we have addressed that already with the patches I've previously cited.
The class I am referring to is useless hooks returning a default value and imposing a denial / enforcement when they are not supposed to.
If a LSM hook's default value were to result in an undesirable behavior within the kernel that would be an issue regardless of what LSMs were involved and we would either need to modify the hook and/or the default value. I am not convinced that we can adequately solve this entire class of problems simply by allowing one LSM, or even arbitrary combinations of LSMs, to disable or otherwise disconnect themselves from the framework.
Quoted 7 lines
Quoted 6 lines
As the BPF maintainer you are always free to do whatever you like within the scope of the LSM you maintain so long as it does not touch or otherwise impact any of the other LSMs or the LSM framework. If you do affect the other LSMs, or the LSM framework, you need to get an ACK from the associated maintainer. That's pretty much how Linux kernel development works.
Quoted 6 lines
Okay, then let's not make an LSM API, I will handle it within the BPF LSM.
The patch I proposed should not affect any other LSMs and is self contained within BPF LSM:
https://lore.kernel.org/bpf/CACYkzJ6jADoGNuPP3-1wkk-kV7NOQh+eFkU5KEDEZgq9qNNEfg@mail.gmail.com/
The code changes may be self-contained within the BPF LSM, but it does appear that the bpf_lsm_toggle_hook() function directly manipulates the LSM framework hook state which is not something we want to allow - none of the individual LSMs should be directly manipulating the LSM hook state/configuration. Although perhaps I'm missing or not factoring in some context around the patch linked above and that isn't the case?
While I had issues with Kees' comments, at a high level his suggestion of dropping the last patch and moving forward is something you should consider as I don't see this a good path forward with all of the approaches that have been discussed thus far.
Signature · 1 line
paul-moore.com
KP Singh
maintainerRe: [PATCH v13 3/5] security: Replace indirect LSM hook calls with static calls
Quoted 43 lines · Paul Moore
Quoted 12 lines · KP Singh
Quoted 3 lines · Paul Moore
Quoted 2 lines · KP Singh
Quoted 1 line · Paul Moore
On Wed, Jul 3, 2024 at 7:08 PM KP Singh <[email protected]> wrote:
[...]
Quoted 5 lines
Quoted 4 lines
Paul, I am talking about eliminating a class of bugs, but you don't seem to get the point and you are fixated on the very instance of this bug class.
Quoted 6 lines
Quoted 3 lines
I do understand that you are trying to eliminate a class of bugs, the point I'm trying to make is that I believe we have addressed that already with the patches I've previously cited.
The class I am referring to is useless hooks returning a default value and imposing a denial / enforcement when they are not supposed to.
If a LSM hook's default value were to result in an undesirable behavior within the kernel that would be an issue regardless of what LSMs were involved and we would either need to modify the hook and/or the default value. I am not convinced that we can adequately solve this entire class of problems simply by allowing one LSM, or even arbitrary combinations of LSMs, to disable or otherwise disconnect themselves from the framework.
Quoted 7 lines
Quoted 6 lines
As the BPF maintainer you are always free to do whatever you like within the scope of the LSM you maintain so long as it does not touch or otherwise impact any of the other LSMs or the LSM framework. If you do affect the other LSMs, or the LSM framework, you need to get an ACK from the associated maintainer. That's pretty much how Linux kernel development works.
Quoted 6 lines
Okay, then let's not make an LSM API, I will handle it within the BPF LSM.
The patch I proposed should not affect any other LSMs and is self contained within BPF LSM:
https://lore.kernel.org/bpf/CACYkzJ6jADoGNuPP3-1wkk-kV7NOQh+eFkU5KEDEZgq9qNNEfg@mail.gmail.com/
Quoted 7 lines
The code changes may be self-contained within the BPF LSM, but it does appear that the
bpf_lsm_toggle_hook()function directly manipulates the LSM framework hook state which is not something we want to allow - none of the individual LSMs should be directly manipulating the LSM hook state/configuration. Although perhaps I'm missing or not factoring in some context around the patch linked above and that isn't the case?
I think you are ignoring my point that BPF does not want to add extraneous function calls which at the least result in extra overhead.
You have ignored the fact that BPF LSM never wanted these empty
callbacks and you still continue to ignore it. Sigh, I will drop it now and will propose it as a separate patch so that we can at least unblock the static call series.
- KP
Quoted 7 lines
While I had issues with Kees' comments, at a high level his suggestion of dropping the last patch and moving forward is something you should consider as I don't see this a good path forward with all of the approaches that have been discussed thus far.
--
paul-moore.com
Paul Moore
maintainerRe: [PATCH v13 3/5] security: Replace indirect LSM hook calls with static calls
Quoted 6 lines · KP Singh
Quoted 5 lines · Paul Moore
Quoted 4 lines · KP Singh
Quoted 3 lines · Paul Moore
Quoted 2 lines · KP Singh
Quoted 1 line · Paul Moore
On Wed, Jul 3, 2024 at 7:08 PM KP Singh <[email protected]> wrote:
...
Quoted 2 lines
I think you are ignoring my point that BPF does not want to add extraneous function calls which at the least result in extra overhead.
I haven't been ignoring you on that point, see my previous comment:
"Correctness first, maintainability second, performance third. That's my current priority and I feel the maintainability hit doesn't justify the performance win at this point in time. Besides, we're already expecting a big performance boost simply by moving to static_calls."
Quoted 4 lines
You have ignored the fact that BPF LSM never wanted these empty
callbacks and you still continue to ignore it. Sigh, I will drop it now and will propose it as a separate patch so that we can at least unblock the static call series.
I didn't comment on that because it isn't very relevant at this point in time, what matters is the current status quo and the proposed change. In this particular case I'm not going to debate decisions made by previous maintainers, my focus is on what we currently have in-tree and what/how/why people want to change.
You've got a path forward with the bulk of this patchset, if you want to scuttle it over the last patch in the series that is up to you, but in my opinion that seems like a lost opportunity.
--
paul-moore.com
KP Singh
maintainerRe: [PATCH v13 4/5] security: Update non standard hooks to use static calls
In reply to Paul Moore[...]
Quoted 6 lines
Quoted 6 lines
security/security.c +0
-0@@ -948,10 +948,48 @@ out: \RC ; \ })
Quoted 14 lines
security/security.c out: \ · line 955
Quoted 9 lines
security/security.c +5
-4-#define lsm_for_each_hook (scall,NAME ) \ - for (scall = static_calls_table.NAME; \ - scall - static_calls_table.NAME < MAX_LSM_COUNT; scall++) \ - if (static_key_enabled(&scall->active->key)) +/* +* Can be used in the context passed to lsm_for_each_hook to get the lsmid of the +* current hook +*/ +#define current_lsmid () _hook_lsmidSee my comments below about
security_getselfattr(), I think we can drop thecurrent_lsmid()macro. If we really must keep it, we need to rename it to something else as it clashes too much with the othercurrent_XXX()macros/functions which are useful outside of our wacky macros.
call_hook_with_lsmid is a pattern used by quite a few hooks, happy to update the name.
What do you think about __security_hook_lsm_id().
Quoted 66 lines
Quoted 38 lines
security/security.c +36
-0+#define __CALL_HOOK (NUM ,HOOK ,RC ,BLOCK_BEFORE ,BLOCK_AFTER , ...) \ +do { \ + int __maybe_unused _hook_lsmid; \ + \ + if (static_branch_unlikely(&SECURITY_HOOK_ACTIVE_KEY(HOOK, NUM))) { \ + _hook_lsmid = static_calls_table.HOOK[NUM].hl->lsmid->id; \ + BLOCK_BEFORE \ + RC = static_call(LSM_STATIC_CALL(HOOK, NUM))(__VA_ARGS__); \ + BLOCK_AFTER \ + } \ +} while (0); + +#define lsm_for_each_hook (HOOK ,RC ,BLOCK_AFTER , ...) \ + LSM_LOOP_UNROLL(__CALL_HOOK, HOOK, RC, ;, BLOCK_AFTER, __VA_ARGS__) + +#define call_hook_with_lsmid (HOOK ,LSMID , ...) \ +({ \ + __label__ out; \ + int RC = LSM_RET_DEFAULT(HOOK); \ + \ + LSM_LOOP_UNROLL(__CALL_HOOK, HOOK, RC, \ +/* BLOCK BEFORE INVOCATION */ \ + { \ +if (current_lsmid () !=LSMID ) \ +continue ; \ + }, \ +/* END BLOCK BEFORE INVOCATION */ \ +/* BLOCK AFTER INVOCATION */ \ + { \ + goto out; \ + }, \ +/* END BLOCK AFTER INVOCATION */ \ + __VA_ARGS__); \ +out: \ +RC ; \ +})/* Security operations */ ...
Quoted 25 lines
security/security.c +11
-7@@ -1581,15 +1629,19 @@ int security_sb_set_mnt_opts(struct super_block *sb,unsigned long kern_flags, unsigned long *set_kern_flags) { - struct lsm_static_call *scall; int rc = mnt_opts ? -EOPNOTSUPP : LSM_RET_DEFAULT(sb_set_mnt_opts); -lsm_for_each_hook (scall, sb_set_mnt_opts) { - rc = scall->hl->hook.sb_set_mnt_opts (sb, mnt_opts, kern_flags, - set_kern_flags); -if (rc !=LSM_RET_DEFAULT (sb_set_mnt_opts)) -break ; - } +lsm_for_each_hook ( + sb_set_mnt_opts, rc, +/* BLOCK AFTER INVOCATION */ + { + if (rc != LSM_RET_DEFAULT(sb_set_mnt_opts)) + goto out; + }, +/* END BLOCK AFTER INVOCATION */ + sb, mnt_opts, kern_flags, set_kern_flags); + +out:return rc;
Quoted 8 lines
security/security.c int security_sb_set_mnt_opts(struct super_block *sb, · line 1647
Quoted 2 lines
security/security.c +0
-0}EXPORT_SYMBOL (security_sb_set_mnt_opts);I know I was the one who asked to implement the
static_callsfor *all* of the LSM functions - thank you for doing that - but I think we can all agree that some of the resulting code is pretty awful. I'm probably missing something important, but would an apporach similar to the pseudo code below work?
This does not work.
The special macro you are defining does not have the static_call invocation and if you add that bit it's basically the __CALL_HOOK macro or __CALL_STATIC_INT, __CALL_STATIC_VOID macro inlined everywhere, I tried implementing it but it gets very dirty.
Quoted 87 lines
#define call_int_hook_special(HOOK, RC, LABEL, ...) \ LSM_LOOP_UNROLL(HOOK##_SPECIAL, RC, HOOK, LABEL, __VA_ARGS__) int security_sb_set_mnt_opts(...) { int rc = LSM_RET_DEFAULT(sb_set_mnt_opts); #define sb_set_mnt_opts_SPECIAL \ do { \ if (rc != LSM_RET_DEFAULT(sb_set_mnt_opts)) \ goto out; \ } while (0)rc = call_int_hook_special(sb_set_mnt_opts, rc, out, ...);out: return rc; }Quoted 67 lines
security/security.c +35
-20@@ -4040,7 +4099,6 @@ EXPORT_SYMBOL(security_d_instantiate);int security_getselfattr (unsigned int attr,struct lsm_ctx __user *uctx,u32 __user *size,u32 flags) { -struct lsm_static_call *scall;struct lsm_ctx lctx = { .id =LSM_ID_UNDEF , };u8 __user *base = (u8 __user *)uctx;u32 entrysize;@@ -4078,31 +4136,42 @@ int security_getselfattr(unsigned int attr, struct lsm_ctx __user *uctx,* In the usual case gather all the data from the LSMs. * In the single case only get the data from the LSM specified. */ -lsm_for_each_hook (scall, getselfattr) { -if (single && lctx.id != scall->hl->lsmid->id) -continue ; - entrysize = left; -if (base) - uctx = (struct lsm_ctx __user *)(base + total); - rc = scall->hl->hook.getselfattr (attr, uctx, &entrysize, flags); -if (rc == -EOPNOTSUPP ) { - rc =0 ; -continue ; - } -if (rc == -E2BIG ) { - rc =0 ; - left =0 ; - toobig = true; - }else if (rc <0 ) -return rc; -else - left -= entrysize; +LSM_LOOP_UNROLL ( + __CALL_HOOK, getselfattr, rc, +/* BLOCK BEFORE INVOCATION */ + { +if (single && lctx.id !=current_lsmid ()) +continue ; + entrysize = left; +if (base) + uctx = (struct lsm_ctx __user *)(base + total); + }, +/* END BLOCK BEFORE INVOCATION */ +/* BLOCK AFTER INVOCATION */ + { +if (rc == -EOPNOTSUPP ) { + rc =0 ; + }else { +if (rc == -E2BIG ) { + rc =0 ; + left =0 ; + toobig = true; + }else if (rc <0 ) +return rc; +else + left -= entrysize; + + total += entrysize; + count += rc; +if (single) + goto out; + } + }, +/* END BLOCK AFTER INVOCATION */ + attr, uctx, &entrysize, flags); + +out:
Quoted 14 lines
security/security.c int security_getselfattr(unsigned int attr, struct lsm_ctx __user *uctx, · line 4177
Quoted 8 lines
security/security.c +0
-5- total += entrysize; - count += rc; -if (single) -break ; - }if (put_user (total, size))return -EFAULT ;if (toobig)I think we may need to admit defeat with
security_getselfattr()and leave it as-is, the above is just too ugly to live. I'd suggest adding a comment explaining that it wasn't converted due to complexity and the resulting awfulness.
I think your position on fixing everything is actually a valid one for security, which is why I did not contest it.
The security_getselfattr is called very close to the syscall boundary and the closer to the boundary the call is, the greater control the attacker has over arguments and the easier it is to mount the attack. This is why LSM indirect calls are a lucrative target because they happen fairly early in the transition from user to kernel. security_getselfattr is literally just in a SYSCALL_DEFINE
From a security perspective we should not leave this open.
- KP
Quoted 2 lines
--
paul-moore.com
Paul Moore
maintainerRe: [PATCH v13 4/5] security: Update non standard hooks to use static calls
Quoted 2 lines · KP Singh
Quoted 2 lines
Quoted 2 lines
security/security.c +0
-0
...
Quoted 15 lines
Quoted 6 lines
Quoted 6 lines
security/security.c +2
-4-#define lsm_for_each_hook (scall,NAME ) \ - for (scall = static_calls_table.NAME; \ - scall - static_calls_table.NAME < MAX_LSM_COUNT; scall++) \ - if (static_key_enabled(&scall->active->key)) +/* +* Can be used in the context passed to lsm_for_each_hook to get the lsmid of the Quoted 8 lines
Quoted 3 lines
+ * current hook
+ */
+#definecurrent_lsmid()_hook_lsmidSee my comments below about
security_getselfattr(), I think we can drop thecurrent_lsmid()macro. If we really must keep it, we need to rename it to something else as it clashes too much with the othercurrent_XXX()macros/functions which are useful outside of our wacky macros.
Quoted 4 lines
call_hook_with_lsmidis a pattern used by quite a few hooks, happy to update the name.What do you think about
__security_hook_lsm_id().
I guess we can't get rid of it due to the crazy macro stuff with loop unrolling, BEFORE/AFTER blocks, etc. Ooof. If you were looking for another example of why I don't really like these patches, this would be a good candidate.
More below ...
Quoted 6 lines
Quoted 5 lines
I know I was the one who asked to implement the
static_callsfor *all* of the LSM functions - thank you for doing that - but I think we can all agree that some of the resulting code is pretty awful. I'm probably missing something important, but would an apporach similar to the pseudo code below work?
Quoted 6 lines
This does not work.
The special macro you are defining does not have the
static_callinvocation and if you add that bit it's basically the__CALL_HOOKmacro or__CALL_STATIC_INT,__CALL_STATIC_VOIDmacro inlined everywhere, I tried implementing it but it gets very dirty.
Thanks for testing it out. Perhaps trying to move all of these hooks to use the static_call approach was a mistake. I realize you're doing your best adapting the static_call API to support multiple LSMs, but it just doesn't look like a good fit to me for the "unconventional" hooks here in this patch.
Quoted 8 lines
Quoted 4 lines
I think we may need to admit defeat with
security_getselfattr()and leave it as-is, the above is just too ugly to live. I'd suggest adding a comment explaining that it wasn't converted due to complexity and the resulting awfulness.I think your position on fixing everything is actually a valid one for security, which is why I did not contest it.
Quoted 6 lines
The
security_getselfattris called very close to the syscall boundary and the closer to the boundary the call is, the greater control the attacker has over arguments and the easier it is to mount the attack. This is why LSM indirect calls are a lucrative target because they happen fairly early in the transition from user to kernel.security_getselfattris literally just in aSYSCALL_DEFINE
I recognize that your comments are in reference to that last flaw rooted in the hardware that used indirect calls at an attack vector, but wasn't that resolved through other means? I never saw the PoC or had time to follow up on whatever mitigation was ultimately merged (if any). However, my understanding is that the move to static_calls is not strictly necessary to patch over that particular hardware flaw, it is just a we-really-want-this for either a performance or a non-specific security reason; pick your favorite of the two based on your audience.
Regardless, since none of the previous suggestions/options proved to be workable, I'm going to suggest we just kill this patch too and move forward with the others. I had hoped we could get the changes in this patch cleaned up, but it doesn't look like that is going to be the case, or at least not within a week or two, so let's drop it and we can always reconsider this in the future if a cleaner implementation is presented.
Signature · 1 line
paul-moore.com
Re: [PATCH v13 4/5] security: Update non standard hooks to use static calls
In reply to KP SinghQuoted 21 lines · KP Singh
[...]
Quoted 6 lines
Quoted 6 lines
security/security.c +0
-0@@ -948,10 +948,48 @@ out: \RC ; \ })Quoted 13 lines
security/security.c out: \ · line 955
Quoted 9 lines
security/security.c +5
-4-#define lsm_for_each_hook (scall,NAME ) \ - for (scall = static_calls_table.NAME; \ - scall - static_calls_table.NAME < MAX_LSM_COUNT; scall++) \ - if (static_key_enabled(&scall->active->key)) +/* +* Can be used in the context passed to lsm_for_each_hook to get the lsmid of the +* current hook +*/ +#define current_lsmid () _hook_lsmidSee my comments below about
security_getselfattr(), I think we can drop thecurrent_lsmid()macro. If we really must keep it, we need to rename it to something else as it clashes too much with the othercurrent_XXX()macros/functions which are useful outside of our wacky macros.
Quoted 4 lines
call_hook_with_lsmidis a pattern used by quite a few hooks, happy to update the name.What do you think about
__security_hook_lsm_id().
I really dislike it. The security prefix (even with __) tells the developer in security.c that the code is used elsewhere. How about lsm_hook_current_id()?
Quoted 195 lines
Quoted 65 lines
Quoted 39 lines
security/security.c +36
-0+#define __CALL_HOOK (NUM ,HOOK ,RC ,BLOCK_BEFORE ,BLOCK_AFTER , ...) \ +do { \ + int __maybe_unused _hook_lsmid; \ + \ + if (static_branch_unlikely(&SECURITY_HOOK_ACTIVE_KEY(HOOK, NUM))) { \ + _hook_lsmid = static_calls_table.HOOK[NUM].hl->lsmid->id; \ + BLOCK_BEFORE \ + RC = static_call(LSM_STATIC_CALL(HOOK, NUM))(__VA_ARGS__); \ + BLOCK_AFTER \ + } \ +} while (0); + +#define lsm_for_each_hook (HOOK ,RC ,BLOCK_AFTER , ...) \ + LSM_LOOP_UNROLL(__CALL_HOOK, HOOK, RC, ;, BLOCK_AFTER, __VA_ARGS__) + +#define call_hook_with_lsmid (HOOK ,LSMID , ...) \ +({ \ + __label__ out; \ + int RC = LSM_RET_DEFAULT(HOOK); \ + \ + LSM_LOOP_UNROLL(__CALL_HOOK, HOOK, RC, \ +/* BLOCK BEFORE INVOCATION */ \ + { \ +if (current_lsmid () !=LSMID ) \ +continue ; \ + }, \ +/* END BLOCK BEFORE INVOCATION */ \ +/* BLOCK AFTER INVOCATION */ \ + { \ + goto out; \ + }, \ +/* END BLOCK AFTER INVOCATION */ \ + __VA_ARGS__); \ +out: \ +RC ; \ +})/* Security operations */ ...
Quoted 24 lines
security/security.c +11
-7@@ -1581,15 +1629,19 @@ int security_sb_set_mnt_opts(struct super_block *sb,unsigned long kern_flags, unsigned long *set_kern_flags) { - struct lsm_static_call *scall; int rc = mnt_opts ? -EOPNOTSUPP : LSM_RET_DEFAULT(sb_set_mnt_opts); -lsm_for_each_hook (scall, sb_set_mnt_opts) { - rc = scall->hl->hook.sb_set_mnt_opts (sb, mnt_opts, kern_flags, - set_kern_flags); -if (rc !=LSM_RET_DEFAULT (sb_set_mnt_opts)) -break ; - } +lsm_for_each_hook ( + sb_set_mnt_opts, rc, +/* BLOCK AFTER INVOCATION */ + { + if (rc != LSM_RET_DEFAULT(sb_set_mnt_opts)) + goto out; + }, +/* END BLOCK AFTER INVOCATION */ + sb, mnt_opts, kern_flags, set_kern_flags); + +out:Quoted 8 lines
security/security.c int security_sb_set_mnt_opts(struct super_block *sb, · line 1647
Quoted 3 lines
security/security.c +0
-0return rc; }EXPORT_SYMBOL (security_sb_set_mnt_opts);I know I was the one who asked to implement the
static_callsfor *all* of the LSM functions - thank you for doing that - but I think we can all agree that some of the resulting code is pretty awful. I'm probably missing something important, but would an apporach similar to the pseudo code below work?This does not work.
The special macro you are defining does not have the
static_callinvocation and if you add that bit it's basically the__CALL_HOOKmacro or__CALL_STATIC_INT,__CALL_STATIC_VOIDmacro inlined everywhere, I tried implementing it but it gets very dirty.Quoted 86 lines
#define call_int_hook_special(HOOK, RC, LABEL, ...) \ LSM_LOOP_UNROLL(HOOK##_SPECIAL, RC, HOOK, LABEL, __VA_ARGS__) int security_sb_set_mnt_opts(...) { int rc = LSM_RET_DEFAULT(sb_set_mnt_opts); #define sb_set_mnt_opts_SPECIAL \ do { \ if (rc != LSM_RET_DEFAULT(sb_set_mnt_opts)) \ goto out; \ } while (0)rc = call_int_hook_special(sb_set_mnt_opts, rc, out, ...);out: return rc; }Quoted 67 lines
security/security.c +35
-20@@ -4040,7 +4099,6 @@ EXPORT_SYMBOL(security_d_instantiate);int security_getselfattr (unsigned int attr,struct lsm_ctx __user *uctx,u32 __user *size,u32 flags) { -struct lsm_static_call *scall;struct lsm_ctx lctx = { .id =LSM_ID_UNDEF , };u8 __user *base = (u8 __user *)uctx;u32 entrysize;@@ -4078,31 +4136,42 @@ int security_getselfattr(unsigned int attr, struct lsm_ctx __user *uctx,* In the usual case gather all the data from the LSMs. * In the single case only get the data from the LSM specified. */ -lsm_for_each_hook (scall, getselfattr) { -if (single && lctx.id != scall->hl->lsmid->id) -continue ; - entrysize = left; -if (base) - uctx = (struct lsm_ctx __user *)(base + total); - rc = scall->hl->hook.getselfattr (attr, uctx, &entrysize, flags); -if (rc == -EOPNOTSUPP ) { - rc =0 ; -continue ; - } -if (rc == -E2BIG ) { - rc =0 ; - left =0 ; - toobig = true; - }else if (rc <0 ) -return rc; -else - left -= entrysize; +LSM_LOOP_UNROLL ( + __CALL_HOOK, getselfattr, rc, +/* BLOCK BEFORE INVOCATION */ + { +if (single && lctx.id !=current_lsmid ()) +continue ; + entrysize = left; +if (base) + uctx = (struct lsm_ctx __user *)(base + total); + }, +/* END BLOCK BEFORE INVOCATION */ +/* BLOCK AFTER INVOCATION */ + { +if (rc == -EOPNOTSUPP ) { + rc =0 ; + }else { +if (rc == -E2BIG ) { + rc =0 ; + left =0 ; + toobig = true; + }else if (rc <0 ) +return rc; +else + left -= entrysize; + + total += entrysize; + count += rc; +if (single) + goto out; + } + }, +/* END BLOCK AFTER INVOCATION */ + attr, uctx, &entrysize, flags); + +out:Quoted 13 lines
security/security.c int security_getselfattr(unsigned int attr, struct lsm_ctx __user *uctx, · line 4177
Quoted 8 lines
security/security.c +0
-5- total += entrysize; - count += rc; -if (single) -break ; - }if (put_user (total, size))return -EFAULT ;if (toobig)I think we may need to admit defeat with
security_getselfattr()and leave it as-is, the above is just too ugly to live. I'd suggest adding a comment explaining that it wasn't converted due to complexity and the resulting awfulness.I think your position on fixing everything is actually a valid one for security, which is why I did not contest it.
The
security_getselfattris called very close to the syscall boundary and the closer to the boundary the call is, the greater control the attacker has over arguments and the easier it is to mount the attack. This is why LSM indirect calls are a lucrative target because they happen fairly early in the transition from user to kernel.security_getselfattris literally just in aSYSCALL_DEFINEQuoted 1 line
From a security perspective we should not leave this open.
- KP
Quoted 2 lines
--
paul-moore.com
Paul Moore
maintainerRe: [PATCH v13 4/5] security: Update non standard hooks to use static calls
Quoted 22 lines · Casey Schaufler
Quoted 21 lines · KP Singh
[...]
Quoted 19 lines
Quoted 5 lines
security/security.c +0
-0@@ -948,10 +948,48 @@ out: \RC ; \ })security/security.c out: \ · line 955
Quoted 10 lines
security/security.c +5
-4-#define lsm_for_each_hook (scall,NAME ) \ - for (scall = static_calls_table.NAME; \ - scall - static_calls_table.NAME < MAX_LSM_COUNT; scall++) \ - if (static_key_enabled(&scall->active->key)) +/* +* Can be used in the context passed to lsm_for_each_hook to get the lsmid of the +* current hook +*/ +#define current_lsmid () _hook_lsmidSee my comments below about
security_getselfattr(), I think we can drop thecurrent_lsmid()macro. If we really must keep it, we need to rename it to something else as it clashes too much with the othercurrent_XXX()macros/functions which are useful outside of our wacky macros.
Quoted 8 lines
Quoted 4 lines
call_hook_with_lsmidis a pattern used by quite a few hooks, happy to update the name.What do you think about
__security_hook_lsm_id().I really dislike it. The security prefix (even with __) tells the developer in
security.cthat the code is used elsewhere. How aboutlsm_hook_current_id()?
See my reply. There is enough ugliness in converting the hooks in this particular patch that I think we need to shelve this patch too.
Signature · 1 line
paul-moore.com