Fanotify fastpath handler
Song Liu
maintainerv2 cover Fanotify fastpath handler
Overview of v2:
Patch 1/7 adds logic to write fastpath handlers in kernel modules. Patch 2/7 adds a sample of a fastpath handler in a kernel module. Patch 3/7 to 5/7 are preparation work on BPF side. Patch 6/7 adds logic to write fastpath handlers in bpf programs. Patch 7/7 is a selftest and example of bpf based fastpath handler.
Changes v1 => v2:
1. Add sysfs entries for fastpath handler.
2. Rewrite the sample and bpf selftest to handle subtree monitoring. This requires quite some work from BPF side to properly handle inode, dentry, etc.
3. Add CONFIG_FANOTIFY_FASTPATH.
4. Add more documents.
TODO of v2:
1. Enable prviate (not added to global list) bpf based fastpath handlers.
4. Man pages.
From v1 RFC:
This RFC set introduces in-kernel fastpath handler for fanotify. The fastpath handler can be used to handle/filter some events without going through userspace.
In LPC 2024, multiple talks covered use cases of monitoring a subtree in the VFS (fanotify: [1], bpf/lsm: [2]). This work is inspired by these discussions. Reliably monitoring of a subtree with low overhead is a hard problem. We do not claim this set fully solves problem. But we think this work can be a very useful building block of the solution to this problem.
The fastpath handler can be implemented with built-in logic, in a kernel module, or a bpf program. The fastpath handler is attached to a fsnotify group. With current implementation, the multiple fastpath handlers are maintained in a global list. Only users with CAP_SYS_ADMIN can add fastpath handlers to the list by loading a kernel module. User without CAP_SYS_ADMIN can attach a loaded fastpath handler to fanotify instances. During the attach operation, the fastpath handler can take an argument. This enables non-CAP_SYSADMIN users to customize/configure the fastpath handler, for example, with a specific allowlist/denylist.
As the patchset grows to 1000+ lines (including samples and tests), I would like some feedback before pushing it further.
[1] https://lpc.events/event/18/contributions/1717/
[2] https://lpc.events/event/18/contributions/1940/
Song Liu (7):
fanotify: Introduce fanotify fastpath handler samples/fanotify: Add a sample fanotify fastpath handler
bpf: Make bpf inode storage available to tracing programs
bpf: fs: Add three kfuncs
bpf: Allow bpf map hold reference on dentry
fanotify: Enable bpf based fanotify fastpath handler selftests/bpf: Add test for BPF based fanotify fastpath handler
MAINTAINERS | 1 + fs/Makefile | 2 +- fs/bpf_fs_kfuncs.c | 51 +- fs/inode.c | 2 + fs/notify/fanotify/Kconfig | 13 + fs/notify/fanotify/Makefile | 1 + fs/notify/fanotify/fanotify.c | 29 ++ fs/notify/fanotify/fanotify_fastpath.c | 448 ++++++++++++++++++ fs/notify/fanotify/fanotify_user.c | 7 + include/linux/bpf.h | 9 + include/linux/bpf_lsm.h | 29 -- include/linux/fanotify.h | 131 +++++ include/linux/fs.h | 4 + include/linux/fsnotify_backend.h | 4 + include/uapi/linux/fanotify.h | 25 + kernel/bpf/Makefile | 3 +- kernel/bpf/bpf_inode_storage.c | 176 +++++-- kernel/bpf/bpf_lsm.c | 4 - kernel/bpf/helpers.c | 14 +- kernel/bpf/verifier.c | 6 + kernel/trace/bpf_trace.c | 8 + samples/Kconfig | 20 +- samples/Makefile | 2 +- samples/fanotify/.gitignore | 1 + samples/fanotify/Makefile | 5 +- samples/fanotify/fastpath-mod.c | 82 ++++ samples/fanotify/fastpath-user.c | 111 +++++ security/bpf/hooks.c | 7 - tools/testing/selftests/bpf/bpf_kfuncs.h | 5 + tools/testing/selftests/bpf/config | 2 + .../testing/selftests/bpf/prog_tests/fan_fp.c | 264 +++++++++++ tools/testing/selftests/bpf/progs/fan_fp.c | 154 ++++++ 32 files changed, 1530 insertions(+), 90 deletions(-) create mode 100644 fs/notify/fanotify/fanotify_fastpath.c create mode 100644 samples/fanotify/fastpath-mod.c create mode 100644 samples/fanotify/fastpath-user.c create mode 100644 tools/testing/selftests/bpf/prog_tests/fan_fp.c create mode 100644 tools/testing/selftests/bpf/progs/fan_fp.c
--
2.43.5
Song Liu
maintainerPATCH v2 1/7 fanotify: Introduce fanotify fastpath handler
fanotify fastpath handler enables handling fanotify events within the kernel, and thus saves a trip to the user space. fanotify fastpath handler can be useful in many use cases. For example, if a user is only interested in events for some files in side a directory, a fastpath handler can be used to filter out irrelevant events.
fanotify fastpath handler is attached to fsnotify_group. At most one fastpath handler can be attached to a fsnotify_group. The attach/detach of fastpath handlers are controlled by two new ioctls on the fanotify fds:FAN_IOC_ADD_FP and FAN_IOC_DEL_FP.
fanotify fastpath handler is packaged in a kernel module. In the future, it is also possible to package fastpath handler in a BPF program. Since loading modules requires CAP_SYS_ADMIN, _loading_ fanotify fastpath handler in kernel modules is limited to CAP_SYS_ADMIN. However, non-SYS_CAP_ADMIN users can _attach_ fastpath handler loaded by sys admin to their fanotify fds. To make fanotify fastpath handler more useful for non-CAP_SYS_ADMIN users, a fastpath handler can take arguments at attach time.
sysfs entry /sys/kernel/fanotify_fastpath is added to help users know which fastpath handlers are available. At the moment, files are added for each fastpath handler: flags, desc, and init_args.
--- fs/notify/fanotify/Kconfig | 13 ++ fs/notify/fanotify/Makefile | 1 + fs/notify/fanotify/fanotify.c | 29 +++ fs/notify/fanotify/fanotify_fastpath.c | 282 +++++++++++++++++++++++++ fs/notify/fanotify/fanotify_user.c | 7 + include/linux/fanotify.h | 131 ++++++++++++ include/linux/fsnotify_backend.h | 4 + include/uapi/linux/fanotify.h | 25 +++ 8 files changed, 492 insertions(+) create mode 100644 fs/notify/fanotify/fanotify_fastpath.c
fs/notify/fanotify/Kconfig +13 -0
@@ -24,3 +24,16 @@ config FANOTIFY_ACCESS_PERMISSIONS hierarchical storage management systems. If unsure, say N. + +config FANOTIFY_FASTPATH + bool "fanotify fastpath handler" + depends on FANOTIFY + default y + help + Say Y here if you want to use fanotify in kernel fastpath handler. + The fastpath handler can be implemented in a kernel module or a + BPF program. The fastpath handler can speed up fanotify in many + use cases. For example, when the listener is only interested in + a subset of events. + + If unsure, say Y.
fs/notify/fanotify/Makefile +1 -0
@@ -1,2 +1,3 @@ # SPDX-License-Identifier: GPL-2.0-only obj-$(CONFIG_FANOTIFY) += fanotify.o fanotify_user.o +obj-$(CONFIG_FANOTIFY_FASTPATH) += fanotify_fastpath.o
fs/notify/fanotify/fanotify.c +29 -0
@@ -18,6 +18,8 @@#include "fanotify.h" +extern struct srcu_struct fsnotify_mark_srcu; +static bool fanotify_path_equal (const struct path *p1,const struct path *p2) {return p1->mnt == p2->mnt && p1->dentry == p2->dentry;
@@ -888,6 +890,7 @@ static int fanotify_handle_event(struct fsnotify_group *group, u32 mask,struct fsnotify_event *fsn_event;__kernel_fsid_t fsid = {};u32 match_mask =0 ; +struct fanotify_fastpath_hook *fp_hook__maybe_unused ;BUILD_BUG_ON (FAN_ACCESS !=FS_ACCESS );BUILD_BUG_ON (FAN_MODIFY !=FS_MODIFY );
@@ -933,6 +936,27 @@ static int fanotify_handle_event(struct fsnotify_group *group, u32 mask,if (FAN_GROUP_FLAG (group,FANOTIFY_FID_BITS )) fsid =fanotify_get_fsid (iter_info); +#ifdef CONFIG_FANOTIFY_FASTPATH + fp_hook =srcu_dereference (group->fanotify_data.fp_hook, &fsnotify_mark_srcu); +if (fp_hook) { +struct fanotify_fastpath_event fp_event = { + .mask = mask, + .data = data, + .data_type = data_type, + .dir = dir, + .file_name = file_name, + .fsid = &fsid, + .match_mask = match_mask, + }; + + ret = fp_hook->ops->fp_handler (group, fp_hook, &fp_event); +if (ret ==FAN_FP_RET_SKIP_EVENT ) { + ret =0 ; + goto finish; + } + } +#endif + event =fanotify_alloc_event (group, mask, data, data_type, dir, file_name, &fsid, match_mask); ret = -ENOMEM ;
@@ -976,6 +1000,11 @@ static void fanotify_free_group_priv(struct fsnotify_group *group)if (mempool_initialized (&group->fanotify_data.error_events_pool))mempool_exit (&group->fanotify_data.error_events_pool); + +#ifdef CONFIG_FANOTIFY_FASTPATH +if (group->fanotify_data.fp_hook) +fanotify_fastpath_hook_free (group->fanotify_data.fp_hook); +#endif }static void fanotify_free_path_event (struct fanotify_event *event)
fs/notify/fanotify/fanotify_fastpath.c new file +282 -0
282 lines not on this page · raw message
fs/notify/fanotify/fanotify_user.c +7 -0
@@ -987,6 +987,13 @@ static long fanotify_ioctl(struct file *file, unsigned int cmd, unsigned long arspin_unlock (&group->notification_lock); ret =put_user (send_len, (int __user *) p);break ; +case FAN_IOC_ADD_FP : + ret =fanotify_fastpath_add (group, p); +break ; +case FAN_IOC_DEL_FP : +fanotify_fastpath_del (group); + ret =0 ; +break ; }return ret;
include/linux/fanotify.h +131 -0
141 lines not on this page · raw message
include/linux/fsnotify_backend.h +4 -0
@@ -117,6 +117,7 @@ struct fsnotify_fname;struct fsnotify_iter_info ;struct mem_cgroup ; +struct fanotify_fastpath_hook ;/* * Each group much define these ops. The fsnotify infrastructure will call
@@ -255,6 +256,9 @@ struct fsnotify_group {int f_flags;/* event_f_flags from fanotify_init() */ struct ucounts *ucounts;mempool_t error_events_pool; +#ifdef CONFIG_FANOTIFY_FASTPATH +struct fanotify_fastpath_hook __rcu *fp_hook; +#endif /* CONFIG_FANOTIFY_FASTPATH */ } fanotify_data;#endif /* CONFIG_FANOTIFY */ };
include/uapi/linux/fanotify.h +25 -0
@@ -3,6 +3,7 @@#define _UAPI_LINUX_FANOTIFY_H#include <linux/types.h> +#include <linux/ioctl.h> /* the following events that user-space can register for */ #define FAN_ACCESS 0x00000001/* File was accessed */
@@ -243,4 +244,28 @@ struct fanotify_response_info_audit_rule { (long)(meta)->event_len >= (long)FAN_EVENT_METADATA_LEN && \ (long)(meta)->event_len <= (long)(len)) +#define FAN_FP_NAME_MAX 64 +#define FAN_FP_ARGS_MAX 64 + +/* This is the arguments used to add fastpath handler to a group. */ +struct fanotify_fastpath_args { +char name[FAN_FP_NAME_MAX ]; + +__u32 version; +__u32 flags; + +/* +* user space pointer to the init args of fastpath handler, +* up to init_args_len (<= FAN_FP_ARGS_MAX). +*/ +__u64 init_args; +/* size of init_args */ +__u32 init_args_size; +} __attribute__((__packed__ )); + +#define FAN_IOC_MAGIC 'F' + +#define FAN_IOC_ADD_FP _IOW(FAN_IOC_MAGIC, 0, struct fanotify_fastpath_args) +#define FAN_IOC_DEL_FP _IOW(FAN_IOC_MAGIC, 1, char[FAN_FP_NAME_MAX]) +#endif /* _UAPI_LINUX_FANOTIFY_H */
Signature · 1 line
2.43.5
Amir Goldstein
reviewerRe: [RFC/PATCH v2 bpf-next fanotify 1/7] fanotify: Introduce fanotify fastpath handler
Quoted 104 lines · Song Liu
fanotify fastpath handler enables handling fanotify events within the kernel, and thus saves a trip to the user space. fanotify fastpath handler can be useful in many use cases. For example, if a user is only interested in events for some files in side a directory, a fastpath handler can be used to filter out irrelevant events.
fanotify fastpath handler is attached to
fsnotify_group. At most one fastpath handler can be attached to afsnotify_group. The attach/detach of fastpath handlers are controlled by two new ioctls on the fanotify fds:FAN_IOC_ADD_FPandFAN_IOC_DEL_FP.fanotify fastpath handler is packaged in a kernel module. In the future, it is also possible to package fastpath handler in a BPF program. Since loading modules requires
CAP_SYS_ADMIN, _loading_ fanotify fastpath handler in kernel modules is limited toCAP_SYS_ADMIN. However, non-SYS_CAP_ADMIN users can _attach_ fastpath handler loaded by sys admin to their fanotify fds. To make fanotify fastpath handler more useful for non-CAP_SYS_ADMIN users, a fastpath handler can take arguments at attach time.sysfs entry
/sys/kernel/fanotify_fastpathis added to help users know which fastpath handlers are available. At the moment, files are added for each fastpath handler: flags, desc, andinit_args.Signed-off-by Song Liu <[email protected]>--- fs/notify/fanotify/Kconfig | 13 ++ fs/notify/fanotify/Makefile | 1 + fs/notify/fanotify/fanotify.c | 29 +++ fs/notify/fanotify/fanotify_fastpath.c | 282 +++++++++++++++++++++++++ fs/notify/fanotify/fanotify_user.c | 7 + include/linux/fanotify.h | 131 ++++++++++++ include/linux/fsnotify_backend.h | 4 + include/uapi/linux/fanotify.h | 25 +++ 8 files changed, 492 insertions(+) create mode 100644 fs/notify/fanotify/fanotify_fastpath.cfs/notify/fanotify/Kconfig +13
-0@@ -24,3 +24,16 @@ config FANOTIFY_ACCESS_PERMISSIONS hierarchical storage management systems. If unsure, say N. + +config FANOTIFY_FASTPATH + bool "fanotify fastpath handler" + depends on FANOTIFY + default y + help + Say Y here if you want to use fanotify in kernel fastpath handler. + The fastpath handler can be implemented in a kernel module or a + BPF program. The fastpath handler can speed up fanotify in many + use cases. For example, when the listener is only interested in + a subset of events. + + If unsure, say Y.fs/notify/fanotify/Makefile +1
-0@@ -1,2 +1,3 @@ # SPDX-License-Identifier: GPL-2.0-only obj-$(CONFIG_FANOTIFY) += fanotify.o fanotify_user.o +obj-$(CONFIG_FANOTIFY_FASTPATH) += fanotify_fastpath.ofs/notify/fanotify/fanotify.c +14
-0@@ -18,6 +18,8 @@#include "fanotify.h" +extern struct srcu_struct fsnotify_mark_srcu; +static bool fanotify_path_equal (const struct path *p1,const struct path *p2) {return p1->mnt == p2->mnt && p1->dentry == p2->dentry;@@ -888,6 +890,7 @@ static int fanotify_handle_event(struct fsnotify_group *group, u32 mask,struct fsnotify_event *fsn_event;__kernel_fsid_t fsid = {};u32 match_mask =0 ; +struct fanotify_fastpath_hook *fp_hook__maybe_unused ;BUILD_BUG_ON (FAN_ACCESS !=FS_ACCESS );BUILD_BUG_ON (FAN_MODIFY !=FS_MODIFY );@@ -933,6 +936,27 @@ static int fanotify_handle_event(struct fsnotify_group *group, u32 mask,if (FAN_GROUP_FLAG (group,FANOTIFY_FID_BITS )) fsid =fanotify_get_fsid (iter_info); +#ifdef CONFIG_FANOTIFY_FASTPATH + fp_hook = srcu_dereference(group->fanotify_data.fp_hook, &fsnotify_mark_srcu); + if (fp_hook) { + struct fanotify_fastpath_event fp_event = { + .mask = mask, + .data = data, + .data_type = data_type, + .dir = dir, + .file_name = file_name, + .fsid = &fsid, + .match_mask = match_mask,
fs/notify/fanotify/fanotify.c static int fanotify_handle_event(struct fsnotify_group *group, u32 mask, · line 959
Quoted 10 lines
fs/notify/fanotify/fanotify.c +10
-0+ }; + + ret = fp_hook->ops->fp_handler (group, fp_hook, &fp_event); +if (ret ==FAN_FP_RET_SKIP_EVENT ) { + ret =0 ; + goto finish; + } + } +#endif +
To me it makes sense that the fastpath module could also return a negative (deny) result for permission events.
Is there a specific reason that you did not handle this or just didn't think of this option?
Thanks,
Amir.
Re: [RFC/PATCH v2 bpf-next fanotify 1/7] fanotify: Introduce fanotify fastpath handler
Hi Amir,
Quoted 1 line · Amir Goldstein
[...]
Quoted 17 lines
Quoted 17 lines
fs/notify/fanotify/fanotify.c +16
-0+#ifdef CONFIG_FANOTIFY_FASTPATH + fp_hook =srcu_dereference (group->fanotify_data.fp_hook, &fsnotify_mark_srcu); +if (fp_hook) { +struct fanotify_fastpath_event fp_event = { + .mask = mask, + .data = data, + .data_type = data_type, + .dir = dir, + .file_name = file_name, + .fsid = &fsid, + .match_mask = match_mask, + }; + + ret = fp_hook->ops->fp_handler (group, fp_hook, &fp_event); +if (ret ==FAN_FP_RET_SKIP_EVENT ) { + ret =0 ;
Quoted 8 lines
fs/notify/fanotify/fanotify.c static int fanotify_handle_event(struct fsnotify_group *group, u32 mask, · line 959
Quoted 5 lines
fs/notify/fanotify/fanotify.c +5
-0+ goto finish; + } + } +#endif +To me it makes sense that the fastpath module could also return a negative (deny) result for permission events.
Yes, this should just work. And I actually plan to use it.
Quoted 2 lines
Is there a specific reason that you did not handle this or just didn't think of this option?
But I haven't tested permission events yet. At first glance, maybe we just need to change the above code a bit, as:
Quoted 4 lines
fs/notify/fanotify/fanotify.c static int fanotify_handle_event(struct fsnotify_group *group, u32 mask, · line 956
Quoted 4 lines
f (ret ==
FAN_FP_RET_SKIP_EVENT) {fs/notify/fanotify/fanotify.c +3
-0+ ret =0 ; + goto finish; + }
if (ret != FAN_FP_RET_SEND_TO_USERSPACE) {
if (ret == FAN_FP_RET_SKIP_EVENT)
ret = 0;
goto finish;
}Well, I guess we should change the value of FAN_FP_RET_SEND_TO_USERSPACE, so that this condition will look better.
We may also consider reorder the code so that we do not call fsnotify_prepare_user_wait() when the fastpath handles the event.
Does this look reasonable?
Thanks,
Song
Amir Goldstein
reviewerRe: [RFC/PATCH v2 bpf-next fanotify 1/7] fanotify: Introduce fanotify fastpath handler
Quoted 53 lines · Song Liu
Hi Amir,
Quoted 1 line · Amir Goldstein
[...]
Quoted 17 lines
Quoted 17 lines
fs/notify/fanotify/fanotify.c +16
-0+#ifdef CONFIG_FANOTIFY_FASTPATH + fp_hook =srcu_dereference (group->fanotify_data.fp_hook, &fsnotify_mark_srcu); +if (fp_hook) { +struct fanotify_fastpath_event fp_event = { + .mask = mask, + .data = data, + .data_type = data_type, + .dir = dir, + .file_name = file_name, + .fsid = &fsid, + .match_mask = match_mask, + }; + + ret = fp_hook->ops->fp_handler (group, fp_hook, &fp_event); +if (ret ==FAN_FP_RET_SKIP_EVENT ) { + ret =0 ;Quoted 8 lines
fs/notify/fanotify/fanotify.c static int fanotify_handle_event(struct fsnotify_group *group, u32 mask, · line 959
Quoted 5 lines
fs/notify/fanotify/fanotify.c +5
-0+ goto finish; + } + } +#endif +To me it makes sense that the fastpath module could also return a negative (deny) result for permission events.
Yes, this should just work. And I actually plan to use it.
Quoted 2 lines
Is there a specific reason that you did not handle this or just didn't think of this option?
But I haven't tested permission events yet. At first glance, maybe we just need to change the above code a bit, as:
Quoted 4 lines
fs/notify/fanotify/fanotify.c static int fanotify_handle_event(struct fsnotify_group *group, u32 mask, · line 956
Quoted 4 lines
f (ret ==
FAN_FP_RET_SKIP_EVENT) {fs/notify/fanotify/fanotify.c +3
-0+ ret =0 ; + goto finish; + }if (ret != FAN_FP_RET_SEND_TO_USERSPACE) { if (ret == FAN_FP_RET_SKIP_EVENT) ret = 0; goto finish; }
Quoted 7 lines
Well, I guess we should change the value of
FAN_FP_RET_SEND_TO_USERSPACE, so that this condition will look better.We may also consider reorder the code so that we do not call
fsnotify_prepare_user_wait()when the fastpath handles the event.Does this look reasonable?
Yes.
Thanks,
Amir.
kernel test robot
build botRe: [RFC/PATCH v2 bpf-next fanotify 1/7] fanotify: Introduce fanotify fastpath handler
In reply to Song Liukernel test robot noticed the following build warnings:
Hi Song,
[This is a private test report for your RFC patch.] kernel test robot noticed the following build warnings:
[auto build test WARNING on jack-fs/fsnotify]
[also build test WARNING on linus/master bpf/master v6.12-rc7 next-20241115] [cannot apply to bpf-next/master]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Song-Liu/fanotify-Introduce-fanotify-fastpath-handler/20241114-164824 base: https://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs.git fsnotify patch link: https://lore.kernel.org/r/20241114084345.1564165-2-song%40kernel.org patch subject: [RFC/PATCH v2 bpf-next fanotify 1/7] fanotify: Introduce fanotify fastpath handler config: i386-allmodconfig (https://download.01.org/0day-ci/archive/20241117/[email protected]/config) compiler: gcc-12 (Debian 12.2.0-14) 12.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20241117/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
fs/notify/fanotify/fanotify_fastpath.c: In function 'fanotify_fastpath_add':
>> fs/notify/fanotify/fanotify_fastpath.c:219:55: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
219 | if (copy_from_user(init_args, (void __user *)args.init_args,
| ^
Kconfig warnings: (for reference only)
WARNING: unmet direct dependencies detected for GET_FREE_REGION Depends on [n]: SPARSEMEM [=n]
Selected by [m]:
- RESOURCE_KUNIT_TEST [=m] && RUNTIME_TESTING_MENU [=y] && KUNIT [=m]
vim +219 fs/notify/fanotify/fanotify_fastpath.c
153
154 /*
155 * fanotify_fastpath_add - Add a fastpath handler to fsnotify_group.
156 *
157 * Add a fastpath handler from fp_list to a fsnotify_group.
158 *
159 * @group: fsnotify_group that will have add
160 * @argp: fanotify_fastpath_args that specifies the fastpath handler
161 * and the init arguments of the fastpath handler.
162 *
163 * Returns:
164 * 0 - on success;
165 * -EEXIST - fastpath handler of the same name already exists.
166 */
167 int fanotify_fastpath_add(struct fsnotify_group *group,
168 struct fanotify_fastpath_args __user *argp)
169 {
170 struct fanotify_fastpath_hook *fp_hook;
171 struct fanotify_fastpath_ops *fp_ops;
172 struct fanotify_fastpath_args args;
173 void *init_args = NULL;
174 int ret = 0;
175
176 ret = copy_from_user(&args, argp, sizeof(args));
177 if (ret)
178 return -EFAULT;
179
180 if (args.version != 1 || args.flags || args.init_args_size > FAN_FP_ARGS_MAX)
181 return -EINVAL;
182
183 args.name[FAN_FP_NAME_MAX - 1] = '\0';
184
185 fsnotify_group_lock(group);
186
187 if (rcu_access_pointer(group->fanotify_data.fp_hook)) {
188 fsnotify_group_unlock(group);
189 return -EBUSY;
190 }
191
192 fp_hook = kzalloc(sizeof(*fp_hook), GFP_KERNEL);
193 if (!fp_hook) {
194 ret = -ENOMEM;
195 goto out;
196 }
197
198 spin_lock(&fp_list_lock);
199 fp_ops = fanotify_fastpath_find(args.name);
200 if (!fp_ops || !try_module_get(fp_ops->owner)) {
201 spin_unlock(&fp_list_lock);
202 ret = -ENOENT;
203 goto err_free_hook;
204 }
205 spin_unlock(&fp_list_lock);
206
207 if (!capable(CAP_SYS_ADMIN) && (fp_ops->flags & FAN_FP_F_SYS_ADMIN_ONLY)) {
208 ret = -EPERM;
209 goto err_module_put;
210 }
211
212 if (fp_ops->fp_init) {
213 if (args.init_args_size) {
214 init_args = kzalloc(args.init_args_size, GFP_KERNEL);
215 if (!init_args) {
216 ret = -ENOMEM;
217 goto err_module_put;
218 }
> 219 if (copy_from_user(init_args, (void __user *)args.init_args,
220 args.init_args_size)) {
221 ret = -EFAULT;
222 goto err_free_args;
223 }
224
225 }
226 ret = fp_ops->fp_init(fp_hook, init_args);
227 if (ret)
228 goto err_free_args;
229 kfree(init_args);
230 }
231 fp_hook->ops = fp_ops;
232 rcu_assign_pointer(group->fanotify_data.fp_hook, fp_hook);
233
234 out:
235 fsnotify_group_unlock(group);
236 return ret;
237
238 err_free_args:
239 kfree(init_args);
240 err_module_put:
241 module_put(fp_ops->owner);
242 err_free_hook:
243 kfree(fp_hook);
244 goto out;
245 }
246
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Song Liu
maintainerPATCH v2 2/7 samples/fanotify: Add a sample fanotify fastpath handler
In reply to Song LiuThis fastpath handler monitors a subtree inside a mount point. To use it:
[root] insmod ./fastpath-mod.ko
[root] mkdir -p /tmp/a/b/c/d
[root] ./fastpath-user /tmp/ /tmp/a/b &
[root] touch /tmp/xx # Doesn't generate event
[root]# touch /tmp/a/xxa # Doesn't generate event
[root]# touch /tmp/a/b/xxab # Generates an event
Accessing file xxab # this is the output from fastpath-user
[root@]# touch /tmp/a/b/c/xxabc # Generates an event Accessing file xxabc # this is the output from fastpath-user
--- MAINTAINERS | 1 + samples/Kconfig | 20 +++++- samples/Makefile | 2 +- samples/fanotify/.gitignore | 1 + samples/fanotify/Makefile | 5 +- samples/fanotify/fastpath-mod.c | 82 +++++++++++++++++++++++ samples/fanotify/fastpath-user.c | 111 +++++++++++++++++++++++++++++++ 7 files changed, 219 insertions(+), 3 deletions(-) create mode 100644 samples/fanotify/fastpath-mod.c create mode 100644 samples/fanotify/fastpath-user.c
MAINTAINERS +1 -0
@@ -8658,6 +8658,7 @@ S: Maintained F: fs/notify/fanotify/ F: include/linux/fanotify.h F: include/uapi/linux/fanotify.h +F: samples/fanotify/ FARADAY FOTG210 USB2 DUAL-ROLE CONTROLLER M: Linus Walleij <[email protected]>
samples/Kconfig +19 -1
@@ -149,15 +149,33 @@ config SAMPLE_CONNECTOR with it. See also Documentation/driver-api/connector.rst +config SAMPLE_FANOTIFY + bool "Build fanotify monitoring sample" + depends on FANOTIFY && CC_CAN_LINK && HEADERS_INSTALL + help + When enabled, this builds samples for fanotify. + There multiple samples for fanotify. Please see the + following configs for more details of these + samples. + config SAMPLE_FANOTIFY_ERROR bool "Build fanotify error monitoring sample" - depends on FANOTIFY && CC_CAN_LINK && HEADERS_INSTALL + depends on SAMPLE_FANOTIFY help When enabled, this builds an example code that uses the FAN_FS_ERROR fanotify mechanism to monitor filesystem errors. See also Documentation/admin-guide/filesystem-monitoring.rst. +config SAMPLE_FANOTIFY_FASTPATH + tristate "Build fanotify fastpath sample" + depends on SAMPLE_FANOTIFY && m + help + When enabled, this builds kernel module that contains a + fanotify fastpath handler. + The fastpath handler filters out certain filename + prefixes for the fanotify user. + config SAMPLE_HIDRAW bool "hidraw sample" depends on CC_CAN_LINK && HEADERS_INSTALL
samples/Makefile +1 -1
@@ -6,7 +6,7 @@ subdir-$(CONFIG_SAMPLE_ANDROID_BINDERFS) += binderfs subdir-$(CONFIG_SAMPLE_CGROUP) += cgroup obj-$(CONFIG_SAMPLE_CONFIGFS) += configfs/ obj-$(CONFIG_SAMPLE_CONNECTOR) += connector/ -obj-$(CONFIG_SAMPLE_FANOTIFY_ERROR) += fanotify/ +obj-$(CONFIG_SAMPLE_FANOTIFY) += fanotify/ subdir-$(CONFIG_SAMPLE_HIDRAW) += hidraw obj-$(CONFIG_SAMPLE_HW_BREAKPOINT) += hw_breakpoint/ obj-$(CONFIG_SAMPLE_KDB) += kdb/
samples/fanotify/.gitignore +1 -0
@@ -1 +1,2 @@ fs-monitor +fastpath-user
samples/fanotify/Makefile +4 -1
@@ -1,5 +1,8 @@ # SPDX-License-Identifier: GPL-2.0-only -userprogs-always-y += fs-monitor +userprogs-always-$(CONFIG_SAMPLE_FANOTIFY_ERROR) += fs-monitor userccflags += -I usr/include -Wall +obj-$(CONFIG_SAMPLE_FANOTIFY_FASTPATH) += fastpath-mod.o + +userprogs-always-$(CONFIG_SAMPLE_FANOTIFY_FASTPATH) += fastpath-user
samples/fanotify/fastpath-mod.c new file +82 -0
82 lines not on this page · raw message
samples/fanotify/fastpath-user.c new file +111 -0
111 lines not on this page · raw message
Signature · 1 line
2.43.5
Song Liu
maintainerPATCH v2 3/7 bpf: Make bpf inode storage available to tracing programs
In reply to Song LiuUse the same recursion voidance mechanism as task local storage.
---
Do not apply this. Another version of this with selftest is submitted separately, with proper selftests.
--- fs/inode.c | 2 + include/linux/bpf.h | 9 ++ include/linux/bpf_lsm.h | 29 ------ include/linux/fs.h | 4 + kernel/bpf/Makefile | 3 +- kernel/bpf/bpf_inode_storage.c | 176 +++++++++++++++++++++++++-------- kernel/bpf/bpf_lsm.c | 4 - kernel/trace/bpf_trace.c | 8 ++ security/bpf/hooks.c | 7 -- 9 files changed, 159 insertions(+), 83 deletions(-)
fs/inode.c +2 -0
@@ -250,6 +250,8 @@ EXPORT_SYMBOL(free_inode_nonrcu);static void i_callback (struct rcu_head *head) { struct inode *inode = container_of(head, struct inode, i_rcu); + +bpf_inode_storage_free (inode);if (inode->free_inode) inode->free_inode (inode);else
include/linux/bpf.h +9 -0
@@ -2630,6 +2630,7 @@ struct bpf_link *bpf_link_by_id(u32 id);const struct bpf_func_proto *bpf_base_func_proto (enum bpf_func_id func_id,const struct bpf_prog *prog);void bpf_task_storage_free (struct task_struct *task); +void bpf_inode_storage_free (struct inode *inode);void bpf_cgrp_storage_free (struct cgroup *cgroup);bool bpf_prog_has_kfunc_call (const struct bpf_prog *prog); const struct btf_func_model *
@@ -2900,6 +2901,10 @@ static inline void bpf_task_storage_free(struct task_struct *task) { } +static inline void bpf_inode_storage_free (struct inode *inode) +{ +} +static inline bool bpf_prog_has_kfunc_call (const struct bpf_prog *prog) {return false;
@@ -3263,6 +3268,10 @@ extern const struct bpf_func_proto bpf_task_storage_get_recur_proto;extern const struct bpf_func_proto bpf_task_storage_get_proto;extern const struct bpf_func_proto bpf_task_storage_delete_recur_proto;extern const struct bpf_func_proto bpf_task_storage_delete_proto; +extern const struct bpf_func_proto bpf_inode_storage_get_proto; +extern const struct bpf_func_proto bpf_inode_storage_get_recur_proto; +extern const struct bpf_func_proto bpf_inode_storage_delete_proto; +extern const struct bpf_func_proto bpf_inode_storage_delete_recur_proto;extern const struct bpf_func_proto bpf_for_each_map_elem_proto;extern const struct bpf_func_proto bpf_btf_find_by_name_kind_proto;extern const struct bpf_func_proto bpf_sk_setsockopt_proto;
include/linux/bpf_lsm.h +0 -29
@@ -19,31 +19,12 @@#include <linux/lsm_hook_defs.h> #undef LSM_HOOK -struct bpf_storage_blob { -struct bpf_local_storage __rcu *storage; -}; - -extern struct lsm_blob_sizes bpf_lsm_blob_sizes; -int bpf_lsm_verify_prog (struct bpf_verifier_log *vlog,const struct bpf_prog *prog);bool bpf_lsm_is_sleepable_hook (u32 btf_id);bool bpf_lsm_is_trusted (const struct bpf_prog *prog); -static inline struct bpf_storage_blob *bpf_inode ( -const struct inode *inode) -{ -if (unlikely (!inode->i_security)) -return NULL ; - -return inode->i_security + bpf_lsm_blob_sizes.lbs_inode; -} - -extern const struct bpf_func_proto bpf_inode_storage_get_proto; -extern const struct bpf_func_proto bpf_inode_storage_delete_proto; -void bpf_inode_storage_free (struct inode *inode); -void bpf_lsm_find_cgroup_shim (const struct bpf_prog *prog,bpf_func_t *bpf_func); int bpf_lsm_get_retval_range(const struct bpf_prog *prog,
@@ -66,16 +47,6 @@ static inline int bpf_lsm_verify_prog(struct bpf_verifier_log *vlog,return -EOPNOTSUPP ; } -static inline struct bpf_storage_blob *bpf_inode ( -const struct inode *inode) -{ -return NULL ; -} - -static inline void bpf_inode_storage_free (struct inode *inode) -{ -} -static inline void bpf_lsm_find_cgroup_shim (const struct bpf_prog *prog,bpf_func_t *bpf_func) {
include/linux/fs.h +4 -0
@@ -79,6 +79,7 @@ struct fs_context;struct fs_parameter_spec ;struct fileattr ;struct iomap_ops ; +struct bpf_local_storage ;extern void __init inode_init (void );extern void __init inode_init_early (void );
@@ -648,6 +649,9 @@ struct inode {#ifdef CONFIG_SECURITY void *i_security;#endif +#ifdef CONFIG_BPF_SYSCALL +struct bpf_local_storage __rcu *i_bpf_storage; +#endif /* Stat data, not accessed from path walking */ unsigned long i_ino;
kernel/bpf/Makefile +1 -2
@@ -10,8 +10,7 @@ obj-$(CONFIG_BPF_SYSCALL) += syscall.o verifier.o inode.o helpers.o tnum.o log.o obj-$(CONFIG_BPF_SYSCALL) += bpf_iter.o map_iter.o task_iter.o prog_iter.o link_iter.o obj-$(CONFIG_BPF_SYSCALL) += hashtab.o arraymap.o percpu_freelist.o bpf_lru_list.o lpm_trie.o map_in_map.o bloom_filter.o obj-$(CONFIG_BPF_SYSCALL) += local_storage.o queue_stack_maps.o ringbuf.o -obj-$(CONFIG_BPF_SYSCALL) += bpf_local_storage.o bpf_task_storage.o -obj-${CONFIG_BPF_LSM} += bpf_inode_storage.o +obj-$(CONFIG_BPF_SYSCALL) += bpf_local_storage.o bpf_task_storage.o bpf_inode_storage.o obj-$(CONFIG_BPF_SYSCALL) += disasm.o mprog.o obj-$(CONFIG_BPF_JIT) += trampoline.o obj-$(CONFIG_BPF_SYSCALL) += btf.o memalloc.o
kernel/bpf/bpf_inode_storage.c +135 -41
284 lines not on this page · raw message
kernel/bpf/bpf_lsm.c +0 -4
@@ -231,10 +231,6 @@ bpf_lsm_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) }switch (func_id) { -case BPF_FUNC_inode_storage_get: -return &bpf_inode_storage_get_proto; -case BPF_FUNC_inode_storage_delete: -return &bpf_inode_storage_delete_proto;#ifdef CONFIG_NET case BPF_FUNC_sk_storage_get:return &bpf_sk_storage_get_proto;
kernel/trace/bpf_trace.c +8 -0
@@ -1529,6 +1529,14 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog)if (bpf_prog_check_recur (prog))return &bpf_task_storage_delete_recur_proto;return &bpf_task_storage_delete_proto; +case BPF_FUNC_inode_storage_get: +if (bpf_prog_check_recur (prog)) +return &bpf_inode_storage_get_recur_proto; +return &bpf_inode_storage_get_proto; +case BPF_FUNC_inode_storage_delete: +if (bpf_prog_check_recur (prog)) +return &bpf_inode_storage_delete_recur_proto; +return &bpf_inode_storage_delete_proto;case BPF_FUNC_for_each_map_elem:return &bpf_for_each_map_elem_proto;case BPF_FUNC_snprintf:
security/bpf/hooks.c +0 -7
@@ -12,8 +12,6 @@ static struct security_hook_list bpf_lsm_hooks[] __ro_after_init = { LSM_HOOK_INIT(NAME, bpf_lsm_##NAME),#include <linux/lsm_hook_defs.h> #undef LSM_HOOK -LSM_HOOK_INIT (inode_free_security, bpf_inode_storage_free), - LSM_HOOK_INIT(task_free, bpf_task_storage_free), };static const struct lsm_id bpf_lsmid = {
@@ -29,12 +27,7 @@ static int __init bpf_lsm_init(void)return 0 ; } -struct lsm_blob_sizes bpf_lsm_blob_sizes__ro_after_init = { - .lbs_inode =sizeof (struct bpf_storage_blob ), -}; -DEFINE_LSM (bpf) = { .name ="bpf" , .init = bpf_lsm_init, - .blobs = &bpf_lsm_blob_sizes };
Signature · 1 line
2.43.5
Song Liu
maintainerPATCH v2 4/7 bpf: fs: Add three kfuncs
In reply to Song LiuAdd the following kfuncs:
- bpf_iput
- bpf_dput
- bpf_is_subdir
These kfuncs can be used by bpf fanotify fastpath.
Both bpf_iput and bpf_dput are marked as KF_SLEEPABLE | KF_RELEASE. They will be used to release reference on inode and dentry.
bpf_is_subdir is marked as KF_RCU. It will be used to take rcu protected pointers, for example, kptr saved to a bpf map.
--- fs/bpf_fs_kfuncs.c | 41 +++++++++++++++++++++++++++++++++++++++++ kernel/bpf/verifier.c | 1 + 2 files changed, 42 insertions(+)
fs/bpf_fs_kfuncs.c +41 -0
@@ -152,6 +152,44 @@ __bpf_kfunc int bpf_get_file_xattr(struct file *file, const char *name__str,return bpf_get_dentry_xattr (dentry, name__str, value_p); } +/** +* bpf_iput - Drop a reference on the inode +* +* @inode: inode to drop reference. +* +* Drop a refcount on inode. +*/ +__bpf_kfunc void bpf_iput(struct inode *inode) +{ +iput (inode); +} + +/** +* bpf_dput - Drop a reference on the dentry +* +* @dentry: dentry to drop reference. +* +* Drop a refcount on dentry. +*/ +__bpf_kfunc void bpf_dput(struct dentry *dentry) +{ +dput (dentry); +} + +/** +* bpf_is_subdir - is new dentry a subdirectory of old_dentry +* @new_dentry: new dentry +* @old_dentry: old dentry +* +* Returns true if new_dentry is a subdirectory of the parent (at any depth). +* Returns false otherwise. +* Caller must ensure that "new_dentry" is pinned before calling is_subdir() +*/ +__bpf_kfunc bool bpf_is_subdir(struct dentry *new_dentry, struct dentry *old_dentry) +{ +return is_subdir (new_dentry, old_dentry); +} +__bpf_kfunc_end_defs ();BTF_KFUNCS_START (bpf_fs_kfunc_set_ids)
@@ -161,6 +199,9 @@ BTF_ID_FLAGS(func, bpf_put_file, KF_RELEASE) BTF_ID_FLAGS(func, bpf_path_d_path, KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_get_dentry_xattr, KF_SLEEPABLE | KF_TRUSTED_ARGS) BTF_ID_FLAGS(func, bpf_get_file_xattr, KF_SLEEPABLE | KF_TRUSTED_ARGS) +BTF_ID_FLAGS(func, bpf_iput, KF_SLEEPABLE | KF_RELEASE) +BTF_ID_FLAGS(func, bpf_dput, KF_SLEEPABLE | KF_RELEASE) +BTF_ID_FLAGS(func, bpf_is_subdir, KF_RCU)BTF_KFUNCS_END (bpf_fs_kfunc_set_ids) static int bpf_fs_kfuncs_filter(const struct bpf_prog *prog, u32 kfunc_id)
kernel/bpf/verifier.c +1 -0
@@ -5432,6 +5432,7 @@ BTF_ID(struct, bpf_cpumask)#endif BTF_ID(struct, task_struct) BTF_ID(struct, bpf_crypto_ctx) +BTF_ID(struct, dentry)BTF_SET_END (rcu_protected_types) static bool rcu_protected_object(const struct btf *btf, u32 btf_id)
Signature · 1 line
2.43.5
Song Liu
maintainerPATCH v2 5/7 bpf: Allow bpf map hold reference on dentry
In reply to Song LiuTo save a dentry in a bpf map, proper logic is needed to free the dentry properly on map termination.
--- kernel/bpf/helpers.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
kernel/bpf/helpers.c +13 -1
@@ -3004,6 +3004,12 @@ __bpf_kfunc int bpf_copy_from_user_str(void *dst, u32 dst__sz, const void __userreturn ret +1 ; } +__bpf_kfunc void bpf_dentry_release_dtor(struct dentry *dentry) +{ +dput (dentry); +} +CFI_NOSEAL (bpf_dentry_release_dtor); +__bpf_kfunc_end_defs ();BTF_KFUNCS_START (generic_btf_ids)
@@ -3046,6 +3052,8 @@ static const struct btf_kfunc_id_set generic_kfunc_set = { BTF_ID_LIST(generic_dtor_ids) BTF_ID(struct, task_struct) BTF_ID(func, bpf_task_release_dtor) +BTF_ID(struct, dentry) +BTF_ID(func, bpf_dentry_release_dtor) #ifdef CONFIG_CGROUPS BTF_ID(struct, cgroup) BTF_ID(func, bpf_cgroup_release_dtor)
@@ -3105,11 +3113,15 @@ static int __init kfunc_init(void) .btf_id = generic_dtor_ids[0], .kfunc_btf_id = generic_dtor_ids[1] }, -#ifdef CONFIG_CGROUPS { .btf_id = generic_dtor_ids[2], .kfunc_btf_id = generic_dtor_ids[3] }, +#ifdef CONFIG_CGROUPS + { + .btf_id = generic_dtor_ids[4], + .kfunc_btf_id = generic_dtor_ids[5] + }, #endif };
Signature · 1 line
2.43.5
Song Liu
maintainerPATCH v2 6/7 fanotify: Enable bpf based fanotify fastpath handler
In reply to Song LiuAllow user to write fanotify fastpath handlers with bpf programs.
Major changes:
1. Make kfuncs in fs/bpf_fs_kfuncs.c available to STRUCT_OPS programs.
2. Add kfunc bpf_fanotify_data_inode, bpf_fanotify_data_dentry. e. Add struct_ops bpf_fanotify_fastpath_ops.
TODO:
1. With current logic, the bpf based fastpath handler is added to the global list, and thus available to all users. This is similar to bpf based tcp congestion algorithms. It is possible to add an API so that the bpf based handler is not added to global list, which is similar to hid-bpf. I plan to add that API later.
--- fs/Makefile | 2 +- fs/bpf_fs_kfuncs.c | 10 +- fs/notify/fanotify/fanotify_fastpath.c | 172 ++++++++++++++++++++++++- kernel/bpf/verifier.c | 5 + 4 files changed, 183 insertions(+), 6 deletions(-)
fs/Makefile +1 -1
@@ -129,4 +129,4 @@ obj-$(CONFIG_EFIVAR_FS) += efivarfs/ obj-$(CONFIG_EROFS_FS) += erofs/ obj-$(CONFIG_VBOXSF_FS) += vboxsf/ obj-$(CONFIG_ZONEFS_FS) += zonefs/ -obj-$(CONFIG_BPF_LSM) += bpf_fs_kfuncs.o +obj-$(CONFIG_BPF_SYSCALL) += bpf_fs_kfuncs.o
fs/bpf_fs_kfuncs.c +8 -2
@@ -207,7 +207,8 @@ BTF_KFUNCS_END(bpf_fs_kfunc_set_ids)static int bpf_fs_kfuncs_filter (const struct bpf_prog *prog,u32 kfunc_id) {if (!btf_id_set8_contains (&bpf_fs_kfunc_set_ids, kfunc_id) || - prog->type ==BPF_PROG_TYPE_LSM ) + prog->type ==BPF_PROG_TYPE_LSM || + prog->type ==BPF_PROG_TYPE_STRUCT_OPS )return 0 ;return -EACCES ; }
@@ -220,7 +221,12 @@ static const struct btf_kfunc_id_set bpf_fs_kfunc_set = {static int __init bpf_fs_kfuncs_init (void ) { -return register_btf_kfunc_id_set (BPF_PROG_TYPE_LSM , &bpf_fs_kfunc_set); +int ret; + + ret =register_btf_kfunc_id_set (BPF_PROG_TYPE_LSM , &bpf_fs_kfunc_set); + ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_STRUCT_OPS , &bpf_fs_kfunc_set); + +return ret; }late_initcall (bpf_fs_kfuncs_init);
fs/notify/fanotify/fanotify_fastpath.c +169 -3
199 lines not on this page · raw message
kernel/bpf/verifier.c +5 -0
@@ -6529,6 +6529,10 @@ BTF_TYPE_SAFE_TRUSTED(struct dentry) {struct inode *d_inode; }; +BTF_TYPE_SAFE_TRUSTED(struct fanotify_fastpath_event) { +struct inode *dir; +}; + BTF_TYPE_SAFE_TRUSTED_OR_NULL(struct socket) {struct sock *sk; };
@@ -6564,6 +6568,7 @@ static bool type_is_trusted(struct bpf_verifier_env *env,BTF_TYPE_EMIT (BTF_TYPE_SAFE_TRUSTED(struct linux_binprm ));BTF_TYPE_EMIT (BTF_TYPE_SAFE_TRUSTED(struct file ));BTF_TYPE_EMIT (BTF_TYPE_SAFE_TRUSTED(struct dentry )); +BTF_TYPE_EMIT (BTF_TYPE_SAFE_TRUSTED(struct fanotify_fastpath_event ));return btf_nested_type_is_trusted (&env->log, reg, field_name, btf_id,"__safe_trusted" ); }
Signature · 1 line
2.43.5
kernel test robot
build botRe: [RFC/PATCH v2 bpf-next fanotify 6/7] fanotify: Enable bpf based fanotify fastpath handler
kernel test robot noticed the following build errors:
Hi Song,
[This is a private test report for your RFC patch.] kernel test robot noticed the following build errors:
[auto build test ERROR on jack-fs/fsnotify]
[also build test ERROR on linus/master bpf/master v6.12-rc7] [cannot apply to bpf-next/master next-20241115]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Song-Liu/fanotify-Introduce-fanotify-fastpath-handler/20241114-164824 base: https://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs.git fsnotify patch link: https://lore.kernel.org/r/20241114084345.1564165-7-song%40kernel.org patch subject: [RFC/PATCH v2 bpf-next fanotify 6/7] fanotify: Enable bpf based fanotify fastpath handler config: x86_64-randconfig-161-20241117 (https://download.01.org/0day-ci/archive/20241117/[email protected]/config) compiler: clang version 19.1.3 (https://github.com/llvm/llvm-project ab51eccf88f5321e7c60591c5546b254b6afab99) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20241117/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
In file included from fs/notify/fanotify/fanotify_fastpath.c:5:
In file included from include/linux/bpf.h:21:
In file included from include/linux/kallsyms.h:13:
In file included from include/linux/mm.h:2213:
include/linux/vmstat.h:504:43: warning: arithmetic between different enumeration types ('enum zone_stat_item' and 'enum numa_stat_item') [-Wenum-enum-conversion]
504 | return vmstat_text[NR_VM_ZONE_STAT_ITEMS +
| ~~~~~~~~~~~~~~~~~~~~~ ^
505 | item];
| ~~~~
include/linux/vmstat.h:511:43: warning: arithmetic between different enumeration types ('enum zone_stat_item' and 'enum numa_stat_item') [-Wenum-enum-conversion]
511 | return vmstat_text[NR_VM_ZONE_STAT_ITEMS +
| ~~~~~~~~~~~~~~~~~~~~~ ^
512 | NR_VM_NUMA_EVENT_ITEMS +
| ~~~~~~~~~~~~~~~~~~~~~~
include/linux/vmstat.h:518:36: warning: arithmetic between different enumeration types ('enum node_stat_item' and 'enum lru_list') [-Wenum-enum-conversion]
518 | return node_stat_name(NR_LRU_BASE + lru) + 3; // skip "nr_"
| ~~~~~~~~~~~ ^ ~~~
>> fs/notify/fanotify/fanotify_fastpath.c:347:7: error: call to undeclared function 'bpf_tracing_btf_ctx_access'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] 347 | if (!bpf_tracing_btf_ctx_access(off, size, type, prog, info))
| ^
3 warnings and 1 error generated.
vim +/bpf_tracing_btf_ctx_access +347 fs/notify/fanotify/fanotify_fastpath.c
341
342 static bool bpf_fanotify_fastpath_is_valid_access(int off, int size,
343 enum bpf_access_type type,
344 const struct bpf_prog *prog,
345 struct bpf_insn_access_aux *info)
346 {
> 347 if (!bpf_tracing_btf_ctx_access(off, size, type, prog, info)) 348 return false;
349
350 return true;
351 }
352
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
kernel test robot
build botRe: [RFC/PATCH v2 bpf-next fanotify 6/7] fanotify: Enable bpf based fanotify fastpath handler
In reply to Song Liukernel test robot noticed the following build errors:
Hi Song,
[This is a private test report for your RFC patch.] kernel test robot noticed the following build errors:
[auto build test ERROR on jack-fs/fsnotify]
[also build test ERROR on linus/master bpf/master v6.12-rc7] [cannot apply to bpf-next/master next-20241115]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Song-Liu/fanotify-Introduce-fanotify-fastpath-handler/20241114-164824 base: https://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs.git fsnotify patch link: https://lore.kernel.org/r/20241114084345.1564165-7-song%40kernel.org patch subject: [RFC/PATCH v2 bpf-next fanotify 6/7] fanotify: Enable bpf based fanotify fastpath handler config: loongarch-randconfig-001-20241117 (https://download.01.org/0day-ci/archive/20241117/[email protected]/config) compiler: loongarch64-linux-gcc (GCC) 14.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20241117/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
fs/notify/fanotify/fanotify_fastpath.c: In function 'bpf_fanotify_fastpath_is_valid_access':
>> fs/notify/fanotify/fanotify_fastpath.c:347:14: error: implicit declaration of function 'bpf_tracing_btf_ctx_access' [-Wimplicit-function-declaration]
347 | if (!bpf_tracing_btf_ctx_access(off, size, type, prog, info))
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from fs/notify/fanotify/fanotify_fastpath.c:5:
fs/notify/fanotify/fanotify_fastpath.c: In function 'bpf_fanotify_fastpath_struct_ops_init':
include/linux/bpf.h:1860:50: warning: statement with no effect [-Wunused-value]
1860 | #define register_bpf_struct_ops(st_ops, type) ({ (void *)(st_ops); 0; })
| ^~~~~~~~~~~~~~~~
fs/notify/fanotify/fanotify_fastpath.c:445:22: note: in expansion of macro 'register_bpf_struct_ops'
445 | ret = ret ?: register_bpf_struct_ops(&bpf_fanotify_fastpath_ops, fanotify_fastpath_ops);
| ^~~~~~~~~~~~~~~~~~~~~~~vim +/bpf_tracing_btf_ctx_access +347 fs/notify/fanotify/fanotify_fastpath.c
341
342 static bool bpf_fanotify_fastpath_is_valid_access(int off, int size,
343 enum bpf_access_type type,
344 const struct bpf_prog *prog,
345 struct bpf_insn_access_aux *info)
346 {
> 347 if (!bpf_tracing_btf_ctx_access(off, size, type, prog, info)) 348 return false;
349
350 return true;
351 }
352
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Song Liu
maintainerPATCH v2 7/7 selftests/bpf: Add test for BPF based fanotify fastpath handler
In reply to Song LiuThis test shows a simplified logic that monitors a subtree. This is simplified as it doesn't handle all the scenarios, such as:
1) moving a subsubtree into/outof the being monitoring subtree;
2) mount point inside the being monitored subtree
Therefore, this is not to show a way to reliably monitor a subtree. Instead, this is to test the functionalities of bpf based fastpath. To really monitor a subtree reliably, we will need more complex logic.
Overview of the logic:
1. fanotify is created for the whole file system (/tmp).
2. dentry of the subtree root is saved in map subtree_root.
3. bpf_is_subdir() is used to check whether a fanotify event happens inside the subtree. Only events happened in the subtree are passed to userspace.
4. A bpf map (inode_storage_map) is used to cache result from
bpf_is_subdir().
5. subsubtree moving is not handled. This is because we don't yet have a good way to walk a subtree from BPF (something similar to d_walk).
--- tools/testing/selftests/bpf/bpf_kfuncs.h | 5 + tools/testing/selftests/bpf/config | 2 + .../testing/selftests/bpf/prog_tests/fan_fp.c | 264 ++++++++++++++++++ tools/testing/selftests/bpf/progs/fan_fp.c | 154 ++++++++++ 4 files changed, 425 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/fan_fp.c create mode 100644 tools/testing/selftests/bpf/progs/fan_fp.c
tools/testing/selftests/bpf/bpf_kfuncs.h +5 -0
@@ -87,4 +87,9 @@ struct dentry;*/ extern int bpf_get_dentry_xattr(struct dentry *dentry, const char *name, struct bpf_dynptr *value_ptr) __ksym __weak; + +struct fanotify_fastpath_event ; +extern struct inode *bpf_fanotify_data_inode (struct fanotify_fastpath_event *event) __ksym__weak ; +extern void bpf_iput(struct inode *inode) __ksym __weak; +extern bool bpf_is_subdir(struct dentry *new_dentry, struct dentry *old_dentry) __ksym __weak;#endif
tools/testing/selftests/bpf/config +2 -0
@@ -24,6 +24,8 @@ CONFIG_DEBUG_INFO_BTF=y CONFIG_DEBUG_INFO_DWARF4=y CONFIG_DUMMY=y CONFIG_DYNAMIC_FTRACE=y +CONFIG_FANOTIFY=y +CONFIG_FANOTIFY_FASTPATH=y CONFIG_FPROBE=y CONFIG_FTRACE_SYSCALLS=y CONFIG_FUNCTION_ERROR_INJECTION=y
tools/testing/selftests/bpf/prog_tests/fan_fp.c new file +264 -0
264 lines not on this page · raw message
tools/testing/selftests/bpf/progs/fan_fp.c new file +154 -0
154 lines not on this page · raw message
Signature · 1 line
2.43.5
Re: [RFC/PATCH v2 bpf-next fanotify 7/7] selftests/bpf: Add test for BPF based fanotify fastpath handler
tools/testing/selftests/bpf/progs/fan_fp.c line 122
Quoted 6 lines · Song Liu
tools/testing/selftests/bpf/progs/fan_fp.c new file +5
-0+ +if (bpf_is_subdir (dentry, v->dentry)) + ret =FAN_FP_RET_SEND_TO_USERSPACE ; +else + ret =FAN_FP_RET_SKIP_EVENT ;
It seems to me that all these patches and feature additions to fanotify, new kfuncs, etc are done just to do the above filtering by subdir ?
If so, just hard code this logic as an extra flag to fanotify ? So it can filter all events by subdir.
bpf programmability makes sense when it needs to express user space policy. Here it's just a filter by subdir. bpf hammer doesn't look like the right tool for this use case.
Re: [RFC/PATCH v2 bpf-next fanotify 7/7] selftests/bpf: Add test for BPF based fanotify fastpath handler
Quoted 16 lines · Alexei Starovoitov
tools/testing/selftests/bpf/progs/fan_fp.c line 122
Quoted 6 lines · Song Liu
tools/testing/selftests/bpf/progs/fan_fp.c new file +5
-0+ +if (bpf_is_subdir (dentry, v->dentry)) + ret =FAN_FP_RET_SEND_TO_USERSPACE ; +else + ret =FAN_FP_RET_SKIP_EVENT ;It seems to me that all these patches and feature additions to fanotify, new kfuncs, etc are done just to do the above filtering by subdir ?
Quoted 5 lines
If so, just hard code this logic as an extra flag to fanotify ? So it can filter all events by subdir.
bpf programmability makes sense when it needs to express user space policy. Here it's just a filter by subdir. bpf hammer doesn't look like the right tool for this use case.
Current version is indeed tailored towards the subtree monitoring use case. This is mostly because feedback on v1 mostly focused on this use case. V1 itself actually had some other use cases.
In practice, fanotify fastpath can benefit from bpf programmability. For example, with bpf programmability, we can combine fanotify and BPF LSM in some security use cases. If some security rules only applies to a few files, a directory, or a subtree, we can use fanotify to only monitor these files. LSM hooks, such as security_file_open(), are always global. The overhead is higher if we are only interested in a few files.
Does this make sense?
Thanks,
Song
Re: [RFC/PATCH v2 bpf-next fanotify 7/7] selftests/bpf: Add test for BPF based fanotify fastpath handler
Quoted 23 lines · Song Liu
Quoted 17 lines · Alexei Starovoitov
tools/testing/selftests/bpf/progs/fan_fp.c line 122
Quoted 6 lines · Song Liu
tools/testing/selftests/bpf/progs/fan_fp.c new file +5
-0+ +if (bpf_is_subdir (dentry, v->dentry)) + ret =FAN_FP_RET_SEND_TO_USERSPACE ; +else + ret =FAN_FP_RET_SKIP_EVENT ;It seems to me that all these patches and feature additions to fanotify, new kfuncs, etc are done just to do the above filtering by subdir ?
Quoted 5 lines
If so, just hard code this logic as an extra flag to fanotify ? So it can filter all events by subdir.
bpf programmability makes sense when it needs to express user space policy. Here it's just a filter by subdir. bpf hammer doesn't look like the right tool for this use case.
Quoted 4 lines
Current version is indeed tailored towards the subtree monitoring use case. This is mostly because feedback on v1 mostly focused on this use case. V1 itself actually had some other use cases.
like?
Quoted 10 lines
In practice, fanotify fastpath can benefit from bpf programmability. For example, with bpf programmability, we can combine fanotify and BPF LSM in some security use cases. If some security rules only applies to a few files, a directory, or a subtree, we can use fanotify to only monitor these files. LSM hooks, such as
security_file_open(), are always global. The overhead is higher if we are only interested in a few files.Does this make sense?
Not yet.
This fanotify bpf filtering only reduces the number of events sent to user space.
How is it supposed to interact with bpf-lsm?
Say, security policy applies to /usr/bin/*
so lsm suppose to act on all files and subdirs in there. How fanotify helps ?
Re: [RFC/PATCH v2 bpf-next fanotify 7/7] selftests/bpf: Add test for BPF based fanotify fastpath handler
Quoted 28 lines · Alexei Starovoitov
Quoted 23 lines · Song Liu
Quoted 22 lines · Alexei Starovoitov
tools/testing/selftests/bpf/progs/fan_fp.c line 122
Quoted 6 lines · Song Liu
tools/testing/selftests/bpf/progs/fan_fp.c new file +5
-0+ +if (bpf_is_subdir (dentry, v->dentry)) + ret =FAN_FP_RET_SEND_TO_USERSPACE ; +else + ret =FAN_FP_RET_SKIP_EVENT ;It seems to me that all these patches and feature additions to fanotify, new kfuncs, etc are done just to do the above filtering by subdir ?
If so, just hard code this logic as an extra flag to fanotify ? So it can filter all events by subdir.
bpf programmability makes sense when it needs to express user space policy. Here it's just a filter by subdir. bpf hammer doesn't look like the right tool for this use case.
Quoted 6 lines
Quoted 4 lines
Current version is indeed tailored towards the subtree monitoring use case. This is mostly because feedback on v1 mostly focused on this use case. V1 itself actually had some other use cases.
like?
samples/fanotify in v1 shows pattern that matches file prefix (no BPF). selftests/bpf in v1 shows a pattern where we propagate a flag in inode local storage from parent directory to newly created children directory.
Quoted 10 lines
Quoted 10 lines
In practice, fanotify fastpath can benefit from bpf programmability. For example, with bpf programmability, we can combine fanotify and BPF LSM in some security use cases. If some security rules only applies to a few files, a directory, or a subtree, we can use fanotify to only monitor these files. LSM hooks, such as
security_file_open(), are always global. The overhead is higher if we are only interested in a few files.
Quoted 6 lines
Quoted 1 line
Does this make sense?
Not yet.
This fanotify bpf filtering only reduces the number of events sent to user space.
How is it supposed to interact with bpf-lsm?
Ah, I didn't explain this part. fanotify+bpf fastpath can do more reducing number of events sent to user space. It can also be used in tracing use cases. For example, we can implement a filetop tool that only monitors a specific directory, a specific device, or a specific mount point. It can also reject some file access (fanotify permission mode). I should have showed these features in a sample and/or selftest.
Quoted 3 lines
Say, security policy applies to
/usr/bin/*
so lsm suppose to act on all files and subdirs in there. How fanotify helps ?
LSM hooks are always global. It is up to the BPF program to filter out irrelevant events. This filtering is sometimes expensive (match d_patch) and inaccurate (maintain a map of target inodes, etc.). OTOH, fanotify has built-in filtering before the BPF program triggers. When multiple BPF programs are monitoring open() for different subdirectories, fanotify based solution will not trigger all these BPF programs for all the open() in the system.
Does this answer the questions?
Thanks,
Song
Re: [RFC/PATCH v2 bpf-next fanotify 7/7] selftests/bpf: Add test for BPF based fanotify fastpath handler
Quoted 58 lines · Song Liu
Quoted 29 lines · Alexei Starovoitov
Quoted 23 lines · Song Liu
Quoted 22 lines · Alexei Starovoitov
tools/testing/selftests/bpf/progs/fan_fp.c line 122
Quoted 6 lines · Song Liu
tools/testing/selftests/bpf/progs/fan_fp.c new file +5
-0+ +if (bpf_is_subdir (dentry, v->dentry)) + ret =FAN_FP_RET_SEND_TO_USERSPACE ; +else + ret =FAN_FP_RET_SKIP_EVENT ;It seems to me that all these patches and feature additions to fanotify, new kfuncs, etc are done just to do the above filtering by subdir ?
If so, just hard code this logic as an extra flag to fanotify ? So it can filter all events by subdir.
bpf programmability makes sense when it needs to express user space policy. Here it's just a filter by subdir. bpf hammer doesn't look like the right tool for this use case.Quoted 6 lines
Quoted 4 lines
Current version is indeed tailored towards the subtree monitoring use case. This is mostly because feedback on v1 mostly focused on this use case. V1 itself actually had some other use cases.
like?
samples/fanotifyin v1 shows pattern that matches file prefix (no BPF). selftests/bpf in v1 shows a pattern where we propagate a flag in inode local storage from parent directory to newly created children directory.Quoted 10 lines
Quoted 10 lines
In practice, fanotify fastpath can benefit from bpf programmability. For example, with bpf programmability, we can combine fanotify and BPF LSM in some security use cases. If some security rules only applies to a few files, a directory, or a subtree, we can use fanotify to only monitor these files. LSM hooks, such as
security_file_open(), are always global. The overhead is higher if we are only interested in a few files.Quoted 6 lines
Quoted 1 line
Does this make sense?
Not yet.
This fanotify bpf filtering only reduces the number of events sent to user space.
How is it supposed to interact with bpf-lsm?
Quoted 8 lines
Ah, I didn't explain this part. fanotify+bpf fastpath can do more reducing number of events sent to user space. It can also be used in tracing use cases. For example, we can implement a filetop tool that only monitors a specific directory, a specific device, or a specific mount point. It can also reject some file access (fanotify permission mode). I should have showed these features in a sample and/or selftest.
I bet bpf tracing can filetop already.
Not as efficient, but tracing isn't going to be running 24/7.
Quoted 15 lines
Quoted 4 lines
Say, security policy applies to
/usr/bin/*
so lsm suppose to act on all files and subdirs in there. How fanotify helps ?LSM hooks are always global. It is up to the BPF program to filter out irrelevant events. This filtering is sometimes expensive (match
d_patch) and inaccurate (maintain a map of target inodes, etc.). OTOH, fanotify has built-in filtering before the BPF program triggers. When multiple BPF programs are monitoringopen()for different subdirectories, fanotify based solution will not trigger all these BPF programs for all theopen()in the system.
Quoted 1 line
Does this answer the questions?
No. Above is too much hand waving.
I think bpf-lsm hook fires before fanotify, so bpf-lsm prog implementing some security policy has to decide right at the moment what to do with, say, security_file_open(). fanotify with or without bpf fastpath is too late.
In general fanotify is not for security. It's notifying user space of events that already happened, so I don't see how these two can be combined.
Re: [RFC/PATCH v2 bpf-next fanotify 7/7] selftests/bpf: Add test for BPF based fanotify fastpath handler
Quoted 3 lines · Alexei Starovoitov
[...]
Quoted 33 lines
Quoted 6 lines
Quoted 5 lines
Not yet.
This fanotify bpf filtering only reduces the number of events sent to user space.
How is it supposed to interact with bpf-lsm?Quoted 8 lines
Ah, I didn't explain this part. fanotify+bpf fastpath can do more reducing number of events sent to user space. It can also be used in tracing use cases. For example, we can implement a filetop tool that only monitors a specific directory, a specific device, or a specific mount point. It can also reject some file access (fanotify permission mode). I should have showed these features in a sample and/or selftest.
I bet bpf tracing can filetop already.
Not as efficient, but tracing isn't going to be running 24/7.Quoted 15 lines
Quoted 4 lines
Say, security policy applies to
/usr/bin/*
so lsm suppose to act on all files and subdirs in there. How fanotify helps ?LSM hooks are always global. It is up to the BPF program to filter out irrelevant events. This filtering is sometimes expensive (match
d_patch) and inaccurate (maintain a map of target inodes, etc.). OTOH, fanotify has built-in filtering before the BPF program triggers. When multiple BPF programs are monitoringopen()for different subdirectories, fanotify based solution will not trigger all these BPF programs for all theopen()in the system.
Quoted 8 lines
Quoted 1 line
Does this answer the questions?
No. Above is too much hand waving.
I think bpf-lsm hook fires before fanotify, so bpf-lsm prog implementing some security policy has to decide right at the moment what to do with, say,
security_file_open(). fanotify with or without bpf fastpath is too late.
Actually, fanotify in permission mode can stop a file open. In current upstream code, fsnotify hook fsnotify_open_perm is actually part of security_file_open(). It will be moved to do_dentry_open(), right after security_file_open(). This move is done by 1cda52f1b461 in linux-next.
In practice, we are not likely to use BPF LSM and fanotify on the same hook at the same time. Instead, we can use BPF LSM hooks to gather information and use fanotify to make allow/deny decision, or vice versa.
Quoted 3 lines
In general fanotify is not for security. It's notifying user space of events that already happened, so I don't see how these two can be combined.
fanotify is actually used by AntiVirus softwares. For example, CalmAV (https://www.clamav.net/) uses fanotify for its Linux version (it also supports Window and MacOS).
I guess I didn't state the motivation clearly. So let me try it now.
Tracing is a critical part of a security solution. With LSM, blocking an operation is straightforward. However, knowing which operation should be blocked is not always easy. Also, security hooks (LSM or fanotify) sit in the critical path of user requests. It is very important to optimize the latency of a security hook. Ideally, the tracing logic should gather all the information ahead of time, and make the actual hook fast.
For example, if security_file_open() only needs to read a flag from inode local storage, the overhead is minimal and predictable. If security_file_open() has to walk the dentry tree, or call d_path(), the overhead will be much higher. fanotify_file_perm() provides another level of optimization over security_file_open(). If a file is not being monitored, fanotify will not generate the event.
Security solutions hold higher bars for the tracing logic:
- It needs to be accurate, as false positives and false negatives can be very annoying and/or harmful.
- It needs to be efficient, as security daemons run 24/7.
Given these requirements of security solutions, I believe it is important to optimize tracing logic as much as possible. And BPF based fanotify fastpath handler can bring non-trivials benefit to BPF based security solutions.
fanotify also has a feature that LSM doesn't provide. When a file is accessed, user space daemon can get a fd on this file from fanotify. OTOH, LSM can only send an ino or a path to user space, which is not always reliable.
I hope this starts to make sense...
Thanks,
Song
Re: [RFC/PATCH v2 bpf-next fanotify 7/7] selftests/bpf: Add test for BPF based fanotify fastpath handler
Quoted 8 lines · Song Liu
Quoted 6 lines
I think bpf-lsm hook fires before fanotify, so bpf-lsm prog implementing some security policy has to decide right at the moment what to do with, say,
security_file_open(). fanotify with or without bpf fastpath is too late.Actually, fanotify in permission mode can stop a file open.
The proposed patch 1 did:
+/* Return value of fp_handler */
+enum fanotify_fastpath_return {
+ /* The event should be sent to user space */
+ FAN_FP_RET_SEND_TO_USERSPACE = 0,
+ /* The event should NOT be sent to user space */
+ FAN_FP_RET_SKIP_EVENT = 1,
+};
It looked like a read-only notification to user space where bpf prog is merely a filter.
Quoted 4 lines
In current upstream code, fsnotify hook
fsnotify_open_permis actually part ofsecurity_file_open(). It will be moved todo_dentry_open(), right aftersecurity_file_open(). This move is done by1cda52f1b461in linux-next.
Separating fsnotify from LSM makes sense.
Quoted 4 lines
In practice, we are not likely to use BPF LSM and fanotify on the same hook at the same time. Instead, we can use BPF LSM hooks to gather information and use fanotify to make allow/deny decision, or vice versa.
Pick one.
If the proposal is changing to let fsnotify-bpf prog to deny file_open then it's a completely different discussion.
In such a case make it clear upfront that fsnotify will rely on CONFIG_FANOTIFY_ACCESS_PERMISSIONS and
bpf-lsm part of file access will not be used,
since interaction of two callbacks at file_open makes little sense.
Quoted 7 lines
Quoted 3 lines
In general fanotify is not for security. It's notifying user space of events that already happened, so I don't see how these two can be combined.
fanotify is actually used by AntiVirus softwares. For example, CalmAV (https://www.clamav.net/) uses fanotify for its Linux version (it also supports Window and MacOS).
It's relying on user space to send back FANOTIFY_PERM_EVENTS ?
fsnotify_open_perm->fsnotify->send_to_group->fanotify_handle_event.
is a pretty long path to call bpf prog and preparing a giant 'struct fanotify_fastpath_event' is not going to fast either.
If we want to accelerate that with bpf it needs to be done sooner with negligible overhead.
Quoted 13 lines
I guess I didn't state the motivation clearly. So let me try it now.
Tracing is a critical part of a security solution. With LSM, blocking an operation is straightforward. However, knowing which operation should be blocked is not always easy. Also, security hooks (LSM or fanotify) sit in the critical path of user requests. It is very important to optimize the latency of a security hook. Ideally, the tracing logic should gather all the information ahead of time, and make the actual hook fast.
Quoted 8 lines
For example, if
security_file_open()only needs to read a flag from inode local storage, the overhead is minimal and predictable. Ifsecurity_file_open()has to walk the dentry tree, or calld_path(), the overhead will be much higher.fanotify_file_perm()provides another level of optimization oversecurity_file_open(). If a file is not being monitored, fanotify will not generate the event.
I agree with motivation, but don't see this in the patches. The overhead to call into bpf prog is big.
Even if prog does nothing it's still going to be slower.
Quoted 10 lines
Security solutions hold higher bars for the tracing logic:
- It needs to be accurate, as false positives and false negatives can be very annoying and/or harmful.
- It needs to be efficient, as security daemons run 24/7.Given these requirements of security solutions, I believe it is important to optimize tracing logic as much as possible. And BPF based fanotify fastpath handler can bring non-trivials benefit to BPF based security solutions.
Doing everything in the kernel is certainly faster than going back and forth to user space,
but bpf-lsm should be able to do the same already.
Without patch 1 and only patches 4,5 that add few kfuncs, bpf-lsm prog will be able to remember subtree dentry and do the same is_subdir() to deny.
The patch 7 stays pretty much as-is. All in bpf-lsm. Close to zero overhead without long chain of fsnotify callbacks.
Quoted 5 lines
fanotify also has a feature that LSM doesn't provide. When a file is accessed, user space daemon can get a fd on this file from fanotify. OTOH, LSM can only send an ino or a path to user space, which is not always reliable.
That sounds useful, but we're mixing too many things. If user space cares about fd it will be using the existing mechanism with all accompanied overhead. fsnotify-bpf can barely accelerate anything, since user space makes ultimate decisions.
If user space is not in the driving seat then existing bpf-lsm plus few kfuncs to remember dentry and call is_subdir() will do the job and no need for patch 1.
Re: [RFC/PATCH v2 bpf-next fanotify 7/7] selftests/bpf: Add test for BPF based fanotify fastpath handler
Quoted 24 lines · Alexei Starovoitov
Quoted 8 lines · Song Liu
Quoted 6 lines
I think bpf-lsm hook fires before fanotify, so bpf-lsm prog implementing some security policy has to decide right at the moment what to do with, say,
security_file_open(). fanotify with or without bpf fastpath is too late.Actually, fanotify in permission mode can stop a file open.
The proposed patch 1 did:
(quoted diff) +7
-0+/* Return value of fp_handler */ +enum fanotify_fastpath_return { + /* The event should be sent to user space */ + FAN_FP_RET_SEND_TO_USERSPACE = 0, + /* The event should NOT be sent to user space */ + FAN_FP_RET_SKIP_EVENT = 1, +};
Quoted 2 lines
It looked like a read-only notification to user space where bpf prog is merely a filter.
Yep. As Amir also pointed out, this part needs more work and clarifications.
Quoted 13 lines
Quoted 5 lines
In current upstream code, fsnotify hook
fsnotify_open_permis actually part ofsecurity_file_open(). It will be moved todo_dentry_open(), right aftersecurity_file_open(). This move is done by1cda52f1b461in linux-next.Separating fsnotify from LSM makes sense.
Quoted 4 lines
In practice, we are not likely to use BPF LSM and fanotify on the same hook at the same time. Instead, we can use BPF LSM hooks to gather information and use fanotify to make allow/deny decision, or vice versa.
Quoted 6 lines
Pick one.
If the proposal is changing to let fsnotify-bpf prog to denyfile_openthen it's a completely different discussion.In such a case make it clear upfront that fsnotify will rely on
CONFIG_FANOTIFY_ACCESS_PERMISSIONSand
I am not sure whether we should limit fsnotify-bpf to only work with CONFIG_FANOTIFY_ACCESS_PERMISSIONS. I still think it can be useful just as a filter. But I guess we can start with this dependency.
Quoted 2 lines
bpf-lsm part of file access will not be used,
since interaction of two callbacks atfile_openmakes little sense.
Agreed that having two hooks on file_open doesn't make sense. I was actually thinking about combining fsnotify-bpf with other LSM hooks. But I agree we should start as simple as possible.
Quoted 10 lines
Quoted 8 lines
Quoted 4 lines
In general fanotify is not for security. It's notifying user space of events that already happened, so I don't see how these two can be combined.
fanotify is actually used by AntiVirus softwares. For
example, CalmAV (https://urldefense.com/v3/__https://www.clamav.net/__;!!Bt8RZUm9aw!4p7JB_E8RuNLldz_TYR07jnW5kHbr4H2FMm9vLbShKOBMznXwES7Dlt5_R6B_-HMzgV3Qk_9WKlhmjHSpYHRhTb2WM3hIg$ ) uses fanotify for its Linux version (it also supports Window and MacOS).It's relying on user space to send back
FANOTIFY_PERM_EVENTS?
Yes, it goes all the way to another process, and comes back.
Quoted 9 lines
fsnotify_open_perm->fsnotify->send_to_group->fanotify_handle_event.is a pretty long path to call bpf prog and preparing a giant 'struct
fanotify_fastpath_event' is not going to fast either.If we want to accelerate that with bpf it needs to be done sooner with negligible overhead.
Agreed. This is actually something I have been thinking since the beginning of this work: Shall it be fanotify-bpf or fsnotify-bpf. Given we have more materials, this is a good time to have broader discussions on this.
@all, please chime in whether we should redo this as fsnotify-bpf. AFAICT:
Pros of fanotify-bpf:
- There is existing user space that we can leverage/reuse.
Pros of fsnotify-bpf:
- Faster fast path.
Another major pros/cons did I miss?
Quoted 22 lines
Quoted 13 lines
I guess I didn't state the motivation clearly. So let me try it now.
Tracing is a critical part of a security solution. With LSM, blocking an operation is straightforward. However, knowing which operation should be blocked is not always easy. Also, security hooks (LSM or fanotify) sit in the critical path of user requests. It is very important to optimize the latency of a security hook. Ideally, the tracing logic should gather all the information ahead of time, and make the actual hook fast.
Quoted 8 lines
For example, if
security_file_open()only needs to read a flag from inode local storage, the overhead is minimal and predictable. Ifsecurity_file_open()has to walk the dentry tree, or calld_path(), the overhead will be much higher.fanotify_file_perm()provides another level of optimization oversecurity_file_open(). If a file is not being monitored, fanotify will not generate the event.
Quoted 1 line
I agree with motivation, but don't see this in the patches.
Agreed. I should definitely do better job in this.
Quoted 18 lines
The overhead to call into bpf prog is big.
Even if prog does nothing it's still going to be slower.Quoted 10 lines
Security solutions hold higher bars for the tracing logic:
- It needs to be accurate, as false positives and false negatives can be very annoying and/or harmful.
- It needs to be efficient, as security daemons run 24/7.Given these requirements of security solutions, I believe it is important to optimize tracing logic as much as possible. And BPF based fanotify fastpath handler can bring non-trivials benefit to BPF based security solutions.
Doing everything in the kernel is certainly faster than going back and forth to user space,
but bpf-lsm should be able to do the same already.
Quoted 5 lines
Without patch 1 and only patches 4,5 that add few kfuncs, bpf-lsm prog will be able to remember subtree dentry and do the same
is_subdir()to deny.
The patch 7 stays pretty much as-is. All in bpf-lsm. Close to zero overhead without long chain of fsnotify callbacks.
One of the advantages of fanotify-bpf or fsnotify-bpf is that it only calls the BPF program for being monitored files. OTOH, BPF LSM program is always global.
Quoted 7 lines
Quoted 6 lines
fanotify also has a feature that LSM doesn't provide. When a file is accessed, user space daemon can get a fd on this file from fanotify. OTOH, LSM can only send an ino or a path to user space, which is not always reliable.
Quoted 8 lines
That sounds useful, but we're mixing too many things. If user space cares about fd it will be using the existing mechanism with all accompanied overhead. fsnotify-bpf can barely accelerate anything, since user space makes ultimate decisions.
If user space is not in the driving seat then existing bpf-lsm plus few kfuncs to remember dentry and callis_subdir()will do the job and no need for patch 1.
In many cases, we only need the user space to look into the file when necessary. For example, when a binary is first written, the user space daemon will scan through it (for virus, etc.) and mark it as safe/dangerous in some BPF maps. Later, when the file is opened for execution, f[s|a]notify-bpf can make the allow/deny decision based on the information in BPF maps.
Thanks again for your review and inputs!
Song
Re: [RFC/PATCH v2 bpf-next fanotify 7/7] selftests/bpf: Add test for BPF based fanotify fastpath handler
Quoted 3 lines · Song Liu
[...]
Quoted 19 lines
Quoted 10 lines
fsnotify_open_perm->fsnotify->send_to_group->fanotify_handle_event.is a pretty long path to call bpf prog and preparing a giant 'struct
fanotify_fastpath_event' is not going to fast either.If we want to accelerate that with bpf it needs to be done sooner with negligible overhead.
Agreed. This is actually something I have been thinking since the beginning of this work: Shall it be fanotify-bpf or fsnotify-bpf. Given we have more materials, this is a good time to have broader discussions on this.
@all, please chime in whether we should redo this as fsnotify-bpf. AFAICT:
Quoted 7 lines
Pros of fanotify-bpf:
- There is existing user space that we can leverage/reuse.Pros of fsnotify-bpf:
- Faster fast path.Another major pros/cons did I miss?
Adding more thoughts on this: I think it makes more sense to go with fanotify-bpf. This is because one of the benefits of fsnotify/fanotify over LSM solutions is the built-in event filtering of events. While this call chain is a bit long:
fsnotify_open_perm->fsnotify->send_to_group->fanotify_handle_event.
There are built-in filtering in fsnotify() and send_to_group(), so logics in the call chain are useful.
struct fanotify_fastpath_event is indeed big. But I think we need to pass these information to the fastpath handler either way.
Overall, I think current fastpath design makes sense, though there are things we need to fix (as Amir and Alexei pointed out). Please let me know comments and suggestions on this.
Thanks,
Song
Re: [RFC/PATCH v2 bpf-next fanotify 7/7] selftests/bpf: Add test for BPF based fanotify fastpath handler
Quoted 38 lines · Song Liu
Quoted 4 lines · Song Liu
[...]
Quoted 19 lines
Quoted 10 lines
fsnotify_open_perm->fsnotify->send_to_group->fanotify_handle_event.is a pretty long path to call bpf prog and preparing a giant 'struct
fanotify_fastpath_event' is not going to fast either.If we want to accelerate that with bpf it needs to be done sooner with negligible overhead.
Agreed. This is actually something I have been thinking since the beginning of this work: Shall it be fanotify-bpf or fsnotify-bpf. Given we have more materials, this is a good time to have broader discussions on this.
@all, please chime in whether we should redo this as fsnotify-bpf. AFAICT:
Quoted 7 lines
Pros of fanotify-bpf:
- There is existing user space that we can leverage/reuse.Pros of fsnotify-bpf:
- Faster fast path.Another major pros/cons did I miss?
Adding more thoughts on this: I think it makes more sense to go with fanotify-bpf. This is because one of the benefits of fsnotify/fanotify over LSM solutions is the built-in event filtering of events. While this call chain is a bit long:
Quoted 4 lines
fsnotify_open_perm->fsnotify->send_to_group->fanotify_handle_event.There are built-in filtering in
fsnotify()andsend_to_group(), so logics in the call chain are useful.
fsnotify_marks based filtering happens in fsnotify. No need to do more indirect calls to get to fanotify.
I would add the bpf struct_ops hook right before send_to_group or inside of it.
Not sure whether fsnotify_group concept should be reused or avoided.
Per inode mark/mask filter should stay.
Quoted 3 lines
struct
fanotify_fastpath_eventis indeed big. But I think we need to pass these information to the fastpath handler either way.
Disagree.
That was the old way of hooking bpf bits in. uapi/bpf.h is full of such "context" structs. xpd_md, bpf_tcp_sock, etc.
They pack fields into one struct only because
old style bpf has one input argument: ctx. struct_ops doesn't have this limitation.
Pass things like path/dentry/inode/whatever pointers directly. No need to pack into fanotify_fastpath_event.
Quoted 4 lines
Overall, I think current fastpath design makes sense, though there are things we need to fix (as Amir and Alexei pointed out). Please let me know comments and suggestions on this.
On one side you're arguing that extra indirection for inode local storage due to inode->i_secruity is needed for performance,
but on the other side you're not worried about the deep call stack of fsnotify->fanotify and argument packing which add way more overhead than i_security hop.
Re: [RFC/PATCH v2 bpf-next fanotify 7/7] selftests/bpf: Add test for BPF based fanotify fastpath handler
Hi Alexei,
Quoted 1 line · Alexei Starovoitov
[...]
Quoted 27 lines
Quoted 22 lines
Quoted 9 lines
Agreed. This is actually something I have been thinking since the beginning of this work: Shall it be fanotify-bpf or fsnotify-bpf. Given we have more materials, this is a good time to have broader discussions on this.
@all, please chime in whether we should redo this as fsnotify-bpf. AFAICT:
Quoted 7 lines
Pros of fanotify-bpf:
- There is existing user space that we can leverage/reuse.Pros of fsnotify-bpf:
- Faster fast path.Another major pros/cons did I miss?
Adding more thoughts on this: I think it makes more sense to go with fanotify-bpf. This is because one of the benefits of fsnotify/fanotify over LSM solutions is the built-in event filtering of events. While this call chain is a bit long:
Quoted 4 lines
fsnotify_open_perm->fsnotify->send_to_group->fanotify_handle_event.There are built-in filtering in
fsnotify()andsend_to_group(), so logics in the call chain are useful.
Quoted 8 lines
fsnotify_marksbased filtering happens in fsnotify. No need to do more indirect calls to get to fanotify.I would add the bpf
struct_opshook right beforesend_to_groupor inside of it.
Not sure whetherfsnotify_groupconcept should be reused or avoided.
Per inode mark/mask filter should stay.
We still need fsnotify_group. It matches each fanotify file descriptor.
Moving struct_ops hook inside send_to_group does save us an indirect call. But this also means we need to introduce the fastpath concept to both fsnotify and fanotify. I personally don't really like duplications like this (see the big BUILD_BUG_ON array in fanotify_handle_event).
OTOH, maybe the benefit of one fewer indirect call justifies the extra complexity. Let me think more about it.
Quoted 5 lines
Quoted 4 lines
struct
fanotify_fastpath_eventis indeed big. But I think we need to pass these information to the fastpath handler either way.
Quoted 9 lines
Disagree.
That was the old way of hooking bpf bits in.uapi/bpf.his full of such "context" structs.xpd_md,bpf_tcp_sock, etc.
They pack fields into one struct only because
old style bpf has one input argument: ctx.struct_opsdoesn't have this limitation.
Pass things like path/dentry/inode/whatever pointers directly. No need to pack intofanotify_fastpath_event.
OK. I am convinced on this one. I will adjust the code to remove fanotify_fastpath_event.
Quoted 9 lines
Quoted 5 lines
Overall, I think current fastpath design makes sense, though there are things we need to fix (as Amir and Alexei pointed out). Please let me know comments and suggestions on this.
On one side you're arguing that extra indirection for inode local storage due to
inode->i_secruityis needed for performance,
The biggest issue with inode_local_storage in i_security is not the extra indirection and thus the extra latency. The bigger problem is, once we make inode local storage available to tracing programs, we will not disable it at boot time. Therefore, the extra indirection through i_security doesn't give us any memory savings. Instead, it will cause latency and memory fragmentations. IOW, we are paying the cost for no benefits at all.
Quoted 3 lines
but on the other side you're not worried about the deep call stack of
fsnotify->fanotifyand argument packing which add way more overhead thani_securityhop.
I think the difference is one indirect call. But that may worth the work. I will think more about it. Also, I would really appreciate other folks' input.
Thanks again for your review!
Song
Amir Goldstein
reviewerRe: [RFC/PATCH v2 bpf-next fanotify 7/7] selftests/bpf: Add test for BPF based fanotify fastpath handler
Quoted 21 lines · Song Liu
Hi Alexei,
Quoted 1 line · Alexei Starovoitov
[...]
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
Agreed. This is actually something I have been thinking since the beginning of this work: Shall it be fanotify-bpf or fsnotify-bpf. Given we have more materials, this is a good time to have broader discussions on this.
@all, please chime in whether we should redo this as fsnotify-bpf. AFAICT:
Pros of fanotify-bpf:
- There is existing user space that we can leverage/reuse.Pros of fsnotify-bpf:
- Faster fast path.Another major pros/cons did I miss?
Sorry, I forgot to address this earlier.
First and foremost, I don't like the brand name "fast path" for this feature. The word "fast" implies that avoiding an indirect call is meaningful and I don't think this is the case here.
I would rather rebrand this feature as "fanotify filter program". I am going to make a controversial argument:
I anticipate that in the more common use of kernel filter for fanotify the filter will be *likely* to skip sending events to userspace, for example, when watching a small subtree in a large filesystem.
In that case, the *likely* result is that a context switch to userspace is avoided, hence, the addition of one indirect call is insignificant in comparison.
If we later decide that my assumption is wrong and it is important to optimize out the indirect call when skipping userspace, we could do that - it is not a problem, but for now, this discussion seems like premature optimization to me.
Quoted 14 lines
Quoted 14 lines
Quoted 10 lines
Adding more thoughts on this: I think it makes more sense to go with fanotify-bpf. This is because one of the benefits of fsnotify/fanotify over LSM solutions is the built-in event filtering of events. While this call chain is a bit long:
fsnotify_open_perm->fsnotify->send_to_group->fanotify_handle_event.There are built-in filtering in
fsnotify()andsend_to_group(), so logics in the call chain are useful.
fsnotify_marksbased filtering happens in fsnotify. No need to do more indirect calls to get to fanotify.
Quoted 9 lines
Quoted 5 lines
I would add the bpf
struct_opshook right beforesend_to_groupor inside of it.
Not sure whetherfsnotify_groupconcept should be reused or avoided.
Per inode mark/mask filter should stay.We still need
fsnotify_group. It matches each fanotify file descriptor.
We need the filter to be per fsnotify_group.
Unlike LSM, fsnotify/fanotify is a pubsub service for many fs event consumers (a.k.a groups).
The feature is a filter per event consumer, not a global filter for all event consumers.
Quoted 11 lines
Moving
struct_opshook insidesend_to_groupdoes save us an indirect call. But this also means we need to introduce the fastpath concept to both fsnotify and fanotify. I personally don't really like duplications like this (see the bigBUILD_BUG_ONarray infanotify_handle_event).OTOH, maybe the benefit of one fewer indirect call justifies the extra complexity. Let me think more about it.
I need to explain something about fsnotify vs. fanotify
in order to argue why the feature should be "fanotify", but the bottom line is that is should not be too hard to avoid the indirect call even if the feature is introduced through fanotify API as I think that it should be.
TLDR:
The fsnotify_backend abstraction has become somewhat of a theater of abstraction over time, because the feature distance between fanotify backend and all the rest has grew quite large.
The logic in send_to_group() is *seemingly* the generic fsnotify logic, but not really, because only fanotify has ignore masks and only fanotify has mark types (inode,mount,sb).
This difference is encoded by the group->ops->handle_event() operation that only fanotify implements.
All the rest of the backends implement the simpler ->handle_inode_event().
Similarly, the group->private union is always dominated by the size of group->fanotify_data, so there is no big difference if we place group->fp_hook (or ->filter_hook) outside of fanotify_data, so that we can query and call it from send_to_group() saving the indirect call to ->handle_event().
That still leaves the question if we need to call fanotify_group_event_mask() before the filter hook.
fanotify_group_event_mask() does several things, but I think that the only thing relevant before the filter hook is this line:
/*
* Send the event depending on event flags in mark mask.
*/
if (!fsnotify_mask_applicable(mark->mask, ondir, type))
continue;This code is related to the two "built-in fanotify filters", namely FAN_ONDIR and FAN_EVENT_ON_CHILD.
These built-in filters are so lame that they serve as a good example why a programmable filter is a better idea.
For example, users need to opt-in for events on directories, but they cannot request events only on directories.
Historically, the "generic" abstraction in send_to_group() has dealt with the non-generic fanotify ignore mask, but has not dealt with these non-generic fanotify built-in filters.
However, since commit 31a371e419c8 ("fanotify: prepare for setting event flags in ignore mask"), send_to_group() is already aware of those fanotify built-in filters.
So unless I am missing something, if we align the marks iteration loop in send_to_group() to look exactly like the marks iteration loop in fanotify_group_event_mask(), there should be no problem to call the filter hook directly, right before calling ->handle_event().
Hope this brain dump helps.
Thanks,
Amir.
Re: [RFC/PATCH v2 bpf-next fanotify 7/7] selftests/bpf: Add test for BPF based fanotify fastpath handler
Hi Amir,
Thanks for sharing these thoughts. I will take a closer look later, as I am not awake enough to fully understand everything.
Quoted 1 line · Amir Goldstein
[...]
Quoted 12 lines
Quoted 11 lines
Moving
struct_opshook insidesend_to_groupdoes save us an indirect call. But this also means we need to introduce the fastpath concept to both fsnotify and fanotify. I personally don't really like duplications like this (see the bigBUILD_BUG_ONarray infanotify_handle_event).OTOH, maybe the benefit of one fewer indirect call justifies the extra complexity. Let me think more about it.
Quoted 5 lines
I need to explain something about fsnotify vs. fanotify
in order to argue why the feature should be "fanotify", but the bottom line is that is should not be too hard to avoid the indirect call even if the feature is introduced through fanotify API as I think that it should be.
When I first looked into this, I thought about "whether there will be a use case that uses fsnotify but not fanotify". I didn't get any conclusion on this back then. But according to this thread, I think we are pretty confident that future use cases (such as FAN_PRE_ACCESS) will have a fanotify part. If this is the case, I think fanotify-bpf makes more sense.
Quoted 14 lines
TLDR:
Thefsnotify_backendabstraction has become somewhat of a theater of abstraction over time, because the feature distance between fanotify backend and all the rest has grew quite large.The logic in
send_to_group()is *seemingly* the generic fsnotify logic, but not really, because only fanotify has ignore masks and only fanotify has mark types (inode,mount,sb).This difference is encoded by the
group->ops->handle_event()operation that only fanotify implements.
All the rest of the backends implement the simpler ->handle_inode_event().
Quoted 8 lines
Similarly, the
group->privateunion is always dominated by the size ofgroup->fanotify_data, so there is no big difference if we placegroup->fp_hook(or ->filter_hook) outside offanotify_data, so that we can query and call it fromsend_to_group()saving the indirect call to ->handle_event().That still leaves the question if we need to call
fanotify_group_event_mask()before the filter hook.
I was trying Alexei's idea to move the API to fsnotify, and got stucked at fanotify_group_event_mask(). It appears we should always honor these built-in filters.
Quoted 15 lines
fanotify_group_event_mask()does several things, but I think that the only thing relevant before the filter hook is this line:/* * Send the event depending on event flags in mark mask. */ if (!fsnotify_mask_applicable(mark->mask, ondir, type)) continue;This code is related to the two "built-in fanotify filters", namely
FAN_ONDIRandFAN_EVENT_ON_CHILD.
These built-in filters are so lame that they serve as a good example why a programmable filter is a better idea.
For example, users need to opt-in for events on directories, but they cannot request events only on directories.
Quoted 7 lines
Historically, the "generic" abstraction in
send_to_group()has dealt with the non-generic fanotify ignore mask, but has not dealt with these non-generic fanotify built-in filters.However, since commit
31a371e419c8("fanotify: prepare for setting event flags in ignore mask"),send_to_group()is already aware of those fanotify built-in filters.
I will continue on this tomorrow. It is time to get some sleep. :)
Quoted 7 lines
So unless I am missing something, if we align the marks iteration loop in
send_to_group()to look exactly like the marks iteration loop infanotify_group_event_mask(), there should be no problem to call the filter hook directly, right before calling ->handle_event().Hope this brain dump helps.
Thanks again for your input!
Song
Quoted 3 lines
Thanks,
Amir.
Amir Goldstein
reviewerRe: [RFC/PATCH v2 bpf-next fanotify 7/7] selftests/bpf: Add test for BPF based fanotify fastpath handler
In reply to Alexei StarovoitovQuoted 13 lines · Alexei Starovoitov
tools/testing/selftests/bpf/progs/fan_fp.c line 122
Quoted 6 lines · Song Liu
tools/testing/selftests/bpf/progs/fan_fp.c new file +5
-0+ +if (bpf_is_subdir (dentry, v->dentry)) + ret =FAN_FP_RET_SEND_TO_USERSPACE ; +else + ret =FAN_FP_RET_SKIP_EVENT ;It seems to me that all these patches and feature additions to fanotify, new kfuncs, etc are done just to do the above filtering by subdir ?
Quoted 5 lines
If so, just hard code this logic as an extra flag to fanotify ? So it can filter all events by subdir.
bpf programmability makes sense when it needs to express user space policy. Here it's just a filter by subdir. bpf hammer doesn't look like the right tool for this use case.
Good question.
Speaking as someone who has made several attempts to design efficient subtree filtering in fanotify, it is not as easy as it sounds.
I recently implemented a method that could be used for "practical" subdir filtering in userspace, not before Jan has questioned if we should go directly to subtree filtering with bpf [1].
This is not the only filter that was proposed for fanotify, where bpf filter came as an alternative proposal [2], but subtree filtering is by far the most wanted filter.
The problem with implementing a naive is_subtree() filter in fanotify is the unbounded cost to be paid by every user for every fs access when M such filters are installed deep in the fs tree.
Making this more efficient then becomes a matter of trading of memory (inode/path cache size) and performance and depends on the size and depth of the watched filesystem.
This engineering decision *is* the userspace policy that can be expressed by a bpf program.
As you may know, Linux is lagging behind Win and MacOS w.r.t subtree filtering for fs events.
MacOS/FreeBSD took the userspace approach with fseventsd [3]. If you Google "fseventsd", you will get results with "High CPU and Memory Usage" for as far as the browser can scroll.
I hope the bpf-aided early subtree filtering technology would be able to reduce some of this overhead to facilitate a better engineering solution, but that remains to be proven...
Thanks,
Amir.
[1] https://lore.kernel.org/linux-fsdevel/[email protected]/
[2] https://lore.kernel.org/linux-fsdevel/[email protected]/
[3] https://developer.apple.com/library/archive/documentation/Darwin/Conceptual/FSEvents_ProgGuide/TechnologyOverview/TechnologyOverview.html#//apple_ref/doc/uid/TP40005289-CH3-SW1
Re: [RFC/PATCH v2 bpf-next fanotify 7/7] selftests/bpf: Add test for BPF based fanotify fastpath handler
Quoted 27 lines · Amir Goldstein
Quoted 13 lines · Alexei Starovoitov
tools/testing/selftests/bpf/progs/fan_fp.c line 122
Quoted 6 lines · Song Liu
tools/testing/selftests/bpf/progs/fan_fp.c new file +5
-0+ +if (bpf_is_subdir (dentry, v->dentry)) + ret =FAN_FP_RET_SEND_TO_USERSPACE ; +else + ret =FAN_FP_RET_SKIP_EVENT ;It seems to me that all these patches and feature additions to fanotify, new kfuncs, etc are done just to do the above filtering by subdir ?
Quoted 5 lines
If so, just hard code this logic as an extra flag to fanotify ? So it can filter all events by subdir.
bpf programmability makes sense when it needs to express user space policy. Here it's just a filter by subdir. bpf hammer doesn't look like the right tool for this use case.Good question.
Speaking as someone who has made several attempts to design efficient subtree filtering in fanotify, it is not as easy as it sounds.
Quoted 7 lines
I recently implemented a method that could be used for "practical" subdir filtering in userspace, not before Jan has questioned if we should go directly to subtree filtering with bpf [1].
This is not the only filter that was proposed for fanotify, where bpf filter came as an alternative proposal [2], but subtree filtering is by far the most wanted filter.
Interesting :)
Thanks for the links. Long threads back then.
Looks like bpf filtering was proposed 2 and 4 years ago and Song's current attempt is the first real prototype that actually implements what was discussed for so long?
Cool. I guess it has legs then.
Quoted 3 lines
The problem with implementing a naive
is_subtree()filter in fanotify is the unbounded cost to be paid by every user for every fs access when M such filters are installed deep in the fs tree.
Agree that the concern is real.
I just don't see how filtering in bpf vs filtering in the kernel is going to be any different.is_subdir() is fast, so either kernel or bpf prog calling it on every file_open doesn't seem like a big deal.
Are you saying that 'naive is_subdir' would call is_subdir M times for every such filter ?
I guess it can be optimized. When these M filters are installed the kernel can check whether they're subdirs of each other and filter a subset of M filters with a single is_subdir() call. Same logic can be in either bpf prog or in the kernel.
Quoted 6 lines
Making this more efficient then becomes a matter of trading of memory (inode/path cache size) and performance and depends on the size and depth of the watched filesystem.
This engineering decision *is* the userspace policy that can be expressed by a bpf program.
Quoted 6 lines
As you may know, Linux is lagging behind Win and MacOS w.r.t subtree filtering for fs events.
MacOS/FreeBSD took the userspace approach with fseventsd [3]. If you Google "fseventsd", you will get results with "High CPU and Memory Usage" for as far as the browser can scroll.
Yeah. Fun.
Looking at your 2 year old attempt it seems the key part was to make sure inodes are not pinned for filtering purpose? How would you call is_subdir() then if parent dentry is not dget() ? Or meaning of 'pinning' is different?
Quoted 3 lines
I hope the bpf-aided early subtree filtering technology would be able to reduce some of this overhead to facilitate a better engineering solution, but that remains to be proven...
Sure. Let's explore this further.
Quoted 6 lines
Thanks,
Amir.[1] https://lore.kernel.org/linux-fsdevel/[email protected]/
[2] https://lore.kernel.org/linux-fsdevel/[email protected]/
[3] https://developer.apple.com/library/archive/documentation/Darwin/Conceptual/FSEvents_ProgGuide/TechnologyOverview/TechnologyOverview.html#//apple_ref/doc/uid/TP40005289-CH3-SW1