fanotify: novel file access notification and permission system
Subsystems (12)
cover fanotify: novel file access notification and permission system
the following is a file notification and access system intended to allow a variety of userspace programs to get information about filesystem events no matter where or how they happen on a system and use that in conjunction with the actual on disk data related to that event to provide additional services such as file change indexing or content based antivirus scanning. Minor changes are almost certainly possible to make this notification and access interface usable for HSMs. fscking all notify is generally refered to as fanotify, or for the weak of heart you can call it file access notify system. The ideas behind this code are based on talpa the GPL antivirus interface originally written by Sophos and on the feedback from lkml and malware-list. This is however a complete rewrite from scratch.
The last patch set addressed every complaint/critisism from lists. This set adds a couple new features, some documentation, and a couple fixes to the networking headers to support bi-arch machines.
**fanotify-executive-summary**
fanotify has a number of event types and only sends events for S_ISREG() files. These events incluse open, read, write, and permissions checking for open and read. The permissions checking events require that the listener return some sort of allow/deny/more_time response as the original process blocks until it gets an event (or times out) listeners may register a group which will get notifications about any combination of these events, it will be up to the listener to determine what events they are interested in hearing or mediating access decisions for.
groups are a construct in which userspace indicates what priority (only really used for permission type events) and what type of events its listeners want to hear. A single group may have unlimited listeners but each event will only go to ONE listener. Two groups may register for the same type of events and one listener in EACH group will get a copy of the event.
The user interface for fanotify is all through sockopt calls. Userspace pulls events from the kernel using getsockopt and sends responses to the kernel using setsockopt. These are the only socket operations defined for PF_FANOTIFY sockets.
---
Eric Paris (21):
fanotify: add Documentation
fanotify: allow fastpath entries to survive inode modification
fanotify: evict misbehaving clients
fanotify: all userspace to set timeouts
fanotify: add option to clear all fastpaths
fanotify: send file f_flags along with notifications
fanotify: send tgid with notification messages
fanotify: send pid with fanotify notification events
fanotify: ability for userspace to delay responses
fanotify: user interface for access decisions
fanotify: give a special access permission check
fanotify: blocking and access granting
fanotify: add group priorities
fanotify: add a userspace interface for fastpaths
fanotify: fastpath to ignore certain in core inodes
fanotify: add a userspace interface for fanotify notifications
fanotify: make use of the new fsnotify_open_exec calls
fsnotify: sys_execve and sys_uselib do not call into fsnotify
fanotify: fscking all notify, system wide file access notification
fsnotify: pass a file instead of an inode to open, read, and write filesystem notification: create fs/notify to contain all fs notification
Documentation/filesystems/fanotify/fanotify.txt | 214 ++++++ .../filesystems/fanotify/fanotify_tester.c | 255 +++++++ fs/Kconfig | 39 - fs/Makefile | 5 fs/aio.c | 7 fs/compat.c | 5 fs/dnotify.c | 194 ----- fs/exec.c | 15 fs/inode.c | 6 fs/inotify.c | 773 -------------------- fs/inotify_user.c | 778 -------------------- fs/nfsd/vfs.c | 4 fs/notify/Kconfig | 52 + fs/notify/Makefile | 6 fs/notify/access.c | 222 ++++++ fs/notify/dnotify.c | 194 +++++ fs/notify/fanotify.c | 152 ++++ fs/notify/fanotify.h | 112 +++ fs/notify/fastpath.c | 264 +++++++ fs/notify/group.c | 163 ++++ fs/notify/inotify.c | 773 ++++++++++++++++++++ fs/notify/inotify_user.c | 778 ++++++++++++++++++++ fs/notify/notification.c | 277 +++++++ fs/open.c | 7 fs/read_write.c | 14 include/linux/Kbuild | 1 include/linux/fanotify.h | 197 +++++ include/linux/fs.h | 5 include/linux/fsnotify.h | 39 + include/linux/sched.h | 1 include/linux/socket.h | 5 mm/mmap.c | 7 mm/mprotect.c | 6 mm/nommu.c | 7 net/Makefile | 1 net/core/sock.c | 6 net/fanotify/Makefile | 5 net/fanotify/af_fanotify.c | 295 ++++++++ net/fanotify/af_fanotify.h | 20 + 39 files changed, 4096 insertions(+), 1808 deletions(-) create mode 100644 Documentation/filesystems/fanotify/fanotify.txt create mode 100644 Documentation/filesystems/fanotify/fanotify_tester.c delete mode 100644 fs/dnotify.c delete mode 100644 fs/inotify.c delete mode 100644 fs/inotify_user.c create mode 100644 fs/notify/Kconfig create mode 100644 fs/notify/Makefile create mode 100644 fs/notify/access.c create mode 100644 fs/notify/dnotify.c create mode 100644 fs/notify/fanotify.c create mode 100644 fs/notify/fanotify.h create mode 100644 fs/notify/fastpath.c create mode 100644 fs/notify/group.c create mode 100644 fs/notify/inotify.c create mode 100644 fs/notify/inotify_user.c create mode 100644 fs/notify/notification.c create mode 100644 include/linux/fanotify.h create mode 100644 net/fanotify/Makefile create mode 100644 net/fanotify/af_fanotify.c create mode 100644 net/fanotify/af_fanotify.h
Signature · 1 line
Signature
PATCH 1/21 filesystem notification: create fs/notify to contain all fs notification
Adding yet another filesystem notification system it seemed like a good idea to clean up fs/ but creating an fs/notify and putting everything there.
---
fs/Kconfig | 39 -- fs/Makefile | 5 fs/dnotify.c | 194 ----------- fs/inotify.c | 773 ---------------------------------------------- fs/inotify_user.c | 778 ---------------------------------------------- fs/notify/Kconfig | 38 ++ fs/notify/Makefile | 4 fs/notify/dnotify.c | 194 +++++++++++ fs/notify/inotify.c | 773 ++++++++++++++++++++++++++++++++++++++++++++++ fs/notify/inotify_user.c | 778 ++++++++++++++++++++++++++++++++++++++++++++++ 10 files changed, 1789 insertions(+), 1787 deletions(-) delete mode 100644 fs/dnotify.c delete mode 100644 fs/inotify.c delete mode 100644 fs/inotify_user.c create mode 100644 fs/notify/Kconfig create mode 100644 fs/notify/Makefile create mode 100644 fs/notify/dnotify.c create mode 100644 fs/notify/inotify.c create mode 100644 fs/notify/inotify_user.c
fs/Kconfig +1 -38
@@ -270,44 +270,7 @@ config OCFS2_COMPAT_JBD endif # BLOCK -config DNOTIFY - bool "Dnotify support" - default y - help - Dnotify is a directory-based per-fd file change notification system - that uses signals to communicate events to user-space. There exist - superior alternatives, but some applications may still rely on - dnotify. - - If unsure, say Y. - -config INOTIFY - bool "Inotify file change notification support" - default y - ---help--- - Say Y here to enable inotify support. Inotify is a file change - notification system and a replacement for dnotify. Inotify fixes - numerous shortcomings in dnotify and introduces several new features - including multiple file events, one-shot support, and unmount - notification. - - For more information, see <file:Documentation/filesystems/inotify.txt> - - If unsure, say Y. - -config INOTIFY_USER - bool "Inotify support for userspace" - depends on INOTIFY - default y - ---help--- - Say Y here to enable inotify support for userspace, including the - associated system calls. Inotify allows monitoring of both files and - directories via a single open fd. Events are read from the file - descriptor, which is also select()- and poll()-able. - - For more information, see <file:Documentation/filesystems/inotify.txt> - - If unsure, say Y. +source "fs/notify/Kconfig" config QUOTA bool "Quota support"
fs/Makefile +1 -4
@@ -20,8 +20,7 @@ obj-y += no-block.o endif obj-$(CONFIG_BLK_DEV_INTEGRITY) += bio-integrity.o -obj-$(CONFIG_INOTIFY) += inotify.o -obj-$(CONFIG_INOTIFY_USER) += inotify_user.o +obj-y += notify/ obj-$(CONFIG_EPOLL) += eventpoll.o obj-$(CONFIG_ANON_INODES) += anon_inodes.o obj-$(CONFIG_SIGNALFD) += signalfd.o
@@ -57,8 +56,6 @@ obj-$(CONFIG_QFMT_V1) += quota_v1.o obj-$(CONFIG_QFMT_V2) += quota_v2.o obj-$(CONFIG_QUOTACTL) += quota.o -obj-$(CONFIG_DNOTIFY) += dnotify.o - obj-$(CONFIG_PROC_FS) += proc/ obj-y += partitions/ obj-$(CONFIG_SYSFS) += sysfs/
fs/dnotify.c deleted +0 -194
194 lines not on this page · raw message
fs/inotify.c deleted +0 -773
773 lines not on this page · raw message
fs/inotify_user.c deleted +0 -778
778 lines not on this page · raw message
fs/notify/Kconfig new file +38 -0
@@ -0,0 +1,38 @@ +config DNOTIFY + bool "Dnotify support" + default y + help + Dnotify is a directory-based per-fd file change notification system + that uses signals to communicate events to user-space. There exist + superior alternatives, but some applications may still rely on + dnotify. + + If unsure, say Y. + +config INOTIFY + bool "Inotify file change notification support" + default y + ---help--- + Say Y here to enable inotify support. Inotify is a file change + notification system and a replacement for dnotify. Inotify fixes + numerous shortcomings in dnotify and introduces several new features + including multiple file events, one-shot support, and unmount + notification. + + For more information, see <file:Documentation/filesystems/inotify.txt> + + If unsure, say Y. + +config INOTIFY_USER + bool "Inotify support for userspace" + depends on INOTIFY + default y + ---help--- + Say Y here to enable inotify support for userspace, including the + associated system calls. Inotify allows monitoring of both files and + directories via a single open fd. Events are read from the file + descriptor, which is also select()- and poll()-able. + + For more information, see <file:Documentation/filesystems/inotify.txt> + + If unsure, say Y.
fs/notify/Makefile new file +4 -0
@@ -0,0 +1,4 @@ +obj-$(CONFIG_INOTIFY) += inotify.o +obj-$(CONFIG_INOTIFY_USER) += inotify_user.o + +obj-$(CONFIG_DNOTIFY) += dnotify.o
fs/notify/dnotify.c new file +194 -0
194 lines not on this page · raw message
fs/notify/inotify.c new file +773 -0
773 lines not on this page · raw message
fs/notify/inotify_user.c new file +778 -0
184 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Part of the diff is not shown here. View the raw message.
PATCH 2/21 fsnotify: pass a file instead of an inode to open, read, and write
In reply to Eric Parisfanotify, the upcoming notification system actually needs a f_path so it can do opens in the context of listeners, and it needs a file so it can get f_flags from the original process. Close already was passing a file.
---
fs/compat.c | 5 ++--- fs/nfsd/vfs.c | 4 ++-- fs/open.c | 2 +- fs/read_write.c | 8 ++++---- include/linux/fsnotify.h | 9 ++++++--- 5 files changed, 15 insertions(+), 13 deletions(-)
fs/compat.c +2 -3
@@ -1158,11 +1158,10 @@ out:if (iov != iovstack)kfree (iov);if ((ret + (type ==READ )) >0 ) { -struct dentry *dentry = file->f_path.dentry;if (type ==READ ) -fsnotify_access (dentry); +fsnotify_access (file);else -fsnotify_modify (dentry); +fsnotify_modify (file); }return ret; }
fs/nfsd/vfs.c +2 -2
@@ -940,7 +940,7 @@ nfsd_vfs_read(struct svc_rqst *rqstp, struct svc_fh *fhp, struct file *file, nfsdstats.io_read += host_err; *count = host_err; err =0 ; -fsnotify_access (file->f_path.dentry); +fsnotify_access (file); }else err =nfserrno (host_err); out:
@@ -1007,7 +1007,7 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp, struct file *file,set_fs (oldfs);if (host_err >=0 ) { nfsdstats.io_write += cnt; -fsnotify_modify (file->f_path.dentry); +fsnotify_modify (file); }/* clear setuid/setgid flag after write */
fs/open.c +1 -1
@@ -1020,7 +1020,7 @@ long do_sys_open(int dfd, const char __user *filename, int flags, int mode)put_unused_fd (fd); fd =PTR_ERR (f); }else { -fsnotify_open (f->f_path.dentry); +fsnotify_open (f);fd_install (fd, f); } }
fs/read_write.c +4 -4
@@ -280,7 +280,7 @@ ssize_t vfs_read(struct file *file, char __user *buf, size_t count, loff_t *pos)else ret =do_sync_read (file, buf, count, pos);if (ret >0 ) { -fsnotify_access (file->f_path.dentry); +fsnotify_access (file);add_rchar (current, ret); }inc_syscr (current);
@@ -335,7 +335,7 @@ ssize_t vfs_write(struct file *file, const char __user *buf, size_t count, loff_else ret =do_sync_write (file, buf, count, pos);if (ret >0 ) { -fsnotify_modify (file->f_path.dentry); +fsnotify_modify (file);add_wchar (current, ret); }inc_syscw (current);
@@ -626,9 +626,9 @@ out:kfree (iov);if ((ret + (type ==READ )) >0 ) {if (type ==READ ) -fsnotify_access (file->f_path.dentry); +fsnotify_access (file);else -fsnotify_modify (file->f_path.dentry); +fsnotify_modify (file); }return ret; }
include/linux/fsnotify.h +6 -3
@@ -136,8 +136,9 @@ static inline void fsnotify_mkdir(struct inode *inode, struct dentry *dentry)/* * fsnotify_access - file was read */ -static inline void fsnotify_access (struct dentry *dentry) +static inline void fsnotify_access (struct file *file) { +struct dentry *dentry = file->f_path.dentry;struct inode *inode = dentry->d_inode;u32 mask =IN_ACCESS ;
@@ -152,8 +153,9 @@ static inline void fsnotify_access(struct dentry *dentry)/* * fsnotify_modify - file was modified */ -static inline void fsnotify_modify (struct dentry *dentry) +static inline void fsnotify_modify (struct file *file) { +struct dentry *dentry = file->f_path.dentry;struct inode *inode = dentry->d_inode;u32 mask =IN_MODIFY ;
@@ -168,8 +170,9 @@ static inline void fsnotify_modify(struct dentry *dentry)/* * fsnotify_open - file was opened */ -static inline void fsnotify_open (struct dentry *dentry) +static inline void fsnotify_open (struct file *file) { +struct dentry *dentry = file->f_path.dentry;struct inode *inode = dentry->d_inode;u32 mask =IN_OPEN ;
PATCH 3/21 fanotify: fscking all notify, system wide file access notification
In reply to Eric ParisA new system wide file access notification system.
---
fs/notify/Kconfig | 14 ++ fs/notify/Makefile | 2 fs/notify/fanotify.c | 99 +++++++++++++++++ fs/notify/fanotify.h | 63 +++++++++++ fs/notify/group.c | 110 +++++++++++++++++++ fs/notify/notification.c | 266 ++++++++++++++++++++++++++++++++++++++++++++++ include/linux/Kbuild | 1 include/linux/fanotify.h | 95 ++++++++++++++++ include/linux/fsnotify.h | 14 ++ include/linux/sched.h | 1 10 files changed, 661 insertions(+), 4 deletions(-) create mode 100644 fs/notify/fanotify.c create mode 100644 fs/notify/fanotify.h create mode 100644 fs/notify/group.c create mode 100644 fs/notify/notification.c create mode 100644 include/linux/fanotify.h
fs/notify/Kconfig +14 -0
@@ -36,3 +36,17 @@ config INOTIFY_USER For more information, see <file:Documentation/filesystems/inotify.txt> If unsure, say Y. + +config FANOTIFY + bool "Filesystem wide access notification" + select SECURITY + default y + ---help--- + Say Y here to enable fanotify suport. fanotify is a system wide + file access notification interface. Events are read from from a + single open fd and in doing so a fd is created in the reading process + which points to the same data as the one on which the event occured. + + For more information, see <file:Documentation/filesystems/fanotify.txt> + + If unsure, say Y.
fs/notify/Makefile +2 -0
@@ -2,3 +2,5 @@ obj-$(CONFIG_INOTIFY) += inotify.o obj-$(CONFIG_INOTIFY_USER) += inotify_user.o obj-$(CONFIG_DNOTIFY) += dnotify.o + +obj-$(CONFIG_FANOTIFY) += fanotify.o notification.o group.o
fs/notify/fanotify.c new file +99 -0
99 lines not on this page · raw message
fs/notify/fanotify.h new file +63 -0
63 lines not on this page · raw message
fs/notify/group.c new file +110 -0
110 lines not on this page · raw message
fs/notify/notification.c new file +266 -0
266 lines not on this page · raw message
include/linux/Kbuild +1 -0
@@ -246,6 +246,7 @@ unifdef-y += inet_diag.h unifdef-y += in.h unifdef-y += in6.h unifdef-y += inotify.h +unifdef-y += fanotify.h unifdef-y += input.h unifdef-y += ip.h unifdef-y += ipc.h
include/linux/fanotify.h new file +95 -0
95 lines not on this page · raw message
include/linux/fsnotify.h +10 -4
@@ -13,6 +13,7 @@#include <linux/dnotify.h> #include <linux/inotify.h> +#include <linux/fanotify.h> #include <linux/audit.h> /*
@@ -148,6 +149,7 @@ static inline void fsnotify_access(struct file *file)dnotify_parent (dentry,DN_ACCESS );inotify_dentry_parent_queue_event (dentry, mask,0 , dentry->d_name.name);inotify_inode_queue_event (inode, mask,0 ,NULL ,NULL ); +fanotify (file,FAN_ACCESS ); }/*
@@ -165,6 +167,7 @@ static inline void fsnotify_modify(struct file *file)dnotify_parent (dentry,DN_MODIFY );inotify_dentry_parent_queue_event (dentry, mask,0 , dentry->d_name.name);inotify_inode_queue_event (inode, mask,0 ,NULL ,NULL ); +fanotify (file,FAN_MODIFY ); }/*
@@ -181,6 +184,7 @@ static inline void fsnotify_open(struct file *file)inotify_dentry_parent_queue_event (dentry, mask,0 , dentry->d_name.name);inotify_inode_queue_event (inode, mask,0 ,NULL ,NULL ); +fanotify (file,FAN_OPEN ); }/*
@@ -192,13 +196,15 @@ static inline void fsnotify_close(struct file *file)struct inode *inode = dentry->d_inode;const char *name = dentry->d_name.name;fmode_t mode = file->f_mode; -u32 mask = (mode &FMODE_WRITE ) ?IN_CLOSE_WRITE :IN_CLOSE_NOWRITE ; +u32 in_mask = (mode &FMODE_WRITE ) ?IN_CLOSE_WRITE :IN_CLOSE_NOWRITE ; +u32 fan_mask = (mode &FMODE_WRITE ) ?FAN_CLOSE_WRITE :FAN_CLOSE_NOWRITE ;if (S_ISDIR (inode->i_mode)) - mask |=IN_ISDIR ; + in_mask |=IN_ISDIR ; -inotify_dentry_parent_queue_event (dentry, mask,0 , name); -inotify_inode_queue_event (inode, mask,0 ,NULL ,NULL ); +inotify_dentry_parent_queue_event (dentry, in_mask,0 , name); +inotify_inode_queue_event (inode, in_mask,0 ,NULL ,NULL ); +fanotify (file, fan_mask); }/*
include/linux/sched.h +1 -0
@@ -1538,6 +1538,7 @@ extern cputime_t task_gtime(struct task_struct *p);#define PF_EXITING 0x00000004/* getting shut down */ #define PF_EXITPIDONE 0x00000008/* pi exit done on shut down */ #define PF_VCPU 0x00000010/* I'm a virtual CPU */ +#define PF_NOFACCESS 0x00000020/* exclude from all faccesses */ #define PF_FORKNOEXEC 0x00000040/* forked but didn't exec */ #define PF_SUPERPRIV 0x00000100/* used super-user privileges */ #define PF_DUMPCORE 0x00000200/* dumped core */
PATCH 4/21 fsnotify: sys_execve and sys_uselib do not call into fsnotify
In reply to Eric Parissys_execve and sys_uselib do not call into fsnotify so inotify, dnotify, and importantly to me fanotify do not see opens on things which are going to be exectued. Create a generic fsnotify hook for these paths.
---
fs/exec.c | 5 +++++ include/linux/fsnotify.h | 7 +++++++ 2 files changed, 12 insertions(+), 0 deletions(-)
fs/exec.c +5 -0
@@ -51,6 +51,7 @@#include <linux/audit.h> #include <linux/tracehook.h> #include <linux/kmod.h> +#include <linux/fsnotify.h> #include <asm/uaccess.h> #include <asm/mmu_context.h>
@@ -135,6 +136,8 @@ asmlinkage long sys_uselib(const char __user * library)if (IS_ERR (file)) goto out; +fsnotify_open_exec (file); + error = -ENOEXEC ;if (file->f_op) {struct linux_binfmt * fmt;
@@ -687,6 +690,8 @@ struct file *open_exec(const char *name)if (IS_ERR (file))return file; +fsnotify_open_exec (file); + err =deny_write_access (file);if (err) {fput (file);
include/linux/fsnotify.h +7 -0
@@ -171,6 +171,13 @@ static inline void fsnotify_modify(struct file *file) }/* +* fsnotify_open_exec - file was opened by execve of uselib +*/ +static inline void fsnotify_open_exec (struct file *file) +{ +} + +/* * fsnotify_open - file was opened */ static inline void fsnotify_open (struct file *file)
Re: [PATCH =-v3 04/21] fsnotify: sys_execve and sys_uselib do not call into fsnotify
Quoted 3 lines · Eric Paris
sys_execveandsys_uselibdo not call into fsnotify so inotify, dnotify, and importantly to me fanotify do not see opens on things which are going to be exectued. Create a generic fsnotify hook for these paths.
This should be submitted as a bugfix on it's own to the current codebase.
Re: [PATCH =-v3 04/21] fsnotify: sys_execve and sys_uselib do not call into fsnotify
Quoted 7 lines · Christoph Hellwig
Quoted 3 lines · Eric Paris
sys_execveandsys_uselibdo not call into fsnotify so inotify, dnotify, and importantly to me fanotify do not see opens on things which are going to be exectued. Create a generic fsnotify hook for these paths.This should be submitted as a bugfix on it's own to the current codebase.
I'll look into this.
PATCH 5/21 fanotify: make use of the new fsnotify_open_exec calls
In reply to Eric ParisThis function sends fanotify events for opens which we know are being used for exec. These are basically just systecalls to sys_execve and sys_uselib
---
include/linux/fanotify.h | 4 +++- include/linux/fsnotify.h | 3 ++- 2 files changed, 5 insertions(+), 2 deletions(-)
include/linux/fanotify.h +3 -1
@@ -14,13 +14,15 @@#define FAN_MODIFY 0x00000002/* File was modified */ #define FAN_CLOSE_NOWRITE 0x00000004/* Unwrittable file closed */ #define FAN_CLOSE_WRITE 0x00000008/* Writtable file closed */ -#define FAN_OPEN 0x00000010/* File was opened */ +#define FAN_OPEN_NOEXEC 0x00000010/* File was opened */ +#define FAN_OPEN_EXEC 0x00000020/* File was opened with the intention of being exec'ed */ /* FIXME currently Q's have no limit.... */ #define FAN_Q_OVERFLOW 0x80000000/* Event queued overflowed */ /* helper events */ #define FAN_CLOSE (FAN_CLOSE_WRITE | FAN_CLOSE_NOWRITE)/* close */ +#define FAN_OPEN (FAN_OPEN_NOEXEC | FAN_OPEN_EXEC)/* open */ /* * All of the events - we build the list by hand so that we can add flags in
include/linux/fsnotify.h +2 -1
@@ -175,6 +175,7 @@ static inline void fsnotify_modify(struct file *file)*/ static inline void fsnotify_open_exec (struct file *file) { +fanotify (file,FAN_OPEN_EXEC ); }/*
@@ -191,7 +192,7 @@ static inline void fsnotify_open(struct file *file)inotify_dentry_parent_queue_event (dentry, mask,0 , dentry->d_name.name);inotify_inode_queue_event (inode, mask,0 ,NULL ,NULL ); -fanotify (file,FAN_OPEN ); +fanotify (file,FAN_OPEN_NOEXEC ); }/*
PATCH 6/21 fanotify: add a userspace interface for fanotify notifications
In reply to Eric Parisnotifications from fanotify need a userspace interface. We are going to try to do it with sockets. Scary....
---
include/linux/fanotify.h | 29 ++++- include/linux/socket.h | 5 + net/Makefile | 1 net/core/sock.c | 6 + net/fanotify/Makefile | 5 + net/fanotify/af_fanotify.c | 240 ++++++++++++++++++++++++++++++++++++++++++++ net/fanotify/af_fanotify.h | 20 ++++ 7 files changed, 294 insertions(+), 12 deletions(-) create mode 100644 net/fanotify/Makefile create mode 100644 net/fanotify/af_fanotify.c create mode 100644 net/fanotify/af_fanotify.h
include/linux/fanotify.h +21 -8
@@ -34,17 +34,23 @@ FAN_CLOSE |\ FAN_OPEN) -#ifdef __KERNEL__#include <linux/types.h> -#else -#include <stdint.h> -#include <sys/types.h> -#endif +struct fanotify_addr { +__u32 group_num; +__u32 mask; +__u32 timeout; +__u32 unused[9 ]; +} __attribute__((packed)); + +/* struct used for FANOTIFY_GET_EVENT */ struct fanotify_event_metadata { -int32_t fd; -uint32_t mask; -}; +__s32 fd; +__u32 mask; +} __attribute__((packed)); + +/* fanotify getsockopt optvals */ +#define FANOTIFY_GET_EVENT 1#ifdef __KERNEL__
@@ -74,6 +80,8 @@ extern void fanotify(struct file *file, unsigned int mask);extern void fanotify_get_group (struct fanotify_group *group);extern struct fanotify_group *fanotify_find_group (unsigned int group_num,unsigned int mask);extern void fanotify_put_group (struct fanotify_group *group); + +extern int fanotify_create_event_fd (struct fanotify_group *group,struct fanotify_event_metadata *data,int nonblock);#else static inline void fanotify (struct file *file,unsigned int mask)
@@ -90,6 +98,11 @@ static inline struct fanotify_group *fanotify_find_group(unsigned int group_num,static inline extern void fanotify_put_group (struct fanotify_group *group) {} +static inline int fanotify_create_event_fd (struct fanotify_group *group,struct fanotify_event_metadata *data,int nonblock) +{ +memset (data,0 ,sizeof (struct fanotify_event_metadata )); +return 0 ; +}#endif /* CONFIG_FANOTIFY */ #endif /* __KERNEL __ */
include/linux/socket.h +4 -1
@@ -191,7 +191,8 @@ struct ucred {#define AF_RXRPC 33/* RxRPC sockets */ #define AF_ISDN 34/* mISDN sockets */ #define AF_PHONET 35/* Phonet sockets */ -#define AF_MAX 36/* For now.. */ +#define AF_FANOTIFY 36/* fscking all access sockets */ +#define AF_MAX 37/* For now.. */ /* Protocol families, same as address families. */ #define PF_UNSPEC AF_UNSPEC
@@ -229,6 +230,7 @@ struct ucred {#define PF_RXRPC AF_RXRPC#define PF_ISDN AF_ISDN#define PF_PHONET AF_PHONET +#define PF_FANOTIFY AF_FANOTIFY#define PF_MAX AF_MAX/* Maximum queue length specifiable by listen. */
@@ -298,6 +300,7 @@ struct ucred {#define SOL_PPPOL2TP 273#define SOL_BLUETOOTH 274#define SOL_PNPIPE 275 +#define SOL_FANOTIFY 276/* IPX options */ #define IPX_TYPE 1
net/Makefile +1 -0
@@ -44,6 +44,7 @@ obj-$(CONFIG_ATM) += atm/ obj-$(CONFIG_DECNET) += decnet/ obj-$(CONFIG_ECONET) += econet/ obj-$(CONFIG_PHONET) += phonet/ +obj-$(CONFIG_FANOTIFY) += fanotify/ ifneq ($(CONFIG_VLAN_8021Q),) obj-y += 8021q/ endif
net/core/sock.c +3 -3
@@ -155,7 +155,7 @@ static const char *af_family_key_strings[AF_MAX+1] = {"sk_lock-27" ,"sk_lock-28" ,"sk_lock-AF_CAN" ,"sk_lock-AF_TIPC" ,"sk_lock-AF_BLUETOOTH" ,"sk_lock-IUCV" ,"sk_lock-AF_RXRPC" ,"sk_lock-AF_ISDN" ,"sk_lock-AF_PHONET" , -"sk_lock-AF_MAX" +"sk_lock-AF_FANOTIFY" ,"sk_lock-AF_MAX" };static const char *af_family_slock_key_strings[AF_MAX +1 ] = { "slock-AF_UNSPEC", "slock-AF_UNIX" , "slock-AF_INET" ,
@@ -170,7 +170,7 @@ static const char *af_family_slock_key_strings[AF_MAX+1] = {"slock-27" ,"slock-28" ,"slock-AF_CAN" ,"slock-AF_TIPC" ,"slock-AF_BLUETOOTH" ,"slock-AF_IUCV" ,"slock-AF_RXRPC" ,"slock-AF_ISDN" ,"slock-AF_PHONET" , -"slock-AF_MAX" +"slock=AF_FANOTIFY" ,"slock-AF_MAX" };static const char *af_family_clock_key_strings[AF_MAX +1 ] = { "clock-AF_UNSPEC", "clock-AF_UNIX" , "clock-AF_INET" ,
@@ -185,7 +185,7 @@ static const char *af_family_clock_key_strings[AF_MAX+1] = {"clock-27" ,"clock-28" ,"clock-AF_CAN" ,"clock-AF_TIPC" ,"clock-AF_BLUETOOTH" ,"clock-AF_IUCV" ,"clock-AF_RXRPC" ,"clock-AF_ISDN" ,"clock-AF_PHONET" , -"clock-AF_MAX" +"clock-AF_FANOTIFY" ,"clock-AF_MAX" };#endif
net/fanotify/Makefile new file +5 -0
@@ -0,0 +1,5 @@ +# +# Makefile for the fanotify AF. +# + +obj-$(CONFIG_FANOTIFY) += af_fanotify.o
net/fanotify/af_fanotify.c new file +240 -0
240 lines not on this page · raw message
net/fanotify/af_fanotify.h new file +20 -0
@@ -0,0 +1,20 @@ +#ifndef _LINUX_AF_FANOTIFY_H +#define _LINUX_AF_FANOTIFY_H + +#ifdef __KERNEL__ + +#include <linux/fanotify.h> +#include <net/sock.h> + +struct fanotify_sock { +struct sock sock; +struct fanotify_group *group; +}; + +static inline struct fanotify_sock *fan_sk (struct sock *sock) +{ +return (struct fanotify_sock *)sock; +} + +#endif /* __KERNEL__ */ +#endif /* _LINUX_AF_NET_H */
PATCH 7/21 fanotify: fastpath to ignore certain in core inodes
In reply to Eric ParisAllows a process to ignore certain events for the given in core inodes. This does not preclude the need to keep a userspace or xattr based cache if a fanotifier listener wants something with a different lifetime than in core inode.
---
fs/inode.c | 6 + fs/notify/Makefile | 2 fs/notify/fanotify.c | 19 ++++ fs/notify/fanotify.h | 34 +++++++ fs/notify/fastpath.c | 230 ++++++++++++++++++++++++++++++++++++++++++++++ fs/notify/group.c | 3 + include/linux/fanotify.h | 17 +++ include/linux/fs.h | 5 + include/linux/fsnotify.h | 8 ++ 9 files changed, 323 insertions(+), 1 deletions(-) create mode 100644 fs/notify/fastpath.c
fs/inode.c +6 -0
@@ -21,6 +21,7 @@#include <linux/cdev.h> #include <linux/bootmem.h> #include <linux/inotify.h> +#include <linux/fsnotify.h> #include <linux/mount.h> /*
@@ -183,6 +184,10 @@ static struct inode *alloc_inode(struct super_block *sb) } inode->i_private =NULL ; inode->i_mapping = mapping; +#ifdef CONFIG_FANOTIFY +INIT_LIST_HEAD (&inode->fastpath_entries); +rwlock_init (&inode->fastpath_rwlock); +#endif }return inode; }
@@ -191,6 +196,7 @@ void destroy_inode(struct inode *inode) {BUG_ON (inode_has_buffers (inode));security_inode_free (inode); +fsnotify_inode_delete (inode);if (inode->i_sb->s_op->destroy_inode) inode->i_sb->s_op->destroy_inode (inode);else
fs/notify/Makefile +1 -1
@@ -3,4 +3,4 @@ obj-$(CONFIG_INOTIFY_USER) += inotify_user.o obj-$(CONFIG_DNOTIFY) += dnotify.o -obj-$(CONFIG_FANOTIFY) += fanotify.o notification.o group.o +obj-$(CONFIG_FANOTIFY) += fanotify.o notification.o group.o fastpath.o
fs/notify/fanotify.c +19 -0
@@ -35,6 +35,15 @@struct dentry *fanotify_fs_root; +void fanotify_inode_delete (struct inode *inode) +{ +if (likely (list_empty (&fanotify_groups))) +return ; + +fanotify_fastpath_clear (inode); +} +EXPORT_SYMBOL_GPL (fanotify_inode_delete); +void fanotify (struct file *file,unsigned int mask) {struct fanotify_group *group;
@@ -52,6 +61,8 @@ void fanotify(struct file *file, unsigned int mask)if (!S_ISREG (inode->i_mode))return ; +if (mask &FAN_MODIFY ) +fanotify_fastpath_clear (inode);/* * SRCU!! the groups list is very very much read only and the path is * very hot (assuming something is using fanotify) Not blocking while
@@ -67,6 +78,8 @@ void fanotify(struct file *file, unsigned int mask) idx =srcu_read_lock (&fanotify_grp_srcu_struct);list_for_each_entry_rcu (group, &fanotify_groups, group_list) {if (mask & group->mask) { +if (fanotify_is_fastpath (file, mask, group)) +continue ;if (!event) { event =create_event (file, mask);/* shit, we OOM'd and now we can't tell, lets hope something else blows up */
@@ -94,6 +107,12 @@ static __init int fanotify_init(void)if (ret)return ret; + ret =fanotify_fastpath_init (); +if (ret) { +fanotify_notification_uninit (); +return ret; + } +return init_srcu_struct (&fanotify_grp_srcu_struct); }__initcall (fanotify_init);
fs/notify/fanotify.h +34 -0
@@ -46,6 +46,31 @@ struct fanotify_event {atomic_t refcnt;/* how many groups still are using/need to send this event */ }; +/* +* a fastpath is simply an entry attached to an in core inode which allows an +* fanotify listener to indicate they are no longer interested in events of +* a type matching mask. +* +* these are flushed when an inode is evicted from core or when the inode is +* modified (as seen by fsnotify_access) +*/ +struct fanotify_fastpath_entry { +struct fanotify_group *group;/* group this fastpath entry is for */ +unsigned int mask;/* mask this fastpath entry is for */ +struct inode *inode;/* inode this entry is associated with */ +/* +* killcnt starts at 0 and is incremented when this entry will be killed +* since multiple processes my try to kill it at the same time. +* One might try to kill it from walking the i_list when an inode is being +* kicked out of cache and one might try to kill it when an fanotify group +* is being unregistered. The last one to finish cleaning everything up +* and decrement it to 0 will do the actual killing +*/ +atomic_t killcnt; +struct list_head i_list;/* list of fastpath_entries by inode->fastpath_entries */ +struct list_head g_list;/* list of fastpath_entries by group->fastpath_entries */ +}; +extern struct srcu_struct fanotify_grp_srcu_struct;extern struct list_head fanotify_groups;
@@ -60,4 +85,13 @@ extern struct fanotify_event_holder *alloc_event_holder(void);extern void fanotify_destroy_event_holder (struct fanotify_event_holder *holder);extern __init int fanotify_notification_init (void );extern __init int fanotify_notification_uninit (void ); + +extern void fanotify_fastpath_get (struct fanotify_fastpath_entry *entry); +extern void fanotify_fastpath_put (struct fanotify_fastpath_entry *entry); +extern int fanotify_is_fastpath (struct file *file,unsigned int mask,struct fanotify_group *group); +extern void fanotify_fastpath_clear_group (struct fanotify_group *group); +extern void fanotify_fastpath_clear (struct inode *inode); +extern __init int fanotify_fastpath_init (void ); +extern __init int fanotify_fastpath_uninit (void ); +#endif /* _LINUX_FANOTIFY_PRIVATE_H */
fs/notify/fastpath.c new file +230 -0
230 lines not on this page · raw message
fs/notify/group.c +3 -0
@@ -87,6 +87,9 @@ void fanotify_kill_group(struct fanotify_group *group)/* clear the notification queue of all events */ fanotify_notification_clear_group (group); +/* clear all fastpath entries for this group */ +fanotify_fastpath_clear_group (group); +kfree (group); }
include/linux/fanotify.h +17 -0
@@ -71,17 +71,25 @@ struct fanotify_group {struct mutex notification_mutex;/* protect the notification_list */ struct list_head notification_list;/* list of event_holder this group needs to send to userspace */ wait_queue_head_t notification_waitq;/* read() on the notification file blocks on this waitq */ + +/* stores all fastapth entries assoc with this group so they can be cleaned on unregister */ +struct mutex fastpath_mutex;/* protect fastpath_entries list */ +struct list_head fastpath_entries;/* all fastpath entries for this group */ };#ifdef CONFIG_FANOTIFY extern void fanotify (struct file *file,unsigned int mask); +extern void fanotify_inode_delete (struct inode *inode);extern void fanotify_get_group (struct fanotify_group *group);extern struct fanotify_group *fanotify_find_group (unsigned int group_num,unsigned int mask);extern void fanotify_put_group (struct fanotify_group *group); +/* things called from the socket */ extern int fanotify_create_event_fd (struct fanotify_group *group,struct fanotify_event_metadata *data,int nonblock); +extern int fanotify_fastpath_add (struct fanotify_group *group,int fd,unsigned int mask); +#else static inline void fanotify (struct file *file,unsigned int mask)
@@ -103,6 +111,15 @@ static inline int fanotify_create_event_fd(struct fanotify_group *group, structmemset (data,0 ,sizeof (struct fanotify_event_metadata ));return 0 ; } + +static inline int fanotify_fastpath_add (struct fanotify_group *group,int fd,unsigned int mask) +{ +return 0 ; +} + +static inline void fanotify_inode_delete (struct inode *inode) +{} +#endif /* CONFIG_FANOTIFY */ #endif /* __KERNEL __ */
include/linux/fs.h +5 -0
@@ -674,6 +674,11 @@ struct inode {struct mutex inotify_mutex;/* protects the watches list */ #endif +#ifdef CONFIG_FANOTIFY +struct list_head fastpath_entries;/* fanotify fastpath entries protected by group */ +rwlock_t fastpath_rwlock;/* protect the fastpath entries list */ +#endif +unsigned long i_state;unsigned long dirtied_when;/* jiffies of first dirtying */
include/linux/fsnotify.h +8 -0
@@ -82,6 +82,14 @@ static inline void fsnotify_nameremove(struct dentry *dentry, int isdir) }/* +* fsnotify_inode_delete - and inode is being evicted from cache, clean up is needed +*/ +static inline void fsnotify_inode_delete (struct inode *inode) +{ +fanotify_inode_delete (inode); +} + +/* * fsnotify_inoderemove - an inode is going away */ static inline void fsnotify_inoderemove (struct inode *inode)
Re: [PATCH =-v3 07/21] fanotify: fastpath to ignore certain in core inodes
Quoted 4 lines · Eric Paris
Allows a process to ignore certain events for the given in core inodes. This does not preclude the need to keep a userspace or xattr based cache if a fanotifier listener wants something with a different lifetime than in core inode.
Please just do this in userspace instead of bloating the inode.
Re: [PATCH =-v3 07/21] fanotify: fastpath to ignore certain in core inodes
Quoted 7 lines · On Wed, 12 Nov 2008 11:50:55 -0500 Christoph Hellwig <[email protected]> wrote:
Quoted 4 lines · Eric Paris
Allows a process to ignore certain events for the given in core inodes. This does not preclude the need to keep a userspace or xattr based cache if a fanotifier listener wants something with a different lifetime than in core inode.
Please just do this in userspace instead of bloating the inode.
Sounds a good way of ruining the performance.
Alan
Re: [PATCH =-v3 07/21] fanotify: fastpath to ignore certain in core inodes
Quoted 1 line · Alan Cox
Sounds a good way of ruining the performance.
I don't think you can ruine performance of a system with AV systems running even more. While bloating the inode does cause enormous problems in file serving or other extremly metadata intensive workloads.
Re: [PATCH =-v3 07/21] fanotify: fastpath to ignore certain in core inodes
Quoted 6 lines · Christoph Hellwig
Quoted 1 line · Alan Cox
Sounds a good way of ruining the performance.
I don't think you can ruine performance of a system with AV systems running even more. While bloating the inode does cause enormous problems in file serving or other extremly metadata intensive workloads.
Kernel Build on a 32 way machine:
Stock kernel: 9 minutes 12 seconds
fanotify no in kernel fastpath: 95 minutes 12 seconds Only events AV wants with in kernel fastpath: 10 minutes 35 seconds
Now I could probably redo the in kernel cache as some sort of per group inode hash table of entries which wouldn't have to bloat the inode but it would be much more expensive in terms of performance. A lock and an list if you want to use fanotify is as small as it can be made an is exactly the same method taken by inotify.
************
I probably should have put this up in the patch description rather than burried down in the Documentation directory but here it is:
The need for fastpaths (or calling what it really is, an in kernel cache) has been questioned. I decided to include a little unscientific data here. On a 32 way machine a make -j 32 took about 9 minutes 12 seconds. With that same machine running having one group and 32 listeners receiving every fanotify event that the kernel could send to userspace while the listeners were responding to accesses as fast as they could (just a very tight get event, allow loop) it took 95 minutes 12 seconds. Same process with in kernel fastpaths/cache results took 19 minutes 5 seconds. More reasonable event requirements and a single listener took 10 minutes 35 sec. So about a 15% perf hit to do any kind of permission checking to userspace. Anyway, the need for fastpaths is quite clear.
Re: [PATCH =-v3 07/21] fanotify: fastpath to ignore certain in core inodes
Quoted 4 lines · Eric Paris
Kernel Build on a 32 way machine:
Stock kernel: 9 minutes 12 seconds
fanotify no in kernel fastpath: 95 minutes 12 seconds Only events AV wants with in kernel fastpath: 10 minutes 35 seconds
Can you provide the number of calls into the listeners for each type of fanotify event? It would provide a better picture of the scope of the issue.
I find your non-fastpath numbers quite alarming.
The DazukoFS stackable filesystem can also be used for userspace listeners to acknowledge/handle filesystem events. (I only mention DazukoFS here so that there is some kind of metric comparison that can be used.) I find a 15% overhead in communication to be too expensive. And a 934.7% performance hit is... well...
The DazukoFS stackable filesystem only provides file open events, but its numbers may still be of interest here:
kernel build on tmpfs: 72m36.884s
kernel build on tmpfs+DazukoFS: 75m30.569s
That is an overhead of 4%. Your results show that fanotify (with fastpaths!) has an overhead of 15%.
My test was done using Linux 2.6.32 and DazukoFS 3.1.2. The Linux source and build were located on tmpfs to minimize hard drive hits. The machine was a 600MHz PowerPC, which is why the build times were so long.
During the test, the DazukoFS listeners recorded a total of 991,907 file open events.
IMHO event filtering should happen in userspace. My experience is that the context switches can be quite cheap. Perhaps if we know which fanotify events (and how many) occur within your test, we can begin to understand why your results are what they are.
John Ogness
Re: [PATCH =-v3 07/21] fanotify: fastpath to ignore certain in core inodes
In reply to Eric Parisinclude/linux/fs.h struct inode { · line 680
Quoted 4 lines · Eric Paris
include/linux/fs.h +4
-0+#ifdef CONFIG_FANOTIFY +struct list_head fastpath_entries;/* fanotify fastpath entries protected by group */ +rwlock_t fastpath_rwlock;/* protect the fastpath entries list */ +#endif
Are you really sure those rwlocks actually gain you performance?
rwlocks are really bad, the critical section must be short because its non-preemptable, but both the lock and unlock are atomic ops that can (and usually will) bounce cachelines. So you often get into the situation where your performance is limited by the cacheline bouncing.
There's also a starvation case in there, where the cacheline is so hot on one package/node that another package/node doesn't get it.
Regular spinlocks are usually faster in those cases (and on those archs that have ticket locks avoid the starvation case too).
And for those cases where the read side is long enough to not be dominated by the cacheline bouncing, RCU is usually the best way out.
PATCH 8/21 fanotify: add a userspace interface for fastpaths
In reply to Eric Paristurns out the fastpath code is useless without an interface. This one works.
---
fs/notify/group.c | 3 +++ include/linux/fanotify.h | 9 +++++++++ net/fanotify/af_fanotify.c | 36 +++++++++++++++++++++++++++++++++++- 3 files changed, 47 insertions(+), 1 deletions(-)
fs/notify/group.c +3 -0
@@ -74,6 +74,9 @@ struct fanotify_group *fanotify_find_group(unsigned int group_num, unsigned intINIT_LIST_HEAD (&group->notification_list);init_waitqueue_head (&group->notification_waitq); +mutex_init (&group->fastpath_mutex); +INIT_LIST_HEAD (&group->fastpath_entries); +/* add it */ list_add_rcu (&group->group_list, &fanotify_groups);
include/linux/fanotify.h +9 -0
@@ -52,6 +52,15 @@ struct fanotify_event_metadata {/* fanotify getsockopt optvals */ #define FANOTIFY_GET_EVENT 1 +/* struct used for FANOTIFY_SET_FASTPATH */ +struct fanotify_so_fastpath { +__s32 fd; +__u32 mask; +} __attribute__((packed)); + +/* fanotify setsockopt optvals */ +#define FANOTIFY_SET_FASTPATH 1 +#ifdef __KERNEL__#include <linux/fs.h>
net/fanotify/af_fanotify.c +35 -1
@@ -156,6 +156,40 @@ static int fan_bind(struct socket *sock, struct sockaddr *addr, int addr_len)return 0 ; } +static int fan_setsockopt (struct socket *sock,int level,int optname,char __user *optval,int optlen) +{ +struct fanotify_sock *fan_sock; +struct fanotify_group *group; +union { +struct fanotify_so_fastpath fastpath; + } data; +int ret =0 ; + +if (sock->state !=SS_CONNECTED ) +return -EBADF ; + +if (level !=SOL_FANOTIFY ) +return -ENOPROTOOPT ; + + fan_sock =fan_sk (sock->sk); + group = fan_sock->group; + +switch (optname) { +case FANOTIFY_SET_FASTPATH : +if (optlen <sizeof (struct fanotify_so_fastpath )) +return -ENOMEM ; + ret =copy_from_user (&data.fastpath, optval,sizeof (struct fanotify_so_fastpath )); +if (ret) +return ret; + ret =fanotify_fastpath_add (group, data.fastpath.fd, data.fastpath.mask); +break ; +default : +return -ENOPROTOOPT ; + } + +return ret; +} +static int fan_getsockopt (struct socket *sock,int level,int optname,char __user *optval,int __user *optlen) {struct fanotify_sock *fan_sock;
@@ -218,7 +252,7 @@ static const struct proto_ops fanotify_proto_ops = { .ioctl = sock_no_ioctl, .listen = sock_no_listen, .shutdown = sock_no_shutdown, - .setsockopt = sock_no_setsockopt, + .setsockopt = fan_setsockopt, .getsockopt = fan_getsockopt, .sendmsg = sock_no_sendmsg, .recvmsg = sock_no_recvmsg,
PATCH 9/21 fanotify: add group priorities
In reply to Eric ParisIn preperation for blocking fanotify calls group priorities must be added. async events will be sent as quickly as possible without waiting for a response.
---
fs/notify/group.c | 30 ++++++++++++++++++++++++------ include/linux/fanotify.h | 5 ++++- net/fanotify/af_fanotify.c | 2 +- 3 files changed, 29 insertions(+), 8 deletions(-)
fs/notify/group.c +24 -6
@@ -42,15 +42,16 @@ void fanotify_get_group(struct fanotify_group *group)atomic_inc (&group->refcnt); } -struct fanotify_group *fanotify_find_group (unsigned int group_num,unsigned int mask) +struct fanotify_group *fanotify_find_group (unsigned int priority,unsigned int group_num,unsigned int mask) {struct fanotify_group *group_iter;struct fanotify_group *group =NULL ;mutex_lock (&fanotify_grp_mutex);list_for_each_entry_rcu (group_iter, &fanotify_groups, group_list) { -if (group_iter->group_num == group_num) { -if (group_iter->mask == mask) { +if (group_iter->priority == priority) { +if ((group_iter->mask == mask) && + (group_iter->group_num == group_num)) {fanotify_get_group (group_iter); group = group_iter; }else
@@ -69,6 +70,7 @@ struct fanotify_group *fanotify_find_group(unsigned int group_num, unsigned int group->group_num = group_num; group->mask = mask; + group->priority = priority;mutex_init (&group->notification_mutex);INIT_LIST_HEAD (&group->notification_list);
@@ -77,9 +79,26 @@ struct fanotify_group *fanotify_find_group(unsigned int group_num, unsigned intmutex_init (&group->fastpath_mutex);INIT_LIST_HEAD (&group->fastpath_entries); -/* add it */ -list_add_rcu (&group->group_list, &fanotify_groups); +/* Do we need to be the first entry? */ +if (list_empty (&fanotify_groups)) { +list_add_rcu (&group->group_list, &fanotify_groups); + goto out; + } + +list_for_each_entry (group_iter, &fanotify_groups, group_list) { +/* insert in front of this one? */ +if (priority < group_iter->priority) { +/* I used list_add_tail() to insert in front of group_iter... */ +list_add_tail_rcu (&group->group_list, &group_iter->group_list); +break ; + } +/* are we at the end? if so insert at end */ +if (list_is_last (&group_iter->group_list, &fanotify_groups)) { +list_add_tail_rcu (&group->group_list, &fanotify_groups); +break ; + } + } out:mutex_unlock (&fanotify_grp_mutex);return group;
@@ -100,7 +119,6 @@ void fanotify_put_group(struct fanotify_group *group) {mutex_lock (&fanotify_grp_mutex);if (atomic_dec_and_test (&group->refcnt)) { -list_del_rcu (&group->group_list);mutex_unlock (&fanotify_grp_mutex);
include/linux/fanotify.h +4 -1
@@ -37,6 +37,7 @@#include <linux/types.h> struct fanotify_addr { +__u32 priority;__u32 group_num;__u32 mask;__u32 timeout;
@@ -84,6 +85,8 @@ struct fanotify_group {/* stores all fastapth entries assoc with this group so they can be cleaned on unregister */ struct mutex fastpath_mutex;/* protect fastpath_entries list */ struct list_head fastpath_entries;/* all fastpath entries for this group */ + +unsigned int priority;/* order this group should receive msgs. low first */ };#ifdef CONFIG_FANOTIFY
@@ -92,7 +95,7 @@ extern void fanotify(struct file *file, unsigned int mask);extern void fanotify_inode_delete (struct inode *inode);extern void fanotify_get_group (struct fanotify_group *group); -extern struct fanotify_group *fanotify_find_group (unsigned int group_num,unsigned int mask); +extern struct fanotify_group *fanotify_find_group (unsigned int priority,unsigned int group_num,unsigned int mask);extern void fanotify_put_group (struct fanotify_group *group);/* things called from the socket */
net/fanotify/af_fanotify.c +1 -1
@@ -145,7 +145,7 @@ static int fan_bind(struct socket *sock, struct sockaddr *addr, int addr_len)return -EINVAL ; fan_sock =fan_sk (sock->sk); - fan_sock->group =fanotify_find_group (fan_addr->group_num, fan_addr->mask); + fan_sock->group =fanotify_find_group (fan_addr->priority, fan_addr->group_num, fan_addr->mask);if (IS_ERR (fan_sock->group))return PTR_ERR (fan_sock->group);
PATCH 10/21 fanotify: blocking and access granting
In reply to Eric ParisThis patch introduces blocking an access granting for fanotify. Events are sent to userspace and the original process is blocked (assuming not O_NONBLOCK) for at most the amount of time determined by the group. Total maximum blocking time is the sum of all groups possible blocking time.
O_NONBLOCK processes trying to read and a group registered to require blocking/access control on read MUST have a fastpath entry in order to ever make progress.
---
fs/aio.c | 7 ++ fs/notify/Makefile | 2 - fs/notify/access.c | 164 ++++++++++++++++++++++++++++++++++++++++++++++ fs/notify/fanotify.c | 46 +++++++++++-- fs/notify/fanotify.h | 17 ++++- fs/notify/group.c | 8 ++ fs/notify/notification.c | 2 + fs/open.c | 5 + fs/read_write.c | 6 ++ include/linux/fanotify.h | 35 +++++++++- 10 files changed, 278 insertions(+), 14 deletions(-) create mode 100644 fs/notify/access.c
fs/aio.c +7 -0
@@ -31,6 +31,7 @@#include <linux/workqueue.h> #include <linux/security.h> #include <linux/eventfd.h> +#include <linux/fanotify.h> #include <asm/kmap_types.h> #include <asm/uaccess.h>
@@ -1454,6 +1455,9 @@ static ssize_t aio_setup_iocb(struct kiocb *kiocb) ret =security_file_permission (file,MAY_READ );if (unlikely (ret))break ; + ret =fanotify (file,FAN_ACCESS_PERM ); +if (unlikely (ret)) +break ; ret =aio_setup_single_vector (kiocb);if (ret)break ;
@@ -1486,6 +1490,9 @@ static ssize_t aio_setup_iocb(struct kiocb *kiocb) ret =security_file_permission (file,MAY_READ );if (unlikely (ret))break ; + ret =fanotify (file,FAN_ACCESS_PERM ); +if (unlikely (ret)) +break ; ret =aio_setup_vectored_rw (READ , kiocb);if (ret)break ;
fs/notify/Makefile +1 -1
@@ -3,4 +3,4 @@ obj-$(CONFIG_INOTIFY_USER) += inotify_user.o obj-$(CONFIG_DNOTIFY) += dnotify.o -obj-$(CONFIG_FANOTIFY) += fanotify.o notification.o group.o fastpath.o +obj-$(CONFIG_FANOTIFY) += fanotify.o notification.o group.o fastpath.o access.o
fs/notify/access.c new file +164 -0
164 lines not on this page · raw message
fs/notify/fanotify.c +40 -6
75 lines not on this page · raw message
fs/notify/fanotify.h +13 -4
@@ -14,12 +14,14 @@#include <linux/wait.h> /* -* A single event can be queued in multiple group->notification_lists. +* A single event can be queued in multiple group->notification_lists and to at +* most one group->access_list at the same time. * -* each group->notification_list will point to an event_holer which in turns points -* to the actual event that needs to be sent to userspace. +* each group->notification_list or group->access_list will point to an +* event_holer which in turns points to the actual event that needs to be sent +* to userspace. * -* Seemed cheaper to create a refcnt'd event and a small holder for every group +* Seemed cheaper to create a refcnt'd event and a small holder for every list * than create a different event for every group * */
@@ -44,6 +46,9 @@ struct fanotify_event {struct path path;/* path from the original access */ unsigned int mask;/* the type of access */ atomic_t refcnt;/* how many groups still are using/need to send this event */ +/* if waiting for a userspace access answer this is the cookie they will send back */ +unsigned long cookie; +unsigned int response;/* userspace answer to question */ };/*
@@ -94,4 +99,8 @@ extern void fanotify_fastpath_clear(struct inode *inode);extern __init int fanotify_fastpath_init (void );extern __init int fanotify_fastpath_uninit (void ); +extern int fanotify_add_event_to_access (struct fanotify_group *group,struct fanotify_event *event); +extern int fanotify_get_response_from_access (struct fanotify_group *group,struct fanotify_event *event); +extern void fanotify_access_clear_group (struct fanotify_group *group); +#endif /* _LINUX_FANOTIFY_PRIVATE_H */
fs/notify/group.c +8 -0
@@ -79,6 +79,11 @@ struct fanotify_group *fanotify_find_group(unsigned int priority, unsigned int gmutex_init (&group->fastpath_mutex);INIT_LIST_HEAD (&group->fastpath_entries); +atomic_set (&group->cookie,0 ); +mutex_init (&group->access_mutex); +INIT_LIST_HEAD (&group->access_list); +init_waitqueue_head (&group->access_waitq); +/* Do we need to be the first entry? */ if (list_empty (&fanotify_groups)) {list_add_rcu (&group->group_list, &fanotify_groups);
@@ -112,6 +117,9 @@ void fanotify_kill_group(struct fanotify_group *group)/* clear all fastpath entries for this group */ fanotify_fastpath_clear_group (group); +/* set an allow for all outstanding access decisions */ +fanotify_access_clear_group (group); +kfree (group); }
fs/notify/notification.c +2 -0
@@ -176,6 +176,7 @@ struct fanotify_event *create_event(struct file *file, unsigned int mask) event->path.mnt = file->f_path.mnt;path_get (&event->path); + event->cookie =0 ; event->mask = mask;WARN_ON (!event->path.dentry);
@@ -241,6 +242,7 @@ int fanotify_create_event_fd(struct fanotify_group *group, struct fanotify_event data->fd = client_fd; data->mask = event->mask; + data->cookie = event->cookie;fanotify_put_event (event);
fs/open.c +5 -0
@@ -29,6 +29,7 @@#include <linux/rcupdate.h> #include <linux/audit.h> #include <linux/falloc.h> +#include <linux/fanotify.h> int vfs_statfs (struct dentry *dentry,struct kstatfs *buf) {
@@ -820,6 +821,10 @@ static struct file *__dentry_open(struct dentry *dentry, struct vfsmount *mnt,if (error) goto cleanup_all; + error =fanotify (f,FAN_OPEN_PERM ); +if (error) + goto cleanup_all; +if (!open && f->f_op) open = f->f_op->open;if (open) {
fs/read_write.c +6 -0
@@ -16,6 +16,7 @@#include <linux/syscalls.h> #include <linux/pagemap.h> #include <linux/splice.h> +#include <linux/fanotify.h> #include "read_write.h" #include <asm/uaccess.h>
@@ -223,6 +224,11 @@ int rw_verify_area(int read_write, struct file *file, loff_t *ppos, size_t count read_write == READ ? MAY_READ : MAY_WRITE);if (retval)return retval; +if (read_write ==READ ) { + retval =fanotify (file,FAN_ACCESS_PERM ); +if (retval) +return retval; + }return count >MAX_RW_COUNT ?MAX_RW_COUNT : count; }
include/linux/fanotify.h +32 -3
@@ -17,6 +17,10 @@#define FAN_OPEN_NOEXEC 0x00000010/* File was opened */ #define FAN_OPEN_EXEC 0x00000020/* File was opened with the intention of being exec'ed */ +/* userspace may also request blocking for permission checks for open and read */ +#define FAN_ACCESS_PERM 0x00000100 +#define FAN_OPEN_PERM 0x00000200 +/* FIXME currently Q's have no limit.... */ #define FAN_Q_OVERFLOW 0x80000000/* Event queued overflowed */
@@ -33,6 +37,16 @@ FAN_MODIFY |\ FAN_CLOSE |\ FAN_OPEN) +/* +* Like the above list of events only these are the event types in which the +* kernel will wait for answers. +*/ +#define FAN_ALL_EVENTS_PERM (FAN_OPEN_PERM |\ + FAN_ACCESS_PERM) + +/* answers will need to be either allow or deny */ +#define FAN_ALLOW 0x00000001 +#define FAN_DENY 0x00000002#include <linux/types.h>
@@ -48,6 +62,7 @@ struct fanotify_addr {struct fanotify_event_metadata {__s32 fd;__u32 mask; +__u64 cookie; } __attribute__((packed));/* fanotify getsockopt optvals */
@@ -86,12 +101,18 @@ struct fanotify_group {struct mutex fastpath_mutex;/* protect fastpath_entries list */ struct list_head fastpath_entries;/* all fastpath entries for this group */ +/* needed to track outstanding requests we expect to hear from userspace */ +atomic_long_t cookie;/* next cookie send to userspace for a decision */ +struct mutex access_mutex;/* protect access_list list */ +struct list_head access_list;/* event_holder we need an answer from userspace */ +wait_queue_head_t access_waitq;/* we wait here for userspace access decisions */ +unsigned int priority;/* order this group should receive msgs. low first */ };#ifdef CONFIG_FANOTIFY -extern void fanotify (struct file *file,unsigned int mask); +extern int fanotify (struct file *file,unsigned int mask);extern void fanotify_inode_delete (struct inode *inode);extern void fanotify_get_group (struct fanotify_group *group);
@@ -101,11 +122,14 @@ extern void fanotify_put_group(struct fanotify_group *group);/* things called from the socket */ extern int fanotify_create_event_fd (struct fanotify_group *group,struct fanotify_event_metadata *data,int nonblock);extern int fanotify_fastpath_add (struct fanotify_group *group,int fd,unsigned int mask); +extern int fanotify_process_access_response (struct fanotify_group *group,unsigned long cookie,unsigned int response);#else -static inline void fanotify (struct file *file,unsigned int mask) -{} +static inline int fanotify (struct file *file,unsigned int mask) +{ +return 0 ; +}static inline void fanotify_get_group (struct fanotify_group *group) {}
@@ -132,6 +156,11 @@ static inline int fanotify_fastpath_add(struct fanotify_group *group, int fd, unstatic inline void fanotify_inode_delete (struct inode *inode) {} +static inline int fanotify_process_access_response (struct fanotify_group *group,unsigned long cookie,unsigned int response) +{ +return 0 ; +} +#endif /* CONFIG_FANOTIFY */ #endif /* __KERNEL __ */
PATCH 11/21 fanotify: give a special access permission check
In reply to Eric Parisadd a special FANOTIFY_ACCESS_EXEC_PERM check. This will be applied any place a process tries to access a file with execute permissions. Mainly sys_execve, sys_uselib, and mmap'ing a file for exec.
---
fs/aio.c | 4 ++-- fs/exec.c | 10 ++++++++++ fs/read_write.c | 2 +- include/linux/fanotify.h | 6 ++++-- mm/mmap.c | 7 +++++++ mm/mprotect.c | 6 ++++++ mm/nommu.c | 7 +++++++ 7 files changed, 37 insertions(+), 5 deletions(-)
fs/aio.c +2 -2
@@ -1455,7 +1455,7 @@ static ssize_t aio_setup_iocb(struct kiocb *kiocb) ret =security_file_permission (file,MAY_READ );if (unlikely (ret))break ; - ret =fanotify (file,FAN_ACCESS_PERM ); + ret =fanotify (file,FAN_ACCESS_NOEXEC_PERM );if (unlikely (ret))break ; ret =aio_setup_single_vector (kiocb);
@@ -1490,7 +1490,7 @@ static ssize_t aio_setup_iocb(struct kiocb *kiocb) ret =security_file_permission (file,MAY_READ );if (unlikely (ret))break ; - ret =fanotify (file,FAN_ACCESS_PERM ); + ret =fanotify (file,FAN_ACCESS_NOEXEC_PERM );if (unlikely (ret))break ; ret =aio_setup_vectored_rw (READ , kiocb);
fs/exec.c +10 -0
@@ -137,6 +137,11 @@ asmlinkage long sys_uselib(const char __user * library) goto out;fsnotify_open_exec (file); + error =fanotify (file,FAN_ACCESS_EXEC_PERM ); +if (error) { +fput (file); + goto out; + } error = -ENOEXEC ;if (file->f_op) {
@@ -691,6 +696,11 @@ struct file *open_exec(const char *name)return file;fsnotify_open_exec (file); + err =fanotify (file,FAN_ACCESS_EXEC_PERM ); +if (err) { +fput (file); + goto out; + } err =deny_write_access (file);if (err) {
fs/read_write.c +1 -1
@@ -225,7 +225,7 @@ int rw_verify_area(int read_write, struct file *file, loff_t *ppos, size_t countif (retval)return retval;if (read_write ==READ ) { - retval =fanotify (file,FAN_ACCESS_PERM ); + retval =fanotify (file,FAN_ACCESS_NOEXEC_PERM );if (retval)return retval; }
include/linux/fanotify.h +4 -2
@@ -18,8 +18,9 @@#define FAN_OPEN_EXEC 0x00000020/* File was opened with the intention of being exec'ed */ /* userspace may also request blocking for permission checks for open and read */ -#define FAN_ACCESS_PERM 0x00000100 -#define FAN_OPEN_PERM 0x00000200 +#define FAN_ACCESS_NOEXEC_PERM 0x00000100 +#define FAN_ACCESS_EXEC_PERM 0x00000200 +#define FAN_OPEN_PERM 0x00000400/* FIXME currently Q's have no limit.... */ #define FAN_Q_OVERFLOW 0x80000000/* Event queued overflowed */
@@ -27,6 +28,7 @@/* helper events */ #define FAN_CLOSE (FAN_CLOSE_WRITE | FAN_CLOSE_NOWRITE)/* close */ #define FAN_OPEN (FAN_OPEN_NOEXEC | FAN_OPEN_EXEC)/* open */ +#define FAN_ACCESS_PERM (FAN_ACCESS_NOEXEC_PERM | FAN_ACCESS_EXEC_PERM)/* access perm */ /* * All of the events - we build the list by hand so that we can add flags in
mm/mmap.c +7 -0
@@ -18,6 +18,7 @@#include <linux/init.h> #include <linux/file.h> #include <linux/fs.h> +#include <linux/fanotify.h> #include <linux/personality.h> #include <linux/security.h> #include <linux/hugetlb.h>
@@ -1027,6 +1028,12 @@ unsigned long do_mmap_pgoff(struct file * file, unsigned long addr,default :return -EINVAL ; } + +if (prot &PROT_EXEC ) { + error =fanotify (file,FAN_ACCESS_EXEC_PERM ); +if (error) +return error; + } }else {switch (flags &MAP_TYPE ) {case MAP_SHARED :
mm/mprotect.c +6 -0
@@ -16,6 +16,7 @@#include <linux/fs.h> #include <linux/highmem.h> #include <linux/security.h> +#include <linux/fanotify.h> #include <linux/mempolicy.h> #include <linux/personality.h> #include <linux/syscalls.h>
@@ -294,6 +295,11 @@ sys_mprotect(unsigned long start, size_t len, unsigned long prot)if (error) goto out; +if (vma->vm_file && (prot &PROT_EXEC )) + error =fanotify (vma->vm_file,FAN_ACCESS_EXEC_PERM ); +if (error) + goto out; + tmp = vma->vm_end;if (tmp > end) tmp = end;
mm/nommu.c +7 -0
@@ -18,6 +18,7 @@#include <linux/mman.h> #include <linux/swap.h> #include <linux/file.h> +#include <linux/fanotify.h> #include <linux/highmem.h> #include <linux/pagemap.h> #include <linux/slab.h>
@@ -731,6 +732,12 @@ static int validate_mmap_request(struct file *file,/* backing file is not executable, try to copy */ capabilities &= ~BDI_CAP_MAP_DIRECT ; } + +if (prot &PROT_EXEC ) { + ret =fanotify (file,FAN_ACCESS_EXEC_PERM ); +if (ret) +return ret; + } }else {/* anonymous mappings are always memory backed and can be
Re: [PATCH =-v3 11/21] fanotify: give a special access permission check
fs/exec.c asmlinkage long sys_uselib(const char __user * library) · line 144
Quoted 6 lines · Eric Paris
fs/exec.c +5
-0fsnotify_open_exec (file); + error =fanotify (file,FAN_ACCESS_EXEC_PERM ); +if (error) { +fput (file); + goto out; + }
Adding your own fanotify calls next to the fsnotify calls completely defeats th purpose of these. Please make sure we have one set of hooks. And given that we now have three notification schemes I think it's getting time that you also unifify the actuall backends (buffering, etc) instead of faning out at the fsnotify layer and just leave dnotify and inotify as tiny userspace interface layers.
Re: [PATCH =-v3 11/21] fanotify: give a special access permission check
Quoted 10 lines · Christoph Hellwig
fs/exec.c asmlinkage long sys_uselib(const char __user * library) · line 144
Quoted 6 lines · Eric Paris
fs/exec.c +5
-0fsnotify_open_exec (file); + error =fanotify (file,FAN_ACCESS_EXEC_PERM ); +if (error) { +fput (file); + goto out; + }Adding your own fanotify calls next to the fsnotify calls completely defeats th purpose of these. Please make sure we have one set of hooks.
Absolutely in this case. Would you prefer I funneled ALL fanotify blocking calls through fsnotify? I did back in patch #10 add fanotify calls next to some security calls (it also might be possible for me to pull the fsnotify calls of these paths up rather than introduce new fsnotify hooks, I'm not yet certain.)
Quoted 4 lines
And given that we now have three notification schemes I think it's getting time that you also unifify the actuall backends (buffering, etc) instead of faning out at the fsnotify layer and just leave dnotify and inotify as tiny userspace interface layers.
doing something like this would obviously require in kernel fastpaths
(as i'm sure you are aware.) I guess then I get to use the space in
inode of both dnotify and inotify. Like I said I'll try to see what I can do down this path.
No matter what I think this is going to have to be merged first as fanotify is quite broad in comparison to either d or i notify.
-Eric
PATCH 12/21 fanotify: user interface for access decisions
In reply to Eric Paristurns out that access decisions are useless if there is no way to do anything about them. So we add an interface. Alan will hate it, but at least it works...
---
include/linux/fanotify.h | 7 +++++++ net/fanotify/af_fanotify.c | 9 +++++++++ 2 files changed, 16 insertions(+), 0 deletions(-)
include/linux/fanotify.h +7 -0
@@ -76,8 +76,15 @@ struct fanotify_so_fastpath {__u32 mask; } __attribute__((packed)); +/* struct used for FANOTIFY_SEND_RESPONSE */ +struct fanotify_so_access { +__u64 cookie; +__u32 response; +} __attribute__((packed)); +/* fanotify setsockopt optvals */ #define FANOTIFY_SET_FASTPATH 1 +#define FANOTIFY_SEND_RESPONSE 2#ifdef __KERNEL__
net/fanotify/af_fanotify.c +9 -0
@@ -162,6 +162,7 @@ static int fan_setsockopt(struct socket *sock, int level, int optname, char __usstruct fanotify_group *group;union {struct fanotify_so_fastpath fastpath; +struct fanotify_so_access access; } data;int ret =0 ;
@@ -183,6 +184,14 @@ static int fan_setsockopt(struct socket *sock, int level, int optname, char __usreturn ret; ret =fanotify_fastpath_add (group, data.fastpath.fd, data.fastpath.mask);break ; +case FANOTIFY_SEND_RESPONSE : +if (optlen <sizeof (struct fanotify_so_access )) +return -ENOMEM ; + ret =copy_from_user (&data.access, optval,sizeof (struct fanotify_so_access )); +if (ret) +return ret; + ret =fanotify_process_access_response (group, data.access.cookie, data.access.response); +break ;default :return -ENOPROTOOPT ; }
PATCH 13/21 fanotify: ability for userspace to delay responses
In reply to Eric ParisUserspace may know that it is behind, may know it is going to take a while to get a good response, or have any number of other reasons to block an access request. FAN_RESTTIMER will reset the timeout for an fanotify access decision response.
---
fs/notify/access.c | 31 +++++++++++++++++++++++++++---- fs/notify/fanotify.h | 1 + fs/notify/notification.c | 2 ++ include/linux/fanotify.h | 1 + 4 files changed, 31 insertions(+), 4 deletions(-)
fs/notify/access.c +27 -4
@@ -55,6 +55,8 @@ int fanotify_add_event_to_access(struct fanotify_group *group, struct fanotify_eint fanotify_get_response_from_access (struct fanotify_group *group,struct fanotify_event *event) {int ret; + +retry: ret =wait_event_interruptible_timeout (group->access_waitq, event->response,msecs_to_jiffies (5000 ));
@@ -89,13 +91,25 @@ int fanotify_get_response_from_access(struct fanotify_group *group, struct fanot }/* userspace responded, convert to something usable */ +spin_lock (&event->response_lock);switch (event->response) {case FAN_ALLOW : -return 0 ; + ret =0 ; +break ;case FAN_DENY : + ret = -EPERM ; +break ; +case FAN_RESETTIMER : + event->response =0 ; +spin_unlock (&event->response_lock); + goto retry;default : -return -EPERM ; + ret = -EPERM ; +break ; } +spin_unlock (&event->response_lock); + +return ret; }int fanotify_process_access_response (struct fanotify_group *group,unsigned long cookie,unsigned int response)
@@ -108,7 +122,7 @@ int fanotify_process_access_response(struct fanotify_group *group, unsigned long* userspace can send a valid responce or we will clean it up after the * timeout */ -if (response & ~(FAN_ALLOW |FAN_DENY )) +if (response & ~(FAN_ALLOW |FAN_DENY |FAN_RESETTIMER ))return -EINVAL ;mutex_lock (&group->access_mutex);
@@ -117,6 +131,11 @@ int fanotify_process_access_response(struct fanotify_group *group, unsigned longcontinue ; event = holder->event; + +/* FAN_RESETTIMER should be left on the list for later.... */ +if (response ==FAN_RESETTIMER ) +break ; +spin_lock (&event->holder_spinlock); holder->event =NULL ;/* as soon as we do this the event.holder might be reused */
@@ -132,9 +151,13 @@ int fanotify_process_access_response(struct fanotify_group *group, unsigned longif (!event)return -ENOENT ; +spin_lock (&event->response_lock); event->response = response; +spin_unlock (&event->response_lock);wake_up (&group->access_waitq); -fanotify_put_event (event); + +if (response !=FAN_RESETTIMER ) +fanotify_put_event (event);return 0 ; }
fs/notify/fanotify.h +1 -0
@@ -48,6 +48,7 @@ struct fanotify_event {atomic_t refcnt;/* how many groups still are using/need to send this event */ /* if waiting for a userspace access answer this is the cookie they will send back */ unsigned long cookie; +spinlock_t response_lock;/* protects response */ unsigned int response;/* userspace answer to question */ };
fs/notify/notification.c +2 -0
@@ -27,6 +27,7 @@#include <linux/path.h> #include <linux/sched.h> #include <linux/slab.h> +#include <linux/spinlock.h> #include <linux/fanotify.h> #include "fanotify.h"
@@ -169,6 +170,7 @@ struct fanotify_event *create_event(struct file *file, unsigned int mask) event->holder.event =NULL ;INIT_LIST_HEAD (&event->holder.event_list);atomic_set (&event->refcnt,1 ); +spin_lock_init (&event->response_lock);spin_lock_init (&event->holder_spinlock);
include/linux/fanotify.h +1 -0
@@ -49,6 +49,7 @@/* answers will need to be either allow or deny */ #define FAN_ALLOW 0x00000001#define FAN_DENY 0x00000002 +#define FAN_RESETTIMER 0x00000004#include <linux/types.h>
PATCH 14/21 fanotify: send pid with fanotify notification events
In reply to Eric ParisOften things like AV scanners may want to allow access to 'bad' files based on the process making the access help by sending the pid of the originally acting process.
---
fs/notify/fanotify.h | 1 + fs/notify/notification.c | 3 +++ include/linux/fanotify.h | 1 + 3 files changed, 5 insertions(+), 0 deletions(-)
fs/notify/fanotify.h +1 -0
@@ -48,6 +48,7 @@ struct fanotify_event {atomic_t refcnt;/* how many groups still are using/need to send this event */ /* if waiting for a userspace access answer this is the cookie they will send back */ unsigned long cookie; +pid_t pid;/* pid of the original process */ spinlock_t response_lock;/* protects response */ unsigned int response;/* userspace answer to question */ };
fs/notify/notification.c +3 -0
@@ -181,6 +181,8 @@ struct fanotify_event *create_event(struct file *file, unsigned int mask) event->cookie =0 ; event->mask = mask; + event->pid = current->pid; +WARN_ON (!event->path.dentry);WARN_ON (!event->path.mnt);
@@ -245,6 +247,7 @@ int fanotify_create_event_fd(struct fanotify_group *group, struct fanotify_event data->fd = client_fd; data->mask = event->mask; data->cookie = event->cookie; + data->pid = event->pid;fanotify_put_event (event);
include/linux/fanotify.h +1 -0
@@ -66,6 +66,7 @@ struct fanotify_event_metadata {__s32 fd;__u32 mask;__u64 cookie; +pid_t pid; } __attribute__((packed));/* fanotify getsockopt optvals */
PATCH 15/21 fanotify: send tgid with notification messages
In reply to Eric ParisSend tgid of original process to fanotify listeners
---
fs/notify/fanotify.h | 1 + fs/notify/notification.c | 2 ++ include/linux/fanotify.h | 1 + 3 files changed, 4 insertions(+), 0 deletions(-)
fs/notify/fanotify.h +1 -0
@@ -49,6 +49,7 @@ struct fanotify_event {/* if waiting for a userspace access answer this is the cookie they will send back */ unsigned long cookie;pid_t pid;/* pid of the original process */ +pid_t tgid;/* tgid of the original process */ spinlock_t response_lock;/* protects response */ unsigned int response;/* userspace answer to question */ };
fs/notify/notification.c +2 -0
@@ -182,6 +182,7 @@ struct fanotify_event *create_event(struct file *file, unsigned int mask) event->mask = mask; event->pid = current->pid; + event->tgid = current->tgid;WARN_ON (!event->path.dentry);WARN_ON (!event->path.mnt);
@@ -248,6 +249,7 @@ int fanotify_create_event_fd(struct fanotify_group *group, struct fanotify_event data->mask = event->mask; data->cookie = event->cookie; data->pid = event->pid; + data->tgid = event->tgid;fanotify_put_event (event);
include/linux/fanotify.h +1 -0
@@ -67,6 +67,7 @@ struct fanotify_event_metadata {__u32 mask;__u64 cookie;pid_t pid; +pid_t tgid; } __attribute__((packed));/* fanotify getsockopt optvals */
PATCH 16/21 fanotify: send file f_flags along with notifications
In reply to Eric Parisfanotify listeners may be interested in what flags the file associated with a particular event were set. This sends f_flags along with the event metadata.
---
fs/notify/fanotify.h | 1 + fs/notify/notification.c | 2 ++ include/linux/fanotify.h | 1 + 3 files changed, 4 insertions(+), 0 deletions(-)
fs/notify/fanotify.h +1 -0
@@ -50,6 +50,7 @@ struct fanotify_event {unsigned long cookie;pid_t pid;/* pid of the original process */ pid_t tgid;/* tgid of the original process */ +unsigned int f_flags;/* f_flags from the original file */ spinlock_t response_lock;/* protects response */ unsigned int response;/* userspace answer to question */ };
fs/notify/notification.c +2 -0
@@ -183,6 +183,7 @@ struct fanotify_event *create_event(struct file *file, unsigned int mask) event->pid = current->pid; event->tgid = current->tgid; + event->f_flags = file->f_flags;WARN_ON (!event->path.dentry);WARN_ON (!event->path.mnt);
@@ -250,6 +251,7 @@ int fanotify_create_event_fd(struct fanotify_group *group, struct fanotify_event data->cookie = event->cookie; data->pid = event->pid; data->tgid = event->tgid; + data->f_flags = event->f_flags;fanotify_put_event (event);
include/linux/fanotify.h +1 -0
@@ -68,6 +68,7 @@ struct fanotify_event_metadata {__u64 cookie;pid_t pid;pid_t tgid; +__u32 f_flags; } __attribute__((packed));/* fanotify getsockopt optvals */
PATCH 17/21 fanotify: add option to clear all fastpaths
In reply to Eric Parisfanotify users because of a change in decision policy may need to clear all fastpath entries. This simply adds a new setsockopt call which will do exactly that.
---
include/linux/fanotify.h | 1 + net/fanotify/af_fanotify.c | 3 +++ 2 files changed, 4 insertions(+), 0 deletions(-)
include/linux/fanotify.h +1 -0
@@ -89,6 +89,7 @@ struct fanotify_so_access {/* fanotify setsockopt optvals */ #define FANOTIFY_SET_FASTPATH 1#define FANOTIFY_SEND_RESPONSE 2 +#define FANOTIFY_CLEAR_ALL_FASTPATH 3#ifdef __KERNEL__
net/fanotify/af_fanotify.c +3 -0
@@ -192,6 +192,9 @@ static int fan_setsockopt(struct socket *sock, int level, int optname, char __usreturn ret; ret =fanotify_process_access_response (group, data.access.cookie, data.access.response);break ; +case FANOTIFY_CLEAR_ALL_FASTPATH : +fanotify_fastpath_clear_group (group); +break ;default :return -ENOPROTOOPT ; }
PATCH 18/21 fanotify: all userspace to set timeouts
In reply to Eric Parisfanotify when used to make access decisions has a default 5 second timeout. This patch will allow userspace listeners to change their timeout on a group wide basis. Userspace listeners will still be able to reset the timeout for individual events.
---
fs/notify/access.c | 2 +- fs/notify/fanotify.h | 1 - fs/notify/group.c | 2 ++ include/linux/fanotify.h | 8 ++++++++ net/fanotify/af_fanotify.c | 9 +++++++++ 5 files changed, 20 insertions(+), 2 deletions(-)
fs/notify/access.c +1 -1
@@ -59,7 +59,7 @@ int fanotify_get_response_from_access(struct fanotify_group *group, struct fanot retry: ret =wait_event_interruptible_timeout (group->access_waitq, event->response, -msecs_to_jiffies (5000 )); +msecs_to_jiffies (group->timeout));/* Timeout or signal? */ if (ret <=0 ) {struct fanotify_event_holder *holder;
fs/notify/fanotify.h +0 -1
@@ -98,7 +98,6 @@ extern __init int fanotify_notification_uninit(void);extern void fanotify_fastpath_get (struct fanotify_fastpath_entry *entry);extern void fanotify_fastpath_put (struct fanotify_fastpath_entry *entry);extern int fanotify_is_fastpath (struct file *file,unsigned int mask,struct fanotify_group *group); -extern void fanotify_fastpath_clear_group (struct fanotify_group *group);extern void fanotify_fastpath_clear (struct inode *inode);extern __init int fanotify_fastpath_init (void );extern __init int fanotify_fastpath_uninit (void );
fs/notify/group.c +2 -0
@@ -84,6 +84,8 @@ struct fanotify_group *fanotify_find_group(unsigned int priority, unsigned int gINIT_LIST_HEAD (&group->access_list);init_waitqueue_head (&group->access_waitq); + group->timeout =5000 ; +/* Do we need to be the first entry? */ if (list_empty (&fanotify_groups)) {list_add_rcu (&group->group_list, &fanotify_groups);
include/linux/fanotify.h +8 -0
@@ -90,6 +90,7 @@ struct fanotify_so_access {#define FANOTIFY_SET_FASTPATH 1#define FANOTIFY_SEND_RESPONSE 2#define FANOTIFY_CLEAR_ALL_FASTPATH 3 +#define FANOTIFY_SET_TIMEOUT 4#ifdef __KERNEL__
@@ -121,6 +122,8 @@ struct fanotify_group {struct list_head access_list;/* event_holder we need an answer from userspace */ wait_queue_head_t access_waitq;/* we wait here for userspace access decisions */ +unsigned int timeout;/* timeout to wait for access requests in msec */ +unsigned int priority;/* order this group should receive msgs. low first */ };
@@ -137,6 +140,7 @@ extern void fanotify_put_group(struct fanotify_group *group);extern int fanotify_create_event_fd (struct fanotify_group *group,struct fanotify_event_metadata *data,int nonblock);extern int fanotify_fastpath_add (struct fanotify_group *group,int fd,unsigned int mask);extern int fanotify_process_access_response (struct fanotify_group *group,unsigned long cookie,unsigned int response); +extern void fanotify_fastpath_clear_group (struct fanotify_group *group);#else
@@ -175,6 +179,10 @@ static inline int fanotify_process_access_response(struct fanotify_group *group,return 0 ; } +static inline void fanotify_fastpath_clear_group (struct fanotify_group *group) +{} + +#endif /* CONFIG_FANOTIFY */ #endif /* __KERNEL __ */
net/fanotify/af_fanotify.c +9 -0
@@ -163,6 +163,7 @@ static int fan_setsockopt(struct socket *sock, int level, int optname, char __usunion {struct fanotify_so_fastpath fastpath;struct fanotify_so_access access; +uint32_t timeout; } data;int ret =0 ;
@@ -195,6 +196,14 @@ static int fan_setsockopt(struct socket *sock, int level, int optname, char __uscase FANOTIFY_CLEAR_ALL_FASTPATH :fanotify_fastpath_clear_group (group);break ; +case FANOTIFY_SET_TIMEOUT : +if (optlen <sizeof (uint32_t )) +return -ENOMEM ; + ret =copy_from_user (&data.timeout, optval,sizeof (uint32_t )); +if (ret) +return ret; + group->timeout = data.timeout; +break ;default :return -ENOPROTOOPT ; }
Re: [PATCH =-v3 18/21] fanotify: all userspace to set timeouts
Quoted 4 lines · Eric Paris
fanotify when used to make access decisions has a default 5 second timeout. This patch will allow userspace listeners to change their timeout on a group wide basis. Userspace listeners will still be able to reset the timeout for individual events.
Really, I think you start to over-engineer. I think this whole stuff would have a much higher chance if you could come up with a staged set of patchsets, ala:
(1) unify the current *notify schemes
(2) add a filesystem wide notify scheme
(3) add blocking filesystem notifiers
and then maybe a fourth with all the misc little things that I doubt will have much of a chance.
Re: [PATCH =-v3 18/21] fanotify: all userspace to set timeouts
Quoted 10 lines · Christoph Hellwig
Quoted 4 lines · Eric Paris
fanotify when used to make access decisions has a default 5 second timeout. This patch will allow userspace listeners to change their timeout on a group wide basis. Userspace listeners will still be able to reset the timeout for individual events.
Really, I think you start to over-engineer. I think this whole stuff would have a much higher chance if you could come up with a staged set of patchsets, ala:
Quoted 6 lines
(1) unify the current *notify schemes
(2) add a filesystem wide notify scheme
(3) add blocking filesystem notifiersand then maybe a fourth with all the misc little things that I doubt will have much of a chance.
This patch set is mostly lined up from most useful to least and every patch comiles and runs on its own. The request for per listener timeouts is based on the array of potential users. Mainly it was put in to allow HSMs to set a high timeout while they ran off and pulled a tape out of the robot. I already let them reset the timeout if they need more time, so I'm find with dropping it if it is so hard to handle.
I'll take a look at #1 but #2 and #3 is exactly what I did. I added fanotify. I added an interface for it. And then I flushed out #2 to the point it was useful on it's own (and off list I've gotten e-mail indicating we have non-AV/malware people interested in using this part of fanotify )
#3 is filled out from patches 10-13. 14+ are all misc but aside from this particular patch I can't see any other way to achieve the intended goals.
I'd love to hear any ideas on how to unitfy the three. Heck I'd be glad to hear any idea how to unify even inotify and dnotify since all three have such wildly different lifetimes and implementations..
I'll poke at this but for now I see my patches as (mostly) clean, useful, and complete.
-Eric
PATCH 19/21 fanotify: evict misbehaving clients
In reply to Eric Parisfanotify clients that fail to respond to access requests withing their timeout or which registered for read permissions checking but fail to then set fastpaths on O_NONBLOCK files should be evicted. I don't want misbehaving clients to be able to bring down the system.
This patch tracks the number of times we hit one of these catagories. If we have 10 more misses than hits we will evict the group from the global fanotify group list.
register for read permission checks and do not then set fastpaths on
---
fs/notify/access.c | 35 +++++++++++++++++++++++++++++++++++ fs/notify/fanotify.h | 1 + fs/notify/group.c | 21 ++++++++++++++++++++- include/linux/fanotify.h | 6 +++++- 4 files changed, 61 insertions(+), 2 deletions(-)
fs/notify/access.c +35 -0
@@ -29,6 +29,35 @@#include <linux/fanotify.h> #include "fanotify.h" +static inline void userspace_success (struct fanotify_group *group) +{ +if (likely (group->misbehave_count ==0 )) +return ; + +spin_lock (&group->misbehave_lock); +if (group->misbehave_count >0 ) + group->misbehave_count--; +spin_unlock (&group->misbehave_lock); +} + +static inline void userspace_error (struct fanotify_group *group) +{ +int evict =0 ; +spin_lock (&group->misbehave_lock); + group->misbehave_count++; +if (group->misbehave_count >10 ) { + group->misbehave_count =10 ; + evict =1 ; +/* don't allow it to wrap */ +if (unlikely (group->misbehave_count >=INT_MAX )) + group->misbehave_count =10 ; + } +spin_unlock (&group->misbehave_lock); + +if (evict) +fanotify_evict_group (group); +} +int fanotify_add_event_to_access (struct fanotify_group *group,struct fanotify_event *event) {struct fanotify_event_holder *holder;
@@ -83,6 +112,10 @@ retry: }mutex_unlock (&group->access_mutex); +/* we timed out, userspace screwed up */ +if (!ret) { +userspace_error (group); + }/* * if we took a signal, return ERESTARTSYS * if we timed out userspace is broken so return ALLOW
@@ -90,6 +123,8 @@ retry:return ret; } +userspace_success (group); +/* userspace responded, convert to something usable */ spin_lock (&event->response_lock);switch (event->response) {
fs/notify/fanotify.h +1 -0
@@ -82,6 +82,7 @@ struct fanotify_fastpath_entry {extern struct srcu_struct fanotify_grp_srcu_struct;extern struct list_head fanotify_groups; +extern void fanotify_evict_group (struct fanotify_group *group);extern int fanotify_check_notif_queue (struct fanotify_group *group);extern void fanotify_get_event (struct fanotify_event *event);
fs/notify/group.c +20 -1
@@ -86,6 +86,10 @@ struct fanotify_group *fanotify_find_group(unsigned int priority, unsigned int g group->timeout =5000 ; + group->evicted =0 ; +spin_lock_init (&group->misbehave_lock); + group->misbehave_count =0 ; +/* Do we need to be the first entry? */ if (list_empty (&fanotify_groups)) {list_add_rcu (&group->group_list, &fanotify_groups);
@@ -125,11 +129,26 @@ void fanotify_kill_group(struct fanotify_group *group)kfree (group); } +/* +* called when a group is misbehaving and needs to be kicked out. group will +* not get really cleared up until clients exit, but it won't be able to hurt +* the system any more. +*/ +void fanotify_evict_group (struct fanotify_group *group) +{ +mutex_lock (&fanotify_grp_mutex); +if (!group->evicted) +list_del_rcu (&group->group_list); + group->evicted =1 ; +mutex_unlock (&fanotify_grp_mutex); +} +void fanotify_put_group (struct fanotify_group *group) {mutex_lock (&fanotify_grp_mutex);if (atomic_dec_and_test (&group->refcnt)) { -list_del_rcu (&group->group_list); +if (!group->evicted) +list_del_rcu (&group->group_list);mutex_unlock (&fanotify_grp_mutex);synchronize_srcu (&fanotify_grp_srcu_struct);
include/linux/fanotify.h +5 -1
@@ -104,7 +104,8 @@ struct fanotify_so_access {struct fanotify_group {struct list_head group_list;/* list of all groups on the system */ unsigned int group_num;/* the 'name' of the event */ -unsigned int mask;/* mask of events this group cares about */ +unsigned int mask;/* mask of events this group cares about */ +unsigned int evicted;/* 1 if this group has been evicted for misbehaving */ atomic_t refcnt;/* num of processes with a special file open */ /* needed to send notification to userspace */
@@ -124,6 +125,9 @@ struct fanotify_group {unsigned int timeout;/* timeout to wait for access requests in msec */ +unsigned int misbehave_count;/* net misbehaviours. increment on failure, dec on success */ +spinlock_t misbehave_lock;/* protect misbehave_count */ +unsigned int priority;/* order this group should receive msgs. low first */ };
PATCH 20/21 fanotify: allow fastpath entries to survive inode modification
In reply to Eric ParisA listener may wish to choose to ignore inodes even if they have been modified. Setting a fastpath with FAN_SURVIVE_WRITE in the mask will cause the fastpath to not be removed on inode modification. The only way to clear this fastpath will be to either clear all fastpaths for the group, have the inode evicted from core, or to destroy the group and create a new one. This is useful for things like a DB file where the scanner will not able or interested in hearing about events even if the file is modified.
---
fs/notify/fanotify.c | 4 ++-- fs/notify/fanotify.h | 4 +++- fs/notify/fastpath.c | 36 +++++++++++++++++++++++++++++++++++- include/linux/fanotify.h | 3 +++ 4 files changed, 43 insertions(+), 4 deletions(-)
fs/notify/fanotify.c +2 -2
@@ -40,7 +40,7 @@ void fanotify_inode_delete(struct inode *inode)if (likely (list_empty (&fanotify_groups)))return ; -fanotify_fastpath_clear (inode); +fanotify_fastpath_clear (inode,FASTPATH_CLEAR_WRITE ); }EXPORT_SYMBOL_GPL (fanotify_inode_delete);
@@ -62,7 +62,7 @@ int fanotify(struct file *file, unsigned int mask)return 0 ;if (mask &FAN_MODIFY ) -fanotify_fastpath_clear (inode); +fanotify_fastpath_clear (inode,FASTPATH_SAVE_WRITE );/* * SRCU!! the groups list is very very much read only and the path is * very hot (assuming something is using fanotify) Not blocking while
fs/notify/fanotify.h +3 -1
@@ -96,10 +96,12 @@ extern void fanotify_destroy_event_holder(struct fanotify_event_holder *holder);extern __init int fanotify_notification_init (void );extern __init int fanotify_notification_uninit (void ); +#define FASTPATH_CLEAR_WRITE 0 +#define FASTPATH_SAVE_WRITE 1extern void fanotify_fastpath_get (struct fanotify_fastpath_entry *entry);extern void fanotify_fastpath_put (struct fanotify_fastpath_entry *entry);extern int fanotify_is_fastpath (struct file *file,unsigned int mask,struct fanotify_group *group); -extern void fanotify_fastpath_clear (struct inode *inode); +extern void fanotify_fastpath_clear (struct inode *inode,int save_write);extern __init int fanotify_fastpath_init (void );extern __init int fanotify_fastpath_uninit (void );
fs/notify/fastpath.c +35 -1
@@ -118,15 +118,49 @@ void fanotify_fastpath_clear_group(struct fanotify_group *group)mutex_unlock (&group->fastpath_mutex); } -void fanotify_fastpath_clear (struct inode *inode) +void fanotify_fastpath_clear (struct inode *inode,int save_write) {struct fanotify_fastpath_entry *entry;struct fanotify_group *group; +struct fanotify_fastpath_entry *first_moved =NULL ;write_lock (&inode->fastpath_rwlock);while (!list_empty (&inode->fastpath_entries)) { entry = list_first_entry(&inode->fastpath_entries, struct fanotify_fastpath_entry, i_list); +/* +* Basic idea is that we move entries that need to survive +* to the end of the list and stop deleting when we hit the +* first entry we moved to the back. +* +* Be careful though because first_moved could be freed as +* soon as we drop the fastpath_rwlock either because its +* group is being freed or because there was a group wide +* fastpath flush. So no dereferencing first_moved! +*/ +if ((entry->mask &FAN_SURVIVE_WRITE ) && (save_write ==FASTPATH_SAVE_WRITE )) { +/* +* if this is the first fastpath for this inode that is +* surviving we need to remember it so we know when we +* walked the entire list of fastpaths +*/ +if (!first_moved) + first_moved = entry; +/* +* If we saved this entry previously and found it on +* the list a second time that means we walked the +* entire list. So we are finished +*/ +else if (first_moved == entry) +break ; + +/* We need to kick this inode to the end. */ +list_move_tail (&entry->i_list, &inode->fastpath_entries); + +/* done with this fastpath, leave the rwlock held */ +continue ; + } +/* make sure the entry survives until off both lists */ fanotify_fastpath_get (entry);list_del_init (&entry->i_list);
include/linux/fanotify.h +3 -0
@@ -22,6 +22,9 @@#define FAN_ACCESS_EXEC_PERM 0x00000200#define FAN_OPEN_PERM 0x00000400 +/* any fastpath with this set will not be cleared on inode modification */ +#define FAN_SURVIVE_WRITE 0x00001000 +/* FIXME currently Q's have no limit.... */ #define FAN_Q_OVERFLOW 0x80000000/* Event queued overflowed */
PATCH 21/21 fanotify: add Documentation
In reply to Eric ParisAdd an example/test program which exercises all of fanotify's abilities and include all of my writeup about fanotify in Documentation.
---
Documentation/filesystems/fanotify/fanotify.txt | 214 +++++++++++++++++ .../filesystems/fanotify/fanotify_tester.c | 255 ++++++++++++++++++++ 2 files changed, 469 insertions(+), 0 deletions(-) create mode 100644 Documentation/filesystems/fanotify/fanotify.txt create mode 100644 Documentation/filesystems/fanotify/fanotify_tester.c
Documentation/filesystems/fanotify/fanotify.txt new file +214 -0
@@ -0,0 +1,214 @@ +fanotify is a new fscking all notify subsystem. Much as inotify +provides notification of filesystem activity for some registered subset +of inodes fanotify provides notification of filesystem activity for ALL +of the system's S_ISREG() files. fanotify has a smaller number of +notifications than inotify. + +User interface work can be found in net/fanotify. Events are pulled from +the kernel using getsockopt() and responses are sent to the kernel using +setsockopt. An example program which implements most if not all of the +fanotify features can be found in the documentation directory. + +The main implementation is in fs/notify/. My layout is like so + +fanotify.c --- all functions called from the main kernel +group.c --- groups are my implementation of my multiple listeners + you can register different groups to get different + event types. +notification.c --- implementation surrounding the sending of events to a + userspace listener. +fastpath.c --- implementation surrounding the addition of inode + fastpath entries for performance. +access.c --- implementation surrounding the processing of + responses from userspace when the events require + a response. + +GROUPS AND EVENTS: +A "group" registers with fanotify and in doing so indicates what that +group should be called, what types of events it wishes to receive and +in what order it should receive events relative to other groups. An +"event" is simply a notification about some filesystem action. The list +of all fanotify events are read, write, open_for_exec, open, +close_was_not_writable, close_was_writable, open_need_access_decision, +read_for exec_need_access_decision, read_need access_decisions. Any number +of groups may be created for any subset of event types. One group may +register to get reads and writes while another maybe register for opens +and read_need_access_desision. + +Any number of userspace listeners may be active in a single group. Each +group will get ONE copy of a given filesystem event. If there are 10 +listeners in a single group and one fanotify event is generated only ONE +of those listeners will get the event. If more than one group registers +for the same type of event one listener in EACH GROUP will get a copy of +that event. + +It all starts when a listener registers a group. Registering a group +is as done by binding group information to a fanotify socket. Simple +operation is something like. + +addr.name = 123 +addr.priority = 456 +addr.mask = 0x002 + +sock = socket(PF_FANOTIFY); +fd = bind(sock, addr); + +123 is just the name of the group, used so it is unlikely two listeners +will accidentally use the same priority and mask and stumble into each +others events. 456 is the priority (only interesting for blocking/access +events, will describe later) and 0x002 is FAN_WRITE. If one wanted read +and write you would use 0x03 = (FAN_ACCESS | FAN_WRITE). If this is the +first time a process has bound to this address a struct fanotify_group +is allocated and initialized (see fanotify_find_group()). The group is +added to a kernel global list called groups. + +The listener should now call getsockopt(fd, FANOTIFY_GET_EVENT,...). +Since at this point there are no events this will block waiting for an +event. + +Now lets say the original process calls open(). Open is going to happen +exactly as before until it gets to the fsnotify code (this is where both +inotify and dnotify hook into the kernel.) From fsnotify we will call +into the function fanotify() with the mask FAN_OPEN. We will then walk +to global groups list (which is ordered by priority, low first) looking +for any groups which want to receive notification about FAN_OPEN and +let's say we will find the group '123' that was registered above. An +fanotify_event is allocated and any data we want the listener process to +get about the original process is added to the fanotify_event. The +event contains a struct path with the dentry and vfsmount from the open +done by the original process. + +Now we call add_event_to_group_notification() to add the event to the +group->notification_list. This function has a little bit of magic. +Since an event may be needed in multiple groups notification_list I +created a helper structure, a struct fanotify_event_holder. Each entry +in the group->event_list points to a unique event_holder which in turn +points to the ref counted event in question. + +(assuming 2 groups) +group1->notification_list ==> fanotify_event_holder1 ==> single_fanotify_event +group2->notification_list ==> fanotify_event_holder2 ==> single_fanotify_event + +The magic is that since we will always need at least 1 holder I embedded +one fanotify_event_holder inside an fanotity_event. This means that +when removing an event from the group->notification_list we may need to +free the fanotify_event_holder (if it was allocated seperately) or we +may need to just clear it (if it was the embedded holder.) + +After the event is added to the group->notification_list we wake up the +listener processes. The original process never blocked and at this +point and is returning to userspace with the completed open. + +Simultaneously the listener process will now remove the event from the +group->notification_list, see remove_event_from_group_notification(). +Create a new file, fd, and install such in the listener process, see +fanotify_notification_read(). We will put_event (since this group +is finished with it) and will return the getsockopt() call to userspace. + +The listener process will get a structure filled back from teh +getsockopt call which will include an fd and some metadata about that +fd. This includes things like the original files f_flags, the original +processes pid and things like that. + +The listener process must call close() when it is finished with this +new fd. But lets assume the listener, for whatever reason, decides it +doesn't want to hear any more of this type of message for this inode. +That means the listener process needs to "create a fastpath" entry. To +do this the listener process will need to call setsockopt(fd, +FANOTIY_SET_FASTPATH, ...) the struct with the setsockopt will include +things like what kind of events we don't want to hear about and what +file this is associated with. All fastpaths will be cleared on the next +fsnotify write event (happens the same place in the code as mtime +update) + +Inside the kernel (fanotify_fastpath_add()) what happens is that we will +create a new fanotify_fastpath_entry for the group and mask in question +and attach it to the inode. The next time a process opens the inode in +question we will search the global groups list for a group that matches +the mask and we will look at the inode to see if there is a fastpath +entry for this group and mask. If there is an entry no event will be +added to the group->event_list. + +The need for fastpaths (or calling what it really is, an in kernel +cache) has been questioned. I decided to include a little unscientific data +here. On a 32 way machine a make -j 32 took about 9 minutes 12 seconds. +With that same machine running having one group and 32 listeners receiving +every fanotify event that the kernel could send to userspace while the +listeners were responding to accesses as fast as they could (just a very tight get +event, allow loop) it took 95 minutes 12 seconds. Same process with in kernel +fastpaths/cache results took 19 minutes 5 seconds. More reasonable event +requirements and a single listener took 10 minutes 35 sec. So about a 15% +perf hit to do any kind of permission checking to userspace. Anyway, the need +for fastpaths is quite clear. + +Assuming the event was for FAN_OPEN_PERM much of the above is the +same. Biggest difference is that the place from the kernel we will call +into the fanotify code is different (fsnotify is not in a good place to +provide security hooks). If a group is found that wants this event the +event is added to the group->access_list AND to the group->event_list. +The original process is then blocked for a (now fixed 5 second) timeout +waiting for the event to get a non-zero event->response on the +group->access_waitq. + +The listener process will get a notification exactly as above from the +notification file but this time will need to write an answer to the +access file. The answer is again a simple string indicating the cookie +and the response (allow/deny). If a response is received from userspace +the event is removed from the group->access_list and the original +process is woken up to continue, either by looking for the next group or +by returning -EPERM. Userspace may also return FAN_RESETTIMER which +will reset the 5 second timeout. A badly behaving userspace may hang an +open indeffinetly. + +If the original process times out waiting for the listener process to +give a response we currently just allow the security access. + +An interesting part of the code is fastpath cleanup handling. Any time +fanotify gets a FAN_MODIFY event we clear the fastpath entries for the +associated inode. This means our notification and access decisions are +NOT race free but the races are small. This is not perfect security. A +'problomatic' sequence of events would be like such + +process1 calls read on file +listener scans file and finds it safe +process2 writes to file +listener creates a fastpath entry for file + +Maybe I should add an explicit clear fastpath request so userspace can +close that race when it gets the write notification on file (if it so +chooses.) Remember I'm not trying to protect against a rogue process +intelligently and actively attacking the system. I'm trying to stop a +correctly functioning yum from reading a bad rpm that it downloaded. +I'm trying to stop an NFS server from accepting bad files and then +apache serving them out on the net. + +There is the also a race between when a process calls mmap() (at that +point we get an event and scan) and when the process actually faults in +a page which may have been changed by a write from another process. Its +not perfect, but it's damn sure better than what we have now. + +Object lifetime: + +fanotify_group - exists from registration to unregistration. +Unregistration racing with some the associated special file notification +open was a bitch to figure out but eventually I based it on the groups +existence in the global groups list. + +fanotify_event - created inside the main fanotify loop which runs the +events list. An event lasts until both the main loop ends AND the event +is no longer needed by all groups for which it was queued. A refcnt on +the event is taken every time it is added to a group notification/access +list and is dropped when the group has removed the event from the list +and is finished with its contents. + +fanotify_event_holder - allocated when an event is added to a group +notification/access list. destroyed when an event is removed from a +notification/access list. There is the special case of the embedded +holder inside the fanotify_event. The embedded holder is assumed to be +available for use if holder->event_list is empty. + +fanotify_fastpath_entry - created when a process writes to the fastpath +special file and added to the inode list. This entry is destroy in 3 +possible places. If an inode has a modify event we flush them all. If +an inode is eviced from core we flush them all. If a group is +unregistered we flush them all for that group.
Documentation/filesystems/fanotify/fanotify_tester.c new file +255 -0
255 lines not on this page · raw message