Skip to contentSkip to conversation

Rework file::show_fdinfo method to use seq-files engine, v2

PATCH In review Cyrill Gorcunov 3 participants 10 messages Nov 6, 2013

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

cover Rework file::show_fdinfo method to use seq-files engine, v2

Hi, in criu we intensively use additional information provided by /proc/<pid>/fdinfo/<fd> particular to the object opened file represents. The output is printed out by calling the file_operations::show_fdinfo method.

The implementation implies that data is getting printed in one pass and returns error in case if seq buffer is overflowed, which works fine while data fit one memory page. For epoll it's about 73 epoll targets can be printed, for notify system -- significantly less depending on notify type and file handler. This limitation didn't cause any problems in applications we're checkpointing and restoring at moment, but might become a problem in future.

So to resolve the problem I considered two ways

1) Don't return error if seq-buffer overflowed thus, one show() method is called the kernel will notice that buffer is overflowed and double its size returning -EAGAIN. But I fear such interface might be misused (say process creates epoll with a number of target descriptors, then application opens a big number of fdinfo reader causing kernel to allocate big seq-buffer for each reader).

2) Instead of printing fdinfo in one pass rather provide seq-operations pointer in file_operations, and subsystems which need it simply hook own seq-operations here. The main code in fs/proc/fd.c will call for underlied seq-operation methods.

The second way looks more preferrable from my POV, here is the series. Please take a look, thanks. Any comments are highly appreciated!

#1 lore raw

PATCH 1/7 seq-file: Export single_ helpers

The single_ helpers will be used in fdinfo output to eliminate code duplication.

Cc Pavel Emelyanov <[email protected]>
Cc Oleg Nesterov <[email protected]>
Cc Andrey Vagin <[email protected]>
Cc Al Viro <[email protected]>
Cc Alexey Dobriyan <[email protected]>
Cc James Bottomley <[email protected]>
Cc "Aneesh Kumar K.V" <[email protected]>
Cc Alexey Dobriyan <[email protected]>
Cc Matthew Helsley <[email protected]>
Cc "J. Bruce Fields" <[email protected]>
Cc "Aneesh Kumar K.V" <[email protected]>
Cc Andrew Morton <[email protected]>
---
 fs/seq_file.c            |    9 ++++++---
 include/linux/seq_file.h |    3 +++
 2 files changed, 9 insertions(+), 3 deletions(-)
fs/seq_file.c +6 -3
Index: linux-2.6.git/fs/seq_file.c
===================================================================
@@ -565,20 +565,23 @@ int seq_bitmap_list(struct seq_file *m,
 }
 EXPORT_SYMBOL(seq_bitmap_list);
 
-static void *single_start(struct seq_file *p, loff_t *pos)
+void *single_start(struct seq_file *p, loff_t *pos)
 {
 	return NULL + (*pos == 0);
 }
+EXPORT_SYMBOL(single_start);
 
-static void *single_next(struct seq_file *p, void *v, loff_t *pos)
+void *single_next(struct seq_file *p, void *v, loff_t *pos)
 {
 	++*pos;
 	return NULL;
 }
+EXPORT_SYMBOL(single_next);
 
-static void single_stop(struct seq_file *p, void *v)
+void single_stop(struct seq_file *p, void *v)
 {
 }
+EXPORT_SYMBOL(single_stop);
 
 int single_open(struct file *file, int (*show)(struct seq_file *, void *),
 		void *data)
include/linux/seq_file.h +3 -0
Index: linux-2.6.git/include/linux/seq_file.h
===================================================================
@@ -122,6 +122,9 @@ static inline int seq_nodemask_list(stru
 	return seq_bitmap_list(m, mask->bits, MAX_NUMNODES);
 }
 
+void *single_start(struct seq_file *, loff_t *);
+void *single_next(struct seq_file *, void *, loff_t *);
+void single_stop(struct seq_file *, void *);
 int single_open(struct file *, int (*)(struct seq_file *, void *), void *);
 int single_open_size(struct file *, int (*)(struct seq_file *, void *), void *, size_t);
 int single_release(struct inode *, struct file *);
#2 lore raw

Re: [patch 1/7] seq-file: Export single_ helpers

Quoted 2 lines · Cyrill Gorcunov

The single_ helpers will be used in fdinfo output to eliminate code duplication.

There is no need to export symbols just to use them in procfs or eventfd, which are always built in.

#3 lore raw

Re: [patch 1/7] seq-file: Export single_ helpers

Quoted 6 lines · Christoph Hellwig
Quoted 2 lines · Cyrill Gorcunov

The single_ helpers will be used in fdinfo output to eliminate code duplication.

There is no need to export symbols just to use them in procfs or eventfd, which are always built in.

I still need to declare them (they were static), so since single-open already exported I thought I need to export the rest of single_ helper as well.

#4 lore raw

PATCH 2/7 procfs: Introduce sequential fdinfo engine

In reply to Cyrill Gorcunov

At moment the fdinfo operations (ie the output from /proc/$pid/fdinfo/$fd) are generating output in one pass, allocating one page buffer as a maximum. But in case of lots of event polls watchees (75 and more) read procedure will fail because one page is not enough to provide all information associated with watch descriptors.

Instead, provide proc_fdinfo structure where each file hook own seq_operations for fdinfo output and the seq-file engine will be able to traverse over the all records and provide the reader complete information increasing buffer size if needed.

Once all fdinfo callers are converted (addressed in next patches) the former @show_fdinfo member will be removed.

Note the patch doesn't change current output it's rather preparing the ground for next patches.

Cc Pavel Emelyanov <[email protected]>
Cc Oleg Nesterov <[email protected]>
Cc Andrey Vagin <[email protected]>
Cc Al Viro <[email protected]>
Cc Alexey Dobriyan <[email protected]>
Cc James Bottomley <[email protected]>
Cc "Aneesh Kumar K.V" <[email protected]>
Cc Alexey Dobriyan <[email protected]>
Cc Matthew Helsley <[email protected]>
Cc "J. Bruce Fields" <[email protected]>
Cc "Aneesh Kumar K.V" <[email protected]>
Cc Alexey Dobriyan <[email protected]>
Cc Andrew Morton <[email protected]>
---
 fs/proc/fd.c       |  141 ++++++++++++++++++++++++++++++++++++++++-------------
 include/linux/fs.h |    1 
 2 files changed, 109 insertions(+), 33 deletions(-)
fs/proc/fd.c +108 -33
Index: linux-2.6.git/fs/proc/fd.c
===================================================================
@@ -14,60 +14,135 @@
 #include "internal.h"
 #include "fd.h"
 
+struct proc_fdinfo {
+	struct seq_file		seq;	/* Make sure it's first */
+	struct seq_operations	*op;
+	struct file		*f_file;
+	unsigned int		f_flags;
+};
+
+#define seq_info(m)	(container_of(m, struct proc_fdinfo, seq))
+
+static void *seq_start(struct seq_file *m, loff_t *pos)
+{
+	struct proc_fdinfo *info = seq_info(m);
+
+	if (*pos == 0)
+		seq_printf(m, "pos:\t%lli\nflags:\t0%o\n",
+			   (long long)info->f_file->f_pos, info->f_flags);
+
+	return info->op ? info->op->start(m, pos) : NULL + (*pos == 0);
+}
+
+static void seq_stop(struct seq_file *m, void *v)
+{
+	struct proc_fdinfo *info = seq_info(m);
+
+	if (info->op)
+		info->op->stop(m, v);
+}
+
+static void *seq_next(struct seq_file *m, void *p, loff_t *pos)
+{
+	struct proc_fdinfo *info = seq_info(m);
+
+	if (info->op)
+		return info->op->next(m, p, pos);
+
+	++*pos;
+	return NULL;
+}
+
 static int seq_show(struct seq_file *m, void *v)
 {
+	struct proc_fdinfo *info = seq_info(m);
+
+	if (info->op)
+		return info->op->show(m, v);
+
+	/* Legacy interface */
+	if (info->f_file->f_op->show_fdinfo)
+		return info->f_file->f_op->show_fdinfo(m, info->f_file);
+
+	return 0;
+}
+
+static const struct seq_operations fdinfo_seq_ops = {
+	.start	= seq_start,
+	.stop	= seq_stop,
+	.next	= seq_next,
+	.show	= seq_show,
+};
+
+static int seq_fdinfo_open(struct inode *inode, struct file *file)
+{
 	struct files_struct *files = NULL;
-	int f_flags = 0, ret = -ENOENT;
-	struct file *file = NULL;
+	struct proc_fdinfo *info;
 	struct task_struct *task;
+	int ret;
+
+	info = kzalloc(sizeof(*info), GFP_KERNEL);
+	if (!info)
+		return -ENOMEM;
+	file->private_data = &info->seq;
+
+	ret = seq_open(file, &fdinfo_seq_ops);
+	if (!ret) {
+		ret = -ENOENT;
 
-	task = get_proc_task(m->private);
-	if (!task)
-		return -ENOENT;
-
-	files = get_files_struct(task);
-	put_task_struct(task);
-
-	if (files) {
-		int fd = proc_fd(m->private);
-
-		spin_lock(&files->file_lock);
-		file = fcheck_files(files, fd);
-		if (file) {
-			struct fdtable *fdt = files_fdtable(files);
-
-			f_flags = file->f_flags;
-			if (close_on_exec(fd, fdt))
-				f_flags |= O_CLOEXEC;
+		task = get_proc_task(inode);
+		if (task) {
+			files = get_files_struct(task);
+			put_task_struct(task);
+		}
 
-			get_file(file);
-			ret = 0;
+		if (files) {
+			struct seq_file *m = file->private_data;
+			int fd = proc_fd(inode);
+
+			spin_lock(&files->file_lock);
+			info->f_file = fcheck_files(files, fd);
+			if (info->f_file) {
+				struct fdtable *fdt = files_fdtable(files);
+
+				info->f_flags = info->f_file->f_flags & ~O_CLOEXEC;
+				if (close_on_exec(fd, fdt))
+					info->f_flags |= O_CLOEXEC;
+				info->op = info->f_file->f_op->fdinfo_op;
+				m->private = info->f_file;
+
+				get_file(info->f_file);
+				ret = 0;
+			}
+			spin_unlock(&files->file_lock);
+			put_files_struct(files);
 		}
-		spin_unlock(&files->file_lock);
-		put_files_struct(files);
 	}
 
-	if (!ret) {
-                seq_printf(m, "pos:\t%lli\nflags:\t0%o\n",
-			   (long long)file->f_pos, f_flags);
-		if (file->f_op->show_fdinfo)
-			ret = file->f_op->show_fdinfo(m, file);
-		fput(file);
+	if (ret) {
+		if (info->f_file)
+			fput(info->f_file);
+		file->private_data = NULL;
+		kfree(info);
 	}
 
 	return ret;
 }
 
-static int seq_fdinfo_open(struct inode *inode, struct file *file)
+static int seq_fdinfo_release(struct inode *inode, struct file *file)
 {
-	return single_open(file, seq_show, inode);
+	struct seq_file *m = file->private_data;
+	struct proc_fdinfo *info = seq_info(m);
+
+	fput(info->f_file);
+	return seq_release(inode, file);
 }
 
 static const struct file_operations proc_fdinfo_file_operations = {
 	.open		= seq_fdinfo_open,
 	.read		= seq_read,
 	.llseek		= seq_lseek,
-	.release	= single_release,
+	.release	= seq_fdinfo_release,
 };
 
 static int tid_fd_revalidate(struct dentry *dentry, unsigned int flags)
include/linux/fs.h +1 -0
Index: linux-2.6.git/include/linux/fs.h
===================================================================
@@ -1552,6 +1552,7 @@ struct file_operations {
 	long (*fallocate)(struct file *file, int mode, loff_t offset,
 			  loff_t len);
 	int (*show_fdinfo)(struct seq_file *m, struct file *f);
+	struct seq_operations *fdinfo_op;
 };
 
 struct inode_operations {
#5 lore raw

PATCH 3/7 epoll: Use sequential fdinfo engine

In reply to Cyrill Gorcunov
Cc Pavel Emelyanov <[email protected]>
Cc Oleg Nesterov <[email protected]>
Cc Andrey Vagin <[email protected]>
Cc Al Viro <[email protected]>
Cc Alexey Dobriyan <[email protected]>
Cc James Bottomley <[email protected]>
Cc "Aneesh Kumar K.V" <[email protected]>
Cc Alexey Dobriyan <[email protected]>
Cc Matthew Helsley <[email protected]>
Cc "J. Bruce Fields" <[email protected]>
Cc "Aneesh Kumar K.V" <[email protected]>
Cc Andrew Morton <[email protected]>
---
 fs/eventpoll.c |   46 ++++++++++++++++++++++++++++++++++------------
 1 file changed, 34 insertions(+), 12 deletions(-)
fs/eventpoll.c +34 -12
Index: linux-2.6.git/fs/eventpoll.c
===================================================================
@@ -833,32 +833,54 @@ static unsigned int ep_eventpoll_poll(st
 }
 
 #ifdef CONFIG_PROC_FS
-static int ep_show_fdinfo(struct seq_file *m, struct file *f)
+static void *seq_start(struct seq_file *m, loff_t *pos)
 {
-	struct eventpoll *ep = f->private_data;
+	struct eventpoll *ep = ((struct file *)m->private)->private_data;
 	struct rb_node *rbp;
-	int ret = 0;
+	loff_t num = *pos;
 
 	mutex_lock(&ep->mtx);
 	for (rbp = rb_first(&ep->rbr); rbp; rbp = rb_next(rbp)) {
-		struct epitem *epi = rb_entry(rbp, struct epitem, rbn);
-
-		ret = seq_printf(m, "tfd: %8d events: %8x data: %16llx\n",
-				 epi->ffd.fd, epi->event.events,
-				 (long long)epi->event.data);
-		if (ret)
-			break;
+		if (num-- == 0)
+			return rbp;
 	}
+	return NULL;
+}
+
+static void seq_stop(struct seq_file *m, void *v)
+{
+	struct eventpoll *ep = ((struct file *)m->private)->private_data;
 	mutex_unlock(&ep->mtx);
+}
 
-	return ret;
+static void *seq_next(struct seq_file *m, void *p, loff_t *pos)
+{
+	++*pos;
+	return (void *)rb_next(p);
 }
+
+static int seq_show(struct seq_file *m, void *v)
+{
+	struct epitem *epi = rb_entry(v, struct epitem, rbn);
+
+	seq_printf(m, "tfd: %8d events: %8x data: %16llx\n",
+		   epi->ffd.fd, epi->event.events,
+		   (long long)epi->event.data);
+	return 0;
+}
+
+static struct seq_operations ep_fdinfo_op = {
+	.start		= seq_start,
+	.next		= seq_next,
+	.stop		= seq_stop,
+	.show		= seq_show,
+};
 #endif
 
 /* File callbacks that implement the eventpoll file behaviour */
 static const struct file_operations eventpoll_fops = {
 #ifdef CONFIG_PROC_FS
-	.show_fdinfo	= ep_show_fdinfo,
+	.fdinfo_op	= &ep_fdinfo_op,
 #endif
 	.release	= ep_eventpoll_release,
 	.poll		= ep_eventpoll_poll,
#6 lore raw

PATCH 4/7 eventfd: Use sequential fdinfo engine

In reply to Cyrill Gorcunov
Cc Pavel Emelyanov <[email protected]>
Cc Oleg Nesterov <[email protected]>
Cc Andrey Vagin <[email protected]>
Cc Al Viro <[email protected]>
Cc Alexey Dobriyan <[email protected]>
Cc James Bottomley <[email protected]>
Cc "Aneesh Kumar K.V" <[email protected]>
Cc Alexey Dobriyan <[email protected]>
Cc Matthew Helsley <[email protected]>
Cc "J. Bruce Fields" <[email protected]>
Cc "Aneesh Kumar K.V" <[email protected]>
Cc Andrew Morton <[email protected]>
---
 fs/eventfd.c |   19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)
fs/eventfd.c +12 -7
Index: linux-2.6.git/fs/eventfd.c
===================================================================
@@ -287,23 +287,28 @@ static ssize_t eventfd_write(struct file
 }
 
 #ifdef CONFIG_PROC_FS
-static int eventfd_show_fdinfo(struct seq_file *m, struct file *f)
+static int seq_show(struct seq_file *m, void *v)
 {
-	struct eventfd_ctx *ctx = f->private_data;
-	int ret;
+	struct eventfd_ctx *ctx = ((struct file *)m->private)->private_data;
 
 	spin_lock_irq(&ctx->wqh.lock);
-	ret = seq_printf(m, "eventfd-count: %16llx\n",
-			 (unsigned long long)ctx->count);
+	seq_printf(m, "eventfd-count: %16llx\n", (unsigned long long)ctx->count);
 	spin_unlock_irq(&ctx->wqh.lock);
 
-	return ret;
+	return 0;
 }
+
+static struct seq_operations eventfd_fdinfo_op = {
+	.start		= single_start,
+	.next		= single_next,
+	.stop		= single_stop,
+	.show		= seq_show,
+};
 #endif
 
 static const struct file_operations eventfd_fops = {
 #ifdef CONFIG_PROC_FS
-	.show_fdinfo	= eventfd_show_fdinfo,
+	.fdinfo_op	= &eventfd_fdinfo_op,
 #endif
 	.release	= eventfd_release,
 	.poll		= eventfd_poll,
#7 lore raw

PATCH 5/7 signalfd: Use sequential fdinfo engine

In reply to Cyrill Gorcunov
Cc Pavel Emelyanov <[email protected]>
Cc Oleg Nesterov <[email protected]>
Cc Andrey Vagin <[email protected]>
Cc Al Viro <[email protected]>
Cc Alexey Dobriyan <[email protected]>
Cc James Bottomley <[email protected]>
Cc "Aneesh Kumar K.V" <[email protected]>
Cc Alexey Dobriyan <[email protected]>
Cc Matthew Helsley <[email protected]>
Cc "J. Bruce Fields" <[email protected]>
Cc "Aneesh Kumar K.V" <[email protected]>
Cc Andrew Morton <[email protected]>
---
 fs/signalfd.c |   13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)
fs/signalfd.c +10 -3
Index: linux-2.6.git/fs/signalfd.c
===================================================================
@@ -230,9 +230,9 @@ static ssize_t signalfd_read(struct file
 }
 
 #ifdef CONFIG_PROC_FS
-static int signalfd_show_fdinfo(struct seq_file *m, struct file *f)
+static int seq_show(struct seq_file *m, void *v)
 {
-	struct signalfd_ctx *ctx = f->private_data;
+	struct signalfd_ctx *ctx = ((struct file *)m->private)->private_data;
 	sigset_t sigmask;
 
 	sigmask = ctx->sigmask;
@@ -241,11 +241,18 @@ static int signalfd_show_fdinfo(struct s
 
 	return 0;
 }
+
+static struct seq_operations signalfd_fdinfo_op = {
+	.start		= single_start,
+	.next		= single_next,
+	.stop		= single_stop,
+	.show		= seq_show,
+};
 #endif
 
 static const struct file_operations signalfd_fops = {
 #ifdef CONFIG_PROC_FS
-	.show_fdinfo	= signalfd_show_fdinfo,
+	.fdinfo_op	= &signalfd_fdinfo_op,
 #endif
 	.release	= signalfd_release,
 	.poll		= signalfd_poll,
#8 lore raw

PATCH 6/7 fsnotify: Use sequential fdinfo engine

In reply to Cyrill Gorcunov
Cc Pavel Emelyanov <[email protected]>
Cc Oleg Nesterov <[email protected]>
Cc Andrey Vagin <[email protected]>
Cc Al Viro <[email protected]>
Cc Alexey Dobriyan <[email protected]>
Cc James Bottomley <[email protected]>
Cc "Aneesh Kumar K.V" <[email protected]>
Cc Alexey Dobriyan <[email protected]>
Cc Matthew Helsley <[email protected]>
Cc "J. Bruce Fields" <[email protected]>
Cc "Aneesh Kumar K.V" <[email protected]>
Cc Andrew Morton <[email protected]>
---
 fs/notify/fanotify/fanotify_user.c |    4 
 fs/notify/fdinfo.c                 |  167 +++++++++++++++++++++++--------------
 fs/notify/fdinfo.h                 |   12 --
 fs/notify/inotify/inotify_user.c   |    4 
 4 files changed, 116 insertions(+), 71 deletions(-)
fs/notify/fanotify/fanotify_user.c +3 -1
Index: linux-2.6.git/fs/notify/fanotify/fanotify_user.c
===================================================================
@@ -432,7 +432,9 @@ static long fanotify_ioctl(struct file *
 }
 
 static const struct file_operations fanotify_fops = {
-	.show_fdinfo	= fanotify_show_fdinfo,
+#ifdef CONFIG_PROC_FS
+	.fdinfo_op	= &fanotify_fdinfo_op,
+#endif
 	.poll		= fanotify_poll,
 	.read		= fanotify_read,
 	.write		= fanotify_write,
fs/notify/fdinfo.c +107 -60

242 lines not on this page · raw message

fs/notify/fdinfo.h +3 -9
Index: linux-2.6.git/fs/notify/fdinfo.h
===================================================================
@@ -4,24 +4,18 @@
 #include <linux/errno.h>
 #include <linux/proc_fs.h>
 
-struct seq_file;
-struct file;
+struct seq_operations;
 
 #ifdef CONFIG_PROC_FS
 
 #ifdef CONFIG_INOTIFY_USER
-extern int inotify_show_fdinfo(struct seq_file *m, struct file *f);
+extern struct seq_operations inotify_fdinfo_op;
 #endif
 
 #ifdef CONFIG_FANOTIFY
-extern int fanotify_show_fdinfo(struct seq_file *m, struct file *f);
+extern struct seq_operations fanotify_fdinfo_op;
 #endif
 
-#else /* CONFIG_PROC_FS */
-
-#define inotify_show_fdinfo	NULL
-#define fanotify_show_fdinfo	NULL
-
 #endif /* CONFIG_PROC_FS */
 
 #endif /* __FSNOTIFY_FDINFO_H__ */
fs/notify/inotify/inotify_user.c +3 -1
Index: linux-2.6.git/fs/notify/inotify/inotify_user.c
===================================================================
@@ -327,7 +327,9 @@ static long inotify_ioctl(struct file *f
 }
 
 static const struct file_operations inotify_fops = {
-	.show_fdinfo	= inotify_show_fdinfo,
+#ifdef CONFIG_PROC_FS
+	.fdinfo_op	= &inotify_fdinfo_op,
+#endif
 	.poll		= inotify_poll,
 	.read		= inotify_read,
 	.fasync		= fsnotify_fasync,
#9 lore raw

PATCH 7/7 procfs: Drop legacy show_fdinfo file operation

In reply to Cyrill Gorcunov

All former users of show_fdinfo are converted to sequential output engine thus we don't need it anymore.

Cc Pavel Emelyanov <[email protected]>
Cc Oleg Nesterov <[email protected]>
Cc Andrey Vagin <[email protected]>
Cc Al Viro <[email protected]>
Cc Alexey Dobriyan <[email protected]>
Cc James Bottomley <[email protected]>
Cc "Aneesh Kumar K.V" <[email protected]>
Cc Alexey Dobriyan <[email protected]>
Cc Matthew Helsley <[email protected]>
Cc "J. Bruce Fields" <[email protected]>
Cc "Aneesh Kumar K.V" <[email protected]>
Cc Andrew Morton <[email protected]>
---
 fs/proc/fd.c       |    5 -----
 include/linux/fs.h |    1 -
 2 files changed, 6 deletions(-)
fs/proc/fd.c +0 -5
Index: linux-2.6.git/fs/proc/fd.c
===================================================================
@@ -59,11 +59,6 @@ static int seq_show(struct seq_file *m,
 
 	if (info->op)
 		return info->op->show(m, v);
-
-	/* Legacy interface */
-	if (info->f_file->f_op->show_fdinfo)
-		return info->f_file->f_op->show_fdinfo(m, info->f_file);
-
 	return 0;
 }
 
include/linux/fs.h +0 -1
Index: linux-2.6.git/include/linux/fs.h
===================================================================
@@ -1551,7 +1551,6 @@ struct file_operations {
 	int (*setlease)(struct file *, long, struct file_lock **);
 	long (*fallocate)(struct file *file, int mode, loff_t offset,
 			  loff_t len);
-	int (*show_fdinfo)(struct seq_file *m, struct file *f);
 	struct seq_operations *fdinfo_op;
 };