Skip to contentSkip to conversation

Some cleanup in the pipe code

PATCH In review Andi Kleen 6 participants 17 messages Aug 14, 2006 – Aug 18, 2006 1 message not in the archive

Subsystems (4)
lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): 20060814 [email protected]

Andi Kleen

author
#1 lore raw

PATCH 1/3 Some cleanup in the pipe code

Split the big and hard to read do_pipe function into smaller pieces.

This creates new create_write_pipe/free_write_pipe/create_read_pipe functions. These functions are made global so that they can be used by other parts of the kernel.

The resulting code is more generic and easier to read and has cleaner error handling and less gotos.

include/linux/fs.h +3 -0
Index: linux/include/linux/fs.h
===================================================================
@@ -1563,6 +1563,9 @@ static inline void allow_write_access(st
 		atomic_inc(&file->f_dentry->d_inode->i_writecount);
 }
 extern int do_pipe(int *);
+extern struct file *create_read_pipe(struct file *f);
+extern struct file *create_write_pipe(void);
+extern void free_write_pipe(struct file *);
 
 extern int open_namei(int dfd, const char *, int, int, struct nameidata *);
 extern int may_open(struct nameidata *, int, int);
fs/pipe.c +90 -59

177 lines not on this page · raw message

author
#2 lore raw

PATCH 2/3 Create call_usermodehelper_pipe()

In reply to a message not in the archive

A new member in the ever growing family of call_usermode* functions is born. The new call_usermodehelper_pipe() function allows to pipe data to the stdin of the called user mode progam and behaves otherwise like the normal call_usermodehelp() (except that it always waits for the child to finish)

To be used in a follow up patch.

kernel/kmod.c +54 -1
Index: linux/kernel/kmod.c
===================================================================
@@ -122,6 +122,7 @@ struct subprocess_info {
 	struct key *ring;
 	int wait;
 	int retval;
+	struct file *stdin;
 };
 
 /*
@@ -145,12 +146,26 @@ static int ____call_usermodehelper(void 
 
 	key_put(old_session);
 
+	/* Install input pipe when needed */
+	if (sub_info->stdin) {
+		struct files_struct *f = current->files;
+		struct fdtable *fdt;
+		/* no races because files should be private here */
+		sys_close(0);
+		fd_install(0, sub_info->stdin);
+		spin_lock(&f->file_lock);
+		fdt = files_fdtable(f);
+		FD_SET(0, fdt->open_fds);
+		FD_CLR(0, fdt->close_on_exec);
+		spin_unlock(&f->file_lock);
+	}
+
 	/* We can run anywhere, unlike our parent keventd(). */
 	set_cpus_allowed(current, CPU_MASK_ALL);
 
 	retval = -EPERM;
 	if (current->fs->root)
-		retval = execve(sub_info->path, sub_info->argv,sub_info->envp);
+		retval = execve(sub_info->path, sub_info->argv, sub_info->envp);
 
 	/* Exec failed? */
 	sub_info->retval = retval;
@@ -257,6 +272,44 @@ int call_usermodehelper_keys(char *path,
 }
 EXPORT_SYMBOL(call_usermodehelper_keys);
 
+int call_usermodehelper_pipe(char *path, char **argv, char **envp,
+			     struct file **filp)
+{
+	DECLARE_COMPLETION(done);
+	struct subprocess_info sub_info = {
+		.complete	= &done,
+		.path		= path,
+		.argv		= argv,
+		.envp		= envp,
+		.retval		= 0,
+	};
+	struct file *f;
+	DECLARE_WORK(work, __call_usermodehelper, &sub_info);
+
+	if (!khelper_wq)
+		return -EBUSY;
+
+	if (path[0] == '\0')
+		return 0;
+
+	f = create_write_pipe();
+	if (!f)
+		return -ENOMEM;
+	*filp = f;
+
+	f = create_read_pipe(f);
+	if (!f) {
+		free_write_pipe(*filp);
+		return -ENOMEM;
+	}
+	sub_info.stdin = f;
+
+	queue_work(khelper_wq, &work);
+	wait_for_completion(&done);
+	return sub_info.retval;
+}
+EXPORT_SYMBOL(call_usermodehelper_pipe);
+
 void __init usermodehelper_init(void)
 {
 	khelper_wq = create_singlethread_workqueue("khelper");
include/linux/kmod.h +4 -0
Index: linux/include/linux/kmod.h
===================================================================
@@ -47,4 +47,8 @@ call_usermodehelper(char *path, char **a
 
 extern void usermodehelper_init(void);
 
+struct file;
+extern int call_usermodehelper_pipe(char *path, char *argv[], char *envp[],
+				    struct file **filp);
+
 #endif /* __LINUX_KMOD_H__ */
#3 lore raw

Re: [PATCH] [2/3] Create call_usermodehelper_pipe()

kernel/kmod.c static int ____call_usermodehelper(void · line 161

Quoted 13 lines · On Mon, 14 Aug 2006 13:27:31 +0200 (CEST) Andi Kleen <[email protected]> wrote:
kernel/kmod.c +13 -0
+	/* Install input pipe when needed */
+	if (sub_info->stdin) {
+		struct files_struct *f = current->files;
+		struct fdtable *fdt;
+		/* no races because files should be private here */
+		sys_close(0);
+		fd_install(0, sub_info->stdin);
+		spin_lock(&f->file_lock);
+		fdt = files_fdtable(f);
+		FD_SET(0, fdt->open_fds);
+		FD_CLR(0, fdt->close_on_exec);
+		spin_unlock(&f->file_lock);
+	}

This is all going to be run by kernel threads, and all kernel threads share current->files=&init_files.

So I suspect that if two coredumps happen at the same time bad things will happen. Like a BUG() in fd_install()?

#4 lore raw

Re: [PATCH] [2/3] Create call_usermodehelper_pipe()

Quoted 19 lines · On Tue, 15 Aug 2006 14:22:25 -0700 Andrew Morton <[email protected]> wrote:

kernel/kmod.c static int ____call_usermodehelper(void · line 161

Quoted 13 lines · On Mon, 14 Aug 2006 13:27:31 +0200 (CEST) Andi Kleen <[email protected]> wrote:
kernel/kmod.c +13 -0
+	/* Install input pipe when needed */
+	if (sub_info->stdin) {
+		struct files_struct *f = current->files;
+		struct fdtable *fdt;
+		/* no races because files should be private here */
+		sys_close(0);
+		fd_install(0, sub_info->stdin);
+		spin_lock(&f->file_lock);
+		fdt = files_fdtable(f);
+		FD_SET(0, fdt->open_fds);
+		FD_CLR(0, fdt->close_on_exec);
+		spin_unlock(&f->file_lock);
+	}

This is all going to be run by kernel threads, and all kernel threads share current->files=&init_files.

hm, that's wrong, isn't it - we're not using CLONE_FILES...

So what we have is a copy of init_files. So the sys_close(0) shouldn't be needed?

#5 lore raw

Re: [PATCH] [2/3] Create call_usermodehelper_pipe()

current->files=&init_files.

Quoted 2 lines

hm, that's wrong, isn't it - we're not using CLONE_FILES...

Yes, I thought I got that right but you shocked me quickly :)

Quoted 2 lines

So what we have is a copy of init_files. So the sys_close(0) shouldn't be needed?

Probably not. It was just paranoia I guess.

-Andi

author
#6 lore raw

PATCH 3/3 Support piping into commands in /proc/sys/kernel/core_pattern

In reply to a message not in the archive

Using the infrastructure created in previous patches implement support to pipe core dumps into programs.

This is done by overloading the existing core_pattern sysctl with a new syntax:

|program

When the first character of the pattern is a '|' the kernel will instead threat the rest of the pattern as a command to run. The core dump will be written to the standard input of that program instead of to a file.

This is useful for having automatic core dump analysis without filling up disks. The program can do some simple analysis and save only a summary of the core dump.

The core dump proces will run with the privileges and in the name space of the process that caused the core dump.

I also increased the core pattern size to 128 bytes so that longer command lines fit.

Most of the changes comes from allowing core dumps without seeks. They are fairly straight forward though.

One small incompatibility is that if someone had a core pattern previously that started with '|' they will get suddenly new behaviour. I think that's unlikely to be a real problem though.

fs/exec.c +18 -5
Index: linux/fs/exec.c
===================================================================
@@ -58,7 +58,7 @@
 #endif
 
 int core_uses_pid;
-char core_pattern[65] = "core";
+char core_pattern[128] = "core";
 int suid_dumpable = 0;
 
 EXPORT_SYMBOL(suid_dumpable);
@@ -1475,6 +1475,7 @@ int do_coredump(long signr, int exit_cod
 	int retval = 0;
 	int fsuid = current->fsuid;
 	int flag = 0;
+	int ispipe = 0;
 
 	binfmt = current->binfmt;
 	if (!binfmt || !binfmt->core_dump)
@@ -1516,22 +1517,34 @@ int do_coredump(long signr, int exit_cod
  	lock_kernel();
 	format_corename(corename, core_pattern, signr);
 	unlock_kernel();
-	file = filp_open(corename, O_CREAT | 2 | O_NOFOLLOW | O_LARGEFILE | flag, 0600);
+ 	if (corename[0] == '|') {
+		/* SIGPIPE can happen, but it's just never processed */
+ 		if(call_usermodehelper_pipe(corename+1, NULL, NULL, &file)) {
+ 			printk(KERN_INFO "Core dump to %s pipe failed\n",
+			       corename);
+ 			goto fail_unlock;
+ 		}
+		ispipe = 1;
+ 	} else
+ 		file = filp_open(corename,
+				 O_CREAT | 2 | O_NOFOLLOW | O_LARGEFILE, 0600);
 	if (IS_ERR(file))
 		goto fail_unlock;
 	inode = file->f_dentry->d_inode;
 	if (inode->i_nlink > 1)
 		goto close_fail;	/* multiple links - don't dump */
-	if (d_unhashed(file->f_dentry))
+	if (!ispipe && d_unhashed(file->f_dentry))
 		goto close_fail;
 
-	if (!S_ISREG(inode->i_mode))
+	/* AK: actually i see no reason to not allow this for named pipes etc.,
+	   but keep the previous behaviour for now. */
+	if (!ispipe && !S_ISREG(inode->i_mode))
 		goto close_fail;
 	if (!file->f_op)
 		goto close_fail;
 	if (!file->f_op->write)
 		goto close_fail;
-	if (do_truncate(file->f_dentry, 0, 0, file) != 0)
+	if (!ispipe && do_truncate(file->f_dentry, 0, 0, file) != 0)
 		goto close_fail;
 
 	retval = binfmt->core_dump(signr, regs, file);
fs/binfmt_elf.c +42 -32

144 lines not on this page · raw message

kernel/sysctl.c +1 -1
Index: linux/kernel/sysctl.c
===================================================================
@@ -293,7 +293,7 @@ static ctl_table kern_table[] = {
 		.ctl_name	= KERN_CORE_PATTERN,
 		.procname	= "core_pattern",
 		.data		= core_pattern,
-		.maxlen		= 64,
+		.maxlen		= 128,
 		.mode		= 0644,
 		.proc_handler	= &proc_dostring,
 		.strategy	= &sysctl_string,
#7 lore raw

Re: [PATCH] [3/3] Support piping into commands in /proc/sys/kernel/core_pattern

Quoted 2 lines · On Mon, 14 Aug 2006 13:27:32 +0200 (CEST) Andi Kleen <[email protected]> wrote:

The core dump proces will run with the privileges and in the name space of the process that caused the core dump.

Don't think so. __call_usermodehelper() is executed by the khelper kernel thread.

author
#8 lore raw

Re: [PATCH] [3/3] Support piping into commands in /proc/sys/kernel/core_pattern

Quoted 7 lines · Andrew Morton
Quoted 2 lines · On Mon, 14 Aug 2006 13:27:32 +0200 (CEST) Andi Kleen <[email protected]> wrote:

The core dump proces will run with the privileges and in the name space of the process that caused the core dump.

Don't think so. __call_usermodehelper() is executed by the khelper kernel thread.

Yes, you're right the comment is wrong. It does run as root instead with global name space. I needed that because otherwise it needed a global writable directory for the core dumps.

Feel free to change it in your version.

That happens when you write the description months later than the code ...

-Andi

#9 lore raw

Re: [PATCH] [3/3] Support piping into commands in /proc/sys/kernel/core_pattern

In reply to Andi Kleen
Quoted 8 lines · Andi Kleen

Using the infrastructure created in previous patches implement support to pipe core dumps into programs.

This is done by overloading the existing core_pattern sysctl with a new syntax:

|program

Very nice, do you happen to have a program that can accept this kind of input for crash dumps? I'm guessing that the embedded people will really want this functionality.

thanks,

greg k-h

#10 lore raw

Re: [PATCH] [3/3] Support piping into commands in /proc/sys/kernel/core_pattern

Quoted 5 lines · On Wed, 16 Aug 2006 01:43:54 -0700 Greg KH <[email protected]> wrote:
Quoted 4 lines · Andi Kleen

Using the infrastructure created in previous patches implement support to pipe core dumps into programs.

Quoted 8 lines
Quoted 4 lines

This is done by overloading the existing core_pattern sysctl with a new syntax:

|program

Very nice, do you happen to have a program that can accept this kind of input for crash dumps? I'm guessing that the embedded people will really want this functionality.

I had a cheesy demo/prototype. Basically it wrote the dump to a file again, ran gdb on it to get a backtrace and wrote the summary to a shared directory. Then there was a simple CGI script to generate a "top 10" crashes HTML listing.

Unfortunately this still had the disadvantage to needing full disk space for a dump except for deleting it afterwards (in fact it was worse because over the pipe holes didn't work so if you have a holey address map it would require more space).

Fortunately gdb seems to be happy to handle /proc/pid/fd/xxx input pipes as cores (at least it worked with zsh's =(cat core) syntax), so it would be likely possible to do it without temporary space with a simple wrapper that calls it in the right way. I ran out of time before doing that though.

The demo prototype scripts weren't very good. If there is really interest I can dig them out (they are currently on a laptop disk on the desk with the laptop itself being in service), but I would recommend to rewrite them for any serious application of this and fix the disk space problem.

Also to be really useful it should probably find a way to automatically fetch the debuginfos (I cheated and just installed them in advance)

If nobody else does it I can probably do the rewrite myself again at some point.

My hope at some point was that desktops would support it in their builtin crash reporters, but at least the KDE people I talked too seemed to be happy with their user space only solution.

-Andi

#11 lore raw

Re: [PATCH] [3/3] Support piping into commands in /proc/sys/kernel/core_pattern

Quoted 24 lines · On Wed, 16 Aug 2006 11:18:01 +0200 Andi Kleen <[email protected]> wrote:
Quoted 3 lines

Very nice, do you happen to have a program that can accept this kind of input for crash dumps? I'm guessing that the embedded people will really want this functionality.

I had a cheesy demo/prototype. Basically it wrote the dump to a file again, ran gdb on it to get a backtrace and wrote the summary to a shared directory. Then there was a simple CGI script to generate a "top 10" crashes HTML listing.

Unfortunately this still had the disadvantage to needing full disk space for a dump except for deleting it afterwards (in fact it was worse because over the pipe holes didn't work so if you have a holey address map it would require more space).

Fortunately gdb seems to be happy to handle /proc/pid/fd/xxx input pipes as cores (at least it worked with zsh's =(cat core) syntax), so it would be likely possible to do it without temporary space with a simple wrapper that calls it in the right way. I ran out of time before doing that though.

The demo prototype scripts weren't very good. If there is really interest I can dig them out (they are currently on a laptop disk on the desk with the laptop itself being in service), but I would recommend to rewrite them for any serious application of this and fix the disk space problem.

Quoted 8 lines

Also to be really useful it should probably find a way to automatically fetch the debuginfos (I cheated and just installed them in advance)

If nobody else does it I can probably do the rewrite myself again at some point.

My hope at some point was that desktops would support it in their builtin crash reporters, but at least the KDE people I talked too seemed to be happy with their user space only solution.

It doens't sounds like there's particularly strong userspace "pull" for this feature?

#12 lore raw

Re: [PATCH] [3/3] Support piping into commands in /proc/sys/kernel/core_pattern

Quoted 2 lines

It doens't sounds like there's particularly strong userspace "pull" for this feature?

Several people from the embedded area wrote me privately
it would be useful for them. Also I think once it's in the main kernel there will be more incentive for user space to use it and I'm optimistic it will get some adoption (ok I guess I should write some better documentation, but there was no obvious place for it)

-Andi

#13 lore raw

Re: [PATCH] [3/3] Support piping into commands in /proc/sys/kernel/core_pattern

Ar Iau, 2006-08-17 am 11:46 +0200, ysgrifennodd Andi Kleen:

Quoted 5 lines

Several people from the embedded area wrote me privately
it would be useful for them. Also I think once it's in the main kernel there will be more incentive for user space to use it and I'm optimistic it will get some adoption (ok I guess I should write some better documentation, but there was no obvious place for it)

I don't believe that piping as such as neccessarily the right model, but the ability to intercept and processes core dumps from user space is asked for by many enterprise users as well. They want to know about, capture, analyse and process core dumps, often centrally and in automated form.

Alan

#14 lore raw

Re: [PATCH] [3/3] Support piping into commands in /proc/sys/kernel/core_pattern

Quoted 7 lines · On Thu, 17 Aug 2006 12:27:44 +0100 Alan Cox <[email protected]> wrote:

Ar Iau, 2006-08-17 am 11:46 +0200, ysgrifennodd Andi Kleen:

Quoted 5 lines

Several people from the embedded area wrote me privately
it would be useful for them. Also I think once it's in the main kernel there will be more incentive for user space to use it and I'm optimistic it will get some adoption (ok I guess I should write some better documentation, but there was no obvious place for it)

Quoted 6 lines

I don't believe that piping as such as neccessarily the right model, but the ability to intercept and processes core dumps from user space is asked for by many enterprise users as well. They want to know about, capture, analyse and process core dumps, often centrally and in automated form.

OK, fair enough.

Now let's think about security. Patches against ptrace, coredump and procfs give me the creeps because we've had (relatively) so many problems in those areas in the past.

So I'd suggest that we should look at this code and think about it in a really twisted fashion - does it open any exploits? I can't think of any, which is worth practically zero, but I don't see how this differs from /proc/sys/kernel/modprobe.

But still. Is this code secure?

#15 lore raw

Re: [PATCH] [3/3] Support piping into commands in /proc/sys/kernel/core_pattern

Quoted 1 line

But still. Is this code secure?

Any auditing from third parties appreciated.

I don't know of any obvious flaws (at least assuming the pipe handler isn't insecure code), but then I'm biased.

-Andi

#16 lore raw

Re: [PATCH] [3/3] Support piping into commands in /proc/sys/kernel/core_pattern

Quoted 6 lines · Andi Kleen
Quoted 1 line

But still. Is this code secure?

Any auditing from third parties appreciated.

I don't know of any obvious flaws (at least assuming the pipe handler isn't insecure code), but then I'm biased.

one angle is... is the userspace handler trusted code by root? Or is it allowed to be per user/user defined binary? In the later case there are all kinds of DoS scenarios possible; in the former it's root doing that to himself ;)

Signature · 1 line
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
#17 lore raw

Re: [PATCH] [3/3] Support piping into commands in /proc/sys/kernel/core_pattern

Quoted 5 lines · Arjan van de Ven
Quoted 4 lines · Andi Kleen
Quoted 1 line

But still. Is this code secure?

Any auditing from third parties appreciated.

Quoted 7 lines
Quoted 2 lines

I don't know of any obvious flaws (at least assuming the pipe handler isn't insecure code), but then I'm biased.

one angle is... is the userspace handler trusted code by root? Or is it allowed to be per user/user defined binary? In the later case there are all kinds of DoS scenarios possible; in the former it's root doing that to himself ;)

so one angle that needs to be resolved is the selinux interaction, where root!=root to some degree. If "any root" can control the sysctl, then this could lead to information leaks, or worse, to a "limited root" to "full root" escalation...

Quoted 1 line
Signature · 1 line
if you want to mail me at work (you don't), use arjan (at) linux.intel.com