Skip to contentSkip to conversation

v9fs does not honor open file handles on anonymous files

DISCUSSION Richard Yao 2 participants 3 messages Dec 31, 2013 VFS

lore
Share & tools
lore
#0 lore raw

I have a small shell script:

#!/bin/bash
cat <<-EOF
EOF

Running this causes bash to fork via clone(), set fd=0 to point to an empty file in /tmp, unlink it and then execve cat. Specifically, something like this;

[pid 3699] open("/tmp/sh-thd-1388524249",
O_WRONLY|O_CREAT|O_EXCL|O_TRUNC, 0600) = 3

[pid  3699] open("/tmp/sh-thd-1388524249", O_RDONLY) = 4
[pid  3699] close(3)                    = 0
[pid  3699] unlink("/tmp/sh-thd-1388524249") = 0
[pid  3699] dup2(4, 0)                  = 0
[pid  3699] close(4)                    = 0
[pid  3699] execve("/bin/cat", ["cat"], [/* 22 vars */]) = 0

One of the first things that cat does is fstat fd=0. This informs cat that the file is empty and it will quit. If /tmp is on other filesystems, cat will fstat fd=0, receive a return code of 0 from fstat, print nothing and then quit normally. If /tmp is on 9pfs, cat will fstat fd=0, receive ENOENT from fstat, print cat: -: No such file or directory to stderr and die.

It seems that v9fs_vfs_unlink() is killing the file while we still have open file handles. I have confirmed that this behavior occurs on Linux 3.13.0-rc6. This breaks several things when Gentoo is on a 9p rootfs (e.g. gcc-config, any emerge command that involves a C compiler, etcetera) inside QEMU. I have placed /tmp and /var/tmp/portage on a tmpfs as a hack-fix, but it would be better to get this fixed.

I doubt that I will write a patch to fix this. I am sending the details to the list so the 9p maintainers or any other interested individual can fix it.

#1 lore raw

Hi,

Richard Yao wrote on Tue, Dec 31, 2013 :

Quoted 24 lines

#!/bin/bash
cat <<-EOF
EOF

Running this causes bash to fork via clone(), set fd=0 to point to an empty file in /tmp, unlink it and then execve cat. Specifically, something like this;

[pid 3699] open("/tmp/sh-thd-1388524249",
O_WRONLY|O_CREAT|O_EXCL|O_TRUNC, 0600) = 3

[pid  3699] open("/tmp/sh-thd-1388524249", O_RDONLY) = 4
[pid  3699] close(3)                    = 0
[pid  3699] unlink("/tmp/sh-thd-1388524249") = 0
[pid  3699] dup2(4, 0)                  = 0
[pid  3699] close(4)                    = 0
[pid  3699] execve("/bin/cat", ["cat"], [/* 22 vars */]) = 0

It seems that v9fs_vfs_unlink() is killing the file while we still have open file handles. I have confirmed that this behavior occurs on Linux 3.13.0-rc6. This breaks several things when Gentoo is on a 9p rootfs (e.g. gcc-config, any emerge command that involves a C compiler, etcetera) inside QEMU. I have placed /tmp and /var/tmp/portage on a tmpfs as a hack-fix, but it would be better to get this fixed.

Quoted 3 lines

I doubt that I will write a patch to fix this. I am sending the details to the list so the 9p maintainers or any other interested individual can fix it.

I'm not sure if it is the client's job to remember the file has been unlinked and only really unlink it after all the file handles are closed or if we should expect the server to do it.

It might not matter in the case of qemu acting as a 9p server, but there are a couple of network 9p2000.L servers out there (diod[1] and nfs-ganesha[2]), and if the file is open on one client and unlinked on another client.. How can the second client wait properly?

For what's it's worth, nfs-ganesha already behaves properly and this will work with /tmp being a 9P mount off it.

It might be worth looking into qemu's code and see if it wouldn't be easy to hold the unlink ? I've got to admit I honestly have no clue there.
(or at least send them a copy of your mail :))

[1] diod "Distributed I/O daemon" - http://code.google.com/p/diod/
[2] nfs-ganesha - https://github.com/nfs-ganesha/nfs-ganesha

Cheers,

Signature · 1 line
Dominique Martinet
#2 lore raw
Quoted 1 line · Dominique Martinet

Hi,

Thanks for the fast reply.

Quoted 25 lines

Richard Yao wrote on Tue, Dec 31, 2013 :

Quoted 24 lines

#!/bin/bash
cat <<-EOF
EOF

Running this causes bash to fork via clone(), set fd=0 to point to an empty file in /tmp, unlink it and then execve cat. Specifically, something like this;

[pid 3699] open("/tmp/sh-thd-1388524249",
O_WRONLY|O_CREAT|O_EXCL|O_TRUNC, 0600) = 3

[pid  3699] open("/tmp/sh-thd-1388524249", O_RDONLY) = 4
[pid  3699] close(3)                    = 0
[pid  3699] unlink("/tmp/sh-thd-1388524249") = 0
[pid  3699] dup2(4, 0)                  = 0
[pid  3699] close(4)                    = 0
[pid  3699] execve("/bin/cat", ["cat"], [/* 22 vars */]) = 0

It seems that v9fs_vfs_unlink() is killing the file while we still have open file handles. I have confirmed that this behavior occurs on Linux 3.13.0-rc6. This breaks several things when Gentoo is on a 9p rootfs (e.g. gcc-config, any emerge command that involves a C compiler, etcetera) inside QEMU. I have placed /tmp and /var/tmp/portage on a tmpfs as a hack-fix, but it would be better to get this fixed.

Quoted 7 lines
Quoted 3 lines

I doubt that I will write a patch to fix this. I am sending the details to the list so the 9p maintainers or any other interested individual can fix it.

I'm not sure if it is the client's job to remember the file has been unlinked and only really unlink it after all the file handles are closed or if we should expect the server to do it.

The answer to this is largely philosophical in nature, but it is a good question to ask. Quite honestly, I have no preference, but I was not sure who was responsible. Having caught another bug that I thought was in QEMU that turned out to be in Linux, I decided to ping the Linux folks first this time.

Speaking of which, that patch did not get much attention because I did not send it to the right lists. I will resend it soon:

http://www.spinics.net/lists/linux-virtualization/msg21716.html

Quoted 5 lines

It might not matter in the case of qemu acting as a 9p server, but there are a couple of network 9p2000.L servers out there (diod[1] and nfs-ganesha[2]), and if the file is open on one client and unlinked on another client.. How can the second client wait properly?

Quoted 8 lines

For what's it's worth, nfs-ganesha already behaves properly and this will work with /tmp being a 9P mount off it.

It might be worth looking into qemu's code and see if it wouldn't be easy to hold the unlink ? I've got to admit I honestly have no clue there.
(or at least send them a copy of your mail :))

If nfs-ganesha handles this properly, then I am going to say this is QEMU's bug. I had CCed Aneesh, who I am told is responsible for the QEMU 9p code, so hopefully we will hear from him soon. Otherwise, I will resend to the QEMU list.

Quoted 7 lines

[1] diod "Distributed I/O daemon" - http://code.google.com/p/diod/
[2] nfs-ganesha - https://github.com/nfs-ganesha/nfs-ganesha

Cheers,