Containers over 9p in 3.14

Hello,

We are using 9p to run containers on top of remote filesystems, and it works correctly using the 3.13.11 kernel. However, there were a bunch of updates in the fs/9p area in 3.14 which seem to have broken the namespaces support for containers.

A simple description of how we run the container over 9p is:
- on the remote source host we run diod as 9p server
- on the local host we mount the filesystem in /some/path
- also locally, we run
contain /some/path /bin/bash

In 3.14 (3.14.4 to be exact), the contain command can no longer mount the dev filesystem via mount("tmpfs", "dev", "tmpfs", 0, "mode=0755") after unsharing IPC, NS, USER, UTS and NET, but before unsharing PID. The above call returns an EPERM. You can look at the very simple code in contain.c and mount.c at [1].

This call used to work fine in 3.13. I haven't tried to bisect and find out the exact patch that introduces the problem, but it's one of "git log --oneline b26d4cd.. fs/9p".

I can provide you with any additional information that might be needed.

Thank you for any feedback.

Cheers,
Ailn.

[1] https://github.com/arachsys/containers

Hi,

Alin Dobre wrote on Tue, May 20, 2014 at 06:03:54PM +0100:

For curiosity, I wouldn't mind seeing your exact mount options ? (cache if any, access, posixacl if set, etc..)

I've got no experience with containers/no setup I could try it with right now, but I recall 3.14 broke the xattr_set call - you might want to try to revert this one:
* bdd5c28 9p: fix return value in case in v9fs_fid_xattr_set()

Philippe Deniel posted about it on this list a while ago but nothing that I could see got done to actually fix it, I guess I'll submit something today or tomorrow with proper fix (e.g. keep negative value if there's an error to report, but 0 on success)

I'll have a harder look if this isn't enough.

Dominique Martinet

That sounds very weird. The filesystem used should be orthogonal.

That is weird.

There has been a bunch of work to keep from abusing the ability to mount, that added some additional constraints.

For sys and proc those constraints pretty much mean other versions of those filesystems need to be mounted somewhere or you are not allowd to mount a fresh copy.

I can't think of any additional rules for tmpfs off of the top of my head.

I can't imagine off the top of my head how 9p or any specific filesystem would be involved.

If you are building your own kernels and this is easily reproducible it may make sense to instrument up the mount call with printks can see where the failure is coming from.

Unless it is a basic 9p bug where directories don't appear as directories in some cases my deep hunch is that you have found some way to run afoul of the restrictions on mount points.

Eric

Hi,

Alin Dobre wrote on Tue, May 20, 2014 at 06:03:54PM +0100:

For curiosity, I wouldn't mind seeing your exact mount options ? (cache if any, access, posixacl if set, etc..)

I've got no experience with containers/no setup I could try it with right now, but I recall 3.14 broke the xattr_set call - you might want to try to revert this one:
* bdd5c28 9p: fix return value in case in v9fs_fid_xattr_set()

Philippe Deniel posted about it on this list a while ago but nothing that I could see got done to actually fix it, I guess I'll submit something today or tomorrow with proper fix (e.g. keep negative value if there's an error to report, but 0 on success)

I'll have a harder look if this isn't enough.

Dominique Martinet

v9fs_fid_xattr_set is supposed to return 0 on success.

This corrects the behaviour introduced in commit bdd5c28dcb8330b9074404cc92a0b83aae5606a
"9p: fix return value in case in v9fs_fid_xattr_set()"

(The function returns a negative error on error, as expected)

---
 fs/9p/xattr.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index 04133a1..f95e01e 100644
@@ -156,7 +156,7 @@ int v9fs_fid_xattr_set(struct p9_fid *fid, const char *name,
 		offset += write_count;
 		value_len -= write_count;
 	}
-	retval = offset;
+	retval = 0;
 err:
 	p9_client_clunk(fid);
 	return retval;

1.9.0

Nothing fancy, really. I mounted it manually with:
mount -t 9p -o trans=tcp,aname=/some/path,version=9p2000.L 127.0.0.1 /mount/point. I can take a look at /proc/mounts to tell you all the mount options, including default ones if you need.

I noticed you sent a patch, I will try that and come back with the result.

Thanks a lot for the support, Dominique!

Cheers,
Alin.