Mount remount operations propagating from container to host and other containers.

Hey all,

Been running into an ugly situation with LXC-Tools that seems to be pointing up a real serious leakage from containers. If you have a mount inside a container (presumably a bind mount in this case), if the container does a mount -o remount (say rw->ro or ro->rw) this propagates to the host mount points (all the way to the primary mount point for that partition in some cases) and is reflected in other containers.

This first show up with containers running full VM's running on a mounted fs (aot the host / fs) were causing the real mounted fs to become ro when they were shut down (the VM was remounting its rootfs as ro and it was leaking out of the container).

I've since confirmed that and encountered it trying to have a shared ro mounted fs in a container using bind mounts (bind mounts since 2.6.26 have allowed setting the ro flag on individual mount points) and discovering that one container could make it rw and then all the other containers would see it as rw as well! If a container made a mount point ro, all the other containers would see it as ro and the mount point for the entire real fs in the host would become ro! This is very not good. That's a pretty serious leakage from the containers out to the host.

Is this a problem with the container isolation or some problem in creating the container?

I'm running and testing on a Fedora 12 system with a 2.6.32 kernel. Not related (I don't think) but I have also noted that linux-utils-ng on F16 seems to also have a bug irt something similar here. If I mount a directory from a mounted partition onto another location and then make that other location ro, the entire partition becomes ro. BUT! If I then make the partition rw, that does not propagate back up and the bind mount remains ro.

What should work is this:

Partition /export
Directory /export/readonly

mount --bind /export/readonly /srv/readonly

At this point, /export and /srv/readonly are both rw

mount -o remount,ro /srv/readonly

Now. both /export and /srv/readonly are ro! This is wrong. Only /srv/readonly is suppose to be ro!

Now, running...

mount -o remount,rw /export

now, /export is rw and /srv/readonly is readonly.

Back to containers...

If I have /srv/readonly mounted in several containers (same mount point) it's ro in the host and in the containers...

Running this in one container:

mount -o remount,rw /srv/readonly

(I seriously wish this would NOT WORK AT ALL, but it does. I don't want the container to be able to write to that partition at all, like the media was RO. Anybody have any ideas on that one?)

Now /srv/readonly is rw in the host and all the containers!

(EVEN WORSE!)

Running this in one container:

mount -o remount,ro /srv/readonly

NOW /srv/readonly is ro in all the containers and /export is ro in the host. NOT GOOD.

Thoughts?

Regards,
Mike

Michael H. Warfield (AI4NB) | (770) 985-6132 |  mhw-BetbSzk+GohWk0Htik3J/[email protected]
   /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
   NIC whois: MHW9          | An optimist believes we live in the best of all
 PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!

_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers

I think you'll want to mount --make-rslave or --make-rprivate during container setup. Distros vary with how they leave / set up, and if it is all rshared from the start, then yeah your containers' mount actions will propagate backward.

That is what the work to make VFS respect hierarchical user namespaces is for. Alas it is a perennial victim of time shortages. It would let you mark that fs as being owned by root in the initial user namespace. Then root in a child user namespace would only get the world access rights (like any other stranger user) to the fs and its files. If an fs or files should be owned by user 500 in a container, then inode->i_uid will actually store 500, the fs will store a user ns id (which only a particular uid in init_user_ns can associate with a container) in which user 500 is valid, and optionally userids to use for file access from higher user namespaces. So the file is owned by userid 500 in userns 5 which is a child of init_user_ns, userns 5 is owned by userid 1000 in init_user_ns, so the file is owned by userid 1000 in init_user_ns.

There are quite a few threads in the containers mailing list archive between Eric and I about this, if you're interested in the background.

Part of the reason this work seems to refuse to get past 5-patch prototypes is that only Eric and I seem to be interested. Maybe now that there is a serious growing user base around lxc that will change.

-serge