procfs reports unexpected mnt_id and path for overlayfs files

DISCUSSION Andrew Vagin 3 participants 6 messages Apr 15, 2015 – Oct 12, 2015

lore

Share & tools

lore

Andrew Vagin

author

2015-04-15 08:50 UTC #0 lore raw

Hello Miklos and Everyone else,

We have a report that a container with overlayfs can't be checkpointed.

To dump file descriptors we get information from /proc/pid/fd/ and /proc/pid/fdinfo.

But in case of overlayfs we see unexpected values there.

/proc/PID/fdinfo/FD contains mnt_id which isn't presented in /proc/pid/mountinfo. /proc/PID/fd/FD does not contain an absolute path to the opened file.

Can someone help to fix these issues?

Here is a sequance of commands to reproduce the problem:

[root@ ]# mount -t overlay overlay -olowerdir=a,upperdir=b,workdir=c z
[root@ ]# cat /proc/self/mountinfo | grep over
163 77 0:42 / /home/avagin/tmp/z rw,relatime shared:109 - overlay overlay rw,lowerdir=a,upperdir=b,workdir=c [root@ ]# ls -l /home/avagin/tmp/z/a
-rw-r--r-- 1 root root 0 Apr 14 09:43 /home/avagin/tmp/z/a
[root@ ]# ipython
In [1]: fd = open("/home/avagin/tmp/z/a")

In [2]:
[1]+ Stopped ipython
[root@ ]# ps -C ipython
PID TTY TIME CMD
8868 pts/6 00:00:00 ipython
[root@ ]# cat /proc/8868/fdinfo/5
pos: 0
flags: 0100000
mnt_id: 164

# we expect to see mnt_id: 163

[root@ ]# ls -l /proc/8868/fd/5
lr-x------ 1 root root 64 Apr 15 11:21 /proc/8868/fd/5 -> /a

# we expect to see /proc/8868/fd/5 -> /home/avagin/tmp/z/a

Thanks,
Andrew

Miklos Szeredi

2015-04-15 15:07 UTC #1 lore raw

Quoted 4 lines · Andrew Vagin

Hello Miklos and Everyone else,
We have a report that a container with overlayfs can't be checkpointed.

Quoted 8 lines

To dump file descriptors we get information from /proc/pid/fd/ and /proc/pid/fdinfo.
But in case of overlayfs we see unexpected values there.
/proc/PID/fdinfo/FD contains mnt_id which isn't presented in /proc/pid/mountinfo. /proc/PID/fd/FD does not contain an absolute path to the opened file.
Can someone help to fix these issues?

See this:

https://lkml.org/lkml/2015/3/20/372

Al didn't like that patch. But it definitely will be fixed, one way or the other, in the near future.

Thanks,
Miklos

Saied Kazemi

2015-04-15 17:47 UTC #2 lore raw

Andrew, thank you for reporting and providing an easy way to reproduce the problem.

Miklos, thanks for the quick reply. If you'd like to see the issue with checkpointing and restoring Docker containers using OverlayFS, you can use the docker binary here https://github.com/SaiedKazemi/docker/releases.

--Saied

Quoted 26 lines · Miklos Szeredi

Quoted 5 lines · Andrew Vagin
Hello Miklos and Everyone else,
We have a report that a container with overlayfs can't be checkpointed.
To dump file descriptors we get information from /proc/pid/fd/ and
/proc/pid/fdinfo.
Quoted 4 lines
But in case of overlayfs we see unexpected values there.
/proc/PID/fdinfo/FD contains mnt_id which isn't presented in
/proc/pid/mountinfo.
Quoted 3 lines
/proc/PID/fd/FD does not contain an absolute path to the opened file.
Can someone help to fix these issues?
See this:
https://lkml.org/lkml/2015/3/20/372
Al didn't like that patch. But it definitely will be fixed, one way or the other, in the near future.
Thanks,
Miklos

Saied Kazemi

2015-05-29 21:29 UTC #3 lore raw

In reply to Miklos Szeredi

Hi Miklos,

Is there an update on this patch? I checked OverlayFS in 4.1-rc5 and noticed that the issue is still there.

I also observed something interesting/confusing applying this patch to 3.18. When Docker 1.5 sets up a container, the path into OverlayFS for a bind mounted file is not visible in /proc/<pid>/mountinfo which means that the patch has been effective. But when using Docker 1.7, the path is visible again. See below as an example for /etc/hosts:

Using Docker 1.5

# docker run -d busybox:latest ...
# cat /proc/<pid>/mountinfo
...
123 114 8:2 /docker/containers/<container-id>/hosts /etc/hosts
rw,relatime - ext4 /dev/sda2 rw,data=ordered

Using Docker 1.7

# docker run -d busybox:latest ...
# cat /proc/<pid>/mountinfo
...
77 68 8:2 /docker/containers/<container-id>/hosts
/sda2/docker/overlay/<container-id>/merged/etc/hosts rw,relatime -
ext4 /dev/sda2 rw,data=ordered

I verified that the mount system call to set up the container's filesystem is identical between Docker 1.5 and 1.7. Any idea as to what may be causing this problem?

--Saied

Quoted 23 lines · Miklos Szeredi

Quoted 4 lines · Andrew Vagin
Hello Miklos and Everyone else,
We have a report that a container with overlayfs can't be checkpointed.
Quoted 8 lines
To dump file descriptors we get information from /proc/pid/fd/ and /proc/pid/fdinfo.
But in case of overlayfs we see unexpected values there.
/proc/PID/fdinfo/FD contains mnt_id which isn't presented in /proc/pid/mountinfo. /proc/PID/fd/FD does not contain an absolute path to the opened file.
Can someone help to fix these issues?
See this:
https://lkml.org/lkml/2015/3/20/372
Al didn't like that patch. But it definitely will be fixed, one way or the other, in the near future.
Thanks,
Miklos

Saied Kazemi

2015-07-06 17:45 UTC #4 lore raw

In reply to Miklos Szeredi

Any updates on this?

Thanks,
--Saied

Quoted 23 lines · Miklos Szeredi

Quoted 4 lines · Andrew Vagin
Hello Miklos and Everyone else,
We have a report that a container with overlayfs can't be checkpointed.
Quoted 8 lines
To dump file descriptors we get information from /proc/pid/fd/ and /proc/pid/fdinfo.
But in case of overlayfs we see unexpected values there.
/proc/PID/fdinfo/FD contains mnt_id which isn't presented in /proc/pid/mountinfo. /proc/PID/fd/FD does not contain an absolute path to the opened file.
Can someone help to fix these issues?
See this:
https://lkml.org/lkml/2015/3/20/372
Al didn't like that patch. But it definitely will be fixed, one way or the other, in the near future.
Thanks,
Miklos

Miklos Szeredi

2015-10-12 17:16 UTC #5 lore raw

Quoted 13 lines · Saied Kazemi

Quoted 12 lines · Miklos Szeredi
Quoted 11 lines · Andrew Vagin
Hello Miklos and Everyone else,
We have a report that a container with overlayfs can't be checkpointed.
To dump file descriptors we get information from /proc/pid/fd/ and /proc/pid/fdinfo.
But in case of overlayfs we see unexpected values there.
/proc/PID/fdinfo/FD contains mnt_id which isn't presented in /proc/pid/mountinfo. /proc/PID/fd/FD does not contain an absolute path to the opened file.

Quoted 8 lines

Quoted 8 lines
Quoted 1 line
Can someone help to fix these issues?
See this:
https://lkml.org/lkml/2015/3/20/372
Al didn't like that patch. But it definitely will be fixed, one way or the other, in the near future.

Should be fixed in 4.2.

Thanks,
Miklos