2.6.35: unshare(NEWNS) does not work inside a container anymore?
I just noticed a regression - immediately after updating kernel from 2.6.32 to 2.6.35 (I skipped .33 and .34). Namely, unshare(CLONE_NEWNS) stopped workin from within a container, like this:
unshare(CLONE_NEWNS) = -1 EINVAL (Invalid argument)
There's no other fancy stuff going on around, just plain unshare and exec a new shell.
What's wrong with 2.6.35 in this context?
Thanks.
/mjt
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 9 lines · Michael Tokarev
I just noticed a regression - immediately after updating kernel from 2.6.32 to 2.6.35 (I skipped .33 and .34). Namely,
unshare(CLONE_NEWNS)stopped workin from within a container, like this:
unshare(CLONE_NEWNS)= -1 EINVAL (Invalid argument)There's no other fancy stuff going on around, just plain unshare and exec a new shell.
I'm not seeing this behavior. I'm on 2.6.35-19-generic (ubuntu maverick), created a lucid container with the standard template, and tested with ns_exec
(git clone git://git.sr71.net/~hallyn/cr_tests.git; git checkout ns_exec; make ns_exec; ns_exec -m /bin/bash; play with mounts; exit)
Can you give us /proc/self/status and capsh --print output from inside the container before you try to unshare, and maybe strace output from the program you were using?
Quoted 9 lines
What's wrong with 2.6.35 in this context?
Thanks.
/mjt
_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 11 lines · Serge E. Hallyn
Quoted 9 lines · Michael Tokarev
I just noticed a regression - immediately after updating kernel from 2.6.32 to 2.6.35 (I skipped .33 and .34). Namely,
unshare(CLONE_NEWNS)stopped workin from within a container, like this:
unshare(CLONE_NEWNS)= -1 EINVAL (Invalid argument)There's no other fancy stuff going on around, just plain unshare and exec a new shell.
Quoted 6 lines
I'm not seeing this behavior. I'm on 2.6.35-19-generic (ubuntu maverick), created a lucid container with the standard template, and tested with
ns_exec(git clone git://git.sr71.net/~hallyn/cr_tests.git; git checkout ns_exec; make ns_exec; ns_exec -m /bin/bash; play with mounts; exit)
This one is not using unshare(2), it is using clone(2) syscall.
I asked about unshare. In particular, lxc-unshare fails withing the container the same way too -- it too uses unshare().
Quoted 3 lines
Can you give us
/proc/self/statusand capsh --print output from inside the container before you try to unshare, and maybe strace output from the program you were using?
Sure.
# cat /proc/self/status Name: cat State: R (running) Tgid: 2663 Pid: 2663 PPid: 2660 TracerPid: 0 Uid: 0 0 0 0 Gid: 0 0 0 0 FDSize: 256 Groups: 0 VmPeak: 4944 kB VmSize: 4944 kB VmLck: 0 kB VmHWM: 232 kB VmRSS: 232 kB VmData: 160 kB VmStk: 136 kB VmExe: 40 kB VmLib: 1388 kB VmPTE: 24 kB VmSwap: 0 kB Threads: 1 SigQ: 4/63178 SigPnd: 0000000000000000 ShdPnd: 0000000000000000 SigBlk: 0000000000000000 SigIgn: 0000000000000000 SigCgt: 0000000000000000 CapInh: 0000000000000000 CapPrm: ffffffffffbfffff CapEff: ffffffffffbfffff CapBnd: ffffffffffbfffff Cpus_allowed: f Cpus_allowed_list: 0-3 Mems_allowed: 1 Mems_allowed_list: 0 voluntary_ctxt_switches: 3 nonvoluntary_ctxt_switches: 2
# capsh --print Current: =ep cap_sys_boot-ep Bounding set =cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_nice,cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin Securebits: 00/0x0 secure-noroot: no (unlocked) secure-no-suid-fixup: no (unlocked) secure-keep-caps: no (unlocked) uid=0
# strace clone --fs bash
execve("/usr/sbin/clone", ["clone", "--fs", "bash"], [/* 15 vars */]) = 0
brk(0) = 0x834c000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xf76f1000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=18528, ...}) = 0
mmap2(NULL, 18528, PROT_READ, MAP_PRIVATE, 3, 0) = 0xf76ec000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/i686/cmov/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320m\1\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=1327556, ...}) = 0
mmap2(NULL, 1337704, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xf75a5000
mprotect(0xf76e5000, 4096, PROT_NONE) = 0
mmap2(0xf76e6000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x140) = 0xf76e6000
mmap2(0xf76e9000, 10600, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xf76e9000
close(3) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xf75a4000
set_thread_area({entry_number:-1 -> 12, base_addr:0xf75a46c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0
mprotect(0xf76e6000, 8192, PROT_READ) = 0
mprotect(0xf770f000, 4096, PROT_READ) = 0
munmap(0xf76ec000, 18528) = 0
unshare(CLONE_NEWNS) = -1 EINVAL (Invalid argument)
write(2, "clone: unshare: Invalid argument"..., 33clone: unshare: Invalid argument
) = 33
exit_group(1) = ?The source of this clone program is available at
http://www.corpit.ru/mjt/clone.c - I use it for
a long time, it works on this same machine
outside of containers, and it worked in 2.6.32.
Thanks!
/mjt
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 12 lines · Michael Tokarev
Quoted 11 lines · Serge E. Hallyn
Quoted 9 lines · Michael Tokarev
I just noticed a regression - immediately after updating kernel from 2.6.32 to 2.6.35 (I skipped .33 and .34). Namely,
unshare(CLONE_NEWNS)stopped workin from within a container, like this:
unshare(CLONE_NEWNS)= -1 EINVAL (Invalid argument)There's no other fancy stuff going on around, just plain unshare and exec a new shell.
Quoted 8 lines
Quoted 6 lines
I'm not seeing this behavior. I'm on 2.6.35-19-generic (ubuntu maverick), created a lucid container with the standard template, and tested with
ns_exec(git clone git://git.sr71.net/~hallyn/cr_tests.git; git checkout ns_exec; make ns_exec; ns_exec -m /bin/bash; play with mounts; exit)This one is not using unshare(2), it is using clone(2) syscall.
That's only the case if you do 'ns_exec -cm'.
Quoted 2 lines
I asked about unshare. In particular, lxc-unshare fails withing the container the same way too -- it too uses
unshare().
lxc-unshare -s MOUNT /bin/bash passes here too.
Quoted 84 lines
Quoted 3 lines
Can you give us
/proc/self/statusand capsh --print output from inside the container before you try to unshare, and maybe strace output from the program you were using?Sure.
# cat /proc/self/status Name: cat State: R (running) Tgid: 2663 Pid: 2663 PPid: 2660 TracerPid: 0 Uid: 0 0 0 0 Gid: 0 0 0 0 FDSize: 256 Groups: 0 VmPeak: 4944 kB VmSize: 4944 kB VmLck: 0 kB VmHWM: 232 kB VmRSS: 232 kB VmData: 160 kB VmStk: 136 kB VmExe: 40 kB VmLib: 1388 kB VmPTE: 24 kB VmSwap: 0 kB Threads: 1 SigQ: 4/63178 SigPnd: 0000000000000000 ShdPnd: 0000000000000000 SigBlk: 0000000000000000 SigIgn: 0000000000000000 SigCgt: 0000000000000000 CapInh: 0000000000000000 CapPrm: ffffffffffbfffff CapEff: ffffffffffbfffff CapBnd: ffffffffffbfffff Cpus_allowed: f Cpus_allowed_list: 0-3 Mems_allowed: 1 Mems_allowed_list: 0 voluntary_ctxt_switches: 3 nonvoluntary_ctxt_switches: 2# capsh --print Current: =ep cap_sys_boot-ep Bounding set =cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_nice,cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin Securebits: 00/0x0 secure-noroot: no (unlocked) secure-no-suid-fixup: no (unlocked) secure-keep-caps: no (unlocked) uid=0# strace clone --fs bash execve("/usr/sbin/clone", ["clone", "--fs", "bash"], [/* 15 vars */]) = 0 brk(0) = 0x834c000 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xf76f1000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) open("/etc/ld.so.cache", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=18528, ...}) = 0 mmap2(NULL, 18528, PROT_READ, MAP_PRIVATE, 3, 0) = 0xf76ec000 close(3) = 0 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) open("/lib/i686/cmov/libc.so.6", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320m\1\0004\0\0\0"..., 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=1327556, ...}) = 0 mmap2(NULL, 1337704, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xf75a5000 mprotect(0xf76e5000, 4096, PROT_NONE) = 0 mmap2(0xf76e6000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x140) = 0xf76e6000 mmap2(0xf76e9000, 10600, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xf76e9000 close(3) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xf75a4000 set_thread_area({entry_number:-1 -> 12, base_addr:0xf75a46c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 mprotect(0xf76e6000, 8192, PROT_READ) = 0 mprotect(0xf770f000, 4096, PROT_READ) = 0 munmap(0xf76ec000, 18528) = 0 unshare(CLONE_NEWNS) = -1 EINVAL (Invalid argument) write(2, "clone: unshare: Invalid argument"..., 33clone: unshare: Invalid argument ) = 33 exit_group(1) = ?
Quoted 4 lines
The source of this clone program is available at
http://www.corpit.ru/mjt/clone.c - I use it for
a long time, it works on this same machine
outside of containers, and it worked in 2.6.32.
Hm, is working for me. You're on a plain upstream 2.6.35, as in commitid 9fe6206f400646a2322096b56c59891d530e8d51 ?
I see nothing obvious in your output, unfortunately.
-serge
A message that is not in the archive (sent off-list or not yet received): [email protected]
01.09.2010 23:41, Serge E. Hallyn wrote:
[]
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
unshare(CLONE_NEWNS)= -1 EINVAL (Invalid argument)
[]
Quoted 4 lines
Quoted 2 lines
Quoted 1 line
ns_exec-m/bin/bash; play with mounts; exit)This one is not using unshare(2), it is using clone(2) syscall.
That's only the case if you do '
ns_exec-cm'.
Oh. I missed that.
[]
Quoted 7 lines
Quoted 4 lines
The source of this clone program is available at
http://www.corpit.ru/mjt/clone.c - I use it for a long time, it works on this same machine outside of containers, and it worked in 2.6.32.Hm, is working for me. You're on a plain upstream 2.6.35, as in commitid
9fe6206f400646a2322096b56c59891d530e8d51?
No, it's 2.6.35.4 - last stable. Plain 2.6.35 works (or fails) the same for me as 2.6.35 - this one:
http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.35.tar.bz2
But I see at least one possible difference: I run 64bit kernel and a 32bit userspace, including lxc tools and unshare code. Lemme check with 64bit (native) userspace....
/mjt
Quoted 11 lines · Michael Tokarev
I just noticed a regression - immediately after updating kernel from 2.6.32 to 2.6.35 (I skipped .33 and .34). Namely,
unshare(CLONE_NEWNS)stopped workin from within a container, like this:
unshare(CLONE_NEWNS)= -1 EINVAL (Invalid argument)There's no other fancy stuff going on around, just plain unshare and exec a new shell.
What's wrong with 2.6.35 in this context?
So, after discussing this on IRC and doing some discovery, it turned out to be new (in 2.6.35) cgroup subsystem -- block I/O controller (CONFIG_BLK_CGROUP). This one does not allow more than 1 level of nesting, so, for example, it is impossible to create a subdirectory in another cgroup dir in cgroupfs:
mkdir /dev/cgroup/foo -- this one succeeds, but mkdir /dev/cgroup/foo/bar -- this fails as long
as blkio mount option is enabled. Once disabled, it works again.
In 2.6.35 block/blk-cgroup.c, blkiocg_create() there's the following code:
/* Currently we do not support hierarchy deeper than two level (0,1) */ if (parent != cgroup->top_cgroup)
return ERR_PTR(-EINVAL);
In 2.6.36-tobe it were changed to
return ERR_PTR(-EPERM);
but the issue remains anyway. What is problematic here is that blkio is different from all other cgroups in this very respect (not allowing nesting), but there's no indication of this fact anywhere. At least, the above quoted place warranrs a WARN() or WARN_ONCE() to tell the user what's going on - or else it's very difficult to debug.
Speaking of real solution, it looks like disallowing nesting should be done in a different way. Maybe allow creation of a subcontainer but reset the limits in there and catch attempts to set them, - I dunno. Or, don't clone whole cgroup hierarchy on CLONE_NEWNS only.
Current situation is too restrictive IMHO - blkio controller is useful for a container like LXC, but currently it implies that one can't create even a new filesystem namespace within it.
Thanks.
/mjt