Net containers config and usage

All,

I'm new to linux containers and am interested in the network containers
support, but am having some troubles getting it working & could use some help.

I saw that the code released to 2.6.26 was incompatible with sysfs, but
that 2.6.29 should have support for both.

So, I built:

    linux-2.6.29-rc1
    iproute2.2.6.25 plus the patch for containers support
    lxc-0.5.2

I can now run lxc-unshare -n /bin/tcsh And, as expected, that process only sees loopback.

But I'm having trouble assigning an interface over to that container.

I tried doing:

     # echo <pid> > /sys/class/net/eth4/new_ns_pid

But there are no entries in sysfs called new_ns_pid.

I also tried doing ip link set eth4 newns <pid>
but I get a netlink error:
RTNETLINK answers: Invalid argument

Any tips on what I might be doing wrong, or how to get this working? Since I can do the lxc-unshare, it seems like I'm part way there, but can't move network devices between namespaces.

Quoted 7 lines

But I'm having trouble assigning an interface over to that container.

I tried doing:

     # echo <pid> > /sys/class/net/eth4/new_ns_pid

But there are no entries in sysfs called new_ns_pid.

Yeah I'm not sure where that command came from.

To move eth4 into a new
netns, you'd want to do

ip link set eth4 netns $pid_of_shell_in_ns

See http://lxc.sourceforge.net/network/configuration.php for more info.

Quoted 5 lines · Serge E. Hallyn

Quoted 5 lines

But I'm having trouble assigning an interface over to that container.

I tried doing:

     # echo <pid> > /sys/class/net/eth4/new_ns_pid

Quoted 4 lines

Quoted 2 lines

But there are no entries in sysfs called new_ns_pid.

Yeah I'm not sure where that command came from.

OK - just FYI, I got that command from http://lxc.sourceforge.net/network.php

Quoted 10 lines

To move eth4 into a new
netns, you'd want to do

ip link set eth4 netns $pid_of_shell_in_ns

See http://lxc.sourceforge.net/network/configuration.php for more info.

Trying to use the "ip link set" command, I get a different problem - it reports an RTNETLINK error:

[root@C100273 iproute2-2.6.25]# ps aux | grep tcsh

 root     21918  0.0  0.0  71036  2084 pts/1    S+   16:47   0:00 -bin/tcsh
 root     22008  0.0  0.0  61144   716 pts/2    R+   16:57   0:00 grep tcsh

[root@C100273 iproute2-2.6.25]# ip/ip link set eth4 netns 21918 RTNETLINK answers: Invalid argument

Any idea what could cause that error?

Quoted 7 lines · [email protected] wrote:

Quoted 5 lines · Serge E. Hallyn

Quoted 5 lines

But I'm having trouble assigning an interface over to that container.

I tried doing:

     # echo <pid> > /sys/class/net/eth4/new_ns_pid

Quoted 7 lines

Quoted 4 lines

Quoted 2 lines

But there are no entries in sysfs called new_ns_pid.

Yeah I'm not sure where that command came from.

OK - just FYI, I got that command from http://lxc.sourceforge.net/network.php

You are right, it is not up-to-date.

Quoted 20 lines

Quoted 10 lines

To move eth4 into a new
netns, you'd want to do

ip link set eth4 netns $pid_of_shell_in_ns

See http://lxc.sourceforge.net/network/configuration.php for more info.

Trying to use the "ip link set" command, I get a different problem - it reports an RTNETLINK error:

[root@C100273 iproute2-2.6.25]# ps aux | grep tcsh

 root     21918  0.0  0.0  71036  2084 pts/1    S+   16:47   0:00 -bin/tcsh
 root     22008  0.0  0.0  61144   716 pts/2    R+   16:57   0:00 grep tcsh

[root@C100273 iproute2-2.6.25]# ip/ip link set eth4 netns 21918 RTNETLINK answers: Invalid argument

Quoted 2 lines

Any idea what could cause that error?

Assuming your prompt is showing the right version, try a version of iproute2 >= 2.6.26

Quoted 11 lines

Quoted 9 lines

[root@C100273 iproute2-2.6.25]# ps aux | grep tcsh

 root     21918  0.0  0.0  71036  2084 pts/1    S+   16:47   0:00 -bin/tcsh
 root     22008  0.0  0.0  61144   716 pts/2    R+   16:57   0:00 grep tcsh

[root@C100273 iproute2-2.6.25]# ip/ip link set eth4 netns 21918 RTNETLINK answers: Invalid argument

Any idea what could cause that error?

Assuming your prompt is showing the right version, try a version of iproute2 >= 2.6.26

Right, this was a bit of a pain for me - when you install iproute, by default it goes into another location, so you probably have to specify /usr/local/sbin/ip or something. Just a guess, but that may be your problem here :)

-serge

Quoted 12 lines

Quoted 11 lines

Quoted 9 lines

[root@C100273 iproute2-2.6.25]# ps aux | grep tcsh

 root     21918  0.0  0.0  71036  2084 pts/1    S+   16:47   0:00 -bin/tcsh
 root     22008  0.0  0.0  61144   716 pts/2    R+   16:57   0:00 grep tcsh

[root@C100273 iproute2-2.6.25]# ip/ip link set eth4 netns 21918 RTNETLINK answers: Invalid argument

Any idea what could cause that error?

Assuming your prompt is showing the right version, try a version of iproute2 >= 2.6.26

Quoted 6 lines

Right, this was a bit of a pain for me - when you install iproute, by default it goes into another location, so you probably have to specify /usr/local/sbin/ip or something. Just a guess, but that may be your problem here :)

-serge

I was running 2.6.25 plus the patch from here:

    http://marc.info/?l=linux-netdev&m=120905226611742&q=raw

But I tried getting 2.6.26 and I have the same problem:

[root@C100273 iproute2-2.6.26]# ip/ip link set eth4 netns 24755 RTNETLINK answers: Invalid argument

I'm pretty certain I'm running the right "version" of the command. It does install it to /usr/sbin/ip, but if I run the version that came with the distribution (in /sbin/ip), I get a different error reporting netns as garbage. So.. since it at least parses the netns argument and sends the netlink request, it at least seems to be making it to the kernel.

Does the kernel.org version of 2.6.29-rc1 have everything I need?

Any other ideas?

Thanks,
Chris
_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers

Works for me in 2.6.27.
Did you enable network namespace support in the kernel ?

Guenter

Quoted 12 lines

Quoted 6 lines

Quoted 6 lines

[root@C100273 iproute2-2.6.25]# ps aux | grep tcsh

 root     21918  0.0  0.0  71036  2084 pts/1    S+   16:47   0:00 -bin/tcsh
 root     22008  0.0  0.0  61144   716 pts/2    R+   16:57   0:00 grep tcsh

[root@C100273 iproute2-2.6.25]# ip/ip link set eth4 netns 21918 RTNETLINK answers: Invalid argument

Quoted 6 lines

Quoted 3 lines

Any idea what could cause that error?

Assuming your prompt is showing the right version, try a version of iproute2 >= 2.6.26

Quoted 7 lines

Right, this was a bit of a pain for me - when you install iproute, by default it goes into another location, so you probably have to specify /usr/local/sbin/ip or something. Just a guess, but that may be your problem here :)

-serge

I was running 2.6.25 plus the patch from here:

    http://marc.info/?l=linux-netdev&m=120905226611742&q=raw

But I tried getting 2.6.26 and I have the same problem:

[root@C100273 iproute2-2.6.26]# ip/ip link set eth4 netns 24755 RTNETLINK answers: Invalid argument

I'm pretty certain I'm running the right "version" of the command. It does install it to /usr/sbin/ip, but if I run the version that came with the distribution (in /sbin/ip), I get a different error reporting netns as garbage. So.. since it at least parses the netns argument and sends the netlink request, it at least seems to be making it to the kernel.

Does the kernel.org version of 2.6.29-rc1 have everything I need?

Any other ideas?

Quoted 14 lines

I tried getting 2.6.26 and I have the same problem:

[root@C100273 iproute2-2.6.26]# ip/ip link set eth4 netns 24755 RTNETLINK answers: Invalid argument

I'm pretty certain I'm running the right "version" of the command. It does install it to /usr/sbin/ip, but if I run the version that came with the
distribution (in /sbin/ip), I get a different error reporting netns as garbage.
So.. since it at least parses the netns argument and sends the netlink request,
it at least seems to be making it to the kernel.

Quoted 4 lines

Does the kernel.org version of 2.6.29-rc1 have everything I need?

Any other ideas?

Is eth4 a physical network device or a virtual device (like a tunnel) renamed eth4 ?

Can you send the output of the "ip link" command and the value of /sys/class/net/eth4/features to check if the network device is not flagged as a non-movable.

thanks.
-- Daniel

Quoted 21 lines · Daniel Lezcano

Quoted 14 lines

I tried getting 2.6.26 and I have the same problem:

[root@C100273 iproute2-2.6.26]# ip/ip link set eth4 netns 24755 RTNETLINK answers: Invalid argument

I'm pretty certain I'm running the right "version" of the command. It does install it to /usr/sbin/ip, but if I run the version that came with the
distribution (in /sbin/ip), I get a different error reporting netns as garbage.
So.. since it at least parses the netns argument and sends the netlink request,
it at least seems to be making it to the kernel.

Quoted 4 lines

Does the kernel.org version of 2.6.29-rc1 have everything I need?

Any other ideas?

Is eth4 a physical network device or a virtual device (like a tunnel) renamed eth4 ?

Quoted 6 lines

Can you send the output of the "ip link" command and the value of /sys/class/net/eth4/features to check if the network device is not flagged as a non-movable.

thanks.
-- Daniel

It's a physical device - uses the e1000e driver.
Here's the output of both commands.

[root@C100273 iproute2-2.6.26]# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:1c:c4:dc:44:b2 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
    link/ether 00:1c:c4:48:35:f5 brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
    link/ether 00:1c:c4:dc:44:b0 brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
    link/ether 00:1c:c4:48:35:f4 brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:1c:c4:48:35:f7 brd ff:ff:ff:ff:ff:ff
7: eth5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
    link/ether 00:1c:c4:48:35:f6 brd ff:ff:ff:ff:ff:ff
8: br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
    link/ether 66:f4:32:9f:12:df brd ff:ff:ff:ff:ff:ff

[root@C100273 iproute2-2.6.26]# cat /sys/class/net/eth4/features 0x111ba9

Thanks,
Chris
_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers

As far as I recall, if you have sysfs active and use the sysfs patch to let you configure both sysfs and network namespaces, you can only move virtual interfaces into a network namespace.

Guenter

Quoted 6 lines · Guenter Roeck

As far as I recall, if you have sysfs active and use the sysfs patch to let you configure both sysfs and network namespaces, you can only move virtual interfaces into a network namespace.

Guenter

Ah ! yes, you are right :)

The current upstream implementation allowing sysfs and netns to coexist together has one restriction, the physical network devices can not be moved if sysfs is enabled in the kernel. This is why Chris can not move the physical network device with this version of the kernel. This restriction will be set until the sysfs per namespace is fully supported.

This restriction does not exist with with the previous kernel version with the sysfs per namespace patchset.

-- Daniel

Quoted 21 lines · Daniel Lezcano

Quoted 7 lines · Guenter Roeck

As far as I recall, if you have sysfs active and use the sysfs patch to let you configure both sysfs and network namespaces, you can only move virtual interfaces into a network namespace.

Guenter

Ah ! yes, you are right :)

The current upstream implementation allowing sysfs and netns to coexist together has one restriction, the physical network devices can not be moved if sysfs is enabled in the kernel. This is why Chris can not move the physical network device with this version of the kernel. This restriction will be set until the sysfs per namespace is fully supported.

This restriction does not exist with with the previous kernel version with the sysfs per namespace patchset.

Quoted 3 lines

-- Daniel

Ah, great, thanks to all for your help on this.
Do you have any rough estimate when the support for sysfs per namespace will

Quoted 27 lines · [email protected] wrote:

Quoted 24 lines · Daniel Lezcano

Quoted 7 lines · Guenter Roeck

As far as I recall, if you have sysfs active and use the sysfs patch to let you configure both sysfs and network namespaces, you can only move virtual interfaces into a network namespace.

Guenter

Ah ! yes, you are right :)

The current upstream implementation allowing sysfs and netns to coexist together has one restriction, the physical network devices can not be moved if sysfs is enabled in the kernel. This is why Chris can not move the physical network device with this version of the kernel. This restriction will be set until the sysfs per namespace is fully supported.

This restriction does not exist with with the previous kernel version with the sysfs per namespace patchset.

-- Daniel

Quoted 3 lines

Ah, great, thanks to all for your help on this.
Do you have any rough estimate when the support for sysfs per namespace will

The sysfs per namespace has been rejected because of some design problems related with the sysfs itself.
Perhaps Eric can tell more about that...

Quoted 28 lines · Daniel Lezcano

Quoted 27 lines · [email protected] wrote:

Quoted 24 lines · Daniel Lezcano

Quoted 7 lines · Guenter Roeck

As far as I recall, if you have sysfs active and use the sysfs patch to let you configure both sysfs and network namespaces, you can only move virtual interfaces into a network namespace.

Guenter

Ah ! yes, you are right :)

The current upstream implementation allowing sysfs and netns to coexist together has one restriction, the physical network devices can not be moved if sysfs is enabled in the kernel. This is why Chris can not move the physical network device with this version of the kernel. This restriction will be set until the sysfs per namespace is fully supported.

This restriction does not exist with with the previous kernel version with the sysfs per namespace patchset.

-- Daniel

Quoted 6 lines

Quoted 3 lines

Ah, great, thanks to all for your help on this.
Do you have any rough estimate when the support for sysfs per namespace will

The sysfs per namespace has been rejected because of some design problems related with the sysfs itself.
Perhaps Eric can tell more about that...

Chris, in the meantime, is using the physical device an absolute necessity, or could you work around it for now using a veth tunnel?

Even if Eric has been working on the sysfs locking rework quietly the last few months, i'd expect several months of back-and-forth trying to prove that the rework is correct...

-serge

Quoted 42 lines · Serge E. Hallyn

Quoted 32 lines · Daniel Lezcano

Quoted 30 lines · [email protected] wrote:

Quoted 27 lines · Daniel Lezcano

Quoted 8 lines · Guenter Roeck

As far as I recall, if you have sysfs active and use the sysfs patch to let you configure both sysfs and network namespaces, you can only move virtual interfaces into a network namespace.

Guenter

Ah ! yes, you are right :)

The current upstream implementation allowing sysfs and netns to coexist together has one restriction, the physical network devices can not be moved if sysfs is enabled in the kernel. This is why Chris can not move the physical network device with this version of the kernel. This restriction will be set until the sysfs per namespace is fully supported.

This restriction does not exist with with the previous kernel version with the sysfs per namespace patchset.

-- Daniel

Quoted 8 lines

Quoted 4 lines

Ah, great, thanks to all for your help on this.
Do you have any rough estimate when the support for sysfs per namespace will

The sysfs per namespace has been rejected because of some design problems related with the sysfs itself.
Perhaps Eric can tell more about that...

Quoted 9 lines

Chris, in the meantime, is using the physical device an absolute necessity, or could you work around it for now using a veth tunnel?

Even if Eric has been working on the sysfs locking rework quietly the last few months, i'd expect several months of back-and-forth trying to prove that the rework is correct...

-serge

Yes, ultimately we'll need the physical device inside the same namespace as our application. Our application does a lot of management on the interface, monitoring things like the interface's link-pulse and such, and that wouldn't be available through a virtual interface. We can always redesign things to have the management portion run in the namespace with the physical interface, but for performance reasons we'd eventually want the physical interface to be directly inside the namespace anyway - so that would probably be wasted effort.

Quoted 14 lines

Yes, ultimately we'll need the physical device inside the same namespace as our application. Our application does a lot of management on the interface,
monitoring things like the interface's link-pulse and such, and that wouldn't
be available through a virtual interface. We can always redesign things to have the management portion run in the namespace with the physical interface,
but for performance reasons we'd eventually want the physical interface to be
directly inside the namespace anyway - so that would probably be wasted effort.

Did you tried with the macvlan ?

Quoted 14 lines

Quoted 14 lines

Yes, ultimately we'll need the physical device inside the same namespace as our application. Our application does a lot of management on the interface,
monitoring things like the interface's link-pulse and such, and that wouldn't
be available through a virtual interface. We can always redesign things to have the management portion run in the namespace with the physical interface,
but for performance reasons we'd eventually want the physical interface to be
directly inside the namespace anyway - so that would probably be wasted effort.

Quoted 2 lines

Did you tried with the macvlan ?

I just tried it - and it does work - I can move the vlan interfaces into the
namespace. But that's not a solution for us - we really need to support
communications out to other nodes on the network that we can't count on using vlan.

Quoted 10 lines · [email protected] wrote:

Quoted 2 lines

Did you tried with the macvlan ?

I just tried it - and it does work - I can move the vlan interfaces into the
namespace. But that's not a solution for us - we really need to support
communications out to other nodes on the network that we can't count on using
vlan.

Perhaps I am misunderstand but the macvlan is just an "eth aliased" interface, so you should but able to communicate with the outside world and keep the same performances as the physical device, no ? The only drawback with the macvlan is you can not communicate on the same host between different namespaces. We did some benchmark last year wih the macvlan and we got very good results.

Quoted 11 lines · Daniel Lezcano

Quoted 10 lines · [email protected] wrote:

Quoted 2 lines

Did you tried with the macvlan ?

I just tried it - and it does work - I can move the vlan interfaces into the
namespace. But that's not a solution for us - we really need to support
communications out to other nodes on the network that we can't count on using
vlan.

Quoted 5 lines

Perhaps I am misunderstand but the macvlan is just an "eth aliased" interface, so you should but able to communicate with the outside world and keep the same performances as the physical device, no ? The only drawback with the macvlan is you can not communicate on the same host between different namespaces.

We did some benchmark last year wih the

Quoted 1 line

macvlan and we got very good results.

Oh great, I confused the macvlan support with 802.1q vlan interfaces (as added through vconfig). macvlan does look very promising - I was able to create a macvlan interface and assign it to my namespace - looks pretty good!

Quoted 10 lines · [email protected] wrote:

Quoted 8 lines · Daniel Lezcano

Quoted 6 lines · Guenter Roeck

As far as I recall, if you have sysfs active and use the sysfs patch to let you configure both sysfs and network namespaces, you can only move virtual interfaces into a network namespace.

Guenter

Quoted 7 lines

Quoted 7 lines

Ah ! yes, you are right :)

The current upstream implementation allowing sysfs and netns to coexist together has one restriction, the physical network devices can not be moved if sysfs is enabled in the kernel. This is why Chris can not move the physical network device with this version of the kernel.
This restriction will be set until the sysfs per namespace is fully

supported.

Quoted 8 lines

Quoted 7 lines

This restriction does not exist with with the previous kernel version with the sysfs per namespace patchset.

-- Daniel

Quoted 3 lines

Ah, great, thanks to all for your help on this.
Do you have any rough estimate when the support for sysfs per namespace will

The sysfs per namespace has been rejected because of some design problems related with the sysfs itself.
Perhaps Eric can tell more about that...