containers and cgroups mini-summit @ Linux Plumbers
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31). The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management, checkpoint-restore and so on.
We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and let me know if you are going to come.
I probably forgot a few more people (such as, I am not sure who else from Google is working on cgroups stuff), so fill free to forward this to anyone you believe should go, or just let me know whom I missed.
Regards,
Kir.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 11 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.
I won't be there. I hope it'll be possible to call in remotely one way or another.
Topics I'm most interested in right now, beside completion of user namespace (if anything needs to be discussed there), is syslog namespace and what to do about loop devices and udev/uevents (devices namespace?). Checkpoint/restart is moving along nicely, perhaps only a progress update and discussion of any limitations?
Quoted 8 lines
I probably forgot a few more people (such as, I am not sure who else from Google is working
on cgroups stuff), so fill free to forward this to anyone you believe should go,
or just let me know whom I missed.Regards,
Kir.
Quoted 11 lines · Serge Hallyn
Quoted 10 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.
We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.
Quoted 2 lines
I won't be there. I hope it'll be possible to call in remotely one way or another.
We will try to set up something (Skype? Google+ Hangout? A conventional phone line from event organizers?) but from some previous experience it can't be very successful if there are more than about 10 people in the room.
Quoted 5 lines
Topics I'm most interested in right now, beside completion of user namespace (if anything needs to be discussed there), is syslog namespace and what to do about loop devices and udev/uevents (devices namespace?). Checkpoint/restart is moving along nicely, perhaps only a progress update and discussion of any limitations?
Thanks, will add it to the list.
Quoted 9 lines
Quoted 9 lines
I probably forgot a few more people (such as, I am not sure who else from Google is working
on cgroups stuff), so fill free to forward this to anyone you believe should go,
or just let me know whom I missed.Regards,
Kir.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 12 lines · Kir Kolyshkin
Quoted 11 lines · Serge Hallyn
Quoted 10 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.
We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.
Quoted 8 lines
Quoted 2 lines
I won't be there. I hope it'll be possible to call in remotely one way or another.
We will try to set up something (Skype? Google+ Hangout? A conventional phone line
from event organizers?) but from some previous experience it can't be very successful
if there are more than about 10 people in the room.
Yeah, I remember we tried it in 2007 or so...
hangouts crash every computer I own. I'm fine with skype or a phone line.
Quoted 7 lines
Quoted 5 lines
Topics I'm most interested in right now, beside completion of user namespace (if anything needs to be discussed there), is syslog namespace and what to do about loop devices and udev/uevents (devices namespace?). Checkpoint/restart is moving along nicely, perhaps only a progress update and discussion of any limitations?
Thanks, will add it to the list.
Great, thanks.
Quoted 10 lines
Quoted 8 lines
Quoted 8 lines
I probably forgot a few more people (such as, I am not sure who else from Google is working
on cgroups stuff), so fill free to forward this to anyone you believe should go,
or just let me know whom I missed.Regards,
Kir.
-serge
Quoted 11 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.
I wish I could, but this year's LPC date conflict with my holidays.
Sorry.
Thanks.
Quoted 9 lines
I probably forgot a few more people (such as, I am not sure who else from Google is working
on cgroups stuff), so fill free to forward this to anyone you believe should go,
or just let me know whom I missed.Regards,
Kir.
Hi,
Quoted 8 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.
Quoted 4 lines
We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.
I will be at LPC, so I should be able to attend.
Quoted 5 lines
I probably forgot a few more people (such as, I am not sure who else from Google is working
on cgroups stuff), so fill free to forward this to anyone you believe should go,
or just let me know whom I missed.
I would be interested mainly in discussing how cgroups core is changing, imposing a set of rules on how new subsystems are being designed (hierarchical vs non-hierarchical, and all that mess).
Dhaval
Quoted 12 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.
Quoted 5 lines
I probably forgot a few more people (such as, I am not sure who else from Google is working
on cgroups stuff), so fill free to forward this to anyone you believe should go,
or just let me know whom I missed.
I can be and would love to be there if I get an invite and get to know in advance. I'll need to work out my travel plans/budget to see if I can get an approval. I think there are other interesting cases for containers especially on new generation, new form factor devices.
Thanks,
Balbir
A message that is not in the archive (sent off-list or not yet received): CAKTCnzmhWZa=_+EE9rA63NuJf3kQ-bXNYStXmZXt8r2w-N02ag-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Quoted 19 lines · Balbir Singh
Quoted 12 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.Quoted 5 lines
I probably forgot a few more people (such as, I am not sure who else from Google is working
on cgroups stuff), so fill free to forward this to anyone you believe should go,
or just let me know whom I missed.
Quoted 4 lines
I can be and would love to be there if I get an invite and get to know in advance. I'll need to work out my travel plans/budget to see if I can get an approval. I think there are other interesting cases for containers especially on new generation, new form factor devices.
You don't need an invite: This is Plumbers, not the kernel summit. Plumbers Mini Summits/Micro Conferences are open to all registered Plumbers attendees.
James
Quoted 12 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.
Quoted 5 lines
I probably forgot a few more people (such as, I am not sure who else from Google is working
on cgroups stuff), so fill free to forward this to anyone you believe should go,
or just let me know whom I missed.
Hi Kir,
I have a presentation for LPC and I am awaiting the approval for the funding. If it is accepted I will be there.
One point to address could be the time virtualization.
Thanks
-- Daniel
Signature · 5 lines
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook | <http://twitter.com/#!/linaroorg> Twitter | <http://www.linaro.org/linaro-blog/> Blog
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 19 lines · Daniel Lezcano
Quoted 12 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.Quoted 5 lines
I probably forgot a few more people (such as, I am not sure who else from Google is working
on cgroups stuff), so fill free to forward this to anyone you believe should go,
or just let me know whom I missed.
Quoted 7 lines
Hi Kir,
I have a presentation for LPC and I am awaiting the approval for the funding. If it is accepted I will be there.
One point to address could be the time virtualization.
What exactly do you have in mind for that ?
Hello,
I think I can be there.
Thanks.
Signature · 1 line
tejun
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 8 lines · Tejun Heo
Hello,
I think I can be there.
Thanks.
--
tejun
Likewise,
- Paul
Quoted 11 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.
I'll be there.
Quoted 9 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31). The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management, checkpoint-restore and so on.
We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and let me know if you are going to come.
I think I'll be there.
-Kame
Quoted 12 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.
Quoted 8 lines
I probably forgot a few more people (such as, I am not sure who else from Google is working
on cgroups stuff), so fill free to forward this to anyone you believe should go,
or just let me know whom I missed.Regards,
Kir.
BTW, sorry for not replying before (vacations + post-vacations laziness)
I would be interested in adding /proc virtualization to the discussion. By now it seems userspace would be the best place for that to happen, in a fuse overlay. I know Daniel has an initial implementation of that, and it would be good to have it as library that both OpenVZ and LXC (and whoever else wants) can use.
Shouldn't take much time...
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 24 lines · Glauber Costa
Quoted 12 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.Quoted 8 lines
I probably forgot a few more people (such as, I am not sure who else from Google is working
on cgroups stuff), so fill free to forward this to anyone you believe should go,
or just let me know whom I missed.Regards,
Kir.BTW, sorry for not replying before (vacations + post-vacations laziness)
Quoted 7 lines
I would be interested in adding /proc virtualization to the discussion. By now it seems userspace would be the best place for that to happen, in a fuse overlay. I know Daniel has an initial implementation of that, and it would be good to have it as library that both OpenVZ and LXC (and whoever else wants) can use.
Shouldn't take much time...
What would you need proc virtualization for?
Eric
A message that is not in the archive (sent off-list or not yet received): 871uk0b1p4.fsf-aS9lmoZGLiVWk0Htik3J/[email protected]
Quoted 32 lines · Eric W. Biederman
Quoted 30 lines · Glauber Costa
Quoted 12 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.Quoted 8 lines
I probably forgot a few more people (such as, I am not sure who else from Google is working
on cgroups stuff), so fill free to forward this to anyone you believe should go,
or just let me know whom I missed.Regards,
Kir.BTW, sorry for not replying before (vacations + post-vacations laziness)
I would be interested in adding /proc virtualization to the discussion. By now it seems userspace would be the best place for that to happen, in a fuse overlay. I know Daniel has an initial implementation of that, and it would be good to have it as library that both OpenVZ and LXC (and whoever else wants) can use.
Quoted 4 lines
Quoted 1 line
Shouldn't take much time...
What would you need proc virtualization for?
proc provides a lot of information that userspace tools rely upon. For instance, when running top, you can draw per-process figures from what we have now, but you can't make sense of percentages without aggregating container-wide information.
When you read /proc/cpuinfo, as well, you would expect to see something that matches your container configuration.
"free" is another example. The list go on.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 43 lines · Glauber Costa
Quoted 32 lines · Eric W. Biederman
Quoted 30 lines · Glauber Costa
Quoted 12 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.Quoted 8 lines
I probably forgot a few more people (such as, I am not sure who else from Google is working
on cgroups stuff), so fill free to forward this to anyone you believe should go,
or just let me know whom I missed.Regards,
Kir.BTW, sorry for not replying before (vacations + post-vacations laziness)
I would be interested in adding /proc virtualization to the discussion. By now it seems userspace would be the best place for that to happen, in a fuse overlay. I know Daniel has an initial implementation of that, and it would be good to have it as library that both OpenVZ and LXC (and whoever else wants) can use.
Quoted 4 lines
Quoted 1 line
Shouldn't take much time...
What would you need proc virtualization for?
proc provides a lot of information that userspace tools rely upon. For instance, when running top, you can draw per-process figures from what we have now, but you can't make sense of percentages without aggregating container-wide information.
Quoted 6 lines
When you read
/proc/cpuinfo, as well, you would expect to see something that matches your container configuration."free" is another example. The list go on.
Also, Eric: Will you be around LPC? Would you be willing to hold a session about pid/user namespaces ?
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 2 lines · Glauber Costa
Also, Eric: Will you be around LPC? Would you be willing to hold a session about pid/user namespaces ?
Tenatively. My plans are not fixed in stone but I have some intention of figuring that out.
Eric
Quoted 43 lines · Glauber Costa
Quoted 32 lines · Eric W. Biederman
Quoted 30 lines · Glauber Costa
Quoted 12 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.Quoted 8 lines
I probably forgot a few more people (such as, I am not sure who else from Google is working
on cgroups stuff), so fill free to forward this to anyone you believe should go,
or just let me know whom I missed.Regards,
Kir.BTW, sorry for not replying before (vacations + post-vacations laziness)
I would be interested in adding /proc virtualization to the discussion. By now it seems userspace would be the best place for that to happen, in a fuse overlay. I know Daniel has an initial implementation of that, and it would be good to have it as library that both OpenVZ and LXC (and whoever else wants) can use.
Quoted 4 lines
Quoted 1 line
Shouldn't take much time...
What would you need proc virtualization for?
proc provides a lot of information that userspace tools rely upon. For instance, when running top, you can draw per-process figures from what we have now, but you can't make sense of percentages without aggregating container-wide information.
Quoted 4 lines
When you read
/proc/cpuinfo, as well, you would expect to see something that matches your container configuration."free" is another example. The list go on.
Another interesting feature IMHO would be the per-cgroup loadavg. A typical use case could be a monitoring system that wants to know which containers are more overloaded than others, instead of using a single system-wide measure in /proc/loadavg.
-Andrea
Quoted 12 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.
Quoted 8 lines
I probably forgot a few more people (such as, I am not sure who else from Google is working
on cgroups stuff), so fill free to forward this to anyone you believe should go,
or just let me know whom I missed.Regards,
Kir.
I just came up with the following preliminary list of sessions:
http://wiki.linuxplumbersconf.org/2012:containers
Since people mostly said what they wanted to talk about, but without extensive descriptions, I took the liberty of coming up with a small text for each in the blueprints. If you believe this is inaccurate, or would like to see it extended (although I personally don't see the point about going into very formal and deep details here), just let me know and I will edit it.
This is all still subject to change.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 19 lines · Glauber Costa
Quoted 18 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.I probably forgot a few more people (such as, I am not sure who else from Google is working
on cgroups stuff), so fill free to forward this to anyone you believe should go,
or just let me know whom I missed.
Quoted 6 lines
Quoted 2 lines
Regards,
Kir.I just came up with the following preliminary list of sessions:
Thanks. We could also add devices namespace to the list. I'm fine NOT having it in the list, as I think we're agreed that it should come after user namespaces are complete.
Quoted 23 lines · Serge Hallyn
Quoted 22 lines · Glauber Costa
Quoted 20 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.I probably forgot a few more people (such as, I am not sure who else from Google is working
on cgroups stuff), so fill free to forward this to anyone you believe should go,
or just let me know whom I missed.Regards,
Kir.
Quoted 8 lines
Quoted 3 lines
I just came up with the following preliminary list of sessions:
Thanks. We could also add devices namespace to the list. I'm fine NOT having it in the list, as I think we're agreed that it should come after user namespaces are complete.
How about a "future of namespaces" generic placeholder?
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 26 lines · Glauber Costa
Quoted 25 lines · Serge Hallyn
Quoted 24 lines · Glauber Costa
Quoted 12 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.Quoted 8 lines
I probably forgot a few more people (such as, I am not sure who else from Google is working
on cgroups stuff), so fill free to forward this to anyone you believe should go,
or just let me know whom I missed.Regards,
Kir.I just came up with the following preliminary list of sessions:
Quoted 7 lines
Quoted 6 lines
Quoted 1 line
Thanks. We could also add devices namespace to the list. I'm fine NOT having it in the list, as I think we're agreed that it should come after user namespaces are complete.
How about a "future of namespaces" generic placeholder?
Sure. And I suppose syslog and time ns could also fall under that. Issues to discuss include:
1. what other namespaces do people expect to need?
2. justifications for the proposed namespaces, and other potential ways to address each. I.e., for devices ns, two justifications are (1) per-container loop devices (could be pre-allocated with containers aware of which /dev/loopN index they may use), and (2) filtering uevents only to appropriate containers to limit the uevent storm when containers do udevadm trigger --action=add (not sure, but we can brainstorm other solutions).
3. semantics for the namespace - I.e syslog namespace, how do we decide whether/when to unshare it, is it owned by a user namespace or a pid namespace, etc.
Quoted 4 lines · Glauber Costa
I just came up with the following preliminary list of sessions:
Quoted 8 lines
Since people mostly said what they wanted to talk about, but without extensive descriptions, I took the liberty of coming up with a small text for each in the blueprints. If you believe this is inaccurate, or would like to see it extended (although I personally don't see the point about going into very formal and deep details here), just let me know and I will edit it.
This is all still subject to change.
Something that just came up recently and worth looking at if it hasn't already be resolved.
The network namespace, the user namespace, and the memory control group are not meshing well.
In particular we need some additional checks for an unprivileged user who can set tcp_mem. If you are the creator of a network namespace you should at least be able to set the values down. I don't know at all about increasing the amount of memory consumed by the tcp stack.
The non-nesting nature of memory control groups with respect to the network stack also seems very bizarre.
Another old issue is that unless I have missed something control groups are still broken for generic use in containers. Does anyone care? Are there any plans on fixing this issue?
Eric
A message that is not in the archive (sent off-list or not yet received): 877gtr6uo5.fsf-aS9lmoZGLiVWk0Htik3J/[email protected]
Quoted 18 lines · Eric W. Biederman
Quoted 4 lines · Glauber Costa
I just came up with the following preliminary list of sessions:
Quoted 8 lines
Since people mostly said what they wanted to talk about, but without extensive descriptions, I took the liberty of coming up with a small text for each in the blueprints. If you believe this is inaccurate, or would like to see it extended (although I personally don't see the point about going into very formal and deep details here), just let me know and I will edit it.
This is all still subject to change.
Something that just came up recently and worth looking at if it hasn't already be resolved.
Quoted 7 lines
The network namespace, the user namespace, and the memory control group are not meshing well.
In particular we need some additional checks for an unprivileged user who can set
tcp_mem. If you are the creator of a network namespace you should at least be able to set the values down. I don't know at all about increasing the amount of memory consumed by the tcp stack.
This is between the user namespace and net namespace only, right ?
To be quite honest, I haven't looked thoroughly at UNS after your last work. How do you yourself believe this should be?
Quoted 3 lines
The non-nesting nature of memory control groups with respect to the network stack also seems very bizarre.
Correction:
The non-nesting nature of memory control groups is very bizarre. No need for modifiers. It does support nesting, though. Just that it is selectable, and not the default. But there is work in progress to change that.
Quoted 7 lines
Another old issue is that unless I have missed something control groups are still broken for generic use in containers. Does anyone care? Are there any plans on fixing this issue?
Eric
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 6 lines
Quoted 6 lines
Another old issue is that unless I have missed something control groups are still broken for generic use in containers. Does anyone care? Are there any plans on fixing this issue?
What is "generic use in containers" ? I am using them alright, but not sure if this counts as generic or specific =)
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 8 lines · Glauber Costa
Quoted 5 lines
Quoted 5 lines
Another old issue is that unless I have missed something control groups are still broken for generic use in containers. Does anyone care? Are there any plans on fixing this issue?
What is "generic use in containers" ? I am using them alright, but not sure if this counts as generic or specific =)
The general container use case would be.
- Create a new mount namespace.
- Create fresh mounts of all of the control groups like I would do at boot, with no consideration to any other control group state.
- Start forking processes.
The expected semantics would be something like chroot for control groups, where all of the control groups that are created by fresh mounts are relative to whatever state the process of being in a control group that the process that mounted them was in.
Last I looked the closest you could come to that was bind mounts, and even with bind mounts you get into weird things where control groups are bound into hierarchies and you may be running a distribution that wants it's control groups bound into different hierarchies.
Last I looked this was just about a total disaster, and the only thing that allowed systemd to run in containers was the fact that systemd did not user controllers.
Eric
A message that is not in the archive (sent off-list or not yet received): 874nou6bx1.fsf-aS9lmoZGLiVWk0Htik3J/[email protected]
Hello, Eric.
Quoted 9 lines · Eric W. Biederman
- Create a new mount namespace.
- Create fresh mounts of all of the control groups like I would do at boot, with no consideration to any other control group state.
- Start forking processes.The expected semantics would be something like chroot for control groups, where all of the control groups that are created by fresh mounts are relative to whatever state the process of being in a control group that the process that mounted them was in.
No, any attempt to build namespace support into cgroup core code will be nacked with strong prejudice. I still think it was a mistake to add that to sysfs. Thankfully, procfs is going the FUSE way and I hope in time we could convert sysfs to a similar mechanism and deprecate the in-kernel support.
So, no, no, no, no, no, no, no, no, no, no, no, no. :P
Signature · 1 line
tejun
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 8 lines · Tejun Heo
Hello, Eric.
Quoted 5 lines · Eric W. Biederman
- Create a new mount namespace.
- Create fresh mounts of all of the control groups like I would do at boot, with no consideration to any other control group state.
- Start forking processes.
Quoted 7 lines
Quoted 4 lines
The expected semantics would be something like chroot for control groups, where all of the control groups that are created by fresh mounts are relative to whatever state the process of being in a control group that the process that mounted them was in.
No, any attempt to build namespace support into cgroup core code will be nacked with strong prejudice.
The cgroup code was only merged with the understanding that this support was simple to add and it would be added. I am sorry that no one had the sense to follow up and make certain that promise was not fullfilled.
Quoted 1 line
I still think it was a mistake to add that to sysfs.
sysfs fundamentally can not represent all of the network devices in the hierarchy of objects that it chose.
sysfs does not have namespace hacks. Sysfs has hacks for the limitations of the hiearchary of devices that was choosen for a sysfs user space ABI.
Quoted 1 line
Thankfully, procfs is going the FUSE way.
No procfs is not going the FUSE way. Hacks for programs that misuse information in procfs is going the FUSE way.
The best example is there currently is not a good method for programs to figure out how parellel it is productive to be so the programs read /proc/cpuinfo and get the count of cpus. Control groups can limit you to fewer cpus but those programs have figured that out yet.
But ultimately fuse for procfs is about the rare case where people want to lie to applications, because it is easier to lie to applications then to disabuse the applications of their mistaken asumptions.
I have not seen a single suggest that any of the other procfs bits can go away.
Quoted 2 lines
and I hope in time we could convert sysfs to a similar mechanism and deprecate the in-kernel support.
I have nothing that even suggests there is a reasonable possibility of using fuse to deprecate any of the proc or sysfs support.
Quoted 1 line
So, no, no, no, no, no, no, no, no, no, no, no, no. :P
Bahahahahahahaha! :P
I sort of wish I had the energy to tackle this. As it is control groups hierarchies have very severe usablilty problems supporting one of their core use cases.
We should have our interfaces designed such that it is possible to run nested init's without hacks, and the only significant piece left on the hacks pile is control groups.
Control group hiearchies are are really strange piece of work whose design makes very little sense to me.
I think all I want from control groups is that a process that is bound into a control group hiearchy when it mounts that hiearchy will get not the normal control group root but instead the dentry of the directory for that processes place in the control group hiearchy.
What is that maybe 15-30 lines of code to look up the right dentry?
Eric
A message that is not in the archive (sent off-list or not yet received): 87a9ymwcsv.fsf-aS9lmoZGLiVWk0Htik3J/[email protected]
Hey, Eric.
Quoted 6 lines · Eric W. Biederman
Quoted 2 lines
No, any attempt to build namespace support into cgroup core code will be nacked with strong prejudice.
The cgroup code was only merged with the understanding that this support was simple to add and it would be added. I am sorry that no one had the sense to follow up and make certain that promise was not fullfilled.
Good chunk of cgroup is messy and I'm likely to continue to break a lot of whatever promises that have been made. :)
Quoted 4 lines
Quoted 1 line
Thankfully, procfs is going the FUSE way.
No procfs is not going the FUSE way. Hacks for programs that misuse information in procfs is going the FUSE way.
All those were proposed to be solved by "teaching" kernel procfs how to present itself differently.
Quoted 4 lines
The best example is there currently is not a good method for programs to figure out how parellel it is productive to be so the programs read
/proc/cpuinfoand get the count of cpus. Control groups can limit you to fewer cpus but those programs have figured that out yet.
Yeah, and you can handle that too nicely with FUSE. More on this later.
Quoted 3 lines
But ultimately fuse for procfs is about the rare case where people want to lie to applications, because it is easier to lie to applications then to disabuse the applications of their mistaken asumptions.
I don't think so. They are necessary parts of representing a properly scoped environment.
Quoted 2 lines
I have not seen a single suggest that any of the other procfs bits can go away.
I think I made that a couple times now. I definitely intend to push things that way. Or, at least, I'll bark as hard as I can against adding more namespace stuff to system pseudo filesystems.
Quoted 5 lines
Quoted 2 lines
and I hope in time we could convert sysfs to a similar mechanism and deprecate the in-kernel support.
I have nothing that even suggests there is a reasonable possibility of using fuse to deprecate any of the proc or sysfs support.
Why not? If there's some deficiency in FUSE or notification mechanisms in pseudo FSes, let's fix them.
Quoted 3 lines
Quoted 1 line
So, no, no, no, no, no, no, no, no, no, no, no, no. :P
Bahahahahahahaha! :P
:)
Quoted 10 lines
I sort of wish I had the energy to tackle this. As it is control groups hierarchies have very severe usablilty problems supporting one of their core use cases.
We should have our interfaces designed such that it is possible to run nested init's without hacks, and the only significant piece left on the hacks pile is control groups.
Control group hiearchies are are really strange piece of work whose design makes very little sense to me.
cgroupfs is riddled with confused designs but this is not it. The confusion is that namespace should play a major role in the design of system pseudo filesystems and that it can be achieved by playing peekaboo with dentries.
It obfuscates the code for niche use case - which in itself could be acceptable if that's the only / best way to achieve that - while not even being able to serve the said use case properly.
Thanks.
Signature · 1 line
tejun
Quoted 28 lines · Eric W. Biederman
Quoted 8 lines · Glauber Costa
Quoted 5 lines
Quoted 5 lines
Another old issue is that unless I have missed something control groups are still broken for generic use in containers. Does anyone care? Are there any plans on fixing this issue?
What is "generic use in containers" ? I am using them alright, but not sure if this counts as generic or specific =)
The general container use case would be.
- Create a new mount namespace.
- Create fresh mounts of all of the control groups like I would do at boot, with no consideration to any other control group state.
- Start forking processes.The expected semantics would be something like chroot for control groups, where all of the control groups that are created by fresh mounts are relative to whatever state the process of being in a control group that the process that mounted them was in.
Last I looked the closest you could come to that was bind mounts, and even with bind mounts you get into weird things where control groups are bound into hierarchies and you may be running a distribution that wants it's control groups bound into different hierarchies.
Quoted 5 lines
Last I looked this was just about a total disaster, and the only thing that allowed systemd to run in containers was the fact that systemd did not user controllers.
Eric
(Sorry, please disregard my last email :)
Yes, what we do now in ubuntu quantal is the bind mounts you mention, and only optionally (using a startup hook).
Each container is brought up in say /sys/fs/cgroup/devices/lxc/container1/container1.real, and that dir is bind-mounted under /sys/fs/cgroup/devices in the guest. The guest is not allowed to mount cgroup fs himself.
It's certainly not ideal (and in cases where cgroup allows you to raise your own limits, worthless). The 'fake cgroup root' has been mentioned before to address this. Definately worth discussing.
thanks,
-serge
Quoted 9 lines · Serge Hallyn
(Sorry, please disregard my last email :)
Yes, what we do now in ubuntu quantal is the bind mounts you mention, and only optionally (using a startup hook).
Each container is brought up in say/sys/fs/cgroup/devices/lxc/container1/container1.real, and that dir is bind-mounted under/sys/fs/cgroup/devicesin the guest. The guest is not allowed to mount cgroup fs himself.
Quoted 3 lines
It's certainly not ideal (and in cases where cgroup allows you to raise your own limits, worthless). The 'fake cgroup root' has been mentioned before to address this. Definately worth discussing.
It is going to be interesting to see how all of the unprivileged operations work when the user-namespaces start allowing unprivileged users to do things (3.7 timeframe I hope).
I can see it making things both easier and harder. I would hope not actually being root will make it easier to keep from raising your own limits.
Running some operations as non-root will catch other places off guard where people were definitely expecting nothing of the kind.
There are a couple of networking memory limits exposed through sysctl that I don't expect we want everyone changing, that I need to figure out how to separate out from the rest. A concept that hasn't existed before.
Eric
Quoted 27 lines · Glauber Costa
Quoted 18 lines · Eric W. Biederman
Quoted 4 lines · Glauber Costa
I just came up with the following preliminary list of sessions:
Quoted 8 lines
Since people mostly said what they wanted to talk about, but without extensive descriptions, I took the liberty of coming up with a small text for each in the blueprints. If you believe this is inaccurate, or would like to see it extended (although I personally don't see the point about going into very formal and deep details here), just let me know and I will edit it.
This is all still subject to change.
Something that just came up recently and worth looking at if it hasn't already be resolved.
Quoted 7 lines
The network namespace, the user namespace, and the memory control group are not meshing well.
In particular we need some additional checks for an unprivileged user who can set
tcp_mem. If you are the creator of a network namespace you should at least be able to set the values down. I don't know at all about increasing the amount of memory consumed by the tcp stack.
Quoted 4 lines
This is between the user namespace and net namespace only, right ?
To be quite honest, I haven't looked thoroughly at UNS after your last work. How do you yourself believe this should be?
I looked a little deeper and there are a few more places in the networking stack besides tcp_mem, that are setting memory limits that unprivileged users should not be able to touch.
What I would expect one of.
- A global limit accross network namespaces that the per netns limit can not allow you to escape.
- Something like rlimits where the limit can be reduced but not increased.
- capability checks that prevent anyone except the global root from changing the value (of course this has the problem that creating a fresh network namespace allows memory limit escaping).
The driving factor is that in the 3.7 time frame it will be possible to create user namespaces and network namespaces as unprivileged users. So we have to be careful that we setup the limits so that the global root on the host can set limits that are not trivially overridden.
Quoted 9 lines
Quoted 2 lines
The non-nesting nature of memory control groups with respect to the network stack also seems very bizarre.
Correction:
The non-nesting nature of memory control groups is very bizarre. No need for modifiers. It does support nesting, though. Just that it is selectable, and not the default. But there is work in progress to change that.
Which leads to another bit of fun. It is possible to create containers in containers in containers, which has interesting implications for control groups in general, and especially interesting implications for control groups that don't nest.
Eric
Quoted 26 lines · Eric W. Biederman
Quoted 4 lines · Glauber Costa
I just came up with the following preliminary list of sessions:
Quoted 8 lines
Since people mostly said what they wanted to talk about, but without extensive descriptions, I took the liberty of coming up with a small text for each in the blueprints. If you believe this is inaccurate, or would like to see it extended (although I personally don't see the point about going into very formal and deep details here), just let me know and I will edit it.
This is all still subject to change.
Something that just came up recently and worth looking at if it hasn't already be resolved.
The network namespace, the user namespace, and the memory control group are not meshing well.
In particular we need some additional checks for an unprivileged user who can set
tcp_mem. If you are the creator of a network namespace you should at least be able to set the values down. I don't know at all about increasing the amount of memory consumed by the tcp stack.
Quoted 7 lines
The non-nesting nature of memory control groups with respect to the network stack also seems very bizarre.
Another old issue is that unless I have missed something control groups are still broken for generic use in containers. Does anyone care? Are there any plans on fixing this issue?
Can you elaborate? Which specific breakages are you thinking of?
Hi,
Quoted 11 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.
I'd like to give a short presentation on what we would like to do with cgroups and ConnMan including a demonstration. The demo will show what will happpen when orthogonal cgroup hierarchies are getting used by more than just systemd. Also I'd like to trigger some discussion on the future of the networking controllers (net_prio, net_cls).
cheers,
daniel
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 15 lines · Daniel Wagner
Hi,
Quoted 11 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.
Quoted 6 lines
I'd like to give a short presentation on what we would like to do with cgroups and ConnMan including a demonstration. The demo will show what will happpen when orthogonal cgroup hierarchies are getting used by more than just systemd. Also I'd like to trigger some discussion on the future of the networking controllers (
net_prio,net_cls).
Daniel, could you fill up the details of this proposal in the plumbers system? I believe kir sent out the details on how to do it somewhere down this thread...
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hi Glauber,
Quoted 23 lines · Glauber Costa
Quoted 15 lines · Daniel Wagner
Hi,
Quoted 11 lines · Kir Kolyshkin
Gentlemen,
We are organizing containers mini-summit during next Linux Plumbers (San Diego, August 29-31).
The idea is to gather and discuss everything relevant to namespaces, cgroups, resource management,
checkpoint-restore and so on.We are trying to come up with a list of topics to discuss, so please reply with topic suggestions, and
let me know if you are going to come.Quoted 6 lines
I'd like to give a short presentation on what we would like to do with cgroups and ConnMan including a demonstration. The demo will show what will happpen when orthogonal cgroup hierarchies are getting used by more than just systemd. Also I'd like to trigger some discussion on the future of the networking controllers (
net_prio,net_cls).
Quoted 3 lines
Daniel, could you fill up the details of this proposal in the plumbers system? I believe kir sent out the details on how to do it somewhere down this thread...
Thanks, just added this one here:
https://blueprints.launchpad.net/lpc/+spec/lpc2012-cont-network
cheers,
daniel