package/systemd: hybrid cgroupfs hierarchy for docker compatibility
PATCH 1/1 package/systemd: hybrid cgroupfs hierarchy for docker compatibility
Docker fails to start with "Devices cgroup isn't mounted" as of systemd 243. According to the systemd documentation:
systemd now defaults to the "unified" cgroup hierarchy setup during build-time, i.e. -Ddefault-hierarchy=unified is now the build-time default. Previously, -Ddefault-hierarchy=hybrid was the default. [...] Downstream production distributions might want to continue to use -Ddefault-hierarchy=hybrid (or even =legacy) for their builds as unfortunately the popular container managers have not caught up with the kernel API changes.
Changing this option to "hybrid" or "legacy" fixes the Docker startup.
Reference: https://github.com/opencontainers/runc/issues/654
--- package/systemd/systemd.mk | 1 + 1 file changed, 1 insertion(+)
package/systemd/systemd.mk +1 -0
@@ -26,6 +26,7 @@ SYSTEMD_CONF_OPTS += \ -Dima=false \ -Dldconfig=false \ -Ddefault-dnssec=no \ + -Ddefault-hierarchy=hybrid \ -Dtests=false \ -Dsplit-bin=true \ -Dsplit-usr=false \
Signature · 1 line
2.24.0
PATCH 1/1 package/systemd: hybrid cgroupfs hierarchy for docker compatibility
Quoted 25 lines · Christian Stewart
Docker fails to start with "Devices cgroup isn't mounted" as of systemd 243. According to the systemd documentation:
systemd now defaults to the "unified" cgroup hierarchy setup during build-time, i.e. -Ddefault-hierarchy=unified is now the build-time default. Previously, -Ddefault-hierarchy=hybrid was the default. [...] Downstream production distributions might want to continue to use -Ddefault-hierarchy=hybrid (or even =legacy) for their builds as unfortunately the popular container managers have not caught up with the kernel API changes.
Changing this option to "hybrid" or "legacy" fixes the Docker startup.
Reference: https://github.com/opencontainers/runc/issues/654
Signed-off-by Christian Stewart <[email protected]>--- package/systemd/systemd.mk | 1 + 1 file changed, 1 insertion(+)package/systemd/systemd.mk +0
-0
package/systemd/systemd.mk SYSTEMD_CONF_OPTS += \ · line 29
Quoted 4 lines
package/systemd/systemd.mk +1
-0@@ -26,6 +26,7 @@ SYSTEMD_CONF_OPTS += \ -Dima=false \ -Dldconfig=false \ -Ddefault-dnssec=no \ + -Ddefault-hierarchy=hybrid \
This should probably be conditionally enabled only when packages that don't support the unified cgroup hierarchy are enabled.
Quoted 6 lines
package/systemd/systemd.mk +0
-1-Dtests=false \ -Dsplit-bin=true \ -Dsplit-usr=false \ --2.24.0
PATCH 1/1 package/systemd: hybrid cgroupfs hierarchy for docker compatibility
Keeping a list like that is going to be tricky...
Let's just hope the pressure on docker is strong enough and in a version or two we can move to unified
also:
Le lun. 11 nov. 2019 ? 10:05, James Hilliard <[email protected]> a ?crit :
Quoted 45 lines
Quoted 2 lines · Christian Stewart
Docker fails to start with "Devices cgroup isn't mounted" as of systemd
243.
Quoted 4 lines
According to the systemd documentation:
systemd now defaults to the "unified" cgroup hierarchy setup during build-time, i.e. -Ddefault-hierarchy=unified is now the build-time
default.
Quoted 1 line
Previously, -Ddefault-hierarchy=hybrid was the default. [...]
Downstream
Quoted 2 lines
production distributions might want to continue to use -Ddefault-hierarchy=hybrid (or even =legacy) for their builds as
unfortunately
Quoted 1 line
the popular container managers have not caught up with the kernel API
changes.
Quoted 15 lines
Changing this option to "hybrid" or "legacy" fixes the Docker startup.
Reference: https://github.com/opencontainers/runc/issues/654
Signed-off-by Christian Stewart <[email protected]>--- package/systemd/systemd.mk | 1 + 1 file changed, 1 insertion(+)package/systemd/systemd.mk +0
-0package/systemd/systemd.mk SYSTEMD_CONF_OPTS += \ · line 29
Quoted 4 lines
package/systemd/systemd.mk +1
-0@@ -26,6 +26,7 @@ SYSTEMD_CONF_OPTS += \ -Dima=false \ -Dldconfig=false \ -Ddefault-dnssec=no \ + -Ddefault-hierarchy=hybrid \This should probably be conditionally enabled only when packages that don't support the unified cgroup hierarchy are enabled.
Quoted 6 lines
package/systemd/systemd.mk +0
-1-Dtests=false \ -Dsplit-bin=true \ -Dsplit-usr=false \ --2.24.0
Signature · 21 lines
[image: SMILE] <http://www.smile.eu/> 20 rue des Jardins 92600 Asni?res-sur-Seine *J?r?my ROSEN* Architecte technique [image: email] jeremy.rosen at smile.fr [image: phone] +33 6 88 25 87 42 [image: url] http://www.smile.eu [image: Twitter] <https://twitter.com/GroupeSmile> [image: Facebook] <https://www.facebook.com/smileopensource> [image: LinkedIn] <https://www.linkedin.com/company/smile> [image: Github] <https://github.com/Smile-SA> [image: D?couvrez l?univers Smile, rendez-vous sur smile.eu] <https://www.smile.eu/fr/publications/livres-blancs/yocto?utm_source=signature&utm_medium=email&utm_campaign=signature> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20191111/5ce8548b/attachment.html>
PATCH 1/1 package/systemd: hybrid cgroupfs hierarchy for docker compatibility
Hello,
Quoted 1 line · On Mon, 11 Nov 2019 11:09:27 +0100 J?r?my ROSEN <[email protected]> wrote:
Keeping a list like that is going to be tricky...
You don't really have to keep a list. You can have a blind option:
config BR2_PACKAGE_SYSTEMD_NEEDS_HYBRID_HIERARCHY
bool
systemd.mk would observe this boolean to enable (or not) the hybrid hierarchy, and packages that need it would select this boolean.
That being said, is there any drawback in unconditionally using the hybrid hierarchy? James, since you suggested to make it conditional, do you have some arguments/feedback?
Thanks,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 1/1 package/systemd: hybrid cgroupfs hierarchy for docker compatibility
It's about what implementation of cgroup is used by systemd to maintain the cgroup-tree of processes
* legacy is v1
* hybrid is both v1 and v2
* unified is v2 only
(note that only the default is changed, this can be changed via the kernel command-line at boot time)
v1 is deprecated on the kernel side because it was... a mess. The systemd people are migrating stuff from v1 to v2 as new controllers are added to v2, but at this point there is little left that is v1 only thus the change of the default value
Very few user-space projects care about cgroups : mainly systemd and container-managers. Container-manager need to interact with systemd as described in
https://systemd.io/CGROUP_DELEGATION.html which allows to drop v1 and to correctly spread the responsibility between systemd and the container managers without
stepping on each other's toes.
Also note that Fedora31 will use the unified cgroup hierarchy. A good summary of adoption status :
https://medium.com/nttlabs/cgroup-v2-596d035be4d7
At that point, I think staying in hybrid mode is the safest bet :
everything in user-space hasn't migrated yet.
Let Fedora take the fire and force everybody to migrate, and let's revisit the subject once (at least) Docker has migrated...
Le lun. 11 nov. 2019 ? 15:39, Thomas Petazzoni <[email protected]> a ?crit :
Quoted 27 lines
Hello,
Quoted 1 line · On Mon, 11 Nov 2019 11:09:27 +0100 J?r?my ROSEN <[email protected]> wrote:
Keeping a list like that is going to be tricky...
You don't really have to keep a list. You can have a blind option:
config
BR2_PACKAGE_SYSTEMD_NEEDS_HYBRID_HIERARCHY
boolsystemd.mk would observe this boolean to enable (or not) the hybrid hierarchy, and packages that need it would select this boolean.
That being said, is there any drawback in unconditionally using the hybrid hierarchy? James, since you suggested to make it conditional, do you have some arguments/feedback?
Thanks,
Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
Signature · 21 lines
[image: SMILE] <http://www.smile.eu/> 20 rue des Jardins 92600 Asni?res-sur-Seine *J?r?my ROSEN* Architecte technique [image: email] jeremy.rosen at smile.fr [image: phone] +33 6 88 25 87 42 [image: url] http://www.smile.eu [image: Twitter] <https://twitter.com/GroupeSmile> [image: Facebook] <https://www.facebook.com/smileopensource> [image: LinkedIn] <https://www.linkedin.com/company/smile> [image: Github] <https://github.com/Smile-SA> [image: D?couvrez l?univers Smile, rendez-vous sur smile.eu] <https://www.smile.eu/fr/publications/livres-blancs/yocto?utm_source=signature&utm_medium=email&utm_campaign=signature> -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20191111/9acb432c/attachment.html>
PATCH 1/1 package/systemd: hybrid cgroupfs hierarchy for docker compatibility
Hello,
(If possible, you should bottom post instead of top post. Thanks!)
Quoted 25 lines · On Mon, 11 Nov 2019 15:56:04 +0100 J?r?my ROSEN <[email protected]> wrote:
It's about what implementation of cgroup is used by systemd to maintain the cgroup-tree of processes
* legacy is v1
* hybrid is both v1 and v2
* unified is v2 only
(note that only the default is changed, this can be changed via the kernel command-line at boot time)v1 is deprecated on the kernel side because it was... a mess. The systemd people are migrating stuff from v1 to v2 as new controllers are added to v2, but at this point there is little left that is v1 only thus the change of the default value
Very few user-space projects care about cgroups : mainly systemd and container-managers. Container-manager need to interact with systemd as described in
https://systemd.io/CGROUP_DELEGATION.html which allows to drop v1 and to correctly spread the responsibility between systemd and the container managers without
stepping on each other's toes.
Also note that Fedora31 will use the unified cgroup hierarchy. A good summary of adoption status :
https://medium.com/nttlabs/cgroup-v2-596d035be4d7
Quoted 4 lines
At that point, I think staying in hybrid mode is the safest bet :
everything in user-space hasn't migrated yet.
Let Fedora take the fire and force everybody to migrate, and let's revisit the subject once (at least) Docker has migrated...
OK, thanks a lot for this explanation, much clearer!
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 1/1 package/systemd: hybrid cgroupfs hierarchy for docker compatibility
In reply to Christian StewartQuoted 10 lines · On Mon, 11 Nov 2019 00:33:48 -0800 Christian Stewart <[email protected]> wrote:
Docker fails to start with "Devices cgroup isn't mounted" as of systemd 243. According to the systemd documentation:
systemd now defaults to the "unified" cgroup hierarchy setup during build-time, i.e. -Ddefault-hierarchy=unified is now the build-time default. Previously, -Ddefault-hierarchy=hybrid was the default. [...] Downstream production distributions might want to continue to use -Ddefault-hierarchy=hybrid (or even =legacy) for their builds as unfortunately the popular container managers have not caught up with the kernel API changes.
Quoted 8 lines
Changing this option to "hybrid" or "legacy" fixes the Docker startup.
Reference: https://github.com/opencontainers/runc/issues/654
Signed-off-by Christian Stewart <[email protected]>--- package/systemd/systemd.mk | 1 + 1 file changed, 1 insertion(+)
Applied to master, thanks. It would be nice to have some runtime tests around Docker, to verify that it is still working properly at runtime, so that we can catch this kind of issues.
Thanks!
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 1/1 package/systemd: hybrid cgroupfs hierarchy for docker compatibility
Hi all,
(re-sending this from the correct email address)
Quoted 2 lines · Jeremy ROSEN
v1 is deprecated on the kernel side because it was... a mess. The systemd people are migrating stuff from v1 to v2 as new controllers a=
re
Quoted 2 lines
added to v2, but at this point there is little left that is v1 only thus the change of the default value
Quoted 6 lines · Thomas Petazzoni
Quoted 6 lines
systemd.mk would observe this boolean to enable (or not) the hybrid hierarchy, and packages that need it would select this boolean.
That being said, is there any drawback in unconditionally using the hybrid hierarchy? James, since you suggested to make it conditional, do you have some arguments/feedback?
I originally planned to submit this patch with a 3-state option for which mode is desired (legacy, hybrid, merged) and then having Docker and other packages depend on hybrid or legacy.
However, I realized that this would be quite a lot of work to then trace down all of the packages that potentially need the hybrid configuration, and that in the name of "not breaking userspace" it makes sense for Buildroot to follow the direction in the systemd NEWS and just default to "hybrid" for now.
If someone wants to do the above work to make the 3-state option, we could merge that first, with "hybrid" as the default, and later add dependencies from Docker -> systemd for that option.
Quoted 4 lines
At that point, I think staying in hybrid mode is the safest bet :
everything in user-space hasn't migrated yet.
Let Fedora take the fire and force everybody to migrate, and let's revisit the subject once (at least) Docker has migrated...
Agreed, we can probably wait until Docker has finished their migration to begin seriously considering moving off of "hybrid."
Best regards,
Christian
Quoted 31 lines · Thomas Petazzoni
Quoted 1 line · On Mon, 11 Nov 2019 00:33:48 -0800 Christian Stewart <[email protected]> wrote:
Docker fails to start with "Devices cgroup isn't mounted" as of systemd
243.
Quoted 4 lines
According to the systemd documentation:
systemd now defaults to the "unified" cgroup hierarchy setup during build-time, i.e. -Ddefault-hierarchy=unified is now the build-time
default.
Quoted 1 line
Previously, -Ddefault-hierarchy=hybrid was the default. [...]
Downstream
Quoted 2 lines
production distributions might want to continue to use -Ddefault-hierarchy=hybrid (or even =legacy) for their builds as
unfortunately
Quoted 1 line
the popular container managers have not caught up with the kernel API
changes.
Quoted 9 lines
Changing this option to "hybrid" or "legacy" fixes the Docker startup.
Reference: https://github.com/opencontainers/runc/issues/654
Signed-off-by Christian Stewart <[email protected]>--- package/systemd/systemd.mk | 1 + 1 file changed, 1 insertion(+)Applied to master, thanks. It would be nice to have some runtime tests around Docker, to verify that it is still working properly at runtime, so that we can catch this kind of issues.
Quoted 8 lines
Thanks!
Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20191111/4c017e02/attachment.html>