Virtualizing /proc/sys/kernel/random/boot_id per container ?
One of the features that SystemD folks have asked us to fix in LXC, is to make sure that /proc/sys/kernel/random/boot_id changes each time a container is started.
The current semantics are that this file produces a new random UUID each time the host OS is booted. Obviously each time we start a container now, they just see the host's random boot_id, so from a container's POV this does not change each time it starts.
There seems to be general agreement that, aside from the PID directories, changes to data in proc should be done by a FUSE filesystem overlay of some kind. We could use that mechanism to fix 'boot_id' in userspace, but I'm wondering if this is a better candidate for dealing with in kernel space, since as well as the /proc/sys tree, the data is also visible via the sysctl() system call which a FUSE overlay won't address.
The kernel doesn't have a real concept of a 'container' to associate a boot_id value with as such, but maybe it is reasonable to associate a boot_id value with each PID namespace ?
Regards,
Daniel
Signature · 4 lines
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 3 lines · Daniel P. Berrange
One of the features that SystemD folks have asked us to fix in LXC, is to make sure that
/proc/sys/kernel/random/boot_idchanges each time a container is started.
There may be a good reason for this. Most of the time what I have seen of kernel requests from the direction of SystemD is that while there may be a real problem but usually their imagined solution is not a particularly good solution. So a description of the problem is needed.
Justifying something with just SystemD wants this is a good way to get a nack.
Quoted 4 lines
The current semantics are that this file produces a new random UUID each time the host OS is booted. Obviously each time we start a container now, they just see the host's random
boot_id, so from a container's POV this does not change each time it starts.
That is correct. As I recall the contract with boot_id is to provide a unique per boot value to assist in dealing with boots etc. I seem to recall emacs uses the combination of hostname+boot_id to help generate unique lock files names.
I would definitely need a refresher on how boot_id is used in practice by applications other than SystemD before I could suggest a good design.
There is also a question of uptime.
Quoted 3 lines
There seems to be general agreement that, aside from the PID directories, changes to data in proc should be done by a FUSE filesystem overlay of some kind.
No. I have yet to see a justification for using FUSE in containers on top of proc files.
I have seen a lot of bad ideas suggested like hacking /proc/cpuinfo instead of providing a proper mechanism to tell applications how parallel they can/should be.
For hacks and controversial ideas FUSE is good because it makes it someone else's problem and it means it isn't something we have to support in the kernel for the indefinite future. At the same time in general a FUSE solution does not really solve anything it just sort of papers over a problem.
For some problems papering over them is good enough, for other problems they really should be solved properly.
Quoted 4 lines
We could use that mechanism to fix '
boot_id' in userspace, but I'm wondering if this is a better candidate for dealing with in kernel space, since as well as the/proc/systree, the data is also visible via thesysctl()system call which a FUSE overlay won't address.
Any application that uses the sysctl() system call needs to be fixed. When I looked years ago the number of applications using sysctl() could be numbered on one hand and most of those applications were the fedora installer, and the fedora installer hasn't used sysctl.
Quoted 3 lines
The kernel doesn't have a real concept of a 'container' to associate a
boot_idvalue with as such, but maybe it is reasonable to associate aboot_idvalue with each PID namespace ?
There is also the question of uptime and clocks and things like that.
The utsnamespace might be a more resasonable place to tack on that kind of extended functionality.
Just changing boot_id itself and not all of the other bits that track when we have booted does not seem reasonable.
Once we can sort out the details a kernel implementation should be quite trivial. It just requires the appropriate sysctl registration dance.
Eric
A message that is not in the archive (sent off-list or not yet received): 878vcwjabu.fsf-aS9lmoZGLiVWk0Htik3J/[email protected]
Quoted 6 lines · Eric W. Biederman
Quoted 3 lines · Daniel P. Berrange
One of the features that SystemD folks have asked us to fix in LXC, is to make sure that
/proc/sys/kernel/random/boot_idchanges each time a container is started.
Quoted 7 lines
There may be a good reason for this. Most of the time what I have seen of kernel requests from the direction of SystemD is that while there may be a real problem but usually their imagined solution is not a particularly good solution. So a description of the problem is needed.
Justifying something with just SystemD wants this is a good way to get a nack.
SystemD records log messages for all system services in their journal. They can show you all log messages for the current service execution, all log messages for a service since system boot, or all log messsages ever. The boot_id value is used as a unique tag to allow grouping of the log messages per system boot. When we run systemd inside a container we want to get that grouping of log messages generated by services inside the container, to take account of the container boot, not the host boot. Hence the desire to have the boot_id value reflect when a container is booted.
Quoted 10 lines
Quoted 4 lines
The current semantics are that this file produces a new random UUID each time the host OS is booted. Obviously each time we start a container now, they just see the host's random
boot_id, so from a container's POV this does not change each time it starts.That is correct. As I recall the contract with
boot_idis to provide a unique per boot value to assist in dealing with boots etc. I seem to recall emacs uses the combination of hostname+boot_id to help generate unique lock files names.
Quoted 4 lines
I would definitely need a refresher on how
boot_idis used in practice by applications other than SystemD before I could suggest a good design.There is also a question of uptime.
Agreed, as you say, this is one of many other /proc values needing virtualizing for container.
Quoted 11 lines
Quoted 3 lines
There seems to be general agreement that, aside from the PID directories, changes to data in proc should be done by a FUSE filesystem overlay of some kind.
No. I have yet to see a justification for using FUSE in containers on top of proc files.
I have seen a lot of bad ideas suggested like hacking
/proc/cpuinfoinstead of providing a proper mechanism to tell applications how parallel they can/should be.
Quoted 8 lines
For hacks and controversial ideas FUSE is good because it makes it someone else's problem and it means it isn't something we have to support in the kernel for the indefinite future. At the same time in general a FUSE solution does not really solve anything it just sort of papers over a problem.
For some problems papering over them is good enough, for other problems they really should be solved properly.
Ok, well I guess things aren't as clear cut as I understood then. I've been told that FUSE was the desired approach to dealing with all the various files in /proc that might need changing for containers. Personally I don't much care what approach is used - if the kernel wants to do more stuff that's fine with my from a libvirt LXC POV. I'll just follow whatever the consensus is in this area.
Quoted 9 lines
Quoted 4 lines
We could use that mechanism to fix '
boot_id' in userspace, but I'm wondering if this is a better candidate for dealing with in kernel space, since as well as the/proc/systree, the data is also visible via thesysctl()system call which a FUSE overlay won't address.Any application that uses the
sysctl()system call needs to be fixed. When I looked years ago the number of applications usingsysctl()could be numbered on one hand and most of those applications were the fedora installer, and the fedora installer hasn't used sysctl.
Ok, I did wonder whether anyone would actually use sysctl() instead of reading /proc/sys. If we can ignore the sysctl that gives us more options.
Quoted 6 lines
Quoted 3 lines
The kernel doesn't have a real concept of a 'container' to associate a
boot_idvalue with as such, but maybe it is reasonable to associate aboot_idvalue with each PID namespace ?There is also the question of uptime and clocks and things like that.
Quoted 8 lines
The utsnamespace might be a more resasonable place to tack on that kind of extended functionality.
Just changing
boot_iditself and not all of the other bits that track when we have booted does not seem reasonable.Once we can sort out the details a kernel implementation should be quite trivial. It just requires the appropriate sysctl registration dance.
Ok, I'll try to identify a list of other related parts which need changing wrt boot.
Thanks for the feedback.
Regards,
Daniel
Signature · 4 lines
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 15 lines · Daniel P. Berrange
Quoted 6 lines · Eric W. Biederman
Quoted 3 lines · Daniel P. Berrange
One of the features that SystemD folks have asked us to fix in LXC, is to make sure that
/proc/sys/kernel/random/boot_idchanges each time a container is started.Quoted 7 lines
There may be a good reason for this. Most of the time what I have seen of kernel requests from the direction of SystemD is that while there may be a real problem but usually their imagined solution is not a particularly good solution. So a description of the problem is needed.
Justifying something with just SystemD wants this is a good way to get a nack.
Quoted 9 lines
SystemD records log messages for all system services in their journal. They can show you all log messages for the current service execution, all log messages for a service since system boot, or all log messsages ever. The
boot_idvalue is used as a unique tag to allow grouping of the log messages per system boot. When we run systemd inside a container we want to get that grouping of log messages generated by services inside the container, to take account of the container boot, not the host boot. Hence the desire to have theboot_idvalue reflect when a container is booted.
Since SystemD post-dates containers and since the logging feature is not currently in wide use that use case is completely non-persuasive.
So far this just sounds like a plain SystemD bug and something that can be easily changed at this point in time.
It has been a long time but my fuzzy memory says that the originial boot_id justification was based on use cases that could not be solved any other way.
My memory says it was this thread https://lkml.org/lkml/1999/5/31/233
that inspired the implementation of boot_id. However reading the current emacs source code it appears emacs gave up before boot_id was implemented and stats /var/run/random-seed (which we seem to have removed) or looks in wtmp or utmp for the latest boot record.
I did a quick grep through the binaries on my system and I could not find anything using /proc/sys/random/boot_id.
That suggests to me that the proper solution is to actually just remove boot_id.
Hmm. And then there is other interesting detail. What should boot_id return after the processes have migrated from one system to another.
Quoted 10 lines
Quoted 10 lines
Quoted 4 lines
The current semantics are that this file produces a new random UUID each time the host OS is booted. Obviously each time we start a container now, they just see the host's random
boot_id, so from a container's POV this does not change each time it starts.That is correct. As I recall the contract with
boot_idis to provide a unique per boot value to assist in dealing with boots etc. I seem to recall emacs uses the combination of hostname+boot_id to help generate unique lock files names.
Quoted 7 lines
Quoted 4 lines
I would definitely need a refresher on how
boot_idis used in practice by applications other than SystemD before I could suggest a good design.There is also a question of uptime.
Agreed, as you say, this is one of many other /proc values needing virtualizing for container.
If you think of it as virtualization and you figure the requirement is to exactly replicate a non-containerized system you won't come up with suggestions that make sense to implement.
For the most part the semantics of namespaces exist to support process migration.
Quoted 20 lines
Quoted 11 lines
Quoted 3 lines
There seems to be general agreement that, aside from the PID directories, changes to data in proc should be done by a FUSE filesystem overlay of some kind.
No. I have yet to see a justification for using FUSE in containers on top of proc files.
I have seen a lot of bad ideas suggested like hacking
/proc/cpuinfoinstead of providing a proper mechanism to tell applications how parallel they can/should be.Quoted 8 lines
For hacks and controversial ideas FUSE is good because it makes it someone else's problem and it means it isn't something we have to support in the kernel for the indefinite future. At the same time in general a FUSE solution does not really solve anything it just sort of papers over a problem.
For some problems papering over them is good enough, for other problems they really should be solved properly.
Quoted 6 lines
Ok, well I guess things aren't as clear cut as I understood then. I've been told that FUSE was the desired approach to dealing with all the various files in /proc that might need changing for containers. Personally I don't much care what approach is used - if the kernel wants to do more stuff that's fine with my from a libvirt LXC POV. I'll just follow whatever the consensus is in this area.
Largely what I have seen is a bunch of half thought out hacks and the consensus being (ick don't bother me...). In which case FUSE is a good answer as it doesn't obligate anyone to maintain or care about the code, except those who want the hack.
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
We could use that mechanism to fix '
boot_id' in userspace, but I'm wondering if this is a better candidate for dealing with in kernel space, since as well as the/proc/systree, the data is also visible via thesysctl()system call which a FUSE overlay won't address.
Quoted 8 lines
Quoted 4 lines
Any application that uses the
sysctl()system call needs to be fixed. When I looked years ago the number of applications usingsysctl()could be numbered on one hand and most of those applications were the fedora installer, and the fedora installer hasn't used sysctl.Ok, I did wonder whether anyone would actually use
sysctl()instead of reading/proc/sys. If we can ignore the sysctl that gives us more options.
Most definitely. The warning sysctl spews when you figure out how to call it should be a good clue.
Quoted 12 lines
Quoted 12 lines
Quoted 3 lines
The kernel doesn't have a real concept of a 'container' to associate a
boot_idvalue with as such, but maybe it is reasonable to associate aboot_idvalue with each PID namespace ?There is also the question of uptime and clocks and things like that.
The utsnamespace might be a more resasonable place to tack on that kind of extended functionality.
Just changing
boot_iditself and not all of the other bits that track when we have booted does not seem reasonable.
Quoted 7 lines
Quoted 2 lines
Once we can sort out the details a kernel implementation should be quite trivial. It just requires the appropriate sysctl registration dance.
Ok, I'll try to identify a list of other related parts which need changing wrt boot.
Thanks for the feedback.
I hope it helps.
There may be a justification and a good case for messing with boot_id but I don't currently see it.
What I see (so far) is SystemD unnecessarily tying itself to linux implemenation details.
Eric
A message that is not in the archive (sent off-list or not yet received): 87bohrhqai.fsf-aS9lmoZGLiVWk0Htik3J/[email protected]
Quoted 43 lines · Eric W. Biederman
Quoted 15 lines · Daniel P. Berrange
Quoted 6 lines · Eric W. Biederman
Quoted 3 lines · Daniel P. Berrange
One of the features that SystemD folks have asked us to fix in LXC, is to make sure that
/proc/sys/kernel/random/boot_idchanges each time a container is started.Quoted 7 lines
There may be a good reason for this. Most of the time what I have seen of kernel requests from the direction of SystemD is that while there may be a real problem but usually their imagined solution is not a particularly good solution. So a description of the problem is needed.
Justifying something with just SystemD wants this is a good way to get a nack.
Quoted 9 lines
SystemD records log messages for all system services in their journal. They can show you all log messages for the current service execution, all log messages for a service since system boot, or all log messsages ever. The
boot_idvalue is used as a unique tag to allow grouping of the log messages per system boot. When we run systemd inside a container we want to get that grouping of log messages generated by services inside the container, to take account of the container boot, not the host boot. Hence the desire to have theboot_idvalue reflect when a container is booted.Since SystemD post-dates containers and since the logging feature is not currently in wide use that use case is completely non-persuasive.
So far this just sounds like a plain SystemD bug and something that can be easily changed at this point in time.
It has been a long time but my fuzzy memory says that the originial
boot_idjustification was based on use cases that could not be solved any other way.My memory says it was this thread https://lkml.org/lkml/1999/5/31/233
that inspired the implementation ofboot_id. However reading the current emacs source code it appears emacs gave up beforeboot_idwas implemented and stats/var/run/random-seed(which we seem to have removed) or looks in wtmp or utmp for the latest boot record.
Quoted 9 lines
I did a quick grep through the binaries on my system and I could not find anything using
/proc/sys/random/boot_id.That suggests to me that the proper solution is to actually just remove
boot_id.Hmm. And then there is other interesting detail. What should
boot_idreturn after the processes have migrated from one system to another.
Since this would be a per-boot id, this clearly has to be carried over with migration, along with all the tons of data we already carry.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 50 lines · Glauber Costa
Quoted 49 lines · Eric W. Biederman
Quoted 15 lines · Daniel P. Berrange
Quoted 6 lines · Eric W. Biederman
Quoted 3 lines · Daniel P. Berrange
One of the features that SystemD folks have asked us to fix in LXC, is to make sure that
/proc/sys/kernel/random/boot_idchanges each time a container is started.Quoted 7 lines
There may be a good reason for this. Most of the time what I have seen of kernel requests from the direction of SystemD is that while there may be a real problem but usually their imagined solution is not a particularly good solution. So a description of the problem is needed.
Justifying something with just SystemD wants this is a good way to get a nack.
Quoted 9 lines
SystemD records log messages for all system services in their journal. They can show you all log messages for the current service execution, all log messages for a service since system boot, or all log messsages ever. The
boot_idvalue is used as a unique tag to allow grouping of the log messages per system boot. When we run systemd inside a container we want to get that grouping of log messages generated by services inside the container, to take account of the container boot, not the host boot. Hence the desire to have theboot_idvalue reflect when a container is booted.Since SystemD post-dates containers and since the logging feature is not currently in wide use that use case is completely non-persuasive.
So far this just sounds like a plain SystemD bug and something that can be easily changed at this point in time.
It has been a long time but my fuzzy memory says that the originial
boot_idjustification was based on use cases that could not be solved any other way.My memory says it was this thread https://lkml.org/lkml/1999/5/31/233
that inspired the implementation ofboot_id. However reading the current emacs source code it appears emacs gave up beforeboot_idwas implemented and stats/var/run/random-seed(which we seem to have removed) or looks in wtmp or utmp for the latest boot record.I did a quick grep through the binaries on my system and I could not find anything using
/proc/sys/random/boot_id.That suggests to me that the proper solution is to actually just remove
boot_id.
Quoted 6 lines
Quoted 3 lines
Hmm. And then there is other interesting detail. What should
boot_idreturn after the processes have migrated from one system to another.Since this would be a per-boot id, this clearly has to be carried over with migration, along with all the tons of data we already carry.
The twist of course is what does a boot mean. If we are really after machine boots than the current behavior is correct.
Looking back in the archives the desired behavior appears to be a value that can be used to see if a pid value must be stale.
As a stale pid detector boot_id is pretty lousy. Pids can still be reused.
Still a role as a stale pid detector makes it clear which namespace boot_id should be in and how we should treat boot_id upon migration.
You can only serve as a stale pid detector if you are in the pid namespace.
So at this point patches are welcome. Hopefully with a summary of the discussion.
Eric
A message that is not in the archive (sent off-list or not yet received): 87r4qi6g6k.fsf-aS9lmoZGLiVWk0Htik3J/[email protected]
Quoted 71 lines · Eric W. Biederman
Quoted 50 lines · Glauber Costa
Quoted 49 lines · Eric W. Biederman
Quoted 15 lines · Daniel P. Berrange
Quoted 6 lines · Eric W. Biederman
Quoted 3 lines · Daniel P. Berrange
One of the features that SystemD folks have asked us to fix in LXC, is to make sure that
/proc/sys/kernel/random/boot_idchanges each time a container is started.Quoted 7 lines
There may be a good reason for this. Most of the time what I have seen of kernel requests from the direction of SystemD is that while there may be a real problem but usually their imagined solution is not a particularly good solution. So a description of the problem is needed.
Justifying something with just SystemD wants this is a good way to get a nack.
Quoted 9 lines
SystemD records log messages for all system services in their journal. They can show you all log messages for the current service execution, all log messages for a service since system boot, or all log messsages ever. The
boot_idvalue is used as a unique tag to allow grouping of the log messages per system boot. When we run systemd inside a container we want to get that grouping of log messages generated by services inside the container, to take account of the container boot, not the host boot. Hence the desire to have theboot_idvalue reflect when a container is booted.Since SystemD post-dates containers and since the logging feature is not currently in wide use that use case is completely non-persuasive.
So far this just sounds like a plain SystemD bug and something that can be easily changed at this point in time.
It has been a long time but my fuzzy memory says that the originial
boot_idjustification was based on use cases that could not be solved any other way.My memory says it was this thread https://lkml.org/lkml/1999/5/31/233
that inspired the implementation ofboot_id. However reading the current emacs source code it appears emacs gave up beforeboot_idwas implemented and stats/var/run/random-seed(which we seem to have removed) or looks in wtmp or utmp for the latest boot record.I did a quick grep through the binaries on my system and I could not find anything using
/proc/sys/random/boot_id.That suggests to me that the proper solution is to actually just remove
boot_id.Quoted 6 lines
Quoted 3 lines
Hmm. And then there is other interesting detail. What should
boot_idreturn after the processes have migrated from one system to another.Since this would be a per-boot id, this clearly has to be carried over with migration, along with all the tons of data we already carry.
The twist of course is what does a boot mean. If we are really after machine boots than the current behavior is correct.
Looking back in the archives the desired behavior appears to be a value that can be used to see if a pid value must be stale.
As a stale pid detector
boot_idis pretty lousy. Pids can still be reused.Still a role as a stale pid detector makes it clear which namespace
boot_idshould be in and how we should treatboot_idupon migration.
Quoted 8 lines
You can only serve as a stale pid detector if you are in the pid namespace.
So at this point patches are welcome. Hopefully with a summary of the discussion.
Eric
Your discussion about boot_id being a limited solution is totally valid. But it is orthogonal to the question of whether or not a container should have it.
I took a look at this, and I think the kernel should be in perfect position to do it. FUSE is welcome so far for things that are really ill-defined in the kernel, such as data coming from cgroups, which has no concept of visibility.
boot_id as a pid namespace id is a very well defined concept. We just need an interface to set it up to make it stable across migration. Maybe we can accept writes to this file as valid, provided the pid namespace has only the init process.
Then any tool could clone, mount proc, set this id, and continue normally. Any objections ?
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 7 lines · Glauber Costa
boot_idas a pid namespace id is a very well defined concept. We just need an interface to set it up to make it stable across migration. Maybe we can accept writes to this file as valid, provided the pid namespace has only the init process.Then any tool could clone, mount proc, set this id, and continue normally. Any objections ?
Ok, the above is totally jet-lag induced garbage. I totally forgot this is a sysctl interface.
We do per-netns sysctls just fine, why can't we do them here as well ?
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 6 lines · Glauber Costa
Quoted 5 lines · Glauber Costa
boot_idas a pid namespace id is a very well defined concept. We just need an interface to set it up to make it stable across migration. Maybe we can accept writes to this file as valid, provided the pid namespace has only the init process.
Quoted 7 lines
Quoted 2 lines
Then any tool could clone, mount proc, set this id, and continue normally. Any objections ?
Ok, the above is totally jet-lag induced garbage. I totally forgot this is a sysctl interface.
We do per-netns sysctls just fine, why can't we do them here as well ?
Yes. This is a sysctl.
The definition of boot_id is that it is for detecting stale pids. So it should be per pid-namespace not per-netns.
The sysctl infrastructure supports per pid-namespace sysctls as easily as per-netns sysctls.
Well almost as easily as the glue code to write a register_pidns_sysctl hasn't been written. But however the existing hack of looking at current works fine for the moment as well.
Ultimately I want to get us to /proc/<pid>/sys/ so we can look at each processes sysctls and tweak them. But that isn't this weeks project.
Eric
Quoted 18 lines · Glauber Costa
Quoted 18 lines
The twist of course is what does a boot mean. If we are really after machine boots than the current behavior is correct.
Looking back in the archives the desired behavior appears to be a value that can be used to see if a pid value must be stale.
As a stale pid detector
boot_idis pretty lousy. Pids can still be reused.Still a role as a stale pid detector makes it clear which namespace
boot_idshould be in and how we should treatboot_idupon migration.You can only serve as a stale pid detector if you are in the pid namespace.
So at this point patches are welcome. Hopefully with a summary of the discussion.
Quoted 6 lines
Quoted 2 lines
Eric
Your discussion about
boot_idbeing a limited solution is totally valid. But it is orthogonal to the question of whether or not a container should have it.
The important part is that boot_id as originally conceived is an identifier to be used in conjunction with pids. Therefore boot_id is a proper pid_namespace component, and there are no semantic problems with putting it in the kernel.
Quoted 2 lines
I took a look at this, and I think the kernel should be in perfect position to do it.
Agreed.
Quoted 1 line
boot_idas a pid namespace id is a very well defined concept.
Agreed.
A reference to the history and the definition needs to be in the patch description.
Quoted 4 lines
We just
need an interface to set it up to make it stable across migration. Maybe we can accept writes to this file as valid, provided the pid namespace has only the init process.
I think the easy solution is to take advantage of the fact that boot_id is initialized lazily. Don't allow writes if boot_id has already been read.
Quoted 2 lines
Then any tool could clone, mount proc, set this id, and continue normally. Any objections ?
My biggest concern is that creating multiple pid namespaces might allow a way to drain the entropy pool in a way that we don't allow normal users.
This is important to look at as with a little luck we will have unprivileged creation of user namespaces and pid namespaces in the near future.
Eric
A message that is not in the archive (sent off-list or not yet received): 878vcq5ekx.fsf-aS9lmoZGLiVWk0Htik3J/[email protected]
Quoted 39 lines · Eric W. Biederman
Quoted 15 lines · Glauber Costa
Quoted 15 lines
The twist of course is what does a boot mean. If we are really after machine boots than the current behavior is correct.
Looking back in the archives the desired behavior appears to be a value that can be used to see if a pid value must be stale.
As a stale pid detector
boot_idis pretty lousy. Pids can still be reused.Still a role as a stale pid detector makes it clear which namespace
boot_idshould be in and how we should treatboot_idupon migration.You can only serve as a stale pid detector if you are in the pid namespace.
Quoted 6 lines
Quoted 2 lines
So at this point patches are welcome. Hopefully with a summary of the discussion.
Your discussion about
boot_idbeing a limited solution is totally valid. But it is orthogonal to the question of whether or not a container should have it.The important part is that
boot_idas originally conceived is an identifier to be used in conjunction with pids. Thereforeboot_idis a properpid_namespacecomponent, and there are no semantic problems with putting it in the kernel.Quoted 1 line
boot_idas a pid namespace id is a very well defined concept.Agreed.
A reference to the history and the definition needs to be in the patch description.
Quoted 2 lines
Then any tool could clone, mount proc, set this id, and continue normally. Any objections ?
Quoted 7 lines
My biggest concern is that creating multiple pid namespaces might allow a way to drain the entropy pool in a way that we don't allow normal users.
This is important to look at as with a little luck we will have unprivileged creation of user namespaces and pid namespaces in the near future.
Unprivileged users can already ask the kernel to generate random UUIDs on demand. eg
$ sysctl kernel.random.uuid kernel.random.uuid = 76199778-8f6d-4fae-a45d-2c4cc0bca62a $ sysctl kernel.random.uuid kernel.random.uuid = 00c7d637-f94d-4df6-82c3-34ad97dd782e ...etc...
So allocating a new random UUID for boot_id each time a pid namespace is created does not appear to make the situation worse wrt entropy usage by unprivileged users.
Regards,
Daniel
Signature · 4 lines
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Quoted 34 lines · Eric W. Biederman
Quoted 18 lines · Glauber Costa
Quoted 18 lines
The twist of course is what does a boot mean. If we are really after machine boots than the current behavior is correct.
Looking back in the archives the desired behavior appears to be a value that can be used to see if a pid value must be stale.
As a stale pid detector
boot_idis pretty lousy. Pids can still be reused.Still a role as a stale pid detector makes it clear which namespace
boot_idshould be in and how we should treatboot_idupon migration.You can only serve as a stale pid detector if you are in the pid namespace.
So at this point patches are welcome. Hopefully with a summary of the discussion.
Quoted 6 lines
Quoted 2 lines
Eric
Your discussion about
boot_idbeing a limited solution is totally valid. But it is orthogonal to the question of whether or not a container should have it.The important part is that
boot_idas originally conceived is an identifier to be used in conjunction with pids. Thereforeboot_idis a properpid_namespacecomponent, and there are no semantic problems with putting it in the kernel.Quoted 2 lines
I took a look at this, and I think the kernel should be in perfect position to do it.
Quoted 8 lines
Agreed.
Quoted 1 line
boot_idas a pid namespace id is a very well defined concept.Agreed.
A reference to the history and the definition needs to be in the patch description.
Huh. I must have glossed over that part of this thread. If you both agree to that, then I'm fine with it - I'll go read the history when the new patch comes by.
thanks,
-serge
Quoted 68 lines · Eric W. Biederman
Quoted 50 lines · Glauber Costa
Quoted 49 lines · Eric W. Biederman
Quoted 15 lines · Daniel P. Berrange
Quoted 6 lines · Eric W. Biederman
Quoted 3 lines · Daniel P. Berrange
One of the features that SystemD folks have asked us to fix in LXC, is to make sure that
/proc/sys/kernel/random/boot_idchanges each time a container is started.Quoted 7 lines
There may be a good reason for this. Most of the time what I have seen of kernel requests from the direction of SystemD is that while there may be a real problem but usually their imagined solution is not a particularly good solution. So a description of the problem is needed.
Justifying something with just SystemD wants this is a good way to get a nack.
Quoted 9 lines
SystemD records log messages for all system services in their journal. They can show you all log messages for the current service execution, all log messages for a service since system boot, or all log messsages ever. The
boot_idvalue is used as a unique tag to allow grouping of the log messages per system boot. When we run systemd inside a container we want to get that grouping of log messages generated by services inside the container, to take account of the container boot, not the host boot. Hence the desire to have theboot_idvalue reflect when a container is booted.Since SystemD post-dates containers and since the logging feature is not currently in wide use that use case is completely non-persuasive.
So far this just sounds like a plain SystemD bug and something that can be easily changed at this point in time.
It has been a long time but my fuzzy memory says that the originial
boot_idjustification was based on use cases that could not be solved any other way.My memory says it was this thread https://lkml.org/lkml/1999/5/31/233
that inspired the implementation ofboot_id. However reading the current emacs source code it appears emacs gave up beforeboot_idwas implemented and stats/var/run/random-seed(which we seem to have removed) or looks in wtmp or utmp for the latest boot record.I did a quick grep through the binaries on my system and I could not find anything using
/proc/sys/random/boot_id.That suggests to me that the proper solution is to actually just remove
boot_id.Quoted 6 lines
Quoted 3 lines
Hmm. And then there is other interesting detail. What should
boot_idreturn after the processes have migrated from one system to another.Since this would be a per-boot id, this clearly has to be carried over with migration, along with all the tons of data we already carry.
The twist of course is what does a boot mean. If we are really after machine boots than the current behavior is correct.
Looking back in the archives the desired behavior appears to be a value that can be used to see if a pid value must be stale.
As a stale pid detector
boot_idis pretty lousy. Pids can still be reused.
Quoted 8 lines
Still a role as a stale pid detector makes it clear which namespace
boot_idshould be in and how we should treatboot_idupon migration.You can only serve as a stale pid detector if you are in the pid namespace.
So at this point patches are welcome. Hopefully with a summary of the discussion.
I don't understand why this should be provided by the kernel. Especially given that we've proven that everyone really wants this to be per-container as well.
So why not just have init, on startup, create a /run/boot_id file, perhaps by sha1summing the time at which it started perhaps plus some nonce?
Quoted 78 lines · Serge Hallyn
Quoted 68 lines · Eric W. Biederman
Quoted 50 lines · Glauber Costa
Quoted 49 lines · Eric W. Biederman
Quoted 15 lines · Daniel P. Berrange
Quoted 6 lines · Eric W. Biederman
Quoted 3 lines · Daniel P. Berrange
One of the features that SystemD folks have asked us to fix in LXC, is to make sure that
/proc/sys/kernel/random/boot_idchanges each time a container is started.Quoted 7 lines
There may be a good reason for this. Most of the time what I have seen of kernel requests from the direction of SystemD is that while there may be a real problem but usually their imagined solution is not a particularly good solution. So a description of the problem is needed.
Justifying something with just SystemD wants this is a good way to get a nack.
Quoted 9 lines
SystemD records log messages for all system services in their journal. They can show you all log messages for the current service execution, all log messages for a service since system boot, or all log messsages ever. The
boot_idvalue is used as a unique tag to allow grouping of the log messages per system boot. When we run systemd inside a container we want to get that grouping of log messages generated by services inside the container, to take account of the container boot, not the host boot. Hence the desire to have theboot_idvalue reflect when a container is booted.Since SystemD post-dates containers and since the logging feature is not currently in wide use that use case is completely non-persuasive.
So far this just sounds like a plain SystemD bug and something that can be easily changed at this point in time.
It has been a long time but my fuzzy memory says that the originial
boot_idjustification was based on use cases that could not be solved any other way.My memory says it was this thread https://lkml.org/lkml/1999/5/31/233
that inspired the implementation ofboot_id. However reading the current emacs source code it appears emacs gave up beforeboot_idwas implemented and stats/var/run/random-seed(which we seem to have removed) or looks in wtmp or utmp for the latest boot record.I did a quick grep through the binaries on my system and I could not find anything using
/proc/sys/random/boot_id.That suggests to me that the proper solution is to actually just remove
boot_id.Quoted 6 lines
Quoted 3 lines
Hmm. And then there is other interesting detail. What should
boot_idreturn after the processes have migrated from one system to another.Since this would be a per-boot id, this clearly has to be carried over with migration, along with all the tons of data we already carry.
The twist of course is what does a boot mean. If we are really after machine boots than the current behavior is correct.
Looking back in the archives the desired behavior appears to be a value that can be used to see if a pid value must be stale.
As a stale pid detector
boot_idis pretty lousy. Pids can still be reused.Quoted 8 lines
Still a role as a stale pid detector makes it clear which namespace
boot_idshould be in and how we should treatboot_idupon migration.You can only serve as a stale pid detector if you are in the pid namespace.
So at this point patches are welcome. Hopefully with a summary of the discussion.
Quoted 7 lines
I don't understand why this should be provided by the kernel. Especially given that we've proven that everyone really wants this to be per-container as well.
So why not just have init, on startup, create a
/run/boot_idfile, perhaps by sha1summing the time at which it started perhaps plus some nonce?
Why shouldn't it provided by the kernel?, is the real question
The way I see it, every file we need to setup from the outside is a hassle. Among many other things, it is just asking for duplication of efforts among multiple userspaces.
netns does this for its proc files. The only reason we don't do it for cgroups-driven file, is that the semantics is very ill-defined. For this file, it doesn't seem to be the case.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 79 lines · Glauber Costa
Quoted 78 lines · Serge Hallyn
Quoted 76 lines · Eric W. Biederman
Quoted 50 lines · Glauber Costa
Quoted 49 lines · Eric W. Biederman
Quoted 15 lines · Daniel P. Berrange
Quoted 6 lines · Eric W. Biederman
Quoted 3 lines · Daniel P. Berrange
One of the features that SystemD folks have asked us to fix in LXC, is to make sure that
/proc/sys/kernel/random/boot_idchanges each time a container is started.Quoted 7 lines
There may be a good reason for this. Most of the time what I have seen of kernel requests from the direction of SystemD is that while there may be a real problem but usually their imagined solution is not a particularly good solution. So a description of the problem is needed.
Justifying something with just SystemD wants this is a good way to get a nack.
Quoted 9 lines
SystemD records log messages for all system services in their journal. They can show you all log messages for the current service execution, all log messages for a service since system boot, or all log messsages ever. The
boot_idvalue is used as a unique tag to allow grouping of the log messages per system boot. When we run systemd inside a container we want to get that grouping of log messages generated by services inside the container, to take account of the container boot, not the host boot. Hence the desire to have theboot_idvalue reflect when a container is booted.Since SystemD post-dates containers and since the logging feature is not currently in wide use that use case is completely non-persuasive.
So far this just sounds like a plain SystemD bug and something that can be easily changed at this point in time.
It has been a long time but my fuzzy memory says that the originial
boot_idjustification was based on use cases that could not be solved any other way.My memory says it was this thread https://lkml.org/lkml/1999/5/31/233
that inspired the implementation ofboot_id. However reading the current emacs source code it appears emacs gave up beforeboot_idwas implemented and stats/var/run/random-seed(which we seem to have removed) or looks in wtmp or utmp for the latest boot record.I did a quick grep through the binaries on my system and I could not find anything using
/proc/sys/random/boot_id.That suggests to me that the proper solution is to actually just remove
boot_id.Quoted 6 lines
Quoted 3 lines
Hmm. And then there is other interesting detail. What should
boot_idreturn after the processes have migrated from one system to another.Since this would be a per-boot id, this clearly has to be carried over with migration, along with all the tons of data we already carry.
The twist of course is what does a boot mean. If we are really after machine boots than the current behavior is correct.
Looking back in the archives the desired behavior appears to be a value that can be used to see if a pid value must be stale.
As a stale pid detector
boot_idis pretty lousy. Pids can still be reused.Still a role as a stale pid detector makes it clear which namespace
boot_idshould be in and how we should treatboot_idupon migration.You can only serve as a stale pid detector if you are in the pid namespace.
So at this point patches are welcome. Hopefully with a summary of the discussion.
Quoted 8 lines
Quoted 7 lines
I don't understand why this should be provided by the kernel. Especially given that we've proven that everyone really wants this to be per-container as well.
So why not just have init, on startup, create a
/run/boot_idfile, perhaps by sha1summing the time at which it started perhaps plus some nonce?Why shouldn't it provided by the kernel?, is the real question
Because it's not the right place. The origin of this thread proves that people want a per-init, not per-kernel, value.
Quoted 7 lines
The way I see it, every file we need to setup from the outside is a hassle. Among many other things, it is just asking for duplication of efforts among multiple userspaces.
netns does this for its proc files. The only reason we don't do it for cgroups-driven file, is that the semantics is very ill-defined. For this file, it doesn't seem to be the case.
But it is the case. How do you intend to have the kernel decide what value to put in there for a process in a container, or in a chroot?
-serge
Quoted 84 lines · Serge Hallyn
Quoted 83 lines · Glauber Costa
Quoted 82 lines · Serge Hallyn
Quoted 68 lines · Eric W. Biederman
Quoted 50 lines · Glauber Costa
Quoted 49 lines · Eric W. Biederman
Quoted 15 lines · Daniel P. Berrange
Quoted 6 lines · Eric W. Biederman
Quoted 3 lines · Daniel P. Berrange
One of the features that SystemD folks have asked us to fix in LXC, is to make sure that
/proc/sys/kernel/random/boot_idchanges each time a container is started.Quoted 7 lines
There may be a good reason for this. Most of the time what I have seen of kernel requests from the direction of SystemD is that while there may be a real problem but usually their imagined solution is not a particularly good solution. So a description of the problem is needed.
Justifying something with just SystemD wants this is a good way to get a nack.
Quoted 9 lines
SystemD records log messages for all system services in their journal. They can show you all log messages for the current service execution, all log messages for a service since system boot, or all log messsages ever. The
boot_idvalue is used as a unique tag to allow grouping of the log messages per system boot. When we run systemd inside a container we want to get that grouping of log messages generated by services inside the container, to take account of the container boot, not the host boot. Hence the desire to have theboot_idvalue reflect when a container is booted.Since SystemD post-dates containers and since the logging feature is not currently in wide use that use case is completely non-persuasive.
So far this just sounds like a plain SystemD bug and something that can be easily changed at this point in time.
It has been a long time but my fuzzy memory says that the originial
boot_idjustification was based on use cases that could not be solved any other way.My memory says it was this thread https://lkml.org/lkml/1999/5/31/233
that inspired the implementation ofboot_id. However reading the current emacs source code it appears emacs gave up beforeboot_idwas implemented and stats/var/run/random-seed(which we seem to have removed) or looks in wtmp or utmp for the latest boot record.I did a quick grep through the binaries on my system and I could not find anything using
/proc/sys/random/boot_id.That suggests to me that the proper solution is to actually just remove
boot_id.Quoted 6 lines
Quoted 3 lines
Hmm. And then there is other interesting detail. What should
boot_idreturn after the processes have migrated from one system to another.Since this would be a per-boot id, this clearly has to be carried over with migration, along with all the tons of data we already carry.
The twist of course is what does a boot mean. If we are really after machine boots than the current behavior is correct.
Looking back in the archives the desired behavior appears to be a value that can be used to see if a pid value must be stale.
As a stale pid detector
boot_idis pretty lousy. Pids can still be reused.Quoted 8 lines
Still a role as a stale pid detector makes it clear which namespace
boot_idshould be in and how we should treatboot_idupon migration.You can only serve as a stale pid detector if you are in the pid namespace.
So at this point patches are welcome. Hopefully with a summary of the discussion.
I don't understand why this should be provided by the kernel. Especially given that we've proven that everyone really wants this to be per-container as well.
Quoted 8 lines
Quoted 4 lines
Quoted 3 lines
So why not just have init, on startup, create a
/run/boot_idfile, perhaps by sha1summing the time at which it started perhaps plus some nonce?Why shouldn't it provided by the kernel?, is the real question
Because it's not the right place. The origin of this thread proves that people want a per-init, not per-kernel, value.
Not all files provided by the kernel are "per-kernel". /proc/self is full of per-namespace stuff.
Quoted 11 lines
Quoted 7 lines
The way I see it, every file we need to setup from the outside is a hassle. Among many other things, it is just asking for duplication of efforts among multiple userspaces.
netns does this for its proc files. The only reason we don't do it for cgroups-driven file, is that the semantics is very ill-defined. For this file, it doesn't seem to be the case.
But it is the case. How do you intend to have the kernel decide what value to put in there for a process in a container, or in a chroot?
one value per pidns.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 105 lines · Glauber Costa
Quoted 84 lines · Serge Hallyn
Quoted 83 lines · Glauber Costa
Quoted 82 lines · Serge Hallyn
Quoted 68 lines · Eric W. Biederman
Quoted 50 lines · Glauber Costa
Quoted 49 lines · Eric W. Biederman
Quoted 15 lines · Daniel P. Berrange
Quoted 6 lines · Eric W. Biederman
Quoted 3 lines · Daniel P. Berrange
One of the features that SystemD folks have asked us to fix in LXC, is to make sure that
/proc/sys/kernel/random/boot_idchanges each time a container is started.Quoted 7 lines
There may be a good reason for this. Most of the time what I have seen of kernel requests from the direction of SystemD is that while there may be a real problem but usually their imagined solution is not a particularly good solution. So a description of the problem is needed.
Justifying something with just SystemD wants this is a good way to get a nack.
Quoted 9 lines
SystemD records log messages for all system services in their journal. They can show you all log messages for the current service execution, all log messages for a service since system boot, or all log messsages ever. The
boot_idvalue is used as a unique tag to allow grouping of the log messages per system boot. When we run systemd inside a container we want to get that grouping of log messages generated by services inside the container, to take account of the container boot, not the host boot. Hence the desire to have theboot_idvalue reflect when a container is booted.Since SystemD post-dates containers and since the logging feature is not currently in wide use that use case is completely non-persuasive.
So far this just sounds like a plain SystemD bug and something that can be easily changed at this point in time.
It has been a long time but my fuzzy memory says that the originial
boot_idjustification was based on use cases that could not be solved any other way.My memory says it was this thread https://lkml.org/lkml/1999/5/31/233
that inspired the implementation ofboot_id. However reading the current emacs source code it appears emacs gave up beforeboot_idwas implemented and stats/var/run/random-seed(which we seem to have removed) or looks in wtmp or utmp for the latest boot record.I did a quick grep through the binaries on my system and I could not find anything using
/proc/sys/random/boot_id.That suggests to me that the proper solution is to actually just remove
boot_id.Quoted 6 lines
Quoted 3 lines
Hmm. And then there is other interesting detail. What should
boot_idreturn after the processes have migrated from one system to another.Since this would be a per-boot id, this clearly has to be carried over with migration, along with all the tons of data we already carry.
The twist of course is what does a boot mean. If we are really after machine boots than the current behavior is correct.
Looking back in the archives the desired behavior appears to be a value that can be used to see if a pid value must be stale.
As a stale pid detector
boot_idis pretty lousy. Pids can still be reused.Quoted 8 lines
Still a role as a stale pid detector makes it clear which namespace
boot_idshould be in and how we should treatboot_idupon migration.You can only serve as a stale pid detector if you are in the pid namespace.
So at this point patches are welcome. Hopefully with a summary of the discussion.
I don't understand why this should be provided by the kernel. Especially given that we've proven that everyone really wants this to be per-container as well.
Quoted 8 lines
Quoted 4 lines
Quoted 3 lines
So why not just have init, on startup, create a
/run/boot_idfile, perhaps by sha1summing the time at which it started perhaps plus some nonce?Why shouldn't it provided by the kernel?, is the real question
Because it's not the right place. The origin of this thread proves that people want a per-init, not per-kernel, value.
Not all files provided by the kernel are "per-kernel".
/proc/selfis full of per-namespace stuff.Quoted 8 lines
Quoted 7 lines
The way I see it, every file we need to setup from the outside is a hassle. Among many other things, it is just asking for duplication of efforts among multiple userspaces.
netns does this for its proc files. The only reason we don't do it for cgroups-driven file, is that the semantics is very ill-defined. For this file, it doesn't seem to be the case.
Quoted 5 lines
Quoted 3 lines
But it is the case. How do you intend to have the kernel decide what value to put in there for a process in a container, or in a chroot?
one value per pidns.
ok. (So should it be called /proc/pidns_uuid? Well, whatever. No objection from me - thanks.)
-serge
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 106 lines · Serge E. Hallyn
Quoted 105 lines · Glauber Costa
Quoted 84 lines · Serge Hallyn
Quoted 83 lines · Glauber Costa
Quoted 82 lines · Serge Hallyn
Quoted 68 lines · Eric W. Biederman
Quoted 50 lines · Glauber Costa
Quoted 49 lines · Eric W. Biederman
Quoted 15 lines · Daniel P. Berrange
Quoted 6 lines · Eric W. Biederman
Quoted 3 lines · Daniel P. Berrange
One of the features that SystemD folks have asked us to fix in LXC, is to make sure that
/proc/sys/kernel/random/boot_idchanges each time a container is started.Quoted 7 lines
There may be a good reason for this. Most of the time what I have seen of kernel requests from the direction of SystemD is that while there may be a real problem but usually their imagined solution is not a particularly good solution. So a description of the problem is needed.
Justifying something with just SystemD wants this is a good way to get a nack.
Quoted 9 lines
SystemD records log messages for all system services in their journal. They can show you all log messages for the current service execution, all log messages for a service since system boot, or all log messsages ever. The
boot_idvalue is used as a unique tag to allow grouping of the log messages per system boot. When we run systemd inside a container we want to get that grouping of log messages generated by services inside the container, to take account of the container boot, not the host boot. Hence the desire to have theboot_idvalue reflect when a container is booted.Since SystemD post-dates containers and since the logging feature is not currently in wide use that use case is completely non-persuasive.
So far this just sounds like a plain SystemD bug and something that can be easily changed at this point in time.
It has been a long time but my fuzzy memory says that the originial
boot_idjustification was based on use cases that could not be solved any other way.My memory says it was this thread https://lkml.org/lkml/1999/5/31/233
that inspired the implementation ofboot_id. However reading the current emacs source code it appears emacs gave up beforeboot_idwas implemented and stats/var/run/random-seed(which we seem to have removed) or looks in wtmp or utmp for the latest boot record.I did a quick grep through the binaries on my system and I could not find anything using
/proc/sys/random/boot_id.That suggests to me that the proper solution is to actually just remove
boot_id.Quoted 6 lines
Quoted 3 lines
Hmm. And then there is other interesting detail. What should
boot_idreturn after the processes have migrated from one system to another.Since this would be a per-boot id, this clearly has to be carried over with migration, along with all the tons of data we already carry.
The twist of course is what does a boot mean. If we are really after machine boots than the current behavior is correct.
Looking back in the archives the desired behavior appears to be a value that can be used to see if a pid value must be stale.
As a stale pid detector
boot_idis pretty lousy. Pids can still be reused.Quoted 8 lines
Still a role as a stale pid detector makes it clear which namespace
boot_idshould be in and how we should treatboot_idupon migration.You can only serve as a stale pid detector if you are in the pid namespace.
So at this point patches are welcome. Hopefully with a summary of the discussion.
I don't understand why this should be provided by the kernel. Especially given that we've proven that everyone really wants this to be per-container as well.
Quoted 8 lines
Quoted 4 lines
Quoted 3 lines
So why not just have init, on startup, create a
/run/boot_idfile, perhaps by sha1summing the time at which it started perhaps plus some nonce?Why shouldn't it provided by the kernel?, is the real question
Because it's not the right place. The origin of this thread proves that people want a per-init, not per-kernel, value.
Not all files provided by the kernel are "per-kernel".
/proc/selfis full of per-namespace stuff.Quoted 8 lines
Quoted 7 lines
The way I see it, every file we need to setup from the outside is a hassle. Among many other things, it is just asking for duplication of efforts among multiple userspaces.
netns does this for its proc files. The only reason we don't do it for cgroups-driven file, is that the semantics is very ill-defined. For this file, it doesn't seem to be the case.
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
But it is the case. How do you intend to have the kernel decide what value to put in there for a process in a container, or in a chroot?
one value per pidns.
ok. (So should it be called
/proc/pidns_uuid? Well, whatever. No objection from me - thanks.)
/proc/sys/kernel/boot_id.
Someday we will get the plumbing right in the kernel so that can be /proc/sys -> /proc/self/sys and /proc/self/sys/kernel/boot_id
The origin of boot_id was so that emacs could implement distributed locking in userspace by creating a symlink from .#filename to user-WI0L6dQK/[email protected]:boot_id.
Ultimately emacs opted to just stat /var/run/random-seed or to grovel through utmp or wtmp to find the last boot record.
Of course /var/run/random-seed is now named something like /var/lib/urandom/random-seed as distributions continue their relentless pursuit to break userspace.
But ultimately boot_id was defined as something you can use to detect stale pids and stale lockfiles. Since the original definition was a uuid to detect stale pids, that seems a reasonable justification for keeping it in the pid_namespace. Boot_id isn't the best name in that case but shrug.
Eric
A message that is not in the archive (sent off-list or not yet received): 87vcft1shu.fsf-aS9lmoZGLiVWk0Htik3J/[email protected]
Quoted 39 lines · Eric W. Biederman
Quoted 12 lines · Serge E. Hallyn
Quoted 11 lines · Glauber Costa
Not all files provided by the kernel are "per-kernel".
/proc/selfis full of per-namespace stuff.Quoted 8 lines
Quoted 7 lines
The way I see it, every file we need to setup from the outside is a hassle. Among many other things, it is just asking for duplication of efforts among multiple userspaces.
netns does this for its proc files. The only reason we don't do it for cgroups-driven file, is that the semantics is very ill-defined. For this file, it doesn't seem to be the case.
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
But it is the case. How do you intend to have the kernel decide what value to put in there for a process in a container, or in a chroot?
one value per pidns.
ok. (So should it be called
/proc/pidns_uuid? Well, whatever. No objection from me - thanks.)
/proc/sys/kernel/boot_id.Someday we will get the plumbing right in the kernel so that can be
/proc/sys->/proc/self/sysand/proc/self/sys/kernel/boot_idThe origin of
boot_idwas so that emacs could implement distributed locking in userspace by creating a symlink from .#filename to user-WI0L6dQK/[email protected]:boot_id.Ultimately emacs opted to just stat
/var/run/random-seedor to grovel through utmp or wtmp to find the last boot record.Of course
/var/run/random-seedis now named something like/var/lib/urandom/random-seedas distributions continue their relentless pursuit to break userspace.
Quoted 5 lines
But ultimately
boot_idwas defined as something you can use to detect stale pids and stale lockfiles. Since the original definition was a uuid to detect stale pids, that seems a reasonable justification for keeping it in thepid_namespace.Boot_idisn't the best name in that case but shrug.
Ok, so reading through this thread, my understanding is that any patch for this needs to work as follows:
- Associate '/proc/sys/kernel/random/boot_id' with the pid namespace
- Allow boot_id to be written to, only if it has not yet been read in the current pid namespace. (for migration use case)
- Lazy generate a UUID for boot_id on first read in the current pid namespace, only if it has not previously been written to.
- Add file to Documentation/ explaining the use case for the boot_id file and its semantics wrt to namespaces.
Regards,
Daniel
Signature · 4 lines
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Quoted 110 lines · Serge E. Hallyn
Quoted 109 lines · Glauber Costa
Quoted 84 lines · Serge Hallyn
Quoted 83 lines · Glauber Costa
Quoted 82 lines · Serge Hallyn
Quoted 68 lines · Eric W. Biederman
Quoted 50 lines · Glauber Costa
Quoted 49 lines · Eric W. Biederman
Quoted 15 lines · Daniel P. Berrange
Quoted 6 lines · Eric W. Biederman
Quoted 3 lines · Daniel P. Berrange
One of the features that SystemD folks have asked us to fix in LXC, is to make sure that
/proc/sys/kernel/random/boot_idchanges each time a container is started.Quoted 7 lines
There may be a good reason for this. Most of the time what I have seen of kernel requests from the direction of SystemD is that while there may be a real problem but usually their imagined solution is not a particularly good solution. So a description of the problem is needed.
Justifying something with just SystemD wants this is a good way to get a nack.
Quoted 9 lines
SystemD records log messages for all system services in their journal. They can show you all log messages for the current service execution, all log messages for a service since system boot, or all log messsages ever. The
boot_idvalue is used as a unique tag to allow grouping of the log messages per system boot. When we run systemd inside a container we want to get that grouping of log messages generated by services inside the container, to take account of the container boot, not the host boot. Hence the desire to have theboot_idvalue reflect when a container is booted.Since SystemD post-dates containers and since the logging feature is not currently in wide use that use case is completely non-persuasive.
So far this just sounds like a plain SystemD bug and something that can be easily changed at this point in time.
It has been a long time but my fuzzy memory says that the originial
boot_idjustification was based on use cases that could not be solved any other way.My memory says it was this thread https://lkml.org/lkml/1999/5/31/233
that inspired the implementation ofboot_id. However reading the current emacs source code it appears emacs gave up beforeboot_idwas implemented and stats/var/run/random-seed(which we seem to have removed) or looks in wtmp or utmp for the latest boot record.I did a quick grep through the binaries on my system and I could not find anything using
/proc/sys/random/boot_id.That suggests to me that the proper solution is to actually just remove
boot_id.Quoted 6 lines
Quoted 3 lines
Hmm. And then there is other interesting detail. What should
boot_idreturn after the processes have migrated from one system to another.Since this would be a per-boot id, this clearly has to be carried over with migration, along with all the tons of data we already carry.
The twist of course is what does a boot mean. If we are really after machine boots than the current behavior is correct.
Looking back in the archives the desired behavior appears to be a value that can be used to see if a pid value must be stale.
As a stale pid detector
boot_idis pretty lousy. Pids can still be reused.Quoted 8 lines
Still a role as a stale pid detector makes it clear which namespace
boot_idshould be in and how we should treatboot_idupon migration.You can only serve as a stale pid detector if you are in the pid namespace.
So at this point patches are welcome. Hopefully with a summary of the discussion.
I don't understand why this should be provided by the kernel. Especially given that we've proven that everyone really wants this to be per-container as well.
Quoted 8 lines
Quoted 4 lines
Quoted 3 lines
So why not just have init, on startup, create a
/run/boot_idfile, perhaps by sha1summing the time at which it started perhaps plus some nonce?Why shouldn't it provided by the kernel?, is the real question
Because it's not the right place. The origin of this thread proves that people want a per-init, not per-kernel, value.
Not all files provided by the kernel are "per-kernel".
/proc/selfis full of per-namespace stuff.Quoted 11 lines
Quoted 7 lines
The way I see it, every file we need to setup from the outside is a hassle. Among many other things, it is just asking for duplication of efforts among multiple userspaces.
netns does this for its proc files. The only reason we don't do it for cgroups-driven file, is that the semantics is very ill-defined. For this file, it doesn't seem to be the case.
But it is the case. How do you intend to have the kernel decide what value to put in there for a process in a container, or in a chroot?
Quoted 7 lines
Quoted 1 line
one value per pidns.
ok. (So should it be called
/proc/pidns_uuid? Well, whatever. No objection from me - thanks.)-serge
For completeness, I believe it should live in the same place it lives today, and become a symlink from /proc/self/boot_id.
Consistent with what we have today for other values like this.
Quoted 19 lines · Eric W. Biederman
Quoted 3 lines · Daniel P. Berrange
One of the features that SystemD folks have asked us to fix in LXC, is to make sure that
/proc/sys/kernel/random/boot_idchanges each time a container is started.There may be a good reason for this. Most of the time what I have seen of kernel requests from the direction of SystemD is that while there may be a real problem but usually their imagined solution is not a particularly good solution. So a description of the problem is needed.
Justifying something with just SystemD wants this is a good way to get a nack.
Quoted 4 lines
The current semantics are that this file produces a new random UUID each time the host OS is booted. Obviously each time we start a container now, they just see the host's random
boot_id, so from a container's POV this does not change each time it starts.
Quoted 7 lines
That is correct. As I recall the contract with
boot_idis to provide a unique per boot value to assist in dealing with boots etc. I seem to recall emacs uses the combination of hostname+boot_id to help generate unique lock files names.I would definitely need a refresher on how
boot_idis used in practice by applications other than SystemD before I could suggest a good design.
This post seems to describe what emacs wants boot_id for:
http://marc.info/?l=linux-kernel&m=93613053109494&w=2
With this info, I think emacs inside a container would expect the boot_id to change each time the container is started, so they can detect stale locks from an emacs instance in a previous boot of the container.
Daniel
Signature · 4 lines
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 28 lines · Daniel P. Berrange
Quoted 19 lines · Eric W. Biederman
Quoted 3 lines · Daniel P. Berrange
One of the features that SystemD folks have asked us to fix in LXC, is to make sure that
/proc/sys/kernel/random/boot_idchanges each time a container is started.There may be a good reason for this. Most of the time what I have seen of kernel requests from the direction of SystemD is that while there may be a real problem but usually their imagined solution is not a particularly good solution. So a description of the problem is needed.
Justifying something with just SystemD wants this is a good way to get a nack.
Quoted 4 lines
The current semantics are that this file produces a new random UUID each time the host OS is booted. Obviously each time we start a container now, they just see the host's random
boot_id, so from a container's POV this does not change each time it starts.Quoted 7 lines
That is correct. As I recall the contract with
boot_idis to provide a unique per boot value to assist in dealing with boots etc. I seem to recall emacs uses the combination of hostname+boot_id to help generate unique lock files names.I would definitely need a refresher on how
boot_idis used in practice by applications other than SystemD before I could suggest a good design.
Quoted 7 lines
This post seems to describe what emacs wants
boot_idfor:http://marc.info/?l=linux-kernel&m=93613053109494&w=2
With this info, I think emacs inside a container would expect the
boot_idto change each time the container is started, so they can detect stale locks from an emacs instance in a previous boot of the container.
Thanks that patch does clarify the original purpose.
Unfortunately the lines of communication were crossed because emacs 24.1 most certainly does not use /proc/sys/kernel/random/boot_id.
Eric
Quoted 33 lines · Eric W. Biederman
Quoted 3 lines · Daniel P. Berrange
One of the features that SystemD folks have asked us to fix in LXC, is to make sure that
/proc/sys/kernel/random/boot_idchanges each time a container is started.There may be a good reason for this. Most of the time what I have seen of kernel requests from the direction of SystemD is that while there may be a real problem but usually their imagined solution is not a particularly good solution. So a description of the problem is needed.
Justifying something with just SystemD wants this is a good way to get a nack.
Quoted 4 lines
The current semantics are that this file produces a new random UUID each time the host OS is booted. Obviously each time we start a container now, they just see the host's random
boot_id, so from a container's POV this does not change each time it starts.That is correct. As I recall the contract with
boot_idis to provide a unique per boot value to assist in dealing with boots etc. I seem to recall emacs uses the combination of hostname+boot_id to help generate unique lock files names.I would definitely need a refresher on how
boot_idis used in practice by applications other than SystemD before I could suggest a good design.There is also a question of uptime.
Quoted 3 lines
There seems to be general agreement that, aside from the PID directories, changes to data in proc should be done by a FUSE filesystem overlay of some kind.
Quoted 6 lines
No. I have yet to see a justification for using FUSE in containers on top of proc files.
I have seen a lot of bad ideas suggested like hacking
/proc/cpuinfoinstead of providing a proper mechanism to tell applications how parallel they can/should be.
Should we be talking about a new set of library functions to gather info like available memory and cpus, etc? The library functions could internally take into account the per-host procfiles, cgroup info, and, as the need arises, new sources of information.
-serge
Quoted 37 lines · Serge Hallyn
Quoted 36 lines · Eric W. Biederman
Quoted 3 lines · Daniel P. Berrange
One of the features that SystemD folks have asked us to fix in LXC, is to make sure that
/proc/sys/kernel/random/boot_idchanges each time a container is started.There may be a good reason for this. Most of the time what I have seen of kernel requests from the direction of SystemD is that while there may be a real problem but usually their imagined solution is not a particularly good solution. So a description of the problem is needed.
Justifying something with just SystemD wants this is a good way to get a nack.
Quoted 4 lines
The current semantics are that this file produces a new random UUID each time the host OS is booted. Obviously each time we start a container now, they just see the host's random
boot_id, so from a container's POV this does not change each time it starts.That is correct. As I recall the contract with
boot_idis to provide a unique per boot value to assist in dealing with boots etc. I seem to recall emacs uses the combination of hostname+boot_id to help generate unique lock files names.I would definitely need a refresher on how
boot_idis used in practice by applications other than SystemD before I could suggest a good design.There is also a question of uptime.
Quoted 3 lines
There seems to be general agreement that, aside from the PID directories, changes to data in proc should be done by a FUSE filesystem overlay of some kind.
No. I have yet to see a justification for using FUSE in containers on top of proc files.
Quoted 9 lines
Quoted 3 lines
I have seen a lot of bad ideas suggested like hacking
/proc/cpuinfoinstead of providing a proper mechanism to tell applications how parallel they can/should be.Should we be talking about a new set of library functions to gather info like available memory and cpus, etc? The library functions could internally take into account the per-host procfiles, cgroup info, and, as the need arises, new sources of information.
I think this makes a lot of sense.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 42 lines · Glauber Costa
Quoted 41 lines · Serge Hallyn
Quoted 39 lines · Eric W. Biederman
Quoted 3 lines · Daniel P. Berrange
One of the features that SystemD folks have asked us to fix in LXC, is to make sure that
/proc/sys/kernel/random/boot_idchanges each time a container is started.There may be a good reason for this. Most of the time what I have seen of kernel requests from the direction of SystemD is that while there may be a real problem but usually their imagined solution is not a particularly good solution. So a description of the problem is needed.
Justifying something with just SystemD wants this is a good way to get a nack.
Quoted 4 lines
The current semantics are that this file produces a new random UUID each time the host OS is booted. Obviously each time we start a container now, they just see the host's random
boot_id, so from a container's POV this does not change each time it starts.That is correct. As I recall the contract with
boot_idis to provide a unique per boot value to assist in dealing with boots etc. I seem to recall emacs uses the combination of hostname+boot_id to help generate unique lock files names.I would definitely need a refresher on how
boot_idis used in practice by applications other than SystemD before I could suggest a good design.There is also a question of uptime.
Quoted 3 lines
There seems to be general agreement that, aside from the PID directories, changes to data in proc should be done by a FUSE filesystem overlay of some kind.
No. I have yet to see a justification for using FUSE in containers on top of proc files.
I have seen a lot of bad ideas suggested like hacking
/proc/cpuinfoinstead of providing a proper mechanism to tell applications how parallel they can/should be.
Quoted 7 lines
Quoted 5 lines
Should we be talking about a new set of library functions to gather info like available memory and cpus, etc? The library functions could internally take into account the per-host procfiles, cgroup info, and, as the need arises, new sources of information.
I think this makes a lot of sense.
Cool. I can't do it today, but it might be worth starting a list (on wiki?) of information userspace wants (uptime, cpuinfo, meminfo, etc) so we can come up with a reasonable api.
-serge
Quoted 40 lines · Eric W. Biederman
Quoted 3 lines · Daniel P. Berrange
One of the features that SystemD folks have asked us to fix in LXC, is to make sure that
/proc/sys/kernel/random/boot_idchanges each time a container is started.There may be a good reason for this. Most of the time what I have seen of kernel requests from the direction of SystemD is that while there may be a real problem but usually their imagined solution is not a particularly good solution. So a description of the problem is needed.
Justifying something with just SystemD wants this is a good way to get a nack.
Quoted 4 lines
The current semantics are that this file produces a new random UUID each time the host OS is booted. Obviously each time we start a container now, they just see the host's random
boot_id, so from a container's POV this does not change each time it starts.That is correct. As I recall the contract with
boot_idis to provide a unique per boot value to assist in dealing with boots etc. I seem to recall emacs uses the combination of hostname+boot_id to help generate unique lock files names.I would definitely need a refresher on how
boot_idis used in practice by applications other than SystemD before I could suggest a good design.There is also a question of uptime.
Quoted 3 lines
There seems to be general agreement that, aside from the PID directories, changes to data in proc should be done by a FUSE filesystem overlay of some kind.
No. I have yet to see a justification for using FUSE in containers on top of proc files.
I have seen a lot of bad ideas suggested like hacking
/proc/cpuinfoinstead of providing a proper mechanism to tell applications how parallel they can/should be.
Quoted 6 lines
For hacks and controversial ideas FUSE is good because it makes it someone else's problem and it means it isn't something we have to support in the kernel for the indefinite future. At the same time in general a FUSE solution does not really solve anything it just sort of papers over a problem.
Two main points here:
I would love to see this solved transparently by the kernel. In all previous attempts, this seemed to fall under "controversial idea" as you described above.
It is hard to call it "consensus", indeed, but this idea gets quite a lot of traction (I am far from the only one), specifically because it allows us to present userspace with something in the time being. Besides, the interface is just the standard /proc interface. If the kernel eventually does something here, we just have to replace it.
About mechanism for communicating with applications, this is completely orthogonal. Applications will keep requiring the standard interface for a while. We can, however, do a lot better with a library for those, using some other non-standard mechanism that helps containers and non-containers alike. But this is in addition to, at least for the time being.