kdbus for 4.1-rc1
Quoted 7 lines · Havoc Pennington
btw if I can make a suggestion, it's quite confusing to talk about "dbus" unqualified when we are talking about implementation issues, since it muddles bus daemon vs. clients, and also since there are lots of implementations of the client bindings:
Quoted 8 lines
For the bus daemon, the only two implementations I know of are the original one (which uses libdbus as its binding) and kdbus, though.
I would expect there's no question the bus daemon can be faster, maybe say 1.5x raw sockets instead of 2.5x, or whatever - something on that order. Should probably simply stipulate this for discussion purposes:
"someone could optimize the crap out of the bus daemon". The kdbus question is about whether to eliminate this daemon entirely.
So the question is if one of the justifications for moving the daemon into kernel space is that it's performance is crap, then I think it is useful to determine whether a fully optimized userspace daemon would be good enough.
After all, we can go down the Novell Netware path and push arbitrary web servers, ldap servers, etc. all into the kernel on the excuse of "the performance would be faster". But that begs the question of how much performance improvements can be made purely in userspace, and ignores all of the security and stability costs of moving more and more code into the kernel.
So the question I have is why in the world do we want to be able to support 1.5x raw sockets for a bus speed? What's the use case where that kind of performance is required for a bus based system, and is that a world we really want to live in? I find dbus to be extremely hard to debug when my desktop starts doing things I don't want it to do. The fact that it might be flinging around hundreds of thousands of messages, and that this is something we want to encourage, doesn't make me feel any more kindly inclined towards dbus or kdbus....
- Ted
Quoted 5 lines · Theodore Ts'o
So the question is if one of the justifications for moving the daemon into kernel space is that it's performance is crap, then I think it is useful to determine whether a fully optimized userspace daemon would be good enough.
Yeah. I don't know how you answer that, because the answer is probably "it would be good enough for some things and not for other things." It depends on whether an app is sending enough data to be too slow, and it depends on the hardware, right.
What I think we might know: the userspace:kernel time-to-send ratio should always be around 2:1, if both of them are similarly-implemented, because the userspace version has about 2x the work to do.
The actual wall-clock time of course depends on the hardware and what's being sent.
If there was a deviation from 2:1 in a benchmark, it might be because of implementation issues - so for example libdbus+dbus-daemon might be 3:1 or 5:1 to sd-dbus+kdbus, because sd-dbus isn't as bloated as libdbus, say. That isn't telling you anything about kernel vs. userspace architecture, the extra ratio above 2:1 is only telling you about userspace implementation quality.
For purposes of deciding what to put in kernel - the differences between dbus client implementations (sd-dbus, libdbus, gdbus, etc.) seem like irrelevant noise to me.
Re: the slippery slope to LDAP in the kernel - my questions would be things like 1) what are non-performance reasons to have dbus in the kernel, such as early boot or security considerations; 2) does LDAP in kernel give these kind of 2:1 gains; 3) is there a simpler way to get the 2:1 gain for dbus...
Others can answer those better than I can.
I _would_ say that dbus is more "generic" than something like LDAP; dbus is specific to the use-case of coordinating processes on a single machine, but it isn't specific to any particular application, and it's been used for lots of different applications. On my laptop, which is a pretty normal fedora 21 as far as I know:
$ rpm -q --whatrequires 'libdbus-1.so.3()(64bit)' | wc -l 113
this omits anyone using a different binding, it's only libdbus users.
Quoted 3 lines
I find dbus to be extremely hard to debug when my desktop starts doing things I don't want it to do. The fact that it might be flinging around hundreds of thousands of messages, and that this is something we want to encourage,
This particular argument doesn't resonate with me ... if dbus is hard to debug, it's not as if "ad hoc application-specific sidechannel somebody cooked up" is going to be easier.
People aren't usually making up data to send around just because they can. If they need to send an audio stream, and dbus is too slow, they'll send it another ad hoc way, but it ultimately has to get sent. Same for most data, it is the size it is and it needs to go where it needs to go, for some what-the-user-wants-to-do kind of reason.
If apps have to, they say "I'm sorry Dave I can't do that - you can't software-decode 4K video on your 300mhz ARM" - of course.
Havoc
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Havoc" == Havoc Pennington <[email protected]> writes:
Quoted 5 lines · Theodore Ts'o
Quoted 5 lines
So the question is if one of the justifications for moving the daemon into kernel space is that it's performance is crap, then I think it is useful to determine whether a fully optimized userspace daemon would be good enough.
Havoc> Yeah. I don't know how you answer that, because the answer is Havoc> probably "it would be good enough for some things and not for Havoc> other things." It depends on whether an app is sending enough Havoc> data to be too slow, and it depends on the hardware, right.
So what happens if we put kdbus into the kernel and it's still too slow? What then?
Havoc> What I think we might know: the userspace:kernel time-to-send Havoc> ratio should always be around 2:1, if both of them are Havoc> similarly-implemented, because the userspace version has about Havoc> 2x the work to do.
I'm not sure I agree with this statement, just putting something into the kernel doesn't magically make the work go away, and the overhead people are talking about won't change if applications and libraries keep opening/closing the connection to the bus all the time.
Havoc> The actual wall-clock time of course depends on the hardware Havoc> and what's being sent.
Havoc> If there was a deviation from 2:1 in a benchmark, it might be Havoc> because of implementation issues - so for example Havoc> libdbus+dbus-daemon might be 3:1 or 5:1 to sd-dbus+kdbus, Havoc> because sd-dbus isn't as bloated as libdbus, say. That isn't Havoc> telling you anything about kernel vs. userspace architecture, Havoc> the extra ratio above 2:1 is only telling you about userspace Havoc> implementation quality.
Which is also telling you that maybe userspace could be improved more, before it needs to even think about going into the kernel?
Havoc> For purposes of deciding what to put in kernel - the Havoc> differences between dbus client implementations (sd-dbus, Havoc> libdbus, gdbus, etc.) seem like irrelevant noise to me.
Havoc> Re: the slippery slope to LDAP in the kernel - my questions Havoc> would be things like 1) what are non-performance reasons to Havoc> have dbus in the kernel, such as early boot or security Havoc> considerations; 2) does LDAP in kernel give these kind of 2:1 Havoc> gains; 3) is there a simpler way to get the 2:1 gain for Havoc> dbus...
Havoc> Others can answer those better than I can.
Havoc> I _would_ say that dbus is more "generic" than something like Havoc> LDAP; dbus is specific to the use-case of coordinating Havoc> processes on a single machine, but it isn't specific to any Havoc> particular application, and it's been used for lots of Havoc> different applications. On my laptop, which is a pretty normal Havoc> fedora 21 as far as I know:
LDAP is pretty damn generic, in that you can put pretty large objects into it, and pretty large OUs, etc. So why would it be a candidate for going into the kernel? And why is kdbus so important in the kernel as well? People have talked about it needing to be there for bootup, but isn't that why we ripped out RAID detection and such from the kernel and built initramfs, so that there's LESS in the kernel, and more in an early userspace? Same idea with dbus in my opinion.
Havoc> $ rpm -q --whatrequires 'libdbus-1.so.3()(64bit)' | wc -l Havoc> 113
Havoc> this omits anyone using a different binding, it's only libdbus users.
Quoted 3 lines
Quoted 3 lines
I find dbus to be extremely hard to debug when my desktop starts doing things I don't want it to do. The fact that it might be flinging around hundreds of thousands of messages, and that this is something we want to encourage,
Havoc> This particular argument doesn't resonate with me ... if dbus Havoc> is hard to debug, it's not as if "ad hoc application-specific Havoc> sidechannel somebody cooked up" is going to be easier.
When Ted is saying it's hard to debug... then maybe it's a bit crappy in design or implementation?
Havoc> People aren't usually making up data to send around just because they Havoc> can. If they need to send an audio stream, and dbus is too slow, Havoc> they'll send it another ad hoc way, but it ultimately has to get sent. Havoc> Same for most data, it is the size it is and it needs to go where it Havoc> needs to go, for some what-the-user-wants-to-do kind of reason.
Havoc> If apps have to, they say "I'm sorry Dave I can't do that - you Havoc> can't software-decode 4K video on your 300mhz ARM" - of course.
So why DOES audio need to go via DBUS? What about video? Why shouldn't that go via dbus as well?
If one userspace implementation is so crappy, why can't that implementation be tossed and a better one done? Or why can't they just optimize/tune it in userspace instead?
John
Quoted 7 lines · John Stoffel
Havoc> Yeah. I don't know how you answer that, because the answer is Havoc> probably "it would be good enough for some things and not for Havoc> other things." It depends on whether an app is sending enough Havoc> data to be too slow, and it depends on the hardware, right.
So what happens if we put kdbus into the kernel and it's still too slow? What then?
What my above paragraph was intended to mean is: I don't understand what it means to ask about a "too slow" fixed line here. Every time you make it substantively faster, it works for more apps or on slower hardware, presumably. You dial the speed, and you include or exclude certain app ideas accordingly.
I think dbus works for lots of purposes now, despite being slow. Lots of people are using it. In many uses, super-slow-dbus might be 1% of the profile of whatever the user-visible functionality is, and nobody cares how fast dbus is. In other uses, they might.
Some people are saying they would use it in more ways if it were faster and/or available in early boot and/or whatever else. I'm not those people, because right now I'm not working on dbus or anything using dbus. They would have to say what's 'fast enough' for them.
"What happens if unix sockets are too slow? what then?" - it's not a coherent question. It's always relative to what you're trying to do, surely.
Quoted 2 lines
I'm not sure I agree with this statement, just putting something into the kernel doesn't magically make the work go away
The kdbus guys should really explain this. I have my understanding of it but theirs will be more accurate.
Quoted 2 lines
Which is also telling you that maybe userspace could be improved more, before it needs to even think about going into the kernel?
I imagine people have already improved the part of userspace they are thinking of keeping (sd-dbus, replacing libdbus) and they don't want to rewrite dbus-daemon only to immediately discard it. (The part of the "dbus" overall system which hasn't been rewritten and optimized is the daemon, which could be dropped completely in kdbus-world.)
It's not especially mysterious what's slow about the existing daemon implementation, in my opinion; it's been the same for 10 years. The rough outline of speeding it up would be to replace libdbus with something sd-dbus-like, and then do a round of profiling and tuning. The 2012 email I linked to earlier had some other ideas. But this is a lot of work, it isn't "just" port to sd-dbus, the daemon is strongly entangled with libdbus right now. I don't blame people for being unmotivated on this if they believe it's a dead end.
In that same 2012 email you'll notice I advised doing exactly what Linus suggests; do the userspace tuning rather than quote "arguing with kernel developers":
http://lists.freedesktop.org/archives/dbus/2012-March/015024.html
But I do admire that people felt kdbus was the right answer so have gone for it anyway, and I do think Linux as a complete OS (kernel+userspace) deserves a great answer in this problem space.
Quoted 8 lines
LDAP is pretty damn generic, in that you can put pretty large objects into it, and pretty large OUs, etc. So why would it be a candidate for going into the kernel? And why is kdbus so important in the kernel as well? People have talked about it needing to be there for bootup, but isn't that why we ripped out RAID detection and such from the kernel and built initramfs, so that there's LESS in the kernel, and more in an early userspace? Same idea with dbus in my opinion.
I don't have a well-developed philosophy on what should be in the kernel or not. That is something the kernel maintainers have to answer. My main concern here is that people understand what dbus is about historically, so they don't do silly stuff - whether cargo cult keeping a 'feature' that was always a bad idea, or speeding it up by breaking intentional and important semantics, or whatever.
When I see people saying they don't understand what dbus is because they have no idea how a Linux workstation userspace is put together, that's something I can help with.
When I see people saying maybe it isn't worth the complexity to put this in the kernel if it's only an N% speedup, I can see that, I'm not going to say that's wrong or right. It depends to me on what apps are enabled by the N%, or whether early boot and other factors are important.
Quoted 2 lines
When Ted is saying it's hard to debug... then maybe it's a bit crappy in design or implementation?
Or maybe he just doesn't know how to debug it, honestly. I find the kernel hard to debug because I know very little about it. I find the desktop simple to debug, at least as simple as debugging millions of lines of code can be. The difference is that I have never done kernel debugging and I'm already familiar with how the desktop works.
dbus has tools that log every message and let you explore and introspect everything on it, etc. - it works for me.
Quoted 6 lines
So why DOES audio need to go via DBUS? What about video? Why shouldn't that go via dbus as well?
If one userspace implementation is so crappy, why can't that implementation be tossed and a better one done? Or why can't they just optimize/tune it in userspace instead?
In this email I listed what I could remember app developers bringing up when told to use a sidechannel instead of dbus:
http://article.gmane.org/gmane.linux.kernel/1935002
I can't speak to what makes sense for audio or video, but I'm sure people who work on those things could.
Re: why can't it be done in userspace, the only thing I'd repeat again here is that when people mention ways to speed up the bus daemon in userspace, they often sound like they would abandon one or more of the semantic guarantees of dbus (usually ordering, sometimes things like the guaranteed-correct sender information or whatever). And _maybe_ some of those guarantees are worth abandoning, but I'd be very careful with it.
Havoc
Quoted 7 lines · John Stoffel
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Havoc" == Havoc Pennington <[email protected]> writes:
Quoted 3 lines · Theodore Ts'o
Quoted 3 lines
I find dbus to be extremely hard to debug when my desktop starts doing things I don't want it to do. The fact that it might be flinging around hundreds of thousands of messages, and that this is something we want to encourage,
Quoted 6 lines
Havoc> This particular argument doesn't resonate with me ... if dbus Havoc> is hard to debug, it's not as if "ad hoc application-specific Havoc> sidechannel somebody cooked up" is going to be easier.
When Ted is saying it's hard to debug... then maybe it's a bit crappy in design or implementation?
There is a very nice tool to debug the traffic for kdbus.
http://lists.freedesktop.org/archives/dbus/2014-March/016178.html
Also the patched wireshark makes it as easy as analyzing network traffic.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Quoted 7 lines · John Stoffel
LDAP is pretty damn generic, in that you can put pretty large objects into it, and pretty large OUs, etc. So why would it be a candidate for going into the kernel? And why is kdbus so important in the kernel as well? People have talked about it needing to be there for bootup, but isn't that why we ripped out RAID detection and such from the kernel and built initramfs, so that there's LESS in the kernel, and more in an early userspace? Same idea with dbus in my opinion.
Let me elaborate on the initramfs/shutdown situation a little bit more, because I have to deal with that every day.
Because of the "let's move everything to userspace" sentiment we nowadays have the situation, that we need a lot of tools to setup the root device.
Be it LVM on IMSM or iSCSI multipath, the initramfs has to setup the network (with bridging, bonding, etc.), the iSCSI connection, assemble the raid, the LVM, open crypto devices, etc...
And if something goes wrong, you want to have a shell, see all the logs and debug things.
Now over the time we moved away from simple shell scripts (without any logging) and static compiled special versions for the initramfs to a mini distribution in the initramfs, which simplifies maintenance and improves reliability.
Basically you want to use the same tools in the initramfs (and shutdown) which you already have and use in your real root, with the same configuration files and the same interfaces and the same code paths.
Therefore systemd is started in dracut created initramfs, which starts journald for logging. The same basic systemd targets exist in the initramfs as on the real root, so normally you don't have to cope with specialized versions for the initramfs.
The target here is to have the same IPC mechanism from the very beginning to the very end. No crappy fallback mechanisms in case a daemon is not running or has crashed, no creepy transition from initramfs root to real root to shutdown root.
We already have such transitions like: systemd, journald, mdmon [1], etc. systemd has to serialize itself, journald's file descriptors are transitioned over, mdmon jumps through hoops. Remember you want to get rid of open files and executables and have to reexec everything, if you transition from the initramfs root to the real root, and also from the real root to the shutdown root.
We really don't want the IPC mechanism to be in a flux state. All tools have to fallback to a non-standard mechanism in that case.
If I have to pull in a dbus daemon in the initramfs, we still have the chicken and egg problem for PID 1 talking to the logging daemon and starting dbus.
systemd cannot talk to journald via dbus unless dbus-daemon is started, dbus cannot log anything on startup, if journald is not running, etc...
dbus-daemon would have to transition to the real root, and from the real root to the shutdown root, without losing state.
Of course this can all be done, but it would involve fallback mechanisms, which we want to get rid off. Hopefully, you don't suggest to merge dbus with PID 1. Also with a daemon, you will lose the points mentioned in the cover mail :
* Security: The peers which communicate do not have to trust each other, as the only trustworthy component in the game is the kernel which adds metadata and ensures that all data passed as payload is either copied or sealed, so that the receiver can parse the data without having to protect against changing memory while parsing buffers. Also, all the data transfer is controlled by the kernel, so that LSMs can track and control what is going on, without involving userspace. Because of the LSM issue, security people are much happier with this model than the current scheme of having to hook into dbus to mediate things.
* Being in the kernel closes a lot of races which can't be fixed with the current userspace solutions. For example, with kdbus, there is a way a client can disconnect from a bus, but do so only if no further messages present in its queue, which is crucial for implementing race-free "exit-on-idle" services
* Eavesdropping on the kernel level, so privileged users can hook into the message stream without hacking support for that into their userspace processes
* A number of smaller benefits: for example kdbus learned a way to peek full messages without dequeing them, which is really useful for logging metadata when handling bus-activation requests.
I don't care, if the kdbus speedup is only marginal.
In my ideal world, there is a standard IPC mechanism from the beginning to the end, which does not rely on any process running (except the kernel) and which is used by _all_ tools, be it a system daemon providing information and interfaces about device assembly or network setup tools or end user desktop processes.
dbus _is_ such an easy, flexible standard IPC mechanism. Of course, you can invent the wheel again (NIH, "we know better") and wait and see, if that works out. Until then the whole common IPC problem is unresolved and Linux distributions are just a collection of random software with no common interoperability and home grown interfaces.
[1] transitioning mdmon is one of the critical parts for an IMSM raid array. Also running an executable from a disk, which the executable is monitoring, and which stops functioning, if the executable is not responding is insane.
Thanks for reading...
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQIcBAEBCAAGBQJVQNL1AAoJEANOs3ABTfJw0BUQAJgj2RNKR8L7xVPwH2GovmST nioOl6sg9u2m8NgYM6TJUJI3yHbOOiRVCTXHb9fmkTk/hBDxsT+X4lFevh0mDLJu Y5bk1RwGn8Ail3GLR6il9RhlNMKEqN2Ik4Ey26IdxQkOhIIAy9IfrNBdsdoNpJ7I P7qhP8J1DKfmIlgryrXy/mTZ1Nl1m6UlpMZHDSqlnPWuT/iJn0wORbs319fgAQx/ kkPvgSqTGkDetHGNzYmghgRzimNBR5ZftH0HS3Chq6rXPiSbdct/dE8VkQRiEWYo k6tE83qJr9KbSdBFqnbznVaOpTCQatdanVPBzzz4DTkuSKBlAxIbdXRaFsJCSnKp 7r+h8q+AgdALJXEyx5AyBeh8/dK1a/PsMzOtYZg6FXAz211geTxHeY8bTdOrzys9 kJGwUbbq4rIyvseEl53+Ugh2qZQptDKCj6F46H3iuhsOyUbPXzg1E7K8w2gApwSY L/eLEcQw+TApULyEhDrQqXlFBPz4vFP38mHNQ6T1Yt3sJuVoU12dOQNN6836htpe h4ijpaTbUkFV8b/7xgGqOlSBio4iSppybXfiBtHT7NBa4da1L+WG0xT+nR8RSMxd Gblt9ECZmbay6SIMYQBhntZD5Hs76iSJl0j2i9zg8E1pBw8O5w0jvlA02fOz2pkp wQsPrxNdlkBxFHVtf/3V
=Dc23
-----END PGP SIGNATURE-----
Quoted 15 lines · Harald Hoyer
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256Quoted 7 lines · John Stoffel
LDAP is pretty damn generic, in that you can put pretty large objects into it, and pretty large OUs, etc. So why would it be a candidate for going into the kernel? And why is kdbus so important in the kernel as well? People have talked about it needing to be there for bootup, but isn't that why we ripped out RAID detection and such from the kernel and built initramfs, so that there's LESS in the kernel, and more in an early userspace? Same idea with dbus in my opinion.
Let me elaborate on the initramfs/shutdown situation a little bit more, because I have to deal with that every day.
Quoted 8 lines
Because of the "let's move everything to userspace" sentiment we nowadays have the situation, that we need a lot of tools to setup the root device.
Be it LVM on IMSM or iSCSI multipath, the initramfs has to setup the network (with bridging, bonding, etc.), the iSCSI connection, assemble the raid, the LVM, open crypto devices, etc...
And if something goes wrong, you want to have a shell, see all the logs and debug things.
None of these tools depend on dbus (_desktop_ bus).
Quoted 26 lines
Now over the time we moved away from simple shell scripts (without any logging) and static compiled special versions for the initramfs to a mini distribution in the initramfs, which simplifies maintenance and improves reliability.
Basically you want to use the same tools in the initramfs (and shutdown) which you already have and use in your real root, with the same configuration files and the same interfaces and the same code paths.
Therefore systemd is started in dracut created initramfs, which starts journald for logging. The same basic systemd targets exist in the initramfs as on the real root, so normally you don't have to cope with specialized versions for the initramfs.
The target here is to have the same IPC mechanism from the very beginning to the very end. No crappy fallback mechanisms in case a daemon is not running or has crashed, no creepy transition from initramfs root to real root to shutdown root.
We already have such transitions like: systemd, journald, mdmon [1], etc. systemd has to serialize itself, journald's file descriptors are transitioned over, mdmon jumps through hoops. Remember you want to get rid of open files and executables and have to reexec everything, if you transition from the initramfs root to the real root, and also from the real root to the shutdown root.
Quoted 8 lines
We really don't want the IPC mechanism to be in a flux state. All tools have to fallback to a non-standard mechanism in that case.
If I have to pull in a dbus daemon in the initramfs, we still have the chicken and egg problem for PID 1 talking to the logging daemon and starting dbus.
systemd cannot talk to journald via dbus unless dbus-daemon is started, dbus cannot log anything on startup, if journald is not running, etc...
The only reason why you need dbus in your initramfs is because of systemd and its tools, isn't it?
Quoted 5 lines
In my ideal world, there is a standard IPC mechanism from the beginning to the end, which does not rely on any process running (except the kernel) and which is used by _all_ tools, be it a system daemon providing information and interfaces about device assembly or network setup tools or end user desktop processes.
It depends how you define "beginning". To me an initramfs is a *very* minimal tool to prepare the rootfs and nothing more (no udev, no systemd, no "mini distro").
If the initramfs fails to do its job it can print to the console like the kernel does if it fails
at a very early stage.
Signature · 2 lines
Thanks, //richard
Quoted 7 lines · Richard Weinberger
It depends how you define "beginning". To me an initramfs is a *very* minimal tool to prepare the rootfs and nothing more (no udev, no systemd, no "mini distro").
If the initramfs fails to do its job it can print to the console like the kernel does if it fails
at a very early stage.
Your solution might work for your small personal needs, but not for our customers.
Am 29.04.2015 um 15:38 schrieb Harald Hoyer:
Quoted 10 lines · Richard Weinberger
Quoted 7 lines
It depends how you define "beginning". To me an initramfs is a *very* minimal tool to prepare the rootfs and nothing more (no udev, no systemd, no "mini distro").
If the initramfs fails to do its job it can print to the console like the kernel does if it fails
at a very early stage.Your solution might work for your small personal needs, but not for our customers.
Correct, I don't know your customers, all I know are my customers. :-)
What feature do your customers need?
I mean, I fully agree with you that an initramfs must not fail silently but how does dbus help there? If it fails to mount the rootfs there is not much it can do.
Thanks,
//richard
Quoted 14 lines · Richard Weinberger
Am 29.04.2015 um 15:38 schrieb Harald Hoyer:
Quoted 10 lines · Richard Weinberger
Quoted 7 lines
It depends how you define "beginning". To me an initramfs is a *very* minimal tool to prepare the rootfs and nothing more (no udev, no systemd, no "mini distro").
If the initramfs fails to do its job it can print to the console like the kernel does if it fails
at a very early stage.Your solution might work for your small personal needs, but not for our customers.
Correct, I don't know your customers, all I know are my customers. :-)
Quoted 8 lines
What feature do your customers need?
I mean, I fully agree with you that an initramfs must not fail silently but how does dbus help there? If it fails to mount the rootfs there is not much it can do.Thanks,
//richard
We don't handcraft the initramfs script for every our customers, therefore we have to generically support hotplug, persistent device names, persistent interface names, network connectivity in the initramfs, user input handling for passwords, fonts, keyboard layouts, fips, fsck, repair tools for file systems, raid assembly, LVM assembly, multipath, crypto devices, live images, iSCSI, FCoE, all kinds of filesystems with their quirks, IBM z-series support, resume from hibernation, […]
And no, this is not a simple minimal tool.
Am 29.04.2015 um 16:01 schrieb Harald Hoyer:
Quoted 24 lines · Richard Weinberger
Quoted 14 lines
Am 29.04.2015 um 15:38 schrieb Harald Hoyer:
Quoted 10 lines · Richard Weinberger
Quoted 7 lines
It depends how you define "beginning". To me an initramfs is a *very* minimal tool to prepare the rootfs and nothing more (no udev, no systemd, no "mini distro").
If the initramfs fails to do its job it can print to the console like the kernel does if it fails
at a very early stage.Your solution might work for your small personal needs, but not for our customers.
Correct, I don't know your customers, all I know are my customers. :-)
Quoted 8 lines
What feature do your customers need?
I mean, I fully agree with you that an initramfs must not fail silently but how does dbus help there? If it fails to mount the rootfs there is not much it can do.Thanks,
//richard
Quoted 7 lines
We don't handcraft the initramfs script for every our customers, therefore we have to generically support hotplug, persistent device names, persistent interface names, network connectivity in the initramfs, user input handling for passwords, fonts, keyboard layouts, fips, fsck, repair tools for file systems, raid assembly, LVM assembly, multipath, crypto devices, live images, iSCSI, FCoE, all kinds of filesystems with their quirks, IBM z-series support, resume from hibernation, […]
This is correct. But which of these tools/features depend on dbus?
Thanks,
//richard
P.s: Please don't drop the CC list.
Quoted 33 lines · Richard Weinberger
Am 29.04.2015 um 16:01 schrieb Harald Hoyer:
Quoted 24 lines · Richard Weinberger
Quoted 14 lines
Am 29.04.2015 um 15:38 schrieb Harald Hoyer:
Quoted 10 lines · Richard Weinberger
Quoted 7 lines
It depends how you define "beginning". To me an initramfs is a *very* minimal tool to prepare the rootfs and nothing more (no udev, no systemd, no "mini distro").
If the initramfs fails to do its job it can print to the console like the kernel does if it fails
at a very early stage.Your solution might work for your small personal needs, but not for our customers.
Correct, I don't know your customers, all I know are my customers. :-)
Quoted 8 lines
What feature do your customers need?
I mean, I fully agree with you that an initramfs must not fail silently but how does dbus help there? If it fails to mount the rootfs there is not much it can do.Thanks,
//richardQuoted 7 lines
We don't handcraft the initramfs script for every our customers, therefore we have to generically support hotplug, persistent device names, persistent interface names, network connectivity in the initramfs, user input handling for passwords, fonts, keyboard layouts, fips, fsck, repair tools for file systems, raid assembly, LVM assembly, multipath, crypto devices, live images, iSCSI, FCoE, all kinds of filesystems with their quirks, IBM z-series support, resume from hibernation, […]
Quoted 1 line
This is correct. But which of these
tools/featuresdepend on dbus?
I would love to add dbus support to all of them and use it, so I can connect them all more easily. No need for them to invent their own version of IPC, which can only be used by their own tool set.
Am 29.04.2015 um 16:11 schrieb Harald Hoyer:
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
We don't handcraft the initramfs script for every our customers, therefore we have to generically support hotplug, persistent device names, persistent interface names, network connectivity in the initramfs, user input handling for passwords, fonts, keyboard layouts, fips, fsck, repair tools for file systems, raid assembly, LVM assembly, multipath, crypto devices, live images, iSCSI, FCoE, all kinds of filesystems with their quirks, IBM z-series support, resume from hibernation, […]
Quoted 5 lines
Quoted 1 line
This is correct. But which of these
tools/featuresdepend on dbus?I would love to add dbus support to all of them and use it, so I can connect them all more easily. No need for them to invent their own version of IPC, which can only be used by their own tool set.
Why/how do you need to connect them?
Sorry for being persistent but as I use most of these tools too (also in initramfs) I'm very curious.
Many of us grumpy kernel devs simply don't know all the use case of you have to cover. So, please explain. :-)
Thanks,
//richard
Quoted 15 lines · Richard Weinberger
Am 29.04.2015 um 16:11 schrieb Harald Hoyer:
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
We don't handcraft the initramfs script for every our customers, therefore we have to generically support hotplug, persistent device names, persistent interface names, network connectivity in the initramfs, user input handling for passwords, fonts, keyboard layouts, fips, fsck, repair tools for file systems, raid assembly, LVM assembly, multipath, crypto devices, live images, iSCSI, FCoE, all kinds of filesystems with their quirks, IBM z-series support, resume from hibernation, […]
Quoted 5 lines
Quoted 1 line
This is correct. But which of these
tools/featuresdepend on dbus?I would love to add dbus support to all of them and use it, so I can connect them all more easily. No need for them to invent their own version of IPC, which can only be used by their own tool set.
Quoted 7 lines
Why/how do you need to connect them?
Sorry for being persistent but as I use most of these tools too (also in initramfs) I'm very curious.Many of us grumpy kernel devs simply don't know all the use case of you have to cover. So, please explain. :-)
Well, using shell scripts I connected all of these tools in the earlier versions of dracut [1]. Been there, done that.
When using bash to wait for an interface to come up [2] or doing dhcp [3], the (at least my) pain threshold is reached, and you want something more sophisticated.
So, one starts eyeing NetworkManager or systemd-networkd. Both of them have CLI tools and helpers and these tools and helpers talk to each other with (guess what?) an IPC mechanism, which happens to be DBUS (because it's the IPC of choice, if you don't want to reinvent the wheel).
But let's not pinpoint that to network alone. Parsing output of tools with shell scripts is horrible, slow, fragile, error prone.
Sure, I can write one binary to rule them all, pull out all the code from all tools I need, but for me an IPC mechanism sounds a lot better. And it should be _one_ common IPC mechanism and not a plethora of them. It should feel like an operating system and not like a bunch of thrown together software, which is glued together with some magic shell scripts.
[1] https://dracut.wiki.kernel.org/index.php/Main_Page
[2]
http://git.kernel.org/cgit/boot/dracut/dracut.git/tree/modules.d/40network/net-lib.sh#n483
[3] http://pkgs.fedoraproject.org/cgit/dhcp.git/tree/dhclient-script
Am 29.04.2015 um 16:53 schrieb Harald Hoyer:
Quoted 30 lines · Richard Weinberger
Quoted 15 lines
Am 29.04.2015 um 16:11 schrieb Harald Hoyer:
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
We don't handcraft the initramfs script for every our customers, therefore we have to generically support hotplug, persistent device names, persistent interface names, network connectivity in the initramfs, user input handling for passwords, fonts, keyboard layouts, fips, fsck, repair tools for file systems, raid assembly, LVM assembly, multipath, crypto devices, live images, iSCSI, FCoE, all kinds of filesystems with their quirks, IBM z-series support, resume from hibernation, […]
Quoted 5 lines
Quoted 1 line
This is correct. But which of these
tools/featuresdepend on dbus?I would love to add dbus support to all of them and use it, so I can connect them all more easily. No need for them to invent their own version of IPC, which can only be used by their own tool set.
Quoted 7 lines
Why/how do you need to connect them?
Sorry for being persistent but as I use most of these tools too (also in initramfs) I'm very curious.Many of us grumpy kernel devs simply don't know all the use case of you have to cover. So, please explain. :-)
Well, using shell scripts I connected all of these tools in the earlier versions of dracut [1]. Been there, done that.
When using bash to wait for an interface to come up [2] or doing dhcp [3], the (at least my) pain threshold is reached, and you want something more sophisticated.
Quoted 7 lines
So, one starts eyeing NetworkManager or systemd-networkd. Both of them have CLI tools and helpers and these tools and helpers talk to each other with (guess what?) an IPC mechanism, which happens to be DBUS (because it's the IPC of choice, if you don't want to reinvent the wheel).
But let's not pinpoint that to network alone. Parsing output of tools with shell scripts is horrible, slow, fragile, error prone.
So, you want to replace bash by dbus?
I'll stop now with arguing.
Let's agree to disagree.
Quoted 5 lines
Sure, I can write one binary to rule them all, pull out all the code from all tools I need, but for me an IPC mechanism sounds a lot better. And it should be _one_ common IPC mechanism and not a plethora of them. It should feel like an operating system and not like a bunch of thrown together software, which is glued together with some magic shell scripts.
This is how UNIX works. ;)
Thanks,
//richard
Quoted 5 lines · Harald Hoyer
Sure, I can write one binary to rule them all, pull out all the code from all tools I need, but for me an IPC mechanism sounds a lot better. And it should be _one_ common IPC mechanism and not a plethora of them. It should feel like an operating system and not like a bunch of thrown together software, which is glued together with some magic shell scripts.
And so requiring wireshark (and X?) in initramfs to debug problems once dbus is introduced is better?
I would think shell scripts are *easier* to debug when things go wrong, especially in a minimal environment such as an initial ram disk. Having had to debug problems in a distro initramfs when trying to help a customer bring up a FC boot disk long ago in another life, I'm certain I would rather debug problems while on site at a classified machine room[1] using shell scripts, and trying to debug dbus is something that would be infinitely worse.
- Ted
[1] So no laptop, no google, no access to sources to figure out random dbus messages, etc.
Quoted 18 lines · Theodore Ts'o
Quoted 5 lines · Harald Hoyer
Sure, I can write one binary to rule them all, pull out all the code from all tools I need, but for me an IPC mechanism sounds a lot better. And it should be _one_ common IPC mechanism and not a plethora of them. It should feel like an operating system and not like a bunch of thrown together software, which is glued together with some magic shell scripts.
And so requiring wireshark (and X?) in initramfs to debug problems once dbus is introduced is better?
I would think shell scripts are *easier* to debug when things go wrong, especially in a minimal environment such as an initial ram disk. Having had to debug problems in a distro initramfs when trying to help a customer bring up a FC boot disk long ago in another life, I'm certain I would rather debug problems while on site at a classified machine room[1] using shell scripts, and trying to debug dbus is something that would be infinitely worse.
Quoted 4 lines
- Ted
[1] So no laptop, no google, no access to sources to figure out random dbus messages, etc.
Likewise.
I keep hearing from people that shell scripting is hard, it really isn't compared to a number of other scripting languages, you just need to actually learn to do it right (which is getting more and more difficult these days cause fewer and fewer CS schools are teaching Unix).
Quoted 13 lines · when Austin S Hemmelgarn
Quoted 12 lines · Theodore Ts'o
Quoted 5 lines · Harald Hoyer
Sure, I can write one binary to rule them all, pull out all the code from all tools I need, but for me an IPC mechanism sounds a lot better. And it should be _one_ common IPC mechanism and not a plethora of them. It should feel like an operating system and not like a bunch of thrown together software, which is glued together with some magic shell scripts.
And so requiring wireshark (and X?) in initramfs to debug problems once dbus is introduced is better?
I would think shell scripts are *easier* to debug when things go wrong,
[...]
Quoted 5 lines
I keep hearing from people that shell scripting is hard, it really isn't compared to a number of other scripting languages, you just need to actually learn to do it right (which is getting more and more difficult these days cause fewer and fewer CS schools are teaching Unix).
My 2/100 of a currency of your choice.
As much as I like(ed) shell scripts as a boot up tool and disliked obscure boot-up procedures of some operating system, I can't help but notice that GNU/Linux distributions have become very sophisticated/complcated (cross out if not applicable). Personally I feel that this degree of coplexity can't be supported by shell scripts piping data around. It does not scale. I am not 100% sure a new IPC is the answer, simply because I do not have experience to be so. It definitely can be and the problem, as I see it, is real.
(The alternative answer is PowerShells capability to pipe objects. I don't like it and I thik it's not a full answer.)
Regardless, of initrd issues I feel there is a need of a local IPC that is more capable than UDS. Linus Torvalds is probably right that dbus-daemon is everything but effictient. I disagree, however, that it can be optimised and therefore solve *all* issues kdbus is trying to address. dbus-deamon, by design, can't some things. It can't transmitt large payloads without copying them. It can't be made race-free.
Kind regards,
Signature · 3 lines
Łukasz Stelmach Samsung R&D Institute Poland Samsung Electronics
Am 30.04.2015 um 11:05 schrieb Łukasz Stelmach:
Quoted 6 lines
Regardless, of initrd issues I feel there is a need of a local IPC that is more capable than UDS. Linus Torvalds is probably right that dbus-daemon is everything but effictient. I disagree, however, that it can be optimised and therefore solve *all* issues kdbus is trying to address. dbus-deamon, by design, can't some things. It can't transmitt large payloads without copying them. It can't be made race-free.
This is true.
But as long dbus-deamon is not optimized as much as possible there is no reason to force push kdbus.
As soon dbus-deamon exploits all kernel interfaces as much it can and it still needs work (may it performance or other stuff) we can think of new kernel features which can help dbus-deamon.
Thanks,
//richard
Quoted 9 lines · when Richard Weinberger
Am 30.04.2015 um 11:05 schrieb Łukasz Stelmach:
Quoted 7 lines
Regardless, of initrd issues I feel there is a need of a local IPC that is more capable than UDS. Linus Torvalds is probably right that dbus-daemon is everything but effictient. I disagree, however, that it can be optimised and therefore solve *all* issues kdbus is trying to address. dbus-deamon, by design, can't some things. It can't transmitt large payloads without copying them. It can't be made race-free.
Quoted 6 lines
This is true.
But as long dbus-deamon is not optimized as much as possible there is no reason to force push kdbus.
As soon dbus-deamon exploits all kernel interfaces as much it can and it still needs work (may it performance or other stuff) we can think of new kernel features which can help dbus-deamon.
I may not be well informed about kernel interfaces, but there are some use cases no dbus-daemon optimisation can make work properly because of rece-conditons introduced by the user-space based message router.
For example, a service can't aquire credentials of a client process that actually sent a request (it can, but it can't trust them). The service can't be protected by LSM on a bus that is driven by dbus-daemon. Yes, dbus-daemon, can check client's and srevice's labels and enforce a policy but it is going to be the daemon and not the LSM code in the kernel.
Signature · 3 lines
Łukasz Stelmach Samsung R&D Institute Poland Samsung Electronics
Am 30.04.2015 um 12:19 schrieb Łukasz Stelmach:
Quoted 21 lines
Quoted 9 lines · when Richard Weinberger
Am 30.04.2015 um 11:05 schrieb Łukasz Stelmach:
Quoted 7 lines
Regardless, of initrd issues I feel there is a need of a local IPC that is more capable than UDS. Linus Torvalds is probably right that dbus-daemon is everything but effictient. I disagree, however, that it can be optimised and therefore solve *all* issues kdbus is trying to address. dbus-deamon, by design, can't some things. It can't transmitt large payloads without copying them. It can't be made race-free.
Quoted 6 lines
This is true.
But as long dbus-deamon is not optimized as much as possible there is no reason to force push kdbus.
As soon dbus-deamon exploits all kernel interfaces as much it can and it still needs work (may it performance or other stuff) we can think of new kernel features which can help dbus-deamon.I may not be well informed about kernel interfaces, but there are some use cases no dbus-daemon optimisation can make work properly because of rece-conditons introduced by the user-space based message router.
Quoted 6 lines
For example, a service can't aquire credentials of a client process that actually sent a request (it can, but it can't trust them). The service can't be protected by LSM on a bus that is driven by dbus-daemon. Yes, dbus-daemon, can check client's and srevice's labels and enforce a policy but it is going to be the daemon and not the LSM code in the kernel.
That's why I said we can think of new kernel features if they are needed. But they current sink or swim approach of kdbus folks is also not the solution. As I said, if dbus-daemon utilizes the kernel interface as much as possible we can think of new features.
Thanks,
//richard
Quoted 30 lines · when Richard Weinberger
Am 30.04.2015 um 12:19 schrieb Łukasz Stelmach:
Quoted 22 lines
Quoted 9 lines · when Richard Weinberger
Am 30.04.2015 um 11:05 schrieb Łukasz Stelmach:
Quoted 7 lines
Regardless, of initrd issues I feel there is a need of a local IPC that is more capable than UDS. Linus Torvalds is probably right that dbus-daemon is everything but effictient. I disagree, however, that it can be optimised and therefore solve *all* issues kdbus is trying to address. dbus-deamon, by design, can't some things. It can't transmitt large payloads without copying them. It can't be made race-free.
Quoted 6 lines
This is true.
But as long dbus-deamon is not optimized as much as possible there is no reason to force push kdbus.
As soon dbus-deamon exploits all kernel interfaces as much it can and it still needs work (may it performance or other stuff) we can think of new kernel features which can help dbus-deamon.I may not be well informed about kernel interfaces, but there are some use cases no dbus-daemon optimisation can make work properly because of rece-conditons introduced by the user-space based message router.
Quoted 6 lines
For example, a service can't aquire credentials of a client process that actually sent a request (it can, but it can't trust them). The service can't be protected by LSM on a bus that is driven by dbus-daemon. Yes, dbus-daemon, can check client's and srevice's labels and enforce a policy but it is going to be the daemon and not the LSM code in the kernel.
Quoted 4 lines
That's why I said we can think of new kernel features if they are needed. But they current sink or swim approach of kdbus folks is also not the solution. As I said, if dbus-daemon utilizes the kernel interface as much as possible we can think of new features.
What kernel interfaces do you suggest to use to solve the issues I mentioned in the second paragraph: race conditions, LSM support (for example)?
BTW. Does anyone know how microkernel-based OSes implement sockets?
Signature · 3 lines
Łukasz Stelmach Samsung R&D Institute Poland Samsung Electronics
Am 30.04.2015 um 14:16 schrieb Łukasz Stelmach:
Quoted 31 lines
Quoted 30 lines · when Richard Weinberger
Am 30.04.2015 um 12:19 schrieb Łukasz Stelmach:
Quoted 28 lines
Quoted 9 lines · when Richard Weinberger
Am 30.04.2015 um 11:05 schrieb Łukasz Stelmach:
Quoted 7 lines
Regardless, of initrd issues I feel there is a need of a local IPC that is more capable than UDS. Linus Torvalds is probably right that dbus-daemon is everything but effictient. I disagree, however, that it can be optimised and therefore solve *all* issues kdbus is trying to address. dbus-deamon, by design, can't some things. It can't transmitt large payloads without copying them. It can't be made race-free.
Quoted 6 lines
This is true.
But as long dbus-deamon is not optimized as much as possible there is no reason to force push kdbus.
As soon dbus-deamon exploits all kernel interfaces as much it can and it still needs work (may it performance or other stuff) we can think of new kernel features which can help dbus-deamon.I may not be well informed about kernel interfaces, but there are some use cases no dbus-daemon optimisation can make work properly because of rece-conditons introduced by the user-space based message router.
For example, a service can't aquire credentials of a client process that actually sent a request (it can, but it can't trust them). The service can't be protected by LSM on a bus that is driven by dbus-daemon. Yes, dbus-daemon, can check client's and srevice's labels and enforce a policy but it is going to be the daemon and not the LSM code in the kernel.
Quoted 8 lines
Quoted 4 lines
That's why I said we can think of new kernel features if they are needed. But they current sink or swim approach of kdbus folks is also not the solution. As I said, if dbus-daemon utilizes the kernel interface as much as possible we can think of new features.
What kernel interfaces do you suggest to use to solve the issues I mentioned in the second paragraph: race conditions, LSM support (for example)?
The question is whether it makes sense to collect this kind of meta data. I really like Andy and Alan's idea improve AF_UNIX or revive AF_BUS.
Thanks,
//richard
Quoted 7 lines · when Richard Weinberger
Am 30.04.2015 um 14:16 schrieb Łukasz Stelmach:
Quoted 6 lines
Quoted 5 lines · when Richard Weinberger
Am 30.04.2015 um 12:19 schrieb Łukasz Stelmach:
Quoted 4 lines
Quoted 3 lines · when Richard Weinberger
Am 30.04.2015 um 11:05 schrieb Łukasz Stelmach:
Quoted 2 lines
Regardless, of initrd issues I feel there is a need of a local IPC that is more capable than UDS.
[...]
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 6 lines
For example, a service can't aquire credentials of a client process that actually sent a request (it can, but it can't trust them). The service can't be protected by LSM on a bus that is driven by dbus-daemon. Yes, dbus-daemon, can check client's and srevice's labels and enforce a policy but it is going to be the daemon and not the LSM code in the kernel.
That's why I said we can think of new kernel features if they are needed. But they current sink or swim approach of kdbus folks is also not the solution. As I said, if dbus-daemon utilizes the kernel interface as much as possible we can think of new features.
Quoted 6 lines
Quoted 3 lines
What kernel interfaces do you suggest to use to solve the issues I mentioned in the second paragraph: race conditions, LSM support (for example)?
The question is whether it makes sense to collect this kind of meta data. I really like Andy and Alan's idea improve
AF_UNIXor reviveAF_BUS.
Race conditions have nothing to do with metadata. Neither has LSM support.
AF_UNIX with multicast support wouldn't be AF_UNIX anymore.
AF_BUS? I haven't followed the discussion back then. Why do you think it is better than kdbus?
Signature · 3 lines
Łukasz Stelmach Samsung R&D Institute Poland Samsung Electronics
Am 30.04.2015 um 14:40 schrieb Łukasz Stelmach:
Quoted 25 lines
Quoted 7 lines · when Richard Weinberger
Am 30.04.2015 um 14:16 schrieb Łukasz Stelmach:
Quoted 6 lines
Quoted 5 lines · when Richard Weinberger
Am 30.04.2015 um 12:19 schrieb Łukasz Stelmach:
Quoted 4 lines
Quoted 3 lines · when Richard Weinberger
Am 30.04.2015 um 11:05 schrieb Łukasz Stelmach:
Quoted 2 lines
Regardless, of initrd issues I feel there is a need of a local IPC that is more capable than UDS.
[...]
Quoted 16 lines
Quoted 15 lines
Quoted 11 lines
Quoted 6 lines
For example, a service can't aquire credentials of a client process that actually sent a request (it can, but it can't trust them). The service can't be protected by LSM on a bus that is driven by dbus-daemon. Yes, dbus-daemon, can check client's and srevice's labels and enforce a policy but it is going to be the daemon and not the LSM code in the kernel.
That's why I said we can think of new kernel features if they are needed. But they current sink or swim approach of kdbus folks is also not the solution. As I said, if dbus-daemon utilizes the kernel interface as much as possible we can think of new features.
What kernel interfaces do you suggest to use to solve the issues I mentioned in the second paragraph: race conditions, LSM support (for example)?
Quoted 5 lines
Quoted 2 lines
The question is whether it makes sense to collect this kind of meta data. I really like Andy and Alan's idea improve
AF_UNIXor reviveAF_BUS.Race conditions have nothing to do with metadata. Neither has LSM support.
Sorry, I thought you mean the races while collecting metadata in userspace...
Quoted 4 lines
AF_UNIXwith multicast support wouldn't beAF_UNIXanymore.
AF_BUS? I haven't followed the discussion back then. Why do you think it is better than kdbus?
Please see https://lwn.net/Articles/641278/
Thanks,
//richard
Quoted 26 lines · when Richard Weinberger
Am 30.04.2015 um 14:40 schrieb Łukasz Stelmach:
Quoted 25 lines
Quoted 7 lines · when Richard Weinberger
Am 30.04.2015 um 14:16 schrieb Łukasz Stelmach:
Quoted 6 lines
Quoted 5 lines · when Richard Weinberger
Am 30.04.2015 um 12:19 schrieb Łukasz Stelmach:
Quoted 4 lines
Quoted 3 lines · when Richard Weinberger
Am 30.04.2015 um 11:05 schrieb Łukasz Stelmach:
Quoted 2 lines
Regardless, of initrd issues I feel there is a need of a local IPC that is more capable than UDS.
[...]
Quoted 16 lines
Quoted 15 lines
Quoted 11 lines
Quoted 6 lines
For example, a service can't aquire credentials of a client process that actually sent a request (it can, but it can't trust them). The service can't be protected by LSM on a bus that is driven by dbus-daemon. Yes, dbus-daemon, can check client's and srevice's labels and enforce a policy but it is going to be the daemon and not the LSM code in the kernel.
That's why I said we can think of new kernel features if they are needed. But they current sink or swim approach of kdbus folks is also not the solution. As I said, if dbus-daemon utilizes the kernel interface as much as possible we can think of new features.
What kernel interfaces do you suggest to use to solve the issues I mentioned in the second paragraph: race conditions, LSM support (for example)?
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
The question is whether it makes sense to collect this kind of meta data. I really like Andy and Alan's idea improve
AF_UNIXor reviveAF_BUS.Race conditions have nothing to do with metadata. Neither has LSM support.
Sorry, I thought you mean the races while collecting metadata in userspace...
My bad, some reace conditions *are* associated with collecting metadata but ont all. It is impossible (correct me if I am wrong) to implement reliable die-on-idle with dbus-daemon.
Quoted 6 lines
Quoted 4 lines
AF_UNIXwith multicast support wouldn't beAF_UNIXanymore.
AF_BUS? I haven't followed the discussion back then. Why do you think it is better than kdbus?Please see https://lwn.net/Articles/641278/
Thanks. If I understand correctly, the author suggests using EBPF on a receiveing socket side for receiving multicast messages. This is nice if you care about introducing (or not) (too?) much of new code. However, AFAICT it may be more computationally complex than Bloom filters because you need to run EBPF on every receiving socket instead of getting a list of a few of them to copy data to. Of course for small number of receivers the "constant" cost of running the Bloom filter may be higher.
Kind regards,
Signature · 3 lines
Łukasz Stelmach Samsung R&D Institute Poland Samsung Electronics
Am 30.04.2015 um 16:52 schrieb Łukasz Stelmach:
Quoted 5 lines
Quoted 1 line
Sorry, I thought you mean the races while collecting metadata in userspace...
My bad, some reace conditions *are* associated with collecting metadata but ont all. It is impossible (correct me if I am wrong) to implement reliable die-on-idle with dbus-daemon.
IIRC Andy gave some ideas howto deal with that.
i.e. https://lkml.org/lkml/2015/4/29/622
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
AF_UNIXwith multicast support wouldn't beAF_UNIXanymore.
AF_BUS? I haven't followed the discussion back then. Why do you think it is better than kdbus?Please see https://lwn.net/Articles/641278/
Quoted 7 lines
Thanks. If I understand correctly, the author suggests using EBPF on a receiveing socket side for receiving multicast messages. This is nice if you care about introducing (or not) (too?) much of new code. However, AFAICT it may be more computationally complex than Bloom filters because you need to run EBPF on every receiving socket instead of getting a list of a few of them to copy data to. Of course for small number of receivers the "constant" cost of running the Bloom filter may be higher.
To make the story short, the kdbus *concept* needs much more
thought. There are many ideas out there howto deal with dbus issues without introducing an ad-hoc solution. AF_BUS is just one of them.
IMHO AF_BUS would be nice but the decision is not up to me.
Thanks,
//richard
2015-04-30 22:52 GMT+08:00 Łukasz Stelmach <[email protected]>:
Quoted 46 lines
Quoted 26 lines · when Richard Weinberger
Am 30.04.2015 um 14:40 schrieb Łukasz Stelmach:
Quoted 25 lines
Quoted 7 lines · when Richard Weinberger
Am 30.04.2015 um 14:16 schrieb Łukasz Stelmach:
Quoted 6 lines
Quoted 5 lines · when Richard Weinberger
Am 30.04.2015 um 12:19 schrieb Łukasz Stelmach:
Quoted 4 lines
Quoted 3 lines · when Richard Weinberger
Am 30.04.2015 um 11:05 schrieb Łukasz Stelmach:
Quoted 2 lines
Regardless, of initrd issues I feel there is a need of a local IPC that is more capable than UDS.
[...]
Quoted 16 lines
Quoted 15 lines
Quoted 11 lines
Quoted 6 lines
For example, a service can't aquire credentials of a client process that actually sent a request (it can, but it can't trust them). The service can't be protected by LSM on a bus that is driven by dbus-daemon. Yes, dbus-daemon, can check client's and srevice's labels and enforce a policy but it is going to be the daemon and not the LSM code in the kernel.
That's why I said we can think of new kernel features if they are needed. But they current sink or swim approach of kdbus folks is also not the solution. As I said, if dbus-daemon utilizes the kernel interface as much as possible we can think of new features.
What kernel interfaces do you suggest to use to solve the issues I mentioned in the second paragraph: race conditions, LSM support (for example)?
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
The question is whether it makes sense to collect this kind of meta data. I really like Andy and Alan's idea improve
AF_UNIXor reviveAF_BUS.Race conditions have nothing to do with metadata. Neither has LSM support.
Sorry, I thought you mean the races while collecting metadata in userspace...
My bad, some reace conditions *are* associated with collecting metadata but ont all. It is impossible (correct me if I am wrong) to implement reliable die-on-idle with dbus-daemon.
Quoted 6 lines
Quoted 4 lines
AF_UNIXwith multicast support wouldn't beAF_UNIXanymore.
AF_BUS? I haven't followed the discussion back then. Why do you think it is better than kdbus?Please see https://lwn.net/Articles/641278/
Quoted 7 lines
Thanks. If I understand correctly, the author suggests using EBPF on a receiveing socket side for receiving multicast messages. This is nice if you care about introducing (or not) (too?) much of new code. However, AFAICT it may be more computationally complex than Bloom filters because you need to run EBPF on every receiving socket instead of getting a list of a few of them to copy data to. Of course for small number of receivers the "constant" cost of running the Bloom filter may be higher.
Still think about the idea of implementing KDBUS in the form of socket.
What about using __multicast group__ instead of EBPF, to send/receive multicast message?
(Which can implement the bloom filter as follows ?) E.g.
Sender:
send to multi_address
Receivers:
if ((multi_address & joined_address) == joined_address) {
/* a message for us */
}Then we can further apply EBFP to remove the "False positive" case, which will otherwise wake up user space code, and let it clear "False positive" case.
Signature · 3 lines
Regards, - cee1
Am Mittwoch, 29. April 2015, 11:03:41 schrieb Theodore Ts'o:
Quoted 11 lines · Harald Hoyer
Quoted 6 lines
Sure, I can write one binary to rule them all, pull out all the code from all tools I need, but for me an IPC mechanism sounds a lot better. And it should be _one_ common IPC mechanism and not a plethora of them. It should feel like an operating system and not like a bunch of thrown together software, which is glued together with some magic shell scripts.
And so requiring wireshark (and X?) in initramfs to debug problems once dbus is introduced is better?
Quoted 7 lines
I would think shell scripts are *easier* to debug when things go wrong, especially in a minimal environment such as an initial ram disk. Having had to debug problems in a distro initramfs when trying to help a customer bring up a FC boot disk long ago in another life, I'm certain I would rather debug problems while on site at a classified machine room[1] using shell scripts, and trying to debug dbus is something that would be infinitely worse.
Later in boot process I have seen some debugging issues with a systemd governed userspace:
Bug 1213778 - drops into emergency mode without any error message if it cannot find a filesystem in /etc/fstab
https://bugzilla.redhat.com/show_bug.cgi?id=1213778
Bug 1213781 - does not start ssh service if a filesystem in /etc/fstab cannot be mounted
https://bugzilla.redhat.com/show_bug.cgi?id=1213781
With shell scripts I had error messages for these, here I had to browse the output of journalctl -xb to find out.
And then what do I do if dbus communication is not working for some reason? Will I then be able to use journalctl at all?
Not that proper error reporting can´t be added in systemd and the dependency handling can´t be fixed towards some more sanity like for example "no /boot mount, no worries, I still start that ssh service for you", but currently for me this is a clear and heavy regression when I compare this with sysvinit boot behavior.
I do fix things when being dropped to initramfs shell, I added some script snippet for supporting BTRFS RAID 1 while it wasn´t yet supported in Debian (dunno if it is for booting in Jessie, but as my bug reports have not been closed yet, maybe bot), I still want to be able to do these kinds of things.
Ciao,
Signature · 2 lines
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7
Quoted 34 lines · Harald Hoyer
Quoted 33 lines · Richard Weinberger
Am 29.04.2015 um 16:01 schrieb Harald Hoyer:
Quoted 31 lines · Richard Weinberger
Quoted 14 lines
Am 29.04.2015 um 15:38 schrieb Harald Hoyer:
Quoted 10 lines · Richard Weinberger
Quoted 7 lines
It depends how you define "beginning". To me an initramfs is a *very* minimal tool to prepare the rootfs and nothing more (no udev, no systemd, no "mini distro").
If the initramfs fails to do its job it can print to the console like the kernel does if it fails
at a very early stage.Your solution might work for your small personal needs, but not for our customers.
Correct, I don't know your customers, all I know are my customers. :-)
Quoted 8 lines
What feature do your customers need?
I mean, I fully agree with you that an initramfs must not fail silently but how does dbus help there? If it fails to mount the rootfs there is not much it can do.Thanks,
//richardWe don't handcraft the initramfs script for every our customers, therefore we have to generically support hotplug, persistent device names, persistent interface names, network connectivity in the initramfs, user input handling for passwords, fonts, keyboard layouts, fips, fsck, repair tools for file systems, raid assembly, LVM assembly, multipath, crypto devices, live images, iSCSI, FCoE, all kinds of filesystems with their quirks, IBM z-series support, resume from hibernation, […]
Quoted 6 lines
Quoted 1 line
This is correct. But which of these
tools/featuresdepend on dbus?I would love to add dbus support to all of them and use it, so I can connect them all more easily. No need for them to invent their own version of IPC, which can only be used by their own tool set.
Resume is built into the kernel, so no need for IPC there. Keymaps, fonts, and fsck need no IPC either. FIPS related stuff should need no IPC. Anything to do with the Device Mapper and hotplug should just need uevents. While I can kind of see you wanting to have lvmetad in the initramfs for use with LVM, I've seen all kinds of reports of issues caused by that. I can also kind of understand wanting some kind of unified IPC for the netboot related stuff, DBus is still serious overkill for any of that IMHO. As things stand currently, the few things in that list that I know actually use IPC for anything get by just fine (and much faster) using just UDS.
Am 29.04.2015 um 16:46 schrieb Austin S Hemmelgarn:
Quoted 37 lines · Harald Hoyer
Quoted 36 lines · Richard Weinberger
Quoted 34 lines
Am 29.04.2015 um 16:01 schrieb Harald Hoyer:
Quoted 24 lines · Richard Weinberger
Quoted 14 lines
Am 29.04.2015 um 15:38 schrieb Harald Hoyer:
Quoted 10 lines · Richard Weinberger
Quoted 7 lines
It depends how you define "beginning". To me an initramfs is a *very* minimal tool to prepare the rootfs and nothing more (no udev, no systemd, no "mini distro").
If the initramfs fails to do its job it can print to the console like the kernel does if it fails
at a very early stage.Your solution might work for your small personal needs, but not for our customers.
Correct, I don't know your customers, all I know are my customers. :-)
Quoted 8 lines
What feature do your customers need?
I mean, I fully agree with you that an initramfs must not fail silently but how does dbus help there? If it fails to mount the rootfs there is not much it can do.Thanks,
//richardQuoted 7 lines
We don't handcraft the initramfs script for every our customers, therefore we have to generically support hotplug, persistent device names, persistent interface names, network connectivity in the initramfs, user input handling for passwords, fonts, keyboard layouts, fips, fsck, repair tools for file systems, raid assembly, LVM assembly, multipath, crypto devices, live images, iSCSI, FCoE, all kinds of filesystems with their quirks, IBM z-series support, resume from hibernation, […]
This is correct. But which of these
tools/featuresdepend on dbus?
Quoted 8 lines
Quoted 4 lines
I would love to add dbus support to all of them and use it, so I can connect them all more easily. No need for them to invent their own version of IPC, which can only be used by their own tool set.
Resume is built into the kernel, so no need for IPC there. Keymaps, fonts, and fsck need no IPC either. FIPS related stuff should need no IPC. Anything to do with the Device Mapper and hotplug should just need uevents. While I can kind of see you wanting to have lvmetad in the initramfs for use with LVM, I've seen all kinds of reports of issues caused by that. I can also kind of understand wanting some kind of unified IPC for the netboot related stuff, DBus is still serious overkill for any of that IMHO. As things stand currently, the few things in that list that I know actually use IPC for anything get by just fine (and much faster) using just UDS.
Even if dbus is really needed you can embed it into the initramfs.
What you need is a good bootstrap procedure between dbus/systemd within the initramfs and the real rootfs.
Thanks,
//richard
Quoted 44 lines · Austin S Hemmelgarn
Quoted 37 lines · Harald Hoyer
Quoted 36 lines · Richard Weinberger
Am 29.04.2015 um 16:01 schrieb Harald Hoyer:
Quoted 34 lines · Richard Weinberger
Quoted 16 lines
Am 29.04.2015 um 15:38 schrieb Harald Hoyer:
Quoted 10 lines · Richard Weinberger
Quoted 8 lines
It depends how you define "beginning". To me an initramfs is a *very* minimal
tool to prepare the rootfs and nothing more (no udev, no systemd, no "mini distro").
If the initramfs fails to do its job it can print to the console like the kernel does if it fails
at a very early stage.Quoted 2 lines
Your solution might work for your small personal needs, but not for our customers.
Correct, I don't know your customers, all I know are my customers. :-)
Quoted 8 lines
What feature do your customers need?
I mean, I fully agree with you that an initramfs must not fail silently but how does dbus help there? If it fails to mount the rootfs there is not much it can do.Thanks,
//richardWe don't handcraft the initramfs script for every our customers, therefore we have to generically support hotplug, persistent device names, persistent interface names, network connectivity in the initramfs, user input handling for
passwords, fonts, keyboard layouts, fips, fsck, repair tools for file systems, raid assembly, LVM assembly, multipath, crypto devices, live images, iSCSI, FCoE, all kinds of filesystems with their quirks, IBM z-series support, resume from hibernation, […]Quoted 6 lines
Quoted 1 line
This is correct. But which of these
tools/featuresdepend on dbus?I would love to add dbus support to all of them and use it, so I can connect them all more easily. No need for them to invent their own version of IPC, which can only be used by their own tool set.
Quoted 11 lines
Resume is built into the kernel, so no need for IPC there. Keymaps, fonts, and fsck need no IPC either. FIPS related stuff should need no IPC. Anything to do with the Device Mapper and hotplug should just need uevents. While I can kind of see you wanting to have lvmetad in the initramfs for use with LVM, I've seen all kinds of reports of issues caused by that. I can also kind of understand wanting some kind of unified IPC for the netboot related stuff, DBus is still serious overkill for any of that IMHO. As things stand currently, the few things in that list that I know actually use IPC for anything get by just fine (and much faster) using just UDS.
He asked what customers need, because he does not need udev, systemd, "mini distro".
Most of the stuff does not work without udev and something like systemd.
And all of the stuff I mentioned together forms a "mini distro" for me.
Quoted 2 lines · Harald Hoyer
Most of the stuff does not work without udev and something like systemd.
That's funny, apparently the initramfs images I've been using for multiple months now on server systems at work which don't have systemd, udev, or dbus, and do LVM/RAID assembly, network configuration, crypto devices, multipath, many different filesystems, and a number of other oddball configurations due to the insanity that is the software I have to deal with from our company, don't work. I wonder how my systems are booting successfully 100% of the time then?
Quoted 11 lines · Austin S Hemmelgarn
Quoted 2 lines · Harald Hoyer
Most of the stuff does not work without udev and something like systemd.
That's funny, apparently the initramfs images I've been using for multiple months now on server systems at work which don't have systemd, udev, or dbus, and do LVM/RAID assembly, network configuration, crypto devices, multipath, many different filesystems, and a number of other oddball configurations due to the insanity that is the software I have to deal with from our company, don't work. I wonder how my systems are booting successfully 100% of the time then?
Then you should probably open source your initramfs, so we can all benefit from it and use it for all distributions.
Quoted 13 lines · Harald Hoyer
Quoted 11 lines · Austin S Hemmelgarn
Quoted 2 lines · Harald Hoyer
Most of the stuff does not work without udev and something like systemd.
That's funny, apparently the initramfs images I've been using for multiple months now on server systems at work which don't have systemd, udev, or dbus, and do LVM/RAID assembly, network configuration, crypto devices, multipath, many different filesystems, and a number of other oddball configurations due to the insanity that is the software I have to deal with from our company, don't work. I wonder how my systems are booting successfully 100% of the time then?
Quoted 3 lines
Then you should probably open source your initramfs, so we can all benefit from it and use it for all distributions.
It's (mostly, aside from a couple of overlays to deal with the hoops I have to jump through to get some of our software working) just the standard one generated by Gentoo's 'genkernel' program (specifically the version from the genkernel-ng package), although now that I actually look at it, it might have udev in it, although I'm certain the ones that I have don't have systemd or dbus.
Am Mittwoch, 29. April 2015, 17:22:08 schrieb Harald Hoyer:
Quoted 11 lines · Austin S Hemmelgarn
Quoted 10 lines · Harald Hoyer
Quoted 2 lines
Most of the stuff does not work without udev and something like systemd.>
That's funny, apparently the initramfs images I've been using for multiple months now on server systems at work which don't have systemd, udev, or dbus, and do LVM/RAID assembly, network configuration, crypto devices, multipath, many different filesystems, and a number of other oddball configurations due to the insanity that is the software I have to deal with from our company, don't work. I wonder how my systems are booting successfully 100% of the time then?
Quoted 2 lines
Then you should probably open source your initramfs, so we can all benefit from it and use it for all distributions.
Do you really think that the tooling will make that much of a difference?
I think there will always be cases where a initramfs will not work until adapted to it. And then its nice, to be able to do things like this:
merkaba:~> cat /etc/initramfs-tools/scripts/local-top/btrfs #!/bin/sh
PREREQ="lvm"prereqs()
{
echo $PREREQ
}
case $1 in
prereqs)
prereqs
exit 0;esac
. /scripts/functions
log_begin_msg "Initializing BTRFS RAID-1."
modprobe btrfs
vgchange -ay
btrfs device scan
log_end_msg
How would I add support for some configuration that a systemd or purely dracut + udev based initramfs does not support *yet*, on my own?
Yes, one can argue, why doesn´t Debian support it already, but heck, I can do it myself and report a bug about it, without having to fire up a C compiler in order to fix things. I may be able to do this myself, but at a much higher cost in time.
Above thing works so long already that I even often forgot about it.
That said, if I still get a chance to execute a script at some time, a dracut based initramfs may just be totally fine with it, but I want this possibility and a shell to fix things up myself it they go wrong. And while I do not get the need for having systemd in the initramfs at all, I might be fine with it, if I can fix things up myself in case of problems.
Ciao,
Signature · 2 lines
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Harald" == Harald Hoyer <[email protected]> writes:
Quoted 7 lines · Richard Weinberger
Quoted 7 lines
It depends how you define "beginning". To me an initramfs is a *very* minimal tool to prepare the rootfs and nothing more (no udev, no systemd, no "mini distro").
If the initramfs fails to do its job it can print to the console like the kernel does if it fails
at a very early stage.
Harald> Your solution might work for your small personal needs, but Harald> not for our customers.
Arguing that your needs outweight mine because you have customers ain't gonna fly... I don't care about your customers and why should I? I'm not getting any money from them. Nor do I make any money from Linux kernel though as an IT person, I support Linux all day long. Do my requirements get listened to as well?
If your customers wnat this feature, you're more than welcome to fork the kernel and support it yourself. Oh wait... Redhat does that
already. So what's the problem? Just put it into RHEL (which I use
I admit, along with Debian/Mint) and be done with it.
Quoted 5 lines · John Stoffel
If your customers wnat this feature, you're more than welcome to fork the kernel and support it yourself. Oh wait... Redhat does that
already. So what's the problem? Just put it into RHEL (which I use
I admit, along with Debian/Mint) and be done with it.
Red Hat tries very hard to push things upstream. It's policy is to not keep things for themselves, but always work with the community. That way, everyone benefits. Ideally, we should come up with a solution that works for all.
-- Steve
Am Mittwoch, 29. April 2015, 13:39:42 schrieb Steven Rostedt:
Quoted 10 lines · John Stoffel
Quoted 4 lines
If your customers wnat this feature, you're more than welcome to fork the kernel and support it yourself. Oh wait... Redhat does that
already. So what's the problem? Just put it into RHEL (which I use
I admit, along with Debian/Mint) and be done with it.Red Hat tries very hard to push things upstream. It's policy is to not keep things for themselves, but always work with the community. That way, everyone benefits. Ideally, we should come up with a solution that works for all.
I think work with the community is a two-way process.
Two way as in actually really *listening* to feedback instead of trying to push things as much as possible, believing to be *right* about things. I honestly dislike the "I know it better than you, go away" kind of attitude I have seen again and again. Here, in systemd-devel (where I unsubscribed again as I saw no use in continuing the discussion there) and even in Debian mailinglists and bug reports.
Wherever I look what I call the "systemd" approach triggers intense polarity and resistance. With that I do not say there is something wrong about it, yet, I ask myself, why is that? And my best answer I came up with up to now comes back to how proponents of the new, different, not necessary better or worse, way, treat feedback.
There I found to some extent: taking the feedback into account and actually adressing it. Especially when the feedback fitted into the new way of doing things.
Yet I also found:
- "I know it better than you, go away."
- "Please only stick to pure technical reasons" as in "Whats wrong with the code?" disregarding any concerns about the *concept* and about different oppinions about whether kdbus code actually really belongs into the kernel
- Ignoring it
So I still say the issues are not purely technical. So I think a purely technical as in "what´s wrong with the code?" approach will not address the core of this discussion and the strong resistance against merging kdbus into the kernel.
It sometimes appears to me like childs arguing about whether to paint their favorite toy red or green. I think a healthy approach might be to agree to disagree and work from there. That would at least break the "I am right", "No, I am right" cycle.
Ciao,
Signature · 2 lines
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Steven" == Steven Rostedt <[email protected]> writes:
Quoted 5 lines · John Stoffel
Quoted 5 lines
If your customers wnat this feature, you're more than welcome to fork the kernel and support it yourself. Oh wait... Redhat does that
already. So what's the problem? Just put it into RHEL (which I use
I admit, along with Debian/Mint) and be done with it.
Steven> Red Hat tries very hard to push things upstream. It's policy Steven> is to not keep things for themselves, but always work with the Steven> community. That way, everyone benefits. Ideally, we should Steven> come up with a solution that works for all.
Yeah, I agree they have been good. I'm just reacting to the off the cuff comment of "my customers need it" which isn't a justification for this feature, esp when it hasn't been shown to be needed in the kernel.
We went through alot of this with tux the in-kernel httpd server, and pushing other stuff out to user-space over the years. Why this needs to come in isn't clear. Or why not just a small part needing to come in with the rest in userspace.
John
Quoted 4 lines · John Stoffel
If your customers wnat this feature, you're more than welcome to fork the kernel and support it yourself. Oh wait... Redhat does that
already. So what's the problem? Just put it into RHEL (which I use
I admit, along with Debian/Mint) and be done with it.
Harald,
If you make the RHEL initramfs harder to debug in the field, I will await the time when some Red Hat field engineers will need to do the same sort of thing I have had to do in the field, and be amused when they want to shake you very warmly by the throat. :-)
Seriously, keep things as simple as possible in the initramfs; don't use complicated bus protocols; that way lies madness. Enterprise systems aren't constantly booting (or they shouldn't be, if your kernels are sufficiently reliable :-), so trying to optimize for an extra 2 or 3 seconds worth of boot time really, REALLY isn't worth it.
- Ted
Quoted 13 lines · Theodore Ts'o
Quoted 4 lines · John Stoffel
If your customers wnat this feature, you're more than welcome to fork the kernel and support it yourself. Oh wait... Redhat does that
already. So what's the problem? Just put it into RHEL (which I use
I admit, along with Debian/Mint) and be done with it.Harald,
If you make the RHEL initramfs harder to debug in the field, I will await the time when some Red Hat field engineers will need to do the same sort of thing I have had to do in the field, and be amused when they want to shake you very warmly by the throat. :-)
Quoted 5 lines
Seriously, keep things as simple as possible in the initramfs; don't use complicated bus protocols; that way lies madness. Enterprise systems aren't constantly booting (or they shouldn't be, if your kernels are sufficiently reliable :-), so trying to optimize for an extra 2 or 3 seconds worth of boot time really, REALLY isn't worth it.
I've had Enterprise systems where I could hit power on two boxes, and finish the OS install on one before the other has even finished POST and look for the boot media. I did this 5 years ago, before the "let's speed up boot" push started.
Admittedly, this wasn't a stock distro boot/install, it was my own optimized one, but it also wasn't as optimized and automated as it could have been (several points where the installer needed to pick items from a menu and enter values)
David Lang
Quoted 29 lines · David Lang
Quoted 15 lines · Theodore Ts'o
Quoted 5 lines · John Stoffel
If your customers wnat this feature, you're more than welcome to fork the kernel and support it yourself. Oh wait... Redhat does that
already. So what's the problem? Just put it into RHEL (which I use
I admit, along with Debian/Mint) and be done with it.Harald,
If you make the RHEL initramfs harder to debug in the field, I will await the time when some Red Hat field engineers will need to do the same sort of thing I have had to do in the field, and be amused when they want to shake you very warmly by the throat. :-)
Quoted 5 lines
Seriously, keep things as simple as possible in the initramfs; don't use complicated bus protocols; that way lies madness. Enterprise systems aren't constantly booting (or they shouldn't be, if your kernels are sufficiently reliable :-), so trying to optimize for an extra 2 or 3 seconds worth of boot time really, REALLY isn't worth it.
I've had Enterprise systems where I could hit power on two boxes, and finish the OS install on one before the other has even finished POST and look for the boot media. I did this 5 years ago, before the "let's speed up boot" push started.
Quoted 5 lines
Admittedly, this wasn't a stock distro boot/install, it was my own optimized one, but it also wasn't as optimized and automated as it could have been (several points where the installer needed to pick items from a menu and enter values)
You guys might have missed this new industry trend, I think they call it virtualisation,
I hear it's going to be big, you might want to look into it.
Dave.
Quoted 36 lines · Dave Airlie
Quoted 29 lines · David Lang
Quoted 15 lines · Theodore Ts'o
Quoted 5 lines · John Stoffel
If your customers wnat this feature, you're more than welcome to fork the kernel and support it yourself. Oh wait... Redhat does that
already. So what's the problem? Just put it into RHEL (which I use
I admit, along with Debian/Mint) and be done with it.Harald,
If you make the RHEL initramfs harder to debug in the field, I will await the time when some Red Hat field engineers will need to do the same sort of thing I have had to do in the field, and be amused when they want to shake you very warmly by the throat. :-)
Quoted 5 lines
Seriously, keep things as simple as possible in the initramfs; don't use complicated bus protocols; that way lies madness. Enterprise systems aren't constantly booting (or they shouldn't be, if your kernels are sufficiently reliable :-), so trying to optimize for an extra 2 or 3 seconds worth of boot time really, REALLY isn't worth it.
I've had Enterprise systems where I could hit power on two boxes, and finish the OS install on one before the other has even finished POST and look for the boot media. I did this 5 years ago, before the "let's speed up boot" push started.
Quoted 5 lines
Admittedly, this wasn't a stock distro boot/install, it was my own optimized one, but it also wasn't as optimized and automated as it could have been (several points where the installer needed to pick items from a menu and enter values)
Quoted 4 lines
You guys might have missed this new industry trend, I think they call it virtualisation,
I hear it's going to be big, you might want to look into it.
So what do you run your virtual machines on? you still have to put an OS on the hardware to support your VMs. Virtualization doesn't eliminate servers (as much as some cloud advocates like to claim it does)
And virtualization has overhead, sometimes very significant overhead, so it's not always the right answer.
David Lang
Quoted 21 lines
Quoted 15 lines
Quoted 8 lines
I've had Enterprise systems where I could hit power on two boxes, and finish
the OS install on one before the other has even finished POST and look for
the boot media. I did this 5 years ago, before the "let's speed up boot" push started.Quoted 6 lines
Admittedly, this wasn't a stock distro boot/install, it was my own optimized
one, but it also wasn't as optimized and automated as it could have been (several points where the installer needed to pick items from a menu and enter values)Quoted 4 lines
You guys might have missed this new industry trend, I think they call it virtualisation,
I hear it's going to be big, you might want to look into it.
Quoted 7 lines
So what do you run your virtual machines on? you still have to put an OS on the hardware to support your VMs. Virtualization doesn't eliminate servers (as much as some cloud advocates like to claim it does)
And virtualization has overhead, sometimes very significant overhead, so it's not always the right answer.
Thanks for proving my point, RHEL as a distro runs in both scenarios, optimising one is important at the moment, that fact that it might speed up boot on server which take 15 mins to POST is a side effect.
For some reason people seem to think their one use case is all that matters,
Dave.
Quoted 64 lines · Harald Hoyer
Quoted 7 lines · John Stoffel
LDAP is pretty damn generic, in that you can put pretty large objects into it, and pretty large OUs, etc. So why would it be a candidate for going into the kernel? And why is kdbus so important in the kernel as well? People have talked about it needing to be there for bootup, but isn't that why we ripped out RAID detection and such from the kernel and built initramfs, so that there's LESS in the kernel, and more in an early userspace? Same idea with dbus in my opinion.
Let me elaborate on the initramfs/shutdown situation a little bit more, because I have to deal with that every day.
Because of the "let's move everything to userspace" sentiment we nowadays have the situation, that we need a lot of tools to setup the root device.
Be it LVM on IMSM or iSCSI multipath, the initramfs has to setup the network (with bridging, bonding, etc.), the iSCSI connection, assemble the raid, the LVM, open crypto devices, etc...
And if something goes wrong, you want to have a shell, see all the logs and debug things.Now over the time we moved away from simple shell scripts (without any logging) and static compiled special versions for the initramfs to a mini distribution in the initramfs, which simplifies maintenance and improves reliability.
Basically you want to use the same tools in the initramfs (and shutdown) which you already have and use in your real root, with the same configuration files and the same interfaces and the same code paths.
Therefore systemd is started in dracut created initramfs, which starts journald for logging. The same basic systemd targets exist in the initramfs as on the real root, so normally you don't have to cope with specialized versions for the initramfs.
The target here is to have the same IPC mechanism from the very beginning to the very end. No crappy fallback mechanisms in case a daemon is not running or has crashed, no creepy transition from initramfs root to real root to shutdown root.
We already have such transitions like: systemd, journald, mdmon [1], etc. systemd has to serialize itself, journald's file descriptors are transitioned over, mdmon jumps through hoops. Remember you want to get rid of open files and executables and have to reexec everything, if you transition from the initramfs root to the real root, and also from the real root to the shutdown root.
We really don't want the IPC mechanism to be in a flux state. All tools have to fallback to a non-standard mechanism in that case.
If I have to pull in a dbus daemon in the initramfs, we still have the chicken and egg problem for PID 1 talking to the logging daemon and starting dbus.
systemd cannot talk to journald via dbus unless dbus-daemon is started, dbus cannot log anything on startup, if journald is not running, etc...dbus-daemon would have to transition to the real root, and from the real root to the shutdown root, without losing state.
Of course this can all be done, but it would involve fallback mechanisms, which we want to get rid off. Hopefully, you don't suggest to merge dbus with PID 1. Also with a daemon, you will lose the points mentioned in the cover mail :
Quoted 10 lines
* Security: The peers which communicate do not have to trust each other, as the only trustworthy component in the game is the kernel which adds metadata and ensures that all data passed as payload is either copied or sealed, so that the receiver can parse the data without having to protect against changing memory while parsing buffers. Also, all the data transfer is controlled by the kernel, so that LSMs can track and control what is going on, without involving userspace. Because of the LSM issue, security people are much happier with this model than the current scheme of having to hook into dbus to mediate things.
I just want to caution that this justification for kdbus is not fully realized in the current implementation. As it currently stands, there are no LSM hook calls in the kdbus tree beyond metadata collection of security labels. I know there have been experimental proof-of-concept patches floating around for LSM hooks for kdbus but they aren't merged as of yet, nor are there any real implementations of the hooks for the security modules. If kdbus is merged, we need to make sure that the LSM support is integrated before it gets enabled in any distros or we'll have a completely unmediated IPC channel, i.e. a bypass of any security policy restrictions on IPC.
It is also interesting that kdbus allows impersonation of any credential, including security label, by "privileged" clients, where privileged simply means it either has CAP_IPC_OWNER or owns (euid matches uid) the bus. That seems wrong at least for security labels; either we should not support impersonation of security labels at all or at least it should be controlled by the security module based on its own logic and attributes, not based on CAP_IPC_OWNER or a uid match.
Has anyone even reviewed the privilege and UID-based model of kdbus for its implications with respect to discretionary access control?
Quoted 9 lines
* Being in the kernel closes a lot of races which can't be fixed with the current userspace solutions. For example, with kdbus, there is a way a client can disconnect from a bus, but do so only if no further messages present in its queue, which is crucial for implementing race-free "exit-on-idle" services
* Eavesdropping on the kernel level, so privileged users can hook into the message stream without hacking support for that into their userspace processes
This one worried me a bit, particularly the statement that such eavesdropping is unobservable by any other participant on the bus. Seems a bit prone to abuse, particularly since it can be done by any privileged client, not merely the process that originally created the bus?
Quoted 3 lines · Stephen Smalley
As it currently stands, there
are no LSM hook calls in the kdbus tree beyond metadata collection of security labels.
SELinux and AppArmor are the two particularly interesting LSMs here:
those are the ones that have support for user-space mediation in dbus-daemon, and hence the ones for which replacing dbus-daemon with kdbus, without LSM hooks, would be a regression.
Quoted 4 lines
It is also interesting that kdbus allows impersonation of any credential, including security label, by "privileged" clients, where privileged simply means it either has
CAP_IPC_OWNERor owns (euid matches uid) the bus.
FWIW, this particular feature is *not* one of those that are necessary for feature parity with dbus-daemon. There's no API for making dbus-daemon fake its clients' credentials; if you can ptrace it, then you can of course subvert it arbitrarily, but nothing less hackish than that is currently offered.
Quoted 9 lines · Harald Hoyer
Quoted 3 lines
* Eavesdropping on the kernel level, so privileged users can hook into the message stream without hacking support for that into their userspace processes
This one worried me a bit, particularly the statement that such eavesdropping is unobservable by any other participant on the bus. Seems a bit prone to abuse, particularly since it can be done by any privileged client, not merely the process that originally created the bus?
For feature parity with dbus-daemon, the fact that eavesdropping/monitoring *exists* is necessary (it's a widely used developer/sysadmin feature) but the precise mechanics of how you get it are not necessarily set in stone. In particular, if you think kdbus' definition of "are you privileged?" may be too broad, that seems a valid question to be asking.
In traditional D-Bus, individual users can normally eavesdrop/monitor on their own session buses (which are not a security boundary, unless specially reconfigured), and this is a useful property; on non-LSM systems without special configuration, each user should ideally be able to monitor their own kdbus user bus, too.
The system bus *is* a security boundary, and administrative privileges should be required to eavesdrop on it. At a high level, someone with "full root privileges" should be able to eavesdrop, and ordinary users should not; there are various possible criteria for distinguishing between those two extremes, and I have no opinion on whether CAP_IPC_OWNER is the most appropriate cutoff point.
In dbus-daemon, LSMs with integration code in dbus-daemon have the opportunity to mediate eavesdropping specially. SELinux does not currently do this (as far as I can see), but AppArmor does, so AppArmor-confined processes are not normally allowed to eavesdrop on the session bus (even though the same user's unconfined processes may). That seems like one of the obvious places for an LSM hook in kdbus.
Having eavesdropping be unobservable means that applications cannot change their behaviour while they are being watched, either maliciously (to hide from investigation) or accidentally (bugs that only happen when not being debugged are the hardest to fix). dbus-daemon's traditional implementation of eavesdropping has had side-effects in the past, which is undesirable, and is addressed by the new monitoring interface in version 1.9. kdbus' version of eavesdropping is quite similar to the new monitoring interface.
Signature · 4 lines
Simon McVittie Collabora Ltd. <http://www.collabora.com/> For context, I am a D-Bus maintainer, but neither the original designer of D-Bus nor a kdbus developer.
Quoted 11 lines · Simon McVittie
Quoted 4 lines · Stephen Smalley
It is also interesting that kdbus allows impersonation of any credential, including security label, by "privileged" clients, where privileged simply means it either has
CAP_IPC_OWNERor owns (euid matches uid) the bus.FWIW, this particular feature is *not* one of those that are necessary for feature parity with dbus-daemon. There's no API for making dbus-daemon fake its clients' credentials; if you can ptrace it, then you can of course subvert it arbitrarily, but nothing less hackish than that is currently offered.
Then I'd be inclined to drop it from kdbus unless some compelling use case exists, and even then, I don't believe that CAP_IPC_OWNER or bus-owner uid match is sufficient even for forging credentials other than the security label. For socket credentials passing, for example, the kernel checks CAP_SYS_ADMIN for pid forging, CAP_SETUID for uid forging, and CAP_SETGID for gid forging. And I don't believe we support any form of forging of the security label on socket credentials.
Quoted 20 lines
For feature parity with dbus-daemon, the fact that eavesdropping/monitoring *exists* is necessary (it's a widely used developer/sysadmin feature) but the precise mechanics of how you get it are not necessarily set in stone. In particular, if you think kdbus' definition of "are you privileged?" may be too broad, that seems a valid question to be asking.
In traditional D-Bus, individual users can normally eavesdrop/monitor on their own session buses (which are not a security boundary, unless specially reconfigured), and this is a useful property; on non-LSM systems without special configuration, each user should ideally be able to monitor their own kdbus user bus, too.
The system bus *is* a security boundary, and administrative privileges should be required to eavesdrop on it. At a high level, someone with "full root privileges" should be able to eavesdrop, and ordinary users should not; there are various possible criteria for distinguishing between those two extremes, and I have no opinion on whether
CAP_IPC_OWNERis the most appropriate cutoff point.
Quoted 6 lines
In dbus-daemon, LSMs with integration code in dbus-daemon have the opportunity to mediate eavesdropping specially. SELinux does not currently do this (as far as I can see), but AppArmor does, so AppArmor-confined processes are not normally allowed to eavesdrop on the session bus (even though the same user's unconfined processes may). That seems like one of the obvious places for an LSM hook in kdbus.
Yes, we would want to control this in SELinux; I suspect that either the eavesdropping functionality did not exist in dbus-daemon at the time of the original dbus-daemon SELinux integration or it was an oversight.
Quoted 8 lines
Having eavesdropping be unobservable means that applications cannot change their behaviour while they are being watched, either maliciously (to hide from investigation) or accidentally (bugs that only happen when not being debugged are the hardest to fix). dbus-daemon's traditional implementation of eavesdropping has had side-effects in the past, which is undesirable, and is addressed by the new monitoring interface in version 1.9. kdbus' version of eavesdropping is quite similar to the new monitoring interface.
Am Mittwoch, 29. April 2015, 14:47:53 schrieb Harald Hoyer:
Quoted 8 lines
We really don't want the IPC mechanism to be in a flux state. All tools have to fallback to a non-standard mechanism in that case.
If I have to pull in a dbus daemon in the initramfs, we still have the chicken and egg problem for PID 1 talking to the logging daemon and starting dbus.
systemd cannot talk to journald via dbus unless dbus-daemon is started, dbus cannot log anything on startup, if journald is not running, etc...
Do I get this right that it is basically a userspace *design* decision that you use as a reason to have kdbus inside the kernel?
Is it really necessary to use DBUS for talking to journald? And does it really matter that much if any message before starting up dbus do not appear in the log? /proc/kmsg is a ring buffer, it can still be copied over later.
I remember this kind of reason not not having cgroup management in a separate process, but these are both in userspace.
"We have done it this way in userspace, thus this needs to be in kernel" doesn´t sound quite convincing to me as an argument for having dbus inside the kernel. Userspace uses the API the kernel and glibc provide, yes, it makes sense to look at what userspace needs, but designing some things in userspace and then requiring support for these design decisions in the kernel just doesn´t sound quite right to me.
Signature · 2 lines
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7
Quoted 10 lines · Martin Steigerwald
Am Mittwoch, 29. April 2015, 14:47:53 schrieb Harald Hoyer:
Quoted 8 lines
We really don't want the IPC mechanism to be in a flux state. All tools have to fallback to a non-standard mechanism in that case.
If I have to pull in a dbus daemon in the initramfs, we still have the chicken and egg problem for PID 1 talking to the logging daemon and starting dbus.
systemd cannot talk to journald via dbus unless dbus-daemon is started, dbus cannot log anything on startup, if journald is not running, etc...
Quoted 7 lines
Do I get this right that it is basically a userspace *design* decision that you use as a reason to have kdbus inside the kernel?
Is it really necessary to use DBUS for talking to journald? And does it really matter that much if any message before starting up dbus do not appear in the log?
/proc/kmsgis a ring buffer, it can still be copied over later.
I've been getting the early boot messages in my logs for decades (assuming the system doesn't fail before the syslog daemon is started). It sometimes has required setting a larger than default ringbuffer in the kernel, but that's easy enough to do.
David Lang
Quoted 5 lines · Harald Hoyer
Of course this can all be done, but it would involve fallback mechanisms, which we want to get rid off. Hopefully, you don't suggest to merge dbus with PID 1. Also with a daemon, you will lose the points mentioned in the cover mail :
Quoted 10 lines
* Security: The peers which communicate do not have to trust each other, as the only trustworthy component in the game is the kernel which adds metadata and ensures that all data passed as payload is either copied or sealed, so that the receiver can parse the data without having to protect against changing memory while parsing buffers. Also, all the data transfer is controlled by the kernel, so that LSMs can track and control what is going on, without involving userspace. Because of the LSM issue, security people are much happier with this model than the current scheme of having to hook into dbus to mediate things.
Other security people prefer code to stay out of the kernel when possible.
Also, this metadata argument is still invalid. Sockets can be improved for this purpose.
Quoted 6 lines
* Being in the kernel closes a lot of races which can't be fixed with the current userspace solutions. For example, with kdbus, there is a way a client can disconnect from a bus, but do so only if no further messages present in its queue, which is crucial for implementing race-free "exit-on-idle" services
This can be implemented in userspace.
Client to dbus daemon: may I exit now?
Dbus daemon to client: yes (and no more messages) or no
Quoted 5 lines
* Eavesdropping on the kernel level, so privileged users can hook into the message stream without hacking support for that into their userspace processes
Why would it be a hack in userspace but not a hack in the kernel?
Quoted 3 lines
* A number of smaller benefits: for example kdbus learned a way to peek full messages without dequeing them, which is really useful for logging metadata when handling bus-activation requests.
MSG_PEEK?
Quoted 9 lines
I don't care, if the kdbus speedup is only marginal.
In my ideal world, there is a standard IPC mechanism from the beginning to the end, which does not rely on any process running (except the kernel) and which is used by _all_ tools, be it a system daemon providing information and interfaces about device assembly or network setup tools or end user desktop processes.
Quoted 5 lines
dbus _is_ such an easy, flexible standard IPC mechanism. Of course, you can invent the wheel again (NIH, "we know better") and wait and see, if that works out. Until then the whole common IPC problem is unresolved and Linux distributions are just a collection of random software with no common interoperability and home grown interfaces.
I don't think anyone is suggesting throwing out dbus.
Quoted 5 lines
[1] transitioning mdmon is one of the critical parts for an IMSM raid array. Also running an executable from a disk, which the executable is monitoring, and which stops functioning, if the executable is not responding is insane.
That sounds like an excellent reason to just keep running the initramfs copy, and I don't see why IPC has anything to do with where the executable comes from.
--Andy
Quoted 8 lines · Andy Lutomirski
Quoted 6 lines · Harald Hoyer
* Being in the kernel closes a lot of races which can't be fixed with the current userspace solutions. For example, with kdbus, there is a way a client can disconnect from a bus, but do so only if no further messages present in its queue, which is crucial for implementing race-free "exit-on-idle" services
Quoted 5 lines
This can be implemented in userspace.
Client to dbus daemon: may I exit now?
Dbus daemon to client: yes (and no more messages) or no
Depending on how this is implemented, there would be a potential issue if a message arrived for the client after the daemon told it it could exit, but before it finished shutdown, in which case the message might get lost.
Quoted 12 lines · Austin S Hemmelgarn
Quoted 11 lines · Andy Lutomirski
Quoted 7 lines · Harald Hoyer
* Being in the kernel closes a lot of races which can't be fixed with the current userspace solutions. For example, with kdbus, there is a way a client can disconnect from a bus, but do so only if no further messages present in its queue, which is crucial for implementing race-free "exit-on-idle" services
Quoted 9 lines
Quoted 5 lines
This can be implemented in userspace.
Client to dbus daemon: may I exit now?
Dbus daemon to client: yes (and no more messages) or noDepending on how this is implemented, there would be a potential issue if a message arrived for the client after the daemon told it it could exit, but before it finished shutdown, in which case the message might get lost.
Then implement it the right way? The client sends some kind of sequence number with its request.
--Andy
Quoted 19 lines · Andy Lutomirski
Quoted 17 lines · Austin S Hemmelgarn
Quoted 16 lines · Andy Lutomirski
Quoted 7 lines · Harald Hoyer
* Being in the kernel closes a lot of races which can't be fixed with the current userspace solutions. For example, with kdbus, there is a way a client can disconnect from a bus, but do so only if no further messages present in its queue, which is crucial for implementing race-free "exit-on-idle" services
This can be implemented in userspace.
Client to dbus daemon: may I exit now?
Dbus daemon to client: yes (and no more messages) or no
Quoted 7 lines
Quoted 4 lines
Depending on how this is implemented, there would be a potential issue if a message arrived for the client after the daemon told it it could exit, but before it finished shutdown, in which case the message might get lost.
Then implement it the right way? The client sends some kind of sequence number with its request.
so any app in the system can prevent any other app from exiting/restarting by just sending it the equivalent of a ping over dbus?
preventing an app from exiting because there are unhandled messages doesn't mean that those messages are going to be handled, just that they will get read and dropped on the floor by an app trying to exit. Sometimes you will just end up with a hung app that can't process messages and needs to be restarted, but can't be restarted because there are pending messages.
The problem with "guaranteed delivery" messages is that things _will_ go wrong that will cause the messages to not be received and processed. At that point you have the choice of loosing some messages or freezing your entire system (you can buffer them for some time, but eventually you will run out of buffer space)
We see this all the time in the logging world, people configure their systems for reliable delivery of log messages to a remote machine, then when that remote machine goes down and can't receive messages (or a network issue blocks the traffic), the sending machine blocks and causes an outage.
Being too strict about guaranteeing delivery just doesn't work. You must have a mechanism to abort and throw away unprocessed messages. If this means disconnecting the receiver so that there are no missing messages to the receiver, that's a valid choice. But preventing a receiver from exiting because it hasn't processed a message is not a valid choice.
David Lang
Quoted 38 lines · David Lang
Quoted 32 lines · Andy Lutomirski
Quoted 20 lines · Austin S Hemmelgarn
Quoted 18 lines · Andy Lutomirski
Quoted 10 lines · Harald Hoyer
* Being in the kernel closes a lot of races which can't be fixed with the current userspace solutions. For example, with kdbus, there is
a
way a client can disconnect from a bus, but do so only if no
further
messages present in its queue, which is crucial for implementing race-free "exit-on-idle" servicesThis can be implemented in userspace.
Quoted 9 lines
Quoted 3 lines
Client to dbus daemon: may I exit now?
Dbus daemon to client: yes (and no more messages) or noDepending on how this is implemented, there would be a potential issue if a
message arrived for the client after the daemon told it it could exit, but
before it finished shutdown, in which case the message might get lost.Quoted 2 lines
Then implement it the right way? The client sends some kind of sequence number with its request.
Quoted 8 lines
so any app in the system can prevent any other app from exiting/restarting by just sending it the equivalent of a ping over dbus?
preventing an app from exiting because there are unhandled messages doesn't mean that those messages are going to be handled, just that they will get read and dropped on the floor by an app trying to exit. Sometimes you will just end up with a hung app that can't process messages and needs to be restarted, but can't be restarted because there are pending messages.
I think this consideration is more or less the same whether it's handled in the kernel or in userspace, though.
--Andy
Quoted 42 lines · Andy Lutomirski
Quoted 41 lines · David Lang
Quoted 32 lines · Andy Lutomirski
Quoted 20 lines · Austin S Hemmelgarn
Quoted 18 lines · Andy Lutomirski
Quoted 10 lines · Harald Hoyer
* Being in the kernel closes a lot of races which can't be fixed with the current userspace solutions. For example, with kdbus, there is
a
way a client can disconnect from a bus, but do so only if no
further
messages present in its queue, which is crucial for implementing race-free "exit-on-idle" servicesThis can be implemented in userspace.
Quoted 9 lines
Quoted 3 lines
Client to dbus daemon: may I exit now?
Dbus daemon to client: yes (and no more messages) or noDepending on how this is implemented, there would be a potential issue if a
message arrived for the client after the daemon told it it could exit, but
before it finished shutdown, in which case the message might get lost.Quoted 2 lines
Then implement it the right way? The client sends some kind of sequence number with its request.
so any app in the system can prevent any other app from exiting/restarting by just sending it the equivalent of a ping over dbus?
Quoted 8 lines
Quoted 5 lines
preventing an app from exiting because there are unhandled messages doesn't mean that those messages are going to be handled, just that they will get read and dropped on the floor by an app trying to exit. Sometimes you will just end up with a hung app that can't process messages and needs to be restarted, but can't be restarted because there are pending messages.
I think this consideration is more or less the same whether it's handled in the kernel or in userspace, though.
If the justification for why this needs to be in the kernel is that you can't reliably prevent apps from exiting if there are pending messages, then the answer of "preventing apps from exiting if there are pending messages isn't a sane thing to try and do" is a direct counter to that justification for including it in the kernel.
David Lang
Hi
Quoted 2 lines · David Lang
If the justification for why this needs to be in the kernel is that you can't reliably prevent apps from exiting if there are pending messages, [...]
It's not.
Quoted 3 lines
the answer of "preventing apps from exiting if there are pending messages isn't a sane thing to try and do" is a direct counter to that justification for including it in the kernel.
It's optionally used for reliable exit-on-idle.
Thanks
David
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"David" == David Herrmann <[email protected]> writes:
David> Hi
Quoted 2 lines · David Lang
Quoted 2 lines
If the justification for why this needs to be in the kernel is that you can't reliably prevent apps from exiting if there are pending messages, [...]
David> It's not.
Quoted 3 lines
Quoted 3 lines
the answer of "preventing apps from exiting if there are pending messages isn't a sane thing to try and do" is a direct counter to that justification for including it in the kernel.
David> It's optionally used for reliable exit-on-idle.
Then why is there a critical race that must be solved in the kernel if it's optional? And can you please describe in more detail what this 'exit-on-idle' thing is and how it works and why you would use it?
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Austin" == Austin S Hemmelgarn <[email protected]> writes:
Quoted 8 lines · Andy Lutomirski
Quoted 8 lines · Harald Hoyer
Quoted 6 lines
* Being in the kernel closes a lot of races which can't be fixed with the current userspace solutions. For example, with kdbus, there is a way a client can disconnect from a bus, but do so only if no further messages present in its queue, which is crucial for implementing race-free "exit-on-idle" services
Quoted 5 lines
Quoted 5 lines
This can be implemented in userspace.
Client to dbus daemon: may I exit now?
Dbus daemon to client: yes (and no more messages) or no
Austin> Depending on how this is implemented, there would be a Austin> potential issue if a message arrived for the client after the Austin> daemon told it it could exit, but before it finished shutdown, Austin> in which case the message might get lost.
What makes anyone think they can guarrantee that a message is even received? I could see the daemon sending the message and the client getting a segfault and dumping core. What then? How would kdbus solve this type of "race" anyway?
Can anyone give a concrete example of one of the races that are closed here? That's been one of the missing examples. And remember, there's no perfection. Even in the kernel we just had a discussion about missed/missing IPIs and lost processor interrupts, etc. Expecting perfection is just asking for trouble.
That's why there are timeouts, retries and just giving up and throwing an exception.
John
Quoted 85 lines · Harald Hoyer
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256Quoted 1 line · John Stoffel
LDAP is pretty damn generic, in that you can put pretty large objects
into
Quoted 1 line
it, and pretty large OUs, etc. So why would it be a candidate for
going
Quoted 1 line
into the kernel? And why is kdbus so important in the kernel as
well?
Quoted 1 line
People have talked about it needing to be there for bootup, but isn't
that
Quoted 3 lines
why we ripped out RAID detection and such from the kernel and built initramfs, so that there's LESS in the kernel, and more in an early userspace? Same idea with dbus in my opinion.
Let me elaborate on the initramfs/shutdown situation a little bit more, because I have to deal with that every day.
Because of the "let's move everything to userspace" sentiment we nowadays
have the situation, that we need a lot of tools to setup the root device.Be it LVM on IMSM or iSCSI multipath, the initramfs has to setup the network
(with bridging, bonding, etc.), the iSCSI connection, assemble the raid, the
LVM, open crypto devices, etc...
And if something goes wrong, you want to have a shell, see all the logs and
debug things.Now over the time we moved away from simple shell scripts (without any logging) and static compiled special versions for the initramfs to a mini
distribution in the initramfs, which simplifies maintenance and improves
reliability.Basically you want to use the same tools in the initramfs (and shutdown)
which you already have and use in your real root, with the same configuration
files and the same interfaces and the same code paths.Therefore systemd is started in dracut created initramfs, which starts journald for logging. The same basic systemd targets exist in the initramfs
as on the real root, so normally you don't have to cope with specialized
versions for the initramfs.The target here is to have the same IPC mechanism from the very beginning to
the very end. No crappy fallback mechanisms in case a daemon is not running
or has crashed, no creepy transition from initramfs root to real root to
shutdown root.We already have such transitions like: systemd, journald, mdmon [1], etc.
systemd has to serialize itself, journald's file descriptors are transitioned
over, mdmon jumps through hoops. Remember you want to get rid of open files
and executables and have to reexec everything, if you transition from the
initramfs root to the real root, and also from the real root to the shutdown
root.We really don't want the IPC mechanism to be in a flux state. All tools have
to fallback to a non-standard mechanism in that case.If I have to pull in a dbus daemon in the initramfs, we still have the chicken and egg problem for PID 1 talking to the logging daemon and starting
dbus.
systemd cannot talk to journald via dbus unless dbus-daemon is started, dbus
cannot log anything on startup, if journald is not running, etc...
Quoted 3 lines
dbus-daemon would have to transition to the real root, and from the real root
to the shutdown root, without losing state.
Which does not sound fundamentally hard.
Unify the roots, and make /run or wherever the dbus socket lives always available. As long as your initramfs has the latest versions of software there is no need for any tricky transitions except to upgrade software on a running system.
Quoted 3 lines
Of course this can all be done, but it would involve fallback mechanisms,
which we want to get rid off.
Only if you design things poorly.
Quoted 4 lines
Hopefully, you don't suggest to merge
dbus with
PID 1. Also with a daemon, you will lose the points mentioned in the cover mail
I don't see how something that is inappropriate to be in PID 1 is better in PID 0.
Quoted 11 lines
I don't care, if the kdbus speedup is only marginal.
In my ideal world, there is a standard IPC mechanism from the beginning to
the end, which does not rely on any process running (except the kernel) and
which is used by _all_ tools, be it a system daemon providing information and
interfaces about device assembly or network setup tools or end user desktop
processes.
And that is a beautiful dream and an absolutely rubbish way to get there. If the performance is not top notch everything can not use your beautiful IPC mechanism. Which means your dream fails.
Good performance is a hard requirement to get where you want to be.
Quoted 8 lines
dbus _is_ such an easy, flexible standard IPC mechanism. Of course, you can
invent the wheel again (NIH, "we know better") and wait and see, if that
works out. Until then the whole common IPC problem is unresolved and Linux
distributions are just a collection of random software with no common interoperability and home grown interfaces.
kdbus seems to be the NIH "we know better better" approach. Many of it's design decisions we have chosen to differently elsewhere in the kernel because the have caused problems. When these issues have been pointed out in review people have blown off leading to the current mess.
Furthermore I don't know that I have seen people arguing for transporting something other than dbus messages but rather I have seen people pointing out there have been many excellent IPC mechanisms that are simpler and faster for the same kind of task and suggesting mapping dbus to better kernel primitives might be productive.
But seriously if you want to have one IPC mechanism to rule them all you won't succeed in convincing everyone with the currently sloppily designed kdbus code.
Performance matters, simplicity matters, being able to explain design decisions matter.
Eric
Quoted 3 lines · Harald Hoyer
Until then the whole common IPC problem is unresolved and Linux distributions are just a collection of random software with no common interoperability and home grown interfaces.
I don't know how I managed to not notice this comment before, but I find it particularly hilarious.
The part about 'no common interoperability' is just plain BS with the exception of some of the insanity being touted by systemd advocates and the insanity that is accessibility software on linux, you can easily string together pretty much arbitrary strings of commands using fifo's to achieve almost anything; the actual interoperability issues (WRT to the command line at least, which is where all the stuff you are complaining about works) come up only with stuff (like journald for example) that just refuses to use text interfaces on the command-line. Also, the 'home grown interfaces' you are complaining about are used on every operating system (not just Linux or other Unix progeny) every day, and no amount of better IPC is going to stop that; furthermore, almost every current 'standard' protocol or interface used on the internet started out as a 'home grown interface' (TCP/IP immediately comes to mind, followed shortly by NFS, SMTP, WebDAV, SSH, XML, JSON, and a whole slew of others).
Quoted 7 lines · Havoc Pennington
btw if I can make a suggestion, it's quite confusing to talk about "dbus" unqualified when we are talking about implementation issues, since it muddles bus daemon vs. clients, and also since there are lots of implementations of the client bindings:
Quoted 8 lines
For the bus daemon, the only two implementations I know of are the original one (which uses libdbus as its binding) and kdbus, though.
I would expect there's no question the bus daemon can be faster, maybe say 1.5x raw sockets instead of 2.5x, or whatever - something on that order. Should probably simply stipulate this for discussion purposes:
"someone could optimize the crap out of the bus daemon". The kdbus question is about whether to eliminate this daemon entirely.
As I'm seeing things, we aren't talking about 1.5x vs 2.5x, we're talking about 1000x
If the examples that are being used to show the performance advantage of kdbus vs normal dbus are doing the wrong thing, then we need to get some other examples available to people who don't live and breath dbus that 'so things right' so that the kernel developers can see what you think is the real problem and how kdbus addresses it.
So far, this 'wrong' example is the only thing that's been posted to show the performance advantage of kdbus.
David Lang
Quoted 8 lines · David Lang
If the examples that are being used to show the performance advantage of kdbus vs normal dbus are doing the wrong thing, then we need to get some other examples available to people who don't live and breath dbus that 'so things right' so that the kernel developers can see what you think is the real problem and how kdbus addresses it.
So far, this 'wrong' example is the only thing that's been posted to show the performance advantage of kdbus.
I'm hopeful someone will do that.
fwiw, I would be suspicious of a broken benchmark if it didn't show:
* the bus daemon means an extra read/parse and marshal/write per message, so 4 vs. 2
* the existence of the bus daemon therefore makes a message send/receive take roughly twice as long
https://lwn.net/Articles/580194/ has a bit more elaboration about number of copies, validations, and context switches in each case.
Quoted 1 line
From what I can tell, the core performance claim for kdbus is that for
a userspace daemon to be a routing intermediary, it has to receive and re-send messages. If the baseline performance of IPC is the cost to send once and receive once, adding the daemon means there's twice as much to do (1 more receive, 1 more send). However fast you make send/receive, the daemon always means there are twice as many send/receives as there would be with no daemon.
If that isn't what a benchmark shows, then there's a mystery to explain... (one disruption to the ratio of course could be if the clients use a much faster or slower dbus lib than the daemon)
As noted many times, of course this 2x penalty for the daemon was a conscious tradeoff - kdbus is trying to escape the tradeoff in order to extend usage of dbus to more use cases. Given the tradeoff, _existing_ uses of dbus seem to prefer the performance hit to the loss of useful semantics, but potential new users would like to or need to have both.
That LWN article lists some other non-performance rationales for kdbus too, of course.
Aside: earlier I referred to the systemd dbus client binding without a link, the link appears to be:
http://cgit.freedesktop.org/systemd/systemd/tree/src/libsystemd/sd-bus
Havoc
Quoted 22 lines · Havoc Pennington
Quoted 8 lines · David Lang
If the examples that are being used to show the performance advantage of kdbus vs normal dbus are doing the wrong thing, then we need to get some other examples available to people who don't live and breath dbus that 'so things right' so that the kernel developers can see what you think is the real problem and how kdbus addresses it.
So far, this 'wrong' example is the only thing that's been posted to show the performance advantage of kdbus.
I'm hopeful someone will do that.
fwiw, I would be suspicious of a broken benchmark if it didn't show:
* the bus daemon means an extra read/parse and marshal/write per message, so 4 vs. 2
* the existence of the bus daemon therefore makes a message send/receive take roughly twice as longhttps://lwn.net/Articles/580194/ has a bit more elaboration about number of copies, validations, and context switches in each case.
Quoted 7 lines
From what I can tell, the core performance claim for kdbus is that for a userspace daemon to be a routing intermediary, it has to receive and re-send messages. If the baseline performance of IPC is the cost to send once and receive once, adding the daemon means there's twice as much to do (1 more receive, 1 more send). However fast you make send/receive, the daemon always means there are twice as many send/receives as there would be with no daemon.
there are twice as many context switches, nobody disputes that, the question is if it matters.
It doesn't matter if the message router is in kernel space or user space, it still needs to read/parse, marshal/write the data, so you aren't saving that time due to it being in the kernel.
Quoted 10 lines
If that isn't what a benchmark shows, then there's a mystery to explain... (one disruption to the ratio of course could be if the clients use a much faster or slower dbus lib than the daemon)
As noted many times, of course this 2x penalty for the daemon was a conscious tradeoff - kdbus is trying to escape the tradeoff in order to extend usage of dbus to more use cases. Given the tradeoff, _existing_ uses of dbus seem to prefer the performance hit to the loss of useful semantics, but potential new users would like to or need to have both.
If there is a 2x performance improvement for being in the kernel, but a 100x performance improvement from fixing the userspace code, the effort should be spent on the userspace code, not on moving things to kernel space.
Remember the Tux in-kernel webserver? it showed performance improvements from putting the http daemon in the kernel, and a lot of the arguments about it sound very similar (reduced context switches, etc)
David Lang
Quoted 55 lines · David Lang
Quoted 25 lines · Havoc Pennington
Quoted 10 lines · David Lang
If the examples that are being used to show the performance advantage of kdbus vs normal dbus are doing the wrong thing, then we need to get some other examples available to people who don't live and breath dbus that 'so
things right' so that the kernel developers can see what you think is the real problem and how kdbus addresses it.So far, this 'wrong' example is the only thing that's been posted to show the performance advantage of kdbus.
I'm hopeful someone will do that.
fwiw, I would be suspicious of a broken benchmark if it didn't show:
* the bus daemon means an extra read/parse and marshal/write per message, so 4 vs. 2
* the existence of the bus daemon therefore makes a message send/receive take roughly twice as longhttps://lwn.net/Articles/580194/ has a bit more elaboration about number of copies, validations, and context switches in each case.
Quoted 7 lines
From what I can tell, the core performance claim for kdbus is that for a userspace daemon to be a routing intermediary, it has to receive and re-send messages. If the baseline performance of IPC is the cost to send once and receive once, adding the daemon means there's twice as much to do (1 more receive, 1 more send). However fast you make send/receive, the daemon always means there are twice as many send/receives as there would be with no daemon.
there are twice as many context switches, nobody disputes that, the question is if it matters.
It doesn't matter if the message router is in kernel space or user space, it still needs to read/parse, marshal/write the data, so you aren't saving that time due to it being in the kernel.
Quoted 10 lines
If that isn't what a benchmark shows, then there's a mystery to explain... (one disruption to the ratio of course could be if the clients use a much faster or slower dbus lib than the daemon)
As noted many times, of course this 2x penalty for the daemon was a conscious tradeoff - kdbus is trying to escape the tradeoff in order to extend usage of dbus to more use cases. Given the tradeoff, _existing_ uses of dbus seem to prefer the performance hit to the loss of useful semantics, but potential new users would like to or need to have both.
Quoted 3 lines
If there is a 2x performance improvement for being in the kernel, but a 100x performance improvement from fixing the userspace code, the effort should be spent on the userspace code, not on moving things to kernel space.
I would guess that, if we compared a highly optimized userspace implementation to a kernel implementation, we'd see less than 2x difference. After all, a userspace daemon doesn't really need to unmarshal and re-marshal anything except headers. For large messages, we could use splice and avoid a couple of copies, too.
If the scheduler became a bottleneck, it could be interesting to add something like a send-and-poll primitive. I suspect that some workloads currently do unnecessary context switches with only standard POSIX primitives. If A sends a message to B, then there's a brief window in which both A and B are runnable. Ideally we wouldn't context switch until A calls poll or epoll_wait, but I don't know how well that works in practice.
There's more room for generic improvements than just that. At LSF/MM we were talking about more scalable epoll variants that would allow a multithreaded daemon to be woken up on the core that received incoming data. That would allow an efficient multi-queue dbus with fewer migrations and IPIs.
At some point, I'd like to implement PCID on x86 (if no one beats me to it, and this is a low priority for me), which will allow us to skip expensive TLB flushes while context switching. I have no idea whether ARM can do something similar.
--Andy
Quoted 8 lines · Havoc Pennington
From what I can tell, the core performance claim for kdbus is that for a userspace daemon to be a routing intermediary, it has to receive and re-send messages. If the baseline performance of IPC is the cost to send once and receive once, adding the daemon means there's twice as much to do (1 more receive, 1 more send). However fast you make send/receive, the daemon always means there are twice as many send/receives as there would be with no daemon.
HOWEVER.
That's only a good optimization strategy if the code is optimized to begin with.
If the code spends 10x as much time in user space in "overhead" as it actually spends in the kernel, the proper place to optimize is to get rid of the 10x. That will make things much faster.
Once user space is lean and mean, at that point do I believe that "ok, let's add kernel code for the last bit of performance". But as it is right now, anybody who works on kdbus and claims that _performance_ is the reason for their work is just looking at teh wrong piece of the puzzle.
Now, there may be *other* reasons why kdbus is a good idea. But quite frankly, every time somebody asks "why", performance seems to be one of the main answers.
And quite frankly, that *stinks*.
Do proper optimizations of the actual real costs before starting to work on kernel stuff. It's *stupid* to add a kernel driver to get 2x improvement, when there's a 10x bloat in user space.
Is that really so hard to see? I don't think it is at *all* appropriate to say "we're a f*cking bloated pig, but we're too lazy to fix the bloat and the primary performance problems, so we'll add a kernel interface to partially hide the issue".
That is particularly true because if you fix the user-level performance problems, you may notice that there was something stupid in the interfaces, and some of the kernel interface design was wrong.
Linus
Quoted 18 lines · Linus Torvalds
Quoted 4 lines · Linus Torvalds
IOW, all the people who say that it's about avoiding context switches are probably just full of shit. It's not about context switches, it's about bad user-level code.
Just to make sure, I did a system-wide profile (so that you can actually see the overhead of context switching better), and that didn't change the picture.
The scheduler overhead *might* be 1% or so.
So really. The people who talk about how kdbus improves performance are just full of sh*t. Yes, it improves things, but the improvement seems to be 100% "incidental", in that it avoids a few trips down the user-space problems.
Quoted 4 lines
The real problems seem to be in dbus memory management (suggestion: keep a small per-thread cache of those message allocations) and to a smaller degree in the crazy utf8 validation (why the f*ck does it do that anyway?), with some locking problems thrown in for good measure.
In case someone actually still reads this, I guess the global rw_lock in gobject/gtype.c is one of the culprits. Every GType instance allocation/ deallocation is serialized using this lock, which pretty much disqualifies GObject from being used for anything scalable to multiple threads.
GStreamer used to have serious performance issues due to that, which AFAIK have been solved by removing GType from GStreamer core in the 1.0 release.
Regards,
Signature · 5 lines
Jindrich Makovicka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in Please read the FAQ at http://www.tux.org/lkml/
Quoted 26 lines · Jindrich Makovicka
Quoted 15 lines
Quoted 3 lines
IOW, all the people who say that it's about avoiding context switches are probably just full of shit. It's not about context switches, it's about bad user-level code.
Just to make sure, I did a system-wide profile (so that you can actually see the overhead of context switching better), and that didn't change the picture.
The scheduler overhead *might* be 1% or so.
So really. The people who talk about how kdbus improves performance are just full of sh*t. Yes, it improves things, but the improvement seems to be 100% "incidental", in that it avoids a few trips down the user-space problems.
Quoted 4 lines
The real problems seem to be in dbus memory management (suggestion: keep a small per-thread cache of those message allocations) and to a smaller degree in the crazy utf8 validation (why the f*ck does it do that anyway?), with some locking problems thrown in for good measure.
In case someone actually still reads this, I guess the global
rw_lockingobject/gtype.cis one of the culprits. Every GType instance allocation/ deallocation is serialized using this lock, which pretty much disqualifies GObject from being used for anything scalable to multiple threads.
Quoted 3 lines
GStreamer used to have serious performance issues due to that, which AFAIK have been solved by removing GType from GStreamer core in the 1.0 release.
This is interesting piece of information, but you unfortunately dropped everybody from CC, so it's very likely that it's going to be lost in the noise.
Could you please resend with CC list restored?
Thanks,
Signature · 6 lines
Jiri Kosina SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in Please read the FAQ at http://www.tux.org/lkml/
Quoted 18 lines · On Mon, 27 Apr 2015 15:14:49 -0700 Linus Torvalds…
Quoted 4 lines · Linus Torvalds
IOW, all the people who say that it's about avoiding context switches are probably just full of shit. It's not about context switches, it's about bad user-level code.
Just to make sure, I did a system-wide profile (so that you can actually see the overhead of context switching better), and that didn't change the picture.
The scheduler overhead *might* be 1% or so.
So really. The people who talk about how kdbus improves performance are just full of sh*t. Yes, it improves things, but the improvement seems to be 100% "incidental", in that it avoids a few trips down the user-space problems.
Quoted 4 lines
The real problems seem to be in dbus memory management (suggestion:
keep a small per-thread cache of those message allocations) and to a smaller degree in the crazy utf8 validation (why the f*ck does it do that anyway?), with some locking problems thrown in for good measure.
In case someone actually still reads this, I guess the global rw_lock in gobject/gtype.c, used by the GDbus binding, is one of the culprits. Every GType instance allocation/ deallocation is serialized using this lock, which pretty much disqualifies GObject from being used for anything scalable to multiple threads.
GStreamer used to have serious performance issues due to that, which AFAIK have been solved by removing GType from GStreamer core in the 1.0 release.
Regards,
Signature · 4 lines
Jindrich Makovicka -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in Please read the FAQ at http://www.tux.org/lkml/
Quoted 11 lines · Linus Torvalds
Quoted 4 lines · Linus Torvalds
IOW, all the people who say that it's about avoiding context switches are probably just full of shit. It's not about context switches, it's about bad user-level code.
Just to make sure, I did a system-wide profile (so that you can actually see the overhead of context switching better), and that didn't change the picture.
Quoted 6 lines
The scheduler overhead *might* be 1% or so.
So really. The people who talk about how kdbus improves performance are just full of sh*t. Yes, it improves things, but the improvement seems to be 100% "incidental", in that it avoids a few trips down the user-space problems.
I was interested how plain UDS performs compared to the dbus-client/dbus-server benchmark when doing a similar transaction (RPC call from client1 to client2 via a server, i.e 4 send() and 4 recv() syscalls per RPC msg). Since I had worked on socket code for some project anyway, I decided to write a stupid little benchmark.
On my machine, dbus-client/dbus-server needs ~200us per call (1024 byte msg), UDS "dbus call" needs ~23us. Of course, someone who cares about performance wouldn't use sync RPC via a message broker, so I added single-client and async mode to the benchmark for comparison. Async mode not only decreases scheduling overhead, it also
can use two CPU cores, so it's more than twice as fast.
./server dbus
(you need to run two clients, the timing loop starts
when the second client connects)
./client sync 4096 1000000
22.757250 s, 43942 msg/s, 22.8 us/msg, 171.638 MB/s
./client async 4096 1000000
8.197482 s, 121989 msg/s, 8.2 us/msg, 476.488 MB/s
./server single
(only a single client talks to the server)
./client sync 4096 1000000
10.980143 s, 91073 msg/s, 11.0 us/msg, 355.733 MB/s
./client async 4096 1000000
3.041953 s, 328736 msg/s, 3.0 us/msg, 1284.044 MB/sIn all cases 1 msg means "send request + receive response".
Johannes
/* UDS server */
#include <errno.h>
#include <poll.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/socket.h>
#include <sys/timerfd.h>
#include <sys/un.h>
// use abstract address
#define SOCKET "\0udsbench.socket"
static int die(const char *msg)
{
if (errno)
fprintf(stderr, "%s: error %d %m\n", msg, errno);
else
fprintf(stderr, "%s\n", msg);
if (errno != ECONNRESET)
exit(EXIT_FAILURE);
return 0;
}int main(int argc, char *argv[])
{
struct sockaddr_un addr = {
.sun_family = AF_UNIX,
.sun_path = SOCKET,
};
int sock, client1, client2 = -1, rc, len;
struct pollfd pfd[2];
char buf[65536];
unsigned long cnt = 0;
bool single = false;
if (argc != 2)
die("usage: server {single|dbus}");
if (!strcmp(argv[1], "single"))
single = true;
printf("running in %s mode\n", single ? "single" : "dbus");
sock = socket(AF_UNIX, SOCK_SEQPACKET, 0);
if (sock < 0)
die("can't create socket");
if (bind(sock, (struct sockaddr *) &addr, sizeof(addr)) < 0)
die("can't bind address");
if (listen(sock, 5) < 0)
die("can't listen");
printf("waiting for client 1\n");
client1 = accept(sock, NULL, NULL);
if (client1 < 0)
die("accept");
if (!single) {
printf("waiting for client 2\n");
client2 = accept(sock, NULL, NULL);
if (client2 < 0)
die("accept");
write(client2, "\01", 1);
}
write(client1, "\0", 1);
printf("enter event loop\n");
pfd[0].fd = client1;
pfd[1].fd = client2;
pfd[0].events = pfd[1].events = POLLIN;
for (;;) {
rc = poll(pfd, single ? 1 : 2, -1);
if (rc < 0)
die("poll");
if (pfd[0].revents & POLLIN) {
len = read(client1, buf, sizeof(buf));
if (len < 0) {
die("read from client 1");
break;
}
if (len == 0) {
printf("client 1 EOF\n");
break;
}
rc = write(single ? client1 : client2, buf, len);
if (len != rc) {
die("write to client 2");
break;
}
cnt++;
}
if (pfd[1].revents & POLLIN) {
len = read(client2, buf, sizeof(buf));
if (len < 0) {
die("read from client 2");
break;
}
if (len == 0) {
printf("client 2 EOF\n");
break;
}
rc = write(client1, buf, len);
if (len != rc) {
die("write to client 1");
break;
}
cnt++;
}
}
printf("passed %lu messages\n", cnt);
return EXIT_SUCCESS;
}/* UDS client */
#include <alloca.h>
#include <errno.h>
#include <poll.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <unistd.h>
#include <sys/socket.h>
#include <sys/timerfd.h>
#include <sys/un.h>
// use abstract address
#define SOCKET "\0udsbench.socket"
static int die(const char *msg)
{
if (errno)
fprintf(stderr, "%s: error %d %m\n", msg, errno);
else
fprintf(stderr, "%s\n", msg);
if (errno != EPIPE)
exit(EXIT_FAILURE);
return 0;
}int main(int argc, char *argv[])
{
struct sockaddr_un addr = {
.sun_family = AF_UNIX,
.sun_path = SOCKET,
};
int sock, rc, client = 1, i;
char *buf;
struct timespec start, end;
double duration;
bool async = false;
struct pollfd pfd;
typeof(&read) f1, f2;
long msglen, loops;
if (argc != 4)
die("usage: client {sync|async} msglen loops");
if (!strcmp(argv[1], "async"))
async = true;
msglen = strtoul(argv[2], NULL, 0);
loops = strtoul(argv[3], NULL, 0);
printf("running in %s mode, msg size %lu, %lu loops\n",
async ? "async" : "sync", msglen, loops);
buf = alloca(msglen);
sock = socket(AF_UNIX, SOCK_SEQPACKET, 0);
if (sock < 0)
die("can't create socket");
if (connect(sock, (struct sockaddr *) &addr, sizeof(addr)) < 0)
die("can't connect");
printf("waiting for other client\n");
rc = read(sock, buf, 1);
if (rc != 1) {
die("read");
exit(EXIT_FAILURE);
}
if (buf[0] != '\0')
client = 2;
printf("this is client %d\n", client);
clock_gettime(CLOCK_MONOTONIC, &start);
if (client == 1)
f1 = (typeof(&read))write, f2 = read;
else
f1 = read, f2 = (typeof(&read))write;
if (async && client == 1) {
pfd.fd = sock;
pfd.events = POLLIN | POLLOUT;
for (i = 0; i < loops; ) {
rc = poll(&pfd, 1, -1);
if (rc == -1)
die("poll");
if (pfd.revents & POLLOUT) {
rc = write(sock, buf, msglen);
if (rc != msglen) {
die("write");
break;
}
}
if (pfd.revents & POLLIN) {
rc = read(sock, buf, msglen);
if (rc != msglen) {
die("read");
break;
}
i++;
}
}
}
else {
for (i = 0; i < loops; i++) {
rc = f1(sock, buf, msglen);
if (rc != msglen) {
die(f1 == read ? "read" : "write");
break;
}
rc = f2(sock, buf, msglen);
if (rc != msglen) {
die(f2 == read ? "read" : "write");
break;
}
}
}
clock_gettime(CLOCK_MONOTONIC, &end);
duration = end.tv_sec - start.tv_sec +
(end.tv_nsec - start.tv_nsec) * 1e-9;
printf("%f s, %.0f msg/s, %.1f us/msg, %.3f MB/s\n", duration,
loops / duration, duration * 1e6 / loops,
(loops * msglen >> 20) / duration);
return EXIT_SUCCESS;
}CFLAGS := -O3 -Wall
CC := gcc
all: client server
clean:
rm -f client server
Quoted 2 lines · Linus Torvalds
If somebody wants to speed up dbus, they should likely look at the user-space code, not the kernel side.
To be more precise, your profile seems to show a lot of the gdbus (glib bindings) user space code. (And the blocking version of this benchmark *is* doing something ridiculous, by blocking for every round trip, which is the one performance mistake the docs say over and over not to make.)
There are at least two other C bindings (a plain-C one in systemd and the original libdbus).
If someone wanted to get the noise out of the picture I imagine the plain-C bindings in systemd might have a lot less in the way of allocations and locks than gdbus, though I haven't looked at them. Those systemd bindings are also the ones people asking for more performance are probably using (because they are talking about early boot, system services, etc.)
Quoted 3 lines
My guess is that pretty much the entirely of the quoted kdbus "speedup" isn't because it speeds up any kernel side thing, it's because it avoids the user-space crap in the dbus server.
The dbus bus daemon doesn't link to any g_ functions, fwiw, when interpreting these profiles. Though nobody would claim the bus daemon is fast, it is on the order of a few times slower than a raw socket last I checked (which was a long time ago) ... here are some old threads:
http://lists.freedesktop.org/pipermail/dbus/2004-November/001779.html
http://lists.freedesktop.org/archives/dbus/2012-March/015024.html
In 2004, the libdbus parsing/validation/malloc/etc. overhead was 2.5x a raw socket without the bus daemon, and about twice that with the bus daemon (since the daemon adds another read and another write per message). I'm not aware of any reason this would have changed dramatically, though it doesn't mean there isn't one.
Havoc
Quoted 10 lines · Linus Torvalds
Quoted 5 lines · Lukasz Skalski
To perform tests I've created two simple apps:
- server: http://fpaste.org/215157/
- client: http://fpaste.org/215156/So since Andy reported that dbus seems to be a few orders of magnitude too slow, I tried to build these apps to see what it even does.
Quoted 7 lines
They don't buidl on F21. You seem to be using features that are too new to exist even in fairly modern distros:
server.c:47:24: error: ‘G_BUS_TYPE_USER’ undeclared
so I can't even see what dbus does *now*.
I've just explained it in my mail to Andy. As it was discussed some time ago with GLib developers, we introduced two new bus types called "user" (G_BUS_TYPE_USER) and "machine" (G_BUS_TYPE_MACHINE). At this moment, these are only available on GLib devel branch, so I should replace G_BUS_TYPE_USER with G_BUS_TYPE_SESSION before I posted my benchmark apps - sorry for that.
Quoted 7 lines
That said, either you're running your test on a potato, or dbus is seriously screwed up. No way should it take 4+ seconds to send a 1000b message to back and forth 20k times. But as mentioned, I can't even see what it's doing right now.
Linus
Signature · 4 lines
Lukasz Skalski Samsung R&D Institute Poland Samsung Electronics [email protected]
Quoted 3 lines · Greg Kroah-Hartman
Did I miss anything else here? Are there any technical reasons I'm forgetting about for why this can't be pulled in as-is for this merge window?
Maybe I get again accused of ``being a jerk'' but I still dare to ask about Boris' unanswered question:
http://marc.info/?l=linux-kernel&m=142969313220781
In fact Boris and I are currently reviewing the code but it is a slow process as we both have day jobs...
AFACT Steven is also looking at it.
Signature · 2 lines
Thanks, //richard
Quoted 9 lines · Richard Weinberger
Quoted 3 lines · Greg Kroah-Hartman
Did I miss anything else here? Are there any technical reasons I'm forgetting about for why this can't be pulled in as-is for this merge window?
Maybe I get again accused of ``being a jerk'' but I still dare to ask about Boris' unanswered question:
http://marc.info/?l=linux-kernel&m=142969313220781
No, I'm not going to say you are being a jerk for asking for a response, only when you say things like "the code is too big" :)
I go reply now, I didn't really understand it the first time around, and still don't the second time either...
Quoted 2 lines
In fact Boris and I are currently reviewing the code but it is a slow process as we both have day jobs...
That's great, thanks for doing it.
greg k-h
Kdbus needs meaningful review (was: Re: [GIT PULL] kdbus for 4.1-rc1)
In reply to Greg Kroah-HartmanQuoted 15 lines · Greg Kroah-Hartman
Quoted 6 lines · Greg Kroah-Hartman
The following changes since commit
9eccca0843205f87c00404b663188b88eb248051:Linux 4.0-rc3 (2015-03-08 16:09:09 -0700)
are available in the git repository at:
Quoted 7 lines
git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git/tags/kdbus-4.1-rc1for you to fetch changes up to
9fb9cd0f4434a23487b6ef3237e733afae90e336:kdbus: avoid the use of struct timespec (2015-04-10 14:34:53 +0200)
----------------------------------------------------------------
Quoted 8 lines
Given this has been a crazy email thread, let's try to figure out what the status is here.
Al Viro pointed out some odd locking (r/w lock only used in write mode), and asked for some more documentation / description of the object model used here. David provided that, and will send a minor fix for the rw lock, so I think that issue is now resolved. David has created a few other minor changes based on Al's review that I will forward on later.
As I recall Al could not even trace through all of the locking, and that prevented him looking at any of the bigger issues. That does not qualify as fixed with a little patch or two and everything thing is fixed.
Perhaps with the minor patch or two Al could potentially actually review the code.
Quoted 5 lines
Andy's concerns about the capability stuff has been hashed out in multiple threads here. The kernel code isn't buggy as-designed or implemented from what we can all tell, it's just that the new functionality isn't liked by everyone, which is totally fair, but not a reason to declare that the function isn't useful.
There are in fact implementation bugs and you asserion that there are none is the largest reason this code should not be merged. You are turning a blind eye to problems.
Quoted 5 lines
Alan, and others, want a tiny, generic, multi-cast IPC method that also works across networks. They feel that this is something that D-Bus might be able to use in the future in userspace to build on top of. Lots of people have said they want something like this for years, but that doesn't address the issue here with kdbus, which is a very specific
Quoted 8 lines
solution for a very common and wide-spread usage model that Linux userspace relies on today. I too would love to see such an IPC be created, and two years ago thought it would be possible to achieve here. But over time, and in working with the D-Bus model and requirements, it just didn't happen here. Given that no one has ever been able to accomplish such a thing in the past means that it's either impossible to do, or that no one really wants such a thing bad enough to actually do the work :)
What is the rush?
Quoted 3 lines
Did I miss anything else here? Are there any technical reasons I'm forgetting about for why this can't be pulled in as-is for this merge window?
****The code has not been meaningfully or properly reviewed.****
Greg you are pushing entirely too hard for this code to get it. When someone pushes as hard as you are doing, inevitiable problems get through.
Greg it is my professional opinion that the code smells. There are all sort of missteps and oversights that indicate that almost certainly that something important has been overlooked.
I do not believe this code is yet up to the standards we want for core kernel code.
This code has astonishingly complex interactions with all kinds of other kernel subsystems and concerns. As a community we should understand them and accept them before letting them in.
The only way I have seen anything make meaninful progress with those kinds of interactions is for the pieces to be teased apart. And then the code incrementally added to with all of the right people being pulled into the discussion.
I suspect removing all of the extensions to the capabilities of dbus-1 would be a good start for getting a piece of code that could be meaningfully reviewed. Then the controversial bits can be addressed on their own. As it is there is too much for to properly address any one issue.
Greg this process has fundamentally not given people time to understand the code, the interactions or the complexities. The discussions show that.
Further by refusing to tease apart the pieces. By refusing to allow other people time to understand this code. By refusing to give an inch and admit anyone else has a valid point real problems, and real issues can not be revealed and fixed.
With such a pig headed direction I do not believe that kdbus is in any sense ready for merging.
Eric
p.s. One of the issues of smell that I have been talking about is I see in kdbus patterns of code construction that have caused real world performance problems, and real world security issues. And those issues get ignored when brought up.
p.p.s Not that this complaint is not in any sense new you have been ignoring people who try to bring up meaningful issues for a long time. The fact that when people bring up uncomfortable points about the kdbus code they get routingely blown off certainly contributes to the lack of meaningful review as it is not rewarding to work with someone who does not listen to criticism. At this point the strongest possible language and the strongest possible push back are being used because everything else is routinely swept under the rug.