kdbus for 4.1-rc1

The following changes since commit 9eccca0843205f87c00404b663188b88eb248051:

Linux 4.0-rc3 (2015-03-08 16:09:09 -0700)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git/ tags/kdbus-4.1-rc1

for you to fetch changes up to 9fb9cd0f4434a23487b6ef3237e733afae90e336:

kdbus: avoid the use of struct timespec (2015-04-10 14:34:53 +0200)

----------------------------------------------------------------
kdbus for 4.1-rc1

Here's the kdbus pull request for 4.1-rc1.

It's been under development for many years now, and been in linux-next for many months, and has undergone loads of testing a review and even a few good arguments. It comes with full documentation and tests.

There has been a few complaints about the code, notably from people who don't like the use of metadata in the bus messages. That is actually one of the main features here, as we can get this data in a secure and reliable way, and it's something that userspace requires today. So while it does look "odd" to people who are not familiar with dbus, this is something that finally fixes a number of almost unfixable races in the current dbus implementations.

The rest of this pull request message comes from the kdbus patch posting messages as sent to lkml previously:

Reasons kdbus should be in the kernel, instead of userspace as it is currently done today includes the following:

* Performance: Fewer process context switches, fewer copies, fewer syscalls, larger memory chunks via memfd. This is really important for a whole class of userspace programs that are ported from other operating systems that are run on tiny ARM systems that rely on hundreds of thousands of messages passed at boot time, and at "critical" times in their user interaction loops. DBus is not used for performance sensitive applications because DBus is slow. We want to make it fast so we can finally use it for low-latency, high-throughput applications. A simple DBus method-call+reply takes 200us on an up-to-date test machine, with kdbus it takes 8us (with UDS about 2us). If the packet size is increased from 8k to 128k, kdbus even beats UDS due to single-copy transfers.

* Security: The peers which communicate do not have to trust each other, as the only trustworthy component in the game is the kernel which adds metadata and ensures that all data passed as payload is either copied or sealed, so that the receiver can parse the data

without having to protect against changing memory while parsing buffers. Also, all the data transfer is controlled by the kernel, so that LSMs can track and control what is going on, without involving userspace. Because of the LSM issue, security people are much happier with this model than the current scheme of having to hook into dbus to mediate things.
* More types of metadata can be attached to messages than in userspace

Quoted 21 lines · Greg Kroah-Hartman

The following changes since commit 9eccca0843205f87c00404b663188b88eb248051:

Linux 4.0-rc3 (2015-03-08 16:09:09 -0700)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git/ tags/kdbus-4.1-rc1

for you to fetch changes up to 9fb9cd0f4434a23487b6ef3237e733afae90e336:

kdbus: avoid the use of struct timespec (2015-04-10 14:34:53 +0200)

----------------------------------------------------------------
kdbus for 4.1-rc1

Here's the kdbus pull request for 4.1-rc1.

It's been under development for many years now, and been in linux-next for many months, and has undergone loads of testing a review and even a few good arguments. It comes with full documentation and tests.

Quoted 7 lines

There has been a few complaints about the code, notably from people who don't like the use of metadata in the bus messages. That is actually one of the main features here, as we can get this data in a secure and reliable way, and it's something that userspace requires today. So while it does look "odd" to people who are not familiar with dbus, this is something that finally fixes a number of almost unfixable races in the current dbus implementations.

And the code that transfers the meta-data is wrong.

It is generally not something that userspace requires today, certainly userspace is not using it.

You are exporting a weird set of information in a unique way that makes it race free enough to make ``security'' decisions upon but the data in general is not appropriate to make those decisions.

I remain opposed to this half thought out trash of an ABI for the meta-data.

Just because something happens to be exported in a DEBUG api today does not make it appropriate for userspace to run around making security decisions with that information.

Quoted 4 lines

* Performance: Fewer process context switches, fewer copies, fewer syscalls, larger memory chunks via memfd. This is really important for a whole class of userspace programs that are ported from other operating systems that are run on tiny ARM systems that rely on

Quoted 8 lines

hundreds of thousands of messages passed at boot time, and at "critical" times in their user interaction loops. DBus is not used for performance sensitive applications because DBus is slow. We want to make it fast so we can finally use it for low-latency, high-throughput applications. A simple DBus method-call+reply takes 200us on an up-to-date test machine, with kdbus it takes 8us (with UDS about 2us). If the packet size is increased from 8k to 128k, kdbus even beats UDS due to single-copy transfers.

And with a good design kdbus could be faster.

Quoted 4 lines

* Security: The peers which communicate do not have to trust each other, as the only trustworthy component in the game is the kernel which adds metadata and ensures that all data passed as payload is either copied or sealed, so that the receiver can parse the data

Quoted 8 lines

without having to protect against changing memory while parsing buffers. Also, all the data transfer is controlled by the kernel, so that LSMs can track and control what is going on, without involving userspace. Because of the LSM issue, security people are much happier with this model than the current scheme of having to hook into dbus to mediate things.
* More types of metadata can be attached to messages than in userspace

The meta-data is poorly thought and and much of it is not appropriate for making security decisions anywhere except in the kernel.

All I have seen with the meta-data discussion is sticking heads in the sand and resubmitting and hoping your reviewers go away.

If you won't do a good responsible job on this before the code is merged how can we possibly expect you to do a good job later. Or is this going to be another API where userspace will be broken at arbitrary moments by arbitrary users?

How are you going to fix the security issues your poor API comes with it when then are eventually spelled out clearly and to fix them means breaking everyones desktop environment?

Quoted 2 lines

I remain opposed to this half thought out trash of an ABI for the meta-data.

You don't have to enable the metadata if you don't want to use it, it's an option :)

Quoted 5 lines · Greg Kroah-Hartman

Quoted 2 lines

I remain opposed to this half thought out trash of an ABI for the meta-data.

You don't have to enable the metadata if you don't want to use it, it's an option :)

Wasn't this also an argument for CONFIG_CGROUPS?
Now we're forced to enable it by default to boot a recent distro and CONFIG_CGROUPS is still not fixed.

Quoted 8 lines

Quoted 5 lines · Greg Kroah-Hartman

Quoted 2 lines

I remain opposed to this half thought out trash of an ABI for the meta-data.

You don't have to enable the metadata if you don't want to use it, it's an option :)

Quoted 3 lines

Wasn't this also an argument for CONFIG_CGROUPS?
Now we're forced to enable it by default to boot a recent distro and CONFIG_CGROUPS is still not fixed.

CONFIG_CGROUPS is "not fixed"? I think Tejun would like to have some words with you :)

Am 13.04.2015 um 21:54 schrieb Greg Kroah-Hartman:

Quoted 13 lines · Richard Weinberger

Quoted 11 lines

Quoted 5 lines · Greg Kroah-Hartman

Quoted 2 lines

I remain opposed to this half thought out trash of an ABI for the meta-data.

You don't have to enable the metadata if you don't want to use it, it's an option :)

Wasn't this also an argument for CONFIG_CGROUPS?
Now we're forced to enable it by default to boot a recent distro and CONFIG_CGROUPS is still not fixed.

Quoted 2 lines

CONFIG_CGROUPS is "not fixed"? I think Tejun would like to have some words with you :)

Tejun is working on it and does a *very* good job. But as long the unified hirarchy is not complete/stable we're facing issues.
Ever tried to run systemd a linux container? ;)

Quoted 15 lines

Am 13.04.2015 um 21:54 schrieb Greg Kroah-Hartman:

Quoted 13 lines · Richard Weinberger

Quoted 11 lines

Quoted 5 lines · Greg Kroah-Hartman

Quoted 2 lines

I remain opposed to this half thought out trash of an ABI for the meta-data.

You don't have to enable the metadata if you don't want to use it, it's an option :)

Wasn't this also an argument for CONFIG_CGROUPS?
Now we're forced to enable it by default to boot a recent distro and CONFIG_CGROUPS is still not fixed.

Quoted 6 lines

Quoted 2 lines

CONFIG_CGROUPS is "not fixed"? I think Tejun would like to have some words with you :)

Tejun is working on it and does a *very* good job. But as long the unified hirarchy is not complete/stable we're facing issues.
Ever tried to run systemd a linux container? ;)

Works just fine for me, I do it daily. Here's how I spin up a debian image on my local filesystem, running systemd within it just swimmingly:
sudo systemd-nspawn -D debian/ /sbin/init

Also works just fine with gentoo and arch images, both of which I use on a weekly basis in this manner.

Perhaps you are doing something odd that prevents this from working for you?

Quoted 2 lines

I remain opposed to this half thought out trash of an ABI for the meta-data.

You don't have to enable the metadata if you don't want to use it, it's an option :)

Quoted 5 lines · Greg Kroah-Hartman

Quoted 2 lines

I remain opposed to this half thought out trash of an ABI for the meta-data.

You don't have to enable the metadata if you don't want to use it, it's an option :)

OK, _that_ argument needs to be stomped out. It had been used before, and it was a deliberate scam. There is no such thing as optional kernel interface, especially when udev/dbus/systemd crowd is nearby. We'd been through that excuse before; remember how devtmpfs was pushed in as "optional"?

This is a huge red flag. On the level of "I need your account information to transfer $200M you might have inherited from my deceased client".

Just to recap how it went the last time around: Kay kept pushing his piece of code into the tree, claiming that it was optional, that nobody who doesn't like it has to enable it, so what's the problem? OK, in it went. And pretty soon udev (maintained by the same... meticulously honorable person) had stopped working on the kernels that didn't have that enabled.

We had been there before. To paraphrase another... meticulously honorable person, "if you didn't want something relied upon, why have you put it into the kernel?" Said person is on the record as having no problem whatsoever with adding dependencies to the bottom of userland stack.

IMO either it's OK without "if you don't like it, don't enable it", or it should not be merged at all.

Quoted 5 lines · Greg Kroah-Hartman

Quoted 2 lines

I remain opposed to this half thought out trash of an ABI for the meta-data.

You don't have to enable the metadata if you don't want to use it, it's an option :)

OK, _that_ argument needs to be stomped out. It had been used before, and it was a deliberate scam. There is no such thing as optional kernel interface, especially when udev/dbus/systemd crowd is nearby. We'd been through that excuse before; remember how devtmpfs was pushed in as "optional"?

This is a huge red flag. On the level of "I need your account information to transfer $200M you might have inherited from my deceased client".

Just to recap how it went the last time around: Kay kept pushing his piece of code into the tree, claiming that it was optional, that nobody who doesn't like it has to enable it, so what's the problem? OK, in it went. And pretty soon udev (maintained by the same... meticulously honorable person) had stopped working on the kernels that didn't have that enabled.

We had been there before. To paraphrase another... meticulously honorable person, "if you didn't want something relied upon, why have you put it into the kernel?" Said person is on the record as having no problem whatsoever with adding dependencies to the bottom of userland stack.

Quoted 4 lines

We had been there before. To paraphrase another... meticulously honorable person, "if you didn't want something relied upon, why have you put it into the kernel?" Said person is on the record as having no problem whatsoever with adding dependencies to the bottom of userland stack.

It appears that, if kdbus is merged, upstream udev may end up requiring it:

http://lists.freedesktop.org/archives/systemd-devel/2014-May/019657.html

On Wed, Apr 15, 2015 at 3:36 AM, Andy Lutomirski <[email protected]

Quoted 9 lines

Quoted 7 lines · wrote:

We had been there before. To paraphrase another... meticulously honorable
person, "if you didn't want something relied upon, why have you put it into the
kernel?" Said person is on the record as having no problem whatsoever with
adding dependencies to the bottom of userland stack.

Quoted 4 lines

It appears that, if kdbus is merged, upstream udev may end up requiring it:

http://lists.freedesktop.org/archives/systemd-devel/2014-May/019657.html

Why so surprised?
kdbus will be a major hard-dependency for every non-trivial userland. Like cgroups...

it seems [systemd] has now mandated group scheduling.

Quoted 1 line · Mike Galbraith

it seems [systemd] has now mandated group scheduling.

What makes you think so?

systemctl mask [email protected]

Quoted 1 line · Mike Galbraith

it seems [systemd] has now mandated group scheduling.

What makes you think so? Was it the fact that by default you have a populated /sys/fs/cgroup/cpu/ hierarchy? This is either because some unit requests the use of the cpu controller using one of the CPU*= directives from systemd.resource-control(5), or (perhaps more likely) because there is a privileged unit with Delegate=yes. The most likely candidate is [email protected], and so you could try preventing it from starting:
systemctl mask [email protected]

Note that systemd still works without group scheduling or any cgroup subsystems enabled in the kernel:

  $ grep GROUP .config
  CONFIG_CGROUPS=y

Quoted 5 lines · Andy Lutomirski

Quoted 4 lines

We had been there before. To paraphrase another... meticulously honorable person, "if you didn't want something relied upon, why have you put it into the kernel?" Said person is on the record as having no problem whatsoever with adding dependencies to the bottom of userland stack.

Quoted 3 lines

It appears that, if kdbus is merged, upstream udev may end up requiring it:

http://lists.freedesktop.org/archives/systemd-devel/2014-May/019657.html

Why so surprised?
kdbus will be a major hard-dependency for every non-trivial userland. Like cgroups...

Quoted 10 lines · Andy Lutomirski

Quoted 8 lines

Quoted 4 lines

We had been there before. To paraphrase another... meticulously honorable person, "if you didn't want something relied upon, why have you put it into the kernel?" Said person is on the record as having no problem whatsoever with adding dependencies to the bottom of userland stack.

It appears that, if kdbus is merged, upstream udev may end up requiring it:

http://lists.freedesktop.org/archives/systemd-devel/2014-May/019657.html

Quoted 3 lines

Why so surprised?
kdbus will be a major hard-dependency for every non-trivial userland. Like cgroups...

Maybe because things like cgroups, and kdbus in the future, solves a need that the developers in that area have to solve problems and provide functionality that their users require?

Look, us kernel developers only work on one huge, multithreaded, global state binary. Our experience in multi-application interactions with shared state and permission requirements is usually quite limited. If you don't trust the developers of those programs outside the kernel, don't use them, there are still distros out there that don't require them.

Am 15.04.2015 um 10:48 schrieb Greg Kroah-Hartman:

Quoted 15 lines · Richard Weinberger

Quoted 13 lines · Andy Lutomirski

Quoted 8 lines

Quoted 4 lines

We had been there before. To paraphrase another... meticulously honorable person, "if you didn't want something relied upon, why have you put it into the kernel?" Said person is on the record as having no problem whatsoever with adding dependencies to the bottom of userland stack.

It appears that, if kdbus is merged, upstream udev may end up requiring it:

http://lists.freedesktop.org/archives/systemd-devel/2014-May/019657.html

Why so surprised?
kdbus will be a major hard-dependency for every non-trivial userland. Like cgroups...

Quoted 3 lines

Maybe because things like cgroups, and kdbus in the future, solves a need that the developers in that area have to solve problems and provide functionality that their users require?

I agree that a high level bus is needed and dbus is not perfect. But this does not mean that we need a in-kernel dbus in any case.

Quoted 6 lines

Look, us kernel developers only work on one huge, multithreaded, global state binary. Our experience in multi-application interactions with shared state and permission requirements is usually quite limited. If you don't trust the developers of those programs outside the kernel, don't use them, there are still distros out there that don't require them.

We're all forced to use cgroups, systemd, udev unless we want to have busybox as userland. That's a fact.

systemd and its dependencies are not a bad thing per se. But we have to be very sure that new hard-dependencies are in well shape before we push them into the kernel.

Quoted 2 lines

We're all forced to use cgroups, systemd, udev unless we want to have busybox as userland. That's a fact.

Is that a problem?

Quoted 4 lines · Greg Kroah-Hartman

Quoted 2 lines

We're all forced to use cgroups, systemd, udev unless we want to have busybox as userland. That's a fact.

Is that a problem?

I'm amazed that you're really actually asking that question :-(

Quoted 4 lines · Greg Kroah-Hartman

Quoted 3 lines

Quoted 2 lines

We're all forced to use cgroups, systemd, udev unless we want to have busybox as userland. That's a fact.

Quoted 3 lines

Quoted 1 line

Is that a problem?

I'm amazed that you're really actually asking that question :-(

Really? Why can't userspace rely on the features that the kernel provides them? If not, why would the feature be created and supported by us kernel developers in the first place?

Am 15.04.2015 um 11:27 schrieb Greg Kroah-Hartman:

Quoted 9 lines · Borislav Petkov

Quoted 7 lines · Greg Kroah-Hartman

Quoted 4 lines

Quoted 2 lines

We're all forced to use cgroups, systemd, udev unless we want to have busybox as userland. That's a fact.

Is that a problem?

I'm amazed that you're really actually asking that question :-(

Quoted 3 lines

Really? Why can't userspace rely on the features that the kernel provides them? If not, why would the feature be created and supported by us kernel developers in the first place?

This IMHO not the problem.
But if we add a new component to the kernel which *will* be used by almost every userland out there (systemd won the "init wars") we have to make sure that we're all fine with it.

Andy and Eric have some very valid concerns.

Quoted 14 lines

Am 15.04.2015 um 11:27 schrieb Greg Kroah-Hartman:

Quoted 12 lines · Borislav Petkov

Quoted 7 lines · Greg Kroah-Hartman

Quoted 4 lines

Quoted 2 lines

We're all forced to use cgroups, systemd, udev unless we want to have busybox as userland. That's a fact.

Is that a problem?

I'm amazed that you're really actually asking that question :-(

Really? Why can't userspace rely on the features that the kernel provides them? If not, why would the feature be created and supported by us kernel developers in the first place?

Quoted 4 lines

This IMHO not the problem.
But if we add a new component to the kernel which *will* be used by almost every userland out there (systemd won the "init wars") we have to make sure that we're all fine with it.

Sure, but why would this be different from any other kernel feature that we add? We have to be sure we are fine with everything we merge, as we are saying we are going to maintain this stuff for forever.

Quoted 1 line

Andy and Eric have some very valid concerns.

I've tried to address Andy's concerns, Eric is not being very specific, so there's nothing I can do there :)

Quoted 7 lines · Borislav Petkov

Quoted 4 lines · Greg Kroah-Hartman

Quoted 2 lines

We're all forced to use cgroups, systemd, udev unless we want to have busybox as userland. That's a fact.

Is that a problem?

I'm amazed that you're really actually asking that question :-(

Really? Why can't userspace rely on the features that the kernel provides them?

Quoted 9 lines · Greg Kroah-Hartman

Quoted 7 lines · Borislav Petkov

Quoted 4 lines · Greg Kroah-Hartman

Quoted 2 lines

We're all forced to use cgroups, systemd, udev unless we want to have busybox as userland. That's a fact.

Is that a problem?

I'm amazed that you're really actually asking that question :-(

Quoted 2 lines

Really? Why can't userspace rely on the features that the kernel provides them?

Userspace can do whatever it wants. As long as I'm not being *forced* to do what userspace thinks is the right thing.

It seems to me that since that whole systemd* debacle started, we're forgetting the choice aspect.

And dammit, I want my choice. I want to be able to choose what I'm running. Not run what someone else thought what would be good for me to run. If I wanted that, I'd long switched to windoze or äbble.

So stop making this discussion be about "oh those horrid systemd developers, I don't want their code as my init system" as that's not what any of this is about at all. It's about the patches being proposed, and the API involved in it. Please stick to that.

You have to trust someone to help make your system work together in a unified way. If you can't trust your distro's engineers, then either start your own distro, or only run busybox on top of a kernel. You really don't have much other "choice" than that :)

And obviously there is a lack of trust. And once kdbus is in, we must use it, or support our own distro where we just do not have the time.

Personally, I'm fine with getting something in that will help userspace tools work better. The issue I see, mostly from the side lines as I haven't totally submerged myself into the dbus protocol (I think I should spend some time to do just that), this is going too fast. Once it is in the kernel, whatever ABI we expose is locked in stone. There's no changing it. We need to make sure that this is well thought out. People seem to be of the impression that the current dbus design has flaws, but because everything relies on it we must still push it into the kernel because it mimics what is out there in user space. I disagree.

As others have said. We do not need to follow the dbus design. If we can supply a better transport layer than what the kernel supplies today, then tools will eventually merge to it away from dbus. Perhaps the kernel can supply just enough to have dbus improve its speed, but not with the entire complex solution that kdbus is presenting today.

The in-kernel code isn't a lot (again, 13k lines, smaller than almost all of the drivers you are using today on an individual basis) It's

Quoted 2 lines · Greg Kroah-Hartman

The in-kernel code isn't a lot (again, 13k lines, smaller than almost all of the drivers you are using today on an individual basis) It's

I originally didn't want to comment on this, but now that you are making this argument for 3rd or 4th time, I can't really resist. What exactly are you trying to "prove" by the 13k-lines argument?

mm/vmscan.c is less that 4k lines. Does that sole fact mean that the whole memory reclaim is trivial to review?

We merge subsystems with new userspace apis that are large than this all the time. I'm trying to say this isn't something "unusual" at all.

Quoted 6 lines

I originally didn't want to comment on this, but now that you are making this argument for 3rd or 4th time, I can't really resist. What exactly are you trying to "prove" by the 13k-lines argument?

mm/vmscan.c is less that 4k lines. Does that sole fact mean that the whole memory reclaim is trivial to review?

I'm trying to say that it's not a ton of code. lines of code are of course not a valid way to judge complexity, and I'm not trying to say that. I am trying to point out that it isn't "huge" by comparing it to other chunks of code that we all know and love.

We merge subsystems with new userspace apis that are large than this all the time. I'm trying to say this isn't something "unusual" at all.

Quoted 3 lines · Steven Rostedt

And obviously there is a lack of trust. And once kdbus is in, we must use it, or support our own distro where we just do not have the time.

Just like cgroups, and ftrace :)

Quoted 10 lines

Personally, I'm fine with getting something in that will help userspace tools work better. The issue I see, mostly from the side lines as I haven't totally submerged myself into the dbus protocol (I think I should spend some time to do just that), this is going too fast. Once it is in the kernel, whatever ABI we expose is locked in stone. There's no changing it. We need to make sure that this is well thought out. People seem to be of the impression that the current dbus design has flaws, but because everything relies on it we must still push it into the kernel because it mimics what is out there in user space. I disagree.

"fast"? Are you kidding me? This stuff has been under active, public, development for over two years. We have been posting public patches, asking for review and comments for _months_ now. Given that there were no more specific review comments on the patch set, and its success in linux-next for almost the entire 4.0 development cycle, I asked it to be merged.

I don't know too many other kernel features/drivers that have taken this long, or done this "slowly", do you?

Quoted 6 lines

As others have said. We do not need to follow the dbus design. If we can supply a better transport layer than what the kernel supplies today, then tools will eventually merge to it away from dbus. Perhaps the kernel can supply just enough to have dbus improve its speed, but not with the entire complex solution that kdbus is presenting today.

I originally thought this would work too. 8 months of work later, I was proven wrong, that will not work. Or it imposes too much additional work on userspace that really makes no sense at all. The in-kernel code isn't a lot (again, 13k lines, smaller than almost all of the drivers you are using today on an individual basis) It's also really fast, but with benchmarks, David and Andy have found some minor bottlenecks that can make things faster.

Yes it seems complex, but read the documentation to get an idea of what is happening here. I think you will get a better appreciation of what is going on.

I read a bit of the documentation, but not enough. I really need to sit down and play with code. That's the way I learn and understand.

Quoted 2 lines

I don't know too many other kernel features/drivers that have taken this long, or done this "slowly", do you?

What other features/drivers that you know introduce a major new IPC user space interface that will be a core component of the system?

Quoted 5 lines · Steven Rostedt

Quoted 2 lines

I don't know too many other kernel features/drivers that have taken this long, or done this "slowly", do you?

What other features/drivers that you know introduce a major new IPC user space interface that will be a core component of the system?

We've been merging these about one every other kernel release for a while now. Look at the drivers/misc/mic/

Quoted 3 lines

Yes it seems complex, but read the documentation to get an idea of what is happening here. I think you will get a better appreciation of what is going on.

I read a bit of the documentation, but not enough. I really need to sit down and play with code. That's the way I learn and understand.

Quoted 3 lines · Greg Kroah-Hartman

Quoted 2 lines · Borislav Petkov

Quoted 1 line

On Wed, Apr 15, 2015 at 11:20:34AM +0200, Greg Kroah-Hartman

Quoted 7 lines

Quoted 7 lines

Quoted 6 lines

Quoted 4 lines

Quoted 2 lines

We're all forced to use cgroups, systemd, udev unless we want to have busybox as userland. That's a fact.

Is that a problem?

I'm amazed that you're really actually asking that question :-(

Quoted 8 lines

Quoted 2 lines

Really? Why can't userspace rely on the features that the kernel provides them?

Userspace can do whatever it wants. As long as I'm not being *forced* to do what userspace thinks is the right thing.

It seems to me that since that whole systemd* debacle started, we're forgetting the choice aspect.

What "choice" aspect? Surely you aren't going to make the "Linux is about choice" argument are you?

Quoted 4 lines

And dammit, I want my choice. I want to be able to choose what I'm running. Not run what someone else thought what would be good for me to
run. If I wanted that, I'd long switched to windoze or äbble.

Oh crap, you went there :)

Take a look at http://www.islinuxaboutchoice.com/ please.

Quoted 16 lines

Am 15.04.2015 um 10:48 schrieb Greg Kroah-Hartman:

Quoted 15 lines · Richard Weinberger

Quoted 13 lines · Andy Lutomirski

Quoted 8 lines

Quoted 4 lines

We had been there before. To paraphrase another... meticulously honorable person, "if you didn't want something relied upon, why have you put it into the kernel?" Said person is on the record as having no problem whatsoever with adding dependencies to the bottom of userland stack.

It appears that, if kdbus is merged, upstream udev may end up requiring it:

http://lists.freedesktop.org/archives/systemd-devel/2014-May/019657.html

Why so surprised?
kdbus will be a major hard-dependency for every non-trivial userland. Like cgroups...

Quoted 6 lines

Quoted 3 lines

Maybe because things like cgroups, and kdbus in the future, solves a need that the developers in that area have to solve problems and provide functionality that their users require?

I agree that a high level bus is needed and dbus is not perfect. But this does not mean that we need a in-kernel dbus in any case.

So what do you propose to solve the issues presented in my original email about the usecases that this code addresses?

Quoted 9 lines

Quoted 6 lines

Look, us kernel developers only work on one huge, multithreaded, global state binary. Our experience in multi-application interactions with shared state and permission requirements is usually quite limited. If you don't trust the developers of those programs outside the kernel, don't use them, there are still distros out there that don't require them.

We're all forced to use cgroups, systemd, udev unless we want to have busybox as userland. That's a fact.

Is that a problem?

Quoted 3 lines

systemd and its dependencies are not a bad thing per se. But we have to be very sure that new hard-dependencies are in well shape before we push them into the kernel.

That's fine, and normal, and I expect it. But please provide technical reasons why the proposal is not acceptable, like Andy has done in this thread.

Look, us kernel developers only work on one huge, multithreaded, global state binary. Our experience in multi-application interactions with shared state and permission requirements is usually quite limited. If you don't trust the developers of those programs outside the kernel, don't use them, there are still distros out there that don't require them.

dbus is not an appropriate design for a kernel messaging layer for a variety of reasons. That's not to say dbus shouldn't be able to use a fast kernel messaging layer, or that one shouldn't exist.

dbus is basically a very large very specialized and somewhat flawed policy engine on top of what should be simple messaging. The two need splitting apart.

Abstract low level messaging layers are not a new concept. V7 unix had one experimentally. It's about getting the separation right.

IMHO that probably involves getting the right people in the right place together - dbus designers, MPI and realtime people, kernel folks and possibly also some of the hardware messaging folk.

In filesystem terms

- stop writing a dbus only file system
- figure out what a messaging "vfs" looks like
- figure out what an clean low level kernel model looks like
- figure out what has to be where to put the policy in userspace

What might also be worth review is how much dbus traffic actually ought to be an object store implemented say with tmpfs and inotify type functionality (or extensions of that) so that you can set/read/enumerate/get change notifications on properties.

IMHO that probably involves getting the right people in the right place together - dbus designers, MPI and realtime people, kernel folks and possibly also some of the hardware messaging folk.

Quoted 4 lines · One Thousand Gnomes

IMHO that probably involves getting the right people in the right place together - dbus designers, MPI and realtime people, kernel folks and possibly also some of the hardware messaging folk.

/me continues on as a broken record

I suggest that we can do this at Linux Plumbers, and then follow up at Kernel Summit, for those that can (or wont) attend plumbers.

-- Steve

Quoted 4 lines · One Thousand Gnomes

IMHO that probably involves getting the right people in the right place together - dbus designers, MPI and realtime people, kernel folks and possibly also some of the hardware messaging folk.

/me continues on as a broken record

I suggest that we can do this at Linux Plumbers, and then follow up at Kernel Summit, for those that can (or wont) attend plumbers.

Quoted 2 lines

I suggest that we can do this at Linux Plumbers, and then follow up at Kernel Summit, for those that can (or wont) attend plumbers.

I really doubt this will work for Plumbers, sorry. And technical things don't work well, if at all, at Kernel Summit.

We have had meetings about this at the past two Plumbers conferences, where none of these things came up (i.e. dislike of the D-Bus model).

I'll be glad to discuss this at both places, but let's try to work through the technical things through email, as really, that's the best place for it.

Al just proved this by pointing out some issues to be resolved (RW lock only used as a W lock, odd atomic values and locking without documenting the lifecycles, etc.) And that's the way this is supposed to work, nothing new/different here that I can see.

Quoted 3 lines · On Wed, 15 Apr 2015 18:35:20 +0200 Greg Kroah-Hartman…

Quoted 2 lines

I suggest that we can do this at Linux Plumbers, and then follow up at Kernel Summit, for those that can (or wont) attend plumbers.

Quoted 5 lines

I really doubt this will work for Plumbers, sorry. And technical things don't work well, if at all, at Kernel Summit.

We have had meetings about this at the past two Plumbers conferences, where none of these things came up (i.e. dislike of the D-Bus model).

But were the people that are not liking it at those conference sessions?

Quoted 4 lines

I'll be glad to discuss this at both places, but let's try to work through the technical things through email, as really, that's the best place for it.

Quoted 4 lines

Al just proved this by pointing out some issues to be resolved (RW lock only used as a W lock, odd atomic values and locking without documenting the lifecycles, etc.) And that's the way this is supposed to work, nothing new/different here that I can see.

But you are missing one of the complaints that I'm reading from people. The proposed ABI is too complex. Do we really want to jump into having to support another tty layer?

One thing that I think may be really worth doing is that everyone on this thread that has not yet done so, write a simple dbus application to try to understand its design. Break it down to the requirements that are needed, and discuss that.

Is there a reason that this patch must go in this merge window?

Having something this controversial take place during the merge window suggests its a bit premature to push in now.

Quoted 1 line

But were the people that are not liking it at those conference sessions?

People who don't like a topic, usually go to a session about it, why would they? :)

Quoted 3 lines

But you are missing one of the complaints that I'm reading from people. The proposed ABI is too complex. Do we really want to jump into having to support another tty layer?

Don't make idle comments, the tty layer is far more complex and larger

than the kdbus code, with much nastier issues and problems. And we handle that just fine :)

As far as the "support" issue, we have 4 people who are all experienced, senior kernel developers who are signed up to maintain this. There's more experience here for this one MAINTAINERS entry per line of code than I have seen in quite some time.

Quoted 4 lines

One thing that I think may be really worth doing is that everyone on this thread that has not yet done so, write a simple dbus application to try to understand its design. Break it down to the requirements that are needed, and discuss that.

I've done that, it's hard, use the gdbus interface instead, it makes your life much easier.

I'll again refer to ALSA here, no one writes a "raw" ALSA program, they all use the library to interact with the kernel. Do that here, there are wonderful dbus libraries out there, for all languages. Use them instead.

Quoted 2 lines

Is there a reason that this patch must go in this merge window?

What makes this merge window any different from any other? Again, I explained why I asked it to be merged at this point in time. If people have technical issues with it, I'll be more than glad to work them out and merge it later, there's no "hard and fast deadline" anyone is asking for here.

Quoted 3 lines

Having something this controversial take place during the merge window suggests its a bit premature to push in now.

"take place"? Have you been ignoring these patches posted numerous times for many months? This is the point in time to ask for code to be merged, just like any other code, nothing is special here.

Don't make idle comments, the tty layer is far more complex and larger than the kdbus code, with much nastier issues and problems. And we handle that just fine :)

I'll again refer to ALSA here, no one writes a "raw" ALSA program, they all use the library to interact with the kernel. Do that here, there are wonderful dbus libraries out there, for all languages. Use them instead.

Quoted 1 line

Is there a reason that this patch must go in this merge window?

What makes this merge window any different from any other? Again, I explained why I asked it to be merged at this point in time. If people have technical issues with it, I'll be more than glad to work them out and merge it later, there's no "hard and fast deadline" anyone is asking for here.

Quoted 3 lines

Having something this controversial take place during the merge window suggests its a bit premature to push in now.

"take place"? Have you been ignoring these patches posted numerous times for many months? This is the point in time to ask for code to be merged, just like any other code, nothing is special here.

On Wed, 15 Apr 2015 19:31:45 +0200

Quoted 3 lines

Don't make idle comments, the tty layer is far more complex and larger than the kdbus code, with much nastier issues and problems. And we handle that just fine :)

The tty layer is the way it is because of design decisions dating back 20 years that were (with hindsight) wrong coupled with the fact that POSIX took a lot of the behavioural guarantees from an armwaving claim about what Unix(tm) implemented without thinking about how to implement them (as far as I can tell - given many of the guarantees are broken in Unix!)

Quoted 4 lines

I'll again refer to ALSA here, no one writes a "raw" ALSA program, they all use the library to interact with the kernel. Do that here, there are wonderful dbus libraries out there, for all languages. Use them instead.

Agreed entirely - I don't disagree that we need a fast messaging layer. The question is what bits belong in kernel. Go wants one, JMS wants one, porting from stuff like QNX wants one (although they use the POSIX API on QNX), MPI wants one (but with some useful and subtly different semantics), various embedded things from tiny uKernels want one.

The question is what the kernel bit should actually look like, and how many we need.

My guess is that we actually have three of the big use cases covered

- futexes and shared memory cover the tiny uKernel emulation bits (and on a lawnmower engine sized ARM thats probably the only way to get the speed approaching that of a tiny rtos)
- posix queues cover things like QNX porting
- publish/subscribe - via tmpfs

but we don't cover

- multicasting
- some types of credential and authority passing
- scatter/gather without excessive userspace wakes

This is functionally identical to passing AF_UNIX socket fds over SCM_RIGHTS, but I want something much lighter weight.

Quoted 2 lines · Andy Lutomirski

This is functionally identical to passing AF_UNIX socket fds over SCM_RIGHTS, but I want something much lighter weight.

Most of the weight in SCM_RIGHTS comes from the fact that you can pass AF_UNIX sockets over it, which requires a garbage collector. Exclude that and suddenly it becomes very cheap...

Quoted 8 lines · Al Viro

Quoted 2 lines · Andy Lutomirski

This is functionally identical to passing AF_UNIX socket fds over SCM_RIGHTS, but I want something much lighter weight.

Most of the weight in SCM_RIGHTS comes from the fact that you can pass AF_UNIX sockets over it, which requires a garbage collector. Exclude that and suddenly it becomes very cheap...

I should have been more specific. I don't mean the performance of SCM_RIGHTS itself; I mean the memory overhead of keeping tons of fds around, each with their socket data structures and buffers.

I think that dbus could be quite efficiently implemented with a userspace daemon that just introduces peers to each other, but the fd explosion could be rather bad for some use cases.

I'll be the first to admit that I don't have a clean API in mind. There was a lightweight fd proposal way back when, but it never went anywhere, and it might not be suitable anyway.

Quoted 14 lines · Andy Lutomirski

Quoted 8 lines · Al Viro

Quoted 2 lines · Andy Lutomirski

This is functionally identical to passing AF_UNIX socket fds over SCM_RIGHTS, but I want something much lighter weight.

Most of the weight in SCM_RIGHTS comes from the fact that you can pass AF_UNIX sockets over it, which requires a garbage collector. Exclude that and suddenly it becomes very cheap...

I should have been more specific. I don't mean the performance of SCM_RIGHTS itself; I mean the memory overhead of keeping tons of fds around, each with their socket data structures and buffers.

Quoted 7 lines

I think that dbus could be quite efficiently implemented with a userspace daemon that just introduces peers to each other, but the fd explosion could be rather bad for some use cases.

I'll be the first to admit that I don't have a clean API in mind. There was a lightweight fd proposal way back when, but it never went anywhere, and it might not be suitable anyway.

Wait, are you talking about the overhead of descriptors used for capability tokens (essentially zero - one system-wide struct file per capability + one pointer in descriptor table of anyone who holds it + two bits in bitmaps in the sam descriptor tables) or about the overhead of descriptors used to send/receive those over? The latter don't have to be sockets at all - they could bloody well be files on some ipcfs, or character device, or FIFOs, etc.

Huh, interesting.

I was imagining that each of a server's peers (capability holders) would have a fresh struct file, but maybe this wouldn't be needed at all. You'd still need a way to get replies to your request, but the API could just as easily be:

int send_to_capability(int dest, int source, const void *data, size_t len, ...);

where dest would be the destination's fd and source would be whatever receive queue I expect the response on.

So maybe this is feasible. It doesn't solve broadcasts, but dbus unicast could easily layer over a facility like this and the context switch problem would go away for unicast.

Heck, I'd use it for my own proprietary stuff, too. It would be way easier than the absurd tangle of socketpairs I currently use.

Quoted 12 lines · Andy Lutomirski

Huh, interesting.

I was imagining that each of a server's peers (capability holders) would have a fresh struct file, but maybe this wouldn't be needed at all. You'd still need a way to get replies to your request, but the API could just as easily be:

int send_to_capability(int dest, int source, const void *data, size_t len, ...);

where dest would be the destination's fd and source would be whatever receive queue I expect the response on.

Quoted 6 lines

So maybe this is feasible. It doesn't solve broadcasts, but dbus unicast could easily layer over a facility like this and the context switch problem would go away for unicast.

Heck, I'd use it for my own proprietary stuff, too. It would be way easier than the absurd tangle of socketpairs I currently use.

BTW, the main issue with AF_UNIX passing is that recepient isn't asleep awaiting for descriptors - they are thrown by sender at whoever's receiving and sit there until somebody gets around to picking them.

_IF_ we had
client: I want a desciptor <goes to sleep, interruptibly>
kernel: assign it a sequence number
server: sees request (including sequence number)
server: give this fd to originator of request #N
kernel: check if originator is still there, insert the damn thing into their descriptor table if they still are and return the obtained number or
server: tell the originator of request #N to fuck off
kernel: check if originator is still there and gleefully pass the "fuck off" if they still are

we wouldn't have the in-flight state at all, and there goes the garbage collection shite. With some elaboration, it could even carry the authentication traffic - "fuck off" might be "answer this challenge", with the next "I want a descriptor" carrying reply...

I wonder if we could get away with having the receiver pre-allocate some placeholder fds and then have the kernel replace a placeholder with a passed fd immediately when the fd is sent and enqueue *that* in the cmsg data. If you send an fd to someone who hasn't assigned any placeholders to the receiving socket, then you get an error.

Quoted 5 lines · Andy Lutomirski

I wonder if we could get away with having the receiver pre-allocate some placeholder fds and then have the kernel replace a placeholder with a passed fd immediately when the fd is sent and enqueue *that* in the cmsg data. If you send an fd to someone who hasn't assigned any placeholders to the receiving socket, then you get an error.

*UGH*

It's a really bad idea. The thing is, descriptor table that isn't shared is assumed to be unchanged. So when fdget() looks a file up, it doesn't have to bump its refcount - the reference in descriptor table itself will stay. Conversely, fdput() doesn't have to drop it in such case (we encode whether we need to drop into struct fd returned by fdget() and passed to fdput()).

That relies on no third-party modifications of descriptor table and yes, the effect _is_ noticable - playing with struct file refcounts does result in considerable overhead.

If recepient sits in "gimme a descriptor", we are fine - if descriptor table was shared, the other users would be doing full refcount song and dance and if it wasn't, recepient is the sole user _and_ it isn't betwee fdget() and fdput() at the moment. With your "replace the dummies when sending" trick we break all of that - we don't know what the recepient is doing at the moment and for all we know they might be in the middle of something like e.g. fstat() on your placeholder. With rather unpleasant effects...

Quoted 14 lines · Andy Lutomirski

Quoted 8 lines · Al Viro

Quoted 2 lines · Andy Lutomirski

This is functionally identical to passing AF_UNIX socket fds over SCM_RIGHTS, but I want something much lighter weight.

Most of the weight in SCM_RIGHTS comes from the fact that you can pass AF_UNIX sockets over it, which requires a garbage collector. Exclude that and suddenly it becomes very cheap...

I should have been more specific. I don't mean the performance of SCM_RIGHTS itself; I mean the memory overhead of keeping tons of fds around, each with their socket data structures and buffers.

Quoted 7 lines

I think that dbus could be quite efficiently implemented with a userspace daemon that just introduces peers to each other, but the fd explosion could be rather bad for some use cases.

I'll be the first to admit that I don't have a clean API in mind. There was a lightweight fd proposal way back when, but it never went anywhere, and it might not be suitable anyway.

Wait, are you talking about the overhead of descriptors used for capability tokens (essentially zero - one system-wide struct file per capability + one pointer in descriptor table of anyone who holds it + two bits in bitmaps in the sam descriptor tables) or about the overhead of descriptors used to send/receive those over? The latter don't have to be sockets at all - they could bloody well be files on some ipcfs, or character device, or FIFOs, etc.

Also, getting the really high performance stuff right would be nice. Binder has one thing going for it (IIRC -- I've talked about it to some of the authors, but I've never so much as glanced at the code):
it has a primitive to send and wait for a reply. This reduces the load on scheduler.

Quoted 5 lines · Greg Kroah-Hartman

'systemctl reboot' calls a bunch of other things to determine if you have local access to the machine, or permissions to reboot the machine (i.e. CAP_SYS_BOOT), and other things that polkit might allow you to do, and then, it decides to reboot or not. That happens today, right? I don't understand the argument here.

And what exactly is the argument that this is the way it should be implemnted?

Why can't it just rely on the kernel to provide final answer to "to reboot or not to reboot, that is the question"?

At the end of the day, it's the kernel that decides whether it will really ultimately ask the platform to reboot.

If, for whatever reason (which might be completely invisible to userspace) kernel decides not to do so, userspace has to be able to recover from such failure in any case.

to a shutdown PID 1 process and finally a transition back to the initial ramdisk so that we can unmount the root file system even.

Quoted 3 lines

to a shutdown PID 1 process and finally a transition back to the initial ramdisk so that we can unmount the root file system even.

Is that wishful thinking or actually implemented somewhere?

Moreover, the daemon performing the shutdown tasks is necessarily always privileged enough to do so, so calling into the kernel and see what happens is completely the wrong thing to do (it would simply succeed). What matters is if the client calling the daemon is sufficiently privileged. If the client has the capabilites necessary to call the reboot syscall directly, it makes no sense to disallow them from doing a clean reboot. It would be like giving someone access to pull the power plug, but not allow them to shutdown the machine cleanly.

To conclude, the kernel makes the decision for allowing reboot() to succeed based on CAP_SYS_BOOT, so when we decide whether or not to perform the preparation steps, we really must also use CAP_SYS_BOOT. If we are more restrictive, it does not gain us anything as people with CAP_SYS_BOOT can just circumvent our logic and "pull the plug" by calling reboot() directly. If we are less restrictive and for instance check for uid==0 it would essentially mean that we have added a way to circumvent the dropping of CAP_SYS_BOOT.

HeHeHe. You mean all I need to do to get around all of the logging servers is capture CAP_SYS_BOOT? Say like just capture this crazy watchdog program that doesn't run as root so that it can only reboot the system? HeHeHe So I can just trigger a clean reboot wait for journald, auditd, and syslog all to shut down and then do evil things to the machine without having to worry about erasing forensic evidence?

Quoted 7 lines · Eric W. Biederman

HeHeHe. You mean all I need to do to get around all of the logging servers is capture CAP_SYS_BOOT? Say like just capture this crazy watchdog program that doesn't run as root so that it can only reboot the system? HeHeHe So I can just trigger a clean reboot wait for journald, auditd, and syslog all to shut down and then do evil things to the machine without having to worry about erasing forensic evidence?

CAP_SYS_BOOT gives you kexec, and kexec with init=/bin/sh lets you do anything. You added that in dc009d92435f99498cbc579ce76bf28e837e2c14 and now the horse is long gone. Don't give CAP_SYS_BOOT to anything you don't trust with full privileges.

On Mon, Apr 13, 2015 at 1:22 PM, Al Viro <[email protected]>

Quoted 25 lines · Greg Kroah-Hartman

Quoted 6 lines

Quoted 2 lines

I remain opposed to this half thought out trash of an ABI for the meta-data.

You don't have to enable the metadata if you don't want to use it, it's
an option :)

OK, _that_ argument needs to be stomped out. It had been used before, and it was a deliberate scam. There is no such thing as optional kernel interface, especially when udev/dbus/systemd crowd is nearby. We'd been through that excuse before; remember how devtmpfs was pushed in as "optional"?

This is a huge red flag. On the level of "I need your account information to transfer $200M you might have inherited from my deceased client".

Just to recap how it went the last time around: Kay kept pushing his piece of code into the tree, claiming that it was optional, that nobody who doesn't like it has to enable it, so what's the problem? OK, in it went. And pretty soon udev (maintained by the same... meticulously honorable person) had stopped working on the kernels that didn't have that enabled.

Quoted 5 lines

We had been there before. To paraphrase another... meticulously honorable person, "if you didn't want something relied upon, why have you put it into the kernel?" Said person is on the record as having no problem whatsoever with adding dependencies to the bottom of userland stack.

It appears that, if kdbus is merged, upstream udev may end up requiring it:

http://lists.freedesktop.org/archives/systemd-devel/2014-May/019657.html

Grumble.

Am Dienstag, 14. April 2015, 18:36:28 schrieb Andy Lutomirski:

Quoted 1 line

On Mon, Apr 13, 2015 at 1:22 PM, Al Viro <[email protected]>

Quoted 34 lines · wrote:

Quoted 25 lines · Greg Kroah-Hartman

Quoted 6 lines

Quoted 2 lines

I remain opposed to this half thought out trash of an ABI for the meta-data.

You don't have to enable the metadata if you don't want to use it, it's
an option :)

OK, _that_ argument needs to be stomped out. It had been used before, and it was a deliberate scam. There is no such thing as optional kernel interface, especially when udev/dbus/systemd crowd is nearby. We'd been through that excuse before; remember how devtmpfs was pushed in as "optional"?

This is a huge red flag. On the level of "I need your account information to transfer $200M you might have inherited from my deceased client".

Just to recap how it went the last time around: Kay kept pushing his piece of code into the tree, claiming that it was optional, that nobody who doesn't like it has to enable it, so what's the problem? OK, in it went. And pretty soon udev (maintained by the same... meticulously honorable person) had stopped working on the kernels that didn't have that enabled.

Quoted 5 lines

We had been there before. To paraphrase another... meticulously honorable person, "if you didn't want something relied upon, why have you put it into the kernel?" Said person is on the record as having no problem whatsoever with adding dependencies to the bottom of userland stack.

It appears that, if kdbus is merged, upstream udev may end up requiring it:

Quoted 3 lines

http://lists.freedesktop.org/archives/systemd-devel/2014-May/019657.html

Grumble.

Honestly, I think that tightly coupling systemd and udev to certain kernel versions in lock step is crap.

That you require some minimum version after some reasonable time, sure. But in lockstep? Seriously.

I certainly do not want a broken system just cause I have to load an older kernel version for some reason.

And yes, I think its good not to force just about any userspace idea into the kernel.

Quoted 5 lines

Am Dienstag, 14. April 2015, 18:36:28 schrieb Andy Lutomirski:

Quoted 1 line

On Mon, Apr 13, 2015 at 1:22 PM, Al Viro <[email protected]>

Quoted 1 line · wrote:

Quoted 1 line

On Mon, Apr 13, 2015 at 09:42:17PM +0200, Greg Kroah-Hartman

Quoted 42 lines

Quoted 42 lines

Quoted 30 lines

Quoted 8 lines

Quoted 3 lines

I remain opposed to this half thought out trash of an ABI for the
meta-data.

You don't have to enable the metadata if you don't want to use it,
it's
an option :)

OK, _that_ argument needs to be stomped out. It had been used before,
and it was a deliberate scam. There is no such thing as optional kernel interface, especially when udev/dbus/systemd crowd is nearby.
We'd been through that excuse before; remember how devtmpfs was pushed in as "optional"?

This is a huge red flag. On the level of "I need your account information to transfer $200M you might have inherited from my deceased client".

Just to recap how it went the last time around: Kay kept pushing his
piece of code into the tree, claiming that it was optional, that nobody who doesn't like it has to enable it, so what's the problem?
OK, in it went. And pretty soon udev (maintained by the same... meticulously honorable person) had stopped working on the kernels that didn't have that enabled.

Quoted 7 lines

We had been there before. To paraphrase another... meticulously honorable person, "if you didn't want something relied upon, why have
you put it into the kernel?" Said person is on the record as having
no problem whatsoever with adding dependencies to the bottom of userland stack.

It appears that, if kdbus is merged, upstream udev may end up requiring
it:

Quoted 7 lines

Quoted 4 lines

http://lists.freedesktop.org/archives/systemd-devel/2014-May/019657.
html

Grumble.

Honestly, I think that tightly coupling systemd and udev to certain kernel versions in lock step is crap.

Where do you see that happening?

Quoted 3 lines

That you require some minimum version after some reasonable time, sure.
But in lockstep? Seriously.

Has that happened in the past? Look at the minimum requirements of systemd/udev today, something like the 3.7 kernel release, many years old.

To make this clear, we expect that systemd and kernels are updated in lockstep. We explicitly do not support really old kernels with really (which means 3.4 right now), but even that should be taken with a grain of salt, as we already made clear that soon after kdbus is merged into the kernel we'll probably make a hard requirement on it from the systemd side.

Quoted 2 lines

I certainly do not want a broken system just cause I have to load an older kernel version for some reason.

No one does. But, work with your distribution if you end up with something like this. Remember, the goal is that you can always run newer kernels on older userspace, as that is something that we kernel developers can enforce. Userspace programs have other requirements / communities, it's up to them to decide what their oldest kernel version they wish to support. Hint, even glibc makes these kinds of requirements, it's nothing new at all here, so why is this even an issue?

Quoted 2 lines

And yes, I think its good not to force just about any userspace idea into the kernel.

Do you have any technical objections to the patch as proposed?

Am Mittwoch, 15. April 2015, 10:32:19 schrieb Greg Kroah-Hartman:

Quoted 6 lines · Martin Steigerwald

Quoted 5 lines

Am Dienstag, 14. April 2015, 18:36:28 schrieb Andy Lutomirski:

Quoted 1 line

On Mon, Apr 13, 2015 at 1:22 PM, Al Viro <[email protected]>

Quoted 1 line · wrote:

Quoted 1 line

On Mon, Apr 13, 2015 at 09:42:17PM +0200, Greg Kroah-Hartman

Quoted 52 lines · wrote:

Quoted 42 lines

Quoted 42 lines

Quoted 30 lines

Quoted 8 lines

Quoted 3 lines

I remain opposed to this half thought out trash of an ABI for the
meta-data.

You don't have to enable the metadata if you don't want to use it,
it's
an option :)

OK, _that_ argument needs to be stomped out. It had been used before,
and it was a deliberate scam. There is no such thing as optional kernel interface, especially when udev/dbus/systemd crowd is nearby.
We'd been through that excuse before; remember how devtmpfs was pushed in as "optional"?

This is a huge red flag. On the level of "I need your account information to transfer $200M you might have inherited from my deceased client".

Just to recap how it went the last time around: Kay kept pushing his
piece of code into the tree, claiming that it was optional, that nobody who doesn't like it has to enable it, so what's the problem?
OK, in it went. And pretty soon udev (maintained by the same... meticulously honorable person) had stopped working on the kernels that didn't have that enabled.

Quoted 7 lines

We had been there before. To paraphrase another... meticulously honorable person, "if you didn't want something relied upon, why have
you put it into the kernel?" Said person is on the record as having
no problem whatsoever with adding dependencies to the bottom of userland stack.

It appears that, if kdbus is merged, upstream udev may end up requiring
it:

Quoted 7 lines

Quoted 4 lines

http://lists.freedesktop.org/archives/systemd-devel/2014-May/019657.
html

Grumble.

Honestly, I think that tightly coupling systemd and udev to certain kernel versions in lock step is crap.

Where do you see that happening?

Quoted 7 lines

Quoted 3 lines

That you require some minimum version after some reasonable time, sure.
But in lockstep? Seriously.

Has that happened in the past? Look at the minimum requirements of systemd/udev today, something like the 3.7 kernel release, many years old.

I refer to the linked mailing list post from Lennart as I quote here:

Quoted 6 lines

To make this clear, we expect that systemd and kernels are updated in lockstep. We explicitly do not support really old kernels with really (which means 3.4 right now), but even that should be taken with a grain of salt, as we already made clear that soon after kdbus is merged into the kernel we'll probably make a hard requirement on it from the systemd side.

Thats plenty clear, isn´t it? As soond as kdbus is merged into kernel, systemd will depend on it, and then… if I need to go back to older kernel, I have to downgrade systemd as well?

Quoted 11 lines

Quoted 2 lines

I certainly do not want a broken system just cause I have to load an older kernel version for some reason.

No one does. But, work with your distribution if you end up with something like this. Remember, the goal is that you can always run newer kernels on older userspace, as that is something that we kernel developers can enforce. Userspace programs have other requirements / communities, it's up to them to decide what their oldest kernel version they wish to support. Hint, even glibc makes these kinds of requirements, it's nothing new at all here, so why is this even an issue?

Its no issue for me that systemd required kernel 3.7. But… what Lennart announces above regarding kdbus reads quite differently.

Quoted 4 lines

Quoted 2 lines

And yes, I think its good not to force just about any userspace idea into the kernel.

Do you have any technical objections to the patch as proposed?

If I had, I would have written it. I explained already that I see that kernel developers have strong technical objections with kdbus. And that I think it is important to acknowledge it, instead of telling them, that the API is required from userspace, userspace people know what they do, and they should just go away with their concerns.

Thats at least how I received quite some of your responses.

Well and I raised an eyebrow on the busname matching rules and the capability stuff. Yet, I didn´t comment on it, cause I didn´t look at it in-depth. I just ask you to take those seriously who did.

Quoted 19 lines

Quoted 7 lines

Quoted 3 lines

And yes, I think its good not to force just about any userspace idea
into the kernel.

Do you have any technical objections to the patch as proposed?

If I had, I would have written it. I explained already that I see that
kernel developers have strong technical objections with kdbus. And that I think it is important to acknowledge it, instead of telling them, that the API is required from userspace, userspace people know what they do, and they should just go away with their concerns.

Quoted 7 lines

Thats at least how I received quite some of your responses.

Well and I raised an eyebrow on the busname matching rules and the capability stuff. Yet, I didn´t comment on it, cause I didn´t look at it in-depth. I just ask you to take those seriously who did.

I take technical comments very seriously, where have I not? If you have technical reasons why the current implementation has problems, please let me know, and I will be glad to address them.

From what I read you basically answered all technical comments like in:

Am Mittwoch, 15. April 2015, 11:02:12 schrieb Greg Kroah-Hartman:

Quoted 27 lines

Quoted 19 lines

Quoted 7 lines

Quoted 3 lines

And yes, I think its good not to force just about any userspace idea
into the kernel.

Do you have any technical objections to the patch as proposed?

If I had, I would have written it. I explained already that I see that
kernel developers have strong technical objections with kdbus. And that I think it is important to acknowledge it, instead of telling them, that the API is required from userspace, userspace people know what they do, and they should just go away with their concerns.

Quoted 7 lines

Thats at least how I received quite some of your responses.

Well and I raised an eyebrow on the busname matching rules and the capability stuff. Yet, I didn´t comment on it, cause I didn´t look at it in-depth. I just ask you to take those seriously who did.

Quoted 5 lines

I take technical comments very seriously, where have I not? If you have technical reasons why the current implementation has problems, please let me know, and I will be glad to address them.

From what I read you basically answered all technical comments like in:

The dbus API is like it is for a very good reason, everyone is using it and everyone agrees. Capabilities are used in userspace for good reason and so on.

But I see, here, not everyone does.

Most of your answers didn´t seem to address the concerns raised of having this in the *kernel*. Especially the security concerns.

Thats what I meant with "And yes, I think its good not to force just about any userspace into the kernel". I think arguing with this is how userspace does it pattern, even if it truly is for a very good reason, is not sufficient as argument for having it in the kernel.

I am just looking at the argumentative pattern here. If other kernel developers complain about how hard it is to review and wrap their mind around the kdbus patches… I am scared at just trying to understand the patches. So no technical complaints from me. I did not nack it nor do I see myself in the position to nack it.

Quoted 21 lines · Al Viro

Quoted 5 lines · Greg Kroah-Hartman

Quoted 2 lines

I remain opposed to this half thought out trash of an ABI for the meta-data.

You don't have to enable the metadata if you don't want to use it, it's an option :)

OK, _that_ argument needs to be stomped out. It had been used before, and it was a deliberate scam. There is no such thing as optional kernel interface, especially when udev/dbus/systemd crowd is nearby. We'd been through that excuse before; remember how devtmpfs was pushed in as "optional"?

This is a huge red flag. On the level of "I need your account information to transfer $200M you might have inherited from my deceased client".

Just to recap how it went the last time around: Kay kept pushing his piece of code into the tree, claiming that it was optional, that nobody who doesn't like it has to enable it, so what's the problem? OK, in it went. And pretty soon udev (maintained by the same... meticulously honorable person) had stopped working on the kernels that didn't have that enabled.

Quoted 4 lines

We had been there before. To paraphrase another... meticulously honorable person, "if you didn't want something relied upon, why have you put it into the kernel?" Said person is on the record as having no problem whatsoever with adding dependencies to the bottom of userland stack.

It appears that, if kdbus is merged, upstream udev may end up requiring it:

http://lists.freedesktop.org/archives/systemd-devel/2014-May/019657.html

Quoted 21 lines · Greg Kroah-Hartman

The following changes since commit 9eccca0843205f87c00404b663188b88eb248051:

Linux 4.0-rc3 (2015-03-08 16:09:09 -0700)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git/ tags/kdbus-4.1-rc1

for you to fetch changes up to 9fb9cd0f4434a23487b6ef3237e733afae90e336:

kdbus: avoid the use of struct timespec (2015-04-10 14:34:53 +0200)

----------------------------------------------------------------
kdbus for 4.1-rc1

Here's the kdbus pull request for 4.1-rc1.

It's been under development for many years now, and been in linux-next for many months, and has undergone loads of testing a review and even a few good arguments. It comes with full documentation and tests.

Quoted 7 lines

There has been a few complaints about the code, notably from people who don't like the use of metadata in the bus messages. That is actually one of the main features here, as we can get this data in a secure and reliable way, and it's something that userspace requires today. So while it does look "odd" to people who are not familiar with dbus, this is something that finally fixes a number of almost unfixable races in the current dbus implementations.

And the code that transfers the meta-data is wrong.

Quoted 31 lines · [email protected] (Eric W. Biederman) writes:

Quoted 21 lines · Greg Kroah-Hartman

The following changes since commit 9eccca0843205f87c00404b663188b88eb248051:

Linux 4.0-rc3 (2015-03-08 16:09:09 -0700)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git/ tags/kdbus-4.1-rc1

for you to fetch changes up to 9fb9cd0f4434a23487b6ef3237e733afae90e336:

kdbus: avoid the use of struct timespec (2015-04-10 14:34:53 +0200)

----------------------------------------------------------------
kdbus for 4.1-rc1

Here's the kdbus pull request for 4.1-rc1.

It's been under development for many years now, and been in linux-next for many months, and has undergone loads of testing a review and even a few good arguments. It comes with full documentation and tests.

Quoted 7 lines

There has been a few complaints about the code, notably from people who don't like the use of metadata in the bus messages. That is actually one of the main features here, as we can get this data in a secure and reliable way, and it's something that userspace requires today. So while it does look "odd" to people who are not familiar with dbus, this is something that finally fixes a number of almost unfixable races in the current dbus implementations.

Quoted 1 line

And the code that transfers the meta-data is wrong.

In fact it is worse than I thought.

With an userspace application able to give meaning to any of the bits of meta-data that are passed (capabilities, cgroup, security labels, etc) that in the fullness of time dropping in them will grant you more permissions somewhere.

Which means that it becomes impossible to change anything. Impossible to jail anything. It in fact becomes impossible to do anything right.

Which means the ultimate result of the direction kdbus is going is a world where nothing can be done without introducing a security issue or breaking userspace.

So as far as I can tell kdbus has a fundamental design flaw.

My apologies for being the bearer of bad news.

Quoted 31 lines · [email protected] (Eric W. Biederman) writes:

Quoted 28 lines · Greg Kroah-Hartman

The following changes since commit 9eccca0843205f87c00404b663188b88eb248051:

Linux 4.0-rc3 (2015-03-08 16:09:09 -0700)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git/ tags/kdbus-4.1-rc1

for you to fetch changes up to 9fb9cd0f4434a23487b6ef3237e733afae90e336:

kdbus: avoid the use of struct timespec (2015-04-10 14:34:53 +0200)

----------------------------------------------------------------
kdbus for 4.1-rc1

Here's the kdbus pull request for 4.1-rc1.

It's been under development for many years now, and been in linux-next for many months, and has undergone loads of testing a review and even a few good arguments. It comes with full documentation and tests.

There has been a few complaints about the code, notably from people who don't like the use of metadata in the bus messages. That is actually one of the main features here, as we can get this data in a secure and reliable way, and it's something that userspace requires today. So while it does look "odd" to people who are not familiar with dbus, this is something that finally fixes a number of almost unfixable races in the current dbus implementations.

Quoted 1 line

And the code that transfers the meta-data is wrong.

In fact it is worse than I thought.

Quoted 5 lines · Eric W. Biederman

Quoted 1 line · [email protected] (Eric W. Biederman) writes:

And the code that transfers the meta-data is wrong.

In fact it is worse than I thought.

Please see the email response I just wrote to Andy about this, it should address these misconceptions.

- The userspace interface for capability bits has a version number that determines the quantity and the meaning of the bits, kdbus does not pass that number to userspace.

- There is a well defined translation of capability bits between user namespaces kdbus does not perform that translation.

- Access to the capability bits is guarded with PTRACE_MAY_READ kdbus does not honor that and thus leaks information.

- Usage of the capability bits by userspace is a layering violation.

- The layering violation results in a privilege escalation (by definiton) whenever userspace uses the presence of the capability bits to allow anything.

- Another kind of privilege escalation happens when userspace makes a decision based on the absense of in kernel capability bits.

Quoted 2 lines · Eric W. Biederman

- Access to the capability bits is guarded with PTRACE_MAY_READ kdbus does not honor that and thus leaks information.

Now, this is likely not a real problem.

Yes, when you try to read other processes capabilities, you need PTRACE_MAY_READ to see them. HOWEVER, that's not really what a kdbus message would do - it doesn't "read somebody elses capabilities". When you do a kdbus write, you export your *own* capabilities. If you don't want others to know what privileges you have, then you shouldn't be using kdbus.

It's like saying "you need PTRACE_MAY_READ" to be able to read the process image of another process. True. But if that other process does a "write()" system call, then the written data will contain the data from that process. That's how write works.

If kdbus were a general purpose IPC tool

and if the libraries would
expose nice knobs like "set this flag if and only if you want to assert CAP_WHATEVER to the server", then maybe this would be okay.

Quoted 2 lines · Andy Lutomirski

If kdbus were a general purpose IPC tool

.. but it's not ..

Quoted 3 lines

and if the libraries would
expose nice knobs like "set this flag if and only if you want to assert CAP_WHATEVER to the server", then maybe this would be okay.

I really don't agree.

The whole notion that you should be able to be anonymous when communicating is *wrong*.

Now, when you talk across machines using TCP, there's no identity that you can trust, so anonymity is kind of enforced. But locally, if you want to connect with somebody else, I actually think that tryin gto be anonymous is just stupid - and more than that - plain wrong.

Yeah, if you do a pipe, that's one thing. You don't "connect" to somebody else with a pipe, you just create both end points. So there is little point in having identifying information for pipes. But when you connect to a service, it just *makes sense* for the other end to know about you. They should know your user ID, they should know your identity (pid or whatever), and they should know your capabilities.

If you don't want that, then you use some anonymizing service, and it because *your* problem. But a server that gets connected by different people should know who it gets connected by.

Unix domain sockets simply got this wrong.

I don't think thi sis the problem of kdbus. There may be *other&* problems, but I think it's very reasonable to just have as a basic *requirement* that when you get connected, you get to know who connects you, and what their rights are. Because it really *is* something fundamental, and something important to know. Is it some random nobody, or is it a system service?

If I was a service writer, that would be *the* most basic requirement I would have. No ifs, buts or maybes about it.

Where do you draw the line?

Quoted 2 lines · Andy Lutomirski

If kdbus were a general purpose IPC tool

.. but it's not ..

Quoted 4 lines · Linus Torvalds

Quoted 2 lines · Andy Lutomirski

If kdbus were a general purpose IPC tool

Quoted 2 lines

.. but it's not ..

Except, IIRC, that was one of the stated design goals in the original patch set. I'm pretty sure that i remember a rather verbose exposition that pretty much could be summarized as "Linux has no general purpose IPC in the kernel, this fixes that"

Quoted 5 lines · Austin S Hemmelgarn

Quoted 4 lines · Linus Torvalds

Quoted 2 lines · Andy Lutomirski

If kdbus were a general purpose IPC tool

Quoted 7 lines

Quoted 2 lines

.. but it's not ..

Except, IIRC, that was one of the stated design goals in the original patch set. I'm pretty sure that i remember a rather verbose exposition that pretty much could be summarized as "Linux has no general purpose IPC in the kernel, this fixes that"

Did I say that somewhere? Here's what the patchset has always started with every time I have posted it for review, starting back last year in October:

kdbus is a kernel-level IPC implementation that aims for resemblance to the the protocol layer with the existing userspace D-Bus daemon while enabling some features that couldn't be implemented before in userspace.

2+ years ago, I had the dream that maybe we could make kdbus into the "general purpose IPC layer for the kernel", but in working through all of the issues, and the requirements of the userspace users and protocols, it just really didn't work out that way, sorry.

Except, IIRC, that was one of the stated design goals in the original patch set. I'm pretty sure that i remember a rather verbose exposition that pretty much could be summarized as "Linux has no general purpose IPC in the kernel, this fixes that"

If we say it isn't "general purpose" we could mean more than one thing -

- it's a complete system / batteries-included, with a defined
protocol, vs. a "make your own protocol" kit
- it isn't especially appropriate as a cross-machine protocol,
whether you mean within a cluster or across the internet
- it isn't portable in a very useful way (it kind of runs on
windows/mac but isn't the native way of doing things there)

On the other hand, it is "general purpose" in the sense that so many apps and services are using it for so many purposes already (i.e. it isn't tied to a particular kind of app or service).

Just out of curiosity, would you like to change something in dbus design, if you didn't have to worry about ABI breaks and the like?

* the byte order marker in every message is silly

Conditional byte order is worse than silly - it's terminally stupid.

Conditional byte order is worse than silly - it's terminally stupid.

In other words, think networking, which statically just decided to use big-endian. Sure, that was the wrong choice in the end, but even

Quoted 4 lines · Linus Torvalds

Conditional byte order is worse than silly - it's terminally stupid.

In other words, think networking, which statically just decided to use big-endian. Sure, that was the wrong choice in the end, but even

Why was that wrong? Any pointers to further details?

http://www.barrgroup.com/Embedded-Systems/How-To/Big-Endian-Little-Endian
"Serious run-time performance penalties occur when using TCP/IP on a little endian processor."

Quoted 3 lines · Michele Curti

http://www.barrgroup.com/Embedded-Systems/How-To/Big-Endian-Little-Endian
"Serious run-time performance penalties occur when using TCP/IP on a little endian processor."

This URL lacks the numbers to proof such claim.

Quoted 4 lines

Conditional byte order is worse than silly - it's terminally stupid.

In other words, think networking, which statically just decided to use big-endian. Sure, that was the wrong choice in the end, but even

Why was that wrong? Any pointers to further details?

Becase the architecture that is running on overwhelming majority of today's world computers is little-endian, and therefore has to convert all the time.

Quoted 4 lines · Linus Torvalds

Conditional byte order is worse than silly - it's terminally stupid.

In other words, think networking, which statically just decided to use big-endian. Sure, that was the wrong choice in the end, but even

Why was that wrong? Any pointers to further details?

So even today, by all means make your protocols or disk images use big-endian byte formats. But do it unconditionally. Don't make the mistake of encoding the byte order as part of the data, and then dynamically switching things (or not) around.

Quoted 4 lines · Michele Curti

Just out of curiosity, would you like to change something in dbus design, if you didn't have to worry about ABI breaks and the like?

Good question. I can't remember any big-picture things, I'm sure the current maintainers and users have a longer list. :-) There are a variety of little small things, some examples I can immediately think of:

* the ad hoc authentication protocol is sort of ugly
* the byte order marker in every message is silly
* protocol version in every message is useless
* Ryan Lortie's nice fixes in GVariant, which I think kdbus adopts (
https://people.gnome.org/~ryanl/gvariant-serialisation.pdf ), for the most part these are 'cleanups' but nullable types ("maybe" types for Haskell fans) are a notable semantic addition
* specify how it works on Windows, the Windows port last I checked
(years ago) didn't do things in a Windows-sensible way
* specify what happens when resource limits are reached
* wouldn't use XML for introspection data these days
http://dbus.freedesktop.org/doc/dbus-specification.html#introspection-format

Quoted 2 lines

- Access to the capability bits is guarded with PTRACE_MAY_READ kdbus does not honor that and thus leaks information.

Now, this is likely not a real problem.

Yes, when you try to read other processes capabilities, you need PTRACE_MAY_READ to see them. HOWEVER, that's not really what a kdbus message would do - it doesn't "read somebody elses capabilities". When you do a kdbus write, you export your *own* capabilities. If you don't want others to know what privileges you have, then you shouldn't be using kdbus.

So I think that one is a real and serious bug. But the other complaints seem to be off the mark. It seems quite reasonable to me to say that a recipient should be able to distinguish between *root* sending it a dbus message to take down the system, and some random luser doing the same.

Quoted 2 lines · Eric W. Biederman

- There is a well defined translation of capability bits between user namespaces kdbus does not perform that translation.

Now this looks like a big oversight, and serious.

If you have a capability inside a lower namespace, and can fool the other end of a kdbus connection into thinking that you have that capability globally, then that sounds like a very obvious and bad security issue. This needs fixing. That said, it's likely a fairly simple fix.

Quoted 2 lines

- Access to the capability bits is guarded with PTRACE_MAY_READ kdbus does not honor that and thus leaks information.

Now, this is likely not a real problem.

Yes, when you try to read other processes capabilities, you need PTRACE_MAY_READ to see them. HOWEVER, that's not really what a kdbus message would do - it doesn't "read somebody elses capabilities". When you do a kdbus write, you export your *own* capabilities. If you don't want others to know what privileges you have, then you shouldn't be using kdbus.

And the code that transfers the meta-data is wrong.

It is generally not something that userspace requires today, certainly userspace is not using it.

You are exporting a weird set of information in a unique way that makes it race free enough to make ``security'' decisions upon but the data in general is not appropriate to make those decisions.

I remain opposed to this half thought out trash of an ABI for the meta-data.

Just because something happens to be exported in a DEBUG api today does not make it appropriate for userspace to run around making security decisions with that information.

Nacked-by "Eric W. Biederman" <[email protected]>

I think it is premature to be merging kdbus. You have fuddamental issues that can not be fixed once the ABI is frozen.

The semantics of the meta-data you export are extremely poorly defined.

Quoted 16 lines · Eric W. Biederman

And the code that transfers the meta-data is wrong.

It is generally not something that userspace requires today, certainly userspace is not using it.

You are exporting a weird set of information in a unique way that makes it race free enough to make ``security'' decisions upon but the data in general is not appropriate to make those decisions.

I remain opposed to this half thought out trash of an ABI for the meta-data.

Just because something happens to be exported in a DEBUG api today does not make it appropriate for userspace to run around making security decisions with that information.

Quoted 6 lines

Nacked-by "Eric W. Biederman" <[email protected]>

I think it is premature to be merging kdbus. You have fuddamental issues that can not be fixed once the ABI is frozen.

The semantics of the meta-data you export are extremely poorly defined.

Not only that - it looks like a serious amount of work on each sent packet. So I did some staring, correct me if I missed something:

kdbus_cmd_send - KDBUS_CMD_SEND, ioctl cmd, copy stuff from userspace
|-> kdbus_kmsg_new_from_cmd(), kmalloc+memset + prepare a *lot* of stuff like:

    |-> m->proc_meta = kdbus_meta_proc_new();
	m->conn_meta = kdbus_meta_conn_new();
	...
    |-> kdbus_bus_broadcast(conn->ep->bus, conn, kmsg); let's look at the broadcast mode
        |-> hash_for_each(bus->conn_hash, i, conn_dst, hentry) { 	iterate over hash buckets, O(256)

	    |-> kdbus_meta_proc_collect(kmsg->proc_meta, attach_flags);	collect a *lot* of stuff from current etc
	    |-> kdbus_meta_conn_collect(kmsg->conn_meta, kmsg, conn_src, attach_flags); collect more stuff

and this happens on *every* send. A *lot* of work.

Now multiply that by the amount of messages this thing is going to send per second. It piles up. So you have the overhead right then and there in the design without even being able to fix it. Or at least pretty damn hard to fix.

So unless I'm missing something, this right there is a design problem.

Why can't this messaging be done with a nifty O(1) scheme like sending parties issuing auth tokens and whatever and the kernel doing the arbitration and distribution of those tokens?

That gets you sandboxing, dropping privileges and whatever else fancy containers people wanna do for free. Token recipient has the token - that's all that counts.

I don't know what O(256) means here, O notation usually is used to show the complexity of a function, so this really is almost always the same amount of time, based on using the hash function.

Yes, these looks like a lot of stuff but it's still really fast.

And we need it.

Hm, this seems to be to be O(1), pretty constant, we do the same amount of work all the time.

I don't understand what a token provides that is different from what is happening here, please explain. How can that be faster than what we do today?

Quoted 3 lines · Greg Kroah-Hartman

I don't know what O(256) means here, O notation usually is used to show the complexity of a function, so this really is almost always the same amount of time, based on using the hash function.

This is iterating over 256 hash buckets. So O(n) complexity. Better?

Hi

Quoted 6 lines · Borislav Petkov

Quoted 3 lines · Greg Kroah-Hartman

I don't know what O(256) means here, O notation usually is used to show the complexity of a function, so this really is almost always the same amount of time, based on using the hash function.

This is iterating over 256 hash buckets. So O(n) complexity. Better?

No it's not. O(256) equals O(1).

Quoted 1 line

No it's not. O(256) equals O(1).

Yeah, that's absolutely correct.
I think Boris wanted to say that iterating over all hash buckets can be costly.

No it's not. O(256) equals O(1).

Quoted 1 line · David Herrmann

No it's not. O(256) equals O(1).

Ok, you're right. Maybe O() was not the right thing to use when trying to point out that iterating over 256 hash buckets and then following the chain in each bucket per packet broadcast looks like a lot.

Quoted 7 lines · Borislav Petkov

Quoted 1 line · David Herrmann

No it's not. O(256) equals O(1).

Ok, you're right. Maybe O() was not the right thing to use when trying to point out that iterating over 256 hash buckets and then following the chain in each bucket per packet broadcast looks like a lot.

Heh. I guess you could call it an "expensive O(1)". While big-O notation is useful for describing algorithm scalability with respect to input size, it falls flat on its face when trying to articulate impact in measurable units.

Quoted 9 lines · Steven Noonan

Quoted 7 lines · Borislav Petkov

Quoted 1 line · David Herrmann

No it's not. O(256) equals O(1).

Ok, you're right. Maybe O() was not the right thing to use when trying to point out that iterating over 256 hash buckets and then following the chain in each bucket per packet broadcast looks like a lot.

Quoted 4 lines

Heh. I guess you could call it an "expensive O(1)". While big-O notation is useful for describing algorithm scalability with respect to input size, it falls flat on its face when trying to articulate impact in measurable units.

Right, so in thinking about this more today, on a fresh head, it still is O(n) because we do broadcast the packet to n recipients - the hash_for_each() thing iterates over 256 hash buckets and also follows the linked list chain in each bucket. Its length is depending on how many connections are in the bucket, i.e. recipients. And I'd guess that number changes dynamically so probably linear.

And then there's the collection of, let's call it metadata of questionable use, *per* packet which is pretty expensive in my book. It becomes even more expensive if it is completely useless as in, the receiving side doesn't need it all.

Now, one might argue that you have to do O(n) work when broadcasting to n recipients anyway and you can't get that cheaper but maybe the design is not optimal. Maybe it could be made to not broadcast at all, or broadcast to a subset of recipients, only those which are actually interested in the broadcast.

Sure, for broadcasts, we have to walk the list of peers connected to the bus and see which one is interested in a particular message. We do that

by looking at the match rules of each of them, which are based on well-known names, IDs, notification types or bloom filters. The policy logic limits this further, as receivers of a broadcast must have TALK access to the sender.

If these rules let a message pass, all the metadata that the receiving peer asked for (by setting a flag at connect time) is collected, unless it has been collected already for some other peer for the same message. In other words, in worst case, we collect all the metadata items exactly once per message.

If none of the connections with permissive match/policy rules for a message is interested in any metadata items, nothing will be collected at all.

If the receiving side doesn't need it, it shouldn't opt-in for that piece of information.

The metadata logic is really only there so receiving peers are directly supplied with information that they would otherwise look up themselves from /proc or something. Also, we collect metadata at send time and for every message intentionally, so that it reflects the state of the sender at the time of sending. This way, the information is not subject to races of asynchronous lookups.

Quoted 5 lines

Now, one might argue that you have to do O(n) work when broadcasting to n recipients anyway and you can't get that cheaper but maybe the design is not optimal. Maybe it could be made to not broadcast at all, or broadcast to a subset of recipients, only those which are actually interested in the broadcast.

That's exactly what happens :) There are some more details on this in kdbus.match(7).

Quoted 2 lines · Daniel Mack

Sure, for broadcasts, we have to walk the list of peers connected to the bus and see which one is interested in a particular message. We do that

And this "... we have to walk the list ..." right there raises the alarm. Can this walking of elements where you know they wouldn't match be avoided?

Quoted 4 lines

by looking at the match rules of each of them, which are based on well-known names, IDs, notification types or bloom filters. The policy logic limits this further, as receivers of a broadcast must have TALK access to the sender.

So it sounds to me like there are characteristics which can already prepare lists of recipients interested in some sort of message. So would it be possible for recipients to "register" for such messages and the sending side would simply iterate a list of solely interested recipients?

This will definitely save you the iteration over all n connections and would make the metadata collection probably not needed (or at least a subset of it) because recipients will have to establish eligibility for receiving a certain message at register time and once they're on the list, you implicitly know why they're there.

Quoted 3 lines

If none of the connections with permissive match/policy rules for a message is interested in any metadata items, nothing will be collected at all.

But we still iterate through there and look at the arg @what and ->collected. And this is useless work which can be avoided IMHO.

Quoted 8 lines

Quoted 5 lines

Now, one might argue that you have to do O(n) work when broadcasting to n recipients anyway and you can't get that cheaper but maybe the design is not optimal. Maybe it could be made to not broadcast at all, or broadcast to a subset of recipients, only those which are actually interested in the broadcast.

That's exactly what happens :) There are some more details on this in kdbus.match(7).

But this is not for KDBUS_DST_ID_BROADCAST types, right?

Quoted 2 lines

Hm, this seems to be to be O(1), pretty constant, we do the same amount of work all the time.

The same *pile* of unnecessary and needless work. You go and collect *all* that data on *every* packet send?!

Quoted 5 lines · Borislav Petkov

Quoted 2 lines

Hm, this seems to be to be O(1), pretty constant, we do the same amount of work all the time.

The same *pile* of unnecessary and needless work. You go and collect *all* that data on *every* packet send?!

No, not at all, the metadata is cached, we only collect that for the first message sent, if we didn't know it already, or we do it on the "open" of the connection, depending on what we are gathering metadata for.

The mc->collected test right before collecting the specific metadata is that "cached or not" test.

Quoted 8 lines · Borislav Petkov

Quoted 6 lines

Quoted 3 lines

Hm, this seems to be to be O(1), pretty constant, we do the same amount
of work all the time.

The same *pile* of unnecessary and needless work. You go and collect *all* that data on *every* packet send?!

Quoted 8 lines

No, not at all, the metadata is cached, we only collect that for the first message sent, if we didn't know it already, or we do it on the "open" of the connection, depending on what we are gathering metadata for.

The mc->collected test right before collecting the specific metadata is
that "cached or not" test.

Oh wait, no, there are some send-time metadata that is collected for every message, see Linus's email for more details about that. Maybe this can be changed to cache things even more than we currently do.

it's early, shouldn't write emails before coffee...

David had some flamegraphs floating around that showed where all the time on transmit / receive was being spent, and I don't think that the metadata area was all that relevant, but I can't find them anymore to say for sure. There are other areas that can be sped up on the send path, but perf data is the best way to verify this.

Quoted 7 lines · Greg Kroah-Hartman

Quoted 5 lines · Borislav Petkov

Quoted 2 lines

Hm, this seems to be to be O(1), pretty constant, we do the same amount of work all the time.

The same *pile* of unnecessary and needless work. You go and collect *all* that data on *every* packet send?!

Quoted 7 lines

No, not at all, the metadata is cached, we only collect that for the first message sent, if we didn't know it already, or we do it on the "open" of the connection, depending on what we are gathering metadata for.

The mc->collected test right before collecting the specific metadata is that "cached or not" test.

Oh wait, no, there are some send-time metadata that is collected for every message, see Linus's email for more details about that. Maybe this can be changed to cache things even more than we currently do.

it's early, shouldn't write emails before coffee...

David had some flamegraphs floating around that showed where all the time on transmit / receive was being spent, and I don't think that the metadata area was all that relevant, but I can't find them anymore to say for sure. There are other areas that can be sped up on the send path, but perf data is the best way to verify this.

The following changes since commit 9eccca0843205f87c00404b663188b88eb248051:

Linux 4.0-rc3 (2015-03-08 16:09:09 -0700)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git/ tags/kdbus-4.1-rc1

for you to fetch changes up to 9fb9cd0f4434a23487b6ef3237e733afae90e336:

kdbus: avoid the use of struct timespec (2015-04-10 14:34:53 +0200)

----------------------------------------------------------------
kdbus for 4.1-rc1

Here's the kdbus pull request for 4.1-rc1.

It's been under development for many years now, and been in linux-next for many months, and has undergone loads of testing a review and even a few good arguments. It comes with full documentation and tests.

There has been a few complaints about the code, notably from people who don't like the use of metadata in the bus messages. That is actually one of the main features here, as we can get this data in a secure and reliable way, and it's something that userspace requires today. So while it does look "odd" to people who are not familiar with dbus, this is something that finally fixes a number of almost unfixable races in the current dbus implementations.

Quoted 21 lines · Greg Kroah-Hartman

The following changes since commit 9eccca0843205f87c00404b663188b88eb248051:

Linux 4.0-rc3 (2015-03-08 16:09:09 -0700)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git/ tags/kdbus-4.1-rc1

for you to fetch changes up to 9fb9cd0f4434a23487b6ef3237e733afae90e336:

kdbus: avoid the use of struct timespec (2015-04-10 14:34:53 +0200)

----------------------------------------------------------------
kdbus for 4.1-rc1

Here's the kdbus pull request for 4.1-rc1.

It's been under development for many years now, and been in linux-next for many months, and has undergone loads of testing a review and even a few good arguments. It comes with full documentation and tests.

Quoted 7 lines

There has been a few complaints about the code, notably from people who don't like the use of metadata in the bus messages. That is actually one of the main features here, as we can get this data in a secure and reliable way, and it's something that userspace requires today. So while it does look "odd" to people who are not familiar with dbus, this is something that finally fixes a number of almost unfixable races in the current dbus implementations.

While I generally like the concept of having a better in-kernel IPC mechanism, after some consideration I don't think this belongs in the kernel in its current form. Here's why.

First, the naming is counterintuitive. There are "endpoints", but you don't send messages to endpoints. In fact, an basic kdbus setup will have exactly one endpoint AFAICT. Wtf? This makes talking about it awkward.

A lot of the design seems to be to violate the concept of "mechanism, not policy". Kdbus is very much a port of userspace dbus to the kernel, and it appears to be a port designed to preserve some questionable design decisions instead of learning from them.

For example, kdbus sticks a whole policy database in the kernel, but that policy database (AFAICT -- holy crap it's overcomplicated) is *not* a simple set of rules like "if A then allow B". Instead it has really weird dependencies not on what name you're sending to but on what *other* names the thing you're sending to has. Sorry, but this way lies (a) the inability for a large set of developers to understand what's going on and (b) security bugs. Also, the result probably can't be reused as part of a non-legacy-filled sensible design

Kdbus claims to be very fast. Unfortunately, requests for a broad set of benchmarks have mostly been ignored, my attempts to benchmark it (admittedly I didn't try that hard) were several times worse than published figures, and, most tellingly, *no one* has claimed that kdbus is faster than AF_UNIX. In fact, everyone seems to acknowledge that kdbus is several times slower than AF_UNIX.

The metadata thing is problematic. It seems to be intended to serve two purposes: data gathering for logging and authentication. Unfortunately, it has issues. There are no fewer than *three* metadata capture points: creation of a bus, connection to a bus, and sending of a message. The kdbus authors like to point out that these are all optional, but IMO that's bunk. Someone will write a userspace library that rejects messages from people who don't enable all of them, then then we're screwed.

Why are we screwed? Because any kdbus client *won't know which metadata matters*. That means that we automatically have the worst of all worlds, not the best. Also, the bus creation metadata is completely worthless for anything other than logging, but someone will use it for something other than logging, at which point it's vulnerable to a DoS. No one has explained to my satisfaction why this isn't a problem.

Also, the metadata code captures things that are, in my book completely unacceptable, such as cmdline and (!) capabilities. I bet that the cmdline capture is extra special fscked up when cgroups and such are in play because *it reads from the sender's VM*. IOW it's insecure and pointless. (OK, it has a point: logging. But I really don't think that belongs in the kernel.)

Quoted 31 lines · Andy Lutomirski

Quoted 21 lines · Greg Kroah-Hartman

The following changes since commit 9eccca0843205f87c00404b663188b88eb248051:

Linux 4.0-rc3 (2015-03-08 16:09:09 -0700)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git/ tags/kdbus-4.1-rc1

for you to fetch changes up to 9fb9cd0f4434a23487b6ef3237e733afae90e336:

kdbus: avoid the use of struct timespec (2015-04-10 14:34:53 +0200)

----------------------------------------------------------------
kdbus for 4.1-rc1

Here's the kdbus pull request for 4.1-rc1.

It's been under development for many years now, and been in linux-next for many months, and has undergone loads of testing a review and even a few good arguments. It comes with full documentation and tests.

Quoted 7 lines

There has been a few complaints about the code, notably from people who don't like the use of metadata in the bus messages. That is actually one of the main features here, as we can get this data in a secure and reliable way, and it's something that userspace requires today. So while it does look "odd" to people who are not familiar with dbus, this is something that finally fixes a number of almost unfixable races in the current dbus implementations.

Quoted 8 lines

While I generally like the concept of having a better in-kernel IPC mechanism, after some consideration I don't think this belongs in the kernel in its current form. Here's why.

First, the naming is counterintuitive. There are "endpoints", but you don't send messages to endpoints. In fact, an basic kdbus setup will have exactly one endpoint AFAICT. Wtf? This makes talking about it awkward.

Did you read the documentation? We've been over this before, and it should all be addressed in the documentation based on this coming up.

Quoted 5 lines

A lot of the design seems to be to violate the concept of "mechanism, not policy". Kdbus is very much a port of userspace dbus to the kernel, and it appears to be a port designed to preserve some questionable design decisions instead of learning from them.

Quoted 8 lines

For example, kdbus sticks a whole policy database in the kernel, but that policy database (AFAICT -- holy crap it's overcomplicated) is *not* a simple set of rules like "if A then allow B". Instead it has really weird dependencies not on what name you're sending to but on what *other* names the thing you're sending to has. Sorry, but this way lies (a) the inability for a large set of developers to understand what's going on and (b) security bugs. Also, the result probably can't be reused as part of a non-legacy-filled sensible design

What policy database? Matching messages to subscribers? That's the same type of "database" that other ipc subsystems need/want, there's nothing radical here.

And the benchmarks and source were posted by David previously, with full details, this is the first time I've heard you could not reproduce them using that code.

Quoted 9 lines

The metadata thing is problematic. It seems to be intended to serve two purposes: data gathering for logging and authentication. Unfortunately, it has issues. There are no fewer than *three* metadata capture points: creation of a bus, connection to a bus, and sending of a message. The kdbus authors like to point out that these are all optional, but IMO that's bunk. Someone will write a userspace library that rejects messages from people who don't enable all of them, then then we're screwed.

Remember, you asked for it to be optional, it wasn't in the beginning :)

So let's make it not optional, great. And the capture points are in different places as it is different data and entry points.

Quoted 8 lines

Why are we screwed? Because any kdbus client *won't know which metadata matters*. That means that we automatically have the worst of all worlds, not the best. Also, the bus creation metadata is completely worthless for anything other than logging, but someone will use it for something other than logging, at which point it's vulnerable to a DoS. No one has explained to my satisfaction why this isn't a problem.

I don't think the creation data is worthless, I'm pretty sure the SELinux people are using it to validate things, but I could be wrong. Others on the cc: know more about that than I do and can provide details.

Quoted 7 lines

Also, the metadata code captures things that are, in my book completely unacceptable, such as cmdline and (!) capabilities. I bet that the cmdline capture is extra special fscked up when cgroups and such are in play because *it reads from the sender's VM*. IOW it's insecure and pointless. (OK, it has a point: logging. But I really don't think that belongs in the kernel.)

The sender's vm is what is wanted here. And cmdline is something that userspace gets today, and does things with, as does SELinux, and auditing. Same for capabilities, it's not insecure and pointless, it's the same thing that is provided to userspace, and userspace makes decisions on today, independent of kdbus/dbus.

Quoted 58 lines · Greg Kroah-Hartman

Quoted 31 lines · Andy Lutomirski

Quoted 21 lines · Greg Kroah-Hartman

The following changes since commit 9eccca0843205f87c00404b663188b88eb248051:

Linux 4.0-rc3 (2015-03-08 16:09:09 -0700)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git/ tags/kdbus-4.1-rc1

for you to fetch changes up to 9fb9cd0f4434a23487b6ef3237e733afae90e336:

kdbus: avoid the use of struct timespec (2015-04-10 14:34:53 +0200)

----------------------------------------------------------------
kdbus for 4.1-rc1

Here's the kdbus pull request for 4.1-rc1.

It's been under development for many years now, and been in linux-next for many months, and has undergone loads of testing a review and even a few good arguments. It comes with full documentation and tests.

Quoted 7 lines

There has been a few complaints about the code, notably from people who don't like the use of metadata in the bus messages. That is actually one of the main features here, as we can get this data in a secure and reliable way, and it's something that userspace requires today. So while it does look "odd" to people who are not familiar with dbus, this is something that finally fixes a number of almost unfixable races in the current dbus implementations.

Quoted 8 lines

While I generally like the concept of having a better in-kernel IPC mechanism, after some consideration I don't think this belongs in the kernel in its current form. Here's why.

First, the naming is counterintuitive. There are "endpoints", but you don't send messages to endpoints. In fact, an basic kdbus setup will have exactly one endpoint AFAICT. Wtf? This makes talking about it awkward.

Did you read the documentation? We've been over this before, and it should all be addressed in the documentation based on this coming up.

Quoted 5 lines

A lot of the design seems to be to violate the concept of "mechanism, not policy". Kdbus is very much a port of userspace dbus to the kernel, and it appears to be a port designed to preserve some questionable design decisions instead of learning from them.

Quoted 8 lines

For example, kdbus sticks a whole policy database in the kernel, but that policy database (AFAICT -- holy crap it's overcomplicated) is *not* a simple set of rules like "if A then allow B". Instead it has really weird dependencies not on what name you're sending to but on what *other* names the thing you're sending to has. Sorry, but this way lies (a) the inability for a large set of developers to understand what's going on and (b) security bugs. Also, the result probably can't be reused as part of a non-legacy-filled sensible design

Quoted 3 lines

What policy database? Matching messages to subscribers? That's the same type of "database" that other ipc subsystems need/want, there's nothing radical here.

Let me quote from the latest version of the kdbus docs:

Note that TALK access is checked against all names of a connection. For example, if a connection owns both <constant>'org.foo.bar'</constant> and <constant>'org.blah.baz'</constant>, and the policy database allows <constant>'org.blah.baz'</constant> to be talked to by WORLD, then this permission is also granted to <constant>'org.foo.bar'</constant>. That might sound illogical, but after all, we allow messages to be directed to either the ID or a well-known name, and policy is applied to the connection, not the name. In other words, the effective TALK policy for a connection is the most permissive of all names the connection owns.

In my humble opinion, this paragraph speaks for itself. The design is bad, full stop.

Quoted 3 lines

And the benchmarks and source were posted by David previously, with full details, this is the first time I've heard you could not reproduce them using that code.

No it's not. But I got bored and didn't try again.

Quoted 2 lines

Quoted 2 lines

The metadata thing is problematic. It seems to be intended to serve two purposes: data gathering for logging and authentication.

Quoted 11 lines

Quoted 6 lines

Unfortunately, it has issues. There are no fewer than *three* metadata capture points: creation of a bus, connection to a bus, and sending of a message. The kdbus authors like to point out that these are all optional, but IMO that's bunk. Someone will write a userspace library that rejects messages from people who don't enable all of them, then then we're screwed.

Remember, you asked for it to be optional, it wasn't in the beginning :)

So let's make it not optional, great. And the capture points are in different places as it is different data and entry points.

Then I'll have to find a way to embolden my NACK further. My point is that capturing garbage like cmdline and capabilities (again, that latter part is completely unacceptable under any circumstances whatsoever) on behalf of *all* senders is a disaster. If it's optional, then I can at least hope that userspace will honor the optionality and let everything turn it off. If it's mandatory, then kdbus is just unsafe to use to send messages to untrusted parties.

Quoted 7 lines

Quoted 7 lines

Why are we screwed? Because any kdbus client *won't know which metadata matters*. That means that we automatically have the worst of all worlds, not the best. Also, the bus creation metadata is completely worthless for anything other than logging, but someone will use it for something other than logging, at which point it's vulnerable to a DoS. No one has explained to my satisfaction why this isn't a problem.

Quoted 7 lines

Quoted 6 lines

Also, the metadata code captures things that are, in my book completely unacceptable, such as cmdline and (!) capabilities. I bet that the cmdline capture is extra special fscked up when cgroups and such are in play because *it reads from the sender's VM*. IOW it's insecure and pointless. (OK, it has a point: logging. But I really don't think that belongs in the kernel.)

Quoted 5 lines

The sender's vm is what is wanted here. And cmdline is something that userspace gets today, and does things with, as does SELinux, and auditing. Same for capabilities, it's not insecure and pointless, it's the same thing that is provided to userspace, and userspace makes decisions on today, independent of kdbus/dbus.

Is there anything that userspace makes decisions on based on capabilities? If so, please tell me and I'll entertain myself by writing exploits for them.

The fact that some existing userspace does awful things does *not* justify adding new kernel mechanisms with which to repeat those mistakes.

Quoted 76 lines · Andy Lutomirski

Quoted 58 lines · Greg Kroah-Hartman

Quoted 31 lines · Andy Lutomirski

Quoted 21 lines · Greg Kroah-Hartman

The following changes since commit 9eccca0843205f87c00404b663188b88eb248051:

Linux 4.0-rc3 (2015-03-08 16:09:09 -0700)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git/ tags/kdbus-4.1-rc1

for you to fetch changes up to 9fb9cd0f4434a23487b6ef3237e733afae90e336:

kdbus: avoid the use of struct timespec (2015-04-10 14:34:53 +0200)

----------------------------------------------------------------
kdbus for 4.1-rc1

Here's the kdbus pull request for 4.1-rc1.

It's been under development for many years now, and been in linux-next for many months, and has undergone loads of testing a review and even a few good arguments. It comes with full documentation and tests.

Quoted 7 lines

There has been a few complaints about the code, notably from people who don't like the use of metadata in the bus messages. That is actually one of the main features here, as we can get this data in a secure and reliable way, and it's something that userspace requires today. So while it does look "odd" to people who are not familiar with dbus, this is something that finally fixes a number of almost unfixable races in the current dbus implementations.

Quoted 8 lines

While I generally like the concept of having a better in-kernel IPC mechanism, after some consideration I don't think this belongs in the kernel in its current form. Here's why.

First, the naming is counterintuitive. There are "endpoints", but you don't send messages to endpoints. In fact, an basic kdbus setup will have exactly one endpoint AFAICT. Wtf? This makes talking about it awkward.

Did you read the documentation? We've been over this before, and it should all be addressed in the documentation based on this coming up.

Quoted 5 lines

A lot of the design seems to be to violate the concept of "mechanism, not policy". Kdbus is very much a port of userspace dbus to the kernel, and it appears to be a port designed to preserve some questionable design decisions instead of learning from them.

Quoted 8 lines

For example, kdbus sticks a whole policy database in the kernel, but that policy database (AFAICT -- holy crap it's overcomplicated) is *not* a simple set of rules like "if A then allow B". Instead it has really weird dependencies not on what name you're sending to but on what *other* names the thing you're sending to has. Sorry, but this way lies (a) the inability for a large set of developers to understand what's going on and (b) security bugs. Also, the result probably can't be reused as part of a non-legacy-filled sensible design

Quoted 3 lines

What policy database? Matching messages to subscribers? That's the same type of "database" that other ipc subsystems need/want, there's nothing radical here.

Let me quote from the latest version of the kdbus docs:

Note that TALK access is checked against all names of a connection. For example, if a connection owns both <constant>'org.foo.bar'</constant> and <constant>'org.blah.baz'</constant>, and the policy database allows <constant>'org.blah.baz'</constant> to be talked to by WORLD, then this permission is also granted to <constant>'org.foo.bar'</constant>. That might sound illogical, but after all, we allow messages to be directed to either the ID or a well-known name, and policy is applied to the connection, not the name. In other words, the effective TALK policy for a connection is the most permissive of all names the connection owns.

Quoted 2 lines

In my humble opinion, this paragraph speaks for itself. The design is bad, full stop.

First off, thanks for reading the docs, I appreciate that. But realize also, that this is straight from the D-Bus spec. We aren't doing anything "radical" here, this is what your desktop uses that you are typing your email from.

Yes, it's an unfortunate design, but one that we are all stuck with (think of it as having to implement code for horrid hardware that you have to get to work properly.)

There are many applications out there
which don't address messages to their well-known name destination but to the ID which they looked up earlier and cached. In fact, that behavior is the default in the gdbus library implementation.

If a connection owns two names, and one is more permissive than the other one, an attacker could as well choose the more openly configured name to get a message delivered. That's nothing we can protect from really. So ideally you never do that, just like you shouldn't do that in an network configuration with DNS, if you want to manage access properly.

The logic here is comparable to IP vs. DNS

Quoted 8 lines

Then I'll have to find a way to embolden my NACK further. My point is that capturing garbage like cmdline and capabilities (again, that latter part is completely unacceptable under any circumstances whatsoever) on behalf of *all* senders is a disaster. If it's optional, then I can at least hope that userspace will honor the optionality and let everything turn it off. If it's mandatory, then kdbus is just unsafe to use to send messages to untrusted parties.

It's opted in by the receiving peer if the task implementing a service wants to access these pieces of information. It is optional, and the documentation clearly states that userspace should cope with this, and also, when they are available we make sure to provide the correct race-free information.

As said many times before, an application can do so already today with information from other API file systems, so why is this suddenly a problem when kdbus optionally offers the exact same information along with each transmitted message? Yes, we all "hate" capabilities, but userspace uses them, and gets access to them all the time through the POSIX apis (capget(), cap_get_pid(), capgetp(), etc.) and through /proc/pid/status. They are something that we have to support and handle properly.

In the very first submission of kdbus, we stated that we want to allow userspace methods to access these same bits to be able to make decisions about permissions. And to do so in a race-free manner, which is very hard, if not almost impossible, to do so from userspace alone.

For instance, if a task has CAP_NET_ADMIN set, we can use that information in order to allow or disallow certain actions to be taken by a privileged process. Or, if a client that has the capability to call reboot (i.e. have CAP_SYS_REBOOT) makes the D-Bus call to reboot the system, the system daemon listening for that message knows that yes, at the time that the client made that call, it really did have that capability so it is ok to actually reboot the system.

Instead of trying to use SCM_CREDENTIALS to get the pid and another round of cap_get_pid() and the like, all of which are susceptable to racing and all sorts of other horrors, that are insecure, we can provide this information in an atomic, and secure way.

The kernel today, and userspace, relies on capabilities all the time (i.e. almost every syscall), how are they something that is somehow not valid to use and support?

And of course, as Eric will point out, capabailities are not translatable across user namespaces, which is a problem. Because of this, we dispose of that piece of metadata information when a message crosses a user namespace boundry. This is the right thing to do, which is not the case for almost all other kernel apis which report bogus capabilies when user namespaces are crossed.

So we implemented this correctly, and somehow that is a feature so bad that both you and Eric think the whole baby should be thrown out? How else should this be implemented?

As documented in the original email on this thread, Tizen wants to use this, as it solves a real need that they have. Their workarounds involve using custom UDS sockets, but the latency involved is horrid and unacceptable. Using a kdbus message solves this issue for them, allowing UI rendering to work properly/quickly.

Again, capabilities are something we all require and rely on today, passing the current capability on to a recipient isn't a way to raise privileges at all, but rather, properly determine if they are present at sending time, if wanted. How does that create an insecure system? What am I missing that is so bad here with the design we have?

Quoted 85 lines · Greg Kroah-Hartman

Quoted 76 lines · Andy Lutomirski

Quoted 58 lines · Greg Kroah-Hartman

Quoted 31 lines · Andy Lutomirski

Quoted 21 lines · Greg Kroah-Hartman

The following changes since commit 9eccca0843205f87c00404b663188b88eb248051:

Linux 4.0-rc3 (2015-03-08 16:09:09 -0700)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git/ tags/kdbus-4.1-rc1

for you to fetch changes up to 9fb9cd0f4434a23487b6ef3237e733afae90e336:

kdbus: avoid the use of struct timespec (2015-04-10 14:34:53 +0200)

----------------------------------------------------------------
kdbus for 4.1-rc1

Here's the kdbus pull request for 4.1-rc1.

It's been under development for many years now, and been in linux-next for many months, and has undergone loads of testing a review and even a few good arguments. It comes with full documentation and tests.

Quoted 7 lines

There has been a few complaints about the code, notably from people who don't like the use of metadata in the bus messages. That is actually one of the main features here, as we can get this data in a secure and reliable way, and it's something that userspace requires today. So while it does look "odd" to people who are not familiar with dbus, this is something that finally fixes a number of almost unfixable races in the current dbus implementations.

Quoted 8 lines

While I generally like the concept of having a better in-kernel IPC mechanism, after some consideration I don't think this belongs in the kernel in its current form. Here's why.

First, the naming is counterintuitive. There are "endpoints", but you don't send messages to endpoints. In fact, an basic kdbus setup will have exactly one endpoint AFAICT. Wtf? This makes talking about it awkward.

Did you read the documentation? We've been over this before, and it should all be addressed in the documentation based on this coming up.

Quoted 5 lines

A lot of the design seems to be to violate the concept of "mechanism, not policy". Kdbus is very much a port of userspace dbus to the kernel, and it appears to be a port designed to preserve some questionable design decisions instead of learning from them.

Quoted 8 lines

For example, kdbus sticks a whole policy database in the kernel, but that policy database (AFAICT -- holy crap it's overcomplicated) is *not* a simple set of rules like "if A then allow B". Instead it has really weird dependencies not on what name you're sending to but on what *other* names the thing you're sending to has. Sorry, but this way lies (a) the inability for a large set of developers to understand what's going on and (b) security bugs. Also, the result probably can't be reused as part of a non-legacy-filled sensible design

Quoted 3 lines

What policy database? Matching messages to subscribers? That's the same type of "database" that other ipc subsystems need/want, there's nothing radical here.

Let me quote from the latest version of the kdbus docs:

Note that TALK access is checked against all names of a connection. For example, if a connection owns both <constant>'org.foo.bar'</constant> and <constant>'org.blah.baz'</constant>, and the policy database allows <constant>'org.blah.baz'</constant> to be talked to by WORLD, then this permission is also granted to <constant>'org.foo.bar'</constant>. That might sound illogical, but after all, we allow messages to be directed to either the ID or a well-known name, and policy is applied to the connection, not the name. In other words, the effective TALK policy for a connection is the most permissive of all names the connection owns.

Quoted 2 lines

In my humble opinion, this paragraph speaks for itself. The design is bad, full stop.

First off, thanks for reading the docs, I appreciate that. But realize also, that this is straight from the D-Bus spec. We aren't doing anything "radical" here, this is what your desktop uses that you are typing your email from.

Quoted 3 lines

Yes, it's an unfortunate design, but one that we are all stuck with (think of it as having to implement code for horrid hardware that you have to get to work properly.)

I agree. You've sent a pull request for an unfortunate design. I don't think that unfortunate design belongs in the kernel. If it says in userspace, then user programmers could potentially fix it some day.

You might not like the design, but it is a valid design. Again, we don't refuse to support hardware that is designed badly.

Quoted 2 lines · Greg Kroah-Hartman

You might not like the design, but it is a valid design. Again, we don't refuse to support hardware that is designed badly.

Yeah except the small difference that unlike this, we can't change hardware.

Quoted 6 lines · Borislav Petkov

Quoted 2 lines · Greg Kroah-Hartman

You might not like the design, but it is a valid design. Again, we don't refuse to support hardware that is designed badly.

Yeah except the small difference that unlike this, we can't change hardware.

And we can't change the design/implementation of many things, again, it's not the kernel's job to prevent something, just because we don't like the RFC, from being accepted.

Quoted 8 lines · Greg Kroah-Hartman

Quoted 6 lines · Borislav Petkov

Quoted 2 lines · Greg Kroah-Hartman

You might not like the design, but it is a valid design. Again, we don't refuse to support hardware that is designed badly.

Yeah except the small difference that unlike this, we can't change hardware.

Quoted 3 lines

And we can't change the design/implementation of many things, again, it's not the kernel's job to prevent something, just because we don't like the RFC, from being accepted.

Translate, please. What exactly will be prevented by NAK on your Fine Piece Of Software? Not dbus working as it does, surely?

It's not going to stop anything from working, it's just going to stop some programs from being able to do things they really want to do (see the first email for examples.)

Yes, we could make this live outside the kernel tree, but that's not the way we work anymore.

We merge things that are useful, that match our
security and coding requirements, and are going to be maintained by people we trust.

Quoted 3 lines · Greg Kroah-Hartman

It's not going to stop anything from working, it's just going to stop some programs from being able to do things they really want to do (see the first email for examples.)

Until it is made "mandatory" as Al said earlier.

Quoted 6 lines

Yes, we could make this live outside the kernel tree, but that's not the way we work anymore.

We merge things that are useful, that match our
security and coding requirements, and are going to be maintained by people we trust.

We trust? I'm not going to even comment on that.

And frankly, merging a useful piece of code sounds completely different to me than this serious backlash I'm reading from the sidelines.

If you have a technical reason for why this code shouldn't be merged, great, please let me know and we can work to address that. Andy and Al have spent time reviewing and giving us comments, and that's wonderful and valuable and is why I treat their comments seriously. If you are interested in the code, please review it, otherwise I don't see what this adds to the conversation at all, do you?

Quoted 6 lines · Greg Kroah-Hartman

If you have a technical reason for why this code shouldn't be merged, great, please let me know and we can work to address that. Andy and Al have spent time reviewing and giving us comments, and that's wonderful and valuable and is why I treat their comments seriously. If you are interested in the code, please review it, otherwise I don't see what this adds to the conversation at all, do you?

You've actually touched another issue I see here, and that is -- the code is complex like crazy.

I've spent big part of past two days trying to get my head around it, but I am still far away from getting at least the 1000 miles overview of how exactly the message passing is designed.

I understand that the primary reason for this complexity is probably the dbus protocol specification itself.

But the problem really is that I don't think you've received even a single
Reviewed-by: from someone who hasn't been directly involved in developing the code, right?

For something that's potentially such a core mechanism as a completely new, massively-adopted IPC, this does send a warning singal.

I've asked for it, but finding people to review code is hard, as you know. It's only 13k lines long, smaller than a serial port driver (my unit of code review), so it's not all that big.

It's smaller than the USB3 host controller driver as well, and very few people ever reviewed that beast :)

Quoted 2 lines

For something that's potentially such a core mechanism as a completely new, massively-adopted IPC, this does send a warning singal.

If you know of a way to force others to review code, please let me know.

al@duke:~/linux/trees/vfs$ git grep -n -w kdbus_node_idr_lock
ipc/kdbus/node.c:237:static DECLARE_RWSEM(kdbus_node_idr_lock);
ipc/kdbus/node.c:340:   down_write(&kdbus_node_idr_lock);
ipc/kdbus/node.c:344:   up_write(&kdbus_node_idr_lock);
ipc/kdbus/node.c:444:           down_write(&kdbus_node_idr_lock);
ipc/kdbus/node.c:452:           up_write(&kdbus_node_idr_lock);

Do you see anything wrong with that? Or with things like that:

                mutex_lock(&pos->lock);
                v_pre = atomic_read(&pos->active);
                if (v_pre >= 0)
                        atomic_add_return(KDBUS_NODE_BIAS, &pos->active);
                else if (v_pre == KDBUS_NODE_NEW)
                        atomic_set(&pos->active, KDBUS_NODE_RELEASE_DIRECT);
                mutex_unlock(&pos->lock);

What are the locking rules for ->active/->waitq/->lock? Are those the outermost thing in the hierarchy? Or is that dependent on the node location? It sure as hell is outside of (at least) ->mmap_sem (by way of kdbus_conn_connect() establishing that ->active/->waitq is outside of ->conn_rwlock, which due to kdbus_bus_broadcast() nests outside of anything taken by kdbus_meta_proc_collect(), which includes ->mmap_sem) and that alone brings in a lot...

Document your goddamn locking, would you? It *IS* new code, and you, as you say, had very few people working on it, so you don't have the excuses for the mess existing in older parts of the tree.

Locking complexity in there is easily as bad as that of VFS sans the RCU fun; sure, I can spend a week and (hopefully) document it for you, but I would really prefer if you guys had done that. And I *do* appreciate the comments in node.c, but they are nowhere near enough.

Quoted 4 lines

But the problem really is that I don't think you've received even a single
Reviewed-by: from someone who hasn't been directly involved in developing the code, right?

I've asked for it, but finding people to review code is hard, as you

know. It's only 13k lines long, smaller than a serial port driver (my unit of code review), so it's not all that big.

It's smaller than the USB3 host controller driver as well, and very few people ever reviewed that beast :)

Quoted 2 lines

For something that's potentially such a core mechanism as a completely new, massively-adopted IPC, this does send a warning singal.

If you know of a way to force others to review code, please let me know.

If you really don't like userspace using features the kernel provides you, well, there's nothing I can say that will change that odd feeling, sorry.

Really? Who in that MAINTAINERS file entry do you not trust?

I don't understand what this means. If you have a technical reason for why this code shouldn't be merged, great, please let me know and we can work to address that. Andy and Al have spent time reviewing and giving us comments, and that's wonderful and valuable and is why I treat their comments seriously. If you are interested in the code, please review it,

Quoted 3 lines · Greg Kroah-Hartman

If you really don't like userspace using features the kernel provides you, well, there's nothing I can say that will change that odd feeling, sorry.

Are you even reading what people are saying?

I don't like the mandatory(!) aspect of this, which it will eventually become. There is this thing called "choice", remember?

Quoted 1 line

Really? Who in that MAINTAINERS file entry do you not trust?

The fact that you're still pushing for this current design *in the face* of people pointing out serious design flaws with this makes me not really trust you.

Quoted 6 lines

I don't understand what this means. If you have a technical reason for why this code shouldn't be merged, great, please let me know and we can work to address that. Andy and Al have spent time reviewing and giving us comments, and that's wonderful and valuable and is why I treat their comments seriously. If you are interested in the code, please review it,

Yeah, I took a brief look at the code. It is overcomplicated.

If I were to review it properly, I'd ask you to split it in small patchsets. Hell, I'm pretty sure you would do the same for code you don't know if you were in my shoes.

Also, considering the complexity of this patchset, it doesn't have a single Reviewed-by by an external party. If this were any other submission, it would've been kicked to the curb a long time ago.

Quoted 3 lines · Greg Kroah-Hartman

Quoted 2 lines · Borislav Petkov

Quoted 1 line

On Tue, Apr 14, 2015 at 09:23:57PM +0200, Greg Kroah-Hartman

Quoted 7 lines

Quoted 7 lines

Quoted 6 lines

Quoted 3 lines

You might not like the design, but it is a valid design. Again, we
don't refuse to support hardware that is designed badly.

Yeah except the small difference that unlike this, we can't change hardware.

Quoted 7 lines

Quoted 4 lines

And we can't change the design/implementation of many things, again, it's not the kernel's job to prevent something, just because we don't
like the RFC, from being accepted.

Translate, please. What exactly will be prevented by NAK on your Fine Piece Of Software? Not dbus working as it does, surely?

I don't understand. You can not like the D-Bus model (and accordingly the X11 model), but to prevent users from wanting to use it in a more secure, and faster way by implementing it like we have seems very odd to me.

It's not going to stop anything from working, it's just going to stop some programs from being able to do things they really want to do (see the first email for examples.)

Yes, we could make this live outside the kernel tree, but that's not the way we work anymore. We merge things that are useful, that match our security and coding requirements, and are going to be maintained by people we trust. To have the only major objection be "we don't like the way the protocol is designed because we know better, sorry", isn't ok at all.

I don't understand. You can not like the D-Bus model (and accordingly the X11 model),

Quoted 2 lines · Greg Kroah-Hartman

I don't understand. You can not like the D-Bus model (and accordingly the X11 model),

I thought that the general hatred level of the X11 "model" and the protocol lead to al the efforts to reimplement this properly ... in userspace (for example Wayland, right?).

I don't think anyone was ever seriously suggesting "X11 model is broken, so let's push it to kernel" ... ?

Quoted 2 lines · Greg Kroah-Hartman

I don't understand. You can not like the D-Bus model (and accordingly the X11 model),

I thought that the general hatred level of the X11 "model" and the protocol lead to al the efforts to reimplement this properly ... in userspace (for example Wayland, right?).

I don't think anyone was ever seriously suggesting "X11 model is broken, so let's push it to kernel" ... ?

Quoted 11 lines · On Wed, 15 Apr 2015 00:39:22 +0200 (CEST) Jiri Kosina <[email protected]> wrote:

Quoted 2 lines · Greg Kroah-Hartman

I don't understand. You can not like the D-Bus model (and accordingly the X11 model),

I thought that the general hatred level of the X11 "model" and the protocol lead to al the efforts to reimplement this properly ... in userspace (for example Wayland, right?).

I don't think anyone was ever seriously suggesting "X11 model is broken, so let's push it to kernel" ... ?

The X11 model is *nothing* to do with the dbus/kdbus model. X11 does properties by attaching them to windows. Those properties can be monitored for changes and they can be queried. Setting them is asynchronous, querying them is sync or with the newer event based libraries can be async. X11 properties are network safe, handled through the same X11 authority as everything else. Two apps can happily run on different systems sharing a display over the network and sharing and responding to changes in X11 properties - and it just works.

The Gnome people tried to re-invent X11 properties and embedding badly with CORBA, then with dbus, despite the fact the Andrew system could already do it really fast and cleanly even before Gnome was thought of.

There is no comparison between the elegance of X11 property setting and a chunk of proposed kernel code that is half the size of a tiny X server!

The dbus model is also flawed in a load of other ways in user space because message handling in the hands of people with no concept of systemic performance analysis just leads to disaster. One of the big reasons dbus is so "slow" isn't that dbus is "slow", it's that the crapware on top of it makes *thousands* of dbus queries.

If you must do it in kernel why not use the Android binder - it's awful, broken, and dubiously secure, but at least we'd still only have one awful, broken dubiously secure rpc/property layer in kernel.

"It's the issue that a stateful bus is required for applications that is the main point I'm trying to get across."

That would be the "if dbus crashes I have to reboot" design flaw of Gnome and friends. The only state you need is beyond the endpoints. It's a message passing system. If you think message passing needs state then I'd take a look at the internet. State belongs in the end points.

It's telling that I can lose and recover my internet connection without rebooting but not my desktops internal messaging.

Quoted 2 lines

There is no comparison between the elegance of X11 property setting and a chunk of proposed kernel code that is half the size of a tiny X server!

Hey, take that up with Havoc, he made the comparison :)

Quoted 5 lines

The dbus model is also flawed in a load of other ways in user space because message handling in the hands of people with no concept of systemic performance analysis just leads to disaster. One of the big reasons dbus is so "slow" isn't that dbus is "slow", it's that the crapware on top of it makes *thousands* of dbus queries.

There's the issue of thousands of dbus queries, and then there's the issue that making those queries takes a measurable amount of time. We can fix the later one, the first one, well, not so much, but we can provide the resources for them to make a faster system if they want to.

The internet model with state in the endpoints doesn't always transfer properly to local applications, see Havoc's email for the details about that.

Quoted 2 lines

It's telling that I can lose and recover my internet connection without rebooting but not my desktops internal messaging.

Yes, as those are totally different things, let's not mix the issue up here please.

Quoted 4 lines

Quoted 2 lines

There is no comparison between the elegance of X11 property setting and a chunk of proposed kernel code that is half the size of a tiny X server!

Hey, take that up with Havoc, he made the comparison :)

And it concerns me you blindly repeat it without realising its wrong.

Quoted 6 lines

Quoted 5 lines

The dbus model is also flawed in a load of other ways in user space because message handling in the hands of people with no concept of systemic performance analysis just leads to disaster. One of the big reasons dbus is so "slow" isn't that dbus is "slow", it's that the crapware on top of it makes *thousands* of dbus queries.

Quoted 4 lines

There's the issue of thousands of dbus queries, and then there's the issue that making those queries takes a measurable amount of time. We can fix the later one, the first one, well, not so much, but we can provide the resources for them to make a faster system if they want to.

If you fix the thousands of queries problem do you need kernel help at all.

Quoted 3 lines

The internet model with state in the endpoints doesn't always transfer properly to local applications, see Havoc's email for the details about that.

URL ?

(note how beautifully btw the stateless network and the URL string will become a reference to state)

Quoted 5 lines

Quoted 2 lines

It's telling that I can lose and recover my internet connection without rebooting but not my desktops internal messaging.

Yes, as those are totally different things, let's not mix the issue up here please.

They are *NOT* different things. They are fundamental properties of the underlying architecture. I worked on stateful networks and still have the scars. It is a fundamental property of stateful network that every time any key component goes castors up you lose the lot. It is a fairly fundamental property of stateless networks that equipment going castors up has no material impact on the network

The internet is built upon three fundamental breakthroughs in technology

- That stateless networks scale and can be reliable while stateful ones cannot scale and cannot be fixed to do so

- That flow control is possible over a stateless network

- That efficient data routing is possible over a stateless network

Those are absolutely critical parts of any network or messaging implementation.

operating systems anymore. Those "legacy" oses provide a system bus that allows them to send thousands of queries just fine, but when moving to Linux, we don't have anything other than D-Bus, so their library is ported to use it, and they have to handle their old applications that need/want the zillions of messages.

People take those stateless models and build stateful ones on top of them, yes, it's great. But you still need a stateful model somewhere in order to be able to achieve many things (think a shopping cart application).

Anyway, this is getting off-topic, there is very little "state" in the kdbus kernel code here, other than a naming database that Havoc and Simon explain the need for, and the normal lifecycle of kdbus "nodes" (new, linked, active, inactive, drained, freed).

Quoted 2 lines

There is no comparison between the elegance of X11 property setting and a chunk of proposed kernel code that is half the size of a tiny X server!

Hey, take that up with Havoc, he made the comparison :)

There's the issue of thousands of dbus queries, and then there's the issue that making those queries takes a measurable amount of time. We can fix the later one, the first one, well, not so much, but we can provide the resources for them to make a faster system if they want to.

Quoted 5 lines · Greg Kroah-Hartman

There's the issue of thousands of dbus queries, and then there's the issue that making those queries takes a measurable amount of time. We can fix the later one, the first one, well, not so much, but we can provide the resources for them to make a faster system if they want to.

I'll argue that you can't fix the later one. One thing that I've observed over the years of having faster computers is, as soon as you make it faster, people will write slower software.

Currently the issue is that we have thousands of dbus queries, you make dbus 10x faster, I guarantee that people will write software with 10 thousand dbus queries and we are no better off than we are today.

Quoted 3 lines

I agree. You've sent a pull request for an unfortunate design. I don't think that unfortunate design belongs in the kernel. If it says in userspace, then user programmers could potentially fix it some day.

You might not like the design, but it is a valid design. Again, we don't refuse to support hardware that is designed badly. Or support protocols we don't necessarily like, that's not the job of a kernel or operating system.

Quoted 4 lines · Greg Kroah-Hartman

Quoted 3 lines

I agree. You've sent a pull request for an unfortunate design. I don't think that unfortunate design belongs in the kernel. If it says in userspace, then user programmers could potentially fix it some day.

Quoted 4 lines

You might not like the design, but it is a valid design. Again, we don't refuse to support hardware that is designed badly. Or support protocols we don't necessarily like, that's not the job of a kernel or operating system.

Bullshit. The problem you seem to deliberately ignore is that once it's in the kernel, it's impossible to eradicate. It's not just a crap design, it's a crap design you are taking in as-is.

And no, "the sole consumer of that API knows better, so bend over" is not a good idea. We have shitloads of examples when single-consumer APIs turned into screaming horrors; taking that in over the objections to API design, merely on "they do it that way, who the hell we are to say they are wrong?" is insane.

Quoted 11 lines · Al Viro

Quoted 8 lines · Greg Kroah-Hartman

Quoted 3 lines

I agree. You've sent a pull request for an unfortunate design. I don't think that unfortunate design belongs in the kernel. If it says in userspace, then user programmers could potentially fix it some day.

You might not like the design, but it is a valid design. Again, we don't refuse to support hardware that is designed badly. Or support protocols we don't necessarily like, that's not the job of a kernel or operating system.

Quoted 5 lines

And no, "the sole consumer of that API knows better, so bend over" is not a good idea. We have shitloads of examples when single-consumer APIs turned into screaming horrors; taking that in over the objections to API design, merely on "they do it that way, who the hell we are to say they are wrong?" is insane.

Again, in this domain, the design is sound. So much so that everyone who works in that area moved toward it (KDE, Qt, Go, etc.) We might not think it makes sense, and it did take me a while to wrap my head around it, but to call it "crap" is unfair, sorry.

Quoted 19 lines · Greg Kroah-Hartman

Quoted 11 lines · Al Viro

Quoted 8 lines · Greg Kroah-Hartman

Quoted 3 lines

I agree. You've sent a pull request for an unfortunate design. I don't think that unfortunate design belongs in the kernel. If it says in userspace, then user programmers could potentially fix it some day.

You might not like the design, but it is a valid design. Again, we don't refuse to support hardware that is designed badly. Or support protocols we don't necessarily like, that's not the job of a kernel or operating system.

Quoted 6 lines

And no, "the sole consumer of that API knows better, so bend over" is not
a good idea. We have shitloads of examples when single-consumer APIs turned into screaming horrors; taking that in over the objections to API design, merely on "they do it that way, who the hell we are to say they are wrong?" is insane.

Quoted 5 lines

Again, in this domain, the design is sound. So much so that everyone who works in that area moved toward it (KDE, Qt, Go, etc.) We might not think it makes sense, and it did take me a while to wrap my head around it, but to call it "crap" is unfair, sorry.

The reason that 'everyone who works in this area' adopted is not as much that the design is sound (I'm not arguing whether it is or isn't in this case) as it is that none of them could come up with anything better.

The reason that 'everyone who works in this area' adopted is not as much that the design is sound (I'm not arguing whether it is or isn't in this case) as it is that none of them could come up with anything better.

Actually most message passing code uses things like JMS and the various MQ libraries. Most IoT uses things other than dbus, small deep embedded never uses dbus.

In the desktop space dbus wins because its very very easy to use and by network effects. Everything else related already talks via dbus, so you are going to have to talk dbus anyway to get anything done.

Quoted 11 lines · Greg Kroah-Hartman

I've asked for it, but finding people to review code is hard, as you know. It's only 13k lines long, smaller than a serial port driver (my unit of code review), so it's not all that big.

It's smaller than the USB3 host controller driver as well, and very few people ever reviewed that beast :)

Quoted 2 lines

For something that's potentially such a core mechanism as a completely new, massively-adopted IPC, this does send a warning singal.

If you know of a way to force others to review code, please let me know.

Have it in a less nasty state, perhaps? Random question:

al@duke:~/linux/trees/vfs$ git grep -n -w kdbus_node_idr_lock
ipc/kdbus/node.c:237:static DECLARE_RWSEM(kdbus_node_idr_lock);
ipc/kdbus/node.c:340:   down_write(&kdbus_node_idr_lock);
ipc/kdbus/node.c:344:   up_write(&kdbus_node_idr_lock);
ipc/kdbus/node.c:444:           down_write(&kdbus_node_idr_lock);
ipc/kdbus/node.c:452:           up_write(&kdbus_node_idr_lock);

Do you see anything wrong with that? Or with things like that:

                mutex_lock(&pos->lock);
                v_pre = atomic_read(&pos->active);
                if (v_pre >= 0)
                        atomic_add_return(KDBUS_NODE_BIAS, &pos->active);
                else if (v_pre == KDBUS_NODE_NEW)
                        atomic_set(&pos->active, KDBUS_NODE_RELEASE_DIRECT);
                mutex_unlock(&pos->lock);

What are the locking rules for ->active/->waitq/->lock? Are those the outermost thing in the hierarchy? Or is that dependent on the node location? It sure as hell is outside of (at least) ->mmap_sem (by way of kdbus_conn_connect() establishing that ->active/->waitq is outside of ->conn_rwlock, which due to kdbus_bus_broadcast() nests outside of anything taken by kdbus_meta_proc_collect(), which includes ->mmap_sem) and that alone brings in a lot...

Document your goddamn locking, would you? It *IS* new code, and you, as you say, had very few people working on it, so you don't have the excuses for the mess existing in older parts of the tree.

I'm working on patches to add more comments similar to how we did in node.c. For now, please see my explanations below:

node->lock is the _innermost_ lock.
node->active implements revoke
support for nodes. It follows what kernfs->active does and isn't a lock in particular. We kinda treat it as rwsem, where down_write() is the outer-most lock in kdbus and _only_ called without any other lock held (kdbus_node_deactivate()). Read-side, we never ever block on the "lock", but only use try-lock. If it fails, the node is dead/revoked. Therefore, the read-side of 'active' nests almost arbitrarily. We hold 'active'-references almost everywhere, to make sure a node is not destroyed while we use it. However, we never sleep for an indefinite time while holding it.

Given that the write-side is the outer-most lock in kdbus, it doesn't dead-lock against the try-lock readers.

Locking order (outer-most to inner-most):
1) domain->lock
2) names->rwlock
3) endpoint->lock
4) bus->conn_rwlock
5) policy->entries_rwlock
6) connection->lock
7) metadata->lock

mmap_sem nests below metadata->lock. With the rcu-protected exe_file patches by Davidlohr Bueso, we can even drop that dependency. They have kinda stalled, though.

Then we have a bunch of data structure protection, which can be called from any context:
* bus->notify_lock
* pool->lock
* match->mdb_rwlock
* node->lock

Lastly, there're 2 locks which nest around everything and must not be taken with any lock held:
* handle->rwlock (taken in ioctl-entry)

* bus->notify_flush_lock (taken in work-queue)

General object stacking is:
domain -> bus -> endpoint -> policy -> connection -> {metadata,pool,match,node} The conn_rwlock protection of the conn-list locks on kdbus_bus is the only lock that doesn't follow this ordering.

Quoted 14 lines · David Herrmann

I'm working on patches to add more comments similar to how we did in node.c. For now, please see my explanations below:

node->lock is the _innermost_ lock.
node->active implements revoke
support for nodes. It follows what kernfs->active does and isn't a lock in particular. We kinda treat it as rwsem, where down_write() is the outer-most lock in kdbus and _only_ called without any other lock held (kdbus_node_deactivate()). Read-side, we never ever block on the "lock", but only use try-lock. If it fails, the node is dead/revoked. Therefore, the read-side of 'active' nests almost arbitrarily. We hold 'active'-references almost everywhere, to make sure a node is not destroyed while we use it. However, we never sleep for an indefinite time while holding it.

Umm... Theoretically, but ->mmap_sem being under it means that it might involve something like an NFS server timing out, so the latency might suck very badly.

Quoted 2 lines

Given that the write-side is the outer-most lock in kdbus, it doesn't dead-lock against the try-lock readers.

Huh? I see at least this call chain:
kdbus_handle_ioctl_control()
kdbus_node_acquire()
kdbus_cmd_bus_make()
kdbus_node_deactivate()
Granted, it won't be the _same_ node (otherwise you'd deadlock solid right there and then), but it means that your locking order is sensitive to something about nodes; it's not entirely determined by the lock type.

Quoted 20 lines

Locking order (outer-most to inner-most):
1) domain->lock
2) names->rwlock
3) endpoint->lock
4) bus->conn_rwlock
5) policy->entries_rwlock
6) connection->lock
7) metadata->lock

mmap_sem nests below metadata->lock. With the rcu-protected exe_file patches by Davidlohr Bueso, we can even drop that dependency. They have kinda stalled, though.

Then we have a bunch of data structure protection, which can be called from any context:
* bus->notify_lock
* pool->lock
* match->mdb_rwlock
* node->lock

Quoted 3 lines

Lastly, there're 2 locks which nest around everything and must not be taken with any lock held:
* handle->rwlock (taken in ioctl-entry)

as well as in ->poll(), for completeness sake. The latter, BTW, isn't nice - kdbus is far from being the only thing that does it, but having ->poll() block can be somewhat surprising...

Quoted 1 line

* bus->notify_flush_lock (taken in work-queue)

Hmm... That needs some care - it means that it nests inside anything held by callers of cancel_delayed_work_sync() on the corresponding work. AFAICS, there's at least one call chain leading to that from kdbus_node_deactivate() (via ->release_cb == kdbus_ep_release -> kdbus_conn_disconnect -> cancel_delayed_work_sync(&conn->work)) wait for kdbus_reply_list_scan_work -> kdbus_notify_flush grabs ->notify_flush_lock). Tracking back further is harder - not all call sites of kdbus_node_deactivate() can lead to that...

BTW, it's not only done in wq callbacks - there's a direct chain from kdbus_conn_disconnect() as well (both through kdbus_name_release_all -> kdbus_notify_flush and directly through kdbus_notify_flush()). And from ioctl(), by many paths, while we are at it, but that only means that it nests inside handle->rwlock, and _that_ is really the outermost.

What nests inside that one? It definitely a part of hierarchy - it can't be excluded from deadlock analysis as effectively outermost. As for the stuff under it... registry->rwlock is obvious, what else?

Quoted 3 lines

The reason that 'everyone who works in this area' adopted is not as much that the design is sound (I'm not arguing whether it is or isn't in this case) as it is that none of them could come up with anything better.

Actually most message passing code uses things like JMS and the various MQ libraries. Most IoT uses things other than dbus, small deep embedded never uses dbus.

Quoted 7 lines · One Thousand Gnomes

Quoted 3 lines

The reason that 'everyone who works in this area' adopted is not as much that the design is sound (I'm not arguing whether it is or isn't in this case) as it is that none of them could come up with anything better.

Actually most message passing code uses things like JMS and the various MQ libraries. Most IoT uses things other than dbus, small deep embedded never uses dbus.

For what it's worth: we more and more use dbus for small deep embedded systems, IoT, loosely coupled industrial control applications etc.

Quoted 23 lines · Greg Kroah-Hartman

Quoted 14 lines · Al Viro

Quoted 9 lines · Greg Kroah-Hartman

Quoted 3 lines

I agree. You've sent a pull request for an unfortunate design. I don't think that unfortunate design belongs in the kernel. If it says in userspace, then user programmers could potentially fix it some day.

You might not like the design, but it is a valid design. Again, we don't refuse to support hardware that is designed badly. Or support protocols we don't necessarily like, that's not the job of a kernel or operating system.

Quoted 5 lines

And no, "the sole consumer of that API knows better, so bend over" is not a good idea. We have shitloads of examples when single-consumer APIs turned into screaming horrors; taking that in over the objections to API design, merely on "they do it that way, who the hell we are to say they are wrong?" is insane.

Quoted 5 lines

Again, in this domain, the design is sound. So much so that everyone who works in that area moved toward it (KDE, Qt, Go, etc.) We might not think it makes sense, and it did take me a while to wrap my head around it, but to call it "crap" is unfair, sorry.

The reason that 'everyone who works in this area' adopted is not as much that the design is sound (I'm not arguing whether it is or isn't in this case) as it is that none of them could come up with anything better.

I'd like to see D-Bus in the kernel (kdbus), if that's going to make D-Bus faster.

See this application taking 15 seconds to start just because D-Bus is too slow.

https://bugs.kde.org/show_bug.cgi?id=342682

Hopefully kdbus solves problems such as this one.

Quoted 8 lines · Diego Viola

I'd like to see D-Bus in the kernel (kdbus), if that's going to make D-Bus faster.

See this application taking 15 seconds to start just because D-Bus is too slow.

https://bugs.kde.org/show_bug.cgi?id=342682

Hopefully kdbus solves problems such as this one.

That bug really doesn't look like it would be solved by kdbus, I don't see a ton of messages being sent as the issue, do you? It seems like something is timing out and then continuing on with the application startup.

But, you can try it out, grab the kernel patch, enable it in systemd, and try it for yourself and let us know!

thanks,

greg k-h

Quoted 1 line

Quoted 1 line

Quoted 1 line

Quoted 1 line

"Greg" == Greg Kroah-Hartman <[email protected]> writes:

Quoted 87 lines

Quoted 85 lines · Greg Kroah-Hartman

Quoted 76 lines · Andy Lutomirski

Quoted 58 lines · Greg Kroah-Hartman

Quoted 31 lines · Andy Lutomirski

Quoted 21 lines · Greg Kroah-Hartman

The following changes since commit 9eccca0843205f87c00404b663188b88eb248051:

Linux 4.0-rc3 (2015-03-08 16:09:09 -0700)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git/ tags/kdbus-4.1-rc1

for you to fetch changes up to 9fb9cd0f4434a23487b6ef3237e733afae90e336:

kdbus: avoid the use of struct timespec (2015-04-10 14:34:53 +0200)

----------------------------------------------------------------
kdbus for 4.1-rc1

Here's the kdbus pull request for 4.1-rc1.

It's been under development for many years now, and been in linux-next for many months, and has undergone loads of testing a review and even a few good arguments. It comes with full documentation and tests.

Quoted 7 lines

There has been a few complaints about the code, notably from people who don't like the use of metadata in the bus messages. That is actually one of the main features here, as we can get this data in a secure and reliable way, and it's something that userspace requires today. So while it does look "odd" to people who are not familiar with dbus, this is something that finally fixes a number of almost unfixable races in the current dbus implementations.

Quoted 8 lines

While I generally like the concept of having a better in-kernel IPC mechanism, after some consideration I don't think this belongs in the kernel in its current form. Here's why.

First, the naming is counterintuitive. There are "endpoints", but you don't send messages to endpoints. In fact, an basic kdbus setup will have exactly one endpoint AFAICT. Wtf? This makes talking about it awkward.

Did you read the documentation? We've been over this before, and it should all be addressed in the documentation based on this coming up.

Quoted 5 lines

A lot of the design seems to be to violate the concept of "mechanism, not policy". Kdbus is very much a port of userspace dbus to the kernel, and it appears to be a port designed to preserve some questionable design decisions instead of learning from them.

Quoted 8 lines

For example, kdbus sticks a whole policy database in the kernel, but that policy database (AFAICT -- holy crap it's overcomplicated) is *not* a simple set of rules like "if A then allow B". Instead it has really weird dependencies not on what name you're sending to but on what *other* names the thing you're sending to has. Sorry, but this way lies (a) the inability for a large set of developers to understand what's going on and (b) security bugs. Also, the result probably can't be reused as part of a non-legacy-filled sensible design

Quoted 3 lines

What policy database? Matching messages to subscribers? That's the same type of "database" that other ipc subsystems need/want, there's nothing radical here.

Let me quote from the latest version of the kdbus docs:

Note that TALK access is checked against all names of a connection. For example, if a connection owns both <constant>'org.foo.bar'</constant> and <constant>'org.blah.baz'</constant>, and the policy database allows <constant>'org.blah.baz'</constant> to be talked to by WORLD, then this permission is also granted to <constant>'org.foo.bar'</constant>. That might sound illogical, but after all, we allow messages to be directed to either the ID or a well-known name, and policy is applied to the connection, not the name. In other words, the effective TALK policy for a connection is the most permissive of all names the connection owns.

Quoted 2 lines

In my humble opinion, this paragraph speaks for itself. The design is bad, full stop.

First off, thanks for reading the docs, I appreciate that. But realize also, that this is straight from the D-Bus spec. We aren't doing anything "radical" here, this is what your desktop uses that you are typing your email from.

Quoted 7 lines

Quoted 3 lines

Yes, it's an unfortunate design, but one that we are all stuck with (think of it as having to implement code for horrid hardware that you have to get to work properly.)

I agree. You've sent a pull request for an unfortunate design. I don't think that unfortunate design belongs in the kernel. If it says in userspace, then user programmers could potentially fix it some day.

So I think that's the steps people need to take, give concrete example of how DBUS is better than anything else out there and won't cause more problems down the line.

I believe that Linux Plumbers is still accepting MicroConferences. I wonder if this would be a good one to have. Try to get everyone face to face and talk about how exactly kdbus should be implemented in the kernel.

So definitely +1 from me for this discussion to happen, being it either LPC (which I will unfortunately probably have to miss due to personal reaons this year) or KS. It might help people like me, who have trouble understanding why we need it, and LKML discussions don't provide enough answers for them.

Quoted 4 lines · Steven Rostedt

I believe that Linux Plumbers is still accepting MicroConferences. I wonder if this would be a good one to have. Try to get everyone face to face and talk about how exactly kdbus should be implemented in the kernel.

I personally would even put more emphasis on a session that would first focus on "why", before we look at "how".

I have already asked about this during the earlier RFC submissions, but the only "take-home message" I took from that discussion was "because it's faster than what we currently have". I don't find that a sufficient justification by itself for something so complex (with potential implications all over the place for the whole Linux ecosystem), especially given the fact we already have sealed memfds zerocopy etc (and I am not even talking about the "infinite set-in-stone userspace API" implications this has).

Quoted 10 lines · Jiri Kosina

Quoted 4 lines · Steven Rostedt

I believe that Linux Plumbers is still accepting MicroConferences. I wonder if this would be a good one to have. Try to get everyone face to face and talk about how exactly kdbus should be implemented in the kernel.

I personally would even put more emphasis on a session that would first focus on "why", before we look at "how".

Quoted 8 lines

I have already asked about this during the earlier RFC submissions, but the only "take-home message" I took from that discussion was "because it's faster than what we currently have". I don't find that a sufficient justification by itself for something so complex (with potential implications all over the place for the whole Linux ecosystem), especially given the fact we already have sealed memfds zerocopy etc (and I am not even talking about the "infinite set-in-stone userspace API" implications this has).

I wrote many many lines of "why" in the patch submissions, and in the first email in this thread. Are any of those specific solutions and "why" reasons not correct in your opinion? If so, great, please let me know.

But to say that no one is focusing on "why" is a slight to those of us who have been providing just that.

For me the biggest issue is the container problem: it's really hard to containerise kdbus because of the stateful nature of the protocol and the fact that it has a well known system bus. Separation into domains works for OS containers, but application containers need more fluidity. It's not unlike the same problem on windows: Windows application containers are very difficult to do because the global registry means that OLE handlers all have to run inside your container as well (effectively making it an OS container). I'm sure, since we already have a lot of containers people going to plumbers, that we can get them to turn up for the discussion.