Devices accessibility control group (v4)

include/linux/devscontrol.h new file +2 -0

new file mode 100644
index 0000000..04c168b
@@ -0,0 +1,12 @@
+#ifndef __DEVS_CONTROL_H__
+#define __DEVS_CONTROL_H__

Changes from v3:
* Ported on 2.6.25-rc3-mm1;
* Re-splitted into smaller pieces;
* Added more comments to tricky places.

This controller allows to tune the devices accessibility by tasks, i.e. grant full access for /dev/null, /dev/zero etc, grant read-only access to IDE devices and completely hide SCSI disks.

Tasks still can call mknod to create device files, regardless of whether the particular device is visible or accessible, but they may not be able to open it later.

This one hides under CONFIG_CGROUP_DEVS option.

To play with it - run a standard procedure:

 # mount -t container none /cont/devs -o devices
 # mkdir /cont/devs/0
 # echo -n $$ > /cont/devs/0/tasks

and tune device permissions.

Finally, here's the control group, which makes full use of the interfaces, declared in the previous patches.

Signed-off-by Pavel Emelyanov <[email protected]>

---
 Documentation/controllers/devices.txt |   61 +++++++

Quoted 8 lines · Pavel Emelyanov

Changes from v3:
* Ported on 2.6.25-rc3-mm1;
* Re-splitted into smaller pieces;
* Added more comments to tricky places.

This controller allows to tune the devices accessibility by tasks, i.e. grant full access for /dev/null, /dev/zero etc, grant read-only access to IDE devices and completely hide SCSI disks.

From within the kernel itself? The kernel should not be keeping track
of the mode of devices, that's what the filesystem holding /dev is for. Those modes change all the time depending on the device plugged in, and the user using the "console". Why should the kernel need to worry about any of this?

Quoted 6 lines

Tasks still can call mknod to create device files, regardless of whether the particular device is visible or accessible, but they may not be able to open it later.

This one hides under CONFIG_CGROUP_DEVS option.

Quoted 5 lines

To play with it - run a standard procedure:

 # mount -t container none /cont/devs -o devices
 # mkdir /cont/devs/0
 # echo -n $$ > /cont/devs/0/tasks

What is /cont/ for?

Quoted 1 line

and tune device permissions.

How is this done?

Why would the kernel care about this stuff?

confused,

greg k-h

Quoted 10 lines · Greg KH

Quoted 8 lines · Pavel Emelyanov

Changes from v3:
* Ported on 2.6.25-rc3-mm1;
* Re-splitted into smaller pieces;
* Added more comments to tricky places.

This controller allows to tune the devices accessibility by tasks, i.e. grant full access for /dev/null, /dev/zero etc, grant read-only access to IDE devices and completely hide SCSI disks.

Quoted 5 lines

From within the kernel itself? The kernel should not be keeping track
of the mode of devices, that's what the filesystem holding /dev is for. Those modes change all the time depending on the device plugged in, and the user using the "console". Why should the kernel need to worry about any of this?

These are distinct from the permissions on device files. No matter what the permissions on the device files, a task in a devcg cgroup which isn't allowed write to chardev 4:64 will not be able to write to /dev/ttyS0.

The purpose is to prevent a root task from granting itself access to certain devices. Without this, the only option currently is to take CAP_MKNOD out of the capability bounding set for a container and make sure that /dev is set up right (and enforce nodev for mounts). In itself that doesn't sound so bad and it was my preference at first,

but the argument is that things like udev should be able to run in a container, and will object about not being able to create devices.

Quoted 6 lines

Quoted 6 lines

Tasks still can call mknod to create device files, regardless of whether the particular device is visible or accessible, but they may not be able to open it later.

This one hides under CONFIG_CGROUP_DEVS option.

Quoted 7 lines

Quoted 5 lines

To play with it - run a standard procedure:

 # mount -t container none /cont/devs -o devices
 # mkdir /cont/devs/0
 # echo -n $$ > /cont/devs/0/tasks

What is /cont/ for?

cgroups used to be called containers, so 'cont' is presumably shorthand for container.

Quoted 5 lines

Quoted 1 line

and tune device permissions.

How is this done?

Why would the kernel care about this stuff?

Because there is no way for userspace to restrict a root process in a container from accessing whatever devices it wants.

Quoted 15 lines · Serge E. Hallyn

Quoted 14 lines · Greg KH

Quoted 8 lines · Pavel Emelyanov

Changes from v3:
* Ported on 2.6.25-rc3-mm1;
* Re-splitted into smaller pieces;
* Added more comments to tricky places.

This controller allows to tune the devices accessibility by tasks, i.e. grant full access for /dev/null, /dev/zero etc, grant read-only access to IDE devices and completely hide SCSI disks.

From within the kernel itself? The kernel should not be keeping track
of the mode of devices, that's what the filesystem holding /dev is for. Those modes change all the time depending on the device plugged in, and the user using the "console". Why should the kernel need to worry about any of this?

Quoted 4 lines

These are distinct from the permissions on device files. No matter what the permissions on the device files, a task in a devcg cgroup which isn't allowed write to chardev 4:64 will not be able to write to /dev/ttyS0.

Then why not do that from userspace with a different /dev, or with a LSM?

Quoted 5 lines

The purpose is to prevent a root task from granting itself access to certain devices. Without this, the only option currently is to take CAP_MKNOD out of the capability bounding set for a container and make sure that /dev is set up right (and enforce nodev for mounts). In itself that doesn't sound so bad and it was my preference at first,

that would be my preference as well.

Quoted 2 lines

but the argument is that things like udev should be able to run in a container, and will object about not being able to create devices.

No reason you can't modify udev to do something like this. At the

worse, just disable udev warning messages, that is pretty trivial to do.

This really makes it seem like this kernel change is not needed at all.

Quoted 16 lines

Quoted 4 lines

Quoted 4 lines

Tasks still can call mknod to create device files, regardless of whether the particular device is visible or accessible, but they may not be able to open it later.

Quoted 8 lines

Quoted 7 lines

This one hides under CONFIG_CGROUP_DEVS option.

To play with it - run a standard procedure:

 # mount -t container none /cont/devs -o devices
 # mkdir /cont/devs/0
 # echo -n $$ > /cont/devs/0/tasks

What is /cont/ for?

cgroups used to be called containers, so 'cont' is presumably shorthand for container.

Quoted 1 line

Quoted 1 line

and tune device permissions.

Quoted 5 lines

Quoted 3 lines

How is this done?

Why would the kernel care about this stuff?

Because there is no way for userspace to restrict a root process in a container from accessing whatever devices it wants.

LSM does this quite well today, why reinvent the wheel and put wierd hooks in other parts of the kernel that do not need them at all?

thanks,

greg k-h

Quoted 12 lines · On Wed, 05 Mar 2008 20:37:40 +0300 Pavel Emelyanov <[email protected]> wrote:

include/linux/devscontrol.h new file +6 -0

new file mode 100644
index 0000000..04c168b
@@ -0,0 +1,12 @@
+#ifndef __DEVS_CONTROL_H__
+#define __DEVS_CONTROL_H__
+static inline struct kobj_map *task_cdev_map(struct task_struct *tsk)
+{
+	return NULL;
+}

include/linux/devscontrol.h line 12

Quoted 6 lines

include/linux/devscontrol.h new file +6 -0

+
+static inline struct kobj_map *task_bdev_map(struct task_struct *tsk)
+{
+	return NULL;
+}
+#endif

This doesn't include sufficient headers to be compileable.

I'm sure there are lots of headers like this. But we regularly need to fix them.

Quoted 21 lines · Greg KH

Quoted 19 lines · Serge E. Hallyn

Quoted 6 lines · Greg KH

Quoted 5 lines · Pavel Emelyanov

Changes from v3:
* Ported on 2.6.25-rc3-mm1;
* Re-splitted into smaller pieces;
* Added more comments to tricky places.

Quoted 8 lines

Quoted 3 lines

This controller allows to tune the devices accessibility by tasks, i.e. grant full access for /dev/null, /dev/zero etc, grant read-only access to IDE devices and completely hide SCSI disks.

From within the kernel itself? The kernel should not be keeping track
of the mode of devices, that's what the filesystem holding /dev is for. Those modes change all the time depending on the device plugged in, and the user using the "console". Why should the kernel need to worry about any of this?

These are distinct from the permissions on device files. No matter what the permissions on the device files, a task in a devcg cgroup which isn't allowed write to chardev 4:64 will not be able to write to /dev/ttyS0.

Quoted 2 lines

Then why not do that from userspace with a different /dev, or with a LSM?

Different dev is not suitable, since task may still call mknod to create device it needs and use it. This is not about comfortable use, this is about security.

LSM approach was proposed, but that required some API to configure the permissions.

This API done via control groups, so there were no difference between this approach and that. Except for this one doesn't create one more level of filtering at the top of kobject lookup and thus is simpler and faster.

Quoted 7 lines

Quoted 5 lines

The purpose is to prevent a root task from granting itself access to certain devices. Without this, the only option currently is to take CAP_MKNOD out of the capability bounding set for a container and make sure that /dev is set up right (and enforce nodev for mounts). In itself that doesn't sound so bad and it was my preference at first,

that would be my preference as well.

That approach relies on a proper user space setup inside a container, but this creates security holes, since container user may ignore all these "requirements".

Quoted 4 lines

Quoted 2 lines

but the argument is that things like udev should be able to run in a container, and will object about not being able to create devices.

No reason you can't modify udev to do something like this. At the

Sure we _can_ modify udev, but the problem is that users of virtualisation solutions often (very often) use old software (e.g. set up some out-dated distribution inside a container), so trick with modified udev simply won't work in many cases.

Quoted 21 lines · Greg KH

Quoted 19 lines · Serge E. Hallyn

Quoted 6 lines · Greg KH

Quoted 5 lines · Pavel Emelyanov

Changes from v3:
* Ported on 2.6.25-rc3-mm1;
* Re-splitted into smaller pieces;
* Added more comments to tricky places.

Quoted 8 lines

Quoted 3 lines

This controller allows to tune the devices accessibility by tasks, i.e. grant full access for /dev/null, /dev/zero etc, grant read-only access to IDE devices and completely hide SCSI disks.

From within the kernel itself? The kernel should not be keeping track
of the mode of devices, that's what the filesystem holding /dev is for. Those modes change all the time depending on the device plugged in, and the user using the "console". Why should the kernel need to worry about any of this?

These are distinct from the permissions on device files. No matter what the permissions on the device files, a task in a devcg cgroup which isn't allowed write to chardev 4:64 will not be able to write to /dev/ttyS0.

Quoted 2 lines

Then why not do that from userspace with a different /dev, or with a LSM?

Different dev is not suitable, since task may still call mknod to create device it needs and use it. This is not about comfortable use, this is about security.

Quoted 21 lines · Pavel Emelyanov

Quoted 20 lines · Greg KH

Quoted 19 lines · Serge E. Hallyn

Quoted 6 lines · Greg KH

Quoted 5 lines · Pavel Emelyanov

Changes from v3:
* Ported on 2.6.25-rc3-mm1;
* Re-splitted into smaller pieces;
* Added more comments to tricky places.

Quoted 8 lines

Quoted 3 lines

This controller allows to tune the devices accessibility by tasks, i.e. grant full access for /dev/null, /dev/zero etc, grant read-only access to IDE devices and completely hide SCSI disks.

From within the kernel itself? The kernel should not be keeping track
of the mode of devices, that's what the filesystem holding /dev is for. Those modes change all the time depending on the device plugged in, and the user using the "console". Why should the kernel need to worry about any of this?

These are distinct from the permissions on device files. No matter what the permissions on the device files, a task in a devcg cgroup which isn't allowed write to chardev 4:64 will not be able to write to /dev/ttyS0.

Quoted 5 lines

Quoted 2 lines

Then why not do that from userspace with a different /dev, or with a LSM?

Different dev is not suitable, since task may still call mknod to create device it needs and use it. This is not about comfortable use, this is about security.

And you may still take out mknod capability...

Quoted 8 lines · On Wed, 05 Mar 2008 20:37:40 +0300 Pavel Emelyanov <[email protected]> wrote:

include/linux/devscontrol.h new file +2 -0

new file mode 100644
index 0000000..04c168b
@@ -0,0 +1,12 @@
+#ifndef __DEVS_CONTROL_H__
+#define __DEVS_CONTROL_H__

include/linux/devscontrol.h line 12

Quoted 10 lines

include/linux/devscontrol.h new file +10 -0

+static inline struct kobj_map *task_cdev_map(struct task_struct *tsk)
+{
+	return NULL;
+}
+
+static inline struct kobj_map *task_bdev_map(struct task_struct *tsk)
+{
+	return NULL;
+}
+#endif

This doesn't include sufficient headers to be compileable.

I'm sure there are lots of headers like this. But we regularly need to fix them.

Quoted 5 lines

This doesn't include sufficient headers to be compileable.

I'm sure there are lots of headers like this. But we regularly need to fix them.

Not sure, whether this is still relevant after Greg's comments, but that's the -fix patch for this one. (It will cause a conflict with the 9th patch.)

Quoted 5 lines · On Fri, 07 Mar 2008 12:22:01 +0300 Pavel Emelyanov <[email protected]> wrote:

Quoted 5 lines

This doesn't include sufficient headers to be compileable.

I'm sure there are lots of headers like this. But we regularly need to fix them.

Quoted 2 lines

Not sure, whether this is still relevant after Greg's comments, but that's the -fix patch for this one. (It will cause a conflict with the 9th patch.)

Well. Where do we stand with this? afaict the state of play is:

Greg: do it in udev
Pavel: but people want to run old distros in containers

Realistically, when is the mainline kernel likely to have sufficient container functionality which is sufficiently well-tested for people to actually be able to do that? And how much longer will it take for that kernel.org functionality to propagate out into non-bleeding-edge distros?

Altogether we're looking at one to three years, aren't we? By then, perhaps a lot of "old" distros are already udev-based.

otoh, my experience upgrading old kernels to new udev has not been a good one (ie: it didn't work).

Quoted 10 lines · Andrew Morton

Quoted 7 lines · On Fri, 07 Mar 2008 12:22:01 +0300 Pavel Emelyanov <[email protected]> wrote:

Quoted 5 lines

This doesn't include sufficient headers to be compileable.

I'm sure there are lots of headers like this. But we regularly need to fix them.

Not sure, whether this is still relevant after Greg's comments, but that's the -fix patch for this one. (It will cause a conflict with the 9th patch.)

Quoted 4 lines

Well. Where do we stand with this? afaict the state of play is:

Greg: do it in udev
Pavel: but people want to run old distros in containers

Actually no.

Greg: Use LSM for this

Pavel: My approach just makes maps per-group, while LSM will
bring a new level of filtering/lookup on device open path

Quoted 4 lines

Realistically, when is the mainline kernel likely to have sufficient container functionality which is sufficiently well-tested for people to actually be able to do that? And how much longer will it take for that kernel.org functionality to propagate out into non-bleeding-edge distros?

The fact is that we have users of OpenVZ and even Virtuozzo, that still use redhat-9 as in containers. So even if this is ready in 5 years, there will always be someone who sets the outdated (by that time) fedora-core-8 and find out, that his udev refuses to work.

Quoted 10 lines · Pavel Emelyanov

Quoted 9 lines · Andrew Morton

Quoted 7 lines · On Fri, 07 Mar 2008 12:22:01 +0300 Pavel Emelyanov <[email protected]> wrote:

Quoted 5 lines

This doesn't include sufficient headers to be compileable.

I'm sure there are lots of headers like this. But we regularly need to fix them.

Not sure, whether this is still relevant after Greg's comments, but that's the -fix patch for this one. (It will cause a conflict with the 9th patch.)

Quoted 7 lines

Quoted 4 lines

Well. Where do we stand with this? afaict the state of play is:

Greg: do it in udev
Pavel: but people want to run old distros in containers

Actually no.

Greg: Use LSM for this

Yes, that is my recommendation.

Quoted 2 lines

Pavel: My approach just makes maps per-group, while LSM will
bring a new level of filtering/lookup on device open path

Huh? You are still doing that same "filtering/lookup" by modifying the maps code. The result should be exactly the same.

Why do you not want to use the LSM interface? That is exactly what it is there for, don't go creating new hooks into the kernel for the exact same functionality.

Opening a dev node is not on any "fast path" that you need to be concerned about a few extra calls within the kernel.

And, I think in the end your patch would be much smaller and easier to understand and review and maintain overall.

Quoted 8 lines

Quoted 4 lines

Realistically, when is the mainline kernel likely to have sufficient container functionality which is sufficiently well-tested for people to actually be able to do that? And how much longer will it take for that kernel.org functionality to propagate out into non-bleeding-edge distros?

The fact is that we have users of OpenVZ and even Virtuozzo, that still use redhat-9 as in containers. So even if this is ready in 5 years, there will always be someone who sets the outdated (by that time) fedora-core-8 and find out, that his udev refuses to work.

That's fine, use the LSM interface, no need to change userspace at all.

Although I think your requirement of using new kernels on very old distros is going to cause you more problems than you realize in the end...

thanks,

greg k-h

Quoted 19 lines · Greg KH

Quoted 10 lines · Pavel Emelyanov

Quoted 9 lines · Andrew Morton

Quoted 7 lines · On Fri, 07 Mar 2008 12:22:01 +0300 Pavel Emelyanov <[email protected]> wrote:

Quoted 5 lines

This doesn't include sufficient headers to be compileable.

I'm sure there are lots of headers like this. But we regularly need to fix them.

Not sure, whether this is still relevant after Greg's comments, but that's the -fix patch for this one. (It will cause a conflict with the 9th patch.)

Quoted 7 lines

Quoted 4 lines

Well. Where do we stand with this? afaict the state of play is:

Greg: do it in udev
Pavel: but people want to run old distros in containers

Actually no.

Greg: Use LSM for this

Quoted 7 lines

Yes, that is my recommendation.

Quoted 2 lines

Pavel: My approach just makes maps per-group, while LSM will
bring a new level of filtering/lookup on device open path

Huh? You are still doing that same "filtering/lookup" by modifying the maps code. The result should be exactly the same.

No - this lookup was there before to get struct kobject from the dev_t, I just make it look up in another map.

Quoted 3 lines

Why do you not want to use the LSM interface? That is exactly what it is there for, don't go creating new hooks into the kernel for the exact same functionality.

_No_new_hooks_ - just the map is get from task, not from a static variable.

I have basically three objections against LSM.

1. LSM is not stackable, so loading this tiny module with devices access rights will block other security modules;

2. Turning CONFIG_SECURITY on immediately causes all the other hooks to get called. This affects performance on critical paths, like process creation/destruction, network flow and so on. This impact is small, but noticeable;

3. With LSM turned on we'll have to "virtualize" it, i.e. make its work safe in a container. I don't presume to judge how much work will have to be done in this area, so the result patch would be even larger and maybe will duplicate functionality, which is currently in cgroups. OTOH, cgroups already provide the ways to correctly delegate proper rights to containers.

Quoted 5 lines

Opening a dev node is not on any "fast path" that you need to be concerned about a few extra calls within the kernel.

And, I think in the end your patch would be much smaller and easier to understand and review and maintain overall.

Hardly - the largest part of my patch is cgroup manipulations. The part that makes the char and block layers switch to new map ac check the permissions is 10-20 lines of new code.

But with LSM I will still need this API.

Again, I object to this as you are driving a new security policy infrastructure into the device node logic where it does not belong as we already have this functionality in the LSM interface today. Please use that one instead and don't clutter up the kernel with "one-off" security changes like this one.

Please try the LSM interface and see what happens. If, after you have created a patch, you still have objections, please post it for review and I will be glad to revisit my opinion at that time.

Quoted 28 lines · Pavel Emelyanov

Quoted 19 lines · Greg KH

Quoted 10 lines · Pavel Emelyanov

Quoted 9 lines · Andrew Morton

Quoted 7 lines · On Fri, 07 Mar 2008 12:22:01 +0300 Pavel Emelyanov <[email protected]> wrote:

Quoted 5 lines

This doesn't include sufficient headers to be compileable.

I'm sure there are lots of headers like this. But we regularly need to fix them.

Not sure, whether this is still relevant after Greg's comments, but that's the -fix patch for this one. (It will cause a conflict with the 9th patch.)

Quoted 7 lines

Quoted 4 lines

Well. Where do we stand with this? afaict the state of play is:

Greg: do it in udev
Pavel: but people want to run old distros in containers

Actually no.

Greg: Use LSM for this

Quoted 7 lines

Yes, that is my recommendation.

Quoted 2 lines

Pavel: My approach just makes maps per-group, while LSM will
bring a new level of filtering/lookup on device open path

Huh? You are still doing that same "filtering/lookup" by modifying the maps code. The result should be exactly the same.

Quoted 8 lines

No - this lookup was there before to get struct kobject from the dev_t, I just make it look up in another map.

Quoted 3 lines

Why do you not want to use the LSM interface? That is exactly what it is there for, don't go creating new hooks into the kernel for the exact same functionality.

_No_new_hooks_ - just the map is get from task, not from a static variable.

The new "hook" is that we now have ways of setting different maps, and you added a whole infrastructure to set these permissions from userspace, as well as track them within the kernel. That's a security policy that you are putting into place within the kernel, just like LSM is supposed to be used for.

Quoted 4 lines

I have basically three objections against LSM.

1. LSM is not stackable, so loading this tiny module with devices access rights will block other security modules;

Do you really want to run other LSMs within a containerd kernel? Is that a requirement? It would seem to run counter to the main goal of containers to me.

Quoted 4 lines

2. Turning CONFIG_SECURITY on immediately causes all the other hooks to get called. This affects performance on critical paths, like process creation/destruction, network flow and so on. This impact is small, but noticeable;

The last time this was brought up, it was not noticable, except for some network paths. And even then, the number was lost in the noise from what I saw. I think with a containered machine, you have bigger things to be worried about :)

Quoted 6 lines

3. With LSM turned on we'll have to "virtualize" it, i.e. make its work safe in a container. I don't presume to judge how much work will have to be done in this area, so the result patch would be even larger and maybe will duplicate functionality, which is currently in cgroups. OTOH, cgroups already provide the ways to correctly delegate proper rights to containers.

No, your lsm would be your "virtualize" policy. I don't think you would have to do any additional work here, but could be wrong. Would like to see the code to prove it.

Quoted 11 lines

Quoted 5 lines

Opening a dev node is not on any "fast path" that you need to be concerned about a few extra calls within the kernel.

And, I think in the end your patch would be much smaller and easier to understand and review and maintain overall.

Hardly - the largest part of my patch is cgroup manipulations. The part that makes the char and block layers switch to new map ac check the permissions is 10-20 lines of new code.

But with LSM I will still need this API.

Yes, but your LSM hooks will be smaller than the code modifications to the map logic :)

Again, I object to this as you are driving a new security policy infrastructure into the device node logic where it does not belong as we already have this functionality in the LSM interface today. Please use that one instead and don't clutter up the kernel with "one-off" security changes like this one.

Please try the LSM interface and see what happens. If, after you have

created a patch, you still have objections, please post it for review and I will be glad to revisit my opinion at that time.

thanks,

greg k-h

...

Until user namespaces are complete, selinux seems the only good solution to offer isolation.

Quoted 3 lines

Do you really want to run other LSMs within a containerd kernel? Is that a requirement? It would seem to run counter to the main goal of containers to me.

Until user namespaces are complete, selinux seems the only good solution to offer isolation.

Quoted 42 lines

Quoted 4 lines

2. Turning CONFIG_SECURITY on immediately causes all the other hooks to get called. This affects performance on critical paths, like process creation/destruction, network flow and so on. This impact is small, but noticeable;

The last time this was brought up, it was not noticable, except for some network paths. And even then, the number was lost in the noise from what I saw. I think with a containered machine, you have bigger things to be worried about :)

Quoted 6 lines

3. With LSM turned on we'll have to "virtualize" it, i.e. make its work safe in a container. I don't presume to judge how much work will have to be done in this area, so the result patch would be even larger and maybe will duplicate functionality, which is currently in cgroups. OTOH, cgroups already provide the ways to correctly delegate proper rights to containers.

No, your lsm would be your "virtualize" policy. I don't think you would have to do any additional work here, but could be wrong. Would like to see the code to prove it.

Quoted 11 lines

Quoted 5 lines

Opening a dev node is not on any "fast path" that you need to be concerned about a few extra calls within the kernel.

And, I think in the end your patch would be much smaller and easier to understand and review and maintain overall.

Hardly - the largest part of my patch is cgroup manipulations. The part that makes the char and block layers switch to new map ac check the permissions is 10-20 lines of new code.

But with LSM I will still need this API.

Yes, but your LSM hooks will be smaller than the code modifications to the map logic :)

Again, I object to this as you are driving a new security policy infrastructure into the device node logic where it does not belong as we already have this functionality in the LSM interface today. Please use that one instead and don't clutter up the kernel with "one-off" security changes like this one.

Quoted 1 line

Please try the LSM interface and see what happens. If, after you have

https://lists.linux-foundation.org/pipermail/containers/2007-November/008589.html

This was just a proof of concept for discussion.

Our conclusion (including my own) was that it would be nicer to have the controls not be shoed in using lsm but rather at the level Pavel was doing.

Quoted 4 lines · --- "Serge E. Hallyn

...

Until user namespaces are complete, selinux seems the only good solution to offer isolation.

Smack does it better and cheaper. (Unless you define good==selinux) (insert smiley)

Quoted 6 lines · Serge E. Hallyn

Quoted 3 lines

Do you really want to run other LSMs within a containerd kernel? Is that a requirement? It would seem to run counter to the main goal of containers to me.

Until user namespaces are complete, selinux seems the only good solution to offer isolation.

Great, use that instead :)

Quoted 42 lines

Quoted 42 lines

Quoted 4 lines

2. Turning CONFIG_SECURITY on immediately causes all the other hooks to get called. This affects performance on critical paths, like process creation/destruction, network flow and so on. This impact is small, but noticeable;

The last time this was brought up, it was not noticable, except for some network paths. And even then, the number was lost in the noise from what I saw. I think with a containered machine, you have bigger things to be worried about :)

Quoted 6 lines

3. With LSM turned on we'll have to "virtualize" it, i.e. make its work safe in a container. I don't presume to judge how much work will have to be done in this area, so the result patch would be even larger and maybe will duplicate functionality, which is currently in cgroups. OTOH, cgroups already provide the ways to correctly delegate proper rights to containers.

No, your lsm would be your "virtualize" policy. I don't think you would have to do any additional work here, but could be wrong. Would like to see the code to prove it.

Quoted 11 lines

Quoted 5 lines

Opening a dev node is not on any "fast path" that you need to be concerned about a few extra calls within the kernel.

And, I think in the end your patch would be much smaller and easier to understand and review and maintain overall.

Hardly - the largest part of my patch is cgroup manipulations. The part that makes the char and block layers switch to new map ac check the permissions is 10-20 lines of new code.

But with LSM I will still need this API.

Yes, but your LSM hooks will be smaller than the code modifications to the map logic :)

Again, I object to this as you are driving a new security policy infrastructure into the device node logic where it does not belong as we already have this functionality in the LSM interface today. Please use that one instead and don't clutter up the kernel with "one-off" security changes like this one.

Quoted 5 lines

Quoted 1 line

Please try the LSM interface and see what happens. If, after you have

https://lists.linux-foundation.org/pipermail/containers/2007-November/008589.html

This was just a proof of concept for discussion.

I don't see the LSM patch in that posting, only a Makefile change that adds it to the build.

Quoted 3 lines

Our conclusion (including my own) was that it would be nicer to have the controls not be shoed in using lsm but rather at the level Pavel was doing.

Why? What is the difference? I don't see that discussion anywhere in that post.

Why add add-hock security stuff to the kernel all over the place and not use the LSM interface that we already have? If LSM doesn't provide the exact right hooks needed, we can always change it (and no, we should not be adding kobj_maps hooks to LSM).

thanks,

greg k-h

From 4266131c40b629e3b04c0d9d01569a95fa967e3e Mon Sep 17 00:00:00 2001

Quoted 5 lines · Casey Schaufler

Quoted 2 lines · --- "Serge E. Hallyn

...

Quoted 5 lines

Quoted 2 lines

Until user namespaces are complete, selinux seems the only good solution to offer isolation.

Smack does it better and cheaper. (Unless you define good==selinux) (insert smiley)

Ah, thanks - I hadn't looked into it, but yes IIUC smack should definately work. I'll have to give that a shot.

(A basic selinux policy module to isolate a container was pretty simple, but providing finer-grained intra-container access seems to take some changes to the base refpolicy. I've been waiting a few weeks to find time to work on that.)

Quoted 7 lines · Serge E. Hallyn

Quoted 6 lines · Casey Schaufler

Quoted 3 lines · --- "Serge E. Hallyn

...

Until user namespaces are complete, selinux seems the only good

solution

Quoted 7 lines

Quoted 4 lines

Quoted 1 line

to offer isolation.

Smack does it better and cheaper. (Unless you define good==selinux) (insert smiley)

Ah, thanks - I hadn't looked into it, but yes IIUC smack should definately work. I'll have to give that a shot.

Not if you want to confine uid 0. smack doesn't control capabilities, even the ones used to override it.

So you'd have to at least configure your per-process bset and file caps rather carefully. And even then you have to watch out for things with CAP_MAC* or CAP_SETPCAP.

Quoted 21 lines · --- Stephen Smalley

Quoted 7 lines · Serge E. Hallyn

Quoted 6 lines · Casey Schaufler

Quoted 3 lines · --- "Serge E. Hallyn

...

Until user namespaces are complete, selinux seems the only good

solution

Quoted 7 lines

Quoted 4 lines

Quoted 1 line

to offer isolation.

Smack does it better and cheaper. (Unless you define good==selinux) (insert smiley)

Ah, thanks - I hadn't looked into it, but yes IIUC smack should definately work. I'll have to give that a shot.

Not if you want to confine uid 0. smack doesn't control capabilities, even the ones used to override it.

Quoted 3 lines

So you'd have to at least configure your per-process bset and file caps rather carefully. And even then you have to watch out for things with CAP_MAC* or CAP_SETPCAP.

Shrug. As if getting 800,000 lines of policy definition for a thousand applications completely correct is going to be easier.

Quoted 5 lines · Greg KH

Quoted 4 lines · Serge E. Hallyn

Quoted 3 lines

Do you really want to run other LSMs within a containerd kernel? Is that a requirement? It would seem to run counter to the main goal of containers to me.

Quoted 4 lines

Quoted 2 lines

Until user namespaces are complete, selinux seems the only good solution to offer isolation.

Great, use that instead :)

That can't work as is since you can't specify major:minor in policy.

So all we could do again is simply refuse all mknod, which we can already do with per-process capability bounding sets.

Quoted 6 lines · Serge E. Hallyn

Quoted 5 lines · Greg KH

Quoted 4 lines · Serge E. Hallyn

Quoted 3 lines

Do you really want to run other LSMs within a containerd kernel? Is that a requirement? It would seem to run counter to the main goal of containers to me.

Quoted 6 lines

Quoted 4 lines

Quoted 2 lines

Until user namespaces are complete, selinux seems the only good solution to offer isolation.

Great, use that instead :)

That can't work as is since you can't specify major:minor in policy.

Your LSM can not, or the LSM interface does not allow this to happen?

Quoted 2 lines

So all we could do again is simply refuse all mknod, which we can already do with per-process capability bounding sets.

I thought we passed that info down to the LSM module, can't you do your selection at that point in time?

And then, just mediate open() like always, right?

Quoted 10 lines · Greg KH

Quoted 9 lines · Serge E. Hallyn

Quoted 8 lines · Greg KH

Quoted 6 lines · Serge E. Hallyn

Quoted 3 lines

Do you really want to run other LSMs within a containerd kernel? Is that a requirement? It would seem to run counter to the main goal of containers to me.

Until user namespaces are complete, selinux seems the only good solution to offer isolation.

Quoted 5 lines

Quoted 3 lines

Quoted 1 line

Great, use that instead :)

That can't work as is since you can't specify major:minor in policy.

Your LSM can not, or the LSM interface does not allow this to happen?

No my lsm in fact does, you just can't do it with selinux policy at the moment. I was still responding to your "just use selinux" :)

Quoted 7 lines

Quoted 2 lines

So all we could do again is simply refuse all mknod, which we can already do with per-process capability bounding sets.

I thought we passed that info down to the LSM module, can't you do your selection at that point in time?

And then, just mediate open() like always, right?

Yup, the patch I included inline does that.

An LSM can address the problem. It just felt like more of a patch-over-the-real-problem kind of solution.

Quoted 11 lines · Serge E. Hallyn

Quoted 10 lines · Greg KH

Quoted 9 lines · Serge E. Hallyn

Quoted 8 lines · Greg KH

Quoted 6 lines · Serge E. Hallyn

Quoted 3 lines

Do you really want to run other LSMs within a containerd kernel? Is that a requirement? It would seem to run counter to the main goal of containers to me.

Until user namespaces are complete, selinux seems the only good solution to offer isolation.

Quoted 8 lines

Quoted 5 lines

Quoted 3 lines

Quoted 1 line

Great, use that instead :)

That can't work as is since you can't specify major:minor in policy.

Your LSM can not, or the LSM interface does not allow this to happen?

No my lsm in fact does, you just can't do it with selinux policy at the moment. I was still responding to your "just use selinux" :)

I never said "use selinux", do you think I am crazy? :)

Just use your own lsm, that's all I recommended.

Quoted 9 lines

Quoted 7 lines

Quoted 2 lines

So all we could do again is simply refuse all mknod, which we can already do with per-process capability bounding sets.

I thought we passed that info down to the LSM module, can't you do your selection at that point in time?

And then, just mediate open() like always, right?

Yup, the patch I included inline does that.

Great. But don't put that other file in the core kernel, put it in security/ please.

Quoted 2 lines

An LSM can address the problem. It just felt like more of a patch-over-the-real-problem kind of solution.

I disagree, it sounds exactly like what LSM is for.

From ff9f9a190d8664275fb86c7d5b2153fd79b990d7 Mon Sep 17 00:00:00 2001

Quoted 4 lines

I have basically three objections against LSM.

1. LSM is not stackable, so loading this tiny module with devices access rights will block other security modules;

Do you really want to run other LSMs within a containerd kernel? Is that a requirement? It would seem to run counter to the main goal of containers to me.

Quoted 4 lines

2. Turning CONFIG_SECURITY on immediately causes all the other hooks to get called. This affects performance on critical paths, like process creation/destruction, network flow and so on. This impact is small, but noticeable;

The last time this was brought up, it was not noticable, except for some network paths. And even then, the number was lost in the noise from what I saw. I think with a containered machine, you have bigger things to be worried about :)

Quoted 6 lines

3. With LSM turned on we'll have to "virtualize" it, i.e. make its work safe in a container. I don't presume to judge how much work will have to be done in this area, so the result patch would be even larger and maybe will duplicate functionality, which is currently in cgroups. OTOH, cgroups already provide the ways to correctly delegate proper rights to containers.

No, your lsm would be your "virtualize" policy. I don't think you would have to do any additional work here, but could be wrong. Would like to see the code to prove it.

Quoted 10 lines

Quoted 5 lines

Opening a dev node is not on any "fast path" that you need to be concerned about a few extra calls within the kernel.

And, I think in the end your patch would be much smaller and easier to understand and review and maintain overall.

Hardly - the largest part of my patch is cgroup manipulations. The part that makes the char and block layers switch to new map ac check the permissions is 10-20 lines of new code.

But with LSM I will still need this API.

Yes, but your LSM hooks will be smaller than the code modifications to the map logic :)

Again, I object to this as you are driving a new security policy infrastructure into the device node logic where it does not belong as we already have this functionality in the LSM interface today. Please use that one instead and don't clutter up the kernel with "one-off" security changes like this one.

Please try the LSM interface and see what happens. If, after you have created a patch, you still have objections, please post it for review and I will be glad to revisit my opinion at that time.

thanks,

greg k-h

Besides, I've measured some things - the lat_syscall test for open from lmbench test suite and the nptl perf test. Here are the results:

sec nosec

open    3.0980s  3.0709s
nptl    2.7746s  2.7710s

So we have 0.88% loss in open and ~0.15% with nptl. I know, this is not that much, but it is noticeable. Besides, this is only two tests, digging deeper may reveal more.

Let alone the fact that simply turning the CONFIG_SECURITY to 'y' puts +8Kb to the vmlinux...

I think, I finally agree with you and Al Viro, that the kobj mapper is not the right place to put the filtering, but taking the above numbers into account, can we put the "hooks" into the #else /* CONFIG_SECURITY */ versions of security_inode_permission/security_file_permission/etc?

Quoted 7 lines · Pavel Emelyanov

Besides, I've measured some things - the lat_syscall test for open from lmbench test suite and the nptl perf test. Here are the results:

sec nosec

open    3.0980s  3.0709s
nptl    2.7746s  2.7710s

Quoted 3 lines

So we have 0.88% loss in open and ~0.15% with nptl. I know, this is not that much, but it is noticeable. Besides, this is only two tests, digging deeper may reveal more.

I think that is in the noise of sampling if you run that test many more times.

Quoted 7 lines

Let alone the fact that simply turning the CONFIG_SECURITY to 'y' puts +8Kb to the vmlinux...

I think, I finally agree with you and Al Viro, that the kobj mapper is not the right place to put the filtering, but taking the above numbers into account, can we put the "hooks" into the #else /* CONFIG_SECURITY */ versions of security_inode_permission/security_file_permission/etc?

Ask the security module interface maintainers about this, not me :)

good luck,

greg k-h

Quoted 8 lines · Greg KH

Quoted 7 lines · Pavel Emelyanov

Besides, I've measured some things - the lat_syscall test for open from lmbench test suite and the nptl perf test. Here are the results:

sec nosec

open    3.0980s  3.0709s
nptl    2.7746s  2.7710s

Quoted 6 lines

Quoted 3 lines

So we have 0.88% loss in open and ~0.15% with nptl. I know, this is not that much, but it is noticeable. Besides, this is only two tests, digging deeper may reveal more.

I think that is in the noise of sampling if you run that test many more times.

These numbers are average values of 20 runs of each test. I didn't provide the measurement accuracy, but the abs(open.sec - open.nosec) is greater than it.

Quoted 8 lines

Quoted 7 lines

Let alone the fact that simply turning the CONFIG_SECURITY to 'y' puts +8Kb to the vmlinux...

I think, I finally agree with you and Al Viro, that the kobj mapper is not the right place to put the filtering, but taking the above numbers into account, can we put the "hooks" into the #else /* CONFIG_SECURITY */ versions of security_inode_permission/security_file_permission/etc?

Quoted 1 line

Ask the security module interface maintainers about this, not me :)

OK :) Thanks for your time, Greg.

So, Serge, since you already have a LSM-based version, maybe you can change it with the proposed "fix" and send it to LSM maintainers for review?

Quoted 4 lines

good luck,

greg k-h

Quoted 28 lines · Pavel Emelyanov

Quoted 8 lines · Greg KH

Quoted 7 lines · Pavel Emelyanov

Besides, I've measured some things - the lat_syscall test for open from lmbench test suite and the nptl perf test. Here are the results:

sec nosec

open    3.0980s  3.0709s
nptl    2.7746s  2.7710s

Quoted 6 lines

Quoted 3 lines

So we have 0.88% loss in open and ~0.15% with nptl. I know, this is not that much, but it is noticeable. Besides, this is only two tests, digging deeper may reveal more.

I think that is in the noise of sampling if you run that test many more times.

These numbers are average values of 20 runs of each test. I didn't provide the measurement accuracy, but the abs(open.sec - open.nosec) is greater than it.

Quoted 8 lines

Quoted 7 lines

Let alone the fact that simply turning the CONFIG_SECURITY to 'y' puts +8Kb to the vmlinux...

I think, I finally agree with you and Al Viro, that the kobj mapper is not the right place to put the filtering, but taking the above numbers into account, can we put the "hooks" into the #else /* CONFIG_SECURITY */ versions of security_inode_permission/security_file_permission/etc?

Quoted 7 lines

Quoted 1 line

Ask the security module interface maintainers about this, not me :)

OK :) Thanks for your time, Greg.

So, Serge, since you already have a LSM-based version, maybe you can change it with the proposed "fix" and send it to LSM maintainers for review?

To take the point of view of someone who neither wants containers nor LSM but just a fast box,

you're asking me to introduce LSM hooks for the !SECURITY case? :)

I can give it a shot, but I expect some complaints. Now at least the _mknod hook shouldn't be a hotpath, and I suppose I can add yet an #ifdef inside the !SECURITY version of security_inode_permission(). I still expect some complaints though. I'll send something soon.

Quoted 40 lines · Serge E. Hallyn

Quoted 28 lines · Pavel Emelyanov

Quoted 8 lines · Greg KH

Quoted 7 lines · Pavel Emelyanov

Besides, I've measured some things - the lat_syscall test for open from lmbench test suite and the nptl perf test. Here are the results:

sec nosec

open    3.0980s  3.0709s
nptl    2.7746s  2.7710s

Quoted 6 lines

Quoted 3 lines

So we have 0.88% loss in open and ~0.15% with nptl. I know, this is not that much, but it is noticeable. Besides, this is only two tests, digging deeper may reveal more.

I think that is in the noise of sampling if you run that test many more times.

These numbers are average values of 20 runs of each test. I didn't provide the measurement accuracy, but the abs(open.sec - open.nosec) is greater than it.

Quoted 8 lines

Quoted 7 lines

Let alone the fact that simply turning the CONFIG_SECURITY to 'y' puts +8Kb to the vmlinux...

I think, I finally agree with you and Al Viro, that the kobj mapper is not the right place to put the filtering, but taking the above numbers into account, can we put the "hooks" into the #else /* CONFIG_SECURITY */ versions of security_inode_permission/security_file_permission/etc?

Quoted 7 lines

Quoted 1 line

Ask the security module interface maintainers about this, not me :)

OK :) Thanks for your time, Greg.

So, Serge, since you already have a LSM-based version, maybe you can change it with the proposed "fix" and send it to LSM maintainers for review?

To take the point of view of someone who neither wants containers nor LSM but just a fast box,

Quoted 6 lines

you're asking me to introduce LSM hooks for the !SECURITY case? :)

I can give it a shot, but I expect some complaints. Now at least the _mknod hook shouldn't be a hotpath, and I suppose I can add yet an #ifdef inside the !SECURITY version of security_inode_permission(). I still expect some complaints though. I'll send something soon.

Not sure I'm following the plot here, but please don't do anything that will prohibit the use of containers/namespaces with security modules like SELinux/Smack. Yes, that's a legitimate use case, and there will be people who will want to do that - they serve different but complementary purposes (containers are _not_ a substitute for MAC). We don't want them to be exclusive of one another.

Quoted 28 lines · Pavel Emelyanov

Quoted 8 lines · Greg KH

Quoted 7 lines · Pavel Emelyanov

Besides, I've measured some things - the lat_syscall test for open from lmbench test suite and the nptl perf test. Here are the results:

sec nosec

open    3.0980s  3.0709s
nptl    2.7746s  2.7710s

Quoted 5 lines

Quoted 3 lines

So we have 0.88% loss in open and ~0.15% with nptl. I know, this is not that much, but it is noticeable. Besides, this is only two tests, digging deeper may reveal more.

I think that is in the noise of sampling if you run that test many more times.

These numbers are average values of 20 runs of each test. I didn't provide the measurement accuracy, but the abs(open.sec - open.nosec) is greater than it.

Quoted 8 lines

Quoted 7 lines

Let alone the fact that simply turning the CONFIG_SECURITY to 'y' puts +8Kb to the vmlinux...

I think, I finally agree with you and Al Viro, that the kobj mapper is not the right place to put the filtering, but taking the above numbers into account, can we put the "hooks" into the #else /* CONFIG_SECURITY */ versions of security_inode_permission/security_file_permission/etc?

Ask the security module interface maintainers about this, not me :)

OK :) Thanks for your time, Greg.

Quoted 3 lines

So, Serge, since you already have a LSM-based version, maybe you can change it with the proposed "fix" and send it to LSM maintainers for review?

To take the point of view of someone who neither wants containers nor LSM but just a fast box,

you're asking me to introduce LSM hooks for the !SECURITY case? :)

I can give it a shot, but I expect some complaints. Now at least the _mknod hook shouldn't be a hotpath, and I suppose I can add yet an #ifdef inside the !SECURITY version of security_inode_permission(). I still expect some complaints though. I'll send something soon.

thanks,
-serge

Quoted 4 lines

Quoted 4 lines

good luck,

greg k-h

Quoted 42 lines · Serge E. Hallyn

Quoted 28 lines · Pavel Emelyanov

Quoted 8 lines · Greg KH

Quoted 7 lines · Pavel Emelyanov

Besides, I've measured some things - the lat_syscall test for open from lmbench test suite and the nptl perf test. Here are the results:

sec nosec

open    3.0980s  3.0709s
nptl    2.7746s  2.7710s

Quoted 6 lines

Quoted 3 lines

So we have 0.88% loss in open and ~0.15% with nptl. I know, this is not that much, but it is noticeable. Besides, this is only two tests, digging deeper may reveal more.

I think that is in the noise of sampling if you run that test many more times.

These numbers are average values of 20 runs of each test. I didn't provide the measurement accuracy, but the abs(open.sec - open.nosec) is greater than it.

Quoted 8 lines

Quoted 7 lines

Let alone the fact that simply turning the CONFIG_SECURITY to 'y' puts +8Kb to the vmlinux...

I think, I finally agree with you and Al Viro, that the kobj mapper is not the right place to put the filtering, but taking the above numbers into account, can we put the "hooks" into the #else /* CONFIG_SECURITY */ versions of security_inode_permission/security_file_permission/etc?

Quoted 7 lines

Quoted 1 line

Ask the security module interface maintainers about this, not me :)

OK :) Thanks for your time, Greg.

So, Serge, since you already have a LSM-based version, maybe you can change it with the proposed "fix" and send it to LSM maintainers for review?

To take the point of view of someone who neither wants containers nor LSM but just a fast box,

you're asking me to introduce LSM hooks for the !SECURITY case? :)

Quoted 4 lines

I can give it a shot, but I expect some complaints. Now at least the _mknod hook shouldn't be a hotpath, and I suppose I can add yet an #ifdef inside the !SECURITY version of security_inode_permission(). I still expect some complaints though. I'll send something soon.

Not sure I'm following the plot here, but please don't do anything that will prohibit the use of containers/namespaces with security modules like SELinux/Smack.

Yes, that's a legitimate use case, and there will
be people who will want to do that - they serve different but complementary purposes (containers are _not_ a substitute for MAC). We don't want them to be exclusive of one another.

Quoted 48 lines · Stephen Smalley

Quoted 40 lines · Serge E. Hallyn

Quoted 28 lines · Pavel Emelyanov

Quoted 8 lines · Greg KH

Quoted 7 lines · Pavel Emelyanov

Besides, I've measured some things - the lat_syscall test for open from lmbench test suite and the nptl perf test. Here are the results:

sec nosec

open    3.0980s  3.0709s
nptl    2.7746s  2.7710s

Quoted 6 lines

Quoted 3 lines

So we have 0.88% loss in open and ~0.15% with nptl. I know, this is not that much, but it is noticeable. Besides, this is only two tests, digging deeper may reveal more.

I think that is in the noise of sampling if you run that test many more times.

These numbers are average values of 20 runs of each test. I didn't provide the measurement accuracy, but the abs(open.sec - open.nosec) is greater than it.

Quoted 8 lines

Quoted 7 lines

Let alone the fact that simply turning the CONFIG_SECURITY to 'y' puts +8Kb to the vmlinux...

I think, I finally agree with you and Al Viro, that the kobj mapper is not the right place to put the filtering, but taking the above numbers into account, can we put the "hooks" into the #else /* CONFIG_SECURITY */ versions of security_inode_permission/security_file_permission/etc?

Quoted 7 lines

Quoted 1 line

Ask the security module interface maintainers about this, not me :)

OK :) Thanks for your time, Greg.

So, Serge, since you already have a LSM-based version, maybe you can change it with the proposed "fix" and send it to LSM maintainers for review?

To take the point of view of someone who neither wants containers nor LSM but just a fast box,

Quoted 6 lines

you're asking me to introduce LSM hooks for the !SECURITY case? :)

I can give it a shot, but I expect some complaints. Now at least the _mknod hook shouldn't be a hotpath, and I suppose I can add yet an #ifdef inside the !SECURITY version of security_inode_permission(). I still expect some complaints though. I'll send something soon.

Quoted 6 lines

Not sure I'm following the plot here, but please don't do anything that will prohibit the use of containers/namespaces with security modules like SELinux/Smack. Yes, that's a legitimate use case, and there will be people who will want to do that - they serve different but complementary purposes (containers are _not_ a substitute for MAC). We don't want them to be exclusive of one another.

Also, note that "real" device labeling and access control (i.e. bind a label to a device in the kernel irrespective of what device node is used to access it so that a process that can create any device nodes at all can't effectively bypass all device access controls just by creating an arbitrary node to any device in a type accessible to it) is already called out on our kernel todo list for SELinux, and contributions there would be welcome.

...

Not sure I'm following the plot here, but please don't do anything that will prohibit the use of containers/namespaces with security modules like SELinux/Smack. Yes, that's a legitimate use case, and there will be people who will want to do that - they serve different but complementary purposes (containers are _not_ a substitute for MAC). We don't want them to be exclusive of one another.