Skip to contentSkip to conversation

Special Kernel Modification

DISCUSSION Lonnie Cumberland 11 participants 21 messages Nov 5, 2001 – Nov 6, 2001

lore
Share & tools
lore
#0 lore raw

Hello All,

I am new to this list and am very interested in any help that you might be able to give me for a particular Linux project that I am working on.

I am currently using a Pentium III 500Mhz running Mandrake 8.0 Linux and Kernel 2.4.3 as a development system.

The basic problem that I am running into is that I am working on an Internet related project and thus need to ensure various types of document security for the eventual users of this system, if things go well.

I have look into using things like "chroot" to restrict the users for this very special server, but that solution is not what we need.

I am building a special xserver that will allow users to login and then the xserver will run a single application such as StarOffice. When the user exits from the application then the Xserver will log them out.

My problem is that I need to find a way to prevent the user from navigating out of their home directories.

I have also looking the possiblility of writing my own filesystem, but I am told that this needs to be done at the VFS level.

Is there someone who might be able to give me some information on how I could add a few lines to the VFS filesystem so that I might set some type of extended attribute to prevent users from navigating out of the locations.

Any help would be greatly appreciated,
Lonnie Cumberland

#1 lore raw
Quoted 2 lines

I have look into using things like "chroot" to restrict the users for this very special server, but that solution is not what we need.

It sounds like it is to me

Quoted 2 lines

My problem is that I need to find a way to prevent the user from navigating out of their home directories.

Then you must put al the files in their home directories. Alternatively with later 2.4 you can use bind mounts to remount the application file systems below the user.

Quoted 4 lines

Is there someone who might be able to give me some information on how I could add a few lines to the VFS filesystem so that I might set some type of extended attribute to prevent users from navigating out of the locations.

It isnt down to attributes - how you can run a binary or load a shared library you cant see.

You might also want to see http://www.nsa.gov/selinux, but that would require a lot of careful policy setup

#2 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

can you point me to a site so i can learn more about that binding of mounts? i recently heard something about that in plan 9, and it sounds very interesting. union mounting they call it also, correct?

Quoted 20 lines · Alan Cox
Quoted 2 lines

I have look into using things like "chroot" to restrict the users for this very special server, but that solution is not what we need.

It sounds like it is to me

Quoted 2 lines

My problem is that I need to find a way to prevent the user from navigating out of their home directories.

Then you must put al the files in their home directories. Alternatively with later 2.4 you can use bind mounts to remount the application file systems below the user.

Quoted 4 lines

Is there someone who might be able to give me some information on how I could add a few lines to the VFS filesystem so that I might set some type of extended attribute to prevent users from navigating out of the locations.

It isnt down to attributes - how you can run a binary or load a shared library you cant see.

Quoted 9 lines

You might also want to see http://www.nsa.gov/selinux, but that would require a lot of careful policy setup
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

- --
Phil Sorber
AIM: PSUdaemon
IRC: irc.openprojects.net #psulug PSUdaemon
GnuPG: keyserver - pgp.mit.edu
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE75d+9Xm6Gwek+iaQRAgnVAKCBpdA6EzpXoT/SIffK4yuPviHENgCggVq6 +6Dn7AzjrsT+S7GavhNudSI=
=eEGW
-----END PGP SIGNATURE-----

#3 lore raw
In reply to Lonnie Cumberland
Quoted 2 lines · Lonnie Cumberland

I have look into using things like "chroot" to restrict the users for this very special server, but that solution is not what we need.

....

Quoted 4 lines

Is there someone who might be able to give me some information on how I could add a few lines to the VFS filesystem so that I might set some type of extended attribute to prevent users from navigating out of the locations.

I fail to see the difference between "chroot" and "preventing users from navigating out of locations". Would you care to clarify what was wrong was chroot that you believe you can solve with a different approach? -Ryan

#4 lore raw
In reply to Lonnie Cumberland
Quoted 5 lines · Lonnie Cumberland

I have look into using things like "chroot" to restrict the users for this very special server, but that solution is not what we need.

My problem is that I need to find a way to prevent the user from navigating out of their home directories.

chroot() will do that pretty well, but if you want more you can always take a look at vserver:

http://www.solucorp.qc.ca/miscprj/s_context.hc

Not as sophisticated as NSA's selinux, but that also means it's much easier to get security because it's just easier to setup ;)

regards,

Rik

Signature · 3 lines
DMCA, SSSCA, W3C?  Who cares?  http://thefreeworld.net/

http://www.surriel.com/		http://distro.conectiva.com/
#5 lore raw
In reply to Lonnie Cumberland
Quoted 2 lines · [email protected] said:

My problem is that I need to find a way to prevent the user from navigating out of their home directories.

A virtual machine would be an administratively easy way of doing this.

Let the 'app' be a VM with the real apps installed inside. The users would effectively be confined to a *file* on the host, not merely their home directories.

My (biased :-) recommendation would be User-mode Linux
(http://user-mode-linux.sourceforge.net), but VMWare would work as well.

Jeff

#6 lore raw

Hey Jeff,

Quoted 9 lines

A virtual machine would be an administratively easy way of doing this.

Let the 'app' be a VM with the real apps installed inside. The users would effectively be confined to a *file* on the host, not merely their home directories.

My (biased :-) recommendation would be User-mode Linux
(http://user-mode-linux.sourceforge.net), but VMWare would work as well.

I'm afraid that would scale horribly on anything greater than a very modest load:

1) There is a fairly significant overhead due to running inside a VM, which would certainly pile up over multiple invocations. Besides the overhead of the VM, you have the have the additional overhead caused by such things as having multiple running kernels doing exactly the same thing, all running seperate copies of the required system daemons.
2) Upgrading/changing the system configuration would require modifying -all- of the VMs. This could be partially alleviated by using an NFS root, but then the overhead mentioned above becomes even more looming
3) Implicitly shared memory no longer works too well, and this is one of the reasons Unix is so strong as a terminal server:
You have thirty people running StarOffice from /usr/bin, and because
they've all mmap()ed the same binary, the read only portions are all shared, and will definitely stay in memory in the event of memory pressure. The same things goes for all the libraries and mmap()'ed data files StarOffice uses, including some pretty hefty ones such as glibc. Even many of the writable pages will be shared until someone decides to write to one of them, and then the writer will get a private copy.
Now, try 30 people all running from VMs. They all must have a seperate copy
of the otherwise sharable portions of StarOffice, and all of its dependancies. Memory pressure quickly rises, and it will start swapping out one copy of StarOffice in favour of an identical portion of another copy (!). System quickly becomes unusable.

So, to summarize, you could use a VM, and it'd work well for containment, but it'd be pretty useless as far as performance, ease of administration, and scaling go. Comments?

-Ryan

#7 lore raw
Quoted 3 lines · [email protected] said:

So, to summarize, you could use a VM, and it'd work well for containment, but it'd be pretty useless as far as performance, ease of administration, and scaling go.

Yeah, that's mostly wrong.

Quoted 1 line

Comments?

They follow...

Quoted 6 lines

1) There is a fairly significant overhead due to running inside a VM, which would certainly pile up over multiple invocations. Besides the overhead of the VM, you have the have the additional overhead caused by such things as having multiple running kernels doing exactly the same thing, all running seperate copies of the required system daemons.

This can be minimized. What the original poster asked for is a one-application jail. So, you boot up a filesystem that runs only that app, perhaps even as the init process. No system daemons, nothing but the app and whatever it runs.

Quoted 5 lines

Now, try 30 people all running from VMs. They all must have a seperate copy of the otherwise sharable portions of StarOffice, and all of its
dependancies. Memory pressure quickly rises, and it will start
swapping out one copy of StarOffice in favour of an identical portion of another copy (!). System quickly becomes unusable.

Nope. Duplicate block caching can be eliminated by making the root filesystem a COW device (http://user-mode-linux.sourceforge.net/shared_fs.html) and mounting it sychronously. COWing it will allow it to be shared among all VMs, with any changes being stored in the private COW file. Mounting it synchronous will disable caching in the VM. So, for a number of virtual machines, there will be, in total, slightly more than one copy of their data in the host's memory.

Another possibility is hostfs. You can directly mount a host directory inside UML. That can obviously be shared between UMLs, so you again eliminate all the duplication.

Quoted 3 lines

2) Upgrading/changing the system configuration would require modifying -all- of the VMs. This could be partially alleviated by using an NFS root, but then the overhead mentioned above becomes even more looming

This is a problem for a COWed block device, but it's trivial for a shared hostfs directory. Just dump the new bits in.

Jeff

#8 lore raw

Jeff,

Quoted 1 line

Mounting it synchronous will disable caching in the VM.

Who told you that? Synchronous mounting turns off write buffering. Even with "-o sync" writes will still end up in the page cache, they'll just be commited immediately. In case you don't believe me, here's a trivial test on a block device mounted -o sync (ext3):

bodnar42:/mnt# time cat linux-2.4.13.tar.bz2 > /dev/null
real    0m2.837s
user    0m0.040s
sys     0m0.350s
bodnar42:/mnt# time cat linux-2.4.13.tar.bz2 > /dev/null
real    0m0.328s
user    0m0.010s
sys     0m0.240s

bodnar42:/mnt#

Trust me, the factor of 8 performace improvement the second time is not due to lucky head positioning or anything, that's coming straight out of cache.

Quoted 3 lines

Another possibility is hostfs. You can directly mount a host directory inside UML. That can obviously be shared between UMLs, so you again eliminate all the duplication.

Er, it will be shared in the host's context, but each VM will still have their own copy in the page cache. This is no better than a COW'ed block device

-Ryan

#9 lore raw
Quoted 5 lines · [email protected] said:
Quoted 1 line

Mounting it synchronous will disable caching in the VM.

Who told
you that? Synchronous mounting turns off write buffering. Even with "-o sync" writes will still end up in the page cache, they'll just be commited immediately.

Ummm, how about O_DIRECT instead of O_SYNC (or maybe as well, my googling hasn't been clear on whether O_DIRECT bypasses the cache on writes as well)?

All IO bypasses the virtual machine cache, so there's no multiple caching.

Quoted 3 lines

Er, it will be shared in the host's context, but each VM will still have their own copy in the page cache. This is no better than a COW'ed block device

Nope. You O_DIRECT here as well.

Jeff

#10 lore raw
Quoted 12 lines · you
Quoted 6 lines · [email protected] said:
Quoted 1 line

Mounting it synchronous will disable caching in the VM.

Who told
you that? Synchronous mounting turns off write buffering. Even with "-o sync" writes will still end up in the page cache, they'll just be commited immediately.

Ummm, how about O_DIRECT instead of O_SYNC (or maybe as well, my googling hasn't been clear on whether O_DIRECT bypasses the cache on writes as well)?

Nope, last I checked O_DIRECT enforces buffer and file offset alignment. Normal apps wouldn't work very well at all. Maybe UML needs some hacks around the whole caching issue?

-Ryan

#11 lore raw
Quoted 2 lines · [email protected] said:

Nope, last I checked O_DIRECT enforces buffer and file offset alignment. Normal apps wouldn't work very well at all.

Except that the apps won't be the ones doing O_DIRECT IO. It'll be UML, and it can presumably provide whatever alignment is required.

Quoted 1 line

Maybe UML needs some hacks around the whole caching issue?

Not that I can see...

Jeff

#12 lore raw
In reply to Ryan Cumming
Quoted 9 lines · Jeff Dike
Quoted 5 lines · [email protected] said:
Quoted 1 line

Mounting it synchronous will disable caching in the VM.

Who told
you that? Synchronous mounting turns off write buffering. Even with "-o sync" writes will still end up in the page cache, they'll just be commited immediately.

Ummm, how about O_DIRECT instead of O_SYNC (or maybe as well, my googling hasn't been clear on whether O_DIRECT bypasses the cache on writes as well)?

O_DIRECT is synchronous but only in terms of data, if you want the metadata to be synchronous as well you need to open with O_SYNC|O_DIRECT, and even in such case all the metadata reads will came from cache.

For example if you only care about being able to reach the data after a crash (not about the inode info) in a file with all its logical blocks mapped to physical blcoks (no holes) and then you fsync, later you can as well avoid O_SYNC and you still don't risk to lose data after a crash because the block mappings never changes, if you grow/shrink the file you definitely need O_SYNC to be sure the O_DIRECT data is still there after a crash instead.

Andrea

#13 lore raw
Quoted 4 lines · [email protected] said:

O_DIRECT is synchronous but only in terms of data, if you want the metadata to be synchronous as well you need to open with O_SYNC|O_DIRECT, and even in such case all the metadata reads will came from cache.

That's OK. The goal is to cram as many UMLs onto a host as possible by eliminating caching as much as possible. If the metadata/data ratio is small, then the metadata caching probably won't be noticable.

Jeff

#14 lore raw
In reply to Andrea Arcangeli
Quoted 2 lines · Jeff Dike

eliminating caching as much as possible. If the metadata/data ratio is small, then the metadata caching probably won't be noticable.

yes, of course the metadata/data ratio is very small, O_DIRECT isn't slower than rawio infact (assuming the file isn't fragmented).

Andrea

#15 lore raw
In reply to Lonnie Cumberland

Hello Ryan,

Quoted 1 line

From what I can see. With chrooting, I have to make a complete "fake" system an

then place the users below that into a home directory, or make a complete "fake" system for each user.

I was trying to find a simple solution that would allow for:

I was initially thinking about something like this for each user:

/system (real) /dev/hda4 (chrooted also)

      |
      /bin
      /etc
      /lib

/home (each user chrooted)

     |
     /user1
     |     |
     |     /system (mounted /dev/hda4)
     |            |
     |            /bin
     |            /etc
     |            /lib
     |
     /user2
     |     |
     |     /system (mounted /dev/hda4)
     |            |
     |            /bin
     |            /etc
     |            /lib
     |
     /user n
          |
          /system (mounted /dev/hda4)
                 |
                 /bin
                 /etc
                 /lib

The basic problem is that I did not want, for example "user2" to be able to "cd .." or some thing to go out of user2

I was hoping to be able to accomplish this at the filesystem level somehow, and possibly without the need to mount the /dev/hda4 onto each /home/user/system, or without having to make entire copies of the chrooted environment for each user.

Cheers,
Lonnie

Quoting Ryan Cumming <[email protected]>:

Quoted 19 lines · Lonnie Cumberland
Quoted 1 line

I have look into using things like "chroot" to restrict the users

for

Quoted 1 line

this very special server, but that solution is not what we need.

....

Quoted 1 line

Is there someone who might be able to give me some information on how

I

Quoted 2 lines

could add a few lines to the VFS filesystem so that I might set some type of extended attribute to prevent users from navigating out of

the

Quoted 1 line

locations.

I fail to see the difference between "chroot" and "preventing users from

navigating out of locations". Would you care to clarify what was wrong was
chroot that you believe you can solve with a different approach? -Ryan

#16 lore raw
Quoted 6 lines · On Sun, Nov 04, 2001 at 19:29:01 -0500, [email protected] wrote:

From what I can see. With chrooting, I have to make a complete "fake" system an then place the users below that into a home directory, or make a complete "fake" system for each user.

I was trying to find a simple solution that would allow for:

Quoted 7 lines

I was initially thinking about something like this for each user:

/system (real) /dev/hda4 (chrooted also)

      |
      /bin
      /etc
      /lib

chtrunk (http://noid.sf.net/chtrunk.html) can set up the namespace dynamically for you. Instead of creating a complete system by hand and run chroot, just run (you don't need to be root):

chtrunk -s /bin /etc /lib /home/user -c program_to_run

This will give that program access to /bin, /etc, /lib and the home directory, but nothing more.

You can use

chtrunk -s /bin /etc /lib /home/user /tmp=/home/user/tmp -c program

to give every user their own private /tmp.

As a bonus, the suid/sgid bits will have no effect for these users, which will prevent them from becoming root through buggy suid programs.

Jörgen

#17 lore raw

Thanks Jorgen,

I am sure that this will help as it looks like what I might need....

Thanks again,
Lonnie

Quoting Jorgen Cederlof <[email protected]>:

Quoted 38 lines · On Sun, Nov 04, 2001 at 19:29:01 -0500, [email protected] wrote:
Quoted 7 lines

From what I can see. With chrooting, I have to make a complete "fake" system an then place the users below that into a home directory, or make a complete "fake" system for each user.

I was trying to find a simple solution that would allow for:

Quoted 7 lines

I was initially thinking about something like this for each user:

/system (real) /dev/hda4 (chrooted also)

      |
      /bin
      /etc
      /lib

chtrunk (http://noid.sf.net/chtrunk.html) can set up the namespace dynamically for you. Instead of creating a complete system by hand and run chroot, just run (you don't need to be root):

chtrunk -s /bin /etc /lib /home/user -c program_to_run

This will give that program access to /bin, /etc, /lib and the home directory, but nothing more.

You can use

chtrunk -s /bin /etc /lib /home/user /tmp=/home/user/tmp -c program

to give every user their own private /tmp.

As a bonus, the suid/sgid bits will have no effect for these users, which will prevent them from becoming root through buggy suid programs.

Jörgen

#18 lore raw
In reply to Lonnie Cumberland

On Sun, 2001-11-04 19:29:01 -0500, [email protected] <[email protected]> wrote in message <[email protected]>:

Quoted 6 lines

The basic problem is that I did not want, for example "user2" to be able to "cd .." or some thing to go out of user2

I was hoping to be able to accomplish this at the filesystem level somehow, and possibly without the need to mount the /dev/hda4 onto each /home/user/system, or without having to make entire copies of the chrooted environment for each user.

Isn't it the simplest of all to set all the user's file and directory modes to sth likt 0700? Then, every user may only access his/her files. (S)he cannot look at other user's files.

MfG, JBG

Signature · 1 line
Jan-Benedict Glaw . [email protected] . +49-172-7608481
#19 lore raw
In reply to Lonnie Cumberland

Um, what about chmod command? You don't want people to be able to see directories like /bin? You would have those anyway in a chroot environment. Use chmod to turn off read and execute(search) permissions to anything they don't
need to run (ie /bin, /usr/bin, /lib, etc). Sorry if that sounds dumb, but there's not much information to go on.

Also, for a special xserver, all you need to make it run staroffice or something else, is change the
shell script that starts the x server and/or the user's session. Xdm has a bunch of options for that.

There are also kernel "capabilities"...

----- Original Message -----
From: Lonnie Cumberland <[email protected]>
To: <[email protected]>
Sent: Sunday, November 04, 2001 4:01 PM
Subject: Special Kernel Modification

Quoted 3 lines

The basic problem that I am running into is that I am working on an Internet related project and thus need to ensure various types of document security for the eventual users of this system, if things go

well.

Quoted 28 lines

I have look into using things like "chroot" to restrict the users for this very special server, but that solution is not what we need.

I am building a special xserver that will allow users to login and then the xserver will run a single application such as StarOffice. When the user exits from the application then the Xserver will log them out.

My problem is that I need to find a way to prevent the user from navigating out of their home directories.

I have also looking the possiblility of writing my own filesystem, but I am told that this needs to be done at the VFS level.

Is there someone who might be able to give me some information on how I could add a few lines to the VFS filesystem so that I might set some type of extended attribute to prevent users from navigating out of the locations.

Any help would be greatly appreciated,
Lonnie Cumberland

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#20 lore raw
In reply to Lonnie Cumberland
Quoted 6 lines · On Sun, Nov 04, 2001 at 07:29:01PM -0500, [email protected] wrote:

Hello Ryan,

Quoted 1 line

From what I can see. With chrooting, I have to make a complete "fake" system an

then place the users below that into a home directory, or make a complete "fake" system for each user.

Quoted 7 lines

The basic problem is that I did not want, for example "user2" to be able to "cd .." or some thing to go out of user2

I was hoping to be able to accomplish this at the filesystem level somehow, and possibly without the need to mount the /dev/hda4 onto each /home/user/system, or without having to make entire copies of the chrooted environment for each user.

Chroot will allow you to keel a user within a certain directory tree, and as long as you use hard links on the same FS, you won't waste much space for each chroot...

Also, why don't you want the users to be able to see the executable directories? They're only writable by root, right?

If you set each user's home directory to mode 0700 no other user will be able to cd into that dir...

The real question is why do you want to split each user so much?

Mike