Skip to contentSkip to conversation

replace SYSV shared memory with shm filesystem

PATCH In review Christoph Rohland 6 participants 9 messages Jan 10, 2000 – Jan 11, 2000

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 replace SYSV shared memory with shm filesystem

Hi folks,

This patch implements a minimal filesystem for shared memory. It replaces/reuses the existing SYSV shm code so you now have to mount the fs to be able to use SYSV SHM. But in turn we now have everything in place to implement posix shm. This also obsoletes vm_private_data in vm_area_struct.

Also it is now possible to do e.g. 'rm /dev/shm/*' instead of this terrible 'ipcrm shm xx' :-)

You can mount the fs only once. This will probably break shm in chrooted environments.

It is heavily tested from UP to 8way SMP with HIGHMEM. On the latter the shm handling triggers some mm bugs under heavy swapping. But these problems also exists in the stock 2.3.36 kernel (I cannot boot 2.3.3[78] on my 8way). I will report these problems later in a separate mail.

Greetings
Christoph

Documentation/Changes +3 -0
diff -uNr 2.3.38/Documentation/Changes c38/Documentation/Changes
@@ -72,6 +72,9 @@
 General Information
 ===================
 
+   To use System V shared memory, you have to mount the shm filesystem
+somewhere.
+
    <CTRL><ALT><DEL> now performs a cold reboot instead of a warm reboot
 for increased hardware compatibility.  If you want a warm reboot and
 know it works on your hardware, add a "reboot=warm" command line option
fs/exec.c +0 -1
diff -uNr 2.3.38/fs/exec.c c38/fs/exec.c
@@ -322,7 +322,6 @@
 		mpnt->vm_ops = NULL;
 		mpnt->vm_pgoff = 0;
 		mpnt->vm_file = NULL;
-		mpnt->vm_private_data = (void *) 0;
 		vmlist_modify_lock(current->mm);
 		insert_vm_struct(current->mm, mpnt);
 		vmlist_modify_unlock(current->mm);
fs/namei.c +1 -1
diff -uNr 2.3.38/fs/namei.c c38/fs/namei.c
@@ -991,7 +991,7 @@
 	return error;
 }
 
-static inline int do_unlink(const char * name)
+int do_unlink(const char * name)
 {
 	int error;
 	struct dentry *dir;
include/linux/fs.h +1 -0
diff -uNr 2.3.38/include/linux/fs.h c38/include/linux/fs.h
@@ -865,6 +865,7 @@
 extern struct dentry * open_namei(const char *, int, int);
 extern struct dentry * do_mknod(const char *, int, dev_t);
 extern int do_pipe(int *);
+extern int do_unlink(const char * name);
 
 /* fs/dcache.c -- generic fs support functions */
 extern int is_subdir(struct dentry *, struct dentry *);
include/linux/mm.h +0 -1
diff -uNr 2.3.38/include/linux/mm.h c38/include/linux/mm.h
@@ -60,7 +60,6 @@
 	struct vm_operations_struct * vm_ops;
 	unsigned long vm_pgoff;		/* offset in PAGE_SIZE units, *not* PAGE_CACHE_SIZE */
 	struct file * vm_file;
-	void * vm_private_data;		/* was vm_pte (shared mem) */
 };
 
 /*
ipc/shm.c +562 -294

1,141 lines not on this page · raw message

mm/mmap.c +0 -4
diff -uNr 2.3.38/mm/mmap.c c38/mm/mmap.c
@@ -271,7 +271,6 @@
 	vma->vm_ops = NULL;
 	vma->vm_pgoff = pgoff;
 	vma->vm_file = NULL;
-	vma->vm_private_data = NULL;
 
 	/* Clear old maps */
 	error = -ENOMEM;
@@ -547,7 +546,6 @@
 		mpnt->vm_ops = area->vm_ops;
 		mpnt->vm_pgoff = area->vm_pgoff + ((end - area->vm_start) >> PAGE_SHIFT);
 		mpnt->vm_file = area->vm_file;
-		mpnt->vm_private_data = area->vm_private_data;
 		if (mpnt->vm_file)
 			get_file(mpnt->vm_file);
 		if (mpnt->vm_ops && mpnt->vm_ops->open)
@@ -780,7 +778,6 @@
 	vma->vm_ops = NULL;
 	vma->vm_pgoff = 0;
 	vma->vm_file = NULL;
-	vma->vm_private_data = NULL;
 
 	/*
 	 * merge_segments may merge our vma, so we can't refer to it
@@ -927,7 +924,6 @@
 
 		/* To share, we must have the same file, operations.. */
 		if ((mpnt->vm_file != prev->vm_file)||
-		    (mpnt->vm_private_data != prev->vm_private_data)	||
 		    (mpnt->vm_ops != prev->vm_ops)	||
 		    (mpnt->vm_flags != prev->vm_flags)	||
 		    (prev->vm_end != mpnt->vm_start))
#1 lore raw
Quoted 4 lines

replaces/reuses the existing SYSV shm code so you now have to mount the fs to be able to use SYSV SHM. But in turn we now have everything in place to implement posix shm. This also obsoletes vm_private_data in vm_area_struct.

Umm no. It obsoletes vm_private_data for existing merged file systems. The stackable file systems still need this field (cryptfs, lofs etc)

Quoted 2 lines

Also it is now possible to do e.g. 'rm /dev/shm/*' instead of this terrible 'ipcrm shm xx' :-)

Nice

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.nl.linux.org/Linux-MM/

#2 lore raw
Quoted 8 lines · Alan Cox
Quoted 4 lines

replaces/reuses the existing SYSV shm code so you now have to mount the fs to be able to use SYSV SHM. But in turn we now have everything in place to implement posix shm. This also obsoletes vm_private_data in vm_area_struct.

Umm no. It obsoletes vm_private_data for existing merged file systems. The stackable file systems still need this field (cryptfs, lofs etc)

Hey, that's a pity :-( I liked this cleanup. Do they really need it?

Quoted 4 lines
Quoted 2 lines

Also it is now possible to do e.g. 'rm /dev/shm/*' instead of this terrible 'ipcrm shm xx' :-)

Nice

Yup :-)

Greetings
Christoph
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.nl.linux.org/Linux-MM/

#3 lore raw
In reply to Christoph Rohland
Quoted 2 lines · Christoph Rohland

You can mount the fs only once. This will probably break shm in chrooted environments.

I think it would be best to code it in such a way that you can mount multiple instances of shmfs, in such a way that the differently chrooted programs cannot see each other's shared memory. That should make it a bit more difficult to get out of a chroot() jail...

(security buffs, please tell me if I'm full of it)

cheers,

Rik
--
The Internet is not a network of computers. It is a network of people. That is its real strength.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.nl.linux.org/Linux-MM/

#4 lore raw
In reply to Christoph Rohland
Quoted 7 lines · Christoph Rohland

Hi folks,

This patch implements a minimal filesystem for shared memory. It replaces/reuses the existing SYSV shm code so you now have to mount the fs to be able to use SYSV SHM. But in turn we now have everything in place to implement posix shm. This also obsoletes vm_private_data in vm_area_struct.

I planed to map the Unix Sockets abstract name space to a file system for some time now. Because it would be silly to write another file system just for that rather obscure feature, would it be possible to use a subdirectory in your new shm filesystem? I haven't looked at the code at all yet, and don't know if it can even deal with directories and special devices. Do you have objections to such a direction?

-Andi

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.nl.linux.org/Linux-MM/

#5 lore raw
Quoted 9 lines · Andi Kleen
Quoted 7 lines · Christoph Rohland

Hi folks,

This patch implements a minimal filesystem for shared memory. It replaces/reuses the existing SYSV shm code so you now have to mount the fs to be able to use SYSV SHM. But in turn we now have everything in place to implement posix shm. This also obsoletes vm_private_data in vm_area_struct.

Quoted 7 lines

I planed to map the Unix Sockets abstract name space to a file system for some time now. Because it would be silly to write another file system just for that rather obscure feature, would it be possible to use a subdirectory in your new shm filesystem? I haven't looked at the code at all yet, and don't know if it can even deal with directories and special devices. Do you have objections to such a direction?

In the current state this is not possible. The shm fs does not support directories and only regular files (which you can only mmap, no read/write support).

But we could later extend the fs to support directories and special files. The Unix Sockets could probably also use the same mechanisms for locating the special fs like SYSV ipc does.

With these changes we also should then be able to mount the fs several times. So we also get the chroot case fixed.

Greetings
Christoph
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.nl.linux.org/Linux-MM/

#6 lore raw
Quoted 23 lines
Quoted 9 lines · Andi Kleen
Quoted 7 lines · Christoph Rohland

Hi folks,

This patch implements a minimal filesystem for shared memory. It replaces/reuses the existing SYSV shm code so you now have to mount the fs to be able to use SYSV SHM. But in turn we now have everything in place to implement posix shm. This also obsoletes vm_private_data in vm_area_struct.

Quoted 7 lines

I planed to map the Unix Sockets abstract name space to a file system for some time now. Because it would be silly to write another file system just for that rather obscure feature, would it be possible to use a subdirectory in your new shm filesystem? I haven't looked at the code at all yet, and don't know if it can even deal with directories and special devices. Do you have objections to such a direction?

In the current state this is not possible. The shm fs does not support directories and only regular files (which you can only mmap, no read/write support).

Quoted 6 lines

But we could later extend the fs to support directories and special files. The Unix Sockets could probably also use the same mechanisms for locating the special fs like SYSV ipc does.

With these changes we also should then be able to mount the fs several times. So we also get the chroot case fixed.

If the idea is to integrate the unix domain sockets at some point,
then I'd like to suggest that _both_ shm and unix domain sockets get a subtree. Ie.

/kernfs/unix_domain_sockets/*
/kernfs/sysv_shared_memory/*

Instead of...

/shm/unix_domain_sockets/*
/shm/* (apart from unix_domain_sockets, and maybe others)

Obviously example names might be too long for your tastes.

This also makes it easier to add new directories (don't say it won't
happen look at proc :).

Indeed a possibly useful one that I can think of straight away is...

/kernfs/users/<uid>/

Where it could store entries to inodes that no longer have a file
associated with them (Ie. like the proc/self/fd/* [1], but for each user).

I'm not saying you should implement the fs above, just that it's
going to hinder it (or anything like it) if you put shm in the root.

Signature · 7 lines
James Antill -- [email protected]
I am always an optimist, but frankly there is no hope.
   -Hosni Mubarek
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.nl.linux.org/Linux-MM/
#7 lore raw

Oh no! Not another /proc. Please do a different fs, perhaps you can share some code with shmfs, but not the namespace!

Richard.

Quoted 48 lines · James Antill
Quoted 22 lines
Quoted 9 lines · Andi Kleen
Quoted 7 lines · Christoph Rohland

Hi folks,

This patch implements a minimal filesystem for shared memory. It replaces/reuses the existing SYSV shm code so you now have to mount the fs to be able to use SYSV SHM. But in turn we now have everything in place to implement posix shm. This also obsoletes vm_private_data in vm_area_struct.

Quoted 7 lines

I planed to map the Unix Sockets abstract name space to a file system for some time now. Because it would be silly to write another file system just for that rather obscure feature, would it be possible to use a subdirectory in your new shm filesystem? I haven't looked at the code at all yet, and don't know if it can even deal with directories and special devices. Do you have objections to such a direction?

In the current state this is not possible. The shm fs does not support directories and only regular files (which you can only mmap, no read/write support).

Quoted 6 lines

But we could later extend the fs to support directories and special files. The Unix Sockets could probably also use the same mechanisms for locating the special fs like SYSV ipc does.

With these changes we also should then be able to mount the fs several times. So we also get the chroot case fixed.

If the idea is to integrate the unix domain sockets at some point,
then I'd like to suggest that _both_ shm and unix domain sockets get a subtree. Ie.

/kernfs/unix_domain_sockets/*
/kernfs/sysv_shared_memory/*

Instead of...

/shm/unix_domain_sockets/*
/shm/* (apart from unix_domain_sockets, and maybe others)

Obviously example names might be too long for your tastes.

This also makes it easier to add new directories (don't say it won't
happen look at proc :).

Indeed a possibly useful one that I can think of straight away is...

Quoted 18 lines

/kernfs/users/<uid>/

Where it could store entries to inodes that no longer have a file
associated with them (Ie. like the proc/self/fd/* [1], but for each user).

I'm not saying you should implement the fs above, just that it's
going to hinder it (or anything like it) if you put shm in the root.

Signature · 7 lines
James Antill -- [email protected]
I am always an optimist, but frankly there is no hope.
   -Hosni Mubarek
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.nl.linux.org/Linux-MM/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.nl.linux.org/Linux-MM/

#8 lore raw
In reply to James Antill
Quoted 6 lines · James Antill

If the idea is to integrate the unix domain sockets at some point,
then I'd like to suggest that _both_ shm and unix domain sockets get a subtree. Ie.

/kernfs/unix_domain_sockets/*
/kernfs/sysv_shared_memory/*

I think Unix Sockets should use a new instance of the fs and mount it to a private place.

Greetings
Christoph
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.nl.linux.org/Linux-MM/