swapfs for 2.4.1-pre

PATCH New J . A . Magallon 4 participants 14 messages Jan 23, 2001 – Feb 6, 2001

Subsystems (7)

Memory management Documentation VFS Library code +3 more subsystems

lore

Share & tools

lore

J . A . Magallon

author

2001-01-23 19:53 UTC #0 lore raw

Hello all...

I have seen the patch for 2.4.0-ac9 to use shmfs as tmpfs, change to the new name 'swapfs', etc..

Is there the same patch for 2.4.1-pre10 ?

Signature · 9 lines

J.A. Magallon                                                      $> cd pub
mailto:[email protected]                                          $> more beer

Linux werewolf 2.4.0-ac10 #1 SMP Sat Jan 20 10:43:18 CET 2001 i686

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

Christoph Rohland

2001-02-01 21:34 UTC #1 lore raw

PATCH 1/1 tmpfs for 2.4.1

Hi,

here is the latest version of my tmpfs patch against 2.4.1

Have fun
Christoph

Documentation/Changes +0 ~~-14~~

diff -uNr 2.4.1/Documentation/Changes 2.4.1-tmpfs/Documentation/Changes
@@ -114,20 +114,6 @@
 DevFS is now in the kernel.  See Documentation/filesystems/devfs/* in
 the kernel source tree for all the gory details.
 
-System V shared memory is now implemented via a virtual filesystem.
-You do not have to mount it to use it. SYSV shared memory limits are
-set via /proc/sys/kernel/shm{max,all,mni}.  You should mount the
-filesystem under /dev/shm to be able to use POSIX shared
-memory. Adding the following line to /etc/fstab should take care of
-things:
-
-none		/dev/shm	shm		defaults	0 0
-
-Remember to create the directory that you intend to mount shm on if
-necessary (The entry is automagically created if you use devfs). You
-can set limits for the number of blocks and inodes used by the
-filesystem with the mount options nr_blocks and nr_inodes.
-
 The Logical Volume Manager (LVM) is now in the kernel.  If you want to
 use this, you'll need to install the necessary LVM toolset.

Documentation/Configure.help +32 ~~-19~~

69 lines not on this page · raw message

arch/i386/kernel/setup.c +1 -1

diff -uNr 2.4.1/arch/i386/kernel/setup.c 2.4.1-tmpfs/arch/i386/kernel/setup.c
@@ -559,7 +559,7 @@
 				 * blow away any automatically generated
 				 * size
 				 */
-				unsigned long start_at, mem_size;
+				unsigned long long start_at, mem_size;
  
 				if (usermem == 0) {
 					/* first time in: zap the whitelist

fs/Config.in +1 -0

diff -uNr 2.4.1/fs/Config.in 2.4.1-tmpfs/fs/Config.in
@@ -31,6 +31,7 @@
 	int 'JFFS debugging verbosity (0 = quiet, 3 = noisy)' CONFIG_JFFS_FS_VERBOSE 0
 fi
 tristate 'Compressed ROM file system support' CONFIG_CRAMFS
+bool 'Virtual memory file system support (former shm fs)' CONFIG_TMPFS
 tristate 'Simple RAM-based file system support' CONFIG_RAMFS
 
 tristate 'ISO 9660 CDROM file system support' CONFIG_ISO9660_FS

include/linux/fs.h +2 -0

diff -uNr 2.4.1/include/linux/fs.h 2.4.1-tmpfs/include/linux/fs.h
@@ -1213,6 +1213,7 @@
 	}
 	return inode;
 }
+extern void remove_suid(struct inode *inode);
 
 extern void insert_inode_hash(struct inode *);
 extern void remove_inode_hash(struct inode *);

@@ -1260,6 +1261,7 @@
 int block_truncate_page(struct address_space *, loff_t, get_block_t *);
 
 extern int generic_file_mmap(struct file *, struct vm_area_struct *);
+extern int file_read_actor(read_descriptor_t * desc, struct page *page, unsigned long offset, unsigned long size);
 extern ssize_t generic_file_read(struct file *, char *, size_t, loff_t *);
 extern ssize_t generic_file_write(struct file *, const char *, size_t, loff_t *);
 extern void do_generic_file_read(struct file *, loff_t *, read_descriptor_t *, read_actor_t);

include/linux/kernel.h +1 -1

diff -uNr 2.4.1/include/linux/kernel.h 2.4.1-tmpfs/include/linux/kernel.h
@@ -62,7 +62,7 @@
 extern int vsprintf(char *buf, const char *, va_list);
 extern int get_option(char **str, int *pint);
 extern char *get_options(char *str, int nints, int *ints);
-extern unsigned long memparse(char *ptr, char **retptr);
+extern unsigned long long memparse(char *ptr, char **retptr);
 extern void dev_probe_lock(void);
 extern void dev_probe_unlock(void);

include/linux/mm.h +2 -2

diff -uNr 2.4.1/include/linux/mm.h 2.4.1-tmpfs/include/linux/mm.h
@@ -200,8 +200,8 @@
 					smp_mb__before_clear_bit(); \
 					if (!test_and_clear_bit(PG_locked, &(page)->flags)) BUG(); \
 					smp_mb__after_clear_bit(); \
-					if (waitqueue_active(&page->wait)) \
-						wake_up(&page->wait); \
+					if (waitqueue_active(&(page)->wait)) \
+						wake_up(&(page)->wait); \
 				} while (0)
 #define PageError(page)		test_bit(PG_error, &(page)->flags)
 #define SetPageError(page)	set_bit(PG_error, &(page)->flags)

include/linux/shmem_fs.h +1 -0

diff -uNr 2.4.1/include/linux/shmem_fs.h 2.4.1-tmpfs/include/linux/shmem_fs.h
@@ -19,6 +19,7 @@
 
 struct shmem_inode_info {
 	spinlock_t	lock;
+	unsigned long	max_index;
 	swp_entry_t	i_direct[SHMEM_NR_DIRECT]; /* for the first blocks */
 	swp_entry_t   **i_indirect; /* doubly indirect blocks */
 	unsigned long	swapped;

lib/cmdline.c +2 -2

diff -uNr 2.4.1/lib/cmdline.c 2.4.1-tmpfs/lib/cmdline.c
@@ -93,9 +93,9 @@
  *	megabyte, or one gigabyte, respectively.
  */
 
-unsigned long memparse (char *ptr, char **retptr)
+unsigned long long memparse (char *ptr, char **retptr)
 {
-	unsigned long ret = simple_strtoul (ptr, retptr, 0);
+	unsigned long long ret = simple_strtoul (ptr, retptr, 0);
 
 	switch (**retptr) {
 	case 'G':

mm/filemap.c +2 -2

diff -uNr 2.4.1/mm/filemap.c 2.4.1-tmpfs/mm/filemap.c
@@ -1209,7 +1209,7 @@
 	UPDATE_ATIME(inode);
 }
 
-static int file_read_actor(read_descriptor_t * desc, struct page *page, unsigned long offset, unsigned long size)
+int file_read_actor(read_descriptor_t * desc, struct page *page, unsigned long offset, unsigned long size)
 {
 	char *kaddr;
 	unsigned long left, count = desc->count;

@@ -2408,7 +2408,7 @@
 	return page;
 }
 
-static inline void remove_suid(struct inode *inode)
+inline void remove_suid(struct inode *inode)
 {
 	unsigned int mode;

mm/shmem.c +542 ~~-195~~

1,026 lines not on this page · raw message

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

H. Peter Anvin

2001-02-01 23:50 UTC #2 lore raw

Re: [patch] tmpfs for 2.4.1

In reply to J . A . Magallon

Followup to:  <[email protected]>
By author:    Christoph Rohland <[email protected]>
In newsgroup: linux.dev.kernel

Quoted 22 lines

Hi,

here is the latest version of my tmpfs patch against 2.4.1

Have fun
Christoph

diff -uNr 2.4.1/Documentation/Changes 2.4.1-tmpfs/Documentation/Changes

Documentation/Changes +0 -7

@@ -114,20 +114,6 @@
 DevFS is now in the kernel.  See Documentation/filesystems/devfs/* in
 the kernel source tree for all the gory details.
 
-System V shared memory is now implemented via a virtual filesystem.
-You do not have to mount it to use it. SYSV shared memory limits are
-set via /proc/sys/kernel/shm{max,all,mni}.  You should mount the
-filesystem under /dev/shm to be able to use POSIX shared
-memory. Adding the following line to /etc/fstab should take care of
-things:
-

Documentation/Changes line 119

Quoted 10 lines

Documentation/Changes +0 -7

-none		/dev/shm	shm		defaults	0 0
-
-Remember to create the directory that you intend to mount shm on if
-necessary (The entry is automagically created if you use devfs). You
-can set limits for the number of blocks and inodes used by the
-filesystem with the mount options nr_blocks and nr_inodes.
-
 The Logical Volume Manager (LVM) is now in the kernel.  If you want to
 use this, you'll need to install the necessary LVM toolset.

What happened with this being a management tool for shared memory segments?!

-hpa

Signature · 7 lines

<[email protected]> at work, <[email protected]> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

Christoph Rohland

2001-02-02 09:53 UTC #3 lore raw

Re: [patch] tmpfs for 2.4.1

In reply to J . A . Magallon

Quoted 2 lines · H. Peter Anvin

What happened with this being a management tool for shared memory segments?!

Unfortunately we lost this ability in the 2.4.0-test series. SYSV shm now works only on an internal mounted instance and does not link the directory entry to the deleted state of the segment.

IMNSHO the new implementation is so much cleaner that it was worth it. Probably we should fix ipcrm to be more flexible.

Greetings
Christoph

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

J . A . Magallon

author

2001-02-02 20:53 UTC #4 lore raw

Re: [patch] tmpfs for 2.4.1

In reply to J . A . Magallon

Quoted 9 lines · On 02.02 Christoph Rohland wrote:

Quoted 2 lines · H. Peter Anvin
What happened with this being a management tool for shared memory segments?!
Unfortunately we lost this ability in the 2.4.0-test series. SYSV shm now works only on an internal mounted instance and does not link the directory entry to the deleted state of the segment.

Mmmmmm, does this mean that mounting /dev/shm is no more needed ? One step more towards easy 2.2 <-> 2.4 switching...

Signature · 9 lines

J.A. Magallon                                                      $> cd pub
mailto:[email protected]                                          $> more beer

Linux werewolf 2.4.1-ac1 #2 SMP Fri Feb 2 00:19:04 CET 2001 i686

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

H. Peter Anvin

2001-02-02 20:56 UTC #5 lore raw

Re: [patch] tmpfs for 2.4.1

In reply to J . A . Magallon

Quoted 7 lines · J . A . Magallon

Quoted 5 lines · On 02.02 Christoph Rohland wrote:
Quoted 2 lines · H. Peter Anvin
What happened with this being a management tool for shared memory segments?!

Quoted 8 lines

Quoted 4 lines
Unfortunately we lost this ability in the 2.4.0-test series. SYSV shm now works only on an internal mounted instance and does not link the directory entry to the deleted state of the segment.
Mmmmmm, does this mean that mounting /dev/shm is no more needed ? One step more towards easy 2.2 <-> 2.4 switching...

In some ways it's kind of sad. I found the /dev/shm interface to be rather appealing :)

-hpa

Signature · 7 lines

<[email protected]> at work, <[email protected]> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

J . A . Magallon

author

2001-02-03 00:07 UTC #6 lore raw

Re: [patch] tmpfs for 2.4.1

In reply to J . A . Magallon

Quoted 13 lines · On 02.02 H. Peter Anvin wrote:

Quoted 12 lines · J . A . Magallon
Quoted 9 lines · On 02.02 Christoph Rohland wrote:
Quoted 2 lines · H. Peter Anvin
What happened with this being a management tool for shared memory segments?!
Unfortunately we lost this ability in the 2.4.0-test series. SYSV shm now works only on an internal mounted instance and does not link the directory entry to the deleted state of the segment.

Quoted 7 lines

Quoted 3 lines
Mmmmmm, does this mean that mounting /dev/shm is no more needed ? One step more towards easy 2.2 <-> 2.4 switching...
In some ways it's kind of sad. I found the /dev/shm interface to be rather appealing :)

I did not get the chance to deal too much with it, but apart from moving functionality from userspace (ipcs) to kernel (ls), what were/could be the benefits of /dev/shm ?. Can you create a shared memory segment by simply creating a file there, or it is just a picture of what is in kernelspace?.

First time I saw that I thought: what could happen if /dev/shm is shared in a cluster ? or, lets suppose that /dev/shm is a logical volume made by addition of some nfs mounted volumes, one of each node, so one piece of the shm fs is local and other remote...kinda DSM/NUMA...?

(just too much marijuana late at night...)

Signature · 9 lines

J.A. Magallon                                                      $> cd pub
mailto:[email protected]                                          $> more beer

Linux werewolf 2.4.1-ac1 #2 SMP Fri Feb 2 00:19:04 CET 2001 i686

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

Christoph Rohland

2001-02-03 14:25 UTC #7 lore raw

Re: [patch] tmpfs for 2.4.1

In reply to J . A . Magallon

Quoted 2 lines · H. Peter Anvin

Quoted 2 lines
Mmmmmm, does this mean that mounting /dev/shm is no more needed ? One step more towards easy 2.2 <-> 2.4 switching...

Yes, it is no longer needed. You will need for POSIX shm, but there are not a lot of program out there using it.

Quoted 2 lines

In some ways it's kind of sad. I found the /dev/shm interface to be rather appealing :)

I totally agree :(

Christoph

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

Christoph Rohland

2001-02-03 14:58 UTC #8 lore raw

Re: [patch] tmpfs for 2.4.1

In reply to J . A . Magallon

Quoted 4 lines · J . A . Magallon

I did not get the chance to deal too much with it, but apart from moving functionality from userspace (ipcs) to kernel (ls), what were/could be the benefits of /dev/shm ?. Can you create a shared memory segment by simply creating a file there, or it is just a picture of what is in kernelspace?.

The most appealing thing to me was rm -f /dev/shm/.IPC* :-) So I should make a patch to ipcrm to allow multiple segments (and wildcards?).

You could not create SYSV shm segments with open, but you could delete them with rm and list the with ls.

Quoted 4 lines

First time I saw that I thought: what could happen if /dev/shm is shared in a cluster ? or, lets suppose that /dev/shm is a logical volume made by addition of some nfs mounted volumes, one of each node, so one piece of the shm fs is local and other remote...kinda DSM/NUMA...?

No, this was never possible. It was only a fs interface to local kernel objects (and still is).

Quoted 1 line

(just too much marijuana late at night...)

Oh, you are allowed to dream ;-)

Greetings
Christoph

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

H. Peter Anvin

2001-02-03 20:28 UTC #9 lore raw

Re: [patch] tmpfs for 2.4.1

In reply to J . A . Magallon

Quoted 9 lines · Christoph Rohland

Quoted 2 lines · H. Peter Anvin
Quoted 2 lines
Mmmmmm, does this mean that mounting /dev/shm is no more needed ? One step more towards easy 2.2 <-> 2.4 switching...
Yes, it is no longer needed. You will need for POSIX shm, but there are not a lot of program out there using it.

Do you need it for POSIX shm or not... if so, I would say you do need it (even if it's going to take some time until POSIX shm becomes widely used.)

-hpa

Signature · 7 lines

<[email protected]> at work, <[email protected]> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

J . A . Magallon

author

2001-02-03 21:16 UTC #10 lore raw

Re: [patch] tmpfs for 2.4.1

In reply to J . A . Magallon

Quoted 7 lines · On 02.03 H. Peter Anvin wrote:

Quoted 6 lines · Christoph Rohland
Quoted 2 lines · H. Peter Anvin
Quoted 2 lines
Mmmmmm, does this mean that mounting /dev/shm is no more needed ? One step more towards easy 2.2 <-> 2.4 switching...

Quoted 8 lines

Quoted 3 lines
Yes, it is no longer needed. You will need for POSIX shm, but there are not a lot of program out there using it.
Do you need it for POSIX shm or not... if so, I would say you do need it (even if it's going to take some time until POSIX shm becomes widely used.)

There was a post recently (that now I can't find), that said the shm management was done with an interal fs. Was that Posix or sysv shm ?

Signature · 9 lines

J.A. Magallon                                                      $> cd pub
mailto:[email protected]                                          $> more beer

Linux werewolf 2.4.1-ac2 #1 SMP Sat Feb 3 10:45:59 CET 2001 i686

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

Christoph Rohland

2001-02-04 09:14 UTC #11 lore raw

Re: [patch] tmpfs for 2.4.1

In reply to J . A . Magallon

Quoted 3 lines · H. Peter Anvin

Do you need it for POSIX shm or not... if so, I would say you do need it (even if it's going to take some time until POSIX shm becomes widely used.)

Yes, you need it. glibc 2.2 will search for a shm fs on shm_open. And without it fails. And the recommendation in the Configure.help is to mount it.

But you do not need to mount it for any pre 2.4 functionality.

Greetings
Christoph

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

Christoph Rohland

2001-02-04 09:49 UTC #12 lore raw

Re: [patch] tmpfs for 2.4.1

In reply to J . A . Magallon

Quoted 2 lines · J . A . Magallon

There was a post recently (that now I can't find), that said the shm management was done with an interal fs. Was that Posix or sysv shm ?

SYSV shm and shared anonymous mappings are using a kern_mount of shm/tmpfs. So the CONFIG_TMPFS does only make the directory/read/write-handling conditional. The rest is unconditionally compiled into the kernel.

Greetings
Christoph

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

Christoph Rohland

2001-02-06 09:13 UTC #13 lore raw

Re: VM question (ramfs abuse)

In reply to J . A . Magallon

Hi Paul,

Quoted 15 lines · On Mon, 5 Feb 2001, [email protected] wrote:

Christoph Rohland <[email protected]>, on Sun Feb 04, 2001 [10:53:26 AM] said:
@>Paul <[email protected]> writes:
@>> I finally managed to coax the cursor over to mutt and quit it. Then things @>> were instantly fine and I could remove 'blob'.
@>> My question is, why wasnt any swap used during this time? Ramfs
@>> may not have any backing store?
@>
@>Because RAMFS lives in _physical_ ram. Grab my tmpfs patch and you @>will have ramfs + swapping and accounting. But set a limit (Mount @>option size) to it before doing anything like
@>'dd if=/dev/zero of=blob' ;-)
@>
	Dear Christoph;

	First, thankyou for the reply. Heh. Meant to send to the list,

Quoted 9 lines

but got only you. Oh well.
I have been testing tmpfs on 2.4.1. I havent encountered any
problems, but I have a question. (with a limit, the pathological dd worked just fine :) For fun, I did a
'time (make dep && make clean && make bzImage)' using 2.2.18, 2.4.1, and various fs's as /tmp. (reiser, ext2, ramfs, tmpfs) Both using and not using the -pipe gcc option during the build.
I was somewhat suprised that each time was the same (within, like
1/10th of a percent)

You probably have to stress some more since the page cache is _very_ effective on Linux. I can bring down a kernel compile on a 8way machine with parallel make from ~44 seconds to ~40 seconds which is a real speedup ;-)

Actually I really doubt a real speed benefit for normal users with tmpfs but there may be other benefits:

- 'add swap on any device and your tmpfs grows' can be really valuable.
- Automatic cleanup of /tmp on reboot
- No traces of temporary files on backing store may help the crypto people. (But no if you swap that argument is hosed.)
- ???

There are some people out there which really wanted to have this and it was a minor task to add the full support to shm fs.

Greetings
Christoph

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/