cramfs filesystem patch

Here's a patch for the cramfs filesystem. Lots of improvements and a new cramfsck program, see below for the full list of changes.

It only modifies cramfs code (aside from adding cramfs to struct super_block) and aims to be completely backward-compatible. All old cramfs images will still work and new cramfs images are mountable on old kernels if you avoid using holes (requires 2.3.39 or later) and you don't pad the beginning of the cramfs filesystem for a boot sector (and presumably, you have a new kernel in that case).

Most of this code has been fairly well-tested, but a few features are newer and less well-tested (like the directory sorting).

If you use cramfs, please try it out and let me know if you have any feedback or comments (please Cc me if you respond here).

Dan

------------------------------------------------------------------------

superblock changes:
- revised fsid field (still unique, contains useful fs info:
a CRC, edition number for implementation-specific versioning, block and file counts for statfs)
- size field is now used
- new feature flags (fsid version 2, holes, shifted root for boot loader, directory sorting)

inode.c
- proper checking for device size (torvalds)
- speed up IO a bit - read in the block asynchronously rather than using bread() on them one by one (torvalds)
- early directory lookup exit (since directories are now sorted)
- add reporting of blocks and blksize for statfs
- removed superfluous test for cramfs signature

cramfsck.c
- new program to fsck/extract cramfs images

mkcramfs.c
- CRC added to filesystem image
- allow holes to be created (see -z option)
- support for a cramfs boot loader (see -p option)
- support for inserting an kernel image into a cramfs image (see -i option)
- directory sorting (more consistent images, allows faster lookups)
- bug fix: allocate enough RAM for the fs image (small images)
- bug fix: work around a bug in ramfs where it would incorrectly report the number of blocks for file (a problem when creating cramfs images from a ramfs filesystem)
- options to set name/edition number in superblock (-n and -e, respectively)

cramfs.txt:
- added entry for /etc/magic to cramfs.txt documentation

cleanup:
- moved width definitions to cramfs.h
- define special cramfs super block

diff -ur linux-2.4.0-test10.orig/Documentation/filesystems/cramfs.txt linux-2.4.0-test10/Documentation/filesystems/cramfs.txt
@@ -47,6 +47,21 @@
 mind the filesystem becoming unreadable to future kernels.
 
 
+For /usr/share/magic
+------------------
+
+0	long	0x28cd3d45	Linux cramfs
+>4	long	x		size %d
+>8	long	x		flags 0x%x
+>12	long	x		future 0x%x
+>16	string	>\0		signature "%.16s"
+>32	long	x		fsid.crc 0x%x
+>36	long	x		fsid.edition %d
+>40	long	x		fsid.blocks %d
+>44	long	x		fsid.files %d
+>48	string	>\0		name "%.16s"
+
+
 Hacker Notes
 ------------
 

@@ -1,7 +1,7 @@
-CFLAGS = -Wall -O2
+CFLAGS = -Wall -O2 -g
 CPPFLAGS = -I../../fs/cramfs
 LDLIBS = -lz
-PROGS = mkcramfs
+PROGS = mkcramfs cramfsck
 
 all: $(PROGS)
 

@@ -634,6 +634,7 @@
 #include <linux/udf_fs_sb.h>
 #include <linux/ncp_fs_sb.h>
 #include <linux/usbdev_fs_sb.h>
+#include <linux/cramfs_fs_sb.h>
 
 extern struct list_head super_blocks;
 

@@ -681,6 +682,7 @@
 		struct udf_sb_info	udf_sb;
 		struct ncp_sb_info	ncpfs_sb;
 		struct usbdev_sb_info   usbdevfs_sb;
+		struct cramfs_sb_info	cramfs_sb;
 		void			*generic_sbp;
 	} u;
 	/*

@@ -0,0 +1,14 @@
+#ifndef _CRAMFS_FS_SB
+#define _CRAMFS_FS_SB
+
+/*
+ * cramfs super-block data in memory
+ */
+struct cramfs_sb_info {
+			unsigned long size;
+			unsigned long blocks;
+			unsigned long files;
+			unsigned long flags;
+};
+
+#endif

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

I've read the patch, and the added benefits of sorting seem usefull. I have some general questions with regard to direction for Cramfs while the subject is broached however.

On the Agenda we use cramfs as our storage mechanism, with some linear addressing patches since it's on flash, and the memory copies inherent with a block device are unnecesary in our case. (And we kill the buffer of compressed data in cramfs, as it's unnecessary as well) Going forward though we're working on some other modifications of the the cramfs filesystem that allow it todo some other stuff that requires an expansion of the inode, Linus mentions in the README for cramfs that it may be usefull to expand it, but I wanted to get peoples reactions on expanding it by 32bits, and using a part of that to denote file information like type of compression, etc.

I had worked previously on hacking in LZO compression into cramfs, but hadn't had time to finish when called onto other things. The other thing we're working on with ROMDISK is putting in XIP mode for binaries/libs that we're going to leave uncompressed, and aligned so they don't need to be copied to memory before executing. I'd like to integrate XIP into cramfs at some point to allow the flash releases from agenda to be released as a single unit, but doing so necesitates an expansion of the inode to denote this information about certain executables/libs. So what do you all think of expanding the inode for this purpose, and is loss of backward compatibility a massivly evil thing, so when I make this patch I need to keep information based on the magic of the cramfs filesystem to denote versioning information to allow for the difference in the inode.

Thanks,
Shane.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

I'd like to see these patches as well. They may be useful on the iPAQ (and similar hardware like my Yopy here... ;-)

I wish some hardware vendor out there would build an x86 box that used memory addressable flash from 0 up and RAM up higher. A simple Linux kernel boot loader could then replace the BIOS. Flash would be less expensive than 8086 lower memory windowed flash hacks.

Has any work been done on e2compr in 2.4 yet?

Tim Riker - http://rikers.org/ - short SIGs! <g>
All I need to know I could have learned in Kindergarten
... if I'd just been paying attention.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/

Patch inserted. It's not ready for kernel inclusion or anything..., there are some remaining issues to fix that I haven't figured out good ways to handle yet. For instance, we still use a /dev/rom to bootstrap the kernel as to which file system module to pull up. Then all the read/writes completely by pass /dev/rom. Silly, I know, but I haven't figured out a smart way to deal with this particular problem.

Sorry for the late reply..., your message got lost in a sea of mail. You might find some other interesting stuff on the ftp.agendacomputing.com ftp site. XIP MTD flash read/write, which I haven't forward ported to MTDs CVS yet. (Working on it) But that would allow if you guys can get XIP kernel working in ARM like we have in MIPS to read and write to flash. Other randomn stuff, and things get added with a slightly normal frequency. One benefit with this patch is copying, the other is dumping that 32k buffer, and giving it back to pageable memory.

Thanks,
Shane.

diff -urN linux.orig/fs/Config.in linux/fs/Config.in
@@ -29,6 +29,10 @@
 	int 'JFFS debugging verbosity (0 = quiet, 3 = noisy)' CONFIG_JFFS_FS_VERBOSE 0
 fi
 tristate 'Compressed ROM file system support' CONFIG_CRAMFS
+dep_mbool 'Linear addressing for CRAMFS' CONFIG_CRAMFS_LINEAR_ADDRESSING $CONFIG_CRAMFS
+if [ "$CONFIG_CRAMFS_LINEAR_ADDRESSING" != "n" ] ; then
+	hex 'Starting address for CRAMFS filesystem' CONFIG_CRAMFS_LA_ADDRESS bf200008
+fi
 tristate 'Simple RAM-based file system support' CONFIG_RAMFS
 
 tristate 'ISO 9660 CDROM file system support' CONFIG_ISO9660_FS

diff -urN linux.orig/fs/cramfs/inode.c linux/fs/cramfs/inode.c
@@ -11,6 +11,20 @@
  * The actual compression is based on zlib, see the other files.
  */
 
+/* Linear Addressing code
+ *
+ * Copyright (C) 2000 Shane Nay.
+ *
+ * Allows you to have a linearly addressed cramfs filesystem.
+ * Saves the need for buffer, and the munging of the buffer.
+ * Savings a bit over 32k with default PAGE_SIZE, BUFFER_SIZE
+ * etc.  Usefull on embedded platform with ROM :-).
+ *
+ * Downsides- Currently linear addressed cramfs partitions
+ * don't co-exist with block cramfs partitions.
+ *
+ */
+
 #include <linux/module.h>
 #include <linux/fs.h>
 #include <linux/pagemap.h>

@@ -68,6 +82,23 @@
 	return inode;
 }
 
+#if defined(CONFIG_CRAMFS_CRAMFS_LINEAR_ADDRESSING) && defined (CONFIG_CRAMFS_LA_ADDRESS)
+
+/*
+ * Returns a pointer to the linearly addressed filesystem.
+ * Simple byte size pointer addition.
+ */
+static unsigned char* romdisk_top=(unsigned char*) CONFIG_CRAMFS_LA_ADDRESS;
+
+static void *cramfs_read(struct super_block *sb, unsigned int offset, unsigned int len)
+{
+	if (!len)
+		return NULL;
+	return romdisk_top + offset;
+}
+
+#else /* !CONFIG_CRAMFS_LINEAR_ADDRESSING aka Regular block mode */
+
 /*
  * We have our own block cache: don't fill up the buffer cache
  * with the rom-image, because the way the filesystem is set

@@ -149,6 +180,8 @@
 	}
 	return read_buffers[buffer] + offset;
 }
+
+#endif
 			
 
 static struct super_block * cramfs_read_super(struct super_block *sb, void *data, int silent)

@@ -161,10 +194,11 @@
 	set_blocksize(sb->s_dev, PAGE_CACHE_SIZE);
 	sb->s_blocksize = PAGE_CACHE_SIZE;
 	sb->s_blocksize_bits = PAGE_CACHE_SHIFT;
 
+#if !( defined(CONFIG_CRAMFS_CRAMFS_LINEAR_ADDRESSING) && defined (CONFIG_CRAMFS_LA_ADDRESS) )
 	/* Invalidate the read buffers on mount: think disk change.. */
 	for (i = 0; i < READ_BUFFERS; i++)
 		buffer_blocknr[i] = -1;
+#endif
 
 	/* Read the first block and get the superblock from it */

memcpy(&super, cramfs_read(sb, 0, sizeof(super)), sizeof(super));

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/