iso9660 inodes beyond 4GB

PATCH In review Paul Serice 2 participants 3 messages May 31, 2004 – Jun 1, 2004 Isofs filesystem

Share & tools

Paul Serice

author

2004-05-31 17:54 UTC #0 lore raw

PATCH 1/1 iso9660 inodes beyond 4GB

This is my third attempt to patch the isofs code. It is a completely different patch from the previous two. I do not think there are any trade-offs with this patch.

The problem it addresses is that the current iso9660 file system cannot reach inodes located beyond the 4GB barrier. This is caused by using the inode number as the byte offset of the inode data. Being 32-bits wide, the inode number is unable to reach inode data that does not reside on the first 4GB of the file system.

This causes real problems with "growisofs"

    http://fy.chalmers.se/~appro/linux/DVD+RW/#isofs4gb

and my pet project "shunt"

http://www.serice.net/shunt/

This patch switches the isofs code from iget() to iget5_locked() which allows extra data to be passed into isofs_read_inode() so that inode data anywhere on the disk can be reached.

The inode number scheme was also changed. Continuing to use the byte offset would have resulted in non-unique inodes in many common situations, but because the inode number no longer plays any role in reading the meta-data off the disk, I was free to set the inode number to some unique characteristic of the file. I have chosen to use the block offset which is also 32-bits wide.

As a practical matter, these new inode numbers should always be unique for directories (which to me is the important case). They will also be unique for other file types provided you do not have a specially mastered image that has two files that share the same starting block (which is allowed by the standard). Compared to the current scheme, this should be a substantial reduction in the likelihood of assigning non-unique inode numbers.

Lastly, the current code uses the default export_operations to handle accessing the file system through NFS. The problem with this is that the default NFS operations assume that iget() works which is no longer the case because of the necessary switch to iget5_locked(). So, I had to implement the NFS operations too.

I did not, however, implement the NFS get_parent() method. So, the default method which just returns an error is used. This is not a reduction in functionality because the current code also uses the default method. If someone can explain what I need to do to trigger this error, I will gladly try to implement the method.

Thanks,
Paul Serice

fs/isofs/Makefile +1 -1

diff -uprN -X dontdiff linux-2.6.7-rc2/fs/isofs/Makefile linux-2.6.7-rc2-isofs.3/fs/isofs/Makefile
@@ -4,7 +4,7 @@
 
 obj-$(CONFIG_ISO9660_FS) += isofs.o
 
-isofs-objs-y 			:= namei.o inode.o dir.o util.o rock.o
+isofs-objs-y 			:= namei.o inode.o dir.o util.o rock.o export.o
 isofs-objs-$(CONFIG_JOLIET)	+= joliet.o
 isofs-objs-$(CONFIG_ZISOFS)	+= compress.o
 isofs-objs			:= $(isofs-objs-y)

fs/isofs/dir.c +3 -3

diff -uprN -X dontdiff linux-2.6.7-rc2/fs/isofs/dir.c linux-2.6.7-rc2-isofs.3/fs/isofs/dir.c
@@ -106,8 +106,8 @@ static int do_isofs_readdir(struct inode
 {
 	unsigned long bufsize = ISOFS_BUFFER_SIZE(inode);
 	unsigned char bufbits = ISOFS_BUFFER_BITS(inode);
-	unsigned int block, offset;
-	int inode_number = 0;	/* Quiet GCC */
+	unsigned long block, offset;
+	unsigned long inode_number = 0;	/* Quiet GCC */
 	struct buffer_head *bh = NULL;
 	int len;
 	int map;

@@ -130,7 +130,7 @@ static int do_isofs_readdir(struct inode
 
 		de = (struct iso_directory_record *) (bh->b_data + offset);
 		if (first_de)
-			inode_number = (bh->b_blocknr << bufbits) + offset;
+			inode_number = isofs_get_ino(de);
 
 		de_len = *(unsigned char *) de;

fs/isofs/export.c +142 -0

142 lines not on this page · raw message

fs/isofs/inode.c +103 ~~-50~~

305 lines not on this page · raw message

fs/isofs/namei.c +18 -9

diff -uprN -X dontdiff linux-2.6.7-rc2/fs/isofs/namei.c linux-2.6.7-rc2-isofs.3/fs/isofs/namei.c
@@ -17,6 +17,7 @@
 #include <linux/config.h>	/* Joliet? */
 #include <linux/smp_lock.h>
 #include <linux/buffer_head.h>
+#include <linux/dcache.h>
 
 #include <asm/uaccess.h>

@@ -59,12 +60,12 @@ isofs_cmp(struct dentry * dentry, const 
  */
 static unsigned long
 isofs_find_entry(struct inode *dir, struct dentry *dentry,
+	unsigned long *block_rv, unsigned long* offset_rv,
 	char * tmpname, struct iso_directory_record * tmpde)
 {
-	unsigned long inode_number;
 	unsigned long bufsize = ISOFS_BUFFER_SIZE(dir);
 	unsigned char bufbits = ISOFS_BUFFER_BITS(dir);
-	unsigned int block, f_pos, offset;
+	unsigned long block, f_pos, offset, block_saved, offset_saved;
 	struct buffer_head * bh = NULL;
 	struct isofs_sb_info *sbi = ISOFS_SB(dir->i_sb);

@@ -87,7 +88,6 @@ isofs_find_entry(struct inode *dir, stru
 		}
 
 		de = (struct iso_directory_record *) (bh->b_data + offset);
-		inode_number = (bh->b_blocknr << bufbits) + offset;
 
 		de_len = *(unsigned char *) de;
 		if (!de_len) {

@@ -99,6 +99,8 @@ isofs_find_entry(struct inode *dir, stru
 			continue;
 		}
 
+		block_saved = bh->b_blocknr;
+		offset_saved = offset;
 		offset += de_len;
 		f_pos += de_len;

@@ -151,7 +153,9 @@ isofs_find_entry(struct inode *dir, stru
 		}
 		if (match) {
 			if (bh) brelse(bh);
-			return inode_number;
+                        *block_rv = block_saved;
+                        *offset_rv = offset_saved;
+			return 1;
 		}
 	}
 	if (bh) brelse(bh);

@@ -160,7 +164,8 @@ isofs_find_entry(struct inode *dir, stru
 
 struct dentry *isofs_lookup(struct inode * dir, struct dentry * dentry, struct nameidata *nd)
 {
-	unsigned long ino;
+	int found;
+	unsigned long block, offset;
 	struct inode *inode;
 	struct page *page;

@@ -171,19 +176,23 @@ struct dentry *isofs_lookup(struct inode
 		return ERR_PTR(-ENOMEM);
 
 	lock_kernel();
-	ino = isofs_find_entry(dir, dentry, page_address(page),
-			       1024 + page_address(page));
+	found = isofs_find_entry(dir, dentry,
+				 &block, &offset,
+				 page_address(page),
+				 1024 + page_address(page));
 	__free_page(page);
 
 	inode = NULL;
-	if (ino) {
-		inode = iget(dir->i_sb, ino);
+	if (found) {
+		inode = isofs_iget(dir->i_sb, block, offset);
 		if (!inode) {
 			unlock_kernel();
 			return ERR_PTR(-EACCES);
 		}
 	}
 	unlock_kernel();
+	if (inode)
+		return d_splice_alias(inode, dentry);
 	d_add(dentry, inode);
 	return NULL;
 }

fs/isofs/rock.c +7 -8

diff -uprN -X dontdiff linux-2.6.7-rc2/fs/isofs/rock.c linux-2.6.7-rc2-isofs.3/fs/isofs/rock.c
@@ -306,9 +306,7 @@ int parse_rock_ridge_inode_internal(stru
 	goto out;
       case SIG('C','L'):
 	ISOFS_I(inode)->i_first_extent = isonum_733(rr->u.CL.location);
-	reloc = iget(inode->i_sb,
-		     (ISOFS_I(inode)->i_first_extent <<
-		      ISOFS_SB(inode->i_sb)->s_log_zone_size));
+	reloc = isofs_iget(inode->i_sb, ISOFS_I(inode)->i_first_extent, 0);
 	if (!reloc)
 		goto out;
 	inode->i_mode = reloc->i_mode;

@@ -447,15 +445,15 @@ int parse_rock_ridge_inode(struct iso_di
 static int rock_ridge_symlink_readpage(struct file *file, struct page *page)
 {
 	struct inode *inode = page->mapping->host;
+        struct iso_inode_info *ei = ISOFS_I(inode);
 	char *link = kmap(page);
 	unsigned long bufsize = ISOFS_BUFFER_SIZE(inode);
-	unsigned char bufbits = ISOFS_BUFFER_BITS(inode);
 	struct buffer_head *bh;
 	char *rpnt = link;
 	unsigned char *pnt;
 	struct iso_directory_record *raw_inode;
 	CONTINUE_DECLS;
-	int block;
+	unsigned long block, offset;
 	int sig;
 	int len;
 	unsigned char *chr;

@@ -464,20 +462,21 @@ static int rock_ridge_symlink_readpage(s
 	if (!ISOFS_SB(inode->i_sb)->s_rock)
 		panic ("Cannot have symlink with high sierra variant of iso filesystem\n");
 
-	block = inode->i_ino >> bufbits;
+	block = ei->i_iget5_block;
 	lock_kernel();
 	bh = sb_bread(inode->i_sb, block);
 	if (!bh)
 		goto out_noread;
 
-	pnt = (unsigned char *) bh->b_data + (inode->i_ino & (bufsize - 1));
+        offset = ei->i_iget5_offset;
+	pnt = (unsigned char *) bh->b_data + offset;
 
 	raw_inode = (struct iso_directory_record *) pnt;
 
 	/*
 	 * If we go past the end of the buffer, there is some sort of error.
 	 */
-	if ((inode->i_ino & (bufsize - 1)) + *pnt > bufsize)
+	if (offset + *pnt > bufsize)
 		goto out_bad_span;
 
 	/* Now test for possible Rock Ridge extensions which will override

include/linux/iso_fs.h +10 -0

diff -uprN -X dontdiff linux-2.6.7-rc2/include/linux/iso_fs.h linux-2.6.7-rc2-isofs.3/include/linux/iso_fs.h
@@ -231,9 +231,19 @@ extern struct dentry *isofs_lookup(struc
 extern struct buffer_head *isofs_bread(struct inode *, sector_t);
 extern int isofs_get_blocks(struct inode *, sector_t, struct buffer_head **, unsigned long);
 
+extern struct inode *isofs_iget(struct super_block *sb,
+                                unsigned long block,
+                                unsigned long offset);
+
+static inline unsigned long isofs_get_ino(struct iso_directory_record *d)
+{
+	return (unsigned long)isonum_733(d->extent);
+}
+
 extern struct inode_operations isofs_dir_inode_operations;
 extern struct file_operations isofs_dir_operations;
 extern struct address_space_operations isofs_symlink_aops;
+extern struct export_operations isofs_export_ops;
 
 /* The following macros are used to check for memory leaks. */
 #ifdef LEAK_CHECK

include/linux/iso_fs_i.h +4 -1

diff -uprN -X dontdiff linux-2.6.7-rc2/include/linux/iso_fs_i.h linux-2.6.7-rc2-isofs.3/include/linux/iso_fs_i.h
@@ -13,10 +13,13 @@ enum isofs_file_format {
  * iso fs inode data in memory
  */
 struct iso_inode_info {
+	unsigned long i_iget5_block;
+	unsigned long i_iget5_offset;
 	unsigned int i_first_extent;
 	unsigned char i_file_format;
 	unsigned char i_format_parm[3];
-	unsigned long i_next_section_ino;
+	unsigned long i_next_section_block;
+	unsigned long i_next_section_offset;
 	off_t i_section_size;
 	struct inode vfs_inode;
 };

Neil Brown

2004-06-01 04:18 UTC #1 lore raw

Quoted 3 lines · On Monday May 31, [email protected] wrote:

This is my third attempt to patch the isofs code. It is a completely different patch from the previous two. I do not think there are any trade-offs with this patch.

snip

Quoted 7 lines

Lastly, the current code uses the default export_operations to handle accessing the file system through NFS. The problem with this is that the default NFS operations assume that iget() works which is no longer the case because of the necessary switch to iget5_locked(). So, I had to implement the NFS operations too.

Quoted 6 lines

I did not, however, implement the NFS get_parent() method. So, the default method which just returns an error is used. This is not a reduction in functionality because the current code also uses the default method. If someone can explain what I need to do to trigger this error, I will gladly try to implement the method.

The easiest way to trigger the error is:
export the filesystem and mount it on some client. On the client, cd into some subdirectory. On the server, unexport, unmount, remount, reexport
(i.e. simulate a reboot).
On the client "ls -l"

That should cause the server to try to call get_parent to find the path back up to the root of the filesystem.

A quick glance at the rest of the export related code suggests that it is all ok.

NeilBrown

Paul Serice

author

2004-06-01 13:43 UTC #2 lore raw

Quoted 9 lines

The easiest way to trigger the error is:
export the filesystem and mount it on some client. On the client, cd into some subdirectory. On the server, unexport, unmount, remount, reexport
(i.e. simulate a reboot).
On the client "ls -l"
That should cause the server to try to call get_parent to find the path back up to the root of the filesystem.

Thanks, that definitely triggers an error.

Quoted 2 lines

A quick glance at the rest of the export related code suggests that it is all ok.

I'm not sure who I have to thank, but the code in fs/exportfs made it much easier than expected.

Paul Serice