Skip to contentSkip to conversation

LBD fixes for Linux 2.6.10 [2/2]

RFC PATCH In review Tony Battersby 3 participants 4 messages Jan 11, 2005 – Jan 13, 2005

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 LBD fixes for Linux 2.6.10 [2/2]

Here is an "example" patch to fix some of the LBD issues with various filesystems (ext3, reiserfs, xfs). Unfortunately it looks like there are many more LBD problems with the filesystems that I didn't fix, so I am just calling this an "example" patch that shows some of what needs to be done, but doesn't fix everything. I am not a filesystem expert, so I would prefer to leave it to others to do a full audit of all the filesystems for LBD-compliance. Specifically there are two things to check:

1) Use sector_t rather than long or unsigned long for block numbers. For auditing, just search for all occurrences of "long" in the filesystem code and make sure it is not referring to a block number.

2) Make sure page->index is typecasted to either loff_t or sector_t before shifting left (e.g. "(sector_t) page->index << (PAGE_CACHE_SHIFT - s->s_blocksize_bits)"). For auditing, just search for "PAGE_CACHE_SHIFT".

It looks like XFS is pretty much OK with regards to LBD-compliance, but ext3 and reiserfs are definitely not. I haven't really investigated other filesystems, so I don't know about any others. If a filesystem cannot be made LBD-compliant (whether for technical reasons or because of lack of interest), could we at least make sure that the fact is documented somewhere and that mkfs or mount fails if the partition extends past sector 0xffffffff?

I haven't tested these changes at all, so please review and comment.

Anthony J. Battersby
Cybernetics

fs/ext3/inode.c +5 -3
diff -urpN linux-2.6.10/fs/ext3/inode.c linux-2.6.10-lbd-fix/fs/ext3/inode.c
@@ -292,7 +292,7 @@ static inline int verify_chain(Indirect 
  */
 
 static int ext3_block_to_path(struct inode *inode,
-			long i_block, int offsets[4], int *boundary)
+			sector_t i_block, int offsets[4], int *boundary)
 {
 	int ptrs = EXT3_ADDR_PER_BLOCK(inode->i_sb);
 	int ptrs_bits = EXT3_ADDR_PER_BLOCK_BITS(inode->i_sb);
@@ -1595,7 +1595,8 @@ static int ext3_block_truncate_page(hand
 {
 	unsigned long index = from >> PAGE_CACHE_SHIFT;
 	unsigned offset = from & (PAGE_CACHE_SIZE-1);
-	unsigned blocksize, iblock, length, pos;
+	unsigned blocksize, length, pos;
+	sector_t iblock;
 	struct inode *inode = mapping->host;
 	struct buffer_head *bh;
 	int err;
@@ -1603,7 +1604,8 @@ static int ext3_block_truncate_page(hand
 
 	blocksize = inode->i_sb->s_blocksize;
 	length = blocksize - (offset & (blocksize - 1));
-	iblock = index << (PAGE_CACHE_SHIFT - inode->i_sb->s_blocksize_bits);
+	iblock = (sector_t) index <<
+	         (PAGE_CACHE_SHIFT - inode->i_sb->s_blocksize_bits);
 
 	if (!page_has_buffers(page))
 		create_empty_buffers(page, blocksize, 0);
fs/reiserfs/inode.c +8 -8
diff -urpN linux-2.6.10/fs/reiserfs/inode.c linux-2.6.10-lbd-fix/fs/reiserfs/inode.c
@@ -203,10 +203,10 @@ static inline void set_block_dev_mapped 
 // files which were created in the earlier version can not be longer,
 // than 2 gb
 //
-static int file_capable (struct inode * inode, long block)
+static int file_capable (struct inode * inode, sector_t block)
 {
     if (get_inode_item_key_version (inode) != KEY_FORMAT_3_5 || // it is new file.
-	block < (1 << (31 - inode->i_sb->s_blocksize_bits))) // old file, but 'block' is inside of 2gb
+	block < (1U << (31 - inode->i_sb->s_blocksize_bits))) // old file, but 'block' is inside of 2gb
 	return 1;
 
     return 0;
@@ -244,7 +244,7 @@ static int file_capable (struct inode * 
 // Please improve the english/clarity in the comment above, as it is
 // hard to understand.
 
-static int _get_block_create_0 (struct inode * inode, long block,
+static int _get_block_create_0 (struct inode * inode, sector_t block,
 				 struct buffer_head * bh_result,
 				 int args)
 {
@@ -559,7 +559,7 @@ out:
 }
 
 static inline int _allocate_block(struct reiserfs_transaction_handle *th,
-			   long block,
+			   sector_t block,
                            struct inode *inode, 
 			   b_blocknr_t *allocated_block_nr, 
 			   struct path * path,
@@ -2063,7 +2063,7 @@ out:
 
 static int map_block_for_writepage(struct inode *inode, 
 			       struct buffer_head *bh_result, 
-                               unsigned long block) {
+                               sector_t block) {
     struct reiserfs_transaction_handle th ;
     int fs_gen ;
     struct item_head tmp_ih ;
@@ -2074,7 +2074,7 @@ static int map_block_for_writepage(struc
     INITIALIZE_PATH(path) ;
     int pos_in_item ;
     int jbegin_count = JOURNAL_PER_BALANCE_CNT ;
-    loff_t byte_offset = (block << inode->i_sb->s_blocksize_bits) + 1 ;
+    loff_t byte_offset = ((loff_t) block << inode->i_sb->s_blocksize_bits) + 1 ;
     int retval ;
     int use_get_block = 0 ;
     int bytes_copied = 0 ;
@@ -2212,7 +2212,7 @@ static int reiserfs_write_full_page(stru
     struct inode *inode = page->mapping->host ;
     unsigned long end_index = inode->i_size >> PAGE_CACHE_SHIFT ;
     int error = 0;
-    unsigned long block ;
+    sector_t block;
     struct buffer_head *head, *bh;
     int partial = 0 ;
     int nr = 0;
@@ -2252,7 +2252,7 @@ static int reiserfs_write_full_page(stru
 	kunmap_atomic(kaddr, KM_USER0) ;
     }
     bh = head ;
-    block = page->index << (PAGE_CACHE_SHIFT - s->s_blocksize_bits) ;
+    block = (sector_t) page->index << (PAGE_CACHE_SHIFT - s->s_blocksize_bits) ;
     /* first map all the buffers, logging any direct items we find */
     do {
 	if ((checked || buffer_dirty(bh)) && (!buffer_mapped(bh) ||
fs/xfs/linux-2.6/xfs_aops.c +1 -1
diff -urpN linux-2.6.10/fs/xfs/linux-2.6/xfs_aops.c linux-2.6.10-lbd-fix/fs/xfs/linux-2.6/xfs_aops.c
@@ -71,7 +71,7 @@ xfs_page_trace(
 	bhv_desc_t	*bdp;
 	vnode_t		*vp = LINVFS_GET_VP(inode);
 	loff_t		isize = i_size_read(inode);
-	loff_t		offset = page->index << PAGE_CACHE_SHIFT;
+	loff_t		offset = (loff_t) page->index << PAGE_CACHE_SHIFT;
 	int		delalloc = -1, unmapped = -1, unwritten = -1;
 
 	if (page_has_buffers(page))
#1 lore raw
Quoted 9 lines · Tony Battersby

Here is an "example" patch to fix some of the LBD issues with various filesystems (ext3, reiserfs, xfs). Unfortunately it looks like there are many more LBD problems with the filesystems that I didn't fix, so I am just calling this an "example" patch that shows some of what needs to be done, but doesn't fix everything. I am not a filesystem expert, so I would prefer to leave it to others to do a full audit of all the filesystems for LBD-compliance. Specifically there are two things to check:

Quoted 3 lines

1) Use sector_t rather than long or unsigned long for block numbers. For auditing, just search for all occurrences of "long" in the filesystem code and make sure it is not referring to a block number.

That's actually not wise. For ext3, the block numbers _on disc_ are 32-bit in size, so making them sector_t in memory is just a waste. reiserfs seems to be the same. OTOH, XFS does use 64-bit blocks on disc, so should use an explicitly 64-bit size in memory too.

Signature · 6 lines
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain
#2 lore raw
Quoted 4 lines · Matthew Wilcox

That's actually not wise. For ext3, the block numbers _on disc_ are 32-bit in size, so making them sector_t in memory is just a waste. reiserfs seems to be the same. OTOH, XFS does use 64-bit blocks on disc, so should use an explicitly 64-bit size in memory too.

XFS has compile-time switches for 32/64 bit in memory. On Linux it's 64bit with either a 64bit host or CONFIG_LBD.

#3 lore raw
In reply to Matthew Wilcox
Quoted 10 lines
Quoted 2 lines

1) Use sector_t rather than long or unsigned long for block numbers. For auditing, just search for all occurrences of "long" in

the filesystem

Quoted 1 line

code and make sure it is not referring to a block number.

That's actually not wise. For ext3, the block numbers _on disc_ are 32-bit in size, so making them sector_t in memory is just a waste. reiserfs seems to be the same. OTOH, XFS does use 64-bit blocks on disc,
so should use an explicitly 64-bit size in memory too.

Thanks, that's why I was looking for feedback before going further. Is there anywhere in the fs-specific code that deals with real sector numbers rather than fs block numbers? Any variable that deals with real sector numbers should be sector_t, even if the filesystem uses only 32-bit block numbers. For example, an ext3 filesystem might be created in the 3-4 TB range of a 4 TB disk, in which case the fs block numbers fit into a 32-bit quantity but the real sector numbers need to be 64-bit because of the large starting offset. If the fs-specific code doesn't deal with real sector numbers, then there is probably nothing that needs to be done for ext3 or reiserfs.

Anthony J. Battersby
Cybernetics