LBD fixes for Linux 2.6.10 [2/2]
Subsystems (2)
RFC PATCH 1/1 LBD fixes for Linux 2.6.10 [2/2]
Here is an "example" patch to fix some of the LBD issues with various filesystems (ext3, reiserfs, xfs). Unfortunately it looks like there are many more LBD problems with the filesystems that I didn't fix, so I am just calling this an "example" patch that shows some of what needs to be done, but doesn't fix everything. I am not a filesystem expert, so I would prefer to leave it to others to do a full audit of all the filesystems for LBD-compliance. Specifically there are two things to check:
1) Use sector_t rather than long or unsigned long for block numbers. For auditing, just search for all occurrences of "long" in the filesystem code and make sure it is not referring to a block number.
2) Make sure page->index is typecasted to either loff_t or sector_t before shifting left (e.g. "(sector_t) page->index << (PAGE_CACHE_SHIFT - s->s_blocksize_bits)"). For auditing, just search for "PAGE_CACHE_SHIFT".
It looks like XFS is pretty much OK with regards to LBD-compliance, but ext3 and reiserfs are definitely not. I haven't really investigated other filesystems, so I don't know about any others. If a filesystem cannot be made LBD-compliant (whether for technical reasons or because of lack of interest), could we at least make sure that the fact is documented somewhere and that mkfs or mount fails if the partition extends past sector 0xffffffff?
I haven't tested these changes at all, so please review and comment.
Anthony J. Battersby
Cybernetics
fs/ext3/inode.c +5 -3
@@ -292,7 +292,7 @@ static inline int verify_chain(Indirect*/ static int ext3_block_to_path (struct inode *inode, -long i_block,int offsets[4 ],int *boundary) +sector_t i_block,int offsets[4 ],int *boundary) {int ptrs =EXT3_ADDR_PER_BLOCK (inode->i_sb);int ptrs_bits =EXT3_ADDR_PER_BLOCK_BITS (inode->i_sb);
@@ -1595,7 +1595,8 @@ static int ext3_block_truncate_page(hand {unsigned long index = from >>PAGE_CACHE_SHIFT ;unsigned offset = from & (PAGE_CACHE_SIZE -1 ); -unsigned blocksize, iblock, length, pos; +unsigned blocksize, length, pos; +sector_t iblock;struct inode *inode = mapping->host;struct buffer_head *bh;int err;
@@ -1603,7 +1604,8 @@ static int ext3_block_truncate_page(hand blocksize = inode->i_sb->s_blocksize; length = blocksize - (offset & (blocksize -1 )); - iblock = index << (PAGE_CACHE_SHIFT - inode->i_sb->s_blocksize_bits); + iblock = (sector_t ) index << + (PAGE_CACHE_SHIFT - inode->i_sb->s_blocksize_bits);if (!page_has_buffers (page))create_empty_buffers (page, blocksize,0 );
fs/reiserfs/inode.c +8 -8
@@ -203,10 +203,10 @@ static inline void set_block_dev_mapped// files which were created in the earlier version can not be longer, // than 2 gb // -static int file_capable (struct inode * inode,long block) +static int file_capable (struct inode * inode,sector_t block) {if (get_inode_item_key_version (inode) !=KEY_FORMAT_3_5 ||// it is new file. - block < (1 << (31 - inode->i_sb->s_blocksize_bits)))// old file, but 'block' is inside of 2gb + block < (1U << (31 - inode->i_sb->s_blocksize_bits)))// old file, but 'block' is inside of 2gb return 1 ;return 0 ;
@@ -244,7 +244,7 @@ static int file_capable (struct inode *// Please improve the english/clarity in the comment above, as it is // hard to understand. -static int _get_block_create_0 (struct inode * inode,long block, +static int _get_block_create_0 (struct inode * inode,sector_t block,struct buffer_head * bh_result,int args) {
@@ -559,7 +559,7 @@ out: } static inline int _allocate_block(struct reiserfs_transaction_handle *th, - long block, + sector_t block, struct inode *inode, b_blocknr_t *allocated_block_nr, struct path * path,
@@ -2063,7 +2063,7 @@ out:static int map_block_for_writepage (struct inode *inode,struct buffer_head *bh_result, -unsigned long block) { +sector_t block) {struct reiserfs_transaction_handle th ;int fs_gen ;struct item_head tmp_ih ;
@@ -2074,7 +2074,7 @@ static int map_block_for_writepage(strucINITIALIZE_PATH (path) ;int pos_in_item ;int jbegin_count =JOURNAL_PER_BALANCE_CNT ; -loff_t byte_offset = (block << inode->i_sb->s_blocksize_bits) +1 ; +loff_t byte_offset = ((loff_t ) block << inode->i_sb->s_blocksize_bits) +1 ;int retval ;int use_get_block =0 ;int bytes_copied =0 ;
@@ -2212,7 +2212,7 @@ static int reiserfs_write_full_page(strustruct inode *inode = page->mapping->host ;unsigned long end_index = inode->i_size >>PAGE_CACHE_SHIFT ;int error =0 ; -unsigned long block ; +sector_t block;struct buffer_head *head, *bh;int partial =0 ;int nr =0 ;
@@ -2252,7 +2252,7 @@ static int reiserfs_write_full_page(strukunmap_atomic (kaddr,KM_USER0 ) ; } bh = head ; - block = page->index << (PAGE_CACHE_SHIFT - s->s_blocksize_bits) ; + block = (sector_t ) page->index << (PAGE_CACHE_SHIFT - s->s_blocksize_bits) ;/* first map all the buffers, logging any direct items we find */ do { if ((checked || buffer_dirty(bh)) && (!buffer_mapped(bh) ||
fs/xfs/linux-2.6/xfs_aops.c +1 -1
@@ -71,7 +71,7 @@ xfs_page_trace(bhv_desc_t *bdp;vnode_t *vp =LINVFS_GET_VP (inode);loff_t isize =i_size_read (inode); -loff_t offset = page->index <<PAGE_CACHE_SHIFT ; +loff_t offset = (loff_t ) page->index <<PAGE_CACHE_SHIFT ;int delalloc =-1 , unmapped =-1 , unwritten =-1 ;if (page_has_buffers (page))
Quoted 9 lines · Tony Battersby
Here is an "example" patch to fix some of the LBD issues with various filesystems (ext3, reiserfs, xfs). Unfortunately it looks like there are many more LBD problems with the filesystems that I didn't fix, so I am just calling this an "example" patch that shows some of what needs to be done, but doesn't fix everything. I am not a filesystem expert, so I would prefer to leave it to others to do a full audit of all the filesystems for LBD-compliance. Specifically there are two things to check:
Quoted 3 lines
1) Use
sector_trather than long or unsigned long for block numbers. For auditing, just search for all occurrences of "long" in the filesystem code and make sure it is not referring to a block number.
That's actually not wise. For ext3, the block numbers _on disc_ are 32-bit in size, so making them sector_t in memory is just a waste. reiserfs seems to be the same. OTOH, XFS does use 64-bit blocks on disc, so should use an explicitly 64-bit size in memory too.
Signature · 6 lines
"Next the statesmen will invent cheap lies, putting the blame upon the nation that is attacked, and every man will be glad of those conscience-soothing falsities, and will diligently study them, and refuse to examine any refutations of them; and thus he will by and by convince himself that the war is just, and will thank God for the better sleep he enjoys after this process of grotesque self-deception." -- Mark Twain
Quoted 4 lines · Matthew Wilcox
That's actually not wise. For ext3, the block numbers
_ondisc_are 32-bit in size, so making themsector_tin memory is just a waste. reiserfs seems to be the same. OTOH, XFS does use 64-bit blocks on disc, so should use an explicitly 64-bit size in memory too.
XFS has compile-time switches for 32/64 bit in memory. On Linux it's 64bit with either a 64bit host or CONFIG_LBD.
Quoted 10 lines
Quoted 2 lines
1) Use
sector_trather than long or unsigned long for block numbers. For auditing, just search for all occurrences of "long" inthe filesystem
Quoted 1 line
code and make sure it is not referring to a block number.
That's actually not wise. For ext3, the block numbers
_ondisc_are 32-bit in size, so making themsector_tin memory is just a waste. reiserfs seems to be the same. OTOH, XFS does use 64-bit blocks on disc,
so should use an explicitly 64-bit size in memory too.
Thanks, that's why I was looking for feedback before going further. Is there anywhere in the fs-specific code that deals with real sector numbers rather than fs block numbers? Any variable that deals with real sector numbers should be sector_t, even if the filesystem uses only 32-bit block numbers. For example, an ext3 filesystem might be created in the 3-4 TB range of a 4 TB disk, in which case the fs block numbers fit into a 32-bit quantity but the real sector numbers need to be 64-bit because of the large starting offset. If the fs-specific code doesn't deal with real sector numbers, then there is probably nothing that needs to be done for ext3 or reiserfs.
Anthony J. Battersby
Cybernetics