Skip to contentSkip to conversation

32/64 bit b_rsector type

DISCUSSION David Chow 4 participants 5 messages Mar 24, 2004 – Mar 25, 2004 VFS

lore
Share & tools
lore

David Chow

author
#0 lore raw

Dear all,

On a native 32-bit Linux system, the bh->b_resetor (type unsigned long) max is limited to 16TB. For 64 bit systems, I know it can get through this point, but what about if I chanage the bh->b_rsector in fs.h to u64 (unsigned long long on 32-bit systems)? Can I simply break through the 16TB on 32-bit Linux by doing so? Please advice with possiblility of impacts. Thanks.

regards,
David Chow

#1 lore raw
Quoted 6 lines · David Chow

On a native 32-bit Linux system, the bh->b_resetor (type unsigned long) max is limited to 16TB. For 64 bit systems, I know it can get through this point, but what about if I chanage the bh->b_rsector in fs.h to u64 (unsigned long long on 32-bit systems)? Can I simply break through the 16TB on 32-bit Linux by doing so? Please advice with possiblility of impacts. Thanks.

The 16TB limit on a block device is due to the page cache -- 4k pages * 4GB of 'unsigned long index'. Linus has drawn a line in the sand on that one -- index is not going to be u64. The only other ways to go bigger than this are:

- Use a 64-bit machine (duh ;-)
- Stop using the page cache for blockdevs (not popular)
- Steal some bits from elsewhere to increase the size

The last one is probably best exemplified by pgcl. That increases the page size so you can have, say, 16k * 4GB which would get you to a 64GB blockdev limit, or 64k * 4GB limiting you to 256GB.

Signature · 6 lines
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain
#2 lore raw
Quoted 5 lines

what about if I chanage the bh->b_rsector in fs.h to u64

Quoted 3 lines

(unsigned long long on 32-bit systems)? Can I simply break through the 16TB on 32-bit Linux by doing so? Please advice with possiblility of impacts. Thanks.

Quoted 7 lines

The 16TB limit on a block device is due to the page cache -- 4k pages * 4GB of 'unsigned long index'.
...
The only other ways to go bigger than this are:
...
- Stop using the page cache for blockdevs (not popular)
...

Why would we have to stop? Doesn't this just mean access through the page cache would remain limited to 16TB? What if I don't really care about access to the device with read() -- I just want to put a filesystem on it?
Would the simple change the bh->b_rsector enable that?

#3 lore raw
Quoted 5 lines · Bryan Henderson

Why would we have to stop? Doesn't this just mean access through the page cache would remain limited to 16TB? What if I don't really care about access to the device with read() -- I just want to put a filesystem on it?
Would the simple change the bh->b_rsector enable that?

mkfs and fsck may have an issue with the inability to do reads and writes to the device.

Signature · 2 lines
David Kleikamp
IBM Linux Technology Center
author
#4 lore raw

I think page cache will probably limited by the hashing functions which is type sensitive to page->index.

Dave Kleikamp 提到:

Quoted 9 lines · Bryan Henderson
Quoted 8 lines

Why would we have to stop? Doesn't this just mean access through the page cache would remain limited to 16TB? What if I don't really care about access to the device with read() -- I just want to put a filesystem on it? Would the simple change the bh->b_rsector enable that?

For me, I would like to just to put a specialized LVM or 64-bit nbd server on it.

Quoted 3 lines

mkfs and fsck may have an issue with the inability to do reads and writes to the device.

For my issue (which I originally brought up) is not targeted to use with fsck or mkfs . I am researching to build a cluster IP-based storage infrastructure with special interconnects. We will use software-based RAID drivers which I can't see direct benefits from using 64-bit machines. For cost reasons, I would really want to stick with 32-bit machines wiht high speed CPUs but with the ability to access blocks beyond 16TB.

Large pages is not my direction, because pages will be send/receive from multiple IP NIC's which is not a good idea to break things up nor to send large chunks.

For page cache or buffer cache or any cache, page->index and bh->b_rsector will definitely have to change to u64 to fullfil the requirement. And of course, fsck and mkfs will break if this is true, fs drivers will also have to be patched and checked for type corrections. Still, I would like to hear more voices from experienced fs developers (like you guys) to explore potential issues before I actually modify the code.

Thanks for pointing out. ^_^

regards,
David Chow

Quoted 2 lines

-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html