remove 2TB block device limit

There's now a patch available against 2.5.15, and the BK repository has been updated to v2.5.15 as well:

    http://www.gelato.unsw.edu.au/patches/2.5.15-largefile-patch
    bk://gelato.unsw.edu.au:2023/

Peter C
[email protected]

This looks really good, I'd like to see something like that merged soon! Some comments:

- please move the sector_t typedef from <linux/types.h> to <asm/types.h>, so 64 bit arches don't have to have the CONFIG_ option at all, some 32bit plattforms that are unlikely to ever support large disks (m68k comes to mind) can make it 32bit unconditionally and some like i386 can use a config option.
- sector_div should move to a common header (blkdev.h?)

And something related to general sector_t usage:

- what about sector_t vs daddr_t? Linux still has daddr_t, but it is still always 32bit, I think a big s/sector_t/daddr_t/ would fit the traditional unix way of doing disk addressing
- why is the get_block block argument a sector_t? It presents a logical filesystem block which usually is larger than the sector, not to mention that for the usual blocksize == PAGE_SIZE case a ulong is enough as that is the same size the pagecache limit triggers.

Christoph> On Mon, May 13, 2002 at 08:28:30PM +1000, Peter Chubb

Christoph> This looks really good, I'd like to see something like that Christoph> merged soon! Some comments:

Christoph> - please move the sector_t typedef from <linux/types.h> to Christoph> <asm/types.h>, so 64 bit arches don't have to have the Christoph> CONFIG_ option at all, some 32bit plattforms that are Christoph> unlikely to ever support large disks (m68k comes to mind) Christoph> can make it 32bit unconditionally and some like i386 can Christoph> use a config option. - sector_div should move to a common Christoph> header (blkdev.h?)

That's not a bad idea, I'll do it.

Christoph> And something related to general sector_t usage:

Christoph> - what about sector_t vs daddr_t? Linux still has Christoph> daddr_t, but it is still always 32bit, I think a big Christoph> s/sector_t/daddr_t/ would fit the traditional unix way of Christoph> doing disk addressing

Yes I considered that, but daddr_t is exported to userland by the tape ioctls, and is defined to be 32-bit, so it'd require out-of-kernel changes.

Besides, Jens had introduced sector_t for the purpose of counting blocks and sectors, so I thought I may as well use it. One could argue that it's misnamed (personally, I liked Ben's `blkoff_t' for offset in blocks), but it's been thare for four months or so, and was already being used in likely places throughout the block layer. I just extended its use to all the places I thought were necessary (there may be some paths that I've missed; but I hope not).

Christoph> - why is the get_block block argument Christoph> a sector_t? It presents a logical filesystem block which Christoph> usually is larger than the sector, not to mention that for Christoph> the usual blocksize == PAGE_SIZE case a ulong is enough as Christoph> that is the same size the pagecache limit triggers.

For filesystems that *can* handle logical filesystem blocks beyond the 2^32 limit (i.e., that use >32bit offsets in their on-disc format), the get_block() argument has to be > 32bits long. At the moment that's only JFS and XFS, but reiserfs version 4 looks as if it might go that way. We'll need this especially when the pagecache limit is gone.

Besides, blocksize is not usually pagesize. ext[23] uses 1k or 4k blocks depending on the size and expected use of the filesystem; alpha pagesize is usually 8k, for example. The arm uses 4k, 16k or 32k pagesizes depending on the model.

So on 32-bit systems, ulong is not enough. (in fact if you look at jfs, the first thing jfs_get_block does is convert the block number arg to a 64-bit number).

Peter C

[snip]

NTFS uses signed 64 bits for all offsets on disk, too. And yes at the moment the pagecache limit is also a problem which we just ignore in the hope that the kernel will have gone to 64 bits by the time devices grow that large as to start using > 32 bits of blocks/pages...

NTFS does that, too, but not quite immediately... (-;

Best regards,

Anton

Anton Altaparmakov <aia21 at cantab.net> (replace at with @)
Linux NTFS maintainer / IRC: #ntfs on irc.openprojects.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

PAGE_CACHE_SIZE can also grow, so 32 bit architectures are further away from the page cache limit on than it seems.

Daniel

I think Christoph's point is that a pagecache index is not a sector number. We agree that we need to plan for taking it to 64 bits, but it should be something different. Like pageindex_t, or whatever.

This:

@@ -148,7 +148,7 @@ struct vm_operations_struct {
 typedef struct page {
 	struct list_head list;		/* ->mapping has some page lists. */
 	struct address_space *mapping;	/* The inode (or ...) we belong to. */
-	unsigned long index;		/* Our offset within mapping. */
+	sector_t index;			/* Our offset within mapping. */
 	atomic_t count;			/* Usage count, see below. */
 	unsigned long flags;		/* atomic flags, some possibly
 					   updated asynchronously */

looks rather silly, no?

-

Christoph> - why is the get_block block argument a sector_t? It Christoph> presents a logical filesystem block which usually is larger Christoph> than the sector, not to mention that for the usual Christoph> blocksize == PAGE_SIZE case a ulong is enough as that is Christoph> the same size the pagecache limit triggers.

Andrew> I think Christoph's point is that a pagecache index is not a Andrew> sector number. We agree that we need to plan for taking it to Andrew> 64 bits, but it should be something different. Like Andrew> pageindex_t, or whatever.

I'll let Christoph speak for himself, but my point is that get_block() is an interface exported from the filesystem. It should be possible to specify any logical block number that the filesystem
supports. That the current VM system on 32-bit machines will never
request a block beyond 2^32 is a (one-day-soon-to-be-removed) current limitation.

Peter C

I don't think we want to increase it. First it grow struct page by 32bits and second 64bit arithmetic on 32bit plattforms is still very expensive. I'd rather see PAGE_CACHE_SIZE growing to address the issues.

It itsn't. get_block is a callback for the generic block-based filesystem library routines.