ext2 large block size > page size
Hello everyone,
As far as I can tell from a quick Google search, there was a push in 2007 to add support for large blocks in some file-systems, in particular ext2.
e.g. cf. http://lwn.net/Articles/239090/
Was this ever accepted into the main line?
(It seems to have lived within -mm for a while)
A few years later, I'm trying to mount a HDD which was ext2-formatted with 32k blocks.
And mount is failing (with a strange "block size too small" error message).
I'm trying this in Fedora 14
2.6.35.6 kernel
1.41.12 e2fsprogs
Is the support for large blocks in ext2 not compiled in this generic kernel ? Or was the patch never accepted ?
I'm working with a ST Microelectronics set-top box. Here are a few performance results for a 2TB USB HDD:
block size 4k : format = 151 s / mount = 242 s block size 8k : format = 52 s / mount = 71 s block size 16k : format = 30 s / mount = 36 s block size 32k : format = 18 s / mount = 19 s
Using 4kB blocks makes mount too slow on the STB, which is why I'd like to use larger blocks. It would be nice if the movies recorded on the STB could also be read on a Linux PC.
Signature · 1 line
Regards.
Hello again,
Quoted 5 lines · Mason
As far as I can tell from a quick Google search, there was a push in 2007 to add support for large blocks in some file-systems, in particular ext2.
e.g. cf. http://lwn.net/Articles/239090/
The page above describes one of the proposed patch, and the following article by Jonathan Corbet gives an overview of the issues.
http://lwn.net/Articles/232757/
Quoted 6 lines
Was this ever accepted into the main line?
(It seems to have lived within -mm for a while)A few years later, I'm trying to mount a HDD which was ext2-formatted with 32k blocks.
Quoted 6 lines
And mount is failing (with a strange "block size too small" error message).
I'm trying this in Fedora 14
2.6.35.6 kernel
1.41.12 e2fsprogs
To add insult to injury, Ext2Fsd(*) an ext2 IFS (Installable File System) for Windows, accepts large-block ext2 partitions.
Quoted 10 lines
Is the support for large blocks in ext2 not compiled in this generic kernel ? Or was the patch never accepted ?
I'm working with a ST Microelectronics set-top box. Here are a few performance results for a 2TB USB HDD:
block size 4k : format = 151 s / mount = 242 s block size 8k : format = 52 s / mount = 71 s block size 16k : format = 30 s / mount = 36 s block size 32k : format = 18 s / mount = 19 s
Quoted 4 lines
Using 4kB blocks makes mount too slow on the STB, which is why I'd like to use larger blocks. It would be nice if the movies recorded on the STB could also be read on a Linux PC.
Is there, perhaps, a better place to discuss this issue?
Signature · 1 line
Regards, Mason
Quoted 10 lines · Mason
Hello everyone,
As far as I can tell from a quick Google search, there was a push in 2007 to add support for large blocks in some file-systems, in particular ext2.
e.g. cf. http://lwn.net/Articles/239090/
Was this ever accepted into the main line?
(It seems to have lived within -mm for a while)
Nope, it never was (as you've by now figured out).
Quoted 11 lines
I'm working with a ST Microelectronics set-top box. Here are a few performance results for a 2TB USB HDD:
block size 4k : format = 151 s / mount = 242 s block size 8k : format = 52 s / mount = 71 s block size 16k : format = 30 s / mount = 36 s block size 32k : format = 18 s / mount = 19 sUsing 4kB blocks makes mount too slow on the STB, which is why I'd like to use larger blocks. It would be nice if the movies recorded on the STB could also be read on a Linux PC.
My guess is the mount time slowness is caused an ancient kernel running on the ST Microelectronics box which is doing mount-time sanity checks. You can disable this with the mount option -o nocheck.
A lot of the rationale for larger block sizes was obviated by the use of more advanced file systems, such as ext4, which have other methods of dealing with the inefficiencies caused by smaller block sizes. If your main complaint with using a 4k block size on the set-top box was the mount-time slowness, that can be fixed with the nocheck mount option.
Regards,
- Ted
[ Adding Christoph Lameter to the CC list ]
Hello Ted,
Quoted 20 lines · Ted Ts'o
Quoted 8 lines · Mason
As far as I can tell from a quick Google search, there was a push in 2007 to add support for large blocks in some file-systems, in particular ext2.
e.g. cf. http://lwn.net/Articles/239090/
Was this ever accepted into the main line?
(It seems to have lived within -mm for a while)Nope, it never was (as you've by now figured out).
Quoted 7 lines
I'm working with a ST Microelectronics set-top box. Here are a few performance results for a 2TB USB HDD:
block size 4k : format = 151 s / mount = 242 s block size 8k : format = 52 s / mount = 71 s block size 16k : format = 30 s / mount = 36 s block size 32k : format = 18 s / mount = 19 s
Quoted 8 lines
Quoted 4 lines
Using 4kB blocks makes mount too slow on the STB, which is why I'd like to use larger blocks. It would be nice if the movies recorded on the STB could also be read on a Linux PC.
My guess is the mount time slowness is caused an ancient kernel running on the ST Microelectronics box which is doing mount-time sanity checks. You can disable this with the mount option -o nocheck.
Unfortunately, the operating system of the set-top box is not Linux (ST has only very recently started migrating to Linux).
Their "legacy" OS (OS+ running on top of OS21) provides (proprietary, I suppose) implementations of FAT32 and ext2.
I used to create FAT32 partitions, until I plugged a 2-TB USB HDD in the STB:
format = 123 s
mount = 62 s
fsck = 517 s (!!)
I switched to ext2, hoping to avoid the need to fsck, thanks to the soft updates mode. Problem is, I can't read the files on a Linux PC if I use large blocks, and performance takes a dive if I use "normal" blocks.
Quoted 6 lines
A lot of the rationale for larger block sizes was obviated by the use of more advanced file systems, such as ext4, which have other methods of dealing with the inefficiencies caused by smaller block sizes. If your main complaint with using a 4k block size on the set-top box was the mount-time slowness, that can be fixed with the nocheck mount option.
Unfortunately, the mount function in this OS accepts only two flags: RDONLY and RDWR :-(
If I understand correctly, I'm screwed, right? :-)
Signature · 1 line
Regards.
Quoted 4 lines · Mason
Quoted 4 lines
Quoted 2 lines
Was this ever accepted into the main line?
(It seems to have lived within -mm for a while)Nope, it never was (as you've by now figured out).
I was not able to get the base patchset in that allowed
the page cache to work with different page sizes. Reason given was that the functionality that this was used for would never have a chance to be accepted.
Quoted 6 lines
Quoted 6 lines
A lot of the rationale for larger block sizes was obviated by the use of more advanced file systems, such as ext4, which have other methods of dealing with the inefficiencies caused by smaller block sizes. If your main complaint with using a 4k block size on the set-top box was the mount-time slowness, that can be fixed with the nocheck mount option.
The rationale for the large block size patchset was to avoid having to handle small 4k chunks both in the hardware and the OS paths. That rationale is still there. The fixes to file systems address meta data issues at a higher level. What you can do on the page level (and what we have done) is basically improve locking but that will at some point no longer be enough since the I/O sizes keep on increasing. At some point we need to be able to handle larger physical chunks.
Andrea's work in getting THP committed to the kernel lays some more groundwork for future possibilities. We could f.e. have a base 2M page size for the page cache at some point (some form of my patches to allow the page cache to function with different orders are required though).
With larger sizes also come fragmentation issues. We have continually added more means to handle defragmentation do compaction. At some point we should be able to handle larger contiguous blocks and be able to implement larger block sizes.