Skip to contentSkip to conversation

ext2resize

linux-lvm Lennert Buytenhek 2 participants 3 messages Jul 5, 1999

lore
Share & tools
lore
#0 lore raw
Quoted 10 lines
Quoted 8 lines · Lennert Buytenhek

I replied: "Depending on who you talk to of course. :-) The max. number of groups for ext2 is 1024 I believe. One extra gd block per block gives you room for 32*8=256mb expansion (assuming 1kb blocks). This will cost you at most 1 meg of reserved gd blocks. Seems like a fair price. The max. number of gd blocks is 32. So doing this when making an fs will cost you at most 32*1024 blocks, which is 32mb with 1k blocks. On modern drives, you'll probably not even notice a 32mb loss. Unless you have a lot of partitions, of course...."

Are you sure that the max number of GDT blocks is 32? For a 1kB block

Yes, for a 1kb block size. One of my ext2 linux kernel headers #defines the max number of groups to be 1024. Times 32 bytes per group descriptor is 32kb, which is 32 blocks on 1kb. Unless the header is wrong, of course.

Quoted 7 lines

size, this would give a limit of 32 GDT blocks * 32 GD/GDT block * 8k blocks/GD = 8GB max FS size. With 4kB blocks we grow 4x for larger data blocks, 4x for more GD/GDT block, and 4x for more blocks/GD, so 512 GB max, not the expected 4TB limit. If we wanted to reach 4TB with 1kB blocks (possible since block numbers are 32-bit unsigned), then we would need 512*32 GDT blocks, or 200% !!! of all FS space, while with 4kB blocks we need 256 GDT blocks, or 1/32 of FS space.

Yes, well, I didn't invent this. Most of the people will use larger block sizes then, anyway.

Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

Mike had also suggested that when we are doing a major FS (offline) reorg, we could start removing blocks from the inode table instead of data blocks as there are usually free inodes in each group, but not always data blocks...

I think it's not worth the complexity. First of all all your inodes will

be

Quoted 6 lines
Quoted 3 lines

renumbered. You'll need a full directory scan-and-replace for inodes which is very crash-sensitive. On the other hand, relocating a block is atomic.

There are two other possibilities:
1) "#define EXT2_EXPAND_INO 7" and put it in include/linux/ext2_fs.h and

allocate

Quoted 1 line

the first few data blocks in each group to this inode to reserve them.

Would be nice. But this way the blocks are really reserved and can't be used for anything else. Why don't we say: "every ext2 fs must have 32 gdt blocks per block group" then? Same idea.

Quoted 1 line

2) Have the "reserved" blocks blocks be the first few data blocks in each

group,

Quoted 1 line

rather than all in a single group. This way the blocks can be used by

root when

Quoted 1 line

the FS is very full, or they can be used for expanding the filesystem.

The one

Quoted 1 line

problem is that you may not have your extra blocks when you need them

(ie when the

Quoted 1 line

FS is full) if it is root that is filling the FS. The plus is that you

don't keep

Quoted 1 line

more spare blocks that normally nobody uses in the FS.

How do you tell the kernel that it should use those 'reserved' blocks only when
free space is low? compat/rocompat/incompat feature?

Lennert Buytenhek
<[email protected]>

#2 lore raw
In reply to Lennert Buytenhek
Quoted 9 lines · Lennert Buytenhek
Quoted 9 lines
Quoted 8 lines · Lennert Buytenhek

I replied: "Depending on who you talk to of course. :-) The max. number of groups for ext2 is 1024 I believe. One extra gd block per block gives you room for 32*8=256mb expansion (assuming 1kb blocks). This will cost you at most 1 meg of reserved gd blocks. Seems like a fair price. The max. number of gd blocks is 32. So doing this when making an fs will cost you at most 32*1024 blocks, which is 32mb with 1k blocks. On modern drives, you'll probably not even notice a 32mb loss. Unless you have a lot of partitions, of course...."

Quoted 7 lines
Quoted 1 line

Are you sure that the max number of GDT blocks is 32? For a 1kB block

Yes, for a 1kb block size. One of my ext2 linux kernel headers #defines the max number of groups to be 1024. Times 32 bytes per group descriptor is 32kb, which is 32 blocks on 1kb. Unless the header is wrong, of course.

I don't think this is correct. I have an ext2 filesystem that is 52GB. It appears to have 200+ blocks in the GDT and as I recall 6000+ block groups. I made a 86GB filesystem the other day with 1k blocks and it had 10000+ block groups. With 4k blocks there were 650+ block groups.

Quoted 9 lines
Quoted 8 lines

size, this would give a limit of 32 GDT blocks * 32 GD/GDT block * 8k blocks/GD = 8GB max FS size. With 4kB blocks we grow 4x for larger data blocks, 4x for more GD/GDT block, and 4x for more blocks/GD, so 512 GB max, not the expected 4TB limit. If we wanted to reach 4TB with 1kB blocks (possible since block numbers are 32-bit unsigned), then we would need 512*32 GDT blocks, or 200% !!! of all FS space, while with 4kB blocks we need 256 GDT blocks, or 1/32 of FS space.

Quoted 3 lines

Yes, well, I didn't invent this. Most of the people will use larger block sizes then, anyway.

It does seem peculiar that the largest block size is limited to 4k. 8k would seem to be a reasonable size to me.

It seems that ext2 is not really suited for large filesystems: seems like there is too much redundancy in the block groups that causes slow downs in operations like mount, etc.; e2fsck takes hours on a 52GB filesystem.

Are there any projects underway to develop a new filesystem that is more suitable for large filesystems?

John