How to resize to an bigger then 16TB ext4 filesysteem on Debian
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hello everyone,
https://packages.debian.org/wheezy/e2fsprogs
# resize2fs /dev/lvm0-vol/storage resize2fs 1.42.5 (29-Jul-2012) resize2fs: New size too large to be expressed in 32 bits
(will the command work with e2fsprogs 1.42.9-3?)
# tune2fs -l /dev/lvm0-vol/storage http://paste.debian.net/92889/
Suggestions has to be stable enough for production systems.
I now did a lvreduce -L 16TB /dev/lvm0-vol/storage and resize2fs /dev/lvm0-vol/storage (its running now)
# resize2fs /dev/lvm0-vol/storage resize2fs 1.42.5 (29-Jul-2012) Filesystem at /dev/lvm0-vol/storage is mounted on /srv/storage; on-line resizing required old_desc_blocks = 464, new_desc_blocks = 1024 Performing an on-line resize of /dev/lvm0-vol/storage to 4294967295 (4k) blocks.
Kind regards,
Jelle de Jong
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
iJwEAQECAAYFAlNGgFAACgkQ1WclBW9j5HnDbQP/YRQvZq0HPJ4brpBas+95xeP3 lOnN0OzEOEpjC9pY7J9ppWfkvOkBx7ayjt8MqmJgluwVBNQIRUU3ctftaI7cUPTH abgj0JHLICCjSIs3nVzQo9pq8URnhTbDXDcXLBemglT5wa+Potlz4YYFA+9Cd/ty G2jfKBiaNCN17Nmikjo=
=bwbX
-----END PGP SIGNATURE-----
Quoted 9 lines · Jelle de Jong
Hello everyone,
https://packages.debian.org/wheezy/e2fsprogs
# resize2fs /dev/lvm0-vol/storage resize2fs 1.42.5 (29-Jul-2012) resize2fs: New size too large to be expressed in 32 bits
Quoted 6 lines
(will the command work with e2fsprogs 1.42.9-3?)
# tune2fs -l /dev/lvm0-vol/storage http://paste.debian.net/92889/Suggestions has to be stable enough for production systems.
I'm going to assume the file system was created without having the 64-bit file system feature set. (Running dumpe2fs -h /dev/lvm0-vol/storage can verify this).
If that's true, and you require a production-stable way of doing this, there's really only one thing I can suggest:
1) Backup your file system completely
2) Get the development version of e2fsprogs, and apply a set of in-development patches that haven't yet been merged
3) Use this in-development resize2fs to convert to a 64-bit file system, realizing that it will have completely rewrite your inode tables. Cross your fingers.
4) If it works, you're done. If not, you can restore from backups. :-)
So basically steps 2 and 3 are ways of optimizing needing to do a full restore of your file system.....
Cheers,
- Ted
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Quoted 30 lines · Theodore Ts'o
Quoted 8 lines · Jelle de Jong
Hello everyone,
https://packages.debian.org/wheezy/e2fsprogs
# resize2fs /dev/lvm0-vol/storage resize2fs 1.42.5 (29-Jul-2012) resize2fs: New size too large to be expressed in 32 bitsQuoted 6 lines
(will the command work with e2fsprogs 1.42.9-3?)
# tune2fs -l /dev/lvm0-vol/storage http://paste.debian.net/92889/Suggestions has to be stable enough for production systems.
I'm going to assume the file system was created without having the 64-bit file system feature set. (Running dumpe2fs -h
/dev/lvm0-vol/storagecan verify this).If that's true, and you require a production-stable way of doing this, there's really only one thing I can suggest:
1) Backup your file system completely 2) Get the development version of e2fsprogs, and apply a set of in-development patches that haven't yet been merged 3) Use this in-development resize2fs to convert to a 64-bit file system, realizing that it will have completely rewrite your inode tables. Cross your fingers. 4) If it works, you're done. If not, you can restore from backups. :-)
Quoted 2 lines
So basically steps 2 and 3 are ways of optimizing needing to do a full restore of your file system.....
Thank you Ted for your quick reply.
In october 2012 I created the ext4 file system with the command:
mkfs.ext4 -m 1 /dev/lvm0-vol/storage
I assumed by using ext4 with lvm2 I would have had a solid basis for some future storage expeditions. Apparently I made a mistake somewhere but I don't know where yet or did I hit a bug?
# tune2fs -l /dev/lvm0-vol/storage # dumpe2fs -h /dev/lvm0-vol/storage http://paste.debian.net/plain/92953
Where can I see that my ext4 file system is limed to 16TB?
Your suggestion to use a development version of e2fsprogs doesn't really suit my needs for a stable solution that is maintainable.
1. I can temporary transfer all the data to an other system and recreate the ext4 file system, but will this be stable with version 1.42.5 and how much TB can the file-system handle?
2. I can put current the 16TB ext4 file system in production and wait for e2fsprogs support in Debian stable or backports to arrive and resize the to 22TB later. When do you expect this functionality to arrive in as stable release of e2fsprogs?
Thank you for your help.
Kind regards,
Jelle de Jong
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
iJwEAQECAAYFAlNGs7IACgkQ1WclBW9j5Hl5NgQAjwMLRPu3KQuBesD7BSHkUrf3 JcaFMD90vvAYmbKMbMumGhR86BA1NXVBfeEA6DnrZUrPzly1VBglCOwCJzw52GCc TZHp3h5RD/Ce1c8/T+VD4DrVxhjIMUAyUT+/eGN008mHtrOfRppnFlcutNxZ5RZJ oGksAsJ5i7hWmeilqcc=
=ECQP
-----END PGP SIGNATURE-----
Quoted 8 lines · Jelle de Jong
In october 2012 I created the ext4 file system with the command:
mkfs.ext4 -m 1/dev/lvm0-vol/storageI assumed by using ext4 with lvm2 I would have had a solid basis for some future storage expeditions. Apparently I made a mistake somewhere but I don't know where yet or did I hit a bug?
Quoted 5 lines
# tune2fs -l /dev/lvm0-vol/storage # dumpe2fs -h /dev/lvm0-vol/storage http://paste.debian.net/plain/92953Where can I see that my ext4 file system is limed to 16TB?
So here's the problem. The original ext2/3 block group descriptors only had room for 32-bit block numbers. So in order to resize the file system past 16TB, it was necessary to change the layout of the block group descriptors, and change them from needing 32 bytes to 64 bytes per block group.
So in order to make this change there was a new "64-bit" feature flag that changes the size of the block group descriptors. We didn't enable this feature at first, because we wanted to make sure the 64-bit support was well shaken down and "production ready" before it was enabled.
(In particular there were some online resizing bugs relating to 64-bit file systems that wasn't ironed out until relatively recently, so you should be glad we didn't enable this feature by default. :-)
Quoted 2 lines
Your suggestion to use a development version of e2fsprogs doesn't really suit my needs for a stable solution that is maintainable.
Well, unfortunately, there is no way to do what you request using the stable version of e2fsprogs. Sorry.
Quoted 3 lines
1. I can temporary transfer all the data to an other system and recreate the ext4 file system, but will this be stable with version 1.42.5 and how much TB can the file-system handle?
You should definitely go to the 1.42.9 version of e2fsprogs to make sure you have the latest fixes.
As far as the kernel is concerned, are you still using the Debian stable 3.2 kernel? The problem is that not all of the bug fixes get backported to a stable kernel which is that old. One of the value adds that a company like Red Hat provides is that they have someone paid to try to support ext4 on older kernels. But there have been some online resize fixes on the kernel side since 3.2, and I don't think all of them have been backported because they are fairly complex and involved patches.
If you're not willing to use the kernel from Debian testing, it may very well be that you will be better off waiting for Debian testing to ship before you even try to use ext4 file systems > that 16TB. I really can't guarantee that all of the necessary fixes have been backported to the 3.2 stable kernel.
Is there some way you could live with two large file systems instead of trying to create one very large file system?
Regards,
- Ted