ext4: add the spec for ext4 subdirectory limit(was: Problems with the max value for create directory)
PATCH 1/1 ext4: add the spec for ext4 subdirectory limit(was: Problems with the max value for create directory)
When using hash index, too many long-subdirectory-names and 1024 blocksize, the subdirectory limit may be less than 32000.
The problem is that indexed diretories have a limit that only allows the trees to be two levels deep. The fanout is normally big enough that this is effectively not a problem, but if you use too many long-names, and a 1024 blocksize, you will run into htree limit much more quickly.
The users may be puzzled when they can't create more than 32000 subdirecties with using hash index, too many long-names and 1024 blocksize. It should be remind of in the ext4 document.
--- Documentation/filesystems/ext4.txt | 8 +++++++- 1 files changed, 7 insertions(+), 1 deletions(-)
Documentation/filesystems/ext4.txt +7 -1
@@ -85,7 +85,7 @@ Note: More extensive information for getting started with ext4 can be * extent format more robust in face of on-disk corruption due to magics, * internal redundancy in tree * improved file allocation (multi-block alloc) -* fix 32000 subdirectory limit +* fix 32000 subdirectory limit (*1) * nsec timestamps for mtime, atime, ctime, create time * inode version field on disk (NFSv4, Lustre) * reduced e2fsck time via uninit_bg feature
@@ -100,6 +100,12 @@ Note: More extensive information for getting started with ext4 can be * efficent new ordered mode in JBD2 and ext4(avoid using buffer head to force the ordering) +(*1) Note +Basically, the limit depends on blocksize and length of subdirectory names in +the directory. If blocksize is too small (like 1024) and you use too many +long-names, you may reach a smaller limit. When dir index is off, the limit is +65000. + 2.2 Candidate features for future inclusion * Online defrag (patches available but not well tested)
I really don't think this tweak to Documentation/filesystems/ext4.txt is necessary, but if we going to do this, I want to tighten up the explanation.
- Ted
ext4: Add fine print for the 32000 subdirectory limit
Some poeple are reading the ext4 feature list too literally and create dubious test cases involving very long filenames and 1k blocksize and then complain when they run into an htree-imposed limit. So add fine print to the "fix 32000 subdirectory limit" ext4 feature.
Documentation/filesystems/ext4.txt +4 -1
@@ -85,7 +85,7 @@ Note: More extensive information for getting started with ext4 can be * extent format more robust in face of on-disk corruption due to magics, * internal redundancy in tree * improved file allocation (multi-block alloc) -* fix 32000 subdirectory limit +* lift 32000 subdirectory limit imposed by i_links_count[1] * nsec timestamps for mtime, atime, ctime, create time * inode version field on disk (NFSv4, Lustre) * reduced e2fsck time via uninit_bg feature
@@ -100,6 +100,9 @@ Note: More extensive information for getting started with ext4 can be * efficent new ordered mode in JBD2 and ext4(avoid using buffer head to force the ordering) +[1] Filesystems with a block size of 1k may see a limit imposed by the +directory hash tree having a maximum depth of two. + 2.2 Candidate features for future inclusion * Online defrag (patches available but not well tested)
Re: [PATCH -v2]ext4: add the spec for ext4 subdirectory limit(was:Problems with the max value for create directory)
Quoted 6 lines · Theodore Tso
I really don't think this tweak to
Documentation/filesystems/ext4.txtis necessary, but if we going to do this, I want to tighten up the explanation.- Ted
I agree.
Thanks.
Quoted 34 lines
ext4: Add fine print for the 32000 subdirectory limit
Some poeple are reading the ext4 feature list too literally and create dubious test cases involving very long filenames and 1k blocksize and then complain when they run into an htree-imposed limit. So add fine print to the "fix 32000 subdirectory limit" ext4 feature.
Signed-off-by "Theodore Ts'o" <[email protected]>Documentation/filesystems/ext4.txt +4
-1@@ -85,7 +85,7 @@ Note: More extensive information for getting started with ext4 can be * extent format more robust in face of on-disk corruption due to magics, * internal redundancy in tree * improved file allocation (multi-block alloc) -* fix 32000 subdirectory limit +* lift 32000 subdirectory limit imposed by i_links_count[1] * nsec timestamps for mtime, atime, ctime, create time * inode version field on disk (NFSv4, Lustre) * reduced e2fsck time via uninit_bg feature@@ -100,6 +100,9 @@ Note: More extensive information for getting started with ext4 can be * efficent new ordered mode in JBD2 and ext4(avoid using buffer head to force the ordering) +[1] Filesystems with a block size of 1k may see a limit imposed by the +directory hash tree having a maximum depth of two. + 2.2 Candidate features for future inclusion * Online defrag (patches available but not well tested)