Skip to contentSkip to conversation

Good place to drop preallocation on directories

DISCUSSION Jan Kara 2 participants 3 messages Nov 19, 2008 – Nov 20, 2008 VFS

lore
Share & tools
lore

Jan Kara

author reviewer
#0 lore raw

Hi!

I have a problem in UDF so I thought I'll post it here if someone has a
good idea how to solve it.
UDF does preallocation on directories (on files a well, but that is not a
problem). Now this helps against directory fragmentation but there is an issue with it: When to drop this preallocation? Currently it is done in udf_clear_inode() which has two problems:
1) udf_clear_inode() is really called to late to do anything like freeing space from inode and UDF has to play some nasty tricks (like explicitely syncing inode, calling invalidate_inode_buffers() once more) to make everything work. Using udf_drop_inode() would be much better but that is called under i_lock so it's not really usable (unless I'd drop i_lock in it but then it would even nastier).
2) When you create lots of directories, they use quite some additional space for preallocation and there's not really a got way how to get rid of preallocated blocks than dropping dcache or umount + mount the filesystem.

So after all I'm thinking whether directory preallocation is really worth
the trouble. But before I go and rip it I'd like to check whether someone does not have some clever idea :).

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
#1 lore raw
Quoted 18 lines · Jan Kara

I have a problem in UDF so I thought I'll post it here if someone has a
good idea how to solve it.
UDF does preallocation on directories (on files a well, but that is not a
problem). Now this helps against directory fragmentation but there is an issue with it: When to drop this preallocation? Currently it is done in udf_clear_inode() which has two problems:
1) udf_clear_inode() is really called to late to do anything like freeing space from inode and UDF has to play some nasty tricks (like explicitely syncing inode, calling invalidate_inode_buffers() once more) to make everything work. Using udf_drop_inode() would be much better but that is called under i_lock so it's not really usable (unless I'd drop i_lock in it but then it would even nastier).
2) When you create lots of directories, they use quite some additional space for preallocation and there's not really a got way how to get rid of preallocated blocks than dropping dcache or umount + mount the filesystem.

Quoted 3 lines

So after all I'm thinking whether directory preallocation is really worth
the trouble. But before I go and rip it I'd like to check whether someone does not have some clever idea :).

Problem 2 has lru written all over, don't you think? Problem 1 is similar to something I'm debugging this very moment. With some luck my solution might work for you as well.

However, I don't see a big difference between files and directories yet. Maybe because there shouldn't be, or maybe because I've been missing something critical. Could you elaborate on that?

Jörn

Signature · 6 lines
It's just what we asked for, but not what we want!
-- anonymous
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
author reviewer
#2 lore raw
Quoted 19 lines · Jörn Engel
Quoted 18 lines · Jan Kara

I have a problem in UDF so I thought I'll post it here if someone has a
good idea how to solve it.
UDF does preallocation on directories (on files a well, but that is not a
problem). Now this helps against directory fragmentation but there is an issue with it: When to drop this preallocation? Currently it is done in udf_clear_inode() which has two problems:
1) udf_clear_inode() is really called to late to do anything like freeing space from inode and UDF has to play some nasty tricks (like explicitely syncing inode, calling invalidate_inode_buffers() once more) to make everything work. Using udf_drop_inode() would be much better but that is called under i_lock so it's not really usable (unless I'd drop i_lock in it but then it would even nastier).
2) When you create lots of directories, they use quite some additional space for preallocation and there's not really a got way how to get rid of preallocated blocks than dropping dcache or umount + mount the filesystem.

Quoted 7 lines
Quoted 3 lines

So after all I'm thinking whether directory preallocation is really worth
the trouble. But before I go and rip it I'd like to check whether someone does not have some clever idea :).

Problem 2 has lru written all over, don't you think? Problem 1 is similar to something I'm debugging this very moment. With some luck my solution might work for you as well.

Hmm, yes, I was thinking of implementing something like LRU. I just
thought it might be a bit overkill. But maybe not...

Quoted 3 lines

However, I don't see a big difference between files and directories yet. Maybe because there shouldn't be, or maybe because I've been missing something critical. Could you elaborate on that?

For files I use ->release() callback to drop preallocation. So
preallocation is dropped on last close of a file which is not completely optimal but generally good enough.
Thanks for your ideas.

Honza

Signature · 6 lines
Jan Kara <[email protected]>
SUSE Labs, CR
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html