ext4: flush delalloc blocks when space is low
Subsystems (2)
PATCH 2/2 ext4: flush delalloc blocks when space is low
Creating many small files in rapid succession on a small filesystem can lead to spurious ENOSPC; on a 104MB filesystem:
for i in seq 1 22500; do
echo -n > $SCRATCH_MNT/$i
echo XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX > $SCRATCH_MNT/$i
done
leads to ENOSPC even though after a sync, 40% of the fs is free again.
This is because we reserve worst-case metadata for delalloc writes, and when data is allocated that worst-case reservation is not usually needed.
When freespace is low, kicking off an async writeback will start converting that worst-case space usage into something more realistic, almost always freeing up space to continue.
This resolves the testcase for me, and survives all 4 generic ENOSPC tests in xfstests.
We'll still need a hard synchronous sync to squeeze out the last bit, but this fixes things up to a large degree.
---
fs/ext4/inode.c +9 -2
@@ -3024,11 +3024,18 @@ static int ext4_nonda_switch(struct super_block *sb)if (2 * free_blocks <3 * dirty_blocks || free_blocks < (dirty_blocks +EXT4_FREEBLOCKS_WATERMARK )) {/* -* free block count is less that 150% of dirty blocks -* or free blocks is less that watermark +* free block count is less than 150% of dirty blocks +* or free blocks is less than watermark */ return 1 ; } +/* +* Even if we don't switch but are nearing capacity, +* start pushing delalloc when 1/2 of free blocks are dirty. +*/ +if (free_blocks <2 * dirty_blocks) +writeback_inodes_sb_if_idle (sb); +return 0 ; }
Quoted 19 lines
Creating many small files in rapid succession on a small filesystem can lead to spurious ENOSPC; on a 104MB filesystem:
for i in
seq 1 22500; do
echo -n > $SCRATCH_MNT/$i
echo XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX > $SCRATCH_MNT/$i
doneleads to ENOSPC even though after a sync, 40% of the fs is free again.
This is because we reserve worst-case metadata for delalloc writes, and when data is allocated that worst-case reservation is not usually needed.
When freespace is low, kicking off an async writeback will start converting that worst-case space usage into something more realistic, almost always freeing up space to continue.
Quoted 7 lines
This resolves the testcase for me, and survives all 4 generic ENOSPC tests in xfstests.
We'll still need a hard synchronous sync to squeeze out the last bit, but this fixes things up to a large degree.
Signed-off-by Eric Sandeen <[email protected]>
Looks good.
Honza
Quoted 32 lines
---
fs/ext4/inode.c +9
-3@@ -3024,11 +3024,18 @@ static int ext4_nonda_switch(struct super_block *sb)if (2 * free_blocks <3 * dirty_blocks || free_blocks < (dirty_blocks +EXT4_FREEBLOCKS_WATERMARK )) {/* -* free block count is less that 150% of dirty blocks -* or free blocks is less that watermark +* free block count is less than 150% of dirty blocks +* or free blocks is less than watermark */ return 1 ; } +/* +* Even if we don't switch but are nearing capacity, +* start pushing delalloc when 1/2 of free blocks are dirty. +*/ +if (free_blocks <2 * dirty_blocks) +writeback_inodes_sb_if_idle (sb); +return 0 ; } --To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 2 lines
Jan Kara <[email protected]> SuSE CR Labs
[email protected]
maintainerQuoted 19 lines · Jan Kara
Quoted 19 lines
Creating many small files in rapid succession on a small filesystem can lead to spurious ENOSPC; on a 104MB filesystem:
for i in
seq 1 22500; do
echo -n > $SCRATCH_MNT/$i
echo XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX > $SCRATCH_MNT/$i
doneleads to ENOSPC even though after a sync, 40% of the fs is free again.
This is because we reserve worst-case metadata for delalloc writes, and when data is allocated that worst-case reservation is not usually needed.
When freespace is low, kicking off an async writeback will start converting that worst-case space usage into something more realistic, almost always freeing up space to continue.
Quoted 7 lines
Quoted 7 lines
This resolves the testcase for me, and survives all 4 generic ENOSPC tests in xfstests.
We'll still need a hard synchronous sync to squeeze out the last bit, but this fixes things up to a large degree.
Signed-off-by Eric Sandeen <[email protected]>
Thanks, added to the ext4 patch queue.
- Ted