ext4: Reserve metadata if writing into uninitialized
PATCH 1/3 ext4: Reserve metadata if writing into uninitialized
Currently in delalloc write we do not reserve any space if we're writing into the uninitialized extent. This is ok for data, because the space has already been allocated so we do not have to do data reservation, however we have to reserve metadata for uninitialized extent conversion on writeback.
Add new ext4_da_reserve_metadata() function to only reserve metadata blocks for delayed allocation an use it if we're writing delayed blocks into unwritten extent to reserve metadata for extent conversion.
The problem can be reproduced with xfstest 083 on bigalloc file system. With this patch I can not reproduce the problem anymore.
--- fs/ext4/ext4.h | 1 + fs/ext4/inode.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++++----- 2 files changed, 70 insertions(+), 7 deletions(-)
fs/ext4/ext4.h +1 -0
@@ -581,6 +581,7 @@ enum {#define EXT4_GET_BLOCKS_NO_LOCK 0x0100/* Do not put hole in extent cache */ #define EXT4_GET_BLOCKS_NO_PUT_HOLE 0x0200 +#define EXT4_GET_BLOCKS_METADATA_RESERVED 0x0400/* * Flags used by ext4_free_blocks
fs/ext4/inode.c +69 -7
119 lines not on this page · raw message
Signature · 1 line
1.7.7.6
PATCH 2/3 ext4: Update reserved space after the 'correction'
Currently in ext4_ext_map_blocks() in delayed allocation writeback we would update the reservation and after that check whether we claimed cluster outside of the range of the allocation and if so, we'll give the block back to the reservation pool.
However this also means that if the number of reserved data block dropped to zero before the correction, we would release all the metadata reservation as well, however we might still need it because the we're not done with the delayed allocation and there might be more blocks to come. This will result in error messages such as:
EXT4-fs warning (device sdb): ext4_da_update_reserve_space:361: ino 12, allocated 1 with only 0 reserved metadata blocks (releasing 1 blocks with reserved 1 data blocks)
This will only happen on bigalloc file system and it can be easily reproduced using fiemap-tester from xfstests like this:
./src/fiemap-tester -m DHDHDHDHD -S -p0 /mnt/test/file
Or using xfstests such as 225.
Fix this by doing the correction first and updating the reservation after that so that we do not accidentally decrease i_reserved_data_blocks to zero.
--- fs/ext4/extents.c | 12 +++++++++--- 1 files changed, 9 insertions(+), 3 deletions(-)
fs/ext4/extents.c +9 -3
@@ -4106,9 +4106,6 @@ got_allocated_blocks: } }else {BUG_ON (allocated_clusters < reserved_clusters); -/* We will claim quota for all newly allocated blocks.*/ -ext4_da_update_reserve_space (inode, allocated_clusters, -1 );if (reserved_clusters < allocated_clusters) {struct ext4_inode_info *ei =EXT4_I (inode); int reservation = allocated_clusters -
@@ -4159,6 +4156,15 @@ got_allocated_blocks: ei->i_reserved_data_blocks += reservation;spin_unlock (&ei->i_block_reservation_lock); } +/* +* We will claim quota for all newly allocated blocks. +* We're updating the reserved space *after* the +* correction above so we do not accidentally free +* all the metadata reservation because we might +* actually need it later on. +*/ +ext4_da_update_reserve_space (inode, allocated_clusters, +1 ); } }
Signature · 1 line
1.7.7.6
Re: [PATCH 2/3] ext4: Update reserved space after the 'correction'
Quoted 20 lines · Lukas Czerner
Currently in
ext4_ext_map_blocks()in delayed allocation writeback we would update the reservation and after that check whether we claimed cluster outside of the range of the allocation and if so, we'll give the block back to the reservation pool.However this also means that if the number of reserved data block dropped to zero before the correction, we would release all the metadata reservation as well, however we might still need it because the we're not done with the delayed allocation and there might be more blocks to come. This will result in error messages such as:
EXT4-fs warning (device sdb): ext4_da_update_reserve_space:361: ino 12, allocated 1 with only 0 reserved metadata blocks (releasing 1 blocks with reserved 1 data blocks)
This will only happen on bigalloc file system and it can be easily reproduced using fiemap-tester from xfstests like this:
./src/fiemap-tester -m DHDHDHDHD -S -p0
/mnt/test/file
Quoted 7 lines
Or using xfstests such as 225.
Fix this by doing the correction first and updating the reservation after that so that we do not accidentally decrease
i_reserved_data_blocksto zero.Signed-off-by Lukas Czerner <[email protected]>
Thanks, applied.
- Ted
PATCH 3/3 ext4: reserve metadata block for every delayed write
In reply to Lukas CzernerCurrently we only reserve space (data+metadata) in delayed allocation if we're allocating from new cluster (which is always in non-bigalloc file system) which is ok for data blocks, because we reserve whole cluster.
However we have to reserve metadata for every delayed block we're going to write because every block could potentially require metedata block when we need to grow the extent tree.
--- fs/ext4/inode.c | 14 ++++++++++++-- 1 files changed, 12 insertions(+), 2 deletions(-)
fs/ext4/inode.c +12 -2
@@ -1905,8 +1905,11 @@ add_delayed:* XXX: __block_prepare_write() unmaps passed block, * is it OK? */ -/* If the block was allocated from previously allocated cluster, -* then we dont need to reserve it again. */ +/* +* If the block was allocated from previously allocated cluster, +* then we don't need to reserve it again. However we still need +* to reserve metadata for every block we're going to write. +*/ if (!(map->m_flags &EXT4_MAP_FROM_CLUSTER )) { ret =ext4_da_reserve_space (inode, iblock);if (ret) {
@@ -1914,6 +1917,13 @@ add_delayed: retval = ret; goto out_unlock; } + }else { + ret =ext4_da_reserve_metadata (inode, iblock); +if (ret) { +/* not enough space to reserve */ + retval = ret; + goto out_unlock; + } } ret = ext4_es_insert_extent(inode, map->m_lblk, map->m_len,
Signature · 1 line
1.7.7.6
Re: [PATCH 3/3] ext4: reserve metadata block for every delayed write
Quoted 9 lines · Lukas Czerner
Currently we only reserve space (data+metadata) in delayed allocation if we're allocating from new cluster (which is always in non-bigalloc file system) which is ok for data blocks, because we reserve whole cluster.
However we have to reserve metadata for every delayed block we're going to write because every block could potentially require metedata block when we need to grow the extent tree.
Signed-off-by Lukas Czerner <[email protected]>
Thanks, applied.
- Ted
Quoted 6 lines · Lukas Czerner
Currently in delalloc write we do not reserve any space if we're writing into the uninitialized extent. This is ok for data, because the space has already been allocated so we do not have to do data reservation, however we have to reserve metadata for uninitialized extent conversion on writeback.
Quoted 8 lines
Add new
ext4_da_reserve_metadata()function to only reserve metadata blocks for delayed allocation an use it if we're writing delayed blocks into unwritten extent to reserve metadata for extent conversion.The problem can be reproduced with xfstest 083 on bigalloc file system. With this patch I can not reproduce the problem anymore.
Signed-off-by Lukas Czerner <[email protected]>
Thanks, applied.
- Ted