mark buffer_head mapping preallocate area as new during write_begin with delayed allocation
Subsystems (2)
RFC PATCH 1/1 mark buffer_head mapping preallocate area as new during write_begin with delayed allocation
We need to mark the buffer_head mapping prealloc space as new during write_begin. Otherwise we don't zero out the page cache content properly for a partial write. This will cause file corruption with preallocation.
--- fs/ext4/inode.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)
fs/ext4/inode.c +2 -0
@@ -2323,6 +2323,8 @@ static int ext4_da_get_block_prep(struct inode *inode, sector_t iblock,set_buffer_delay (bh_result); }else if (ret >0 ) { bh_result->b_size = (ret << inode->i_blkbits); +if (buffer_unwritten (bh_result)) +set_buffer_new (bh_result); ret =0 ; }
Signature · 1 line
tg: (51bec03..) preallocate_corruption (depends on: ext4_lock_group_conversion)
Quoted 16 lines · Aneesh Kumar K.V
We need to mark the
buffer_headmapping prealloc space as new duringwrite_begin. Otherwise we don't zero out the page cache content properly for a partial write. This will cause file corruption with preallocation.Signed-off-by Aneesh Kumar K.V <[email protected]>--- fs/ext4/inode.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)fs/ext4/inode.c +0
-0
fs/ext4/inode.c static int ext4_da_get_block_prep(struct inode *inode, sector_t iblock, · line 2330
Quoted 8 lines
fs/ext4/inode.c +2
-0@@ -2323,6 +2323,8 @@ static int ext4_da_get_block_prep(struct inode *inode, sector_t iblock,set_buffer_delay (bh_result); }else if (ret >0 ) { bh_result->b_size = (ret << inode->i_blkbits); +if (buffer_unwritten (bh_result)) +set_buffer_new (bh_result); ret =0 ; }
Hm, yep, that sure does break things. For some future regression test:
[root@inode tmp]# /root/fallocate -l 8k testfile
[root@inode tmp]# dd if=/dev/zero of=testfile bs=1 count=10 conv=notrunc 10+0 records in
10+0 records out
10 bytes (10 B) copied, 5.1491e-05 s, 194 kB/s
[root@inode tmp]# hexdump -C testfile
<much garbage ensues>
This looks pretty reasonable; Aneesh & I talked online and found that xfs has a somewhat similar fix:
commit 549054afadae44889c0b40d4c3bfb0207b98d5a0
Author: David Chinner <[email protected]> Date: Sat Feb 10 18:36:35 2007 +1100
[XFS] Fix sub-block zeroing for buffered writes into unwritten extents.
When writing less than a filesystem block of data into an unwritten
extent
via buffered I/O, __xfs_get_blocks fails to set the buffer new flag.
As a
result, the generic code will not zero either edge of the block
resulting
in garbage being written to disk either side of the real data. Set the buffer new state on bufferd writes to unwritten extents to ensure that zeroing occurs.
-Eric
Quoted 17 lines · 在 2009-04-28二的 00:35 +0530,Aneesh Kumar K.V写道:
We need to mark the
buffer_headmapping prealloc space as new duringwrite_begin. Otherwise we don't zero out the page cache content properly for a partial write. This will cause file corruption with preallocation.Signed-off-by Aneesh Kumar K.V <[email protected]>--- fs/ext4/inode.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)fs/ext4/inode.c +0
-0
fs/ext4/inode.c static int ext4_da_get_block_prep(struct inode *inode, sector_t iblock, · line 2330
Quoted 8 lines
fs/ext4/inode.c +2
-0@@ -2323,6 +2323,8 @@ static int ext4_da_get_block_prep(struct inode *inode, sector_t iblock,set_buffer_delay (bh_result); }else if (ret >0 ) { bh_result->b_size = (ret << inode->i_blkbits); +if (buffer_unwritten (bh_result)) +set_buffer_new (bh_result); ret =0 ; }
Thanks Aneesh.
Just to share with list, I have seen garbage content show up on a preallocated but later partially written blocks. This only happens with delayed allocation. The test simply preallocate 2blocks to a new file, then write a few bytes to the beginning of file(less than a block), and od shows the first block the written content followed by garbage filled to the end of the first block.
After examing the code, we did set the buffer as new for nondelalloc, as the create flag passed to ext4_ext_get_blocks() is 1, while for delalloc case, ext4_get_blocks_prep() calling ext4_ext_get_block() with create =0, which leads to the code path that forget to set the bh as new if the block is preallocated.
This patch is mostly correct except forget to set the bh_result->bdev, which caused the fs blow out.
The updated patch fixed the problem for me.
fs/ext4/inode.c +3 -0
@@ -2177,7 +2177,10 @@ static int ext4_da_get_block_prep(structset_buffer_new (bh_result);set_buffer_delay (bh_result); }else if (ret >0 ) { +if (buffer_unwritten (bh_result)) +set_buffer_new (bh_result); bh_result->b_size = (ret << inode->i_blkbits); + bh_result->b_bdev = inode->i_sb->s_bdev; ret =0 ; }
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 36 lines · Mingming Cao
Quoted 17 lines · 在 2009-04-28二的 00:35 +0530,Aneesh Kumar K.V写道:
We need to mark the
buffer_headmapping prealloc space as new duringwrite_begin. Otherwise we don't zero out the page cache content properly for a partial write. This will cause file corruption with preallocation.Signed-off-by Aneesh Kumar K.V <[email protected]>--- fs/ext4/inode.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)fs/ext4/inode.c +0
-0fs/ext4/inode.c static int ext4_da_get_block_prep(struct inode *inode, sector_t iblock, · line 2330
Quoted 8 lines
fs/ext4/inode.c +2
-0@@ -2323,6 +2323,8 @@ static int ext4_da_get_block_prep(struct inode *inode, sector_t iblock,set_buffer_delay (bh_result); }else if (ret >0 ) { bh_result->b_size = (ret << inode->i_blkbits); +if (buffer_unwritten (bh_result)) +set_buffer_new (bh_result); ret =0 ; }Thanks Aneesh.
Just to share with list, I have seen garbage content show up on a preallocated but later partially written blocks. This only happens with delayed allocation. The test simply preallocate 2blocks to a new file, then write a few bytes to the beginning of file(less than a block), and od shows the first block the written content followed by garbage filled to the end of the first block.
Quoted 8 lines
After examing the code, we did set the buffer as new for nondelalloc, as the create flag passed to
ext4_ext_get_blocks()is 1, while for delalloc case,ext4_get_blocks_prep()callingext4_ext_get_block()with create =0, which leads to the code path that forget to set the bh as new if the block is preallocated.This patch is mostly correct except forget to set the
bh_result->bdev, which caused the fs blow out.
Yep, I saw the oops too.
Quoted 9 lines
The updated patch fixed the problem for me.
Signed-off-by Mingming Cao <[email protected]>Index:
linux-2.6.28-rc6/fs/ext4/inode.c
===================================================================
fs/ext4/inode.c static int ext4_da_get_block_prep(struct · line 2185
Quoted 9 lines
fs/ext4/inode.c +3
-0@@ -2177,7 +2177,10 @@ static int ext4_da_get_block_prep(structset_buffer_new (bh_result);set_buffer_delay (bh_result); }else if (ret >0 ) { +if (buffer_unwritten (bh_result)) +set_buffer_new (bh_result); bh_result->b_size = (ret << inode->i_blkbits); + bh_result->b_bdev = inode->i_sb->s_bdev; ret =0 ; }
It may be just me, but I'd like to sort out why we now need to set b_bdev here just because we set it as new, and it wasn't necessary before...? If it's obvious I'm not yet seeing it :)
-Eric
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Apr 27, 2009 at 04:04:54PM -0700, Mingming Cao wrote:
.....
fs/ext4/inode.c static int ext4_da_get_block_prep(struct · line 2183
Quoted 13 lines
Index:
linux-2.6.28-rc6/fs/ext4/inode.c
===================================================================fs/ext4/inode.c +3
-0@@ -2177,7 +2177,10 @@ static int ext4_da_get_block_prep(structset_buffer_new (bh_result);set_buffer_delay (bh_result); }else if (ret >0 ) { +if (buffer_unwritten (bh_result)) +set_buffer_new (bh_result); bh_result->b_size = (ret << inode->i_blkbits); + bh_result->b_bdev = inode->i_sb->s_bdev;
Updated patch to set bh_result->b_dev. I also added comments in the source to explain whey we need to mark buffer_head new. Also updated single line patch summary. I will send the update (-v2) patch.
-aneesh
Quoted 13 lines · Aneesh Kumar K.V
On Mon, Apr 27, 2009 at 04:04:54PM -0700, Mingming Cao wrote:
.....Quoted 10 lines
Index:
linux-2.6.28-rc6/fs/ext4/inode.c
===================================================================fs/ext4/inode.c +1
-0@@ -2177,7 +2177,10 @@ static int ext4_da_get_block_prep(structset_buffer_new (bh_result);set_buffer_delay (bh_result); }else if (ret >0 ) { +if (buffer_unwritten (bh_result))
Quoted 8 lines
fs/ext4/inode.c static int ext4_da_get_block_prep(struct · line 2183
Quoted 3 lines
fs/ext4/inode.c +2
-0+set_buffer_new (bh_result); bh_result->b_size = (ret << inode->i_blkbits); + bh_result->b_bdev = inode->i_sb->s_bdev;Updated patch to set
bh_result->b_dev. I also added comments in the source to explain whey we need to markbuffer_headnew. Also updated single line patch summary. I will send the update (-v2) patch.
Looking at the source again i guess setting just b_dev is not enough. unmap_underlying_metadata looks at the mapping block number, which we don't have in case on unwritten buffer_head. How about the below patch ? It involve vfs changes. But i guess it is correct with respect to the meaning of BH_New (Disk mapping was newly created by get_block). I guess BH_New implies BH_Mapped.
I haven't tested the patch yet. Also it should be split into multiple patches. It also a fix a problem where we missed an unamp_underlying_metadata in case of delayed allocated blocks. I guess that can also cause corruption with delayed allocation.
From: Aneesh Kumar K.V <[email protected]>
Subject: [PATCH -V3] ext4: Fix sub-block zeroing for buffered writes into unwritten extents.
We need to mark the buffer_head mapping prealloc space as new during write_begin. Otherwise we don't zero out the page cache content properly for a partial write. This will cause file corruption with preallocation.
--- fs/buffer.c | 9 ++++++++- fs/ext4/inode.c | 8 +++++--- 2 files changed, 13 insertions(+), 4 deletions(-)
fs/buffer.c +8 -1
@@ -1867,15 +1867,22 @@ static int __block_prepare_write(struct inode *inode, struct page *page, err =get_block (inode, block, bh,1 );if (err)break ; -if (buffer_new (bh)) { +if (buffer_new (bh))unmap_underlying_metadata (bh->b_bdev, bh->b_blocknr); +if (buffer_new (bh) ||buffer_unwritten (bh) || +buffer_delay (bh)) {if (PageUptodate (page)) {clear_buffer_new (bh);set_buffer_uptodate (bh);mark_buffer_dirty (bh);continue ; } +/* +* sub-block writes into unwritten or +* delayed buffer should result in zero out +* of the rest of the buffer +*/ if (block_end > to || block_start < from) zero_user_segments(page, to, block_end,
fs/ext4/inode.c +5 -3
@@ -1892,13 +1892,17 @@ static void mpage_put_bnr_to_bhs(struct mpage_da_data *mpd, sector_t logical,if (buffer_delay (bh)) { bh->b_blocknr = pblock;clear_buffer_delay (bh); +set_buffer_mapped (bh); bh->b_bdev = inode->i_sb->s_bdev; +unmap_underlying_metadata (bh->b_bdev, + pblock); }else if (buffer_unwritten (bh)) { bh->b_blocknr = pblock;clear_buffer_unwritten (bh);set_buffer_mapped (bh); -set_buffer_new (bh); bh->b_bdev = inode->i_sb->s_bdev; +unmap_underlying_metadata (bh->b_bdev, + pblock); }else if (buffer_mapped (bh))BUG_ON (bh->b_blocknr != pblock);
@@ -2318,8 +2322,6 @@ static int ext4_da_get_block_prep(struct inode *inode, sector_t iblock,/* not enough space to reserve */ return ret; -map_bh (bh_result, inode->i_sb,0 ); -set_buffer_new (bh_result);set_buffer_delay (bh_result); }else if (ret >0 ) { bh_result->b_size = (ret << inode->i_blkbits);
Signature · 1 line
tg: (2084a87..) preallocate_corruption (depends on: ext4_lock_group_conversion)
Theodore Tso
maintainerQuoted 7 lines · Aneesh Kumar K.V
Looking at the source again i guess setting just
b_devis not enough.unmap_underlying_metadatalooks at the mapping block number, which we don't have in case on unwrittenbuffer_head. How about the below patch ? It involve vfs changes. But i guess it is correct with respect to the meaning ofBH_New(Disk mapping was newly created byget_block). I guessBH_NewimpliesBH_Mapped.
Argh. So we have multiple problems going on here. One is the original problem, namely that of a partial write into an preallocated block can leave garbage behind in that unitialized block.
The other problem seems to be in the case of a delayed allocation write, where we return a buffer_head which is marked new, and this causes block_prepare_write() to call unmap_underlying_metadata(dev, 0).
In theory this could cause problems if we try installing a new bootloader in the filesystem's boot block while there's a delayed writes happening in the background, since we could end up discarding the write to the boot sector. We've lived with this for quite a wihle though.
My concern with making the fs/buffer.c changes is that we need to make sure it doesn't break any of the other filesystems, so that's going to make it hard to try to slip this with 2.6.30-rc4 nearly upon us. (Silly question; why doesn't XFS get caught by this?)
So the question is do we try to fix both bugs with one patch, and very likely have to wait until 2.6.31 before the patch is incorporated? Or do we fix the second bug using an ext4-only fix, with the knowledge that post 2.6.30, we'll need undo most of it and fix it properly with a change that involves fs/buffer.c?
My preference is for the former, unless we belive the 2nd bug is serious enough that we really need to address it ASAP (in which case we have a lot of work ahead of us in terms of coordinating with the
other filesystem developers). What do other folks think?
- Ted
Quoted 9 lines · Theodore Tso
Quoted 7 lines · Aneesh Kumar K.V
Looking at the source again i guess setting just
b_devis not enough.unmap_underlying_metadatalooks at the mapping block number, which we don't have in case on unwrittenbuffer_head. How about the below patch ? It involve vfs changes. But i guess it is correct with respect to the meaning ofBH_New(Disk mapping was newly created byget_block). I guessBH_NewimpliesBH_Mapped.
Quoted 7 lines
Argh. So we have multiple problems going on here. One is the original problem, namely that of a partial write into an preallocated block can leave garbage behind in that unitialized block.
The other problem seems to be in the case of a delayed allocation write, where we return a
buffer_headwhich is marked new, and this causesblock_prepare_write()to callunmap_underlying_metadata(dev, 0).
Not just that. On block allocation we are not calling unmap_underlying_metadata(dev, blocknumber) for delayed allocated blocks. That would imply file corruption.
Quoted 18 lines
In theory this could cause problems if we try installing a new bootloader in the filesystem's boot block while there's a delayed writes happening in the background, since we could end up discarding the write to the boot sector. We've lived with this for quite a wihle though.
My concern with making the
fs/buffer.cchanges is that we need to make sure it doesn't break any of the other filesystems, so that's going to make it hard to try to slip this with 2.6.30-rc4 nearly upon us. (Silly question; why doesn't XFS get caught by this?)So the question is do we try to fix both bugs with one patch, and very likely have to wait until 2.6.31 before the patch is incorporated? Or do we fix the second bug using an ext4-only fix, with the knowledge that post 2.6.30, we'll need undo most of it and fix it properly with a change that involves
fs/buffer.c?
Quoted 4 lines
My preference is for the former, unless we belive the 2nd bug is serious enough that we really need to address it ASAP (in which case we have a lot of work ahead of us in terms of coordinating with the
other filesystem developers). What do other folks think?
The original reported problem is something really easy to reproduce. So i guess if we can have a ext4 local change that would fix the original problem that would be good. Considering that map_bh(bdev, 0) didn't create any issues till now, what we can do is to do a similar update for unwritten_buffer in ext4_da_block_write_prep. That's the v2 version of the patch with the below addition
bh_result->b_blocknr = 0;
-aneesh
Theodore Tso
maintainerQuoted 14 lines · Aneesh Kumar K.V
Quoted 13 lines · Theodore Tso
Quoted 7 lines · Aneesh Kumar K.V
Looking at the source again i guess setting just
b_devis not enough.unmap_underlying_metadatalooks at the mapping block number, which we don't have in case on unwrittenbuffer_head. How about the below patch ? It involve vfs changes. But i guess it is correct with respect to the meaning ofBH_New(Disk mapping was newly created byget_block). I guessBH_NewimpliesBH_Mapped.Argh. So we have multiple problems going on here. One is the original problem, namely that of a partial write into an preallocated block can leave garbage behind in that unitialized block.
Quoted 7 lines
Quoted 3 lines
The other problem seems to be in the case of a delayed allocation write, where we return a
buffer_headwhich is marked new, and this causesblock_prepare_write()to callunmap_underlying_metadata(dev, 0).Not just that. On block allocation we are not calling
unmap_underlying_metadata(dev, blocknumber)for delayed allocated blocks. That would imply file corruption.
I don't think I'm following you . If we write into block that was delayed allocated. Are you saying we might get in trouble of the delayed allocation block is mmap'ed in?
Quoted 7 lines
The original reported problem is something really easy to reproduce. So i guess if we can have a ext4 local change that would fix the original problem that would be good. Considering that
map_bh(bdev, 0)didn't create any issues till now, what we can do is to do a similar update forunwritten_bufferinext4_da_block_write_prep. That's the v2 version of the patch with the below additionbh_result->b_blocknr = 0;
OK, I can put togehter a patch to do this. Whatever we do, I think we're going to need a *lot* of testing.
- Ted
Quoted 19 lines · Theodore Tso
Quoted 18 lines · Aneesh Kumar K.V
Quoted 16 lines · Theodore Tso
Quoted 7 lines · Aneesh Kumar K.V
Looking at the source again i guess setting just
b_devis not enough.unmap_underlying_metadatalooks at the mapping block number, which we don't have in case on unwrittenbuffer_head. How about the below patch ? It involve vfs changes. But i guess it is correct with respect to the meaning ofBH_New(Disk mapping was newly created byget_block). I guessBH_NewimpliesBH_Mapped.Argh. So we have multiple problems going on here. One is the original problem, namely that of a partial write into an preallocated block can leave garbage behind in that unitialized block.
The other problem seems to be in the case of a delayed allocation write, where we return a
buffer_headwhich is marked new, and this causesblock_prepare_write()to callunmap_underlying_metadata(dev, 0).
Quoted 7 lines
Quoted 3 lines
Not just that. On block allocation we are not calling
unmap_underlying_metadata(dev, blocknumber)for delayed allocated blocks. That would imply file corruption.I don't think I'm following you . If we write into block that was delayed allocated. Are you saying we might get in trouble of the delayed allocation block is mmap'ed in?
We allocate blocks for delayed buffer during writepage. Now we need to make sure after getting the blocks we drop the old buffer_head mapping that we may have with this particular block attached to the block device. That is done by calling unmap_underlying_metadata. Now the current code doesn't call unmap_underlying_metadata for delayed allocated blocks. That would mean we can see corrupt files if old buffer_head mapping gets synced to disk AFTER we write the new buffer_head mapping.
Quoted 11 lines
Quoted 8 lines
The original reported problem is something really easy to reproduce. So i guess if we can have a ext4 local change that would fix the original problem that would be good. Considering that
map_bh(bdev, 0)didn't create any issues till now, what we can do is to do a similar update forunwritten_bufferinext4_da_block_write_prep. That's the v2 version of the patch with the below additionbh_result->b_blocknr = 0;OK, I can put togehter a patch to do this. Whatever we do, I think we're going to need a *lot* of testing.
I just sent -v3 version
-aneesh
Quoted 2 lines · Aneesh Kumar K.V
Quoted 1 line · Theodore Tso
On Tue, Apr 28, 2009 at 10:05:54PM +0530, Aneesh Kumar K.V wrote:
...
Quoted 10 lines
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
The other problem seems to be in the case of a delayed allocation write, where we return a
buffer_headwhich is marked new, and this causesblock_prepare_write()to callunmap_underlying_metadata(dev, 0).Not just that. On block allocation we are not calling
unmap_underlying_metadata(dev, blocknumber)for delayed allocated blocks. That would imply file corruption.I don't think I'm following you . If we write into block that was delayed allocated. Are you saying we might get in trouble of the delayed allocation block is mmap'ed in?
Quoted 8 lines
We allocate blocks for delayed buffer during writepage. Now we need to make sure after getting the blocks we drop the old
buffer_headmapping that we may have with this particular block attached to the block device. That is done by callingunmap_underlying_metadata. Now the current code doesn't callunmap_underlying_metadatafor delayed allocated blocks. That would mean we can see corrupt files if oldbuffer_headmapping gets synced to disk AFTER we write the newbuffer_headmapping.
Talking w/ Aneesh on IRC, I don't see how we can have stray dirty mappings lying around for this block device unless someone is writing directly to the mounted block device, which I don't think is ever considered safe ...
I'm not quite sure what the call to __unmap_underlying_blocks() in mpage_da_map_blocks() is for, I guess?
-Eric
Jan Kara
reviewerQuoted 24 lines
Quoted 2 lines · Aneesh Kumar K.V
Quoted 1 line · Theodore Tso
On Tue, Apr 28, 2009 at 10:05:54PM +0530, Aneesh Kumar K.V wrote:
...
Quoted 10 lines
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
The other problem seems to be in the case of a delayed allocation write, where we return a
buffer_headwhich is marked new, and this causesblock_prepare_write()to callunmap_underlying_metadata(dev, 0).Not just that. On block allocation we are not calling
unmap_underlying_metadata(dev, blocknumber)for delayed allocated blocks. That would imply file corruption.I don't think I'm following you . If we write into block that was delayed allocated. Are you saying we might get in trouble of the delayed allocation block is mmap'ed in?
Quoted 8 lines
We allocate blocks for delayed buffer during writepage. Now we need to make sure after getting the blocks we drop the old
buffer_headmapping that we may have with this particular block attached to the block device. That is done by callingunmap_underlying_metadata. Now the current code doesn't callunmap_underlying_metadatafor delayed allocated blocks. That would mean we can see corrupt files if oldbuffer_headmapping gets synced to disk AFTER we write the newbuffer_headmapping.
Quoted 7 lines
Talking w/ Aneesh on IRC, I don't see how we can have stray dirty mappings lying around for this block device unless someone is writing directly to the mounted block device, which I don't think is ever considered safe ...
I'm not quite sure what the call to
__unmap_underlying_blocks()inmpage_da_map_blocks()is for, I guess?
For ext3 / ext4 I think we don't need unmap_underlying_blocks() since
before we reallocate a block, we make sure that the transaction freeing the block is committed and clear all dirty bits from freed blocks.
But for more careless filesystems, if they reallocate metadata block
as a data block and don't clear the dirty bit in blockdev mapping, unmap_underlying_blocks() does it for them.
Honza
Signature · 2 lines
Jan Kara <[email protected]> SuSE CR Labs
Quoted 27 lines · Jan Kara
Quoted 27 lines
Quoted 2 lines · Aneesh Kumar K.V
Quoted 1 line · Theodore Tso
On Tue, Apr 28, 2009 at 10:05:54PM +0530, Aneesh Kumar K.V wrote:
...
Quoted 9 lines
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
The other problem seems to be in the case of a delayed allocation write, where we return a
buffer_headwhich is marked new, and this causesblock_prepare_write()to callunmap_underlying_metadata(dev, 0).Not just that. On block allocation we are not calling
unmap_underlying_metadata(dev, blocknumber)for delayed allocated blocks. That would imply file corruption.I don't think I'm following you . If we write into block that was delayed allocated. Are you saying we might get in trouble of the delayed allocation block is mmap'ed in?
Quoted 8 lines
We allocate blocks for delayed buffer during writepage. Now we need to make sure after getting the blocks we drop the old
buffer_headmapping that we may have with this particular block attached to the block device. That is done by callingunmap_underlying_metadata. Now the current code doesn't callunmap_underlying_metadatafor delayed allocated blocks. That would mean we can see corrupt files if oldbuffer_headmapping gets synced to disk AFTER we write the newbuffer_headmapping.Talking w/ Aneesh on IRC, I don't see how we can have stray dirty mappings lying around for this block device unless someone is writing directly to the mounted block device, which I don't think is ever considered safe ...
Quoted 8 lines
Quoted 2 lines
I'm not quite sure what the call to
__unmap_underlying_blocks()inmpage_da_map_blocks()is for, I guess?For ext3 / ext4 I think we don't need
unmap_underlying_blocks()since
before we reallocate a block, we make sure that the transaction freeing the block is committed and clear all dirty bits from freed blocks.
But for more careless filesystems, if they reallocate metadata block
as a data block and don't clear the dirty bit in blockdev mapping,unmap_underlying_blocks()does it for them.
That's what I thought - so I was wondering why we have specific calls to this in ext4:
mpage_da_map_blocks
__unmap_underlying_blocks for (i = 0; i < blocks; i++) unmap_underlying_metadata
-Eric
Jan Kara
reviewerQuoted 37 lines · Eric Sandeen
Quoted 27 lines · Jan Kara
Quoted 27 lines
Quoted 2 lines · Aneesh Kumar K.V
Quoted 1 line · Theodore Tso
On Tue, Apr 28, 2009 at 10:05:54PM +0530, Aneesh Kumar K.V wrote:
...
Quoted 9 lines
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
The other problem seems to be in the case of a delayed allocation write, where we return a
buffer_headwhich is marked new, and this causesblock_prepare_write()to callunmap_underlying_metadata(dev, 0).Not just that. On block allocation we are not calling
unmap_underlying_metadata(dev, blocknumber)for delayed allocated blocks. That would imply file corruption.I don't think I'm following you . If we write into block that was delayed allocated. Are you saying we might get in trouble of the delayed allocation block is mmap'ed in?
Quoted 8 lines
We allocate blocks for delayed buffer during writepage. Now we need to make sure after getting the blocks we drop the old
buffer_headmapping that we may have with this particular block attached to the block device. That is done by callingunmap_underlying_metadata. Now the current code doesn't callunmap_underlying_metadatafor delayed allocated blocks. That would mean we can see corrupt files if oldbuffer_headmapping gets synced to disk AFTER we write the newbuffer_headmapping.Talking w/ Aneesh on IRC, I don't see how we can have stray dirty mappings lying around for this block device unless someone is writing directly to the mounted block device, which I don't think is ever considered safe ...
Quoted 8 lines
Quoted 2 lines
I'm not quite sure what the call to
__unmap_underlying_blocks()inmpage_da_map_blocks()is for, I guess?For ext3 / ext4 I think we don't need
unmap_underlying_blocks()since
before we reallocate a block, we make sure that the transaction freeing the block is committed and clear all dirty bits from freed blocks.
But for more careless filesystems, if they reallocate metadata block
as a data block and don't clear the dirty bit in blockdev mapping,unmap_underlying_blocks()does it for them.
Quoted 7 lines
That's what I thought - so I was wondering why we have specific calls to this in ext4:
mpage_da_map_blocks__unmap_underlying_blocks for (i = 0; i < blocks; i++) unmap_underlying_metadata
Hmm, OK. So maybe change it warn on dirty blockdev buffer and if the warning
does not trigger we can believe that our theory is right ;).
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Quoted 35 lines · Theodore Tso
Quoted 14 lines · Aneesh Kumar K.V
Quoted 13 lines · Theodore Tso
Quoted 7 lines · Aneesh Kumar K.V
Looking at the source again i guess setting just
b_devis not enough.unmap_underlying_metadatalooks at the mapping block number, which we don't have in case on unwrittenbuffer_head. How about the below patch ? It involve vfs changes. But i guess it is correct with respect to the meaning ofBH_New(Disk mapping was newly created byget_block). I guessBH_NewimpliesBH_Mapped.Argh. So we have multiple problems going on here. One is the original problem, namely that of a partial write into an preallocated block can leave garbage behind in that unitialized block.
Quoted 7 lines
Quoted 3 lines
The other problem seems to be in the case of a delayed allocation write, where we return a
buffer_headwhich is marked new, and this causesblock_prepare_write()to callunmap_underlying_metadata(dev, 0).Not just that. On block allocation we are not calling
unmap_underlying_metadata(dev, blocknumber)for delayed allocated blocks. That would imply file corruption.I don't think I'm following you . If we write into block that was delayed allocated. Are you saying we might get in trouble of the delayed allocation block is mmap'ed in?
Quoted 7 lines
The original reported problem is something really easy to reproduce. So i guess if we can have a ext4 local change that would fix the original problem that would be good. Considering that
map_bh(bdev, 0)didn't create any issues till now, what we can do is to do a similar update forunwritten_bufferinext4_da_block_write_prep. That's the v2 version of the patch with the below additionbh_result->b_blocknr = 0;
Quoted 4 lines
OK, I can put togehter a patch to do this. Whatever we do, I think we're going to need a *lot* of testing.
- Ted
Aneesh, Eric and I discussed this online today, we find a separate issue, the lookup on the preallocated extent doesn't set the buffer_mapped(), so loop up/write to the same preallocated block multiple times (e.g. write 1 byte at a time, for 10 bytes total) will end up calling ext4_get_blocks_wrap() multiple times.
It seems reasonable to set the buffer mapped for preallocated buffer, with blocknr set to the real mapped block number (rather than faked -1 for the buffer blocknr in the V3 proposed fix for partial write garbage issue), and later reply on unwritten flag to force the writepage()/mpage_da_map_blocks calls get_block() to do the unintialized extent split. But this change seems require more thoughts and heavy auditing, and not as urgency as the data corruption problem.
Mingming
Quoted 22 lines · Theodore Tso
Quoted 6 lines · Aneesh Kumar K.V
Looking at the source again i guess setting just
b_devis not enough.unmap_underlying_metadatalooks at the mapping block number, which we don't have in case on unwrittenbuffer_head. How about the below patch ? It involve vfs changes. But i guess it is correct with respect to the meaning ofBH_New(Disk mapping was newly created byget_block). I guessBH_NewimpliesBH_Mapped.Argh. So we have multiple problems going on here. One is the original problem, namely that of a partial write into an preallocated block can leave garbage behind in that unitialized block.
The other problem seems to be in the case of a delayed allocation write, where we return a
buffer_headwhich is marked new, and this causesblock_prepare_write()to callunmap_underlying_metadata(dev, 0).In theory this could cause problems if we try installing a new bootloader in the filesystem's boot block while there's a delayed writes happening in the background, since we could end up discarding the write to the boot sector. We've lived with this for quite a wihle though.
Quoted 4 lines
My concern with making the
fs/buffer.cchanges is that we need to make sure it doesn't break any of the other filesystems, so that's going to make it hard to try to slip this with 2.6.30-rc4 nearly upon us. (Silly question; why doesn't XFS get caught by this?)
I'm not sure offhand. All xfs does is this in the get_block path:
* With sub-block writes into unwritten extents we also need to mark
* the buffer as new so that the unwritten parts of the buffer gets
* correctly zeroed.
*/
if (create &&
((!buffer_mapped(bh_result) && !buffer_uptodate(bh_result)) ||
(offset >= i_size_read(inode)) ||
(iomap.iomap_flags & (IOMAP_NEW|IOMAP_UNWRITTEN))))
set_buffer_new(bh_result);so it returns with BH_New as well.
Quoted 5 lines
So the question is do we try to fix both bugs with one patch, and very likely have to wait until 2.6.31 before the patch is incorporated? Or do we fix the second bug using an ext4-only fix, with the knowledge that post 2.6.30, we'll need undo most of it and fix it properly with a change that involves
fs/buffer.c?
I have the sense that this might need a bit more digging around, and I finally got stuff out of the way to do so :)
-Eric