Skip to contentSkip to conversation

ocfs2: Fill-in the unused portion of the block with zeros by dio_zero_block()

PATCH New jiangyiwen 5 participants 11 messages Sep 9, 2015 – Sep 15, 2015 OCFS2

lore
Share & tools
lore

jiangyiwen

author
#0 lore raw

PATCH 1/1 ocfs2: Fill-in the unused portion of the block with zeros by dio_zero_block()

A simplified test case is (this case from Ryan):
1) dd if=/dev/zero of=/mnt/hello bs=512 count=1 oflag=direct;
2) truncate /mnt/hello -s 2097152
file 'hello' is not exist before test. After this command, file 'hello' should be all zero. But 512~4096 is some random data.

Setting bh state to new when get a new block, if so, direct_io_worker()->dio_zero_block() will fill-in the unused portion of the block with zero.

---
 fs/ocfs2/aops.c | 1 +
 1 file changed, 1 insertion(+)
fs/ocfs2/aops.c +1 -0
index 1a35c61..bd106b9 100644
@@ -581,6 +581,7 @@ static int ocfs2_direct_IO_get_blocks(struct inode *inode, sector_t iblock,
 			ret = -EIO;
 			goto bail;
 		}
+		set_buffer_new(bh_result);
 	}
 
 	/*
Signature · 1 line
1.8.3.4
#1 lore raw

PATCH 1/1 ocfs2: Fill-in the unused portion of the block with zeros by dio_zero_block()

Quoted 11 lines · jiangyiwen

A simplified test case is (this case from Ryan):
1) dd if=/dev/zero of=/mnt/hello bs=512 count=1 oflag=direct;
2) truncate /mnt/hello -s 2097152
file 'hello' is not exist before test. After this command, file 'hello' should be all zero. But 512~4096 is some random data.

Setting bh state to new when get a new block, if so, direct_io_worker()->dio_zero_block() will fill-in the unused portion of the block with zero.

Quoted 17 lines
---
 fs/ocfs2/aops.c | 1 +
 1 file changed, 1 insertion(+)
fs/ocfs2/aops.c +1 -0
index 1a35c61..bd106b9 100644
@@ -581,6 +581,7 @@ static int ocfs2_direct_IO_get_blocks(struct inode *inode, sector_t iblock,
 			ret = -EIO;
 			goto bail;
 		}
+		set_buffer_new(bh_result);
 	}
 
 	/*
#2 lore raw

PATCH 1/1 ocfs2: Fill-in the unused portion of the block with zeros by dio_zero_block()

In reply to jiangyiwen

Hi Yiwen,

I try to reproduce this case, but it didn't act like you describe.
What I did:
===
laptop:/mnt/shared # mount | grep ocfs2
ocfs2_dlmfs on /dlm type ocfs2_dlmfs (rw,relatime)
/dev/sda3 on /mnt/shared type ocfs2 (rw,relatime,_netdev,heartbeat=local,nointr,data=ordered,errors=remount-ro,atime_quantum=60,coherency=full,user_xattr,acl) laptop:/mnt/shared # dd if=/dev/zero of=/mnt/shared/hello bs=512 count=1 oflag=direct
1+0 records in
1+0 records out
512 bytes (512 B) copied, 0.000684382 s, 748 kB/s
laptop:/mnt/shared # truncate hello -s 2097152
laptop:/mnt/shared # cat hello ======> nothing
laptop:/mnt/shared # uname -r
3.16.7-21-desktop
===

Did I do something wrong? If I misunderstand, please correct me. Thanks.

Quoted 28 lines
Quoted 1 line
Quoted 1 line

A simplified test case is (this case from Ryan):
1) dd if=/dev/zero of=/mnt/hello bs=512 count=1 oflag=direct;
2) truncate /mnt/hello -s 2097152
file 'hello' is not exist before test. After this command, file 'hello' should be all zero. But 512~4096 is some random data.

Setting bh state to new when get a new block, if so, direct_io_worker()->dio_zero_block() will fill-in the unused portion of the block with zero.

---

 fs/ocfs2/aops.c | 1 + 
 1 file changed, 1 insertion(+) 
fs/ocfs2/aops.c +1 -0
index 1a35c61..bd106b9 100644 
@@ -581,6 +581,7 @@ static int ocfs2_direct_IO_get_blocks(struct inode  
 *inode, sector_t iblock, 
 			ret = -EIO; 
 			goto bail; 
 		} 
+		set_buffer_new(bh_result); 
 	} 
 
Quoted 1 line

/*

--
Eric, Ren

#3 lore raw

PATCH 1/1 ocfs2: Fill-in the unused portion of the block with zeros by dio_zero_block()

Hi Zhen,

Quoted 14 lines · Zhen Ren

Hi Yiwen,

I try to reproduce this case, but it didn't act like you describe.
What I did:
===
laptop:/mnt/shared # mount | grep ocfs2
ocfs2_dlmfs on /dlm type ocfs2_dlmfs (rw,relatime)
/dev/sda3 on /mnt/shared type ocfs2 (rw,relatime,_netdev,heartbeat=local,nointr,data=ordered,errors=remount-ro,atime_quantum=60,coherency=full,user_xattr,acl) laptop:/mnt/shared # dd if=/dev/zero of=/mnt/shared/hello bs=512 count=1 oflag=direct
1+0 records in
1+0 records out
512 bytes (512 B) copied, 0.000684382 s, 748 kB/s
laptop:/mnt/shared # truncate hello -s 2097152
laptop:/mnt/shared # cat hello ======> nothing

I don't think 'cat' works here.
Could you please use hexdump?

Quoted 52 lines

laptop:/mnt/shared # uname -r
3.16.7-21-desktop
===

Did I do something wrong? If I misunderstand, please correct me. Thanks.

Quoted 28 lines
Quoted 1 line
Quoted 1 line

A simplified test case is (this case from Ryan):
1) dd if=/dev/zero of=/mnt/hello bs=512 count=1 oflag=direct;
2) truncate /mnt/hello -s 2097152
file 'hello' is not exist before test. After this command, file 'hello' should be all zero. But 512~4096 is some random data.

Setting bh state to new when get a new block, if so, direct_io_worker()->dio_zero_block() will fill-in the unused portion of the block with zero.

---

 fs/ocfs2/aops.c | 1 + 
 1 file changed, 1 insertion(+) 
fs/ocfs2/aops.c +1 -0
index 1a35c61..bd106b9 100644 
@@ -581,6 +581,7 @@ static int ocfs2_direct_IO_get_blocks(struct inode  
 *inode, sector_t iblock, 
 			ret = -EIO; 
 			goto bail; 
 		} 
+		set_buffer_new(bh_result); 
 	} 
 
Quoted 1 line

/*

--
Eric, Ren

_______________________________________________
Ocfs2-devel mailing list
Ocfs2-devel at oss.oracle.com
https://oss.oracle.com/mailman/listinfo/ocfs2-devel

#4 lore raw

PATCH 1/1 ocfs2: Fill-in the unused portion of the block with zeros by dio_zero_block()

Hi Joseph and Yiwen,

Sorry for that misoperation.

This time, I did the same cmds using "hexdump" on ocfs2 and btrfs. The testing results look the same:
==
laptop:/mnt/shared # dd if=/dev/zero of=hello bs=512 count=1 oflag=direct ===> on ocfs2
1+0 records in
1+0 records out
512 bytes (512 B) copied, 0.000834866 s, 613 kB/s laptop:/mnt/shared # hexdump hello -C
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| *
00000200
laptop:/mnt/shared # truncate hello -s 2097152
laptop:/mnt/shared # truncate hello -s 2097152
laptop:/mnt/shared # hexdump hello -C
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| *
00200000

laptop:~ # dd if=/dev/zero of=hello bs=512 count=1 oflag=direct ===> on btrfs
1+0 records in
1+0 records out
512 bytes (512 B) copied, 0.000201759 s, 2.5 MB/s
laptop:~ # hexdump hello -C
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| *
00000200
laptop:~ # truncate hello -s 2097152
laptop:~ # hexdump hello -C
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| *
00200000
==

BTW, Yiwen, I'm confused by your comment and commit message.
1. >> file 'hello' should be all zero. But 512~4096 is some random data.
2. it means in file area 512~4096 is zero not random data.

Anyway, I didn't look into this case, but just try to understand it. So, if there's any problem, let it go ;-)

Thanks.
--
Eric Ren

Quoted 1 line
Quoted 1 line
Quoted 1 line

Joseph Qi <[email protected]> 09/09/15 6:04 PM >>>

Hi Zhen,

Quoted 14 lines · Zhen Ren

Hi Yiwen,

I try to reproduce this case, but it didn't act like you describe.
What I did:
===
laptop:/mnt/shared # mount | grep ocfs2
ocfs2_dlmfs on /dlm type ocfs2_dlmfs (rw,relatime)
/dev/sda3 on /mnt/shared type ocfs2 (rw,relatime,_netdev,heartbeat=local,nointr,data=ordered,errors=remount-ro,atime_quantum=60,coherency=full,user_xattr,acl) laptop:/mnt/shared # dd if=/dev/zero of=/mnt/shared/hello bs=512 count=1 oflag=direct
1+0 records in
1+0 records out
512 bytes (512 B) copied, 0.000684382 s, 748 kB/s
laptop:/mnt/shared # truncate hello -s 2097152
laptop:/mnt/shared # cat hello ======> nothing

I don't think 'cat' works here.
Could you please use hexdump?

Quoted 52 lines

laptop:/mnt/shared # uname -r
3.16.7-21-desktop
===

Did I do something wrong? If I misunderstand, please correct me. Thanks.

Quoted 28 lines
Quoted 1 line
Quoted 1 line

A simplified test case is (this case from Ryan):
1) dd if=/dev/zero of=/mnt/hello bs=512 count=1 oflag=direct;
2) truncate /mnt/hello -s 2097152
file 'hello' is not exist before test. After this command, file 'hello' should be all zero. But 512~4096 is some random data.

Setting bh state to new when get a new block, if so, direct_io_worker()->dio_zero_block() will fill-in the unused portion of the block with zero.

---

 fs/ocfs2/aops.c | 1 + 
 1 file changed, 1 insertion(+) 
fs/ocfs2/aops.c +1 -0
index 1a35c61..bd106b9 100644 
@@ -581,6 +581,7 @@ static int ocfs2_direct_IO_get_blocks(struct inode  
 *inode, sector_t iblock, 
 			ret = -EIO; 
 			goto bail; 
 		} 
+		set_buffer_new(bh_result); 
 	} 
 
Quoted 1 line

/*

--
Eric, Ren

_______________________________________________
Ocfs2-devel mailing list
Ocfs2-devel at oss.oracle.com
https://oss.oracle.com/mailman/listinfo/ocfs2-devel

author
#5 lore raw

PATCH 1/1 ocfs2: Fill-in the unused portion of the block with zeros by dio_zero_block()

Quoted 37 lines · Zhen Ren

Hi Joseph and Yiwen,

Sorry for that misoperation.

This time, I did the same cmds using "hexdump" on ocfs2 and btrfs. The testing results look the same:
==
laptop:/mnt/shared # dd if=/dev/zero of=hello bs=512 count=1 oflag=direct ===> on ocfs2
1+0 records in
1+0 records out
512 bytes (512 B) copied, 0.000834866 s, 613 kB/s laptop:/mnt/shared # hexdump hello -C
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| *
00000200
laptop:/mnt/shared # truncate hello -s 2097152
laptop:/mnt/shared # truncate hello -s 2097152
laptop:/mnt/shared # hexdump hello -C
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| *
00200000

laptop:~ # dd if=/dev/zero of=hello bs=512 count=1 oflag=direct ===> on btrfs
1+0 records in
1+0 records out
512 bytes (512 B) copied, 0.000201759 s, 2.5 MB/s
laptop:~ # hexdump hello -C
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| *
00000200
laptop:~ # truncate hello -s 2097152
laptop:~ # hexdump hello -C
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| *
00200000
==

Quoted 6 lines

BTW, Yiwen, I'm confused by your comment and commit message.
1. >> file 'hello' should be all zero. But 512~4096 is some random data.
2. it means in file area 512~4096 is zero not random data.

Anyway, I didn't look into this case, but just try to understand it. So, if there's any problem, let it go ;-)

sorry, my description is wrong, 512~4096 is zero.

Thanks,
Yiwen Jiang

Quoted 85 lines

Thanks.
--
Eric Ren

Quoted 1 line
Quoted 1 line
Quoted 1 line

Joseph Qi <[email protected]> 09/09/15 6:04 PM >>>

Hi Zhen,

Quoted 14 lines · Zhen Ren

Hi Yiwen,

I try to reproduce this case, but it didn't act like you describe.
What I did:
===
laptop:/mnt/shared # mount | grep ocfs2
ocfs2_dlmfs on /dlm type ocfs2_dlmfs (rw,relatime)
/dev/sda3 on /mnt/shared type ocfs2 (rw,relatime,_netdev,heartbeat=local,nointr,data=ordered,errors=remount-ro,atime_quantum=60,coherency=full,user_xattr,acl) laptop:/mnt/shared # dd if=/dev/zero of=/mnt/shared/hello bs=512 count=1 oflag=direct
1+0 records in
1+0 records out
512 bytes (512 B) copied, 0.000684382 s, 748 kB/s
laptop:/mnt/shared # truncate hello -s 2097152
laptop:/mnt/shared # cat hello ======> nothing

I don't think 'cat' works here.
Could you please use hexdump?

Quoted 46 lines

laptop:/mnt/shared # uname -r
3.16.7-21-desktop
===

Did I do something wrong? If I misunderstand, please correct me. Thanks.

Quoted 28 lines
Quoted 1 line
Quoted 1 line

A simplified test case is (this case from Ryan):
1) dd if=/dev/zero of=/mnt/hello bs=512 count=1 oflag=direct;
2) truncate /mnt/hello -s 2097152
file 'hello' is not exist before test. After this command, file 'hello' should be all zero. But 512~4096 is some random data.

Setting bh state to new when get a new block, if so, direct_io_worker()->dio_zero_block() will fill-in the unused portion of the block with zero.

---

 fs/ocfs2/aops.c | 1 + 
 1 file changed, 1 insertion(+) 
fs/ocfs2/aops.c +1 -0
index 1a35c61..bd106b9 100644 
@@ -581,6 +581,7 @@ static int ocfs2_direct_IO_get_blocks(struct inode  
 *inode, sector_t iblock, 
 			ret = -EIO; 
 			goto bail; 
 		} 
+		set_buffer_new(bh_result); 
 	} 
 
Quoted 1 line

/*

--
Eric, Ren

Quoted 6 lines

_______________________________________________
Ocfs2-devel mailing list
Ocfs2-devel at oss.oracle.com
https://oss.oracle.com/mailman/listinfo/ocfs2-devel

.

author
#6 lore raw

PATCH 1/1 ocfs2: Fill-in the unused portion of the block with zeros by dio_zero_block()

In reply to Zhen Ren
Quoted 14 lines · Zhen Ren

Hi Yiwen,

I try to reproduce this case, but it didn't act like you describe.
What I did:
===
laptop:/mnt/shared # mount | grep ocfs2
ocfs2_dlmfs on /dlm type ocfs2_dlmfs (rw,relatime)
/dev/sda3 on /mnt/shared type ocfs2 (rw,relatime,_netdev,heartbeat=local,nointr,data=ordered,errors=remount-ro,atime_quantum=60,coherency=full,user_xattr,acl) laptop:/mnt/shared # dd if=/dev/zero of=/mnt/shared/hello bs=512 count=1 oflag=direct
1+0 records in
1+0 records out
512 bytes (512 B) copied, 0.000684382 s, 748 kB/s
laptop:/mnt/shared # truncate hello -s 2097152
laptop:/mnt/shared # cat hello ======> nothing

Please use this command to read binary file "hexdump -C /mnt/shared/hello", you will see the result like this:
linux-ZnXfWq:/mnt/ocfs2 # hexdump -C hello
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| *
00200000
it means in file area 512~4096 is zero not random data.

Quoted 48 lines

laptop:/mnt/shared # uname -r
3.16.7-21-desktop
===

Did I do something wrong? If I misunderstand, please correct me. Thanks.

Quoted 28 lines
Quoted 1 line
Quoted 1 line

A simplified test case is (this case from Ryan):
1) dd if=/dev/zero of=/mnt/hello bs=512 count=1 oflag=direct;
2) truncate /mnt/hello -s 2097152
file 'hello' is not exist before test. After this command, file 'hello' should be all zero. But 512~4096 is some random data.

Setting bh state to new when get a new block, if so, direct_io_worker()->dio_zero_block() will fill-in the unused portion of the block with zero.

---

 fs/ocfs2/aops.c | 1 + 
 1 file changed, 1 insertion(+) 
fs/ocfs2/aops.c +1 -0
index 1a35c61..bd106b9 100644 
@@ -581,6 +581,7 @@ static int ocfs2_direct_IO_get_blocks(struct inode  
 *inode, sector_t iblock, 
 			ret = -EIO; 
 			goto bail; 
 		} 
+		set_buffer_new(bh_result); 
 	} 
 
Quoted 1 line

/*

--
Eric, Ren

.

#7 lore raw

PATCH 1/1 ocfs2: Fill-in the unused portion of the block with zeros by dio_zero_block()

In reply to jiangyiwen

Hi Yiwen,

I'm working on this issue. The patch will be send out soon. And the issue that do not support file hole will be fixed too. I have proved it can pass all ltp-aiodio test cases, and has better performance. ;-)

Thanks,
Ryan

Quoted 27 lines · jiangyiwen

A simplified test case is (this case from Ryan):
1) dd if=/dev/zero of=/mnt/hello bs=512 count=1 oflag=direct;
2) truncate /mnt/hello -s 2097152
file 'hello' is not exist before test. After this command, file 'hello' should be all zero. But 512~4096 is some random data.

Setting bh state to new when get a new block, if so, direct_io_worker()->dio_zero_block() will fill-in the unused portion of the block with zero.

---
  fs/ocfs2/aops.c | 1 +
  1 file changed, 1 insertion(+)
fs/ocfs2/aops.c +1 -0
index 1a35c61..bd106b9 100644
@@ -581,6 +581,7 @@ static int ocfs2_direct_IO_get_blocks(struct inode *inode, sector_t iblock,
  			ret = -EIO;
  			goto bail;
  		}
+		set_buffer_new(bh_result);
  	}
 
  	/*
#8 lore raw

PATCH 1/1 ocfs2: Fill-in the unused portion of the block with zeros by dio_zero_block()

Quoted 5 lines · On Thu, 10 Sep 2015 09:53:11 +0800 ryding <[email protected]> wrote:

Hi Yiwen,

I'm working on this issue. The patch will be send out soon. And the issue that do not support file hole will be fixed too. I have proved it can pass all ltp-aiodio test cases, and has better performance. ;-)

Yes, your "ocfs2: fill in the unused portion of the block with zeros by dio_zero_block()" conflicts with this patch a little.

Does your patchset need alteration to fix this bug?

#9 lore raw

PATCH 1/1 ocfs2: Fill-in the unused portion of the block with zeros by dio_zero_block()

Quoted 11 lines · Andrew Morton
Quoted 5 lines · On Thu, 10 Sep 2015 09:53:11 +0800 ryding <[email protected]> wrote:

Hi Yiwen,

I'm working on this issue. The patch will be send out soon. And the issue that do not support file hole will be fixed too. I have proved it can pass all ltp-aiodio test cases, and has better performance. ;-)

Yes, your "ocfs2: fill in the unused portion of the block with zeros by dio_zero_block()" conflicts with this patch a little.

Does your patchset need alteration to fix this bug?

My patchset "ocfs2: fix ocfs2 direct io code patch to support sparse file and
data ordering semantics" has already include the fix to this bug. Yiwen's patch
is no longer needed if my patchset is accepted.

#10 lore raw

PATCH 1/1 ocfs2: Fill-in the unused portion of the block with zeros by dio_zero_block()

In reply to jiangyiwen
Quoted 15 lines · On Wed, 9 Sep 2015 09:55:16 +0800 jiangyiwen <[email protected]> wrote:

A simplified test case is (this case from Ryan):
1) dd if=/dev/zero of=/mnt/hello bs=512 count=1 oflag=direct;
2) truncate /mnt/hello -s 2097152
file 'hello' is not exist before test. After this command, file 'hello' should be all zero. But 512~4096 is some random data.

Setting bh state to new when get a new block, if so, direct_io_worker()->dio_zero_block() will fill-in the unused portion of the block with zero.

...

fs/ocfs2/aops.c static int ocfs2_direct_IO_get_blocks(struct inode *inode, sector_t iblock, · line 586

Quoted 6 lines
fs/ocfs2/aops.c +1 -0
@@ -581,6 +581,7 @@ static int ocfs2_direct_IO_get_blocks(struct inode *inode, sector_t iblock,
 			ret = -EIO;
 			goto bail;
 		}
+		set_buffer_new(bh_result);
 	}
 

You're working against an old kernel. I did this:

fs/ocfs2/aops.c renamed from fs/ocfs2/aops.c~ocfs2-fill-in-the-unused-portion-of-the-block-with-zeros-by-dio_zero_block +1 -0
@@ -589,6 +589,7 @@ static int ocfs2_direct_IO_get_blocks(st
 			ret = -EIO;
 			goto bail;
 		}
+		set_buffer_new(bh_result);
 		up_write(&OCFS2_I(inode)->ip_alloc_sem);
 	}
 

Probably we could run set_buffer_new() after the up_write(), which would decrease lok hold times a little bit.