Skip to contentSkip to conversation

f2fs: support finding extents after isize

PATCH In review Fan Li 4 participants 18 messages Dec 30, 2015 – Jan 5, 2016 F2FS file system

lore
Share & tools
lore

Fan Li

author
#0 lore raw

PATCH 2/3 f2fs: support finding extents after isize

f2fs allows preallocation beyond isize, but f2fs_fiemap only look up extents within isize. Therefore add this support.

Note: It's possible that there are holes after isize, for example, fallocate multiple discontinuous extents after isize with FALLOC_FL_KEEP_SIZE set. Since I can tell no differences between EOF and holes from return of get_data_block, I'm afaid this patch can't support such scenarios.

---
 fs/f2fs/data.c |    7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
fs/f2fs/data.c +1 -6
index a9a4d89..f89cf07 100644
@@ -798,12 +798,6 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 	isize = i_size_read(inode);
 
 	mutex_lock(&inode->i_mutex);
-	if (start >= isize)
-		goto out;
-
-	if (start + len > isize)
-		len = isize - start;
-
 	if (logical_to_blk(inode, len) == 0)
 		len = blk_to_logical(inode, 1);
 
@@ -829,6 +823,7 @@ next:
 		 * punch holes beyond isize and keep size unchanged.
 		 */
 		flags |= FIEMAP_EXTENT_LAST;
+		last_blk = start_blk - 1;
 	}
 
 	if (size)
Signature · 4 lines
1.7.9.5


------------------------------------------------------------------------------
maintainer
#1 lore raw

Hi,

Quoted 8 lines · Fan Li

f2fs allows preallocation beyond isize, but f2fs_fiemap only look up extents within isize. Therefore add this support.

Note: It's possible that there are holes after isize, for example, fallocate multiple discontinuous extents after isize with FALLOC_FL_KEEP_SIZE set. Since I can tell no differences between EOF and holes from return of get_data_block, I'm afaid this patch can't support such scenarios.

As you mentioned, preallocated block beyond isize can be allocated in f2fs, and we are trying to support mapping extents across whole data space of inode, so why we treat theses extents inside i_size and outside i_size separately? IMO, instead using i_size, we should use max blocks as boundary.

Most important, this interface still can't support finding all extents after i_size, which looks buggy for our user.

Thanks,

Quoted 26 lines
---
 fs/f2fs/data.c |    7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
fs/f2fs/data.c +0 -6
index a9a4d89..f89cf07 100644
@@ -798,12 +798,6 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 	isize = i_size_read(inode);
 
 	mutex_lock(&inode->i_mutex);
-	if (start >= isize)
-		goto out;
-
-	if (start + len > isize)
-		len = isize - start;
-
 	if (logical_to_blk(inode, len) == 0)
 		len = blk_to_logical(inode, 1);
 

fs/f2fs/data.c next: · line 829

Quoted 7 lines
fs/f2fs/data.c +1 -0
@@ -829,6 +823,7 @@ next:
 		 * punch holes beyond isize and keep size unchanged.
 		 */
 		flags |= FIEMAP_EXTENT_LAST;
+		last_blk = start_blk - 1;
 	}
 
 	if (size)

------------------------------------------------------------------------------

author
#2 lore raw
Quoted 15 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Wednesday, December 30, 2015 9:28 PM
To: Fan Li; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Hi,

Quoted 3 lines · Fan Li

f2fs allows preallocation beyond isize, but f2fs_fiemap only look up extents within isize. Therefore add this support.

Quoted 8 lines
Quoted 5 lines

Note: It's possible that there are holes after isize, for example, fallocate multiple discontinuous extents after isize with FALLOC_FL_KEEP_SIZE set. Since I can tell no differences between EOF and holes from return of get_data_block, I'm afaid this patch can't support such scenarios.

As you mentioned, preallocated block beyond isize can be allocated in f2fs, and we are trying to support mapping extents across whole data space of inode, so why we treat theses extents inside i_size and outside i_size separately? IMO, instead using i_size,

we

Quoted 3 lines

should use max blocks as boundary.

Most important, this interface still can't support finding all extents after i_size, which looks buggy for our user.

Notice that this issue exists before my patch, by adding this patch, at least now it can support more scenarios such as fallocate a range right after isize. I'd say it's an improvement.

use max blocks as boundary would get us the precise result, but it also means after we reach the EOF, we still need to look up every block between the EOF and sb-> s_maxbytes to make sure the EOF is true, that's about 4TB or 10^9 blocks.
And it affects all scenarios where the search range covers the last extent in the file, not just extents beyond isize. I think this price is too high to pay.

I was hoping that I can make f2fs_map_block return an EOF to solve this problem some time later, or anyone have a better idea?

Quoted 34 lines

Thanks,

Quoted 31 lines
---
 fs/f2fs/data.c |    7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
fs/f2fs/data.c index a9a4d89..f89cf07 renamed from fs/f2fs/data.c +0 -0

100644

fs/f2fs/data.c +1 -6
@@ -798,12 +798,6 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 	isize = i_size_read(inode);
 
 	mutex_lock(&inode->i_mutex);
-	if (start >= isize)
-		goto out;
-
-	if (start + len > isize)
-		len = isize - start;
-
 	if (logical_to_blk(inode, len) == 0)
 		len = blk_to_logical(inode, 1);
 
@@ -829,6 +823,7 @@ next:
 		 * punch holes beyond isize and keep size unchanged.
 		 */
 		flags |= FIEMAP_EXTENT_LAST;
+		last_blk = start_blk - 1;
 	}
 
Quoted 2 lines

fs/f2fs/data.c next: · line 829

Quoted 2 lines
fs/f2fs/data.c +0 -0
 	if (size)

------------------------------------------------------------------------------

#3 lore raw
Quoted 33 lines

-----Original Message-----
From: Fan Li [mailto:[email protected]]
Sent: Thursday, December 31, 2015 11:37 AM
To: 'Chao Yu'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 13 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Wednesday, December 30, 2015 9:28 PM
To: Fan Li; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Hi,

Quoted 3 lines · Fan Li

f2fs allows preallocation beyond isize, but f2fs_fiemap only look up extents within isize. Therefore add this support.

Quoted 7 lines
Quoted 5 lines

Note: It's possible that there are holes after isize, for example, fallocate multiple discontinuous extents after isize with FALLOC_FL_KEEP_SIZE set. Since I can tell no differences between EOF and holes from return of get_data_block, I'm afaid this patch can't support such scenarios.

As you mentioned, preallocated block beyond isize can be allocated in f2fs, and we are trying

to support mapping extents across

Quoted 1 line

whole data space of inode, so why we treat theses extents inside i_size and outside i_size

separately? IMO, instead using i_size,
we

Quoted 8 lines
Quoted 3 lines

should use max blocks as boundary.

Most important, this interface still can't support finding all extents after i_size, which

looks buggy for our user.

Notice that this issue exists before my patch, by adding this patch, at least now it can support more scenarios
such as fallocate a range right after isize. I'd say it's an improvement.

Nope, what I'm talking about is *correctness* of our ->fiemap interface, but you're trying to avoid it by saying "support more cases, it's an improvement". That doesn't make any sense to me, since correctness issue still not be fixed.

Quoted 9 lines

use max blocks as boundary would get us the precise result, but it also means after we reach the EOF, we still
need to look up every block between the EOF and sb-> s_maxbytes to make sure the EOF is true, that's
about 4TB or 10^9 blocks.
And it affects all scenarios where the search range covers the last extent in the file, not just extents beyond
isize. I think this price is too high to pay.

That's another performance issue.

Quoted 4 lines

I was hoping that I can make f2fs_map_block return an EOF to solve this problem some time later, or anyone
have a better idea?

At least we can seek valid dnode block like the way llseek use.

In addition, for most cases, few of i_nid[5] in f2fs_inode will be NULL, we could skip searching all dnode block in such non-allocated indirect node, instead of searching dnode block f2fs_map_block one by one.

Thanks,

Quoted 35 lines
Quoted 35 lines

Thanks,

Quoted 31 lines
---
 fs/f2fs/data.c |    7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
fs/f2fs/data.c index a9a4d89..f89cf07 renamed from fs/f2fs/data.c +0 -0

100644

fs/f2fs/data.c +1 -6
@@ -798,12 +798,6 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 	isize = i_size_read(inode);
 
 	mutex_lock(&inode->i_mutex);
-	if (start >= isize)
-		goto out;
-
-	if (start + len > isize)
-		len = isize - start;
-
 	if (logical_to_blk(inode, len) == 0)
 		len = blk_to_logical(inode, 1);
 
@@ -829,6 +823,7 @@ next:
 		 * punch holes beyond isize and keep size unchanged.
 		 */
 		flags |= FIEMAP_EXTENT_LAST;
+		last_blk = start_blk - 1;
 	}
 
Quoted 9 lines
Quoted 2 lines

fs/f2fs/data.c next: · line 829

Quoted 2 lines
fs/f2fs/data.c +0 -0
 	if (size)

------------------------------------------------------------------------------
_______________________________________________
Linux-f2fs-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

------------------------------------------------------------------------------

author
#4 lore raw
Quoted 49 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Thursday, December 31, 2015 2:34 PM
To: 'Fan Li'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: RE: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 40 lines

-----Original Message-----
From: Fan Li [mailto:[email protected]]
Sent: Thursday, December 31, 2015 11:37 AM
To: 'Chao Yu'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 14 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Wednesday, December 30, 2015 9:28 PM
To: Fan Li; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Hi,

Quoted 3 lines · Fan Li

f2fs allows preallocation beyond isize, but f2fs_fiemap only look up extents within isize. Therefore add this support.

Quoted 8 lines
Quoted 5 lines

Note: It's possible that there are holes after isize, for example, fallocate multiple discontinuous extents after isize with FALLOC_FL_KEEP_SIZE set. Since I can tell no differences between EOF and holes from return of get_data_block, I'm afaid this patch can't support such scenarios.

As you mentioned, preallocated block beyond isize can be allocated in f2fs, and we are trying

to support mapping extents across

Quoted 2 lines

whole data space of inode, so why we treat theses extents inside i_size and outside i_size

separately? IMO, instead using i_size, we

Quoted 4 lines

should use max blocks as boundary.

Most important, this interface still can't support finding all extents after i_size, which

Quoted 7 lines
Quoted 5 lines

looks buggy for our user.

Notice that this issue exists before my patch, by adding this patch, at least now it can support more scenarios such as fallocate a range right after isize. I'd say it's an improvement.

Nope, what I'm talking about is *correctness* of our ->fiemap interface, but you're trying to avoid it by saying "support more

cases,

Quoted 1 line

it's an improvement". That doesn't make any sense to me, since correctness issue still not be fixed.

I'm not sure what you mean by avoiding, I think the comment and reply I written has already stated the issue and limitation of this patch.
Now there are two suggestions:
1. support one more scenario, and all old scenarios are dealt like before, but it still can't support discontinuous extent after isize.
2. support all scenarios, but sacrifice performance for lots of common scenarios by checking about 10^9 blocks.

I think we can all agree both ideas have their flaws.
The only divergence is that I vote the first, and you the second. I think the most important thing is that it works fluently in most scenarios, and you think is that it works in every scenarios even it's very slow.

I think my method is more pratical, but balance between performance and utility seems to be an Eternal problem.

Quoted 13 lines
Quoted 9 lines

use max blocks as boundary would get us the precise result, but it also means after we reach the EOF, we still need to look up every block between the EOF and sb-> s_maxbytes to make sure the EOF is true, that's about 4TB or 10^9 blocks.
And it affects all scenarios where the search range covers the last extent in the file, not just extents beyond isize. I think this price is too high to pay.

That's another performance issue.

Quoted 1 line
Quoted 6 lines
Quoted 2 lines

I was hoping that I can make f2fs_map_block return an EOF to solve this problem some time later, or anyone have a better idea?

At least we can seek valid dnode block like the way llseek use.

In addition, for most cases, few of i_nid[5] in f2fs_inode will be NULL, we could skip searching all dnode block in such

non-allocated

Quoted 49 lines

indirect node, instead of searching dnode block f2fs_map_block one by one.

Thanks,

Quoted 45 lines
Quoted 36 lines

Thanks,

Quoted 32 lines
---
 fs/f2fs/data.c |    7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
fs/f2fs/data.c index renamed from fs/f2fs/data.c +0 -0

a9a4d89..f89cf07
100644

fs/f2fs/data.c +1 -6
@@ -798,12 +798,6 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 	isize = i_size_read(inode);
 
 	mutex_lock(&inode->i_mutex);
-	if (start >= isize)
-		goto out;
-
-	if (start + len > isize)
-		len = isize - start;
-
 	if (logical_to_blk(inode, len) == 0)
 		len = blk_to_logical(inode, 1);
 
@@ -829,6 +823,7 @@ next:
 		 * punch holes beyond isize and keep size unchanged.
 		 */
 		flags |= FIEMAP_EXTENT_LAST;
+		last_blk = start_blk - 1;
 	}
 
Quoted 2 lines

fs/f2fs/data.c next: · line 829

Quoted 2 lines
fs/f2fs/data.c +0 -0
 	if (size)

----------------------------------------------------------------------
-------- _______________________________________________ Linux-f2fs-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

------------------------------------------------------------------------------

maintainer
#5 lore raw

Hello,

Quoted 57 lines · Fan Li
Quoted 49 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Thursday, December 31, 2015 2:34 PM
To: 'Fan Li'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: RE: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 40 lines

-----Original Message-----
From: Fan Li [mailto:[email protected]]
Sent: Thursday, December 31, 2015 11:37 AM
To: 'Chao Yu'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 14 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Wednesday, December 30, 2015 9:28 PM
To: Fan Li; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Hi,

Quoted 3 lines · Fan Li

f2fs allows preallocation beyond isize, but f2fs_fiemap only look up extents within isize. Therefore add this support.

Quoted 8 lines
Quoted 5 lines

Note: It's possible that there are holes after isize, for example, fallocate multiple discontinuous extents after isize with FALLOC_FL_KEEP_SIZE set. Since I can tell no differences between EOF and holes from return of get_data_block, I'm afaid this patch can't support such scenarios.

As you mentioned, preallocated block beyond isize can be allocated in f2fs, and we are trying

to support mapping extents across

Quoted 2 lines

whole data space of inode, so why we treat theses extents inside i_size and outside i_size

separately? IMO, instead using i_size, we

Quoted 4 lines

should use max blocks as boundary.

Most important, this interface still can't support finding all extents after i_size, which

Quoted 7 lines
Quoted 5 lines

looks buggy for our user.

Notice that this issue exists before my patch, by adding this patch, at least now it can support more scenarios such as fallocate a range right after isize. I'd say it's an improvement.

Nope, what I'm talking about is *correctness* of our ->fiemap interface, but you're trying to avoid it by saying "support more

cases,

Quoted 8 lines
Quoted 1 line

it's an improvement". That doesn't make any sense to me, since correctness issue still not be fixed.

I'm not sure what you mean by avoiding, I think the comment and reply I written has already stated the issue and limitation of this patch.
Now there are two suggestions:
1. support one more scenario, and all old scenarios are dealt like before, but it still can't support discontinuous extent after isize.
2. support all scenarios, but sacrifice performance for lots of common scenarios by checking about 10^9 blocks.

IMO, we can think about #2 whether there is an efficient way.

How many cases does this incur?
One is fallocate with keeping i_size, ana other?

How about adding FADVISE_OVER_ISIZE to represent inode has blocks beyond i_size? Then, we can set this flag in fallocate and reset it in f2fs_truncate.

Thanks,

Quoted 20 lines

I think we can all agree both ideas have their flaws.
The only divergence is that I vote the first, and you the second. I think the most important thing is that it works fluently in most scenarios, and you think is that it works in every scenarios even it's very slow.

I think my method is more pratical, but balance between performance and utility seems to be an Eternal problem.

Quoted 13 lines
Quoted 9 lines

use max blocks as boundary would get us the precise result, but it also means after we reach the EOF, we still need to look up every block between the EOF and sb-> s_maxbytes to make sure the EOF is true, that's about 4TB or 10^9 blocks.
And it affects all scenarios where the search range covers the last extent in the file, not just extents beyond isize. I think this price is too high to pay.

That's another performance issue.

Quoted 1 line
Quoted 56 lines
Quoted 6 lines
Quoted 2 lines

I was hoping that I can make f2fs_map_block return an EOF to solve this problem some time later, or anyone have a better idea?

At least we can seek valid dnode block like the way llseek use.

In addition, for most cases, few of i_nid[5] in f2fs_inode will be NULL, we could skip searching all dnode block in such

non-allocated

Quoted 49 lines

indirect node, instead of searching dnode block f2fs_map_block one by one.

Thanks,

Quoted 45 lines
Quoted 36 lines

Thanks,

Quoted 32 lines
---
 fs/f2fs/data.c |    7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
fs/f2fs/data.c index renamed from fs/f2fs/data.c +0 -0

a9a4d89..f89cf07
100644

fs/f2fs/data.c +1 -6
@@ -798,12 +798,6 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 	isize = i_size_read(inode);
 
 	mutex_lock(&inode->i_mutex);
-	if (start >= isize)
-		goto out;
-
-	if (start + len > isize)
-		len = isize - start;
-
 	if (logical_to_blk(inode, len) == 0)
 		len = blk_to_logical(inode, 1);
 
@@ -829,6 +823,7 @@ next:
 		 * punch holes beyond isize and keep size unchanged.
 		 */
 		flags |= FIEMAP_EXTENT_LAST;
+		last_blk = start_blk - 1;
 	}
 
Quoted 2 lines

fs/f2fs/data.c next: · line 829

Quoted 2 lines
fs/f2fs/data.c +0 -0
 	if (size)

----------------------------------------------------------------------
-------- _______________________________________________ Linux-f2fs-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

------------------------------------------------------------------------------

author
#6 lore raw
Quoted 80 lines

-----Original Message-----
From: Jaegeuk Kim [mailto:[email protected]]
Sent: Monday, January 04, 2016 2:56 PM
To: Fan Li
Cc: 'Chao Yu'; [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Hello,

Quoted 59 lines · Fan Li
Quoted 50 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Thursday, December 31, 2015 2:34 PM
To: 'Fan Li'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: RE: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 40 lines

-----Original Message-----
From: Fan Li [mailto:[email protected]]
Sent: Thursday, December 31, 2015 11:37 AM
To: 'Chao Yu'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 14 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Wednesday, December 30, 2015 9:28 PM
To: Fan Li; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Hi,

Quoted 3 lines · Fan Li

f2fs allows preallocation beyond isize, but f2fs_fiemap only look up extents within isize. Therefore add this support.

Quoted 8 lines
Quoted 5 lines

Note: It's possible that there are holes after isize, for example, fallocate multiple discontinuous extents after isize with FALLOC_FL_KEEP_SIZE set. Since I can tell no differences between EOF and holes from return of get_data_block, I'm afaid this patch can't support such scenarios.

As you mentioned, preallocated block beyond isize can be allocated in f2fs, and we are trying

to support mapping extents across

Quoted 2 lines

whole data space of inode, so why we treat theses extents inside i_size and outside i_size

separately? IMO, instead using i_size, we

Quoted 4 lines

should use max blocks as boundary.

Most important, this interface still can't support finding all extents after i_size, which

Quoted 8 lines
Quoted 5 lines

looks buggy for our user.

Notice that this issue exists before my patch, by adding this patch, at least now it can support more scenarios such as fallocate a range right after isize. I'd say it's an improvement.

Nope, what I'm talking about is *correctness* of our ->fiemap interface, but you're trying to avoid it by saying "support more

cases,

Quoted 8 lines
Quoted 1 line

it's an improvement". That doesn't make any sense to me, since correctness issue still not be fixed.

I'm not sure what you mean by avoiding, I think the comment and reply
I written has already stated the issue and limitation of this patch.
Now there are two suggestions:
1. support one more scenario, and all old scenarios are dealt like
before, but it still can't support discontinuous extent after isize.
2. support all scenarios, but sacrifice performance for lots of common scenarios by checking about 10^9 blocks.

Quoted 7 lines

IMO, we can think about #2 whether there is an efficient way.

How many cases does this incur?
One is fallocate with keeping i_size, ana other?

How about adding FADVISE_OVER_ISIZE to represent inode has blocks beyond i_size? Then, we can set this flag in fallocate and reset it in f2fs_truncate.

I have a similar idea that add an actual size which marks the end of last extent, so we can know if the current extent is the last one, even without searching for extents behind.

But there is a problem I still can't figure out, after truncate an extent at the end of file beyond isize , how do I know where the new last extent ends or if there are still extents beyond isize? after all, the extents beyond isize could be discontinuous.

Quoted 7 lines

Thanks,

Quoted 4 lines

I think we can all agree both ideas have their flaws.
The only divergence is that I vote the first, and you the second. I
think the most important thing is that it works fluently in most scenarios, and you think is that it works in every scenarios

even it's

Quoted 75 lines

very slow.

Quoted 74 lines

I think my method is more pratical, but balance between performance and utility seems to be an Eternal problem.

Quoted 13 lines
Quoted 9 lines

use max blocks as boundary would get us the precise result, but it also means after we reach the EOF, we still need to look up every block between the EOF and sb-> s_maxbytes to make sure the EOF is true, that's about 4TB or 10^9 blocks.
And it affects all scenarios where the search range covers the last extent in the file, not just extents beyond isize. I think this price is too high to pay.

That's another performance issue.

Quoted 1 line
Quoted 7 lines
Quoted 2 lines

I was hoping that I can make f2fs_map_block return an EOF to solve this problem some time later, or anyone have a better idea?

At least we can seek valid dnode block like the way llseek use.

In addition, for most cases, few of i_nid[5] in f2fs_inode will be NULL, we could skip searching all dnode block in such

non-allocated

Quoted 50 lines

indirect node, instead of searching dnode block f2fs_map_block one by one.

Thanks,

Quoted 46 lines
Quoted 36 lines

Thanks,

Quoted 32 lines
---
 fs/f2fs/data.c |    7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
fs/f2fs/data.c index renamed from fs/f2fs/data.c +0 -0

a9a4d89..f89cf07
100644

fs/f2fs/data.c +1 -6
@@ -798,12 +798,6 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo,
 	isize = i_size_read(inode);
 
 	mutex_lock(&inode->i_mutex);
-	if (start >= isize)
-		goto out;
-
-	if (start + len > isize)
-		len = isize - start;
-
 	if (logical_to_blk(inode, len) == 0)
 		len = blk_to_logical(inode, 1);
 
@@ -829,6 +823,7 @@ next:
 		 * punch holes beyond isize and keep size unchanged.
 		 */
 		flags |= FIEMAP_EXTENT_LAST;
+		last_blk = start_blk - 1;
 	}
 
Quoted 2 lines

fs/f2fs/data.c next: · line 829

Quoted 2 lines
fs/f2fs/data.c +0 -0
 	if (size)

------------------------------------------------------------------
----
-------- _______________________________________________ Linux-f2fs-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

------------------------------------------------------------------------------

maintainer
#7 lore raw

Hi,

Quoted 82 lines · Fan Li
Quoted 82 lines

-----Original Message-----
From: Jaegeuk Kim [mailto:[email protected]]
Sent: Monday, January 04, 2016 2:56 PM
To: Fan Li
Cc: 'Chao Yu'; [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Hello,

Quoted 59 lines · Fan Li
Quoted 50 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Thursday, December 31, 2015 2:34 PM
To: 'Fan Li'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: RE: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 40 lines

-----Original Message-----
From: Fan Li [mailto:[email protected]]
Sent: Thursday, December 31, 2015 11:37 AM
To: 'Chao Yu'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 14 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Wednesday, December 30, 2015 9:28 PM
To: Fan Li; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Hi,

Quoted 3 lines · Fan Li

f2fs allows preallocation beyond isize, but f2fs_fiemap only look up extents within isize. Therefore add this support.

Quoted 8 lines
Quoted 5 lines

Note: It's possible that there are holes after isize, for example, fallocate multiple discontinuous extents after isize with FALLOC_FL_KEEP_SIZE set. Since I can tell no differences between EOF and holes from return of get_data_block, I'm afaid this patch can't support such scenarios.

As you mentioned, preallocated block beyond isize can be allocated in f2fs, and we are trying

to support mapping extents across

Quoted 2 lines

whole data space of inode, so why we treat theses extents inside i_size and outside i_size

separately? IMO, instead using i_size, we

Quoted 4 lines

should use max blocks as boundary.

Most important, this interface still can't support finding all extents after i_size, which

Quoted 8 lines
Quoted 5 lines

looks buggy for our user.

Notice that this issue exists before my patch, by adding this patch, at least now it can support more scenarios such as fallocate a range right after isize. I'd say it's an improvement.

Nope, what I'm talking about is *correctness* of our ->fiemap interface, but you're trying to avoid it by saying "support more

cases,

Quoted 8 lines
Quoted 1 line

it's an improvement". That doesn't make any sense to me, since correctness issue still not be fixed.

I'm not sure what you mean by avoiding, I think the comment and reply
I written has already stated the issue and limitation of this patch.
Now there are two suggestions:
1. support one more scenario, and all old scenarios are dealt like
before, but it still can't support discontinuous extent after isize.
2. support all scenarios, but sacrifice performance for lots of common scenarios by checking about 10^9 blocks.

IMO, we can think about #2 whether there is an efficient way.

Quoted 8 lines
Quoted 5 lines

How many cases does this incur?
One is fallocate with keeping i_size, ana other?

How about adding FADVISE_OVER_ISIZE to represent inode has blocks beyond i_size? Then, we can set this flag in fallocate and reset it in f2fs_truncate.

I have a similar idea that add an actual size which marks the end of last extent, so we can know if the current extent is the last one, even without searching for extents behind.

Where do you want to store that size in disk?

Quoted 3 lines

But there is a problem I still can't figure out, after truncate an extent at the end of file beyond isize , how do I know where the new last extent ends or if there are still extents beyond isize? after all, the extents beyond isize could be discontinuous.

So, that's why I proposed a flag instead of a kind of i_disksize. We can just set the flag, only if a file *may* have a extent beyond i_size in fallocate, and unset it through f2fs_truncate.
Moreover, I don't expect that this happens so frequently.

Thanks,

Quoted 4 lines
Quoted 4 lines

Thanks,

------------------------------------------------------------------------------

author
#8 lore raw
Quoted 98 lines

-----Original Message-----
From: Jaegeuk Kim [mailto:[email protected]]
Sent: Tuesday, January 05, 2016 7:14 AM
To: Fan Li
Cc: 'Chao Yu'; [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Hi,

Quoted 86 lines · Fan Li
Quoted 86 lines

-----Original Message-----
From: Jaegeuk Kim [mailto:[email protected]]
Sent: Monday, January 04, 2016 2:56 PM
To: Fan Li
Cc: 'Chao Yu'; [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Hello,

Quoted 59 lines · Fan Li
Quoted 50 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Thursday, December 31, 2015 2:34 PM
To: 'Fan Li'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: RE: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 40 lines

-----Original Message-----
From: Fan Li [mailto:[email protected]]
Sent: Thursday, December 31, 2015 11:37 AM
To: 'Chao Yu'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 14 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Wednesday, December 30, 2015 9:28 PM
To: Fan Li; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Hi,

Quoted 3 lines · Fan Li

f2fs allows preallocation beyond isize, but f2fs_fiemap only look up extents within isize. Therefore add this support.

Quoted 8 lines
Quoted 5 lines

Note: It's possible that there are holes after isize, for example, fallocate multiple discontinuous extents after isize with FALLOC_FL_KEEP_SIZE set. Since I can tell no differences between EOF and holes from return of get_data_block, I'm afaid this patch can't support such scenarios.

As you mentioned, preallocated block beyond isize can be allocated in f2fs, and we are trying

to support mapping extents across

Quoted 2 lines

whole data space of inode, so why we treat theses extents inside i_size and outside i_size

separately? IMO, instead using i_size, we

Quoted 4 lines

should use max blocks as boundary.

Most important, this interface still can't support finding all extents after i_size, which

Quoted 8 lines
Quoted 5 lines

looks buggy for our user.

Notice that this issue exists before my patch, by adding this patch, at least now it can support more scenarios such as fallocate a range right after isize. I'd say it's an improvement.

Nope, what I'm talking about is *correctness* of our ->fiemap interface, but you're trying to avoid it by saying "support more

cases,

Quoted 8 lines
Quoted 1 line

it's an improvement". That doesn't make any sense to me, since correctness issue still not be fixed.

I'm not sure what you mean by avoiding, I think the comment and
reply I written has already stated the issue and limitation of this patch.
Now there are two suggestions:
1. support one more scenario, and all old scenarios are dealt like
before, but it still can't support discontinuous extent after isize.
2. support all scenarios, but sacrifice performance for lots of common scenarios by checking about 10^9 blocks.

IMO, we can think about #2 whether there is an efficient way.

How many cases does this incur?
One is fallocate with keeping i_size, ana other?

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

How about adding FADVISE_OVER_ISIZE to represent inode has blocks beyond i_size? Then, we can set this flag in fallocate and reset it in f2fs_truncate.

I have a similar idea that add an actual size which marks the end of
last extent, so we can know if the current extent is the last one, even without searching for extents behind.

Where do you want to store that size in disk?

I'm still not very confident about this idea, so I didn't really think that through yet, inode may be a proper place.
Or we just write a special function to find it, and call it only when fiemap is called and disk size isn't initiated yet. After all this isn't frequently-used.

Quoted 8 lines
Quoted 4 lines

But there is a problem I still can't figure out, after truncate an
extent at the end of file beyond isize , how do I know where the new
last extent ends or if there are still extents beyond isize? after all, the extents beyond isize could be discontinuous.

So, that's why I proposed a flag instead of a kind of i_disksize.
We can just set the flag, only if a file *may* have a extent beyond i_size in fallocate, and unset it through f2fs_truncate. Moreover, I don't expect that this happens so frequently.

if flag could indicate "may", will i_disksize do a better job in the same way?
let i_disksize be the end of the last extent that *may* exist beyond i_size, set it also in fallocate,when truncate, if the length is still beyond isize, we set the new length as i_disksize, so in worse scenario we won't have to search up to s_maxbytes.

Quoted 7 lines

Thanks,

Quoted 4 lines
Quoted 4 lines

Thanks,

------------------------------------------------------------------------------

maintainer
#9 lore raw

Hi,

Quoted 61 lines
Quoted 43 lines
Quoted 43 lines
Quoted 43 lines
Quoted 29 lines
Quoted 20 lines
Quoted 20 lines
Quoted 12 lines · Fan Li
Quoted 8 lines

f2fs allows preallocation beyond isize, but f2fs_fiemap only look up extents within isize. Therefore add this support.

Note: It's possible that there are holes after isize, for example, fallocate multiple discontinuous extents after isize with FALLOC_FL_KEEP_SIZE set. Since I can tell no differences between EOF and holes from return of get_data_block, I'm afaid this patch can't support such scenarios.

As you mentioned, preallocated block beyond isize can be allocated in f2fs, and we are trying

to support mapping extents across

Quoted 2 lines

whole data space of inode, so why we treat theses extents inside i_size and outside i_size

separately? IMO, instead using i_size, we

Quoted 4 lines

should use max blocks as boundary.

Most important, this interface still can't support finding all extents after i_size, which

Quoted 8 lines
Quoted 5 lines

looks buggy for our user.

Notice that this issue exists before my patch, by adding this patch, at least now it can support more scenarios such as fallocate a range right after isize. I'd say it's an improvement.

Nope, what I'm talking about is *correctness* of our ->fiemap interface, but you're trying to avoid it by saying "support more

cases,

Quoted 8 lines
Quoted 1 line

it's an improvement". That doesn't make any sense to me, since correctness issue still not be fixed.

I'm not sure what you mean by avoiding, I think the comment and
reply I written has already stated the issue and limitation of this patch.
Now there are two suggestions:
1. support one more scenario, and all old scenarios are dealt like
before, but it still can't support discontinuous extent after isize.
2. support all scenarios, but sacrifice performance for lots of common scenarios by checking about 10^9 blocks.

IMO, we can think about #2 whether there is an efficient way.

How many cases does this incur?
One is fallocate with keeping i_size, ana other?

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

How about adding FADVISE_OVER_ISIZE to represent inode has blocks beyond i_size? Then, we can set this flag in fallocate and reset it in f2fs_truncate.

I have a similar idea that add an actual size which marks the end of
last extent, so we can know if the current extent is the last one, even without searching for extents behind.

Where do you want to store that size in disk?

I'm still not very confident about this idea, so I didn't really think that through yet, inode may be a proper place.
Or we just write a special function to find it, and call it only when fiemap is called and disk size isn't initiated yet. After all this isn't frequently-used.

Quoted 5 lines
Quoted 4 lines

But there is a problem I still can't figure out, after truncate an
extent at the end of file beyond isize , how do I know where the new
last extent ends or if there are still extents beyond isize? after all, the extents beyond isize could be discontinuous.

Quoted 8 lines
Quoted 3 lines

So, that's why I proposed a flag instead of a kind of i_disksize.
We can just set the flag, only if a file *may* have a extent beyond i_size in fallocate, and unset it through f2fs_truncate. Moreover, I don't expect that this happens so frequently.

if flag could indicate "may", will i_disksize do a better job in the same way?
let i_disksize be the end of the last extent that *may* exist beyond i_size, set it also in fallocate,when truncate, if the length is still beyond isize, we set the new length as i_disksize, so in worse scenario we won't have to search up to s_maxbytes.

My real concern is that there no space for i_disksize.

Thanks,

------------------------------------------------------------------------------

author
#10 lore raw
Quoted 72 lines

-----Original Message-----
From: Jaegeuk Kim [mailto:[email protected]]
Sent: Tuesday, January 05, 2016 11:08 AM
To: Fan Li; ""@jaegeuk-2.local
Cc: 'Chao Yu'; [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Hi,

Quoted 61 lines
Quoted 44 lines
Quoted 44 lines
Quoted 44 lines
Quoted 30 lines
Quoted 22 lines
Quoted 22 lines
Quoted 12 lines · Fan Li
Quoted 8 lines

f2fs allows preallocation beyond isize, but
f2fs_fiemap only look up extents within isize. Therefore add this support.

Note: It's possible that there are holes after isize,
for example, fallocate multiple discontinuous extents after isize with FALLOC_FL_KEEP_SIZE set. Since I can
tell no differences between EOF and holes from return
of get_data_block, I'm afaid this patch can't support such scenarios.

As you mentioned, preallocated block beyond isize can be allocated in f2fs, and we are trying

to support mapping extents across

Quoted 2 lines

whole data space of inode, so why we treat theses extents inside i_size and outside i_size

separately? IMO, instead using i_size, we

Quoted 4 lines

should use max blocks as boundary.

Most important, this interface still can't support finding all extents after i_size, which

looks buggy for our user.

Quoted 7 lines
Quoted 3 lines

Notice that this issue exists before my patch, by adding
this patch, at least now it can support more scenarios
such as fallocate a range right after isize. I'd say it's an improvement.

Nope, what I'm talking about is *correctness* of our ->fiemap interface, but you're trying to avoid it by saying "support more

cases,

Quoted 8 lines
Quoted 1 line

it's an improvement". That doesn't make any sense to me, since correctness issue still not be fixed.

I'm not sure what you mean by avoiding, I think the comment
and reply I written has already stated the issue and limitation of this patch.
Now there are two suggestions:
1. support one more scenario, and all old scenarios are dealt
like before, but it still can't support discontinuous extent after isize.
2. support all scenarios, but sacrifice performance for lots of common scenarios by checking about 10^9 blocks.

IMO, we can think about #2 whether there is an efficient way.

How many cases does this incur?
One is fallocate with keeping i_size, ana other?

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

How about adding FADVISE_OVER_ISIZE to represent inode has blocks beyond i_size? Then, we can set this flag in fallocate and reset it in f2fs_truncate.

I have a similar idea that add an actual size which marks the end
of last extent, so we can know if the current extent is the last one, even without searching for extents behind.

Where do you want to store that size in disk?

I'm still not very confident about this idea, so I didn't really think
that through yet, inode may be a proper place.
Or we just write a special function to find it, and call it only when fiemap is called and disk size isn't initiated yet. After all this isn't frequently-used.

Quoted 4 lines
Quoted 4 lines

But there is a problem I still can't figure out, after truncate
an extent at the end of file beyond isize , how do I know where
the new last extent ends or if there are still extents beyond isize? after all, the extents beyond isize could be

discontinuous.

Quoted 11 lines
Quoted 9 lines
Quoted 4 lines

So, that's why I proposed a flag instead of a kind of i_disksize.
We can just set the flag, only if a file *may* have a extent beyond i_size in fallocate, and unset it through f2fs_truncate. Moreover, I don't expect that this happens so frequently.

if flag could indicate "may", will i_disksize do a better job in the same way?
let i_disksize be the end of the last extent that *may* exist beyond
i_size, set it also in fallocate,when truncate, if the length is still
beyond isize, we set the new length as i_disksize, so in worse scenario we won't have to search up to s_maxbytes.

My real concern is that there no space for i_disksize.

I see. How about combine both ideas, use flag as you propose, and if fiemap ever needs to look up the last extent beyond isize, we calculate i_disksize first and cache it in f2fs_inode_info, use it to determine the last extent.

This way we don't have to look up all blocks up to s_maxbytes, and need no extra space in disk.

Quoted 2 lines

Thanks,

------------------------------------------------------------------------------

maintainer
#11 lore raw

Hi,

...

Quoted 46 lines
Quoted 32 lines
Quoted 32 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 4 lines
Quoted 3 lines

Nope, what I'm talking about is *correctness* of our ->fiemap interface, but you're trying to avoid it by saying "support more

cases,

Quoted 8 lines
Quoted 1 line

it's an improvement". That doesn't make any sense to me, since correctness issue still not be fixed.

I'm not sure what you mean by avoiding, I think the comment
and reply I written has already stated the issue and limitation of this patch.
Now there are two suggestions:
1. support one more scenario, and all old scenarios are dealt
like before, but it still can't support discontinuous extent after isize.
2. support all scenarios, but sacrifice performance for lots of common scenarios by checking about 10^9 blocks.

IMO, we can think about #2 whether there is an efficient way.

How many cases does this incur?
One is fallocate with keeping i_size, ana other?

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

How about adding FADVISE_OVER_ISIZE to represent inode has blocks beyond i_size? Then, we can set this flag in fallocate and reset it in f2fs_truncate.

I have a similar idea that add an actual size which marks the end
of last extent, so we can know if the current extent is the last one, even without searching for extents behind.

Where do you want to store that size in disk?

I'm still not very confident about this idea, so I didn't really think
that through yet, inode may be a proper place.
Or we just write a special function to find it, and call it only when fiemap is called and disk size isn't initiated yet. After all this isn't frequently-used.

Quoted 1 line
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

But there is a problem I still can't figure out, after truncate
an extent at the end of file beyond isize , how do I know where
the new last extent ends or if there are still extents beyond isize? after all, the extents beyond isize could be

discontinuous.

Quoted 10 lines
Quoted 9 lines
Quoted 4 lines

So, that's why I proposed a flag instead of a kind of i_disksize.
We can just set the flag, only if a file *may* have a extent beyond i_size in fallocate, and unset it through f2fs_truncate. Moreover, I don't expect that this happens so frequently.

if flag could indicate "may", will i_disksize do a better job in the same way?
let i_disksize be the end of the last extent that *may* exist beyond
i_size, set it also in fallocate,when truncate, if the length is still
beyond isize, we set the new length as i_disksize, so in worse scenario we won't have to search up to s_maxbytes.

Quoted 7 lines
Quoted 1 line

My real concern is that there no space for i_disksize.

I see. How about combine both ideas, use flag as you propose, and if fiemap ever needs to look up the last extent beyond isize, we calculate i_disksize first and cache it in f2fs_inode_info, use it to determine the last extent.

This way we don't have to look up all blocks up to s_maxbytes, and need no extra space in disk.

Looks good to me.

Thanks,

------------------------------------------------------------------------------

#12 lore raw
In reply to Jaegeuk Kim

Hi all,

Quoted 77 lines

-----Original Message-----
From: Jaegeuk Kim [mailto:[email protected]]
Sent: Monday, January 04, 2016 2:56 PM
To: Fan Li
Cc: 'Chao Yu'; [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Hello,

Quoted 49 lines · Fan Li
Quoted 49 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Thursday, December 31, 2015 2:34 PM
To: 'Fan Li'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: RE: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 40 lines

-----Original Message-----
From: Fan Li [mailto:[email protected]]
Sent: Thursday, December 31, 2015 11:37 AM
To: 'Chao Yu'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 14 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Wednesday, December 30, 2015 9:28 PM
To: Fan Li; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Hi,

Quoted 3 lines · Fan Li

f2fs allows preallocation beyond isize, but f2fs_fiemap only look up extents within isize. Therefore add this support.

Quoted 8 lines
Quoted 5 lines

Note: It's possible that there are holes after isize, for example, fallocate multiple discontinuous extents after isize with FALLOC_FL_KEEP_SIZE set. Since I can tell no differences between EOF and holes from return of get_data_block, I'm afaid this patch can't support such scenarios.

As you mentioned, preallocated block beyond isize can be allocated in f2fs, and we are trying

to support mapping extents across

Quoted 2 lines

whole data space of inode, so why we treat theses extents inside i_size and outside i_size

separately? IMO, instead using i_size, we

Quoted 4 lines

should use max blocks as boundary.

Most important, this interface still can't support finding all extents after i_size, which

Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

looks buggy for our user.

Notice that this issue exists before my patch, by adding this patch, at least now it can support more scenarios such as fallocate a range right after isize. I'd say it's an improvement.

Nope, what I'm talking about is *correctness* of our ->fiemap interface, but you're trying

to avoid it by saying "support more

Quoted 2 lines

cases,

Quoted 1 line

it's an improvement". That doesn't make any sense to me, since correctness issue still not

be fixed.

Quoted 2 lines

I'm not sure what you mean by avoiding, I think the comment and reply I written has already

stated the issue and

Quoted 3 lines

limitation of this patch.
Now there are two suggestions:
1. support one more scenario, and all old scenarios are dealt like before, but it still can't

support discontinuous extent after

Quoted 8 lines
Quoted 2 lines

isize.
2. support all scenarios, but sacrifice performance for lots of common scenarios by checking

about 10^9 blocks.

IMO, we can think about #2 whether there is an efficient way.

How many cases does this incur?
One is fallocate with keeping i_size, ana other?

AFAIK, no more.

Quoted 3 lines

How about adding FADVISE_OVER_ISIZE to represent inode has blocks beyond i_size? Then, we can set this flag in fallocate and reset it in f2fs_truncate.

Append write in such inode could destroy this convention, right?

Thanks,

Quoted 48 lines

Thanks,

Quoted 3 lines

I think we can all agree both ideas have their flaws.
The only divergence is that I vote the first, and you the second. I think the most important

thing is that it works fluently in most

Quoted 3 lines

scenarios, and you think is that it works in every scenarios even it's very slow.

I think my method is more pratical, but balance between performance and utility seems to be

an Eternal problem.

Quoted 14 lines
Quoted 14 lines
Quoted 10 lines

use max blocks as boundary would get us the precise result, but it also means after we reach the EOF, we still need to look up every block between the EOF and sb-> s_maxbytes to make sure the EOF is true, that's about 4TB or 10^9 blocks.
And it affects all scenarios where the search range covers the last extent in the file, not just extents beyond isize. I think this price is too high to pay.

That's another performance issue.

Quoted 1 line
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

I was hoping that I can make f2fs_map_block return an EOF to solve this problem some time later, or anyone have a better idea?

At least we can seek valid dnode block like the way llseek use.

In addition, for most cases, few of i_nid[5] in f2fs_inode will be NULL, we could skip searching

all dnode block in such

Quoted 16 lines

non-allocated

Quoted 15 lines

indirect node, instead of searching dnode block f2fs_map_block one by one.

Thanks,

Quoted 11 lines
Quoted 11 lines

Thanks,

Quoted 7 lines
---
 fs/f2fs/data.c |    7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
Quoted 35 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

fs/f2fs/data.c int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info

Quoted 6 lines
fs/f2fs/data.c index renamed from fs/f2fs/data.c +0 -0

a9a4d89..f89cf07
100644

fs/f2fs/data.c +0 -0
@@ -798,12 +798,6 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info

*fieinfo,

Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Quoted 19 lines
Quoted 19 lines
fs/f2fs/data.c +1 -6
 	isize = i_size_read(inode);
 
 	mutex_lock(&inode->i_mutex);
-	if (start >= isize)
-		goto out;
-
-	if (start + len > isize)
-		len = isize - start;
-
 	if (logical_to_blk(inode, len) == 0)
 		len = blk_to_logical(inode, 1);
 
@@ -829,6 +823,7 @@ next:
 		 * punch holes beyond isize and keep size unchanged.
 		 */
 		flags |= FIEMAP_EXTENT_LAST;
+		last_blk = start_blk - 1;
 	}
 
Quoted 2 lines

fs/f2fs/data.c next: · line 829

Quoted 2 lines
fs/f2fs/data.c +0 -0
 	if (size)

----------------------------------------------------------------------
-------- _______________________________________________ Linux-f2fs-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

------------------------------------------------------------------------------

maintainer
#13 lore raw

Hi,

...

Quoted 21 lines
Quoted 12 lines
Quoted 1 line
Quoted 1 line

Nope, what I'm talking about is *correctness* of our ->fiemap interface, but you're trying

to avoid it by saying "support more

Quoted 2 lines

cases,

Quoted 1 line

it's an improvement". That doesn't make any sense to me, since correctness issue still not

be fixed.

Quoted 2 lines

I'm not sure what you mean by avoiding, I think the comment and reply I written has already

stated the issue and

Quoted 3 lines

limitation of this patch.
Now there are two suggestions:
1. support one more scenario, and all old scenarios are dealt like before, but it still can't

support discontinuous extent after

Quoted 8 lines
Quoted 2 lines

isize.
2. support all scenarios, but sacrifice performance for lots of common scenarios by checking

about 10^9 blocks.

IMO, we can think about #2 whether there is an efficient way.

How many cases does this incur?
One is fallocate with keeping i_size, ana other?

Quoted 7 lines

AFAIK, no more.

Quoted 3 lines

How about adding FADVISE_OVER_ISIZE to represent inode has blocks beyond i_size? Then, we can set this flag in fallocate and reset it in f2fs_truncate.

Append write in such inode could destroy this convention, right?

Right, but we have another chance to reset the flag, when fiemap checks the end of allocated space.

Thanks,

Quoted 86 lines

Thanks,

Quoted 83 lines

Thanks,

Quoted 3 lines

I think we can all agree both ideas have their flaws.
The only divergence is that I vote the first, and you the second. I think the most important

thing is that it works fluently in most

Quoted 3 lines

scenarios, and you think is that it works in every scenarios even it's very slow.

I think my method is more pratical, but balance between performance and utility seems to be

an Eternal problem.

Quoted 14 lines
Quoted 14 lines
Quoted 10 lines

use max blocks as boundary would get us the precise result, but it also means after we reach the EOF, we still need to look up every block between the EOF and sb-> s_maxbytes to make sure the EOF is true, that's about 4TB or 10^9 blocks.
And it affects all scenarios where the search range covers the last extent in the file, not just extents beyond isize. I think this price is too high to pay.

That's another performance issue.

Quoted 1 line
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

I was hoping that I can make f2fs_map_block return an EOF to solve this problem some time later, or anyone have a better idea?

At least we can seek valid dnode block like the way llseek use.

In addition, for most cases, few of i_nid[5] in f2fs_inode will be NULL, we could skip searching

all dnode block in such

Quoted 16 lines

non-allocated

Quoted 15 lines

indirect node, instead of searching dnode block f2fs_map_block one by one.

Thanks,

Quoted 11 lines
Quoted 11 lines

Thanks,

Quoted 7 lines
---
 fs/f2fs/data.c |    7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

fs/f2fs/data.c int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info

Quoted 6 lines
fs/f2fs/data.c index renamed from fs/f2fs/data.c +0 -0

a9a4d89..f89cf07
100644

fs/f2fs/data.c +0 -0
@@ -798,12 +798,6 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info

*fieinfo,

Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Quoted 19 lines
Quoted 19 lines
fs/f2fs/data.c +1 -6
 	isize = i_size_read(inode);
 
 	mutex_lock(&inode->i_mutex);
-	if (start >= isize)
-		goto out;
-
-	if (start + len > isize)
-		len = isize - start;
-
 	if (logical_to_blk(inode, len) == 0)
 		len = blk_to_logical(inode, 1);
 
@@ -829,6 +823,7 @@ next:
 		 * punch holes beyond isize and keep size unchanged.
 		 */
 		flags |= FIEMAP_EXTENT_LAST;
+		last_blk = start_blk - 1;
 	}
 
Quoted 2 lines

fs/f2fs/data.c next: · line 829

Quoted 2 lines
fs/f2fs/data.c +0 -0
 	if (size)

----------------------------------------------------------------------
-------- _______________________________________________ Linux-f2fs-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

------------------------------------------------------------------------------

#14 lore raw
Quoted 38 lines

-----Original Message-----
From: Jaegeuk Kim [mailto:[email protected]]
Sent: Tuesday, January 05, 2016 11:30 AM
To: Chao Yu
Cc: 'Fan Li'; [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Hi,

...

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Nope, what I'm talking about is *correctness* of our ->fiemap interface, but you're

trying

Quoted 3 lines
Quoted 3 lines

to avoid it by saying "support more

Quoted 2 lines

cases,

Quoted 1 line

it's an improvement". That doesn't make any sense to me, since correctness issue still

not

Quoted 7 lines
Quoted 7 lines

be fixed.

Quoted 2 lines

I'm not sure what you mean by avoiding, I think the comment and reply I written has already

stated the issue and

Quoted 3 lines

limitation of this patch.
Now there are two suggestions:
1. support one more scenario, and all old scenarios are dealt like before, but it still

can't

Quoted 13 lines
Quoted 9 lines

support discontinuous extent after

Quoted 2 lines

isize.
2. support all scenarios, but sacrifice performance for lots of common scenarios by checking

about 10^9 blocks.

IMO, we can think about #2 whether there is an efficient way.

How many cases does this incur?
One is fallocate with keeping i_size, ana other?

AFAIK, no more.

Quoted 1 line
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

How about adding FADVISE_OVER_ISIZE to represent inode has blocks beyond i_size? Then, we can set this flag in fallocate and reset it in f2fs_truncate.

Append write in such inode could destroy this convention, right?

Right, but we have another chance to reset the flag, when fiemap checks the end of allocated space.

Agreed, :)

Thanks,

Quoted 32 lines

Thanks,

Quoted 13 lines

Thanks,

Quoted 10 lines

Thanks,

Quoted 3 lines

I think we can all agree both ideas have their flaws.
The only divergence is that I vote the first, and you the second. I think the most important

thing is that it works fluently in most

Quoted 3 lines

scenarios, and you think is that it works in every scenarios even it's very slow.

I think my method is more pratical, but balance between performance and utility seems

to be

Quoted 15 lines
Quoted 15 lines

an Eternal problem.

Quoted 14 lines
Quoted 14 lines
Quoted 10 lines

use max blocks as boundary would get us the precise result, but it also means after we reach the EOF, we still need to look up every block between the EOF and sb-> s_maxbytes to make sure the EOF is true, that's about 4TB or 10^9 blocks.
And it affects all scenarios where the search range covers the last extent in the file, not just extents beyond isize. I think this price is too high to pay.

That's another performance issue.

Quoted 1 line
Quoted 59 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

I was hoping that I can make f2fs_map_block return an EOF to solve this problem some time later, or anyone have a better idea?

At least we can seek valid dnode block like the way llseek use.

In addition, for most cases, few of i_nid[5] in f2fs_inode will be NULL, we could skip

searching

Quoted 52 lines
Quoted 52 lines

all dnode block in such

Quoted 16 lines

non-allocated

Quoted 15 lines

indirect node, instead of searching dnode block f2fs_map_block one by one.

Thanks,

Quoted 11 lines
Quoted 11 lines

Thanks,

Quoted 7 lines
---
 fs/f2fs/data.c |    7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

fs/f2fs/data.c int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info

Quoted 6 lines
fs/f2fs/data.c index renamed from fs/f2fs/data.c +0 -0

a9a4d89..f89cf07
100644

fs/f2fs/data.c +0 -0
@@ -798,12 +798,6 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info

*fieinfo,

Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Quoted 19 lines
Quoted 19 lines
fs/f2fs/data.c +1 -6
 	isize = i_size_read(inode);
 
 	mutex_lock(&inode->i_mutex);
-	if (start >= isize)
-		goto out;
-
-	if (start + len > isize)
-		len = isize - start;
-
 	if (logical_to_blk(inode, len) == 0)
 		len = blk_to_logical(inode, 1);
 
@@ -829,6 +823,7 @@ next:
 		 * punch holes beyond isize and keep size unchanged.
 		 */
 		flags |= FIEMAP_EXTENT_LAST;
+		last_blk = start_blk - 1;
 	}
 
Quoted 2 lines

fs/f2fs/data.c next: · line 829

Quoted 2 lines
fs/f2fs/data.c +0 -0
 	if (size)

----------------------------------------------------------------------
-------- _______________________________________________ Linux-f2fs-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

------------------------------------------------------------------------------

#15 lore raw
In reply to Fan Li
Quoted 63 lines

-----Original Message-----
From: Fan Li [mailto:[email protected]]
Sent: Monday, January 04, 2016 1:57 PM
To: 'Chao Yu'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: RE: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 47 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Thursday, December 31, 2015 2:34 PM
To: 'Fan Li'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: RE: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 40 lines

-----Original Message-----
From: Fan Li [mailto:[email protected]]
Sent: Thursday, December 31, 2015 11:37 AM
To: 'Chao Yu'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 14 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Wednesday, December 30, 2015 9:28 PM
To: Fan Li; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Hi,

Quoted 3 lines · Fan Li

f2fs allows preallocation beyond isize, but f2fs_fiemap only look up extents within isize. Therefore add this support.

Quoted 8 lines
Quoted 5 lines

Note: It's possible that there are holes after isize, for example, fallocate multiple discontinuous extents after isize with FALLOC_FL_KEEP_SIZE set. Since I can tell no differences between EOF and holes from return of get_data_block, I'm afaid this patch can't support such scenarios.

As you mentioned, preallocated block beyond isize can be allocated in f2fs, and we are trying

to support mapping extents across

Quoted 2 lines

whole data space of inode, so why we treat theses extents inside i_size and outside i_size

separately? IMO, instead using i_size, we

Quoted 4 lines

should use max blocks as boundary.

Most important, this interface still can't support finding all extents after i_size, which

Quoted 7 lines
Quoted 5 lines

looks buggy for our user.

Notice that this issue exists before my patch, by adding this patch, at least now it can support more scenarios such as fallocate a range right after isize. I'd say it's an improvement.

Nope, what I'm talking about is *correctness* of our ->fiemap interface, but you're trying

Quoted 8 lines

to avoid it by saying "support more
cases,

Quoted 1 line

it's an improvement". That doesn't make any sense to me, since correctness issue still not

be fixed.

I'm not sure what you mean by avoiding, I think the comment and reply I written has already stated the issue and
limitation of this patch.

What I mean here is it's better to stand in user's viewpoint, let the interface return the correct result, since user only refer the manual of interface, rather than comments in patch or reply in email.

Quoted 9 lines

Now there are two suggestions:
1. support one more scenario, and all old scenarios are dealt like before, but it still can't support discontinuous extent after
isize.
2. support all scenarios, but sacrifice performance for lots of common scenarios by checking about 10^9 blocks.

I think we can all agree both ideas have their flaws. The only divergence is that I vote the first, and you the second.

I didn't vote the second, IMO, it's better support all cases firstly, at least, we should not let user experience 'sometimes success, sometimes fail' in our ->fiemap. Then, if there are performance issue for common cases, we could try to do some improvement on it as I mentioned below.

I think Jaegeuk has better idea for the performance issue, please refer to his comments.

Thanks,

Quoted 43 lines

I think the most important
thing is that it works fluently in most
scenarios, and you think is that it works in every scenarios even it's very slow.

I think my method is more pratical, but balance between performance and utility seems to be an Eternal problem.

Quoted 13 lines
Quoted 9 lines

use max blocks as boundary would get us the precise result, but it also means after we reach the EOF, we still need to look up every block between the EOF and sb-> s_maxbytes to make sure the EOF is true, that's about 4TB or 10^9 blocks.
And it affects all scenarios where the search range covers the last extent in the file, not just extents beyond isize. I think this price is too high to pay.

That's another performance issue.

Quoted 1 line
Quoted 6 lines
Quoted 2 lines

I was hoping that I can make f2fs_map_block return an EOF to solve this problem some time later, or anyone have a better idea?

At least we can seek valid dnode block like the way llseek use.

In addition, for most cases, few of i_nid[5] in f2fs_inode will be NULL, we could skip searching

all dnode block in such
non-allocated

Quoted 15 lines

indirect node, instead of searching dnode block f2fs_map_block one by one.

Thanks,

Quoted 11 lines
Quoted 11 lines

Thanks,

Quoted 7 lines
---
 fs/f2fs/data.c |    7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
Quoted 35 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

fs/f2fs/data.c int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info

Quoted 6 lines
fs/f2fs/data.c index renamed from fs/f2fs/data.c +0 -0

a9a4d89..f89cf07
100644

fs/f2fs/data.c +0 -0
@@ -798,12 +798,6 @@ int f2fs_fiemap(struct inode *inode, struct fiemap_extent_info

*fieinfo,

Quoted 28 lines
Quoted 28 lines
Quoted 19 lines
Quoted 19 lines
fs/f2fs/data.c +1 -6
 	isize = i_size_read(inode);
 
 	mutex_lock(&inode->i_mutex);
-	if (start >= isize)
-		goto out;
-
-	if (start + len > isize)
-		len = isize - start;
-
 	if (logical_to_blk(inode, len) == 0)
 		len = blk_to_logical(inode, 1);
 
@@ -829,6 +823,7 @@ next:
 		 * punch holes beyond isize and keep size unchanged.
 		 */
 		flags |= FIEMAP_EXTENT_LAST;
+		last_blk = start_blk - 1;
 	}
 
Quoted 2 lines

fs/f2fs/data.c next: · line 829

Quoted 2 lines
fs/f2fs/data.c +0 -0
 	if (size)

----------------------------------------------------------------------
-------- _______________________________________________ Linux-f2fs-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

------------------------------------------------------------------------------

author
#16 lore raw
Quoted 77 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Monday, January 04, 2016 6:00 PM
To: 'Fan Li'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: RE: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 68 lines

-----Original Message-----
From: Fan Li [mailto:[email protected]]
Sent: Monday, January 04, 2016 1:57 PM
To: 'Chao Yu'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: RE: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 48 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Thursday, December 31, 2015 2:34 PM
To: 'Fan Li'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: RE: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 40 lines

-----Original Message-----
From: Fan Li [mailto:[email protected]]
Sent: Thursday, December 31, 2015 11:37 AM
To: 'Chao Yu'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 14 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Wednesday, December 30, 2015 9:28 PM
To: Fan Li; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Hi,

Quoted 3 lines · Fan Li

f2fs allows preallocation beyond isize, but f2fs_fiemap only look up extents within isize. Therefore add this support.

Quoted 8 lines
Quoted 5 lines

Note: It's possible that there are holes after isize, for example, fallocate multiple discontinuous extents after isize with FALLOC_FL_KEEP_SIZE set. Since I can tell no differences between EOF and holes from return of get_data_block, I'm afaid this patch can't support such scenarios.

As you mentioned, preallocated block beyond isize can be allocated in f2fs, and we are trying

to support mapping extents across

Quoted 2 lines

whole data space of inode, so why we treat theses extents inside i_size and outside i_size

separately? IMO, instead using i_size, we

Quoted 4 lines

should use max blocks as boundary.

Most important, this interface still can't support finding all extents after i_size, which

Quoted 8 lines
Quoted 5 lines

looks buggy for our user.

Notice that this issue exists before my patch, by adding this patch, at least now it can support more scenarios such as fallocate a range right after isize. I'd say it's an improvement.

Nope, what I'm talking about is *correctness* of our ->fiemap interface, but you're trying

to avoid it by saying "support more
cases,

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

it's an improvement". That doesn't make any sense to me, since correctness issue still not

be fixed.

I'm not sure what you mean by avoiding, I think the comment and reply I written has already stated the issue and limitation of this patch.

What I mean here is it's better to stand in user's viewpoint, let the interface return the correct result, since user only refer

the manual

Quoted 3 lines

of interface, rather than comments in patch or reply in email.

From viewpoint of users, this patch makes the functionality of fiemap a little closer to the manual,

it supports one more scenario that manual says, why isn't it an improvement?

Besides there are a lot of examples in kernel, that is written in comment and different from the manual, the old version of this function happens to be one of them, are you saying that if we don't solve this, we shouldn't support this function at all?

Of course there are solutions at the end, as I said in reply, I have an idea about further improvement, but what's wrong with improving it one step at a time?

Quoted 11 lines
Quoted 9 lines

Now there are two suggestions:
1. support one more scenario, and all old scenarios are dealt like before, but it still can't support discontinuous extent after isize.
2. support all scenarios, but sacrifice performance for lots of common scenarios by checking about 10^9 blocks.

I think we can all agree both ideas have their flaws. The only divergence is that I vote the first, and you the second.

I didn't vote the second, IMO, it's better support all cases firstly, at least, we should not let user experience 'sometimes

success,

Quoted 88 lines

sometimes fail' in our ->fiemap. Then, if there are performance issue for common cases, we could try to do some improvement on it as I mentioned below.

I think Jaegeuk has better idea for the performance issue, please refer to his comments.

Thanks,

Quoted 81 lines

I think the most important
thing is that it works fluently in most scenarios, and you think is that it works in every scenarios even it's very slow.

I think my method is more pratical, but balance between performance and utility seems to be an Eternal problem.

Quoted 13 lines
Quoted 9 lines

use max blocks as boundary would get us the precise result, but it also means after we reach the EOF, we still need to look up every block between the EOF and sb-> s_maxbytes to make sure the EOF is true, that's about 4TB or 10^9 blocks.
And it affects all scenarios where the search range covers the last extent in the file, not just extents beyond isize. I think this price is too high to pay.

That's another performance issue.

Quoted 1 line
Quoted 7 lines
Quoted 2 lines

I was hoping that I can make f2fs_map_block return an EOF to solve this problem some time later, or anyone have a better idea?

At least we can seek valid dnode block like the way llseek use.

In addition, for most cases, few of i_nid[5] in f2fs_inode will be NULL, we could skip searching

all dnode block in such
non-allocated

Quoted 15 lines

indirect node, instead of searching dnode block f2fs_map_block one by one.

Thanks,

Quoted 11 lines
Quoted 11 lines

Thanks,

Quoted 7 lines
---
 fs/f2fs/data.c |    7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
fs/f2fs/data.c index renamed from fs/f2fs/data.c +0 -0

a9a4d89..f89cf07
100644

fs/f2fs/data.c +0 -0
@@ -798,12 +798,6 @@ int f2fs_fiemap(struct inode *inode,

struct fiemap_extent_info

*fieinfo,

Quoted 29 lines
Quoted 29 lines
Quoted 19 lines
Quoted 19 lines
fs/f2fs/data.c +1 -6
 	isize = i_size_read(inode);
 
 	mutex_lock(&inode->i_mutex);
-	if (start >= isize)
-		goto out;
-
-	if (start + len > isize)
-		len = isize - start;
-
 	if (logical_to_blk(inode, len) == 0)
 		len = blk_to_logical(inode, 1);
@@ -829,6 +823,7 @@ next:
 		 * punch holes beyond isize and keep size unchanged.
 		 */
 		flags |= FIEMAP_EXTENT_LAST;
+		last_blk = start_blk - 1;
 	}
 
Quoted 2 lines

fs/f2fs/data.c next: · line 829

Quoted 2 lines
fs/f2fs/data.c +0 -0
 	if (size)

------------------------------------------------------------------
----
-------- _______________________________________________ Linux-f2fs-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

------------------------------------------------------------------------------

#17 lore raw
Quoted 106 lines

-----Original Message-----
From: Fan Li [mailto:[email protected]]
Sent: Monday, January 04, 2016 6:55 PM
To: 'Chao Yu'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: RE: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 75 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Monday, January 04, 2016 6:00 PM
To: 'Fan Li'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: RE: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 68 lines

-----Original Message-----
From: Fan Li [mailto:[email protected]]
Sent: Monday, January 04, 2016 1:57 PM
To: 'Chao Yu'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: RE: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 48 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Thursday, December 31, 2015 2:34 PM
To: 'Fan Li'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: RE: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 40 lines

-----Original Message-----
From: Fan Li [mailto:[email protected]]
Sent: Thursday, December 31, 2015 11:37 AM
To: 'Chao Yu'; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Quoted 14 lines

-----Original Message-----
From: Chao Yu [mailto:[email protected]]
Sent: Wednesday, December 30, 2015 9:28 PM
To: Fan Li; 'Jaegeuk Kim'
Cc: [email protected]
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: support finding extents after isize

Hi,

Quoted 3 lines · Fan Li

f2fs allows preallocation beyond isize, but f2fs_fiemap only look up extents within isize. Therefore add this support.

Quoted 8 lines
Quoted 5 lines

Note: It's possible that there are holes after isize, for example, fallocate multiple discontinuous extents after isize with FALLOC_FL_KEEP_SIZE set. Since I can tell no differences between EOF and holes from return of get_data_block, I'm afaid this patch can't support such scenarios.

As you mentioned, preallocated block beyond isize can be allocated in f2fs, and we are trying

to support mapping extents across

Quoted 2 lines

whole data space of inode, so why we treat theses extents inside i_size and outside i_size

separately? IMO, instead using i_size, we

Quoted 4 lines

should use max blocks as boundary.

Most important, this interface still can't support finding all extents after i_size, which

Quoted 8 lines
Quoted 5 lines

looks buggy for our user.

Notice that this issue exists before my patch, by adding this patch, at least now it can support more scenarios such as fallocate a range right after isize. I'd say it's an improvement.

Nope, what I'm talking about is *correctness* of our ->fiemap interface, but you're trying

to avoid it by saying "support more
cases,

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

it's an improvement". That doesn't make any sense to me, since correctness issue still not

be fixed.

I'm not sure what you mean by avoiding, I think the comment and reply I written has already stated the issue and limitation of this patch.

What I mean here is it's better to stand in user's viewpoint, let the interface return the

correct result, since user only refer
the manual

Quoted 1 line

of interface, rather than comments in patch or reply in email.

From viewpoint of users, this patch makes the functionality of fiemap a little closer to the manual,
it supports one more scenario that manual says, why isn't it an improvement?

Besides there are a lot of examples in kernel, that is written in comment and different from the manual,
the old version of this function happens to be one of them, are you saying that if we don't solve this,
we shouldn't support this function at all?

Quoted 3 lines

Of course there are solutions at the end, as I said in reply, I have an idea about further improvement,
but what's wrong with improving it one step at a time?

In brief, as we discussed in private, we have reached an agreement that we will continue to improve on this patch to solve related issue as Jaegeuk suggested.

Thanks,

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Now there are two suggestions:
1. support one more scenario, and all old scenarios are dealt like before, but it still can't support discontinuous extent after isize.
2. support all scenarios, but sacrifice performance for lots of common scenarios by checking about 10^9 blocks.

Quoted 95 lines
Quoted 4 lines
Quoted 2 lines

I think we can all agree both ideas have their flaws. The only divergence is that I vote the first, and you the second.

I didn't vote the second, IMO, it's better support all cases firstly, at least, we should

not let user experience 'sometimes
success,

Quoted 1 line

sometimes fail' in our ->fiemap. Then, if there are performance issue for common cases, we

could try to do some improvement on it

Quoted 87 lines

as I mentioned below.

I think Jaegeuk has better idea for the performance issue, please refer to his comments.

Thanks,

Quoted 81 lines

I think the most important
thing is that it works fluently in most scenarios, and you think is that it works in every scenarios even it's very slow.

I think my method is more pratical, but balance between performance and utility seems to be an Eternal problem.

Quoted 13 lines
Quoted 9 lines

use max blocks as boundary would get us the precise result, but it also means after we reach the EOF, we still need to look up every block between the EOF and sb-> s_maxbytes to make sure the EOF is true, that's about 4TB or 10^9 blocks.
And it affects all scenarios where the search range covers the last extent in the file, not just extents beyond isize. I think this price is too high to pay.

That's another performance issue.

Quoted 1 line
Quoted 7 lines
Quoted 2 lines

I was hoping that I can make f2fs_map_block return an EOF to solve this problem some time later, or anyone have a better idea?

At least we can seek valid dnode block like the way llseek use.

In addition, for most cases, few of i_nid[5] in f2fs_inode will be NULL, we could skip searching

all dnode block in such
non-allocated

Quoted 15 lines

indirect node, instead of searching dnode block f2fs_map_block one by one.

Thanks,

Quoted 11 lines
Quoted 11 lines

Thanks,

Quoted 7 lines
---
 fs/f2fs/data.c |    7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
fs/f2fs/data.c index renamed from fs/f2fs/data.c +0 -0

a9a4d89..f89cf07
100644

fs/f2fs/data.c +0 -0
@@ -798,12 +798,6 @@ int f2fs_fiemap(struct inode *inode,

struct fiemap_extent_info

*fieinfo,

Quoted 29 lines
Quoted 29 lines
Quoted 19 lines
Quoted 19 lines
fs/f2fs/data.c +1 -6
 	isize = i_size_read(inode);
 
 	mutex_lock(&inode->i_mutex);
-	if (start >= isize)
-		goto out;
-
-	if (start + len > isize)
-		len = isize - start;
-
 	if (logical_to_blk(inode, len) == 0)
 		len = blk_to_logical(inode, 1);
@@ -829,6 +823,7 @@ next:
 		 * punch holes beyond isize and keep size unchanged.
 		 */
 		flags |= FIEMAP_EXTENT_LAST;
+		last_blk = start_blk - 1;
 	}
 
Quoted 2 lines

fs/f2fs/data.c next: · line 829

Quoted 2 lines
fs/f2fs/data.c +0 -0
 	if (size)

------------------------------------------------------------------
----
-------- _______________________________________________ Linux-f2fs-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

------------------------------------------------------------------------------