iomap: ->iomap_end() error handling fixes
Subsystems (2)
cover iomap: ->iomap_end() error handling fixes
Hi all,
This is a couple small error handling fixes for ->iomap_end() errors (via iomap_iter()). The immediate problem here was that the ->iomap_end() error return started overriding an iter.status error code, which on ext4 happened to trigger dio fallback to buffered I/O in some cases. Jan has actually fixed that separately in ext4 [1], but I wanted to take an independent look at iomap to see if it is worth fixing as well.
The more I poked around the more it seemed like it's more appropriate to return the initial error code in iter.status if one is pending. I also eventually noticed the DAX vs. reflink case documented in patch 2, which further tweaks the error handling and supports the former reasoning.
These are separate patches because they are separate issues, but I'm still doing some testing and wanted to see if there was any initial feedback before dropping the RFC. Thoughts?
Brian
[1] https://lore.kernel.org/linux-ext4/[email protected]/
Brian Foster (2):
iomap: prioritize iter.status error over ->iomap_end()
iomap: revert the iomap_iter pos on ->iomap_end() error
fs/iomap/iter.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-)
Signature · 1 line
2.51.0
RFC PATCH 1/2 iomap: prioritize iter.status error over ->iomap_end()
Jan Kara reports that commit bc264fea0f6f subtly changed error handling behavior in iomap_iter() in the case where both iter.status and ->iomap_end() return error codes. Previously, iter.status had priority and would return to the caller regardless of the ->iomap_end() result. After the change, an ->iomap_end() error returns immediately.
This had the unexpected side effect of enabling a DIO fallback to buffered write on ext4 because ->iomap_end() could return -ENOTBLK and overload an -EINVAL error from the core iomap direct I/O code.
This has been fixed independently in ext4, but nonetheless the change in iomap was unintentional. Since other filesystems may use this in similar ways, restore long standing behavior and always return the value of iter.status if it happens to contain an error code.
--- fs/iomap/iter.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/iomap/iter.c +1 -1
@@ -80,7 +80,7 @@ int iomap_iter(struct iomap_iter *iter, const struct iomap_ops *ops)iomap_length_trim (iter, iter->iter_start_pos, olen), advanced, iter->flags, &iter->iomap); -if (ret <0 && !advanced) +if (ret <0 && !advanced && !iter->status)return ret; }
Signature · 1 line
2.51.0
Re: [PATCH RFC 1/2] iomap: prioritize iter.status error over ->iomap_end()
Looks good:
Re: [PATCH RFC 1/2] iomap: prioritize iter.status error over ->iomap_end()
In reply to Brian FosterQuoted 17 lines · Brian Foster
Jan Kara reports that commit
bc264fea0f6fsubtly changed error handling behavior iniomap_iter()in the case where both iter.status and ->iomap_end() return error codes. Previously, iter.status had priority and would return to the caller regardless of the ->iomap_end() result. After the change, an ->iomap_end() error returns immediately.This had the unexpected side effect of enabling a DIO fallback to buffered write on ext4 because ->iomap_end() could return
-ENOTBLKand overload an-EINVALerror from the core iomap direct I/O code.This has been fixed independently in ext4, but nonetheless the change in iomap was unintentional. Since other filesystems may use this in similar ways, restore long standing behavior and always return the value of iter.status if it happens to contain an error code.
Quoted 3 lines
Fixes bc264fea0f6f ("iomap: support incremental iomap_iter advances")Diagnosed-by Jan Kara <[email protected]>Signed-off-by Brian Foster <[email protected]>
Looks good. Feel free to add:
Honza
Quoted 20 lines
--- fs/iomap/iter.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/iomap/iter.c +1
-1@@ -80,7 +80,7 @@ int iomap_iter(struct iomap_iter *iter, const struct iomap_ops *ops)iomap_length_trim (iter, iter->iter_start_pos, olen), advanced, iter->flags, &iter->iomap); -if (ret <0 && !advanced) +if (ret <0 && !advanced && !iter->status)return ret; }Signature · 1 line
2.51.0
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Re: [PATCH RFC 1/2] iomap: prioritize iter.status error over ->iomap_end()
In reply to Brian FosterQuoted 17 lines · Brian Foster
Jan Kara reports that commit
bc264fea0f6fsubtly changed error handling behavior iniomap_iter()in the case where both iter.status and ->iomap_end() return error codes. Previously, iter.status had priority and would return to the caller regardless of the ->iomap_end() result. After the change, an ->iomap_end() error returns immediately.This had the unexpected side effect of enabling a DIO fallback to buffered write on ext4 because ->iomap_end() could return
-ENOTBLKand overload an-EINVALerror from the core iomap direct I/O code.This has been fixed independently in ext4, but nonetheless the change in iomap was unintentional. Since other filesystems may use this in similar ways, restore long standing behavior and always return the value of iter.status if it happens to contain an error code.
Quoted 4 lines
Fixes bc264fea0f6f ("iomap: support incremental iomap_iter advances")Diagnosed-by Jan Kara <[email protected]>Signed-off-by Brian Foster <[email protected]>---
Looks good to me. Please feel free to add:
Quoted 18 lines
fs/iomap/iter.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)fs/iomap/iter.c +1
-1@@ -80,7 +80,7 @@ int iomap_iter(struct iomap_iter *iter, const struct iomap_ops *ops)iomap_length_trim (iter, iter->iter_start_pos, olen), advanced, iter->flags, &iter->iomap); -if (ret <0 && !advanced) +if (ret <0 && !advanced && !iter->status)return ret; }Signature · 1 line
2.51.0
RFC PATCH 2/2 iomap: revert the iomap_iter pos on ->iomap_end() error
In reply to Brian FosterAn iomap op iteration should not be considered successful if ->iomap_end() fails. Most ->iomap_end() callbacks do not return errors, and for those that do we return the error to the caller, but this is still not sufficient in some corner cases.
For example, if a DAX write to a shared iomap fails at ->iomap_end() on XFS, this means the remap of shared blocks from the COW fork to the data fork has possibly failed. In turn this means that just written data may not be accessible in the file. dax_iomap_rw() returns partial success over a returned error code and the operation has already advanced iter.pos by the time ->iomap_end() is called. This means that dax_iomap_rw() can return more bytes processed than have been completed successfully, including partial success instead of an error code if the first iteration happens to fail.
To address this problem, first tweak the ->iomap_end() error handling logic to run regardless of whether the current iteration advanced the iter. Next, revert pos in the error handling path. Add a new helper to undo the changes from iomap_iter_advance(). It is static to start since the only initial user is in iomap_iter.c.
--- fs/iomap/iter.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-)
fs/iomap/iter.c +19 -1
@@ -27,6 +27,22 @@ int iomap_iter_advance(struct iomap_iter *iter, u64 *count)return 0 ; } +/** +* iomap_iter_revert - revert the iterator position +* @iter: iteration structure +* @count: number of bytes to revert +* +* Revert the iterator position by the specified number of bytes, undoing +* the effect of a previous iomap_iter_advance() call. The count must not +* exceed the amount previously advanced in the current iter. +*/ +static void iomap_iter_revert (struct iomap_iter *iter,u64 count) +{ + count =min_t (u64, iter->pos - iter->iter_start_pos, count); + iter->pos -= count; + iter->len += count; +} +static inline void iomap_iter_done (struct iomap_iter *iter) {WARN_ON_ONCE (iter->iomap.offset > iter->pos);
@@ -80,8 +96,10 @@ int iomap_iter(struct iomap_iter *iter, const struct iomap_ops *ops)iomap_length_trim (iter, iter->iter_start_pos, olen), advanced, iter->flags, &iter->iomap); -if (ret <0 && !advanced && !iter->status) +if (ret <0 && !iter->status) { +iomap_iter_revert (iter, advanced);return ret; + } }/* detect old return semantics where this would advance */
Signature · 1 line
2.51.0
Re: [PATCH RFC 2/2] iomap: revert the iomap_iter pos on ->iomap_end() error
Quoted 55 lines · Brian Foster
An iomap op iteration should not be considered successful if ->iomap_end() fails. Most ->iomap_end() callbacks do not return errors, and for those that do we return the error to the caller, but this is still not sufficient in some corner cases.
For example, if a DAX write to a shared iomap fails at ->iomap_end() on XFS, this means the remap of shared blocks from the COW fork to the data fork has possibly failed. In turn this means that just written data may not be accessible in the file.
dax_iomap_rw()returns partial success over a returned error code and the operation has already advanced iter.pos by the time ->iomap_end() is called. This means thatdax_iomap_rw()can return more bytes processed than have been completed successfully, including partial success instead of an error code if the first iteration happens to fail.To address this problem, first tweak the ->iomap_end() error handling logic to run regardless of whether the current iteration advanced the iter. Next, revert pos in the error handling path. Add a new helper to undo the changes from
iomap_iter_advance(). It is static to start since the only initial user is iniomap_iter.c.Signed-off-by Brian Foster <[email protected]>--- fs/iomap/iter.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-)fs/iomap/iter.c +16
-0@@ -27,6 +27,22 @@ int iomap_iter_advance(struct iomap_iter *iter, u64 *count)return 0 ; } +/** +* iomap_iter_revert - revert the iterator position +* @iter: iteration structure +* @count: number of bytes to revert +* +* Revert the iterator position by the specified number of bytes, undoing +* the effect of a previous iomap_iter_advance() call. The count must not +* exceed the amount previously advanced in the current iter. +*/ +static void iomap_iter_revert (struct iomap_iter *iter,u64 count) +{ + count =min_t (u64, iter->pos - iter->iter_start_pos, count); + iter->pos -= count; + iter->len += count; +} +static inline void iomap_iter_done (struct iomap_iter *iter) {WARN_ON_ONCE (iter->iomap.offset > iter->pos);
fs/iomap/iter.c int iomap_iter(struct iomap_iter *iter, const struct iomap_ops *ops) · line 102
Quoted 8 lines
fs/iomap/iter.c +3
-1@@ -80,8 +96,10 @@ int iomap_iter(struct iomap_iter *iter, const struct iomap_ops *ops)iomap_length_trim (iter, iter->iter_start_pos, olen), advanced, iter->flags, &iter->iomap); -if (ret <0 && !advanced && !iter->status) +if (ret <0 && !iter->status) { +iomap_iter_revert (iter, advanced);return ret; + }
Should iomap_iter_revert() also be called in the "if (iter->status < 0)" case a few lines below? I think otherwise, that leads to the same problem in dax_iomap_rw() you pointed out in the commit message.
Thanks,
Joanne
Quoted 7 lines
fs/iomap/iter.c +0
-1}/* detect old return semantics where this would advance */ --2.51.0
Re: [PATCH RFC 2/2] iomap: revert the iomap_iter pos on ->iomap_end() error
Quoted 60 lines · Joanne Koong
Quoted 59 lines · Brian Foster
An iomap op iteration should not be considered successful if ->iomap_end() fails. Most ->iomap_end() callbacks do not return errors, and for those that do we return the error to the caller, but this is still not sufficient in some corner cases.
For example, if a DAX write to a shared iomap fails at ->iomap_end() on XFS, this means the remap of shared blocks from the COW fork to the data fork has possibly failed. In turn this means that just written data may not be accessible in the file.
dax_iomap_rw()returns partial success over a returned error code and the operation has already advanced iter.pos by the time ->iomap_end() is called. This means thatdax_iomap_rw()can return more bytes processed than have been completed successfully, including partial success instead of an error code if the first iteration happens to fail.To address this problem, first tweak the ->iomap_end() error handling logic to run regardless of whether the current iteration advanced the iter. Next, revert pos in the error handling path. Add a new helper to undo the changes from
iomap_iter_advance(). It is static to start since the only initial user is iniomap_iter.c.Signed-off-by Brian Foster <[email protected]>--- fs/iomap/iter.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-)fs/iomap/iter.c +16
-1@@ -27,6 +27,22 @@ int iomap_iter_advance(struct iomap_iter *iter, u64 *count)return 0 ; } +/** +* iomap_iter_revert - revert the iterator position +* @iter: iteration structure +* @count: number of bytes to revert +* +* Revert the iterator position by the specified number of bytes, undoing +* the effect of a previous iomap_iter_advance() call. The count must not +* exceed the amount previously advanced in the current iter. +*/ +static void iomap_iter_revert (struct iomap_iter *iter,u64 count) +{ + count =min_t (u64, iter->pos - iter->iter_start_pos, count); + iter->pos -= count; + iter->len += count; +} +static inline void iomap_iter_done (struct iomap_iter *iter) {WARN_ON_ONCE (iter->iomap.offset > iter->pos);@@ -80,8 +96,10 @@ int iomap_iter(struct iomap_iter *iter, const struct iomap_ops *ops)iomap_length_trim (iter, iter->iter_start_pos, olen), advanced, iter->flags, &iter->iomap); -if (ret <0 && !advanced && !iter->status)
Quoted 9 lines
fs/iomap/iter.c int iomap_iter(struct iomap_iter *iter, const struct iomap_ops *ops) · line 102
Quoted 4 lines
fs/iomap/iter.c +3
-0+if (ret <0 && !iter->status) { +iomap_iter_revert (iter, advanced);return ret; + }Should
iomap_iter_revert()also be called in the "if (iter->status< 0)" case a few lines below? I think otherwise, that leads to the same problem indax_iomap_rw()you pointed out in the commit message.
My thinking was that I wanted to try for the invariant that the operation/iteration is responsible to set the iter appropriately in the event that it returns an error in iter.status. I.e., either not advance or revert if appropriate.
This is more consistent with how the iter is advanced and I suspect will help prevent potential whack a mole issues with inconsistent expectations for error handling at the iomap_iter() level. I actually had iomap_iter_revert() non-static originally, but changed it since I didn't spot anywhere it needed to be called as of yet. I could have certainly missed something though. Did you have a particular sequence in mind, or were just thinking in general?
FWIW, I suspect there's a reasonable argument for doing the same for ->iomap_end() and make the callback responsible for reverting if necessary. I went the way in this patch just because it seemed more simple given the limited scope, but that may not always be the case and/or may just be cleaner. I can take a closer look at that if there are stronger opinions..? Thanks for the feedback.
Brian
Quoted 10 lines
Thanks,
JoanneQuoted 7 lines
fs/iomap/iter.c +0
-1}/* detect old return semantics where this would advance */ --2.51.0
Re: [PATCH RFC 2/2] iomap: revert the iomap_iter pos on ->iomap_end() error
Quoted 77 lines · Brian Foster
Quoted 60 lines · Joanne Koong
Quoted 59 lines · Brian Foster
An iomap op iteration should not be considered successful if ->iomap_end() fails. Most ->iomap_end() callbacks do not return errors, and for those that do we return the error to the caller, but this is still not sufficient in some corner cases.
For example, if a DAX write to a shared iomap fails at ->iomap_end() on XFS, this means the remap of shared blocks from the COW fork to the data fork has possibly failed. In turn this means that just written data may not be accessible in the file.
dax_iomap_rw()returns partial success over a returned error code and the operation has already advanced iter.pos by the time ->iomap_end() is called. This means thatdax_iomap_rw()can return more bytes processed than have been completed successfully, including partial success instead of an error code if the first iteration happens to fail.To address this problem, first tweak the ->iomap_end() error handling logic to run regardless of whether the current iteration advanced the iter. Next, revert pos in the error handling path. Add a new helper to undo the changes from
iomap_iter_advance(). It is static to start since the only initial user is iniomap_iter.c.Signed-off-by Brian Foster <[email protected]>--- fs/iomap/iter.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-)fs/iomap/iter.c +16
-1@@ -27,6 +27,22 @@ int iomap_iter_advance(struct iomap_iter *iter, u64 *count)return 0 ; } +/** +* iomap_iter_revert - revert the iterator position +* @iter: iteration structure +* @count: number of bytes to revert +* +* Revert the iterator position by the specified number of bytes, undoing +* the effect of a previous iomap_iter_advance() call. The count must not +* exceed the amount previously advanced in the current iter. +*/ +static void iomap_iter_revert (struct iomap_iter *iter,u64 count) +{ + count =min_t (u64, iter->pos - iter->iter_start_pos, count); + iter->pos -= count; + iter->len += count; +} +static inline void iomap_iter_done (struct iomap_iter *iter) {WARN_ON_ONCE (iter->iomap.offset > iter->pos);@@ -80,8 +96,10 @@ int iomap_iter(struct iomap_iter *iter, const struct iomap_ops *ops)iomap_length_trim (iter, iter->iter_start_pos, olen), advanced, iter->flags, &iter->iomap); -if (ret <0 && !advanced && !iter->status)Quoted 9 lines
fs/iomap/iter.c int iomap_iter(struct iomap_iter *iter, const struct iomap_ops *ops) · line 102
Quoted 4 lines
fs/iomap/iter.c +3
-0+if (ret <0 && !iter->status) { +iomap_iter_revert (iter, advanced);return ret; + }Should
iomap_iter_revert()also be called in the "if (iter->status< 0)" case a few lines below? I think otherwise, that leads to the same problem indax_iomap_rw()you pointed out in the commit message.My thinking was that I wanted to try for the invariant that the operation/iteration is responsible to set the iter appropriately in the event that it returns an error in iter.status. I.e., either not advance or revert if appropriate.
Quoted 7 lines
This is more consistent with how the iter is advanced and I suspect will help prevent potential whack a mole issues with inconsistent expectations for error handling at the
iomap_iter()level. I actually hadiomap_iter_revert()non-static originally, but changed it since I didn't spot anywhere it needed to be called as of yet. I could have certainly missed something though. Did you have a particular sequence in mind, or were just thinking in general?
Thanks for explaining your thought process. That reasoning makes sense to me.
Originally I thought the dax_iomap_rw() sequence needed a iomap_iter_revert() but looking at it again, I'm realizing now that that function is intended to return successfully even if the writes in further iterations fail.
Thanks,
Joanne
Quoted 26 lines
FWIW, I suspect there's a reasonable argument for doing the same for ->iomap_end() and make the callback responsible for reverting if necessary. I went the way in this patch just because it seemed more simple given the limited scope, but that may not always be the case and/or may just be cleaner. I can take a closer look at that if there are stronger opinions..? Thanks for the feedback.
Quoted 5 lines
Quoted 5 lines
returns partial success over a returned error code and the operation has already advanced iter.pos by the time ->iomap_end() is called. This means that
dax_iomap_rw()can return more bytes processed than have been completed successfully, including partial success instead of an error code if the first iteration happens to fail.Brian
Quoted 10 lines
Thanks,
JoanneQuoted 7 lines
} /* detect old return semantics where this would advance */--
2.51.0
Re: [PATCH RFC 2/2] iomap: revert the iomap_iter pos on ->iomap_end() error
In reply to Brian FosterQuoted 15 lines · Brian Foster
An iomap op iteration should not be considered successful if ->iomap_end() fails. Most ->iomap_end() callbacks do not return errors, and for those that do we return the error to the caller, but this is still not sufficient in some corner cases.
For example, if a DAX write to a shared iomap fails at ->iomap_end() on XFS, this means the remap of shared blocks from the COW fork to the data fork has possibly failed. In turn this means that just written data may not be accessible in the file.
dax_iomap_rw()returns partial success over a returned error code and the operation has already advanced iter.pos by the time ->iomap_end() is called. This means thatdax_iomap_rw()can return more bytes processed than have been completed successfully, including partial success instead of an error code if the first iteration happens to fail.
Quoted 7 lines
To address this problem, first tweak the ->iomap_end() error handling logic to run regardless of whether the current iteration advanced the iter. Next, revert pos in the error handling path. Add a new helper to undo the changes from
iomap_iter_advance(). It is static to start since the only initial user is iniomap_iter.c.Signed-off-by Brian Foster <[email protected]>
Looks sensible to me. Feel free to add:
Honza
Quoted 46 lines
--- fs/iomap/iter.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-)fs/iomap/iter.c +19
-1@@ -27,6 +27,22 @@ int iomap_iter_advance(struct iomap_iter *iter, u64 *count)return 0 ; } +/** +* iomap_iter_revert - revert the iterator position +* @iter: iteration structure +* @count: number of bytes to revert +* +* Revert the iterator position by the specified number of bytes, undoing +* the effect of a previous iomap_iter_advance() call. The count must not +* exceed the amount previously advanced in the current iter. +*/ +static void iomap_iter_revert (struct iomap_iter *iter,u64 count) +{ + count =min_t (u64, iter->pos - iter->iter_start_pos, count); + iter->pos -= count; + iter->len += count; +} +static inline void iomap_iter_done (struct iomap_iter *iter) {WARN_ON_ONCE (iter->iomap.offset > iter->pos);@@ -80,8 +96,10 @@ int iomap_iter(struct iomap_iter *iter, const struct iomap_ops *ops)iomap_length_trim (iter, iter->iter_start_pos, olen), advanced, iter->flags, &iter->iomap); -if (ret <0 && !advanced && !iter->status) +if (ret <0 && !iter->status) { +iomap_iter_revert (iter, advanced);return ret; + } }/* detect old return semantics where this would advance */ Signature · 1 line
2.51.0
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR