linux-next: manual merge of the mm-stable tree with the ext4 tree
Hi all,
Today's linux-next merge of the mm-stable tree got a conflict in:
fs/ext4/inode.c
between commits:
726432969963 ("ext4: use nr_to_write directly in mpage_prepare_extent_to_map()") 9ff6a9153c8f ("ext4: move page unlocking out of mpage_submit_page()") b4d26e70a755 ("ext4: move mpage_page_done() calls after error handling") 9b18c23c131a ("ext4: convert data=journal writeback to use ext4_writepages()")
from the ext4 tree and commits:
50ead2537441 ("ext4: convert mpage_prepare_extent_to_map() to use filemap_get_folios_tag()") d585bdbeb79a ("fs: convert writepage_t callback to pass a folio")
from the mm-stable tree.
I have no idea how to fix this up, so I have used the ext4 tree from next-20230217 for today.
Signature · 2 lines
Cheers, Stephen Rothwell
Quoted 15 lines · Stephen Rothwell
Hi all,
Today's linux-next merge of the mm-stable tree got a conflict in:
fs/ext4/inode.cbetween commits:
726432969963 ("ext4: use
nr_to_writedirectly inmpage_prepare_extent_to_map()")9ff6a9153c8f("ext4: move page unlocking out ofmpage_submit_page()")b4d26e70a755("ext4: movempage_page_done()calls after error handling")9b18c23c131a("ext4: convert data=journal writeback to useext4_writepages()")from the ext4 tree and commits:
Quoted 7 lines
50ead2537441("ext4: convertmpage_prepare_extent_to_map()to usefilemap_get_folios_tag()")d585bdbeb79a("fs: convertwritepage_tcallback to pass a folio")from the mm-stable tree.
I have no idea how to fix this up, so I have used the ext4 tree from next-20230217 for today.
Yeah, this one's a mess. Probably the easiest solution is for Ted to pull in 247f9e1feef4 and 50ead2537441 and then redo Jan's patches on top of them.
Hi all,
Quoted 20 lines · On Mon, 20 Feb 2023 14:27:00 +0000 Matthew Wilcox <[email protected]> wrote:
Quoted 18 lines · Stephen Rothwell
Hi all,
Today's linux-next merge of the mm-stable tree got a conflict in:
fs/ext4/inode.cbetween commits:
726432969963 ("ext4: use
nr_to_writedirectly inmpage_prepare_extent_to_map()")9ff6a9153c8f("ext4: move page unlocking out ofmpage_submit_page()")b4d26e70a755("ext4: movempage_page_done()calls after error handling")9b18c23c131a("ext4: convert data=journal writeback to useext4_writepages()")from the ext4 tree and commits:
50ead2537441("ext4: convertmpage_prepare_extent_to_map()to usefilemap_get_folios_tag()")d585bdbeb79a("fs: convertwritepage_tcallback to pass a folio")
Quoted 8 lines
Quoted 4 lines
from the mm-stable tree.
I have no idea how to fix this up, so I have used the ext4 tree from next-20230217 for today.
Yeah, this one's a mess. Probably the easiest solution is for Ted to pull in
247f9e1feef4and50ead2537441and then redo Jan's patches on top of them.
I am still using the ext4 tree from next-20230217 (which was empty relative to Linus' tree).
So, I have to ask why all these commits were only published to linux-next over the weekend?
Signature · 2 lines
Cheers, Stephen Rothwell
Hi all,
Quoted 30 lines · On Tue, 21 Feb 2023 17:54:36 +1100 Stephen Rothwell <[email protected]>…
Quoted 19 lines · On Mon, 20 Feb 2023 14:27:00 +0000 Matthew Wilcox <[email protected]> wrote:
Quoted 17 lines · Stephen Rothwell
Today's linux-next merge of the mm-stable tree got a conflict in:
fs/ext4/inode.cbetween commits:
726432969963 ("ext4: use
nr_to_writedirectly inmpage_prepare_extent_to_map()")9ff6a9153c8f("ext4: move page unlocking out ofmpage_submit_page()")b4d26e70a755("ext4: movempage_page_done()calls after error handling")9b18c23c131a("ext4: convert data=journal writeback to useext4_writepages()")from the ext4 tree and commits:
50ead2537441("ext4: convertmpage_prepare_extent_to_map()to usefilemap_get_folios_tag()")d585bdbeb79a("fs: convertwritepage_tcallback to pass a folio")Quoted 8 lines
Quoted 4 lines
from the mm-stable tree.
I have no idea how to fix this up, so I have used the ext4 tree from next-20230217 for today.
Yeah, this one's a mess. Probably the easiest solution is for Ted to pull in
247f9e1feef4and50ead2537441and then redo Jan's patches on top of them.
Quoted 5 lines
I am still using the ext4 tree from next-20230217 (which was empty relative to Linus' tree).
So, I have to ask why all these commits were only published to linux-next over the weekend?
I still have not been able to figure out a merge resolution for this, so the ext4 tree still has had no testing in linux-next and the merge window is progressing. Does anyone want to help with this?
Signature · 2 lines
Cheers, Stephen Rothwell
Quoted 4 lines · Stephen Rothwell
I still have not been able to figure out a merge resolution for this, so the ext4 tree still has had no testing in linux-next and the merge window is progressing. Does anyone want to help with this?
I think at worst, Linus will have to resolve this mess himself when PR for ext4 is submitted and merged in this merge window.
Thanks.
Signature · 1 line
An old man doll... just what I always wanted! - Clara
Hi Bagas,
Quoted 8 lines · On Fri, 24 Feb 2023 09:42:49 +0700 Bagas Sanjaya <[email protected]> wrote:
Quoted 3 lines · Stephen Rothwell
I still have not been able to figure out a merge resolution for this, so the ext4 tree still has had no testing in linux-next and the merge window is progressing. Does anyone want to help with this?
I think at worst, Linus will have to resolve this mess himself when PR for ext4 is submitted and merged in this merge window.
In which case we will have an ext4 filesystem in Linus' kernel that has no testing ... and if someone has done the testing, then they can at least supply the merge resolution that they tested.
Signature · 2 lines
Cheers, Stephen Rothwell
Quoted 4 lines · Stephen Rothwell
In which case we will have an ext4 filesystem in Linus' kernel that has no testing ... and if someone has done the testing, then they can at least supply the merge resolution that they tested.
Well, the good news is that it will have more testing than the cifs resolution I did - if only because I actually have ext4 filesystems that I'd test with.
I'm not claiming that's great either, of course. I think this whole "convert from find_get_pages_range_tag() to filemap_get_folios_tag() with very different semantics at the same time as all the writepages work is done" was an absolutely *horrible* idea.
I actually considered doing the cifs merge by just keeping the old format, and making a wrapper around "filemap_get_folios_tag()" that would work one page at a time, just to avoid the whole "do two changes at once" issue.
But then I - probably from some excessive hubris - decided I could just rewrite the whole thing without testing, and just get it right on the first try.
I did have DavidH's suggestion to look at, but decided that was buggy, so mine is actually fairly different.
Yes. I do think "excessive hubris" is probably the right word choice. But "insanity" probably also comes close ;)
Linus
Sorry for the delay in taking a look at things; this week has been horribly crazy busy for me.
The conflict is with the folio changes in mm-stable and with Jan's "ext4: Cleanup data=journal writeback path" patch series. Both of these touch mpage_prepare_extent_to_map() in fs/ext4/inode.c quite extensively.
[1] https://lore.kernel.org/r/[email protected]
I've just tried taking Jan's patch series and applying it linux-next (which contains the mm-stable branch). Dealing with each chage one by one, there were conflits with patches #1, #4, #5, and #7. One at the time, the conflict resolutions weren't _that_ bad, but looking at the combined conflict after doing a "git merge" it was quite scary indeed.
Maybe Linus's git merging-fu is much stronger than mine, but I certainly wouldn't wnat to try to resolve it just using "git merge"!
So here's my proposed path forawrd.
1) So far the only testing Jan's patch series (modified so it sits on top of linux-next) is "it builds, ship it". So I'll kick off a full xfstests test on that series, and make sure I don't see any regressions. After that, I'll post it on linux-ext4 for Jan to examine.
(Since it's well after work hours in Europe on a Friday, Jan probably won't get to it until Monday, which is fine.)
2) I'll drop Jan's patch set from the ext4 dev branch, and run the following full xfstests runs on this dev "lite" branch. (a) the dev "lite" branch by itself. (b) the dev "lite" branch merged with tip of Linux's tree. (c) the dev "lite" branch merged with linux-next.
99.9% of the time, when there are problems, they are detected by my full set of xfstests regression testing, since random users running linux-next tend not stress file system all that much, and the 0-day bot doesn't do nearly as much testing as I do. (I test a dozen different ext4 fs configs[2], while the 0-day tests only a single one.)
[2] https://github.com/tytso/xfstests-bld/blob/master/test-appliance/files/root/fs/ext4/cfg/all.list
3) I'll then send the ext4 dev branch (minus the data=writepage cleanups) as a pull request to Linus. Next week, after Jan has a chance to review my patch conflict resolutions, I'll send a second pull request with the data=writepage cleanups. (As usual, I'll do my full set of test runs before sending the pull request.)
Linus, are you OK with this plan?
- Ted
Quoted 8 lines · Theodore Ts'o
3) I'll then send the ext4 dev branch (minus the data=writepage cleanups) as a pull request to Linus. Next week, after Jan has a chance to review my patch conflict resolutions, I'll send a second pull request with the data=writepage cleanups. (As usual, I'll do my full set of test runs before sending the pull request.)
Linus, are you OK with this plan?
Yeah. I think the whole "do two completely different things to the same area in parallel" was horrible, and we should strive to avoid that kind of thing in filesystems in general.
So let's just serialize them instead of doing them at the same time and try to merge them together.
Even if the merge is then successful, it just seems unnecessarily risky from a "what if somebody reports a problem, and it bisects to the merge, and then we have something subtle going on".
Linus