Missing fix backports detected by syzbot
Hi Greg, Sasha,
A number of commits were identified[1] by syzbot as non-backported fixes for the fuzzer-detected findings in various Linux LTS trees.
[1] https://syzkaller.appspot.com/upstream/backports
Please consider backporting the following commits to LTS v6.1:9a8ec9e8ebb5a7c0cfbce2d6b4a6b67b2b78e8f3 "Bluetooth: SCO: Fix possible circular locking dependency on sco_connect_cfm" (fixes 9a8ec9e) 3dcaa192ac2159193bc6ab57bc5369dcb84edd8e "Bluetooth: SCO: fix sco_conn related locking and validity issues" 3f5424790d4377839093b68c12b130077a4e4510 "ext4: fix inode tree inconsistency caused by ENOMEM" 7b0151caf73a656b75b550e361648430233455a0 "KVM: x86: Remove WARN sanity check on hypervisor timer vs. UNINITIALIZED vCPU" c2efd13a2ed4f29bf9ef14ac2fbb7474084655f8 "udf: Limit file size to 4TB" 4b827b3f305d1fcf837265f1e12acc22ee84327c "xfs: remove WARN when dquot cache insertion fails"
These were verified to apply cleanly on top of v6.1.107 and to build/boot.
The following commits to LTS v5.15:8216776ccff6fcd40e3fdaa109aa4150ebe760b3 "ext4: reject casefold inode flag without casefold feature" c2efd13a2ed4f29bf9ef14ac2fbb7474084655f8 "udf: Limit file size to 4TB"
These were verified to apply cleanly on top of v5.15.165 and to build/boot.
The following commits to LTS v5.10:04e568a3b31cfbd545c04c8bfc35c20e5ccfce0f "ext4: handle redirtying in ext4_bio_write_page()" 2a1fc7dc36260fbe74b6ca29dc6d9088194a2115 "KVM: x86: Suppress MMIO that is triggered during task switch emulation" 2454ad83b90afbc6ed2c22ec1310b624c40bf0d3 "fs: Restrict lock_two_nondirectories() to non-directory inodes" (fixes 2454ad) 33ab231f83cc12d0157711bbf84e180c3be7d7bc "fs: don't assume arguments are non-NULL"
These were verified to apply cleanly on top of v5.10.224 and to build/boot.
There are also a lot of syzbot-detected fix commits that did not apply cleanly, but the conflicts seem to be quite straightforward to resolve manually. Could you please share what the current process is with respect to such fix patches? For example, are you sending emails asking developers to adjust the non-applied patch (if they want), or is it the other way around -- you expect the authors to be proactive and send the adjusted patch versions themselves?
Some sample commits, which failed to apply to v6.1.107:ff91059932401894e6c86341915615c5eb0eca48 "bpf, sockmap: Prevent lock inversion deadlock in map delete elem" f8f210dc84709804c9f952297f2bfafa6ea6b4bd "btrfs: calculate the right space for delayed refs when updating global reserve"
Signature · 1 line
Aleksandr
Greg KH
maintainerQuoted 7 lines · Aleksandr Nogikh
There are also a lot of syzbot-detected fix commits that did not apply cleanly, but the conflicts seem to be quite straightforward to resolve manually. Could you please share what the current process is with respect to such fix patches? For example, are you sending emails asking developers to adjust the non-applied patch (if they want), or is it the other way around -- you expect the authors to be proactive and send the adjusted patch versions themselves?
We expect someone to send us patches, we aren't going to be able to do it ourselves :)
thanks,
greg k-h
Greg KH
maintainerQuoted 5 lines · Aleksandr Nogikh
Hi Greg, Sasha,
A number of commits were identified[1] by syzbot as non-backported fixes for the fuzzer-detected findings in various Linux LTS trees.
Quoted 7 lines
[1] https://syzkaller.appspot.com/upstream/backports
Please consider backporting the following commits to LTS v6.1:
9a8ec9e8ebb5a7c0cfbce2d6b4a6b67b2b78e8f3"Bluetooth: SCO: Fix possible circular locking dependency onsco_connect_cfm" (fixes 9a8ec9e)3dcaa192ac2159193bc6ab57bc5369dcb84edd8e"Bluetooth: SCO: fixsco_connrelated locking and validity issues"3f5424790d4377839093b68c12b130077a4e4510"ext4: fix inode tree inconsistency caused by ENOMEM"7b0151caf73a656b75b550e361648430233455a0"KVM: x86: Remove WARN sanity check on hypervisor timer vs. UNINITIALIZED vCPU"
Note, for kvm, and for:
Quoted 1 line
4b827b3f305d1fcf837265f1e12acc22ee84327c"xfs: remove WARN when dquot cache insertion fails"
xfs patches, we need explicit approval from the subsystem maintainers to take them into stable as they were not marked for such.
Please work with them to get those patches merged if you wish to see them in stable trees.
thanks,
greg k-h
Greg KH
maintainerQuoted 15 lines · Aleksandr Nogikh
Hi Greg, Sasha,
A number of commits were identified[1] by syzbot as non-backported fixes for the fuzzer-detected findings in various Linux LTS trees.
[1] https://syzkaller.appspot.com/upstream/backports
Please consider backporting the following commits to LTS v6.1:
9a8ec9e8ebb5a7c0cfbce2d6b4a6b67b2b78e8f3"Bluetooth: SCO: Fix possible circular locking dependency onsco_connect_cfm" (fixes 9a8ec9e)3dcaa192ac2159193bc6ab57bc5369dcb84edd8e"Bluetooth: SCO: fixsco_connrelated locking and validity issues"3f5424790d4377839093b68c12b130077a4e4510"ext4: fix inode tree inconsistency caused by ENOMEM"7b0151caf73a656b75b550e361648430233455a0"KVM: x86: Remove WARN sanity check on hypervisor timer vs. UNINITIALIZED vCPU"c2efd13a2ed4f29bf9ef14ac2fbb7474084655f8"udf: Limit file size to 4TB"4b827b3f305d1fcf837265f1e12acc22ee84327c"xfs: remove WARN when dquot cache insertion fails"
Quoted 5 lines
These were verified to apply cleanly on top of v6.1.107 and to build/boot.
The following commits to LTS v5.15:
8216776ccff6fcd40e3fdaa109aa4150ebe760b3"ext4: reject casefold inode flag without casefold feature"
Wait, what about 6.1 for this? We can't move to a new kernel and have a regression.
Quoted 7 lines
c2efd13a2ed4f29bf9ef14ac2fbb7474084655f8"udf: Limit file size to 4TB"These were verified to apply cleanly on top of v5.15.165 and to build/boot.
The following commits to LTS v5.10:
04e568a3b31cfbd545c04c8bfc35c20e5ccfce0f"ext4: handle redirtying inext4_bio_write_page()"
Same here, what about 5.15.y?
Quoted 3 lines
2a1fc7dc36260fbe74b6ca29dc6d9088194a2115"KVM: x86: Suppress MMIO that is triggered during task switch emulation"2454ad83b90afbc6ed2c22ec1310b624c40bf0d3"fs: Restrictlock_two_nondirectories()to non-directory inodes" (fixes 2454ad)33ab231f83cc12d0157711bbf84e180c3be7d7bc"fs: don't assume arguments are non-NULL"
Why are these last two needed?
Can you provide full lists of what needs to go to what tree, and better yet, tested patch series for this type of thing in the future?
thanks,
greg k-h
Hi Greg,
Thank you very much for your comments!
Quoted 7 lines · Greg KH
Note, for kvm, and for:
Quoted 1 line
4b827b3f305d1fcf837265f1e12acc22ee84327c"xfs: remove WARN when dquot cache insertion fails"xfs patches, we need explicit approval from the subsystem maintainers to take them into stable as they were not marked for such.
Is it specific only to some subsystems (like kvm and xfs), or is it due to some general rule like "if the commit was not initially marked as a stable backport candidate, you need an approval from the subsystem maintainer"?
Quoted 17 lines · Greg KH
Quoted 15 lines · Aleksandr Nogikh
Hi Greg, Sasha,
A number of commits were identified[1] by syzbot as non-backported fixes for the fuzzer-detected findings in various Linux LTS trees.
[1] https://syzkaller.appspot.com/upstream/backports
Please consider backporting the following commits to LTS v6.1:
9a8ec9e8ebb5a7c0cfbce2d6b4a6b67b2b78e8f3"Bluetooth: SCO: Fix possible circular locking dependency onsco_connect_cfm" (fixes 9a8ec9e)3dcaa192ac2159193bc6ab57bc5369dcb84edd8e"Bluetooth: SCO: fixsco_connrelated locking and validity issues"3f5424790d4377839093b68c12b130077a4e4510"ext4: fix inode tree inconsistency caused by ENOMEM"7b0151caf73a656b75b550e361648430233455a0"KVM: x86: Remove WARN sanity check on hypervisor timer vs. UNINITIALIZED vCPU"c2efd13a2ed4f29bf9ef14ac2fbb7474084655f8"udf: Limit file size to 4TB"4b827b3f305d1fcf837265f1e12acc22ee84327c"xfs: remove WARN when dquot cache insertion fails"
Quoted 8 lines
Quoted 5 lines
These were verified to apply cleanly on top of v6.1.107 and to build/boot.
The following commits to LTS v5.15:
8216776ccff6fcd40e3fdaa109aa4150ebe760b3"ext4: reject casefold inode flag without casefold feature"Wait, what about 6.1 for this? We can't move to a new kernel and have a regression.
Indeed!
Syzbot currently constructs missing backports lists independently for each fuzzed LTS, so the page [1] does have some holes. But in any case, it's totally reasonable that if we backport a commit to an older kernel, newer ones should also get it.
Quoted 9 lines
Quoted 8 lines
c2efd13a2ed4f29bf9ef14ac2fbb7474084655f8"udf: Limit file size to 4TB"These were verified to apply cleanly on top of v5.15.165 and to build/boot.
The following commits to LTS v5.10:
04e568a3b31cfbd545c04c8bfc35c20e5ccfce0f"ext4: handle redirtying inext4_bio_write_page()"
Quoted 7 lines
Same here, what about 5.15.y?
Quoted 3 lines
2a1fc7dc36260fbe74b6ca29dc6d9088194a2115"KVM: x86: Suppress MMIO that is triggered during task switch emulation"2454ad83b90afbc6ed2c22ec1310b624c40bf0d3"fs: Restrictlock_two_nondirectories()to non-directory inodes" (fixes 2454ad)33ab231f83cc12d0157711bbf84e180c3be7d7bc"fs: don't assume arguments are non-NULL"Why are these last two needed?
The last two address the "WARNING: bad unlock balance in unlock_two_nondirectories" bug on a Linux 5.10-based kernel. The crash report is very similar to
https://syzkaller.appspot.com/bug?id=32c54626e170a6b327ca2c8ae4c1aea666a8c20b
Quoted 3 lines
Can you provide full lists of what needs to go to what tree, and better yet, tested patch series for this type of thing in the future?
Sure!
Could you please clarify a bit what should be the criteria for the full lists and what exact kinds of tests you mean?
If the list only contains the patches that apply cleanly, it is unfortunately not very big anyway since the vast majority of the detected missing backports just cannot be cherry-picked automatically. I guess, otherwise most of them would have already been backported :) I plan to try to manually adjust and resend some of them to better understand how much manual effort it actually requires.
For the commits I listed so far, I checked that they apply without problems, whether there are fix commits for them and whether the kernel builds and boots fine. It should be easy to also verify whether syzbot is able to reproduce the bug after the fix is cherry-picked. What are some other tests that would be great to have run?
Signature · 1 line
Aleksandr
Quoted 4 lines
thanks,
greg k-h
Greg KH
maintainerQuoted 10 lines · Aleksandr Nogikh
Hi Greg,
Thank you very much for your comments!
Quoted 5 lines · Greg KH
Note, for kvm, and for:
Quoted 1 line
4b827b3f305d1fcf837265f1e12acc22ee84327c"xfs: remove WARN when dquot cache insertion fails"
Quoted 7 lines
Quoted 2 lines
xfs patches, we need explicit approval from the subsystem maintainers to take them into stable as they were not marked for such.
Is it specific only to some subsystems (like kvm and xfs), or is it due to some general rule like "if the commit was not initially marked as a stable backport candidate, you need an approval from the subsystem maintainer"?
We have a list, see this file:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/stable-queue.git/tree/ignore_list
Quoted 19 lines
Quoted 19 lines
Quoted 18 lines
A number of commits were identified[1] by syzbot as non-backported fixes for the fuzzer-detected findings in various Linux LTS trees.
[1] https://syzkaller.appspot.com/upstream/backports
Please consider backporting the following commits to LTS v6.1:
9a8ec9e8ebb5a7c0cfbce2d6b4a6b67b2b78e8f3"Bluetooth: SCO: Fix possible circular locking dependency onsco_connect_cfm" (fixes 9a8ec9e)3dcaa192ac2159193bc6ab57bc5369dcb84edd8e"Bluetooth: SCO: fixsco_connrelated locking and validity issues"3f5424790d4377839093b68c12b130077a4e4510"ext4: fix inode tree inconsistency caused by ENOMEM"7b0151caf73a656b75b550e361648430233455a0"KVM: x86: Remove WARN sanity check on hypervisor timer vs. UNINITIALIZED vCPU"c2efd13a2ed4f29bf9ef14ac2fbb7474084655f8"udf: Limit file size to 4TB"4b827b3f305d1fcf837265f1e12acc22ee84327c"xfs: remove WARN when dquot cache insertion fails"These were verified to apply cleanly on top of v6.1.107 and to build/boot.
The following commits to LTS v5.15:
8216776ccff6fcd40e3fdaa109aa4150ebe760b3"ext4: reject casefold inode flag without casefold feature"
Quoted 8 lines
Quoted 2 lines
Wait, what about 6.1 for this? We can't move to a new kernel and have a regression.
Indeed!
Syzbot currently constructs missing backports lists independently for each fuzzed LTS, so the page [1] does have some holes. But in any case, it's totally reasonable that if we backport a commit to an older kernel, newer ones should also get it.
It's actually a requirement, and I am pretty sure, is documented somewhere...
Quoted 14 lines
Quoted 14 lines
Quoted 7 lines
c2efd13a2ed4f29bf9ef14ac2fbb7474084655f8"udf: Limit file size to 4TB"These were verified to apply cleanly on top of v5.15.165 and to build/boot.
The following commits to LTS v5.10:
04e568a3b31cfbd545c04c8bfc35c20e5ccfce0f"ext4: handle redirtying inext4_bio_write_page()"Same here, what about 5.15.y?
Quoted 3 lines
2a1fc7dc36260fbe74b6ca29dc6d9088194a2115"KVM: x86: Suppress MMIO that is triggered during task switch emulation"2454ad83b90afbc6ed2c22ec1310b624c40bf0d3"fs: Restrictlock_two_nondirectories()to non-directory inodes" (fixes 2454ad)33ab231f83cc12d0157711bbf84e180c3be7d7bc"fs: don't assume arguments are non-NULL"
Quoted 6 lines
Quoted 1 line
Why are these last two needed?
The last two address the "WARNING: bad unlock balance in
unlock_two_nondirectories" bug on a Linux 5.10-based kernel. The crash report is very similar to
https://syzkaller.appspot.com/bug?id=32c54626e170a6b327ca2c8ae4c1aea666a8c20b
As the changelog does not show that, I'd prefer to get an ack from the developers involved before accepting core vfs changes that are not obviously a bugfix and no one has run into before in the wild.
Also, just lockdep fixes to add markings aren't always needed to be backported, UNLESS someone is reporting that this is annoying their system tests, as sometimes happens.
Quoted 6 lines
Quoted 2 lines
Can you provide full lists of what needs to go to what tree, and better yet, tested patch series for this type of thing in the future?
Sure!
Could you please clarify a bit what should be the criteria for the full lists and what exact kinds of tests you mean?
A full list of the git ids, for each branch, you didn't do that here as a bunch were missed for newer kernels.
Quoted 4 lines
If the list only contains the patches that apply cleanly, it is unfortunately not very big anyway since the vast majority of the detected missing backports just cannot be cherry-picked automatically. I guess, otherwise most of them would have already been backported :)
Sometimes, but if they are not explicitly marked to be backported with a
cc: stable line, there is no guarantee at all that they will be.
Quoted 8 lines
I plan to try to manually adjust and resend some of them to better understand how much manual effort it actually requires.
For the commits I listed so far, I checked that they apply without problems, whether there are fix commits for them and whether the kernel builds and boots fine. It should be easy to also verify whether syzbot is able to reproduce the bug after the fix is cherry-picked. What are some other tests that would be great to have run?
That's all a great start, try doing that first and we can go from there.
thanks,
greg k-h