Skip to contentSkip to conversation

net/9p: fix infinite loop in p9_client_rpc on fatal signal

PATCH In review Vasiliy Kovalev 2 participants 9 messages Apr 15 – Jun 22 9P file system

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 net/9p: fix infinite loop in p9_client_rpc on fatal signal

When p9_client_rpc() is called with type P9_TFLUSH and the transport has no peer (e.g. fd transport backed by pipes with no 9p server), a fatal signal causes an infinite loop:

again:

	err = io_wait_event_killable(req->wq, ...)
	/* SIGKILL wakes the task, returns -ERESTARTSYS */

	if (err == -ERESTARTSYS && c->status == Connected &&
		type == P9_TFLUSH) {
		sigpending = 1;
		clear_thread_flag(TIF_SIGPENDING);
		goto again;
	}

clear_thread_flag() clears TIF_SIGPENDING before jumping back to io_wait_event_killable(). signal_pending_state() checks TIF_SIGPENDING, finds it zero, and the task goes to sleep again. The task can only wake on the next signal delivery that calls signal_wake_up() and sets TIF_SIGPENDING again. When that happens the loop repeats, clears TIF_SIGPENDING, and sleeps again indefinitely.

This is triggered in practice by coredump_wait(): when a thread in a multi-threaded process causes a coredump (e.g. via SIGSYS from Syscall User Dispatch), coredump_wait() sends SIGKILL to all other threads and waits for them to call mm_release(). If one of those threads is blocked in p9_client_rpc() over an fd transport with no peer, it enters the P9_TFLUSH loop and never calls mm_release(), so coredump_wait() stalls forever:

INFO: task syz.0.18:676 blocked for more than 143 seconds.
      Not tainted 6.12.77+ #1
task:syz.0.18 state:D stack:27600 pid:676 tgid:673 ppid:630 flags:0x00000004
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5344 [inline]
 __schedule+0xcb4/0x5d50 kernel/sched/core.c:6724
 __schedule_loop kernel/sched/core.c:6801 [inline]
 schedule+0xe5/0x350 kernel/sched/core.c:6816
 schedule_timeout+0x253/0x290 kernel/time/timer.c:2593
 do_wait_for_common kernel/sched/completion.c:95 [inline]
 __wait_for_common+0x409/0x600 kernel/sched/completion.c:116
 wait_for_common kernel/sched/completion.c:127 [inline]
 wait_for_completion_state+0x1d/0x40 kernel/sched/completion.c:264
 coredump_wait fs/coredump.c:448 [inline]
 do_coredump+0x854/0x4350 fs/coredump.c:629
 get_signal+0x1425/0x2730 kernel/signal.c:2903
 arch_do_signal_or_restart+0x81/0x880 arch/x86/kernel/signal.c:337
 exit_to_user_mode_loop kernel/entry/common.c:111 [inline]
 exit_to_user_mode_prepare include/linux/entry-common.h:328 [inline]
 __syscall_exit_to_user_mode_work kernel/entry/common.c:207 [inline]
 syscall_exit_to_user_mode+0xf9/0x160 kernel/entry/common.c:218
 do_syscall_64+0x102/0x220 arch/x86/entry/common.c:84
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
 </TASK>

Fix: check fatal_signal_pending() before clearing TIF_SIGPENDING in the P9_TFLUSH retry loop. At that point TIF_SIGPENDING is still set, so fatal_signal_pending() works correctly. If a fatal signal is pending, jump to recalc_sigpending to restore TIF_SIGPENDING and return -ERESTARTSYS to the caller.

The same defect is present in stable kernels back to 5.4. On those kernels the infinite loop is broken earlier by a second SIGKILL from the parent process (e.g. kill_and_wait() retrying after a timeout), resulting in a zombie process and a shutdown delay rather than a permanent D-state hang, but the underlying flaw is the same.

Found by Linux Verification Center (linuxtesting.org) with Syzkaller.

Fixes 91b8534fa8f5 ("9p: make rpc code common and rework flush code")
---
 net/9p/client.c | 2 ++
 1 file changed, 2 insertions(+)
net/9p/client.c +2 -0
index f0dcf252af7e..748b92d3f0c1 100644
@@ -600,6 +600,8 @@ p9_client_rpc(struct p9_client *c, int8_t type, const char *fmt, ...)
 
 	if (err == -ERESTARTSYS && c->status == Connected &&
 	    type == P9_TFLUSH) {
+		if (fatal_signal_pending(current))
+			goto recalc_sigpending;
 		sigpending = 1;
 		clear_thread_flag(TIF_SIGPENDING);
 		goto again;
Signature · 1 line
2.50.1
#1 lore raw

Vasiliy Kovalev wrote on Wed, Apr 15, 2026 at 06:52:37PM +0300:

Quoted 7 lines

The same defect is present in stable kernels back to 5.4. On those kernels the infinite loop is broken earlier by a second SIGKILL from the parent process (e.g. kill_and_wait() retrying after a timeout), resulting in a zombie process and a shutdown delay rather than a permanent D-state hang, but the underlying flaw is the same.

Found by Linux Verification Center (linuxtesting.org) with Syzkaller.

It's unfortunately not that simple, just leaving here will leak things.

There was another patch last month, help is welcome, please see https://lore.kernel.org/r/[email protected]

(By the way, what's with the rekindled interest on this?...)

Signature · 1 line
Dominique Martinet | Asmadeus
#2 lore raw

Hi Dominique,

Quoted 10 lines · Dominique Martinet

Vasiliy Kovalev wrote on Wed, Apr 15, 2026 at 06:52:37PM +0300:

Quoted 7 lines

The same defect is present in stable kernels back to 5.4. On those kernels the infinite loop is broken earlier by a second SIGKILL from the parent process (e.g. kill_and_wait() retrying after a timeout), resulting in a zombie process and a shutdown delay rather than a permanent D-state hang, but the underlying flaw is the same.

Found by Linux Verification Center (linuxtesting.org) with Syzkaller.

It's unfortunately not that simple, just leaving here will leak things.

Following up on your concern about leaking things (as mentioned in the thread with Shigeru Yoshida's patch) - I did some tracing with printk inside p9_req_put and p9_tag_remove to see exactly what happens to the request lifecycle with my proposed patch vs Shigeru's approach:

net/9p/client.c +3 -0
index 748b92d3f0c1..4e8f4e0195b4 100644
@@ -274,10 +274,13 @@ static void p9_tag_remove(struct p9_client *c, struct p9_req_t *r)
        spin_lock_irqsave(&c->lock, flags);
        idr_remove(&c->reqs, tag);
        spin_unlock_irqrestore(&c->lock, flags);
+       printk("tag %d removed from IDR\n", tag);
 }
 
 int p9_req_put(struct p9_client *c, struct p9_req_t *r)
 {
+       printk("p9_req_put req %p tag %d refcount before=%d\n",
+         r, r->tc.tag, refcount_read(&r->refcount));
        if (refcount_dec_and_test(&r->refcount)) {
                p9_tag_remove(c, r);
 

There is a significant difference.

If we use Shigeru's approach [1] (preventing the flush from being sent/processed using !fatal_signal_pending(current) in the condition), the original request (tag 0) is never freed from the IDR. It creates a definitive memory and tag leak:

[root@localhost]# ./repro
executing program
[   50.613200] p9_req_put req ff11000110157cb0 tag 65535 refcount before=3
[   50.663513] p9_req_put req ff11000110157cb0 tag 65535 refcount before=2
[   50.665156] p9_req_put req ff11000110157cb0 tag 65535 refcount before=1
[   50.666593] tag 65535 removed from IDR
executing program
[   50.688232] p9_req_put req ff11000110157ba0 tag 65535 refcount before=3
[   50.739634] p9_req_put req ff11000110157ba0 tag 65535 refcount before=2
[   50.741170] p9_req_put req ff11000110157ba0 tag 65535 refcount before=1
[   50.742587] tag 65535 removed from IDR
executing program
[   50.775708] p9_req_put req ff11000110157a90 tag 65535 refcount before=3
[   50.825104] p9_req_put req ff11000110157a90 tag 65535 refcount before=2
[   50.827719] p9_req_put req ff11000110157a90 tag 65535 refcount before=1
[   50.830432] tag 65535 removed from IDR
executing program
...

[1] https://lore.kernel.org/all/[email protected]/

However, with my patch (jumping to recalc_sigpending *inside* the P9_TFLUSH wait loop), the logic works cleanly due to the refcount mechanism:

[root@localhost]# ./repro
executing program
[   48.875093] p9_req_put req ff110001056fdcb0 tag 65535 refcount before=3
[   48.925981] p9_req_put req ff110001056fdba0 tag 0 refcount before=3
[   53.873969] p9_req_put req ff110001056fdba0 tag 0 refcount before=2
[   53.876638] p9_req_put req ff110001056fdcb0 tag 65535 refcount before=2
[   53.879882] p9_req_put req ff110001056fdba0 tag 0 refcount before=1
[   53.882505] tag 0 removed from IDR
[   53.884184] p9_req_put req ff110001056fdcb0 tag 65535 refcount before=1
[   53.886905] tag 65535 removed from IDR
executing program
[   53.914764] p9_req_put req ff1100010ca97a90 tag 65535 refcount before=3
[   53.967191] p9_req_put req ff1100010ca97980 tag 0 refcount before=3
[   58.909626] p9_req_put req ff1100010ca97980 tag 0 refcount before=2
[   58.912420] p9_req_put req ff1100010ca97a90 tag 65535 refcount before=2
[   58.915358] p9_req_put req ff1100010ca97980 tag 0 refcount before=1
[   58.918478] tag 0 removed from IDR
[   58.920730] p9_req_put req ff1100010ca97a90 tag 65535 refcount before=1
[   58.923715] tag 65535 removed from IDR
executing program
[   58.946905] p9_req_put req ff1100010c5a3870 tag 65535 refcount before=3
[   58.998260] p9_req_put req ff1100010c5a3760 tag 0 refcount before=3
[   63.945960] p9_req_put req ff1100010c5a3760 tag 0 refcount before=2
[   63.948549] p9_req_put req ff1100010c5a3870 tag 65535 refcount before=2
[   63.951153] p9_req_put req ff1100010c5a3760 tag 0 refcount before=1
[   63.953651] tag 0 removed from IDR
[   63.957254] p9_req_put req ff1100010c5a3870 tag 65535 refcount before=1
[   63.960655] tag 65535 removed from IDR
executing program
...

1. The dying thread breaks out of the loop and calls p9_req_put(). The refcount correctly drops from 2 to 1.

2. The tag is NOT removed from the IDR yet, preventing any Use-After-Free. The request is safely "orphaned".

3. Shortly after, when the parent process closes the pipe/socket (or the transport is torn down due to process exit), the transport layer cleans up the pending requests. It calls the final p9_req_put(), refcount hits 0, and the tag is properly removed from the IDR.

My logs clearly show refcount before=1 -> tag 0 removed from IDR upon transport teardown, proving there is no permanent leak when breaking out of the TFLUSH loop this way.

Since this safely resolves the permanent D-state hang during coredump_wait() without causing the UAF or leaks that plagued earlier kernels (pre-728356dedeff ("9p: Add refcount to p9_req_t")), do you think this pragmatic fix is acceptable for mainline and stable?

Quoted 2 lines

There was another patch last month, help is welcome, please see https://lore.kernel.org/r/[email protected]

While the ideal long-term goal is the asynchronous implementation (as seen in your 9p-async-v2 branch [2]), this patch serves as a reliable intermediate solution for a critical regression.

[2] https://github.com/martinetd/linux/commits/9p-async-v2

Quoted 2 lines

(By the way, what's with the rekindled interest on this?...)

The issue was identified by Syzkaller, and I have a stable C reproducer that consistently triggers the hang. There is also a similar report on Syzbot with a nearly identical reproducer:
https://syzkaller.appspot.com/text?tag=ReproC&x=156aa534580000

Signature · 2 lines
Thanks,
Vasiliy
#3 lore raw

Vasiliy Kovalev wrote on Thu, Apr 16, 2026 at 03:49:59PM +0300:

Quoted 93 lines
Quoted 1 line

It's unfortunately not that simple, just leaving here will leak things.

Following up on your concern about leaking things (as mentioned in the thread with Shigeru Yoshida's patch) - I did some tracing with printk inside p9_req_put and p9_tag_remove to see exactly what happens to the request lifecycle with my proposed patch vs Shigeru's approach:

net/9p/client.c +3 -0
index 748b92d3f0c1..4e8f4e0195b4 100644
@@ -274,10 +274,13 @@ static void p9_tag_remove(struct p9_client *c, struct
 p9_req_t *r)
        spin_lock_irqsave(&c->lock, flags);
        idr_remove(&c->reqs, tag);
        spin_unlock_irqrestore(&c->lock, flags);
+       printk("tag %d removed from IDR\n", tag);
 }
 
 int p9_req_put(struct p9_client *c, struct p9_req_t *r)
 {
+       printk("p9_req_put req %p tag %d refcount before=%d\n",
+         r, r->tc.tag, refcount_read(&r->refcount));
        if (refcount_dec_and_test(&r->refcount)) {
                p9_tag_remove(c, r);

There is a significant difference.

If we use Shigeru's approach [1] (preventing the flush from being sent/processed using !fatal_signal_pending(current) in the condition), the original request (tag 0) is never freed from the IDR. It creates a definitive memory and tag leak:

[root@localhost]# ./repro
executing program
[   50.613200] p9_req_put req ff11000110157cb0 tag 65535 refcount before=3
[   50.663513] p9_req_put req ff11000110157cb0 tag 65535 refcount before=2
[   50.665156] p9_req_put req ff11000110157cb0 tag 65535 refcount before=1
[   50.666593] tag 65535 removed from IDR
executing program
[   50.688232] p9_req_put req ff11000110157ba0 tag 65535 refcount before=3
[   50.739634] p9_req_put req ff11000110157ba0 tag 65535 refcount before=2
[   50.741170] p9_req_put req ff11000110157ba0 tag 65535 refcount before=1
[   50.742587] tag 65535 removed from IDR
executing program
[   50.775708] p9_req_put req ff11000110157a90 tag 65535 refcount before=3
[   50.825104] p9_req_put req ff11000110157a90 tag 65535 refcount before=2
[   50.827719] p9_req_put req ff11000110157a90 tag 65535 refcount before=1
[   50.830432] tag 65535 removed from IDR
executing program
...

[1]
https://lore.kernel.org/all/[email protected]/

However, with my patch (jumping to recalc_sigpending *inside* the P9_TFLUSH wait loop), the logic works cleanly due to the refcount mechanism:

[root@localhost]# ./repro
executing program
[   48.875093] p9_req_put req ff110001056fdcb0 tag 65535 refcount before=3
[   48.925981] p9_req_put req ff110001056fdba0 tag 0 refcount before=3
[   53.873969] p9_req_put req ff110001056fdba0 tag 0 refcount before=2
[   53.876638] p9_req_put req ff110001056fdcb0 tag 65535 refcount before=2
[   53.879882] p9_req_put req ff110001056fdba0 tag 0 refcount before=1
[   53.882505] tag 0 removed from IDR
[   53.884184] p9_req_put req ff110001056fdcb0 tag 65535 refcount before=1
[   53.886905] tag 65535 removed from IDR
executing program
[   53.914764] p9_req_put req ff1100010ca97a90 tag 65535 refcount before=3
[   53.967191] p9_req_put req ff1100010ca97980 tag 0 refcount before=3
[   58.909626] p9_req_put req ff1100010ca97980 tag 0 refcount before=2
[   58.912420] p9_req_put req ff1100010ca97a90 tag 65535 refcount before=2
[   58.915358] p9_req_put req ff1100010ca97980 tag 0 refcount before=1
[   58.918478] tag 0 removed from IDR
[   58.920730] p9_req_put req ff1100010ca97a90 tag 65535 refcount before=1
[   58.923715] tag 65535 removed from IDR
executing program
[   58.946905] p9_req_put req ff1100010c5a3870 tag 65535 refcount before=3
[   58.998260] p9_req_put req ff1100010c5a3760 tag 0 refcount before=3
[   63.945960] p9_req_put req ff1100010c5a3760 tag 0 refcount before=2
[   63.948549] p9_req_put req ff1100010c5a3870 tag 65535 refcount before=2
[   63.951153] p9_req_put req ff1100010c5a3760 tag 0 refcount before=1
[   63.953651] tag 0 removed from IDR
[   63.957254] p9_req_put req ff1100010c5a3870 tag 65535 refcount before=1
[   63.960655] tag 65535 removed from IDR
executing program
...

1. The dying thread breaks out of the loop and calls p9_req_put(). The refcount correctly drops from 2 to 1.

Quoted 7 lines

2. The tag is NOT removed from the IDR yet, preventing any Use-After-Free. The request is safely "orphaned".

3. Shortly after, when the parent process closes the pipe/socket (or the transport is torn down due to process exit), the transport layer cleans up the pending requests. It calls the final p9_req_put(), refcount hits 0, and the tag is properly removed from the IDR.

Thanks for taking the time to verify

So if I understand this right, with a sane server, the tag will be used until umount or a reply (to either the flush or the original request)?

That sounds sane enough (and definitely better then Shigeru's patch)

Quoted 4 lines

Since this safely resolves the permanent D-state hang during coredump_wait() without causing the UAF or leaks that plagued earlier kernels (pre-728356dedeff ("9p: Add refcount to p9_req_t")), do you think this pragmatic fix is acceptable for mainline and stable?

I'll need to find time (or someone) to perform some heavy tests with this -- basically my problem is that these hangs are easy to test with syzcaller but these are not ever supposed to happen with a real (correct) server, so while I agree not having stuck threads would be helpful to improve test efficiency I don't see it as a "real" problem (as in, 9p mount is a privileged operation, so normal users aren't supposed to mess around with a pipe/fd mount like syzcaller does)

Basically I'd want to "just" mess around with a real system on a 9p mount and play with ^C/sigkills and see what happens, but I don't have time for that short term, so this will likely be a few months

Quoted 4 lines

While the ideal long-term goal is the asynchronous implementation (as
seen in your 9p-async-v2 branch [2]), this patch serves as a reliable intermediate solution for a critical regression.
[2] https://github.com/martinetd/linux/commits/9p-async-v2

iirc one of the problem with the async branch is that the process would quit immediately on, say, ^C, before the IO has completed, but it's possible for the server to process the IO (and not the flush) afterwards and you'd get something that's not supposed to happen e.g.

p1 p2

write(1)
^C/sigkill
flush sent but process exit without waiting for server ack
1 not written yet
write(2) in same spot
write(2) done
write(1) completes
data isn't 2 as expected after p2 completed

So it's quite possible async isn't the way to go, but that there is no good solution for this
(given this is true even without async on sigkill: if we have something that works safely, there's no reason to wait only for non-fatal signals...)

Quoted 7 lines
Quoted 2 lines

(By the way, what's with the rekindled interest on this?...)

The issue was identified by Syzkaller, and I have a stable C reproducer that consistently triggers the hang. There is also a similar report on Syzbot with a nearly identical reproducer:
https://syzkaller.appspot.com/text?tag=ReproC&x=156aa534580000

Yes, it's been a known problem since syzcaller started testing 9p... many years ago.
Back then requests didn't have a refcount, so it's quite likely nobody looked hard enough after this became possible.

Thanks,

Signature · 1 line
Dominique Martinet | Asmadeus
#4 lore raw

On 4/17/26 01:52, Dominique Martinet wrote>

Quoted 2 lines

So if I understand this right, with a sane server, the tag will be used until umount or a reply (to either the flush or the original request)?

From what I tested — until a reply. Tested on latest master.

Added virtio-9p to QEMU cmdline:

-virtfs
local,path=/tmp/shared,mount_tag=p9host,security_model=none,id=p9host

Mounted in VM:

mount -t 9p -o trans=virtio,version=9p2000.L,access=any p9host
/mnt/p9real

Tested with and without patch:

  # SIGKILL
  dd if=/dev/zero of=/mnt/p9real/bigfile bs=1M count=500 &
  PID=$!; sleep 1 && kill -KILL $PID; wait $PID

  # SIGINT
  dd if=/dev/zero of=/mnt/p9real/bigfile bs=1M count=500 &
  PID=$!; sleep 1 && kill -INT $PID

SIGKILL — transport cleanup picks up the remaining ref:

p9_req_put req ff11000108aa8... tag 0 refcount before=2 p9_req_put req ff11000108aa8... tag 0 refcount before=2 tag 0 removed from IDR
[1]+ Killed dd ...

SIGINT — server replies and drops the last ref:

p9_req_put req ff1100010e073... tag 0 refcount before=2 p9_req_put req ff1100010e073... tag 0 refcount before=1 tag 0 removed from IDR
1+0 records in
[1]+ Interrupt dd ...

No hangs, no leaks, no regressions with or without patch in either case.

Syzkaller reproducer (trans=fd, no peer): without patch — permanent D-state, process never exits. With patch, each iteration completes cleanly and both tags are properly removed from IDR:

p9_req_put tag 65535 refcount 3->2->1, removed from IDR
p9_req_put tag 0 refcount 3->2->1, removed from IDR

Also verified forced kill of the reproducer — tags cleaned up correctly.

Severity is low and likely unreachable in production, but it slows down syzkaller — the hung process ties up a worker slot until the harness kills it by timeout (143s on our setup).

Signature · 2 lines
Thanks,
Vasiliy
#5 lore raw

Vasiliy Kovalev wrote on Sun, Apr 19, 2026 at 11:22:11AM +0300:

Quoted 5 lines

On 4/17/26 01:52, Dominique Martinet wrote>

Quoted 2 lines

So if I understand this right, with a sane server, the tag will be used until umount or a reply (to either the flush or the original request)?

From what I tested — until a reply. Tested on latest master.

Thanks for all the tests and following up

I'm still a little scared of this but I can't fault what you've found, I've picked this up for testing When I Find Time™ and will submit to Linus hopefully for next cycle if you don't hear back from me

Signature · 1 line
Dominique
#6 lore raw
In reply to Dominique Martinet

Dominique Martinet wrote on Fri, Apr 17, 2026 at 07:52:52AM +0900:

Quoted 22 lines
Quoted 4 lines

While the ideal long-term goal is the asynchronous implementation (as
seen in your 9p-async-v2 branch [2]), this patch serves as a reliable intermediate solution for a critical regression.
[2] https://github.com/martinetd/linux/commits/9p-async-v2

iirc one of the problem with the async branch is that the process would quit immediately on, say, ^C, before the IO has completed, but it's possible for the server to process the IO (and not the flush) afterwards and you'd get something that's not supposed to happen e.g.

p1 p2

write(1)
^C/sigkill
flush sent but process exit without waiting for server ack
1 not written yet
write(2) in same spot
write(2) done
write(1) completes
data isn't 2 as expected after p2 completed

Quoted 4 lines

So it's quite possible async isn't the way to go, but that there is no good solution for this
(given this is true even without async on sigkill: if we have something that works safely, there's no reason to wait only for non-fatal signals...)

Sorry to come back to this after two months but I'm still a bit worried about this patch, and just came back to it as I'm about to send the PR to Linus...
And I'm still thinking about the problem above, or rather possible variants involving cache (e.g. write going through the server, but client believing it didn't because the response didn't make it in time)

.. But the thing is, I couldn't actually hit the if (fatal_signal_pending(current)) you added (adding some print statement):
- if cache is enabled, the actual I/Os are done by the vfs in the background, so any kill to user processes won't have any impact (and thus I guess my main worry about cache is alleviated there)
- with cache=none I'm not sure why I can't hit it, I tried with an external server, breaking on the write() call while running dd, and killing dd with SIGKILL a few times but that doesn't appear to be enough? (task still stuck in write > rpc > flush > rpc, but it doesn't appear to ever get out of io_wait_event_killable() even when I hammer it with more signals?)

So, given that my worry with cache is irrelevant (runs in background & won't ever hit this), I can't seem to hit this with what I consider to be normal workloads, and assuming it does fix your problems given you were able to test it... I'll leave it in and send to Linus now but I'd appreciate clarifications on how to test this more thoroughly as time permits...
(I honestly probably should drop the patch at this point, but it'll still be time to revert if I figure something out in the next few weeks given it's been in -next for almost 2 months already)

Thanks,

Signature · 1 line
Dominique Martinet
#7 lore raw
Quoted 58 lines · Dominique Martinet

Dominique Martinet wrote on Fri, Apr 17, 2026 at 07:52:52AM +0900:

Quoted 22 lines
Quoted 4 lines

While the ideal long-term goal is the asynchronous implementation (as
seen in your 9p-async-v2 branch [2]), this patch serves as a reliable intermediate solution for a critical regression.
[2] https://github.com/martinetd/linux/commits/9p-async-v2

iirc one of the problem with the async branch is that the process would quit immediately on, say, ^C, before the IO has completed, but it's possible for the server to process the IO (and not the flush) afterwards and you'd get something that's not supposed to happen e.g.

p1 p2

write(1)
^C/sigkill
flush sent but process exit without waiting for server ack
1 not written yet
write(2) in same spot
write(2) done
write(1) completes
data isn't 2 as expected after p2 completed

Quoted 4 lines

So it's quite possible async isn't the way to go, but that there is no good solution for this
(given this is true even without async on sigkill: if we have something that works safely, there's no reason to wait only for non-fatal signals...)

Sorry to come back to this after two months but I'm still a bit worried about this patch, and just came back to it as I'm about to send the PR to Linus...
And I'm still thinking about the problem above, or rather possible variants involving cache (e.g. write going through the server, but client believing it didn't because the response didn't make it in time)

.. But the thing is, I couldn't actually hit the if (fatal_signal_pending(current)) you added (adding some print statement):
- if cache is enabled, the actual I/Os are done by the vfs in the background, so any kill to user processes won't have any impact (and thus I guess my main worry about cache is alleviated there)
- with cache=none I'm not sure why I can't hit it, I tried with an external server, breaking on the write() call while running dd, and killing dd with SIGKILL a few times but that doesn't appear to be enough? (task still stuck in write > rpc > flush > rpc, but it doesn't appear to ever get out of io_wait_event_killable() even when I hammer it with more signals?)

So, given that my worry with cache is irrelevant (runs in background & won't ever hit this), I can't seem to hit this with what I consider to be normal workloads, and assuming it does fix your problems given you were able to test it... I'll leave it in and send to Linus now but I'd appreciate clarifications on how to test this more thoroughly as time permits...
(I honestly probably should drop the patch at this point, but it'll still be time to revert if I figure something out in the next few weeks given it's been in -next for almost 2 months already)

Quoted 1 line

Thanks,

Quoting myself from April: "Severity is low and likely unreachable in production, but it slows down syzkaller — the hung process ties up a worker slot until the harness kills it by timeout (143s on our setup)."

The deterministic path is the syzkaller C reproducer:
https://syzkaller.appspot.com/x/repro.c?x=156aa534580000

What it does:
1) mounts 9p with trans=fd, rfdno/wfdno pointing to open fds with nothing speaking the 9p protocol on the other side
- RFLUSH can never arrive;
2) the 9p rpc from mount parks a thread in io_wait_event_killable;
3) another thread triggers SIGSEGV via prctl(PR_SET_MM) + brk()

     corruption -> coredump_wait;

4) the harness's kill_and_wait() fires 5s later.

To make both branches visible, debug diff on top of the patch:

net/9p/client.c +5 -1
@@ -600,8 +600,12 @@ p9_client_rpc(...)
        if (err == -ERESTARTSYS && c->status == Connected &&
            type == P9_TFLUSH) {
-               if (fatal_signal_pending(current))
+               pr_info("9p-dbg: TFLUSH retry hit, fatal=%d\n",
+                       fatal_signal_pending(current));
+               if (fatal_signal_pending(current)) {
+                       pr_info("9p-dbg: bailing out via recalc_sigpending\n");
                        goto recalc_sigpending;
+               }
                sigpending = 1;
                clear_thread_flag(TIF_SIGPENDING);
                goto again;
 

In the VM:

  # gcc repro.c -o repro
  # ./repro

dmesg fires on every iteration:

[root@localhost repro]# ./repro
executing program

[  126.254054] repro[363]: segfault at 558a42e9ff30 ip 0000558a42e9ff30 sp 00007f3225ee4e80 error 14 likely on CPU 0 (core 0, socket 0)
[  126.258095] Code: Unable to access opcode bytes at 0x558a42e9ff06.
[  131.199937] 9pnet: 9p-dbg: TFLUSH retry hit, fatal=1
[  131.201868] 9pnet: 9p-dbg: bailing out via recalc_sigpending
executing program
[  131.270955] repro[366]: segfault at 558a42e9ff30 ip 0000558a42e9ff30 sp 00007f3225ee4e80 error 14 likely on CPU 3 (core 3, socket 0)
[  131.275131] Code: Unable to access opcode bytes at 0x558a42e9ff06.
[  136.219066] 9pnet: 9p-dbg: TFLUSH retry hit, fatal=1
[  136.221359] 9pnet: 9p-dbg: bailing out via recalc_sigpending
executing program
[  136.290772] repro[369]: segfault at 558a42e9ff30 ip 0000558a42e9ff30 sp 00007f3225ee4e80 error 14 likely on CPU 2 (core 2, socket 0)
[  136.295901] Code: Unable to access opcode bytes at 0x558a42e9ff06.
[  141.237955] 9pnet: 9p-dbg: TFLUSH retry hit, fatal=1
[  141.239800] 9pnet: 9p-dbg: bailing out via recalc_sigpending

executing program
...

Without the patch the second pr_info never appears and the task hangs in D-state.

On a real server I couldn't reproduce this by hand. The reproducer hits the branch deterministically (logs above); why hand-issued SIGKILLs don't get there is a kernel signal-delivery question outside the path this patch touches, and I didn't dig into it.

Feel free to revert if anything turns up in the next weeks.

Signature · 2 lines
Thanks,
Vasiliy
#8 lore raw

Vasiliy Kovalev wrote on Mon, Jun 22, 2026 at 01:06:21AM +0300:

Quoted 6 lines

The deterministic path is the syzkaller C reproducer:
https://syzkaller.appspot.com/x/repro.c?x=156aa534580000

[...]

dmesg fires on every iteration:

Thanks for the quick reply, I've confirmed this repro "works", as in gets out of the loop as expected.

Quoted 4 lines

On a real server I couldn't reproduce this by hand. The reproducer hits the branch deterministically (logs above); why hand-issued SIGKILLs don't get there is a kernel signal-delivery question outside the path this patch touches, and I didn't dig into it.

Right, from my added logs, when doing it "by hand", I don't think the thread goes out of the wait_event_killable() the second time around.

I agree it's slightly out of scope, but any behaviour we don't understand is something that could be subject to change, so I'll try to dig a bit further as time allows.

Quoted 1 line

Feel free to revert if anything turns up in the next weeks.

Thanks, for now Linus took the patch, I'll let you know if there is any change.

Signature · 1 line
Dominique Martinet | Asmadeus