Skip to contentSkip to conversation

liveupdate: suppress TCP RST during post-kexec restore window

PATCH In review Li Chen 2 participants 8 messages Jan 30 – Feb 3

Subsystems (5)
lore
Share & tools
lore

Li Chen

author
#0 lore raw

cover liveupdate: suppress TCP RST during post-kexec restore window

LUO supports kexec-based live update where userspace restores preserved state after the new kernel boots. For established TCP connections that are restored in userspace (e.g. CRIU), there is an unavoidable window where the new kernel has no socket yet but the peer can still send packets. The TCP no-socket path replies with a RST to segments with ACK set, causing the peer to immediately tear down the connection.

This series tracks a bounded "restore window" in which incoming LUO sessions are expected to be retrieved and finished. The window ends automatically when all incoming sessions are retrieved and finished (or their session file descriptors are closed), can be terminated early via the global LIVEUPDATE_IOCTL_RESTORE_DONE, and is also bounded by a hard timeout (15 minutes) so it cannot remain open indefinitely.

On top of that, an optional cmdline knob, liveupdate_tcp_rst_suppress=on, drops established (ACK && !SYN) segments that would otherwise trigger a RST while the restore window is in progress. Default is off and requires liveupdate=on.

Tested with an established TCP stream across LUO kexec + CRIU restore; calling RESTORE_DONE after restore avoids TCP RST and the connection continues.

Li Chen (3):
liveupdate: track post-kexec restore window
liveupdate: bound and control post-kexec restore window
liveupdate: suppress TCP RST during post-kexec restore window

 .../admin-guide/kernel-parameters.txt         | 10 +++
 include/linux/liveupdate.h                    | 26 ++++++
 include/uapi/linux/liveupdate.h               | 34 ++++++++
 kernel/liveupdate/luo_core.c                  | 31 +++++++
 kernel/liveupdate/luo_internal.h              |  5 ++
 kernel/liveupdate/luo_session.c               | 85 ++++++++++++++++++-
 net/ipv4/tcp_ipv4.c                           |  5 ++
 net/ipv6/tcp_ipv6.c                           |  5 ++
 8 files changed, 198 insertions(+), 3 deletions(-)
Signature · 1 line
2.52.0
author
#1 lore raw

PATCH 1/3 liveupdate: track post-kexec restore window

A kexec-based live update introduces a window after the new kernel boots where userspace needs to retrieve and restore preserved sessions.

Provide liveupdate_restore_in_progress() backed by a counter of incoming sessions left, and make session finishing idempotent to avoid double finish paths causing counter underflow.

---
 include/linux/liveupdate.h       | 11 +++++++++
 kernel/liveupdate/luo_core.c     |  2 ++
 kernel/liveupdate/luo_internal.h |  4 ++++
 kernel/liveupdate/luo_session.c  | 41 +++++++++++++++++++++++++++++++-
 4 files changed, 57 insertions(+), 1 deletion(-)
include/linux/liveupdate.h +11 -0
index ed81e7b31a9f..406a6e2dd4a1 100644
@@ -217,6 +217,12 @@ struct liveupdate_flb {
 /* Return true if live update orchestrator is enabled */
 bool liveupdate_enabled(void);
 
+/*
+ * Return true during a kexec-based live update boot while userspace is still
+ * restoring preserved sessions/resources.
+ */
+bool liveupdate_restore_in_progress(void);
+
 /* Called during kexec to tell LUO that entered into reboot */
 int liveupdate_reboot(void);
 
@@ -238,6 +244,11 @@ static inline bool liveupdate_enabled(void)
 	return false;
 }
 
+static inline bool liveupdate_restore_in_progress(void)
+{
+	return false;
+}
+
 static inline int liveupdate_reboot(void)
 {
 	return 0;
kernel/liveupdate/luo_core.c +2 -0
index 7a9ef16b37d8..19c91843fbdb 100644
@@ -128,6 +128,8 @@ static int __init luo_early_startup(void)
 	if (err)
 		return err;
 
+	luo_session_restore_window_init();
+
 	err = luo_flb_setup_incoming(luo_global.fdt_in);
 
 	return err;
kernel/liveupdate/luo_internal.h +4 -0
index 6115d6a4054d..8aa4c5b0101b 100644
@@ -72,6 +72,8 @@ struct luo_file_set {
  *              previous kernel) sessions.
  * @retrieved:  A boolean flag indicating whether this session has been
  *              retrieved by a consumer in the new kernel.
+ * @finished:   A boolean flag indicating whether this session has been
+ *              successfully finished in the new kernel.
  * @file_set:   A set of files that belong to this session.
  * @mutex:      protects fields in the luo_session.
  */
@@ -80,6 +82,7 @@ struct luo_session {
 	struct luo_session_ser *ser;
 	struct list_head list;
 	bool retrieved;
+	bool finished;
 	struct luo_file_set file_set;
 	struct mutex mutex;
 };
@@ -88,6 +91,7 @@ int luo_session_create(const char *name, struct file **filep);
 int luo_session_retrieve(const char *name, struct file **filep);
 int __init luo_session_setup_outgoing(void *fdt);
 int __init luo_session_setup_incoming(void *fdt);
+void __init luo_session_restore_window_init(void);
 int luo_session_serialize(void);
 int luo_session_deserialize(void);
 bool luo_session_quiesce(void);
kernel/liveupdate/luo_session.c +40 -1

60 lines not on this page · raw message

Signature · 1 line
2.52.0
author
#2 lore raw

PATCH 2/3 liveupdate: bound and control post-kexec restore window

In reply to Li Chen

The restore window can remain open indefinitely if userspace never finishes some incoming sessions. Bound the window with a hard timeout and add a RESTORE_DONE ioctl on /dev/liveupdate so an orchestrator can end it explicitly once restoration is complete.

---
 include/linux/liveupdate.h       |  4 +++
 include/uapi/linux/liveupdate.h  | 34 ++++++++++++++++++
 kernel/liveupdate/luo_core.c     | 15 ++++++++
 kernel/liveupdate/luo_internal.h |  1 +
 kernel/liveupdate/luo_session.c  | 59 ++++++++++++++++++++++++++------
 5 files changed, 103 insertions(+), 10 deletions(-)
include/linux/liveupdate.h +4 -0
index 406a6e2dd4a1..301d3e94516e 100644
@@ -220,6 +220,10 @@ bool liveupdate_enabled(void);
 /*
  * Return true during a kexec-based live update boot while userspace is still
  * restoring preserved sessions/resources.
+ *
+ * The restore window ends automatically once all incoming sessions have been
+ * retrieved and finished. Userspace can also terminate the window explicitly
+ * (and early) via LIVEUPDATE_IOCTL_RESTORE_DONE.
  */
 bool liveupdate_restore_in_progress(void);
 
include/uapi/linux/liveupdate.h +34 -0
index 30bc66ee9436..357137d9a78c 100644
@@ -51,6 +51,7 @@ enum {
 	LIVEUPDATE_CMD_BASE = 0x00,
 	LIVEUPDATE_CMD_CREATE_SESSION = LIVEUPDATE_CMD_BASE,
 	LIVEUPDATE_CMD_RETRIEVE_SESSION = 0x01,
+	LIVEUPDATE_CMD_RESTORE_DONE = 0x02,
 };
 
 /* ioctl commands for session file descriptors */
@@ -118,6 +119,39 @@ struct liveupdate_ioctl_retrieve_session {
 #define LIVEUPDATE_IOCTL_RETRIEVE_SESSION \
 	_IO(LIVEUPDATE_IOCTL_TYPE, LIVEUPDATE_CMD_RETRIEVE_SESSION)
 
+/**
+ * struct liveupdate_ioctl_restore_done - ioctl(LIVEUPDATE_IOCTL_RESTORE_DONE)
+ * @size:     Input; sizeof(struct liveupdate_ioctl_restore_done)
+ * @reserved: Input; Must be zero. Reserved for future use.
+ *
+ * Marks the completion of the post-kexec restore window.
+ *
+ * After a live update (kexec), userspace needs time to restore preserved
+ * sessions/resources. Some kernel subsystems may apply temporary compatibility
+ * behavior during this window. Userspace should call this ioctl once it has
+ * completed restoration and wants normal kernel behavior to resume, even if
+ * some incoming sessions are left unused.
+ *
+ * This ioctl updates global state and is not tied to a specific live update
+ * session file descriptor. A typical userspace agent calls it once per live
+ * update, after it has restored the required state.
+ *
+ * Note that the restore window may also end automatically once all incoming
+ * sessions have been retrieved and finished (or their session file
+ * descriptors have been closed). This ioctl is intended for cases where an
+ * agent intentionally does not retrieve all incoming sessions or does not want
+ * to wait for them to be finished/closed.
+ *
+ * Return: 0 on success, negative error code on failure.
+ */
+struct liveupdate_ioctl_restore_done {
+	__u32		size;
+	__u32		reserved;
+};
+
+#define LIVEUPDATE_IOCTL_RESTORE_DONE \
+	_IO(LIVEUPDATE_IOCTL_TYPE, LIVEUPDATE_CMD_RESTORE_DONE)
+
 /* Session specific IOCTLs */
 
 /**
kernel/liveupdate/luo_core.c +15 -0
index 19c91843fbdb..fb6a73c08979 100644
@@ -340,6 +340,18 @@ static int luo_ioctl_retrieve_session(struct luo_ucmd *ucmd)
 	return err;
 }
 
+static int luo_ioctl_restore_done(struct luo_ucmd *ucmd)
+{
+	struct liveupdate_ioctl_restore_done *argp = ucmd->cmd;
+
+	if (argp->reserved)
+		return -EINVAL;
+
+	luo_session_restore_done();
+
+	return luo_ucmd_respond(ucmd, sizeof(*argp));
+}
+
 static int luo_open(struct inode *inodep, struct file *filep)
 {
 	struct luo_device_state *ldev = container_of(filep->private_data,
@@ -371,6 +383,7 @@ static int luo_release(struct inode *inodep, struct file *filep)
 union ucmd_buffer {
 	struct liveupdate_ioctl_create_session create;
 	struct liveupdate_ioctl_retrieve_session retrieve;
+	struct liveupdate_ioctl_restore_done restore_done;
 };
 
 struct luo_ioctl_op {
@@ -395,6 +408,8 @@ static const struct luo_ioctl_op luo_ioctl_ops[] = {
 		 struct liveupdate_ioctl_create_session, name),
 	IOCTL_OP(LIVEUPDATE_IOCTL_RETRIEVE_SESSION, luo_ioctl_retrieve_session,
 		 struct liveupdate_ioctl_retrieve_session, name),
+	IOCTL_OP(LIVEUPDATE_IOCTL_RESTORE_DONE, luo_ioctl_restore_done,
+		 struct liveupdate_ioctl_restore_done, reserved),
 };
 
 static long luo_ioctl(struct file *filep, unsigned int cmd, unsigned long arg)
kernel/liveupdate/luo_internal.h +1 -0
index 8aa4c5b0101b..03a5b27498be 100644
@@ -96,6 +96,7 @@ int luo_session_serialize(void);
 int luo_session_deserialize(void);
 bool luo_session_quiesce(void);
 void luo_session_resume(void);
+void luo_session_restore_done(void);
 
 int luo_preserve_file(struct luo_file_set *file_set, u64 token, int fd);
 void luo_file_unpreserve_files(struct luo_file_set *file_set);
kernel/liveupdate/luo_session.c +49 -10

100 lines not on this page · raw message

Signature · 1 line
2.52.0
author
#3 lore raw

PATCH 3/3 liveupdate: suppress TCP RST during post-kexec restore window

In reply to Li Chen

During a kexec-based live update, userspace may restore established TCP connections after the new kernel has booted (e.g. via CRIU). Any packet arriving for a not-yet-restored socket will hit the no-socket path and trigger a TCP RST, causing the peer to immediately drop the connection. Add an optional cmdline knob, liveupdate_tcp_rst_suppress=, to drop such packets while liveupdate_restore_in_progress() is true. Only segments with ACK set and SYN clear are dropped, and the default behavior remains unchanged.
Document the liveupdate_tcp_rst_suppress cmdline parameter.

---
 Documentation/admin-guide/kernel-parameters.txt | 10 ++++++++++
 include/linux/liveupdate.h                      | 11 +++++++++++
 kernel/liveupdate/luo_core.c                    | 14 ++++++++++++++
 kernel/liveupdate/luo_session.c                 |  1 +
 net/ipv4/tcp_ipv4.c                             |  5 +++++
 net/ipv6/tcp_ipv6.c                             |  5 +++++
 6 files changed, 46 insertions(+)
Documentation/admin-guide/kernel-parameters.txt +10 -0
index 3097e4266d76..b73347a0aefd 100644
@@ -3442,6 +3442,16 @@ Kernel parameters
 			If there are multiple matching configurations changing
 			the same attribute, the last one is used.
 
+	liveupdate_tcp_rst_suppress=	[KNL,EARLY]
+			Format: <bool>
+			When enabled, drop packets for established connections
+			(ACK set, SYN clear) that would otherwise trigger a RST
+			in the LUO post-kexec restore window.
+			This is useful when userspace restores sockets after
+			kexec (e.g. via CRIU).
+			Requires liveupdate=on.
+			Default: off.
+
 	lockd.nlm_grace_period=P  [NFS] Assign grace period.
 			Format: <integer>
 
include/linux/liveupdate.h +11 -0
index 301d3e94516e..6ca740ec19d4 100644
@@ -227,6 +227,12 @@ bool liveupdate_enabled(void);
  */
 bool liveupdate_restore_in_progress(void);
 
+/*
+ * Return true when TCP RST suppression is enabled for the post-kexec restore
+ * window.
+ */
+bool liveupdate_tcp_rst_suppress_enabled(void);
+
 /* Called during kexec to tell LUO that entered into reboot */
 int liveupdate_reboot(void);
 
@@ -253,6 +259,11 @@ static inline bool liveupdate_restore_in_progress(void)
 	return false;
 }
 
+static inline bool liveupdate_tcp_rst_suppress_enabled(void)
+{
+	return false;
+}
+
 static inline int liveupdate_reboot(void)
 {
 	return 0;
kernel/liveupdate/luo_core.c +14 -0
index fb6a73c08979..0ed5c9ce1421 100644
@@ -64,6 +64,7 @@
 
 static struct {
 	bool enabled;
+	bool tcp_rst_suppress;
 	void *fdt_out;
 	void *fdt_in;
 	u64 liveupdate_num;
@@ -75,6 +76,13 @@ static int __init early_liveupdate_param(char *buf)
 }
 early_param("liveupdate", early_liveupdate_param);
 
+static int __init early_liveupdate_tcp_rst_suppress_param(char *buf)
+{
+	return kstrtobool(buf, &luo_global.tcp_rst_suppress);
+}
+early_param("liveupdate_tcp_rst_suppress",
+	    early_liveupdate_tcp_rst_suppress_param);
+
 static int __init luo_early_startup(void)
 {
 	phys_addr_t fdt_phys;
@@ -259,6 +267,12 @@ bool liveupdate_enabled(void)
 	return luo_global.enabled;
 }
 
+bool liveupdate_tcp_rst_suppress_enabled(void)
+{
+	return liveupdate_enabled() && luo_global.tcp_rst_suppress;
+}
+EXPORT_SYMBOL_GPL(liveupdate_tcp_rst_suppress_enabled);
+
 /**
  * DOC: LUO ioctl Interface
  *
kernel/liveupdate/luo_session.c +1 -0
index 2c7dd3b12303..427ae74061ba 100644
@@ -146,6 +146,7 @@ bool liveupdate_restore_in_progress(void)
 {
 	return atomic_long_read(&liveupdate_incoming_sessions_left) > 0;
 }
+EXPORT_SYMBOL_GPL(liveupdate_restore_in_progress);
 
 void __init luo_session_restore_window_init(void)
 {
net/ipv4/tcp_ipv4.c +5 -0
index f8a9596e8f4d..9a95f3dbf39a 100644
@@ -56,6 +56,7 @@
 #include <linux/fips.h>
 #include <linux/jhash.h>
 #include <linux/init.h>
+#include <linux/liveupdate.h>
 #include <linux/times.h>
 #include <linux/slab.h>
 #include <linux/sched.h>
@@ -2349,6 +2350,10 @@ int tcp_v4_rcv(struct sk_buff *skb)
 bad_packet:
 		__TCP_INC_STATS(net, TCP_MIB_INERRS);
 	} else {
+		if (liveupdate_tcp_rst_suppress_enabled() &&
+		    liveupdate_restore_in_progress() &&
+		    th->ack && !th->syn)
+			goto discard_it;
 		tcp_v4_send_reset(NULL, skb, sk_rst_convert_drop_reason(drop_reason));
 	}
 
net/ipv6/tcp_ipv6.c +5 -0
index 280fe5978559..c2e680eba041 100644
@@ -40,6 +40,7 @@
 #include <linux/icmpv6.h>
 #include <linux/random.h>
 #include <linux/indirect_call_wrapper.h>
+#include <linux/liveupdate.h>
 
 #include <net/aligned_data.h>
 #include <net/tcp.h>
@@ -1900,6 +1901,10 @@ INDIRECT_CALLABLE_SCOPE int tcp_v6_rcv(struct sk_buff *skb)
 bad_packet:
 		__TCP_INC_STATS(net, TCP_MIB_INERRS);
 	} else {
+		if (liveupdate_tcp_rst_suppress_enabled() &&
+		    liveupdate_restore_in_progress() &&
+		    th->ack && !th->syn)
+			goto discard_it;
 		tcp_v6_send_reset(NULL, skb, sk_rst_convert_drop_reason(drop_reason));
 	}
 
Signature · 1 line
2.52.0
maintainer
#4 lore raw
Quoted 4 lines · On Fri, 30 Jan 2026 22:51:19 +0800 Li Chen wrote:

During a kexec-based live update, userspace may restore established TCP connections after the new kernel has booted (e.g. via CRIU). Any packet arriving for a not-yet-restored socket will hit the no-socket path and trigger a TCP RST, causing the peer to immediately drop the connection.

Can you not add a filter to simply drop those packets until workload is running again? It'd actually be less racy than this hac^w patch ...

author
#5 lore raw

Hi Jakub,

Quoted 9 lines · On Fri, 30 Jan 2026 22:51:19 +0800 Li Chen wrote:
Quoted 4 lines

During a kexec-based live update, userspace may restore established TCP connections after the new kernel has booted (e.g. via CRIU). Any packet arriving for a not-yet-restored socket will hit the no-socket path and trigger a TCP RST, causing the peer to immediately drop the connection.

Can you not add a filter to simply drop those packets until workload is running again? It'd actually be less racy than this hac^w patch ...

Thanks for the suggestion.

When you say "add a filter", do you mean installing a temporary drop rule (nftables/iptables/tc) in the network domain which does not get rebooted by kexec (e.g. LB/ToR/host firewall), so packets never reach the new kernel until the workload is restored and ready?

If you meant a filter inside the kexec'ed kernel, I'm worried it won't cover the critical window: kexec resets the ruleset, so we'd have to install the drop rule extremely early (initramfs) before any packets hit the no-socket path, which still seems inherently racy.

If the expectation is to drain/blackhole traffic externally and re-enable it once the workload is running again, I can rework the series to keep only the restore-window tracking plus a clear "restore done" control plane, and rely on the external filter for the data plane.

Regards
Li

maintainer
#6 lore raw
Quoted 17 lines · On Sun, 01 Feb 2026 09:44:27 +0800 Li Chen wrote:
Quoted 9 lines · On Fri, 30 Jan 2026 22:51:19 +0800 Li Chen wrote:
Quoted 4 lines

During a kexec-based live update, userspace may restore established TCP connections after the new kernel has booted (e.g. via CRIU). Any packet arriving for a not-yet-restored socket will hit the no-socket path and trigger a TCP RST, causing the peer to immediately drop the connection.

Can you not add a filter to simply drop those packets until workload is running again? It'd actually be less racy than this hac^w patch ...

Thanks for the suggestion.

When you say "add a filter", do you mean installing a temporary drop rule (nftables/iptables/tc) in the network domain which does not get rebooted by kexec (e.g. LB/ToR/host firewall), so packets never reach the new kernel until the workload is restored and ready?

Quoted 4 lines

If you meant a filter inside the kexec'ed kernel, I'm worried it won't cover the critical window: kexec resets the ruleset, so we'd have to install the drop rule extremely early (initramfs) before any packets hit the no-socket path, which still seems inherently racy.

I'm not sure what your flow is exactly, but I assume you drive the workload restore from user space already?

author
#7 lore raw

Hi Jakub,

---- On Tue, 03 Feb 2026 08:53:20 +0800 Jakub Kicinski <[email protected]> wrote ---

Quoted 18 lines · On Sun, 01 Feb 2026 09:44:27 +0800 Li Chen wrote:
Quoted 17 lines
Quoted 9 lines · On Fri, 30 Jan 2026 22:51:19 +0800 Li Chen wrote:
Quoted 4 lines

During a kexec-based live update, userspace may restore established TCP connections after the new kernel has booted (e.g. via CRIU). Any packet arriving for a not-yet-restored socket will hit the no-socket path and trigger a TCP RST, causing the peer to immediately drop the connection.

Can you not add a filter to simply drop those packets until workload is running again? It'd actually be less racy than this hac^w patch ...

Thanks for the suggestion.

When you say "add a filter", do you mean installing a temporary drop rule (nftables/iptables/tc) in the network domain which does not get rebooted by kexec (e.g. LB/ToR/host firewall), so packets never reach the new kernel until the workload is restored and ready?

Quoted 8 lines
Quoted 4 lines

If you meant a filter inside the kexec'ed kernel, I'm worried it won't cover the critical window: kexec resets the ruleset, so we'd have to install the drop rule extremely early (initramfs) before any packets hit the no-socket path, which still seems inherently racy.

I'm not sure what your flow is exactly, but I assume you drive the workload restore from user space already?

Yes, in our PoC setup the post-kexec restore flow is driven from initramfs / early userspace.

We pass an initramfs via kexec --initrd and install a temporary iptables INPUT DROP rule from a dracut pre-mount hook (keyed by a cmdline like luo_tcp_drop_port=...). In our external-peer test this avoids the early TCP RST window; the peer just retransmits/timeouts until CRIU restore recreates the socket.

The downside is that it makes initramfs heavier (iptables userspace + required xtables extensions, and it relies on legacy iptables filter support being available early). Not sure this is a great general solution, but it can work when initramfs is under our control.

Regards,
Li​