Skip to contentSkip to conversation

xenconsole: Add connected flag

PATCH Superseded Jason Andryuk 3 participants 7 messages Aug 7, 2025 – Aug 19, 2025

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xenconsole: Add connected flag

With hyperlaunch, a domU can start before its console ring is connected by xenconsoled. With nothing emptying the ring, it can quickly fill during boot. In domU_write_console(), __write_console() returns 0 when the ring is full. This loops spins until xenconsoled starts emptying the ring:

        while (len) {
                ssize_t sent = __write_console(cons, data, len);

                if (sent < 0)
                        return sent;

                data += sent;
                len -= sent;

                if (unlikely(len))
                        HYPERVISOR_sched_op(SCHEDOP_yield, NULL);
        }

The goal of this patch is to add a way for the frontend to know when a console is connected. This patch adds a new flag to the end of the console ring structure. It is used for the backend to indicate that it has connected and started servicing the page.

The two values are
XENCONSOLE_DISCONNECTED 1
XENCONSOLE_CONNECTED 0

XENCONSOLE_DISCONNECTED indicates to the guest that ring is disconnected, so it will not be serviced. The guest can avoid writing into it in that case. A domU can use console hypercalls and only transition to the ring when it is connected and won't fill and block.

Once the backend (xenconsoled) maps and starts servicing the console, the flag will be set to XENCONSOLE_CONNECTED (0) to indicate the backend state to the frontend.

The connected value as 0 will be match the default of a zero-ed console page. Hyperlaunch can set the flag to XENCONSOLE_DISCONNECTED and let xenconsoled set to XENCONSOLE_CONNECTED.

Old domU hvc_xen drivers won't check the flag.
New domU hvc_xen running on a new xen/xenconsoled will work properly. New domU hvc_xen on old xen/xenconsoled should only see a 0 for the flag and behave as if connected.

---
v1:
Remove evtchn notify call
Set connected later when there is no error

RFC v3:
Flip flag values so 0 is connected.

The other option would be to add:
uint32_t features
uint32_t connected

New domUs would check features for a magic value and/or flag to know they can rely on connected transitioning.

I think making XENCONSOLE_CONNECTED == 0 side steps the need for an additional features field. As long as assuming zero-ed memory is acceptable. However, this only matters for a hyperlaunched guest - xenconsoled will normally readily connect the console and set the value.

This assumes that existing frontends are not using the flag space for some other use.

Removed idea:
Send an event channel notification to let the domU know that xenconsoled is connected. Xenstored does similar, but for xenstore, the xenstore driver owns the event channel/irq and can rebind it. For hvc_xen, the hvc subsystem owns the irq, so it isn't readily available for rebinding. This is not implemented.

I had the idea for the kernel to use a static key and switch writing from the hypercall to the PV ring once connected. It didn't actually work in my short attempt - I think changing the static key from within an interupt was wrong. I fell back to just checking the flag directly without an optimization. That would work and would make the event channel notification unnecessary.

---
 tools/console/daemon/io.c       |  4 ++++
 xen/include/public/io/console.h | 13 +++++++++++++
 2 files changed, 17 insertions(+)
tools/console/daemon/io.c +4 -0
index bb739bdb8c..c278752ff1 100644
@@ -781,6 +781,10 @@ static int console_create_ring(struct console *con)
 		con->log_fd = create_console_log(con);
 
  out:
+	/* Mark the console connected. */
+	if (!err && con->interface)
+		con->interface->flag = XENCONSOLE_CONNECTED;
+
 	return err;
 }
 
xen/include/public/io/console.h +13 -0
index 4509b4b689..c16ef42ef4 100644
@@ -19,6 +19,19 @@ struct xencons_interface {
     char out[2048];
     XENCONS_RING_IDX in_cons, in_prod;
     XENCONS_RING_IDX out_cons, out_prod;
+/*
+ * Flag values signaling from backend to frontend whether the console is
+ * connected.  i.e. Whether it will be serviced and emptied.
+ *
+ * The flag starts as disconnected.
+ */
+#define XENCONSOLE_DISCONNECTED 1
+/*
+ * The flag is set to connected when the backend connects and the console
+ * will be serviced.
+ */
+#define XENCONSOLE_CONNECTED    0
+    uint32_t flag;
 };
 
 #ifdef XEN_WANT_FLEX_CONSOLE_RING
Signature · 1 line
2.50.1
#1 lore raw
Quoted 33 lines · Jason Andryuk

With hyperlaunch, a domU can start before its console ring is connected by xenconsoled. With nothing emptying the ring, it can quickly fill during boot. In domU_write_console(), __write_console() returns 0 when the ring is full. This loops spins until xenconsoled starts emptying the ring:

         while (len) {
                 ssize_t sent = __write_console(cons, data, len);

                 if (sent < 0)
                         return sent;

                 data += sent;
                 len -= sent;

                 if (unlikely(len))
                         HYPERVISOR_sched_op(SCHEDOP_yield, NULL);
         }

The goal of this patch is to add a way for the frontend to know when a console is connected. This patch adds a new flag to the end of the console ring structure. It is used for the backend to indicate that it has connected and started servicing the page.

The two values are
XENCONSOLE_DISCONNECTED 1
XENCONSOLE_CONNECTED 0

XENCONSOLE_DISCONNECTED indicates to the guest that ring is disconnected, so it will not be serviced. The guest can avoid writing into it in that case. A domU can use console hypercalls and only transition to the ring when it is connected and won't fill and block.

Quoted 7 lines

Once the backend (xenconsoled) maps and starts servicing the console, the flag will be set to XENCONSOLE_CONNECTED (0) to indicate the backend state to the frontend.

The connected value as 0 will be match the default of a zero-ed console page. Hyperlaunch can set the flag to XENCONSOLE_DISCONNECTED and let xenconsoled set to XENCONSOLE_CONNECTED.

I think libxenguest should set XENCONSOLE_DISCONNECTED as well (see below).

Quoted 7 lines

Old domU hvc_xen drivers won't check the flag.
New domU hvc_xen running on a new xen/xenconsoled will work properly. New domU hvc_xen on old xen/xenconsoled should only see a 0 for the flag and behave as if connected.

Adapt the title of the patch?

Quoted 12 lines

---
v1:
Remove evtchn notify call
Set connected later when there is no error

RFC v3:
Flip flag values so 0 is connected.

The other option would be to add:
uint32_t features
uint32_t connected

Quoted 7 lines

New domUs would check features for a magic value and/or flag to know they can rely on connected transitioning.

I think making XENCONSOLE_CONNECTED == 0 side steps the need for an additional features field. As long as assuming zero-ed memory is acceptable. However, this only matters for a hyperlaunched guest - xenconsoled will normally readily connect the console and set the value.

I'd like to consider other cases as well, e.g. a console driver domain. So any instance creating a domain with a console ring page should set the flag initially to "disconnected".

Quoted 10 lines

This assumes that existing frontends are not using the flag space for some other use.

Removed idea:
Send an event channel notification to let the domU know that xenconsoled is connected. Xenstored does similar, but for xenstore, the xenstore driver owns the event channel/irq and can rebind it. For hvc_xen, the hvc subsystem owns the irq, so it isn't readily available for rebinding. This is not implemented.

Quoted 4 lines

I had the idea for the kernel to use a static key and switch writing from the hypercall to the PV ring once connected. It didn't actually work in my short attempt - I think changing the static key from within an interupt was wrong. I fell back to just checking the flag directly

You'd need to do the static key changing from a worker thread instead.

Juergen

#2 lore raw
Quoted 45 lines · Jürgen Groß
Quoted 33 lines · Jason Andryuk

With hyperlaunch, a domU can start before its console ring is connected by xenconsoled.  With nothing emptying the ring, it can quickly fill during boot.  In domU_write_console(), __write_console() returns 0 when the ring is full.  This loops spins until xenconsoled starts emptying the ring:

         while (len) {
                 ssize_t sent = __write_console(cons, data, len);

                 if (sent < 0)
                         return sent;

                 data += sent;
                 len -= sent;

                 if (unlikely(len)) HYPERVISOR_sched_op(SCHEDOP_yield, NULL); }

The goal of this patch is to add a way for the frontend to know when a console is connected.  This patch adds a new flag to the end of the console ring structure.  It is used for the backend to indicate that it has connected and started servicing the page.

The two values are
XENCONSOLE_DISCONNECTED 1
XENCONSOLE_CONNECTED    0

XENCONSOLE_DISCONNECTED indicates to the guest that ring is disconnected, so it will not be serviced.  The guest can avoid writing into it in that case.  A domU can use console hypercalls and only transition to the ring when it is connected and won't fill and block.

Quoted 7 lines

Once the backend (xenconsoled) maps and starts servicing the console, the flag will be set to XENCONSOLE_CONNECTED (0) to indicate the backend state to the frontend.

The connected value as 0 will be match the default of a zero-ed console page.  Hyperlaunch can set the flag to XENCONSOLE_DISCONNECTED and let xenconsoled set to XENCONSOLE_CONNECTED.

I think libxenguest should set XENCONSOLE_DISCONNECTED as well (see below).

Quoted 1 line
Quoted 8 lines
Quoted 6 lines

Old domU hvc_xen drivers won't check the flag.
New domU hvc_xen running on a new xen/xenconsoled will work properly. New domU hvc_xen on old xen/xenconsoled should only see a 0 for the flag and behave as if connected.

Adapt the title of the patch?

With Jan's naming suggestion, it'll be "xenconsole: Add connection field"

Quoted 15 lines
Quoted 15 lines

---
v1:
Remove evtchn notify call
Set connected later when there is no error

RFC v3:
Flip flag values so 0 is connected.

The other option would be to add:
uint32_t features
uint32_t connected

New domUs would check features for a magic value and/or flag to know they can rely on connected transitioning.

Quoted 8 lines
Quoted 4 lines

I think making XENCONSOLE_CONNECTED == 0 side steps the need for an additional features field.  As long as assuming zero-ed memory is acceptable.  However, this only matters for a hyperlaunched guest - xenconsoled will normally readily connect the console and set the value.

I'd like to consider other cases as well, e.g. a console driver domain. So any instance creating a domain with a console ring page should set the flag initially to "disconnected".

Sure, that makes sense.

I don't have immediate plans for this, but I had the additional thought of placing the event channel inside the page like we do for xenstore. Today xenconsoled uses xenstore for the ring-ref and event channel, thought it tries the reserved console grant first. With the reserved grant, and the event channel inside, that dependency would be removed. Xenconsoled still needs some way to know when domains come and go, and it depends on xenstore's @introduceDomain/@releaseDomain for that. So it couldn't be completely decoupled without some other mechanism.

Quoted 10 lines
Quoted 10 lines

This assumes that existing frontends are not using the flag space for some other use.

Removed idea:
Send an event channel notification to let the domU know that xenconsoled is connected.  Xenstored does similar, but for xenstore, the xenstore driver owns the event channel/irq and can rebind it.  For hvc_xen, the hvc subsystem owns the irq, so it isn't readily available for rebinding. This is not implemented.

Quoted 6 lines
Quoted 4 lines

I had the idea for the kernel to use a static key and switch writing from the hypercall to the PV ring once connected.  It didn't actually work in my short attempt - I think changing the static key from within an interupt was wrong.  I fell back to just checking the flag directly

You'd need to do the static key changing from a worker thread instead.

Thanks. I'll investigate.

Regards,
Jason

#3 lore raw
In reply to Jürgen Groß
Quoted 69 lines · Jürgen Groß
Quoted 33 lines · Jason Andryuk

With hyperlaunch, a domU can start before its console ring is connected by xenconsoled.  With nothing emptying the ring, it can quickly fill during boot.  In domU_write_console(), __write_console() returns 0 when the ring is full.  This loops spins until xenconsoled starts emptying the ring:

         while (len) {
                 ssize_t sent = __write_console(cons, data, len);

                 if (sent < 0)
                         return sent;

                 data += sent;
                 len -= sent;

                 if (unlikely(len)) HYPERVISOR_sched_op(SCHEDOP_yield, NULL); }

The goal of this patch is to add a way for the frontend to know when a console is connected.  This patch adds a new flag to the end of the console ring structure.  It is used for the backend to indicate that it has connected and started servicing the page.

The two values are
XENCONSOLE_DISCONNECTED 1
XENCONSOLE_CONNECTED    0

XENCONSOLE_DISCONNECTED indicates to the guest that ring is disconnected, so it will not be serviced.  The guest can avoid writing into it in that case.  A domU can use console hypercalls and only transition to the ring when it is connected and won't fill and block.

Quoted 7 lines

Once the backend (xenconsoled) maps and starts servicing the console, the flag will be set to XENCONSOLE_CONNECTED (0) to indicate the backend state to the frontend.

The connected value as 0 will be match the default of a zero-ed console page.  Hyperlaunch can set the flag to XENCONSOLE_DISCONNECTED and let xenconsoled set to XENCONSOLE_CONNECTED.

I think libxenguest should set XENCONSOLE_DISCONNECTED as well (see below).

Quoted 7 lines

Old domU hvc_xen drivers won't check the flag.
New domU hvc_xen running on a new xen/xenconsoled will work properly. New domU hvc_xen on old xen/xenconsoled should only see a 0 for the flag and behave as if connected.

Adapt the title of the patch?

Quoted 15 lines

---
v1:
Remove evtchn notify call
Set connected later when there is no error

RFC v3:
Flip flag values so 0 is connected.

The other option would be to add:
uint32_t features
uint32_t connected

New domUs would check features for a magic value and/or flag to know they can rely on connected transitioning.

Quoted 8 lines
Quoted 4 lines

I think making XENCONSOLE_CONNECTED == 0 side steps the need for an additional features field.  As long as assuming zero-ed memory is acceptable.  However, this only matters for a hyperlaunched guest - xenconsoled will normally readily connect the console and set the value.

I'd like to consider other cases as well, e.g. a console driver domain. So any instance creating a domain with a console ring page should set the flag initially to "disconnected".

Setting disconnected for domain creation is fine. Looking at libxenguest, there is also domain restore. There the console could be set to disconnected again before domain restore. Again, this should work and xenconsoled would set it connected again. I originally intended for a single one way transition disconnected -> connected.

Alternatively, restore could skip setting disconnected and just assume xenconsoled will promptly attach. Restore implies a toolstack is running, so there isn't the indefinite time period that is involved with hyperlaunch/dom0less. But I guess an actively changing flag accurately shows the state, so that is preferable.

Quoted 10 lines
Quoted 10 lines

This assumes that existing frontends are not using the flag space for some other use.

Removed idea:
Send an event channel notification to let the domU know that xenconsoled is connected.  Xenstored does similar, but for xenstore, the xenstore driver owns the event channel/irq and can rebind it.  For hvc_xen, the hvc subsystem owns the irq, so it isn't readily available for rebinding. This is not implemented.

Quoted 6 lines
Quoted 4 lines

I had the idea for the kernel to use a static key and switch writing from the hypercall to the PV ring once connected.  It didn't actually work in my short attempt - I think changing the static key from within an interupt was wrong.  I fell back to just checking the flag directly

You'd need to do the static key changing from a worker thread instead.

My static key idea has an issue. The flag needs to be per-instance, primary console and any additional PV consoles, but the kernel has only a single function to handle all of them. Either the primary console needs dedicated ops, or the the flag would need to be checked in the function. If the flag will toggle back and forth, then a static key may not be appropriate.

Regards,
Jason

#4 lore raw
Quoted 75 lines · Jason Andryuk
Quoted 74 lines · Jürgen Groß
Quoted 33 lines · Jason Andryuk

With hyperlaunch, a domU can start before its console ring is connected by xenconsoled.  With nothing emptying the ring, it can quickly fill during boot.  In domU_write_console(), __write_console() returns 0 when the ring is full.  This loops spins until xenconsoled starts emptying the ring:

         while (len) {
                 ssize_t sent = __write_console(cons, data, len);

                 if (sent < 0)
                         return sent;

                 data += sent;
                 len -= sent;

                 if (unlikely(len)) HYPERVISOR_sched_op(SCHEDOP_yield, NULL); }

The goal of this patch is to add a way for the frontend to know when a console is connected.  This patch adds a new flag to the end of the console ring structure.  It is used for the backend to indicate that it has connected and started servicing the page.

The two values are
XENCONSOLE_DISCONNECTED 1
XENCONSOLE_CONNECTED    0

XENCONSOLE_DISCONNECTED indicates to the guest that ring is disconnected, so it will not be serviced.  The guest can avoid writing into it in that case.  A domU can use console hypercalls and only transition to the ring when it is connected and won't fill and block.

Quoted 7 lines

Once the backend (xenconsoled) maps and starts servicing the console, the flag will be set to XENCONSOLE_CONNECTED (0) to indicate the backend state to the frontend.

The connected value as 0 will be match the default of a zero-ed console page.  Hyperlaunch can set the flag to XENCONSOLE_DISCONNECTED and let xenconsoled set to XENCONSOLE_CONNECTED.

I think libxenguest should set XENCONSOLE_DISCONNECTED as well (see below).

Quoted 7 lines

Old domU hvc_xen drivers won't check the flag.
New domU hvc_xen running on a new xen/xenconsoled will work properly. New domU hvc_xen on old xen/xenconsoled should only see a 0 for the flag and behave as if connected.

Adapt the title of the patch?

Quoted 19 lines

---
v1:
Remove evtchn notify call
Set connected later when there is no error

RFC v3:
Flip flag values so 0 is connected.

The other option would be to add:
uint32_t features
uint32_t connected

New domUs would check features for a magic value and/or flag to know they can rely on connected transitioning.

I think making XENCONSOLE_CONNECTED == 0 side steps the need for an additional features field.  As long as assuming zero-ed memory is acceptable.  However, this only matters for a hyperlaunched guest - xenconsoled will normally readily connect the console and set the value.

Quoted 7 lines
Quoted 3 lines

I'd like to consider other cases as well, e.g. a console driver domain. So any instance creating a domain with a console ring page should set the flag initially to "disconnected".

Setting disconnected for domain creation is fine.  Looking at libxenguest, there is also domain restore.  There the console could be set to disconnected again before domain restore.  Again, this should work and xenconsoled would set it connected again.  I originally intended for a single one way transition disconnected -> connected.

Alternatively, restore could skip setting disconnected and just assume xenconsoled will promptly attach.  Restore implies a toolstack is running, so there isn't the indefinite time period that is involved with hyperlaunch/ dom0less.  But I guess an actively changing flag accurately shows the state, so that is preferable.

I agree.

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

This assumes that existing frontends are not using the flag space for some other use.

Removed idea:
Send an event channel notification to let the domU know that xenconsoled is connected.  Xenstored does similar, but for xenstore, the xenstore driver owns the event channel/irq and can rebind it.  For hvc_xen, the hvc subsystem owns the irq, so it isn't readily available for rebinding. This is not implemented.

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

I had the idea for the kernel to use a static key and switch writing from the hypercall to the PV ring once connected.  It didn't actually work in my short attempt - I think changing the static key from within an interupt was wrong.  I fell back to just checking the flag directly

You'd need to do the static key changing from a worker thread instead.

My static key idea has an issue.  The flag needs to be per-instance, primary console and any additional PV consoles, but the kernel has only a single function to handle all of them.  Either the primary console needs dedicated ops, or the the flag would need to be checked in the function.  If the flag will toggle back and forth, then a static key may not be appropriate.

Its not as if consoles are high throughput devices. So testing the flag instead of using a static key shouldn't be a big issue.

Juergen

#5 lore raw
In reply to Jason Andryuk
Quoted 10 lines · Jason Andryuk
xen/include/public/io/console.h +4 -0
@@ -19,6 +19,19 @@ struct xencons_interface {
     char out[2048];
     XENCONS_RING_IDX in_cons, in_prod;
     XENCONS_RING_IDX out_cons, out_prod;
+/*
+ * Flag values signaling from backend to frontend whether the console is
+ * connected.  i.e. Whether it will be serviced and emptied.
+ *

xen/include/public/io/console.h struct xencons_interface { · line 35

Quoted 10 lines
xen/include/public/io/console.h +9 -0
+ * The flag starts as disconnected.
+ */
+#define XENCONSOLE_DISCONNECTED 1
+/*
+ * The flag is set to connected when the backend connects and the console
+ * will be serviced.
+ */
+#define XENCONSOLE_CONNECTED    0
+    uint32_t flag;
 };

Even if the field name is singular, there's the possible reading of everything together as the connected state merely taking one bit, with other bits available for future use. IOW I think the field wants giving a less generic name, and perhaps also shrinking to unsigned char (or uint8_t, but char is being used in the structure already anyway).

Jan

#6 lore raw
Quoted 19 lines · Jan Beulich
Quoted 18 lines · Jason Andryuk
xen/include/public/io/console.h +12 -0
@@ -19,6 +19,19 @@ struct xencons_interface {
      char out[2048];
      XENCONS_RING_IDX in_cons, in_prod;
      XENCONS_RING_IDX out_cons, out_prod;
+/*
+ * Flag values signaling from backend to frontend whether the console is
+ * connected.  i.e. Whether it will be serviced and emptied.
+ *
+ * The flag starts as disconnected.
+ */
+#define XENCONSOLE_DISCONNECTED 1
+/*
+ * The flag is set to connected when the backend connects and the console
+ * will be serviced.
+ */
+#define XENCONSOLE_CONNECTED    0
Quoted 8 lines

xen/include/public/io/console.h struct xencons_interface { · line 35

Quoted 2 lines
xen/include/public/io/console.h +1 -0
+    uint32_t flag;
  };

Even if the field name is singular, there's the possible reading of everything together as the connected state merely taking one bit, with other bits available for future use. IOW I think the field wants giving a less generic name, and perhaps also shrinking to unsigned char (or uint8_t, but char is being used in the structure already anyway).

It could become a bit in a multi-flag field, dut the structure has a good bit of space to the end of the page, A named field, "connection", sounds good.

Thanks,
Jason