Skip to contentSkip to conversation

tools/libxl: make default of max event channels dependant on vcpus

PATCH Superseded Juergen Gross 3 participants 8 messages Mar 26, 2020

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 tools/libxl: make default of max event channels dependant on vcpus

Today the maximum number of event channels for a guest is defaulting to 1023. For large guests with lots of vcpus this is not enough, as e.g. the Linux kernel uses 7 event channels per vcpu, limiting the guest to about 140 vcpus.

Instead of requiring to specify the allowed number of event channels via the "event_channels" domain config option, make the default depend on the maximum number of vcpus of the guest.

In order not to regress current configs use 1023 as the minimum default setting.

---
 tools/libxl/libxl_create.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/libxl/libxl_create.c +1 -1
index e7cb2dbc2b..eddd0e98e5 100644
@@ -226,7 +226,7 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
             b_info->iomem[i].gfn = b_info->iomem[i].start;
 
     if (!b_info->event_channels)
-        b_info->event_channels = 1023;
+        b_info->event_channels = min(1023, b_info->max_vcpus * 8 + 255);
 
     libxl__arch_domain_build_info_setdefault(gc, b_info);
     libxl_defbool_setdefault(&b_info->dm_restrict, false);
Signature · 1 line
2.16.4
#1 lore raw
Quoted 4 lines · Juergen Gross

Today the maximum number of event channels for a guest is defaulting to 1023. For large guests with lots of vcpus this is not enough, as e.g. the Linux kernel uses 7 event channels per vcpu, limiting the guest to about 140 vcpus.

I don't think any particular guest OS should "dictate" the default here. What if another OS (or even a future version of Linux) requires more than 8? And why would the value depend on number of vCPU-s but not number and kind of virtual devices?

Jan

#2 lore raw
Quoted 10 lines · Jan Beulich
Quoted 4 lines · Juergen Gross

Today the maximum number of event channels for a guest is defaulting to 1023. For large guests with lots of vcpus this is not enough, as e.g. the Linux kernel uses 7 event channels per vcpu, limiting the guest to about 140 vcpus.

I don't think any particular guest OS should "dictate" the default here. What if another OS (or even a future version of Linux) requires more than 8? And why would the value depend on number of vCPU-s but not number and kind of virtual devices?

It is a rough estimate. And with HVM guests being limited to 128 cpus anyway, this problem is more relevant for PV guests, limiting the available options for guest OS's.

You can always specify a larger value in the guest config after all.

Juergen

#3 lore raw
Quoted 7 lines · Jürgen Groß
Quoted 6 lines · Jan Beulich
Quoted 4 lines · Juergen Gross

Today the maximum number of event channels for a guest is defaulting to 1023. For large guests with lots of vcpus this is not enough, as e.g. the Linux kernel uses 7 event channels per vcpu, limiting the guest to about 140 vcpus.

Quoted 8 lines
Quoted 4 lines

I don't think any particular guest OS should "dictate" the default here. What if another OS (or even a future version of Linux) requires more than 8? And why would the value depend on number of vCPU-s but not number and kind of virtual devices?

It is a rough estimate. And with HVM guests being limited to 128 cpus anyway, this problem is more relevant for PV guests, limiting the available options for guest OS's.

How many evtchns would e.g. mini-os require? I.e. wouldn't such stubdoms get a much larger than necessary allowance this way?

Jan

#4 lore raw
Quoted 13 lines · Jan Beulich
Quoted 12 lines · Jürgen Groß
Quoted 10 lines · Jan Beulich
Quoted 4 lines · Juergen Gross

Today the maximum number of event channels for a guest is defaulting to 1023. For large guests with lots of vcpus this is not enough, as e.g. the Linux kernel uses 7 event channels per vcpu, limiting the guest to about 140 vcpus.

I don't think any particular guest OS should "dictate" the default here. What if another OS (or even a future version of Linux) requires more than 8? And why would the value depend on number of vCPU-s but not number and kind of virtual devices?

Quoted 6 lines
Quoted 3 lines

It is a rough estimate. And with HVM guests being limited to 128 cpus anyway, this problem is more relevant for PV guests, limiting the available options for guest OS's.

How many evtchns would e.g. mini-os require? I.e. wouldn't such stubdoms get a much larger than necessary allowance this way?

mini-os doesn't support SMP. So (if not configured with lots of vcpus, which doesn't make sense) they still get 1023 event channels per default. Which is much more as they need, but the same as today.

Xenstore stubdom is unlimited like dom0, BTW.

Juergen

#5 lore raw
In reply to Juergen Gross
Quoted 22 lines · Juergen Gross

Today the maximum number of event channels for a guest is defaulting to 1023. For large guests with lots of vcpus this is not enough, as e.g. the Linux kernel uses 7 event channels per vcpu, limiting the guest to about 140 vcpus.

Instead of requiring to specify the allowed number of event channels via the "event_channels" domain config option, make the default depend on the maximum number of vcpus of the guest.

In order not to regress current configs use 1023 as the minimum default setting.

---
  tools/libxl/libxl_create.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
tools/libxl/libxl_create.c +0 -0
index e7cb2dbc2b..eddd0e98e5 100644

tools/libxl/libxl_create.c int libxl__domain_build_info_setdefault(libxl__gc *gc, · line 229

Quoted 5 lines
tools/libxl/libxl_create.c +1 -1
@@ -226,7 +226,7 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
              b_info->iomem[i].gfn = b_info->iomem[i].start;
 
      if (!b_info->event_channels)
-        b_info->event_channels = 1023;
+        b_info->event_channels = min(1023, b_info->max_vcpus * 8 + 255);

Sorry, forgot to do "stg refresh", this should be max(), of course.

Juergen

#6 lore raw
In reply to Juergen Gross
Quoted 5 lines · Juergen Gross

Today the maximum number of event channels for a guest is defaulting to 1023. For large guests with lots of vcpus this is not enough, as e.g. the Linux kernel uses 7 event channels per vcpu, limiting the guest to about 140 vcpus.

Quoted 8 lines

Instead of requiring to specify the allowed number of event channels via the "event_channels" domain config option, make the default depend on the maximum number of vcpus of the guest.

In order not to regress current configs use 1023 as the minimum default setting.

:(

I suspect I regressed this by making it not actually dead code.

1023 is the 32bit EVT 2L ABI limit, but until I moved max_evtchn into the createdomain hypercall, I'm pretty sure a limit never used to get set.

If this is the case, then unlimited should be the default.

~Andrew

#7 lore raw
Quoted 13 lines · Andrew Cooper
Quoted 12 lines · Juergen Gross

Today the maximum number of event channels for a guest is defaulting to 1023. For large guests with lots of vcpus this is not enough, as e.g. the Linux kernel uses 7 event channels per vcpu, limiting the guest to about 140 vcpus.

Instead of requiring to specify the allowed number of event channels via the "event_channels" domain config option, make the default depend on the maximum number of vcpus of the guest.

In order not to regress current configs use 1023 as the minimum default setting.

Quoted 8 lines
Quoted 1 line

:(

I suspect I regressed this by making it not actually dead code.

1023 is the 32bit EVT 2L ABI limit, but until I moved max_evtchn into the createdomain hypercall, I'm pretty sure a limit never used to get set.

In 4.11 I'm seeing it being set via xc_domain_set_max_evtchn().

Juergen