Skip to contentSkip to conversation

domain_create: honour global grant/maptrack frame limits...

PATCH In review Paul Durrant 2 participants 4 messages Nov 13, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 domain_create: honour global grant/maptrack frame limits...

...when their values are larger than the per-domain configured limits.

---
After mining through commits it is still unclear to me exactly when Xen stopped honouring the global values, but I really think this commit should be back-ported to stable trees as it was a behavioural change that can cause domUs to fail in non-obvious ways.

---
 xen/common/domain.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
xen/common/domain.c +12 -2
index 611116c7fc..aad6d55b82 100644
@@ -335,6 +335,7 @@ struct domain *domain_create(domid_t domid,
     enum { INIT_watchdog = 1u<<1,
            INIT_evtchn = 1u<<3, INIT_gnttab = 1u<<4, INIT_arch = 1u<<5 };
     int err, init_status = 0;
+    unsigned int max_grant_frames, max_maptrack_frames;
 
     if ( config && (err = sanitise_domain_config(config)) )
         return ERR_PTR(err);
@@ -456,8 +457,17 @@ struct domain *domain_create(domid_t domid,
             goto fail;
         init_status |= INIT_evtchn;
 
-        if ( (err = grant_table_init(d, config->max_grant_frames,
-                                     config->max_maptrack_frames)) != 0 )
+        /*
+         * Make sure that the configured values don't reduce any
+         * global command line override.
+         */
+        max_grant_frames = max(config->max_grant_frames,
+                               opt_max_grant_frames);
+        max_maptrack_frames = max(config->max_maptrack_frames,
+                                  opt_max_maptrack_frames);
+
+        if ( (err = grant_table_init(d, max_grant_frames,
+                                     max_maptrack_frames)) != 0 )
             goto fail;
         init_status |= INIT_gnttab;
 
Signature · 7 lines
2.17.1


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw

Sorry, the Cc list got dropped... I'll re-send.

Paul

Quoted 53 lines

-----Original Message-----
From: Paul Durrant <[email protected]>
Sent: 13 November 2019 13:47
To: [email protected]
Cc: Durrant, Paul <[email protected]>
Subject: [PATCH] domain_create: honour global grant/maptrack frame limits...

...when their values are larger than the per-domain configured limits.

---
After mining through commits it is still unclear to me exactly when Xen stopped honouring the global values, but I really think this commit should be back-ported to stable trees as it was a behavioural change that can cause domUs to fail in non-obvious ways.

---
 xen/common/domain.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
xen/common/domain.c +12 -3
index 611116c7fc..aad6d55b82 100644
@@ -335,6 +335,7 @@ struct domain *domain_create(domid_t domid,
     enum { INIT_watchdog = 1u<<1,
            INIT_evtchn = 1u<<3, INIT_gnttab = 1u<<4, INIT_arch = 1u<<5 };
     int err, init_status = 0;
+    unsigned int max_grant_frames, max_maptrack_frames;
 
     if ( config && (err = sanitise_domain_config(config)) )
         return ERR_PTR(err);
@@ -456,8 +457,17 @@ struct domain *domain_create(domid_t domid,
             goto fail;
         init_status |= INIT_evtchn;
 
-        if ( (err = grant_table_init(d, config->max_grant_frames,
-                                     config->max_maptrack_frames)) != 0 )
+        /*
+         * Make sure that the configured values don't reduce any
+         * global command line override.
+         */
+        max_grant_frames = max(config->max_grant_frames,
+                               opt_max_grant_frames);
+        max_maptrack_frames = max(config->max_maptrack_frames,
+                                  opt_max_maptrack_frames);
+
+        if ( (err = grant_table_init(d, max_grant_frames,
+                                     max_maptrack_frames)) != 0 )
             goto fail;
         init_status |= INIT_gnttab;
 
--
Quoted 1 line

2.17.1

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#2 lore raw
In reply to Paul Durrant
Quoted 8 lines · Paul Durrant

...when their values are larger than the per-domain configured limits.

---
After mining through commits it is still unclear to me exactly when Xen stopped honouring the global values, but I really think this commit should be back-ported to stable trees as it was a behavioural change that can cause domUs to fail in non-obvious ways.

-1.  Overriding toolstack settings like this is the same kind of "bad" as silently converting HAP => Shadow.

In particular, this breaks one of points of the original per-domain work to deliberately allow stub xenstored to be configured with tiny grant/maptrack tables.

You also break the setting of these parameters in xl.conf.

I'm not defending how the interface changed subtly/unexpected; that was bad and we should have done better, but this change is just as bad in the opposite direction.

The way to fix this is to plumb the Xen default parameters though so that, in the absence of any explicit configuration (vm.cfg or xl.conf), libxl can then use "xen cmdline" as a source of configuration, before falling back to hardcoded numbers.

~Andrew

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#3 lore raw
Quoted 18 lines

-----Original Message-----
From: Andrew Cooper <[email protected]>
Sent: 13 November 2019 14:05
To: Durrant, Paul <[email protected]>; [email protected]
Subject: Re: [Xen-devel] [PATCH] domain_create: honour global grant/maptrack frame limits...

Quoted 6 lines · Paul Durrant

...when their values are larger than the per-domain configured limits.

---
After mining through commits it is still unclear to me exactly when Xen stopped honouring the global values, but I really think this commit

should

Quoted 2 lines

be back-ported to stable trees as it was a behavioural change that can cause domUs to fail in non-obvious ways.

Quoted 6 lines

-1.  Overriding toolstack settings like this is the same kind of "bad" as silently converting HAP => Shadow.

In particular, this breaks one of points of the original per-domain work to deliberately allow stub xenstored to be configured with tiny grant/maptrack tables.

Ok, but IMO subtly breaking domUs in the process is not really acceptable behaviour.

Quoted 2 lines

You also break the setting of these parameters in xl.conf.

No I don't. xl.conf can still increase values over the command line.

Quoted 10 lines

I'm not defending how the interface changed subtly/unexpected; that was bad and we should have done better, but this change is just as bad in the opposite direction.

The way to fix this is to plumb the Xen default parameters though so that, in the absence of any explicit configuration (vm.cfg or xl.conf), libxl can then use "xen cmdline" as a source of configuration, before falling back to hardcoded numbers.

I agree that is the best way to fix it, but not so easy to backport. So my proposal (via email a few days ago) was that regressions are fixed immediately in this way and then a *proper* fix is done moving forward, which I shall base upon Juergen's patches which should allow easy retrieval of the command line values.

Paul

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel