xenctrl.ml: improve scalability of domain_getinfolist
Get series with b4
b4 am [email protected]Versions newest first
- v1 Latest2 patches 4 repliesIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/2 | xenctrl.ml: make domain_getinfolist tail recursivetools/ | +17 | 0 replies | |
| 2/2 | xenctrl: use larger chunksize in domain_getinfolisttools/ | +1 | 0 replies |
Reviews trailers per patch
None of the 2 patches has a Reviewed-by or Acked-by of their own; the series has one for all of them.
| Patch | Pau Ruiz Safont | Christian Lindig |
|---|---|---|
| Series | A-b | |
| 1/2 | T-b | |
| 2/2 | T-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH for-4.17 v1 0/2] xenctrl.ml: improve scalability of domain_getinfolist
Pau has performed some performance tests by booting 1000 mirage unikernels test VMs and shutting them down.
We've noticed on the flamegraphs that a lot of time is spent in Xenctrl domain_getinfolist, 17.7% of overall time
(This needs to be multiplied by 16 because Dom0 100% usage = 16 vCPUs) In particular time is spent in camlXenctrl___getlist_339
as can be seen from this flamegraph, and it also creates a very deep call stack:
https://cdn.jsdelivr.net/gh/edwintorok/xen@xenctrl-coverletter/docs/tmp/perf-merge-boot.svg?x=948.9&y=2213
After some algorithmic improvements to the code now the function barely shows up at all on a flamegraph, taking only 0.02%.
The function is called camlXenctrl___getlist_343, but that is just due to the changed arguments, still the same function:
https://cdn.jsdelivr.net/gh/edwintorok/xen@xenctrl-coverletter/docs/tmp/perf-xen-boot-1150.svg?x=1188.0&y=1941&s=infolist
It was calling the Xen hypercall ~500*1000 times for 1000 VMs, and instead it is now calling it "only" 1000 times.
I would suggest to try to take this in 4.17 given the massive improvement in scalability (number of VMs on a Xen host).
There are further improvements possible here, but they'll be in xenopsd (part of XAPI) to avoid calling domain_getinfolist and just usedomain_getinfo: the only reason it needs use infolist is that it does the lookup by VM UUID and not by domid, but it could have a small cache of UUID->domid mappings and then call just domain_getinfo (or get the mapping from xenstore if not in the cache), but it looks like that improvement is not even needed if this function barely registers on a flamegraph now.
P.S.: the mirage test VM is a very old PV version, at some point we'll repeat the test with a Solo5 based PVH one.
Edwin Török (2):
xenctrl.ml: make domain_getinfolist tail recursive
xenctrl: use larger chunksize in domain_getinfolist
tools/ocaml/libs/xc/xenctrl.ml | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-)
Signature · 1 line
2.34.1