Skip to contentSkip to conversation

xenballoond (memory overcommit) scripts

PATCH In review Dan Magenheimer 2 participants 4 messages Jul 1, 2008

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xenballoond (memory overcommit) scripts

Attached is the current xenballoond script-set I talked about at Xen Summit 2008 that supports memory overcommit.

I've had a number of requests for the scripts and, though more polishing would be nice, it makes sense to push them upstream so that others in the community can try/test them and improve on them.

Note that there is no impact on any xen installation or on any guest unless the scripts are intentionally installed and configured on one or more guests.

See the README and conf files for more info.

All files are new so, in addition to the patch, these hg add commands will need to be done in the main tree.

hg add tools/xenballoond
hg add tools/xenballoond/xenballoond.init
hg add tools/xenballoond/xenballoond
hg add tools/xenballoond/xenballoon.conf
hg add tools/xenballoond/xenballoon.README
hg add tools/xenballoond/xenballoon-monitor

Thanks,
Dan

tools/xenballoon/xenballoon-monitor new file +43 -0
diff -r 0972fc23b504 tools/xenballoon/xenballoon-monitor
@@ -0,0 +1,43 @@
+#!/bin/bash
+#
+# xenballoon-monitor - monitor certain stats from xenballoond
+#   (run in dom0 with "watch -d xenballoon-monitor" for xentop-like output)
+#
+# Copyright (C) 2009 Oracle Corporation and/or its affiliates.
+# All rights reserved
+# Written by: Dan Magenheimer <[email protected]>
+#
+# Hint: Use "xm sched-credit -d 0 -w 2000" to watch on heavily loaded machines
+#
+echo "id   mem-kb  tgt-kb  commit   swapin  swapout      pgin     pgout active(sec)"
+for i in `xenstore-list /local/domain`; do
+ if [ "$i" -ne 0 ]; then
+ tot=0; tgt=0; sin=0; sout=0; pgin=0; pgout=0; cmt=0; up=0; idle=0; act=0;
+ if xenstore-exists /local/domain/$i/memory/meminfo; then
+  tot=`xenstore-read /local/domain/$i/memory/meminfo | grep MemTotal \
+   | sed 's/[^1-9]*\([1-9][0-9]*\).*/\1/'`
+  cmt=`xenstore-read /local/domain/$i/memory/meminfo | grep Committed_AS \
+   | sed 's/[^1-9]*\([1-9][0-9]*\).*/\1/'`
+ fi
+ if xenstore-exists /local/domain/$i/memory/selftarget; then
+  tgt=`xenstore-read /local/domain/$i/memory/selftarget`
+ fi
+ if xenstore-exists /local/domain/$i/memory/vmstat; then
+  sin=`xenstore-read /local/domain/$i/memory/vmstat | grep pswpin \
+ 	| cut -d" " -f2`
+  sout=`xenstore-read /local/domain/$i/memory/vmstat | grep pswpout \
+ 	| cut -d" " -f2`
+  pgin=`xenstore-read /local/domain/$i/memory/vmstat | grep pgpgin \
+ 	| cut -d" " -f2`
+  pgout=`xenstore-read /local/domain/$i/memory/vmstat | grep pgout \
+  	| cut -d" " -f2`
+ fi
+ if xenstore-exists /local/domain/$i/memory/uptime; then
+  up=`xenstore-read /local/domain/$i/memory/uptime | cut -d" " -f1`
+  idle=`xenstore-read /local/domain/$i/memory/uptime | cut -d" " -f2`
+  act=`echo $up - $idle | bc -iq`
+ fi
+ printf "%2d %8d%8d%8d%9d%9d%10d%10d%10.2f\n" $i $tot $tgt $cmt $sin $sout $pgin $pgout $act
+ fi
+done
+echo Free memory: `xm info | grep free | sed 's/[^1-9]*\([1-9][0-9]*\).*/\1/'` MB
tools/xenballoon/xenballoon.conf new file +91 -0

91 lines not on this page · raw message

tools/xenballoon/xenballoond new file +205 -0

205 lines not on this page · raw message

tools/xenballoon/xenballoond.README new file +82 -0

82 lines not on this page · raw message

tools/xenballoon/xenballoond.init new file +91 -0

91 lines not on this page · raw message

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#1 lore raw

Hello,

is it a good idea to run a memory balloon process in a domU? As you know I've tested your xenbus selfballooning, I've thought this make more sense?

I thought it would be more secure and better for policing reasons to run in dom0?

I've just tested the script and it works fine, now i will try it for a short periode...

greetings
Viets

Quoted 38 lines · Dan Magenheimer

Attached is the current xenballoond script-set I talked about at Xen Summit 2008 that supports memory overcommit.

I've had a number of requests for the scripts and, though more polishing would be nice, it makes sense to push them upstream so that others in the community can try/test them and improve on them.

Note that there is no impact on any xen installation or on any guest unless the scripts are intentionally installed and configured on one or more guests.

See the README and conf files for more info.

All files are new so, in addition to the patch, these hg add commands will need to be done in the main tree.

hg add tools/xenballoond
hg add tools/xenballoond/xenballoond.init
hg add tools/xenballoond/xenballoond
hg add tools/xenballoond/xenballoon.conf
hg add tools/xenballoond/xenballoon.README
hg add tools/xenballoond/xenballoon-monitor

Thanks,
Dan

------------------------------------------------------------------------

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#2 lore raw

Hi Viets --

A guest can't be ballooned without its "permission". The original implementation had the selfballooning in the guest's balloon driver, which could be rmmod'd inside the guest, so I don't think the old model was more secure than the new, which puts the selfballooning in a daemon. The worst that a malicious guest can do in either case is ensure it always gets all the memory that's assigned to it.

Or have you thought of a different attack scenario?

Thanks for the testing. Make sure you try running "watch -d xenballoond-monitor" in domain0.

Thanks,
Dan

Quoted 55 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]]
Sent: Tuesday, July 01, 2008 6:06 AM
To: [email protected]
Cc: [email protected]
Subject: Re: [Xen-devel] [PATCH] xenballoond (memory overcommit) scripts

Hello,

is it a good idea to run a memory balloon process in a domU? As you know
I've tested your xenbus selfballooning, I've thought this make more sense?

I thought it would be more secure and better for policing reasons to run
in dom0?

I've just tested the script and it works fine, now i will try it for a short periode...

greetings
Viets

Quoted 28 lines · Dan Magenheimer

Attached is the current xenballoond script-set I talked about at Xen Summit 2008 that supports memory overcommit.

I've had a number of requests for the scripts and, though more polishing would be nice, it makes sense to push them upstream so that others in the community can try/test them and improve on them.

Note that there is no impact on any xen installation or on any guest unless the scripts are intentionally installed and configured on one or more guests.

See the README and conf files for more info.

All files are new so, in addition to the patch, these hg add commands will need to be done in the main tree.

hg add tools/xenballoond
hg add tools/xenballoond/xenballoond.init
hg add tools/xenballoond/xenballoond
hg add tools/xenballoond/xenballoon.conf
hg add tools/xenballoond/xenballoon.README
hg add tools/xenballoond/xenballoon-monitor

Quoted 14 lines
Quoted 5 lines

Thanks,
Dan

--------------------------------------------------------------
----------

Quoted 5 lines

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#3 lore raw

Hey,

ok, you're right, but I thought to change a kernel is harder than changing a skript on a domU...

Your script inspired me to make an own script working on dom0, it isn't beautiful, but my first test were successfully, tomorrow I will test it more.

I will post my script just to know whether it is worth to improve it.

First of all I changed the DomU kernel, that I've got the Memory Information over xenbus.

/linux-2.6.18-xen.hg-xenU.bak/fs/proc/proc_misc.c
54a55,58

Quoted 4 lines

//Edited by me
#include <xen/xenbus.h>
//Edited by me

205a210,216

Quoted 7 lines

//Edited by Me

(quoted diff) +0 -0
 		(void)xenbus_printf(XBT_NIL, "memory", "Committed_AS",
                                "%lu", K(committed) );
 		(void)xenbus_printf(XBT_NIL, "memory", "MemTotal",
                                "%lu", K(i.totalram) );

//Edited by Me

After that I just edit your script, as I say it is not finished, but just wanna post it for feedback..

#/bin/bash

curkb() {
kb=$(xenstore-read /local/domain/$1/memory/MemTotal); RETVAL=$kb
return # value returned in RETVAL in kB
}

downhysteresis() {
# RETVAL=$XENBALLOON_AUTO_DOWNHYSTERESIS
# if [ $xenstore_enabled = "true" ]; then
# if xenstore-exists memory/downhysteresis ; then
# RETVAL=xenstore-read memory/downhysteresis
# fi
# fi
RETVAL=10;
return;
}

uphysteresis() {
# RETVAL=$XENBALLOON_AUTO_UPHYSTERESIS
# if [ $xenstore_enabled = "true" ]; then
# if xenstore-exists memory/uphysteresis ; then
# RETVAL=xenstore-read memory/uphysteresis
# fi
# fi
RETVAL=1;
return
}

minmb() {
RETVAL=$XENBALLOON_MINMEM;
if [ $RETVAL -ne 0 ]; then
return $RETVAL
fi
kb=$(cat $maxkb);
let "mb=$kb/1024";
let "pages=$kb/4";

  # this algorithm from drivers/xen/balloon/balloon.c:minimum_target()
  # which was added to balloon.c in 2008 to avoid ballooning too small
  # it is unnecessary here except to accomodate pre-2008 balloon drivers
  # note that ranges are adjusted because a VM with "memory=1024"
  # gets somewhat less than 1024MB
  if [ $mb -lt 125 ]; then
    let RETVAL="$(( 8 + ($pages >> 9) ))"
  elif [ $mb -lt 500 ]; then
    let RETVAL="$(( 40 + ($pages >> 10) ))"
  elif [ $mb -lt 2000 ]; then
    let RETVAL="$(( 104 + ($pages >> 11) ))"
  else
    let RETVAL="$(( 296 + ($pages >> 13) ))"
  fi
  return	# value returned in RETVAL in mB
}

selftarget() {
tgtkb=$(xenstore-read /local/domain/$1/memory/Committed_AS); minmb;
let "minbytes=$RETVAL*1024*1024";
let "tgtbytes=$tgtkb*1024";
if [ $tgtbytes -lt $minbytes ]; then

    let "tgtbytes=$minbytes";

fi
RETVAL=$tgtbytes # value returned in RETVAL in bytes return
}

balloon_to_target() {
if [ "$2" -eq 1 ]; then

    selftarget $1;
    tgtbytes=$RETVAL;
   else
     let "tgtbytes=$(( $1 * 1024 ))";
  fi;
    curkb $1;
    let "curbytes=$RETVAL*1024"
    if [ $curbytes -gt $tgtbytes ]; then
      downhysteresis;
      downhys=$RETVAL
      if [ $downhys -ne 0 ]; then
        let "tgtbytes=$(( $curbytes - \
        ( ( $curbytes - $tgtbytes ) / $downhys ) ))"
      fi
    else if [ $curbytes -lt $tgtbytes ]; then
      uphysteresis
      uphys=$RETVAL
      let "tgtbytes=$(( $curbytes + \
      ( ( $tgtbytes - $curbytes ) / $uphys ) ))"
    fi

fi
let "tgt=$(( $tgtbytes/1024/1024 ))";
xm mem-set $1 $tgt;
let "tgtkb=$(( $tgtbytes/1024 ))" xenstore-write /local/domain/$1/memory/selftarget $tgtkb
}

selfballoon_eval() {
if xenstore-exists /local/domain/$1/memory/selfballoon; then

    RETVAL=$(xenstore-read /local/domain/$1/memory/selfballoon);
    if [ $RETVAL -eq 1 ]; then
      selfballoon_enabled=true;
      return;
    fi;

fi;
selfballoon_enabled=$XENBALLOON_SELF;
return;
}

maxkb=1073741824;

. /etc/sysconfig/xenballoon.conf

while true; do
for i in $(xm list | awk '{print $2;'} | fgrep -v 'ID' | fgrep -v '0'); do

#    curkb=$(xenstore-read /local/domain/$i/memory/MemTotal);
    selfballoon_eval $i;
#    if [ $selfballoon_enabled = "true" ]; then
      tgtkb=$(xenstore-read /local/domain/$i/memory/target);
      balloon_to_target $i $tgtkb;

# fi;
done;
sleep $XENBALLOON_INTERVAL;
done

greetings
Viets

Quoted 85 lines · Dan Magenheimer

Hi Viets --

A guest can't be ballooned without its "permission". The original implementation had the selfballooning in the guest's balloon driver, which could be rmmod'd inside the guest, so I don't think the old model was more secure than the new, which puts the selfballooning in a daemon. The worst that a malicious guest can do in either case is ensure it always gets all the memory that's assigned to it.

Or have you thought of a different attack scenario?

Thanks for the testing. Make sure you try running "watch -d xenballoond-monitor" in domain0.

Thanks,
Dan

Quoted 66 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]]
Sent: Tuesday, July 01, 2008 6:06 AM
To: [email protected]
Cc: [email protected]
Subject: Re: [Xen-devel] [PATCH] xenballoond (memory overcommit) scripts

Hello,

is it a good idea to run a memory balloon process in a domU? As you know
I've tested your xenbus selfballooning, I've thought this make more sense?

I thought it would be more secure and better for policing reasons to run
in dom0?

I've just tested the script and it works fine, now i will try it for a short periode...

greetings
Viets

Quoted 26 lines · Dan Magenheimer

Attached is the current xenballoond script-set I talked about at Xen Summit 2008 that supports memory overcommit.

I've had a number of requests for the scripts and, though more polishing would be nice, it makes sense to push them upstream so that others in the community can try/test them and improve on them.

Note that there is no impact on any xen installation or on any guest unless the scripts are intentionally installed and configured on one or more guests.

See the README and conf files for more info.

All files are new so, in addition to the patch, these hg add commands will need to be done in the main tree.

hg add tools/xenballoond
hg add tools/xenballoond/xenballoond.init
hg add tools/xenballoond/xenballoond
hg add tools/xenballoond/xenballoon.conf
hg add tools/xenballoond/xenballoon.README
hg add tools/xenballoond/xenballoon-monitor

Quoted 7 lines

Thanks,
Dan

--------------------------------------------------------------
----------

Quoted 4 lines

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

Signature · 5 lines
Torben Viets                              w               [email protected]
n@work Internet Informationssysteme GmbH  o      http://support.work.de
Wandalenweg 5                             r   Tel.: +49 40 23 88 09 - 0
D-20097 Hamburg                           k   Fax:  +49 40 23 88 09 -29
HR B 61 668  - Amtsgericht Hamburg 	      	      Gf Jan Diegelmann