KSM with Debian GNU/Linux
Hi all!
I have understood that in addition to support from the kernel and qemu-kvm, there must be support in libc. So in testing I was doing on Lenny, it did not work using libc from Debian repositories. Now that squeeze is frozen, anyone knows if it will support KSM?
Has anyone had success making KSM works on Lenny? Michael?
Thanks in advance for your reply.
Regards,
Daniel
Signature · 2 lines
Fingerprint: BFB3 08D6 B4D1 31B2 72B9 29CE 6696 BF1B 14E6 1D37 Powered by Debian GNU/Linux Lenny - Linux user #188.598
Quoted 6 lines · Daniel Bareiro
Hi all!
I have understood that in addition to support from the kernel and qemu-kvm, there must be support in libc. So in testing I was doing on Lenny, it did not work using libc from Debian repositories. Now that squeeze is frozen, anyone knows if it will support KSM?
KSM is working in qemu-kvm packages in bpo, for quite long time already. It has nothing to do with libc, and now it does not even require recent-enough kernel headers to compile. What's needed is recent-enough kernel that actually implements the feature. The kernel in Lenny does not KSM support, but the one in bpo has. Also note that Lenny's kernel is too old for kvm.
/mjt
Hi,
I don't have clue what glibc is contained in debian, but it seems to me that at least compile-time KSM support DOES depend on glibc. I'm using RHEL5-based system and KSM support doesn't get compiled in by default.
But since I use new kernel, I add following to CFLAGS while compiling qemu-kvm and then it works:
-DMADV_MERGEABLE=12 -DMADV_UNMERGEABLE=13
just my 5cents
regards
nik
Quoted 22 lines · Michael Tokarev
Quoted 6 lines · Daniel Bareiro
Hi all!
I have understood that in addition to support from the kernel and qemu-kvm, there must be support in libc. So in testing I was doing on Lenny, it did not work using libc from Debian repositories. Now that squeeze is frozen, anyone knows if it will support KSM?
KSM is working in qemu-kvm packages in bpo, for quite long time already. It has nothing to do with libc, and now it does not even require recent-enough kernel headers to compile. What's needed is recent-enough kernel that actually implements the feature. The kernel in Lenny does not KSM support, but the one in bpo has. Also note that Lenny's kernel is too old for kvm.
/mjt
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 13 lines
------------------------------------- Ing. Nikola CIPRICH LinuxBox.cz, s.r.o. 28. rijna 168, 709 01 Ostrava tel.: +420 596 603 142 fax: +420 596 621 273 mobil: +420 777 093 799 www.linuxbox.cz mobil servis: +420 737 238 656 email servis: [email protected] -------------------------------------
09.08.2010 15:00, Nikola Ciprich пишет:
Quoted 5 lines
Hi,
I don't have clue what glibc is contained in debian, but it seems to me that at least compile-time KSM support DOES depend on glibc. I'm using RHEL5-based system and KSM support doesn't get compiled in by default.
For KSM to be compiled in, one need to have kernel headers of the corresponding system interface (madvise() system call). That is not part of glibc (even while madvise function is declared by glibc) but part of kernel headers which glibc uses itself during compilation.
In debian that comes in linux-libc-dev package, some other distribution may bundle it together with glibc but that's pure packaging stuff, these headers does not come from glibc sources.
Quoted 3 lines
But since I use new kernel, I add following to CFLAGS while compiling qemu-kvm and then it works:
-DMADV_MERGEABLE=12 -DMADV_UNMERGEABLE=13
This is what the patch in debian package does, to get KSM-ready binary even without recent-enough kernel headers (like on Lenny).
/mjt
Quoted 1 line · Nikola Ciprich
Hi,
Hi, Nikola.
Quoted 7 lines
I don't have clue what glibc is contained in debian, but it seems to me that at least compile-time KSM support DOES depend on glibc. I'm using RHEL5-based system and KSM support doesn't get compiled in by default.
But since I use new kernel, I add following to CFLAGS while compiling qemu-kvm and then it works:
-DMADV_MERGEABLE=12 -DMADV_UNMERGEABLE=13
My comments on the dependency of glibc support was based on reading this bug report [1].
Regards,
Daniel
[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=556631
Signature · 2 lines
Fingerprint: BFB3 08D6 B4D1 31B2 72B9 29CE 6696 BF1B 14E6 1D37 Powered by Debian GNU/Linux Lenny - Linux user #188.598
Quoted 5 lines · Michael Tokarev
Quoted 4 lines
I have understood that in addition to support from the kernel and qemu-kvm, there must be support in libc. So in testing I was doing on Lenny, it did not work using libc from Debian repositories. Now that squeeze is frozen, anyone knows if it will support KSM?
Quoted 7 lines
KSM is working in qemu-kvm packages in bpo, for quite long time already. It has nothing to do with libc, and now it does not even require recent-enough kernel headers to compile. What's needed is recent-enough kernel that actually implements the feature. The kernel in Lenny does not KSM support, but the one in bpo has. Also note that Lenny's kernel is too old for kvm.
I'm using Linux 2.6.32.3 compiled by myself with the kernel.org source code with support for KSM:
# cat /boot/config-2.6.32.3-dgb | grep KSM CONFIG_KSM=y
I'm also using qemu-kvm 0.12.1.2 compiled by myself with the source code of SourceForge.
These are the checks that I was doing in the VMHost:
# ll /sys/kernel/mm/ksm/ total 0 -r--r--r-- 1 root root 4096 ago 9 06:28 full_scans -rw-r--r-- 1 root root 4096 ago 9 06:28 max_kernel_pages -r--r--r-- 1 root root 4096 ago 9 06:28 pages_shared -r--r--r-- 1 root root 4096 ago 9 06:28 pages_sharing -rw-r--r-- 1 root root 4096 ago 9 06:28 pages_to_scan -r--r--r-- 1 root root 4096 ago 9 06:28 pages_unshared -r--r--r-- 1 root root 4096 ago 9 06:28 pages_volatile -rw-r--r-- 1 root root 4096 ago 9 06:27 run -rw-r--r-- 1 root root 4096 ago 9 06:28 sleep_millisecs
# cat /sys/kernel/mm/ksm/max_kernel_pages 253500
But KSM seems not work:
# cat /sys/kernel/mm/ksm/pages_sharing 0
Should I have some additional consideration when compiling qemu-kvm?
Thanks for your reply.
Regards,
Daniel
Signature · 2 lines
Fingerprint: BFB3 08D6 B4D1 31B2 72B9 29CE 6696 BF1B 14E6 1D37 Powered by Debian GNU/Linux Lenny - Linux user #188.598
Quoted 16 lines · Daniel Bareiro
Quoted 5 lines · Michael Tokarev
Quoted 4 lines
I have understood that in addition to support from the kernel and qemu-kvm, there must be support in libc. So in testing I was doing on Lenny, it did not work using libc from Debian repositories. Now that squeeze is frozen, anyone knows if it will support KSM?
Quoted 7 lines
KSM is working in qemu-kvm packages in bpo, for quite long time already. It has nothing to do with libc, and now it does not even require recent-enough kernel headers to compile. What's needed is recent-enough kernel that actually implements the feature. The kernel in Lenny does not KSM support, but the one in bpo has. Also note that Lenny's kernel is too old for kvm.
Quoted 8 lines
I'm using Linux 2.6.32.3 compiled by myself with the kernel.org source code with support for KSM:
# cat /boot/config-2.6.32.3-dgb | grep KSM CONFIG_KSM=yI'm also using qemu-kvm 0.12.1.2 compiled by myself with the source code of SourceForge.
Note that this one has numerous bugs, some of which involves data corruption. JFYI.
Quoted 15 lines
These are the checks that I was doing in the VMHost:
# ll /sys/kernel/mm/ksm/ total 0(quoted diff) +0
-9-r--r--r-- 1 root root 4096 ago 9 06:28 full_scans -rw-r--r-- 1 root root 4096 ago 9 06:28 max_kernel_pages -r--r--r-- 1 root root 4096 ago 9 06:28 pages_shared -r--r--r-- 1 root root 4096 ago 9 06:28 pages_sharing -rw-r--r-- 1 root root 4096 ago 9 06:28 pages_to_scan -r--r--r-- 1 root root 4096 ago 9 06:28 pages_unshared -r--r--r-- 1 root root 4096 ago 9 06:28 pages_volatile -rw-r--r-- 1 root root 4096 ago 9 06:27 run -rw-r--r-- 1 root root 4096 ago 9 06:28 sleep_millisecs
Quoted 7 lines
# cat /sys/kernel/mm/ksm/max_kernel_pages 253500But KSM seems not work:
# cat /sys/kernel/mm/ksm/pages_sharing 0
Did you actually enable it in the kernel? Did you read the ksm docs shipped with the kernel?
Quoted 1 line
Should I have some additional consideration when compiling qemu-kvm?
Please note what Nikola Ciprich said. If you don't have kernel headers that defines MADV_MERGEABLE, you'll have to define it when compiling kvm.
Note again that debian qemu-kvm package (you're asking about debian after all) includes support for KSM. It also includes a small patch to #define that symbol.
/mjt
Hi, Michael.
Quoted 5 lines · Michael Tokarev
Quoted 2 lines
I'm also using qemu-kvm 0.12.1.2 compiled by myself with the source code of SourceForge.
Note that this one has numerous bugs, some of which involves data corruption. JFYI.
Thanks for the observation.
Quoted 18 lines
Quoted 18 lines
These are the checks that I was doing in the VMHost:
# ll /sys/kernel/mm/ksm/ total 0(quoted diff) +0
-9-r--r--r-- 1 root root 4096 ago 9 06:28 full_scans -rw-r--r-- 1 root root 4096 ago 9 06:28 max_kernel_pages -r--r--r-- 1 root root 4096 ago 9 06:28 pages_shared -r--r--r-- 1 root root 4096 ago 9 06:28 pages_sharing -rw-r--r-- 1 root root 4096 ago 9 06:28 pages_to_scan -r--r--r-- 1 root root 4096 ago 9 06:28 pages_unshared -r--r--r-- 1 root root 4096 ago 9 06:28 pages_volatile -rw-r--r-- 1 root root 4096 ago 9 06:27 run -rw-r--r-- 1 root root 4096 ago 9 06:28 sleep_millisecs# cat /sys/kernel/mm/ksm/max_kernel_pages 253500
Quoted 7 lines
Quoted 4 lines
But KSM seems not work:
# cat /sys/kernel/mm/ksm/pages_sharing 0Did you actually enable it in the kernel? Did you read the ksm docs shipped with the kernel?
Yes, I forgot to mention that I had enabled it:
# cat /sys/kernel/mm/ksm/run 1
Quoted 9 lines
Quoted 1 line
Should I have some additional consideration when compiling qemu-kvm?
Please note what Nikola Ciprich said. If you don't have kernel headers that defines
MADV_MERGEABLE, you'll have to define it when compiling kvm.Note again that debian qemu-kvm package (you're asking about debian after all) includes support for KSM. It also includes a small patch to #define that symbol.
Keeping the kernel I had compiled and installing the qemu-kvm package in Backports, now KSM is working:
# cat /sys/kernel/mm/ksm/pages_sharing 181406
This indicates that this was a problem in compiling qemu-kvm since the kernel is still exactly the same. Now, according to what I was seeing, headers of my kernel defines MADV_MERGEABLE:
# cat /usr/src/linux-2.6.32.3/debian/linux-headers-2.6.32.3-dgb/usr/src/linux-headers-2.6.32.3-dgb/include/asm-generic/mman-common.h | grep MERGEABLE #define MADV_MERGEABLE 12 /* KSM may merge identical pages */ #define MADV_UNMERGEABLE 13 /* KSM may not merge identical pages */
Would it be that perhaps in my compilation of qemu-kvm, ksm was not working because I would have to indicate the path /usr/src/linux-2.6.32.3/debian/...?
Thanks for your reply.
Regards,
Daniel
Signature · 2 lines
Fingerprint: BFB3 08D6 B4D1 31B2 72B9 29CE 6696 BF1B 14E6 1D37 Powered by Debian GNU/Linux Lenny - Linux user #188.598
Hi, all!
Quoted 5 lines · Daniel Bareiro
Keeping the kernel I had compiled and installing the qemu-kvm package in Backports, now KSM is working:
# cat /sys/kernel/mm/ksm/pages_sharing 181406
Looking at the statistics of the values obtained running 15 virtual machines totaling 10.7 GB on a 4 GB VMHost, I get the following, which is a very interesting memory savings:
# for ii in /sys/kernel/mm/ksm/* ; do echo -n "$ii: " ; cat $ii ; done /sys/kernel/mm/ksm/full_scans: 4114 /sys/kernel/mm/ksm/max_kernel_pages: 253500 /sys/kernel/mm/ksm/pages_shared: 67064 /sys/kernel/mm/ksm/pages_sharing: 510990 /sys/kernel/mm/ksm/pages_to_scan: 100 /sys/kernel/mm/ksm/pages_unshared: 448079 /sys/kernel/mm/ksm/pages_volatile: 13595 /sys/kernel/mm/ksm/run: 1 /sys/kernel/mm/ksm/sleep_millisecs: 20
# free
total used free shared buffers cached
Mem: 4056468 2578728 1477740 0 3736 62156
-/+ buffers/cache: 2512836 1543632
Swap: 497848 25972 471876Some recommendation about tunning of KSM?
I've no very clear about the difference between page_shared and page_sharing. Somebody could clarify it?
Thanks for your reply.
Regards,
Daniel
Signature · 2 lines
Fingerprint: BFB3 08D6 B4D1 31B2 72B9 29CE 6696 BF1B 14E6 1D37 Powered by Debian GNU/Linux Lenny - Linux user #188.598