KVM slow LAMP guest

Hi,

I have a LAMP server as guest in KVM. Whenever the server is idle for some time it takes about 30 seconds to load a Wordpress site.
If the server is not idle the site shows up in max 5 seconds. I've already turned of power management in the guest by passing

GRUB_CMDLINE_LINUX_DEFAULT="apm=off"

in /etc/default/grub. This has no effect.
Does KVM do some power management on guests? If so, how do I turn this off for my LAMP guest?

Best, Hansa

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 49 lines · Hansa

Quoted 32 lines · David Matlack

Quoted 29 lines · David Matlack

Quoted 10 lines · Hansa

Hi,

I have a LAMP server as guest in KVM. Whenever the server is idle for some
time it takes about 30 seconds to load a Wordpress site. If the server is not idle the site shows up in max 5 seconds. I've already
turned of power management in the guest by passing

Quoted 6 lines

GRUB_CMDLINE_LINUX_DEFAULT="apm=off"

in /etc/default/grub. This has no effect.
Does KVM do some power management on guests? If so, how do I turn this off
for my LAMP guest?

KVM doesn't do any power management of guests. But if everything is idle on
the host (including your guest), then host power management could kick in.
Have you tried playing with host pm?

Could you try running your workload with the guest kernel parameter "idle=poll"
and let me know the performance?

Quoted 6 lines

Quoted 3 lines

Also, if you are running Linux 4.0 or later on the host, could you try running
your workload with the KVM module parameter "halt_poll_ns=500000"?

Hansa-- Did you ever get a chance to run your workload with these changes?

Hi David,

Sorry I didn't respond to your mail earlier and nice of you to check back. To be honest no I haven't. To much work overload on my side. I've just added the idle=poll to the guests' grub and rebooted the server. I'll let it run a few day's to check if it helps. Btw: I'm running 2.6 linux kernel here so I'm not sure if I can add the "halt_poll_ns" param to KVM.

Quoted 1 line

Best, Hansa

Wow! That makes real difference!
The site was never that fast! Although CPU load is constantly on 100% any chance I can tweak this param? And is there a similar param for Windows guests?

(adding back the kvm mailing list)

Quoted 43 lines · C. Bröcker

Quoted 42 lines · Hansa

Quoted 28 lines · David Matlack

Quoted 26 lines · David Matlack

Quoted 9 lines · Hansa

Hi,

I have a LAMP server as guest in KVM. Whenever the server is idle for some
time it takes about 30 seconds to load a Wordpress site. If the server is not idle the site shows up in max 5 seconds. I've already
turned of power management in the guest by passing

Quoted 6 lines

GRUB_CMDLINE_LINUX_DEFAULT="apm=off"

in /etc/default/grub. This has no effect.
Does KVM do some power management on guests? If so, how do I turn this off
for my LAMP guest?

KVM doesn't do any power management of guests. But if everything is idle on
the host (including your guest), then host power management could kick in.
Have you tried playing with host pm?

Could you try running your workload with the guest kernel parameter "idle=poll"
and let me know the performance?

Quoted 5 lines

Quoted 3 lines

Also, if you are running Linux 4.0 or later on the host, could you try running
your workload with the KVM module parameter "halt_poll_ns=500000"?

Hansa-- Did you ever get a chance to run your workload with these changes?

Hi David,

Sorry I didn't respond to your mail earlier and nice of you to check back. To be honest no I haven't. To much work overload on my side. I've just added the idle=poll to the guests' grub and rebooted the server. I'll let it run a few day's to check if it helps. Btw: I'm running 2.6 linux kernel here so I'm not sure if I can add the "halt_poll_ns" param to KVM.

Quoted 5 lines

Quoted 1 line

Best, Hansa

Wow! That makes real difference!
The site was never that fast! Although CPU load is constantly on 100% any chance I can tweak this param? And is there a similar param for Windows guests?

Since idle=poll gave a noticeable improvement, your workload is a good candidate for halt_poll_ns. This is a host KVM module parameter. It will give you a similar speedup, it will work independent of your guest os, and won't peg your cpu usage at 100% during periods of no activity. If you don't have halt_poll_ns available (your host kernel older than 4.0), is upgrading to a newer kernel or backporting the patch [1] an option?

I'm not sure of any more guest params you can tweak (linux or windows), but I'll let you know if I come up with anything.

[1] https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=f7819512996361280b86259222456fcf15aad926

Quoted 7 lines · Hansa

Quoted 3 lines

[1]
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=f7819512996361280b86259222456fcf15aad926

Sorry David.
That is not an option. It's a production server on 2.6 kernel. 4.0 kernel is not in Base repos and I don't know which impact installing the newer kernel version will have on this server. Any other options?

You could backport the patch to the version you have.

What OS is this? Is it RHEL/CentOS? If so, halt_poll_ns will be in 6.7 which will be out in a few days/weeks.

Paolo

Quoted 7 lines · Hansa

Quoted 3 lines

[1]
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=f7819512996361280b86259222456fcf15aad926

Sorry David.
That is not an option. It's a production server on 2.6 kernel. 4.0 kernel is not in Base repos and I don't know which impact installing the newer kernel version will have on this server. Any other options?

You could backport the patch to the version you have.

What OS is this? Is it RHEL/CentOS? If so, halt_poll_ns will be in 6.7 which will be out in a few days/weeks.

Paolo

Quoted 5 lines

What OS is this? Is it RHEL/CentOS? If so, halt_poll_ns will be in 6.7 which will be out in a few days/weeks.

Paolo

OK. As said CentOS 6.6.
But where do I put this parameter?

Regards, Hansa

Quoted 6 lines · Hansa

Quoted 4 lines

What OS is this? Is it RHEL/CentOS? If so, halt_poll_ns will be in 6.7 which will be out in a few days/weeks.

Paolo

OK. As said CentOS 6.6.
But where do I put this parameter?

You can add "kvm.halt_poll_ns=500000" to the kernel command line. If you have the parameter, you have the /sys/module/kvm/parameters/halt_poll_ns file.

Quoted 6 lines · C. Bröcker

Quoted 4 lines

What OS is this? Is it RHEL/CentOS? If so, halt_poll_ns will be in 6.7 which will be out in a few days/weeks.

Paolo

OK. As said CentOS 6.6.
But where do I put this parameter?

You can add "kvm.halt_poll_ns=500000" to the kernel command line. If you have the parameter, you have the /sys/module/kvm/parameters/halt_poll_ns file.

Quoted 7 lines · Paolo Bonzini

Quoted 6 lines · C. Bröcker

Quoted 4 lines

What OS is this? Is it RHEL/CentOS? If so, halt_poll_ns will be in 6.7 which will be out in a few days/weeks.

Paolo

OK. As said CentOS 6.6.
But where do I put this parameter?

Quoted 3 lines

You can add "kvm.halt_poll_ns=500000" to the kernel command line. If you have the parameter, you have the /sys/module/kvm/parameters/halt_poll_ns file.

Hi,

I upgraded to the CentOS 6.7 release which came out last month and as promised the halt_poll_ns parameter was available.
Last week I tested the availability status every 5 minutes on my Wordpress VM's with the halt_poll_ns kernel param set on DOM0. I'm pleased to announce that it solves the problem!

My guess is that a lot people out there have the same problem but do not know the solution. Why not put it in the kernel by default?

Thanks a lot for all the help!
- Hansa

Signature · 3 lines

To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Quoted 13 lines · Hansa

Quoted 10 lines · Paolo Bonzini

Quoted 6 lines · C. Bröcker

Quoted 4 lines

What OS is this? Is it RHEL/CentOS? If so, halt_poll_ns will be in 6.7 which will be out in a few days/weeks.

Paolo

OK. As said CentOS 6.6.
But where do I put this parameter?

You can add "kvm.halt_poll_ns=500000" to the kernel command line. If you have the parameter, you have the /sys/module/kvm/parameters/halt_poll_ns file.

Hi,

Quoted 2 lines

I upgraded to the CentOS 6.7 release which came out last month and as promised the halt_poll_ns parameter was available.
Last week I tested the availability status every 5 minutes on my Wordpress VM's with the halt_poll_ns kernel param set on DOM0. I'm pleased to announce that it solves the problem!

How much seconds to load your Wordpress site this time?

Regards,
Wanpeng Li

Quoted 13 lines · Hansa

Quoted 10 lines · Paolo Bonzini

Quoted 6 lines · C. Bröcker

Quoted 4 lines

What OS is this? Is it RHEL/CentOS? If so, halt_poll_ns will be in 6.7 which will be out in a few days/weeks.

Paolo

OK. As said CentOS 6.6.
But where do I put this parameter?

You can add "kvm.halt_poll_ns=500000" to the kernel command line. If you have the parameter, you have the /sys/module/kvm/parameters/halt_poll_ns file.

Hi,

Quoted 2 lines

I upgraded to the CentOS 6.7 release which came out last month and as promised the halt_poll_ns parameter was available.
Last week I tested the availability status every 5 minutes on my Wordpress VM's with the halt_poll_ns kernel param set on DOM0. I'm pleased to announce that it solves the problem!

How much seconds to load your Wordpress site this time?

Regards,
Wanpeng Li

Quoted 14 lines · Wanpeng Li

Quoted 13 lines · Hansa

Quoted 10 lines · Paolo Bonzini

Quoted 6 lines · C. Bröcker

Quoted 4 lines

What OS is this? Is it RHEL/CentOS? If so, halt_poll_ns will be in 6.7 which will be out in a few days/weeks.

Paolo

OK. As said CentOS 6.6.
But where do I put this parameter?

You can add "kvm.halt_poll_ns=500000" to the kernel command line. If you have the parameter, you have the /sys/module/kvm/parameters/halt_poll_ns file.

Hi,

Quoted 7 lines

Quoted 2 lines

I upgraded to the CentOS 6.7 release which came out last month and as promised the halt_poll_ns parameter was available.
Last week I tested the availability status every 5 minutes on my Wordpress VM's with the halt_poll_ns kernel param set on DOM0. I'm pleased to announce that it solves the problem!

How much seconds to load your Wordpress site this time?

Regards,
Wanpeng Li

The average is around 0.4 seconds to load my "heaviest" site on my slowest machine.

On the VM server I issued the command below every eleven minutes:

date >> curltest-file; _
top -b -n 1 | sed -n '7,12p' >> curltest-file; _
curl -o /dev/null -s -w"time_total: %{time_total}\\n" https://my.domain.com | perl -pe 'BEGIN {use POSIX;} print strftime("%Y-%m-%d %H:%M:%S ", localtime)' >> curltest-file

This gives me the total time for displaying my site on a local machine. It also includes a 'top' command to display which processes are running at each sample. All is saved in a file called curltest-file.

I found 7 occurrences in my curltest-file of a time_total larger than 20 seconds. Top however didn't show any significant CPU or IO activity at those sampled times. Further investigations shows me that they are related to a known (gravatar) issue in the Wordpress Jetpack plugin. I didn't include these samples in the average total.

Cheers and good luck tweaking your sites!
Best, Hansa

Signature · 3 lines

To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Quoted 6 lines

On the VM server I issued the command below every eleven minutes:

date >> curltest-file; _
top -b -n 1 | sed -n '7,12p' >> curltest-file; _
curl -o /dev/null -s -w"time_total: %{time_total}\\n" https://my.domain.com | perl -pe 'BEGIN {use POSIX;} print strftime("%Y-%m-%d %H:%M:%S ", localtime)' >> curltest-file

Quoted 3 lines

This gives me the total time for displaying my site on a local machine. It also includes a 'top' command to display which processes are running at each sample. All is saved in a file called curltest-file.

I found 7 occurrences in my curltest-file of a time_total larger than 20 seconds. Top however didn't show any significant CPU or IO activity at those sampled times. Further investigations shows me that they are related to a known (gravatar) issue in the Wordpress Jetpack plugin. I didn't include these samples in the average total.

If you just use halt_poll_ns or both halt_poll_ns and idle=poll in guest?

Quoted 19 lines · Wanpeng Li

Quoted 18 lines

On the VM server I issued the command below every eleven minutes:

date >> curltest-file; _
top -b -n 1 | sed -n '7,12p' >> curltest-file; _ curl -o /dev/null -s -w"time_total: %{time_total}\\n"
https://my.domain.com | perl -pe 'BEGIN {use POSIX;} print strftime("%Y-%m-%d %H:%M:%S ", localtime)' >> curltest-file

This gives me the total time for displaying my site on a local machine. It also includes a 'top' command to display which processes are running at each sample. All is saved in a file called curltest-file.

I found 7 occurrences in my curltest-file of a time_total larger than 20 seconds. Top however didn't show any significant CPU or IO activity at those sampled times. Further investigations shows me that they are related to a known (gravatar) issue in the Wordpress Jetpack plugin. I didn't include these samples in the average total.

Quoted 3 lines

If you just use halt_poll_ns or both halt_poll_ns and idle=poll in guest?

I just use kvm.halt_poll_ns=500000
Should I try some different tests?