kvm with Windows optimization

This is a request for testing of an experimental kvm feature that dramatically accelerates some Windows releases (when running with the ACPI HAL, and especially with guest SMP). The feature detects accesses by the guest to the Task Priority Register (TPR) and patches them at runtime to a kvm-friendly code snipped that is provided by the BIOS.

The upshot of all that is that
http://kvm.qumranet.com/kvmwiki/Windows_ACPI_Workaround is no longer required, and Windows SMP no longer runs like a dog; it's quite snappy from my very limited testing.

Please download the test release from
http://people.qumranet.com/avi/tpr-opt-1.tar.gz and give it a spin. Once it has received some exposure, I'll merge it into mainline.

Credit for the original idea is due to Ben Serebrin.

--
error compiling committee.c: too many arguments to function

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

On compilation I get the following error.

../qemu-kvm.h:41: error: redefinition of 'kvm_log_tpr' ../qemu-kvm.h:41: error: previous definition of 'kvm_log_tpr' was here make[2]: *** [i8259.o] Error 1
make[2]: Leaving directory `/root/Download/software/tpr-opt-1/qemu/x86_64-softmmu' make[1]: *** [subdir-x86_64-softmmu] Error 2
make[1]: Leaving directory `/root/Download/software/tpr-opt-1/qemu'
make: *** [qemu] Error 2

Any help would be appreciated. Thanks.

Quoted 11 lines · Haydn Solomon

On compilation I get the following error.

../qemu-kvm.h:41: error: redefinition of 'kvm_log_tpr' ../qemu-kvm.h:41: error: previous definition of 'kvm_log_tpr' was here make[2]: *** [i8259.o] Error 1
make[2]: Leaving directory `/root/Download/software/tpr-opt-1/qemu/x86_64-softmmu' make[1]: *** [subdir-x86_64-softmmu] Error 2
make[1]: Leaving directory `/root/Download/software/tpr-opt-1/qemu'
make: *** [qemu] Error 2

Quoted 1 line

Any help would be appreciated. Thanks.

I uploaded a new version that fixes this. Same location.

Now back to reviewing and applying patches.

--
error compiling committee.c: too many arguments to function

Thanks, that was quick.

Quoted 2 lines · On Thu, 25 Oct 2007 13:39:33 -0400 "Haydn Solomon"…

Thanks, that was quick.

Finally, this (pre)release solved the long standing Java IE Plugin lockup problem for me. Thanks a lot!

Quoted 11 lines · Jindrich Makovicka

Quoted 2 lines · On Thu, 25 Oct 2007 13:39:33 -0400 "Haydn Solomon"…

Thanks, that was quick.

Finally, this (pre)release solved the long standing Java IE Plugin lockup problem for me. Thanks a lot!

No, this release is meant to cause problems, not fix them.

(but thanks for the reminder)

--
error compiling committee.c: too many arguments to function

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

I also had a problem installing windows xp. Got the following error

Quoted 2 lines · Haydn Solomon

I also had a problem installing windows xp. Got the following error

I guess I'll have to test that too.

Quoted 6 lines · Avi Kivity

Quoted 2 lines · Haydn Solomon

I also had a problem installing windows xp. Got the following error

Quoted 2 lines

I guess I'll have to test that too.

I hit it again, this time about half-way through the second stage install of Windows XP (last time it was almost immediately). This is with your updated tree. I'll next try to reproduce with the latest git.

static int mmu_topup_memory_caches(struct kvm_vcpu *vcpu)

{
    int r;

    kvm_mmu_free_some_pages(vcpu);
    r = mmu_topup_memory_cache(&vcpu->mmu_pte_chain_cache,
                   pte_chain_cache, 4);
    if (r)
        goto out;
    r = mmu_topup_memory_cache(&vcpu->mmu_rmap_desc_cache,
                   rmap_desc_cache, 1);
    if (r)
        goto out;

    r = mmu_topup_memory_cache_page(&vcpu->mmu_page_cache, 8);
    if (r)
        goto out;
    r = mmu_topup_memory_cache(&vcpu->mmu_page_header_cache,
                   mmu_page_header_cache, 4);
out:
    return r;
}

Quoted 7 lines · Anthony Liguori

Quoted 6 lines · Avi Kivity

Quoted 2 lines · Haydn Solomon

I also had a problem installing windows xp. Got the following error

Quoted 6 lines

Quoted 2 lines

I guess I'll have to test that too.

I hit it again, this time about half-way through the second stage install of Windows XP (last time it was almost immediately). This is with your updated tree. I'll next try to reproduce with the latest git.

Can you try upping the 'min' parameter below:

Quoted 13 lines

static int mmu_topup_memory_caches(struct kvm_vcpu *vcpu)

{
    int r;

    kvm_mmu_free_some_pages(vcpu);
    r = mmu_topup_memory_cache(&vcpu->mmu_pte_chain_cache,
                   pte_chain_cache, 4);
    if (r)
        goto out;
    r = mmu_topup_memory_cache(&vcpu->mmu_rmap_desc_cache,
                   rmap_desc_cache, 1);
    if (r)
        goto out;

Quoted 8 lines

    r = mmu_topup_memory_cache_page(&vcpu->mmu_page_cache, 8);
    if (r)
        goto out;
    r = mmu_topup_memory_cache(&vcpu->mmu_page_header_cache,
                   mmu_page_header_cache, 4);
out:
    return r;
}

These are the (4, 1, 8, 4) values in the call to mmu_topup_memory_cache. Perhaps one of them is too low.

Quoted 24 lines

Quoted 22 lines

static int mmu_topup_memory_caches(struct kvm_vcpu *vcpu)

{
    int r;

    kvm_mmu_free_some_pages(vcpu);
    r = mmu_topup_memory_cache(&vcpu->mmu_pte_chain_cache,
                   pte_chain_cache, 4);
    if (r)
        goto out;
    r = mmu_topup_memory_cache(&vcpu->mmu_rmap_desc_cache,
                   rmap_desc_cache, 1);
    if (r)
        goto out;
    r = mmu_topup_memory_cache_page(&vcpu->mmu_page_cache, 8);
    if (r)
        goto out;
    r = mmu_topup_memory_cache(&vcpu->mmu_page_header_cache,
                   mmu_page_header_cache, 4);
out:
    return r;
}

Quoted 2 lines

These are the (4, 1, 8, 4) values in the call to mmu_topup_memory_cache. Perhaps one of them is too low.

Sure. Would this be affected at all by your tpr patch?

IIUC, if this is the problem, it should be reproducible with the latest git too?

Quoted 24 lines · Anthony Liguori

Quoted 24 lines

Quoted 22 lines

static int mmu_topup_memory_caches(struct kvm_vcpu *vcpu)

{
    int r;

    kvm_mmu_free_some_pages(vcpu);
    r = mmu_topup_memory_cache(&vcpu->mmu_pte_chain_cache,
                   pte_chain_cache, 4);
    if (r)
        goto out;
    r = mmu_topup_memory_cache(&vcpu->mmu_rmap_desc_cache,
                   rmap_desc_cache, 1);
    if (r)
        goto out;
    r = mmu_topup_memory_cache_page(&vcpu->mmu_page_cache, 8);
    if (r)
        goto out;
    r = mmu_topup_memory_cache(&vcpu->mmu_page_header_cache,
                   mmu_page_header_cache, 4);
out:
    return r;
}

Quoted 4 lines

Quoted 2 lines

These are the (4, 1, 8, 4) values in the call to mmu_topup_memory_cache. Perhaps one of them is too low.

Sure. Would this be affected at all by your tpr patch?

I believe not, but the code doesn't care what I believe.

Quoted 2 lines

IIUC, if this is the problem, it should be reproducible with the latest git too?

One difference is that the tpr patch disables nx. That causes Windows to go into 32-bit paging mode (nice that it has both pae and nonpae in the same kernel), which may change things.

You can try booting your host with nx disabled to get the same effect (or disable nx cpuid in kvm).

--
error compiling committee.c: too many arguments to function

I was able to complete an installation of windows xp using -smp 1. However, I would like to test on an existing virtual machine with applications already installed on it. Changing the HAL to ACPI should be enough for me to test the windows optimization right feature right?

Quoted 24 lines · Anthony Liguori

Quoted 24 lines

Quoted 22 lines

static int mmu_topup_memory_caches(struct kvm_vcpu *vcpu)

{
    int r;

    kvm_mmu_free_some_pages(vcpu);
    r = mmu_topup_memory_cache(&vcpu->mmu_pte_chain_cache,
                   pte_chain_cache, 4);
    if (r)
        goto out;
    r = mmu_topup_memory_cache(&vcpu->mmu_rmap_desc_cache,
                   rmap_desc_cache, 1);
    if (r)
        goto out;
    r = mmu_topup_memory_cache_page(&vcpu->mmu_page_cache, 8);
    if (r)
        goto out;
    r = mmu_topup_memory_cache(&vcpu->mmu_page_header_cache,
                   mmu_page_header_cache, 4);
out:
    return r;
}

Quoted 4 lines

Quoted 2 lines

These are the (4, 1, 8, 4) values in the call to mmu_topup_memory_cache. Perhaps one of them is too low.

Sure. Would this be affected at all by your tpr patch?

I believe not, but the code doesn't care what I believe.

Quoted 2 lines

IIUC, if this is the problem, it should be reproducible with the latest git too?

One difference is that the tpr patch disables nx. That causes Windows to go into 32-bit paging mode (nice that it has both pae and nonpae in the same kernel), which may change things.

You can try booting your host with nx disabled to get the same effect (or disable nx cpuid in kvm).

Quoted 35 lines · Avi Kivity

Quoted 22 lines · Anthony Liguori

Quoted 22 lines

Quoted 22 lines

static int mmu_topup_memory_caches(struct kvm_vcpu *vcpu)

{
    int r;

    kvm_mmu_free_some_pages(vcpu);
    r = mmu_topup_memory_cache(&vcpu->mmu_pte_chain_cache,
                   pte_chain_cache, 4);
    if (r)
        goto out;
    r = mmu_topup_memory_cache(&vcpu->mmu_rmap_desc_cache,
                   rmap_desc_cache, 1);
    if (r)
        goto out;
    r = mmu_topup_memory_cache_page(&vcpu->mmu_page_cache, 8);
    if (r)
        goto out;
    r = mmu_topup_memory_cache(&vcpu->mmu_page_header_cache,
                   mmu_page_header_cache, 4);
out:
    return r;
}

Quoted 5 lines

Quoted 3 lines

These are the (4, 1, 8, 4) values in the call to mmu_topup_memory_cache. Perhaps one of them is too low.

Sure. Would this be affected at all by your tpr patch?

I believe not, but the code doesn't care what I believe.

Quoted 3 lines

IIUC, if this is the problem, it should be reproducible with the latest git too?

Quoted 7 lines

One difference is that the tpr patch disables nx. That causes Windows to go into 32-bit paging mode (nice that it has both pae and nonpae in the same kernel), which may change things.

You can try booting your host with nx disabled to get the same effect (or disable nx cpuid in kvm).

I've disabled NX in KVM and that didn't reproduce the issue in the current git.

If I double all of the memory caches, I can't seem to reproduce. However, as soon as I reduce rmap_desc_cache down to 1, I can reproduce.

I'll try to see if just setting the rmap_desc_cache line to 2 is enough to make the problem go away.

How can the guest provoke this BUG() based on the cache size? Should the cache size only affect performance?

Quoted 47 lines · Anthony Liguori

Quoted 34 lines · Avi Kivity

Quoted 22 lines · Anthony Liguori

Quoted 22 lines

Quoted 22 lines

static int mmu_topup_memory_caches(struct kvm_vcpu *vcpu)

{
    int r;

    kvm_mmu_free_some_pages(vcpu);
    r = mmu_topup_memory_cache(&vcpu->mmu_pte_chain_cache,
                   pte_chain_cache, 4);
    if (r)
        goto out;
    r = mmu_topup_memory_cache(&vcpu->mmu_rmap_desc_cache,
                   rmap_desc_cache, 1);
    if (r)
        goto out;
    r = mmu_topup_memory_cache_page(&vcpu->mmu_page_cache, 8);
    if (r)
        goto out;
    r = mmu_topup_memory_cache(&vcpu->mmu_page_header_cache,
                   mmu_page_header_cache, 4);
out:
    return r;
}

Quoted 4 lines

Quoted 3 lines

These are the (4, 1, 8, 4) values in the call to mmu_topup_memory_cache. Perhaps one of them is too low.

Sure. Would this be affected at all by your tpr patch?

I believe not, but the code doesn't care what I believe.

Quoted 3 lines

IIUC, if this is the problem, it should be reproducible with the latest git too?

Quoted 7 lines

One difference is that the tpr patch disables nx. That causes Windows to go into 32-bit paging mode (nice that it has both pae and nonpae in the same kernel), which may change things.

You can try booting your host with nx disabled to get the same effect (or disable nx cpuid in kvm).

I've disabled NX in KVM and that didn't reproduce the issue in the current git.

Quoted 10 lines

If I double all of the memory caches, I can't seem to reproduce. However, as soon as I reduce rmap_desc_cache down to 1, I can reproduce.

I'll try to see if just setting the rmap_desc_cache line to 2 is enough to make the problem go away.

How can the guest provoke this BUG() based on the cache size? Should the cache size only affect performance?

The memory caches are a little misnamed; they're really preallocation buffers.

They serve two purposes: to avoid allocation in atomic contexts (that's no longer needed since preempt notifiers) and to avoid complex error recovery paths. We make sure there are enough objects to satisfy worst case behavior and assume all allocations will work.

Regarding the rmap memory cache failure, I can't think of a reason why we'll need to add more than one rmap entry per fault.

Quoted 24 lines · Anthony Liguori

Quoted 24 lines

Quoted 22 lines

static int mmu_topup_memory_caches(struct kvm_vcpu *vcpu)

{
    int r;

    kvm_mmu_free_some_pages(vcpu);
    r = mmu_topup_memory_cache(&vcpu->mmu_pte_chain_cache,
                   pte_chain_cache, 4);
    if (r)
        goto out;
    r = mmu_topup_memory_cache(&vcpu->mmu_rmap_desc_cache,
                   rmap_desc_cache, 1);
    if (r)
        goto out;
    r = mmu_topup_memory_cache_page(&vcpu->mmu_page_cache, 8);
    if (r)
        goto out;
    r = mmu_topup_memory_cache(&vcpu->mmu_page_header_cache,
                   mmu_page_header_cache, 4);
out:
    return r;
}

Quoted 3 lines

Quoted 1 line

These are the (4, 1, 8, 4) values in the call to mmu_topup_memory_cache. Perhaps one of them is too low.

Sure. Would this be affected at all by your tpr patch?

Quoted 25 lines · Avi Kivity

Quoted 24 lines · Anthony Liguori

Quoted 24 lines

Quoted 22 lines

static int mmu_topup_memory_caches(struct kvm_vcpu *vcpu)

{
    int r;

    kvm_mmu_free_some_pages(vcpu);
    r = mmu_topup_memory_cache(&vcpu->mmu_pte_chain_cache,
                   pte_chain_cache, 4);
    if (r)
        goto out;
    r = mmu_topup_memory_cache(&vcpu->mmu_rmap_desc_cache,
                   rmap_desc_cache, 1);
    if (r)
        goto out;
    r = mmu_topup_memory_cache_page(&vcpu->mmu_page_cache, 8);
    if (r)
        goto out;
    r = mmu_topup_memory_cache(&vcpu->mmu_page_header_cache,
                   mmu_page_header_cache, 4);
out:
    return r;
}

Quoted 4 lines

Quoted 4 lines

Quoted 2 lines

These are the (4, 1, 8, 4) values in the call to mmu_topup_memory_cache. Perhaps one of them is too low.

Sure. Would this be affected at all by your tpr patch?

Using the following patch, I'm getting really good results. I'll keep running test cases tonight and see if I can trigger anything.

Windows is *much* faster with your patch, very nice work!

Regards,

Anthony Liguori

Using the following patch, I'm getting really good results.

Quoted 6 lines · Avi Kivity

Quoted 2 lines · Haydn Solomon

I also had a problem installing windows xp. Got the following error

Quoted 2 lines

I guess I'll have to test that too.

I hit it again, this time about half-way through the second stage install of Windows XP (last time it was almost immediately). This is with your updated tree. I'll next try to reproduce with the latest git.

Regards,

Anthony Liguori

[ 1577.621100] ------------[ cut here ]------------
[ 1577.621106] kernel BUG at
/home/anthony/tmp/2/tpr-opt-1/kernel/mmu.c:308!
[ 1577.621108] invalid opcode: 0000 [1] SMP
[ 1577.621111] CPU 1
[ 1577.621113] Modules linked in: tun kvm_intel kvm i915 drm af_packet 

rfcomm l2cap thinkpad_acpi ppdev acpi_cpufreq cpufreq_stats cpufreq_ondemand freq_table cpufreq_userspace cpufreq_conservative cpufreq_powersave sbs ac battery bay dock container button video ipv6 bridge ipt_REJECT xt_state xt_tcpudp iptable_filter ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nfnetlink ip_tables x_tables sbp2 lp arc4 ecb blkcipher snd_hda_intel snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy iwl4965 iwlwifi_mac80211 snd_seq_oss parport_pc snd_seq_midi snd_rawmidi parport cfg80211 pcmcia hci_usb bluetooth snd_seq_midi_event snd_seq snd_timer snd_seq_device intel_agp sdhci mmc_core serio_raw psmouse pcspkr yenta_socket rsrc_nonstatic pcmcia_core snd soundcore shpchp pci_hotplug snd_page_alloc joydev evdev ext3 jbd mbcache usbhid hid sg sr_mod sd_mod cdrom ata_piix ehci_hcd ohci1394 ieee1394 ata_generic libata scsi_mod uhci_hcd usbcore e1000 thermal processor fan fuse apparmor commoncap

[ 1577.621172] Pid: 7788, comm: qemu-system-x86 Not tainted 
2.6.22-14-generic #1
[ 1577.621175] RIP: 0010:[<ffffffff8837d905>]  [<ffffffff8837d905>] 
:kvm:mmu_memory_cache_alloc+0x25/0x30
[ 1577.621188] RSP: 0018:ffff8100441819b8  EFLAGS: 00010246
[ 1577.621190] RAX: 0000000000000000 RBX: ffffc200010c1968 RCX: 
ffff81001eebc080
[ 1577.621193] RDX: ffff81001eebc088 RSI: 0000000000000028 RDI: 
ffff8100388543e0
[ 1577.621195] RBP: ffff81000377f060 R08: 0000000000000000 R09: 
0000000000000004
[ 1577.621197] R10: 00000000081ed163 R11: 0000000000000000 R12: 
ffff810038854000
[ 1577.621200] R13: 0000000000000000 R14: ffff81000377f060 R15: 
ffff810038854000
[ 1577.621202] FS:  00002b7827310d40(0000) GS:ffff81000102c280(0000) 
knlGS:0000000000000000
[ 1577.621205] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
[ 1577.621207] CR2: 0000000001002ff8 CR3: 0000000054f45000 CR4: 
00000000000026e0
[ 1577.621210] Process qemu-system-x86 (pid: 7788, threadinfo 
ffff810044180000, task ffff81000d4d8dc0)
[ 1577.621212] Stack:  ffffc200010c1968 ffffffff8837da16 
00000000441c6043 0000000000000002
[ 1577.621216]  00000000441c6000 ffffffff8837e1ad 00002b7828552000 
00000000000081ed
[ 1577.621220]  0000000000000000 0000000000000000 00000000081ed000 
00000000081ed000
[ 1577.621223] Call Trace:
[ 1577.621233]  [<ffffffff8837da16>] :kvm:rmap_add+0xf6/0x110
[ 1577.621243]  [<ffffffff8837e1ad>] 
:kvm:paging32_set_pte_common+0x10d/0x2b0
[ 1577.621258]  [<ffffffff8837e399>] :kvm:paging32_set_pte+0x49/0x50
[ 1577.621269]  [<ffffffff8837e9dd>] :kvm:kvm_mmu_pte_write+0x33d/0x3b0
[ 1577.621287]  [<ffffffff883786cd>] 
:kvm:emulator_write_emulated_onepage+0x9d/0x100
[ 1577.621298]  [<ffffffff88380da8>] :kvm:x86_emulate_insn+0x5e8/0x4450
[ 1577.621308]  [<ffffffff88396467>] :kvm_intel:vmcs_readl+0x17/0x20
[ 1577.621320]  [<ffffffff88378cff>] :kvm:emulate_instruction+0x1af/0x2f0
[ 1577.621328]  [<ffffffff88399090>]
:kvm_intel:handle_exception+0x240/0x2b0
[ 1577.621336]  [<ffffffff88385ab9>] :kvm:apic_update_ppr+0x29/0x60
[ 1577.621347]  [<ffffffff8837aa74>] :kvm:kvm_vcpu_ioctl+0xe54/0x11b0
[ 1577.621355]  [<ffffffff8026e000>] file_read_actor+0x0/0x150
[ 1577.621366]  [<ffffffff8022cacc>] task_rq_lock+0x4c/0x90
[ 1577.621372]  [<ffffffff8022f30c>] try_to_wake_up+0x5c/0x3f0
[ 1577.621385]  [<ffffffff802a84f5>] core_sys_select+0x275/0x300
[ 1577.621391]  [<ffffffff8022ca03>] __wake_up+0x43/0x70
[ 1577.621402]  [<ffffffff80240719>] __dequeue_signal+0x19/0x1d0
[ 1577.621409]  [<ffffffff8023ffce>] recalc_sigpending+0xe/0x30
[ 1577.621412]  [<ffffffff8024212c>] dequeue_signal+0x5c/0x190
[ 1577.621416]  [<ffffffff8024f4f9>] getnstimeofday+0x39/0xa0
[ 1577.621423]  [<ffffffff802a6e05>] do_ioctl+0x35/0xe0
[ 1577.621428]  [<ffffffff802a6f24>] vfs_ioctl+0x74/0x2d0
[ 1577.621435]  [<ffffffff802a7215>] sys_ioctl+0x95/0xb0
[ 1577.621442]  [<ffffffff80209e8e>] system_call+0x7e/0x83
[ 1577.621452]
[ 1577.621453]
[ 1577.621454] Code: 0f 0b eb fe 0f 1f 80 00 00 00 00 48 81 c7 e0 03 00 
00 be 28
[ 1577.621462] RIP  [<ffffffff8837d905>] 
:kvm:mmu_memory_cache_alloc+0x25/0x30
[ 1577.621471]  RSP <ffff8100441819b8>

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Quoted 5 lines · Jindrich Makovicka

On Thu, 25 Oct 2007 13:39:33 -0400
"Haydn Solomon" <[email protected]> wrote:

Quoted 8 lines

Quoted 3 lines

Thanks, that was quick.

Finally, this (pre)release solved the long standing Java IE Plugin lockup problem for me. Thanks a lot!

No, this release is meant to cause problems, not fix them.

(but thanks for the reminder)

Avi Kivity wrote:
It's probably due to disabling NX (which is an unwanted side effect of this). Java generates executable code, so it probably needs to modify protection attributes. Maybe we're bad at this.

Can you check the attached patch? Against normal kvm, not the windows accelerator.

This is a request for testing of an experimental kvm feature that dramatically accelerates some Windows releases (when running with the ACPI HAL, and especially with guest SMP). The feature detects accesses by the guest to the Task Priority Register (TPR) and patches them at runtime to a kvm-friendly code snipped that is provided by the BIOS.

The upshot of all that is that http://kvm.qumranet.com/kvmwiki/Windows_ACPI_Workaround is no longer required, and Windows SMP no longer runs like a dog; it's quite snappy from my very limited testing.

Please download the test release from http://people.qumranet.com/avi/tpr-opt-1.tar.gz and give it a spin. Once it has received some exposure, I'll merge it into mainline.

Credit for the original idea is due to Ben Serebrin.

Quoted 16 lines · Avi Kivity

This is a request for testing of an experimental kvm feature that dramatically accelerates some Windows releases (when running with the ACPI HAL, and especially with guest SMP). The feature detects accesses by the guest to the Task Priority Register (TPR) and patches them at runtime to a kvm-friendly code snipped that is provided by the BIOS.

The upshot of all that is that
http://kvm.qumranet.com/kvmwiki/Windows_ACPI_Workaround is no longer required, and Windows SMP no longer runs like a dog; it's quite snappy from my very limited testing.

Please download the test release from
http://people.qumranet.com/avi/tpr-opt-1.tar.gz and give it a spin. Once it has received some exposure, I'll merge it into mainline.

Quoted 3 lines

Credit for the original idea is due to Ben Serebrin.

Sweet!

I needed the following patch to compile. Will let you know how testing goes.

Quoted 21 lines · Anthony Liguori

Quoted 19 lines · Avi Kivity

This is a request for testing of an experimental kvm feature that dramatically accelerates some Windows releases (when running with the ACPI HAL, and especially with guest SMP). The feature detects accesses by the guest to the Task Priority Register (TPR) and patches them at runtime to a kvm-friendly code snipped that is provided by the BIOS.

The upshot of all that is that
http://kvm.qumranet.com/kvmwiki/Windows_ACPI_Workaround is no longer required, and Windows SMP no longer runs like a dog; it's quite snappy from my very limited testing.

Please download the test release from
http://people.qumranet.com/avi/tpr-opt-1.tar.gz and give it a spin. Once it has received some exposure, I'll merge it into mainline.

Credit for the original idea is due to Ben Serebrin.

Quoted 5 lines

Sweet!

I needed the following patch to compile. Will let you know how testing goes.

I already applied that, plus another minor fix, so I recommend re-downloading. Should have given it a new name.

Quoted 29 lines · Avi Kivity

Quoted 27 lines · Anthony Liguori

Quoted 12 lines · Avi Kivity

This is a request for testing of an experimental kvm feature that dramatically accelerates some Windows releases (when running with the ACPI HAL, and especially with guest SMP). The feature detects accesses by the guest to the Task Priority Register (TPR) and patches them at runtime to a kvm-friendly code snipped that is provided by the BIOS.

The upshot of all that is that
http://kvm.qumranet.com/kvmwiki/Windows_ACPI_Workaround is no longer required, and Windows SMP no longer runs like a dog; it's quite snappy from my very limited testing.

Quoted 8 lines

Please download the test release from
http://people.qumranet.com/avi/tpr-opt-1.tar.gz and give it a spin. Once it has received some exposure, I'll merge it into mainline.

Credit for the original idea is due to Ben Serebrin.

Sweet!

I needed the following patch to compile. Will let you know how testing goes.

Quoted 2 lines

I already applied that, plus another minor fix, so I recommend re-downloading. Should have given it a new name.

What's the minor fix? During the second stage of installing winxp, I just got the following BUG(). Have to reboot to check and see if this is specific to your branch.

This is a request for testing of an experimental kvm feature that dramatically accelerates some Windows releases (when running with the ACPI HAL, and especially with guest SMP). The feature detects accesses by the guest to the Task Priority Register (TPR) and patches them at runtime to a kvm-friendly code snipped that is provided by the BIOS.

The upshot of all that is that
http://kvm.qumranet.com/kvmwiki/Windows_ACPI_Workaround is no longer required, and Windows SMP no longer runs like a dog; it's quite snappy from my very limited testing.

Please download the test release from
http://people.qumranet.com/avi/tpr-opt-1.tar.gz and give it a spin. Once it has received some exposure, I'll merge it into mainline.

Credit for the original idea is due to Ben Serebrin.

Quoted 15 lines · Avi Kivity

This is a request for testing of an experimental kvm feature that dramatically accelerates some Windows releases (when running with the ACPI HAL, and especially with guest SMP). The feature detects accesses by the guest to the Task Priority Register (TPR) and patches them at runtime to a kvm-friendly code snipped that is provided by the BIOS.

The upshot of all that is that
http://kvm.qumranet.com/kvmwiki/Windows_ACPI_Workaround is no longer required, and Windows SMP no longer runs like a dog; it's quite snappy from my very limited testing.

Please download the test release from
http://people.qumranet.com/avi/tpr-opt-1.tar.gz and give it a spin. Once it has received some exposure, I'll merge it into mainline.

Quoted 3 lines

Credit for the original idea is due to Ben Serebrin.

BTW, is it wise to enable this by default for all guests? Ignoring the fact that we're modifying guest's memory without its knowledge, if a guest unmaps the VA mappings for the BIOS then all sorts of problems could occur

BTW, is it wise to enable this by default for all guests? Ignoring the
fact that we're modifying guest's memory without its knowledge, if a guest unmaps the VA mappings for the BIOS then all sorts of problems could occur.

Quoted 7 lines

BTW, is it wise to enable this by default for all guests? Ignoring the
fact that we're modifying guest's memory without its knowledge, if a guest unmaps the VA mappings for the BIOS then all sorts of problems could occur.

Good movement!

But, Vista won't work with patching. The memory guard in vista will make the system refuse to service. Maybe we need a option to check if it is vista.

Quoted 9 lines · Eddie

Quoted 7 lines

BTW, is it wise to enable this by default for all guests? Ignoring the
fact that we're modifying guest's memory without its knowledge, if a guest unmaps the VA mappings for the BIOS then all sorts of problems could occur.

Good movement!

Quoted 5 lines

But, Vista won't work with patching. The memory guard in vista will make the system refuse to service. Maybe we need a option to check if it is vista.

Thanks for the hint; I'll add that. I think Vista hits the tpr much less frequently, so it can work well without the optimization.

Quoted 6 lines

Quoted 2 lines

Thanks for the hint; I'll add that. I think Vista hits the tpr much less frequently, so it can work well without the optimization.

Curious: How do u know ROM is mapped 0xf00000000 in Windows? Eddie

    kvm_get_sregs(kvm_context, env->cpu_index, &sregs);

    probe = (rip & 0xf0000000) + 0xe0000;
    phys = map_addr(&sregs, probe, &perms);
    if (phys != 0xe0000)
	return 0;
    printf("bios ok at %lx\n", (long)probe);

Quoted 7 lines

BTW, is it wise to enable this by default for all guests? Ignoring the
fact that we're modifying guest's memory without its knowledge, if a guest unmaps the VA mappings for the BIOS then all sorts of problems could occur.

Good movement!

But, Vista won't work with patching. The memory guard in vista will make the system refuse to service. Maybe we need a option to check if it is vista.

Eddie

Quoted 8 lines

Good movement!

But, Vista won't work with patching. The memory guard in vista will make the system refuse to service. Maybe we need a option to check if it is vista.

Are you sure that Vista isn't just verifying memory during startup/driver load? Seems odd to periodically scan memory.

In the recently released Windows Hypervisor specification, in section 13.2.2 (Local APIC Memory-mapped Accesses), it's dictated that the APIC can only be accessed through 6 instructions. They are all >= 5 byte instructions. When I first saw that, I figured they were also patching.

Quoted 11 lines · Anthony Liguori

Quoted 7 lines

Good movement!

But, Vista won't work with patching. The memory guard in vista will make the system refuse to service. Maybe we need a option to check if it is vista.

Are you sure that Vista isn't just verifying memory during startup/driver load? Seems odd to periodically scan memory.

Quoted 6 lines

In the recently released Windows Hypervisor specification, in section 13.2.2 (Local APIC Memory-mapped Accesses), it's dictated that the APIC can only be accessed through 6 instructions. They are all >= 5 byte instructions. When I first saw that, I figured they were also patching.

Six? My hack patches five. Which one did I miss?

Quoted 19 lines · Avi Kivity

Quoted 17 lines · Anthony Liguori

Quoted 7 lines

Good movement!

But, Vista won't work with patching. The memory guard in vista will make the system refuse to service. Maybe we need a option to check if it is vista.

Are you sure that Vista isn't just verifying memory during startup/driver load? Seems odd to periodically scan memory.

In the recently released Windows Hypervisor specification, in section 13.2.2 (Local APIC Memory-mapped Accesses), it's dictated that the APIC can only be accessed through 6 instructions. They are all >= 5 byte instructions. When I first saw that, I figured they were also patching.

Quoted 1 line

Six? My hack patches five. Which one did I miss?

FF /6 PUSH m32

Also note, that it says that all addresses are required to be 4-byte aligned.

Quoted 20 lines · Anthony Liguori

Quoted 19 lines · Avi Kivity

Quoted 17 lines · Anthony Liguori

Quoted 7 lines

Good movement!

But, Vista won't work with patching. The memory guard in vista will make the system refuse to service. Maybe we need a option to check if it is vista.

Are you sure that Vista isn't just verifying memory during startup/driver load? Seems odd to periodically scan memory.

In the recently released Windows Hypervisor specification, in section 13.2.2 (Local APIC Memory-mapped Accesses), it's dictated that the APIC can only be accessed through 6 instructions. They are all >= 5 byte instructions. When I first saw that, I figured they were also patching.

Quoted 4 lines

Quoted 1 line

Six? My hack patches five. Which one did I miss?

FF /6 PUSH m32

Thanks. That's easy to add.

Quoted 2 lines

Also note, that it says that all addresses are required to be 4-byte aligned.

Which addresses? The tpr address is obviously 4 byte aligned.

The instruction pointer?!

Quoted 7 lines

BTW, is it wise to enable this by default for all guests? Ignoring the
fact that we're modifying guest's memory without its knowledge, if a guest unmaps the VA mappings for the BIOS then all sorts of problems could occur.

Good movement!

But, Vista won't work with patching. The memory guard in vista will make the system refuse to service. Maybe we need a option to check if it is vista.

Eddie

Quoted 9 lines · Eddie

Quoted 7 lines

BTW, is it wise to enable this by default for all guests? Ignoring the
fact that we're modifying guest's memory without its knowledge, if a guest unmaps the VA mappings for the BIOS then all sorts of problems could occur.

Good movement!

Quoted 4 lines

But, Vista won't work with patching. The memory guard in vista will make the system refuse to service. Maybe we need a option to check if it is vista.

FWIW, I successfully installed and booted Vista Enterprise Edition with the TPR optimization patch.

Quoted 12 lines · Anthony Liguori

Quoted 10 lines · Eddie

Quoted 8 lines

BTW, is it wise to enable this by default for all guests? Ignoring the
fact that we're modifying guest's memory without its knowledge, if a guest unmaps the VA mappings for the BIOS then all sorts of problems could occur.

Good movement!

Quoted 9 lines

Quoted 5 lines

But, Vista won't work with patching. The memory guard in vista will make the system refuse to service. Maybe we need a option to check if it is vista.

FWIW, I successfully installed and booted Vista Enterprise Edition with the TPR optimization patch.

32 bits or 64 bits?
See
http://blogs.msdn.com/windowsvistasecurity/archive/2006/08/11/695993.asp
x

It says:

There are many brand new security features in Windows Vista, but Kernel Patch Protection is actually not one of them. Kernel Patch Protection was first supported on x64 (AMD64 and Intel EMT64T) CPU architecture versions of Microsoft Windows including Microsoft Windows Server 2003 SP1 and Windows XP Microsoft Windows XP Professional x64 Edition. (Patch protection is currently not supported on x86 or ia64 architectures.) Though, as the use of 64-bit computers is increasing, Windows Vista users will end up benefiting most from this technology.

Seems Vista 32 is not using patchguard yet, not sure about next SP. If above is true, it looks like we are OK for current stage. thx,eddie

Quoted 28 lines · Eddie

Quoted 9 lines · Anthony Liguori

Quoted 7 lines · Eddie

Quoted 7 lines

BTW, is it wise to enable this by default for all guests? Ignoring the
fact that we're modifying guest's memory without its knowledge, if a guest unmaps the VA mappings for the BIOS then all sorts of problems could occur.

Quoted 10 lines

Quoted 6 lines

Good movement!

But, Vista won't work with patching. The memory guard in vista will make the system refuse to service. Maybe we need a option to check if it is vista.

FWIW, I successfully installed and booted Vista Enterprise Edition with the TPR optimization patch.

32 bits or 64 bits?
See
http://blogs.msdn.com/windowsvistasecurity/archive/2006/08/11/695993.asp
x

It says:

Quoted 12 lines

There are many brand new security features in Windows Vista, but Kernel Patch Protection is actually not one of them. Kernel Patch Protection was first supported on x64 (AMD64 and Intel EMT64T) CPU architecture versions of Microsoft Windows including Microsoft Windows Server 2003 SP1 and Windows XP Microsoft Windows XP Professional x64 Edition. (Patch protection is currently not supported on x86 or ia64 architectures.) Though, as the use of 64-bit computers is increasing, Windows Vista users will end up benefiting most from this technology. Seems Vista 32 is not using patchguard yet, not sure about next SP. If above is true, it looks like we are OK for current stage. thx,eddie

I tested 32-bit Vista. We'll have to try 64-bit Vista and see if it gets upset.

Quoted 44 lines · Anthony Liguori

Quoted 42 lines · Eddie

Quoted 9 lines · Anthony Liguori

Quoted 7 lines · Eddie

Quoted 7 lines

BTW, is it wise to enable this by default for all guests? Ignoring the
fact that we're modifying guest's memory without its knowledge, if a guest unmaps the VA mappings for the BIOS then all sorts of problems could occur.

Quoted 10 lines

Quoted 6 lines

Good movement!

But, Vista won't work with patching. The memory guard in vista will make the system refuse to service. Maybe we need a option to check if it is vista.

FWIW, I successfully installed and booted Vista Enterprise Edition with the TPR optimization patch.

32 bits or 64 bits?
See
http://blogs.msdn.com/windowsvistasecurity/archive/2006/08/11/695993.asp

x

It says:

There are many brand new security features in Windows Vista, but Kernel Patch Protection is actually not one of them. Kernel Patch Protection was first supported on x64 (AMD64 and Intel EMT64T) CPU architecture versions of Microsoft Windows including Microsoft Windows Server 2003 SP1 and Windows XP Microsoft Windows XP Professional x64 Edition. (Patch
protection is currently not supported on x86 or ia64 architectures.) Though, as the use of 64-bit computers is increasing, Windows Vista users will end up benefiting most from this technology. Seems Vista 32 is not using patchguard yet, not sure about next SP. If above is true, it looks like we are OK for current stage. thx,eddie

Quoted 3 lines

I tested 32-bit Vista. We'll have to try 64-bit Vista and see if it gets upset.

btw, it may have not triggered because the hack didn't recognize Vista's memory map.

This is a request for testing of an experimental kvm feature that dramatically accelerates some Windows releases (when running with the ACPI HAL, and especially with guest SMP). The feature detects accesses by the guest to the Task Priority Register (TPR) and patches them at runtime to a kvm-friendly code snipped that is provided by the BIOS.

The upshot of all that is that
http://kvm.qumranet.com/kvmwiki/Windows_ACPI_Workaround is no longer required, and Windows SMP no longer runs like a dog; it's quite snappy from my very limited testing.

Please download the test release from
http://people.qumranet.com/avi/tpr-opt-1.tar.gz and give it a spin. Once it has received some exposure, I'll merge it into mainline.

Credit for the original idea is due to Ben Serebrin.

--
error compiling committee.c: too many arguments to function

2007/10/25, Avi Kivity <avi-atKUWr5tajBWk0Htik3J/[email protected]>:

Quoted 11 lines

This is a request for testing of an experimental kvm feature that dramatically accelerates some Windows releases (when running with the ACPI HAL, and especially with guest SMP). The feature detects accesses by the guest to the Task Priority Register (TPR) and patches them at runtime to a kvm-friendly code snipped that is provided by the BIOS.

The upshot of all that is that
http://kvm.qumranet.com/kvmwiki/Windows_ACPI_Workaround is no longer required, and Windows SMP no longer runs like a dog; it's quite snappy from my very limited testing.

Quoted 9 lines

Please download the test release from
http://people.qumranet.com/avi/tpr-opt-1.tar.gz and give it a spin. Once it has received some exposure, I'll merge it into mainline.

Credit for the original idea is due to Ben Serebrin.

--
error compiling committee.c: too many arguments to function

Hi Avi,

On an intel machine running 64 bits ubuntu with kernel 2.6.20, using the archive downloaded this morning at 10 o'clock GMT with the following md5sum:

Quoted 25 lines · Gildas

2007/10/25, Avi Kivity <avi-atKUWr5tajBWk0Htik3J/[email protected]>:

Quoted 11 lines

This is a request for testing of an experimental kvm feature that dramatically accelerates some Windows releases (when running with the ACPI HAL, and especially with guest SMP). The feature detects accesses by the guest to the Task Priority Register (TPR) and patches them at runtime to a kvm-friendly code snipped that is provided by the BIOS.

The upshot of all that is that
http://kvm.qumranet.com/kvmwiki/Windows_ACPI_Workaround is no longer required, and Windows SMP no longer runs like a dog; it's quite snappy from my very limited testing.

Quoted 9 lines

Please download the test release from
http://people.qumranet.com/avi/tpr-opt-1.tar.gz and give it a spin. Once it has received some exposure, I'll merge it into mainline.

Credit for the original idea is due to Ben Serebrin.

--
error compiling committee.c: too many arguments to function

Hi Avi,

Quoted 5 lines

On an intel machine running 64 bits ubuntu with kernel 2.6.20, using the archive downloaded this morning at 10 o'clock GMT with the following md5sum:

Right, installs seem not to like this release, but running an existing VM seems to work fine (mostly).

Quoted 4 lines

Quoted 1 line

Right, installs seem not to like this release, but running an existing VM seems to work fine (mostly).

A windows 2000 sp4 installation crashes as well (this time with WORKER_THREAD_RETURNED_A_BAD_IRQL). This was reproduceable. I've restarted the installation with -no-kvm-irq and it seems to work fine so long (it didn't crash where it crashed previously).

I've restarted windows xp install and this time I had the following:

[86852.991254] ------------[ cut here ]------------
[86852.991260] kernel BUG at /home/gl03/sources/tpr-opt-1/kernel/mmu.c:308!

Hi,
is it with kvm possible to remove a hardware (for example: a second PCI-E Graphiccard)- mybe on boottime like by xen.
And add dedicated this removed hardware (for example: a second PCI-E Graphiccard) to a guest system (for example: to improve the 3D-Graphic).

With best regards,
Michael

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

-----Original Message-----
From: kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org [mailto:kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org] On Behalf Of Avi Kivity
Sent: 2007年10月26日 1:24
To: kvm-devel
Subject: [kvm-devel] [RFT] kvm with Windows optimization

This is a request for testing of an experimental kvm feature that dramatically accelerates some Windows releases (when running with the ACPI HAL, and especially with guest SMP). The feature detects accesses by the guest to the Task Priority Register (TPR) and patches them at runtime to a kvm-friendly code snipped that is provided by the BIOS.

The upshot of all that is that
http://kvm.qumranet.com/kvmwiki/Windows_ACPI_Workaround is no longer required, and Windows SMP no longer runs like a dog; it's quite snappy from my very limited testing.

Please download the test release from
http://people.qumranet.com/avi/tpr-opt-1.tar.gz and give it a spin. Once it has received some exposure, I'll merge it into mainline.

Credit for the original idea is due to Ben Serebrin.

--
error compiling committee.c: too many arguments to function

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

I didn't some tests against the release.
Basically, it works well, and no any new regressions has been introduced. The booting speed of windows is also much faster.

But I still failed to boot 64bit smp XP and 2k3.
And after run a series of tests, I found a defunct qemu process.
############################################################
[root@vt-dp8 ~]# ps aux|grep qemu

root     12645  0.0  0.0  60284   720 pts/7    S+   22:30   0:00 grep qemu
root     20656  0.3  0.0      0     0 ?        Zl   20:37   0:20 [qemu-system-x86] <defunct>

############################################################
I checked the log, it happened when creating 64bit vista. If run the test case separately, the issue will not happen.

Quoted 8 lines · Yunfeng

I didn't some tests against the release.
Basically, it works well, and no any new regressions has been introduced. The booting speed of windows is also much faster.

But I still failed to boot 64bit smp XP and 2k3.
And after run a series of tests, I found a defunct qemu process.
############################################################
[root@vt-dp8 ~]# ps aux|grep qemu

Quoted 8 lines

root     12645  0.0  0.0  60284   720 pts/7    S+   22:30   0:00 grep qemu
root     20656  0.3  0.0      0     0 ?        Zl   20:37   0:20 [qemu-system-x86] <defunct>

############################################################
I checked the log, it happened when creating 64bit vista. If run the test case separately, the issue will not happen.

A bug was fixed in mainline for this issue, so it is not related to the optimization. I still need to make sure it doesn't harm 64-bit Windows.

I'll rebase and release a new version in a few days.

------------------------------------------------------------------------

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
------------------------------------------------------------------------

_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Quoted 20 lines · Avi Kivity

Quoted 17 lines · Yunfeng

I didn't some tests against the release.
Basically, it works well, and no any new regressions has been introduced. The booting speed of windows is also much faster.

But I still failed to boot 64bit smp XP and 2k3.
And after run a series of tests, I found a defunct qemu process.
############################################################
[root@vt-dp8 ~]# ps aux|grep qemu

root     12645  0.0  0.0  60284   720 pts/7    S+   22:30   0:00 grep qemu
root     20656  0.3  0.0      0     0 ?        Zl   20:37   0:20 [qemu-system-x86] <defunct>

############################################################
I checked the log, it happened when creating 64bit vista. If run the test case separately, the issue will not happen.

Quoted 4 lines

A bug was fixed in mainline for this issue, so it is not related to the optimization. I still need to make sure it doesn't harm 64-bit Windows.

Does 64-bit Windows ever access the TPR through MMIO? Wouldn't it just use cr8?