Trying to switch EPTP for execute-protecting guest pages
Hi all,
I'm playing around with EPTs and kvm to track execution in the guest. I've created a separate set of EPTs (and copied the last level entries from the real tables, minus execute permissions) but I'm not getting exits where I expect. I also have code in handle_ept_violation to preserve those permissions for any non-execute ept violations.
Here is what I am calling within a VM Exit handler:
---kvm_mmu_unload(vcpu); vcpu->arch.mmu.root_hpa = eptp; kvm_x86_ops->set_tdp_cr3(vcpu, eptp); kvm_mmu_load(vcpu); kvm_flush_remote_tlbs(vcpu->kvm);
---
I think some of this is overkill, but am I missing something? I think I may need to flush the rmaps too, but I'm not exactly sure how.
I am using Haswell if that matters.
Thank you very much for your help!
--Zak
Paolo Bonzini
maintainerQuoted 15 lines · Zachary J
I'm playing around with EPTs and kvm to track execution in the guest. I've created a separate set of EPTs (and copied the last level entries from the real tables, minus execute permissions) but I'm not getting exits where I expect. I also have code in
handle_ept_violationto preserve those permissions for any non-execute ept violations.Here is what I am calling within a VM Exit handler:
---kvm_mmu_unload(vcpu);vcpu->arch.mmu.root_hpa= eptp;kvm_x86_ops->set_tdp_cr3(vcpu, eptp);kvm_mmu_load(vcpu);kvm_flush_remote_tlbs(vcpu->kvm);
---
Quoted 2 lines
I think some of this is overkill, but am I missing something? I think I may need to flush the rmaps too, but I'm not exactly sure how.
My suggestion is:
1) use tracing and check that kvm_mmu_get_page is being called correctly.
2) there is already code for write protection. Try copying that code instead of doing a complete reimplementation.
Paolo
Quoted 21 lines · Paolo Bonzini
Quoted 15 lines · Zachary J
I'm playing around with EPTs and kvm to track execution in the guest. I've created a separate set of EPTs (and copied the last level entries from the real tables, minus execute permissions) but I'm not getting exits where I expect. I also have code in
handle_ept_violationto preserve those permissions for any non-execute ept violations.Here is what I am calling within a VM Exit handler:
---kvm_mmu_unload(vcpu);vcpu->arch.mmu.root_hpa= eptp;kvm_x86_ops->set_tdp_cr3(vcpu, eptp);kvm_mmu_load(vcpu);kvm_flush_remote_tlbs(vcpu->kvm);
---Quoted 2 lines
I think some of this is overkill, but am I missing something? I think I may need to flush the rmaps too, but I'm not exactly sure how.
Quoted 9 lines
My suggestion is:
1) use tracing and check that
kvm_mmu_get_pageis being called correctly.2) there is already code for write protection. Try copying that code instead of doing a complete reimplementation.
Paolo
1) Will do, thanks!
2) Got it. Let's say I want to work with a copy of the extended page tables instead of the original, what would be the best way to do so? Right now I'm traversing the full tables using root_hpa, but if there's a better way using the spte interface, I would prefer that.
Thanks so much!
--Zak
Paolo Bonzini
maintainerQuoted 2 lines · Zachary J
2) Got it. Let's say I want to work with a copy of the extended page tables instead of the original, what would be the best way to do so?
Why would you want that? It's difficult to give an answer without understanding what you're doing. Notice that KVM pretty much always leaves the X bit set (__direct_map uses ACC_ALL for the pte_access parameter) so it's easy to go from your copy of the extended page tables to the original.
I'm not sure if this is your problem, but perhaps you want to record in the role whether the page comes from your version or the original? The role is like the hash key, if the role is the same you get the same PTE.
Paolo
Quoted 2 lines
Right now I'm traversing the full tables using
root_hpa, but if there's a better way using the spte interface, I would prefer that.
Quoted 12 lines · Paolo Bonzini
Quoted 2 lines · Zachary J
2) Got it. Let's say I want to work with a copy of the extended page tables instead of the original, what would be the best way to do so?
Why would you want that? It's difficult to give an answer without understanding what you're doing. Notice that KVM pretty much always leaves the X bit set (
__direct_mapusesACC_ALLfor thepte_accessparameter) so it's easy to go from your copy of the extended page tables to the original.
Reply sent offlist.
Quoted 4 lines
I'm not sure if this is your problem, but perhaps you want to record in the role whether the page comes from your version or the original? The role is like the hash key, if the role is the same you get the same PTE.
This is extremely helpful, I had not noticed this. I'm using my new root_hpa as the base_role.word - does that make sense? I just tried it and I seem to get EPT_VIOLATIONS that I was expecting, but missing.
Thanks a ton, it appears that the role was exactly the thing I was looking for! --Zak
Paolo Bonzini
maintainerQuoted 7 lines · Zachary J
Quoted 3 lines
I'm not sure if this is your problem, but perhaps you want to record in the role whether the page comes from your version or the original? The role is like the hash key, if the role is the same you get the same PTE.
This is extremely helpful, I had not noticed this. I'm using my new
root_hpaas the base_role.word - does that make sense? I just tried it and I seem to getEPT_VIOLATIONSthat I was expecting, but missing.
I think you should add a new bit to the role meaning "should I clear some X bits?" :) that is computed based on the VCPU state. For an example see commit 699023e2 ("KVM: x86: add SMM to the MMU role, support SMRAM address space"), which does
+ context->base_role.smm = is_smm(vcpu);
in init_kvm_tdp_mmu. BTW, based on what you told me offlist, what you are doing should also just work with shadow page tables.
Paolo
Quoted 2 lines
Thanks a ton, it appears that the role was exactly the thing I was looking for!