x86/emulate: Send vm_event from emulate
PATCH v9 1/1 x86/emulate: Send vm_event from emulate
A/D bit writes (on page walks) can be considered benign by an introspection agent, so receiving vm_events for them is a pessimization. We try here to optimize by filtering these events out.
Currently, we are fully emulating the instruction at RIP when the hardware sees an EPT fault with npfec.kind != npfec_kind_with_gla. This is, however, incorrect, because the instruction at RIP might legitimately cause an EPT fault of its own while accessing a _different_ page from the original one, where A/D were set.
The solution is to perform the whole emulation, while ignoring EPT restrictions for the walk part, and taking them into account for the "actual" emulating of the instruction at RIP. When we send out a vm_event, we don't want the emulation to complete, since in that case we won't be able to veto whatever it is doing. That would mean that we can't actually prevent any malicious activity, instead we'd only be able to report on it.
When we see a "send-vm_event" case while emulating, we need to first send the event out and then suspend the emulation (return X86EMUL_RETRY). After the emulation stops we'll call hvm_vm_event_do_resume() again after the introspection agent treats the event and resumes the guest. There, the instruction at RIP will be fully emulated (with the EPT ignored) if the introspection application allows it, and the guest will continue to run past the instruction.
A common example is if the hardware exits because of an EPT fault caused by a page walk, p2m_mem_access_check() decides if it is going to send a vm_event. If the vm_event was sent and it would be treated so it runs the instruction at RIP, that instruction might also hit a protected page and provoke a vm_event.
Now if npfec.kind == npfec_kind_in_gpt and d->arch.monitor.inguest_pagefault_disabled is true then we are in the page walk case and we can do this emulation optimization and emulate the page walk while ignoring the EPT, but don't ignore the EPT for the emulation of the actual instruction.
In the first case we would have 2 EPT events, in the second case we would have 1 EPT event if the instruction at the RIP triggers an EPT event.
We use hvmemul_map_linear_addr() to intercept r/w access and __hvm_copy() to intercept exec access.
hvm_emulate_send_vm_event() can return false if there was no violation, if there was an error from monitor_traps() or p2m_get_mem_access(). Returning false if p2m_get_mem_access() fails is needed because the EPT entry will have rwx memory access rights.
NOTE: hvm_emulate_send_vm_event() assumes the caller will check arch.vm_event->send_event
---
Changes since V8:
- Replace npfec.kind != npfec_kind_with_gla with
npfec.kind == npfec_kind_in_gpt
- Update commit message
- Add unlikely(curr->arch.vm_event)
- Move goto out in the loop in hvmemul_map_linear_addr()
- Rename hvm_emulate_send_vm_event() to hvm_monitor_check_ept()
and move it to hvm/monitor.c
--- xen/arch/x86/hvm/emulate.c | 17 ++++--- xen/arch/x86/hvm/hvm.c | 8 ++++ xen/arch/x86/hvm/monitor.c | 75 +++++++++++++++++++++++++++++++ xen/arch/x86/mm/mem_access.c | 3 +- xen/include/asm-x86/hvm/monitor.h | 3 ++ xen/include/asm-x86/vm_event.h | 2 + 6 files changed, 101 insertions(+), 7 deletions(-)
xen/arch/x86/hvm/emulate.c +11 -6
@@ -544,10 +544,11 @@ static void *hvmemul_map_linear_addr( struct hvm_emulate_ctxt *hvmemul_ctxt) {struct vcpu *curr = current; -void *err, *mapping; +void *err =NULL , *mapping;unsigned int nr_frames = ((linear + bytes - !!bytes) >>PAGE_SHIFT ) - (linear >>PAGE_SHIFT ) +1 ;unsigned int i; +gfn_t gfn;/* * mfn points to the next free slot. All used slots have a page reference
@@ -582,7 +583,7 @@ static void *hvmemul_map_linear_addr(ASSERT (mfn_x (*mfn) ==0 ); res =hvm_translate_get_page (curr, addr, true, pfec, - &pfinfo, &page,NULL , &p2mt); + &pfinfo, &page, &gfn, &p2mt);switch ( res ) {
@@ -596,7 +597,6 @@ static void *hvmemul_map_linear_addr( goto out;case HVMTRANS_bad_gfn_to_mfn: - err =NULL ; goto out;case HVMTRANS_gfn_paged_out:
@@ -619,13 +619,18 @@ static void *hvmemul_map_linear_addr( }if ( p2mt == p2m_ioreq_server ) - { - err =NULL ; goto out; - }ASSERT (p2mt == p2m_ram_logdirty || !p2m_is_readonly (p2mt)); } + +if (unlikely (curr->arch.vm_event) && + curr->arch.vm_event->send_event && +hvm_monitor_check_ept (addr, gfn, pfec, npfec_kind_with_gla) ) + { + err =ERR_PTR (~X86EMUL_RETRY ); + goto out; + } }/* Entire access within a single frame? */
xen/arch/x86/hvm/hvm.c +8 -0
@@ -3224,6 +3224,14 @@ static enum hvm_translation_result __hvm_copy(return HVMTRANS_bad_gfn_to_mfn; } +if (unlikely (v->arch.vm_event) && + v->arch.vm_event->send_event && +hvm_monitor_check_ept (addr, gfn, pfec, npfec_kind_with_gla) ) + { +put_page (page); +return HVMTRANS_gfn_paged_out; + } + p = (char *)__map_domain_page (page) + (addr & ~PAGE_MASK );if ( flags & HVMCOPY_to_guest )
xen/arch/x86/hvm/monitor.c +75 -0
89 lines not on this page · raw message
xen/arch/x86/mm/mem_access.c +2 -1
@@ -212,8 +212,9 @@ bool p2m_mem_access_check(paddr_t gpa, unsigned long gla, }if (vm_event_check_ring (d->vm_event_monitor) && d->arch.monitor.inguest_pagefault_disabled && - npfec.kind != npfec_kind_with_gla )/* don't send a mem_event */ + npfec.kind == npfec_kind_in_gpt )/* don't send a mem_event */ { + v->arch.vm_event->send_event = true;hvm_emulate_one_vm_event (EMUL_KIND_NORMAL , TRAP_invalid_op,X86_EVENT_NO_EC );return true;
xen/include/asm-x86/hvm/monitor.h +3 -0
@@ -49,6 +49,9 @@ void hvm_monitor_interrupt(unsigned int vector, unsigned int type, unsigned int err, uint64_t cr2);bool hvm_monitor_emul_unimplemented (void ); +bool hvm_monitor_check_ept (unsigned long gla,gfn_t gfn,uint32_t pfec, +uint16_t kind); +#endif /* __ASM_X86_HVM_MONITOR_H__ */ /*
xen/include/asm-x86/vm_event.h +2 -0
@@ -36,6 +36,8 @@ struct arch_vm_event {bool set_gprs;/* A sync vm_event has been sent and we're not done handling it. */ bool sync_event; +/* Send mem access events from emulator */ +bool send_event; };int vm_event_init_domain (struct domain *d);
Signature · 7 lines
2.17.1 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 32 lines · Alexandru Stefan ISAILA
A/D bit writes (on page walks) can be considered benign by an introspection agent, so receiving
vm_eventsfor them is a pessimization. We try here to optimize by filtering these events out.
Currently, we are fully emulating the instruction at RIP when the hardware sees an EPT fault with npfec.kind !=npfec_kind_with_gla. This is, however, incorrect, because the instruction at RIP might legitimately cause an EPT fault of its own while accessing a _different_ page from the original one, where A/D were set.
The solution is to perform the whole emulation, while ignoring EPT restrictions for the walk part, and taking them into account for the "actual" emulating of the instruction at RIP. When we send out avm_event, we don't want the emulation to complete, since in that case we won't be able to veto whatever it is doing. That would mean that we can't actually prevent any malicious activity, instead we'd only be able to report on it.
When we see a "send-vm_event" case while emulating, we need to first send the event out and then suspend the emulation (returnX86EMUL_RETRY). After the emulation stops we'll callhvm_vm_event_do_resume()again after the introspection agent treats the event and resumes the guest. There, the instruction at RIP will be fully emulated (with the EPT ignored) if the introspection application allows it, and the guest will continue to run past the instruction.A common example is if the hardware exits because of an EPT fault caused by a page walk,
p2m_mem_access_check()decides if it is going to send avm_event. If thevm_eventwas sent and it would be treated so it runs the instruction at RIP, that instruction might also hit a protected page and provoke avm_event.Now if npfec.kind ==
npfec_kind_in_gptandd->arch.monitor.inguest_pagefault_disabledis true then we are in the page walk case and we can do this emulation optimization and emulate the page walk while ignoring the EPT, but don't ignore the EPT for the emulation of the actual instruction.
Quoted 5 lines
In the first case we would have 2 EPT events, in the second case we would have 1 EPT event if the instruction at the RIP triggers an EPT event.
We use
hvmemul_map_linear_addr()to intercept r/w access and__hvm_copy()to intercept exec access.
Just like said for v8 - this doesn't look to match the implementation.
Quoted 4 lines
hvm_emulate_send_vm_event()can return false if there was no violation, if there was an error frommonitor_traps()orp2m_get_mem_access(). Returning false ifp2m_get_mem_access()fails is needed because the EPT entry will have rwx memory access rights.
I have to admit I still don't understand this reasoning, but I guess I should leave it to the VM event maintainers to judge. In particular it's unclear to me why p2m_get_mem_access() failure would imply rwx access.
xen/arch/x86/hvm/emulate.c static void *hvmemul_map_linear_addr( · line 547
Quoted 8 lines
xen/arch/x86/hvm/emulate.c +1
-1@@ -544,10 +544,11 @@ static void *hvmemul_map_linear_addr( struct hvm_emulate_ctxt *hvmemul_ctxt) {struct vcpu *curr = current; -void *err, *mapping; +void *err =NULL , *mapping;
As also said during v8 review, I don't think this (and the related) changes is needed anymore now that you've moved your new goto into the loop.
Quoted 5 lines
xen/arch/x86/hvm/emulate.c +1
-0@@ -215,6 +217,79 @@ void hvm_monitor_interrupt(unsigned int vector, unsigned int type,monitor_traps (current,1 , &req); } +/*
xen/arch/x86/hvm/emulate.c void hvm_monitor_interrupt(unsigned int vector, unsigned int type, · line 230
Quoted 10 lines
xen/arch/x86/hvm/emulate.c +10
-0+* Send memory access vm_events based on pfec. Returns true if the event was +* sent and false for p2m_get_mem_access() error, no violation and event send +* error. Assumes the caller will check arch.vm_event->send_event. +* +* NOTE: p2m_get_mem_access() can fail if the entry was not found in the EPT +* (in which case access to it is unrestricted, so no violations can occur). +* In this cases it is fine to continue the emulation. +*/ +bool hvm_monitor_check_ept (unsigned long gla,gfn_t gfn,uint32_t pfec, +uint16_t kind)
Why did you choose to have "ept" in the name and also mention it in the commit? Is there anything in here which isn't generic p2m?
xen/arch/x86/mm/mem_access.c bool p2m_mem_access_check(paddr_t gpa, unsigned long gla, · line 217
Quoted 10 lines
xen/arch/x86/mm/mem_access.c +2
-1@@ -212,8 +212,9 @@ bool p2m_mem_access_check(paddr_t gpa, unsigned long gla, }if (vm_event_check_ring (d->vm_event_monitor) && d->arch.monitor.inguest_pagefault_disabled && - npfec.kind != npfec_kind_with_gla )/* don't send a mem_event */ + npfec.kind == npfec_kind_in_gpt )/* don't send a mem_event */ { + v->arch.vm_event->send_event = true;
Since I'm being puzzled every time I see this: The comment and the line you add look to be in curious disagreement. Do you perhaps want to extend it to include something like "right away", or make it e.g. "try to avoid sending a mem event"? Personally I think it wouldn't hurt to even mention the "why" here.
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 51 lines · Jan Beulich
Quoted 32 lines · Alexandru Stefan ISAILA
A/D bit writes (on page walks) can be considered benign by an introspection agent, so receiving
vm_eventsfor them is a pessimization. We try here to optimize by filtering these events out.
Currently, we are fully emulating the instruction at RIP when the hardware sees an EPT fault with npfec.kind !=npfec_kind_with_gla. This is, however, incorrect, because the instruction at RIP might legitimately cause an EPT fault of its own while accessing a _different_ page from the original one, where A/D were set.
The solution is to perform the whole emulation, while ignoring EPT restrictions for the walk part, and taking them into account for the "actual" emulating of the instruction at RIP. When we send out avm_event, we don't want the emulation to complete, since in that case we won't be able to veto whatever it is doing. That would mean that we can't actually prevent any malicious activity, instead we'd only be able to report on it.
When we see a "send-vm_event" case while emulating, we need to first send the event out and then suspend the emulation (returnX86EMUL_RETRY). After the emulation stops we'll callhvm_vm_event_do_resume()again after the introspection agent treats the event and resumes the guest. There, the instruction at RIP will be fully emulated (with the EPT ignored) if the introspection application allows it, and the guest will continue to run past the instruction.A common example is if the hardware exits because of an EPT fault caused by a page walk,
p2m_mem_access_check()decides if it is going to send avm_event. If thevm_eventwas sent and it would be treated so it runs the instruction at RIP, that instruction might also hit a protected page and provoke avm_event.Now if npfec.kind ==
npfec_kind_in_gptandd->arch.monitor.inguest_pagefault_disabledis true then we are in the page walk case and we can do this emulation optimization and emulate the page walk while ignoring the EPT, but don't ignore the EPT for the emulation of the actual instruction.Quoted 5 lines
In the first case we would have 2 EPT events, in the second case we would have 1 EPT event if the instruction at the RIP triggers an EPT event.
We use
hvmemul_map_linear_addr()to intercept r/w access and__hvm_copy()to intercept exec access.Just like said for v8 - this doesn't look to match the implementation.
Quoted 4 lines
hvm_emulate_send_vm_event()can return false if there was no violation, if there was an error frommonitor_traps()orp2m_get_mem_access(). Returning false ifp2m_get_mem_access()fails is needed because the EPT entry will have rwx memory access rights.I have to admit I still don't understand this reasoning, but I guess I should leave it to the VM event maintainers to judge. In particular it's unclear to me why
p2m_get_mem_access()failure would imply rwx access.
Quoted 12 lines
xen/arch/x86/hvm/emulate.c static void *hvmemul_map_linear_addr( · line 547
Quoted 8 lines
xen/arch/x86/hvm/emulate.c +1
-1@@ -544,10 +544,11 @@ static void *hvmemul_map_linear_addr( struct hvm_emulate_ctxt *hvmemul_ctxt) {struct vcpu *curr = current; -void *err, *mapping; +void *err =NULL , *mapping;As also said during v8 review, I don't think this (and the related) changes is needed anymore now that you've moved your new goto into the loop.
I thought it is simpler to init err with NULL but you are right there is no need for this in this patch. I will revert the changes.
Quoted 5 lines
Quoted 5 lines
xen/arch/x86/hvm/emulate.c +0
-0@@ -215,6 +217,79 @@ void hvm_monitor_interrupt(unsigned int vector, unsigned int type,monitor_traps (current,1 , &req); }
Quoted 14 lines
xen/arch/x86/hvm/emulate.c void hvm_monitor_interrupt(unsigned int vector, unsigned int type, · line 230
Quoted 11 lines
xen/arch/x86/hvm/emulate.c +11
-0+/* +* Send memory access vm_events based on pfec. Returns true if the event was +* sent and false for p2m_get_mem_access() error, no violation and event send +* error. Assumes the caller will check arch.vm_event->send_event. +* +* NOTE: p2m_get_mem_access() can fail if the entry was not found in the EPT +* (in which case access to it is unrestricted, so no violations can occur). +* In this cases it is fine to continue the emulation. +*/ +bool hvm_monitor_check_ept (unsigned long gla,gfn_t gfn,uint32_t pfec, +uint16_t kind)Why did you choose to have "ept" in the name and also mention it in the commit? Is there anything in here which isn't generic p2m?
The name was suggested by Razvan Cojocaru. I have no preference in the name. Maybe Tamas can suggest a good one.
Quoted 10 lines
Quoted 10 lines
xen/arch/x86/mm/mem_access.c +1
-1@@ -212,8 +212,9 @@ bool p2m_mem_access_check(paddr_t gpa, unsigned long gla, }if (vm_event_check_ring (d->vm_event_monitor) && d->arch.monitor.inguest_pagefault_disabled && - npfec.kind != npfec_kind_with_gla )/* don't send a mem_event */ + npfec.kind == npfec_kind_in_gpt )/* don't send a mem_event */ {
Quoted 8 lines
xen/arch/x86/mm/mem_access.c bool p2m_mem_access_check(paddr_t gpa, unsigned long gla, · line 217
Quoted 1 line
xen/arch/x86/mm/mem_access.c +1
-0+ v->arch.vm_event->send_event = true;Since I'm being puzzled every time I see this: The comment and the line you add look to be in curious disagreement. Do you perhaps want to extend it to include something like "right away", or make it e.g. "try to avoid sending a mem event"? Personally I think it wouldn't hurt to even mention the "why" here.
I agree, I will update that comment.
Thanks,
Alex
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 19 lines · Alexandru Stefan ISAILA
Quoted 15 lines · Jan Beulich
xen/arch/x86/hvm/monitor.c void hvm_monitor_interrupt(unsigned int vector, unsigned int type, · line 230
Quoted 11 lines · Alexandru Stefan ISAILA
xen/arch/x86/hvm/monitor.c +11
-0+/* +* Send memory access vm_events based on pfec. Returns true if the event was +* sent and false for p2m_get_mem_access() error, no violation and event send +* error. Assumes the caller will check arch.vm_event->send_event. +* +* NOTE: p2m_get_mem_access() can fail if the entry was not found in the EPT +* (in which case access to it is unrestricted, so no violations can occur). +* In this cases it is fine to continue the emulation. +*/ +bool hvm_monitor_check_ept (unsigned long gla,gfn_t gfn,uint32_t pfec, +uint16_t kind)Why did you choose to have "ept" in the name and also mention it in the commit? Is there anything in here which isn't generic p2m?
Quoted 2 lines
The name was suggested by Razvan Cojocaru. I have no preference in the name. Maybe Tamas can suggest a good one.
I've suggested "ept" in the name because "regular" emulation ignores it, and this function takes it into account, hence the "check_ept" (which I thought would be read together). It's fine to change it to something else.
Thanks,
Razvan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 20 lines · Razvan Cojocaru
Quoted 19 lines · Alexandru Stefan ISAILA
Quoted 15 lines · Jan Beulich
xen/arch/x86/hvm/monitor.c void hvm_monitor_interrupt(unsigned int vector, unsigned int type, · line 230
Quoted 11 lines · Alexandru Stefan ISAILA
xen/arch/x86/hvm/monitor.c +11
-0+/* +* Send memory access vm_events based on pfec. Returns true if the event was +* sent and false for p2m_get_mem_access() error, no violation and event send +* error. Assumes the caller will check arch.vm_event->send_event. +* +* NOTE: p2m_get_mem_access() can fail if the entry was not found in the EPT +* (in which case access to it is unrestricted, so no violations can occur). +* In this cases it is fine to continue the emulation. +*/ +bool hvm_monitor_check_ept (unsigned long gla,gfn_t gfn,uint32_t pfec, +uint16_t kind)Why did you choose to have "ept" in the name and also mention it in the commit? Is there anything in here which isn't generic p2m?
Quoted 6 lines
Quoted 2 lines
The name was suggested by Razvan Cojocaru. I have no preference in the name. Maybe Tamas can suggest a good one.
I've suggested "ept" in the name because "regular" emulation ignores it, and this function takes it into account, hence the "
check_ept" (which I thought would be read together). It's fine to change it to something else.
Then "check_p2m" perhaps?
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 21 lines · Jan Beulich
Quoted 20 lines · Razvan Cojocaru
Quoted 19 lines · Alexandru Stefan ISAILA
Quoted 15 lines · Jan Beulich
xen/arch/x86/hvm/monitor.c void hvm_monitor_interrupt(unsigned int vector, unsigned int type, · line 230
Quoted 11 lines · Alexandru Stefan ISAILA
xen/arch/x86/hvm/monitor.c +11
-0+/* +* Send memory access vm_events based on pfec. Returns true if the event was +* sent and false for p2m_get_mem_access() error, no violation and event send +* error. Assumes the caller will check arch.vm_event->send_event. +* +* NOTE: p2m_get_mem_access() can fail if the entry was not found in the EPT +* (in which case access to it is unrestricted, so no violations can occur). +* In this cases it is fine to continue the emulation. +*/ +bool hvm_monitor_check_ept (unsigned long gla,gfn_t gfn,uint32_t pfec, +uint16_t kind)Why did you choose to have "ept" in the name and also mention it in the commit? Is there anything in here which isn't generic p2m?
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
The name was suggested by Razvan Cojocaru. I have no preference in the name. Maybe Tamas can suggest a good one.
I've suggested "ept" in the name because "regular" emulation ignores it, and this function takes it into account, hence the "
check_ept" (which I thought would be read together). It's fine to change it to something else.Then "
check_p2m" perhaps?
Sounds good to me.
Thanks,
Razvan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel