Skip to contentSkip to conversation

viridian: fix assertion failure

PATCH Superseded Paul Durrant 2 participants 5 messages Nov 26, 2018

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 viridian: fix assertion failure

Whilst attempting to crash an apparently wedged Windows domain using 'xen-hvmcrash' I managed to trigger the following ASSERT:

(XEN) Assertion '!vp->ptr' failed at viridian.c:607

with stack:

(XEN)    [<ffff82d08032c55d>] viridian_map_guest_page+0x1b4/0x1b6
(XEN)    [<ffff82d08032b1db>] viridian_synic_load_vcpu_ctxt+0x39/0x3b
(XEN)    [<ffff82d08032b90d>] viridian.c#viridian_load_vcpu_ctxt+0x93/0xcc
(XEN)    [<ffff82d0803096d6>] hvm_load+0x10e/0x19e
(XEN)    [<ffff82d080274c6d>] arch_do_domctl+0xb74/0x25b4
(XEN)    [<ffff82d0802068ab>] do_domctl+0x16f7/0x19d8

This happened because viridian_map_guest_page() was not written to cope with being called multiple times, but this is unfortunately exactly what happens when xen-hvmcrash re-loads the domain context (having clobbered the values of RIP).

This patch simply makes viridian_map_guest_page() return immediately if it finds the page already mapped (i.e. vp->ptr != NULL).

---
Cc: Jan Beulich <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: Wei Liu <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>

---
 xen/arch/x86/hvm/viridian/viridian.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
xen/arch/x86/hvm/viridian/viridian.c +2 -1
index 7d73f41de6..b99501eea2 100644
@@ -604,7 +604,8 @@ void viridian_map_guest_page(struct vcpu *v, struct viridian_page *vp)
     unsigned long gmfn = vp->msr.fields.pfn;
     struct page_info *page = get_page_from_gfn(d, gmfn, NULL, P2M_ALLOC);
 
-    ASSERT(!vp->ptr);
+    if ( vp->ptr )
+        return;
 
     if ( !page )
         goto fail;
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw
Quoted 37 lines · Paul Durrant

Whilst attempting to crash an apparently wedged Windows domain using 'xen-hvmcrash' I managed to trigger the following ASSERT:

(XEN) Assertion '!vp->ptr' failed at viridian.c:607

with stack:

(XEN)    [<ffff82d08032c55d>] viridian_map_guest_page+0x1b4/0x1b6
(XEN)    [<ffff82d08032b1db>] viridian_synic_load_vcpu_ctxt+0x39/0x3b
(XEN)    [<ffff82d08032b90d>] viridian.c#viridian_load_vcpu_ctxt+0x93/0xcc
(XEN)    [<ffff82d0803096d6>] hvm_load+0x10e/0x19e
(XEN)    [<ffff82d080274c6d>] arch_do_domctl+0xb74/0x25b4
(XEN)    [<ffff82d0802068ab>] do_domctl+0x16f7/0x19d8

This happened because viridian_map_guest_page() was not written to cope with being called multiple times, but this is unfortunately exactly what happens when xen-hvmcrash re-loads the domain context (having clobbered the values of RIP).

This patch simply makes viridian_map_guest_page() return immediately if it finds the page already mapped (i.e. vp->ptr != NULL).

---
Cc: Jan Beulich <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: Wei Liu <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>

---
 xen/arch/x86/hvm/viridian/viridian.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
xen/arch/x86/hvm/viridian/viridian.c +0 -0
index 7d73f41de6..b99501eea2 100644

xen/arch/x86/hvm/viridian/viridian.c void viridian_map_guest_page(struct vcpu *v, struct viridian_page *vp) · line 608

Quoted 6 lines
xen/arch/x86/hvm/viridian/viridian.c +2 -1
@@ -604,7 +604,8 @@ void viridian_map_guest_page(struct vcpu *v, struct viridian_page *vp)
     unsigned long gmfn = vp->msr.fields.pfn;
     struct page_info *page = get_page_from_gfn(d, gmfn, NULL, P2M_ALLOC);
 
-    ASSERT(!vp->ptr);
+    if ( vp->ptr )
+        return;

Isn't this likely to get the page reference counting out of sync? You will return without a put_page, and viridian_unmap_guest_page will set vp->ptr = NULL and call put_page_and_type without taking into account the reference count.

If viridian_map_guest_page needs to be called multiple times you likely need some reference counting AFAICT, so that the last call to viridian_unmap_guest_page is the one that releases the page and sets vp->ptr = NULL.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#2 lore raw
Quoted 54 lines

-----Original Message-----
From: Roger Pau Monne
Sent: 26 November 2018 15:04
To: Paul Durrant <[email protected]>
Cc: [email protected]; Jan Beulich <[email protected]>; Andrew Cooper <[email protected]>; Wei Liu <[email protected]>
Subject: Re: [PATCH] viridian: fix assertion failure

Quoted 10 lines · Paul Durrant

Whilst attempting to crash an apparently wedged Windows domain using 'xen-hvmcrash' I managed to trigger the following ASSERT:

(XEN) Assertion '!vp->ptr' failed at viridian.c:607

with stack:

(XEN)    [<ffff82d08032c55d>] viridian_map_guest_page+0x1b4/0x1b6
(XEN)    [<ffff82d08032b1db>] viridian_synic_load_vcpu_ctxt+0x39/0x3b
(XEN)    [<ffff82d08032b90d>]

viridian.c#viridian_load_vcpu_ctxt+0x93/0xcc

Quoted 10 lines
(XEN)    [<ffff82d0803096d6>] hvm_load+0x10e/0x19e
(XEN)    [<ffff82d080274c6d>] arch_do_domctl+0xb74/0x25b4
(XEN)    [<ffff82d0802068ab>] do_domctl+0x16f7/0x19d8

This happened because viridian_map_guest_page() was not written to cope with being called multiple times, but this is unfortunately exactly what happens when xen-hvmcrash re-loads the domain context (having clobbered the values of RIP).

This patch simply makes viridian_map_guest_page() return immediately if

it

Quoted 8 lines

finds the page already mapped (i.e. vp->ptr != NULL).

---
Cc: Jan Beulich <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: Wei Liu <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>

Review

Quoted 5 lines
---
 xen/arch/x86/hvm/viridian/viridian.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
(quoted diff) +0 -0

b/xen/arch/x86/hvm/viridian/viridian.c

xen/arch/x86/hvm/viridian/viridian.c void viridian_map_guest_page(struct vcpu *v, struct

Quoted 4 lines

index 7d73f41de6..b99501eea2 100644

xen/arch/x86/hvm/viridian/viridian.c +0 -0
@@ -604,7 +604,8 @@ void viridian_map_guest_page(struct vcpu *v, struct

viridian_page *vp)

xen/arch/x86/hvm/viridian/viridian.c void viridian_map_guest_page(struct vcpu *v, struct · line 605

Quoted 2 lines
xen/arch/x86/hvm/viridian/viridian.c +0 -0
     unsigned long gmfn = vp->msr.fields.pfn;
     struct page_info *page = get_page_from_gfn(d, gmfn, NULL,

P2M_ALLOC);

Quoted 1 line
Quoted 9 lines

xen/arch/x86/hvm/viridian/viridian.c void viridian_map_guest_page(struct vcpu *v, struct viridian_page *vp) · line 608

Quoted 3 lines
xen/arch/x86/hvm/viridian/viridian.c +2 -1
-    ASSERT(!vp->ptr);
+    if ( vp->ptr )
+        return;

Isn't this likely to get the page reference counting out of sync? You will return without a put_page, and viridian_unmap_guest_page will set vp->ptr = NULL and call put_page_and_type without taking into account the reference count.

Damn. Missed that the side effect was in the initializer. I'll need to move it below the check.

Quoted 4 lines

If viridian_map_guest_page needs to be called multiple times you likely need some reference counting AFAICT, so that the last call to viridian_unmap_guest_page is the one that releases the page and sets vp->ptr = NULL.

I shouldn't need to reference count. When the MSR is set by the guest there is always an unmap then a map. It is just the context load case that should only map on the first call (to handle resume correctly).

Paul

Quoted 2 lines

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#3 lore raw
Quoted 64 lines · Paul Durrant
Quoted 54 lines

-----Original Message-----
From: Roger Pau Monne
Sent: 26 November 2018 15:04
To: Paul Durrant <[email protected]>
Cc: [email protected]; Jan Beulich <[email protected]>; Andrew Cooper <[email protected]>; Wei Liu <[email protected]>
Subject: Re: [PATCH] viridian: fix assertion failure

Quoted 10 lines · Paul Durrant

Whilst attempting to crash an apparently wedged Windows domain using 'xen-hvmcrash' I managed to trigger the following ASSERT:

(XEN) Assertion '!vp->ptr' failed at viridian.c:607

with stack:

(XEN)    [<ffff82d08032c55d>] viridian_map_guest_page+0x1b4/0x1b6
(XEN)    [<ffff82d08032b1db>] viridian_synic_load_vcpu_ctxt+0x39/0x3b
(XEN)    [<ffff82d08032b90d>]

viridian.c#viridian_load_vcpu_ctxt+0x93/0xcc

Quoted 10 lines
(XEN)    [<ffff82d0803096d6>] hvm_load+0x10e/0x19e
(XEN)    [<ffff82d080274c6d>] arch_do_domctl+0xb74/0x25b4
(XEN)    [<ffff82d0802068ab>] do_domctl+0x16f7/0x19d8

This happened because viridian_map_guest_page() was not written to cope with being called multiple times, but this is unfortunately exactly what happens when xen-hvmcrash re-loads the domain context (having clobbered the values of RIP).

This patch simply makes viridian_map_guest_page() return immediately if

it

Quoted 8 lines

finds the page already mapped (i.e. vp->ptr != NULL).

---
Cc: Jan Beulich <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: Wei Liu <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>

Review

Quoted 5 lines
---
 xen/arch/x86/hvm/viridian/viridian.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
(quoted diff) +0 -0

b/xen/arch/x86/hvm/viridian/viridian.c

xen/arch/x86/hvm/viridian/viridian.c void viridian_map_guest_page(struct vcpu *v, struct

Quoted 4 lines

index 7d73f41de6..b99501eea2 100644

xen/arch/x86/hvm/viridian/viridian.c +0 -0
@@ -604,7 +604,8 @@ void viridian_map_guest_page(struct vcpu *v, struct

viridian_page *vp)

xen/arch/x86/hvm/viridian/viridian.c void viridian_map_guest_page(struct vcpu *v, struct · line 605

Quoted 2 lines
xen/arch/x86/hvm/viridian/viridian.c +0 -0
     unsigned long gmfn = vp->msr.fields.pfn;
     struct page_info *page = get_page_from_gfn(d, gmfn, NULL,

P2M_ALLOC);

Quoted 1 line
Quoted 9 lines

xen/arch/x86/hvm/viridian/viridian.c void viridian_map_guest_page(struct vcpu *v, struct viridian_page *vp) · line 608

Quoted 3 lines
xen/arch/x86/hvm/viridian/viridian.c +2 -1
-    ASSERT(!vp->ptr);
+    if ( vp->ptr )
+        return;

Isn't this likely to get the page reference counting out of sync? You will return without a put_page, and viridian_unmap_guest_page will set vp->ptr = NULL and call put_page_and_type without taking into account the reference count.

Quoted 8 lines

Damn. Missed that the side effect was in the initializer. I'll need to move it below the check.

Quoted 4 lines

If viridian_map_guest_page needs to be called multiple times you likely need some reference counting AFAICT, so that the last call to viridian_unmap_guest_page is the one that releases the page and sets vp->ptr = NULL.

I shouldn't need to reference count. When the MSR is set by the guest there is always an unmap then a map. It is just the context load case that should only map on the first call (to handle resume correctly).

OK, I haven't looked at the usage, but if there's a paired number of maps/unmaps adding a reference counter to viridian_page is fairly trivial and would make the implementation of map/unmap always correct regardless of it's usage.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#4 lore raw
Quoted 88 lines

-----Original Message-----
From: Roger Pau Monne
Sent: 26 November 2018 15:32
To: Paul Durrant <[email protected]>
Cc: [email protected]; Jan Beulich <[email protected]>; Andrew Cooper <[email protected]>; Wei Liu <[email protected]>
Subject: Re: [PATCH] viridian: fix assertion failure

Quoted 6 lines · Paul Durrant
Quoted 6 lines

-----Original Message-----
From: Roger Pau Monne
Sent: 26 November 2018 15:04
To: Paul Durrant <[email protected]>
Cc: [email protected]; Jan Beulich <[email protected]>; Andrew Cooper <[email protected]>; Wei Liu

<[email protected]>

Quoted 6 lines
Quoted 6 lines

Subject: Re: [PATCH] viridian: fix assertion failure

Quoted 3 lines · Paul Durrant

Whilst attempting to crash an apparently wedged Windows domain using 'xen-hvmcrash' I managed to trigger the following ASSERT:

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

(XEN) Assertion '!vp->ptr' failed at viridian.c:607

with stack:

(XEN) [<ffff82d08032c55d>] viridian_map_guest_page+0x1b4/0x1b6
(XEN) [<ffff82d08032b1db>]

viridian_synic_load_vcpu_ctxt+0x39/0x3b

Quoted 7 lines
Quoted 7 lines
Quoted 1 line

(XEN) [<ffff82d08032b90d>]

viridian.c#viridian_load_vcpu_ctxt+0x93/0xcc

Quoted 5 lines
(XEN)    [<ffff82d0803096d6>] hvm_load+0x10e/0x19e
(XEN)    [<ffff82d080274c6d>] arch_do_domctl+0xb74/0x25b4
(XEN)    [<ffff82d0802068ab>] do_domctl+0x16f7/0x19d8

This happened because viridian_map_guest_page() was not written to

cope

Quoted 1 line
Quoted 1 line
Quoted 1 line

with being called multiple times, but this is unfortunately exactly

what

Quoted 1 line
Quoted 1 line
Quoted 1 line

happens when xen-hvmcrash re-loads the domain context (having

clobbered

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

the values of RIP).

This patch simply makes viridian_map_guest_page() return immediately

if

Quoted 13 lines
Quoted 13 lines

it

Quoted 12 lines

finds the page already mapped (i.e. vp->ptr != NULL).

---
Cc: Jan Beulich <[email protected]>
Cc: Andrew Cooper <[email protected]>
Cc: Wei Liu <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>

---
 xen/arch/x86/hvm/viridian/viridian.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
Quoted 6 lines
Quoted 6 lines

Review

Quoted 1 line
(quoted diff) +0 -0

b/xen/arch/x86/hvm/viridian/viridian.c

xen/arch/x86/hvm/viridian/viridian.c void viridian_map_guest_page(struct vcpu *v,

Quoted 4 lines

index 7d73f41de6..b99501eea2 100644

xen/arch/x86/hvm/viridian/viridian.c +0 -0
@@ -604,7 +604,8 @@ void viridian_map_guest_page(struct vcpu *v,

struct

Quoted 9 lines
Quoted 9 lines

viridian_page *vp)

xen/arch/x86/hvm/viridian/viridian.c void viridian_map_guest_page(struct vcpu *v, · line 605

Quoted 2 lines
xen/arch/x86/hvm/viridian/viridian.c +0 -0
     unsigned long gmfn = vp->msr.fields.pfn;
     struct page_info *page = get_page_from_gfn(d, gmfn, NULL,

P2M_ALLOC);

Quoted 4 lines
xen/arch/x86/hvm/viridian/viridian.c +2 -1
-    ASSERT(!vp->ptr);
+    if ( vp->ptr )
+        return;
Quoted 7 lines
Quoted 5 lines

Isn't this likely to get the page reference counting out of sync? You will return without a put_page, and viridian_unmap_guest_page will set vp->ptr = NULL and call put_page_and_type without taking into account the reference count.

Damn. Missed that the side effect was in the initializer. I'll need to

move it below the check.

Quoted 6 lines
Quoted 5 lines

If viridian_map_guest_page needs to be called multiple times you likely need some reference counting AFAICT, so that the last call to viridian_unmap_guest_page is the one that releases the page and sets vp->ptr = NULL.

Quoted 9 lines
Quoted 1 line

I shouldn't need to reference count. When the MSR is set by the guest

there is always an unmap then a map. It is just the context load case that should only map on the first call (to handle resume correctly).

OK, I haven't looked at the usage, but if there's a paired number of maps/unmaps adding a reference counter to viridian_page is fairly trivial and would make the implementation of map/unmap always correct regardless of it's usage.

No, maps and unmaps are not paired. You can have an arbitrary number of maps, but only one unmap. Anyway, v2 coming in a minute.

Paul

Quoted 1 line

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel