Extend ioreq-server to support page write protection
cover Extend ioreq-server to support page write protection
ioreq-server is proposed to forward PIO and MMIO request to multiple device models according to the io range. XenGT (Intel Graphics Virtualization technology, please refer to https://01.org/xen/blogs/srclarkx/2013/graphics-virtualization-
xengt) driver reside in Dom0 as a virtual graphics device model also need to trap and emulate the guest's write operation to some specific memory pages, like the memory pages used by guest graphics driver as PPGTT(per-process graphics translation table). We add an new p2m type "p2m_ram_wp" to trap the page write operation. Page of this new p2m type is read only and for write, the request will go to device model via ioreq-server.
Wei Ye (2):
x86: add p2m_ram_wp
ioreq-server: Support scatter page forwarding
tools/libxc/xc_domain.c | 32 ++++++ tools/libxc/xenctrl.h | 18 ++++ xen/arch/x86/hvm/hvm.c | 214 +++++++++++++++++++++++++++++++++++++- xen/arch/x86/mm/p2m-ept.c | 1 + xen/include/asm-x86/hvm/domain.h | 14 +++ xen/include/asm-x86/p2m.h | 8 +- xen/include/public/hvm/hvm_op.h | 12 +++ 7 files changed, 296 insertions(+), 3 deletions(-)
Signature · 1 line
1.7.9.5
Quoted 16 lines
Quoted 1 line
Quoted 1 line · On 28.07.14 at 19:55, <[email protected]> wrote:
ioreq-server is proposed to forward PIO and MMIO request to multiple device models according to the io range. XenGT (Intel Graphics Virtualization technology,
please refer to
https://01.org/xen/blogs/srclarkx/2013/graphics-virtualization-
xengt) driver reside in Dom0 as a virtual graphics device model also need to trap
and emulate the guest's write operation to some specific memory pages, like the
memory pages used by guest graphics driver as PPGTT(per-process graphics translation table). We add an new p2m type "p2m_ram_wp" to trap the page write
operation. Page of this new p2m type is read only and for write, the request will
go to device model via ioreq-server.
So how is this write-protection supposed to work on the IOMMU side when sharing page tables?
Jan
Quoted 25 lines
-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Monday, July 28, 2014 4:25 PM
To: Ye, Wei
Cc: [email protected]; [email protected]; [email protected]; [email protected]; xen- [email protected]; [email protected]
Subject: Re: [PATCH v1 0/2] Extend ioreq-server to support page write protectionQuoted 12 lines
Quoted 1 line
Quoted 1 line · On 28.07.14 at 19:55, <[email protected]> wrote:
ioreq-server is proposed to forward PIO and MMIO request to multiple device models according to the io range. XenGT (Intel Graphics Virtualization technology, please refer to
https://01.org/xen/blogs/srclarkx/2013/graphics-virtualization-
xengt) driver reside in Dom0 as a virtual graphics device model also need to trap and emulate the guest's write operation to some specific memory pages, like the memory pages used by guest graphics driver as PPGTT(per-process graphics translation table). We add an new p2m type "p2m_ram_wp" to trap the page write operation. Page of this new p2m type is read only and for write, the request will go to device model via ioreq-server.
Quoted 2 lines
So how is this write-protection supposed to work on the IOMMU side when sharing page tables?
Thanks for pointing out this question. Write-protection is not supposed to work when sharing page tables between EPT and vt-d. An explicit command line "iommu=no-sharept" should be setted for avoiding undesirable iommu fault.
Quoted 2 lines
Jan
Quoted 26 lines
Quoted 26 lines
Quoted 1 line · On 04.08.14 at 07:05, <[email protected]> wrote:
-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Monday, July 28, 2014 4:25 PM
To: Ye, Wei
Cc: [email protected]; [email protected]; [email protected]; [email protected]; xen- [email protected]; [email protected]
Subject: Re: [PATCH v1 0/2] Extend ioreq-server to support page write protectionQuoted 12 lines
Quoted 1 line
Quoted 1 line · On 28.07.14 at 19:55, <[email protected]> wrote:
ioreq-server is proposed to forward PIO and MMIO request to multiple device models according to the io range. XenGT (Intel Graphics Virtualization technology, please refer to
https://01.org/xen/blogs/srclarkx/2013/graphics-virtualization-
xengt) driver reside in Dom0 as a virtual graphics device model also need to trap and emulate the guest's write operation to some specific memory pages, like the memory pages used by guest graphics driver as PPGTT(per-process graphics translation table). We add an new p2m type "p2m_ram_wp" to trap the page write operation. Page of this new p2m type is read only and for write, the request will go to device model via ioreq-server.
Quoted 7 lines
Quoted 2 lines
So how is this write-protection supposed to work on the IOMMU side when sharing page tables?
Thanks for pointing out this question. Write-protection is not supposed to work when sharing page tables between EPT and vt-d.
An explicit command line "iommu=no-sharept" should be setted for avoiding undesirable iommu fault.
Requiring the unconditional use of a specific command line option is certainly fine for experimental code, but not beyond that. Behavior should be correct by default in production code.
But what's worse here: The option _allows_ device side writes from the guest. Why would device side writes be okay, but CPU side ones not?
Jan
Quoted 37 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Monday, August 04, 2014 12:35 AMQuoted 26 lines
Quoted 26 lines
Quoted 1 line · On 04.08.14 at 07:05, <[email protected]> wrote:
-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Monday, July 28, 2014 4:25 PM
To: Ye, Wei
Cc: [email protected]; [email protected]; [email protected]; [email protected]; xen- [email protected]; [email protected]
Subject: Re: [PATCH v1 0/2] Extend ioreq-server to support page write protectionQuoted 12 lines
Quoted 1 line
Quoted 1 line · On 28.07.14 at 19:55, <[email protected]> wrote:
ioreq-server is proposed to forward PIO and MMIO request to multiple device models according to the io range. XenGT (Intel Graphics Virtualization technology, please refer to
https://01.org/xen/blogs/srclarkx/2013/graphics-virtualization-
xengt) driver reside in Dom0 as a virtual graphics device model also need to trap and emulate the guest's write operation to some specific memory pages, like the memory pages used by guest graphics driver as PPGTT(per-process graphics translation table). We add an new p2m type "p2m_ram_wp" to trap the page write operation. Page of this new p2m type is read only and for write, the request will go to device model via ioreq-server.Quoted 7 lines
Quoted 2 lines
So how is this write-protection supposed to work on the IOMMU side when sharing page tables?
Thanks for pointing out this question. Write-protection is not supposed to work when sharing page tables between EPT and vt-d.
An explicit command line "iommu=no-sharept" should be setted for avoiding undesirable iommu fault.
Quoted 8 lines
Requiring the unconditional use of a specific command line option is certainly fine for experimental code, but not beyond that. Behavior should be correct by default in production code.
But what's worse here: The option _allows_ device side writes from the guest. Why would device side writes be okay, but CPU side ones not?
right, whether ept is shared or not doesn't address the concern here. In both cases we need maintain the same p2m view between CPU and device side, otherwise it's broken...
One option is to treat wp similar to logdirty, i.e. exclusive to VT-d device assignment, until in the future VT-d supports page fault. We can provide a boot option to override the default policy if user thinks OK.
2nd option, like Wei mentioned in another mail, is to treat such write protected PPGTT page tables as MMIO. new hypercall is required to change the p2m type between p2m_io and p2m_ram, based on allocation/free of guest page table. This way may impact performance on read though.
Comments?
Thanks
Kevin
Quoted 6 lines
Quoted 1 line
Quoted 1 line · On 04.08.14 at 23:34, <[email protected]> wrote:
2nd option, like Wei mentioned in another mail, is to treat such write protected PPGTT page tables as MMIO. new hypercall is required to change the p2m type between
p2m_ioandp2m_ram, based on allocation/free of guest page table. This way may impact performance on read though.
Depends on whether you mean mmio_dm or mmio_direct here.
Jan
Quoted 17 lines
-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Tuesday, August 5, 2014 2:35 PM
To: Tian, Kevin
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Ye, Wei; Lv, Zhiyuan; [email protected]; [email protected]; [email protected]
Subject: RE: [PATCH v1 0/2] Extend ioreq-server to support page write protectionQuoted 6 lines
Quoted 1 line
Quoted 1 line · On 04.08.14 at 23:34, <[email protected]> wrote:
2nd option, like Wei mentioned in another mail, is to treat such write protected PPGTT page tables as MMIO. new hypercall is required to change the p2m type between
p2m_ioandp2m_ram, based on allocation/free of guest page table. This way may impact performance on read though.
Quoted 3 lines
Depends on whether you mean
mmio_dmormmio_directhere.Jan
It's p2m_mmio_dm.
Quoted 18 lines
Quoted 18 lines
Quoted 1 line · On 05.08.14 at 08:46, <[email protected]> wrote:
-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Tuesday, August 5, 2014 2:35 PM
To: Tian, Kevin
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Ye, Wei; Lv, Zhiyuan; [email protected]; [email protected]; [email protected]
Subject: RE: [PATCH v1 0/2] Extend ioreq-server to support page write protectionQuoted 6 lines
Quoted 1 line
Quoted 1 line · On 04.08.14 at 23:34, <[email protected]> wrote:
2nd option, like Wei mentioned in another mail, is to treat such write protected PPGTT page tables as MMIO. new hypercall is required to change the p2m type between
p2m_ioandp2m_ram, based on allocation/free of guest page table. This way may impact performance on read though.
Quoted 3 lines
Quoted 1 line
Depends on whether you mean
mmio_dmormmio_directhere.It's
p2m_mmio_dm.
In which case - how would this help? You'd still not be able to deal with device side accesses. (And as a side note - a hypercall to convert between p2m_ram_rw and p2m_mmio_dm already exists:HVMOP_set_mem_type.)
Jan
Tian, Kevin wrote on 2014-08-05:
Quoted 49 lines
Quoted 32 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Monday, August 04, 2014 12:35 AMQuoted 21 lines
Quoted 1 line
Quoted 1 line · On 04.08.14 at 07:05, <[email protected]> wrote:
Jan Beulich wrote on 2014-07-28:
Quoted 17 lines
[email protected]; [email protected]
Subject: Re: [PATCH v1 0/2] Extend ioreq-server to support page write protectionQuoted 12 lines
Quoted 1 line
Quoted 1 line · On 28.07.14 at 19:55, <[email protected]> wrote:
ioreq-server is proposed to forward PIO and MMIO request to multiple device models according to the io range. XenGT (Intel Graphics Virtualization technology, please refer to
https://01.org/xen/blogs/srclarkx/2013/graphics-virtualization-
xengt) driver reside in Dom0 as a virtual graphics device model also need to trap and emulate the guest's write operation to some specific memory pages, like the memory pages used by guest graphics driver as PPGTT(per-process graphics translation table). We add an new p2m type "p2m_ram_wp" to trap the page write operation. Page of this new p2m type is read only and for write, the request will go to device model via ioreq-server.Quoted 7 lines
Quoted 2 lines
So how is this write-protection supposed to work on the IOMMU side when sharing page tables?
Thanks for pointing out this question. Write-protection is not supposed to work when sharing page tables between EPT and vt-d. An explicit command line "iommu=no-sharept" should be setted for avoiding undesirable iommu fault.
Quoted 7 lines
Requiring the unconditional use of a specific command line option is certainly fine for experimental code, but not beyond that. Behavior should be correct by default in production code.
But what's worse here: The option _allows_ device side writes from the guest. Why would device side writes be okay, but CPU side ones not?
right, whether ept is shared or not doesn't address the concern here. In both cases we need maintain the same p2m view between CPU and device side, otherwise it's broken...
One option is to treat wp similar to logdirty, i.e. exclusive to VT-d device assignment, until in the future VT-d supports page fault. We can provide a boot option to override the default policy if user thinks OK.
Quoted 7 lines
2nd option, like Wei mentioned in another mail, is to treat such write protected PPGTT page tables as MMIO. new hypercall is required to change the p2m type between
p2m_ioandp2m_ram, based on allocation/free of guest page table. This way may impact performance on read though.Comments?
Another solution is using the EPT misconfiguration mechanism like what Xen does for MTRR emulation currently.
Quoted 8 lines
Thanks
Kevin_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel
Best regards,
Yang
Quoted 57 lines
Quoted 1 line
Quoted 1 line · On 05.08.14 at 09:35, <[email protected]> wrote:
Tian, Kevin wrote on 2014-08-05:
Quoted 55 lines
Quoted 32 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Monday, August 04, 2014 12:35 AMQuoted 21 lines
Quoted 1 line
Quoted 1 line · On 04.08.14 at 07:05, <[email protected]> wrote:
Jan Beulich wrote on 2014-07-28:
Quoted 17 lines
[email protected]; [email protected]
Subject: Re: [PATCH v1 0/2] Extend ioreq-server to support page write protectionQuoted 12 lines
Quoted 1 line
Quoted 1 line · On 28.07.14 at 19:55, <[email protected]> wrote:
ioreq-server is proposed to forward PIO and MMIO request to multiple device models according to the io range. XenGT (Intel Graphics Virtualization technology, please refer to
https://01.org/xen/blogs/srclarkx/2013/graphics-virtualization-
xengt) driver reside in Dom0 as a virtual graphics device model also need to trap and emulate the guest's write operation to some specific memory pages, like the memory pages used by guest graphics driver as PPGTT(per-process graphics translation table). We add an new p2m type "p2m_ram_wp" to trap the page write operation. Page of this new p2m type is read only and for write, the request will go to device model via ioreq-server.Quoted 7 lines
Quoted 2 lines
So how is this write-protection supposed to work on the IOMMU side when sharing page tables?
Thanks for pointing out this question. Write-protection is not supposed to work when sharing page tables between EPT and vt-d. An explicit command line "iommu=no-sharept" should be setted for avoiding undesirable iommu fault.
Quoted 7 lines
Requiring the unconditional use of a specific command line option is certainly fine for experimental code, but not beyond that. Behavior should be correct by default in production code.
But what's worse here: The option _allows_ device side writes from the guest. Why would device side writes be okay, but CPU side ones not?
right, whether ept is shared or not doesn't address the concern here. In both cases we need maintain the same p2m view between CPU and device side, otherwise it's broken...
One option is to treat wp similar to logdirty, i.e. exclusive to VT-d device assignment, until in the future VT-d supports page fault. We can provide a boot option to override the default policy if user thinks OK.
2nd option, like Wei mentioned in another mail, is to treat such write protected PPGTT page tables as MMIO. new hypercall is required to change the p2m type between
p2m_ioandp2m_ram, based on allocation/free of guest page table. This way may impact performance on read though.
Quoted 4 lines
Quoted 1 line
Comments?
Another solution is using the EPT misconfiguration mechanism like what Xen does for MTRR emulation currently.
That would cause faults on reads as well, making it necessary to emulate them.
Jan
Quoted 69 lines
-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Tuesday, August 5, 2014 3:52 PM
To: Tian, Kevin; Ye, Wei; Zhang, Yang Z
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Lv, Zhiyuan; [email protected]; [email protected]; [email protected]
Subject: RE: [Xen-devel] [PATCH v1 0/2] Extend ioreq-server to support page write protectionQuoted 57 lines
Quoted 1 line
Quoted 1 line · On 05.08.14 at 09:35, <[email protected]> wrote:
Tian, Kevin wrote on 2014-08-05:
Quoted 55 lines
Quoted 32 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Monday, August 04, 2014 12:35 AMQuoted 21 lines
Quoted 1 line
Quoted 1 line · On 04.08.14 at 07:05, <[email protected]> wrote:
Jan Beulich wrote on 2014-07-28:
Quoted 17 lines
[email protected]; [email protected]
Subject: Re: [PATCH v1 0/2] Extend ioreq-server to support page write protectionQuoted 12 lines
Quoted 1 line
Quoted 1 line · On 28.07.14 at 19:55, <[email protected]> wrote:
ioreq-server is proposed to forward PIO and MMIO request to multiple device models according to the io range. XenGT (Intel Graphics Virtualization technology, please refer to
https://01.org/xen/blogs/srclarkx/2013/graphics-virtualization-
xengt) driver reside in Dom0 as a virtual graphics device model also need to trap and emulate the guest's write operation to some specific memory pages, like the memory pages used by guest graphics driver as PPGTT(per-process graphics translation table). We add an new p2m type "p2m_ram_wp" to trap the page write operation. Page of this new p2m type is read only and for write, the request will go to device model via ioreq-server.Quoted 7 lines
Quoted 2 lines
So how is this write-protection supposed to work on the IOMMU side when sharing page tables?
Thanks for pointing out this question. Write-protection is not supposed to work when sharing page tables between EPT and vt-d. An explicit command line "iommu=no-sharept" should be setted for avoiding undesirable iommu fault.
Quoted 7 lines
Requiring the unconditional use of a specific command line option is certainly fine for experimental code, but not beyond that. Behavior should be correct by default in production code.
But what's worse here: The option _allows_ device side writes from the guest. Why would device side writes be okay, but CPU side ones not?
right, whether ept is shared or not doesn't address the concern here. In both cases we need maintain the same p2m view between CPU and device side, otherwise it's broken...
One option is to treat wp similar to logdirty, i.e. exclusive to VT-d device assignment, until in the future VT-d supports page fault. We can provide a boot option to override the default policy if user thinks OK.
2nd option, like Wei mentioned in another mail, is to treat such write protected PPGTT page tables as MMIO. new hypercall is required to change the p2m type between
p2m_ioandp2m_ram, based on allocation/free of guest page table. This way may impact performance on read though.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Comments?
Another solution is using the EPT misconfiguration mechanism like what Xen does for MTRR emulation currently.
That would cause faults on reads as well, making it necessary to emulate them.
In our usage case, the read access is much less than write (zero read in Linux guest and 1/4 for read access compared to write in windows guest).
Quoted 2 lines
Jan
Quoted 60 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Tuesday, August 05, 2014 12:52 AMQuoted 57 lines
Quoted 1 line
Quoted 1 line · On 05.08.14 at 09:35, <[email protected]> wrote:
Tian, Kevin wrote on 2014-08-05:
Quoted 55 lines
Quoted 32 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Monday, August 04, 2014 12:35 AMQuoted 21 lines
Quoted 1 line
Quoted 1 line · On 04.08.14 at 07:05, <[email protected]> wrote:
Jan Beulich wrote on 2014-07-28:
Quoted 17 lines
[email protected]; [email protected]
Subject: Re: [PATCH v1 0/2] Extend ioreq-server to support page write protectionQuoted 12 lines
Quoted 1 line
Quoted 1 line · On 28.07.14 at 19:55, <[email protected]> wrote:
ioreq-server is proposed to forward PIO and MMIO request to multiple device models according to the io range. XenGT (Intel Graphics Virtualization technology, please refer to
https://01.org/xen/blogs/srclarkx/2013/graphics-virtualization-
xengt) driver reside in Dom0 as a virtual graphics device model also need to trap and emulate the guest's write operation to some specific memory pages, like the memory pages used by guest graphics driver as PPGTT(per-process graphics translation table). We add an new p2m type "p2m_ram_wp" to trap the page write operation. Page of this new p2m type is read only and for write, the request will go to device model via ioreq-server.Quoted 7 lines
Quoted 2 lines
So how is this write-protection supposed to work on the IOMMU side when sharing page tables?
Thanks for pointing out this question. Write-protection is not supposed to work when sharing page tables between EPT and vt-d. An explicit command line "iommu=no-sharept" should be setted for avoiding undesirable iommu fault.
Quoted 7 lines
Requiring the unconditional use of a specific command line option is certainly fine for experimental code, but not beyond that. Behavior should be correct by default in production code.
But what's worse here: The option _allows_ device side writes from the guest. Why would device side writes be okay, but CPU side ones not?
right, whether ept is shared or not doesn't address the concern here. In both cases we need maintain the same p2m view between CPU and device side, otherwise it's broken...
One option is to treat wp similar to logdirty, i.e. exclusive to VT-d device assignment, until in the future VT-d supports page fault. We can provide a boot option to override the default policy if user thinks OK.
2nd option, like Wei mentioned in another mail, is to treat such write protected PPGTT page tables as MMIO. new hypercall is required to change the p2m type between
p2m_ioandp2m_ram, based on allocation/free of guest page table. This way may impact performance on read though.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Comments?
Another solution is using the EPT misconfiguration mechanism like what Xen does for MTRR emulation currently.
That would cause faults on reads as well, making it necessary to emulate them.
Then it's same effect of p2m_mmio_dm.
Thanks
Kevin
Tian, Kevin wrote on 2014-08-05:
Quoted 62 lines
Quoted 62 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Tuesday, August 05, 2014 12:52 AMQuoted 59 lines
Quoted 58 lines
Quoted 34 lines
On 05.08.14 at 09:35, <[email protected]> wrote: Tian, Kevin wrote on 2014-08-05: From: Jan Beulich [mailto:[email protected]]
Sent: Monday, August 04, 2014 12:35 AMQuoted 22 lines
Quoted 1 line
Quoted 1 line · On 04.08.14 at 07:05, <[email protected]> wrote:
Jan Beulich wrote on 2014-07-28:
Quoted 18 lines
[email protected]; [email protected]
Subject: Re: [PATCH v1 0/2] Extend ioreq-server to support page write protectionQuoted 13 lines
Quoted 1 line
Quoted 1 line · On 28.07.14 at 19:55, <[email protected]> wrote:
ioreq-server is proposed to forward PIO and MMIO request to
multiple device models according to the io range. XenGT
(Intel Graphics Virtualization technology, please refer to
https://01.org/xen/blogs/srclarkx/2013/graphics-virtualizatio
n-
xengt) driver reside in Dom0 as a virtual graphics device
model also need to trap and emulate the guest's write
operation to some specific memory pages, like the memory
pages used by guest graphics driver as PPGTT(per-process graphics translation table). We add an new p2m type "p2m_ram_wp" to trap the page write
operation. Page of this new p2m type is read only and for
write, the request will go to device model via ioreq-server.Quoted 7 lines
Quoted 2 lines
So how is this write-protection supposed to work on the IOMMU side when sharing page tables?
Thanks for pointing out this question. Write-protection is not supposed to work when sharing page tables between EPT and vt-d. An explicit command line "iommu=no-sharept" should be setted for avoiding undesirable iommu fault.
Quoted 7 lines
Requiring the unconditional use of a specific command line option is certainly fine for experimental code, but not beyond that. Behavior should be correct by default in production code.
But what's worse here: The option _allows_ device side writes from the guest. Why would device side writes be okay, but CPU side ones not?
right, whether ept is shared or not doesn't address the concern here. In both cases we need maintain the same p2m view between CPU and device side, otherwise it's broken...
One option is to treat wp similar to logdirty, i.e. exclusive to VT-d device assignment, until in the future VT-d supports page fault. We can provide a boot option to override the default policy if user thinks OK.
2nd option, like Wei mentioned in another mail, is to treat such write protected PPGTT page tables as MMIO. new hypercall is required to change the p2m type between
p2m_ioandp2m_ram, based on allocation/free of guest page table. This way may impact performance on read though.Comments?
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Another solution is using the EPT misconfiguration mechanism like what Xen does for MTRR emulation currently.
That would cause faults on reads as well, making it necessary to emulate them.
Then it's same effect of
p2m_mmio_dm.
p2m_mmio_dm will break VT-d but EPT misconfiguration doesn't.
Quoted 3 lines
Thanks
Kevin
Best regards,
Yang
Quoted 71 lines
From: Zhang, Yang Z
Sent: Tuesday, August 05, 2014 7:11 PMTian, Kevin wrote on 2014-08-05:
Quoted 16 lines
Quoted 16 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Tuesday, August 05, 2014 12:52 AMQuoted 13 lines
Quoted 13 lines
Quoted 13 lines
On 05.08.14 at 09:35, <[email protected]> wrote: Tian, Kevin wrote on 2014-08-05: From: Jan Beulich [mailto:[email protected]]
Sent: Monday, August 04, 2014 12:35 AMQuoted 9 lines
Quoted 1 line
Quoted 1 line · On 04.08.14 at 07:05, <[email protected]> wrote:
Jan Beulich wrote on 2014-07-28:
Quoted 5 lines
[email protected]; [email protected]
Subject: Re: [PATCH v1 0/2] Extend ioreq-server to support page write protectionQuoted 1 line
Quoted 1 line
Quoted 1 line · On 28.07.14 at 19:55, <[email protected]> wrote:
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
ioreq-server is proposed to forward PIO and MMIO request to multiple device models according to the io range. XenGT (Intel Graphics Virtualization technology, please refer to
https://01.org/xen/blogs/srclarkx/2013/graphics-virtualizatio
n-
xengt) driver reside in Dom0 as a virtual graphics device model also need to trap and emulate the guest's write operation to some specific memory pages, like the memory pages used by guest graphics driver as PPGTT(per-process graphicstranslation table).
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 6 lines
Quoted 3 lines
We add an new p2m type "
p2m_ram_wp" to trap the page write operation. Page of this new p2m type is read only and for write, the request will go to device model via ioreq-server.So how is this write-protection supposed to work on the IOMMU side when sharing page tables?
Thanks for pointing out this question. Write-protection is not supposed to work when sharing page tables between EPT and vt-d. An explicit command line "iommu=no-sharept" should be setted for avoiding undesirable iommu fault.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Requiring the unconditional use of a specific command line option is certainly fine for experimental code, but not beyond that. Behavior should be correct by default in production code.
But what's worse here: The option _allows_ device side writes from the guest. Why would device side writes be okay, but CPU side
ones not?
Quoted 22 lines
Quoted 22 lines
Quoted 21 lines
Quoted 11 lines
Quoted 1 line
right, whether ept is shared or not doesn't address the concern here. In both cases we need maintain the same p2m view between CPU and device side, otherwise it's broken...
One option is to treat wp similar to logdirty, i.e. exclusive to VT-d device assignment, until in the future VT-d supports page fault. We can provide a boot option to override the default policy if user thinks OK.
Quoted 7 lines
2nd option, like Wei mentioned in another mail, is to treat such write protected PPGTT page tables as MMIO. new hypercall is required to change the p2m type between
p2m_ioandp2m_ram, based on allocation/free of guest page table. This way may impact performance on read though.Comments?
Another solution is using the EPT misconfiguration mechanism like what Xen does for MTRR emulation currently.
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
That would cause faults on reads as well, making it necessary to emulate them.
Then it's same effect of
p2m_mmio_dm.
p2m_mmio_dmwill break VT-d but EPT misconfiguration doesn't.
I don't understand. Treating it as emulated io just means no valid p2m entry in EPT/VT-d. Note XenGT itself doesn't require VT-d, while you can still assign other devices this way.
Thanks
Kevin
Tian, Kevin wrote on 2014-08-06:
Quoted 83 lines
Quoted 70 lines
From: Zhang, Yang Z
Sent: Tuesday, August 05, 2014 7:11 PMTian, Kevin wrote on 2014-08-05:
Quoted 66 lines
Quoted 66 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Tuesday, August 05, 2014 12:52 AMQuoted 62 lines
Quoted 52 lines
Quoted 34 lines
On 05.08.14 at 09:35, <[email protected]> wrote: Tian, Kevin wrote on 2014-08-05: From: Jan Beulich [mailto:[email protected]]
Sent: Monday, August 04, 2014 12:35 AMQuoted 21 lines
Quoted 1 line
Quoted 1 line · On 04.08.14 at 07:05, <[email protected]> wrote:
Jan Beulich wrote on 2014-07-28:
Quoted 17 lines
[email protected]; [email protected]
Subject: Re: [PATCH v1 0/2] Extend ioreq-server to support page write protectionQuoted 12 lines
Quoted 1 line
Quoted 1 line · On 28.07.14 at 19:55, <[email protected]> wrote:
ioreq-server is proposed to forward PIO and MMIO request to multiple device models according to the io range. XenGT (Intel Graphics Virtualization technology, please refer to
https://01.org/xen/blogs/srclarkx/2013/graphics-virtualizat io n- xengt) driver reside in Dom0 as a virtual graphics device model also need to trap and emulate the guest's write operation to some specific memory pages, like the memory pages used by guest graphics driver as PPGTT(per-process graphics translation table). We add an new p2m type "p2m_ram_wp" to trap the page write operation. Page of this new p2m type is read only and for write, the request will go to device model via ioreq-server.Quoted 7 lines
Quoted 2 lines
So how is this write-protection supposed to work on the IOMMU side when sharing page tables?
Thanks for pointing out this question. Write-protection is not supposed to work when sharing page tables between EPT and vt-d. An explicit command line "iommu=no-sharept" should be setted for avoiding undesirable iommu fault.
Quoted 8 lines
Requiring the unconditional use of a specific command line option is certainly fine for experimental code, but not beyond that. Behavior should be correct by default in production code.
But what's worse here: The option _allows_ device side writes from the guest. Why would device side writes be okay, but CPU side ones not?
right, whether ept is shared or not doesn't address the concern here. In both cases we need maintain the same p2m view between CPU and device side, otherwise it's broken...
One option is to treat wp similar to logdirty, i.e. exclusive to VT-d device assignment, until in the future VT-d supports page fault. We can provide a boot option to override the default policy if user thinks OK.
Quoted 7 lines
2nd option, like Wei mentioned in another mail, is to treat such write protected PPGTT page tables as MMIO. new hypercall is required to change the p2m type between
p2m_ioandp2m_ram, based on allocation/free of guest page table. This way may impact performance on read though.Comments?
Another solution is using the EPT misconfiguration mechanism like what Xen does for MTRR emulation currently.
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
That would cause faults on reads as well, making it necessary to emulate them.
Then it's same effect of
p2m_mmio_dm.
p2m_mmio_dmwill break VT-d but EPT misconfiguration doesn't.I don't understand. Treating it as emulated io just means no valid p2m entry in EPT/VT-d. Note XenGT itself doesn't require VT-d, while you can still assign other devices this way.
Quoted 5 lines
From guest's view, it doesn't know the page is not valid in EPT/VT-d table. So if guest CPU touch this page, then hypervisor will intercept the access from EPT violation and handle it. But if guest uses that page as DMA buffer (a malicious guest may do it), then VT-d fault happens. Since DMA is not restartable, guest will never receive the right data.
Thanks
Kevin
Best regards,
Yang
Quoted 88 lines
From: Zhang, Yang Z
Sent: Tuesday, August 05, 2014 7:40 PMTian, Kevin wrote on 2014-08-06:
Quoted 33 lines
Quoted 33 lines
From: Zhang, Yang Z
Sent: Tuesday, August 05, 2014 7:11 PMTian, Kevin wrote on 2014-08-05:
Quoted 29 lines
Quoted 29 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Tuesday, August 05, 2014 12:52 AMQuoted 26 lines
Quoted 26 lines
Quoted 26 lines
On 05.08.14 at 09:35, <[email protected]> wrote: Tian, Kevin wrote on 2014-08-05: From: Jan Beulich [mailto:[email protected]]
Sent: Monday, August 04, 2014 12:35 AMQuoted 21 lines
Quoted 1 line
Quoted 1 line · On 04.08.14 at 07:05, <[email protected]> wrote:
Jan Beulich wrote on 2014-07-28:
Quoted 17 lines
[email protected]; [email protected]
Subject: Re: [PATCH v1 0/2] Extend ioreq-server to support page write protectionQuoted 12 lines
Quoted 1 line
Quoted 1 line · On 28.07.14 at 19:55, <[email protected]> wrote:
ioreq-server is proposed to forward PIO and MMIO request to multiple device models according to the io range. XenGT (Intel Graphics Virtualization technology, please refer to
https://01.org/xen/blogs/srclarkx/2013/graphics-virtualizat io n- xengt) driver reside in Dom0 as a virtual graphics device model also need to trap and emulate the guest's write operation to some specific memory pages, like the memory pages used by guest graphics driver as PPGTT(per-process graphics translation table). We add an new p2m type "p2m_ram_wp" to trap the page write operation. Page of this new p2m type is read only and for write, the request will go to device model via ioreq-server.Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
So how is this write-protection supposed to work on the IOMMU side when sharing page tables?
Thanks for pointing out this question. Write-protection is not supposed to work when sharing page tables between EPT and
vt-d.
Quoted 36 lines
Quoted 36 lines
Quoted 36 lines
Quoted 35 lines
Quoted 27 lines
Quoted 27 lines
Quoted 11 lines
Quoted 2 lines
An explicit command line "iommu=no-sharept" should be setted for avoiding undesirable iommu fault.
Requiring the unconditional use of a specific command line option is certainly fine for experimental code, but not beyond that. Behavior should be correct by default in production code.
But what's worse here: The option _allows_ device side writes from the guest. Why would device side writes be okay, but CPU side ones not?
right, whether ept is shared or not doesn't address the concern here. In both cases we need maintain the same p2m view between CPU and device side, otherwise it's broken...
One option is to treat wp similar to logdirty, i.e. exclusive to VT-d device assignment, until in the future VT-d supports page fault. We can provide a boot option to override the default policy if user thinks OK.
2nd option, like Wei mentioned in another mail, is to treat such write protected PPGTT page tables as MMIO. new hypercall is required to change the p2m type between
p2m_ioandp2m_ram, based on allocation/free of guest page table. This way may impact performance on read though.Quoted 4 lines
Quoted 1 line
Comments?
Another solution is using the EPT misconfiguration mechanism like what Xen does for MTRR emulation currently.
That would cause faults on reads as well, making it necessary to emulate them.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Then it's same effect of
p2m_mmio_dm.
p2m_mmio_dmwill break VT-d but EPT misconfiguration doesn't.I don't understand. Treating it as emulated io just means no valid p2m entry in EPT/VT-d. Note XenGT itself doesn't require VT-d, while you can still assign other devices this way.
Quoted 6 lines
From guest's view, it doesn't know the page is not valid in EPT/VT-d table. So if guest CPU touch this page, then hypervisor will intercept the access from EPT violation and handle it. But if guest uses that page as DMA buffer (a malicious guest may do it), then VT-d fault happens. Since DMA is not restartable, guest will never receive the right data.
as you said, it's malicious guest, that's why we need zap the VT-d entries to avoid mallicous DMA to GPU page tables. In sane case the page is allocated by gfx driver so other device drivers with VT-d assigned devices shouldn't use it as DMA target. There is no 'right data' to be ensured in such case. :-)
Thanks
Kevin
Quoted 104 lines
From: Tian, Kevin
Sent: Tuesday, August 05, 2014 7:49 PMQuoted 31 lines
From: Zhang, Yang Z
Sent: Tuesday, August 05, 2014 7:40 PMTian, Kevin wrote on 2014-08-06:
Quoted 27 lines
Quoted 27 lines
From: Zhang, Yang Z
Sent: Tuesday, August 05, 2014 7:11 PMTian, Kevin wrote on 2014-08-05:
Quoted 23 lines
Quoted 23 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Tuesday, August 05, 2014 12:52 AMQuoted 20 lines
Quoted 20 lines
Quoted 20 lines
On 05.08.14 at 09:35, <[email protected]> wrote: Tian, Kevin wrote on 2014-08-05: From: Jan Beulich [mailto:[email protected]]
Sent: Monday, August 04, 2014 12:35 AMQuoted 15 lines
Quoted 1 line
Quoted 1 line · On 04.08.14 at 07:05, <[email protected]> wrote:
Jan Beulich wrote on 2014-07-28:
Quoted 11 lines
[email protected]; [email protected]
Subject: Re: [PATCH v1 0/2] Extend ioreq-server to support page write protectionQuoted 7 lines
Quoted 1 line
On 28.07.14 at 19:55, <[email protected]> wrote:
ioreq-server is proposed to forward PIO and MMIO request to multiple device models according to the io range. XenGT (Intel Graphics Virtualization technology, please refer to
https://01.org/xen/blogs/srclarkx/2013/graphics-virtualizat io n- xengt) driver reside in Dom0 as a virtual graphics device model also need to trap and emulate the guest's writeoperation
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
to some specific memory pages, like the memory pages used by guest graphics driver as PPGTT(per-process graphics translation table). We add an new p2m type "
p2m_ram_wp" to trap thepage
Quoted 47 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
write operation. Page of this new p2m type is read only and for write, the request will go to device model via ioreq-server.
So how is this write-protection supposed to work on the IOMMU side when sharing page tables?
Thanks for pointing out this question. Write-protection is not supposed to work when sharing page tables between EPT and
vt-d.
Quoted 38 lines
Quoted 38 lines
Quoted 37 lines
Quoted 27 lines
Quoted 27 lines
Quoted 27 lines
Quoted 11 lines
Quoted 2 lines
An explicit command line "iommu=no-sharept" should be setted for avoiding undesirable iommu fault.
Requiring the unconditional use of a specific command line option is certainly fine for experimental code, but not beyond that. Behavior should be correct by default in production code.
But what's worse here: The option _allows_ device side writes from the guest. Why would device side writes be okay, but CPU side ones not?
right, whether ept is shared or not doesn't address the concern here. In both cases we need maintain the same p2m view between CPU and device side, otherwise it's broken...
One option is to treat wp similar to logdirty, i.e. exclusive to VT-d device assignment, until in the future VT-d supports page fault. We can provide a boot option to override the default policy if user thinks OK.
2nd option, like Wei mentioned in another mail, is to treat such write protected PPGTT page tables as MMIO. new hypercall is required to change the p2m type between
p2m_ioandp2m_ram, based on allocation/free of guest page table. This way may impact performance on read though.Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Comments?
Another solution is using the EPT misconfiguration mechanism like what Xen does for MTRR emulation currently.
That would cause faults on reads as well, making it necessary to emulate them.
Then it's same effect of
p2m_mmio_dm.Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
p2m_mmio_dmwill break VT-d but EPT misconfiguration doesn't.I don't understand. Treating it as emulated io just means no valid p2m entry in EPT/VT-d. Note XenGT itself doesn't require VT-d, while you can still assign other devices this way.
From guest's view, it doesn't know the page is not valid in EPT/VT-d table. So
if
Quoted 1 line
guest CPU touch this page, then hypervisor will intercept the access from
EPT
Quoted 1 line
violation and handle it. But if guest uses that page as DMA buffer (a
malicious
Quoted 1 line
guest may do it), then VT-d fault happens. Since DMA is not restartable,
guest
Quoted 2 lines
will never receive the right data.
Quoted 8 lines
as you said, it's malicious guest, that's why we need zap the VT-d entries to avoid
mallicous DMA to GPU page tables. In sane case the page is allocated by gfx driver
so other device drivers with VT-d assigned devices shouldn't use it as DMA target.
There is no 'right data' to be ensured in such case. :-)
Think about a malicious guest may DMA to any holes...
Thanks
Kevin
Tian, Kevin wrote on 2014-08-06:
Quoted 40 lines
Quoted 40 lines
From: Tian, Kevin
Sent: Tuesday, August 05, 2014 7:49 PMQuoted 37 lines
From: Zhang, Yang Z
Sent: Tuesday, August 05, 2014 7:40 PMTian, Kevin wrote on 2014-08-06:
Quoted 33 lines
Quoted 33 lines
From: Zhang, Yang Z
Sent: Tuesday, August 05, 2014 7:11 PMTian, Kevin wrote on 2014-08-05:
Quoted 29 lines
Quoted 29 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Tuesday, August 05, 2014 12:52 AMQuoted 26 lines
Quoted 26 lines
Quoted 26 lines · On 05.08.14 at 09:35, <[email protected]> wrote:
Tian, Kevin wrote on 2014-08-05: From: Jan Beulich [mailto:[email protected]]
Sent: Monday, August 04, 2014 12:35 AMQuoted 21 lines
Quoted 1 line
Quoted 1 line · On 04.08.14 at 07:05, <[email protected]> wrote:
Jan Beulich wrote on 2014-07-28:
Quoted 17 lines
[email protected]; [email protected]
Subject: Re: [PATCH v1 0/2] Extend ioreq-server to support page write protectionQuoted 13 lines
On 28.07.14 at 19:55, <[email protected]> wrote:
ioreq-server is proposed to forward PIO and MMIO request to
multiple device models according to the io range. XenGT
(Intel Graphics Virtualization technology, please refer to
https://01.org/xen/blogs/srclarkx/2013/graphics-virtual iz
at io n- xengt) driver reside in Dom0 as a virtual graphics
device model also need to trap and emulate the guest's write
operation to some specific memory pages, like the memory
pages used by guest graphics driver as PPGTT(per-process
graphics translation table). We add an new p2m type
"p2m_ram_wp" to trap the page write operation. Page of this
new p2m type is read only and for write, the request will go
to device model via
Quoted 61 lines
ioreq-server.
Quoted 60 lines
Quoted 51 lines
Quoted 42 lines
Quoted 42 lines
Quoted 42 lines
Quoted 41 lines
Quoted 33 lines
Quoted 33 lines
Quoted 9 lines
Quoted 8 lines
Quoted 3 lines
So how is this write-protection supposed to work on the IOMMU side when sharing page tables?
Thanks for pointing out this question. Write-protection is not supposed to work when sharing page tables between EPT and vt-d. An explicit command line "iommu=no-sharept" should be setted for avoiding undesirable iommu fault.
Quoted 8 lines
Requiring the unconditional use of a specific command line option is certainly fine for experimental code, but not beyond that. Behavior should be correct by default in production code.
But what's worse here: The option _allows_ device side writes from the guest. Why would device side writes be okay, but CPU side ones not?
right, whether ept is shared or not doesn't address the concern here. In both cases we need maintain the same p2m view between CPU and device side, otherwise it's broken...
One option is to treat wp similar to logdirty, i.e. exclusive to VT-d device assignment, until in the future VT-d supports page fault. We can provide a boot option to override the default policy if user thinks OK.
2nd option, like Wei mentioned in another mail, is to treat such write protected PPGTT page tables as MMIO. new hypercall is required to change the p2m type between
p2m_ioandp2m_ram, based on allocation/free of guest page table. This way may impact performance on read though.Quoted 4 lines
Quoted 1 line
Comments?
Another solution is using the EPT misconfiguration mechanism like what Xen does for MTRR emulation currently.
That would cause faults on reads as well, making it necessary to emulate them.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Then it's same effect of
p2m_mmio_dm.
p2m_mmio_dmwill break VT-d but EPT misconfiguration doesn't.I don't understand. Treating it as emulated io just means no valid p2m entry in EPT/VT-d. Note XenGT itself doesn't require VT-d, while you can still assign other devices this way.
Quoted 7 lines
From guest's view, it doesn't know the page is not valid in EPT/VT-d table. So if guest CPU touch this page, then hypervisor will intercept the access from EPT violation and handle it. But if guest uses that page as DMA buffer (a malicious guest may do it), then VT-d fault happens. Since DMA is not restartable, guest will never receive the right data.
as you said, it's malicious guest, that's why we need zap the VT-d
Malicious guest is just an example. We can assume a normal guest never does it.
Quoted 3 lines
Quoted 3 lines
entries to avoid mallicous DMA to GPU page tables. In sane case the page is allocated by gfx driver so other device drivers with VT-d assigned devices shouldn't use it as DMA target.
What about a guest really does it?
Quoted 4 lines
Quoted 2 lines
There is no 'right data' to be ensured in such case. :-)
Think about a malicious guest may DMA to any holes...
Other hole is not a RAM, so a VT-d fault is acceptable.
Quoted 3 lines
Thanks
Kevin
Best regards,
Yang
Quoted 101 lines · Yang Z
Tian, Kevin wrote on 2014-08-06:
Quoted 100 lines
Quoted 40 lines
From: Tian, Kevin
Sent: Tuesday, August 05, 2014 7:49 PMQuoted 37 lines
From: Zhang, Yang Z
Sent: Tuesday, August 05, 2014 7:40 PMTian, Kevin wrote on 2014-08-06:
Quoted 33 lines
Quoted 33 lines
From: Zhang, Yang Z
Sent: Tuesday, August 05, 2014 7:11 PMTian, Kevin wrote on 2014-08-05:
Quoted 29 lines
Quoted 29 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Tuesday, August 05, 2014 12:52 AMQuoted 26 lines
Quoted 26 lines
Quoted 26 lines · On 05.08.14 at 09:35, <[email protected]> wrote:
Tian, Kevin wrote on 2014-08-05: From: Jan Beulich [mailto:[email protected]]
Sent: Monday, August 04, 2014 12:35 AMQuoted 21 lines
Quoted 1 line
On 04.08.14 at 07:05, <[email protected]> wrote:
Jan Beulich wrote on 2014-07-28:
Quoted 17 lines
[email protected]; [email protected]
Subject: Re: [PATCH v1 0/2] Extend ioreq-server to
support page write protectionOn 28.07.14 at 19:55, <[email protected]> wrote:
ioreq-server is proposed to forward PIO and MMIO request to
multiple device models according to the io range. XenGT
(Intel Graphics Virtualization technology, please refer to
https://01.org/xen/blogs/srclarkx/2013/graphics-virtual iz
at io n- xengt) driver reside in Dom0 as a virtual graphics
device model also need to trap and emulate the guest's write
operation to some specific memory pages, like the memory
pages used by guest graphics driver as PPGTT(per-process
graphics translation table). We add an new p2m type
"p2m_ram_wp" to trap the page write operation. Page of this
new p2m type is read only and for write, the request will go
to device model viaioreq-server.
Quoted 59 lines
Quoted 58 lines
Quoted 42 lines
Quoted 42 lines
Quoted 42 lines
Quoted 41 lines
Quoted 33 lines
Quoted 33 lines
Quoted 9 lines
Quoted 8 lines
Quoted 3 lines
So how is this write-protection supposed to work on the
IOMMU side when sharing page tables?Thanks for pointing out this question. Write-protection is not supposed to work when sharing page tables between EPT and vt-d. An explicit command line "iommu=no-sharept" should be setted for avoiding undesirable iommu fault.
Quoted 8 lines
Requiring the unconditional use of a specific command line option is certainly fine for experimental code, but not beyond that. Behavior should be correct by default in production code.
But what's worse here: The option _allows_ device side writes from the guest. Why would device side writes be okay, but CPU side ones not?
right, whether ept is shared or not doesn't address the concern here. In both cases we need maintain the same p2m view between CPU and device side, otherwise it's broken...
One option is to treat wp similar to logdirty, i.e. exclusive to VT-d device assignment, until in the future VT-d supports page fault. We can provide a boot option to override the default policy if user thinks OK.
2nd option, like Wei mentioned in another mail, is to treat such write protected PPGTT page tables as MMIO. new hypercall is required to change the p2m type between
p2m_ioandp2m_ram, based on allocation/free of guest page table. This way may impact performance on read though.Quoted 4 lines
Quoted 1 line
Comments?
Another solution is using the EPT misconfiguration mechanism like what Xen does for MTRR emulation currently.
That would cause faults on reads as well, making it necessary to emulate them.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Then it's same effect of
p2m_mmio_dm.
p2m_mmio_dmwill break VT-d but EPT misconfiguration doesn't.I don't understand. Treating it as emulated io just means no valid p2m entry in EPT/VT-d. Note XenGT itself doesn't require VT-d, while you can still assign other devices this way.
From guest's view, it doesn't know the page is not valid in EPT/VT-d table. So if guest CPU touch this page, then hypervisor will intercept the access from EPT violation and handle it. But if guest uses that page as DMA buffer (a malicious guest may do it), then VT-d fault happens. Since DMA is not restartable, guest will never receive the right data.
Quoted 3 lines
Quoted 1 line
Quoted 1 line
as you said, it's malicious guest, that's why we need zap the VT-d
Malicious guest is just an example. We can assume a normal guest never does it.
What!? No you can't. That is thundering towards an XSA! The normal guest can become compromised.
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
entries to avoid mallicous DMA to GPU page tables. In sane case the page is allocated by gfx driver so other device drivers with VT-d assigned devices shouldn't use it as DMA target.
What about a guest really does it?
Oh, here you say it can do it. A bit confusing.
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
There is no 'right data' to be ensured in such case. :-)
Think about a malicious guest may DMA to any holes...
Other hole is not a RAM, so a VT-d fault is acceptable.
I feel that we talking about the same issue that had been raised before about EPT and VT-d sharing a page and having issues with DMAing in RAM regions (like into the framebuffer).
And my understanding is that Intel is going to fix it, except that it is on the 'low-priority'. It sounds to me that it should be raised to a higher priority and be taken care of.
Quoted 15 lines
Quoted 4 lines
Thanks
KevinBest regards,
Yang_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel
Quoted 110 lines
From: Konrad Rzeszutek Wilk [mailto:[email protected]]
Sent: Wednesday, August 06, 2014 8:01 AMQuoted 30 lines · Yang Z
Tian, Kevin wrote on 2014-08-06:
Quoted 29 lines
Quoted 29 lines
From: Tian, Kevin
Sent: Tuesday, August 05, 2014 7:49 PMQuoted 26 lines
From: Zhang, Yang Z
Sent: Tuesday, August 05, 2014 7:40 PMTian, Kevin wrote on 2014-08-06:
Quoted 22 lines
Quoted 22 lines
From: Zhang, Yang Z
Sent: Tuesday, August 05, 2014 7:11 PMTian, Kevin wrote on 2014-08-05:
Quoted 18 lines
Quoted 18 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Tuesday, August 05, 2014 12:52 AMQuoted 15 lines
Quoted 15 lines
Quoted 15 lines · On 05.08.14 at 09:35, <[email protected]> wrote:
Tian, Kevin wrote on 2014-08-05: From: Jan Beulich [mailto:[email protected]]
Sent: Monday, August 04, 2014 12:35 AMQuoted 10 lines
On 04.08.14 at 07:05, <[email protected]> wrote:
Jan Beulich wrote on 2014-07-28:
[email protected]; [email protected]
Subject: Re: [PATCH v1 0/2] Extend ioreq-server to
support page write protectionOn 28.07.14 at 19:55, <[email protected]> wrote:
ioreq-server is proposed to forward PIO and MMIO requestto
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
multiple device models according to the io range. XenGT
(Intel Graphics Virtualization technology, please refer to
https://01.org/xen/blogs/srclarkx/2013/graphics-virtual iz
at io n- xengt) driver reside in Dom0 as a virtual graphics
device model also need to trap and emulate the guest'swrite
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
operation to some specific memory pages, like the memory
pages used by guest graphics driver as PPGTT(per-process
graphics translation table). We add an new p2m type
"p2m_ram_wp" to trap the page write operation. Page ofthis
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
new p2m type is read only and for write, the request will go
to device model viaioreq-server.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
So how is this write-protection supposed to work on the
IOMMU side when sharing page tables?Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Thanks for pointing out this question. Write-protection is not
supposed to work when sharing page tables between EPT and
vt-d. An explicit command line "iommu=no-sharept" should be
setted for avoiding undesirable iommu fault.Requiring the unconditional use of a specific command line option is certainly fine for experimental code, but not beyond
that.
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 6 lines
Behavior should be correct by default in production code.
But what's worse here: The option _allows_ device side writes from the guest. Why would device side writes be okay, but CPU side ones not?
right, whether ept is shared or not doesn't address the concern here. In both cases we need maintain the same p2m view
between
Quoted 39 lines
Quoted 39 lines
Quoted 39 lines
Quoted 38 lines
Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
Quoted 21 lines
Quoted 13 lines
Quoted 13 lines
CPU and device side, otherwise it's broken...
One option is to treat wp similar to logdirty, i.e. exclusive to VT-d device assignment, until in the future VT-d supports page fault. We can provide a boot option to override the default policy if user thinks OK.
2nd option, like Wei mentioned in another mail, is to treat such write protected PPGTT page tables as MMIO. new hypercall is required to change the p2m type between
p2m_ioandp2m_ram, based on allocation/free of guest page table. This way may impact performance on read though.Quoted 4 lines
Quoted 1 line
Comments?
Another solution is using the EPT misconfiguration mechanism like what Xen does for MTRR emulation currently.
That would cause faults on reads as well, making it necessary to emulate them.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Then it's same effect of
p2m_mmio_dm.
p2m_mmio_dmwill break VT-d but EPT misconfiguration doesn't.I don't understand. Treating it as emulated io just means no valid p2m entry in EPT/VT-d. Note XenGT itself doesn't require VT-d, while you can still assign other devices this way.
From guest's view, it doesn't know the page is not valid in EPT/VT-d table. So if guest CPU touch this page, then hypervisor will intercept the access from EPT violation and handle it. But if guest uses that page as DMA buffer (a malicious guest may do it), then VT-d fault happens. Since DMA is not restartable, guest will never receive the right data.
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
as you said, it's malicious guest, that's why we need zap the VT-d
Malicious guest is just an example. We can assume a normal guest never
does it.
What!? No you can't. That is thundering towards an XSA! The normal guest can become compromised.
It would be a severe security hole to write-protect only the CPU path while leaving DMA writes valid. When we say a page is write-protected, it must be consistently enforced in both CPU/DMA paths. Now there is no other way to achieve it lacking of VT-d page fault, except treating it fully as a emulated MMIO range (means removing mapping in both EPT/VT-d page table), or excluding VT-d like this patch originally does.
Quoted 17 lines
Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
entries to avoid mallicous DMA to GPU page tables. In sane case the page is allocated by gfx driver so other device drivers with VT-d assigned devices shouldn't use it as DMA target.
What about a guest really does it?
Oh, here you say it can do it. A bit confusing.
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
There is no 'right data' to be ensured in such case. :-)
Think about a malicious guest may DMA to any holes...
Other hole is not a RAM, so a VT-d fault is acceptable.
Quoted 8 lines
I feel that we talking about the same issue that had been raised before about EPT and VT-d sharing a page and having issues with DMAing in RAM regions (like into the framebuffer).
And my understanding is that Intel is going to fix it, except that it is on the 'low-priority'. It sounds to me that it should be raised to a higher priority and be taken care of.
I'm not in original discussion loop, but my feeling is w/o VT-d fault support anything related to write-protection (e.g. logdirty) will remains a gap here (i.e. the feature has to be exclusive to VT-d usage)
Thanks
Kevin
Quoted 9 lines
Quoted 8 lines
Quoted 1 line · On 06.08.14 at 18:08, <[email protected]> wrote:
From: Konrad Rzeszutek Wilk [mailto:[email protected]]
Quoted 1 line · Yang Z
Malicious guest is just an example. We can assume a normal guest never
does it.
What!? No you can't. That is thundering towards an XSA! The normal guest can become compromised.
Quoted 6 lines
It would be a severe security hole to write-protect only the CPU path while leaving DMA writes valid. When we say a page is write-protected, it must be consistently enforced in both CPU/DMA paths. Now there is no other way to achieve it lacking of VT-d page fault, except treating it fully as a emulated MMIO range (means removing mapping in both EPT/VT-d page table), or excluding VT-d like this patch originally does.
A fully emulated range means more than just removing the mapping
though: No passed through device must access the memory range, since you can't emulate such accesses. While this is fine for existing uses (LAPIC, HPET, ...), I don't think it is for things like a frame buffer. If otoh talk is exclusively about GPU page tables, then this ought to be fine, but would - to me - not fit with other aspects discussed in the context of these patches (like the supposedly large number of pages needing to be tracked for a guest).
Jan
Quoted 19 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Wednesday, August 06, 2014 11:45 PMQuoted 9 lines
Quoted 8 lines
Quoted 1 line · On 06.08.14 at 18:08, <[email protected]> wrote:
From: Konrad Rzeszutek Wilk [mailto:[email protected]]
Quoted 1 line · Yang Z
Malicious guest is just an example. We can assume a normal guest never
does it.
What!? No you can't. That is thundering towards an XSA! The normal guest can become compromised.
Quoted 6 lines
It would be a severe security hole to write-protect only the CPU path while leaving DMA writes valid. When we say a page is write-protected, it must be consistently enforced in both CPU/DMA paths. Now there is no other way to achieve it lacking of VT-d page fault, except treating it fully as a emulated MMIO range (means removing mapping in both EPT/VT-d page table), or excluding VT-d like this patch originally does.
Quoted 5 lines
A fully emulated range means more than just removing the mapping
though: No passed through device must access the memory range, since you can't emulate such accesses. While this is fine for existing uses (LAPIC, HPET, ...), I don't think it is for things like a frame buffer. If otoh talk is exclusively about GPU page tables, then this
I don't understand here. why would a pass-through device wants to access frame buffer (I think you meant 'virtual' frame buffer)? Note XenGT itself doesn't use VT-d, and we didn't see such usage of having another device DMA to GPU resource. Then it's just same as any existing emulated MMIO ranges e.g. on a virtual NIC, where no pass-through device would access them.
Quoted 4 lines
ought to be fine, but would - to me - not fit with other aspects discussed in the context of these patches (like the supposedly large number of pages needing to be tracked for a guest).
So what's your suggestion here?
Thanks
Kevin
Quoted 17 lines
Quoted 11 lines
Quoted 1 line · On 07.08.14 at 18:28, <[email protected]> wrote:
From: Jan Beulich [mailto:[email protected]]
Sent: Wednesday, August 06, 2014 11:45 PMQuoted 7 lines
Quoted 1 line
Quoted 1 line · On 06.08.14 at 18:08, <[email protected]> wrote:
It would be a severe security hole to write-protect only the CPU path while leaving DMA writes valid. When we say a page is write-protected, it must be consistently enforced in both CPU/DMA paths. Now there is no other way to achieve it lacking of VT-d page fault, except treating it fully as a emulated MMIO range (means removing mapping in both EPT/VT-d page table), or excluding VT-d like this patch originally does.
Quoted 5 lines
A fully emulated range means more than just removing the mapping
though: No passed through device must access the memory range, since you can't emulate such accesses. While this is fine for existing uses (LAPIC, HPET, ...), I don't think it is for things like a frame buffer. If otoh talk is exclusively about GPU page tables, then this
Quoted 6 lines
I don't understand here. why would a pass-through device wants to access frame buffer (I think you meant 'virtual' frame buffer)? Note XenGT itself doesn't use VT-d, and we didn't see such usage of having another device DMA to GPU resource. Then it's just same as any existing emulated MMIO ranges e.g. on a virtual NIC, where no pass-through device would access them.
This is the same as the discussion about VT-d code not supporting large pages right now: An OS _might_ do clever things (merely dumping the frame buffer verbatim to disk or network would be one of the more trivial examples). Current OSes (or more precisely their drivers) may not do this, but I can only repeat my fundamental position here: We should not make assumptions about HVM guest behavior except in cases where we want to optimize certain things.
Quoted 5 lines
Quoted 3 lines
ought to be fine, but would - to me - not fit with other aspects discussed in the context of these patches (like the supposedly large number of pages needing to be tracked for a guest).
So what's your suggestion here?
I can't answer that without you clarifying whether talk earlier on was about the frame buffer or page tables.
Jan
Quoted 31 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Thursday, August 07, 2014 11:33 PMQuoted 5 lines
Quoted 5 lines
Quoted 1 line · On 07.08.14 at 18:28, <[email protected]> wrote:
From: Jan Beulich [mailto:[email protected]]
Sent: Wednesday, August 06, 2014 11:45 PMQuoted 2 lines
Quoted 1 line
Quoted 1 line · On 06.08.14 at 18:08, <[email protected]> wrote:
It would be a severe security hole to write-protect only the CPU path
while
Quoted 1 line
Quoted 1 line
Quoted 1 line
leaving DMA writes valid. When we say a page is write-protected, it must
be
Quoted 1 line
Quoted 1 line
Quoted 1 line
consistently enforced in both CPU/DMA paths. Now there is no other way
to
Quoted 1 line
Quoted 1 line
Quoted 1 line
achieve it lacking of VT-d page fault, except treating it fully as a
emulated
Quoted 9 lines
Quoted 8 lines
Quoted 2 lines
MMIO range (means removing mapping in both EPT/VT-d page table), or excluding VT-d like this patch originally does.
A fully emulated range means more than just removing the mapping
though: No passed through device must access the memory range, since you can't emulate such accesses. While this is fine for existing uses (LAPIC, HPET, ...), I don't think it is for things like a frame buffer. If otoh talk is exclusively about GPU page tables, then thisQuoted 6 lines
I don't understand here. why would a pass-through device wants to access frame buffer (I think you meant 'virtual' frame buffer)? Note XenGT itself doesn't use VT-d, and we didn't see such usage of having another device DMA to GPU resource. Then it's just same as any existing emulated MMIO ranges e.g. on a virtual NIC, where no pass-through device would access them.
Quoted 7 lines
This is the same as the discussion about VT-d code not supporting large pages right now: An OS _might_ do clever things (merely dumping the frame buffer verbatim to disk or network would be one of the more trivial examples). Current OSes (or more precisely their drivers) may not do this, but I can only repeat my fundamental position here: We should not make assumptions about HVM guest behavior except in cases where we want to optimize certain things.
with a complete HW support yes we should make assumption for guest behavior (e.g. with VT-x we can support any type of OS), but when HW support in VT-d is not fully ready, then making some assumption makes sense especially when no real usage is observed in this case.
Quoted 10 lines
Quoted 6 lines
Quoted 4 lines
ought to be fine, but would - to me - not fit with other aspects discussed in the context of these patches (like the supposedly large number of pages needing to be tracked for a guest).
So what's your suggestion here?
I can't answer that without you clarifying whether talk earlier on was about the frame buffer or page tables.
In our case it's only GPU page table. XenGT assigns frame buffer to the VM so there's no concern in that path.
Thanks
Kevin
Quoted 45 lines
From: Tian, Kevin
Sent: Friday, August 08, 2014 9:02 AMQuoted 10 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Thursday, August 07, 2014 11:33 PMQuoted 5 lines
Quoted 5 lines
Quoted 1 line · On 07.08.14 at 18:28, <[email protected]> wrote:
From: Jan Beulich [mailto:[email protected]]
Sent: Wednesday, August 06, 2014 11:45 PMQuoted 2 lines
Quoted 1 line
Quoted 1 line · On 06.08.14 at 18:08, <[email protected]> wrote:
It would be a severe security hole to write-protect only the CPU path
while
Quoted 1 line
Quoted 1 line
Quoted 1 line
leaving DMA writes valid. When we say a page is write-protected, it
must
Quoted 2 lines
be
Quoted 1 line
Quoted 1 line
Quoted 1 line
consistently enforced in both CPU/DMA paths. Now there is no other
way
Quoted 4 lines
to
Quoted 1 line
Quoted 1 line
Quoted 1 line
achieve it lacking of VT-d page fault, except treating it fully as a
emulated
Quoted 1 line
Quoted 1 line
Quoted 1 line
MMIO range (means removing mapping in both EPT/VT-d page table),
or
Quoted 15 lines
Quoted 8 lines
Quoted 7 lines
Quoted 1 line
excluding VT-d like this patch originally does.
A fully emulated range means more than just removing the mapping
though: No passed through device must access the memory range, since you can't emulate such accesses. While this is fine for existing uses (LAPIC, HPET, ...), I don't think it is for things like a frame buffer. If otoh talk is exclusively about GPU page tables, then thisQuoted 6 lines
I don't understand here. why would a pass-through device wants to access frame buffer (I think you meant 'virtual' frame buffer)? Note XenGT itself doesn't use VT-d, and we didn't see such usage of having another device DMA to GPU resource. Then it's just same as any existing emulated MMIO ranges e.g. on a virtual NIC, where no pass-through device would access them.
Quoted 7 lines
This is the same as the discussion about VT-d code not supporting large pages right now: An OS _might_ do clever things (merely dumping the frame buffer verbatim to disk or network would be one of the more trivial examples). Current OSes (or more precisely their drivers) may not do this, but I can only repeat my fundamental position here: We should not make assumptions about HVM guest behavior except in cases where we want to optimize certain things.
Quoted 1 line
with a complete HW support yes we should make assumption for guest
should -> shouldn't
Quoted 4 lines
behavior (e.g. with VT-x we can support any type of OS), but when HW support in VT-d is not fully ready, then making some assumption makes sense especially when no real usage is observed in this case.
Some summary of the discussion:
Without vt-d page fault support, a general write-protection is not possible. It's impossible to protect the write of both CPU and DMA path. But note that XenGT itself doesn't use VT-d, and there is no another usage of having another device DMA to GPU resource. Or even other device use the protect page for its DMA buffer, like a malicious guest, there is no security hole since it can only access it's our memory range. So, just keep protecting the CPU write path may be feasible for this moment.
Another way is treat the page as MMIO ranges(means removing mapping in both EPD/VT-D page table). In this way, both read/write access should be forwarded and emulated. For XenGT usage case, we observed that for the Linux guest, there is no read access to the ppgtt page tables. And for the windows guest, the read access is just about 20% of write. The read overhead is relatively small, so treat the pages as MMIO range may be possible.
Any comments?
Regards
Wei
Quoted 70 lines
-----Original Message-----
From: Tian, Kevin
Sent: Saturday, August 9, 2014 12:04 AM
To: Jan Beulich
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Dugger, Donald D; Ye, Wei; Zhang, Yang Z; Lv, Zhiyuan; [email protected]; Konrad Rzeszutek Wilk; [email protected]; [email protected]
Subject: RE: [Xen-devel] [PATCH v1 0/2] Extend ioreq-server to support page write protectionQuoted 51 lines
From: Tian, Kevin
Sent: Friday, August 08, 2014 9:02 AMQuoted 12 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Thursday, August 07, 2014 11:33 PMQuoted 6 lines
Quoted 6 lines
Quoted 1 line · On 07.08.14 at 18:28, <[email protected]> wrote:
From: Jan Beulich [mailto:[email protected]]
Sent: Wednesday, August 06, 2014 11:45 PMQuoted 3 lines
Quoted 1 line
Quoted 1 line · On 06.08.14 at 18:08, <[email protected]> wrote:
It would be a severe security hole to write-protect only the CPU path
while
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
leaving DMA writes valid. When we say a page is
write-protected, itmust
Quoted 3 lines
be
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
consistently enforced in both CPU/DMA paths. Now there is no other
way
Quoted 6 lines
to
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
achieve it lacking of VT-d page fault, except treating it fully as a
emulated
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
MMIO range (means removing mapping in both EPT/VT-d page table),
or
Quoted 16 lines
Quoted 9 lines
Quoted 8 lines
Quoted 1 line
excluding VT-d like this patch originally does.
A fully emulated range means more than just removing the mapping
though: No passed through device must access the memory range, since you can't emulate such accesses. While this is fine for existing uses (LAPIC, HPET, ...), I don't think it is for things like a frame buffer. If otoh talk is exclusively about GPU page tables, then thisQuoted 6 lines
I don't understand here. why would a pass-through device wants to access frame buffer (I think you meant 'virtual' frame buffer)? Note XenGT itself doesn't use VT-d, and we didn't see such usage of having another device DMA to GPU resource. Then it's just same as any existing emulated MMIO ranges e.g. on a virtual NIC, where no pass-through device would access them.
Quoted 7 lines
This is the same as the discussion about VT-d code not supporting large pages right now: An OS _might_ do clever things (merely dumping the frame buffer verbatim to disk or network would be one of the more trivial examples). Current OSes (or more precisely their drivers) may not do this, but I can only repeat my fundamental position here: We should not make assumptions about HVM guest behavior except in cases where we want to optimize certain things.
Quoted 1 line
with a complete HW support yes we should make assumption for guest
should -> shouldn't
Quoted 4 lines
behavior (e.g. with VT-x we can support any type of OS), but when HW support in VT-d is not fully ready, then making some assumption makes sense especially when no real usage is observed in this case.
Quoted 6 lines · Wei
Some summary of the discussion:
Without vt-d page fault support, a general write-protection is not possible. It's impossible to protect the write of both CPU and DMA path. But note that XenGT itself doesn't use VT-d, and there is no another usage of having another device DMA to GPU resource.
None that you're aware of. But there might be some out-of-tree users?
Quoted 4 lines
Or even
other device use the protect page for its DMA buffer, like a malicious guest, there is no security hole since it can only access it's our memory range.
Not true at all! If the guest has a read-only mapping it must be read-only for a reason. For example, it might have a read-only grant mapping of another VM's memory. Or an out-of-VM security tool might be enforcing W^X permissions.
Quoted 1 line
So, just keep protecting the CPU write path may be feasible for this moment.
I don't think so.
Quoted 7 lines
Another way is treat the page as MMIO ranges(means removing mapping in both EPD/VT-D page table). In this way, both read/write access should be forwarded and emulated. For XenGT usage case, we observed that for the Linux guest, there is no read access to the ppgtt page tables. And for the windows guest, the read access is just about 20% of write. The read overhead is relatively small, so treat the pages as MMIO range may be possible.
That might work for XenGT but it doesn't address the general problem at all. AFAICT the only _safe_ way to make a page read-only to a guest that can issue DMA is to remove it from the IOMMU tables - and that means having separate EPT and vtD tables, until hardware support for restartable IOMMU faults comes along.
Cheers,
Tim.
Quoted 4 lines
From: Tim Deegan [mailto:[email protected]]
Sent: Wednesday, August 13, 2014 1:39 AMAt 23:15 +0000 on 12 Aug (1407881757), Ye, Wei wrote:
Quoted 8 lines
Quoted 6 lines
Some summary of the discussion:
Without vt-d page fault support, a general write-protection is not possible. It's impossible to protect the write of both CPU and DMA path. But note that XenGT itself doesn't use VT-d, and there is no another usage of having another device DMA to GPU resource.
None that you're aware of. But there might be some out-of-tree users?
If there is, then it has to be in the unsupported feature list. I think it's fair enough since we all know w/o complete VT-d page fault support there's no solution for such usage. Pretty much like old days big real mode tricks before hardware support is ready.
Quoted 10 lines
Quoted 5 lines
Or even
other device use the protect page for its DMA buffer, like a malicious guest, there is no security hole since it can only access it's our memory range.Not true at all! If the guest has a read-only mapping it must be read-only for a reason. For example, it might have a read-only grant mapping of another VM's memory. Or an out-of-VM security tool might be enforcing W^X permissions.
sorry original statement is inaccurate. there is no cross-VM security hole since EPT/VT-d are fully under VMM's control, but yes out-of-VM security tool might be broken...
Quoted 5 lines
Quoted 2 lines
So, just keep protecting the CPU write path may be feasible for this
moment.
I don't think so.
Could we introduce a cpu-write-protection attribute only for functional usages which don't care about DMA path like in XenGT? Then out-of-VM security tool relying on complete write-protection in both CPU/DMA paths still need to wait full VT-d page fault. I know it's not clean, but want to hear your comments anything we can do here in current stage. :-)
Quoted 9 lines
Quoted 8 lines
Another way is treat the page as MMIO ranges(means removing mapping in both EPD/VT-D page table). In this way, both read/write access should be forwarded and emulated. For XenGT usage case, we observed that for the Linux guest, there is no read access to the ppgtt page tables. And for the windows guest, the read access is just about 20% of write. The read overhead is relatively small, so treat the pages as MMIO range may be possible.
Quoted 5 lines
That might work for XenGT but it doesn't address the general problem at all. AFAICT the only _safe_ way to make a page read-only to a guest that can issue DMA is to remove it from the IOMMU tables - and that means having separate EPT and vtD tables, until hardware support for restartable IOMMU faults comes along.
Aren't we talking about a similar option? ours is more conservative, not only removing from IOMMU table, but also removing from EPT so both r/w are trapped which serves the read-only requirements with performance penalty. along this way we didn't aim to provide a general write-protection mechanism, but just to extend ioreq server to support scattered mmio ranges. If adding a new p2m type is deemed dirty, we may go to this option since it can meet XenGT's requirement, and doesn't force other usages to go same route.
Actually separate or shared EPT/VT-d tables doesn't mean anything here. Removing the page from IOMMU tables still breaks guest's attempt. You protected it from write, but didn't emulate the DMA so it's still broken.
Thanks
Kevin
Quoted 11 lines · Kevin
Quoted 11 lines
From: Tim Deegan [mailto:[email protected]]
Sent: Wednesday, August 13, 2014 1:39 AMQuoted 6 lines · Wei
Some summary of the discussion:
Without vt-d page fault support, a general write-protection is not possible. It's impossible to protect the write of both CPU and DMA path. But note that XenGT itself doesn't use VT-d, and there is no another usage of having another device DMA to GPU resource.
Quoted 6 lines
Quoted 1 line
None that you're aware of. But there might be some out-of-tree users?
If there is, then it has to be in the unsupported feature list. I think it's fair enough since we all know w/o complete VT-d page fault support there's no solution for such usage. Pretty much like old days big real mode tricks before hardware support is ready.
I think there's confusion here about what we're taling about. Yes, until we have h/w support for restarting faulting DMA the IOMMU feature set will be incomplete. The important thing is that it should fail safe -- i.e. if a guest has a read-only mapping of a page it is better to disallow DMA altogether than to allow read-write DMA.
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
Or even
other device use the protect page for its DMA buffer, like a malicious guest, there is no security hole since it can only access it's our memory range.
Quoted 8 lines
Quoted 4 lines
Not true at all! If the guest has a read-only mapping it must be read-only for a reason. For example, it might have a read-only grant mapping of another VM's memory. Or an out-of-VM security tool might be enforcing W^X permissions.
sorry original statement is inaccurate. there is no cross-VM security hole since EPT/VT-d are fully under VMM's control, but yes out-of-VM security tool might be broken...
As I just said: "it might have a read-only grant mapping of another VM's memory". We can't let it DMA into that mapping.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
So, just keep protecting the CPU write path may be feasible for this
moment.
I don't think so.
Could we introduce a cpu-write-protection attribute only for functional usages which don't care about DMA path like in XenGT?
Possibly. Sorry for going over things that have probably already been covered but AIUI you are saying that XenGT needs to allow full DMA access to this memory area but only read-only CPU access. If that's what's needed then I think a new type with these semantics is the best way of doing it (and requiring no shared IOMMU/HAP tables). But:
- it's pointless if the guest can make the device issue DMA for it, (with a blitting engine or similar) since it can just use that DMA to make the writes it wants; and
- for the same reason, you will need to give the GPU its own set of IOMMU tables, separate from the ones used for other devices that the guest controls. Otherwise the guest can use its NIC to DMA into the read-only area.
Quoted 14 lines
Quoted 8 lines
Quoted 7 lines
Another way is treat the page as MMIO ranges(means removing mapping in both EPD/VT-D page table). In this way, both read/write access should be forwarded and emulated. For XenGT usage case, we observed that for the Linux guest, there is no read access to the ppgtt page tables. And for the windows guest, the read access is just about 20% of write. The read overhead is relatively small, so treat the pages as MMIO range may be possible.
Quoted 5 lines
That might work for XenGT but it doesn't address the general problem at all. AFAICT the only _safe_ way to make a page read-only to a guest that can issue DMA is to remove it from the IOMMU tables - and that means having separate EPT and vtD tables, until hardware support for restartable IOMMU faults comes along.
Quoted 3 lines
Aren't we talking about a similar option? ours is more conservative, not only removing from IOMMU table, but also removing from EPT so both r/w are trapped which serves the read-only requirements with performance penalty.
Yes they are similar. I would say mine is more conservative in a different way - not using the emulator unneccesarily. :)
Cheers,
Tim.
Quoted 22 lines
From: Tim Deegan [mailto:[email protected]]
Sent: Thursday, August 14, 2014 1:08 AMQuoted 11 lines · Kevin
Quoted 11 lines
From: Tim Deegan [mailto:[email protected]]
Sent: Wednesday, August 13, 2014 1:39 AMQuoted 6 lines · Wei
Some summary of the discussion:
Without vt-d page fault support, a general write-protection is not possible. It's impossible to protect the write of both CPU and DMA path. But note that XenGT itself doesn't use VT-d, and there is no another usage of having another device DMA to GPU resource.
Quoted 6 lines
Quoted 1 line
None that you're aware of. But there might be some out-of-tree users?
If there is, then it has to be in the unsupported feature list. I think it's fair enough since we all know w/o complete VT-d page fault support there's no solution for such usage. Pretty much like old days big real mode tricks before hardware support is ready.
Quoted 5 lines
I think there's confusion here about what we're taling about. Yes, until we have h/w support for restarting faulting DMA the IOMMU feature set will be incomplete. The important thing is that it should fail safe -- i.e. if a guest has a read-only mapping of a page it is better to disallow DMA altogether than to allow read-write DMA.
Yes, there did be some confusions here. :-)
I think we are on the same page that, complete 'read-only' or complete 'write-protection' is impossible w/o restarting faulting DMA, because there is no way to enforce the access permission in the IOMMU side.
Then whatever 'read-only' or 'write-protection" discussion now is a partial solution where only CPU path is protected, with the assumption that no valid usage of using DMA to access the concerned memory area. And then I agree disallowing DMA becomes a fail-safe means to capture violating usages.
Quoted 23 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Or even
other device use the protect page for its DMA buffer, like a malicious guest, there is no security hole since it can only access it's our memory range.Quoted 8 lines
Quoted 4 lines
Not true at all! If the guest has a read-only mapping it must be read-only for a reason. For example, it might have a read-only grant mapping of another VM's memory. Or an out-of-VM security tool might be enforcing W^X permissions.
sorry original statement is inaccurate. there is no cross-VM security hole since EPT/VT-d are fully under VMM's control, but yes out-of-VM security tool might be broken...
As I just said: "it might have a read-only grant mapping of another VM's memory". We can't let it DMA into that mapping.
Quoted 5 lines
Quoted 4 lines
Quoted 1 line
So, just keep protecting the CPU write path may be feasible for this
moment.
I don't think so.
Quoted 8 lines
Quoted 2 lines
Could we introduce a cpu-write-protection attribute only for functional usages which don't care about DMA path like in XenGT?
Possibly. Sorry for going over things that have probably already been covered but AIUI you are saying that XenGT needs to allow full DMA access to this memory area but only read-only CPU access. If that's what's needed then I think a new type with these semantics is the best way of doing it (and requiring no shared IOMMU/HAP tables). But:
It's about GPU page table write-protection, so XenGT needs to trap writes in both CPU/DMA paths:
* CPU writes are captured by read-only permission in EPT
* GPU writes are captured by scanning GPU commands before they're submitted for execution, as part of the 'mediation' concept in XenGT mediated pass-through architecture.
Note XenGT doesn't use IOMMU because w/o SR-IOV a single device entry can't be used to server multiple VMs.
So XenGT can achieve true 'write-protection' w/o IOMMU restarting fault support. However it's very GPU specific. As a general implementation, I think a new type with below semantics should be enough:
- only CPU access to the protected area, so it's removed from IOMMU page table
- require IOMMU/HAP seperate
How would you call such a new type then? just p2m_write_protection or p2m_cpu_write_protection? (still need to differentiate write-protection from read-only, since the latter implies no writes)
Quoted 3 lines
- it's pointless if the guest can make the device issue DMA for it, (with a blitting engine or similar) since it can just use that DMA to make the writes it wants; and
Based on above explanation it's not a problem
Quoted 4 lines
- for the same reason, you will need to give the GPU its own set of IOMMU tables, separate from the ones used for other devices that the guest controls. Otherwise the guest can use its NIC to DMA into the read-only area.
XenGT doesn't use IOMMU. So it's enough as long as we zap the range from all IOMMU page tables
in the future when we have SR-IOV GPU support, there's no need to shadow GPU page table then, and thus no such a requirement of write-protection.
Quoted 16 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Another way is treat the page as MMIO ranges(means removing
mapping
Quoted 13 lines
Quoted 12 lines
Quoted 6 lines
in both EPD/VT-D page table). In this way, both read/write access should be forwarded and emulated. For XenGT usage case, we observed that for the Linux guest, there is no read access to the ppgtt page tables. And for the windows guest, the read access is just about 20% of write. The read overhead is relatively small, so treat the pages as MMIO range may be possible.
That might work for XenGT but it doesn't address the general problem at all. AFAICT the only _safe_ way to make a page read-only to a guest that can issue DMA is to remove it from the IOMMU tables - and that means having separate EPT and vtD tables, until hardware support for restartable IOMMU faults comes along.
Quoted 7 lines
Quoted 3 lines
Aren't we talking about a similar option? ours is more conservative, not only removing from IOMMU table, but also removing from EPT so both r/w are trapped which serves the read-only requirements with performance penalty.
Yes they are similar. I would say mine is more conservative in a different way - not using the emulator unneccesarily. :)
yeah from different view. Our original thought is if new p2m type is tricky, then treating it as mmio could be an easier thing to extend. Now since you also agree adding a new type is reasonable, then we can continue this direction (actually original patch is along this road, but didn't zap IOMMU pages. :-))
Thanks
Kevin
Quoted 13 lines · Kevin
Quoted 9 lines
From: Tim Deegan [mailto:[email protected]]
Quoted 2 lines
Could we introduce a cpu-write-protection attribute only for functional usages which don't care about DMA path like in XenGT?
Possibly. Sorry for going over things that have probably already been covered but AIUI you are saying that XenGT needs to allow full DMA access to this memory area but only read-only CPU access. If that's what's needed then I think a new type with these semantics is the best way of doing it (and requiring no shared IOMMU/HAP tables). But:
It's about GPU page table write-protection, so XenGT needs to trap writes in both CPU/DMA paths:
Quoted 7 lines
* CPU writes are captured by read-only permission in EPT
* GPU writes are captured by scanning GPU commands before they're submitted for execution, as part of the 'mediation' concept in XenGT mediated pass-through architecture.Note XenGT doesn't use IOMMU because w/o SR-IOV a single device entry can't be used to server multiple VMs.
Oh, I see. So you don't actually want the GPUPTs mapped for DMA at all. That's much easier.
Quoted 7 lines
So XenGT can achieve true 'write-protection' w/o IOMMU restarting fault support. However it's very GPU specific. As a general implementation, I think a new type with below semantics should be enough:
- only CPU access to the protected area, so it's removed from IOMMU page table
- require IOMMU/HAP seperate
I think we might want to make those tables separate in general. IIRC we were discussing that in another thread already. But that aside, I think a plain read-only mapping will DTRT for you -- it allows read-only DMA from the GPUPTs, but that's no worse that the read-only CPU mapping.
After that, the only odd behaviour is that you're emulating writes to read-only memory. And that's where the new type is helpful: to say that we should be emulating writes rather than dropping them.
Quoted 3 lines
How would you call such a new type then? just
p2m_write_protectionorp2m_cpu_write_protection? (still need to differentiate write-protection from read-only, since the latter implies no writes)
p2m_mmio_write_dm?
Now that (I hope) I understand better what you need, I can see how just using plain mmio_dm and emulating reads would be a plausible solution too. So I think I'd be happy with either that or a new type.
Cheers,
Tim.
Quoted 32 lines
From: Tim Deegan [mailto:[email protected]]
Sent: Thursday, August 14, 2014 1:25 PMQuoted 13 lines · Kevin
Quoted 9 lines
From: Tim Deegan [mailto:[email protected]]
Quoted 2 lines
Could we introduce a cpu-write-protection attribute only for functional usages which don't care about DMA path like in XenGT?
Possibly. Sorry for going over things that have probably already been covered but AIUI you are saying that XenGT needs to allow full DMA access to this memory area but only read-only CPU access. If that's what's needed then I think a new type with these semantics is the best way of doing it (and requiring no shared IOMMU/HAP tables). But:
It's about GPU page table write-protection, so XenGT needs to trap writes in both CPU/DMA paths:
Quoted 7 lines
* CPU writes are captured by read-only permission in EPT
* GPU writes are captured by scanning GPU commands before they're submitted for execution, as part of the 'mediation' concept in XenGT mediated pass-through architecture.Note XenGT doesn't use IOMMU because w/o SR-IOV a single device entry can't be used to server multiple VMs.
Oh, I see. So you don't actually want the GPUPTs mapped for DMA at all. That's much easier.
Quoted 4 lines
So XenGT can achieve true 'write-protection' w/o IOMMU restarting fault support. However it's very GPU specific. As a general implementation, I think a new type with below semantics should be enough:
Quoted 8 lines
Quoted 3 lines
- only CPU access to the protected area, so it's removed from IOMMU page table
- require IOMMU/HAP seperateI think we might want to make those tables separate in general. IIRC we were discussing that in another thread already. But that aside, I think a plain read-only mapping will DTRT for you -- it allows read-only DMA from the GPUPTs, but that's no worse that the read-only CPU mapping.
for 'plain read-only mapping' you mean existing p2m_ram_ro?
btw just curious how we handle p2m_ram_ro today. Does it play any trick to fail-safe in IOMMU side, e.g. zap the mapping or not? Otherwise when it allows read-only DMA there's no way to prevent DMA writes either. :-)
Quoted 4 lines
After that, the only odd behaviour is that you're emulating writes to read-only memory. And that's where the new type is helpful: to say that we should be emulating writes rather than dropping them.
so p2m_mmio_write_dm would go together with existing p2m_ram_ro in most condition checks, with only difference on final policy of emulation vs. drop.
Quoted 11 lines
Quoted 4 lines
How would you call such a new type then? just
p2m_write_protectionorp2m_cpu_write_protection? (still need to differentiate write-protection from read-only, since the latter implies no writes)
p2m_mmio_write_dm?Now that (I hope) I understand better what you need, I can see how just using plain
mmio_dmand emulating reads would be a plausible solution too. So I think I'd be happy with either that or a new type.
yes either way works. for now let's try the new p2m type since it is more intuitive regarding to 'write-protection'. :-)
Thanks
Kevin
Quoted 4 lines
Quoted 1 line
Quoted 1 line · On 15.08.14 at 00:53, <[email protected]> wrote:
btw just curious how we handle
p2m_ram_rotoday. Does it play any trick to fail-safe in IOMMU side, e.g. zap the mapping or not? Otherwise when it allows read-only DMA there's no way to prevent DMA writes either. :-)
Why would that be? A page marked r/o in the IOMMU page tables ought to very well be protected against DMA writes. It's just that these will cause faults, which - if happening too frequently - will result in the device getting turned off, quite likely (implicitly) killing the guest.
Jan
Quoted 8 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Thursday, August 14, 2014 4:12 PMQuoted 4 lines
Quoted 1 line
Quoted 1 line · On 15.08.14 at 00:53, <[email protected]> wrote:
btw just curious how we handle
p2m_ram_rotoday. Does it play any trick to fail-safe in IOMMU side, e.g. zap the mapping or not? Otherwise when it allows read-only DMA there's no way to prevent DMA writes either. :-)
Quoted 6 lines
Why would that be? A page marked r/o in the IOMMU page tables ought to very well be protected against DMA writes. It's just that these will cause faults, which - if happening too frequently - will result in the device getting turned off, quite likely (implicitly) killing the guest.
yes, as protection it's enough, and it's only a problem for emulation. sorry for messed msg after a long discussion thread. :/
Thanks
Kevin
Quoted 126 lines
From: Tian, Kevin
Sent: Wednesday, August 06, 2014 9:09 AMQuoted 34 lines
From: Konrad Rzeszutek Wilk [mailto:[email protected]]
Sent: Wednesday, August 06, 2014 8:01 AMQuoted 30 lines · Yang Z
Tian, Kevin wrote on 2014-08-06:
Quoted 29 lines
Quoted 29 lines
From: Tian, Kevin
Sent: Tuesday, August 05, 2014 7:49 PMQuoted 26 lines
From: Zhang, Yang Z
Sent: Tuesday, August 05, 2014 7:40 PMTian, Kevin wrote on 2014-08-06:
Quoted 22 lines
Quoted 22 lines
From: Zhang, Yang Z
Sent: Tuesday, August 05, 2014 7:11 PMTian, Kevin wrote on 2014-08-05:
Quoted 18 lines
Quoted 18 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Tuesday, August 05, 2014 12:52 AMQuoted 15 lines
Quoted 15 lines
Quoted 15 lines · On 05.08.14 at 09:35, <[email protected]> wrote:
Tian, Kevin wrote on 2014-08-05: From: Jan Beulich
[mailto:[email protected]]
Sent: Monday, August 04, 2014 12:35 AMOn 04.08.14 at 07:05, <[email protected]> wrote:
Jan Beulich wrote on 2014-07-28:
[email protected]; [email protected]
Subject: Re: [PATCH v1 0/2] Extend ioreq-server to
support page write protectionOn 28.07.14 at 19:55, <[email protected]> wrote:
ioreq-server is proposed to forward PIO and MMIOrequest
Quoted 8 lines
to
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
multiple device models according to the io range. XenGT
(Intel Graphics Virtualization technology, please refer to
https://01.org/xen/blogs/srclarkx/2013/graphics-virtual iz
at io n- xengt) driver reside in Dom0 as a virtual graphics
device model also need to trap and emulate the guest'swrite
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
operation to some specific memory pages, like the
memory
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
pages used by guest graphics driver as PPGTT(per-process
graphics translation table). We add an new p2m type
"p2m_ram_wp" to trap the page write operation. Page ofthis
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
new p2m type is read only and for write, the request will
go
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
to device model via
ioreq-server.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
So how is this write-protection supposed to work on the
IOMMU side when sharing page tables?Thanks for pointing out this question. Write-protection is not
supposed to work when sharing page tables between EPTand
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
vt-d. An explicit command line "iommu=no-sharept" should
be
Quoted 54 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
setted for avoiding undesirable iommu fault.
Requiring the unconditional use of a specific command line
option is certainly fine for experimental code, but not beyondthat.
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 6 lines
Behavior should be correct by default in production code.
But what's worse here: The option _allows_ device side
writes from the guest. Why would device side writes be
okay, but CPU side ones not?right, whether ept is shared or not doesn't address the concern here. In both cases we need maintain the same p2m view
between
Quoted 39 lines
Quoted 39 lines
Quoted 39 lines
Quoted 38 lines
Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
Quoted 21 lines
Quoted 13 lines
Quoted 13 lines
CPU and device side, otherwise it's broken...
One option is to treat wp similar to logdirty, i.e. exclusive to VT-d device assignment, until in the future VT-d supports page fault. We can provide a boot option to override the default policy if user thinks OK.
2nd option, like Wei mentioned in another mail, is to treat such write protected PPGTT page tables as MMIO. new hypercall is required to change the p2m type between
p2m_ioandp2m_ram, based on allocation/free of guest page table. This way may impact performance on read though.Quoted 4 lines
Quoted 1 line
Comments?
Another solution is using the EPT misconfiguration mechanism like what Xen does for MTRR emulation currently.
That would cause faults on reads as well, making it necessary to emulate them.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Then it's same effect of
p2m_mmio_dm.
p2m_mmio_dmwill break VT-d but EPT misconfiguration doesn't.I don't understand. Treating it as emulated io just means no valid p2m entry in EPT/VT-d. Note XenGT itself doesn't require VT-d, while you can still assign other devices this way.
From guest's view, it doesn't know the page is not valid in EPT/VT-d table. So if guest CPU touch this page, then hypervisor will intercept the access from EPT violation and handle it. But if guest uses that page as DMA buffer (a malicious guest may do it), then VT-d fault happens. Since DMA is not restartable, guest will never receive the right data.
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line
as you said, it's malicious guest, that's why we need zap the VT-d
Malicious guest is just an example. We can assume a normal guest never
does it.
What!? No you can't. That is thundering towards an XSA! The normal guest can become compromised.
Quoted 7 lines
It would be a severe security hole to write-protect only the CPU path while leaving DMA writes valid. When we say a page is write-protected, it must be consistently enforced in both CPU/DMA paths. Now there is no other way to achieve it lacking of VT-d page fault, except treating it fully as a emulated MMIO range (means removing mapping in both EPT/VT-d page table), or excluding VT-d like this patch originally does.
Two corrections:
- it's still a functional problem instead of security problem. with EPT/VT-d a malicious VM anyway can only access its own memory.
- A general write-protection mechanism is not possible until VT-d page fault is there. The options just talk about XenGT specific requirement, which only cares about CPU modification to the GPU page table.
Thanks
Kevin
PATCH 1/2 x86: add p2m_ram_wp
In reply to Wei YeAdd a new p2m type p2m_ram_wp. Page of p2m_ram_wp is read only, and write will go to the device model for emulation.
--- xen/arch/x86/hvm/hvm.c | 8 +++++++- xen/arch/x86/mm/p2m-ept.c | 1 + xen/include/asm-x86/p2m.h | 8 +++++++- 3 files changed, 15 insertions(+), 2 deletions(-)
xen/arch/x86/hvm/hvm.c +7 -1
@@ -2738,7 +2738,8 @@ int hvm_hap_nested_page_fault(paddr_t gpa,* If this GFN is emulated MMIO or marked as read-only, pass the fault * to the mmio handler. */ -if ( (p2mt == p2m_mmio_dm) || +if ( (p2mt == p2m_mmio_dm) || + (p2mt == p2m_ram_wp) || (access_w && (p2mt == p2m_ram_ro)) ) {put_gfn (p2m->domain, gfn);
@@ -3829,6 +3830,11 @@ static enum hvm_copy_result __hvm_copy(put_page (page);return HVMCOPY_unhandleable; } +if (p2m_is_wp_ram (p2mt) ) + { +put_page (page); +return HVMCOPY_bad_gfn_to_mfn; + } p = (char *)__map_domain_page (page) + (addr & ~PAGE_MASK );
xen/arch/x86/mm/p2m-ept.c +1 -0
@@ -125,6 +125,7 @@ static void ept_p2m_type_to_flags(ept_entry_t *entry, p2m_type_t type, p2m_acces entry->mfn);break ;case p2m_ram_logdirty: +case p2m_ram_wp:case p2m_ram_ro:case p2m_ram_shared: entry->r = entry->x =1 ;
xen/include/asm-x86/p2m.h +7 -1
@@ -71,6 +71,7 @@ typedef enum { p2m_ram_shared =12 ,/* Shared or sharable memory */ p2m_ram_broken =13 ,/* Broken page, access cause domain crash */ p2m_map_foreign =14 ,/* ram pages from foreign domain */ + p2m_ram_wp =15 ,/* Read-only; write go to the device model */ } p2m_type_t;/*
@@ -116,7 +117,8 @@ typedef unsigned int p2m_query_t; | p2m_to_mask(p2m_ram_paging_out) \ |p2m_to_mask (p2m_ram_paged) \ |p2m_to_mask (p2m_ram_paging_in) \ - | p2m_to_mask(p2m_ram_shared)) + |p2m_to_mask (p2m_ram_shared) \ + | p2m_to_mask(p2m_ram_wp))/* Types that represent a physmap hole that is ok to replace with a shared * entry */
@@ -167,6 +169,9 @@ typedef unsigned int p2m_query_t;* and must not be touched. */ #define P2M_BROKEN_TYPES (p2m_to_mask(p2m_ram_broken)) +/* Write protection types */ +#define P2M_WP_TYPES (p2m_to_mask(p2m_ram_wp)) +/* Useful predicates */ #define p2m_is_ram (_t) (p2m_to_mask(_t) & P2M_RAM_TYPES)#define p2m_is_hole (_t) (p2m_to_mask(_t) & P2M_HOLE_TYPES)
@@ -191,6 +196,7 @@ typedef unsigned int p2m_query_t;#define p2m_is_any_ram (_t) (p2m_to_mask(_t) & \ (P2M_RAM_TYPES | P2M_GRANT_TYPES | \ p2m_to_mask(p2m_map_foreign))) +#define p2m_is_wp_ram (_t) (p2m_to_mask(_t) & P2M_WP_TYPES)/* Per-p2m-table state */ struct p2m_domain {
Signature · 1 line
1.7.9.5
Re: [PATCH v1 1/2] x86: add p2m_ram_wp
Quoted 5 lines
Quoted 1 line
Quoted 1 line · On 28.07.14 at 19:55, <[email protected]> wrote:
xen/arch/x86/hvm/hvm.c | 8 +++++++- xen/arch/x86/mm/p2m-ept.c | 1 + xen/include/asm-x86/p2m.h | 8 +++++++- 3 files changed, 15 insertions(+), 2 deletions(-)
This can't be complete: At the very least p2m-pt.c also needs a change similar to the one to p2m-ept.c.
xen/arch/x86/hvm/hvm.c int hvm_hap_nested_page_fault(paddr_t gpa, · line 2743
Quoted 10 lines
xen/arch/x86/hvm/hvm.c +2
-1@@ -2738,7 +2738,8 @@ int hvm_hap_nested_page_fault(paddr_t gpa,* If this GFN is emulated MMIO or marked as read-only, pass the fault * to the mmio handler. */ -if ( (p2mt == p2m_mmio_dm) || +if ( (p2mt == p2m_mmio_dm) || + (p2mt == p2m_ram_wp) || (access_w && (p2mt == p2m_ram_ro)) )
Comparing your change with the surrounding existing code, you would pass even reads happening to fault (perhaps for another reason) to the DM, other than done for p2m_ram_ro. I don't think that's correct.
xen/arch/x86/hvm/hvm.c static enum hvm_copy_result __hvm_copy( · line 3837
Quoted 9 lines
xen/arch/x86/hvm/hvm.c +5
-0@@ -3829,6 +3830,11 @@ static enum hvm_copy_result __hvm_copy(put_page (page);return HVMCOPY_unhandleable; } +if (p2m_is_wp_ram (p2mt) ) + { +put_page (page); +return HVMCOPY_bad_gfn_to_mfn; + }
And again this change can't be complete: __hvm_clear() would also need a similar change.
Quoted 11 lines
xen/arch/x86/hvm/hvm.c +3
-0@@ -167,6 +169,9 @@ typedef unsigned int p2m_query_t;* and must not be touched. */ #define P2M_BROKEN_TYPES (p2m_to_mask(p2m_ram_broken)) +/* Write protection types */ +#define P2M_WP_TYPES (p2m_to_mask(p2m_ram_wp)) +/* Useful predicates */ #define p2m_is_ram (_t) (p2m_to_mask(_t) & P2M_RAM_TYPES)#define p2m_is_hole (_t) (p2m_to_mask(_t) & P2M_HOLE_TYPES)
xen/arch/x86/hvm/hvm.c typedef unsigned int p2m_query_t; · line 199
Quoted 4 lines
xen/arch/x86/hvm/hvm.c +1
-0@@ -191,6 +196,7 @@ typedef unsigned int p2m_query_t;#define p2m_is_any_ram (_t) (p2m_to_mask(_t) & \ (P2M_RAM_TYPES | P2M_GRANT_TYPES | \ p2m_to_mask(p2m_map_foreign))) +#define p2m_is_wp_ram (_t) (p2m_to_mask(_t) & P2M_WP_TYPES)
To me such single-type classes don't seem very useful. Agreed there are a number of pre-existing ones, but for classes where future extensions are rather hard to imagine I wouldn't needlessly add classes right away. But Tim (whom you failed to Cc anyway) will have the final say here.
Jan
Re: [PATCH v1 1/2] x86: add p2m_ram_wp
Quoted 11 lines
-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Monday, July 28, 2014 4:32 PM
To: Ye, Wei
Cc: [email protected]; [email protected]; [email protected]; [email protected]; xen- [email protected]; [email protected]; Tim Deegan
Subject: Re: [PATCH v1 1/2] x86: addp2m_ram_wp
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line · On 28.07.14 at 19:55, <[email protected]> wrote:
xen/arch/x86/hvm/hvm.c | 8 +++++++- xen/arch/x86/mm/p2m-ept.c | 1 + xen/include/asm-x86/p2m.h | 8 +++++++- 3 files changed, 15 insertions(+), 2 deletions(-)This can't be complete: At the very least
p2m-pt.calso needs a change similar to the one top2m-ept.c.
Yes, got it.
Quoted 17 lines
xen/arch/x86/hvm/hvm.c int hvm_hap_nested_page_fault(paddr_t gpa, · line 2743
Quoted 11 lines
xen/arch/x86/hvm/hvm.c +2
-1@@ -2738,7 +2738,8 @@ int hvm_hap_nested_page_fault(paddr_t gpa,* If this GFN is emulated MMIO or marked as read-only, pass the fault * to the mmio handler. */ -if ( (p2mt == p2m_mmio_dm) || +if ( (p2mt == p2m_mmio_dm) || + (p2mt == p2m_ram_wp) || (access_w && (p2mt == p2m_ram_ro)) )Comparing your change with the surrounding existing code, you would pass even reads happening to fault (perhaps for another reason) to the DM, other than done for
p2m_ram_ro. I don't think that's correct.
Yes, should also access_w like the p2m_ram_ro.
Quoted 4 lines
Quoted 4 lines
xen/arch/x86/hvm/hvm.c +0
-0@@ -3829,6 +3830,11 @@ static enum hvm_copy_result __hvm_copy(put_page (page);return HVMCOPY_unhandleable; }
Quoted 9 lines
xen/arch/x86/hvm/hvm.c static enum hvm_copy_result __hvm_copy( · line 3837
Quoted 5 lines
xen/arch/x86/hvm/hvm.c +5
-0+if (p2m_is_wp_ram (p2mt) ) + { +put_page (page); +return HVMCOPY_bad_gfn_to_mfn; + }And again this change can't be complete:
__hvm_clear()would also need a similar change.
Got it and thanks.
Quoted 13 lines
Quoted 13 lines
xen/arch/x86/hvm/hvm.c +3
-0@@ -167,6 +169,9 @@ typedef unsigned int p2m_query_t;* and must not be touched. */ #define P2M_BROKEN_TYPES (p2m_to_mask(p2m_ram_broken)) +/* Write protection types */ +#define P2M_WP_TYPES (p2m_to_mask(p2m_ram_wp)) +/* Useful predicates */ #define p2m_is_ram (_t) (p2m_to_mask(_t) & P2M_RAM_TYPES)#define p2m_is_hole (_t) (p2m_to_mask(_t) & P2M_HOLE_TYPES)@@ -191,6 +196,7 @@ typedef unsigned int p2m_query_t;#define p2m_is_any_ram (_t) (p2m_to_mask(_t) & \ (P2M_RAM_TYPES | P2M_GRANT_TYPES | \
Quoted 9 lines
xen/arch/x86/hvm/hvm.c typedef unsigned int p2m_query_t; · line 199
Quoted 2 lines
xen/arch/x86/hvm/hvm.c +1
-0p2m_to_mask(p2m_map_foreign))) +#define p2m_is_wp_ram (_t) (p2m_to_mask(_t) & P2M_WP_TYPES)To me such single-type classes don't seem very useful. Agreed there are a number of pre-existing ones, but for classes where future extensions are rather hard to imagine I wouldn't needlessly add classes right away. But Tim (whom you failed to Cc anyway) will have the final say here.
How about using the existing p2m_mmio_dm, in that way, both read & write access will go to device model for emulation.
Quoted 1 line
Jan
Re: [PATCH v1 1/2] x86: add p2m_ram_wp
Quoted 25 lines
Quoted 16 lines
Quoted 1 line · On 04.08.14 at 07:10, <[email protected]> wrote:
From: Jan Beulich [mailto:[email protected]]
Quoted 14 lines
Quoted 1 line
Quoted 1 line · On 28.07.14 at 19:55, <[email protected]> wrote:
xen/include/asm-x86/p2m.h +3
-0@@ -167,6 +169,9 @@ typedef unsigned int p2m_query_t;* and must not be touched. */ #define P2M_BROKEN_TYPES (p2m_to_mask(p2m_ram_broken)) +/* Write protection types */ +#define P2M_WP_TYPES (p2m_to_mask(p2m_ram_wp)) +/* Useful predicates */ #define p2m_is_ram (_t) (p2m_to_mask(_t) & P2M_RAM_TYPES)#define p2m_is_hole (_t) (p2m_to_mask(_t) & P2M_HOLE_TYPES)@@ -191,6 +196,7 @@ typedef unsigned int p2m_query_t;#define p2m_is_any_ram (_t) (p2m_to_mask(_t) & \ (P2M_RAM_TYPES | P2M_GRANT_TYPES | \Quoted 8 lines
xen/include/asm-x86/p2m.h typedef unsigned int p2m_query_t; · line 199
Quoted 2 lines
xen/include/asm-x86/p2m.h +1
-0p2m_to_mask(p2m_map_foreign))) +#define p2m_is_wp_ram (_t) (p2m_to_mask(_t) & P2M_WP_TYPES)To me such single-type classes don't seem very useful. Agreed there are a number of pre-existing ones, but for classes where future extensions are rather hard to imagine I wouldn't needlessly add classes right away. But Tim (whom you failed to Cc anyway) will have the final say here.
Quoted 2 lines
How about using the existing
p2m_mmio_dm, in that way, both read & write access will go to device model for emulation.
Perhaps you didn't read what I wrote the way I intended it: I'm not opposed to the new P2M type. What I dislike is the single type class that you introduce. To check for that in the code, you could as well use direct comparisons with p2m_ram_wp.
Jan
Re: [PATCH v1 1/2] x86: add p2m_ram_wp
Quoted 38 lines
-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Monday, August 4, 2014 3:38 PM
To: Ye, Wei
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Tian, Kevin; Lv, Zhiyuan; [email protected]; [email protected]; Tim Deegan
Subject: RE: [PATCH v1 1/2] x86: addp2m_ram_wpQuoted 7 lines
Quoted 7 lines
Quoted 1 line · On 04.08.14 at 07:10, <[email protected]> wrote:
From: Jan Beulich [mailto:[email protected]]
Quoted 5 lines
Quoted 1 line
Quoted 1 line · On 28.07.14 at 19:55, <[email protected]> wrote:
xen/include/asm-x86/p2m.h +0
-0@@ -167,6 +169,9 @@ typedef unsigned int p2m_query_t;* and must not be touched. */ #define P2M_BROKEN_TYPES (p2m_to_mask(p2m_ram_broken))Quoted 5 lines
Quoted 5 lines
xen/include/asm-x86/p2m.h typedef unsigned int p2m_query_t; · line 176
Quoted 5 lines
xen/include/asm-x86/p2m.h +3
-0+/* Write protection types */ +#define P2M_WP_TYPES (p2m_to_mask(p2m_ram_wp)) +/* Useful predicates */ #define p2m_is_ram (_t) (p2m_to_mask(_t) & P2M_RAM_TYPES)#define
Quoted 1 line
Quoted 1 line
Quoted 1 line
p2m_is_hole(_t)(p2m_to_mask(_t)&P2M_HOLE_TYPES) @@ -191,6+196,7
Quoted 12 lines
Quoted 11 lines
Quoted 5 lines
@@ typedef unsigned int
p2m_query_t;#define p2m_is_any_ram(_t) (p2m_to_mask(_t) & \ (P2M_RAM_TYPES | P2M_GRANT_TYPES | \ p2m_to_mask(p2m_map_foreign)))+#define
p2m_is_wp_ram(_t)(p2m_to_mask(_t)&P2M_WP_TYPES)To me such single-type classes don't seem very useful. Agreed there are a number of pre-existing ones, but for classes where future extensions are rather hard to imagine I wouldn't needlessly add classes right away. But Tim (whom you failed to Cc anyway) will have the final say here.
Quoted 7 lines
Quoted 2 lines
How about using the existing
p2m_mmio_dm, in that way, both read & write access will go to device model for emulation.Perhaps you didn't read what I wrote the way I intended it: I'm not opposed to the new P2M type. What I dislike is the single type class that you introduce. To check for that in the code, you could as well use direct comparisons with
p2m_ram_wp.
Sorry for my misunderstanding. Agree that this single class may be unnecessary. So I'll remove this class type and compare it directly with p2m_ram_wp. But another option maybe we can remove this new introduced p2m type, since it doesn't work well when page table sharing with IOMMU case. And just using the existing p2m type "p2m_mmio_dm".
Quoted 2 lines
Jan
PATCH 2/2 ioreq-server: Support scatter page forwarding
In reply to Wei YeExtend the interface to support add/remove scatter page list to be forwarded by a dedicated ioreq-server instance. Check and select a right ioreq-server instance for forwarding the wirte operation for a write protected page.
--- tools/libxc/xc_domain.c | 32 ++++++ tools/libxc/xenctrl.h | 18 ++++ xen/arch/x86/hvm/hvm.c | 206 +++++++++++++++++++++++++++++++++++++- xen/include/asm-x86/hvm/domain.h | 14 +++ xen/include/public/hvm/hvm_op.h | 12 +++ 5 files changed, 281 insertions(+), 1 deletion(-)
tools/libxc/xc_domain.c +32 -0
@@ -1485,6 +1485,38 @@ int xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch, domid_t domid,return rc; } +int xc_hvm_map_pages_to_ioreq_server (xc_interface *xch,domid_t domid, +ioservid_t id,uint16_t set, +uint16_t nr_pages,unsigned long *gpfn) +{ +DECLARE_HYPERCALL ; +DECLARE_HYPERCALL_BUFFER (xen_hvm_map_pages_to_ioreq_server_t, arg); +int pg, rc =-1 ; + +if ( arg ==NULL ) +return -1 ; + +if (nr_pages >MAX_MAP_BATCH_PAGES ) + goto out; + + hypercall.op = __HYPERVISOR_hvm_op; + hypercall.arg[0 ] = HVMOP_map_pages_to_ioreq_server; + hypercall.arg[1 ] =HYPERCALL_BUFFER_AS_ARG (arg); + + arg->domid = domid; + arg->id = id; + arg->set = set; + arg->nr_pages = nr_pages; +for ( pg =0 ; pg < nr_pages; pg++ ) + arg->page_list[pg] = gpfn[pg]; + + rc =do_xen_hypercall (xch, &hypercall); + +out: +xc_hypercall_buffer_free (xch, arg); +return rc; +} +int xc_hvm_destroy_ioreq_server (xc_interface *xch,domid_t domid,ioservid_t id)
tools/libxc/xenctrl.h +18 -0
@@ -1943,6 +1943,24 @@ int xc_hvm_unmap_pcidev_from_ioreq_server(xc_interface *xch, uint8_t function);/** +* This function registers/deregisters a set of pages to be write protected. +* +* @parm xch a handle to an open hypervisor interface. +* @parm domid the domain id to be serviced +* @parm id the IOREQ Server id. +* @parm set whether registers or deregisters: 1 for register, 0 for deregister +* @parm nr_pages the number of pages +* @parm pgfn the guest page frame number list +* @return 0 on success, -1 on failure. +*/ +int xc_hvm_map_pages_to_ioreq_server (xc_interface *xch, +domid_t domid, +ioservid_t id, +uint16_t set, +uint16_t nr_pages, +unsigned long *gpfn); + +/** * This function destroys an IOREQ Server. * * @parm xch a handle to an open hypervisor interface.
xen/arch/x86/hvm/hvm.c +205 -1
255 lines not on this page · raw message
xen/include/asm-x86/hvm/domain.h +14 -0
@@ -48,6 +48,16 @@ struct hvm_ioreq_vcpu {evtchn_port_t ioreq_evtchn; }; +struct pglist_hash_table { +struct pglist_hash_table *next; +unsigned long gpfn; +}; +#define PGLIST_HASH_SIZE_SHIFT 8 +#define PGLIST_HASH_SIZE (1 << PGLIST_HASH_SIZE_SHIFT) +#define pglist_hash (x) ((x) % PGLIST_HASH_SIZE) +#define PGLIST_INVALID_GPFN 0 +#define PGLIST_HASH_ENTRY_SIZE sizeof(struct pglist_hash_table) +#define NR_IO_RANGE_TYPES (HVMOP_IO_RANGE_PCI + 1)#define MAX_NR_IO_RANGES 256
@@ -70,6 +80,10 @@ struct hvm_ioreq_server {evtchn_port_t bufioreq_evtchn;struct rangeset *range[NR_IO_RANGE_TYPES ];bool_t enabled; + +/* scatter page list need write protection */ +struct pglist_hash_table *pglist_hash_base; +spinlock_t pglist_hash_lock; };struct hvm_domain {
xen/include/public/hvm/hvm_op.h +12 -0
@@ -367,6 +367,18 @@ struct xen_hvm_set_ioreq_server_state {typedef struct xen_hvm_set_ioreq_server_state xen_hvm_set_ioreq_server_state_t ;DEFINE_XEN_GUEST_HANDLE (xen_hvm_set_ioreq_server_state_t); +#define HVMOP_map_pages_to_ioreq_server 23 +#define MAX_MAP_BATCH_PAGES 128 +struct xen_hvm_map_pages_to_ioreq_server { +domid_t domid;/* IN - domain to be serviced */ +ioservid_t id;/* IN - server id */ +uint16_t set;/* IN - 1: map pages, 0: unmap pages */ +uint16_t nr_pages;/* IN - page number */ +unsigned long page_list[MAX_MAP_BATCH_PAGES ];/* IN - gpfn list */ +}; +typedef struct xen_hvm_map_pages_to_ioreq_server xen_hvm_map_pages_to_ioreq_server_t ; +DEFINE_XEN_GUEST_HANDLE (xen_hvm_map_pages_to_ioreq_server_t); +#endif /* defined(__XEN__) || defined(__XEN_TOOLS__) */ #endif /* __XEN_PUBLIC_HVM_HVM_OP_H__ */
Signature · 1 line
1.7.9.5
Re: [PATCH v1 2/2] ioreq-server: Support scatter page forwarding
xen/arch/x86/hvm/hvm.c static void hvm_ioreq_server_remove_all_vcpus(struct hvm_ioreq_server *s) · line 750
Quoted 5 lines
Quoted 1 line
Quoted 1 line · On 28.07.14 at 19:55, <[email protected]> wrote:
xen/arch/x86/hvm/hvm.c +4
-0+static struct pglist_hash_table *hvm_ioreq_server_lookup_pglist_hash_table ( +struct pglist_hash_table *pglist_ht,unsigned long gpfn) +{ +int index =pglist_hash (gpfn);
Here and elsewhere: "unsigned int" for the hash index.
xen/arch/x86/hvm/hvm.c static void hvm_ioreq_server_remove_all_vcpus(struct hvm_ioreq_server *s) · line 765
Quoted 6 lines
xen/arch/x86/hvm/hvm.c +6
-0+static void hvm_ioreq_server_free_hash_chain (struct pglist_hash_table *chain) +{ +struct pglist_hash_table *p = chain; +struct pglist_hash_table *n; + +while (p)
Coding style.
Quoted 4 lines
xen/arch/x86/hvm/hvm.c +4
-0+static int hvm_ioreq_server_pglist_hash_add (struct pglist_hash_table *pglist_ht, +unsigned long gpfn) +{ +int index =pglist_hash (gpfn);
xen/arch/x86/hvm/hvm.c static void hvm_ioreq_server_remove_all_vcpus(struct hvm_ioreq_server *s) · line 795
Quoted 10 lines
xen/arch/x86/hvm/hvm.c +10
-0+struct pglist_hash_table *ne; + +if (hvm_ioreq_server_lookup_pglist_hash_table (pglist_ht, gpfn) !=NULL ) +return -EINVAL ; + +if ( pglist_ht[index].gpfn ==PGLIST_INVALID_GPFN ) + pglist_ht[index].gpfn = gpfn; +else + { + ne =xmalloc_bytes (sizeof (pglist_ht[0 ]));
xmalloc() please.
Quoted 8 lines
xen/arch/x86/hvm/hvm.c +8
-0+static int hvm_ioreq_server_pglist_hash_rem (struct pglist_hash_table *pglist_ht, +unsigned long gpfn) +{ +int index =pglist_hash (gpfn); +struct pglist_hash_table *next, *prev; + +if ( pglist_ht[index].gpfn == gpfn ) + pglist_ht[index].gpfn =PGLIST_INVALID_GPFN ;
xen/arch/x86/hvm/hvm.c static void hvm_ioreq_server_remove_all_vcpus(struct hvm_ioreq_server *s) · line 823
Quoted 10 lines
xen/arch/x86/hvm/hvm.c +10
-0+else + { + prev = &pglist_ht[index]; +while (1 ) + { + next = prev->next; +if ( !next ) + { +printk ("hvm_ioreq_server_pglist_hash_rem hash_table %p remove" +" %lx not found\n" , pglist_ht, gpfn);
The value of this log message is questionable anyway, but there's no way for this to be acceptable without a suitably low guest level being set.
xen/arch/x86/hvm/hvm.c static int hvm_ioreq_server_init(struct · line 1047
Quoted 9 lines
xen/arch/x86/hvm/hvm.c +3
-0@@ -948,7 +1040,14 @@ static int hvm_ioreq_server_init(struct hvm_ioreq_server *s, struct domain *d, spin_lock_init(&s->lock);INIT_LIST_HEAD (&s->ioreq_vcpu_list);spin_lock_init (&s->bufioreq_lock); +spin_lock_init (&s->pglist_hash_lock); + rc = -ENOMEM ; + s->pglist_hash_base =xmalloc_bytes (PGLIST_HASH_ENTRY_SIZE *PGLIST_HASH_SIZE );
xmalloc_array() please.
xen/arch/x86/hvm/hvm.c static int hvm_ioreq_server_init(struct hvm_ioreq_server *s, struct domain *d, · line 1049
Quoted 3 lines
xen/arch/x86/hvm/hvm.c +3
-0+if ( s->pglist_hash_base ==NULL ) + goto fail1; +memset (s->pglist_hash_base,0 ,PGLIST_HASH_ENTRY_SIZE *PGLIST_HASH_SIZE );
And together with this, xzalloc_array().
Quoted 22 lines
xen/arch/x86/hvm/hvm.c +22
-0+static int hvm_map_pages_to_ioreq_server (struct domain *d,ioservid_t id, +uint16_t set,uint16_t nr_pages, +unsigned long *gpfn) +{ +struct hvm_ioreq_server *s; +int rc; +unsigned int i; +p2m_type_t ot, nt; + +if ( set ) + { + ot = p2m_ram_rw; + nt = p2m_ram_wp; + } +else + { + ot = p2m_ram_wp; + nt = p2m_ram_rw; + } + +spin_lock (&d->arch.hvm_domain.ioreq_server.lock); +
xen/arch/x86/hvm/hvm.c static int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id, · line 1415
Quoted 10 lines
xen/arch/x86/hvm/hvm.c +10
-0+ rc = -ENOENT ; +list_for_each_entry ( s, + &d->arch.hvm_domain.ioreq_server.list, + list_entry ) + { +if ( s == d->arch.hvm_domain.default_ioreq_server ) +continue ; + +if ( s->id == id ) + {
Consistency please: Either both if()-s use "continue" (and perhaps get combined into one), or (less desirable imo) both get combined to use a {} body.
xen/arch/x86/hvm/hvm.c static int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id, · line 1420
Quoted 5 lines
xen/arch/x86/hvm/hvm.c +5
-0+spin_lock (&s->pglist_hash_lock); + +for ( i =0 ; i < nr_pages; i++ ) + { +p2m_change_type_one (d, gpfn[i], ot, nt);
Error handling?
xen/arch/x86/hvm/hvm.c static int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id, · line 1431
Quoted 11 lines
xen/arch/x86/hvm/hvm.c +11
-0+if ( set ) + rc =hvm_ioreq_server_pglist_hash_add (s->pglist_hash_base, gpfn[i]); +else + rc =hvm_ioreq_server_pglist_hash_rem (s->pglist_hash_base, gpfn[i]); + +if ( rc !=0 ) +break ; + } + +spin_unlock (&s->pglist_hash_lock); +flush_tlb_mask (d->domain_dirty_cpumask);
Why?
xen/arch/x86/hvm/hvm.c static struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d, · line 2457
Quoted 6 lines
xen/arch/x86/hvm/hvm.c +2
-0@@ -2294,6 +2453,8 @@ static struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,uint32_t cf8;uint8_t type;uint64_t addr; +unsigned long gpfn; +struct pglist_hash_table *he;
These ought to be moved down into a more narrow scope.
xen/arch/x86/hvm/hvm.c static struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d, · line 2530
Quoted 10 lines
xen/arch/x86/hvm/hvm.c +6
-0@@ -2361,6 +2522,15 @@ static struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d,if (rangeset_contains_range (r, addr, end) )return s; + gpfn = addr >>PAGE_SHIFT ; +spin_lock (&s->pglist_hash_lock); + he =hvm_ioreq_server_lookup_pglist_hash_table (s->pglist_hash_base, gpfn); +spin_unlock (&s->pglist_hash_lock); + +if ( he ) {
Coding style again.
xen/arch/x86/hvm/hvm.c static struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d, · line 2533
Quoted 3 lines
xen/arch/x86/hvm/hvm.c +3
-0+return s; + } +
And now the most fundamental question: Why is the existing (range set based) mechanism not enough? Is that because we limit the number of entries permitted on the range set? If so, the obvious question is - why is there no limit being enforced for your new type of pages?
Quoted 5 lines
xen/include/asm-x86/hvm/domain.h +0
-0@@ -48,6 +48,16 @@ struct hvm_ioreq_vcpu {evtchn_port_t ioreq_evtchn; };
xen/include/asm-x86/hvm/domain.h struct hvm_ioreq_vcpu { · line 59
Quoted 10 lines
xen/include/asm-x86/hvm/domain.h +9
-0+struct pglist_hash_table { +struct pglist_hash_table *next; +unsigned long gpfn; +}; +#define PGLIST_HASH_SIZE_SHIFT 8 +#define PGLIST_HASH_SIZE (1 << PGLIST_HASH_SIZE_SHIFT) +#define pglist_hash (x) ((x) % PGLIST_HASH_SIZE) +#define PGLIST_INVALID_GPFN 0 +#define PGLIST_HASH_ENTRY_SIZE sizeof(struct pglist_hash_table)
None of this has any point in being in a header: There's only a single source file using these, so they should be private to that one.
xen/include/public/hvm/hvm_op.h struct xen_hvm_set_ioreq_server_state { · line 371
Quoted 8 lines
xen/include/public/hvm/hvm_op.h +2
-0@@ -367,6 +367,18 @@ struct xen_hvm_set_ioreq_server_state {typedef struct xen_hvm_set_ioreq_server_state xen_hvm_set_ioreq_server_state_t ;DEFINE_XEN_GUEST_HANDLE (xen_hvm_set_ioreq_server_state_t); +#define HVMOP_map_pages_to_ioreq_server 23 +#define MAX_MAP_BATCH_PAGES 128
Too generic a name.
xen/include/public/hvm/hvm_op.h struct xen_hvm_set_ioreq_server_state { · line 376
Quoted 5 lines
xen/include/public/hvm/hvm_op.h +5
-0+struct xen_hvm_map_pages_to_ioreq_server { +domid_t domid;/* IN - domain to be serviced */ +ioservid_t id;/* IN - server id */ +uint16_t set;/* IN - 1: map pages, 0: unmap pages */ +uint16_t nr_pages;/* IN - page number */
"page count" or "number of pages".
xen/include/public/hvm/hvm_op.h struct xen_hvm_set_ioreq_server_state { · line 377
Quoted 1 line
xen/include/public/hvm/hvm_op.h +1
-0+unsigned long page_list[MAX_MAP_BATCH_PAGES ];/* IN - gpfn list */
No "unsigned long" in public interfaces. And no variable width types in interfaces not being subject to compat mode translation.
Jan
Re: [PATCH v1 2/2] ioreq-server: Support scatter page forwarding
Quoted 20 lines
-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: Monday, July 28, 2014 4:58 PM
To: Ye, Wei
Cc: [email protected]; [email protected]; [email protected]; [email protected]; xen- [email protected]; [email protected]
Subject: Re: [PATCH v1 2/2] ioreq-server: Support scatter page forwardingQuoted 2 lines
Quoted 1 line
Quoted 1 line · On 28.07.14 at 19:55, <[email protected]> wrote:
+static struct
pglist_hash_table*hvm_ioreq_server_lookup_pglist_hash_table(
Review
Quoted 3 lines
(quoted diff) +3
-0+ struct pglist_hash_table *pglist_ht, +unsigned long gpfn) { + int index = pglist_hash(gpfn);Here and elsewhere: "unsigned int" for the hash index.
Quoted 8 lines
xen/arch/x86/hvm/hvm.c static void hvm_ioreq_server_remove_all_vcpus(struct hvm_ioreq_server *s) · line 765
Quoted 6 lines
xen/arch/x86/hvm/hvm.c +6
-0+static void hvm_ioreq_server_free_hash_chain (struct pglist_hash_table +*chain) { +struct pglist_hash_table *p = chain; +struct pglist_hash_table *n; + +while (p)Coding style.
Will improve, thanks.
Quoted 17 lines
Quoted 2 lines
+static int hvm_ioreq_server_pglist_hash_add(struct
pglist_hash_table
*pglist_ht,xen/arch/x86/hvm/hvm.c static void hvm_ioreq_server_remove_all_vcpus(struct hvm_ioreq_server *s)
Quoted 5 lines
xen/arch/x86/hvm/hvm.c +5
-0+ unsigned long gpfn) { +int index =pglist_hash (gpfn); +struct pglist_hash_table *ne; + + if ( hvm_ioreq_server_lookup_pglist_hash_table(pglist_ht, gpfn) !=NULL )
xen/arch/x86/hvm/hvm.c static void hvm_ioreq_server_remove_all_vcpus(struct hvm_ioreq_server *s) · line 795
Quoted 7 lines
xen/arch/x86/hvm/hvm.c +7
-0+return -EINVAL ; + +if ( pglist_ht[index].gpfn ==PGLIST_INVALID_GPFN ) + pglist_ht[index].gpfn = gpfn; +else + { + ne =xmalloc_bytes (sizeof (pglist_ht[0 ]));
Quoted 1 line
xmalloc()please.
Ok.
Quoted 18 lines
Quoted 2 lines
+static int hvm_ioreq_server_pglist_hash_rem(struct
pglist_hash_table
*pglist_ht,Quoted 5 lines
xen/arch/x86/hvm/hvm.c +5
-0+ unsigned long gpfn) { +int index =pglist_hash (gpfn); +struct pglist_hash_table *next, *prev; + +if ( pglist_ht[index].gpfn == gpfn )xen/arch/x86/hvm/hvm.c static void hvm_ioreq_server_remove_all_vcpus(struct hvm_ioreq_server *s)
Quoted 10 lines
xen/arch/x86/hvm/hvm.c +10
-0+ pglist_ht[index].gpfn =PGLIST_INVALID_GPFN ; + else + { + prev = &pglist_ht[index]; +while (1 ) + { + next = prev->next; +if ( !next ) + { + printk("hvm_ioreq_server_pglist_hash_rem hash_table %p
Quoted 5 lines
remove"
Quoted 1 line
+ " %lx not found\n",
pglist_ht, gpfn);The value of this log message is questionable anyway, but there's no way for this to be acceptable without a suitably low guest level being set.
You mean should also print out the VM id in this log?
Quoted 14 lines
xen/arch/x86/hvm/hvm.c static int hvm_ioreq_server_init(struct · line 1048
Quoted 11 lines
xen/arch/x86/hvm/hvm.c +4
-0@@ -948,7 +1040,14 @@ static int hvm_ioreq_server_init(struct hvm_ioreq_server *s, struct domain *d, spin_lock_init(&s->lock);INIT_LIST_HEAD (&s->ioreq_vcpu_list);spin_lock_init (&s->bufioreq_lock); +spin_lock_init (&s->pglist_hash_lock); + rc = -ENOMEM ; + s->pglist_hash_base =xmalloc_bytes (PGLIST_HASH_ENTRY_SIZE * +PGLIST_HASH_SIZE );
xmalloc_array()please
Quoted 7 lines
xen/arch/x86/hvm/hvm.c static int hvm_ioreq_server_init(struct hvm_ioreq_server *s, struct domain *d,
Quoted 4 lines
xen/arch/x86/hvm/hvm.c +4
-0+if ( s->pglist_hash_base ==NULL ) + goto fail1; +memset (s->pglist_hash_base,0 ,PGLIST_HASH_ENTRY_SIZE * +PGLIST_HASH_SIZE );And together with this,
xzalloc_array().
Ok.
Quoted 28 lines
Review static int hvm_ioreq_server_init(struct · line 1053
Quoted 1 line
(quoted diff) +1
-0+static int hvm_map_pages_to_ioreq_server(struct domain *d, ioservid_tid,
Quoted 26 lines
xen/arch/x86/hvm/hvm.c +26
-0+ uint16_t set, uint16_t nr_pages, + unsigned long *gpfn) { + struct hvm_ioreq_server *s; +int rc; +unsigned int i; +p2m_type_t ot, nt; + +if ( set ) + { + ot = p2m_ram_rw; + nt = p2m_ram_wp; + } +else + { + ot = p2m_ram_wp; + nt = p2m_ram_rw; + } + +spin_lock (&d->arch.hvm_domain.ioreq_server.lock); + + rc = -ENOENT ; +list_for_each_entry ( s, + &d->arch.hvm_domain.ioreq_server.list, + list_entry ) + { +if ( s == d->arch.hvm_domain.default_ioreq_server )
Quoted 8 lines
xen/arch/x86/hvm/hvm.c static int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id, · line 1415
Quoted 4 lines
xen/arch/x86/hvm/hvm.c +4
-0+continue ; + +if ( s->id == id ) + {Consistency please: Either both if()-s use "continue" (and perhaps get combined into one), or (less desirable imo) both get combined to use a {} body.
Ok.
Quoted 8 lines
xen/arch/x86/hvm/hvm.c static int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id, · line 1420
Quoted 6 lines
xen/arch/x86/hvm/hvm.c +5
-0+spin_lock (&s->pglist_hash_lock); + +for ( i =0 ; i < nr_pages; i++ ) + { +p2m_change_type_one (d, gpfn[i], ot, nt);Error handling?
Forget error handling, thanks for your reminding.
Quoted 4 lines
Review
Quoted 3 lines
(quoted diff) +2
-0+ if ( set ) + rc = hvm_ioreq_server_pglist_hash_add(s->pglist_hash_base,gpfn[i]);
Quoted 12 lines
xen/arch/x86/hvm/hvm.c static int hvm_set_ioreq_server_state(struct domain *d, ioservid_t id, · line 1431
Quoted 10 lines
xen/arch/x86/hvm/hvm.c +10
-0+else + rc = +hvm_ioreq_server_pglist_hash_rem (s->pglist_hash_base, gpfn[i]); + +if ( rc !=0 ) +break ; + } + +spin_unlock (&s->pglist_hash_lock); +flush_tlb_mask (d->domain_dirty_cpumask);Why?
Will remove it.
Quoted 24 lines
Review static struct hvm_ioreq_server
Quoted 2 lines
(quoted diff) +0
-0@@ -2294,6 +2453,8 @@ static struct hvm_ioreq_server*hvm_select_ioreq_server(struct domain *d,
xen/arch/x86/hvm/hvm.c static struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d, · line 2457
Quoted 5 lines
xen/arch/x86/hvm/hvm.c +2
-0uint32_t cf8;uint8_t type;uint64_t addr; +unsigned long gpfn; +struct pglist_hash_table *he;These ought to be moved down into a more narrow scope.
Review static struct hvm_ioreq_server
Quoted 1 line
(quoted diff) +0
-0@@ -2361,6 +2522,15 @@ static struct hvm_ioreq_server*hvm_select_ioreq_server(struct domain *d,
xen/arch/x86/hvm/hvm.c static struct hvm_ioreq_server *hvm_select_ioreq_server(struct domain *d, · line 2530
Quoted 10 lines
xen/arch/x86/hvm/hvm.c +6
-0if (rangeset_contains_range (r, addr, end) )return s; + gpfn = addr >>PAGE_SHIFT ; +spin_lock (&s->pglist_hash_lock); + he =hvm_ioreq_server_lookup_pglist_hash_table (s- pglist_hash_base, gpfn); +spin_unlock (&s->pglist_hash_lock); + +if ( he ) {
Quoted 6 lines
Coding style again.
Quoted 3 lines
xen/arch/x86/hvm/hvm.c +3
-0+return s; + } +
Ok.
Quoted 5 lines
And now the most fundamental question: Why is the existing (range set based) mechanism not enough? Is that because we limit the number of entries permitted on the range set? If so, the obvious question is - why is there no limit being enforced for your new type of pages?
In our usage, we should write protect all the pages allocated by GFX driver using as Per-process graphics translation table, This kind of pages is much more than MAX_NR_IO_RANGES(256). So we need a scatter page list for containing all the pages.
Quoted 6 lines
Quoted 6 lines
xen/include/asm-x86/hvm/domain.h +0
-0@@ -48,6 +48,16 @@ struct hvm_ioreq_vcpu {evtchn_port_t ioreq_evtchn; };
Quoted 12 lines
xen/include/asm-x86/hvm/domain.h struct hvm_ioreq_vcpu { · line 59
Quoted 9 lines
xen/include/asm-x86/hvm/domain.h +9
-0+struct pglist_hash_table { +struct pglist_hash_table *next; +unsigned long gpfn; +}; +#define PGLIST_HASH_SIZE_SHIFT 8 +#define PGLIST_HASH_SIZE (1 << PGLIST_HASH_SIZE_SHIFT) +#define pglist_hash (x) ((x) % PGLIST_HASH_SIZE) +#define PGLIST_INVALID_GPFN 0 +#define PGLIST_HASH_ENTRY_SIZE sizeof(struct pglist_hash_table)None of this has any point in being in a header: There's only a single source file using these, so they should be private to that one.
Will move to hvm.c
Quoted 5 lines
xen/include/public/hvm/hvm_op.h struct xen_hvm_set_ioreq_server_state
Quoted 4 lines
xen/include/public/hvm/hvm_op.h +0
-0@@ -367,6 +367,18 @@ struct xen_hvm_set_ioreq_server_state{ typedef
Quoted 8 lines
xen/include/public/hvm/hvm_op.h struct xen_hvm_set_ioreq_server_state { · line 371
Quoted 6 lines
struct
xen_hvm_set_ioreq_server_statexen_hvm_set_ioreq_server_state_t;DEFINE_XEN_GUEST_HANDLE(xen_hvm_set_ioreq_server_state_t);xen/include/public/hvm/hvm_op.h +2
-0+#define HVMOP_map_pages_to_ioreq_server 23 +#define MAX_MAP_BATCH_PAGES 128Too generic a name.
How about MAX_IOREQSVR_MAP_BATCH_PAGES ?
Quoted 8 lines
xen/include/public/hvm/hvm_op.h struct xen_hvm_set_ioreq_server_state { · line 376
Quoted 6 lines
xen/include/public/hvm/hvm_op.h +5
-0+struct xen_hvm_map_pages_to_ioreq_server { +domid_t domid;/* IN - domain to be serviced */ +ioservid_t id;/* IN - server id */ +uint16_t set;/* IN - 1: map pages, 0: unmap pages */ +uint16_t nr_pages;/* IN - page number */ "page count" or "number of pages".
Should be page count.
Quoted 7 lines
Quoted 3 lines
+ unsigned long page_list[MAX_MAP_BATCH_PAGES]; /* IN - gpfn list
+ */No "unsigned long" in public interfaces. And no variable width types in interfaces not being subject to compat mode translation.
I'll change to use uint64_t.
Quoted 1 line
Jan
Re: [PATCH v1 2/2] ioreq-server: Support scatter page forwarding
Quoted 20 lines
Quoted 20 lines
Quoted 1 line · On 04.08.14 at 07:41, <[email protected]> wrote:
From: Jan Beulich [mailto:[email protected]]
Quoted 2 lines
Quoted 1 line
Quoted 1 line · On 28.07.14 at 19:55, <[email protected]> wrote:
+static int hvm_ioreq_server_pglist_hash_rem(struct
pglist_hash_table
*pglist_ht,Quoted 15 lines
xen/arch/x86/hvm/hvm.c +15
-0+ unsigned long gpfn) { +int index =pglist_hash (gpfn); +struct pglist_hash_table *next, *prev; + +if ( pglist_ht[index].gpfn == gpfn ) + pglist_ht[index].gpfn =PGLIST_INVALID_GPFN ; +else + { + prev = &pglist_ht[index]; +while (1 ) + { + next = prev->next; +if ( !next ) + { + printk("hvm_ioreq_server_pglist_hash_rem hash_table %p
Quoted 7 lines
Quoted 5 lines
remove"
Quoted 1 line
+ " %lx not found\n",
pglist_ht, gpfn);The value of this log message is questionable anyway, but there's no way for this to be acceptable without a suitably low guest level being set.
You mean should also print out the VM id in this log?
That's a secondary thing: Of course you need to ask yourself what use you could make of the message if you see it in some log, including when there was more than one VM running. But my point was that this lacks a suitable XENLOG_G_* prefix, thus representing a security issue (guest being able to flood the hypervisor log).
Quoted 10 lines
Quoted 5 lines
And now the most fundamental question: Why is the existing (range set based) mechanism not enough? Is that because we limit the number of entries permitted on the range set? If so, the obvious question is - why is there no limit being enforced for your new type of pages?
In our usage, we should write protect all the pages allocated by GFX driver using as Per-process graphics translation table,
This kind of pages is much more thanMAX_NR_IO_RANGES(256). So we need a scatter page list for containing all the pages.
So am I reading this right that you could do with the existing mechanism if the limit was higher? Together with you not answering the question about the lack of an enforced limit in your new mechanism, this clearly tells me that you should use the existing one, raising the limit and - if necessary - dealing with the (security) fallout.
Quoted 8 lines
Quoted 8 lines
xen/include/public/hvm/hvm_op.h struct xen_hvm_set_ioreq_server_state
Quoted 3 lines
xen/include/public/hvm/hvm_op.h +0
-0@@ -367,6 +367,18 @@ struct xen_hvm_set_ioreq_server_state{ typedef
Quoted 4 lines
struct
xen_hvm_set_ioreq_server_statexen_hvm_set_ioreq_server_state_t;DEFINE_XEN_GUEST_HANDLE(xen_hvm_set_ioreq_server_state_t);
Quoted 6 lines
Quoted 4 lines
xen/include/public/hvm/hvm_op.h struct xen_hvm_set_ioreq_server_state { · line 371
Quoted 2 lines
xen/include/public/hvm/hvm_op.h +2
-0+#define HVMOP_map_pages_to_ioreq_server 23 +#define MAX_MAP_BATCH_PAGES 128Too generic a name.
How about
MAX_IOREQSVR_MAP_BATCH_PAGES?
Butter, but still insufficient. XEN_IOREQSRV_MAX_MAP_BATCH_PAGES would go in the direction of being acceptable in a public header. But with the comments earlier on I suppose this will go away anyway.
Jan
Re: [PATCH v1 2/2] ioreq-server: Support scatter page forwarding
Quoted 49 lines
From: Jan Beulich [mailto:[email protected]]
Sent: Monday, August 04, 2014 12:47 AMQuoted 6 lines
Quoted 6 lines
Quoted 1 line · On 04.08.14 at 07:41, <[email protected]> wrote:
From: Jan Beulich [mailto:[email protected]]
Quoted 2 lines
Quoted 1 line
Quoted 1 line · On 28.07.14 at 19:55, <[email protected]> wrote:
+static int hvm_ioreq_server_pglist_hash_rem(struct
pglist_hash_table
*pglist_ht,Quoted 1 line
+ unsigned long
gpfn) {
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
xen/arch/x86/hvm/hvm.c +6
-0+int index =pglist_hash (gpfn); +struct pglist_hash_table *next, *prev; + +if ( pglist_ht[index].gpfn == gpfn ) + pglist_ht[index].gpfn =PGLIST_INVALID_GPFN ; +else Quoted 8 lines
Quoted 8 lines
xen/arch/x86/hvm/hvm.c static void hvm_ioreq_server_remove_all_vcpus(struct hvm_ioreq_server *s)
Quoted 8 lines
xen/arch/x86/hvm/hvm.c +8
-0+ { + prev = &pglist_ht[index]; +while (1 ) + { + next = prev->next; +if ( !next ) + { + printk("hvm_ioreq_server_pglist_hash_rem
hash_table%pQuoted 4 lines
Quoted 4 lines
remove"
Quoted 1 line
+ " %lx not found\n",
pglist_ht, gpfn);The value of this log message is questionable anyway, but there's no way
for
Quoted 3 lines
Quoted 1 line
this to be acceptable without a suitably low guest level being set.
You mean should also print out the VM id in this log?
That's a secondary thing: Of course you need to ask yourself what use you could make of the message if you see it in some log, including when there was more than one VM running. But my point was that this lacks a suitable XENLOG_G_* prefix, thus representing a security issue (guest being able to flood the hypervisor log).
Quoted 9 lines
Quoted 5 lines
And now the most fundamental question: Why is the existing (range set based) mechanism not enough? Is that because we limit the number of entries permitted on the range set? If so, the obvious question is - why is there no limit being enforced for your new type of pages?
In our usage, we should write protect all the pages allocated by GFX driver using as Per-process graphics translation table,
This kind of pages is much more thanMAX_NR_IO_RANGES(256). So we
Quoted 9 lines
need a
Quoted 1 line
scatter page list for containing all the pages.
So am I reading this right that you could do with the existing mechanism if the limit was higher? Together with you not answering the question about the lack of an enforced limit in your new mechanism, this clearly tells me that you should use the existing one, raising the limit and - if necessary - dealing with the (security) fallout.
essentially those pages are not contiguous and dynamic, and thus a fixed range based structure even with a higher limitation doesn't fit here. Just think about a structure similar to what we have done to maintain shadow pfn is required here. A hash list is more flexible to serve such purpose.
Thanks
Kevin
Re: [PATCH v1 2/2] ioreq-server: Support scatter page forwarding
Quoted 9 lines
Quoted 8 lines
Quoted 1 line · On 04.08.14 at 23:39, <[email protected]> wrote:
From: Jan Beulich [mailto:[email protected]]
So am I reading this right that you could do with the existing mechanism if the limit was higher? Together with you not answering the question about the lack of an enforced limit in your new mechanism, this clearly tells me that you should use the existing one, raising the limit and - if necessary - dealing with the (security) fallout.
Quoted 4 lines
essentially those pages are not contiguous and dynamic, and thus a fixed range based structure even with a higher limitation doesn't fit here. Just think about a structure similar to what we have done to maintain shadow pfn is required here. A hash list is more flexible to serve such purpose.
I don't follow - range sets aren't a "fixed range based structure" to me. All we impose is a limit on the number of ranges you may have in place to avoid unbounded amounts of memory being tied up by a domain, or unbounded amounts of time are spent in accessing that tree. Obviously the same would need to be done for any new mechanism, so a new mechanism is only first choice if it provides other benefits over the original one.
Jan