drm/i915: Move the release of PT page to the upper caller
Subsystems (2)
PATCH 1/1 drm/i915: Move the release of PT page to the upper caller
a PT page will be released if it doesn't contain any meaningful mappings during PPGTT page table shrinking. The PT entry in the upper level will be set to a scratch entry.
Normally this works nicely, but in virtualization world, the PPGTT page table is tracked by hypervisor. Releasing the PT page before modifying the upper level PT entry would cause extra efforts.
As the tracked page has been returned to OS before losing track from hypervisor, it could be written in any pattern. Hypervisor has to recognize if a page is still being used as a PT page by validating these writing patterns. It's complicated. Better let the guest modify the PT entry in upper level PT first, then release the PT page.
--- drivers/gpu/drm/i915/i915_gem_gtt.c | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-)
drivers/gpu/drm/i915/i915_gem_gtt.c +7 -11
@@ -736,10 +736,8 @@ static bool gen8_ppgtt_clear_pt(struct i915_address_space *vm,bitmap_clear (pt->used_ptes, pte, num_entries); -if (bitmap_empty (pt->used_ptes,GEN8_PTES )) { -free_pt (to_i915 (vm->dev), pt); +if (bitmap_empty (pt->used_ptes,GEN8_PTES ))return true; - } pt_vaddr =kmap_px (pt);
@@ -775,13 +773,12 @@ static bool gen8_ppgtt_clear_pd(struct i915_address_space *vm, pde_vaddr =kmap_px (pd); pde_vaddr[pde] = scratch_pde;kunmap_px (ppgtt, pde_vaddr); +free_pt (to_i915 (vm->dev), pt); } } -if (bitmap_empty (pd->used_pdes,I915_PDES )) { -free_pd (to_i915 (vm->dev), pd); +if (bitmap_empty (pd->used_pdes,I915_PDES ))return true; - }return false; }
@@ -795,7 +792,6 @@ static bool gen8_ppgtt_clear_pdp(struct i915_address_space *vm, uint64_t length) {struct i915_hw_ppgtt *ppgtt =i915_vm_to_ppgtt (vm); -struct drm_i915_private *dev_priv =to_i915 (vm->dev);struct i915_page_directory *pd;uint64_t pdpe;gen8_ppgtt_pdpe_t *pdpe_vaddr;
@@ -813,16 +809,14 @@ static bool gen8_ppgtt_clear_pdp(struct i915_address_space *vm, pdpe_vaddr[pdpe] = scratch_pdpe;kunmap_px (ppgtt, pdpe_vaddr); } +free_pd (to_i915 (vm->dev), pd); } }mark_tlbs_dirty (ppgtt); -if (USES_FULL_48BIT_PPGTT (dev_priv) && -bitmap_empty (pdp->used_pdpes,I915_PDPES_PER_PDP (dev_priv))) { -free_pdp (dev_priv, pdp); +if (bitmap_empty (pdp->used_pdpes,I915_PDPES_PER_PDP (dev_priv)))return true; - }return false; }
@@ -836,6 +830,7 @@ static void gen8_ppgtt_clear_pml4(struct i915_address_space *vm, uint64_t start, uint64_t length) { +struct drm_i915_private *dev_priv =to_i915 (vm->dev);struct i915_hw_ppgtt *ppgtt =i915_vm_to_ppgtt (vm);struct i915_page_directory_pointer *pdp;uint64_t pml4e;
@@ -854,6 +849,7 @@ static void gen8_ppgtt_clear_pml4(struct i915_address_space *vm, pml4e_vaddr =kmap_px (pml4); pml4e_vaddr[pml4e] = scratch_pml4e;kunmap_px (ppgtt, pml4e_vaddr); +free_pdp (dev_priv, pdp); } } }
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Patchwork
CI✓ Fi.CI.BAT: success for drm/i915: Move the release of PT page to the upper caller
== Series Details ==
== Series Details ==
Series: drm/i915: Move the release of PT page to the upper caller
URL : https://patchwork.freedesktop.org/series/15651/
State : success
== Summary ==
Series 15651v1 drm/i915: Move the release of PT page to the upper caller
https://patchwork.freedesktop.org/api/1.0/series/15651/revisions/1/mbox/
fi-bdw-5557u total:244 pass:229 dwarn:0 dfail:0 fail:0 skip:15 fi-bsw-n3050 total:244 pass:204 dwarn:0 dfail:0 fail:0 skip:40 fi-bxt-t5700 total:244 pass:216 dwarn:0 dfail:0 fail:0 skip:28 fi-byt-j1900 total:244 pass:216 dwarn:0 dfail:0 fail:0 skip:28 fi-byt-n2820 total:244 pass:212 dwarn:0 dfail:0 fail:0 skip:32 fi-hsw-4770 total:244 pass:224 dwarn:0 dfail:0 fail:0 skip:20 fi-hsw-4770r total:244 pass:224 dwarn:0 dfail:0 fail:0 skip:20 fi-ilk-650 total:244 pass:191 dwarn:0 dfail:0 fail:0 skip:53 fi-ivb-3520m total:244 pass:222 dwarn:0 dfail:0 fail:0 skip:22 fi-ivb-3770 total:244 pass:222 dwarn:0 dfail:0 fail:0 skip:22 fi-kbl-7200u total:244 pass:222 dwarn:0 dfail:0 fail:0 skip:22 fi-skl-6260u total:244 pass:230 dwarn:0 dfail:0 fail:0 skip:14 fi-skl-6700hq total:244 pass:223 dwarn:0 dfail:0 fail:0 skip:21 fi-skl-6700k total:244 pass:222 dwarn:1 dfail:0 fail:0 skip:21 fi-skl-6770hq total:244 pass:230 dwarn:0 dfail:0 fail:0 skip:14 fi-snb-2520m total:244 pass:212 dwarn:0 dfail:0 fail:0 skip:32 fi-snb-2600 total:244 pass:211 dwarn:0 dfail:0 fail:0 skip:33
d6149d212b69a8e1d9229fe80fca034a0abe1d0e drm-intel-nightly: 2016y-11m-21d-12h-48m-13s UTC integration manifest a7b4f2e drm/i915: Move the release of PT page to the upper caller
== Logs ==
For more details see: https://intel-gfx-ci.01.org/CI/Patchwork_3069/
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Hi guys:
Would you mind to have a quick review on this patch? :P The linux
guest under GVT-g couldn't boot up without this patch in the newer kernel.
Thanks,
Zhi.
Quoted 94 lines · Zhi Wang
a PT page will be released if it doesn't contain any meaningful mappings during PPGTT page table shrinking. The PT entry in the upper level will be set to a scratch entry.
Normally this works nicely, but in virtualization world, the PPGTT page table is tracked by hypervisor. Releasing the PT page before modifying the upper level PT entry would cause extra efforts.
As the tracked page has been returned to OS before losing track from hypervisor, it could be written in any pattern. Hypervisor has to recognize if a page is still being used as a PT page by validating these writing patterns. It's complicated. Better let the guest modify the PT entry in upper level PT first, then release the PT page.
Cc Michał Winiarski <[email protected]>Cc Michel Thierry <[email protected]>Cc Joonas Lahtinen <[email protected]>Cc Chris Wilson <[email protected]>Cc Zhenyu Wang <[email protected]>Cc Zhiyuan Lv <[email protected]>Signed-off-by Zhi Wang <[email protected]>--- drivers/gpu/drm/i915/i915_gem_gtt.c | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-)drivers/gpu/drm/i915/i915_gem_gtt.c +6
-11@@ -736,10 +736,8 @@ static bool gen8_ppgtt_clear_pt(struct i915_address_space *vm,bitmap_clear (pt->used_ptes, pte, num_entries); -if (bitmap_empty (pt->used_ptes,GEN8_PTES )) { -free_pt (to_i915 (vm->dev), pt); +if (bitmap_empty (pt->used_ptes,GEN8_PTES ))return true; - } pt_vaddr =kmap_px (pt);@@ -775,13 +773,12 @@ static bool gen8_ppgtt_clear_pd(struct i915_address_space *vm, pde_vaddr =kmap_px (pd); pde_vaddr[pde] = scratch_pde;kunmap_px (ppgtt, pde_vaddr); +free_pt (to_i915 (vm->dev), pt); } } -if (bitmap_empty (pd->used_pdes,I915_PDES )) { -free_pd (to_i915 (vm->dev), pd); +if (bitmap_empty (pd->used_pdes,I915_PDES ))return true; - }return false; }@@ -795,7 +792,6 @@ static bool gen8_ppgtt_clear_pdp(struct i915_address_space *vm, uint64_t length) {struct i915_hw_ppgtt *ppgtt =i915_vm_to_ppgtt (vm); -struct drm_i915_private *dev_priv =to_i915 (vm->dev);struct i915_page_directory *pd;uint64_t pdpe;gen8_ppgtt_pdpe_t *pdpe_vaddr;@@ -813,16 +809,14 @@ static bool gen8_ppgtt_clear_pdp(struct i915_address_space *vm, pdpe_vaddr[pdpe] = scratch_pdpe;kunmap_px (ppgtt, pdpe_vaddr); } +free_pd (to_i915 (vm->dev), pd); } }mark_tlbs_dirty (ppgtt); -if (USES_FULL_48BIT_PPGTT (dev_priv) && -bitmap_empty (pdp->used_pdpes,I915_PDPES_PER_PDP (dev_priv))) { -free_pdp (dev_priv, pdp); +if (bitmap_empty (pdp->used_pdpes,I915_PDPES_PER_PDP (dev_priv)))return true; - }return false; }@@ -836,6 +830,7 @@ static void gen8_ppgtt_clear_pml4(struct i915_address_space *vm, uint64_t start, uint64_t length) { +struct drm_i915_private *dev_priv =to_i915 (vm->dev);struct i915_hw_ppgtt *ppgtt =i915_vm_to_ppgtt (vm);struct i915_page_directory_pointer *pdp;uint64_t pml4e;
drivers/gpu/drm/i915/i915_gem_gtt.c static void gen8_ppgtt_clear_pml4(struct i915_address_space *vm, · line 855
Quoted 7 lines
drivers/gpu/drm/i915/i915_gem_gtt.c +1
-0@@ -854,6 +849,7 @@ static void gen8_ppgtt_clear_pml4(struct i915_address_space *vm, pml4e_vaddr =kmap_px (pml4); pml4e_vaddr[pml4e] = scratch_pml4e;kunmap_px (ppgtt, pml4e_vaddr); +free_pdp (dev_priv, pdp); } } }
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Chris Wilson
Reviewed-byQuoted 5 lines · Zhi Wang
Hi guys:
Would you mind to have a quick review on this patch? :P The
linux guest under GVT-g couldn't boot up without this patch in the newer kernel.
Quoted 25 lines
Thanks,
Zhi.Quoted 21 lines · Zhi Wang
a PT page will be released if it doesn't contain any meaningful mappings during PPGTT page table shrinking. The PT entry in the upper level will be set to a scratch entry.
Normally this works nicely, but in virtualization world, the PPGTT page table is tracked by hypervisor. Releasing the PT page before modifying the upper level PT entry would cause extra efforts.
As the tracked page has been returned to OS before losing track from hypervisor, it could be written in any pattern. Hypervisor has to recognize if a page is still being used as a PT page by validating these writing patterns. It's complicated. Better let the guest modify the PT entry in upper level PT first, then release the PT page.
Cc Michał Winiarski <[email protected]>Cc Michel Thierry <[email protected]>Cc Joonas Lahtinen <[email protected]>Cc Chris Wilson <[email protected]>Cc Zhenyu Wang <[email protected]>Cc Zhiyuan Lv <[email protected]>Signed-off-by Zhi Wang <[email protected]>
The original didn't make it to me, so I just assumed a list issue.
For the record, I'd like to have some more detail on just when the hv is doing the page scanning. It makes it sound like you are actively scanning an idle range.
-Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 32 lines · Chris Wilson
Quoted 5 lines · Zhi Wang
Hi guys:
Would you mind to have a quick review on this patch? :P The
linux guest under GVT-g couldn't boot up without this patch in the newer kernel.Quoted 25 lines
Thanks,
Zhi.Quoted 21 lines · Zhi Wang
a PT page will be released if it doesn't contain any meaningful mappings during PPGTT page table shrinking. The PT entry in the upper level will be set to a scratch entry.
Normally this works nicely, but in virtualization world, the PPGTT page table is tracked by hypervisor. Releasing the PT page before modifying the upper level PT entry would cause extra efforts.
As the tracked page has been returned to OS before losing track from hypervisor, it could be written in any pattern. Hypervisor has to recognize if a page is still being used as a PT page by validating these writing patterns. It's complicated. Better let the guest modify the PT entry in upper level PT first, then release the PT page.
Cc Michał Winiarski <[email protected]>Cc Michel Thierry <[email protected]>Cc Joonas Lahtinen <[email protected]>Cc Chris Wilson <[email protected]>Cc Zhenyu Wang <[email protected]>Cc Zhiyuan Lv <[email protected]>Signed-off-by Zhi Wang <[email protected]>
Quoted 7 lines
The original didn't make it to me, so I just assumed a list issue.
For the record, I'd like to have some more detail on just when the hv is doing the page scanning. It makes it sound like you are actively scanning an idle range.
Reviewed-by Chris Wilson <[email protected]>
Note that this should be queued for 4.9 fix otherwise 4.9 will be broken as guest kernel version for GVT-g.
Thanks
Signature · 3 lines
Open Source Technology Center, Intel ltd. $gpg --keyserver wwwkeys.pgp.net --recv-keys 4D781827
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Hi Chris, thanks for the reply! Without this patch, we have to do pattern scan to identify if the page is still being used as a PT page. :( It's complicated.
Originally, all the guest shadow PPGTT pages will be set to write-protected by HV. When guest makes a change in its page table, it will be trapped by HV.
For example:
Guest updates a PTE entry in a PTE PT page, then HV will be notified and populate the shadow PPGTT page table accordingly. Now let's say a PTE PT is empty (all mappings are pointing to scratch page) and will be freed in the newer kernel. If guest freed the PT page first, this page could be used by anyone while HV is still treating this page as PT page, then things goes wrong here. HV has to identify the page is not being used as PT page anymore at this time.
While if the guest updated the upper level PT entry first, HV will know the PTE PT page will not be used as a page table page, and stop track that page at this time. Then following accesses to that page will not be trapped by HV and surely HV will not see unrecognized PT entry writing (this page may be used by other guy at this time.)
Thanks,
Zhi.
Quoted 42 lines
-----Original Message-----
From: Chris Wilson [mailto:[email protected]]
Sent: Tuesday, November 22, 2016 10:38 PM
To: Wang, Zhi A
Cc: [email protected]; Winiarski, Michal; Thierry, Michel; Joonas Lahtinen; Zhenyu Wang; Lv, Zhiyuan
Subject: Re: [PATCH] drm/i915: Move the release of PT page to the upper callerQuoted 5 lines · Zhi Wang
Hi guys:
Would you mind to have a quick review on this patch? :P The linux
guest under GVT-g couldn't boot up without this patch in the newer kernel.Quoted 25 lines
Thanks,
Zhi.Quoted 21 lines · Zhi Wang
a PT page will be released if it doesn't contain any meaningful mappings during PPGTT page table shrinking. The PT entry in the upper level will be set to a scratch entry.
Normally this works nicely, but in virtualization world, the PPGTT page table is tracked by hypervisor. Releasing the PT page before modifying the upper level PT entry would cause extra efforts.
As the tracked page has been returned to OS before losing track from hypervisor, it could be written in any pattern. Hypervisor has to recognize if a page is still being used as a PT page by validating these writing patterns. It's complicated. Better let the guest modify the PT entry in upper level PT first, then release the PT page.
Cc Michał Winiarski <[email protected]>Cc Michel Thierry <[email protected]>Cc Joonas Lahtinen <[email protected]>Cc Chris Wilson <[email protected]>Cc Zhenyu Wang <[email protected]>Cc Zhiyuan Lv <[email protected]>Signed-off-by Zhi Wang <[email protected]>The original didn't make it to me, so I just assumed a list issue.
Quoted 8 lines
For the record, I'd like to have some more detail on just when the hv is doing the page scanning. It makes it sound like you are actively scanning an idle range.
Reviewed-by Chris Wilson <[email protected]> -Chris--
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
It's also the style how CPU page table is managed...
Quoted 71 lines
From: Wang, Zhi A
Sent: Wednesday, November 23, 2016 3:37 PMHi Chris, thanks for the reply! Without this patch, we have to do pattern scan to identify if the page is still being used as a PT page. :( It's complicated.
Originally, all the guest shadow PPGTT pages will be set to write-protected by HV. When guest makes a change in its page table, it will be trapped by HV.
For example:
Guest updates a PTE entry in a PTE PT page, then HV will be notified and populate the shadow PPGTT page table accordingly. Now let's say a PTE PT is empty (all mappings are pointing to scratch page) and will be freed in the newer kernel. If guest freed the PT page first, this page could be used by anyone while HV is still treating this page as PT page, then things goes wrong here. HV has to identify the page is not being used as PT page anymore at this time.While if the guest updated the upper level PT entry first, HV will know the PTE PT page will not be used as a page table page, and stop track that page at this time. Then following accesses to that page will not be trapped by HV and surely HV will not see unrecognized PT entry writing (this page may be used by other guy at this time.)
Thanks,
Zhi.Quoted 46 lines
-----Original Message-----
From: Chris Wilson [mailto:[email protected]]
Sent: Tuesday, November 22, 2016 10:38 PM
To: Wang, Zhi A
Cc: [email protected]; Winiarski, Michal; Thierry, Michel; Joonas Lahtinen; Zhenyu Wang; Lv, Zhiyuan
Subject: Re: [PATCH] drm/i915: Move the release of PT page to the upper callerQuoted 5 lines · Zhi Wang
Hi guys:
Would you mind to have a quick review on this patch? :P The linux
guest under GVT-g couldn't boot up without this patch in the newer kernel.Quoted 25 lines
Thanks,
Zhi.Quoted 21 lines · Zhi Wang
a PT page will be released if it doesn't contain any meaningful mappings during PPGTT page table shrinking. The PT entry in the upper level will be set to a scratch entry.
Normally this works nicely, but in virtualization world, the PPGTT page table is tracked by hypervisor. Releasing the PT page before modifying the upper level PT entry would cause extra efforts.
As the tracked page has been returned to OS before losing track from hypervisor, it could be written in any pattern. Hypervisor has to recognize if a page is still being used as a PT page by validating these writing patterns. It's complicated. Better let the guest modify the PT entry in upper level PT first, then release the PT page.
Cc Michał Winiarski <[email protected]>Cc Michel Thierry <[email protected]>Cc Joonas Lahtinen <[email protected]>Cc Chris Wilson <[email protected]>Cc Zhenyu Wang <[email protected]>Cc Zhiyuan Lv <[email protected]>Signed-off-by Zhi Wang <[email protected]>The original didn't make it to me, so I just assumed a list issue.
For the record, I'd like to have some more detail on just when the hv is doing the page scanning. It makes it sound like you are actively scanning an idle range.
Quoted 8 lines
Quoted 4 lines
Reviewed-by Chris Wilson <[email protected]> -Chris--
Chris Wilson, Intel Open Source Technology Centre_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 1 line · Kevin
It's also the style how CPU page table is managed...
But it is not the style of how the GPU page table behaves, which is what the code expects. My only concern here is if the fact that the page is not idle has further rammifactions.
-Chris
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 8 lines · Zhi Wang
a PT page will be released if it doesn't contain any meaningful mappings during PPGTT page table shrinking. The PT entry in the upper level will be set to a scratch entry.
Normally this works nicely, but in virtualization world, the PPGTT page table is tracked by hypervisor. Releasing the PT page before modifying the upper level PT entry would cause extra efforts.
Quoted 5 lines
As the tracked page has been returned to OS before losing track from hypervisor, it could be written in any pattern. Hypervisor has to recognize if a page is still being used as a PT page by validating these writing patterns. It's complicated. Better let the guest modify the PT entry in upper level PT first, then release the PT page.
-Michał
Quoted 10 lines
Cc Michał Winiarski <[email protected]>Cc Michel Thierry <[email protected]>Cc Joonas Lahtinen <[email protected]>Cc Chris Wilson <[email protected]>Cc Zhenyu Wang <[email protected]>Cc Zhiyuan Lv <[email protected]>Signed-off-by Zhi Wang <[email protected]>--- drivers/gpu/drm/i915/i915_gem_gtt.c | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-)
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Thanks Michał! Have a good day. :P
Quoted 15 lines · Michał Winiarski
Quoted 8 lines · Zhi Wang
a PT page will be released if it doesn't contain any meaningful mappings during PPGTT page table shrinking. The PT entry in the upper level will be set to a scratch entry.
Normally this works nicely, but in virtualization world, the PPGTT page table is tracked by hypervisor. Releasing the PT page before modifying the upper level PT entry would cause extra efforts.
Quoted 5 lines
As the tracked page has been returned to OS before losing track from hypervisor, it could be written in any pattern. Hypervisor has to recognize if a page is still being used as a PT page by validating these writing patterns. It's complicated. Better let the guest modify the PT entry in upper level PT first, then release the PT page.
Quoted 14 lines
Reviewed-by Michał Winiarski <[email protected]>-Michał
Quoted 10 lines
Cc Michał Winiarski <[email protected]>Cc Michel Thierry <[email protected]>Cc Joonas Lahtinen <[email protected]>Cc Chris Wilson <[email protected]>Cc Zhenyu Wang <[email protected]>Cc Zhiyuan Lv <[email protected]>Signed-off-by Zhi Wang <[email protected]>--- drivers/gpu/drm/i915/i915_gem_gtt.c | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-)
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx