drm/xen-front: Make shmem backed display buffer coherent
Subsystems (3)
PATCH v2 1/1 drm/xen-front: Make shmem backed display buffer coherent
From: Oleksandr Andrushchenko <[email protected]>
When GEM backing storage is allocated with drm_gem_get_pages the backing pages may be cached, thus making it possible that the backend sees only partial content of the buffer which may lead to screen artifacts. Make sure that the frontend's memory is coherent and the backend always sees correct display buffer content.
---
Changes since v1:
- Remove GFP_USER|__GFP_DMA32 mapping flags (Gerd)
- Use drm_prime_pages_to_sg directly (Noralf)
drivers/gpu/drm/xen/xen_drm_front_gem.c | 38 ++++++++++++++++++++++--- 1 file changed, 34 insertions(+), 4 deletions(-)
drivers/gpu/drm/xen/xen_drm_front_gem.c +34 -4
@@ -32,8 +32,11 @@ struct xen_gem_object {/* set for buffers allocated by the backend */ bool be_alloc; -/* this is for imported PRIME buffer */ -struct sg_table *sgt_imported; +/* +* this is for imported PRIME buffer or the one allocated via +* drm_gem_get_pages. +*/ +struct sg_table *sgt; }; static inline struct xen_gem_object *
@@ -124,8 +127,28 @@ static struct xen_gem_object *gem_create(struct drm_device *dev, size_t size) goto fail; } + xen_obj->sgt =drm_prime_pages_to_sg (xen_obj->pages, + xen_obj->num_pages); +if (IS_ERR_OR_NULL (xen_obj->sgt)) { + ret =PTR_ERR (xen_obj->sgt); + xen_obj->sgt =NULL ; + goto fail_put_pages; + } + +if (!dma_map_sg (dev->dev, xen_obj->sgt->sgl, xen_obj->sgt->nents, +DMA_BIDIRECTIONAL )) { + ret = -EFAULT ; + goto fail_free_sgt; + } +return xen_obj; +fail_free_sgt: +sg_free_table (xen_obj->sgt); + xen_obj->sgt =NULL ; +fail_put_pages: +drm_gem_put_pages (&xen_obj->base, xen_obj->pages, true, false); + xen_obj->pages =NULL ; fail:DRM_ERROR ("Failed to allocate buffer with size %zu\n" , size);return ERR_PTR (ret);
@@ -148,7 +171,7 @@ void xen_drm_front_gem_free_object_unlocked(struct drm_gem_object *gem_obj)struct xen_gem_object *xen_obj =to_xen_gem_obj (gem_obj);if (xen_obj->base.import_attach) { -drm_prime_gem_destroy (&xen_obj->base, xen_obj->sgt_imported); +drm_prime_gem_destroy (&xen_obj->base, xen_obj->sgt);gem_free_pages_array (xen_obj); }else {if (xen_obj->pages) {
@@ -157,6 +180,13 @@ void xen_drm_front_gem_free_object_unlocked(struct drm_gem_object *gem_obj) xen_obj->pages);gem_free_pages_array (xen_obj); }else { +if (xen_obj->sgt) { +dma_unmap_sg (xen_obj->base.dev->dev, + xen_obj->sgt->sgl, + xen_obj->sgt->nents, +DMA_BIDIRECTIONAL ); +sg_free_table (xen_obj->sgt); + }drm_gem_put_pages (&xen_obj->base, xen_obj->pages, true, false); }
@@ -202,7 +232,7 @@ xen_drm_front_gem_import_sg_table(struct drm_device *dev,if (ret <0 )return ERR_PTR (ret); - xen_obj->sgt_imported = sgt; + xen_obj->sgt = sgt; ret =drm_prime_sg_to_page_addr_arrays (sgt, xen_obj->pages,NULL , xen_obj->num_pages);
Signature · 1 line
2.20.1
Hi,
drivers/gpu/drm/xen/xen_drm_front_gem.c static struct xen_gem_object *gem_create(struct drm_device *dev, size_t size) · line 142
Quoted 5 lines
drivers/gpu/drm/xen/xen_drm_front_gem.c +5
-0+if (!dma_map_sg (dev->dev, xen_obj->sgt->sgl, xen_obj->sgt->nents, +DMA_BIDIRECTIONAL )) { + ret = -EFAULT ; + goto fail_free_sgt; + }
Hmm, so it seems the arm guys could not come up with a suggestion how to solve that one in a better way. Ok, lets go with this then.
But didn't we agree that this deserves a comment exmplaining the purpose of the dma_map_sg() call is to flush caches and that there is no actual DMA happening here?
cheers,
Gerd
Hi,
drivers/gpu/drm/xen/xen_drm_front_gem.c static struct xen_gem_object *gem_create(struct drm_device *dev, size_t size) · line 142
Quoted 5 lines
drivers/gpu/drm/xen/xen_drm_front_gem.c +5
-0+if (!dma_map_sg (dev->dev, xen_obj->sgt->sgl, xen_obj->sgt->nents, +DMA_BIDIRECTIONAL )) { + ret = -EFAULT ; + goto fail_free_sgt; + }
Hmm, so it seems the arm guys could not come up with a suggestion how to solve that one in a better way. Ok, lets go with this then.
But didn't we agree that this deserves a comment exmplaining the purpose of the dma_map_sg() call is to flush caches and that there is no actual DMA happening here?
cheers,
Gerd
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 11 lines · Gerd Hoffmann
Hi,
drivers/gpu/drm/xen/xen_drm_front_gem.c static struct xen_gem_object *gem_create(struct drm_device *dev, size_t size) · line 142
Quoted 5 lines
drivers/gpu/drm/xen/xen_drm_front_gem.c +5
-0+if (!dma_map_sg (dev->dev, xen_obj->sgt->sgl, xen_obj->sgt->nents, +DMA_BIDIRECTIONAL )) { + ret = -EFAULT ; + goto fail_free_sgt; + }Hmm, so it seems the arm guys could not come up with a suggestion how to solve that one in a better way. Ok, lets go with this then.
Quoted 3 lines
But didn't we agree that this deserves a comment exmplaining the purpose of the
dma_map_sg()call is to flush caches and that there is no actual DMA happening here?
Using a dma mapping call to flush caches is complete no-go. But the real question is why you'd even want to flush cashes if you do not want a dma mapping?
This whole issue keeps getting more and more confusing.
Quoted 11 lines · Gerd Hoffmann
Hi,
drivers/gpu/drm/xen/xen_drm_front_gem.c static struct xen_gem_object *gem_create(struct drm_device *dev, size_t size) · line 142
Quoted 5 lines
drivers/gpu/drm/xen/xen_drm_front_gem.c +5
-0+if (!dma_map_sg (dev->dev, xen_obj->sgt->sgl, xen_obj->sgt->nents, +DMA_BIDIRECTIONAL )) { + ret = -EFAULT ; + goto fail_free_sgt; + }Hmm, so it seems the arm guys could not come up with a suggestion how to solve that one in a better way. Ok, lets go with this then.
Quoted 3 lines
But didn't we agree that this deserves a comment exmplaining the purpose of the
dma_map_sg()call is to flush caches and that there is no actual DMA happening here?
Using a dma mapping call to flush caches is complete no-go. But the real question is why you'd even want to flush cashes if you do not want a dma mapping?
This whole issue keeps getting more and more confusing.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Oleksandr Andrushchenko
maintainerQuoted 10 lines · Gerd Hoffmann
Hi,drivers/gpu/drm/xen/xen_drm_front_gem.c static struct xen_gem_object *gem_create(struct drm_device *dev, size_t size) · line 142
Quoted 5 lines
drivers/gpu/drm/xen/xen_drm_front_gem.c +5
-0+if (!dma_map_sg (dev->dev, xen_obj->sgt->sgl, xen_obj->sgt->nents, +DMA_BIDIRECTIONAL )) { + ret = -EFAULT ; + goto fail_free_sgt; + }Hmm, so it seems the arm guys could not come up with a suggestion how to solve that one in a better way. Ok, lets go with this then.
Quoted 3 lines
But didn't we agree that this deserves a comment exmplaining the purpose of the
dma_map_sg()call is to flush caches and that there is no actual DMA happening here?
My bad, will add the comment
Quoted 3 lines
cheers,
Gerd
Thank you,
Oleksandr
Oleksandr Andrushchenko
maintainerQuoted 10 lines · Gerd Hoffmann
Hi,drivers/gpu/drm/xen/xen_drm_front_gem.c static struct xen_gem_object *gem_create(struct drm_device *dev, size_t size) · line 142
Quoted 5 lines
drivers/gpu/drm/xen/xen_drm_front_gem.c +5
-0+if (!dma_map_sg (dev->dev, xen_obj->sgt->sgl, xen_obj->sgt->nents, +DMA_BIDIRECTIONAL )) { + ret = -EFAULT ; + goto fail_free_sgt; + }Hmm, so it seems the arm guys could not come up with a suggestion how to solve that one in a better way. Ok, lets go with this then.
Quoted 3 lines
But didn't we agree that this deserves a comment exmplaining the purpose of the
dma_map_sg()call is to flush caches and that there is no actual DMA happening here?
My bad, will add the comment
Quoted 3 lines
cheers,
Gerd
Thank you,
Oleksandr
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Oleksandr Andrushchenko
maintainerQuoted 11 lines · Christoph Hellwig
Quoted 10 lines · Gerd Hoffmann
Hi,drivers/gpu/drm/xen/xen_drm_front_gem.c static struct xen_gem_object *gem_create(struct drm_device *dev, size_t size) · line 142
Quoted 5 lines
drivers/gpu/drm/xen/xen_drm_front_gem.c +5
-0+if (!dma_map_sg (dev->dev, xen_obj->sgt->sgl, xen_obj->sgt->nents, +DMA_BIDIRECTIONAL )) { + ret = -EFAULT ; + goto fail_free_sgt; + }Hmm, so it seems the arm guys could not come up with a suggestion how to solve that one in a better way. Ok, lets go with this then.
Quoted 8 lines
Quoted 3 lines
But didn't we agree that this deserves a comment exmplaining the purpose of the
dma_map_sg()call is to flush caches and that there is no actual DMA happening here?Using a dma mapping call to flush caches is complete no-go. But the real question is why you'd even want to flush cashes if you do not want a dma mapping?
This whole issue keeps getting more and more confusing.
Well, I don't really do DMA here, but instead the buffers in question are shared with other Xen domain, so effectively it could be thought of some sort of DMA here, where the "device" is that remote domain. If the buffers are not flushed then the remote part sees some inconsistency which in my case results in artifacts on screen while displaying the buffers. When buffers are allocated via DMA API then there are no artifacts; if buffers are allocated with shmem + DMA mapping then there are no artifacts as well.
The only offending use-case is when I use shmem backed buffers, but do not flush them
Oleksandr Andrushchenko
maintainerQuoted 11 lines · Christoph Hellwig
Quoted 10 lines · Gerd Hoffmann
Hi,drivers/gpu/drm/xen/xen_drm_front_gem.c static struct xen_gem_object *gem_create(struct drm_device *dev, size_t size) · line 142
Quoted 5 lines
drivers/gpu/drm/xen/xen_drm_front_gem.c +5
-0+if (!dma_map_sg (dev->dev, xen_obj->sgt->sgl, xen_obj->sgt->nents, +DMA_BIDIRECTIONAL )) { + ret = -EFAULT ; + goto fail_free_sgt; + }Hmm, so it seems the arm guys could not come up with a suggestion how to solve that one in a better way. Ok, lets go with this then.
Quoted 8 lines
Quoted 3 lines
But didn't we agree that this deserves a comment exmplaining the purpose of the
dma_map_sg()call is to flush caches and that there is no actual DMA happening here?Using a dma mapping call to flush caches is complete no-go. But the real question is why you'd even want to flush cashes if you do not want a dma mapping?
This whole issue keeps getting more and more confusing.
Well, I don't really do DMA here, but instead the buffers in question are shared with other Xen domain, so effectively it could be thought of some sort of DMA here, where the "device" is that remote domain. If the buffers are not flushed then the remote part sees some inconsistency which in my case results in artifacts on screen while displaying the buffers. When buffers are allocated via DMA API then there are no artifacts; if buffers are allocated with shmem + DMA mapping then there are no artifacts as well.
The only offending use-case is when I use shmem backed buffers, but do not flush them
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 12 lines · Oleksandr Andrushchenko
Quoted 1 line
This whole issue keeps getting more and more confusing.
Well, I don't really do DMA here, but instead the buffers in question are shared with other Xen domain, so effectively it could be thought of some sort of DMA here, where the "device" is that remote domain. If the buffers are not flushed then the remote part sees some inconsistency which in my case results in artifacts on screen while displaying the buffers. When buffers are allocated via DMA API then there are no artifacts; if buffers are allocated with shmem + DMA mapping then there are no artifacts as well.
The only offending use-case is when I use shmem backed buffers, but do not flush them
The right answer would be to implement cache maintainance hooks for this case in the Xen arch code. These would basically look the same as the low-level cache maintainance used by the DMA ops, but without going through the DMA mapping layer, in fact they should probably reuse the same low-level assembly routines.
I don't think this is the first usage of such Xen buffer sharing, so what do the other users do?
Quoted 12 lines · Oleksandr Andrushchenko
Quoted 1 line
This whole issue keeps getting more and more confusing.
Well, I don't really do DMA here, but instead the buffers in question are shared with other Xen domain, so effectively it could be thought of some sort of DMA here, where the "device" is that remote domain. If the buffers are not flushed then the remote part sees some inconsistency which in my case results in artifacts on screen while displaying the buffers. When buffers are allocated via DMA API then there are no artifacts; if buffers are allocated with shmem + DMA mapping then there are no artifacts as well.
The only offending use-case is when I use shmem backed buffers, but do not flush them
The right answer would be to implement cache maintainance hooks for this case in the Xen arch code. These would basically look the same as the low-level cache maintainance used by the DMA ops, but without going through the DMA mapping layer, in fact they should probably reuse the same low-level assembly routines.
I don't think this is the first usage of such Xen buffer sharing, so what do the other users do?
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Oleksandr Andrushchenko
maintainerQuoted 13 lines · Christoph Hellwig
Quoted 12 lines · Oleksandr Andrushchenko
Quoted 1 line
This whole issue keeps getting more and more confusing.
Well, I don't really do DMA here, but instead the buffers in question are shared with other Xen domain, so effectively it could be thought of some sort of DMA here, where the "device" is that remote domain. If the buffers are not flushed then the remote part sees some inconsistency which in my case results in artifacts on screen while displaying the buffers. When buffers are allocated via DMA API then there are no artifacts; if buffers are allocated with shmem + DMA mapping then there are no artifacts as well.
The only offending use-case is when I use shmem backed buffers, but do not flush them
Quoted 8 lines
The right answer would be to implement cache maintainance hooks for this case in the Xen arch code. These would basically look the same as the low-level cache maintainance used by the DMA ops, but without going through the DMA mapping layer, in fact they should probably reuse the same low-level assembly routines.
I don't think this is the first usage of such Xen buffer sharing, so what do the other users do?
I'll have to get even deeper into it. Initially I looked at the code, but didn't find anything useful. Or maybe I have just overlooked obvious things there
Thank you,
Oleksandr
Oleksandr Andrushchenko
maintainerQuoted 13 lines · Christoph Hellwig
Quoted 12 lines · Oleksandr Andrushchenko
Quoted 1 line
This whole issue keeps getting more and more confusing.
Well, I don't really do DMA here, but instead the buffers in question are shared with other Xen domain, so effectively it could be thought of some sort of DMA here, where the "device" is that remote domain. If the buffers are not flushed then the remote part sees some inconsistency which in my case results in artifacts on screen while displaying the buffers. When buffers are allocated via DMA API then there are no artifacts; if buffers are allocated with shmem + DMA mapping then there are no artifacts as well.
The only offending use-case is when I use shmem backed buffers, but do not flush them
Quoted 8 lines
The right answer would be to implement cache maintainance hooks for this case in the Xen arch code. These would basically look the same as the low-level cache maintainance used by the DMA ops, but without going through the DMA mapping layer, in fact they should probably reuse the same low-level assembly routines.
I don't think this is the first usage of such Xen buffer sharing, so what do the other users do?
I'll have to get even deeper into it. Initially I looked at the code, but didn't find anything useful. Or maybe I have just overlooked obvious things there
Thank you,
Oleksandr
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
(+ Stefano)
Hi,
Sorry for jumping late in the conversation.
Quoted 20 lines · Oleksandr Andrushchenko
Quoted 19 lines · Christoph Hellwig
Quoted 12 lines · Oleksandr Andrushchenko
Quoted 1 line
This whole issue keeps getting more and more confusing.
Well, I don't really do DMA here, but instead the buffers in question are shared with other Xen domain, so effectively it could be thought of some sort of DMA here, where the "device" is that remote domain. If the buffers are not flushed then the remote part sees some inconsistency which in my case results in artifacts on screen while displaying the buffers. When buffers are allocated via DMA API then there are no artifacts; if buffers are allocated with shmem + DMA mapping then there are no artifacts as well.
The only offending use-case is when I use shmem backed buffers, but do not flush themThe right answer would be to implement cache maintainance hooks for this case in the Xen arch code. These would basically look the same as the low-level cache maintainance used by the DMA ops, but without going through the DMA mapping layer, in fact they should probably reuse the same low-level assembly routines.
Quoted 5 lines
Quoted 2 lines
I don't think this is the first usage of such Xen buffer sharing, so what do the other users do?
I'll have to get even deeper into it. Initially I looked at the code, but didn't find anything useful. Or maybe I have just overlooked obvious things there
From Xen on Arm ABI:
"All memory which is shared with other entities in the system (including the hypervisor and other guests) must reside in memory which is mapped as Normal Inner Write-Back Outer Write-Back Inner-Shareable. This applies to:
- hypercall arguments passed via a pointer to guest memory.
- memory shared via the grant table mechanism (including PV I/O rings etc).
- memory shared with the hypervisor (struct shared_info, struct vcpu_info, the grant table, etc).
"
So you should not need any cache maintenance here. Can you provide more details on the memory attribute you use for memory shared in both the backend and frontend?
Cheers,
Quoted 8 lines
Thank you,
Oleksandr
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Signature · 1 line
Julien Grall
(+ Stefano)
Hi,
Sorry for jumping late in the conversation.
Quoted 20 lines · Oleksandr Andrushchenko
Quoted 19 lines · Christoph Hellwig
Quoted 12 lines · Oleksandr Andrushchenko
Quoted 1 line
This whole issue keeps getting more and more confusing.
Well, I don't really do DMA here, but instead the buffers in question are shared with other Xen domain, so effectively it could be thought of some sort of DMA here, where the "device" is that remote domain. If the buffers are not flushed then the remote part sees some inconsistency which in my case results in artifacts on screen while displaying the buffers. When buffers are allocated via DMA API then there are no artifacts; if buffers are allocated with shmem + DMA mapping then there are no artifacts as well.
The only offending use-case is when I use shmem backed buffers, but do not flush themThe right answer would be to implement cache maintainance hooks for this case in the Xen arch code. These would basically look the same as the low-level cache maintainance used by the DMA ops, but without going through the DMA mapping layer, in fact they should probably reuse the same low-level assembly routines.
Quoted 5 lines
Quoted 2 lines
I don't think this is the first usage of such Xen buffer sharing, so what do the other users do?
I'll have to get even deeper into it. Initially I looked at the code, but didn't find anything useful. Or maybe I have just overlooked obvious things there
From Xen on Arm ABI:
"All memory which is shared with other entities in the system (including the hypervisor and other guests) must reside in memory which is mapped as Normal Inner Write-Back Outer Write-Back Inner-Shareable. This applies to:
- hypercall arguments passed via a pointer to guest memory.
- memory shared via the grant table mechanism (including PV I/O rings etc).
- memory shared with the hypervisor (struct shared_info, struct vcpu_info, the grant table, etc).
"
So you should not need any cache maintenance here. Can you provide more details on the memory attribute you use for memory shared in both the backend and frontend?
Cheers,
Quoted 8 lines
Thank you,
Oleksandr
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Signature · 6 lines
Julien Grall _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Oleksandr Andrushchenko
maintainerQuoted 47 lines · Julien Grall
(+ Stefano)
Hi,
Sorry for jumping late in the conversation.
Quoted 21 lines · Oleksandr Andrushchenko
Quoted 20 lines · Christoph Hellwig
Quoted 12 lines · Oleksandr Andrushchenko
Quoted 1 line
This whole issue keeps getting more and more confusing.
Well, I don't really do DMA here, but instead the buffers in question are shared with other Xen domain, so effectively it could be thought of some sort of DMA here, where the "device" is that remote domain. If the buffers are not flushed then the remote part sees some inconsistency which in my case results in artifacts on screen while displaying the buffers. When buffers are allocated via DMA API then there are no artifacts; if buffers are allocated with shmem + DMA mapping then there are no artifacts as well.
The only offending use-case is when I use shmem backed buffers, but do not flush themThe right answer would be to implement cache maintainance hooks for this case in the Xen arch code. These would basically look the same as the low-level cache maintainance used by the DMA ops, but without going through the DMA mapping layer, in fact they should probably reuse the same low-level assembly routines.
Quoted 5 lines
Quoted 2 lines
I don't think this is the first usage of such Xen buffer sharing, so what do the other users do?
I'll have to get even deeper into it. Initially I looked at the code, but didn't find anything useful. Or maybe I have just overlooked obvious things there
From Xen on Arm ABI:
"All memory which is shared with other entities in the system (including the hypervisor and other guests) must reside in memory which is mapped as Normal Inner Write-Back Outer Write-Back Inner-Shareable.
This applies to:
- hypercall arguments passed via a pointer to guest memory.
- memory shared via the grant table mechanism (including PV I/O rings etc).
- memory shared with the hypervisor (structshared_info, structvcpu_info, the grant table, etc).
"
Quoted 4 lines
So you should not need any cache maintenance here. Can you provide more details on the memory attribute you use for memory shared in both the backend and frontend?
It takes quite some time to collect this (because many components are involved in the
use-case), but for now the pages in the guest domain are:
!PTE_RDONLY + PTE_PXN + PTE_SHARED + PTE_AF + PTE_UXN + PTE_ATTRINDX(MT_NORMAL)
Quoted 11 lines
Cheers,
Quoted 8 lines
Thank you,
Oleksandr
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Oleksandr Andrushchenko
maintainerQuoted 47 lines · Julien Grall
(+ Stefano)
Hi,
Sorry for jumping late in the conversation.
Quoted 21 lines · Oleksandr Andrushchenko
Quoted 20 lines · Christoph Hellwig
Quoted 12 lines · Oleksandr Andrushchenko
Quoted 1 line
This whole issue keeps getting more and more confusing.
Well, I don't really do DMA here, but instead the buffers in question are shared with other Xen domain, so effectively it could be thought of some sort of DMA here, where the "device" is that remote domain. If the buffers are not flushed then the remote part sees some inconsistency which in my case results in artifacts on screen while displaying the buffers. When buffers are allocated via DMA API then there are no artifacts; if buffers are allocated with shmem + DMA mapping then there are no artifacts as well.
The only offending use-case is when I use shmem backed buffers, but do not flush themThe right answer would be to implement cache maintainance hooks for this case in the Xen arch code. These would basically look the same as the low-level cache maintainance used by the DMA ops, but without going through the DMA mapping layer, in fact they should probably reuse the same low-level assembly routines.
Quoted 5 lines
Quoted 2 lines
I don't think this is the first usage of such Xen buffer sharing, so what do the other users do?
I'll have to get even deeper into it. Initially I looked at the code, but didn't find anything useful. Or maybe I have just overlooked obvious things there
From Xen on Arm ABI:
"All memory which is shared with other entities in the system (including the hypervisor and other guests) must reside in memory which is mapped as Normal Inner Write-Back Outer Write-Back Inner-Shareable.
This applies to:
- hypercall arguments passed via a pointer to guest memory.
- memory shared via the grant table mechanism (including PV I/O rings etc).
- memory shared with the hypervisor (structshared_info, structvcpu_info, the grant table, etc).
"
Quoted 4 lines
So you should not need any cache maintenance here. Can you provide more details on the memory attribute you use for memory shared in both the backend and frontend?
It takes quite some time to collect this (because many components are involved in the
use-case), but for now the pages in the guest domain are:
!PTE_RDONLY + PTE_PXN + PTE_SHARED + PTE_AF + PTE_UXN + PTE_ATTRINDX(MT_NORMAL)
Quoted 11 lines
Cheers,
Quoted 8 lines
Thank you,
Oleksandr
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Hello,
Quoted 46 lines · Oleksandr Andrushchenko
Quoted 41 lines · Julien Grall
Quoted 21 lines · Oleksandr Andrushchenko
Quoted 20 lines · Christoph Hellwig
Quoted 12 lines · Oleksandr Andrushchenko
Quoted 1 line
This whole issue keeps getting more and more confusing.
Well, I don't really do DMA here, but instead the buffers in question are shared with other Xen domain, so effectively it could be thought of some sort of DMA here, where the "device" is that remote domain. If the buffers are not flushed then the remote part sees some inconsistency which in my case results in artifacts on screen while displaying the buffers. When buffers are allocated via DMA API then there are no artifacts; if buffers are allocated with shmem + DMA mapping then there are no artifacts as well.
The only offending use-case is when I use shmem backed buffers, but do not flush themThe right answer would be to implement cache maintainance hooks for this case in the Xen arch code. These would basically look the same as the low-level cache maintainance used by the DMA ops, but without going through the DMA mapping layer, in fact they should probably reuse the same low-level assembly routines.
Quoted 5 lines
Quoted 2 lines
I don't think this is the first usage of such Xen buffer sharing, so what do the other users do?
I'll have to get even deeper into it. Initially I looked at the code, but didn't find anything useful. Or maybe I have just overlooked obvious things there
From Xen on Arm ABI:
"All memory which is shared with other entities in the system (including the hypervisor and other guests) must reside in memory which is mapped as Normal Inner Write-Back Outer Write-Back Inner-Shareable.
This applies to:
- hypercall arguments passed via a pointer to guest memory.
- memory shared via the grant table mechanism (including PV I/O rings etc).
- memory shared with the hypervisor (structshared_info, structvcpu_info, the grant table, etc).
"Quoted 4 lines
So you should not need any cache maintenance here. Can you provide more details on the memory attribute you use for memory shared in both the backend and frontend?
Quoted 5 lines
It takes quite some time to collect this (because many components are involved in the
use-case), but for now the pages in the guest domain are:
!PTE_RDONLY +PTE_PXN+PTE_SHARED+PTE_AF+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)
So that's the attribute for the page mapped in the frontend, right? How about the backend side?
Also, could that page be handed to the graphic card correctly? If so, is your graphic card coherent?
If one of your components is mapping with non-cacheable attributes then you are already not following the Xen Arm ABI. In that case, we would need to discuss how to extend the ABI.
Cheers,
Signature · 1 line
Julien Grall
Hello,
Quoted 46 lines · Oleksandr Andrushchenko
Quoted 41 lines · Julien Grall
Quoted 21 lines · Oleksandr Andrushchenko
Quoted 20 lines · Christoph Hellwig
Quoted 12 lines · Oleksandr Andrushchenko
Quoted 1 line
This whole issue keeps getting more and more confusing.
Well, I don't really do DMA here, but instead the buffers in question are shared with other Xen domain, so effectively it could be thought of some sort of DMA here, where the "device" is that remote domain. If the buffers are not flushed then the remote part sees some inconsistency which in my case results in artifacts on screen while displaying the buffers. When buffers are allocated via DMA API then there are no artifacts; if buffers are allocated with shmem + DMA mapping then there are no artifacts as well.
The only offending use-case is when I use shmem backed buffers, but do not flush themThe right answer would be to implement cache maintainance hooks for this case in the Xen arch code. These would basically look the same as the low-level cache maintainance used by the DMA ops, but without going through the DMA mapping layer, in fact they should probably reuse the same low-level assembly routines.
Quoted 5 lines
Quoted 2 lines
I don't think this is the first usage of such Xen buffer sharing, so what do the other users do?
I'll have to get even deeper into it. Initially I looked at the code, but didn't find anything useful. Or maybe I have just overlooked obvious things there
From Xen on Arm ABI:
"All memory which is shared with other entities in the system (including the hypervisor and other guests) must reside in memory which is mapped as Normal Inner Write-Back Outer Write-Back Inner-Shareable.
This applies to:
- hypercall arguments passed via a pointer to guest memory.
- memory shared via the grant table mechanism (including PV I/O rings etc).
- memory shared with the hypervisor (structshared_info, structvcpu_info, the grant table, etc).
"Quoted 4 lines
So you should not need any cache maintenance here. Can you provide more details on the memory attribute you use for memory shared in both the backend and frontend?
Quoted 5 lines
It takes quite some time to collect this (because many components are involved in the
use-case), but for now the pages in the guest domain are:
!PTE_RDONLY +PTE_PXN+PTE_SHARED+PTE_AF+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)
So that's the attribute for the page mapped in the frontend, right? How about the backend side?
Also, could that page be handed to the graphic card correctly? If so, is your graphic card coherent?
If one of your components is mapping with non-cacheable attributes then you are already not following the Xen Arm ABI. In that case, we would need to discuss how to extend the ABI.
Cheers,
Signature · 6 lines
Julien Grall _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Oleksandr Andrushchenko
maintainerHello, Julien!
Quoted 49 lines · Julien Grall
Hello,
Quoted 46 lines · Oleksandr Andrushchenko
Quoted 45 lines · Julien Grall
Quoted 21 lines · Oleksandr Andrushchenko
Quoted 20 lines · Christoph Hellwig
Quoted 12 lines · Oleksandr Andrushchenko
Quoted 1 line
This whole issue keeps getting more and more confusing.
Well, I don't really do DMA here, but instead the buffers in question are shared with other Xen domain, so effectively it could be thought of some sort of DMA here, where the "device" is that remote domain. If the buffers are not flushed then the remote part sees some inconsistency which in my case results in artifacts on screen while displaying the buffers. When buffers are allocated via DMA API then there are no artifacts; if buffers are allocated with shmem + DMA mapping then there are no artifacts as well.
The only offending use-case is when I use shmem backed buffers, but do not flush themThe right answer would be to implement cache maintainance hooks for this case in the Xen arch code. These would basically look the same as the low-level cache maintainance used by the DMA ops, but without going through the DMA mapping layer, in fact they should probably reuse the same low-level assembly routines.
Quoted 5 lines
Quoted 2 lines
I don't think this is the first usage of such Xen buffer sharing, so what do the other users do?
I'll have to get even deeper into it. Initially I looked at the code, but didn't find anything useful. Or maybe I have just overlooked obvious things there
From Xen on Arm ABI:
"All memory which is shared with other entities in the system (including the hypervisor and other guests) must reside in memory which is mapped as Normal Inner Write-Back Outer Write-Back Inner-Shareable.
This applies to:
- hypercall arguments passed via a pointer to guest memory.
- memory shared via the grant table mechanism (including PV I/O rings etc).
- memory shared with the hypervisor (structshared_info, structvcpu_info, the grant table, etc).
"So you should not need any cache maintenance here. Can you provide more details on the memory attribute you use for memory shared in both the backend and frontend?
Quoted 8 lines
Quoted 5 lines
It takes quite some time to collect this (because many components are involved in the
use-case), but for now the pages in the guest domain are:
!PTE_RDONLY +PTE_PXN+PTE_SHARED+PTE_AF+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)So that's the attribute for the page mapped in the frontend, right? How about the backend side?
Please see below
Quoted 2 lines
Also, could that page be handed to the graphic card correctly?
Yes, if we use zero-copying. But please see below
Quoted 1 line
If so, is your graphic card coherent?
Yes, it is
Quoted 7 lines
If one of your components is mapping with non-cacheable attributes then you are already not following the Xen Arm ABI. In that case, we would need to discuss how to extend the ABI.
Cheers,
Well, I didn't get the attributes of pages at the backend side, but IMO those
do not matter in my use-case (for simplicity I am not using zero-copying at backend side):
1. Frontend device allocates display buffer pages which come from shmem and have these attributes:
!PTE_RDONLY + PTE_PXN + PTE_SHARED + PTE_AF + PTE_UXN + PTE_ATTRINDX(MT_NORMAL)
2. Frontend grants references to these pages and shares those with the backend
3. Backend is a user-space application (Weston client), so it uses gntdev kernel
driver to mmap the pages. The pages, which are used by gntdev, are those coming
from the Xen balloon driver and I believe they are all normal memory and shouldn't be non-cached.
4. Once the frontend starts displaying it flips the buffers and backend does *memcpy*
from the frontend-backend shared buffer into Weston's buffer. This means no HW at the backend side touches the shared buffer.
5. I can see distorted picture.
Previously I used setup with zero-copying, so then the picture becomes more complicated
in terms of buffers and how those used by the backed, but anyways it seems that the
very basic scenario with memory copying doesn't work for me.
Using DMA API on frontend's side does help - no artifacts are seen. This is why I'm thinking that this is related to frontend/kernel side rather then to
the backend side. This is why I'm thinking this is related to caches and trying to figure
out what can be done here instead of using DMA API.
Thank you,
Olkesandr
Oleksandr Andrushchenko
maintainerHello, Julien!
Quoted 49 lines · Julien Grall
Hello,
Quoted 46 lines · Oleksandr Andrushchenko
Quoted 45 lines · Julien Grall
Quoted 21 lines · Oleksandr Andrushchenko
Quoted 20 lines · Christoph Hellwig
Quoted 12 lines · Oleksandr Andrushchenko
Quoted 1 line
This whole issue keeps getting more and more confusing.
Well, I don't really do DMA here, but instead the buffers in question are shared with other Xen domain, so effectively it could be thought of some sort of DMA here, where the "device" is that remote domain. If the buffers are not flushed then the remote part sees some inconsistency which in my case results in artifacts on screen while displaying the buffers. When buffers are allocated via DMA API then there are no artifacts; if buffers are allocated with shmem + DMA mapping then there are no artifacts as well.
The only offending use-case is when I use shmem backed buffers, but do not flush themThe right answer would be to implement cache maintainance hooks for this case in the Xen arch code. These would basically look the same as the low-level cache maintainance used by the DMA ops, but without going through the DMA mapping layer, in fact they should probably reuse the same low-level assembly routines.
Quoted 5 lines
Quoted 2 lines
I don't think this is the first usage of such Xen buffer sharing, so what do the other users do?
I'll have to get even deeper into it. Initially I looked at the code, but didn't find anything useful. Or maybe I have just overlooked obvious things there
From Xen on Arm ABI:
"All memory which is shared with other entities in the system (including the hypervisor and other guests) must reside in memory which is mapped as Normal Inner Write-Back Outer Write-Back Inner-Shareable.
This applies to:
- hypercall arguments passed via a pointer to guest memory.
- memory shared via the grant table mechanism (including PV I/O rings etc).
- memory shared with the hypervisor (structshared_info, structvcpu_info, the grant table, etc).
"So you should not need any cache maintenance here. Can you provide more details on the memory attribute you use for memory shared in both the backend and frontend?
Quoted 8 lines
Quoted 5 lines
It takes quite some time to collect this (because many components are involved in the
use-case), but for now the pages in the guest domain are:
!PTE_RDONLY +PTE_PXN+PTE_SHARED+PTE_AF+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)So that's the attribute for the page mapped in the frontend, right? How about the backend side?
Please see below
Quoted 2 lines
Also, could that page be handed to the graphic card correctly?
Yes, if we use zero-copying. But please see below
Quoted 1 line
If so, is your graphic card coherent?
Yes, it is
Quoted 7 lines
If one of your components is mapping with non-cacheable attributes then you are already not following the Xen Arm ABI. In that case, we would need to discuss how to extend the ABI.
Cheers,
Well, I didn't get the attributes of pages at the backend side, but IMO those
do not matter in my use-case (for simplicity I am not using zero-copying at backend side):
1. Frontend device allocates display buffer pages which come from shmem and have these attributes:
!PTE_RDONLY + PTE_PXN + PTE_SHARED + PTE_AF + PTE_UXN + PTE_ATTRINDX(MT_NORMAL)
2. Frontend grants references to these pages and shares those with the backend
3. Backend is a user-space application (Weston client), so it uses gntdev kernel
driver to mmap the pages. The pages, which are used by gntdev, are those coming
from the Xen balloon driver and I believe they are all normal memory and shouldn't be non-cached.
4. Once the frontend starts displaying it flips the buffers and backend does *memcpy*
from the frontend-backend shared buffer into Weston's buffer. This means no HW at the backend side touches the shared buffer.
5. I can see distorted picture.
Previously I used setup with zero-copying, so then the picture becomes more complicated
in terms of buffers and how those used by the backed, but anyways it seems that the
very basic scenario with memory copying doesn't work for me.
Using DMA API on frontend's side does help - no artifacts are seen. This is why I'm thinking that this is related to frontend/kernel side rather then to
the backend side. This is why I'm thinking this is related to caches and trying to figure
out what can be done here instead of using DMA API.
Thank you,
Olkesandr
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 1 line · Oleksandr Andrushchenko
Hello, Julien!
Hi,
Quoted 5 lines · Julien Grall
Well, I didn't get the attributes of pages at the backend side, but IMO those
do not matter in my use-case (for simplicity I am not using zero-copying at backend side):
They are actually important no matter what is your use case. If you access the same physical page with different attributes, then you are asking for trouble.
This is why Xen imposes all the pages shared to have their memory attributes following some rules. Actually, speaking with Mark R., we may want to tight a bit more the attributes.
Quoted 5 lines
1. Frontend device allocates display buffer pages which come from shmem and have these attributes:
!PTE_RDONLY +PTE_PXN+PTE_SHARED+PTE_AF+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)
My knowledge of Xen DRM is inexistent. However, looking at the code in 5.0-rc2, I don't seem to find the same attributes. For instance xen_drm_front_gem_prime_vmap and gem_mmap_obj are using pgprot_writecombine. So it looks like, the mapping will be non-cacheable on Arm64.
Can you explain how you came up to these attributes?
Quoted 24 lines
2. Frontend grants references to these pages and shares those with the backend
3. Backend is a user-space application (Weston client), so it uses gntdev kernel
driver to mmap the pages. The pages, which are used by gntdev, are those coming
from the Xen balloon driver and I believe they are all normal memory and shouldn't be non-cached.4. Once the frontend starts displaying it flips the buffers and backend does *memcpy*
from the frontend-backend shared buffer into Weston's buffer. This means no HW at the backend side touches the shared buffer.5. I can see distorted picture.
Previously I used setup with zero-copying, so then the picture becomes more complicated
in terms of buffers and how those used by the backed, but anyways it seems that the
very basic scenario with memory copying doesn't work for me.
Quoted 6 lines
Using DMA API on frontend's side does help - no artifacts are seen. This is why I'm thinking that this is related to frontend/kernel side rather then to
the backend side. This is why I'm thinking this is related to caches and trying to figure
out what can be done here instead of using DMA API.
We actually never required to use cache flush in other PV protocol, so I still don't understand why the PV DRM should be different here.
To me, it looks like that you are either missing some barriers or the memory attributes are not correct.
Cheers,
Signature · 1 line
Julien Grall
Quoted 1 line · Oleksandr Andrushchenko
Hello, Julien!
Hi,
Quoted 5 lines · Julien Grall
Well, I didn't get the attributes of pages at the backend side, but IMO those
do not matter in my use-case (for simplicity I am not using zero-copying at backend side):
They are actually important no matter what is your use case. If you access the same physical page with different attributes, then you are asking for trouble.
This is why Xen imposes all the pages shared to have their memory attributes following some rules. Actually, speaking with Mark R., we may want to tight a bit more the attributes.
Quoted 5 lines
1. Frontend device allocates display buffer pages which come from shmem and have these attributes:
!PTE_RDONLY +PTE_PXN+PTE_SHARED+PTE_AF+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)
My knowledge of Xen DRM is inexistent. However, looking at the code in 5.0-rc2, I don't seem to find the same attributes. For instance xen_drm_front_gem_prime_vmap and gem_mmap_obj are using pgprot_writecombine. So it looks like, the mapping will be non-cacheable on Arm64.
Can you explain how you came up to these attributes?
Quoted 24 lines
2. Frontend grants references to these pages and shares those with the backend
3. Backend is a user-space application (Weston client), so it uses gntdev kernel
driver to mmap the pages. The pages, which are used by gntdev, are those coming
from the Xen balloon driver and I believe they are all normal memory and shouldn't be non-cached.4. Once the frontend starts displaying it flips the buffers and backend does *memcpy*
from the frontend-backend shared buffer into Weston's buffer. This means no HW at the backend side touches the shared buffer.5. I can see distorted picture.
Previously I used setup with zero-copying, so then the picture becomes more complicated
in terms of buffers and how those used by the backed, but anyways it seems that the
very basic scenario with memory copying doesn't work for me.
Quoted 6 lines
Using DMA API on frontend's side does help - no artifacts are seen. This is why I'm thinking that this is related to frontend/kernel side rather then to
the backend side. This is why I'm thinking this is related to caches and trying to figure
out what can be done here instead of using DMA API.
We actually never required to use cache flush in other PV protocol, so I still don't understand why the PV DRM should be different here.
To me, it looks like that you are either missing some barriers or the memory attributes are not correct.
Cheers,
Signature · 6 lines
Julien Grall _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Oleksandr Andrushchenko
maintainerHello, Julien!
Sorry for the late reply - it took quite some time to collect the data requested.
Quoted 14 lines · Julien Grall
Quoted 1 line · Oleksandr Andrushchenko
Hello, Julien!
Hi,
Quoted 6 lines · Julien Grall
Well, I didn't get the attributes of pages at the backend side, but IMO those
do not matter in my use-case (for simplicity I am not using zero-copying at
backend side):
Quoted 3 lines
They are actually important no matter what is your use case. If you access the same physical page with different attributes, then you are asking for trouble.
So, we have:
DomU: frontend side
====================
!PTE_RDONLY + PTE_PXN + PTE_SHARED + PTE_AF + PTE_UXN + PTE_ATTRINDX(MT_NORMAL)
DomD: backend side
====================PTE_USER + !PTE_RDONLY + PTE_PXN + PTE_NG + PTE_CONT + PTE_TABLE_BIT + PTE_UXN + PTE_ATTRINDX(MT_NORMAL)
From the above it seems that I don't violate cached/non-cached
agreement here
Quoted 11 lines
This is why Xen imposes all the pages shared to have their memory attributes following some rules. Actually, speaking with Mark R., we may want to tight a bit more the attributes.
Quoted 5 lines
1. Frontend device allocates display buffer pages which come from shmem and have these attributes:
!PTE_RDONLY +PTE_PXN+PTE_SHARED+PTE_AF+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)
Quoted 7 lines
My knowledge of Xen DRM is inexistent. However, looking at the code in 5.0-rc2, I don't seem to find the same attributes. For instance
xen_drm_front_gem_prime_vmapandgem_mmap_objare usingpgprot_writecombine. So it looks like, the mapping will be non-cacheable on Arm64.Can you explain how you came up to these attributes?
pgprot_writecombine is PTE_ATTRINDX(MT_NORMAL_NC), so it seems to be applicable here? [1]
Quoted 32 lines
Quoted 25 lines
2. Frontend grants references to these pages and shares those with the backend
3. Backend is a user-space application (Weston client), so it uses gntdev kernel
driver to mmap the pages. The pages, which are used by gntdev, are those coming
from the Xen balloon driver and I believe they are all normal memory and shouldn't be non-cached.4. Once the frontend starts displaying it flips the buffers and backend does *memcpy*
from the frontend-backend shared buffer into Weston's buffer. This means no HW at the backend side touches the shared buffer.5. I can see distorted picture.
Previously I used setup with zero-copying, so then the picture becomes more complicated
in terms of buffers and how those used by the backed, but anyways it seems that the
very basic scenario with memory copying doesn't work for me.Quoted 6 lines
Using DMA API on frontend's side does help - no artifacts are seen. This is why I'm thinking that this is related to frontend/kernel side rather then to
the backend side. This is why I'm thinking this is related to caches and trying to figure
out what can be done here instead of using DMA API.
Quoted 2 lines
We actually never required to use cache flush in other PV protocol, so I still don't understand why the PV DRM should be different here.
Well, you are right. But at the same time not flushing the buffer makes troubles,
so this is why I am trying to figure out what is wrong here.
Quoted 2 lines
To me, it looks like that you are either missing some barriers
Barriers for the buffer? Not sure what you mean here. Even more, we have a use case
when the buffer is not touched by CPU in DomD and is solely owned by the HW.
Quoted 1 line
or the memory attributes are not correct.
Please see above - I do need your advice here
Quoted 3 lines
Cheers,
Thank you very much for your time,
Oleksandr
[1]
https://elixir.bootlin.com/linux/v5.0-rc2/source/arch/arm64/include/asm/pgtable.h#L414
Oleksandr Andrushchenko
maintainerHello, Julien!
Sorry for the late reply - it took quite some time to collect the data requested.
Quoted 14 lines · Julien Grall
Quoted 1 line · Oleksandr Andrushchenko
Hello, Julien!
Hi,
Quoted 6 lines · Julien Grall
Well, I didn't get the attributes of pages at the backend side, but IMO those
do not matter in my use-case (for simplicity I am not using zero-copying at
backend side):
Quoted 3 lines
They are actually important no matter what is your use case. If you access the same physical page with different attributes, then you are asking for trouble.
So, we have:
DomU: frontend side
====================
!PTE_RDONLY + PTE_PXN + PTE_SHARED + PTE_AF + PTE_UXN + PTE_ATTRINDX(MT_NORMAL)
DomD: backend side
====================PTE_USER + !PTE_RDONLY + PTE_PXN + PTE_NG + PTE_CONT + PTE_TABLE_BIT + PTE_UXN + PTE_ATTRINDX(MT_NORMAL)
From the above it seems that I don't violate cached/non-cached
agreement here
Quoted 11 lines
This is why Xen imposes all the pages shared to have their memory attributes following some rules. Actually, speaking with Mark R., we may want to tight a bit more the attributes.
Quoted 5 lines
1. Frontend device allocates display buffer pages which come from shmem and have these attributes:
!PTE_RDONLY +PTE_PXN+PTE_SHARED+PTE_AF+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)
Quoted 7 lines
My knowledge of Xen DRM is inexistent. However, looking at the code in 5.0-rc2, I don't seem to find the same attributes. For instance
xen_drm_front_gem_prime_vmapandgem_mmap_objare usingpgprot_writecombine. So it looks like, the mapping will be non-cacheable on Arm64.Can you explain how you came up to these attributes?
pgprot_writecombine is PTE_ATTRINDX(MT_NORMAL_NC), so it seems to be applicable here? [1]
Quoted 32 lines
Quoted 25 lines
2. Frontend grants references to these pages and shares those with the backend
3. Backend is a user-space application (Weston client), so it uses gntdev kernel
driver to mmap the pages. The pages, which are used by gntdev, are those coming
from the Xen balloon driver and I believe they are all normal memory and shouldn't be non-cached.4. Once the frontend starts displaying it flips the buffers and backend does *memcpy*
from the frontend-backend shared buffer into Weston's buffer. This means no HW at the backend side touches the shared buffer.5. I can see distorted picture.
Previously I used setup with zero-copying, so then the picture becomes more complicated
in terms of buffers and how those used by the backed, but anyways it seems that the
very basic scenario with memory copying doesn't work for me.Quoted 6 lines
Using DMA API on frontend's side does help - no artifacts are seen. This is why I'm thinking that this is related to frontend/kernel side rather then to
the backend side. This is why I'm thinking this is related to caches and trying to figure
out what can be done here instead of using DMA API.
Quoted 2 lines
We actually never required to use cache flush in other PV protocol, so I still don't understand why the PV DRM should be different here.
Well, you are right. But at the same time not flushing the buffer makes troubles,
so this is why I am trying to figure out what is wrong here.
Quoted 2 lines
To me, it looks like that you are either missing some barriers
Barriers for the buffer? Not sure what you mean here. Even more, we have a use case
when the buffer is not touched by CPU in DomD and is solely owned by the HW.
Quoted 1 line
or the memory attributes are not correct.
Please see above - I do need your advice here
Quoted 3 lines
Cheers,
Thank you very much for your time,
Oleksandr
[1]
https://elixir.bootlin.com/linux/v5.0-rc2/source/arch/arm64/include/asm/pgtable.h#L414
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 1 line · Oleksandr Andrushchenko
Hello, Julien!
Hi,
Quoted 18 lines · Julien Grall
Quoted 14 lines · Oleksandr Andrushchenko
Quoted 1 line
Hello, Julien!
Hi,
Quoted 6 lines · Julien Grall
Well, I didn't get the attributes of pages at the backend side, but IMO those
do not matter in my use-case (for simplicity I am not using zero-copying at
backend side):Quoted 3 lines
They are actually important no matter what is your use case. If you access the same physical page with different attributes, then you are asking for trouble.
Quoted 6 lines
So, we have:
DomU: frontend side
====================
!PTE_RDONLY +PTE_PXN+PTE_SHARED+PTE_AF+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)
I still don't understand how you came up with MT_NORMAL when you seem to confirm...
Quoted 25 lines
DomD: backend side
====================PTE_USER+ !PTE_RDONLY +PTE_PXN+PTE_NG+PTE_CONT+PTE_TABLE_BIT+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)From the above it seems that I don't violate cached/non-cached
agreement hereQuoted 17 lines
This is why Xen imposes all the pages shared to have their memory attributes following some rules. Actually, speaking with Mark R., we may want to tight a bit more the attributes.
Quoted 5 lines
1. Frontend device allocates display buffer pages which come from shmem and have these attributes:
!PTE_RDONLY +PTE_PXN+PTE_SHARED+PTE_AF+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)My knowledge of Xen DRM is inexistent. However, looking at the code in 5.0-rc2, I don't seem to find the same attributes. For instance
xen_drm_front_gem_prime_vmapandgem_mmap_objare usingpgprot_writecombine. So it looks like, the mapping will be non-cacheable on Arm64.
Quoted 3 lines
Quoted 1 line
Can you explain how you came up to these attributes?
pgprot_writecombineisPTE_ATTRINDX(MT_NORMAL_NC), so it seems to be applicable here? [1]
... that MT_NORMAL_NC is used for the frontend pages.
MT_NORMAL_NC is different from MT_NORMAL. The use of the former will result to non-cacheable memory while the latter will result to cacheable memory.
To me, this looks like the exact reason why you see artifact on the display buffer. As the author of this code, can you explain why you decided to use pgprot_writecombine here instead of relying on the default VMA prot?
[...]
Quoted 5 lines
Quoted 2 lines
We actually never required to use cache flush in other PV protocol, so I still don't understand why the PV DRM should be different here.
Well, you are right. But at the same time not flushing the buffer makes troubles,
so this is why I am trying to figure out what is wrong here.
The cache flush is likely hiding the real problem rather than solving it.
Quoted 3 lines
Quoted 2 lines
To me, it looks like that you are either missing some barriers
Barriers for the buffer? Not sure what you mean here.
If you share information between two entities, you may need some ordering so the information are seen consistently by the consumer side. This can be achieved by using barriers.
Quoted 3 lines
Even more, we have
a use case
when the buffer is not touched by CPU in DomD and is solely owned by the HW.
Memory ordering issues are subtle. The fact that one of your use-case works does not imply that barriers are not necessary. I am not saying there are a missing barriers, I am only pointed out potential reasons.
Anyway, I don't think your problem is a missing barriers here. It is more likely because of mismatch memory attributes (see above).
Cheers,
Signature · 1 line
Julien Grall
Quoted 1 line · Oleksandr Andrushchenko
Hello, Julien!
Hi,
Quoted 18 lines · Julien Grall
Quoted 14 lines · Oleksandr Andrushchenko
Quoted 1 line
Hello, Julien!
Hi,
Quoted 6 lines · Julien Grall
Well, I didn't get the attributes of pages at the backend side, but IMO those
do not matter in my use-case (for simplicity I am not using zero-copying at
backend side):Quoted 3 lines
They are actually important no matter what is your use case. If you access the same physical page with different attributes, then you are asking for trouble.
Quoted 6 lines
So, we have:
DomU: frontend side
====================
!PTE_RDONLY +PTE_PXN+PTE_SHARED+PTE_AF+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)
I still don't understand how you came up with MT_NORMAL when you seem to confirm...
Quoted 25 lines
DomD: backend side
====================PTE_USER+ !PTE_RDONLY +PTE_PXN+PTE_NG+PTE_CONT+PTE_TABLE_BIT+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)From the above it seems that I don't violate cached/non-cached
agreement hereQuoted 17 lines
This is why Xen imposes all the pages shared to have their memory attributes following some rules. Actually, speaking with Mark R., we may want to tight a bit more the attributes.
Quoted 5 lines
1. Frontend device allocates display buffer pages which come from shmem and have these attributes:
!PTE_RDONLY +PTE_PXN+PTE_SHARED+PTE_AF+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)My knowledge of Xen DRM is inexistent. However, looking at the code in 5.0-rc2, I don't seem to find the same attributes. For instance
xen_drm_front_gem_prime_vmapandgem_mmap_objare usingpgprot_writecombine. So it looks like, the mapping will be non-cacheable on Arm64.
Quoted 3 lines
Quoted 1 line
Can you explain how you came up to these attributes?
pgprot_writecombineisPTE_ATTRINDX(MT_NORMAL_NC), so it seems to be applicable here? [1]
... that MT_NORMAL_NC is used for the frontend pages.
MT_NORMAL_NC is different from MT_NORMAL. The use of the former will result to non-cacheable memory while the latter will result to cacheable memory.
To me, this looks like the exact reason why you see artifact on the display buffer. As the author of this code, can you explain why you decided to use pgprot_writecombine here instead of relying on the default VMA prot?
[...]
Quoted 5 lines
Quoted 2 lines
We actually never required to use cache flush in other PV protocol, so I still don't understand why the PV DRM should be different here.
Well, you are right. But at the same time not flushing the buffer makes troubles,
so this is why I am trying to figure out what is wrong here.
The cache flush is likely hiding the real problem rather than solving it.
Quoted 3 lines
Quoted 2 lines
To me, it looks like that you are either missing some barriers
Barriers for the buffer? Not sure what you mean here.
If you share information between two entities, you may need some ordering so the information are seen consistently by the consumer side. This can be achieved by using barriers.
Quoted 3 lines
Even more, we have
a use case
when the buffer is not touched by CPU in DomD and is solely owned by the HW.
Memory ordering issues are subtle. The fact that one of your use-case works does not imply that barriers are not necessary. I am not saying there are a missing barriers, I am only pointed out potential reasons.
Anyway, I don't think your problem is a missing barriers here. It is more likely because of mismatch memory attributes (see above).
Cheers,
Signature · 6 lines
Julien Grall _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Oleksandr Andrushchenko
maintainerQuoted 72 lines · Julien Grall
Quoted 1 line · Oleksandr Andrushchenko
Hello, Julien!
Hi,
Quoted 19 lines · Julien Grall
Quoted 15 lines · Oleksandr Andrushchenko
Quoted 1 line
Hello, Julien!
Hi,
Quoted 7 lines · Julien Grall
Well, I didn't get the attributes of pages at the backend side, but IMO
those
do not matter in my use-case (for simplicity I am not using zero-copying at
backend side):Quoted 3 lines
They are actually important no matter what is your use case. If you access the same physical page with different attributes, then you are asking for trouble.
Quoted 6 lines
So, we have:
DomU: frontend side
====================
!PTE_RDONLY +PTE_PXN+PTE_SHARED+PTE_AF+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)I still don't understand how you came up with
MT_NORMALwhen you seem to confirm...Quoted 26 lines
DomD: backend side
====================PTE_USER+ !PTE_RDONLY +PTE_PXN+PTE_NG+PTE_CONT+PTE_TABLE_BIT+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)From the above it seems that I don't violate cached/non-cached agreement here
Quoted 18 lines
This is why Xen imposes all the pages shared to have their memory attributes following some rules. Actually, speaking with Mark R., we may want to tight a bit more the attributes.
Quoted 6 lines
1. Frontend device allocates display buffer pages which come from shmem
and have these attributes:
!PTE_RDONLY +PTE_PXN+PTE_SHARED+PTE_AF+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)My knowledge of Xen DRM is inexistent. However, looking at the code in 5.0-rc2, I don't seem to find the same attributes. For instance
xen_drm_front_gem_prime_vmapandgem_mmap_objare usingpgprot_writecombine. So it looks like, the mapping will be non-cacheable on Arm64.Quoted 3 lines
Quoted 1 line
Can you explain how you came up to these attributes?
pgprot_writecombineisPTE_ATTRINDX(MT_NORMAL_NC), so it seems to be applicable here? [1]... that
MT_NORMAL_NCis used for the frontend pages.
MT_NORMAL_NCis different fromMT_NORMAL. The use of the former will result to non-cacheable memory while the latter will result to cacheable memory.
Quoted 6 lines
To me, this looks like the exact reason why you see artifact on the display buffer. As the author of this code, can you explain why you decided to use
pgprot_writecombinehere instead of relying on the default VMA prot?[...]
Sorry for late reply, it took me quite some time to re-check all the use-cases
that we have with PV DRM.
First of all I found a bug in my debug code which reported page attributes and
now I can confirm that all the use-cases sue MT_NORMAL, both backend and frontend.
You are right: there is no reason to have pgprot_writecombine and indeed I have
to rely on almost default VMA prot. I came up with the following (used by omap drm,
for example):
(quoted diff) +0 -0
b/drivers/gpu/drm/xen/xen_drm_front_gem.c
index ae28ad4b4254..867800a2ed42 100644
drivers/gpu/drm/xen/xen_drm_front_gem.c +1 -2
@@ -238,8 +238,7 @@ static int gem_mmap_obj(struct xen_gem_object *xen_obj, vma->vm_flags &= ~VM_PFNMAP; vma->vm_flags |= VM_MIXEDMAP; vma->vm_pgoff = 0; - vma->vm_page_prot = -pgprot_writecombine (vm_get_page_prot (vma->vm_flags)); + vma->vm_page_prot = vm_get_page_prot(vma->vm_flags); { int cnt = xen_obj->num_pages > 5 ? 5 : xen_obj->num_pages;
@@ -295,7 +294,7 @@ void *xen_drm_front_gem_prime_vmap(struct
drm_gem_object *gem_obj)
return NULL;
return vmap(xen_obj->pages, xen_obj->num_pages,
- VM_MAP, pgprot_writecombine(PAGE_KERNEL)); + VM_MAP, PAGE_KERNEL);
}
With the above all the artifacts are gone now as page attributes are the same across
all domains. So, I consider this as a fix and will send it as v3 or better drop this patch and have a new one.
THANK YOU for helping figuring this out!
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
We actually never required to use cache flush in other PV protocol, so I still don't understand why the PV DRM should be different here.
Well, you are right. But at the same time not flushing the buffer makes troubles,
so this is why I am trying to figure out what is wrong here.The cache flush is likely hiding the real problem rather than solving it.
It does hide the real issue. And I have confirmed this
Quoted 13 lines
Quoted 3 lines
Quoted 2 lines
To me, it looks like that you are either missing some barriers
Barriers for the buffer? Not sure what you mean here.
If you share information between two entities, you may need some ordering so the information are seen consistently by the consumer side. This can be achieved by using barriers.
Quoted 4 lines
Even more, we have
a use case
when the buffer is not touched by CPU in DomD and is solely owned by the HW.
Quoted 7 lines
Memory ordering issues are subtle. The fact that one of your use-case works does not imply that barriers are not necessary. I am not saying there are a missing barriers, I am only pointed out potential reasons.
Anyway, I don't think your problem is a missing barriers here. It is more likely because of mismatch memory attributes (see above).
Yes, please see above
Quoted 2 lines
Cheers,
Thank you all for helping with the correct fix,
Oleksandr
Oleksandr Andrushchenko
maintainerQuoted 72 lines · Julien Grall
Quoted 1 line · Oleksandr Andrushchenko
Hello, Julien!
Hi,
Quoted 19 lines · Julien Grall
Quoted 15 lines · Oleksandr Andrushchenko
Quoted 1 line
Hello, Julien!
Hi,
Quoted 7 lines · Julien Grall
Well, I didn't get the attributes of pages at the backend side, but IMO
those
do not matter in my use-case (for simplicity I am not using zero-copying at
backend side):Quoted 3 lines
They are actually important no matter what is your use case. If you access the same physical page with different attributes, then you are asking for trouble.
Quoted 6 lines
So, we have:
DomU: frontend side
====================
!PTE_RDONLY +PTE_PXN+PTE_SHARED+PTE_AF+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)I still don't understand how you came up with
MT_NORMALwhen you seem to confirm...Quoted 26 lines
DomD: backend side
====================PTE_USER+ !PTE_RDONLY +PTE_PXN+PTE_NG+PTE_CONT+PTE_TABLE_BIT+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)From the above it seems that I don't violate cached/non-cached agreement here
Quoted 18 lines
This is why Xen imposes all the pages shared to have their memory attributes following some rules. Actually, speaking with Mark R., we may want to tight a bit more the attributes.
Quoted 6 lines
1. Frontend device allocates display buffer pages which come from shmem
and have these attributes:
!PTE_RDONLY +PTE_PXN+PTE_SHARED+PTE_AF+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)My knowledge of Xen DRM is inexistent. However, looking at the code in 5.0-rc2, I don't seem to find the same attributes. For instance
xen_drm_front_gem_prime_vmapandgem_mmap_objare usingpgprot_writecombine. So it looks like, the mapping will be non-cacheable on Arm64.Quoted 3 lines
Quoted 1 line
Can you explain how you came up to these attributes?
pgprot_writecombineisPTE_ATTRINDX(MT_NORMAL_NC), so it seems to be applicable here? [1]... that
MT_NORMAL_NCis used for the frontend pages.
MT_NORMAL_NCis different fromMT_NORMAL. The use of the former will result to non-cacheable memory while the latter will result to cacheable memory.
Quoted 6 lines
To me, this looks like the exact reason why you see artifact on the display buffer. As the author of this code, can you explain why you decided to use
pgprot_writecombinehere instead of relying on the default VMA prot?[...]
Sorry for late reply, it took me quite some time to re-check all the use-cases
that we have with PV DRM.
First of all I found a bug in my debug code which reported page attributes and
now I can confirm that all the use-cases sue MT_NORMAL, both backend and frontend.
You are right: there is no reason to have pgprot_writecombine and indeed I have
to rely on almost default VMA prot. I came up with the following (used by omap drm,
for example):
(quoted diff) +0 -0
b/drivers/gpu/drm/xen/xen_drm_front_gem.c
index ae28ad4b4254..867800a2ed42 100644
drivers/gpu/drm/xen/xen_drm_front_gem.c +1 -2
@@ -238,8 +238,7 @@ static int gem_mmap_obj(struct xen_gem_object *xen_obj, vma->vm_flags &= ~VM_PFNMAP; vma->vm_flags |= VM_MIXEDMAP; vma->vm_pgoff = 0; - vma->vm_page_prot = -pgprot_writecombine (vm_get_page_prot (vma->vm_flags)); + vma->vm_page_prot = vm_get_page_prot(vma->vm_flags); { int cnt = xen_obj->num_pages > 5 ? 5 : xen_obj->num_pages;
@@ -295,7 +294,7 @@ void *xen_drm_front_gem_prime_vmap(struct
drm_gem_object *gem_obj)
return NULL;
return vmap(xen_obj->pages, xen_obj->num_pages,
- VM_MAP, pgprot_writecombine(PAGE_KERNEL)); + VM_MAP, PAGE_KERNEL);
}
With the above all the artifacts are gone now as page attributes are the same across
all domains. So, I consider this as a fix and will send it as v3 or better drop this patch and have a new one.
THANK YOU for helping figuring this out!
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
We actually never required to use cache flush in other PV protocol, so I still don't understand why the PV DRM should be different here.
Well, you are right. But at the same time not flushing the buffer makes troubles,
so this is why I am trying to figure out what is wrong here.The cache flush is likely hiding the real problem rather than solving it.
It does hide the real issue. And I have confirmed this
Quoted 13 lines
Quoted 3 lines
Quoted 2 lines
To me, it looks like that you are either missing some barriers
Barriers for the buffer? Not sure what you mean here.
If you share information between two entities, you may need some ordering so the information are seen consistently by the consumer side. This can be achieved by using barriers.
Quoted 4 lines
Even more, we have
a use case
when the buffer is not touched by CPU in DomD and is solely owned by the HW.
Quoted 7 lines
Memory ordering issues are subtle. The fact that one of your use-case works does not imply that barriers are not necessary. I am not saying there are a missing barriers, I am only pointed out potential reasons.
Anyway, I don't think your problem is a missing barriers here. It is more likely because of mismatch memory attributes (see above).
Yes, please see above
Quoted 2 lines
Cheers,
Thank you all for helping with the correct fix,
Oleksandr
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Oleksandr Andrushchenko
maintainerDropped in favor of https://lkml.org/lkml/2019/1/29/910
Quoted 109 lines · Julien Grall
Quoted 1 line · Oleksandr Andrushchenko
Hello, Julien!
Hi,
Quoted 19 lines · Julien Grall
Quoted 15 lines · Oleksandr Andrushchenko
Quoted 1 line
Hello, Julien!
Hi,
Quoted 7 lines · Julien Grall
Well, I didn't get the attributes of pages at the backend side, but IMO
those
do not matter in my use-case (for simplicity I am not using zero-copying at
backend side):Quoted 3 lines
They are actually important no matter what is your use case. If you access the same physical page with different attributes, then you are asking for trouble.
Quoted 6 lines
So, we have:
DomU: frontend side
====================
!PTE_RDONLY +PTE_PXN+PTE_SHARED+PTE_AF+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)I still don't understand how you came up with
MT_NORMALwhen you seem to confirm...Quoted 26 lines
DomD: backend side
====================PTE_USER+ !PTE_RDONLY +PTE_PXN+PTE_NG+PTE_CONT+PTE_TABLE_BIT+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)From the above it seems that I don't violate cached/non-cached agreement here
Quoted 18 lines
This is why Xen imposes all the pages shared to have their memory attributes following some rules. Actually, speaking with Mark R., we may want to tight a bit more the attributes.
Quoted 6 lines
1. Frontend device allocates display buffer pages which come from shmem
and have these attributes:
!PTE_RDONLY +PTE_PXN+PTE_SHARED+PTE_AF+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)My knowledge of Xen DRM is inexistent. However, looking at the code in 5.0-rc2, I don't seem to find the same attributes. For instance
xen_drm_front_gem_prime_vmapandgem_mmap_objare usingpgprot_writecombine. So it looks like, the mapping will be non-cacheable on Arm64.Quoted 3 lines
Quoted 1 line
Can you explain how you came up to these attributes?
pgprot_writecombineisPTE_ATTRINDX(MT_NORMAL_NC), so it seems to be applicable here? [1]... that
MT_NORMAL_NCis used for the frontend pages.
MT_NORMAL_NCis different fromMT_NORMAL. The use of the former will result to non-cacheable memory while the latter will result to cacheable memory.To me, this looks like the exact reason why you see artifact on the display buffer. As the author of this code, can you explain why you decided to use
pgprot_writecombinehere instead of relying on the default VMA prot?[...]
Quoted 5 lines
Quoted 2 lines
We actually never required to use cache flush in other PV protocol, so I still don't understand why the PV DRM should be different here.
Well, you are right. But at the same time not flushing the buffer makes troubles,
so this is why I am trying to figure out what is wrong here.The cache flush is likely hiding the real problem rather than solving it.
Quoted 3 lines
Quoted 2 lines
To me, it looks like that you are either missing some barriers
Barriers for the buffer? Not sure what you mean here.
If you share information between two entities, you may need some ordering so the information are seen consistently by the consumer side. This can be achieved by using barriers.
Quoted 4 lines
Even more, we have
a use case
when the buffer is not touched by CPU in DomD and is solely owned by the HW.Memory ordering issues are subtle. The fact that one of your use-case works does not imply that barriers are not necessary. I am not saying there are a missing barriers, I am only pointed out potential reasons.
Anyway, I don't think your problem is a missing barriers here. It is more likely because of mismatch memory attributes (see above).
Cheers,
Oleksandr Andrushchenko
maintainerDropped in favor of https://lkml.org/lkml/2019/1/29/910
Quoted 100 lines · Julien Grall
Quoted 1 line · Oleksandr Andrushchenko
Hello, Julien!
Hi,
Quoted 19 lines · Julien Grall
Quoted 15 lines · Oleksandr Andrushchenko
Quoted 1 line
Hello, Julien!
Hi,
Quoted 7 lines · Julien Grall
Well, I didn't get the attributes of pages at the backend side, but IMO
those
do not matter in my use-case (for simplicity I am not using zero-copying at
backend side):Quoted 3 lines
They are actually important no matter what is your use case. If you access the same physical page with different attributes, then you are asking for trouble.
Quoted 6 lines
So, we have:
DomU: frontend side
====================
!PTE_RDONLY +PTE_PXN+PTE_SHARED+PTE_AF+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)I still don't understand how you came up with
MT_NORMALwhen you seem to confirm...Quoted 26 lines
DomD: backend side
====================PTE_USER+ !PTE_RDONLY +PTE_PXN+PTE_NG+PTE_CONT+PTE_TABLE_BIT+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)From the above it seems that I don't violate cached/non-cached agreement here
Quoted 18 lines
This is why Xen imposes all the pages shared to have their memory attributes following some rules. Actually, speaking with Mark R., we may want to tight a bit more the attributes.
Quoted 6 lines
1. Frontend device allocates display buffer pages which come from shmem
and have these attributes:
!PTE_RDONLY +PTE_PXN+PTE_SHARED+PTE_AF+PTE_UXN+PTE_ATTRINDX(MT_NORMAL)My knowledge of Xen DRM is inexistent. However, looking at the code in 5.0-rc2, I don't seem to find the same attributes. For instance
xen_drm_front_gem_prime_vmapandgem_mmap_objare usingpgprot_writecombine. So it looks like, the mapping will be non-cacheable on Arm64.Quoted 3 lines
Quoted 1 line
Can you explain how you came up to these attributes?
pgprot_writecombineisPTE_ATTRINDX(MT_NORMAL_NC), so it seems to be applicable here? [1]... that
MT_NORMAL_NCis used for the frontend pages.
MT_NORMAL_NCis different fromMT_NORMAL. The use of the former will result to non-cacheable memory while the latter will result to cacheable memory.To me, this looks like the exact reason why you see artifact on the display buffer. As the author of this code, can you explain why you decided to use
pgprot_writecombinehere instead of relying on the default VMA prot?[...]
Quoted 5 lines
Quoted 2 lines
We actually never required to use cache flush in other PV protocol, so I still don't understand why the PV DRM should be different here.
Well, you are right. But at the same time not flushing the buffer makes troubles,
so this is why I am trying to figure out what is wrong here.The cache flush is likely hiding the real problem rather than solving it.
Quoted 3 lines
Quoted 2 lines
To me, it looks like that you are either missing some barriers
Barriers for the buffer? Not sure what you mean here.
If you share information between two entities, you may need some ordering so the information are seen consistently by the consumer side. This can be achieved by using barriers.
Quoted 4 lines
Even more, we have
a use case
when the buffer is not touched by CPU in DomD and is solely owned by the HW.
Quoted 9 lines
Memory ordering issues are subtle. The fact that one of your use-case works does not imply that barriers are not necessary. I am not saying there are a missing barriers, I am only pointed out potential reasons.
Anyway, I don't think your problem is a missing barriers here. It is more likely because of mismatch memory attributes (see above).
Cheers,
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel