drm: return false in drm_arch_can_wc_memory() for ARM
Subsystems (2)
PATCH 1/1 drm: return false in drm_arch_can_wc_memory() for ARM
I'm not familiar enough with ARM to know if write combining is actually an architectural limitation or if it's an issue with the PCIe IPs used on various platforms, but so far everyone that has tried to run radeon hardware on ARM has had to disable it. So let's just make it official.
--- include/drm/drm_cache.h | 2 ++ 1 file changed, 2 insertions(+)
include/drm/drm_cache.h +2 -0
@@ -47,6 +47,8 @@ static inline bool drm_arch_can_wc_memory(void)return false;#elif defined(CONFIG_MIPS ) && defined(CONFIG_CPU_LOONGSON3 )return false; +#elif defined(CONFIG_ARM ) || defined(CONFIG_ARM64 ) +return false;#else return true;#endif
Signature · 6 lines
2.13.6 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 20181220145657.304-1-alexander.deucher-5C7GfCeVMHo@public.gmane.org
Daniel Vetter
Reviewed-byQuoted 5 lines · Alex Deucher
I'm not familiar enough with ARM to know if write combining is actually an architectural limitation or if it's an issue with the PCIe IPs used on various platforms, but so far everyone that has tried to run radeon hardware on ARM has had to disable it. So let's just make it official.
wc on arm is Really Complicated (tm) afaiui. There's issues with aliasing mappings and stuff, so you need to allocate your wc memory from special pools. So probably best to just disable it until we figure this out.
Quoted 1 line
Signed-off-by Alex Deucher <[email protected]>
Quoted 24 lines
--- include/drm/drm_cache.h | 2 ++ 1 file changed, 2 insertions(+)include/drm/drm_cache.h +2
-0@@ -47,6 +47,8 @@ static inline bool drm_arch_can_wc_memory(void)return false;#elif defined(CONFIG_MIPS ) && defined(CONFIG_CPU_LOONGSON3 )return false; +#elif defined(CONFIG_ARM ) || defined(CONFIG_ARM64 ) +return false;#else return true;#endif Signature · 6 lines
2.13.6 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 20181220153619.GP21184-dv86pmgwkMBes7Z6vYuT8azUEOm+Xw19@public.gmane.org
Quoted 10 lines · Daniel Vetter
Quoted 5 lines · Alex Deucher
I'm not familiar enough with ARM to know if write combining is actually an architectural limitation or if it's an issue with the PCIe IPs used on various platforms, but so far everyone that has tried to run radeon hardware on ARM has had to disable it. So let's just make it official.
wc on arm is Really Complicated (tm) afaiui. There's issues with aliasing mappings and stuff, so you need to allocate your wc memory from special pools. So probably best to just disable it until we figure this out.
I believe both of you are conflating different issues under the wrong name. Write combining happens all the time with Arm, the ARMv8 architecture is a weakly-ordered model of memory so hardware is allowed to re-order or combine memory access as they seem fit.
A while ago I did run an AMD GPU card on my Juno dev board and it worked (for a very limited definition of worked, I've only validated the fact that I could get an fbcon and could run un-accelerated X11). So I would be interested if Alex could share some of the scenarios where people are seeing failures.
As for aliasing, yeah, having multiple aliases to the same piece of memory is a bad thing. The problem arises when devices on the PCI bus have memory allocated as device memory (which on Arm is non-cacheable and non-reorderable), but the PCI bus effectively acts as a write-combiner which changes the order of transactions. Therefore, for devices that have local memory associated with them (i.e. more than just register accesses) one should allocate memory in the first place that is Device-GRE (gathering, reordering and early-access). Otherwise, problems will surface that are not visible on x86 as that is a strongly ordered architecture.
Quoted 4 lines
Quoted 2 lines
Signed-off-by Alex Deucher <[email protected]>Reviewed-by Daniel Vetter <[email protected]>
Given that this API is only used by AMD I'm OK for now with the change, but I think in general it is misleading and we should work towards fixing radeon and amd drivers.
Best regards,
Liviu
Quoted 34 lines
Quoted 25 lines
--- include/drm/drm_cache.h | 2 ++ 1 file changed, 2 insertions(+)include/drm/drm_cache.h +2
-0@@ -47,6 +47,8 @@ static inline bool drm_arch_can_wc_memory(void)return false;#elif defined(CONFIG_MIPS ) && defined(CONFIG_CPU_LOONGSON3 )return false; +#elif defined(CONFIG_ARM ) || defined(CONFIG_ARM64 ) +return false;#else return true;#endif Signature · 6 lines
2.13.6 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-develSignature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Signature · 12 lines
====================
| I would like to |
| fix the world, |
| but they're not |
| giving me the |
\ source code! /
---------------
¯\_(ツ)_/¯
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfxA message that is not in the archive (sent off-list or not yet received): 20181221141644.GB22341-A/Nd4k6kWRHZROr8t4l/[email protected]
Quoted 18 lines · Liviu Dudau
Quoted 10 lines · Daniel Vetter
Quoted 5 lines · Alex Deucher
I'm not familiar enough with ARM to know if write combining is actually an architectural limitation or if it's an issue with the PCIe IPs used on various platforms, but so far everyone that has tried to run radeon hardware on ARM has had to disable it. So let's just make it official.
wc on arm is Really Complicated (tm) afaiui. There's issues with aliasing mappings and stuff, so you need to allocate your wc memory from special pools. So probably best to just disable it until we figure this out.
I believe both of you are conflating different issues under the wrong name. Write combining happens all the time with Arm, the ARMv8 architecture is a weakly-ordered model of memory so hardware is allowed to re-order or combine memory access as they seem fit.
Quoted 5 lines
A while ago I did run an AMD GPU card on my Juno dev board and it worked (for a very limited definition of worked, I've only validated the fact that I could get an fbcon and could run un-accelerated X11). So I would be interested if Alex could share some of the scenarios where people are seeing failures.
Here's an example:
https://bugs.freedesktop.org/show_bug.cgi?id=108625
But there are probably 5 or 6 other cases where people have emailed me or our team directly with issues on ARM resolved by disabling WC. Generally the driver seems to load ok, but then hangs as soon as you try and use acceleration from userspace or we end up with page flipping timeouts. Not really sure what the issue is. Michel suggested maybe ARM has a cacheable kernel mapping of all "normal" system memory, and having
both that mapping and another non-cacheable mapping of the same page can result in bad behaviour.
Quoted 11 lines
As for aliasing, yeah, having multiple aliases to the same piece of memory is a bad thing. The problem arises when devices on the PCI bus have memory allocated as device memory (which on Arm is non-cacheable and non-reorderable), but the PCI bus effectively acts as a write-combiner which changes the order of transactions. Therefore, for devices that have local memory associated with them (i.e. more than just register accesses) one should allocate memory in the first place that is Device-GRE (gathering, reordering and early-access). Otherwise, problems will surface that are not visible on x86 as that is a strongly ordered architecture.
PCI framebuffer BARs are mapped on the CPU with WC. We also use uncached WC mappings for system memory in cases where it's not likely we will be doing any CPU reads. When accessing system memory, the GPU can either do a CPU cache snooped transaction or a non-snooped transaction. The non-snooped transaction has lower latency and better throughput since it doesn't have to snoop the CPU cache.
Quoted 9 lines
Quoted 5 lines
Quoted 3 lines
Signed-off-by Alex Deucher <[email protected]>Reviewed-by Daniel Vetter <[email protected]>Given that this API is only used by AMD I'm OK for now with the change, but I think in general it is misleading and we should work towards fixing radeon and amd drivers.
Alternatively, we could just disable WC in the amdgpu driver on ARM. I'm not sure to what extent other drivers are using WC in general or have been tested on ARM.
Alex
Quoted 41 lines
Best regards,
LiviuQuoted 21 lines
Quoted 21 lines
--- include/drm/drm_cache.h | 2 ++ 1 file changed, 2 insertions(+)include/drm/drm_cache.h +2
-1@@ -47,6 +47,8 @@ static inline bool drm_arch_can_wc_memory(void)return false;#elif defined(CONFIG_MIPS ) && defined(CONFIG_CPU_LOONGSON3 )return false; +#elif defined(CONFIG_ARM ) || defined(CONFIG_ARM64 ) +return false;#else return true;#endif --2.13.6
Quoted 13 lines
Quoted 4 lines
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel--
====================
Quoted 7 lines
| I would like to |
| fix the world, |
| but they're not |
| giving me the |
\ source code! /
---------------
¯\_(ツ)_/¯_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): CADnq5_P9VZMhC_4j3F4hqHdD6Ey0=totPuNhF3hYBzkbgHsPpg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Quoted 62 lines · Alex Deucher
Quoted 18 lines · Liviu Dudau
Quoted 10 lines · Daniel Vetter
Quoted 5 lines · Alex Deucher
I'm not familiar enough with ARM to know if write combining is actually an architectural limitation or if it's an issue with the PCIe IPs used on various platforms, but so far everyone that has tried to run radeon hardware on ARM has had to disable it. So let's just make it official.
wc on arm is Really Complicated (tm) afaiui. There's issues with aliasing mappings and stuff, so you need to allocate your wc memory from special pools. So probably best to just disable it until we figure this out.
I believe both of you are conflating different issues under the wrong name. Write combining happens all the time with Arm, the ARMv8 architecture is a weakly-ordered model of memory so hardware is allowed to re-order or combine memory access as they seem fit.
Quoted 5 lines
A while ago I did run an AMD GPU card on my Juno dev board and it worked (for a very limited definition of worked, I've only validated the fact that I could get an fbcon and could run un-accelerated X11). So I would be interested if Alex could share some of the scenarios where people are seeing failures.
Here's an example:
https://bugs.freedesktop.org/show_bug.cgi?id=108625
But there are probably 5 or 6 other cases where people have emailed me or our team directly with issues on ARM resolved by disabling WC. Generally the driver seems to load ok, but then hangs as soon as you try and use acceleration from userspace or we end up with page flipping timeouts. Not really sure what the issue is. Michel suggested maybe ARM has a cacheable kernel mapping of all "normal" system memory, and having
both that mapping and another non-cacheable mapping of the same page can result in bad behaviour.Quoted 11 lines
As for aliasing, yeah, having multiple aliases to the same piece of memory is a bad thing. The problem arises when devices on the PCI bus have memory allocated as device memory (which on Arm is non-cacheable and non-reorderable), but the PCI bus effectively acts as a write-combiner which changes the order of transactions. Therefore, for devices that have local memory associated with them (i.e. more than just register accesses) one should allocate memory in the first place that is Device-GRE (gathering, reordering and early-access). Otherwise, problems will surface that are not visible on x86 as that is a strongly ordered architecture.
PCI framebuffer BARs are mapped on the CPU with WC. We also use uncached WC mappings for system memory in cases where it's not likely we will be doing any CPU reads. When accessing system memory, the GPU can either do a CPU cache snooped transaction or a non-snooped transaction. The non-snooped transaction has lower latency and better throughput since it doesn't have to snoop the CPU cache.
Quoted 6 lines
Quoted 5 lines
Quoted 3 lines
Signed-off-by Alex Deucher <[email protected]>Reviewed-by Daniel Vetter <daniel.vetter-/[email protected]>
Quoted 7 lines
Quoted 3 lines
Given that this API is only used by AMD I'm OK for now with the change, but I think in general it is misleading and we should work towards fixing radeon and amd drivers.
Alternatively, we could just disable WC in the amdgpu driver on ARM. I'm not sure to what extent other drivers are using WC in general or have been tested on ARM.
FWIW, I use WC mappings of BOs on V3D (shmem) and VC4 (cma). V3D is totally stable. VC4 I've heard reports of stability issues long-term but I don't think it's related. I don't do any cached mappings of my BOs, though.
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx