drm/amdgpu: access stolen VRAM directly on KV/KB
Subsystems (2)
PATCH 1/2 drm/amdgpu: access stolen VRAM directly on KV/KB
From: Christian König <[email protected]>
We don't need to use the PCI BAR on APUs. This allows us to access the full VRAM directly without being limited by the BAR size.
--- drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c +8 -1
@@ -375,9 +375,16 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev)/* size in MB on si */ adev->mc.mc_vram_size =RREG32 (mmCONFIG_MEMSIZE) *1024ULL *1024ULL ; adev->mc.real_vram_size =RREG32 (mmCONFIG_MEMSIZE) *1024ULL *1024ULL ; - adev->mc.visible_vram_size = adev->mc.aper_size; + +#ifdef CONFIG_X86_64 +if (adev->flags &AMD_IS_APU ) { + adev->mc.aper_base =RREG32 (mmMC_VM_FB_OFFSET) <<22ULL ; + adev->mc.aper_size = adev->mc.real_vram_size; + } +#endif /* In case the PCI BAR is larger than the actual amount of vram */ + adev->mc.visible_vram_size = adev->mc.aper_size;if (adev->mc.visible_vram_size > adev->mc.real_vram_size) adev->mc.visible_vram_size = adev->mc.real_vram_size;
Signature · 6 lines
2.7.4 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 1478510481-1768-1-git-send-email-deathsimple-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org
PATCH 2/2 drm/amdgpu: access stolen VRAM directly on CZ
From: Christian König <[email protected]>
We don't need to use the PCI BAR on APUs. This allows us to access the full VRAM directly without being limited by the BAR size.
--- drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c +8 -1
@@ -462,9 +462,16 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev)/* size in MB on si */ adev->mc.mc_vram_size =RREG32 (mmCONFIG_MEMSIZE) *1024ULL *1024ULL ; adev->mc.real_vram_size =RREG32 (mmCONFIG_MEMSIZE) *1024ULL *1024ULL ; - adev->mc.visible_vram_size = adev->mc.aper_size; + +#ifdef CONFIG_X86_64 +if (adev->flags &AMD_IS_APU ) { + adev->mc.aper_base =RREG32 (mmMC_VM_FB_OFFSET) <<22ULL ; + adev->mc.aper_size = adev->mc.real_vram_size; + } +#endif /* In case the PCI BAR is larger than the actual amount of vram */ + adev->mc.visible_vram_size = adev->mc.aper_size;if (adev->mc.visible_vram_size > adev->mc.real_vram_size) adev->mc.visible_vram_size = adev->mc.real_vram_size;
Signature · 6 lines
2.7.4 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 1478510481-1768-2-git-send-email-deathsimple-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org
Re: [PATCH 2/2] drm/amdgpu: access stolen VRAM directly on CZ
Quoted 4 lines · Christian König
From: Christian König <[email protected]>
We don't need to use the PCI BAR on APUs. This allows us to access the full VRAM directly without being limited by the BAR size.
I'm feeling coherency issues, has this approach been validated with the hw team?
Dave.
Quoted 31 lines
Signed-off-by Christian König <[email protected]>--- drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c +8
-2@@ -462,9 +462,16 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev)/* size in MB on si */ adev->mc.mc_vram_size =RREG32 (mmCONFIG_MEMSIZE) *1024ULL *1024ULL ; adev->mc.real_vram_size =RREG32 (mmCONFIG_MEMSIZE) *1024ULL *1024ULL ; - adev->mc.visible_vram_size = adev->mc.aper_size; + +#ifdef CONFIG_X86_64 +if (adev->flags &AMD_IS_APU ) { + adev->mc.aper_base =RREG32 (mmMC_VM_FB_OFFSET) <<22ULL ; + adev->mc.aper_size = adev->mc.real_vram_size; + } +#endif /* In case the PCI BAR is larger than the actual amount of vram */ + adev->mc.visible_vram_size = adev->mc.aper_size;if (adev->mc.visible_vram_size > adev->mc.real_vram_size) adev->mc.visible_vram_size = adev->mc.real_vram_size; --2.7.4
Quoted 4 lines
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): CAPM=9twmcyE4fOzZXJexdfViNx32vkc39sciQ3Ha+7UhetJJKA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Re: [PATCH 2/2] drm/amdgpu: access stolen VRAM directly on CZ
Quoted 7 lines · Dave Airlie
Quoted 4 lines · Christian König
From: Christian König <[email protected]>
We don't need to use the PCI BAR on APUs. This allows us to access the full VRAM directly without being limited by the BAR size.
I'm feeling coherency issues, has this approach been validated with the hw team?
I raised this concern internally as well, but the response indicated that it should be fine. Apparently x86 defines that write-combined writes complete before any access to UC memory (such as writing to an MMIO register or doorbell to submit a GPU operation).
Signature · 6 lines
Earthling Michel Dänzer | http://www.amd.com Libre software enthusiast | Mesa and X developer _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Michel Dänzer
Reviewed-byQuoted 6 lines · Christian König
From: Christian König <[email protected]>
We don't need to use the PCI BAR on APUs. This allows us to access the full VRAM directly without being limited by the BAR size.
Signed-off-by Christian König <[email protected]>
The series is
We could also drop the HDP flushes / invalidates in this case, right?
Signature · 6 lines
Earthling Michel Dänzer | http://www.amd.com Libre software enthusiast | Mesa and X developer _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 9d218698-d588-090a-eb47-d0142cfdb623-otUistvHUpPR7s880joybQ@public.gmane.org
Am 07.11.2016 um 10:29 schrieb Michel Dänzer:
Quoted 6 lines · Christian König
Quoted 5 lines
From: Christian König <[email protected]>
We don't need to use the PCI BAR on APUs. This allows us to access the full VRAM directly without being limited by the BAR size.
Quoted 7 lines
Quoted 1 line
Signed-off-by Christian König <[email protected]>The series is
Reviewed-by Michel Dänzer <[email protected]>We could also drop the HDP flushes / invalidates in this case, right?
Could be, but I'm not 100% sure how those are actually wired up on APUs.
Might be that they are NOPs anyway, but could be that at least the flushes are still vital.
Regards,
Christian.
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 7be5b56d-fae3-d71b-485f-01227172859d-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org
Quoted 10 lines · Christian König
Am 07.11.2016 um 10:29 schrieb Michel Dänzer:
Quoted 9 lines · Christian König
Quoted 6 lines
From: Christian König <[email protected]>
We don't need to use the PCI BAR on APUs. This allows us to access the full VRAM directly without being limited by the BAR size.
Signed-off-by Christian König <[email protected]>The series is
Quoted 8 lines
Quoted 4 lines
Reviewed-by Michel Dänzer <[email protected]>We could also drop the HDP flushes / invalidates in this case, right?
Could be, but I'm not 100% sure how those are actually wired up on APUs.
Might be that they are NOPs anyway, but could be that at least the flushes are still vital.
My assumption is that since CPU access to VRAM doesn't go through the HDP at all with this change, HDP flushes / invalidates can't have any useful effect.
Signature · 6 lines
Earthling Michel Dänzer | http://www.amd.com Libre software enthusiast | Mesa and X developer _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 8372afd6-8a51-1102-99a3-28a719088fa6-otUistvHUpPR7s880joybQ@public.gmane.org
Am 08.11.2016 um 08:31 schrieb Michel Dänzer:
Quoted 14 lines · Christian König
Quoted 13 lines
Am 07.11.2016 um 10:29 schrieb Michel Dänzer:
Quoted 12 lines · Christian König
Quoted 6 lines
From: Christian König <[email protected]>
We don't need to use the PCI BAR on APUs. This allows us to access the full VRAM directly without being limited by the BAR size.
Signed-off-by Christian König <[email protected]>The series is
Reviewed-by Michel Dänzer <[email protected]>
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
We could also drop the HDP flushes / invalidates in this case, right?
Could be, but I'm not 100% sure how those are actually wired up on APUs.
Might be that they are NOPs anyway, but could be that at least the flushes are still vital.
My assumption is that since CPU access to VRAM doesn't go through the HDP at all with this change, HDP flushes / invalidates can't have any useful effect.
Well writes to GTT still use the HDP if I understand it correctly.
So we still need to handle either flushes or invalidations (I doesn't of hand remember which was that).
Regards,
Christian.
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): f04a0ad6-e2cb-98c9-cb09-8af3218b9dca-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org
Quoted 20 lines · Christian König
Am 08.11.2016 um 08:31 schrieb Michel Dänzer:
Quoted 19 lines · Christian König
Quoted 18 lines
Am 07.11.2016 um 10:29 schrieb Michel Dänzer:
Quoted 6 lines · Christian König
Quoted 5 lines
From: Christian König <[email protected]>
We don't need to use the PCI BAR on APUs. This allows us to access the full VRAM directly without being limited by the BAR size.
Quoted 7 lines
Quoted 1 line
Signed-off-by Christian König <[email protected]>The series is
Reviewed-by Michel Dänzer <[email protected]>We could also drop the HDP flushes / invalidates in this case, right?
Could be, but I'm not 100% sure how those are actually wired up on APUs.
Might be that they are NOPs anyway, but could be that at least the flushes are still vital.
Quoted 8 lines
Quoted 3 lines
My assumption is that since CPU access to VRAM doesn't go through the HDP at all with this change, HDP flushes / invalidates can't have any useful effect.
Well writes to GTT still use the HDP if I understand it correctly.
So we still need to handle either flushes or invalidations (I doesn't of hand remember which was that).
Do you mean GPU access to GTT? If so, not sure why it would be writes only?
Signature · 6 lines
Earthling Michel Dänzer | http://www.amd.com Libre software enthusiast | Mesa and X developer _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 930d7e26-a894-c119-e5ec-b6dc1dae1ea4-otUistvHUpPR7s880joybQ@public.gmane.org
Quoted 35 lines
-----Original Message-----
From: amd-gfx [mailto:[email protected]] On Behalf Of Michel Dänzer
Sent: Tuesday, November 08, 2016 4:37 AM
To: Christian König
Cc: [email protected]
Subject: Re: [PATCH 1/2] drm/amdgpu: access stolen VRAM directly on KV/KBQuoted 17 lines · Christian König
Am 08.11.2016 um 08:31 schrieb Michel Dänzer:
Quoted 16 lines · Christian König
Quoted 15 lines
Am 07.11.2016 um 10:29 schrieb Michel Dänzer:
Quoted 6 lines · Christian König
Quoted 5 lines
From: Christian König <[email protected]>
We don't need to use the PCI BAR on APUs. This allows us to access the full VRAM directly without being limited by the BAR size.
Quoted 7 lines
Quoted 1 line
Signed-off-by Christian König <[email protected]>The series is
Reviewed-by Michel Dänzer <[email protected]>We could also drop the HDP flushes / invalidates in this case, right?
Could be, but I'm not 100% sure how those are actually wired up on
APUs.
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
Might be that they are NOPs anyway, but could be that at least the flushes are still vital.
My assumption is that since CPU access to VRAM doesn't go through the HDP at all with this change, HDP flushes / invalidates can't have any useful effect.
Quoted 6 lines
Quoted 4 lines
Well writes to GTT still use the HDP if I understand it correctly.
So we still need to handle either flushes or invalidations (I doesn't of hand remember which was that).
Do you mean GPU access to GTT? If so, not sure why it would be writes only?
The HDP has a read cache and a write queue. It only affects access via the BAR (either the CPU or another PCIe device). AFAIK, the APU access would only use the HDP if you mapped the BAR via the GART.
Alex
Quoted 9 lines
--
Earthling Michel Dänzer | http://www.amd.com
Libre software enthusiast | Mesa and X developer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): MWHPR12MB16944B5018FC8CFF710EF3FFF7A60-Gy0DoCVfaSW4WA4dJ5YXGAdYzm3356FpvxpqHgZTriW3zl9H0oFU5g@public.gmane.org
Am 08.11.2016 um 14:36 schrieb Deucher, Alexander:
Quoted 33 lines
Quoted 33 lines
-----Original Message-----
From: amd-gfx [mailto:[email protected]] On Behalf Of Michel Dänzer
Sent: Tuesday, November 08, 2016 4:37 AM
To: Christian König
Cc: [email protected]
Subject: Re: [PATCH 1/2] drm/amdgpu: access stolen VRAM directly on KV/KBQuoted 17 lines · Christian König
Am 08.11.2016 um 08:31 schrieb Michel Dänzer:
Quoted 16 lines · Christian König
Quoted 15 lines
Am 07.11.2016 um 10:29 schrieb Michel Dänzer:
Quoted 6 lines · Christian König
Quoted 5 lines
From: Christian König <[email protected]>
We don't need to use the PCI BAR on APUs. This allows us to access the full VRAM directly without being limited by the BAR size.
Quoted 7 lines
Quoted 1 line
Signed-off-by Christian König <[email protected]>The series is
Reviewed-by Michel Dänzer <[email protected]>We could also drop the HDP flushes / invalidates in this case, right?
Could be, but I'm not 100% sure how those are actually wired up on
APUs.
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
Might be that they are NOPs anyway, but could be that at least the flushes are still vital.
My assumption is that since CPU access to VRAM doesn't go through the HDP at all with this change, HDP flushes / invalidates can't have any useful effect.
Quoted 6 lines
Quoted 5 lines
Quoted 4 lines
Well writes to GTT still use the HDP if I understand it correctly.
So we still need to handle either flushes or invalidations (I doesn't of hand remember which was that).
Do you mean GPU access to GTT? If so, not sure why it would be writes only?
The HDP has a read cache and a write queue. It only affects access via the BAR (either the CPU or another PCIe device). AFAIK, the APU access would only use the HDP if you mapped the BAR via the GART.
Ok, my understanding was that the HDP was responsible for both directions. E.g. writes/reads from the CPU side to the BAR as well as writes/reads from the GPU to system memory.
Since this needs intense testing anyway I will just wait for the first two patches to land, wait for a release to make sure they don't cause problems and then disable HDP flushes on APUs as well.
Christian.
Quoted 11 lines
Alex
Quoted 8 lines
--
Earthling Michel Dänzer | http://www.amd.com
Libre software enthusiast | Mesa and X developer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Quoted 4 lines
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
As a next step, you could also remove HDP flushing on APUs.
Regards,
Felix
Quoted 22 lines · Christian König
From: Christian König <[email protected]>
We don't need to use the PCI BAR on APUs. This allows us to access the full VRAM directly without being limited by the BAR size.
Signed-off-by Christian König <[email protected]>--- drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c +3
-1@@ -375,9 +375,16 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev)/* size in MB on si */ adev->mc.mc_vram_size =RREG32 (mmCONFIG_MEMSIZE) *1024ULL *1024ULL ; adev->mc.real_vram_size =RREG32 (mmCONFIG_MEMSIZE) *1024ULL *1024ULL ; - adev->mc.visible_vram_size = adev->mc.aper_size; + +#ifdef CONFIG_X86_64 +if (adev->flags &AMD_IS_APU ) {
drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c static int gmc_v7_0_mc_init(struct amdgpu_device *adev) · line 390
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c +5
-0+ adev->mc.aper_base =RREG32 (mmMC_VM_FB_OFFSET) <<22ULL ; + adev->mc.aper_size = adev->mc.real_vram_size; + } +#endif /* In case the PCI BAR is larger than the actual amount of vram */ + adev->mc.visible_vram_size = adev->mc.aper_size;if (adev->mc.visible_vram_size > adev->mc.real_vram_size) adev->mc.visible_vram_size = adev->mc.real_vram_size;
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx