Skip to contentSkip to conversation

Skip IH re-route on Vega SR-IOV

PATCH In review Trigger Huang 2 participants 7 messages May 7, 2019 4 messages not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Skip IH re-route on Vega SR-IOV

IH re-route is not supported on Vega SR-IOV, need to be skipped

Trigger Huang (2):
drm/amdgpu: Skip IH reroute in Vega10 SR-IOV VF
drm/amdgpu: Skip IH reroute in Vega20 SR-IOV VF

 drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 4 ++++
 drivers/gpu/drm/amd/amdgpu/psp_v3_1.c  | 4 ++++
 2 files changed, 8 insertions(+)
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1557202115-24444-1-git-send-email-Trigger.Huang-5C7GfCeVMHo@public.gmane.org

#2 lore raw

PATCH 1/2 drm/amdgpu: Skip IH reroute in Vega10 SR-IOV VF

IH reroute commands are not supported on Vega10 VF

---
 drivers/gpu/drm/amd/amdgpu/psp_v3_1.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/gpu/drm/amd/amdgpu/psp_v3_1.c +4 -0
index 143f0fa..9d6e603 100644
@@ -260,6 +260,10 @@ static void psp_v3_1_reroute_ih(struct psp_context *psp)
 	struct amdgpu_device *adev = psp->adev;
 	uint32_t tmp;
 
+	/* reroute_ih is not supported on SR_IOV VF */
+	if (amdgpu_sriov_vf(adev))
+		return;
+
 	/* Change IH ring for VMC */
 	tmp = REG_SET_FIELD(0, IH_CLIENT_CFG_DATA, CREDIT_RETURN_ADDR, 0x1244b);
 	tmp = REG_SET_FIELD(tmp, IH_CLIENT_CFG_DATA, CLIENT_TYPE, 1);
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#3 lore raw

PATCH 2/2 drm/amdgpu: Skip IH reroute in Vega20 SR-IOV VF

In reply to a message not in the archive

IH reroute commands are not supported on Vega20 VF

---
 drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/gpu/drm/amd/amdgpu/psp_v11_0.c +4 -0
index b91df7b..4bdd70a 100644
@@ -232,6 +232,10 @@ static void psp_v11_0_reroute_ih(struct psp_context *psp)
 	struct amdgpu_device *adev = psp->adev;
 	uint32_t tmp;
 
+	/* reroute_ih is not supported on SR_IOV VF */
+	if (amdgpu_sriov_vf(adev))
+		return;
+
 	/* Change IH ring for VMC */
 	tmp = REG_SET_FIELD(0, IH_CLIENT_CFG_DATA, CREDIT_RETURN_ADDR, 0x1244b);
 	tmp = REG_SET_FIELD(tmp, IH_CLIENT_CFG_DATA, CLIENT_TYPE, 1);
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#4 lore raw
In reply to a message not in the archive

We intentionally didn't do this to make sure that the commands are ignored by the PSP firmware.

I have no strong opinion on if we should do this or not, but the PSP firmware guys might have.

Christian.

Am 07.05.19 um 06:08 schrieb Trigger Huang:

Quoted 10 lines

IH re-route is not supported on Vega SR-IOV, need to be skipped

Trigger Huang (2):
drm/amdgpu: Skip IH reroute in Vega10 SR-IOV VF drm/amdgpu: Skip IH reroute in Vega20 SR-IOV VF

  drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 4 ++++
  drivers/gpu/drm/amd/amdgpu/psp_v3_1.c  | 4 ++++
  2 files changed, 8 insertions(+)

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1b5b40b0-0d3c-8374-f423-5c50617fca9a-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#6 lore raw

Hi Christian,

On Vega10 SR-IOV VF, I injected a 'real' VMC page fault from user space, using the modified amdgpu_test.

[   19.127874] amdgpu 0000:00:08.0: [gfxhub] no-retry page fault (src_id:0 ring:174 vmid:1 pasid:32768, for process amdgpu_test pid 1071 thread amdgpu_test pid 1071)
[   19.130037] amdgpu 0000:00:08.0:   in page starting at address 0x0000000800000000 from 27

And see this interrupt is still from IH0 amdgpu_irq_handler, which can prove this feature is not working under SR-IOV.

I suggest to remove this feature from SR-IOV, as my concern is, some weird bugs may be cased by it in the Virtualization heavy stress test. In the future, maybe we can request PSP team to add this support for SR-IOV.

Thanks & Best Wishes,
Trigger Huang

-----Original Message-----
From: Christian König <[email protected]>
Sent: Tuesday, May 07, 2019 3:37 PM
To: Huang, Trigger <[email protected]>; [email protected]
Subject: Re: [PATCH 0/2] Skip IH re-route on Vega SR-IOV

[CAUTION: External Email]

We intentionally didn't do this to make sure that the commands are ignored by the PSP firmware.

I have no strong opinion on if we should do this or not, but the PSP firmware guys might have.

Christian.

Am 07.05.19 um 06:08 schrieb Trigger Huang:

Quoted 10 lines

IH re-route is not supported on Vega SR-IOV, need to be skipped

Trigger Huang (2):
drm/amdgpu: Skip IH reroute in Vega10 SR-IOV VF drm/amdgpu: Skip IH reroute in Vega20 SR-IOV VF

  drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 4 ++++
  drivers/gpu/drm/amd/amdgpu/psp_v3_1.c  | 4 ++++
  2 files changed, 8 insertions(+)

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): MN2PR12MB34084FE909CDC9AF45A6D624FE310-rweVpJHSKTo4ZTiO+qW6TgdYzm3356FpvxpqHgZTriW3zl9H0oFU5g@public.gmane.org

#8 lore raw

Hi Trigger,

Quoted 1 line

And see this interrupt is still from IH0 amdgpu_irq_handler, which can prove this feature is not working under SR-IOV.

In this case this change is a clear NAK.

Quoted 1 line

I suggest to remove this feature from SR-IOV, as my concern is, some weird bugs may be cased by it in the Virtualization heavy stress test.

And I really think we should keep it to make sure that we have the same handling for bare metal as for SRIOV.

Quoted 1 line

In the future, maybe we can request PSP team to add this support for SR-IOV.

We will never be able to use this under SRIOV because of limitation in the hardware.

What we could maybe do is check the response code from the PSP firmware if it correctly ignored the commands under SR-IOV, but I think the response code is the same for ignoring as for executing the commands.

Regards,
Christian.

Am 07.05.19 um 10:54 schrieb Huang, Trigger:

Quoted 35 lines

Hi Christian,

On Vega10 SR-IOV VF, I injected a 'real' VMC page fault from user space, using the modified amdgpu_test.

[   19.127874] amdgpu 0000:00:08.0: [gfxhub] no-retry page fault (src_id:0 ring:174 vmid:1 pasid:32768, for process amdgpu_test pid 1071 thread amdgpu_test pid 1071)
[   19.130037] amdgpu 0000:00:08.0:   in page starting at address 0x0000000800000000 from 27

And see this interrupt is still from IH0 amdgpu_irq_handler, which can prove this feature is not working under SR-IOV.

I suggest to remove this feature from SR-IOV, as my concern is, some weird bugs may be cased by it in the Virtualization heavy stress test. In the future, maybe we can request PSP team to add this support for SR-IOV.

Thanks & Best Wishes,
Trigger Huang

-----Original Message-----
From: Christian König <[email protected]>
Sent: Tuesday, May 07, 2019 3:37 PM
To: Huang, Trigger <[email protected]>; [email protected]
Subject: Re: [PATCH 0/2] Skip IH re-route on Vega SR-IOV

[CAUTION: External Email]

We intentionally didn't do this to make sure that the commands are ignored by the PSP firmware.

I have no strong opinion on if we should do this or not, but the PSP firmware guys might have.

Christian.

Am 07.05.19 um 06:08 schrieb Trigger Huang:

Quoted 6 lines

IH re-route is not supported on Vega SR-IOV, need to be skipped

Trigger Huang (2):
drm/amdgpu: Skip IH reroute in Vega10 SR-IOV VF drm/amdgpu: Skip IH reroute in Vega20 SR-IOV VF

Quoted 8 lines
Quoted 4 lines
   drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 4 ++++
   drivers/gpu/drm/amd/amdgpu/psp_v3_1.c  | 4 ++++
   2 files changed, 8 insertions(+)

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 3bc56691-9d61-8709-31a4-6ee98b79b397-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#10 lore raw

OK, thanks for the detailed background, before I didn't know the limitation in the hardware.

Thanks & Best Wishes,
Trigger Huang

-----Original Message-----
From: Christian König <[email protected]>
Sent: Tuesday, May 07, 2019 5:04 PM
To: Huang, Trigger <[email protected]>; Koenig, Christian <[email protected]>; [email protected]
Subject: Re: [PATCH 0/2] Skip IH re-route on Vega SR-IOV

[CAUTION: External Email]

Hi Trigger,

Quoted 1 line

And see this interrupt is still from IH0 amdgpu_irq_handler, which can prove this feature is not working under SR-IOV.

In this case this change is a clear NAK.

Quoted 1 line

I suggest to remove this feature from SR-IOV, as my concern is, some weird bugs may be cased by it in the Virtualization heavy stress test.

And I really think we should keep it to make sure that we have the same handling for bare metal as for SRIOV.

Quoted 1 line

In the future, maybe we can request PSP team to add this support for SR-IOV.

We will never be able to use this under SRIOV because of limitation in the hardware.

What we could maybe do is check the response code from the PSP firmware if it correctly ignored the commands under SR-IOV, but I think the response code is the same for ignoring as for executing the commands.

Regards,
Christian.

Am 07.05.19 um 10:54 schrieb Huang, Trigger:

Quoted 36 lines

Hi Christian,

On Vega10 SR-IOV VF, I injected a 'real' VMC page fault from user space, using the modified amdgpu_test.

[   19.127874] amdgpu 0000:00:08.0: [gfxhub] no-retry page fault (src_id:0 ring:174 vmid:1 pasid:32768, for process amdgpu_test pid 1071 thread amdgpu_test pid 1071)
[   19.130037] amdgpu 0000:00:08.0:   in page starting at address 0x0000000800000000 from 27

And see this interrupt is still from IH0 amdgpu_irq_handler, which can prove this feature is not working under SR-IOV.

I suggest to remove this feature from SR-IOV, as my concern is, some weird bugs may be cased by it in the Virtualization heavy stress test. In the future, maybe we can request PSP team to add this support for SR-IOV.

Thanks & Best Wishes,
Trigger Huang

-----Original Message-----
From: Christian König <[email protected]>
Sent: Tuesday, May 07, 2019 3:37 PM
To: Huang, Trigger <[email protected]>; [email protected]
Subject: Re: [PATCH 0/2] Skip IH re-route on Vega SR-IOV

[CAUTION: External Email]

We intentionally didn't do this to make sure that the commands are ignored by the PSP firmware.

I have no strong opinion on if we should do this or not, but the PSP firmware guys might have.

Christian.

Am 07.05.19 um 06:08 schrieb Trigger Huang:

Quoted 6 lines

IH re-route is not supported on Vega SR-IOV, need to be skipped

Trigger Huang (2):
drm/amdgpu: Skip IH reroute in Vega10 SR-IOV VF drm/amdgpu: Skip IH reroute in Vega20 SR-IOV VF

Quoted 8 lines
Quoted 4 lines
   drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 4 ++++
   drivers/gpu/drm/amd/amdgpu/psp_v3_1.c  | 4 ++++
   2 files changed, 8 insertions(+)

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx