Skip to contentSkip to conversation

drm/amdgpu: Return -EINVAL when whole gpu reset happened

PATCH NAK Liu ChengZhe 3 participants 5 messages Dec 9, 2020 – Dec 10, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/amdgpu: Return -EINVAL when whole gpu reset happened

If CS init return -ECANCELED, UMD will free and create new context. Job in this new context could conitnue exexcuting. In the case of BACO or mode 1, we can't allow this happpen. Because VRAM has lost after whole gpu reset, the job can't guarantee to succeed.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +7 -2
index 85e48c29a57c..2a98f58134ed 100644
@@ -120,6 +120,7 @@ static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs
 	uint64_t *chunk_array;
 	unsigned size, num_ibs = 0;
 	uint32_t uf_offset = 0;
+	uint32_t vramlost_count = 0;
 	int i;
 	int ret;
 
@@ -140,7 +141,11 @@ static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs
 
 	/* skip guilty context job */
 	if (atomic_read(&p->ctx->guilty) == 1) {
-		ret = -ECANCELED;
+		vramlost_count = atomic_read(&p->adev->vram_lost_counter);
+		if (p->ctx->vram_lost_counter != vramlost_count)
+			ret = -EINVAL;
+		else
+			ret = -ECANCELED;
 		goto free_chunk;
 	}
 
@@ -246,7 +251,7 @@ static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs
 		goto free_all_kdata;
 
 	if (p->ctx->vram_lost_counter != p->job->vram_lost_counter) {
-		ret = -ECANCELED;
+		ret = -EINVAL;
 		goto free_all_kdata;
 	}
 
Signature · 6 lines
2.25.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
maintainer NAK
#1 lore raw

Am 09.12.20 um 10:46 schrieb Liu ChengZhe:

Quoted 4 lines

If CS init return -ECANCELED, UMD will free and create new context. Job in this new context could conitnue exexcuting. In the case of BACO or mode 1, we can't allow this happpen. Because VRAM has lost after whole gpu reset, the job can't guarantee to succeed.

NAK, this is intentional.

When ECANCELED is returned UMD should create new context after a GPU reset to get back into an usable state and continue to submit jobs.

Regards,
Christian.

Quoted 32 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 9 +++++++--
  1 file changed, 7 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +6 -1
index 85e48c29a57c..2a98f58134ed 100644
@@ -120,6 +120,7 @@ static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs
  	uint64_t *chunk_array;
  	unsigned size, num_ibs = 0;
  	uint32_t uf_offset = 0;
+	uint32_t vramlost_count = 0;
  	int i;
  	int ret;
 
@@ -140,7 +141,11 @@ static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs
 
  	/* skip guilty context job */
  	if (atomic_read(&p->ctx->guilty) == 1) {
-		ret = -ECANCELED;
+		vramlost_count = atomic_read(&p->adev->vram_lost_counter);
+		if (p->ctx->vram_lost_counter != vramlost_count)
+			ret = -EINVAL;
+		else
+			ret = -ECANCELED;
  		goto free_chunk;
  	}
 

drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs · line 257

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +1 -1
@@ -246,7 +251,7 @@ static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs
  		goto free_all_kdata;
 
  	if (p->ctx->vram_lost_counter != p->job->vram_lost_counter) {
-		ret = -ECANCELED;
+		ret = -EINVAL;
  		goto free_all_kdata;
  	}
 

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

author NAK
#2 lore raw

[AMD Public Use]

Yeah, we discussed this issue again. We think it's better UMD make some changes instead of changing in KMD. If FLR happened, it's OK for UMD create new context and continue to submit jobs. However, if BACO or mode 1 reset happens, of course UMD could also submit jobs, but these jobs can't use any resource create before the reset, including page table. Because all the contents in VRAM has lost after BACO or mode 1 reset, which including APP's buffer.

-----Original Message-----
From: Koenig, Christian <[email protected]>
Sent: Wednesday, December 9, 2020 6:06 PM
To: Liu, Cheng Zhe <[email protected]>; [email protected]
Cc: Tuikov, Luben <[email protected]>; Deucher, Alexander <[email protected]>; Xiao, Jack <[email protected]>; Zhang, Hawking <[email protected]>; Xu, Feifei <[email protected]>; Wang, Kevin(Yang) <[email protected]>; Yuan, Xiaojie <[email protected]>
Subject: Re: [PATCH] drm/amdgpu: Return -EINVAL when whole gpu reset happened

Am 09.12.20 um 10:46 schrieb Liu ChengZhe:

Quoted 4 lines

If CS init return -ECANCELED, UMD will free and create new context. Job in this new context could conitnue exexcuting. In the case of BACO or mode 1, we can't allow this happpen. Because VRAM has lost after whole gpu reset, the job can't guarantee to succeed.

NAK, this is intentional.

When ECANCELED is returned UMD should create new context after a GPU reset to get back into an usable state and continue to submit jobs.

Regards,
Christian.

Quoted 34 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 9 +++++++--
  1 file changed, 7 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +1 -0
index 85e48c29a57c..2a98f58134ed 100644
@@ -120,6 +120,7 @@ static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs
  	uint64_t *chunk_array;
  	unsigned size, num_ibs = 0;
  	uint32_t uf_offset = 0;
+	uint32_t vramlost_count = 0;
  	int i;
  	int ret;
 
@@ -140,7 +141,11 @@ static int amdgpu_cs_parser_init(struct 

amdgpu_cs_parser *p, union drm_amdgpu_cs

  	/* skip guilty context job */
  	if (atomic_read(&p->ctx->guilty) == 1) {
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +5 -1
-		ret = -ECANCELED;
+		vramlost_count = atomic_read(&p->adev->vram_lost_counter);
+		if (p->ctx->vram_lost_counter != vramlost_count)
+			ret = -EINVAL;
+		else
+			ret = -ECANCELED;
  		goto free_chunk;
  	}

drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs · line 257

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +1 -1
@@ -246,7 +251,7 @@ static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs
  		goto free_all_kdata;
 
  	if (p->ctx->vram_lost_counter != p->job->vram_lost_counter) {
-		ret = -ECANCELED;
+		ret = -EINVAL;
  		goto free_all_kdata;
  	}
 

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#3 lore raw

Hi Cheng,

well with whom you discussed that? Such discussions should happen on the mailing list and at minimum include Alex and me.

Anyway as discussed before this is not acceptable. We already discussed this years ago and the behavior is fixed and not changeable.

Regards,
Christian.

Am 10.12.20 um 03:25 schrieb Liu, Cheng Zhe:

Quoted 60 lines

[AMD Public Use]

Yeah, we discussed this issue again. We think it's better UMD make some changes instead of changing in KMD. If FLR happened, it's OK for UMD create new context and continue to submit jobs. However, if BACO or mode 1 reset happens, of course UMD could also submit jobs, but these jobs can't use any resource create before the reset, including page table. Because all the contents in VRAM has lost after BACO or mode 1 reset, which including APP's buffer.

-----Original Message-----
From: Koenig, Christian <[email protected]>
Sent: Wednesday, December 9, 2020 6:06 PM
To: Liu, Cheng Zhe <[email protected]>; [email protected]
Cc: Tuikov, Luben <[email protected]>; Deucher, Alexander <[email protected]>; Xiao, Jack <[email protected]>; Zhang, Hawking <[email protected]>; Xu, Feifei <[email protected]>; Wang, Kevin(Yang) <[email protected]>; Yuan, Xiaojie <[email protected]>
Subject: Re: [PATCH] drm/amdgpu: Return -EINVAL when whole gpu reset happened

Am 09.12.20 um 10:46 schrieb Liu ChengZhe:

Quoted 4 lines

If CS init return -ECANCELED, UMD will free and create new context. Job in this new context could conitnue exexcuting. In the case of BACO or mode 1, we can't allow this happpen. Because VRAM has lost after whole gpu reset, the job can't guarantee to succeed.

NAK, this is intentional.

When ECANCELED is returned UMD should create new context after a GPU reset to get back into an usable state and continue to submit jobs.

Regards,
Christian.

Quoted 35 lines
---
   drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 9 +++++++--
   1 file changed, 7 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +1 -0
index 85e48c29a57c..2a98f58134ed 100644
@@ -120,6 +120,7 @@ static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs
   	uint64_t *chunk_array;
   	unsigned size, num_ibs = 0;
   	uint32_t uf_offset = 0;
+	uint32_t vramlost_count = 0;
   	int i;
   	int ret;
 
@@ -140,7 +141,11 @@ static int amdgpu_cs_parser_init(struct

amdgpu_cs_parser *p, union drm_amdgpu_cs

   	/* skip guilty context job */
   	if (atomic_read(&p->ctx->guilty) == 1) {
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +5 -1
-		ret = -ECANCELED;
+		vramlost_count = atomic_read(&p->adev->vram_lost_counter);
+		if (p->ctx->vram_lost_counter != vramlost_count)
+			ret = -EINVAL;
+		else
+			ret = -ECANCELED;
   		goto free_chunk;
   	}
@@ -246,7 +251,7 @@ static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs
   		goto free_all_kdata;
 
Quoted 10 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs · line 257

Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +1 -1
   	if (p->ctx->vram_lost_counter != p->job->vram_lost_counter) {
-		ret = -ECANCELED;
+		ret = -EINVAL;
   		goto free_all_kdata;
   	}
 

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#4 lore raw

[AMD Public Use]

Hi Christian,

I discussed with @Liu, Monk and @Chen, Horace, it hasn't decided yet. I'll add you and Alex to the mailing list.

-----Original Message-----
From: Christian König <[email protected]>
Sent: Thursday, December 10, 2020 5:29 PM
To: Liu, Cheng Zhe <[email protected]>; Koenig, Christian <[email protected]>; [email protected]
Cc: Xiao, Jack <[email protected]>; Xu, Feifei <[email protected]>; Wang, Kevin(Yang) <[email protected]>; Tuikov, Luben <[email protected]>; Deucher, Alexander <[email protected]>; Yuan, Xiaojie <[email protected]>; Zhang, Hawking <[email protected]>
Subject: Re: [PATCH] drm/amdgpu: Return -EINVAL when whole gpu reset happened

Hi Cheng,

well with whom you discussed that? Such discussions should happen on the mailing list and at minimum include Alex and me.

Anyway as discussed before this is not acceptable. We already discussed this years ago and the behavior is fixed and not changeable.

Regards,
Christian.

Am 10.12.20 um 03:25 schrieb Liu, Cheng Zhe:

Quoted 66 lines

[AMD Public Use]

Yeah, we discussed this issue again. We think it's better UMD make some changes instead of changing in KMD. If FLR happened, it's OK for UMD create new context and continue to submit jobs. However, if BACO or mode 1 reset happens, of course UMD could also submit jobs, but these jobs can't use any resource create before the reset, including page table. Because all the contents in VRAM has lost after BACO or mode 1 reset, which including APP's buffer.

-----Original Message-----
From: Koenig, Christian <[email protected]>
Sent: Wednesday, December 9, 2020 6:06 PM
To: Liu, Cheng Zhe <[email protected]>; [email protected]
Cc: Tuikov, Luben <[email protected]>; Deucher, Alexander <[email protected]>; Xiao, Jack <[email protected]>; Zhang, Hawking <[email protected]>; Xu, Feifei <[email protected]>; Wang, Kevin(Yang) <[email protected]>; Yuan, Xiaojie <[email protected]>
Subject: Re: [PATCH] drm/amdgpu: Return -EINVAL when whole gpu reset happened

Am 09.12.20 um 10:46 schrieb Liu ChengZhe:

Quoted 4 lines

If CS init return -ECANCELED, UMD will free and create new context. Job in this new context could conitnue exexcuting. In the case of BACO or mode 1, we can't allow this happpen. Because VRAM has lost after whole gpu reset, the job can't guarantee to succeed.

NAK, this is intentional.

When ECANCELED is returned UMD should create new context after a GPU reset to get back into an usable state and continue to submit jobs.

Regards,
Christian.

Quoted 35 lines
---
   drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 9 +++++++--
   1 file changed, 7 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +1 -0
index 85e48c29a57c..2a98f58134ed 100644
@@ -120,6 +120,7 @@ static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs
   	uint64_t *chunk_array;
   	unsigned size, num_ibs = 0;
   	uint32_t uf_offset = 0;
+	uint32_t vramlost_count = 0;
   	int i;
   	int ret;
 
@@ -140,7 +141,11 @@ static int amdgpu_cs_parser_init(struct 

amdgpu_cs_parser *p, union drm_amdgpu_cs

   	/* skip guilty context job */
   	if (atomic_read(&p->ctx->guilty) == 1) {
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +5 -1
-		ret = -ECANCELED;
+		vramlost_count = atomic_read(&p->adev->vram_lost_counter);
+		if (p->ctx->vram_lost_counter != vramlost_count)
+			ret = -EINVAL;
+		else
+			ret = -ECANCELED;
   		goto free_chunk;
   	}
@@ -246,7 +251,7 @@ static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs
   		goto free_all_kdata;
 
Quoted 16 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs · line 257

Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +1 -1
   	if (p->ctx->vram_lost_counter != p->job->vram_lost_counter) {
-		ret = -ECANCELED;
+		ret = -EINVAL;
   		goto free_all_kdata;
   	}
 

_______________________________________________
amd-gfx mailing list
[email protected]
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flist
s.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfx&amp;data=04%7C01%7CCh engZhe.Liu%40amd.com%7C56e4519b79184cdab3e508d89cedf8ba%7C3dd8961fe488 4e608e11a82d994e183d%7C0%7C0%7C637431893767203117%7CUnknown%7CTWFpbGZs b3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D %7C3000&amp;sdata=sQIc%2FhxC%2BotX49yD7Vhw5FfxL96Nz4IPAkcXMUVZS6c%3D&a mp;reserved=0

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx