Skip to contentSkip to conversation

drm/amdgpu: fix missed gpu info firmware when cache firmware during S3

PATCH In review Huang Rui 4 participants 9 messages Jun 6, 2017 5 messages not in the archive

Subsystems (2)
lore
Share & tools
lore

Huang Rui

author
#0 lore raw

PATCH 1/1 drm/amdgpu: fix missed gpu info firmware when cache firmware during S3

gpu_info firmware is released after data is used. But when system enters into suspend, upper class driver will cache all firmware names. At that time, gpu_info will be failing to load. It seems an upper class issue, that we should not release gpu_info firmware until device finished.

[  903.236589] cache_firmware: amdgpu/vega10_sdma1.bin
[  903.236590] fw_set_page_data: fw-amdgpu/vega10_sdma1.bin buf=ffff88041eee10c0 data=ffffc90002561000 size=17408
[  903.236591] cache_firmware: amdgpu/vega10_sdma1.bin ret=0
[  903.464160] __allocate_fw_buf: fw-amdgpu/vega10_gpu_info.bin buf=ffff88041eee2c00
[  903.471815] (NULL device *): loading /lib/firmware/updates/4.11.0-custom/amdgpu/vega10_gpu_info.bin failed with error -2
[  903.482870] (NULL device *): loading /lib/firmware/updates/amdgpu/vega10_gpu_info.bin failed with error -2
[  903.492716] (NULL device *): loading /lib/firmware/4.11.0-custom/amdgpu/vega10_gpu_info.bin failed with error -2
[  903.503156] (NULL device *): direct-loading amdgpu/vega10_gpu_info.bin
---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  3 +++
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 18 ++++++++++--------
 2 files changed, 13 insertions(+), 8 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +3 -0
index 8658643..54ee050 100644
@@ -1272,6 +1272,9 @@ struct amdgpu_firmware {
 	const struct amdgpu_psp_funcs *funcs;
 	struct amdgpu_bo *rbuf;
 	struct mutex mutex;
+
+	/* gpu info firmware data pointer */
+	const struct firmware *fw;
 };
 
 /*
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +10 -8
index 6883fe1..af8f8b3 100644
@@ -1426,12 +1426,13 @@ static void amdgpu_device_enable_virtual_display(struct amdgpu_device *adev)
 
 static int amdgpu_device_parse_gpu_info_fw(struct amdgpu_device *adev)
 {
-	const struct firmware *fw;
 	const char *chip_name;
 	char fw_name[30];
 	int err;
 	const struct gpu_info_firmware_header_v1_0 *hdr;
 
+	adev->firmware.fw = NULL;
+
 	switch (adev->asic_type) {
 	case CHIP_TOPAZ:
 	case CHIP_TONGA:
@@ -1466,14 +1467,14 @@ static int amdgpu_device_parse_gpu_info_fw(struct amdgpu_device *adev)
 	}
 
 	snprintf(fw_name, sizeof(fw_name), "amdgpu/%s_gpu_info.bin", chip_name);
-	err = request_firmware(&fw, fw_name, adev->dev);
+	err = request_firmware(&adev->firmware.fw, fw_name, adev->dev);
 	if (err) {
 		dev_err(adev->dev,
 			"Failed to load gpu_info firmware \"%s\"\n",
 			fw_name);
 		goto out;
 	}
-	err = amdgpu_ucode_validate(fw);
+	err = amdgpu_ucode_validate(adev->firmware.fw);
 	if (err) {
 		dev_err(adev->dev,
 			"Failed to validate gpu_info firmware \"%s\"\n",
@@ -1481,14 +1482,14 @@ static int amdgpu_device_parse_gpu_info_fw(struct amdgpu_device *adev)
 		goto out;
 	}
 
-	hdr = (const struct gpu_info_firmware_header_v1_0 *)fw->data;
+	hdr = (const struct gpu_info_firmware_header_v1_0 *)adev->firmware.fw->data;
 	amdgpu_ucode_print_gpu_info_hdr(&hdr->header);
 
 	switch (hdr->version_major) {
 	case 1:
 	{
 		const struct gpu_info_firmware_v1_0 *gpu_info_fw =
-			(const struct gpu_info_firmware_v1_0 *)(fw->data +
+			(const struct gpu_info_firmware_v1_0 *)(adev->firmware.fw->data +
 								le32_to_cpu(hdr->header.ucode_array_offset_bytes));
 
 		adev->gfx.config.max_shader_engines = gpu_info_fw->gc_num_se;
@@ -1513,9 +1514,6 @@ static int amdgpu_device_parse_gpu_info_fw(struct amdgpu_device *adev)
 		goto out;
 	}
 out:
-	release_firmware(fw);
-	fw = NULL;
-
 	return err;
 }
 
@@ -2313,6 +2311,10 @@ void amdgpu_device_fini(struct amdgpu_device *adev)
 	amdgpu_fence_driver_fini(adev);
 	amdgpu_fbdev_fini(adev);
 	r = amdgpu_fini(adev);
+	if (adev->firmware.fw) {
+		release_firmware(adev->firmware.fw);
+		adev->firmware.fw = NULL;
+	}
 	adev->accel_working = false;
 	/* free i2c buses */
 	if (!amdgpu_device_has_dc_support(adev))
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#2 lore raw
In reply to a message not in the archive

Hi Ray,

mhm, indeed a nice catch.

But why do we need to load the gpu info after resume in the first place?

I mean we already know what GPU we have, loading it again looks superfluous to me.

Regards,
Christian.

Am 06.06.2017 um 08:24 schrieb Huang Rui:

Quoted 98 lines

gpu_info firmware is released after data is used. But when system enters into suspend, upper class driver will cache all firmware names. At that time, gpu_info will be failing to load. It seems an upper class issue, that we should not release gpu_info firmware until device finished.

[  903.236589] cache_firmware: amdgpu/vega10_sdma1.bin
[  903.236590] fw_set_page_data: fw-amdgpu/vega10_sdma1.bin buf=ffff88041eee10c0 data=ffffc90002561000 size=17408
[  903.236591] cache_firmware: amdgpu/vega10_sdma1.bin ret=0
[  903.464160] __allocate_fw_buf: fw-amdgpu/vega10_gpu_info.bin buf=ffff88041eee2c00
[  903.471815] (NULL device *): loading /lib/firmware/updates/4.11.0-custom/amdgpu/vega10_gpu_info.bin failed with error -2
[  903.482870] (NULL device *): loading /lib/firmware/updates/amdgpu/vega10_gpu_info.bin failed with error -2
[  903.492716] (NULL device *): loading /lib/firmware/4.11.0-custom/amdgpu/vega10_gpu_info.bin failed with error -2
[  903.503156] (NULL device *): direct-loading amdgpu/vega10_gpu_info.bin
---
  drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  3 +++
  drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 18 ++++++++++--------
  2 files changed, 13 insertions(+), 8 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +3 -0
index 8658643..54ee050 100644
@@ -1272,6 +1272,9 @@ struct amdgpu_firmware {
  	const struct amdgpu_psp_funcs *funcs;
  	struct amdgpu_bo *rbuf;
  	struct mutex mutex;
+
+	/* gpu info firmware data pointer */
+	const struct firmware *fw;
  };
 
  /*
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +6 -8
index 6883fe1..af8f8b3 100644
@@ -1426,12 +1426,13 @@ static void amdgpu_device_enable_virtual_display(struct amdgpu_device *adev)
 
  static int amdgpu_device_parse_gpu_info_fw(struct amdgpu_device *adev)
  {
-	const struct firmware *fw;
  	const char *chip_name;
  	char fw_name[30];
  	int err;
  	const struct gpu_info_firmware_header_v1_0 *hdr;
 
+	adev->firmware.fw = NULL;
+
  	switch (adev->asic_type) {
  	case CHIP_TOPAZ:
  	case CHIP_TONGA:
@@ -1466,14 +1467,14 @@ static int amdgpu_device_parse_gpu_info_fw(struct amdgpu_device *adev)
  	}
 
  	snprintf(fw_name, sizeof(fw_name), "amdgpu/%s_gpu_info.bin", chip_name);
-	err = request_firmware(&fw, fw_name, adev->dev);
+	err = request_firmware(&adev->firmware.fw, fw_name, adev->dev);
  	if (err) {
  		dev_err(adev->dev,
  			"Failed to load gpu_info firmware \"%s\"\n",
  			fw_name);
  		goto out;
  	}
-	err = amdgpu_ucode_validate(fw);
+	err = amdgpu_ucode_validate(adev->firmware.fw);
  	if (err) {
  		dev_err(adev->dev,
  			"Failed to validate gpu_info firmware \"%s\"\n",
@@ -1481,14 +1482,14 @@ static int amdgpu_device_parse_gpu_info_fw(struct amdgpu_device *adev)
  		goto out;
  	}
 
-	hdr = (const struct gpu_info_firmware_header_v1_0 *)fw->data;
+	hdr = (const struct gpu_info_firmware_header_v1_0 *)adev->firmware.fw->data;
  	amdgpu_ucode_print_gpu_info_hdr(&hdr->header);
 
  	switch (hdr->version_major) {
  	case 1:
  	{
  		const struct gpu_info_firmware_v1_0 *gpu_info_fw =
-			(const struct gpu_info_firmware_v1_0 *)(fw->data +
+			(const struct gpu_info_firmware_v1_0 *)(adev->firmware.fw->data +
  								le32_to_cpu(hdr->header.ucode_array_offset_bytes));
 
  		adev->gfx.config.max_shader_engines = gpu_info_fw->gc_num_se;
@@ -1513,9 +1514,6 @@ static int amdgpu_device_parse_gpu_info_fw(struct amdgpu_device *adev)
  		goto out;
  	}
  out:
-	release_firmware(fw);
-	fw = NULL;
-
  	return err;
  }
 

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c void amdgpu_device_fini(struct amdgpu_device *adev) · line 2320

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +4 -0
@@ -2313,6 +2311,10 @@ void amdgpu_device_fini(struct amdgpu_device *adev)
  	amdgpu_fence_driver_fini(adev);
  	amdgpu_fbdev_fini(adev);
  	r = amdgpu_fini(adev);
+	if (adev->firmware.fw) {
+		release_firmware(adev->firmware.fw);
+		adev->firmware.fw = NULL;
+	}
  	adev->accel_working = false;
  	/* free i2c buses */
  	if (!amdgpu_device_has_dc_support(adev))

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

author
#4 lore raw
In reply to a message not in the archive
Quoted 9 lines · Christian König

Hi Ray,

mhm, indeed a nice catch.

But why do we need to load the gpu info after resume in the first place?

I mean we already know what GPU we have, loading it again looks superfluous to me.

Yes, I agree with you. That's also my orignal opinion. But we encountered a random buggy when we were calling device_cache_fw_images.

[  558.288976] cache_firmware: amdgpu/vega10_sdma1.bin
[  558.288976] cache_firmware: amdgpu/vega10_sdma.bin ret=0
[  558.288981] fw_set_page_data: fw-amdgpu/vega10_sdma1.bin buf=ffff8803f1e64a80 data=ffffc90002411000 size=17408
[  558.288981] cache_firmware: amdgpu/vega10_sdma1.bin ret=0
[  558.288997] BUG: unable to handle kernel NULL pointer dereference at 0000000000000008
[  558.289001] IP: devres_for_each_res+0x5e/0x100
[  558.289001] PGD 0
[  558.289002] Oops: 0000 [#3] SMP
[  558.289003] Modules linked in: joydev hid_generic usbhid amdgpu(OE) ttm(OE) drm_kms_helper(OE) drm(OE) i2c_algo_bit fb_sys_fops syscopyarea sysfillrect sysimgblt rpcsec_gss_krb5 nfsv4 nfs fscache snd_hda_codec_realtek snd_hda_codec_generic snd_hda_codec_hdmi snd_hda_intel snd_hda_codec snd_hda_core intel_rapl snd_hwdep x86_pkg_temp_thermal intel_powerclamp snd_pcm kvm_intel snd_seq_midi kvm snd_seq_midi_event snd_rawmidi snd_seq snd_seq_device snd_timer irqbypass snd crct10dif_pclmul soundcore crc32_pclmul ghash_clmulni_intel pcbc mei_me aesni_intel shpchp mei aes_x86_64 crypto_simd glue_helper mac_hid cryptd acpi_pad tpm_infineon nfsd auth_rpcgss nfs_acl coretemp lockd grace sunrpc parport_pc ppdev lp parport autofs4 e1000e ptp nvme mxm_wmi ahci i2c_hid pps_core libahci nvme_core wmi video hid
[  558.289027] CPU: 0 PID: 3742 Comm: pm-suspend Tainted: G      D    OE   4.11.0-custom #7
[  558.289027] Hardware name: Gigabyte Technology Co., Ltd. Z170XP-SLI/Z170XP-SLI-CF, BIOS F20 11/04/2016
[  558.289027] task: ffff8803ebdcd940 task.stack: ffffc900029b0000
[  558.289029] RIP: 0010:devres_for_each_res+0x5e/0x100
[  558.289029] RSP: 0018:ffffc900029b3bc8 EFLAGS: 00010086
[  558.289030] RAX: 000000000000001d RBX: ffff880426aa0c18 RCX: 0000000000000000
[  558.289030] RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000000000000092
[  558.289031] RBP: ffffc900029b3c20 R08: 000000000000001d R09: ffffffff821ee601
[  558.289031] R10: 000000000000141d R11: 0000000000000000 R12: ffffffff81566590
[  558.289032] R13: ffffffff81566870 R14: ffffc900029b3c30 R15: ffff880426aa0e98
[  558.289032] FS:  00007f63006c3700(0000) GS:ffff88043ec00000(0000) knlGS:0000000000000000
[  558.289033] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  558.289033] CR2: 0000000000000008 CR3: 00000003f257e000 CR4: 00000000003406f0
[  558.289034] Call Trace:
[  558.289036]  ? alloc_fw_cache_entry+0x60/0x60
[  558.289037]  ? request_firmware_nowait+0x140/0x140
[  558.289038]  dev_cache_fw_image+0x46/0x120
[  558.289039]  ? request_firmware_nowait+0x140/0x140
[  558.289040]  dpm_for_each_dev+0x44/0x70
[  558.289041]  fw_pm_notify+0x164/0x190
[  558.289043]  ? prepare_to_wait_event+0x110/0x110
[  558.289044]  notifier_call_chain+0x49/0x70
[  558.289046]  __blocking_notifier_call_chain+0x4d/0x70
[  558.289047]  __pm_notifier_call_chain+0x1f/0x40
[  558.289047]  pm_suspend+0x27f/0x3a0
[  558.289048]  state_store+0x80/0xf0
[  558.289050]  kobj_attr_store+0xf/0x20
[  558.289051]  sysfs_kf_write+0x3a/0x50
[  558.289053]  kernfs_fop_write+0xff/0x180
[  558.289054]  __vfs_write+0x28/0x120
[  558.289056]  ? apparmor_file_permission+0x1a/0x20

So then I check these functions and find gpu_info errors. The random buggy cannot be reproduced constantly.But we expected it can pass more than 30 cycles of S3 suspend and resume. Any ideas?

Thanks,
Ray
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#5 lore raw
Quoted 3 lines

Yes, I agree with you. That's also my orignal opinion. But we encountered a random buggy when we were calling device_cache_fw_images.

That looks like an upstream bug in device_cache_fw_images.

We should probably open a bug report and ping the maintainer. Most likely we are not correctly using the FW interface or trigger a rare bug or something like this.

Quoted 3 lines

So then I check these functions and find gpu_info errors. The random buggy cannot be reproduced constantly.But we expected it can pass more than 30 cycles of S3 suspend and resume. Any ideas?

I think the real solution is to just stop calling amdgpu_device_parse_gpu_info_fw() during resume.

That function just sets up the adev->gfx.config fields and that is unnecessary after resume.

Regards,
Christian.

Am 06.06.2017 um 10:33 schrieb Huang Rui:

Quoted 55 lines · Christian König
Quoted 9 lines

Hi Ray,

mhm, indeed a nice catch.

But why do we need to load the gpu info after resume in the first place?

I mean we already know what GPU we have, loading it again looks superfluous to me.

Yes, I agree with you. That's also my orignal opinion. But we encountered a random buggy when we were calling device_cache_fw_images.

[  558.288976] cache_firmware: amdgpu/vega10_sdma1.bin
[  558.288976] cache_firmware: amdgpu/vega10_sdma.bin ret=0
[  558.288981] fw_set_page_data: fw-amdgpu/vega10_sdma1.bin buf=ffff8803f1e64a80 data=ffffc90002411000 size=17408
[  558.288981] cache_firmware: amdgpu/vega10_sdma1.bin ret=0
[  558.288997] BUG: unable to handle kernel NULL pointer dereference at 0000000000000008
[  558.289001] IP: devres_for_each_res+0x5e/0x100
[  558.289001] PGD 0
[  558.289002] Oops: 0000 [#3] SMP
[  558.289003] Modules linked in: joydev hid_generic usbhid amdgpu(OE) ttm(OE) drm_kms_helper(OE) drm(OE) i2c_algo_bit fb_sys_fops syscopyarea sysfillrect sysimgblt rpcsec_gss_krb5 nfsv4 nfs fscache snd_hda_codec_realtek snd_hda_codec_generic snd_hda_codec_hdmi snd_hda_intel snd_hda_codec snd_hda_core intel_rapl snd_hwdep x86_pkg_temp_thermal intel_powerclamp snd_pcm kvm_intel snd_seq_midi kvm snd_seq_midi_event snd_rawmidi snd_seq snd_seq_device snd_timer irqbypass snd crct10dif_pclmul soundcore crc32_pclmul ghash_clmulni_intel pcbc mei_me aesni_intel shpchp mei aes_x86_64 crypto_simd glue_helper mac_hid cryptd acpi_pad tpm_infineon nfsd auth_rpcgss nfs_acl coretemp lockd grace sunrpc parport_pc ppdev lp parport autofs4 e1000e ptp nvme mxm_wmi ahci i2c_hid pps_core libahci nvme_core wmi video hid
[  558.289027] CPU: 0 PID: 3742 Comm: pm-suspend Tainted: G      D    OE   4.11.0-custom #7
[  558.289027] Hardware name: Gigabyte Technology Co., Ltd. Z170XP-SLI/Z170XP-SLI-CF, BIOS F20 11/04/2016
[  558.289027] task: ffff8803ebdcd940 task.stack: ffffc900029b0000
[  558.289029] RIP: 0010:devres_for_each_res+0x5e/0x100
[  558.289029] RSP: 0018:ffffc900029b3bc8 EFLAGS: 00010086
[  558.289030] RAX: 000000000000001d RBX: ffff880426aa0c18 RCX: 0000000000000000
[  558.289030] RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000000000000092
[  558.289031] RBP: ffffc900029b3c20 R08: 000000000000001d R09: ffffffff821ee601
[  558.289031] R10: 000000000000141d R11: 0000000000000000 R12: ffffffff81566590
[  558.289032] R13: ffffffff81566870 R14: ffffc900029b3c30 R15: ffff880426aa0e98
[  558.289032] FS:  00007f63006c3700(0000) GS:ffff88043ec00000(0000) knlGS:0000000000000000
[  558.289033] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  558.289033] CR2: 0000000000000008 CR3: 00000003f257e000 CR4: 00000000003406f0
[  558.289034] Call Trace:
[  558.289036]  ? alloc_fw_cache_entry+0x60/0x60
[  558.289037]  ? request_firmware_nowait+0x140/0x140
[  558.289038]  dev_cache_fw_image+0x46/0x120
[  558.289039]  ? request_firmware_nowait+0x140/0x140
[  558.289040]  dpm_for_each_dev+0x44/0x70
[  558.289041]  fw_pm_notify+0x164/0x190
[  558.289043]  ? prepare_to_wait_event+0x110/0x110
[  558.289044]  notifier_call_chain+0x49/0x70
[  558.289046]  __blocking_notifier_call_chain+0x4d/0x70
[  558.289047]  __pm_notifier_call_chain+0x1f/0x40
[  558.289047]  pm_suspend+0x27f/0x3a0
[  558.289048]  state_store+0x80/0xf0
[  558.289050]  kobj_attr_store+0xf/0x20
[  558.289051]  sysfs_kf_write+0x3a/0x50
[  558.289053]  kernfs_fop_write+0xff/0x180
[  558.289054]  __vfs_write+0x28/0x120
[  558.289056]  ? apparmor_file_permission+0x1a/0x20
Quoted 6 lines

So then I check these functions and find gpu_info errors. The random buggy cannot be reproduced constantly.But we expected it can pass more than 30 cycles of S3 suspend and resume. Any ideas?

Thanks,
Ray

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#7 lore raw
In reply to a message not in the archive
Quoted 10 lines · Christian König
Quoted 3 lines

Yes, I agree with you. That's also my orignal opinion. But we encountered a random buggy when we were calling device_cache_fw_images.

That looks like an upstream bug in device_cache_fw_images.

We should probably open a bug report and ping the maintainer. Most likely we are not correctly using the FW interface or trigger a rare bug or something like this.

Quoted 7 lines
Quoted 4 lines

So then I check these functions and find gpu_info errors. The random buggy cannot be reproduced constantly.But we expected it can pass more than 30 cycles
of S3 suspend and resume. Any ideas?

I think the real solution is to just stop calling
amdgpu_device_parse_gpu_info_fw() during resume.

Right. we only need to parse the firmware once during startup.

Alex

Quoted 98 lines

That function just sets up the adev->gfx.config fields and that is unnecessary after resume.

Regards,
Christian.

Am 06.06.2017 um 10:33 schrieb Huang Rui:

Quoted 79 lines · Christian König
Quoted 10 lines

Hi Ray,

mhm, indeed a nice catch.

But why do we need to load the gpu info after resume in the first place?

I mean we already know what GPU we have, loading it again looks superfluous to me.

Yes, I agree with you. That's also my orignal opinion. But we encountered a random buggy when we were calling device_cache_fw_images.

[  558.288976] cache_firmware: amdgpu/vega10_sdma1.bin
[  558.288976] cache_firmware: amdgpu/vega10_sdma.bin ret=0
[  558.288981] fw_set_page_data: fw-amdgpu/vega10_sdma1.bin
buf=ffff8803f1e64a80 data=ffffc90002411000 size=17408
[  558.288981] cache_firmware: amdgpu/vega10_sdma1.bin ret=0
[  558.288997] BUG: unable to handle kernel NULL pointer dereference at
0000000000000008
[  558.289001] IP: devres_for_each_res+0x5e/0x100
[  558.289001] PGD 0
[  558.289002] Oops: 0000 [#3] SMP
[  558.289003] Modules linked in: joydev hid_generic usbhid amdgpu(OE)

ttm(OE) drm_kms_helper(OE) drm(OE) i2c_algo_bit fb_sys_fops syscopyarea sysfillrect sysimgblt rpcsec_gss_krb5 nfsv4 nfs fscache snd_hda_codec_realtek snd_hda_codec_generic snd_hda_codec_hdmi snd_hda_intel snd_hda_codec snd_hda_core intel_rapl snd_hwdep x86_pkg_temp_thermal intel_powerclamp snd_pcm kvm_intel snd_seq_midi kvm snd_seq_midi_event snd_rawmidi snd_seq snd_seq_device snd_timer irqbypass snd crct10dif_pclmul soundcore crc32_pclmul ghash_clmulni_intel pcbc mei_me aesni_intel shpchp mei aes_x86_64 crypto_simd glue_helper mac_hid cryptd acpi_pad tpm_infineon nfsd auth_rpcgss nfs_acl coretemp lockd grace sunrpc parport_pc ppdev lp parport autofs4 e1000e ptp nvme mxm_wmi ahci i2c_hid pps_core libahci nvme_core wmi video hid

[  558.289027] CPU: 0 PID: 3742 Comm: pm-suspend Tainted: G      D    OE
4.11.0-custom #7
[  558.289027] Hardware name: Gigabyte Technology Co., Ltd.
Z170XP-SLI/Z170XP-SLI-CF, BIOS F20 11/04/2016
[  558.289027] task: ffff8803ebdcd940 task.stack: ffffc900029b0000
[  558.289029] RIP: 0010:devres_for_each_res+0x5e/0x100
[  558.289029] RSP: 0018:ffffc900029b3bc8 EFLAGS: 00010086
[  558.289030] RAX: 000000000000001d RBX: ffff880426aa0c18 RCX:
0000000000000000
[  558.289030] RDX: 0000000000000001 RSI: 0000000000000000 RDI:
0000000000000092
[  558.289031] RBP: ffffc900029b3c20 R08: 000000000000001d R09:
ffffffff821ee601
[  558.289031] R10: 000000000000141d R11: 0000000000000000 R12:
ffffffff81566590
[  558.289032] R13: ffffffff81566870 R14: ffffc900029b3c30 R15:
ffff880426aa0e98
[  558.289032] FS:  00007f63006c3700(0000) GS:ffff88043ec00000(0000)
knlGS:0000000000000000
[  558.289033] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  558.289033] CR2: 0000000000000008 CR3: 00000003f257e000 CR4:
00000000003406f0
[  558.289034] Call Trace:
[  558.289036]  ? alloc_fw_cache_entry+0x60/0x60
[  558.289037]  ? request_firmware_nowait+0x140/0x140
[  558.289038]  dev_cache_fw_image+0x46/0x120
[  558.289039]  ? request_firmware_nowait+0x140/0x140
[  558.289040]  dpm_for_each_dev+0x44/0x70
[  558.289041]  fw_pm_notify+0x164/0x190
[  558.289043]  ? prepare_to_wait_event+0x110/0x110
[  558.289044]  notifier_call_chain+0x49/0x70
[  558.289046]  __blocking_notifier_call_chain+0x4d/0x70
[  558.289047]  __pm_notifier_call_chain+0x1f/0x40
[  558.289047]  pm_suspend+0x27f/0x3a0
[  558.289048]  state_store+0x80/0xf0
[  558.289050]  kobj_attr_store+0xf/0x20
[  558.289051]  sysfs_kf_write+0x3a/0x50
[  558.289053]  kernfs_fop_write+0xff/0x180
[  558.289054]  __vfs_write+0x28/0x120
[  558.289056]  ? apparmor_file_permission+0x1a/0x20
Quoted 7 lines

So then I check these functions and find gpu_info errors. The random buggy cannot be reproduced constantly.But we expected it can pass more than 30 cycles
of S3 suspend and resume. Any ideas?

Thanks,
Ray

Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#9 lore raw
In reply to a message not in the archive
Quoted 17 lines · Alex Deucher
Quoted 15 lines · Christian König
Quoted 3 lines

Yes, I agree with you. That's also my orignal opinion. But we encountered a random buggy when we were calling device_cache_fw_images.

That looks like an upstream bug in device_cache_fw_images.

We should probably open a bug report and ping the maintainer. Most likely we are not correctly using the FW interface or trigger a rare bug or something like this.

Quoted 4 lines

So then I check these functions and find gpu_info errors. The random buggy cannot be reproduced constantly.But we expected it can pass more than 30 cycles
of S3 suspend and resume. Any ideas?

Quoted 4 lines
Quoted 2 lines

I think the real solution is to just stop calling
amdgpu_device_parse_gpu_info_fw() during resume.

Right. we only need to parse the firmware once during startup.

How are hitting this on resume? amdgpu_device_parse_gpu_info_fw() is called indirectly from amdgpu_device_init() which is only called once at driver load time.

Alex

Quoted 105 lines

Alex

Quoted 102 lines

That function just sets up the adev->gfx.config fields and that is unnecessary after resume.

Regards,
Christian.

Am 06.06.2017 um 10:33 schrieb Huang Rui:

Quoted 79 lines · Christian König
Quoted 10 lines

Hi Ray,

mhm, indeed a nice catch.

But why do we need to load the gpu info after resume in the first place?

I mean we already know what GPU we have, loading it again looks superfluous to me.

Yes, I agree with you. That's also my orignal opinion. But we encountered a random buggy when we were calling device_cache_fw_images.

[  558.288976] cache_firmware: amdgpu/vega10_sdma1.bin
[  558.288976] cache_firmware: amdgpu/vega10_sdma.bin ret=0
[  558.288981] fw_set_page_data: fw-amdgpu/vega10_sdma1.bin
buf=ffff8803f1e64a80 data=ffffc90002411000 size=17408
[  558.288981] cache_firmware: amdgpu/vega10_sdma1.bin ret=0
[  558.288997] BUG: unable to handle kernel NULL pointer dereference at
0000000000000008
[  558.289001] IP: devres_for_each_res+0x5e/0x100
[  558.289001] PGD 0
[  558.289002] Oops: 0000 [#3] SMP
[  558.289003] Modules linked in: joydev hid_generic usbhid amdgpu(OE)

ttm(OE) drm_kms_helper(OE) drm(OE) i2c_algo_bit fb_sys_fops syscopyarea sysfillrect sysimgblt rpcsec_gss_krb5 nfsv4 nfs fscache snd_hda_codec_realtek snd_hda_codec_generic snd_hda_codec_hdmi snd_hda_intel snd_hda_codec snd_hda_core intel_rapl snd_hwdep x86_pkg_temp_thermal intel_powerclamp snd_pcm kvm_intel snd_seq_midi kvm snd_seq_midi_event snd_rawmidi snd_seq snd_seq_device snd_timer irqbypass snd crct10dif_pclmul soundcore crc32_pclmul ghash_clmulni_intel pcbc mei_me aesni_intel shpchp mei aes_x86_64 crypto_simd glue_helper mac_hid cryptd acpi_pad tpm_infineon nfsd auth_rpcgss nfs_acl coretemp lockd grace sunrpc parport_pc ppdev lp parport autofs4 e1000e ptp nvme mxm_wmi ahci i2c_hid pps_core libahci nvme_core wmi video hid

[  558.289027] CPU: 0 PID: 3742 Comm: pm-suspend Tainted: G      D    OE
4.11.0-custom #7
[  558.289027] Hardware name: Gigabyte Technology Co., Ltd.
Z170XP-SLI/Z170XP-SLI-CF, BIOS F20 11/04/2016
[  558.289027] task: ffff8803ebdcd940 task.stack: ffffc900029b0000
[  558.289029] RIP: 0010:devres_for_each_res+0x5e/0x100
[  558.289029] RSP: 0018:ffffc900029b3bc8 EFLAGS: 00010086
[  558.289030] RAX: 000000000000001d RBX: ffff880426aa0c18 RCX:
0000000000000000
[  558.289030] RDX: 0000000000000001 RSI: 0000000000000000 RDI:
0000000000000092
[  558.289031] RBP: ffffc900029b3c20 R08: 000000000000001d R09:
ffffffff821ee601
[  558.289031] R10: 000000000000141d R11: 0000000000000000 R12:
ffffffff81566590
[  558.289032] R13: ffffffff81566870 R14: ffffc900029b3c30 R15:
ffff880426aa0e98
[  558.289032] FS:  00007f63006c3700(0000) GS:ffff88043ec00000(0000)
knlGS:0000000000000000
[  558.289033] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  558.289033] CR2: 0000000000000008 CR3: 00000003f257e000 CR4:
00000000003406f0
[  558.289034] Call Trace:
[  558.289036]  ? alloc_fw_cache_entry+0x60/0x60
[  558.289037]  ? request_firmware_nowait+0x140/0x140
[  558.289038]  dev_cache_fw_image+0x46/0x120
[  558.289039]  ? request_firmware_nowait+0x140/0x140
[  558.289040]  dpm_for_each_dev+0x44/0x70
[  558.289041]  fw_pm_notify+0x164/0x190
[  558.289043]  ? prepare_to_wait_event+0x110/0x110
[  558.289044]  notifier_call_chain+0x49/0x70
[  558.289046]  __blocking_notifier_call_chain+0x4d/0x70
[  558.289047]  __pm_notifier_call_chain+0x1f/0x40
[  558.289047]  pm_suspend+0x27f/0x3a0
[  558.289048]  state_store+0x80/0xf0
[  558.289050]  kobj_attr_store+0xf/0x20
[  558.289051]  sysfs_kf_write+0x3a/0x50
[  558.289053]  kernfs_fop_write+0xff/0x180
[  558.289054]  __vfs_write+0x28/0x120
[  558.289056]  ? apparmor_file_permission+0x1a/0x20
Quoted 7 lines

So then I check these functions and find gpu_info errors. The random buggy cannot be reproduced constantly.But we expected it can pass more than 30 cycles
of S3 suspend and resume. Any ideas?

Thanks,
Ray

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Reviewed-by
#12 lore raw
In reply to Huang Rui
Quoted 24 lines · Huang Rui
Quoted 23 lines · Alex Deucher
Quoted 21 lines · Alex Deucher
Quoted 10 lines · Christian König
Quoted 3 lines

Yes, I agree with you. That's also my orignal opinion. But we encountered a random buggy when we were calling device_cache_fw_images.

That looks like an upstream bug in device_cache_fw_images.

We should probably open a bug report and ping the maintainer. Most likely we are not correctly using the FW interface or trigger a rare bug or something like this.

Quoted 7 lines
Quoted 4 lines

So then I check these functions and find gpu_info errors. The random buggy cannot be reproduced constantly.But we expected it can pass more than 30 cycles
of S3 suspend and resume. Any ideas?

I think the real solution is to just stop calling
amdgpu_device_parse_gpu_info_fw() during resume.

Right. we only need to parse the firmware once during startup.

Quoted 7 lines
Quoted 4 lines

How are hitting this on resume? amdgpu_device_parse_gpu_info_fw() is called indirectly from amdgpu_device_init() which is only called once at driver load time.

Yes, I also noted it. So I am confused with why firmware_class will still cache it during suspend.

I guess request_firmware expects the driver to keep the firmware around until the driver is unloaded. Just one comment about the patch:

@@ -1272,6 +1272,9 @@ struct amdgpu_firmware {
        const struct amdgpu_psp_funcs *funcs;
        struct amdgpu_bo *rbuf;
        struct mutex mutex;

+
+ /* gpu info firmware data pointer */
+ const struct firmware *fw;
};

Call this gpu_info_fw rather than just fw. With that, the patch is:

Alex
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

author
#11 lore raw
In reply to a message not in the archive
Quoted 18 lines · Alex Deucher
Quoted 17 lines · Alex Deucher
Quoted 15 lines · Christian König
Quoted 3 lines

Yes, I agree with you. That's also my orignal opinion. But we encountered a random buggy when we were calling device_cache_fw_images.

That looks like an upstream bug in device_cache_fw_images.

We should probably open a bug report and ping the maintainer. Most likely we are not correctly using the FW interface or trigger a rare bug or something like this.

Quoted 4 lines

So then I check these functions and find gpu_info errors. The random buggy cannot be reproduced constantly.But we expected it can pass more than 30 cycles
of S3 suspend and resume. Any ideas?

Quoted 9 lines
Quoted 4 lines
Quoted 2 lines

I think the real solution is to just stop calling
amdgpu_device_parse_gpu_info_fw() during resume.

Right. we only need to parse the firmware once during startup.

How are hitting this on resume? amdgpu_device_parse_gpu_info_fw() is called indirectly from amdgpu_device_init() which is only called once at driver load time.

Yes, I also noted it. So I am confused with why firmware_class will still cache it during suspend.

Thanks,
Ray
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

author
#13 lore raw
Quoted 24 lines · Huang Rui
Quoted 23 lines · Alex Deucher
Quoted 21 lines · Alex Deucher
Quoted 10 lines · Christian K?nig
Quoted 3 lines

Yes, I agree with you. That's also my orignal opinion. But we encountered a random buggy when we were calling device_cache_fw_images.

That looks like an upstream bug in device_cache_fw_images.

We should probably open a bug report and ping the maintainer. Most likely we are not correctly using the FW interface or trigger a rare bug or something like this.

Quoted 7 lines
Quoted 4 lines

So then I check these functions and find gpu_info errors. The random buggy cannot be reproduced constantly.But we expected it can pass more than 30 cycles
of S3 suspend and resume. Any ideas?

I think the real solution is to just stop calling
amdgpu_device_parse_gpu_info_fw() during resume.

Right. we only need to parse the firmware once during startup.

Quoted 8 lines
Quoted 4 lines

How are hitting this on resume? amdgpu_device_parse_gpu_info_fw() is called indirectly from amdgpu_device_init() which is only called once at driver load time.

Yes, I also noted it. So I am confused with why firmware_class will still cache it during suspend.

At that time, we already have released gpu_info firmware data. It seems a bug of upper layer.

Thanks,
Ray
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx