Skip to contentSkip to conversation

drm/amd/amdgpu: Allow broadcast on debugfs read (v2)

PATCH In review Tom St Denis 5 participants 10 messages Oct 9, 2016 – Oct 19, 2016 9 messages not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/amd/amdgpu: Allow broadcast on debugfs read (v2)

Allow any of the se/sh/instance fields to be
specified as a broadcast by submitting 0x3FF.

(v2) Fix broadcast range checking

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +9 -2
index 4217e754f99e..04c4aee70452 100644
@@ -2574,6 +2574,13 @@ static ssize_t amdgpu_debugfs_regs_read(struct file *f, char __user *buf,
 		se_bank = (*pos >> 24) & 0x3FF;
 		sh_bank = (*pos >> 34) & 0x3FF;
 		instance_bank = (*pos >> 44) & 0x3FF;
+
+		if (se_bank == 0x3FF)
+			se_bank = 0xFFFFFFFF;
+		if (sh_bank == 0x3FF)
+			sh_bank = 0xFFFFFFFF;
+		if (instance_bank == 0x3FF)
+			instance_bank = 0xFFFFFFFF;
 		use_bank = 1;
 	} else {
 		use_bank = 0;
@@ -2582,8 +2589,8 @@ static ssize_t amdgpu_debugfs_regs_read(struct file *f, char __user *buf,
 	*pos &= 0x3FFFF;
 
 	if (use_bank) {
-		if (sh_bank >= adev->gfx.config.max_sh_per_se ||
-		    se_bank >= adev->gfx.config.max_shader_engines)
+		if ((sh_bank != 0xFFFFFFFF && sh_bank >= adev->gfx.config.max_sh_per_se) ||
+		    (se_bank != 0xFFFFFFFF && se_bank >= adev->gfx.config.max_shader_engines))
 			return -EINVAL;
 		mutex_lock(&adev->grbm_idx_mutex);
 		amdgpu_gfx_select_se_sh(adev, se_bank,
Signature · 6 lines
2.10.0

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): [email protected]

#2 lore raw
Quoted 6 lines · Tom St Denis

Allow any of the se/sh/instance fields to be
specified as a broadcast by submitting 0x3FF.

(v2) Fix broadcast range checking

I'm not sure how a "broadcast read" is supposed to work? I can only see the register specs talking about broadcast in connection with writes.

Signature · 6 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): a2035199-429c-d377-b354-bf42148d3820-cl+VPiYnx/[email protected]

#4 lore raw

That actually makes sense. It came up because mmPA_SC_RASTER_CONFIG (and _1) are read by libdrm with a selection of 0/*/* for se/sh/instance so I wanted to be able to reproduce that via the debugfs interface for testing.

But ya, if you use the broadcast bit who are you reading from then ... hmm..

Tom

________________________________
From: amd-gfx <amd-gfx-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org> on behalf of Michel Dänzer <michel.daenzer-cl+VPiYnx/[email protected]>
Sent: Monday, October 10, 2016 23:52
To: Tom St Denis
Cc: [email protected]
Subject: Re: [PATCH] drm/amd/amdgpu: Allow broadcast on debugfs read (v2)

Quoted 6 lines · Tom St Denis

Allow any of the se/sh/instance fields to be
specified as a broadcast by submitting 0x3FF.

(v2) Fix broadcast range checking

I'm not sure how a "broadcast read" is supposed to work? I can only see the register specs talking about broadcast in connection with writes.

--
Earthling Michel Dänzer | http://www.amd.com
Graphics, Processors and Immersive VR Solutions | AMD <http://www.amd.com/>
www.amd.com
Explore a wide range of innovative next generation computing processors, graphics, and Immersive VR solutions by Advanced Micro Devices (AMD). Visit AMD.com now!

Libre software enthusiast | Mesa and X developer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
amd-gfx Info Page - lists.freedesktop.org<https://lists.freedesktop.org/mailman/listinfo/amd-gfx>
lists.freedesktop.org
To see the collection of prior postings to the list, visit the amd-gfx Archives. Using amd-gfx: To post a message to all the list members, send email ...

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): CY4PR12MB176836D549B404152F5A1040F7DA0-rpdhrqHFk06yjjPBNVDk/[email protected]

#6 lore raw

I briefly remember reading in some hardware doc that this actually has a defined behavior.

It was something about applying either bit wise "and" or "or" on the resulting bits and returning that to the reader.

This way you can check with a single broadcast read if an error occurred or if any of the instances are busy.

But if you don't actually use it I suggest to not implement this.

Regards,
Christian.

Am 11.10.2016 um 12:26 schrieb StDenis, Tom:

Quoted 21 lines

That actually makes sense. It came up because mmPA_SC_RASTER_CONFIG (and _1) are read by libdrm with a selection of 0/*/* for se/sh/instance so I wanted to be able to reproduce that via the debugfs interface for testing.

But ya, if you use the broadcast bit who are you reading from then ... hmm..

Tom

------------------------------------------------------------------------
*From:* amd-gfx <amd-gfx-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org> on behalf of Michel Dänzer <michel.daenzer-cl+VPiYnx/[email protected]> *Sent:* Monday, October 10, 2016 23:52
*To:* Tom St Denis
*Cc:* [email protected]
*Subject:* Re: [PATCH] drm/amd/amdgpu: Allow broadcast on debugfs read (v2)

Quoted 3 lines · Tom St Denis

Allow any of the se/sh/instance fields to be
specified as a broadcast by submitting 0x3FF.

Quoted 33 lines
Quoted 3 lines

(v2) Fix broadcast range checking

I'm not sure how a "broadcast read" is supposed to work? I can only see the register specs talking about broadcast in connection with writes.

Signature · 24 lines
Earthling Michel Dänzer               | http://www.amd.com <http://www.amd.com>
Graphics, Processors and Immersive VR Solutions | AMD <http://www.amd.com/>
www.amd.com
Explore a wide range of innovative next generation computing processors, graphics, and Immersive VR solutions by Advanced Micro Devices (AMD). Visit AMD.com now!



Libre software enthusiast             |             Mesa and X developer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
amd-gfx Info Page - lists.freedesktop.org <https://lists.freedesktop.org/mailman/listinfo/amd-gfx>
lists.freedesktop.org
To see the collection of prior postings to the list, visit the amd-gfx Archives. Using amd-gfx: To post a message to all the list members, send email ...





_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 942ca6f5-fb0b-5f6f-7a87-1ffd6335be5d-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org

#8 lore raw

It's used by the UMD though they read from 0/*/* when reading the RASTER_CONFIG registers (which may be a bug...)

I only added it because the previous interface was either */*/* or x/y/z (where none are *). Now it supports arbitrary bank selections.

Tom

________________________________
From: Christian König <[email protected]>
Sent: Tuesday, October 11, 2016 08:30
To: StDenis, Tom; Michel Dänzer
Cc: [email protected]
Subject: Re: [PATCH] drm/amd/amdgpu: Allow broadcast on debugfs read (v2)

I briefly remember reading in some hardware doc that this actually has a defined behavior.

It was something about applying either bit wise "and" or "or" on the resulting bits and returning that to the reader.

This way you can check with a single broadcast read if an error occurred or if any of the instances are busy.

But if you don't actually use it I suggest to not implement this.

Regards,
Christian.

Am 11.10.2016 um 12:26 schrieb StDenis, Tom:

That actually makes sense. It came up because mmPA_SC_RASTER_CONFIG (and _1) are read by libdrm with a selection of 0/*/* for se/sh/instance so I wanted to be able to reproduce that via the debugfs interface for testing.

But ya, if you use the broadcast bit who are you reading from then ... hmm..

Tom

________________________________
From: amd-gfx <amd-gfx-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org><mailto:[email protected]> on behalf of Michel Dänzer <[email protected]><mailto:michel.daenzer-cl+VPiYnx/[email protected]>
Sent: Monday, October 10, 2016 23:52
To: Tom St Denis
Cc: [email protected]<mailto:[email protected]>
Subject: Re: [PATCH] drm/amd/amdgpu: Allow broadcast on debugfs read (v2)

Quoted 6 lines · Tom St Denis

Allow any of the se/sh/instance fields to be
specified as a broadcast by submitting 0x3FF.

(v2) Fix broadcast range checking

I'm not sure how a "broadcast read" is supposed to work? I can only see the register specs talking about broadcast in connection with writes.

--
Earthling Michel Dänzer | http://www.amd.com
Graphics, Processors and Immersive VR Solutions | AMD <http://www.amd.com/>
www.amd.com<http://www.amd.com>
Explore a wide range of innovative next generation computing processors, graphics, and Immersive VR solutions by Advanced Micro Devices (AMD). Visit AMD.com now!

Libre software enthusiast | Mesa and X developer
_______________________________________________
amd-gfx mailing list [email protected]<mailto:[email protected]>
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
amd-gfx Info Page - lists.freedesktop.org<https://lists.freedesktop.org/mailman/listinfo/amd-gfx>
lists.freedesktop.org
To see the collection of prior postings to the list, visit the amd-gfx Archives. Using amd-gfx: To post a message to all the list members, send email ...

_______________________________________________
amd-gfx mailing list [email protected]<mailto:[email protected]>
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): BN6PR12MB1764A6A7F768C662DC5866EBF7DA0-/b2+HYfkarS51p+Siuid1wdYzm3356FpvxpqHgZTriW3zl9H0oFU5g@public.gmane.org

#10 lore raw
Quoted 2 lines · Tom

It's used by the UMD though they read from 0/*/* when reading the RASTER_CONFIG registers (which may be a bug...)

We should probably clarify what userspace is trying to do there, and whether the hardware actually does that.

Signature · 6 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): d1a48843-4541-1f68-eac1-21821998b735-cl+VPiYnx/[email protected]

#12 lore raw

It comes from amdgpu_query_gpu_info_init()

        for (i = 0; i < (int)dev->info.num_shader_engines; i++) {
                unsigned instance = (i << AMDGPU_INFO_MMR_SE_INDEX_SHIFT) |
                                    (AMDGPU_INFO_MMR_SH_INDEX_MASK <<
                                     AMDGPU_INFO_MMR_SH_INDEX_SHIFT);

                r = amdgpu_read_mm_registers(dev, 0x263d, 1, instance, 0,
                                             &dev->info.backend_disable[i]);

This effectively reads from 0/* where the kernel adds the instance of * so it's 0/*/*. That line was last changed by Alex

0936139536380 (Alex Deucher 2015-04-20 12:04:22 -0400 174) (AMDGPU_INFO_MMR_SH_INDEX_MASK <<

I still don't get why this is a reason to hit pause on the patch(es) though. They're root only, you can still not use * if you don't want to and furthermore I'm already using the write debugfs content (to debug waveforms).

Tom

________________________________
From: Michel Dänzer <michel.daenzer-cl+VPiYnx/[email protected]>
Sent: Tuesday, October 11, 2016 20:34
To: StDenis, Tom; Christian König
Cc: [email protected]
Subject: Re: [PATCH] drm/amd/amdgpu: Allow broadcast on debugfs read (v2)

Quoted 2 lines · Tom

It's used by the UMD though they read from 0/*/* when reading the RASTER_CONFIG registers (which may be a bug...)

We should probably clarify what userspace is trying to do there, and whether the hardware actually does that.

--
Earthling Michel Dänzer | http://www.amd.com
Graphics, Processors and Immersive VR Solutions | AMD <http://www.amd.com/>
www.amd.com
Explore a wide range of innovative next generation computing processors, graphics, and Immersive VR solutions by Advanced Micro Devices (AMD). Visit AMD.com now!

Libre software enthusiast | Mesa and X developer

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): CY4PR12MB1768476CAA76C1F8BE2D429FF7DD0-rpdhrqHFk06yjjPBNVDk/[email protected]

#14 lore raw
Quoted 11 lines · Tom

It comes from amdgpu_query_gpu_info_init()

        for (i = 0; i < (int)dev->info.num_shader_engines; i++) {
                unsigned instance = (i << AMDGPU_INFO_MMR_SE_INDEX_SHIFT) |
                                    (*AMDGPU_INFO_MMR_SH_INDEX_MASK*<<
                                     AMDGPU_INFO_MMR_SH_INDEX_SHIFT);

                r = amdgpu_read_mm_registers(dev, 0x263d, 1, instance, 0,
                                             &dev->info.backend_disable[i]);
Quoted 5 lines

This effectively reads from 0/* where the kernel adds the instance of * so it's 0/*/*. That line was last changed by Alex

*0936139536380* (Alex Deucher 2015-04-20 12:04:22 -0400 174)
(AMDGPU_INFO_MMR_SH_INDEX_MASK <<

As a side note, following that to the end in the kernel code, I noticed an interesting minor difference between the AMDGPU_INFO_READ_MMR_REG functionality used by this code and the debugfs interface:

With AMDGPU_INFO_READ_MMR_REG, the effect is that amdgpu_asic_read_register() doesn't call amdgpu_gfx_select_se_sh() at all before reading the register, so the read is performed from whichever SH instance is currently selected.

Whereas with this patch, amdgpu_debugfs_regs_read() calls amdgpu_gfx_select_se_sh(adev, se_bank, 0xFFFFFFFF, instance_bank) before the register read, which translates to only the SH_BROADCAST_WRITES bit being set for the SH instance index.

The end result should be the same though, since amdgpu_gfx_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff) is normally called after every register read.

Quoted 2 lines

I still don't get why this is a reason to hit pause on the patch(es) though.

At the very least, it should be documented in an appropriate place (commit log and/or code, or any other place where the debugfs interface semantics are documented) what actually happens when passing all ones for the SE/SH index. Does the hardware ignore the *_BROADCAST_WRITES bit for reads, so they're performed from instance 0, or does it combine the values from all instances with logical and/or?

Signature · 7 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 9625361e-271c-ed34-d92a-0ea1ca0ca7a1-otUistvHUpPR7s880joybQ@public.gmane.org

#16 lore raw
Quoted 40 lines · Michel Dänzer
Quoted 11 lines · Tom

It comes from amdgpu_query_gpu_info_init()

        for (i = 0; i < (int)dev->info.num_shader_engines; i++) {
                unsigned instance = (i << AMDGPU_INFO_MMR_SE_INDEX_SHIFT) |
                                    (*AMDGPU_INFO_MMR_SH_INDEX_MASK*<<
                                     AMDGPU_INFO_MMR_SH_INDEX_SHIFT);

                r = amdgpu_read_mm_registers(dev, 0x263d, 1, instance, 0,
                                             &dev->info.backend_disable[i]);
Quoted 5 lines

This effectively reads from 0/* where the kernel adds the instance of * so it's 0/*/*. That line was last changed by Alex

*0936139536380* (Alex Deucher 2015-04-20 12:04:22 -0400 174)
(AMDGPU_INFO_MMR_SH_INDEX_MASK <<

As a side note, following that to the end in the kernel code, I noticed an interesting minor difference between the AMDGPU_INFO_READ_MMR_REG functionality used by this code and the debugfs interface:

With AMDGPU_INFO_READ_MMR_REG, the effect is that amdgpu_asic_read_register() doesn't call amdgpu_gfx_select_se_sh() at all before reading the register, so the read is performed from whichever SH instance is currently selected.

Whereas with this patch, amdgpu_debugfs_regs_read() calls amdgpu_gfx_select_se_sh(adev, se_bank, 0xFFFFFFFF, instance_bank) before the register read, which translates to only the SH_BROADCAST_WRITES bit being set for the SH instance index.

The end result should be the same though, since amdgpu_gfx_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff) is normally called after every register read.

Quoted 2 lines

I still don't get why this is a reason to hit pause on the patch(es) though.

Quoted 6 lines

At the very least, it should be documented in an appropriate place (commit log and/or code, or any other place where the debugfs interface semantics are documented) what actually happens when passing all ones for the SE/SH index. Does the hardware ignore the *_BROADCAST_WRITES bit for reads, so they're performed from instance 0, or does it combine the values from all instances with logical and/or?

I'm not sure how to interpret the fact that this patch has landed without any changes or followups.

FWIW, I'm still interested in (pointers to) information about what the libdrm_amdgpu code above expects and what the hardware does for reads with the broadcast bit enabled, from anyone.

Signature · 6 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): dbf837f3-d9e6-a618-bd78-54f17c7b7d2c-otUistvHUpPR7s880joybQ@public.gmane.org

#18 lore raw

The broadcast read was required to attempt to debug a pro-stack problem which I believe the "cache gca config" patches are meant to mitigate. The broadcast write support is required to select CUs for wave readings. libdrm performs a broadcast read when reading raster config data so apparently that's a thing.

Since it's a debugfs interface any stability problems a broadcast read may or may not cause isn't really a concern for day to day operations. FWIW I've never yet seen an issue with broadcast reading raster config registers.

But to put it simply, the patch was part of a series that was blocking other work and there wasn't sufficient cause to NAK it.

Tom

________________________________
From: Michel Dänzer <[email protected]>
Sent: Tuesday, October 18, 2016 23:49
To: StDenis, Tom; Christian König
Cc: [email protected]
Subject: Re: [PATCH] drm/amd/amdgpu: Allow broadcast on debugfs read (v2)

Quoted 40 lines · Michel Dänzer
Quoted 11 lines · Tom

It comes from amdgpu_query_gpu_info_init()

        for (i = 0; i < (int)dev->info.num_shader_engines; i++) {
                unsigned instance = (i << AMDGPU_INFO_MMR_SE_INDEX_SHIFT) |
                                    (*AMDGPU_INFO_MMR_SH_INDEX_MASK*<<
                                     AMDGPU_INFO_MMR_SH_INDEX_SHIFT);

                r = amdgpu_read_mm_registers(dev, 0x263d, 1, instance, 0,
                                             &dev->info.backend_disable[i]);
Quoted 5 lines

This effectively reads from 0/* where the kernel adds the instance of * so it's 0/*/*. That line was last changed by Alex

*0936139536380* (Alex Deucher 2015-04-20 12:04:22 -0400 174)
(AMDGPU_INFO_MMR_SH_INDEX_MASK <<

As a side note, following that to the end in the kernel code, I noticed an interesting minor difference between the AMDGPU_INFO_READ_MMR_REG functionality used by this code and the debugfs interface:

With AMDGPU_INFO_READ_MMR_REG, the effect is that amdgpu_asic_read_register() doesn't call amdgpu_gfx_select_se_sh() at all before reading the register, so the read is performed from whichever SH instance is currently selected.

Whereas with this patch, amdgpu_debugfs_regs_read() calls amdgpu_gfx_select_se_sh(adev, se_bank, 0xFFFFFFFF, instance_bank) before the register read, which translates to only the SH_BROADCAST_WRITES bit being set for the SH instance index.

The end result should be the same though, since amdgpu_gfx_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff) is normally called after every register read.

Quoted 2 lines

I still don't get why this is a reason to hit pause on the patch(es) though.

Quoted 6 lines

At the very least, it should be documented in an appropriate place (commit log and/or code, or any other place where the debugfs interface semantics are documented) what actually happens when passing all ones for the SE/SH index. Does the hardware ignore the *_BROADCAST_WRITES bit for reads, so they're performed from instance 0, or does it combine the values from all instances with logical and/or?

I'm not sure how to interpret the fact that this patch has landed without any changes or followups.

FWIW, I'm still interested in (pointers to) information about what the libdrm_amdgpu code above expects and what the hardware does for reads with the broadcast bit enabled, from anyone.

--
Earthling Michel Dänzer | http://www.amd.com
Graphics, Processors and Immersive VR Solutions | AMD <http://www.amd.com/>
www.amd.com
Explore a wide range of innovative next generation computing processors, graphics, and Immersive VR solutions by Advanced Micro Devices (AMD). Visit AMD.com now!

Libre software enthusiast | Mesa and X developer

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx