Skip to contentSkip to conversation

drm/radeon/bo: add some fallback placements for VRAM only objects.

PATCH In review Dave Airlie 3 participants 10 messages Mar 19, 2010 – Mar 28, 2010

Subsystems (2)
lore
Share & tools
lore

Dave Airlie

author maintainer
#0 lore raw

PATCH 1/2 drm/radeon/bo: add some fallback placements for VRAM only objects.

From: Dave Airlie <[email protected]>

On constrained r100 systems compiz would fail to start due to a lack of memory, we can just fallback place the objects rather than completely failing it works a lot better.

---
 drivers/gpu/drm/radeon/radeon.h        |    2 ++
 drivers/gpu/drm/radeon/radeon_object.c |   10 +++++++---
 2 files changed, 9 insertions(+), 3 deletions(-)
drivers/gpu/drm/radeon/radeon.h +2 -0
index 72ed251..69585bb 100644
@@ -238,7 +238,9 @@ struct radeon_bo {
 	struct list_head		list;
 	/* Protected by tbo.reserved */
 	u32				placements[3];
+	u32				busy_placements[3];
 	struct ttm_placement		placement;
+	struct ttm_placement		busy_placement;
 	struct ttm_buffer_object	tbo;
 	struct ttm_bo_kmap_obj		kmap;
 	unsigned			pin_count;
drivers/gpu/drm/radeon/radeon_object.c +7 -3
index fc9d00a..3580c75 100644
@@ -65,15 +65,19 @@ bool radeon_ttm_bo_is_radeon_bo(struct ttm_buffer_object *bo)
 
 void radeon_ttm_placement_from_domain(struct radeon_bo *rbo, u32 domain)
 {
-	u32 c = 0;
+	u32 c = 0, b = 0;
 
 	rbo->placement.fpfn = 0;
 	rbo->placement.lpfn = 0;
 	rbo->placement.placement = rbo->placements;
-	rbo->placement.busy_placement = rbo->placements;
+	rbo->placement.busy_placement = rbo->busy_placements;
 	if (domain & RADEON_GEM_DOMAIN_VRAM)
 		rbo->placements[c++] = TTM_PL_FLAG_WC | TTM_PL_FLAG_UNCACHED |
 					TTM_PL_FLAG_VRAM;
+	/* add busy placement to TTM if VRAM is only option */
+	if (domain == RADEON_GEM_DOMAIN_VRAM) {
+		rbo->busy_placements[b++] = TTM_PL_MASK_CACHING | TTM_PL_FLAG_TT;
+	}
 	if (domain & RADEON_GEM_DOMAIN_GTT)
 		rbo->placements[c++] = TTM_PL_MASK_CACHING | TTM_PL_FLAG_TT;
 	if (domain & RADEON_GEM_DOMAIN_CPU)
@@ -81,7 +85,7 @@ void radeon_ttm_placement_from_domain(struct radeon_bo *rbo, u32 domain)
 	if (!c)
 		rbo->placements[c++] = TTM_PL_MASK_CACHING | TTM_PL_FLAG_SYSTEM;
 	rbo->placement.num_placement = c;
-	rbo->placement.num_busy_placement = c;
+	rbo->placement.num_busy_placement = b;
 }
 
 int radeon_bo_create(struct radeon_device *rdev, struct drm_gem_object *gobj,
Signature · 10 lines
1.6.6.1


------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
--
author maintainer
#1 lore raw

PATCH 2/2 drm/radeon/kms: don't print error on -ERESTARTSYS.

From: Dave Airlie <[email protected]>

We can get this if the user moves the mouse when we are waiting to move some stuff around in the validate. Don't fail.

---
 drivers/gpu/drm/radeon/radeon_cs.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)
drivers/gpu/drm/radeon/radeon_cs.c +2 -1
index 8d99f70..3346d9e 100644
@@ -239,7 +239,8 @@ int radeon_cs_ioctl(struct drm_device *dev, void *data, struct drm_file *filp)
 	}
 	r = radeon_cs_parser_relocs(&parser);
 	if (r) {
-		DRM_ERROR("Failed to parse relocation !\n");
+		if (r != -ERESTARTSYS)
+			DRM_ERROR("Failed to parse relocation %d!\n", r);
 		radeon_cs_parser_fini(&parser, r);
 		mutex_unlock(&rdev->cs_mutex);
 		return r;
Signature · 10 lines
1.6.6.1


------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
--
#2 lore raw

Re: [PATCH 2/2] drm/radeon/kms: don't print error on -ERESTARTSYS.

Quoted 25 lines · Dave Airlie

From: Dave Airlie <[email protected]>

We can get this if the user moves the mouse when we are waiting to move some stuff around in the validate. Don't fail.

---
 drivers/gpu/drm/radeon/radeon_cs.c |    3 ++-

 1 files changed, 2 insertions(+), 1 deletions(-)
drivers/gpu/drm/radeon/radeon_cs.c +2 -2
index 8d99f70..3346d9e 100644
@@ -239,7 +239,8 @@ int radeon_cs_ioctl(struct drm_device *dev, void *data, struct drm_file *filp)
        }
        r = radeon_cs_parser_relocs(&parser);
        if (r) {
-               DRM_ERROR("Failed to parse relocation !\n");
+               if (r != -ERESTARTSYS)
+                       DRM_ERROR("Failed to parse relocation %d!\n", r);
                radeon_cs_parser_fini(&parser, r);
                mutex_unlock(&rdev->cs_mutex);
                return r;
--
Quoted 1 line

1.6.6.1

Do we want to interrupt the parser?

If we do then user space needs updating so mesa/ddx knows to submit cs again if return code is showing that parsing was interrupted.

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
--

#3 lore raw

Re: [PATCH 2/2] drm/radeon/kms: don't print error on -ERESTARTSYS.

Quoted 30 lines · Pauli Nieminen
Quoted 25 lines · Dave Airlie

From: Dave Airlie <[email protected]>

We can get this if the user moves the mouse when we are waiting to move some stuff around in the validate. Don't fail.

---
 drivers/gpu/drm/radeon/radeon_cs.c |    3 ++-

 1 files changed, 2 insertions(+), 1 deletions(-)
drivers/gpu/drm/radeon/radeon_cs.c +2 -2
index 8d99f70..3346d9e 100644
@@ -239,7 +239,8 @@ int radeon_cs_ioctl(struct drm_device *dev, void *data, struct drm_file *filp)
        }
        r = radeon_cs_parser_relocs(&parser);
        if (r) {
-               DRM_ERROR("Failed to parse relocation !\n");
+               if (r != -ERESTARTSYS)
+                       DRM_ERROR("Failed to parse relocation %d!\n", r);
                radeon_cs_parser_fini(&parser, r);
                mutex_unlock(&rdev->cs_mutex);
                return r;
--
Quoted 1 line

1.6.6.1

Do we want to interrupt the parser?

Quoted 3 lines

If we do then user space needs updating so mesa/ddx knows to submit cs again if return code is showing that parsing was interrupted.

Forget that comment. I was somehow reading it as -EINTR.

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
--

#4 lore raw
In reply to Dave Airlie
Quoted 7 lines · Dave Airlie

From: Dave Airlie <[email protected]>

On constrained r100 systems compiz would fail to start due to a lack of memory, we can just fallback place the objects rather than completely failing it works a lot better.

This change seems to trigger or at least greatly expedite GPU lockups on my PowerBook. With the change applied, my normal X session locked up the GPU after just a few minutes several times. Now with it reverted it's back to the previous stability.

I don't know why that is - maybe something doesn't properly deal with BOs getting placed differently in some cases now - but anyway I suspect the implications of this change haven't been fully thought through: The log message sounds as though the change was mainly written with radeon_bo_create() / radeon_bo_list_validate() in mind, but radeon_ttm_placement_from_domain() is also called from other places:

* radeon_bo_pin(): The change could lead to a BO being pinned to GTT instead of VRAM, which would probably be bad.
* radeon_evict_flags(): The change might have undesirable consequences here as well, not sure.

I don't think the way we currently use the same arrays for normal and busy placement makes a lot of sense, but we probably need a better mechanism to specify which placements are desirable / acceptable in each case.

Signature · 19 lines
Earthling Michel Dänzer           |                http://www.vmware.com
Libre software enthusiast         |          Debian, X and DRI developer






------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
--
_______________________________________________
Dri-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dri-devel
author maintainer
#5 lore raw

2010/3/25 Michel Dänzer <[email protected]>:

Quoted 7 lines · Dave Airlie
Quoted 6 lines

From: Dave Airlie <[email protected]>

On constrained r100 systems compiz would fail to start due to a lack of memory, we can just fallback place the objects rather than completely failing it works a lot better.

Quoted 6 lines
Quoted 1 line

This change seems to trigger or at least greatly expedite GPU lockups on my PowerBook. With the change applied, my normal X session locked up the GPU after just a few minutes several times. Now with it reverted it's back to the previous stability.

Care to try in pci mode? see if helps, it might be just straining AGP a bit more,
maybe try 1x as well if you aren't already in it.

Quoted 8 lines

I don't know why that is - maybe something doesn't properly deal with BOs getting placed differently in some cases now - but anyway I suspect the implications of this change haven't been fully thought through: The log message sounds as though the change was mainly written with radeon_bo_create() / radeon_bo_list_validate() in mind, but radeon_ttm_placement_from_domain() is also called from other places:

Quoted 4 lines

     * radeon_bo_pin(): The change could lead to a BO being pinned to GTT instead of VRAM, which would probably be bad.
     * radeon_evict_flags(): The change might have undesirable consequences here as well, not sure.

The first might be bad, but the second should be okay, I'll take a closer look in the morning.

Quoted 4 lines

I don't think the way we currently use the same arrays for normal and busy placement makes a lot of sense, but we probably need a better mechanism to specify which placements are desirable / acceptable in each case.

Well its not really a huge matrix of choices, I'm guessing we need more info from userspace, or use the info better. Though in theory everything should work in GTT or VRAM equally well just slower.

Dave.

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
--

#6 lore raw
Quoted 10 lines · Dave Airlie

2010/3/25 Michel Dänzer <[email protected]>:

Quoted 9 lines · Dave Airlie
Quoted 7 lines

From: Dave Airlie <[email protected]>

On constrained r100 systems compiz would fail to start due to a lack of memory, we can just fallback place the objects rather than completely failing it works a lot better.

Quoted 7 lines
Quoted 4 lines

This change seems to trigger or at least greatly expedite GPU lockups on my PowerBook. With the change applied, my normal X session locked up the GPU after just a few minutes several times. Now with it reverted it's back to the previous stability.

Care to try in pci mode? see if helps, it might be just straining AGP a bit more,

Ugh, k I'll try... but that incurs such a huge performance hit that the result might not be very meaningful I'm afraid.

Quoted 1 line

maybe try 1x as well if you aren't already in it.

I am, for some reason neither 2x nor 4x has ever worked reliably with KMS although 4x was rock solid with UMS.

Quoted 7 lines
Quoted 7 lines

I don't know why that is - maybe something doesn't properly deal with BOs getting placed differently in some cases now - but anyway I suspect the implications of this change haven't been fully thought through: The log message sounds as though the change was mainly written with radeon_bo_create() / radeon_bo_list_validate() in mind, but radeon_ttm_placement_from_domain() is also called from other places:

Quoted 7 lines
Quoted 4 lines

* radeon_bo_pin(): The change could lead to a BO being pinned to GTT instead of VRAM, which would probably be bad.
* radeon_evict_flags(): The change might have undesirable consequences here as well, not sure.

The first might be bad, but the second should be okay, I'll take a closer look in the morning.

What about that there are now usually no busy placements specified, couldn't that be a problem?

Quoted 8 lines
Quoted 4 lines

I don't think the way we currently use the same arrays for normal and busy placement makes a lot of sense, but we probably need a better mechanism to specify which placements are desirable / acceptable in each case.

Well its not really a huge matrix of choices, I'm guessing we need more info from userspace, or use the info better. Though in theory everything should work in GTT or VRAM equally well just slower.

E.g. scanout from GTT could easily exceed the available bandwidth.

Signature · 14 lines
Earthling Michel Dänzer           |                http://www.vmware.com
Libre software enthusiast         |          Debian, X and DRI developer

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
--
_______________________________________________
Dri-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dri-devel
#7 lore raw
Quoted 16 lines · Michel Dänzer
Quoted 15 lines · Dave Airlie

2010/3/25 Michel Dänzer <[email protected]>:

Quoted 13 lines · Dave Airlie
Quoted 7 lines

From: Dave Airlie <[email protected]>

On constrained r100 systems compiz would fail to start due to a lack of memory, we can just fallback place the objects rather than completely failing it works a lot better.

This change seems to trigger or at least greatly expedite GPU lockups on my PowerBook. With the change applied, my normal X session locked up the GPU after just a few minutes several times. Now with it reverted it's back to the previous stability.

Quoted 5 lines
Quoted 2 lines

Care to try in pci mode? see if helps, it might be just straining AGP a bit more,

Ugh, k I'll try... but that incurs such a huge performance hit that the result might not be very meaningful I'm afraid.

It didn't lock up in a couple of hours of suffering through PCI, so maybe it is an AGP problem, or maybe PCI is just too slow to trigger it... More likely the former though I guess.

Quoted 12 lines
Quoted 12 lines
Quoted 11 lines

I don't know why that is - maybe something doesn't properly deal with BOs getting placed differently in some cases now - but anyway I suspect the implications of this change haven't been fully thought through: The log message sounds as though the change was mainly written with radeon_bo_create() / radeon_bo_list_validate() in mind, but radeon_ttm_placement_from_domain() is also called from other places:

* radeon_bo_pin(): The change could lead to a BO being pinned to GTT instead of VRAM, which would probably be bad.
* radeon_evict_flags(): The change might have undesirable consequences here as well, not sure.

Quoted 5 lines
Quoted 2 lines

The first might be bad, but the second should be okay, I'll take a closer look in the morning.

What about that there are now usually no busy placements specified, couldn't that be a problem?

FWIW I tried re-using the normal placements for missing busy placements, didn't help.

Signature · 14 lines
Earthling Michel Dänzer           |                http://www.vmware.com
Libre software enthusiast         |          Debian, X and DRI developer

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
--
_______________________________________________
Dri-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dri-devel
#8 lore raw
In reply to Dave Airlie
Quoted 10 lines · Dave Airlie

2010/3/25 Michel Dänzer <[email protected]>:

Quoted 9 lines · Dave Airlie
Quoted 7 lines

From: Dave Airlie <[email protected]>

On constrained r100 systems compiz would fail to start due to a lack of memory, we can just fallback place the objects rather than completely failing it works a lot better.

Quoted 7 lines
Quoted 4 lines

This change seems to trigger or at least greatly expedite GPU lockups on my PowerBook. With the change applied, my normal X session locked up the GPU after just a few minutes several times. Now with it reverted it's back to the previous stability.

Care to try in pci mode? see if helps, it might be just straining AGP a bit more, maybe try 1x as well if you aren't already in it.

After various more tests, the gist of it seems to be that rendering to GTT with the 3D engine tends to lock up whereas the 2D and DMA engines work fine (otherwise I probably would have run into this a long time ago at BO eviction time). Maybe some kind of synchronization issue resulting in the 3D engine trying to access GTT memory which isn't bound yet / anymore?

Quoted 7 lines
Quoted 7 lines

I don't know why that is - maybe something doesn't properly deal with BOs getting placed differently in some cases now - but anyway I suspect the implications of this change haven't been fully thought through: The log message sounds as though the change was mainly written with radeon_bo_create() / radeon_bo_list_validate() in mind, but radeon_ttm_placement_from_domain() is also called from other places:

Quoted 6 lines
Quoted 4 lines

* radeon_bo_pin(): The change could lead to a BO being pinned to GTT instead of VRAM, which would probably be bad.
* radeon_evict_flags(): The change might have undesirable consequences here as well, not sure.

The first might be bad, [...]

I actually saw this happen once for the scanout BO of a second X server, and it resulted in pretty spectacular cascaded failure. This confirmed my suspicion that it's a very bad idea for radeon_bo_pin().

In summary I'm afraid this isn't really a good solution for what you were trying to achieve, and it's actually breaking things that were working before, so I think it should be reverted.

Signature · 14 lines
Earthling Michel Dänzer           |                http://www.vmware.com
Libre software enthusiast         |          Debian, X and DRI developer

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
--
_______________________________________________
Dri-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dri-devel
#9 lore raw
Quoted 19 lines · Michel Dänzer
Quoted 10 lines · Dave Airlie

2010/3/25 Michel Dänzer <[email protected]>:

Quoted 9 lines · Dave Airlie
Quoted 7 lines

From: Dave Airlie <[email protected]>

On constrained r100 systems compiz would fail to start due to a lack of memory, we can just fallback place the objects rather than completely failing it works a lot better.

Quoted 7 lines
Quoted 4 lines

This change seems to trigger or at least greatly expedite GPU lockups on my PowerBook. With the change applied, my normal X session locked up the GPU after just a few minutes several times. Now with it reverted it's back to the previous stability.

Care to try in pci mode? see if helps, it might be just straining AGP a bit more, maybe try 1x as well if you aren't already in it.

Quoted 6 lines

After various more tests, the gist of it seems to be that rendering to GTT with the 3D engine tends to lock up whereas the 2D and DMA engines work fine (otherwise I probably would have run into this a long time ago at BO eviction time). Maybe some kind of synchronization issue resulting in the 3D engine trying to access GTT memory which isn't bound yet / anymore?

I think I've confirmed this theory by changing the AGP driver to bind the AGP scratch page instead of really unbinding a GTT entry. That prevents the lockups from occurring. Even so though, this change hurts performance, presumably because BOs evicted from VRAM (or not going in in the first place) aren't getting (back) in.

Signature · 14 lines
Earthling Michel Dänzer           |                http://www.vmware.com
Libre software enthusiast         |          Debian, X and DRI developer

------------------------------------------------------------------------------
Download Intel&#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
--
_______________________________________________
Dri-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dri-devel