Skip to contentSkip to conversation

New instmem implementation for Tegra

PATCH Superseded Alexandre Courbot 3 participants 8 messages Sep 3, 2015 6 messages not in the archive

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover New instmem implementation for Tegra

Due to the lack of implicit synchronization between CPU and GPU on Tegra systems (like what PCI provides for dGPUs), the instmem implementation of GK20A/GM20B relied on the slow, legacy PRAMIN so that CPU accesses used the same path as GPU, making sure we see the same data on both ends.

The recent Nouveau refactoring introduced acquire/release functions on instmem that help us understand the bounds of accesses to instobjs. We can now use these to perform any needed synchronization, and use direct CPU accesses. For Tegra, provided that we use a write-combined CPU mapping, we just need to make sure the GPU L2 cache is flushed before we access instobjs and invalidated after.

This patch series first introduces flush/invalidate hooks on LTC so we can manage the L2 ourselves, and an implementation for GF100+ GPUs. The last patch then changes GK20A's instmem implementation to use CPU mappings and manage the GPU L2 in the new acquire/release hooks.

This results in a lot of L2 operations, but thankfully flushing/invalidating an empty L2 returns immediately, so the performance gain is largely positive. In the future it would be nice to move L2 management to a higher level though.

Alexandre Courbot (3):
ltc: add hooks for invalidate and flush
ltc/gf100: add flush/invalidate functions
instmem/gk20a: use direct CPU writes

 drm/nouveau/include/nvkm/subdev/ltc.h   |   4 +
 drm/nouveau/nvkm/subdev/instmem/gk20a.c | 360 +++++++++++++++++++++++---------
 drm/nouveau/nvkm/subdev/ltc/base.c      |  14 ++
 drm/nouveau/nvkm/subdev/ltc/gf100.c     |  36 ++++
 drm/nouveau/nvkm/subdev/ltc/gk104.c     |   2 +
 drm/nouveau/nvkm/subdev/ltc/gm107.c     |   2 +
 drm/nouveau/nvkm/subdev/ltc/priv.h      |   5 +
 7 files changed, 326 insertions(+), 97 deletions(-)
Signature · 6 lines
2.5.1

_______________________________________________
Nouveau mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/nouveau

A message that is not in the archive (sent off-list or not yet received): 1441262566-4141-1-git-send-email-acourbot-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org

#2 lore raw

PATCH 1/3 ltc: add hooks for invalidate and flush

These are useful for systems without a coherent CPU/GPU bus. For such systems we may need to maintain the L2 ourselves.

---
 drm/nouveau/include/nvkm/subdev/ltc.h |  3 +++
 drm/nouveau/nvkm/subdev/ltc/base.c    | 14 ++++++++++++++
 drm/nouveau/nvkm/subdev/ltc/priv.h    |  3 +++
 3 files changed, 20 insertions(+)
drm/nouveau/include/nvkm/subdev/ltc.h +3 -0
index c773b5e958b4..5464fcf482f1 100644
@@ -30,6 +30,9 @@ void nvkm_ltc_tags_clear(struct nvkm_ltc *, u32 first, u32 count);
 int nvkm_ltc_zbc_color_get(struct nvkm_ltc *, int index, const u32[4]);
 int nvkm_ltc_zbc_depth_get(struct nvkm_ltc *, int index, const u32);
 
+void nvkm_ltc_invalidate(struct nvkm_ltc *);
+void nvkm_ltc_flush(struct nvkm_ltc *);
+
 int gf100_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
 int gk104_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
 int gm107_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
drm/nouveau/nvkm/subdev/ltc/base.c +14 -0
index 930d25b6e63c..85b1464c0194 100644
@@ -67,6 +67,20 @@ nvkm_ltc_zbc_depth_get(struct nvkm_ltc *ltc, int index, const u32 depth)
 	return index;
 }
 
+void
+nvkm_ltc_invalidate(struct nvkm_ltc *ltc)
+{
+	if (ltc->func->invalidate)
+		ltc->func->invalidate(ltc);
+}
+
+void
+nvkm_ltc_flush(struct nvkm_ltc *ltc)
+{
+	if (ltc->func->flush)
+		ltc->func->flush(ltc);
+}
+
 static void
 nvkm_ltc_intr(struct nvkm_subdev *subdev)
 {
drm/nouveau/nvkm/subdev/ltc/priv.h +3 -0
index 4e05037cc99f..6f66bd03f829 100644
@@ -17,6 +17,9 @@ struct nvkm_ltc_func {
 	int zbc;
 	void (*zbc_clear_color)(struct nvkm_ltc *, int, const u32[4]);
 	void (*zbc_clear_depth)(struct nvkm_ltc *, int, const u32);
+
+	void (*invalidate)(struct nvkm_ltc *);
+	void (*flush)(struct nvkm_ltc *);
 };
 
 int gf100_ltc_oneinit(struct nvkm_ltc *);
Signature · 6 lines
2.5.1

_______________________________________________
Nouveau mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/nouveau
#3 lore raw

PATCH 2/3 ltc/gf100: add flush/invalidate functions

In reply to a message not in the archive

Allow clients to manually flush and invalidate L2. This will be useful for Tegra systems for which we want to write instmem using the CPU.

---
 drm/nouveau/include/nvkm/subdev/ltc.h |  1 +
 drm/nouveau/nvkm/subdev/ltc/gf100.c   | 36 +++++++++++++++++++++++++++++++++++
 drm/nouveau/nvkm/subdev/ltc/gk104.c   |  2 ++
 drm/nouveau/nvkm/subdev/ltc/gm107.c   |  2 ++
 drm/nouveau/nvkm/subdev/ltc/priv.h    |  2 ++
 5 files changed, 43 insertions(+)
drm/nouveau/include/nvkm/subdev/ltc.h +1 -0
index 5464fcf482f1..3d4dbbf9aab3 100644
@@ -35,5 +35,6 @@ void nvkm_ltc_flush(struct nvkm_ltc *);
 
 int gf100_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
 int gk104_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
+int gk20a_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
 int gm107_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
 #endif
drm/nouveau/nvkm/subdev/ltc/gf100.c +36 -0
index 45ac765b753e..42ae77533865 100644
@@ -122,6 +122,40 @@ gf100_ltc_intr(struct nvkm_ltc *ltc)
 	}
 }
 
+void
+gf100_ltc_invalidate(struct nvkm_ltc *ltc)
+{
+	struct nvkm_device *device = ltc->subdev.device;
+	s64 taken;
+
+	nvkm_wr32(device, 0x70004, 0x00000001);
+	if ((taken = nvkm_msec(device, 2000,
+		if ((nvkm_rd32(device, 0x70004) & 0x00000003) == 0x00000000)
+			break;
+	)) < 0)
+		nvkm_warn(&ltc->subdev, "L2 invalidate timeout\n");
+
+	if (taken > 0)
+		nvkm_debug(&ltc->subdev, "LTC invalidate took %lld ns\n", taken);
+}
+
+void
+gf100_ltc_flush(struct nvkm_ltc *ltc)
+{
+	struct nvkm_device *device = ltc->subdev.device;
+	s64 taken;
+
+	nvkm_wr32(device, 0x70010, 0x00000001);
+	if ((taken = nvkm_msec(device, 2000,
+		if ((nvkm_rd32(device, 0x70010) & 0x00000003) == 0x00000000)
+			break;
+	)) < 0)
+		nvkm_warn(&ltc->subdev, "L2 flush timeout\n");
+
+	if (taken > 0)
+		nvkm_debug(&ltc->subdev, "LTC flush took %lld ns\n", taken);
+}
+
 /* TODO: Figure out tag memory details and drop the over-cautious allocation.
  */
 int
@@ -215,6 +249,8 @@ gf100_ltc = {
 	.zbc = 16,
 	.zbc_clear_color = gf100_ltc_zbc_clear_color,
 	.zbc_clear_depth = gf100_ltc_zbc_clear_depth,
+	.invalidate = gf100_ltc_invalidate,
+	.flush = gf100_ltc_flush,
 };
 
 int
drm/nouveau/nvkm/subdev/ltc/gk104.c +2 -0
index 839e6b4c597b..b4f6e0034d58 100644
@@ -45,6 +45,8 @@ gk104_ltc = {
 	.zbc = 16,
 	.zbc_clear_color = gf100_ltc_zbc_clear_color,
 	.zbc_clear_depth = gf100_ltc_zbc_clear_depth,
+	.invalidate = gf100_ltc_invalidate,
+	.flush = gf100_ltc_flush,
 };
 
 int
drm/nouveau/nvkm/subdev/ltc/gm107.c +2 -0
index 389331bb63ba..3043bbfd7384 100644
@@ -138,6 +138,8 @@ gm107_ltc = {
 	.zbc = 16,
 	.zbc_clear_color = gm107_ltc_zbc_clear_color,
 	.zbc_clear_depth = gm107_ltc_zbc_clear_depth,
+	.invalidate = gf100_ltc_invalidate,
+	.flush = gf100_ltc_flush,
 };
 
 int
drm/nouveau/nvkm/subdev/ltc/priv.h +2 -0
index 6f66bd03f829..4e3755b82769 100644
@@ -29,4 +29,6 @@ void gf100_ltc_cbc_clear(struct nvkm_ltc *, u32, u32);
 void gf100_ltc_cbc_wait(struct nvkm_ltc *);
 void gf100_ltc_zbc_clear_color(struct nvkm_ltc *, int, const u32[4]);
 void gf100_ltc_zbc_clear_depth(struct nvkm_ltc *, int, const u32);
+void gf100_ltc_invalidate(struct nvkm_ltc *);
+void gf100_ltc_flush(struct nvkm_ltc *);
 #endif
Signature · 6 lines
2.5.1

_______________________________________________
Nouveau mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/nouveau

A message that is not in the archive (sent off-list or not yet received): 1441262566-4141-3-git-send-email-acourbot-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org

#5 lore raw

Re: [PATCH 2/3] ltc/gf100: add flush/invalidate functions

Quoted 29 lines · Alexandre Courbot

Allow clients to manually flush and invalidate L2. This will be useful for Tegra systems for which we want to write instmem using the CPU.

---
 drm/nouveau/include/nvkm/subdev/ltc.h |  1 +
 drm/nouveau/nvkm/subdev/ltc/gf100.c   | 36 +++++++++++++++++++++++++++++++++++
 drm/nouveau/nvkm/subdev/ltc/gk104.c   |  2 ++
 drm/nouveau/nvkm/subdev/ltc/gm107.c   |  2 ++
 drm/nouveau/nvkm/subdev/ltc/priv.h    |  2 ++
 5 files changed, 43 insertions(+)
drm/nouveau/include/nvkm/subdev/ltc.h +1 -0
index 5464fcf482f1..3d4dbbf9aab3 100644
@@ -35,5 +35,6 @@ void nvkm_ltc_flush(struct nvkm_ltc *);
 
 int gf100_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
 int gk104_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
+int gk20a_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
 int gm107_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
 #endif
drm/nouveau/nvkm/subdev/ltc/gf100.c +0 -0
index 45ac765b753e..42ae77533865 100644
@@ -122,6 +122,40 @@ gf100_ltc_intr(struct nvkm_ltc *ltc)
        }

drm/nouveau/nvkm/subdev/ltc/gf100.c gf100_ltc_intr(struct nvkm_ltc *ltc) · line 132

Quoted 10 lines
drm/nouveau/nvkm/subdev/ltc/gf100.c +8 -0
 }
 
+void
+gf100_ltc_invalidate(struct nvkm_ltc *ltc)
+{
+       struct nvkm_device *device = ltc->subdev.device;
+       s64 taken;
+
+       nvkm_wr32(device, 0x70004, 0x00000001);
+       if ((taken = nvkm_msec(device, 2000,

I don't suppose you have access to information on more realistic timeouts? I'd like to improve all the potential 2s timeout values across the driver in general, to avoid things hanging for a long time when the GPU craps itself :)

Quoted 78 lines
drm/nouveau/nvkm/subdev/ltc/gf100.c +28 -0
+               if ((nvkm_rd32(device, 0x70004) & 0x00000003) == 0x00000000)
+                       break;
+       )) < 0)
+               nvkm_warn(&ltc->subdev, "L2 invalidate timeout\n");
+
+       if (taken > 0)
+               nvkm_debug(&ltc->subdev, "LTC invalidate took %lld ns\n", taken);
+}
+
+void
+gf100_ltc_flush(struct nvkm_ltc *ltc)
+{
+       struct nvkm_device *device = ltc->subdev.device;
+       s64 taken;
+
+       nvkm_wr32(device, 0x70010, 0x00000001);
+       if ((taken = nvkm_msec(device, 2000,
+               if ((nvkm_rd32(device, 0x70010) & 0x00000003) == 0x00000000)
+                       break;
+       )) < 0)
+               nvkm_warn(&ltc->subdev, "L2 flush timeout\n");
+
+       if (taken > 0)
+               nvkm_debug(&ltc->subdev, "LTC flush took %lld ns\n", taken);
+}
+
 /* TODO: Figure out tag memory details and drop the over-cautious allocation.
  */
 int
@@ -215,6 +249,8 @@ gf100_ltc = {
        .zbc = 16,
        .zbc_clear_color = gf100_ltc_zbc_clear_color,
        .zbc_clear_depth = gf100_ltc_zbc_clear_depth,
+       .invalidate = gf100_ltc_invalidate,
+       .flush = gf100_ltc_flush,
 };
 
 int
drm/nouveau/nvkm/subdev/ltc/gk104.c +2 -0
index 839e6b4c597b..b4f6e0034d58 100644
@@ -45,6 +45,8 @@ gk104_ltc = {
        .zbc = 16,
        .zbc_clear_color = gf100_ltc_zbc_clear_color,
        .zbc_clear_depth = gf100_ltc_zbc_clear_depth,
+       .invalidate = gf100_ltc_invalidate,
+       .flush = gf100_ltc_flush,
 };
 
 int
drm/nouveau/nvkm/subdev/ltc/gm107.c +2 -0
index 389331bb63ba..3043bbfd7384 100644
@@ -138,6 +138,8 @@ gm107_ltc = {
        .zbc = 16,
        .zbc_clear_color = gm107_ltc_zbc_clear_color,
        .zbc_clear_depth = gm107_ltc_zbc_clear_depth,
+       .invalidate = gf100_ltc_invalidate,
+       .flush = gf100_ltc_flush,
 };
 
 int
drm/nouveau/nvkm/subdev/ltc/priv.h +2 -1
index 6f66bd03f829..4e3755b82769 100644
@@ -29,4 +29,6 @@ void gf100_ltc_cbc_clear(struct nvkm_ltc *, u32, u32);
 void gf100_ltc_cbc_wait(struct nvkm_ltc *);
 void gf100_ltc_zbc_clear_color(struct nvkm_ltc *, int, const u32[4]);
 void gf100_ltc_zbc_clear_depth(struct nvkm_ltc *, int, const u32);
+void gf100_ltc_invalidate(struct nvkm_ltc *);
+void gf100_ltc_flush(struct nvkm_ltc *);
 #endif
--

2.5.1

Quoted 4 lines

_______________________________________________
Nouveau mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/nouveau

_______________________________________________
Nouveau mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/nouveau

A message that is not in the archive (sent off-list or not yet received): CACAvsv6vbLBHGemF6jCAO=43cBK+u45Nbiw4dQZNENkx3=kgMQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#7 lore raw

Re: [PATCH 2/3] ltc/gf100: add flush/invalidate functions

Quoted 32 lines · Ben Skeggs
Quoted 31 lines · Alexandre Courbot

Allow clients to manually flush and invalidate L2. This will be useful for Tegra systems for which we want to write instmem using the CPU.

---
 drm/nouveau/include/nvkm/subdev/ltc.h |  1 +
 drm/nouveau/nvkm/subdev/ltc/gf100.c   | 36 +++++++++++++++++++++++++++++++++++
 drm/nouveau/nvkm/subdev/ltc/gk104.c   |  2 ++
 drm/nouveau/nvkm/subdev/ltc/gm107.c   |  2 ++
 drm/nouveau/nvkm/subdev/ltc/priv.h    |  2 ++
 5 files changed, 43 insertions(+)
drm/nouveau/include/nvkm/subdev/ltc.h +1 -0
index 5464fcf482f1..3d4dbbf9aab3 100644
@@ -35,5 +35,6 @@ void nvkm_ltc_flush(struct nvkm_ltc *);
 
 int gf100_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
 int gk104_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
+int gk20a_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
 int gm107_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
 #endif
drm/nouveau/nvkm/subdev/ltc/gf100.c +0 -0
index 45ac765b753e..42ae77533865 100644
@@ -122,6 +122,40 @@ gf100_ltc_intr(struct nvkm_ltc *ltc)
        }
 }
 
Quoted 12 lines

drm/nouveau/nvkm/subdev/ltc/gf100.c gf100_ltc_intr(struct nvkm_ltc *ltc) · line 132

Quoted 8 lines
drm/nouveau/nvkm/subdev/ltc/gf100.c +8 -0
+void
+gf100_ltc_invalidate(struct nvkm_ltc *ltc)
+{
+       struct nvkm_device *device = ltc->subdev.device;
+       s64 taken;
+
+       nvkm_wr32(device, 0x70004, 0x00000001);
+       if ((taken = nvkm_msec(device, 2000,

I don't suppose you have access to information on more realistic timeouts? I'd like to improve all the potential 2s timeout values across the driver in general, to avoid things hanging for a long time when the GPU craps itself :)

The longest values I have ever seen are ~270ms (only during driver initialization, and at low clock speeds), but anyway this should never ever timeout. If it does, then we will have bigger issues than 2 second timeouts. :)

Btw, wouldn't it be worth to restore macros like the old nvkm_wait() to avoid having to write this verbose code every time we need to wait on a register value?
_______________________________________________
Nouveau mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/nouveau

A message that is not in the archive (sent off-list or not yet received): CAAVeFuLp8O995rE6v7gnx=rJBeu3=r=EkhMU5G89HBC8hry+Sg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#9 lore raw

Re: [PATCH 2/3] ltc/gf100: add flush/invalidate functions

Quoted 46 lines · Alexandre Courbot
Quoted 32 lines · Ben Skeggs
Quoted 31 lines · Alexandre Courbot

Allow clients to manually flush and invalidate L2. This will be useful for Tegra systems for which we want to write instmem using the CPU.

---
 drm/nouveau/include/nvkm/subdev/ltc.h |  1 +
 drm/nouveau/nvkm/subdev/ltc/gf100.c   | 36 +++++++++++++++++++++++++++++++++++
 drm/nouveau/nvkm/subdev/ltc/gk104.c   |  2 ++
 drm/nouveau/nvkm/subdev/ltc/gm107.c   |  2 ++
 drm/nouveau/nvkm/subdev/ltc/priv.h    |  2 ++
 5 files changed, 43 insertions(+)
drm/nouveau/include/nvkm/subdev/ltc.h +1 -0
index 5464fcf482f1..3d4dbbf9aab3 100644
@@ -35,5 +35,6 @@ void nvkm_ltc_flush(struct nvkm_ltc *);
 
 int gf100_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
 int gk104_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
+int gk20a_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
 int gm107_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
 #endif
drm/nouveau/nvkm/subdev/ltc/gf100.c +0 -0
index 45ac765b753e..42ae77533865 100644
@@ -122,6 +122,40 @@ gf100_ltc_intr(struct nvkm_ltc *ltc)
        }
 }
 
Quoted 12 lines

drm/nouveau/nvkm/subdev/ltc/gf100.c gf100_ltc_intr(struct nvkm_ltc *ltc) · line 132

Quoted 8 lines
drm/nouveau/nvkm/subdev/ltc/gf100.c +8 -0
+void
+gf100_ltc_invalidate(struct nvkm_ltc *ltc)
+{
+       struct nvkm_device *device = ltc->subdev.device;
+       s64 taken;
+
+       nvkm_wr32(device, 0x70004, 0x00000001);
+       if ((taken = nvkm_msec(device, 2000,

I don't suppose you have access to information on more realistic timeouts? I'd like to improve all the potential 2s timeout values across the driver in general, to avoid things hanging for a long time when the GPU craps itself :)

Quoted 4 lines

The longest values I have ever seen are ~270ms (only during driver initialization, and at low clock speeds), but anyway this should never ever timeout. If it does, then we will have bigger issues than 2 second timeouts. :)

Indeed it shouldn't, but when things get horribly messed up (as happens occasionally :P) it's annoying and can potentially stall the whole system until things are sorted out. The patch is fine as-is, I was mostly wondering if NVIDIA have data on a maximum time this should ever take that we could use instead.

Quoted 4 lines

Btw, wouldn't it be worth to restore macros like the old nvkm_wait() to avoid having to write this verbose code every time we need to wait on a register value?

I'm fine doing that, the reason I didn't in the first place was that I'd planned on not having the constant 2s timeout everywhere (as already mentioned) :)
_______________________________________________
Nouveau mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/nouveau

A message that is not in the archive (sent off-list or not yet received): CACAvsv7G-=OxciY7gUkaofM_qf2Efx3uPxaFCad8Dt6rzK--9A-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#11 lore raw

Re: [PATCH 2/3] ltc/gf100: add flush/invalidate functions

Quoted 51 lines · Ben Skeggs
Quoted 46 lines · Alexandre Courbot
Quoted 32 lines · Ben Skeggs
Quoted 31 lines · Alexandre Courbot

Allow clients to manually flush and invalidate L2. This will be useful for Tegra systems for which we want to write instmem using the CPU.

---
 drm/nouveau/include/nvkm/subdev/ltc.h |  1 +
 drm/nouveau/nvkm/subdev/ltc/gf100.c   | 36 +++++++++++++++++++++++++++++++++++
 drm/nouveau/nvkm/subdev/ltc/gk104.c   |  2 ++
 drm/nouveau/nvkm/subdev/ltc/gm107.c   |  2 ++
 drm/nouveau/nvkm/subdev/ltc/priv.h    |  2 ++
 5 files changed, 43 insertions(+)
drm/nouveau/include/nvkm/subdev/ltc.h +1 -0
index 5464fcf482f1..3d4dbbf9aab3 100644
@@ -35,5 +35,6 @@ void nvkm_ltc_flush(struct nvkm_ltc *);
 
 int gf100_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
 int gk104_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
+int gk20a_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
 int gm107_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
 #endif
drm/nouveau/nvkm/subdev/ltc/gf100.c +0 -0
index 45ac765b753e..42ae77533865 100644
@@ -122,6 +122,40 @@ gf100_ltc_intr(struct nvkm_ltc *ltc)
        }
 }
 
Quoted 12 lines

drm/nouveau/nvkm/subdev/ltc/gf100.c gf100_ltc_intr(struct nvkm_ltc *ltc) · line 132

Quoted 8 lines
drm/nouveau/nvkm/subdev/ltc/gf100.c +8 -0
+void
+gf100_ltc_invalidate(struct nvkm_ltc *ltc)
+{
+       struct nvkm_device *device = ltc->subdev.device;
+       s64 taken;
+
+       nvkm_wr32(device, 0x70004, 0x00000001);
+       if ((taken = nvkm_msec(device, 2000,

I don't suppose you have access to information on more realistic timeouts? I'd like to improve all the potential 2s timeout values across the driver in general, to avoid things hanging for a long time when the GPU craps itself :)

Quoted 4 lines

The longest values I have ever seen are ~270ms (only during driver initialization, and at low clock speeds), but anyway this should never ever timeout. If it does, then we will have bigger issues than 2 second timeouts. :)

Quoted 5 lines

Indeed it shouldn't, but when things get horribly messed up (as happens occasionally :P) it's annoying and can potentially stall the whole system until things are sorted out. The patch is fine as-is, I was mostly wondering if NVIDIA have data on a maximum time this should ever take that we could use instead.

In nvgpu we do 200 retries with a delay of 5us in between, so we wait at most for 1ms. Yeah, that sounds like a better measure than 2s. :)

Btw, when I wrote 270ms, I meant 270us.

1ms would be nice in that case. Let me test with it and resend this patch with better values.

Quoted 8 lines
Quoted 5 lines

Btw, wouldn't it be worth to restore macros like the old nvkm_wait() to avoid having to write this verbose code every time we need to wait on a register value?

I'm fine doing that, the reason I didn't in the first place was that I'd planned on not having the constant 2s timeout everywhere (as already mentioned) :)

We can just add one extra argument that specifies how long we are willing to wait. :P
_______________________________________________
Nouveau mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/nouveau

A message that is not in the archive (sent off-list or not yet received): CAAVeFu+rJGAML-D5SsivRcv6zMm6-KdKhKV61gqsbSgsVJ3fpA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#13 lore raw

Re: [PATCH 2/3] ltc/gf100: add flush/invalidate functions

Quoted 71 lines · Alexandre Courbot
Quoted 51 lines · Ben Skeggs
Quoted 46 lines · Alexandre Courbot
Quoted 32 lines · Ben Skeggs
Quoted 31 lines · Alexandre Courbot

Allow clients to manually flush and invalidate L2. This will be useful for Tegra systems for which we want to write instmem using the CPU.

---
 drm/nouveau/include/nvkm/subdev/ltc.h |  1 +
 drm/nouveau/nvkm/subdev/ltc/gf100.c   | 36 +++++++++++++++++++++++++++++++++++
 drm/nouveau/nvkm/subdev/ltc/gk104.c   |  2 ++
 drm/nouveau/nvkm/subdev/ltc/gm107.c   |  2 ++
 drm/nouveau/nvkm/subdev/ltc/priv.h    |  2 ++
 5 files changed, 43 insertions(+)
drm/nouveau/include/nvkm/subdev/ltc.h +1 -0
index 5464fcf482f1..3d4dbbf9aab3 100644
@@ -35,5 +35,6 @@ void nvkm_ltc_flush(struct nvkm_ltc *);
 
 int gf100_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
 int gk104_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
+int gk20a_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
 int gm107_ltc_new(struct nvkm_device *, int, struct nvkm_ltc **);
 #endif
drm/nouveau/nvkm/subdev/ltc/gf100.c +0 -0
index 45ac765b753e..42ae77533865 100644
@@ -122,6 +122,40 @@ gf100_ltc_intr(struct nvkm_ltc *ltc)
        }
 }
 
Quoted 12 lines

drm/nouveau/nvkm/subdev/ltc/gf100.c gf100_ltc_intr(struct nvkm_ltc *ltc) · line 132

Quoted 8 lines
drm/nouveau/nvkm/subdev/ltc/gf100.c +8 -0
+void
+gf100_ltc_invalidate(struct nvkm_ltc *ltc)
+{
+       struct nvkm_device *device = ltc->subdev.device;
+       s64 taken;
+
+       nvkm_wr32(device, 0x70004, 0x00000001);
+       if ((taken = nvkm_msec(device, 2000,

I don't suppose you have access to information on more realistic timeouts? I'd like to improve all the potential 2s timeout values across the driver in general, to avoid things hanging for a long time when the GPU craps itself :)

Quoted 4 lines

The longest values I have ever seen are ~270ms (only during driver initialization, and at low clock speeds), but anyway this should never ever timeout. If it does, then we will have bigger issues than 2 second timeouts. :)

Quoted 5 lines

Indeed it shouldn't, but when things get horribly messed up (as happens occasionally :P) it's annoying and can potentially stall the whole system until things are sorted out. The patch is fine as-is, I was mostly wondering if NVIDIA have data on a maximum time this should ever take that we could use instead.

In nvgpu we do 200 retries with a delay of 5us in between, so we wait at most for 1ms. Yeah, that sounds like a better measure than 2s. :)

Btw, when I wrote 270ms, I meant 270us.

1ms would be nice in that case. Let me test with it and resend this patch with better values.

Quoted 5 lines
Quoted 5 lines

Btw, wouldn't it be worth to restore macros like the old nvkm_wait() to avoid having to write this verbose code every time we need to wait on a register value?

Quoted 6 lines
Quoted 3 lines

I'm fine doing that, the reason I didn't in the first place was that I'd planned on not having the constant 2s timeout everywhere (as already mentioned) :)

We can just add one extra argument that specifies how long we are willing to wait. :P

Works for me :)
_______________________________________________
Nouveau mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/nouveau