Skip to contentSkip to conversation

drm/nouveau: Preparatory work for GV11B support

PATCH New Thierry Reding 4 participants 14 messages Sep 16, 2019 – Sep 20, 2019 5 messages not in the archive

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover drm/nouveau: Preparatory work for GV11B support

From: Thierry Reding <[email protected]>

Hi Ben,

these are a couple of patches that are in preparation for adding GV11B support. The fundamental issue that these are trying to solve is that the GV11B is the first Tegra incarnation of the GPU where the aperture really matters. All prior generations would accept any of them.

For dGPUs we usually allocate memory in VRAM, so the default aperture
(0) is correct. However, on Tegra the buffers are allocated in system memory, and since the GPU actually cares about the aperture, we need to ensure that the aperture field is written in all the necessary places.

This series of patches does three things: the first two patches make it easier to debug aperture related faults by actually reading the aperture information from the fault information registers. The second patch is actually only a small cleanup.

Patches 3-5 unify the aperture values. All generations have the same definitions for these, so there's little use in separating them out into callbacks.

Finally, patch 6 writes the aperture field in the places where required. I've used these patches to test my initial support for GV11B. This is enough to get me through the driver probe without any faults, but I have not made much progress on secboot support yet, so I can't use the GV11B to do anything very interesting yet.

I should also note that this is completely untested on dGPU because I don't currently have a way of testing them. I'm working on that, but in the meantime it'd be great if somebody could give this set a quick spin on a dGPU to confirm that these don't break.

Thierry

Thierry Reding (6):
drm/nouveau: fault: Store aperture in fault information drm/nouveau: fault: Widen engine field
drm/nouveau: Remove bogus gk20a aperture callback drm/nouveau: Implement nvkm_memory_aperture() drm/nouveau: Remove unused nvkm_vmm_func->aper() implementations drm/nouveau: Program aperture field where necessary

 .../drm/nouveau/include/nvkm/core/memory.h    | 28 +++++++++++++++++++
 .../drm/nouveau/include/nvkm/subdev/fault.h   |  1 +
 .../gpu/drm/nouveau/nvkm/engine/fifo/gk104.c  |  3 +-
 .../nouveau/nvkm/engine/fifo/gpfifogk104.c    |  7 +++--
 .../nouveau/nvkm/engine/fifo/gpfifogv100.c    |  5 ++--
 .../gpu/drm/nouveau/nvkm/engine/fifo/gv100.c  |  7 ++++-
 .../gpu/drm/nouveau/nvkm/subdev/bar/gf100.c   |  3 +-
 .../gpu/drm/nouveau/nvkm/subdev/fault/gv100.c |  3 +-
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.h |  3 --
 .../drm/nouveau/nvkm/subdev/mmu/vmmgf100.c    | 21 ++------------
 .../drm/nouveau/nvkm/subdev/mmu/vmmgk104.c    |  2 --
 .../drm/nouveau/nvkm/subdev/mmu/vmmgk20a.c    | 12 --------
 .../drm/nouveau/nvkm/subdev/mmu/vmmgm200.c    |  2 --
 .../drm/nouveau/nvkm/subdev/mmu/vmmgm20b.c    |  2 --
 .../drm/nouveau/nvkm/subdev/mmu/vmmgp100.c    |  8 ++----
 .../drm/nouveau/nvkm/subdev/mmu/vmmgp10b.c    |  1 -
 .../drm/nouveau/nvkm/subdev/mmu/vmmgv100.c    |  1 -
 .../drm/nouveau/nvkm/subdev/mmu/vmmtu102.c    |  1 -
 18 files changed, 55 insertions(+), 55 deletions(-)
Signature · 6 lines
2.23.0

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

A message that is not in the archive (sent off-list or not yet received): 20190916151757.10953-1-thierry.reding-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#2 lore raw

PATCH 1/6 drm/nouveau: fault: Store aperture in fault information

From: Thierry Reding <[email protected]>

The fault information register contains data about the aperture that caused the failure. This can be useful in debugging aperture related programming bugs.

---
 drivers/gpu/drm/nouveau/include/nvkm/subdev/fault.h | 1 +
 drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.c    | 3 ++-
 drivers/gpu/drm/nouveau/nvkm/subdev/fault/gv100.c   | 1 +
 3 files changed, 4 insertions(+), 1 deletion(-)
drivers/gpu/drm/nouveau/include/nvkm/subdev/fault.h +1 -0
index 97322f95b3ee..1cc862bc1122 100644
@@ -21,6 +21,7 @@ struct nvkm_fault_data {
 	u64  addr;
 	u64  inst;
 	u64  time;
+	u8 aperture;
 	u8 engine;
 	u8  valid;
 	u8    gpc;
drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.c +2 -1
index 5d4b695cab8e..81cbe1cc4804 100644
@@ -519,9 +519,10 @@ gk104_fifo_fault(struct nvkm_fifo *base, struct nvkm_fault_data *info)
 	chan = nvkm_fifo_chan_inst_locked(&fifo->base, info->inst);
 
 	nvkm_error(subdev,
-		   "fault %02x [%s] at %016llx engine %02x [%s] client %02x "
+		   "fault %02x [%s] at %016llx aperture %02x engine %02x [%s] client %02x "
 		   "[%s%s] reason %02x [%s] on channel %d [%010llx %s]\n",
 		   info->access, ea ? ea->name : "", info->addr,
+		   info->aperture,
 		   info->engine, ee ? ee->name : en,
 		   info->client, ct, ec ? ec->name : "",
 		   info->reason, er ? er->name : "", chan ? chan->chid : -1,
drivers/gpu/drm/nouveau/nvkm/subdev/fault/gv100.c +1 -0
index 6747f09c2dc3..b5e32295237b 100644
@@ -138,6 +138,7 @@ gv100_fault_intr_fault(struct nvkm_fault *fault)
 	info.inst = ((u64)insthi << 32) | (info0 & 0xfffff000);
 	info.time = 0;
 	info.engine = (info0 & 0x000000ff);
+	info.aperture = (info0 & 0x00000c00) >> 10;
 	info.valid  = (info1 & 0x80000000) >> 31;
 	info.gpc    = (info1 & 0x1f000000) >> 24;
 	info.hub    = (info1 & 0x00100000) >> 20;
Signature · 6 lines
2.23.0

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

A message that is not in the archive (sent off-list or not yet received): 20190916151757.10953-2-thierry.reding-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#4 lore raw

Re: [PATCH 1/6] drm/nouveau: fault: Store aperture in fault information

Quoted 6 lines · Thierry Reding

From: Thierry Reding <[email protected]>

The fault information register contains data about the aperture that caused the failure. This can be useful in debugging aperture related programming bugs.

Should this be parsed for fault buffer entries too?

Quoted 51 lines
---
 drivers/gpu/drm/nouveau/include/nvkm/subdev/fault.h | 1 +
 drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.c    | 3 ++-
 drivers/gpu/drm/nouveau/nvkm/subdev/fault/gv100.c   | 1 +
 3 files changed, 4 insertions(+), 1 deletion(-)
drivers/gpu/drm/nouveau/include/nvkm/subdev/fault.h +1 -0
index 97322f95b3ee..1cc862bc1122 100644
@@ -21,6 +21,7 @@ struct nvkm_fault_data {
        u64  addr;
        u64  inst;
        u64  time;
+       u8 aperture;
        u8 engine;
        u8  valid;
        u8    gpc;
drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.c +2 -1
index 5d4b695cab8e..81cbe1cc4804 100644
@@ -519,9 +519,10 @@ gk104_fifo_fault(struct nvkm_fifo *base, struct nvkm_fault_data *info)
        chan = nvkm_fifo_chan_inst_locked(&fifo->base, info->inst);
 
        nvkm_error(subdev,
-                  "fault %02x [%s] at %016llx engine %02x [%s] client %02x "
+                  "fault %02x [%s] at %016llx aperture %02x engine %02x [%s] client %02x "
                   "[%s%s] reason %02x [%s] on channel %d [%010llx %s]\n",
                   info->access, ea ? ea->name : "", info->addr,
+                  info->aperture,
                   info->engine, ee ? ee->name : en,
                   info->client, ct, ec ? ec->name : "",
                   info->reason, er ? er->name : "", chan ? chan->chid : -1,
drivers/gpu/drm/nouveau/nvkm/subdev/fault/gv100.c +1 -1
index 6747f09c2dc3..b5e32295237b 100644
@@ -138,6 +138,7 @@ gv100_fault_intr_fault(struct nvkm_fault *fault)
        info.inst = ((u64)insthi << 32) | (info0 & 0xfffff000);
        info.time = 0;
        info.engine = (info0 & 0x000000ff);
+       info.aperture = (info0 & 0x00000c00) >> 10;
        info.valid  = (info1 & 0x80000000) >> 31;
        info.gpc    = (info1 & 0x1f000000) >> 24;
        info.hub    = (info1 & 0x00100000) >> 20;
--

2.23.0

Quoted 4 lines

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

A message that is not in the archive (sent off-list or not yet received): CACAvsv70b1-LJoaP1ZL2hvy9xMcO1WXqh0ibGzNfpB81ybgrTg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#6 lore raw

Re: [PATCH 1/6] drm/nouveau: fault: Store aperture in fault information

Quoted 8 lines · Ben Skeggs
Quoted 6 lines · Thierry Reding

From: Thierry Reding <[email protected]>

The fault information register contains data about the aperture that caused the failure. This can be useful in debugging aperture related programming bugs.

Should this be parsed for fault buffer entries too?

Yes, it probably should. Will fix that in v2.

Thanks,
Thierry

Quoted 52 lines
Quoted 52 lines
---
 drivers/gpu/drm/nouveau/include/nvkm/subdev/fault.h | 1 +
 drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.c    | 3 ++-
 drivers/gpu/drm/nouveau/nvkm/subdev/fault/gv100.c   | 1 +
 3 files changed, 4 insertions(+), 1 deletion(-)
drivers/gpu/drm/nouveau/include/nvkm/subdev/fault.h +1 -0
index 97322f95b3ee..1cc862bc1122 100644
@@ -21,6 +21,7 @@ struct nvkm_fault_data {
        u64  addr;
        u64  inst;
        u64  time;
+       u8 aperture;
        u8 engine;
        u8  valid;
        u8    gpc;
drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.c +2 -1
index 5d4b695cab8e..81cbe1cc4804 100644
@@ -519,9 +519,10 @@ gk104_fifo_fault(struct nvkm_fifo *base, struct nvkm_fault_data *info)
        chan = nvkm_fifo_chan_inst_locked(&fifo->base, info->inst);
 
        nvkm_error(subdev,
-                  "fault %02x [%s] at %016llx engine %02x [%s] client %02x "
+                  "fault %02x [%s] at %016llx aperture %02x engine %02x [%s] client %02x "
                   "[%s%s] reason %02x [%s] on channel %d [%010llx %s]\n",
                   info->access, ea ? ea->name : "", info->addr,
+                  info->aperture,
                   info->engine, ee ? ee->name : en,
                   info->client, ct, ec ? ec->name : "",
                   info->reason, er ? er->name : "", chan ? chan->chid : -1,
drivers/gpu/drm/nouveau/nvkm/subdev/fault/gv100.c +1 -1
index 6747f09c2dc3..b5e32295237b 100644
@@ -138,6 +138,7 @@ gv100_fault_intr_fault(struct nvkm_fault *fault)
        info.inst = ((u64)insthi << 32) | (info0 & 0xfffff000);
        info.time = 0;
        info.engine = (info0 & 0x000000ff);
+       info.aperture = (info0 & 0x00000c00) >> 10;
        info.valid  = (info1 & 0x80000000) >> 31;
        info.gpc    = (info1 & 0x1f000000) >> 24;
        info.hub    = (info1 & 0x00100000) >> 20;
--

2.23.0

Quoted 4 lines
Quoted 4 lines

_______________________________________________
Nouveau mailing list [email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

#7 lore raw

PATCH 2/6 drm/nouveau: fault: Widen engine field

In reply to a message not in the archive

From: Thierry Reding <[email protected]>

The engine field in the FIFO fault information registers is actually 9 bits wide.

---
 drivers/gpu/drm/nouveau/nvkm/subdev/fault/gv100.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/nouveau/nvkm/subdev/fault/gv100.c +1 -1
index b5e32295237b..28306c5f6651 100644
@@ -137,8 +137,8 @@ gv100_fault_intr_fault(struct nvkm_fault *fault)
 	info.addr = ((u64)addrhi << 32) | addrlo;
 	info.inst = ((u64)insthi << 32) | (info0 & 0xfffff000);
 	info.time = 0;
-	info.engine = (info0 & 0x000000ff);
 	info.aperture = (info0 & 0x00000c00) >> 10;
+	info.engine = (info0 & 0x000001ff);
 	info.valid  = (info1 & 0x80000000) >> 31;
 	info.gpc    = (info1 & 0x1f000000) >> 24;
 	info.hub    = (info1 & 0x00100000) >> 20;
Signature · 6 lines
2.23.0

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

A message that is not in the archive (sent off-list or not yet received): 20190916151757.10953-3-thierry.reding-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org

#9 lore raw

Re: [PATCH 2/6] drm/nouveau: fault: Widen engine field

Quoted 5 lines · Thierry Reding

From: Thierry Reding <[email protected]>

The engine field in the FIFO fault information registers is actually 9 bits wide.

Looks like this is true for fault buffer parsing too.

Quoted 23 lines
---
 drivers/gpu/drm/nouveau/nvkm/subdev/fault/gv100.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/nouveau/nvkm/subdev/fault/gv100.c +1 -2
index b5e32295237b..28306c5f6651 100644
@@ -137,8 +137,8 @@ gv100_fault_intr_fault(struct nvkm_fault *fault)
        info.addr = ((u64)addrhi << 32) | addrlo;
        info.inst = ((u64)insthi << 32) | (info0 & 0xfffff000);
        info.time = 0;
-       info.engine = (info0 & 0x000000ff);
        info.aperture = (info0 & 0x00000c00) >> 10;
+       info.engine = (info0 & 0x000001ff);
        info.valid  = (info1 & 0x80000000) >> 31;
        info.gpc    = (info1 & 0x1f000000) >> 24;
        info.hub    = (info1 & 0x00100000) >> 20;
--

2.23.0

Quoted 4 lines

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

#10 lore raw

Re: [Nouveau] [PATCH 2/6] drm/nouveau: fault: Widen engine field

Quoted 7 lines · Ben Skeggs
Quoted 5 lines · Thierry Reding

From: Thierry Reding <[email protected]>

The engine field in the FIFO fault information registers is actually 9 bits wide.

Looks like this is true for fault buffer parsing too.

Yes, I'll add that in v2.

Thierry

Quoted 23 lines
Quoted 23 lines
---
 drivers/gpu/drm/nouveau/nvkm/subdev/fault/gv100.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/nouveau/nvkm/subdev/fault/gv100.c +1 -2
index b5e32295237b..28306c5f6651 100644
@@ -137,8 +137,8 @@ gv100_fault_intr_fault(struct nvkm_fault *fault)
        info.addr = ((u64)addrhi << 32) | addrlo;
        info.inst = ((u64)insthi << 32) | (info0 & 0xfffff000);
        info.time = 0;
-       info.engine = (info0 & 0x000000ff);
        info.aperture = (info0 & 0x00000c00) >> 10;
+       info.engine = (info0 & 0x000001ff);
        info.valid  = (info1 & 0x80000000) >> 31;
        info.gpc    = (info1 & 0x1f000000) >> 24;
        info.hub    = (info1 & 0x00100000) >> 20;
--

2.23.0

Quoted 4 lines
Quoted 4 lines

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#11 lore raw

PATCH 3/6 drm/nouveau: Remove bogus gk20a aperture callback

In reply to a message not in the archive

From: Thierry Reding <[email protected]>

The gk20a (as well as all subsequent Tegra instantiations of the GPU) do in fact use the same apertures as regular GPUs. Prior to gv11b there are no checks in hardware for the aperture, so we get away with setting VRAM as the aperture for buffers that are actually in system memory.

---
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.h      |  1 -
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgk20a.c | 10 ----------
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgm20b.c |  4 ++--
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp10b.c |  2 +-
 4 files changed, 3 insertions(+), 14 deletions(-)
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.h +0 -1
index fb3a9e8bb9cd..9862f44ac8b5 100644
@@ -212,7 +212,6 @@ void gf100_vmm_flush(struct nvkm_vmm *, int);
 void gf100_vmm_invalidate(struct nvkm_vmm *, u32 type);
 void gf100_vmm_invalidate_pdb(struct nvkm_vmm *, u64 addr);
 
-int gk20a_vmm_aper(enum nvkm_memory_target);
 int gk20a_vmm_valid(struct nvkm_vmm *, void *, u32, struct nvkm_vmm_map *);
 
 int gm200_vmm_new_(const struct nvkm_vmm_func *, const struct nvkm_vmm_func *,
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgk20a.c +0 -10
index 16d7bf727292..999b953505b3 100644
@@ -25,16 +25,6 @@
 
 #include <core/memory.h>
 
-int
-gk20a_vmm_aper(enum nvkm_memory_target target)
-{
-	switch (target) {
-	case NVKM_MEM_TARGET_NCOH: return 0;
-	default:
-		return -EINVAL;
-	}
-}
-
 int
 gk20a_vmm_valid(struct nvkm_vmm *vmm, void *argv, u32 argc,
 		struct nvkm_vmm_map *map)
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgm20b.c +2 -2
index 7a6066d886cd..f5d7819c4a40 100644
@@ -25,7 +25,7 @@ static const struct nvkm_vmm_func
 gm20b_vmm_17 = {
 	.join = gm200_vmm_join,
 	.part = gf100_vmm_part,
-	.aper = gk20a_vmm_aper,
+	.aper = gf100_vmm_aper,
 	.valid = gk20a_vmm_valid,
 	.flush = gf100_vmm_flush,
 	.invalidate_pdb = gf100_vmm_invalidate_pdb,
@@ -41,7 +41,7 @@ static const struct nvkm_vmm_func
 gm20b_vmm_16 = {
 	.join = gm200_vmm_join,
 	.part = gf100_vmm_part,
-	.aper = gk20a_vmm_aper,
+	.aper = gf100_vmm_aper,
 	.valid = gk20a_vmm_valid,
 	.flush = gf100_vmm_flush,
 	.invalidate_pdb = gf100_vmm_invalidate_pdb,
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp10b.c +1 -1
index 180c8f006e32..ffe84ea2f7d9 100644
@@ -43,7 +43,7 @@ static const struct nvkm_vmm_func
 gp10b_vmm = {
 	.join = gp100_vmm_join,
 	.part = gf100_vmm_part,
-	.aper = gk20a_vmm_aper,
+	.aper = gf100_vmm_aper,
 	.valid = gp10b_vmm_valid,
 	.flush = gp100_vmm_flush,
 	.mthd = gp100_vmm_mthd,
Signature · 6 lines
2.23.0

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau
#12 lore raw

Re: [Nouveau] [PATCH 3/6] drm/nouveau: Remove bogus gk20a aperture callback

Quoted 7 lines · Thierry Reding

From: Thierry Reding <[email protected]>

The gk20a (as well as all subsequent Tegra instantiations of the GPU) do in fact use the same apertures as regular GPUs. Prior to gv11b there are no checks in hardware for the aperture, so we get away with setting VRAM as the aperture for buffers that are actually in system memory.

Can GK20A take comptags with aperture set to system memory? For some reason I can recall, I was under the impression PTEs needed to be pointed at "vidmem" (despite them actually accessing system memory anyway) on Tegra parts for compression to work? I could be mistaken though.

Ben.

Quoted 80 lines
---
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.h      |  1 -
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgk20a.c | 10 ----------
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgm20b.c |  4 ++--
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp10b.c |  2 +-
 4 files changed, 3 insertions(+), 14 deletions(-)
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.h +0 -1
index fb3a9e8bb9cd..9862f44ac8b5 100644
@@ -212,7 +212,6 @@ void gf100_vmm_flush(struct nvkm_vmm *, int);
 void gf100_vmm_invalidate(struct nvkm_vmm *, u32 type);
 void gf100_vmm_invalidate_pdb(struct nvkm_vmm *, u64 addr);
 
-int gk20a_vmm_aper(enum nvkm_memory_target);
 int gk20a_vmm_valid(struct nvkm_vmm *, void *, u32, struct nvkm_vmm_map *);
 
 int gm200_vmm_new_(const struct nvkm_vmm_func *, const struct nvkm_vmm_func *,
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgk20a.c +0 -10
index 16d7bf727292..999b953505b3 100644
@@ -25,16 +25,6 @@
 
 #include <core/memory.h>
 
-int
-gk20a_vmm_aper(enum nvkm_memory_target target)
-{
-       switch (target) {
-       case NVKM_MEM_TARGET_NCOH: return 0;
-       default:
-               return -EINVAL;
-       }
-}
-
 int
 gk20a_vmm_valid(struct nvkm_vmm *vmm, void *argv, u32 argc,
                struct nvkm_vmm_map *map)
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgm20b.c +2 -2
index 7a6066d886cd..f5d7819c4a40 100644
@@ -25,7 +25,7 @@ static const struct nvkm_vmm_func
 gm20b_vmm_17 = {
        .join = gm200_vmm_join,
        .part = gf100_vmm_part,
-       .aper = gk20a_vmm_aper,
+       .aper = gf100_vmm_aper,
        .valid = gk20a_vmm_valid,
        .flush = gf100_vmm_flush,
        .invalidate_pdb = gf100_vmm_invalidate_pdb,
@@ -41,7 +41,7 @@ static const struct nvkm_vmm_func
 gm20b_vmm_16 = {
        .join = gm200_vmm_join,
        .part = gf100_vmm_part,
-       .aper = gk20a_vmm_aper,
+       .aper = gf100_vmm_aper,
        .valid = gk20a_vmm_valid,
        .flush = gf100_vmm_flush,
        .invalidate_pdb = gf100_vmm_invalidate_pdb,
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp10b.c +1 -2
index 180c8f006e32..ffe84ea2f7d9 100644
@@ -43,7 +43,7 @@ static const struct nvkm_vmm_func
 gp10b_vmm = {
        .join = gp100_vmm_join,
        .part = gf100_vmm_part,
-       .aper = gk20a_vmm_aper,
+       .aper = gf100_vmm_aper,
        .valid = gp10b_vmm_valid,
        .flush = gp100_vmm_flush,
        .mthd = gp100_vmm_mthd,
--

2.23.0

Quoted 4 lines

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

A message that is not in the archive (sent off-list or not yet received): CACAvsv6AzWvpPLaOKahpJErTAMCJet4_4mBvRC8Z+e4+bwaD4w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#14 lore raw

Re: [PATCH 3/6] drm/nouveau: Remove bogus gk20a aperture callback

Quoted 8 lines · Ben Skeggs
Quoted 7 lines · Thierry Reding

From: Thierry Reding <[email protected]>

The gk20a (as well as all subsequent Tegra instantiations of the GPU) do in fact use the same apertures as regular GPUs. Prior to gv11b there are no checks in hardware for the aperture, so we get away with setting VRAM as the aperture for buffers that are actually in system memory.

Quoted 5 lines

Can GK20A take comptags with aperture set to system memory? For some reason I can recall, I was under the impression PTEs needed to be pointed at "vidmem" (despite them actually accessing system memory anyway) on Tegra parts for compression to work? I could be mistaken though.

I don't think GK20A supports comptags at all. I think this wasn't introduced until GM20B. nvgpu has some "gk20a" code to flush comptags, but that's only used on GM20B and later.

Anyway, my understanding is that on all of GK20A, GM20B and GP10B the aperture field is completely ignored. I think that also goes for comptags. nvgpu in particular never requests comptags to be allocated from vidmem. See:

	https://nv-tegra.nvidia.com/gitweb/?p=linux-nvgpu.git;a=blob;f=drivers/gpu/nvgpu/os/linux/ltc.c;h=baeb20b2e539cc6cb70667ce168603546678dc73;hb=2081ce686bfd4deb461b4130df424d592000ff88#l30

There are two callers of that, both passing "false" for the vidmem_alloc parameter, so comptags always do end up in system memory for Tegra.

That said, I'll go confirm that with one of our experts and get back to you.

Thierry

Quoted 83 lines

Ben.

Quoted 80 lines
---
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.h      |  1 -
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgk20a.c | 10 ----------
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgm20b.c |  4 ++--
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp10b.c |  2 +-
 4 files changed, 3 insertions(+), 14 deletions(-)
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.h +0 -1
index fb3a9e8bb9cd..9862f44ac8b5 100644
@@ -212,7 +212,6 @@ void gf100_vmm_flush(struct nvkm_vmm *, int);
 void gf100_vmm_invalidate(struct nvkm_vmm *, u32 type);
 void gf100_vmm_invalidate_pdb(struct nvkm_vmm *, u64 addr);
 
-int gk20a_vmm_aper(enum nvkm_memory_target);
 int gk20a_vmm_valid(struct nvkm_vmm *, void *, u32, struct nvkm_vmm_map *);
 
 int gm200_vmm_new_(const struct nvkm_vmm_func *, const struct nvkm_vmm_func *,
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgk20a.c +0 -10
index 16d7bf727292..999b953505b3 100644
@@ -25,16 +25,6 @@
 
 #include <core/memory.h>
 
-int
-gk20a_vmm_aper(enum nvkm_memory_target target)
-{
-       switch (target) {
-       case NVKM_MEM_TARGET_NCOH: return 0;
-       default:
-               return -EINVAL;
-       }
-}
-
 int
 gk20a_vmm_valid(struct nvkm_vmm *vmm, void *argv, u32 argc,
                struct nvkm_vmm_map *map)
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgm20b.c +2 -2
index 7a6066d886cd..f5d7819c4a40 100644
@@ -25,7 +25,7 @@ static const struct nvkm_vmm_func
 gm20b_vmm_17 = {
        .join = gm200_vmm_join,
        .part = gf100_vmm_part,
-       .aper = gk20a_vmm_aper,
+       .aper = gf100_vmm_aper,
        .valid = gk20a_vmm_valid,
        .flush = gf100_vmm_flush,
        .invalidate_pdb = gf100_vmm_invalidate_pdb,
@@ -41,7 +41,7 @@ static const struct nvkm_vmm_func
 gm20b_vmm_16 = {
        .join = gm200_vmm_join,
        .part = gf100_vmm_part,
-       .aper = gk20a_vmm_aper,
+       .aper = gf100_vmm_aper,
        .valid = gk20a_vmm_valid,
        .flush = gf100_vmm_flush,
        .invalidate_pdb = gf100_vmm_invalidate_pdb,
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp10b.c +1 -2
index 180c8f006e32..ffe84ea2f7d9 100644
@@ -43,7 +43,7 @@ static const struct nvkm_vmm_func
 gp10b_vmm = {
        .join = gp100_vmm_join,
        .part = gf100_vmm_part,
-       .aper = gk20a_vmm_aper,
+       .aper = gf100_vmm_aper,
        .valid = gp10b_vmm_valid,
        .flush = gp100_vmm_flush,
        .mthd = gp100_vmm_mthd,
--

2.23.0

Quoted 4 lines
Quoted 4 lines

_______________________________________________
Nouveau mailing list [email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

#15 lore raw

Re: [Nouveau] [PATCH 3/6] drm/nouveau: Remove bogus gk20a aperture callback

Quoted 24 lines · Thierry Reding
Quoted 8 lines · Ben Skeggs
Quoted 7 lines · Thierry Reding

From: Thierry Reding <[email protected]>

The gk20a (as well as all subsequent Tegra instantiations of the GPU) do in fact use the same apertures as regular GPUs. Prior to gv11b there are no checks in hardware for the aperture, so we get away with setting VRAM as the aperture for buffers that are actually in system memory.

Quoted 5 lines

Can GK20A take comptags with aperture set to system memory? For some reason I can recall, I was under the impression PTEs needed to be pointed at "vidmem" (despite them actually accessing system memory anyway) on Tegra parts for compression to work? I could be mistaken though.

I don't think GK20A supports comptags at all. I think this wasn't introduced until GM20B. nvgpu has some "gk20a" code to flush comptags, but that's only used on GM20B and later.

Anyway, my understanding is that on all of GK20A, GM20B and GP10B the aperture field is completely ignored. I think that also goes for comptags. nvgpu in particular never requests comptags to be allocated from vidmem. See:

Quoted 7 lines
	https://nv-tegra.nvidia.com/gitweb/?p=linux-nvgpu.git;a=blob;f=drivers/gpu/nvgpu/os/linux/ltc.c;h=baeb20b2e539cc6cb70667ce168603546678dc73;hb=2081ce686bfd4deb461b4130df424d592000ff88#l30

There are two callers of that, both passing "false" for the vidmem_alloc parameter, so comptags always do end up in system memory for Tegra.

That said, I'll go confirm that with one of our experts and get back to you.

So it looks like you're right and indeed GK20A and later (up to, but not including, GV11B that is) comptags do indeed need to be mapped as vidmem even if they reside in sysmem.

Conceptually I think what we want to do is decide about the aperture at allocation time. So if we allocate comptags we would need to force the aperture to be VRAM on the iGPUs where necessary, even if they are really allocated in system memory. Ultimately the end result is the same of course, but I think this way around is a better representation of this particular hardware quirk and allows us to keep the unification that this patch series achieves.

But I'll have to look into this and see what I can come up with.

Thierry

Quoted 86 lines

Thierry

Quoted 83 lines

Ben.

Quoted 80 lines
---
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.h      |  1 -
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgk20a.c | 10 ----------
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgm20b.c |  4 ++--
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp10b.c |  2 +-
 4 files changed, 3 insertions(+), 14 deletions(-)
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.h +0 -1
index fb3a9e8bb9cd..9862f44ac8b5 100644
@@ -212,7 +212,6 @@ void gf100_vmm_flush(struct nvkm_vmm *, int);
 void gf100_vmm_invalidate(struct nvkm_vmm *, u32 type);
 void gf100_vmm_invalidate_pdb(struct nvkm_vmm *, u64 addr);
 
-int gk20a_vmm_aper(enum nvkm_memory_target);
 int gk20a_vmm_valid(struct nvkm_vmm *, void *, u32, struct nvkm_vmm_map *);
 
 int gm200_vmm_new_(const struct nvkm_vmm_func *, const struct nvkm_vmm_func *,
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgk20a.c +0 -10
index 16d7bf727292..999b953505b3 100644
@@ -25,16 +25,6 @@
 
 #include <core/memory.h>
 
-int
-gk20a_vmm_aper(enum nvkm_memory_target target)
-{
-       switch (target) {
-       case NVKM_MEM_TARGET_NCOH: return 0;
-       default:
-               return -EINVAL;
-       }
-}
-
 int
 gk20a_vmm_valid(struct nvkm_vmm *vmm, void *argv, u32 argc,
                struct nvkm_vmm_map *map)
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgm20b.c +2 -2
index 7a6066d886cd..f5d7819c4a40 100644
@@ -25,7 +25,7 @@ static const struct nvkm_vmm_func
 gm20b_vmm_17 = {
        .join = gm200_vmm_join,
        .part = gf100_vmm_part,
-       .aper = gk20a_vmm_aper,
+       .aper = gf100_vmm_aper,
        .valid = gk20a_vmm_valid,
        .flush = gf100_vmm_flush,
        .invalidate_pdb = gf100_vmm_invalidate_pdb,
@@ -41,7 +41,7 @@ static const struct nvkm_vmm_func
 gm20b_vmm_16 = {
        .join = gm200_vmm_join,
        .part = gf100_vmm_part,
-       .aper = gk20a_vmm_aper,
+       .aper = gf100_vmm_aper,
        .valid = gk20a_vmm_valid,
        .flush = gf100_vmm_flush,
        .invalidate_pdb = gf100_vmm_invalidate_pdb,
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp10b.c +1 -2
index 180c8f006e32..ffe84ea2f7d9 100644
@@ -43,7 +43,7 @@ static const struct nvkm_vmm_func
 gp10b_vmm = {
        .join = gp100_vmm_join,
        .part = gf100_vmm_part,
-       .aper = gk20a_vmm_aper,
+       .aper = gf100_vmm_aper,
        .valid = gp10b_vmm_valid,
        .flush = gp100_vmm_flush,
        .mthd = gp100_vmm_mthd,
--

2.23.0

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#16 lore raw

PATCH 4/6 drm/nouveau: Implement nvkm_memory_aperture()

In reply to a message not in the archive

From: Thierry Reding <[email protected]>

The aperture of a buffer is always specific to where its memory was allocated from. Furthermore, the encoding of the aperture is always the same, regardless of GPU generation.

Implement the memory target to aperture conversion in one central place and make the aperture independent of the VMM.

Note that we no longer return a negative error code for unsupported apertures. First, this should never happen to begin with and is a programming error, which is why we have a WARN already. Second, the standard aperture (0, VRAM) should be correct for the vast majority of memory objects. Lastly, the aperture also needs to be programmed into many registers and instance blocks. Having to check for error codes at every step of the way would make this very unwieldy. If in any case there is ever a problem with the aperture being wrong, let us rely on the WARN to tell us about it.

---
 .../drm/nouveau/include/nvkm/core/memory.h    | 28 +++++++++++++++++++
 .../drm/nouveau/nvkm/subdev/mmu/vmmgf100.c    |  7 ++---
 .../drm/nouveau/nvkm/subdev/mmu/vmmgp100.c    |  7 ++---
 3 files changed, 34 insertions(+), 8 deletions(-)
drivers/gpu/drm/nouveau/include/nvkm/core/memory.h +28 -0
index b23bf6109f2d..29c60fbed167 100644
@@ -64,6 +64,34 @@ void nvkm_memory_tags_put(struct nvkm_memory *, struct nvkm_device *,
 #define nvkm_memory_map(p,o,vm,va,av,ac)                                       \
 	(p)->func->map((p),(o),(vm),(va),(av),(ac))
 
+static inline u32
+nvkm_memory_aperture(struct nvkm_memory *mem)
+{
+	enum nvkm_memory_target target = nvkm_memory_target(mem);
+
+	switch (target) {
+	case NVKM_MEM_TARGET_VRAM:
+		return 0;
+
+	case NVKM_MEM_TARGET_HOST:
+		return 2;
+
+	case NVKM_MEM_TARGET_NCOH:
+		return 3;
+
+	default:
+		break;
+	}
+
+	/*
+	 * This is invalid, so warn about this loudly. However, return 0 to
+	 * avoid writing garbage into registers. 0 is the VRAM aperture and
+	 * might still work in most cases.
+	 */
+	WARN(1, "invalid memory target: %d\n", target);
+	return 0;
+}
+
 /* accessor macros - kmap()/done() must bracket use of the other accessor
  * macros to guarantee correct behaviour across all chipsets
  */
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgf100.c +3 -4
index ab6424faf84c..ffa64c0d3eda 100644
@@ -248,8 +248,9 @@ gf100_vmm_valid(struct nvkm_vmm *vmm, void *argv, u32 argc,
 	struct nvkm_device *device = vmm->mmu->subdev.device;
 	struct nvkm_memory *memory = map->memory;
 	u8  kind, priv, ro, vol;
-	int kindn, aper, ret = -ENOSYS;
+	int kindn, ret = -ENOSYS;
 	const u8 *kindm;
+	u32 aper;
 
 	map->next = (1 << page->shift) >> 8;
 	map->type = map->ctag = 0;
@@ -270,9 +271,7 @@ gf100_vmm_valid(struct nvkm_vmm *vmm, void *argv, u32 argc,
 		return ret;
 	}
 
-	aper = vmm->func->aper(target);
-	if (WARN_ON(aper < 0))
-		return aper;
+	aper = nvkm_memory_aperture(map->memory);
 
 	kindm = vmm->mmu->func->kind(vmm->mmu, &kindn);
 	if (kind >= kindn || kindm[kind] == 0xff) {
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp100.c +3 -4
index b4f519768d5e..4a1a658328e5 100644
@@ -321,8 +321,9 @@ gp100_vmm_valid(struct nvkm_vmm *vmm, void *argv, u32 argc,
 	struct nvkm_device *device = vmm->mmu->subdev.device;
 	struct nvkm_memory *memory = map->memory;
 	u8  kind, priv, ro, vol;
-	int kindn, aper, ret = -ENOSYS;
+	int kindn, ret = -ENOSYS;
 	const u8 *kindm;
+	u32 aper;
 
 	map->next = (1ULL << page->shift) >> 4;
 	map->type = 0;
@@ -343,9 +344,7 @@ gp100_vmm_valid(struct nvkm_vmm *vmm, void *argv, u32 argc,
 		return ret;
 	}
 
-	aper = vmm->func->aper(target);
-	if (WARN_ON(aper < 0))
-		return aper;
+	aper = nvkm_memory_aperture(map->memory);
 
 	kindm = vmm->mmu->func->kind(vmm->mmu, &kindn);
 	if (kind >= kindn || kindm[kind] == 0xff) {
Signature · 6 lines
2.23.0

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau
#17 lore raw

PATCH 5/6 drm/nouveau: Remove unused nvkm_vmm_func->aper() implementations

In reply to a message not in the archive

From: Thierry Reding <[email protected]>

These implementations are now all unused. Remove them.

---
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.h      |  2 --
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgf100.c | 14 --------------
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgk104.c |  2 --
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgk20a.c |  2 --
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgm200.c |  2 --
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgm20b.c |  2 --
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp100.c |  1 -
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp10b.c |  1 -
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgv100.c |  1 -
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmtu102.c |  1 -
 10 files changed, 28 deletions(-)
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.h +0 -2
index 9862f44ac8b5..767870c2d24c 100644
@@ -140,7 +140,6 @@ struct nvkm_vmm_func {
 	int (*join)(struct nvkm_vmm *, struct nvkm_memory *inst);
 	void (*part)(struct nvkm_vmm *, struct nvkm_memory *inst);
 
-	int (*aper)(enum nvkm_memory_target);
 	int (*valid)(struct nvkm_vmm *, void *argv, u32 argc,
 		     struct nvkm_vmm_map *);
 	void (*flush)(struct nvkm_vmm *, int depth);
@@ -206,7 +205,6 @@ int gf100_vmm_new_(const struct nvkm_vmm_func *, const struct nvkm_vmm_func *,
 int gf100_vmm_join_(struct nvkm_vmm *, struct nvkm_memory *, u64 base);
 int gf100_vmm_join(struct nvkm_vmm *, struct nvkm_memory *);
 void gf100_vmm_part(struct nvkm_vmm *, struct nvkm_memory *);
-int gf100_vmm_aper(enum nvkm_memory_target);
 int gf100_vmm_valid(struct nvkm_vmm *, void *, u32, struct nvkm_vmm_map *);
 void gf100_vmm_flush(struct nvkm_vmm *, int);
 void gf100_vmm_invalidate(struct nvkm_vmm *, u32 type);
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgf100.c +0 -14
index ffa64c0d3eda..ccf5a92d7b54 100644
@@ -318,18 +318,6 @@ gf100_vmm_valid(struct nvkm_vmm *vmm, void *argv, u32 argc,
 	return 0;
 }
 
-int
-gf100_vmm_aper(enum nvkm_memory_target target)
-{
-	switch (target) {
-	case NVKM_MEM_TARGET_VRAM: return 0;
-	case NVKM_MEM_TARGET_HOST: return 2;
-	case NVKM_MEM_TARGET_NCOH: return 3;
-	default:
-		return -EINVAL;
-	}
-}
-
 void
 gf100_vmm_part(struct nvkm_vmm *vmm, struct nvkm_memory *inst)
 {
@@ -370,7 +358,6 @@ static const struct nvkm_vmm_func
 gf100_vmm_17 = {
 	.join = gf100_vmm_join,
 	.part = gf100_vmm_part,
-	.aper = gf100_vmm_aper,
 	.valid = gf100_vmm_valid,
 	.flush = gf100_vmm_flush,
 	.invalidate_pdb = gf100_vmm_invalidate_pdb,
@@ -385,7 +372,6 @@ static const struct nvkm_vmm_func
 gf100_vmm_16 = {
 	.join = gf100_vmm_join,
 	.part = gf100_vmm_part,
-	.aper = gf100_vmm_aper,
 	.valid = gf100_vmm_valid,
 	.flush = gf100_vmm_flush,
 	.invalidate_pdb = gf100_vmm_invalidate_pdb,
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgk104.c +0 -2
index 0b59c01fd146..8efd147fa930 100644
@@ -68,7 +68,6 @@ static const struct nvkm_vmm_func
 gk104_vmm_17 = {
 	.join = gf100_vmm_join,
 	.part = gf100_vmm_part,
-	.aper = gf100_vmm_aper,
 	.valid = gf100_vmm_valid,
 	.flush = gf100_vmm_flush,
 	.invalidate_pdb = gf100_vmm_invalidate_pdb,
@@ -83,7 +82,6 @@ static const struct nvkm_vmm_func
 gk104_vmm_16 = {
 	.join = gf100_vmm_join,
 	.part = gf100_vmm_part,
-	.aper = gf100_vmm_aper,
 	.valid = gf100_vmm_valid,
 	.flush = gf100_vmm_flush,
 	.invalidate_pdb = gf100_vmm_invalidate_pdb,
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgk20a.c +0 -2
index 999b953505b3..774b6fe9d4a9 100644
@@ -45,7 +45,6 @@ static const struct nvkm_vmm_func
 gk20a_vmm_17 = {
 	.join = gf100_vmm_join,
 	.part = gf100_vmm_part,
-	.aper = gf100_vmm_aper,
 	.valid = gk20a_vmm_valid,
 	.flush = gf100_vmm_flush,
 	.invalidate_pdb = gf100_vmm_invalidate_pdb,
@@ -60,7 +59,6 @@ static const struct nvkm_vmm_func
 gk20a_vmm_16 = {
 	.join = gf100_vmm_join,
 	.part = gf100_vmm_part,
-	.aper = gf100_vmm_aper,
 	.valid = gk20a_vmm_valid,
 	.flush = gf100_vmm_flush,
 	.invalidate_pdb = gf100_vmm_invalidate_pdb,
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgm200.c +0 -2
index 2e61af02d4d8..c18f1c296de5 100644
@@ -110,7 +110,6 @@ static const struct nvkm_vmm_func
 gm200_vmm_17 = {
 	.join = gm200_vmm_join,
 	.part = gf100_vmm_part,
-	.aper = gf100_vmm_aper,
 	.valid = gf100_vmm_valid,
 	.flush = gf100_vmm_flush,
 	.invalidate_pdb = gf100_vmm_invalidate_pdb,
@@ -126,7 +125,6 @@ static const struct nvkm_vmm_func
 gm200_vmm_16 = {
 	.join = gm200_vmm_join,
 	.part = gf100_vmm_part,
-	.aper = gf100_vmm_aper,
 	.valid = gf100_vmm_valid,
 	.flush = gf100_vmm_flush,
 	.invalidate_pdb = gf100_vmm_invalidate_pdb,
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgm20b.c +0 -2
index f5d7819c4a40..d6f61403de34 100644
@@ -25,7 +25,6 @@ static const struct nvkm_vmm_func
 gm20b_vmm_17 = {
 	.join = gm200_vmm_join,
 	.part = gf100_vmm_part,
-	.aper = gf100_vmm_aper,
 	.valid = gk20a_vmm_valid,
 	.flush = gf100_vmm_flush,
 	.invalidate_pdb = gf100_vmm_invalidate_pdb,
@@ -41,7 +40,6 @@ static const struct nvkm_vmm_func
 gm20b_vmm_16 = {
 	.join = gm200_vmm_join,
 	.part = gf100_vmm_part,
-	.aper = gf100_vmm_aper,
 	.valid = gk20a_vmm_valid,
 	.flush = gf100_vmm_flush,
 	.invalidate_pdb = gf100_vmm_invalidate_pdb,
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp100.c +0 -1
index 4a1a658328e5..d7d1b2e4fb61 100644
@@ -487,7 +487,6 @@ static const struct nvkm_vmm_func
 gp100_vmm = {
 	.join = gp100_vmm_join,
 	.part = gf100_vmm_part,
-	.aper = gf100_vmm_aper,
 	.valid = gp100_vmm_valid,
 	.flush = gp100_vmm_flush,
 	.mthd = gp100_vmm_mthd,
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp10b.c +0 -1
index ffe84ea2f7d9..a41ebdbdc216 100644
@@ -43,7 +43,6 @@ static const struct nvkm_vmm_func
 gp10b_vmm = {
 	.join = gp100_vmm_join,
 	.part = gf100_vmm_part,
-	.aper = gf100_vmm_aper,
 	.valid = gp10b_vmm_valid,
 	.flush = gp100_vmm_flush,
 	.mthd = gp100_vmm_mthd,
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgv100.c +0 -1
index f0e21f63253a..b1260bdeef9e 100644
@@ -63,7 +63,6 @@ static const struct nvkm_vmm_func
 gv100_vmm = {
 	.join = gv100_vmm_join,
 	.part = gf100_vmm_part,
-	.aper = gf100_vmm_aper,
 	.valid = gp100_vmm_valid,
 	.flush = gp100_vmm_flush,
 	.mthd = gp100_vmm_mthd,
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmtu102.c +0 -1
index be91cffc3b52..078319f890b6 100644
@@ -53,7 +53,6 @@ static const struct nvkm_vmm_func
 tu102_vmm = {
 	.join = gv100_vmm_join,
 	.part = gf100_vmm_part,
-	.aper = gf100_vmm_aper,
 	.valid = gp100_vmm_valid,
 	.flush = tu102_vmm_flush,
 	.mthd = gp100_vmm_mthd,
Signature · 6 lines
2.23.0

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau
#18 lore raw

PATCH 6/6 drm/nouveau: Program aperture field where necessary

In reply to Thierry Reding

From: Thierry Reding <[email protected]>

Some registers and instance block entries need the aperture to be programmed correctly. This is important on recent Tegra GPUs where the GPU actually checks the value of this field and faults if an invalid aperture is programmed.

For example GV11B no longer supports VRAM and all memory is already allocated from system (coherent or non-coherent), so make sure to also program the right aperture.

---
 drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifogk104.c | 7 +++++--
 drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifogv100.c | 5 +++--
 drivers/gpu/drm/nouveau/nvkm/engine/fifo/gv100.c       | 7 ++++++-
 drivers/gpu/drm/nouveau/nvkm/subdev/bar/gf100.c        | 3 ++-
 4 files changed, 16 insertions(+), 6 deletions(-)
drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifogk104.c +5 -2
index 728a1edbf98c..843ebb41dbc6 100644
@@ -201,6 +201,7 @@ gk104_fifo_gpfifo_fini(struct nvkm_fifo_chan *base)
 void
 gk104_fifo_gpfifo_init(struct nvkm_fifo_chan *base)
 {
+	u32 aperture = nvkm_memory_aperture(base->inst->memory) << 28;
 	struct gk104_fifo_chan *chan = gk104_fifo_chan(base);
 	struct gk104_fifo *fifo = chan->fifo;
 	struct nvkm_device *device = fifo->base.engine.subdev.device;
@@ -208,7 +209,7 @@ gk104_fifo_gpfifo_init(struct nvkm_fifo_chan *base)
 	u32 coff = chan->base.chid * 8;
 
 	nvkm_mask(device, 0x800004 + coff, 0x000f0000, chan->runl << 16);
-	nvkm_wr32(device, 0x800000 + coff, 0x80000000 | addr);
+	nvkm_wr32(device, 0x800000 + coff, 0x80000000 | aperture | addr);
 
 	if (list_empty(&chan->head) && !chan->killed) {
 		gk104_fifo_runlist_insert(fifo, chan);
@@ -250,6 +251,7 @@ gk104_fifo_gpfifo_new_(struct gk104_fifo *fifo, u64 *runlists, u16 *chid,
 	unsigned long engm;
 	u64 subdevs = 0;
 	u64 usermem;
+	u32 target;
 
 	if (!vmm || runlist < 0 || runlist >= fifo->runlist_nr)
 		return -EINVAL;
@@ -303,10 +305,11 @@ gk104_fifo_gpfifo_new_(struct gk104_fifo *fifo, u64 *runlists, u16 *chid,
 		nvkm_wo32(fifo->user.mem, usermem + i, 0x00000000);
 	nvkm_done(fifo->user.mem);
 	usermem = nvkm_memory_addr(fifo->user.mem) + usermem;
+	target = nvkm_memory_aperture(fifo->user.mem);
 
 	/* RAMFC */
 	nvkm_kmap(chan->base.inst);
-	nvkm_wo32(chan->base.inst, 0x08, lower_32_bits(usermem));
+	nvkm_wo32(chan->base.inst, 0x08, lower_32_bits(usermem) | target);
 	nvkm_wo32(chan->base.inst, 0x0c, upper_32_bits(usermem));
 	nvkm_wo32(chan->base.inst, 0x10, 0x0000face);
 	nvkm_wo32(chan->base.inst, 0x30, 0xfffff902);
drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifogv100.c +3 -2
index a7462cf59d65..97d084ffcfd5 100644
@@ -132,7 +132,7 @@ gv100_fifo_gpfifo_new_(const struct nvkm_fifo_chan_func *func,
 	unsigned long engm;
 	u64 subdevs = 0;
 	u64 usermem, mthd;
-	u32 size;
+	u32 size, target;
 
 	if (!vmm || runlist < 0 || runlist >= fifo->runlist_nr)
 		return -EINVAL;
@@ -183,6 +183,7 @@ gv100_fifo_gpfifo_new_(const struct nvkm_fifo_chan_func *func,
 		nvkm_wo32(fifo->user.mem, usermem + i, 0x00000000);
 	nvkm_done(fifo->user.mem);
 	usermem = nvkm_memory_addr(fifo->user.mem) + usermem;
+	target = nvkm_memory_target(fifo->user.mem);
 
 	/* Allocate fault method buffer (magics come from nvgpu). */
 	size = nvkm_rd32(device, 0x104028); /* NV_PCE_PCE_MAP */
@@ -200,7 +201,7 @@ gv100_fifo_gpfifo_new_(const struct nvkm_fifo_chan_func *func,
 
 	/* RAMFC */
 	nvkm_kmap(chan->base.inst);
-	nvkm_wo32(chan->base.inst, 0x008, lower_32_bits(usermem));
+	nvkm_wo32(chan->base.inst, 0x008, lower_32_bits(usermem) | target);
 	nvkm_wo32(chan->base.inst, 0x00c, upper_32_bits(usermem));
 	nvkm_wo32(chan->base.inst, 0x010, 0x0000face);
 	nvkm_wo32(chan->base.inst, 0x030, 0x7ffff902);
drivers/gpu/drm/nouveau/nvkm/engine/fifo/gv100.c +6 -1
index 6ee1bb32a071..449f669f43b0 100644
@@ -32,11 +32,16 @@ void
 gv100_fifo_runlist_chan(struct gk104_fifo_chan *chan,
 			struct nvkm_memory *memory, u32 offset)
 {
+	struct nvkm_memory *instmem = chan->base.inst->memory;
 	struct nvkm_memory *usermem = chan->fifo->user.mem;
 	const u64 user = nvkm_memory_addr(usermem) + (chan->base.chid * 0x200);
 	const u64 inst = chan->base.inst->addr;
+	u64 target;
 
-	nvkm_wo32(memory, offset + 0x0, lower_32_bits(user));
+	target = (u64)nvkm_memory_aperture(usermem) << 6 |
+		 (u64)nvkm_memory_aperture(instmem) << 4;
+
+	nvkm_wo32(memory, offset + 0x0, lower_32_bits(user) | target);
 	nvkm_wo32(memory, offset + 0x4, upper_32_bits(user));
 	nvkm_wo32(memory, offset + 0x8, lower_32_bits(inst) | chan->base.chid);
 	nvkm_wo32(memory, offset + 0xc, upper_32_bits(inst));
drivers/gpu/drm/nouveau/nvkm/subdev/bar/gf100.c +2 -1
index a3dcb09a40ee..5b193a7bf5de 100644
@@ -54,7 +54,8 @@ gf100_bar_bar1_init(struct nvkm_bar *base)
 	struct nvkm_device *device = base->subdev.device;
 	struct gf100_bar *bar = gf100_bar(base);
 	const u32 addr = nvkm_memory_addr(bar->bar[1].inst) >> 12;
-	nvkm_wr32(device, 0x001704, 0x80000000 | addr);
+	u32 target = nvkm_memory_aperture(bar->bar[1].inst) << 28;
+	nvkm_wr32(device, 0x001704, 0x80000000 | target | addr);
 }
 
 struct nvkm_vmm *
Signature · 6 lines
2.23.0

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel