Skip to contentSkip to conversation

NV50/GF100 behind constrained hierarchies

PATCH In review Jon Derrick 2 participants 7 messages Mar 16, 2019 – Mar 28, 2019 2 messages not in the archive

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover NV50/GF100 behind constrained hierarchies

Hi Ben,

I've been working with an mmio-constrained pci hierarchy intended almost solely for nvme devices and switches. Binding nouveau to an NV50-based gpu results in a kernel panic as the device cannot be fully mapped.

I've made modifications in nv50 and vmm to unbind the driver from this hierarchy, and modified gf100 assuming it will have the same issue.

1/4 also includes a fix where nv50 doesn't look like it checks the return value for nvkm_vmm_new for bar1. I don't know if this was intentional (eg, it could work without it).

I'm wholly unfamiliar with linux graphics drivers so let me know if there's a better place for this.

Jon Derrick (4):
drm/nouveau/bar/nv50: check bar1 vmm return value drm/nouveau/bar/nv50: ensure BAR is mapped drm/nouveau/bar/gf100: ensure BAR is mapped drm/nouveau/mmu: qualify vmm during dtor

 drivers/gpu/drm/nouveau/nvkm/subdev/bar/gf100.c |  2 ++
 drivers/gpu/drm/nouveau/nvkm/subdev/bar/nv50.c  | 14 +++++++++++---
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.c   |  2 +-
 3 files changed, 14 insertions(+), 4 deletions(-)
Signature · 6 lines
2.14.4

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

A message that is not in the archive (sent off-list or not yet received): 20190316000518.3081-1-jonathan.derrick-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org

#2 lore raw

PATCH 1/4 drm/nouveau/bar/nv50: check bar1 vmm return value

Check bar1's new vmm creation return value for errors.

---
 drivers/gpu/drm/nouveau/nvkm/subdev/bar/nv50.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/gpu/drm/nouveau/nvkm/subdev/bar/nv50.c +2 -0
index 157b076a1272..8e64b19f3f8a 100644
@@ -168,6 +168,8 @@ nv50_bar_oneinit(struct nvkm_bar *base)
 
 	ret = nvkm_vmm_new(device, start, limit-- - start, NULL, 0,
 			   &bar1_lock, "bar1", &bar->bar1_vmm);
+	if (ret)
+		return ret;
 
 	atomic_inc(&bar->bar1_vmm->engref[NVKM_SUBDEV_BAR]);
 	bar->bar1_vmm->debug = bar->base.subdev.debug;
Signature · 6 lines
2.14.4

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau
#3 lore raw

PATCH 2/4 drm/nouveau/bar/nv50: ensure BAR is mapped

In reply to a message not in the archive

If the BAR is zero size, it indicates it was never successfully mapped. Ensure that the BAR is valid during initialization before attempting to use it.

---
 drivers/gpu/drm/nouveau/nvkm/subdev/bar/nv50.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
drivers/gpu/drm/nouveau/nvkm/subdev/bar/nv50.c +9 -3
index 8e64b19f3f8a..f23a0ccc2bec 100644
@@ -109,7 +109,7 @@ nv50_bar_oneinit(struct nvkm_bar *base)
 	struct nvkm_device *device = bar->base.subdev.device;
 	static struct lock_class_key bar1_lock;
 	static struct lock_class_key bar2_lock;
-	u64 start, limit;
+	u64 start, limit, size;
 	int ret;
 
 	ret = nvkm_gpuobj_new(device, 0x20000, 0, false, NULL, &bar->mem);
@@ -127,7 +127,10 @@ nv50_bar_oneinit(struct nvkm_bar *base)
 
 	/* BAR2 */
 	start = 0x0100000000ULL;
-	limit = start + device->func->resource_size(device, 3);
+	size = device->func->resource_size(device, 3);
+	if (!size)
+		return -ENOMEM;
+	limit = start + size;
 
 	ret = nvkm_vmm_new(device, start, limit-- - start, NULL, 0,
 			   &bar2_lock, "bar2", &bar->bar2_vmm);
@@ -164,7 +167,10 @@ nv50_bar_oneinit(struct nvkm_bar *base)
 
 	/* BAR1 */
 	start = 0x0000000000ULL;
-	limit = start + device->func->resource_size(device, 1);
+	size = device->func->resource_size(device, 1);
+	if (!size)
+		return -ENOMEM;
+	limit = start + size;
 
 	ret = nvkm_vmm_new(device, start, limit-- - start, NULL, 0,
 			   &bar1_lock, "bar1", &bar->bar1_vmm);
Signature · 6 lines
2.14.4

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau
#4 lore raw

PATCH 3/4 drm/nouveau/bar/gf100: ensure BAR is mapped

In reply to a message not in the archive

If the BAR is zero size, it indicates it was never successfully mapped. Ensure that the BAR is valid during initialization before attempting to use it.

---
 drivers/gpu/drm/nouveau/nvkm/subdev/bar/gf100.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/gpu/drm/nouveau/nvkm/subdev/bar/gf100.c +2 -0
index a3ba7f50198b..a3dcb09a40ee 100644
@@ -94,6 +94,8 @@ gf100_bar_oneinit_bar(struct gf100_bar *bar, struct gf100_barN *bar_vm,
 		return ret;
 
 	bar_len = device->func->resource_size(device, bar_nr);
+	if (!bar_len)
+		return -ENOMEM;
 	if (bar_nr == 3 && bar->bar2_halve)
 		bar_len >>= 1;
 
Signature · 6 lines
2.14.4

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau
#5 lore raw

PATCH 4/4 drm/nouveau/mmu: qualify vmm during dtor

In reply to a message not in the archive

If the BAR initialization failed it may leave the vmm structure in an unitialized state, leading to a null-pointer-dereference when the vmm is dereferenced during teardown.

---
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.c +1 -1
index fa93f964e6a4..41640e0584ac 100644
@@ -1783,7 +1783,7 @@ nvkm_vmm_get(struct nvkm_vmm *vmm, u8 page, u64 size, struct nvkm_vma **pvma)
 void
 nvkm_vmm_part(struct nvkm_vmm *vmm, struct nvkm_memory *inst)
 {
-	if (inst && vmm->func->part) {
+	if (inst && vmm && vmm->func->part) {
 		mutex_lock(&vmm->mutex);
 		vmm->func->part(vmm, inst);
 		mutex_unlock(&vmm->mutex);
Signature · 6 lines
2.14.4

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau
#6 lore raw
In reply to a message not in the archive
Quoted 2 lines · Jon Derrick

Hi Ben,

Hey John,

Quoted 7 lines

I've been working with an mmio-constrained pci hierarchy intended almost solely for nvme devices and switches. Binding nouveau to an NV50-based gpu results in a kernel panic as the device cannot be fully mapped.

I've made modifications in nv50 and vmm to unbind the driver from this hierarchy, and modified gf100 assuming it will have the same issue.

It would have the same issue indeed.

Quoted 4 lines

1/4 also includes a fix where nv50 doesn't look like it checks the return value for nvkm_vmm_new for bar1. I don't know if this was intentional (eg, it could work without it).

No, that's a good catch. It would have worked at one point in time, but would be a NULL-ptr deref since a rework that was done a while ago.

Quoted 3 lines

I'm wholly unfamiliar with linux graphics drivers so let me know if there's a better place for this.

This is the correct place, and I've merged all the patches into my tree.

Just as a note/reminder for the future (mostly to myself), we should already be able to deal with BAR2 not being available, there's a fallback in place already.

BAR1 is perhaps a bit trickier, but there are other (slow) HW mechanisms available that we could use if we need to care about making the driver work behind these hierarchies in the future.

Thank you,
Ben.

Quoted 12 lines

Jon Derrick (4):
drm/nouveau/bar/nv50: check bar1 vmm return value drm/nouveau/bar/nv50: ensure BAR is mapped drm/nouveau/bar/gf100: ensure BAR is mapped drm/nouveau/mmu: qualify vmm during dtor

 drivers/gpu/drm/nouveau/nvkm/subdev/bar/gf100.c |  2 ++
 drivers/gpu/drm/nouveau/nvkm/subdev/bar/nv50.c  | 14 +++++++++++---
 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.c   |  2 +-
 3 files changed, 14 insertions(+), 4 deletions(-)
Quoted 7 lines

--
2.14.4

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

A message that is not in the archive (sent off-list or not yet received): CACAvsv6kX8=Om+Nq2HWrwya9yba6rs1MXirHi0=zwTSFzkqy6Q-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#8 lore raw
Quoted 4 lines · Ben Skeggs
Quoted 1 line · Jon Derrick

Hi Ben,

Hey John,

Quoted 8 lines
Quoted 6 lines

I've been working with an mmio-constrained pci hierarchy intended almost solely for nvme devices and switches. Binding nouveau to an NV50-based gpu results in a kernel panic as the device cannot be fully mapped.

I've made modifications in nv50 and vmm to unbind the driver from this hierarchy, and modified gf100 assuming it will have the same issue.

It would have the same issue indeed.

Cheers. I was able to find a gf100 based card and confirmed the fix.

Thanks

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau