Skip to contentSkip to conversation

Enable atomic modesetting by default

PATCH Superseded Lyude Paul 2 participants 8 messages Jul 30

Subsystems (3)

v3 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

Lyude Paul

author maintainer
#0 lore raw

v3 cover Enable atomic modesetting by default

Nouveau is one of the very few modern hardware drivers in the kernel that doesn't have atomic modesetting enabled by default, in part because when it was originally written by Ben there wasn't much in the way of good atomic modesetting clients to actually test things out with.

Nowadays however, atomic modesetting is very much the norm - and support in userspace for non-atomic drivers is starting to bitrot a bit - leading to its own set of issues. At the same time, many of those issues are fixed by just turning on atomic in nouveau. Plus, I've been running nouveau with atomic modesetting on by default for most of the machines I work on, and I've already fixed quite a number of issues to the point where things seem quite stable (excluding a single screen flashing bug on my desktop, which I am not particularly convinced has anything to do with atomic modesetting).

This patch series enables atomic modesetting by default, at least for NV50 and newer. NV04 is left unchanged, as atomic modesetting was never implemented for it. This series also enforces leaving atomic modesetting disabled on <NV50, and adds the value of nouveau.atomic to our kernel debug output.

Previous version of this patch series:

	https://patchwork.freedesktop.org/series/171310/

Lyude Paul (5):
drm/nouveau: Fix cleanup bug in nouveau_drm_device_new() drm/nouveau: Print the nouveau.atomic parameter in
nouveau_display_options()
drm/nouveau: Fix drm_driver struct/nouveau.atomic parameter handling drm/nouveau/kms: Only allow enabling atomic modesetting on nv50+ drm/nouveau/kms/nv50-: Enable atomic modesetting by default

 drivers/gpu/drm/nouveau/nouveau_drm.c | 65 ++++++++++++++-------------
 drivers/gpu/drm/nouveau/nouveau_drv.h |  1 +
 2 files changed, 36 insertions(+), 30 deletions(-)

base-commit: 35b6cf07851a2b430b0eefbcb7b5a856f9030264

Signature · 1 line
2.55.0
author maintainer
#1 lore raw

PATCH v3 1/5 drm/nouveau: Fix cleanup bug in nouveau_drm_device_new()

Sashiko caught this while reviewing the patches for enabling atomic by default - if we fail to allocate the DRM device pointer, we'll attempt to free the error pointer that it returns rather than the actual struct. Let's fix this while we're at it.

---
V3:
* Don't use devm, that will just break during unbind (Sashiko).

 drivers/gpu/drm/nouveau/nouveau_drm.c | 28 +++++++++++++++------------
 1 file changed, 16 insertions(+), 12 deletions(-)
drivers/gpu/drm/nouveau/nouveau_drm.c +16 -12
index 4d1ad718e09b7..3dfe21cd4bde4 100644
@@ -749,7 +749,7 @@ nouveau_drm_device_new(const struct drm_driver *drm_driver, struct device *paren
 	drm->dev = drm_dev_alloc(drm_driver, parent);
 	if (IS_ERR(drm->dev)) {
 		ret = PTR_ERR(drm->dev);
-		goto done;
+		goto err_free_drm;
 	}
 
 	drm->dev->dev_private = drm;
@@ -762,39 +762,43 @@ nouveau_drm_device_new(const struct drm_driver *drm_driver, struct device *paren
 	ret = nvif_driver_init(NULL, nouveau_config, nouveau_debug, "drm",
 			       nouveau_name(drm->dev), &drm->_client);
 	if (ret)
-		goto done;
+		goto err_device_del;
 
 	ret = nvif_device_ctor(&drm->_client, "drmDevice", &drm->device);
 	if (ret) {
 		NV_ERROR(drm, "Device allocation failed: %d\n", ret);
-		goto done;
+		goto err_device_del;
 	}
 
 	ret = nvif_device_map(&drm->device);
 	if (ret) {
 		NV_ERROR(drm, "Failed to map PRI: %d\n", ret);
-		goto done;
+		goto err_device_del;
 	}
 
 	ret = nvif_mclass(&drm->device.object, mmus);
 	if (ret < 0) {
 		NV_ERROR(drm, "No supported MMU class\n");
-		goto done;
+		goto err_device_del;
 	}
 
 	ret = nvif_mmu_ctor(&drm->device.object, "drmMmu", mmus[ret].oclass, &drm->mmu);
 	if (ret) {
 		NV_ERROR(drm, "MMU allocation failed: %d\n", ret);
-		goto done;
+		goto err_device_del;
 	}
 
-done:
-	if (ret) {
-		nouveau_drm_device_del(drm);
-		drm = NULL;
-	}
+	return 0;
+
+err_free_drm:
+	kfree(drm);
+
+	return ERR_PTR(ret);
+
+err_device_del:
+	nouveau_drm_device_del(drm);
 
-	return ret ? ERR_PTR(ret) : drm;
+	return ERR_PTR(ret);
 }
 
 /*
Signature · 1 line
2.55.0
#2 lore raw

Re: [PATCH v3 1/5] drm/nouveau: Fix cleanup bug in nouveau_drm_device_new()

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] The patch erroneously changes the success return value of `nouveau_drm_device_new()` from…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The patch erroneously changes the success return value of nouveau_drm_device_new() from returning the allocated drm struct pointer to returning 0. In C, returning 0 in a pointer context evaluates to a NULL pointer, which bypasses IS_ERR() checks in callers and leads to an immediate kernel panic upon dereference. --

--- Patch [1]: [PATCH v3 1/5] drm/nouveau: Fix cleanup bug in nouveau_drm_device_new() --- commit 764ec8f5aa922a913d49de424c5e71dadb787207
Author: Lyude Paul <[email protected]>

drm/nouveau: Fix cleanup bug in nouveau_drm_device_new()

This patch refactors the error handling in nouveau_drm_device_new() to avoid freeing an error pointer if the DRM device pointer allocation fails. It replaces the single done label with specific error labels for different cleanup paths.

Quoted 4 lines
drivers/gpu/drm/nouveau/nouveau_drm.c +0 -0
index 4d1ad718e09b7..3dfe21cd4bde4 100644

[ ... ]

Quoted 1 line
drivers/gpu/drm/nouveau/nouveau_drm.c +0 -0
@@ -762,39 +762,43 @@ nouveau_drm_device_new(const struct drm_driver *drm_driver, struct device *paren

[ ... ]

drivers/gpu/drm/nouveau/nouveau_drm.c nouveau_drm_device_new(const struct drm_driver *drm_driver, struct device *paren · line 791

Quoted 13 lines
drivers/gpu/drm/nouveau/nouveau_drm.c +2 -6
 	ret = nvif_mmu_ctor(&drm->device.object, "drmMmu", mmus[ret].oclass, &drm->mmu);
 	if (ret) {
 		NV_ERROR(drm, "MMU allocation failed: %d\n", ret);
-		goto done;
+		goto err_device_del;
 	}
 
-done:
-	if (ret) {
-		nouveau_drm_device_del(drm);
-		drm = NULL;
-	}
+	return 0;

[Severity: High]
Does returning 0 here evaluate to a NULL pointer since the function's return type is a pointer to struct nouveau_drm?

Callers like nouveau_drm_probe() check the return value using IS_ERR():

drivers/gpu/drm/nouveau/nouveau_drm.c:nouveau_drm_probe() {
	...
	drm = nouveau_drm_device_new(&driver_pci, &pdev->dev, device);
	if (IS_ERR(drm)) {
		ret = PTR_ERR(drm);
		goto fail_nvkm;
	}

	ret = nouveau_drm_device_init(drm);
	...
}

Because IS_ERR(NULL) is false, the error check is bypassed. The NULL pointer is then passed to nouveau_drm_device_init() where it is immediately dereferenced:

drivers/gpu/drm/nouveau/nouveau_drm.c:nouveau_drm_device_init() {
	struct drm_device *dev = drm->dev;
	...
}

Could this lead to a kernel panic during device probe? Should this return the allocated drm pointer instead of 0?

Quoted 12 lines
drivers/gpu/drm/nouveau/nouveau_drm.c +9 -1
+
+err_free_drm:
+	kfree(drm);
+
+	return ERR_PTR(ret);
+
+err_device_del:
+	nouveau_drm_device_del(drm);
 
-	return ret ? ERR_PTR(ret) : drm;
+	return ERR_PTR(ret);
 }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author maintainer
#3 lore raw

PATCH v3 2/5 drm/nouveau: Print the nouveau.atomic parameter in nouveau_display_options()

In reply to Lyude Paul

Seems like we never remembered to start printing the value for this, so let's start to aid in troubleshooting in case we run into any issues with atomic.

---
 drivers/gpu/drm/nouveau/nouveau_drm.c | 1 +
 1 file changed, 1 insertion(+)
drivers/gpu/drm/nouveau/nouveau_drm.c +1 -0
index 3dfe21cd4bde4..6940e774ace07 100644
@@ -1420,6 +1420,7 @@ static void nouveau_display_options(void)
 	DRM_DEBUG_DRIVER("... debug        : %s\n", nouveau_debug);
 	DRM_DEBUG_DRIVER("... noaccel      : %d\n", nouveau_noaccel);
 	DRM_DEBUG_DRIVER("... modeset      : %d\n", nouveau_modeset);
+	DRM_DEBUG_DRIVER("... atomic       : %d\n", nouveau_atomic);
 	DRM_DEBUG_DRIVER("... runpm        : %d\n", nouveau_runtime_pm);
 	DRM_DEBUG_DRIVER("... vram_pushbuf : %d\n", nouveau_vram_pushbuf);
 	DRM_DEBUG_DRIVER("... hdmimhz      : %d\n", nouveau_hdmimhz);
Signature · 1 line
2.55.0
author maintainer
#4 lore raw

PATCH v3 3/5 drm/nouveau: Fix drm_driver struct/nouveau.atomic parameter handling

In reply to Lyude Paul

The way we handled the nouveau.atomic module parameter before was fairly broken, and had a number of issues:

- It was only ever actually parsed in the case of PCI devices.
- When nouveau.atomic was enabled, it would add the cap for atomic modesetting to the global driver_pci structure. This meant that if one GPU on a system supported atomic and another didn't, it would still get enabled for both.

Looking into this exposed further silliness in the way that we actually handle the drm_driver struct. We have one global structure for platform devices, and another for PCI devices - both of which are literally identical.

So before we start preparing to enable atomic modesetting by default, let's fix this. Instead of sharing driver_pci and driver_platform, we move the drm_driver struct we use over to the nouveau_drm struct, and then copy the contents of driver_stub over to it. We then convert driver_stub to a const, and move the handling of the nouveau.atomic module parameter into nouveau_drm_device_new(), and conditionally add the atomic modesetting capability to the embedded drm_driver struct.

Doing this is also preferable, as the next step for enabling atomic modesetting by default will be ensuring that we don't enable it for legacy devices that still don't support it. This requires only checking the atomic modesetting module parameter after the NVIF device is ready, as this allows us to check the GPU family that nouveau is running on.

---
V2:
* s/driver_pci/drm_driver/
* Dynamically allocate drm_driver struct, get rid of duplicate global driver structs to fix another Sashiko issue.
V3:
* Don't use devm (sashiko)

 drivers/gpu/drm/nouveau/nouveau_drm.c | 27 +++++++++++----------------
 drivers/gpu/drm/nouveau/nouveau_drv.h |  1 +
 2 files changed, 12 insertions(+), 16 deletions(-)
drivers/gpu/drm/nouveau/nouveau_drm.c +11 -16
index 6940e774ace07..95daf2643e1df 100644
@@ -111,9 +111,7 @@ MODULE_PARM_DESC(runpm, "disable (0), force enable (1), optimus only default (-1
 static int nouveau_runtime_pm = -1;
 module_param_named(runpm, nouveau_runtime_pm, int, 0400);
 
-static struct drm_driver driver_stub;
-static struct drm_driver driver_pci;
-static struct drm_driver driver_platform;
+static const struct drm_driver driver_stub;
 
 #ifdef CONFIG_DEBUG_FS
 struct dentry *nouveau_debugfs_root;
@@ -727,8 +725,7 @@ nouveau_drm_device_del(struct nouveau_drm *drm)
 }
 
 static struct nouveau_drm *
-nouveau_drm_device_new(const struct drm_driver *drm_driver, struct device *parent,
-		       struct nvkm_device *device)
+nouveau_drm_device_new(struct device *parent, struct nvkm_device *device)
 {
 	static const struct nvif_mclass
 	mmus[] = {
@@ -744,9 +741,10 @@ nouveau_drm_device_new(const struct drm_driver *drm_driver, struct device *paren
 	if (!drm)
 		return ERR_PTR(-ENOMEM);
 
+	drm->drm_driver = driver_stub;
 	drm->nvkm = device;
 
-	drm->dev = drm_dev_alloc(drm_driver, parent);
+	drm->dev = drm_dev_alloc(&drm->drm_driver, parent);
 	if (IS_ERR(drm->dev)) {
 		ret = PTR_ERR(drm->dev);
 		goto err_free_drm;
@@ -770,6 +768,9 @@ nouveau_drm_device_new(const struct drm_driver *drm_driver, struct device *paren
 		goto err_device_del;
 	}
 
+	if (nouveau_atomic)
+		drm->drm_driver.driver_features |= DRIVER_ATOMIC;
+
 	ret = nvif_device_map(&drm->device);
 	if (ret) {
 		NV_ERROR(drm, "Failed to map PRI: %d\n", ret);
@@ -877,16 +878,13 @@ static int nouveau_drm_probe(struct pci_dev *pdev,
 		return ret;
 
 	/* Remove conflicting drivers (vesafb, efifb etc). */
-	ret = aperture_remove_conflicting_pci_devices(pdev, driver_pci.name);
+	ret = aperture_remove_conflicting_pci_devices(pdev, driver_stub.name);
 	if (ret)
 		goto fail_nvkm;
 
 	pci_set_master(pdev);
 
-	if (nouveau_atomic)
-		driver_pci.driver_features |= DRIVER_ATOMIC;
-
-	drm = nouveau_drm_device_new(&driver_pci, &pdev->dev, device);
+	drm = nouveau_drm_device_new(&pdev->dev, device);
 	if (IS_ERR(drm)) {
 		ret = PTR_ERR(drm);
 		goto fail_nvkm;
@@ -1363,7 +1361,7 @@ nouveau_driver_fops = {
 	.fop_flags = FOP_UNSIGNED_OFFSET,
 };
 
-static struct drm_driver
+static const struct drm_driver
 driver_stub = {
 	.driver_features = DRIVER_GEM |
 			   DRIVER_SYNCOBJ | DRIVER_SYNCOBJ_TIMELINE |
@@ -1460,7 +1458,7 @@ nouveau_platform_device_create(const struct nvkm_device_tegra_func *func,
 	if (err)
 		goto err_free;
 
-	drm = nouveau_drm_device_new(&driver_platform, &pdev->dev, *pdevice);
+	drm = nouveau_drm_device_new(&pdev->dev, *pdevice);
 	if (IS_ERR(drm)) {
 		err = PTR_ERR(drm);
 		goto err_free;
@@ -1485,9 +1483,6 @@ nouveau_drm_init(void)
 {
 	int ret;
 
-	driver_pci = driver_stub;
-	driver_platform = driver_stub;
-
 	nouveau_display_options();
 
 	if (nouveau_modeset == -1) {
drivers/gpu/drm/nouveau/nouveau_drv.h +1 -0
index 5fc75dc750ed0..5cc0001134e74 100644
@@ -216,6 +216,7 @@ struct nouveau_drm {
 
 	struct nouveau_cli client;
 	struct drm_device *dev;
+	struct drm_driver drm_driver;
 
 	struct list_head clients;
 
Signature · 1 line
2.55.0
#5 lore raw

Re: [PATCH v3 3/5] drm/nouveau: Fix drm_driver struct/nouveau.atomic parameter handling

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

New issues:
- [High] Embedding the drm_driver struct inside the dynamically allocated nouveau_drm struct causes a Use-After-Free when the device is unplugged but userspace holds open DRM file descriptors.

Pre-existing issues:
- [Critical] nouveau_drm_device_new() incorrectly returns 0 (NULL) on success, causing an immediate kernel panic during device probe. --

--- Patch [3]: [PATCH v3 3/5] drm/nouveau: Fix drm_driver struct/nouveau.atomic parameter handling --- commit d192ca1eced4bd2c1c15cea6e0c5ae59ed261add
Author: Lyude Paul <[email protected]>

drm/nouveau: Fix drm_driver struct/nouveau.atomic parameter handling

The way we handled the nouveau.atomic module parameter before was fairly broken, and had a number of issues:

[ ... ]

Quoted 4 lines
drivers/gpu/drm/nouveau/nouveau_drm.c +0 -0
index 6940e774ace07..95daf2643e1df 100644

[ ... ]

drivers/gpu/drm/nouveau/nouveau_drm.c nouveau_drm_device_del(struct nouveau_drm *drm) · line 729

Quoted 8 lines
drivers/gpu/drm/nouveau/nouveau_drm.c +1 -2
@@ -727,8 +725,7 @@ nouveau_drm_device_del(struct nouveau_drm *drm)
 }
 
 static struct nouveau_drm *
-nouveau_drm_device_new(const struct drm_driver *drm_driver, struct device *parent,
-		       struct nvkm_device *device)
+nouveau_drm_device_new(struct device *parent, struct nvkm_device *device)
 {

[Severity: Critical]
This is a pre-existing issue (introduced earlier in the series), but does nouveau_drm_device_new() incorrectly return 0 (NULL) on success?

At the end of nouveau_drm_device_new(), it returns 0:

	return 0;

err_free_drm:

This causes callers like nouveau_drm_probe() to incorrectly bypass the IS_ERR() check, as IS_ERR(NULL) is false:

nouveau_drm_probe()

	drm = nouveau_drm_device_new(&pdev->dev, device);
	if (IS_ERR(drm)) {
		ret = PTR_ERR(drm);
		goto fail_nvkm;
	}

	ret = pci_enable_device(pdev);
	if (ret)
		goto fail_drm;

	ret = nouveau_drm_device_init(drm);

The NULL pointer is then passed to nouveau_drm_device_init(), which immediately dereferences it and causes a kernel panic during device probe.

[ ... ]

drivers/gpu/drm/nouveau/nouveau_drv.h struct nouveau_drm { · line 219

Quoted 9 lines
drivers/gpu/drm/nouveau/nouveau_drv.h +1 -0
index 5fc75dc750ed0..5cc0001134e74 100644
@@ -216,6 +216,7 @@ struct nouveau_drm {
 
 	struct nouveau_cli client;
 	struct drm_device *dev;
+	struct drm_driver drm_driver;

[Severity: High]
Can embedding the drm_driver struct inside the dynamically allocated nouveau_drm struct cause a use-after-free when the device is unplugged?

When the device is unbound, nouveau_drm_device_del() drops the driver's reference to the DRM device and then immediately frees the nouveau_drm struct using kfree():

drivers/gpu/drm/nouveau/nouveau_drm.c:nouveau_drm_device_del() {
	if (drm->dev)
		drm_dev_put(drm->dev);
	...
	kfree(drm);
}

Because drm_driver is now embedded in nouveau_drm, the drm_driver struct is also freed here.

If userspace still has an open file descriptor, the drm_device remains alive. When userspace later closes the file descriptor, DRM core functions (such as drm_release) will attempt to dereference dev->driver (e.g., dev->driver->postclose), resulting in a use-after-free.

Quoted 3 lines
drivers/gpu/drm/nouveau/nouveau_drv.h +0 -0
 	struct list_head clients;
 
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
author maintainer
#6 lore raw

PATCH v3 4/5 drm/nouveau/kms: Only allow enabling atomic modesetting on nv50+

In reply to Lyude Paul

Atomic modesetting support was never added for pre-nv50 chipsets, so make sure we don't allow it to be forced on. Additionally, print a small warning when it's not supported.

---
V2:
* Check against info.family, not info.chipset

 drivers/gpu/drm/nouveau/nouveau_drm.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/gpu/drm/nouveau/nouveau_drm.c +6 -2
index 95daf2643e1df..80fdedc27189f 100644
@@ -768,8 +768,12 @@ nouveau_drm_device_new(struct device *parent, struct nvkm_device *device)
 		goto err_device_del;
 	}
 
-	if (nouveau_atomic)
-		drm->drm_driver.driver_features |= DRIVER_ATOMIC;
+	if (nouveau_atomic) {
+		if (drm->device.info.family >= NV_DEVICE_INFO_V0_TESLA)
+			drm->drm_driver.driver_features |= DRIVER_ATOMIC;
+		else
+			NV_WARN(drm, "Atomic modesetting not supported (needs nv50+)\n");
+	}
 
 	ret = nvif_device_map(&drm->device);
 	if (ret) {
Signature · 1 line
2.55.0
author maintainer
#7 lore raw

PATCH v3 5/5 drm/nouveau/kms/nv50-: Enable atomic modesetting by default

In reply to Lyude Paul

Nouveau is one of the very few modern hardware drivers in the kernel that doesn't have atomic modesetting enabled by default, in part because when it was originally written by Ben there wasn't much in the way of good atomic modesetting clients to actually test things out with.

Nowadays however, atomic modesetting is very much the norm - and support in userspace for non-atomic drivers is starting to bitrot a bit - leading to its own set of issues. At the same time, many of those issues are fixed by just turning on atomic in nouveau. Plus, I've been running nouveau with atomic modesetting on by default for most of the machines I work on, and I've already fixed quite a number of issues to the point where things seem quite stable (excluding a single screen flashing bug on my desktop, which I am not particularly convinced has anything to do with atomic modesetting).

Now that we've protected against breaking things for chipsets where atomic isn't supported (<nv50) - let's enable it by default on generations of hardware that support it. We'll leave the module parameter around for the time being, as turning it off may be helpful in the possibility that we hit regressions.

---
 drivers/gpu/drm/nouveau/nouveau_drm.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
drivers/gpu/drm/nouveau/nouveau_drm.c +5 -4
index 80fdedc27189f..b3b90d354b5ca 100644
@@ -103,8 +103,9 @@ MODULE_PARM_DESC(modeset, "enable driver (default: auto, "
 int nouveau_modeset = -1;
 module_param_named(modeset, nouveau_modeset, int, 0400);
 
-MODULE_PARM_DESC(atomic, "Expose atomic ioctl (default: disabled)");
-static int nouveau_atomic = 0;
+MODULE_PARM_DESC(atomic, "Expose atomic ioctl (default: auto, "
+			 "0 = disabled, 1 = enabled)");
+static int nouveau_atomic = -1;
 module_param_named(atomic, nouveau_atomic, int, 0400);
 
 MODULE_PARM_DESC(runpm, "disable (0), force enable (1), optimus only default (-1)");
@@ -768,10 +769,10 @@ nouveau_drm_device_new(struct device *parent, struct nvkm_device *device)
 		goto err_device_del;
 	}
 
-	if (nouveau_atomic) {
+	if (nouveau_atomic != 0) {
 		if (drm->device.info.family >= NV_DEVICE_INFO_V0_TESLA)
 			drm->drm_driver.driver_features |= DRIVER_ATOMIC;
-		else
+		else if (nouveau_atomic == 1)
 			NV_WARN(drm, "Atomic modesetting not supported (needs nv50+)\n");
 	}
 
Signature · 1 line
2.55.0