Skip to contentSkip to conversation

xen/pciback: Prevent NULL pointer dereference in quirks_show

PATCH In review Stefan Nuernberger 3 participants 6 messages Dec 6, 2019 – Dec 9, 2019

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xen/pciback: Prevent NULL pointer dereference in quirks_show

From: Uwe Dannowski <[email protected]>

Reading /sys/bus/pci/drivers/pciback/quirks while unbinding can result in dereferencing a NULL pointer. Instead, skip printing information about the dangling quirk.

---
 drivers/xen/xen-pciback/pci_stub.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/xen/xen-pciback/pci_stub.c +2 -0
index 097410a7cdb7..da725e474294 100644
@@ -1346,6 +1346,8 @@ static ssize_t quirks_show(struct device_driver *drv, char *buf)
 				   quirk->devid.subdevice);
 
 		dev_data = pci_get_drvdata(quirk->pdev);
+		if (!dev_data)
+			continue;
 
 		list_for_each_entry(cfg_entry, &dev_data->config_fields, list) {
 			field = cfg_entry->field;
Signature · 12 lines
2.23.0




Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879
#1 lore raw
Quoted 21 lines · Stefan Nuernberger

From: Uwe Dannowski <[email protected]>

Reading /sys/bus/pci/drivers/pciback/quirks while unbinding can result in dereferencing a NULL pointer. Instead, skip printing information about the dangling quirk.

---
 drivers/xen/xen-pciback/pci_stub.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/xen/xen-pciback/pci_stub.c +0 -0
index 097410a7cdb7..da725e474294 100644

drivers/xen/xen-pciback/pci_stub.c static ssize_t quirks_show(struct device_driver *drv, char *buf) · line 1352

Quoted 7 lines
drivers/xen/xen-pciback/pci_stub.c +2 -0
@@ -1346,6 +1346,8 @@ static ssize_t quirks_show(struct device_driver *drv, char *buf)
 				   quirk->devid.subdevice);
 
 		dev_data = pci_get_drvdata(quirk->pdev);
+		if (!dev_data)
+			continue;
 
 		list_for_each_entry(cfg_entry, &dev_data->config_fields, list) {

Couldn't you have the same race here?

-boris

Quoted 1 line
drivers/xen/xen-pciback/pci_stub.c +0 -0
 			field = cfg_entry->field;
#2 lore raw
Quoted 33 lines · Boris Ostrovsky
Quoted 32 lines · Stefan Nuernberger

From: Uwe Dannowski <[email protected]>

Reading /sys/bus/pci/drivers/pciback/quirks while unbinding can result
in dereferencing a NULL pointer. Instead, skip printing information about the dangling quirk.

---
 drivers/xen/xen-pciback/pci_stub.c | 2 ++

 1 file changed, 2 insertions(+)
drivers/xen/xen- renamed from drivers/xen/xen-pciback/pci_stub.c +0 -0

pciback/pci_stub.c
index 097410a7cdb7..da725e474294 100644

drivers/xen/xen-pciback/pci_stub.c +0 -0
@@ -1346,6 +1346,8 @@ static ssize_t quirks_show(struct
 device_driver *drv, char *buf)
  				   quirk->devid.subdevice);


  dev_data = pci_get_drvdata(quirk->pdev);

drivers/xen/xen-pciback/pci_stub.c +2 -0
+		if (!dev_data)
+			continue;


  list_for_each_entry(cfg_entry, &dev_data-

Quoted 2 lines
Quoted 1 line
Quoted 1 line

config_fields, list) {

Couldn't you have the same race here?

Not quite the same, but it might not be entirely safe yet. The 'quirks_show' takes the 'device_ids_lock' and races with unbind / 'pcistub_device_release' "which takes device_lock mutex". So this might now be a UAF read access instead of a NULL pointer dereference. We have not observed adversarial effects in our testing (compared to the obvious issues with NULL pointer) but that's not a guarantee of course.

So should quirks_show actually be protected by pcistub_devices_lock instead as are other functions that access dev_data? Does it need both locks in that case?

-Stefan

Quoted 5 lines

-boris

Quoted 2 lines

  field = cfg_entry->field;

Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879

#3 lore raw
Quoted 33 lines · Stefan
Quoted 32 lines · Boris Ostrovsky
Quoted 31 lines · Stefan Nuernberger

From: Uwe Dannowski <[email protected]>

Reading /sys/bus/pci/drivers/pciback/quirks while unbinding can result
in dereferencing a NULL pointer. Instead, skip printing information about the dangling quirk.

---
 drivers/xen/xen-pciback/pci_stub.c | 2 ++

 1 file changed, 2 insertions(+)
drivers/xen/xen- renamed from drivers/xen/xen-pciback/pci_stub.c +0 -0

pciback/pci_stub.c
index 097410a7cdb7..da725e474294 100644

drivers/xen/xen-pciback/pci_stub.c +0 -0
@@ -1346,6 +1346,8 @@ static ssize_t quirks_show(struct
 device_driver *drv, char *buf)
  				   quirk->devid.subdevice);


  dev_data = pci_get_drvdata(quirk->pdev);

drivers/xen/xen-pciback/pci_stub.c +2 -0
+		if (!dev_data)
+			continue;


  list_for_each_entry(cfg_entry, &dev_data-

Quoted 6 lines
Quoted 2 lines
Quoted 1 line
Quoted 1 line

config_fields, list) {

Couldn't you have the same race here?

Not quite the same, but it might not be entirely safe yet. The 'quirks_show' takes the 'device_ids_lock' and races with unbind / 'pcistub_device_release' "which takes device_lock mutex". So this might now be a UAF read access instead of a NULL pointer dereference.

Yes, that's what I meant (although I don't see much difference in this context).

Quoted 7 lines

We have
not observed adversarial effects in our testing (compared to the obvious issues with NULL pointer) but that's not a guarantee of course.

So should quirks_show actually be protected by pcistub_devices_lock instead as are other functions that access dev_data? Does it need both locks in that case?

device_ids_lock protects device_ids list, which is not what you are trying to access, so that doesn't look like right lock to hold. And AFAICT pcistub_devices_lock is not held when device data is cleared in pcistub_device_release() (which I think is where we are racing).

-boris

#4 lore raw
Quoted 12 lines · Boris Ostrovsky
Quoted 11 lines · Stefan
Quoted 9 lines · Boris Ostrovsky
Quoted 6 lines · Stefan Nuernberger

From: Uwe Dannowski <[email protected]>

  list_for_each_entry(cfg_entry, &dev_data-

Quoted 2 lines

config_fields, list) {

Couldn't you have the same race here?

Quoted 8 lines
Quoted 5 lines

Not quite the same, but it might not be entirely safe yet. The 'quirks_show' takes the 'device_ids_lock' and races with unbind / 'pcistub_device_release' "which takes device_lock mutex". So this might
now be a UAF read access instead of a NULL pointer dereference.

Yes, that's what I meant (although I don't see much difference in this
context).

Well, the NULL ptr access causes an instant kernel panic whereas we have not attributed crashes to the possible UAF read until now.

Quoted 10 lines
Quoted 10 lines

 We have
not observed adversarial effects in our testing (compared to the obvious issues with NULL pointer) but that's not a guarantee of course.

So should quirks_show actually be protected by pcistub_devices_lock instead as are other functions that access dev_data? Does it need both
locks in that case?

Quoted 5 lines

device_ids_lock protects device_ids list, which is not what you are trying to access, so that doesn't look like right lock to hold. And AFAICT pcistub_devices_lock is not held when device data is cleared in
pcistub_device_release() (which I think is where we are racing).

Indeed. The xen_pcibk_quirks list does not have a separate lock to protect it. It's either modified under 'pcistub_devices_lock', from pcistub_remove(), or iterated over with the 'device_ids_lock' held in quirks_show(). Also the quirks list is amended from pcistub_init_device()
    -> xen_pcibk_config_init_dev()
      -> xen_pcibk_config_quirks_init()
without holding any lock at all. In fact the pcistub_init_devices_late() and pcistub_seize() functions deliberately release the pcistub_devices_lock before calling pcistub_init_device(). This looks broken.

The race is between
  pcistub_remove()
    -> pcistub_device_put()
      -> pcistub_device_release()
on one side and the quirks_show() on the other side. The problematic quirk is freed from the xen_pcibk_quirks list in pcistub_remove() early on under pcistub_devices_lock before the associated dev_data is freed eventually. So switching from device_ids_lock to pcistub_devices_lock in quirks_show() could be sufficient to always have valid dev_data for all quirks in the list.

There is also pcistub_put_pci_dev() possibly in the race, called from xen_pcibk_remove_device(), or xen_pcibk_xenbus_remove(), or pcistub_remove(). The pcistub_remove() call site is safe when we switch to pcistub_devices_lock (same reasoning as above). For the others I currently do not see when the quirks are ever freed?

- Stefan

Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879

#5 lore raw
Quoted 34 lines · Stefan
Quoted 8 lines · Boris Ostrovsky
Quoted 7 lines · Stefan
Quoted 6 lines · Boris Ostrovsky
Quoted 4 lines · Stefan Nuernberger

From: Uwe Dannowski <[email protected]>

  list_for_each_entry(cfg_entry, &dev_data-

Quoted 1 line

config_fields, list) {

Couldn't you have the same race here?

Quoted 8 lines
Quoted 5 lines

Not quite the same, but it might not be entirely safe yet. The 'quirks_show' takes the 'device_ids_lock' and races with unbind / 'pcistub_device_release' "which takes device_lock mutex". So this might
now be a UAF read access instead of a NULL pointer dereference.

Yes, that's what I meant (although I don't see much difference in this
context).

Well, the NULL ptr access causes an instant kernel panic whereas we have not attributed crashes to the possible UAF read until now.

Quoted 9 lines
Quoted 9 lines

 We have
not observed adversarial effects in our testing (compared to the obvious issues with NULL pointer) but that's not a guarantee of course.

So should quirks_show actually be protected by pcistub_devices_lock instead as are other functions that access dev_data? Does it need both
locks in that case?

Quoted 5 lines

device_ids_lock protects device_ids list, which is not what you are trying to access, so that doesn't look like right lock to hold. And AFAICT pcistub_devices_lock is not held when device data is cleared in
pcistub_device_release() (which I think is where we are racing).

Quoted 11 lines

Indeed. The xen_pcibk_quirks list does not have a separate lock to protect it. It's either modified under 'pcistub_devices_lock', from pcistub_remove(), or iterated over with the 'device_ids_lock' held in quirks_show(). Also the quirks list is amended from
  pcistub_init_device()
    -> xen_pcibk_config_init_dev()
      -> xen_pcibk_config_quirks_init()
without holding any lock at all. In fact the pcistub_init_devices_late() and pcistub_seize() functions deliberately release the pcistub_devices_lock before calling pcistub_init_device(). This looks broken.

Indeed.

Quoted 11 lines

The race is between
  pcistub_remove()
    -> pcistub_device_put()
      -> pcistub_device_release()
on one side and the quirks_show() on the other side. The problematic quirk is freed from the xen_pcibk_quirks list in pcistub_remove() early on under pcistub_devices_lock before the associated dev_data is freed eventually. So switching from device_ids_lock to pcistub_devices_lock in quirks_show() could be sufficient to always have valid dev_data for all quirks in the list.

Yes, that should do it. (I missed xen_pcibk_config_quirk_release() call, which is why I wasn't sure pcistub_devices_lock is held where necessary).

Quoted 6 lines

There is also pcistub_put_pci_dev() possibly in the race, called from xen_pcibk_remove_device(), or xen_pcibk_xenbus_remove(), or pcistub_remove(). The pcistub_remove() call site is safe when we switch to pcistub_devices_lock (same reasoning as above). For the others I currently do not see when the quirks are ever freed?

I wonder whether we should call xen_pcibk_config_quirk_release() from pcistub_device_release() under pcistub_devices_lock.

-boris