virtio: disable notification hardening by default
Subsystems (4)
PATCH v2 1/1 virtio: disable notification hardening by default
We try to harden virtio device notifications in 8b4ec69d7e09 ("virtio:
harden vring IRQ"). It works with the assumption that the driver or core can properly call virtio_device_ready() at the right place. Unfortunately, this seems to be not true and uncover various bugs of the existing drivers, mainly the issue of using virtio_device_ready() incorrectly.
So let's having a Kconfig option and disable it by default. It gives us a breath to fix the drivers and then we can consider to enable it by default.
---
Changes since V1:
- tweak the Kconfig prompt
- don't hold spinlock for IRQ path in s390
--- drivers/s390/virtio/virtio_ccw.c | 4 ++++ drivers/virtio/Kconfig | 11 +++++++++++ drivers/virtio/virtio.c | 2 ++ drivers/virtio/virtio_ring.c | 12 ++++++++++++ include/linux/virtio_config.h | 2 ++ 5 files changed, 31 insertions(+)
drivers/s390/virtio/virtio_ccw.c +4 -0
@@ -1136,8 +1136,10 @@ static void virtio_ccw_int_handler(struct ccw_device *cdev, vcdev->err = -EIO ; }virtio_ccw_check_activity (vcdev, activity); +#ifdef CONFIG_VIRTIO_HARDEN_NOTIFICATION /* Interrupts are disabled here */ read_lock (&vcdev->irq_lock); +#endif for_each_set_bit (i,indicators (vcdev),sizeof (*indicators (vcdev)) *BITS_PER_BYTE ) {/* The bit clear must happen before the vring kick. */
@@ -1146,7 +1148,9 @@ static void virtio_ccw_int_handler(struct ccw_device *cdev, vq =virtio_ccw_vq_by_ind (vcdev, i);vring_interrupt (0 , vq); } +#ifdef CONFIG_VIRTIO_HARDEN_NOTIFICATION read_unlock (&vcdev->irq_lock); +#endif if (test_bit (0 ,indicators2 (vcdev))) {virtio_config_changed (&vcdev->vdev);clear_bit (0 ,indicators2 (vcdev));
drivers/virtio/Kconfig +11 -0
@@ -35,6 +35,17 @@ menuconfig VIRTIO_MENU if VIRTIO_MENU +config VIRTIO_HARDEN_NOTIFICATION + bool "Harden virtio notification" + help + Enable this to harden the device notifications and supress + the ones that are illegal. + + Experimental: not all drivers handle this correctly at this + point. + + If unsure, say N. + config VIRTIO_PCI tristate "PCI driver for virtio devices" depends on PCI
drivers/virtio/virtio.c +2 -0
@@ -220,6 +220,7 @@ static int virtio_features_ok(struct virtio_device *dev)* */ void virtio_reset_device (struct virtio_device *dev) { +#ifdef CONFIG_VIRTIO_HARDEN_NOTIFICATION /* * The below virtio_synchronize_cbs() guarantees that any * interrupt for this line arriving after
@@ -228,6 +229,7 @@ void virtio_reset_device(struct virtio_device *dev)*/ virtio_break_device (dev);virtio_synchronize_cbs (dev); +#endif dev->config->reset (dev); }
drivers/virtio/virtio_ring.c +12 -0
@@ -1688,7 +1688,11 @@ static struct virtqueue *vring_create_virtqueue_packed( vq->we_own_ring = true; vq->notify = notify; vq->weak_barriers = weak_barriers; +#ifdef CONFIG_VIRTIO_HARDEN_NOTIFICATION vq->broken = true; +#else + vq->broken = false; +#endif vq->last_used_idx =0 ; vq->event_triggered = false; vq->num_added =0 ;
@@ -2135,9 +2139,13 @@ irqreturn_t vring_interrupt(int irq, void *_vq) }if (unlikely (vq->broken)) { +#ifdef CONFIG_VIRTIO_HARDEN_NOTIFICATION dev_warn_once (&vq->vq.vdev->dev,"virtio vring IRQ raised before DRIVER_OK" );return IRQ_NONE ; +#else +return IRQ_HANDLED ; +#endif }/* Just a hint for performance: so it's ok that this can be racy! */
@@ -2180,7 +2188,11 @@ struct virtqueue *__vring_new_virtqueue(unsigned int index, vq->we_own_ring = false; vq->notify = notify; vq->weak_barriers = weak_barriers; +#ifdef CONFIG_VIRTIO_HARDEN_NOTIFICATION vq->broken = true; +#else + vq->broken = false; +#endif vq->last_used_idx =0 ; vq->event_triggered = false; vq->num_added =0 ;
include/linux/virtio_config.h +2 -0
@@ -257,6 +257,7 @@ void virtio_device_ready(struct virtio_device *dev)WARN_ON (status &VIRTIO_CONFIG_S_DRIVER_OK ); +#ifdef CONFIG_VIRTIO_HARDEN_NOTIFICATION /* * The virtio_synchronize_cbs() makes sure vring_interrupt() * will see the driver specific setup if it sees vq->broken
@@ -264,6 +265,7 @@ void virtio_device_ready(struct virtio_device *dev)*/ virtio_synchronize_cbs (dev);__virtio_unbreak_device (dev); +#endif /* * The transport should ensure the visibility of vq->broken * before setting DRIVER_OK. See the comments for the transport
Signature · 1 line
2.25.1
Cornelia Huck
maintainerQuoted 29 lines · Jason Wang
We try to harden virtio device notifications in
8b4ec69d7e09("virtio:
harden vring IRQ"). It works with the assumption that the driver or core can properly callvirtio_device_ready()at the right place. Unfortunately, this seems to be not true and uncover various bugs of the existing drivers, mainly the issue of usingvirtio_device_ready()incorrectly.So let's having a
Kconfigoption and disable it by default. It gives us a breath to fix the drivers and then we can consider to enable it by default.Signed-off-by Jason Wang <[email protected]>---
Changes since V1:
- tweak theKconfigprompt
- don't hold spinlock for IRQ path in s390--- drivers/s390/virtio/virtio_ccw.c | 4 ++++ drivers/virtio/Kconfig | 11 +++++++++++ drivers/virtio/virtio.c | 2 ++ drivers/virtio/virtio_ring.c | 12 ++++++++++++ include/linux/virtio_config.h | 2 ++ 5 files changed, 31 insertions(+)drivers/s390/virtio/virtio_ccw.c +0
-0
drivers/s390/virtio/virtio_ccw.c static void virtio_ccw_int_handler(struct ccw_device *cdev, · line 1141
Quoted 6 lines
drivers/s390/virtio/virtio_ccw.c +1
-0@@ -1136,8 +1136,10 @@ static void virtio_ccw_int_handler(struct ccw_device *cdev, vcdev->err = -EIO ; }virtio_ccw_check_activity (vcdev, activity); +#ifdef CONFIG_VIRTIO_HARDEN_NOTIFICATION /* Interrupts are disabled here */ read_lock (&vcdev->irq_lock);
Should we add a comment that this pairs with virtio_ccw_synchronize_cbs()? Just to avoid future headscratching as to why this lock is only needed when notification hardening is enabled.
Quoted 19 lines
drivers/s390/virtio/virtio_ccw.c +3
-0+#endif for_each_set_bit (i,indicators (vcdev),sizeof (*indicators (vcdev)) *BITS_PER_BYTE ) {/* The bit clear must happen before the vring kick. */ @@ -1146,7 +1148,9 @@ static void virtio_ccw_int_handler(struct ccw_device *cdev, vq =virtio_ccw_vq_by_ind (vcdev, i);vring_interrupt (0 , vq); } +#ifdef CONFIG_VIRTIO_HARDEN_NOTIFICATION read_unlock (&vcdev->irq_lock); +#endif if (test_bit (0 ,indicators2 (vcdev))) {virtio_config_changed (&vcdev->vdev);clear_bit (0 ,indicators2 (vcdev));drivers/virtio/Kconfig +0
-0
drivers/virtio/Kconfig menuconfig VIRTIO_MENU · line 42
Quoted 8 lines
drivers/virtio/Kconfig +5
-0@@ -35,6 +35,17 @@ menuconfig VIRTIO_MENU if VIRTIO_MENU +config VIRTIO_HARDEN_NOTIFICATION + bool "Harden virtio notification" + help + Enable this to harden the device notifications and supress + the ones that are illegal.
"...and suppress those that happen at a time where notifications are illegal." ?
drivers/virtio/Kconfig menuconfig VIRTIO_MENU · line 45
Quoted 3 lines
drivers/virtio/Kconfig +3
-0+ + Experimental: not all drivers handle this correctly at this + point.
"Note that several drivers still have bugs that may cause crashes or hangs when correct handling of notifications is enforced; depending on the subset of drivers and devices you use, this may or may not work."
Or is that too verbose?
drivers/virtio/Kconfig menuconfig VIRTIO_MENU · line 51
Quoted 6 lines
drivers/virtio/Kconfig +3
-0+ + If unsure, say N. + config VIRTIO_PCI tristate "PCI driver for virtio devices" depends on PCI
The ifdeffery looks a big ugly, but I don't have a better idea.
Quoted 27 lines · Cornelia Huck
Quoted 24 lines · Jason Wang
We try to harden virtio device notifications in
8b4ec69d7e09("virtio:
harden vring IRQ"). It works with the assumption that the driver or core can properly callvirtio_device_ready()at the right place. Unfortunately, this seems to be not true and uncover various bugs of the existing drivers, mainly the issue of usingvirtio_device_ready()incorrectly.So let's having a
Kconfigoption and disable it by default. It gives us a breath to fix the drivers and then we can consider to enable it by default.Signed-off-by Jason Wang <[email protected]>---
Changes since V1:
- tweak theKconfigprompt
- don't hold spinlock for IRQ path in s390--- drivers/s390/virtio/virtio_ccw.c | 4 ++++ drivers/virtio/Kconfig | 11 +++++++++++ drivers/virtio/virtio.c | 2 ++ drivers/virtio/virtio_ring.c | 12 ++++++++++++ include/linux/virtio_config.h | 2 ++ 5 files changed, 31 insertions(+)
Quoted 15 lines
drivers/s390/virtio/virtio_ccw.c static void virtio_ccw_int_handler(struct ccw_device *cdev, · line 1141
Quoted 11 lines
drivers/s390/virtio/virtio_ccw.c +1
-0@@ -1136,8 +1136,10 @@ static void virtio_ccw_int_handler(struct ccw_device *cdev, vcdev->err = -EIO ; }virtio_ccw_check_activity (vcdev, activity); +#ifdef CONFIG_VIRTIO_HARDEN_NOTIFICATION /* Interrupts are disabled here */ read_lock (&vcdev->irq_lock);Should we add a comment that this pairs with
virtio_ccw_synchronize_cbs()? Just to avoid future headscratching as to why this lock is only needed when notification hardening is enabled.
Fine.
Quoted 23 lines
Quoted 23 lines
drivers/s390/virtio/virtio_ccw.c +3
-0+#endif for_each_set_bit (i,indicators (vcdev),sizeof (*indicators (vcdev)) *BITS_PER_BYTE ) {/* The bit clear must happen before the vring kick. */ @@ -1146,7 +1148,9 @@ static void virtio_ccw_int_handler(struct ccw_device *cdev, vq =virtio_ccw_vq_by_ind (vcdev, i);vring_interrupt (0 , vq); } +#ifdef CONFIG_VIRTIO_HARDEN_NOTIFICATION read_unlock (&vcdev->irq_lock); +#endif if (test_bit (0 ,indicators2 (vcdev))) {virtio_config_changed (&vcdev->vdev);clear_bit (0 ,indicators2 (vcdev));drivers/virtio/Kconfig +0
-0@@ -35,6 +35,17 @@ menuconfig VIRTIO_MENU if VIRTIO_MENU
Quoted 8 lines
drivers/virtio/Kconfig menuconfig VIRTIO_MENU · line 42
Quoted 5 lines
drivers/virtio/Kconfig +5
-0+config VIRTIO_HARDEN_NOTIFICATION + bool "Harden virtio notification" + help + Enable this to harden the device notifications and supress + the ones that are illegal."...and suppress those that happen at a time where notifications are illegal." ?
Ok.
Quoted 10 lines
drivers/virtio/Kconfig menuconfig VIRTIO_MENU · line 45
Quoted 4 lines
drivers/virtio/Kconfig +3
-0+ + Experimental: not all drivers handle this correctly at this + point."Note that several drivers still have bugs that may cause crashes or hangs when correct handling of notifications is enforced; depending on the subset of drivers and devices you use, this may or may not work."
Or is that too verbose?
Looks fine.
Quoted 9 lines
drivers/virtio/Kconfig menuconfig VIRTIO_MENU · line 51
Quoted 7 lines
drivers/virtio/Kconfig +3
-0+ + If unsure, say N. + config VIRTIO_PCI tristate "PCI driver for virtio devices" depends on PCIThe ifdeffery looks a big ugly, but I don't have a better idea.
I guess you meant the ccw part, I leave the spinlock here in V1, but Michael prefers to have that.
In the future, we may consider removing that, one possible way is to have a per driver boolean for the hardening.
Tanks
Quoted 1 line
Cornelia Huck
maintainerQuoted 6 lines · Jason Wang
Quoted 2 lines · Cornelia Huck
The ifdeffery looks a big ugly, but I don't have a better idea.
I guess you meant the ccw part, I leave the spinlock here in V1, but Michael prefers to have that.
Not doing the locking dance is good; I think the #ifdefs all over are a bit ugly, but as I said, I can't think of a good, less-ugly way...
Quoted 2 lines
In the future, we may consider removing that, one possible way is to have a per driver boolean for the hardening.
As in "we've reviewed and tested this driver, so let's turn it on for every device bound to it"?
_______________________________________________
Virtualization mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Quoted 5 lines · Cornelia Huck
Quoted 2 lines · Jason Wang
Quoted 1 line · Cornelia Huck
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
The ifdeffery looks a big ugly, but I don't have a better idea.
I guess you meant the ccw part, I leave the spinlock here in V1, but Michael prefers to have that.
Not doing the locking dance is good; I think the #ifdefs all over are a bit ugly, but as I said, I can't think of a good, less-ugly way...
Probably, but this is the way that is used by other subsystems. E.g CONFIG_HARDEN_USERCOPY etc.
Quoted 6 lines
Quoted 3 lines
In the future, we may consider removing that, one possible way is to have a per driver boolean for the hardening.
As in "we've reviewed and tested this driver, so let's turn it on for every device bound to it"?
Right.
Thanks