VIRTIO_F_IO_BARRIER: use I/O barriers in driver
RFC 1/1 VIRTIO_F_IO_BARRIER: use I/O barriers in driver
There will be hardware virtio devices in the future, which require drivers to use the barriers suitable for I/O device, compared with software virtio devices which just require drivers to use the barriers suitable for CPU core.
To fix the ordering issue for hardware virtio devices, add a new feature: VIRTIO_F_IO_BARRIER. When negotiated, driver will use the barriers suitable for I/O device.
--- content.tex | 9 +++++++++ 1 file changed, 9 insertions(+)
content.tex +9 -0
@@ -5445,6 +5445,8 @@ Descriptors} and \ref{sec:Packed Virtqueues / Indirect Flag: Scatter-Gather Supp that drivers pass extra data (besides identifying the Virtqueue) in their device notifications. See \ref{sec:Virtqueues / Driver notifications}~\nameref{sec:Virtqueues / Driver notifications}. + \item[VIRTIO_F_IO_BARRIER(37)] This feature indicates that the device + needs the driver to use the barriers suitable for hardware device. \end{description} \drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}
@@ -5460,6 +5462,10 @@ addresses to the device. A driver SHOULD accept VIRTIO_F_RING_PACKED if it is offered. +A driver SHOULD accept VIRTIO_F_IO_BARRIER if it is offered. +If VIRTIO_F_IO_BARRIER has been negotiated, a driver MUST use +the barriers suitable for hardware device. + \devicenormative{\section}{Reserved Feature Bits}{Reserved Feature Bits} A device MUST offer VIRTIO_F_VERSION_1. A device MAY fail to operate further
@@ -5473,6 +5479,9 @@ accepted. If VIRTIO_F_IN_ORDER has been negotiated, a device MUST use buffers in the same order in which they have been available. +A device MAY fail to operate further if VIRTIO_F_IO_BARRIER +is not accepted. + \section{Legacy Interface: Reserved Feature Bits}\label{sec:Reserved Feature Bits / Legacy Interface: Reserved Feature Bits} Transitional devices MAY offer the following:
Signature · 6 lines
2.11.0 --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 8 lines · Tiwei Bie
There will be hardware virtio devices in the future, which require drivers to use the barriers suitable for I/O device, compared with software virtio devices which just require drivers to use the barriers suitable for CPU core.
To fix the ordering issue for hardware virtio devices, add a new feature:
VIRTIO_F_IO_BARRIER. When negotiated, driver will use the barriers suitable for I/O device.
Unfortunately this doesn't work as mentioned earlier. Virtio live migration assumes that features are safe if you have them on the destination but not on the source; this feature however works the opposite way.
Paolo
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 6 lines · Paolo Bonzini
Quoted 5 lines · Tiwei Bie
There will be hardware virtio devices in the future, which require drivers to use the barriers suitable for I/O device, compared with software virtio devices which just require drivers to use the barriers suitable for CPU core.
Quoted 8 lines
Quoted 3 lines
To fix the ordering issue for hardware virtio devices, add a new feature:
VIRTIO_F_IO_BARRIER. When negotiated, driver will use the barriers suitable for I/O device.Unfortunately this doesn't work as mentioned earlier. Virtio live migration assumes that features are safe if you have them on the destination but not on the source; this feature however works the opposite way.
If my understanding is correct, I think Michael suggested to add code to disallow migration between setups with and without the feature:
http://lists.nongnu.org/archive/html/qemu-devel/2018-04/msg03094.html
"""
Right. We'll need a flag for this feature for starters. It doesn't exist :) Paolo also points out that we should then add code to disallow migration between setups with and without the feature.
"""
If we do it in the opposite way (i.e. use SMP barriers when feature is negotiated), the problem is that, old kernels will go into error silently when using hardware virtio devices.
Best regards,
Tiwei Bie
Quoted 8 lines
Paolo
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 10 lines · Paolo Bonzini
Quoted 8 lines · Tiwei Bie
There will be hardware virtio devices in the future, which require drivers to use the barriers suitable for I/O device, compared with software virtio devices which just require drivers to use the barriers suitable for CPU core.
To fix the ordering issue for hardware virtio devices, add a new feature:
VIRTIO_F_IO_BARRIER. When negotiated, driver will use the barriers suitable for I/O device.
Quoted 6 lines
Unfortunately this doesn't work as mentioned earlier. Virtio live migration assumes that features are safe if you have them on the destination but not on the source; this feature however works the opposite way.
Paolo
That's a qemu code property, and it's because it does not migrate host properties right now. When implementing this feature we can add migration of host features.
But that's only if we want to catch mis-use. We can also punt it up the stack and say feature flags must match on source and destination, and if they do not the problem is between the keyboard and the chair.
Signature · 5 lines
MST --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 8 lines · Michael S. Tsirkin
Quoted 4 lines
Unfortunately this doesn't work as mentioned earlier. Virtio live migration assumes that features are safe if you have them on the destination but not on the source; this feature however works the opposite way.
That's a qemu code property, and it's because it does not migrate host properties right now. When implementing this feature we can add migration of host features.
You don't even need to migrate host features; it's just that guest feature negotiation is repeated on the destination, and live migration reports an error if the negotiation fails. The guest feature negotiation is complicated a little bit by this feature, but I guess it can work...
Paolo
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Quoted 19 lines · Tiwei Bie
There will be hardware virtio devices in the future, which require drivers to use the barriers suitable for I/O device, compared with software virtio devices which just require drivers to use the barriers suitable for CPU core.
To fix the ordering issue for hardware virtio devices, add a new feature:
VIRTIO_F_IO_BARRIER. When negotiated, driver will use the barriers suitable for I/O device.Signed-off-by Tiwei Bie <[email protected]>--- content.tex | 9 +++++++++ 1 file changed, 9 insertions(+)content.tex +0
-0
content.tex Descriptors} and \ref{sec:Packed Virtqueues / Indirect Flag: Scatter-Gather Supp · line 5449
Quoted 5 lines
content.tex +2
-0@@ -5445,6 +5445,8 @@ Descriptors} and \ref{sec:Packed Virtqueues / Indirect Flag: Scatter-Gather Supp that drivers pass extra data (besides identifying the Virtqueue) in their device notifications. See \ref{sec:Virtqueues / Driver notifications}~\nameref{sec:Virtqueues / Driver notifications}. + \item[VIRTIO_F_IO_BARRIER(37)] This feature indicates that the device + needs the driver to use the barriers suitable for hardware device.
s/hardware device/hardware devices/ (plural)
content.tex addresses to the device. · line 5467
Quoted 10 lines
content.tex +3
-0\end{description} \drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}@@ -5460,6 +5462,10 @@ addresses to the device. A driver SHOULD accept VIRTIO_F_RING_PACKED if it is offered. +A driver SHOULD accept VIRTIO_F_IO_BARRIER if it is offered. +If VIRTIO_F_IO_BARRIER has been negotiated, a driver MUST use +the barriers suitable for hardware device.
s/hardware device/hardware devices/ (plural)
Please add a little context so the purpose is clear to readers (I wouldn't be 100% sure what the spec means without more detail):
Some transports require barriers to ensure hardware devices have a consistent view of memory. When devices are implemented in software a weaker form of barrier may be sufficient and yield better performance. This feature bit indicates whether the full hardware barrier is necessary.
Quoted 41 lines · Stefan Hajnoczi
Quoted 19 lines · Tiwei Bie
There will be hardware virtio devices in the future, which require drivers to use the barriers suitable for I/O device, compared with software virtio devices which just require drivers to use the barriers suitable for CPU core.
To fix the ordering issue for hardware virtio devices, add a new feature:
VIRTIO_F_IO_BARRIER. When negotiated, driver will use the barriers suitable for I/O device.Signed-off-by Tiwei Bie <[email protected]>--- content.tex | 9 +++++++++ 1 file changed, 9 insertions(+)content.tex +0
-0content.tex Descriptors} and \ref{sec:Packed Virtqueues / Indirect Flag: Scatter-Gather Supp · line 5449
Quoted 5 lines
content.tex +2
-0@@ -5445,6 +5445,8 @@ Descriptors} and \ref{sec:Packed Virtqueues / Indirect Flag: Scatter-Gather Supp that drivers pass extra data (besides identifying the Virtqueue) in their device notifications. See \ref{sec:Virtqueues / Driver notifications}~\nameref{sec:Virtqueues / Driver notifications}. + \item[VIRTIO_F_IO_BARRIER(37)] This feature indicates that the device + needs the driver to use the barriers suitable for hardware device.s/hardware device/hardware devices/ (plural)
content.tex addresses to the device. · line 5467
Quoted 10 lines
content.tex +3
-0\end{description} \drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}@@ -5460,6 +5462,10 @@ addresses to the device. A driver SHOULD accept VIRTIO_F_RING_PACKED if it is offered. +A driver SHOULD accept VIRTIO_F_IO_BARRIER if it is offered. +If VIRTIO_F_IO_BARRIER has been negotiated, a driver MUST use +the barriers suitable for hardware device.s/hardware device/hardware devices/ (plural)
Quoted 8 lines
Please add a little context so the purpose is clear to readers (I wouldn't be 100% sure what the spec means without more detail):
Some transports require barriers to ensure hardware devices have a consistent view of memory. When devices are implemented in software a weaker form of barrier may be sufficient and yield better performance. This feature bit indicates whether the full hardware barrier is necessary.
Thank you very much for the review and suggestion! :)
Best regards,
Tiwei Bie
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]