rework notifications terminology

Let us unify the wording when talking about notifications. This change establishes the terms available buffer notification for what was usually simply called notification or virtqueue notification in v1.0 and used buffer notification for what was usually called interrupt.

The term configuration change notification in kept where called so and consolidated where it's called configuration change interrupt or similar.

The changes done here are limited to the core part, and don't conceptually involve neither the transports nor the devices (references are updated though). Future changes should address these parts.

Quoted 9 lines · Halil Pasic

Let us unify the wording when talking about notifications. This change establishes the terms available buffer notification for what was usually simply called notification or virtqueue notification in v1.0 and used buffer notification for what was usually called interrupt.

The term configuration change notification in kept where called so and consolidated where it's called configuration change interrupt or similar.

Quoted 3 lines

The changes done here are limited to the core part, and don't conceptually involve neither the transports nor the devices (references are updated though). Future changes should address these parts.

Nice, I think the cleanup is worthwhile.

Review

Quoted 3 lines

\begin{lstlisting}

(quoted diff) +1 -1

-virtq_disable_interrupts(vq);
+virtq_disable_notifications(vq);

This name is ambiguous. Only used buffer notifications are disabled, not configuration change notifications.

How about:

  virtq_disable_used_buffer_notifications(vq);

Quoted 9 lines · Stefan Hajnoczi

Review

Quoted 3 lines · Halil Pasic

\begin{lstlisting}

(quoted diff) +1 -1

-virtq_disable_interrupts(vq);
+virtq_disable_notifications(vq);  

This name is ambiguous. Only used buffer notifications are disabled, not configuration change notifications.

One could argue that the only virtqueue scoped notification received by the driver are used buffer notifications. That's why I thought this would be sufficient (btw. if it's ambiguous it was ambiguous before too as both options are called interrupts).

But the more straight-forward the better.

Quoted 5 lines

How about:

  virtq_disable_used_buffer_notifications(vq);

In absence of further discussion I will take this for the next version.

Quoted 9 lines · Halil Pasic

Let us unify the wording when talking about notifications. This change establishes the terms available buffer notification for what was usually simply called notification or virtqueue notification in v1.0 and used buffer notification for what was usually called interrupt.

The term configuration change notification in kept where called so and consolidated where it's called configuration change interrupt or similar.

Quoted 3 lines

The changes done here are limited to the core part, and don't conceptually involve neither the transports nor the devices (references are updated though). Future changes should address these parts.

Nice, I think the cleanup is worthwhile.

Review

Quoted 3 lines

\begin{lstlisting}

(quoted diff) +1 -1

-virtq_disable_interrupts(vq);
+virtq_disable_notifications(vq);

This name is ambiguous. Only used buffer notifications are disabled, not configuration change notifications.

How about:

  virtq_disable_used_buffer_notifications(vq);

Quoted 14 lines · Stefan Hajnoczi

Quoted 12 lines · Halil Pasic

Let us unify the wording when talking about notifications. This change establishes the terms available buffer notification for what was usually simply called notification or virtqueue notification in v1.0 and used buffer notification for what was usually called interrupt.

The term configuration change notification in kept where called so and consolidated where it's called configuration change interrupt or similar.

The changes done here are limited to the core part, and don't conceptually involve neither the transports nor the devices (references are updated though). Future changes should address these parts.

Quoted 1 line

Nice, I think the cleanup is worthwhile.

I agree. I wondered if we should use the term "used buffer interrupt" and "available buffer notification". In the common case I think it would be clearer, though there are cases such as vhost-pci where the roles are swapped.

Quoted 5 lines

Quoted 1 line

Nice, I think the cleanup is worthwhile.

I agree. I wondered if we should use the term "used buffer interrupt" and "available buffer notification". In the common case I think it would be clearer, though there are cases such as vhost-pci where the roles are swapped.

If it is swapped in some cases, it is bound to cause confusion for those. I'd vote for using "notification" in both cases, as this patch is doing.

Let's introduce notifications as basic virtio facility early on. This shall not only increase the cohesion between core and transport description by having a well-defined place where notifications are introduced, but also give us the opportunity to explain some discrepancies.

Namely notifications send by the device to the driver were often called

interrupts prior v1.1. Getting completely rid of that terminology is

however not viable in case of some names.

Signed-off-by Halil Pasic <[email protected]>

---
 content.tex |   31 +++++++++++++++++++++++++++++++
 1 files changed, 31 insertions(+), 0 deletions(-)

content.tex +6 -0

index 4ccb823..87cc0e2 100644
@@ -8,6 +8,7 @@ device consists of the following parts:
 \begin{itemize}
 \item Device status field
 \item Feature bits
+\item Notifications
 \item Device Configuration space
 \item One or more virtqueues
 \end{itemize}

@@ -150,6 +151,36 @@ requirements documented within these legacy interface sections.
 Specification text within these sections generally does not apply
 to non-transitional devices.
 
+\section{Notifications}\label{sec:Basic Facilities of a Virtio Device
+/ Notifications}
+
+The notion of sending a notification (driver to device or device
+to driver) plays an important role in this specification. The

The various notifications are introduced and specified in the common (i.e. transport agnostic) portion of this specification. How notifications are realised for a given transport is something each transport has to specify.

Let's make the relationship between the virtio-ccw terms and the common terms more obvious.

Signed-off-by Halil Pasic <[email protected]>

---
 content.tex |   41 +++++++++++++++++++++++++++++++++++++++++
 1 files changed, 41 insertions(+), 0 deletions(-)

content.tex +0 -0

index 87cc0e2..27aa17b 100644

content.tex +5 -0

+\end{itemize} 
+
+\subsubsection{Channel Commands for Virtio}\label{sec:Virtio Transport Options / Virtio
+over channel I/O / Basic Concepts/ Channel Commands for Virtio}
+
 In addition to the basic channel commands, virtio-ccw defines a
 set of channel commands related to configuration and operation of
 virtio:

@@ -1958,6 +1969,36 @@ virtio:
 #define CCW_CMD_READ_STATUS 0x72
 \end{lstlisting}
 

content.tex +4 -0

+initialization is described in sections \ref{sec:Virtio Transport Options
+/ Virtio over channel I/O / Device Initialization / Setting Up Indicators
+/ Setting Up Classic Queue Indicators} and \ref{sec:Virtio Transport
+Options / Virtio over channel I/O / Device Initialization / Setting Up

On Wed, 11 Apr 2018 00:11:27 +0200
Halil Pasic <[email protected]> wrote:

[Have not yet looked at your other patches, on my list.]

Quoted 21 lines

The various notifications are introduced and specified in the common (i.e. transport agnostic) portion of this specification. How notifications are realised for a given transport is something each transport has to specify.

Let's make the relationship between the virtio-ccw terms and the common terms more obvious.

Signed-off-by Halil Pasic <[email protected]>

---
 content.tex |   41 +++++++++++++++++++++++++++++++++++++++++
 1 files changed, 41 insertions(+), 0 deletions(-)

content.tex +0 -0

index 87cc0e2..27aa17b 100644
@@ -1938,6 +1938,17 @@ Bytes & Description & Contents \\
 \hline
 \end{tabular}
 

content.tex Bytes & Description & Contents \\ · line 1944

Quoted 4 lines

content.tex +4 -0

+A virtio-ccw proxy device facilitates:
+\begin{itemize} 
+\item Discovery and attachment of  virtio devices (as described above).
+\item Initialization of vitqueues and transport specific facilities (using

s/vitqueues/virtqueues/

content.tex Bytes & Description & Contents \\ · line 1945

Quoted 1 line

content.tex +1 -0

+      custom channel commands).

s/custom/virtio-specific/ ?

In a sense, all channel commands other than the basic ones are 'custom' :) They are always device or control unit type specific, only obeying some rules.

content.tex Bytes & Description & Contents \\ · line 1946

Quoted 1 line

content.tex +1 -0

+\item Notifications (via hypercall and virtual interrupts).

Why 'virtual' interrupts? Better call this 'I/O interrupts' (includes both classic and adapter interrupts)?

Quoted 12 lines

content.tex +5 -0

+\end{itemize} 
+
+\subsubsection{Channel Commands for Virtio}\label{sec:Virtio Transport Options / Virtio
+over channel I/O / Basic Concepts/ Channel Commands for Virtio}
+
 In addition to the basic channel commands, virtio-ccw defines a
 set of channel commands related to configuration and operation of
 virtio:

@@ -1958,6 +1969,36 @@ virtio:
 #define CCW_CMD_READ_STATUS 0x72
 \end{lstlisting}
 

content.tex virtio: · line 1981

Quoted 10 lines

content.tex +10 -0

+\subsubsection{Notifications}\label{sec:Virtio Transport Options / Virtio
+over channel I/O / Basic Concepts/ Notifications}
+
+Available buffer notifications are realized as a hypercall. No additional
+setup by the driver is needed. The operation of available buffer
+notifications is described in section \ref{sec:Virtio Transport Options /
+Virtio over channel I/O / Device Operation / Guest->Host Notification}.
+
+Used buffer notifications are realized either as so called classic or as
+adapter interrupts depending on a transport level negotiation. The

"as so-called classic or adapter I/O interrupts"?

(I'd really like a reference to I/O interrupts here... especially as the old, never standardized s390 transport used external interrupts.)

Quoted 8 lines

content.tex +8 -0

+initialization is described in sections \ref{sec:Virtio Transport Options
+/ Virtio over channel I/O / Device Initialization / Setting Up Indicators
+/ Setting Up Classic Queue Indicators} and \ref{sec:Virtio Transport
+Options / Virtio over channel I/O / Device Initialization / Setting Up
+Indicators / Setting Up Two-Stage Queue Indicators} respectively.  The
+operation of each flavor is described in sections \ref{sec:Virtio
+Transport Options / Virtio over channel I/O / Device Operation /
+Host->Guest Notification / Notification via Classic I/O Interrupts} and

content.tex virtio: · line 1995

Quoted 6 lines

content.tex +6 -0

+\ref{sec:Virtio Transport Options / Virtio over channel I/O / Device
+Operation / Host->Guest Notification / Notification via Adapter I/O
+Interrupts} respectively. 
+
+Configuration change notifications are done using so called classic
+interrupts. The initialization is described in section \ref{sec:Virtio

"so-called classic I/O interrupts"

Quoted 8 lines

content.tex +6 -0

+Transport Options / Virtio over channel I/O / Device Initialization /
+Setting Up Indicators / Setting Up Configuration Change Indicators} and
+the operation in section \ref{sec:Virtio Transport Options / Virtio over
+channel I/O / Device Operation / Host->Guest Notification / Notification
+via Classic I/O Interrupts}.
+
 \devicenormative{\subsubsection}{Basic Concepts}{Virtio Transport Options / Virtio over channel I/O / Basic Concepts}
 

content.tex virtio: · line 2004

Quoted 1 line

content.tex +0 -0

 The virtio-ccw device acts like a normal channel device, as specified

In general, I like this.

Quoted 5 lines · Cornelia Huck

Quoted 1 line · On Wed, 11 Apr 2018 00:11:27 +0200 Halil Pasic <[email protected]> wrote:

+\item Notifications (via hypercall and virtual interrupts).

Quoted 3 lines

Why 'virtual' interrupts? Better call this 'I/O interrupts' (includes both classic and adapter interrupts)?

The idea was 'hypercall' and 'virtual' should harmonize well. These I/O interrupts are kind of 'real' from the perspective of the virtual machine, but are 'virtual' from the perspective of HW and AR perspective.

What I mean, there is AFAIU no way to implement a control unit and device combo in HW attach it to a z box and do virtio over CIO naively.

Even with classic I/O interrupts we have to do set indicator + inject subchannel interrupt to realize a notification. This is however form core perspective one notification/even/interrupt.

So this is why I added this 'virtual' (to hint it may not fit anything one can find in the PoP perfectly).

Quoted 13 lines

Quoted 13 lines

+\end{itemize}

content.tex +4 -0

+
+\subsubsection{Channel Commands for Virtio}\label{sec:Virtio Transport Options / Virtio
+over channel I/O / Basic Concepts/ Channel Commands for Virtio}
+
 In addition to the basic channel commands, virtio-ccw defines a
 set of channel commands related to configuration and operation of
 virtio:

@@ -1958,6 +1969,36 @@ virtio:
 #define CCW_CMD_READ_STATUS 0x72
 \end{lstlisting}
 

Quoted 12 lines

content.tex virtio: · line 1981

Quoted 10 lines

content.tex +10 -0

+\subsubsection{Notifications}\label{sec:Virtio Transport Options / Virtio
+over channel I/O / Basic Concepts/ Notifications}
+
+Available buffer notifications are realized as a hypercall. No additional
+setup by the driver is needed. The operation of available buffer
+notifications is described in section \ref{sec:Virtio Transport Options /
+Virtio over channel I/O / Device Operation / Guest->Host Notification}.
+
+Used buffer notifications are realized either as so called classic or as
+adapter interrupts depending on a transport level negotiation. The  

"as so-called classic or adapter I/O interrupts"?

Valid. These are indeed called 'adapter I/O interrupts' through out this spec. I was i a hurry to write up something demonstrating he idea, so I did not check this. I think these are usually called 'adapter interrupts' (without the I/O in between) elsewhere, but internal integrity is more important.

I will take it.

Quoted 1 line

Quoted 3 lines

(I'd really like a reference to I/O interrupts here... especially as the old, never standardized s390 transport used external interrupts.)

You mean with the wording you proposed, or something more? If something more could you do a patch on top (later)?

Quoted 10 lines · On Wed, 11 Apr 2018 15:42:34 +0200 Halil Pasic <[email protected]> wrote:

Quoted 8 lines · Cornelia Huck

Quoted 1 line · On Wed, 11 Apr 2018 00:11:27 +0200 Halil Pasic <[email protected]> wrote:

+\item Notifications (via hypercall and virtual interrupts).

Why 'virtual' interrupts? Better call this 'I/O interrupts' (includes both classic and adapter interrupts)?

Quoted 3 lines

The idea was 'hypercall' and 'virtual' should harmonize well. These I/O interrupts are kind of 'real' from the perspective of the virtual machine, but are 'virtual' from the perspective of HW and AR perspective.

Yes, but that's an implementation detail. I/O interrupts follow the same architecture in any case, there's nothing special about I/O interrupts for virtio.

Quoted 4 lines

What I mean, there is AFAIU no way to implement a control unit and device combo in HW attach it to a z box and do virtio over CIO naively.

It does not seem completely impossible (I/O interrupts are abstractions already). The diagnose notification might be a problem, though :)

Quoted 4 lines

Even with classic I/O interrupts we have to do set indicator + inject subchannel interrupt to realize a notification. This is however form core perspective one notification/even/interrupt.

But the I/O interrupt + indicators combination already exits (cf. QDIO). I don't think we should single out virtio.

Quoted 3 lines

So this is why I added this 'virtual' (to hint it may not fit anything one can find in the PoP perfectly).

I certainly would welcome addition of the adapter interrupt architecture to the PoP :)

Quoted 35 lines

Quoted 13 lines

Quoted 13 lines

+\end{itemize}

content.tex +4 -0

+
+\subsubsection{Channel Commands for Virtio}\label{sec:Virtio Transport Options / Virtio
+over channel I/O / Basic Concepts/ Channel Commands for Virtio}
+
 In addition to the basic channel commands, virtio-ccw defines a
 set of channel commands related to configuration and operation of
 virtio:

@@ -1958,6 +1969,36 @@ virtio:
 #define CCW_CMD_READ_STATUS 0x72
 \end{lstlisting}
 

Quoted 12 lines

content.tex virtio: · line 1981

Quoted 10 lines

content.tex +10 -0

+\subsubsection{Notifications}\label{sec:Virtio Transport Options / Virtio
+over channel I/O / Basic Concepts/ Notifications}
+
+Available buffer notifications are realized as a hypercall. No additional
+setup by the driver is needed. The operation of available buffer
+notifications is described in section \ref{sec:Virtio Transport Options /
+Virtio over channel I/O / Device Operation / Guest->Host Notification}.
+
+Used buffer notifications are realized either as so called classic or as
+adapter interrupts depending on a transport level negotiation. The  

"as so-called classic or adapter I/O interrupts"?

Valid. These are indeed called 'adapter I/O interrupts' through out this spec. I was i a hurry to write up something demonstrating he idea, so I did not check this. I think these are usually called 'adapter interrupts' (without the I/O in between) elsewhere, but internal integrity is more important.

I will take it.

Quoted 1 line

Quoted 6 lines

Quoted 3 lines

(I'd really like a reference to I/O interrupts here... especially as the old, never standardized s390 transport used external interrupts.)

You mean with the wording you proposed, or something more? If something more could you do a patch on top (later)?

I think simply adding "I/O" should be enough.

Quoted 17 lines · Cornelia Huck

Quoted 13 lines · On Wed, 11 Apr 2018 15:42:34 +0200 Halil Pasic <[email protected]> wrote:

Quoted 8 lines · Cornelia Huck

Quoted 1 line · On Wed, 11 Apr 2018 00:11:27 +0200 Halil Pasic <[email protected]> wrote:

+\item Notifications (via hypercall and virtual interrupts).

Why 'virtual' interrupts? Better call this 'I/O interrupts' (includes both classic and adapter interrupts)?

The idea was 'hypercall' and 'virtual' should harmonize well. These I/O interrupts are kind of 'real' from the perspective of the virtual machine, but are 'virtual' from the perspective of HW and AR perspective.

Quoted 4 lines

Yes, but that's an implementation detail. I/O interrupts follow the same architecture in any case, there's nothing special about I/O interrupts for virtio.

I'm not sure about that. For instance, there is this check your notifiers unsolicited subchannel associated I/O interruption. I don't think this is just plain old channel I/O interrupt handling.

But you seem bothered by this 'virtual' and I respect that. So how about 'Notifications (via hypercall and a combination of I/O interrupts and indicator bits).'?

While discussing XXXX using available and used buffer notifications

consistently for virqueue notifications (driver to device and vice versa respectively) seemed like a good idea.

It turned out surprisingly invasive however, and I find it necessary to confirm, this is really the path we want to walk, before investing even more time.

This series limits itself to the core and to the ccw transport. It ain't typical to the device types to make statements about notifications, but at least net has some words on interrupts. I did not get these immediately so I've left that out for now.

I choose ccw as demonstrator on how to bridge the abstract with the transport specific, because that is the transport I'm most familiar with. I do not expect difficulties with the other ones.

Halil Pasic (3):
notifications: unify notifications wording in core notifications:notifications as basic virtio facility
ccw: map common notifications terminology to ccw

  cl-os.tex       |    2 +-
  conformance.tex |    8 ++--
  content.tex     |   92 ++++++++++++++++++++++++++++++++++++++++++++++++++-----
  packed-ring.tex |   59 +++++++++++++++++++---------------
  split-ring.tex  |   72 +++++++++++++++++++++++++------------------
  5 files changed, 164 insertions(+), 69 deletions(-)

ping

@Michael: Any feedback from you or should I prepare a non-rfc that deals with the other transports too.

Quoted 1 line

While discussing XXXX using available and used buffer notifications

s;XXXX;'https://www.mail-archive.com/[email protected]/msg02774.html

Quoted 29 lines

consistently for virqueue notifications (driver to device and vice versa respectively) seemed like a good idea.

It turned out surprisingly invasive however, and I find it necessary to confirm, this is really the path we want to walk, before investing even more time.

This series limits itself to the core and to the ccw transport. It ain't typical to the device types to make statements about notifications, but at least net has some words on interrupts. I did not get these immediately so I've left that out for now.

I choose ccw as demonstrator on how to bridge the abstract with the transport specific, because that is the transport I'm most familiar with. I do not expect difficulties with the other ones.

Halil Pasic (3):
notifications: unify notifications wording in core notifications:notifications as basic virtio facility
ccw: map common notifications terminology to ccw

  cl-os.tex       |    2 +-
  conformance.tex |    8 ++--
  content.tex     |   92 ++++++++++++++++++++++++++++++++++++++++++++++++++-----
  packed-ring.tex |   59 +++++++++++++++++++---------------
  split-ring.tex  |   72 +++++++++++++++++++++++++------------------
  5 files changed, 164 insertions(+), 69 deletions(-)