Skip to contentSkip to conversation

acknowledgements: add members and non-members

PATCH In review Michael S. Tsirkin 3 participants 4 messages Mar 27, 2019 – Mar 29, 2019 1 message not in the archive Arm/actions semi architecture

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 acknowledgements: add members and non-members

Add all current members as participants.
Add all participant names collected from list, jira and github.

---

Fixes from v1:
add two reporters from mailing list (mellanox)

 acknowledgements.tex | 72 ++++++++++++++++++++++++++++++++++++++------
 1 file changed, 63 insertions(+), 9 deletions(-)
acknowledgements.tex +63 -9
index 233e4a1..b2feeaf 100644
@@ -6,47 +6,101 @@ \chapter{Acknowledgements}\label{chap:Acknowledgements}
 The following individuals have participated in the creation of this specification and are gratefully acknowledged:
 
 \begin{oasistitlesection}{Participants}
+Allen Chia, Oracle	\newline
 Amit Shah,	Red Hat	\newline
 Amos Kong,	Red Hat	\newline
 Anthony Liguori,	IBM	\newline
-Bruce Rogers,	Novell	\newline
+Bruce Rogers, SUSE	\newline
 Bryan Venteicher,	NetApp	\newline
+Chandra Thyamagondlu, Xilinx	\newline
+Chet Ensign, OASIS	\newline
 Cornelia Huck,	Red Hat	\newline
+Cunming, Liang, Intel	\newline
+Damjan, Marion, Cisco	\newline
 Daniel Kiper,	Oracle	\newline
+Fang Chen, Huawei	\newline
+Fang You, Huawei	\newline
 Geoff Brown,	Machine-to-Machine Intelligence (M2MI) Corporation	\newline
+Gerd Hoffmann, Red Hat	\newline
 Gershon Janssen,	Individual Member	\newline
+Grant Likely, ARM	\newline
+Haggai Eran,	Mellanox	\newline
 Halil Pasic,	IBM	\newline
 James Bottomley,	Parallels IP Holdings GmbH	\newline
+Jani Kokkonen, Huawei	\newline
+Jan Kiszka,	Siemens AG	\newline
+Jens Freimann, Red Hat	\newline
 Jian Zhou,	Huawei	\newline
+Karen Xie, Xilinx	\newline
+Kumar Sanghvi, Xilinx	\newline
 Lei Gong,	Huawei	\newline
+Lior Narkis,	Mellanox	\newline
 Luiz Capitulino,	Red Hat	\newline
+Marc-André Lureau, Red Hat	\newline
+Mark Gray, Intel	\newline
 Michael S. Tsirkin,	Red Hat	\newline
+Mihai Carabas,	Oracle	\newline
+Nishank Trivedi, NetApp	\newline
 Paolo Bonzini,	Red Hat	\newline
-Pawel Moll,	ARM \newline
+Paul Mundt, Huawei	\newline
+Pawel Moll,	ARM	\newline
 Peng Long,	Huawei	\newline
-Richard Sohn,	Alcatel-Lucent \newline
+Piotr Uminski, Intel	\newline
+Qian Xum, Intel	\newline
+Richard Sohn,	Alcatel-Lucent	\newline
 Rusty Russell,	IBM	\newline
 Sasha Levin,	Oracle	\newline
 Sergey Tverdyshev,	Thales e-Security	\newline
 Stefan Hajnoczi,	Red Hat	\newline
+Sundar Mohan, Xilinx	\newline
 Tom Lyon,	Samya Systems, Inc.	\newline
+Victor Kaplansky, Red Hat	\newline
+Vijay Balakrishna,	Oracle	\newline
+Wei Wang, Intel	\newline
+Xin Zeng, Intel	\newline
 \end{oasistitlesection}
 
 The following non-members have provided valuable feedback on this
 specification and are gratefully acknowledged:
 
 \begin{oasistitlesection}{Reviewers}
+Aaron Conole,	Red Hat	\newline
 Andrew Thornton,  Google \newline
 Arun Subbarao,	LynuxWorks	\newline
+Baptiste Reynal,	Virtual Open Systems	\newline
 Brian Foley,  ARM \newline
+Changpeng Liu,	Intel	\newline
+Christian Pinto,	Virtual Open Systems	\newline
+Christoffer Dall,	ARM	\newline
+Christian Borntraeger,	IBM	\newline
+Daniel Marcovitch,	Mellanox	\newline
 David Alan Gilbert, Red Hat \newline
+David Hildenbrand,	Red Hat	\newline
 Fam Zheng, Red Hat	\newline
-Gerd Hoffmann, Red Hat	\newline
-Halil Pasic,	IBM	\newline
+Gil Savir,	Intel	\newline
+Gonglei (Arei),	Huawei	\newline
+Hannes Reiencke, SUSE	\newline
+Jacques Durand,	Fujutsu	\newline
 Jason Wang, Red Hat \newline
-Laura Novich, Red Hat	\newline
-Patrick Durusau,	Technical Advisory Board, OASIS	\newline
-Thomas Huth,	Red Hat	\newline
-Yan Vugenfirer, Red Hat / Daynix	\newline
+Jean-Philippe Brucker,	ARM	\newline
+Jonathan Helman,	Oracle	\newline
 Kevin Lo,	MSI	\newline
+Laura Novich, Red Hat	\newline
+Ladi Prosek,	Red Hat	\newline
+Longpeng (Mike),	Huawei	\newline
+Maxime Coquelin,	Red Hat	\newline
+Namhyung Kim,	LG	\newline
+Patrick Durusau,	Technical Advisory Board, OASIS	\newline
+Robin Cover,	Technical Advisory Board, OASIS	\newline
+Sameeh Jubran,	Red Hat / Daynix	\newline
+Sridhar Samudrala,	Intel	\newline
+Stefano Garzarella,	Red Hat	\newline
+Thomas Huth,	Red Hat	\newline
+Tiwei Bie,	Intel	\newline
+Tomáš Golembiovský,	Red Hat	\newline
+Victor Kaplansky,	Red Hat	\newline
+Yan Vugenfirer, Red Hat / Daynix	\newline
+Will Deacon,	ARM	\newline
+Yuri Benditovich,	Red Hat / Daynix	\newline
+Zhoujian,	Huawei	\newline
 \end{oasistitlesection}
Signature · 6 lines
MST

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that 
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php 
Reviewed-by on v2 1/1
#1 lore raw
Quoted 11 lines · Michael S. Tsirkin

Add all current members as participants.
Add all participant names collected from list, jira and github.

---

Fixes from v1:
add two reporters from mailing list (mellanox)

 acknowledgements.tex | 72 ++++++++++++++++++++++++++++++++++++++------
 1 file changed, 63 insertions(+), 9 deletions(-)

I haven't checked the spelling of names or whether the companies are correct. The overall formatting looks good though.

A message that is not in the archive (sent off-list or not yet received): [email protected]

#3 lore raw

Sure, will do. And which affiliation should I list? Napatech?

Quoted 300 lines · Lars Ganrot

Hi Michael,

(Sending this to your account, not the list, since it isn't of general interest).

I have no knowledge of the community's threshold for acknowledging non-members, but would like to ensure my contribution regarding VIRTIO_F_IN_ORDER (attached) is not left out by accident.

If too minor for inclusion, I gracefully accept that decision.

Best Regards,

-Lars

-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of Michael S. Tsirkin
Sent: 27. marts 2019 23:33
To: [email protected]; [email protected]
Subject: [virtio-dev] [PATCH v2] acknowledgements: add members and non-members

Add all current members as participants.
Add all participant names collected from list, jira and github.

---

Fixes from v1:
add two reporters from mailing list (mellanox)

 acknowledgements.tex | 72 ++++++++++++++++++++++++++++++++++++++------
 1 file changed, 63 insertions(+), 9 deletions(-)
acknowledgements.tex +63 -10
@@ -6,47 +6,101 @@ \chapter{Acknowledgements}\label{chap:Acknowledgements}
 The following individuals have participated in the creation of this specification and are gratefully acknowledged:
 
 \begin{oasistitlesection}{Participants}
+Allen Chia, Oracle\newline
 Amit Shah,Red Hat\newline
 Amos Kong,Red Hat\newline
 Anthony Liguori,IBM\newline
-Bruce Rogers,Novell\newline
+Bruce Rogers, SUSE\newline
 Bryan Venteicher,NetApp\newline
+Chandra Thyamagondlu, Xilinx\newline
+Chet Ensign, OASIS\newline
 Cornelia Huck,Red Hat\newline
+Cunming, Liang, Intel\newline
+Damjan, Marion, Cisco\newline
 Daniel Kiper,Oracle\newline
+Fang Chen, Huawei\newline
+Fang You, Huawei\newline
 Geoff Brown,Machine-to-Machine Intelligence (M2MI) Corporation\newline
+Gerd Hoffmann, Red Hat\newline
 Gershon Janssen,Individual Member\newline
+Grant Likely, ARM\newline
+Haggai Eran,Mellanox\newline
 Halil Pasic,IBM\newline
 James Bottomley,Parallels IP Holdings GmbH\newline
+Jani Kokkonen, Huawei\newline
+Jan Kiszka,Siemens AG\newline
+Jens Freimann, Red Hat\newline
 Jian Zhou,Huawei\newline
+Karen Xie, Xilinx\newline
+Kumar Sanghvi, Xilinx\newline
 Lei Gong,Huawei\newline
+Lior Narkis,Mellanox\newline
 Luiz Capitulino,Red Hat\newline
+Marc-André Lureau, Red Hat\newline
+Mark Gray, Intel\newline
 Michael S. Tsirkin,Red Hat\newline
+Mihai Carabas,Oracle\newline
+Nishank Trivedi, NetApp\newline
 Paolo Bonzini,Red Hat\newline
-Pawel Moll,ARM \newline
+Paul Mundt, Huawei\newline
+Pawel Moll,ARM\newline
 Peng Long,Huawei\newline
-Richard Sohn,Alcatel-Lucent \newline
+Piotr Uminski, Intel\newline
+Qian Xum, Intel\newline
+Richard Sohn,Alcatel-Lucent\newline
 Rusty Russell,IBM\newline
 Sasha Levin,Oracle\newline
 Sergey Tverdyshev,Thales e-Security\newline
 Stefan Hajnoczi,Red Hat\newline
+Sundar Mohan, Xilinx\newline
 Tom Lyon,Samya Systems, Inc.\newline
+Victor Kaplansky, Red Hat\newline
+Vijay Balakrishna,Oracle\newline
+Wei Wang, Intel\newline
+Xin Zeng, Intel\newline
 \end{oasistitlesection}
 
 The following non-members have provided valuable feedback on this  specification and are gratefully acknowledged:
 
 \begin{oasistitlesection}{Reviewers}
+Aaron Conole,Red Hat\newline
 Andrew Thornton,  Google \newline
 Arun Subbarao,LynuxWorks\newline
+Baptiste Reynal,Virtual Open Systems\newline
 Brian Foley,  ARM \newline
+Changpeng Liu,Intel\newline
+Christian Pinto,Virtual Open Systems\newline
+Christoffer Dall,ARM\newline
+Christian Borntraeger,IBM\newline
+Daniel Marcovitch,Mellanox\newline
 David Alan Gilbert, Red Hat \newline
+David Hildenbrand,Red Hat\newline
 Fam Zheng, Red Hat\newline
-Gerd Hoffmann, Red Hat\newline
-Halil Pasic,IBM\newline
+Gil Savir,Intel\newline
+Gonglei (Arei),Huawei\newline
+Hannes Reiencke, SUSE\newline
+Jacques Durand,Fujutsu\newline
 Jason Wang, Red Hat \newline
-Laura Novich, Red Hat\newline
-Patrick Durusau,Technical Advisory Board, OASIS\newline
-Thomas Huth,Red Hat\newline
-Yan Vugenfirer, Red Hat / Daynix\newline
+Jean-Philippe Brucker,ARM\newline
+Jonathan Helman,Oracle\newline
 Kevin Lo,MSI\newline
+Laura Novich, Red Hat\newline
+Ladi Prosek,Red Hat\newline
+Longpeng (Mike),Huawei\newline
+Maxime Coquelin,Red Hat\newline
+Namhyung Kim,LG\newline
+Patrick Durusau,Technical Advisory Board, OASIS\newline
+Robin Cover,Technical Advisory Board, OASIS\newline
+Sameeh Jubran,Red Hat / Daynix\newline
+Sridhar Samudrala,Intel\newline
+Stefano Garzarella,Red Hat\newline
+Thomas Huth,Red Hat\newline
+Tiwei Bie,Intel\newline
+Tomáš Golembiovský,Red Hat\newline
+Victor Kaplansky,Red Hat\newline
+Yan Vugenfirer, Red Hat / Daynix\newline
+Will Deacon,ARM\newline
+Yuri Benditovich,Red Hat / Daynix\newline
+Zhoujian,Huawei\newline
 \end{oasistitlesection}
--

MST

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

Disclaimer: This email and any files transmitted with it may contain confidential information intended for the addressee(s) only. The information is not to be surrendered or copied to unauthorized persons. If you have received this communication in error, please notify the sender immediately and delete this e-mail from your system.

Date: Fri, 20 Oct 2017 09:50:03 +0000
From: Lars Ganrot <[email protected]>
To: "[email protected]" <[email protected]>
Subject: Re: [virtio-dev] packed ring layout proposal v3
Message-ID: <[email protected]>

Hi Michael,

I'm trying to understand your Sep 28 13:32 example (text copied below). I've in-lined my questions [#lga#].

Quoted 11 lines

Let's assume device promised to consume packets in order

ring size = 2

Ring is 0 initialized.

Device initially polls DESC[0].flags for WRAP bit to change.

driver adds:

Quoted 7 lines

DESC[0].addr = 1234
DESC[0].id = 0
DESC[0].flags = DESC_DRIVER | DESC_MORE | DESC_WRAP

and

DESC[0].addr = 5678

[#lga#] Is index 0 above a typo (makes more sense if it is 1)?

Quoted 11 lines

DESC[1].id = 1
DESC[1].flags = DESC_DRIVER | DESC_WRAP

it now starts polling DESC[0] flags.

Device reads 1234, executes it, does not use it.

Device reads 5678, executes it, and uses it:

DESC[0].id = 1
DESC[0].flags = 0

[#lga#] Should above line be: "DESC[0].flags = DESC[1].flags & DESC_WRAP"?

Quoted 5 lines

Device now polls DESC[0].flags for WRAP bit to change.

Now driver sees that DRIVER bit has been cleared, so it nows that id is valid. I sees id 1, therefore id 0 and 1 has been read and are safe to overwrite.

[#lga#] At this point, has the device returned both buffers *(1234) and *(5678) to the driver? [#lga#] If so, how would out-of-order completion avoid head of line blocking? [#lga#] E.g. the device is done with id=1 and *(5678), but not id=0 and *(1234).

Quoted 11 lines

So it writes it out. It wrapped around to beginning of ring, so it flips the WRAP bit to 0 on all descriptors now:

DESC[0].addr = 9ABC
DESC[0].id = 0
DESC[0].flags = DESC_DRIVER | DESC_MORE

DESC[0].addr = DEF0
DESC[0].id = 1
DESC[0].flags = DESC_DRIVER

[#lga#] Index typo in all 3 lines above? DESC[1] makes more sense.

Quoted 8 lines

Next round wrap will be 1 again.

To summarise:

DRIVER bit is used by driver to detect device has used one or more descriptors. WRAP is is used by device to detect driver has made a new descriptor available.

Best Regards,

-Lars

Date: Wed, 29 Nov 2017 08:12:23 +0000
From: Lars Ganrot <[email protected]>
To: "Michael S. Tsirkin" <[email protected]>
CC: "[email protected]" <[email protected]>
Subject: RE: [virtio-dev] [PATCH RFC] packed ring layout spec v5
Message-ID: <[email protected]>
In-Reply-To: <[email protected]>

Quoted 9 lines

From: Michael S. Tsirkin [mailto:[email protected]]
Sent: 28. november 2017 15:37

Quoted 5 lines

If the device always returns buffers in order, then couldn't the driver skip the step of reading the used.ring for read-only buffers (e.g. TX for net devices)? The used.idx tells how many buffers were returned, and since they are returned in the same order as the driver sent them, it knows what their indices are. This would then save one cache-miss in the old structure too.

Quoted 4 lines

True. That would be another variant to support though.

I doubt it'll outperform this one but I didn't test it specifically. Care trying to implement it?

Agreed, and I don't see it as competing with the packed ring, however if there are low hanging fruits, that improve performance of the non-ring structure (in at least some significant use cases) they could be worth considering as part of a rev 1.1 specification too. I'll see what can be done for a prototype.

Quoted 22 lines
Quoted 6 lines

And a follow-up questions would then be: if a device always returns buffers in order, does the v1.0 specification not require drivers to reuse descriptors in the same order as they are returned? I think 3.2.1.1 implies that at least. If so, wouldn't new descriptors always be placed back2back in the descriptor table (contiguous memory)?

You probably mean this:
1. Get the next free descriptor table entry, d

and you interpret "next" here as "next in ring order".

I'm not sure everyone follows this interpretation though.

E.g. Linux does:
static void detach_buf(struct vring_virtqueue *vq, unsigned int head,

                       void **ctx)
{
...
        vq->vring.desc[i].next = cpu_to_virtio16(vq->vq.vdev, vq->free_head);
        vq->free_head = head;
Quoted 4 lines

So descriptors are added at head of the free list. Next is interpreted as next on this list. E.g. with a single request in flight, it looks like a single descriptor will keep getting reused.

I guess there isn't an explicit enough requirement in v1.0 to claim right or wrong with regards to this. Enforcing it could however be made part of a driver requirement imposed by the new IN_ORDER feature bit. Thus the IN_ORDER feature bit for the non-ring would be defined to enforce that the descriptor indices are always processed in-order by both the device and the driver.

My reason for this is to ensure that new descriptors are placed in a contiguous range of the descriptor table, which should improve the L1$ prefetcher hit rate for batching, and also provide means for efficient DMA in case of HW-offload. With knowledge of the number of elements in each buffer it could maybe also be possible to calculate the descriptor index range to DMA.

Quoted 3 lines

BR,

-Lars

---------------------------------------------------------------------
To unsubscribe from this mail list, you must leave the OASIS TC that generates this mail. Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php

#4 lore raw

Correct.

Thank you, Sir!

Best Regards,

-Lars

-----Original Message-----
From: Michael S. Tsirkin <[email protected]>
Sent: 28. marts 2019 18:36
To: Lars Ganrot <[email protected]>
Cc: [email protected]; [email protected]
Subject: Re: [virtio-dev] [PATCH v2] acknowledgements: add members and non-members

Sure, will do. And which affiliation should I list? Napatech?

Quoted 303 lines · Lars Ganrot

Hi Michael,

(Sending this to your account, not the list, since it isn't of general interest).

I have no knowledge of the community's threshold for acknowledging non-members, but would like to ensure my contribution regarding VIRTIO_F_IN_ORDER (attached) is not left out by accident.

If too minor for inclusion, I gracefully accept that decision.

Best Regards,

-Lars

-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of Michael S. Tsirkin
Sent: 27. marts 2019 23:33
To: [email protected]; [email protected]
Subject: [virtio-dev] [PATCH v2] acknowledgements: add members and non-members

Add all current members as participants.
Add all participant names collected from list, jira and github.

---

Fixes from v1:
add two reporters from mailing list (mellanox)

 acknowledgements.tex | 72 
++++++++++++++++++++++++++++++++++++++------
 1 file changed, 63 insertions(+), 9 deletions(-)
acknowledgements.tex index renamed from acknowledgements.tex +0 -0

233e4a1..b2feeaf 100644

acknowledgements.tex +8 -1
@@ -6,47 +6,101 @@ 
 \chapter{Acknowledgements}\label{chap:Acknowledgements}
 The following individuals have participated in the creation of this specification and are gratefully acknowledged:
 
 \begin{oasistitlesection}{Participants}
+Allen Chia, Oracle\newline
 Amit Shah,Red Hat\newline
 Amos Kong,Red Hat\newline
 Anthony Liguori,IBM\newline
-Bruce Rogers,Novell\newline
+Bruce Rogers, SUSE\newline
 Bryan Venteicher,NetApp\newline
+Chandra Thyamagondlu, Xilinx\newline
+Chet Ensign, OASIS\newline
 Cornelia Huck,Red Hat\newline
+Cunming, Liang, Intel\newline
+Damjan, Marion, Cisco\newline
 Daniel Kiper,Oracle\newline
+Fang Chen, Huawei\newline
+Fang You, Huawei\newline
 Geoff Brown,Machine-to-Machine Intelligence (M2MI) 

Corporation\newline

acknowledgements.tex +18 -2
+Gerd Hoffmann, Red Hat\newline
 Gershon Janssen,Individual Member\newline
+Grant Likely, ARM\newline
+Haggai Eran,Mellanox\newline
 Halil Pasic,IBM\newline
 James Bottomley,Parallels IP Holdings GmbH\newline
+Jani Kokkonen, Huawei\newline
+Jan Kiszka,Siemens AG\newline
+Jens Freimann, Red Hat\newline
 Jian Zhou,Huawei\newline
+Karen Xie, Xilinx\newline
+Kumar Sanghvi, Xilinx\newline
 Lei Gong,Huawei\newline
+Lior Narkis,Mellanox\newline
 Luiz Capitulino,Red Hat\newline
+Marc-André Lureau, Red Hat\newline
+Mark Gray, Intel\newline
 Michael S. Tsirkin,Red Hat\newline
+Mihai Carabas,Oracle\newline
+Nishank Trivedi, NetApp\newline
 Paolo Bonzini,Red Hat\newline
-Pawel Moll,ARM \newline
+Paul Mundt, Huawei\newline
+Pawel Moll,ARM\newline
 Peng Long,Huawei\newline
-Richard Sohn,Alcatel-Lucent \newline
+Piotr Uminski, Intel\newline
+Qian Xum, Intel\newline
+Richard Sohn,Alcatel-Lucent\newline
 Rusty Russell,IBM\newline
 Sasha Levin,Oracle\newline
 Sergey Tverdyshev,Thales e-Security\newline  Stefan Hajnoczi,Red 

Hat\newline

acknowledgements.tex +16 -4
+Sundar Mohan, Xilinx\newline
 Tom Lyon,Samya Systems, Inc.\newline
+Victor Kaplansky, Red Hat\newline
+Vijay Balakrishna,Oracle\newline
+Wei Wang, Intel\newline
+Xin Zeng, Intel\newline
 \end{oasistitlesection}
 
 The following non-members have provided valuable feedback on this  specification and are gratefully acknowledged:
 
 \begin{oasistitlesection}{Reviewers}
+Aaron Conole,Red Hat\newline
 Andrew Thornton,  Google \newline
 Arun Subbarao,LynuxWorks\newline
+Baptiste Reynal,Virtual Open Systems\newline
 Brian Foley,  ARM \newline
+Changpeng Liu,Intel\newline
+Christian Pinto,Virtual Open Systems\newline Christoffer 
+Dall,ARM\newline Christian Borntraeger,IBM\newline Daniel 
+Marcovitch,Mellanox\newline
 David Alan Gilbert, Red Hat \newline
+David Hildenbrand,Red Hat\newline
 Fam Zheng, Red Hat\newline
-Gerd Hoffmann, Red Hat\newline
-Halil Pasic,IBM\newline
+Gil Savir,Intel\newline
+Gonglei (Arei),Huawei\newline
+Hannes Reiencke, SUSE\newline
+Jacques Durand,Fujutsu\newline
 Jason Wang, Red Hat \newline
-Laura Novich, Red Hat\newline
-Patrick Durusau,Technical Advisory Board, OASIS\newline -Thomas 

Huth,Red Hat\newline -Yan Vugenfirer, Red Hat / Daynix\newline

acknowledgements.tex +14 -1
+Jean-Philippe Brucker,ARM\newline
+Jonathan Helman,Oracle\newline
 Kevin Lo,MSI\newline
+Laura Novich, Red Hat\newline
+Ladi Prosek,Red Hat\newline
+Longpeng (Mike),Huawei\newline
+Maxime Coquelin,Red Hat\newline
+Namhyung Kim,LG\newline
+Patrick Durusau,Technical Advisory Board, OASIS\newline Robin 
+Cover,Technical Advisory Board, OASIS\newline Sameeh Jubran,Red Hat / 
+Daynix\newline Sridhar Samudrala,Intel\newline Stefano Garzarella,Red 
+Hat\newline Thomas Huth,Red Hat\newline Tiwei Bie,Intel\newline Tomáš 
+Golembiovský,Red Hat\newline Victor Kaplansky,Red Hat\newline Yan 
+Vugenfirer, Red Hat / Daynix\newline Will Deacon,ARM\newline Yuri 
+Benditovich,Red Hat / Daynix\newline Zhoujian,Huawei\newline
 \end{oasistitlesection}
--

MST

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]

Disclaimer: This email and any files transmitted with it may contain confidential information intended for the addressee(s) only. The information is not to be surrendered or copied to unauthorized persons. If you have received this communication in error, please notify the sender immediately and delete this e-mail from your system.

Date: Fri, 20 Oct 2017 09:50:03 +0000
From: Lars Ganrot <[email protected]>
To: "[email protected]" <[email protected]>
Subject: Re: [virtio-dev] packed ring layout proposal v3
Message-ID: <[email protected]>

Hi Michael,

I'm trying to understand your Sep 28 13:32 example (text copied below). I've in-lined my questions [#lga#].

Quoted 11 lines

Let's assume device promised to consume packets in order

ring size = 2

Ring is 0 initialized.

Device initially polls DESC[0].flags for WRAP bit to change.

driver adds:

Quoted 7 lines

DESC[0].addr = 1234
DESC[0].id = 0
DESC[0].flags = DESC_DRIVER | DESC_MORE | DESC_WRAP

and

DESC[0].addr = 5678

[#lga#] Is index 0 above a typo (makes more sense if it is 1)?

Quoted 11 lines

DESC[1].id = 1
DESC[1].flags = DESC_DRIVER | DESC_WRAP

it now starts polling DESC[0] flags.

Device reads 1234, executes it, does not use it.

Device reads 5678, executes it, and uses it:

DESC[0].id = 1
DESC[0].flags = 0

[#lga#] Should above line be: "DESC[0].flags = DESC[1].flags & DESC_WRAP"?

Quoted 5 lines

Device now polls DESC[0].flags for WRAP bit to change.

Now driver sees that DRIVER bit has been cleared, so it nows that id is valid. I sees id 1, therefore id 0 and 1 has been read and are safe to overwrite.

[#lga#] At this point, has the device returned both buffers *(1234) and *(5678) to the driver? [#lga#] If so, how would out-of-order completion avoid head of line blocking? [#lga#] E.g. the device is done with id=1 and *(5678), but not id=0 and *(1234).

Quoted 11 lines

So it writes it out. It wrapped around to beginning of ring, so it flips the WRAP bit to 0 on all descriptors now:

DESC[0].addr = 9ABC
DESC[0].id = 0
DESC[0].flags = DESC_DRIVER | DESC_MORE

DESC[0].addr = DEF0
DESC[0].id = 1
DESC[0].flags = DESC_DRIVER

[#lga#] Index typo in all 3 lines above? DESC[1] makes more sense.

Quoted 8 lines

Next round wrap will be 1 again.

To summarise:

DRIVER bit is used by driver to detect device has used one or more descriptors. WRAP is is used by device to detect driver has made a new descriptor available.

Best Regards,

-Lars

Date: Wed, 29 Nov 2017 08:12:23 +0000
From: Lars Ganrot <[email protected]>
To: "Michael S. Tsirkin" <[email protected]>
CC: "[email protected]" <[email protected]>
Subject: RE: [virtio-dev] [PATCH RFC] packed ring layout spec v5
Message-ID: <[email protected]>
In-Reply-To: <[email protected]>

Quoted 9 lines

From: Michael S. Tsirkin [mailto:[email protected]]
Sent: 28. november 2017 15:37

Quoted 5 lines

If the device always returns buffers in order, then couldn't the
driver skip the step of reading the used.ring for read-only buffers
(e.g. TX for net devices)? The used.idx tells how many buffers were
returned, and since they are returned in the same order as the
driver sent them, it knows what their indices are. This would then save one cache-miss in the old structure too.

Quoted 4 lines

True. That would be another variant to support though.

I doubt it'll outperform this one but I didn't test it specifically. Care trying to implement it?

Agreed, and I don't see it as competing with the packed ring, however
if there are low hanging fruits, that improve performance of the
non-ring structure (in at least some significant use cases) they could
be worth considering as part of a rev 1.1 specification too. I'll see what can be done for a prototype.

Quoted 22 lines
Quoted 6 lines

And a follow-up questions would then be: if a device always
returns buffers in order, does the v1.0 specification not require
drivers to reuse descriptors in the same order as they are
returned? I think 3.2.1.1 implies that at least. If so, wouldn't
new descriptors always be placed back2back in the descriptor table (contiguous memory)?

You probably mean this:
1. Get the next free descriptor table entry, d

and you interpret "next" here as "next in ring order".

I'm not sure everyone follows this interpretation though.

E.g. Linux does:
static void detach_buf(struct vring_virtqueue *vq, unsigned int head,

                       void **ctx)
{
...
        vq->vring.desc[i].next = cpu_to_virtio16(vq->vq.vdev, vq->free_head);
        vq->free_head = head;
Quoted 4 lines

So descriptors are added at head of the free list. Next is interpreted as next on this list. E.g. with a single request in flight, it looks like a single descriptor will keep getting reused.

I guess there isn't an explicit enough requirement in v1.0 to claim
right or wrong with regards to this. Enforcing it could however be
made part of a driver requirement imposed by the new IN_ORDER feature
bit. Thus the IN_ORDER feature bit for the non-ring would be defined
to enforce that the descriptor indices are always processed in-order by both the device and the driver.

My reason for this is to ensure that new descriptors are placed in a
contiguous range of the descriptor table, which should improve the L1$
prefetcher hit rate for batching, and also provide means for efficient DMA in case of HW-offload. With knowledge of the number of elements in each buffer it could maybe
also be possible to calculate the descriptor index range to DMA.

BR,

-Lars