doc: announce ring ABI and API changes
PATCH 1/1 doc: announce ring ABI and API changes
In order to support the non-blocking ring[1], one ABI change and one API change are required in librte_ring. This commit updates the deprecation notice to pave the way for their inclusion in 19.05.
[1] http://mails.dpdk.org/archives/dev/2019-January/123475.html
--- doc/guides/rel_notes/deprecation.rst | 11 +++++++++++ 1 file changed, 11 insertions(+)
doc/guides/rel_notes/deprecation.rst +11 -0
@@ -83,3 +83,14 @@ Deprecation Notices - The size and layout of ``rte_cryptodev_qp_conf`` and syntax of ``rte_cryptodev_queue_pair_setup`` will change to to allow to use two different mempools for crypto and device private sessions. + +* ring: two changes are planned for rte_ring in v19.05: + + - The ring head and tail values are planned to be changed from ``uint32_t`` + to ``size_t``. This reduces the likelihood of wrap-around to effectively + zero for 64-bit builds, which is important in avoiding the ABA problem in + the upcoming non-blocking ring implementation. (32-bit builds are + unaffected by this change.) + - rte_ring_get_memsize() will get a new ``flags`` parameter, so it can + calculate the memory required for rings that require more than 8B per entry + (such as the upcoming non-blocking ring).
Signature · 1 line
2.13.6
Quoted 20 lines · On Tue, 15 Jan 2019 17:59:34 -0600 Gage Eads <[email protected]> wrote:
In order to support the non-blocking ring[1], one ABI change and one API change are required in
librte_ring. This commit updates the deprecation notice to pave the way for their inclusion in 19.05.[1] http://mails.dpdk.org/archives/dev/2019-January/123475.html
Signed-off-by Gage Eads <[email protected]>--- doc/guides/rel_notes/deprecation.rst | 11 +++++++++++ 1 file changed, 11 insertions(+)doc/guides/rel_notes/deprecation.rst +1
-0@@ -83,3 +83,14 @@ Deprecation Notices - The size and layout of ``rte_cryptodev_qp_conf`` and syntax of ``rte_cryptodev_queue_pair_setup`` will change to to allow to use two different mempools for crypto and device private sessions. +
doc/guides/rel_notes/deprecation.rst Deprecation Notices · line 96
Quoted 10 lines
doc/guides/rel_notes/deprecation.rst +10
-0+* ring: two changes are planned for rte_ring in v19.05: + + - The ring head and tail values are planned to be changed from ``uint32_t`` + to ``size_t``. This reduces the likelihood of wrap-around to effectively + zero for 64-bit builds, which is important in avoiding the ABA problem in + the upcoming non-blocking ring implementation. (32-bit builds are + unaffected by this change.) + - rte_ring_get_memsize() will get a new ``flags`` parameter, so it can + calculate the memory required for rings that require more than 8B per entry + (such as the upcoming non-blocking ring).
Would it be possible to support new and old ring types, either through naming tricks and/or new ring flag? Changing things like ring buffer and mbuf are basically a flag day for all users.
I admit to having a personal interest in this since the API/ABI churn is this project causes vendors to stay on older code. And older code does not correctly support newer networks.
Quoted 48 lines
-----Original Message-----
From: Stephen Hemminger [mailto:[email protected]]
Sent: Tuesday, January 15, 2019 6:34 PM
To: Eads, Gage <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; Richardson, Bruce <[email protected]>; Ananyev, Konstantin <[email protected]>
Subject: Re: [PATCH] doc: announce ring ABI and API changesQuoted 20 lines · On Tue, 15 Jan 2019 17:59:34 -0600 Gage Eads <[email protected]> wrote:
In order to support the non-blocking ring[1], one ABI change and one API change are required in
librte_ring. This commit updates the deprecation notice to pave the way for their inclusion in 19.05.[1] http://mails.dpdk.org/archives/dev/2019-January/123475.html
Signed-off-by Gage Eads <[email protected]>--- doc/guides/rel_notes/deprecation.rst | 11 +++++++++++ 1 file changed, 11 insertions(+)doc/guides/rel_notes/deprecation.rst +0
-0@@ -83,3 +83,14 @@ Deprecation Notices - The size and layout of ``rte_cryptodev_qp_conf`` and syntax of ``rte_cryptodev_queue_pair_setup`` will change to to allow to use two different mempools for crypto and device private sessions.doc/guides/rel_notes/deprecation.rst Deprecation Notices · line 95
Quoted 10 lines
doc/guides/rel_notes/deprecation.rst +10
-0+ +* ring: two changes are planned for rte_ring in v19.05: + + - The ring head and tail values are planned to be changed from ``uint32_t`` + to ``size_t``. This reduces the likelihood of wrap-around to effectively + zero for 64-bit builds, which is important in avoiding the ABA problem in + the upcoming non-blocking ring implementation. (32-bit builds are + unaffected by this change.) + - rte_ring_get_memsize() will get a new ``flags`` parameter, so it can + calculate the memory required for rings that require more than 8B perentry
doc/guides/rel_notes/deprecation.rst Deprecation Notices · line 96
Quoted 1 line
doc/guides/rel_notes/deprecation.rst +1
-0+ (such as the upcoming non-blocking ring).
Quoted 7 lines
Would it be possible to support new and old ring types, either through naming tricks and/or new ring flag? Changing things like ring buffer and mbuf are basically a flag day for all users.
I admit to having a personal interest in this since the API/ABI churn is this project causes vendors to stay on older code. And older code does not correctly support newer networks.
Fair enough -- I appreciate the additional context wrt avoiding churn.
This might be doable with the following change:
"
@@ -70,6 +70,15 @@ struct rte_ring_headtail {
uint32_t single; /**< True if single prod/cons */
};
+/* 64-bit version of rte_ring_headtail, for use by rings that need to avoid
+ * head/tail wrap-around.
+ */
+struct rte_ring_headtail_64 {
+ volatile uint64_t head; /**< Prod/consumer head. */ + volatile uint64_t tail; /**< Prod/consumer tail. */ + uint32_t single; /**< True if single prod/cons */
+};
+
/**
* An RTE ring structure.
*
@@ -97,11 +106,19 @@ struct rte_ring {
char pad0 __rte_cache_aligned; /**< empty cache line */
/** Ring producer status. */
- struct rte_ring_headtail prod __rte_cache_aligned;
+ RTE_STD_C11
+ union {+ struct rte_ring_headtail prod __rte_cache_aligned;
+ struct rte_ring_headtail_64 prod_64 __rte_cache_aligned;
+ };
char pad1 __rte_cache_aligned; /**< empty cache line */
/** Ring consumer status. */
- struct rte_ring_headtail cons __rte_cache_aligned;
+ RTE_STD_C11
+ union {+ struct rte_ring_headtail cons __rte_cache_aligned;
+ struct rte_ring_headtail_64 cons_64 __rte_cache_aligned;
+ };
char pad2 __rte_cache_aligned; /**< empty cache line */
};
"
The ABI compatibility hinges on the fact that today's prod and cons are both padded out to a full cache line, and the 64-bit version fits within a single cache line. (Confirmed with pahole.)
abi-compliance-checker reports two issues, but both appear to be false positives:
1. "Field cons has been removed from this type"
2. "Field prod has been removed from this type"
I need to do more work to see whether/how the ring functions are affected by such a change, but I first want to check if the community agrees with this approach. Note that I don't see any way to avoid the API change to rte_ring_get_memsize, but I doubt that would have near the impact of a ring data structure change.
Thanks,
Gage
PATCH v2 1/1 doc: announce ring API change
In reply to Gage EadsIn order to support the non-blocking ring[1], an API change (additional argument to rte_ring_get_memsize()) is required in librte_ring. This commit updates the deprecation notice to pave the way for its inclusion in 19.05.
[1] http://mails.dpdk.org/archives/dev/2019-January/123774.html
---
v2:
- Drop the ABI change notice
doc/guides/rel_notes/deprecation.rst | 6 ++++++ 1 file changed, 6 insertions(+)
doc/guides/rel_notes/deprecation.rst +6 -0
@@ -83,3 +83,9 @@ Deprecation Notices - The size and layout of ``rte_cryptodev_qp_conf`` and syntax of ``rte_cryptodev_queue_pair_setup`` will change to to allow to use two different mempools for crypto and device private sessions. + +* ring: two changes are planned for rte_ring in v19.05: + + - rte_ring_get_memsize() will get a new ``flags`` parameter, so it can + calculate the memory required for rings that require more than 8B per entry + (such as the upcoming non-blocking ring).
Signature · 1 line
2.13.6
PATCH v3 1/1 doc: announce ring API change
In order to support the non-blocking ring[1], an API change (additional argument to rte_ring_get_memsize()) is required in librte_ring. This commit updates the deprecation notice to pave the way for its inclusion in 19.05.
[1] http://mails.dpdk.org/archives/dev/2019-January/123774.html
---
v3:
- "two changes are planned" -> "one change is planned"
v2:
- Drop the ABI change notice
doc/guides/rel_notes/deprecation.rst | 6 ++++++ 1 file changed, 6 insertions(+)
doc/guides/rel_notes/deprecation.rst +6 -0
@@ -83,3 +83,9 @@ Deprecation Notices - The size and layout of ``rte_cryptodev_qp_conf`` and syntax of ``rte_cryptodev_queue_pair_setup`` will change to to allow to use two different mempools for crypto and device private sessions. + +* ring: one change is planned for rte_ring in v19.05: + + - rte_ring_get_memsize() will get a new ``flags`` parameter, so it can + calculate the memory required for rings that require more than 8B per entry + (such as the upcoming non-blocking ring).
Signature · 1 line
2.13.6
Re: [PATCH v3] doc: announce ring API change
18/01/2019 16:31, Gage Eads:
Quoted 8 lines
In order to support the non-blocking ring[1], an API change (additional argument to
rte_ring_get_memsize()) is required inlibrte_ring. This commit updates the deprecation notice to pave the way for its inclusion in 19.05.[1] http://mails.dpdk.org/archives/dev/2019-January/123774.html
Signed-off-by Gage Eads <[email protected]>
Nobody agreed on this change.
Gage, do you still want to push non-blocking ring in 19.05?
---
Quoted 5 lines
doc/guides/rel_notes/deprecation.rst +5
-0+* ring: one change is planned for rte_ring in v19.05: + + - rte_ring_get_memsize() will get a new ``flags`` parameter, so it can + calculate the memory required for rings that require more than 8B per entry + (such as the upcoming non-blocking ring).
Re: [PATCH v3] doc: announce ring API change
Quoted 18 lines
-----Original Message-----
From: Thomas Monjalon [mailto:[email protected]]
Sent: Friday, February 1, 2019 5:17 AM
To: [email protected]; Eads, Gage <[email protected]>
Cc: [email protected]; [email protected]; Richardson, Bruce <[email protected]>; Ananyev, Konstantin <[email protected]>; [email protected]; Mcnamara, John <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v3] doc: announce ring API change18/01/2019 16:31, Gage Eads:
Quoted 5 lines
In order to support the non-blocking ring[1], an API change (additional argument to
rte_ring_get_memsize()) is required inlibrte_ring. This commit updates the deprecation notice to pave the way for its inclusion in 19.05.
Quoted 8 lines
Quoted 3 lines
[1] http://mails.dpdk.org/archives/dev/2019-January/123774.html
Signed-off-by Gage Eads <[email protected]>Nobody agreed on this change.
Gage, do you still want to push non-blocking ring in 19.05?
It was optimistic to try for agreement before the 19.02 release :) -- no, let's delay it until 19.08. I'll edit and resubmit this patch.
Quoted 13 lines
---
doc/guides/rel_notes/deprecation.rst Deprecation Notices
Quoted 4 lines
doc/guides/rel_notes/deprecation.rst +4
-0+* ring: one change is planned for rte_ring in v19.05: + + - rte_ring_get_memsize() will get a new ``flags`` parameter, so it can + calculate the memory required for rings that require more than 8B perentry
Quoted 1 line
+ (such as the upcoming non-blocking ring).
PATCH v4 1/1 doc: announce ring API change
In reply to Gage EadsIn order to support the non-blocking ring[1], an API change (additional argument to rte_ring_get_memsize()) is required in librte_ring. This commit updates the deprecation notice to pave the way for its inclusion in 19.08.
[1] http://mails.dpdk.org/archives/dev/2019-January/124162.html
---
v4:
- 19.05 -> 19.08
v3:
- "two changes are planned" -> "one change is planned"
v2:
- Drop the ABI change notice
doc/guides/rel_notes/deprecation.rst | 6 ++++++ 1 file changed, 6 insertions(+)
doc/guides/rel_notes/deprecation.rst +6 -0
@@ -83,3 +83,9 @@ Deprecation Notices - The size and layout of ``rte_cryptodev_qp_conf`` and syntax of ``rte_cryptodev_queue_pair_setup`` will change to to allow to use two different mempools for crypto and device private sessions. + +* ring: one change is planned for rte_ring in v19.08: + + - rte_ring_get_memsize() will get a new ``flags`` parameter, so it can + calculate the memory required for rings that require more than 8B per entry + (such as the upcoming non-blocking ring).
Signature · 1 line
2.13.6
Re: [dpdk-dev] [PATCH v4] doc: announce ring API change
01/02/2019 15:36, Gage Eads:
Quoted 8 lines
In order to support the non-blocking ring[1], an API change (additional argument to
rte_ring_get_memsize()) is required inlibrte_ring. This commit updates the deprecation notice to pave the way for its inclusion in 19.08.[1] http://mails.dpdk.org/archives/dev/2019-January/124162.html
Signed-off-by Gage Eads <[email protected]>
There is still no agreement on this change?
Re: [dpdk-dev] [PATCH v4] doc: announce ring API change
Quoted 6 lines
01/02/2019 15:36, Gage Eads:
Quoted 5 lines
In order to support the non-blocking ring[1], an API change (additional argument to
rte_ring_get_memsize()) is required inlibrte_ring. This commit updates the deprecation notice to pave the way for its inclusion in 19.08.
Quoted 6 lines
Quoted 3 lines
[1] http://mails.dpdk.org/archives/dev/2019-January/124162.html
Signed-off-by Gage Eads <[email protected]>There is still no agreement on this change?
Still none. I was hoping this discussion (http://mails.dpdk.org/archives/dev/2019-April/129229.html) would lead to some clear direction, but at this point the effort is stalled.
Re: [dpdk-dev] [PATCH v4] doc: announce ring API change
Quoted 6 lines · Gage
Quoted 6 lines
01/02/2019 15:36, Gage Eads:
Quoted 5 lines
In order to support the non-blocking ring[1], an API change (additional argument to
rte_ring_get_memsize()) is required inlibrte_ring. This commit updates the deprecation notice to pave the way for its inclusion in 19.08.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
[1] http://mails.dpdk.org/archives/dev/2019-January/124162.html
Signed-off-by Gage Eads <[email protected]>There is still no agreement on this change?
Still none. I was hoping this discussion (http://mails.dpdk.org/archives/dev/2019-April/129229.html) would lead to some clear direction, but at this point the effort is stalled.
The fundamental tradeoff is between non-blocking rings and ABI breakage.
Why not have a new ring type for non-blocking rings since non-blocking rings are not necessary for all use cases.
Re: [dpdk-dev] [PATCH v4] doc: announce ring API change
Quoted 13 lines · Stephen Hemminger
On Fri, 10 May 2019 14:53:56 +0000
"Eads, Gage" <[email protected]> wrote:01/02/2019 15:36, Gage Eads:
In order to support the non-blocking ring[1], an API change (additional argument torte_ring_get_memsize()) is required inlibrte_ring. This commit updates the deprecation notice to pave the way for its inclusion in 19.08.[1] http://mails.dpdk.org/archives/dev/2019-January/124162.html
Signed-off-by Gage Eads <[email protected]>
Quoted 8 lines
There is still no agreement on this change?
Still none. I was hoping this discussion (
http://mails.dpdk.org/archives/dev/2019-April/129229.html) would lead to some clear direction, but at this point the effort is stalled.The fundamental tradeoff is between non-blocking rings and ABI breakage.
It is also possible to do "non-blocking" (but not lock-free) rings with the original element size (a pointer per ring slot) as implemented here:
https://github.com/ARM-software/progress64/blob/master/src/p64_ringbuf.c
Some extra (head&tail) metadata is required but I think there is space for that in the rte_ring structure.
Quoted 2 lines
Why not have a new ring type for non-blocking rings since non-blocking rings are not necessary for all use cases.
I proposed a new library ("rte_lfring") with lock-free rings here:
https://mails.dpdk.org/archives/dev/2019-January/124242.html
The lock-free design should be the same as in Gage's patch.
rte_lfring could of course be part of the rte_ring library.
Signature · 2 lines
Ola Liljedahl, Networking System Architect, Arm Phone +46706866373, Skype ola.liljedahl
Re: [dpdk-dev] [PATCH v4] doc: announce ring API change
Quoted 42 lines
-----Original Message-----
From: Ola Liljedahl [mailto:[email protected]]
Sent: Friday, May 10, 2019 10:19 AM
To: [email protected]; Eads, Gage <[email protected]>
Cc: [email protected]; nd <[email protected]>; Ananyev, Konstantin <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; [email protected]; Honnappa Nagarahalli <[email protected]>; Gavin Hu (Arm Technology China) <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v4] doc: announce ring API changeQuoted 13 lines · Stephen Hemminger
On Fri, 10 May 2019 14:53:56 +0000
"Eads, Gage" <[email protected]> wrote:01/02/2019 15:36, Gage Eads:
In order to support the non-blocking ring[1], an API change (additional argument torte_ring_get_memsize()) is required inlibrte_ring. This commit updates the deprecation notice to pave the way for its inclusion in 19.08.[1] http://mails.dpdk.org/archives/dev/2019-January/124162.html
Signed-off-by Gage Eads <[email protected]>Quoted 8 lines
There is still no agreement on this change?
Still none. I was hoping this discussion (
http://mails.dpdk.org/archives/dev/2019-April/129229.html) would lead to some clear direction, but at this point the effort is stalled.The fundamental tradeoff is between non-blocking rings and ABI breakage.
It is also possible to do "non-blocking" (but not lock-free) rings with the original element size (a pointer per ring slot) as implemented here:
https://github.com/ARM-software/progress64/blob/master/src/p64_ringbuf.c
Some extra (head&tail) metadata is required but I think there is space for that in therte_ringstructure.
Quoted 9 lines
Quoted 2 lines
Why not have a new ring type for non-blocking rings since non-blocking rings are not necessary for all use cases.
I proposed a new library ("
rte_lfring") with lock-free rings here:
https://mails.dpdk.org/archives/dev/2019-January/124242.html
The lock-free design should be the same as in Gage's patch.
rte_lfringcould of course be part of therte_ringlibrary.
Just read through the API/ABI stability discussion (https://mails.dpdk.org/archives/dev/2019-April/128969.html). I'll drop my patchset and work on supporting this lfring API instead.
Thanks,
Gage
Quoted 4 lines
--
Ola Liljedahl, Networking System Architect, Arm Phone +46706866373, Skype ola.liljedahl
Re: [dpdk-dev] [PATCH v4] doc: announce ring API change
Quoted 49 lines · Gage
Quoted 49 lines
-----Original Message-----
From: Ola Liljedahl [mailto:[email protected]]
Sent: Friday, May 10, 2019 10:19 AM
To: [email protected]; Eads, Gage <[email protected]>
Cc: [email protected]; nd <[email protected]>; Ananyev, Konstantin <[email protected]>; [email protected]; Richardson, Bruce <[email protected]>; [email protected]; [email protected]; Honnappa Nagarahalli <[email protected]>; Gavin Hu (Arm Technology China) <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v4] doc: announce ring API changeQuoted 13 lines · Stephen Hemminger
On Fri, 10 May 2019 14:53:56 +0000
"Eads, Gage" <[email protected]> wrote:01/02/2019 15:36, Gage Eads:
In order to support the non-blocking ring[1], an API change (additional argument torte_ring_get_memsize()) is required inlibrte_ring. This commit updates the deprecation notice to pave the way for its inclusion in 19.08.[1] http://mails.dpdk.org/archives/dev/2019-January/124162.html
Signed-off-by Gage Eads <[email protected]>Quoted 8 lines
There is still no agreement on this change?
Still none. I was hoping this discussion (
http://mails.dpdk.org/archives/dev/2019-April/129229.html) would lead to some clear direction, but at this point the effort is stalled.The fundamental tradeoff is between non-blocking rings and ABI breakage.
It is also possible to do "non-blocking" (but not lock-free) rings with the original element size (a pointer per ring slot) as implemented here:
https://github.com/ARM-software/progress64/blob/master/src/p64_ringbuf.c
Some extra (head&tail) metadata is required but I think there is space for that in therte_ringstructure.Quoted 2 lines
Why not have a new ring type for non-blocking rings since non-blocking rings are not necessary for all use cases.
I proposed a new library ("
rte_lfring") with lock-free rings here:
https://mails.dpdk.org/archives/dev/2019-January/124242.html
The lock-free design should be the same as in Gage's patch.
Quoted 6 lines
Quoted 2 lines
rte_lfringcould of course be part of therte_ringlibrary.Just read through the API/ABI stability discussion
(https://mails.dpdk.org/archives/dev/2019-April/128969.html). I'll drop my patchset and work on supporting this lfring API instead.
+1
Given the discussions related to ABI stability, it looks better to implement this in another library.
Olivier