generic rte atomic APIs deprecate proposal
<snip>
Quoted 83 lines
Subject: Re: [PATCH v3 00/12] generic rte atomic APIs deprecate proposal
18/03/2020 15:01, Van Haaren, Harry:
Quoted 11 lines
Hi Phil & Honnappa,
From: Phil Yang <[email protected]>
Quoted 8 lines
DPDK provides generic
rte_atomicAPIs to do several atomic operations. These APIs are using the deprecated__syncbuilt-ins and enforce full memory barriers on aarch64. However, full barriers are not necessary in many use cases. In order to address such use cases, C language offers
C11 atomic APIs. The C11 atomic APIs provide finer memory barrier control by making use of the memory ordering parameter provided by theuser.
Quoted 8 lines
Quoted 8 lines
Various patches submitted in the past [2] and the patches in this series indicate significant performance gains on multiple aarch64 CPUs and no performance loss on x86.
But the existing
rte_atomicAPI implementations cannot be changed as the APIs do not take the memory ordering parameter. The only choice available is replacing the usage of therte_atomicAPIs with C11 atomic APIs. In order to make this change, the following steps areproposed:
Quoted 11 lines
Quoted 4 lines
[1] deprecate
rte_atomicAPIs so that future patches do not userte_atomicAPIs (a script is added to flag the usages).
[2] refactor the code that usesrte_atomicAPIs to use c11 atomic APIs.On [1] above, I feel deprecating DPDKs atomic functions and failing checkpatch is a bit sudden. Perhaps noting that in a future release (20.11?) DPDK will move to a
C11 based atomics model is a more gradual step to achieving the goal, and at that point add a checkpatch warning for additions of rte_atomic*?Quoted 9 lines
More on [2] in context below.
The above is my point-of-view, of course I'd like more people from the DPDK community to provide their input too.
Quoted 3 lines
This patchset contains:
1) the checkpatch script changes to flagrte_atomicAPI usage in patches.
2) changes to programmer guide describing writing efficient code foraarch64.
Quoted 5 lines
Quoted 5 lines
3) changes to various libraries to make use of c11 atomic APIs.
We are planning to replicate this idea across all the other libraries, drivers, examples, test applications. In the next phase, we will add changes to the mbuf, the EAL interrupts and the event timer
adapter libraries.
Quoted 3 lines
About ~6/12 patches of this C11 set are targeting the Service Cores area of DPDK. I have some concerns over increased complexity of C11
implementation vs the (already complex)
rte_atomicimplementation today.Quoted 2 lines
I see other patchsets enabling C11 across other DPDK components, so maybe we should also discuss C11 enabling in a wider context that just
service cores?
Quoted 3 lines
I don't think it fair to expect all developers to be well versed in C11 atomic semantics like understanding the complex interactions between
the various C11 RELEASE, AQUIRE barriers requires.
Quoted 4 lines
As maintainer of Service Cores I'm hesitant to accept the large-scale refactor of atomic-implementation, as it could lead to racey bugs that are likely extremely difficult to track down. (The recent race-on-exit has
proven the difficulty in reproducing, and that's with an atomics model I'm quite familiar with).
Quoted 4 lines
Let me be very clear: I don't wish to block a C11 atomic implementation, but I'd like to discuss how we (DPDK community) can best port-to and maintain a complex multi-threaded service library with
best-in-class performance for the workload.
Quoted 7 lines
To put some discussions/solutions on the table:
- Shared Maintainership of a component?
Split in functionality and C11 atomics implementation Obviously there would be collaboration required in such a case.
- Maybe shared maintainership is too much?
A gentlemans/womans agreement of "helping out" with C11 atomicsdebug is enough?
Quoted 2 lines
Quoted 7 lines
Quoted 2 lines
Hope my concerns are understandable, and of course input/feedback welcomed! -Harry
Thanks for raising the issue Harry.
I think we should have at least two official maintainers for C11 atomics in general.
Sure, I can volunteer.
Quoted 2 lines
C11 conversion is a progressive effort being done, and should be merged step by step.
Agree, the changes need to be understood, it is not a search and replace effort. The changes will come-in in stages unless others join the effort.
The concern I have is about the new patches that get added. I think we need to stop the new patches from using rte_atomic APIs, otherwise we might be making these changes forever.
Quoted 2 lines
If C11 maintainers fail to fix some issues on time, then we can hold the effort. Does it make sense?
I am fine with this approach. But, I think we need to have a deadline in mind to complete the work.
Quoted 1 line
Quoted 91 lines · Honnappa Nagarahalli
<snip>
Quoted 88 lines
Subject: Re: [PATCH v3 00/12] generic rte atomic APIs deprecate proposal
18/03/2020 15:01, Van Haaren, Harry:
Quoted 11 lines
Hi Phil & Honnappa,
From: Phil Yang <[email protected]>
Quoted 8 lines
DPDK provides generic
rte_atomicAPIs to do several atomic operations. These APIs are using the deprecated__syncbuilt-ins and enforce full memory barriers on aarch64. However, full barriers are not necessary in many use cases. In order to address such use cases, C language offers
C11 atomic APIs. The C11 atomic APIs provide finer memory barrier control by making use of the memory ordering parameter provided by theuser.
Quoted 8 lines
Quoted 8 lines
Various patches submitted in the past [2] and the patches in this series indicate significant performance gains on multiple aarch64 CPUs and no performance loss on x86.
But the existing
rte_atomicAPI implementations cannot be changed as the APIs do not take the memory ordering parameter. The only choice available is replacing the usage of therte_atomicAPIs with C11 atomic APIs. In order to make this change, the following steps areproposed:
Quoted 11 lines
Quoted 4 lines
[1] deprecate
rte_atomicAPIs so that future patches do not userte_atomicAPIs (a script is added to flag the usages).
[2] refactor the code that usesrte_atomicAPIs to use c11 atomic APIs.On [1] above, I feel deprecating DPDKs atomic functions and failing checkpatch is a bit sudden. Perhaps noting that in a future release (20.11?) DPDK will move to a
C11 based atomics model is a more gradual step to achieving the goal, and at that point add a checkpatch warning for additions of rte_atomic*?Quoted 9 lines
More on [2] in context below.
The above is my point-of-view, of course I'd like more people from the DPDK community to provide their input too.
Quoted 3 lines
This patchset contains:
1) the checkpatch script changes to flagrte_atomicAPI usage in patches.
2) changes to programmer guide describing writing efficient code foraarch64.
Quoted 5 lines
Quoted 5 lines
3) changes to various libraries to make use of c11 atomic APIs.
We are planning to replicate this idea across all the other libraries, drivers, examples, test applications. In the next phase, we will add changes to the mbuf, the EAL interrupts and the event timer
adapter libraries.
Quoted 3 lines
About ~6/12 patches of this C11 set are targeting the Service Cores area of DPDK. I have some concerns over increased complexity of C11
implementation vs the (already complex)
rte_atomicimplementation today.Quoted 2 lines
I see other patchsets enabling C11 across other DPDK components, so maybe we should also discuss C11 enabling in a wider context that just
service cores?
Quoted 3 lines
I don't think it fair to expect all developers to be well versed in C11 atomic semantics like understanding the complex interactions between
the various C11 RELEASE, AQUIRE barriers requires.
Quoted 4 lines
As maintainer of Service Cores I'm hesitant to accept the large-scale refactor of atomic-implementation, as it could lead to racey bugs that are likely extremely difficult to track down. (The recent race-on-exit has
proven the difficulty in reproducing, and that's with an atomics model I'm quite familiar with).
Quoted 4 lines
Let me be very clear: I don't wish to block a C11 atomic implementation, but I'd like to discuss how we (DPDK community) can best port-to and maintain a complex multi-threaded service library with
best-in-class performance for the workload.
Quoted 7 lines
To put some discussions/solutions on the table:
- Shared Maintainership of a component?
Split in functionality and C11 atomics implementation Obviously there would be collaboration required in such a case.
- Maybe shared maintainership is too much?
A gentlemans/womans agreement of "helping out" with C11 atomicsdebug is enough?
Quoted 4 lines
Hope my concerns are understandable, and of course input/feedback welcomed! -Harry
Thanks for raising the issue Harry.
Quoted 8 lines
Quoted 2 lines
I think we should have at least two official maintainers for C11 atomics in general.
Sure, I can volunteer.
Quoted 2 lines
C11 conversion is a progressive effort being done, and should be merged step by step.
Agree, the changes need to be understood, it is not a search and replace effort. The changes will come-in in stages unless others join the effort.
The concern I have is about the new patches that get added. I think we need to stop the new patches from usingrte_atomicAPIs, otherwise we might be making these changes forever.
+1. We must define a time frame otherwise we might be making these changes forever.
Quoted 7 lines
Quoted 3 lines
If C11 maintainers fail to fix some issues on time, then we can hold the effort. Does it make sense?
I am fine with this approach. But, I think we need to have a deadline in mind to complete the work.
Quoted 1 line
<snip>
Quoted 23 lines
Quoted 9 lines
Subject: [PATCH v3 00/12] generic rte atomic APIs deprecate proposal
DPDK provides generic
rte_atomicAPIs to do several atomic operations. These APIs are using the deprecated__syncbuilt-ins and enforce full memory barriers on aarch64. However, full barriers are not necessary in many use cases. In order to address such use cases, C language offers
C11 atomic APIs. The C11 atomic APIs provide finer memory barrier control by making use of the memory ordering parameter provided by theuser.
Quoted 4 lines
Various patches submitted in the past [2] and the patches in this series indicate significant performance gains on multiple aarch64 CPUs and no performance loss on x86.
Quoted 8 lines
But the existing
rte_atomicAPI implementations cannot be changed as the APIs do not take the memory ordering parameter. The only choice available is replacing the usage of therte_atomicAPIs with C11 atomic APIs. In order to make this change, the following steps are proposed:[1] deprecate
rte_atomicAPIs so that future patches do not userte_atomicAPIs (a script is added to flag the usages).
[2] refactor the code that usesrte_atomicAPIs to use c11 atomic APIs.
Quoted 5 lines
On [1] above, I feel deprecating DPDKs atomic functions and failing checkpatch is a bit sudden. Perhaps noting that in a future release (20.11?) DPDK will move to a
C11 based atomics model is a more gradual step to achieving the goal, and at that point add a checkpatch warning for additions of rte_atomic*?
We have been working on changing existing usages of rte_atomic APIs in DPDK to use C11 atomics. Usually, the x.11 releases have significant amount of changes (not sure how many would use rte_atomic APIs). I would prefer that in 20.11 no additional code is added using rte_atomics APIs. However, I am open to suggestions on the exact time frame. Once we decide on the release, I think it makes sense to add a 'warning' in the checkpatch to indicate the deprecation timeline and add an 'error' after the release.
Quoted 13 lines
More on [2] in context below.
The above is my point-of-view, of course I'd like more people from the DPDK community to provide their input too.
Quoted 3 lines
This patchset contains:
1) the checkpatch script changes to flagrte_atomicAPI usage in patches.
2) changes to programmer guide describing writing efficient code foraarch64.
Quoted 2 lines
3) changes to various libraries to make use of c11 atomic APIs.
Quoted 8 lines
Quoted 3 lines
We are planning to replicate this idea across all the other libraries, drivers, examples, test applications. In the next phase, we will add changes to the mbuf, the EAL interrupts and the event timer adapter
libraries.
About ~6/12 patches of this C11 set are targeting the Service Cores area of DPDK. I have some concerns over increased complexity of C11 implementation vs the (already complex)
rte_atomicimplementation today.
I agree that it C11 changes are complex, especially if one is starting out to understand what these APIs provide. From my experience, once few underlying concepts are understood, reviewing or making changes do not take too much time.
Quoted 2 lines
I see other patchsets enabling C11 across other DPDK components, so maybe we should also discuss C11 enabling in a wider context that just service cores?
Yes, agree. We are in the process of making changes to other areas as well.
Quoted 4 lines
I don't think it fair to expect all developers to be well versed in C11 atomic semantics like understanding the complex interactions between the various C11 RELEASE, AQUIRE barriers requires.
C11 has been around from sometime now. To my surprise, OVS already uses C11 APIs extensively. VPP has been accepting C11 related changes from past couple of years. Having said that, I agree in general that not everyone is well versed.
Quoted 2 lines
As maintainer of Service Cores I'm hesitant to accept the large-scale refactor
Right now, the patches are split into multiple commits. If required I can host a call to go over simple C11 API usages (sufficient to cover the usage in service core) and the changes in this patch. If you find that particular areas need more understanding I can work on providing additional information such as memory order ladder diagrams. Please let me know what you think.
Quoted 10 lines
of atomic-implementation, as it could lead to racey bugs that are likely extremely difficult to track down. (The recent race-on-exit has proven the difficulty in reproducing, and that's with an atomics model I'm quite familiar with).
Let me be very clear: I don't wish to block a C11 atomic implementation, but I'd like to discuss how we (DPDK community) can best port-to and maintain a complex multi-threaded service library with best-in-class performance for the workload.
Quoted 7 lines
To put some discussions/solutions on the table:
- Shared Maintainership of a component?
Split in functionality and C11 atomics implementation Obviously there would be collaboration required in such a case.
- Maybe shared maintainership is too much?
A gentlemans/womans agreement of "helping out" with C11 atomics debug
is enough?
I think shared maintainer ship could be too much as there are many changes. But, I and other engineers from Arm (I would include Arm ecosystem as well) can definitely help out on debug and reviews involving C11 APIs. (I see Thomas's reply on this topic).
Quoted 4 lines
Hope my concerns are understandable, and of course input/feedback welcomed! -Harry
No worries at all. We are here to help and make this as easy as possible.
Quoted 3 lines
PS: Apologies for the delay in reply - was OOO on Irish national holiday.
Same here, was on vacation for 3 days.
Quoted 43 lines
Quoted 43 lines
v3:
add libatomic dependency for 32-bit clangv2:
1. fix Clang '-Wincompatible-pointer-types' WARNING.
2. fix typos.Honnappa Nagarahalli (2):
service: avoid race condition for MT unsafe service
service: identify service running on another core correctlyPhil Yang (10):
doc: add generic atomic deprecation section
devtools: prevent use of rte atomic APIs in future patches eal/build: add libatomic dependency for 32-bit clang
build: remove redundant code
vhost: optimize broadcast rarp sync with c11 atomic
ipsec: optimize with c11 atomic for sa outbound sqn update
service: remove rte prefix from static functions
service: remove redundant code
service: optimize with c11 one-way barrier
service: relax barriers with C11 atomic operationsdevtools/checkpatches.sh | 9 ++ doc/guides/prog_guide/writing_efficient_code.rst | 60 +++++++- drivers/event/octeontx/meson.build | 5 - drivers/event/octeontx2/meson.build | 5 - drivers/event/opdl/meson.build | 5 - lib/librte_eal/common/rte_service.c | 175 ++++++++++++---------- - lib/librte_eal/meson.build | 6 + lib/librte_ipsec/ipsec_sqn.h | 3 +- lib/librte_ipsec/sa.h | 2 +- lib/librte_rcu/meson.build | 5 - lib/librte_vhost/vhost.h | 2 +- lib/librte_vhost/vhost_user.c | 7 +- lib/librte_vhost/virtio_net.c | 16 ++- 13 files changed, 181 insertions(+), 119 deletions(-)--
2.7.4
+ Erik as there are similar changes to timer library
<snip>
Quoted 83 lines
Quoted 11 lines
Quoted 11 lines
Subject: [PATCH v3 00/12] generic rte atomic APIs deprecate proposal
DPDK provides generic
rte_atomicAPIs to do several atomic operations. These APIs are using the deprecated__syncbuilt-ins and enforce full memory barriers on aarch64. However, full barriers are not necessary in many use cases. In order to address such use cases, C language offers
C11 atomic APIs. The C11 atomic APIs provide finer memory barrier control by making use of the memory ordering parameter provided by theQuoted 9 lines
user.
Quoted 8 lines
Various patches submitted in the past [2] and the patches in this series indicate significant performance gains on multiple aarch64 CPUs and no performance loss on x86.
But the existing
rte_atomicAPI implementations cannot be changed as the APIs do not take the memory ordering parameter. The only choice available is replacing the usage of therte_atomicAPIs with C11 atomic APIs. In order to make this change, the following steps areproposed:
Quoted 10 lines
Quoted 4 lines
[1] deprecate
rte_atomicAPIs so that future patches do not userte_atomicAPIs (a script is added to flag the usages).
[2] refactor the code that usesrte_atomicAPIs to use c11 atomic APIs.On [1] above, I feel deprecating DPDKs atomic functions and failing checkpatch is a bit sudden. Perhaps noting that in a future release (20.11?) DPDK will move to a
C11 based atomics model is a more gradual step to achieving the goal, and at that point add a checkpatch warning for additions of rte_atomic*?We have been working on changing existing usages of
rte_atomicAPIs in DPDK to use C11 atomics. Usually, the x.11 releases have significant amount of changes (not sure how many would userte_atomicAPIs). I would prefer that in 20.11 no additional code is added usingrte_atomicsAPIs. However, I am open to suggestions on the exact time frame.
Once we decide on the release, I think it makes sense to add a 'warning' in the checkpatch to indicate the deprecation timeline and add an 'error' after the release.Quoted 17 lines
More on [2] in context below.
The above is my point-of-view, of course I'd like more people from the DPDK community to provide their input too.
Quoted 3 lines
This patchset contains:
1) the checkpatch script changes to flagrte_atomicAPI usage in patches.
2) changes to programmer guide describing writing efficient code foraarch64.
Quoted 6 lines
3) changes to various libraries to make use of c11 atomic APIs.
We are planning to replicate this idea across all the other libraries, drivers, examples, test applications. In the next phase, we will add changes to the mbuf, the EAL interrupts and the event timer adapter
Quoted 5 lines
libraries.
About ~6/12 patches of this C11 set are targeting the Service Cores area of DPDK. I have some concerns over increased complexity of C11 implementation vs the (already complex)
rte_atomicimplementation today.I agree that it C11 changes are complex, especially if one is starting out to understand what these APIs provide. From my experience, once few underlying concepts are understood, reviewing or making changes do not take too much time.
Quoted 2 lines
I see other patchsets enabling C11 across other DPDK components, so maybe we should also discuss C11 enabling in a wider context that just
service cores?
Yes, agree. We are in the process of making changes to other areas as well.Quoted 5 lines
I don't think it fair to expect all developers to be well versed in C11 atomic semantics like understanding the complex interactions between the various
C11 RELEASE, AQUIRE barriers requires.C11 has been around from sometime now. To my surprise, OVS already uses C11 APIs extensively. VPP has been accepting C11 related changes from past couple of years. Having said that, I agree in general that not everyone is well versed.
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
As maintainer of Service Cores I'm hesitant to accept the large-scale refactor
Right now, the patches are split into multiple commits. If required I can host a call to go over simple C11 API usages (sufficient to cover the usage in service core) and the changes in this patch. If you find that particular areas need more understanding I can work on providing additional information such as memory order ladder diagrams. Please let me know what you think.
When I started working with C11 APIs, I had referred to the following blogs.
https://preshing.com/20120913/acquire-and-release-semantics/
https://preshing.com/20130702/the-happens-before-relation/
https://preshing.com/20130823/the-synchronizes-with-relation/
These will be helpful to understand the changes.
Quoted 78 lines
Quoted 11 lines
of atomic-implementation, as it could lead to racey bugs that are likely extremely difficult to track down. (The recent race-on-exit has proven the difficulty in reproducing, and that's with an atomics model I'm quite familiar with).
Let me be very clear: I don't wish to block a C11 atomic implementation, but I'd like to discuss how we (DPDK community) can best port-to and maintain a complex multi-threaded service library with best-in-class performance for the workload.
Quoted 7 lines
To put some discussions/solutions on the table:
- Shared Maintainership of a component?
Split in functionality and C11 atomics implementation Obviously there would be collaboration required in such a case.
- Maybe shared maintainership is too much?
A gentlemans/womans agreement of "helping out" with C11 atomics
debug is enough?I think shared maintainer ship could be too much as there are many changes. But, I and other engineers from Arm (I would include Arm ecosystem as well) can definitely help out on debug and reviews involving C11 APIs. (I see Thomas's reply on this topic).
Quoted 4 lines
Hope my concerns are understandable, and of course input/feedback welcomed! -Harry
No worries at all. We are here to help and make this as easy as possible.
Quoted 3 lines
PS: Apologies for the delay in reply - was OOO on Irish national holiday.
Same here, was on vacation for 3 days.
Quoted 43 lines
Quoted 43 lines
v3:
add libatomic dependency for 32-bit clangv2:
1. fix Clang '-Wincompatible-pointer-types' WARNING.
2. fix typos.Honnappa Nagarahalli (2):
service: avoid race condition for MT unsafe service
service: identify service running on another core correctlyPhil Yang (10):
doc: add generic atomic deprecation section
devtools: prevent use of rte atomic APIs in future patches eal/build: add libatomic dependency for 32-bit clang
build: remove redundant code
vhost: optimize broadcast rarp sync with c11 atomic
ipsec: optimize with c11 atomic for sa outbound sqn update
service: remove rte prefix from static functions
service: remove redundant code
service: optimize with c11 one-way barrier
service: relax barriers with C11 atomic operationsdevtools/checkpatches.sh | 9 ++ doc/guides/prog_guide/writing_efficient_code.rst | 60 +++++++- drivers/event/octeontx/meson.build | 5 - drivers/event/octeontx2/meson.build | 5 - drivers/event/opdl/meson.build | 5 - lib/librte_eal/common/rte_service.c | 175 ++++++++++++---------- - lib/librte_eal/meson.build | 6 + lib/librte_ipsec/ipsec_sqn.h | 3 +- lib/librte_ipsec/sa.h | 2 +- lib/librte_rcu/meson.build | 5 - lib/librte_vhost/vhost.h | 2 +- lib/librte_vhost/vhost_user.c | 7 +- lib/librte_vhost/virtio_net.c | 16 ++- 13 files changed, 181 insertions(+), 119 deletions(-)--
2.7.4
Quoted 52 lines
-----Original Message-----
From: Honnappa Nagarahalli <[email protected]>
Sent: Friday, March 20, 2020 6:32 PM
To: Van Haaren, Harry <[email protected]>; Phil Yang <[email protected]>; [email protected]; Ananyev, Konstantin <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Gavin Hu <[email protected]>; Ruifeng Wang <[email protected]>; Joyce Kong <[email protected]>; Carrillo, Erik G <[email protected]>; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>; nd <[email protected]>
Subject: RE: [PATCH v3 00/12] generic rte atomic APIs deprecate proposal+ Erik as there are similar changes to timer library
<snip>
Quoted 16 lines
Quoted 16 lines
Quoted 11 lines
Subject: [PATCH v3 00/12] generic rte atomic APIs deprecate proposal
DPDK provides generic
rte_atomicAPIs to do several atomic operations. These APIs are using the deprecated__syncbuilt-ins and enforce full memory barriers on aarch64. However, full barriers are not necessary in many use cases. In order to address such use cases, C language offers
C11 atomic APIs. The C11 atomic APIs provide finer memory barrier control by making use of the memory ordering parameter provided by theuser.
Quoted 4 lines
Various patches submitted in the past [2] and the patches in this series indicate significant performance gains on multiple aarch64 CPUs and no performance loss on x86.
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
But the existing
rte_atomicAPI implementations cannot be changed as the APIs do not take the memory ordering parameter. The only choice available is replacing the usage of therte_atomicAPIs with C11 atomic APIs. In order to make this change, the following steps areproposed:
Quoted 4 lines
Quoted 4 lines
[1] deprecate
rte_atomicAPIs so that future patches do not userte_atomicAPIs (a script is added to flag the usages).
[2] refactor the code that usesrte_atomicAPIs to use c11 atomicAPIs.
Quoted 7 lines
Quoted 6 lines
On [1] above, I feel deprecating DPDKs atomic functions and failing checkpatch is a bit sudden. Perhaps noting that in a future release (20.11?) DPDK will move to a
C11 based atomics model is a more gradual step to achieving the goal, and at that point add a checkpatch warning for additions of rte_atomic*?We have been working on changing existing usages of
rte_atomicAPIs inDPDK
Quoted 9 lines
Quoted 5 lines
to use C11 atomics. Usually, the x.11 releases have significant amount of changes (not sure how many would use
rte_atomicAPIs). I would prefer that in 20.11 no additional code is added usingrte_atomicsAPIs. However, I am open to suggestions on the exact time frame.
Once we decide on the release, I think it makes sense to add a 'warning'in the
Quoted 1 line
checkpatch to indicate the deprecation timeline and add an 'error' after
the
Quoted 1 line
release.
The above sounds reasonable - mainly let's not block any code that exists or is being developed today using rte_atomic_* APIs from making a release.
Quoted 26 lines
Quoted 8 lines
Quoted 8 lines
More on [2] in context below.
The above is my point-of-view, of course I'd like more people from the DPDK community to provide their input too.
Quoted 2 lines
This patchset contains:
1) the checkpatch script changes to flagrte_atomicAPI usage inpatches.
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
2) changes to programmer guide describing writing efficient code for
aarch64.
Quoted 6 lines
3) changes to various libraries to make use of c11 atomic APIs.
We are planning to replicate this idea across all the other libraries, drivers, examples, test applications. In the next phase, we will add changes to the mbuf, the EAL interrupts and the event timer adapter
Quoted 6 lines
Quoted 5 lines
libraries.
About ~6/12 patches of this C11 set are targeting the Service Cores area of DPDK. I have some concerns over increased complexity of C11 implementation vs the (already complex)
rte_atomicimplementation today.I agree that it C11 changes are complex, especially if one is starting out
to
Quoted 2 lines
understand what these APIs provide. From my experience, once few underlying concepts are understood, reviewing or making changes do not
Quoted 18 lines
take
Quoted 6 lines
too much time.
Quoted 2 lines
I see other patchsets enabling C11 across other DPDK components, so maybe we should also discuss C11 enabling in a wider context that just
service cores?
Yes, agree. We are in the process of making changes to other areas aswell.
Quoted 10 lines
Quoted 6 lines
I don't think it fair to expect all developers to be well versed in C11 atomic semantics like understanding the complex interactions between the various
C11 RELEASE, AQUIRE barriers requires.C11 has been around from sometime now. To my surprise, OVS already uses C11 APIs extensively. VPP has been accepting C11 related changes from past couple of years. Having said that, I agree in general that not everyone is well versed.
Fair point - like so many things, once familiar with it, it becomes easy :)
Quoted 11 lines
Quoted 3 lines
Quoted 2 lines
As maintainer of Service Cores I'm hesitant to accept the large-scale refactor
Right now, the patches are split into multiple commits. If required I can
host a
Quoted 1 line
call to go over simple C11 API usages (sufficient to cover the usage in
service
Quoted 1 line
core) and the changes in this patch. If you find that particular areas
need more
Quoted 1 line
understanding I can work on providing additional information such as
memory
Quoted 1 line
order ladder diagrams. Please let me know what you think.
Thanks for the offer - I will need to do my due diligence on reiview before taking up any of your or other C11 folks time.
Quoted 6 lines
When I started working with C11 APIs, I had referred to the following blogs.
https://preshing.com/20120913/acquire-and-release-semantics/
https://preshing.com/20130702/the-happens-before-relation/
https://preshing.com/20130823/the-synchronizes-with-relation/These will be helpful to understand the changes.
Thanks, indeed good articles. I found the following slide deck particularly informative due to the fantastic diagrams (eg, slide 23):
https://mariadb.org/wp-content/uploads/2017/11/2017-11-Memory-barriers.pdf
That said, finding a nice diagram and understanding the implications of actually using it is different! I hope to properly review the service-cores patches next week.
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
of atomic-implementation, as it could lead to racey bugs that are likely extremely difficult to track down. (The recent race-on-exit has proven the difficulty in reproducing, and that's with an atomics model I'm quite familiar with).
Let me be very clear: I don't wish to block a C11 atomic implementation, but I'd like to discuss how we (DPDK community) can best port-to and maintain a complex multi-threaded service library with best-in-class performance for the workload.
To put some discussions/solutions on the table:
- Shared Maintainership of a component?
Split in functionality and C11 atomics implementation Obviously there would be collaboration required in such a case.
- Maybe shared maintainership is too much?
A gentlemans/womans agreement of "helping out" with C11 atomics
debug is enough?
Quoted 6 lines
Quoted 1 line
I think shared maintainer ship could be too much as there are many
changes.
Quoted 1 line
But, I and other engineers from Arm (I would include Arm ecosystem as
well)
Quoted 2 lines
can definitely help out on debug and reviews involving C11 APIs. (I see Thomas's reply on this topic).
Thanks for the offer - as above, ball on my side, I'll go review.
<snip>
Quoted 10 lines · Phil Yang
DPDK provides generic
rte_atomicAPIs to do several atomic operations. These APIs are using the deprecated__syncbuilt-ins and enforce full memory barriers on aarch64. However, full barriers are not necessary in many use cases. In order to address such use cases, C language offers C11 atomic APIs. The C11 atomic APIs provide finer memory barrier control by making use of the memory ordering parameter provided by the user. Various patches submitted in the past [2] and the patches in this series indicate significant performance gains on multiple aarch64 CPUs and no performance loss on x86.
Quoted 4 lines
But the existing
rte_atomicAPI implementations cannot be changed as the APIs do not take the memory ordering parameter. The only choice available is replacing the usage of therte_atomicAPIs with C11 atomic APIs. In order to make this change, the following steps are proposed:
First of all I must say I much support the effort of introducing C11 atomics or something equivalent into DPDK, across the board.
What's being proposed however, is not to use C11 atomics, but rather the GCC built-ins designed to allow an efficient C11 atomics implementation. The C11 atomic API is found in <stdatomic.h>. Also, the <rte_atomic.h> API is not using __sync. It doesn't dictate any particular implementation at all.
I don't think directly accessing GCC built-ins across the whole DPDK code base sounds like a good idea at all.
Beyond just being plain ugly, and setting a bad precedence, using built-ins directly also effectively prevents API extensions. Although C11 is new and shiny, I'm sure there will come a day when we want to extend this API, to make it easier for consumers and avoid code duplication. Some parts of the DPDK code base already today define their own __atomic_* functions. Bad idea to use the "__*" namespace, especially in a way that has a real risk of future collisions. It's also confusing for anyone reading the code, since they are led to believe it's a GCC built-in.
Direct calls to GCC built-ins also prevents the use of any other implementation than the GCC built-ins, if some ISA or ISA implementation would benefit from this. This should be avoided of course, so it's just a minor objection.
I think the right way to go about this is not to deprecate <rte_atomic.h>. Rather, <rte_atomic.h> should be reshaped into something that closely maps to the GCC built-ins for C11 (which seems more convenient than real C11 atomics). The parts of <rte_atomic.h> that doesn't fit the new model, should be deprecated.
To summarize, I'm not in favor of deprecating <rte_atomic.h>. If we should deprecate anything, it's directly accessing compiler built-ins.
Quoted 51 lines
[1] deprecate
rte_atomicAPIs so that future patches do not userte_atomicAPIs (a script is added to flag the usages).
[2] refactor the code that usesrte_atomicAPIs to use c11 atomic APIs.This patchset contains:
1) the checkpatch script changes to flagrte_atomicAPI usage in patches.
2) changes to programmer guide describing writing efficient code for aarch64.
3) changes to various libraries to make use of c11 atomic APIs.We are planning to replicate this idea across all the other libraries, drivers, examples, test applications. In the next phase, we will add changes to the mbuf, the EAL interrupts and the event timer adapter libraries.
v3:
add libatomic dependency for 32-bit clangv2:
1. fix Clang '-Wincompatible-pointer-types' WARNING.
2. fix typos.Honnappa Nagarahalli (2):
service: avoid race condition for MT unsafe service
service: identify service running on another core correctlyPhil Yang (10):
doc: add generic atomic deprecation section
devtools: prevent use of rte atomic APIs in future patches eal/build: add libatomic dependency for 32-bit clang
build: remove redundant code
vhost: optimize broadcast rarp sync with c11 atomic
ipsec: optimize with c11 atomic for sa outbound sqn update
service: remove rte prefix from static functions
service: remove redundant code
service: optimize with c11 one-way barrier
service: relax barriers with C11 atomic operationsdevtools/checkpatches.sh | 9 ++ doc/guides/prog_guide/writing_efficient_code.rst | 60 +++++++- drivers/event/octeontx/meson.build | 5 - drivers/event/octeontx2/meson.build | 5 - drivers/event/opdl/meson.build | 5 - lib/librte_eal/common/rte_service.c | 175 ++++++++++++----------- lib/librte_eal/meson.build | 6 + lib/librte_ipsec/ipsec_sqn.h | 3 +- lib/librte_ipsec/sa.h | 2 +- lib/librte_rcu/meson.build | 5 - lib/librte_vhost/vhost.h | 2 +- lib/librte_vhost/vhost_user.c | 7 +- lib/librte_vhost/virtio_net.c | 16 ++- 13 files changed, 181 insertions(+), 119 deletions(-)
v4 cover generic rte atomic APIs deprecate proposal
In reply to Phil YangDPDK provides generic rte_atomic APIs to do several atomic operations. These APIs are using the deprecated __sync built-ins and enforce full memory barriers on aarch64. However, full barriers are not necessary in many use cases. In order to address such use cases, C language offers C11 atomic APIs. The C11 atomic APIs provide finer memory barrier control by making use of the memory ordering parameter provided by the user. Various patches submitted in the past [2] and the patches in this series indicate significant performance gains on multiple aarch64 CPUs and no performance loss on x86.
But the existing rte_atomic API implementations cannot be changed as the APIs do not take the memory ordering parameter. The only choice available is replacing the usage of the rte_atomic APIs with C11 atomic APIs. In order to make this change, the following steps are proposed:
[1] deprecate rte_atomic APIs so that future patches do not use rte_atomic APIs (a script is added to flag the usages).
[2] refactor the code that uses rte_atomic APIs to use c11 atomic APIs.
This patchset contains:
1) changes to programmer guide describing writing efficient code for aarch64.
2) the checkpatch script changes to flag rte_atomicNN_xxx API usage in patches.
3) wraps up compiler __atomic built-ins with explicit memory ordering parameter.
v4:
1. add reader-writer concurrency case describing.
2. claim maintainership of c11 atomics code for each platforms.
3. flag rte_atomicNN_xxx in new patches for modules that have been converted to c11 style.
4. flag __sync_xxx built-ins in new patches.
5. wraps up compiler atomic built-ins
6. move the changes of libraries which make use of c11 atomic APIs out of this patchset.
v3:
add libatomic dependency for 32-bit clang
v2:
1. fix Clang '-Wincompatible-pointer-types' WARNING.
2. fix typos.
Phil Yang (4):
doc: add generic atomic deprecation section
maintainers: claim maintainers of c11 atomics code
devtools: prevent use of rte atomic APIs in future patches eal/atomic: add wrapper for c11 atomics
MAINTAINERS | 4 + devtools/checkpatches.sh | 23 ++++ doc/guides/prog_guide/writing_efficient_code.rst | 139 ++++++++++++++++++++++- lib/librte_eal/include/generic/rte_atomic_c11.h | 139 +++++++++++++++++++++++ lib/librte_eal/include/meson.build | 1 + 5 files changed, 305 insertions(+), 1 deletion(-) create mode 100644 lib/librte_eal/include/generic/rte_atomic_c11.h
Signature · 1 line
2.7.4
PATCH v4 1/4 doc: add generic atomic deprecation section
Add deprecating the generic rte_atomic_xx APIs to c11 atomic built-ins guide and examples.
--- doc/guides/prog_guide/writing_efficient_code.rst | 139 ++++++++++++++++++++++- 1 file changed, 138 insertions(+), 1 deletion(-)
doc/guides/prog_guide/writing_efficient_code.rst +138 -1
@@ -167,7 +167,13 @@ but with the added cost of lower throughput. Locks and Atomic Operations --------------------------- -Atomic operations imply a lock prefix before the instruction, +This section describes some key considerations when using locks and atomic +operations in the DPDK environment. + +Locks +~~~~~ + +On x86, atomic operations imply a lock prefix before the instruction, causing the processor's LOCK# signal to be asserted during execution of the following instruction. This has a big impact on performance in a multicore environment.
@@ -176,6 +182,137 @@ It can often be replaced by other solutions like per-lcore variables. Also, some locking techniques are more efficient than others. For instance, the Read-Copy-Update (RCU) algorithm can frequently replace simple rwlocks. +Atomic Operations: Use C11 Atomic Built-ins +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +DPDK `generic rte_atomic <https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_eal/common/include/generic/rte_atomic.h>`_ operations are +implemented by `__sync built-ins <https://gcc.gnu.org/onlinedocs/gcc/_005f_005fsync-Builtins.html>`_. +These __sync built-ins result in full barriers on aarch64, which are unnecessary +in many use cases. They can be replaced by `__atomic built-ins <https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html>`_ that +conform to the C11 memory model and provide finer memory order control. + +So replacing the rte_atomic operations with __atomic built-ins might improve +performance for aarch64 machines. `More details <https://www.dpdk.org/wp-content/uploads/sites/35/2019/10/StateofC11Code.pdf>`_. + +Some typical optimization cases are listed below: + +Atomicity +^^^^^^^^^ + +Some use cases require atomicity alone, the ordering of the memory operations +does not matter. For example the packets statistics in the `vhost <https://github.com/DPDK/dpdk/blob/v20.02/examples/vhost/main.c#L796>`_ example application. + +It just updates the number of transmitted packets, no subsequent logic depends +on these counters. So the RELAXED memory ordering is sufficient: + +.. code-block:: c + + static __rte_always_inline void + virtio_xmit(struct vhost_dev *dst_vdev, struct vhost_dev *src_vdev, + struct rte_mbuf *m) + { + ... + ... + if (enable_stats) { + __atomic_add_fetch(&dst_vdev->stats.rx_total_atomic, 1, __ATOMIC_RELAXED); + __atomic_add_fetch(&dst_vdev->stats.rx_atomic, ret, __ATOMIC_RELAXED); + ... + } + } + +One-way Barrier +^^^^^^^^^^^^^^^ + +Some use cases allow for memory reordering in one way while requiring memory +ordering in the other direction. + +For example, the memory operations before the `lock <https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_eal/common/include/generic/rte_spinlock.h#L66>`_ can move to the +critical section, but the memory operations in the critical section cannot move +above the lock. In this case, the full memory barrier in the CAS operation can +be replaced to ACQUIRE. On the other hand, the memory operations after the +`unlock <https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_eal/common/include/generic/rte_spinlock.h#L88>`_ can move to the critical section, but the memory operations in the +critical section cannot move below the unlock. So the full barrier in the STORE +operation can be replaced with RELEASE. + +Reader-Writer Concurrency +^^^^^^^^^^^^^^^^^^^^^^^^^ +Lock-free reader-writer concurrency is one of the common use cases in DPDK. + +The payload or the data that the writer wants to communicate to the reader, +can be written with RELAXED memory order. However, the guard variable should +be written with RELEASE memory order. This ensures that the store to guard +variable is observable only after the store to payload is observable. +Refer to `rte_hash insert <https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_hash/rte_cuckoo_hash.c#L737>`_ for an example. + +.. code-block:: c + + static inline int32_t + rte_hash_cuckoo_insert_mw(const struct rte_hash *h, + ... + int32_t *ret_val) + { + ... + ... + + /* Insert new entry if there is room in the primary + * bucket. + */ + for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) { + /* Check if slot is available */ + if (likely(prim_bkt->key_idx[i] == EMPTY_SLOT)) { + prim_bkt->sig_current[i] = sig; + /* Store to signature and key should not + * leak after the store to key_idx. i.e. + * key_idx is the guard variable for signature + * and key. + */ + __atomic_store_n(&prim_bkt->key_idx[i], + new_idx, + __ATOMIC_RELEASE); + break; + } + } + + ... + } + +Correspondingly, on the reader side, the guard variable should be read +with ACQUIRE memory order. The payload or the data the writer communicated, +can be read with RELAXED memory order. This ensures that, if the store to +guard variable is observable, the store to payload is also observable. Refer to `rte_hash lookup <https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_hash/rte_cuckoo_hash.c#L1215>`_ for an example. + +.. code-block:: c + + static inline int32_t + search_one_bucket_lf(const struct rte_hash *h, const void *key, uint16_t sig, + void **data, const struct rte_hash_bucket *bkt) + { + ... + + for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) { + .... + if (bkt->sig_current[i] == sig) { + key_idx = __atomic_load_n(&bkt->key_idx[i], + __ATOMIC_ACQUIRE); + if (key_idx != EMPTY_SLOT) { + k = (struct rte_hash_key *) ((char *)keys + + key_idx * h->key_entry_size); + + if (rte_hash_cmp_eq(key, k->key, h) == 0) { + if (data != NULL) { + *data = __atomic_load_n(&k->pdata, + __ATOMIC_ACQUIRE); + } + + /* + * Return index where key is stored, + * subtracting the first dummy index + */ + return key_idx - 1; + } + ... + } + Coding Considerations ---------------------
Signature · 1 line
2.7.4
PATCH v4 2/4 maintainers: claim maintainers of c11 atomics code
In reply to Phil YangAdd the maintainership of c11 atomics code.
--- MAINTAINERS | 4 ++++ 1 file changed, 4 insertions(+)
MAINTAINERS +4 -0
@@ -266,6 +266,10 @@ F: lib/librte_eal/include/rte_random.h F: lib/librte_eal/common/rte_random.c F: app/test/test_rand_perf.c +C11 Code Maintainer +M: Honnappa Nagarahalli <honnappa,nagarahalli.arm.com> +M: David Christensen <[email protected]> + ARM v7 M: Jan Viktorin <[email protected]> M: Ruifeng Wang <[email protected]>
Signature · 1 line
2.7.4
Re: [dpdk-dev] [PATCH v4 2/4] maintainers: claim maintainers of c11 atomics code
12/05/2020 10:03, Phil Yang:
MAINTAINERS F: lib/librte_eal/include/rte_random.h · line 270
Quoted 9 lines
Add the maintainership of c11 atomics code.
Signed-off-by Phil Yang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>---
MAINTAINERS +2
-0+C11 Code Maintainer +M: Honnappa Nagarahalli <honnappa,nagarahalli.arm.com>
Typo: comma in the email address.
That's what happens when you don't copy-paste...
Quoted 1 line
+M: David Christensen <[email protected]>
Re: [dpdk-dev] [PATCH v4 2/4] maintainers: claim maintainers of c11 atomics code
Quoted 18 lines
-----Original Message-----
From: Thomas Monjalon <[email protected]>
Sent: Monday, May 25, 2020 7:11 AM
To: Phil Yang <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Honnappa Nagarahalli <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; nd <[email protected]>
Subject: Re: [dpdk-dev] [PATCH v4 2/4] maintainers: claim maintainers of c11 atomics code12/05/2020 10:03, Phil Yang:
Quoted 2 lines
Add the maintainership of c11 atomics code.
Quoted 10 lines
MAINTAINERS F: lib/librte_eal/include/rte_random.h · line 270
Quoted 7 lines
Signed-off-by Phil Yang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>---
MAINTAINERS +2
-0+C11 Code Maintainer +M: Honnappa Nagarahalli <honnappa,nagarahalli.arm.com>Typo: comma in the email address.
That's what happens when you don't copy-paste...
Thanks. Yes, I should have been more careful.
I will update in the next version.
Thanks,
Phil
Quoted 4 lines
Quoted 2 lines
+M: David Christensen <[email protected]>
PATCH v4 3/4 devtools: prevent use of rte atomic APIs in future patches
In reply to Phil YangIn order to deprecate the rte_atomic APIs, prevent the patches from using rte_atomic APIs in the converted modules and compilers __sync built-ins in all modules.
The converted modules:lib/librte_distributorlib/librte_hashlib/librte_knilib/librte_lpmlib/librte_rculib/librte_ringlib/librte_stacklib/librte_vhostlib/librte_timerlib/librte_ipsecdrivers/event/octeontxdrivers/event/octeontx2drivers/event/opdldrivers/net/bnx2xdrivers/net/hinicdrivers/net/hns3drivers/net/memifdrivers/net/thunderxdrivers/net/virtio
examples/l2fwd-event
--- devtools/checkpatches.sh | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)
devtools/checkpatches.sh +23 -0
@@ -69,6 +69,29 @@ check_forbidden_additions() { # <patch> -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ "$1" || res=1 + # refrain from new additions of 16/32/64 bits rte_atomic_xxx() + # multiple folders and expressions are separated by spaces + awk -v FOLDERS="lib/librte_distributor lib/librte_hash lib/librte_kni + lib/librte_lpm lib/librte_rcu lib/librte_ring + lib/librte_stack lib/librte_vhost drivers/event/octeontx + drivers/event/octeontx2 drivers/event/opdl + drivers/net/bnx2x drivers/net/hinic drivers/net/hns3 + drivers/net/memif drivers/net/thunderx + drivers/net/virtio examples/l2fwd-event" \ + -v EXPRESSIONS="rte_atomic[0-9][0-9]_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of rte_atomicNN_xxx APIs not allowed, use rte_atomic_xxx APIs' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from using compiler __sync built-ins + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="__sync_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of __sync_xxx built-ins not allowed, use rte_atomic_xxx APIs' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + # svg figures must be included with wildcard extension # because of png conversion for pdf docs awk -v FOLDERS='doc' \
Signature · 1 line
2.7.4
PATCH v4 4/4 eal/atomic: add wrapper for c11 atomics
In reply to Phil YangWraps up compiler c11 atomic built-ins with explicit memory ordering parameter.
--- lib/librte_eal/include/generic/rte_atomic_c11.h | 139 ++++++++++++++++++++++++ lib/librte_eal/include/meson.build | 1 + 2 files changed, 140 insertions(+) create mode 100644 lib/librte_eal/include/generic/rte_atomic_c11.h
lib/librte_eal/include/generic/rte_atomic_c11.h new file +139 -0
@@ -0,0 +1,139 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2020 Arm Limited +*/ + +#ifndef _RTE_ATOMIC_C11_H_ +#define _RTE_ATOMIC_C11_H_ + +#include <rte_common.h> + +/** +* @file +* c11 atomic operations +* +* This file wraps up compiler (GCC) c11 atomic built-ins. +* https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html +*/ + +#define memory_order_relaxed __ATOMIC_RELAXED +#define memory_order_consume __ATOMIC_CONSUME +#define memory_order_acquire __ATOMIC_ACQUIRE +#define memory_order_release __ATOMIC_RELEASE +#define memory_order_acq_rel __ATOMIC_ACQ_REL +#define memory_order_seq_cst __ATOMIC_SEQ_CST + +/* Generic atomic load. +* It returns the contents of *PTR. +* +* The valid memory order variants are: +* memory_order_relaxed +* memory_order_consume +* memory_order_acquire +* memory_order_seq_cst +*/ +#define rte_atomic_load (PTR ,MO ) \ + (__extension__ ({ \ + typeof(PTR) _ptr = (PTR); \ + typeof(*_ptr) _ret; \ + __atomic_load(_ptr, &_ret, (MO)); \ + _ret; \ + })) + +/* Generic atomic store. +* It stores the value of VAL into *PTR. +* +* The valid memory order variants are: +* memory_order_relaxed +* memory_order_release +* memory_order_seq_cst +*/ +#define rte_atomic_store (PTR ,VAL ,MO ) \ + (__extension__ ({ \ + typeof(PTR) _ptr = (PTR); \ + typeof(*_ptr) _val = (VAL); \ + __atomic_store(_ptr, &_val, (MO)); \ + })) + +/* Generic atomic exchange. +* It stores the value of VAL into *PTR. +* It returns the original value of *PTR. +* +* The valid memory order variants are: +* memory_order_relaxed +* memory_order_acquire +* memory_order_release +* memory_order_acq_rel +* memory_order_seq_cst +*/ +#define rte_atomic_exchange (PTR ,VAL ,MO ) \ + (__extension__ ({ \ + typeof(PTR) _ptr = (PTR); \ + typeof(*_ptr) _val = (VAL); \ + typeof(*_ptr) _ret; \ + __atomic_exchange(_ptr, &_val, &_ret, (MO)); \ + _ret; \ + })) + +/* Generic atomic compare and exchange. +* It compares the contents of *PTR with the contents of *EXP. +* If equal, the operation is a read-modify-write operation that +* writes DES into *PTR. +* If they are not equal, the operation is a read and the current +* contents of *PTR are written into *EXP. +* +* The weak compare_exchange may fail spuriously and the strong +* variation will never fails spuriously. +* +* If DES is written into *PTR then true is returned and memory is +* affected according to the memory order specified by SUC_MO. +* There are no restrictions on what memory order can be used here. +* +* Otherwise, false is returned and memory is affected according to +* FAIL_MO. This memory order cannot be memory_order_release nor +* memory_order_acq_rel. It also cannot be a stronger order than that +* specified by SUC_MO. +*/ +#define rte_atomic_compare_exchange_weak (PTR ,EXP ,DES ,SUC_MO ,FAIL_MO ) \ + (__extension__ ({ \ + typeof(PTR) _ptr = (PTR); \ + typeof(*_ptr) _des = (DES); \ + __atomic_compare_exchange(_ptr, (EXP), &_des, 1, \ + (SUC_MO), (FAIL_MO)); \ + })) + +#define rte_atomic_compare_exchange_strong (PTR ,EXP ,DES ,SUC_MO ,FAIL_MO ) \ + (__extension__ ({ \ + typeof(PTR) _ptr = (PTR); \ + typeof(*_ptr) _des = (DES); \ + __atomic_compare_exchange(_ptr, (EXP), &_des, 0, \ + (SUC_MO), (FAIL_MO)); \ + })) + +#define rte_atomic_fetch_add (PTR ,VAL ,MO ) \ + __atomic_fetch_add((PTR), (VAL), (MO)) +#define rte_atomic_fetch_sub (PTR ,VAL ,MO ) \ + __atomic_fetch_sub((PTR), (VAL), (MO)) +#define rte_atomic_fetch_or (PTR ,VAL ,MO ) \ + __atomic_fetch_or((PTR), (VAL), (MO)) +#define rte_atomic_fetch_xor (PTR ,VAL ,MO ) \ + __atomic_fetch_xor((PTR), (VAL), (MO)) +#define rte_atomic_fetch_and (PTR ,VAL ,MO ) \ + __atomic_fetch_and((PTR), (VAL), (MO)) + +#define rte_atomic_add_fetch (PTR ,VAL ,MO ) \ + __atomic_add_fetch((PTR), (VAL), (MO)) +#define rte_atomic_sub_fetch (PTR ,VAL ,MO ) \ + __atomic_sub_fetch((PTR), (VAL), (MO)) +#define rte_atomic_or_fetch (PTR ,VAL ,MO ) \ + __atomic_or_fetch((PTR), (VAL), (MO)) +#define rte_atomic_xor_fetch (PTR ,VAL ,MO ) \ + __atomic_xor_fetch((PTR), (VAL), (MO)) +#define rte_atomic_and_fetch (PTR ,VAL ,MO ) \ + __atomic_and_fetch((PTR), (VAL), (MO)) + +/* Synchronization fence between threads based on +* the specified memory order. +*/ +#define rte_atomic_thread_fence (MO ) __atomic_thread_fence((MO)) + +#endif /* _RTE_ATOMIC_C11_H_ */
lib/librte_eal/include/meson.build +1 -0
@@ -51,6 +51,7 @@ headers += files( # special case install the generic headers, since they go in a subdir generic_headers = files( 'generic/rte_atomic.h', + 'generic/rte_atomic_c11.h', 'generic/rte_byteorder.h', 'generic/rte_cpuflags.h', 'generic/rte_cycles.h',
Signature · 1 line
2.7.4
Re: [dpdk-dev] [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
Quoted 34 lines
From: Phil Yang [mailto:[email protected]]
Sent: Tuesday, May 12, 2020 10:03 AMWraps up compiler c11 atomic built-ins with explicit memory ordering parameter.
Signed-off-by Phil Yang <[email protected]>--- lib/librte_eal/include/generic/rte_atomic_c11.h | 139 ++++++++++++++++++++++++ lib/librte_eal/include/meson.build | 1 + 2 files changed, 140 insertions(+) create mode 100644 lib/librte_eal/include/generic/rte_atomic_c11.hlib/librte_eal/include/generic/rte_atomic_c11.h new file +13
-0@@ -0,0 +1,139 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2020 Arm Limited +*/ + +#ifndef _RTE_ATOMIC_C11_H_ +#define _RTE_ATOMIC_C11_H_ + +#include <rte_common.h> + +/** +* @file +* c11 atomic operations +*
lib/librte_eal/include/generic/rte_atomic_c11.h line 23
Quoted 10 lines
lib/librte_eal/include/generic/rte_atomic_c11.h new file +10
-0+* This file wraps up compiler (GCC) c11 atomic built-ins. +* https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html +*/ + +#define memory_order_relaxed __ATOMIC_RELAXED +#define memory_order_consume __ATOMIC_CONSUME +#define memory_order_acquire __ATOMIC_ACQUIRE +#define memory_order_release __ATOMIC_RELEASE +#define memory_order_acq_rel __ATOMIC_ACQ_REL +#define memory_order_seq_cst __ATOMIC_SEQ_CST
Why redefine these instead of using the original names?
If we need to redefine them, they should be upper case and RTE_ prefixed.
Quoted 52 lines
lib/librte_eal/include/generic/rte_atomic_c11.h new file +52
-0+ +/* Generic atomic load. +* It returns the contents of *PTR. +* +* The valid memory order variants are: +* memory_order_relaxed +* memory_order_consume +* memory_order_acquire +* memory_order_seq_cst +*/ +#define rte_atomic_load (PTR ,MO ) \ + (__extension__ ({ \ + typeof(PTR) _ptr = (PTR); \ + typeof(*_ptr) _ret; \ + __atomic_load(_ptr, &_ret, (MO)); \ + _ret; \ + })) + +/* Generic atomic store. +* It stores the value of VAL into *PTR. +* +* The valid memory order variants are: +* memory_order_relaxed +* memory_order_release +* memory_order_seq_cst +*/ +#define rte_atomic_store (PTR ,VAL ,MO ) \ + (__extension__ ({ \ + typeof(PTR) _ptr = (PTR); \ + typeof(*_ptr) _val = (VAL); \ + __atomic_store(_ptr, &_val, (MO)); \ + })) + +/* Generic atomic exchange. +* It stores the value of VAL into *PTR. +* It returns the original value of *PTR. +* +* The valid memory order variants are: +* memory_order_relaxed +* memory_order_acquire +* memory_order_release +* memory_order_acq_rel +* memory_order_seq_cst +*/ +#define rte_atomic_exchange (PTR ,VAL ,MO ) \ + (__extension__ ({ \ + typeof(PTR) _ptr = (PTR); \ + typeof(*_ptr) _val = (VAL); \ + typeof(*_ptr) _ret; \ + __atomic_exchange(_ptr, &_val, &_ret, (MO)); \ + _ret; \ + }))
lib/librte_eal/include/generic/rte_atomic_c11.h line 85
Quoted 10 lines
lib/librte_eal/include/generic/rte_atomic_c11.h new file +10
-0+ +/* Generic atomic compare and exchange. +* It compares the contents of *PTR with the contents of *EXP. +* If equal, the operation is a read-modify-write operation that +* writes DES into *PTR. +* If they are not equal, the operation is a read and the current +* contents of *PTR are written into *EXP. +* +* The weak compare_exchange may fail spuriously and the strong +* variation will never fails spuriously.
"will never fails spuriously" -> "will never fail" / "never fails".
And I suggest that you elaborate what "fail" means here, i.e. what exactly can happen when it fails.
Quoted 70 lines
lib/librte_eal/include/generic/rte_atomic_c11.h new file +11
-0+* +* If DES is written into *PTR then true is returned and memory is +* affected according to the memory order specified by SUC_MO. +* There are no restrictions on what memory order can be used here. +* +* Otherwise, false is returned and memory is affected according to +* FAIL_MO. This memory order cannot be memory_order_release nor +* memory_order_acq_rel. It also cannot be a stronger order than that +* specified by SUC_MO. +*/ +#define rte_atomic_compare_exchange_weak(PTR, EXP, DES, SUC_MO,
FAIL_MO) \lib/librte_eal/include/generic/rte_atomic_c11.h new file +8
-0+ (__extension__ ({ \ +typeof (PTR) _ptr = (PTR ); \ + typeof(*_ptr) _des = (DES); \ +__atomic_compare_exchange (_ptr, (EXP ), &_des,1 , \ + (SUC_MO ), (FAIL_MO )); \ + })) + +#define rte_atomic_compare_exchange_strong(PTR, EXP, DES, SUC_MO,
FAIL_MO) \lib/librte_eal/include/generic/rte_atomic_c11.h new file +35
-0+ (__extension__ ({ \ +typeof (PTR) _ptr = (PTR ); \ + typeof(*_ptr) _des = (DES); \ +__atomic_compare_exchange (_ptr, (EXP ), &_des,0 , \ + (SUC_MO ), (FAIL_MO )); \ + })) + +#define rte_atomic_fetch_add (PTR ,VAL ,MO ) \ + __atomic_fetch_add((PTR), (VAL), (MO)) +#define rte_atomic_fetch_sub (PTR ,VAL ,MO ) \ + __atomic_fetch_sub((PTR), (VAL), (MO)) +#define rte_atomic_fetch_or (PTR ,VAL ,MO ) \ + __atomic_fetch_or((PTR), (VAL), (MO)) +#define rte_atomic_fetch_xor (PTR ,VAL ,MO ) \ + __atomic_fetch_xor((PTR), (VAL), (MO)) +#define rte_atomic_fetch_and (PTR ,VAL ,MO ) \ + __atomic_fetch_and((PTR), (VAL), (MO)) + +#define rte_atomic_add_fetch (PTR ,VAL ,MO ) \ + __atomic_add_fetch((PTR), (VAL), (MO)) +#define rte_atomic_sub_fetch (PTR ,VAL ,MO ) \ + __atomic_sub_fetch((PTR), (VAL), (MO)) +#define rte_atomic_or_fetch (PTR ,VAL ,MO ) \ + __atomic_or_fetch((PTR), (VAL), (MO)) +#define rte_atomic_xor_fetch (PTR ,VAL ,MO ) \ + __atomic_xor_fetch((PTR), (VAL), (MO)) +#define rte_atomic_and_fetch (PTR ,VAL ,MO ) \ + __atomic_and_fetch((PTR), (VAL), (MO)) + +/* Synchronization fence between threads based on +* the specified memory order. +*/ +#define rte_atomic_thread_fence (MO ) __atomic_thread_fence((MO)) + +#endif /* _RTE_ATOMIC_C11_H_ */ lib/librte_eal/include/meson.build +1
-1@@ -51,6 +51,7 @@ headers += files( # special case install the generic headers, since they go in a subdir generic_headers = files( 'generic/rte_atomic.h', + 'generic/rte_atomic_c11.h', 'generic/rte_byteorder.h', 'generic/rte_cpuflags.h', 'generic/rte_cycles.h', --
Quoted 2 lines
2.7.4
Thumbs up for the good function documentation. :-)
Med venlig hilsen / kind regards
- Morten Brørup
Re: [dpdk-dev] [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
Quoted 61 lines
-----Original Message-----
From: Morten Brørup <[email protected]>
Sent: Tuesday, May 12, 2020 7:18 PM
To: Phil Yang <[email protected]>; [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Honnappa Nagarahalli <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; nd <[email protected]>; David Christensen <[email protected]>; nd <[email protected]>
Subject: RE: [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomicsQuoted 34 lines
From: Phil Yang [mailto:[email protected]]
Sent: Tuesday, May 12, 2020 10:03 AMWraps up compiler c11 atomic built-ins with explicit memory ordering parameter.
Signed-off-by Phil Yang <[email protected]>--- lib/librte_eal/include/generic/rte_atomic_c11.h | 139 ++++++++++++++++++++++++ lib/librte_eal/include/meson.build | 1 + 2 files changed, 140 insertions(+) create mode 100644 lib/librte_eal/include/generic/rte_atomic_c11.hlib/librte_eal/include/generic/rte_atomic_c11.h new file +13
-0@@ -0,0 +1,139 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2020 Arm Limited +*/ + +#ifndef _RTE_ATOMIC_C11_H_ +#define _RTE_ATOMIC_C11_H_ + +#include <rte_common.h> + +/** +* @file +* c11 atomic operations +* lib/librte_eal/include/generic/rte_atomic_c11.h line 23
Quoted 10 lines
lib/librte_eal/include/generic/rte_atomic_c11.h new file +10
-0+* This file wraps up compiler (GCC) c11 atomic built-ins. +* https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html +*/ + +#define memory_order_relaxed __ATOMIC_RELAXED +#define memory_order_consume __ATOMIC_CONSUME +#define memory_order_acquire __ATOMIC_ACQUIRE +#define memory_order_release __ATOMIC_RELEASE +#define memory_order_acq_rel __ATOMIC_ACQ_REL +#define memory_order_seq_cst __ATOMIC_SEQ_CSTWhy redefine these instead of using the original names?
Quoted 1 line
If we need to redefine them, they should be upper case and
RTE_prefixed.
Agreed, we don't need to redefine them. I was trying to align with the stdatomic library. I will remove them in the next version.
Quoted 57 lines
Quoted 57 lines
lib/librte_eal/include/generic/rte_atomic_c11.h new file +56
-0+ +/* Generic atomic load. +* It returns the contents of *PTR. +* +* The valid memory order variants are: +* memory_order_relaxed +* memory_order_consume +* memory_order_acquire +* memory_order_seq_cst +*/ +#define rte_atomic_load (PTR ,MO ) \ + (__extension__ ({ \ + typeof(PTR) _ptr = (PTR); \ + typeof(*_ptr) _ret; \ + __atomic_load(_ptr, &_ret, (MO)); \ + _ret; \ + })) + +/* Generic atomic store. +* It stores the value of VAL into *PTR. +* +* The valid memory order variants are: +* memory_order_relaxed +* memory_order_release +* memory_order_seq_cst +*/ +#define rte_atomic_store (PTR ,VAL ,MO ) \ + (__extension__ ({ \ + typeof(PTR) _ptr = (PTR); \ + typeof(*_ptr) _val = (VAL); \ + __atomic_store(_ptr, &_val, (MO)); \ + })) + +/* Generic atomic exchange. +* It stores the value of VAL into *PTR. +* It returns the original value of *PTR. +* +* The valid memory order variants are: +* memory_order_relaxed +* memory_order_acquire +* memory_order_release +* memory_order_acq_rel +* memory_order_seq_cst +*/ +#define rte_atomic_exchange (PTR ,VAL ,MO ) \ + (__extension__ ({ \ + typeof(PTR) _ptr = (PTR); \ + typeof(*_ptr) _val = (VAL); \ + typeof(*_ptr) _ret; \ + __atomic_exchange(_ptr, &_val, &_ret, (MO)); \ + _ret; \ + })) + +/* Generic atomic compare and exchange. +* It compares the contents of *PTR with the contents of *EXP. +* If equal, the operation is a read-modify-write operation that
Quoted 8 lines
lib/librte_eal/include/generic/rte_atomic_c11.h line 85
Quoted 6 lines
lib/librte_eal/include/generic/rte_atomic_c11.h new file +6
-0+* writes DES into *PTR. +* If they are not equal, the operation is a read and the current +* contents of *PTR are written into *EXP. +* +* The weak compare_exchange may fail spuriously and the strong +* variation will never fails spuriously. "will never fails spuriously" -> "will never fail" / "never fails".
Thanks, I will fix it in the next version.
Quoted 3 lines
And I suggest that you elaborate what "fail" means here, i.e. what exactly can happen when it fails.
Yes. That would be better. I will update it in the new version.
Fail spuriously means the compare exchange operation acts as *PTR != *EXP and return false even if they are equal.
Quoted 69 lines
Quoted 13 lines
lib/librte_eal/include/generic/rte_atomic_c11.h new file +11
-0+* +* If DES is written into *PTR then true is returned and memory is +* affected according to the memory order specified by SUC_MO. +* There are no restrictions on what memory order can be used here. +* +* Otherwise, false is returned and memory is affected according to +* FAIL_MO. This memory order cannot be memory_order_release nor +* memory_order_acq_rel. It also cannot be a stronger order than that +* specified by SUC_MO. +*/ +#define rte_atomic_compare_exchange_weak(PTR, EXP, DES, SUC_MO,
FAIL_MO) \lib/librte_eal/include/generic/rte_atomic_c11.h
Quoted 5 lines
lib/librte_eal/include/generic/rte_atomic_c11.h new file +5
-0+ (__extension__ ({ \ + typeof(PTR) _ptr = (PTR); \ + typeof(*_ptr) _des = (DES); \ + __atomic_compare_exchange(_ptr, (EXP), &_des, 1, \ + (SUC_MO), (FAIL_MO));\lib/librte_eal/include/generic/rte_atomic_c11.h
Quoted 9 lines
lib/librte_eal/include/generic/rte_atomic_c11.h new file +3
-0+ })) + +#define rte_atomic_compare_exchange_strong(PTR, EXP, DES, SUC_MO,
FAIL_MO) \lib/librte_eal/include/generic/rte_atomic_c11.h new file +5
-0+ (__extension__ ({ \ + typeof(PTR) _ptr = (PTR); \ + typeof(*_ptr) _des = (DES); \ + __atomic_compare_exchange(_ptr, (EXP), &_des, 0, \ + (SUC_MO), (FAIL_MO));\Quoted 40 lines
lib/librte_eal/include/generic/rte_atomic_c11.h new file +30
-0+ })) + +#define rte_atomic_fetch_add (PTR ,VAL ,MO ) \ + __atomic_fetch_add((PTR), (VAL), (MO)) +#define rte_atomic_fetch_sub (PTR ,VAL ,MO ) \ + __atomic_fetch_sub((PTR), (VAL), (MO)) +#define rte_atomic_fetch_or (PTR ,VAL ,MO ) \ + __atomic_fetch_or((PTR), (VAL), (MO)) +#define rte_atomic_fetch_xor (PTR ,VAL ,MO ) \ + __atomic_fetch_xor((PTR), (VAL), (MO)) +#define rte_atomic_fetch_and (PTR ,VAL ,MO ) \ + __atomic_fetch_and((PTR), (VAL), (MO)) + +#define rte_atomic_add_fetch (PTR ,VAL ,MO ) \ + __atomic_add_fetch((PTR), (VAL), (MO)) +#define rte_atomic_sub_fetch (PTR ,VAL ,MO ) \ + __atomic_sub_fetch((PTR), (VAL), (MO)) +#define rte_atomic_or_fetch (PTR ,VAL ,MO ) \ + __atomic_or_fetch((PTR), (VAL), (MO)) +#define rte_atomic_xor_fetch (PTR ,VAL ,MO ) \ + __atomic_xor_fetch((PTR), (VAL), (MO)) +#define rte_atomic_and_fetch (PTR ,VAL ,MO ) \ + __atomic_and_fetch((PTR), (VAL), (MO)) + +/* Synchronization fence between threads based on +* the specified memory order. +*/ +#define rte_atomic_thread_fence (MO ) __atomic_thread_fence((MO)) + +#endif /* _RTE_ATOMIC_C11_H_ */ lib/librte_eal/include/meson.build +1
-0@@ -51,6 +51,7 @@ headers += files( # special case install the generic headers, since they go in a subdir generic_headers = files( 'generic/rte_atomic.h', + 'generic/rte_atomic_c11.h',
Quoted 8 lines
Quoted 6 lines
lib/librte_eal/include/meson.build +0
-1'generic/rte_byteorder.h', 'generic/rte_cpuflags.h', 'generic/rte_cycles.h', --2.7.4
Thumbs up for the good function documentation. :-)
Thank you for your comments.
Thanks,
Phil
Quoted 6 lines
Med venlig hilsen / kind regards
- Morten Brørup
Re: [dpdk-dev] [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
<snip>
Quoted 51 lines
Quoted 35 lines
Subject: RE: [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
Quoted 33 lines
From: Phil Yang [mailto:[email protected]]
Sent: Tuesday, May 12, 2020 10:03 AMWraps up compiler c11 atomic built-ins with explicit memory ordering parameter.
Signed-off-by Phil Yang <[email protected]>--- lib/librte_eal/include/generic/rte_atomic_c11.h | 139 ++++++++++++++++++++++++ lib/librte_eal/include/meson.build | 1 + 2 files changed, 140 insertions(+) create mode 100644 lib/librte_eal/include/generic/rte_atomic_c11.hlib/librte_eal/include/generic/rte_atomic_c11.h new file +12
-0@@ -0,0 +1,139 @@ +/* SPDX-License-Identifier: BSD-3-Clause +* Copyright(c) 2020 Arm Limited +*/ + +#ifndef _RTE_ATOMIC_C11_H_ +#define _RTE_ATOMIC_C11_H_ + +#include <rte_common.h> + +/** +* @file +* c11 atomic operations Quoted 8 lines
lib/librte_eal/include/generic/rte_atomic_c11.h
Quoted 8 lines
lib/librte_eal/include/generic/rte_atomic_c11.h new file +8
-0+* +* This file wraps up compiler (GCC) c11 atomic built-ins. +* +https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html +*/ + +#define memory_order_relaxed __ATOMIC_RELAXED #define +memory_order_consume __ATOMIC_CONSUME #define
memory_order_acquireQuoted 1 line
lib/librte_eal/include/generic/rte_atomic_c11.h line 21
Quoted 1 line
lib/librte_eal/include/generic/rte_atomic_c11.h new file +1
-0+__ATOMIC_ACQUIRE #define memory_order_release
__ATOMIC_RELEASEQuoted 5 lines
lib/librte_eal/include/generic/rte_atomic_c11.h line 23
Quoted 2 lines
lib/librte_eal/include/generic/rte_atomic_c11.h new file +2
-0+#define memory_order_acq_rel __ATOMIC_ACQ_REL #define +memory_order_seq_cst __ATOMIC_SEQ_CSTWhy redefine these instead of using the original names?
Quoted 5 lines
Quoted 1 line
If we need to redefine them, they should be upper case and
RTE_prefixed.Agreed, we don't need to redefine them. I was trying to align with the stdatomic library.
I will remove them in the next version.
Agree, this will keep it inline with rte_atomic128_cmp_exchange API.
Quoted 160 lines
Quoted 55 lines
Quoted 55 lines
lib/librte_eal/include/generic/rte_atomic_c11.h new file +53
-0+ +/* Generic atomic load. +* It returns the contents of *PTR. +* +* The valid memory order variants are: +* memory_order_relaxed +* memory_order_consume +* memory_order_acquire +* memory_order_seq_cst +*/ +#define rte_atomic_load (PTR ,MO )\ +(__extension__ ({\ +typeof(PTR) _ptr = (PTR);\ +typeof(*_ptr) _ret;\ +__atomic_load(_ptr, &_ret, (MO));\ +_ret;\ +})) + +/* Generic atomic store. +* It stores the value of VAL into *PTR. +* +* The valid memory order variants are: +* memory_order_relaxed +* memory_order_release +* memory_order_seq_cst +*/ +#define rte_atomic_store (PTR ,VAL ,MO )\ (__extension__ ({\ +typeof(PTR) _ptr = (PTR);\ +typeof(*_ptr) _val = (VAL);\ +__atomic_store(_ptr, &_val, (MO));\ +})) + +/* Generic atomic exchange. +* It stores the value of VAL into *PTR. +* It returns the original value of *PTR. +* +* The valid memory order variants are: +* memory_order_relaxed +* memory_order_acquire +* memory_order_release +* memory_order_acq_rel +* memory_order_seq_cst +*/ +#define rte_atomic_exchange (PTR ,VAL ,MO )\ (__extension__ ({\ +typeof(PTR) _ptr = (PTR);\ +typeof(*_ptr) _val = (VAL);\ +typeof(*_ptr) _ret;\ +__atomic_exchange(_ptr, &_val, &_ret, (MO));\ _ret;\ +})) + +/* Generic atomic compare and exchange. +* It compares the contents of *PTR with the contents of *EXP. +* If equal, the operation is a read-modify-write operation that Quoted 8 lines
lib/librte_eal/include/generic/rte_atomic_c11.h line 85
Quoted 6 lines
lib/librte_eal/include/generic/rte_atomic_c11.h new file +6
-0+* writes DES into *PTR. +* If they are not equal, the operation is a read and the current +* contents of *PTR are written into *EXP. +* +* The weak compare_exchange may fail spuriously and the strong +* variation will never fails spuriously. "will never fails spuriously" -> "will never fail" / "never fails".
Thanks, I will fix it in the next version.
Quoted 3 lines
And I suggest that you elaborate what "fail" means here, i.e. what exactly can happen when it fails.
Yes. That would be better. I will update it in the new version. Fail spuriously means the compare exchange operation acts as *PTR != *EXP and return false even if they are equal.
Quoted 65 lines
Quoted 14 lines
lib/librte_eal/include/generic/rte_atomic_c11.h new file +12
-0+* +* If DES is written into *PTR then true is returned and memory is +* affected according to the memory order specified by SUC_MO. +* There are no restrictions on what memory order can be used here. +* +* Otherwise, false is returned and memory is affected according to +* FAIL_MO. This memory order cannot be memory_order_release nor +* memory_order_acq_rel. It also cannot be a stronger order than +that +* specified by SUC_MO. +*/ +#define rte_atomic_compare_exchange_weak(PTR, EXP, DES, SUC_MO,
FAIL_MO) \lib/librte_eal/include/generic/rte_atomic_c11.h
Quoted 5 lines
lib/librte_eal/include/generic/rte_atomic_c11.h new file +5
-0+(__extension__ ({ \ +typeof(PTR) _ptr = (PTR); \ +typeof(*_ptr) _des = (DES); \ +__atomic_compare_exchange(_ptr, (EXP), &_des, 1, \ + (SUC_MO), (FAIL_MO));\lib/librte_eal/include/generic/rte_atomic_c11.h
Quoted 9 lines
lib/librte_eal/include/generic/rte_atomic_c11.h new file +3
-0+})) + +#define rte_atomic_compare_exchange_strong(PTR, EXP, DES, SUC_MO,
FAIL_MO) \lib/librte_eal/include/generic/rte_atomic_c11.h new file +5
-0+(__extension__ ({ \ +typeof(PTR) _ptr = (PTR); \ +typeof(*_ptr) _des = (DES); \ +__atomic_compare_exchange(_ptr, (EXP), &_des, 0, \ + (SUC_MO), (FAIL_MO));\Quoted 35 lines
lib/librte_eal/include/generic/rte_atomic_c11.h new file +26
-0+})) + +#define rte_atomic_fetch_add(PTR, VAL, MO)\ +__atomic_fetch_add((PTR), (VAL), (MO)) #define +rte_atomic_fetch_sub(PTR, VAL, MO)\ __atomic_fetch_sub((PTR), +(VAL), (MO)) #define rte_atomic_fetch_or(PTR, VAL, MO)\ +__atomic_fetch_or((PTR), (VAL), (MO)) #define +rte_atomic_fetch_xor (PTR ,VAL ,MO )\ __atomic_fetch_xor((PTR), +(VAL), (MO)) #define rte_atomic_fetch_and(PTR, VAL, MO)\ +__atomic_fetch_and((PTR), (VAL), (MO)) + +#define rte_atomic_add_fetch (PTR ,VAL ,MO )\ +__atomic_add_fetch((PTR), (VAL), (MO)) #define +rte_atomic_sub_fetch(PTR, VAL, MO)\ __atomic_sub_fetch((PTR), +(VAL), (MO)) #define rte_atomic_or_fetch(PTR, VAL, MO)\ +__atomic_or_fetch((PTR), (VAL), (MO)) #define +rte_atomic_xor_fetch (PTR ,VAL ,MO )\ __atomic_xor_fetch((PTR), +(VAL), (MO)) #define rte_atomic_and_fetch(PTR, VAL, MO)\ +__atomic_and_fetch((PTR), (VAL), (MO)) + +/* Synchronization fence between threads based on +* the specified memory order. +*/ +#define rte_atomic_thread_fence (MO ) __atomic_thread_fence((MO)) + +#endif /* _RTE_ATOMIC_C11_H_ */ lib/librte_eal/include/meson.build +1
-0@@ -51,6 +51,7 @@ headers += files( # special case install the generic headers, since they go in a subdir generic_headers = files( 'generic/rte_atomic.h', +'generic/rte_atomic_c11.h',Quoted 8 lines
Quoted 6 lines
lib/librte_eal/include/meson.build +0
-1'generic/rte_byteorder.h', 'generic/rte_cpuflags.h', 'generic/rte_cycles.h', --2.7.4
Thumbs up for the good function documentation. :-)
Thank you for your comments.
Thanks,
PhilQuoted 6 lines
Med venlig hilsen / kind regards
- Morten Brørup
Re: [dpdk-dev] [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
In reply to Phil YangQuoted 3 lines · Phil Yang
parameter.
Signed-off-by Phil Yang <[email protected] <mailto:[email protected]>>
What is the purpose of having rte_atomic at all?
Is this level of indirection really helping?
Re: [dpdk-dev] [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
<snip>
Subject: Re: [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
On Tue, May 12, 2020 at 4:03 pm, Phil Yang <mailto:[email protected]> wrote:
parameter. Signed-off-by: Phil Yang <mailto:[email protected]>
What is the purpose of having rte_atomic at all?
Is this level of indirection really helping?
[HONNAPPA] (not sure why this email has html format, converted to text format)
I believe you meant, why not use the __atomic_xxx built-ins directly? The only reason for now is handling of __atomic_thread_fence(__ATOMIC_SEQ_CST) for x86. This is equivalent to rte_smp_mb which has an optimized implementation for x86. According to Konstantin, the compiler does not generate optimal code. Wrapping that built-in alone is going to be confusing.
The wrappers also allow us to have our own implementation using inline assembly for compilers versions that do not support C11 atomic built-ins. But, I do not know if there is a need to support those versions.
Re: [dpdk-dev] [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
Quoted 24 lines
From: Honnappa Nagarahalli [mailto:[email protected]]
Sent: Tuesday, May 12, 2020 9:24 PM<snip>
Subject: Re: [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
On Tue, May 12, 2020 at 4:03 pm, Phil Yang <mailto:[email protected]> wrote:
parameter. Signed-off-by: Phil Yang <mailto:[email protected]>
What is the purpose of having
rte_atomicat all?
Is this level of indirection really helping?
[HONNAPPA] (not sure why this email has html format, converted to text format)
I believe you meant, why not use the__atomic_xxxbuilt-ins directly? The only reason for now is handling of__atomic_thread_fence(__ATOMIC_SEQ_CST)for x86. This is equivalent torte_smp_mbwhich has an optimized implementation for x86. According to Konstantin, the compiler does not generate optimal code. Wrapping that built-in alone is going to be confusing.
Quoted 3 lines
The wrappers also allow us to have our own implementation using inline assembly for compilers versions that do not support C11 atomic built- ins. But, I do not know if there is a need to support those versions.
If I recall correctly, someone mentioned that one (or more) of the aging enterprise Linux distributions don't include a compiler with C11 atomics.
I think Stephen is onto something here...
It is silly to add wrappers like this, if the only purpose is to support compilers and distributions that don't properly support an official C standard which is nearly a decade old. The quality and quantity of the DPDK documentation for these functions (including examples, discussions on Stack Overflow, etc.) will be inferior to the documentation of the standard C11 atomics, which increases the probability of incorrect use.
And if some compiler generates code that is suboptimal for a user, then it should be the choice of the user to either accept it or use a better compiler. Using a suboptimal compiler will not only affect the user's DPDK applications, but all applications developed by the user. And if he accepts it for his other applications, he will also accept it for his DPDK applications.
We could introduce some sort of marker or standardized comment to indicate when functions only exist for backwards compatibility with ancient compilers and similar, with a reference to documentation describing why. And when the documented preconditions are no longer relevant, e.g. when those particular enterprise Linux distributions become obsolete, these functions become obsolete too, and should be removed. However, getting rid of obsolete cruft will break the ABI. In other words: Added cruft will never be removed again, so think twice before adding.
Med venlig hilsen / kind regards
- Morten Brørup
Re: [dpdk-dev] [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
<snip>
Quoted 19 lines
Quoted 19 lines
Subject: Re: [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
On Tue, May 12, 2020 at 4:03 pm, Phil Yang <mailto:[email protected]> wrote:
parameter. Signed-off-by: Phil Yang <mailto:[email protected]>
What is the purpose of having
rte_atomicat all?
Is this level of indirection really helping?
[HONNAPPA] (not sure why this email has html format, converted to text format)
I believe you meant, why not use the__atomic_xxxbuilt-ins directly? The only reason for now is handling of__atomic_thread_fence(__ATOMIC_SEQ_CST)for x86. This is equivalent torte_smp_mbwhich has an optimized implementation for x86. According to Konstantin, the compiler does not generate optimal code. Wrapping that built-in alone is going to be confusing.
Quoted 6 lines
Quoted 3 lines
The wrappers also allow us to have our own implementation using inline assembly for compilers versions that do not support C11 atomic built- ins. But, I do not know if there is a need to support those versions.
If I recall correctly, someone mentioned that one (or more) of the aging enterprise Linux distributions don't include a compiler with C11 atomics.
I searched through the mailing list yesterday and I could not find anyone mentioning about compilers not supporting C11 built-ins. However, the C11 atomic APIs (as defined in stdatomic.h) are supported in later versions of the compilers. So, using C11 built-ins gives us better coverage with older compilers (including the ones being used in Intel CI which were the oldest versions mentioned on the mailing list). IMO, we should not be worried about compilers that do not support C11.
Quoted 9 lines
I think Stephen is onto something here...
It is silly to add wrappers like this, if the only purpose is to support compilers and distributions that don't properly support an official C standard which is nearly a decade old. The quality and quantity of the DPDK documentation for these functions (including examples, discussions on Stack Overflow, etc.) will be inferior to the documentation of the standard C11 atomics, which increases the probability of incorrect use.
I agree. I do not want to add them for the sake of adding them. But, I do think that we need to solve the issues in DPDK (if they affect performance) which could be due to tools. As Konstantin suggested, we could do the wrappers only for the __atomic_thread_fence built-in. This will make life lot easier.
Quoted 21 lines
And if some compiler generates code that is suboptimal for a user, then it should be the choice of the user to either accept it or use a better compiler. Using a suboptimal compiler will not only affect the user's DPDK applications, but all applications developed by the user. And if he accepts it for his other applications, he will also accept it for his DPDK applications.
We could introduce some sort of marker or standardized comment to indicate when functions only exist for backwards compatibility with ancient compilers and similar, with a reference to documentation describing why. And when the documented preconditions are no longer relevant, e.g. when those particular enterprise Linux distributions become obsolete, these functions become obsolete too, and should be removed. However, getting rid of obsolete cruft will break the ABI. In other words: Added cruft will never be removed again, so think twice before adding.
Med venlig hilsen / kind regards
- Morten Brørup
Re: [dpdk-dev] [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
In reply to Morten BrørupQuoted 24 lines · Morten Brørup
Quoted 24 lines
From: Honnappa Nagarahalli [mailto:[email protected]]
Sent: Tuesday, May 12, 2020 9:24 PM<snip>
Subject: Re: [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
On Tue, May 12, 2020 at 4:03 pm, Phil Yang <mailto:[email protected]> wrote:
parameter. Signed-off-by: Phil Yang <mailto:[email protected]>
What is the purpose of having
rte_atomicat all?
Is this level of indirection really helping?
[HONNAPPA] (not sure why this email has html format, converted to text format)
I believe you meant, why not use the__atomic_xxxbuilt-ins directly? The only reason for now is handling of__atomic_thread_fence(__ATOMIC_SEQ_CST)for x86. This is equivalent torte_smp_mbwhich has an optimized implementation for x86. According to Konstantin, the compiler does not generate optimal code. Wrapping that built-in alone is going to be confusing.
Quoted 8 lines
Quoted 3 lines
The wrappers also allow us to have our own implementation using inline assembly for compilers versions that do not support C11 atomic built- ins. But, I do not know if there is a need to support those versions.
If I recall correctly, someone mentioned that one (or more) of the aging enterprise Linux distributions don't include a compiler with C11 atomics.
I think Stephen is onto something here...
It is silly to add wrappers like this, if the only purpose is to support compilers and distributions that don't properly support an official C standard which is nearly a decade old. The quality and quantity of the DPDK documentation for these functions (including examples, discussions on Stack Overflow, etc.) will be inferior to the documentation of the standard C11 atomics, which increases the probability of incorrect use.
What's being used in DPDK today, and what's being wrapped here, is not standard C11 atomics - it's a bunch of GCC built-ins. Nothing in the __ namespace is in the standard. It's reserved for the implementation (e.g. compiler).
Quoted 10 lines
And if some compiler generates code that is suboptimal for a user, then it should be the choice of the user to either accept it or use a better compiler. Using a suboptimal compiler will not only affect the user's DPDK applications, but all applications developed by the user. And if he accepts it for his other applications, he will also accept it for his DPDK applications.
We could introduce some sort of marker or standardized comment to indicate when functions only exist for backwards compatibility with ancient compilers and similar, with a reference to documentation describing why. And when the documented preconditions are no longer relevant, e.g. when those particular enterprise Linux distributions become obsolete, these functions become obsolete too, and should be removed. However, getting rid of obsolete cruft will break the ABI. In other words: Added cruft will never be removed again, so think twice before adding.
Med venlig hilsen / kind regards
- Morten Brørup
Re: [dpdk-dev] [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
<snip>
Quoted 37 lines
Quoted 21 lines
Quoted 21 lines
Subject: Re: [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
On Tue, May 12, 2020 at 4:03 pm, Phil Yang <mailto:[email protected]> wrote:
parameter. Signed-off-by: Phil Yang <mailto:[email protected]>
What is the purpose of having
rte_atomicat all?
Is this level of indirection really helping?
[HONNAPPA] (not sure why this email has html format, converted to text
format)
I believe you meant, why not use the__atomic_xxxbuilt-ins directly? The only reason for now is handling of__atomic_thread_fence(__ATOMIC_SEQ_CST)for x86. This is equivalent torte_smp_mbwhich has an optimized implementation for x86. According to Konstantin, the compiler does not generate optimal code. Wrapping that built-in alone is going to be confusing.Quoted 4 lines
Quoted 3 lines
The wrappers also allow us to have our own implementation using inline assembly for compilers versions that do not support C11 atomic built- ins. But, I do not know if there is a need to support those versions.
If I recall correctly, someone mentioned that one (or more) of the aging
enterprise Linux distributions don't include a compiler with C11 atomics.
Quoted 4 lines
I think Stephen is onto something here...
It is silly to add wrappers like this, if the only purpose is to support
compilers and distributions that don't properly support an official C standard which is nearly a decade old. The quality and quantity of the DPDK documentation for these functions (including examples, discussions on Stack Overflow, etc.) will be inferior to the documentation of the standard C11 atomics, which increases the probability of incorrect use.
Quoted 4 lines
What's being used in DPDK today, and what's being wrapped here, is not standard C11 atomics - it's a bunch of GCC built-ins. Nothing in the __ namespace is in the standard. It's reserved for the implementation (e.g. compiler).
I have tried to understand what it mean by 'built-ins', but I have not got a good answer. So, does it mean that the built-in function (same symbol and API interface) may not be available in another C compiler? IMO, this is what matters for DPDK. Currently, the same built-in functions are available in GCC and Clang.
Quoted 23 lines
Quoted 3 lines
And if some compiler generates code that is suboptimal for a user, then it
should be the choice of the user to either accept it or use a better compiler. Using a suboptimal compiler will not only affect the user's DPDK applications, but all applications developed by the user. And if he accepts it for his other applications, he will also accept it for his DPDK applications.
Quoted 2 lines
We could introduce some sort of marker or standardized comment to
indicate when functions only exist for backwards compatibility with ancient compilers and similar, with a reference to documentation describing why. And when the documented preconditions are no longer relevant, e.g. when those particular enterprise Linux distributions become obsolete, these functions become obsolete too, and should be removed. However, getting rid of obsolete cruft will break the ABI. In other words: Added cruft will never be removed again, so think twice before adding.
Quoted 7 lines
Med venlig hilsen / kind regards
- Morten Brørup
Re: [dpdk-dev] [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
Quoted 37 lines · Honnappa Nagarahalli
<snip>
Quoted 35 lines
Quoted 20 lines
Quoted 20 lines
Subject: Re: [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
On Tue, May 12, 2020 at 4:03 pm, Phil Yang <mailto:[email protected]> wrote:
parameter. Signed-off-by: Phil Yang <mailto:[email protected]>
What is the purpose of having
rte_atomicat all?
Is this level of indirection really helping?
[HONNAPPA] (not sure why this email has html format, converted to text
format)
I believe you meant, why not use the__atomic_xxxbuilt-ins directly? The only reason for now is handling of__atomic_thread_fence(__ATOMIC_SEQ_CST)for x86. This is equivalent torte_smp_mbwhich has an optimized implementation for x86. According to Konstantin, the compiler does not generate optimal code. Wrapping that built-in alone is going to be confusing.Quoted 4 lines
Quoted 3 lines
The wrappers also allow us to have our own implementation using inline assembly for compilers versions that do not support C11 atomic built- ins. But, I do not know if there is a need to support those versions.
If I recall correctly, someone mentioned that one (or more) of the aging
enterprise Linux distributions don't include a compiler with C11 atomics.
Quoted 3 lines
I think Stephen is onto something here...
It is silly to add wrappers like this, if the only purpose is to support
compilers and distributions that don't properly support an official C standard which is nearly a decade old. The quality and quantity of the DPDK documentation for these functions (including examples, discussions on Stack Overflow, etc.) will be inferior to the documentation of the standard C11 atomics, which increases the probability of incorrect use.
Quoted 6 lines
Quoted 4 lines
What's being used in DPDK today, and what's being wrapped here, is not standard C11 atomics - it's a bunch of GCC built-ins. Nothing in the __ namespace is in the standard. It's reserved for the implementation (e.g. compiler).
I have tried to understand what it mean by 'built-ins', but I have not got a good answer. So, does it mean that the built-in function (same symbol and API interface) may not be available in another C compiler? IMO, this is what matters for DPDK. Currently, the same built-in functions are available in GCC and Clang.
From what I understand, "built-ins" is GCC terminology for
non-standard, implementation-specific intrinsic functions, built into the compiler. They all reside in the __* namespace.
Since GCC is the industry standard, other compilers are likely to follow, including built-in functions.
Quoted 20 lines
Quoted 20 lines
Quoted 2 lines
And if some compiler generates code that is suboptimal for a user, then it
should be the choice of the user to either accept it or use a better compiler. Using a suboptimal compiler will not only affect the user's DPDK applications, but all applications developed by the user. And if he accepts it for his other applications, he will also accept it for his DPDK applications.
Quoted 1 line
We could introduce some sort of marker or standardized comment to
indicate when functions only exist for backwards compatibility with ancient compilers and similar, with a reference to documentation describing why. And when the documented preconditions are no longer relevant, e.g. when those particular enterprise Linux distributions become obsolete, these functions become obsolete too, and should be removed. However, getting rid of obsolete cruft will break the ABI. In other words: Added cruft will never be removed again, so think twice before adding.
Quoted 6 lines
Med venlig hilsen / kind regards
- Morten Brørup
Re: [dpdk-dev] [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
+ Added people from the related discussion regarding the ARM roadmap [https://mails.dpdk.org/archives/dev/2020-April/162580.html].
Quoted 69 lines
From: Mattias Rönnblom [mailto:[email protected]]
Sent: Wednesday, May 13, 2020 10:17 PMQuoted 3 lines · Honnappa Nagarahalli
<snip>
Quoted 1 line
Quoted 1 line
Quoted 1 line
Subject: Re: [PATCH v4 4/4] eal/atomic: add wrapper for c11
atomics
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
On Tue, May 12, 2020 at 4:03 pm, Phil Yang
<mailto:[email protected]>
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
wrote:
parameter. Signed-off-by: Phil Yang <mailto:[email protected]>
What is the purpose of having
rte_atomicat all?
Is this level of indirection really helping?
[HONNAPPA] (not sure why this email has html format, converted to text
format)
I believe you meant, why not use the__atomic_xxxbuilt-insdirectly?
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The only reason for now is handling of
__atomic_thread_fence(__ATOMIC_SEQ_CST)for x86. This isequivalent
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
to
rte_smp_mbwhich has an optimized implementation for x86. According to Konstantin, the compiler does not generate optimalcode.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Wrapping that built-in alone is going to be confusing.
The wrappers also allow us to have our own implementation using inline assembly for compilers versions that do not support C11
atomic
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
built- ins. But, I do not know if there is a need to support those
versions.
Quoted 1 line
Quoted 1 line
Quoted 1 line
If I recall correctly, someone mentioned that one (or more) of the
aging
Quoted 1 line
Quoted 1 line
enterprise Linux distributions don't include a compiler with C11
atomics.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
I think Stephen is onto something here...
It is silly to add wrappers like this, if the only purpose is to
support
Quoted 1 line
Quoted 1 line
compilers and distributions that don't properly support an official
C standard
Quoted 2 lines
Quoted 2 lines
which is nearly a decade old. The quality and quantity of the DPDK documentation for these functions (including examples, discussions
on Stack
Quoted 1 line
Quoted 1 line
Overflow, etc.) will be inferior to the documentation of the
standard C11
Quoted 4 lines
Quoted 4 lines
atomics, which increases the probability of incorrect use.
What's being used in DPDK today, and what's being wrapped here, is
not
Quoted 1 line
Quoted 1 line
standard C11 atomics - it's a bunch of GCC built-ins. Nothing in the
__
Quoted 1 line
Quoted 1 line
namespace is in the standard. It's reserved for the implementation
(e.g.
Quoted 2 lines
Quoted 1 line
compiler).
I have tried to understand what it mean by 'built-ins', but I have
not got a good answer. So, does it mean that the built-in function (same symbol and API interface) may not be available in another C compiler? IMO, this is what matters for DPDK.
Quoted 1 line
Currently, the same built-in functions are available in GCC and
Clang.
Quoted 8 lines
From what I understand, "built-ins" is GCC terminology for
non-standard, implementation-specific intrinsic functions, built into the compiler. They all reside in the __* namespace.Since GCC is the industry standard, other compilers are likely to follow, including built-in functions.
Timeline:
December 2011: The C11 standard was published [http://www.open-std.org/jtc1/sc22/wg14/www/standards.html].
March 2012: GCC 4.7 was released, introducing the __atomic built-ins [https://gcc.gnu.org/gcc-4.7/changes.html, https://www.gnu.org/software/gcc/gcc-4.7/].
March 2013: GCC 4.8 was released [https://www.gnu.org/software/gcc/gcc-4.8/].
April 2014: GCC 4.9 was released, introducing C11 atomics (incl. <stdatomic.h>) [https://gcc.gnu.org/gcc-4.9/changes.html, https://www.gnu.org/software/gcc/gcc-4.9/].
June 2014: RHEL7 was released [https://access.redhat.com/articles/3078]. (RHEL7 Beta was released in December 2013, which probably explains why the GA release doesn’t include GCC 4.9.)
May 2019 (i.e. one year ago): RHEL8 was released [https://access.redhat.com/articles/3078].
RHEL7 includes GCC 4.8 only [https://access.redhat.com/solutions/19458], and apparently RHEL7 has not been updated to GCC 4.9 with any of its minor releases.
Should the DPDK project be stuck on "industry standard" GCC atomics, unable to use the decade old "official standard" C11 atomics, only because we want to support a six year old enterprise Linux distribution? Red Hat released a new enterprise version a year ago... perhaps it's time for their customers to upgrade, if they want to use the latest and greatest version of DPDK.
Are all the other tools required for building DPDK (in the required versions) included in RHEL7, or do we require developers to install/upgrade any other tools anyway? If so, why not also GCC? DPDK can be used in a cross compilation environment, so we are not requiring RHEL7 users to replace their GCC 4.7 default compiler.
Furthermore, the DPDK Documentation specifies GCC 4.9+ as a system requirement [https://doc.dpdk.org/guides/linux_gsg/sys_reqs.html#compilation-of-the-dpdk]. If we are stuck on GCC 4.8, the documentation should be updated.
Quoted 23 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
And if some compiler generates code that is suboptimal for a user,
then it
Quoted 1 line
Quoted 1 line
should be the choice of the user to either accept it or use a better
compiler.
Quoted 1 line
Quoted 1 line
Using a suboptimal compiler will not only affect the user's DPDK
applications,
Quoted 1 line
Quoted 1 line
but all applications developed by the user. And if he accepts it for
his other
Quoted 3 lines
Quoted 3 lines
applications, he will also accept it for his DPDK applications.
Quoted 1 line
We could introduce some sort of marker or standardized comment to
indicate when functions only exist for backwards compatibility with
ancient
Quoted 1 line
Quoted 1 line
compilers and similar, with a reference to documentation describing
why. And
Quoted 1 line
Quoted 1 line
when the documented preconditions are no longer relevant, e.g. when
those
Quoted 1 line
Quoted 1 line
particular enterprise Linux distributions become obsolete, these
functions
Quoted 2 lines
Quoted 2 lines
become obsolete too, and should be removed. However, getting rid of obsolete cruft will break the ABI. In other words: Added cruft will
never be
Quoted 1 line
Quoted 1 line
removed again, so think twice before adding.
Re: [dpdk-dev] [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
Quoted 88 lines · Morten Brørup
+ Added people from the related discussion regarding the ARM roadmap [https://protect2.fireeye.com/v1/url?k=10efdd7b-4e4f1ed2-10ef9de0-86959e472243-b772fef31e4ae6af&q=1&e=e3b0051e-bb23-4a30-84c7-7e5e80f83325&u=https%3A%2F%2Fmails.dpdk.org%2Farchives%2Fdev%2F2020-April%2F162580.html].
Quoted 68 lines
From: Mattias Rönnblom [mailto:[email protected]]
Sent: Wednesday, May 13, 2020 10:17 PMQuoted 3 lines · Honnappa Nagarahalli
<snip>
Quoted 1 line
Quoted 1 line
Quoted 1 line
Subject: Re: [PATCH v4 4/4] eal/atomic: add wrapper for c11
atomics
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Tue, May 12, 2020 at 4:03 pm, Phil Yang
<mailto:[email protected]>
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
wrote:
parameter. Signed-off-by: Phil Yang <mailto:[email protected]>
What is the purpose of having
rte_atomicat all?
Is this level of indirection really helping?
[HONNAPPA] (not sure why this email has html format, converted to text
format)
I believe you meant, why not use the__atomic_xxxbuilt-insdirectly?
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The only reason for now is handling of
__atomic_thread_fence(__ATOMIC_SEQ_CST)for x86. This isequivalent
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
to
rte_smp_mbwhich has an optimized implementation for x86. According to Konstantin, the compiler does not generate optimalcode.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Wrapping that built-in alone is going to be confusing.
The wrappers also allow us to have our own implementation using inline assembly for compilers versions that do not support C11
atomic
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
built- ins. But, I do not know if there is a need to support those
versions.
Quoted 1 line
Quoted 1 line
Quoted 1 line
If I recall correctly, someone mentioned that one (or more) of the
aging
Quoted 1 line
Quoted 1 line
enterprise Linux distributions don't include a compiler with C11
atomics.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
I think Stephen is onto something here...
It is silly to add wrappers like this, if the only purpose is to
support
Quoted 1 line
Quoted 1 line
compilers and distributions that don't properly support an official
C standard
Quoted 2 lines
Quoted 2 lines
which is nearly a decade old. The quality and quantity of the DPDK documentation for these functions (including examples, discussions
on Stack
Quoted 1 line
Quoted 1 line
Overflow, etc.) will be inferior to the documentation of the
standard C11
Quoted 4 lines
Quoted 4 lines
atomics, which increases the probability of incorrect use.
What's being used in DPDK today, and what's being wrapped here, is
not
Quoted 1 line
Quoted 1 line
standard C11 atomics - it's a bunch of GCC built-ins. Nothing in the
__
Quoted 1 line
Quoted 1 line
namespace is in the standard. It's reserved for the implementation
(e.g.
Quoted 2 lines
Quoted 1 line
compiler).
I have tried to understand what it mean by 'built-ins', but I have
not got a good answer. So, does it mean that the built-in function (same symbol and API interface) may not be available in another C compiler? IMO, this is what matters for DPDK.
Quoted 1 line
Currently, the same built-in functions are available in GCC and
Clang.
Quoted 8 lines
From what I understand, "built-ins" is GCC terminology for
non-standard, implementation-specific intrinsic functions, built into the compiler. They all reside in the __* namespace.Since GCC is the industry standard, other compilers are likely to follow, including built-in functions.
Timeline:
December 2011: The C11 standard was published [https://protect2.fireeye.com/v1/url?k=8e23b012-d08373bb-8e23f089-86959e472243-a2babe7075f8ac38&q=1&e=e3b0051e-bb23-4a30-84c7-7e5e80f83325&u=http%3A%2F%2Fwww.open-std.org%2Fjtc1%2Fsc22%2Fwg14%2Fwww%2Fstandards.html].
March 2012: GCC 4.7 was released, introducing the
__atomicbuilt-ins [https://gcc.gnu.org/gcc-4.7/changes.html, https://www.gnu.org/software/gcc/gcc-4.7/].March 2013: GCC 4.8 was released [https://www.gnu.org/software/gcc/gcc-4.8/].
April 2014: GCC 4.9 was released, introducing C11 atomics (incl. <
stdatomic.h>) [https://gcc.gnu.org/gcc-4.9/changes.html, https://www.gnu.org/software/gcc/gcc-4.9/].
Quoted 8 lines
June 2014: RHEL7 was released [https://access.redhat.com/articles/3078]. (RHEL7 Beta was released in December 2013, which probably explains why the GA release doesn’t include GCC 4.9.)
May 2019 (i.e. one year ago): RHEL8 was released [https://access.redhat.com/articles/3078].
RHEL7 includes GCC 4.8 only [https://access.redhat.com/solutions/19458], and apparently RHEL7 has not been updated to GCC 4.9 with any of its minor releases.
Should the DPDK project be stuck on "industry standard" GCC atomics, unable to use the decade old "official standard" C11 atomics, only because we want to support a six year old enterprise Linux distribution? Red Hat released a new enterprise version a year ago... perhaps it's time for their customers to upgrade, if they want to use the latest and greatest version of DPDK.
Just to be clear - I wasn't arguing for the direct use of GCC built-ins.
The GCC __atomic built-ins (called directly, or via a DPDK wrapper) do have some advantages over C11 atomics. One is that GCC supports 128-bit atomic operations, on certain architectures. <rte_atomic.h> already has a 128-bit compare-exchange. Also, since the GCC built-ins seem not to bother with architectures where atomics would be implemented by means of a lock, they are a little easier to use than <stdatomic.h>.
Quoted 28 lines
Are all the other tools required for building DPDK (in the required versions) included in RHEL7, or do we require developers to install/upgrade any other tools anyway? If so, why not also GCC? DPDK can be used in a cross compilation environment, so we are not requiring RHEL7 users to replace their GCC 4.7 default compiler.
Furthermore, the DPDK Documentation specifies GCC 4.9+ as a system requirement [https://protect2.fireeye.com/v1/url?k=339bad56-6d3b6eff-339bedcd-86959e472243-cb1bf3934c202e3f&q=1&e=e3b0051e-bb23-4a30-84c7-7e5e80f83325&u=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Flinux_gsg%2Fsys_reqs.html%23compilation-of-the-dpdk]. If we are stuck on GCC 4.8, the documentation should be updated.
Quoted 22 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
And if some compiler generates code that is suboptimal for a user,
then it
Quoted 1 line
Quoted 1 line
should be the choice of the user to either accept it or use a better
compiler.
Quoted 1 line
Quoted 1 line
Using a suboptimal compiler will not only affect the user's DPDK
applications,
Quoted 1 line
Quoted 1 line
but all applications developed by the user. And if he accepts it for
his other
Quoted 3 lines
Quoted 3 lines
applications, he will also accept it for his DPDK applications.
Quoted 1 line
We could introduce some sort of marker or standardized comment to
indicate when functions only exist for backwards compatibility with
ancient
Quoted 1 line
Quoted 1 line
compilers and similar, with a reference to documentation describing
why. And
Quoted 1 line
Quoted 1 line
when the documented preconditions are no longer relevant, e.g. when
those
Quoted 1 line
Quoted 1 line
particular enterprise Linux distributions become obsolete, these
functions
Quoted 2 lines
Quoted 2 lines
become obsolete too, and should be removed. However, getting rid of obsolete cruft will break the ABI. In other words: Added cruft will
never be
Quoted 1 line
Quoted 1 line
removed again, so think twice before adding.
Re: [dpdk-dev] [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
<snip>
Quoted 130 lines
Subject: Re: [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
Quoted 1 line · Morten Brørup
+ Added people from the related discussion regarding the ARM roadmap
[https://protect2.fireeye.com/v1/url?k=10efdd7b-4e4f1ed2-10ef9de0-
86959e472243-b772fef31e4ae6af&q=1&e=e3b0051e-bb23-4a30-84c7- 7e5e80f83325&u=https%3A%2F%2Fmails.dpdk.org%2Farchives%2Fdev%2F20 20-April%2F162580.html].Quoted 76 lines
Quoted 76 lines
From: Mattias Rönnblom [mailto:[email protected]]
Sent: Wednesday, May 13, 2020 10:17 PMQuoted 3 lines · Honnappa Nagarahalli
<snip>
Quoted 1 line
Quoted 1 line
Quoted 1 line
Subject: Re: [PATCH v4 4/4] eal/atomic: add wrapper for c11
atomics
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Tue, May 12, 2020 at 4:03 pm, Phil Yang
<mailto:[email protected]>
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
wrote:
parameter. Signed-off-by: Phil Yang <mailto:[email protected]>
What is the purpose of having
rte_atomicat all?
Is this level of indirection really helping?
[HONNAPPA] (not sure why this email has html format, converted to text
format)
I believe you meant, why not use the__atomic_xxxbuilt-insdirectly?
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
The only reason for now is handling of
__atomic_thread_fence(__ATOMIC_SEQ_CST)for x86. This isequivalent
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
to
rte_smp_mbwhich has an optimized implementation for x86. According to Konstantin, the compiler does not generate optimalcode.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Wrapping that built-in alone is going to be confusing.
The wrappers also allow us to have our own implementation using inline assembly for compilers versions that do not support C11
atomic
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
built- ins. But, I do not know if there is a need to support those
versions.
Quoted 1 line
Quoted 1 line
Quoted 1 line
If I recall correctly, someone mentioned that one (or more) of the
aging
Quoted 1 line
Quoted 1 line
enterprise Linux distributions don't include a compiler with C11
atomics.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
I think Stephen is onto something here...
It is silly to add wrappers like this, if the only purpose is to
support
Quoted 1 line
Quoted 1 line
compilers and distributions that don't properly support an official
C standard
Quoted 2 lines
Quoted 2 lines
which is nearly a decade old. The quality and quantity of the DPDK documentation for these functions (including examples, discussions
on Stack
Quoted 1 line
Quoted 1 line
Overflow, etc.) will be inferior to the documentation of the
standard C11
Quoted 4 lines
Quoted 4 lines
atomics, which increases the probability of incorrect use.
What's being used in DPDK today, and what's being wrapped here, is
not
Quoted 2 lines
Quoted 2 lines
standard C11 atomics - it's a bunch of GCC built-ins. Nothing in the
__
Quoted 1 line
Quoted 1 line
namespace is in the standard. It's reserved for the implementation
(e.g.
Quoted 2 lines
Quoted 1 line
compiler).
I have tried to understand what it mean by 'built-ins', but I have
not got a good answer. So, does it mean that the built-in function (same symbol and API interface) may not be available in another C compiler? IMO, this is what matters for DPDK.
Quoted 1 line
Currently, the same built-in functions are available in GCC and
Clang.
From what I understand, "built-ins" is GCC terminology for
non-standard, implementation-specific intrinsic functions, built into the compiler. They all reside in the __* namespace.Quoted 6 lines
Quoted 3 lines
Since GCC is the industry standard, other compilers are likely to follow, including built-in functions.
Timeline:
December 2011: The C11 standard was published
[https://protect2.fireeye.com/v1/url?k=8e23b012-d08373bb-8e23f089-
86959e472243-a2babe7075f8ac38&q=1&e=e3b0051e-bb23-4a30-84c7- 7e5e80f83325&u=http%3A%2F%2Fwww.open- std.org%2Fjtc1%2Fsc22%2Fwg14%2Fwww%2Fstandards.html].Quoted 2 lines
March 2012: GCC 4.7 was released, introducing the
__atomicbuilt-ins[https://gcc.gnu.org/gcc-4.7/changes.html,
https://www.gnu.org/software/gcc/gcc-4.7/].Quoted 2 lines
March 2013: GCC 4.8 was released [https://www.gnu.org/software/gcc/gcc-
4.8/].
Quoted 2 lines
April 2014: GCC 4.9 was released, introducing C11 atomics (incl.
<
stdatomic.h>) [https://gcc.gnu.org/gcc-4.9/changes.html,
https://www.gnu.org/software/gcc/gcc-4.9/].Quoted 7 lines
June 2014: RHEL7 was released
[https://access.redhat.com/articles/3078]. (RHEL7 Beta was released in December 2013, which probably explains why the GA release doesn’t include GCC 4.9.)May 2019 (i.e. one year ago): RHEL8 was released
[https://access.redhat.com/articles/3078].
Quoted 3 lines
RHEL7 includes GCC 4.8 only [https://access.redhat.com/solutions/19458],
and apparently RHEL7 has not been updated to GCC 4.9 with any of its minor releases.
Quoted 2 lines
Should the DPDK project be stuck on "industry standard" GCC atomics,
unable to use the decade old "official standard" C11 atomics, only because we want to support a six year old enterprise Linux distribution? Red Hat released a new enterprise version a year ago... perhaps it's time for their customers to upgrade, if they want to use the latest and greatest version of DPDK.
Just to be clear - I wasn't arguing for the direct use of GCC built-ins.
Quoted 6 lines
The GCC
__atomicbuilt-ins (called directly, or via a DPDK wrapper) do have some advantages over C11 atomics. One is that GCC supports 128-bit atomic operations, on certain architectures. <rte_atomic.h> already has a 128-bit compare-exchange. Also, since the GCC built-ins seem not to bother with architectures where atomics would be implemented by means of a lock, they are a little easier to use than <stdatomic.h>.
IMO, I do not think we should focus on built-ins vs APIs.
1) Built-ins are supported by both GCC and Clang today. If there is a new compiler in the future, most likely it will support these built-ins.
2) I like the fact that the built-ins always require the memory order parameter. stdatomic.h provides some APIs which do not need memory order (just like rte_atomicNN_xxx APIs). This needs us to implement checks in checkpatch script to avoid using such APIs.
3) If we need to replace the built-ins with APIs in the future, it is a simple search and replace.
If the decision to go with built-ins, turns out to be a bad decision, it can be corrected easily.
I think we should focus on the compiler not generating optimal code for __atomic_thread_fence(__ATOMIC_SEQ_CST) for x86. This is the main reason for these wrappers. From what I have seen, DPDK has tried to provide solutions internally for performance issues caused by compilers. Given that we have provided 'rte_atomic128_cmp_exchange' (provided because both the compilers were not generating the 128b compare-exchange), I would say we should just provide wrapper for '__atomic_thread_fence' built-in.
Quoted 7 lines
Quoted 3 lines
Are all the other tools required for building DPDK (in the required versions)
included in RHEL7, or do we require developers to install/upgrade any other tools anyway? If so, why not also GCC? DPDK can be used in a cross compilation environment, so we are not requiring RHEL7 users to replace their GCC 4.7 default compiler.
I have not used RHEL7, Intel CI uses RHEL7, may be they can answer.
Quoted 9 lines
Quoted 3 lines
Furthermore, the DPDK Documentation specifies GCC 4.9+ as a system
requirement [https://protect2.fireeye.com/v1/url?k=339bad56-6d3b6eff-
339bedcd-86959e472243-cb1bf3934c202e3f&q=1&e=e3b0051e-bb23-4a30- 84c7- 7e5e80f83325&u=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Flinux_gsg%2F sys_reqs.html%23compilation-of-the-dpdk]. If we are stuck on GCC 4.8, the documentation should be updated.
This is interesting. Then the CI systems should be upgraded to use GCC 4.9+.
Quoted 27 lines
Quoted 26 lines
Quoted 26 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
And if some compiler generates code that is suboptimal for a user,
then it
Quoted 2 lines
Quoted 2 lines
should be the choice of the user to either accept it or use a better
compiler.
Quoted 1 line
Quoted 1 line
Using a suboptimal compiler will not only affect the user's DPDK
applications,
Quoted 2 lines
Quoted 2 lines
but all applications developed by the user. And if he accepts it for
his other
Quoted 3 lines
Quoted 3 lines
applications, he will also accept it for his DPDK applications.
Quoted 1 line
We could introduce some sort of marker or standardized comment to
indicate when functions only exist for backwards compatibility with
ancient
Quoted 1 line
Quoted 1 line
compilers and similar, with a reference to documentation describing
why. And
Quoted 1 line
Quoted 1 line
when the documented preconditions are no longer relevant, e.g. when
those
Quoted 1 line
Quoted 1 line
particular enterprise Linux distributions become obsolete, these
functions
Quoted 2 lines
Quoted 2 lines
become obsolete too, and should be removed. However, getting rid of obsolete cruft will break the ABI. In other words: Added cruft will
never be
Quoted 1 line
Quoted 1 line
removed again, so think twice before adding.
Re: [dpdk-dev] [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
In reply to Honnappa NagarahalliQuoted 16 lines
<snip>
Subject: Re: [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
On Tue, May 12, 2020 at 4:03 pm, Phil Yang <mailto:[email protected]> wrote:
parameter. Signed-off-by: Phil Yang <mailto:[email protected]>
What is the purpose of having
rte_atomicat all?
Is this level of indirection really helping?
[HONNAPPA] (not sure why this email has html format, converted to text format)
I believe you meant, why not use the__atomic_xxxbuilt-ins directly? The only reason for now is handling of__atomic_thread_fence(__ATOMIC_SEQ_CST)for x86. This is equivalent torte_smp_mbwhich has an optimized implementation for x86. According to Konstantin, the compiler does not generate optimal code. Wrapping that built-in alone is going to be confusing.
Quoted 2 lines
The wrappers also allow us to have our own implementation using inline assembly for compilers versions that do not support C11 atomic built-ins. But, I do not know if there is a need to support those versions.
Few thoughts from my side about that patch:
Yes, for __atomic_thread_fence(__ATOMIC_SEQ_CST) generates full 'mfence' which is quite expensive, and can be a avoided for SMP case.
Though I don't see why we need to create our own wrappers for *all* __atomic buitins. From my perspective it would be sufficient to just introduce few of them:rte_thread_fence_XXX (where XXX - supported memory-orders: RELEASE, ACUIQRE, SEQ_CST, etc.). For all other __atomic built-ins I don't see any problem to use them directly, without introducing any wrappers around.
As a side note, this patch implements rte_atomic_thread_fence() as a simple wrapper around __atomic_thread_fence(), so concern mentioned above is not addressed.
Re: [dpdk-dev] [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
Quoted 24 lines
Quoted 5 lines
<snip>
Subject: Re: [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
On Tue, May 12, 2020 at 4:03 pm, Phil Yang <mailto:[email protected]>
Quoted 10 lines · wrote:
parameter. Signed-off-by: Phil Yang <mailto:[email protected]>
What is the purpose of having
rte_atomicat all?
Is this level of indirection really helping?
[HONNAPPA] (not sure why this email has html format, converted to text format) I believe you meant, why not use the__atomic_xxxbuilt-ins directly? The only reason for now is handling of__atomic_thread_fence(__ATOMIC_SEQ_CST)for x86. This is equivalent to
rte_smp_mbwhich has an optimized implementation for x86.Quoted 1 line
According to Konstantin, the compiler does not generate optimal code.
Wrapping that built-in alone is going to be confusing.
Quoted 3 lines
The wrappers also allow us to have our own implementation using inline assembly for compilers versions that do not support C11 atomic built-ins.
But, I do not know if there is a need to support those versions.
Quoted 10 lines
Few thoughts from my side about that patch:
Yes, for__atomic_thread_fence(__ATOMIC_SEQ_CST)generates full 'mfence' which is quite expensive, and can be a avoided for SMP case. Though I don't see why we need to create our own wrappers for *all*__atomicbuitins.
From my perspective it would be sufficient to just introduce few of them:rte_thread_fence_XXX(where XXX - supported memory-orders: RELEASE, ACUIQRE,SEQ_CST, etc.).
For all other__atomicbuilt-ins I don't see any problem to use them directly, without introducing any wrappers around.
I am all for not doing wrappers for the sake of doing. Here, we were concerned about the uniformity of the code, hence did the wrappers for all. Does, anyone have any concerns with doing the wrappers only for __atomic_thread_fence?
Is there any possibility that the compiler will change in the future to generate the optimized code for x86?
For the API, we already have 'rte_atomic128_cmp_exchange' implemented with C11 semantics, I suggest we keep this one also on the same lines. This would require the memory order to be a parameter.
Quoted 4 lines
As a side note, this patch implements
rte_atomic_thread_fence()as a simple wrapper around__atomic_thread_fence(), so concern mentioned above is not addressed.
Agreed. So, we will just pick the implementation of rte_smp_mb for x86 for this.
Re: [dpdk-dev] [PATCH v4 4/4] eal/atomic: add wrapper for c11 atomics
In reply to Stephen HemmingerQuoted 8 lines · Stephen Hemminger
Quoted 2 lines · Phil Yang
parameter. Signed-off-by: Phil Yang <[email protected] <mailto:[email protected]>>
What is the purpose of having
rte_atomicat all?
Is this level of indirection really helping?
To allow a different implementation than the GCC built-ins, for certain functions and architectures. To allow extensions (i.e. atomic functions that aren't GCC built-ins) in a clean way. To avoid using GCC built-ins directly, both for cosmetic reasons, and that it might cause problem for future compilers.
+ David Christensen
The PPC c11 atomics maintainer.
My apologies, I forgot to cc you on this email.
Thanks,
Phil Yang
Quoted 78 lines
-----Original Message-----
From: Phil Yang <[email protected]>
Sent: Tuesday, May 12, 2020 4:03 PM
To: [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Honnappa Nagarahalli <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Phil Yang <[email protected]>; nd <[email protected]>
Subject: [PATCH v4 0/4] generic rte atomic APIs deprecate proposalDPDK provides generic
rte_atomicAPIs to do several atomic operations. These APIs are using the deprecated__syncbuilt-ins and enforce full memory barriers on aarch64. However, full barriers are not necessary in many use cases. In order to address such use cases, C language offers C11 atomic APIs. The C11 atomic APIs provide finer memory barrier control by making use of the memory ordering parameter provided by the user. Various patches submitted in the past [2] and the patches in this series indicate significant performance gains on multiple aarch64 CPUs and no performance loss on x86.But the existing
rte_atomicAPI implementations cannot be changed as the APIs do not take the memory ordering parameter. The only choice available is replacing the usage of therte_atomicAPIs with C11 atomic APIs. In order to make this change, the following steps are proposed:[1] deprecate
rte_atomicAPIs so that future patches do not userte_atomicAPIs (a script is added to flag the usages).
[2] refactor the code that usesrte_atomicAPIs to use c11 atomic APIs.This patchset contains:
1) changes to programmer guide describing writing efficient code for aarch64.
2) the checkpatch script changes to flagrte_atomicNN_xxxAPI usage in patches.
3) wraps up compiler__atomicbuilt-ins with explicit memory ordering parameter.v4:
1. add reader-writer concurrency case describing.
2. claim maintainership of c11 atomics code for each platforms.
3. flagrte_atomicNN_xxxin new patches for modules that have been converted to
c11 style.
4. flag__sync_xxxbuilt-ins in new patches.
5. wraps up compiler atomic built-ins
6. move the changes of libraries which make use of c11 atomic APIs out of this
patchset.v3:
add libatomic dependency for 32-bit clangv2:
1. fix Clang '-Wincompatible-pointer-types' WARNING.
2. fix typos.Phil Yang (4):
doc: add generic atomic deprecation section
maintainers: claim maintainers of c11 atomics code
devtools: prevent use of rte atomic APIs in future patches eal/atomic: add wrapper for c11 atomicsMAINTAINERS | 4 + devtools/checkpatches.sh | 23 ++++ doc/guides/prog_guide/writing_efficient_code.rst | 139 ++++++++++++++++++++++- lib/librte_eal/include/generic/rte_atomic_c11.h | 139 +++++++++++++++++++++++ lib/librte_eal/include/meson.build | 1 + 5 files changed, 305 insertions(+), 1 deletion(-) create mode 100644 lib/librte_eal/include/generic/rte_atomic_c11.h--
2.7.4
v5 cover generic rte atomic APIs deprecate proposal
In reply to Phil YangDPDK provides generic rte_atomic APIs to do several atomic operations. These APIs are using the deprecated __sync built-ins and enforce full memory barriers on aarch64. However, full barriers are not necessary in many use cases. In order to address such use cases, C language offers C11 atomic APIs. The C11 atomic APIs provide finer memory barrier control by making use of the memory ordering parameter provided by the user. Various patches submitted in the past [2] and the patches in this series indicate significant performance gains on multiple aarch64 CPUs and no performance loss on x86.
But the existing rte_atomic API implementations cannot be changed as the APIs do not take the memory ordering parameter. The only choice available is replacing the usage of the rte_atomic APIs with C11 atomic APIs. In order to make this change, the following steps are proposed:
[1] deprecate rte_atomic APIs so that future patches do not use rte_atomic APIs (a script is added to flag the usages).
[2] refactor the code that uses rte_atomic APIs to use c11 atomic APIs.
This patchset contains:
1) changes to programmer guide describing writing efficient code for aarch64.
2) the checkpatch script changes to flag rte_atomicNN_xxx API usage in patches.
3) wraps up __atomic_thread_fence with explicit memory ordering parameter.
v5:
1. Wraps up __atomic_thread_fence to support optimized code for __ATOMIC_SEQ_CST memory order.
2. Flag __atomic_thread_fence with __ATOMIC_SEQ_CST in new patches.
3. Fix email address typo in patch 2/4.
v4:
1. add reader-writer concurrency case describing.
2. claim maintainership of c11 atomics code for each platforms.
3. flag rte_atomicNN_xxx in new patches for modules that have been converted to c11 style.
4. flag __sync_xxx built-ins in new patches.
5. wraps up compiler atomic built-ins
6. move the changes of libraries which make use of c11 atomic APIs out of this patchset.
v3:
add libatomic dependency for 32-bit clang
v2:
1. fix Clang '-Wincompatible-pointer-types' WARNING.
2. fix typos.
Phil Yang (4):
doc: add generic atomic deprecation section
maintainers: claim maintainers of c11 atomics code
devtools: prevent use of rte atomic APIs in future patches eal/atomic: add wrapper for c11 atomic thread fence
MAINTAINERS | 4 + devtools/checkpatches.sh | 32 ++++++ doc/guides/prog_guide/writing_efficient_code.rst | 139 ++++++++++++++++++++++- lib/librte_eal/arm/include/rte_atomic_32.h | 6 + lib/librte_eal/arm/include/rte_atomic_64.h | 6 + lib/librte_eal/include/generic/rte_atomic.h | 6 + lib/librte_eal/ppc/include/rte_atomic.h | 6 + lib/librte_eal/x86/include/rte_atomic.h | 17 +++ 8 files changed, 215 insertions(+), 1 deletion(-)
Signature · 1 line
2.7.4
PATCH v5 1/4 doc: add generic atomic deprecation section
Add deprecating the generic rte_atomic_xx APIs to c11 atomic built-ins guide and examples.
--- doc/guides/prog_guide/writing_efficient_code.rst | 139 ++++++++++++++++++++++- 1 file changed, 138 insertions(+), 1 deletion(-)
doc/guides/prog_guide/writing_efficient_code.rst +138 -1
@@ -167,7 +167,13 @@ but with the added cost of lower throughput. Locks and Atomic Operations --------------------------- -Atomic operations imply a lock prefix before the instruction, +This section describes some key considerations when using locks and atomic +operations in the DPDK environment. + +Locks +~~~~~ + +On x86, atomic operations imply a lock prefix before the instruction, causing the processor's LOCK# signal to be asserted during execution of the following instruction. This has a big impact on performance in a multicore environment.
@@ -176,6 +182,137 @@ It can often be replaced by other solutions like per-lcore variables. Also, some locking techniques are more efficient than others. For instance, the Read-Copy-Update (RCU) algorithm can frequently replace simple rwlocks. +Atomic Operations: Use C11 Atomic Built-ins +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +DPDK `generic rte_atomic <https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_eal/common/include/generic/rte_atomic.h>`_ operations are +implemented by `__sync built-ins <https://gcc.gnu.org/onlinedocs/gcc/_005f_005fsync-Builtins.html>`_. +These __sync built-ins result in full barriers on aarch64, which are unnecessary +in many use cases. They can be replaced by `__atomic built-ins <https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html>`_ that +conform to the C11 memory model and provide finer memory order control. + +So replacing the rte_atomic operations with __atomic built-ins might improve +performance for aarch64 machines. `More details <https://www.dpdk.org/wp-content/uploads/sites/35/2019/10/StateofC11Code.pdf>`_. + +Some typical optimization cases are listed below: + +Atomicity +^^^^^^^^^ + +Some use cases require atomicity alone, the ordering of the memory operations +does not matter. For example the packets statistics in the `vhost <https://github.com/DPDK/dpdk/blob/v20.02/examples/vhost/main.c#L796>`_ example application. + +It just updates the number of transmitted packets, no subsequent logic depends +on these counters. So the RELAXED memory ordering is sufficient: + +.. code-block:: c + + static __rte_always_inline void + virtio_xmit(struct vhost_dev *dst_vdev, struct vhost_dev *src_vdev, + struct rte_mbuf *m) + { + ... + ... + if (enable_stats) { + __atomic_add_fetch(&dst_vdev->stats.rx_total_atomic, 1, __ATOMIC_RELAXED); + __atomic_add_fetch(&dst_vdev->stats.rx_atomic, ret, __ATOMIC_RELAXED); + ... + } + } + +One-way Barrier +^^^^^^^^^^^^^^^ + +Some use cases allow for memory reordering in one way while requiring memory +ordering in the other direction. + +For example, the memory operations before the `lock <https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_eal/common/include/generic/rte_spinlock.h#L66>`_ can move to the +critical section, but the memory operations in the critical section cannot move +above the lock. In this case, the full memory barrier in the CAS operation can +be replaced to ACQUIRE. On the other hand, the memory operations after the +`unlock <https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_eal/common/include/generic/rte_spinlock.h#L88>`_ can move to the critical section, but the memory operations in the +critical section cannot move below the unlock. So the full barrier in the STORE +operation can be replaced with RELEASE. + +Reader-Writer Concurrency +^^^^^^^^^^^^^^^^^^^^^^^^^ +Lock-free reader-writer concurrency is one of the common use cases in DPDK. + +The payload or the data that the writer wants to communicate to the reader, +can be written with RELAXED memory order. However, the guard variable should +be written with RELEASE memory order. This ensures that the store to guard +variable is observable only after the store to payload is observable. +Refer to `rte_hash insert <https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_hash/rte_cuckoo_hash.c#L737>`_ for an example. + +.. code-block:: c + + static inline int32_t + rte_hash_cuckoo_insert_mw(const struct rte_hash *h, + ... + int32_t *ret_val) + { + ... + ... + + /* Insert new entry if there is room in the primary + * bucket. + */ + for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) { + /* Check if slot is available */ + if (likely(prim_bkt->key_idx[i] == EMPTY_SLOT)) { + prim_bkt->sig_current[i] = sig; + /* Store to signature and key should not + * leak after the store to key_idx. i.e. + * key_idx is the guard variable for signature + * and key. + */ + __atomic_store_n(&prim_bkt->key_idx[i], + new_idx, + __ATOMIC_RELEASE); + break; + } + } + + ... + } + +Correspondingly, on the reader side, the guard variable should be read +with ACQUIRE memory order. The payload or the data the writer communicated, +can be read with RELAXED memory order. This ensures that, if the store to +guard variable is observable, the store to payload is also observable. Refer to `rte_hash lookup <https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_hash/rte_cuckoo_hash.c#L1215>`_ for an example. + +.. code-block:: c + + static inline int32_t + search_one_bucket_lf(const struct rte_hash *h, const void *key, uint16_t sig, + void **data, const struct rte_hash_bucket *bkt) + { + ... + + for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) { + .... + if (bkt->sig_current[i] == sig) { + key_idx = __atomic_load_n(&bkt->key_idx[i], + __ATOMIC_ACQUIRE); + if (key_idx != EMPTY_SLOT) { + k = (struct rte_hash_key *) ((char *)keys + + key_idx * h->key_entry_size); + + if (rte_hash_cmp_eq(key, k->key, h) == 0) { + if (data != NULL) { + *data = __atomic_load_n(&k->pdata, + __ATOMIC_ACQUIRE); + } + + /* + * Return index where key is stored, + * subtracting the first dummy index + */ + return key_idx - 1; + } + ... + } + Coding Considerations ---------------------
Signature · 1 line
2.7.4
PATCH v5 2/4 maintainers: claim maintainers of c11 atomics code
In reply to Phil YangAdd the maintainership of c11 atomics code.
--- MAINTAINERS | 4 ++++ 1 file changed, 4 insertions(+)
MAINTAINERS +4 -0
@@ -265,6 +265,10 @@ F: lib/librte_eal/include/rte_random.h F: lib/librte_eal/common/rte_random.c F: app/test/test_rand_perf.c +C11 Code Maintainer +M: Honnappa Nagarahalli <[email protected]> +M: David Christensen <[email protected]> + ARM v7 M: Jan Viktorin <[email protected]> M: Ruifeng Wang <[email protected]>
Signature · 1 line
2.7.4
PATCH v5 3/4 devtools: prevent use of rte atomic APIs in future patches
In reply to Phil YangIn order to deprecate the rte_atomic APIs, prevent the patches from using rte_atomic APIs in the converted modules and compilers __sync built-ins in all modules.
The converted modules:lib/librte_distributorlib/librte_hashlib/librte_knilib/librte_lpmlib/librte_rculib/librte_ringlib/librte_stacklib/librte_vhostlib/librte_timerlib/librte_ipsecdrivers/event/octeontxdrivers/event/octeontx2drivers/event/opdldrivers/net/bnx2xdrivers/net/hinicdrivers/net/hns3drivers/net/memifdrivers/net/thunderxdrivers/net/virtio
examples/l2fwd-event
On x86 the __atomic_thread_fence(__ATOMIC_SEQ_CST) is quite expensive for SMP case. Flag the new code which use SEQ_CST memory ordering in __atomic_thread_fence API.
--- devtools/checkpatches.sh | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+)
devtools/checkpatches.sh +32 -0
@@ -69,6 +69,38 @@ check_forbidden_additions() { # <patch> -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ "$1" || res=1 + # refrain from new additions of 16/32/64 bits rte_atomic_xxx() + # multiple folders and expressions are separated by spaces + awk -v FOLDERS="lib/librte_distributor lib/librte_hash lib/librte_kni + lib/librte_lpm lib/librte_rcu lib/librte_ring + lib/librte_stack lib/librte_vhost drivers/event/octeontx + drivers/event/octeontx2 drivers/event/opdl + drivers/net/bnx2x drivers/net/hinic drivers/net/hns3 + drivers/net/memif drivers/net/thunderx + drivers/net/virtio examples/l2fwd-event" \ + -v EXPRESSIONS="rte_atomic[0-9][0-9]_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of rte_atomicNN_xxx APIs not allowed, use __atomic_xxx APIs' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from using compiler __sync built-ins + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="__sync_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of __sync_xxx built-ins not allowed, use __atomic_xxx APIs' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from using compiler __atomic_thread_fence(__ATOMIC_SEQ_CST) + # It should be avoided on x86 for SMP case. + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="__atomic_thread_fence\\\(__ATOMIC_SEQ_CST" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of __atomic_thread_fence with SEQ_CST ordering is not allowed, use rte_atomic_thread_fence' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + # svg figures must be included with wildcard extension # because of png conversion for pdf docs awk -v FOLDERS='doc' \
Signature · 1 line
2.7.4
Re: [dpdk-dev] [PATCH v5 3/4] devtools: prevent use of rte atomic APIs in future patches
<snip>
Quoted 71 lines
Subject: [PATCH v5 3/4] devtools: prevent use of rte atomic APIs in future patches
In order to deprecate the
rte_atomicAPIs, prevent the patches from usingrte_atomicAPIs in the converted modules and compilers__syncbuilt-ins in all modules.The converted modules:
lib/librte_distributorlib/librte_hashlib/librte_knilib/librte_lpmlib/librte_rculib/librte_ringlib/librte_stacklib/librte_vhostlib/librte_timerlib/librte_ipsecdrivers/event/octeontxdrivers/event/octeontx2drivers/event/opdldrivers/net/bnx2xdrivers/net/hinicdrivers/net/hns3drivers/net/memifdrivers/net/thunderxdrivers/net/virtio
examples/l2fwd-eventOn x86 the
__atomic_thread_fence(__ATOMIC_SEQ_CST)is quite expensive for SMP case. Flag the new code which useSEQ_CSTmemory ordering in__atomic_thread_fenceAPI.Signed-off-by Phil Yang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>--- devtools/checkpatches.sh | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+)devtools/checkpatches.sh index renamed from devtools/checkpatches.sh +0
-0158087f..5983f05 100755
devtools/checkpatches.sh +5
-0@@ -69,6 +69,38 @@ check_forbidden_additions() { # <patch> -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ "$1" || res=1 + # refrain from new additions of 16/32/64 bits rte_atomic_xxx() + # multiple folders and expressions are separated by spaces + awk -v FOLDERS="lib/librte_distributor lib/librte_hash lib/librte_kni + lib/librte_lpm lib/librte_rcu lib/librte_ring + lib/librte_stack lib/librte_vhost
drivers/event/octeontxdevtools/checkpatches.sh +7
-0+ drivers/event/octeontx2 drivers/event/opdl + drivers/net/bnx2x drivers/net/hinic drivers/net/hns3 + drivers/net/memif drivers/net/thunderx + drivers/net/virtio examples/l2fwd-event" \ + -v EXPRESSIONS="rte_atomic[0-9][0-9]_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of rte_atomicNN_xxx APIs not allowed, use
__atomic_xxxAPIs' \devtools/checkpatches.sh +8
-0+ -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from using compiler __sync built-ins + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="__sync_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of __sync_xxx built-ins not allowed, use
__atomic_xxxAPIs' \
devtools/checkpatches.sh
Quoted 2 lines
devtools/checkpatches.sh +2
-0+ -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1
Can you please add rte_smp_*mb as well?
Quoted 18 lines
devtools/checkpatches.sh +2
-0+ + # refrain from using compiler
__atomic_thread_fence(__ATOMIC_SEQ_CST)devtools/checkpatches.sh +3
-0+ # It should be avoided on x86 for SMP case. + awk -v FOLDERS="lib drivers app examples" \ + -vEXPRESSIONS="__atomic_thread_fence\\\(__ATOMIC_SEQ_CST" \
devtools/checkpatches.sh +2
-0+ -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of __atomic_thread_fence with SEQ_CSTordering is not allowed, use
rte_atomic_thread_fence' \devtools/checkpatches.sh +3
-1+ -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + # svg figures must be included with wildcard extension # because of png conversion for pdf docs awk -v FOLDERS='doc' \ --2.7.4
Re: [dpdk-dev] [PATCH v5 3/4] devtools: prevent use of rte atomic APIs in future patches
Quoted 83 lines
-----Original Message-----
From: Honnappa Nagarahalli <[email protected]>
Sent: Monday, June 29, 2020 12:38 PM
To: Phil Yang <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Ola Liljedahl <[email protected]>; Ruifeng Wang <[email protected]>; Phil Yang <[email protected]>; nd <[email protected]>; Honnappa Nagarahalli <[email protected]>; nd <[email protected]>
Subject: RE: [PATCH v5 3/4] devtools: prevent use of rte atomic APIs in future patches<snip>
Quoted 29 lines
Subject: [PATCH v5 3/4] devtools: prevent use of rte atomic APIs in future patches
In order to deprecate the
rte_atomicAPIs, prevent the patches from usingrte_atomicAPIs in the converted modules and compilers__syncbuilt-ins in all modules.The converted modules:
lib/librte_distributorlib/librte_hashlib/librte_knilib/librte_lpmlib/librte_rculib/librte_ringlib/librte_stacklib/librte_vhostlib/librte_timerlib/librte_ipsecdrivers/event/octeontxdrivers/event/octeontx2drivers/event/opdldrivers/net/bnx2xdrivers/net/hinicdrivers/net/hns3drivers/net/memifdrivers/net/thunderxdrivers/net/virtio
examples/l2fwd-eventQuoted 1 line
On x86 the
__atomic_thread_fence(__ATOMIC_SEQ_CST)is quiteexpensive
Quoted 24 lines
for SMP case. Flag the new code which use
SEQ_CSTmemory ordering in__atomic_thread_fenceAPI.Signed-off-by Phil Yang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>Reviewed-by Honnappa Nagarahalli <[email protected]>--- devtools/checkpatches.sh | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+)devtools/checkpatches.sh index renamed from devtools/checkpatches.sh +0
-0158087f..5983f05 100755
devtools/checkpatches.sh +5
-0@@ -69,6 +69,38 @@ check_forbidden_additions() { # <patch> -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ "$1" || res=1 + # refrain from new additions of 16/32/64 bits rte_atomic_xxx() + # multiple folders and expressions are separated by spaces + awk -v FOLDERS="lib/librte_distributor lib/librte_hash lib/librte_kni + lib/librte_lpm lib/librte_rcu lib/librte_ring + lib/librte_stack lib/librte_vhost
drivers/event/octeontxdevtools/checkpatches.sh check_forbidden_additions() { # <patch>
Quoted 7 lines
devtools/checkpatches.sh +7
-0+ drivers/event/octeontx2 drivers/event/opdl + drivers/net/bnx2x drivers/net/hinic drivers/net/hns3 + drivers/net/memif drivers/net/thunderx + drivers/net/virtio examples/l2fwd-event" \ + -v EXPRESSIONS="rte_atomic[0-9][0-9]_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of rte_atomicNN_xxx APIs not allowed,use
Quoted 13 lines
devtools/checkpatches.sh
Quoted 12 lines
__atomic_xxxAPIs' \devtools/checkpatches.sh +8
-0+ -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from using compiler __sync built-ins + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="__sync_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of __sync_xxx built-ins not allowed, use
__atomic_xxxAPIs' \devtools/checkpatches.sh +2
-0+ -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1Can you please add rte_smp_*mb as well?
Sure. Will do.
Thanks,
Phil
Quoted 19 lines
Quoted 19 lines
devtools/checkpatches.sh +2
-0+ + # refrain from using compiler
__atomic_thread_fence(__ATOMIC_SEQ_CST)devtools/checkpatches.sh +3
-0+ # It should be avoided on x86 for SMP case. + awk -v FOLDERS="lib drivers app examples" \ + -vEXPRESSIONS="__atomic_thread_fence\\\(__ATOMIC_SEQ_CST" \
devtools/checkpatches.sh +2
-0+ -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of __atomic_thread_fence with SEQ_CSTordering is not allowed, use
rte_atomic_thread_fence' \devtools/checkpatches.sh +3
-1+ -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + # svg figures must be included with wildcard extension # because of png conversion for pdf docs awk -v FOLDERS='doc' \ --2.7.4
PATCH v5 4/4 eal/atomic: add wrapper for c11 atomic thread fence
In reply to Phil YangProvide a wrapper for __atomic_thread_fence built-in to support optimized code for __ATOMIC_SEQ_CST memory order for x86 platforms.
--- lib/librte_eal/arm/include/rte_atomic_32.h | 6 ++++++ lib/librte_eal/arm/include/rte_atomic_64.h | 6 ++++++ lib/librte_eal/include/generic/rte_atomic.h | 6 ++++++ lib/librte_eal/ppc/include/rte_atomic.h | 6 ++++++ lib/librte_eal/x86/include/rte_atomic.h | 17 +++++++++++++++++ 5 files changed, 41 insertions(+)
lib/librte_eal/arm/include/rte_atomic_32.h +6 -0
@@ -37,6 +37,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} + #ifdef __cplusplus }#endif
lib/librte_eal/arm/include/rte_atomic_64.h +6 -0
@@ -41,6 +41,12 @@ extern "C" {#define rte_cio_rmb () asm volatile("dmb oshld" : : : "memory") +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} +/*------------------------ 128 bit atomic operations -------------------------*/ #if defined(__ARM_FEATURE_ATOMICS) || defined(RTE_ARM_FEATURE_ATOMICS )
lib/librte_eal/include/generic/rte_atomic.h +6 -0
@@ -158,6 +158,12 @@ static inline void rte_cio_rmb(void); asmvolatile ("" : : :"memory" ); \ }while (0 ) +/** +* Synchronization fence between threads based on the specified +* memory order. +*/ +static inline void rte_atomic_thread_fence(int mo); +/*------------------------- 16 bit atomic operations -------------------------*/ /**
lib/librte_eal/ppc/include/rte_atomic.h +6 -0
@@ -40,6 +40,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} +/*------------------------- 16 bit atomic operations -------------------------*/ /* To be compatible with Power7, use GCC built-in functions for 16 bit * operations */
lib/librte_eal/x86/include/rte_atomic.h +17 -0
@@ -83,6 +83,23 @@ rte_smp_mb(void)#define rte_cio_rmb () rte_compiler_barrier() +/** +* Synchronization fence between threads based on the specified +* memory order. +* +* On x86 the __atomic_thread_fence(__ATOMIC_SEQ_CST) generates +* full 'mfence' which is quite expensive. The optimized +* implementation of rte_smp_mb is used instead. +*/ +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +if (mo == __ATOMIC_SEQ_CST) +rte_smp_mb (); +else +__atomic_thread_fence (mo); +} +/*------------------------- 16 bit atomic operations -------------------------*/ #ifndef RTE_FORCE_INTRINSICS
Signature · 1 line
2.7.4
Re: [dpdk-dev] [PATCH v5 4/4] eal/atomic: add wrapper for c11 atomic thread fence
<snip>
Quoted 104 lines
Subject: [PATCH v5 4/4] eal/atomic: add wrapper for c11 atomic thread fence
Provide a wrapper for
__atomic_thread_fencebuilt-in to support optimized code for__ATOMIC_SEQ_CSTmemory order for x86 platforms.Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Phil Yang <[email protected]>Reviewed-by Ola Liljedahl <[email protected]>--- lib/librte_eal/arm/include/rte_atomic_32.h | 6 ++++++ lib/librte_eal/arm/include/rte_atomic_64.h | 6 ++++++ lib/librte_eal/include/generic/rte_atomic.h | 6 ++++++ lib/librte_eal/ppc/include/rte_atomic.h | 6 ++++++ lib/librte_eal/x86/include/rte_atomic.h | 17 +++++++++++++++++ 5 files changed, 41 insertions(+)lib/librte_eal/arm/include/rte_atomic_32.h +6
-0@@ -37,6 +37,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} + #ifdef __cplusplus }#endif lib/librte_eal/arm/include/rte_atomic_64.h +6
-0@@ -41,6 +41,12 @@ extern "C" {#define rte_cio_rmb () asm volatile("dmb oshld" : : : "memory") +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} +/*------------------------ 128 bit atomic operations -------------------------*/ #if defined(__ARM_FEATURE_ATOMICS) ||
defined(RTE_ARM_FEATURE_ATOMICS)diff --gita/lib/librte_eal/include/generic/rte_atomic.hb/lib/librte_eal/include/generic/rte_atomic.h
index e6ab15a..5b941db 100644lib/librte_eal/include/generic/rte_atomic.h +6
-0@@ -158,6 +158,12 @@ static inline void rte_cio_rmb(void); asmvolatile ("" : : :"memory" ); \ }while (0 ) +/** +* Synchronization fence between threads based on the specified +* memory order. +*/ +static inline void rte_atomic_thread_fence(int mo); +/*------------------------- 16 bit atomic operations -------------------------*/ /** lib/librte_eal/ppc/include/rte_atomic.h +6
-0@@ -40,6 +40,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} +/*------------------------- 16 bit atomic operations -------------------------*/ /* To be compatible with Power7, use GCC built-in functions for 16 bit * operations */ lib/librte_eal/x86/include/rte_atomic.h +6
-0@@ -83,6 +83,23 @@ rte_smp_mb(void)#define rte_cio_rmb () rte_compiler_barrier() +/** +* Synchronization fence between threads based on the specified +* memory order. +* +* On x86 the __atomic_thread_fence(__ATOMIC_SEQ_CST) generates +* full 'mfence' which is quite expensive. The optimized
lib/librte_eal/x86/include/rte_atomic.h rte_smp_mb(void) · line 101
Quoted 10 lines
lib/librte_eal/x86/include/rte_atomic.h +10
-0+* implementation of rte_smp_mb is used instead. +*/ +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +if (mo == __ATOMIC_SEQ_CST) +rte_smp_mb (); +else +__atomic_thread_fence (mo); +}
I think __ATOMIC_SEQ_CST needs to be used rarely. IMO, rte_atomic_thread_fence should be called only for __ATOMIC_SEQ_CST memory order. For all others the __atomic_thread_fence can be used directly. This will help us to stick to using the atomic built-ins in most of the cases.
Konstantin, is this ok for you?
Quoted 6 lines
lib/librte_eal/x86/include/rte_atomic.h +1
-1+/*------------------------- 16 bit atomic operations -------------------------*/ #ifndef RTE_FORCE_INTRINSICS --2.7.4
Re: [dpdk-dev] [PATCH v5 4/4] eal/atomic: add wrapper for c11 atomic thread fence
Quoted 122 lines
<snip>
Quoted 104 lines
Subject: [PATCH v5 4/4] eal/atomic: add wrapper for c11 atomic thread fence
Provide a wrapper for
__atomic_thread_fencebuilt-in to support optimized code for__ATOMIC_SEQ_CSTmemory order for x86 platforms.Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Phil Yang <[email protected]>Reviewed-by Ola Liljedahl <[email protected]>--- lib/librte_eal/arm/include/rte_atomic_32.h | 6 ++++++ lib/librte_eal/arm/include/rte_atomic_64.h | 6 ++++++ lib/librte_eal/include/generic/rte_atomic.h | 6 ++++++ lib/librte_eal/ppc/include/rte_atomic.h | 6 ++++++ lib/librte_eal/x86/include/rte_atomic.h | 17 +++++++++++++++++ 5 files changed, 41 insertions(+)lib/librte_eal/arm/include/rte_atomic_32.h +6
-0@@ -37,6 +37,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} + #ifdef __cplusplus }#endif lib/librte_eal/arm/include/rte_atomic_64.h +6
-0@@ -41,6 +41,12 @@ extern "C" {#define rte_cio_rmb () asm volatile("dmb oshld" : : : "memory") +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} +/*------------------------ 128 bit atomic operations -------------------------*/ #if defined(__ARM_FEATURE_ATOMICS) ||
defined(RTE_ARM_FEATURE_ATOMICS)diff --gita/lib/librte_eal/include/generic/rte_atomic.hb/lib/librte_eal/include/generic/rte_atomic.h
index e6ab15a..5b941db 100644lib/librte_eal/include/generic/rte_atomic.h +6
-0@@ -158,6 +158,12 @@ static inline void rte_cio_rmb(void); asmvolatile ("" : : :"memory" ); \ }while (0 ) +/** +* Synchronization fence between threads based on the specified +* memory order. +*/ +static inline void rte_atomic_thread_fence(int mo); +/*------------------------- 16 bit atomic operations -------------------------*/ /** lib/librte_eal/ppc/include/rte_atomic.h +6
-0@@ -40,6 +40,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} +/*------------------------- 16 bit atomic operations -------------------------*/ /* To be compatible with Power7, use GCC built-in functions for 16 bit * operations */ lib/librte_eal/x86/include/rte_atomic.h +6
-0@@ -83,6 +83,23 @@ rte_smp_mb(void)#define rte_cio_rmb () rte_compiler_barrier() +/** +* Synchronization fence between threads based on the specified +* memory order. +* +* On x86 the __atomic_thread_fence(__ATOMIC_SEQ_CST) generates +* full 'mfence' which is quite expensive. The optimized lib/librte_eal/x86/include/rte_atomic.h rte_smp_mb(void) · line 101
Quoted 10 lines
lib/librte_eal/x86/include/rte_atomic.h +10
-0+* implementation of rte_smp_mb is used instead. +*/ +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +if (mo == __ATOMIC_SEQ_CST) +rte_smp_mb (); +else +__atomic_thread_fence (mo); +}I think
__ATOMIC_SEQ_CSTneeds to be used rarely. IMO,rte_atomic_thread_fenceshould be called only for__ATOMIC_SEQ_CSTmemory order. For all others the__atomic_thread_fencecan be used directly. This will help us to stick to using the atomic built-ins in most of the cases.
Quoted 1 line
Konstantin, is this ok for you?
My preference is to have one generic rte_atomic_thread_fence() for all cases (I.E - current Phil implementation looks good to me).
I think it is more consistent approach and would help to avoid confusion. Konstantin
Re: [dpdk-dev] [PATCH v5 4/4] eal/atomic: add wrapper for c11 atomic thread fence
<snip>
Quoted 28 lines
Quoted 20 lines
Quoted 20 lines
lib/librte_eal/x86/include/rte_atomic.h +11
-0@@ -83,6 +83,23 @@ rte_smp_mb(void)#define rte_cio_rmb () rte_compiler_barrier() +/** +* Synchronization fence between threads based on the specified +* memory order. +* +* On x86 the __atomic_thread_fence(__ATOMIC_SEQ_CST) generates +* full 'mfence' which is quite expensive. The optimized +* implementation of rte_smp_mb is used instead. +*/ +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{Quoted 8 lines
lib/librte_eal/x86/include/rte_atomic.h rte_smp_mb(void) · line 101
Quoted 5 lines
lib/librte_eal/x86/include/rte_atomic.h +5
-0+if (mo == __ATOMIC_SEQ_CST) +rte_smp_mb (); +else +__atomic_thread_fence (mo); +}I think
__ATOMIC_SEQ_CSTneeds to be used rarely. IMO,rte_atomic_thread_fenceshould be called only for__ATOMIC_SEQ_CSTmemory order. For all others the__atomic_thread_fencecan be used
Quoted 8 lines
directly. This will help us to stick to using the atomic built-ins in most of the cases.
Quoted 2 lines
Konstantin, is this ok for you?
My preference is to have one generic
rte_atomic_thread_fence()for all cases (I.E - current Phil implementation looks good to me).
I think it is more consistent approach and would help to avoid confusion.
Ok, I am fine. The script currently checks for __atomic_thread_fence(__ATOMIC_SEQ_CST), we have to change it to check for __atomic_thread_fence
Quoted 1 line
Konstantin
Re: [dpdk-dev] [PATCH v5 4/4] eal/atomic: add wrapper for c11 atomic thread fence
In reply to Phil YangQuoted 103 lines
Provide a wrapper for
__atomic_thread_fencebuilt-in to support optimized code for__ATOMIC_SEQ_CSTmemory order for x86 platforms.Suggested-by Honnappa Nagarahalli <[email protected]>Signed-off-by Phil Yang <[email protected]>Reviewed-by Ola Liljedahl <[email protected]>--- lib/librte_eal/arm/include/rte_atomic_32.h | 6 ++++++ lib/librte_eal/arm/include/rte_atomic_64.h | 6 ++++++ lib/librte_eal/include/generic/rte_atomic.h | 6 ++++++ lib/librte_eal/ppc/include/rte_atomic.h | 6 ++++++ lib/librte_eal/x86/include/rte_atomic.h | 17 +++++++++++++++++ 5 files changed, 41 insertions(+)lib/librte_eal/arm/include/rte_atomic_32.h +6
-0@@ -37,6 +37,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} + #ifdef __cplusplus }#endif lib/librte_eal/arm/include/rte_atomic_64.h +6
-0@@ -41,6 +41,12 @@ extern "C" {#define rte_cio_rmb () asm volatile("dmb oshld" : : : "memory") +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} +/*------------------------ 128 bit atomic operations -------------------------*/ #if defined(__ARM_FEATURE_ATOMICS) || defined(RTE_ARM_FEATURE_ATOMICS )lib/librte_eal/include/generic/rte_atomic.h +6
-0@@ -158,6 +158,12 @@ static inline void rte_cio_rmb(void); asmvolatile ("" : : :"memory" ); \ }while (0 ) +/** +* Synchronization fence between threads based on the specified +* memory order. +*/ +static inline void rte_atomic_thread_fence(int mo); +/*------------------------- 16 bit atomic operations -------------------------*/ /** lib/librte_eal/ppc/include/rte_atomic.h +6
-0@@ -40,6 +40,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} +/*------------------------- 16 bit atomic operations -------------------------*/ /* To be compatible with Power7, use GCC built-in functions for 16 bit * operations */ lib/librte_eal/x86/include/rte_atomic.h +11
-0@@ -83,6 +83,23 @@ rte_smp_mb(void)#define rte_cio_rmb () rte_compiler_barrier() +/** +* Synchronization fence between threads based on the specified +* memory order. +* +* On x86 the __atomic_thread_fence(__ATOMIC_SEQ_CST) generates +* full 'mfence' which is quite expensive. The optimized +* implementation of rte_smp_mb is used instead. +*/ +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{
lib/librte_eal/x86/include/rte_atomic.h rte_smp_mb(void)
Quoted 10 lines
lib/librte_eal/x86/include/rte_atomic.h +6
-1+if (mo == __ATOMIC_SEQ_CST) +rte_smp_mb (); +else +__atomic_thread_fence (mo); +} +/*------------------------- 16 bit atomic operations -------------------------*/ #ifndef RTE_FORCE_INTRINSICS --
Quoted 1 line
2.7.4
v6 cover generic rte atomic APIs deprecate proposal
In reply to Phil YangDPDK provides generic rte_atomic APIs to do several atomic operations. These APIs are using the deprecated __sync built-ins and enforce full memory barriers on aarch64. However, full barriers are not necessary in many use cases. In order to address such use cases, C language offers C11 atomic APIs. The C11 atomic APIs provide finer memory barrier control by making use of the memory ordering parameter provided by the user. Various patches submitted in the past [2] and the patches in this series indicate significant performance gains on multiple aarch64 CPUs and no performance loss on x86.
But the existing rte_atomic API implementations cannot be changed as the APIs do not take the memory ordering parameter. The only choice available is replacing the usage of the rte_atomic APIs with C11 atomic APIs. In order to make this change, the following steps are proposed:
[1] deprecate rte_atomic APIs so that future patches do not use rte_atomic APIs (a script is added to flag the usages).
[2] refactor the code that uses rte_atomic APIs to use c11 atomic APIs.
This patchset contains:
1) changes to programmer guide describing writing efficient code for aarch64.
2) the checkpatch script changes to flag rte_atomicNN_xxx API usage in patches.
3) wraps up __atomic_thread_fence with explicit memory ordering parameter.
v6:
Add check for rte_smp barriers APIs in the new code.
v5:
1. Wraps up __atomic_thread_fence to support optimized code for __ATOMIC_SEQ_CST memory order.
2. Flag __atomic_thread_fence with __ATOMIC_SEQ_CST in new patches.
3. Fix email address typo in patch 2/4.
v4:
1. add reader-writer concurrency case describing.
2. claim maintainership of c11 atomics code for each platforms.
3. flag rte_atomicNN_xxx in new patches for modules that have been converted to c11 style.
4. flag __sync_xxx built-ins in new patches.
5. wraps up compiler atomic built-ins
6. move the changes of libraries which make use of c11 atomic APIs out of this patchset.
v3:
add libatomic dependency for 32-bit clang
v2:
1. fix Clang '-Wincompatible-pointer-types' WARNING.
2. fix typos.
Phil Yang (4):
doc: add generic atomic deprecation section
maintainers: claim maintainers of C11 atomics
devtools: prevent use of rte atomic APIs in future patches eal/atomic: add wrapper for C11 atomic thread fence
MAINTAINERS | 4 + devtools/checkpatches.sh | 40 +++++++ doc/guides/prog_guide/writing_efficient_code.rst | 139 ++++++++++++++++++++++- lib/librte_eal/arm/include/rte_atomic_32.h | 6 + lib/librte_eal/arm/include/rte_atomic_64.h | 6 + lib/librte_eal/include/generic/rte_atomic.h | 6 + lib/librte_eal/ppc/include/rte_atomic.h | 6 + lib/librte_eal/x86/include/rte_atomic.h | 17 +++ 8 files changed, 223 insertions(+), 1 deletion(-)
Signature · 1 line
2.7.4
PATCH v6 1/4 doc: add generic atomic deprecation section
Add deprecating the generic rte_atomic_xx APIs to c11 atomic built-ins guide and examples.
--- doc/guides/prog_guide/writing_efficient_code.rst | 139 ++++++++++++++++++++++- 1 file changed, 138 insertions(+), 1 deletion(-)
doc/guides/prog_guide/writing_efficient_code.rst +138 -1
@@ -167,7 +167,13 @@ but with the added cost of lower throughput. Locks and Atomic Operations --------------------------- -Atomic operations imply a lock prefix before the instruction, +This section describes some key considerations when using locks and atomic +operations in the DPDK environment. + +Locks +~~~~~ + +On x86, atomic operations imply a lock prefix before the instruction, causing the processor's LOCK# signal to be asserted during execution of the following instruction. This has a big impact on performance in a multicore environment.
@@ -176,6 +182,137 @@ It can often be replaced by other solutions like per-lcore variables. Also, some locking techniques are more efficient than others. For instance, the Read-Copy-Update (RCU) algorithm can frequently replace simple rwlocks. +Atomic Operations: Use C11 Atomic Built-ins +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +DPDK `generic rte_atomic <https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_eal/common/include/generic/rte_atomic.h>`_ operations are +implemented by `__sync built-ins <https://gcc.gnu.org/onlinedocs/gcc/_005f_005fsync-Builtins.html>`_. +These __sync built-ins result in full barriers on aarch64, which are unnecessary +in many use cases. They can be replaced by `__atomic built-ins <https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html>`_ that +conform to the C11 memory model and provide finer memory order control. + +So replacing the rte_atomic operations with __atomic built-ins might improve +performance for aarch64 machines. `More details <https://www.dpdk.org/wp-content/uploads/sites/35/2019/10/StateofC11Code.pdf>`_. + +Some typical optimization cases are listed below: + +Atomicity +^^^^^^^^^ + +Some use cases require atomicity alone, the ordering of the memory operations +does not matter. For example the packets statistics in the `vhost <https://github.com/DPDK/dpdk/blob/v20.02/examples/vhost/main.c#L796>`_ example application. + +It just updates the number of transmitted packets, no subsequent logic depends +on these counters. So the RELAXED memory ordering is sufficient: + +.. code-block:: c + + static __rte_always_inline void + virtio_xmit(struct vhost_dev *dst_vdev, struct vhost_dev *src_vdev, + struct rte_mbuf *m) + { + ... + ... + if (enable_stats) { + __atomic_add_fetch(&dst_vdev->stats.rx_total_atomic, 1, __ATOMIC_RELAXED); + __atomic_add_fetch(&dst_vdev->stats.rx_atomic, ret, __ATOMIC_RELAXED); + ... + } + } + +One-way Barrier +^^^^^^^^^^^^^^^ + +Some use cases allow for memory reordering in one way while requiring memory +ordering in the other direction. + +For example, the memory operations before the `lock <https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_eal/common/include/generic/rte_spinlock.h#L66>`_ can move to the +critical section, but the memory operations in the critical section cannot move +above the lock. In this case, the full memory barrier in the CAS operation can +be replaced to ACQUIRE. On the other hand, the memory operations after the +`unlock <https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_eal/common/include/generic/rte_spinlock.h#L88>`_ can move to the critical section, but the memory operations in the +critical section cannot move below the unlock. So the full barrier in the STORE +operation can be replaced with RELEASE. + +Reader-Writer Concurrency +^^^^^^^^^^^^^^^^^^^^^^^^^ +Lock-free reader-writer concurrency is one of the common use cases in DPDK. + +The payload or the data that the writer wants to communicate to the reader, +can be written with RELAXED memory order. However, the guard variable should +be written with RELEASE memory order. This ensures that the store to guard +variable is observable only after the store to payload is observable. +Refer to `rte_hash insert <https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_hash/rte_cuckoo_hash.c#L737>`_ for an example. + +.. code-block:: c + + static inline int32_t + rte_hash_cuckoo_insert_mw(const struct rte_hash *h, + ... + int32_t *ret_val) + { + ... + ... + + /* Insert new entry if there is room in the primary + * bucket. + */ + for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) { + /* Check if slot is available */ + if (likely(prim_bkt->key_idx[i] == EMPTY_SLOT)) { + prim_bkt->sig_current[i] = sig; + /* Store to signature and key should not + * leak after the store to key_idx. i.e. + * key_idx is the guard variable for signature + * and key. + */ + __atomic_store_n(&prim_bkt->key_idx[i], + new_idx, + __ATOMIC_RELEASE); + break; + } + } + + ... + } + +Correspondingly, on the reader side, the guard variable should be read +with ACQUIRE memory order. The payload or the data the writer communicated, +can be read with RELAXED memory order. This ensures that, if the store to +guard variable is observable, the store to payload is also observable. Refer to `rte_hash lookup <https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_hash/rte_cuckoo_hash.c#L1215>`_ for an example. + +.. code-block:: c + + static inline int32_t + search_one_bucket_lf(const struct rte_hash *h, const void *key, uint16_t sig, + void **data, const struct rte_hash_bucket *bkt) + { + ... + + for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) { + .... + if (bkt->sig_current[i] == sig) { + key_idx = __atomic_load_n(&bkt->key_idx[i], + __ATOMIC_ACQUIRE); + if (key_idx != EMPTY_SLOT) { + k = (struct rte_hash_key *) ((char *)keys + + key_idx * h->key_entry_size); + + if (rte_hash_cmp_eq(key, k->key, h) == 0) { + if (data != NULL) { + *data = __atomic_load_n(&k->pdata, + __ATOMIC_ACQUIRE); + } + + /* + * Return index where key is stored, + * subtracting the first dummy index + */ + return key_idx - 1; + } + ... + } + Coding Considerations ---------------------
Signature · 1 line
2.7.4
Re: [dpdk-dev] [PATCH v6 1/4] doc: add generic atomic deprecation section
Interestingly, John, our doc maintainer is not Cc'ed. I add him.
Please use --cc-cmd devtools/get-maintainer.sh I am expecting a review from an x86 maintainer as well. If no maintainer replies, ping them.
07/07/2020 11:50, Phil Yang:
Quoted 2 lines
Add deprecating the generic
rte_atomic_xxAPIs to c11 atomic built-ins guide and examples.
[...]
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables. · line 188
Quoted 4 lines
doc/guides/prog_guide/writing_efficient_code.rst +4
-0+Atomic Operations: Use C11 Atomic Built-ins +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +DPDK `generic rte_atomic <https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_eal/common/include/generic/rte_atomic.h>`_ operations are
Why this github link on 20.02?
Please try to keep lines small.
Quoted 1 line
+implemented by
__sync built-ins <https://gcc.gnu.org/onlinedocs/gcc/_005f_005fsync-Builtins.html>_.
Long links should be on their own line to avoid long lines.
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables. · line 195
Quoted 6 lines
doc/guides/prog_guide/writing_efficient_code.rst +6
-0+These __sync built-ins result in full barriers on aarch64, which are unnecessary +in many use cases. They can be replaced by `__atomic built-ins <https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html>`_ that +conform to the C11 memory model and provide finer memory order control. + +So replacing the rte_atomic operations with __atomic built-ins might improve +performance for aarch64 machines. `More details <https://www.dpdk.org/wp-content/uploads/sites/35/2019/10/StateofC11Code.pdf>`_.
"More details."
Please make a sentence.
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables. · line 203
Quoted 8 lines
doc/guides/prog_guide/writing_efficient_code.rst +8
-0+ +Some typical optimization cases are listed below: + +Atomicity +^^^^^^^^^ + +Some use cases require atomicity alone, the ordering of the memory operations +does not matter. For example the packets statistics in the `vhost <https://github.com/DPDK/dpdk/blob/v20.02/examples/vhost/main.c#L796>`_ example application.
Again github.
If you really want a web link, use code.dpdk.org or doc.dpdk.org/api
But why giving code example at all?
Quoted 8 lines
doc/guides/prog_guide/writing_efficient_code.rst +8
-0+ +It just updates the number of transmitted packets, no subsequent logic depends +on these counters. So the RELAXED memory ordering is sufficient: + +.. code-block:: c + + static __rte_always_inline void + virtio_xmit(struct vhost_dev *dst_vdev, struct vhost_dev *src_vdev,
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables. · line 221
Quoted 10 lines
doc/guides/prog_guide/writing_efficient_code.rst +10
-0+ struct rte_mbuf *m) + { + ... + ... + if (enable_stats) { + __atomic_add_fetch(&dst_vdev->stats.rx_total_atomic, 1, __ATOMIC_RELAXED); + __atomic_add_fetch(&dst_vdev->stats.rx_atomic, ret, __ATOMIC_RELAXED); + ... + } + }
I don't see how adding real code helps here.
Why not just mentioning __atomic_add_fetch and __ATOMIC_RELAXED?
Quoted 4 lines
doc/guides/prog_guide/writing_efficient_code.rst +4
-0+ +One-way Barrier +^^^^^^^^^^^^^^^ +
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables. · line 235
Quoted 10 lines
doc/guides/prog_guide/writing_efficient_code.rst +10
-0+Some use cases allow for memory reordering in one way while requiring memory +ordering in the other direction. + +For example, the memory operations before the `lock <https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_eal/common/include/generic/rte_spinlock.h#L66>`_ can move to the +critical section, but the memory operations in the critical section cannot move +above the lock. In this case, the full memory barrier in the CAS operation can +be replaced to ACQUIRE. On the other hand, the memory operations after the +`unlock <https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_eal/common/include/generic/rte_spinlock.h#L88>`_ can move to the critical section, but the memory operations in the +critical section cannot move below the unlock. So the full barrier in the STORE +operation can be replaced with RELEASE.
Again github links instead of our doxygen.
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables. · line 238
Quoted 3 lines
doc/guides/prog_guide/writing_efficient_code.rst +3
-0+ +Reader-Writer Concurrency +^^^^^^^^^^^^^^^^^^^^^^^^^
No blank line here?
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables. · line 245
Quoted 7 lines
doc/guides/prog_guide/writing_efficient_code.rst +7
-0+Lock-free reader-writer concurrency is one of the common use cases in DPDK. + +The payload or the data that the writer wants to communicate to the reader, +can be written with RELAXED memory order. However, the guard variable should +be written with RELEASE memory order. This ensures that the store to guard +variable is observable only after the store to payload is observable. +Refer to `rte_hash insert <https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_hash/rte_cuckoo_hash.c#L737>`_ for an example.
Hum...
Quoted 60 lines
doc/guides/prog_guide/writing_efficient_code.rst +60
-0+ +.. code-block:: c + + static inline int32_t + rte_hash_cuckoo_insert_mw(const struct rte_hash *h, + ... + int32_t *ret_val) + { + ... + ... + + /* Insert new entry if there is room in the primary + * bucket. + */ + for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) { + /* Check if slot is available */ + if (likely(prim_bkt->key_idx[i] == EMPTY_SLOT)) { + prim_bkt->sig_current[i] = sig; + /* Store to signature and key should not + * leak after the store to key_idx. i.e. + * key_idx is the guard variable for signature + * and key. + */ + __atomic_store_n(&prim_bkt->key_idx[i], + new_idx, + __ATOMIC_RELEASE); + break; + } + } + + ... + } + +Correspondingly, on the reader side, the guard variable should be read +with ACQUIRE memory order. The payload or the data the writer communicated, +can be read with RELAXED memory order. This ensures that, if the store to +guard variable is observable, the store to payload is also observable. Refer to `rte_hash lookup <https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_hash/rte_cuckoo_hash.c#L1215>`_ for an example. + +.. code-block:: c + + static inline int32_t + search_one_bucket_lf(const struct rte_hash *h, const void *key, uint16_t sig, + void **data, const struct rte_hash_bucket *bkt) + { + ... + + for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) { + .... + if (bkt->sig_current[i] == sig) { + key_idx = __atomic_load_n(&bkt->key_idx[i], + __ATOMIC_ACQUIRE); + if (key_idx != EMPTY_SLOT) { + k = (struct rte_hash_key *) ((char *)keys + + key_idx * h->key_entry_size); + + if (rte_hash_cmp_eq(key, k->key, h) == 0) { + if (data != NULL) { + *data = __atomic_load_n(&k->pdata, + __ATOMIC_ACQUIRE); + }
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables. · line 315
Quoted 10 lines
doc/guides/prog_guide/writing_efficient_code.rst +10
-0+ + /* + * Return index where key is stored, + * subtracting the first dummy index + */ + return key_idx - 1; + } + ... + } +
NACK for the big chunks of real code.
Please use words and avoid code.
If you insist on keeping code in doc, I will make you responsible of updating all the code we have already in the doc :)
Re: [dpdk-dev] [PATCH v6 1/4] doc: add generic atomic deprecation section
<snip>
Quoted 34 lines
Subject: Re: [dpdk-dev] [PATCH v6 1/4] doc: add generic atomic deprecation section
Interestingly, John, our doc maintainer is not Cc'ed.
I add him.
Please use --cc-cmddevtools/get-maintainer.shI am expecting a review from an x86 maintainer as well.
If no maintainer replies, ping them.07/07/2020 11:50, Phil Yang:
Quoted 2 lines
Add deprecating the generic
rte_atomic_xxAPIs to c11 atomic built-ins guide and examples.[...]
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables.
Quoted 6 lines
doc/guides/prog_guide/writing_efficient_code.rst +6
-0+Atomic Operations: Use C11 Atomic Built-ins +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +DPDK `generic rte_atomic +<https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_eal/common/inclu +de/generic/rte_atomic.h>`_ operations areWhy this github link on 20.02?
Please try to keep lines small.
Quoted 1 line
+implemented by `__sync built-ins
<https://gcc.gnu.org/onlinedocs/gcc/_005f_005fsync-Builtins.html>`_.
Long links should be on their own line to avoid long lines.
Review
Quoted 3 lines
(quoted diff) +3
-0+These __sync built-ins result in full barriers on aarch64, which are +unnecessary in many use cases. They can be replaced by `__atomic +built-ins <https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html>`_ that conform to the C11 memory model and provide finer memory order control.
Quoted 8 lines
Review
Quoted 3 lines
(quoted diff) +3
-0+ +So replacing the rte_atomic operations with __atomic built-ins might +improve performance for aarch64 machines. `More details<https://www.dpdk.org/wp-
content/uploads/sites/35/2019/10/StateofC11Code.pdf>`_."More details."
Please make a sentence.
The full stop is after the link. But, I think we will remove this link as well.
Quoted 62 lines
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables.
Quoted 9 lines
doc/guides/prog_guide/writing_efficient_code.rst +8
-0+ +Some typical optimization cases are listed below: + +Atomicity +^^^^^^^^^ + +Some use cases require atomicity alone, the ordering of the memory +operations does not matter. For example the packets statistics in the`vhost
<https://github.com/DPDK/dpdk/blob/v20.02/examples/vhost/main.c#L796>`
_ example application.Again github.
If you really want a web link, use code.dpdk.org or doc.dpdk.org/apiBut why giving code example at all?
Review
Quoted 3 lines
(quoted diff) +3
-0+ +It just updates the number of transmitted packets, no subsequent +logic depends on these counters. So the RELAXED memory ordering issufficient:
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables.
Quoted 11 lines
doc/guides/prog_guide/writing_efficient_code.rst +11
-0+ +.. code-block:: c + + static __rte_always_inline void + virtio_xmit(struct vhost_dev *dst_vdev, struct vhost_dev *src_vdev, + struct rte_mbuf *m) + { + ... + ... + if (enable_stats) { + __atomic_add_fetch(&dst_vdev->stats.rx_total_atomic, 1,
__ATOMIC_RELAXED);Quoted 1 line
+ __atomic_add_fetch(&dst_vdev->stats.rx_atomic, ret,
__ATOMIC_RELAXED);doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables. · line 221
Quoted 3 lines
doc/guides/prog_guide/writing_efficient_code.rst +3
-0+ ... + } + }I don't see how adding real code helps here.
Why not just mentioning__atomic_add_fetchand__ATOMIC_RELAXED?Quoted 4 lines
(quoted diff) +4
-0+ +One-way Barrier +^^^^^^^^^^^^^^^ +Review
Quoted 10 lines
(quoted diff) +10
-0+Some use cases allow for memory reordering in one way while requiring +memory ordering in the other direction. + +For example, the memory operations before the `lock +<https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_eal/common/inclu +de/generic/rte_spinlock.h#L66>`_ can move to the critical section, +but the memory operations in the critical section cannot move above +the lock. In this case, the full memory barrier in the CAS operation +can be replaced to ACQUIRE. On the other hand, the memory operations +after the `unlock<https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_eal/common/include/
generic/rte_spinlock.h#L88>`_ can move to the critical section, but the memory operations in the critical section cannot move below the unlock. So the full barrier in the STORE operation can be replaced with RELEASE.
Quoted 7 lines
Again github links instead of our doxygen.
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables. · line 238
Quoted 3 lines
doc/guides/prog_guide/writing_efficient_code.rst +3
-0+ +Reader-Writer Concurrency +^^^^^^^^^^^^^^^^^^^^^^^^^No blank line here?
Will fix
Quoted 93 lines
Quoted 2 lines
+Lock-free reader-writer concurrency is one of the common use cases in
DPDK.
Review
Quoted 5 lines
(quoted diff) +5
-0+ +The payload or the data that the writer wants to communicate to the +reader, can be written with RELAXED memory order. However, the guard +variable should be written with RELEASE memory order. This ensures +that the store to guard variable is observable only after the store topayload is observable.
Quoted 1 line
+Refer to `rte_hash insert
<https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_hash/rte_cuckoo_has
h.c#L737>`_ for an example.Hum...
Quoted 27 lines
doc/guides/prog_guide/writing_efficient_code.rst +27
-0+ +.. code-block:: c + + static inline int32_t + rte_hash_cuckoo_insert_mw(const struct rte_hash *h, + ... + int32_t *ret_val) + { + ... + ... + + /* Insert new entry if there is room in the primary + * bucket. + */ + for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) { + /* Check if slot is available */ + if (likely(prim_bkt->key_idx[i] == EMPTY_SLOT)) { + prim_bkt->sig_current[i] = sig; + /* Store to signature and key should not + * leak after the store to key_idx. i.e. + * key_idx is the guard variable for signature + * and key. + */ + __atomic_store_n(&prim_bkt->key_idx[i], + new_idx, + __ATOMIC_RELEASE); + break;doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables.
Quoted 10 lines
doc/guides/prog_guide/writing_efficient_code.rst +10
-0+ } + } + + ... + } + +Correspondingly, on the reader side, the guard variable should be +read with ACQUIRE memory order. The payload or the data the writer +communicated, can be read with RELAXED memory order. This ensures +that, if the store to guard variable is observable, the store to payload isalso observable. Refer to `rte_hash lookup
<https://github.com/DPDK/dpdk/blob/v20.02/lib/librte_hash/rte_cuckoo_has
h.c#L1215>`_ for an example.doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables.
Quoted 5 lines
doc/guides/prog_guide/writing_efficient_code.rst +5
-0+ +.. code-block:: c + + static inline int32_t + search_one_bucket_lf(const struct rte_hash *h, const void *key,
uint16_tsig,Quoted 18 lines
doc/guides/prog_guide/writing_efficient_code.rst +18
-0+ void **data, const struct rte_hash_bucket *bkt) + { + ... + + for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) { + .... + if (bkt->sig_current[i] == sig) { + key_idx = __atomic_load_n(&bkt->key_idx[i], + __ATOMIC_ACQUIRE); + if (key_idx != EMPTY_SLOT) { + k = (struct rte_hash_key *) ((char *)keys + + key_idx * h->key_entry_size); + + if (rte_hash_cmp_eq(key, k->key, h) == 0) { + if (data != NULL) { + *data = __atomic_load_n(&k->pdata, + __ATOMIC_ACQUIRE); + }doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables. · line 315
Quoted 10 lines
doc/guides/prog_guide/writing_efficient_code.rst +10
-0+ + /* + * Return index where key is stored, + * subtracting the first dummy index + */ + return key_idx - 1; + } + ... + } +NACK for the big chunks of real code.
Please use words and avoid code.
Quoted 2 lines
If you insist on keeping code in doc, I will make you responsible of updating all the code we have already in the doc :)
Ok, understood, will re-spin.
Quoted 1 line
PATCH v6 2/4 maintainers: claim maintainers of C11 atomics
In reply to Phil YangAdd the maintainership of C11 atomics code. Different maintainers focus on different platforms C11 atomics.
--- MAINTAINERS | 4 ++++ 1 file changed, 4 insertions(+)
MAINTAINERS +4 -0
@@ -276,6 +276,10 @@ F: lib/librte_eal/include/rte_random.h F: lib/librte_eal/common/rte_random.c F: app/test/test_rand_perf.c +C11 Atomics Code Maintainer +M: Honnappa Nagarahalli <[email protected]> +M: David Christensen <[email protected]> + ARM v7 M: Jan Viktorin <[email protected]> M: Ruifeng Wang <[email protected]>
Signature · 1 line
2.7.4
Re: [dpdk-dev] [PATCH v6 2/4] maintainers: claim maintainers of C11 atomics
07/07/2020 11:50, Phil Yang:
MAINTAINERS F: lib/librte_eal/include/rte_random.h · line 281
Quoted 7 lines
Add the maintainership of C11 atomics code. Different maintainers focus on different platforms C11 atomics.
MAINTAINERS +3
-0+C11 Atomics Code Maintainer +M: Honnappa Nagarahalli <[email protected]> +M: David Christensen <[email protected]>
I remember we discussed the need for identifying maintainers of C11 atomics code.
However I doubt it will be very useful if there is no file associated with this responsibility.
Maybe that relying on arch maintainers is enough?
Re: [dpdk-dev] [PATCH v6 2/4] maintainers: claim maintainers of C11 atomics
<snip>
Quoted 12 lines
07/07/2020 11:50, Phil Yang:
MAINTAINERS F: lib/librte_eal/include/rte_random.h · line 281
Quoted 7 lines
Add the maintainership of C11 atomics code. Different maintainers focus on different platforms C11 atomics.
MAINTAINERS +3
-0+C11 Atomics Code Maintainer +M: Honnappa Nagarahalli <[email protected]> +M: David Christensen <[email protected]>I remember we discussed the need for identifying maintainers of C11 atomics code.
Yes, it was discussed in the tech-board meeting [1].
[1] https://mails.dpdk.org/archives/dev/2020-April/165143.html
Quoted 4 lines
However I doubt it will be very useful if there is no file associated with this responsibility.
Maybe that relying on arch maintainers is enough?
I already see reviews from multiple engineers (not just arch maintainers) on the mailing list on C11 related patches. I think it might be enough. I am fine to remove this.
Quoted 1 line
Re: [dpdk-dev] [PATCH v6 2/4] maintainers: claim maintainers of C11 atomics
Honnappa Nagarahalli <[email protected]> writes:
<snip>
Quoted 18 lines
Quoted 12 lines
07/07/2020 11:50, Phil Yang:
MAINTAINERS F: lib/librte_eal/include/rte_random.h · line 281
Quoted 7 lines
Add the maintainership of C11 atomics code. Different maintainers focus on different platforms C11 atomics.
MAINTAINERS +3
-0+C11 Atomics Code Maintainer +M: Honnappa Nagarahalli <[email protected]> +M: David Christensen <[email protected]>I remember we discussed the need for identifying maintainers of C11
atomics
Quoted 1 line
code.
Yes, it was discussed in the tech-board meeting [1].
[1] https://mails.dpdk.org/archives/dev/2020-April/165143.html
Quoted 7 lines
Quoted 4 lines
However I doubt it will be very useful if there is no file associated with this responsibility.
Maybe that relying on arch maintainers is enough?
I already see reviews from multiple engineers (not just arch maintainers) on the mailing list on C11 related patches. I think it might be enough. I am fine to remove this.
Removed this patch in the new version.
Thanks,
Phil
PATCH v6 3/4 devtools: prevent use of rte atomic APIs in future patches
In reply to Phil YangIn order to deprecate the rte_atomic and rte_smp barrier APIs, prevent the patches from using these APIs in the converted modules and compilers __sync built-ins in all modules.
The converted modules:lib/librte_distributorlib/librte_hashlib/librte_knilib/librte_lpmlib/librte_rculib/librte_ringlib/librte_stacklib/librte_vhostlib/librte_timerlib/librte_ipsecdrivers/event/octeontxdrivers/event/octeontx2drivers/event/opdldrivers/net/bnx2xdrivers/net/hinicdrivers/net/hns3drivers/net/memifdrivers/net/thunderxdrivers/net/virtio
examples/l2fwd-event
On x86 the __atomic_thread_fence(__ATOMIC_SEQ_CST) is quite expensive for SMP case. Flag the new code which use __atomic_thread_fence API.
--- devtools/checkpatches.sh | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+)
devtools/checkpatches.sh +40 -0
@@ -51,6 +51,13 @@ print_usage () { check_forbidden_additions() { # <patch> res=0 + c11_atomics_dir="lib/librte_distributor lib/librte_hash lib/librte_kni + lib/librte_lpm lib/librte_rcu lib/librte_ring + lib/librte_stack lib/librte_vhost + drivers/event/octeontx drivers/event/octeontx2 + drivers/event/opdl drivers/net/bnx2x drivers/net/hinic + drivers/net/hns3 drivers/net/memif drivers/net/thunderx + drivers/net/virtio examples/l2fwd-event" # refrain from new additions of rte_panic() and rte_exit() # multiple folders and expressions are separated by spaces
@@ -74,6 +81,39 @@ check_forbidden_additions() { # <patch> -v EXPRESSIONS='for[[:space:]]*\\((char|u?int|unsigned|s?size_t)' \ -v RET_ON_FAIL=1 \ -v MESSAGE='Declaring a variable inside for()' \ + + # refrain from new additions of 16/32/64 bits rte_atomic_xxx() + # multiple folders and expressions are separated by spaces + awk -v FOLDERS="$c11_atomics_dir" \ + -v EXPRESSIONS="rte_atomic[0-9][0-9]_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of rte_atomicNN_xxx APIs not allowed, use __atomic_xxx built-ins' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from new additions of rte_smp_XXmb() + # multiple folders and expressions are separated by spaces + awk -v FOLDERS="$c11_atomics_dir" \ + -v EXPRESSIONS="rte_smp_(r|w)?mb\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of rte_smp_r/wmb not allowed, use __atomic_xxx built-ins' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from using compiler __sync built-ins + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="__sync_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of __sync_xxx built-ins not allowed, use __atomic_xxx built-ins' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from using compiler __atomic_thread_fence() + # It should be avoided on x86 for SMP case. + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="__atomic_thread_fence\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of __atomic_thread_fence is not allowed, use rte_atomic_thread_fence' \ -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ "$1" || res=1
Signature · 1 line
2.7.4
PATCH v6 4/4 eal/atomic: add wrapper for C11 atomic thread fence
In reply to Phil YangProvide a wrapper for __atomic_thread_fence built-in to support optimized code for __ATOMIC_SEQ_CST memory order for x86 platforms.
--- lib/librte_eal/arm/include/rte_atomic_32.h | 6 ++++++ lib/librte_eal/arm/include/rte_atomic_64.h | 6 ++++++ lib/librte_eal/include/generic/rte_atomic.h | 6 ++++++ lib/librte_eal/ppc/include/rte_atomic.h | 6 ++++++ lib/librte_eal/x86/include/rte_atomic.h | 17 +++++++++++++++++ 5 files changed, 41 insertions(+)
lib/librte_eal/arm/include/rte_atomic_32.h +6 -0
@@ -37,6 +37,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} + #ifdef __cplusplus }#endif
lib/librte_eal/arm/include/rte_atomic_64.h +6 -0
@@ -41,6 +41,12 @@ extern "C" {#define rte_cio_rmb () asm volatile("dmb oshld" : : : "memory") +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} +/*------------------------ 128 bit atomic operations -------------------------*/ #if defined(__ARM_FEATURE_ATOMICS) || defined(RTE_ARM_FEATURE_ATOMICS )
lib/librte_eal/include/generic/rte_atomic.h +6 -0
@@ -158,6 +158,12 @@ static inline void rte_cio_rmb(void); asmvolatile ("" : : :"memory" ); \ }while (0 ) +/** +* Synchronization fence between threads based on the specified +* memory order. +*/ +static inline void rte_atomic_thread_fence(int mo); +/*------------------------- 16 bit atomic operations -------------------------*/ /**
lib/librte_eal/ppc/include/rte_atomic.h +6 -0
@@ -40,6 +40,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} +/*------------------------- 16 bit atomic operations -------------------------*/ /* To be compatible with Power7, use GCC built-in functions for 16 bit * operations */
lib/librte_eal/x86/include/rte_atomic.h +17 -0
@@ -83,6 +83,23 @@ rte_smp_mb(void)#define rte_cio_rmb () rte_compiler_barrier() +/** +* Synchronization fence between threads based on the specified +* memory order. +* +* On x86 the __atomic_thread_fence(__ATOMIC_SEQ_CST) generates +* full 'mfence' which is quite expensive. The optimized +* implementation of rte_smp_mb is used instead. +*/ +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +if (mo == __ATOMIC_SEQ_CST) +rte_smp_mb (); +else +__atomic_thread_fence (mo); +} +/*------------------------- 16 bit atomic operations -------------------------*/ #ifndef RTE_FORCE_INTRINSICS
Signature · 1 line
2.7.4
v7 cover generic rte atomic APIs deprecate proposal
In reply to Phil YangDPDK provides generic rte_atomic APIs to do several atomic operations. These APIs are using the deprecated __sync built-ins and enforce full memory barriers on aarch64. However, full barriers are not necessary in many use cases. In order to address such use cases, C language offers C11 atomic APIs. The C11 atomic APIs provide finer memory barrier control by making use of the memory ordering parameter provided by the user. Various patches submitted in the past [2] and the patches in this series indicate significant performance gains on multiple aarch64 CPUs and no performance loss on x86.
But the existing rte_atomic API implementations cannot be changed as the APIs do not take the memory ordering parameter. The only choice available is replacing the usage of the rte_atomic APIs with C11 atomic APIs. In order to make this change, the following steps are proposed:
[1] deprecate rte_atomic APIs so that future patches do not use rte_atomic APIs (a script is added to flag the usages).
[2] refactor the code that uses rte_atomic APIs to use c11 atomic APIs.
This patchset contains:
1) changes to programmer guide describing writing efficient code for aarch64.
2) the checkpatch script changes to flag rte_atomicNN_xxx API usage in patches.
3) wraps up __atomic_thread_fence with explicit memory ordering parameter.
v7:
1. Remove code blocks in the guidance.
2. Remove code reference links in the guidance.
3. Remove the update of C11 atomics maintainers.
v6:
Add check for rte_smp barriers APIs in the new code.
v5:
1. Wraps up __atomic_thread_fence to support optimized code for __ATOMIC_SEQ_CST memory order.
2. Flag __atomic_thread_fence with __ATOMIC_SEQ_CST in new patches.
3. Fix email address typo in patch 2/4.
v4:
1. add reader-writer concurrency case describing.
2. claim maintainership of c11 atomics code for each platforms.
3. flag rte_atomicNN_xxx in new patches for modules that have been converted to c11 style.
4. flag __sync_xxx built-ins in new patches.
5. wraps up compiler atomic built-ins
6. move the changes of libraries which make use of c11 atomic APIs out of this patchset.
v3:
add libatomic dependency for 32-bit clang
v2:
1. fix Clang '-Wincompatible-pointer-types' WARNING.
2. fix typos.
Phil Yang (3):
doc: add generic atomic deprecation section
devtools: prevent use of rte atomic APIs in future patches eal/atomic: add wrapper for C11 atomic thread fence
devtools/checkpatches.sh | 40 +++++++++++++++ doc/guides/prog_guide/writing_efficient_code.rst | 64 +++++++++++++++++++++++- lib/librte_eal/arm/include/rte_atomic_32.h | 6 +++ lib/librte_eal/arm/include/rte_atomic_64.h | 6 +++ lib/librte_eal/include/generic/rte_atomic.h | 6 +++ lib/librte_eal/ppc/include/rte_atomic.h | 6 +++ lib/librte_eal/x86/include/rte_atomic.h | 17 +++++++ 7 files changed, 144 insertions(+), 1 deletion(-)
Signature · 1 line
2.7.4
PATCH v7 1/3 doc: add generic atomic deprecation section
Add deprecating the generic rte_atomic_xx APIs to C11 atomic built-ins guide and examples.
--- doc/guides/prog_guide/writing_efficient_code.rst | 64 +++++++++++++++++++++++- 1 file changed, 63 insertions(+), 1 deletion(-)
doc/guides/prog_guide/writing_efficient_code.rst +63 -1
@@ -167,7 +167,13 @@ but with the added cost of lower throughput. Locks and Atomic Operations --------------------------- -Atomic operations imply a lock prefix before the instruction, +This section describes some key considerations when using locks and atomic +operations in the DPDK environment. + +Locks +~~~~~ + +On x86, atomic operations imply a lock prefix before the instruction, causing the processor's LOCK# signal to be asserted during execution of the following instruction. This has a big impact on performance in a multicore environment.
@@ -176,6 +182,62 @@ It can often be replaced by other solutions like per-lcore variables. Also, some locking techniques are more efficient than others. For instance, the Read-Copy-Update (RCU) algorithm can frequently replace simple rwlocks. +Atomic Operations: Use C11 Atomic Built-ins +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +DPDK generic rte_atomic operations are implemented by `__sync built-ins +<https://gcc.gnu.org/onlinedocs/gcc/_005f_005fsync-Builtins.html>`_. +These __sync built-ins result in full barriers on aarch64, which are unnecessary +in many use cases. They can be replaced by `__atomic built-ins +<https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html>`_ +that conform to the C11 memory model and provide finer memory order control. + +So replacing the rte_atomic operations with __atomic built-ins might improve +performance for aarch64 machines. + +Some typical optimization cases are listed below: + +Atomicity +^^^^^^^^^ + +Some use cases require atomicity alone, the ordering of the memory operations +does not matter. For example the packets statistics in ``virtio_xmit()`` +function of ``vhost`` example application. It just updates the number of +transmitted packets, no subsequent logic depends on these counters. So the +RELAXED memory ordering is sufficient. + +One-way Barrier +^^^^^^^^^^^^^^^ + +Some use cases allow for memory reordering in one way while requiring memory +ordering in the other direction. + +For example, the memory operations before the ``rte_spinlock_lock()`` can move +to the critical section, but the memory operations in the critical section +cannot move above the lock. In this case, the full memory barrier in the +compare-and-swap operation can be replaced to ACQUIRE. On the other hand, the +memory operations after the ``rte_spinlock_unlock()`` can move to the critical +section, but the memory operations in the critical section cannot move below +the unlock. So the full barrier in the STORE operation can be replaced with +RELEASE. + +Reader-Writer Concurrency +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Lock-free reader-writer concurrency is one of the common use cases in DPDK. + +The payload or the data that the writer wants to communicate to the reader, +can be written with RELAXED memory order. However, the guard variable should +be written with RELEASE memory order. This ensures that the store to guard +variable is observable only after the store to payload is observable. +Refer to ``rte_hash_cuckoo_insert_mw()`` for an example. + +Correspondingly, on the reader side, the guard variable should be read +with ACQUIRE memory order. The payload or the data the writer communicated, +can be read with RELAXED memory order. This ensures that, if the store to +guard variable is observable, the store to payload is also observable. +Refer to rte_hash ``search_one_bucket_lf()`` for an example. + Coding Considerations ---------------------
Signature · 1 line
2.7.4
Re: [dpdk-dev] [PATCH v7 1/3] doc: add generic atomic deprecation section
Hi Phil,
Few comments.
<snip>
Quoted 1 line
Subject: [PATCH v7 1/3] doc: add generic atomic deprecation section
I think we should change this as follows:
"doc: add optimizations using C11 atomic built-ins"
Quoted 3 lines
Add deprecating the generic
rte_atomic_xxAPIs to C11 atomic built-ins guide and examples.
I think same here, something like following would help:
"Add information about possible optimizations using C11 atomic built-ins"
Quoted 35 lines
Signed-off-by Phil Yang <[email protected]>Signed-off-by Honnappa Nagarahalli <[email protected]>--- doc/guides/prog_guide/writing_efficient_code.rst | 64 +++++++++++++++++++++++- 1 file changed, 63 insertions(+), 1 deletion(-)doc/guides/prog_guide/writing_efficient_code.rst +7
-1@@ -167,7 +167,13 @@ but with the added cost of lower throughput. Locks and Atomic Operations --------------------------- -Atomic operations imply a lock prefix before the instruction, +This section describes some key considerations when using locks and +atomic operations in the DPDK environment. + +Locks +~~~~~ + +On x86, atomic operations imply a lock prefix before the instruction, causing the processor's LOCK# signal to be asserted during execution of thefollowing instruction.
This has a big impact on performance in a multicore environment.doc/guides/prog_guide/writing_efficient_code.rst +0
-0@@ -176,6 +182,62 @@ It can often be replaced by other solutions like per-lcore variables.
Also, some locking techniques are more efficient than others. For instance, the Read-Copy-Update (RCU) algorithm can frequently replace
simple rwlocks.
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables. · line 189
Quoted 5 lines
doc/guides/prog_guide/writing_efficient_code.rst +5
-0+Atomic Operations: Use C11 Atomic Built-ins +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +DPDK generic rte_atomic operations are implemented by `__sync built-ins +<https://gcc.gnu.org/onlinedocs/gcc/_005f_005fsync-Builtins.html>`_.
We can skip the hyper-link. Google search on "__sync built-in functions" leads to this page easily.
doc/guides/prog_guide/writing_efficient_code.rst
Quoted 5 lines
doc/guides/prog_guide/writing_efficient_code.rst +4
-0+These __sync built-ins result in full barriers on aarch64, which are +unnecessary in many use cases. They can be replaced by `__atomic +built-ins +<https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html>`_
Same here, we can skip the hyper-link.
Quoted 5 lines
+that conform to the C11 memory model and provide finer memory order control.
doc/guides/prog_guide/writing_efficient_code.rst +3
-0+ +So replacing the rte_atomic operations with __atomic built-ins might +improve performance for aarch64 machines.
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables.
Quoted 10 lines
doc/guides/prog_guide/writing_efficient_code.rst +10
-0+ +Some typical optimization cases are listed below: + +Atomicity +^^^^^^^^^ + +Some use cases require atomicity alone, the ordering of the memory +operations does not matter. For example the packets statistics in +``virtio_xmit()`` function of ``vhost`` example application. It just +updates the number of transmitted packets, no subsequent logic depends
Instead of referring to the code here, it is better to keep it generic. May be something like below?
"For example, the packet statistics counters need to be incremented atomically but do not need any particular memory ordering. So, RELAXED memory ordering is sufficient".
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables. · line 214
Quoted 8 lines
doc/guides/prog_guide/writing_efficient_code.rst +8
-0+on these counters. So the RELAXED memory ordering is sufficient. + +One-way Barrier +^^^^^^^^^^^^^^^ + +Some use cases allow for memory reordering in one way while requiring +memory ordering in the other direction. +
Spinlock is a good example, making is little more generic would be helpful.
Quoted 1 line
+For example, the memory operations before the ``rte_spinlock_lock()``
Instead of referring to "rte_spinlock_lock" can you just say spinlock lock
Quoted 1 line
+can move to the critical section, but the memory operations in the
^^^ are allowed to
Quoted 1 line
+critical section cannot move above the lock. In this case, the full
^^^^^^ are not allowed to
Quoted 1 line
+memory barrier in the compare-and-swap operation can be replaced to
^^ with
Quoted 1 line
+ACQUIRE. On the other hand, the memory operations after the
^^^^^^^^ ACQUIRE memory order.
Quoted 1 line
+``rte_spinlock_unlock()`` can move to the critical section, but the
^^^ are allowed to
Quoted 1 line
+memory operations in the critical section cannot move below the unlock.
^^^^^^ are not allowed to
Quoted 1 line
+So the full barrier in the STORE operation can be replaced with RELEASE.
I think this above statement can be replaced with something like below:
"So the full barrier in the store operation can use RELEASE memory order."
Quoted 10 lines
doc/guides/prog_guide/writing_efficient_code.rst +10
-0+ +Reader-Writer Concurrency +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Lock-free reader-writer concurrency is one of the common use cases in DPDK. + +The payload or the data that the writer wants to communicate to the +reader, can be written with RELAXED memory order. However, the guard +variable should be written with RELEASE memory order. This ensures that +the store to guard variable is observable only after the store to payload is
Quoted 2 lines
observable.
+Refer to ``rte_hash_cuckoo_insert_mw()`` for an example.
We can remove just this line above.
Quoted 7 lines
doc/guides/prog_guide/writing_efficient_code.rst +5
-0+ +Correspondingly, on the reader side, the guard variable should be read +with ACQUIRE memory order. The payload or the data the writer +communicated, can be read with RELAXED memory order. This ensures that, +if the store to guard variable is observable, the store to payload is alsoobservable.
+Refer torte_hash``search_one_bucket_lf()`` for an example.
Same here, suggest removing just the above line.
Quoted 6 lines
+
Coding Considerations
-----------------------
2.7.4
Re: [dpdk-dev] [PATCH v7 1/3] doc: add generic atomic deprecation section
Hi Honnappa,
Thanks for your comments and suggestions.
I Will update it in the next version.
Hi John,
I would greatly appreciate it if you kindly give me some feedback on this proposal.
Thanks,
Phil
Quoted 150 lines
Hi Phil,
Few comments.<snip>
Quoted 1 line
Subject: [PATCH v7 1/3] doc: add generic atomic deprecation section
I think we should change this as follows:
"doc: add optimizations using C11 atomic built-ins"Quoted 2 lines
Add deprecating the generic
rte_atomic_xxAPIs to C11 atomic built-insguide
Quoted 1 line
and examples.
I think same here, something like following would help:
"Add information about possible optimizations using C11 atomic built-ins"Quoted 16 lines
Signed-off-by Phil Yang <[email protected]>Signed-off-by Honnappa Nagarahalli <[email protected]>--- doc/guides/prog_guide/writing_efficient_code.rst | 64 +++++++++++++++++++++++- 1 file changed, 63 insertions(+), 1 deletion(-)doc/guides/prog_guide/writing_efficient_code.rst +0
-0@@ -167,7 +167,13 @@ but with the added cost of lower throughput. Locks and Atomic Operations ---------------------------doc/guides/prog_guide/writing_efficient_code.rst but with the added cost of lower throughput. · line 177
Quoted 10 lines
doc/guides/prog_guide/writing_efficient_code.rst +7
-1-Atomic operations imply a lock prefix before the instruction, +This section describes some key considerations when using locks and +atomic operations in the DPDK environment. + +Locks +~~~~~ + +On x86, atomic operations imply a lock prefix before the instruction, causing the processor's LOCK# signal to be asserted during execution ofthe
Quoted 7 lines
following instruction.
This has a big impact on performance in a multicore environment.doc/guides/prog_guide/writing_efficient_code.rst +0
-0@@ -176,6 +182,62 @@ It can often be replaced by other solutions like per-lcore variables.
Also, some locking techniques are more efficient than others. For instance, the Read-Copy-Update (RCU) algorithm can frequentlyreplace
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables. · line 189
Quoted 7 lines
simple rwlocks.
doc/guides/prog_guide/writing_efficient_code.rst +5
-0+Atomic Operations: Use C11 Atomic Built-ins +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +DPDK generic rte_atomic operations are implemented by `__sync built-ins +<https://gcc.gnu.org/onlinedocs/gcc/_005f_005fsync-Builtins.html>`_.We can skip the hyper-link. Google search on "
__syncbuilt-in functions" leads to this page easily.doc/guides/prog_guide/writing_efficient_code.rst
Quoted 5 lines
doc/guides/prog_guide/writing_efficient_code.rst +4
-0+These __sync built-ins result in full barriers on aarch64, which are +unnecessary in many use cases. They can be replaced by `__atomic +built-ins +<https://gcc.gnu.org/onlinedocs/gcc/_005f_005fatomic-Builtins.html>`_Same here, we can skip the hyper-link.
Quoted 5 lines
+that conform to the C11 memory model and provide finer memory order control.
doc/guides/prog_guide/writing_efficient_code.rst +3
-0+ +So replacing the rte_atomic operations with __atomic built-ins might +improve performance for aarch64 machines.doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables.
Quoted 10 lines
doc/guides/prog_guide/writing_efficient_code.rst +10
-0+ +Some typical optimization cases are listed below: + +Atomicity +^^^^^^^^^ + +Some use cases require atomicity alone, the ordering of the memory +operations does not matter. For example the packets statistics in +``virtio_xmit()`` function of ``vhost`` example application. It just +updates the number of transmitted packets, no subsequent logicdepends
Instead of referring to the code here, it is better to keep it generic. May be something like below?
"For example, the packet statistics counters need to be incremented atomically but do not need any particular memory ordering. So, RELAXED memory ordering is sufficient".doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables. · line 214
Quoted 8 lines
doc/guides/prog_guide/writing_efficient_code.rst +8
-0+on these counters. So the RELAXED memory ordering is sufficient. + +One-way Barrier +^^^^^^^^^^^^^^^ + +Some use cases allow for memory reordering in one way while requiring +memory ordering in the other direction. +Spinlock is a good example, making is little more generic would be helpful.
Quoted 1 line
+For example, the memory operations before the ``rte_spinlock_lock()``
Instead of referring to "
rte_spinlock_lock" can you just say spinlock lockQuoted 1 line
+can move to the critical section, but the memory operations in the
^^^ are allowed to
Quoted 1 line
+critical section cannot move above the lock. In this case, the full
^^^^^^ are not allowed to
Quoted 1 line
+memory barrier in the compare-and-swap operation can be replaced to
^^ withQuoted 1 line
+ACQUIRE. On the other hand, the memory operations after the
^^^^^^^^ ACQUIRE memory order.Quoted 1 line
+``rte_spinlock_unlock()`` can move to the critical section, but the
^^^ are allowed to
Quoted 1 line
+memory operations in the critical section cannot move below the unlock.
^^^^^^ are not allowed to
Quoted 1 line
+So the full barrier in the STORE operation can be replaced with RELEASE.
I think this above statement can be replaced with something like below:
"So the full barrier in the store operation can use RELEASE memory order."
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables.
Quoted 5 lines
doc/guides/prog_guide/writing_efficient_code.rst +5
-0+ +Reader-Writer Concurrency +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Lock-free reader-writer concurrency is one of the common use cases inDPDK.
Quoted 7 lines
doc/guides/prog_guide/writing_efficient_code.rst +5
-0+ +The payload or the data that the writer wants to communicate to the +reader, can be written with RELAXED memory order. However, the guard +variable should be written with RELEASE memory order. This ensures that +the store to guard variable is observable only after the store to payload isobservable.
+Refer to ``rte_hash_cuckoo_insert_mw()`` for an example.We can remove just this line above.
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables.
Quoted 4 lines
doc/guides/prog_guide/writing_efficient_code.rst +4
-0+ +Correspondingly, on the reader side, the guard variable should be read +with ACQUIRE memory order. The payload or the data the writer +communicated, can be read with RELAXED memory order. This ensuresthat,
Quoted 3 lines
+if the store to guard variable is observable, the store to payload is also observable.
+Refer torte_hash``search_one_bucket_lf()`` for an example.Same here, suggest removing just the above line.
Quoted 6 lines
+
Coding Considerations
-----------------------
2.7.4
PATCH v7 2/3 devtools: prevent use of rte atomic APIs in future patches
In reply to Phil YangIn order to deprecate the rte_atomic and rte_smp barrier APIs, prevent the patches from using these APIs in the converted modules and compilers __sync built-ins in all modules.
The converted modules:lib/librte_distributorlib/librte_hashlib/librte_knilib/librte_lpmlib/librte_rculib/librte_ringlib/librte_stacklib/librte_vhostlib/librte_timerlib/librte_ipsecdrivers/event/octeontxdrivers/event/octeontx2drivers/event/opdldrivers/net/bnx2xdrivers/net/hinicdrivers/net/hns3drivers/net/memifdrivers/net/thunderxdrivers/net/virtio
examples/l2fwd-event
On x86 the __atomic_thread_fence(__ATOMIC_SEQ_CST) is quite expensive for SMP case. Flag the new code which use __atomic_thread_fence API.
--- devtools/checkpatches.sh | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+)
devtools/checkpatches.sh +40 -0
@@ -51,6 +51,13 @@ print_usage () { check_forbidden_additions() { # <patch> res=0 + c11_atomics_dir="lib/librte_distributor lib/librte_hash lib/librte_kni + lib/librte_lpm lib/librte_rcu lib/librte_ring + lib/librte_stack lib/librte_vhost + drivers/event/octeontx drivers/event/octeontx2 + drivers/event/opdl drivers/net/bnx2x drivers/net/hinic + drivers/net/hns3 drivers/net/memif drivers/net/thunderx + drivers/net/virtio examples/l2fwd-event" # refrain from new additions of rte_panic() and rte_exit() # multiple folders and expressions are separated by spaces
@@ -74,6 +81,39 @@ check_forbidden_additions() { # <patch> -v EXPRESSIONS='for[[:space:]]*\\((char|u?int|unsigned|s?size_t)' \ -v RET_ON_FAIL=1 \ -v MESSAGE='Declaring a variable inside for()' \ + + # refrain from new additions of 16/32/64 bits rte_atomic_xxx() + # multiple folders and expressions are separated by spaces + awk -v FOLDERS="$c11_atomics_dir" \ + -v EXPRESSIONS="rte_atomic[0-9][0-9]_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of rte_atomicNN_xxx APIs not allowed, use __atomic_xxx built-ins' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from new additions of rte_smp_XXmb() + # multiple folders and expressions are separated by spaces + awk -v FOLDERS="$c11_atomics_dir" \ + -v EXPRESSIONS="rte_smp_(r|w)?mb\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of rte_smp_r/wmb not allowed, use __atomic_xxx built-ins' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from using compiler __sync built-ins + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="__sync_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of __sync_xxx built-ins not allowed, use __atomic_xxx built-ins' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from using compiler __atomic_thread_fence() + # It should be avoided on x86 for SMP case. + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="__atomic_thread_fence\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='The __atomic_thread_fence is not allowed, use rte_atomic_thread_fence' \ -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ "$1" || res=1
Signature · 1 line
2.7.4
PATCH v7 3/3 eal/atomic: add wrapper for C11 atomic thread fence
In reply to Phil YangProvide a wrapper for __atomic_thread_fence built-in to support optimized code for __ATOMIC_SEQ_CST memory order for x86 platforms.
--- lib/librte_eal/arm/include/rte_atomic_32.h | 6 ++++++ lib/librte_eal/arm/include/rte_atomic_64.h | 6 ++++++ lib/librte_eal/include/generic/rte_atomic.h | 6 ++++++ lib/librte_eal/ppc/include/rte_atomic.h | 6 ++++++ lib/librte_eal/x86/include/rte_atomic.h | 17 +++++++++++++++++ 5 files changed, 41 insertions(+)
lib/librte_eal/arm/include/rte_atomic_32.h +6 -0
@@ -37,6 +37,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} + #ifdef __cplusplus }#endif
lib/librte_eal/arm/include/rte_atomic_64.h +6 -0
@@ -41,6 +41,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} +/*------------------------ 128 bit atomic operations -------------------------*/ #if defined(__ARM_FEATURE_ATOMICS) || defined(RTE_ARM_FEATURE_ATOMICS )
lib/librte_eal/include/generic/rte_atomic.h +6 -0
@@ -158,6 +158,12 @@ static inline void rte_cio_rmb(void); asmvolatile ("" : : :"memory" ); \ }while (0 ) +/** +* Synchronization fence between threads based on the specified +* memory order. +*/ +static inline void rte_atomic_thread_fence(int mo); +/*------------------------- 16 bit atomic operations -------------------------*/ /**
lib/librte_eal/ppc/include/rte_atomic.h +6 -0
@@ -40,6 +40,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} +/*------------------------- 16 bit atomic operations -------------------------*/ /* To be compatible with Power7, use GCC built-in functions for 16 bit * operations */
lib/librte_eal/x86/include/rte_atomic.h +17 -0
@@ -83,6 +83,23 @@ rte_smp_mb(void)#define rte_cio_rmb () rte_compiler_barrier() +/** +* Synchronization fence between threads based on the specified +* memory order. +* +* On x86 the __atomic_thread_fence(__ATOMIC_SEQ_CST) generates +* full 'mfence' which is quite expensive. The optimized +* implementation of rte_smp_mb is used instead. +*/ +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +if (mo == __ATOMIC_SEQ_CST) +rte_smp_mb (); +else +__atomic_thread_fence (mo); +} +/*------------------------- 16 bit atomic operations -------------------------*/ #ifndef RTE_FORCE_INTRINSICS
Signature · 1 line
2.7.4
v8 cover generic rte atomic APIs deprecate proposal
In reply to Phil YangDPDK provides generic rte_atomic APIs to do several atomic operations. These APIs are using the deprecated __sync built-ins and enforce full memory barriers on aarch64. However, full barriers are not necessary in many use cases. In order to address such use cases, C language offers C11 atomic APIs. The C11 atomic APIs provide finer memory barrier control by making use of the memory ordering parameter provided by the user. Various patches submitted in the past [2] and the patches in this series indicate significant performance gains on multiple aarch64 CPUs and no performance loss on x86.
But the existing rte_atomic API implementations cannot be changed as the APIs do not take the memory ordering parameter. The only choice available is replacing the usage of the rte_atomic APIs with C11 atomic APIs. In order to make this change, the following steps are proposed:
[1] deprecate rte_atomic APIs so that future patches do not use rte_atomic APIs (a script is added to flag the usages).
[2] refactor the code that uses rte_atomic APIs to use c11 atomic APIs.
This patchset contains:
1) changes to programmer guide describing writing efficient code for aarch64.
2) the checkpatch script changes to flag rte_atomicNN_xxx API usage in patches.
3) wraps up __atomic_thread_fence with explicit memory ordering parameter.
v8:
Make descriptions more general. (Honnappa)
v7:
1. Remove code blocks in the guidance.
2. Remove code reference links in the guidance.
3. Remove the update of C11 atomics maintainers.
v6:
Add check for rte_smp barriers APIs in the new code.
v5:
1. Wraps up __atomic_thread_fence to support optimized code for __ATOMIC_SEQ_CST memory order.
2. Flag __atomic_thread_fence with __ATOMIC_SEQ_CST in new patches.
3. Fix email address typo in patch 2/4.
v4:
1. add reader-writer concurrency case describing.
2. claim maintainership of c11 atomics code for each platforms.
3. flag rte_atomicNN_xxx in new patches for modules that have been converted to c11 style.
4. flag __sync_xxx built-ins in new patches.
5. wraps up compiler atomic built-ins
6. move the changes of libraries which make use of c11 atomic APIs out of this patchset.
v3:
add libatomic dependency for 32-bit clang
v2:
1. fix Clang '-Wincompatible-pointer-types' WARNING.
2. fix typos.
Phil Yang (3):
doc: add optimizations using C11 atomic built-ins
devtools: prevent use of rte atomic APIs in future patches eal/atomic: add wrapper for C11 atomic thread fence
devtools/checkpatches.sh | 40 ++++++++++++++++ doc/guides/prog_guide/writing_efficient_code.rst | 59 +++++++++++++++++++++++- lib/librte_eal/arm/include/rte_atomic_32.h | 6 +++ lib/librte_eal/arm/include/rte_atomic_64.h | 6 +++ lib/librte_eal/include/generic/rte_atomic.h | 6 +++ lib/librte_eal/ppc/include/rte_atomic.h | 6 +++ lib/librte_eal/x86/include/rte_atomic.h | 17 +++++++ 7 files changed, 139 insertions(+), 1 deletion(-)
Signature · 1 line
2.7.4
PATCH v8 1/3 doc: add optimizations using C11 atomic built-ins
Add information about possible optimizations using C11 atomic built-ins.
--- doc/guides/prog_guide/writing_efficient_code.rst | 59 +++++++++++++++++++++++- 1 file changed, 58 insertions(+), 1 deletion(-)
doc/guides/prog_guide/writing_efficient_code.rst +58 -1
@@ -167,7 +167,13 @@ but with the added cost of lower throughput. Locks and Atomic Operations --------------------------- -Atomic operations imply a lock prefix before the instruction, +This section describes some key considerations when using locks and atomic +operations in the DPDK environment. + +Locks +~~~~~ + +On x86, atomic operations imply a lock prefix before the instruction, causing the processor's LOCK# signal to be asserted during execution of the following instruction. This has a big impact on performance in a multicore environment.
@@ -176,6 +182,57 @@ It can often be replaced by other solutions like per-lcore variables. Also, some locking techniques are more efficient than others. For instance, the Read-Copy-Update (RCU) algorithm can frequently replace simple rwlocks. +Atomic Operations: Use C11 Atomic Built-ins +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +DPDK generic rte_atomic operations are implemented by __sync built-ins. These +__sync built-ins result in full barriers on aarch64, which are unnecessary +in many use cases. They can be replaced by __atomic built-ins that conform to +the C11 memory model and provide finer memory order control. + +So replacing the rte_atomic operations with __atomic built-ins might improve +performance for aarch64 machines. + +Some typical optimization cases are listed below: + +Atomicity +^^^^^^^^^ + +Some use cases require atomicity alone, the ordering of the memory operations +does not matter. For example, the packet statistics counters need to be +incremented atomically but do not need any particular memory ordering. +So, RELAXED memory ordering is sufficient. + +One-way Barrier +^^^^^^^^^^^^^^^ + +Some use cases allow for memory reordering in one way while requiring memory +ordering in the other direction. + +For example, the memory operations before the spinlock lock are allowed to +move to the critical section, but the memory operations in the critical section +are not allowed to move above the lock. In this case, the full memory barrier +in the compare-and-swap operation can be replaced with ACQUIRE memory order. +On the other hand, the memory operations after the spinlock unlock are allowed +to move to the critical section, but the memory operations in the critical +section are not allowed to move below the unlock. So the full barrier in the +store operation can use RELEASE memory order. + +Reader-Writer Concurrency +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Lock-free reader-writer concurrency is one of the common use cases in DPDK. + +The payload or the data that the writer wants to communicate to the reader, +can be written with RELAXED memory order. However, the guard variable should +be written with RELEASE memory order. This ensures that the store to guard +variable is observable only after the store to payload is observable. + +Correspondingly, on the reader side, the guard variable should be read +with ACQUIRE memory order. The payload or the data the writer communicated, +can be read with RELAXED memory order. This ensures that, if the store to +guard variable is observable, the store to payload is also observable. + Coding Considerations ---------------------
Signature · 1 line
2.7.4
Re: [dpdk-dev] [PATCH v8 1/3] doc: add optimizations using C11 atomic built-ins
Hello,
Quoted 2 lines · Phil Yang
Add information about possible optimizations using C11 atomic built-ins.
We are missing a review on this doc update.
Thanks.
Signature · 1 line
David Marchand
Quoted 87 lines
Signed-off-by Phil Yang <[email protected]>Signed-off-by Honnappa Nagarahalli <[email protected]>--- doc/guides/prog_guide/writing_efficient_code.rst | 59 +++++++++++++++++++++++- 1 file changed, 58 insertions(+), 1 deletion(-)doc/guides/prog_guide/writing_efficient_code.rst +58
-2@@ -167,7 +167,13 @@ but with the added cost of lower throughput. Locks and Atomic Operations --------------------------- -Atomic operations imply a lock prefix before the instruction, +This section describes some key considerations when using locks and atomic +operations in the DPDK environment. + +Locks +~~~~~ + +On x86, atomic operations imply a lock prefix before the instruction, causing the processor's LOCK# signal to be asserted during execution of the following instruction. This has a big impact on performance in a multicore environment.@@ -176,6 +182,57 @@ It can often be replaced by other solutions like per-lcore variables. Also, some locking techniques are more efficient than others. For instance, the Read-Copy-Update (RCU) algorithm can frequently replace simple rwlocks. +Atomic Operations: Use C11 Atomic Built-ins +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +DPDK generic rte_atomic operations are implemented by __sync built-ins. These +__sync built-ins result in full barriers on aarch64, which are unnecessary +in many use cases. They can be replaced by __atomic built-ins that conform to +the C11 memory model and provide finer memory order control. + +So replacing the rte_atomic operations with __atomic built-ins might improve +performance for aarch64 machines. + +Some typical optimization cases are listed below: + +Atomicity +^^^^^^^^^ + +Some use cases require atomicity alone, the ordering of the memory operations +does not matter. For example, the packet statistics counters need to be +incremented atomically but do not need any particular memory ordering. +So, RELAXED memory ordering is sufficient. + +One-way Barrier +^^^^^^^^^^^^^^^ + +Some use cases allow for memory reordering in one way while requiring memory +ordering in the other direction. + +For example, the memory operations before the spinlock lock are allowed to +move to the critical section, but the memory operations in the critical section +are not allowed to move above the lock. In this case, the full memory barrier +in the compare-and-swap operation can be replaced with ACQUIRE memory order. +On the other hand, the memory operations after the spinlock unlock are allowed +to move to the critical section, but the memory operations in the critical +section are not allowed to move below the unlock. So the full barrier in the +store operation can use RELEASE memory order. + +Reader-Writer Concurrency +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Lock-free reader-writer concurrency is one of the common use cases in DPDK. + +The payload or the data that the writer wants to communicate to the reader, +can be written with RELAXED memory order. However, the guard variable should +be written with RELEASE memory order. This ensures that the store to guard +variable is observable only after the store to payload is observable. + +Correspondingly, on the reader side, the guard variable should be read +with ACQUIRE memory order. The payload or the data the writer communicated, +can be read with RELAXED memory order. This ensures that, if the store to +guard variable is observable, the store to payload is also observable. + Coding Considerations --------------------- --2.7.4
Re: [dpdk-dev] [PATCH v8 1/3] doc: add optimizations using C11 atomic built-ins
In reply to Phil Yang<snip>
Quoted 6 lines
Subject: [PATCH v8 1/3] doc: add optimizations using C11 atomic built-ins
Add information about possible optimizations using C11 atomic built-ins.
Signed-off-by Phil Yang <[email protected]>Signed-off-by Honnappa Nagarahalli <[email protected]>
Thanks for the changes, they look good now.
David wanted to change 'built-ins' to 'builtins', otherwise
Quoted 93 lines
--- doc/guides/prog_guide/writing_efficient_code.rst | 59 +++++++++++++++++++++++- 1 file changed, 58 insertions(+), 1 deletion(-)doc/guides/prog_guide/writing_efficient_code.rst +7
-1@@ -167,7 +167,13 @@ but with the added cost of lower throughput. Locks and Atomic Operations --------------------------- -Atomic operations imply a lock prefix before the instruction, +This section describes some key considerations when using locks and +atomic operations in the DPDK environment. + +Locks +~~~~~ + +On x86, atomic operations imply a lock prefix before the instruction, causing the processor's LOCK# signal to be asserted during execution of thefollowing instruction.
This has a big impact on performance in a multicore environment.doc/guides/prog_guide/writing_efficient_code.rst +0
-0@@ -176,6 +182,57 @@ It can often be replaced by other solutions like per-lcore variables.
Also, some locking techniques are more efficient than others. For instance, the Read-Copy-Update (RCU) algorithm can frequently replace
simple rwlocks.doc/guides/prog_guide/writing_efficient_code.rst +7
-0+Atomic Operations: Use C11 Atomic Built-ins +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +DPDK generic rte_atomic operations are implemented by __sync built-ins. +These __sync built-ins result in full barriers on aarch64, which are +unnecessary in many use cases. They can be replaced by __atomic +built-ins that conform to the C11 memory model and provide finer memoryorder control.
doc/guides/prog_guide/writing_efficient_code.rst +12
-0+ +So replacing the rte_atomic operations with __atomic built-ins might +improve performance for aarch64 machines. + +Some typical optimization cases are listed below: + +Atomicity +^^^^^^^^^ + +Some use cases require atomicity alone, the ordering of the memory +operations does not matter. For example, the packet statistics counters +need to be incremented atomically but do not need any particular memoryordering.
doc/guides/prog_guide/writing_efficient_code.rst +12
-0+So, RELAXED memory ordering is sufficient. + +One-way Barrier +^^^^^^^^^^^^^^^ + +Some use cases allow for memory reordering in one way while requiring +memory ordering in the other direction. + +For example, the memory operations before the spinlock lock are allowed +to move to the critical section, but the memory operations in the +critical section are not allowed to move above the lock. In this case, +the full memory barrier in the compare-and-swap operation can be replacedwith ACQUIRE memory order.
doc/guides/prog_guide/writing_efficient_code.rst +14
-0+On the other hand, the memory operations after the spinlock unlock are +allowed to move to the critical section, but the memory operations in +the critical section are not allowed to move below the unlock. So the +full barrier in the store operation can use RELEASE memory order. + +Reader-Writer Concurrency +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Lock-free reader-writer concurrency is one of the common use cases in DPDK. + +The payload or the data that the writer wants to communicate to the +reader, can be written with RELAXED memory order. However, the guard +variable should be written with RELEASE memory order. This ensures that +the store to guard variable is observable only after the store to payload isobservable.
doc/guides/prog_guide/writing_efficient_code.rst +5
-0+ +Correspondingly, on the reader side, the guard variable should be read +with ACQUIRE memory order. The payload or the data the writer +communicated, can be read with RELAXED memory order. This ensures that, +if the store to guard variable is observable, the store to payload is alsoobservable.
+
Coding Considerations
-----------------------
2.7.4
Re: [dpdk-dev] [PATCH v8 1/3] doc: add optimizations using C11 atomic built-ins
Honnappa Nagarahalli <[email protected]> writes:
<snip>
Quoted 5 lines
Quoted 5 lines
Subject: [PATCH v8 1/3] doc: add optimizations using C11 atomic built-ins
Add information about possible optimizations using C11 atomic built-ins.
Quoted 7 lines
Quoted 2 lines
Signed-off-by Phil Yang <[email protected]>Signed-off-by Honnappa Nagarahalli <[email protected]>Thanks for the changes, they look good now.
David wanted to change 'built-ins' to 'builtins', otherwise
Reviewed-by Honnappa Nagarahalli <[email protected]>
Will do in the next version.
Thanks.
Quoted 101 lines
Quoted 14 lines
--- doc/guides/prog_guide/writing_efficient_code.rst | 59 +++++++++++++++++++++++- 1 file changed, 58 insertions(+), 1 deletion(-)doc/guides/prog_guide/writing_efficient_code.rst +0
-0@@ -167,7 +167,13 @@ but with the added cost of lower throughput. Locks and Atomic Operations ---------------------------doc/guides/prog_guide/writing_efficient_code.rst but with the added cost of lower throughput. · line 177
Quoted 10 lines
doc/guides/prog_guide/writing_efficient_code.rst +7
-1-Atomic operations imply a lock prefix before the instruction, +This section describes some key considerations when using locks and +atomic operations in the DPDK environment. + +Locks +~~~~~ + +On x86, atomic operations imply a lock prefix before the instruction, causing the processor's LOCK# signal to be asserted during execution ofthe
Quoted 7 lines
following instruction.
This has a big impact on performance in a multicore environment.doc/guides/prog_guide/writing_efficient_code.rst +0
-0@@ -176,6 +182,57 @@ It can often be replaced by other solutions like per-lcore variables.
Also, some locking techniques are more efficient than others. For instance, the Read-Copy-Update (RCU) algorithm can frequentlyreplace
doc/guides/prog_guide/writing_efficient_code.rst
Quoted 9 lines
simple rwlocks.
doc/guides/prog_guide/writing_efficient_code.rst +7
-0+Atomic Operations: Use C11 Atomic Built-ins +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +DPDK generic rte_atomic operations are implemented by __sync built-ins. +These __sync built-ins result in full barriers on aarch64, which are +unnecessary in many use cases. They can be replaced by __atomic +built-ins that conform to the C11 memory model and provide finermemory
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables.
Quoted 13 lines
order control.
doc/guides/prog_guide/writing_efficient_code.rst +12
-0+ +So replacing the rte_atomic operations with __atomic built-ins might +improve performance for aarch64 machines. + +Some typical optimization cases are listed below: + +Atomicity +^^^^^^^^^ + +Some use cases require atomicity alone, the ordering of the memory +operations does not matter. For example, the packet statistics counters +need to be incremented atomically but do not need any particularmemory
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables.
Quoted 13 lines
ordering.
doc/guides/prog_guide/writing_efficient_code.rst +12
-0+So, RELAXED memory ordering is sufficient. + +One-way Barrier +^^^^^^^^^^^^^^^ + +Some use cases allow for memory reordering in one way while requiring +memory ordering in the other direction. + +For example, the memory operations before the spinlock lock are allowed +to move to the critical section, but the memory operations in the +critical section are not allowed to move above the lock. In this case, +the full memory barrier in the compare-and-swap operation can bereplaced
doc/guides/prog_guide/writing_efficient_code.rst
Quoted 10 lines
with ACQUIRE memory order.
doc/guides/prog_guide/writing_efficient_code.rst +9
-0+On the other hand, the memory operations after the spinlock unlock are +allowed to move to the critical section, but the memory operations in +the critical section are not allowed to move below the unlock. So the +full barrier in the store operation can use RELEASE memory order. + +Reader-Writer Concurrency +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Lock-free reader-writer concurrency is one of the common use cases inDPDK.
doc/guides/prog_guide/writing_efficient_code.rst It can often be replaced by other solutions like per-lcore variables.
Quoted 10 lines
doc/guides/prog_guide/writing_efficient_code.rst +5
-0+ +The payload or the data that the writer wants to communicate to the +reader, can be written with RELAXED memory order. However, the guard +variable should be written with RELEASE memory order. This ensures that +the store to guard variable is observable only after the store to payload isobservable.
doc/guides/prog_guide/writing_efficient_code.rst +4
-0+ +Correspondingly, on the reader side, the guard variable should be read +with ACQUIRE memory order. The payload or the data the writer +communicated, can be read with RELAXED memory order. This ensuresthat,
Quoted 8 lines
+if the store to guard variable is observable, the store to payload is also observable.
+
Coding Considerations
-----------------------
2.7.4
PATCH v8 2/3 devtools: prevent use of rte atomic APIs in future patches
In reply to Phil YangIn order to deprecate the rte_atomic and rte_smp barrier APIs, prevent the patches from using these APIs in the converted modules and compilers __sync built-ins in all modules.
The converted modules:lib/librte_distributorlib/librte_hashlib/librte_knilib/librte_lpmlib/librte_rculib/librte_ringlib/librte_stacklib/librte_vhostlib/librte_timerlib/librte_ipsecdrivers/event/octeontxdrivers/event/octeontx2drivers/event/opdldrivers/net/bnx2xdrivers/net/hinicdrivers/net/hns3drivers/net/memifdrivers/net/thunderxdrivers/net/virtio
examples/l2fwd-event
On x86 the __atomic_thread_fence(__ATOMIC_SEQ_CST) is quite expensive for SMP case. Flag the new code which use __atomic_thread_fence API.
--- devtools/checkpatches.sh | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+)
devtools/checkpatches.sh +40 -0
@@ -51,6 +51,13 @@ print_usage () { check_forbidden_additions() { # <patch> res=0 + c11_atomics_dir="lib/librte_distributor lib/librte_hash lib/librte_kni + lib/librte_lpm lib/librte_rcu lib/librte_ring + lib/librte_stack lib/librte_vhost + drivers/event/octeontx drivers/event/octeontx2 + drivers/event/opdl drivers/net/bnx2x drivers/net/hinic + drivers/net/hns3 drivers/net/memif drivers/net/thunderx + drivers/net/virtio examples/l2fwd-event" # refrain from new additions of rte_panic() and rte_exit() # multiple folders and expressions are separated by spaces
@@ -74,6 +81,39 @@ check_forbidden_additions() { # <patch> -v EXPRESSIONS='for[[:space:]]*\\((char|u?int|unsigned|s?size_t)' \ -v RET_ON_FAIL=1 \ -v MESSAGE='Declaring a variable inside for()' \ + + # refrain from new additions of 16/32/64 bits rte_atomic_xxx() + # multiple folders and expressions are separated by spaces + awk -v FOLDERS="$c11_atomics_dir" \ + -v EXPRESSIONS="rte_atomic[0-9][0-9]_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of rte_atomicNN_xxx APIs not allowed, use __atomic_xxx built-ins' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from new additions of rte_smp_XXmb() + # multiple folders and expressions are separated by spaces + awk -v FOLDERS="$c11_atomics_dir" \ + -v EXPRESSIONS="rte_smp_(r|w)?mb\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of rte_smp_[r/w]mb not allowed, use __atomic_xxx built-ins' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from using compiler __sync built-ins + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="__sync_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of __sync_xxx built-ins not allowed, use __atomic_xxx built-ins' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from using compiler __atomic_thread_fence() + # It should be avoided on x86 for SMP case. + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="__atomic_thread_fence\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='The __atomic_thread_fence is not allowed, use rte_atomic_thread_fence' \ -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ "$1" || res=1
Signature · 1 line
2.7.4
Re: [dpdk-dev] [PATCH v8 2/3] devtools: prevent use of rte atomic APIs in future patches
Quoted 4 lines · Phil Yang
In order to deprecate the
rte_atomicandrte_smpbarrier APIs, prevent the patches from using these APIs in the converted modules and compilers__syncbuilt-ins in all modules.
builtins* for the whole patch.
Quoted 37 lines
The converted modules:
lib/librte_distributorlib/librte_hashlib/librte_knilib/librte_lpmlib/librte_rculib/librte_ringlib/librte_stacklib/librte_vhostlib/librte_timerlib/librte_ipsecdrivers/event/octeontxdrivers/event/octeontx2drivers/event/opdldrivers/net/bnx2xdrivers/net/hinicdrivers/net/hns3drivers/net/memifdrivers/net/thunderxdrivers/net/virtio
examples/l2fwd-eventOn x86 the
__atomic_thread_fence(__ATOMIC_SEQ_CST)is quite expensive for SMP case. Flag the new code which use__atomic_thread_fenceAPI.Signed-off-by Phil Yang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>--- devtools/checkpatches.sh | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+)devtools/checkpatches.sh +0
-0
devtools/checkpatches.sh print_usage () { · line 60
Quoted 10 lines
devtools/checkpatches.sh +7
-0@@ -51,6 +51,13 @@ print_usage () { check_forbidden_additions() { # <patch> res=0 + c11_atomics_dir="lib/librte_distributor lib/librte_hash lib/librte_kni + lib/librte_lpm lib/librte_rcu lib/librte_ring + lib/librte_stack lib/librte_vhost + drivers/event/octeontx drivers/event/octeontx2 + drivers/event/opdl drivers/net/bnx2x drivers/net/hinic + drivers/net/hns3 drivers/net/memif drivers/net/thunderx + drivers/net/virtio examples/l2fwd-event"
I prefer a form like:
+ c11_atomics_dir="" + c11_atomics_dir="$c11_atomics_dir drivers/event/octeontx" + c11_atomics_dir="$c11_atomics_dir drivers/event/octeontx2" + c11_atomics_dir="$c11_atomics_dir drivers/event/opdl" + c11_atomics_dir="$c11_atomics_dir drivers/net/bnx2x" + c11_atomics_dir="$c11_atomics_dir drivers/net/hinic" + c11_atomics_dir="$c11_atomics_dir drivers/net/hns3" + c11_atomics_dir="$c11_atomics_dir drivers/net/memif" + c11_atomics_dir="$c11_atomics_dir drivers/net/thunderx" + c11_atomics_dir="$c11_atomics_dir drivers/net/virtio" + c11_atomics_dir="$c11_atomics_dir examples/l2fwd-event" + c11_atomics_dir="$c11_atomics_dir lib/librte_distributor" + c11_atomics_dir="$c11_atomics_dir lib/librte_hash" + c11_atomics_dir="$c11_atomics_dir lib/librte_kni" + c11_atomics_dir="$c11_atomics_dir lib/librte_lpm" + c11_atomics_dir="$c11_atomics_dir lib/librte_rcu" + c11_atomics_dir="$c11_atomics_dir lib/librte_ring" + c11_atomics_dir="$c11_atomics_dir lib/librte_stack" + c11_atomics_dir="$c11_atomics_dir lib/librte_vhost"
Easier to read and update.
devtools/checkpatches.sh check_forbidden_additions() { # <patch> · line 83
Quoted 7 lines
devtools/checkpatches.sh +0
-0# refrain from new additions of rte_panic() and rte_exit() # multiple folders and expressions are separated by spaces@@ -74,6 +81,39 @@ check_forbidden_additions() { # <patch> -v EXPRESSIONS='for[[:space:]]*\\((char|u?int|unsigned|s?size_t)' \ -v RET_ON_FAIL=1 \ -v MESSAGE='Declaring a variable inside for()' \
This hunk breaks the script.
This is not seen by the CI, as the CI uses its local copy of the script.
We are missing a:
+ "$1" || res=1
devtools/checkpatches.sh check_forbidden_additions() { # <patch> · line 90
Quoted 7 lines
devtools/checkpatches.sh +7
-0+ + # refrain from new additions of 16/32/64 bits rte_atomic_xxx() + # multiple folders and expressions are separated by spaces + awk -v FOLDERS="$c11_atomics_dir" \ + -v EXPRESSIONS="rte_atomic[0-9][0-9]_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of rte_atomicNN_xxx APIs not allowed, use __atomic_xxx built-ins' \
Other messages in this script report what is wrong. Example:
-v MESSAGE='Using rte_panic/rte_exit' \
-v MESSAGE='Using compiler attribute directly' \
-v MESSAGE='Declaring a variable inside for()' \So to be consistent,
+ -v MESSAGE='Using rte_atomicNN_xxx' \
devtools/checkpatches.sh check_forbidden_additions() { # <patch> · line 99
Quoted 9 lines
devtools/checkpatches.sh +9
-0+ -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from new additions of rte_smp_XXmb() + # multiple folders and expressions are separated by spaces + awk -v FOLDERS="$c11_atomics_dir" \ + -v EXPRESSIONS="rte_smp_(r|w)?mb\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of rte_smp_[r/w]mb not allowed, use __atomic_xxx built-ins' \
+ -v MESSAGE='Using rte_smp_[r/w]mb' \
devtools/checkpatches.sh check_forbidden_additions() { # <patch> · line 107
Quoted 8 lines
devtools/checkpatches.sh +8
-0+ -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from using compiler __sync built-ins + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="__sync_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of __sync_xxx built-ins not allowed, use __atomic_xxx built-ins' \
+ -v MESSAGE='Using __sync_xxx builtins' \
devtools/checkpatches.sh check_forbidden_additions() { # <patch> · line 116
Quoted 9 lines
devtools/checkpatches.sh +9
-0+ -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from using compiler __atomic_thread_fence() + # It should be avoided on x86 for SMP case. + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="__atomic_thread_fence\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='The __atomic_thread_fence is not allowed, use rte_atomic_thread_fence' \
+ -v MESSAGE='Using __atomic_thread_fence' \
Quoted 3 lines
devtools/checkpatches.sh +0
-0-f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ "$1" || res=1
Signature · 1 line
David Marchand
Re: [dpdk-dev] [PATCH v8 2/3] devtools: prevent use of rte atomic APIs in future patches
16/07/2020 12:48, David Marchand:
Quoted 33 lines · Phil Yang
devtools/checkpatches.sh print_usage () { · line 60
Quoted 9 lines
check_forbidden_additions(){ # <patch>res=0devtools/checkpatches.sh +7
-0+ c11_atomics_dir="lib/librte_distributor lib/librte_hash lib/librte_kni + lib/librte_lpm lib/librte_rcu lib/librte_ring + lib/librte_stack lib/librte_vhost + drivers/event/octeontx drivers/event/octeontx2 + drivers/event/opdl drivers/net/bnx2x drivers/net/hinic + drivers/net/hns3 drivers/net/memif drivers/net/thunderx + drivers/net/virtio examples/l2fwd-event"I prefer a form like:
(quoted diff) +19
-0+ c11_atomics_dir="" + c11_atomics_dir="$c11_atomics_dir drivers/event/octeontx" + c11_atomics_dir="$c11_atomics_dir drivers/event/octeontx2" + c11_atomics_dir="$c11_atomics_dir drivers/event/opdl" + c11_atomics_dir="$c11_atomics_dir drivers/net/bnx2x" + c11_atomics_dir="$c11_atomics_dir drivers/net/hinic" + c11_atomics_dir="$c11_atomics_dir drivers/net/hns3" + c11_atomics_dir="$c11_atomics_dir drivers/net/memif" + c11_atomics_dir="$c11_atomics_dir drivers/net/thunderx" + c11_atomics_dir="$c11_atomics_dir drivers/net/virtio" + c11_atomics_dir="$c11_atomics_dir examples/l2fwd-event" + c11_atomics_dir="$c11_atomics_dir lib/librte_distributor" + c11_atomics_dir="$c11_atomics_dir lib/librte_hash" + c11_atomics_dir="$c11_atomics_dir lib/librte_kni" + c11_atomics_dir="$c11_atomics_dir lib/librte_lpm" + c11_atomics_dir="$c11_atomics_dir lib/librte_rcu" + c11_atomics_dir="$c11_atomics_dir lib/librte_ring" + c11_atomics_dir="$c11_atomics_dir lib/librte_stack" + c11_atomics_dir="$c11_atomics_dir lib/librte_vhost"
Quoted 1 line
Easier to read and update.
Why do we need this list at all?
Are we allowed to add new code with old atomics in other directories?
How bad it is to have a warning on non-converted libs?
Re: [dpdk-dev] [PATCH v8 2/3] devtools: prevent use of rte atomic APIs in future patches
Hello,
Quoted 37 lines · Thomas Monjalon
Subject: Re: [dpdk-dev] [PATCH v8 2/3] devtools: prevent use of rte atomic APIs in future patches
16/07/2020 12:48, David Marchand:
Quoted 33 lines · Phil Yang
devtools/checkpatches.sh print_usage () { · line 60
Quoted 9 lines
check_forbidden_additions(){ # <patch>res=0devtools/checkpatches.sh +7
-0+ c11_atomics_dir="lib/librte_distributor lib/librte_hash lib/librte_kni + lib/librte_lpm lib/librte_rcu lib/librte_ring + lib/librte_stack lib/librte_vhost + drivers/event/octeontx drivers/event/octeontx2 + drivers/event/opdl drivers/net/bnx2x drivers/net/hinic + drivers/net/hns3 drivers/net/memif drivers/net/thunderx + drivers/net/virtio examples/l2fwd-event"I prefer a form like:
(quoted diff) +19
-0+ c11_atomics_dir="" + c11_atomics_dir="$c11_atomics_dir drivers/event/octeontx" + c11_atomics_dir="$c11_atomics_dir drivers/event/octeontx2" + c11_atomics_dir="$c11_atomics_dir drivers/event/opdl" + c11_atomics_dir="$c11_atomics_dir drivers/net/bnx2x" + c11_atomics_dir="$c11_atomics_dir drivers/net/hinic" + c11_atomics_dir="$c11_atomics_dir drivers/net/hns3" + c11_atomics_dir="$c11_atomics_dir drivers/net/memif" + c11_atomics_dir="$c11_atomics_dir drivers/net/thunderx" + c11_atomics_dir="$c11_atomics_dir drivers/net/virtio" + c11_atomics_dir="$c11_atomics_dir examples/l2fwd-event" + c11_atomics_dir="$c11_atomics_dir lib/librte_distributor" + c11_atomics_dir="$c11_atomics_dir lib/librte_hash" + c11_atomics_dir="$c11_atomics_dir lib/librte_kni" + c11_atomics_dir="$c11_atomics_dir lib/librte_lpm" + c11_atomics_dir="$c11_atomics_dir lib/librte_rcu" + c11_atomics_dir="$c11_atomics_dir lib/librte_ring" + c11_atomics_dir="$c11_atomics_dir lib/librte_stack" + c11_atomics_dir="$c11_atomics_dir lib/librte_vhost"
Quoted 6 lines
Quoted 1 line
Easier to read and update.
Why do we need this list at all?
Are we allowed to add new code with old atomics in other directories?
How bad it is to have a warning on non-converted libs?
From my perspective, the pros of this list are :
1. Avoid introducing false warnings in non-converted modules. Otherwise, the maintainers have to wonder if that module is converted or not.
2. Keep non-converted modules compatible. C11 atomic builtins cannot be used directly for rte_atomicXX_t variables.
The cons are :
1. The list needs updating every time we convert a module.
2. The script is not elegant as before.
Thanks,
Phil
Re: [dpdk-dev] [PATCH v8 2/3] devtools: prevent use of rte atomic APIs in future patches
16/07/2020 18:37, Phil Yang:
Quoted 43 lines
Hello,
Quoted 39 lines · Thomas Monjalon
Subject: Re: [dpdk-dev] [PATCH v8 2/3] devtools: prevent use of rte atomic APIs in future patches
16/07/2020 12:48, David Marchand:
Quoted 34 lines · Phil Yang
devtools/checkpatches.sh print_usage () { · line 60
Quoted 9 lines
check_forbidden_additions(){ # <patch>res=0devtools/checkpatches.sh +7
-0+ c11_atomics_dir="lib/librte_distributor lib/librte_hash lib/librte_kni + lib/librte_lpm lib/librte_rcu lib/librte_ring + lib/librte_stack lib/librte_vhost + drivers/event/octeontx drivers/event/octeontx2 + drivers/event/opdl drivers/net/bnx2x drivers/net/hinic + drivers/net/hns3 drivers/net/memif drivers/net/thunderx + drivers/net/virtio examples/l2fwd-event"I prefer a form like:
(quoted diff) +19
-0+ c11_atomics_dir="" + c11_atomics_dir="$c11_atomics_dir drivers/event/octeontx" + c11_atomics_dir="$c11_atomics_dir drivers/event/octeontx2" + c11_atomics_dir="$c11_atomics_dir drivers/event/opdl" + c11_atomics_dir="$c11_atomics_dir drivers/net/bnx2x" + c11_atomics_dir="$c11_atomics_dir drivers/net/hinic" + c11_atomics_dir="$c11_atomics_dir drivers/net/hns3" + c11_atomics_dir="$c11_atomics_dir drivers/net/memif" + c11_atomics_dir="$c11_atomics_dir drivers/net/thunderx" + c11_atomics_dir="$c11_atomics_dir drivers/net/virtio" + c11_atomics_dir="$c11_atomics_dir examples/l2fwd-event" + c11_atomics_dir="$c11_atomics_dir lib/librte_distributor" + c11_atomics_dir="$c11_atomics_dir lib/librte_hash" + c11_atomics_dir="$c11_atomics_dir lib/librte_kni" + c11_atomics_dir="$c11_atomics_dir lib/librte_lpm" + c11_atomics_dir="$c11_atomics_dir lib/librte_rcu" + c11_atomics_dir="$c11_atomics_dir lib/librte_ring" + c11_atomics_dir="$c11_atomics_dir lib/librte_stack" + c11_atomics_dir="$c11_atomics_dir lib/librte_vhost"Easier to read and update.
Quoted 7 lines
Quoted 4 lines
Why do we need this list at all?
Are we allowed to add new code with old atomics in other directories?
How bad it is to have a warning on non-converted libs?From my perspective, the pros of this list are :
1. Avoid introducing false warnings in non-converted modules. Otherwise, the maintainers have to wonder if that module is converted or not.
Don't we want to convert all libs?
If we are adding one more rte_atomic in a lib, we should ask the question why not converting to C11, no?
Quoted 5 lines
2. Keep non-converted modules compatible. C11 atomic builtins cannot be used directly for
rte_atomicXX_tvariables.The cons are :
1. The list needs updating every time we convert a module.
2. The script is not elegant as before.
Re: [dpdk-dev] [PATCH v8 2/3] devtools: prevent use of rte atomic APIs in future patches
<snip>
Quoted 47 lines
Quoted 9 lines
Quoted 9 lines
Subject: Re: [dpdk-dev] [PATCH v8 2/3] devtools: prevent use of rte atomic APIs in future patches
16/07/2020 12:48, David Marchand:
Quoted 4 lines · Phil Yang
Quoted 3 lines
check_forbidden_additions(){ # <patch>res=0+ c11_atomics_dir="lib/librte_distributor
lib/librte_hash
lib/librte_kniQuoted 5 lines
Quoted 5 lines
Quoted 5 lines
devtools/checkpatches.sh print_usage () {
Quoted 5 lines
devtools/checkpatches.sh +5
-0+ lib/librte_lpm lib/librte_rcu lib/librte_ring + lib/librte_stack lib/librte_vhost + drivers/event/octeontx drivers/event/octeontx2 + drivers/event/opdl drivers/net/bnx2x drivers/net/hinic + drivers/net/hns3 drivers/net/memif
drivers/net/thunderxQuoted 31 lines
Quoted 30 lines
Quoted 24 lines
Quoted 1 line
+
drivers/net/virtioexamples/l2fwd-event"I prefer a form like:
(quoted diff) +19
-0+ c11_atomics_dir="" + c11_atomics_dir="$c11_atomics_dir drivers/event/octeontx" + c11_atomics_dir="$c11_atomics_dir drivers/event/octeontx2" + c11_atomics_dir="$c11_atomics_dir drivers/event/opdl" + c11_atomics_dir="$c11_atomics_dir drivers/net/bnx2x" + c11_atomics_dir="$c11_atomics_dir drivers/net/hinic" + c11_atomics_dir="$c11_atomics_dir drivers/net/hns3" + c11_atomics_dir="$c11_atomics_dir drivers/net/memif" + c11_atomics_dir="$c11_atomics_dir drivers/net/thunderx" + c11_atomics_dir="$c11_atomics_dir drivers/net/virtio" + c11_atomics_dir="$c11_atomics_dir examples/l2fwd-event" + c11_atomics_dir="$c11_atomics_dir lib/librte_distributor" + c11_atomics_dir="$c11_atomics_dir lib/librte_hash" + c11_atomics_dir="$c11_atomics_dir lib/librte_kni" + c11_atomics_dir="$c11_atomics_dir lib/librte_lpm" + c11_atomics_dir="$c11_atomics_dir lib/librte_rcu" + c11_atomics_dir="$c11_atomics_dir lib/librte_ring" + c11_atomics_dir="$c11_atomics_dir lib/librte_stack" + c11_atomics_dir="$c11_atomics_dir lib/librte_vhost"Quoted 1 line
Easier to read and update.
Why do we need this list at all?
Are we allowed to add new code with old atomics in other directories?
How bad it is to have a warning on non-converted libs?
Quoted 5 lines
Quoted 2 lines
From my perspective, the pros of this list are :
1. Avoid introducing false warnings in non-converted modules. Otherwise,the maintainers have to wonder if that module is converted or not.
Don't we want to convert all libs?
The goal is to convert all the libs.
Quoted 2 lines
If we are adding one more
rte_atomicin a lib, we should ask the question why not converting to C11, no?
Agree, I am fine with this approach. That will kind of distribute the conversion work as well.
Quoted 6 lines
Quoted 2 lines
2. Keep non-converted modules compatible. C11 atomic builtins cannot be
used directly for
rte_atomicXX_tvariables.Quoted 3 lines
The cons are :
1. The list needs updating every time we convert a module.
This list will go away once all the modules are converted.
Quoted 3 lines
Quoted 1 line
2. The script is not elegant as before.
Re: [dpdk-dev] [PATCH v8 2/3] devtools: prevent use of rte atomic APIs in future patches
Quoted 61 lines · On Thu, 16 Jul 2020 16:59:11 +0000 Honnappa Nagarahalli…
<snip>
Quoted 47 lines
Quoted 9 lines
Quoted 9 lines
Subject: Re: [dpdk-dev] [PATCH v8 2/3] devtools: prevent use of rte atomic APIs in future patches
16/07/2020 12:48, David Marchand:
Quoted 4 lines · Phil Yang
Quoted 3 lines
check_forbidden_additions(){ # <patch>res=0+ c11_atomics_dir="lib/librte_distributor
lib/librte_hash
lib/librte_kniQuoted 5 lines
Quoted 5 lines
Quoted 5 lines
devtools/checkpatches.sh print_usage () {
Quoted 5 lines
devtools/checkpatches.sh +5
-0+ lib/librte_lpm lib/librte_rcu lib/librte_ring + lib/librte_stack lib/librte_vhost + drivers/event/octeontx drivers/event/octeontx2 + drivers/event/opdl drivers/net/bnx2x drivers/net/hinic + drivers/net/hns3 drivers/net/memif
drivers/net/thunderxQuoted 31 lines
Quoted 30 lines
Quoted 24 lines
Quoted 1 line
+
drivers/net/virtioexamples/l2fwd-event"I prefer a form like:
(quoted diff) +19
-0+ c11_atomics_dir="" + c11_atomics_dir="$c11_atomics_dir drivers/event/octeontx" + c11_atomics_dir="$c11_atomics_dir drivers/event/octeontx2" + c11_atomics_dir="$c11_atomics_dir drivers/event/opdl" + c11_atomics_dir="$c11_atomics_dir drivers/net/bnx2x" + c11_atomics_dir="$c11_atomics_dir drivers/net/hinic" + c11_atomics_dir="$c11_atomics_dir drivers/net/hns3" + c11_atomics_dir="$c11_atomics_dir drivers/net/memif" + c11_atomics_dir="$c11_atomics_dir drivers/net/thunderx" + c11_atomics_dir="$c11_atomics_dir drivers/net/virtio" + c11_atomics_dir="$c11_atomics_dir examples/l2fwd-event" + c11_atomics_dir="$c11_atomics_dir lib/librte_distributor" + c11_atomics_dir="$c11_atomics_dir lib/librte_hash" + c11_atomics_dir="$c11_atomics_dir lib/librte_kni" + c11_atomics_dir="$c11_atomics_dir lib/librte_lpm" + c11_atomics_dir="$c11_atomics_dir lib/librte_rcu" + c11_atomics_dir="$c11_atomics_dir lib/librte_ring" + c11_atomics_dir="$c11_atomics_dir lib/librte_stack" + c11_atomics_dir="$c11_atomics_dir lib/librte_vhost"Quoted 1 line
Easier to read and update.
Why do we need this list at all?
Are we allowed to add new code with old atomics in other directories?
How bad it is to have a warning on non-converted libs?Quoted 5 lines
Quoted 2 lines
From my perspective, the pros of this list are :
1. Avoid introducing false warnings in non-converted modules. Otherwise,the maintainers have to wonder if that module is converted or not.
Don't we want to convert all libs?
The goal is to convert all the libs.
Quoted 2 lines
If we are adding one more
rte_atomicin a lib, we should ask the question why not converting to C11, no?Agree, I am fine with this approach. That will kind of distribute the conversion work as well.
Quoted 1 line
Quoted 11 lines
Quoted 5 lines
Quoted 1 line
2. Keep non-converted modules compatible. C11 atomic builtins cannot be
used directly for
rte_atomicXX_tvariables.Quoted 3 lines
The cons are :
1. The list needs updating every time we convert a module.This list will go away once all the modules are converted.
Quoted 3 lines
Quoted 1 line
2. The script is not elegant as before.
Can't the conversion just be automated with something coccinelle?
Re: [dpdk-dev] [PATCH v8 2/3] devtools: prevent use of rte atomic APIs in future patches
<snip>
Quoted 71 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Subject: Re: [dpdk-dev] [PATCH v8 2/3] devtools: prevent use of rte atomic APIs in future patches
16/07/2020 12:48, David Marchand:
Quoted 1 line
On Thu, Jul 16, 2020 at 6:58 AM Phil Yang <[email protected]>
Quoted 9 lines · wrote:
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
check_forbidden_additions(){ # <patch>res=0+ c11_atomics_dir="lib/librte_distributor
+lib/librte_hash
lib/librte_kniQuoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
(quoted diff) +4
-0+ lib/librte_lpm lib/librte_rcu lib/librte_ring + lib/librte_stack lib/librte_vhost + drivers/event/octeontx drivers/event/octeontx2 + drivers/event/opdl drivers/net/bnx2x
drivers/net/hinicQuoted 36 lines
Quoted 36 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+
drivers/net/hns3drivers/net/memif
drivers/net/thunderxQuoted 34 lines
Quoted 24 lines
Quoted 24 lines
Quoted 1 line
+
drivers/net/virtioexamples/l2fwd-event"I prefer a form like:
(quoted diff) +19
-0+ c11_atomics_dir="" + c11_atomics_dir="$c11_atomics_dir drivers/event/octeontx" + c11_atomics_dir="$c11_atomics_dir drivers/event/octeontx2" + c11_atomics_dir="$c11_atomics_dir drivers/event/opdl" + c11_atomics_dir="$c11_atomics_dir drivers/net/bnx2x" + c11_atomics_dir="$c11_atomics_dir drivers/net/hinic" + c11_atomics_dir="$c11_atomics_dir drivers/net/hns3" + c11_atomics_dir="$c11_atomics_dir drivers/net/memif" + c11_atomics_dir="$c11_atomics_dir drivers/net/thunderx" + c11_atomics_dir="$c11_atomics_dir drivers/net/virtio" + c11_atomics_dir="$c11_atomics_dir examples/l2fwd-event" + c11_atomics_dir="$c11_atomics_dir lib/librte_distributor" + c11_atomics_dir="$c11_atomics_dir lib/librte_hash" + c11_atomics_dir="$c11_atomics_dir lib/librte_kni" + c11_atomics_dir="$c11_atomics_dir lib/librte_lpm" + c11_atomics_dir="$c11_atomics_dir lib/librte_rcu" + c11_atomics_dir="$c11_atomics_dir lib/librte_ring" + c11_atomics_dir="$c11_atomics_dir lib/librte_stack" + c11_atomics_dir="$c11_atomics_dir lib/librte_vhost"Quoted 6 lines
Quoted 1 line
Easier to read and update.
Why do we need this list at all?
Are we allowed to add new code with old atomics in other directories?
How bad it is to have a warning on non-converted libs?From my perspective, the pros of this list are :
1. Avoid introducing false warnings in non-converted modules. Otherwise,Quoted 8 lines
Quoted 3 lines
the maintainers have to wonder if that module is converted or not.
Don't we want to convert all libs?
The goal is to convert all the libs.
Quoted 2 lines
If we are adding one more
rte_atomicin a lib, we should ask the question why not converting to C11, no?Agree, I am fine with this approach. That will kind of distribute the
conversion work as well.
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
2. Keep non-converted modules compatible. C11 atomic builtins cannot be
used directly for
rte_atomicXX_tvariables.Quoted 3 lines
The cons are :
1. The list needs updating every time we convert a module.
Quoted 8 lines
Quoted 6 lines
This list will go away once all the modules are converted.
Quoted 3 lines
Quoted 1 line
2. The script is not elegant as before.
Can't the conversion just be automated with something coccinelle?
I do not know what 'coccinelle' means, not much help on the internet as well.
I really have not thought about automation. But, working on these conversions, I have realized that understanding of the synchronizations used is required (basically, understanding the entire module) to ensure the correct barriers are used. We could use the most strongest barrier and functionality would be fine, but the performance would take a hit.
Re: [dpdk-dev] [PATCH v8 2/3] devtools: prevent use of rte atomic APIs in future patches
Quoted 22 lines · On Fri, 17 Jul 2020 22:48:04 +0000 Honnappa Nagarahalli…
Quoted 22 lines
Quoted 6 lines
Quoted 1 line
Don't we want to convert all libs?
The goal is to convert all the libs.
Quoted 2 lines
If we are adding one more
rte_atomicin a lib, we should ask the question why not converting to C11, no?Agree, I am fine with this approach. That will kind of distribute the
conversion work as well.
Quoted 14 lines
Quoted 8 lines
Quoted 4 lines
2. Keep non-converted modules compatible. C11 atomic builtins cannot be
used directly for
rte_atomicXX_tvariables.Quoted 3 lines
The cons are :
1. The list needs updating every time we convert a module.This list will go away once all the modules are converted.
Quoted 3 lines
Quoted 1 line
2. The script is not elegant as before.
Quoted 4 lines
Quoted 1 line
Can't the conversion just be automated with something coccinelle?
I do not know what 'coccinelle' means, not much help on the internet as well.
I really have not thought about automation. But, working on these conversions, I have realized that understanding of the synchronizations used is required (basically, understanding the entire module) to ensure the correct barriers are used. We could use the most strongest barrier and functionality would be fine, but the performance would take a hit.
coccinelle is a semantic patch tool, it allows for changes that are know how to deal with arguments to functions etc. It is widely used in Linux kernel to do tree wide changes (for example adding a new argument to an internal function).
https://www.kernel.org/doc/html/latest/dev-tools/coccinelle.html
Re: [dpdk-dev] [PATCH v8 2/3] devtools: prevent use of rte atomic APIs in future patches
In reply to Honnappa NagarahalliHonnappa Nagarahalli <[email protected]> writes:
<snip>
Quoted 53 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Subject: Re: [dpdk-dev] [PATCH v8 2/3] devtools: prevent use of rte atomic APIs in future patches
Quoted 10 lines
Quoted 5 lines
Quoted 5 lines
16/07/2020 12:48, David Marchand:
Quoted 4 lines · Phil Yang
Quoted 3 lines
check_forbidden_additions(){ # <patch>res=0+ c11_atomics_dir="lib/librte_distributor
lib/librte_hash
lib/librte_kniQuoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
devtools/checkpatches.sh +4
-0+ lib/librte_lpm lib/librte_rcu lib/librte_ring + lib/librte_stack lib/librte_vhost + drivers/event/octeontx drivers/event/octeontx2 + drivers/event/opdl drivers/net/bnx2x
drivers/net/hinicQuoted 37 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
+
drivers/net/hns3drivers/net/memif
drivers/net/thunderxQuoted 26 lines
Quoted 26 lines
Quoted 25 lines
Quoted 1 line
+
drivers/net/virtioexamples/l2fwd-event"I prefer a form like:
(quoted diff) +19
-0+ c11_atomics_dir="" + c11_atomics_dir="$c11_atomics_dir drivers/event/octeontx" + c11_atomics_dir="$c11_atomics_dir drivers/event/octeontx2" + c11_atomics_dir="$c11_atomics_dir drivers/event/opdl" + c11_atomics_dir="$c11_atomics_dir drivers/net/bnx2x" + c11_atomics_dir="$c11_atomics_dir drivers/net/hinic" + c11_atomics_dir="$c11_atomics_dir drivers/net/hns3" + c11_atomics_dir="$c11_atomics_dir drivers/net/memif" + c11_atomics_dir="$c11_atomics_dir drivers/net/thunderx" + c11_atomics_dir="$c11_atomics_dir drivers/net/virtio" + c11_atomics_dir="$c11_atomics_dir examples/l2fwd-event" + c11_atomics_dir="$c11_atomics_dir lib/librte_distributor" + c11_atomics_dir="$c11_atomics_dir lib/librte_hash" + c11_atomics_dir="$c11_atomics_dir lib/librte_kni" + c11_atomics_dir="$c11_atomics_dir lib/librte_lpm" + c11_atomics_dir="$c11_atomics_dir lib/librte_rcu" + c11_atomics_dir="$c11_atomics_dir lib/librte_ring" + c11_atomics_dir="$c11_atomics_dir lib/librte_stack" + c11_atomics_dir="$c11_atomics_dir lib/librte_vhost"Easier to read and update.
Quoted 7 lines
Quoted 4 lines
Why do we need this list at all?
Are we allowed to add new code with old atomics in other directories?
How bad it is to have a warning on non-converted libs?From my perspective, the pros of this list are :
1. Avoid introducing false warnings in non-converted modules. Otherwise,the maintainers have to wonder if that module is converted or not.
Quoted 7 lines
Quoted 1 line
Don't we want to convert all libs?
The goal is to convert all the libs.
Quoted 2 lines
If we are adding one more
rte_atomicin a lib, we should ask the question why not converting to C11, no?Agree, I am fine with this approach. That will kind of distribute the conversion work as well.
Will do in V9.
Thanks,
Phil
Quoted 14 lines
Quoted 3 lines
Quoted 3 lines
2. Keep non-converted modules compatible. C11 atomic builtins cannot
be
Quoted 4 lines
used directly for
rte_atomicXX_tvariables.Quoted 3 lines
The cons are :
1. The list needs updating every time we convert a module.This list will go away once all the modules are converted.
Quoted 3 lines
Quoted 1 line
2. The script is not elegant as before.
Re: [dpdk-dev] [PATCH v8 2/3] devtools: prevent use of rte atomic APIs in future patches
In reply to David MarchandQuoted 13 lines
-----Original Message-----
From: David Marchand <[email protected]>
Sent: Thursday, July 16, 2020 6:49 PM
To: Phil Yang <[email protected]>
Cc: [email protected]; Mcnamara, John <[email protected]>; Honnappa Nagarahalli <[email protected]>; David Christensen <[email protected]>; dev <[email protected]>; [email protected]; Ananyev, Konstantin <[email protected]>; Ola Liljedahl <[email protected]>; Bruce Richardson <[email protected]>; Ruifeng Wang <[email protected]>; nd <[email protected]>
Subject: Re: [PATCH v8 2/3] devtools: prevent use of rte atomic APIs in future patches
Quoted 7 lines
Quoted 4 lines · Phil Yang
In order to deprecate the
rte_atomicandrte_smpbarrier APIs, prevent the patches from using these APIs in the converted modules and compilers__syncbuilt-ins in all modules.builtins* for the whole patch.
Will do.
Quoted 83 lines
Quoted 25 lines
The converted modules:
lib/librte_distributorlib/librte_hashlib/librte_knilib/librte_lpmlib/librte_rculib/librte_ringlib/librte_stacklib/librte_vhostlib/librte_timerlib/librte_ipsecdrivers/event/octeontxdrivers/event/octeontx2drivers/event/opdldrivers/net/bnx2xdrivers/net/hinicdrivers/net/hns3drivers/net/memifdrivers/net/thunderxdrivers/net/virtio
examples/l2fwd-eventQuoted 1 line
On x86 the
__atomic_thread_fence(__ATOMIC_SEQ_CST)is quiteexpensive
Quoted 6 lines
for SMP case. Flag the new code which use
__atomic_thread_fenceAPI.Signed-off-by Phil Yang <[email protected]>Reviewed-by Ruifeng Wang <[email protected]>--- devtools/checkpatches.sh | 40++++++++++++++++++++++++++++++++++++++++
Quoted 7 lines
1 file changed, 40 insertions(+)devtools/checkpatches.sh +0
-0devtools/checkpatches.sh print_usage () { · line 60
Quoted 10 lines
devtools/checkpatches.sh +7
-0@@ -51,6 +51,13 @@ print_usage () { check_forbidden_additions() { # <patch> res=0 + c11_atomics_dir="lib/librte_distributor lib/librte_hash lib/librte_kni + lib/librte_lpm lib/librte_rcu lib/librte_ring + lib/librte_stack lib/librte_vhost + drivers/event/octeontx drivers/event/octeontx2 + drivers/event/opdl drivers/net/bnx2x drivers/net/hinic + drivers/net/hns3 drivers/net/memif drivers/net/thunderx + drivers/net/virtio examples/l2fwd-event"I prefer a form like:
(quoted diff) +19
-0+ c11_atomics_dir="" + c11_atomics_dir="$c11_atomics_dir drivers/event/octeontx" + c11_atomics_dir="$c11_atomics_dir drivers/event/octeontx2" + c11_atomics_dir="$c11_atomics_dir drivers/event/opdl" + c11_atomics_dir="$c11_atomics_dir drivers/net/bnx2x" + c11_atomics_dir="$c11_atomics_dir drivers/net/hinic" + c11_atomics_dir="$c11_atomics_dir drivers/net/hns3" + c11_atomics_dir="$c11_atomics_dir drivers/net/memif" + c11_atomics_dir="$c11_atomics_dir drivers/net/thunderx" + c11_atomics_dir="$c11_atomics_dir drivers/net/virtio" + c11_atomics_dir="$c11_atomics_dir examples/l2fwd-event" + c11_atomics_dir="$c11_atomics_dir lib/librte_distributor" + c11_atomics_dir="$c11_atomics_dir lib/librte_hash" + c11_atomics_dir="$c11_atomics_dir lib/librte_kni" + c11_atomics_dir="$c11_atomics_dir lib/librte_lpm" + c11_atomics_dir="$c11_atomics_dir lib/librte_rcu" + c11_atomics_dir="$c11_atomics_dir lib/librte_ring" + c11_atomics_dir="$c11_atomics_dir lib/librte_stack" + c11_atomics_dir="$c11_atomics_dir lib/librte_vhost"Easier to read and update.
devtools/checkpatches.sh check_forbidden_additions() { # <patch> · line 81
Quoted 5 lines
devtools/checkpatches.sh +0
-0# refrain from new additions of rte_panic() and rte_exit() # multiple folders and expressions are separated by spaces@@ -74,6 +81,39 @@ check_forbidden_additions() { # <patch> -v EXPRESSIONS='for[[:space:]]*\\((char|u?int|unsigned|s?size_t)'\
Quoted 8 lines
devtools/checkpatches.sh check_forbidden_additions() { # <patch> · line 83
Quoted 2 lines
devtools/checkpatches.sh +0
-0-v RET_ON_FAIL=1 \ -v MESSAGE='Declaring a variable inside for()' \This hunk breaks the script.
This is not seen by the CI, as the CI uses its local copy of the script.We are missing a:
+ "$1" || res=1
Good catch. I will fix it in this patch.
Quoted 11 lines
devtools/checkpatches.sh check_forbidden_additions() { # <patch>
Quoted 9 lines
devtools/checkpatches.sh +7
-0+ + # refrain from new additions of 16/32/64 bits rte_atomic_xxx() + # multiple folders and expressions are separated by spaces + awk -v FOLDERS="$c11_atomics_dir" \ + -v EXPRESSIONS="rte_atomic[0-9][0-9]_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of rte_atomicNN_xxx APIs not allowed, use
__atomic_xxxbuilt-ins' \
Quoted 8 lines
Other messages in this script report what is wrong. Example:
-v MESSAGE='Using rte_panic/rte_exit' \ -v MESSAGE='Using compiler attribute directly' \ -v MESSAGE='Declaring a variable inside for()' \So to be consistent,
+ -v MESSAGE='Usingrte_atomicNN_xxx' \
Will do.
Quoted 11 lines
devtools/checkpatches.sh check_forbidden_additions() { # <patch>
Quoted 11 lines
devtools/checkpatches.sh +9
-0+ -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from new additions of rte_smp_XXmb() + # multiple folders and expressions are separated by spaces + awk -v FOLDERS="$c11_atomics_dir" \ + -v EXPRESSIONS="rte_smp_(r|w)?mb\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of rte_smp_[r/w]mb not allowed, use
Quoted 3 lines
__atomic_xxxbuilt-ins' \+ -v MESSAGE='Using rte_smp_[r/w]mb' \
Will do.
Quoted 9 lines
devtools/checkpatches.sh check_forbidden_additions() { # <patch>
Quoted 9 lines
devtools/checkpatches.sh +8
-0+ -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from using compiler __sync built-ins + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="__sync_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Use of __sync_xxx built-ins not allowed, use
Quoted 3 lines
__atomic_xxxbuilt-ins' \+ -v MESSAGE='Using
__sync_xxxbuiltins' \
Will do.
Quoted 11 lines
devtools/checkpatches.sh check_forbidden_additions() { # <patch>
Quoted 11 lines
devtools/checkpatches.sh +9
-0+ -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from using compiler __atomic_thread_fence() + # It should be avoided on x86 for SMP case. + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="__atomic_thread_fence\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='The __atomic_thread_fence is not allowed, use
Quoted 3 lines
rte_atomic_thread_fence' \+ -v MESSAGE='Using
__atomic_thread_fence' \
Will do.
Quoted 9 lines
devtools/checkpatches.sh check_forbidden_additions() { # <patch> · line 119
Quoted 5 lines
devtools/checkpatches.sh +0
-0-f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ "$1" || res=1--
David Marchand
PATCH v8 3/3 eal/atomic: add wrapper for C11 atomic thread fence
In reply to Phil YangProvide a wrapper for __atomic_thread_fence built-in to support optimized code for __ATOMIC_SEQ_CST memory order for x86 platforms.
--- lib/librte_eal/arm/include/rte_atomic_32.h | 6 ++++++ lib/librte_eal/arm/include/rte_atomic_64.h | 6 ++++++ lib/librte_eal/include/generic/rte_atomic.h | 6 ++++++ lib/librte_eal/ppc/include/rte_atomic.h | 6 ++++++ lib/librte_eal/x86/include/rte_atomic.h | 17 +++++++++++++++++ 5 files changed, 41 insertions(+)
lib/librte_eal/arm/include/rte_atomic_32.h +6 -0
@@ -37,6 +37,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} + #ifdef __cplusplus }#endif
lib/librte_eal/arm/include/rte_atomic_64.h +6 -0
@@ -41,6 +41,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} +/*------------------------ 128 bit atomic operations -------------------------*/ #if defined(__ARM_FEATURE_ATOMICS) || defined(RTE_ARM_FEATURE_ATOMICS )
lib/librte_eal/include/generic/rte_atomic.h +6 -0
@@ -158,6 +158,12 @@ static inline void rte_cio_rmb(void); asmvolatile ("" : : :"memory" ); \ }while (0 ) +/** +* Synchronization fence between threads based on the specified +* memory order. +*/ +static inline void rte_atomic_thread_fence(int mo); +/*------------------------- 16 bit atomic operations -------------------------*/ /**
lib/librte_eal/ppc/include/rte_atomic.h +6 -0
@@ -40,6 +40,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} +/*------------------------- 16 bit atomic operations -------------------------*/ /* To be compatible with Power7, use GCC built-in functions for 16 bit * operations */
lib/librte_eal/x86/include/rte_atomic.h +17 -0
@@ -83,6 +83,23 @@ rte_smp_mb(void)#define rte_cio_rmb () rte_compiler_barrier() +/** +* Synchronization fence between threads based on the specified +* memory order. +* +* On x86 the __atomic_thread_fence(__ATOMIC_SEQ_CST) generates +* full 'mfence' which is quite expensive. The optimized +* implementation of rte_smp_mb is used instead. +*/ +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +if (mo == __ATOMIC_SEQ_CST) +rte_smp_mb (); +else +__atomic_thread_fence (mo); +} +/*------------------------- 16 bit atomic operations -------------------------*/ #ifndef RTE_FORCE_INTRINSICS
Signature · 1 line
2.7.4
v9 cover generic rte atomic APIs deprecate proposal
In reply to Phil YangDPDK provides generic rte_atomic APIs to do several atomic operations. These APIs are using the deprecated __sync builtins and enforce full memory barriers on aarch64. However, full barriers are not necessary in many use cases. In order to address such use cases, C language offers C11 atomic APIs. The C11 atomic APIs provide finer memory barrier control by making use of the memory ordering parameter provided by the user. Various patches submitted in the past [2] and the patches in this series indicate significant performance gains on multiple aarch64 CPUs and no performance loss on x86.
But the existing rte_atomic API implementations cannot be changed as the APIs do not take the memory ordering parameter. The only choice available is replacing the usage of the rte_atomic APIs with C11 atomic APIs. In order to make this change, the following steps are proposed:
[1] deprecate rte_atomic APIs so that future patches do not use rte_atomic APIs (a script is added to flag the usages).
[2] refactor the code that uses rte_atomic APIs to use c11 atomic APIs.
This patchset contains:
1) changes to programmer guide describing writing efficient code for aarch64.
2) the checkpatch script changes to flag rte_atomicNN_xxx and rte_smp_[r/w]mb APIs usage in patches.
3) wraps up __atomic_thread_fence with explicit memory ordering parameter.
v9:
1. Change built-ins to builtins. (David)
2. Flag all the new code which use rte_atomicNN_xx and rte_smp_[r/w]mb. (Thomas/Honnappa)
3. Simplify the warning output of the checkpatch script. (David)
v8:
Make descriptions more general. (Honnappa)
v7:
1. Remove code blocks in the guidance.
2. Remove code reference links in the guidance.
3. Remove the update of C11 atomics maintainers.
v6:
Add check for rte_smp barriers APIs in the new code.
v5:
1. Wraps up __atomic_thread_fence to support optimized code for __ATOMIC_SEQ_CST memory order.
2. Flag __atomic_thread_fence with __ATOMIC_SEQ_CST in new patches.
3. Fix email address typo in patch 2/4.
v4:
1. add reader-writer concurrency case describing.
2. claim maintainership of c11 atomics code for each platforms.
3. flag rte_atomicNN_xxx in new patches for modules that have been converted to c11 style.
4. flag __sync_xxx builtins in new patches.
5. wraps up compiler atomic builtins
6. move the changes of libraries which make use of c11 atomic APIs out of this patchset.
v3:
add libatomic dependency for 32-bit clang
v2:
1. fix Clang '-Wincompatible-pointer-types' WARNING.
2. fix typos.
Phil Yang (3):
doc: add optimizations using C11 atomic builtins
devtools: prevent use of rte atomic APIs in future patches eal/atomic: add wrapper for C11 atomic thread fence
devtools/checkpatches.sh | 35 ++++++++++++++ doc/guides/prog_guide/writing_efficient_code.rst | 59 +++++++++++++++++++++++- lib/librte_eal/arm/include/rte_atomic_32.h | 6 +++ lib/librte_eal/arm/include/rte_atomic_64.h | 6 +++ lib/librte_eal/include/generic/rte_atomic.h | 6 +++ lib/librte_eal/ppc/include/rte_atomic.h | 6 +++ lib/librte_eal/x86/include/rte_atomic.h | 17 +++++++ 7 files changed, 134 insertions(+), 1 deletion(-)
Signature · 1 line
2.7.4
PATCH v9 1/3 doc: add optimizations using C11 atomic builtins
Add information about possible optimizations using C11 atomic builtins.
--- doc/guides/prog_guide/writing_efficient_code.rst | 59 +++++++++++++++++++++++- 1 file changed, 58 insertions(+), 1 deletion(-)
doc/guides/prog_guide/writing_efficient_code.rst +58 -1
@@ -167,7 +167,13 @@ but with the added cost of lower throughput. Locks and Atomic Operations --------------------------- -Atomic operations imply a lock prefix before the instruction, +This section describes some key considerations when using locks and atomic +operations in the DPDK environment. + +Locks +~~~~~ + +On x86, atomic operations imply a lock prefix before the instruction, causing the processor's LOCK# signal to be asserted during execution of the following instruction. This has a big impact on performance in a multicore environment.
@@ -176,6 +182,57 @@ It can often be replaced by other solutions like per-lcore variables. Also, some locking techniques are more efficient than others. For instance, the Read-Copy-Update (RCU) algorithm can frequently replace simple rwlocks. +Atomic Operations: Use C11 Atomic Builtins +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +DPDK generic rte_atomic operations are implemented by __sync builtins. These +__sync builtins result in full barriers on aarch64, which are unnecessary +in many use cases. They can be replaced by __atomic builtins that conform to +the C11 memory model and provide finer memory order control. + +So replacing the rte_atomic operations with __atomic builtins might improve +performance for aarch64 machines. + +Some typical optimization cases are listed below: + +Atomicity +^^^^^^^^^ + +Some use cases require atomicity alone, the ordering of the memory operations +does not matter. For example, the packet statistics counters need to be +incremented atomically but do not need any particular memory ordering. +So, RELAXED memory ordering is sufficient. + +One-way Barrier +^^^^^^^^^^^^^^^ + +Some use cases allow for memory reordering in one way while requiring memory +ordering in the other direction. + +For example, the memory operations before the spinlock lock are allowed to +move to the critical section, but the memory operations in the critical section +are not allowed to move above the lock. In this case, the full memory barrier +in the compare-and-swap operation can be replaced with ACQUIRE memory order. +On the other hand, the memory operations after the spinlock unlock are allowed +to move to the critical section, but the memory operations in the critical +section are not allowed to move below the unlock. So the full barrier in the +store operation can use RELEASE memory order. + +Reader-Writer Concurrency +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Lock-free reader-writer concurrency is one of the common use cases in DPDK. + +The payload or the data that the writer wants to communicate to the reader, +can be written with RELAXED memory order. However, the guard variable should +be written with RELEASE memory order. This ensures that the store to guard +variable is observable only after the store to payload is observable. + +Correspondingly, on the reader side, the guard variable should be read +with ACQUIRE memory order. The payload or the data the writer communicated, +can be read with RELAXED memory order. This ensures that, if the store to +guard variable is observable, the store to payload is also observable. + Coding Considerations ---------------------
Signature · 1 line
2.7.4
PATCH v9 2/3 devtools: prevent use of rte atomic APIs in future patches
In reply to Phil YangIn order to deprecate the rte_atomic and rte_smp barrier APIs, prevent the patches from using these APIs and __sync builtins in new code.
On x86 the __atomic_thread_fence(__ATOMIC_SEQ_CST) is quite expensive for SMP case. Flag the new code which use __atomic_thread_fence API.
--- devtools/checkpatches.sh | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+)
devtools/checkpatches.sh +35 -0
@@ -77,6 +77,41 @@ check_forbidden_additions() { # <patch> -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ "$1" || res=1 + # refrain from new additions of 16/32/64 bits rte_atomic_xxx() + # multiple folders and expressions are separated by spaces + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="rte_atomic[0-9][0-9]_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Using rte_atomicNN_xxx' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from new additions of rte_smp_XXmb() + # multiple folders and expressions are separated by spaces + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="rte_smp_(r|w)?mb\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Using rte_smp_[r/w]mb' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from using compiler __sync builtins + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="__sync_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Using __sync_xxx builtins' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from using compiler __atomic_thread_fence() + # It should be avoided on x86 for SMP case. + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="__atomic_thread_fence\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Using __atomic_thread_fence' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + # svg figures must be included with wildcard extension # because of png conversion for pdf docs awk -v FOLDERS='doc' \
Signature · 1 line
2.7.4
Re: [dpdk-dev] [PATCH v9 2/3] devtools: prevent use of rte atomic APIs in future patches
17/07/2020 07:08, Phil Yang:
Quoted 5 lines
In order to deprecate the
rte_atomicandrte_smpbarrier APIs, prevent the patches from using these APIs and__syncbuiltins in new code.On x86 the
__atomic_thread_fence(__ATOMIC_SEQ_CST)is quite expensive for SMP case. Flag the new code which use__atomic_thread_fenceAPI.
When receiving such message error, I would look at the explanations in the git history. Unfortunately this commit log does not reference the documentation. Please add a link and give the 4 direct replacements in this commit log.
Re: [dpdk-dev] [PATCH v9 2/3] devtools: prevent use of rte atomic APIs in future patches
Thomas Monjalon <[email protected]> wrties:
Quoted 11 lines
Quoted 4 lines
In order to deprecate the
rte_atomicandrte_smpbarrier APIs, prevent the patches from using these APIs and__syncbuiltins in new code.On x86 the
__atomic_thread_fence(__ATOMIC_SEQ_CST)is quiteexpensive
Quoted 1 line
for SMP case. Flag the new code which use
__atomic_thread_fenceAPI.When receiving such message error, I would look at the explanations in the git history. Unfortunately this commit log does not reference the documentation. Please add a link and give the 4 direct replacements in this commit log.
I will update it in v10. Thanks.
How about putting it like this? (Truncated the long link to pass the long line check)
"
In order to deprecate the rte_atomic and rte_smp barrier APIs[1], prevent the patches from using these APIs and __sync builtins in new code. Please use __atomic builtins instead of __sync builtins, rte_atomicNN_xxx and rte_smp_[r/w]mb APIs.
On x86 the __atomic_thread_fence(__ATOMIC_SEQ_CST) is quite expensive for SMP case. Flag the new code which use __atomic_thread_fence API. Please use rte_thread_fence API instead of __atomic_thread_fence.
[1] Refer to the locks-and-atomic-operations section in
https://doc.dpdk.org/guides/prog_guide/writing_efficient_code.html.
"
PATCH v9 3/3 eal/atomic: add wrapper for C11 atomic thread fence
In reply to Phil YangProvide a wrapper for __atomic_thread_fence builtins to support optimized code for __ATOMIC_SEQ_CST memory order for x86 platforms.
--- lib/librte_eal/arm/include/rte_atomic_32.h | 6 ++++++ lib/librte_eal/arm/include/rte_atomic_64.h | 6 ++++++ lib/librte_eal/include/generic/rte_atomic.h | 6 ++++++ lib/librte_eal/ppc/include/rte_atomic.h | 6 ++++++ lib/librte_eal/x86/include/rte_atomic.h | 17 +++++++++++++++++ 5 files changed, 41 insertions(+)
lib/librte_eal/arm/include/rte_atomic_32.h +6 -0
@@ -37,6 +37,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} + #ifdef __cplusplus }#endif
lib/librte_eal/arm/include/rte_atomic_64.h +6 -0
@@ -41,6 +41,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} +/*------------------------ 128 bit atomic operations -------------------------*/ #if defined(__ARM_FEATURE_ATOMICS) || defined(RTE_ARM_FEATURE_ATOMICS )
lib/librte_eal/include/generic/rte_atomic.h +6 -0
@@ -158,6 +158,12 @@ static inline void rte_cio_rmb(void); asmvolatile ("" : : :"memory" ); \ }while (0 ) +/** +* Synchronization fence between threads based on the specified +* memory order. +*/ +static inline void rte_atomic_thread_fence(int mo); +/*------------------------- 16 bit atomic operations -------------------------*/ /**
lib/librte_eal/ppc/include/rte_atomic.h +6 -0
@@ -40,6 +40,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} +/*------------------------- 16 bit atomic operations -------------------------*/ /* To be compatible with Power7, use GCC built-in functions for 16 bit * operations */
lib/librte_eal/x86/include/rte_atomic.h +17 -0
@@ -83,6 +83,23 @@ rte_smp_mb(void)#define rte_cio_rmb () rte_compiler_barrier() +/** +* Synchronization fence between threads based on the specified +* memory order. +* +* On x86 the __atomic_thread_fence(__ATOMIC_SEQ_CST) generates +* full 'mfence' which is quite expensive. The optimized +* implementation of rte_smp_mb is used instead. +*/ +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +if (mo == __ATOMIC_SEQ_CST) +rte_smp_mb (); +else +__atomic_thread_fence (mo); +} +/*------------------------- 16 bit atomic operations -------------------------*/ #ifndef RTE_FORCE_INTRINSICS
Signature · 1 line
2.7.4
Re: [dpdk-dev] [PATCH v9 3/3] eal/atomic: add wrapper for C11 atomic thread fence
17/07/2020 07:08, Phil Yang:
Quoted 2 lines
Provide a wrapper for
__atomic_thread_fencebuiltins to support optimized code for__ATOMIC_SEQ_CSTmemory order for x86 platforms.
Logically, it should be the patch 2.
So you can recommend using this wrapper in the explanation of the actual patch 2.
Re: [dpdk-dev] [PATCH v9 3/3] eal/atomic: add wrapper for C11 atomic thread fence
Quoted 6 lines · Thomas Monjalon
Quoted 2 lines
Provide a wrapper for
__atomic_thread_fencebuiltins to support optimized code for__ATOMIC_SEQ_CSTmemory order for x86 platforms.Logically, it should be the patch 2.
So you can recommend using this wrapper in the explanation of the actual patch 2.
Indeed. Thanks.
Will change it in v10.
v10 cover generic rte atomic APIs deprecate proposal
In reply to Phil YangDPDK provides generic rte_atomic APIs to do several atomic operations. These APIs are using the deprecated __sync builtins and enforce full memory barriers on aarch64. However, full barriers are not necessary in many use cases. In order to address such use cases, C language offers C11 atomic APIs. The C11 atomic APIs provide finer memory barrier control by making use of the memory ordering parameter provided by the user. Various patches submitted in the past [2] and the patches in this series indicate significant performance gains on multiple aarch64 CPUs and no performance loss on x86.
But the existing rte_atomic API implementations cannot be changed as the APIs do not take the memory ordering parameter. The only choice available is replacing the usage of the rte_atomic APIs with C11 atomic APIs. In order to make this change, the following steps are proposed:
[1] deprecate rte_atomic APIs so that future patches do not use rte_atomic APIs (a script is added to flag the usages).
[2] refactor the code that uses rte_atomic APIs to use c11 atomic APIs.
This patchset contains:
1) changes to programmer guide describing writing efficient code for aarch64.
2) wraps up __atomic_thread_fence with explicit memory ordering parameter.
3) the checkpatch script changes to flag rte_atomicNN_xxx and rte_smp_[r/w]mb APIs usage in patches.
v10:
1. Rearrange the second and third patches. (Thomas)
2. Refine commit log. (Thomas)
v9:
1. Change built-ins to builtins. (David)
2. Flag all the new code which use rte_atomicNN_xx and rte_smp_[r/w]mb. (Thomas/Honnappa)
3. Simplify the warning output of the checkpatch script. (David)
v8:
Make descriptions more general. (Honnappa)
v7:
1. Remove code blocks in the guidance.
2. Remove code reference links in the guidance.
3. Remove the update of C11 atomics maintainers.
v6:
Add check for rte_smp barriers APIs in the new code.
v5:
1. Wraps up __atomic_thread_fence to support optimized code for __ATOMIC_SEQ_CST memory order.
2. Flag __atomic_thread_fence with __ATOMIC_SEQ_CST in new patches.
3. Fix email address typo in patch 2/4.
v4:
1. add reader-writer concurrency case describing.
2. claim maintainership of c11 atomics code for each platforms.
3. flag rte_atomicNN_xxx in new patches for modules that have been converted to c11 style.
4. flag __sync_xxx builtins in new patches.
5. wraps up compiler atomic builtins
6. move the changes of libraries which make use of c11 atomic APIs out of this patchset.
v3:
add libatomic dependency for 32-bit clang
v2:
1. fix Clang '-Wincompatible-pointer-types' WARNING.
2. fix typos.
Phil Yang (3):
doc: add optimizations using C11 atomic builtins eal/atomic: add wrapper for C11 atomic thread fence
devtools: prevent use of rte atomic APIs in future patches
devtools/checkpatches.sh | 35 ++++++++++++++ doc/guides/prog_guide/writing_efficient_code.rst | 59 +++++++++++++++++++++++- lib/librte_eal/arm/include/rte_atomic_32.h | 6 +++ lib/librte_eal/arm/include/rte_atomic_64.h | 6 +++ lib/librte_eal/include/generic/rte_atomic.h | 6 +++ lib/librte_eal/ppc/include/rte_atomic.h | 6 +++ lib/librte_eal/x86/include/rte_atomic.h | 17 +++++++ 7 files changed, 134 insertions(+), 1 deletion(-)
Signature · 1 line
2.7.4
PATCH v10 1/3 doc: add optimizations using C11 atomic builtins
Add information about possible optimizations using C11 atomic builtins.
--- doc/guides/prog_guide/writing_efficient_code.rst | 59 +++++++++++++++++++++++- 1 file changed, 58 insertions(+), 1 deletion(-)
doc/guides/prog_guide/writing_efficient_code.rst +58 -1
@@ -167,7 +167,13 @@ but with the added cost of lower throughput. Locks and Atomic Operations --------------------------- -Atomic operations imply a lock prefix before the instruction, +This section describes some key considerations when using locks and atomic +operations in the DPDK environment. + +Locks +~~~~~ + +On x86, atomic operations imply a lock prefix before the instruction, causing the processor's LOCK# signal to be asserted during execution of the following instruction. This has a big impact on performance in a multicore environment.
@@ -176,6 +182,57 @@ It can often be replaced by other solutions like per-lcore variables. Also, some locking techniques are more efficient than others. For instance, the Read-Copy-Update (RCU) algorithm can frequently replace simple rwlocks. +Atomic Operations: Use C11 Atomic Builtins +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +DPDK generic rte_atomic operations are implemented by __sync builtins. These +__sync builtins result in full barriers on aarch64, which are unnecessary +in many use cases. They can be replaced by __atomic builtins that conform to +the C11 memory model and provide finer memory order control. + +So replacing the rte_atomic operations with __atomic builtins might improve +performance for aarch64 machines. + +Some typical optimization cases are listed below: + +Atomicity +^^^^^^^^^ + +Some use cases require atomicity alone, the ordering of the memory operations +does not matter. For example, the packet statistics counters need to be +incremented atomically but do not need any particular memory ordering. +So, RELAXED memory ordering is sufficient. + +One-way Barrier +^^^^^^^^^^^^^^^ + +Some use cases allow for memory reordering in one way while requiring memory +ordering in the other direction. + +For example, the memory operations before the spinlock lock are allowed to +move to the critical section, but the memory operations in the critical section +are not allowed to move above the lock. In this case, the full memory barrier +in the compare-and-swap operation can be replaced with ACQUIRE memory order. +On the other hand, the memory operations after the spinlock unlock are allowed +to move to the critical section, but the memory operations in the critical +section are not allowed to move below the unlock. So the full barrier in the +store operation can use RELEASE memory order. + +Reader-Writer Concurrency +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Lock-free reader-writer concurrency is one of the common use cases in DPDK. + +The payload or the data that the writer wants to communicate to the reader, +can be written with RELAXED memory order. However, the guard variable should +be written with RELEASE memory order. This ensures that the store to guard +variable is observable only after the store to payload is observable. + +Correspondingly, on the reader side, the guard variable should be read +with ACQUIRE memory order. The payload or the data the writer communicated, +can be read with RELAXED memory order. This ensures that, if the store to +guard variable is observable, the store to payload is also observable. + Coding Considerations ---------------------
Signature · 1 line
2.7.4
PATCH v10 2/3 eal/atomic: add wrapper for C11 atomic thread fence
In reply to Phil YangProvide a wrapper for __atomic_thread_fence builtins to support optimized code for __ATOMIC_SEQ_CST memory order for x86 platforms.
--- lib/librte_eal/arm/include/rte_atomic_32.h | 6 ++++++ lib/librte_eal/arm/include/rte_atomic_64.h | 6 ++++++ lib/librte_eal/include/generic/rte_atomic.h | 6 ++++++ lib/librte_eal/ppc/include/rte_atomic.h | 6 ++++++ lib/librte_eal/x86/include/rte_atomic.h | 17 +++++++++++++++++ 5 files changed, 41 insertions(+)
lib/librte_eal/arm/include/rte_atomic_32.h +6 -0
@@ -37,6 +37,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} + #ifdef __cplusplus }#endif
lib/librte_eal/arm/include/rte_atomic_64.h +6 -0
@@ -41,6 +41,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} +/*------------------------ 128 bit atomic operations -------------------------*/ #if defined(__ARM_FEATURE_ATOMICS) || defined(RTE_ARM_FEATURE_ATOMICS )
lib/librte_eal/include/generic/rte_atomic.h +6 -0
@@ -158,6 +158,12 @@ static inline void rte_cio_rmb(void); asmvolatile ("" : : :"memory" ); \ }while (0 ) +/** +* Synchronization fence between threads based on the specified +* memory order. +*/ +static inline void rte_atomic_thread_fence(int mo); +/*------------------------- 16 bit atomic operations -------------------------*/ /**
lib/librte_eal/ppc/include/rte_atomic.h +6 -0
@@ -40,6 +40,12 @@ extern "C" {#define rte_cio_rmb () rte_rmb() +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +__atomic_thread_fence (mo); +} +/*------------------------- 16 bit atomic operations -------------------------*/ /* To be compatible with Power7, use GCC built-in functions for 16 bit * operations */
lib/librte_eal/x86/include/rte_atomic.h +17 -0
@@ -83,6 +83,23 @@ rte_smp_mb(void)#define rte_cio_rmb () rte_compiler_barrier() +/** +* Synchronization fence between threads based on the specified +* memory order. +* +* On x86 the __atomic_thread_fence(__ATOMIC_SEQ_CST) generates +* full 'mfence' which is quite expensive. The optimized +* implementation of rte_smp_mb is used instead. +*/ +static __rte_always_inline void +rte_atomic_thread_fence (int mo) +{ +if (mo == __ATOMIC_SEQ_CST) +rte_smp_mb (); +else +__atomic_thread_fence (mo); +} +/*------------------------- 16 bit atomic operations -------------------------*/ #ifndef RTE_FORCE_INTRINSICS
Signature · 1 line
2.7.4
PATCH v10 3/3 devtools: prevent use of rte atomic APIs in future patches
In reply to Phil YangIn order to deprecate the rte_atomic and rte_smp barrier APIs[1], prevent the patches from using these APIs and __sync builtins in new code. Please use __atomic builtins instead of __sync builtins, rte_atomicNN_xxx and rte_smp_[r/w]mb APIs.
On x86 the __atomic_thread_fence(__ATOMIC_SEQ_CST) is quite expensive for SMP case. Flag the new code which use __atomic_thread_fence API. Please use rte_thread_fence API instead of __atomic_thread_fence builtins.
1. Refer to the locks-and-atomic-operations section in
https://doc.dpdk.org/guides/prog_guide/writing_efficient_code.html.
--- devtools/checkpatches.sh | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+)
devtools/checkpatches.sh +35 -0
@@ -77,6 +77,41 @@ check_forbidden_additions() { # <patch> -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ "$1" || res=1 + # refrain from new additions of 16/32/64 bits rte_atomic_xxx() + # multiple folders and expressions are separated by spaces + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="rte_atomic[0-9][0-9]_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Using rte_atomicNN_xxx' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from new additions of rte_smp_XXmb() + # multiple folders and expressions are separated by spaces + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="rte_smp_(r|w)?mb\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Using rte_smp_[r/w]mb' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from using compiler __sync builtins + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="__sync_.*\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Using __sync_xxx builtins' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + + # refrain from using compiler __atomic_thread_fence() + # It should be avoided on x86 for SMP case. + awk -v FOLDERS="lib drivers app examples" \ + -v EXPRESSIONS="__atomic_thread_fence\\\(" \ + -v RET_ON_FAIL=1 \ + -v MESSAGE='Using __atomic_thread_fence' \ + -f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \ + "$1" || res=1 + # svg figures must be included with wildcard extension # because of png conversion for pdf docs awk -v FOLDERS='doc' \
Signature · 1 line
2.7.4
Quoted 20 lines · Phil Yang
DPDK provides generic
rte_atomicAPIs to do several atomic operations. These APIs are using the deprecated__syncbuiltins and enforce full memory barriers on aarch64. However, full barriers are not necessary in many use cases. In order to address such use cases, C language offers C11 atomic APIs. The C11 atomic APIs provide finer memory barrier control by making use of the memory ordering parameter provided by the user. Various patches submitted in the past [2] and the patches in this series indicate significant performance gains on multiple aarch64 CPUs and no performance loss on x86.But the existing
rte_atomicAPI implementations cannot be changed as the APIs do not take the memory ordering parameter. The only choice available is replacing the usage of therte_atomicAPIs with C11 atomic APIs. In order to make this change, the following steps are proposed:[1] deprecate
rte_atomicAPIs so that future patches do not userte_atomicAPIs (a script is added to flag the usages).
[2] refactor the code that usesrte_atomicAPIs to use c11 atomic APIs.
Quoted 5 lines
This patchset contains:
1) changes to programmer guide describing writing efficient code for aarch64.
2) wraps up__atomic_thread_fencewith explicit memory ordering parameter.
3) the checkpatch script changes to flagrte_atomicNN_xxxand rte_smp_[r/w]mb APIs usage in patches.
Series applied, thanks.
The removal of the legacy atomic will be in a future release. Can you prepare a patch updating the deprecation notices to announce this removal?
This will be the occasion to discuss the timing.
Signature · 1 line
David Marchand
Quoted 41 lines
-----Original Message-----
From: David Marchand <[email protected]>
Sent: Friday, July 17, 2020 9:59 PM
To: Phil Yang <[email protected]>
Cc: [email protected]; dev <[email protected]>; Mcnamara, John <[email protected]>; Honnappa Nagarahalli <[email protected]>; David Christensen <[email protected]>; [email protected]; Ananyev, Konstantin <[email protected]>; Ola Liljedahl <[email protected]>; Bruce Richardson <[email protected]>; Ruifeng Wang <[email protected]>; nd <[email protected]>
Subject: Re: [PATCH v10 0/3] generic rte atomic APIs deprecate proposalQuoted 16 lines · Phil Yang
DPDK provides generic
rte_atomicAPIs to do several atomic operations. These APIs are using the deprecated__syncbuiltins and enforce full memory barriers on aarch64. However, full barriers are not necessary in many use cases. In order to address such use cases, C language offers C11 atomic APIs. The C11 atomic APIs provide finer memory barrier control by making use of the memory ordering parameter provided by the user. Various patches submitted in the past [2] and the patches in this series indicate significant performance gains on multiple aarch64 CPUs and no performance loss on x86.But the existing
rte_atomicAPI implementations cannot be changed as the APIs do not take the memory ordering parameter. The only choice available is replacing the usage of therte_atomicAPIs with C11 atomic APIs. In order to make this change, the following steps are proposed:Quoted 6 lines
[1] deprecate
rte_atomicAPIs so that future patches do not userte_atomicAPIs (a script is added to flag the usages).
[2] refactor the code that usesrte_atomicAPIs to use c11 atomic APIs.This patchset contains:
1) changes to programmer guide describing writing efficient code foraarch64.
Quoted 1 line
2) wraps up
__atomic_thread_fencewith explicit memory orderingparameter.
Quoted 1 line
3) the checkpatch script changes to flag
rte_atomicNN_xxxandrte_smp_[r/w]mb
Quoted 8 lines
Quoted 1 line
APIs usage in patches.
Series applied, thanks.
The removal of the legacy atomic will be in a future release. Can you prepare a patch updating the deprecation notices to announce this removal?
This will be the occasion to discuss the timing.
Sure. I think after complete converting for all the drivers and libraries, we can start to remove the legacy atomic APIs. I will prepare a patch for that announcement. We can discuss the details in that thread.
Thanks,
Phil