Skip to contentSkip to conversation

count: Switch from GCC to C11 thread-local storage

PATCH Superseded Elad Lahav 2 participants 13 messages Aug 13, 2022 – Aug 17, 2022

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Elad Lahav

author
#0 lore raw

PATCH 1/1 count: Switch from GCC to C11 thread-local storage

---
 CodeSamples/count/count_end.c | 12 ++++++------
 count/count.tex               |  4 ++--
 2 files changed, 8 insertions(+), 8 deletions(-)
CodeSamples/count/count_end.c +6 -6
index 722ad2f7..5e7b9ee1 100644
@@ -1,6 +1,6 @@
 /*
  * count_end.c: Per-thread statistical counters that provide sum at end.
- *	Uses __thread for each thread's counter.
+ *	Uses _Thread_local for each thread's counter.
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -23,17 +23,17 @@
 #include "../api.h"
 
 //\begin{snippet}[labelbase=ln:count:count_end:whole,commandchars=\\\@\$]
-unsigned long __thread counter = 0;		//\lnlbl{var:b}
+unsigned long _Thread_local counter = 0;		//\lnlbl{var:b}
 unsigned long *counterp[NR_THREADS] = { NULL };
 unsigned long finalcount = 0;
 DEFINE_SPINLOCK(final_mutex);			//\lnlbl{var:e}
 
-static __inline__ void inc_count(void)		//\lnlbl{inc:b}
+static inline void inc_count(void)		//\lnlbl{inc:b}
 {
 	WRITE_ONCE(counter, counter + 1);
 }						//\lnlbl{inc:e}
 
-static __inline__ unsigned long read_count(void)
+static inline unsigned long read_count(void)
 {
 	int t;
 	unsigned long sum;
@@ -48,7 +48,7 @@ static __inline__ unsigned long read_count(void)
 }
 
 #ifndef FCV_SNIPPET
-__inline__ void count_init(void)
+inline void count_init(void)
 {
 }
 #endif /* FCV_SNIPPET */
@@ -73,7 +73,7 @@ void count_unregister_thread(int nthreadsexpected)	//\lnlbl{unreg:b}
 }							//\lnlbl{unreg:e}
 //\end{snippet}
 
-__inline__ void count_cleanup(void)
+inline void count_cleanup(void)
 {
 }
 
count/count.tex +2 -2
index 523789e2..173b964c 100644
@@ -721,8 +721,8 @@ This is the topic of the next section.
 \subsection{Per-Thread-Variable-Based Implementation}
 \label{sec:count:Per-Thread-Variable-Based Implementation}
 
-\GCC\ provides an \apig{__thread} storage class that provides
-per-thread storage.
+The C language, since C11, features a \apig{_Thread_local} storage class that
+provides per-thread storage.
 This can be used as shown in
 \cref{lst:count:Per-Thread Statistical Counters} (\path{count_end.c})
 to implement
Signature · 1 line
2.25.1
author
#1 lore raw
Quoted 2 lines · Elad Lahav

---

This one will probably require some back-and-forth. If we change it here, do other places need to be updated? Have I now removed the explanation of GCC's __thread storage class, which will impact the readability of other sections? Or are you ready to take the plunge and convert all code snippets?

For the record, I did build and test count_end.c.

--Elad

#9 lore raw
In reply to Elad Lahav
Quoted 1 line · Elad Lahav

Nice, thank you!

But did you verify that none of the text describing count_end.c needed changing? (It seems unlikely that any would, but could you please check?)

Quoted 61 lines
---
 CodeSamples/count/count_end.c | 12 ++++++------
 count/count.tex               |  4 ++--
 2 files changed, 8 insertions(+), 8 deletions(-)
CodeSamples/count/count_end.c +6 -6
index 722ad2f7..5e7b9ee1 100644
@@ -1,6 +1,6 @@
 /*
  * count_end.c: Per-thread statistical counters that provide sum at end.
- *	Uses __thread for each thread's counter.
+ *	Uses _Thread_local for each thread's counter.
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -23,17 +23,17 @@
 #include "../api.h"
 
 //\begin{snippet}[labelbase=ln:count:count_end:whole,commandchars=\\\@\$]
-unsigned long __thread counter = 0;		//\lnlbl{var:b}
+unsigned long _Thread_local counter = 0;		//\lnlbl{var:b}
 unsigned long *counterp[NR_THREADS] = { NULL };
 unsigned long finalcount = 0;
 DEFINE_SPINLOCK(final_mutex);			//\lnlbl{var:e}
 
-static __inline__ void inc_count(void)		//\lnlbl{inc:b}
+static inline void inc_count(void)		//\lnlbl{inc:b}
 {
 	WRITE_ONCE(counter, counter + 1);
 }						//\lnlbl{inc:e}
 
-static __inline__ unsigned long read_count(void)
+static inline unsigned long read_count(void)
 {
 	int t;
 	unsigned long sum;
@@ -48,7 +48,7 @@ static __inline__ unsigned long read_count(void)
 }
 
 #ifndef FCV_SNIPPET
-__inline__ void count_init(void)
+inline void count_init(void)
 {
 }
 #endif /* FCV_SNIPPET */
@@ -73,7 +73,7 @@ void count_unregister_thread(int nthreadsexpected)	//\lnlbl{unreg:b}
 }							//\lnlbl{unreg:e}
 //\end{snippet}
 
-__inline__ void count_cleanup(void)
+inline void count_cleanup(void)
 {
 }
 
count/count.tex +0 -0
index 523789e2..173b964c 100644

count/count.tex This is the topic of the next section. · line 725

Quoted 7 lines
count/count.tex +2 -2
@@ -721,8 +721,8 @@ This is the topic of the next section.
 \subsection{Per-Thread-Variable-Based Implementation}
 \label{sec:count:Per-Thread-Variable-Based Implementation}
 
-\GCC\ provides an \apig{__thread} storage class that provides
-per-thread storage.
+The C language, since C11, features a \apig{_Thread_local} storage class that
+provides per-thread storage.

Could you please mention __thread as the old way? Either in the text directly, in parentheses, as a footnote, or as a quick quiz, your choice.

Thanx, Paul

Quoted 6 lines
count/count.tex +0 -0
 This can be used as shown in
 \cref{lst:count:Per-Thread Statistical Counters} (\path{count_end.c})
 to implement
Signature · 1 line
2.25.1
#2 lore raw
In reply to Elad Lahav
Quoted 10 lines · Elad Lahav
Quoted 2 lines · Elad Lahav

---

This one will probably require some back-and-forth. If we change it here, do other places need to be updated? Have I now removed the explanation of GCC's __thread storage class, which will impact the readability of other sections? Or are you ready to take the plunge and convert all code snippets?

For the record, I did build and test count_end.c.

As long as you continue building and testing, checking the descriptions in text of the code snippets (in case there is a mention of __thread there), and verifying that the PDF still builds, as fast as you are willing to go!

Are you seeing the same performance with the new as with the old on your hardware? (No statistically significant difference on my laptop, but figured I should ask.)

Thanx, Paul

author
#10 lore raw
In reply to Paul E. McKenney

Yes, I thought about a footnote along the lines of "previous versions of this book used GCC extensions..." but wasn't sure if you'd approve. I can certainly add that.

--Elad

Quoted 89 lines · Paul E. McKenney
Quoted 1 line · Elad Lahav

Nice, thank you!

But did you verify that none of the text describing count_end.c needed changing? (It seems unlikely that any would, but could you please check?)

Quoted 61 lines
---
 CodeSamples/count/count_end.c | 12 ++++++------
 count/count.tex               |  4 ++--
 2 files changed, 8 insertions(+), 8 deletions(-)
CodeSamples/count/count_end.c +6 -6
index 722ad2f7..5e7b9ee1 100644
@@ -1,6 +1,6 @@
 /*
  * count_end.c: Per-thread statistical counters that provide sum at end.
- *   Uses __thread for each thread's counter.
+ *   Uses _Thread_local for each thread's counter.
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -23,17 +23,17 @@
 #include "../api.h"
 
 //\begin{snippet}[labelbase=ln:count:count_end:whole,commandchars=\\\@\$]
-unsigned long __thread counter = 0;          //\lnlbl{var:b}
+unsigned long _Thread_local counter = 0;             //\lnlbl{var:b}
 unsigned long *counterp[NR_THREADS] = { NULL };
 unsigned long finalcount = 0;
 DEFINE_SPINLOCK(final_mutex);                        //\lnlbl{var:e}
 
-static __inline__ void inc_count(void)               //\lnlbl{inc:b}
+static inline void inc_count(void)           //\lnlbl{inc:b}
 {
      WRITE_ONCE(counter, counter + 1);
 }                                            //\lnlbl{inc:e}
 
-static __inline__ unsigned long read_count(void)
+static inline unsigned long read_count(void)
 {
      int t;
      unsigned long sum;
@@ -48,7 +48,7 @@ static __inline__ unsigned long read_count(void)
 }
 
 #ifndef FCV_SNIPPET
-__inline__ void count_init(void)
+inline void count_init(void)
 {
 }
 #endif /* FCV_SNIPPET */
@@ -73,7 +73,7 @@ void count_unregister_thread(int nthreadsexpected)  //\lnlbl{unreg:b}
 }                                                    //\lnlbl{unreg:e}
 //\end{snippet}
 
-__inline__ void count_cleanup(void)
+inline void count_cleanup(void)
 {
 }
 
count/count.tex +0 -0
index 523789e2..173b964c 100644

count/count.tex This is the topic of the next section. · line 725

Quoted 7 lines
count/count.tex +2 -2
@@ -721,8 +721,8 @@ This is the topic of the next section.
 \subsection{Per-Thread-Variable-Based Implementation}
 \label{sec:count:Per-Thread-Variable-Based Implementation}
 
-\GCC\ provides an \apig{__thread} storage class that provides
-per-thread storage.
+The C language, since C11, features a \apig{_Thread_local} storage class that
+provides per-thread storage.

Could you please mention __thread as the old way? Either in the text directly, in parentheses, as a footnote, or as a quick quiz, your choice.

Thanx, Paul

Quoted 6 lines
count/count.tex +0 -1
 This can be used as shown in
 \cref{lst:count:Per-Thread Statistical Counters} (\path{count_end.c})
 to implement
--

2.25.1

#11 lore raw

That sounds good, thank you!

There are quite a few people on projects for which C11 is not yet a thing. Or that need to look at old code.

But yes, some years hence, perhaps all mention of __thread is dropped from this book. Not today, though. ;-)

Thanx, Paul

Quoted 96 lines · Elad Lahav

Yes, I thought about a footnote along the lines of "previous versions of this book used GCC extensions..." but wasn't sure if you'd approve. I can certainly add that.

--Elad

Quoted 89 lines · Paul E. McKenney
Quoted 1 line · Elad Lahav

Nice, thank you!

But did you verify that none of the text describing count_end.c needed changing? (It seems unlikely that any would, but could you please check?)

Quoted 61 lines
---
 CodeSamples/count/count_end.c | 12 ++++++------
 count/count.tex               |  4 ++--
 2 files changed, 8 insertions(+), 8 deletions(-)
CodeSamples/count/count_end.c +6 -6
index 722ad2f7..5e7b9ee1 100644
@@ -1,6 +1,6 @@
 /*
  * count_end.c: Per-thread statistical counters that provide sum at end.
- *   Uses __thread for each thread's counter.
+ *   Uses _Thread_local for each thread's counter.
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -23,17 +23,17 @@
 #include "../api.h"
 
 //\begin{snippet}[labelbase=ln:count:count_end:whole,commandchars=\\\@\$]
-unsigned long __thread counter = 0;          //\lnlbl{var:b}
+unsigned long _Thread_local counter = 0;             //\lnlbl{var:b}
 unsigned long *counterp[NR_THREADS] = { NULL };
 unsigned long finalcount = 0;
 DEFINE_SPINLOCK(final_mutex);                        //\lnlbl{var:e}
 
-static __inline__ void inc_count(void)               //\lnlbl{inc:b}
+static inline void inc_count(void)           //\lnlbl{inc:b}
 {
      WRITE_ONCE(counter, counter + 1);
 }                                            //\lnlbl{inc:e}
 
-static __inline__ unsigned long read_count(void)
+static inline unsigned long read_count(void)
 {
      int t;
      unsigned long sum;
@@ -48,7 +48,7 @@ static __inline__ unsigned long read_count(void)
 }
 
 #ifndef FCV_SNIPPET
-__inline__ void count_init(void)
+inline void count_init(void)
 {
 }
 #endif /* FCV_SNIPPET */
@@ -73,7 +73,7 @@ void count_unregister_thread(int nthreadsexpected)  //\lnlbl{unreg:b}
 }                                                    //\lnlbl{unreg:e}
 //\end{snippet}
 
-__inline__ void count_cleanup(void)
+inline void count_cleanup(void)
 {
 }
 
count/count.tex +0 -0
index 523789e2..173b964c 100644

count/count.tex This is the topic of the next section. · line 725

Quoted 7 lines
count/count.tex +2 -2
@@ -721,8 +721,8 @@ This is the topic of the next section.
 \subsection{Per-Thread-Variable-Based Implementation}
 \label{sec:count:Per-Thread-Variable-Based Implementation}
 
-\GCC\ provides an \apig{__thread} storage class that provides
-per-thread storage.
+The C language, since C11, features a \apig{_Thread_local} storage class that
+provides per-thread storage.

Could you please mention __thread as the old way? Either in the text directly, in parentheses, as a footnote, or as a quick quiz, your choice.

Thanx, Paul

Quoted 6 lines
count/count.tex +0 -1
 This can be used as shown in
 \cref{lst:count:Per-Thread Statistical Counters} (\path{count_end.c})
 to implement
--

2.25.1

author
#3 lore raw
In reply to Paul E. McKenney

I believe that performance is statistically the same, but I will double check. I assume both GCC and C11 end up using the same underlying mechanism for thread-local storage:

https://uclibc.org/docs/tls.pdf

If not implemented, TLS falls back on pthread_[gs]et_specific(), but again it would be the same for __thread and _Thread_local.

I was about to run all benchmarks on a cute 16-core aarch64 machine I recently bought for testing the scalability of my latest kernel work, but the code requires linking against urcu, and now I'm going down the rabbit hole of building that for QNX. Does the counter code really need this library?

--Elad

Quoted 22 lines · Paul E. McKenney
Quoted 10 lines · Elad Lahav
Quoted 2 lines · Elad Lahav

---

This one will probably require some back-and-forth. If we change it here, do other places need to be updated? Have I now removed the explanation of GCC's __thread storage class, which will impact the readability of other sections? Or are you ready to take the plunge and convert all code snippets?

For the record, I did build and test count_end.c.

As long as you continue building and testing, checking the descriptions in text of the code snippets (in case there is a mention of __thread there), and verifying that the PDF still builds, as fast as you are willing to go!

Are you seeing the same performance with the new as with the old on your hardware? (No statistically significant difference on my laptop, but figured I should ask.)

Thanx, Paul

#4 lore raw
Quoted 8 lines · Elad Lahav

I believe that performance is statistically the same, but I will double check. I assume both GCC and C11 end up using the same underlying mechanism for thread-local storage:

https://uclibc.org/docs/tls.pdf

If not implemented, TLS falls back on pthread_[gs]et_specific(), but again it would be the same for __thread and _Thread_local.

Sounds likely to me, but I have been surprised before.

Quoted 5 lines

I was about to run all benchmarks on a cute 16-core aarch64 machine I recently bought for testing the scalability of my latest kernel work, but the code requires linking against urcu, and now I'm going down the rabbit hole of building that for QNX. Does the counter code really need this library?

Only for one of the count programs, count_end_rcu.c. If you comment that one out of the Makefile, you should be fine. Some other adjustments to other files might be needed, for example, CodeSamples/Makefile.

Probably about the same amount of work to wean CodeSamples/count from that library as to get it working on QNX. Murphy would of course argue that whichever path you choose will be the harder of the two.

Thanx, Paul

Quoted 25 lines

--Elad

Quoted 22 lines · Paul E. McKenney
Quoted 10 lines · Elad Lahav
Quoted 2 lines · Elad Lahav

---

This one will probably require some back-and-forth. If we change it here, do other places need to be updated? Have I now removed the explanation of GCC's __thread storage class, which will impact the readability of other sections? Or are you ready to take the plunge and convert all code snippets?

For the record, I did build and test count_end.c.

As long as you continue building and testing, checking the descriptions in text of the code snippets (in case there is a mention of __thread there), and verifying that the PDF still builds, as fast as you are willing to go!

Are you seeing the same performance with the new as with the old on your hardware? (No statistically significant difference on my laptop, but figured I should ask.)

Thanx, Paul

author
#5 lore raw
Quoted 11 lines · Paul E. McKenney
Quoted 8 lines · Elad Lahav

I believe that performance is statistically the same, but I will double check. I assume both GCC and C11 end up using the same underlying mechanism for thread-local storage:

https://uclibc.org/docs/tls.pdf

If not implemented, TLS falls back on pthread_[gs]et_specific(), but again it would be the same for __thread and _Thread_local.

Sounds likely to me, but I have been surprised before.

Results:

Linux/x86-64, i7-8550U, GCC TLS

elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 408000  n_updates: 689366000  nreaders: 1  nupdaters: 1 duration: 240
ns/read: 588.235  ns/update: 0.348146
elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 443000  n_updates: 762876000  nreaders: 1  nupdaters: 1 duration: 240
ns/read: 541.761  ns/update: 0.314599
elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 395000  n_updates: 666718000  nreaders: 1  nupdaters: 1 duration: 240
ns/read: 607.595  ns/update: 0.359972

Linux/x86-64, i7-8550U, C11 TLS

elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 410000  n_updates: 684880000  nreaders: 1  nupdaters: 1 duration: 240
ns/read: 585.366  ns/update: 0.350426
elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 411000  n_updates: 698148000  nreaders: 1  nupdaters: 1 duration: 240
ns/read: 583.942  ns/update: 0.343767
elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 409000  n_updates: 704072000  nreaders: 1  nupdaters: 1 duration: 240
ns/read: 586.797  ns/update: 0.340874

QNX/aarch64, NXP LX2160A, GCC TLS (emulated)

elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 221000  n_updates: 67876000  nreaders: 1  nupdaters: 1 duration: 240
ns/read: 1085.97  ns/update: 3.53586
elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 227000  n_updates: 67901000  nreaders: 1  nupdaters: 1 duration: 240
ns/read: 1057.27  ns/update: 3.53456
elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 211000  n_updates: 65043000  nreaders: 1  nupdaters: 1 duration: 240
ns/read: 1137.44  ns/update: 3.68987

QNX/aarch64, NXP LX2160A, C11 TLS (emulated)

elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 217000  n_updates: 67814000  nreaders: 1  nupdaters: 1 duration: 240
ns/read: 1105.99  ns/update: 3.53909
elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 223000  n_updates: 67860000  nreaders: 1  nupdaters: 1 duration: 240
ns/read: 1076.23  ns/update: 3.53669
elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 218000  n_updates: 68116000  nreaders: 1  nupdaters: 1 duration: 240
ns/read: 1100.92  ns/update: 3.5234

Looking at the disassembly for the QNX binary I realized that it is still using the emulated TLS option (i.e., the compiler generates a shim layer that uses pthread_[gs]et_specific()). I will need to rebuild the compiler with native TLS support and retest, though I doubt it will have a significant impact.

In any case, both the native and the emulated TLS options show the same results with the GCC and C11 versions.

--Elad

#6 lore raw
Quoted 75 lines · Elad Lahav
Quoted 11 lines · Paul E. McKenney
Quoted 8 lines · Elad Lahav

I believe that performance is statistically the same, but I will double check. I assume both GCC and C11 end up using the same underlying mechanism for thread-local storage:

https://uclibc.org/docs/tls.pdf

If not implemented, TLS falls back on pthread_[gs]et_specific(), but again it would be the same for __thread and _Thread_local.

Sounds likely to me, but I have been surprised before.

Results:

Linux/x86-64, i7-8550U, GCC TLS

elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 408000  n_updates: 689366000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 588.235  ns/update: 0.348146
elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 443000  n_updates: 762876000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 541.761  ns/update: 0.314599
elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 395000  n_updates: 666718000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 607.595  ns/update: 0.359972

Linux/x86-64, i7-8550U, C11 TLS

elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 410000  n_updates: 684880000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 585.366  ns/update: 0.350426
elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 411000  n_updates: 698148000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 583.942  ns/update: 0.343767
elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 409000  n_updates: 704072000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 586.797  ns/update: 0.340874

QNX/aarch64, NXP LX2160A, GCC TLS (emulated)

elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 221000  n_updates: 67876000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 1085.97  ns/update: 3.53586
elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 227000  n_updates: 67901000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 1057.27  ns/update: 3.53456
elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 211000  n_updates: 65043000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 1137.44  ns/update: 3.68987

QNX/aarch64, NXP LX2160A, C11 TLS (emulated)

elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 217000  n_updates: 67814000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 1105.99  ns/update: 3.53909
elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 223000  n_updates: 67860000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 1076.23  ns/update: 3.53669
elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 218000  n_updates: 68116000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 1100.92  ns/update: 3.5234
Quoted 5 lines

Looking at the disassembly for the QNX binary I realized that it is still using the emulated TLS option (i.e., the compiler generates a shim layer that uses pthread_[gs]et_specific()). I will need to rebuild the compiler with native TLS support and retest, though I doubt it will have a significant impact.

That would explain the lack of statistical significance. If there is a change, I would hope that the new style is faster.

Quoted 2 lines

In any case, both the native and the emulated TLS options show the same results with the GCC and C11 versions.

Sounds good, and thank you for checking!

Thanx, Paul

author
#7 lore raw

I also ran the count_end test on Linux/aarch64 on the same board, and got the same results as on QNX. No surprise there, but the discrepancy with the x86_64 results on Linux made me want to double-check that the QNX build is not doing something stupid. I guess that what it shows is that, Apple's M1 chip notwithstanding, ARM still hasn't closed the gap when it comes to raw performance. You can always claim that we are comparing apples and oranges here, but I noticed the same trend on many different boards.

--Elad

Quoted 92 lines · Paul E. McKenney
Quoted 75 lines · Elad Lahav
Quoted 11 lines · Paul E. McKenney
Quoted 8 lines · Elad Lahav

I believe that performance is statistically the same, but I will double check. I assume both GCC and C11 end up using the same underlying mechanism for thread-local storage:

https://uclibc.org/docs/tls.pdf

If not implemented, TLS falls back on pthread_[gs]et_specific(), but again it would be the same for __thread and _Thread_local.

Sounds likely to me, but I have been surprised before.

Results:

Linux/x86-64, i7-8550U, GCC TLS

elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 408000  n_updates: 689366000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 588.235  ns/update: 0.348146
elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 443000  n_updates: 762876000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 541.761  ns/update: 0.314599
elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 395000  n_updates: 666718000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 607.595  ns/update: 0.359972

Linux/x86-64, i7-8550U, C11 TLS

elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 410000  n_updates: 684880000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 585.366  ns/update: 0.350426
elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 411000  n_updates: 698148000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 583.942  ns/update: 0.343767
elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 409000  n_updates: 704072000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 586.797  ns/update: 0.340874

QNX/aarch64, NXP LX2160A, GCC TLS (emulated)

elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 221000  n_updates: 67876000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 1085.97  ns/update: 3.53586
elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 227000  n_updates: 67901000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 1057.27  ns/update: 3.53456
elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 211000  n_updates: 65043000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 1137.44  ns/update: 3.68987

QNX/aarch64, NXP LX2160A, C11 TLS (emulated)

elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 217000  n_updates: 67814000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 1105.99  ns/update: 3.53909
elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 223000  n_updates: 67860000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 1076.23  ns/update: 3.53669
elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 218000  n_updates: 68116000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 1100.92  ns/update: 3.5234
Quoted 5 lines

Looking at the disassembly for the QNX binary I realized that it is still using the emulated TLS option (i.e., the compiler generates a shim layer that uses pthread_[gs]et_specific()). I will need to rebuild the compiler with native TLS support and retest, though I doubt it will have a significant impact.

That would explain the lack of statistical significance. If there is a change, I would hope that the new style is faster.

Quoted 2 lines

In any case, both the native and the emulated TLS options show the same results with the GCC and C11 versions.

Sounds good, and thank you for checking!

Thanx, Paul

#8 lore raw

I am not all that worried about it, mostly curious.

get_timestamp() isn't used all that heavily, and it seems plausible that a 40-nanosecond timestamp would work in rcu_ts.[ch].

Fun times, though!

Thanx, Paul

Quoted 104 lines · Elad Lahav

I also ran the count_end test on Linux/aarch64 on the same board, and got the same results as on QNX. No surprise there, but the discrepancy with the x86_64 results on Linux made me want to double-check that the QNX build is not doing something stupid. I guess that what it shows is that, Apple's M1 chip notwithstanding, ARM still hasn't closed the gap when it comes to raw performance. You can always claim that we are comparing apples and oranges here, but I noticed the same trend on many different boards.

--Elad

Quoted 92 lines · Paul E. McKenney
Quoted 75 lines · Elad Lahav
Quoted 11 lines · Paul E. McKenney
Quoted 8 lines · Elad Lahav

I believe that performance is statistically the same, but I will double check. I assume both GCC and C11 end up using the same underlying mechanism for thread-local storage:

https://uclibc.org/docs/tls.pdf

If not implemented, TLS falls back on pthread_[gs]et_specific(), but again it would be the same for __thread and _Thread_local.

Sounds likely to me, but I have been surprised before.

Results:

Linux/x86-64, i7-8550U, GCC TLS

elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 408000  n_updates: 689366000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 588.235  ns/update: 0.348146
elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 443000  n_updates: 762876000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 541.761  ns/update: 0.314599
elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 395000  n_updates: 666718000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 607.595  ns/update: 0.359972

Linux/x86-64, i7-8550U, C11 TLS

elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 410000  n_updates: 684880000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 585.366  ns/update: 0.350426
elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 411000  n_updates: 698148000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 583.942  ns/update: 0.343767
elahav@lamia:~/src/other/perfbook/CodeSamples/count$ ./count_end
n_reads: 409000  n_updates: 704072000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 586.797  ns/update: 0.340874

QNX/aarch64, NXP LX2160A, GCC TLS (emulated)

elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 221000  n_updates: 67876000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 1085.97  ns/update: 3.53586
elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 227000  n_updates: 67901000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 1057.27  ns/update: 3.53456
elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 211000  n_updates: 65043000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 1137.44  ns/update: 3.68987

QNX/aarch64, NXP LX2160A, C11 TLS (emulated)

elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 217000  n_updates: 67814000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 1105.99  ns/update: 3.53909
elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 223000  n_updates: 67860000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 1076.23  ns/update: 3.53669
elahav@honeycomb:~/src/CodeSamples/count$ ./count_end
n_reads: 218000  n_updates: 68116000  nreaders: 1  nupdaters: 1 duration:
240
ns/read: 1100.92  ns/update: 3.5234
Quoted 5 lines

Looking at the disassembly for the QNX binary I realized that it is still using the emulated TLS option (i.e., the compiler generates a shim layer that uses pthread_[gs]et_specific()). I will need to rebuild the compiler with native TLS support and retest, though I doubt it will have a significant impact.

That would explain the lack of statistical significance. If there is a change, I would hope that the new style is faster.

Quoted 2 lines

In any case, both the native and the emulated TLS options show the same results with the GCC and C11 versions.

Sounds good, and thank you for checking!

Thanx, Paul

author
#12 lore raw
In reply to Paul E. McKenney
Quoted 7 lines · Paul E. McKenney
Quoted 1 line · Elad Lahav

Nice, thank you!

But did you verify that none of the text describing count_end.c needed changing? (It seems unlikely that any would, but could you please check?)

Indeed there were such references. Updated.

Quoted 2 lines

Could you please mention __thread as the old way? Either in the text directly, in parentheses, as a footnote, or as a quick quiz, your choice.

Done. See v2.

--Elad