TSA: make sure QEMU compiles when using clang TSA
cover TSA: make sure QEMU compiles when using clang TSA
This serie aims to enable clang Thread Safety Analysis (TSA) in QEMU. The goal is to use it for our multiqueue project aiming to replace the block layer AioContext lock with a rwlock and make sure the lock is taken correctly everywhere [1].
By default, TSA covers pthread mutexes, therefore when added in QEMU it immediately detects some wrappers using pthread_mutex_lock/unlock without using the proper TSA macros. Since adding such macro requires scanning all possible callers of the affected wrapper, simply use TSA_NO_TSA to suppress the warnings.
[1] = https://lists.gnu.org/archive/html/qemu-devel/2022-12/msg00903.html
Emanuele Giuseppe Esposito (2):
util/qemu-thread-posix: use TSA_NO_TSA to suppress clang TSA warnings bsd-user/mmap: use TSA_NO_TSA to suppress clang TSA warnings
Kevin Wolf (1):
configure: Enable -Wthread-safety if present
configure | 1 + bsd-user/qemu.h | 5 +++-- include/exec/exec-all.h | 5 +++-- include/qemu/thread.h | 14 +++++++++----- util/qemu-thread-posix.c | 2 +- 5 files changed, 17 insertions(+), 10 deletions(-)
Signature · 1 line
2.39.0
PATCH 1/3 util/qemu-thread-posix: use TSA_NO_TSA to suppress clang TSA warnings
QEMU does not compile when enabling clang's thread safety analysis (TSA),
because some functions create wrappers for pthread mutexes but do not use any TSA macro. Therefore the compiler fails.
In order to make the compiler happy and avoid adding all the necessary macros to all callers (lock functions should use TSA_ACQUIRE, while unlock TSA_RELEASE, and this applies to all users of pthread_mutex_lock/pthread_mutex_unlock), simply use TSA_NO_TSA to supppress such warnings.
--- include/qemu/thread.h | 14 +++++++++----- util/qemu-thread-posix.c | 2 +- 2 files changed, 10 insertions(+), 6 deletions(-)
include/qemu/thread.h +9 -5
@@ -3,6 +3,7 @@#include "qemu/processor.h" #include "qemu/atomic.h" +#include "qemu/clang-tsa.h" typedef struct QemuCond QemuCond ;typedef struct QemuSemaphore QemuSemaphore ;
@@ -24,9 +25,12 @@ typedef struct QemuThread QemuThread;void qemu_mutex_init (QemuMutex *mutex);void qemu_mutex_destroy (QemuMutex *mutex); -int qemu_mutex_trylock_impl (QemuMutex *mutex,const char *file,const int line); -void qemu_mutex_lock_impl (QemuMutex *mutex,const char *file,const int line); -void qemu_mutex_unlock_impl (QemuMutex *mutex,const char *file,const int line); +int TSA_NO_TSA qemu_mutex_trylock_impl(QemuMutex *mutex, const char *file, +const int line); +void TSA_NO_TSA qemu_mutex_lock_impl(QemuMutex *mutex, const char *file, +const int line); +void TSA_NO_TSA qemu_mutex_unlock_impl(QemuMutex *mutex, const char *file, +const int line);void qemu_rec_mutex_init (QemuRecMutex *mutex);void qemu_rec_mutex_destroy (QemuRecMutex *mutex);
@@ -153,8 +157,8 @@ void qemu_cond_destroy(QemuCond *cond);*/ void qemu_cond_signal (QemuCond *cond);void qemu_cond_broadcast (QemuCond *cond); -void qemu_cond_wait_impl (QemuCond *cond,QemuMutex *mutex, -const char *file,const int line); +void TSA_NO_TSA qemu_cond_wait_impl(QemuCond *cond, QemuMutex *mutex, +const char *file,const int line);bool qemu_cond_timedwait_impl (QemuCond *cond,QemuMutex *mutex,int ms,const char *file,const int line);
util/qemu-thread-posix.c +1 -1
@@ -223,7 +223,7 @@ void qemu_cond_wait_impl(QemuCond *cond, QemuMutex *mutex, const char *file, conerror_exit (err, __func__); } -static bool +static bool TSA_NO_TSAqemu_cond_timedwait_ts (QemuCond *cond,QemuMutex *mutex,struct timespec *ts,const char *file,const int line) {
Signature · 1 line
2.39.0
Re: [PATCH 1/3] util/qemu-thread-posix: use TSA_NO_TSA to suppress clang TSA warnings
Quoted 22 lines · Emanuele Giuseppe Esposito
QEMU does not compile when enabling clang's thread safety analysis (TSA),
because some functions create wrappers for pthread mutexes but do not use any TSA macro. Therefore the compiler fails.In order to make the compiler happy and avoid adding all the necessary macros to all callers (lock functions should use
TSA_ACQUIRE, while unlockTSA_RELEASE, and this applies to all users of pthread_mutex_lock/pthread_mutex_unlock), simply useTSA_NO_TSAto supppress such warnings.Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- include/qemu/thread.h | 14 +++++++++----- util/qemu-thread-posix.c | 2 +- 2 files changed, 10 insertions(+), 6 deletions(-)include/qemu/thread.h +0
-0
include/qemu/thread.h line 6
Quoted 4 lines
include/qemu/thread.h +1
-0@@ -3,6 +3,7 @@#include "qemu/processor.h" #include "qemu/atomic.h" +#include "qemu/clang-tsa.h"
Missing file? ^^^^^^^^^^
Re: [PATCH 1/3] util/qemu-thread-posix: use TSA_NO_TSA to suppress clang TSA warnings
Am 17/01/2023 um 15:33 schrieb Philippe Mathieu-Daudé:
Quoted 18 lines · Emanuele Giuseppe Esposito
Quoted 17 lines
QEMU does not compile when enabling clang's thread safety analysis (TSA),
because some functions create wrappers for pthread mutexes but do not use any TSA macro. Therefore the compiler fails.In order to make the compiler happy and avoid adding all the necessary macros to all callers (lock functions should use
TSA_ACQUIRE, while unlockTSA_RELEASE, and this applies to all users of pthread_mutex_lock/pthread_mutex_unlock), simply useTSA_NO_TSAto supppress such warnings.Signed-off-by Emanuele Giuseppe Esposito <[email protected]>---
include/qemu/thread.h | 14 +++++++++-----
util/qemu-thread-posix.c| 2 +-2 files changed, 10 insertions(+), 6 deletions(-)
Quoted 11 lines
include/qemu/thread.h line 5
Quoted 8 lines
include/qemu/thread.h +1
-0@@ -3,6 +3,7 @@ #include "qemu/processor.h" #include "qemu/atomic.h" +#include "qemu/clang-tsa.h" Missing file? ^^^^^^^^^^
? Forgot to pull latest changes? I see clang-tsa.h in master
Re: [PATCH 1/3] util/qemu-thread-posix: use TSA_NO_TSA to suppress clang TSA warnings
Quoted 32 lines · Emanuele Giuseppe Esposito
Am 17/01/2023 um 15:33 schrieb Philippe Mathieu-Daudé:
Quoted 18 lines · Emanuele Giuseppe Esposito
Quoted 17 lines
QEMU does not compile when enabling clang's thread safety analysis (TSA),
because some functions create wrappers for pthread mutexes but do not use any TSA macro. Therefore the compiler fails.In order to make the compiler happy and avoid adding all the necessary macros to all callers (lock functions should use
TSA_ACQUIRE, while unlockTSA_RELEASE, and this applies to all users of pthread_mutex_lock/pthread_mutex_unlock), simply useTSA_NO_TSAto supppress such warnings.Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- include/qemu/thread.h | 14 +++++++++----- util/qemu-thread-posix.c | 2 +- 2 files changed, 10 insertions(+), 6 deletions(-)Quoted 11 lines
include/qemu/thread.h line 5
Quoted 8 lines
include/qemu/thread.h +1
-0@@ -3,6 +3,7 @@ #include "qemu/processor.h" #include "qemu/atomic.h" +#include "qemu/clang-tsa.h" Missing file? ^^^^^^^^^^
Quoted 1 line
? Forgot to pull latest changes? I see
clang-tsa.hin master
Oops sorry I forgot to reset a git-bisection, and indeed was based on a older base :\
PATCH 2/3 bsd-user/mmap: use TSA_NO_TSA to suppress clang TSA warnings
In reply to Emanuele Giuseppe EspositoQEMU does not compile when enabling clang's thread safety analysis (TSA),
because some functions create wrappers for pthread mutexes but do not use any TSA macro. Therefore the compiler fails.
In order to make the compiler happy and avoid adding all the
necessary macros to all callers (lock functions should useTSA_ACQUIRE, while unlock TSA_RELEASE, and this applies to allusers of pthread_mutex_lock/pthread_mutex_unlock), simply use TSA_NO_TSA to supppress such warnings.
--- bsd-user/qemu.h | 5 +++-- include/exec/exec-all.h | 5 +++-- 2 files changed, 6 insertions(+), 4 deletions(-)
bsd-user/qemu.h +3 -2
@@ -37,6 +37,7 @@ extern char **environ;#include "target_os_signal.h" #include "target.h" #include "exec/gdbstub.h" +#include "qemu/clang-tsa.h" /* * This struct is used to hold certain information about the image. Basically,
@@ -235,8 +236,8 @@ int target_msync(abi_ulong start, abi_ulong len, int flags);extern unsigned long last_brk;extern abi_ulong mmap_next_start;abi_ulong mmap_find_vma (abi_ulong start,abi_ulong size); -void mmap_fork_start (void ); -void mmap_fork_end (int child); +void TSA_NO_TSA mmap_fork_start(void); +void TSA_NO_TSA mmap_fork_end(int child);/* main.c */ extern char qemu_proc_pathname[];
include/exec/exec-all.h +3 -2
@@ -25,6 +25,7 @@#include "exec/cpu_ldst.h" #endif #include "qemu/interval-tree.h" +#include "qemu/clang-tsa.h" /* allow to see translation results - the slowdown should be negligible, so we leave it */ #define DEBUG_DISAS
@@ -758,8 +759,8 @@ static inline tb_page_addr_t get_page_addr_code(CPUArchState *env, }#if defined(CONFIG_USER_ONLY ) -void mmap_lock (void ); -void mmap_unlock (void ); +void TSA_NO_TSA mmap_lock(void); +void TSA_NO_TSA mmap_unlock(void);bool have_mmap_lock (void );/**
Signature · 1 line
2.39.0
Re: [PATCH 2/3] bsd-user/mmap: use TSA_NO_TSA to suppress clang TSA warnings
Quoted 11 lines · Emanuele Giuseppe Esposito
QEMU does not compile when enabling clang's thread safety analysis (TSA),
because some functions create wrappers for pthread mutexes but do not use any TSA macro. Therefore the compiler fails.In order to make the compiler happy and avoid adding all the necessary macros to all callers (lock functions should use
TSA_ACQUIRE, while unlockTSA_RELEASE, and this applies to allusers of pthread_mutex_lock/pthread_mutex_unlock),
simply useTSA_NO_TSAto supppress such warnings.
I'm not sure I understand this quite right. Maybe a clarifying question will help me understand: Why is this needed for bsd-user but not linux-user? How are they different here?
Warner
Quoted 60 lines
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- bsd-user/qemu.h | 5 +++-- include/exec/exec-all.h | 5 +++-- 2 files changed, 6 insertions(+), 4 deletions(-)bsd-user/qemu.h +1
-0@@ -37,6 +37,7 @@ extern char **environ;#include "target_os_signal.h" #include "target.h" #include "exec/gdbstub.h" +#include "qemu/clang-tsa.h" /* * This struct is used to hold certain information about the image. Basically,
bsd-user/qemu.h +2
-2@@ -235,8 +236,8 @@ int target_msync(abi_ulong start, abi_ulong len, int flags);extern unsigned long last_brk;extern abi_ulong mmap_next_start;abi_ulong mmap_find_vma (abi_ulong start,abi_ulong size); -void mmap_fork_start (void ); -void mmap_fork_end (int child); +void TSA_NO_TSA mmap_fork_start(void); +void TSA_NO_TSA mmap_fork_end(int child);/* main.c */ extern char qemu_proc_pathname[];
include/exec/exec-all.h +1
-0@@ -25,6 +25,7 @@#include "exec/cpu_ldst.h" #endif #include "qemu/interval-tree.h" +#include "qemu/clang-tsa.h" /* allow to see translation results - the slowdown should be negligible, so we leave it */
#defineDEBUG_DISASinclude/exec/exec-all.h +2
-2@@ -758,8 +759,8 @@ static inline tb_page_addr_t get_page_addr_code(CPUArchState *env, }#if defined(CONFIG_USER_ONLY ) -void mmap_lock (void ); -void mmap_unlock (void ); +void TSA_NO_TSA mmap_lock(void); +void TSA_NO_TSA mmap_unlock(void);bool have_mmap_lock (void );/**
--
2.39.0
Re: [PATCH 2/3] bsd-user/mmap: use TSA_NO_TSA to suppress clang TSA warnings
Am 17/01/2023 um 17:16 schrieb Warner Losh:
Quoted 17 lines
On Tue, Jan 17, 2023 at 6:52 AM Emanuele Giuseppe Esposito <[email protected] <mailto:[email protected]>> wrote:
QEMU does not compile when enabling clang's thread safety analysis (TSA),
because some functions create wrappers for pthread mutexes but do not use any TSA macro. Therefore the compiler fails.In order to make the compiler happy and avoid adding all the necessary macros to all callers (lock functions should use
TSA_ACQUIRE, while unlockTSA_RELEASE, and this applies to allusers of pthread_mutex_lock/pthread_mutex_unlock),
simply useTSA_NO_TSAto supppress such warnings.
Quoted 4 lines
I'm not sure I understand this quite right. Maybe a clarifying question will help me understand: Why is this needed for bsd-user but not linux-user? How are they different here?
Honestly I just went and fix the warnings that the compiler was throwing at me. On FreeBSD it complains on bsd-user/mmap.c, but apparently the CI or even compiling locally in linux doesn't create any issue with linux-user.
Weird. But I guess it won't hurt to add TSA_NO_TSA there too.
Emanuele
Quoted 63 lines
Warner
Signed-off-by: Emanuele Giuseppe Esposito <[email protected] <mailto:[email protected]>> --- bsd-user/qemu.h | 5 +++-- include/exec/exec-all.h | 5 +++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/bsd-user/qemu.h b/bsd-user/qemu.h index be6105385e..711fdd1b64 100644 --- a/bsd-user/qemu.h +++ b/bsd-user/qemu.h @@ -37,6 +37,7 @@ extern char **environ; #include "target_os_signal.h" #include "target.h" #include "exec/gdbstub.h" +#include "qemu/clang-tsa.h" /* * This struct is used to hold certain information about the image. Basically, @@ -235,8 +236,8 @@ int target_msync(abi_ulong start, abi_ulong len, int flags); extern unsigned long last_brk; extern abi_ulong mmap_next_start; abi_ulong mmap_find_vma(abi_ulong start, abi_ulong size); -void mmap_fork_start(void); -void mmap_fork_end(int child); +void TSA_NO_TSA mmap_fork_start(void); +void TSA_NO_TSA mmap_fork_end(int child); /* main.c */ extern char qemu_proc_pathname[]; diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h index 25e11b0a8d..4f0c0559ac 100644 --- a/include/exec/exec-all.h +++ b/include/exec/exec-all.h @@ -25,6 +25,7 @@ #include "exec/cpu_ldst.h" #endif #include "qemu/interval-tree.h" +#include "qemu/clang-tsa.h" /* allow to see translation results - the slowdown should be negligible, so we leave it */ #define DEBUG_DISAS @@ -758,8 +759,8 @@ static inline tb_page_addr_t get_page_addr_code(CPUArchState *env, } #if defined(CONFIG_USER_ONLY) -void mmap_lock(void); -void mmap_unlock(void); +void TSA_NO_TSA mmap_lock(void); +void TSA_NO_TSA mmap_unlock(void); bool have_mmap_lock(void); /** -- 2.39.0
Re: [PATCH 2/3] bsd-user/mmap: use TSA_NO_TSA to suppress clang TSA warnings
In reply to Warner LoshAm 17.01.2023 um 17:16 hat Warner Losh geschrieben:
Quoted 14 lines
Quoted 10 lines · Emanuele Giuseppe Esposito
QEMU does not compile when enabling clang's thread safety analysis (TSA),
because some functions create wrappers for pthread mutexes but do not use any TSA macro. Therefore the compiler fails.In order to make the compiler happy and avoid adding all the necessary macros to all callers (lock functions should use
TSA_ACQUIRE, while unlockTSA_RELEASE, and this applies to allusers of pthread_mutex_lock/pthread_mutex_unlock),
simply useTSA_NO_TSAto supppress such warnings.
Quoted 3 lines
I'm not sure I understand this quite right. Maybe a clarifying question will help me understand: Why is this needed for bsd-user but not linux-user? How are they different here?
FreeBSD's pthread headers include TSA annotations for some functions that force us to do something about them (for now: suppress the warnings in their callers) before we can enable -Wthread-safety for the purposes where we really want it. Without this, calling functions like pthread_mutex_lock() would cause compiler errors.
glibc's headers don't contain such annotations, so the same is not necessary on Linux.
Kevin
Warner Losh
Reviewed-byRe: [PATCH 2/3] bsd-user/mmap: use TSA_NO_TSA to suppress clang TSA warnings
Quoted 19 lines · Kevin Wolf
Am 17.01.2023 um 17:16 hat Warner Losh geschrieben:
Quoted 14 lines
Quoted 10 lines · Emanuele Giuseppe Esposito
QEMU does not compile when enabling clang's thread safety analysis (TSA),
because some functions create wrappers for pthread mutexes but do not use any TSA macro. Therefore the compiler fails.In order to make the compiler happy and avoid adding all the necessary macros to all callers (lock functions should use
TSA_ACQUIRE, while unlockTSA_RELEASE, and this applies to allusers of pthread_mutex_lock/pthread_mutex_unlock),
simply useTSA_NO_TSAto supppress such warnings.Quoted 3 lines
I'm not sure I understand this quite right. Maybe a clarifying question will help me understand: Why is this needed for bsd-user but not linux-user? How are they different here?
Quoted 9 lines
FreeBSD's pthread headers include TSA annotations for some functions that force us to do something about them (for now: suppress the warnings in their callers) before we can enable -Wthread-safety for the purposes where we really want it. Without this, calling functions like
pthread_mutex_lock()would cause compiler errors.glibc's headers don't contain such annotations, so the same is not necessary on Linux
Thanks Kevin. With that explanation, these patches and their explanation make perfect sense now. Often when there's a patch to bsd-user but not linux-user, it's because bsd-user needs to do more in some way (which I try to keep up on).
In this case, it's because FreeBSD's libc is a bit ahead of the curve. So I understand why it's needed, and what I need to do next (though I think that I may have to wait for the rest of qemu to be annotated)...
It might be better, though, to put some of this information in the commit message so it isn't just on the mailing list. Just a suggestion:
Re: [PATCH 2/3] bsd-user/mmap: use TSA_NO_TSA to suppress clang TSA warnings
Am 17.01.2023 um 17:43 hat Warner Losh geschrieben:
Quoted 31 lines · Kevin Wolf
Quoted 20 lines
Am 17.01.2023 um 17:16 hat Warner Losh geschrieben:
Quoted 14 lines
Quoted 10 lines · Emanuele Giuseppe Esposito
QEMU does not compile when enabling clang's thread safety analysis (TSA),
because some functions create wrappers for pthread mutexes but do not use any TSA macro. Therefore the compiler fails.In order to make the compiler happy and avoid adding all the necessary macros to all callers (lock functions should use
TSA_ACQUIRE, while unlockTSA_RELEASE, and this applies to allusers of pthread_mutex_lock/pthread_mutex_unlock),
simply useTSA_NO_TSAto supppress such warnings.Quoted 3 lines
I'm not sure I understand this quite right. Maybe a clarifying question will help me understand: Why is this needed for bsd-user but not linux-user? How are they different here?
Quoted 9 lines
FreeBSD's pthread headers include TSA annotations for some functions that force us to do something about them (for now: suppress the warnings in their callers) before we can enable -Wthread-safety for the purposes where we really want it. Without this, calling functions like
pthread_mutex_lock()would cause compiler errors.glibc's headers don't contain such annotations, so the same is not necessary on Linux
Quoted 8 lines
Thanks Kevin. With that explanation, these patches and their explanation make perfect sense now. Often when there's a patch to bsd-user but not linux-user, it's because bsd-user needs to do more in some way (which I try to keep up on).
In this case, it's because FreeBSD's libc is a bit ahead of the curve. So I understand why it's needed, and what I need to do next (though I think that I may have to wait for the rest of qemu to be annotated)...
I assume that the bsd-user part is actually sufficiently independent that you could do proper annotations there if you want.
However, be aware that TSA has some serious limitations with C, so you can't express certain things, and it isn't as strict as it could be (in particular, function pointers bypass it). As long as you have global locks (as opposed to locks in structs), it kind of works, though. Certainly better than nothing.
But it probably means that some of the rest of QEMU may never get the annotations. Also, our primary goal is protecting the block layer, so someone else would have to work on other locks. With checks disabled on individual functions like in this series, it should at least be possible to work on it incrementally.
Quoted 2 lines
It might be better, though, to put some of this information in the commit message so it isn't just on the mailing list.
Yes, I agree. We can tweak the commit messages before merging it.
Quoted 3 lines
Just a suggestion:
Reviewed-by Warner Losh <[email protected]>
Thanks!
Kevin
Re: [PATCH 2/3] bsd-user/mmap: use TSA_NO_TSA to suppress clang TSA warnings
Quoted 42 lines · Kevin Wolf
Am 17.01.2023 um 17:43 hat Warner Losh geschrieben:
Quoted 31 lines · Kevin Wolf
Quoted 20 lines
Am 17.01.2023 um 17:16 hat Warner Losh geschrieben:
Quoted 14 lines
Quoted 10 lines · Emanuele Giuseppe Esposito
QEMU does not compile when enabling clang's thread safety analysis (TSA),
because some functions create wrappers for pthread mutexes but do not use any TSA macro. Therefore the compiler fails.In order to make the compiler happy and avoid adding all the necessary macros to all callers (lock functions should use
TSA_ACQUIRE, while unlockTSA_RELEASE, and this applies to allusers of pthread_mutex_lock/pthread_mutex_unlock),
simply useTSA_NO_TSAto supppress such warnings.Quoted 3 lines
I'm not sure I understand this quite right. Maybe a clarifying question will help me understand: Why is this needed for bsd-user but not linux-user? How are they different here?
Quoted 9 lines
FreeBSD's pthread headers include TSA annotations for some functions that force us to do something about them (for now: suppress the warnings in their callers) before we can enable -Wthread-safety for the purposes where we really want it. Without this, calling functions like
pthread_mutex_lock()would cause compiler errors.glibc's headers don't contain such annotations, so the same is not necessary on Linux
Quoted 8 lines
Thanks Kevin. With that explanation, these patches and their explanation make perfect sense now. Often when there's a patch to bsd-user but not linux-user, it's because bsd-user needs to do more in some way (which I try to keep up on).
In this case, it's because FreeBSD's libc is a bit ahead of the curve. So I understand why it's needed, and what I need to do next (though I think that I may have to wait for the rest of qemu to be annotated)...
Quoted 8 lines
I assume that the bsd-user part is actually sufficiently independent that you could do proper annotations there if you want.
However, be aware that TSA has some serious limitations with C, so you can't express certain things, and it isn't as strict as it could be (in particular, function pointers bypass it). As long as you have global locks (as opposed to locks in structs), it kind of works, though. Certainly better than nothing.
What are the limitations on locks in structs (a common case)?
Stefan
Re: [PATCH 2/3] bsd-user/mmap: use TSA_NO_TSA to suppress clang TSA warnings
Am 17.01.2023 um 21:43 hat Stefan Hajnoczi geschrieben:
Quoted 46 lines · Kevin Wolf
Quoted 45 lines
Am 17.01.2023 um 17:43 hat Warner Losh geschrieben:
Quoted 31 lines · Kevin Wolf
Quoted 20 lines
Am 17.01.2023 um 17:16 hat Warner Losh geschrieben:
Quoted 14 lines
Quoted 10 lines · Emanuele Giuseppe Esposito
QEMU does not compile when enabling clang's thread safety analysis (TSA),
because some functions create wrappers for pthread mutexes but do not use any TSA macro. Therefore the compiler fails.In order to make the compiler happy and avoid adding all the necessary macros to all callers (lock functions should use
TSA_ACQUIRE, while unlockTSA_RELEASE, and this applies to allusers of pthread_mutex_lock/pthread_mutex_unlock),
simply useTSA_NO_TSAto supppress such warnings.Quoted 3 lines
I'm not sure I understand this quite right. Maybe a clarifying question will help me understand: Why is this needed for bsd-user but not linux-user? How are they different here?
Quoted 9 lines
FreeBSD's pthread headers include TSA annotations for some functions that force us to do something about them (for now: suppress the warnings in their callers) before we can enable -Wthread-safety for the purposes where we really want it. Without this, calling functions like
pthread_mutex_lock()would cause compiler errors.glibc's headers don't contain such annotations, so the same is not necessary on Linux
Quoted 8 lines
Thanks Kevin. With that explanation, these patches and their explanation make perfect sense now. Often when there's a patch to bsd-user but not linux-user, it's because bsd-user needs to do more in some way (which I try to keep up on).
In this case, it's because FreeBSD's libc is a bit ahead of the curve. So I understand why it's needed, and what I need to do next (though I think that I may have to wait for the rest of qemu to be annotated)...
I assume that the bsd-user part is actually sufficiently independent that you could do proper annotations there if you want.
Quoted 7 lines
Quoted 5 lines
However, be aware that TSA has some serious limitations with C, so you can't express certain things, and it isn't as strict as it could be (in particular, function pointers bypass it). As long as you have global locks (as opposed to locks in structs), it kind of works, though. Certainly better than nothing.
What are the limitations on locks in structs (a common case)?
TSA_GUARDED_BY() can't refer to a mutex in the same struct in C. You would have to have something like 'this', but it just doesn't exist. (I think in C++ you don't actually need 'this' because name resolution automatically starts at the struct or something - I neither know C++ well enough nor TSA with it, so take this with a grain of salt.)
You can still annotate functions for such structs in C, because then you have a name for the struct, like this:
void lock(Foo *foo) TSA_REQUIRES(foo->mutex);
Kevin
Re: [PATCH 2/3] bsd-user/mmap: use TSA_NO_TSA to suppress clang TSA warnings
Quoted 73 lines · On Wed, Jan 18, 2023, 04:15 Kevin Wolf <[email protected]> wrote:
Am 17.01.2023 um 21:43 hat Stefan Hajnoczi geschrieben:
Quoted 10 lines · Kevin Wolf
Quoted 9 lines
Am 17.01.2023 um 17:43 hat Warner Losh geschrieben:
Quoted 7 lines · Kevin Wolf
Quoted 6 lines
Am 17.01.2023 um 17:16 hat Warner Losh geschrieben:
Quoted 4 lines
Quoted 1 line · Emanuele Giuseppe Esposito
QEMU does not compile when enabling clang's thread safety
analysis
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
(TSA),
because some functions create wrappers for pthread mutexes butdo
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
not use any TSA macro. Therefore the compiler fails.
In order to make the compiler happy and avoid adding all the necessary macros to all callers (lock functions should use
TSA_ACQUIRE, while unlockTSA_RELEASE, and this applies toallusers of
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
pthread_mutex_lock/pthread_mutex_unlock),
simply useTSA_NO_TSAto supppress such warnings.I'm not sure I understand this quite right. Maybe a clarifying
question
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
will help me understand: Why is this needed for bsd-user but not linux-user? How are they different here?
FreeBSD's pthread headers include TSA annotations for some
functions
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
that force us to do something about them (for now: suppress the
warnings
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
in their callers) before we can enable -Wthread-safety for the
purposes
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
where we really want it. Without this, calling functions like
pthread_mutex_lock()would cause compiler errors.glibc's headers don't contain such annotations, so the same is not necessary on Linux
Thanks Kevin. With that explanation, these patches and their
explanation
Quoted 1 line
Quoted 1 line
Quoted 1 line
make perfect sense now. Often when there's a patch to bsd-user but
not
Quoted 1 line
Quoted 1 line
Quoted 1 line
linux-user, it's because bsd-user needs to do more in some way
(which I try
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
to keep up on).
In this case, it's because FreeBSD's libc is a bit ahead of the
curve. So I
Quoted 1 line
Quoted 1 line
Quoted 1 line
understand why it's needed, and what I need to do next (though I
think that
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
I may have to wait for the rest of qemu to be annotated)...
I assume that the bsd-user part is actually sufficiently independent that you could do proper annotations there if you want.
Quoted 7 lines
Quoted 5 lines
However, be aware that TSA has some serious limitations with C, so you can't express certain things, and it isn't as strict as it could be (in particular, function pointers bypass it). As long as you have global locks (as opposed to locks in structs), it kind of works, though. Certainly better than nothing.
What are the limitations on locks in structs (a common case)?
TSA_GUARDED_BY()can't refer to a mutex in the same struct in C. You would have to have something like 'this', but it just doesn't exist. (I think in C++ you don't actually need 'this' because name resolution automatically starts at the struct or something - I neither know C++ well enough nor TSA with it, so take this with a grain of salt.)
Quoted 5 lines
You can still annotate functions for such structs in C, because then you have a name for the struct, like this:
void
lock(Foo *foo)TSA_REQUIRES(foo->mutex);
Thanks for the explanation!
Stefan
Quoted 1 line
Re: [PATCH 2/3] bsd-user/mmap: use TSA_NO_TSA to suppress clang TSA warnings
In reply to Kevin WolfAm 17/01/2023 um 18:17 schrieb Kevin Wolf:
Quoted 56 lines
Am 17.01.2023 um 17:43 hat Warner Losh geschrieben:
Quoted 31 lines · Kevin Wolf
Quoted 20 lines
Am 17.01.2023 um 17:16 hat Warner Losh geschrieben:
Quoted 14 lines
Quoted 10 lines · Emanuele Giuseppe Esposito
QEMU does not compile when enabling clang's thread safety analysis (TSA),
because some functions create wrappers for pthread mutexes but do not use any TSA macro. Therefore the compiler fails.In order to make the compiler happy and avoid adding all the necessary macros to all callers (lock functions should use
TSA_ACQUIRE, while unlockTSA_RELEASE, and this applies to allusers of pthread_mutex_lock/pthread_mutex_unlock),
simply useTSA_NO_TSAto supppress such warnings.Quoted 3 lines
I'm not sure I understand this quite right. Maybe a clarifying question will help me understand: Why is this needed for bsd-user but not linux-user? How are they different here?
Quoted 9 lines
FreeBSD's pthread headers include TSA annotations for some functions that force us to do something about them (for now: suppress the warnings in their callers) before we can enable -Wthread-safety for the purposes where we really want it. Without this, calling functions like
pthread_mutex_lock()would cause compiler errors.glibc's headers don't contain such annotations, so the same is not necessary on Linux
Quoted 8 lines
Thanks Kevin. With that explanation, these patches and their explanation make perfect sense now. Often when there's a patch to bsd-user but not linux-user, it's because bsd-user needs to do more in some way (which I try to keep up on).
In this case, it's because FreeBSD's libc is a bit ahead of the curve. So I understand why it's needed, and what I need to do next (though I think that I may have to wait for the rest of qemu to be annotated)...
I assume that the bsd-user part is actually sufficiently independent that you could do proper annotations there if you want.
However, be aware that TSA has some serious limitations with C, so you can't express certain things, and it isn't as strict as it could be (in particular, function pointers bypass it). As long as you have global locks (as opposed to locks in structs), it kind of works, though. Certainly better than nothing.
But it probably means that some of the rest of QEMU may never get the annotations. Also, our primary goal is protecting the block layer, so someone else would have to work on other locks. With checks disabled on individual functions like in this series, it should at least be possible to work on it incrementally.
Quoted 4 lines
Quoted 2 lines
It might be better, though, to put some of this information in the commit message so it isn't just on the mailing list.
Yes, I agree. We can tweak the commit messages before merging it.
New proposed commit message:
bsd-user/mmap: use TSA_NO_TSA to suppress clang TSA warnings in FreeBSD
FreeBSD implements pthread headers using TSA (thread safety analysis) annotations, therefore when an application is compiled with -Wthread-safety there are some locking/annotation requirements that the user of the pthread API has to follow.
This will also be the case in QEMU, since bsd-user/mmap.c uses the pthread API. Therefore when building it with -Wthread-safety the compiler will throw warnings because the functions are not properly annotated. We need TSA to be enabled because it ensures that the critical sections of an annotated variable are properly locked.
In order to make the compiler happy and avoid adding all the necessary macros to all callers (lock functions should use TSA_ACQUIRE, while unlock TSA_RELEASE, and this applies to all users of pthread_mutex_lock/pthread_mutex_unlock), simply use TSA_NO_TSA to supppress such warnings.
Same message could be applied to patch 1, substituting bsd-user/mmap with util/qemu-thread-posix.
Thank you,
Emanuele
Quoted 9 lines
Quoted 4 lines
Just a suggestion:
Reviewed-by Warner Losh <[email protected]>Thanks!
Kevin
Re: [PATCH 2/3] bsd-user/mmap: use TSA_NO_TSA to suppress clang TSA warnings
Quoted 86 lines · Emanuele Giuseppe Esposito
Am 17/01/2023 um 18:17 schrieb Kevin Wolf:
Quoted 56 lines
Am 17.01.2023 um 17:43 hat Warner Losh geschrieben:
Quoted 31 lines · Kevin Wolf
Quoted 20 lines
Am 17.01.2023 um 17:16 hat Warner Losh geschrieben:
Quoted 14 lines
Quoted 10 lines · Emanuele Giuseppe Esposito
QEMU does not compile when enabling clang's thread safety analysis (TSA),
because some functions create wrappers for pthread mutexes but do not use any TSA macro. Therefore the compiler fails.In order to make the compiler happy and avoid adding all the necessary macros to all callers (lock functions should use
TSA_ACQUIRE, while unlockTSA_RELEASE, and this applies to allusers of pthread_mutex_lock/pthread_mutex_unlock),
simply useTSA_NO_TSAto supppress such warnings.Quoted 3 lines
I'm not sure I understand this quite right. Maybe a clarifying question will help me understand: Why is this needed for bsd-user but not linux-user? How are they different here?
Quoted 9 lines
FreeBSD's pthread headers include TSA annotations for some functions that force us to do something about them (for now: suppress the warnings in their callers) before we can enable -Wthread-safety for the purposes where we really want it. Without this, calling functions like
pthread_mutex_lock()would cause compiler errors.glibc's headers don't contain such annotations, so the same is not necessary on Linux
Quoted 8 lines
Thanks Kevin. With that explanation, these patches and their explanation make perfect sense now. Often when there's a patch to bsd-user but not linux-user, it's because bsd-user needs to do more in some way (which I try to keep up on).
In this case, it's because FreeBSD's libc is a bit ahead of the curve. So I understand why it's needed, and what I need to do next (though I think that I may have to wait for the rest of qemu to be annotated)...
I assume that the bsd-user part is actually sufficiently independent that you could do proper annotations there if you want.
However, be aware that TSA has some serious limitations with C, so you can't express certain things, and it isn't as strict as it could be (in particular, function pointers bypass it). As long as you have global locks (as opposed to locks in structs), it kind of works, though. Certainly better than nothing.
But it probably means that some of the rest of QEMU may never get the annotations. Also, our primary goal is protecting the block layer, so someone else would have to work on other locks. With checks disabled on individual functions like in this series, it should at least be possible to work on it incrementally.
Quoted 4 lines
Quoted 2 lines
It might be better, though, to put some of this information in the commit message so it isn't just on the mailing list.
Yes, I agree. We can tweak the commit messages before merging it.
New proposed commit message:
bsd-user/mmap: use
TSA_NO_TSAto suppress clang TSA warnings in FreeBSDFreeBSD implements pthread headers using TSA (thread safety analysis) annotations, therefore when an application is compiled with -Wthread-safety there are some locking/annotation requirements that the user of the pthread API has to follow.
This will also be the case in QEMU, since
bsd-user/mmap.cuses the pthread API. Therefore when building it with -Wthread-safety the compiler will throw warnings because the functions are not properly annotated. We need TSA to be enabled because it ensures that the critical sections of an annotated variable are properly locked.In order to make the compiler happy and avoid adding all the necessary macros to all callers (lock functions should use
TSA_ACQUIRE, while unlockTSA_RELEASE, and this applies to all users of pthread_mutex_lock/pthread_mutex_unlock), simply useTSA_NO_TSAto supppress such warnings.
Quoted 4 lines
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>Same message could be applied to patch 1, substituting bsd-user/mmap with util/qemu-thread-posix.
Looks good to me.
Stefan
Re: [PATCH 2/3] bsd-user/mmap: use TSA_NO_TSA to suppress clang TSA warnings
In reply to Emanuele Giuseppe EspositoQuoted 87 lines · Emanuele Giuseppe Esposito
Am 17/01/2023 um 18:17 schrieb Kevin Wolf:
Quoted 15 lines
Am 17.01.2023 um 17:43 hat Warner Losh geschrieben:
Quoted 14 lines · Kevin Wolf
Quoted 13 lines
Am 17.01.2023 um 17:16 hat Warner Losh geschrieben:
Quoted 11 lines
Quoted 8 lines · Emanuele Giuseppe Esposito
QEMU does not compile when enabling clang's thread safety analysis (TSA),
because some functions create wrappers for pthread mutexes but do not use any TSA macro. Therefore the compiler fails.In order to make the compiler happy and avoid adding all the necessary macros to all callers (lock functions should use
TSA_ACQUIRE, while unlockTSA_RELEASE, and this applies to allusersof
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
pthread_mutex_lock/pthread_mutex_unlock),
simply useTSA_NO_TSAto supppress such warnings.I'm not sure I understand this quite right. Maybe a clarifying
question
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
will help me understand: Why is this needed for bsd-user but not linux-user? How are they different here?
FreeBSD's pthread headers include TSA annotations for some functions that force us to do something about them (for now: suppress the
warnings
Quoted 11 lines
Quoted 11 lines
Quoted 7 lines
in their callers) before we can enable -Wthread-safety for the purposes where we really want it. Without this, calling functions like
pthread_mutex_lock()would cause compiler errors.glibc's headers don't contain such annotations, so the same is not necessary on Linux
Thanks Kevin. With that explanation, these patches and their explanation make perfect sense now. Often when there's a patch to bsd-user but not linux-user, it's because bsd-user needs to do more in some way (which I
try
Quoted 3 lines
Quoted 3 lines
to keep up on).
In this case, it's because FreeBSD's libc is a bit ahead of the curve.
So I
Quoted 1 line
Quoted 1 line
understand why it's needed, and what I need to do next (though I think
that
Quoted 11 lines
Quoted 1 line
I may have to wait for the rest of qemu to be annotated)...
I assume that the bsd-user part is actually sufficiently independent that you could do proper annotations there if you want.
However, be aware that TSA has some serious limitations with C, so you can't express certain things, and it isn't as strict as it could be (in particular, function pointers bypass it). As long as you have global locks (as opposed to locks in structs), it kind of works, though. Certainly better than nothing.
Quoted 7 lines
But it probably means that some of the rest of QEMU may never get the annotations. Also, our primary goal is protecting the block layer, so someone else would have to work on other locks. With checks disabled on individual functions like in this series, it should at least be possible to work on it incrementally.
Quoted 1 line
It might be better, though, to put some of this information in the
commit
Quoted 3 lines
Quoted 1 line
message so it isn't just on the mailing list.
Yes, I agree. We can tweak the commit messages before merging it.
New proposed commit message:
bsd-user/mmap: use
TSA_NO_TSAto suppress clang TSA warnings in FreeBSDFreeBSD implements pthread headers using TSA (thread safety analysis) annotations, therefore when an application is compiled with -Wthread-safety there are some locking/annotation requirements that the user of the pthread API has to follow.
This will also be the case in QEMU, since
bsd-user/mmap.cuses the pthread API. Therefore when building it with -Wthread-safety the compiler will throw warnings because the functions are not properly annotated. We need TSA to be enabled because it ensures that the critical sections of an annotated variable are properly locked.
Quoted 6 lines
In order to make the compiler happy and avoid adding all the necessary macros to all callers (lock functions should use
TSA_ACQUIRE, while unlockTSA_RELEASE, and this applies to all users of pthread_mutex_lock/pthread_mutex_unlock), simply useTSA_NO_TSAto supppress such warnings.
Looks good to me.
Warner
Quoted 20 lines
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>Same message could be applied to patch 1, substituting bsd-user/mmap with util/qemu-thread-posix.
Thank you,
EmanueleQuoted 9 lines
Quoted 4 lines
Just a suggestion:
Reviewed-by Warner Losh <[email protected]>Thanks!
Kevin
Re: [PATCH 2/3] bsd-user/mmap: use TSA_NO_TSA to suppress clang TSA warnings
In reply to Emanuele Giuseppe EspositoQuoted 10 lines · Emanuele Giuseppe Esposito
QEMU does not compile when enabling clang's thread safety analysis (TSA),
because some functions create wrappers for pthread mutexes but do not use any TSA macro. Therefore the compiler fails.In order to make the compiler happy and avoid adding all the
necessary macros to all callers (lock functions should useTSA_ACQUIRE, while unlockTSA_RELEASE, and this applies to allusers of pthread_mutex_lock/pthread_mutex_unlock), simply useTSA_NO_TSAto supppress such warnings.
Quoted 5 lines
Signed-off-by Emanuele Giuseppe Esposito <[email protected]>--- bsd-user/qemu.h | 5 +++-- include/exec/exec-all.h | 5 +++-- 2 files changed, 6 insertions(+), 4 deletions(-)
As a TSA newbie I'm wondering how would we go about annotating this properly?
Maybe:
1. mmap_lock() ACQUIRE(), mmap_unlock() RELEASE()
2. Find all functions that call mmap_lock() but not mmap_release() and add ACQUIRE().
3. Find all functions that call mmap_unlock() but not mmap_lock() and add RELEASE().
Can you add an item to https://wiki.qemu.org/BiteSizedTasks so someone who wants to spend a few hours auditing the code can do this?
Thanks!
PATCH 3/3 configure: Enable -Wthread-safety if present
In reply to Emanuele Giuseppe EspositoFrom: Kevin Wolf <[email protected]>
This enables clang's thread safety analysis (TSA), which we'll use to statically check the block graph locking.
Message-Id: <[email protected]>
--- configure | 1 + 1 file changed, 1 insertion(+)
configure +1 -0
@@ -1183,6 +1183,7 @@ add_to warn_flags -Wnested-externs add_to warn_flags -Wendif-labels add_to warn_flags -Wexpansion-to-defined add_to warn_flags -Wimplicit-fallthrough=2 +add_to warn_flags -Wthread-safety nowarn_flags= add_to nowarn_flags -Wno-initializer-overrides
Signature · 1 line
2.39.0
Re: [PATCH 3/3] configure: Enable -Wthread-safety if present
Quoted 18 lines · Emanuele Giuseppe Esposito
From: Kevin Wolf <[email protected]>
This enables clang's thread safety analysis (TSA), which we'll use to statically check the block graph locking.
Signed-off-by Kevin Wolf <[email protected]>Message-Id: <[email protected]>
Reviewed-by Emanuele Giuseppe Esposito <[email protected]>Signed-off-by Kevin Wolf <[email protected]>--- configure | 1 + 1 file changed, 1 insertion(+)configure +0
-0
configure add_to warn_flags -Wnested-externs · line 1186
Quoted 4 lines
configure +1
-0@@ -1183,6 +1183,7 @@ add_to warn_flags -Wnested-externs add_to warn_flags -Wendif-labels add_to warn_flags -Wexpansion-to-defined add_to warn_flags -Wimplicit-fallthrough=2 +add_to warn_flags -Wthread-safety
Does this thread safety analysis have any kind of measurable impact on compilation speed ?
Our CI jobs are quite sensitive to any increase in build time.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH 3/3] configure: Enable -Wthread-safety if present
Am 17/01/2023 um 15:02 schrieb Daniel P. Berrangé:
Quoted 27 lines · Emanuele Giuseppe Esposito
Quoted 18 lines
From: Kevin Wolf <[email protected]>
This enables clang's thread safety analysis (TSA), which we'll use to statically check the block graph locking.
Signed-off-by Kevin Wolf <[email protected]>Message-Id: <[email protected]>
Reviewed-by Emanuele Giuseppe Esposito <[email protected]>Signed-off-by Kevin Wolf <[email protected]>--- configure | 1 + 1 file changed, 1 insertion(+)configure +0
-0configure add_to warn_flags -Wnested-externs · line 1186
Quoted 4 lines
configure +1
-0@@ -1183,6 +1183,7 @@ add_to warn_flags -Wnested-externs add_to warn_flags -Wendif-labels add_to warn_flags -Wexpansion-to-defined add_to warn_flags -Wimplicit-fallthrough=2 +add_to warn_flags -Wthread-safetyDoes this thread safety analysis have any kind of measurable impact on compilation speed ?
Quoted 2 lines
Our CI jobs are quite sensitive to any increase in build time.
From a quick run in my machine (Dell PowerEdge R440 with Intel(R) Xeon(R) Gold 5120 CPU @ 2.20GHz, 28 cpus):
without clang:
real 2m46.729s
user 19m24.122s
sys 2m58.643s
with clang:
real 2m45.204s user 19m52.096s sys 2m9.036s
So there should be no significative impact.
I also forgot to mention that this serie fixes the CI failure seen in:
https://gitlab.com/qemu-project/qemu/-/jobs/3479763741
https://gitlab.com/qemu-project/qemu/-/jobs/3479763746
Thank you,
Emanuele
Quoted 4 lines
With regards,
Daniel
Re: [PATCH 3/3] configure: Enable -Wthread-safety if present
Quoted 46 lines · Emanuele Giuseppe Esposito
Am 17/01/2023 um 15:02 schrieb Daniel P. Berrangé:
Quoted 27 lines · Emanuele Giuseppe Esposito
Quoted 18 lines
From: Kevin Wolf <[email protected]>
This enables clang's thread safety analysis (TSA), which we'll use to statically check the block graph locking.
Signed-off-by Kevin Wolf <[email protected]>Message-Id: <[email protected]>
Reviewed-by Emanuele Giuseppe Esposito <[email protected]>Signed-off-by Kevin Wolf <[email protected]>--- configure | 1 + 1 file changed, 1 insertion(+)configure +0
-0configure add_to warn_flags -Wnested-externs · line 1186
Quoted 4 lines
configure +1
-0@@ -1183,6 +1183,7 @@ add_to warn_flags -Wnested-externs add_to warn_flags -Wendif-labels add_to warn_flags -Wexpansion-to-defined add_to warn_flags -Wimplicit-fallthrough=2 +add_to warn_flags -Wthread-safetyDoes this thread safety analysis have any kind of measurable impact on compilation speed ?
Quoted 2 lines
Our CI jobs are quite sensitive to any increase in build time.
From a quick run in my machine (Dell PowerEdge R440 with Intel(R) Xeon(R) Gold 5120 CPU @ 2.20GHz, 28 cpus):
without clang:
real 2m46.729s
user 19m24.122s
sys 2m58.643swith clang:
real 2m45.204s user 19m52.096s sys 2m9.036s
Quoted 6 lines
So there should be no significative impact.
I also forgot to mention that this serie fixes the CI failure seen in:
https://gitlab.com/qemu-project/qemu/-/jobs/3479763741
https://gitlab.com/qemu-project/qemu/-/jobs/3479763746
Odd, that job already has -Wthread-safety included in CFLAGS, which would seem to make this patch redundant, but I don't see where -Wthread-safety came from in that pipeline
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH 3/3] configure: Enable -Wthread-safety if present
Am 17.01.2023 um 16:01 hat Daniel P. Berrangé geschrieben:
Quoted 49 lines · Emanuele Giuseppe Esposito
Quoted 48 lines
Am 17/01/2023 um 15:02 schrieb Daniel P. Berrangé:
Quoted 27 lines · Emanuele Giuseppe Esposito
Quoted 18 lines
From: Kevin Wolf <[email protected]>
This enables clang's thread safety analysis (TSA), which we'll use to statically check the block graph locking.
Signed-off-by Kevin Wolf <[email protected]>Message-Id: <[email protected]>
Reviewed-by Emanuele Giuseppe Esposito <[email protected]>Signed-off-by Kevin Wolf <[email protected]>--- configure | 1 + 1 file changed, 1 insertion(+)configure +0
-0configure add_to warn_flags -Wnested-externs · line 1186
Quoted 4 lines
configure +1
-0@@ -1183,6 +1183,7 @@ add_to warn_flags -Wnested-externs add_to warn_flags -Wendif-labels add_to warn_flags -Wexpansion-to-defined add_to warn_flags -Wimplicit-fallthrough=2 +add_to warn_flags -Wthread-safetyDoes this thread safety analysis have any kind of measurable impact on compilation speed ?
Quoted 2 lines
Our CI jobs are quite sensitive to any increase in build time.
From a quick run in my machine (Dell PowerEdge R440 with Intel(R) Xeon(R) Gold 5120 CPU @ 2.20GHz, 28 cpus):
without clang:
real 2m46.729s
user 19m24.122s
sys 2m58.643swith clang:
real 2m45.204s user 19m52.096s sys 2m9.036sSo there should be no significative impact.
Quoted 8 lines
Quoted 4 lines
I also forgot to mention that this serie fixes the CI failure seen in:
https://gitlab.com/qemu-project/qemu/-/jobs/3479763741
https://gitlab.com/qemu-project/qemu/-/jobs/3479763746Odd, that job already has -Wthread-safety included in CFLAGS, which would seem to make this patch redundant, but I don't see where -Wthread-safety came from in that pipeline
This patch is what was already in the branch in which the above CI tests ran. I dropped it from the pull request because of these build failures on FreeBSD. (Their libc includes TSA annotations, which means that QEMU functions have to be annotated as well when they call pthread locking functions, before we can enable -Wthread-safety.)
So patches 1 and 2 fix the failure that you would otherwise see with this patch.
Kevin
Quoted 1 line · Emanuele Giuseppe Esposito
This serie aims to enable clang Thread Safety Analysis (TSA) in QEMU.
It's worth covering what TSA is and why it's useful:
Thread Safety Analysis "warns about potential race conditions in code. The analysis is completely static (i.e. compile-time); there is no run-time overhead"
"Thread safety analysis works very much like a type system for multi-threaded programs. In addition to declaring the type of data (e.g. int, float, etc.), the programmer can (optionally) declare how access to that data is controlled in a multi-threaded environment. For example, if foo is guarded by the mutex mu, then the analysis will issue a warning whenever a piece of code reads or writes to foo without first locking mu."
Am 17.01.2023 um 14:52 hat Emanuele Giuseppe Esposito geschrieben:
Quoted 5 lines
This serie aims to enable clang Thread Safety Analysis (TSA) in QEMU. The goal is to use it for our multiqueue project aiming to replace the block layer AioContext lock with a rwlock and make sure the lock is taken correctly everywhere [1].
Quoted 7 lines
By default, TSA covers pthread mutexes, therefore when added in QEMU it immediately detects some wrappers using pthread_mutex_lock/unlock without using the proper TSA macros. Since adding such macro requires scanning all possible callers of the affected wrapper, simply use
TSA_NO_TSAto suppress the warnings.[1] = https://lists.gnu.org/archive/html/qemu-devel/2022-12/msg00903.html
Thanks, changed the commit messages as discussed and applied to my block branch.
Kevin