Skip to contentSkip to conversation

qemu/atomic.h: rename atomic_ to qatomic_

PATCH Applied Stefan Hajnoczi 5 participants 7 messages Sep 23, 2020 – Oct 3, 2020

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 qemu/atomic.h: rename atomic_ to qatomic_

clang's C11 atomic_fetch_*() functions only take a C11 atomic type pointer argument. QEMU uses direct types (int, etc) and this causes a compiler error when a QEMU code calls these functions in a source file that also included <stdatomic.h> via a system header file:

  $ CC=clang CXX=clang++ ./configure ... && make
  ../util/async.c:79:17: error: address argument to atomic operation must be a pointer to _Atomic type ('unsigned int *' invalid)

Avoid using atomic_*() names in QEMU's atomic.h since that namespace is used by <stdatomic.h>. Prefix QEMU's APIs with 'q' so that atomic.h and <stdatomic.h> can co-exist. I checked /usr/include on my machine and searched GitHub for existing "qatomic_" users but there seem to be none.

This patch was generated using:

  $ git grep -h -o '\<atomic\(64\)\?_[a-z0-9_]\+' include/qemu/atomic.h | \
    sort -u >/tmp/changed_identifiers
  $ for identifier in $(</tmp/changed_identifiers); do
        sed -i "s%\<$identifier\>%q$identifier%g" \
            $(git grep -I -l "\<$identifier\>")
    done

I manually fixed line-wrap issues and misaligned rST tables.

---
v3:
* Use qatomic_ instead of atomic_ [Paolo]
* The diff of my manual fixups is available here:
https://vmsplice.net/~stefan/atomic-namespace-pre-fixups-v3.diff
- Dropping #ifndef qatomic_fetch_add in atomic.h
- atomic_##X(haddr, val) glue macros not caught by grep
- Keep atomic_add-bench name
- C preprocessor backslash-newline ('\') column alignment
- Line wrapping
v2:
* The diff of my manual fixups is available here:
https://vmsplice.net/~stefan/atomic-namespace-pre-fixups.diff
- Dropping #ifndef qemu_atomic_fetch_add in atomic.h
- atomic_##X(haddr, val) glue macros not caught by grep
- Keep atomic_add-bench name
- C preprocessor backslash-newline ('\') column alignment
- Line wrapping
* Use grep -I to avoid accidentally modifying binary files (RISC-V OpenSBI ELFs) [Eric Blake]
* Tweak .gitorder to show atomic.h changes first [Eric Blake]
* Update grep commands in commit description so reviewers can reproduce mechanical changes [Eric Blake]

---
 include/qemu/atomic.h                         | 248 +++++++++---------
 docs/devel/lockcnt.txt                        |   8 +-
 docs/devel/rcu.txt                            |  34 +--
 accel/tcg/atomic_template.h                   |  20 +-
 include/block/aio-wait.h                      |   4 +-
 include/block/aio.h                           |   8 +-
 include/exec/cpu_ldst.h                       |   2 +-
 include/exec/exec-all.h                       |   6 +-
 include/exec/log.h                            |   6 +-
 include/exec/memory.h                         |   2 +-
 include/exec/ram_addr.h                       |  26 +-
 include/exec/ramlist.h                        |   2 +-
 include/exec/tb-lookup.h                      |   4 +-
 include/hw/core/cpu.h                         |   2 +-
 include/qemu/atomic128.h                      |   6 +-
 include/qemu/bitops.h                         |   2 +-
 include/qemu/coroutine.h                      |   2 +-
 include/qemu/log.h                            |   6 +-
 include/qemu/queue.h                          |   7 +-
 include/qemu/rcu.h                            |  10 +-
 include/qemu/rcu_queue.h                      | 100 +++----
 include/qemu/seqlock.h                        |   8 +-
 include/qemu/stats64.h                        |  28 +-
 include/qemu/thread.h                         |  24 +-
 .../infiniband/hw/vmw_pvrdma/pvrdma_ring.h    |  14 +-
 linux-user/qemu.h                             |   2 +-
 tcg/i386/tcg-target.h                         |   2 +-
 tcg/s390/tcg-target.h                         |   2 +-
 tcg/tci/tcg-target.h                          |   2 +-
 accel/kvm/kvm-all.c                           |  12 +-
 accel/tcg/cpu-exec.c                          |  15 +-
 accel/tcg/cputlb.c                            |  24 +-
 accel/tcg/tcg-all.c                           |   2 +-
 accel/tcg/translate-all.c                     |  55 ++--
 audio/jackaudio.c                             |  18 +-
 block.c                                       |   4 +-
 block/block-backend.c                         |  15 +-
 block/io.c                                    |  48 ++--
 block/nfs.c                                   |   2 +-
 block/sheepdog.c                              |   2 +-
 block/throttle-groups.c                       |  12 +-
 block/throttle.c                              |   4 +-
 blockdev.c                                    |   2 +-
 blockjob.c                                    |   2 +-
 contrib/libvhost-user/libvhost-user.c         |   2 +-
 cpus-common.c                                 |  26 +-
 dump/dump.c                                   |   8 +-
 exec.c                                        |  49 ++--
 hw/core/cpu.c                                 |   6 +-
 hw/display/qxl.c                              |   4 +-
 hw/hyperv/hyperv.c                            |  10 +-
 hw/hyperv/vmbus.c                             |   2 +-
 hw/i386/xen/xen-hvm.c                         |   2 +-
 hw/intc/rx_icu.c                              |  12 +-
 hw/intc/sifive_plic.c                         |   4 +-
 hw/misc/edu.c                                 |  16 +-
 hw/net/virtio-net.c                           |  10 +-
 hw/rdma/rdma_backend.c                        |  18 +-
 hw/rdma/rdma_rm.c                             |   2 +-
 hw/rdma/vmw/pvrdma_dev_ring.c                 |   4 +-
 hw/s390x/s390-pci-bus.c                       |   2 +-
 hw/s390x/virtio-ccw.c                         |   2 +-
 hw/virtio/vhost.c                             |   2 +-
 hw/virtio/virtio-mmio.c                       |   6 +-
 hw/virtio/virtio-pci.c                        |   6 +-
 hw/virtio/virtio.c                            |  16 +-
 hw/xtensa/pic_cpu.c                           |   4 +-
 iothread.c                                    |   6 +-
 linux-user/hppa/cpu_loop.c                    |  11 +-
 linux-user/signal.c                           |   8 +-
 migration/colo-failover.c                     |   4 +-
 migration/migration.c                         |   8 +-
 migration/multifd.c                           |  18 +-
 migration/postcopy-ram.c                      |  34 +--
 migration/rdma.c                              |  34 +--
 monitor/hmp.c                                 |   6 +-
 monitor/misc.c                                |   2 +-
 monitor/monitor.c                             |   6 +-
 qemu-nbd.c                                    |   2 +-
 qga/commands.c                                |  12 +-
 qom/object.c                                  |  20 +-
 scsi/qemu-pr-helper.c                         |   4 +-
 softmmu/cpu-throttle.c                        |  10 +-
 softmmu/cpus.c                                |  42 +--
 softmmu/memory.c                              |   6 +-
 softmmu/vl.c                                  |   2 +-
 target/arm/mte_helper.c                       |   6 +-
 target/hppa/op_helper.c                       |   2 +-
 target/i386/mem_helper.c                      |   2 +-
 target/i386/whpx-all.c                        |   6 +-
 target/riscv/cpu_helper.c                     |   2 +-
 target/s390x/mem_helper.c                     |   4 +-
 target/xtensa/exc_helper.c                    |   4 +-
 target/xtensa/op_helper.c                     |   2 +-
 tcg/tcg.c                                     |  58 ++--
 tcg/tci.c                                     |   2 +-
 tests/atomic64-bench.c                        |  14 +-
 tests/atomic_add-bench.c                      |  14 +-
 tests/iothread.c                              |   2 +-
 tests/qht-bench.c                             |  12 +-
 tests/rcutorture.c                            |  24 +-
 tests/test-aio-multithread.c                  |  52 ++--
 tests/test-logging.c                          |   4 +-
 tests/test-rcu-list.c                         |  38 +--
 tests/test-thread-pool.c                      |  10 +-
 util/aio-posix.c                              |  14 +-
 util/aio-wait.c                               |   2 +-
 util/aio-win32.c                              |   5 +-
 util/async.c                                  |  28 +-
 util/atomic64.c                               |  10 +-
 util/bitmap.c                                 |  14 +-
 util/cacheinfo.c                              |   2 +-
 util/fdmon-epoll.c                            |   4 +-
 util/fdmon-io_uring.c                         |  12 +-
 util/lockcnt.c                                |  52 ++--
 util/log.c                                    |  10 +-
 util/qemu-coroutine-lock.c                    |  18 +-
 util/qemu-coroutine-sleep.c                   |   4 +-
 util/qemu-coroutine.c                         |   6 +-
 util/qemu-sockets.c                           |   4 +-
 util/qemu-thread-posix.c                      |  12 +-
 util/qemu-thread-win32.c                      |  12 +-
 util/qemu-timer.c                             |  12 +-
 util/qht.c                                    |  57 ++--
 util/qsp.c                                    |  50 ++--
 util/rcu.c                                    |  36 +--
 util/stats64.c                                |  34 +--
 docs/devel/atomics.rst                        | 134 +++++-----
 scripts/kernel-doc                            |   2 +-
 tcg/aarch64/tcg-target.c.inc                  |   2 +-
 tcg/mips/tcg-target.c.inc                     |   2 +-
 tcg/ppc/tcg-target.c.inc                      |   6 +-
 tcg/sparc/tcg-target.c.inc                    |   5 +-
 133 files changed, 1041 insertions(+), 1018 deletions(-)
include/qemu/atomic.h +126 -122
index ff72db5115..c1d211a351 100644
@@ -125,49 +125,49 @@
  * no effect on the generated code but not using the atomic primitives
  * will get flagged by sanitizers as a violation.
  */
-#define atomic_read__nocheck(ptr) \
+#define qatomic_read__nocheck(ptr) \
     __atomic_load_n(ptr, __ATOMIC_RELAXED)
 
-#define atomic_read(ptr)                              \
-    ({                                                \
+#define qatomic_read(ptr)                              \
+    ({                                                 \
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
-    atomic_read__nocheck(ptr);                        \
+    qatomic_read__nocheck(ptr);                        \
     })
 
-#define atomic_set__nocheck(ptr, i) \
+#define qatomic_set__nocheck(ptr, i) \
     __atomic_store_n(ptr, i, __ATOMIC_RELAXED)
 
-#define atomic_set(ptr, i)  do {                      \
+#define qatomic_set(ptr, i)  do {                      \
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
-    atomic_set__nocheck(ptr, i);                      \
+    qatomic_set__nocheck(ptr, i);                      \
 } while(0)
 
 /* See above: most compilers currently treat consume and acquire the
- * same, but this slows down atomic_rcu_read unnecessarily.
+ * same, but this slows down qatomic_rcu_read unnecessarily.
  */
 #ifdef __SANITIZE_THREAD__
-#define atomic_rcu_read__nocheck(ptr, valptr)           \
+#define qatomic_rcu_read__nocheck(ptr, valptr)           \
     __atomic_load(ptr, valptr, __ATOMIC_CONSUME);
 #else
-#define atomic_rcu_read__nocheck(ptr, valptr)           \
-    __atomic_load(ptr, valptr, __ATOMIC_RELAXED);       \
+#define qatomic_rcu_read__nocheck(ptr, valptr)           \
+    __atomic_load(ptr, valptr, __ATOMIC_RELAXED);        \
     smp_read_barrier_depends();
 #endif
 
-#define atomic_rcu_read(ptr)                          \
-    ({                                                \
+#define qatomic_rcu_read(ptr)                          \
+    ({                                                 \
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
-    typeof_strip_qual(*ptr) _val;                     \
-    atomic_rcu_read__nocheck(ptr, &_val);             \
-    _val;                                             \
+    typeof_strip_qual(*ptr) _val;                      \
+    qatomic_rcu_read__nocheck(ptr, &_val);             \
+    _val;                                              \
     })
 
-#define atomic_rcu_set(ptr, i) do {                   \
+#define qatomic_rcu_set(ptr, i) do {                   \
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
-    __atomic_store_n(ptr, i, __ATOMIC_RELEASE);       \
+    __atomic_store_n(ptr, i, __ATOMIC_RELEASE);        \
 } while(0)
 
-#define atomic_load_acquire(ptr)                        \
+#define qatomic_load_acquire(ptr)                       \
     ({                                                  \
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE);  \
     typeof_strip_qual(*ptr) _val;                       \
@@ -175,7 +175,7 @@
     _val;                                               \
     })
 
-#define atomic_store_release(ptr, i)  do {              \
+#define qatomic_store_release(ptr, i)  do {             \
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE);  \
     __atomic_store_n(ptr, i, __ATOMIC_RELEASE);         \
 } while(0)
@@ -183,56 +183,61 @@
 
 /* All the remaining operations are fully sequentially consistent */
 
-#define atomic_xchg__nocheck(ptr, i)    ({                  \
+#define qatomic_xchg__nocheck(ptr, i)    ({                 \
     __atomic_exchange_n(ptr, (i), __ATOMIC_SEQ_CST);        \
 })
 
-#define atomic_xchg(ptr, i)    ({                           \
+#define qatomic_xchg(ptr, i)    ({                          \
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE);      \
-    atomic_xchg__nocheck(ptr, i);                           \
+    qatomic_xchg__nocheck(ptr, i);                          \
 })
 
 /* Returns the eventual value, failed or not */
-#define atomic_cmpxchg__nocheck(ptr, old, new)    ({                    \
+#define qatomic_cmpxchg__nocheck(ptr, old, new)    ({                   \
     typeof_strip_qual(*ptr) _old = (old);                               \
     (void)__atomic_compare_exchange_n(ptr, &_old, new, false,           \
                               __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST);      \
     _old;                                                               \
 })
 
-#define atomic_cmpxchg(ptr, old, new)    ({                             \
+#define qatomic_cmpxchg(ptr, old, new)    ({                            \
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE);                  \
-    atomic_cmpxchg__nocheck(ptr, old, new);                             \
+    qatomic_cmpxchg__nocheck(ptr, old, new);                            \
 })
 
 /* Provide shorter names for GCC atomic builtins, return old value */
-#define atomic_fetch_inc(ptr)  __atomic_fetch_add(ptr, 1, __ATOMIC_SEQ_CST)
-#define atomic_fetch_dec(ptr)  __atomic_fetch_sub(ptr, 1, __ATOMIC_SEQ_CST)
+#define qatomic_fetch_inc(ptr)  __atomic_fetch_add(ptr, 1, __ATOMIC_SEQ_CST)
+#define qatomic_fetch_dec(ptr)  __atomic_fetch_sub(ptr, 1, __ATOMIC_SEQ_CST)
 
-#ifndef atomic_fetch_add
-#define atomic_fetch_add(ptr, n) __atomic_fetch_add(ptr, n, __ATOMIC_SEQ_CST)
-#define atomic_fetch_sub(ptr, n) __atomic_fetch_sub(ptr, n, __ATOMIC_SEQ_CST)
-#define atomic_fetch_and(ptr, n) __atomic_fetch_and(ptr, n, __ATOMIC_SEQ_CST)
-#define atomic_fetch_or(ptr, n)  __atomic_fetch_or(ptr, n, __ATOMIC_SEQ_CST)
-#define atomic_fetch_xor(ptr, n) __atomic_fetch_xor(ptr, n, __ATOMIC_SEQ_CST)
-#endif
+#define qatomic_fetch_add(ptr, n) __atomic_fetch_add(ptr, n, __ATOMIC_SEQ_CST)
+#define qatomic_fetch_sub(ptr, n) __atomic_fetch_sub(ptr, n, __ATOMIC_SEQ_CST)
+#define qatomic_fetch_and(ptr, n) __atomic_fetch_and(ptr, n, __ATOMIC_SEQ_CST)
+#define qatomic_fetch_or(ptr, n)  __atomic_fetch_or(ptr, n, __ATOMIC_SEQ_CST)
+#define qatomic_fetch_xor(ptr, n) __atomic_fetch_xor(ptr, n, __ATOMIC_SEQ_CST)
 
-#define atomic_inc_fetch(ptr)    __atomic_add_fetch(ptr, 1, __ATOMIC_SEQ_CST)
-#define atomic_dec_fetch(ptr)    __atomic_sub_fetch(ptr, 1, __ATOMIC_SEQ_CST)
-#define atomic_add_fetch(ptr, n) __atomic_add_fetch(ptr, n, __ATOMIC_SEQ_CST)
-#define atomic_sub_fetch(ptr, n) __atomic_sub_fetch(ptr, n, __ATOMIC_SEQ_CST)
-#define atomic_and_fetch(ptr, n) __atomic_and_fetch(ptr, n, __ATOMIC_SEQ_CST)
-#define atomic_or_fetch(ptr, n)  __atomic_or_fetch(ptr, n, __ATOMIC_SEQ_CST)
-#define atomic_xor_fetch(ptr, n) __atomic_xor_fetch(ptr, n, __ATOMIC_SEQ_CST)
+#define qatomic_inc_fetch(ptr)    __atomic_add_fetch(ptr, 1, __ATOMIC_SEQ_CST)
+#define qatomic_dec_fetch(ptr)    __atomic_sub_fetch(ptr, 1, __ATOMIC_SEQ_CST)
+#define qatomic_add_fetch(ptr, n) __atomic_add_fetch(ptr, n, __ATOMIC_SEQ_CST)
+#define qatomic_sub_fetch(ptr, n) __atomic_sub_fetch(ptr, n, __ATOMIC_SEQ_CST)
+#define qatomic_and_fetch(ptr, n) __atomic_and_fetch(ptr, n, __ATOMIC_SEQ_CST)
+#define qatomic_or_fetch(ptr, n)  __atomic_or_fetch(ptr, n, __ATOMIC_SEQ_CST)
+#define qatomic_xor_fetch(ptr, n) __atomic_xor_fetch(ptr, n, __ATOMIC_SEQ_CST)
 
 /* And even shorter names that return void.  */
-#define atomic_inc(ptr)    ((void) __atomic_fetch_add(ptr, 1, __ATOMIC_SEQ_CST))
-#define atomic_dec(ptr)    ((void) __atomic_fetch_sub(ptr, 1, __ATOMIC_SEQ_CST))
-#define atomic_add(ptr, n) ((void) __atomic_fetch_add(ptr, n, __ATOMIC_SEQ_CST))
-#define atomic_sub(ptr, n) ((void) __atomic_fetch_sub(ptr, n, __ATOMIC_SEQ_CST))
-#define atomic_and(ptr, n) ((void) __atomic_fetch_and(ptr, n, __ATOMIC_SEQ_CST))
-#define atomic_or(ptr, n)  ((void) __atomic_fetch_or(ptr, n, __ATOMIC_SEQ_CST))
-#define atomic_xor(ptr, n) ((void) __atomic_fetch_xor(ptr, n, __ATOMIC_SEQ_CST))
+#define qatomic_inc(ptr) \
+    ((void) __atomic_fetch_add(ptr, 1, __ATOMIC_SEQ_CST))
+#define qatomic_dec(ptr) \
+    ((void) __atomic_fetch_sub(ptr, 1, __ATOMIC_SEQ_CST))
+#define qatomic_add(ptr, n) \
+    ((void) __atomic_fetch_add(ptr, n, __ATOMIC_SEQ_CST))
+#define qatomic_sub(ptr, n) \
+    ((void) __atomic_fetch_sub(ptr, n, __ATOMIC_SEQ_CST))
+#define qatomic_and(ptr, n) \
+    ((void) __atomic_fetch_and(ptr, n, __ATOMIC_SEQ_CST))
+#define qatomic_or(ptr, n) \
+    ((void) __atomic_fetch_or(ptr, n, __ATOMIC_SEQ_CST))
+#define qatomic_xor(ptr, n) \
+    ((void) __atomic_fetch_xor(ptr, n, __ATOMIC_SEQ_CST))
 
 #else /* __ATOMIC_RELAXED */
 
@@ -272,7 +277,7 @@
  * but it is a full barrier at the hardware level.  Add a compiler barrier
  * to make it a full barrier also at the compiler level.
  */
-#define atomic_xchg(ptr, i)    (barrier(), __sync_lock_test_and_set(ptr, i))
+#define qatomic_xchg(ptr, i)    (barrier(), __sync_lock_test_and_set(ptr, i))
 
 #elif defined(_ARCH_PPC)
 
@@ -325,14 +330,14 @@
 /* These will only be atomic if the processor does the fetch or store
  * in a single issue memory operation
  */
-#define atomic_read__nocheck(p)   (*(__typeof__(*(p)) volatile*) (p))
-#define atomic_set__nocheck(p, i) ((*(__typeof__(*(p)) volatile*) (p)) = (i))
+#define qatomic_read__nocheck(p)   (*(__typeof__(*(p)) volatile*) (p))
+#define qatomic_set__nocheck(p, i) ((*(__typeof__(*(p)) volatile*) (p)) = (i))
 
-#define atomic_read(ptr)       atomic_read__nocheck(ptr)
-#define atomic_set(ptr, i)     atomic_set__nocheck(ptr,i)
+#define qatomic_read(ptr)       qatomic_read__nocheck(ptr)
+#define qatomic_set(ptr, i)     qatomic_set__nocheck(ptr,i)
 
 /**
- * atomic_rcu_read - reads a RCU-protected pointer to a local variable
+ * qatomic_rcu_read - reads a RCU-protected pointer to a local variable
  * into a RCU read-side critical section. The pointer can later be safely
  * dereferenced within the critical section.
  *
@@ -342,21 +347,21 @@
  * Inserts memory barriers on architectures that require them (currently only
  * Alpha) and documents which pointers are protected by RCU.
  *
- * atomic_rcu_read also includes a compiler barrier to ensure that
+ * qatomic_rcu_read also includes a compiler barrier to ensure that
  * value-speculative optimizations (e.g. VSS: Value Speculation
  * Scheduling) does not perform the data read before the pointer read
  * by speculating the value of the pointer.
  *
- * Should match atomic_rcu_set(), atomic_xchg(), atomic_cmpxchg().
+ * Should match qatomic_rcu_set(), qatomic_xchg(), qatomic_cmpxchg().
  */
-#define atomic_rcu_read(ptr)    ({                \
-    typeof(*ptr) _val = atomic_read(ptr);         \
+#define qatomic_rcu_read(ptr)    ({               \
+    typeof(*ptr) _val = qatomic_read(ptr);        \
     smp_read_barrier_depends();                   \
     _val;                                         \
 })
 
 /**
- * atomic_rcu_set - assigns (publicizes) a pointer to a new data structure
+ * qatomic_rcu_set - assigns (publicizes) a pointer to a new data structure
  * meant to be read by RCU read-side critical sections.
  *
  * Documents which pointers will be dereferenced by RCU read-side critical
@@ -364,65 +369,64 @@
  * them. It also makes sure the compiler does not reorder code initializing the
  * data structure before its publication.
  *
- * Should match atomic_rcu_read().
+ * Should match qatomic_rcu_read().
  */
-#define atomic_rcu_set(ptr, i)  do {              \
+#define qatomic_rcu_set(ptr, i)  do {             \
     smp_wmb();                                    \
-    atomic_set(ptr, i);                           \
+    qatomic_set(ptr, i);                          \
 } while (0)
 
-#define atomic_load_acquire(ptr)    ({      \
-    typeof(*ptr) _val = atomic_read(ptr);   \
+#define qatomic_load_acquire(ptr)    ({     \
+    typeof(*ptr) _val = qatomic_read(ptr);  \
     smp_mb_acquire();                       \
     _val;                                   \
 })
 
-#define atomic_store_release(ptr, i)  do {  \
+#define qatomic_store_release(ptr, i)  do { \
     smp_mb_release();                       \
-    atomic_set(ptr, i);                     \
+    qatomic_set(ptr, i);                    \
 } while (0)
 
-#ifndef atomic_xchg
+#ifndef qatomic_xchg
 #if defined(__clang__)
-#define atomic_xchg(ptr, i)    __sync_swap(ptr, i)
+#define qatomic_xchg(ptr, i)    __sync_swap(ptr, i)
 #else
 /* __sync_lock_test_and_set() is documented to be an acquire barrier only.  */
-#define atomic_xchg(ptr, i)    (smp_mb(), __sync_lock_test_and_set(ptr, i))
+#define qatomic_xchg(ptr, i)    (smp_mb(), __sync_lock_test_and_set(ptr, i))
 #endif
 #endif
-#define atomic_xchg__nocheck  atomic_xchg
+#define qatomic_xchg__nocheck  qatomic_xchg
 
 /* Provide shorter names for GCC atomic builtins.  */
-#define atomic_fetch_inc(ptr)  __sync_fetch_and_add(ptr, 1)
-#define atomic_fetch_dec(ptr)  __sync_fetch_and_add(ptr, -1)
+#define qatomic_fetch_inc(ptr)  __sync_fetch_and_add(ptr, 1)
+#define qatomic_fetch_dec(ptr)  __sync_fetch_and_add(ptr, -1)
 
-#ifndef atomic_fetch_add
-#define atomic_fetch_add(ptr, n) __sync_fetch_and_add(ptr, n)
-#define atomic_fetch_sub(ptr, n) __sync_fetch_and_sub(ptr, n)
-#define atomic_fetch_and(ptr, n) __sync_fetch_and_and(ptr, n)
-#define atomic_fetch_or(ptr, n) __sync_fetch_and_or(ptr, n)
-#define atomic_fetch_xor(ptr, n) __sync_fetch_and_xor(ptr, n)
-#endif
+#define qatomic_fetch_add(ptr, n) __sync_fetch_and_add(ptr, n)
+#define qatomic_fetch_sub(ptr, n) __sync_fetch_and_sub(ptr, n)
+#define qatomic_fetch_and(ptr, n) __sync_fetch_and_and(ptr, n)
+#define qatomic_fetch_or(ptr, n) __sync_fetch_and_or(ptr, n)
+#define qatomic_fetch_xor(ptr, n) __sync_fetch_and_xor(ptr, n)
 
-#define atomic_inc_fetch(ptr)  __sync_add_and_fetch(ptr, 1)
-#define atomic_dec_fetch(ptr)  __sync_add_and_fetch(ptr, -1)
-#define atomic_add_fetch(ptr, n) __sync_add_and_fetch(ptr, n)
-#define atomic_sub_fetch(ptr, n) __sync_sub_and_fetch(ptr, n)
-#define atomic_and_fetch(ptr, n) __sync_and_and_fetch(ptr, n)
-#define atomic_or_fetch(ptr, n) __sync_or_and_fetch(ptr, n)
-#define atomic_xor_fetch(ptr, n) __sync_xor_and_fetch(ptr, n)
+#define qatomic_inc_fetch(ptr)  __sync_add_and_fetch(ptr, 1)
+#define qatomic_dec_fetch(ptr)  __sync_add_and_fetch(ptr, -1)
+#define qatomic_add_fetch(ptr, n) __sync_add_and_fetch(ptr, n)
+#define qatomic_sub_fetch(ptr, n) __sync_sub_and_fetch(ptr, n)
+#define qatomic_and_fetch(ptr, n) __sync_and_and_fetch(ptr, n)
+#define qatomic_or_fetch(ptr, n) __sync_or_and_fetch(ptr, n)
+#define qatomic_xor_fetch(ptr, n) __sync_xor_and_fetch(ptr, n)
 
-#define atomic_cmpxchg(ptr, old, new) __sync_val_compare_and_swap(ptr, old, new)
-#define atomic_cmpxchg__nocheck(ptr, old, new)  atomic_cmpxchg(ptr, old, new)
+#define qatomic_cmpxchg(ptr, old, new) \
+    __sync_val_compare_and_swap(ptr, old, new)
+#define qatomic_cmpxchg__nocheck(ptr, old, new)  qatomic_cmpxchg(ptr, old, new)
 
 /* And even shorter names that return void.  */
-#define atomic_inc(ptr)        ((void) __sync_fetch_and_add(ptr, 1))
-#define atomic_dec(ptr)        ((void) __sync_fetch_and_add(ptr, -1))
-#define atomic_add(ptr, n)     ((void) __sync_fetch_and_add(ptr, n))
-#define atomic_sub(ptr, n)     ((void) __sync_fetch_and_sub(ptr, n))
-#define atomic_and(ptr, n)     ((void) __sync_fetch_and_and(ptr, n))
-#define atomic_or(ptr, n)      ((void) __sync_fetch_and_or(ptr, n))
-#define atomic_xor(ptr, n)     ((void) __sync_fetch_and_xor(ptr, n))
+#define qatomic_inc(ptr)        ((void) __sync_fetch_and_add(ptr, 1))
+#define qatomic_dec(ptr)        ((void) __sync_fetch_and_add(ptr, -1))
+#define qatomic_add(ptr, n)     ((void) __sync_fetch_and_add(ptr, n))
+#define qatomic_sub(ptr, n)     ((void) __sync_fetch_and_sub(ptr, n))
+#define qatomic_and(ptr, n)     ((void) __sync_fetch_and_and(ptr, n))
+#define qatomic_or(ptr, n)      ((void) __sync_fetch_and_or(ptr, n))
+#define qatomic_xor(ptr, n)     ((void) __sync_fetch_and_xor(ptr, n))
 
 #endif /* __ATOMIC_RELAXED */
 
@@ -436,11 +440,11 @@
 /* This is more efficient than a store plus a fence.  */
 #if !defined(__SANITIZE_THREAD__)
 #if defined(__i386__) || defined(__x86_64__) || defined(__s390x__)
-#define atomic_mb_set(ptr, i)  ((void)atomic_xchg(ptr, i))
+#define qatomic_mb_set(ptr, i)  ((void)qatomic_xchg(ptr, i))
 #endif
 #endif
 
-/* atomic_mb_read/set semantics map Java volatile variables. They are
+/* qatomic_mb_read/set semantics map Java volatile variables. They are
  * less expensive on some platforms (notably POWER) than fully
  * sequentially consistent operations.
  *
@@ -448,58 +452,58 @@
  * use. See docs/devel/atomics.txt for more discussion.
  */
 
-#ifndef atomic_mb_read
-#define atomic_mb_read(ptr)                             \
-    atomic_load_acquire(ptr)
+#ifndef qatomic_mb_read
+#define qatomic_mb_read(ptr)                             \
+    qatomic_load_acquire(ptr)
 #endif
 
-#ifndef atomic_mb_set
-#define atomic_mb_set(ptr, i)  do {                     \
-    atomic_store_release(ptr, i);                       \
+#ifndef qatomic_mb_set
+#define qatomic_mb_set(ptr, i)  do {                    \
+    qatomic_store_release(ptr, i);                      \
     smp_mb();                                           \
 } while(0)
 #endif
 
-#define atomic_fetch_inc_nonzero(ptr) ({                                \
-    typeof_strip_qual(*ptr) _oldn = atomic_read(ptr);                   \
-    while (_oldn && atomic_cmpxchg(ptr, _oldn, _oldn + 1) != _oldn) {   \
-        _oldn = atomic_read(ptr);                                       \
+#define qatomic_fetch_inc_nonzero(ptr) ({                               \
+    typeof_strip_qual(*ptr) _oldn = qatomic_read(ptr);                  \
+    while (_oldn && qatomic_cmpxchg(ptr, _oldn, _oldn + 1) != _oldn) {  \
+        _oldn = qatomic_read(ptr);                                      \
     }                                                                   \
     _oldn;                                                              \
 })
 
 /* Abstractions to access atomically (i.e. "once") i64/u64 variables */
 #ifdef CONFIG_ATOMIC64
-static inline int64_t atomic_read_i64(const int64_t *ptr)
+static inline int64_t qatomic_read_i64(const int64_t *ptr)
 {
     /* use __nocheck because sizeof(void *) might be < sizeof(u64) */
-    return atomic_read__nocheck(ptr);
+    return qatomic_read__nocheck(ptr);
 }
 
-static inline uint64_t atomic_read_u64(const uint64_t *ptr)
+static inline uint64_t qatomic_read_u64(const uint64_t *ptr)
 {
-    return atomic_read__nocheck(ptr);
+    return qatomic_read__nocheck(ptr);
 }
 
-static inline void atomic_set_i64(int64_t *ptr, int64_t val)
+static inline void qatomic_set_i64(int64_t *ptr, int64_t val)
 {
-    atomic_set__nocheck(ptr, val);
+    qatomic_set__nocheck(ptr, val);
 }
 
-static inline void atomic_set_u64(uint64_t *ptr, uint64_t val)
+static inline void qatomic_set_u64(uint64_t *ptr, uint64_t val)
 {
-    atomic_set__nocheck(ptr, val);
+    qatomic_set__nocheck(ptr, val);
 }
 
-static inline void atomic64_init(void)
+static inline void qatomic64_init(void)
 {
 }
 #else /* !CONFIG_ATOMIC64 */
-int64_t  atomic_read_i64(const int64_t *ptr);
-uint64_t atomic_read_u64(const uint64_t *ptr);
-void atomic_set_i64(int64_t *ptr, int64_t val);
-void atomic_set_u64(uint64_t *ptr, uint64_t val);
-void atomic64_init(void);
+int64_t  qatomic_read_i64(const int64_t *ptr);
+uint64_t qatomic_read_u64(const uint64_t *ptr);
+void qatomic_set_i64(int64_t *ptr, int64_t val);
+void qatomic_set_u64(uint64_t *ptr, uint64_t val);
+void qatomic64_init(void);
 #endif /* !CONFIG_ATOMIC64 */
 
 #endif /* QEMU_ATOMIC_H */
docs/devel/lockcnt.txt +4 -4
index 7c099bc6c8..2d85462fe3 100644
@@ -95,7 +95,7 @@ not just frees, though there could be cases where this is not necessary.
 
 Reads, instead, can be done without taking the mutex, as long as the
 readers and writers use the same macros that are used for RCU, for
-example atomic_rcu_read, atomic_rcu_set, QLIST_FOREACH_RCU, etc.  This is
+example qatomic_rcu_read, qatomic_rcu_set, QLIST_FOREACH_RCU, etc.  This is
 because the reads are done outside a lock and a set or QLIST_INSERT_HEAD
 can happen concurrently with the read.  The RCU API ensures that the
 processor and the compiler see all required memory barriers.
@@ -189,7 +189,7 @@ qemu_lockcnt_lock and qemu_lockcnt_unlock:
     if (!xyz) {
         new_xyz = g_new(XYZ, 1);
         ...
-        atomic_rcu_set(&xyz, new_xyz);
+        qatomic_rcu_set(&xyz, new_xyz);
     }
     qemu_lockcnt_unlock(&xyz_lockcnt);
 
@@ -198,7 +198,7 @@ qemu_lockcnt_dec:
 
     qemu_lockcnt_inc(&xyz_lockcnt);
     if (xyz) {
-        XYZ *p = atomic_rcu_read(&xyz);
+        XYZ *p = qatomic_rcu_read(&xyz);
         ...
         /* Accesses can now be done through "p".  */
     }
@@ -222,7 +222,7 @@ the decrement, the locking and the check on count as follows:
 
     qemu_lockcnt_inc(&xyz_lockcnt);
     if (xyz) {
-        XYZ *p = atomic_rcu_read(&xyz);
+        XYZ *p = qatomic_rcu_read(&xyz);
         ...
         /* Accesses can now be done through "p".  */
     }
docs/devel/rcu.txt +17 -17
index 0ce15ba198..cdf002edd8 100644
@@ -130,13 +130,13 @@ The core RCU API is small:
 
             g_free_rcu(&foo, rcu);
 
-     typeof(*p) atomic_rcu_read(p);
+     typeof(*p) qatomic_rcu_read(p);
 
-        atomic_rcu_read() is similar to atomic_load_acquire(), but it makes
+        qatomic_rcu_read() is similar to qatomic_load_acquire(), but it makes
         some assumptions on the code that calls it.  This allows a more
         optimized implementation.
 
-        atomic_rcu_read assumes that whenever a single RCU critical
+        qatomic_rcu_read assumes that whenever a single RCU critical
         section reads multiple shared data, these reads are either
         data-dependent or need no ordering.  This is almost always the
         case when using RCU, because read-side critical sections typically
@@ -144,7 +144,7 @@ The core RCU API is small:
         every update) until reaching a data structure of interest,
         and then read from there.
 
-        RCU read-side critical sections must use atomic_rcu_read() to
+        RCU read-side critical sections must use qatomic_rcu_read() to
         read data, unless concurrent writes are prevented by another
         synchronization mechanism.
 
@@ -152,18 +152,18 @@ The core RCU API is small:
         data structure in a single direction, opposite to the direction
         in which the updater initializes it.
 
-     void atomic_rcu_set(p, typeof(*p) v);
+     void qatomic_rcu_set(p, typeof(*p) v);
 
-        atomic_rcu_set() is similar to atomic_store_release(), though it also
+        qatomic_rcu_set() is similar to qatomic_store_release(), though it also
         makes assumptions on the code that calls it in order to allow a more
         optimized implementation.
 
-        In particular, atomic_rcu_set() suffices for synchronization
+        In particular, qatomic_rcu_set() suffices for synchronization
         with readers, if the updater never mutates a field within a
         data item that is already accessible to readers.  This is the
         case when initializing a new copy of the RCU-protected data
         structure; just ensure that initialization of *p is carried out
-        before atomic_rcu_set() makes the data item visible to readers.
+        before qatomic_rcu_set() makes the data item visible to readers.
         If this rule is observed, writes will happen in the opposite
         order as reads in the RCU read-side critical sections (or if
         there is just one update), and there will be no need for other
@@ -212,7 +212,7 @@ DIFFERENCES WITH LINUX
   programming; not allowing this would prevent upgrading an RCU read-side
   critical section to become an updater.
 
-- atomic_rcu_read and atomic_rcu_set replace rcu_dereference and
+- qatomic_rcu_read and qatomic_rcu_set replace rcu_dereference and
   rcu_assign_pointer.  They take a _pointer_ to the variable being accessed.
 
 - call_rcu is a macro that has an extra argument (the name of the first
@@ -257,7 +257,7 @@ may be used as a restricted reference-counting mechanism.  For example,
 consider the following code fragment:
 
     rcu_read_lock();
-    p = atomic_rcu_read(&foo);
+    p = qatomic_rcu_read(&foo);
     /* do something with p. */
     rcu_read_unlock();
 
@@ -268,7 +268,7 @@ The write side looks simply like this (with appropriate locking):
 
     qemu_mutex_lock(&foo_mutex);
     old = foo;
-    atomic_rcu_set(&foo, new);
+    qatomic_rcu_set(&foo, new);
     qemu_mutex_unlock(&foo_mutex);
     synchronize_rcu();
     free(old);
@@ -277,7 +277,7 @@ If the processing cannot be done purely within the critical section, it
 is possible to combine this idiom with a "real" reference count:
 
     rcu_read_lock();
-    p = atomic_rcu_read(&foo);
+    p = qatomic_rcu_read(&foo);
     foo_ref(p);
     rcu_read_unlock();
     /* do something with p. */
@@ -287,7 +287,7 @@ The write side can be like this:
 
     qemu_mutex_lock(&foo_mutex);
     old = foo;
-    atomic_rcu_set(&foo, new);
+    qatomic_rcu_set(&foo, new);
     qemu_mutex_unlock(&foo_mutex);
     synchronize_rcu();
     foo_unref(old);
@@ -296,7 +296,7 @@ or with call_rcu:
 
     qemu_mutex_lock(&foo_mutex);
     old = foo;
-    atomic_rcu_set(&foo, new);
+    qatomic_rcu_set(&foo, new);
     qemu_mutex_unlock(&foo_mutex);
     call_rcu(foo_unref, old, rcu);
 
@@ -307,7 +307,7 @@ last reference may be dropped on the read side.  Hence you can
 use call_rcu() instead:
 
      foo_unref(struct foo *p) {
-        if (atomic_fetch_dec(&p->refcount) == 1) {
+        if (qatomic_fetch_dec(&p->refcount) == 1) {
             call_rcu(foo_destroy, p, rcu);
         }
     }
@@ -375,7 +375,7 @@ Instead, we store the size of the array with the array itself:
 
     read side:
         rcu_read_lock();
-        struct arr *array = atomic_rcu_read(&global_array);
+        struct arr *array = qatomic_rcu_read(&global_array);
         x = i < array->size ? array->data[i] : -1;
         rcu_read_unlock();
         return x;
@@ -392,7 +392,7 @@ Instead, we store the size of the array with the array itself:
 
             /* Removal phase.  */
             old_array = global_array;
-            atomic_rcu_set(&new_array->data, new_array);
+            qatomic_rcu_set(&new_array->data, new_array);
             synchronize_rcu();
 
             /* Reclamation phase.  */
accel/tcg/atomic_template.h +10 -10
index 26969487d6..0ff7f913e1 100644
@@ -83,7 +83,7 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr,
 #if DATA_SIZE == 16
     ret = atomic16_cmpxchg(haddr, cmpv, newv);
 #else
-    ret = atomic_cmpxchg__nocheck(haddr, cmpv, newv);
+    ret = qatomic_cmpxchg__nocheck(haddr, cmpv, newv);
 #endif
     ATOMIC_MMU_CLEANUP;
     atomic_trace_rmw_post(env, addr, info);
@@ -131,7 +131,7 @@ ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr,
                                          ATOMIC_MMU_IDX);
 
     atomic_trace_rmw_pre(env, addr, info);
-    ret = atomic_xchg__nocheck(haddr, val);
+    ret = qatomic_xchg__nocheck(haddr, val);
     ATOMIC_MMU_CLEANUP;
     atomic_trace_rmw_post(env, addr, info);
     return ret;
@@ -147,7 +147,7 @@ ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr,       \
     uint16_t info = trace_mem_build_info(SHIFT, false, 0, false,    \
                                          ATOMIC_MMU_IDX);           \
     atomic_trace_rmw_pre(env, addr, info);                          \
-    ret = atomic_##X(haddr, val);                                   \
+    ret = qatomic_##X(haddr, val);                                  \
     ATOMIC_MMU_CLEANUP;                                             \
     atomic_trace_rmw_post(env, addr, info);                         \
     return ret;                                                     \
@@ -182,10 +182,10 @@ ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr,       \
                                          ATOMIC_MMU_IDX);           \
     atomic_trace_rmw_pre(env, addr, info);                          \
     smp_mb();                                                       \
-    cmp = atomic_read__nocheck(haddr);                              \
+    cmp = qatomic_read__nocheck(haddr);                             \
     do {                                                            \
         old = cmp; new = FN(old, val);                              \
-        cmp = atomic_cmpxchg__nocheck(haddr, old, new);             \
+        cmp = qatomic_cmpxchg__nocheck(haddr, old, new);            \
     } while (cmp != old);                                           \
     ATOMIC_MMU_CLEANUP;                                             \
     atomic_trace_rmw_post(env, addr, info);                         \
@@ -230,7 +230,7 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr,
 #if DATA_SIZE == 16
     ret = atomic16_cmpxchg(haddr, BSWAP(cmpv), BSWAP(newv));
 #else
-    ret = atomic_cmpxchg__nocheck(haddr, BSWAP(cmpv), BSWAP(newv));
+    ret = qatomic_cmpxchg__nocheck(haddr, BSWAP(cmpv), BSWAP(newv));
 #endif
     ATOMIC_MMU_CLEANUP;
     atomic_trace_rmw_post(env, addr, info);
@@ -280,7 +280,7 @@ ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr,
                                          ATOMIC_MMU_IDX);
 
     atomic_trace_rmw_pre(env, addr, info);
-    ret = atomic_xchg__nocheck(haddr, BSWAP(val));
+    ret = qatomic_xchg__nocheck(haddr, BSWAP(val));
     ATOMIC_MMU_CLEANUP;
     atomic_trace_rmw_post(env, addr, info);
     return BSWAP(ret);
@@ -296,7 +296,7 @@ ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr,       \
     uint16_t info = trace_mem_build_info(SHIFT, false, MO_BSWAP,    \
                                          false, ATOMIC_MMU_IDX);    \
     atomic_trace_rmw_pre(env, addr, info);                          \
-    ret = atomic_##X(haddr, BSWAP(val));                            \
+    ret = qatomic_##X(haddr, BSWAP(val));                           \
     ATOMIC_MMU_CLEANUP;                                             \
     atomic_trace_rmw_post(env, addr, info);                         \
     return BSWAP(ret);                                              \
@@ -329,10 +329,10 @@ ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr,       \
                                          false, ATOMIC_MMU_IDX);    \
     atomic_trace_rmw_pre(env, addr, info);                          \
     smp_mb();                                                       \
-    ldn = atomic_read__nocheck(haddr);                              \
+    ldn = qatomic_read__nocheck(haddr);                             \
     do {                                                            \
         ldo = ldn; old = BSWAP(ldo); new = FN(old, val);            \
-        ldn = atomic_cmpxchg__nocheck(haddr, ldo, BSWAP(new));      \
+        ldn = qatomic_cmpxchg__nocheck(haddr, ldo, BSWAP(new));     \
     } while (ldo != ldn);                                           \
     ATOMIC_MMU_CLEANUP;                                             \
     atomic_trace_rmw_post(env, addr, info);                         \
include/block/aio-wait.h +2 -2
index 716d2639df..b39eefb38d 100644
@@ -80,7 +80,7 @@ extern AioWait global_aio_wait;
     AioWait *wait_ = &global_aio_wait;                             \
     AioContext *ctx_ = (ctx);                                      \
     /* Increment wait_->num_waiters before evaluating cond. */     \
-    atomic_inc(&wait_->num_waiters);                               \
+    qatomic_inc(&wait_->num_waiters);                              \
     if (ctx_ && in_aio_context_home_thread(ctx_)) {                \
         while ((cond)) {                                           \
             aio_poll(ctx_, true);                                  \
@@ -100,7 +100,7 @@ extern AioWait global_aio_wait;
             waited_ = true;                                        \
         }                                                          \
     }                                                              \
-    atomic_dec(&wait_->num_waiters);                               \
+    qatomic_dec(&wait_->num_waiters);                              \
     waited_; })
 
 /**
include/block/aio.h +4 -4
index b2f703fa3f..ec8c5af642 100644
@@ -595,7 +595,7 @@ int64_t aio_compute_timeout(AioContext *ctx);
  */
 static inline void aio_disable_external(AioContext *ctx)
 {
-    atomic_inc(&ctx->external_disable_cnt);
+    qatomic_inc(&ctx->external_disable_cnt);
 }
 
 /**
@@ -608,7 +608,7 @@ static inline void aio_enable_external(AioContext *ctx)
 {
     int old;
 
-    old = atomic_fetch_dec(&ctx->external_disable_cnt);
+    old = qatomic_fetch_dec(&ctx->external_disable_cnt);
     assert(old > 0);
     if (old == 1) {
         /* Kick event loop so it re-arms file descriptors */
@@ -624,7 +624,7 @@ static inline void aio_enable_external(AioContext *ctx)
  */
 static inline bool aio_external_disabled(AioContext *ctx)
 {
-    return atomic_read(&ctx->external_disable_cnt);
+    return qatomic_read(&ctx->external_disable_cnt);
 }
 
 /**
@@ -637,7 +637,7 @@ static inline bool aio_external_disabled(AioContext *ctx)
  */
 static inline bool aio_node_check(AioContext *ctx, bool is_external)
 {
-    return !is_external || !atomic_read(&ctx->external_disable_cnt);
+    return !is_external || !qatomic_read(&ctx->external_disable_cnt);
 }
 
 /**
include/exec/cpu_ldst.h +1 -1
index c14a48f65e..30605edab9 100644
@@ -299,7 +299,7 @@ static inline target_ulong tlb_addr_write(const CPUTLBEntry *entry)
 #if TCG_OVERSIZED_GUEST
     return entry->addr_write;
 #else
-    return atomic_read(&entry->addr_write);
+    return qatomic_read(&entry->addr_write);
 #endif
 }
 
include/exec/exec-all.h +3 -3
index 3cf88272df..1fe28d574f 100644
@@ -89,7 +89,7 @@ void QEMU_NORETURN cpu_loop_exit_atomic(CPUState *cpu, uintptr_t pc);
  */
 static inline bool cpu_loop_exit_requested(CPUState *cpu)
 {
-    return (int32_t)atomic_read(&cpu_neg(cpu)->icount_decr.u32) < 0;
+    return (int32_t)qatomic_read(&cpu_neg(cpu)->icount_decr.u32) < 0;
 }
 
 #if !defined(CONFIG_USER_ONLY)
@@ -487,10 +487,10 @@ struct TranslationBlock {
 
 extern bool parallel_cpus;
 
-/* Hide the atomic_read to make code a little easier on the eyes */
+/* Hide the qatomic_read to make code a little easier on the eyes */
 static inline uint32_t tb_cflags(const TranslationBlock *tb)
 {
-    return atomic_read(&tb->cflags);
+    return qatomic_read(&tb->cflags);
 }
 
 /* current cflags for hashing/comparison */
include/exec/log.h +3 -3
index 3ed797c1c8..86871f403a 100644
@@ -19,7 +19,7 @@ static inline void log_cpu_state(CPUState *cpu, int flags)
 
     if (qemu_log_enabled()) {
         rcu_read_lock();
-        logfile = atomic_rcu_read(&qemu_logfile);
+        logfile = qatomic_rcu_read(&qemu_logfile);
         if (logfile) {
             cpu_dump_state(cpu, logfile->fd, flags);
         }
@@ -49,7 +49,7 @@ static inline void log_target_disas(CPUState *cpu, target_ulong start,
 {
     QemuLogFile *logfile;
     rcu_read_lock();
-    logfile = atomic_rcu_read(&qemu_logfile);
+    logfile = qatomic_rcu_read(&qemu_logfile);
     if (logfile) {
         target_disas(logfile->fd, cpu, start, len);
     }
@@ -60,7 +60,7 @@ static inline void log_disas(void *code, unsigned long size, const char *note)
 {
     QemuLogFile *logfile;
     rcu_read_lock();
-    logfile = atomic_rcu_read(&qemu_logfile);
+    logfile = qatomic_rcu_read(&qemu_logfile);
     if (logfile) {
         disas(logfile->fd, code, size, note);
     }
include/exec/memory.h +1 -1
index f1bb2a7df5..06b85e3a73 100644
@@ -685,7 +685,7 @@ struct FlatView {
 
 static inline FlatView *address_space_to_flatview(AddressSpace *as)
 {
-    return atomic_rcu_read(&as->current_map);
+    return qatomic_rcu_read(&as->current_map);
 }
 
 
include/exec/ram_addr.h +14 -12
index 3ef729a23c..c6d2ef1d07 100644
@@ -164,7 +164,7 @@ static inline bool cpu_physical_memory_get_dirty(ram_addr_t start,
     page = start >> TARGET_PAGE_BITS;
 
     WITH_RCU_READ_LOCK_GUARD() {
-        blocks = atomic_rcu_read(&ram_list.dirty_memory[client]);
+        blocks = qatomic_rcu_read(&ram_list.dirty_memory[client]);
 
         idx = page / DIRTY_MEMORY_BLOCK_SIZE;
         offset = page % DIRTY_MEMORY_BLOCK_SIZE;
@@ -205,7 +205,7 @@ static inline bool cpu_physical_memory_all_dirty(ram_addr_t start,
 
     RCU_READ_LOCK_GUARD();
 
-    blocks = atomic_rcu_read(&ram_list.dirty_memory[client]);
+    blocks = qatomic_rcu_read(&ram_list.dirty_memory[client]);
 
     idx = page / DIRTY_MEMORY_BLOCK_SIZE;
     offset = page % DIRTY_MEMORY_BLOCK_SIZE;
@@ -278,7 +278,7 @@ static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
 
     RCU_READ_LOCK_GUARD();
 
-    blocks = atomic_rcu_read(&ram_list.dirty_memory[client]);
+    blocks = qatomic_rcu_read(&ram_list.dirty_memory[client]);
 
     set_bit_atomic(offset, blocks->blocks[idx]);
 }
@@ -301,7 +301,7 @@ static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
 
     WITH_RCU_READ_LOCK_GUARD() {
         for (i = 0; i < DIRTY_MEMORY_NUM; i++) {
-            blocks[i] = atomic_rcu_read(&ram_list.dirty_memory[i]);
+            blocks[i] = qatomic_rcu_read(&ram_list.dirty_memory[i]);
         }
 
         idx = page / DIRTY_MEMORY_BLOCK_SIZE;
@@ -361,23 +361,25 @@ static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned long *bitmap,
 
         WITH_RCU_READ_LOCK_GUARD() {
             for (i = 0; i < DIRTY_MEMORY_NUM; i++) {
-                blocks[i] = atomic_rcu_read(&ram_list.dirty_memory[i])->blocks;
+                blocks[i] =
+                    qatomic_rcu_read(&ram_list.dirty_memory[i])->blocks;
             }
 
             for (k = 0; k < nr; k++) {
                 if (bitmap[k]) {
                     unsigned long temp = leul_to_cpu(bitmap[k]);
 
-                    atomic_or(&blocks[DIRTY_MEMORY_VGA][idx][offset], temp);
+                    qatomic_or(&blocks[DIRTY_MEMORY_VGA][idx][offset], temp);
 
                     if (global_dirty_log) {
-                        atomic_or(&blocks[DIRTY_MEMORY_MIGRATION][idx][offset],
-                                  temp);
+                        qatomic_or(
+                                &blocks[DIRTY_MEMORY_MIGRATION][idx][offset],
+                                temp);
                     }
 
                     if (tcg_enabled()) {
-                        atomic_or(&blocks[DIRTY_MEMORY_CODE][idx][offset],
-                                  temp);
+                        qatomic_or(&blocks[DIRTY_MEMORY_CODE][idx][offset],
+                                   temp);
                     }
                 }
 
@@ -461,12 +463,12 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
                                         DIRTY_MEMORY_BLOCK_SIZE);
         unsigned long page = BIT_WORD(start >> TARGET_PAGE_BITS);
 
-        src = atomic_rcu_read(
+        src = qatomic_rcu_read(
                 &ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION])->blocks;
 
         for (k = page; k < page + nr; k++) {
             if (src[idx][offset]) {
-                unsigned long bits = atomic_xchg(&src[idx][offset], 0);
+                unsigned long bits = qatomic_xchg(&src[idx][offset], 0);
                 unsigned long new_dirty;
                 new_dirty = ~dest[k];
                 dest[k] |= bits;
include/exec/ramlist.h +1 -1
index bc4faa1b00..26704aa3b0 100644
@@ -19,7 +19,7 @@ typedef struct RAMBlockNotifier RAMBlockNotifier;
  *   rcu_read_lock();
  *
  *   DirtyMemoryBlocks *blocks =
- *       atomic_rcu_read(&ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION]);
+ *       qatomic_rcu_read(&ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION]);
  *
  *   ram_addr_t idx = (addr >> TARGET_PAGE_BITS) / DIRTY_MEMORY_BLOCK_SIZE;
  *   unsigned long *block = blocks.blocks[idx];
include/exec/tb-lookup.h +2 -2
index 26921b6daf..9cf475bb03 100644
@@ -27,7 +27,7 @@ tb_lookup__cpu_state(CPUState *cpu, target_ulong *pc, target_ulong *cs_base,
 
     cpu_get_tb_cpu_state(env, pc, cs_base, flags);
     hash = tb_jmp_cache_hash_func(*pc);
-    tb = atomic_rcu_read(&cpu->tb_jmp_cache[hash]);
+    tb = qatomic_rcu_read(&cpu->tb_jmp_cache[hash]);
 
     cf_mask &= ~CF_CLUSTER_MASK;
     cf_mask |= cpu->cluster_index << CF_CLUSTER_SHIFT;
@@ -44,7 +44,7 @@ tb_lookup__cpu_state(CPUState *cpu, target_ulong *pc, target_ulong *cs_base,
     if (tb == NULL) {
         return NULL;
     }
-    atomic_set(&cpu->tb_jmp_cache[hash], tb);
+    qatomic_set(&cpu->tb_jmp_cache[hash], tb);
     return tb;
 }
 
include/hw/core/cpu.h +1 -1
index 99dc33ffeb..6c34798c8b 100644
@@ -482,7 +482,7 @@ static inline void cpu_tb_jmp_cache_clear(CPUState *cpu)
     unsigned int i;
 
     for (i = 0; i < TB_JMP_CACHE_SIZE; i++) {
-        atomic_set(&cpu->tb_jmp_cache[i], NULL);
+        qatomic_set(&cpu->tb_jmp_cache[i], NULL);
     }
 }
 
include/qemu/atomic128.h +3 -3
index 6b34484e15..ad2bcf45b4 100644
@@ -44,7 +44,7 @@
 #if defined(CONFIG_ATOMIC128)
 static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new)
 {
-    return atomic_cmpxchg__nocheck(ptr, cmp, new);
+    return qatomic_cmpxchg__nocheck(ptr, cmp, new);
 }
 # define HAVE_CMPXCHG128 1
 #elif defined(CONFIG_CMPXCHG128)
@@ -89,12 +89,12 @@ Int128 QEMU_ERROR("unsupported atomic")
 #if defined(CONFIG_ATOMIC128)
 static inline Int128 atomic16_read(Int128 *ptr)
 {
-    return atomic_read__nocheck(ptr);
+    return qatomic_read__nocheck(ptr);
 }
 
 static inline void atomic16_set(Int128 *ptr, Int128 val)
 {
-    atomic_set__nocheck(ptr, val);
+    qatomic_set__nocheck(ptr, val);
 }
 
 # define HAVE_ATOMIC128 1
include/qemu/bitops.h +1 -1
index f55ce8b320..3acbf3384c 100644
@@ -51,7 +51,7 @@ static inline void set_bit_atomic(long nr, unsigned long *addr)
     unsigned long mask = BIT_MASK(nr);
     unsigned long *p = addr + BIT_WORD(nr);
 
-    atomic_or(p, mask);
+    qatomic_or(p, mask);
 }
 
 /**
include/qemu/coroutine.h +1 -1
index dfd261c5b1..84eab6e3bf 100644
@@ -179,7 +179,7 @@ static inline coroutine_fn void qemu_co_mutex_assert_locked(CoMutex *mutex)
      * because the condition will be false no matter whether we read NULL or
      * the pointer for any other coroutine.
      */
-    assert(atomic_read(&mutex->locked) &&
+    assert(qatomic_read(&mutex->locked) &&
            mutex->holder == qemu_coroutine_self());
 }
 
include/qemu/log.h +3 -3
index f4724f7330..9b80660207 100644
@@ -36,7 +36,7 @@ static inline bool qemu_log_separate(void)
     bool res = false;
 
     rcu_read_lock();
-    logfile = atomic_rcu_read(&qemu_logfile);
+    logfile = qatomic_rcu_read(&qemu_logfile);
     if (logfile && logfile->fd != stderr) {
         res = true;
     }
@@ -75,7 +75,7 @@ static inline FILE *qemu_log_lock(void)
 {
     QemuLogFile *logfile;
     rcu_read_lock();
-    logfile = atomic_rcu_read(&qemu_logfile);
+    logfile = qatomic_rcu_read(&qemu_logfile);
     if (logfile) {
         qemu_flockfile(logfile->fd);
         return logfile->fd;
@@ -102,7 +102,7 @@ qemu_log_vprintf(const char *fmt, va_list va)
     QemuLogFile *logfile;
 
     rcu_read_lock();
-    logfile = atomic_rcu_read(&qemu_logfile);
+    logfile = qatomic_rcu_read(&qemu_logfile);
     if (logfile) {
         vfprintf(logfile->fd, fmt, va);
     }
include/qemu/queue.h +4 -3
index 456a5b01ee..e029e7bf66 100644
@@ -218,12 +218,12 @@ struct {                                                                \
         typeof(elm) save_sle_next;                                           \
         do {                                                                 \
             save_sle_next = (elm)->field.sle_next = (head)->slh_first;       \
-        } while (atomic_cmpxchg(&(head)->slh_first, save_sle_next, (elm)) != \
+        } while (qatomic_cmpxchg(&(head)->slh_first, save_sle_next, (elm)) !=\
                  save_sle_next);                                             \
 } while (/*CONSTCOND*/0)
 
 #define QSLIST_MOVE_ATOMIC(dest, src) do {                               \
-        (dest)->slh_first = atomic_xchg(&(src)->slh_first, NULL);        \
+        (dest)->slh_first = qatomic_xchg(&(src)->slh_first, NULL);       \
 } while (/*CONSTCOND*/0)
 
 #define QSLIST_REMOVE_HEAD(head, field) do {                             \
@@ -376,7 +376,8 @@ struct {                                                                \
 /*
  * Simple queue access methods.
  */
-#define QSIMPLEQ_EMPTY_ATOMIC(head) (atomic_read(&((head)->sqh_first)) == NULL)
+#define QSIMPLEQ_EMPTY_ATOMIC(head) \
+    (qatomic_read(&((head)->sqh_first)) == NULL)
 #define QSIMPLEQ_EMPTY(head)        ((head)->sqh_first == NULL)
 #define QSIMPLEQ_FIRST(head)        ((head)->sqh_first)
 #define QSIMPLEQ_NEXT(elm, field)   ((elm)->field.sqe_next)
include/qemu/rcu.h +5 -5
index 0e375ebe13..515d327cf1 100644
@@ -79,8 +79,8 @@ static inline void rcu_read_lock(void)
         return;
     }
 
-    ctr = atomic_read(&rcu_gp_ctr);
-    atomic_set(&p_rcu_reader->ctr, ctr);
+    ctr = qatomic_read(&rcu_gp_ctr);
+    qatomic_set(&p_rcu_reader->ctr, ctr);
 
     /* Write p_rcu_reader->ctr before reading RCU-protected pointers.  */
     smp_mb_placeholder();
@@ -100,12 +100,12 @@ static inline void rcu_read_unlock(void)
      * smp_mb_placeholder(), this ensures writes to p_rcu_reader->ctr
      * are sequentially consistent.
      */
-    atomic_store_release(&p_rcu_reader->ctr, 0);
+    qatomic_store_release(&p_rcu_reader->ctr, 0);
 
     /* Write p_rcu_reader->ctr before reading p_rcu_reader->waiting.  */
     smp_mb_placeholder();
-    if (unlikely(atomic_read(&p_rcu_reader->waiting))) {
-        atomic_set(&p_rcu_reader->waiting, false);
+    if (unlikely(qatomic_read(&p_rcu_reader->waiting))) {
+        qatomic_set(&p_rcu_reader->waiting, false);
         qemu_event_set(&rcu_gp_event);
     }
 }
include/qemu/rcu_queue.h +51 -49

257 lines not on this page · raw message

include/qemu/seqlock.h +4 -4
index 8b6b4ee4bb..ecb7d2c864 100644
@@ -32,7 +32,7 @@ static inline void seqlock_init(QemuSeqLock *sl)
 /* Lock out other writers and update the count.  */
 static inline void seqlock_write_begin(QemuSeqLock *sl)
 {
-    atomic_set(&sl->sequence, sl->sequence + 1);
+    qatomic_set(&sl->sequence, sl->sequence + 1);
 
     /* Write sequence before updating other fields.  */
     smp_wmb();
@@ -43,7 +43,7 @@ static inline void seqlock_write_end(QemuSeqLock *sl)
     /* Write other fields before finalizing sequence.  */
     smp_wmb();
 
-    atomic_set(&sl->sequence, sl->sequence + 1);
+    qatomic_set(&sl->sequence, sl->sequence + 1);
 }
 
 /* Lock out other writers and update the count.  */
@@ -68,7 +68,7 @@ static inline void seqlock_write_unlock_impl(QemuSeqLock *sl, QemuLockable *lock
 static inline unsigned seqlock_read_begin(const QemuSeqLock *sl)
 {
     /* Always fail if a write is in progress.  */
-    unsigned ret = atomic_read(&sl->sequence);
+    unsigned ret = qatomic_read(&sl->sequence);
 
     /* Read sequence before reading other fields.  */
     smp_rmb();
@@ -79,7 +79,7 @@ static inline int seqlock_read_retry(const QemuSeqLock *sl, unsigned start)
 {
     /* Read other fields before reading final sequence.  */
     smp_rmb();
-    return unlikely(atomic_read(&sl->sequence) != start);
+    return unlikely(qatomic_read(&sl->sequence) != start);
 }
 
 #endif
include/qemu/stats64.h +14 -14
index 19a5ac4c56..fdd3d1b8f9 100644
@@ -37,27 +37,27 @@ static inline void stat64_init(Stat64 *s, uint64_t value)
 
 static inline uint64_t stat64_get(const Stat64 *s)
 {
-    return atomic_read__nocheck(&s->value);
+    return qatomic_read__nocheck(&s->value);
 }
 
 static inline void stat64_add(Stat64 *s, uint64_t value)
 {
-    atomic_add(&s->value, value);
+    qatomic_add(&s->value, value);
 }
 
 static inline void stat64_min(Stat64 *s, uint64_t value)
 {
-    uint64_t orig = atomic_read__nocheck(&s->value);
+    uint64_t orig = qatomic_read__nocheck(&s->value);
     while (orig > value) {
-        orig = atomic_cmpxchg__nocheck(&s->value, orig, value);
+        orig = qatomic_cmpxchg__nocheck(&s->value, orig, value);
     }
 }
 
 static inline void stat64_max(Stat64 *s, uint64_t value)
 {
-    uint64_t orig = atomic_read__nocheck(&s->value);
+    uint64_t orig = qatomic_read__nocheck(&s->value);
     while (orig < value) {
-        orig = atomic_cmpxchg__nocheck(&s->value, orig, value);
+        orig = qatomic_cmpxchg__nocheck(&s->value, orig, value);
     }
 }
 #else
@@ -79,7 +79,7 @@ static inline void stat64_add(Stat64 *s, uint64_t value)
     low = (uint32_t) value;
     if (!low) {
         if (high) {
-            atomic_add(&s->high, high);
+            qatomic_add(&s->high, high);
         }
         return;
     }
@@ -101,7 +101,7 @@ static inline void stat64_add(Stat64 *s, uint64_t value)
          * the high 32 bits, so it can race just fine with stat64_add32_carry
          * and even stat64_get!
          */
-        old = atomic_cmpxchg(&s->low, orig, result);
+        old = qatomic_cmpxchg(&s->low, orig, result);
         if (orig == old) {
             return;
         }
@@ -116,7 +116,7 @@ static inline void stat64_min(Stat64 *s, uint64_t value)
     high = value >> 32;
     low = (uint32_t) value;
     do {
-        orig_high = atomic_read(&s->high);
+        orig_high = qatomic_read(&s->high);
         if (orig_high < high) {
             return;
         }
@@ -128,7 +128,7 @@ static inline void stat64_min(Stat64 *s, uint64_t value)
              * the write barrier in stat64_min_slow.
              */
             smp_rmb();
-            orig_low = atomic_read(&s->low);
+            orig_low = qatomic_read(&s->low);
             if (orig_low <= low) {
                 return;
             }
@@ -138,7 +138,7 @@ static inline void stat64_min(Stat64 *s, uint64_t value)
              * we may miss being lucky.
              */
             smp_rmb();
-            orig_high = atomic_read(&s->high);
+            orig_high = qatomic_read(&s->high);
             if (orig_high < high) {
                 return;
             }
@@ -156,7 +156,7 @@ static inline void stat64_max(Stat64 *s, uint64_t value)
     high = value >> 32;
     low = (uint32_t) value;
     do {
-        orig_high = atomic_read(&s->high);
+        orig_high = qatomic_read(&s->high);
         if (orig_high > high) {
             return;
         }
@@ -168,7 +168,7 @@ static inline void stat64_max(Stat64 *s, uint64_t value)
              * the write barrier in stat64_max_slow.
              */
             smp_rmb();
-            orig_low = atomic_read(&s->low);
+            orig_low = qatomic_read(&s->low);
             if (orig_low >= low) {
                 return;
             }
@@ -178,7 +178,7 @@ static inline void stat64_max(Stat64 *s, uint64_t value)
              * we may miss being lucky.
              */
             smp_rmb();
-            orig_high = atomic_read(&s->high);
+            orig_high = qatomic_read(&s->high);
             if (orig_high > high) {
                 return;
             }
include/qemu/thread.h +12 -12
index 4baf4d1715..5435763184 100644
@@ -70,33 +70,33 @@ extern QemuCondTimedWaitFunc qemu_cond_timedwait_func;
             qemu_cond_timedwait_impl(c, m, ms, __FILE__, __LINE__)
 #else
 #define qemu_mutex_lock(m) ({                                           \
-            QemuMutexLockFunc _f = atomic_read(&qemu_mutex_lock_func);  \
+            QemuMutexLockFunc _f = qatomic_read(&qemu_mutex_lock_func); \
             _f(m, __FILE__, __LINE__);                                  \
         })
 
-#define qemu_mutex_trylock(m) ({                                        \
-            QemuMutexTrylockFunc _f = atomic_read(&qemu_mutex_trylock_func); \
-            _f(m, __FILE__, __LINE__);                                  \
+#define qemu_mutex_trylock(m) ({                                              \
+            QemuMutexTrylockFunc _f = qatomic_read(&qemu_mutex_trylock_func); \
+            _f(m, __FILE__, __LINE__);                                        \
         })
 
-#define qemu_rec_mutex_lock(m) ({                                       \
-            QemuRecMutexLockFunc _f = atomic_read(&qemu_rec_mutex_lock_func); \
-            _f(m, __FILE__, __LINE__);                                  \
+#define qemu_rec_mutex_lock(m) ({                                             \
+            QemuRecMutexLockFunc _f = qatomic_read(&qemu_rec_mutex_lock_func);\
+            _f(m, __FILE__, __LINE__);                                        \
         })
 
 #define qemu_rec_mutex_trylock(m) ({                            \
             QemuRecMutexTrylockFunc _f;                         \
-            _f = atomic_read(&qemu_rec_mutex_trylock_func);     \
+            _f = qatomic_read(&qemu_rec_mutex_trylock_func);    \
             _f(m, __FILE__, __LINE__);                          \
         })
 
 #define qemu_cond_wait(c, m) ({                                         \
-            QemuCondWaitFunc _f = atomic_read(&qemu_cond_wait_func);    \
+            QemuCondWaitFunc _f = qatomic_read(&qemu_cond_wait_func);   \
             _f(c, m, __FILE__, __LINE__);                               \
         })
 
 #define qemu_cond_timedwait(c, m, ms) ({                                       \
-            QemuCondTimedWaitFunc _f = atomic_read(&qemu_cond_timedwait_func); \
+            QemuCondTimedWaitFunc _f = qatomic_read(&qemu_cond_timedwait_func);\
             _f(c, m, ms, __FILE__, __LINE__);                                  \
         })
 #endif
@@ -236,7 +236,7 @@ static inline void qemu_spin_lock(QemuSpin *spin)
     __tsan_mutex_pre_lock(spin, 0);
 #endif
     while (unlikely(__sync_lock_test_and_set(&spin->value, true))) {
-        while (atomic_read(&spin->value)) {
+        while (qatomic_read(&spin->value)) {
             cpu_relax();
         }
     }
@@ -261,7 +261,7 @@ static inline bool qemu_spin_trylock(QemuSpin *spin)
 
 static inline bool qemu_spin_locked(QemuSpin *spin)
 {
-    return atomic_read(&spin->value);
+    return qatomic_read(&spin->value);
 }
 
 static inline void qemu_spin_unlock(QemuSpin *spin)
include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h +7 -7
index acd4c8346d..7b4062a1a1 100644
@@ -68,7 +68,7 @@ static inline int pvrdma_idx_valid(uint32_t idx, uint32_t max_elems)
 
 static inline int32_t pvrdma_idx(int *var, uint32_t max_elems)
 {
-	const unsigned int idx = atomic_read(var);
+	const unsigned int idx = qatomic_read(var);
 
 	if (pvrdma_idx_valid(idx, max_elems))
 		return idx & (max_elems - 1);
@@ -77,17 +77,17 @@ static inline int32_t pvrdma_idx(int *var, uint32_t max_elems)
 
 static inline void pvrdma_idx_ring_inc(int *var, uint32_t max_elems)
 {
-	uint32_t idx = atomic_read(var) + 1;	/* Increment. */
+	uint32_t idx = qatomic_read(var) + 1;	/* Increment. */
 
 	idx &= (max_elems << 1) - 1;		/* Modulo size, flip gen. */
-	atomic_set(var, idx);
+	qatomic_set(var, idx);
 }
 
 static inline int32_t pvrdma_idx_ring_has_space(const struct pvrdma_ring *r,
 					      uint32_t max_elems, uint32_t *out_tail)
 {
-	const uint32_t tail = atomic_read(&r->prod_tail);
-	const uint32_t head = atomic_read(&r->cons_head);
+	const uint32_t tail = qatomic_read(&r->prod_tail);
+	const uint32_t head = qatomic_read(&r->cons_head);
 
 	if (pvrdma_idx_valid(tail, max_elems) &&
 	    pvrdma_idx_valid(head, max_elems)) {
@@ -100,8 +100,8 @@ static inline int32_t pvrdma_idx_ring_has_space(const struct pvrdma_ring *r,
 static inline int32_t pvrdma_idx_ring_has_data(const struct pvrdma_ring *r,
 					     uint32_t max_elems, uint32_t *out_head)
 {
-	const uint32_t tail = atomic_read(&r->prod_tail);
-	const uint32_t head = atomic_read(&r->cons_head);
+	const uint32_t tail = qatomic_read(&r->prod_tail);
+	const uint32_t head = qatomic_read(&r->cons_head);
 
 	if (pvrdma_idx_valid(tail, max_elems) &&
 	    pvrdma_idx_valid(head, max_elems)) {
linux-user/qemu.h +1 -1
index a69a0bd347..941ca99722 100644
@@ -146,7 +146,7 @@ typedef struct TaskState {
     /* Nonzero if process_pending_signals() needs to do something (either
      * handle a pending signal or unblock signals).
      * This flag is written from a signal handler so should be accessed via
-     * the atomic_read() and atomic_set() functions. (It is not accessed
+     * the qatomic_read() and qatomic_set() functions. (It is not accessed
      * from multiple threads.)
      */
     int signal_pending;
tcg/i386/tcg-target.h +1 -1
index 99ac1e3958..d2baf796b0 100644
@@ -215,7 +215,7 @@ static inline void tb_target_set_jmp_target(uintptr_t tc_ptr,
                                             uintptr_t jmp_addr, uintptr_t addr)
 {
     /* patch the branch destination */
-    atomic_set((int32_t *)jmp_addr, addr - (jmp_addr + 4));
+    qatomic_set((int32_t *)jmp_addr, addr - (jmp_addr + 4));
     /* no need to flush icache explicitly */
 }
 
tcg/s390/tcg-target.h +1 -1
index 07accabbd1..63c8797bd3 100644
@@ -154,7 +154,7 @@ static inline void tb_target_set_jmp_target(uintptr_t tc_ptr,
 {
     /* patch the branch destination */
     intptr_t disp = addr - (jmp_addr - 2);
-    atomic_set((int32_t *)jmp_addr, disp / 2);
+    qatomic_set((int32_t *)jmp_addr, disp / 2);
     /* no need to flush icache explicitly */
 }
 
tcg/tci/tcg-target.h +1 -1
index 8b90ab71cb..8c1c1d265d 100644
@@ -206,7 +206,7 @@ static inline void tb_target_set_jmp_target(uintptr_t tc_ptr,
                                             uintptr_t jmp_addr, uintptr_t addr)
 {
     /* patch the branch destination */
-    atomic_set((int32_t *)jmp_addr, addr - (jmp_addr + 4));
+    qatomic_set((int32_t *)jmp_addr, addr - (jmp_addr + 4));
     /* no need to flush icache explicitly */
 }
 
accel/kvm/kvm-all.c +6 -6
index ad8b315b35..e4bbf78366 100644
@@ -2379,7 +2379,7 @@ static __thread bool have_sigbus_pending;
 
 static void kvm_cpu_kick(CPUState *cpu)
 {
-    atomic_set(&cpu->kvm_run->immediate_exit, 1);
+    qatomic_set(&cpu->kvm_run->immediate_exit, 1);
 }
 
 static void kvm_cpu_kick_self(void)
@@ -2400,7 +2400,7 @@ static void kvm_eat_signals(CPUState *cpu)
     int r;
 
     if (kvm_immediate_exit) {
-        atomic_set(&cpu->kvm_run->immediate_exit, 0);
+        qatomic_set(&cpu->kvm_run->immediate_exit, 0);
         /* Write kvm_run->immediate_exit before the cpu->exit_request
          * write in kvm_cpu_exec.
          */
@@ -2434,7 +2434,7 @@ int kvm_cpu_exec(CPUState *cpu)
     DPRINTF("kvm_cpu_exec()\n");
 
     if (kvm_arch_process_async_events(cpu)) {
-        atomic_set(&cpu->exit_request, 0);
+        qatomic_set(&cpu->exit_request, 0);
         return EXCP_HLT;
     }
 
@@ -2450,7 +2450,7 @@ int kvm_cpu_exec(CPUState *cpu)
         }
 
         kvm_arch_pre_run(cpu, run);
-        if (atomic_read(&cpu->exit_request)) {
+        if (qatomic_read(&cpu->exit_request)) {
             DPRINTF("interrupt exit requested\n");
             /*
              * KVM requires us to reenter the kernel after IO exits to complete
@@ -2577,7 +2577,7 @@ int kvm_cpu_exec(CPUState *cpu)
         vm_stop(RUN_STATE_INTERNAL_ERROR);
     }
 
-    atomic_set(&cpu->exit_request, 0);
+    qatomic_set(&cpu->exit_request, 0);
     return ret;
 }
 
@@ -2994,7 +2994,7 @@ int kvm_on_sigbus_vcpu(CPUState *cpu, int code, void *addr)
     have_sigbus_pending = true;
     pending_sigbus_addr = addr;
     pending_sigbus_code = code;
-    atomic_set(&cpu->exit_request, 1);
+    qatomic_set(&cpu->exit_request, 1);
     return 0;
 #else
     return 1;
accel/tcg/cpu-exec.c +8 -7
index 66d38f9d85..e10b46283c 100644
@@ -367,7 +367,8 @@ static inline void tb_add_jump(TranslationBlock *tb, int n,
         goto out_unlock_next;
     }
     /* Atomically claim the jump destination slot only if it was NULL */
-    old = atomic_cmpxchg(&tb->jmp_dest[n], (uintptr_t)NULL, (uintptr_t)tb_next);
+    old = qatomic_cmpxchg(&tb->jmp_dest[n], (uintptr_t)NULL,
+                          (uintptr_t)tb_next);
     if (old) {
         goto out_unlock_next;
     }
@@ -407,7 +408,7 @@ static inline TranslationBlock *tb_find(CPUState *cpu,
         tb = tb_gen_code(cpu, pc, cs_base, flags, cf_mask);
         mmap_unlock();
         /* We add the TB in the virtual pc hash table for the fast lookup */
-        atomic_set(&cpu->tb_jmp_cache[tb_jmp_cache_hash_func(pc)], tb);
+        qatomic_set(&cpu->tb_jmp_cache[tb_jmp_cache_hash_func(pc)], tb);
     }
 #ifndef CONFIG_USER_ONLY
     /* We don't take care of direct jumps when address mapping changes in
@@ -536,9 +537,9 @@ static inline bool cpu_handle_interrupt(CPUState *cpu,
      * Ensure zeroing happens before reading cpu->exit_request or
      * cpu->interrupt_request (see also smp_wmb in cpu_exit())
      */
-    atomic_mb_set(&cpu_neg(cpu)->icount_decr.u16.high, 0);
+    qatomic_mb_set(&cpu_neg(cpu)->icount_decr.u16.high, 0);
 
-    if (unlikely(atomic_read(&cpu->interrupt_request))) {
+    if (unlikely(qatomic_read(&cpu->interrupt_request))) {
         int interrupt_request;
         qemu_mutex_lock_iothread();
         interrupt_request = cpu->interrupt_request;
@@ -613,10 +614,10 @@ static inline bool cpu_handle_interrupt(CPUState *cpu,
     }
 
     /* Finally, check if we need to exit to the main loop.  */
-    if (unlikely(atomic_read(&cpu->exit_request))
+    if (unlikely(qatomic_read(&cpu->exit_request))
         || (use_icount
             && cpu_neg(cpu)->icount_decr.u16.low + cpu->icount_extra == 0)) {
-        atomic_set(&cpu->exit_request, 0);
+        qatomic_set(&cpu->exit_request, 0);
         if (cpu->exception_index == -1) {
             cpu->exception_index = EXCP_INTERRUPT;
         }
@@ -642,7 +643,7 @@ static inline void cpu_loop_exec_tb(CPUState *cpu, TranslationBlock *tb,
     }
 
     *last_tb = NULL;
-    insns_left = atomic_read(&cpu_neg(cpu)->icount_decr.u32);
+    insns_left = qatomic_read(&cpu_neg(cpu)->icount_decr.u32);
     if (insns_left < 0) {
         /* Something asked us to stop executing chained TBs; just
          * continue round the main loop. Whatever requested the exit
accel/tcg/cputlb.c +12 -12
index 6489abbf8c..aaf8e46ae5 100644
@@ -312,9 +312,9 @@ void tlb_flush_counts(size_t *pfull, size_t *ppart, size_t *pelide)
     CPU_FOREACH(cpu) {
         CPUArchState *env = cpu->env_ptr;
 
-        full += atomic_read(&env_tlb(env)->c.full_flush_count);
-        part += atomic_read(&env_tlb(env)->c.part_flush_count);
-        elide += atomic_read(&env_tlb(env)->c.elide_flush_count);
+        full += qatomic_read(&env_tlb(env)->c.full_flush_count);
+        part += qatomic_read(&env_tlb(env)->c.part_flush_count);
+        elide += qatomic_read(&env_tlb(env)->c.elide_flush_count);
     }
     *pfull = full;
     *ppart = part;
@@ -349,13 +349,13 @@ static void tlb_flush_by_mmuidx_async_work(CPUState *cpu, run_on_cpu_data data)
     cpu_tb_jmp_cache_clear(cpu);
 
     if (to_clean == ALL_MMUIDX_BITS) {
-        atomic_set(&env_tlb(env)->c.full_flush_count,
+        qatomic_set(&env_tlb(env)->c.full_flush_count,
                    env_tlb(env)->c.full_flush_count + 1);
     } else {
-        atomic_set(&env_tlb(env)->c.part_flush_count,
+        qatomic_set(&env_tlb(env)->c.part_flush_count,
                    env_tlb(env)->c.part_flush_count + ctpop16(to_clean));
         if (to_clean != asked) {
-            atomic_set(&env_tlb(env)->c.elide_flush_count,
+            qatomic_set(&env_tlb(env)->c.elide_flush_count,
                        env_tlb(env)->c.elide_flush_count +
                        ctpop16(asked & ~to_clean));
         }
@@ -693,7 +693,7 @@ void tlb_unprotect_code(ram_addr_t ram_addr)
  * generated code.
  *
  * Other vCPUs might be reading their TLBs during guest execution, so we update
- * te->addr_write with atomic_set. We don't need to worry about this for
+ * te->addr_write with qatomic_set. We don't need to worry about this for
  * oversized guests as MTTCG is disabled for them.
  *
  * Called with tlb_c.lock held.
@@ -711,7 +711,7 @@ static void tlb_reset_dirty_range_locked(CPUTLBEntry *tlb_entry,
 #if TCG_OVERSIZED_GUEST
             tlb_entry->addr_write |= TLB_NOTDIRTY;
 #else
-            atomic_set(&tlb_entry->addr_write,
+            qatomic_set(&tlb_entry->addr_write,
                        tlb_entry->addr_write | TLB_NOTDIRTY);
 #endif
         }
@@ -1138,8 +1138,8 @@ static inline target_ulong tlb_read_ofs(CPUTLBEntry *entry, size_t ofs)
 #if TCG_OVERSIZED_GUEST
     return *(target_ulong *)((uintptr_t)entry + ofs);
 #else
-    /* ofs might correspond to .addr_write, so use atomic_read */
-    return atomic_read((target_ulong *)((uintptr_t)entry + ofs));
+    /* ofs might correspond to .addr_write, so use qatomic_read */
+    return qatomic_read((target_ulong *)((uintptr_t)entry + ofs));
 #endif
 }
 
@@ -1155,11 +1155,11 @@ static bool victim_tlb_hit(CPUArchState *env, size_t mmu_idx, size_t index,
         CPUTLBEntry *vtlb = &env_tlb(env)->d[mmu_idx].vtable[vidx];
         target_ulong cmp;
 
-        /* elt_ofs might correspond to .addr_write, so use atomic_read */
+        /* elt_ofs might correspond to .addr_write, so use qatomic_read */
 #if TCG_OVERSIZED_GUEST
         cmp = *(target_ulong *)((uintptr_t)vtlb + elt_ofs);
 #else
-        cmp = atomic_read((target_ulong *)((uintptr_t)vtlb + elt_ofs));
+        cmp = qatomic_read((target_ulong *)((uintptr_t)vtlb + elt_ofs));
 #endif
 
         if (cmp == page) {
accel/tcg/tcg-all.c +1 -1
index 7098ad96c3..1c664924d7 100644
@@ -65,7 +65,7 @@ static void tcg_handle_interrupt(CPUState *cpu, int mask)
     if (!qemu_cpu_is_self(cpu)) {
         qemu_cpu_kick(cpu);
     } else {
-        atomic_set(&cpu_neg(cpu)->icount_decr.u16.high, -1);
+        qatomic_set(&cpu_neg(cpu)->icount_decr.u16.high, -1);
         if (use_icount &&
             !cpu->can_do_io
             && (mask & ~old_mask) != 0) {
accel/tcg/translate-all.c +28 -27

182 lines not on this page · raw message

audio/jackaudio.c +9 -9
index 72ed7c4929..1e714b30bc 100644
@@ -104,7 +104,7 @@ static void qjack_buffer_create(QJackBuffer *buffer, int channels, int frames)
 static void qjack_buffer_clear(QJackBuffer *buffer)
 {
     assert(buffer->data);
-    atomic_store_release(&buffer->used, 0);
+    qatomic_store_release(&buffer->used, 0);
     buffer->rptr = 0;
     buffer->wptr = 0;
 }
@@ -129,7 +129,7 @@ static int qjack_buffer_write(QJackBuffer *buffer, float *data, int size)
     assert(buffer->data);
     const int samples = size / sizeof(float);
     int frames        = samples / buffer->channels;
-    const int avail   = buffer->frames - atomic_load_acquire(&buffer->used);
+    const int avail   = buffer->frames - qatomic_load_acquire(&buffer->used);
 
     if (frames > avail) {
         frames = avail;
@@ -153,7 +153,7 @@ static int qjack_buffer_write(QJackBuffer *buffer, float *data, int size)
 
     buffer->wptr = wptr;
 
-    atomic_add(&buffer->used, frames);
+    qatomic_add(&buffer->used, frames);
     return frames * buffer->channels * sizeof(float);
 };
 
@@ -161,7 +161,7 @@ static int qjack_buffer_write(QJackBuffer *buffer, float *data, int size)
 static int qjack_buffer_write_l(QJackBuffer *buffer, float **dest, int frames)
 {
     assert(buffer->data);
-    const int avail   = buffer->frames - atomic_load_acquire(&buffer->used);
+    const int avail   = buffer->frames - qatomic_load_acquire(&buffer->used);
     int wptr = buffer->wptr;
 
     if (frames > avail) {
@@ -185,7 +185,7 @@ static int qjack_buffer_write_l(QJackBuffer *buffer, float **dest, int frames)
     }
     buffer->wptr = wptr;
 
-    atomic_add(&buffer->used, frames);
+    qatomic_add(&buffer->used, frames);
     return frames;
 }
 
@@ -195,7 +195,7 @@ static int qjack_buffer_read(QJackBuffer *buffer, float *dest, int size)
     assert(buffer->data);
     const int samples = size / sizeof(float);
     int frames        = samples / buffer->channels;
-    const int avail   = atomic_load_acquire(&buffer->used);
+    const int avail   = qatomic_load_acquire(&buffer->used);
 
     if (frames > avail) {
         frames = avail;
@@ -219,7 +219,7 @@ static int qjack_buffer_read(QJackBuffer *buffer, float *dest, int size)
 
     buffer->rptr = rptr;
 
-    atomic_sub(&buffer->used, frames);
+    qatomic_sub(&buffer->used, frames);
     return frames * buffer->channels * sizeof(float);
 }
 
@@ -228,7 +228,7 @@ static int qjack_buffer_read_l(QJackBuffer *buffer, float **dest, int frames)
 {
     assert(buffer->data);
     int copy       = frames;
-    const int used = atomic_load_acquire(&buffer->used);
+    const int used = qatomic_load_acquire(&buffer->used);
     int rptr       = buffer->rptr;
 
     if (copy > used) {
@@ -252,7 +252,7 @@ static int qjack_buffer_read_l(QJackBuffer *buffer, float **dest, int frames)
     }
     buffer->rptr = rptr;
 
-    atomic_sub(&buffer->used, copy);
+    qatomic_sub(&buffer->used, copy);
     return copy;
 }
 
block.c +2 -2
index 11ab55f80b..f72a2e26e8 100644
@@ -1694,7 +1694,7 @@ static int bdrv_open_common(BlockDriverState *bs, BlockBackend *file,
     }
 
     /* bdrv_new() and bdrv_close() make it so */
-    assert(atomic_read(&bs->copy_on_read) == 0);
+    assert(qatomic_read(&bs->copy_on_read) == 0);
 
     if (bs->open_flags & BDRV_O_COPY_ON_READ) {
         if (!bs->read_only) {
@@ -4436,7 +4436,7 @@ static void bdrv_close(BlockDriverState *bs)
     bs->file = NULL;
     g_free(bs->opaque);
     bs->opaque = NULL;
-    atomic_set(&bs->copy_on_read, 0);
+    qatomic_set(&bs->copy_on_read, 0);
     bs->backing_file[0] = '\0';
     bs->backing_format[0] = '\0';
     bs->total_sectors = 0;
block/block-backend.c +8 -7
index 24dd0670d1..ce78d30794 100644
@@ -1353,12 +1353,12 @@ int blk_make_zero(BlockBackend *blk, BdrvRequestFlags flags)
 
 void blk_inc_in_flight(BlockBackend *blk)
 {
-    atomic_inc(&blk->in_flight);
+    qatomic_inc(&blk->in_flight);
 }
 
 void blk_dec_in_flight(BlockBackend *blk)
 {
-    atomic_dec(&blk->in_flight);
+    qatomic_dec(&blk->in_flight);
     aio_wait_kick();
 }
 
@@ -1720,7 +1720,7 @@ void blk_drain(BlockBackend *blk)
 
     /* We may have -ENOMEDIUM completions in flight */
     AIO_WAIT_WHILE(blk_get_aio_context(blk),
-                   atomic_mb_read(&blk->in_flight) > 0);
+                   qatomic_mb_read(&blk->in_flight) > 0);
 
     if (bs) {
         bdrv_drained_end(bs);
@@ -1739,7 +1739,7 @@ void blk_drain_all(void)
         aio_context_acquire(ctx);
 
         /* We may have -ENOMEDIUM completions in flight */
-        AIO_WAIT_WHILE(ctx, atomic_mb_read(&blk->in_flight) > 0);
+        AIO_WAIT_WHILE(ctx, qatomic_mb_read(&blk->in_flight) > 0);
 
         aio_context_release(ctx);
     }
@@ -2346,6 +2346,7 @@ void blk_io_limits_update_group(BlockBackend *blk, const char *group)
 static void blk_root_drained_begin(BdrvChild *child)
 {
     BlockBackend *blk = child->opaque;
+    ThrottleGroupMember *tgm = &blk->public.throttle_group_member;
 
     if (++blk->quiesce_counter == 1) {
         if (blk->dev_ops && blk->dev_ops->drained_begin) {
@@ -2356,8 +2357,8 @@ static void blk_root_drained_begin(BdrvChild *child)
     /* Note that blk->root may not be accessible here yet if we are just
      * attaching to a BlockDriverState that is drained. Use child instead. */
 
-    if (atomic_fetch_inc(&blk->public.throttle_group_member.io_limits_disabled) == 0) {
-        throttle_group_restart_tgm(&blk->public.throttle_group_member);
+    if (qatomic_fetch_inc(&tgm->io_limits_disabled) == 0) {
+        throttle_group_restart_tgm(tgm);
     }
 }
 
@@ -2374,7 +2375,7 @@ static void blk_root_drained_end(BdrvChild *child, int *drained_end_counter)
     assert(blk->quiesce_counter);
 
     assert(blk->public.throttle_group_member.io_limits_disabled);
-    atomic_dec(&blk->public.throttle_group_member.io_limits_disabled);
+    qatomic_dec(&blk->public.throttle_group_member.io_limits_disabled);
 
     if (--blk->quiesce_counter == 0) {
         if (blk->dev_ops && blk->dev_ops->drained_end) {
block/io.c +24 -24

185 lines not on this page · raw message

block/nfs.c +1 -1
index 61a249a9fc..f86e660374 100644
@@ -721,7 +721,7 @@ nfs_get_allocated_file_size_cb(int ret, struct nfs_context *nfs, void *data,
     }
 
     /* Set task->complete before reading bs->wakeup.  */
-    atomic_mb_set(&task->complete, 1);
+    qatomic_mb_set(&task->complete, 1);
     bdrv_wakeup(task->bs);
 }
 
block/sheepdog.c +1 -1
index cbbebc1aaf..2f5c0eb376 100644
@@ -665,7 +665,7 @@ out:
     srco->co = NULL;
     srco->ret = ret;
     /* Set srco->finished before reading bs->wakeup.  */
-    atomic_mb_set(&srco->finished, true);
+    qatomic_mb_set(&srco->finished, true);
     if (srco->bs) {
         bdrv_wakeup(srco->bs);
     }
block/throttle-groups.c +6 -6
index 4e28365d8d..e2f2813c0f 100644
@@ -228,7 +228,7 @@ static ThrottleGroupMember *next_throttle_token(ThrottleGroupMember *tgm,
      * immediately if it has pending requests. Otherwise we could be
      * forcing it to wait for other member's throttled requests. */
     if (tgm_has_pending_reqs(tgm, is_write) &&
-        atomic_read(&tgm->io_limits_disabled)) {
+        qatomic_read(&tgm->io_limits_disabled)) {
         return tgm;
     }
 
@@ -272,7 +272,7 @@ static bool throttle_group_schedule_timer(ThrottleGroupMember *tgm,
     ThrottleTimers *tt = &tgm->throttle_timers;
     bool must_wait;
 
-    if (atomic_read(&tgm->io_limits_disabled)) {
+    if (qatomic_read(&tgm->io_limits_disabled)) {
         return false;
     }
 
@@ -417,7 +417,7 @@ static void coroutine_fn throttle_group_restart_queue_entry(void *opaque)
 
     g_free(data);
 
-    atomic_dec(&tgm->restart_pending);
+    qatomic_dec(&tgm->restart_pending);
     aio_wait_kick();
 }
 
@@ -434,7 +434,7 @@ static void throttle_group_restart_queue(ThrottleGroupMember *tgm, bool is_write
      * be no timer pending on this tgm at this point */
     assert(!timer_pending(tgm->throttle_timers.timers[is_write]));
 
-    atomic_inc(&tgm->restart_pending);
+    qatomic_inc(&tgm->restart_pending);
 
     co = qemu_coroutine_create(throttle_group_restart_queue_entry, rd);
     aio_co_enter(tgm->aio_context, co);
@@ -544,7 +544,7 @@ void throttle_group_register_tgm(ThrottleGroupMember *tgm,
 
     tgm->throttle_state = ts;
     tgm->aio_context = ctx;
-    atomic_set(&tgm->restart_pending, 0);
+    qatomic_set(&tgm->restart_pending, 0);
 
     qemu_mutex_lock(&tg->lock);
     /* If the ThrottleGroup is new set this ThrottleGroupMember as the token */
@@ -592,7 +592,7 @@ void throttle_group_unregister_tgm(ThrottleGroupMember *tgm)
     }
 
     /* Wait for throttle_group_restart_queue_entry() coroutines to finish */
-    AIO_WAIT_WHILE(tgm->aio_context, atomic_read(&tgm->restart_pending) > 0);
+    AIO_WAIT_WHILE(tgm->aio_context, qatomic_read(&tgm->restart_pending) > 0);
 
     qemu_mutex_lock(&tg->lock);
     for (i = 0; i < 2; i++) {
block/throttle.c +2 -2
index 9a0f38149a..b685166ad4 100644
@@ -217,7 +217,7 @@ static void throttle_reopen_abort(BDRVReopenState *reopen_state)
 static void coroutine_fn throttle_co_drain_begin(BlockDriverState *bs)
 {
     ThrottleGroupMember *tgm = bs->opaque;
-    if (atomic_fetch_inc(&tgm->io_limits_disabled) == 0) {
+    if (qatomic_fetch_inc(&tgm->io_limits_disabled) == 0) {
         throttle_group_restart_tgm(tgm);
     }
 }
@@ -226,7 +226,7 @@ static void coroutine_fn throttle_co_drain_end(BlockDriverState *bs)
 {
     ThrottleGroupMember *tgm = bs->opaque;
     assert(tgm->io_limits_disabled);
-    atomic_dec(&tgm->io_limits_disabled);
+    qatomic_dec(&tgm->io_limits_disabled);
 }
 
 static const char *const throttle_strong_runtime_opts[] = {
blockdev.c +1 -1
index 7f2561081e..bebd3ba1c3 100644
@@ -1604,7 +1604,7 @@ static void external_snapshot_commit(BlkActionState *common)
     /* We don't need (or want) to use the transactional
      * bdrv_reopen_multiple() across all the entries at once, because we
      * don't want to abort all of them if one of them fails the reopen */
-    if (!atomic_read(&state->old_bs->copy_on_read)) {
+    if (!qatomic_read(&state->old_bs->copy_on_read)) {
         bdrv_reopen_set_read_only(state->old_bs, true, NULL);
     }
 
blockjob.c +1 -1
index 470facfd47..98ac8af982 100644
@@ -298,7 +298,7 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp)
     info = g_new0(BlockJobInfo, 1);
     info->type      = g_strdup(job_type_str(&job->job));
     info->device    = g_strdup(job->job.id);
-    info->busy      = atomic_read(&job->job.busy);
+    info->busy      = qatomic_read(&job->job.busy);
     info->paused    = job->job.pause_count > 0;
     info->offset    = job->job.progress.current;
     info->len       = job->job.progress.total;
contrib/libvhost-user/libvhost-user.c +1 -1
index 9d30ff2283..05861b1419 100644
@@ -448,7 +448,7 @@ static void
 vu_log_page(uint8_t *log_table, uint64_t page)
 {
     DPRINT("Logged dirty guest page: %"PRId64"\n", page);
-    atomic_or(&log_table[page / 8], 1 << (page % 8));
+    qatomic_or(&log_table[page / 8], 1 << (page % 8));
 }
 
 static void
cpus-common.c +13 -13
index 34044f4e4c..83475efff7 100644
@@ -148,7 +148,7 @@ void do_run_on_cpu(CPUState *cpu, run_on_cpu_func func, run_on_cpu_data data,
     wi.exclusive = false;
 
     queue_work_on_cpu(cpu, &wi);
-    while (!atomic_mb_read(&wi.done)) {
+    while (!qatomic_mb_read(&wi.done)) {
         CPUState *self_cpu = current_cpu;
 
         qemu_cond_wait(&qemu_work_cond, mutex);
@@ -188,20 +188,20 @@ void start_exclusive(void)
     exclusive_idle();
 
     /* Make all other cpus stop executing.  */
-    atomic_set(&pending_cpus, 1);
+    qatomic_set(&pending_cpus, 1);
 
     /* Write pending_cpus before reading other_cpu->running.  */
     smp_mb();
     running_cpus = 0;
     CPU_FOREACH(other_cpu) {
-        if (atomic_read(&other_cpu->running)) {
+        if (qatomic_read(&other_cpu->running)) {
             other_cpu->has_waiter = true;
             running_cpus++;
             qemu_cpu_kick(other_cpu);
         }
     }
 
-    atomic_set(&pending_cpus, running_cpus + 1);
+    qatomic_set(&pending_cpus, running_cpus + 1);
     while (pending_cpus > 1) {
         qemu_cond_wait(&exclusive_cond, &qemu_cpu_list_lock);
     }
@@ -220,7 +220,7 @@ void end_exclusive(void)
     current_cpu->in_exclusive_context = false;
 
     qemu_mutex_lock(&qemu_cpu_list_lock);
-    atomic_set(&pending_cpus, 0);
+    qatomic_set(&pending_cpus, 0);
     qemu_cond_broadcast(&exclusive_resume);
     qemu_mutex_unlock(&qemu_cpu_list_lock);
 }
@@ -228,7 +228,7 @@ void end_exclusive(void)
 /* Wait for exclusive ops to finish, and begin cpu execution.  */
 void cpu_exec_start(CPUState *cpu)
 {
-    atomic_set(&cpu->running, true);
+    qatomic_set(&cpu->running, true);
 
     /* Write cpu->running before reading pending_cpus.  */
     smp_mb();
@@ -246,17 +246,17 @@ void cpu_exec_start(CPUState *cpu)
      * 3. pending_cpus == 0.  Then start_exclusive is definitely going to
      * see cpu->running == true, and it will kick the CPU.
      */
-    if (unlikely(atomic_read(&pending_cpus))) {
+    if (unlikely(qatomic_read(&pending_cpus))) {
         QEMU_LOCK_GUARD(&qemu_cpu_list_lock);
         if (!cpu->has_waiter) {
             /* Not counted in pending_cpus, let the exclusive item
              * run.  Since we have the lock, just set cpu->running to true
              * while holding it; no need to check pending_cpus again.
              */
-            atomic_set(&cpu->running, false);
+            qatomic_set(&cpu->running, false);
             exclusive_idle();
             /* Now pending_cpus is zero.  */
-            atomic_set(&cpu->running, true);
+            qatomic_set(&cpu->running, true);
         } else {
             /* Counted in pending_cpus, go ahead and release the
              * waiter at cpu_exec_end.
@@ -268,7 +268,7 @@ void cpu_exec_start(CPUState *cpu)
 /* Mark cpu as not executing, and release pending exclusive ops.  */
 void cpu_exec_end(CPUState *cpu)
 {
-    atomic_set(&cpu->running, false);
+    qatomic_set(&cpu->running, false);
 
     /* Write cpu->running before reading pending_cpus.  */
     smp_mb();
@@ -288,11 +288,11 @@ void cpu_exec_end(CPUState *cpu)
      * see cpu->running == false, and it can ignore this CPU until the
      * next cpu_exec_start.
      */
-    if (unlikely(atomic_read(&pending_cpus))) {
+    if (unlikely(qatomic_read(&pending_cpus))) {
         QEMU_LOCK_GUARD(&qemu_cpu_list_lock);
         if (cpu->has_waiter) {
             cpu->has_waiter = false;
-            atomic_set(&pending_cpus, pending_cpus - 1);
+            qatomic_set(&pending_cpus, pending_cpus - 1);
             if (pending_cpus == 1) {
                 qemu_cond_signal(&exclusive_cond);
             }
@@ -346,7 +346,7 @@ void process_queued_cpu_work(CPUState *cpu)
         if (wi->free) {
             g_free(wi);
         } else {
-            atomic_mb_set(&wi->done, true);
+            qatomic_mb_set(&wi->done, true);
         }
     }
     qemu_mutex_unlock(&cpu->work_mutex);
dump/dump.c +4 -4
index 13fda440a4..45da46a952 100644
@@ -1572,7 +1572,7 @@ static void dump_state_prepare(DumpState *s)
 bool dump_in_progress(void)
 {
     DumpState *state = &dump_state_global;
-    return (atomic_read(&state->status) == DUMP_STATUS_ACTIVE);
+    return (qatomic_read(&state->status) == DUMP_STATUS_ACTIVE);
 }
 
 /* calculate total size of memory to be dumped (taking filter into
@@ -1882,7 +1882,7 @@ static void dump_process(DumpState *s, Error **errp)
 
     /* make sure status is written after written_size updates */
     smp_wmb();
-    atomic_set(&s->status,
+    qatomic_set(&s->status,
                (local_err ? DUMP_STATUS_FAILED : DUMP_STATUS_COMPLETED));
 
     /* send DUMP_COMPLETED message (unconditionally) */
@@ -1908,7 +1908,7 @@ DumpQueryResult *qmp_query_dump(Error **errp)
 {
     DumpQueryResult *result = g_new(DumpQueryResult, 1);
     DumpState *state = &dump_state_global;
-    result->status = atomic_read(&state->status);
+    result->status = qatomic_read(&state->status);
     /* make sure we are reading status and written_size in order */
     smp_rmb();
     result->completed = state->written_size;
@@ -2013,7 +2013,7 @@ void qmp_dump_guest_memory(bool paging, const char *file,
               begin, length, &local_err);
     if (local_err) {
         error_propagate(errp, local_err);
-        atomic_set(&s->status, DUMP_STATUS_FAILED);
+        qatomic_set(&s->status, DUMP_STATUS_FAILED);
         return;
     }
 
exec.c +26 -23

173 lines not on this page · raw message

hw/core/cpu.c +3 -3
index 8f65383ffb..c55c09f734 100644
@@ -111,10 +111,10 @@ void cpu_reset_interrupt(CPUState *cpu, int mask)
 
 void cpu_exit(CPUState *cpu)
 {
-    atomic_set(&cpu->exit_request, 1);
+    qatomic_set(&cpu->exit_request, 1);
     /* Ensure cpu_exec will see the exit request after TCG has exited.  */
     smp_wmb();
-    atomic_set(&cpu->icount_decr_ptr->u16.high, -1);
+    qatomic_set(&cpu->icount_decr_ptr->u16.high, -1);
 }
 
 int cpu_write_elf32_qemunote(WriteCoreDumpFunction f, CPUState *cpu,
@@ -261,7 +261,7 @@ static void cpu_common_reset(DeviceState *dev)
     cpu->halted = cpu->start_powered_off;
     cpu->mem_io_pc = 0;
     cpu->icount_extra = 0;
-    atomic_set(&cpu->icount_decr_ptr->u32, 0);
+    qatomic_set(&cpu->icount_decr_ptr->u32, 0);
     cpu->can_do_io = 1;
     cpu->exception_index = -1;
     cpu->crash_occurred = false;
hw/display/qxl.c +2 -2
index 11871340e7..431c107096 100644
@@ -1908,7 +1908,7 @@ static void qxl_send_events(PCIQXLDevice *d, uint32_t events)
     /*
      * Older versions of Spice forgot to define the QXLRam struct
      * with the '__aligned__(4)' attribute. clang 7 and newer will
-     * thus warn that atomic_fetch_or(&d->ram->int_pending, ...)
+     * thus warn that qatomic_fetch_or(&d->ram->int_pending, ...)
      * might be a misaligned atomic access, and will generate an
      * out-of-line call for it, which results in a link error since
      * we don't currently link against libatomic.
@@ -1928,7 +1928,7 @@ static void qxl_send_events(PCIQXLDevice *d, uint32_t events)
 #define ALIGNED_UINT32_PTR(P) ((uint32_t *)P)
 #endif
 
-    old_pending = atomic_fetch_or(ALIGNED_UINT32_PTR(&d->ram->int_pending),
+    old_pending = qatomic_fetch_or(ALIGNED_UINT32_PTR(&d->ram->int_pending),
                                   le_events);
     if ((old_pending & le_events) == le_events) {
         return;
hw/hyperv/hyperv.c +5 -5
index 4b26db1365..cb1074f234 100644
@@ -231,7 +231,7 @@ static void sint_msg_bh(void *opaque)
     HvSintRoute *sint_route = opaque;
     HvSintStagedMessage *staged_msg = sint_route->staged_msg;
 
-    if (atomic_read(&staged_msg->state) != HV_STAGED_MSG_POSTED) {
+    if (qatomic_read(&staged_msg->state) != HV_STAGED_MSG_POSTED) {
         /* status nor ready yet (spurious ack from guest?), ignore */
         return;
     }
@@ -240,7 +240,7 @@ static void sint_msg_bh(void *opaque)
     staged_msg->status = 0;
 
     /* staged message processing finished, ready to start over */
-    atomic_set(&staged_msg->state, HV_STAGED_MSG_FREE);
+    qatomic_set(&staged_msg->state, HV_STAGED_MSG_FREE);
     /* drop the reference taken in hyperv_post_msg */
     hyperv_sint_route_unref(sint_route);
 }
@@ -278,7 +278,7 @@ static void cpu_post_msg(CPUState *cs, run_on_cpu_data data)
     memory_region_set_dirty(&synic->msg_page_mr, 0, sizeof(*synic->msg_page));
 
 posted:
-    atomic_set(&staged_msg->state, HV_STAGED_MSG_POSTED);
+    qatomic_set(&staged_msg->state, HV_STAGED_MSG_POSTED);
     /*
      * Notify the msg originator of the progress made; if the slot was busy we
      * set msg_pending flag in it so it will be the guest who will do EOM and
@@ -301,7 +301,7 @@ int hyperv_post_msg(HvSintRoute *sint_route, struct hyperv_message *src_msg)
     assert(staged_msg);
 
     /* grab the staging area */
-    if (atomic_cmpxchg(&staged_msg->state, HV_STAGED_MSG_FREE,
+    if (qatomic_cmpxchg(&staged_msg->state, HV_STAGED_MSG_FREE,
                        HV_STAGED_MSG_BUSY) != HV_STAGED_MSG_FREE) {
         return -EAGAIN;
     }
@@ -351,7 +351,7 @@ int hyperv_set_event_flag(HvSintRoute *sint_route, unsigned eventno)
     set_mask = BIT_MASK(eventno);
     flags = synic->event_page->slot[sint_route->sint].flags;
 
-    if ((atomic_fetch_or(&flags[set_idx], set_mask) & set_mask) != set_mask) {
+    if ((qatomic_fetch_or(&flags[set_idx], set_mask) & set_mask) != set_mask) {
         memory_region_set_dirty(&synic->event_page_mr, 0,
                                 sizeof(*synic->event_page));
         ret = hyperv_sint_route_set_sint(sint_route);
hw/hyperv/vmbus.c +1 -1
index 6ef895bc35..896e981f85 100644
@@ -747,7 +747,7 @@ static int vmbus_channel_notify_guest(VMBusChannel *chan)
 
     idx = BIT_WORD(chan->id);
     mask = BIT_MASK(chan->id);
-    if ((atomic_fetch_or(&int_map[idx], mask) & mask) != mask) {
+    if ((qatomic_fetch_or(&int_map[idx], mask) & mask) != mask) {
         res = hyperv_sint_route_set_sint(chan->notify_route);
         dirty = len;
     }
hw/i386/xen/xen-hvm.c +1 -1
index cde981bad6..a39a648ca6 100644
@@ -1140,7 +1140,7 @@ static int handle_buffered_iopage(XenIOState *state)
         assert(req.dir == IOREQ_WRITE);
         assert(!req.data_is_ptr);
 
-        atomic_add(&buf_page->read_pointer, qw + 1);
+        qatomic_add(&buf_page->read_pointer, qw + 1);
     }
 
     return req.count;
hw/intc/rx_icu.c +6 -6
index df4b6a8d22..94e17a9dea 100644
@@ -81,8 +81,8 @@ static void rxicu_request(RXICUState *icu, int n_IRQ)
     int enable;
 
     enable = icu->ier[n_IRQ / 8] & (1 << (n_IRQ & 7));
-    if (n_IRQ > 0 && enable != 0 && atomic_read(&icu->req_irq) < 0) {
-        atomic_set(&icu->req_irq, n_IRQ);
+    if (n_IRQ > 0 && enable != 0 && qatomic_read(&icu->req_irq) < 0) {
+        qatomic_set(&icu->req_irq, n_IRQ);
         set_irq(icu, n_IRQ, rxicu_level(icu, n_IRQ));
     }
 }
@@ -124,10 +124,10 @@ static void rxicu_set_irq(void *opaque, int n_IRQ, int level)
     }
     if (issue == 0 && src->sense == TRG_LEVEL) {
         icu->ir[n_IRQ] = 0;
-        if (atomic_read(&icu->req_irq) == n_IRQ) {
+        if (qatomic_read(&icu->req_irq) == n_IRQ) {
             /* clear request */
             set_irq(icu, n_IRQ, 0);
-            atomic_set(&icu->req_irq, -1);
+            qatomic_set(&icu->req_irq, -1);
         }
         return;
     }
@@ -144,11 +144,11 @@ static void rxicu_ack_irq(void *opaque, int no, int level)
     int n_IRQ;
     int max_pri;
 
-    n_IRQ = atomic_read(&icu->req_irq);
+    n_IRQ = qatomic_read(&icu->req_irq);
     if (n_IRQ < 0) {
         return;
     }
-    atomic_set(&icu->req_irq, -1);
+    qatomic_set(&icu->req_irq, -1);
     if (icu->src[n_IRQ].sense != TRG_LEVEL) {
         icu->ir[n_IRQ] = 0;
     }
hw/intc/sifive_plic.c +2 -2
index af611f8db8..f42fd695d8 100644
@@ -89,12 +89,12 @@ static void sifive_plic_print_state(SiFivePLICState *plic)
 
 static uint32_t atomic_set_masked(uint32_t *a, uint32_t mask, uint32_t value)
 {
-    uint32_t old, new, cmp = atomic_read(a);
+    uint32_t old, new, cmp = qatomic_read(a);
 
     do {
         old = cmp;
         new = (old & ~mask) | (value & mask);
-        cmp = atomic_cmpxchg(a, old, new);
+        cmp = qatomic_cmpxchg(a, old, new);
     } while (old != cmp);
 
     return old;
hw/misc/edu.c +8 -8
index 0ff9d1ac78..e935c418d4 100644
@@ -212,7 +212,7 @@ static uint64_t edu_mmio_read(void *opaque, hwaddr addr, unsigned size)
         qemu_mutex_unlock(&edu->thr_mutex);
         break;
     case 0x20:
-        val = atomic_read(&edu->status);
+        val = qatomic_read(&edu->status);
         break;
     case 0x24:
         val = edu->irq_status;
@@ -252,7 +252,7 @@ static void edu_mmio_write(void *opaque, hwaddr addr, uint64_t val,
         edu->addr4 = ~val;
         break;
     case 0x08:
-        if (atomic_read(&edu->status) & EDU_STATUS_COMPUTING) {
+        if (qatomic_read(&edu->status) & EDU_STATUS_COMPUTING) {
             break;
         }
         /* EDU_STATUS_COMPUTING cannot go 0->1 concurrently, because it is only
@@ -260,15 +260,15 @@ static void edu_mmio_write(void *opaque, hwaddr addr, uint64_t val,
          */
         qemu_mutex_lock(&edu->thr_mutex);
         edu->fact = val;
-        atomic_or(&edu->status, EDU_STATUS_COMPUTING);
+        qatomic_or(&edu->status, EDU_STATUS_COMPUTING);
         qemu_cond_signal(&edu->thr_cond);
         qemu_mutex_unlock(&edu->thr_mutex);
         break;
     case 0x20:
         if (val & EDU_STATUS_IRQFACT) {
-            atomic_or(&edu->status, EDU_STATUS_IRQFACT);
+            qatomic_or(&edu->status, EDU_STATUS_IRQFACT);
         } else {
-            atomic_and(&edu->status, ~EDU_STATUS_IRQFACT);
+            qatomic_and(&edu->status, ~EDU_STATUS_IRQFACT);
         }
         break;
     case 0x60:
@@ -322,7 +322,7 @@ static void *edu_fact_thread(void *opaque)
         uint32_t val, ret = 1;
 
         qemu_mutex_lock(&edu->thr_mutex);
-        while ((atomic_read(&edu->status) & EDU_STATUS_COMPUTING) == 0 &&
+        while ((qatomic_read(&edu->status) & EDU_STATUS_COMPUTING) == 0 &&
                         !edu->stopping) {
             qemu_cond_wait(&edu->thr_cond, &edu->thr_mutex);
         }
@@ -347,9 +347,9 @@ static void *edu_fact_thread(void *opaque)
         qemu_mutex_lock(&edu->thr_mutex);
         edu->fact = ret;
         qemu_mutex_unlock(&edu->thr_mutex);
-        atomic_and(&edu->status, ~EDU_STATUS_COMPUTING);
+        qatomic_and(&edu->status, ~EDU_STATUS_COMPUTING);
 
-        if (atomic_read(&edu->status) & EDU_STATUS_IRQFACT) {
+        if (qatomic_read(&edu->status) & EDU_STATUS_IRQFACT) {
             qemu_mutex_lock_iothread();
             edu_raise_irq(edu, FACT_IRQ);
             qemu_mutex_unlock_iothread();
hw/net/virtio-net.c +5 -5
index cb0d27084c..7bf27b9db7 100644
@@ -933,7 +933,7 @@ static void virtio_net_set_features(VirtIODevice *vdev, uint64_t features)
 
     if (virtio_has_feature(features, VIRTIO_NET_F_STANDBY)) {
         qapi_event_send_failover_negotiated(n->netclient_name);
-        atomic_set(&n->primary_should_be_hidden, false);
+        qatomic_set(&n->primary_should_be_hidden, false);
         failover_add_primary(n, &err);
         if (err) {
             n->primary_dev = virtio_connect_failover_devices(n, n->qdev, &err);
@@ -3168,7 +3168,7 @@ static void virtio_net_handle_migration_primary(VirtIONet *n,
     bool should_be_hidden;
     Error *err = NULL;
 
-    should_be_hidden = atomic_read(&n->primary_should_be_hidden);
+    should_be_hidden = qatomic_read(&n->primary_should_be_hidden);
 
     if (!n->primary_dev) {
         n->primary_dev = virtio_connect_failover_devices(n, n->qdev, &err);
@@ -3183,7 +3183,7 @@ static void virtio_net_handle_migration_primary(VirtIONet *n,
                     qdev_get_vmsd(n->primary_dev),
                     n->primary_dev);
             qapi_event_send_unplug_primary(n->primary_device_id);
-            atomic_set(&n->primary_should_be_hidden, true);
+            qatomic_set(&n->primary_should_be_hidden, true);
         } else {
             warn_report("couldn't unplug primary device");
         }
@@ -3234,7 +3234,7 @@ static int virtio_net_primary_should_be_hidden(DeviceListener *listener,
     n->primary_device_opts = device_opts;
 
     /* primary_should_be_hidden is set during feature negotiation */
-    hide = atomic_read(&n->primary_should_be_hidden);
+    hide = qatomic_read(&n->primary_should_be_hidden);
 
     if (n->primary_device_dict) {
         g_free(n->primary_device_id);
@@ -3291,7 +3291,7 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
     if (n->failover) {
         n->primary_listener.should_be_hidden =
             virtio_net_primary_should_be_hidden;
-        atomic_set(&n->primary_should_be_hidden, true);
+        qatomic_set(&n->primary_should_be_hidden, true);
         device_listener_register(&n->primary_listener);
         n->migration_state.notify = virtio_net_migration_state_notifier;
         add_migration_state_change_notifier(&n->migration_state);
hw/rdma/rdma_backend.c +9 -9
index db7e5c8be5..5de010b1fa 100644
@@ -68,7 +68,7 @@ static void free_cqe_ctx(gpointer data, gpointer user_data)
     bctx = rdma_rm_get_cqe_ctx(rdma_dev_res, cqe_ctx_id);
     if (bctx) {
         rdma_rm_dealloc_cqe_ctx(rdma_dev_res, cqe_ctx_id);
-        atomic_dec(&rdma_dev_res->stats.missing_cqe);
+        qatomic_dec(&rdma_dev_res->stats.missing_cqe);
     }
     g_free(bctx);
 }
@@ -81,7 +81,7 @@ static void clean_recv_mads(RdmaBackendDev *backend_dev)
         cqe_ctx_id = rdma_protected_qlist_pop_int64(&backend_dev->
                                                     recv_mads_list);
         if (cqe_ctx_id != -ENOENT) {
-            atomic_inc(&backend_dev->rdma_dev_res->stats.missing_cqe);
+            qatomic_inc(&backend_dev->rdma_dev_res->stats.missing_cqe);
             free_cqe_ctx(GINT_TO_POINTER(cqe_ctx_id),
                          backend_dev->rdma_dev_res);
         }
@@ -123,7 +123,7 @@ static int rdma_poll_cq(RdmaDeviceResources *rdma_dev_res, struct ibv_cq *ibcq)
             }
             total_ne += ne;
         } while (ne > 0);
-        atomic_sub(&rdma_dev_res->stats.missing_cqe, total_ne);
+        qatomic_sub(&rdma_dev_res->stats.missing_cqe, total_ne);
     }
 
     if (ne < 0) {
@@ -195,17 +195,17 @@ static void *comp_handler_thread(void *arg)
 
 static inline void disable_rdmacm_mux_async(RdmaBackendDev *backend_dev)
 {
-    atomic_set(&backend_dev->rdmacm_mux.can_receive, 0);
+    qatomic_set(&backend_dev->rdmacm_mux.can_receive, 0);
 }
 
 static inline void enable_rdmacm_mux_async(RdmaBackendDev *backend_dev)
 {
-    atomic_set(&backend_dev->rdmacm_mux.can_receive, sizeof(RdmaCmMuxMsg));
+    qatomic_set(&backend_dev->rdmacm_mux.can_receive, sizeof(RdmaCmMuxMsg));
 }
 
 static inline int rdmacm_mux_can_process_async(RdmaBackendDev *backend_dev)
 {
-    return atomic_read(&backend_dev->rdmacm_mux.can_receive);
+    return qatomic_read(&backend_dev->rdmacm_mux.can_receive);
 }
 
 static int rdmacm_mux_check_op_status(CharBackend *mad_chr_be)
@@ -555,7 +555,7 @@ void rdma_backend_post_send(RdmaBackendDev *backend_dev,
         goto err_dealloc_cqe_ctx;
     }
 
-    atomic_inc(&backend_dev->rdma_dev_res->stats.missing_cqe);
+    qatomic_inc(&backend_dev->rdma_dev_res->stats.missing_cqe);
     backend_dev->rdma_dev_res->stats.tx++;
 
     return;
@@ -658,7 +658,7 @@ void rdma_backend_post_recv(RdmaBackendDev *backend_dev,
         goto err_dealloc_cqe_ctx;
     }
 
-    atomic_inc(&backend_dev->rdma_dev_res->stats.missing_cqe);
+    qatomic_inc(&backend_dev->rdma_dev_res->stats.missing_cqe);
     backend_dev->rdma_dev_res->stats.rx_bufs++;
 
     return;
@@ -710,7 +710,7 @@ void rdma_backend_post_srq_recv(RdmaBackendDev *backend_dev,
         goto err_dealloc_cqe_ctx;
     }
 
-    atomic_inc(&backend_dev->rdma_dev_res->stats.missing_cqe);
+    qatomic_inc(&backend_dev->rdma_dev_res->stats.missing_cqe);
     backend_dev->rdma_dev_res->stats.rx_bufs++;
     backend_dev->rdma_dev_res->stats.rx_srq++;
 
hw/rdma/rdma_rm.c +1 -1
index 60957f88db..49141d4074 100644
@@ -790,7 +790,7 @@ int rdma_rm_init(RdmaDeviceResources *dev_res, struct ibv_device_attr *dev_attr)
     qemu_mutex_init(&dev_res->lock);
 
     memset(&dev_res->stats, 0, sizeof(dev_res->stats));
-    atomic_set(&dev_res->stats.missing_cqe, 0);
+    qatomic_set(&dev_res->stats.missing_cqe, 0);
 
     return 0;
 }
hw/rdma/vmw/pvrdma_dev_ring.c +2 -2
index c122fe7035..f0bcde74b0 100644
@@ -38,8 +38,8 @@ int pvrdma_ring_init(PvrdmaRing *ring, const char *name, PCIDevice *dev,
     ring->max_elems = max_elems;
     ring->elem_sz = elem_sz;
     /* TODO: Give a moment to think if we want to redo driver settings
-    atomic_set(&ring->ring_state->prod_tail, 0);
-    atomic_set(&ring->ring_state->cons_head, 0);
+    qatomic_set(&ring->ring_state->prod_tail, 0);
+    qatomic_set(&ring->ring_state->cons_head, 0);
     */
     ring->npages = npages;
     ring->pages = g_malloc(npages * sizeof(void *));
hw/s390x/s390-pci-bus.c +1 -1
index 92146a2119..fb4cee87a4 100644
@@ -650,7 +650,7 @@ static uint8_t set_ind_atomic(uint64_t ind_loc, uint8_t to_be_set)
     actual = *ind_addr;
     do {
         expected = actual;
-        actual = atomic_cmpxchg(ind_addr, expected, expected | to_be_set);
+        actual = qatomic_cmpxchg(ind_addr, expected, expected | to_be_set);
     } while (actual != expected);
     cpu_physical_memory_unmap((void *)ind_addr, len, 1, len);
 
hw/s390x/virtio-ccw.c +1 -1
index 8feb3451a0..8d140dc50f 100644
@@ -800,7 +800,7 @@ static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc,
     actual = *ind_addr;
     do {
         expected = actual;
-        actual = atomic_cmpxchg(ind_addr, expected, expected | to_be_set);
+        actual = qatomic_cmpxchg(ind_addr, expected, expected | to_be_set);
     } while (actual != expected);
     trace_virtio_ccw_set_ind(ind_loc, actual, actual | to_be_set);
     cpu_physical_memory_unmap((void *)ind_addr, len, 1, len);
hw/virtio/vhost.c +1 -1
index 1a1384e7a6..0119516254 100644
@@ -90,7 +90,7 @@ static void vhost_dev_sync_region(struct vhost_dev *dev,
         }
         /* Data must be read atomically. We don't really need barrier semantics
          * but it's easier to use atomic_* than roll our own. */
-        log = atomic_xchg(from, 0);
+        log = qatomic_xchg(from, 0);
         while (log) {
             int bit = ctzl(log);
             hwaddr page_addr;
hw/virtio/virtio-mmio.c +3 -3
index f12d1595aa..e1b5c3b81e 100644
@@ -179,7 +179,7 @@ static uint64_t virtio_mmio_read(void *opaque, hwaddr offset, unsigned size)
         }
         return proxy->vqs[vdev->queue_sel].enabled;
     case VIRTIO_MMIO_INTERRUPT_STATUS:
-        return atomic_read(&vdev->isr);
+        return qatomic_read(&vdev->isr);
     case VIRTIO_MMIO_STATUS:
         return vdev->status;
     case VIRTIO_MMIO_CONFIG_GENERATION:
@@ -370,7 +370,7 @@ static void virtio_mmio_write(void *opaque, hwaddr offset, uint64_t value,
         }
         break;
     case VIRTIO_MMIO_INTERRUPT_ACK:
-        atomic_and(&vdev->isr, ~value);
+        qatomic_and(&vdev->isr, ~value);
         virtio_update_irq(vdev);
         break;
     case VIRTIO_MMIO_STATUS:
@@ -496,7 +496,7 @@ static void virtio_mmio_update_irq(DeviceState *opaque, uint16_t vector)
     if (!vdev) {
         return;
     }
-    level = (atomic_read(&vdev->isr) != 0);
+    level = (qatomic_read(&vdev->isr) != 0);
     trace_virtio_mmio_setting_irq(level);
     qemu_set_irq(proxy->irq, level);
 }
hw/virtio/virtio-pci.c +3 -3
index 5bc769f685..02790e3237 100644
@@ -72,7 +72,7 @@ static void virtio_pci_notify(DeviceState *d, uint16_t vector)
         msix_notify(&proxy->pci_dev, vector);
     else {
         VirtIODevice *vdev = virtio_bus_get_device(&proxy->bus);
-        pci_set_irq(&proxy->pci_dev, atomic_read(&vdev->isr) & 1);
+        pci_set_irq(&proxy->pci_dev, qatomic_read(&vdev->isr) & 1);
     }
 }
 
@@ -398,7 +398,7 @@ static uint32_t virtio_ioport_read(VirtIOPCIProxy *proxy, uint32_t addr)
         break;
     case VIRTIO_PCI_ISR:
         /* reading from the ISR also clears it. */
-        ret = atomic_xchg(&vdev->isr, 0);
+        ret = qatomic_xchg(&vdev->isr, 0);
         pci_irq_deassert(&proxy->pci_dev);
         break;
     case VIRTIO_MSI_CONFIG_VECTOR:
@@ -1362,7 +1362,7 @@ static uint64_t virtio_pci_isr_read(void *opaque, hwaddr addr,
 {
     VirtIOPCIProxy *proxy = opaque;
     VirtIODevice *vdev = virtio_bus_get_device(&proxy->bus);
-    uint64_t val = atomic_xchg(&vdev->isr, 0);
+    uint64_t val = qatomic_xchg(&vdev->isr, 0);
     pci_irq_deassert(&proxy->pci_dev);
 
     return val;
hw/virtio/virtio.c +8 -8
index e983025217..3a3d012d9f 100644
@@ -149,8 +149,8 @@ static void virtio_virtqueue_reset_region_cache(struct VirtQueue *vq)
 {
     VRingMemoryRegionCaches *caches;
 
-    caches = atomic_read(&vq->vring.caches);
-    atomic_rcu_set(&vq->vring.caches, NULL);
+    caches = qatomic_read(&vq->vring.caches);
+    qatomic_rcu_set(&vq->vring.caches, NULL);
     if (caches) {
         call_rcu(caches, virtio_free_region_cache, rcu);
     }
@@ -197,7 +197,7 @@ static void virtio_init_region_cache(VirtIODevice *vdev, int n)
         goto err_avail;
     }
 
-    atomic_rcu_set(&vq->vring.caches, new);
+    qatomic_rcu_set(&vq->vring.caches, new);
     if (old) {
         call_rcu(old, virtio_free_region_cache, rcu);
     }
@@ -283,7 +283,7 @@ static void vring_packed_flags_write(VirtIODevice *vdev,
 /* Called within rcu_read_lock().  */
 static VRingMemoryRegionCaches *vring_get_region_caches(struct VirtQueue *vq)
 {
-    return atomic_rcu_read(&vq->vring.caches);
+    return qatomic_rcu_read(&vq->vring.caches);
 }
 
 /* Called within rcu_read_lock().  */
@@ -2007,7 +2007,7 @@ void virtio_reset(void *opaque)
     vdev->queue_sel = 0;
     vdev->status = 0;
     vdev->disabled = false;
-    atomic_set(&vdev->isr, 0);
+    qatomic_set(&vdev->isr, 0);
     vdev->config_vector = VIRTIO_NO_VECTOR;
     virtio_notify_vector(vdev, vdev->config_vector);
 
@@ -2439,13 +2439,13 @@ void virtio_del_queue(VirtIODevice *vdev, int n)
 
 static void virtio_set_isr(VirtIODevice *vdev, int value)
 {
-    uint8_t old = atomic_read(&vdev->isr);
+    uint8_t old = qatomic_read(&vdev->isr);
 
     /* Do not write ISR if it does not change, so that its cacheline remains
      * shared in the common case where the guest does not read it.
      */
[… more lines in the raw message]

Lines of this file are not shown. View the raw message.

hw/xtensa/pic_cpu.c +2 -2
index 1d5982a9e4..6c9447565d 100644

Lines of this file are not shown. View the raw message.

iothread.c +3 -3
index 3a3860a09c..69eff9efbc 100644

Lines of this file are not shown. View the raw message.

linux-user/hppa/cpu_loop.c +6 -5
index 9915456a1d..d7e1ec7722 100644

Lines of this file are not shown. View the raw message.

linux-user/signal.c +4 -4
index 8cf51ffecd..73de934c65 100644

Lines of this file are not shown. View the raw message.

migration/colo-failover.c +2 -2
index b717edc8e2..42453481c4 100644

Lines of this file are not shown. View the raw message.

migration/migration.c +4 -4
index 58a5452471..d9d1e0b190 100644

Lines of this file are not shown. View the raw message.

migration/multifd.c +9 -9
index ac84a61797..fd57378db8 100644

Lines of this file are not shown. View the raw message.

migration/postcopy-ram.c +17 -17
index baf094ba3a..1654ff11a5 100644

Lines of this file are not shown. View the raw message.

migration/rdma.c +17 -17
index 3bd30d46ad..0340841fad 100644

Lines of this file are not shown. View the raw message.

monitor/hmp.c +3 -3
index d598dd02bb..4ecdefd705 100644

Lines of this file are not shown. View the raw message.

monitor/misc.c +1 -1
index 0b1b9b196c..262f2bd951 100644

Lines of this file are not shown. View the raw message.

monitor/monitor.c +3 -3
index b385a3d569..0f32892ad4 100644

Lines of this file are not shown. View the raw message.

qemu-nbd.c +1 -1
index 33476a1000..e39d3b23c1 100644

Lines of this file are not shown. View the raw message.

qga/commands.c +6 -6
index d3fec807c1..3dcd5fbe5c 100644

Lines of this file are not shown. View the raw message.

qom/object.c +10 -10
index a91a6a515a..1196970635 100644

Lines of this file are not shown. View the raw message.

scsi/qemu-pr-helper.c +2 -2
index 57ad830d54..d26faaf91e 100644

Lines of this file are not shown. View the raw message.

softmmu/cpu-throttle.c +5 -5
index 4e6b2818ca..2ec4b8e0bc 100644

Lines of this file are not shown. View the raw message.

softmmu/cpus.c +21 -21

42 lines not on this page · raw message

softmmu/memory.c +3 -3
index d030eb6f7c..da5f90f141 100644

Lines of this file are not shown. View the raw message.

softmmu/vl.c +1 -1
index f7b103467c..50d8c2e42e 100644

Lines of this file are not shown. View the raw message.

target/arm/mte_helper.c +3 -3
index 891306f5b0..5615c6706c 100644

Lines of this file are not shown. View the raw message.

target/hppa/op_helper.c +1 -1
index 5685e303ab..39361d3759 100644

Lines of this file are not shown. View the raw message.

target/i386/mem_helper.c +1 -1
index acf41f8885..3a6d3ae2ef 100644

Lines of this file are not shown. View the raw message.

target/i386/whpx-all.c +3 -3
index c78baac6df..5a0f337a4c 100644

Lines of this file are not shown. View the raw message.

target/riscv/cpu_helper.c +1 -1
index f4c4111536..904899054d 100644

Lines of this file are not shown. View the raw message.

target/s390x/mem_helper.c +2 -2
index a237dec757..bd25eed3e8 100644

Lines of this file are not shown. View the raw message.

target/xtensa/exc_helper.c +2 -2
index 58a64e6d62..2f032bc053 100644

Lines of this file are not shown. View the raw message.

target/xtensa/op_helper.c +1 -1
index 09f4962d00..143476849f 100644

Lines of this file are not shown. View the raw message.

tcg/tcg.c +29 -29

58 lines not on this page · raw message

tcg/tci.c +1 -1
index 46fe9ce63f..82039fd163 100644

Lines of this file are not shown. View the raw message.

tests/atomic64-bench.c +7 -7
index 121a8c14f4..e474753d34 100644

Lines of this file are not shown. View the raw message.

tests/atomic_add-bench.c +7 -7
index 5666f6bbff..f05471ab45 100644

Lines of this file are not shown. View the raw message.

tests/iothread.c +1 -1
index d3a2ee9a01..afde12b4ef 100644

Lines of this file are not shown. View the raw message.

tests/qht-bench.c +6 -6
index 362f03cb03..2e5b70ccd0 100644

Lines of this file are not shown. View the raw message.

tests/rcutorture.c +12 -12
index 732f03abda..de6f649058 100644

Lines of this file are not shown. View the raw message.

tests/test-aio-multithread.c +26 -26

52 lines not on this page · raw message

tests/test-logging.c +2 -2
index 8b1522cfed..ccb819f193 100644

Lines of this file are not shown. View the raw message.

tests/test-rcu-list.c +20 -18
index 92be51ec50..49641e1936 100644

Lines of this file are not shown. View the raw message.

tests/test-thread-pool.c +5 -5
index 0b675923f6..70dc6314a1 100644

Lines of this file are not shown. View the raw message.

util/aio-posix.c +7 -7
index f7f13ebfc2..280f27bb99 100644

Lines of this file are not shown. View the raw message.

util/aio-wait.c +1 -1
index b4877493f8..bdb3d3af22 100644

Lines of this file are not shown. View the raw message.

util/aio-win32.c +3 -2
index 49bd90e62e..e7b1d649e9 100644

Lines of this file are not shown. View the raw message.

util/async.c +14 -14
index 4266745dee..f758354c6a 100644

Lines of this file are not shown. View the raw message.

util/atomic64.c +5 -5
index b198a6c9c8..93037d5b11 100644

Lines of this file are not shown. View the raw message.

util/bitmap.c +7 -7
index 1753ff7f5b..1f201393ae 100644

Lines of this file are not shown. View the raw message.

util/cacheinfo.c +1 -1
index d94dc6adc8..7804c186b6 100644

Lines of this file are not shown. View the raw message.

util/fdmon-epoll.c +2 -2
index fcd989d47d..e11a8a022e 100644

Lines of this file are not shown. View the raw message.

util/fdmon-io_uring.c +6 -6
index 1d14177df0..1461dfa407 100644

Lines of this file are not shown. View the raw message.

util/lockcnt.c +27 -25

52 lines not on this page · raw message

util/log.c +5 -5
index bdb3d712e8..4b423062aa 100644

Lines of this file are not shown. View the raw message.

util/qemu-coroutine-lock.c +9 -9
index 5da5234155..36927b5f88 100644

Lines of this file are not shown. View the raw message.

util/qemu-coroutine-sleep.c +2 -2
index 769a76e57d..8c4dac4fd7 100644

Lines of this file are not shown. View the raw message.

util/qemu-coroutine.c +3 -3
index c3caa6c770..38fb6d3084 100644

Lines of this file are not shown. View the raw message.

util/qemu-sockets.c +2 -2
index 99ce2fd5e6..de4bf7616e 100644

Lines of this file are not shown. View the raw message.

util/qemu-thread-posix.c +6 -6
index b4c2359272..dcff5e7c5d 100644

Lines of this file are not shown. View the raw message.

util/qemu-thread-win32.c +6 -6
index d207b0cb58..cb5aa2018c 100644

Lines of this file are not shown. View the raw message.

util/qemu-timer.c +6 -6
index 878d80fd5e..ca677836cb 100644

Lines of this file are not shown. View the raw message.

util/qht.c +29 -28

57 lines not on this page · raw message

util/qsp.c +25 -25

50 lines not on this page · raw message

util/rcu.c +18 -18
index c4fefa9333..13ac0f75cb 100644

Lines of this file are not shown. View the raw message.

util/stats64.c +17 -17
index 389c365a9e..897613c949 100644

Lines of this file are not shown. View the raw message.

docs/devel/atomics.rst +67 -67

134 lines not on this page · raw message

scripts/kernel-doc +1 -1
index 030b5c8691..40ad782e34 100755

Lines of this file are not shown. View the raw message.

tcg/aarch64/tcg-target.c.inc +1 -1
index 948c35d825..2607fe4ab9 100644

Lines of this file are not shown. View the raw message.

tcg/mips/tcg-target.c.inc +1 -1
index bd5b8e09a0..7aa2073520 100644

Lines of this file are not shown. View the raw message.

tcg/ppc/tcg-target.c.inc +3 -3
index 393c4b30e0..7cb40b0466 100644

Lines of this file are not shown. View the raw message.

tcg/sparc/tcg-target.c.inc +3 -2
index 0f1d91fc21..40bc12290c 100644

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.26.2

Part of the diff is not shown here. View the raw message.

maintainer Acked-by on v3 1/1
#1 lore raw
Quoted 22 lines · Stefan Hajnoczi

clang's C11 atomic_fetch_*() functions only take a C11 atomic type pointer argument. QEMU uses direct types (int, etc) and this causes a compiler error when a QEMU code calls these functions in a source file that also included <stdatomic.h> via a system header file:

  $ CC=clang CXX=clang++ ./configure ... && make
  ../util/async.c:79:17: error: address argument to atomic operation must be a pointer to _Atomic type ('unsigned int *' invalid)

Avoid using atomic_*() names in QEMU's atomic.h since that namespace is used by <stdatomic.h>. Prefix QEMU's APIs with 'q' so that atomic.h and <stdatomic.h> can co-exist. I checked /usr/include on my machine and searched GitHub for existing "qatomic_" users but there seem to be none.

This patch was generated using:

  $ git grep -h -o '\<atomic\(64\)\?_[a-z0-9_]\+' include/qemu/atomic.h | \
    sort -u >/tmp/changed_identifiers
  $ for identifier in $(</tmp/changed_identifiers); do
        sed -i "s%\<$identifier\>%q$identifier%g" \
            $(git grep -I -l "\<$identifier\>")
    done
Quoted 3 lines

I manually fixed line-wrap issues and misaligned rST tables.

Thank you very much!

Paolo

Quoted 7,539 lines

---
v3:
* Use qatomic_ instead of atomic_ [Paolo]
* The diff of my manual fixups is available here:
https://vmsplice.net/~stefan/atomic-namespace-pre-fixups-v3.diff
- Dropping #ifndef qatomic_fetch_add in atomic.h
- atomic_##X(haddr, val) glue macros not caught by grep
- Keep atomic_add-bench name
- C preprocessor backslash-newline ('\') column alignment
- Line wrapping
v2:
* The diff of my manual fixups is available here:
https://vmsplice.net/~stefan/atomic-namespace-pre-fixups.diff
- Dropping #ifndef qemu_atomic_fetch_add in atomic.h
- atomic_##X(haddr, val) glue macros not caught by grep
- Keep atomic_add-bench name
- C preprocessor backslash-newline ('\') column alignment
- Line wrapping
* Use grep -I to avoid accidentally modifying binary files (RISC-V OpenSBI ELFs) [Eric Blake]
* Tweak .gitorder to show atomic.h changes first [Eric Blake]
* Update grep commands in commit description so reviewers can reproduce mechanical changes [Eric Blake]

---
 include/qemu/atomic.h                         | 248 +++++++++---------
 docs/devel/lockcnt.txt                        |   8 +-
 docs/devel/rcu.txt                            |  34 +--
 accel/tcg/atomic_template.h                   |  20 +-
 include/block/aio-wait.h                      |   4 +-
 include/block/aio.h                           |   8 +-
 include/exec/cpu_ldst.h                       |   2 +-
 include/exec/exec-all.h                       |   6 +-
 include/exec/log.h                            |   6 +-
 include/exec/memory.h                         |   2 +-
 include/exec/ram_addr.h                       |  26 +-
 include/exec/ramlist.h                        |   2 +-
 include/exec/tb-lookup.h                      |   4 +-
 include/hw/core/cpu.h                         |   2 +-
 include/qemu/atomic128.h                      |   6 +-
 include/qemu/bitops.h                         |   2 +-
 include/qemu/coroutine.h                      |   2 +-
 include/qemu/log.h                            |   6 +-
 include/qemu/queue.h                          |   7 +-
 include/qemu/rcu.h                            |  10 +-
 include/qemu/rcu_queue.h                      | 100 +++----
 include/qemu/seqlock.h                        |   8 +-
 include/qemu/stats64.h                        |  28 +-
 include/qemu/thread.h                         |  24 +-
 .../infiniband/hw/vmw_pvrdma/pvrdma_ring.h    |  14 +-
 linux-user/qemu.h                             |   2 +-
 tcg/i386/tcg-target.h                         |   2 +-
 tcg/s390/tcg-target.h                         |   2 +-
 tcg/tci/tcg-target.h                          |   2 +-
 accel/kvm/kvm-all.c                           |  12 +-
 accel/tcg/cpu-exec.c                          |  15 +-
 accel/tcg/cputlb.c                            |  24 +-
 accel/tcg/tcg-all.c                           |   2 +-
 accel/tcg/translate-all.c                     |  55 ++--
 audio/jackaudio.c                             |  18 +-
 block.c                                       |   4 +-
 block/block-backend.c                         |  15 +-
 block/io.c                                    |  48 ++--
 block/nfs.c                                   |   2 +-
 block/sheepdog.c                              |   2 +-
 block/throttle-groups.c                       |  12 +-
 block/throttle.c                              |   4 +-
 blockdev.c                                    |   2 +-
 blockjob.c                                    |   2 +-
 contrib/libvhost-user/libvhost-user.c         |   2 +-
 cpus-common.c                                 |  26 +-
 dump/dump.c                                   |   8 +-
 exec.c                                        |  49 ++--
 hw/core/cpu.c                                 |   6 +-
 hw/display/qxl.c                              |   4 +-
 hw/hyperv/hyperv.c                            |  10 +-
 hw/hyperv/vmbus.c                             |   2 +-
 hw/i386/xen/xen-hvm.c                         |   2 +-
 hw/intc/rx_icu.c                              |  12 +-
 hw/intc/sifive_plic.c                         |   4 +-
 hw/misc/edu.c                                 |  16 +-
 hw/net/virtio-net.c                           |  10 +-
 hw/rdma/rdma_backend.c                        |  18 +-
 hw/rdma/rdma_rm.c                             |   2 +-
 hw/rdma/vmw/pvrdma_dev_ring.c                 |   4 +-
 hw/s390x/s390-pci-bus.c                       |   2 +-
 hw/s390x/virtio-ccw.c                         |   2 +-
 hw/virtio/vhost.c                             |   2 +-
 hw/virtio/virtio-mmio.c                       |   6 +-
 hw/virtio/virtio-pci.c                        |   6 +-
 hw/virtio/virtio.c                            |  16 +-
 hw/xtensa/pic_cpu.c                           |   4 +-
 iothread.c                                    |   6 +-
 linux-user/hppa/cpu_loop.c                    |  11 +-
 linux-user/signal.c                           |   8 +-
 migration/colo-failover.c                     |   4 +-
 migration/migration.c                         |   8 +-
 migration/multifd.c                           |  18 +-
 migration/postcopy-ram.c                      |  34 +--
 migration/rdma.c                              |  34 +--
 monitor/hmp.c                                 |   6 +-
 monitor/misc.c                                |   2 +-
 monitor/monitor.c                             |   6 +-
 qemu-nbd.c                                    |   2 +-
 qga/commands.c                                |  12 +-
 qom/object.c                                  |  20 +-
 scsi/qemu-pr-helper.c                         |   4 +-
 softmmu/cpu-throttle.c                        |  10 +-
 softmmu/cpus.c                                |  42 +--
 softmmu/memory.c                              |   6 +-
 softmmu/vl.c                                  |   2 +-
 target/arm/mte_helper.c                       |   6 +-
 target/hppa/op_helper.c                       |   2 +-
 target/i386/mem_helper.c                      |   2 +-
 target/i386/whpx-all.c                        |   6 +-
 target/riscv/cpu_helper.c                     |   2 +-
 target/s390x/mem_helper.c                     |   4 +-
 target/xtensa/exc_helper.c                    |   4 +-
 target/xtensa/op_helper.c                     |   2 +-
 tcg/tcg.c                                     |  58 ++--
 tcg/tci.c                                     |   2 +-
 tests/atomic64-bench.c                        |  14 +-
 tests/atomic_add-bench.c                      |  14 +-
 tests/iothread.c                              |   2 +-
 tests/qht-bench.c                             |  12 +-
 tests/rcutorture.c                            |  24 +-
 tests/test-aio-multithread.c                  |  52 ++--
 tests/test-logging.c                          |   4 +-
 tests/test-rcu-list.c                         |  38 +--
 tests/test-thread-pool.c                      |  10 +-
 util/aio-posix.c                              |  14 +-
 util/aio-wait.c                               |   2 +-
 util/aio-win32.c                              |   5 +-
 util/async.c                                  |  28 +-
 util/atomic64.c                               |  10 +-
 util/bitmap.c                                 |  14 +-
 util/cacheinfo.c                              |   2 +-
 util/fdmon-epoll.c                            |   4 +-
 util/fdmon-io_uring.c                         |  12 +-
 util/lockcnt.c                                |  52 ++--
 util/log.c                                    |  10 +-
 util/qemu-coroutine-lock.c                    |  18 +-
 util/qemu-coroutine-sleep.c                   |   4 +-
 util/qemu-coroutine.c                         |   6 +-
 util/qemu-sockets.c                           |   4 +-
 util/qemu-thread-posix.c                      |  12 +-
 util/qemu-thread-win32.c                      |  12 +-
 util/qemu-timer.c                             |  12 +-
 util/qht.c                                    |  57 ++--
 util/qsp.c                                    |  50 ++--
 util/rcu.c                                    |  36 +--
 util/stats64.c                                |  34 +--
 docs/devel/atomics.rst                        | 134 +++++-----
 scripts/kernel-doc                            |   2 +-
 tcg/aarch64/tcg-target.c.inc                  |   2 +-
 tcg/mips/tcg-target.c.inc                     |   2 +-
 tcg/ppc/tcg-target.c.inc                      |   6 +-
 tcg/sparc/tcg-target.c.inc                    |   5 +-
 133 files changed, 1041 insertions(+), 1018 deletions(-)
include/qemu/atomic.h +126 -122
index ff72db5115..c1d211a351 100644
@@ -125,49 +125,49 @@
  * no effect on the generated code but not using the atomic primitives
  * will get flagged by sanitizers as a violation.
  */
-#define atomic_read__nocheck(ptr) \
+#define qatomic_read__nocheck(ptr) \
     __atomic_load_n(ptr, __ATOMIC_RELAXED)
 
-#define atomic_read(ptr)                              \
-    ({                                                \
+#define qatomic_read(ptr)                              \
+    ({                                                 \
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
-    atomic_read__nocheck(ptr);                        \
+    qatomic_read__nocheck(ptr);                        \
     })
 
-#define atomic_set__nocheck(ptr, i) \
+#define qatomic_set__nocheck(ptr, i) \
     __atomic_store_n(ptr, i, __ATOMIC_RELAXED)
 
-#define atomic_set(ptr, i)  do {                      \
+#define qatomic_set(ptr, i)  do {                      \
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
-    atomic_set__nocheck(ptr, i);                      \
+    qatomic_set__nocheck(ptr, i);                      \
 } while(0)
 
 /* See above: most compilers currently treat consume and acquire the
- * same, but this slows down atomic_rcu_read unnecessarily.
+ * same, but this slows down qatomic_rcu_read unnecessarily.
  */
 #ifdef __SANITIZE_THREAD__
-#define atomic_rcu_read__nocheck(ptr, valptr)           \
+#define qatomic_rcu_read__nocheck(ptr, valptr)           \
     __atomic_load(ptr, valptr, __ATOMIC_CONSUME);
 #else
-#define atomic_rcu_read__nocheck(ptr, valptr)           \
-    __atomic_load(ptr, valptr, __ATOMIC_RELAXED);       \
+#define qatomic_rcu_read__nocheck(ptr, valptr)           \
+    __atomic_load(ptr, valptr, __ATOMIC_RELAXED);        \
     smp_read_barrier_depends();
 #endif
 
-#define atomic_rcu_read(ptr)                          \
-    ({                                                \
+#define qatomic_rcu_read(ptr)                          \
+    ({                                                 \
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
-    typeof_strip_qual(*ptr) _val;                     \
-    atomic_rcu_read__nocheck(ptr, &_val);             \
-    _val;                                             \
+    typeof_strip_qual(*ptr) _val;                      \
+    qatomic_rcu_read__nocheck(ptr, &_val);             \
+    _val;                                              \
     })
 
-#define atomic_rcu_set(ptr, i) do {                   \
+#define qatomic_rcu_set(ptr, i) do {                   \
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
-    __atomic_store_n(ptr, i, __ATOMIC_RELEASE);       \
+    __atomic_store_n(ptr, i, __ATOMIC_RELEASE);        \
 } while(0)
 
-#define atomic_load_acquire(ptr)                        \
+#define qatomic_load_acquire(ptr)                       \
     ({                                                  \
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE);  \
     typeof_strip_qual(*ptr) _val;                       \
@@ -175,7 +175,7 @@
     _val;                                               \
     })
 
-#define atomic_store_release(ptr, i)  do {              \
+#define qatomic_store_release(ptr, i)  do {             \
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE);  \
     __atomic_store_n(ptr, i, __ATOMIC_RELEASE);         \
 } while(0)
@@ -183,56 +183,61 @@
 
 /* All the remaining operations are fully sequentially consistent */
 
-#define atomic_xchg__nocheck(ptr, i)    ({                  \
+#define qatomic_xchg__nocheck(ptr, i)    ({                 \
     __atomic_exchange_n(ptr, (i), __ATOMIC_SEQ_CST);        \
 })
 
-#define atomic_xchg(ptr, i)    ({                           \
+#define qatomic_xchg(ptr, i)    ({                          \
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE);      \
-    atomic_xchg__nocheck(ptr, i);                           \
+    qatomic_xchg__nocheck(ptr, i);                          \
 })
 
 /* Returns the eventual value, failed or not */
-#define atomic_cmpxchg__nocheck(ptr, old, new)    ({                    \
+#define qatomic_cmpxchg__nocheck(ptr, old, new)    ({                   \
     typeof_strip_qual(*ptr) _old = (old);                               \
     (void)__atomic_compare_exchange_n(ptr, &_old, new, false,           \
                               __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST);      \
     _old;                                                               \
 })
 
-#define atomic_cmpxchg(ptr, old, new)    ({                             \
+#define qatomic_cmpxchg(ptr, old, new)    ({                            \
     QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE);                  \
-    atomic_cmpxchg__nocheck(ptr, old, new);                             \
+    qatomic_cmpxchg__nocheck(ptr, old, new);                            \
 })
 
 /* Provide shorter names for GCC atomic builtins, return old value */
-#define atomic_fetch_inc(ptr)  __atomic_fetch_add(ptr, 1, __ATOMIC_SEQ_CST)
-#define atomic_fetch_dec(ptr)  __atomic_fetch_sub(ptr, 1, __ATOMIC_SEQ_CST)
+#define qatomic_fetch_inc(ptr)  __atomic_fetch_add(ptr, 1, __ATOMIC_SEQ_CST)
+#define qatomic_fetch_dec(ptr)  __atomic_fetch_sub(ptr, 1, __ATOMIC_SEQ_CST)
 
-#ifndef atomic_fetch_add
-#define atomic_fetch_add(ptr, n) __atomic_fetch_add(ptr, n, __ATOMIC_SEQ_CST)
-#define atomic_fetch_sub(ptr, n) __atomic_fetch_sub(ptr, n, __ATOMIC_SEQ_CST)
-#define atomic_fetch_and(ptr, n) __atomic_fetch_and(ptr, n, __ATOMIC_SEQ_CST)
-#define atomic_fetch_or(ptr, n)  __atomic_fetch_or(ptr, n, __ATOMIC_SEQ_CST)
-#define atomic_fetch_xor(ptr, n) __atomic_fetch_xor(ptr, n, __ATOMIC_SEQ_CST)
-#endif
+#define qatomic_fetch_add(ptr, n) __atomic_fetch_add(ptr, n, __ATOMIC_SEQ_CST)
+#define qatomic_fetch_sub(ptr, n) __atomic_fetch_sub(ptr, n, __ATOMIC_SEQ_CST)
+#define qatomic_fetch_and(ptr, n) __atomic_fetch_and(ptr, n, __ATOMIC_SEQ_CST)
+#define qatomic_fetch_or(ptr, n)  __atomic_fetch_or(ptr, n, __ATOMIC_SEQ_CST)
+#define qatomic_fetch_xor(ptr, n) __atomic_fetch_xor(ptr, n, __ATOMIC_SEQ_CST)
 
-#define atomic_inc_fetch(ptr)    __atomic_add_fetch(ptr, 1, __ATOMIC_SEQ_CST)
-#define atomic_dec_fetch(ptr)    __atomic_sub_fetch(ptr, 1, __ATOMIC_SEQ_CST)
-#define atomic_add_fetch(ptr, n) __atomic_add_fetch(ptr, n, __ATOMIC_SEQ_CST)
-#define atomic_sub_fetch(ptr, n) __atomic_sub_fetch(ptr, n, __ATOMIC_SEQ_CST)
-#define atomic_and_fetch(ptr, n) __atomic_and_fetch(ptr, n, __ATOMIC_SEQ_CST)
-#define atomic_or_fetch(ptr, n)  __atomic_or_fetch(ptr, n, __ATOMIC_SEQ_CST)
-#define atomic_xor_fetch(ptr, n) __atomic_xor_fetch(ptr, n, __ATOMIC_SEQ_CST)
+#define qatomic_inc_fetch(ptr)    __atomic_add_fetch(ptr, 1, __ATOMIC_SEQ_CST)
+#define qatomic_dec_fetch(ptr)    __atomic_sub_fetch(ptr, 1, __ATOMIC_SEQ_CST)
+#define qatomic_add_fetch(ptr, n) __atomic_add_fetch(ptr, n, __ATOMIC_SEQ_CST)
+#define qatomic_sub_fetch(ptr, n) __atomic_sub_fetch(ptr, n, __ATOMIC_SEQ_CST)
+#define qatomic_and_fetch(ptr, n) __atomic_and_fetch(ptr, n, __ATOMIC_SEQ_CST)
+#define qatomic_or_fetch(ptr, n)  __atomic_or_fetch(ptr, n, __ATOMIC_SEQ_CST)
+#define qatomic_xor_fetch(ptr, n) __atomic_xor_fetch(ptr, n, __ATOMIC_SEQ_CST)
 
 /* And even shorter names that return void.  */
-#define atomic_inc(ptr)    ((void) __atomic_fetch_add(ptr, 1, __ATOMIC_SEQ_CST))
-#define atomic_dec(ptr)    ((void) __atomic_fetch_sub(ptr, 1, __ATOMIC_SEQ_CST))
-#define atomic_add(ptr, n) ((void) __atomic_fetch_add(ptr, n, __ATOMIC_SEQ_CST))
-#define atomic_sub(ptr, n) ((void) __atomic_fetch_sub(ptr, n, __ATOMIC_SEQ_CST))
-#define atomic_and(ptr, n) ((void) __atomic_fetch_and(ptr, n, __ATOMIC_SEQ_CST))
-#define atomic_or(ptr, n)  ((void) __atomic_fetch_or(ptr, n, __ATOMIC_SEQ_CST))
-#define atomic_xor(ptr, n) ((void) __atomic_fetch_xor(ptr, n, __ATOMIC_SEQ_CST))
+#define qatomic_inc(ptr) \
+    ((void) __atomic_fetch_add(ptr, 1, __ATOMIC_SEQ_CST))
+#define qatomic_dec(ptr) \
+    ((void) __atomic_fetch_sub(ptr, 1, __ATOMIC_SEQ_CST))
+#define qatomic_add(ptr, n) \
+    ((void) __atomic_fetch_add(ptr, n, __ATOMIC_SEQ_CST))
+#define qatomic_sub(ptr, n) \
+    ((void) __atomic_fetch_sub(ptr, n, __ATOMIC_SEQ_CST))
+#define qatomic_and(ptr, n) \
+    ((void) __atomic_fetch_and(ptr, n, __ATOMIC_SEQ_CST))
+#define qatomic_or(ptr, n) \
+    ((void) __atomic_fetch_or(ptr, n, __ATOMIC_SEQ_CST))
+#define qatomic_xor(ptr, n) \
+    ((void) __atomic_fetch_xor(ptr, n, __ATOMIC_SEQ_CST))
 
 #else /* __ATOMIC_RELAXED */
 
@@ -272,7 +277,7 @@
  * but it is a full barrier at the hardware level.  Add a compiler barrier
  * to make it a full barrier also at the compiler level.
  */
-#define atomic_xchg(ptr, i)    (barrier(), __sync_lock_test_and_set(ptr, i))
+#define qatomic_xchg(ptr, i)    (barrier(), __sync_lock_test_and_set(ptr, i))
 
 #elif defined(_ARCH_PPC)
 
@@ -325,14 +330,14 @@
 /* These will only be atomic if the processor does the fetch or store
  * in a single issue memory operation
  */
-#define atomic_read__nocheck(p)   (*(__typeof__(*(p)) volatile*) (p))
-#define atomic_set__nocheck(p, i) ((*(__typeof__(*(p)) volatile*) (p)) = (i))
+#define qatomic_read__nocheck(p)   (*(__typeof__(*(p)) volatile*) (p))
+#define qatomic_set__nocheck(p, i) ((*(__typeof__(*(p)) volatile*) (p)) = (i))
 
-#define atomic_read(ptr)       atomic_read__nocheck(ptr)
-#define atomic_set(ptr, i)     atomic_set__nocheck(ptr,i)
+#define qatomic_read(ptr)       qatomic_read__nocheck(ptr)
+#define qatomic_set(ptr, i)     qatomic_set__nocheck(ptr,i)
 
 /**
- * atomic_rcu_read - reads a RCU-protected pointer to a local variable
+ * qatomic_rcu_read - reads a RCU-protected pointer to a local variable
  * into a RCU read-side critical section. The pointer can later be safely
  * dereferenced within the critical section.
  *
@@ -342,21 +347,21 @@
  * Inserts memory barriers on architectures that require them (currently only
  * Alpha) and documents which pointers are protected by RCU.
  *
- * atomic_rcu_read also includes a compiler barrier to ensure that
+ * qatomic_rcu_read also includes a compiler barrier to ensure that
  * value-speculative optimizations (e.g. VSS: Value Speculation
  * Scheduling) does not perform the data read before the pointer read
  * by speculating the value of the pointer.
  *
- * Should match atomic_rcu_set(), atomic_xchg(), atomic_cmpxchg().
+ * Should match qatomic_rcu_set(), qatomic_xchg(), qatomic_cmpxchg().
  */
-#define atomic_rcu_read(ptr)    ({                \
-    typeof(*ptr) _val = atomic_read(ptr);         \
+#define qatomic_rcu_read(ptr)    ({               \
+    typeof(*ptr) _val = qatomic_read(ptr);        \
     smp_read_barrier_depends();                   \
     _val;                                         \
 })
 
 /**
- * atomic_rcu_set - assigns (publicizes) a pointer to a new data structure
+ * qatomic_rcu_set - assigns (publicizes) a pointer to a new data structure
  * meant to be read by RCU read-side critical sections.
  *
  * Documents which pointers will be dereferenced by RCU read-side critical
@@ -364,65 +369,64 @@
  * them. It also makes sure the compiler does not reorder code initializing the
  * data structure before its publication.
  *
- * Should match atomic_rcu_read().
+ * Should match qatomic_rcu_read().
  */
-#define atomic_rcu_set(ptr, i)  do {              \
+#define qatomic_rcu_set(ptr, i)  do {             \
     smp_wmb();                                    \
-    atomic_set(ptr, i);                           \
+    qatomic_set(ptr, i);                          \
 } while (0)
 
-#define atomic_load_acquire(ptr)    ({      \
-    typeof(*ptr) _val = atomic_read(ptr);   \
+#define qatomic_load_acquire(ptr)    ({     \
+    typeof(*ptr) _val = qatomic_read(ptr);  \
     smp_mb_acquire();                       \
     _val;                                   \
 })
 
-#define atomic_store_release(ptr, i)  do {  \
+#define qatomic_store_release(ptr, i)  do { \
     smp_mb_release();                       \
-    atomic_set(ptr, i);                     \
+    qatomic_set(ptr, i);                    \
 } while (0)
 
-#ifndef atomic_xchg
+#ifndef qatomic_xchg
 #if defined(__clang__)
-#define atomic_xchg(ptr, i)    __sync_swap(ptr, i)
+#define qatomic_xchg(ptr, i)    __sync_swap(ptr, i)
 #else
 /* __sync_lock_test_and_set() is documented to be an acquire barrier only.  */
-#define atomic_xchg(ptr, i)    (smp_mb(), __sync_lock_test_and_set(ptr, i))
+#define qatomic_xchg(ptr, i)    (smp_mb(), __sync_lock_test_and_set(ptr, i))
 #endif
 #endif
-#define atomic_xchg__nocheck  atomic_xchg
+#define qatomic_xchg__nocheck  qatomic_xchg
 
 /* Provide shorter names for GCC atomic builtins.  */
-#define atomic_fetch_inc(ptr)  __sync_fetch_and_add(ptr, 1)
-#define atomic_fetch_dec(ptr)  __sync_fetch_and_add(ptr, -1)
+#define qatomic_fetch_inc(ptr)  __sync_fetch_and_add(ptr, 1)
+#define qatomic_fetch_dec(ptr)  __sync_fetch_and_add(ptr, -1)
 
-#ifndef atomic_fetch_add
-#define atomic_fetch_add(ptr, n) __sync_fetch_and_add(ptr, n)
-#define atomic_fetch_sub(ptr, n) __sync_fetch_and_sub(ptr, n)
-#define atomic_fetch_and(ptr, n) __sync_fetch_and_and(ptr, n)
-#define atomic_fetch_or(ptr, n) __sync_fetch_and_or(ptr, n)
-#define atomic_fetch_xor(ptr, n) __sync_fetch_and_xor(ptr, n)
-#endif
+#define qatomic_fetch_add(ptr, n) __sync_fetch_and_add(ptr, n)
+#define qatomic_fetch_sub(ptr, n) __sync_fetch_and_sub(ptr, n)
+#define qatomic_fetch_and(ptr, n) __sync_fetch_and_and(ptr, n)
+#define qatomic_fetch_or(ptr, n) __sync_fetch_and_or(ptr, n)
+#define qatomic_fetch_xor(ptr, n) __sync_fetch_and_xor(ptr, n)
 
-#define atomic_inc_fetch(ptr)  __sync_add_and_fetch(ptr, 1)
-#define atomic_dec_fetch(ptr)  __sync_add_and_fetch(ptr, -1)
-#define atomic_add_fetch(ptr, n) __sync_add_and_fetch(ptr, n)
-#define atomic_sub_fetch(ptr, n) __sync_sub_and_fetch(ptr, n)
-#define atomic_and_fetch(ptr, n) __sync_and_and_fetch(ptr, n)
-#define atomic_or_fetch(ptr, n) __sync_or_and_fetch(ptr, n)
-#define atomic_xor_fetch(ptr, n) __sync_xor_and_fetch(ptr, n)
+#define qatomic_inc_fetch(ptr)  __sync_add_and_fetch(ptr, 1)
+#define qatomic_dec_fetch(ptr)  __sync_add_and_fetch(ptr, -1)
+#define qatomic_add_fetch(ptr, n) __sync_add_and_fetch(ptr, n)
+#define qatomic_sub_fetch(ptr, n) __sync_sub_and_fetch(ptr, n)
+#define qatomic_and_fetch(ptr, n) __sync_and_and_fetch(ptr, n)
+#define qatomic_or_fetch(ptr, n) __sync_or_and_fetch(ptr, n)
+#define qatomic_xor_fetch(ptr, n) __sync_xor_and_fetch(ptr, n)
 
-#define atomic_cmpxchg(ptr, old, new) __sync_val_compare_and_swap(ptr, old, new)
-#define atomic_cmpxchg__nocheck(ptr, old, new)  atomic_cmpxchg(ptr, old, new)
+#define qatomic_cmpxchg(ptr, old, new) \
+    __sync_val_compare_and_swap(ptr, old, new)
+#define qatomic_cmpxchg__nocheck(ptr, old, new)  qatomic_cmpxchg(ptr, old, new)
 
 /* And even shorter names that return void.  */
-#define atomic_inc(ptr)        ((void) __sync_fetch_and_add(ptr, 1))
-#define atomic_dec(ptr)        ((void) __sync_fetch_and_add(ptr, -1))
-#define atomic_add(ptr, n)     ((void) __sync_fetch_and_add(ptr, n))
-#define atomic_sub(ptr, n)     ((void) __sync_fetch_and_sub(ptr, n))
-#define atomic_and(ptr, n)     ((void) __sync_fetch_and_and(ptr, n))
-#define atomic_or(ptr, n)      ((void) __sync_fetch_and_or(ptr, n))
-#define atomic_xor(ptr, n)     ((void) __sync_fetch_and_xor(ptr, n))
+#define qatomic_inc(ptr)        ((void) __sync_fetch_and_add(ptr, 1))
+#define qatomic_dec(ptr)        ((void) __sync_fetch_and_add(ptr, -1))
+#define qatomic_add(ptr, n)     ((void) __sync_fetch_and_add(ptr, n))
+#define qatomic_sub(ptr, n)     ((void) __sync_fetch_and_sub(ptr, n))
+#define qatomic_and(ptr, n)     ((void) __sync_fetch_and_and(ptr, n))
+#define qatomic_or(ptr, n)      ((void) __sync_fetch_and_or(ptr, n))
+#define qatomic_xor(ptr, n)     ((void) __sync_fetch_and_xor(ptr, n))
 
 #endif /* __ATOMIC_RELAXED */
 
@@ -436,11 +440,11 @@
 /* This is more efficient than a store plus a fence.  */
 #if !defined(__SANITIZE_THREAD__)
 #if defined(__i386__) || defined(__x86_64__) || defined(__s390x__)
-#define atomic_mb_set(ptr, i)  ((void)atomic_xchg(ptr, i))
+#define qatomic_mb_set(ptr, i)  ((void)qatomic_xchg(ptr, i))
 #endif
 #endif
 
-/* atomic_mb_read/set semantics map Java volatile variables. They are
+/* qatomic_mb_read/set semantics map Java volatile variables. They are
  * less expensive on some platforms (notably POWER) than fully
  * sequentially consistent operations.
  *
@@ -448,58 +452,58 @@
  * use. See docs/devel/atomics.txt for more discussion.
  */
 
-#ifndef atomic_mb_read
-#define atomic_mb_read(ptr)                             \
-    atomic_load_acquire(ptr)
+#ifndef qatomic_mb_read
+#define qatomic_mb_read(ptr)                             \
+    qatomic_load_acquire(ptr)
 #endif
 
-#ifndef atomic_mb_set
-#define atomic_mb_set(ptr, i)  do {                     \
-    atomic_store_release(ptr, i);                       \
+#ifndef qatomic_mb_set
+#define qatomic_mb_set(ptr, i)  do {                    \
+    qatomic_store_release(ptr, i);                      \
     smp_mb();                                           \
 } while(0)
 #endif
 
-#define atomic_fetch_inc_nonzero(ptr) ({                                \
-    typeof_strip_qual(*ptr) _oldn = atomic_read(ptr);                   \
-    while (_oldn && atomic_cmpxchg(ptr, _oldn, _oldn + 1) != _oldn) {   \
-        _oldn = atomic_read(ptr);                                       \
+#define qatomic_fetch_inc_nonzero(ptr) ({                               \
+    typeof_strip_qual(*ptr) _oldn = qatomic_read(ptr);                  \
+    while (_oldn && qatomic_cmpxchg(ptr, _oldn, _oldn + 1) != _oldn) {  \
+        _oldn = qatomic_read(ptr);                                      \
     }                                                                   \
     _oldn;                                                              \
 })
 
 /* Abstractions to access atomically (i.e. "once") i64/u64 variables */
 #ifdef CONFIG_ATOMIC64
-static inline int64_t atomic_read_i64(const int64_t *ptr)
+static inline int64_t qatomic_read_i64(const int64_t *ptr)
 {
     /* use __nocheck because sizeof(void *) might be < sizeof(u64) */
-    return atomic_read__nocheck(ptr);
+    return qatomic_read__nocheck(ptr);
 }
 
-static inline uint64_t atomic_read_u64(const uint64_t *ptr)
+static inline uint64_t qatomic_read_u64(const uint64_t *ptr)
 {
-    return atomic_read__nocheck(ptr);
+    return qatomic_read__nocheck(ptr);
 }
 
-static inline void atomic_set_i64(int64_t *ptr, int64_t val)
+static inline void qatomic_set_i64(int64_t *ptr, int64_t val)
 {
-    atomic_set__nocheck(ptr, val);
+    qatomic_set__nocheck(ptr, val);
 }
 
-static inline void atomic_set_u64(uint64_t *ptr, uint64_t val)
+static inline void qatomic_set_u64(uint64_t *ptr, uint64_t val)
 {
-    atomic_set__nocheck(ptr, val);
+    qatomic_set__nocheck(ptr, val);
 }
 
-static inline void atomic64_init(void)
+static inline void qatomic64_init(void)
 {
 }
 #else /* !CONFIG_ATOMIC64 */
-int64_t  atomic_read_i64(const int64_t *ptr);
-uint64_t atomic_read_u64(const uint64_t *ptr);
-void atomic_set_i64(int64_t *ptr, int64_t val);
-void atomic_set_u64(uint64_t *ptr, uint64_t val);
-void atomic64_init(void);
+int64_t  qatomic_read_i64(const int64_t *ptr);
+uint64_t qatomic_read_u64(const uint64_t *ptr);
+void qatomic_set_i64(int64_t *ptr, int64_t val);
+void qatomic_set_u64(uint64_t *ptr, uint64_t val);
+void qatomic64_init(void);
 #endif /* !CONFIG_ATOMIC64 */
 
 #endif /* QEMU_ATOMIC_H */
docs/devel/lockcnt.txt +4 -4
index 7c099bc6c8..2d85462fe3 100644
@@ -95,7 +95,7 @@ not just frees, though there could be cases where this is not necessary.
 
 Reads, instead, can be done without taking the mutex, as long as the
 readers and writers use the same macros that are used for RCU, for
-example atomic_rcu_read, atomic_rcu_set, QLIST_FOREACH_RCU, etc.  This is
+example qatomic_rcu_read, qatomic_rcu_set, QLIST_FOREACH_RCU, etc.  This is
 because the reads are done outside a lock and a set or QLIST_INSERT_HEAD
 can happen concurrently with the read.  The RCU API ensures that the
 processor and the compiler see all required memory barriers.
@@ -189,7 +189,7 @@ qemu_lockcnt_lock and qemu_lockcnt_unlock:
     if (!xyz) {
         new_xyz = g_new(XYZ, 1);
...
-        atomic_rcu_set(&xyz, new_xyz);
+        qatomic_rcu_set(&xyz, new_xyz);
     }
     qemu_lockcnt_unlock(&xyz_lockcnt);
@@ -198,7 +198,7 @@ qemu_lockcnt_dec:
 
     qemu_lockcnt_inc(&xyz_lockcnt);
     if (xyz) {
-        XYZ *p = atomic_rcu_read(&xyz);
+        XYZ *p = qatomic_rcu_read(&xyz);
...
         /* Accesses can now be done through "p".  */
     }
@@ -222,7 +222,7 @@ the decrement, the locking and the check on count as follows:
 
     qemu_lockcnt_inc(&xyz_lockcnt);
     if (xyz) {
-        XYZ *p = atomic_rcu_read(&xyz);
+        XYZ *p = qatomic_rcu_read(&xyz);
...
         /* Accesses can now be done through "p".  */
     }
docs/devel/rcu.txt +17 -17
index 0ce15ba198..cdf002edd8 100644
@@ -130,13 +130,13 @@ The core RCU API is small:
 
             g_free_rcu(&foo, rcu);
 
-     typeof(*p) atomic_rcu_read(p);
+     typeof(*p) qatomic_rcu_read(p);
 
-        atomic_rcu_read() is similar to atomic_load_acquire(), but it makes
+        qatomic_rcu_read() is similar to qatomic_load_acquire(), but it makes
         some assumptions on the code that calls it.  This allows a more
         optimized implementation.
 
-        atomic_rcu_read assumes that whenever a single RCU critical
+        qatomic_rcu_read assumes that whenever a single RCU critical
         section reads multiple shared data, these reads are either
         data-dependent or need no ordering.  This is almost always the
         case when using RCU, because read-side critical sections typically
@@ -144,7 +144,7 @@ The core RCU API is small:
         every update) until reaching a data structure of interest,
         and then read from there.
 
-        RCU read-side critical sections must use atomic_rcu_read() to
+        RCU read-side critical sections must use qatomic_rcu_read() to
         read data, unless concurrent writes are prevented by another
         synchronization mechanism.
 
@@ -152,18 +152,18 @@ The core RCU API is small:
         data structure in a single direction, opposite to the direction
         in which the updater initializes it.
 
-     void atomic_rcu_set(p, typeof(*p) v);
+     void qatomic_rcu_set(p, typeof(*p) v);
 
-        atomic_rcu_set() is similar to atomic_store_release(), though it also
+        qatomic_rcu_set() is similar to qatomic_store_release(), though it also
         makes assumptions on the code that calls it in order to allow a more
         optimized implementation.
 
-        In particular, atomic_rcu_set() suffices for synchronization
+        In particular, qatomic_rcu_set() suffices for synchronization
         with readers, if the updater never mutates a field within a
         data item that is already accessible to readers.  This is the
         case when initializing a new copy of the RCU-protected data
         structure; just ensure that initialization of *p is carried out
-        before atomic_rcu_set() makes the data item visible to readers.
+        before qatomic_rcu_set() makes the data item visible to readers.
         If this rule is observed, writes will happen in the opposite
         order as reads in the RCU read-side critical sections (or if
         there is just one update), and there will be no need for other
@@ -212,7 +212,7 @@ DIFFERENCES WITH LINUX
   programming; not allowing this would prevent upgrading an RCU read-side
   critical section to become an updater.
 
-- atomic_rcu_read and atomic_rcu_set replace rcu_dereference and
+- qatomic_rcu_read and qatomic_rcu_set replace rcu_dereference and
   rcu_assign_pointer.  They take a _pointer_ to the variable being accessed.
 
 - call_rcu is a macro that has an extra argument (the name of the first
@@ -257,7 +257,7 @@ may be used as a restricted reference-counting mechanism.  For example,
 consider the following code fragment:
 
     rcu_read_lock();
-    p = atomic_rcu_read(&foo);
+    p = qatomic_rcu_read(&foo);
     /* do something with p. */
     rcu_read_unlock();
 
@@ -268,7 +268,7 @@ The write side looks simply like this (with appropriate locking):
 
     qemu_mutex_lock(&foo_mutex);
     old = foo;
-    atomic_rcu_set(&foo, new);
+    qatomic_rcu_set(&foo, new);
     qemu_mutex_unlock(&foo_mutex);
     synchronize_rcu();
     free(old);
@@ -277,7 +277,7 @@ If the processing cannot be done purely within the critical section, it
 is possible to combine this idiom with a "real" reference count:
 
     rcu_read_lock();
-    p = atomic_rcu_read(&foo);
+    p = qatomic_rcu_read(&foo);
     foo_ref(p);
     rcu_read_unlock();
     /* do something with p. */
@@ -287,7 +287,7 @@ The write side can be like this:
 
     qemu_mutex_lock(&foo_mutex);
     old = foo;
-    atomic_rcu_set(&foo, new);
+    qatomic_rcu_set(&foo, new);
     qemu_mutex_unlock(&foo_mutex);
     synchronize_rcu();
     foo_unref(old);
@@ -296,7 +296,7 @@ or with call_rcu:
 
     qemu_mutex_lock(&foo_mutex);
     old = foo;
-    atomic_rcu_set(&foo, new);
+    qatomic_rcu_set(&foo, new);
     qemu_mutex_unlock(&foo_mutex);
     call_rcu(foo_unref, old, rcu);
 
@@ -307,7 +307,7 @@ last reference may be dropped on the read side.  Hence you can
 use call_rcu() instead:
 
      foo_unref(struct foo *p) {
-        if (atomic_fetch_dec(&p->refcount) == 1) {
+        if (qatomic_fetch_dec(&p->refcount) == 1) {
             call_rcu(foo_destroy, p, rcu);
         }
     }
@@ -375,7 +375,7 @@ Instead, we store the size of the array with the array itself:
 
     read side:
         rcu_read_lock();
-        struct arr *array = atomic_rcu_read(&global_array);
+        struct arr *array = qatomic_rcu_read(&global_array);
         x = i < array->size ? array->data[i] : -1;
         rcu_read_unlock();
         return x;
@@ -392,7 +392,7 @@ Instead, we store the size of the array with the array itself:
 
             /* Removal phase.  */
             old_array = global_array;
-            atomic_rcu_set(&new_array->data, new_array);
+            qatomic_rcu_set(&new_array->data, new_array);
             synchronize_rcu();
 
             /* Reclamation phase.  */
accel/tcg/atomic_template.h +10 -10
index 26969487d6..0ff7f913e1 100644
@@ -83,7 +83,7 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr,
 #if DATA_SIZE == 16
     ret = atomic16_cmpxchg(haddr, cmpv, newv);
 #else
-    ret = atomic_cmpxchg__nocheck(haddr, cmpv, newv);
+    ret = qatomic_cmpxchg__nocheck(haddr, cmpv, newv);
 #endif
     ATOMIC_MMU_CLEANUP;
     atomic_trace_rmw_post(env, addr, info);
@@ -131,7 +131,7 @@ ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr,
                                          ATOMIC_MMU_IDX);
 
     atomic_trace_rmw_pre(env, addr, info);
-    ret = atomic_xchg__nocheck(haddr, val);
+    ret = qatomic_xchg__nocheck(haddr, val);
     ATOMIC_MMU_CLEANUP;
     atomic_trace_rmw_post(env, addr, info);
     return ret;
@@ -147,7 +147,7 @@ ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr,       \
     uint16_t info = trace_mem_build_info(SHIFT, false, 0, false,    \
                                          ATOMIC_MMU_IDX);           \
     atomic_trace_rmw_pre(env, addr, info);                          \
-    ret = atomic_##X(haddr, val);                                   \
+    ret = qatomic_##X(haddr, val);                                  \
     ATOMIC_MMU_CLEANUP;                                             \
     atomic_trace_rmw_post(env, addr, info);                         \
     return ret;                                                     \
@@ -182,10 +182,10 @@ ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr,       \
                                          ATOMIC_MMU_IDX);           \
     atomic_trace_rmw_pre(env, addr, info);                          \
     smp_mb();                                                       \
-    cmp = atomic_read__nocheck(haddr);                              \
+    cmp = qatomic_read__nocheck(haddr);                             \
     do {                                                            \
         old = cmp; new = FN(old, val);                              \
-        cmp = atomic_cmpxchg__nocheck(haddr, old, new);             \
+        cmp = qatomic_cmpxchg__nocheck(haddr, old, new);            \
     } while (cmp != old);                                           \
     ATOMIC_MMU_CLEANUP;                                             \
     atomic_trace_rmw_post(env, addr, info);                         \
@@ -230,7 +230,7 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr,
 #if DATA_SIZE == 16
     ret = atomic16_cmpxchg(haddr, BSWAP(cmpv), BSWAP(newv));
 #else
-    ret = atomic_cmpxchg__nocheck(haddr, BSWAP(cmpv), BSWAP(newv));
+    ret = qatomic_cmpxchg__nocheck(haddr, BSWAP(cmpv), BSWAP(newv));
 #endif
     ATOMIC_MMU_CLEANUP;
     atomic_trace_rmw_post(env, addr, info);
@@ -280,7 +280,7 @@ ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr,
                                          ATOMIC_MMU_IDX);
 
     atomic_trace_rmw_pre(env, addr, info);
-    ret = atomic_xchg__nocheck(haddr, BSWAP(val));
+    ret = qatomic_xchg__nocheck(haddr, BSWAP(val));
     ATOMIC_MMU_CLEANUP;
     atomic_trace_rmw_post(env, addr, info);
     return BSWAP(ret);
@@ -296,7 +296,7 @@ ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr,       \
     uint16_t info = trace_mem_build_info(SHIFT, false, MO_BSWAP,    \
                                          false, ATOMIC_MMU_IDX);    \
     atomic_trace_rmw_pre(env, addr, info);                          \
-    ret = atomic_##X(haddr, BSWAP(val));                            \
+    ret = qatomic_##X(haddr, BSWAP(val));                           \
     ATOMIC_MMU_CLEANUP;                                             \
     atomic_trace_rmw_post(env, addr, info);                         \
     return BSWAP(ret);                                              \
@@ -329,10 +329,10 @@ ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr,       \
                                          false, ATOMIC_MMU_IDX);    \
     atomic_trace_rmw_pre(env, addr, info);                          \
     smp_mb();                                                       \
-    ldn = atomic_read__nocheck(haddr);                              \
+    ldn = qatomic_read__nocheck(haddr);                             \
     do {                                                            \
         ldo = ldn; old = BSWAP(ldo); new = FN(old, val);            \
-        ldn = atomic_cmpxchg__nocheck(haddr, ldo, BSWAP(new));      \
+        ldn = qatomic_cmpxchg__nocheck(haddr, ldo, BSWAP(new));     \
     } while (ldo != ldn);                                           \
     ATOMIC_MMU_CLEANUP;                                             \
     atomic_trace_rmw_post(env, addr, info);                         \
include/block/aio-wait.h +2 -2
index 716d2639df..b39eefb38d 100644
@@ -80,7 +80,7 @@ extern AioWait global_aio_wait;
     AioWait *wait_ = &global_aio_wait;                             \
     AioContext *ctx_ = (ctx);                                      \
     /* Increment wait_->num_waiters before evaluating cond. */     \
-    atomic_inc(&wait_->num_waiters);                               \
+    qatomic_inc(&wait_->num_waiters);                              \
     if (ctx_ && in_aio_context_home_thread(ctx_)) {                \
         while ((cond)) {                                           \
             aio_poll(ctx_, true);                                  \
@@ -100,7 +100,7 @@ extern AioWait global_aio_wait;
             waited_ = true;                                        \
         }                                                          \
     }                                                              \
-    atomic_dec(&wait_->num_waiters);                               \
+    qatomic_dec(&wait_->num_waiters);                              \
     waited_; })
 
 /**
include/block/aio.h +4 -4
index b2f703fa3f..ec8c5af642 100644
@@ -595,7 +595,7 @@ int64_t aio_compute_timeout(AioContext *ctx);
  */
 static inline void aio_disable_external(AioContext *ctx)
 {
-    atomic_inc(&ctx->external_disable_cnt);
+    qatomic_inc(&ctx->external_disable_cnt);
 }
 
 /**
@@ -608,7 +608,7 @@ static inline void aio_enable_external(AioContext *ctx)
 {
     int old;
 
-    old = atomic_fetch_dec(&ctx->external_disable_cnt);
+    old = qatomic_fetch_dec(&ctx->external_disable_cnt);
     assert(old > 0);
     if (old == 1) {
         /* Kick event loop so it re-arms file descriptors */
@@ -624,7 +624,7 @@ static inline void aio_enable_external(AioContext *ctx)
  */
 static inline bool aio_external_disabled(AioContext *ctx)
 {
-    return atomic_read(&ctx->external_disable_cnt);
+    return qatomic_read(&ctx->external_disable_cnt);
 }
 
 /**
@@ -637,7 +637,7 @@ static inline bool aio_external_disabled(AioContext *ctx)
  */
 static inline bool aio_node_check(AioContext *ctx, bool is_external)
 {
-    return !is_external || !atomic_read(&ctx->external_disable_cnt);
+    return !is_external || !qatomic_read(&ctx->external_disable_cnt);
 }
 
 /**
include/exec/cpu_ldst.h +1 -1
index c14a48f65e..30605edab9 100644
@@ -299,7 +299,7 @@ static inline target_ulong tlb_addr_write(const CPUTLBEntry *entry)
 #if TCG_OVERSIZED_GUEST
     return entry->addr_write;
 #else
-    return atomic_read(&entry->addr_write);
+    return qatomic_read(&entry->addr_write);
 #endif
 }
 
include/exec/exec-all.h +3 -3
index 3cf88272df..1fe28d574f 100644
@@ -89,7 +89,7 @@ void QEMU_NORETURN cpu_loop_exit_atomic(CPUState *cpu, uintptr_t pc);
  */
 static inline bool cpu_loop_exit_requested(CPUState *cpu)
 {
-    return (int32_t)atomic_read(&cpu_neg(cpu)->icount_decr.u32) < 0;
+    return (int32_t)qatomic_read(&cpu_neg(cpu)->icount_decr.u32) < 0;
 }
 
 #if !defined(CONFIG_USER_ONLY)
@@ -487,10 +487,10 @@ struct TranslationBlock {
 
 extern bool parallel_cpus;
 
-/* Hide the atomic_read to make code a little easier on the eyes */
+/* Hide the qatomic_read to make code a little easier on the eyes */
 static inline uint32_t tb_cflags(const TranslationBlock *tb)
 {
-    return atomic_read(&tb->cflags);
+    return qatomic_read(&tb->cflags);
 }
 
 /* current cflags for hashing/comparison */
include/exec/log.h +3 -3
index 3ed797c1c8..86871f403a 100644
@@ -19,7 +19,7 @@ static inline void log_cpu_state(CPUState *cpu, int flags)
 
     if (qemu_log_enabled()) {
         rcu_read_lock();
-        logfile = atomic_rcu_read(&qemu_logfile);
+        logfile = qatomic_rcu_read(&qemu_logfile);
         if (logfile) {
             cpu_dump_state(cpu, logfile->fd, flags);
         }
@@ -49,7 +49,7 @@ static inline void log_target_disas(CPUState *cpu, target_ulong start,
 {
     QemuLogFile *logfile;
     rcu_read_lock();
-    logfile = atomic_rcu_read(&qemu_logfile);
+    logfile = qatomic_rcu_read(&qemu_logfile);
     if (logfile) {
         target_disas(logfile->fd, cpu, start, len);
     }
@@ -60,7 +60,7 @@ static inline void log_disas(void *code, unsigned long size, const char *note)
 {
     QemuLogFile *logfile;
     rcu_read_lock();
-    logfile = atomic_rcu_read(&qemu_logfile);
+    logfile = qatomic_rcu_read(&qemu_logfile);
     if (logfile) {
         disas(logfile->fd, code, size, note);
     }
include/exec/memory.h +1 -1
index f1bb2a7df5..06b85e3a73 100644
@@ -685,7 +685,7 @@ struct FlatView {
 
 static inline FlatView *address_space_to_flatview(AddressSpace *as)
 {
-    return atomic_rcu_read(&as->current_map);
+    return qatomic_rcu_read(&as->current_map);
 }
 
 
include/exec/ram_addr.h +14 -12
index 3ef729a23c..c6d2ef1d07 100644
@@ -164,7 +164,7 @@ static inline bool cpu_physical_memory_get_dirty(ram_addr_t start,
     page = start >> TARGET_PAGE_BITS;
 
     WITH_RCU_READ_LOCK_GUARD() {
-        blocks = atomic_rcu_read(&ram_list.dirty_memory[client]);
+        blocks = qatomic_rcu_read(&ram_list.dirty_memory[client]);
 
         idx = page / DIRTY_MEMORY_BLOCK_SIZE;
         offset = page % DIRTY_MEMORY_BLOCK_SIZE;
@@ -205,7 +205,7 @@ static inline bool cpu_physical_memory_all_dirty(ram_addr_t start,
 
     RCU_READ_LOCK_GUARD();
 
-    blocks = atomic_rcu_read(&ram_list.dirty_memory[client]);
+    blocks = qatomic_rcu_read(&ram_list.dirty_memory[client]);
 
     idx = page / DIRTY_MEMORY_BLOCK_SIZE;
     offset = page % DIRTY_MEMORY_BLOCK_SIZE;
@@ -278,7 +278,7 @@ static inline void cpu_physical_memory_set_dirty_flag(ram_addr_t addr,
 
     RCU_READ_LOCK_GUARD();
 
-    blocks = atomic_rcu_read(&ram_list.dirty_memory[client]);
+    blocks = qatomic_rcu_read(&ram_list.dirty_memory[client]);
 
     set_bit_atomic(offset, blocks->blocks[idx]);
 }
@@ -301,7 +301,7 @@ static inline void cpu_physical_memory_set_dirty_range(ram_addr_t start,
 
     WITH_RCU_READ_LOCK_GUARD() {
         for (i = 0; i < DIRTY_MEMORY_NUM; i++) {
-            blocks[i] = atomic_rcu_read(&ram_list.dirty_memory[i]);
+            blocks[i] = qatomic_rcu_read(&ram_list.dirty_memory[i]);
         }
 
         idx = page / DIRTY_MEMORY_BLOCK_SIZE;
@@ -361,23 +361,25 @@ static inline void cpu_physical_memory_set_dirty_lebitmap(unsigned long *bitmap,
 
         WITH_RCU_READ_LOCK_GUARD() {
             for (i = 0; i < DIRTY_MEMORY_NUM; i++) {
-                blocks[i] = atomic_rcu_read(&ram_list.dirty_memory[i])->blocks;
+                blocks[i] =
+                    qatomic_rcu_read(&ram_list.dirty_memory[i])->blocks;
             }
 
             for (k = 0; k < nr; k++) {
                 if (bitmap[k]) {
                     unsigned long temp = leul_to_cpu(bitmap[k]);
 
-                    atomic_or(&blocks[DIRTY_MEMORY_VGA][idx][offset], temp);
+                    qatomic_or(&blocks[DIRTY_MEMORY_VGA][idx][offset], temp);
 
                     if (global_dirty_log) {
-                        atomic_or(&blocks[DIRTY_MEMORY_MIGRATION][idx][offset],
-                                  temp);
+                        qatomic_or(
+                                &blocks[DIRTY_MEMORY_MIGRATION][idx][offset],
+                                temp);
                     }
 
                     if (tcg_enabled()) {
-                        atomic_or(&blocks[DIRTY_MEMORY_CODE][idx][offset],
-                                  temp);
+                        qatomic_or(&blocks[DIRTY_MEMORY_CODE][idx][offset],
+                                   temp);
                     }
                 }
 
@@ -461,12 +463,12 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
                                         DIRTY_MEMORY_BLOCK_SIZE);
         unsigned long page = BIT_WORD(start >> TARGET_PAGE_BITS);
 
-        src = atomic_rcu_read(
+        src = qatomic_rcu_read(
                 &ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION])->blocks;
 
         for (k = page; k < page + nr; k++) {
             if (src[idx][offset]) {
-                unsigned long bits = atomic_xchg(&src[idx][offset], 0);
+                unsigned long bits = qatomic_xchg(&src[idx][offset], 0);
                 unsigned long new_dirty;
                 new_dirty = ~dest[k];
                 dest[k] |= bits;
include/exec/ramlist.h +1 -1
index bc4faa1b00..26704aa3b0 100644
@@ -19,7 +19,7 @@ typedef struct RAMBlockNotifier RAMBlockNotifier;
  *   rcu_read_lock();
  *
  *   DirtyMemoryBlocks *blocks =
- *       atomic_rcu_read(&ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION]);
+ *       qatomic_rcu_read(&ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION]);
  *
  *   ram_addr_t idx = (addr >> TARGET_PAGE_BITS) / DIRTY_MEMORY_BLOCK_SIZE;
  *   unsigned long *block = blocks.blocks[idx];
include/exec/tb-lookup.h +2 -2
index 26921b6daf..9cf475bb03 100644
@@ -27,7 +27,7 @@ tb_lookup__cpu_state(CPUState *cpu, target_ulong *pc, target_ulong *cs_base,
 
     cpu_get_tb_cpu_state(env, pc, cs_base, flags);
     hash = tb_jmp_cache_hash_func(*pc);
-    tb = atomic_rcu_read(&cpu->tb_jmp_cache[hash]);
+    tb = qatomic_rcu_read(&cpu->tb_jmp_cache[hash]);
 
     cf_mask &= ~CF_CLUSTER_MASK;
     cf_mask |= cpu->cluster_index << CF_CLUSTER_SHIFT;
@@ -44,7 +44,7 @@ tb_lookup__cpu_state(CPUState *cpu, target_ulong *pc, target_ulong *cs_base,
     if (tb == NULL) {
         return NULL;
     }
-    atomic_set(&cpu->tb_jmp_cache[hash], tb);
+    qatomic_set(&cpu->tb_jmp_cache[hash], tb);
     return tb;
 }
 
include/hw/core/cpu.h +1 -1
index 99dc33ffeb..6c34798c8b 100644
@@ -482,7 +482,7 @@ static inline void cpu_tb_jmp_cache_clear(CPUState *cpu)
     unsigned int i;
 
     for (i = 0; i < TB_JMP_CACHE_SIZE; i++) {
-        atomic_set(&cpu->tb_jmp_cache[i], NULL);
+        qatomic_set(&cpu->tb_jmp_cache[i], NULL);
     }
 }
 
include/qemu/atomic128.h +3 -3
index 6b34484e15..ad2bcf45b4 100644
@@ -44,7 +44,7 @@
 #if defined(CONFIG_ATOMIC128)
 static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new)
 {
-    return atomic_cmpxchg__nocheck(ptr, cmp, new);
+    return qatomic_cmpxchg__nocheck(ptr, cmp, new);
 }
 # define HAVE_CMPXCHG128 1
 #elif defined(CONFIG_CMPXCHG128)
@@ -89,12 +89,12 @@ Int128 QEMU_ERROR("unsupported atomic")
 #if defined(CONFIG_ATOMIC128)
 static inline Int128 atomic16_read(Int128 *ptr)
 {
-    return atomic_read__nocheck(ptr);
+    return qatomic_read__nocheck(ptr);
 }
 
 static inline void atomic16_set(Int128 *ptr, Int128 val)
 {
-    atomic_set__nocheck(ptr, val);
+    qatomic_set__nocheck(ptr, val);
 }
 
 # define HAVE_ATOMIC128 1
include/qemu/bitops.h +1 -1
index f55ce8b320..3acbf3384c 100644
@@ -51,7 +51,7 @@ static inline void set_bit_atomic(long nr, unsigned long *addr)
     unsigned long mask = BIT_MASK(nr);
     unsigned long *p = addr + BIT_WORD(nr);
 
-    atomic_or(p, mask);
+    qatomic_or(p, mask);
 }
 
 /**
include/qemu/coroutine.h +1 -1
index dfd261c5b1..84eab6e3bf 100644
@@ -179,7 +179,7 @@ static inline coroutine_fn void qemu_co_mutex_assert_locked(CoMutex *mutex)
      * because the condition will be false no matter whether we read NULL or
      * the pointer for any other coroutine.
      */
-    assert(atomic_read(&mutex->locked) &&
+    assert(qatomic_read(&mutex->locked) &&
            mutex->holder == qemu_coroutine_self());
 }
 
include/qemu/log.h +3 -3
index f4724f7330..9b80660207 100644
@@ -36,7 +36,7 @@ static inline bool qemu_log_separate(void)
     bool res = false;
 
     rcu_read_lock();
-    logfile = atomic_rcu_read(&qemu_logfile);
+    logfile = qatomic_rcu_read(&qemu_logfile);
     if (logfile && logfile->fd != stderr) {
         res = true;
     }
@@ -75,7 +75,7 @@ static inline FILE *qemu_log_lock(void)
 {
     QemuLogFile *logfile;
     rcu_read_lock();
-    logfile = atomic_rcu_read(&qemu_logfile);
+    logfile = qatomic_rcu_read(&qemu_logfile);
     if (logfile) {
         qemu_flockfile(logfile->fd);
         return logfile->fd;
@@ -102,7 +102,7 @@ qemu_log_vprintf(const char *fmt, va_list va)
     QemuLogFile *logfile;
 
     rcu_read_lock();
-    logfile = atomic_rcu_read(&qemu_logfile);
+    logfile = qatomic_rcu_read(&qemu_logfile);
     if (logfile) {
         vfprintf(logfile->fd, fmt, va);
     }
include/qemu/queue.h +4 -3
index 456a5b01ee..e029e7bf66 100644
@@ -218,12 +218,12 @@ struct {                                                                \
         typeof(elm) save_sle_next;                                           \
         do {                                                                 \
             save_sle_next = (elm)->field.sle_next = (head)->slh_first;       \
-        } while (atomic_cmpxchg(&(head)->slh_first, save_sle_next, (elm)) != \
+        } while (qatomic_cmpxchg(&(head)->slh_first, save_sle_next, (elm)) !=\
                  save_sle_next);                                             \
 } while (/*CONSTCOND*/0)
 
 #define QSLIST_MOVE_ATOMIC(dest, src) do {                               \
-        (dest)->slh_first = atomic_xchg(&(src)->slh_first, NULL);        \
+        (dest)->slh_first = qatomic_xchg(&(src)->slh_first, NULL);       \
 } while (/*CONSTCOND*/0)
 
 #define QSLIST_REMOVE_HEAD(head, field) do {                             \
@@ -376,7 +376,8 @@ struct {                                                                \
 /*
  * Simple queue access methods.
  */
-#define QSIMPLEQ_EMPTY_ATOMIC(head) (atomic_read(&((head)->sqh_first)) == NULL)
+#define QSIMPLEQ_EMPTY_ATOMIC(head) \
+    (qatomic_read(&((head)->sqh_first)) == NULL)
 #define QSIMPLEQ_EMPTY(head)        ((head)->sqh_first == NULL)
 #define QSIMPLEQ_FIRST(head)        ((head)->sqh_first)
 #define QSIMPLEQ_NEXT(elm, field)   ((elm)->field.sqe_next)
include/qemu/rcu.h +5 -5
index 0e375ebe13..515d327cf1 100644
@@ -79,8 +79,8 @@ static inline void rcu_read_lock(void)
         return;
     }
 
-    ctr = atomic_read(&rcu_gp_ctr);
-    atomic_set(&p_rcu_reader->ctr, ctr);
+    ctr = qatomic_read(&rcu_gp_ctr);
+    qatomic_set(&p_rcu_reader->ctr, ctr);
 
     /* Write p_rcu_reader->ctr before reading RCU-protected pointers.  */
     smp_mb_placeholder();
@@ -100,12 +100,12 @@ static inline void rcu_read_unlock(void)
      * smp_mb_placeholder(), this ensures writes to p_rcu_reader->ctr
      * are sequentially consistent.
      */
-    atomic_store_release(&p_rcu_reader->ctr, 0);
+    qatomic_store_release(&p_rcu_reader->ctr, 0);
 
     /* Write p_rcu_reader->ctr before reading p_rcu_reader->waiting.  */
     smp_mb_placeholder();
-    if (unlikely(atomic_read(&p_rcu_reader->waiting))) {
-        atomic_set(&p_rcu_reader->waiting, false);
+    if (unlikely(qatomic_read(&p_rcu_reader->waiting))) {
+        qatomic_set(&p_rcu_reader->waiting, false);
         qemu_event_set(&rcu_gp_event);
     }
 }
include/qemu/rcu_queue.h +51 -49
index 558961cc27..0e53ddd530 100644
@@ -36,9 +36,9 @@ extern "C" {
 /*
  * List access methods.
  */
-#define QLIST_EMPTY_RCU(head) (atomic_read(&(head)->lh_first) == NULL)
-#define QLIST_FIRST_RCU(head) (atomic_rcu_read(&(head)->lh_first))
-#define QLIST_NEXT_RCU(elm, field) (atomic_rcu_read(&(elm)->field.le_next))
+#define QLIST_EMPTY_RCU(head) (qatomic_read(&(head)->lh_first) == NULL)
+#define QLIST_FIRST_RCU(head) (qatomic_rcu_read(&(head)->lh_first))
+#define QLIST_NEXT_RCU(elm, field) (qatomic_rcu_read(&(elm)->field.le_next))
 
 /*
  * List functions.
@@ -46,7 +46,7 @@ extern "C" {
 
 
 /*
- *  The difference between atomic_read/set and atomic_rcu_read/set
+ *  The difference between qatomic_read/set and qatomic_rcu_read/set
  *  is in the including of a read/write memory barrier to the volatile
  *  access. atomic_rcu_* macros include the memory barrier, the
  *  plain atomic macros do not. Therefore, it should be correct to
@@ -66,7 +66,7 @@ extern "C" {
 #define QLIST_INSERT_AFTER_RCU(listelm, elm, field) do {    \
     (elm)->field.le_next = (listelm)->field.le_next;        \
     (elm)->field.le_prev = &(listelm)->field.le_next;       \
-    atomic_rcu_set(&(listelm)->field.le_next, (elm));       \
+    qatomic_rcu_set(&(listelm)->field.le_next, (elm));      \
     if ((elm)->field.le_next != NULL) {                     \
        (elm)->field.le_next->field.le_prev =                \
         &(elm)->field.le_next;                              \
@@ -82,7 +82,7 @@ extern "C" {
 #define QLIST_INSERT_BEFORE_RCU(listelm, elm, field) do {   \
     (elm)->field.le_prev = (listelm)->field.le_prev;        \
     (elm)->field.le_next = (listelm);                       \
-    atomic_rcu_set((listelm)->field.le_prev, (elm));        \
+    qatomic_rcu_set((listelm)->field.le_prev, (elm));       \
     (listelm)->field.le_prev = &(elm)->field.le_next;       \
 } while (/*CONSTCOND*/0)
 
@@ -95,7 +95,7 @@ extern "C" {
 #define QLIST_INSERT_HEAD_RCU(head, elm, field) do {    \
     (elm)->field.le_prev = &(head)->lh_first;           \
     (elm)->field.le_next = (head)->lh_first;            \
-    atomic_rcu_set((&(head)->lh_first), (elm));         \
+    qatomic_rcu_set((&(head)->lh_first), (elm));        \
     if ((elm)->field.le_next != NULL) {                 \
        (elm)->field.le_next->field.le_prev =            \
         &(elm)->field.le_next;                          \
@@ -112,20 +112,20 @@ extern "C" {
        (elm)->field.le_next->field.le_prev =        \
         (elm)->field.le_prev;                       \
     }                                               \
-    atomic_set((elm)->field.le_prev, (elm)->field.le_next); \
+    qatomic_set((elm)->field.le_prev, (elm)->field.le_next); \
 } while (/*CONSTCOND*/0)
 
 /* List traversal must occur within an RCU critical section.  */
 #define QLIST_FOREACH_RCU(var, head, field)                 \
-        for ((var) = atomic_rcu_read(&(head)->lh_first);    \
+        for ((var) = qatomic_rcu_read(&(head)->lh_first);   \
                 (var);                                      \
-                (var) = atomic_rcu_read(&(var)->field.le_next))
+                (var) = qatomic_rcu_read(&(var)->field.le_next))
 
 /* List traversal must occur within an RCU critical section.  */
 #define QLIST_FOREACH_SAFE_RCU(var, head, field, next_var)           \
-    for ((var) = (atomic_rcu_read(&(head)->lh_first));               \
+    for ((var) = (qatomic_rcu_read(&(head)->lh_first));              \
       (var) &&                                                       \
-          ((next_var) = atomic_rcu_read(&(var)->field.le_next), 1);  \
+          ((next_var) = qatomic_rcu_read(&(var)->field.le_next), 1); \
            (var) = (next_var))
 
 /*
@@ -133,9 +133,10 @@ extern "C" {
  */
 
 /* Simple queue access methods */
-#define QSIMPLEQ_EMPTY_RCU(head)      (atomic_read(&(head)->sqh_first) == NULL)
-#define QSIMPLEQ_FIRST_RCU(head)       atomic_rcu_read(&(head)->sqh_first)
-#define QSIMPLEQ_NEXT_RCU(elm, field)  atomic_rcu_read(&(elm)->field.sqe_next)
+#define QSIMPLEQ_EMPTY_RCU(head) \
+    (qatomic_read(&(head)->sqh_first) == NULL)
+#define QSIMPLEQ_FIRST_RCU(head)       qatomic_rcu_read(&(head)->sqh_first)
+#define QSIMPLEQ_NEXT_RCU(elm, field)  qatomic_rcu_read(&(elm)->field.sqe_next)
 
 /* Simple queue functions */
 #define QSIMPLEQ_INSERT_HEAD_RCU(head, elm, field) do {         \
@@ -143,12 +144,12 @@ extern "C" {
     if ((elm)->field.sqe_next == NULL) {                        \
         (head)->sqh_last = &(elm)->field.sqe_next;              \
     }                                                           \
-    atomic_rcu_set(&(head)->sqh_first, (elm));                  \
+    qatomic_rcu_set(&(head)->sqh_first, (elm));                 \
 } while (/*CONSTCOND*/0)
 
 #define QSIMPLEQ_INSERT_TAIL_RCU(head, elm, field) do {    \
     (elm)->field.sqe_next = NULL;                          \
-    atomic_rcu_set((head)->sqh_last, (elm));               \
+    qatomic_rcu_set((head)->sqh_last, (elm));              \
     (head)->sqh_last = &(elm)->field.sqe_next;             \
 } while (/*CONSTCOND*/0)
 
@@ -157,11 +158,11 @@ extern "C" {
     if ((elm)->field.sqe_next == NULL) {                                \
         (head)->sqh_last = &(elm)->field.sqe_next;                      \
     }                                                                   \
-    atomic_rcu_set(&(listelm)->field.sqe_next, (elm));                  \
+    qatomic_rcu_set(&(listelm)->field.sqe_next, (elm));                 \
 } while (/*CONSTCOND*/0)
 
 #define QSIMPLEQ_REMOVE_HEAD_RCU(head, field) do {                     \
-    atomic_set(&(head)->sqh_first, (head)->sqh_first->field.sqe_next); \
+    qatomic_set(&(head)->sqh_first, (head)->sqh_first->field.sqe_next);\
     if ((head)->sqh_first == NULL) {                                   \
         (head)->sqh_last = &(head)->sqh_first;                         \
     }                                                                  \
@@ -175,7 +176,7 @@ extern "C" {
         while (curr->field.sqe_next != (elm)) {                     \
             curr = curr->field.sqe_next;                            \
         }                                                           \
-        atomic_set(&curr->field.sqe_next,                           \
+        qatomic_set(&curr->field.sqe_next,                          \
                    curr->field.sqe_next->field.sqe_next);           \
         if (curr->field.sqe_next == NULL) {                         \
             (head)->sqh_last = &(curr)->field.sqe_next;             \
@@ -184,13 +185,13 @@ extern "C" {
 } while (/*CONSTCOND*/0)
 
 #define QSIMPLEQ_FOREACH_RCU(var, head, field)                          \
-    for ((var) = atomic_rcu_read(&(head)->sqh_first);                   \
+    for ((var) = qatomic_rcu_read(&(head)->sqh_first);                  \
          (var);                                                         \
-         (var) = atomic_rcu_read(&(var)->field.sqe_next))
+         (var) = qatomic_rcu_read(&(var)->field.sqe_next))
 
 #define QSIMPLEQ_FOREACH_SAFE_RCU(var, head, field, next)                \
-    for ((var) = atomic_rcu_read(&(head)->sqh_first);                    \
-         (var) && ((next) = atomic_rcu_read(&(var)->field.sqe_next), 1); \
+    for ((var) = qatomic_rcu_read(&(head)->sqh_first);                   \
+         (var) && ((next) = qatomic_rcu_read(&(var)->field.sqe_next), 1);\
          (var) = (next))
 
 /*
@@ -198,9 +199,9 @@ extern "C" {
  */
 
 /* Tail queue access methods */
-#define QTAILQ_EMPTY_RCU(head)      (atomic_read(&(head)->tqh_first) == NULL)
-#define QTAILQ_FIRST_RCU(head)       atomic_rcu_read(&(head)->tqh_first)
-#define QTAILQ_NEXT_RCU(elm, field)  atomic_rcu_read(&(elm)->field.tqe_next)
+#define QTAILQ_EMPTY_RCU(head)      (qatomic_read(&(head)->tqh_first) == NULL)
+#define QTAILQ_FIRST_RCU(head)       qatomic_rcu_read(&(head)->tqh_first)
+#define QTAILQ_NEXT_RCU(elm, field)  qatomic_rcu_read(&(elm)->field.tqe_next)
 
 /* Tail queue functions */
 #define QTAILQ_INSERT_HEAD_RCU(head, elm, field) do {                   \
@@ -211,14 +212,14 @@ extern "C" {
     } else {                                                            \
         (head)->tqh_circ.tql_prev = &(elm)->field.tqe_circ;             \
     }                                                                   \
-    atomic_rcu_set(&(head)->tqh_first, (elm));                          \
+    qatomic_rcu_set(&(head)->tqh_first, (elm));                         \
     (elm)->field.tqe_circ.tql_prev = &(head)->tqh_circ;                 \
 } while (/*CONSTCOND*/0)
 
 #define QTAILQ_INSERT_TAIL_RCU(head, elm, field) do {                   \
     (elm)->field.tqe_next = NULL;                                       \
     (elm)->field.tqe_circ.tql_prev = (head)->tqh_circ.tql_prev;         \
-    atomic_rcu_set(&(head)->tqh_circ.tql_prev->tql_next, (elm));        \
+    qatomic_rcu_set(&(head)->tqh_circ.tql_prev->tql_next, (elm));       \
     (head)->tqh_circ.tql_prev = &(elm)->field.tqe_circ;                 \
 } while (/*CONSTCOND*/0)
 
@@ -230,14 +231,14 @@ extern "C" {
     } else {                                                            \
         (head)->tqh_circ.tql_prev = &(elm)->field.tqe_circ;             \
     }                                                                   \
-    atomic_rcu_set(&(listelm)->field.tqe_next, (elm));                  \
+    qatomic_rcu_set(&(listelm)->field.tqe_next, (elm));                 \
     (elm)->field.tqe_circ.tql_prev = &(listelm)->field.tqe_circ;        \
 } while (/*CONSTCOND*/0)
 
 #define QTAILQ_INSERT_BEFORE_RCU(listelm, elm, field) do {                \
     (elm)->field.tqe_circ.tql_prev = (listelm)->field.tqe_circ.tql_prev;  \
     (elm)->field.tqe_next = (listelm);                                    \
-    atomic_rcu_set(&(listelm)->field.tqe_circ.tql_prev->tql_next, (elm)); \
+    qatomic_rcu_set(&(listelm)->field.tqe_circ.tql_prev->tql_next, (elm));\
     (listelm)->field.tqe_circ.tql_prev = &(elm)->field.tqe_circ;          \
 } while (/*CONSTCOND*/0)
 
@@ -248,18 +249,19 @@ extern "C" {
     } else {                                                            \
         (head)->tqh_circ.tql_prev = (elm)->field.tqe_circ.tql_prev;     \
     }                                                                   \
-    atomic_set(&(elm)->field.tqe_circ.tql_prev->tql_next, (elm)->field.tqe_next); \
+    qatomic_set(&(elm)->field.tqe_circ.tql_prev->tql_next,              \
+                (elm)->field.tqe_next);                                 \
     (elm)->field.tqe_circ.tql_prev = NULL;                              \
 } while (/*CONSTCOND*/0)
 
 #define QTAILQ_FOREACH_RCU(var, head, field)                            \
-    for ((var) = atomic_rcu_read(&(head)->tqh_first);                   \
+    for ((var) = qatomic_rcu_read(&(head)->tqh_first);                  \
          (var);                                                         \
-         (var) = atomic_rcu_read(&(var)->field.tqe_next))
+         (var) = qatomic_rcu_read(&(var)->field.tqe_next))
 
 #define QTAILQ_FOREACH_SAFE_RCU(var, head, field, next)                  \
-    for ((var) = atomic_rcu_read(&(head)->tqh_first);                    \
-         (var) && ((next) = atomic_rcu_read(&(var)->field.tqe_next), 1); \
+    for ((var) = qatomic_rcu_read(&(head)->tqh_first);                   \
+         (var) && ((next) = qatomic_rcu_read(&(var)->field.tqe_next), 1);\
          (var) = (next))
 
 /*
@@ -267,23 +269,23 @@ extern "C" {
  */
 
 /* Singly-linked list access methods */
-#define QSLIST_EMPTY_RCU(head)      (atomic_read(&(head)->slh_first) == NULL)
-#define QSLIST_FIRST_RCU(head)       atomic_rcu_read(&(head)->slh_first)
-#define QSLIST_NEXT_RCU(elm, field)  atomic_rcu_read(&(elm)->field.sle_next)
+#define QSLIST_EMPTY_RCU(head)      (qatomic_read(&(head)->slh_first) == NULL)
+#define QSLIST_FIRST_RCU(head)       qatomic_rcu_read(&(head)->slh_first)
+#define QSLIST_NEXT_RCU(elm, field)  qatomic_rcu_read(&(elm)->field.sle_next)
 
 /* Singly-linked list functions */
 #define QSLIST_INSERT_HEAD_RCU(head, elm, field) do {           \
     (elm)->field.sle_next = (head)->slh_first;                  \
-    atomic_rcu_set(&(head)->slh_first, (elm));                  \
+    qatomic_rcu_set(&(head)->slh_first, (elm));                 \
 } while (/*CONSTCOND*/0)
 
 #define QSLIST_INSERT_AFTER_RCU(head, listelm, elm, field) do {         \
     (elm)->field.sle_next = (listelm)->field.sle_next;                  \
-    atomic_rcu_set(&(listelm)->field.sle_next, (elm));                  \
+    qatomic_rcu_set(&(listelm)->field.sle_next, (elm));                 \
 } while (/*CONSTCOND*/0)
 
 #define QSLIST_REMOVE_HEAD_RCU(head, field) do {                       \
-    atomic_set(&(head)->slh_first, (head)->slh_first->field.sle_next); \
+    qatomic_set(&(head)->slh_first, (head)->slh_first->field.sle_next);\
 } while (/*CONSTCOND*/0)
 
 #define QSLIST_REMOVE_RCU(head, elm, type, field) do {              \
@@ -294,19 +296,19 @@ extern "C" {
         while (curr->field.sle_next != (elm)) {                     \
             curr = curr->field.sle_next;                            \
         }                                                           \
-        atomic_set(&curr->field.sle_next,                           \
+        qatomic_set(&curr->field.sle_next,                          \
                    curr->field.sle_next->field.sle_next);           \
     }                                                               \
 } while (/*CONSTCOND*/0)
 
 #define QSLIST_FOREACH_RCU(var, head, field)                          \
-    for ((var) = atomic_rcu_read(&(head)->slh_first);                   \
-         (var);                                                         \
-         (var) = atomic_rcu_read(&(var)->field.sle_next))
+    for ((var) = qatomic_rcu_read(&(head)->slh_first);                \
+         (var);                                                       \
+         (var) = qatomic_rcu_read(&(var)->field.sle_next))
 
-#define QSLIST_FOREACH_SAFE_RCU(var, head, field, next)                \
-    for ((var) = atomic_rcu_read(&(head)->slh_first);                    \
-         (var) && ((next) = atomic_rcu_read(&(var)->field.sle_next), 1); \
+#define QSLIST_FOREACH_SAFE_RCU(var, head, field, next)                   \
+    for ((var) = qatomic_rcu_read(&(head)->slh_first);                    \
+         (var) && ((next) = qatomic_rcu_read(&(var)->field.sle_next), 1); \
          (var) = (next))
 
 #ifdef __cplusplus
include/qemu/seqlock.h +4 -4
index 8b6b4ee4bb..ecb7d2c864 100644
@@ -32,7 +32,7 @@ static inline void seqlock_init(QemuSeqLock *sl)
 /* Lock out other writers and update the count.  */
 static inline void seqlock_write_begin(QemuSeqLock *sl)
 {
-    atomic_set(&sl->sequence, sl->sequence + 1);
+    qatomic_set(&sl->sequence, sl->sequence + 1);
 
     /* Write sequence before updating other fields.  */
     smp_wmb();
@@ -43,7 +43,7 @@ static inline void seqlock_write_end(QemuSeqLock *sl)
     /* Write other fields before finalizing sequence.  */
     smp_wmb();
 
-    atomic_set(&sl->sequence, sl->sequence + 1);
+    qatomic_set(&sl->sequence, sl->sequence + 1);
 }
 
 /* Lock out other writers and update the count.  */
@@ -68,7 +68,7 @@ static inline void seqlock_write_unlock_impl(QemuSeqLock *sl, QemuLockable *lock
 static inline unsigned seqlock_read_begin(const QemuSeqLock *sl)
 {
     /* Always fail if a write is in progress.  */
-    unsigned ret = atomic_read(&sl->sequence);
+    unsigned ret = qatomic_read(&sl->sequence);
 
     /* Read sequence before reading other fields.  */
     smp_rmb();
@@ -79,7 +79,7 @@ static inline int seqlock_read_retry(const QemuSeqLock *sl, unsigned start)
 {
     /* Read other fields before reading final sequence.  */
     smp_rmb();
-    return unlikely(atomic_read(&sl->sequence) != start);
+    return unlikely(qatomic_read(&sl->sequence) != start);
 }
 
 #endif
include/qemu/stats64.h +14 -14
index 19a5ac4c56..fdd3d1b8f9 100644
@@ -37,27 +37,27 @@ static inline void stat64_init(Stat64 *s, uint64_t value)
 
 static inline uint64_t stat64_get(const Stat64 *s)
 {
-    return atomic_read__nocheck(&s->value);
+    return qatomic_read__nocheck(&s->value);
 }
 
 static inline void stat64_add(Stat64 *s, uint64_t value)
 {
-    atomic_add(&s->value, value);
+    qatomic_add(&s->value, value);
 }
 
 static inline void stat64_min(Stat64 *s, uint64_t value)
 {
-    uint64_t orig = atomic_read__nocheck(&s->value);
+    uint64_t orig = qatomic_read__nocheck(&s->value);
     while (orig > value) {
-        orig = atomic_cmpxchg__nocheck(&s->value, orig, value);
+        orig = qatomic_cmpxchg__nocheck(&s->value, orig, value);
     }
 }
 
 static inline void stat64_max(Stat64 *s, uint64_t value)
 {
-    uint64_t orig = atomic_read__nocheck(&s->value);
+    uint64_t orig = qatomic_read__nocheck(&s->value);
     while (orig < value) {
-        orig = atomic_cmpxchg__nocheck(&s->value, orig, value);
+        orig = qatomic_cmpxchg__nocheck(&s->value, orig, value);
     }
 }
 #else
@@ -79,7 +79,7 @@ static inline void stat64_add(Stat64 *s, uint64_t value)
     low = (uint32_t) value;
     if (!low) {
         if (high) {
-            atomic_add(&s->high, high);
+            qatomic_add(&s->high, high);
         }
         return;
     }
@@ -101,7 +101,7 @@ static inline void stat64_add(Stat64 *s, uint64_t value)
          * the high 32 bits, so it can race just fine with stat64_add32_carry
          * and even stat64_get!
          */
-        old = atomic_cmpxchg(&s->low, orig, result);
+        old = qatomic_cmpxchg(&s->low, orig, result);
         if (orig == old) {
             return;
         }
@@ -116,7 +116,7 @@ static inline void stat64_min(Stat64 *s, uint64_t value)
     high = value >> 32;
     low = (uint32_t) value;
     do {
-        orig_high = atomic_read(&s->high);
+        orig_high = qatomic_read(&s->high);
         if (orig_high < high) {
             return;
         }
@@ -128,7 +128,7 @@ static inline void stat64_min(Stat64 *s, uint64_t value)
              * the write barrier in stat64_min_slow.
              */
             smp_rmb();
-            orig_low = atomic_read(&s->low);
+            orig_low = qatomic_read(&s->low);
             if (orig_low <= low) {
                 return;
             }
@@ -138,7 +138,7 @@ static inline void stat64_min(Stat64 *s, uint64_t value)
              * we may miss being lucky.
              */
             smp_rmb();
-            orig_high = atomic_read(&s->high);
+            orig_high = qatomic_read(&s->high);
             if (orig_high < high) {
                 return;
             }
@@ -156,7 +156,7 @@ static inline void stat64_max(Stat64 *s, uint64_t value)
     high = value >> 32;
     low = (uint32_t) value;
     do {
-        orig_high = atomic_read(&s->high);
+        orig_high = qatomic_read(&s->high);
         if (orig_high > high) {
             return;
         }
@@ -168,7 +168,7 @@ static inline void stat64_max(Stat64 *s, uint64_t value)
              * the write barrier in stat64_max_slow.
              */
             smp_rmb();
-            orig_low = atomic_read(&s->low);
+            orig_low = qatomic_read(&s->low);
             if (orig_low >= low) {
                 return;
             }
@@ -178,7 +178,7 @@ static inline void stat64_max(Stat64 *s, uint64_t value)
              * we may miss being lucky.
              */
             smp_rmb();
-            orig_high = atomic_read(&s->high);
+            orig_high = qatomic_read(&s->high);
             if (orig_high > high) {
                 return;
             }
include/qemu/thread.h +12 -12
index 4baf4d1715..5435763184 100644
@@ -70,33 +70,33 @@ extern QemuCondTimedWaitFunc qemu_cond_timedwait_func;
             qemu_cond_timedwait_impl(c, m, ms, __FILE__, __LINE__)
 #else
 #define qemu_mutex_lock(m) ({                                           \
-            QemuMutexLockFunc _f = atomic_read(&qemu_mutex_lock_func);  \
+            QemuMutexLockFunc _f = qatomic_read(&qemu_mutex_lock_func); \
             _f(m, __FILE__, __LINE__);                                  \
         })
 
-#define qemu_mutex_trylock(m) ({                                        \
-            QemuMutexTrylockFunc _f = atomic_read(&qemu_mutex_trylock_func); \
-            _f(m, __FILE__, __LINE__);                                  \
+#define qemu_mutex_trylock(m) ({                                              \
+            QemuMutexTrylockFunc _f = qatomic_read(&qemu_mutex_trylock_func); \
+            _f(m, __FILE__, __LINE__);                                        \
         })
 
-#define qemu_rec_mutex_lock(m) ({                                       \
-            QemuRecMutexLockFunc _f = atomic_read(&qemu_rec_mutex_lock_func); \
-            _f(m, __FILE__, __LINE__);                                  \
+#define qemu_rec_mutex_lock(m) ({                                             \
+            QemuRecMutexLockFunc _f = qatomic_read(&qemu_rec_mutex_lock_func);\
+            _f(m, __FILE__, __LINE__);                                        \
         })
 
 #define qemu_rec_mutex_trylock(m) ({                            \
             QemuRecMutexTrylockFunc _f;                         \
-            _f = atomic_read(&qemu_rec_mutex_trylock_func);     \
+            _f = qatomic_read(&qemu_rec_mutex_trylock_func);    \
             _f(m, __FILE__, __LINE__);                          \
         })
 
 #define qemu_cond_wait(c, m) ({                                         \
-            QemuCondWaitFunc _f = atomic_read(&qemu_cond_wait_func);    \
+            QemuCondWaitFunc _f = qatomic_read(&qemu_cond_wait_func);   \
             _f(c, m, __FILE__, __LINE__);                               \
         })
 
 #define qemu_cond_timedwait(c, m, ms) ({                                       \
-            QemuCondTimedWaitFunc _f = atomic_read(&qemu_cond_timedwait_func); \
+            QemuCondTimedWaitFunc _f = qatomic_read(&qemu_cond_timedwait_func);\
             _f(c, m, ms, __FILE__, __LINE__);                                  \
         })
 #endif
@@ -236,7 +236,7 @@ static inline void qemu_spin_lock(QemuSpin *spin)
     __tsan_mutex_pre_lock(spin, 0);
 #endif
     while (unlikely(__sync_lock_test_and_set(&spin->value, true))) {
-        while (atomic_read(&spin->value)) {
+        while (qatomic_read(&spin->value)) {
             cpu_relax();
         }
     }
@@ -261,7 +261,7 @@ static inline bool qemu_spin_trylock(QemuSpin *spin)
 
 static inline bool qemu_spin_locked(QemuSpin *spin)
 {
-    return atomic_read(&spin->value);
+    return qatomic_read(&spin->value);
 }
 
 static inline void qemu_spin_unlock(QemuSpin *spin)
include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h +7 -7
index acd4c8346d..7b4062a1a1 100644
@@ -68,7 +68,7 @@ static inline int pvrdma_idx_valid(uint32_t idx, uint32_t max_elems)
 
 static inline int32_t pvrdma_idx(int *var, uint32_t max_elems)
 {
-	const unsigned int idx = atomic_read(var);
+	const unsigned int idx = qatomic_read(var);
 
 	if (pvrdma_idx_valid(idx, max_elems))
 		return idx & (max_elems - 1);
@@ -77,17 +77,17 @@ static inline int32_t pvrdma_idx(int *var, uint32_t max_elems)
 
 static inline void pvrdma_idx_ring_inc(int *var, uint32_t max_elems)
 {
-	uint32_t idx = atomic_read(var) + 1;	/* Increment. */
+	uint32_t idx = qatomic_read(var) + 1;	/* Increment. */
 
 	idx &= (max_elems << 1) - 1;		/* Modulo size, flip gen. */
-	atomic_set(var, idx);
+	qatomic_set(var, idx);
 }
 
 static inline int32_t pvrdma_idx_ring_has_space(const struct pvrdma_ring *r,
 					      uint32_t max_elems, uint32_t *out_tail)
 {
-	const uint32_t tail = atomic_read(&r->prod_tail);
-	const uint32_t head = atomic_read(&r->cons_head);
+	const uint32_t tail = qatomic_read(&r->prod_tail);
+	const uint32_t head = qatomic_read(&r->cons_head);
 
 	if (pvrdma_idx_valid(tail, max_elems) &&
 	    pvrdma_idx_valid(head, max_elems)) {
@@ -100,8 +100,8 @@ static inline int32_t pvrdma_idx_ring_has_space(const struct pvrdma_ring *r,
 static inline int32_t pvrdma_idx_ring_has_data(const struct pvrdma_ring *r,
 					     uint32_t max_elems, uint32_t *out_head)
 {
-	const uint32_t tail = atomic_read(&r->prod_tail);
-	const uint32_t head = atomic_read(&r->cons_head);
+	const uint32_t tail = qatomic_read(&r->prod_tail);
+	const uint32_t head = qatomic_read(&r->cons_head);
 
 	if (pvrdma_idx_valid(tail, max_elems) &&
 	    pvrdma_idx_valid(head, max_elems)) {
linux-user/qemu.h +1 -1
index a69a0bd347..941ca99722 100644
@@ -146,7 +146,7 @@ typedef struct TaskState {
     /* Nonzero if process_pending_signals() needs to do something (either
      * handle a pending signal or unblock signals).
      * This flag is written from a signal handler so should be accessed via
-     * the atomic_read() and atomic_set() functions. (It is not accessed
+     * the qatomic_read() and qatomic_set() functions. (It is not accessed
      * from multiple threads.)
      */
     int signal_pending;
tcg/i386/tcg-target.h +1 -1
index 99ac1e3958..d2baf796b0 100644
@@ -215,7 +215,7 @@ static inline void tb_target_set_jmp_target(uintptr_t tc_ptr,
                                             uintptr_t jmp_addr, uintptr_t addr)
 {
     /* patch the branch destination */
-    atomic_set((int32_t *)jmp_addr, addr - (jmp_addr + 4));
+    qatomic_set((int32_t *)jmp_addr, addr - (jmp_addr + 4));
     /* no need to flush icache explicitly */
 }
 
tcg/s390/tcg-target.h +1 -1
index 07accabbd1..63c8797bd3 100644
@@ -154,7 +154,7 @@ static inline void tb_target_set_jmp_target(uintptr_t tc_ptr,
 {
     /* patch the branch destination */
     intptr_t disp = addr - (jmp_addr - 2);
-    atomic_set((int32_t *)jmp_addr, disp / 2);
+    qatomic_set((int32_t *)jmp_addr, disp / 2);
     /* no need to flush icache explicitly */
 }
 
tcg/tci/tcg-target.h +1 -1
index 8b90ab71cb..8c1c1d265d 100644
@@ -206,7 +206,7 @@ static inline void tb_target_set_jmp_target(uintptr_t tc_ptr,
                                             uintptr_t jmp_addr, uintptr_t addr)
 {
     /* patch the branch destination */
-    atomic_set((int32_t *)jmp_addr, addr - (jmp_addr + 4));
+    qatomic_set((int32_t *)jmp_addr, addr - (jmp_addr + 4));
     /* no need to flush icache explicitly */
 }
 
accel/kvm/kvm-all.c +6 -6
index ad8b315b35..e4bbf78366 100644
@@ -2379,7 +2379,7 @@ static __thread bool have_sigbus_pending;
 
 static void kvm_cpu_kick(CPUState *cpu)
 {
-    atomic_set(&cpu->kvm_run->immediate_exit, 1);
+    qatomic_set(&cpu->kvm_run->immediate_exit, 1);
 }
 
 static void kvm_cpu_kick_self(void)
@@ -2400,7 +2400,7 @@ static void kvm_eat_signals(CPUState *cpu)
     int r;
 
     if (kvm_immediate_exit) {
-        atomic_set(&cpu->kvm_run->immediate_exit, 0);
+        qatomic_set(&cpu->kvm_run->immediate_exit, 0);
         /* Write kvm_run->immediate_exit before the cpu->exit_request
          * write in kvm_cpu_exec.
          */
@@ -2434,7 +2434,7 @@ int kvm_cpu_exec(CPUState *cpu)
     DPRINTF("kvm_cpu_exec()\n");
 
     if (kvm_arch_process_async_events(cpu)) {
-        atomic_set(&cpu->exit_request, 0);
+        qatomic_set(&cpu->exit_request, 0);
         return EXCP_HLT;
     }
 
@@ -2450,7 +2450,7 @@ int kvm_cpu_exec(CPUState *cpu)
         }
 
         kvm_arch_pre_run(cpu, run);
-        if (atomic_read(&cpu->exit_request)) {
+        if (qatomic_read(&cpu->exit_request)) {
             DPRINTF("interrupt exit requested\n");
             /*
              * KVM requires us to reenter the kernel after IO exits to complete
@@ -2577,7 +2577,7 @@ int kvm_cpu_exec(CPUState *cpu)
         vm_stop(RUN_STATE_INTERNAL_ERROR);
     }
 
-    atomic_set(&cpu->exit_request, 0);
+    qatomic_set(&cpu->exit_request, 0);
     return ret;
 }
 
@@ -2994,7 +2994,7 @@ int kvm_on_sigbus_vcpu(CPUState *cpu, int code, void *addr)
     have_sigbus_pending = true;
     pending_sigbus_addr = addr;
     pending_sigbus_code = code;
-    atomic_set(&cpu->exit_request, 1);
+    qatomic_set(&cpu->exit_request, 1);
     return 0;
 #else
     return 1;
accel/tcg/cpu-exec.c +8 -7
index 66d38f9d85..e10b46283c 100644
@@ -367,7 +367,8 @@ static inline void tb_add_jump(TranslationBlock *tb, int n,
         goto out_unlock_next;
     }
     /* Atomically claim the jump destination slot only if it was NULL */
-    old = atomic_cmpxchg(&tb->jmp_dest[n], (uintptr_t)NULL, (uintptr_t)tb_next);
+    old = qatomic_cmpxchg(&tb->jmp_dest[n], (uintptr_t)NULL,
+                          (uintptr_t)tb_next);
     if (old) {
         goto out_unlock_next;
     }
@@ -407,7 +408,7 @@ static inline TranslationBlock *tb_find(CPUState *cpu,
         tb = tb_gen_code(cpu, pc, cs_base, flags, cf_mask);
         mmap_unlock();
         /* We add the TB in the virtual pc hash table for the fast lookup */
-        atomic_set(&cpu->tb_jmp_cache[tb_jmp_cache_hash_func(pc)], tb);
+        qatomic_set(&cpu->tb_jmp_cache[tb_jmp_cache_hash_func(pc)], tb);
     }
 #ifndef CONFIG_USER_ONLY
     /* We don't take care of direct jumps when address mapping changes in
@@ -536,9 +537,9 @@ static inline bool cpu_handle_interrupt(CPUState *cpu,
      * Ensure zeroing happens before reading cpu->exit_request or
      * cpu->interrupt_request (see also smp_wmb in cpu_exit())
      */
-    atomic_mb_set(&cpu_neg(cpu)->icount_decr.u16.high, 0);
+    qatomic_mb_set(&cpu_neg(cpu)->icount_decr.u16.high, 0);
 
-    if (unlikely(atomic_read(&cpu->interrupt_request))) {
+    if (unlikely(qatomic_read(&cpu->interrupt_request))) {
         int interrupt_request;
         qemu_mutex_lock_iothread();
         interrupt_request = cpu->interrupt_request;
@@ -613,10 +614,10 @@ static inline bool cpu_handle_interrupt(CPUState *cpu,
     }
 
     /* Finally, check if we need to exit to the main loop.  */
-    if (unlikely(atomic_read(&cpu->exit_request))
+    if (unlikely(qatomic_read(&cpu->exit_request))
         || (use_icount
             && cpu_neg(cpu)->icount_decr.u16.low + cpu->icount_extra == 0)) {
-        atomic_set(&cpu->exit_request, 0);
+        qatomic_set(&cpu->exit_request, 0);
         if (cpu->exception_index == -1) {
             cpu->exception_index = EXCP_INTERRUPT;
         }
@@ -642,7 +643,7 @@ static inline void cpu_loop_exec_tb(CPUState *cpu, TranslationBlock *tb,
     }
 
     *last_tb = NULL;
-    insns_left = atomic_read(&cpu_neg(cpu)->icount_decr.u32);
+    insns_left = qatomic_read(&cpu_neg(cpu)->icount_decr.u32);
     if (insns_left < 0) {
         /* Something asked us to stop executing chained TBs; just
          * continue round the main loop. Whatever requested the exit
accel/tcg/cputlb.c +12 -12
index 6489abbf8c..aaf8e46ae5 100644
@@ -312,9 +312,9 @@ void tlb_flush_counts(size_t *pfull, size_t *ppart, size_t *pelide)
     CPU_FOREACH(cpu) {
         CPUArchState *env = cpu->env_ptr;
 
-        full += atomic_read(&env_tlb(env)->c.full_flush_count);
-        part += atomic_read(&env_tlb(env)->c.part_flush_count);
-        elide += atomic_read(&env_tlb(env)->c.elide_flush_count);
+        full += qatomic_read(&env_tlb(env)->c.full_flush_count);
+        part += qatomic_read(&env_tlb(env)->c.part_flush_count);
+        elide += qatomic_read(&env_tlb(env)->c.elide_flush_count);
     }
     *pfull = full;
     *ppart = part;
@@ -349,13 +349,13 @@ static void tlb_flush_by_mmuidx_async_work(CPUState *cpu, run_on_cpu_data data)
     cpu_tb_jmp_cache_clear(cpu);
 
     if (to_clean == ALL_MMUIDX_BITS) {
-        atomic_set(&env_tlb(env)->c.full_flush_count,
+        qatomic_set(&env_tlb(env)->c.full_flush_count,
                    env_tlb(env)->c.full_flush_count + 1);
     } else {
-        atomic_set(&env_tlb(env)->c.part_flush_count,
+        qatomic_set(&env_tlb(env)->c.part_flush_count,
                    env_tlb(env)->c.part_flush_count + ctpop16(to_clean));
         if (to_clean != asked) {
-            atomic_set(&env_tlb(env)->c.elide_flush_count,
+            qatomic_set(&env_tlb(env)->c.elide_flush_count,
                        env_tlb(env)->c.elide_flush_count +
                        ctpop16(asked & ~to_clean));
         }
@@ -693,7 +693,7 @@ void tlb_unprotect_code(ram_addr_t ram_addr)
  * generated code.
  *
  * Other vCPUs might be reading their TLBs during guest execution, so we update
- * te->addr_write with atomic_set. We don't need to worry about this for
+ * te->addr_write with qatomic_set. We don't need to worry about this for
  * oversized guests as MTTCG is disabled for them.
  *
  * Called with tlb_c.lock held.
@@ -711,7 +711,7 @@ static void tlb_reset_dirty_range_locked(CPUTLBEntry *tlb_entry,
 #if TCG_OVERSIZED_GUEST
             tlb_entry->addr_write |= TLB_NOTDIRTY;
 #else
-            atomic_set(&tlb_entry->addr_write,
+            qatomic_set(&tlb_entry->addr_write,
                        tlb_entry->addr_write | TLB_NOTDIRTY);
 #endif
         }
@@ -1138,8 +1138,8 @@ static inline target_ulong tlb_read_ofs(CPUTLBEntry *entry, size_t ofs)
 #if TCG_OVERSIZED_GUEST
     return *(target_ulong *)((uintptr_t)entry + ofs);
 #else
-    /* ofs might correspond to .addr_write, so use atomic_read */
-    return atomic_read((target_ulong *)((uintptr_t)entry + ofs));
+    /* ofs might correspond to .addr_write, so use qatomic_read */
+    return qatomic_read((target_ulong *)((uintptr_t)entry + ofs));
 #endif
 }
 
@@ -1155,11 +1155,11 @@ static bool victim_tlb_hit(CPUArchState *env, size_t mmu_idx, size_t index,
         CPUTLBEntry *vtlb = &env_tlb(env)->d[mmu_idx].vtable[vidx];
         target_ulong cmp;
 
-        /* elt_ofs might correspond to .addr_write, so use atomic_read */
+        /* elt_ofs might correspond to .addr_write, so use qatomic_read */
 #if TCG_OVERSIZED_GUEST
         cmp = *(target_ulong *)((uintptr_t)vtlb + elt_ofs);
 #else
-        cmp = atomic_read((target_ulong *)((uintptr_t)vtlb + elt_ofs));
+        cmp = qatomic_read((target_ulong *)((uintptr_t)vtlb + elt_ofs));
 #endif
 
         if (cmp == page) {
accel/tcg/tcg-all.c +1 -1
index 7098ad96c3..1c664924d7 100644
@@ -65,7 +65,7 @@ static void tcg_handle_interrupt(CPUState *cpu, int mask)
     if (!qemu_cpu_is_self(cpu)) {
         qemu_cpu_kick(cpu);
     } else {
-        atomic_set(&cpu_neg(cpu)->icount_decr.u16.high, -1);
+        qatomic_set(&cpu_neg(cpu)->icount_decr.u16.high, -1);
         if (use_icount &&
             !cpu->can_do_io
             && (mask & ~old_mask) != 0) {
accel/tcg/translate-all.c +28 -27
index 2d83013633..f44ba9d46d 100644
@@ -377,9 +377,9 @@ static int cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
     restore_state_to_opc(env, tb, data);
 
 #ifdef CONFIG_PROFILER
-    atomic_set(&prof->restore_time,
+    qatomic_set(&prof->restore_time,
                 prof->restore_time + profile_getclock() - ti);
-    atomic_set(&prof->restore_count, prof->restore_count + 1);
+    qatomic_set(&prof->restore_count, prof->restore_count + 1);
 #endif
     return 0;
 }
@@ -509,7 +509,7 @@ static PageDesc *page_find_alloc(tb_page_addr_t index, int alloc)
 
     /* Level 2..N-1.  */
     for (i = v_l2_levels; i > 0; i--) {
-        void **p = atomic_rcu_read(lp);
+        void **p = qatomic_rcu_read(lp);
 
         if (p == NULL) {
             void *existing;
@@ -518,7 +518,7 @@ static PageDesc *page_find_alloc(tb_page_addr_t index, int alloc)
                 return NULL;
             }
             p = g_new0(void *, V_L2_SIZE);
-            existing = atomic_cmpxchg(lp, NULL, p);
+            existing = qatomic_cmpxchg(lp, NULL, p);
             if (unlikely(existing)) {
                 g_free(p);
                 p = existing;
@@ -528,7 +528,7 @@ static PageDesc *page_find_alloc(tb_page_addr_t index, int alloc)
         lp = p + ((index >> (i * V_L2_BITS)) & (V_L2_SIZE - 1));
     }
 
-    pd = atomic_rcu_read(lp);
+    pd = qatomic_rcu_read(lp);
     if (pd == NULL) {
         void *existing;
 
@@ -545,7 +545,7 @@ static PageDesc *page_find_alloc(tb_page_addr_t index, int alloc)
             }
         }
 #endif
-        existing = atomic_cmpxchg(lp, NULL, pd);
+        existing = qatomic_cmpxchg(lp, NULL, pd);
         if (unlikely(existing)) {
 #ifndef CONFIG_USER_ONLY
             {
@@ -1253,7 +1253,7 @@ static void do_tb_flush(CPUState *cpu, run_on_cpu_data tb_flush_count)
     tcg_region_reset_all();
     /* XXX: flush processor icache at this point if cache flush is
        expensive */
-    atomic_mb_set(&tb_ctx.tb_flush_count, tb_ctx.tb_flush_count + 1);
+    qatomic_mb_set(&tb_ctx.tb_flush_count, tb_ctx.tb_flush_count + 1);
 
 done:
     mmap_unlock();
@@ -1265,7 +1265,7 @@ done:
 void tb_flush(CPUState *cpu)
 {
     if (tcg_enabled()) {
-        unsigned tb_flush_count = atomic_mb_read(&tb_ctx.tb_flush_count);
+        unsigned tb_flush_count = qatomic_mb_read(&tb_ctx.tb_flush_count);
 
         if (cpu_in_exclusive_context(cpu)) {
             do_tb_flush(cpu, RUN_ON_CPU_HOST_INT(tb_flush_count));
@@ -1358,7 +1358,7 @@ static inline void tb_remove_from_jmp_list(TranslationBlock *orig, int n_orig)
     int n;
 
     /* mark the LSB of jmp_dest[] so that no further jumps can be inserted */
-    ptr = atomic_or_fetch(&orig->jmp_dest[n_orig], 1);
+    ptr = qatomic_or_fetch(&orig->jmp_dest[n_orig], 1);
     dest = (TranslationBlock *)(ptr & ~1);
     if (dest == NULL) {
         return;
@@ -1369,7 +1369,7 @@ static inline void tb_remove_from_jmp_list(TranslationBlock *orig, int n_orig)
      * While acquiring the lock, the jump might have been removed if the
      * destination TB was invalidated; check again.
      */
-    ptr_locked = atomic_read(&orig->jmp_dest[n_orig]);
+    ptr_locked = qatomic_read(&orig->jmp_dest[n_orig]);
     if (ptr_locked != ptr) {
         qemu_spin_unlock(&dest->jmp_lock);
         /*
@@ -1415,7 +1415,7 @@ static inline void tb_jmp_unlink(TranslationBlock *dest)
 
     TB_FOR_EACH_JMP(dest, tb, n) {
         tb_reset_jump(tb, n);
-        atomic_and(&tb->jmp_dest[n], (uintptr_t)NULL | 1);
+        qatomic_and(&tb->jmp_dest[n], (uintptr_t)NULL | 1);
         /* No need to clear the list entry; setting the dest ptr is enough */
     }
     dest->jmp_list_head = (uintptr_t)NULL;
@@ -1439,7 +1439,7 @@ static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
 
     /* make sure no further incoming jumps will be chained to this TB */
     qemu_spin_lock(&tb->jmp_lock);
-    atomic_set(&tb->cflags, tb->cflags | CF_INVALID);
+    qatomic_set(&tb->cflags, tb->cflags | CF_INVALID);
     qemu_spin_unlock(&tb->jmp_lock);
 
     /* remove the TB from the hash list */
@@ -1466,8 +1466,8 @@ static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
     /* remove the TB from the hash list */
     h = tb_jmp_cache_hash_func(tb->pc);
     CPU_FOREACH(cpu) {
-        if (atomic_read(&cpu->tb_jmp_cache[h]) == tb) {
-            atomic_set(&cpu->tb_jmp_cache[h], NULL);
+        if (qatomic_read(&cpu->tb_jmp_cache[h]) == tb) {
+            qatomic_set(&cpu->tb_jmp_cache[h], NULL);
         }
     }
 
@@ -1478,7 +1478,7 @@ static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
     /* suppress any remaining jumps to this TB */
     tb_jmp_unlink(tb);
 
-    atomic_set(&tcg_ctx->tb_phys_invalidate_count,
+    qatomic_set(&tcg_ctx->tb_phys_invalidate_count,
                tcg_ctx->tb_phys_invalidate_count + 1);
 }
 
@@ -1733,7 +1733,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
 
 #ifdef CONFIG_PROFILER
     /* includes aborted translations because of exceptions */
-    atomic_set(&prof->tb_count1, prof->tb_count1 + 1);
+    qatomic_set(&prof->tb_count1, prof->tb_count1 + 1);
     ti = profile_getclock();
 #endif
 
@@ -1758,8 +1758,9 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
     }
 
 #ifdef CONFIG_PROFILER
-    atomic_set(&prof->tb_count, prof->tb_count + 1);
-    atomic_set(&prof->interm_time, prof->interm_time + profile_getclock() - ti);
+    qatomic_set(&prof->tb_count, prof->tb_count + 1);
+    qatomic_set(&prof->interm_time,
+                prof->interm_time + profile_getclock() - ti);
     ti = profile_getclock();
 #endif
 
@@ -1804,10 +1805,10 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
     tb->tc.size = gen_code_size;
 
 #ifdef CONFIG_PROFILER
-    atomic_set(&prof->code_time, prof->code_time + profile_getclock() - ti);
-    atomic_set(&prof->code_in_len, prof->code_in_len + tb->size);
-    atomic_set(&prof->code_out_len, prof->code_out_len + gen_code_size);
-    atomic_set(&prof->search_out_len, prof->search_out_len + search_size);
+    qatomic_set(&prof->code_time, prof->code_time + profile_getclock() - ti);
+    qatomic_set(&prof->code_in_len, prof->code_in_len + tb->size);
+    qatomic_set(&prof->code_out_len, prof->code_out_len + gen_code_size);
+    qatomic_set(&prof->search_out_len, prof->search_out_len + search_size);
 #endif
 
 #ifdef DEBUG_DISAS
@@ -1869,7 +1870,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
     }
 #endif
 
-    atomic_set(&tcg_ctx->code_gen_ptr, (void *)
+    qatomic_set(&tcg_ctx->code_gen_ptr, (void *)
         ROUND_UP((uintptr_t)gen_code_buf + gen_code_size + search_size,
                  CODE_GEN_ALIGN));
 
@@ -1905,7 +1906,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
         uintptr_t orig_aligned = (uintptr_t)gen_code_buf;
 
         orig_aligned -= ROUND_UP(sizeof(*tb), qemu_icache_linesize);
-        atomic_set(&tcg_ctx->code_gen_ptr, (void *)orig_aligned);
+        qatomic_set(&tcg_ctx->code_gen_ptr, (void *)orig_aligned);
         tb_destroy(tb);
         return existing_tb;
     }
@@ -2273,7 +2274,7 @@ static void tb_jmp_cache_clear_page(CPUState *cpu, target_ulong page_addr)
     unsigned int i, i0 = tb_jmp_cache_hash_page(page_addr);
 
     for (i = 0; i < TB_JMP_PAGE_SIZE; i++) {
-        atomic_set(&cpu->tb_jmp_cache[i0 + i], NULL);
+        qatomic_set(&cpu->tb_jmp_cache[i0 + i], NULL);
     }
 }
 
@@ -2393,7 +2394,7 @@ void dump_exec_info(void)
 
     qemu_printf("\nStatistics:\n");
     qemu_printf("TB flush count      %u\n",
-                atomic_read(&tb_ctx.tb_flush_count));
+                qatomic_read(&tb_ctx.tb_flush_count));
     qemu_printf("TB invalidate count %zu\n",
                 tcg_tb_phys_invalidate_count());
 
@@ -2415,7 +2416,7 @@ void cpu_interrupt(CPUState *cpu, int mask)
 {
     g_assert(qemu_mutex_iothread_locked());
     cpu->interrupt_request |= mask;
-    atomic_set(&cpu_neg(cpu)->icount_decr.u16.high, -1);
+    qatomic_set(&cpu_neg(cpu)->icount_decr.u16.high, -1);
 }
 
 /*
audio/jackaudio.c +9 -9
index 72ed7c4929..1e714b30bc 100644
@@ -104,7 +104,7 @@ static void qjack_buffer_create(QJackBuffer *buffer, int channels, int frames)
 static void qjack_buffer_clear(QJackBuffer *buffer)
 {
     assert(buffer->data);
-    atomic_store_release(&buffer->used, 0);
+    qatomic_store_release(&buffer->used, 0);
     buffer->rptr = 0;
     buffer->wptr = 0;
 }
@@ -129,7 +129,7 @@ static int qjack_buffer_write(QJackBuffer *buffer, float *data, int size)
     assert(buffer->data);
     const int samples = size / sizeof(float);
     int frames        = samples / buffer->channels;
-    const int avail   = buffer->frames - atomic_load_acquire(&buffer->used);
+    const int avail   = buffer->frames - qatomic_load_acquire(&buffer->used);
 
     if (frames > avail) {
         frames = avail;
@@ -153,7 +153,7 @@ static int qjack_buffer_write(QJackBuffer *buffer, float *data, int size)
 
     buffer->wptr = wptr;
 
-    atomic_add(&buffer->used, frames);
+    qatomic_add(&buffer->used, frames);
     return frames * buffer->channels * sizeof(float);
 };
 
@@ -161,7 +161,7 @@ static int qjack_buffer_write(QJackBuffer *buffer, float *data, int size)
 static int qjack_buffer_write_l(QJackBuffer *buffer, float **dest, int frames)
 {
     assert(buffer->data);
-    const int avail   = buffer->frames - atomic_load_acquire(&buffer->used);
+    const int avail   = buffer->frames - qatomic_load_acquire(&buffer->used);
     int wptr = buffer->wptr;
 
     if (frames > avail) {
@@ -185,7 +185,7 @@ static int qjack_buffer_write_l(QJackBuffer *buffer, float **dest, int frames)
     }
     buffer->wptr = wptr;
 
-    atomic_add(&buffer->used, frames);
+    qatomic_add(&buffer->used, frames);
     return frames;
 }
 
@@ -195,7 +195,7 @@ static int qjack_buffer_read(QJackBuffer *buffer, float *dest, int size)
     assert(buffer->data);
     const int samples = size / sizeof(float);
     int frames        = samples / buffer->channels;
-    const int avail   = atomic_load_acquire(&buffer->used);
+    const int avail   = qatomic_load_acquire(&buffer->used);
 
     if (frames > avail) {
         frames = avail;
@@ -219,7 +219,7 @@ static int qjack_buffer_read(QJackBuffer *buffer, float *dest, int size)
 
     buffer->rptr = rptr;
 
-    atomic_sub(&buffer->used, frames);
+    qatomic_sub(&buffer->used, frames);
     return frames * buffer->channels * sizeof(float);
 }
 
@@ -228,7 +228,7 @@ static int qjack_buffer_read_l(QJackBuffer *buffer, float **dest, int frames)
 {
     assert(buffer->data);
     int copy       = frames;
-    const int used = atomic_load_acquire(&buffer->used);
+    const int used = qatomic_load_acquire(&buffer->used);
     int rptr       = buffer->rptr;
 
     if (copy > used) {
@@ -252,7 +252,7 @@ static int qjack_buffer_read_l(QJackBuffer *buffer, float **dest, int frames)
     }
     buffer->rptr = rptr;
 
-    atomic_sub(&buffer->used, copy);
+    qatomic_sub(&buffer->used, copy);
     return copy;
 }
 
block.c +2 -2
index 11ab55f80b..f72a2e26e8 100644
@@ -1694,7 +1694,7 @@ static int bdrv_open_common(BlockDriverState *bs, BlockBackend *file,
     }
 
     /* bdrv_new() and bdrv_close() make it so */
-    assert(atomic_read(&bs->copy_on_read) == 0);
+    assert(qatomic_read(&bs->copy_on_read) == 0);
 
     if (bs->open_flags & BDRV_O_COPY_ON_READ) {
         if (!bs->read_only) {
@@ -4436,7 +4436,7 @@ static void bdrv_close(BlockDriverState *bs)
     bs->file = NULL;
     g_free(bs->opaque);
     bs->opaque = NULL;
-    atomic_set(&bs->copy_on_read, 0);
+    qatomic_set(&bs->copy_on_read, 0);
     bs->backing_file[0] = '\0';
     bs->backing_format[0] = '\0';
     bs->total_sectors = 0;
block/block-backend.c +8 -7
index 24dd0670d1..ce78d30794 100644
@@ -1353,12 +1353,12 @@ int blk_make_zero(BlockBackend *blk, BdrvRequestFlags flags)
 
 void blk_inc_in_flight(BlockBackend *blk)
 {
-    atomic_inc(&blk->in_flight);
+    qatomic_inc(&blk->in_flight);
 }
 
 void blk_dec_in_flight(BlockBackend *blk)
 {
-    atomic_dec(&blk->in_flight);
+    qatomic_dec(&blk->in_flight);
     aio_wait_kick();
 }
 
@@ -1720,7 +1720,7 @@ void blk_drain(BlockBackend *blk)
 
     /* We may have -ENOMEDIUM completions in flight */
     AIO_WAIT_WHILE(blk_get_aio_context(blk),
-                   atomic_mb_read(&blk->in_flight) > 0);
+                   qatomic_mb_read(&blk->in_flight) > 0);
 
     if (bs) {
         bdrv_drained_end(bs);
@@ -1739,7 +1739,7 @@ void blk_drain_all(void)
         aio_context_acquire(ctx);
 
         /* We may have -ENOMEDIUM completions in flight */
-        AIO_WAIT_WHILE(ctx, atomic_mb_read(&blk->in_flight) > 0);
+        AIO_WAIT_WHILE(ctx, qatomic_mb_read(&blk->in_flight) > 0);
 
         aio_context_release(ctx);
     }
@@ -2346,6 +2346,7 @@ void blk_io_limits_update_group(BlockBackend *blk, const char *group)
 static void blk_root_drained_begin(BdrvChild *child)
 {
     BlockBackend *blk = child->opaque;
+    ThrottleGroupMember *tgm = &blk->public.throttle_group_member;
 
     if (++blk->quiesce_counter == 1) {
         if (blk->dev_ops && blk->dev_ops->drained_begin) {
@@ -2356,8 +2357,8 @@ static void blk_root_drained_begin(BdrvChild *child)
     /* Note that blk->root may not be accessible here yet if we are just
      * attaching to a BlockDriverState that is drained. Use child instead. */
 
-    if (atomic_fetch_inc(&blk->public.throttle_group_member.io_limits_disabled) == 0) {
-        throttle_group_restart_tgm(&blk->public.throttle_group_member);
+    if (qatomic_fetch_inc(&tgm->io_limits_disabled) == 0) {
+        throttle_group_restart_tgm(tgm);
     }
 }
 
@@ -2374,7 +2375,7 @@ static void blk_root_drained_end(BdrvChild *child, int *drained_end_counter)
     assert(blk->quiesce_counter);
 
     assert(blk->public.throttle_group_member.io_limits_disabled);
-    atomic_dec(&blk->public.throttle_group_member.io_limits_disabled);
+    qatomic_dec(&blk->public.throttle_group_member.io_limits_disabled);
 
     if (--blk->quiesce_counter == 0) {
         if (blk->dev_ops && blk->dev_ops->drained_end) {
block/io.c +24 -24
index a2389bb38c..11df1889f1 100644
@@ -69,7 +69,7 @@ void bdrv_parent_drained_end_single(BdrvChild *c)
 {
     int drained_end_counter = 0;
     bdrv_parent_drained_end_single_no_poll(c, &drained_end_counter);
-    BDRV_POLL_WHILE(c->bs, atomic_read(&drained_end_counter) > 0);
+    BDRV_POLL_WHILE(c->bs, qatomic_read(&drained_end_counter) > 0);
 }
 
 static void bdrv_parent_drained_end(BlockDriverState *bs, BdrvChild *ignore,
@@ -186,12 +186,12 @@ void bdrv_refresh_limits(BlockDriverState *bs, Error **errp)
  */
 void bdrv_enable_copy_on_read(BlockDriverState *bs)
 {
-    atomic_inc(&bs->copy_on_read);
+    qatomic_inc(&bs->copy_on_read);
 }
 
 void bdrv_disable_copy_on_read(BlockDriverState *bs)
 {
-    int old = atomic_fetch_dec(&bs->copy_on_read);
+    int old = qatomic_fetch_dec(&bs->copy_on_read);
     assert(old >= 1);
 }
 
@@ -219,9 +219,9 @@ static void coroutine_fn bdrv_drain_invoke_entry(void *opaque)
     }
 
     /* Set data->done and decrement drained_end_counter before bdrv_wakeup() */
-    atomic_mb_set(&data->done, true);
+    qatomic_mb_set(&data->done, true);
     if (!data->begin) {
-        atomic_dec(data->drained_end_counter);
+        qatomic_dec(data->drained_end_counter);
     }
     bdrv_dec_in_flight(bs);
 
@@ -248,7 +248,7 @@ static void bdrv_drain_invoke(BlockDriverState *bs, bool begin,
     };
 
     if (!begin) {
-        atomic_inc(drained_end_counter);
+        qatomic_inc(drained_end_counter);
     }
 
     /* Make sure the driver callback completes during the polling phase for
@@ -268,7 +268,7 @@ bool bdrv_drain_poll(BlockDriverState *bs, bool recursive,
         return true;
     }
 
-    if (atomic_read(&bs->in_flight)) {
+    if (qatomic_read(&bs->in_flight)) {
         return true;
     }
 
@@ -382,7 +382,7 @@ void bdrv_do_drained_begin_quiesce(BlockDriverState *bs,
     assert(!qemu_in_coroutine());
 
     /* Stop things in parent-to-child order */
-    if (atomic_fetch_inc(&bs->quiesce_counter) == 0) {
+    if (qatomic_fetch_inc(&bs->quiesce_counter) == 0) {
         aio_disable_external(bdrv_get_aio_context(bs));
     }
 
@@ -473,7 +473,7 @@ static void bdrv_do_drained_end(BlockDriverState *bs, bool recursive,
     bdrv_parent_drained_end(bs, parent, ignore_bds_parents,
                             drained_end_counter);
 
-    old_quiesce_counter = atomic_fetch_dec(&bs->quiesce_counter);
+    old_quiesce_counter = qatomic_fetch_dec(&bs->quiesce_counter);
     if (old_quiesce_counter == 1) {
         aio_enable_external(bdrv_get_aio_context(bs));
     }
@@ -492,7 +492,7 @@ void bdrv_drained_end(BlockDriverState *bs)
 {
     int drained_end_counter = 0;
     bdrv_do_drained_end(bs, false, NULL, false, &drained_end_counter);
-    BDRV_POLL_WHILE(bs, atomic_read(&drained_end_counter) > 0);
+    BDRV_POLL_WHILE(bs, qatomic_read(&drained_end_counter) > 0);
 }
 
 void bdrv_drained_end_no_poll(BlockDriverState *bs, int *drained_end_counter)
@@ -504,7 +504,7 @@ void bdrv_subtree_drained_end(BlockDriverState *bs)
 {
     int drained_end_counter = 0;
     bdrv_do_drained_end(bs, true, NULL, false, &drained_end_counter);
-    BDRV_POLL_WHILE(bs, atomic_read(&drained_end_counter) > 0);
+    BDRV_POLL_WHILE(bs, qatomic_read(&drained_end_counter) > 0);
 }
 
 void bdrv_apply_subtree_drain(BdrvChild *child, BlockDriverState *new_parent)
@@ -526,7 +526,7 @@ void bdrv_unapply_subtree_drain(BdrvChild *child, BlockDriverState *old_parent)
                             &drained_end_counter);
     }
 
-    BDRV_POLL_WHILE(child->bs, atomic_read(&drained_end_counter) > 0);
+    BDRV_POLL_WHILE(child->bs, qatomic_read(&drained_end_counter) > 0);
 }
 
 /*
@@ -553,7 +553,7 @@ static void bdrv_drain_assert_idle(BlockDriverState *bs)
 {
     BdrvChild *child, *next;
 
-    assert(atomic_read(&bs->in_flight) == 0);
+    assert(qatomic_read(&bs->in_flight) == 0);
     QLIST_FOREACH_SAFE(child, &bs->children, next, next) {
         bdrv_drain_assert_idle(child->bs);
     }
@@ -655,7 +655,7 @@ void bdrv_drain_all_end(void)
     }
 
     assert(qemu_get_current_aio_context() == qemu_get_aio_context());
-    AIO_WAIT_WHILE(NULL, atomic_read(&drained_end_counter) > 0);
+    AIO_WAIT_WHILE(NULL, qatomic_read(&drained_end_counter) > 0);
 
     assert(bdrv_drain_all_count > 0);
     bdrv_drain_all_count--;
@@ -675,7 +675,7 @@ void bdrv_drain_all(void)
 static void tracked_request_end(BdrvTrackedRequest *req)
 {
     if (req->serialising) {
-        atomic_dec(&req->bs->serialising_in_flight);
+        qatomic_dec(&req->bs->serialising_in_flight);
     }
 
     qemu_co_mutex_lock(&req->bs->reqs_lock);
@@ -777,7 +777,7 @@ bool bdrv_mark_request_serialising(BdrvTrackedRequest *req, uint64_t align)
 
     qemu_co_mutex_lock(&bs->reqs_lock);
     if (!req->serialising) {
-        atomic_inc(&req->bs->serialising_in_flight);
+        qatomic_inc(&req->bs->serialising_in_flight);
         req->serialising = true;
     }
 
@@ -841,7 +841,7 @@ static int bdrv_get_cluster_size(BlockDriverState *bs)
 
 void bdrv_inc_in_flight(BlockDriverState *bs)
 {
-    atomic_inc(&bs->in_flight);
+    qatomic_inc(&bs->in_flight);
 }
 
 void bdrv_wakeup(BlockDriverState *bs)
@@ -851,7 +851,7 @@ void bdrv_wakeup(BlockDriverState *bs)
 
 void bdrv_dec_in_flight(BlockDriverState *bs)
 {
-    atomic_dec(&bs->in_flight);
+    qatomic_dec(&bs->in_flight);
     bdrv_wakeup(bs);
 }
 
@@ -860,7 +860,7 @@ static bool coroutine_fn bdrv_wait_serialising_requests(BdrvTrackedRequest *self
     BlockDriverState *bs = self->bs;
     bool waited = false;
 
-    if (!atomic_read(&bs->serialising_in_flight)) {
+    if (!qatomic_read(&bs->serialising_in_flight)) {
         return false;
     }
 
@@ -1747,7 +1747,7 @@ int coroutine_fn bdrv_co_preadv_part(BdrvChild *child,
     bdrv_inc_in_flight(bs);
 
     /* Don't do copy-on-read if we read data before write operation */
-    if (atomic_read(&bs->copy_on_read)) {
+    if (qatomic_read(&bs->copy_on_read)) {
         flags |= BDRV_REQ_COPY_ON_READ;
     }
 
@@ -1935,7 +1935,7 @@ bdrv_co_write_req_finish(BdrvChild *child, int64_t offset, uint64_t bytes,
     int64_t end_sector = DIV_ROUND_UP(offset + bytes, BDRV_SECTOR_SIZE);
     BlockDriverState *bs = child->bs;
 
-    atomic_inc(&bs->write_gen);
+    qatomic_inc(&bs->write_gen);
 
     /*
      * Discard cannot extend the image, but in error handling cases, such as
@@ -2768,7 +2768,7 @@ int coroutine_fn bdrv_co_flush(BlockDriverState *bs)
     }
 
     qemu_co_mutex_lock(&bs->reqs_lock);
-    current_gen = atomic_read(&bs->write_gen);
+    current_gen = qatomic_read(&bs->write_gen);
 
     /* Wait until any previous flushes are completed */
     while (bs->active_flush_req) {
@@ -3116,7 +3116,7 @@ void bdrv_io_plug(BlockDriverState *bs)
         bdrv_io_plug(child->bs);
     }
 
-    if (atomic_fetch_inc(&bs->io_plugged) == 0) {
+    if (qatomic_fetch_inc(&bs->io_plugged) == 0) {
         BlockDriver *drv = bs->drv;
         if (drv && drv->bdrv_io_plug) {
             drv->bdrv_io_plug(bs);
@@ -3129,7 +3129,7 @@ void bdrv_io_unplug(BlockDriverState *bs)
     BdrvChild *child;
 
     assert(bs->io_plugged);
-    if (atomic_fetch_dec(&bs->io_plugged) == 1) {
+    if (qatomic_fetch_dec(&bs->io_plugged) == 1) {
         BlockDriver *drv = bs->drv;
         if (drv && drv->bdrv_io_unplug) {
             drv->bdrv_io_unplug(bs);
block/nfs.c +1 -1
index 61a249a9fc..f86e660374 100644
@@ -721,7 +721,7 @@ nfs_get_allocated_file_size_cb(int ret, struct nfs_context *nfs, void *data,
     }
 
     /* Set task->complete before reading bs->wakeup.  */
-    atomic_mb_set(&task->complete, 1);
+    qatomic_mb_set(&task->complete, 1);
     bdrv_wakeup(task->bs);
 }
 
block/sheepdog.c +1 -1
index cbbebc1aaf..2f5c0eb376 100644
@@ -665,7 +665,7 @@ out:
     srco->co = NULL;
     srco->ret = ret;
     /* Set srco->finished before reading bs->wakeup.  */
-    atomic_mb_set(&srco->finished, true);
+    qatomic_mb_set(&srco->finished, true);
     if (srco->bs) {
         bdrv_wakeup(srco->bs);
     }
block/throttle-groups.c +6 -6
index 4e28365d8d..e2f2813c0f 100644
@@ -228,7 +228,7 @@ static ThrottleGroupMember *next_throttle_token(ThrottleGroupMember *tgm,
      * immediately if it has pending requests. Otherwise we could be
      * forcing it to wait for other member's throttled requests. */
     if (tgm_has_pending_reqs(tgm, is_write) &&
-        atomic_read(&tgm->io_limits_disabled)) {
+        qatomic_read(&tgm->io_limits_disabled)) {
         return tgm;
     }
 
@@ -272,7 +272,7 @@ static bool throttle_group_schedule_timer(ThrottleGroupMember *tgm,
     ThrottleTimers *tt = &tgm->throttle_timers;
     bool must_wait;
 
-    if (atomic_read(&tgm->io_limits_disabled)) {
+    if (qatomic_read(&tgm->io_limits_disabled)) {
         return false;
     }
 
@@ -417,7 +417,7 @@ static void coroutine_fn throttle_group_restart_queue_entry(void *opaque)
 
     g_free(data);
 
-    atomic_dec(&tgm->restart_pending);
+    qatomic_dec(&tgm->restart_pending);
     aio_wait_kick();
 }
 
@@ -434,7 +434,7 @@ static void throttle_group_restart_queue(ThrottleGroupMember *tgm, bool is_write
      * be no timer pending on this tgm at this point */
     assert(!timer_pending(tgm->throttle_timers.timers[is_write]));
 
-    atomic_inc(&tgm->restart_pending);
+    qatomic_inc(&tgm->restart_pending);
 
     co = qemu_coroutine_create(throttle_group_restart_queue_entry, rd);
     aio_co_enter(tgm->aio_context, co);
@@ -544,7 +544,7 @@ void throttle_group_register_tgm(ThrottleGroupMember *tgm,
 
     tgm->throttle_state = ts;
     tgm->aio_context = ctx;
-    atomic_set(&tgm->restart_pending, 0);
+    qatomic_set(&tgm->restart_pending, 0);
 
     qemu_mutex_lock(&tg->lock);
     /* If the ThrottleGroup is new set this ThrottleGroupMember as the token */
@@ -592,7 +592,7 @@ void throttle_group_unregister_tgm(ThrottleGroupMember *tgm)
     }
 
     /* Wait for throttle_group_restart_queue_entry() coroutines to finish */
-    AIO_WAIT_WHILE(tgm->aio_context, atomic_read(&tgm->restart_pending) > 0);
+    AIO_WAIT_WHILE(tgm->aio_context, qatomic_read(&tgm->restart_pending) > 0);
 
     qemu_mutex_lock(&tg->lock);
     for (i = 0; i < 2; i++) {
block/throttle.c +2 -2
index 9a0f38149a..b685166ad4 100644
@@ -217,7 +217,7 @@ static void throttle_reopen_abort(BDRVReopenState *reopen_state)
 static void coroutine_fn throttle_co_drain_begin(BlockDriverState *bs)
 {
     ThrottleGroupMember *tgm = bs->opaque;
-    if (atomic_fetch_inc(&tgm->io_limits_disabled) == 0) {
+    if (qatomic_fetch_inc(&tgm->io_limits_disabled) == 0) {
         throttle_group_restart_tgm(tgm);
     }
 }
@@ -226,7 +226,7 @@ static void coroutine_fn throttle_co_drain_end(BlockDriverState *bs)
 {
     ThrottleGroupMember *tgm = bs->opaque;
     assert(tgm->io_limits_disabled);
-    atomic_dec(&tgm->io_limits_disabled);
+    qatomic_dec(&tgm->io_limits_disabled);
 }
 
 static const char *const throttle_strong_runtime_opts[] = {
blockdev.c +1 -1
index 7f2561081e..bebd3ba1c3 100644
@@ -1604,7 +1604,7 @@ static void external_snapshot_commit(BlkActionState *common)
     /* We don't need (or want) to use the transactional
      * bdrv_reopen_multiple() across all the entries at once, because we
      * don't want to abort all of them if one of them fails the reopen */
-    if (!atomic_read(&state->old_bs->copy_on_read)) {
+    if (!qatomic_read(&state->old_bs->copy_on_read)) {
         bdrv_reopen_set_read_only(state->old_bs, true, NULL);
     }
 
blockjob.c +1 -1
index 470facfd47..98ac8af982 100644
@@ -298,7 +298,7 @@ BlockJobInfo *block_job_query(BlockJob *job, Error **errp)
     info = g_new0(BlockJobInfo, 1);
     info->type      = g_strdup(job_type_str(&job->job));
     info->device    = g_strdup(job->job.id);
-    info->busy      = atomic_read(&job->job.busy);
+    info->busy      = qatomic_read(&job->job.busy);
     info->paused    = job->job.pause_count > 0;
     info->offset    = job->job.progress.current;
     info->len       = job->job.progress.total;
contrib/libvhost-user/libvhost-user.c +1 -1
index 9d30ff2283..05861b1419 100644
@@ -448,7 +448,7 @@ static void
 vu_log_page(uint8_t *log_table, uint64_t page)
 {
     DPRINT("Logged dirty guest page: %"PRId64"\n", page);
-    atomic_or(&log_table[page / 8], 1 << (page % 8));
+    qatomic_or(&log_table[page / 8], 1 << (page % 8));
 }
 
 static void
cpus-common.c +13 -13
index 34044f4e4c..83475efff7 100644
@@ -148,7 +148,7 @@ void do_run_on_cpu(CPUState *cpu, run_on_cpu_func func, run_on_cpu_data data,
     wi.exclusive = false;
 
     queue_work_on_cpu(cpu, &wi);
-    while (!atomic_mb_read(&wi.done)) {
+    while (!qatomic_mb_read(&wi.done)) {
         CPUState *self_cpu = current_cpu;
 
         qemu_cond_wait(&qemu_work_cond, mutex);
@@ -188,20 +188,20 @@ void start_exclusive(void)
     exclusive_idle();
 
     /* Make all other cpus stop executing.  */
-    atomic_set(&pending_cpus, 1);
+    qatomic_set(&pending_cpus, 1);
 
     /* Write pending_cpus before reading other_cpu->running.  */
     smp_mb();
     running_cpus = 0;
     CPU_FOREACH(other_cpu) {
-        if (atomic_read(&other_cpu->running)) {
+        if (qatomic_read(&other_cpu->running)) {
             other_cpu->has_waiter = true;
             running_cpus++;
             qemu_cpu_kick(other_cpu);
         }
     }
 
-    atomic_set(&pending_cpus, running_cpus + 1);
+    qatomic_set(&pending_cpus, running_cpus + 1);
     while (pending_cpus > 1) {
         qemu_cond_wait(&exclusive_cond, &qemu_cpu_list_lock);
     }
@@ -220,7 +220,7 @@ void end_exclusive(void)
     current_cpu->in_exclusive_context = false;
 
     qemu_mutex_lock(&qemu_cpu_list_lock);
-    atomic_set(&pending_cpus, 0);
+    qatomic_set(&pending_cpus, 0);
     qemu_cond_broadcast(&exclusive_resume);
     qemu_mutex_unlock(&qemu_cpu_list_lock);
 }
@@ -228,7 +228,7 @@ void end_exclusive(void)
 /* Wait for exclusive ops to finish, and begin cpu execution.  */
 void cpu_exec_start(CPUState *cpu)
 {
-    atomic_set(&cpu->running, true);
+    qatomic_set(&cpu->running, true);
 
     /* Write cpu->running before reading pending_cpus.  */
     smp_mb();
@@ -246,17 +246,17 @@ void cpu_exec_start(CPUState *cpu)
      * 3. pending_cpus == 0.  Then start_exclusive is definitely going to
      * see cpu->running == true, and it will kick the CPU.
      */
-    if (unlikely(atomic_read(&pending_cpus))) {
+    if (unlikely(qatomic_read(&pending_cpus))) {
         QEMU_LOCK_GUARD(&qemu_cpu_list_lock);
         if (!cpu->has_waiter) {
             /* Not counted in pending_cpus, let the exclusive item
              * run.  Since we have the lock, just set cpu->running to true
              * while holding it; no need to check pending_cpus again.
              */
-            atomic_set(&cpu->running, false);
+            qatomic_set(&cpu->running, false);
             exclusive_idle();
             /* Now pending_cpus is zero.  */
-            atomic_set(&cpu->running, true);
+            qatomic_set(&cpu->running, true);
         } else {
             /* Counted in pending_cpus, go ahead and release the
              * waiter at cpu_exec_end.
@@ -268,7 +268,7 @@ void cpu_exec_start(CPUState *cpu)
 /* Mark cpu as not executing, and release pending exclusive ops.  */
 void cpu_exec_end(CPUState *cpu)
 {
-    atomic_set(&cpu->running, false);
+    qatomic_set(&cpu->running, false);
 
     /* Write cpu->running before reading pending_cpus.  */
     smp_mb();
@@ -288,11 +288,11 @@ void cpu_exec_end(CPUState *cpu)
      * see cpu->running == false, and it can ignore this CPU until the
      * next cpu_exec_start.
      */
-    if (unlikely(atomic_read(&pending_cpus))) {
+    if (unlikely(qatomic_read(&pending_cpus))) {
         QEMU_LOCK_GUARD(&qemu_cpu_list_lock);
         if (cpu->has_waiter) {
             cpu->has_waiter = false;
-            atomic_set(&pending_cpus, pending_cpus - 1);
+            qatomic_set(&pending_cpus, pending_cpus - 1);
             if (pending_cpus == 1) {
                 qemu_cond_signal(&exclusive_cond);
             }
@@ -346,7 +346,7 @@ void process_queued_cpu_work(CPUState *cpu)
         if (wi->free) {
             g_free(wi);
         } else {
-            atomic_mb_set(&wi->done, true);
+            qatomic_mb_set(&wi->done, true);
         }
     }
     qemu_mutex_unlock(&cpu->work_mutex);
dump/dump.c +4 -4
index 13fda440a4..45da46a952 100644
@@ -1572,7 +1572,7 @@ static void dump_state_prepare(DumpState *s)
 bool dump_in_progress(void)
 {
     DumpState *state = &dump_state_global;
-    return (atomic_read(&state->status) == DUMP_STATUS_ACTIVE);
+    return (qatomic_read(&state->status) == DUMP_STATUS_ACTIVE);
 }
 
 /* calculate total size of memory to be dumped (taking filter into
@@ -1882,7 +1882,7 @@ static void dump_process(DumpState *s, Error **errp)
 
     /* make sure status is written after written_size updates */
     smp_wmb();
-    atomic_set(&s->status,
+    qatomic_set(&s->status,
                (local_err ? DUMP_STATUS_FAILED : DUMP_STATUS_COMPLETED));
 
     /* send DUMP_COMPLETED message (unconditionally) */
@@ -1908,7 +1908,7 @@ DumpQueryResult *qmp_query_dump(Error **errp)
 {
     DumpQueryResult *result = g_new(DumpQueryResult, 1);
     DumpState *state = &dump_state_global;
-    result->status = atomic_read(&state->status);
+    result->status = qatomic_read(&state->status);
     /* make sure we are reading status and written_size in order */
     smp_rmb();
     result->completed = state->written_size;
@@ -2013,7 +2013,7 @@ void qmp_dump_guest_memory(bool paging, const char *file,
               begin, length, &local_err);
     if (local_err) {
         error_propagate(errp, local_err);
-        atomic_set(&s->status, DUMP_STATUS_FAILED);
+        qatomic_set(&s->status, DUMP_STATUS_FAILED);
         return;
     }
 
exec.c +26 -23
index e34b602bdf..3f5deb35ff 100644
@@ -353,13 +353,13 @@ static MemoryRegionSection *address_space_lookup_region(AddressSpaceDispatch *d,
                                                         hwaddr addr,
                                                         bool resolve_subpage)
 {
-    MemoryRegionSection *section = atomic_read(&d->mru_section);
+    MemoryRegionSection *section = qatomic_read(&d->mru_section);
     subpage_t *subpage;
 
     if (!section || section == &d->map.sections[PHYS_SECTION_UNASSIGNED] ||
         !section_covers_addr(section, addr)) {
         section = phys_page_find(d, addr);
-        atomic_set(&d->mru_section, section);
+        qatomic_set(&d->mru_section, section);
     }
     if (resolve_subpage && section->mr->subpage) {
         subpage = container_of(section->mr, subpage_t, iomem);
@@ -695,7 +695,8 @@ address_space_translate_for_iotlb(CPUState *cpu, int asidx, hwaddr addr,
     IOMMUMemoryRegionClass *imrc;
     IOMMUTLBEntry iotlb;
     int iommu_idx;
-    AddressSpaceDispatch *d = atomic_rcu_read(&cpu->cpu_ases[asidx].memory_dispatch);
+    AddressSpaceDispatch *d =
+        qatomic_rcu_read(&cpu->cpu_ases[asidx].memory_dispatch);
 
     for (;;) {
         section = address_space_translate_internal(d, addr, &addr, plen, false);
@@ -1247,7 +1248,7 @@ static RAMBlock *qemu_get_ram_block(ram_addr_t addr)
 {
     RAMBlock *block;
 
-    block = atomic_rcu_read(&ram_list.mru_block);
+    block = qatomic_rcu_read(&ram_list.mru_block);
     if (block && addr - block->offset < block->max_length) {
         return block;
     }
@@ -1273,7 +1274,7 @@ found:
      *                                        call_rcu(reclaim_ramblock, xxx);
      *                  rcu_read_unlock()
      *
-     * atomic_rcu_set is not needed here.  The block was already published
+     * qatomic_rcu_set is not needed here.  The block was already published
      * when it was placed into the list.  Here we're just making an extra
      * copy of the pointer.
      */
@@ -1321,7 +1322,7 @@ bool cpu_physical_memory_test_and_clear_dirty(ram_addr_t start,
     page = start_page;
 
     WITH_RCU_READ_LOCK_GUARD() {
-        blocks = atomic_rcu_read(&ram_list.dirty_memory[client]);
+        blocks = qatomic_rcu_read(&ram_list.dirty_memory[client]);
         ramblock = qemu_get_ram_block(start);
         /* Range sanity check on the ramblock */
         assert(start >= ramblock->offset &&
@@ -1371,7 +1372,7 @@ DirtyBitmapSnapshot *cpu_physical_memory_snapshot_and_clear_dirty
     dest = 0;
 
     WITH_RCU_READ_LOCK_GUARD() {
-        blocks = atomic_rcu_read(&ram_list.dirty_memory[client]);
+        blocks = qatomic_rcu_read(&ram_list.dirty_memory[client]);
 
         while (page < end) {
             unsigned long idx = page / DIRTY_MEMORY_BLOCK_SIZE;
@@ -2207,7 +2208,7 @@ static void dirty_memory_extend(ram_addr_t old_ram_size,
         DirtyMemoryBlocks *new_blocks;
         int j;
 
-        old_blocks = atomic_rcu_read(&ram_list.dirty_memory[i]);
+        old_blocks = qatomic_rcu_read(&ram_list.dirty_memory[i]);
         new_blocks = g_malloc(sizeof(*new_blocks) +
                               sizeof(new_blocks->blocks[0]) * new_num_blocks);
 
@@ -2220,7 +2221,7 @@ static void dirty_memory_extend(ram_addr_t old_ram_size,
             new_blocks->blocks[j] = bitmap_new(DIRTY_MEMORY_BLOCK_SIZE);
         }
 
-        atomic_rcu_set(&ram_list.dirty_memory[i], new_blocks);
+        qatomic_rcu_set(&ram_list.dirty_memory[i], new_blocks);
 
         if (old_blocks) {
             g_free_rcu(old_blocks, rcu);
@@ -2667,7 +2668,7 @@ RAMBlock *qemu_ram_block_from_host(void *ptr, bool round_offset,
     }
 
     RCU_READ_LOCK_GUARD();
-    block = atomic_rcu_read(&ram_list.mru_block);
+    block = qatomic_rcu_read(&ram_list.mru_block);
     if (block && block->host && host - block->host < block->max_length) {
         goto found;
     }
@@ -2912,7 +2913,7 @@ MemoryRegionSection *iotlb_to_section(CPUState *cpu,
 {
     int asidx = cpu_asidx_from_attrs(cpu, attrs);
     CPUAddressSpace *cpuas = &cpu->cpu_ases[asidx];
-    AddressSpaceDispatch *d = atomic_rcu_read(&cpuas->memory_dispatch);
+    AddressSpaceDispatch *d = qatomic_rcu_read(&cpuas->memory_dispatch);
     MemoryRegionSection *sections = d->map.sections;
 
     return &sections[index & ~TARGET_PAGE_MASK];
@@ -2996,7 +2997,7 @@ static void tcg_commit(MemoryListener *listener)
      * may have split the RCU critical section.
      */
     d = address_space_to_dispatch(cpuas->as);
-    atomic_rcu_set(&cpuas->memory_dispatch, d);
+    qatomic_rcu_set(&cpuas->memory_dispatch, d);
     tlb_flush(cpuas->cpu);
 }
 
@@ -3443,7 +3444,7 @@ void cpu_register_map_client(QEMUBH *bh)
     qemu_mutex_lock(&map_client_list_lock);
     client->bh = bh;
     QLIST_INSERT_HEAD(&map_client_list, client, link);
-    if (!atomic_read(&bounce.in_use)) {
+    if (!qatomic_read(&bounce.in_use)) {
         cpu_notify_map_clients_locked();
     }
     qemu_mutex_unlock(&map_client_list_lock);
@@ -3577,7 +3578,7 @@ void *address_space_map(AddressSpace *as,
     mr = flatview_translate(fv, addr, &xlat, &l, is_write, attrs);
 
     if (!memory_access_is_direct(mr, is_write)) {
-        if (atomic_xchg(&bounce.in_use, true)) {
+        if (qatomic_xchg(&bounce.in_use, true)) {
             *plen = 0;
             return NULL;
         }
@@ -3636,7 +3637,7 @@ void address_space_unmap(AddressSpace *as, void *buffer, hwaddr len,
     qemu_vfree(bounce.buffer);
     bounce.buffer = NULL;
     memory_region_unref(bounce.mr);
-    atomic_mb_set(&bounce.in_use, false);
+    qatomic_mb_set(&bounce.in_use, false);
     cpu_notify_map_clients();
 }
 
@@ -4105,16 +4106,17 @@ int ram_block_discard_disable(bool state)
     int old;
 
     if (!state) {
-        atomic_dec(&ram_block_discard_disabled);
+        qatomic_dec(&ram_block_discard_disabled);
         return 0;
     }
 
     do {
-        old = atomic_read(&ram_block_discard_disabled);
+        old = qatomic_read(&ram_block_discard_disabled);
         if (old < 0) {
             return -EBUSY;
         }
-    } while (atomic_cmpxchg(&ram_block_discard_disabled, old, old + 1) != old);
+    } while (qatomic_cmpxchg(&ram_block_discard_disabled,
+                             old, old + 1) != old);
     return 0;
 }
 
@@ -4123,27 +4125,28 @@ int ram_block_discard_require(bool state)
     int old;
 
     if (!state) {
-        atomic_inc(&ram_block_discard_disabled);
+        qatomic_inc(&ram_block_discard_disabled);
         return 0;
     }
 
     do {
-        old = atomic_read(&ram_block_discard_disabled);
+        old = qatomic_read(&ram_block_discard_disabled);
         if (old > 0) {
             return -EBUSY;
         }
-    } while (atomic_cmpxchg(&ram_block_discard_disabled, old, old - 1) != old);
+    } while (qatomic_cmpxchg(&ram_block_discard_disabled,
+                             old, old - 1) != old);
     return 0;
 }
 
 bool ram_block_discard_is_disabled(void)
 {
-    return atomic_read(&ram_block_discard_disabled) > 0;
+    return qatomic_read(&ram_block_discard_disabled) > 0;
 }
 
 bool ram_block_discard_is_required(void)
 {
-    return atomic_read(&ram_block_discard_disabled) < 0;
+    return qatomic_read(&ram_block_discard_disabled) < 0;
 }
 
 #endif
hw/core/cpu.c +3 -3
index 8f65383ffb..c55c09f734 100644
@@ -111,10 +111,10 @@ void cpu_reset_interrupt(CPUState *cpu, int mask)
 
 void cpu_exit(CPUState *cpu)
 {
-    atomic_set(&cpu->exit_request, 1);
+    qatomic_set(&cpu->exit_request, 1);
     /* Ensure cpu_exec will see the exit request after TCG has exited.  */
     smp_wmb();
-    atomic_set(&cpu->icount_decr_ptr->u16.high, -1);
+    qatomic_set(&cpu->icount_decr_ptr->u16.high, -1);
 }
 
 int cpu_write_elf32_qemunote(WriteCoreDumpFunction f, CPUState *cpu,
@@ -261,7 +261,7 @@ static void cpu_common_reset(DeviceState *dev)
     cpu->halted = cpu->start_powered_off;
     cpu->mem_io_pc = 0;
     cpu->icount_extra = 0;
-    atomic_set(&cpu->icount_decr_ptr->u32, 0);
+    qatomic_set(&cpu->icount_decr_ptr->u32, 0);
     cpu->can_do_io = 1;
     cpu->exception_index = -1;
     cpu->crash_occurred = false;
hw/display/qxl.c +2 -2
index 11871340e7..431c107096 100644
@@ -1908,7 +1908,7 @@ static void qxl_send_events(PCIQXLDevice *d, uint32_t events)
     /*
      * Older versions of Spice forgot to define the QXLRam struct
      * with the '__aligned__(4)' attribute. clang 7 and newer will
-     * thus warn that atomic_fetch_or(&d->ram->int_pending, ...)
+     * thus warn that qatomic_fetch_or(&d->ram->int_pending, ...)
      * might be a misaligned atomic access, and will generate an
      * out-of-line call for it, which results in a link error since
      * we don't currently link against libatomic.
@@ -1928,7 +1928,7 @@ static void qxl_send_events(PCIQXLDevice *d, uint32_t events)
 #define ALIGNED_UINT32_PTR(P) ((uint32_t *)P)
 #endif
 
-    old_pending = atomic_fetch_or(ALIGNED_UINT32_PTR(&d->ram->int_pending),
+    old_pending = qatomic_fetch_or(ALIGNED_UINT32_PTR(&d->ram->int_pending),
                                   le_events);
     if ((old_pending & le_events) == le_events) {
         return;
hw/hyperv/hyperv.c +5 -5
index 4b26db1365..cb1074f234 100644
@@ -231,7 +231,7 @@ static void sint_msg_bh(void *opaque)
     HvSintRoute *sint_route = opaque;
     HvSintStagedMessage *staged_msg = sint_route->staged_msg;
 
-    if (atomic_read(&staged_msg->state) != HV_STAGED_MSG_POSTED) {
+    if (qatomic_read(&staged_msg->state) != HV_STAGED_MSG_POSTED) {
         /* status nor ready yet (spurious ack from guest?), ignore */
         return;
     }
@@ -240,7 +240,7 @@ static void sint_msg_bh(void *opaque)
     staged_msg->status = 0;
 
     /* staged message processing finished, ready to start over */
-    atomic_set(&staged_msg->state, HV_STAGED_MSG_FREE);
+    qatomic_set(&staged_msg->state, HV_STAGED_MSG_FREE);
     /* drop the reference taken in hyperv_post_msg */
     hyperv_sint_route_unref(sint_route);
 }
@@ -278,7 +278,7 @@ static void cpu_post_msg(CPUState *cs, run_on_cpu_data data)
     memory_region_set_dirty(&synic->msg_page_mr, 0, sizeof(*synic->msg_page));
 
 posted:
-    atomic_set(&staged_msg->state, HV_STAGED_MSG_POSTED);
+    qatomic_set(&staged_msg->state, HV_STAGED_MSG_POSTED);
     /*
      * Notify the msg originator of the progress made; if the slot was busy we
      * set msg_pending flag in it so it will be the guest who will do EOM and
@@ -301,7 +301,7 @@ int hyperv_post_msg(HvSintRoute *sint_route, struct hyperv_message *src_msg)
     assert(staged_msg);
 
     /* grab the staging area */
-    if (atomic_cmpxchg(&staged_msg->state, HV_STAGED_MSG_FREE,
+    if (qatomic_cmpxchg(&staged_msg->state, HV_STAGED_MSG_FREE,
                        HV_STAGED_MSG_BUSY) != HV_STAGED_MSG_FREE) {
         return -EAGAIN;
     }
@@ -351,7 +351,7 @@ int hyperv_set_event_flag(HvSintRoute *sint_route, unsigned eventno)
     set_mask = BIT_MASK(eventno);
     flags = synic->event_page->slot[sint_route->sint].flags;
 
-    if ((atomic_fetch_or(&flags[set_idx], set_mask) & set_mask) != set_mask) {
+    if ((qatomic_fetch_or(&flags[set_idx], set_mask) & set_mask) != set_mask) {
         memory_region_set_dirty(&synic->event_page_mr, 0,
                                 sizeof(*synic->event_page));
         ret = hyperv_sint_route_set_sint(sint_route);
hw/hyperv/vmbus.c +1 -1
index 6ef895bc35..896e981f85 100644
@@ -747,7 +747,7 @@ static int vmbus_channel_notify_guest(VMBusChannel *chan)
 
     idx = BIT_WORD(chan->id);
     mask = BIT_MASK(chan->id);
-    if ((atomic_fetch_or(&int_map[idx], mask) & mask) != mask) {
+    if ((qatomic_fetch_or(&int_map[idx], mask) & mask) != mask) {
         res = hyperv_sint_route_set_sint(chan->notify_route);
         dirty = len;
     }
hw/i386/xen/xen-hvm.c +1 -1
index cde981bad6..a39a648ca6 100644
@@ -1140,7 +1140,7 @@ static int handle_buffered_iopage(XenIOState *state)
         assert(req.dir == IOREQ_WRITE);
         assert(!req.data_is_ptr);
 
-        atomic_add(&buf_page->read_pointer, qw + 1);
+        qatomic_add(&buf_page->read_pointer, qw + 1);
     }
 
     return req.count;
hw/intc/rx_icu.c +6 -6
index df4b6a8d22..94e17a9dea 100644
@@ -81,8 +81,8 @@ static void rxicu_request(RXICUState *icu, int n_IRQ)
     int enable;
 
     enable = icu->ier[n_IRQ / 8] & (1 << (n_IRQ & 7));
-    if (n_IRQ > 0 && enable != 0 && atomic_read(&icu->req_irq) < 0) {
-        atomic_set(&icu->req_irq, n_IRQ);
+    if (n_IRQ > 0 && enable != 0 && qatomic_read(&icu->req_irq) < 0) {
+        qatomic_set(&icu->req_irq, n_IRQ);
         set_irq(icu, n_IRQ, rxicu_level(icu, n_IRQ));
     }
 }
@@ -124,10 +124,10 @@ static void rxicu_set_irq(void *opaque, int n_IRQ, int level)
     }
     if (issue == 0 && src->sense == TRG_LEVEL) {
         icu->ir[n_IRQ] = 0;
-        if (atomic_read(&icu->req_irq) == n_IRQ) {
+        if (qatomic_read(&icu->req_irq) == n_IRQ) {
             /* clear request */
             set_irq(icu, n_IRQ, 0);
-            atomic_set(&icu->req_irq, -1);
+            qatomic_set(&icu->req_irq, -1);
         }
         return;
     }
@@ -144,11 +144,11 @@ static void rxicu_ack_irq(void *opaque, int no, int level)
     int n_IRQ;
     int max_pri;
 
-    n_IRQ = atomic_read(&icu->req_irq);
+    n_IRQ = qatomic_read(&icu->req_irq);
     if (n_IRQ < 0) {
         return;
     }
-    atomic_set(&icu->req_irq, -1);
+    qatomic_set(&icu->req_irq, -1);
     if (icu->src[n_IRQ].sense != TRG_LEVEL) {
         icu->ir[n_IRQ] = 0;
     }
hw/intc/sifive_plic.c +2 -2
index af611f8db8..f42fd695d8 100644
@@ -89,12 +89,12 @@ static void sifive_plic_print_state(SiFivePLICState *plic)
 
 static uint32_t atomic_set_masked(uint32_t *a, uint32_t mask, uint32_t value)
 {
-    uint32_t old, new, cmp = atomic_read(a);
+    uint32_t old, new, cmp = qatomic_read(a);
 
     do {
         old = cmp;
         new = (old & ~mask) | (value & mask);
-        cmp = atomic_cmpxchg(a, old, new);
+        cmp = qatomic_cmpxchg(a, old, new);
     } while (old != cmp);
 
     return old;
hw/misc/edu.c +8 -8
index 0ff9d1ac78..e935c418d4 100644
@@ -212,7 +212,7 @@ static uint64_t edu_mmio_read(void *opaque, hwaddr addr, unsigned size)
         qemu_mutex_unlock(&edu->thr_mutex);
         break;
     case 0x20:
-        val = atomic_read(&edu->status);
+        val = qatomic_read(&edu->status);
         break;
     case 0x24:
         val = edu->irq_status;
@@ -252,7 +252,7 @@ static void edu_mmio_write(void *opaque, hwaddr addr, uint64_t val,
         edu->addr4 = ~val;
         break;
     case 0x08:
-        if (atomic_read(&edu->status) & EDU_STATUS_COMPUTING) {
+        if (qatomic_read(&edu->status) & EDU_STATUS_COMPUTING) {
             break;
         }
         /* EDU_STATUS_COMPUTING cannot go 0->1 concurrently, because it is only
@@ -260,15 +260,15 @@ static void edu_mmio_write(void *opaque, hwaddr addr, uint64_t val,
          */
         qemu_mutex_lock(&edu->thr_mutex);
         edu->fact = val;
-        atomic_or(&edu->status, EDU_STATUS_COMPUTING);
+        qatomic_or(&edu->status, EDU_STATUS_COMPUTING);
         qemu_cond_signal(&edu->thr_cond);
         qemu_mutex_unlock(&edu->thr_mutex);
         break;
     case 0x20:
         if (val & EDU_STATUS_IRQFACT) {
-            atomic_or(&edu->status, EDU_STATUS_IRQFACT);
+            qatomic_or(&edu->status, EDU_STATUS_IRQFACT);
         } else {
-            atomic_and(&edu->status, ~EDU_STATUS_IRQFACT);
+            qatomic_and(&edu->status, ~EDU_STATUS_IRQFACT);
         }
         break;
     case 0x60:
@@ -322,7 +322,7 @@ static void *edu_fact_thread(void *opaque)
         uint32_t val, ret = 1;
 
         qemu_mutex_lock(&edu->thr_mutex);
-        while ((atomic_read(&edu->status) & EDU_STATUS_COMPUTING) == 0 &&
+        while ((qatomic_read(&edu->status) & EDU_STATUS_COMPUTING) == 0 &&
                         !edu->stopping) {
             qemu_cond_wait(&edu->thr_cond, &edu->thr_mutex);
         }
@@ -347,9 +347,9 @@ static void *edu_fact_thread(void *opaque)
         qemu_mutex_lock(&edu->thr_mutex);
         edu->fact = ret;
         qemu_mutex_unlock(&edu->thr_mutex);
-        atomic_and(&edu->status, ~EDU_STATUS_COMPUTING);
+        qatomic_and(&edu->status, ~EDU_STATUS_COMPUTING);
 
-        if (atomic_read(&edu->status) & EDU_STATUS_IRQFACT) {
+        if (qatomic_read(&edu->status) & EDU_STATUS_IRQFACT) {
             qemu_mutex_lock_iothread();
             edu_raise_irq(edu, FACT_IRQ);
             qemu_mutex_unlock_iothread();
hw/net/virtio-net.c +5 -5
index cb0d27084c..7bf27b9db7 100644
@@ -933,7 +933,7 @@ static void virtio_net_set_features(VirtIODevice *vdev, uint64_t features)
 
     if (virtio_has_feature(features, VIRTIO_NET_F_STANDBY)) {
         qapi_event_send_failover_negotiated(n->netclient_name);
-        atomic_set(&n->primary_should_be_hidden, false);
+        qatomic_set(&n->primary_should_be_hidden, false);
         failover_add_primary(n, &err);
         if (err) {
             n->primary_dev = virtio_connect_failover_devices(n, n->qdev, &err);
@@ -3168,7 +3168,7 @@ static void virtio_net_handle_migration_primary(VirtIONet *n,
     bool should_be_hidden;
     Error *err = NULL;
 
-    should_be_hidden = atomic_read(&n->primary_should_be_hidden);
+    should_be_hidden = qatomic_read(&n->primary_should_be_hidden);
 
     if (!n->primary_dev) {
         n->primary_dev = virtio_connect_failover_devices(n, n->qdev, &err);
@@ -3183,7 +3183,7 @@ static void virtio_net_handle_migration_primary(VirtIONet *n,
                     qdev_get_vmsd(n->primary_dev),
                     n->primary_dev);
             qapi_event_send_unplug_primary(n->primary_device_id);
-            atomic_set(&n->primary_should_be_hidden, true);
+            qatomic_set(&n->primary_should_be_hidden, true);
         } else {
             warn_report("couldn't unplug primary device");
         }
@@ -3234,7 +3234,7 @@ static int virtio_net_primary_should_be_hidden(DeviceListener *listener,
     n->primary_device_opts = device_opts;
 
     /* primary_should_be_hidden is set during feature negotiation */
-    hide = atomic_read(&n->primary_should_be_hidden);
+    hide = qatomic_read(&n->primary_should_be_hidden);
 
     if (n->primary_device_dict) {
         g_free(n->primary_device_id);
@@ -3291,7 +3291,7 @@ static void virtio_net_device_realize(DeviceState *dev, Error **errp)
     if (n->failover) {
         n->primary_listener.should_be_hidden =
             virtio_net_primary_should_be_hidden;
-        atomic_set(&n->primary_should_be_hidden, true);
+        qatomic_set(&n->primary_should_be_hidden, true);
         device_listener_register(&n->primary_listener);
         n->migration_state.notify = virtio_net_migration_state_notifier;
         add_migration_state_change_notifier(&n->migration_state);
hw/rdma/rdma_backend.c +9 -9
index db7e5c8be5..5de010b1fa 100644
@@ -68,7 +68,7 @@ static void free_cqe_ctx(gpointer data, gpointer user_data)
     bctx = rdma_rm_get_cqe_ctx(rdma_dev_res, cqe_ctx_id);
     if (bctx) {
         rdma_rm_dealloc_cqe_ctx(rdma_dev_res, cqe_ctx_id);
-        atomic_dec(&rdma_dev_res->stats.missing_cqe);
+        qatomic_dec(&rdma_dev_res->stats.missing_cqe);
     }
     g_free(bctx);
 }
@@ -81,7 +81,7 @@ static void clean_recv_mads(RdmaBackendDev *backend_dev)
         cqe_ctx_id = rdma_protected_qlist_pop_int64(&backend_dev->
                                                     recv_mads_list);
         if (cqe_ctx_id != -ENOENT) {
-            atomic_inc(&backend_dev->rdma_dev_res->stats.missing_cqe);
+            qatomic_inc(&backend_dev->rdma_dev_res->stats.missing_cqe);
             free_cqe_ctx(GINT_TO_POINTER(cqe_ctx_id),
                          backend_dev->rdma_dev_res);
         }
@@ -123,7 +123,7 @@ static int rdma_poll_cq(RdmaDeviceResources *rdma_dev_res, struct ibv_cq *ibcq)
             }
             total_ne += ne;
         } while (ne > 0);
-        atomic_sub(&rdma_dev_res->stats.missing_cqe, total_ne);
+        qatomic_sub(&rdma_dev_res->stats.missing_cqe, total_ne);
     }
 
     if (ne < 0) {
@@ -195,17 +195,17 @@ static void *comp_handler_thread(void *arg)
 
 static inline void disable_rdmacm_mux_async(RdmaBackendDev *backend_dev)
 {
-    atomic_set(&backend_dev->rdmacm_mux.can_receive, 0);
+    qatomic_set(&backend_dev->rdmacm_mux.can_receive, 0);
 }
 
 static inline void enable_rdmacm_mux_async(RdmaBackendDev *backend_dev)
 {
-    atomic_set(&backend_dev->rdmacm_mux.can_receive, sizeof(RdmaCmMuxMsg));
+    qatomic_set(&backend_dev->rdmacm_mux.can_receive, sizeof(RdmaCmMuxMsg));
 }
 
 static inline int rdmacm_mux_can_process_async(RdmaBackendDev *backend_dev)
 {
-    return atomic_read(&backend_dev->rdmacm_mux.can_receive);
+    return qatomic_read(&backend_dev->rdmacm_mux.can_receive);
 }
 
 static int rdmacm_mux_check_op_status(CharBackend *mad_chr_be)
@@ -555,7 +555,7 @@ void rdma_backend_post_send(RdmaBackendDev *backend_dev,
         goto err_dealloc_cqe_ctx;
     }
 
-    atomic_inc(&backend_dev->rdma_dev_res->stats.missing_cqe);
+    qatomic_inc(&backend_dev->rdma_dev_res->stats.missing_cqe);
     backend_dev->rdma_dev_res->stats.tx++;
 
     return;
@@ -658,7 +658,7 @@ void rdma_backend_post_recv(RdmaBackendDev *backend_dev,
         goto err_dealloc_cqe_ctx;
     }
 
-    atomic_inc(&backend_dev->rdma_dev_res->stats.missing_cqe);
+    qatomic_inc(&backend_dev->rdma_dev_res->stats.missing_cqe);
     backend_dev->rdma_dev_res->stats.rx_bufs++;
 
     return;
@@ -710,7 +710,7 @@ void rdma_backend_post_srq_recv(RdmaBackendDev *backend_dev,
         goto err_dealloc_cqe_ctx;
     }
 
-    atomic_inc(&backend_dev->rdma_dev_res->stats.missing_cqe);
+    qatomic_inc(&backend_dev->rdma_dev_res->stats.missing_cqe);
     backend_dev->rdma_dev_res->stats.rx_bufs++;
     backend_dev->rdma_dev_res->stats.rx_srq++;
 
hw/rdma/rdma_rm.c +1 -1
index 60957f88db..49141d4074 100644
@@ -790,7 +790,7 @@ int rdma_rm_init(RdmaDeviceResources *dev_res, struct ibv_device_attr *dev_attr)
     qemu_mutex_init(&dev_res->lock);
 
     memset(&dev_res->stats, 0, sizeof(dev_res->stats));
-    atomic_set(&dev_res->stats.missing_cqe, 0);
+    qatomic_set(&dev_res->stats.missing_cqe, 0);
 
     return 0;
 }
hw/rdma/vmw/pvrdma_dev_ring.c +2 -2
index c122fe7035..f0bcde74b0 100644
@@ -38,8 +38,8 @@ int pvrdma_ring_init(PvrdmaRing *ring, const char *name, PCIDevice *dev,
     ring->max_elems = max_elems;
     ring->elem_sz = elem_sz;
     /* TODO: Give a moment to think if we want to redo driver settings
-    atomic_set(&ring->ring_state->prod_tail, 0);
-    atomic_set(&ring->ring_state->cons_head, 0);
+    qatomic_set(&ring->ring_state->prod_tail, 0);
+    qatomic_set(&ring->ring_state->cons_head, 0);
     */
     ring->npages = npages;
     ring->pages = g_malloc(npages * sizeof(void *));
hw/s390x/s390-pci-bus.c +1 -1
index 92146a2119..fb4cee87a4 100644
@@ -650,7 +650,7 @@ static uint8_t set_ind_atomic(uint64_t ind_loc, uint8_t to_be_set)
     actual = *ind_addr;
     do {
         expected = actual;
-        actual = atomic_cmpxchg(ind_addr, expected, expected | to_be_set);
+        actual = qatomic_cmpxchg(ind_addr, expected, expected | to_be_set);
     } while (actual != expected);
     cpu_physical_memory_unmap((void *)ind_addr, len, 1, len);
 
hw/s390x/virtio-ccw.c +1 -1
index 8feb3451a0..8d140dc50f 100644
@@ -800,7 +800,7 @@ static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc,
     actual = *ind_addr;
     do {
         expected = actual;
-        actual = atomic_cmpxchg(ind_addr, expected, expected | to_be_set);
+        actual = qatomic_cmpxchg(ind_addr, expected, expected | to_be_set);
     } while (actual != expected);
     trace_virtio_ccw_set_ind(ind_loc, actual, actual | to_be_set);
     cpu_physical_memory_unmap((void *)ind_addr, len, 1, len);
hw/virtio/vhost.c +1 -1
index 1a1384e7a6..0119516254 100644
@@ -90,7 +90,7 @@ static void vhost_dev_sync_region(struct vhost_dev *dev,
         }
         /* Data must be read atomically. We don't really need barrier semantics
          * but it's easier to use atomic_* than roll our own. */
-        log = atomic_xchg(from, 0);
+        log = qatomic_xchg(from, 0);
         while (log) {
             int bit = ctzl(log);
             hwaddr page_addr;
hw/virtio/virtio-mmio.c +3 -3
index f12d1595aa..e1b5c3b81e 100644
@@ -179,7 +179,7 @@ static uint64_t virtio_mmio_read(void *opaque, hwaddr offset, unsigned size)
         }
         return proxy->vqs[vdev->queue_sel].enabled;
     case VIRTIO_MMIO_INTERRUPT_STATUS:
-        return atomic_read(&vdev->isr);
+        return qatomic_read(&vdev->isr);
     case VIRTIO_MMIO_STATUS:
         return vdev->status;
     case VIRTIO_MMIO_CONFIG_GENERATION:
@@ -370,7 +370,7 @@ static void virtio_mmio_write(void *opaque, hwaddr offset, uint64_t value,
         }
         break;
     case VIRTIO_MMIO_INTERRUPT_ACK:
-        atomic_and(&vdev->isr, ~value);
+        qatomic_and(&vdev->isr, ~value);
         virtio_update_irq(vdev);
         break;
     case VIRTIO_MMIO_STATUS:
@@ -496,7 +496,7 @@ static void virtio_mmio_update_irq(DeviceState *opaque, uint16_t vector)
     if (!vdev) {
         return;
     }
-    level = (atomic_read(&vdev->isr) != 0);
+    level = (qatomic_read(&vdev->isr) != 0);
     trace_virtio_mmio_setting_irq(level);
     qemu_set_irq(proxy->irq, level);
 }
hw/virtio/virtio-pci.c +3 -3
index 5bc769f685..02790e3237 100644
@@ -72,7 +72,7 @@ static void virtio_pci_notify(DeviceState *d, uint16_t vector)
         msix_notify(&proxy->pci_dev, vector);
     else {
         VirtIODevice *vdev = virtio_bus_get_device(&proxy->bus);
-        pci_set_irq(&proxy->pci_dev, atomic_read(&vdev->isr) & 1);
+        pci_set_irq(&proxy->pci_dev, qatomic_read(&vdev->isr) & 1);
     }
 }
 
@@ -398,7 +398,7 @@ static uint32_t virtio_ioport_read(VirtIOPCIProxy *proxy, uint32_t addr)
         break;
     case VIRTIO_PCI_ISR:
         /* reading from the ISR also clears it. */
-        ret = atomic_xchg(&vdev->isr, 0);
+        ret = qatomic_xchg(&vdev->isr, 0);
         pci_irq_deassert(&proxy->pci_dev);
         break;
     case VIRTIO_MSI_CONFIG_VECTOR:
@@ -1362,7 +1362,7 @@ static uint64_t virtio_pci_isr_read(void *opaque, hwaddr addr,
 {
     VirtIOPCIProxy *proxy = opaque;
     VirtIODevice *vdev = virtio_bus_get_device(&proxy->bus);
-    uint64_t val = atomic_xchg(&vdev->isr, 0);
+    uint64_t val = qatomic_xchg(&vdev->isr, 0);
     pci_irq_deassert(&proxy->pci_dev);
 
     return val;
hw/virtio/virtio.c +8 -8
index e983025217..3a3d012d9f 100644
@@ -149,8 +149,8 @@ static void virtio_virtqueue_reset_region_cache(struct VirtQueue *vq)
 {
     VRingMemoryRegionCaches *caches;
 
-    caches = atomic_read(&vq->vring.caches);
-    atomic_rcu_set(&vq->vring.caches, NULL);
+    caches = qatomic_read(&vq->vring.caches);
+    qatomic_rcu_set(&vq->vring.caches, NULL);
     if (caches) {
         call_rcu(caches, virtio_free_region_cache, rcu);
     }
@@ -197,7 +197,7 @@ static void virtio_init_region_cache(VirtIODevice *vdev, int n)
         goto err_avail;
     }
 
-    atomic_rcu_set(&vq->vring.caches, new);
+    qatomic_rcu_set(&vq->vring.caches, new);
     if (old) {
         call_rcu(old, virtio_free_region_cache, rcu);
     }
@@ -283,7 +283,7 @@ static void vring_packed_flags_write(VirtIODevice *vdev,
 /* Called within rcu_read_lock().  */
 static VRingMemoryRegionCaches *vring_get_region_caches(struct VirtQueue *vq)
 {
-    return atomic_rcu_read(&vq->vring.caches);
+    return qatomic_rcu_read(&vq->vring.caches);
 }
 
 /* Called within rcu_read_lock().  */
@@ -2007,7 +2007,7 @@ void virtio_reset(void *opaque)
     vdev->queue_sel = 0;
     vdev->status = 0;
     vdev->disabled = false;
-    atomic_set(&vdev->isr, 0);
+    qatomic_set(&vdev->isr, 0);
     vdev->config_vector = VIRTIO_NO_VECTOR;
     virtio_notify_vector(vdev, vdev->config_vector);
 
@@ -2439,13 +2439,13 @@ void virtio_del_queue(VirtIODevice *vdev, int n)
 
 static void virtio_set_isr(VirtIODevice *vdev, int value)
 {
-    uint8_t old = atomic_read(&vdev->isr);
+    uint8_t old = qatomic_read(&vdev->isr);
 
     /* Do not write ISR if it does not change, so that its cacheline remains
      * shared in the common case where the guest does not read it.
      */
     if ((old & value) != value) {
[… more lines in the raw message]

Lines of this file are not shown. View the raw message.

hw/xtensa/pic_cpu.c +2 -2
index 1d5982a9e4..6c9447565d 100644

Lines of this file are not shown. View the raw message.

iothread.c +3 -3
index 3a3860a09c..69eff9efbc 100644

Lines of this file are not shown. View the raw message.

linux-user/hppa/cpu_loop.c +6 -5
index 9915456a1d..d7e1ec7722 100644

Lines of this file are not shown. View the raw message.

linux-user/signal.c +4 -4
index 8cf51ffecd..73de934c65 100644

Lines of this file are not shown. View the raw message.

migration/colo-failover.c +2 -2
index b717edc8e2..42453481c4 100644

Lines of this file are not shown. View the raw message.

migration/migration.c +4 -4
index 58a5452471..d9d1e0b190 100644

Lines of this file are not shown. View the raw message.

migration/multifd.c +9 -9
index ac84a61797..fd57378db8 100644

Lines of this file are not shown. View the raw message.

migration/postcopy-ram.c +17 -17
index baf094ba3a..1654ff11a5 100644

Lines of this file are not shown. View the raw message.

migration/rdma.c +17 -17
index 3bd30d46ad..0340841fad 100644

Lines of this file are not shown. View the raw message.

monitor/hmp.c +3 -3
index d598dd02bb..4ecdefd705 100644

Lines of this file are not shown. View the raw message.

monitor/misc.c +1 -1
index 0b1b9b196c..262f2bd951 100644

Lines of this file are not shown. View the raw message.

monitor/monitor.c +3 -3
index b385a3d569..0f32892ad4 100644

Lines of this file are not shown. View the raw message.

qemu-nbd.c +1 -1
index 33476a1000..e39d3b23c1 100644

Lines of this file are not shown. View the raw message.

qga/commands.c +6 -6
index d3fec807c1..3dcd5fbe5c 100644

Lines of this file are not shown. View the raw message.

qom/object.c +10 -10
index a91a6a515a..1196970635 100644

Lines of this file are not shown. View the raw message.

scsi/qemu-pr-helper.c +2 -2
index 57ad830d54..d26faaf91e 100644

Lines of this file are not shown. View the raw message.

softmmu/cpu-throttle.c +5 -5
index 4e6b2818ca..2ec4b8e0bc 100644

Lines of this file are not shown. View the raw message.

softmmu/cpus.c +21 -21
index e3b98065c9..ac8940d52e 100644

Lines of this file are not shown. View the raw message.

softmmu/memory.c +3 -3
index d030eb6f7c..da5f90f141 100644

Lines of this file are not shown. View the raw message.

softmmu/vl.c +1 -1
index f7b103467c..50d8c2e42e 100644

Lines of this file are not shown. View the raw message.

target/arm/mte_helper.c +3 -3
index 891306f5b0..5615c6706c 100644

Lines of this file are not shown. View the raw message.

target/hppa/op_helper.c +1 -1
index 5685e303ab..39361d3759 100644

Lines of this file are not shown. View the raw message.

target/i386/mem_helper.c +1 -1
index acf41f8885..3a6d3ae2ef 100644

Lines of this file are not shown. View the raw message.

target/i386/whpx-all.c +3 -3
index c78baac6df..5a0f337a4c 100644

Lines of this file are not shown. View the raw message.

target/riscv/cpu_helper.c +1 -1
index f4c4111536..904899054d 100644

Lines of this file are not shown. View the raw message.

target/s390x/mem_helper.c +2 -2
index a237dec757..bd25eed3e8 100644

Lines of this file are not shown. View the raw message.

target/xtensa/exc_helper.c +2 -2
index 58a64e6d62..2f032bc053 100644

Lines of this file are not shown. View the raw message.

target/xtensa/op_helper.c +1 -1
index 09f4962d00..143476849f 100644

Lines of this file are not shown. View the raw message.

tcg/tcg.c +29 -29
index 62f299e36e..284b1c2a15 100644

Lines of this file are not shown. View the raw message.

tcg/tci.c +1 -1
index 46fe9ce63f..82039fd163 100644

Lines of this file are not shown. View the raw message.

tests/atomic64-bench.c +7 -7
index 121a8c14f4..e474753d34 100644

Lines of this file are not shown. View the raw message.

tests/atomic_add-bench.c +7 -7
index 5666f6bbff..f05471ab45 100644

Lines of this file are not shown. View the raw message.

tests/iothread.c +1 -1
index d3a2ee9a01..afde12b4ef 100644

Lines of this file are not shown. View the raw message.

tests/qht-bench.c +6 -6
index 362f03cb03..2e5b70ccd0 100644

Lines of this file are not shown. View the raw message.

tests/rcutorture.c +12 -12
index 732f03abda..de6f649058 100644

Lines of this file are not shown. View the raw message.

tests/test-aio-multithread.c +26 -26
index d3144be7e0..a555cc8835 100644

Lines of this file are not shown. View the raw message.

tests/test-logging.c +2 -2
index 8b1522cfed..ccb819f193 100644

Lines of this file are not shown. View the raw message.

tests/test-rcu-list.c +20 -18
index 92be51ec50..49641e1936 100644

Lines of this file are not shown. View the raw message.

tests/test-thread-pool.c +5 -5
index 0b675923f6..70dc6314a1 100644

Lines of this file are not shown. View the raw message.

util/aio-posix.c +7 -7
index f7f13ebfc2..280f27bb99 100644

Lines of this file are not shown. View the raw message.

util/aio-wait.c +1 -1
index b4877493f8..bdb3d3af22 100644

Lines of this file are not shown. View the raw message.

util/aio-win32.c +3 -2
index 49bd90e62e..e7b1d649e9 100644

Lines of this file are not shown. View the raw message.

util/async.c +14 -14
index 4266745dee..f758354c6a 100644

Lines of this file are not shown. View the raw message.

util/atomic64.c +5 -5
index b198a6c9c8..93037d5b11 100644

Lines of this file are not shown. View the raw message.

util/bitmap.c +7 -7
index 1753ff7f5b..1f201393ae 100644

Lines of this file are not shown. View the raw message.

util/cacheinfo.c +1 -1
index d94dc6adc8..7804c186b6 100644

Lines of this file are not shown. View the raw message.

util/fdmon-epoll.c +2 -2
index fcd989d47d..e11a8a022e 100644

Lines of this file are not shown. View the raw message.

util/fdmon-io_uring.c +6 -6
index 1d14177df0..1461dfa407 100644

Lines of this file are not shown. View the raw message.

util/lockcnt.c +27 -25
index 4f88dcf8b8..5da36946b1 100644

Lines of this file are not shown. View the raw message.

util/log.c +5 -5
index bdb3d712e8..4b423062aa 100644

Lines of this file are not shown. View the raw message.

util/qemu-coroutine-lock.c +9 -9
index 5da5234155..36927b5f88 100644

Lines of this file are not shown. View the raw message.

util/qemu-coroutine-sleep.c +2 -2
index 769a76e57d..8c4dac4fd7 100644

Lines of this file are not shown. View the raw message.

util/qemu-coroutine.c +3 -3
index c3caa6c770..38fb6d3084 100644

Lines of this file are not shown. View the raw message.

util/qemu-sockets.c +2 -2
index 99ce2fd5e6..de4bf7616e 100644

Lines of this file are not shown. View the raw message.

util/qemu-thread-posix.c +6 -6
index b4c2359272..dcff5e7c5d 100644

Lines of this file are not shown. View the raw message.

util/qemu-thread-win32.c +6 -6
index d207b0cb58..cb5aa2018c 100644

Lines of this file are not shown. View the raw message.

util/qemu-timer.c +6 -6
index 878d80fd5e..ca677836cb 100644

Lines of this file are not shown. View the raw message.

util/qht.c +29 -28
index b2e020c398..079605121b 100644

Lines of this file are not shown. View the raw message.

util/qsp.c +25 -25
index 7d5147f1b2..bacc5fa2f6 100644

Lines of this file are not shown. View the raw message.

util/rcu.c +18 -18
index c4fefa9333..13ac0f75cb 100644

Lines of this file are not shown. View the raw message.

util/stats64.c +17 -17
index 389c365a9e..897613c949 100644

Lines of this file are not shown. View the raw message.

docs/devel/atomics.rst +67 -67
index 445c3b3503..52baa0736d 100644

Lines of this file are not shown. View the raw message.

scripts/kernel-doc +1 -1
index 030b5c8691..40ad782e34 100755

Lines of this file are not shown. View the raw message.

tcg/aarch64/tcg-target.c.inc +1 -1
index 948c35d825..2607fe4ab9 100644

Lines of this file are not shown. View the raw message.

tcg/mips/tcg-target.c.inc +1 -1
index bd5b8e09a0..7aa2073520 100644

Lines of this file are not shown. View the raw message.

tcg/ppc/tcg-target.c.inc +3 -3
index 393c4b30e0..7cb40b0466 100644

Lines of this file are not shown. View the raw message.

tcg/sparc/tcg-target.c.inc +3 -2
index 0f1d91fc21..40bc12290c 100644

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Part of the diff is not shown here. View the raw message.

Reviewed-by on v3 1/1
#2 lore raw
In reply to Stefan Hajnoczi
Quoted 22 lines · Stefan Hajnoczi

clang's C11 atomic_fetch_*() functions only take a C11 atomic type pointer argument. QEMU uses direct types (int, etc) and this causes a compiler error when a QEMU code calls these functions in a source file that also included <stdatomic.h> via a system header file:

  $ CC=clang CXX=clang++ ./configure ... && make
  ../util/async.c:79:17: error: address argument to atomic operation must be a pointer to _Atomic type ('unsigned int *' invalid)

Avoid using atomic_*() names in QEMU's atomic.h since that namespace is used by <stdatomic.h>. Prefix QEMU's APIs with 'q' so that atomic.h and <stdatomic.h> can co-exist. I checked /usr/include on my machine and searched GitHub for existing "qatomic_" users but there seem to be none.

This patch was generated using:

  $ git grep -h -o '\<atomic\(64\)\?_[a-z0-9_]\+' include/qemu/atomic.h | \
    sort -u >/tmp/changed_identifiers
  $ for identifier in $(</tmp/changed_identifiers); do
        sed -i "s%\<$identifier\>%q$identifier%g" \
            $(git grep -I -l "\<$identifier\>")
    done
Quoted 3 lines

I manually fixed line-wrap issues and misaligned rST tables.

Reviewed using 'git-diff --color-words'.

Quoted 24 lines

---
v3:
* Use qatomic_ instead of atomic_ [Paolo]
* The diff of my manual fixups is available here:
https://vmsplice.net/~stefan/atomic-namespace-pre-fixups-v3.diff
- Dropping #ifndef qatomic_fetch_add in atomic.h
- atomic_##X(haddr, val) glue macros not caught by grep
- Keep atomic_add-bench name
- C preprocessor backslash-newline ('\') column alignment
- Line wrapping
v2:
* The diff of my manual fixups is available here:
https://vmsplice.net/~stefan/atomic-namespace-pre-fixups.diff
- Dropping #ifndef qemu_atomic_fetch_add in atomic.h
- atomic_##X(haddr, val) glue macros not caught by grep
- Keep atomic_add-bench name
- C preprocessor backslash-newline ('\') column alignment
- Line wrapping
* Use grep -I to avoid accidentally modifying binary files (RISC-V OpenSBI ELFs) [Eric Blake]
* Tweak .gitorder to show atomic.h changes first [Eric Blake]
* Update grep commands in commit description so reviewers can reproduce mechanical changes [Eric Blake]
---

#3 lore raw
In reply to Stefan Hajnoczi
Patchew URL: https://patchew.org/QEMU/[email protected]/

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series seems to have some coding style problems. See output below for more information:

Type: series
Message-id: [email protected]
Subject: [PATCH v3] qemu/atomic.h: rename atomic_ to qatomic_

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu

 - [tag update]      patchew/[email protected] -> patchew/[email protected]
 * [new tag]         patchew/[email protected] -> patchew/[email protected]
 * [new tag]         patchew/[email protected] -> patchew/[email protected]

Switched to a new branch 'test'
07abb8e qemu/atomic.h: rename atomic_ to qatomic_

=== OUTPUT BEGIN ===
ERROR: Macros with multiple statements should be enclosed in a do - while loop #2797: FILE: include/qemu/atomic.h:152:
+#define qatomic_rcu_read__nocheck(ptr, valptr) \

+    __atomic_load(ptr, valptr, __ATOMIC_RELAXED);        \
     smp_read_barrier_depends();

ERROR: space required before that '*' (ctx:VxB)
#2942: FILE: include/qemu/atomic.h:333:
+#define qatomic_read__nocheck(p) (*(__typeof__(*(p)) volatile*) (p))

                                                                ^

ERROR: Use of volatile is usually wrong, please add a comment #2942: FILE: include/qemu/atomic.h:333:
+#define qatomic_read__nocheck(p) (*(__typeof__(*(p)) volatile*) (p))

ERROR: space required before that '*' (ctx:VxB)
#2943: FILE: include/qemu/atomic.h:334:
+#define qatomic_set__nocheck(p, i) ((*(__typeof__(*(p)) volatile*) (p)) = (i))

                                                                 ^

ERROR: Use of volatile is usually wrong, please add a comment #2943: FILE: include/qemu/atomic.h:334:
+#define qatomic_set__nocheck(p, i) ((*(__typeof__(*(p)) volatile*) (p)) = (i))

ERROR: space required after that ',' (ctx:VxV)
#2948: FILE: include/qemu/atomic.h:337:
+#define qatomic_set(ptr, i) qatomic_set__nocheck(ptr,i)

                                                         ^

ERROR: memory barrier without comment
#3020: FILE: include/qemu/atomic.h:395:
+#define qatomic_xchg(ptr, i) (smp_mb(), __sync_lock_test_and_set(ptr, i))

WARNING: Block comments use a leading /* on a separate line #3094: FILE: include/qemu/atomic.h:447:
+/* qatomic_mb_read/set semantics map Java volatile variables. They are

WARNING: Block comments use a leading /* on a separate line #6177: FILE: util/bitmap.c:214:
+ /* If we avoided the full barrier in qatomic_or(), issue a

WARNING: Block comments use a leading /* on a separate line #7192: FILE: util/rcu.c:85:
+ /* Instead of using qatomic_mb_set for index->waiting, and

WARNING: Block comments use a leading /* on a separate line #7218: FILE: util/rcu.c:154:
+ /* In either case, the qatomic_mb_set below blocks stores that free

total: 7 errors, 4 warnings, 6267 lines checked

Commit 07abb8e79dbe (qemu/atomic.h: rename atomic_ to qatomic_) has style problems, please review. If any of these errors are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1

The full log is available at
http://patchew.org/logs/[email protected]/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

author Applied
#4 lore raw
In reply to Stefan Hajnoczi
Quoted 25 lines · Stefan Hajnoczi

clang's C11 atomic_fetch_*() functions only take a C11 atomic type pointer argument. QEMU uses direct types (int, etc) and this causes a compiler error when a QEMU code calls these functions in a source file that also included <stdatomic.h> via a system header file:

  $ CC=clang CXX=clang++ ./configure ... && make
  ../util/async.c:79:17: error: address argument to atomic operation must be a pointer to _Atomic type ('unsigned int *' invalid)

Avoid using atomic_*() names in QEMU's atomic.h since that namespace is used by <stdatomic.h>. Prefix QEMU's APIs with 'q' so that atomic.h and <stdatomic.h> can co-exist. I checked /usr/include on my machine and searched GitHub for existing "qatomic_" users but there seem to be none.

This patch was generated using:

  $ git grep -h -o '\<atomic\(64\)\?_[a-z0-9_]\+' include/qemu/atomic.h | \
    sort -u >/tmp/changed_identifiers
  $ for identifier in $(</tmp/changed_identifiers); do
        sed -i "s%\<$identifier\>%q$identifier%g" \
            $(git grep -I -l "\<$identifier\>")
    done

I manually fixed line-wrap issues and misaligned rST tables.

Quoted 10 lines

---
v3:
* Use qatomic_ instead of atomic_ [Paolo]
* The diff of my manual fixups is available here:
https://vmsplice.net/~stefan/atomic-namespace-pre-fixups-v3.diff
- Dropping #ifndef qatomic_fetch_add in atomic.h
- atomic_##X(haddr, val) glue macros not caught by grep
- Keep atomic_add-bench name
- C preprocessor backslash-newline ('\') column alignment
- Line wrapping

Thanks, applied quickly due to high risk of conflicts:
https://github.com/stefanha/qemu/commits/block

Stefan

#5 lore raw
In reply to Stefan Hajnoczi
Quoted 25 lines · Stefan Hajnoczi

clang's C11 atomic_fetch_*() functions only take a C11 atomic type pointer argument. QEMU uses direct types (int, etc) and this causes a compiler error when a QEMU code calls these functions in a source file that also included <stdatomic.h> via a system header file:

   $ CC=clang CXX=clang++ ./configure ... && make
   ../util/async.c:79:17: error: address argument to atomic operation must be a pointer to _Atomic type ('unsigned int *' invalid)

Avoid using atomic_*() names in QEMU's atomic.h since that namespace is used by <stdatomic.h>. Prefix QEMU's APIs with 'q' so that atomic.h and <stdatomic.h> can co-exist. I checked /usr/include on my machine and searched GitHub for existing "qatomic_" users but there seem to be none.

This patch was generated using:

   $ git grep -h -o '\<atomic\(64\)\?_[a-z0-9_]\+' include/qemu/atomic.h | \
     sort -u >/tmp/changed_identifiers
   $ for identifier in $(</tmp/changed_identifiers); do
         sed -i "s%\<$identifier\>%q$identifier%g" \
             $(git grep -I -l "\<$identifier\>")
     done

I manually fixed line-wrap issues and misaligned rST tables.

..snip..

Quoted 36 lines
include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h +5 -5
index acd4c8346d..7b4062a1a1 100644
@@ -68,7 +68,7 @@ static inline int pvrdma_idx_valid(uint32_t idx, uint32_t max_elems)
 
  static inline int32_t pvrdma_idx(int *var, uint32_t max_elems)
  {
-	const unsigned int idx = atomic_read(var);
+	const unsigned int idx = qatomic_read(var);
 
  	if (pvrdma_idx_valid(idx, max_elems))
  		return idx & (max_elems - 1);
@@ -77,17 +77,17 @@ static inline int32_t pvrdma_idx(int *var, uint32_t max_elems)
 
  static inline void pvrdma_idx_ring_inc(int *var, uint32_t max_elems)
  {
-	uint32_t idx = atomic_read(var) + 1;	/* Increment. */
+	uint32_t idx = qatomic_read(var) + 1;	/* Increment. */
 
  	idx &= (max_elems << 1) - 1;		/* Modulo size, flip gen. */
-	atomic_set(var, idx);
+	qatomic_set(var, idx);
  }
 
  static inline int32_t pvrdma_idx_ring_has_space(const struct pvrdma_ring *r,
  					      uint32_t max_elems, uint32_t *out_tail)
  {
-	const uint32_t tail = atomic_read(&r->prod_tail);
-	const uint32_t head = atomic_read(&r->cons_head);
+	const uint32_t tail = qatomic_read(&r->prod_tail);
+	const uint32_t head = qatomic_read(&r->cons_head);
 
  	if (pvrdma_idx_valid(tail, max_elems) &&
  	    pvrdma_idx_valid(head, max_elems)) {

include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h static inline int32_t pvrdma_idx_ring_has_space(const struct pvrdma_ring *r, · line 107

Quoted 10 lines
include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h +2 -2
@@ -100,8 +100,8 @@ static inline int32_t pvrdma_idx_ring_has_space(const struct pvrdma_ring *r,
  static inline int32_t pvrdma_idx_ring_has_data(const struct pvrdma_ring *r,
  					     uint32_t max_elems, uint32_t *out_head)
  {
-	const uint32_t tail = atomic_read(&r->prod_tail);
-	const uint32_t head = atomic_read(&r->cons_head);
+	const uint32_t tail = qatomic_read(&r->prod_tail);
+	const uint32_t head = qatomic_read(&r->cons_head);
 
  	if (pvrdma_idx_valid(tail, max_elems) &&
  	    pvrdma_idx_valid(head, max_elems)) {

It looks like the changes in this file are going to get reverted the next time someone does a linux header sync.

maintainer
#6 lore raw
Quoted 44 lines · Matthew Rosato
Quoted 44 lines

diff --git a/include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h b/include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h index acd4c8346d..7b4062a1a1 100644
---
a/include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h
+++
b/include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h

include/standard-headers/drivers/infiniband/hw/vmw_pvrdma/pvrdma_ring.h +2 -2
@@ -68,7 +68,7 @@ static inline int pvrdma_idx_valid(uint32_t idx,
 uint32_t max_elems)
     static inline int32_t pvrdma_idx(int *var, uint32_t max_elems)
   {
-    const unsigned int idx = atomic_read(var);
+    const unsigned int idx = qatomic_read(var);
         if (pvrdma_idx_valid(idx, max_elems))
           return idx & (max_elems - 1);
@@ -77,17 +77,17 @@ static inline int32_t pvrdma_idx(int *var,
 uint32_t max_elems)
     static inline void pvrdma_idx_ring_inc(int *var, uint32_t max_elems)
   {
-    uint32_t idx = atomic_read(var) + 1;    /* Increment. */
+    uint32_t idx = qatomic_read(var) + 1;    /* Increment. */

        idx &= (max_elems << 1) - 1;        /* Modulo size, flip gen. */ -    atomic_set(var, idx);
+    qatomic_set(var, idx);
  }
    static inline int32_t pvrdma_idx_ring_has_space(const struct pvrdma_ring *r,
                            uint32_t max_elems, uint32_t *out_tail) {
-    const uint32_t tail = atomic_read(&r->prod_tail);
-    const uint32_t head = atomic_read(&r->cons_head);
+    const uint32_t tail = qatomic_read(&r->prod_tail);
+    const uint32_t head = qatomic_read(&r->cons_head);
        if (pvrdma_idx_valid(tail, max_elems) && pvrdma_idx_valid(head, max_elems)) {

(quoted diff) +0 -0
@@ -100,8 +100,8 @@ static inline int32_t
 pvrdma_idx_ring_has_space(const struct pvrdma_ring *r,

  static inline int32_t pvrdma_idx_ring_has_data(const struct pvrdma_ring *r,
                           uint32_t max_elems, uint32_t *out_head) {
-    const uint32_t tail = atomic_read(&r->prod_tail); -    const uint32_t head = atomic_read(&r->cons_head);

Quoted 8 lines
Quoted 4 lines

+    const uint32_t tail = qatomic_read(&r->prod_tail); +    const uint32_t head = qatomic_read(&r->cons_head); if (pvrdma_idx_valid(tail, max_elems) && pvrdma_idx_valid(head, max_elems)) {

It looks like the changes in this file are going to get reverted the next time someone does a linux header sync.

Source code should not be at all imported from Linux. The hacks that accomodate pvrdma in update-linux-headers.sh (like s/atomic_t/u32/) really have no place there; the files in include/standard-headers/drivers/infiniband/hw/vmw_pvrdma should all be moved in hw/.

Paolo