Skip to contentSkip to conversation

block/graph-lock: Make WITH_GRAPH_RDLOCK_GUARD() fully checked

PATCH In review Kevin Wolf 3 participants 5 messages Jun 27, 2024 – Jul 11, 2024

Subsystems (2)
lore
Share & tools
lore

Kevin Wolf

author
#0 lore raw

cover block/graph-lock: Make WITH_GRAPH_RDLOCK_GUARD() fully checked

Newer clang versions allow us to check scoped guards more thoroughly. Surprisingly, we only seem to have missed one instance with the old incomplete checks.

Kevin Wolf (2):
block-copy: Fix missing graph lock
block/graph-lock: Make WITH_GRAPH_RDLOCK_GUARD() fully checked

 include/block/graph-lock.h | 21 ++++++++++++++-------
 block/block-copy.c         |  4 +++-
 meson.build                | 14 +++++++++++++-
 3 files changed, 30 insertions(+), 9 deletions(-)
Signature · 1 line
2.45.2
author
#1 lore raw

PATCH 1/2 block-copy: Fix missing graph lock

The graph lock needs to be held when calling bdrv_co_pdiscard(). Fix block_copy_task_entry() to take it for the call.

WITH_GRAPH_RDLOCK_GUARD() was implemented in a weak way because of limitations in clang's Thread Safety Analysis at the time, so that it only asserts that the lock is held (which allows calling functions that require the lock), but we never deal with the unlocking (so even after the scope of the guard, the compiler assumes that the lock is still held). This is why the compiler didn't catch this locking error.

---
 block/block-copy.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
block/block-copy.c +3 -1
index 7e3b378528..cc618e4561 100644
@@ -595,7 +595,9 @@ static coroutine_fn int block_copy_task_entry(AioTask *task)
     if (s->discard_source && ret == 0) {
         int64_t nbytes =
             MIN(t->req.offset + t->req.bytes, s->len) - t->req.offset;
-        bdrv_co_pdiscard(s->source, t->req.offset, nbytes);
+        WITH_GRAPH_RDLOCK_GUARD() {
+            bdrv_co_pdiscard(s->source, t->req.offset, nbytes);
+        }
     }
 
     return ret;
Signature · 1 line
2.45.2
author
#2 lore raw

PATCH 2/2 block/graph-lock: Make WITH_GRAPH_RDLOCK_GUARD() fully checked

In reply to Kevin Wolf

Upstream clang 18 (and backports to clang 17 in Fedora and RHEL) implemented support for __attribute__((cleanup())) in its Thread Safety Analysis, so we can now actually have a proper implementation of WITH_GRAPH_RDLOCK_GUARD() that understands when we acquire and when we release the lock.

-Wthread-safety is now only enabled if the compiler is new enough to understand this pattern. In theory, we could have used some #ifdefs to keep the existing basic checks on old compilers, but as long as someone runs a newer compiler (and our CI does), we will catch locking problems, so it's probably not worth keeping multiple implementations for this.

The implementation can't use g_autoptr any more because the glib macros define wrapper functions that don't have the right TSA attributes, so the compiler would complain about them. Just use the cleanup attribute directly instead.

---
 include/block/graph-lock.h | 21 ++++++++++++++-------
 meson.build                | 14 +++++++++++++-
 2 files changed, 27 insertions(+), 8 deletions(-)
include/block/graph-lock.h +14 -7
index d7545e82d0..dc8d949184 100644
@@ -209,31 +209,38 @@ typedef struct GraphLockable { } GraphLockable;
  * unlocked. TSA_ASSERT_SHARED() makes sure that the following calls know that
  * we hold the lock while unlocking is left unchecked.
  */
-static inline GraphLockable * TSA_ASSERT_SHARED(graph_lock) TSA_NO_TSA coroutine_fn
+static inline GraphLockable * TSA_ACQUIRE_SHARED(graph_lock) coroutine_fn
 graph_lockable_auto_lock(GraphLockable *x)
 {
     bdrv_graph_co_rdlock();
     return x;
 }
 
-static inline void TSA_NO_TSA coroutine_fn
-graph_lockable_auto_unlock(GraphLockable *x)
+static inline void TSA_RELEASE_SHARED(graph_lock) coroutine_fn
+graph_lockable_auto_unlock(GraphLockable **x)
 {
     bdrv_graph_co_rdunlock();
 }
 
-G_DEFINE_AUTOPTR_CLEANUP_FUNC(GraphLockable, graph_lockable_auto_unlock)
+#define GRAPH_AUTO_UNLOCK __attribute__((cleanup(graph_lockable_auto_unlock)))
 
+/*
+ * @var is only used to break the loop after the first iteration.
+ * @unlock_var can't be unlocked and then set to NULL because TSA wants the lock
+ * to be held at the start of every iteration of the loop.
+ */
 #define WITH_GRAPH_RDLOCK_GUARD_(var)                                         \
-    for (g_autoptr(GraphLockable) var = graph_lockable_auto_lock(GML_OBJ_()); \
+    for (GraphLockable *unlock_var GRAPH_AUTO_UNLOCK =                        \
+            graph_lockable_auto_lock(GML_OBJ_()),                             \
+            *var = unlock_var;                                                \
          var;                                                                 \
-         graph_lockable_auto_unlock(var), var = NULL)
+         var = NULL)
 
 #define WITH_GRAPH_RDLOCK_GUARD() \
     WITH_GRAPH_RDLOCK_GUARD_(glue(graph_lockable_auto, __COUNTER__))
 
 #define GRAPH_RDLOCK_GUARD(x)                                       \
-    g_autoptr(GraphLockable)                                        \
+    GraphLockable * GRAPH_AUTO_UNLOCK                               \
     glue(graph_lockable_auto, __COUNTER__) G_GNUC_UNUSED =          \
             graph_lockable_auto_lock(GML_OBJ_())
 
meson.build +13 -1
index 97e00d6f59..b1d5ce5f1d 100644
@@ -624,7 +624,19 @@ warn_flags = [
 ]
 
 if host_os != 'darwin'
-  warn_flags += ['-Wthread-safety']
+  tsa_has_cleanup = cc.compiles('''
+    struct __attribute__((capability("mutex"))) mutex {};
+    void lock(struct mutex *m) __attribute__((acquire_capability(m)));
+    void unlock(struct mutex *m) __attribute__((release_capability(m)));
+
+    void test(void) {
+      struct mutex __attribute__((cleanup(unlock))) m;
+      lock(&m);
+    }
+  ''', args: ['-Wthread-safety', '-Werror'])
+  if tsa_has_cleanup
+    warn_flags += ['-Wthread-safety']
+  endif
 endif
 
 # Set up C++ compiler flags
Signature · 1 line
2.45.2
Reviewed-by on v1 2/2
#3 lore raw
Quoted 100 lines · Kevin Wolf

Upstream clang 18 (and backports to clang 17 in Fedora and RHEL) implemented support for __attribute__((cleanup())) in its Thread Safety Analysis, so we can now actually have a proper implementation of WITH_GRAPH_RDLOCK_GUARD() that understands when we acquire and when we release the lock.

-Wthread-safety is now only enabled if the compiler is new enough to understand this pattern. In theory, we could have used some #ifdefs to keep the existing basic checks on old compilers, but as long as someone runs a newer compiler (and our CI does), we will catch locking problems, so it's probably not worth keeping multiple implementations for this.

The implementation can't use g_autoptr any more because the glib macros define wrapper functions that don't have the right TSA attributes, so the compiler would complain about them. Just use the cleanup attribute directly instead.

---
 include/block/graph-lock.h | 21 ++++++++++++++-------
 meson.build                | 14 +++++++++++++-
 2 files changed, 27 insertions(+), 8 deletions(-)
include/block/graph-lock.h +14 -7
index d7545e82d0..dc8d949184 100644
@@ -209,31 +209,38 @@ typedef struct GraphLockable { } GraphLockable;
  * unlocked. TSA_ASSERT_SHARED() makes sure that the following calls know that
  * we hold the lock while unlocking is left unchecked.
  */
-static inline GraphLockable * TSA_ASSERT_SHARED(graph_lock) TSA_NO_TSA coroutine_fn
+static inline GraphLockable * TSA_ACQUIRE_SHARED(graph_lock) coroutine_fn
 graph_lockable_auto_lock(GraphLockable *x)
 {
     bdrv_graph_co_rdlock();
     return x;
 }
 
-static inline void TSA_NO_TSA coroutine_fn
-graph_lockable_auto_unlock(GraphLockable *x)
+static inline void TSA_RELEASE_SHARED(graph_lock) coroutine_fn
+graph_lockable_auto_unlock(GraphLockable **x)
 {
     bdrv_graph_co_rdunlock();
 }
 
-G_DEFINE_AUTOPTR_CLEANUP_FUNC(GraphLockable, graph_lockable_auto_unlock)
+#define GRAPH_AUTO_UNLOCK __attribute__((cleanup(graph_lockable_auto_unlock)))
 
+/*
+ * @var is only used to break the loop after the first iteration.
+ * @unlock_var can't be unlocked and then set to NULL because TSA wants the lock
+ * to be held at the start of every iteration of the loop.
+ */
 #define WITH_GRAPH_RDLOCK_GUARD_(var)                                         \
-    for (g_autoptr(GraphLockable) var = graph_lockable_auto_lock(GML_OBJ_()); \
+    for (GraphLockable *unlock_var GRAPH_AUTO_UNLOCK =                        \
+            graph_lockable_auto_lock(GML_OBJ_()),                             \
+            *var = unlock_var;                                                \
          var;                                                                 \
-         graph_lockable_auto_unlock(var), var = NULL)
+         var = NULL)
 
 #define WITH_GRAPH_RDLOCK_GUARD() \
     WITH_GRAPH_RDLOCK_GUARD_(glue(graph_lockable_auto, __COUNTER__))
 
 #define GRAPH_RDLOCK_GUARD(x)                                       \
-    g_autoptr(GraphLockable)                                        \
+    GraphLockable * GRAPH_AUTO_UNLOCK                               \
     glue(graph_lockable_auto, __COUNTER__) G_GNUC_UNUSED =          \
             graph_lockable_auto_lock(GML_OBJ_())
 
meson.build +13 -2
index 97e00d6f59..b1d5ce5f1d 100644
@@ -624,7 +624,19 @@ warn_flags = [
 ]
 
 if host_os != 'darwin'
-  warn_flags += ['-Wthread-safety']
+  tsa_has_cleanup = cc.compiles('''
+    struct __attribute__((capability("mutex"))) mutex {};
+    void lock(struct mutex *m) __attribute__((acquire_capability(m)));
+    void unlock(struct mutex *m) __attribute__((release_capability(m)));
+
+    void test(void) {
+      struct mutex __attribute__((cleanup(unlock))) m;
+      lock(&m);
+    }
+  ''', args: ['-Wthread-safety', '-Werror'])
+  if tsa_has_cleanup
+    warn_flags += ['-Wthread-safety']
+  endif
 endif
 
 # Set up C++ compiler flags
--
Quoted 2 lines

2.45.2

Signature · 2 lines
Manos Pitsidianakis
Emulation and Virtualization Engineer at Linaro Ltd
Reviewed-by on the v1 series
#4 lore raw
In reply to Kevin Wolf
Quoted 4 lines · Kevin Wolf

Newer clang versions allow us to check scoped guards more thoroughly. Surprisingly, we only seem to have missed one instance with the old incomplete checks.

Quoted 12 lines

Kevin Wolf (2):
block-copy: Fix missing graph lock
block/graph-lock: Make WITH_GRAPH_RDLOCK_GUARD() fully checked

 include/block/graph-lock.h | 21 ++++++++++++++-------
 block/block-copy.c         |  4 +++-
 meson.build                | 14 +++++++++++++-
 3 files changed, 30 insertions(+), 9 deletions(-)
Signature · 1 line
2.45.2