Skip to contentSkip to conversation

llist: Fix missing memory barrier

PATCH In review Mathieu Desnoyers 4 participants 8 messages Feb 6, 2015 Library code

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 llist: Fix missing memory barrier

A smp_read_barrier_depends() appears to be missing in llist_del_first(). It should only matter for Alpha in practice. Adding it after the check of entry against NULL allows skipping the barrier in a common case.

CC Huang Ying <[email protected]>
CC Paul McKenney <[email protected]>
CC David Howells <[email protected]>
---
 lib/llist.c | 7 +++++++
 1 file changed, 7 insertions(+)
lib/llist.c +7 -0
index f76196d..72861f3 100644
@@ -26,6 +26,7 @@
 #include <linux/export.h>
 #include <linux/interrupt.h>
 #include <linux/llist.h>
+#include <asm/barrier.h>
 
 
 /**
@@ -72,6 +73,12 @@ struct llist_node *llist_del_first(struct llist_head *head)
 		if (entry == NULL)
 			return NULL;
 		old_entry = entry;
+		/*
+		 * Load entry before entry->next. Matches the implicit
+		 * memory barrier before the cmpxchg in llist_add_batch(),
+		 * which ensures entry->next is stored before entry.
+		 */
+		smp_read_barrier_depends();
 		next = entry->next;
 		entry = cmpxchg(&head->first, old_entry, next);
 		if (entry == old_entry)
Signature · 1 line
2.1.4
#1 lore raw

Hi Mathieu,

Quoted 3 lines · Mathieu Desnoyers

A smp_read_barrier_depends() appears to be missing in llist_del_first(). It should only matter for Alpha in practice. Adding it after the check of entry against NULL allows skipping the barrier in a common case.

We recently decided on using lockless_dereference() instead of hard-coding smp_read_barrier_depends()[1]. The advantage is that lockless_dereference() clearly shows what loads are being ordered. Could you resend the patch using that API?

Thanks!

[1] http://lkml.iu.edu/hypermail/linux/kernel/1410.3/04561.html

Quoted 42 lines
CC Huang Ying <[email protected]>
CC Paul McKenney <[email protected]>
CC David Howells <[email protected]>
---
 lib/llist.c | 7 +++++++
 1 file changed, 7 insertions(+)
lib/llist.c +7 -1
index f76196d..72861f3 100644
@@ -26,6 +26,7 @@
 #include <linux/export.h>
 #include <linux/interrupt.h>
 #include <linux/llist.h>
+#include <asm/barrier.h>
 
 
 /**
@@ -72,6 +73,12 @@ struct llist_node *llist_del_first(struct llist_head *head)
                if (entry == NULL)
                        return NULL;
                old_entry = entry;
+               /*
+                * Load entry before entry->next. Matches the implicit
+                * memory barrier before the cmpxchg in llist_add_batch(),
+                * which ensures entry->next is stored before entry.
+                */
+               smp_read_barrier_depends();
                next = entry->next;
                entry = cmpxchg(&head->first, old_entry, next);
                if (entry == old_entry)
--

2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Signature · 1 line
Pranith
#2 lore raw

----- Original Message -----

Quoted 11 lines

From: "Pranith Kumar" <[email protected]>
To: "Mathieu Desnoyers" <[email protected]>
Cc: "Huang Ying" <[email protected]>, "LKML" <[email protected]>, "Paul McKenney" <[email protected]>, "David Howells" <[email protected]>
Sent: Thursday, February 5, 2015 10:44:07 PM
Subject: Re: [PATCH] llist: Fix missing memory barrier

Hi Mathieu,

On Thu, Feb 5, 2015 at 10:06 PM, Mathieu Desnoyers <[email protected]> wrote:

Quoted 8 lines
Quoted 3 lines

A smp_read_barrier_depends() appears to be missing in llist_del_first(). It should only matter for Alpha in practice. Adding it after the check of entry against NULL allows skipping the barrier in a common case.

We recently decided on using lockless_dereference() instead of hard-coding smp_read_barrier_depends()[1]. The advantage is that lockless_dereference() clearly shows what loads are being ordered. Could you resend the patch using that API?

Since llist.h has been introduced prior to 3.18, I'm wondering if it would be worthwhile to submit 2 patches for the purpose of backporting to stable branches:

1) Fix introducing smp_read_barrier_depends() (for master and stable branches)
2) Move master from smp_read_barrier_depends() to lockless_dereference(),

Thoughts ?

Thanks!

Mathieu

Quoted 54 lines

Thanks!

[1] http://lkml.iu.edu/hypermail/linux/kernel/1410.3/04561.html

Quoted 35 lines
CC Huang Ying <[email protected]>
CC Paul McKenney <[email protected]>
CC David Howells <[email protected]>
---
 lib/llist.c | 7 +++++++
 1 file changed, 7 insertions(+)
lib/llist.c +7 -0
index f76196d..72861f3 100644
@@ -26,6 +26,7 @@
 #include <linux/export.h>
 #include <linux/interrupt.h>
 #include <linux/llist.h>
+#include <asm/barrier.h>
 
 
 /**
@@ -72,6 +73,12 @@ struct llist_node *llist_del_first(struct llist_head
 *head)
                if (entry == NULL)
                        return NULL;
                old_entry = entry;
+               /*
+                * Load entry before entry->next. Matches the implicit
+                * memory barrier before the cmpxchg in llist_add_batch(),
+                * which ensures entry->next is stored before entry.
+                */
+               smp_read_barrier_depends();
                next = entry->next;
                entry = cmpxchg(&head->first, old_entry, next);
                if (entry == old_entry)
Quoted 8 lines

--
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

--
Pranith

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#3 lore raw
Quoted 25 lines · Mathieu Desnoyers

----- Original Message -----

Quoted 11 lines

From: "Pranith Kumar" <[email protected]>
To: "Mathieu Desnoyers" <[email protected]>
Cc: "Huang Ying" <[email protected]>, "LKML" <[email protected]>, "Paul McKenney" <[email protected]>, "David Howells" <[email protected]>
Sent: Thursday, February 5, 2015 10:44:07 PM
Subject: Re: [PATCH] llist: Fix missing memory barrier

Hi Mathieu,

On Thu, Feb 5, 2015 at 10:06 PM, Mathieu Desnoyers <[email protected]> wrote:

Quoted 8 lines
Quoted 3 lines

A smp_read_barrier_depends() appears to be missing in llist_del_first(). It should only matter for Alpha in practice. Adding it after the check of entry against NULL allows skipping the barrier in a common case.

We recently decided on using lockless_dereference() instead of hard-coding smp_read_barrier_depends()[1]. The advantage is that lockless_dereference() clearly shows what loads are being ordered. Could you resend the patch using that API?

Since llist.h has been introduced prior to 3.18, I'm wondering if it would be worthwhile to submit 2 patches for the purpose of backporting to stable branches:

Quoted 6 lines

1) Fix introducing smp_read_barrier_depends() (for master and stable branches)
2) Move master from smp_read_barrier_depends() to lockless_dereference(),

Thoughts ?

Yes, why? What code needs these new apis?

confused,

greg k-h

#4 lore raw

----- Original Message -----

Quoted 35 lines

From: "Greg Kroah-Hartman" <[email protected]>
To: "Mathieu Desnoyers" <[email protected]>
Cc: "Pranith Kumar" <[email protected]>, "Huang Ying" <[email protected]>, "LKML" <[email protected]>, "Paul McKenney" <[email protected]>, "David Howells" <[email protected]>
Sent: Friday, February 6, 2015 10:03:59 AM
Subject: Re: [PATCH] llist: Fix missing memory barrier

Quoted 27 lines · Mathieu Desnoyers

----- Original Message -----

Quoted 17 lines

From: "Pranith Kumar" <[email protected]>
To: "Mathieu Desnoyers" <[email protected]>
Cc: "Huang Ying" <[email protected]>, "LKML" <[email protected]>, "Paul McKenney" <[email protected]>, "David Howells" <[email protected]>
Sent: Thursday, February 5, 2015 10:44:07 PM
Subject: Re: [PATCH] llist: Fix missing memory barrier

Hi Mathieu,

Quoted 4 lines · Mathieu Desnoyers

A smp_read_barrier_depends() appears to be missing in llist_del_first().
It should only matter for Alpha in practice. Adding it after the check of entry against NULL allows skipping the barrier in a common case.

Quoted 4 lines

We recently decided on using lockless_dereference() instead of hard-coding smp_read_barrier_depends()[1]. The advantage is that lockless_dereference() clearly shows what loads are being ordered. Could you resend the patch using that API?

Since llist.h has been introduced prior to 3.18, I'm wondering if it would be worthwhile to submit 2 patches for the purpose of backporting to stable branches:

Quoted 8 lines
Quoted 6 lines

1) Fix introducing smp_read_barrier_depends() (for master and stable branches)
2) Move master from smp_read_barrier_depends() to lockless_dereference(),

Thoughts ?

Yes, why? What code needs these new apis?

The subsystems using llist.h in master: IRQ, smp core code, vmalloc, scsi, the block layer, some filesystems... and more. grep for "llist.h" to see the complete list of users.

My question was mainly on how to do the fix process-wise: post-3.18, it implies using the new lockless_dereference() API. pre-3.18, we need to use smp_read_barrier_depends().

As Peter Hurley suggested, using the new API for master and 3.18 seems like the right approach. Then the backports to stable branches can, if needed, use smp_read_barrier_depends() instead. Is that OK with you ?

Thanks,

Mathieu

Quoted 5 lines

confused,

greg k-h

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#5 lore raw
Quoted 49 lines · Mathieu Desnoyers

----- Original Message -----

Quoted 35 lines

From: "Greg Kroah-Hartman" <[email protected]>
To: "Mathieu Desnoyers" <[email protected]>
Cc: "Pranith Kumar" <[email protected]>, "Huang Ying" <[email protected]>, "LKML" <[email protected]>, "Paul McKenney" <[email protected]>, "David Howells" <[email protected]>
Sent: Friday, February 6, 2015 10:03:59 AM
Subject: Re: [PATCH] llist: Fix missing memory barrier

Quoted 27 lines · Mathieu Desnoyers

----- Original Message -----

Quoted 17 lines

From: "Pranith Kumar" <[email protected]>
To: "Mathieu Desnoyers" <[email protected]>
Cc: "Huang Ying" <[email protected]>, "LKML" <[email protected]>, "Paul McKenney" <[email protected]>, "David Howells" <[email protected]>
Sent: Thursday, February 5, 2015 10:44:07 PM
Subject: Re: [PATCH] llist: Fix missing memory barrier

Hi Mathieu,

Quoted 4 lines · Mathieu Desnoyers

A smp_read_barrier_depends() appears to be missing in llist_del_first().
It should only matter for Alpha in practice. Adding it after the check of entry against NULL allows skipping the barrier in a common case.

Quoted 4 lines

We recently decided on using lockless_dereference() instead of hard-coding smp_read_barrier_depends()[1]. The advantage is that lockless_dereference() clearly shows what loads are being ordered. Could you resend the patch using that API?

Since llist.h has been introduced prior to 3.18, I'm wondering if it would be worthwhile to submit 2 patches for the purpose of backporting to stable branches:

Quoted 8 lines
Quoted 6 lines

1) Fix introducing smp_read_barrier_depends() (for master and stable branches)
2) Move master from smp_read_barrier_depends() to lockless_dereference(),

Thoughts ?

Yes, why? What code needs these new apis?

The subsystems using llist.h in master: IRQ, smp core code, vmalloc, scsi, the block layer, some filesystems... and more. grep for "llist.h" to see the complete list of users.

Quoted 8 lines

My question was mainly on how to do the fix process-wise: post-3.18, it implies using the new lockless_dereference() API. pre-3.18, we need to use smp_read_barrier_depends().

As Peter Hurley suggested, using the new API for master and 3.18 seems like the right approach. Then the backports to stable branches can, if needed, use smp_read_barrier_depends() instead. Is that OK with you ?

Sounds fine to me.

greg k-h

#6 lore raw
In reply to Mathieu Desnoyers
Quoted 25 lines · Mathieu Desnoyers

----- Original Message -----

Quoted 11 lines

From: "Pranith Kumar" <[email protected]>
To: "Mathieu Desnoyers" <[email protected]>
Cc: "Huang Ying" <[email protected]>, "LKML" <[email protected]>, "Paul McKenney" <[email protected]>, "David Howells" <[email protected]>
Sent: Thursday, February 5, 2015 10:44:07 PM
Subject: Re: [PATCH] llist: Fix missing memory barrier

Hi Mathieu,

On Thu, Feb 5, 2015 at 10:06 PM, Mathieu Desnoyers <[email protected]> wrote:

Quoted 8 lines
Quoted 3 lines

A smp_read_barrier_depends() appears to be missing in llist_del_first(). It should only matter for Alpha in practice. Adding it after the check of entry against NULL allows skipping the barrier in a common case.

We recently decided on using lockless_dereference() instead of hard-coding smp_read_barrier_depends()[1]. The advantage is that lockless_dereference() clearly shows what loads are being ordered. Could you resend the patch using that API?

Since llist.h has been introduced prior to 3.18, I'm wondering if it would be worthwhile to submit 2 patches for the purpose of backporting to stable branches:

Quoted 6 lines

1) Fix introducing smp_read_barrier_depends() (for master and stable branches)
2) Move master from smp_read_barrier_depends() to lockless_dereference(),

Thoughts ?

Other way around.

The first patch should use lockless_dereference() for mainline.

Then once that's been picked up and has a SHA, then a backport patch for stable using smp_read_barrier_depends() instead before lockless_dereference() was introduced.

Regards,
Peter Hurley

Quoted 60 lines

Thanks!

Mathieu

Quoted 54 lines

Thanks!

[1] http://lkml.iu.edu/hypermail/linux/kernel/1410.3/04561.html

Quoted 35 lines
CC Huang Ying <[email protected]>
CC Paul McKenney <[email protected]>
CC David Howells <[email protected]>
---
 lib/llist.c | 7 +++++++
 1 file changed, 7 insertions(+)
lib/llist.c +7 -0
index f76196d..72861f3 100644
@@ -26,6 +26,7 @@
 #include <linux/export.h>
 #include <linux/interrupt.h>
 #include <linux/llist.h>
+#include <asm/barrier.h>
 
 
 /**
@@ -72,6 +73,12 @@ struct llist_node *llist_del_first(struct llist_head
 *head)
                if (entry == NULL)
                        return NULL;
                old_entry = entry;
+               /*
+                * Load entry before entry->next. Matches the implicit
+                * memory barrier before the cmpxchg in llist_add_batch(),
+                * which ensures entry->next is stored before entry.
+                */
+               smp_read_barrier_depends();
                next = entry->next;
                entry = cmpxchg(&head->first, old_entry, next);
                if (entry == old_entry)
Quoted 8 lines

--
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

--
Pranith

#7 lore raw

----- Original Message -----

Quoted 42 lines

From: "Peter Hurley" <[email protected]>
To: "Mathieu Desnoyers" <[email protected]>, "Pranith Kumar" <[email protected]>
Cc: "Huang Ying" <[email protected]>, "LKML" <[email protected]>, "Paul McKenney" <[email protected]>, "David Howells" <[email protected]>, "Greg Kroah-Hartman" <[email protected]>
Sent: Friday, February 6, 2015 10:17:37 AM
Subject: Re: [PATCH] llist: Fix missing memory barrier

Quoted 26 lines · Mathieu Desnoyers

----- Original Message -----

Quoted 12 lines

From: "Pranith Kumar" <[email protected]>
To: "Mathieu Desnoyers" <[email protected]>
Cc: "Huang Ying" <[email protected]>, "LKML" <[email protected]>, "Paul McKenney" <[email protected]>, "David Howells" <[email protected]>
Sent: Thursday, February 5, 2015 10:44:07 PM
Subject: Re: [PATCH] llist: Fix missing memory barrier

Hi Mathieu,

On Thu, Feb 5, 2015 at 10:06 PM, Mathieu Desnoyers <[email protected]> wrote:

Quoted 8 lines
Quoted 3 lines

A smp_read_barrier_depends() appears to be missing in llist_del_first(). It should only matter for Alpha in practice. Adding it after the check of entry against NULL allows skipping the barrier in a common case.

We recently decided on using lockless_dereference() instead of hard-coding smp_read_barrier_depends()[1]. The advantage is that lockless_dereference() clearly shows what loads are being ordered. Could you resend the patch using that API?

Since llist.h has been introduced prior to 3.18, I'm wondering if it would be worthwhile to submit 2 patches for the purpose of backporting to stable branches:

Quoted 6 lines

1) Fix introducing smp_read_barrier_depends() (for master and stable branches)
2) Move master from smp_read_barrier_depends() to lockless_dereference(),

Thoughts ?

Quoted 7 lines

Other way around.

The first patch should use lockless_dereference() for mainline.

Then once that's been picked up and has a SHA, then a backport patch for stable using smp_read_barrier_depends() instead before lockless_dereference() was introduced.

That sounds like a good approach. I'll wait for a final word from Greg and proceed this way unless there are objections.

Thanks!

Mathieu

Quoted 68 lines

Regards,
Peter Hurley

Quoted 62 lines

Thanks!

Mathieu

Quoted 56 lines

Thanks!

[1] http://lkml.iu.edu/hypermail/linux/kernel/1410.3/04561.html

Quoted 39 lines
CC Huang Ying <[email protected]>
CC Paul McKenney <[email protected]>
CC David Howells <[email protected]>
---
 lib/llist.c | 7 +++++++
 1 file changed, 7 insertions(+)
lib/llist.c +7 -0
index f76196d..72861f3 100644
@@ -26,6 +26,7 @@
 #include <linux/export.h>
 #include <linux/interrupt.h>
 #include <linux/llist.h>
+#include <asm/barrier.h>
 
 
 /**
@@ -72,6 +73,12 @@ struct llist_node *llist_del_first(struct llist_head
 *head)
                if (entry == NULL)
                        return NULL;
                old_entry = entry;
+               /*
+                * Load entry before entry->next. Matches the implicit
+                * memory barrier before the cmpxchg in
 llist_add_batch(),
+                * which ensures entry->next is stored before entry.
+                */
+               smp_read_barrier_depends();
                next = entry->next;
                entry = cmpxchg(&head->first, old_entry, next);
                if (entry == old_entry)

--
2.1.4

Quoted 6 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

--
Pranith

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com