Skip to contentSkip to conversation

net: more try_cmpxchg() conversions

PATCH Applied Eric Dumazet 3 participants 9 messages Nov 15, 2022 – Nov 17, 2022

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover net: more try_cmpxchg() conversions

Adopt try_cmpxchg() and friends in more places, as this is preferred nowadays.

Eric Dumazet (6):
net: mm_account_pinned_pages() optimization
ipv6: fib6_new_sernum() optimization
net: net_{enable|disable}_timestamp() optimizations
net: adopt try_cmpxchg() in napi_schedule_prep() and
napi_complete_done()
net: adopt try_cmpxchg() in napi_{enable|disable}()
net: __sock_gen_cookie() cleanup

 net/core/dev.c       | 42 +++++++++++++++---------------------------
 net/core/skbuff.c    |  5 ++---
 net/core/sock_diag.c | 12 ++++++------
 net/ipv6/ip6_fib.c   |  7 +++----
 4 files changed, 26 insertions(+), 40 deletions(-)
Signature · 1 line
2.38.1.431.g37b22c650d-goog
#1 lore raw

PATCH 1/6 net: mm_account_pinned_pages() optimization

Adopt atomic_long_try_cmpxchg() in mm_account_pinned_pages() as it is slightly more efficient.

Cc Willem de Bruijn <[email protected]>
---
 net/core/skbuff.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
net/core/skbuff.c +2 -3
index 90d085290d49d864b8431f99a19dbda867d9c03b..4bf95e36ed162611e8245c64c6caeecf9f60ef5d 100644
@@ -1267,13 +1267,12 @@ int mm_account_pinned_pages(struct mmpin *mmp, size_t size)
 	max_pg = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT;
 	user = mmp->user ? : current_user();
 
+	old_pg = atomic_long_read(&user->locked_vm);
 	do {
-		old_pg = atomic_long_read(&user->locked_vm);
 		new_pg = old_pg + num_pg;
 		if (new_pg > max_pg)
 			return -ENOBUFS;
-	} while (atomic_long_cmpxchg(&user->locked_vm, old_pg, new_pg) !=
-		 old_pg);
+	} while (!atomic_long_try_cmpxchg(&user->locked_vm, &old_pg, new_pg));
 
 	if (!mmp->user) {
 		mmp->user = get_uid(user);
Signature · 1 line
2.38.1.431.g37b22c650d-goog
#2 lore raw

PATCH 2/6 ipv6: fib6_new_sernum() optimization

In reply to Eric Dumazet

Adopt atomic_try_cmpxchg() which is slightly more efficient.

---
 net/ipv6/ip6_fib.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
net/ipv6/ip6_fib.c +3 -4
index 413f66781e50de62d6b20042d84798e7da59165a..2438da5ff6da810d9f612fc66df4d28510f50f10 100644
@@ -91,13 +91,12 @@ static void fib6_walker_unlink(struct net *net, struct fib6_walker *w)
 
 static int fib6_new_sernum(struct net *net)
 {
-	int new, old;
+	int new, old = atomic_read(&net->ipv6.fib6_sernum);
 
 	do {
-		old = atomic_read(&net->ipv6.fib6_sernum);
 		new = old < INT_MAX ? old + 1 : 1;
-	} while (atomic_cmpxchg(&net->ipv6.fib6_sernum,
-				old, new) != old);
+	} while (!atomic_try_cmpxchg(&net->ipv6.fib6_sernum, &old, new));
+
 	return new;
 }
 
Signature · 1 line
2.38.1.431.g37b22c650d-goog
maintainer Reviewed-by on v1 2/6
#3 lore raw

Re: [PATCH net-next 2/6] ipv6: fib6_new_sernum() optimization

Quoted 19 lines · Eric Dumazet

Adopt atomic_try_cmpxchg() which is slightly more efficient.

---
 net/ipv6/ip6_fib.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
net/ipv6/ip6_fib.c +1 -1
index 413f66781e50de62d6b20042d84798e7da59165a..2438da5ff6da810d9f612fc66df4d28510f50f10 100644
@@ -91,13 +91,12 @@ static void fib6_walker_unlink(struct net *net, struct fib6_walker *w)
 
 static int fib6_new_sernum(struct net *net)
 {
-	int new, old;
+	int new, old = atomic_read(&net->ipv6.fib6_sernum);
 

net/ipv6/ip6_fib.c static void fib6_walker_unlink(struct net *net, struct fib6_walker *w) · line 102

Quoted 10 lines
net/ipv6/ip6_fib.c +2 -3
 	do {
-		old = atomic_read(&net->ipv6.fib6_sernum);
 		new = old < INT_MAX ? old + 1 : 1;
-	} while (atomic_cmpxchg(&net->ipv6.fib6_sernum,
-				old, new) != old);
+	} while (!atomic_try_cmpxchg(&net->ipv6.fib6_sernum, &old, new));
+
 	return new;
 }
 
#4 lore raw

PATCH 3/6 net: net_{enable|disable}_timestamp() optimizations

In reply to Eric Dumazet

Adopting atomic_try_cmpxchg() makes the code cleaner.

---
 net/core/dev.c | 18 ++++++------------
 1 file changed, 6 insertions(+), 12 deletions(-)
net/core/dev.c +6 -12
index 117e830cabb0787ecd3da13bd88f8818fddaddc1..10b56648a9d4a0a709f8e23bb3e114854a4a1b69 100644
@@ -2073,13 +2073,10 @@ static DECLARE_WORK(netstamp_work, netstamp_clear);
 void net_enable_timestamp(void)
 {
 #ifdef CONFIG_JUMP_LABEL
-	int wanted;
+	int wanted = atomic_read(&netstamp_wanted);
 
-	while (1) {
-		wanted = atomic_read(&netstamp_wanted);
-		if (wanted <= 0)
-			break;
-		if (atomic_cmpxchg(&netstamp_wanted, wanted, wanted + 1) == wanted)
+	while (wanted > 0) {
+		if (atomic_try_cmpxchg(&netstamp_wanted, &wanted, wanted + 1))
 			return;
 	}
 	atomic_inc(&netstamp_needed_deferred);
@@ -2093,13 +2090,10 @@ EXPORT_SYMBOL(net_enable_timestamp);
 void net_disable_timestamp(void)
 {
 #ifdef CONFIG_JUMP_LABEL
-	int wanted;
+	int wanted = atomic_read(&netstamp_wanted);
 
-	while (1) {
-		wanted = atomic_read(&netstamp_wanted);
-		if (wanted <= 1)
-			break;
-		if (atomic_cmpxchg(&netstamp_wanted, wanted, wanted - 1) == wanted)
+	while (wanted > 1) {
+		if (atomic_try_cmpxchg(&netstamp_wanted, &wanted, wanted - 1))
 			return;
 	}
 	atomic_dec(&netstamp_needed_deferred);
Signature · 1 line
2.38.1.431.g37b22c650d-goog
#5 lore raw

PATCH 4/6 net: adopt try_cmpxchg() in napi_schedule_prep() and napi_complete_done()

In reply to Eric Dumazet

This makes the code slightly more efficient.

---
 net/core/dev.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)
net/core/dev.c +4 -6
index 10b56648a9d4a0a709f8e23bb3e114854a4a1b69..0c12c7ad04f21da05fef4c60ca5570bff48ec491 100644
@@ -5979,10 +5979,9 @@ EXPORT_SYMBOL(__napi_schedule);
  */
 bool napi_schedule_prep(struct napi_struct *n)
 {
-	unsigned long val, new;
+	unsigned long new, val = READ_ONCE(n->state);
 
 	do {
-		val = READ_ONCE(n->state);
 		if (unlikely(val & NAPIF_STATE_DISABLE))
 			return false;
 		new = val | NAPIF_STATE_SCHED;
@@ -5995,7 +5994,7 @@ bool napi_schedule_prep(struct napi_struct *n)
 		 */
 		new |= (val & NAPIF_STATE_SCHED) / NAPIF_STATE_SCHED *
 						   NAPIF_STATE_MISSED;
-	} while (cmpxchg(&n->state, val, new) != val);
+	} while (!try_cmpxchg(&n->state, &val, new));
 
 	return !(val & NAPIF_STATE_SCHED);
 }
@@ -6063,9 +6062,8 @@ bool napi_complete_done(struct napi_struct *n, int work_done)
 		local_irq_restore(flags);
 	}
 
+	val = READ_ONCE(n->state);
 	do {
-		val = READ_ONCE(n->state);
-
 		WARN_ON_ONCE(!(val & NAPIF_STATE_SCHED));
 
 		new = val & ~(NAPIF_STATE_MISSED | NAPIF_STATE_SCHED |
@@ -6078,7 +6076,7 @@ bool napi_complete_done(struct napi_struct *n, int work_done)
 		 */
 		new |= (val & NAPIF_STATE_MISSED) / NAPIF_STATE_MISSED *
 						    NAPIF_STATE_SCHED;
-	} while (cmpxchg(&n->state, val, new) != val);
+	} while (!try_cmpxchg(&n->state, &val, new));
 
 	if (unlikely(val & NAPIF_STATE_MISSED)) {
 		__napi_schedule(n);
Signature · 1 line
2.38.1.431.g37b22c650d-goog
#6 lore raw

PATCH 5/6 net: adopt try_cmpxchg() in napi_{enable|disable}()

In reply to Eric Dumazet

This makes code a bit cleaner.

---
 net/core/dev.c | 14 +++++---------
 1 file changed, 5 insertions(+), 9 deletions(-)
net/core/dev.c +5 -9
index 0c12c7ad04f21da05fef4c60ca5570bff48ec491..fb943dad96513e0f9eb1c3ce30c5bb7170edea5e 100644
@@ -6397,8 +6397,8 @@ void napi_disable(struct napi_struct *n)
 	might_sleep();
 	set_bit(NAPI_STATE_DISABLE, &n->state);
 
-	for ( ; ; ) {
-		val = READ_ONCE(n->state);
+	val = READ_ONCE(n->state);
+	do {
 		if (val & (NAPIF_STATE_SCHED | NAPIF_STATE_NPSVC)) {
 			usleep_range(20, 200);
 			continue;
@@ -6406,10 +6406,7 @@ void napi_disable(struct napi_struct *n)
 
 		new = val | NAPIF_STATE_SCHED | NAPIF_STATE_NPSVC;
 		new &= ~(NAPIF_STATE_THREADED | NAPIF_STATE_PREFER_BUSY_POLL);
-
-		if (cmpxchg(&n->state, val, new) == val)
-			break;
-	}
+	} while (!try_cmpxchg(&n->state, &val, new));
 
 	hrtimer_cancel(&n->timer);
 
@@ -6426,16 +6423,15 @@ EXPORT_SYMBOL(napi_disable);
  */
 void napi_enable(struct napi_struct *n)
 {
-	unsigned long val, new;
+	unsigned long new, val = READ_ONCE(n->state);
 
 	do {
-		val = READ_ONCE(n->state);
 		BUG_ON(!test_bit(NAPI_STATE_SCHED, &val));
 
 		new = val & ~(NAPIF_STATE_SCHED | NAPIF_STATE_NPSVC);
 		if (n->dev->threaded && n->thread)
 			new |= NAPIF_STATE_THREADED;
-	} while (cmpxchg(&n->state, val, new) != val);
+	} while (!try_cmpxchg(&n->state, &val, new));
 }
 EXPORT_SYMBOL(napi_enable);
 
Signature · 1 line
2.38.1.431.g37b22c650d-goog
#7 lore raw

PATCH 6/6 net: __sock_gen_cookie() cleanup

In reply to Eric Dumazet

Adopt atomic64_try_cmpxchg() and remove the loop,
to make the intent more obvious.

---
 net/core/sock_diag.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
net/core/sock_diag.c +6 -6
index f7cf74cdd3db1f7f0783db88fc28fd876fbac4b6..b11593cae5a09b15a10d6ba35bccc22263cb8fc8 100644
@@ -25,14 +25,14 @@ DEFINE_COOKIE(sock_cookie);
 
 u64 __sock_gen_cookie(struct sock *sk)
 {
-	while (1) {
-		u64 res = atomic64_read(&sk->sk_cookie);
+	u64 res = atomic64_read(&sk->sk_cookie);
 
-		if (res)
-			return res;
-		res = gen_cookie_next(&sock_cookie);
-		atomic64_cmpxchg(&sk->sk_cookie, 0, res);
+	if (!res) {
+		u64 new = gen_cookie_next(&sock_cookie);
+
+		atomic64_try_cmpxchg(&sk->sk_cookie, &res, new);
 	}
+	return res;
 }
 
 int sock_diag_check_cookie(struct sock *sk, const __u32 *cookie)
Signature · 1 line
2.38.1.431.g37b22c650d-goog
#8 lore raw
In reply to Eric Dumazet
This series was applied to netdev/net-next.git (master) by David S. Miller <[email protected]>:

Hello:

This series was applied to netdev/net-next.git (master) by David S. Miller <[email protected]>:

Quoted 12 lines · On Tue, 15 Nov 2022 09:10:55 +0000 you wrote:

Adopt try_cmpxchg() and friends in more places, as this is preferred nowadays.

Eric Dumazet (6):
net: mm_account_pinned_pages() optimization
ipv6: fib6_new_sernum() optimization
net: net_{enable|disable}_timestamp() optimizations
net: adopt try_cmpxchg() in napi_schedule_prep() and
napi_complete_done()
net: adopt try_cmpxchg() in napi_{enable|disable}()
net: __sock_gen_cookie() cleanup

Quoted 1 line

[...]

Here is the summary with links:
- [net-next,1/6] net: mm_account_pinned_pages() optimization

    https://git.kernel.org/netdev/net-next/c/57fc05e8e82d
  - [net-next,2/6] ipv6: fib6_new_sernum() optimization
    https://git.kernel.org/netdev/net-next/c/30189806fbb9
  - [net-next,3/6] net: net_{enable|disable}_timestamp() optimizations
    https://git.kernel.org/netdev/net-next/c/6af645a5b2da
  - [net-next,4/6] net: adopt try_cmpxchg() in napi_schedule_prep() and napi_complete_done()
    https://git.kernel.org/netdev/net-next/c/1462160c7455
  - [net-next,5/6] net: adopt try_cmpxchg() in napi_{enable|disable}()
    https://git.kernel.org/netdev/net-next/c/4ffa1d1c6842
  - [net-next,6/6] net: __sock_gen_cookie() cleanup
    https://git.kernel.org/netdev/net-next/c/4ebf802cf1c6

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html