Skip to contentSkip to conversation

buffer: Avoid setting buffer bits that are already set

PATCH Superseded Kemi Wang 3 participants 6 messages Oct 23, 2017 – Oct 24, 2017

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Kemi Wang

author
#0 lore raw

PATCH 1/1 buffer: Avoid setting buffer bits that are already set

It's expensive to set buffer flags that are already set, because that causes a costly cache line transition.

A common case is setting the "verified" flag during ext4 writes. This patch checks for the flag being set first.

With the AIM7/creat-clo benchmark testing on a 48G ramdisk based-on ext4 file system, we see 3.3%(15431->15936) improvement of aim7.jobs-per-min on a 2-sockets broadwell platform.

What the benchmark does is: it forks 3000 processes, and each process do the following:
a) open a new file
b) close the file
c) delete the file
until loop=100*1000 times.

The original patch is contributed by Andi Kleen.

---
 include/linux/buffer_head.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
include/linux/buffer_head.h +2 -1
index c8dae55..e1799f7 100644
@@ -84,7 +84,8 @@ struct buffer_head {
 #define BUFFER_FNS(bit, name)						\
 static __always_inline void set_buffer_##name(struct buffer_head *bh)	\
 {									\
-	set_bit(BH_##bit, &(bh)->b_state);				\
+	if (!test_bit(BH_##bit, &(bh)->b_state))			\
+		set_bit(BH_##bit, &(bh)->b_state);              \
 }									\
 static __always_inline void clear_buffer_##name(struct buffer_head *bh)	\
 {									\
Signature · 1 line
2.7.4
#1 lore raw
Quoted 10 lines · Kemi Wang

It's expensive to set buffer flags that are already set, because that causes a costly cache line transition.

A common case is setting the "verified" flag during ext4 writes. This patch checks for the flag being set first.

With the AIM7/creat-clo benchmark testing on a 48G ramdisk based-on ext4 file system, we see 3.3%(15431->15936) improvement of aim7.jobs-per-min on a 2-sockets broadwell platform.

Quoted 8 lines

What the benchmark does is: it forks 3000 processes, and each process do the following:
a) open a new file
b) close the file
c) delete the file
until loop=100*1000 times.

The original patch is contributed by Andi Kleen.

We discussed this recently, in reference to this commit:

commit 7fcbbaf18392f0b17c95e2f033c8ccf87eecde1d

Author: Jens Axboe <[email protected]>
Date:   Thu May 22 11:54:16 2014 -0700
    mm/filemap.c: avoid always dirtying mapping->flags on O_DIRECT

which made a massive difference, as the changelog details.

blk-mq uses this extensively as well, where possible. The problem is that it always has to be explained, hence the recent discussion was around perhaps adding

set_bit_if_not_set()
clear_bit_if_set()

or similar functions, to document in a single location why this matters. Additionally, some archs may be able to implement that in an efficient manner.

You can add my reviewed-by to the below, I'll see if I can find some time to implement the above in a nice way. In the mean time, you may want to consider adding a comment to the function explaining why you have done it that way.

Signature · 1 line
Jens Axboe
author
#2 lore raw
Quoted 34 lines · Jens Axboe
Quoted 10 lines · Kemi Wang

It's expensive to set buffer flags that are already set, because that causes a costly cache line transition.

A common case is setting the "verified" flag during ext4 writes. This patch checks for the flag being set first.

With the AIM7/creat-clo benchmark testing on a 48G ramdisk based-on ext4 file system, we see 3.3%(15431->15936) improvement of aim7.jobs-per-min on a 2-sockets broadwell platform.

Quoted 8 lines

What the benchmark does is: it forks 3000 processes, and each process do the following:
a) open a new file
b) close the file
c) delete the file
until loop=100*1000 times.

The original patch is contributed by Andi Kleen.

We discussed this recently, in reference to this commit:

commit 7fcbbaf18392f0b17c95e2f033c8ccf87eecde1d

Author: Jens Axboe <[email protected]>
Date:   Thu May 22 11:54:16 2014 -0700
    mm/filemap.c: avoid always dirtying mapping->flags on O_DIRECT

which made a massive difference, as the changelog details.

blk-mq uses this extensively as well, where possible. The problem is that it always has to be explained, hence the recent discussion was around perhaps adding

Quoted 8 lines

set_bit_if_not_set()
clear_bit_if_set()

or similar functions, to document in a single location why this matters. Additionally, some archs may be able to implement that in an efficient manner.

You can add my reviewed-by to the below,

Thanks.

I'll see if I can find some

Quoted 1 line

time to implement the above in a nice way.

Agree. Maybe something like test_and_set_bit() would be more suitable.

In the mean time, you may

Quoted 3 lines

want to consider adding a comment to the function explaining why you have done it that way.

Sure.

#3 lore raw
Quoted 5 lines · kemi

I'll see if I can find some

Quoted 1 line

time to implement the above in a nice way.

Agree. Maybe something like test_and_set_bit() would be more suitable.

test_and_set_bit is a very different operation for the CPU because it is atomic for both. But we want the initial read to not be atomic.

If you add special functions use a different variant that is only atomic for the set.

-Andi

author
#4 lore raw
Quoted 11 lines · Andi Kleen
Quoted 5 lines · kemi

I'll see if I can find some

Quoted 1 line

time to implement the above in a nice way.

Agree. Maybe something like test_and_set_bit() would be more suitable.

test_and_set_bit is a very different operation for the CPU because it is atomic for both. But we want the initial read to not be atomic.

I meant to express the meaning of test before setting bit. Apologize to make you confused.

Quoted 5 lines

If you add special functions use a different variant that is only atomic for the set.

-Andi

#5 lore raw
Quoted 10 lines · kemi
Quoted 7 lines · Andi Kleen
Quoted 5 lines · kemi

I'll see if I can find some

Quoted 1 line

time to implement the above in a nice way.

Agree. Maybe something like test_and_set_bit() would be more suitable.

Quoted 7 lines
Quoted 4 lines

test_and_set_bit is a very different operation for the CPU because it is atomic for both. But we want the initial read to not be atomic.

I meant to express the meaning of test before setting bit. Apologize to make you confused.

That's why I suggested something like set_bit_if_not_set(), test_and_set_bit() is both already used and has entirely different semantics.

Signature · 1 line
Jens Axboe