1be41b78: +18% increased btrfs write throughput
Hi Josef,
FYI. We are doing 0day performance tests and happen to notice that btrfs write throughput increased considerably during v3.10-11 time frame:
v3.10 v3.11 v3.12 v3.13-rc6
--------------- ------------------------- ------------------------- -------------------------
50619 ~ 1% +17.0% 59209 ~ 2% +18.8% 60159 ~ 2% +20.5% 61007 ~ 0% lkp-ws02/micro/dd-write/11HDD-JBOD-cfq-btrfs-1dd
50619 +17.0% 59209 +18.8% 60159 +20.5% 61007 TOTAL iostat.sdd.wkB/sand it's contributed by
commit 1be41b78bc688fc634bf30965d2be692c99fd11d
Author: Josef Bacik <[email protected]> AuthorDate: Wed Jun 12 13:56:06 2013 -0400 Commit: Josef Bacik <[email protected]> CommitDate: Mon Jul 1 08:52:28 2013 -0400
Btrfs: fix transaction throttling for delayed refs
Dave has this fs_mark script that can make btrfs abort with sufficient amount of ram. This is because with more ram we can keep more dirty metadata in cache which in a round about way makes for many more pending delayed refs. What happens is we end up not throttling the transaction enough so when we go to commit the transaction when we've completely filled the file system we'll abort() because we use all of the space in the global reserve and we still have delayed refs to run. To fix this we need to make the delayed ref flushing and the transaction throttling dependant upon the number of delayed refs that we have instead of how much reserved space is left in the global reserve. With this patch we not only stop aborting transactions but we also get a smoother run speed with fs_mark and it makes us about 10% faster. Thanks,
fs/btrfs/ctree.h | 2 ++ fs/btrfs/extent-tree.c | 61 ++++++++++++++++++++++++++++++++++++++++++++++---- fs/btrfs/transaction.c | 24 +++++++++----------- 3 files changed, 69 insertions(+), 18 deletions(-)
time.elapsed_time
740 ++------------------*------------------------------------------------+
*.. .*.*..* *..*.*..*. |
720 ++ *.*..*.*. *..*.*..*. .*..*. .*..*. .*. |
| *. *..* *. *..*
700 ++ |
| |
680 ++ |
| |
660 ++ |
| |
640 ++ |
| |
620 ++ O O |
O O O O O O O O O O O O O O O O O O O O O O O O |600 ++-------------------------------------------------------------------+
time.voluntary_context_switches
100000 ++------------------*-*----*--------------------------------------+
90000 ++ *. .*..*.*. *. *.. |
*. .. *..* .*.. .*..*.*.*..*.*..*.*..* .*
80000 ++* * * + .* |
70000 ++ *. |
| |
60000 ++ |
50000 ++ |
40000 ++ |
| |
30000 ++ |
20000 ++ |
| |
10000 ++ |
0 O+O--O-O--O-O--O-O--O-O-O--O-O--O-O--O-O--O-O-O--O-O--O-O--O-O----+
time.file_system_inputs
800000 ++--------------------*----*--------------------------------------+
| *. .*..*.*..* *. *.. |
700000 *+ .. *..* .*.. .*..*.*.*..*.*..*.*..* .*
| * * * + .* |
600000 ++ *. |
500000 ++ |
| |
400000 ++ |
| |
300000 ++ |
200000 ++ |
| |
100000 ++ |
| |
0 O+O--O-O--O-O--O-O--O-O-O--O-O--O-O--O-O--O-O-O--O-O--O-O--O-O----+
time.file_system_outputs900000 ++----------------------------------------------------------------+
| *.*.. .*.*. .*. .*.*.*..*. .* |
800000 *+ .. *.*..*.*. *. *.. .*..*.*. *..*.*. + .*.*
700000 ++* * *. |
| |
600000 ++ |
500000 ++ |
| |
400000 ++ |
300000 ++ |
| |
200000 ++ |
100000 ++ |
| |
0 O+O--O-O--O-O--O-O--O-O-O--O-O--O-O--O-O--O-O-O--O-O--O-O--O-O----+
perf-stat.page-faults6.6e+06 ++---------------------------------------------------------------+
| |
6.4e+06 ++ .*.*.. .*.. |
*. .*.*..*.*.*. *.* *. .*. .*..*. |
6.2e+06 ++*. *. *..*.* *..*.*.*..*.*..*.*
| |
6e+06 ++ |
| |
5.8e+06 ++ |
| |
5.6e+06 ++ |
| |
5.4e+06 ++O O |
O O O O O O O O O O O O O O O O O O O O O O O O |5.2e+06 ++---------------------------------------------------------------+
perf-stat.minor-faults
6.6e+06 ++---------------------------------------------------------------+
| |
6.4e+06 ++ .*.*.. .*.. |
*. .*.*..*.*.*. *.* *. .*. .*..*. |
6.2e+06 ++*. *. *..*.* *..*.*.*..*.*..*.*
| |
6e+06 ++ |
| |
5.8e+06 ++ |
| |
5.6e+06 ++ |
| |
5.4e+06 ++O O |
O O O O O O O O O O O O O O O O O O O O O O O O |5.2e+06 ++---------------------------------------------------------------+
vmstat.io.bi
600 ++-----------*-*--*-*--*--*----*-------------------------------------+
| .. *. + |
500 *+.*.*..*.* *..*.*..*.*.. .*.*..*.*..*.*..*.*..*
| *. |
| |
400 ++ |
| |
300 ++ |
| |
200 ++ |
| O O |
| |
100 ++ O |
O O O O O O O O O O O O O O O O O O O O O O |
0 ++-----------------------------------------------O-------------------+
vmstat.io.bo660000 ++----------------------------------------------------------------+
| O O O O O O O O O |
640000 O+ O O O O O O O O O O O O O |
| O O O |
620000 ++ |
| |
600000 ++ |
| |
580000 ++ |
| |
560000 ++ .*.|
| .*. .*.*..*. .*..*.*..*.*..*.*. *
540000 *+*..*.*..*. .*.*.. .*. *. * |
| *..*.*..* * |520000 ++----------------------------------------------------------------+
iostat.sde.avgrq-sz
720 ++-------------------------------------------------------------------+
| O O |
700 ++ O O O O O |
680 O+ O O O O O O O O O O O |
| O O O O O O O |
660 ++ |
| |
640 ++ |
| |
620 ++ |
600 ++ |
| |
580 ++ .*.*..*.*.. .*.*.. .*.. .*. *. .*..|
*..*.*..*.*. *. *.*.. .*..* *. *..*. .. *..* *560 ++----------------------------------*-------------------*------------+
iostat.sde.avgqu-sz
125 ++-------------------------------------------------------------------+
| O |
120 ++ O O |
O O O O O O O O O O O O O O O O |
| O O O O O O O |
115 ++ |
| |
110 ++ |
| |
105 ++ |
| .*.. |
| .*. .* *..*.*..* .*. .*. .*.*.. .*..*
100 *+ *.. .*. + .*..*.*.. + *. *. *. * |
| * *..*.*. * |
95 ++-------------------------------------------------------------------+
iostat.sdf.avgrq-sz720 ++-------------------------------------------------------------------+
| O O |
700 ++ O O O O O O |
680 O+ O O O O O O O O |
| O O O O O O O O O |
660 ++ |
| |
640 ++ |
| |
620 ++ |
600 ++ |
| .* |
580 ++ *.*.. .*..*. + .*.. .*..*.*.. *..|
|.. .*..*.*..* *..*.*.. .*..* *..*.*..* + *560 *+------*---------------------------*---------------------------*----+
iostat.sdg.avgrq-sz
720 ++-------------------------------------------------------------------+
| O O |
700 O+ O O O O |
680 ++ O O O O O O O O O O O O O |
| O O O O O O |
660 ++ |
640 ++ |
| |
620 ++ |
600 ++ |
| .*.. .*.. |
580 ++ .*..* *.*..*. .*..*.*..*. .*.*..*..*.*..*.*..* .*..*
560 ++.* *. *..*.*. * |
*. |540 ++-------------------------------------------------------------------+
iostat.sdd.avgrq-sz
720 ++-------------------------------------------------------------------+
| O |
700 O+ O O O O O O O O |
680 ++ O O O O O O O O O O O O |
| O O O O |
660 ++ |
640 ++ |
| |
620 ++ |
600 ++ |
| |
580 ++ .*..*.*..*.*.. .*..*..*.*.. .*.. .*.*..*.*.. .*.. .*..*
560 *+.* * *.*.. .*..* *. * * |
| * |540 ++-------------------------------------------------------------------+
iostat.sdd.avgqu-sz
120 ++-------------------------------------------------------------------+
| O O O O O O O O O O O O |
115 O+ O O O O O O O O O O O O O |
| |
| |
110 ++ |
| |
105 ++ |
| |
100 ++ *. *.. .*. .*.*..*. .*.*..|
|.. *..*.*..*.*..*. .*..*. .*.. + *. *. *. *
* *. *..* .*..* |
95 ++ * |
| |
90 ++-------------------------------------------------------------------+
iostat.sdh.avgrq-sz720 ++-------------------------------------------------------------------+
| O O |
700 O+ O O O O O O O |
680 ++ O O O O O O O O O O O O |
| O O O O |
660 ++ |
640 ++ |
| |
620 ++ |
600 ++ |
| |
580 ++.*.*..*.*.. .*..*.*.. .*.*..*. .*.*.. .*. .*. .*.*.. .*..*
560 *+ * *. *..*.*. *. *. *. * |
| |540 ++-------------------------------------------------------------------+
iostat.sdh.avgqu-sz
120 ++-------------------------------------------------------------------+
| O O O O O O O O |
| O O O O O O O O O O |
115 O+ O O O O O O |
| O |
| |
110 ++ |
| |
105 ++ |
| |
| |
100 ++.*. *.. .*.. *. .*. .*..|
*. *..*. .*.. .*.. + * .. *. *..*.*..* *
| *..* *.*..*..* .*..* * |
95 ++-----------------------------*-------------------------------------+
iostat.sdi.avgrq-sz720 ++-------------------------------------------------------------------+
| O O |
700 O+ O O O O O O O O |
680 ++ O O O O O O O O O O |
| O O O O O |
660 ++ |
640 ++ |
| |
620 ++ |
600 ++ |
| .*. .*.. .*..|
580 ++ .*.*..*.*..*. .*. *..*. *.. .*..*..*.*..*. .* * |
560 *+.*.*. *. *.. + * *. *
| * |540 ++-------------------------------------------------------------------+
iostat.sdi.avgqu-sz
125 ++-------------------------------------------------------------------+
| |
120 ++ O |
| O O O O O O O |
O O O O O O O O O O O |
115 ++ O O O O O O |
| O |
110 ++ |
| |
105 ++ |
| |
| .*.. .*..|
100 ++ *.*..*.*..*. .*.. .*..*.*..*..*.*..* *. .* |
|.. *..*.*. *.*..*.*..* *. *
95 *+-------------------------------------------------------------------+
iostat.sdj.avgqu-sz120 ++---------------------------------------------------------O---------+
| O O O O O O O |
115 O+ O O O O O O O O O O O O O |
| O O O O |
| |
110 ++ |
| |
105 ++ |
| * |
100 ++.*.*..*. *.. + + .*.. *..|
*. *..*. + *.*.. + *..*.*..* + *
| *..*. .*..*.*..*.*..* * * |
95 ++ *. |
| |
90 ++-------------------------------------------------------------------+
iostat.sdk.avgrq-sz720 ++-------------------------------------------------------------------+
| |
700 ++ O O O O O O O O O |
680 O+ O O O O O O |
| O O O O O O O O O |
660 ++ O |
| |
640 ++ |
| |
620 ++ |
600 ++ |
| |
580 ++.*. .*.. .*.. *.. *. .*.. .*. .*. .*.. .*..*
*. *..* * *. .. *. .. *.. .*..* *. *. *..* * |560 ++------------------*-------*-------*--------------------------------+
iostat.sdk.avgqu-sz
125 ++-------------------------------------------------------------------+
| |
120 ++ O |
O O O O O O O O O O O O |
115 ++ O O O O O O O O O O O O O |
| |
110 ++ |
| |
105 ++ |
| .*.. |
100 ++.*. .*. .*.. .*..* .*.*..*. .*
*. *. *..*. .*..*. .* * *. *..*.*..*.*. |
95 ++ *..*.*. *. |
| |
90 ++-------------------------------------------------------------------+
iostat.sdl.avgrq-sz720 ++-------------------------------------------------------------------+
O |
700 ++ O O O O O O |
680 ++ O O O O O O O O O O O O O O |
| O O O O O |
660 ++ |
640 ++ |
| |
620 ++ |
600 ++ |
| *..|
580 ++ .*..*.*..*.*..* *..*.*..*. .*.*.. .*. .*. .*.*.. + *
560 *+.* + .. *..*.*. *. *. *. * |
| * |540 ++-------------------------------------------------------------------+
iostat.sdl.avgqu-sz
125 ++-------------------------------------------------------------------+
| |
120 ++ O O |
| O O O O O O O O |
O O O O O O O O O O O O O |
115 ++ O O O |
| |
110 ++ |
| |
105 ++ |
| .*. |
| .*..*. .*.*.. .*. .*.*. *.. .*..|
100 ++.* *..* .*..* *.*.. .*. *. *. * *
*. + .*.*. + .. * |
95 ++-------------*------------*----------------------------------------+
iostat.sdm.avgrq-sz700 ++---------------------O-------O-------------------------------------+
O O O O O O O O O O O O |
680 ++ O O O O O O O O O |
| O O |
660 ++ O |
| |
640 ++ |
| |
620 ++ |
| |
600 ++ |
| .*.. *..|
580 ++ *.. .*..* .*.. .*.*..*. .*.*.. .*. .*. .*.*.. + *
| + * * *. *.. .*. *. *. *. * |560 *+-*--------------------------------*--------------------------------+
iostat.sdn.avgrq-sz
700 ++------------------O----------O-------------------------------------+
O O O O O O O O O |
680 ++ O O O O O O O O O O O O O O |
660 ++ O |
| |
640 ++ |
| |
620 ++ |
| |
600 ++ |
580 ++ .*..* .*..|
*..*. .*.*.. .*.*. + .*. .*.. .*.. .* *. .*..* *
560 ++ *. *.*. *. *..* * *. + .. *..* |
| * |540 ++-------------------------------------------------------------------+
Quoted 13 lines · On Fri, 2014-01-03 at 23:54 +-0800, [email protected] wrote:
Hi Josef,
FYI. We are doing 0day performance tests and happen to notice that btrfs write throughput increased considerably during v3.10-11 time frame:
v3.10 v3.11 v3.12 v3.13-rc6
--------------- ------------------------- ------------------------- -------------------------50619 ~ 1% +-17.0% 59209 ~ 2% +-18.8% 60159 ~ 2% +-20.5% 61007 ~ 0% lkp-ws02/micro/dd-write/11HDD-JBOD-cfq-btrfs-1dd 50619 +-17.0% 59209 +-18.8% 60159 +-20.5% 61007 TOTAL iostat.sdd.wkB/sand it's contributed by
Quoted 7 lines
commit
1be41b78bc688fc634bf30965d2be692c99fd11dAuthor: Josef Bacik <[email protected]> AuthorDate: Wed Jun 12 13:56:06 2013 -0400 Commit: Josef Bacik <[email protected]> CommitDate: Mon Jul 1 08:52:28 2013 -0400Btrfs: fix transaction throttling for delayed refs
Bonus points for increasing the performance on purpose. Thanks for running these Wu.
-chris