ceph: add min/max/stdev latency support
v4 cover ceph: add min/max/stdev latency support
From: Xiubo Li <[email protected]>
Changed in V4:
- fix the 32-bit arches div errors by using DIV64_U64_ROUND_CLOSEST instead. [1/4]
- rebase and combine the stdev patch series [3/4][4/4]
- remove the sum latency showing, which makes no sense for debugging, if it is really needed in some case then just do (avg * total) in userland. [4/4]
- switch {read/write/metadata}_latency_sum to atomic type since it will be readed very time when updating the latencies to calculate the stdev. [4/4]
Changed in V2:
- switch spin lock to cmpxchg [1/4]
Changed in V3:
- add the __update_min/max_latency helpers [1/4]
# cat /sys/kernel/debug/ceph/0f923fe5-00e6-4866-bf01-2027cb75e94b.client4150/metrics item total avg_lat(us) min_lat(us) max_lat(us) stdev(us) ----------------------------------------------------------------------------------- read 2312 9000 1000 100000 607.4 write 21777 925000 2000 44551000 29700.3 metadata 6 4179000 1000 21414000 19590.8
item total miss hit
-------------------------------------------------
d_lease 2 0 11 caps 2 14 398418
Xiubo Li (4):
ceph: switch to DIV64_U64_ROUND_CLOSEST to support 32-bit arches
ceph: add min/max latency support for read/write/metadata metrics
ceph: move the metric helpers into one separate file
ceph: add standard deviation support for read/write/metadata perf
metric
fs/ceph/Makefile | 2 +- fs/ceph/debugfs.c | 89 ++++++++++++++++++------ fs/ceph/mds_client.c | 83 +--------------------- fs/ceph/metric.c | 190 +++++++++++++++++++++++++++++++++++++++++++++++++++ fs/ceph/metric.h | 79 +++++++++++---------- 5 files changed, 297 insertions(+), 146 deletions(-) create mode 100644 fs/ceph/metric.c
Signature · 1 line
1.8.3.1
PATCH v4 1/4 ceph: switch to DIV64_U64_ROUND_CLOSEST to support 32-bit arches
From: Xiubo Li <[email protected]>
fs/ceph/debugfs.c:140: undefined reference to `__divdi3' Use math64 helpers to avoid 64-bit div on 32-bit arches.
--- fs/ceph/debugfs.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
fs/ceph/debugfs.c +3 -3
@@ -137,19 +137,19 @@ static int metric_show(struct seq_file *s, void *p) total =percpu_counter_sum (&mdsc->metric.total_reads); sum =percpu_counter_sum (&mdsc->metric.read_latency_sum); sum =jiffies_to_usecs (sum); - avg = total ? sum / total :0 ; + avg = total ?DIV64_U64_ROUND_CLOSEST (sum, total) :0 ;seq_printf (s,"%-14s%-12lld%-16lld%lld\n" ,"read" , total, sum, avg); total =percpu_counter_sum (&mdsc->metric.total_writes); sum =percpu_counter_sum (&mdsc->metric.write_latency_sum); sum =jiffies_to_usecs (sum); - avg = total ? sum / total :0 ; + avg = total ?DIV64_U64_ROUND_CLOSEST (sum, total) :0 ;seq_printf (s,"%-14s%-12lld%-16lld%lld\n" ,"write" , total, sum, avg); total =percpu_counter_sum (&mdsc->metric.total_metadatas); sum =percpu_counter_sum (&mdsc->metric.metadata_latency_sum); sum =jiffies_to_usecs (sum); - avg = total ? sum / total :0 ; + avg = total ?DIV64_U64_ROUND_CLOSEST (sum, total) :0 ;seq_printf (s,"%-14s%-12lld%-16lld%lld\n" ,"metadata" , total, sum, avg);seq_printf (s,"\n" );
Signature · 1 line
1.8.3.1
PATCH v4 2/4 ceph: add min/max latency support for read/write/metadata metrics
In reply to [email protected]From: Xiubo Li <[email protected]>
These will be very useful help diagnose problems.
URL: https://tracker.ceph.com/issues/44533
--- fs/ceph/debugfs.c | 27 +++++++++++++++++++++------ fs/ceph/mds_client.c | 9 +++++++++ fs/ceph/metric.h | 51 ++++++++++++++++++++++++++++++++++++++++++++++++--- 3 files changed, 78 insertions(+), 9 deletions(-)
fs/ceph/debugfs.c +21 -6
@@ -129,28 +129,43 @@ static int metric_show(struct seq_file *s, void *p)struct ceph_fs_client *fsc = s->private;struct ceph_mds_client *mdsc = fsc->mdsc;int i, nr_caps =0 ; -s64 total, sum, avg =0 ; +s64 total, sum, avg =0 , min, max; -seq_printf (s,"item total sum_lat(us) avg_lat(us)\n" ); -seq_printf (s,"-----------------------------------------------------\n" ); +seq_printf (s,"item total sum_lat(us) avg_lat(us) min_lat(us) max_lat(us)\n" ); +seq_printf (s,"-------------------------------------------------------------------------------------\n" ); total =percpu_counter_sum (&mdsc->metric.total_reads); sum =percpu_counter_sum (&mdsc->metric.read_latency_sum); sum =jiffies_to_usecs (sum); avg = total ?DIV64_U64_ROUND_CLOSEST (sum, total) :0 ; -seq_printf (s,"%-14s%-12lld%-16lld%lld\n" ,"read" , total, sum, avg); + min =atomic64_read (&mdsc->metric.read_latency_min); + min =jiffies_to_usecs (min ==S64_MAX ?0 : min); + max =atomic64_read (&mdsc->metric.read_latency_max); + max =jiffies_to_usecs (max); +seq_printf (s,"%-14s%-12lld%-16lld%-16lld%-16lld%lld\n" ,"read" , + total, sum, avg, min, max); total =percpu_counter_sum (&mdsc->metric.total_writes); sum =percpu_counter_sum (&mdsc->metric.write_latency_sum); sum =jiffies_to_usecs (sum); avg = total ?DIV64_U64_ROUND_CLOSEST (sum, total) :0 ; -seq_printf (s,"%-14s%-12lld%-16lld%lld\n" ,"write" , total, sum, avg); + min =atomic64_read (&mdsc->metric.write_latency_min); + min =jiffies_to_usecs (min ==S64_MAX ?0 : min); + max =atomic64_read (&mdsc->metric.write_latency_max); + max =jiffies_to_usecs (max); +seq_printf (s,"%-14s%-12lld%-16lld%-16lld%-16lld%lld\n" ,"write" , + total, sum, avg, min, max); total =percpu_counter_sum (&mdsc->metric.total_metadatas); sum =percpu_counter_sum (&mdsc->metric.metadata_latency_sum); sum =jiffies_to_usecs (sum); avg = total ?DIV64_U64_ROUND_CLOSEST (sum, total) :0 ; -seq_printf (s,"%-14s%-12lld%-16lld%lld\n" ,"metadata" , total, sum, avg); + min =atomic64_read (&mdsc->metric.metadata_latency_min); + min =jiffies_to_usecs (min ==S64_MAX ?0 : min); + max =atomic64_read (&mdsc->metric.metadata_latency_max); + max =jiffies_to_usecs (max); +seq_printf (s,"%-14s%-12lld%-16lld%-16lld%-16lld%lld\n" ,"metadata" , + total, sum, avg, min, max);seq_printf (s,"\n" );seq_printf (s,"item total miss hit\n" );
fs/ceph/mds_client.c +9 -0
@@ -4358,6 +4358,9 @@ static int ceph_mdsc_metric_init(struct ceph_client_metric *metric)if (ret) goto err_read_latency_sum; +atomic64_set (&metric->read_latency_min,S64_MAX ); +atomic64_set (&metric->read_latency_max,0 ); + ret =percpu_counter_init (&metric->total_writes,0 ,GFP_KERNEL );if (ret) goto err_total_writes;
@@ -4366,6 +4369,9 @@ static int ceph_mdsc_metric_init(struct ceph_client_metric *metric)if (ret) goto err_write_latency_sum; +atomic64_set (&metric->write_latency_min,S64_MAX ); +atomic64_set (&metric->write_latency_max,0 ); + ret =percpu_counter_init (&metric->total_metadatas,0 ,GFP_KERNEL );if (ret) goto err_total_metadatas;
@@ -4374,6 +4380,9 @@ static int ceph_mdsc_metric_init(struct ceph_client_metric *metric)if (ret) goto err_metadata_latency_sum; +atomic64_set (&metric->metadata_latency_min,S64_MAX ); +atomic64_set (&metric->metadata_latency_max,0 ); +return 0 ; err_metadata_latency_sum:
fs/ceph/metric.h +48 -3
103 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH v4 3/4 ceph: move the metric helpers into one separate file
In reply to [email protected]From: Xiubo Li <[email protected]>
The inline is not proper any more dues to the helpers becomes larger.
URL: https://tracker.ceph.com/issues/44534
--- fs/ceph/Makefile | 2 +- fs/ceph/mds_client.c | 92 +-------------------------- fs/ceph/metric.c | 173 +++++++++++++++++++++++++++++++++++++++++++++++++++ fs/ceph/metric.h | 87 +++++--------------------- 4 files changed, 190 insertions(+), 164 deletions(-) create mode 100644 fs/ceph/metric.c
fs/ceph/Makefile +1 -1
@@ -8,7 +8,7 @@ obj-$(CONFIG_CEPH_FS) += ceph.o ceph-y := super.o inode.o dir.o file.o locks.o addr.o ioctl.o \ export.o caps.o snap.o xattr.o quota.o io.o \ mds_client.o mdsmap.o strings.o ceph_frag.o \ - debugfs.o util.o + debugfs.o util.o metric.o ceph-$(CONFIG_CEPH_FSCACHE) += cache.o ceph-$(CONFIG_CEPH_FS_POSIX_ACL) += acl.o
fs/ceph/mds_client.c +1 -91
104 lines not on this page · raw message
fs/ceph/metric.c new file +173 -0
173 lines not on this page · raw message
fs/ceph/metric.h +15 -72
97 lines not on this page · raw message
Signature · 1 line
1.8.3.1
PATCH v4 4/4 ceph: add standard deviation support for read/write/metadata perf metric
In reply to [email protected]From: Xiubo Li <[email protected]>
Switch {read/write/metadata}_latency_sum to atomic type and remove {read/write/metadata}_latency_sum showing in the debugfs, which makes no sense.
URL: https://tracker.ceph.com/issues/44534
--- fs/ceph/debugfs.c | 88 +++++++++++++++++++++++++++++++++---------------- fs/ceph/metric.c | 99 ++++++++++++++++++++++++++++++++----------------------- fs/ceph/metric.h | 18 ++++++---- 3 files changed, 129 insertions(+), 76 deletions(-)
fs/ceph/debugfs.c +59 -29
@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0 #include <linux/ceph/ceph_debug.h> +#include <linux/kernel.h> #include <linux/device.h> #include <linux/slab.h> #include <linux/module.h>
@@ -124,48 +125,77 @@ static int mdsc_show(struct seq_file *s, void *p)return 0 ; } +static s64 get_avg (atomic64_t *totalp,atomic64_t *sump,spinlock_t *lockp, +s64 *total) +{ +s64 n, sum, avg =0 ; + +spin_lock (lockp); + n =atomic64_read (totalp); + sum =atomic64_read (sump); +spin_unlock (lockp); + +if (likely (n)) + avg =DIV64_U64_ROUND_CLOSEST (sum, n); + + *total = n; +return avg; +} + +#define METRIC (name, total, avg, min, max, sq) { \ + s64 _total, _avg, _min, _max, _sq, _st, _re = 0; \ + _avg = jiffies_to_usecs(avg); \ + _min = jiffies_to_usecs(min == S64_MAX ? 0 : min); \ + _max = jiffies_to_usecs(max); \ + _total = total - 1; \ + _sq = _total > 0 ? DIV64_U64_ROUND_CLOSEST(sq, _total) : 0; \ + _sq = jiffies_to_usecs(_sq); \ + _st = int_sqrt64(_sq); \ + if (_st > 0) { \ + _re = 5 * (_sq - (_st * _st)); \ + _re = _re > 0 ? _re - 1 : 0; \ + _re = _st > 0 ? div64_s64(_re, _st) : 0; \ + } \ + seq_printf(s, "%-14s%-12lld%-16lld%-16lld%-16lld%lld.%lld\n", \ + name, total, _avg, _min, _max, _st, _re); \ +} +static int metric_show (struct seq_file *s,void *p) {struct ceph_fs_client *fsc = s->private;struct ceph_mds_client *mdsc = fsc->mdsc;int i, nr_caps =0 ; -s64 total, sum, avg =0 , min, max; +s64 total, avg, min, max, sq; -seq_printf (s,"item total sum_lat(us) avg_lat(us) min_lat(us) max_lat(us)\n" ); -seq_printf (s,"-------------------------------------------------------------------------------------\n" ); +seq_printf (s,"item total avg_lat(us) min_lat(us) max_lat(us) stdev(us)\n" ); +seq_printf (s,"-----------------------------------------------------------------------------------\n" ); - total =percpu_counter_sum (&mdsc->metric.total_reads); - sum =percpu_counter_sum (&mdsc->metric.read_latency_sum); - sum =jiffies_to_usecs (sum); - avg = total ?DIV64_U64_ROUND_CLOSEST (sum, total) :0 ; + avg =get_avg (&mdsc->metric.total_reads, + &mdsc->metric.read_latency_sum, + &mdsc->metric.read_latency_lock, + &total); min =atomic64_read (&mdsc->metric.read_latency_min); - min =jiffies_to_usecs (min ==S64_MAX ?0 : min); max =atomic64_read (&mdsc->metric.read_latency_max); - max =jiffies_to_usecs (max); -seq_printf (s,"%-14s%-12lld%-16lld%-16lld%-16lld%lld\n" ,"read" , - total, sum, avg, min, max); - - total =percpu_counter_sum (&mdsc->metric.total_writes); - sum =percpu_counter_sum (&mdsc->metric.write_latency_sum); - sum =jiffies_to_usecs (sum); - avg = total ?DIV64_U64_ROUND_CLOSEST (sum, total) :0 ; + sq =percpu_counter_sum (&mdsc->metric.read_latency_sq_sum); +METRIC ("read" , total, avg, min, max, sq); + + avg =get_avg (&mdsc->metric.total_writes, + &mdsc->metric.write_latency_sum, + &mdsc->metric.write_latency_lock, + &total); min =atomic64_read (&mdsc->metric.write_latency_min); - min =jiffies_to_usecs (min ==S64_MAX ?0 : min); max =atomic64_read (&mdsc->metric.write_latency_max); - max =jiffies_to_usecs (max); -seq_printf (s,"%-14s%-12lld%-16lld%-16lld%-16lld%lld\n" ,"write" , - total, sum, avg, min, max); - - total =percpu_counter_sum (&mdsc->metric.total_metadatas); - sum =percpu_counter_sum (&mdsc->metric.metadata_latency_sum); - sum =jiffies_to_usecs (sum); - avg = total ?DIV64_U64_ROUND_CLOSEST (sum, total) :0 ; + sq =percpu_counter_sum (&mdsc->metric.write_latency_sq_sum); +METRIC ("write" , total, avg, min, max, sq); + + avg =get_avg (&mdsc->metric.total_metadatas, + &mdsc->metric.metadata_latency_sum, + &mdsc->metric.metadata_latency_lock, + &total); min =atomic64_read (&mdsc->metric.metadata_latency_min); - min =jiffies_to_usecs (min ==S64_MAX ?0 : min); max =atomic64_read (&mdsc->metric.metadata_latency_max); - max =jiffies_to_usecs (max); -seq_printf (s,"%-14s%-12lld%-16lld%-16lld%-16lld%lld\n" ,"metadata" , - total, sum, avg, min, max); + sq =percpu_counter_sum (&mdsc->metric.metadata_latency_sq_sum); +METRIC ("metadata" , total, avg, min, max, sq);seq_printf (s,"\n" );seq_printf (s,"item total miss hit\n" );
fs/ceph/metric.c +58 -41
161 lines not on this page · raw message
fs/ceph/metric.h +12 -6
@@ -14,18 +14,24 @@ struct ceph_client_metric {struct percpu_counter i_caps_hit;struct percpu_counter i_caps_mis; -struct percpu_counter total_reads; -struct percpu_counter read_latency_sum; +struct percpu_counter read_latency_sq_sum; +spinlock_t read_latency_lock; +atomic64_t total_reads; +atomic64_t read_latency_sum;atomic64_t read_latency_min;atomic64_t read_latency_max; -struct percpu_counter total_writes; -struct percpu_counter write_latency_sum; +struct percpu_counter write_latency_sq_sum; +spinlock_t write_latency_lock; +atomic64_t total_writes; +atomic64_t write_latency_sum;atomic64_t write_latency_min;atomic64_t write_latency_max; -struct percpu_counter total_metadatas; -struct percpu_counter metadata_latency_sum; +struct percpu_counter metadata_latency_sq_sum; +spinlock_t metadata_latency_lock; +atomic64_t total_metadatas; +atomic64_t metadata_latency_sum;atomic64_t metadata_latency_min;atomic64_t metadata_latency_max; };
Signature · 1 line
1.8.3.1
Ilya Dryomov
maintainerQuoted 40 lines · On Wed, Mar 18, 2020 at 6:46 AM <[email protected]> wrote:
From: Xiubo Li <[email protected]>
Changed in V4:
- fix the 32-bit arches div errors by usingDIV64_U64_ROUND_CLOSESTinstead. [1/4]
- rebase and combine the stdev patch series [3/4][4/4]
- remove the sum latency showing, which makes no sense for debugging, if it is really needed in some case then just do (avg * total) in userland. [4/4]
- switch {read/write/metadata}_latency_sum to atomic type since it will be readed very time when updating the latencies to calculate the stdev. [4/4]Changed in V2:
- switch spin lock to cmpxchg [1/4]Changed in V3:
- add the __update_min/max_latency helpers [1/4]# cat /sys/kernel/debug/ceph/0f923fe5-00e6-4866-bf01-2027cb75e94b.client4150/metrics item total avg_lat(us) min_lat(us) max_lat(us) stdev(us) ----------------------------------------------------------------------------------- read 2312 9000 1000 100000 607.4 write 21777 925000 2000 44551000 29700.3 metadata 6 4179000 1000 21414000 19590.8item total miss hit
-------------------------------------------------d_lease 2 0 11 caps 2 14 398418Xiubo Li (4):
ceph: switch toDIV64_U64_ROUND_CLOSESTto support 32-bit arches
ceph: add min/max latency support for read/write/metadata metrics
ceph: move the metric helpers into one separate file
ceph: add standard deviation support for read/write/metadata perf
metric
Quoted 7 lines
fs/ceph/Makefile | 2 +- fs/ceph/debugfs.c | 89 ++++++++++++++++++------ fs/ceph/mds_client.c | 83 +--------------------- fs/ceph/metric.c | 190 +++++++++++++++++++++++++++++++++++++++++++++++++++ fs/ceph/metric.h | 79 +++++++++++---------- 5 files changed, 297 insertions(+), 146 deletions(-) create mode 100644 fs/ceph/metric.c
Hi Xiubo,
I think these additions need to be merged with your previous series, so that the history is clean. Ideally the whole thing would start with a single patch adding all of the metrics infrastructure to metric.[ch], followed by patches introducing new metrics and ceph_update_*() calls.
Related metrics and ceph_update_*() calls should be added together. No point in splitting read and write OSD latency in two patches as they touch the same functions in addr.c and file.c.
Thanks,
Ilya
Quoted 49 lines · Ilya Dryomov
Quoted 39 lines · On Wed, Mar 18, 2020 at 6:46 AM <[email protected]> wrote:
From: Xiubo Li <[email protected]>
Changed in V4:
- fix the 32-bit arches div errors by usingDIV64_U64_ROUND_CLOSESTinstead. [1/4]
- rebase and combine the stdev patch series [3/4][4/4]
- remove the sum latency showing, which makes no sense for debugging, if it is really needed in some case then just do (avg * total) in userland. [4/4]
- switch {read/write/metadata}_latency_sum to atomic type since it will be readed very time when updating the latencies to calculate the stdev. [4/4]Changed in V2:
- switch spin lock to cmpxchg [1/4]Changed in V3:
- add the __update_min/max_latency helpers [1/4]# cat /sys/kernel/debug/ceph/0f923fe5-00e6-4866-bf01-2027cb75e94b.client4150/metrics item total avg_lat(us) min_lat(us) max_lat(us) stdev(us) ----------------------------------------------------------------------------------- read 2312 9000 1000 100000 607.4 write 21777 925000 2000 44551000 29700.3 metadata 6 4179000 1000 21414000 19590.8item total miss hit
-------------------------------------------------d_lease 2 0 11 caps 2 14 398418Xiubo Li (4):
ceph: switch toDIV64_U64_ROUND_CLOSESTto support 32-bit arches
ceph: add min/max latency support for read/write/metadata metrics
ceph: move the metric helpers into one separate file
ceph: add standard deviation support for read/write/metadata perf
metricQuoted 7 lines
fs/ceph/Makefile | 2 +- fs/ceph/debugfs.c | 89 ++++++++++++++++++------ fs/ceph/mds_client.c | 83 +--------------------- fs/ceph/metric.c | 190 +++++++++++++++++++++++++++++++++++++++++++++++++++ fs/ceph/metric.h | 79 +++++++++++---------- 5 files changed, 297 insertions(+), 146 deletions(-)create mode 100644
fs/ceph/metric.cHi Xiubo,
Quoted 8 lines
I think these additions need to be merged with your previous series, so that the history is clean. Ideally the whole thing would start with a single patch adding all of the metrics infrastructure to metric.[ch], followed by patches introducing new metrics and ceph_update_*() calls.
Related metrics and ceph_update_*() calls should be added together. No point in splitting read and write OSD latency in two patches as they touch the same functions in
addr.candfile.c.
Hi Ilya,
Yeah, it makes sense and I will merge all the related patch series about the metric and post it again.
Thanks
BRs
Quoted 4 lines
Thanks,
Ilya
Quoted 55 lines · Xiubo Li
Quoted 54 lines · Ilya Dryomov
Quoted 39 lines · On Wed, Mar 18, 2020 at 6:46 AM <[email protected]> wrote:
From: Xiubo Li <[email protected]>
Changed in V4:
- fix the 32-bit arches div errors by usingDIV64_U64_ROUND_CLOSESTinstead. [1/4]
- rebase and combine the stdev patch series [3/4][4/4]
- remove the sum latency showing, which makes no sense for debugging, if it is really needed in some case then just do (avg * total) in userland. [4/4]
- switch {read/write/metadata}_latency_sum to atomic type since it will be readed very time when updating the latencies to calculate the stdev. [4/4]Changed in V2:
- switch spin lock to cmpxchg [1/4]Changed in V3:
- add the __update_min/max_latency helpers [1/4]# cat /sys/kernel/debug/ceph/0f923fe5-00e6-4866-bf01-2027cb75e94b.client4150/metrics item total avg_lat(us) min_lat(us) max_lat(us) stdev(us) ----------------------------------------------------------------------------------- read 2312 9000 1000 100000 607.4 write 21777 925000 2000 44551000 29700.3 metadata 6 4179000 1000 21414000 19590.8item total miss hit
-------------------------------------------------d_lease 2 0 11 caps 2 14 398418Xiubo Li (4):
ceph: switch toDIV64_U64_ROUND_CLOSESTto support 32-bit arches
ceph: add min/max latency support for read/write/metadata metrics
ceph: move the metric helpers into one separate file
ceph: add standard deviation support for read/write/metadata perf
metricQuoted 7 lines
fs/ceph/Makefile | 2 +- fs/ceph/debugfs.c | 89 ++++++++++++++++++------ fs/ceph/mds_client.c | 83 +--------------------- fs/ceph/metric.c | 190 +++++++++++++++++++++++++++++++++++++++++++++++++++ fs/ceph/metric.h | 79 +++++++++++---------- 5 files changed, 297 insertions(+), 146 deletions(-)create mode 100644
fs/ceph/metric.cHi Xiubo,
I think these additions need to be merged with your previous series, so that the history is clean. Ideally the whole thing would start with a single patch adding all of the metrics infrastructure to metric.[ch], followed by patches introducing new metrics and ceph_update_*() calls.
Quoted 9 lines
Quoted 3 lines
Related metrics and ceph_update_*() calls should be added together. No point in splitting read and write OSD latency in two patches as they touch the same functions in
addr.candfile.c.Hi Ilya,
Yeah, it makes sense and I will merge all the related patch series about the metric and post it again.
Sounds good. I've gone ahead and dropped all of the metrics patches from the "testing" branch for now. Please resend the whole series and I'll re-merge them.
Thanks,
Signature · 1 line
Jeff Layton <[email protected]>
Quoted 59 lines · Jeff Layton
Quoted 58 lines · Xiubo Li
Quoted 57 lines · Ilya Dryomov
Quoted 39 lines · On Wed, Mar 18, 2020 at 6:46 AM <[email protected]> wrote:
From: Xiubo Li <[email protected]>
Changed in V4:
- fix the 32-bit arches div errors by usingDIV64_U64_ROUND_CLOSESTinstead. [1/4]
- rebase and combine the stdev patch series [3/4][4/4]
- remove the sum latency showing, which makes no sense for debugging, if it is really needed in some case then just do (avg * total) in userland. [4/4]
- switch {read/write/metadata}_latency_sum to atomic type since it will be readed very time when updating the latencies to calculate the stdev. [4/4]Changed in V2:
- switch spin lock to cmpxchg [1/4]Changed in V3:
- add the __update_min/max_latency helpers [1/4]# cat /sys/kernel/debug/ceph/0f923fe5-00e6-4866-bf01-2027cb75e94b.client4150/metrics item total avg_lat(us) min_lat(us) max_lat(us) stdev(us) ----------------------------------------------------------------------------------- read 2312 9000 1000 100000 607.4 write 21777 925000 2000 44551000 29700.3 metadata 6 4179000 1000 21414000 19590.8item total miss hit
-------------------------------------------------d_lease 2 0 11 caps 2 14 398418Xiubo Li (4):
ceph: switch toDIV64_U64_ROUND_CLOSESTto support 32-bit arches
ceph: add min/max latency support for read/write/metadata metrics
ceph: move the metric helpers into one separate file
ceph: add standard deviation support for read/write/metadata perf
metricQuoted 7 lines
fs/ceph/Makefile | 2 +- fs/ceph/debugfs.c | 89 ++++++++++++++++++------ fs/ceph/mds_client.c | 83 +--------------------- fs/ceph/metric.c | 190 +++++++++++++++++++++++++++++++++++++++++++++++++++ fs/ceph/metric.h | 79 +++++++++++---------- 5 files changed, 297 insertions(+), 146 deletions(-)create mode 100644
fs/ceph/metric.cHi Xiubo,
I think these additions need to be merged with your previous series, so that the history is clean. Ideally the whole thing would start with a single patch adding all of the metrics infrastructure to metric.[ch], followed by patches introducing new metrics and ceph_update_*() calls.
Related metrics and ceph_update_*() calls should be added together. No point in splitting read and write OSD latency in two patches as they touch the same functions in
addr.candfile.c.
Quoted 8 lines
Quoted 5 lines
Hi Ilya,
Yeah, it makes sense and I will merge all the related patch series about the metric and post it again.
Sounds good. I've gone ahead and dropped all of the metrics patches from the "testing" branch for now. Please resend the whole series and I'll re-merge them.
Sure, thanks Jeff.
BRs
Quoted 1 line
Thanks,