perf top overwrite mode
v3 cover perf top overwrite mode
From: Kan Liang <[email protected]>
perf_top__mmap_read has severe performance issue in Knights Landing/Mill, when monitoring in heavy load system. It costs several minutes to finish, which is unacceptable.
Currently, perf top is non overwrite mode. For non overwrite mode, it tries to read everything in the ringbuffer and doesn't pause the ringbuffer. Once there are lots of samples delivered persistently, the processing time could be very long. Also, the latest samples could be lost when the ringbuffer is full.
It's better to change it to overwrite mode, which takes a snapshot for the system by pausing the ringbuffer and could significantly reducing the processing time (from several minutes to several seconds). Also, the overwrite mode always keep the latest samples.
Patch 1-7: Introduce new interfaces for generic code to support
overwrite mode for one by one event read.
Discards stale interfaces.
The patches can be merged separately.
Patch 8-12: Add overwrite support to perf top.
Perf top should only support either overwrite or non-overwrite mode.
Switch default mode to overwrite mode
If kernel doesn't support overwrite mode, fall back to non-overwrite mode.
Changes since V2:
- Move duplicate 'map->prev' out of perf_mmap__read. Modify the perf_mmap__read_event accordingly.
- Introduce new interface perf_mmap__read_init to calculate the ringbuffer position
- Check perf_missing_features.write_backward
- Discard stale interfaces perf_mmap__read_backward and perf_mmap__read_catchup
Changes since V1:
- New patches 4-6
- Support both overwrite mode and non-overwrite mode. If kernel doesn't support default overwrite mode, fall back to non-overwrite mode.
Kan Liang (12):
perf evlist: remove stale mmap read for backward
perf mmap: factor out function to find ringbuffer position perf mmap: discard 'prev' in perf_mmap__read
perf mmap: introduce perf_mmap__read_done
perf mmap: introduce perf_mmap__read_event
perf test: update mmap read functions for backward-ring-buffer test perf mmap: discard legacy interface for mmap read
perf top: check per event overwrite term
perf evsel: expose perf_missing_features.write_backward perf top: add overwrite fall back
perf top: switch default mode to overwrite mode
perf top: check the latency of perf_top__mmap_read
tools/perf/builtin-top.c | 139 +++++++++++++++++++++++++--- tools/perf/tests/backward-ring-buffer.c | 7 +- tools/perf/ui/browsers/hists.c | 12 ++- tools/perf/util/evlist.c | 17 ---- tools/perf/util/evlist.h | 4 - tools/perf/util/evsel.c | 5 + tools/perf/util/evsel.h | 2 + tools/perf/util/mmap.c | 158 ++++++++++++++++++-------------- tools/perf/util/mmap.h | 10 +- 9 files changed, 244 insertions(+), 110 deletions(-)
Signature · 1 line
2.5.5
PATCH v3 1/12 perf evlist: remove stale mmap read for backward
From: Kan Liang <[email protected]>
perf_evlist__mmap_read_catchup and perf_evlist__mmap_read_backward are only for overwrite mode.
But they read the evlist->mmap buffer which is for non-overwrite mode.
It did not bring any serious problem yet, because there is no one use it.
Remove the unused interfaces.
--- tools/perf/util/evlist.c | 17 ----------------- tools/perf/util/evlist.h | 4 ---- 2 files changed, 21 deletions(-)
tools/perf/util/evlist.c +0 -17
@@ -714,28 +714,11 @@ union perf_event *perf_evlist__mmap_read_forward(struct perf_evlist *evlist, intreturn perf_mmap__read_forward (md); } -union perf_event *perf_evlist__mmap_read_backward (struct perf_evlist *evlist,int idx) -{ -struct perf_mmap *md = &evlist->mmap[idx]; - -/* -* No need to check messup for backward ring buffer: -* We can always read arbitrary long data from a backward -* ring buffer unless we forget to pause it before reading. -*/ -return perf_mmap__read_backward (md); -} -union perf_event *perf_evlist__mmap_read (struct perf_evlist *evlist,int idx) {return perf_evlist__mmap_read_forward (evlist, idx); } -void perf_evlist__mmap_read_catchup (struct perf_evlist *evlist,int idx) -{ -perf_mmap__read_catchup (&evlist->mmap[idx]); -} -void perf_evlist__mmap_consume (struct perf_evlist *evlist,int idx) {perf_mmap__consume (&evlist->mmap[idx], false);
tools/perf/util/evlist.h +0 -4
@@ -132,10 +132,6 @@ union perf_event *perf_evlist__mmap_read(struct perf_evlist *evlist, int idx);union perf_event *perf_evlist__mmap_read_forward (struct perf_evlist *evlist,int idx); -union perf_event *perf_evlist__mmap_read_backward (struct perf_evlist *evlist, -int idx); -void perf_evlist__mmap_read_catchup (struct perf_evlist *evlist,int idx); -void perf_evlist__mmap_consume (struct perf_evlist *evlist,int idx);int perf_evlist__open (struct perf_evlist *evlist);
Signature · 1 line
2.5.5
PATCH v3 2/12 perf mmap: factor out function to find ringbuffer position
In reply to [email protected]From: Kan Liang <[email protected]>
The perf record has specific codes to calculate the ringbuffer position for both overwrite and non-overwrite mode.
The perf top will support both modes later.
It is useful to make the specific codes generic.
Introduce a new interface perf_mmap__read_init() to find ringbuffer position.
Add a check for map->refcnt in perf_mmap__read_init(). 'size' is needed for both perf_mmap__read_init() and perf_mmap__push(). Have to calculate in each function.
--- tools/perf/util/mmap.c | 62 ++++++++++++++++++++++++++++++++++---------------- tools/perf/util/mmap.h | 2 ++ 2 files changed, 45 insertions(+), 19 deletions(-)
tools/perf/util/mmap.c +43 -19
@@ -267,41 +267,65 @@ static int overwrite_rb_find_range(void *buf, int mask, u64 head, u64 *start, u6return -1 ; } -int perf_mmap__push (struct perf_mmap *md,bool overwrite, -void *to,int push (void *to,void *buf,size_t size)) +/* +* Report the start and end of the available data in ringbuffer +*/ +int perf_mmap__read_init (struct perf_mmap *map,bool overwrite, +u64 *start,u64 *end) { -u64 head =perf_mmap__read_head (md); -u64 old = md->prev; -u64 end = head, start = old; -unsigned char *data = md->base + page_size; +u64 head =perf_mmap__read_head (map); +u64 old = map->prev; +unsigned char *data = map->base + page_size;unsigned long size; -void *buf; -int rc =0 ; - start = overwrite ? head : old; - end = overwrite ? old : head; +/* +* Check if event was unmapped due to a POLLHUP/POLLERR. +*/ +if (!refcount_read (&map->refcnt)) +return -EINVAL ; -if (start == end) -return 0 ; + *start = overwrite ? head : old; + *end = overwrite ? old : head; - size = end - start; -if (size > (unsigned long )(md->mask) +1 ) { +if (*start == *end) +return -EAGAIN ; + + size = *end - *start; +if (size > (unsigned long )(map->mask) +1 ) {if (!overwrite) {WARN_ONCE (1 ,"failed to keep up with mmap data. (warn only once)\n" ); - md->prev = head; -perf_mmap__consume (md, overwrite); -return 0 ; + map->prev = head; +perf_mmap__consume (map, overwrite); +return -EAGAIN ; }/* * Backward ring buffer is full. We still have a chance to read * most of data from it. */ -if (overwrite_rb_find_range (data, md->mask, head, &start, &end)) -return -1 ; +if (overwrite_rb_find_range (data, map->mask, head, start, end)) +return -EINVAL ; } +return 0 ; +} + +int perf_mmap__push (struct perf_mmap *md,bool overwrite, +void *to,int push (void *to,void *buf,size_t size)) +{ +u64 head =perf_mmap__read_head (md); +u64 end, start; +unsigned char *data = md->base + page_size; +unsigned long size; +void *buf; +int rc; + + rc =perf_mmap__read_init (md, overwrite, &start, &end); +if (rc <0 ) +return (rc == -EAGAIN ) ?0 :-1 ; + + size = end - start;if ((start & md->mask) + size != (end & md->mask)) { buf = &data[start & md->mask]; size = md->mask +1 - (start & md->mask);
tools/perf/util/mmap.h +2 -0
@@ -94,4 +94,6 @@ int perf_mmap__push(struct perf_mmap *md, bool backward,size_t perf_mmap__mmap_len (struct perf_mmap *map); +int perf_mmap__read_init (struct perf_mmap *map,bool overwrite, +u64 *start,u64 *end);#endif /*__PERF_MMAP_H */
Signature · 1 line
2.5.5
Re: [PATCH V3 02/12] perf mmap: factor out function to find ringbuffer position
Quoted 7 lines · On Thu, Dec 21, 2017 at 10:08:44AM -0800, [email protected] wrote:
From: Kan Liang <[email protected]>
The perf record has specific codes to calculate the ringbuffer position for both overwrite and non-overwrite mode.
The perf top will support both modes later.
It is useful to make the specific codes generic.
Quoted 5 lines
Introduce a new interface
perf_mmap__read_init()to find ringbuffer position.
Add a check formap->refcntinperf_mmap__read_init(). 'size' is needed for bothperf_mmap__read_init()andperf_mmap__push(). Have to calculate in each function.
it's 2 separate changes then plus 1 not mentioned in changelog.. could you please split this into separate patches:
- Introduce a new interface perf_mmap__read_init ...
- Add a check for map->refcnt in perf_mmap__read_init
- add new EAGAIN return value logic
thanks,
jirka
Quoted 110 lines
Signed-off-by Kan Liang <[email protected]>--- tools/perf/util/mmap.c | 62 ++++++++++++++++++++++++++++++++++---------------- tools/perf/util/mmap.h | 2 ++ 2 files changed, 45 insertions(+), 19 deletions(-)tools/perf/util/mmap.c +43
-19@@ -267,41 +267,65 @@ static int overwrite_rb_find_range(void *buf, int mask, u64 head, u64 *start, u6return -1 ; } -int perf_mmap__push (struct perf_mmap *md,bool overwrite, -void *to,int push (void *to,void *buf,size_t size)) +/* +* Report the start and end of the available data in ringbuffer +*/ +int perf_mmap__read_init (struct perf_mmap *map,bool overwrite, +u64 *start,u64 *end) { -u64 head =perf_mmap__read_head (md); -u64 old = md->prev; -u64 end = head, start = old; -unsigned char *data = md->base + page_size; +u64 head =perf_mmap__read_head (map); +u64 old = map->prev; +unsigned char *data = map->base + page_size;unsigned long size; -void *buf; -int rc =0 ; - start = overwrite ? head : old; - end = overwrite ? old : head; +/* +* Check if event was unmapped due to a POLLHUP/POLLERR. +*/ +if (!refcount_read (&map->refcnt)) +return -EINVAL ; -if (start == end) -return 0 ; + *start = overwrite ? head : old; + *end = overwrite ? old : head; - size = end - start; -if (size > (unsigned long )(md->mask) +1 ) { +if (*start == *end) +return -EAGAIN ; + + size = *end - *start; +if (size > (unsigned long )(map->mask) +1 ) {if (!overwrite) {WARN_ONCE (1 ,"failed to keep up with mmap data. (warn only once)\n" ); - md->prev = head; -perf_mmap__consume (md, overwrite); -return 0 ; + map->prev = head; +perf_mmap__consume (map, overwrite); +return -EAGAIN ; }/* * Backward ring buffer is full. We still have a chance to read * most of data from it. */ -if (overwrite_rb_find_range (data, md->mask, head, &start, &end)) -return -1 ; +if (overwrite_rb_find_range (data, map->mask, head, start, end)) +return -EINVAL ; } +return 0 ; +} + +int perf_mmap__push (struct perf_mmap *md,bool overwrite, +void *to,int push (void *to,void *buf,size_t size)) +{ +u64 head =perf_mmap__read_head (md); +u64 end, start; +unsigned char *data = md->base + page_size; +unsigned long size; +void *buf; +int rc; + + rc =perf_mmap__read_init (md, overwrite, &start, &end); +if (rc <0 ) +return (rc == -EAGAIN ) ?0 :-1 ; + + size = end - start;if ((start & md->mask) + size != (end & md->mask)) { buf = &data[start & md->mask]; size = md->mask +1 - (start & md->mask);tools/perf/util/mmap.h +2
-0@@ -94,4 +94,6 @@ int perf_mmap__push(struct perf_mmap *md, bool backward,size_t perf_mmap__mmap_len (struct perf_mmap *map); +int perf_mmap__read_init (struct perf_mmap *map,bool overwrite, +u64 *start,u64 *end);#endif /*__PERF_MMAP_H */ Signature · 1 line
2.5.5
RE: [PATCH V3 02/12] perf mmap: factor out function to find ringbuffer position
Quoted 15 lines · On Thu, Dec 21, 2017 at 10:08:44AM -0800, [email protected] wrote:
Quoted 7 lines
From: Kan Liang <[email protected]>
The perf record has specific codes to calculate the ringbuffer position for both overwrite and non-overwrite mode. The perf top will support both modes later.
It is useful to make the specific codes generic.Quoted 5 lines
Introduce a new interface
perf_mmap__read_init()to find ringbuffer position.
Add a check formap->refcntinperf_mmap__read_init(). 'size' is needed for bothperf_mmap__read_init()andperf_mmap__push(). Have to calculate in each function.
Quoted 7 lines
it's 2 separate changes then plus 1 not mentioned in changelog.. could you please split this into separate patches:
- Introduce a new interface
perf_mmap__read_init...
- Add a check formap->refcntinperf_mmap__read_init
- add new EAGAIN return value logic
Thanks for the comments, Jirka.
I will split the patch accordingly in V4.
Thanks,
Kan
Quoted 118 lines
thanks,
jirkatools/perf/util/mmap.c static int overwrite_rb_find_range(void *buf, int
Quoted 13 lines
Signed-off-by Kan Liang <[email protected]>--- tools/perf/util/mmap.c | 62 ++++++++++++++++++++++++++++++++++---------------- tools/perf/util/mmap.h | 2 ++ 2 files changed, 45 insertions(+), 19 deletions(-)tools/perf/util/mmap.c index renamed from tools/perf/util/mmap.c +0
-005076e6..3fd4f3c 100644
tools/perf/util/mmap.c +0
-0@@ -267,41 +267,65 @@ static int overwrite_rb_find_range(void *buf, intmask, u64 head, u64 *start, u6
Quoted 33 lines
tools/perf/util/mmap.c +14
-12return -1 ; } -int perf_mmap__push (struct perf_mmap *md,bool overwrite, -void *to,int push (void *to,void *buf,size_t size)) +/* +* Report the start and end of the available data in ringbuffer */ +int perf_mmap__read_init (struct perf_mmap *map,bool overwrite, +u64 *start,u64 *end) { -u64 head =perf_mmap__read_head (md); -u64 old = md->prev; -u64 end = head, start = old; -unsigned char *data = md->base + page_size; +u64 head =perf_mmap__read_head (map); +u64 old = map->prev; +unsigned char *data = map->base + page_size;unsigned long size; -void *buf; -int rc =0 ; - start = overwrite ? head : old; - end = overwrite ? old : head; +/* +* Check if event was unmapped due to a POLLHUP/POLLERR. +*/ +if (!refcount_read (&map->refcnt)) +return -EINVAL ; -if (start == end) -return 0 ; + *start = overwrite ? head : old; + *end = overwrite ? old : head;tools/perf/util/mmap.c static int overwrite_rb_find_range(void *buf, int mask, u64 head, u64 *start, u6
Quoted 10 lines
tools/perf/util/mmap.c +5
-2- size = end - start; -if (size > (unsigned long )(md->mask) +1 ) { +if (*start == *end) +return -EAGAIN ; + + size = *end - *start; +if (size > (unsigned long )(map->mask) +1 ) {if (!overwrite) { WARN_ONCE(1, "failed to keep up with mmap data.(warn only
Quoted 5 lines
tools/perf/util/mmap.c +0
-3once)\n"); - md->prev = head; -perf_mmap__consume (md, overwrite); -return 0 ;tools/perf/util/mmap.c static int overwrite_rb_find_range(void *buf, int mask, u64 head, u64 *start, u6
Quoted 10 lines
tools/perf/util/mmap.c +3
-1+ map->prev = head; +perf_mmap__consume (map, overwrite); +return -EAGAIN ; }/* * Backward ring buffer is full. We still have a chance to read * most of data from it. */ - if (overwrite_rb_find_range(data, md->mask, head, &start,&end))
tools/perf/util/mmap.c static int overwrite_rb_find_range(void *buf, int · line 307
Quoted 2 lines
tools/perf/util/mmap.c +1
-1-return -1 ; + if (overwrite_rb_find_range(data, map->mask, head, start,end))
Quoted 38 lines
tools/perf/util/mmap.c +18
-0+return -EINVAL ; } +return 0 ; +} + +int perf_mmap__push (struct perf_mmap *md,bool overwrite, +void *to,int push (void *to,void *buf,size_t size)) { +u64 head =perf_mmap__read_head (md); +u64 end, start; +unsigned char *data = md->base + page_size; +unsigned long size; +void *buf; +int rc; + + rc =perf_mmap__read_init (md, overwrite, &start, &end); +if (rc <0 ) +return (rc == -EAGAIN ) ?0 :-1 ; + + size = end - start;if ((start & md->mask) + size != (end & md->mask)) { buf = &data[start & md->mask]; size = md->mask + 1 - (start & md->mask); diff --git
a/tools/perf/util/mmap.hb/tools/perf/util/mmap.hindex d640273..abe9b9f 100644tools/perf/util/mmap.h +2
-0@@ -94,4 +94,6 @@ int perf_mmap__push(struct perf_mmap *md, bool backward, size_t perf_mmap__mmap_len(struct perf_mmap *map); +int perf_mmap__read_init (struct perf_mmap *map,bool overwrite, +u64 *start,u64 *end);#endif /*__PERF_MMAP_H */
--
2.5.5
Re: [PATCH V3 02/12] perf mmap: factor out function to find ringbuffer position
In reply to [email protected]On Thu, Dec 21, 2017 at 10:08:44AM -0800, [email protected] wrote:
SNIP
Quoted 30 lines
tools/perf/util/mmap.c +15
-10+/* +* Report the start and end of the available data in ringbuffer +*/ +int perf_mmap__read_init (struct perf_mmap *map,bool overwrite, +u64 *start,u64 *end) { -u64 head =perf_mmap__read_head (md); -u64 old = md->prev; -u64 end = head, start = old; -unsigned char *data = md->base + page_size; +u64 head =perf_mmap__read_head (map); +u64 old = map->prev; +unsigned char *data = map->base + page_size;unsigned long size; -void *buf; -int rc =0 ; - start = overwrite ? head : old; - end = overwrite ? old : head; +/* +* Check if event was unmapped due to a POLLHUP/POLLERR. +*/ +if (!refcount_read (&map->refcnt)) +return -EINVAL ; -if (start == end) -return 0 ; + *start = overwrite ? head : old; + *end = overwrite ? old : head;
tools/perf/util/mmap.c static int overwrite_rb_find_range(void *buf, int mask, u64 head, u64 *start, u6 · line 296
Quoted 10 lines
tools/perf/util/mmap.c +5
-2- size = end - start; -if (size > (unsigned long )(md->mask) +1 ) { +if (*start == *end) +return -EAGAIN ; + + size = *end - *start; +if (size > (unsigned long )(map->mask) +1 ) {if (!overwrite) {WARN_ONCE (1 ,"failed to keep up with mmap data. (warn only once)\n" );
I know you did not change this, but is this leg even possible in !overwrite mode? I think kernel will throw away the data, keep the head and wait for tail to be read by user..
jirka
Quoted 6 lines
tools/perf/util/mmap.c +3
-3- md->prev = head; -perf_mmap__consume (md, overwrite); -return 0 ; + map->prev = head; +perf_mmap__consume (map, overwrite); +return -EAGAIN ;
SNIP
RE: [PATCH V3 02/12] perf mmap: factor out function to find ringbuffer position
Quoted 47 lines · On Thu, Dec 21, 2017 at 10:08:44AM -0800, [email protected] wrote:
SNIP
Quoted 29 lines
tools/perf/util/mmap.c +15
-10+/* +* Report the start and end of the available data in ringbuffer +*/ +int perf_mmap__read_init (struct perf_mmap *map,bool overwrite, +u64 *start,u64 *end) { -u64 head =perf_mmap__read_head (md); -u64 old = md->prev; -u64 end = head, start = old; -unsigned char *data = md->base + page_size; +u64 head =perf_mmap__read_head (map); +u64 old = map->prev; +unsigned char *data = map->base + page_size;unsigned long size; -void *buf; -int rc =0 ; - start = overwrite ? head : old; - end = overwrite ? old : head; +/* +* Check if event was unmapped due to a POLLHUP/POLLERR. +*/ +if (!refcount_read (&map->refcnt)) +return -EINVAL ; -if (start == end) -return 0 ; + *start = overwrite ? head : old; + *end = overwrite ? old : head;tools/perf/util/mmap.c static int overwrite_rb_find_range(void *buf, int mask, u64 head, u64 *start, u6
Quoted 10 lines
tools/perf/util/mmap.c +5
-2- size = end - start; -if (size > (unsigned long )(md->mask) +1 ) { +if (*start == *end) +return -EAGAIN ; + + size = *end - *start; +if (size > (unsigned long )(map->mask) +1 ) {if (!overwrite) { WARN_ONCE(1, "failed to keep up with mmap data.(warn only once)\n");
Quoted 1 line
Quoted 3 lines
I know you did not change this, but is this leg even possible in !overwrite mode? I think kernel will throw away the data, keep the head and wait for tail to be read by user..
Right, it should not happen in !overwrite mode. I guess it's just sanity check.
It should not bring any problems.
I think I will still keep it for V4 if no objection?
Thanks,
Kan
Quoted 11 lines
jirka
Quoted 6 lines
tools/perf/util/mmap.c +3
-3- md->prev = head; -perf_mmap__consume (md, overwrite); -return 0 ; + map->prev = head; +perf_mmap__consume (map, overwrite); +return -EAGAIN ;SNIP
Re: [PATCH V3 02/12] perf mmap: factor out function to find ringbuffer position
Quoted 47 lines · Kan
Quoted 47 lines · On Thu, Dec 21, 2017 at 10:08:44AM -0800, [email protected] wrote:
SNIP
Quoted 29 lines
tools/perf/util/mmap.c +15
-10+/* +* Report the start and end of the available data in ringbuffer +*/ +int perf_mmap__read_init (struct perf_mmap *map,bool overwrite, +u64 *start,u64 *end) { -u64 head =perf_mmap__read_head (md); -u64 old = md->prev; -u64 end = head, start = old; -unsigned char *data = md->base + page_size; +u64 head =perf_mmap__read_head (map); +u64 old = map->prev; +unsigned char *data = map->base + page_size;unsigned long size; -void *buf; -int rc =0 ; - start = overwrite ? head : old; - end = overwrite ? old : head; +/* +* Check if event was unmapped due to a POLLHUP/POLLERR. +*/ +if (!refcount_read (&map->refcnt)) +return -EINVAL ; -if (start == end) -return 0 ; + *start = overwrite ? head : old; + *end = overwrite ? old : head;tools/perf/util/mmap.c static int overwrite_rb_find_range(void *buf, int mask, u64 head, u64 *start, u6
Quoted 10 lines
tools/perf/util/mmap.c +5
-2- size = end - start; -if (size > (unsigned long )(md->mask) +1 ) { +if (*start == *end) +return -EAGAIN ; + + size = *end - *start; +if (size > (unsigned long )(map->mask) +1 ) {if (!overwrite) { WARN_ONCE(1, "failed to keep up with mmap data.(warn only once)\n");
Quoted 1 line
Quoted 8 lines
Quoted 3 lines
I know you did not change this, but is this leg even possible in !overwrite mode? I think kernel will throw away the data, keep the head and wait for tail to be read by user..
Right, it should not happen in !overwrite mode. I guess it's just sanity check.
It should not bring any problems.
I think I will still keep it for V4 if no objection?
ok
jirka
PATCH v3 3/12 perf mmap: discard 'prev' in perf_mmap__read
In reply to [email protected]From: Kan Liang <[email protected]>
'start' and 'prev' are duplicate in perf_mmap__read()
Use 'map->prev' to replace 'start' in perf_mmap__read_*().
--- tools/perf/util/mmap.c | 28 ++++++++++------------------ 1 file changed, 10 insertions(+), 18 deletions(-)
tools/perf/util/mmap.c +10 -18
@@ -22,29 +22,27 @@ size_t perf_mmap__mmap_len(struct perf_mmap *map)/* When check_messup is true, 'end' must points to a good entry */ static union perf_event *perf_mmap__read (struct perf_mmap *map, -u64 start,u64 end,u64 *prev) +u64 *start,u64 end) {unsigned char *data = map->base + page_size;union perf_event *event =NULL ; -int diff = end - start; +int diff = end - *start;if (diff >= (int )sizeof (event->header)) {size_t size; - event = (union perf_event *)&data[start & map->mask]; + event = (union perf_event *)&data[*start & map->mask]; size = event->header.size; -if (size <sizeof (event->header) || diff < (int )size) { - event =NULL ; - goto broken_event; - } +if (size <sizeof (event->header) || diff < (int )size) +return NULL ;/* * Event straddles the mmap boundary -- header should always * be inside due to u64 alignment of output. */ -if ((start & map->mask) + size != ((start + size) & map->mask)) { -unsigned int offset = start; +if ((*start & map->mask) + size != ((*start + size) & map->mask)) { +unsigned int offset = *start;unsigned int len =min (sizeof (*event), size), cpy;void *dst = map->event_copy;
@@ -59,20 +57,15 @@ static union perf_event *perf_mmap__read(struct perf_mmap *map, event = (union perf_event *)map->event_copy; } - start += size; + *start += size; } -broken_event: -if (prev) - *prev = start; -return event; }union perf_event *perf_mmap__read_forward (struct perf_mmap *map) {u64 head; -u64 old = map->prev;/* * Check if event was unmapped due to a POLLHUP/POLLERR.
@@ -82,13 +75,12 @@ union perf_event *perf_mmap__read_forward(struct perf_mmap *map) head =perf_mmap__read_head (map); -return perf_mmap__read (map, old, head, &map->prev); +return perf_mmap__read (map, &map->prev, head); }union perf_event *perf_mmap__read_backward (struct perf_mmap *map) {u64 head, end; -u64 start = map->prev;/* * Check if event was unmapped due to a POLLHUP/POLLERR.
@@ -118,7 +110,7 @@ union perf_event *perf_mmap__read_backward(struct perf_mmap *map)else end = head + map->mask +1 ; -return perf_mmap__read (map, start, end, &map->prev); +return perf_mmap__read (map, &map->prev, end); }void perf_mmap__read_catchup (struct perf_mmap *map)
Signature · 1 line
2.5.5
Re: [PATCH V3 03/12] perf mmap: discard 'prev' in perf_mmap__read
Quoted 17 lines · On Thu, Dec 21, 2017 at 10:08:45AM -0800, [email protected] wrote:
From: Kan Liang <[email protected]>
'start' and 'prev' are duplicate in
perf_mmap__read()Use '
map->prev' to replace 'start' in perf_mmap__read_*().Suggested-by Wang Nan <[email protected]>Signed-off-by Kan Liang <[email protected]>--- tools/perf/util/mmap.c | 28 ++++++++++------------------ 1 file changed, 10 insertions(+), 18 deletions(-)tools/perf/util/mmap.c +0
-0
tools/perf/util/mmap.c size_t perf_mmap__mmap_len(struct perf_mmap *map) · line 29
Quoted 10 lines
tools/perf/util/mmap.c +2
-2@@ -22,29 +22,27 @@ size_t perf_mmap__mmap_len(struct perf_mmap *map)/* When check_messup is true, 'end' must points to a good entry */ static union perf_event *perf_mmap__read (struct perf_mmap *map, -u64 start,u64 end,u64 *prev) +u64 *start,u64 end) {unsigned char *data = map->base + page_size;union perf_event *event =NULL ; -int diff = end - start; +int diff = end - *start;
I'd like more if the arg was pointer, like 'u64 *startp' and you kept using the 'u64 start' value in the body.. but can't see any technical benefit behind it to force it ;-)
jirka
Quoted 40 lines
tools/perf/util/mmap.c +6
-12if (diff >= (int )sizeof (event->header)) {size_t size; - event = (union perf_event *)&data[start & map->mask]; + event = (union perf_event *)&data[*start & map->mask]; size = event->header.size; -if (size <sizeof (event->header) || diff < (int )size) { - event =NULL ; - goto broken_event; - } +if (size <sizeof (event->header) || diff < (int )size) +return NULL ;/* * Event straddles the mmap boundary -- header should always * be inside due to u64 alignment of output. */ -if ((start & map->mask) + size != ((start + size) & map->mask)) { -unsigned int offset = start; +if ((*start & map->mask) + size != ((*start + size) & map->mask)) { +unsigned int offset = *start;unsigned int len =min (sizeof (*event), size), cpy;void *dst = map->event_copy;@@ -59,20 +57,15 @@ static union perf_event *perf_mmap__read(struct perf_mmap *map, event = (union perf_event *)map->event_copy; } - start += size; + *start += size; } -broken_event: -if (prev) - *prev = start; -return event; }
SNIP
RE: [PATCH V3 03/12] perf mmap: discard 'prev' in perf_mmap__read
Quoted 20 lines · On Thu, Dec 21, 2017 at 10:08:45AM -0800, [email protected] wrote:
tools/perf/util/mmap.c size_t perf_mmap__mmap_len(struct perf_mmap
Quoted 17 lines
From: Kan Liang <[email protected]>
'start' and 'prev' are duplicate in
perf_mmap__read()Use '
map->prev' to replace 'start' in perf_mmap__read_*().Suggested-by Wang Nan <[email protected]>Signed-off-by Kan Liang <[email protected]>--- tools/perf/util/mmap.c | 28 ++++++++++------------------ 1 file changed, 10 insertions(+), 18 deletions(-)tools/perf/util/mmap.c +0
-0@@ -22,29 +22,27 @@ size_t perf_mmap__mmap_len(struct perf_mmap*map)
Quoted 1 line
Quoted 14 lines
tools/perf/util/mmap.c size_t perf_mmap__mmap_len(struct perf_mmap *map) · line 29
Quoted 9 lines
/* When
check_messupis true, 'end' must points to a good entry */ static unionperf_event*perf_mmap__read(structperf_mmap*map,tools/perf/util/mmap.c +2
-2- u64 start, u64 end, u64 *prev) + u64 *start, u64 end) {unsigned char *data = map->base + page_size;union perf_event *event =NULL ; -int diff = end - start; +int diff = end - *start;I'd like more if the arg was pointer, like 'u64 *startp' and you kept using the 'u64 start' value in the body.. but can't see any technical benefit behind it to force it ;-)
I will rename it as startp in V4.
Thanks,
Kan
Quoted 48 lines
jirka
Quoted 7 lines
tools/perf/util/mmap.c +1
-1if (diff >= (int )sizeof (event->header)) {size_t size; - event = (union perf_event *)&data[start & map->mask]; + event = (union perf_event *)&data[*start & map->mask]; size = event->header.size;tools/perf/util/mmap.c size_t perf_mmap__mmap_len(struct perf_mmap *map)
Quoted 10 lines
tools/perf/util/mmap.c +2
-4-if (size <sizeof (event->header) || diff < (int )size) { - event =NULL ; - goto broken_event; - } +if (size <sizeof (event->header) || diff < (int )size) +return NULL ;/* * Event straddles the mmap boundary -- header should always
tools/perf/util/mmap.c size_t perf_mmap__mmap_len(struct perf_mmap *map)
Quoted 3 lines
tools/perf/util/mmap.c +0
-1* be inside due to u64 alignment of output. */ -if ((start & map->mask) + size != ((start + size) & map->mask)){
tools/perf/util/mmap.c static union perf_event *perf_mmap__read(struct
Quoted 8 lines
tools/perf/util/mmap.c +1
-1-unsigned int offset = start; + if ((*start & map->mask) + size != ((*start + size) & map-mask)) {
tools/perf/util/mmap.c +1
-0+unsigned int offset = *start;unsigned int len =min (sizeof (*event), size), cpy;void *dst = map->event_copy;@@ -59,20 +57,15 @@ static union perf_event *perf_mmap__read(struct
perf_mmap*map,Quoted 13 lines
tools/perf/util/mmap.c +1
-5event = (union perf_event *)map->event_copy; } - start += size; + *start += size; } -broken_event: -if (prev) - *prev = start; -return event; }SNIP
PATCH v3 4/12 perf mmap: introduce perf_mmap__read_done
In reply to [email protected]From: Kan Liang <[email protected]>
The direction of overwrite mode is backward. The last mmap__read_event will set tail to map->prev. Need to correct the map->prev to head which is the end of next read.
It will be used later.
--- tools/perf/util/mmap.c | 11 +++++++++++ tools/perf/util/mmap.h | 1 + 2 files changed, 12 insertions(+)
tools/perf/util/mmap.c +11 -0
@@ -343,3 +343,14 @@ int perf_mmap__push(struct perf_mmap *md, bool overwrite, out:return rc; } + +/* +* Mandatory for overwrite mode +* The direction of overwrite mode is backward. +* The last mmap__read_event will set tail to map->prev. +* Need to correct the map->prev to head which is the end of next read. +*/ +void perf_mmap__read_done (struct perf_mmap *map) +{ + map->prev =perf_mmap__read_head (map); +}
tools/perf/util/mmap.h +1 -0
@@ -96,4 +96,5 @@ size_t perf_mmap__mmap_len(struct perf_mmap *map);int perf_mmap__read_init (struct perf_mmap *map,bool overwrite,u64 *start,u64 *end); +void perf_mmap__read_done (struct perf_mmap *map);#endif /*__PERF_MMAP_H */
Signature · 1 line
2.5.5
Namhyung Kim
maintainerRe: [PATCH V3 04/12] perf mmap: introduce perf_mmap__read_done
Hello,
Quoted 5 lines · On Thu, Dec 21, 2017 at 10:08:46AM -0800, [email protected] wrote:
From: Kan Liang <[email protected]>
The direction of overwrite mode is backward. The last
mmap__read_eventwill set tail tomap->prev. Need to correct themap->prevto head which is the end of next read.
Why do you update the map->prev needlessly then? I think we don't need it for overwrite/backward mode, right?
Also I guess the current code might miss some events since the head can be different between _read_init() and _read_done(), no?
Thanks,
Namhyung
Quoted 41 lines
It will be used later.
Signed-off-by Kan Liang <[email protected]>--- tools/perf/util/mmap.c | 11 +++++++++++ tools/perf/util/mmap.h | 1 + 2 files changed, 12 insertions(+)tools/perf/util/mmap.c +11
-0@@ -343,3 +343,14 @@ int perf_mmap__push(struct perf_mmap *md, bool overwrite, out:return rc; } + +/* +* Mandatory for overwrite mode +* The direction of overwrite mode is backward. +* The last mmap__read_event will set tail to map->prev. +* Need to correct the map->prev to head which is the end of next read. +*/ +void perf_mmap__read_done (struct perf_mmap *map) +{ + map->prev =perf_mmap__read_head (map); +}tools/perf/util/mmap.h +1
-0@@ -96,4 +96,5 @@ size_t perf_mmap__mmap_len(struct perf_mmap *map);int perf_mmap__read_init (struct perf_mmap *map,bool overwrite,u64 *start,u64 *end); +void perf_mmap__read_done (struct perf_mmap *map);#endif /*__PERF_MMAP_H */ Signature · 1 line
2.5.5
RE: [PATCH V3 04/12] perf mmap: introduce perf_mmap__read_done
Quoted 11 lines
Hello,
Quoted 5 lines · On Thu, Dec 21, 2017 at 10:08:46AM -0800, [email protected] wrote:
From: Kan Liang <[email protected]>
The direction of overwrite mode is backward. The last
mmap__read_eventwill set tail tomap->prev. Need to correct themap->prevto head which is the end of next read.Why do you update the
map->prevneedlessly then? I think we don't need it for overwrite/backward mode, right?
The map->prev is needless only when the overwrite does really happen in ringbuffer. In a light load system or with big ringbuffer, the unprocessed data will not be overwritten. So it's necessary to keep an pointer to indicate the last position.
Overwrite mode is backward, but the event processing is always forward. So map->prev has to be updated in __read_done().
Quoted 4 lines
Also I guess the current code might miss some events since the head can be different between
_read_init()and_read_done(), no?
The overwrite mode requires the ring buffer to be paused during processing. The head is unchanged between __read_init() and __read_done().
The event during the pause should be missed. But I think it has little impact for the accuracy of the snapshot and can be tolerant for perf top.
I mentioned it in the change log of patch 11/12.
I also removed the lost events checking for perf top.
Thanks,
Kan
Quoted 46 lines
Thanks,
Namhyungtools/perf/util/mmap.c int perf_mmap__push(struct perf_mmap *md,
Quoted 14 lines
It will be used later.
Signed-off-by Kan Liang <[email protected]>--- tools/perf/util/mmap.c | 11 +++++++++++ tools/perf/util/mmap.h | 1(quoted diff) +1
-0+ 2 files changed, 12 insertions(+)tools/perf/util/mmap.c index renamed from tools/perf/util/mmap.c +0
-0a844a2f..4aaeb64 100644
tools/perf/util/mmap.c +0
-0@@ -343,3 +343,14 @@ int perf_mmap__push(struct perf_mmap *md,bool
tools/perf/util/mmap.h size_t perf_mmap__mmap_len(struct perf_mmap
Quoted 18 lines
tools/perf/util/mmap.c +9
-0overwrite, out:return rc; } + +/* +* Mandatory for overwrite mode +* The direction of overwrite mode is backward. +* The last mmap__read_event will set tail to map->prev. +* Need to correct the map->prev to head which is the end of next read. +*/ +void perf_mmap__read_done (struct perf_mmap *map) { + map->prev =perf_mmap__read_head (map); }tools/perf/util/mmap.h index renamed from tools/perf/util/mmap.h +0
-0abe9b9f..2df27c1 100644
tools/perf/util/mmap.h +0
-0@@ -96,4 +96,5 @@ size_t perf_mmap__mmap_len(struct perf_mmap*map);
Quoted 8 lines
tools/perf/util/mmap.h +1
-1int perf_mmap__read_init (struct perf_mmap *map,bool overwrite,u64 *start,u64 *end); +void perf_mmap__read_done (struct perf_mmap *map);#endif /*__PERF_MMAP_H */ --2.5.5
Namhyung Kim
maintainerRe: [PATCH V3 04/12] perf mmap: introduce perf_mmap__read_done
Hi Kan,
Quoted 12 lines · Kan
Quoted 11 lines
Hello,
Quoted 5 lines · On Thu, Dec 21, 2017 at 10:08:46AM -0800, [email protected] wrote:
From: Kan Liang <[email protected]>
The direction of overwrite mode is backward. The last
mmap__read_eventwill set tail tomap->prev. Need to correct themap->prevto head which is the end of next read.Why do you update the
map->prevneedlessly then? I think we don't need it for overwrite/backward mode, right?
Quoted 6 lines
The
map->previs needless only when the overwrite does really happen in ringbuffer. In a light load system or with big ringbuffer, the unprocessed data will not be overwritten. So it's necessary to keep an pointer to indicate the last position.Overwrite mode is backward, but the event processing is always forward. So
map->prevhas to be updated in__read_done().
Yep, I meant that updating map->prev in every perf_mmap__read_event() is unnecessary for the overwrite mode. It only needs to be set in perf_mmap__read_done(), right?
Quoted 8 lines
Quoted 5 lines
Also I guess the current code might miss some events since the head can be different between
_read_init()and_read_done(), no?The overwrite mode requires the ring buffer to be paused during processing. The head is unchanged between
__read_init()and__read_done().
Ah, ok then. Maybe we could read the head once, and use it during processing.
Thanks,
Namhyung
Quoted 5 lines
The event during the pause should be missed. But I think it has little impact for the accuracy of the snapshot and can be tolerant for perf top.
I mentioned it in the change log of patch 11/12.
I also removed the lost events checking for perf top.
RE: [PATCH V3 04/12] perf mmap: introduce perf_mmap__read_done
Quoted 22 lines
Hi Kan,
Quoted 6 lines · Kan
Quoted 6 lines
Hello,
Quoted 3 lines · On Thu, Dec 21, 2017 at 10:08:46AM -0800, [email protected] wrote:
From: Kan Liang <[email protected]>
The direction of overwrite mode is backward. The last
mmap__read_eventQuoted 4 lines
Quoted 4 lines
Quoted 2 lines
will set tail to
map->prev. Need to correct themap->prevto head which is the end of next read.Why do you update the
map->prevneedlessly then? I think we don'tneed it
Quoted 3 lines
Quoted 1 line
for overwrite/backward mode, right?
The
map->previs needless only when the overwrite does really happen inringbuffer.
Quoted 1 line
In a light load system or with big ringbuffer, the unprocessed data will not
be
Quoted 1 line
overwritten. So it's necessary to keep an pointer to indicate the last
Quoted 8 lines
position.
Quoted 3 lines
Overwrite mode is backward, but the event processing is always forward. So
map->prevhas to be updated in__read_done().Yep, I meant that updating
map->previn everyperf_mmap__read_event()is unnecessary for the overwrite mode. It only needs to be set inperf_mmap__read_done(), right?
Right, for overwrite, only updating the map->prev in perf_mmap__read_done() is enough.
But for non-overwrite, we have to update map->prev.
It will be used by perf_mmap__consume() later to write the ring buffer tail. So I specially handle the non-overwrite as below in patch 5/12.
+ event = perf_mmap__read(map, start, end);
+
+ if (!overwrite)
+ map->prev = *start;
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Also I guess the current code might miss some events since the head can
be
Quoted 3 lines
Quoted 2 lines
different between
_read_init()and_read_done(), no?
Quoted 6 lines
Quoted 1 line
The overwrite mode requires the ring buffer to be paused during
processing.
Quoted 1 line
The head is unchanged between
__read_init()and__read_done().Ah, ok then. Maybe we could read the head once, and use it during processing.
Yes, it only needs to read head once for overwrite mode. But for non-overwrite, we have to read the head in every perf_mmap__read_event(). Because the head is floating. The non-overwrite is specially handled in patch 5/12 as well.
+ /* non-overwirte doesn't pause the ringbuffer */
+ if (!overwrite)
+ end = perf_mmap__read_head(map);
Thanks,
Kan
Quoted 11 lines
Thanks,
NamhyungQuoted 2 lines
The event during the pause should be missed. But I think it has little impact
for
Quoted 3 lines
the accuracy of the snapshot and can be tolerant for perf top. I mentioned it in the change log of patch 11/12.
I also removed the lost events checking for perf top.
Namhyung Kim
maintainerRe: [PATCH V3 04/12] perf mmap: introduce perf_mmap__read_done
Hi,
Quoted 31 lines · Kan
Quoted 22 lines
Hi Kan,
Quoted 6 lines · Kan
Quoted 6 lines
Hello,
Quoted 3 lines · On Thu, Dec 21, 2017 at 10:08:46AM -0800, [email protected] wrote:
From: Kan Liang <[email protected]>
The direction of overwrite mode is backward. The last
mmap__read_eventQuoted 4 lines
Quoted 4 lines
Quoted 2 lines
will set tail to
map->prev. Need to correct themap->prevto head which is the end of next read.Why do you update the
map->prevneedlessly then? I think we don'tneed it
Quoted 3 lines
Quoted 1 line
for overwrite/backward mode, right?
The
map->previs needless only when the overwrite does really happen inringbuffer.
Quoted 1 line
In a light load system or with big ringbuffer, the unprocessed data will not
be
Quoted 1 line
overwritten. So it's necessary to keep an pointer to indicate the last
Quoted 8 lines
position.
Quoted 3 lines
Overwrite mode is backward, but the event processing is always forward. So
map->prevhas to be updated in__read_done().Yep, I meant that updating
map->previn everyperf_mmap__read_event()is unnecessary for the overwrite mode. It only needs to be set inperf_mmap__read_done(), right?
Quoted 6 lines
Right, for overwrite, only updating the
map->previnperf_mmap__read_done()is enough.But for non-overwrite, we have to update
map->prev.
It will be used byperf_mmap__consume()later to write the ring buffer tail. So I specially handle the non-overwrite as below in patch 5/12.
Oh I can see it in the patch 5, thanks.
Quoted 16 lines
(quoted diff) +4
-0+ event = perf_mmap__read(map, start, end); + + if (!overwrite) + map->prev = *start;Quoted 12 lines
Quoted 4 lines
Quoted 4 lines
Also I guess the current code might miss some events since the head can
be
Quoted 4 lines
Quoted 2 lines
different between
_read_init()and_read_done(), no?The overwrite mode requires the ring buffer to be paused during
processing.
Quoted 1 line
The head is unchanged between
__read_init()and__read_done().
Quoted 7 lines
Quoted 2 lines
Ah, ok then. Maybe we could read the head once, and use it during processing.
Yes, it only needs to read head once for overwrite mode. But for non-overwrite, we have to read the head in every
perf_mmap__read_event(). Because the head is floating. The non-overwrite is specially handled in patch 5/12 as well.
Right, I understand it for the non-overwrite mode.
But, for the overwrite mode, my concern was that it might be possible that it reads a stale head in __read_init() (even after it paused the ring buffer) and reads an update head in __read_done(). Then it's gonna miss some records. I'm not sure whether it reads the same head in __read_init() and __read_done() by the pause.
Thanks,
Namhyung
Quoted 3 lines
(quoted diff) +3
-0+ /* non-overwirte doesn't pause the ringbuffer */ + if (!overwrite) + end = perf_mmap__read_head(map);
RE: [PATCH V3 04/12] perf mmap: introduce perf_mmap__read_done
Quoted 19 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Also I guess the current code might miss some events since the head
can
Quoted 8 lines
Quoted 8 lines
be
Quoted 4 lines
Quoted 2 lines
different between
_read_init()and_read_done(), no?The overwrite mode requires the ring buffer to be paused during
processing.
Quoted 1 line
The head is unchanged between
__read_init()and__read_done().Quoted 7 lines
Quoted 2 lines
Ah, ok then. Maybe we could read the head once, and use it during processing.
Yes, it only needs to read head once for overwrite mode. But for non-overwrite, we have to read the head in every
perf_mmap__read_event(). Because the head is floating. The non-overwrite is specially handled in patch 5/12 as well.
Quoted 8 lines
Right, I understand it for the non-overwrite mode.
But, for the overwrite mode, my concern was that it might be possible that it reads a stale head in
__read_init()(even after it paused the ring buffer) and reads an update head in__read_done(). Then it's gonna miss some records. I'm not sure whether it reads the same head in__read_init()and__read_done()by the pause.
The only scenario which may cause the different 'head' may be as below. The 'rb->head' is updated in __perf_output_begin(), but haven’t been assigned to 'pc->data_head' for perf tool. During this period, the 'paused' is set and __read_init() reads head.
But this scenario never happens because of the ringbuffer lock.
Otherwise, I cannot imagine any other scenarios which may causes the different 'head' in __read_init() and __read_done() with ringbuffer paused. Please let me know if there is an example.
There would be some records miss. But it's only because the ringbuffer is paused. The head should keep the same.
I also did some tests and dump the 'head' in __read_init() and __read_done() with ringbuffer paused. I didn't see any difference either.
Thanks,
Kan
Quoted 7 lines
Thanks,
NamhyungQuoted 3 lines
(quoted diff) +3
-0+ /* non-overwirte doesn't pause the ringbuffer */ + if (!overwrite) + end = perf_mmap__read_head(map);
Namhyung Kim
maintainerRe: [PATCH V3 04/12] perf mmap: introduce perf_mmap__read_done
Hi,
Quoted 28 lines · Kan
Quoted 19 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Also I guess the current code might miss some events since the head
can
Quoted 8 lines
Quoted 8 lines
be
Quoted 4 lines
Quoted 2 lines
different between
_read_init()and_read_done(), no?The overwrite mode requires the ring buffer to be paused during
processing.
Quoted 1 line
The head is unchanged between
__read_init()and__read_done().Quoted 7 lines
Quoted 2 lines
Ah, ok then. Maybe we could read the head once, and use it during processing.
Yes, it only needs to read head once for overwrite mode. But for non-overwrite, we have to read the head in every
perf_mmap__read_event(). Because the head is floating. The non-overwrite is specially handled in patch 5/12 as well.Quoted 8 lines
Right, I understand it for the non-overwrite mode.
But, for the overwrite mode, my concern was that it might be possible that it reads a stale head in
__read_init()(even after it paused the ring buffer) and reads an update head in__read_done(). Then it's gonna miss some records. I'm not sure whether it reads the same head in__read_init()and__read_done()by the pause.
Quoted 5 lines
The only scenario which may cause the different 'head' may be as below. The '
rb->head' is updated in__perf_output_begin(), but haven’t been assigned to 'pc->data_head' for perf tool. During this period, the 'paused' is set and__read_init()reads head.
But this scenario never happens because of the ringbuffer lock.
Which lock did you say?
Quoted 4 lines
Otherwise, I cannot imagine any other scenarios which may causes the different 'head' in
__read_init()and__read_done()with ringbuffer paused. Please let me know if there is an example.
Maybe I'm missing something. But I don't know what makes it guarantee to see the updated data_head written by another cpu before the pause.
Quoted 3 lines
There would be some records miss. But it's only because the ringbuffer is paused. The head should keep the same.
Hmm.. yes. It's gonna miss some records anyway, then I don't care about it anymore.
Quoted 3 lines
I also did some tests and dump the 'head' in
__read_init()and__read_done()with ringbuffer paused. I didn't see any difference either.
Maybe on a weak-ordered machine? Not sure though.
But as I said, I don't care anymore since we cannot avoid some missing records.
Thanks,
Namhyung
RE: [PATCH V3 04/12] perf mmap: introduce perf_mmap__read_done
Quoted 32 lines
Hi,
Quoted 2 lines · Kan
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Also I guess the current code might miss some events since the
head
Quoted 26 lines
Quoted 25 lines
can
Quoted 8 lines
Quoted 8 lines
be
Quoted 4 lines
Quoted 2 lines
different between
_read_init()and_read_done(), no?The overwrite mode requires the ring buffer to be paused during
processing.
Quoted 1 line
The head is unchanged between
__read_init()and__read_done().Quoted 7 lines
Quoted 2 lines
Ah, ok then. Maybe we could read the head once, and use it during processing.
Yes, it only needs to read head once for overwrite mode. But for non-overwrite, we have to read the head in every
perf_mmap__read_event(). Because the head is floating. The non-overwrite is specially handled in patch 5/12 as well.Right, I understand it for the non-overwrite mode.
But, for the overwrite mode, my concern was that it might be possible that it reads a stale head in
__read_init()(even after it paused the ring buffer) and reads an update head in__read_done(). Then it's gonna miss some records. I'm not sure whether it reads the same head in__read_init()and__read_done()by the pause.
Quoted 8 lines
Quoted 5 lines
The only scenario which may cause the different 'head' may be as below. The '
rb->head' is updated in__perf_output_begin(), but haven’t been assigned to 'pc->data_head' for perf tool. During this period, the 'paused' is set and__read_init()reads head.
But this scenario never happens because of the ringbuffer lock.Which lock did you say?
The RCU lock.
Quoted 9 lines
Quoted 5 lines
Otherwise, I cannot imagine any other scenarios which may causes the different 'head' in
__read_init()and__read_done()with ringbuffer paused. Please let me know if there is an example.Maybe I'm missing something. But I don't know what makes it guarantee to see the updated
data_headwritten by another cpu before the pause.
I think it should be the kernel's responsibility. It's described in the changelog of Commit 86e7972f690c "perf/ring_buffer: Introduce new ioctl options to pause and resume the ring-buffer"
Quoted 8 lines
Quoted 4 lines
There would be some records miss. But it's only because the ringbuffer is paused. The head should keep the same.
Hmm.. yes. It's gonna miss some records anyway, then I don't care about it anymore.
OK.
Thanks for the review.
Thanks,
Kan
Re: [PATCH V3 04/12] perf mmap: introduce perf_mmap__read_done
In reply to [email protected]Quoted 19 lines · On Thu, Dec 21, 2017 at 10:08:46AM -0800, [email protected] wrote:
From: Kan Liang <[email protected]>
The direction of overwrite mode is backward. The last
mmap__read_eventwill set tail tomap->prev. Need to correct themap->prevto head which is the end of next read.It will be used later.
Signed-off-by Kan Liang <[email protected]>--- tools/perf/util/mmap.c | 11 +++++++++++ tools/perf/util/mmap.h | 1 + 2 files changed, 12 insertions(+)tools/perf/util/mmap.c +0
-0
tools/perf/util/mmap.c int perf_mmap__push(struct perf_mmap *md, bool overwrite, · line 350
Quoted 8 lines
tools/perf/util/mmap.c +5
-0@@ -343,3 +343,14 @@ int perf_mmap__push(struct perf_mmap *md, bool overwrite, out:return rc; } + +/* +* Mandatory for overwrite mode +* The direction of overwrite mode is backward. +* The last mmap__read_event will set tail to map->prev.
you mean perf_mmap__read function in this comment, right?
jirka
Quoted 19 lines
tools/perf/util/mmap.c +6
-0+* Need to correct the map->prev to head which is the end of next read. +*/ +void perf_mmap__read_done (struct perf_mmap *map) +{ + map->prev =perf_mmap__read_head (map); +}tools/perf/util/mmap.h +1
-0@@ -96,4 +96,5 @@ size_t perf_mmap__mmap_len(struct perf_mmap *map);int perf_mmap__read_init (struct perf_mmap *map,bool overwrite,u64 *start,u64 *end); +void perf_mmap__read_done (struct perf_mmap *map);#endif /*__PERF_MMAP_H */ Signature · 1 line
2.5.5
RE: [PATCH V3 04/12] perf mmap: introduce perf_mmap__read_done
Quoted 21 lines · On Thu, Dec 21, 2017 at 10:08:46AM -0800, [email protected] wrote:
tools/perf/util/mmap.c int perf_mmap__push(struct perf_mmap *md,
Quoted 19 lines
From: Kan Liang <[email protected]>
The direction of overwrite mode is backward. The last
mmap__read_eventwill set tail tomap->prev. Need to correct themap->prevto head which is the end of next read.It will be used later.
Signed-off-by Kan Liang <[email protected]>--- tools/perf/util/mmap.c | 11 +++++++++++ tools/perf/util/mmap.h | 1 + 2 files changed, 12 insertions(+)tools/perf/util/mmap.c +0
-0@@ -343,3 +343,14 @@ int perf_mmap__push(struct perf_mmap *md,bool overwrite,
Quoted 11 lines
tools/perf/util/mmap.c int perf_mmap__push(struct perf_mmap *md, bool overwrite, · line 350
Quoted 8 lines
tools/perf/util/mmap.c +5
-0out:return rc; } + +/* +* Mandatory for overwrite mode +* The direction of overwrite mode is backward. +* The last mmap__read_event will set tail to map->prev. you mean
perf_mmap__readfunction in this comment, right?
Right.
Will fix it in V4.
Thanks,
Kan
Quoted 22 lines
jirka
tools/perf/util/mmap.h size_t perf_mmap__mmap_len(struct perf_mmap
Quoted 11 lines
tools/perf/util/mmap.c +6
-0+* Need to correct the map->prev to head which is the end of next read. +*/ +void perf_mmap__read_done (struct perf_mmap *map) +{ + map->prev =perf_mmap__read_head (map); +}tools/perf/util/mmap.h +0
-0@@ -96,4 +96,5 @@ size_t perf_mmap__mmap_len(struct perf_mmap*map);
Quoted 8 lines
tools/perf/util/mmap.h +1
-1int perf_mmap__read_init (struct perf_mmap *map,bool overwrite,u64 *start,u64 *end); +void perf_mmap__read_done (struct perf_mmap *map);#endif /*__PERF_MMAP_H */ --2.5.5
PATCH v3 5/12 perf mmap: introduce perf_mmap__read_event
In reply to [email protected]From: Kan Liang <[email protected]>
Except perf record, other perf tools read events one by one from ring buffer by perf_mmap__read_forward. But it only supports non-overwrite mode.
Introduce perf_mmap__read_event to support both non-overwrite and overwrite mode.
Usage:perf_mmap__read_init()
while(event = perf_mmap__read_event()) {
//process the event
perf_mmap__consume()
}perf_mmap__read_done()
It cannot use perf_mmap__read_backward. Because it always read the stale buffer which is already processed. Furthermore, the forward and backward concepts have been removed. The perf_mmap__read_backward will be replaced and discarded later.
--- tools/perf/util/mmap.c | 39 +++++++++++++++++++++++++++++++++++++++ tools/perf/util/mmap.h | 4 ++++ 2 files changed, 43 insertions(+)
tools/perf/util/mmap.c +39 -0
@@ -113,6 +113,45 @@ union perf_event *perf_mmap__read_backward(struct perf_mmap *map)return perf_mmap__read (map, &map->prev, end); } +/* +* Read event from ring buffer one by one. +* Return one event for each call. +* +* Usage: +* perf_mmap__read_init() +* while(event = perf_mmap__read_event()) { +* //process the event +* perf_mmap__consume() +* } +* perf_mmap__read_done() +*/ +union perf_event *perf_mmap__read_event (struct perf_mmap *map, +bool overwrite, +u64 *start,u64 end) +{ +union perf_event *event; + +/* +* Check if event was unmapped due to a POLLHUP/POLLERR. +*/ +if (!refcount_read (&map->refcnt)) +return NULL ; + +if (start ==NULL ) +return NULL ; + +/* non-overwirte doesn't pause the ringbuffer */ +if (!overwrite) + end =perf_mmap__read_head (map); + + event =perf_mmap__read (map, start, end); + +if (!overwrite) + map->prev = *start; + +return event; +} +void perf_mmap__read_catchup (struct perf_mmap *map) {u64 head;
tools/perf/util/mmap.h +4 -0
@@ -89,6 +89,10 @@ static inline void perf_mmap__write_tail(struct perf_mmap *md, u64 tail)union perf_event *perf_mmap__read_forward (struct perf_mmap *map);union perf_event *perf_mmap__read_backward (struct perf_mmap *map); +union perf_event *perf_mmap__read_event (struct perf_mmap *map, +bool overwrite, +u64 *start,u64 end); +int perf_mmap__push (struct perf_mmap *md,bool backward,void *to,int push (void *to,void *buf,size_t size));
Signature · 1 line
2.5.5
PATCH v3 6/12 perf test: update mmap read functions for backward-ring-buffer test
In reply to [email protected]From: Kan Liang <[email protected]>
Use the new perf_mmap__read_* interfaces for overwrite ringbuffer test.
--- tools/perf/tests/backward-ring-buffer.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
tools/perf/tests/backward-ring-buffer.c +5 -2
@@ -31,10 +31,12 @@ static int count_samples(struct perf_evlist *evlist, int *sample_count,int i;for (i =0 ; i < evlist->nr_mmaps; i++) { +struct perf_mmap *map = &evlist->overwrite_mmap[i];union perf_event *event; +u64 start, end; -perf_mmap__read_catchup (&evlist->overwrite_mmap[i]); -while ((event =perf_mmap__read_backward (&evlist->overwrite_mmap[i])) !=NULL ) { +perf_mmap__read_init (map, true, &start, &end); +while ((event =perf_mmap__read_event (map, true, &start, end)) !=NULL ) {const u32 type = event->header.type;switch (type) {
@@ -49,6 +51,7 @@ static int count_samples(struct perf_evlist *evlist, int *sample_count,return TEST_FAIL ; } } +perf_mmap__read_done (map); }return TEST_OK ; }
Signature · 1 line
2.5.5
PATCH v3 7/12 perf mmap: discard legacy interface for mmap read
In reply to [email protected]From: Kan Liang <[email protected]>
Discards perf_mmap__read_backward and perf_mmap__read_catchup. No tools use them.
There are tools still use perf_mmap__read_forward. Keep it, but add comments to point to the new interface for future use.
--- tools/perf/util/mmap.c | 50 ++++---------------------------------------------- tools/perf/util/mmap.h | 3 --- 2 files changed, 4 insertions(+), 49 deletions(-)
tools/perf/util/mmap.c +4 -46
@@ -63,6 +63,10 @@ static union perf_event *perf_mmap__read(struct perf_mmap *map,return event; } +/* +* legacy interface for mmap read. +* Don't use it. Use perf_mmap__read_event(). +*/ union perf_event *perf_mmap__read_forward (struct perf_mmap *map) {u64 head;
@@ -78,41 +82,6 @@ union perf_event *perf_mmap__read_forward(struct perf_mmap *map)return perf_mmap__read (map, &map->prev, head); } -union perf_event *perf_mmap__read_backward (struct perf_mmap *map) -{ -u64 head, end; - -/* -* Check if event was unmapped due to a POLLHUP/POLLERR. -*/ -if (!refcount_read (&map->refcnt)) -return NULL ; - - head =perf_mmap__read_head (map); -if (!head) -return NULL ; - -/* -* 'head' pointer starts from 0. Kernel minus sizeof(record) form -* it each time when kernel writes to it, so in fact 'head' is -* negative. 'end' pointer is made manually by adding the size of -* the ring buffer to 'head' pointer, means the validate data can -* read is the whole ring buffer. If 'end' is positive, the ring -* buffer has not fully filled, so we must adjust 'end' to 0. -* -* However, since both 'head' and 'end' is unsigned, we can't -* simply compare 'end' against 0. Here we compare '-head' and -* the size of the ring buffer, where -head is the number of bytes -* kernel write to the ring buffer. -*/ -if (-head < (u64)(map->mask +1 )) - end =0 ; -else - end = head + map->mask +1 ; - -return perf_mmap__read (map, &map->prev, end); -} -/* * Read event from ring buffer one by one. * Return one event for each call.
@@ -152,17 +121,6 @@ union perf_event *perf_mmap__read_event(struct perf_mmap *map,return event; } -void perf_mmap__read_catchup (struct perf_mmap *map) -{ -u64 head; - -if (!refcount_read (&map->refcnt)) -return ; - - head =perf_mmap__read_head (map); - map->prev = head; -} -static bool perf_mmap__empty (struct perf_mmap *map) {return perf_mmap__read_head (map) == map->prev && !map->auxtrace_mmap.base;
tools/perf/util/mmap.h +0 -3
@@ -65,8 +65,6 @@ void perf_mmap__put(struct perf_mmap *map);void perf_mmap__consume (struct perf_mmap *map,bool overwrite); -void perf_mmap__read_catchup (struct perf_mmap *md); -static inline u64 perf_mmap__read_head (struct perf_mmap *mm) {struct perf_event_mmap_page *pc = mm->base;
@@ -87,7 +85,6 @@ static inline void perf_mmap__write_tail(struct perf_mmap *md, u64 tail) }union perf_event *perf_mmap__read_forward (struct perf_mmap *map); -union perf_event *perf_mmap__read_backward (struct perf_mmap *map); union perf_event *perf_mmap__read_event(struct perf_mmap *map, bool overwrite,
Signature · 1 line
2.5.5
Re: [PATCH V3 07/12] perf mmap: discard legacy interface for mmap read
Quoted 19 lines · On Thu, Dec 21, 2017 at 10:08:49AM -0800, [email protected] wrote:
From: Kan Liang <[email protected]>
Discards
perf_mmap__read_backwardandperf_mmap__read_catchup. No tools use them.There are tools still use
perf_mmap__read_forward. Keep it, but add comments to point to the new interface for future use.Signed-off-by Kan Liang <[email protected]>--- tools/perf/util/mmap.c | 50 ++++---------------------------------------------- tools/perf/util/mmap.h | 3 --- 2 files changed, 4 insertions(+), 49 deletions(-)tools/perf/util/mmap.c +0
-0
tools/perf/util/mmap.c static union perf_event *perf_mmap__read(struct perf_mmap *map, · line 69
Quoted 7 lines
tools/perf/util/mmap.c +4
-0@@ -63,6 +63,10 @@ static union perf_event *perf_mmap__read(struct perf_mmap *map,return event; } +/* +* legacy interface for mmap read. +* Don't use it. Use perf_mmap__read_event(). +*/
could we get rid of it then? looks like it's not much work, seems it's used only in:
perf_evlist__mmap_read
perf_evlist__mmap_read_forward
it'd prove the new interface work correctly for both cases
thanks,
jirka
Quoted 6 lines
tools/perf/util/mmap.c +0
-0union perf_event *perf_mmap__read_forward (struct perf_mmap *map) {u64 head;@@ -78,41 +82,6 @@ union perf_event *perf_mmap__read_forward(struct perf_mmap *map)return perf_mmap__read (map, &map->prev, head); }
SNIP
RE: [PATCH V3 07/12] perf mmap: discard legacy interface for mmap read
Quoted 25 lines · On Thu, Dec 21, 2017 at 10:08:49AM -0800, [email protected] wrote:
Quoted 3 lines
From: Kan Liang <[email protected]>
Discards
perf_mmap__read_backwardandperf_mmap__read_catchup.No tools
tools/perf/util/mmap.c static union perf_event *perf_mmap__read(struct
Quoted 16 lines
use them.
There are tools still use
perf_mmap__read_forward. Keep it, but add comments to point to the new interface for future use.Signed-off-by Kan Liang <[email protected]>--- tools/perf/util/mmap.c | 50 ++++---------------------------------------------- tools/perf/util/mmap.h | 3 --- 2 files changed, 4 insertions(+), 49 deletions(-)tools/perf/util/mmap.c +0
-0@@ -63,6 +63,10 @@ static union perf_event *perf_mmap__read(struct
perf_mmap*map,Quoted 3 lines
tools/perf/util/mmap.c +0
-0return event; }
Quoted 8 lines
tools/perf/util/mmap.c static union perf_event *perf_mmap__read(struct perf_mmap *map, · line 69
Quoted 4 lines
tools/perf/util/mmap.c +4
-0+/* +* legacy interface for mmap read. +* Don't use it. Use perf_mmap__read_event(). +*/ could we get rid of it then? looks like it's not much work, seems it's used only in:
To get rid of it, it has to introduce the whole perf_mmap__read_init/_read_event/_done.
Besides perf top, kvm and trace need to be changed. There are also 11 perf test cases need to be changed as well.
I think that would make current patch series too huge. I can submit a separated patch series later to get rid of the legacy interface. Is it OK?
Thanks,
Kan
Quoted 17 lines
perf_evlist__mmap_read
perf_evlist__mmap_read_forwardit'd prove the new interface work correctly for both cases
thanks,
jirkatools/perf/util/mmap.c union perf_event
Quoted 4 lines
tools/perf/util/mmap.c +0
-0union perf_event *perf_mmap__read_forward (struct perf_mmap *map) {u64 head;@@ -78,41 +82,6 @@ union perf_event*perf_mmap__read_forward(struct
perf_mmap*map)Quoted 2 lines
tools/perf/util/mmap.c +0
-0return perf_mmap__read (map, &map->prev, head); }SNIP
Re: [PATCH V3 07/12] perf mmap: discard legacy interface for mmap read
Quoted 37 lines · Kan
Quoted 25 lines · On Thu, Dec 21, 2017 at 10:08:49AM -0800, [email protected] wrote:
Quoted 3 lines
From: Kan Liang <[email protected]>
Discards
perf_mmap__read_backwardandperf_mmap__read_catchup.No tools
tools/perf/util/mmap.c static union perf_event *perf_mmap__read(struct
Quoted 16 lines
use them.
There are tools still use
perf_mmap__read_forward. Keep it, but add comments to point to the new interface for future use.Signed-off-by Kan Liang <[email protected]>--- tools/perf/util/mmap.c | 50 ++++---------------------------------------------- tools/perf/util/mmap.h | 3 --- 2 files changed, 4 insertions(+), 49 deletions(-)tools/perf/util/mmap.c +0
-0@@ -63,6 +63,10 @@ static union perf_event *perf_mmap__read(struct
perf_mmap*map,Quoted 3 lines
tools/perf/util/mmap.c +0
-0return event; }Quoted 8 lines
tools/perf/util/mmap.c static union perf_event *perf_mmap__read(struct perf_mmap *map, · line 69
Quoted 4 lines
tools/perf/util/mmap.c +4
-0+/* +* legacy interface for mmap read. +* Don't use it. Use perf_mmap__read_event(). +*/ could we get rid of it then? looks like it's not much work, seems it's used only in:
To get rid of it, it has to introduce the whole perf_mmap__read_init/_read_event/_done.
Quoted 6 lines
Besides perf top, kvm and trace need to be changed. There are also 11 perf test cases need to be changed as well.
I think that would make current patch series too huge. I can submit a separated patch series later to get rid of the legacy interface. Is it OK?
ok
jirka
PATCH v3 8/12 perf top: check per event overwrite term
In reply to [email protected]From: Kan Liang <[email protected]>
Per event overwrite term is not forbidden in perf top, which can bring problems. Because perf top only support non-overwrite mode.
Check and forbid inconsistent per event overwrite term in the evlist. Make it possible to support either non-overwrite or overwrite mode. The overwrite mode is forbidden now, which will be removed when the overwrite mode is supported later.
--- tools/perf/builtin-top.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+)
tools/perf/builtin-top.c +60 -0
@@ -881,6 +881,56 @@ static void perf_top__mmap_read(struct perf_top *top)perf_top__mmap_read_idx (top, i); } +/* +* Check per event overwrite term. +* perf top supports consistent mode for all events. +* Return -1 if the per event terms set but not consistent. +*/ +static int perf_top_overwrite_check (struct perf_top *top) +{ +struct record_opts *opts = &top->record_opts; +struct perf_evlist *evlist = top->evlist; +struct perf_evsel_config_term *term; +struct list_head *config_terms; +struct perf_evsel *evsel; +int set, overwrite =-1 ; + +evlist__for_each_entry (evlist, evsel) { + set =-1 ; + config_terms = &evsel->config_terms; +list_for_each_entry (term, config_terms, list) { +if (term->type ==PERF_EVSEL__CONFIG_TERM_OVERWRITE ) + set = term->val.overwrite ?1 :0 ; + } + +/* no term for current and previous event (likely) */ +if ((overwrite <0 ) && (set <0 )) +continue ; + +/* has term for both current and previous event, compare */ +if ((overwrite >=0 ) && (set >=0 ) && (overwrite != set)) +return -1 ; + +/* no term for current event but has term for previous one */ +if ((overwrite >=0 ) && (set <0 )) +return -1 ; + +/* has term for current event */ +if ((overwrite <0 ) && (set >=0 )) { +/* if it's first event, set overwrite */ +if (evsel ==perf_evlist__first (evlist)) + overwrite = set; +else +return -1 ; + } + } + +if ((overwrite >=0 ) && (opts->overwrite != overwrite)) + opts->overwrite = overwrite; + +return 0 ; +} +static int perf_top__start_counters (struct perf_top *top) {char msg[BUFSIZ ];
@@ -890,6 +940,16 @@ static int perf_top__start_counters(struct perf_top *top)perf_evlist__config (evlist, opts, &callchain_param); +if (perf_top_overwrite_check (top)) { +ui__error ("perf top support consistent mode for all events\n" ); + goto out_err; + } + +if (opts->overwrite) { +ui__error ("not support overwrite mode yet\n" ); + goto out_err; + } +evlist__for_each_entry (evlist, counter) { try_again: if (perf_evsel__open(counter, top->evlist->cpus,
Signature · 1 line
2.5.5
Re: [PATCH V3 08/12] perf top: check per event overwrite term
Quoted 20 lines · On Thu, Dec 21, 2017 at 10:08:50AM -0800, [email protected] wrote:
From: Kan Liang <[email protected]>
Per event overwrite term is not forbidden in perf top, which can bring problems. Because perf top only support non-overwrite mode.
Check and forbid inconsistent per event overwrite term in the evlist. Make it possible to support either non-overwrite or overwrite mode. The overwrite mode is forbidden now, which will be removed when the overwrite mode is supported later.
Signed-off-by Kan Liang <[email protected]>--- tools/perf/builtin-top.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+)tools/perf/builtin-top.c +0
-0
tools/perf/builtin-top.c static void perf_top__mmap_read(struct perf_top *top) · line 887
Quoted 7 lines
tools/perf/builtin-top.c +4
-0@@ -881,6 +881,56 @@ static void perf_top__mmap_read(struct perf_top *top)perf_top__mmap_read_idx (top, i); } +/* +* Check per event overwrite term. +* perf top supports consistent mode for all events. +* Return -1 if the per event terms set but not consistent.
please list the rules for overwrite in the comment are they just top specific?
SNIP
Quoted 17 lines
tools/perf/builtin-top.c +13
-0+if (evsel ==perf_evlist__first (evlist)) + overwrite = set; +else +return -1 ; + } + } + +if ((overwrite >=0 ) && (opts->overwrite != overwrite)) + opts->overwrite = overwrite; + +return 0 ; +} +static int perf_top__start_counters (struct perf_top *top) {char msg[BUFSIZ ];
tools/perf/builtin-top.c static int perf_top__start_counters(struct perf_top *top) · line 941
Quoted 2 lines
tools/perf/builtin-top.c +0
-0@@ -890,6 +940,16 @@ static int perf_top__start_counters(struct perf_top *top)perf_evlist__config (evlist, opts, &callchain_param);
so perf_evlist__config uses opts->overwrite, which you set in your perf_top_overwrite_check call.. I'd think you need to call it sooner
jirka
RE: [PATCH V3 08/12] perf top: check per event overwrite term
Quoted 26 lines · On Thu, Dec 21, 2017 at 10:08:50AM -0800, [email protected] wrote:
Quoted 5 lines
From: Kan Liang <[email protected]>
Per event overwrite term is not forbidden in perf top, which can bring problems. Because perf top only support non-overwrite mode.
Quoted 8 lines
Check and forbid inconsistent per event overwrite term in the evlist. Make it possible to support either non-overwrite or overwrite mode. The overwrite mode is forbidden now, which will be removed when the overwrite mode is supported later.
Signed-off-by Kan Liang <[email protected]>--- tools/perf/builtin-top.c | 60++++++++++++++++++++++++++++++++++++++++++++++++
tools/perf/builtin-top.c static void perf_top__mmap_read(struct perf_top
Quoted 7 lines
1 file changed, 60 insertions(+)tools/perf/builtin-top.c +0
-0@@ -881,6 +881,56 @@ static void perf_top__mmap_read(struct perf_top*top)
Quoted 3 lines
tools/perf/builtin-top.c +0
-0perf_top__mmap_read_idx (top, i); }
Quoted 7 lines
tools/perf/builtin-top.c static void perf_top__mmap_read(struct perf_top *top) · line 887
Quoted 4 lines
tools/perf/builtin-top.c +4
-0+/* +* Check per event overwrite term. +* perf top supports consistent mode for all events. +* Return -1 if the per event terms set but not consistent. please list the rules for overwrite in the comment are they just top specific?
Yes, it's only for perf top.
It doesn't support that some events are overwrite mode and other events are non-overwrite mode.
I will refine the comments.
Quoted 22 lines
SNIP
tools/perf/builtin-top.c static int perf_top__start_counters(struct
Quoted 17 lines
tools/perf/builtin-top.c +13
-0+if (evsel ==perf_evlist__first (evlist)) + overwrite = set; +else +return -1 ; + } + } + +if ((overwrite >=0 ) && (opts->overwrite != overwrite)) + opts->overwrite = overwrite; + +return 0 ; +} +static int perf_top__start_counters (struct perf_top *top) {char msg[BUFSIZ ];@@ -890,6 +940,16 @@ static int perf_top__start_counters(struct
perf_top*top)Quoted 1 line
Quoted 6 lines
Quoted 1 line
perf_evlist__config(evlist, opts, &callchain_param);so
perf_evlist__configusesopts->overwrite, which you set in yourperf_top_overwrite_checkcall.. I'd think you need to call it sooner
User may set per-event mode, which is processed in perf_evlist__config It's possible that the per-event mode is not same as opts->overwrite. If so, perf_evlist__config actually uses per-event mode.
perf_top_overwrite_check will do the check. If the per-event mode is inconsistent as opts->overwrite, updating the opts->overwrite.
It has to be called after the perf_evlist__config.
Thanks,
Kan
Re: [PATCH V3 08/12] perf top: check per event overwrite term
Quoted 68 lines · Kan
Quoted 26 lines · On Thu, Dec 21, 2017 at 10:08:50AM -0800, [email protected] wrote:
Quoted 5 lines
From: Kan Liang <[email protected]>
Per event overwrite term is not forbidden in perf top, which can bring problems. Because perf top only support non-overwrite mode.
Quoted 8 lines
Check and forbid inconsistent per event overwrite term in the evlist. Make it possible to support either non-overwrite or overwrite mode. The overwrite mode is forbidden now, which will be removed when the overwrite mode is supported later.
Signed-off-by Kan Liang <[email protected]>--- tools/perf/builtin-top.c | 60++++++++++++++++++++++++++++++++++++++++++++++++
tools/perf/builtin-top.c static void perf_top__mmap_read(struct perf_top
Quoted 7 lines
1 file changed, 60 insertions(+)tools/perf/builtin-top.c +0
-0@@ -881,6 +881,56 @@ static void perf_top__mmap_read(struct perf_top*top)
Quoted 3 lines
tools/perf/builtin-top.c +0
-0perf_top__mmap_read_idx (top, i); }Quoted 7 lines
tools/perf/builtin-top.c static void perf_top__mmap_read(struct perf_top *top) · line 887
Quoted 4 lines
tools/perf/builtin-top.c +4
-0+/* +* Check per event overwrite term. +* perf top supports consistent mode for all events. +* Return -1 if the per event terms set but not consistent. please list the rules for overwrite in the comment are they just top specific?
Yes, it's only for perf top.
It doesn't support that some events are overwrite mode and other events are non-overwrite mode.
I will refine the comments.Quoted 22 lines
SNIP
tools/perf/builtin-top.c static int perf_top__start_counters(struct
Quoted 17 lines
tools/perf/builtin-top.c +13
-0+if (evsel ==perf_evlist__first (evlist)) + overwrite = set; +else +return -1 ; + } + } + +if ((overwrite >=0 ) && (opts->overwrite != overwrite)) + opts->overwrite = overwrite; + +return 0 ; +} +static int perf_top__start_counters (struct perf_top *top) {char msg[BUFSIZ ];@@ -890,6 +940,16 @@ static int perf_top__start_counters(struct
perf_top*top)Quoted 1 line
Quoted 6 lines
Quoted 1 line
perf_evlist__config(evlist, opts, &callchain_param);so
perf_evlist__configusesopts->overwrite, which you set in yourperf_top_overwrite_checkcall.. I'd think you need to call it sooner
Quoted 8 lines
User may set per-event mode, which is processed in
perf_evlist__configIt's possible that the per-event mode is not same asopts->overwrite. If so,perf_evlist__configactually uses per-event mode.
perf_top_overwrite_checkwill do the check. If the per-event mode is inconsistent asopts->overwrite, updating theopts->overwrite.It has to be called after the
perf_evlist__config.
let's see the rules then first, also wrt opts->overwrite, which is used later on I assume
thanks,
jirka
RE: [PATCH V3 08/12] perf top: check per event overwrite term
Quoted 73 lines · Kan
Quoted 71 lines
Quoted 26 lines · On Thu, Dec 21, 2017 at 10:08:50AM -0800, [email protected] wrote:
Quoted 5 lines
From: Kan Liang <[email protected]>
Per event overwrite term is not forbidden in perf top, which can bring problems. Because perf top only support non-overwrite mode.
Quoted 8 lines
Check and forbid inconsistent per event overwrite term in the evlist. Make it possible to support either non-overwrite or overwrite mode. The overwrite mode is forbidden now, which will be removed when the overwrite mode is supported later.
Signed-off-by Kan Liang <[email protected]>--- tools/perf/builtin-top.c | 60++++++++++++++++++++++++++++++++++++++++++++++++
Quoted 8 lines
1 file changed, 60 insertions(+)tools/perf/builtin-top.c +0
-0@@ -881,6 +881,56 @@ static void perf_top__mmap_read(struct
perf_top*top)
Quoted 2 lines
perf_top__mmap_read_idx(top, i); }Quoted 7 lines
Quoted 4 lines
+/*
+ * Check per event overwrite term.
+ * perf top supports consistent mode for all events.
+ * Return -1 if the per event terms set but not consistent.please list the rules for overwrite in the comment are they just top specific?
Yes, it's only for perf top.
It doesn't support that some events are overwrite mode and other events are non-overwrite mode.
I will refine the comments.Quoted 21 lines
SNIP
tools/perf/builtin-top.c static int perf_top__start_counters(struct
Quoted 16 lines
tools/perf/builtin-top.c +13
-0+if (evsel ==perf_evlist__first (evlist)) + overwrite = set; +else +return -1 ; + } + } + +if ((overwrite >=0 ) && (opts->overwrite != overwrite)) + opts->overwrite = overwrite; + +return 0 ; +} +static int perf_top__start_counters (struct perf_top *top) {char msg[BUFSIZ ];@@ -890,6 +940,16 @@ static int perf_top__start_counters(struct
perf_top*top)Quoted 1 line
Quoted 6 lines
Quoted 1 line
perf_evlist__config(evlist, opts, &callchain_param);so
perf_evlist__configusesopts->overwrite, which you set in yourperf_top_overwrite_checkcall.. I'd think you need to call it soonerUser may set per-event mode, which is processed in
perf_evlist__configIt's possible that the per-event mode is not same asopts->overwrite. If so,perf_evlist__configactually uses per-event mode.
Quoted 8 lines
Quoted 4 lines
perf_top_overwrite_checkwill do the check. If the per-event mode is inconsistent asopts->overwrite, updating theopts->overwrite.It has to be called after the
perf_evlist__config.let's see the rules then first, also wrt
opts->overwrite, which is used later on I assume
The rules for perf_top_overwrite_check is quite simple.
- All events have same per-event mode.
E.g. "cpu/cpu-cycles,no-overwrite/,cpu/instructions,no-overwrite/ Using the per-event setting to replace the opts->overwrite if they are different.
- Events have different per-event mode.
E.g. "cpu/cpu-cycles,overwrite/,cpu/instructions,no-overwrite/" Error out.
- Some of the event set per-event mode, but some not.
E.g. "cpu/cpu-cycles, /,cpu/instructions,no-overwrite/"
Error out.
There is no --overwrite option support for perf top.
We have a default opts->overwrite.
In perf_evlist__config, attr->write_backward will be set according to opts->overwrite at first. But if there is per-event mode, the attr->write_backward will be updated accordingly.
There may be inconsistent between opts->overwrite and attr->write_backward.
perf_top_overwrite_check will be called right after perf_evlist__config. It can fix the inconsistence by updating the opts->overwrite or error out. The opts->overwrite will be used by fallback and _mmap_read() later.
Thanks,
Kan
RE: [PATCH V3 08/12] perf top: check per event overwrite term
In reply to Jiri OlsaQuoted 103 lines
Quoted 64 lines · Kan
Quoted 62 lines
Quoted 26 lines · On Thu, Dec 21, 2017 at 10:08:50AM -0800, [email protected] wrote:
Quoted 5 lines
From: Kan Liang <[email protected]>
Per event overwrite term is not forbidden in perf top, which can bring problems. Because perf top only support non-overwrite mode.
Quoted 8 lines
Check and forbid inconsistent per event overwrite term in the evlist. Make it possible to support either non-overwrite or overwrite mode. The overwrite mode is forbidden now, which will be removed when the overwrite mode is supported later.
Signed-off-by Kan Liang <[email protected]>--- tools/perf/builtin-top.c | 60++++++++++++++++++++++++++++++++++++++++++++++++
Quoted 8 lines
1 file changed, 60 insertions(+)tools/perf/builtin-top.c +0
-0@@ -881,6 +881,56 @@ static void perf_top__mmap_read(struct
perf_top*top)
Quoted 2 lines
perf_top__mmap_read_idx(top, i); }Quoted 7 lines
Quoted 4 lines
+/*
+ * Check per event overwrite term.
+ * perf top supports consistent mode for all events.
+ * Return -1 if the per event terms set but not consistent.please list the rules for overwrite in the comment are they just top specific?
Yes, it's only for perf top.
It doesn't support that some events are overwrite mode and other events are non-overwrite mode.
I will refine the comments.Quoted 23 lines
SNIP
tools/perf/builtin-top.c static int perf_top__start_counters(struct
Quoted 16 lines
tools/perf/builtin-top.c +13
-0+if (evsel ==perf_evlist__first (evlist)) + overwrite = set; +else +return -1 ; + } + } + +if ((overwrite >=0 ) && (opts->overwrite != overwrite)) + opts->overwrite = overwrite; + +return 0 ; +} +static int perf_top__start_counters (struct perf_top *top) {char msg[BUFSIZ ];@@ -890,6 +940,16 @@ static int perf_top__start_counters(struct
perf_top*top)Quoted 2 lines
tools/perf/builtin-top.c +0
-0perf_evlist__config (evlist, opts, &callchain_param);Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
so
perf_evlist__configusesopts->overwrite, which you set in yourperf_top_overwrite_checkcall.. I'd think you need to call it soonerUser may set per-event mode, which is processed in
perf_evlist__configIt's possible that the per-event mode is not same as
opts->overwrite.Quoted 10 lines
Quoted 6 lines
If so,
perf_evlist__configactually uses per-event mode.
perf_top_overwrite_checkwill do the check. If the per-event mode is inconsistent asopts->overwrite, updating theopts->overwrite.It has to be called after the
perf_evlist__config.let's see the rules then first, also wrt
opts->overwrite, which is used later on I assumeThe rules for
perf_top_overwrite_checkis quite simple.
- All events have same per-event mode.
E.g. "cpu/cpu-cycles,no-overwrite/,cpu/instructions,no-overwrite/ Using the per-event setting to replace theopts->overwriteif they are
different.
- Events have different per-event mode.
E.g. "cpu/cpu-cycles,overwrite/,cpu/instructions,no-overwrite/" Error out.
- Some of the event set per-event mode, but some not.
E.g. "cpu/cpu-cycles, /,cpu/instructions,no-overwrite/" Error out.There is no --overwrite option support for perf top.
We have a defaultopts->overwrite.
Inperf_evlist__config,attr->write_backwardwill be set according toopts->overwriteat first. But if there is per-event mode, theattr->write_backwardwill be updated accordingly.
There may be inconsistent betweenopts->overwriteandattr->write_backward.
Quoted 4 lines
perf_top_overwrite_checkwill be called right afterperf_evlist__config. It can fix the inconsistence by updating theopts->overwriteor error out. Theopts->overwritewill be used by fallback and_mmap_read()later.
Check the code again.Perf_top_overwrite_check() reply on evsel->config_terms to check the terms, which is already parsed before perf_evlist__config(). You are right, we can call it earlier.
I will move the codes right before the perf_evlist__config() in V4.
Thanks,
Kan
PATCH v3 9/12 perf evsel: expose perf_missing_features.write_backward
In reply to [email protected]From: Kan Liang <[email protected]>
perf top need it to handle overwrite fallback later.
--- tools/perf/util/evsel.c | 5 +++++ tools/perf/util/evsel.h | 2 ++ 2 files changed, 7 insertions(+)
tools/perf/util/evsel.c +5 -0
@@ -1669,6 +1669,11 @@ static bool ignore_missing_thread(struct perf_evsel *evsel,return true; } +bool is_write_backward_fail (void ) +{ +return perf_missing_features.write_backward; +} +int perf_evsel__open (struct perf_evsel *evsel,struct cpu_map *cpus,struct thread_map *threads) {
tools/perf/util/evsel.h +2 -0
@@ -448,4 +448,6 @@ int perf_event_attr__fprintf(FILE *fp, struct perf_event_attr *attr,struct perf_env *perf_evsel__env (struct perf_evsel *evsel); +bool is_write_backward_fail (void ); +#endif /* __PERF_EVSEL_H */
Signature · 1 line
2.5.5
PATCH v3 10/12 perf top: add overwrite fall back
In reply to [email protected]From: Kan Liang <[email protected]>
Switch to non-overwrite mode if kernel doesnot support overwrite ringbuffer.
It's only effect when overwrite mode is supported. No change to current behavior.
--- tools/perf/builtin-top.c | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+)
tools/perf/builtin-top.c +36 -0
@@ -931,6 +931,27 @@ static int perf_top_overwrite_check(struct perf_top *top)return 0 ; } +static int perf_top_overwrite_fallback (struct perf_top *top, +struct perf_evsel *evsel) +{ +struct record_opts *opts = &top->record_opts; +struct perf_evlist *evlist = top->evlist; +struct perf_evsel *counter; + +if (!opts->overwrite) +return 0 ; + +/* only fall back when first event fails */ +if (evsel !=perf_evlist__first (evlist)) +return 0 ; + +evlist__for_each_entry (evlist, counter) + counter->attr.write_backward = false; + opts->overwrite = false; +ui__warning ("fall back to non-overwrite mode\n" ); +return 1 ; +} +static int perf_top__start_counters (struct perf_top *top) {char msg[BUFSIZ ];
@@ -954,6 +975,21 @@ static int perf_top__start_counters(struct perf_top *top) try_again:if (perf_evsel__open (counter, top->evlist->cpus, top->evlist->threads) <0 ) { + +/* +* Specially handle overwrite fall back. +* Because perf top is the only tool which has +* overwrite mode by default, support +* both overwrite and non-overwrite mode, and +* require consistent mode for all events. +* +* May move it to generic code with more tools +* have similar attribute. +*/ +if (is_write_backward_fail () && +perf_top_overwrite_fallback (top, counter)) + goto try_again; +if (perf_evsel__fallback (counter, errno, msg,sizeof (msg))) {if (verbose >0 )ui__warning ("%s\n" , msg);
Signature · 1 line
2.5.5
Re: [PATCH V3 10/12] perf top: add overwrite fall back
Quoted 31 lines · On Thu, Dec 21, 2017 at 10:08:52AM -0800, [email protected] wrote:
From: Kan Liang <[email protected]>
Switch to non-overwrite mode if kernel doesnot support overwrite ringbuffer.
It's only effect when overwrite mode is supported. No change to current behavior.
Signed-off-by Kan Liang <[email protected]>--- tools/perf/builtin-top.c | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+)tools/perf/builtin-top.c +10
-0@@ -931,6 +931,27 @@ static int perf_top_overwrite_check(struct perf_top *top)return 0 ; } +static int perf_top_overwrite_fallback (struct perf_top *top, +struct perf_evsel *evsel) +{ +struct record_opts *opts = &top->record_opts; +struct perf_evlist *evlist = top->evlist; +struct perf_evsel *counter; + +if (!opts->overwrite) +return 0 ; +
tools/perf/builtin-top.c static int perf_top_overwrite_check(struct perf_top *top) · line 953
Quoted 10 lines
tools/perf/builtin-top.c +10
-0+/* only fall back when first event fails */ +if (evsel !=perf_evlist__first (evlist)) +return 0 ; + +evlist__for_each_entry (evlist, counter) + counter->attr.write_backward = false; + opts->overwrite = false; +ui__warning ("fall back to non-overwrite mode\n" ); +return 1 ; +}
we already do that for evsel in perf_evsel__open.. could we make this fallback on one place only?
checking the code, why don't we follow the perf_missing_features.write_backward like we do for other features in the perf_evsel__open and set write_backward accordingly, like in attached patch.
jirka
---
tools/perf/util/evsel.c +2 -3
@@ -1683,9 +1683,6 @@ int perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus,int pid =-1 , err;enum {NO_CHANGE ,SET_TO_MAX ,INCREASED_MAX } set_rlimit =NO_CHANGE ; -if (perf_missing_features.write_backward && evsel->attr.write_backward) -return -EINVAL ; -if (cpus ==NULL ) {static struct cpu_map *empty_cpu_map;
@@ -1725,6 +1722,8 @@ int perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus, } fallback_missing_features: +if (perf_missing_features.write_backward) + evsel->attr.write_backward = false;if (perf_missing_features.clockid_wrong) evsel->attr.clockid =CLOCK_MONOTONIC ;/* should always work */ if (perf_missing_features.clockid) {
RE: [PATCH V3 10/12] perf top: add overwrite fall back
Quoted 47 lines · On Thu, Dec 21, 2017 at 10:08:52AM -0800, [email protected] wrote:
tools/perf/builtin-top.c static int perf_top_overwrite_check(struct
Quoted 18 lines
From: Kan Liang <[email protected]>
Switch to non-overwrite mode if kernel doesnot support overwrite ringbuffer.
It's only effect when overwrite mode is supported. No change to current behavior.
Signed-off-by Kan Liang <[email protected]>--- tools/perf/builtin-top.c | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+)tools/perf/builtin-top.c +0
-0@@ -931,6 +931,27 @@ static int perf_top_overwrite_check(struct
perf_top*top)Quoted 13 lines
tools/perf/builtin-top.c +10
-0return 0 ; } +static int perf_top_overwrite_fallback (struct perf_top *top, +struct perf_evsel *evsel) +{ +struct record_opts *opts = &top->record_opts; +struct perf_evlist *evlist = top->evlist; +struct perf_evsel *counter; + +if (!opts->overwrite) +return 0 ; +tools/perf/builtin-top.c static int perf_top_overwrite_check(struct · line 953
Quoted 10 lines
tools/perf/builtin-top.c +10
-0+/* only fall back when first event fails */ +if (evsel !=perf_evlist__first (evlist)) +return 0 ; + +evlist__for_each_entry (evlist, counter) + counter->attr.write_backward = false; + opts->overwrite = false; +ui__warning ("fall back to non-overwrite mode\n" ); +return 1 ; +}we already do that for evsel in
perf_evsel__open.. could we make this fallback on one place only?
Quoted 5 lines
checking the code, why don't we follow the perf_missing_features.write_backward
like we do for other features in theperf_evsel__openand setwrite_backwardaccordingly, like in attached patch.
No, the per-event fallback is explicitly disabled in perf_evsel__open. You may refer to commit 32a951b4fd6b ("perf evlist: Drop redundant evsel->overwrite indicator)".
Now, only perf top supports the whole fallback. So it is specially handled.
Thanks,
Kan
Quoted 32 lines
jirka
---
tools/perf/util/evsel.c +0
-3@@ -1683,9 +1683,6 @@ int perf_evsel__open(struct perf_evsel *evsel,struct cpu_map *cpus, int pid = -1, err;enum {NO_CHANGE ,SET_TO_MAX ,INCREASED_MAX } set_rlimit =NO_CHANGE ; -if (perf_missing_features.write_backward && evsel- >attr.write_backward) -return -EINVAL ; -tools/perf/util/evsel.c +2
-0if (cpus ==NULL ) {static struct cpu_map *empty_cpu_map;@@ -1725,6 +1722,8 @@ int perf_evsel__open(struct perf_evsel *evsel, struct cpu_map *cpus, } fallback_missing_features: +if (perf_missing_features.write_backward) + evsel->attr.write_backward = false;if (perf_missing_features.clockid_wrong) evsel->attr.clockid =CLOCK_MONOTONIC ;/* should always work */
if (perf_missing_features.clockid) {
Re: [PATCH V3 10/12] perf top: add overwrite fall back
Quoted 53 lines · Kan
Quoted 47 lines · On Thu, Dec 21, 2017 at 10:08:52AM -0800, [email protected] wrote:
tools/perf/builtin-top.c static int perf_top_overwrite_check(struct
Quoted 18 lines
From: Kan Liang <[email protected]>
Switch to non-overwrite mode if kernel doesnot support overwrite ringbuffer.
It's only effect when overwrite mode is supported. No change to current behavior.
Signed-off-by Kan Liang <[email protected]>--- tools/perf/builtin-top.c | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+)tools/perf/builtin-top.c +0
-0@@ -931,6 +931,27 @@ static int perf_top_overwrite_check(struct
perf_top*top)Quoted 13 lines
tools/perf/builtin-top.c +10
-0return 0 ; } +static int perf_top_overwrite_fallback (struct perf_top *top, +struct perf_evsel *evsel) +{ +struct record_opts *opts = &top->record_opts; +struct perf_evlist *evlist = top->evlist; +struct perf_evsel *counter; + +if (!opts->overwrite) +return 0 ; +tools/perf/builtin-top.c static int perf_top_overwrite_check(struct · line 953
Quoted 10 lines
tools/perf/builtin-top.c +10
-0+/* only fall back when first event fails */ +if (evsel !=perf_evlist__first (evlist)) +return 0 ; + +evlist__for_each_entry (evlist, counter) + counter->attr.write_backward = false; + opts->overwrite = false; +ui__warning ("fall back to non-overwrite mode\n" ); +return 1 ; +}we already do that for evsel in
perf_evsel__open.. could we make this fallback on one place only?Quoted 5 lines
checking the code, why don't we follow the perf_missing_features.write_backward
like we do for other features in theperf_evsel__openand setwrite_backwardaccordingly, like in attached patch.
Quoted 5 lines
No, the per-event fallback is explicitly disabled in
perf_evsel__open. You may refer to commit32a951b4fd6b("perf evlist: Drop redundantevsel->overwriteindicator)".Now, only perf top supports the whole fallback. So it is specially handled.
ok
jirka
PATCH v3 11/12 perf top: switch default mode to overwrite mode
In reply to [email protected]From: Kan Liang <[email protected]>
perf_top__mmap_read has severe performance issue in Knights Landing/Mill, when monitoring in heavy load system. It costs several minutes to finish, which is unacceptable.
Currently, perf top is non overwrite mode. For non overwrite mode, it tries to read everything in the ringbuffer and doesn't pause the ringbuffer. Once there are lots of samples delivered persistently, the processing time could be very long. Also, the latest samples could be lost when the ringbuffer is full.
For overwrite mode, it takes a snapshot for the system by pausing the ringbuffer, which could significantly reducing the processing time. Also, the overwrite mode always keep the latest samples. Considering the real time requirement for perf top, the overwrite mode is more suitable for perf top.
Actually, perf top was overwrite mode. It is changed to non overwrite mode since commit 93fc64f14472 ("perf top: Switch to non overwrite mode"). It's better to change it back to overwrite mode by default.
For the kernel which doesn't support overwrite mode, it will fall back to non overwrite mode.
There would be some records lost in overwrite mode because of pausing the ringbuffer. It has little impact for the accuracy of the snapshot and could be tolerant.
The lost events checking is removed.
For overwrite mode, unconditionally wait 100 ms before each snapshot. It also reduce the overhead caused by pausing ringbuffer, especially on light load system.
--- tools/perf/builtin-top.c | 44 ++++++++++++++++++++++++------------------ tools/perf/ui/browsers/hists.c | 12 +++++++++--- 2 files changed, 34 insertions(+), 22 deletions(-)
tools/perf/builtin-top.c +25 -19
@@ -283,16 +283,6 @@ static void perf_top__print_sym_table(struct perf_top *top)printf ("%-*.*s\n" , win_width, win_width, graph_dotted_line); -if (hists->stats.nr_lost_warned != - hists->stats.nr_events[PERF_RECORD_LOST ]) { - hists->stats.nr_lost_warned = - hists->stats.nr_events[PERF_RECORD_LOST ]; -color_fprintf (stdout,PERF_COLOR_RED , -"WARNING: LOST %d chunks, Check IO/CPU overload" , - hists->stats.nr_lost_warned); - ++printed; - } -if (top->sym_filter_entry) {perf_top__show_details (top);return ;
@@ -807,15 +797,23 @@ static void perf_event__process_sample(struct perf_tool *tool,static void perf_top__mmap_read_idx (struct perf_top *top,int idx) { +struct record_opts *opts = &top->record_opts; +struct perf_evlist *evlist = top->evlist;struct perf_sample sample;struct perf_evsel *evsel; +struct perf_mmap *md;struct perf_session *session = top->session;union perf_event *event;struct machine *machine; +u64 end, start;int ret; -while ((event =perf_evlist__mmap_read (top->evlist, idx)) !=NULL ) { - ret =perf_evlist__parse_sample (top->evlist, event, &sample); + md = opts->overwrite ? &evlist->overwrite_mmap[idx] : &evlist->mmap[idx]; +if (perf_mmap__read_init (md, opts->overwrite, &start, &end) <0 ) +return ; + +while ((event =perf_mmap__read_event (md, opts->overwrite, &start, end)) !=NULL ) { + ret =perf_evlist__parse_sample (evlist, event, &sample);if (ret) {pr_err ("Can't parse sample, err = %d\n" , ret); goto next_event;
@@ -869,16 +867,28 @@ static void perf_top__mmap_read_idx(struct perf_top *top, int idx) }else ++session->evlist->stats.nr_unknown_events; next_event: -perf_evlist__mmap_consume (top->evlist, idx); +perf_mmap__consume (md, opts->overwrite); } + +perf_mmap__read_done (md); }static void perf_top__mmap_read (struct perf_top *top) { +bool overwrite = top->record_opts.overwrite; +struct perf_evlist *evlist = top->evlist;int i; +if (overwrite) +perf_evlist__toggle_bkw_mmap (evlist,BKW_MMAP_DATA_PENDING ); +for (i =0 ; i < top->evlist->nr_mmaps; i++)perf_top__mmap_read_idx (top, i); + +if (overwrite) { +perf_evlist__toggle_bkw_mmap (evlist,BKW_MMAP_EMPTY ); +perf_evlist__toggle_bkw_mmap (evlist,BKW_MMAP_RUNNING ); + } }/*
@@ -966,11 +976,6 @@ static int perf_top__start_counters(struct perf_top *top) goto out_err; } -if (opts->overwrite) { -ui__error ("not support overwrite mode yet\n" ); - goto out_err; - } -evlist__for_each_entry (evlist, counter) { try_again: if (perf_evsel__open(counter, top->evlist->cpus,
@@ -1129,7 +1134,7 @@ static int __cmd_top(struct perf_top *top)perf_top__mmap_read (top); -if (hits == top->samples) +if (opts->overwrite || (hits == top->samples)) ret =perf_evlist__poll (top->evlist,100 );if (resize) {
@@ -1223,6 +1228,7 @@ int cmd_top(int argc, const char **argv) .uses_mmap = true, }, .proc_map_timeout = 500, + .overwrite = 1, }, .max_stack = sysctl_perf_event_max_stack, .sym_pcnt_filter = 5,
tools/perf/ui/browsers/hists.c +9 -3
@@ -638,8 +638,13 @@ int hist_browser__run(struct hist_browser *browser, const char *help) nr_entries =hist_browser__nr_entries (browser);ui_browser__update_nr_entries (&browser->b, nr_entries); -if (browser->hists->stats.nr_lost_warned != - browser->hists->stats.nr_events[PERF_RECORD_LOST ]) { +/* +* Don't print lost events warning for perf top, +* because it is overwrite mode. +* Perf top is the only tool which has hbt timer. +*/ +if ((browser->hists->stats.nr_lost_warned != + browser->hists->stats.nr_events[PERF_RECORD_LOST ]) && !hbt) { browser->hists->stats.nr_lost_warned = browser->hists->stats.nr_events[PERF_RECORD_LOST ];ui_browser__warn_lost_events (&browser->b);
@@ -3203,7 +3208,8 @@ static int perf_evsel_menu__run(struct perf_evsel_menu *menu,case K_TIMER : hbt->timer (hbt->arg); -if (!menu->lost_events_warned && menu->lost_events) { +if (!menu->lost_events_warned && + menu->lost_events && !hbt) {ui_browser__warn_lost_events (&menu->b); menu->lost_events_warned = true; }
Signature · 1 line
2.5.5
Re: [PATCH V3 11/12] perf top: switch default mode to overwrite mode
On Thu, Dec 21, 2017 at 10:08:53AM -0800, [email protected] wrote:
SNIP
Quoted 7 lines
tools/perf/builtin-top.c +0
-0.max_stack = sysctl_perf_event_max_stack, .sym_pcnt_filter = 5,tools/perf/ui/browsers/hists.c +0
-0
tools/perf/ui/browsers/hists.c int hist_browser__run(struct hist_browser *browser, const char *help) · line 645
Quoted 10 lines
tools/perf/ui/browsers/hists.c +5
-2@@ -638,8 +638,13 @@ int hist_browser__run(struct hist_browser *browser, const char *help) nr_entries =hist_browser__nr_entries (browser);ui_browser__update_nr_entries (&browser->b, nr_entries); -if (browser->hists->stats.nr_lost_warned != - browser->hists->stats.nr_events[PERF_RECORD_LOST ]) { +/* +* Don't print lost events warning for perf top, +* because it is overwrite mode. +* Perf top is the only tool which has hbt timer. +*/
I hate that warning, but still should it be ommited only for overwrite case? also please separate this change from the rest of the patch
jirka
Quoted 18 lines
tools/perf/ui/browsers/hists.c +4
-1+if ((browser->hists->stats.nr_lost_warned != + browser->hists->stats.nr_events[PERF_RECORD_LOST ]) && !hbt) { browser->hists->stats.nr_lost_warned = browser->hists->stats.nr_events[PERF_RECORD_LOST ];ui_browser__warn_lost_events (&browser->b);@@ -3203,7 +3208,8 @@ static int perf_evsel_menu__run(struct perf_evsel_menu *menu,case K_TIMER : hbt->timer (hbt->arg); -if (!menu->lost_events_warned && menu->lost_events) { +if (!menu->lost_events_warned && + menu->lost_events && !hbt) {ui_browser__warn_lost_events (&menu->b); menu->lost_events_warned = true; }Signature · 1 line
2.5.5
RE: [PATCH V3 11/12] perf top: switch default mode to overwrite mode
Quoted 16 lines
SNIP
tools/perf/ui/browsers/hists.c int hist_browser__run(struct hist_browser
Quoted 7 lines
tools/perf/builtin-top.c +0
-0.max_stack = sysctl_perf_event_max_stack, .sym_pcnt_filter = 5,tools/perf/ui/browsers/hists.c +0
-0@@ -638,8 +638,13 @@ int hist_browser__run(struct hist_browser*browser, const char *help)
tools/perf/ui/browsers/hists.c int hist_browser__run(struct hist_browser · line 639
Quoted 2 lines
tools/perf/ui/browsers/hists.c +0
-0nr_entries =hist_browser__nr_entries (browser); ui_browser__update_nr_entries(&browser->b,
nr_entries);Quoted 1 line
Quoted 11 lines
tools/perf/ui/browsers/hists.c int hist_browser__run(struct hist_browser *browser, const char *help) · line 645
Quoted 8 lines
tools/perf/ui/browsers/hists.c +0
-2-if (browser->hists->stats.nr_lost_warned != - browser->hists-stats.nr_events[PERF_RECORD_LOST]) {
tools/perf/ui/browsers/hists.c +5
-0+/* +* Don't print lost events warning for perf top, +* because it is overwrite mode. +* Perf top is the only tool which has hbt timer. +*/ I hate that warning, but still should it be ommited only for overwrite case? also please separate this change from the rest of the patch
Sure.
Will do it.
Thanks,
Kan
Quoted 26 lines
jirka
tools/perf/ui/browsers/hists.c static int perf_evsel_menu__run(struct
Quoted 9 lines
tools/perf/ui/browsers/hists.c +2
-0+ if ((browser->hists->stats.nr_lost_warned != + browser->hists-stats.nr_events[PERF_RECORD_LOST]) && !hbt) {
tools/perf/ui/browsers/hists.c +0
-0browser->hists->stats.nr_lost_warned = browser->hists-stats.nr_events[PERF_RECORD_LOST];
ui_browser__warn_lost_events(&browser-
b);tools/perf/ui/browsers/hists.c +0
-0@@ -3203,7 +3208,8 @@ static int perf_evsel_menu__run(struct
perf_evsel_menu*menu,Quoted 13 lines
tools/perf/ui/browsers/hists.c +2
-1case K_TIMER : hbt->timer (hbt->arg); -if (!menu->lost_events_warned && menu- lost_events) { +if (!menu->lost_events_warned && + menu->lost_events && !hbt) {ui_browser__warn_lost_events (&menu->b); menu->lost_events_warned = true;}
--
2.5.5
RE: [PATCH V3 11/12] perf top: switch default mode to overwrite mode
In reply to Jiri OlsaQuoted 17 lines · On Thu, Dec 21, 2017 at 10:08:53AM -0800, [email protected] wrote:
SNIP
tools/perf/ui/browsers/hists.c int hist_browser__run(struct hist_browser
Quoted 7 lines
tools/perf/builtin-top.c +0
-0.max_stack = sysctl_perf_event_max_stack, .sym_pcnt_filter = 5,tools/perf/ui/browsers/hists.c +0
-0@@ -638,8 +638,13 @@ int hist_browser__run(struct hist_browser*browser, const char *help)
tools/perf/ui/browsers/hists.c int hist_browser__run(struct hist_browser · line 639
Quoted 2 lines
tools/perf/ui/browsers/hists.c +0
-0nr_entries =hist_browser__nr_entries (browser); ui_browser__update_nr_entries(&browser->b,
nr_entries);Quoted 1 line
Quoted 12 lines
tools/perf/ui/browsers/hists.c int hist_browser__run(struct hist_browser *browser, const char *help) · line 645
Quoted 8 lines
tools/perf/ui/browsers/hists.c +0
-2-if (browser->hists->stats.nr_lost_warned != - browser->hists-stats.nr_events[PERF_RECORD_LOST]) {
tools/perf/ui/browsers/hists.c +5
-0+/* +* Don't print lost events warning for perf top, +* because it is overwrite mode. +* Perf top is the only tool which has hbt timer. +*/ I hate that warning, but still should it be ommited only for overwrite case? also please separate this change from the rest of the patch
OK. Will do it.
Thanks,
Kan
Quoted 25 lines
jirka
tools/perf/ui/browsers/hists.c static int perf_evsel_menu__run(struct
Quoted 9 lines
tools/perf/ui/browsers/hists.c +2
-0+ if ((browser->hists->stats.nr_lost_warned != + browser->hists-stats.nr_events[PERF_RECORD_LOST]) && !hbt) {
tools/perf/ui/browsers/hists.c +0
-0browser->hists->stats.nr_lost_warned = browser->hists-stats.nr_events[PERF_RECORD_LOST];
ui_browser__warn_lost_events(&browser-
b);tools/perf/ui/browsers/hists.c +0
-0@@ -3203,7 +3208,8 @@ static int perf_evsel_menu__run(struct
perf_evsel_menu*menu,Quoted 13 lines
tools/perf/ui/browsers/hists.c +2
-1case K_TIMER : hbt->timer (hbt->arg); -if (!menu->lost_events_warned && menu- lost_events) { +if (!menu->lost_events_warned && + menu->lost_events && !hbt) {ui_browser__warn_lost_events (&menu->b); menu->lost_events_warned = true;}
--
2.5.5
Re: [PATCH V3 11/12] perf top: switch default mode to overwrite mode
In reply to [email protected]Quoted 48 lines · On Thu, Dec 21, 2017 at 10:08:53AM -0800, [email protected] wrote:
From: Kan Liang <[email protected]>
perf_top__mmap_readhas severe performance issue in Knights Landing/Mill, when monitoring in heavy load system. It costs several minutes to finish, which is unacceptable.Currently, perf top is non overwrite mode. For non overwrite mode, it tries to read everything in the ringbuffer and doesn't pause the ringbuffer. Once there are lots of samples delivered persistently, the processing time could be very long. Also, the latest samples could be lost when the ringbuffer is full.
For overwrite mode, it takes a snapshot for the system by pausing the ringbuffer, which could significantly reducing the processing time. Also, the overwrite mode always keep the latest samples. Considering the real time requirement for perf top, the overwrite mode is more suitable for perf top.
Actually, perf top was overwrite mode. It is changed to non overwrite mode since commit
93fc64f14472("perf top: Switch to non overwrite mode"). It's better to change it back to overwrite mode by default.For the kernel which doesn't support overwrite mode, it will fall back to non overwrite mode.
There would be some records lost in overwrite mode because of pausing the ringbuffer. It has little impact for the accuracy of the snapshot and could be tolerant.
The lost events checking is removed.For overwrite mode, unconditionally wait 100 ms before each snapshot. It also reduce the overhead caused by pausing ringbuffer, especially on light load system.
Signed-off-by Kan Liang <[email protected]>--- tools/perf/builtin-top.c | 44 ++++++++++++++++++++++++------------------ tools/perf/ui/browsers/hists.c | 12 +++++++++--- 2 files changed, 34 insertions(+), 22 deletions(-)tools/perf/builtin-top.c +0
-0@@ -283,16 +283,6 @@ static void perf_top__print_sym_table(struct perf_top *top)printf ("%-*.*s\n" , win_width, win_width, graph_dotted_line);
tools/perf/builtin-top.c static void perf_top__print_sym_table(struct perf_top *top) · line 295
Quoted 10 lines
tools/perf/builtin-top.c +0
-10-if (hists->stats.nr_lost_warned != - hists->stats.nr_events[PERF_RECORD_LOST ]) { - hists->stats.nr_lost_warned = - hists->stats.nr_events[PERF_RECORD_LOST ]; -color_fprintf (stdout,PERF_COLOR_RED , -"WARNING: LOST %d chunks, Check IO/CPU overload" , - hists->stats.nr_lost_warned); - ++printed; - } -
should this stay for non-overwrite mode?
jirka
PATCH v3 12/12 perf top: check the latency of perf_top__mmap_read
In reply to [email protected]From: Kan Liang <[email protected]>
The latency of perf_top__mmap_read should be lower than refresh time. If not, give some hints to reduce the latency.
--- tools/perf/builtin-top.c | 9 +++++++++ 1 file changed, 9 insertions(+)
tools/perf/builtin-top.c +9 -0
@@ -877,8 +877,10 @@ static void perf_top__mmap_read(struct perf_top *top) {bool overwrite = top->record_opts.overwrite;struct perf_evlist *evlist = top->evlist; +unsigned long long start, end;int i; + start =rdclock ();if (overwrite)perf_evlist__toggle_bkw_mmap (evlist,BKW_MMAP_DATA_PENDING );
@@ -889,6 +891,13 @@ static void perf_top__mmap_read(struct perf_top *top)perf_evlist__toggle_bkw_mmap (evlist,BKW_MMAP_EMPTY );perf_evlist__toggle_bkw_mmap (evlist,BKW_MMAP_RUNNING ); } + end =rdclock (); + +if ((end - start) > (unsigned long long )top->delay_secs *NSEC_PER_SEC ) +ui__warning ("Too slow to read ring buffer.\n" +"Please try increasing the period (-c) or\n" +"decreasing the freq (-F) or\n" +"limiting the number of CPUs (-C)\n" ); }/*
Signature · 1 line
2.5.5