perf pollfd series
Arnaldo Carvalho de Melo
maintainercover perf pollfd series
Hi,
This is an alternative series to the one Jiri Olsa posted to use the
fixes he made to the kernel side to allow tooling to notice that a thread had exited by looking at the pollfd.revents looking for POLLHUP notifications.
Once all event file descriptors are removed from the evlist pollfd array,
tools can make a decision about exiting or telling the user about what happened, asking to guidance on what to do next.
The main difference in this approach is that a new class, which Jiri
called 'poller' and I called 'fdarray', grew up from what was in evlist->pollfd and associated operations, while Jiri first introduced a new class and then made tooling use it.
The details of the implementation should be clear on the changelog
comments, please let me know if you see any problems, and if I can have your acked-by/tested-by/whatever-else tags to get this moving forward.
Ah, there is still one missing thing which is to make the hists browser
in live mode be notified that all monitored events are POLLHUP'ed, will get to that in followup patches.
The kernel bits were sent together with my latest pull req to Ingo,
this series is on top of that branch and is available at:
git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git perf/pollfd
- Arnaldo
Arnaldo Carvalho de Melo (10):
perf evlist: Introduce perf_evlist__filter_pollfd method
perf tests: Add test for perf_evlist__filter_pollfd()
perf evlist: Monitor POLLERR and POLLHUP events too
perf record: Filter out POLLHUP'ed file descriptors
perf trace: Filter out POLLHUP'ed file descriptors
perf evlist: Allow growing pollfd on add method
perf tests: Add pollfd growing test
perf kvm stat live: Use perf_evlist__add_pollfd() instead of local equivalent perf evlist: Introduce poll method for common code idiom
tools lib api: Adopt fdarray class from perf's evlist
Arnaldo Carvalho de Melo
maintainerPATCH 1/10 perf evlist: Introduce perf_evlist__filter_pollfd method
From: Arnaldo Carvalho de Melo <[email protected]>
To remove all entries in evlist->pollfd[] that have revents matching at least one of the bits in the specified mask.
It'll adjust evlist->nr_fds to the number of unfiltered fds and will return this value, as a convenience and to avoid requiring direct access to internal state of perf_evlist objects.
This will be used after polling the evlist fds so that we remove fds that were closed by the kernel.
--- tools/perf/util/evlist.c | 16 ++++++++++++++++ tools/perf/util/evlist.h | 2 ++ 2 files changed, 18 insertions(+)
tools/perf/util/evlist.c +16 -0
@@ -428,6 +428,22 @@ void perf_evlist__add_pollfd(struct perf_evlist *evlist, int fd) evlist->nr_fds++; } +int perf_evlist__filter_pollfd (struct perf_evlist *evlist,short revents_and_mask) +{ +int fd =0 , nr_fds =0 ; + +while (fd < evlist->nr_fds) { +if ((evlist->pollfd[fd].revents & revents_and_mask) ==0 ) + ++nr_fds; + +if (++fd != nr_fds) + evlist->pollfd[nr_fds] = evlist->pollfd[fd]; + } + + evlist->nr_fds = nr_fds; +return nr_fds; +} +static void perf_evlist__id_hash (struct perf_evlist *evlist,struct perf_evsel *evsel,int cpu,int thread,u64 id)
tools/perf/util/evlist.h +2 -0
@@ -84,6 +84,8 @@ void perf_evlist__id_add(struct perf_evlist *evlist, struct perf_evsel *evsel,void perf_evlist__add_pollfd (struct perf_evlist *evlist,int fd); +int perf_evlist__filter_pollfd (struct perf_evlist *evlist,short revents_and_mask); +struct perf_evsel *perf_evlist__id2evsel (struct perf_evlist *evlist,u64 id);struct perf_sample_id *perf_evlist__id2sid (struct perf_evlist *evlist,u64 id);
Signature · 1 line
1.9.3
Adrian Hunter
reviewerRe: [PATCH 01/10] perf evlist: Introduce perf_evlist__filter_pollfd method
Quoted 35 lines · Arnaldo Carvalho de Melo
From: Arnaldo Carvalho de Melo <[email protected]>
To remove all entries in evlist->pollfd[] that have revents matching at least one of the bits in the specified mask.
It'll adjust
evlist->nr_fdsto the number of unfiltered fds and will return this value, as a convenience and to avoid requiring direct access to internal state ofperf_evlistobjects.This will be used after polling the evlist fds so that we remove fds that were closed by the kernel.
Cc Adrian Hunter <[email protected]>Cc David Ahern <[email protected]>Cc Don Zickus <[email protected]>Cc Frederic Weisbecker <[email protected]>Cc Ingo Molnar <[email protected]>Cc Jiri Olsa <[email protected]>Cc Mike Galbraith <[email protected]>Cc Namhyung Kim <[email protected]>Cc Paul Mackerras <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Stephane Eranian <[email protected]>Signed-off-by Arnaldo Carvalho de Melo <[email protected]>--- tools/perf/util/evlist.c | 16 ++++++++++++++++ tools/perf/util/evlist.h | 2 ++ 2 files changed, 18 insertions(+)tools/perf/util/evlist.c +0
-0
tools/perf/util/evlist.c void perf_evlist__add_pollfd(struct perf_evlist *evlist, int fd) · line 435
Quoted 8 lines
tools/perf/util/evlist.c +5
-0@@ -428,6 +428,22 @@ void perf_evlist__add_pollfd(struct perf_evlist *evlist, int fd) evlist->nr_fds++; } +int perf_evlist__filter_pollfd (struct perf_evlist *evlist,short revents_and_mask) +{ +int fd =0 , nr_fds =0 ; + +while (fd < evlist->nr_fds) {
A for loop is clearer e.g.
for (fd = 0; fd < evlist->nr_fds; fd++) {tools/perf/util/evlist.c void perf_evlist__add_pollfd(struct perf_evlist *evlist, int fd) · line 440
Quoted 5 lines
tools/perf/util/evlist.c +5
-0+if ((evlist->pollfd[fd].revents & revents_and_mask) ==0 ) + ++nr_fds; + +if (++fd != nr_fds) + evlist->pollfd[nr_fds] = evlist->pollfd[fd];
That looks like it will go off the end of the array. Shouldn't it be:
if ((evlist->pollfd[fd].revents & revents_and_mask) == 0) {
if (fd != nr_fds)
evlist->pollfd[nr_fds] = evlist->pollfd[fd];
nr_fds += 1;
}Quoted 23 lines
tools/perf/util/evlist.c +6
-0+ } + + evlist->nr_fds = nr_fds; +return nr_fds; +} +static void perf_evlist__id_hash (struct perf_evlist *evlist,struct perf_evsel *evsel,int cpu,int thread,u64 id)tools/perf/util/evlist.h +2
-0@@ -84,6 +84,8 @@ void perf_evlist__id_add(struct perf_evlist *evlist, struct perf_evsel *evsel,void perf_evlist__add_pollfd (struct perf_evlist *evlist,int fd); +int perf_evlist__filter_pollfd (struct perf_evlist *evlist,short revents_and_mask); +struct perf_evsel *perf_evlist__id2evsel (struct perf_evlist *evlist,u64 id);struct perf_sample_id *perf_evlist__id2sid (struct perf_evlist *evlist,u64 id);
Arnaldo Carvalho de Melo
maintainerPATCH 2/10 perf tests: Add test for perf_evlist__filter_pollfd()
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
That will use a synthetic evlist with just what is touched by this new method to check that it works as expected.
Output in verbose mode:
$ perf test -v pollfd 33: Filter fds with revents mask in a pollfd array : --- start --- filtering all but pollfd[2]: before: 5 [ 5, 4, 3, 2, 1 ] after: 1 [ 3 ] filtering all but (pollfd[0], pollfd[3]): before: 5 [ 5, 4, 3, 2, 1 ] after: 2 [ 5, 2 ] test child finished with 0 ---- end ---- Filter fds with revents mask in a pollfd array: Ok $
--- tools/perf/Makefile.perf | 1 + tools/perf/tests/builtin-test.c | 4 ++ tools/perf/tests/evlist.c | 103 ++++++++++++++++++++++++++++++++++++++++ tools/perf/tests/tests.h | 1 + 4 files changed, 109 insertions(+) create mode 100644 tools/perf/tests/evlist.c
tools/perf/Makefile.perf +1 -0
@@ -399,6 +399,7 @@ LIB_OBJS += $(OUTPUT)tests/open-syscall-tp-fields.o LIB_OBJS += $(OUTPUT)tests/mmap-basic.o LIB_OBJS += $(OUTPUT)tests/perf-record.o LIB_OBJS += $(OUTPUT)tests/rdpmc.o +LIB_OBJS += $(OUTPUT)tests/evlist.o LIB_OBJS += $(OUTPUT)tests/evsel-roundtrip-name.o LIB_OBJS += $(OUTPUT)tests/evsel-tp-sched.o LIB_OBJS += $(OUTPUT)tests/pmu.o
tools/perf/tests/builtin-test.c +4 -0
@@ -158,6 +158,10 @@ static struct test { .func = test__switch_tracking, }, { + .desc = "Filter fds with revents mask in a pollfd array", + .func = test__perf_evlist__filter_pollfd, + }, + { .func = NULL, }, };
tools/perf/tests/evlist.c new file +103 -0
103 lines not on this page · raw message
tools/perf/tests/tests.h +1 -0
@@ -49,6 +49,7 @@ int test__thread_mg_share(void);int test__hists_output (void );int test__hists_cumulate (void );int test__switch_tracking (void ); +int test__perf_evlist__filter_pollfd (void );#if defined(__x86_64__) || defined(__i386__) || defined(__arm__)#ifdef HAVE_DWARF_UNWIND_SUPPORT
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 3/10 perf evlist: Monitor POLLERR and POLLHUP events too
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
We want to know when the fd went away, like when a monitored thread exits.
If we do not monitor such events, then the tools will wait forever on events from a vanished thread, like when running:
$ sleep 5s & $ perf record -p `pidof sleep`
This builds upon the kernel patch by Jiri Olsa that actually makes a poll on those file descriptors to return POLLHUP.
It is also needed to change the tools to use perf_evlist__filter_pollfd() to check if there are remainings fds to monitor or if all are gone, in which case they will exit the poll/mmap/read loop.
--- tools/perf/util/evlist.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/evlist.c +1 -1
@@ -424,7 +424,7 @@ void perf_evlist__add_pollfd(struct perf_evlist *evlist, int fd) {fcntl (fd,F_SETFL ,O_NONBLOCK ); evlist->pollfd[evlist->nr_fds].fd = fd; - evlist->pollfd[evlist->nr_fds].events =POLLIN ; + evlist->pollfd[evlist->nr_fds].events =POLLIN |POLLERR |POLLHUP ; evlist->nr_fds++; }
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 4/10 perf record: Filter out POLLHUP'ed file descriptors
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
So that we don't continue polling on vanished file descriptors, i.e. file descriptors for events monitoring threads that exited.
I.e. the following 'perf record' command now exits as expected, instead of staying in an eternal loop:
$ sleep 5s & $ perf record -p `pidof sleep`
--- tools/perf/builtin-record.c | 3 +++ 1 file changed, 3 insertions(+)
tools/perf/builtin-record.c +3 -0
@@ -467,6 +467,9 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)if (err >0 || (err <0 && errno ==EINTR )) err =0 ; waking++; + +if (perf_evlist__filter_pollfd (rec->evlist,POLLERR |POLLHUP ) ==0 ) + done =1 ; }/*
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 5/10 perf trace: Filter out POLLHUP'ed file descriptors
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
So that we don't continue polling on vanished file descriptors, i.e. file descriptors for events monitoring threads that exited.
I.e. the following 'trace' command now exits as expected, instead of staying in an eternal loop:
$ sleep 5s &
$ trace -p `pidof sleep`--- tools/perf/builtin-trace.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tools/perf/builtin-trace.c +2 -1
@@ -2171,7 +2171,8 @@ next_event:if (trace->nr_events == before) {int timeout = done ?100 :-1 ; -if (poll (evlist->pollfd, evlist->nr_fds, timeout) >0 ) +if (poll (evlist->pollfd, evlist->nr_fds, timeout) >0 && +perf_evlist__filter_pollfd (evlist,POLLERR |POLLHUP ) >0 ) goto again; }else { goto again;
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 6/10 perf evlist: Allow growing pollfd on add method
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
This way we will be able to add more file descriptors to be polled, like stdin or some timer fd.
At this point we might as well yank the pollfd class from evlist so that it can be used in other places.
--- tools/perf/util/evlist.c | 40 +++++++++++++++++++++++++++++++++++----- tools/perf/util/evlist.h | 5 +++-- 2 files changed, 38 insertions(+), 7 deletions(-)
tools/perf/util/evlist.c +35 -5
@@ -402,7 +402,21 @@ int perf_evlist__enable_event_idx(struct perf_evlist *evlist,return perf_evlist__enable_event_thread (evlist, evsel, idx); } -static int perf_evlist__alloc_pollfd (struct perf_evlist *evlist) +static int perf_evlist__grow_pollfd (struct perf_evlist *evlist,int hint) +{ +int nr_fds_alloc = evlist->nr_fds_alloc + hint; +size_t size =sizeof (struct pollfd ) * nr_fds_alloc; +struct pollfd *pollfd =realloc (evlist->pollfd, size); + +if (pollfd ==NULL ) +return -ENOMEM ; + + evlist->nr_fds_alloc = nr_fds_alloc; + evlist->pollfd = pollfd; +return 0 ; +} + +int perf_evlist__alloc_pollfd (struct perf_evlist *evlist) {int nr_cpus =cpu_map__nr (evlist->cpus);int nr_threads =thread_map__nr (evlist->threads);
@@ -416,16 +430,28 @@ static int perf_evlist__alloc_pollfd(struct perf_evlist *evlist) nfds += nr_cpus * nr_threads; } - evlist->pollfd =malloc (sizeof (struct pollfd ) * nfds); -return evlist->pollfd !=NULL ?0 : -ENOMEM ; +if (evlist->nr_fds_alloc - evlist->nr_fds < nfds && +perf_evlist__grow_pollfd (evlist, nfds) <0 ) +return -ENOMEM ; + +return 0 ; } -void perf_evlist__add_pollfd (struct perf_evlist *evlist,int fd) +int perf_evlist__add_pollfd (struct perf_evlist *evlist,int fd) { +/* +* XXX: 64 is arbitrary, just not to call realloc at each fd. +* Find a better autogrowing heuristic +*/ +if (evlist->nr_fds == evlist->nr_fds_alloc && +perf_evlist__grow_pollfd (evlist,64 ) <0 ) +return -ENOMEM ; +fcntl (fd,F_SETFL ,O_NONBLOCK ); evlist->pollfd[evlist->nr_fds].fd = fd; evlist->pollfd[evlist->nr_fds].events =POLLIN |POLLERR |POLLHUP ; evlist->nr_fds++; +return 0 ; }int perf_evlist__filter_pollfd (struct perf_evlist *evlist,short revents_and_mask)
@@ -713,7 +739,11 @@ static int __perf_evlist__mmap(struct perf_evlist *evlist, int idx,return -1 ; } -perf_evlist__add_pollfd (evlist, fd); +if (perf_evlist__add_pollfd (evlist, fd) <0 ) { +__perf_evlist__munmap (evlist->mmap[idx].base, idx); +return -1 ; + } +return 0 ; }
tools/perf/util/evlist.h +3 -2
@@ -30,6 +30,7 @@ struct perf_evlist {int nr_entries;int nr_groups;int nr_fds; +int nr_fds_alloc;int nr_mmaps;size_t mmap_len;int id_pos;
@@ -82,8 +83,8 @@ perf_evlist__find_tracepoint_by_name(struct perf_evlist *evlist,void perf_evlist__id_add (struct perf_evlist *evlist,struct perf_evsel *evsel,int cpu,int thread,u64 id); -void perf_evlist__add_pollfd (struct perf_evlist *evlist,int fd); - +int perf_evlist__add_pollfd (struct perf_evlist *evlist,int fd); +int perf_evlist__alloc_pollfd (struct perf_evlist *evlist);int perf_evlist__filter_pollfd (struct perf_evlist *evlist,short revents_and_mask);struct perf_evsel *perf_evlist__id2evsel (struct perf_evlist *evlist,u64 id);
Signature · 1 line
1.9.3
Re: [PATCH 06/10] perf evlist: Allow growing pollfd on add method
On Fri, Aug 22, 2014 at 05:59:46PM -0300, Arnaldo Carvalho de Melo wrote:
SNIP
Quoted 20 lines
tools/perf/util/evlist.c +10
-1-void perf_evlist__add_pollfd (struct perf_evlist *evlist,int fd) +int perf_evlist__add_pollfd (struct perf_evlist *evlist,int fd) { +/* +* XXX: 64 is arbitrary, just not to call realloc at each fd. +* Find a better autogrowing heuristic +*/ +if (evlist->nr_fds == evlist->nr_fds_alloc && +perf_evlist__grow_pollfd (evlist,64 ) <0 ) +return -ENOMEM ; +fcntl (fd,F_SETFL ,O_NONBLOCK ); evlist->pollfd[evlist->nr_fds].fd = fd; evlist->pollfd[evlist->nr_fds].events =POLLIN |POLLERR |POLLHUP ; evlist->nr_fds++; +return 0 ; }int perf_evlist__filter_pollfd (struct perf_evlist *evlist,short revents_and_mask)
tools/perf/util/evlist.c static int __perf_evlist__mmap(struct perf_evlist *evlist, int idx, · line 743
Quoted 6 lines
tools/perf/util/evlist.c +2
-1@@ -713,7 +739,11 @@ static int __perf_evlist__mmap(struct perf_evlist *evlist, int idx,return -1 ; } -perf_evlist__add_pollfd (evlist, fd); +if (perf_evlist__add_pollfd (evlist, fd) <0 ) { +__perf_evlist__munmap (evlist->mmap[idx].base, idx);
unmap does not look necessary here.. should be handeled by out_unmap in both perf_evlist__mmap_per_thread and perf_evlist__mmap_per_cpu
jirka
Re: [PATCH 06/10] perf evlist: Allow growing pollfd on add method
In reply to Arnaldo Carvalho de MeloOn Fri, Aug 22, 2014 at 05:59:46PM -0300, Arnaldo Carvalho de Melo wrote:
SNIP
Quoted 21 lines
tools/perf/util/evlist.c +15
-1-static int perf_evlist__alloc_pollfd (struct perf_evlist *evlist) +static int perf_evlist__grow_pollfd (struct perf_evlist *evlist,int hint) +{ +int nr_fds_alloc = evlist->nr_fds_alloc + hint; +size_t size =sizeof (struct pollfd ) * nr_fds_alloc; +struct pollfd *pollfd =realloc (evlist->pollfd, size); + +if (pollfd ==NULL ) +return -ENOMEM ; + + evlist->nr_fds_alloc = nr_fds_alloc; + evlist->pollfd = pollfd; +return 0 ; +} + +int perf_evlist__alloc_pollfd (struct perf_evlist *evlist) {int nr_cpus =cpu_map__nr (evlist->cpus);int nr_threads =thread_map__nr (evlist->threads);
tools/perf/util/evlist.c static int perf_evlist__alloc_pollfd(struct perf_evlist *evlist) · line 435
Quoted 8 lines
tools/perf/util/evlist.c +3
-2@@ -416,16 +430,28 @@ static int perf_evlist__alloc_pollfd(struct perf_evlist *evlist) nfds += nr_cpus * nr_threads; } - evlist->pollfd =malloc (sizeof (struct pollfd ) * nfds); -return evlist->pollfd !=NULL ?0 : -ENOMEM ; +if (evlist->nr_fds_alloc - evlist->nr_fds < nfds && +perf_evlist__grow_pollfd (evlist, nfds) <0 ) +return -ENOMEM ;
hum, so do we still need perf_evlist__alloc_pollfd? we grow any time we need inside perf_evlist__add_pollfd..
jirka
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 06/10] perf evlist: Allow growing pollfd on add method
Em Mon, Aug 25, 2014 at 11:47:47AM +0200, Jiri Olsa escreveu:
Quoted 25 lines · Arnaldo Carvalho de Melo
Quoted 24 lines
tools/perf/util/evlist.c +15
-1-static int perf_evlist__alloc_pollfd (struct perf_evlist *evlist) +static int perf_evlist__grow_pollfd (struct perf_evlist *evlist,int hint) +{ +int nr_fds_alloc = evlist->nr_fds_alloc + hint; +size_t size =sizeof (struct pollfd ) * nr_fds_alloc; +struct pollfd *pollfd =realloc (evlist->pollfd, size); + +if (pollfd ==NULL ) +return -ENOMEM ; + + evlist->nr_fds_alloc = nr_fds_alloc; + evlist->pollfd = pollfd; +return 0 ; +} + +int perf_evlist__alloc_pollfd (struct perf_evlist *evlist) {int nr_cpus =cpu_map__nr (evlist->cpus);int nr_threads =thread_map__nr (evlist->threads);@@ -416,16 +430,28 @@ static int perf_evlist__alloc_pollfd(struct perf_evlist *evlist) nfds += nr_cpus * nr_threads; }
Quoted 8 lines
tools/perf/util/evlist.c static int perf_evlist__alloc_pollfd(struct perf_evlist *evlist) · line 435
Quoted 5 lines
tools/perf/util/evlist.c +3
-2- evlist->pollfd =malloc (sizeof (struct pollfd ) * nfds); -return evlist->pollfd !=NULL ?0 : -ENOMEM ; +if (evlist->nr_fds_alloc - evlist->nr_fds < nfds && +perf_evlist__grow_pollfd (evlist, nfds) <0 ) +return -ENOMEM ;hum, so do we still need
perf_evlist__alloc_pollfd? we grow any time we need insideperf_evlist__add_pollfd..
Humm, yeah, it could conceivably be removed and then we would rely always on add_pollfd() autogrowing it as needed. But then, I thought that code using this could optimize for the case where it knows in advance how many entries it will need, just like perf_evlist does.
I.e. the first thing builtin-kvm.c should do would be to grow it by two more entries, because it knows it wants to add just two more entries, in advance, etc.
- Arnaldo
Arnaldo Carvalho de Melo
maintainerPATCH 7/10 perf tests: Add pollfd growing test
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
[acme@ssdandy linux]$ perf test "Add fd"
34: Add fd to pollfd array, making it autogrow : Ok
[acme@ssdandy linux]$ perf test -v "Add fd"
34: Add fd to pollfd array, making it autogrow :
--- start ---
test child forked, pid 19817
before growing array: 2 [ 1, 2 ] after 3rd add_pollfd: 3 [ 1, 2, 35 ] after 4th add_pollfd: 4 [ 1, 2, 35, 88 ]
test child finished with 0
---- end ----
Add fd to pollfd array, making it autogrow: Ok [acme@ssdandy linux]$
--- tools/perf/tests/builtin-test.c | 4 ++ tools/perf/tests/evlist.c | 111 ++++++++++++++++++++++++++++++++++++++++ tools/perf/tests/tests.h | 1 + 3 files changed, 116 insertions(+)
tools/perf/tests/builtin-test.c +4 -0
@@ -162,6 +162,10 @@ static struct test { .func = test__perf_evlist__filter_pollfd, }, { + .desc = "Add fd to pollfd array, making it autogrow", + .func = test__perf_evlist__add_pollfd, + }, + { .func = NULL, }, };
tools/perf/tests/evlist.c +111 -0
119 lines not on this page · raw message
tools/perf/tests/tests.h +1 -0
@@ -50,6 +50,7 @@ int test__hists_output(void);int test__hists_cumulate (void );int test__switch_tracking (void );int test__perf_evlist__filter_pollfd (void ); +int test__perf_evlist__add_pollfd (void );#if defined(__x86_64__) || defined(__i386__) || defined(__arm__)#ifdef HAVE_DWARF_UNWIND_SUPPORT
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 8/10 perf kvm stat live: Use perf_evlist__add_pollfd() instead of local equivalent
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
Since we can add file descriptors to the evlist pollfd and it will autogrow, no need to copy all events to a local pollfd array, just add the timer and stdin file descriptors.
--- tools/perf/builtin-kvm.c | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-)
tools/perf/builtin-kvm.c +9 -13
@@ -921,15 +921,8 @@ static int kvm_events_live_report(struct perf_kvm_stat *kvm)signal (SIGINT , sig_handler);signal (SIGTERM , sig_handler); -/* copy pollfds -- need to add timerfd and stdin */ +/* use pollfds -- need to add timerfd and stdin */ nr_fds = kvm->evlist->nr_fds; - pollfds =zalloc (sizeof (struct pollfd ) * (nr_fds +2 )); -if (!pollfds) { - err = -ENOMEM ; - goto out; - } -memcpy (pollfds, kvm->evlist->pollfd, -sizeof (struct pollfd ) * kvm->evlist->nr_fds);/* add timer fd */ if (perf_kvm__timerfd_create (kvm) <0 ) {
@@ -937,17 +930,21 @@ static int kvm_events_live_report(struct perf_kvm_stat *kvm) goto out; } - pollfds[nr_fds].fd = kvm->timerfd; - pollfds[nr_fds].events =POLLIN ; +if (perf_evlist__add_pollfd (kvm->evlist, kvm->timerfd)) + goto out; + nr_fds++; - pollfds[nr_fds].fd =fileno (stdin); - pollfds[nr_fds].events =POLLIN ; +if (perf_evlist__add_pollfd (kvm->evlist,fileno (stdin))) + goto out; + nr_stdin = nr_fds; nr_fds++;if (fd_set_nonblock (fileno (stdin)) !=0 ) goto out; + pollfds = kvm->evlist->pollfd; +/* everything is good - enable the events and process */ perf_evlist__enable (kvm->evlist);
@@ -981,7 +978,6 @@ out:close (kvm->timerfd);tcsetattr (0 ,TCSAFLUSH , &save); -free (pollfds);return err; }
Signature · 1 line
1.9.3
Re: [PATCH 08/10] perf kvm stat live: Use perf_evlist__add_pollfd() instead of local equivalent
Quoted 71 lines · Arnaldo Carvalho de Melo
From: Arnaldo Carvalho de Melo <[email protected]>
Since we can add file descriptors to the evlist pollfd and it will autogrow, no need to copy all events to a local pollfd array, just add the timer and stdin file descriptors.
Cc Adrian Hunter <[email protected]>Cc Corey Ashford <[email protected]>Cc David Ahern <[email protected]>Cc Frederic Weisbecker <[email protected]>Cc Ingo Molnar <[email protected]>Cc Jean Pihet <[email protected]>Cc Jiri Olsa <[email protected]>Cc Namhyung Kim <[email protected]>Cc Paul Mackerras <[email protected]>Cc Peter Zijlstra <[email protected]>Signed-off-by Arnaldo Carvalho de Melo <[email protected]>--- tools/perf/builtin-kvm.c | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-)tools/perf/builtin-kvm.c +9
-12@@ -921,15 +921,8 @@ static int kvm_events_live_report(struct perf_kvm_stat *kvm)signal (SIGINT , sig_handler);signal (SIGTERM , sig_handler); -/* copy pollfds -- need to add timerfd and stdin */ +/* use pollfds -- need to add timerfd and stdin */ nr_fds = kvm->evlist->nr_fds; - pollfds =zalloc (sizeof (struct pollfd ) * (nr_fds +2 )); -if (!pollfds) { - err = -ENOMEM ; - goto out; - } -memcpy (pollfds, kvm->evlist->pollfd, -sizeof (struct pollfd ) * kvm->evlist->nr_fds);/* add timer fd */ if (perf_kvm__timerfd_create (kvm) <0 ) {@@ -937,17 +930,21 @@ static int kvm_events_live_report(struct perf_kvm_stat *kvm) goto out; } - pollfds[nr_fds].fd = kvm->timerfd; - pollfds[nr_fds].events =POLLIN ; +if (perf_evlist__add_pollfd (kvm->evlist, kvm->timerfd)) + goto out; + nr_fds++; - pollfds[nr_fds].fd =fileno (stdin); - pollfds[nr_fds].events =POLLIN ; +if (perf_evlist__add_pollfd (kvm->evlist,fileno (stdin))) + goto out; + nr_stdin = nr_fds; nr_fds++;if (fd_set_nonblock (fileno (stdin)) !=0 ) goto out; + pollfds = kvm->evlist->pollfd; +/* everything is good - enable the events and process */ perf_evlist__enable (kvm->evlist);
tools/perf/builtin-kvm.c out: · line 983
Quoted 7 lines
tools/perf/builtin-kvm.c +0
-1@@ -981,7 +978,6 @@ out:close (kvm->timerfd);tcsetattr (0 ,TCSAFLUSH , &save); -free (pollfds);return err; }
Much simpler. Looks ok to me.
David
Arnaldo Carvalho de Melo
maintainerPATCH 9/10 perf evlist: Introduce poll method for common code idiom
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
Since we have access two evlist members in all these poll calls, provide a helper.
This will also help to make the patch introducing the pollfd class more clear, as the evlist specific uses will be hiden away perf_evlist__poll().
--- tools/perf/builtin-record.c | 2 +- tools/perf/builtin-top.c | 4 ++-- tools/perf/builtin-trace.c | 2 +- tools/perf/tests/open-syscall-tp-fields.c | 2 +- tools/perf/tests/perf-record.c | 2 +- tools/perf/tests/task-exit.c | 2 +- tools/perf/util/evlist.c | 5 +++++ tools/perf/util/evlist.h | 2 ++ tools/perf/util/python.c | 2 +- 9 files changed, 15 insertions(+), 8 deletions(-)
tools/perf/builtin-record.c +1 -1
@@ -459,7 +459,7 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)if (hits == rec->samples) {if (done)break ; - err =poll (rec->evlist->pollfd, rec->evlist->nr_fds,-1 ); + err =perf_evlist__poll (rec->evlist,-1 );/* * Propagate error, only if there's any. Ignore positive * number of returned events and interrupt error.
tools/perf/builtin-top.c +2 -2
@@ -964,7 +964,7 @@ static int __cmd_top(struct perf_top *top)perf_evlist__enable (top->evlist);/* Wait for a minimal set of events before starting the snapshot */ -poll (top->evlist->pollfd, top->evlist->nr_fds,100 ); +perf_evlist__poll (top->evlist,100 );perf_top__mmap_read (top);
@@ -991,7 +991,7 @@ static int __cmd_top(struct perf_top *top)perf_top__mmap_read (top);if (hits == top->samples) - ret =poll (top->evlist->pollfd, top->evlist->nr_fds,100 ); + ret =perf_evlist__poll (top->evlist,100 ); } ret =0 ;
tools/perf/builtin-trace.c +1 -1
@@ -2171,7 +2171,7 @@ next_event:if (trace->nr_events == before) {int timeout = done ?100 :-1 ; -if (poll (evlist->pollfd, evlist->nr_fds, timeout) >0 && +if (perf_evlist__poll (evlist, timeout) >0 &&perf_evlist__filter_pollfd (evlist,POLLERR |POLLHUP ) >0 ) goto again; }else {
tools/perf/tests/open-syscall-tp-fields.c +1 -1
@@ -105,7 +105,7 @@ int test__syscall_open_tp_fields(void) }if (nr_events == before) -poll (evlist->pollfd, evlist->nr_fds,10 ); +perf_evlist__poll (evlist,10 );if (++nr_polls >5 ) {pr_debug ("%s: no events!\n" , __func__);
tools/perf/tests/perf-record.c +1 -1
@@ -268,7 +268,7 @@ int test__PERF_RECORD(void)* perf_event_attr.wakeup_events, just PERF_EVENT_SAMPLE does. */ if (total_events == before && false) -poll (evlist->pollfd, evlist->nr_fds,-1 ); +perf_evlist__poll (evlist,-1 );sleep (1 );if (++wakeups >5 ) {
tools/perf/tests/task-exit.c +1 -1
@@ -105,7 +105,7 @@ retry: }if (!exited || !nr_exit) { -poll (evlist->pollfd, evlist->nr_fds,-1 ); +perf_evlist__poll (evlist,-1 ); goto retry; }
tools/perf/util/evlist.c +5 -0
@@ -470,6 +470,11 @@ int perf_evlist__filter_pollfd(struct perf_evlist *evlist, short revents_and_masreturn nr_fds; } +int perf_evlist__poll (struct perf_evlist *evlist,int timeout) +{ +return poll (evlist->pollfd, evlist->nr_fds, timeout); +} +static void perf_evlist__id_hash (struct perf_evlist *evlist,struct perf_evsel *evsel,int cpu,int thread,u64 id)
tools/perf/util/evlist.h +2 -0
@@ -87,6 +87,8 @@ int perf_evlist__add_pollfd(struct perf_evlist *evlist, int fd);int perf_evlist__alloc_pollfd (struct perf_evlist *evlist);int perf_evlist__filter_pollfd (struct perf_evlist *evlist,short revents_and_mask); +int perf_evlist__poll (struct perf_evlist *evlist,int timeout); +struct perf_evsel *perf_evlist__id2evsel (struct perf_evlist *evlist,u64 id);struct perf_sample_id *perf_evlist__id2sid (struct perf_evlist *evlist,u64 id);
tools/perf/util/python.c +1 -1
@@ -736,7 +736,7 @@ static PyObject *pyrf_evlist__poll(struct pyrf_evlist *pevlist,if (!PyArg_ParseTupleAndKeywords (args, kwargs,"|i" , kwlist, &timeout))return NULL ; - n =poll (evlist->pollfd, evlist->nr_fds, timeout); + n =perf_evlist__poll (evlist, timeout);if (n <0 ) {PyErr_SetFromErrno (PyExc_OSError);return NULL ;
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 10/10 tools lib api: Adopt fdarray class from perf's evlist
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
The extensible file description array that grew in the perf_evlist class can be useful for other tools, as it is not something that only evlists need, so move it to tools/lib/api/fd to ease sharing it.
--- tools/lib/api/Makefile | 7 +++- tools/lib/api/fd/poll.c | 105 ++++++++++++++++++++++++++++++++++++++++++++++ tools/lib/api/fd/poll.h | 31 ++++++++++++++ tools/perf/builtin-kvm.c | 4 +- tools/perf/tests/evlist.c | 67 +++++++++++++++-------------- tools/perf/util/evlist.c | 52 ++++------------------- tools/perf/util/evlist.h | 5 +-- tools/perf/util/python.c | 4 +- 8 files changed, 191 insertions(+), 84 deletions(-) create mode 100644 tools/lib/api/fd/poll.c create mode 100644 tools/lib/api/fd/poll.h
tools/lib/api/Makefile +6 -1
@@ -10,9 +10,14 @@ LIB_OBJS= LIB_H += fs/debugfs.h LIB_H += fs/fs.h +# See comment below about piggybacking... +LIB_H += fd/poll.h LIB_OBJS += $(OUTPUT)fs/debugfs.o LIB_OBJS += $(OUTPUT)fs/fs.o +# XXX piggybacking here, need to introduce libapikfd, or rename this +# to plain libapik.a and make it have it all api goodies +LIB_OBJS += $(OUTPUT)fd/poll.o LIBFILE = libapikfs.a
@@ -29,7 +34,7 @@ $(LIBFILE): $(LIB_OBJS) $(LIB_OBJS): $(LIB_H) libapi_dirs: - $(QUIET_MKDIR)mkdir -p $(OUTPUT)fs/ + $(QUIET_MKDIR)mkdir -p $(OUTPUT){fs,fd}/ $(OUTPUT)%.o: %.c libapi_dirs $(QUIET_CC)$(CC) -o $@ -c $(ALL_CFLAGS) $<
tools/lib/api/fd/poll.c new file +105 -0
105 lines not on this page · raw message
tools/lib/api/fd/poll.h new file +31 -0
@@ -0,0 +1,31 @@ +#ifndef __API_FD_POLL__ +#define __API_FD_POLL__ + +#include <stdio.h> + +struct pollfd ; + +struct fdarray { +int nr; +int nr_alloc; +struct pollfd *entries; +}; + +void fdarray__init (struct fdarray *fda); +void fdarray__exit (struct fdarray *fda); + +struct fdarray *fdarray__new (int nr_alloc); +void fdarray__delete (struct fdarray *fda); + +int fdarray__add (struct fdarray *fda,int fd); +int fdarray__poll (struct fdarray *fda,int timeout); +int fdarray__filter (struct fdarray *fda,short revents_and_mask); +int fdarray__grow (struct fdarray *fda,int extra); +int fdarray__fprintf (struct fdarray *fda,FILE *fp); + +static inline int fdarray__available_entries (struct fdarray *fda) +{ +return fda->nr_alloc - fda->nr; +} + +#endif /* __API_FD_POLL__ */
tools/perf/builtin-kvm.c +2 -2
@@ -922,7 +922,7 @@ static int kvm_events_live_report(struct perf_kvm_stat *kvm)signal (SIGTERM , sig_handler);/* use pollfds -- need to add timerfd and stdin */ - nr_fds = kvm->evlist->nr_fds; + nr_fds = kvm->evlist->pollfd.nr;/* add timer fd */ if (perf_kvm__timerfd_create (kvm) <0 ) {
@@ -943,7 +943,7 @@ static int kvm_events_live_report(struct perf_kvm_stat *kvm)if (fd_set_nonblock (fileno (stdin)) !=0 ) goto out; - pollfds = kvm->evlist->pollfd; + pollfds = kvm->evlist->pollfd.entries;/* everything is good - enable the events and process */ perf_evlist__enable (kvm->evlist);
tools/perf/tests/evlist.c +35 -32
154 lines not on this page · raw message
tools/perf/util/evlist.c +8 -44
96 lines not on this page · raw message
tools/perf/util/evlist.h +2 -3
@@ -2,6 +2,7 @@#define __PERF_EVLIST_H 1#include <linux/list.h> +#include <api/fd/poll.h> #include <stdio.h> #include "../perf.h" #include "event.h"
@@ -29,8 +30,6 @@ struct perf_evlist {struct hlist_head heads[PERF_EVLIST__HLIST_SIZE ];int nr_entries;int nr_groups; -int nr_fds; -int nr_fds_alloc;int nr_mmaps;size_t mmap_len;int id_pos;
@@ -41,8 +40,8 @@ struct perf_evlist {pid_t pid; } workload;bool overwrite; +struct fdarray pollfd;struct perf_mmap *mmap; -struct pollfd *pollfd;struct thread_map *threads;struct cpu_map *cpus;struct perf_evsel *selected;
tools/perf/util/python.c +2 -2
@@ -753,9 +753,9 @@ static PyObject *pyrf_evlist__get_pollfd(struct pyrf_evlist *pevlist,PyObject *list =PyList_New (0 );int i; -for (i =0 ; i < evlist->nr_fds; ++i) { +for (i =0 ; i < evlist->pollfd.nr; ++i) {PyObject *file; -FILE *fp =fdopen (evlist->pollfd[i].fd,"r" ); +FILE *fp =fdopen (evlist->pollfd.entries[i].fd,"r" );if (fp ==NULL ) goto free_list;
Signature · 1 line
1.9.3
Re: [PATCH 10/10] tools lib api: Adopt fdarray class from perf's evlist
Hi Arnaldo,
Quoted 6 lines · Arnaldo Carvalho de Melo
tools/lib/api/fd/poll.c new file +6
-0+int fdarray__add (struct fdarray *fda,int fd) +{ +/* +* XXX: 64 is arbitrary, just not to call realloc at each fd. +* Find a better autogrowing heuristic +*/
tools/lib/api/fd/poll.c line 74
Quoted 10 lines
tools/lib/api/fd/poll.c new file +10
-0+if (fda->nr == fda->nr_alloc && +fdarray__grow (fda,64 ) <0 ) +return -ENOMEM ; + +fcntl (fd,F_SETFL ,O_NONBLOCK ); + fda->entries[fda->nr].fd = fd; + fda->entries[fda->nr].events =POLLIN |POLLERR |POLLHUP ; + fda->nr++; +return 0 ; +}
To be more generic api, I think it'd be better receiving events from user rather than hard-coding. Also it might be useful to let user sets a grow hint (during init?) as well.
Thanks,
Namhyung
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 10/10] tools lib api: Adopt fdarray class from perf's evlist
Em Tue, Aug 26, 2014 at 04:46:18PM +0900, Namhyung Kim escreveu:
Quoted 14 lines
Hi Arnaldo,
Quoted 11 lines · Arnaldo Carvalho de Melo
tools/lib/api/fd/poll.c new file +11
-0+int fdarray__add (struct fdarray *fda,int fd) +{ +/* +* XXX: 64 is arbitrary, just not to call realloc at each fd. +* Find a better autogrowing heuristic +*/ +if (fda->nr == fda->nr_alloc && +fdarray__grow (fda,64 ) <0 ) +return -ENOMEM ; + +fcntl (fd,F_SETFL ,O_NONBLOCK );
Quoted 8 lines
tools/lib/api/fd/poll.c line 74
Quoted 5 lines
tools/lib/api/fd/poll.c new file +5
-0+ fda->entries[fda->nr].fd = fd; + fda->entries[fda->nr].events =POLLIN |POLLERR |POLLHUP ; + fda->nr++; +return 0 ; +}To be more generic api, I think it'd be better receiving events from user rather than hard-coding. Also it might be useful to let user
Agreed, will do.
Quoted 1 line
sets a grow hint (during init?) as well.
Sure, that can be done as well, and with that, ditch that hard coded value.
Thanks,
- Arnaldo
Quoted 30 lines · Arnaldo Carvalho de Melo
Hi,
This is an alternative series to the one Jiri Olsa posted to use the
fixes he made to the kernel side to allow tooling to notice that a thread had exited by looking at the pollfd.revents looking for POLLHUP notifications.Once all event file descriptors are removed from the evlist pollfd array,
tools can make a decision about exiting or telling the user about what happened, asking to guidance on what to do next.The main difference in this approach is that a new class, which Jiri
called 'poller' and I called 'fdarray', grew up from what was inevlist->pollfdand associated operations, while Jiri first introduced a new class and then made tooling use it.The details of the implementation should be clear on the changelog
comments, please let me know if you see any problems, and if I can have your acked-by/tested-by/whatever-else tags to get this moving forward.Ah, there is still one missing thing which is to make the hists browser
in live mode be notified that all monitored events are POLLHUP'ed, will get to that in followup patches.The kernel bits were sent together with my latest pull req to Ingo,
this series is on top of that branch and is available at:
git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.gitperf/pollfd- Arnaldo
Quoted 11 lines
Arnaldo Carvalho de Melo (10):
perf evlist: Introduceperf_evlist__filter_pollfdmethod
perf tests: Add test forperf_evlist__filter_pollfd()
perf evlist: Monitor POLLERR and POLLHUP events too
perf record: Filter out POLLHUP'ed file descriptors
perf trace: Filter out POLLHUP'ed file descriptors
perf evlist: Allow growing pollfd on add method
perf tests: Add pollfd growing test
perf kvm stat live: Useperf_evlist__add_pollfd()instead of local equivalent perf evlist: Introduce poll method for common code idiom
tools lib api: Adopt fdarray class from perf's evlist
fdarray name seems too generic for this object, fdpoll looks more suitable.. also given that the file name is poll.[ch] ;-)
anyway, except for what I've already commented:
jirka
Arnaldo Carvalho de Melo
maintainerEm Mon, Aug 25, 2014 at 01:04:31PM +0200, Jiri Olsa escreveu:
Quoted 2 lines
fdarray name seems too generic for this object, fdpoll looks more suitable.. also given that the file name is poll.[ch] ;-)
I thought about 'struct pollfds', then thought that having 'poll' in the name of the class and in its main routine, i.e. pollfds__poll, would be too much.
I thought of it as an extensible array of file descriptors that right now has just poll related operations, but could have some other operations, perhaps the associated mmaps in evlist, for instance.
I ended up using poll.[ch] because struct pollfd came from poll.h, but yeah, that is inconsistent with poll being just one of the operations for an extensible array of file descriptors, so it should be renamed accordingly to 'fdarray.[ch]'.
That was my reasoning.
I'm looking at the other comments, would send a fix for what Adrian commented but thought about an optimization, will do that after lunch, hopefully food will prevent me from adding more bugs 8-)
Quoted 3 lines
anyway, except for what I've already commented:
Acked-by Jiri Olsa <[email protected]>
Thanks!
- Arnaldo