perf pollfd series v2
Arnaldo Carvalho de Melo
maintainercover perf pollfd series v2
Hi,
This is an alternative series to the one Jiri Olsa posted to use the
fixes he made to the kernel side to allow tooling to notice that a thread had exited by looking at the pollfd.revents looking for POLLHUP notifications.
Once all event file descriptors are removed from the evlist pollfd array,
tools can make a decision about exiting or telling the user about what happened, asking to guidance on what to do next.
The main difference in this approach is that a new class, which Jiri
called 'poller' and I called 'fdarray', grew up from what was in evlist->pollfd and associated operations, while Jiri first introduced a new class and then made tooling use it.
The details of the implementation should be clear on the changelog
comments, please let me know if you see any problems, and if I can have your acked-by/tested-by/whatever-else tags to get this moving forward.
Ah, there is still one missing thing which is to make the hists browser
in live mode be notified that all monitored events are POLLHUP'ed, will get to that in followup patches.
The kernel bits were sent together with my latest pull req to Ingo,
this series is on top of that branch and is available at:
git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git perf/fdarray
--------------------------------------------------------------------
v2: The kernel bits were already merged by Ingo, this is should address the comments made for the first RFC, and has a hopefully improved/clearer fdarray__filter() method, with changes made from thinking about comments made by Adrian.
I made fdarray__init() receive the fdarray__add() autogrow hint, as suggested by Namhyung.
It also makes fdarray__add() receive the revents mask instead of using what makes sense to perf_evlist__add_pollfd(), also suggested by Namhyung.
I renamed the fd/poll.[ch] files to fd/array.[ch], from the discussion had about naming with Jiri.
I added some Acked-by and Reviewed-by tags, please let me know if i should stick some more, I really appreciate those tags!
Ah, in this branch there are some more patches, as it was made on top of my perf/core branch, below are the stats for just the changesets related to this RFC.
Thanks!
- Arnaldo
Arnaldo Carvalho de Melo (10):
perf evlist: Introduce perf_evlist__filter_pollfd method perf tests: Add test for perf_evlist__filter_pollfd() perf evlist: Monitor POLLERR and POLLHUP events too perf record: Filter out POLLHUP'ed file descriptors perf trace: Filter out POLLHUP'ed file descriptors
perf evlist: Allow growing pollfd on add method
perf tests: Add pollfd growing test
perf kvm stat live: Use perf_evlist__add_pollfd() instead of local
equivalent
perf evlist: Introduce poll method for common code idiom tools lib api: Adopt fdarray class from perf's evlist
tools/lib/api/Makefile | 7 +- tools/lib/api/fd/array.c | 108 +++++++++++++++ tools/lib/api/fd/array.h | 32 +++++ tools/perf/Makefile.perf | 1 + tools/perf/builtin-kvm.c | 24 ++-- tools/perf/builtin-record.c | 5 +- tools/perf/builtin-top.c | 4 +- tools/perf/builtin-trace.c | 3 +- tools/perf/tests/builtin-test.c | 8 ++ tools/perf/tests/evlist.c | 217 ++++++++++++++++++++++++++++++ tools/perf/tests/open-syscall-tp-fields.c | 2 +- tools/perf/tests/perf-record.c | 2 +- tools/perf/tests/task-exit.c | 2 +- tools/perf/tests/tests.h | 2 + tools/perf/util/evlist.c | 37 +++-- tools/perf/util/evlist.h | 10 +- tools/perf/util/python.c | 6 +- 17 files changed, 431 insertions(+), 39 deletions(-) create mode 100644 tools/lib/api/fd/array.c create mode 100644 tools/lib/api/fd/array.h create mode 100644 tools/perf/tests/evlist.c
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 1/10 perf evlist: Introduce perf_evlist__filter_pollfd method
From: Arnaldo Carvalho de Melo <[email protected]>
To remove all entries in evlist->pollfd[] that have revents matching at least one of the bits in the specified mask.
It'll adjust evlist->nr_fds to the number of unfiltered fds and will return this value, as a convenience and to avoid requiring direct access to internal state of perf_evlist objects.
This will be used after polling the evlist fds so that we remove fds that were closed by the kernel.
--- tools/perf/util/evlist.c | 21 +++++++++++++++++++++ tools/perf/util/evlist.h | 2 ++ 2 files changed, 23 insertions(+)
tools/perf/util/evlist.c +21 -0
@@ -428,6 +428,27 @@ void perf_evlist__add_pollfd(struct perf_evlist *evlist, int fd) evlist->nr_fds++; } +int perf_evlist__filter_pollfd (struct perf_evlist *evlist,short revents_and_mask) +{ +int fd, nr_fds =0 ; + +if (evlist->nr_fds ==0 ) +return 0 ; + +for (fd =0 ; fd < evlist->nr_fds; ++fd) { +if (evlist->pollfd[fd].revents & revents_and_mask) +continue ; + +if (fd != nr_fds) + evlist->pollfd[nr_fds] = evlist->pollfd[fd]; + + ++nr_fds; + } + + evlist->nr_fds = nr_fds; +return nr_fds; +} +static void perf_evlist__id_hash (struct perf_evlist *evlist,struct perf_evsel *evsel,int cpu,int thread,u64 id)
tools/perf/util/evlist.h +2 -0
@@ -84,6 +84,8 @@ void perf_evlist__id_add(struct perf_evlist *evlist, struct perf_evsel *evsel,void perf_evlist__add_pollfd (struct perf_evlist *evlist,int fd); +int perf_evlist__filter_pollfd (struct perf_evlist *evlist,short revents_and_mask); +struct perf_evsel *perf_evlist__id2evsel (struct perf_evlist *evlist,u64 id);struct perf_sample_id *perf_evlist__id2sid (struct perf_evlist *evlist,u64 id);
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 2/10 perf tests: Add test for perf_evlist__filter_pollfd()
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
That will use a synthetic evlist with just what is touched by this new method to check that it works as expected.
Output in verbose mode:
$ perf test -v pollfd 33: Filter fds with revents mask in a pollfd array : --- start --- filtering all but pollfd[2]: before: 5 [ 5, 4, 3, 2, 1 ] after: 1 [ 3 ] filtering all but (pollfd[0], pollfd[3]): before: 5 [ 5, 4, 3, 2, 1 ] after: 2 [ 5, 2 ] test child finished with 0 ---- end ---- Filter fds with revents mask in a pollfd array: Ok $
--- tools/perf/Makefile.perf | 1 + tools/perf/tests/builtin-test.c | 4 ++ tools/perf/tests/evlist.c | 103 ++++++++++++++++++++++++++++++++++++++++ tools/perf/tests/tests.h | 1 + 4 files changed, 109 insertions(+) create mode 100644 tools/perf/tests/evlist.c
tools/perf/Makefile.perf +1 -0
@@ -399,6 +399,7 @@ LIB_OBJS += $(OUTPUT)tests/open-syscall-tp-fields.o LIB_OBJS += $(OUTPUT)tests/mmap-basic.o LIB_OBJS += $(OUTPUT)tests/perf-record.o LIB_OBJS += $(OUTPUT)tests/rdpmc.o +LIB_OBJS += $(OUTPUT)tests/evlist.o LIB_OBJS += $(OUTPUT)tests/evsel-roundtrip-name.o LIB_OBJS += $(OUTPUT)tests/evsel-tp-sched.o LIB_OBJS += $(OUTPUT)tests/pmu.o
tools/perf/tests/builtin-test.c +4 -0
@@ -158,6 +158,10 @@ static struct test { .func = test__switch_tracking, }, { + .desc = "Filter fds with revents mask in a pollfd array", + .func = test__perf_evlist__filter_pollfd, + }, + { .func = NULL, }, };
tools/perf/tests/evlist.c new file +103 -0
103 lines not on this page · raw message
tools/perf/tests/tests.h +1 -0
@@ -49,6 +49,7 @@ int test__thread_mg_share(void);int test__hists_output (void );int test__hists_cumulate (void );int test__switch_tracking (void ); +int test__perf_evlist__filter_pollfd (void );#if defined(__x86_64__) || defined(__i386__) || defined(__arm__)#ifdef HAVE_DWARF_UNWIND_SUPPORT
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 3/10 perf evlist: Monitor POLLERR and POLLHUP events too
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
We want to know when the fd went away, like when a monitored thread exits.
If we do not monitor such events, then the tools will wait forever on events from a vanished thread, like when running:
$ sleep 5s & $ perf record -p `pidof sleep`
This builds upon the kernel patch by Jiri Olsa that actually makes a poll on those file descriptors to return POLLHUP.
It is also needed to change the tools to use perf_evlist__filter_pollfd() to check if there are remainings fds to monitor or if all are gone, in which case they will exit the poll/mmap/read loop.
--- tools/perf/util/evlist.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/evlist.c +1 -1
@@ -424,7 +424,7 @@ void perf_evlist__add_pollfd(struct perf_evlist *evlist, int fd) {fcntl (fd,F_SETFL ,O_NONBLOCK ); evlist->pollfd[evlist->nr_fds].fd = fd; - evlist->pollfd[evlist->nr_fds].events =POLLIN ; + evlist->pollfd[evlist->nr_fds].events =POLLIN |POLLERR |POLLHUP ; evlist->nr_fds++; }
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 4/10 perf record: Filter out POLLHUP'ed file descriptors
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
So that we don't continue polling on vanished file descriptors, i.e. file descriptors for events monitoring threads that exited.
I.e. the following 'perf record' command now exits as expected, instead of staying in an eternal loop:
$ sleep 5s & $ perf record -p `pidof sleep`
--- tools/perf/builtin-record.c | 3 +++ 1 file changed, 3 insertions(+)
tools/perf/builtin-record.c +3 -0
@@ -467,6 +467,9 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)if (err >0 || (err <0 && errno ==EINTR )) err =0 ; waking++; + +if (perf_evlist__filter_pollfd (rec->evlist,POLLERR |POLLHUP ) ==0 ) + done =1 ; }/*
Signature · 1 line
1.9.3
Adrian Hunter
reviewerRe: [PATCH 04/10] perf record: Filter out POLLHUP'ed file descriptors
Quoted 34 lines · Arnaldo Carvalho de Melo
From: Arnaldo Carvalho de Melo <[email protected]>
So that we don't continue polling on vanished file descriptors, i.e. file descriptors for events monitoring threads that exited.
I.e. the following 'perf record' command now exits as expected, instead of staying in an eternal loop:
$ sleep 5s & $ perf record -p `pidof sleep`Reported-by Jiri Olsa <[email protected]>Acked-by Jiri Olsa <[email protected]>Cc Adrian Hunter <[email protected]>Cc David Ahern <[email protected]>Cc Don Zickus <[email protected]>Cc Frederic Weisbecker <[email protected]>Cc Jiri Olsa <[email protected]>Cc Mike Galbraith <[email protected]>Cc Namhyung Kim <[email protected]>Cc Paul Mackerras <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Stephane Eranian <[email protected]>Signed-off-by Arnaldo Carvalho de Melo <[email protected]>--- tools/perf/builtin-record.c | 3 +++ 1 file changed, 3 insertions(+)tools/perf/builtin-record.c +0
-0
tools/perf/builtin-record.c static int __cmd_record(struct record *rec, int argc, const char **argv) · line 471
Quoted 5 lines
tools/perf/builtin-record.c +2
-0@@ -467,6 +467,9 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)if (err >0 || (err <0 && errno ==EINTR )) err =0 ; waking++; + +if (perf_evlist__filter_pollfd (rec->evlist,POLLERR |POLLHUP ) ==0 )
If the poll fds only include the ones mmapped, then mightn't it filter out ones still in use? e.g. what if you record two processes and one exits?
Quoted 5 lines
tools/perf/builtin-record.c +1
-0+ done =1 ; }/*
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 04/10] perf record: Filter out POLLHUP'ed file descriptors
Em Thu, Sep 04, 2014 at 03:32:08PM +0300, Adrian Hunter escreveu:
Quoted 11 lines · Arnaldo Carvalho de Melo
Quoted 10 lines
So that we don't continue polling on vanished file descriptors, i.e. file descriptors for events monitoring threads that exited.
I.e. the following 'perf record' command now exits as expected, instead of staying in an eternal loop:
$ sleep 5s & $ perf record -p `pidof sleep`Reported-by Jiri Olsa <[email protected]>
<SNIP>
Quoted 9 lines
tools/perf/builtin-record.c static int __cmd_record(struct record *rec, int argc, const char **argv) · line 470
Quoted 6 lines
+++
b/tools/perf/builtin-record.ctools/perf/builtin-record.c +1
-0@@ -467,6 +467,9 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)if (err >0 || (err <0 && errno ==EINTR )) err =0 ; waking++; +if (perf_evlist__filter_pollfd (rec->evlist,POLLERR |POLLHUP ) ==0 )If the poll fds only include the ones mmapped, then mightn't it filter out ones still in use? e.g. what if you record two processes and one exits?
Humm, without looking at the code, we would be needlessly looking at the mmaps for the threads that exited.
I.e. we need to associate at the fdarray level the mmaps for the descriptors, so that we can, at fdarray__filter() time, munmap the POLLHUPed descriptors, right?
I'll take a look at the code to see what is needed.
But then this makes the patch set good for a class of problems while maintaining existing behaviour for the case you outlined, i.e. this is a partial fix and should go now, with further patches on top of it?
- Arnaldo
Adrian Hunter
reviewerRe: [PATCH 04/10] perf record: Filter out POLLHUP'ed file descriptors
Quoted 30 lines · Arnaldo Carvalho de Melo
Em Thu, Sep 04, 2014 at 03:32:08PM +0300, Adrian Hunter escreveu:
Quoted 11 lines · Arnaldo Carvalho de Melo
Quoted 10 lines
So that we don't continue polling on vanished file descriptors, i.e. file descriptors for events monitoring threads that exited.
I.e. the following 'perf record' command now exits as expected, instead of staying in an eternal loop:
$ sleep 5s & $ perf record -p `pidof sleep`Reported-by Jiri Olsa <[email protected]><SNIP>
Quoted 9 lines
tools/perf/builtin-record.c static int __cmd_record(struct record *rec, int argc, const char **argv) · line 470
Quoted 6 lines
+++
b/tools/perf/builtin-record.ctools/perf/builtin-record.c +1
-0@@ -467,6 +467,9 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)if (err >0 || (err <0 && errno ==EINTR )) err =0 ; waking++; +if (perf_evlist__filter_pollfd (rec->evlist,POLLERR |POLLHUP ) ==0 )If the poll fds only include the ones mmapped, then mightn't it filter out ones still in use? e.g. what if you record two processes and one exits?
Humm, without looking at the code, we would be needlessly looking at the mmaps for the threads that exited.
I.e. we need to associate at the fdarray level the mmaps for the descriptors, so that we can, at
fdarray__filter()time, munmap the POLLHUPed descriptors, right?
Quoted 5 lines
I'll take a look at the code to see what is needed.
But then this makes the patch set good for a class of problems while maintaining existing behaviour for the case you outlined, i.e. this is a partial fix and should go now, with further patches on top of it?
No I was meaning something different. For example, 'perf record' opens an event for 2 processes per-cpu and gets 4 file descriptors:
task1 task2
cpu0 fd0 fd1
cpu1 fd2 fd3
Now, perf record will mmap fd0 and fd2 and set-output fd1->fd0 and fd3->fd2.
pollfds includes only fd0 and fd2.
But if task2 exits, the POLLHUP will appear on fd1 and fd3.
I think Jiri's patchset changed pollfds to include all fds for that reason.
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 04/10] perf record: Filter out POLLHUP'ed file descriptors
Em Fri, Sep 05, 2014 at 11:42:59AM +0300, Adrian Hunter escreveu:
Quoted 9 lines · Arnaldo Carvalho de Melo
Quoted 1 line
Em Thu, Sep 04, 2014 at 03:32:08PM +0300, Adrian Hunter escreveu:
No I was meaning something different. For example, 'perf record' opens an event for 2 processes per-cpu and gets 4 file descriptors:
task1 task2cpu0 fd0 fd1
cpu1 fd2 fd3
Quoted 6 lines
Now, perf record will mmap fd0 and fd2 and set-output
fd1->fd0andfd3->fd2.pollfds includes only fd0 and fd2.
But if task2 exits, the POLLHUP will appear on fd1 and fd3.
So? We are not interested in fd1 and fd3, since all our reading is done on fd0 and fd2 mmaps, no?
I.e. when we ask the kernel to point fd B to fd A's mmap (what you called set-output) and fd B inserts an event into fd A's mmap ring buffer, we get fd A poll return as POLLRD, no?
Have to check... Otherwise we would have to poll all fds all the time, not just the ones mmaping, right?
Quoted 1 line
I think Jiri's patchset changed pollfds to include all fds for that reason.
It did? I have to look again, probably went together with other changes, has it?
- Arnaldo
Re: [PATCH 04/10] perf record: Filter out POLLHUP'ed file descriptors
Quoted 13 lines · Arnaldo Carvalho de Melo
Em Fri, Sep 05, 2014 at 11:42:59AM +0300, Adrian Hunter escreveu:
Quoted 11 lines · Arnaldo Carvalho de Melo
Quoted 1 line
Em Thu, Sep 04, 2014 at 03:32:08PM +0300, Adrian Hunter escreveu:
No I was meaning something different. For example, 'perf record' opens an event for 2 processes per-cpu and gets 4 file descriptors:
task1 task2cpu0 fd0 fd1
cpu1 fd2 fd3Now, perf record will mmap fd0 and fd2 and set-output
fd1->fd0andfd3->fd2.
Quoted 6 lines
Quoted 3 lines
pollfds includes only fd0 and fd2.
But if task2 exits, the POLLHUP will appear on fd1 and fd3.
So? We are not interested in fd1 and fd3, since all our reading is done on fd0 and fd2 mmaps, no?
hm, what if task1 (fd0, fd2) exits first.. perf record will exit, but it still has to read task2..?
Quoted 9 lines
I.e. when we ask the kernel to point fd B to fd A's mmap (what you called set-output) and fd B inserts an event into fd A's mmap ring buffer, we get fd A poll return as POLLRD, no?
Have to check... Otherwise we would have to poll all fds all the time, not just the ones mmaping, right?
Quoted 1 line
I think Jiri's patchset changed pollfds to include all fds for that reason.
hm, I did not think of that.. ;-) I needed more grained feedback for future features like cpu hotplug
Quoted 3 lines
It did? I have to look again, probably went together with other changes, has it?
it was done by replacing 'int' with 'struct poll_item' for evsel::fd xyarray
jirka
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 04/10] perf record: Filter out POLLHUP'ed file descriptors
Em Sat, Sep 06, 2014 at 10:39:15PM +0200, Jiri Olsa escreveu:
Quoted 16 lines · Arnaldo Carvalho de Melo
Quoted 14 lines
Em Fri, Sep 05, 2014 at 11:42:59AM +0300, Adrian Hunter escreveu:
Quoted 13 lines · Arnaldo Carvalho de Melo
Quoted 1 line
Em Thu, Sep 04, 2014 at 03:32:08PM +0300, Adrian Hunter escreveu:
No I was meaning something different. For example, 'perf record' opens an event for 2 processes per-cpu and gets 4 file descriptors:
task1 task2cpu0 fd0 fd1
cpu1 fd2 fd3Now, perf record will mmap fd0 and fd2 and set-output
fd1->fd0andfd3->fd2.pollfds includes only fd0 and fd2.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
But if task2 exits, the POLLHUP will appear on fd1 and fd3.
So? We are not interested in fd1 and fd3, since all our reading is done on fd0 and fd2 mmaps, no?
hm, what if task1 (fd0, fd2) exits first.. perf record will exit, but it still has to read task2..?
Ok, what happens in that case, i.e. when the fds that were set to be the ones to be polled, gets nuked, does the set-output command gets just undone? Or does the mmap stands, receiving the events from the remaining fds and the polling notifications get sent to, in this case, fd3 and fd1?
I'll look at the kernel code for that...
Quoted 11 lines
Quoted 8 lines
I.e. when we ask the kernel to point fd B to fd A's mmap (what you called set-output) and fd B inserts an event into fd A's mmap ring buffer, we get fd A poll return as POLLRD, no?
Have to check... Otherwise we would have to poll all fds all the time, not just the ones mmaping, right?
Quoted 1 line
I think Jiri's patchset changed pollfds to include all fds for that reason.
hm, I did not think of that.. ;-) I needed more grained feedback for future features like cpu hotplug
So this is good for something you didn't tried to fix (and document) but good for something that may be nice in the future? Grumpf, we have already way too much stuff that will be eventually used but is not used right now :-\
Quoted 4 lines
Quoted 2 lines
It did? I have to look again, probably went together with other changes, has it?
it was done by replacing 'int' with 'struct
poll_item' forevsel::fdxyarray
Well, will look into that as soon as I see the need for it. I.e. after I clarify the discussion above.
- Arnaldo
Re: [PATCH 04/10] perf record: Filter out POLLHUP'ed file descriptors
Quoted 22 lines · Arnaldo Carvalho de Melo
Em Sat, Sep 06, 2014 at 10:39:15PM +0200, Jiri Olsa escreveu:
Quoted 20 lines · Arnaldo Carvalho de Melo
Quoted 19 lines
Em Fri, Sep 05, 2014 at 11:42:59AM +0300, Adrian Hunter escreveu:
Quoted 9 lines · Arnaldo Carvalho de Melo
Quoted 1 line
Em Thu, Sep 04, 2014 at 03:32:08PM +0300, Adrian Hunter escreveu:
No I was meaning something different. For example, 'perf record' opens an event for 2 processes per-cpu and gets 4 file descriptors:
task1 task2cpu0 fd0 fd1
cpu1 fd2 fd3Quoted 6 lines
Now, perf record will mmap fd0 and fd2 and set-output
fd1->fd0andfd3->fd2.pollfds includes only fd0 and fd2.
But if task2 exits, the POLLHUP will appear on fd1 and fd3.
So? We are not interested in fd1 and fd3, since all our reading is done on fd0 and fd2 mmaps, no?
Quoted 8 lines
Quoted 2 lines
hm, what if task1 (fd0, fd2) exits first.. perf record will exit, but it still has to read task2..?
Ok, what happens in that case, i.e. when the fds that were set to be the ones to be polled, gets nuked, does the set-output command gets just undone? Or does the mmap stands, receiving the events from the remaining fds and the polling notifications get sent to, in this case, fd3 and fd1?
mmaps stays for fd1 and fd3.. and they get poll notifications as well, we just do not check/poll them now
Quoted 12 lines
I'll look at the kernel code for that...
Quoted 8 lines
Quoted 8 lines
I.e. when we ask the kernel to point fd B to fd A's mmap (what you called set-output) and fd B inserts an event into fd A's mmap ring buffer, we get fd A poll return as POLLRD, no?
Have to check... Otherwise we would have to poll all fds all the time, not just the ones mmaping, right?
Quoted 1 line
I think Jiri's patchset changed pollfds to include all fds for that reason.
Quoted 8 lines
Quoted 2 lines
hm, I did not think of that.. ;-) I needed more grained feedback for future features like cpu hotplug
So this is good for something you didn't tried to fix (and document) but good for something that may be nice in the future? Grumpf, we have already way too much stuff that will be eventually used but is not used right now :-\
IMO it's more clear to poll pm all event FDs.. and now with the case Adrian described it seems necessary anyway
jirka
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 04/10] perf record: Filter out POLLHUP'ed file descriptors
Em Mon, Sep 08, 2014 at 04:04:54PM +0200, Jiri Olsa escreveu:
Quoted 26 lines · Arnaldo Carvalho de Melo
Quoted 24 lines
Em Sat, Sep 06, 2014 at 10:39:15PM +0200, Jiri Olsa escreveu:
Quoted 23 lines · Arnaldo Carvalho de Melo
Quoted 13 lines
Em Fri, Sep 05, 2014 at 11:42:59AM +0300, Adrian Hunter escreveu:
Quoted 11 lines · Arnaldo Carvalho de Melo
Quoted 1 line
Em Thu, Sep 04, 2014 at 03:32:08PM +0300, Adrian Hunter escreveu:
No I was meaning something different. For example, 'perf record' opens an event for 2 processes per-cpu and gets 4 file descriptors:
task1 task2cpu0 fd0 fd1
cpu1 fd2 fd3Now, perf record will mmap fd0 and fd2 and set-output
fd1->fd0andfd3->fd2.Quoted 6 lines
Quoted 3 lines
pollfds includes only fd0 and fd2.
But if task2 exits, the POLLHUP will appear on fd1 and fd3.
So? We are not interested in fd1 and fd3, since all our reading is done on fd0 and fd2 mmaps, no?
hm, what if task1 (fd0, fd2) exits first.. perf record will exit, but it still has to read task2..?
Quoted 8 lines
Quoted 5 lines
Ok, what happens in that case, i.e. when the fds that were set to be the ones to be polled, gets nuked, does the set-output command gets just undone? Or does the mmap stands, receiving the events from the remaining fds and the polling notifications get sent to, in this case, fd3 and fd1?
mmaps stays for fd1 and fd3.. and they get poll notifications as well, we just do not check/poll them now
So what you're saying is that we should have been polling all the fds all the time?
Because after all we end up trying to consume everything in all the ring buffers when just one of them gets a POLLRD anyway...
Quoted 14 lines
Quoted 14 lines
I'll look at the kernel code for that...
Quoted 11 lines
Quoted 8 lines
I.e. when we ask the kernel to point fd B to fd A's mmap (what you called set-output) and fd B inserts an event into fd A's mmap ring buffer, we get fd A poll return as POLLRD, no?
Have to check... Otherwise we would have to poll all fds all the time, not just the ones mmaping, right?
Quoted 1 line
I think Jiri's patchset changed pollfds to include all fds for that reason.
hm, I did not think of that.. ;-) I needed more grained feedback for future features like cpu hotplug
Quoted 7 lines
Quoted 4 lines
So this is good for something you didn't tried to fix (and document) but good for something that may be nice in the future? Grumpf, we have already way too much stuff that will be eventually used but is not used right now :-\
IMO it's more clear to poll pm all event FDs.. and now with the case Adrian described it seems necessary anyway
I would have to check why was that we were polling just the one where the mmap is done, I don't recall being the one to do it, probably who did it thought that since the ring buffer is there, it was enough (and possibly scaled better, dunno) to do the polling in just one of them.
- Arnaldo
Re: [PATCH 04/10] perf record: Filter out POLLHUP'ed file descriptors
On Mon, Sep 08, 2014 at 11:33:17AM -0300, Arnaldo Carvalho de Melo wrote:
SNIP
Quoted 26 lines
So what you're saying is that we should have been polling all the fds all the time?
Because after all we end up trying to consume everything in all the ring buffers when just one of them gets a POLLRD anyway...
Quoted 18 lines
Quoted 18 lines
I'll look at the kernel code for that...
Quoted 11 lines
Quoted 8 lines
I.e. when we ask the kernel to point fd B to fd A's mmap (what you called set-output) and fd B inserts an event into fd A's mmap ring buffer, we get fd A poll return as POLLRD, no?
Have to check... Otherwise we would have to poll all fds all the time, not just the ones mmaping, right?
Quoted 1 line
I think Jiri's patchset changed pollfds to include all fds for that reason.
hm, I did not think of that.. ;-) I needed more grained feedback for future features like cpu hotplug
So this is good for something you didn't tried to fix (and document) but good for something that may be nice in the future? Grumpf, we have already way too much stuff that will be eventually used but is not used right now :-\
Quoted 7 lines
Quoted 2 lines
IMO it's more clear to poll pm all event FDs.. and now with the case Adrian described it seems necessary anyway
I would have to check why was that we were polling just the one where the mmap is done, I don't recall being the one to do it, probably who did it thought that since the ring buffer is there, it was enough (and possibly scaled better, dunno) to do the polling in just one of them.
for read notification it's ok to poll just for one event, because they all share same ringbuffer and perf_poll checks if there's ANY new data
for the hup notification I think we need to poll all of them
jirka
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 04/10] perf record: Filter out POLLHUP'ed file descriptors
Em Mon, Sep 08, 2014 at 05:10:17PM +0200, Jiri Olsa escreveu:
Quoted 9 lines · Arnaldo Carvalho de Melo
Quoted 7 lines
Quoted 2 lines
IMO it's more clear to poll pm all event FDs.. and now with the case Adrian described it seems necessary anyway
I would have to check why was that we were polling just the one where the mmap is done, I don't recall being the one to do it, probably who did it thought that since the ring buffer is there, it was enough (and possibly scaled better, dunno) to do the polling in just one of them.
Quoted 4 lines
for read notification it's ok to poll just for one event, because they all share same ringbuffer and
perf_pollchecks if there's ANY new datafor the hup notification I think we need to poll all of them
Yeah, I'm convinced of this, I'm working on a patch to make it look at all file descriptors at poll time.
I'm doing it on top of a patch that will close the mmap when it gets a POLLHUP, as discussed recently on this thread, in a response I gave to Adrian.
But since multiple fds share an mmap, we can only close that mmap when all fds are HUPed, i.e. we need to reference count struct perf_mmap, which is what I am doing.
I.e. the fist mmaps and sets perf_mmap.nfds to 1, the next one, just after doing that PERF_EVENT_IOC_SET_OUTPUT will bump perf_mmap.nfds, unmap gets replaced by mmap_put, that decs and calls munmap when it hits zero, yadda, yadda.
- Arnaldo
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 04/10] perf record: Filter out POLLHUP'ed file descriptors
Em Mon, Sep 08, 2014 at 12:38:24PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 10 lines
Em Mon, Sep 08, 2014 at 05:10:17PM +0200, Jiri Olsa escreveu:
Quoted 8 lines · Arnaldo Carvalho de Melo
Quoted 7 lines
Quoted 2 lines
IMO it's more clear to poll pm all event FDs.. and now with the case Adrian described it seems necessary anyway
I would have to check why was that we were polling just the one where the mmap is done, I don't recall being the one to do it, probably who did it thought that since the ring buffer is there, it was enough (and possibly scaled better, dunno) to do the polling in just one of them.
Quoted 7 lines
Quoted 4 lines
for read notification it's ok to poll just for one event, because they all share same ringbuffer and
perf_pollchecks if there's ANY new datafor the hup notification I think we need to poll all of them
Yeah, I'm convinced of this, I'm working on a patch to make it look at all file descriptors at poll time.
Done, its at:
this is in a refreshed branch so that it gets in the right time in the patch kit
Comment:
-----
perf evlist: We need to poll all event file descriptors
Because we want to notice when they get POLLHUP'ed, so that we can figure out when all threads exited in a workload being monitored.
We can't just monitor the fds that were mmaped, we need to notice when all the fds that were PERF_EVENT_IOC_SET_OUTPUT'ed too, because the mmap stays even after the fd that originally was used to do the mmap call went away, its only when all the set-output fds for a mmap are gone that the mmap is.
-----
Full branch is available at:
git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git perf/fdarray.v2
Web interface:
https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/fdarray.v2
Quoted 4 lines
I'm doing it on top of a patch that will close the mmap when it gets a POLLHUP, as discussed recently on this thread, in a response I gave to Adrian.
Quoted 8 lines
But since multiple fds share an mmap, we can only close that mmap when all fds are HUPed, i.e. we need to reference count struct
perf_mmap, which is what I am doing.I.e. the fist mmaps and sets perf_mmap.nfds to 1, the next one, just after doing that
PERF_EVENT_IOC_SET_OUTPUTwill bump perf_mmap.nfds, unmap gets replaced bymmap_put, that decs and calls munmap when it hits zero, yadda, yadda.
Should be ok, now, with the above patch (kinda one liner, just moving the add_pollfd call) + the patches at the top of the branch that introduce the mmap refcounting and the fdarray facility to store some info (what I think you called poll_item in your patchset) associated to each entry in the file descriptor array).
Please let me know if you can take a look at the branch via the web interface to see if that solves the problems you reported before I repost them or if you prefer that I repost so that you can check via e-mail.
I will repost it one final time, after lunch ;-)
- Arnaldo
tip-bot for Arnaldo Carvalho de Melo
merge bot[tip:perf/core] perf evlist: Refcount mmaps
In reply to Arnaldo Carvalho de Meloperf evlist: Refcount mmaps
Commit-ID: 82396986032915c1572bfb74b224fcc2e4e8ba7c Gitweb: http://git.kernel.org/tip/82396986032915c1572bfb74b224fcc2e4e8ba7c Author: Arnaldo Carvalho de Melo <[email protected]> AuthorDate: Mon, 8 Sep 2014 13:26:35 -0300 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Thu, 25 Sep 2014 16:46:55 -0300
perf evlist: Refcount mmaps
We need to know how many fds are using a perf mmap via PERF_EVENT_IOC_SET_OUTPUT, so that we can know when to ditch an mmap, refcount it.
v2: Automatically unmap it when the refcount hits one, which will happen when all fds are filtered by perf_evlist__filter_pollfd(), in later patches.
--- tools/perf/util/evlist.c | 47 +++++++++++++++++++++++++++++++++++++++++++++-- tools/perf/util/evlist.h | 6 ++++++ 2 files changed, 51 insertions(+), 2 deletions(-)
tools/perf/util/evlist.c +45 -2
@@ -28,6 +28,8 @@#define FD (e, x, y) (*(int *)xyarray__entry(e->fd, x, y))#define SID (e, x, y) xyarray__entry(e->sample_id, x, y) +static void __perf_evlist__munmap (struct perf_evlist *evlist,int idx); +void perf_evlist__init (struct perf_evlist *evlist,struct cpu_map *cpus,struct thread_map *threads) {
@@ -651,14 +653,36 @@ union perf_event *perf_evlist__mmap_read(struct perf_evlist *evlist, int idx)return event; } +static bool perf_mmap__empty (struct perf_mmap *md) +{ +return perf_mmap__read_head (md) != md->prev; +} + +static void perf_evlist__mmap_get (struct perf_evlist *evlist,int idx) +{ + ++evlist->mmap[idx].refcnt; +} + +static void perf_evlist__mmap_put (struct perf_evlist *evlist,int idx) +{ +BUG_ON (evlist->mmap[idx].refcnt ==0 ); + +if (--evlist->mmap[idx].refcnt ==0 ) +__perf_evlist__munmap (evlist, idx); +} +void perf_evlist__mmap_consume (struct perf_evlist *evlist,int idx) { +struct perf_mmap *md = &evlist->mmap[idx]; +if (!evlist->overwrite) { -struct perf_mmap *md = &evlist->mmap[idx];unsigned int old = md->prev;perf_mmap__write_tail (md, old); } + +if (md->refcnt ==1 &&perf_mmap__empty (md)) +perf_evlist__mmap_put (evlist, idx); }static void __perf_evlist__munmap (struct perf_evlist *evlist,int idx)
@@ -666,6 +690,7 @@ static void __perf_evlist__munmap(struct perf_evlist *evlist, int idx)if (evlist->mmap[idx].base !=NULL ) {munmap (evlist->mmap[idx].base, evlist->mmap_len); evlist->mmap[idx].base =NULL ; + evlist->mmap[idx].refcnt =0 ; } }
@@ -699,6 +724,20 @@ struct mmap_params {static int __perf_evlist__mmap (struct perf_evlist *evlist,int idx,struct mmap_params *mp,int fd) { +/* +* The last one will be done at perf_evlist__mmap_consume(), so that we +* make sure we don't prevent tools from consuming every last event in +* the ring buffer. +* +* I.e. we can get the POLLHUP meaning that the fd doesn't exist +* anymore, but the last events for it are still in the ring buffer, +* waiting to be consumed. +* +* Tools can chose to ignore this at their own discretion, but the +* evlist layer can't just drop it when filtering events in +* perf_evlist__filter_pollfd(). +*/ + evlist->mmap[idx].refcnt =2 ; evlist->mmap[idx].prev =0 ; evlist->mmap[idx].mask = mp->mask; evlist->mmap[idx].base = mmap(NULL, evlist->mmap_len, mp->prot,
@@ -734,10 +773,14 @@ static int perf_evlist__mmap_per_evsel(struct perf_evlist *evlist, int idx, }else {if (ioctl (fd,PERF_EVENT_IOC_SET_OUTPUT , *output) !=0 )return -1 ; + +perf_evlist__mmap_get (evlist, idx); } -if (perf_evlist__add_pollfd (evlist, fd) <0 ) +if (perf_evlist__add_pollfd (evlist, fd) <0 ) { +perf_evlist__mmap_put (evlist, idx);return -1 ; + }if ((evsel->attr.read_format &PERF_FORMAT_ID ) &&perf_evlist__id_add_fd (evlist, evsel, cpu, thread, fd) <0 )
tools/perf/util/evlist.h +6 -0
@@ -18,9 +18,15 @@ struct record_opts;#define PERF_EVLIST__HLIST_BITS 8#define PERF_EVLIST__HLIST_SIZE (1 << PERF_EVLIST__HLIST_BITS) +/** +* struct perf_mmap - perf's ring buffer mmap details +* +* @refcnt - e.g. code using PERF_EVENT_IOC_SET_OUTPUT to share this +*/ struct perf_mmap {void *base;int mask; +int refcnt;unsigned int prev;char event_copy[PERF_SAMPLE_MAX_SIZE ]; };
tip-bot for Arnaldo Carvalho de Melo
merge bot[tip:perf/core] perf evlist: We need to poll all event file descriptors
In reply to Jiri Olsaperf evlist: We need to poll all event file descriptors
Commit-ID: 033fa713db66b96d5779e6a93d50ff821bc1abd2 Gitweb: http://git.kernel.org/tip/033fa713db66b96d5779e6a93d50ff821bc1abd2 Author: Arnaldo Carvalho de Melo <[email protected]> AuthorDate: Mon, 8 Sep 2014 12:55:12 -0300 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitDate: Thu, 25 Sep 2014 16:46:54 -0300
perf evlist: We need to poll all event file descriptors
Because we want to notice when they get POLLHUP'ed, so that we can figure out when all threads exited in a workload being monitored.
We can't just monitor the fds that were mmaped, we need to notice when all the fds that were PERF_EVENT_IOC_SET_OUTPUT'ed too, because the mmap stays even after the fd that originally was used to do the mmap call went away, its only when all the set-output fds for a mmap are gone that the mmap is.
--- tools/perf/util/evlist.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/evlist.c +2 -2
@@ -717,8 +717,6 @@ static int __perf_evlist__mmap(struct perf_evlist *evlist, int idx, evlist->mmap[idx].base =NULL ;return -1 ; } - -perf_evlist__add_pollfd (evlist, fd);return 0 ; }
@@ -745,6 +743,8 @@ static int perf_evlist__mmap_per_evsel(struct perf_evlist *evlist, int idx,return -1 ; } +perf_evlist__add_pollfd (evlist, fd); +if ((evsel->attr.read_format &PERF_FORMAT_ID ) &&perf_evlist__id_add_fd (evlist, evsel, cpu, thread, fd) <0 )return -1 ;
Arnaldo Carvalho de Melo
maintainerPATCH 5/10 perf trace: Filter out POLLHUP'ed file descriptors
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
So that we don't continue polling on vanished file descriptors, i.e. file descriptors for events monitoring threads that exited.
I.e. the following 'trace' command now exits as expected, instead of staying in an eternal loop:
$ sleep 5s &
$ trace -p `pidof sleep`--- tools/perf/builtin-trace.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tools/perf/builtin-trace.c +2 -1
@@ -2171,7 +2171,8 @@ next_event:if (trace->nr_events == before) {int timeout = done ?100 :-1 ; -if (poll (evlist->pollfd, evlist->nr_fds, timeout) >0 ) +if (poll (evlist->pollfd, evlist->nr_fds, timeout) >0 && +perf_evlist__filter_pollfd (evlist,POLLERR |POLLHUP ) >0 ) goto again; }else { goto again;
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 6/10 perf evlist: Allow growing pollfd on add method
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
This way we will be able to add more file descriptors to be polled, like stdin or some timer fd.
At this point we might as well yank the pollfd class from evlist so that it can be used in other places.
--- tools/perf/util/evlist.c | 40 +++++++++++++++++++++++++++++++++++----- tools/perf/util/evlist.h | 5 +++-- 2 files changed, 38 insertions(+), 7 deletions(-)
tools/perf/util/evlist.c +35 -5
@@ -402,7 +402,21 @@ int perf_evlist__enable_event_idx(struct perf_evlist *evlist,return perf_evlist__enable_event_thread (evlist, evsel, idx); } -static int perf_evlist__alloc_pollfd (struct perf_evlist *evlist) +static int perf_evlist__grow_pollfd (struct perf_evlist *evlist,int hint) +{ +int nr_fds_alloc = evlist->nr_fds_alloc + hint; +size_t size =sizeof (struct pollfd ) * nr_fds_alloc; +struct pollfd *pollfd =realloc (evlist->pollfd, size); + +if (pollfd ==NULL ) +return -ENOMEM ; + + evlist->nr_fds_alloc = nr_fds_alloc; + evlist->pollfd = pollfd; +return 0 ; +} + +int perf_evlist__alloc_pollfd (struct perf_evlist *evlist) {int nr_cpus =cpu_map__nr (evlist->cpus);int nr_threads =thread_map__nr (evlist->threads);
@@ -416,16 +430,28 @@ static int perf_evlist__alloc_pollfd(struct perf_evlist *evlist) nfds += nr_cpus * nr_threads; } - evlist->pollfd =malloc (sizeof (struct pollfd ) * nfds); -return evlist->pollfd !=NULL ?0 : -ENOMEM ; +if (evlist->nr_fds_alloc - evlist->nr_fds < nfds && +perf_evlist__grow_pollfd (evlist, nfds) <0 ) +return -ENOMEM ; + +return 0 ; } -void perf_evlist__add_pollfd (struct perf_evlist *evlist,int fd) +int perf_evlist__add_pollfd (struct perf_evlist *evlist,int fd) { +/* +* XXX: 64 is arbitrary, just not to call realloc at each fd. +* Find a better autogrowing heuristic +*/ +if (evlist->nr_fds == evlist->nr_fds_alloc && +perf_evlist__grow_pollfd (evlist,64 ) <0 ) +return -ENOMEM ; +fcntl (fd,F_SETFL ,O_NONBLOCK ); evlist->pollfd[evlist->nr_fds].fd = fd; evlist->pollfd[evlist->nr_fds].events =POLLIN |POLLERR |POLLHUP ; evlist->nr_fds++; +return 0 ; }int perf_evlist__filter_pollfd (struct perf_evlist *evlist,short revents_and_mask)
@@ -718,7 +744,11 @@ static int __perf_evlist__mmap(struct perf_evlist *evlist, int idx,return -1 ; } -perf_evlist__add_pollfd (evlist, fd); +if (perf_evlist__add_pollfd (evlist, fd) <0 ) { +__perf_evlist__munmap (evlist->mmap[idx].base, idx); +return -1 ; + } +return 0 ; }
tools/perf/util/evlist.h +3 -2
@@ -30,6 +30,7 @@ struct perf_evlist {int nr_entries;int nr_groups;int nr_fds; +int nr_fds_alloc;int nr_mmaps;size_t mmap_len;int id_pos;
@@ -82,8 +83,8 @@ perf_evlist__find_tracepoint_by_name(struct perf_evlist *evlist,void perf_evlist__id_add (struct perf_evlist *evlist,struct perf_evsel *evsel,int cpu,int thread,u64 id); -void perf_evlist__add_pollfd (struct perf_evlist *evlist,int fd); - +int perf_evlist__add_pollfd (struct perf_evlist *evlist,int fd); +int perf_evlist__alloc_pollfd (struct perf_evlist *evlist);int perf_evlist__filter_pollfd (struct perf_evlist *evlist,short revents_and_mask);struct perf_evsel *perf_evlist__id2evsel (struct perf_evlist *evlist,u64 id);
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 7/10 perf tests: Add pollfd growing test
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
[acme@ssdandy linux]$ perf test "Add fd"
34: Add fd to pollfd array, making it autogrow : Ok
[acme@ssdandy linux]$ perf test -v "Add fd"
34: Add fd to pollfd array, making it autogrow :
--- start ---
test child forked, pid 19817
before growing array: 2 [ 1, 2 ] after 3rd add_pollfd: 3 [ 1, 2, 35 ] after 4th add_pollfd: 4 [ 1, 2, 35, 88 ]
test child finished with 0
---- end ----
Add fd to pollfd array, making it autogrow: Ok [acme@ssdandy linux]$
--- tools/perf/tests/builtin-test.c | 4 ++ tools/perf/tests/evlist.c | 111 ++++++++++++++++++++++++++++++++++++++++ tools/perf/tests/tests.h | 1 + 3 files changed, 116 insertions(+)
tools/perf/tests/builtin-test.c +4 -0
@@ -162,6 +162,10 @@ static struct test { .func = test__perf_evlist__filter_pollfd, }, { + .desc = "Add fd to pollfd array, making it autogrow", + .func = test__perf_evlist__add_pollfd, + }, + { .func = NULL, }, };
tools/perf/tests/evlist.c +111 -0
119 lines not on this page · raw message
tools/perf/tests/tests.h +1 -0
@@ -50,6 +50,7 @@ int test__hists_output(void);int test__hists_cumulate (void );int test__switch_tracking (void );int test__perf_evlist__filter_pollfd (void ); +int test__perf_evlist__add_pollfd (void );#if defined(__x86_64__) || defined(__i386__) || defined(__arm__)#ifdef HAVE_DWARF_UNWIND_SUPPORT
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 8/10 perf kvm stat live: Use perf_evlist__add_pollfd() instead of local equivalent
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
Since we can add file descriptors to the evlist pollfd and it will autogrow, no need to copy all events to a local pollfd array, just add the timer and stdin file descriptors.
--- tools/perf/builtin-kvm.c | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-)
tools/perf/builtin-kvm.c +9 -13
@@ -919,15 +919,8 @@ static int kvm_events_live_report(struct perf_kvm_stat *kvm)signal (SIGINT , sig_handler);signal (SIGTERM , sig_handler); -/* copy pollfds -- need to add timerfd and stdin */ +/* use pollfds -- need to add timerfd and stdin */ nr_fds = kvm->evlist->nr_fds; - pollfds =zalloc (sizeof (struct pollfd ) * (nr_fds +2 )); -if (!pollfds) { - err = -ENOMEM ; - goto out; - } -memcpy (pollfds, kvm->evlist->pollfd, -sizeof (struct pollfd ) * kvm->evlist->nr_fds);/* add timer fd */ if (perf_kvm__timerfd_create (kvm) <0 ) {
@@ -935,17 +928,21 @@ static int kvm_events_live_report(struct perf_kvm_stat *kvm) goto out; } - pollfds[nr_fds].fd = kvm->timerfd; - pollfds[nr_fds].events =POLLIN ; +if (perf_evlist__add_pollfd (kvm->evlist, kvm->timerfd)) + goto out; + nr_fds++; - pollfds[nr_fds].fd =fileno (stdin); - pollfds[nr_fds].events =POLLIN ; +if (perf_evlist__add_pollfd (kvm->evlist,fileno (stdin))) + goto out; + nr_stdin = nr_fds; nr_fds++;if (fd_set_nonblock (fileno (stdin)) !=0 ) goto out; + pollfds = kvm->evlist->pollfd; +/* everything is good - enable the events and process */ perf_evlist__enable (kvm->evlist);
@@ -979,7 +976,6 @@ out:close (kvm->timerfd);tcsetattr (0 ,TCSAFLUSH , &save); -free (pollfds);return err; }
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 9/10 perf evlist: Introduce poll method for common code idiom
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
Since we have access two evlist members in all these poll calls, provide a helper.
This will also help to make the patch introducing the pollfd class more clear, as the evlist specific uses will be hiden away perf_evlist__poll().
--- tools/perf/builtin-record.c | 2 +- tools/perf/builtin-top.c | 4 ++-- tools/perf/builtin-trace.c | 2 +- tools/perf/tests/open-syscall-tp-fields.c | 2 +- tools/perf/tests/perf-record.c | 2 +- tools/perf/tests/task-exit.c | 2 +- tools/perf/util/evlist.c | 5 +++++ tools/perf/util/evlist.h | 2 ++ tools/perf/util/python.c | 2 +- 9 files changed, 15 insertions(+), 8 deletions(-)
tools/perf/builtin-record.c +1 -1
@@ -459,7 +459,7 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)if (hits == rec->samples) {if (done)break ; - err =poll (rec->evlist->pollfd, rec->evlist->nr_fds,-1 ); + err =perf_evlist__poll (rec->evlist,-1 );/* * Propagate error, only if there's any. Ignore positive * number of returned events and interrupt error.
tools/perf/builtin-top.c +2 -2
@@ -964,7 +964,7 @@ static int __cmd_top(struct perf_top *top)perf_evlist__enable (top->evlist);/* Wait for a minimal set of events before starting the snapshot */ -poll (top->evlist->pollfd, top->evlist->nr_fds,100 ); +perf_evlist__poll (top->evlist,100 );perf_top__mmap_read (top);
@@ -991,7 +991,7 @@ static int __cmd_top(struct perf_top *top)perf_top__mmap_read (top);if (hits == top->samples) - ret =poll (top->evlist->pollfd, top->evlist->nr_fds,100 ); + ret =perf_evlist__poll (top->evlist,100 ); } ret =0 ;
tools/perf/builtin-trace.c +1 -1
@@ -2171,7 +2171,7 @@ next_event:if (trace->nr_events == before) {int timeout = done ?100 :-1 ; -if (poll (evlist->pollfd, evlist->nr_fds, timeout) >0 && +if (perf_evlist__poll (evlist, timeout) >0 &&perf_evlist__filter_pollfd (evlist,POLLERR |POLLHUP ) >0 ) goto again; }else {
tools/perf/tests/open-syscall-tp-fields.c +1 -1
@@ -105,7 +105,7 @@ int test__syscall_open_tp_fields(void) }if (nr_events == before) -poll (evlist->pollfd, evlist->nr_fds,10 ); +perf_evlist__poll (evlist,10 );if (++nr_polls >5 ) {pr_debug ("%s: no events!\n" , __func__);
tools/perf/tests/perf-record.c +1 -1
@@ -268,7 +268,7 @@ int test__PERF_RECORD(void)* perf_event_attr.wakeup_events, just PERF_EVENT_SAMPLE does. */ if (total_events == before && false) -poll (evlist->pollfd, evlist->nr_fds,-1 ); +perf_evlist__poll (evlist,-1 );sleep (1 );if (++wakeups >5 ) {
tools/perf/tests/task-exit.c +1 -1
@@ -105,7 +105,7 @@ retry: }if (!exited || !nr_exit) { -poll (evlist->pollfd, evlist->nr_fds,-1 ); +perf_evlist__poll (evlist,-1 ); goto retry; }
tools/perf/util/evlist.c +5 -0
@@ -475,6 +475,11 @@ int perf_evlist__filter_pollfd(struct perf_evlist *evlist, short revents_and_masreturn nr_fds; } +int perf_evlist__poll (struct perf_evlist *evlist,int timeout) +{ +return poll (evlist->pollfd, evlist->nr_fds, timeout); +} +static void perf_evlist__id_hash (struct perf_evlist *evlist,struct perf_evsel *evsel,int cpu,int thread,u64 id)
tools/perf/util/evlist.h +2 -0
@@ -87,6 +87,8 @@ int perf_evlist__add_pollfd(struct perf_evlist *evlist, int fd);int perf_evlist__alloc_pollfd (struct perf_evlist *evlist);int perf_evlist__filter_pollfd (struct perf_evlist *evlist,short revents_and_mask); +int perf_evlist__poll (struct perf_evlist *evlist,int timeout); +struct perf_evsel *perf_evlist__id2evsel (struct perf_evlist *evlist,u64 id);struct perf_sample_id *perf_evlist__id2sid (struct perf_evlist *evlist,u64 id);
tools/perf/util/python.c +1 -1
@@ -736,7 +736,7 @@ static PyObject *pyrf_evlist__poll(struct pyrf_evlist *pevlist,if (!PyArg_ParseTupleAndKeywords (args, kwargs,"|i" , kwlist, &timeout))return NULL ; - n =poll (evlist->pollfd, evlist->nr_fds, timeout); + n =perf_evlist__poll (evlist, timeout);if (n <0 ) {PyErr_SetFromErrno (PyExc_OSError);return NULL ;
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 10/10 tools lib api: Adopt fdarray class from perf's evlist
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
The extensible file description array that grew in the perf_evlist class can be useful for other tools, as it is not something that only evlists need, so move it to tools/lib/api/fd to ease sharing it.
--- tools/lib/api/Makefile | 7 ++- tools/lib/api/fd/array.c | 108 ++++++++++++++++++++++++++++++++++++++++++++++ tools/lib/api/fd/array.h | 32 ++++++++++++++ tools/perf/builtin-kvm.c | 4 +- tools/perf/tests/evlist.c | 67 ++++++++++++++-------------- tools/perf/util/evlist.c | 57 ++++-------------------- tools/perf/util/evlist.h | 5 +-- tools/perf/util/python.c | 4 +- 8 files changed, 195 insertions(+), 89 deletions(-) create mode 100644 tools/lib/api/fd/array.c create mode 100644 tools/lib/api/fd/array.h
tools/lib/api/Makefile +6 -1
@@ -10,9 +10,14 @@ LIB_OBJS= LIB_H += fs/debugfs.h LIB_H += fs/fs.h +# See comment below about piggybacking... +LIB_H += fd/array.h LIB_OBJS += $(OUTPUT)fs/debugfs.o LIB_OBJS += $(OUTPUT)fs/fs.o +# XXX piggybacking here, need to introduce libapikfd, or rename this +# to plain libapik.a and make it have it all api goodies +LIB_OBJS += $(OUTPUT)fd/array.o LIBFILE = libapikfs.a
@@ -29,7 +34,7 @@ $(LIBFILE): $(LIB_OBJS) $(LIB_OBJS): $(LIB_H) libapi_dirs: - $(QUIET_MKDIR)mkdir -p $(OUTPUT)fs/ + $(QUIET_MKDIR)mkdir -p $(OUTPUT){fs,fd}/ $(OUTPUT)%.o: %.c libapi_dirs $(QUIET_CC)$(CC) -o $@ -c $(ALL_CFLAGS) $<
tools/lib/api/fd/array.c new file +108 -0
108 lines not on this page · raw message
tools/lib/api/fd/array.h new file +32 -0
@@ -0,0 +1,32 @@ +#ifndef __API_FD_ARRAY__ +#define __API_FD_ARRAY__ + +#include <stdio.h> + +struct pollfd ; + +struct fdarray { +int nr; +int nr_alloc; +int nr_autogrow; +struct pollfd *entries; +}; + +void fdarray__init (struct fdarray *fda,int nr_autogrow); +void fdarray__exit (struct fdarray *fda); + +struct fdarray *fdarray__new (int nr_alloc,int nr_autogrow); +void fdarray__delete (struct fdarray *fda); + +int fdarray__add (struct fdarray *fda,int fd,short revents); +int fdarray__poll (struct fdarray *fda,int timeout); +int fdarray__filter (struct fdarray *fda,short revents); +int fdarray__grow (struct fdarray *fda,int extra); +int fdarray__fprintf (struct fdarray *fda,FILE *fp); + +static inline int fdarray__available_entries (struct fdarray *fda) +{ +return fda->nr_alloc - fda->nr; +} + +#endif /* __API_FD_ARRAY__ */
tools/perf/builtin-kvm.c +2 -2
@@ -920,7 +920,7 @@ static int kvm_events_live_report(struct perf_kvm_stat *kvm)signal (SIGTERM , sig_handler);/* use pollfds -- need to add timerfd and stdin */ - nr_fds = kvm->evlist->nr_fds; + nr_fds = kvm->evlist->pollfd.nr;/* add timer fd */ if (perf_kvm__timerfd_create (kvm) <0 ) {
@@ -941,7 +941,7 @@ static int kvm_events_live_report(struct perf_kvm_stat *kvm)if (fd_set_nonblock (fileno (stdin)) !=0 ) goto out; - pollfds = kvm->evlist->pollfd; + pollfds = kvm->evlist->pollfd.entries;/* everything is good - enable the events and process */ perf_evlist__enable (kvm->evlist);
tools/perf/tests/evlist.c +35 -32
154 lines not on this page · raw message
tools/perf/util/evlist.c +8 -49
101 lines not on this page · raw message
tools/perf/util/evlist.h +2 -3
@@ -2,6 +2,7 @@#define __PERF_EVLIST_H 1#include <linux/list.h> +#include <api/fd/array.h> #include <stdio.h> #include "../perf.h" #include "event.h"
@@ -29,8 +30,6 @@ struct perf_evlist {struct hlist_head heads[PERF_EVLIST__HLIST_SIZE ];int nr_entries;int nr_groups; -int nr_fds; -int nr_fds_alloc;int nr_mmaps;size_t mmap_len;int id_pos;
@@ -41,8 +40,8 @@ struct perf_evlist {pid_t pid; } workload;bool overwrite; +struct fdarray pollfd;struct perf_mmap *mmap; -struct pollfd *pollfd;struct thread_map *threads;struct cpu_map *cpus;struct perf_evsel *selected;
tools/perf/util/python.c +2 -2
@@ -753,9 +753,9 @@ static PyObject *pyrf_evlist__get_pollfd(struct pyrf_evlist *pevlist,PyObject *list =PyList_New (0 );int i; -for (i =0 ; i < evlist->nr_fds; ++i) { +for (i =0 ; i < evlist->pollfd.nr; ++i) {PyObject *file; -FILE *fp =fdopen (evlist->pollfd[i].fd,"r" ); +FILE *fp =fdopen (evlist->pollfd.entries[i].fd,"r" );if (fp ==NULL ) goto free_list;
Signature · 1 line
1.9.3