perf pollfd v3
Subsystems (2)
Arnaldo Carvalho de Melo
maintainercover perf pollfd v3
From: Arnaldo Carvalho de Melo <[email protected]>
Hi,
Main different to what was discussed in v2[1]:
Poll all event file descriptors, not just the ones mmaped.
This is a bug that was present before, noticed by Adrian Hunter while
reviewing this patchkit, i.e. we were polling just the mmaped ones, because then we would look at all the ring buffers when just one of the polled descriptors was ready to read.
But this could lead to delays in polling when one of the mmaped fds was
closed and we would then not be notified when another, associated to it by means of PERF_EVENT_IOC_SET_OUTPUT, had events available.
Also refcount the mmaps associated by means of PERF_EVENT_IOC_SET_OUTPUT,
so that we can properly unmap a ring buffer to which all its file descriptors had been closed, for longer running apps.
It is all available on my tree:
git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git perf/fdarray.v2
I kept the v3 for the series as I initially called it pollfd, but since the
class that grew out of it was named fdarray, thus the fdarray.v2 branch name.
Please let me know if there are still any other problems, now back to
fixing up my last perf/core pull req wrt those external proggies problems.
- Arnaldo
[1] http://lkml.kernel.org/r/[email protected]
Arnaldo Carvalho de Melo (14):
perf evlist: Introduce perf_evlist__filter_pollfd method
perf tests: Add test for perf_evlist__filter_pollfd()
perf evlist: Monitor POLLERR and POLLHUP events too
perf evlist: We need to poll all event file descriptors
perf record: Filter out POLLHUP'ed file descriptors
perf trace: Filter out POLLHUP'ed file descriptors
perf evlist: Allow growing pollfd on add method
perf tests: Add pollfd growing test
perf kvm stat live: Use perf_evlist__add_pollfd() instead of local equivalent perf evlist: Introduce poll method for common code idiom
tools lib api: Adopt fdarray class from perf's evlist
perf evlist: Refcount mmaps
tools lib fd array: Allow associating an integer cookie with each entry perf evlist: Unmap ring buffer when fd is nuked
tools/lib/api/Makefile | 7 +- tools/lib/api/fd/array.c | 128 ++++++++++++++++++ tools/lib/api/fd/array.h | 34 +++++ tools/perf/Makefile.perf | 1 + tools/perf/builtin-kvm.c | 24 ++-- tools/perf/builtin-record.c | 5 +- tools/perf/builtin-top.c | 4 +- tools/perf/builtin-trace.c | 3 +- tools/perf/tests/builtin-test.c | 8 ++ tools/perf/tests/evlist.c | 217 ++++++++++++++++++++++++++++++ tools/perf/tests/open-syscall-tp-fields.c | 2 +- tools/perf/tests/perf-record.c | 2 +- tools/perf/tests/task-exit.c | 2 +- tools/perf/tests/tests.h | 2 + tools/perf/util/evlist.c | 77 +++++++++-- tools/perf/util/evlist.h | 16 ++- tools/perf/util/python.c | 6 +- 17 files changed, 499 insertions(+), 39 deletions(-) create mode 100644 tools/lib/api/fd/array.c create mode 100644 tools/lib/api/fd/array.h create mode 100644 tools/perf/tests/evlist.c
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 1/14 perf evlist: Introduce perf_evlist__filter_pollfd method
From: Arnaldo Carvalho de Melo <[email protected]>
To remove all entries in evlist->pollfd[] that have revents matching at least one of the bits in the specified mask.
It'll adjust evlist->nr_fds to the number of unfiltered fds and will return this value, as a convenience and to avoid requiring direct access to internal state of perf_evlist objects.
This will be used after polling the evlist fds so that we remove fds that were closed by the kernel.
--- tools/perf/util/evlist.c | 21 +++++++++++++++++++++ tools/perf/util/evlist.h | 2 ++ 2 files changed, 23 insertions(+)
tools/perf/util/evlist.c +21 -0
@@ -428,6 +428,27 @@ void perf_evlist__add_pollfd(struct perf_evlist *evlist, int fd) evlist->nr_fds++; } +int perf_evlist__filter_pollfd (struct perf_evlist *evlist,short revents_and_mask) +{ +int fd, nr_fds =0 ; + +if (evlist->nr_fds ==0 ) +return 0 ; + +for (fd =0 ; fd < evlist->nr_fds; ++fd) { +if (evlist->pollfd[fd].revents & revents_and_mask) +continue ; + +if (fd != nr_fds) + evlist->pollfd[nr_fds] = evlist->pollfd[fd]; + + ++nr_fds; + } + + evlist->nr_fds = nr_fds; +return nr_fds; +} +static void perf_evlist__id_hash (struct perf_evlist *evlist,struct perf_evsel *evsel,int cpu,int thread,u64 id)
tools/perf/util/evlist.h +2 -0
@@ -84,6 +84,8 @@ void perf_evlist__id_add(struct perf_evlist *evlist, struct perf_evsel *evsel,void perf_evlist__add_pollfd (struct perf_evlist *evlist,int fd); +int perf_evlist__filter_pollfd (struct perf_evlist *evlist,short revents_and_mask); +struct perf_evsel *perf_evlist__id2evsel (struct perf_evlist *evlist,u64 id);struct perf_sample_id *perf_evlist__id2sid (struct perf_evlist *evlist,u64 id);
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 2/14 perf tests: Add test for perf_evlist__filter_pollfd()
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
That will use a synthetic evlist with just what is touched by this new method to check that it works as expected.
Output in verbose mode:
$ perf test -v pollfd 33: Filter fds with revents mask in a pollfd array : --- start --- filtering all but pollfd[2]: before: 5 [ 5, 4, 3, 2, 1 ] after: 1 [ 3 ] filtering all but (pollfd[0], pollfd[3]): before: 5 [ 5, 4, 3, 2, 1 ] after: 2 [ 5, 2 ] test child finished with 0 ---- end ---- Filter fds with revents mask in a pollfd array: Ok $
--- tools/perf/Makefile.perf | 1 + tools/perf/tests/builtin-test.c | 4 ++ tools/perf/tests/evlist.c | 103 ++++++++++++++++++++++++++++++++++++++++ tools/perf/tests/tests.h | 1 + 4 files changed, 109 insertions(+) create mode 100644 tools/perf/tests/evlist.c
tools/perf/Makefile.perf +1 -0
@@ -399,6 +399,7 @@ LIB_OBJS += $(OUTPUT)tests/open-syscall-tp-fields.o LIB_OBJS += $(OUTPUT)tests/mmap-basic.o LIB_OBJS += $(OUTPUT)tests/perf-record.o LIB_OBJS += $(OUTPUT)tests/rdpmc.o +LIB_OBJS += $(OUTPUT)tests/evlist.o LIB_OBJS += $(OUTPUT)tests/evsel-roundtrip-name.o LIB_OBJS += $(OUTPUT)tests/evsel-tp-sched.o LIB_OBJS += $(OUTPUT)tests/pmu.o
tools/perf/tests/builtin-test.c +4 -0
@@ -158,6 +158,10 @@ static struct test { .func = test__switch_tracking, }, { + .desc = "Filter fds with revents mask in a pollfd array", + .func = test__perf_evlist__filter_pollfd, + }, + { .func = NULL, }, };
tools/perf/tests/evlist.c new file +103 -0
103 lines not on this page · raw message
tools/perf/tests/tests.h +1 -0
@@ -49,6 +49,7 @@ int test__thread_mg_share(void);int test__hists_output (void );int test__hists_cumulate (void );int test__switch_tracking (void ); +int test__perf_evlist__filter_pollfd (void );#if defined(__x86_64__) || defined(__i386__) || defined(__arm__)#ifdef HAVE_DWARF_UNWIND_SUPPORT
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 3/14 perf evlist: Monitor POLLERR and POLLHUP events too
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
We want to know when the fd went away, like when a monitored thread exits.
If we do not monitor such events, then the tools will wait forever on events from a vanished thread, like when running:
$ sleep 5s & $ perf record -p `pidof sleep`
This builds upon the kernel patch by Jiri Olsa that actually makes a poll on those file descriptors to return POLLHUP.
It is also needed to change the tools to use perf_evlist__filter_pollfd() to check if there are remainings fds to monitor or if all are gone, in which case they will exit the poll/mmap/read loop.
--- tools/perf/util/evlist.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/evlist.c +1 -1
@@ -424,7 +424,7 @@ void perf_evlist__add_pollfd(struct perf_evlist *evlist, int fd) {fcntl (fd,F_SETFL ,O_NONBLOCK ); evlist->pollfd[evlist->nr_fds].fd = fd; - evlist->pollfd[evlist->nr_fds].events =POLLIN ; + evlist->pollfd[evlist->nr_fds].events =POLLIN |POLLERR |POLLHUP ; evlist->nr_fds++; }
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 4/14 perf evlist: We need to poll all event file descriptors
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
Because we want to notice when they get POLLHUP'ed, so that we can figure out when all threads exited in a workload being monitored.
We can't just monitor the fds that were mmaped, we need to notice when all the fds that were PERF_EVENT_IOC_SET_OUTPUT'ed too, because the mmap stays even after the fd that originally was used to do the mmap call went away, its only when all the set-output fds for a mmap are gone that the mmap is.
--- tools/perf/util/evlist.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/evlist.c +2 -2
@@ -717,8 +717,6 @@ static int __perf_evlist__mmap(struct perf_evlist *evlist, int idx, evlist->mmap[idx].base =NULL ;return -1 ; } - -perf_evlist__add_pollfd (evlist, fd);return 0 ; }
@@ -745,6 +743,8 @@ static int perf_evlist__mmap_per_evsel(struct perf_evlist *evlist, int idx,return -1 ; } +perf_evlist__add_pollfd (evlist, fd); +if ((evsel->attr.read_format &PERF_FORMAT_ID ) &&perf_evlist__id_add_fd (evlist, evsel, cpu, thread, fd) <0 )return -1 ;
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 5/14 perf record: Filter out POLLHUP'ed file descriptors
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
So that we don't continue polling on vanished file descriptors, i.e. file descriptors for events monitoring threads that exited.
I.e. the following 'perf record' command now exits as expected, instead of staying in an eternal loop:
$ sleep 5s & $ perf record -p `pidof sleep`
--- tools/perf/builtin-record.c | 3 +++ 1 file changed, 3 insertions(+)
tools/perf/builtin-record.c +3 -0
@@ -467,6 +467,9 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)if (err >0 || (err <0 && errno ==EINTR )) err =0 ; waking++; + +if (perf_evlist__filter_pollfd (rec->evlist,POLLERR |POLLHUP ) ==0 ) + done =1 ; }/*
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 6/14 perf trace: Filter out POLLHUP'ed file descriptors
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
So that we don't continue polling on vanished file descriptors, i.e. file descriptors for events monitoring threads that exited.
I.e. the following 'trace' command now exits as expected, instead of staying in an eternal loop:
$ sleep 5s &
$ trace -p `pidof sleep`--- tools/perf/builtin-trace.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tools/perf/builtin-trace.c +2 -1
@@ -2171,7 +2171,8 @@ next_event:if (trace->nr_events == before) {int timeout = done ?100 :-1 ; -if (poll (evlist->pollfd, evlist->nr_fds, timeout) >0 ) +if (poll (evlist->pollfd, evlist->nr_fds, timeout) >0 && +perf_evlist__filter_pollfd (evlist,POLLERR |POLLHUP ) >0 ) goto again; }else { goto again;
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 7/14 perf evlist: Allow growing pollfd on add method
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
This way we will be able to add more file descriptors to be polled, like stdin or some timer fd.
At this point we might as well yank the pollfd class from evlist so that it can be used in other places.
--- tools/perf/util/evlist.c | 38 +++++++++++++++++++++++++++++++++----- tools/perf/util/evlist.h | 5 +++-- 2 files changed, 36 insertions(+), 7 deletions(-)
tools/perf/util/evlist.c +33 -5
@@ -402,7 +402,21 @@ int perf_evlist__enable_event_idx(struct perf_evlist *evlist,return perf_evlist__enable_event_thread (evlist, evsel, idx); } -static int perf_evlist__alloc_pollfd (struct perf_evlist *evlist) +static int perf_evlist__grow_pollfd (struct perf_evlist *evlist,int hint) +{ +int nr_fds_alloc = evlist->nr_fds_alloc + hint; +size_t size =sizeof (struct pollfd ) * nr_fds_alloc; +struct pollfd *pollfd =realloc (evlist->pollfd, size); + +if (pollfd ==NULL ) +return -ENOMEM ; + + evlist->nr_fds_alloc = nr_fds_alloc; + evlist->pollfd = pollfd; +return 0 ; +} + +int perf_evlist__alloc_pollfd (struct perf_evlist *evlist) {int nr_cpus =cpu_map__nr (evlist->cpus);int nr_threads =thread_map__nr (evlist->threads);
@@ -416,16 +430,28 @@ static int perf_evlist__alloc_pollfd(struct perf_evlist *evlist) nfds += nr_cpus * nr_threads; } - evlist->pollfd =malloc (sizeof (struct pollfd ) * nfds); -return evlist->pollfd !=NULL ?0 : -ENOMEM ; +if (evlist->nr_fds_alloc - evlist->nr_fds < nfds && +perf_evlist__grow_pollfd (evlist, nfds) <0 ) +return -ENOMEM ; + +return 0 ; } -void perf_evlist__add_pollfd (struct perf_evlist *evlist,int fd) +int perf_evlist__add_pollfd (struct perf_evlist *evlist,int fd) { +/* +* XXX: 64 is arbitrary, just not to call realloc at each fd. +* Find a better autogrowing heuristic +*/ +if (evlist->nr_fds == evlist->nr_fds_alloc && +perf_evlist__grow_pollfd (evlist,64 ) <0 ) +return -ENOMEM ; +fcntl (fd,F_SETFL ,O_NONBLOCK ); evlist->pollfd[evlist->nr_fds].fd = fd; evlist->pollfd[evlist->nr_fds].events =POLLIN |POLLERR |POLLHUP ; evlist->nr_fds++; +return 0 ; }int perf_evlist__filter_pollfd (struct perf_evlist *evlist,short revents_and_mask)
@@ -717,6 +743,7 @@ static int __perf_evlist__mmap(struct perf_evlist *evlist, int idx, evlist->mmap[idx].base =NULL ;return -1 ; } +return 0 ; }
@@ -743,7 +770,8 @@ static int perf_evlist__mmap_per_evsel(struct perf_evlist *evlist, int idx,return -1 ; } -perf_evlist__add_pollfd (evlist, fd); +if (perf_evlist__add_pollfd (evlist, fd) <0 ) +return -1 ;if ((evsel->attr.read_format &PERF_FORMAT_ID ) &&perf_evlist__id_add_fd (evlist, evsel, cpu, thread, fd) <0 )
tools/perf/util/evlist.h +3 -2
@@ -30,6 +30,7 @@ struct perf_evlist {int nr_entries;int nr_groups;int nr_fds; +int nr_fds_alloc;int nr_mmaps;size_t mmap_len;int id_pos;
@@ -82,8 +83,8 @@ perf_evlist__find_tracepoint_by_name(struct perf_evlist *evlist,void perf_evlist__id_add (struct perf_evlist *evlist,struct perf_evsel *evsel,int cpu,int thread,u64 id); -void perf_evlist__add_pollfd (struct perf_evlist *evlist,int fd); - +int perf_evlist__add_pollfd (struct perf_evlist *evlist,int fd); +int perf_evlist__alloc_pollfd (struct perf_evlist *evlist);int perf_evlist__filter_pollfd (struct perf_evlist *evlist,short revents_and_mask);struct perf_evsel *perf_evlist__id2evsel (struct perf_evlist *evlist,u64 id);
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 8/14 perf tests: Add pollfd growing test
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
[acme@ssdandy linux]$ perf test "Add fd"
34: Add fd to pollfd array, making it autogrow : Ok
[acme@ssdandy linux]$ perf test -v "Add fd"
34: Add fd to pollfd array, making it autogrow :
--- start ---
test child forked, pid 19817
before growing array: 2 [ 1, 2 ] after 3rd add_pollfd: 3 [ 1, 2, 35 ] after 4th add_pollfd: 4 [ 1, 2, 35, 88 ]
test child finished with 0
---- end ----
Add fd to pollfd array, making it autogrow: Ok [acme@ssdandy linux]$
--- tools/perf/tests/builtin-test.c | 4 ++ tools/perf/tests/evlist.c | 111 ++++++++++++++++++++++++++++++++++++++++ tools/perf/tests/tests.h | 1 + 3 files changed, 116 insertions(+)
tools/perf/tests/builtin-test.c +4 -0
@@ -162,6 +162,10 @@ static struct test { .func = test__perf_evlist__filter_pollfd, }, { + .desc = "Add fd to pollfd array, making it autogrow", + .func = test__perf_evlist__add_pollfd, + }, + { .func = NULL, }, };
tools/perf/tests/evlist.c +111 -0
119 lines not on this page · raw message
tools/perf/tests/tests.h +1 -0
@@ -50,6 +50,7 @@ int test__hists_output(void);int test__hists_cumulate (void );int test__switch_tracking (void );int test__perf_evlist__filter_pollfd (void ); +int test__perf_evlist__add_pollfd (void );#if defined(__x86_64__) || defined(__i386__) || defined(__arm__)#ifdef HAVE_DWARF_UNWIND_SUPPORT
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 9/14 perf kvm stat live: Use perf_evlist__add_pollfd() instead of local equivalent
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
Since we can add file descriptors to the evlist pollfd and it will autogrow, no need to copy all events to a local pollfd array, just add the timer and stdin file descriptors.
--- tools/perf/builtin-kvm.c | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-)
tools/perf/builtin-kvm.c +9 -13
@@ -919,15 +919,8 @@ static int kvm_events_live_report(struct perf_kvm_stat *kvm)signal (SIGINT , sig_handler);signal (SIGTERM , sig_handler); -/* copy pollfds -- need to add timerfd and stdin */ +/* use pollfds -- need to add timerfd and stdin */ nr_fds = kvm->evlist->nr_fds; - pollfds =zalloc (sizeof (struct pollfd ) * (nr_fds +2 )); -if (!pollfds) { - err = -ENOMEM ; - goto out; - } -memcpy (pollfds, kvm->evlist->pollfd, -sizeof (struct pollfd ) * kvm->evlist->nr_fds);/* add timer fd */ if (perf_kvm__timerfd_create (kvm) <0 ) {
@@ -935,17 +928,21 @@ static int kvm_events_live_report(struct perf_kvm_stat *kvm) goto out; } - pollfds[nr_fds].fd = kvm->timerfd; - pollfds[nr_fds].events =POLLIN ; +if (perf_evlist__add_pollfd (kvm->evlist, kvm->timerfd)) + goto out; + nr_fds++; - pollfds[nr_fds].fd =fileno (stdin); - pollfds[nr_fds].events =POLLIN ; +if (perf_evlist__add_pollfd (kvm->evlist,fileno (stdin))) + goto out; + nr_stdin = nr_fds; nr_fds++;if (fd_set_nonblock (fileno (stdin)) !=0 ) goto out; + pollfds = kvm->evlist->pollfd; +/* everything is good - enable the events and process */ perf_evlist__enable (kvm->evlist);
@@ -979,7 +976,6 @@ out:close (kvm->timerfd);tcsetattr (0 ,TCSAFLUSH , &save); -free (pollfds);return err; }
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 10/14 perf evlist: Introduce poll method for common code idiom
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
Since we have access two evlist members in all these poll calls, provide a helper.
This will also help to make the patch introducing the pollfd class more clear, as the evlist specific uses will be hiden away perf_evlist__poll().
--- tools/perf/builtin-record.c | 2 +- tools/perf/builtin-top.c | 4 ++-- tools/perf/builtin-trace.c | 2 +- tools/perf/tests/open-syscall-tp-fields.c | 2 +- tools/perf/tests/perf-record.c | 2 +- tools/perf/tests/task-exit.c | 2 +- tools/perf/util/evlist.c | 5 +++++ tools/perf/util/evlist.h | 2 ++ tools/perf/util/python.c | 2 +- 9 files changed, 15 insertions(+), 8 deletions(-)
tools/perf/builtin-record.c +1 -1
@@ -459,7 +459,7 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)if (hits == rec->samples) {if (done)break ; - err =poll (rec->evlist->pollfd, rec->evlist->nr_fds,-1 ); + err =perf_evlist__poll (rec->evlist,-1 );/* * Propagate error, only if there's any. Ignore positive * number of returned events and interrupt error.
tools/perf/builtin-top.c +2 -2
@@ -964,7 +964,7 @@ static int __cmd_top(struct perf_top *top)perf_evlist__enable (top->evlist);/* Wait for a minimal set of events before starting the snapshot */ -poll (top->evlist->pollfd, top->evlist->nr_fds,100 ); +perf_evlist__poll (top->evlist,100 );perf_top__mmap_read (top);
@@ -991,7 +991,7 @@ static int __cmd_top(struct perf_top *top)perf_top__mmap_read (top);if (hits == top->samples) - ret =poll (top->evlist->pollfd, top->evlist->nr_fds,100 ); + ret =perf_evlist__poll (top->evlist,100 ); } ret =0 ;
tools/perf/builtin-trace.c +1 -1
@@ -2171,7 +2171,7 @@ next_event:if (trace->nr_events == before) {int timeout = done ?100 :-1 ; -if (poll (evlist->pollfd, evlist->nr_fds, timeout) >0 && +if (perf_evlist__poll (evlist, timeout) >0 &&perf_evlist__filter_pollfd (evlist,POLLERR |POLLHUP ) >0 ) goto again; }else {
tools/perf/tests/open-syscall-tp-fields.c +1 -1
@@ -105,7 +105,7 @@ int test__syscall_open_tp_fields(void) }if (nr_events == before) -poll (evlist->pollfd, evlist->nr_fds,10 ); +perf_evlist__poll (evlist,10 );if (++nr_polls >5 ) {pr_debug ("%s: no events!\n" , __func__);
tools/perf/tests/perf-record.c +1 -1
@@ -268,7 +268,7 @@ int test__PERF_RECORD(void)* perf_event_attr.wakeup_events, just PERF_EVENT_SAMPLE does. */ if (total_events == before && false) -poll (evlist->pollfd, evlist->nr_fds,-1 ); +perf_evlist__poll (evlist,-1 );sleep (1 );if (++wakeups >5 ) {
tools/perf/tests/task-exit.c +1 -1
@@ -105,7 +105,7 @@ retry: }if (!exited || !nr_exit) { -poll (evlist->pollfd, evlist->nr_fds,-1 ); +perf_evlist__poll (evlist,-1 ); goto retry; }
tools/perf/util/evlist.c +5 -0
@@ -475,6 +475,11 @@ int perf_evlist__filter_pollfd(struct perf_evlist *evlist, short revents_and_masreturn nr_fds; } +int perf_evlist__poll (struct perf_evlist *evlist,int timeout) +{ +return poll (evlist->pollfd, evlist->nr_fds, timeout); +} +static void perf_evlist__id_hash (struct perf_evlist *evlist,struct perf_evsel *evsel,int cpu,int thread,u64 id)
tools/perf/util/evlist.h +2 -0
@@ -87,6 +87,8 @@ int perf_evlist__add_pollfd(struct perf_evlist *evlist, int fd);int perf_evlist__alloc_pollfd (struct perf_evlist *evlist);int perf_evlist__filter_pollfd (struct perf_evlist *evlist,short revents_and_mask); +int perf_evlist__poll (struct perf_evlist *evlist,int timeout); +struct perf_evsel *perf_evlist__id2evsel (struct perf_evlist *evlist,u64 id);struct perf_sample_id *perf_evlist__id2sid (struct perf_evlist *evlist,u64 id);
tools/perf/util/python.c +1 -1
@@ -736,7 +736,7 @@ static PyObject *pyrf_evlist__poll(struct pyrf_evlist *pevlist,if (!PyArg_ParseTupleAndKeywords (args, kwargs,"|i" , kwlist, &timeout))return NULL ; - n =poll (evlist->pollfd, evlist->nr_fds, timeout); + n =perf_evlist__poll (evlist, timeout);if (n <0 ) {PyErr_SetFromErrno (PyExc_OSError);return NULL ;
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 11/14 tools lib api: Adopt fdarray class from perf's evlist
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
The extensible file description array that grew in the perf_evlist class can be useful for other tools, as it is not something that only evlists need, so move it to tools/lib/api/fd to ease sharing it.
v2: Don't use {} like in:
libapi_dirs:
$(QUIET_MKDIR)mkdir -p $(OUTPUT){fs,fd}/in Makefiles, as it will not work in some systems, as in ubuntu13.10.
--- tools/lib/api/Makefile | 7 ++- tools/lib/api/fd/array.c | 108 ++++++++++++++++++++++++++++++++++++++++++++++ tools/lib/api/fd/array.h | 32 ++++++++++++++ tools/perf/builtin-kvm.c | 4 +- tools/perf/tests/evlist.c | 67 ++++++++++++++-------------- tools/perf/util/evlist.c | 57 ++++-------------------- tools/perf/util/evlist.h | 5 +-- tools/perf/util/python.c | 4 +- 8 files changed, 195 insertions(+), 89 deletions(-) create mode 100644 tools/lib/api/fd/array.c create mode 100644 tools/lib/api/fd/array.h
tools/lib/api/Makefile +6 -1
@@ -10,9 +10,14 @@ LIB_OBJS= LIB_H += fs/debugfs.h LIB_H += fs/fs.h +# See comment below about piggybacking... +LIB_H += fd/array.h LIB_OBJS += $(OUTPUT)fs/debugfs.o LIB_OBJS += $(OUTPUT)fs/fs.o +# XXX piggybacking here, need to introduce libapikfd, or rename this +# to plain libapik.a and make it have it all api goodies +LIB_OBJS += $(OUTPUT)fd/array.o LIBFILE = libapikfs.a
@@ -29,7 +34,7 @@ $(LIBFILE): $(LIB_OBJS) $(LIB_OBJS): $(LIB_H) libapi_dirs: - $(QUIET_MKDIR)mkdir -p $(OUTPUT)fs/ + $(QUIET_MKDIR)mkdir -p $(OUTPUT)fd $(OUTPUT)fs $(OUTPUT)%.o: %.c libapi_dirs $(QUIET_CC)$(CC) -o $@ -c $(ALL_CFLAGS) $<
tools/lib/api/fd/array.c new file +108 -0
108 lines not on this page · raw message
tools/lib/api/fd/array.h new file +32 -0
@@ -0,0 +1,32 @@ +#ifndef __API_FD_ARRAY__ +#define __API_FD_ARRAY__ + +#include <stdio.h> + +struct pollfd ; + +struct fdarray { +int nr; +int nr_alloc; +int nr_autogrow; +struct pollfd *entries; +}; + +void fdarray__init (struct fdarray *fda,int nr_autogrow); +void fdarray__exit (struct fdarray *fda); + +struct fdarray *fdarray__new (int nr_alloc,int nr_autogrow); +void fdarray__delete (struct fdarray *fda); + +int fdarray__add (struct fdarray *fda,int fd,short revents); +int fdarray__poll (struct fdarray *fda,int timeout); +int fdarray__filter (struct fdarray *fda,short revents); +int fdarray__grow (struct fdarray *fda,int extra); +int fdarray__fprintf (struct fdarray *fda,FILE *fp); + +static inline int fdarray__available_entries (struct fdarray *fda) +{ +return fda->nr_alloc - fda->nr; +} + +#endif /* __API_FD_ARRAY__ */
tools/perf/builtin-kvm.c +2 -2
@@ -920,7 +920,7 @@ static int kvm_events_live_report(struct perf_kvm_stat *kvm)signal (SIGTERM , sig_handler);/* use pollfds -- need to add timerfd and stdin */ - nr_fds = kvm->evlist->nr_fds; + nr_fds = kvm->evlist->pollfd.nr;/* add timer fd */ if (perf_kvm__timerfd_create (kvm) <0 ) {
@@ -941,7 +941,7 @@ static int kvm_events_live_report(struct perf_kvm_stat *kvm)if (fd_set_nonblock (fileno (stdin)) !=0 ) goto out; - pollfds = kvm->evlist->pollfd; + pollfds = kvm->evlist->pollfd.entries;/* everything is good - enable the events and process */ perf_evlist__enable (kvm->evlist);
tools/perf/tests/evlist.c +35 -32
154 lines not on this page · raw message
tools/perf/util/evlist.c +8 -49
101 lines not on this page · raw message
tools/perf/util/evlist.h +2 -3
@@ -2,6 +2,7 @@#define __PERF_EVLIST_H 1#include <linux/list.h> +#include <api/fd/array.h> #include <stdio.h> #include "../perf.h" #include "event.h"
@@ -29,8 +30,6 @@ struct perf_evlist {struct hlist_head heads[PERF_EVLIST__HLIST_SIZE ];int nr_entries;int nr_groups; -int nr_fds; -int nr_fds_alloc;int nr_mmaps;size_t mmap_len;int id_pos;
@@ -41,8 +40,8 @@ struct perf_evlist {pid_t pid; } workload;bool overwrite; +struct fdarray pollfd;struct perf_mmap *mmap; -struct pollfd *pollfd;struct thread_map *threads;struct cpu_map *cpus;struct perf_evsel *selected;
tools/perf/util/python.c +2 -2
@@ -753,9 +753,9 @@ static PyObject *pyrf_evlist__get_pollfd(struct pyrf_evlist *pevlist,PyObject *list =PyList_New (0 );int i; -for (i =0 ; i < evlist->nr_fds; ++i) { +for (i =0 ; i < evlist->pollfd.nr; ++i) {PyObject *file; -FILE *fp =fdopen (evlist->pollfd[i].fd,"r" ); +FILE *fp =fdopen (evlist->pollfd.entries[i].fd,"r" );if (fp ==NULL ) goto free_list;
Signature · 1 line
1.9.3
PATCH 1/1 tools lib fd array: Do not set fd as non blocking evlist
On Wed, Sep 10, 2014 at 11:08:46AM -0300, Arnaldo Carvalho de Melo wrote:
SNIP
Quoted 11 lines
(quoted diff) +11
-0+} + +void fdarray__exit(struct fdarray *fda) +{ + free(fda->entries); + fdarray__init(fda, 0); +} + +void fdarray__delete(struct fdarray *fda) +{ + fdarray__exit(fda);
Review
Quoted 10 lines
(quoted diff) +10
-0+ free(fda); +} + +int fdarray__add(struct fdarray *fda, int fd, short revents) +{ + if (fda->nr == fda->nr_alloc && + fdarray__grow(fda, fda->nr_autogrow) < 0) + return -ENOMEM; + + fcntl(fd, F_SETFL, O_NONBLOCK);
also I spot this one and couldn't think of reason for it, attached patch makes no behaviour difference for me..
I might be missing something, but I dont see any blocking operation in perf related data reads. The git log history says it was there since early days.
jirka
---
There's no reason for the current user of this API to have non blocking fds. Also it should be up to user to set this, not this object.
--- tools/lib/api/fd/array.c | 1 - 1 files changed, 0 insertions(+), 1 deletions(-)
tools/lib/api/fd/array.c +0 -1
@@ -78,7 +78,6 @@ int fdarray__add(struct fdarray *fda, int fd, short revents) fdarray__grow(fda, fda->nr_autogrow) < 0)return -ENOMEM ; -fcntl (fd,F_SETFL ,O_NONBLOCK ); fda->entries[fda->nr].fd = fd; fda->entries[fda->nr].events = revents; fda->nr++;
Signature · 1 line
1.7.7.6
Arnaldo Carvalho de Melo
maintainerRe: [PATCH] tools lib fd array: Do not set fd as non blocking evlist
Em Thu, Sep 11, 2014 at 05:09:43PM +0200, Jiri Olsa escreveu:
Quoted 20 lines · Arnaldo Carvalho de Melo
SNIP
Quoted 16 lines
(quoted diff) +16
-0+} + +void fdarray__exit(struct fdarray *fda) +{ + free(fda->entries); + fdarray__init(fda, 0); +} + +void fdarray__delete(struct fdarray *fda) +{ + fdarray__exit(fda); + free(fda); +} + +int fdarray__add(struct fdarray *fda, int fd, short revents) +{
Quoted 8 lines
Review
Quoted 5 lines
(quoted diff) +5
-0+ if (fda->nr == fda->nr_alloc && + fdarray__grow(fda, fda->nr_autogrow) < 0) + return -ENOMEM; + + fcntl(fd, F_SETFL, O_NONBLOCK);also I spot this one and couldn't think of reason for it, attached patch makes no behaviour difference for me..
Have to look why it is there, perhaps there is some changeset specifically made for this, will do some research...
Quoted 3 lines
I might be missing something, but I dont see any blocking operation in perf related data reads. The git log history says it was there since early days.
Oops, you did that research already, have you followed the history all the way to when this code lived in Documentation/ ?
But yes, I agree with your comment that this is something up to the users to do, not for a general purpose class as fdarray is set out to be.
I think the way to do this is to not move this fcntl when introducing the fdarray class, but instead leave it at the perf_evlist__add_pollfd() function, then, in a later patch, if we determine that it is not needed at all, nuke it, ok?
- Arnaldo
Quoted 39 lines
jirka
---
There's no reason for the current user of this API to have non blocking fds. Also it should be up to user to set this, not this object.Cc Adrian Hunter <[email protected]>Cc Arnaldo Carvalho de Melo <[email protected]>Cc Borislav Petkov <[email protected]>Cc Corey Ashford <[email protected]>Cc David Ahern <[email protected]>Cc Frederic Weisbecker <[email protected]>Cc Ingo Molnar <[email protected]>Cc Jean Pihet <[email protected]>Cc Namhyung Kim <[email protected]>Cc Paul Mackerras <[email protected]>Cc Peter Zijlstra <[email protected]>Signed-off-by Jiri Olsa <[email protected]>--- tools/lib/api/fd/array.c | 1 - 1 files changed, 0 insertions(+), 1 deletions(-)tools/lib/api/fd/array.c +0
-1@@ -78,7 +78,6 @@ int fdarray__add(struct fdarray *fda, int fd, short revents) fdarray__grow(fda, fda->nr_autogrow) < 0)return -ENOMEM ; -fcntl (fd,F_SETFL ,O_NONBLOCK ); fda->entries[fda->nr].fd = fd; fda->entries[fda->nr].events = revents; fda->nr++;Signature · 1 line
1.7.7.6
Re: [PATCH] tools lib fd array: Do not set fd as non blocking evlist
Em Thu, Sep 11, 2014 at 12:27:26PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 9 lines
Em Thu, Sep 11, 2014 at 05:09:43PM +0200, Jiri Olsa escreveu:
Quoted 4 lines
Review
Quoted 1 line
(quoted diff) +1
-0+ fcntl(fd, F_SETFL, O_NONBLOCK);also I spot this one and couldn't think of reason for it, attached patch makes no behaviour difference for me..
Have to look why it is there, perhaps there is some changeset specifically made for this, will do some research...
Quoted 6 lines
Quoted 3 lines
I might be missing something, but I dont see any blocking operation in perf related data reads. The git log history says it was there since early days.
Oops, you did that research already, have you followed the history all the way to when this code lived in
Documentation/?
Since day one, found using:
$ git log -p --follow tools/perf/builtin-top.c
commit e0143bad9dbf2a8fad4c5430562bceba196b66ea
Author: Ingo Molnar <[email protected]> Date: Mon Mar 23 21:29:59 2009 +0100
perf_counter: add sample user-space to Documentation/perf_counter/
Initial version of kerneltop.c and perfstat.c.
Documentation/perf_counter/kerneltop.c +0 -0
Ingo, do we really need that O_NONBLOCK thing there?
- Arnaldo
Re: [PATCH 11/14] tools lib api: Adopt fdarray class from perf's evlist
In reply to Arnaldo Carvalho de MeloOn Wed, Sep 10, 2014 at 11:08:46AM -0300, Arnaldo Carvalho de Melo wrote:
SNIP
Quoted 6 lines
tools/lib/api/fd/array.c new file +6
-0+if (entries ==NULL ) +return -ENOMEM ; + + fda->nr_alloc = nr_alloc; + fda->entries = entries; +return 0 ;
tools/lib/api/fd/array.c line 41
Quoted 10 lines
tools/lib/api/fd/array.c new file +10
-0+} + +struct fdarray *fdarray__new (int nr_alloc,int nr_autogrow) +{ +struct fdarray *fda =calloc (1 ,sizeof (*fda)); + +if (fda !=NULL ) { +if (fdarray__grow (fda, nr_alloc)) { +free (fda); + fda =NULL ;
you could use zfree
jirka
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 11/14] tools lib api: Adopt fdarray class from perf's evlist
Em Fri, Sep 12, 2014 at 02:58:22PM +0200, Jiri Olsa escreveu:
Quoted 8 lines · Arnaldo Carvalho de Melo
SNIP
Quoted 3 lines
tools/lib/api/fd/array.c new file +3
-0+struct fdarray *fdarray__new (int nr_alloc,int nr_autogrow) +{ +struct fdarray *fda =calloc (1 ,sizeof (*fda));
Quoted 6 lines
tools/lib/api/fd/array.c line 41
Quoted 4 lines
tools/lib/api/fd/array.c new file +4
-0+if (fda !=NULL ) { +if (fdarray__grow (fda, nr_alloc)) { +free (fda); + fda =NULL ;you could use zfree
Yeah, we could, if zfree had moved from tools/perf/util/ to tools/lib/, which is has not yet.
I thought about doing it now, but when doing some grepping in the kernel sources I realized that zfree is already taken by... zlib, so I'll leave this for later, when we figure out how to properly export this, perhaps renaming it to freez()?
- Arnaldo
Re: [PATCH 11/14] tools lib api: Adopt fdarray class from perf's evlist
Quoted 16 lines · Arnaldo Carvalho de Melo
Em Fri, Sep 12, 2014 at 02:58:22PM +0200, Jiri Olsa escreveu:
Quoted 8 lines · Arnaldo Carvalho de Melo
SNIP
Quoted 3 lines
tools/lib/api/fd/array.c new file +3
-0+struct fdarray *fdarray__new (int nr_alloc,int nr_autogrow) +{ +struct fdarray *fda =calloc (1 ,sizeof (*fda));Quoted 6 lines
tools/lib/api/fd/array.c line 41
Quoted 4 lines
tools/lib/api/fd/array.c new file +4
-0+if (fda !=NULL ) { +if (fdarray__grow (fda, nr_alloc)) { +free (fda); + fda =NULL ;you could use zfree
Quoted 7 lines
Yeah, we could, if zfree had moved from
tools/perf/util/totools/lib/, which is has not yet.I thought about doing it now, but when doing some grepping in the kernel sources I realized that zfree is already taken by... zlib, so I'll leave this for later, when we figure out how to properly export this, perhaps renaming it to
freez()?
aargh.. haven't realized it's not in libs ;-) freez sounds ok
jirka
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 11/14] tools lib api: Adopt fdarray class from perf's evlist
Em Fri, Sep 12, 2014 at 04:16:32PM +0200, Jiri Olsa escreveu:
Quoted 11 lines · Arnaldo Carvalho de Melo
Quoted 9 lines
Em Fri, Sep 12, 2014 at 02:58:22PM +0200, Jiri Olsa escreveu:
Quoted 5 lines · Arnaldo Carvalho de Melo
tools/lib/api/fd/array.c line 41
Quoted 2 lines
tools/lib/api/fd/array.c new file +2
-0+free (fda); + fda =NULL ;you could use zfree
Yeah, we could, if zfree had moved from
tools/perf/util/totools/lib/, which is has not yet.
Quoted 6 lines
Quoted 4 lines
I thought about doing it now, but when doing some grepping in the kernel sources I realized that zfree is already taken by... zlib, so I'll leave this for later, when we figure out how to properly export this, perhaps renaming it to
freez()?aargh.. haven't realized it's not in libs ;-) freez sounds ok
Yeah, to me as well, but then zfree() was such a nice pick and looked ok at the time :-)
I'm leaving this for later, different patchkit that would sweep thru tools/ and find this pattern, make it use freez() (or some other name), etc.
- Arnaldo
Re: [PATCH 11/14] tools lib api: Adopt fdarray class from perf's evlist
Quoted 6 lines · Arnaldo Carvalho de Melo
Yeah, to me as well, but then
zfree()was such a nice pick and looked ok at the time :-)I'm leaving this for later, different patchkit that would sweep thru
tools/and find this pattern, make it usefreez()(or some other name), etc.
freez() sounds like something gets frozen :-P and not freed.
Signature · 3 lines
Regards/Gruss,
Boris.
--Arnaldo Carvalho de Melo
maintainerRe: [PATCH 11/14] tools lib api: Adopt fdarray class from perf's evlist
Em Fri, Sep 12, 2014 at 06:54:07PM +0200, Borislav Petkov escreveu:
Quoted 9 lines · Arnaldo Carvalho de Melo
Quoted 6 lines
Yeah, to me as well, but then
zfree()was such a nice pick and looked ok at the time :-)I'm leaving this for later, different patchkit that would sweep thru
tools/and find this pattern, make it usefreez()(or some other name), etc.
freez()sounds like something gets frozen :-P and not freed.
ok, note taken.
I liked it when trying something different from zfree().
I'm not that successfull at landgrabs recently, people at Fedora don't want to use the 'trace' alias for 'perf trace', even it not being used by anybody so far.
Ideas of how to name zfree()?
- Arnaldo
Re: [PATCH 11/14] tools lib api: Adopt fdarray class from perf's evlist
Quoted 3 lines · Arnaldo Carvalho de Melo
I'm not that successfull at landgrabs recently, people at Fedora don't want to use the 'trace' alias for 'perf trace', even it not being used by anybody so far.
WTF, it's just a name...
Quoted 1 line
Ideas of how to name
zfree()?
zo_free() for "free and zero out" maybe. I always try for the naming to still be small rather than spelled out.
Signature · 3 lines
Regards/Gruss,
Boris.
--Re: [PATCH 11/14] tools lib api: Adopt fdarray class from perf's evlist
In reply to Arnaldo Carvalho de MeloOn Wed, Sep 10, 2014 at 11:08:46AM -0300, Arnaldo Carvalho de Melo wrote:
SNIP
Quoted 13 lines
tools/lib/api/fd/array.c new file +13
-0+if (entries ==NULL ) +return -ENOMEM ; + + fda->nr_alloc = nr_alloc; + fda->entries = entries; +return 0 ; +} + +struct fdarray *fdarray__new (int nr_alloc,int nr_autogrow) +{ +struct fdarray *fda =calloc (1 ,sizeof (*fda)); + +if (fda !=NULL ) {
tools/lib/api/fd/array.c line 48
Quoted 10 lines
tools/lib/api/fd/array.c new file +10
-0+if (fdarray__grow (fda, nr_alloc)) { +free (fda); + fda =NULL ; + }else { + fda->nr_autogrow = nr_autogrow; + } + } + +return fda; +}
fdarray__new is never used..
jirka
Arnaldo Carvalho de Melo
maintainerPATCH 12/14 perf evlist: Refcount mmaps
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
We need to know how many fds are using a perf mmap via PERF_EVENT_IOC_SET_OUTPUT, so that we can know when to ditch an mmap, refcount it.
--- tools/perf/util/evlist.c | 21 ++++++++++++++++++++- tools/perf/util/evlist.h | 6 ++++++ 2 files changed, 26 insertions(+), 1 deletion(-)
tools/perf/util/evlist.c +20 -1
@@ -664,6 +664,7 @@ static void __perf_evlist__munmap(struct perf_evlist *evlist, int idx)if (evlist->mmap[idx].base !=NULL ) {munmap (evlist->mmap[idx].base, evlist->mmap_len); evlist->mmap[idx].base =NULL ; + evlist->mmap[idx].nfds =0 ; } }
@@ -697,6 +698,7 @@ struct mmap_params {static int __perf_evlist__mmap (struct perf_evlist *evlist,int idx,struct mmap_params *mp,int fd) { + evlist->mmap[idx].nfds =1 ; evlist->mmap[idx].prev =0 ; evlist->mmap[idx].mask = mp->mask; evlist->mmap[idx].base = mmap(NULL, evlist->mmap_len, mp->prot,
@@ -711,6 +713,19 @@ static int __perf_evlist__mmap(struct perf_evlist *evlist, int idx,return 0 ; } +static void perf_evlist__mmap_get (struct perf_evlist *evlist,int idx) +{ + ++evlist->mmap[idx].nfds; +} + +static void perf_evlist__mmap_put (struct perf_evlist *evlist,int idx) +{ +BUG_ON (evlist->mmap[idx].nfds ==0 ); + +if (--evlist->mmap[idx].nfds ==0 ) +__perf_evlist__munmap (evlist, idx); +} +static int perf_evlist__mmap_per_evsel (struct perf_evlist *evlist,int idx,struct mmap_params *mp,int cpu,int thread,int *output)
@@ -732,10 +747,14 @@ static int perf_evlist__mmap_per_evsel(struct perf_evlist *evlist, int idx, }else {if (ioctl (fd,PERF_EVENT_IOC_SET_OUTPUT , *output) !=0 )return -1 ; + +perf_evlist__mmap_get (evlist, idx); } -if (perf_evlist__add_pollfd (evlist, fd) <0 ) +if (perf_evlist__add_pollfd (evlist, fd) <0 ) { +perf_evlist__mmap_put (evlist, idx);return -1 ; + }if ((evsel->attr.read_format &PERF_FORMAT_ID ) &&perf_evlist__id_add_fd (evlist, evsel, cpu, thread, fd) <0 )
tools/perf/util/evlist.h +6 -0
@@ -18,9 +18,15 @@ struct record_opts;#define PERF_EVLIST__HLIST_BITS 8#define PERF_EVLIST__HLIST_SIZE (1 << PERF_EVLIST__HLIST_BITS) +/** +* struct perf_mmap - perf's ring buffer mmap details +* +* @nfds - file descriptors pointing here via PERF_EVENT_IOC_SET_OUTPUT +*/ struct perf_mmap {void *base;int mask; +int nfds;unsigned int prev;char event_copy[PERF_SAMPLE_MAX_SIZE ]; };
Signature · 1 line
1.9.3
Arnaldo Carvalho de Melo
maintainerPATCH 13/14 tools lib fd array: Allow associating an integer cookie with each entry
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
We will use this in perf's evlist class so that it can, at fdarray__filter() time, to unmap the associated ring buffer.
We may need to have further info associated with each fdarray entry, in that case we'll make that int array a 'union fdarray_priv' one and put a pointer there so that users can stash whatever they want there. For now, an int is enough tho.
--- tools/lib/api/fd/array.c | 28 ++++++++++++++++++++++++---- tools/lib/api/fd/array.h | 4 +++- tools/perf/util/evlist.c | 2 +- 3 files changed, 28 insertions(+), 6 deletions(-)
tools/lib/api/fd/array.c +24 -4
@@ -13,21 +13,31 @@void fdarray__init (struct fdarray *fda,int nr_autogrow) { fda->entries =NULL ; + fda->priv =NULL ; fda->nr = fda->nr_alloc =0 ; fda->nr_autogrow = nr_autogrow; }int fdarray__grow (struct fdarray *fda,int nr) { +void *priv;int nr_alloc = fda->nr_alloc + nr; +size_t psize =sizeof (fda->priv[0 ]) * nr_alloc;size_t size =sizeof (struct pollfd ) * nr_alloc;struct pollfd *entries =realloc (fda->entries, size);if (entries ==NULL )return -ENOMEM ; + priv =realloc (fda->priv, psize); +if (priv ==NULL ) { +free (entries); +return -ENOMEM ; + } + fda->nr_alloc = nr_alloc; fda->entries = entries; + fda->priv = priv;return 0 ; }
@@ -50,6 +60,7 @@ struct fdarray *fdarray__new(int nr_alloc, int nr_autogrow)void fdarray__exit (struct fdarray *fda) {free (fda->entries); +free (fda->priv);fdarray__init (fda,0 ); }
@@ -61,6 +72,8 @@ void fdarray__delete(struct fdarray *fda)int fdarray__add (struct fdarray *fda,int fd,short revents) { +int pos = fda->nr; +if (fda->nr == fda->nr_alloc &&fdarray__grow (fda, fda->nr_autogrow) <0 )return -ENOMEM ;
@@ -69,10 +82,11 @@ int fdarray__add(struct fdarray *fda, int fd, short revents) fda->entries[fda->nr].fd = fd; fda->entries[fda->nr].events = revents; fda->nr++; -return 0 ; +return pos; } -int fdarray__filter (struct fdarray *fda,short revents) +int fdarray__filter (struct fdarray *fda,short revents, +void (*entry_destructor)(struct fdarray *fda,int fd)) {int fd, nr =0 ;
@@ -80,11 +94,17 @@ int fdarray__filter(struct fdarray *fda, short revents)return 0 ;for (fd =0 ; fd < fda->nr; ++fd) { -if (fda->entries[fd].revents & revents) +if (fda->entries[fd].revents & revents) { +if (entry_destructor) +entry_destructor (fda, fd); +continue ; + } -if (fd != nr) +if (fd != nr) { fda->entries[nr] = fda->entries[fd]; + fda->priv[nr] = fda->priv[fd]; + } ++nr; }
tools/lib/api/fd/array.h +3 -1
@@ -10,6 +10,7 @@ struct fdarray {int nr_alloc;int nr_autogrow;struct pollfd *entries; +int *priv; };void fdarray__init (struct fdarray *fda,int nr_autogrow);
@@ -20,7 +21,8 @@ void fdarray__delete(struct fdarray *fda);int fdarray__add (struct fdarray *fda,int fd,short revents);int fdarray__poll (struct fdarray *fda,int timeout); -int fdarray__filter (struct fdarray *fda,short revents); +int fdarray__filter (struct fdarray *fda,short revents, +void (*entry_destructor)(struct fdarray *fda,int fd));int fdarray__grow (struct fdarray *fda,int extra);int fdarray__fprintf (struct fdarray *fda,FILE *fp);
tools/perf/util/evlist.c +1 -1
@@ -431,7 +431,7 @@ int perf_evlist__add_pollfd(struct perf_evlist *evlist, int fd)int perf_evlist__filter_pollfd (struct perf_evlist *evlist,short revents_and_mask) { -return fdarray__filter (&evlist->pollfd, revents_and_mask); +return fdarray__filter (&evlist->pollfd, revents_and_mask,NULL ); }int perf_evlist__poll (struct perf_evlist *evlist,int timeout)
Signature · 1 line
1.9.3
Re: [PATCH 13/14] tools lib fd array: Allow associating an integer cookie with each entry
On Wed, Sep 10, 2014 at 11:08:48AM -0300, Arnaldo Carvalho de Melo wrote:
SNIP
Quoted 9 lines
}
tools/lib/api/fd/array.c +2
-1-int fdarray__filter (struct fdarray *fda,short revents) +int fdarray__filter (struct fdarray *fda,short revents, +void (*entry_destructor)(struct fdarray *fda,int fd)) {int fd, nr =0 ;
tools/lib/api/fd/array.c int fdarray__filter(struct fdarray *fda, short revents) · line 99
Quoted 7 lines
tools/lib/api/fd/array.c +3
-1@@ -80,11 +94,17 @@ int fdarray__filter(struct fdarray *fda, short revents)return 0 ;for (fd =0 ; fd < fda->nr; ++fd) { -if (fda->entries[fd].revents & revents) +if (fda->entries[fd].revents & revents) { +if (entry_destructor) +entry_destructor (fda, fd);
the desctructor callback could have the 'priv' as an argument so we dont need to touch fdarray internals in upper layer
Quoted 17 lines
tools/lib/api/fd/array.c +5
-1+continue ; + } -if (fd != nr) +if (fd != nr) { fda->entries[nr] = fda->entries[fd]; + fda->priv[nr] = fda->priv[fd]; + } ++nr; }tools/lib/api/fd/array.h +0
-0
tools/lib/api/fd/array.h struct fdarray { · line 13
Quoted 4 lines
tools/lib/api/fd/array.h +1
-0@@ -10,6 +10,7 @@ struct fdarray {int nr_alloc;int nr_autogrow;struct pollfd *entries; +int *priv;
I like the idea of private pointer for each fd, but I think it should be 'void*' to keep the library generic. The 'int*' is related only to the evlist usage of this.
jirka
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 13/14] tools lib fd array: Allow associating an integer cookie with each entry
Em Thu, Sep 11, 2014 at 12:33:19PM +0200, Jiri Olsa escreveu:
Quoted 1 line · Arnaldo Carvalho de Melo
SNIP
Quoted 9 lines
tools/lib/api/fd/array.c int fdarray__filter(struct fdarray *fda, short revents) · line 97
Quoted 6 lines
tools/lib/api/fd/array.c +3
-1@@ -80,11 +94,17 @@ int fdarray__filter(struct fdarray *fda, short revents)for (fd =0 ; fd < fda->nr; ++fd) { -if (fda->entries[fd].revents & revents) +if (fda->entries[fd].revents & revents) { +if (entry_destructor) +entry_destructor (fda, fd);the desctructor callback could have the 'priv' as an argument so we dont need to touch fdarray internals in upper layer
I thought about that, but then I would have to pass as well the pollfd entry, that is not in priv, i.e. the callback may want to know exactly what were the POLL{RD,WR,HUP,ER) in fdarray->entryes[fd].revents.
Yeah, sometimes it is good to avoid users touching the internals, but that doesn't need to be something set in stone, i.e. we can consider fda->revents and priv->priv to "public" :-)
SNIP
Quoted 8 lines
tools/lib/api/fd/array.h struct fdarray { · line 11
Quoted 4 lines
tools/lib/api/fd/array.h +2
-0+++ b/tools/lib/api/fd/array.h@@ -10,6 +10,7 @@ struct fdarray {struct pollfd *entries; +int *priv;I like the idea of private pointer for each fd, but I think it should be 'void*' to keep the library generic. The 'int*' is related only to the evlist usage of this.
Right, from the changelog comment for this patch:
------------------------------
We may need to have further info associated with each fdarray entry,
in that case we'll make that int array a 'union fdarray_priv' one and put a pointer there so that users can stash whatever they want there. For now, an int is enough tho.
------------------------------
I had it as:
struct fdarray {
...
union {
int idx;
void *ptr;
} priv;
...
}But then I had no use whatsoever for the ->ptr one at this point, so I just nuked it, to keep _just_ what is used _right now_, and added the comment to the changelog :-)
Heck, even the comment mentions "union fdarray_priv", because at one point I was passing that to the destructor, as you suggested, but then I removed it to an unnamed union, because passing just the index would allow access to ->priv[] and ->entries[].
- Arnaldo
Re: [PATCH 13/14] tools lib fd array: Allow associating an integer cookie with each entry
On Thu, Sep 11, 2014 at 10:29:11AM -0300, Arnaldo Carvalho de Melo wrote:
SNIP
Quoted 24 lines
Quoted 3 lines
I like the idea of private pointer for each fd, but I think it should be 'void*' to keep the library generic. The 'int*' is related only to the evlist usage of this.
Right, from the changelog comment for this patch:
------------------------------
We may need to have further info associated with each fdarray entry,
in that case we'll make that int array a 'unionfdarray_priv' one and put a pointer there so that users can stash whatever they want there. For now, an int is enough tho.
------------------------------I had it as:
struct fdarray { ... union { int idx; void *ptr; } priv; ... }
Quoted 3 lines
But then I had no use whatsoever for the ->ptr one at this point, so I just nuked it, to keep _just_ what is used
_rightnow_, and added the comment to the changelog :-)
if we are treeting tools/lib/api as 'external' lib, I think we should use 'void *' for priv and let the user retype it to whatever he wants
but I dont care/insist here too much.. it just seems strange to me ;-)
jirka
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 13/14] tools lib fd array: Allow associating an integer cookie with each entry
Em Thu, Sep 11, 2014 at 04:59:45PM +0200, Jiri Olsa escreveu:
Quoted 9 lines · Arnaldo Carvalho de Melo
Quoted 3 lines
But then I had no use whatsoever for the ->ptr one at this point, so I just nuked it, to keep _just_ what is used
_rightnow_, and added the comment to the changelog :-)if we are treeting
tools/lib/apias 'external' lib, I think we should use 'void *' for priv and let the user retype it to whatever he wantsbut I dont care/insist here too much.. it just seems strange to me ;-)
Ok, so perhaps it should be:
struct fdarray {
...
union {
int idx;
} *priv;
...
}And then perf's evlist will use:
fda->priv[fd].idx; Then, when we get the need for a pointer, we can go there and
add it to the union.
This will make tools/lib/api/fd/ a little bit more friendly to
use outside the kernel sources, since then a new version will be source code compatible (but not necessarily binary compatible).
If we commit to that we can't just go on and change all its
users as we can now do for things like 'struct sk_buff', 'struct sock' and any other kernel data structures.
There are costs in making it 'external' in the sense you're
implying. I want to make it a bit less 'external' (in your sense), at least while we initially implement new stuff like this one.
My preference is for us to move things to tools/lib/ and
initially support use of it for tools that live in the kernel sources.
If and when people find it useful for things outside the kernel
sources, if we think that the code is perfect (ha ha) we can add some comment saying that that particular source file can't be changed in a way that will break out-of-the-kernel-tree sources.
- Arnaldo
Re: [PATCH 13/14] tools lib fd array: Allow associating an integer cookie with each entry
Quoted 23 lines · Arnaldo Carvalho de Melo
Em Thu, Sep 11, 2014 at 04:59:45PM +0200, Jiri Olsa escreveu:
Quoted 9 lines · Arnaldo Carvalho de Melo
Quoted 3 lines
But then I had no use whatsoever for the ->ptr one at this point, so I just nuked it, to keep _just_ what is used
_rightnow_, and added the comment to the changelog :-)if we are treeting
tools/lib/apias 'external' lib, I think we should use 'void *' for priv and let the user retype it to whatever he wantsbut I dont care/insist here too much.. it just seems strange to me ;-)
Ok, so perhaps it should be:
struct fdarray { ... union { int idx; } *priv; ... }And then perf's evlist will use:
Quoted 8 lines
fda->priv[fd].idx; Then, when we get the need for a pointer, we can go there andadd it to the union.
This will make
tools/lib/api/fd/a little bit more friendly to
use outside the kernel sources, since then a new version will be source code compatible (but not necessarily binary compatible).
you lost me ;-)
could you please make an example of the !compatibility you mentioned?
Quoted 6 lines
If we commit to that we can't just go on and change all its
users as we can now do for things like 'structsk_buff', 'struct sock' and any other kernel data structures.
There are costs in making it 'external' in the sense you're
implying. I want to make it a bit less 'external' (in your sense), at least while we initially implement new stuff like this one.
and the costs?
also I did not mean that union above, what I meant was:
struct fdarray {
...
void *priv;
...
}jirka
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 13/14] tools lib fd array: Allow associating an integer cookie with each entry
Em Thu, Sep 11, 2014 at 05:35:27PM +0200, Jiri Olsa escreveu:
Quoted 29 lines · Arnaldo Carvalho de Melo
Quoted 28 lines
Em Thu, Sep 11, 2014 at 04:59:45PM +0200, Jiri Olsa escreveu:
Quoted 9 lines · Arnaldo Carvalho de Melo
Quoted 3 lines
But then I had no use whatsoever for the ->ptr one at this point, so I just nuked it, to keep _just_ what is used
_rightnow_, and added the comment to the changelog :-)if we are treeting
tools/lib/apias 'external' lib, I think we should use 'void *' for priv and let the user retype it to whatever he wantsbut I dont care/insist here too much.. it just seems strange to me ;-)
Ok, so perhaps it should be:
struct fdarray { ... union { int idx; } *priv; ... }And then perf's evlist will use:
fda->priv[fd].idx; Then, when we get the need for a pointer, we can go there andadd it to the union.
Quoted 7 lines
Quoted 3 lines
This will make
tools/lib/api/fd/a little bit more friendly to
use outside the kernel sources, since then a new version will be source code compatible (but not necessarily binary compatible).you lost me ;-)
could you please make an example of the !compatibility you mentioned?
If I leave it as is now, when I add that void * pointer to each entry in fdarray->priv[], I will have to change fdarray's users, i.e. right now I will have to go and change evlist.h, i.e. every place that has:
fda->priv[fd]; will have to be changed to: fda->priv[fd].idx;
If I instead change it to:
struct fdarray {
...
union {
int idx;
} *priv;
...
}
Then I will not have to change anything when I add that void * pointer to the union.
Quoted 9 lines
Quoted 6 lines
If we commit to that we can't just go on and change all its
users as we can now do for things like 'structsk_buff', 'struct sock' and any other kernel data structures.
There are costs in making it 'external' in the sense you're
implying. I want to make it a bit less 'external' (in your sense), at least while we initially implement new stuff like this one.and the costs?
Quoted 7 lines
also I did not mean that union above, what I meant was:
struct fdarray { ... void *priv; ... }
Yeah, that is what I am trying to avoid. This will make it use sizeof(void *) for each entry, and I have no use for that right now. All I need is sizeof(int) for each entry.
Remember, this is an array, so its not just a matter of changing:
int *priv;
to:
void *priv;
But since it is an array, it will use, in this case, double the space, on 64-bit, for no gain _right now_.
- Arnaldo
Re: [PATCH 13/14] tools lib fd array: Allow associating an integer cookie with each entry
Quoted 58 lines · Arnaldo Carvalho de Melo
Em Thu, Sep 11, 2014 at 05:35:27PM +0200, Jiri Olsa escreveu:
Quoted 29 lines · Arnaldo Carvalho de Melo
Quoted 28 lines
Em Thu, Sep 11, 2014 at 04:59:45PM +0200, Jiri Olsa escreveu:
Quoted 9 lines · Arnaldo Carvalho de Melo
Quoted 3 lines
But then I had no use whatsoever for the ->ptr one at this point, so I just nuked it, to keep _just_ what is used
_rightnow_, and added the comment to the changelog :-)if we are treeting
tools/lib/apias 'external' lib, I think we should use 'void *' for priv and let the user retype it to whatever he wantsbut I dont care/insist here too much.. it just seems strange to me ;-)
Ok, so perhaps it should be:
struct fdarray { ... union { int idx; } *priv; ... }And then perf's evlist will use:
fda->priv[fd].idx; Then, when we get the need for a pointer, we can go there andadd it to the union.
Quoted 7 lines
Quoted 3 lines
This will make
tools/lib/api/fd/a little bit more friendly to
use outside the kernel sources, since then a new version will be source code compatible (but not necessarily binary compatible).you lost me ;-)
could you please make an example of the !compatibility you mentioned?
If I leave it as is now, when I add that void * pointer to each entry in fdarray->priv[], I will have to change fdarray's users, i.e. right now I will have to go and change
evlist.h, i.e. every place that has:fda->priv[fd]; will have to be changed to: fda->priv[fd].idx;If I instead change it to:
(quoted diff) +0
-0struct fdarray { union { int idx; } *priv;}
Quoted 2 lines
Then I will not have to change anything when I add that void * pointer to the union.
I dont think there's any trouble like that if you keep just void* priv pointer and let users to store there whatever they like
Quoted 18 lines
Quoted 10 lines
Quoted 7 lines
If we commit to that we can't just go on and change all its
users as we can now do for things like 'structsk_buff', 'struct sock' and any other kernel data structures.
There are costs in making it 'external' in the sense you're
implying. I want to make it a bit less 'external' (in your sense), at least while we initially implement new stuff like this one.and the costs?
Quoted 7 lines
also I did not mean that union above, what I meant was:
struct fdarray { ... void *priv; ... }
Quoted 3 lines
Yeah, that is what I am trying to avoid. This will make it use
sizeof(void *)for each entry, and I have no use for that right now. All I need is sizeof(int) for each entry.
hm? but you're using pointer (int*) not int .. size will be sizeof(unsigned long) anyway
jirka
Arnaldo Carvalho de Melo
maintainerPATCH 14/14 perf evlist: Unmap ring buffer when fd is nuked
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
As noticed by receiving a POLLHUP for its pollfd entry.
--- tools/perf/util/evlist.c | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-)
tools/perf/util/evlist.c +26 -3
@@ -25,6 +25,8 @@#include <linux/bitops.h> #include <linux/hash.h> +static void perf_evlist__mmap_put (struct perf_evlist *evlist,int idx); +#define FD (e, x, y) (*(int *)xyarray__entry(e->fd, x, y))#define SID (e, x, y) xyarray__entry(e->sample_id, x, y)
@@ -424,14 +426,35 @@ int perf_evlist__alloc_pollfd(struct perf_evlist *evlist)return 0 ; } +static int __perf_evlist__add_pollfd (struct perf_evlist *evlist,int fd,int idx) +{ +int pos =fdarray__add (&evlist->pollfd, fd,POLLIN |POLLERR |POLLHUP ); +/* +* Save the idx so that when we filter out fds POLLHUP'ed we can +* close the associated evlist->mmap[] entry. +*/ +if (pos >=0 ) + evlist->pollfd.priv[pos] = idx; + +return pos; +} +int perf_evlist__add_pollfd (struct perf_evlist *evlist,int fd) { -return fdarray__add (&evlist->pollfd, fd,POLLIN |POLLERR |POLLHUP ); +return __perf_evlist__add_pollfd (evlist, fd,-1 ); +} + +static void perf_evlist__munmap_filtered (struct fdarray *fda,int fd) +{ + struct perf_evlist *evlist = container_of(fda, struct perf_evlist, pollfd); + +perf_evlist__mmap_put (evlist, fda->priv[fd]); }int perf_evlist__filter_pollfd (struct perf_evlist *evlist,short revents_and_mask) { -return fdarray__filter (&evlist->pollfd, revents_and_mask,NULL ); +return fdarray__filter (&evlist->pollfd, revents_and_mask, + perf_evlist__munmap_filtered); }int perf_evlist__poll (struct perf_evlist *evlist,int timeout)
@@ -751,7 +774,7 @@ static int perf_evlist__mmap_per_evsel(struct perf_evlist *evlist, int idx,perf_evlist__mmap_get (evlist, idx); } -if (perf_evlist__add_pollfd (evlist, fd) <0 ) { +if (__perf_evlist__add_pollfd (evlist, fd, idx) <0 ) {perf_evlist__mmap_put (evlist, idx);return -1 ; }
Signature · 1 line
1.9.3
Re: [PATCH 14/14] perf evlist: Unmap ring buffer when fd is nuked
On Wed, Sep 10, 2014 at 11:08:49AM -0300, Arnaldo Carvalho de Melo wrote:
SNIP
Review
Quoted 13 lines
(quoted diff) +10
-1+} + int perf_evlist__add_pollfd(struct perf_evlist *evlist, int fd) { - return fdarray__add(&evlist->pollfd, fd, POLLIN | POLLERR | POLLHUP); + return __perf_evlist__add_pollfd(evlist, fd, -1); +} + +static void perf_evlist__munmap_filtered(struct fdarray *fda, int fd) +{ + struct perf_evlist *evlist = container_of(fda, struct perf_evlist, pollfd); + + perf_evlist__mmap_put(evlist, fda->priv[fd]);
we cannot do this.. because of the way we read the map
getting error or hup does not mean the mmap is empty, it can still have data, which we loose if we unmap it
following test will get data only with attached patch:
---
term1:
$ cat
term2:
$ cat perf record -p `pgrep cat`
term1:
^D
---
we get poll READ notification based on the wattermart settings, which by default is half size of the ring buffer.. so for small amount of perf data we dont get the poll read notification
I think we need to handle this in the record command context and read out the mmap before we unmap it
jirka
Quoted 22 lines
}
int
perf_evlist__filter_pollfd(struct perf_evlist *evlist, short revents_and_mask){(quoted diff) +3
-2- return fdarray__filter(&evlist->pollfd, revents_and_mask, NULL); + return fdarray__filter(&evlist->pollfd, revents_and_mask, + perf_evlist__munmap_filtered); } int perf_evlist__poll(struct perf_evlist *evlist, int timeout)@@ -751,7 +774,7 @@ static int perf_evlist__mmap_per_evsel(struct perf_evlist *evlist, int idx, perf_evlist__mmap_get(evlist, idx); } - if (perf_evlist__add_pollfd(evlist, fd) < 0) { + if (__perf_evlist__add_pollfd(evlist, fd, idx) < 0) { perf_evlist__mmap_put(evlist, idx); return -1; }Signature · 1 line
1.9.3
---
tools/perf/util/evlist.c +1 -0
@@ -448,6 +448,7 @@ static void perf_evlist__munmap_filtered(struct fdarray *fda, int fd) { struct perf_evlist *evlist = container_of(fda, struct perf_evlist, pollfd); +if (0 )perf_evlist__mmap_put (evlist, fda->priv[fd]); }
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 14/14] perf evlist: Unmap ring buffer when fd is nuked
Em Thu, Sep 11, 2014 at 02:27:51PM +0200, Jiri Olsa escreveu:
Quoted 18 lines · Arnaldo Carvalho de Melo
SNIP
Review
Quoted 11 lines
int
perf_evlist__add_pollfd(struct perf_evlist *evlist, int fd){(quoted diff) +7
-1- return fdarray__add(&evlist->pollfd, fd, POLLIN | POLLERR | POLLHUP); + return __perf_evlist__add_pollfd(evlist, fd, -1); +} +static void perf_evlist__munmap_filtered(struct fdarray *fda, int fd) +{ + struct perf_evlist *evlist = container_of(fda, struct perf_evlist, pollfd); + + perf_evlist__mmap_put(evlist, fda->priv[fd]);we cannot do this.. because of the way we read the map
Quoted 2 lines
getting error or hup does not mean the mmap is empty, it can still have data, which we loose if we unmap it
Understood, good catch, so I think that since associating with the mmap is done automagically by the evlist class, it should continue, as I did, doing the mmap_put's to have it in pairs, and then we have two choices:
1. the user, i.e. the tool, does an extra perf_evlist__mmap_get() to make sure that it exits the loop with the mmaps in place, since it will have that extra refcount, and then drain the buffers one last time, then to the final put, or leave it there to be reaped unconditionally at perf_evlist__delete()
2. Do this all automagically in the evlist layer(), i.e. start the perf_mmap->nr_fds (that gets renamed, as this is no longer the number of fds, but just a generic refcount) at 2, and when confirming the mmap read, in perf_evlist__mmap_consume(), check if the refcount is 1 and if there are no more events, do the final mmap_put.
I think #2 is better, no?
I.e. tools remain as they are, just doing the filtering, that could even be renamed from perf_evlist__filter_pollfd() to perf_evlist__eof(), to use some well known TLA for "no more things to read". :-)
I.e. the less the tools are _required_ to know about how events are laid out and dealt with, concentrating just on _consuming_ those events, the better.
- Arnaldo
Quoted 60 lines
following test will get data only with attached patch:
---
term1:$ catterm2:
$ cat perf record -p `pgrep cat`term1:
^D
---we get poll READ notification based on the wattermart settings, which by default is half size of the ring buffer.. so for small amount of perf data we dont get the poll read notification
I think we need to handle this in the record command context and read out the mmap before we unmap it
jirka
Quoted 22 lines
}
int
perf_evlist__filter_pollfd(struct perf_evlist *evlist, short revents_and_mask){(quoted diff) +3
-2- return fdarray__filter(&evlist->pollfd, revents_and_mask, NULL); + return fdarray__filter(&evlist->pollfd, revents_and_mask, + perf_evlist__munmap_filtered); } int perf_evlist__poll(struct perf_evlist *evlist, int timeout)@@ -751,7 +774,7 @@ static int perf_evlist__mmap_per_evsel(struct perf_evlist *evlist, int idx, perf_evlist__mmap_get(evlist, idx); } - if (perf_evlist__add_pollfd(evlist, fd) < 0) { + if (__perf_evlist__add_pollfd(evlist, fd, idx) < 0) { perf_evlist__mmap_put(evlist, idx); return -1; }Signature · 1 line
1.9.3---
tools/perf/util/evlist.c +1
-0@@ -448,6 +448,7 @@ static void perf_evlist__munmap_filtered(struct fdarray *fda, int fd) { struct perf_evlist *evlist = container_of(fda, struct perf_evlist, pollfd); +if (0 )perf_evlist__mmap_put (evlist, fda->priv[fd]); }
Quoted 23 lines · Arnaldo Carvalho de Melo
From: Arnaldo Carvalho de Melo <[email protected]>
Hi,
Main different to what was discussed in v2[1]:
Poll all event file descriptors, not just the ones mmaped.
This is a bug that was present before, noticed by Adrian Hunter while
reviewing this patchkit, i.e. we were polling just the mmaped ones, because then we would look at all the ring buffers when just one of the polled descriptors was ready to read.But this could lead to delays in polling when one of the mmaped fds was
closed and we would then not be notified when another, associated to it by means ofPERF_EVENT_IOC_SET_OUTPUT, had events available.Also refcount the mmaps associated by means of
PERF_EVENT_IOC_SET_OUTPUT,
so that we can properly unmap a ring buffer to which all its file descriptors had been closed, for longer running apps.It is all available on my tree:
Quoted 8 lines
git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.gitperf/fdarray.v2I kept the v3 for the series as I initially called it pollfd, but since the
class that grew out of it was named fdarray, thus the fdarray.v2 branch name.Please let me know if there are still any other problems, now back to
fixing up my last perf/core pull req wrt those external proggies problems.
hum got this one:
[jolsa@krava perf]$ ./perf record ls
Error:
The sys_perf_event_open() syscall returned with 22 (Invalid argument) for event (cycles). /bin/dmesg may provide additional information.
No CONFIG_PERF_EVENTS=y kernel support configured?
*** Error in ./perf': free(): invalid next size (fast): 0x00000000020d2290 *** *** Error in ./perf': malloc(): memory corruption: 0x00000000020d22b0 ***
[jolsa@krava perf]$ pstack 20706
#0 pthread_once () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_once.S:94 #1 0x0000003408109d1c in __GI___backtrace (array=array@entry=0x7fff5c073d60, size=size@entry=64) at ../sysdeps/x86_64/backtrace.c:103 #2 0x0000003408075d34 in __libc_message (do_abort=2, fmt=fmt@entry=0x340817e568 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/unix/sysv/linux/libc_fatal.c:176 #3 0x000000340807e21c in malloc_printerr (ptr=0x20d22b0, str=0x340817bcbf "malloc(): memory corruption", action=<optimized out>) at malloc.c:4937 #4 _int_malloc (av=0x34083ba780 <main_arena>, bytes=56) at malloc.c:3411 #5 0x000000340808002c in __GI___libc_malloc (bytes=56) at malloc.c:2863 #6 0x0000003407c0d379 in _dl_map_object_deps (map=map@entry=0x7fe140d0ba18, preloads=preloads@entry=0x0, npreloads=npreloads@entry=0, trace_mode=trace_mode@entry=0, open_mode=open_mode@entry=-2147483648) at dl-deps.c:515 #7 0x0000003407c138bc in dl_open_worker (a=a@entry=0x7fff5c0748f8) at dl-open.c:265 #8 0x0000003407c0f304 in _dl_catch_error (objname=objname@entry=0x7fff5c0748e8, errstring=errstring@entry=0x7fff5c0748f0, mallocedp=mallocedp@entry=0x7fff5c0748e0, operate=operate@entry=0x3407c13740 <dl_open_worker>, args=args@entry=0x7fff5c0748f8) at dl-error.c:177 #9 0x0000003407c131eb in _dl_open (file=0x340817aaa6 "libgcc_s.so.1", mode=-2147483647, caller_dlopen=0x3408109c05 <init+21>, nsid=-2, argc=3, argv=<optimized out>, env=0x20ae470) at dl-open.c:656
#10 0x00000034081305d2 in do_dlopen (ptr=ptr@entry=0x7fff5c074b00) at dl-libc.c:87
#11 0x0000003407c0f304 in _dl_catch_error (objname=0x7fff5c074ae0, errstring=0x7fff5c074af0, mallocedp=0x7fff5c074ad0, operate=0x3408130590 <do_dlopen>, args=0x7fff5c074b00) at dl-error.c:177 #12 0x0000003408130692 in dlerror_run (args=0x7fff5c074b00, operate=0x3408130590 <do_dlopen>) at dl-libc.c:46
#13 __GI___libc_dlopen_mode (name=name@entry=0x340817aaa6 "libgcc_s.so.1", mode=mode@entry=-2147483647) at dl-libc.c:163
#14 0x0000003408109c05 in init () at ../sysdeps/x86_64/backtrace.c:52
#15 0x0000003408c0ca40 in pthread_once () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_once.S:103
#16 0x0000003408109d1c in __GI___backtrace (array=array@entry=0x7fff5c074dc0, size=size@entry=64) at ../sysdeps/x86_64/backtrace.c:103
#17 0x0000003408075d34 in __libc_message (do_abort=do_abort@entry=2, fmt=fmt@entry=0x340817e568 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/unix/sysv/linux/libc_fatal.c:176 #18 0x000000340807d0b8 in malloc_printerr (ptr=<optimized out>, str=0x340817e690 "free(): invalid next size (fast)", action=3) at malloc.c:4937
#19 _int_free (av=0x34083ba780 <main_arena>, p=<optimized out>, have_lock=0) at malloc.c:3789
#20 0x0000000000465e72 in perf_evsel__free_fd (evsel=0x20af190) at util/evsel.c:786
#21 perf_evsel__close (evsel=evsel@entry=0x20af190, ncpus=<optimized out>, nthreads=nthreads@entry=1) at util/evsel.c:1139
#22 0x000000000045f77d in perf_evlist__close (evlist=0x20ae8b0) at util/evlist.c:1148
#23 perf_evlist__delete (evlist=0x20ae8b0) at util/evlist.c:114
#24 0x000000000042b878 in cmd_record (argc=<optimized out>, argv=<optimized out>, prefix=<optimized out>) at builtin-record.c:967
#25 0x000000000041c455 in run_builtin (p=p@entry=0x815e70 <commands+144>, argc=argc@entry=2, argv=argv@entry=0x7fff5c077c50) at perf.c:331
#26 0x000000000041bc70 in handle_internal_command (argv=0x7fff5c077c50, argc=2) at perf.c:390
#27 run_argv (argv=0x7fff5c0779d0, argcp=0x7fff5c0779dc) at perf.c:434
#28 main (argc=2, argv=0x7fff5c077c50) at perf.c:549
jirka
On Thu, Sep 11, 2014 at 01:33:36PM +0200, Jiri Olsa wrote:
SNIP
Quoted 9 lines
#20
0x0000000000465e72inperf_evsel__free_fd(evsel=0x20af190) atutil/evsel.c:786
#21perf_evsel__close(evsel=evsel@entry=0x20af190, ncpus=<optimized out>, nthreads=nthreads@entry=1) atutil/evsel.c:1139#220x000000000045f77dinperf_evlist__close(evlist=0x20ae8b0) atutil/evlist.c:1148
#23perf_evlist__delete(evlist=0x20ae8b0) atutil/evlist.c:114
#240x000000000042b878incmd_record(argc=<optimized out>, argv=<optimized out>, prefix=<optimized out>) at builtin-record.c:967 #250x000000000041c455inrun_builtin(p=p@entry=0x815e70 <commands+144>, argc=argc@entry=2, argv=argv@entry=0x7fff5c077c50) at perf.c:331 #260x000000000041bc70inhandle_internal_command(argv=0x7fff5c077c50, argc=2) at perf.c:390
#27run_argv(argv=0x7fff5c0779d0, argcp=0x7fff5c0779dc) at perf.c:434
#28 main (argc=2, argv=0x7fff5c077c50) at perf.c:549
so the reason was that my fd lib stuff did not get rebuilt..
you probably want to add attached change, before there's the fix for the apik library
jirka
---
tools/perf/Makefile.perf +1 -1
@@ -769,7 +769,7 @@ $(LIBTRACEEVENT)-clean: install-traceevent-plugins: $(LIBTRACEEVENT) $(QUIET_SUBDIR0)$(TRACE_EVENT_DIR) $(LIBTRACEEVENT_FLAGS) install_plugins -LIBAPIKFS_SOURCES = $(wildcard $(LIB_PATH)fs/*.[ch]) +LIBAPIKFS_SOURCES = $(wildcard $(LIB_PATH)fs/*.[ch] $(LIB_PATH)fd/*.[ch]) # if subdir is set, we've been called from above so target has been built # already
Arnaldo Carvalho de Melo
maintainerEm Thu, Sep 11, 2014 at 01:48:25PM +0200, Jiri Olsa escreveu:
Quoted 14 lines · Jiri Olsa
SNIP
Quoted 9 lines
#20
0x0000000000465e72inperf_evsel__free_fd(evsel=0x20af190) atutil/evsel.c:786
#21perf_evsel__close(evsel=evsel@entry=0x20af190, ncpus=<optimized out>, nthreads=nthreads@entry=1) atutil/evsel.c:1139#220x000000000045f77dinperf_evlist__close(evlist=0x20ae8b0) atutil/evlist.c:1148
#23perf_evlist__delete(evlist=0x20ae8b0) atutil/evlist.c:114
#240x000000000042b878incmd_record(argc=<optimized out>, argv=<optimized out>, prefix=<optimized out>) at builtin-record.c:967 #250x000000000041c455inrun_builtin(p=p@entry=0x815e70 <commands+144>, argc=argc@entry=2, argv=argv@entry=0x7fff5c077c50) at perf.c:331 #260x000000000041bc70inhandle_internal_command(argv=0x7fff5c077c50, argc=2) at perf.c:390
#27run_argv(argv=0x7fff5c0779d0, argcp=0x7fff5c0779dc) at perf.c:434
#28 main (argc=2, argv=0x7fff5c077c50) at perf.c:549
Quoted 1 line
so the reason was that my fd lib stuff did not get rebuilt..
Thanks a lot! I missed that one, will fold it into the patch that introduces fdarray and add a v2: comment attributing credit to you, so that bisection works.
- Arnaldo
Quoted 20 lines
you probably want to add attached change, before there's the fix for the apik library
jirka
---
tools/perf/Makefile.perf +1
-1@@ -769,7 +769,7 @@ $(LIBTRACEEVENT)-clean: install-traceevent-plugins: $(LIBTRACEEVENT) $(QUIET_SUBDIR0)$(TRACE_EVENT_DIR) $(LIBTRACEEVENT_FLAGS) install_plugins -LIBAPIKFS_SOURCES = $(wildcard $(LIB_PATH)fs/*.[ch]) +LIBAPIKFS_SOURCES = $(wildcard $(LIB_PATH)fs/*.[ch] $(LIB_PATH)fd/*.[ch]) # if subdir is set, we've been called from above so target has been built # already
Arnaldo Carvalho de Melo
maintainerEm Thu, Sep 11, 2014 at 10:30:29AM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 9 lines
Em Thu, Sep 11, 2014 at 01:48:25PM +0200, Jiri Olsa escreveu:
Quoted 1 line
so the reason was that my fd lib stuff did not get rebuilt..
Thanks a lot! I missed that one, will fold it into the patch that introduces fdarray and add a v2: comment attributing credit to you, so that bisection works.
Quoted 2 lines
you probably want to add attached change, before there's the fix for the apik library
Done, need just to move that fcntl O_NONBLOCK from fdarray__add() to perf_evlist__add_pollfd(), what I have now is at:
https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/fdarray.v3
In case you want to see if itaddresses your concerns.
The last cset should address what you pointed out about unmapping only when the events gets completely drained from the ring buffer.
I'll do the fcntl() fix, test your drain-last-events scenario and repost.
- Arnaldo
Arnaldo Carvalho de Melo
maintainerEm Thu, Sep 11, 2014 at 06:36:38PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 16 lines
Em Thu, Sep 11, 2014 at 10:30:29AM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 9 lines
Em Thu, Sep 11, 2014 at 01:48:25PM +0200, Jiri Olsa escreveu:
Quoted 1 line
so the reason was that my fd lib stuff did not get rebuilt..
Thanks a lot! I missed that one, will fold it into the patch that introduces fdarray and add a v2: comment attributing credit to you, so that bisection works.
Quoted 2 lines
you probably want to add attached change, before there's the fix for the apik library
Done, need just to move that fcntl
O_NONBLOCKfromfdarray__add()toperf_evlist__add_pollfd(), what I have now is at:https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/fdarray.v3
Quoted 7 lines
In case you want to see if itaddresses your concerns.
The last cset should address what you pointed out about unmapping only when the events gets completely drained from the ring buffer.
I'll do the
fcntl()fix, test your drain-last-events scenario and repost.
Done, updated it there:
https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/fdarray.v4
Its on top of my last perf/core branch, i.e. with builtin-record.c using perf_evlist__mmap_consume() and it and builtin-trace.c doing one last mmap_read loop to consume what is left after all fds for a mmap are closed.
I'll wait a bit before reposting, probably Jiri will not be able to comment this week, but I would like to at least post the URL for this latest v4 kit.
Adrian, if you could take a look at it, would be really great :-)
- Arnaldo
Quoted 34 lines · Arnaldo Carvalho de Melo
Em Thu, Sep 11, 2014 at 06:36:38PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 16 lines
Em Thu, Sep 11, 2014 at 10:30:29AM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 9 lines
Em Thu, Sep 11, 2014 at 01:48:25PM +0200, Jiri Olsa escreveu:
Quoted 1 line
so the reason was that my fd lib stuff did not get rebuilt..
Thanks a lot! I missed that one, will fold it into the patch that introduces fdarray and add a v2: comment attributing credit to you, so that bisection works.
Quoted 2 lines
you probably want to add attached change, before there's the fix for the apik library
Done, need just to move that fcntl
O_NONBLOCKfromfdarray__add()toperf_evlist__add_pollfd(), what I have now is at:https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/fdarray.v3
Quoted 7 lines
In case you want to see if itaddresses your concerns.
The last cset should address what you pointed out about unmapping only when the events gets completely drained from the ring buffer.
I'll do the
fcntl()fix, test your drain-last-events scenario and repost.Done, updated it there:
https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/fdarray.v4
Its on top of my last perf/core branch, i.e. with
builtin-record.cusingperf_evlist__mmap_consume()and it andbuiltin-trace.cdoing one lastmmap_readloop to consume what is left after all fds for a mmap are closed.
Quoted 5 lines
I'll wait a bit before reposting, probably Jiri will not be able to comment this week, but I would like to at least post the URL for this latest v4 kit.
Adrian, if you could take a look at it, would be really great :-)
I checked the branch.. the last patch brakes the functionality of perf record for me
I tried attached change instead.. but then I realized we dont actually need the final unmap for record command, because we always open 1 mmap for each CPU and then do the output ioctl for any other process we monitor for each CPU.. so all the mmaps stay until last process is dead or we exit
maybe we could have this code just to be complete, or I missed something ;-)
jirka
---
tools/perf/builtin-record.c +6 -2
@@ -244,11 +244,15 @@ static int record__mmap_read_all(struct record *rec)int rc =0 ;for (i =0 ; i < rec->evlist->nr_mmaps; i++) { -if (rec->evlist->mmap[i].base) { -if (record__mmap_read (rec, &rec->evlist->mmap[i]) !=0 ) { +struct perf_mmap *md = &rec->evlist->mmap[i]; + +if (md->base) { +if (record__mmap_read (rec, md) !=0 ) { rc =-1 ; goto out; } +if (md->refcnt ==1 &&perf_mmap__empty (md)) +perf_evlist__mmap_put (rec->evlist, i); } }
tools/perf/util/evlist.c +1 -7
@@ -25,7 +25,6 @@#include <linux/bitops.h> #include <linux/hash.h> -static void perf_evlist__mmap_put (struct perf_evlist *evlist,int idx);static void __perf_evlist__munmap (struct perf_evlist *evlist,int idx);#define FD (e, x, y) (*(int *)xyarray__entry(e->fd, x, y))
@@ -676,17 +675,12 @@ union perf_event *perf_evlist__mmap_read(struct perf_evlist *evlist, int idx)return event; } -static bool perf_mmap__empty (struct perf_mmap *md) -{ -return perf_mmap__read_head (md) != md->prev; -} -static void perf_evlist__mmap_get (struct perf_evlist *evlist,int idx) { ++evlist->mmap[idx].refcnt; } -static void perf_evlist__mmap_put (struct perf_evlist *evlist,int idx) +void perf_evlist__mmap_put (struct perf_evlist *evlist,int idx) {BUG_ON (evlist->mmap[idx].refcnt ==0 );
tools/perf/util/evlist.h +7 -0
@@ -204,6 +204,13 @@ static inline void perf_mmap__write_tail(struct perf_mmap *md, pc->data_tail = tail; } +static inline bool perf_mmap__empty (struct perf_mmap *md) +{ +return perf_mmap__read_head (md) != md->prev; +} + +void perf_evlist__mmap_put (struct perf_evlist *evlist,int idx); +bool perf_evlist__can_select_event (struct perf_evlist *evlist,const char *str);void perf_evlist__to_front (struct perf_evlist *evlist,struct perf_evsel *move_evsel);
Arnaldo Carvalho de Melo
maintainerEm Mon, Sep 22, 2014 at 03:35:26PM +0200, Jiri Olsa escreveu:
Quoted 11 lines · Arnaldo Carvalho de Melo
Quoted 10 lines
Em Thu, Sep 11, 2014 at 06:36:38PM -0300, Arnaldo Carvalho de Melo escreveu:
Done, updated it there:https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/fdarray.v4
Its on top of my last perf/core branch, i.e. with
builtin-record.cusingperf_evlist__mmap_consume()and it andbuiltin-trace.cdoing one lastmmap_readloop to consume what is left after all fds for a mmap are closed.
Quoted 8 lines
Quoted 5 lines
I'll wait a bit before reposting, probably Jiri will not be able to comment this week, but I would like to at least post the URL for this latest v4 kit.
Adrian, if you could take a look at it, would be really great :-)
I checked the branch.. the last patch brakes the functionality of perf record for me
Breaks in what sense?
Quoted 7 lines
I tried attached change instead.. but then I realized we dont actually need the final unmap for record command, because we always open 1 mmap for each CPU and then do the output ioctl for any other process we monitor for each CPU.. so all the mmaps stay until last process is dead or we exit
maybe we could have this code just to be complete,
This is to be complete, trying to find she simples setup + event loop processing + teardown API.
Quoted 1 line
or I missed something ;-)
We'll both find it, eventually ;-)
- Arnaldo
Quoted 6 lines
+++
b/tools/perf/builtin-record.ctools/perf/builtin-record.c +0
-1@@ -244,11 +244,15 @@ static int record__mmap_read_all(struct record *rec)int rc =0 ;for (i =0 ; i < rec->evlist->nr_mmaps; i++) { -if (rec->evlist->mmap[i].base) {
tools/perf/builtin-record.c static int record__mmap_read_all(struct record *rec) · line 255
Quoted 10 lines
tools/perf/builtin-record.c +6
-1-if (record__mmap_read (rec, &rec->evlist->mmap[i]) !=0 ) { +struct perf_mmap *md = &rec->evlist->mmap[i]; + +if (md->base) { +if (record__mmap_read (rec, md) !=0 ) { rc =-1 ; goto out; } +if (md->refcnt ==1 &&perf_mmap__empty (md)) +perf_evlist__mmap_put (rec->evlist, i);
Humm, this should go into perf_evlist__mmap_consume(), is it not there? I.e. record__mmap_read() will have called it, nope? Humm, it seems I haven't rebased this on top of tip/perf/core, that has this at the end of record__mmap_read():
md->prev = old;
perf_evlist__mmap_consume(rec->evlist, idx);Sorry about that, but at least it matches what you did, no?
I just pushed a perf/fdarray.v5 with it.
There is one more thing I realised, that is we can't exit the loop like I do in my series at these places:
commit 593ee89818b3ab3493a9f368c1eb6b2e4c4a9512
perf trace: Filter out POLLHUP'ed file descriptors
commit 2478af9c299f4f7950b8d5d4bca2d2c71d661018
perf record: Filter out POLLHUP'ed file descriptors
I need to reorder the patches so that this draining gets merged with the test for entries in the poll fdarray to drop to zero.
The end result should be the equivalent to what is in perf/fdarray.v5, but if we test right at those two last mentioned patches, we may lose the last few events in a record or trace session.
- Arnaldo
Quoted 49 lines
tools/perf/util/evlist.c +1
-7@@ -25,7 +25,6 @@#include <linux/bitops.h> #include <linux/hash.h> -static void perf_evlist__mmap_put (struct perf_evlist *evlist,int idx);static void __perf_evlist__munmap (struct perf_evlist *evlist,int idx);#define FD (e, x, y) (*(int *)xyarray__entry(e->fd, x, y))@@ -676,17 +675,12 @@ union perf_event *perf_evlist__mmap_read(struct perf_evlist *evlist, int idx)return event; } -static bool perf_mmap__empty (struct perf_mmap *md) -{ -return perf_mmap__read_head (md) != md->prev; -} -static void perf_evlist__mmap_get (struct perf_evlist *evlist,int idx) { ++evlist->mmap[idx].refcnt; } -static void perf_evlist__mmap_put (struct perf_evlist *evlist,int idx) +void perf_evlist__mmap_put (struct perf_evlist *evlist,int idx) {BUG_ON (evlist->mmap[idx].refcnt ==0 );tools/perf/util/evlist.h +7
-0@@ -204,6 +204,13 @@ static inline void perf_mmap__write_tail(struct perf_mmap *md, pc->data_tail = tail; } +static inline bool perf_mmap__empty (struct perf_mmap *md) +{ +return perf_mmap__read_head (md) != md->prev; +} + +void perf_evlist__mmap_put (struct perf_evlist *evlist,int idx); +bool perf_evlist__can_select_event (struct perf_evlist *evlist,const char *str);void perf_evlist__to_front (struct perf_evlist *evlist,struct perf_evsel *move_evsel);
Quoted 16 lines · Arnaldo Carvalho de Melo
Em Mon, Sep 22, 2014 at 03:35:26PM +0200, Jiri Olsa escreveu:
Quoted 15 lines · Arnaldo Carvalho de Melo
Quoted 14 lines
Em Thu, Sep 11, 2014 at 06:36:38PM -0300, Arnaldo Carvalho de Melo escreveu:
Done, updated it there:https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/fdarray.v4
Its on top of my last perf/core branch, i.e. with
builtin-record.cusingperf_evlist__mmap_consume()and it andbuiltin-trace.cdoing one lastmmap_readloop to consume what is left after all fds for a mmap are closed.I'll wait a bit before reposting, probably Jiri will not be able to comment this week, but I would like to at least post the URL for this latest v4 kit.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Adrian, if you could take a look at it, would be really great :-)
I checked the branch.. the last patch brakes the functionality of perf record for me
Breaks in what sense?
perf record -p pid
and killing the pid won't finish perf record
jirka
Arnaldo Carvalho de Melo
maintainerEm Mon, Sep 22, 2014 at 04:51:33PM +0200, Jiri Olsa escreveu:
Quoted 17 lines · Arnaldo Carvalho de Melo
Quoted 15 lines
Em Mon, Sep 22, 2014 at 03:35:26PM +0200, Jiri Olsa escreveu:
Quoted 14 lines · Arnaldo Carvalho de Melo
Quoted 13 lines
Em Thu, Sep 11, 2014 at 06:36:38PM -0300, Arnaldo Carvalho de Melo escreveu:
https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/fdarray.v4Its on top of my last perf/core branch, i.e. with
builtin-record.cusingperf_evlist__mmap_consume()and it andbuiltin-trace.cdoing one lastmmap_readloop to consume what is left after all fds for a mmap are closed.I'll wait a bit before reposting, probably Jiri will not be able to comment this week, but I would like to at least post the URL for this latest v4 kit.
Adrian, if you could take a look at it, would be really great :-)
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
I checked the branch.. the last patch brakes the functionality of perf record for me
Breaks in what sense?
perf record -p pid
and killing the pid won't finish perf record
Should be ok now, after making sure record uses perf_evlist__mmap_consume(), which does what you did in your proposed fix, tried both killing it and letting it exit normally, i.e. a 'sleep 5s' target.
Pushed
https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/fdarray.v6
after moving the changesets for trace and record to be done last, waiting for all fds to be POLLHUP'ed and doing one last draining mmap_read loop over the mmaps.
- Arnaldo
Quoted 32 lines · Arnaldo Carvalho de Melo
Em Mon, Sep 22, 2014 at 04:51:33PM +0200, Jiri Olsa escreveu:
Quoted 17 lines · Arnaldo Carvalho de Melo
Quoted 15 lines
Em Mon, Sep 22, 2014 at 03:35:26PM +0200, Jiri Olsa escreveu:
Quoted 14 lines · Arnaldo Carvalho de Melo
Quoted 13 lines
Em Thu, Sep 11, 2014 at 06:36:38PM -0300, Arnaldo Carvalho de Melo escreveu:
https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/fdarray.v4Its on top of my last perf/core branch, i.e. with
builtin-record.cusingperf_evlist__mmap_consume()and it andbuiltin-trace.cdoing one lastmmap_readloop to consume what is left after all fds for a mmap are closed.I'll wait a bit before reposting, probably Jiri will not be able to comment this week, but I would like to at least post the URL for this latest v4 kit.
Adrian, if you could take a look at it, would be really great :-)
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
I checked the branch.. the last patch brakes the functionality of perf record for me
Breaks in what sense?
perf record -p pid
and killing the pid won't finish perf record
Should be ok now, after making sure record uses
perf_evlist__mmap_consume(), which does what you did in your proposed fix, tried both killing it and letting it exit normally, i.e. a 'sleep 5s' target.
Quoted 5 lines
Pushed
https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/fdarray.v6
after moving the changesets for trace and record to be done last, waiting for all fds to be POLLHUP'ed and doing one last drainingmmap_readloop over the mmaps.
One more thing is I noticed the test 34 is segfaulting, because perf_evlist__munmap_filtered assumes priv pointer setup, which is not done in the test.
otherwise it looks gut
jirka
Arnaldo Carvalho de Melo
maintainerEm Tue, Sep 23, 2014 at 11:26:45AM +0200, Jiri Olsa escreveu:
Quoted 39 lines · Arnaldo Carvalho de Melo
Quoted 32 lines
Em Mon, Sep 22, 2014 at 04:51:33PM +0200, Jiri Olsa escreveu:
Quoted 17 lines · Arnaldo Carvalho de Melo
Quoted 15 lines
Em Mon, Sep 22, 2014 at 03:35:26PM +0200, Jiri Olsa escreveu:
Quoted 14 lines · Arnaldo Carvalho de Melo
Quoted 13 lines
Em Thu, Sep 11, 2014 at 06:36:38PM -0300, Arnaldo Carvalho de Melo escreveu:
https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/fdarray.v4Its on top of my last perf/core branch, i.e. with
builtin-record.cusingperf_evlist__mmap_consume()and it andbuiltin-trace.cdoing one lastmmap_readloop to consume what is left after all fds for a mmap are closed.I'll wait a bit before reposting, probably Jiri will not be able to comment this week, but I would like to at least post the URL for this latest v4 kit.
Adrian, if you could take a look at it, would be really great :-)
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
I checked the branch.. the last patch brakes the functionality of perf record for me
Breaks in what sense?
perf record -p pid
and killing the pid won't finish perf record
Should be ok now, after making sure record uses
perf_evlist__mmap_consume(), which does what you did in your proposed fix, tried both killing it and letting it exit normally, i.e. a 'sleep 5s' target.Quoted 5 lines
Pushed
https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/fdarray.v6
after moving the changesets for trace and record to be done last, waiting for all fds to be POLLHUP'ed and doing one last drainingmmap_readloop over the mmaps.
Quoted 3 lines
One more thing is I noticed the test 34 is segfaulting, because
perf_evlist__munmap_filteredassumes priv pointer setup, which is not done in the test.
Yeah, forgot to update it wrt mmap refcounting.
But this brought to my attention the difficulty in debugging perf test entries these days, because we do a fork + exec there, I tried using gdb and I must be missing something, because I couldn't add a breakpoint inside the tests :-\
I'll try with 'perf probe' instead of with gdb, probably will be a good idea to try eating our own dog food, and overal, for just inserting some printfs, better than using a full-blown debugger like gdb.
- Arnaldo
Quoted 1 line
otherwise it looks gut
Danke!
- Arnaldo
Quoted 44 lines · Arnaldo Carvalho de Melo
Em Tue, Sep 23, 2014 at 11:26:45AM +0200, Jiri Olsa escreveu:
Quoted 39 lines · Arnaldo Carvalho de Melo
Quoted 32 lines
Em Mon, Sep 22, 2014 at 04:51:33PM +0200, Jiri Olsa escreveu:
Quoted 17 lines · Arnaldo Carvalho de Melo
Quoted 15 lines
Em Mon, Sep 22, 2014 at 03:35:26PM +0200, Jiri Olsa escreveu:
Quoted 14 lines · Arnaldo Carvalho de Melo
Quoted 13 lines
Em Thu, Sep 11, 2014 at 06:36:38PM -0300, Arnaldo Carvalho de Melo escreveu:
https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/fdarray.v4Its on top of my last perf/core branch, i.e. with
builtin-record.cusingperf_evlist__mmap_consume()and it andbuiltin-trace.cdoing one lastmmap_readloop to consume what is left after all fds for a mmap are closed.I'll wait a bit before reposting, probably Jiri will not be able to comment this week, but I would like to at least post the URL for this latest v4 kit.
Adrian, if you could take a look at it, would be really great :-)
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
I checked the branch.. the last patch brakes the functionality of perf record for me
Breaks in what sense?
perf record -p pid
and killing the pid won't finish perf record
Should be ok now, after making sure record uses
perf_evlist__mmap_consume(), which does what you did in your proposed fix, tried both killing it and letting it exit normally, i.e. a 'sleep 5s' target.Quoted 5 lines
Pushed
https://git.kernel.org/cgit/linux/kernel/git/acme/linux.git/log/?h=perf/fdarray.v6
after moving the changesets for trace and record to be done last, waiting for all fds to be POLLHUP'ed and doing one last drainingmmap_readloop over the mmaps.Quoted 3 lines
One more thing is I noticed the test 34 is segfaulting, because
perf_evlist__munmap_filteredassumes priv pointer setup, which is not done in the test.
Quoted 6 lines
Yeah, forgot to update it wrt mmap refcounting.
But this brought to my attention the difficulty in debugging perf test entries these days, because we do a fork + exec there, I tried using gdb and I must be missing something, because I couldn't add a breakpoint inside the tests :-\
you need to run following gdb command:
(gdb) set follow-fork-mode child
jirka