perf: Better fit max unprivileged mlock pages for tools needs
PATCH 1/1 perf: Better fit max unprivileged mlock pages for tools needs
The maximum kilobytes of locked memory that an unprivileged user can reserve is of 512 kB = 128 pages by default, scaled to the number of onlined CPUs, which fits well with the tools that use 128 data pages by default.
However tools actually use 129 pages, because they need one more for the user control page. Thus the default mlock threshold is not sufficient for the default tools needs and we always end up to evaluate the constant mlock rlimit policy, which doesn't have this scaling with the number of online CPUs.
Hence, on systems that have more than 16 CPUs, we overlap the rlimit threshold and fail to mmap:
$ perf record ls Error: failed to mmap with 1 (Operation not permitted)
Just increase the max unprivileged mlock threshold by one page so that it supports well perf tools even after 16 CPUs.
--- kernel/perf_event.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
kernel/perf_event.c +2 -1
@@ -145,7 +145,8 @@ static struct srcu_struct pmus_srcu;*/ int sysctl_perf_event_paranoid__read_mostly =1 ; -int sysctl_perf_event_mlock__read_mostly =512 ;/* 'free' kb per user */ +/* Minimum for 128 pages + 1 for the user control page */ +int sysctl_perf_event_mlock__read_mostly =516 ;/* 'free' kb per user */ /* * max perf event sample rate
Signature · 1 line
1.7.3.2
Em Wed, Mar 23, 2011 at 07:29:39PM +0100, Frederic Weisbecker escreveu:
Quoted 11 lines
The maximum kilobytes of locked memory that an unprivileged user can reserve is of 512 kB = 128 pages by default, scaled to the number of onlined CPUs, which fits well with the tools that use 128 data pages by default.
However tools actually use 129 pages, because they need one more for the user control page. Thus the default mlock threshold is not sufficient for the default tools needs and we always end up to evaluate the constant mlock rlimit policy, which doesn't have this scaling with the number of online CPUs.
Quoted 8 lines
Hence, on systems that have more than 16 CPUs, we overlap the rlimit threshold and fail to mmap:
$ perf record ls Error: failed to mmap with 1 (Operation not permitted)Just increase the max unprivileged mlock threshold by one page so that it supports well perf tools even after 16 CPUs.
Thanks!
Quoted 27 lines
Reported-by Han Pingtian <[email protected]>Reported-by Peter Zijlstra <[email protected]>Reported-by Arnaldo Carvalho de Melo <[email protected]>Signed-off-by Frederic Weisbecker <[email protected]>Cc Ingo Molnar <[email protected]>Cc Stephane Eranian <[email protected]>Cc Stable <[email protected]>--- kernel/perf_event.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)kernel/perf_event.c +2
-1@@ -145,7 +145,8 @@ static struct srcu_struct pmus_srcu;*/ int sysctl_perf_event_paranoid__read_mostly =1 ; -int sysctl_perf_event_mlock__read_mostly =512 ;/* 'free' kb per user */ +/* Minimum for 128 pages + 1 for the user control page */ +int sysctl_perf_event_mlock__read_mostly =516 ;/* 'free' kb per user */ /* * max perf event sample rate Signature · 1 line
1.7.3.2
Commit-ID: 880f57318450dbead6a03f9e31a1468924d6dd88 Gitweb: http://git.kernel.org/tip/880f57318450dbead6a03f9e31a1468924d6dd88 Author: Frederic Weisbecker <[email protected]> AuthorDate: Wed, 23 Mar 2011 19:29:39 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Wed, 23 Mar 2011 20:57:04 +0100
perf: Better fit max unprivileged mlock pages for tools needs
The maximum kilobytes of locked memory that an unprivileged user can reserve is of 512 kB = 128 pages by default, scaled to the number of onlined CPUs, which fits well with the tools that use 128 data pages by default.
However tools actually use 129 pages, because they need one more for the user control page. Thus the default mlock threshold is not sufficient for the default tools needs and we always end up to evaluate the constant mlock rlimit policy, which doesn't have this scaling with the number of online CPUs.
Hence, on systems that have more than 16 CPUs, we overlap the rlimit threshold and fail to mmap:
$ perf record ls Error: failed to mmap with 1 (Operation not permitted)
Just increase the max unprivileged mlock threshold by one page so that it supports well perf tools even after 16 CPUs.
LKML-Reference: <[email protected]>
--- kernel/perf_event.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
kernel/perf_event.c +2 -1
@@ -145,7 +145,8 @@ static struct srcu_struct pmus_srcu;*/ int sysctl_perf_event_paranoid__read_mostly =1 ; -int sysctl_perf_event_mlock__read_mostly =512 ;/* 'free' kb per user */ +/* Minimum for 128 pages + 1 for the user control page */ +int sysctl_perf_event_mlock__read_mostly =516 ;/* 'free' kb per user */ /* * max perf event sample rate
kernel/perf_event.c static struct srcu_struct pmus_srcu; · line 149
Quoted 3 lines · Frederic Weisbecker
kernel/perf_event.c +2
-1-int sysctl_perf_event_mlock__read_mostly =512 ;/* 'free' kb per user */ +/* Minimum for 128 pages + 1 for the user control page */ +int sysctl_perf_event_mlock__read_mostly =516 ;/* 'free' kb per user */
This assumes PAGE_SIZE = 4k..
2011/3/25 Peter Zijlstra <[email protected]>:
Quoted 7 lines · Frederic Weisbecker
kernel/perf_event.c static struct srcu_struct pmus_srcu; · line 149
Quoted 4 lines
kernel/perf_event.c +2
-1-int sysctl_perf_event_mlock__read_mostly =512 ;/* 'free' kb per user */ +/* Minimum for 128 pages + 1 for the user control page */ +int sysctl_perf_event_mlock__read_mostly =516 ;/* 'free' kb per user */ This assumes
PAGE_SIZE= 4k..
Yeah, should I rebase the value on top of PAGE_SIZE?
Quoted 10 lines · Frederic Weisbecker
2011/3/25 Peter Zijlstra <[email protected]>:
Quoted 7 lines · Frederic Weisbecker
kernel/perf_event.c static struct srcu_struct pmus_srcu; · line 149
Quoted 4 lines
kernel/perf_event.c +2
-1-int sysctl_perf_event_mlock__read_mostly =512 ;/* 'free' kb per user */ +/* Minimum for 128 pages + 1 for the user control page */ +int sysctl_perf_event_mlock__read_mostly =516 ;/* 'free' kb per user */ This assumes
PAGE_SIZE= 4k..Yeah, should I rebase the value on top of
PAGE_SIZE?
Possibly yeah, I guess that when the base page size if 64k you really can't do anything about wasting the whole first page, comes with the territory.
PATCH 1/1 perf: Rebase max unprivileged mlock threshold on top of page size
So that we ensure we allow 129 pages, even if the page size is different than 4096.
--- kernel/perf_event.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/perf_event.c +1 -1
@@ -146,7 +146,7 @@ static struct srcu_struct pmus_srcu;int sysctl_perf_event_paranoid__read_mostly =1 ;/* Minimum for 128 pages + 1 for the user control page */ -int sysctl_perf_event_mlock__read_mostly =516 ;/* 'free' kb per user */ +int sysctl_perf_event_mlock__read_mostly = (129 *PAGE_SIZE ) /1024 ;/* 'free' kb per user */ /* * max perf event sample rate
Signature · 1 line
1.7.3.2
Re: [PATCH] perf: Rebase max unprivileged mlock threshold on top of page size
Quoted 1 line · Frederic Weisbecker
+int
sysctl_perf_event_mlock__read_mostly= (129 *PAGE_SIZE) / 1024; /* 'free' kb per user */
I'm fairly sure you don't want to do this either:
512 + (PAGE_SIZE / 1024)
should do I think, there's no reason to further inflate things for large page platforms.
Re: [PATCH] perf: Rebase max unprivileged mlock threshold on top of page size
Quoted 7 lines · Peter Zijlstra
Quoted 1 line · Frederic Weisbecker
+int
sysctl_perf_event_mlock__read_mostly= (129 *PAGE_SIZE) / 1024; /* 'free' kb per user */I'm fairly sure you don't want to do this either:
512 + (PAGE_SIZE/ 1024)
should do I think, there's no reason to further inflate things for large page platforms.
Right, that also requires to fix tools that use a default number of pages instead of computing nr_pages on top of page size and a default kb size.
Will respin by fixing both sides.
Thanks.
PATCH v2 1/2 perf: Rebase max unprivileged mlock threshold on top of page size
In reply to Peter ZijlstraEnsure we allow 512 kiB + 1 page for user control without assuming a 4096 page size.
--- kernel/perf_event.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)
kernel/perf_event.c +2 -2
@@ -145,8 +145,8 @@ static struct srcu_struct pmus_srcu;*/ int sysctl_perf_event_paranoid__read_mostly =1 ; -/* Minimum for 128 pages + 1 for the user control page */ -int sysctl_perf_event_mlock__read_mostly =516 ;/* 'free' kb per user */ +/* Minimum for 512 kiB + 1 user control page */ +int sysctl_perf_event_mlock__read_mostly =512 + (PAGE_SIZE /1024 );/* 'free' kiB per user */ /* * max perf event sample rate
Signature · 1 line
1.7.3.2
[tip:perf/urgent] perf: Rebase max unprivileged mlock threshold on top of page size
Commit-ID: 20443384fe090c5f8aeb016e7e85659c5bbdd69f Gitweb: http://git.kernel.org/tip/20443384fe090c5f8aeb016e7e85659c5bbdd69f Author: Frederic Weisbecker <[email protected]> AuthorDate: Thu, 31 Mar 2011 03:33:29 +0200 Committer: Ingo Molnar <[email protected]> CommitDate: Thu, 31 Mar 2011 13:02:54 +0200
perf: Rebase max unprivileged mlock threshold on top of page size
Ensure we allow 512 kiB + 1 page for user control without assuming a 4096 bytes page size.
LKML-Reference: <[email protected]>
--- kernel/perf_event.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)
kernel/perf_event.c +2 -2
@@ -145,8 +145,8 @@ static struct srcu_struct pmus_srcu;*/ int sysctl_perf_event_paranoid__read_mostly =1 ; -/* Minimum for 128 pages + 1 for the user control page */ -int sysctl_perf_event_mlock__read_mostly =516 ;/* 'free' kb per user */ +/* Minimum for 512 kiB + 1 user control page */ +int sysctl_perf_event_mlock__read_mostly =512 + (PAGE_SIZE /1024 );/* 'free' kiB per user */ /* * max perf event sample rate
PATCH 2/2 perf: Mmap 512 kiB by default
In reply to Peter ZijlstraThe default setting of perf record is to mmap 128 pages if the user did not override with -m.
However the page size may vary accross different architecture settings, giving different default size between each.
Moreover the kernel side still has a default max number of mlocked pages of 512 kiB + 1 page for unprivileged users. 128 + 1 pages with page size > 4096 overlaps this threshold.
Thus, better adapt to this limitation and set the default number of pages to fit those 512 kiB + 1 page.
--- tools/perf/builtin-record.c | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-)
tools/perf/builtin-record.c +5 -1
@@ -41,7 +41,7 @@ static u64 user_interval = ULLONG_MAX;static u64 default_interval =0 ;static unsigned int page_size; -static unsigned int mmap_pages =128 ; +static unsigned int mmap_pages =UINT_MAX ;static unsigned int user_freq =UINT_MAX ;static int freq =1000 ;static int output;
@@ -506,6 +506,10 @@ static int __cmd_record(int argc, const char **argv)if (have_tracepoints (&evsel_list->entries))perf_header__set_feat (&session->header,HEADER_TRACE_INFO ); +/* 512 kiB: default amount of unprivileged mlocked memory */ +if (mmap_pages ==UINT_MAX ) + mmap_pages = (512 *1024 ) / page_size; +if (forks) { child_pid =fork ();if (child_pid <0 ) {
Signature · 1 line
1.7.3.2
Re: [PATCH 2/2] perf: Mmap 512 kiB by default
Quoted 36 lines · Frederic Weisbecker
The default setting of perf record is to mmap 128 pages if the user did not override with -m.
However the page size may vary accross different architecture settings, giving different default size between each.Moreover the kernel side still has a default max number of mlocked pages of 512 kiB + 1 page for unprivileged users. 128 + 1 pages with page size > 4096 overlaps this threshold.
Thus, better adapt to this limitation and set the default number of pages to fit those 512 kiB + 1 page.
Signed-off-by Frederic Weisbecker <[email protected]>Cc Ingo Molnar <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Arnaldo Carvalho de Melo <[email protected]>Cc Paul Mackerras <[email protected]>Cc Stephane Eranian <[email protected]>--- tools/perf/builtin-record.c | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-)tools/perf/builtin-record.c +1
-1@@ -41,7 +41,7 @@ static u64 user_interval = ULLONG_MAX;static u64 default_interval =0 ;static unsigned int page_size; -static unsigned int mmap_pages =128 ; +static unsigned int mmap_pages =UINT_MAX ;static unsigned int user_freq =UINT_MAX ;static int freq =1000 ;static int output;
tools/perf/builtin-record.c static int __cmd_record(int argc, const char **argv) · line 515
Quoted 10 lines
tools/perf/builtin-record.c +4
-0@@ -506,6 +506,10 @@ static int __cmd_record(int argc, const char **argv)if (have_tracepoints (&evsel_list->entries))perf_header__set_feat (&session->header,HEADER_TRACE_INFO ); +/* 512 kiB: default amount of unprivileged mlocked memory */ +if (mmap_pages ==UINT_MAX ) + mmap_pages = (512 *1024 ) / page_size; +if (forks) { child_pid =fork ();if (child_pid <0 ) {
Ok, these two patches look good, I'll queue them. However a follow up might be to change the perf-record parameter from nr_pages to kb, which is a much more user friendly interface anyway ;-)
Re: [PATCH 2/2] perf: Mmap 512 kiB by default
Quoted 40 lines · Peter Zijlstra
Quoted 39 lines · Frederic Weisbecker
The default setting of perf record is to mmap 128 pages if the user did not override with -m.
However the page size may vary accross different architecture settings, giving different default size between each.Moreover the kernel side still has a default max number of mlocked pages of 512 kiB + 1 page for unprivileged users. 128 + 1 pages with page size > 4096 overlaps this threshold.
Thus, better adapt to this limitation and set the default number of pages to fit those 512 kiB + 1 page.
Signed-off-by Frederic Weisbecker <[email protected]>Cc Ingo Molnar <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Arnaldo Carvalho de Melo <[email protected]>Cc Paul Mackerras <[email protected]>Cc Stephane Eranian <[email protected]>--- tools/perf/builtin-record.c | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-)tools/perf/builtin-record.c +1
-1@@ -41,7 +41,7 @@ static u64 user_interval = ULLONG_MAX;static u64 default_interval =0 ;static unsigned int page_size; -static unsigned int mmap_pages =128 ; +static unsigned int mmap_pages =UINT_MAX ;static unsigned int user_freq =UINT_MAX ;static int freq =1000 ;static int output;@@ -506,6 +506,10 @@ static int __cmd_record(int argc, const char **argv)if (have_tracepoints (&evsel_list->entries))perf_header__set_feat (&session->header,HEADER_TRACE_INFO );
Quoted 11 lines
tools/perf/builtin-record.c static int __cmd_record(int argc, const char **argv) · line 515
Quoted 7 lines
tools/perf/builtin-record.c +4
-0+/* 512 kiB: default amount of unprivileged mlocked memory */ +if (mmap_pages ==UINT_MAX ) + mmap_pages = (512 *1024 ) / page_size; +if (forks) { child_pid =fork ();if (child_pid <0 ) {Ok, these two patches look good, I'll queue them. However a follow up might be to change the perf-record parameter from
nr_pagesto kb, which is a much more user friendly interface anyway ;-)
Agreed. I just wanted to keep that -m around on top of nr_pages beacuse we have some tools/scripts relying on it, like perf lock, which need to be changed as well.
We need a new option for this I think, and deprecate -m later perhaps, the time for external users to migrate? Or just remove -m but it's certainly going to be replaced soon.
[tip:perf/urgent] perf: mmap 512 kiB by default
In reply to Frederic WeisbeckerCommit-ID: 800cd25c12981c99e5401c3551ffffd99f1c6c85 Gitweb: http://git.kernel.org/tip/800cd25c12981c99e5401c3551ffffd99f1c6c85 Author: Frederic Weisbecker <[email protected]> AuthorDate: Thu, 31 Mar 2011 03:35:24 +0200 Committer: Ingo Molnar <[email protected]> CommitDate: Thu, 31 Mar 2011 13:02:55 +0200
perf: mmap 512 kiB by default
The default setting of perf record is to mmap 128 pages if the user did not override with -m.
However the page size may vary accross different architecture settings, giving different default size between each.
Moreover the kernel side still has a default max number of mlocked pages of 512 kiB + 1 page for unprivileged users. 128 + 1 pages with page size > 4096 overlaps this threshold.
Thus, better adapt to this limitation and set the default number of pages to fit those 512 kiB + 1 page.
LKML-Reference: <[email protected]>
--- tools/perf/builtin-record.c | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-)
tools/perf/builtin-record.c +5 -1
@@ -41,7 +41,7 @@ static u64 user_interval = ULLONG_MAX;static u64 default_interval =0 ;static unsigned int page_size; -static unsigned int mmap_pages =128 ; +static unsigned int mmap_pages =UINT_MAX ;static unsigned int user_freq =UINT_MAX ;static int freq =1000 ;static int output;
@@ -513,6 +513,10 @@ static int __cmd_record(int argc, const char **argv)if (have_tracepoints (&evsel_list->entries))perf_header__set_feat (&session->header,HEADER_TRACE_INFO ); +/* 512 kiB: default amount of unprivileged mlocked memory */ +if (mmap_pages ==UINT_MAX ) + mmap_pages = (512 *1024 ) / page_size; +if (forks) { child_pid =fork ();if (child_pid <0 ) {