perf: Intel PT and 64-bit AUX area size fixes
Subsystems (3)
Adrian Hunter
reviewercover perf: Intel PT and 64-bit AUX area size fixes
Hi
Here are some fixes.
I have taken the liberty of re-posting Marco Cavenati's patch from:
https://lore.kernel.org/lkml/[email protected]/
with updated commit suggested by Dave Hansen.
Adrian Hunter (6):
perf/x86/intel/pt: Fix a topa_entry base address calculation perf/x86/intel/pt: Fix pt_topa_entry_for_page() address calculation
perf: Fix perf_aux_size() for greater-than 32-bit size
perf: Prevent passing zero nr_pages to rb_alloc_aux()
perf: Fix default aux_watermark calculation
perf: Make rb_alloc_aux() return an error immediately if nr_pages <= 0
Marco Cavenati (1):
perf/x86/intel/pt: Fix topa_entry base length
arch/x86/events/intel/pt.c | 4 ++-- arch/x86/events/intel/pt.h | 4 ++-- kernel/events/core.c | 2 ++ kernel/events/internal.h | 2 +- kernel/events/ring_buffer.c | 7 ++++++- 5 files changed, 13 insertions(+), 6 deletions(-)
Regards
Adrian
Adrian Hunter
reviewerPATCH 1/7 perf/x86/intel/pt: Fix topa_entry base length
From: Marco Cavenati <[email protected]>
topa_entry->base needs to store a pfn. It obviously needs to be large enough to store the largest possible x86 pfn which is MAXPHYADDR-PAGE_SIZE (52-12). So it is 4 bits too small.
Increase the size of topa_entry->base from 36 bits to 40 bits.
Note, systems where physical addresses can be 256TiB or more are affected.
[ Adrian: Amend commit message as suggested by Dave Hansen ]
--- arch/x86/events/intel/pt.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/events/intel/pt.h +2 -2
@@ -33,8 +33,8 @@ struct topa_entry { u64 rsvd2 : 1; u64 size : 4; u64 rsvd3 : 2; - u64 base : 36; - u64 rsvd4 : 16; + u64 base : 40; + u64 rsvd4 : 12; };/* TSC to Core Crystal Clock Ratio */
Signature · 1 line
2.34.1
[tip: perf/core] perf/x86/intel/pt: Fix topa_entry base length
The following commit has been merged into the perf/core branch of tip:
The following commit has been merged into the perf/core branch of tip:
Commit-ID: 5638bd722a44bbe97c1a7b3fae5b9efddb3e70ff Gitweb: https://git.kernel.org/tip/5638bd722a44bbe97c1a7b3fae5b9efddb3e70ff Author: Marco Cavenati <[email protected]> AuthorDate: Mon, 24 Jun 2024 23:10:55 +03:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Thu, 04 Jul 2024 16:00:20 +02:00
perf/x86/intel/pt: Fix topa_entry base length
topa_entry->base needs to store a pfn. It obviously needs to be large enough to store the largest possible x86 pfn which is MAXPHYADDR-PAGE_SIZE (52-12). So it is 4 bits too small.
Increase the size of topa_entry->base from 36 bits to 40 bits.
Note, systems where physical addresses can be 256TiB or more are affected.
[ Adrian: Amend commit message as suggested by Dave Hansen ]
--- arch/x86/events/intel/pt.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/events/intel/pt.h +2 -2
@@ -33,8 +33,8 @@ struct topa_entry { u64 rsvd2 : 1; u64 size : 4; u64 rsvd3 : 2; - u64 base : 36; - u64 rsvd4 : 16; + u64 base : 40; + u64 rsvd4 : 12; };/* TSC to Core Crystal Clock Ratio */
Adrian Hunter
reviewerPATCH 2/7 perf/x86/intel/pt: Fix a topa_entry base address calculation
In reply to Adrian Huntertopa_entry->base is a bit-field. Bit-fields are not promoted to a 64-bit type, even if the underlying type is 64-bit, and so, if necessary, must be cast to a larger type when calculations are done.
Fix a topa_entry->base address calculation by adding a cast.
Without the cast, the address was limited to 36-bits i.e. 64GiB.
The address calculation is used on systems that do not support Multiple Entry ToPA (only Broadwell), and affects physical addresses on or above 64GiB. Instead of writing to the correct address, the address comprising the first 36 bits would be written to.
Intel PT snapshot and sampling modes are not affected.
--- arch/x86/events/intel/pt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/events/intel/pt.c +1 -1
@@ -878,7 +878,7 @@ static void pt_update_head(struct pt *pt)*/ static void *pt_buffer_region (struct pt_buffer *buf) { -return phys_to_virt (TOPA_ENTRY (buf->cur, buf->cur_idx)->base <<TOPA_SHIFT ); +return phys_to_virt ((phys_addr_t )TOPA_ENTRY (buf->cur, buf->cur_idx)->base <<TOPA_SHIFT ); }/**
Signature · 1 line
2.34.1
[tip: perf/core] perf/x86/intel/pt: Fix a topa_entry base address calculation
The following commit has been merged into the perf/core branch of tip:
The following commit has been merged into the perf/core branch of tip:
Commit-ID: ad97196379d0b8cb24ef3d5006978a6554e6467f Gitweb: https://git.kernel.org/tip/ad97196379d0b8cb24ef3d5006978a6554e6467f Author: Adrian Hunter <[email protected]> AuthorDate: Mon, 24 Jun 2024 23:10:56 +03:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Thu, 04 Jul 2024 16:00:21 +02:00
perf/x86/intel/pt: Fix a topa_entry base address calculation
topa_entry->base is a bit-field. Bit-fields are not promoted to a 64-bit type, even if the underlying type is 64-bit, and so, if necessary, must be cast to a larger type when calculations are done.
Fix a topa_entry->base address calculation by adding a cast.
Without the cast, the address was limited to 36-bits i.e. 64GiB.
The address calculation is used on systems that do not support Multiple Entry ToPA (only Broadwell), and affects physical addresses on or above 64GiB. Instead of writing to the correct address, the address comprising the first 36 bits would be written to.
Intel PT snapshot and sampling modes are not affected.
--- arch/x86/events/intel/pt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/events/intel/pt.c +1 -1
@@ -878,7 +878,7 @@ static void pt_update_head(struct pt *pt)*/ static void *pt_buffer_region (struct pt_buffer *buf) { -return phys_to_virt (TOPA_ENTRY (buf->cur, buf->cur_idx)->base <<TOPA_SHIFT ); +return phys_to_virt ((phys_addr_t )TOPA_ENTRY (buf->cur, buf->cur_idx)->base <<TOPA_SHIFT ); }/**
Adrian Hunter
reviewerPATCH 3/7 perf/x86/intel/pt: Fix pt_topa_entry_for_page() address calculation
In reply to Adrian HunterCurrently, perf allocates an array of page pointers which is limited in size by MAX_PAGE_ORDER. That in turn limits the maximum Intel PT buffer size to 2GiB. Should that limitation be lifted, the Intel PT driver can support larger sizes, except for one calculation in pt_topa_entry_for_page(), which is limited to 32-bits.
Fix pt_topa_entry_for_page() address calculation by adding a cast.
--- arch/x86/events/intel/pt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/events/intel/pt.c +1 -1
@@ -990,7 +990,7 @@ pt_topa_entry_for_page(struct pt_buffer *buf, unsigned int pg)* order allocations, there shouldn't be many of these. */ list_for_each_entry (topa, &buf->tables, list) { -if (topa->offset + topa->size > pg <<PAGE_SHIFT ) +if (topa->offset + topa->size > (unsigned long )pg <<PAGE_SHIFT ) goto found; }
Signature · 1 line
2.34.1
[tip: perf/core] perf/x86/intel/pt: Fix pt_topa_entry_for_page() address calculation
The following commit has been merged into the perf/core branch of tip:
The following commit has been merged into the perf/core branch of tip:
Commit-ID: 3520b251dcae2b4a27b95cd6f745c54fd658bda5 Gitweb: https://git.kernel.org/tip/3520b251dcae2b4a27b95cd6f745c54fd658bda5 Author: Adrian Hunter <[email protected]> AuthorDate: Mon, 24 Jun 2024 23:10:57 +03:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Thu, 04 Jul 2024 16:00:21 +02:00
perf/x86/intel/pt: Fix pt_topa_entry_for_page() address calculation
Currently, perf allocates an array of page pointers which is limited in size by MAX_PAGE_ORDER. That in turn limits the maximum Intel PT buffer size to 2GiB. Should that limitation be lifted, the Intel PT driver can support larger sizes, except for one calculation in pt_topa_entry_for_page(), which is limited to 32-bits.
Fix pt_topa_entry_for_page() address calculation by adding a cast.
--- arch/x86/events/intel/pt.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/events/intel/pt.c +1 -1
@@ -990,7 +990,7 @@ pt_topa_entry_for_page(struct pt_buffer *buf, unsigned int pg)* order allocations, there shouldn't be many of these. */ list_for_each_entry (topa, &buf->tables, list) { -if (topa->offset + topa->size > pg <<PAGE_SHIFT ) +if (topa->offset + topa->size > (unsigned long )pg <<PAGE_SHIFT ) goto found; }
Adrian Hunter
reviewerPATCH 4/7 perf: Fix perf_aux_size() for greater-than 32-bit size
In reply to Adrian Hunterperf_buffer->aux_nr_pages uses a 32-bit type, so a cast is needed to calculate a 64-bit size.
--- kernel/events/internal.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/events/internal.h +1 -1
@@ -128,7 +128,7 @@ static inline unsigned long perf_data_size(struct perf_buffer *rb)static inline unsigned long perf_aux_size (struct perf_buffer *rb) { -return rb->aux_nr_pages <<PAGE_SHIFT ; +return (unsigned long )rb->aux_nr_pages <<PAGE_SHIFT ; }#define __DEFINE_OUTPUT_COPY_BODY (advance_buf, memcpy_func, ...) \
Signature · 1 line
2.34.1
[tip: perf/core] perf: Fix perf_aux_size() for greater-than 32-bit size
The following commit has been merged into the perf/core branch of tip:
The following commit has been merged into the perf/core branch of tip:
Commit-ID: 3df94a5b1078dfe2b0c03f027d018800faf44c82 Gitweb: https://git.kernel.org/tip/3df94a5b1078dfe2b0c03f027d018800faf44c82 Author: Adrian Hunter <[email protected]> AuthorDate: Mon, 24 Jun 2024 23:10:58 +03:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Thu, 04 Jul 2024 16:00:22 +02:00
perf: Fix perf_aux_size() for greater-than 32-bit size
perf_buffer->aux_nr_pages uses a 32-bit type, so a cast is needed to calculate a 64-bit size.
--- kernel/events/internal.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/events/internal.h +1 -1
@@ -128,7 +128,7 @@ static inline unsigned long perf_data_size(struct perf_buffer *rb)static inline unsigned long perf_aux_size (struct perf_buffer *rb) { -return rb->aux_nr_pages <<PAGE_SHIFT ; +return (unsigned long )rb->aux_nr_pages <<PAGE_SHIFT ; }#define __DEFINE_OUTPUT_COPY_BODY (advance_buf, memcpy_func, ...) \
Adrian Hunter
reviewerPATCH 5/7 perf: Prevent passing zero nr_pages to rb_alloc_aux()
In reply to Adrian Hunternr_pages is unsigned long but gets passed to rb_alloc_aux() as an int, and is stored as an int.
Only power-of-2 values are accepted, so if nr_pages is a 64_bit value, it will be passed to rb_alloc_aux() as zero.
That is not ideal because:
1. the value is incorrect
2. rb_alloc_aux() is at risk of misbehaving, although it manages to return -ENOMEM in that case, it is a result of passing zero to get_order() even though the get_order() result is documented to be undefined in that case.
Fix by simply validating the maximum supported value in the first place. Use -ENOMEM error code for consistency with the current error code that is returned in that case.
--- kernel/events/core.c | 2 ++ 1 file changed, 2 insertions(+)
kernel/events/core.c +2 -0
@@ -6509,6 +6509,8 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)return -EINVAL ; nr_pages = vma_size /PAGE_SIZE ; +if (nr_pages >INT_MAX ) +return -ENOMEM ;mutex_lock (&event->mmap_mutex); ret = -EINVAL ;
Signature · 1 line
2.34.1
[tip: perf/core] perf: Prevent passing zero nr_pages to rb_alloc_aux()
The following commit has been merged into the perf/core branch of tip:
The following commit has been merged into the perf/core branch of tip:
Commit-ID: dbc48c8f41c208082cfa95e973560134489e3309 Gitweb: https://git.kernel.org/tip/dbc48c8f41c208082cfa95e973560134489e3309 Author: Adrian Hunter <[email protected]> AuthorDate: Mon, 24 Jun 2024 23:10:59 +03:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Thu, 04 Jul 2024 16:00:22 +02:00
perf: Prevent passing zero nr_pages to rb_alloc_aux()
nr_pages is unsigned long but gets passed to rb_alloc_aux() as an int, and is stored as an int.
Only power-of-2 values are accepted, so if nr_pages is a 64_bit value, it will be passed to rb_alloc_aux() as zero.
That is not ideal because:
1. the value is incorrect
2. rb_alloc_aux() is at risk of misbehaving, although it manages to return -ENOMEM in that case, it is a result of passing zero to get_order() even though the get_order() result is documented to be undefined in that case.
Fix by simply validating the maximum supported value in the first place. Use -ENOMEM error code for consistency with the current error code that is returned in that case.
--- kernel/events/core.c | 2 ++ 1 file changed, 2 insertions(+)
kernel/events/core.c +2 -0
@@ -6496,6 +6496,8 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)return -EINVAL ; nr_pages = vma_size /PAGE_SIZE ; +if (nr_pages >INT_MAX ) +return -ENOMEM ;mutex_lock (&event->mmap_mutex); ret = -EINVAL ;
Adrian Hunter
reviewerPATCH 6/7 perf: Fix default aux_watermark calculation
In reply to Adrian HunterThe default aux_watermark is half the AUX area buffer size. In general, on a 64-bit architecture, the AUX area buffer size could be a bigger than fits in a 32-bit type, but the calculation does not allow for that possibility.
However the aux_watermark value is recorded in a u32, so should not be more than U32_MAX either.
Fix by doing the calculation in a correctly sized type, and limiting the result to U32_MAX.
--- kernel/events/ring_buffer.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/events/ring_buffer.c +3 -1
@@ -688,7 +688,9 @@ int rb_alloc_aux(struct perf_buffer *rb, struct perf_event *event,* max_order, to aid PMU drivers in double buffering. */ if (!watermark) - watermark = nr_pages << (PAGE_SHIFT -1 ); + watermark = min_t(unsigned long, +U32_MAX , + (unsigned long )nr_pages << (PAGE_SHIFT -1 ));/* * Use aux_watermark as the basis for chunking to
Signature · 1 line
2.34.1
[tip: perf/core] perf: Fix default aux_watermark calculation
The following commit has been merged into the perf/core branch of tip:
The following commit has been merged into the perf/core branch of tip:
Commit-ID: 43deb76b19663a96ec2189d8f4eb9a9dc2d7623f Gitweb: https://git.kernel.org/tip/43deb76b19663a96ec2189d8f4eb9a9dc2d7623f Author: Adrian Hunter <[email protected]> AuthorDate: Mon, 24 Jun 2024 23:11:00 +03:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Thu, 04 Jul 2024 16:00:23 +02:00
perf: Fix default aux_watermark calculation
The default aux_watermark is half the AUX area buffer size. In general, on a 64-bit architecture, the AUX area buffer size could be a bigger than fits in a 32-bit type, but the calculation does not allow for that possibility.
However the aux_watermark value is recorded in a u32, so should not be more than U32_MAX either.
Fix by doing the calculation in a correctly sized type, and limiting the result to U32_MAX.
--- kernel/events/ring_buffer.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/events/ring_buffer.c +3 -1
@@ -688,7 +688,9 @@ int rb_alloc_aux(struct perf_buffer *rb, struct perf_event *event,* max_order, to aid PMU drivers in double buffering. */ if (!watermark) - watermark = nr_pages << (PAGE_SHIFT -1 ); + watermark = min_t(unsigned long, +U32_MAX , + (unsigned long )nr_pages << (PAGE_SHIFT -1 ));/* * Use aux_watermark as the basis for chunking to
Adrian Hunter
reviewerPATCH 7/7 perf: Make rb_alloc_aux() return an error immediately if nr_pages <= 0
In reply to Adrian Hunterrb_alloc_aux() should not be called with nr_pages <= 0. Make it more robust and readable by returning an error immediately in that case.
--- kernel/events/ring_buffer.c | 3 +++ 1 file changed, 3 insertions(+)
kernel/events/ring_buffer.c +3 -0
@@ -682,6 +682,9 @@ int rb_alloc_aux(struct perf_buffer *rb, struct perf_event *event,if (!has_aux (event))return -EOPNOTSUPP ; +if (nr_pages <=0 ) +return -EINVAL ; +if (!overwrite) {/* * Watermark defaults to half the buffer, and so does the
Signature · 1 line
2.34.1
[tip: perf/core] perf: Make rb_alloc_aux() return an error immediately if nr_pages <= 0
The following commit has been merged into the perf/core branch of tip:
The following commit has been merged into the perf/core branch of tip:
Commit-ID: 0ca4da2412da05fb9dd0b5d90dcc8026219f0f29 Gitweb: https://git.kernel.org/tip/0ca4da2412da05fb9dd0b5d90dcc8026219f0f29 Author: Adrian Hunter <[email protected]> AuthorDate: Mon, 24 Jun 2024 23:11:01 +03:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Thu, 04 Jul 2024 16:00:23 +02:00
perf: Make rb_alloc_aux() return an error immediately if nr_pages <= 0
rb_alloc_aux() should not be called with nr_pages <= 0. Make it more robust and readable by returning an error immediately in that case.
--- kernel/events/ring_buffer.c | 3 +++ 1 file changed, 3 insertions(+)
kernel/events/ring_buffer.c +3 -0
@@ -682,6 +682,9 @@ int rb_alloc_aux(struct perf_buffer *rb, struct perf_event *event,if (!has_aux (event))return -EOPNOTSUPP ; +if (nr_pages <=0 ) +return -EINVAL ; +if (!overwrite) {/* * Watermark defaults to half the buffer, and so does the