Skip to contentSkip to conversation

Perf stack unwinding with pointer authentication

PATCH In review James Clark 4 participants 13 messages Oct 20, 2022 – Mar 4

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v2 cover Perf stack unwinding with pointer authentication

Hi,

I've taken up this change from Andrew and made some improvements. It also didn't apply cleanly anymore so I thought I would resend.

I know there were some discussions with Vince Weaver about having something arch specific in perf_event_open, so it would be good to get some more feedback on that. I'm happy to change it so that it's just called PERF_SAMPLE_PTRAUTH, but given how arch specific the output is, I'm not sure if that is best.

Thanks
James

Changes since V1:

* Use weak symbols instead of #ifdefs which simplified things
* Remove the new files as everything fits well into existing ones
* Never allow opening the event if CONFIG_ARM64_PTR_AUTH isn't set
* Include only the kernel side change in this set, the perf tool changes are here [1] which I will also rebase and resubmit
* Rebase onto tip/perf/core (82aad7ff7ac)

[1]: https://lore.kernel.org/lkml/[email protected]/T/#t

Andrew Kilroy (1):
perf arm64: Send pointer auth masks to ring buffer

 arch/arm64/include/asm/perf_event.h | 32 +++++++++++++++++++++++++++++
 arch/arm64/kernel/perf_event.c      | 32 +++++++++++++++++++++++++++++
 include/linux/perf_event.h          | 12 +++++++++++
 include/uapi/linux/perf_event.h     |  4 +++-
 kernel/events/core.c                | 31 ++++++++++++++++++++++++++++
 5 files changed, 110 insertions(+), 1 deletion(-)
Signature · 1 line
2.28.0
#1 lore raw

PATCH v2 1/1 perf arm64: Send pointer auth masks to ring buffer

From: Andrew Kilroy <[email protected]>

Perf report cannot produce callgraphs using dwarf on arm64 where pointer authentication is enabled. This is because libunwind and libdw cannot unmangle instruction pointers that have a pointer authentication code (PAC) embedded in them.

libunwind and libdw need to be given an instruction mask which they can use to arrive at the correct return address that does not contain the PAC.

The bits in the return address that contain the PAC can differ by process, so this patch adds a new sample field PERF_SAMPLE_ARCH_1 to allow the kernel to send the masks up to userspace perf.

This field can be used in a architecture specific fashion, but on arm64, it contains the ptrauth mask information. The event will currently fail to open on architectures other than arm64 if PERF_SAMPLE_ARCH_1 is set. It will also fail to open on arm64 if CONFIG_ARM64_PTR_AUTH isn't set, as the data would always be zeros.

Cc Vince Weaver <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Peter Zijlstra <[email protected]>
---
 arch/arm64/include/asm/perf_event.h | 32 +++++++++++++++++++++++++++++
 arch/arm64/kernel/perf_event.c      | 32 +++++++++++++++++++++++++++++
 include/linux/perf_event.h          | 12 +++++++++++
 include/uapi/linux/perf_event.h     |  4 +++-
 kernel/events/core.c                | 31 ++++++++++++++++++++++++++++
 5 files changed, 110 insertions(+), 1 deletion(-)
arch/arm64/include/asm/perf_event.h +32 -0
index 3eaf462f5752..160fdb8fca1c 100644
@@ -273,4 +273,36 @@ extern unsigned long perf_misc_flags(struct pt_regs *regs);
 	(regs)->pstate = PSR_MODE_EL1h;	\
 }
 
+#ifdef CONFIG_ARM64_PTR_AUTH
+#define HAS_ARCH_SAMPLE_DATA
+/*
+ * Structure holding masks to help userspace stack unwinding
+ * in the presence of arm64 pointer authentication.
+ */
+struct ptrauth_info {
+	/*
+	 * Bits 0, 1, 2, 3, 4 may be set to on, to indicate which keys are being used
+	 * The APIAKEY, APIBKEY, APDAKEY, APDBKEY, or the APGAKEY respectively.
+	 * Where all bits are off, pointer authentication is not in use for the
+	 * process.
+	 */
+	u64 enabled_keys;
+
+	/*
+	 * The on bits represent which bits in an instruction pointer
+	 * constitute the pointer authentication code.
+	 */
+	u64 insn_mask;
+
+	/*
+	 * The on bits represent which bits in a data pointer constitute the
+	 * pointer authentication code.
+	 */
+	u64 data_mask;
+};
+
+struct arch_sample_data {
+	struct ptrauth_info ptrauth;
+};
+#endif /* CONFIG_ARM64_PTR_AUTH */
 #endif
arch/arm64/kernel/perf_event.c +32 -0
index cb69ff1e6138..9c209168e055 100644
@@ -1459,3 +1459,35 @@ void arch_perf_update_userpage(struct perf_event *event,
 	userpg->cap_user_time_zero = 1;
 	userpg->cap_user_time_short = 1;
 }
+
+#ifdef CONFIG_ARM64_PTR_AUTH
+void perf_output_sample_arch_1(struct perf_output_handle *handle,
+			       struct perf_event_header *header,
+			       struct perf_sample_data *data,
+			       struct perf_event *event)
+{
+	perf_output_put(handle, data->arch.ptrauth.enabled_keys);
+	perf_output_put(handle, data->arch.ptrauth.insn_mask);
+	perf_output_put(handle, data->arch.ptrauth.data_mask);
+}
+
+void perf_prepare_sample_arch_1(struct perf_event_header *header,
+				struct perf_sample_data *data,
+				struct perf_event *event,
+				struct pt_regs *regs)
+{
+	int keys_result = ptrauth_get_enabled_keys(current);
+	u64 user_pac_mask = keys_result > 0 ? ptrauth_user_pac_mask() : 0;
+
+	data->arch.ptrauth.enabled_keys = keys_result > 0 ? keys_result : 0;
+	data->arch.ptrauth.insn_mask = user_pac_mask;
+	data->arch.ptrauth.data_mask = user_pac_mask;
+
+	header->size += (3 * sizeof(u64));
+}
+
+int perf_event_open_request_arch_1(void)
+{
+	return 0;
+}
+#endif /* CONFIG_ARM64_PTR_AUTH */
include/linux/perf_event.h +12 -0
index 853f64b6c8c2..f6b0cc93faae 100644
@@ -1065,6 +1065,9 @@ struct perf_sample_data {
 	u64				cgroup;
 	u64				data_page_size;
 	u64				code_page_size;
+#ifdef HAS_ARCH_SAMPLE_DATA
+	struct arch_sample_data		arch;
+#endif
 } ____cacheline_aligned;
 
 /* default value for data source */
@@ -1674,6 +1677,15 @@ int perf_event_exit_cpu(unsigned int cpu);
 extern void __weak arch_perf_update_userpage(struct perf_event *event,
 					     struct perf_event_mmap_page *userpg,
 					     u64 now);
+extern void perf_output_sample_arch_1(struct perf_output_handle *handle,
+				      struct perf_event_header *header,
+				      struct perf_sample_data *data,
+				      struct perf_event *event);
+extern void perf_prepare_sample_arch_1(struct perf_event_header *header,
+				       struct perf_sample_data *data,
+				       struct perf_event *event,
+				       struct pt_regs *regs);
+extern int perf_event_open_request_arch_1(void);
 
 #ifdef CONFIG_MMU
 extern __weak u64 arch_perf_get_page_size(struct mm_struct *mm, unsigned long addr);
include/uapi/linux/perf_event.h +3 -1
index 85be78e0e7f6..3c8349111422 100644
@@ -162,10 +162,12 @@ enum perf_event_sample_format {
 	PERF_SAMPLE_DATA_PAGE_SIZE		= 1U << 22,
 	PERF_SAMPLE_CODE_PAGE_SIZE		= 1U << 23,
 	PERF_SAMPLE_WEIGHT_STRUCT		= 1U << 24,
+	PERF_SAMPLE_ARCH_1			= 1U << 25,
 
-	PERF_SAMPLE_MAX = 1U << 25,		/* non-ABI */
+	PERF_SAMPLE_MAX = 1U << 26,		/* non-ABI */
 };
 
+#define PERF_SAMPLE_ARM64_PTRAUTH PERF_SAMPLE_ARCH_1
 #define PERF_SAMPLE_WEIGHT_TYPE	(PERF_SAMPLE_WEIGHT | PERF_SAMPLE_WEIGHT_STRUCT)
 /*
  * values to program into branch_sample_type when PERF_SAMPLE_BRANCH is set
kernel/events/core.c +31 -0
index b981b879bcd8..8ca0501d608d 100644
@@ -5822,6 +5822,25 @@ void __weak arch_perf_update_userpage(
 {
 }
 
+void __weak perf_output_sample_arch_1(struct perf_output_handle *handle,
+			       struct perf_event_header *header,
+			       struct perf_sample_data *data,
+			       struct perf_event *event)
+{
+}
+
+void __weak perf_prepare_sample_arch_1(struct perf_event_header *header,
+				struct perf_sample_data *data,
+				struct perf_event *event,
+				struct pt_regs *regs)
+{
+}
+
+int __weak perf_event_open_request_arch_1(void)
+{
+	return -EINVAL;
+}
+
 /*
  * Callers need to ensure there can be no nesting of this function, otherwise
  * the seqlock logic goes bad. We can not serialize this because the arch
@@ -7142,6 +7161,9 @@ void perf_output_sample(struct perf_output_handle *handle,
 			perf_aux_sample_output(event, handle, data);
 	}
 
+	if (sample_type & PERF_SAMPLE_ARCH_1)
+		perf_output_sample_arch_1(handle, header, data, event);
+
 	if (!event->attr.watermark) {
 		int wakeup_events = event->attr.wakeup_events;
 
@@ -7466,6 +7488,9 @@ void perf_prepare_sample(struct perf_event_header *header,
 	if (sample_type & PERF_SAMPLE_CODE_PAGE_SIZE)
 		data->code_page_size = perf_get_page_size(data->ip);
 
+	if (sample_type & PERF_SAMPLE_ARCH_1)
+		perf_prepare_sample_arch_1(header, data, event, regs);
+
 	if (sample_type & PERF_SAMPLE_AUX) {
 		u64 size;
 
@@ -12140,6 +12165,12 @@ SYSCALL_DEFINE5(perf_event_open,
 			return err;
 	}
 
+	if (attr.sample_type & PERF_SAMPLE_ARCH_1) {
+		err = perf_event_open_request_arch_1();
+		if (err)
+			return err;
+	}
+
 	/*
 	 * In cgroup mode, the pid argument is used to pass the fd
 	 * opened to the cgroup directory in cgroupfs. The cpu argument
Signature · 1 line
2.28.0
maintainer
#2 lore raw

Re: [PATCH v2 1/1] perf arm64: Send pointer auth masks to ring buffer

Quoted 15 lines · James Clark

From: Andrew Kilroy <[email protected]>

Perf report cannot produce callgraphs using dwarf on arm64 where pointer authentication is enabled. This is because libunwind and libdw cannot unmangle instruction pointers that have a pointer authentication code (PAC) embedded in them.

libunwind and libdw need to be given an instruction mask which they can use to arrive at the correct return address that does not contain the PAC.

The bits in the return address that contain the PAC can differ by process, so this patch adds a new sample field PERF_SAMPLE_ARCH_1 to allow the kernel to send the masks up to userspace perf.

Quoted 5 lines

This field can be used in a architecture specific fashion, but on arm64, it contains the ptrauth mask information. The event will currently fail to open on architectures other than arm64 if PERF_SAMPLE_ARCH_1 is set. It will also fail to open on arm64 if CONFIG_ARM64_PTR_AUTH isn't set, as the data would always be zeros.

A little more information please; wth is pointer authentication? Are we going to be having the same thing with x86 LAM where only a subset of the available bits have meaning to the hardware?

Why do we want the same mask repeated over and over with each sample; should this not be part of the address space (side-band) data?

maintainer
#3 lore raw

Re: [PATCH v2 1/1] perf arm64: Send pointer auth masks to ring buffer

Quoted 16 lines · Peter Zijlstra
Quoted 15 lines · James Clark

From: Andrew Kilroy <[email protected]>

Perf report cannot produce callgraphs using dwarf on arm64 where pointer authentication is enabled. This is because libunwind and libdw cannot unmangle instruction pointers that have a pointer authentication code (PAC) embedded in them.

libunwind and libdw need to be given an instruction mask which they can use to arrive at the correct return address that does not contain the PAC.

The bits in the return address that contain the PAC can differ by process, so this patch adds a new sample field PERF_SAMPLE_ARCH_1 to allow the kernel to send the masks up to userspace perf.

Quoted 7 lines
Quoted 5 lines

This field can be used in a architecture specific fashion, but on arm64, it contains the ptrauth mask information. The event will currently fail to open on architectures other than arm64 if PERF_SAMPLE_ARCH_1 is set. It will also fail to open on arm64 if CONFIG_ARM64_PTR_AUTH isn't set, as the data would always be zeros.

A little more information please; wth is pointer authentication? Are we

Mark got me: https://events.static.linuxfound.org/sites/events/files/slides/slides_23.pdf

Quoted 5 lines

going to be having the same thing with x86 LAM where only a subset of the available bits have meaning to the hardware?

Why do we want the same mask repeated over and over with each sample; should this not be part of the address space (side-band) data?

#4 lore raw

Re: [PATCH v2 1/1] perf arm64: Send pointer auth masks to ring buffer

In reply to Peter Zijlstra
Quoted 16 lines · Peter Zijlstra
Quoted 15 lines · James Clark

From: Andrew Kilroy <[email protected]>

Perf report cannot produce callgraphs using dwarf on arm64 where pointer authentication is enabled. This is because libunwind and libdw cannot unmangle instruction pointers that have a pointer authentication code (PAC) embedded in them.

libunwind and libdw need to be given an instruction mask which they can use to arrive at the correct return address that does not contain the PAC.

The bits in the return address that contain the PAC can differ by process, so this patch adds a new sample field PERF_SAMPLE_ARCH_1 to allow the kernel to send the masks up to userspace perf.

Quoted 7 lines
Quoted 5 lines

This field can be used in a architecture specific fashion, but on arm64, it contains the ptrauth mask information. The event will currently fail to open on architectures other than arm64 if PERF_SAMPLE_ARCH_1 is set. It will also fail to open on arm64 if CONFIG_ARM64_PTR_AUTH isn't set, as the data would always be zeros.

A little more information please; wth is pointer authentication? Are we

Yes the commit message could do with some links and details. I will add more background about the feature if I resend it.

Quoted 2 lines

going to be having the same thing with x86 LAM where only a subset of the available bits have meaning to the hardware?

If LAM is used with things like return addresses on the stack then yes. For data addresses then it wouldn't cause this exact issue.

The problem is that libunwind and perf do pointer arithmetic when unwinding with other values that don't have these bits set. For example computing offset in an mmap where the base address doesn't have an authentication code but a return value does results in an incorrect value. Like x-y=offset is only correct if the PAC bits are masked out first.

Quoted 3 lines

Why do we want the same mask repeated over and over with each sample; should this not be part of the address space (side-band) data?

You are probably right that it could be done that way. The reason that we did it this way was to be consistent with ptrace feature [1] where it is delivered to userspace on a per-process basis. And there is also a prctl for the enabled key types [2] which can be changed dynamically. Particularly for the last reason is why it was done per sample.

Having said that, the enabled keys field is not used by perf, only the mask is used, so I can drop the per sample data until enabled keys is needed, which may be never.

I'm going to assume that perf shouldn't use ptrace because of permissions and conflicts with debuggers, so I could put the mask somewhere like PERF_RECORD_FORK instead of per sample.

[1]:
https://lore.kernel.org/all/[email protected]/
[2]:
https://lore.kernel.org/all/d6609065f8f40397a4124654eb68c9f490b4d477.1616123271.git.pcc@google.com/

#5 lore raw

Re: [PATCH v2 1/1] perf arm64: Send pointer auth masks to ring buffer

Quoted 53 lines · James Clark
Quoted 16 lines · Peter Zijlstra
Quoted 15 lines · James Clark

From: Andrew Kilroy <[email protected]>

Perf report cannot produce callgraphs using dwarf on arm64 where pointer authentication is enabled. This is because libunwind and libdw cannot unmangle instruction pointers that have a pointer authentication code (PAC) embedded in them.

libunwind and libdw need to be given an instruction mask which they can use to arrive at the correct return address that does not contain the PAC.

The bits in the return address that contain the PAC can differ by process, so this patch adds a new sample field PERF_SAMPLE_ARCH_1 to allow the kernel to send the masks up to userspace perf.

Quoted 7 lines
Quoted 5 lines

This field can be used in a architecture specific fashion, but on arm64, it contains the ptrauth mask information. The event will currently fail to open on architectures other than arm64 if PERF_SAMPLE_ARCH_1 is set. It will also fail to open on arm64 if CONFIG_ARM64_PTR_AUTH isn't set, as the data would always be zeros.

A little more information please; wth is pointer authentication? Are we

Yes the commit message could do with some links and details. I will add more background about the feature if I resend it.

Quoted 2 lines

going to be having the same thing with x86 LAM where only a subset of the available bits have meaning to the hardware?

If LAM is used with things like return addresses on the stack then yes. For data addresses then it wouldn't cause this exact issue.

The problem is that libunwind and perf do pointer arithmetic when unwinding with other values that don't have these bits set. For example computing offset in an mmap where the base address doesn't have an authentication code but a return value does results in an incorrect value. Like x-y=offset is only correct if the PAC bits are masked out first.

Quoted 3 lines

Why do we want the same mask repeated over and over with each sample; should this not be part of the address space (side-band) data?

You are probably right that it could be done that way. The reason that we did it this way was to be consistent with ptrace feature [1] where it is delivered to userspace on a per-process basis. And there is also a prctl for the enabled key types [2] which can be changed dynamically. Particularly for the last reason is why it was done per sample.

Quoted 8 lines

Having said that, the enabled keys field is not used by perf, only the mask is used, so I can drop the per sample data until enabled keys is needed, which may be never.

I'm going to assume that perf shouldn't use ptrace because of permissions and conflicts with debuggers, so I could put the mask somewhere like PERF_RECORD_FORK instead of per sample.

Hi Peter,

Sorry for flip flopping, but I've read those threads that I linked and spoke with Kristina and we would like to stick with the per sample implementation after all.

The reason is that in the future there may also be a prctrl for 48/52 bit userspace addressing which would change the mask dynamically in the same way as the enabled keys. Although this isn't possible now it makes sense to do it this way in case of that, and also for consistency with the ptrace feature.

I also think that repeating the mask in this case has a very low impact because if you are doing dwarf unwinding, then the whole stack is saved anyway, so a few extra u64s wouldn't be noticeable.

Are you ok with that and for me to resubmit with the expanded commit message?

Thanks
James

Quoted 5 lines

[1]:
https://lore.kernel.org/all/[email protected]/
[2]:
https://lore.kernel.org/all/d6609065f8f40397a4124654eb68c9f490b4d477.1616123271.git.pcc@google.com/

maintainer
#6 lore raw

Re: [PATCH v2 1/1] perf arm64: Send pointer auth masks to ring buffer

Quoted 10 lines · James Clark

Sorry for flip flopping, but I've read those threads that I linked and spoke with Kristina and we would like to stick with the per sample implementation after all.

The reason is that in the future there may also be a prctrl for 48/52 bit userspace addressing which would change the mask dynamically in the same way as the enabled keys. Although this isn't possible now it makes sense to do it this way in case of that, and also for consistency with the ptrace feature.

Quoted 6 lines

I also think that repeating the mask in this case has a very low impact because if you are doing dwarf unwinding, then the whole stack is saved anyway, so a few extra u64s wouldn't be noticeable.

Are you ok with that and for me to resubmit with the expanded commit message?

But you can send a side-band thing around if/when it changes. Same as all the other stuff. We don't include MMAP data with each event either.

Yes, the DWARF thing is slow as anything. but imagine needing this for something else as well, and then you're stuck with it.

maintainer
#7 lore raw

Re: [PATCH v2 1/1] perf arm64: Send pointer auth masks to ring buffer

In reply to James Clark
Quoted 9 lines · James Clark
Quoted 2 lines

Why do we want the same mask repeated over and over with each sample; should this not be part of the address space (side-band) data?

You are probably right that it could be done that way. The reason that we did it this way was to be consistent with ptrace feature [1] where it is delivered to userspace on a per-process basis. And there is also a prctl for the enabled key types [2] which can be changed dynamically. Particularly for the last reason is why it was done per sample.

Quoted 7 lines

Having said that, the enabled keys field is not used by perf, only the mask is used, so I can drop the per sample data until enabled keys is needed, which may be never.

I'm going to assume that perf shouldn't use ptrace because of permissions and conflicts with debuggers, so I could put the mask somewhere like PERF_RECORD_FORK instead of per sample.

Yeah, or create an new RECORD type which you can also send around at prctl() time.

The only thing that's needed on top of that is exposing the mask somewhere in /proc for existing tasks; which is what perf also uses to syntesize RECORD_MMAP events on startup etc..

#8 lore raw

Re: [PATCH v2 1/1] perf arm64: Send pointer auth masks to ring buffer

Quoted 19 lines · Peter Zijlstra
Quoted 9 lines · James Clark
Quoted 2 lines

Why do we want the same mask repeated over and over with each sample; should this not be part of the address space (side-band) data?

You are probably right that it could be done that way. The reason that we did it this way was to be consistent with ptrace feature [1] where it is delivered to userspace on a per-process basis. And there is also a prctl for the enabled key types [2] which can be changed dynamically. Particularly for the last reason is why it was done per sample.

Quoted 7 lines

Having said that, the enabled keys field is not used by perf, only the mask is used, so I can drop the per sample data until enabled keys is needed, which may be never.

I'm going to assume that perf shouldn't use ptrace because of permissions and conflicts with debuggers, so I could put the mask somewhere like PERF_RECORD_FORK instead of per sample.

Quoted 7 lines

Yeah, or create an new RECORD type which you can also send around at prctl() time.

The only thing that's needed on top of that is exposing the mask somewhere in /proc for existing tasks; which is what perf also uses to syntesize RECORD_MMAP events on startup etc..

Hmm ok, in that case I can just add the /proc interface for now because the mask won't change and we can add the new record type at the point it's needed in the future.

Thanks for the feedback.

#9 lore raw

Re: [PATCH v2 1/1] perf arm64: Send pointer auth masks to ring buffer

Quoted 30 lines · James Clark
Quoted 19 lines · Peter Zijlstra
Quoted 9 lines · James Clark
Quoted 2 lines

Why do we want the same mask repeated over and over with each sample; should this not be part of the address space (side-band) data?

You are probably right that it could be done that way. The reason that we did it this way was to be consistent with ptrace feature [1] where it is delivered to userspace on a per-process basis. And there is also a prctl for the enabled key types [2] which can be changed dynamically. Particularly for the last reason is why it was done per sample.

Quoted 7 lines

Having said that, the enabled keys field is not used by perf, only the mask is used, so I can drop the per sample data until enabled keys is needed, which may be never.

I'm going to assume that perf shouldn't use ptrace because of permissions and conflicts with debuggers, so I could put the mask somewhere like PERF_RECORD_FORK instead of per sample.

Quoted 7 lines

Yeah, or create an new RECORD type which you can also send around at prctl() time.

The only thing that's needed on top of that is exposing the mask somewhere in /proc for existing tasks; which is what perf also uses to syntesize RECORD_MMAP events on startup etc..

Quoted 5 lines

Hmm ok, in that case I can just add the /proc interface for now because the mask won't change and we can add the new record type at the point it's needed in the future.

Thanks for the feedback.

Hi all,

Does anyone know how this issue was eventually resolved? Specifically, is the mask for the pointer authentication bits currently exposed and stored somewhere in perf.data so it can be used when running perf report?

Thanks,
Jakub

reviewer
#10 lore raw

Re: [PATCH v2 1/1] perf arm64: Send pointer auth masks to ring buffer

Quoted 39 lines · Jakub Brnak
Quoted 30 lines · James Clark
Quoted 19 lines · Peter Zijlstra
Quoted 9 lines · James Clark
Quoted 2 lines

Why do we want the same mask repeated over and over with each sample; should this not be part of the address space (side-band) data?

You are probably right that it could be done that way. The reason that we did it this way was to be consistent with ptrace feature [1] where it is delivered to userspace on a per-process basis. And there is also a prctl for the enabled key types [2] which can be changed dynamically. Particularly for the last reason is why it was done per sample.

Quoted 7 lines

Having said that, the enabled keys field is not used by perf, only the mask is used, so I can drop the per sample data until enabled keys is needed, which may be never.

I'm going to assume that perf shouldn't use ptrace because of permissions and conflicts with debuggers, so I could put the mask somewhere like PERF_RECORD_FORK instead of per sample.

Quoted 7 lines

Yeah, or create an new RECORD type which you can also send around at prctl() time.

The only thing that's needed on top of that is exposing the mask somewhere in /proc for existing tasks; which is what perf also uses to syntesize RECORD_MMAP events on startup etc..

Quoted 5 lines

Hmm ok, in that case I can just add the /proc interface for now because the mask won't change and we can add the new record type at the point it's needed in the future.

Thanks for the feedback.

Hi all,

Quoted 9 lines

Does anyone know how this issue was eventually resolved? Specifically, is the mask for the pointer authentication bits currently exposed and stored somewhere in perf.data so it can be used when running perf report?

Thanks,
Jakub

Hi Jakub,

No we never finished this, you're actually the first person to mention it. Can you give a bit more detail about your use case? Maybe we can dig out the old branches and work on it again.

For a summary of the current status, frame pointer unwinds in the kernel are working with pointer auth because the kernel strips them.

We patched libunwind for support on commit f67ef2867bc5 ("Unwind with pointer authentication on arm64"). And libdw on commit 0af6c7289c22 (libdwfl, aarch64: Demangle return addresses using a PAC mask). These two were to support dwarf unwinding, but the kernel changes to expose the mask and Perf changes to pass it to the unwinders were never finished.

Thanks
James

#11 lore raw

Re: [PATCH v2 1/1] perf arm64: Send pointer auth masks to ring buffer

Quoted 61 lines · James Clark
Quoted 39 lines · Jakub Brnak
Quoted 30 lines · James Clark
Quoted 19 lines · Peter Zijlstra
Quoted 9 lines · James Clark
Quoted 2 lines

Why do we want the same mask repeated over and over with each sample; should this not be part of the address space (side-band) data?

You are probably right that it could be done that way. The reason that we did it this way was to be consistent with ptrace feature [1] where it is delivered to userspace on a per-process basis. And there is also a prctl for the enabled key types [2] which can be changed dynamically. Particularly for the last reason is why it was done per sample.

Quoted 7 lines

Having said that, the enabled keys field is not used by perf, only the mask is used, so I can drop the per sample data until enabled keys is needed, which may be never.

I'm going to assume that perf shouldn't use ptrace because of permissions and conflicts with debuggers, so I could put the mask somewhere like PERF_RECORD_FORK instead of per sample.

Quoted 7 lines

Yeah, or create an new RECORD type which you can also send around at prctl() time.

The only thing that's needed on top of that is exposing the mask somewhere in /proc for existing tasks; which is what perf also uses to syntesize RECORD_MMAP events on startup etc..

Quoted 5 lines

Hmm ok, in that case I can just add the /proc interface for now because the mask won't change and we can add the new record type at the point it's needed in the future.

Thanks for the feedback.

Hi all,

Quoted 9 lines

Does anyone know how this issue was eventually resolved? Specifically, is the mask for the pointer authentication bits currently exposed and stored somewhere in perf.data so it can be used when running perf report?

Thanks,
Jakub

Hi Jakub,

No we never finished this, you're actually the first person to mention it. Can you give a bit more detail about your use case? Maybe we can dig out the old branches and work on it again.

For a summary of the current status, frame pointer unwinds in the kernel are working with pointer auth because the kernel strips them.

Quoted 9 lines

We patched libunwind for support on commit f67ef2867bc5 ("Unwind with pointer authentication on arm64"). And libdw on commit 0af6c7289c22 (libdwfl, aarch64: Demangle return addresses using a PAC mask). These two were to support dwarf unwinding, but the kernel changes to expose the mask and Perf changes to pass it to the unwinders were never finished.

Thanks
James

Hi James,

Thanks for the answer. I was investigating a failure in one of the perf tests on AArch64 RHEL-10 that verifies stack unwinding using DWARF, and it was consistently failing. I dug a bit deeper and discovered that DWARF unwinding in perf report does not work correctly at all, and addresses polluted by PAC bits are likely the cause. I stepped through the program and observed that in libdw the code path responsible for masking these bits is executed, but the mask is always set to 0 and therefore has no effect. During the unwinding process, there is a structure that is supposed to store the mask and use it later. However, since perf constructs the unwinding context offline from samples and currently does not store the mask anywhere in perf.data, there is likely a no way to propagate it into that structure. It would be nice if we could push this forward.

Thanks,
Jakub

reviewer
#12 lore raw

Re: [PATCH v2 1/1] perf arm64: Send pointer auth masks to ring buffer

Quoted 85 lines · Jakub Brnak
Quoted 61 lines · James Clark
Quoted 39 lines · Jakub Brnak
Quoted 30 lines · James Clark
Quoted 19 lines · Peter Zijlstra
Quoted 9 lines · James Clark
Quoted 2 lines

Why do we want the same mask repeated over and over with each sample; should this not be part of the address space (side-band) data?

You are probably right that it could be done that way. The reason that we did it this way was to be consistent with ptrace feature [1] where it is delivered to userspace on a per-process basis. And there is also a prctl for the enabled key types [2] which can be changed dynamically. Particularly for the last reason is why it was done per sample.

Quoted 7 lines

Having said that, the enabled keys field is not used by perf, only the mask is used, so I can drop the per sample data until enabled keys is needed, which may be never.

I'm going to assume that perf shouldn't use ptrace because of permissions and conflicts with debuggers, so I could put the mask somewhere like PERF_RECORD_FORK instead of per sample.

Quoted 7 lines

Yeah, or create an new RECORD type which you can also send around at prctl() time.

The only thing that's needed on top of that is exposing the mask somewhere in /proc for existing tasks; which is what perf also uses to syntesize RECORD_MMAP events on startup etc..

Quoted 5 lines

Hmm ok, in that case I can just add the /proc interface for now because the mask won't change and we can add the new record type at the point it's needed in the future.

Thanks for the feedback.

Hi all,

Quoted 9 lines

Does anyone know how this issue was eventually resolved? Specifically, is the mask for the pointer authentication bits currently exposed and stored somewhere in perf.data so it can be used when running perf report?

Thanks,
Jakub

Hi Jakub,

No we never finished this, you're actually the first person to mention it. Can you give a bit more detail about your use case? Maybe we can dig out the old branches and work on it again.

For a summary of the current status, frame pointer unwinds in the kernel are working with pointer auth because the kernel strips them.

Quoted 9 lines

We patched libunwind for support on commit f67ef2867bc5 ("Unwind with pointer authentication on arm64"). And libdw on commit 0af6c7289c22 (libdwfl, aarch64: Demangle return addresses using a PAC mask). These two were to support dwarf unwinding, but the kernel changes to expose the mask and Perf changes to pass it to the unwinders were never finished.

Thanks
James

Hi James,

Thanks for the answer. I was investigating a failure in one of the perf tests on AArch64 RHEL-10 that verifies stack unwinding using DWARF, and it was consistently failing. I dug a bit deeper and discovered that DWARF unwinding in perf report does not work correctly at all, and addresses polluted by PAC bits are likely the cause. I stepped through the program and observed that in libdw the code path responsible for masking these bits is executed, but the mask is always set to 0 and therefore has no effect. During the unwinding process, there is a structure that is supposed to store the mask and use it later. However, since perf constructs the unwinding context offline from samples and currently does not store the mask anywhere in perf.data, there is likely a no way to propagate it into that structure. It would be nice if we could push this forward.

Quoted 3 lines

Thanks,
Jakub

Sure, I can put it back on the todo list.