Skip to contentSkip to conversation

pgo: Fix allocate_node() handling of non-vmlinux nodes

PATCH In review Jarmo Tiitto 2 participants 4 messages Jun 2, 2021

lore
Share & tools
lore
#0 lore raw

cover pgo: Fix allocate_node() handling of non-vmlinux nodes

Currently allocate_node() will reserve nodes even if *p doesn't point into __llvm_prf_data_start - __llvm_prf_data_end range.

This is wrong:

Any instrumented modules that are not part of vmlinux (built-in) will "steal" available nodes away from vmlinux. This can result in exhaustion of nodes for vmlinux and less accurate profile data.

Either way, any profiling data generated by loaded modules, if any, is unusable from vmlinux.profraw.
So just filtter them out.

Jarmo Tiitto (1):
pgo: Fix allocate_node() handling of non-vmlinux nodes.

 kernel/pgo/instrument.c | 4 ++++
 1 file changed, 4 insertions(+)
Signature · 1 line
2.31.1
#1 lore raw

PATCH 1/1 pgo: Fix allocate_node() handling of non-vmlinux nodes.

Currently allocate_node() will reserve nodes even if *p doesn't point into __llvm_prf_data_start - __llvm_prf_data_end range.

Fix it by checking if p points into vmlinux range
and otherwise return NULL.

---
 kernel/pgo/instrument.c | 4 ++++
 1 file changed, 4 insertions(+)
kernel/pgo/instrument.c +4 -0
index 0e07ee1b17d9..9bca535dfa91 100644
@@ -55,6 +55,10 @@ void prf_unlock(unsigned long flags)
 static struct llvm_prf_value_node *allocate_node(struct llvm_prf_data *p,
 						 u32 index, u64 value)
 {
+	/* check if p points into vmlinux. If not, don't allocate. */
+	if (p < __llvm_prf_data_start || p >= __llvm_prf_data_end)
+		return NULL;
+
 	if (&__llvm_prf_vnds_start[current_node + 1] >= __llvm_prf_vnds_end)
 		return NULL; /* Out of nodes */
 
Signature · 1 line
2.31.1
#2 lore raw

Re: [PATCH 1/1] pgo: Fix allocate_node() handling of non-vmlinux nodes.

Quoted 17 lines · Jarmo Tiitto

Currently allocate_node() will reserve nodes even if *p doesn't point into __llvm_prf_data_start - __llvm_prf_data_end range.

Fix it by checking if p points into vmlinux range
and otherwise return NULL.

---
 kernel/pgo/instrument.c | 4 ++++
 1 file changed, 4 insertions(+)
kernel/pgo/instrument.c +0 -0
index 0e07ee1b17d9..9bca535dfa91 100644

kernel/pgo/instrument.c void prf_unlock(unsigned long flags) · line 60

Quoted 6 lines
kernel/pgo/instrument.c +3 -0
@@ -55,6 +55,10 @@ void prf_unlock(unsigned long flags)
 static struct llvm_prf_value_node *allocate_node(struct llvm_prf_data *p,
 						 u32 index, u64 value)
 {
+	/* check if p points into vmlinux. If not, don't allocate. */
+	if (p < __llvm_prf_data_start || p >= __llvm_prf_data_end)
+		return NULL;

This should be a tighter check (struct llvm_prf_data has size, so just checking for p < __llvm_prf_data_end isn't sufficient. I recommend using the memory_contains() helper.

And I think this should be louder as it's entirely unexpected right now. Perhaps:

	if (WARN_ON_ONCE(!memory_contains(__llvm_prf_data_start,
					  __llvm_prf_data_end,
					  p, sizeof(*p))))
		return NULL;
Quoted 7 lines
kernel/pgo/instrument.c +1 -0
+
 	if (&__llvm_prf_vnds_start[current_node + 1] >= __llvm_prf_vnds_end)
 		return NULL; /* Out of nodes */
 
Signature · 1 line
2.31.1
Signature · 1 line
Kees Cook
#3 lore raw

Re: [PATCH 1/1] pgo: Fix allocate_node() handling of non-vmlinux nodes.

Kees Cook wrote keskiviikkona 2. kesäkuuta 2021 20.41.28 EEST:

Quoted 37 lines · Jarmo Tiitto
Quoted 17 lines

Currently allocate_node() will reserve nodes even if *p doesn't point into __llvm_prf_data_start - __llvm_prf_data_end range.

Fix it by checking if p points into vmlinux range
and otherwise return NULL.

---
 kernel/pgo/instrument.c | 4 ++++
 1 file changed, 4 insertions(+)
kernel/pgo/instrument.c +0 -0
index 0e07ee1b17d9..9bca535dfa91 100644

kernel/pgo/instrument.c void prf_unlock(unsigned long flags) · line 60

Quoted 6 lines
kernel/pgo/instrument.c +3 -0
@@ -55,6 +55,10 @@ void prf_unlock(unsigned long flags)
 static struct llvm_prf_value_node *allocate_node(struct llvm_prf_data *p,
 						 u32 index, u64 value)
 {
+	/* check if p points into vmlinux. If not, don't allocate. */
+	if (p < __llvm_prf_data_start || p >= __llvm_prf_data_end)
+		return NULL;

This should be a tighter check (struct llvm_prf_data has size, so just checking for p < __llvm_prf_data_end isn't sufficient. I recommend using the memory_contains() helper.

And I think this should be louder as it's entirely unexpected right now. Perhaps:

	if (WARN_ON_ONCE(!memory_contains(__llvm_prf_data_start,
					  __llvm_prf_data_end,
					  p, sizeof(*p))))
		return NULL;
Quoted 11 lines
Quoted 7 lines
kernel/pgo/instrument.c +1 -0
+
 	if (&__llvm_prf_vnds_start[current_node + 1] >= __llvm_prf_vnds_end)
 		return NULL; /* Out of nodes */
 
Signature · 1 line
2.31.1
Signature · 1 line
Kees Cook

Well, if you do that the WARN_ON_ONCE() will always trigger, unless CONFIG_MODULES=n 😇

This is because 'struct llvm_prf_data *p' argument is expected (at least I think so) to point into __llvm_prf_data section in vmlinux and also into each module's own __llvm_prf_data section.

So in the end the compiler supplied pointer is likely correct, but the current v9 PGO patch attempts to reserve all vnodes from vmlinux __llvm_prf_vnds section, instead of respective module section.

I think it would be normal to ignore pointers here, until my module PGO machinery is ready.

But I agree on using memory_contains() for checking if p is within bounds. I will follow on with v2 of this patch. :-)

-Jarmo