Skip to contentSkip to conversation

samples: bpf: fix bpf programs with kprobe/sys_connect event

PATCH In review Daniel T. Lee 2 participants 10 messages Jun 26, 2020

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/3 samples: bpf: fix bpf programs with kprobe/sys_connect event

Currently, BPF programs with kprobe/sys_connect does not work properly.

Commit 34745aed515c ("samples/bpf: fix kprobe attachment issue on x64") This commit modifies the bpf_load behavior of kprobe events in the x64 architecture. If the current kprobe event target starts with "sys_*", add the prefix "__x64_" to the front of the event.

Appending "__x64_" prefix with kprobe/sys_* event was appropriate as a solution to most of the problems caused by the commit below.

commit d5a00528b58c ("syscalls/core, syscalls/x86: Rename struct pt_regs-based sys_*() to __x64_sys_*()")

However, there is a problem with the sys_connect kprobe event that does not work properly. For __sys_connect event, parameters can be fetched normally, but for __x64_sys_connect, parameters cannot be fetched.

Because of this problem, this commit fixes the sys_connect event by specifying the __sys_connect directly and this will bypass the "__x64_" appending rule of bpf_load.

Fixes 34745aed515c ("samples/bpf: fix kprobe attachment issue on x64")
---
 samples/bpf/map_perf_test_kern.c         | 2 +-
 samples/bpf/test_map_in_map_kern.c       | 2 +-
 samples/bpf/test_probe_write_user_kern.c | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
samples/bpf/map_perf_test_kern.c +1 -1
index 1bb1fdb9cdf8..69ecd717d998 100644
@@ -159,7 +159,7 @@ int stress_percpu_hmap_alloc(struct pt_regs *ctx)
 	return 0;
 }
 
-SEC("kprobe/sys_connect")
+SEC("kprobe/__sys_connect")
 int stress_lru_hmap_alloc(struct pt_regs *ctx)
 {
 	char fmt[] = "Failed at stress_lru_hmap_alloc. ret:%dn";
samples/bpf/test_map_in_map_kern.c +1 -1
index 6cee61e8ce9b..b1562ba2f025 100644
@@ -102,7 +102,7 @@ static __always_inline int do_inline_hash_lookup(void *inner_map, u32 port)
 	return result ? *result : -ENOENT;
 }
 
-SEC("kprobe/sys_connect")
+SEC("kprobe/__sys_connect")
 int trace_sys_connect(struct pt_regs *ctx)
 {
 	struct sockaddr_in6 *in6;
samples/bpf/test_probe_write_user_kern.c +1 -1
index 6579639a83b2..9b3c3918c37d 100644
@@ -26,7 +26,7 @@ struct {
  * This example sits on a syscall, and the syscall ABI is relatively stable
  * of course, across platforms, and over time, the ABI may change.
  */
-SEC("kprobe/sys_connect")
+SEC("kprobe/__sys_connect")
 int bpf_prog1(struct pt_regs *ctx)
 {
 	struct sockaddr_in new_addr, orig_addr = {};
Signature · 1 line
2.25.1
#1 lore raw

PATCH 2/3 samples: bpf: cleanup pointer error check with libbpf

Libbpf has its own helper function to check for errors in the bpf data structure (pointer). And Some codes do not use this libbbpf helper function and check the pointer's error directly.

This commit clean up the existing pointer error check logic with libbpf.

---
 samples/bpf/sampleip_user.c    | 2 +-
 samples/bpf/trace_event_user.c | 2 +-
 samples/bpf/tracex1_user.c     | 2 +-
 samples/bpf/tracex5_user.c     | 2 +-
 samples/bpf/tracex7_user.c     | 2 +-
 5 files changed, 5 insertions(+), 5 deletions(-)
samples/bpf/sampleip_user.c +1 -1
index 921c505bb567..554dfa1cb34d 100644
@@ -186,7 +186,7 @@ int main(int argc, char **argv)
 	}
 
 	prog = bpf_object__find_program_by_name(obj, "do_sample");
-	if (!prog) {
+	if (libbpf_get_error(prog)) {
 		fprintf(stderr, "ERROR: finding a prog in obj file failed\n");
 		goto cleanup;
 	}
samples/bpf/trace_event_user.c +1 -1
index ac1ba368195c..8bcb9b4bfbe6 100644
@@ -318,7 +318,7 @@ int main(int argc, char **argv)
 	}
 
 	prog = bpf_object__find_program_by_name(obj, "bpf_prog1");
-	if (!prog) {
+	if (libbpf_get_error(prog)) {
 		printf("finding a prog in obj file failed\n");
 		goto cleanup;
 	}
samples/bpf/tracex1_user.c +1 -1
index 9d4adb7fd834..d3e01807dcd0 100644
@@ -20,7 +20,7 @@ int main(int ac, char **argv)
 	}
 
 	prog = bpf_object__find_program_by_name(obj, "bpf_prog1");
-	if (!prog) {
+	if (libbpf_get_error(prog)) {
 		fprintf(stderr, "ERROR: finding a prog in obj file failed\n");
 		goto cleanup;
 	}
samples/bpf/tracex5_user.c +1 -1
index 98dad57a96c4..65c753b30121 100644
@@ -53,7 +53,7 @@ int main(int ac, char **argv)
 	}
 
 	prog = bpf_object__find_program_by_name(obj, "bpf_prog1");
-	if (!prog) {
+	if (libbpf_get_error(prog)) {
 		printf("finding a prog in obj file failed\n");
 		goto cleanup;
 	}
samples/bpf/tracex7_user.c +1 -1
index fdcd6580dd73..10c896732139 100644
@@ -22,7 +22,7 @@ int main(int argc, char **argv)
 	}
 
 	prog = bpf_object__find_program_by_name(obj, "bpf_prog1");
-	if (!prog) {
+	if (libbpf_get_error(prog)) {
 		fprintf(stderr, "ERROR: finding a prog in obj file failed\n");
 		goto cleanup;
 	}
Signature · 1 line
2.25.1
#2 lore raw

Re: [PATCH 2/3] samples: bpf: cleanup pointer error check with libbpf

Quoted 10 lines · Daniel T. Lee

Libbpf has its own helper function to check for errors in the bpf data structure (pointer). And Some codes do not use this libbbpf helper function and check the pointer's error directly.

This commit clean up the existing pointer error check logic with libbpf.

---

This entire patch is wrong. bpf_object__find_program_by_name() returns NULL if the program is not found, not an error code.

Quoted 7 lines
 samples/bpf/sampleip_user.c    | 2 +-
 samples/bpf/trace_event_user.c | 2 +-
 samples/bpf/tracex1_user.c     | 2 +-
 samples/bpf/tracex5_user.c     | 2 +-
 samples/bpf/tracex7_user.c     | 2 +-
 5 files changed, 5 insertions(+), 5 deletions(-)

[...]

#3 lore raw

Re: [PATCH 2/3] samples: bpf: cleanup pointer error check with libbpf

2020년 6월 27일 (토) 05:25, Andrii Nakryiko <[email protected]>님이 작성:

Quoted 7 lines · Daniel T. Lee
Quoted 5 lines

Libbpf has its own helper function to check for errors in the bpf data structure (pointer). And Some codes do not use this libbbpf helper function and check the pointer's error directly.

Quoted 9 lines
Quoted 5 lines

This commit clean up the existing pointer error check logic with libbpf.

---

This entire patch is wrong. bpf_object__find_program_by_name() returns NULL if the program is not found, not an error code.

Oops, I'll drop the patch and resend with the next version.

Thanks for your time and effort for the review.
Daniel

Quoted 9 lines
Quoted 7 lines
 samples/bpf/sampleip_user.c    | 2 +-
 samples/bpf/trace_event_user.c | 2 +-
 samples/bpf/tracex1_user.c     | 2 +-
 samples/bpf/tracex5_user.c     | 2 +-
 samples/bpf/tracex7_user.c     | 2 +-
 5 files changed, 5 insertions(+), 5 deletions(-)

[...]

#4 lore raw

PATCH 3/3 samples: bpf: refactor BPF map in map test with libbpf

In reply to Daniel T. Lee

From commit 646f02ffdd49 ("libbpf: Add BTF-defined map-in-map support"), a way to define internal map in BTF-defined map has been added.

Instead of using previous 'inner_map_idx' definition, the structure to be used for the inner map can be directly defined using array directive.

__array(values, struct inner_map)

This commit refactors map in map test program with libbpf by explicitly defining inner map with BTF-defined format.

---
 samples/bpf/Makefile               |  2 +-
 samples/bpf/test_map_in_map_kern.c | 85 +++++++++++++++---------------
 samples/bpf/test_map_in_map_user.c | 53 +++++++++++++++++--
 3 files changed, 91 insertions(+), 49 deletions(-)
samples/bpf/Makefile +1 -1
index ffd0fda536da..78678d4e6842 100644
@@ -93,7 +93,7 @@ sampleip-objs := sampleip_user.o $(TRACE_HELPERS)
 tc_l2_redirect-objs := bpf_load.o tc_l2_redirect_user.o
 lwt_len_hist-objs := bpf_load.o lwt_len_hist_user.o
 xdp_tx_iptunnel-objs := xdp_tx_iptunnel_user.o
-test_map_in_map-objs := bpf_load.o test_map_in_map_user.o
+test_map_in_map-objs := test_map_in_map_user.o
 per_socket_stats_example-objs := cookie_uid_helper_example.o
 xdp_redirect-objs := xdp_redirect_user.o
 xdp_redirect_map-objs := xdp_redirect_map_user.o
samples/bpf/test_map_in_map_kern.c +42 -43

108 lines not on this page · raw message

samples/bpf/test_map_in_map_user.c +48 -5

73 lines not on this page · raw message

Signature · 1 line
2.25.1
#5 lore raw

Re: [PATCH 3/3] samples: bpf: refactor BPF map in map test with libbpf

Quoted 8 lines · Daniel T. Lee

From commit 646f02ffdd49 ("libbpf: Add BTF-defined map-in-map support"), a way to define internal map in BTF-defined map has been added.

Instead of using previous 'inner_map_idx' definition, the structure to be used for the inner map can be directly defined using array directive.

Quoted 7 lines

__array(values, struct inner_map)

This commit refactors map in map test program with libbpf by explicitly defining inner map with BTF-defined format.

---

Thanks for the clean up, looks good except that prog NULL check.

It also seems like this is the last use of bpf_map_def_legacy, do you mind removing it as well?

Quoted 5 lines
 samples/bpf/Makefile               |  2 +-
 samples/bpf/test_map_in_map_kern.c | 85 +++++++++++++++---------------
 samples/bpf/test_map_in_map_user.c | 53 +++++++++++++++++--
 3 files changed, 91 insertions(+), 49 deletions(-)

[...]

samples/bpf/test_map_in_map_user.c static void test_map_in_map(void) · line 130

Quoted 4 lines
        snprintf(filename, sizeof(filename), "%s_kern.o", argv[0]);
samples/bpf/test_map_in_map_user.c +2 -0
+       obj = bpf_object__open_file(filename, NULL);
+       if (libbpf_get_error(obj)) {

this is right, but...

samples/bpf/test_map_in_map_user.c static void test_map_in_map(void) · line 136

Quoted 9 lines
samples/bpf/test_map_in_map_user.c +5 -3
+               fprintf(stderr, "ERROR: opening BPF object file failed\n");
+               return 0;
+       }
 
-       if (load_bpf_file(filename)) {
-               printf("%s", bpf_log_buf);
-               return 1;
+       prog = bpf_object__find_program_by_name(obj, "trace_sys_connect");
+       if (libbpf_get_error(prog)) {

this is wrong. Just NULL check. libbpf APIs are not very consistent with what they return, unfortunately.

Quoted 4 lines
samples/bpf/test_map_in_map_user.c +4 -0
+               printf("finding a prog in obj file failed\n");
+               goto cleanup;
+       }
+

[...]

#6 lore raw

Re: [PATCH 3/3] samples: bpf: refactor BPF map in map test with libbpf

Quoted 12 lines · Andrii Nakryiko
Quoted 10 lines · Daniel T. Lee

From commit 646f02ffdd49 ("libbpf: Add BTF-defined map-in-map support"), a way to define internal map in BTF-defined map has been added.

Instead of using previous 'inner_map_idx' definition, the structure to be used for the inner map can be directly defined using array directive.

__array(values, struct inner_map)

Quoted 8 lines
Quoted 5 lines

This commit refactors map in map test program with libbpf by explicitly defining inner map with BTF-defined format.

---

Thanks for the clean up, looks good except that prog NULL check.

I'll fix this NULL check as well too.

Quoted 3 lines

It also seems like this is the last use of bpf_map_def_legacy, do you mind removing it as well?

Actually, there is one more place that uses bpf_map_def_legacy. map_perf_test_kern.c is the one, and I'm currently working on it, but I'm having difficulty with refactoring this file at the moment.

It has a hash_map map definition named inner_lru_hash_map with BPF_F_NUMA_NODE flag and '.numa_node = 0'.

The bpf_map_def in libbpf has the attribute name map_flags but it does not have the numa_node attribute. Because the numa node for bpf_map_def cannot be explicitly specified, this means that there is no way to set the numa node where the map will be placed at the time of bpf_object__load.

The only approach currently available is not to use libbbpf to handle everything (bpf_object_load), but instead to create a map directly with specifying numa node (bpf_load approach).

bpf_create_map_in_map_node
bpf_create_map_node

I'm trying to stick with the libbpf implementation only, and I'm wondering If I have to create bpf maps manually at _user.c program.

Any advice and suggestions will be greatly appreciated.

Thanks for your time and effort for the review.
Daniel.

Quoted 35 lines
Quoted 6 lines
 samples/bpf/Makefile               |  2 +-
 samples/bpf/test_map_in_map_kern.c | 85 +++++++++++++++---------------
 samples/bpf/test_map_in_map_user.c | 53 +++++++++++++++++--
 3 files changed, 91 insertions(+), 49 deletions(-)

[...]

samples/bpf/test_map_in_map_user.c static void test_map_in_map(void) · line 130

Quoted 4 lines
        snprintf(filename, sizeof(filename), "%s_kern.o", argv[0]);
samples/bpf/test_map_in_map_user.c +2 -0
+       obj = bpf_object__open_file(filename, NULL);
+       if (libbpf_get_error(obj)) {

this is right, but...

samples/bpf/test_map_in_map_user.c static void test_map_in_map(void) · line 136

Quoted 9 lines
samples/bpf/test_map_in_map_user.c +5 -3
+               fprintf(stderr, "ERROR: opening BPF object file failed\n");
+               return 0;
+       }
 
-       if (load_bpf_file(filename)) {
-               printf("%s", bpf_log_buf);
-               return 1;
+       prog = bpf_object__find_program_by_name(obj, "trace_sys_connect");
+       if (libbpf_get_error(prog)) {

this is wrong. Just NULL check. libbpf APIs are not very consistent with what they return, unfortunately.

Quoted 4 lines
samples/bpf/test_map_in_map_user.c +4 -0
+               printf("finding a prog in obj file failed\n");
+               goto cleanup;
+       }
+

[...]

#7 lore raw

Re: [PATCH 3/3] samples: bpf: refactor BPF map in map test with libbpf

Quoted 34 lines · Daniel T. Lee
Quoted 12 lines · Andrii Nakryiko
Quoted 10 lines · Daniel T. Lee

From commit 646f02ffdd49 ("libbpf: Add BTF-defined map-in-map support"), a way to define internal map in BTF-defined map has been added.

Instead of using previous 'inner_map_idx' definition, the structure to be used for the inner map can be directly defined using array directive.

__array(values, struct inner_map)

Quoted 8 lines
Quoted 5 lines

This commit refactors map in map test program with libbpf by explicitly defining inner map with BTF-defined format.

---

Thanks for the clean up, looks good except that prog NULL check.

I'll fix this NULL check as well too.

Quoted 3 lines

It also seems like this is the last use of bpf_map_def_legacy, do you mind removing it as well?

Actually, there is one more place that uses bpf_map_def_legacy. map_perf_test_kern.c is the one, and I'm currently working on it, but I'm having difficulty with refactoring this file at the moment.

Quoted 5 lines

It has a hash_map map definition named inner_lru_hash_map with BPF_F_NUMA_NODE flag and '.numa_node = 0'.

The bpf_map_def in libbpf has the attribute name map_flags but it does not have the numa_node attribute. Because the numa node

It does since 1 or 2 days ago ([0])

[0] https://patchwork.ozlabs.org/project/netdev/patch/[email protected]/

Quoted 8 lines

for bpf_map_def cannot be explicitly specified, this means that there is no way to set the numa node where the map will be placed at the time of bpf_object__load.

The only approach currently available is not to use libbbpf to handle everything (bpf_object_load), but instead to create a map directly with specifying numa node (bpf_load approach).

Quoted 8 lines

bpf_create_map_in_map_node
bpf_create_map_node

I'm trying to stick with the libbpf implementation only, and I'm wondering If I have to create bpf maps manually at _user.c program.

Any advice and suggestions will be greatly appreciated.

It should be super straightforward now with a BTF-defined map supporting numa_node attribute.

Quoted 38 lines

Thanks for your time and effort for the review.
Daniel.

Quoted 35 lines
Quoted 6 lines
 samples/bpf/Makefile               |  2 +-
 samples/bpf/test_map_in_map_kern.c | 85 +++++++++++++++---------------
 samples/bpf/test_map_in_map_user.c | 53 +++++++++++++++++--
 3 files changed, 91 insertions(+), 49 deletions(-)

[...]

samples/bpf/test_map_in_map_user.c static void test_map_in_map(void) · line 130

Quoted 4 lines
        snprintf(filename, sizeof(filename), "%s_kern.o", argv[0]);
samples/bpf/test_map_in_map_user.c +2 -0
+       obj = bpf_object__open_file(filename, NULL);
+       if (libbpf_get_error(obj)) {

this is right, but...

samples/bpf/test_map_in_map_user.c static void test_map_in_map(void) · line 136

Quoted 9 lines
samples/bpf/test_map_in_map_user.c +5 -3
+               fprintf(stderr, "ERROR: opening BPF object file failed\n");
+               return 0;
+       }
 
-       if (load_bpf_file(filename)) {
-               printf("%s", bpf_log_buf);
-               return 1;
+       prog = bpf_object__find_program_by_name(obj, "trace_sys_connect");
+       if (libbpf_get_error(prog)) {

this is wrong. Just NULL check. libbpf APIs are not very consistent with what they return, unfortunately.

Quoted 4 lines
samples/bpf/test_map_in_map_user.c +4 -0
+               printf("finding a prog in obj file failed\n");
+               goto cleanup;
+       }
+

[...]

#8 lore raw

Re: [PATCH 3/3] samples: bpf: refactor BPF map in map test with libbpf

Quoted 58 lines · Andrii Nakryiko
Quoted 34 lines · Daniel T. Lee
Quoted 12 lines · Andrii Nakryiko
Quoted 10 lines · Daniel T. Lee

From commit 646f02ffdd49 ("libbpf: Add BTF-defined map-in-map support"), a way to define internal map in BTF-defined map has been added.

Instead of using previous 'inner_map_idx' definition, the structure to be used for the inner map can be directly defined using array directive.

__array(values, struct inner_map)

Quoted 8 lines
Quoted 5 lines

This commit refactors map in map test program with libbpf by explicitly defining inner map with BTF-defined format.

---

Thanks for the clean up, looks good except that prog NULL check.

I'll fix this NULL check as well too.

Quoted 3 lines

It also seems like this is the last use of bpf_map_def_legacy, do you mind removing it as well?

Actually, there is one more place that uses bpf_map_def_legacy. map_perf_test_kern.c is the one, and I'm currently working on it, but I'm having difficulty with refactoring this file at the moment.

Quoted 5 lines

It has a hash_map map definition named inner_lru_hash_map with BPF_F_NUMA_NODE flag and '.numa_node = 0'.

The bpf_map_def in libbpf has the attribute name map_flags but it does not have the numa_node attribute. Because the numa node

It does since 1 or 2 days ago ([0])

[0] https://patchwork.ozlabs.org/project/netdev/patch/[email protected]/

Quoted 11 lines

for bpf_map_def cannot be explicitly specified, this means that there is no way to set the numa node where the map will be placed at the time of bpf_object__load.

The only approach currently available is not to use libbbpf to handle everything (bpf_object_load), but instead to create a map directly with specifying numa node (bpf_load approach).

bpf_create_map_in_map_node
bpf_create_map_node

Quoted 9 lines
Quoted 5 lines

I'm trying to stick with the libbpf implementation only, and I'm wondering If I have to create bpf maps manually at _user.c program.

Any advice and suggestions will be greatly appreciated.

It should be super straightforward now with a BTF-defined map supporting numa_node attribute.

Awesome, thanks for letting me know!

I will use this new attribute for the map_perf_test refactoring. Problem Solved!

Thanks.

Quoted 38 lines
Quoted 38 lines

Thanks for your time and effort for the review.
Daniel.

Quoted 35 lines
Quoted 6 lines
 samples/bpf/Makefile               |  2 +-
 samples/bpf/test_map_in_map_kern.c | 85 +++++++++++++++---------------
 samples/bpf/test_map_in_map_user.c | 53 +++++++++++++++++--
 3 files changed, 91 insertions(+), 49 deletions(-)

[...]

samples/bpf/test_map_in_map_user.c static void test_map_in_map(void) · line 130

Quoted 4 lines
        snprintf(filename, sizeof(filename), "%s_kern.o", argv[0]);
samples/bpf/test_map_in_map_user.c +2 -0
+       obj = bpf_object__open_file(filename, NULL);
+       if (libbpf_get_error(obj)) {

this is right, but...

samples/bpf/test_map_in_map_user.c static void test_map_in_map(void) · line 136

Quoted 9 lines
samples/bpf/test_map_in_map_user.c +5 -3
+               fprintf(stderr, "ERROR: opening BPF object file failed\n");
+               return 0;
+       }
 
-       if (load_bpf_file(filename)) {
-               printf("%s", bpf_log_buf);
-               return 1;
+       prog = bpf_object__find_program_by_name(obj, "trace_sys_connect");
+       if (libbpf_get_error(prog)) {

this is wrong. Just NULL check. libbpf APIs are not very consistent with what they return, unfortunately.

Quoted 4 lines
samples/bpf/test_map_in_map_user.c +4 -0
+               printf("finding a prog in obj file failed\n");
+               goto cleanup;
+       }
+

[...]

Acked-by on v1 1/3
#9 lore raw
In reply to Daniel T. Lee
Quoted 18 lines · Daniel T. Lee

Currently, BPF programs with kprobe/sys_connect does not work properly.

Commit 34745aed515c ("samples/bpf: fix kprobe attachment issue on x64") This commit modifies the bpf_load behavior of kprobe events in the x64 architecture. If the current kprobe event target starts with "sys_*", add the prefix "__x64_" to the front of the event.

Appending "__x64_" prefix with kprobe/sys_* event was appropriate as a solution to most of the problems caused by the commit below.

commit d5a00528b58c ("syscalls/core, syscalls/x86: Rename struct pt_regs-based sys_*() to __x64_sys_*()")

However, there is a problem with the sys_connect kprobe event that does not work properly. For __sys_connect event, parameters can be fetched normally, but for __x64_sys_connect, parameters cannot be fetched.

Quoted 7 lines

Because of this problem, this commit fixes the sys_connect event by specifying the __sys_connect directly and this will bypass the "__x64_" appending rule of bpf_load.

Fixes 34745aed515c ("samples/bpf: fix kprobe attachment issue on x64")

---

Quoted 5 lines
 samples/bpf/map_perf_test_kern.c         | 2 +-
 samples/bpf/test_map_in_map_kern.c       | 2 +-
 samples/bpf/test_probe_write_user_kern.c | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

[...]