Skip to contentSkip to conversation

lib/bpf hashmap portability and fix

PATCH In review Ian Rogers 2 participants 9 messages May 6, 2020

Subsystems (4)
lore
Share & tools
lore

Ian Rogers

author
#0 lore raw

cover lib/bpf hashmap portability and fix

I'm experimenting with hashmap in perf for perf events within a metric expression. In doing this I encountered some issues with lib/bpf hashmap. Ignoring the perf change, I think these fixes are likely still worthwhile. Thanks!

Ian Rogers (2):
lib/bpf hashmap: increase portability
lib/bpf hashmap: fixes to hashmap__clear

 tools/lib/bpf/hashmap.c | 6 ++++++
 tools/lib/bpf/hashmap.h | 3 +--
 2 files changed, 7 insertions(+), 2 deletions(-)
Signature · 1 line
2.26.2.526.g744177e7f7-goog
author
#1 lore raw

PATCH 1/2 lib/bpf hashmap: increase portability

Don't include libbpf_internal.h as it is unused and has conflicting definitions, for example, with tools/perf/util/debug.h. Fix a non-glibc include path.

---
 tools/lib/bpf/hashmap.h | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
tools/lib/bpf/hashmap.h +1 -2
index bae8879cdf58..d5ef212a55ba 100644
@@ -13,9 +13,8 @@
 #ifdef __GLIBC__
 #include <bits/wordsize.h>
 #else
-#include <bits/reg.h>
+#include <linux/bitops.h>
 #endif
-#include "libbpf_internal.h"
 
 static inline size_t hash_bits(size_t h, int bits)
 {
Signature · 1 line
2.26.2.526.g744177e7f7-goog
#2 lore raw

Re: [PATCH 1/2] lib/bpf hashmap: increase portability

Quoted 15 lines · Ian Rogers

Don't include libbpf_internal.h as it is unused and has conflicting definitions, for example, with tools/perf/util/debug.h. Fix a non-glibc include path.

---
 tools/lib/bpf/hashmap.h | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
tools/lib/bpf/hashmap.h +0 -0
index bae8879cdf58..d5ef212a55ba 100644

tools/lib/bpf/hashmap.h line 16

Quoted 5 lines
tools/lib/bpf/hashmap.h +1 -1
@@ -13,9 +13,8 @@
 #ifdef __GLIBC__
 #include <bits/wordsize.h>
 #else
-#include <bits/reg.h>
+#include <linux/bitops.h>

why this change? It might be ok for libbpf built from kernel source, but it will break Github libbpf.

tools/lib/bpf/hashmap.h line 18

Quoted 2 lines
tools/lib/bpf/hashmap.h +0 -1
 #endif
-#include "libbpf_internal.h"

Dropping this seems ok, don't remember why I had it here in the first place.

Quoted 6 lines
tools/lib/bpf/hashmap.h +0 -1
 static inline size_t hash_bits(size_t h, int bits)
 {
--

2.26.2.526.g744177e7f7-goog

author
#3 lore raw

Re: [PATCH 1/2] lib/bpf hashmap: increase portability

Quoted 12 lines · Andrii Nakryiko
Quoted 10 lines · Ian Rogers

Don't include libbpf_internal.h as it is unused and has conflicting definitions, for example, with tools/perf/util/debug.h. Fix a non-glibc include path.

---
 tools/lib/bpf/hashmap.h | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
Quoted 13 lines

tools/lib/bpf/hashmap.h line 16

Quoted 10 lines
tools/lib/bpf/hashmap.h +1 -1
index bae8879cdf58..d5ef212a55ba 100644
@@ -13,9 +13,8 @@
 #ifdef __GLIBC__
 #include <bits/wordsize.h>
 #else
-#include <bits/reg.h>
+#include <linux/bitops.h>

why this change? It might be ok for libbpf built from kernel source, but it will break Github libbpf.

Without this change my debian based machine wasn't able to build within the kernel tree. I see bits/wordsize.h on the machine. Perhaps the __WORDSIZE computation could just be based on __LP64__ to remove any #include?

Thanks,
Ian

Quoted 11 lines

tools/lib/bpf/hashmap.h line 18

Quoted 2 lines
tools/lib/bpf/hashmap.h +0 -1
 #endif
-#include "libbpf_internal.h"

Dropping this seems ok, don't remember why I had it here in the first place.

Quoted 6 lines
tools/lib/bpf/hashmap.h +0 -1
 static inline size_t hash_bits(size_t h, int bits)
 {
--

2.26.2.526.g744177e7f7-goog

#4 lore raw

Re: [PATCH 1/2] lib/bpf hashmap: increase portability

Quoted 29 lines · Ian Rogers
Quoted 12 lines · Andrii Nakryiko
Quoted 10 lines · Ian Rogers

Don't include libbpf_internal.h as it is unused and has conflicting definitions, for example, with tools/perf/util/debug.h. Fix a non-glibc include path.

---
 tools/lib/bpf/hashmap.h | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
Quoted 13 lines

tools/lib/bpf/hashmap.h line 16

Quoted 10 lines
tools/lib/bpf/hashmap.h +1 -1
index bae8879cdf58..d5ef212a55ba 100644
@@ -13,9 +13,8 @@
 #ifdef __GLIBC__
 #include <bits/wordsize.h>
 #else
-#include <bits/reg.h>
+#include <linux/bitops.h>

why this change? It might be ok for libbpf built from kernel source, but it will break Github libbpf.

Quoted 4 lines

Without this change my debian based machine wasn't able to build within the kernel tree. I see bits/wordsize.h on the machine. Perhaps the __WORDSIZE computation could just be based on __LP64__ to remove any #include?

It might work. Do you mind forking https://github.com/libbpf/libbpf
and trying to execute travis CI tests with such change? It compiles across a range of distros and arches. You might need to set up Travis CI login, hope that's not a problem. Thanks!

Quoted 15 lines

Thanks,
Ian

Quoted 11 lines

tools/lib/bpf/hashmap.h line 18

Quoted 2 lines
tools/lib/bpf/hashmap.h +0 -1
 #endif
-#include "libbpf_internal.h"

Dropping this seems ok, don't remember why I had it here in the first place.

Quoted 6 lines
tools/lib/bpf/hashmap.h +0 -1
 static inline size_t hash_bits(size_t h, int bits)
 {
--

2.26.2.526.g744177e7f7-goog

author
#5 lore raw

Re: [PATCH 1/2] lib/bpf hashmap: increase portability

Quoted 36 lines · Andrii Nakryiko
Quoted 29 lines · Ian Rogers
Quoted 12 lines · Andrii Nakryiko
Quoted 10 lines · Ian Rogers

Don't include libbpf_internal.h as it is unused and has conflicting definitions, for example, with tools/perf/util/debug.h. Fix a non-glibc include path.

---
 tools/lib/bpf/hashmap.h | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
Quoted 13 lines

tools/lib/bpf/hashmap.h line 16

Quoted 10 lines
tools/lib/bpf/hashmap.h +1 -1
index bae8879cdf58..d5ef212a55ba 100644
@@ -13,9 +13,8 @@
 #ifdef __GLIBC__
 #include <bits/wordsize.h>
 #else
-#include <bits/reg.h>
+#include <linux/bitops.h>

why this change? It might be ok for libbpf built from kernel source, but it will break Github libbpf.

Quoted 4 lines

Without this change my debian based machine wasn't able to build within the kernel tree. I see bits/wordsize.h on the machine. Perhaps the __WORDSIZE computation could just be based on __LP64__ to remove any #include?

Quoted 4 lines

It might work. Do you mind forking https://github.com/libbpf/libbpf
and trying to execute travis CI tests with such change? It compiles across a range of distros and arches. You might need to set up Travis CI login, hope that's not a problem. Thanks!

I'll try to find time. Thanks,
Ian

Quoted 15 lines
Quoted 15 lines

Thanks,
Ian

Quoted 11 lines

tools/lib/bpf/hashmap.h line 18

Quoted 2 lines
tools/lib/bpf/hashmap.h +0 -1
 #endif
-#include "libbpf_internal.h"

Dropping this seems ok, don't remember why I had it here in the first place.

Quoted 6 lines
tools/lib/bpf/hashmap.h +0 -1
 static inline size_t hash_bits(size_t h, int bits)
 {
--

2.26.2.526.g744177e7f7-goog

author
#6 lore raw

PATCH 2/2 lib/bpf hashmap: fixes to hashmap__clear

In reply to Ian Rogers

hashmap_find_entry assumes that if buckets is NULL then there are no entries. NULL the buckets in clear to ensure this.
Free hashmap entries and not just the bucket array.

---
 tools/lib/bpf/hashmap.c | 6 ++++++
 1 file changed, 6 insertions(+)
tools/lib/bpf/hashmap.c +6 -0
index 54c30c802070..1a1bca1ff5cd 100644
@@ -59,7 +59,13 @@ struct hashmap *hashmap__new(hashmap_hash_fn hash_fn,
 
 void hashmap__clear(struct hashmap *map)
 {
+	struct hashmap_entry *cur, *tmp;
+	size_t bkt;
+
+	hashmap__for_each_entry_safe(map, cur, tmp, bkt)
+		free(cur);
 	free(map->buckets);
+	map->buckets = NULL;
 	map->cap = map->cap_bits = map->sz = 0;
 }
 
Signature · 1 line
2.26.2.526.g744177e7f7-goog
#7 lore raw

Re: [PATCH 2/2] lib/bpf hashmap: fixes to hashmap__clear

Quoted 7 lines · Ian Rogers

hashmap_find_entry assumes that if buckets is NULL then there are no entries. NULL the buckets in clear to ensure this.
Free hashmap entries and not just the bucket array.

---

This is already fixed in bpf-next ([0]). Seems to be 1-to-1 character by character :)

[0] https://patchwork.ozlabs.org/project/netdev/patch/[email protected]/

Quoted 24 lines
 tools/lib/bpf/hashmap.c | 6 ++++++
 1 file changed, 6 insertions(+)
tools/lib/bpf/hashmap.c +6 -1
index 54c30c802070..1a1bca1ff5cd 100644
@@ -59,7 +59,13 @@ struct hashmap *hashmap__new(hashmap_hash_fn hash_fn,
 
 void hashmap__clear(struct hashmap *map)
 {
+       struct hashmap_entry *cur, *tmp;
+       size_t bkt;
+
+       hashmap__for_each_entry_safe(map, cur, tmp, bkt)
+               free(cur);
        free(map->buckets);
+       map->buckets = NULL;
        map->cap = map->cap_bits = map->sz = 0;
 }
 
--

2.26.2.526.g744177e7f7-goog

author
#8 lore raw

Re: [PATCH 2/2] lib/bpf hashmap: fixes to hashmap__clear

Quoted 7 lines · Andrii Nakryiko
Quoted 5 lines · Ian Rogers

hashmap_find_entry assumes that if buckets is NULL then there are no entries. NULL the buckets in clear to ensure this.
Free hashmap entries and not just the bucket array.

Quoted 7 lines
Quoted 2 lines

---

This is already fixed in bpf-next ([0]). Seems to be 1-to-1 character by character :)

[0] https://patchwork.ozlabs.org/project/netdev/patch/[email protected]/

Thanks!
Ian

Quoted 24 lines
Quoted 24 lines
 tools/lib/bpf/hashmap.c | 6 ++++++
 1 file changed, 6 insertions(+)
tools/lib/bpf/hashmap.c +6 -1
index 54c30c802070..1a1bca1ff5cd 100644
@@ -59,7 +59,13 @@ struct hashmap *hashmap__new(hashmap_hash_fn hash_fn,
 
 void hashmap__clear(struct hashmap *map)
 {
+       struct hashmap_entry *cur, *tmp;
+       size_t bkt;
+
+       hashmap__for_each_entry_safe(map, cur, tmp, bkt)
+               free(cur);
        free(map->buckets);
+       map->buckets = NULL;
        map->cap = map->cap_bits = map->sz = 0;
 }
 
--

2.26.2.526.g744177e7f7-goog