Skip to contentSkip to conversation

libbpf's hashmap use of __WORDSIZE

PATCH Applied Arnaldo Carvalho de Melo 4 participants 7 messages Jun 8, 2020 – Jun 9, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi Andrii,

We've got that hashmap.[ch] copy from libbpf so that we can
build perf in systems where libbpf isn't available, and to make it build in all the containers I regularly test build perf I had to add the patch below, I test build with many versions of both gcc and clang and multiple libcs.

https://gcc.gnu.org/onlinedocs/cpp/Common-Predefined-Macros.html

The way that tools/include/linux/bitops.h has been doing since 2012 is explained in:

http://git.kernel.org/torvalds/c/3f34f6c0233ae055b5

Please take a look and see if you find it acceptable,

Thanks,

- Arnaldo

Warning: Kernel ABI header at 'tools/perf/util/hashmap.h' differs from latest version at 'tools/lib/bpf/hashmap.h' diff -u tools/perf/util/hashmap.h tools/lib/bpf/hashmap.h

$ diff -u tools/lib/bpf/hashmap.h tools/perf/util/hashmap.h
--- tools/lib/bpf/hashmap.h	2020-06-05 13:25:27.822079838 -0300
+++ tools/perf/util/hashmap.h	2020-06-05 13:25:27.838079794 -0300
@@ -10,10 +10,9 @@

#include <stdbool.h>
#include <stddef.h>
-#ifdef __GLIBC__
-#include <bits/wordsize.h>
-#else
-#include <bits/reg.h>
+#include <limits.h>
+#ifndef __WORDSIZE
+#define __WORDSIZE (__SIZEOF_LONG__ * 8)
#endif

static inline size_t hash_bits(size_t h, int bits)

#1 lore raw
Quoted 30 lines · Arnaldo Carvalho de Melo

Hi Andrii,

We've got that hashmap.[ch] copy from libbpf so that we can
build perf in systems where libbpf isn't available, and to make it build in all the containers I regularly test build perf I had to add the patch below, I test build with many versions of both gcc and clang and multiple libcs.

https://gcc.gnu.org/onlinedocs/cpp/Common-Predefined-Macros.html

The way that tools/include/linux/bitops.h has been doing since 2012 is explained in:

http://git.kernel.org/torvalds/c/3f34f6c0233ae055b5

Please take a look and see if you find it acceptable,

Thanks,

- Arnaldo

Warning: Kernel ABI header at 'tools/perf/util/hashmap.h' differs from latest version at 'tools/lib/bpf/hashmap.h' diff -u tools/perf/util/hashmap.h tools/lib/bpf/hashmap.h

$ diff -u tools/lib/bpf/hashmap.h tools/perf/util/hashmap.h
perf/util/hashmap.h 2020-06-05 13:25:27.838079794 -0300 renamed from lib/bpf/hashmap.h 2020-06-05 13:25:27.822079838 -0300 +0 -0
@@ -10,10 +10,9 @@
 

perf/util/hashmap.h 2020-06-05 13:25:27.838079794 -0300 line 16

Quoted 10 lines
perf/util/hashmap.h 2020-06-05 13:25:27.838079794 -0300 renamed from lib/bpf/hashmap.h 2020-06-05 13:25:27.822079838 -0300 +3 -4
 #include <stdbool.h>
 #include <stddef.h>
-#ifdef __GLIBC__
-#include <bits/wordsize.h>
-#else
-#include <bits/reg.h>
+#include <limits.h>
+#ifndef __WORDSIZE
+#define __WORDSIZE (__SIZEOF_LONG__ * 8)
 #endif

This looks fine, I also build-tested it in Travis CI, so all good. There is actually __SIZEOF_SIZE_T__, which is more directly what hash_bits work with, but I don't think it matters for any reasonable system in use :)

So yeah,

Are you going to do this change for libbpf's variant, or should I submit a separate patch?

Quoted 2 lines
perf/util/hashmap.h 2020-06-05 13:25:27.838079794 -0300 renamed from lib/bpf/hashmap.h 2020-06-05 13:25:27.822079838 -0300 +0 -0
 static inline size_t hash_bits(size_t h, int bits)
#2 lore raw

Mi

Quoted 52 lines · Andrii Nakryiko
Quoted 5 lines · Arnaldo Carvalho de Melo

Hi Andrii,

We've got that hashmap.[ch] copy from libbpf so that we can
build perf in systems where libbpf isn't available, and to make it

build

Quoted 1 line

in all the containers I regularly test build perf I had to add the

patch

Quoted 6 lines

below, I test build with many versions of both gcc and clang and multiple libcs.

https://gcc.gnu.org/onlinedocs/cpp/Common-Predefined-Macros.html

The way that tools/include/linux/bitops.h has been doing since 2012

is

Quoted 11 lines

explained in:

http://git.kernel.org/torvalds/c/3f34f6c0233ae055b5

Please take a look and see if you find it acceptable,

Thanks,

- Arnaldo

Warning: Kernel ABI header at 'tools/perf/util/hashmap.h' differs

from latest version at 'tools/lib/bpf/hashmap.h'

Quoted 7 lines

diff -u tools/perf/util/hashmap.h tools/lib/bpf/hashmap.h

$ diff -u tools/lib/bpf/hashmap.h tools/perf/util/hashmap.h
perf/util/hashmap.h 2020-06-05 13:25:27.838079794 -0300 renamed from lib/bpf/hashmap.h 2020-06-05 13:25:27.822079838 -0300 +0 -0
@@ -10,10 +10,9 @@
 

perf/util/hashmap.h 2020-06-05 13:25:27.838079794 -0300 line 16

Quoted 10 lines
perf/util/hashmap.h 2020-06-05 13:25:27.838079794 -0300 renamed from lib/bpf/hashmap.h 2020-06-05 13:25:27.822079838 -0300 +3 -4
 #include <stdbool.h>
 #include <stddef.h>
-#ifdef __GLIBC__
-#include <bits/wordsize.h>
-#else
-#include <bits/reg.h>
+#include <limits.h>
+#ifndef __WORDSIZE
+#define __WORDSIZE (__SIZEOF_LONG__ * 8)
 #endif

This looks fine, I also build-tested it in Travis CI, so all good. There is actually __SIZEOF_SIZE_T__, which is more directly what hash_bits work with, but I don't think it matters for any reasonable system in use :)

Quoted 6 lines

So yeah,

Are you going to do this change for libbpf's variant, or should I submit a separate patch?

I'll send the patch later,

Thanks for checking,

- Arnaldo

Quoted 3 lines
Quoted 3 lines
perf/util/hashmap.h 2020-06-05 13:25:27.838079794 -0300 renamed from lib/bpf/hashmap.h 2020-06-05 13:25:27.822079838 -0300 +0 -0
 static inline size_t hash_bits(size_t h, int bits)
Signature · 1 line
Sent from my Android device with K-9 Mail. Please excuse my brevity.
#3 lore raw

PATCH 1/1 libbpf: Define __WORDSIZE if not available

In reply to Andrii Nakryiko

Some systems, such as Android, don't have a define for __WORDSIZE, do it in terms of __SIZEOF_LONG__, as done in perf since 2012:

http://git.kernel.org/torvalds/c/3f34f6c0233ae055b5

For reference: https://gcc.gnu.org/onlinedocs/cpp/Common-Predefined-Macros.html

I build tested it here and Andrii did some Travis CI build tests too.

maintainer
#4 lore raw

Re: [PATCH] libbpf: Define __WORDSIZE if not available

Hey Arnaldo,

Quoted 11 lines · Arnaldo Carvalho de Melo

Some systems, such as Android, don't have a define for __WORDSIZE, do it in terms of __SIZEOF_LONG__, as done in perf since 2012:

   http://git.kernel.org/torvalds/c/3f34f6c0233ae055b5

For reference: https://gcc.gnu.org/onlinedocs/cpp/Common-Predefined-Macros.html

I build tested it here and Andrii did some Travis CI build tests too.

Diff missing?

Thanks,
Daniel

author Acked-by Andrii Nakryiko
#5 lore raw

Re: [PATCH] libbpf: Define __WORDSIZE if not available

Em Tue, Jun 09, 2020 at 10:37:48PM +0200, Daniel Borkmann escreveu:

Quoted 8 lines

Hey Arnaldo,

Quoted 5 lines · Arnaldo Carvalho de Melo

Some systems, such as Android, don't have a define for __WORDSIZE, do it in terms of __SIZEOF_LONG__, as done in perf since 2012:

   http://git.kernel.org/torvalds/c/3f34f6c0233ae055b5
Quoted 8 lines
Quoted 6 lines

For reference: https://gcc.gnu.org/onlinedocs/cpp/Common-Predefined-Macros.html

I build tested it here and Andrii did some Travis CI build tests too.

Diff missing?

Oh well, sorry about that, EBADCOFFEE or something:

From: Arnaldo Carvalho de Melo <[email protected]>

Some systems, such as Android, don't have a define for __WORDSIZE, do it in terms of __SIZEOF_LONG__, as done in perf since 2012:

http://git.kernel.org/torvalds/c/3f34f6c0233ae055b5

For reference: https://gcc.gnu.org/onlinedocs/cpp/Common-Predefined-Macros.html

I build tested it here and Andrii did some Travis CI build tests too.

---

tools/lib/bpf/hashmap.h +3 -4
index e823b35e7371..df59fd4fc95b 100644
@@ -10,10 +10,9 @@
 
 #include <stdbool.h>
 #include <stddef.h>
-#ifdef __GLIBC__
-#include <bits/wordsize.h>
-#else
-#include <bits/reg.h>
+#include <limits.h>
+#ifndef __WORDSIZE
+#define __WORDSIZE (__SIZEOF_LONG__ * 8)
 #endif
 
 static inline size_t hash_bits(size_t h, int bits)
maintainer Applied
#6 lore raw

Re: [PATCH] libbpf: Define __WORDSIZE if not available

Quoted 25 lines · Arnaldo Carvalho de Melo

Em Tue, Jun 09, 2020 at 10:37:48PM +0200, Daniel Borkmann escreveu:

Quoted 8 lines

Hey Arnaldo,

Quoted 5 lines · Arnaldo Carvalho de Melo

Some systems, such as Android, don't have a define for __WORDSIZE, do it in terms of __SIZEOF_LONG__, as done in perf since 2012:

    http://git.kernel.org/torvalds/c/3f34f6c0233ae055b5
Quoted 8 lines
Quoted 6 lines

For reference: https://gcc.gnu.org/onlinedocs/cpp/Common-Predefined-Macros.html

I build tested it here and Andrii did some Travis CI build tests too.

Diff missing?

Oh well, sorry about that, EBADCOFFEE or something:

From: Arnaldo Carvalho de Melo <[email protected]>

Some systems, such as Android, don't have a define for __WORDSIZE, do it in terms of __SIZEOF_LONG__, as done in perf since 2012:

Quoted 9 lines
    http://git.kernel.org/torvalds/c/3f34f6c0233ae055b5

For reference: https://gcc.gnu.org/onlinedocs/cpp/Common-Predefined-Macros.html

I build tested it here and Andrii did some Travis CI build tests too.

[...]

Applied, thanks!