Skip to contentSkip to conversation

Failed to build arm64 bpf samples with LLVM 3.7 on 4.3 kernel

DISCUSSION Shi, Yang 2 participants 5 messages Nov 10, 2015 – Nov 11, 2015

lore
Share & tools
lore

Shi, Yang

author
#0 lore raw

Hi guys,

I just tried to build arm64 bpf samples on 4.3 kernel, but LLVM complain:

LLVM ERROR: Inline asm not supported by this streamer because we don't have an asm parser for this target

I took some time to bisect it, then found it is caused by sysreg.h

There is inline assembly macro defined in sysreg.h:

asm(
" .irp
num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"

"       .equ    __reg_num_x\\num, \\num\n"
"       .endr\n"
"       .equ    __reg_num_xzr, 31\n"
"\n"
"       .macro  mrs_s, rt, sreg\n"
"       .inst   0xd5200000|(\\sreg)|(__reg_num_\\rt)\n"
"       .endm\n"
"\n"
"       .macro  msr_s, sreg, rt\n"
"       .inst   0xd5000000|(\\sreg)|(__reg_num_\\rt)\n"
"       .endm\n"

);

sysreg.h was not included by any other arm64 header files in 4.2, but in 4.3 it is included by futex.h and uaccess.h, which are included by bpf samples via skbuff.h.

But, it sounds LLVM can't recognize it. I'm not familiar with LLVM, so any suggestion?

Could it be worked around by some LLVM compile flags?

Thanks,
Yang

#1 lore raw
Quoted 27 lines · Yang

Hi guys,

I just tried to build arm64 bpf samples on 4.3 kernel, but LLVM complain:

LLVM ERROR: Inline asm not supported by this streamer because we don't have an asm parser for this target

I took some time to bisect it, then found it is caused by sysreg.h

There is inline assembly macro defined in sysreg.h:

asm(
" .irp
num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"

"       .equ    __reg_num_x\\num, \\num\n"
"       .endr\n"
"       .equ    __reg_num_xzr, 31\n"
"\n"
"       .macro  mrs_s, rt, sreg\n"
"       .inst   0xd5200000|(\\sreg)|(__reg_num_\\rt)\n"
"       .endm\n"
"\n"
"       .macro  msr_s, sreg, rt\n"
"       .inst   0xd5000000|(\\sreg)|(__reg_num_\\rt)\n"
"       .endm\n"

);

Quoted 8 lines

sysreg.h was not included by any other arm64 header files in 4.2, but in 4.3 it is included by futex.h and uaccess.h, which are included by bpf samples via skbuff.h.

But, it sounds LLVM can't recognize it. I'm not familiar with LLVM, so any suggestion?

Could it be worked around by some LLVM compile flags?

hmm this used to work. I don't have arm64 setup anymore. Try hacking it by adding #define __ASM_SYSREG_H to prevent inclusion of that file? all those asms are unused anyway.
tracing samples include kernel headers to have exact struct layout. and #include skbuff.h can be removed from almost all samples. In-kernel 'struct sk_buff' only used inside tracex1_kern.c

author
#2 lore raw
Quoted 32 lines · Alexei Starovoitov
Quoted 31 lines · Yang

Hi guys,

I just tried to build arm64 bpf samples on 4.3 kernel, but LLVM complain:

LLVM ERROR: Inline asm not supported by this streamer because we don't have an asm parser for this target

I took some time to bisect it, then found it is caused by sysreg.h

There is inline assembly macro defined in sysreg.h:

asm(
" .irp
num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"

"       .equ    __reg_num_x\\num, \\num\n"
"       .endr\n"
"       .equ    __reg_num_xzr, 31\n"
"\n"
"       .macro  mrs_s, rt, sreg\n"
"       .inst   0xd5200000|(\\sreg)|(__reg_num_\\rt)\n"
"       .endm\n"
"\n"
"       .macro  msr_s, sreg, rt\n"
"       .inst   0xd5000000|(\\sreg)|(__reg_num_\\rt)\n"
"       .endm\n"

);

sysreg.h was not included by any other arm64 header files in 4.2, but in 4.3 it is included by futex.h and uaccess.h, which are included by bpf samples via skbuff.h.

Quoted 6 lines
Quoted 4 lines

But, it sounds LLVM can't recognize it. I'm not familiar with LLVM, so any suggestion?

Could it be worked around by some LLVM compile flags?

hmm this used to work. I don't have arm64 setup anymore.

Yes, sysreg.h used to be not included by any header file.

Quoted 5 lines

Try hacking it by adding #define __ASM_SYSREG_H to prevent inclusion of that file? all those asms are unused anyway.
tracing samples include kernel headers to have exact struct layout. and #include skbuff.h can be removed from almost all samples. In-kernel 'struct sk_buff' only used inside tracex1_kern.c

The below change works:

         clang $(NOSTDINC_FLAGS) $(LINUXINCLUDE) $(EXTRA_CFLAGS) \
-               -D__KERNEL__ -Wno-unused-value -Wno-pointer-sign \
+               -D__KERNEL__ -D__ASM_SYSREG_H -Wno-unused-value 
-Wno-pointer-sign \
                 -O2 -emit-llvm -c $< -o -| $(LLC) -march=bpf 

-filetype=obj -o $@

Is it ok for upstream too?

Thanks,
Yang

Quoted 1 line
#3 lore raw
Quoted 16 lines · Yang
Quoted 6 lines

Try hacking it by adding #define __ASM_SYSREG_H to prevent inclusion of that file? all those asms are unused anyway.
tracing samples include kernel headers to have exact struct layout. and #include skbuff.h can be removed from almost all samples. In-kernel 'struct sk_buff' only used inside tracex1_kern.c

The below change works:

        clang $(NOSTDINC_FLAGS) $(LINUXINCLUDE) $(EXTRA_CFLAGS) \
(quoted diff) +1 -3
-               -D__KERNEL__ -Wno-unused-value -Wno-pointer-sign \
+               -D__KERNEL__ -D__ASM_SYSREG_H -Wno-unused-value
-Wno-pointer-sign \
                -O2 -emit-llvm -c $< -o -| $(LLC) -march=bpf -filetype=obj
-o $@
Quoted 1 line

Is it ok for upstream too?

It's fragile and ugly, but I don't see a cleaner way for arm64, so yes, please submit it officially.

Thanks!

author
#4 lore raw
Quoted 17 lines · Alexei Starovoitov
Quoted 16 lines · Yang
Quoted 6 lines

Try hacking it by adding #define __ASM_SYSREG_H to prevent inclusion of that file? all those asms are unused anyway.
tracing samples include kernel headers to have exact struct layout. and #include skbuff.h can be removed from almost all samples. In-kernel 'struct sk_buff' only used inside tracex1_kern.c

The below change works:

         clang $(NOSTDINC_FLAGS) $(LINUXINCLUDE) $(EXTRA_CFLAGS) \
(quoted diff) +1 -3
-               -D__KERNEL__ -Wno-unused-value -Wno-pointer-sign \
+               -D__KERNEL__ -D__ASM_SYSREG_H -Wno-unused-value
-Wno-pointer-sign \
                 -O2 -emit-llvm -c $< -o -| $(LLC) -march=bpf -filetype=obj
-o $@
Quoted 4 lines
Quoted 1 line

Is it ok for upstream too?

It's fragile and ugly, but I don't see a cleaner way for arm64, so yes, please submit it officially.

Yes, but anyway I could add some comment to explain the workaround. If anyone else run into such issue in the future, at least they could know how to workaround it.

Thanks,
Yang

Quoted 3 lines

Thanks!