Skip to contentSkip to conversation

Function Return ABI (was Re: Argument passing and ABI: GCC vs clang)

DISCUSSION Alexei Starovoitov 2 participants 2 messages Feb 20 3 messages not in the archive BPF

lore
Share & tools
lore
#3 lore raw
In reply to a message not in the archive
Quoted 67 lines · Vineet Gupta
Quoted 55 lines · Vineet Gupta
Quoted 4 lines · Jose E. Marchesi

Hello people!

Yesterday evening Vineet pointed out in IRC some differences between the way GCC handles argument passing compared to clang/LLVM.

For illustration, here's what a simple test looks like across clang, gcc and x86 and aarch64
https://godbolt.org/z/oerG3eT6f

typedef struct {
     int a;
     short b;
     signed char c;
} my_t;

void foo(signed char, short, int);

char args_setup(my_t *s)

{
     foo(s->c, s->b, s->a);
}

int args_consume (signed char a, short b, int c)

{
   int x = a;
   int y = b;
   int z = c;

   return x + y + z;
}

clang: -O2 -mcpu=v4 | gcc: -O2 -mcpu=v4

                                |
args_setup:                    | args_setup:
         r2 = *(s16 *)(r1 + 4)  |         r3 = *(u32 *) (r1+0)
         w3 = *(u32 *)(r1 + 0)  |         r2 = *(u16 *) (r1+4)
         r1 = *(s8 *)(r1 + 6)   |         r1 = *(u8 *) (r1+6)
         call foo               |         call foo
         exit                   |         exit
                                |
args_consume:                  | args_consume:
         w0 = w2                |         r2 = (s16) r2
         w0 += w1               |         r0 = r3
         w0 += w3               |         r1 = (s8) r1
         exit                   |         w1 += w2
                                |         w0 += w1
                                |         exit

So clang is narrowing the args on caller site while gcc is doing this in caller. This is not consistent and needs to fixed in one of the compilers. Doing this in callee seems like a better/safer approach as doesn't assume caller to always be doing the right thing, specially when mixing bpf user code with kernel etc.

Quoted 5 lines

BTW where is the w<N> assembler notation and semantics (32-bit, zero extension etc) documented. I don't anything relevant in [1]
Presumably it is just a compiler notation so [1] might not be the ideal place.

[1] https://docs.kernel.org/bpf/standardization/instruction-set.html

So the function arguments issue is captured as PR/124171 and I have a tentative fix for that, but...
There's second half of the problem which is promotion of narrow function return values.

Quoted 4 lines

It seems the existing clang ABI is not symmetrical. The function arguments promotion is handled in callee, but function return is handled in caller [1] which is a bit atypical IMO and would need to be handled in gcc as well.

hmm. I thought we already concluded that for clang-x86 and clang-bpf argument extension is handled in *caller*.
The same thing for returns. It's a caller responsibility. gcc-bpf doing args in the callee only is broken and has to be fixed.

Quoted 16 lines

Since we are on the topic of ABI change, I wanted to surface that as well before we go off and tackle that.

Simple example

_Bool bar_bool(void);

int ret_caller(void) {
       if (bar_bool() != 1) return 0; else return 1;
}

char ret_callee(my_t *s)

{
     return s->c;    // c is a char in a struct
}
Quoted 8 lines

ret_caller:

     call    bar_bool
     r0 &= 0xff        <-- ret promotion in caller
     exit

ret_callee:

     r0 = *(u8 *) (r1+6) <-- no ret promotion in callee
     exit

Exactly as it should be because x86 will populate 8-bit sub registers in the callee and bpf side has to do r0 &= 0xff in the caller.

#4 lore raw
Quoted 4 lines · Alexei Starovoitov
Quoted 4 lines

It seems the existing clang ABI is not symmetrical. The function arguments promotion is handled in callee, but function return is handled in caller [1] which is a bit atypical IMO and would need to be handled in gcc as well.

Ignore this as it is clearly wrong (my jetlag and waking up at 3 am is to blame :-)

With PR/124171 gcc-bpf will promote args in caller same as llvm (not callee)

Quoted 4 lines

hmm. I thought we already concluded that for clang-x86 and clang-bpf argument extension is handled in*caller*.
The same thing for returns. It's a caller responsibility. gcc-bpf doing args in the callee only is broken and has to be fixed.

Correct, agreed.

Quoted 27 lines
Quoted 16 lines

Since we are on the topic of ABI change, I wanted to surface that as well before we go off and tackle that.

Simple example

_Bool bar_bool(void);

int ret_caller(void) {
        if (bar_bool() != 1) return 0; else return 1;
}

char ret_callee(my_t *s)

{
      return s->c;    // c is a char in a struct
}
Quoted 8 lines

ret_caller:

      call    bar_bool
      r0 &= 0xff        <-- ret promotion in caller
      exit

ret_callee:

      r0 = *(u8 *) (r1+6) <-- no ret promotion in callee
      exit

Exactly as it should be because x86 will populate 8-bit sub registers in the callee and bpf side has to do r0 &= 0xff in the caller.