clang asm goto issue small reproducer
Quoted 6 lines · Mathieu Desnoyers
Hi Nick,
I have a small reproducer for the asm goto issue we discussed over irc. I have axed away all code that is not the problematic function by hand. Hopefully this won't change the overall optimization context too much.
If I cut the compilation pipeline off at after "middle-end" optimizations, I can see that this_cpu_list_pop returns NULL for test.c while in test-asm.c, it's a phi node of either NULL or head (which looks correct; for the test-asm.c case).
"GVNPass" seems to be converting
return: ; preds =
%cleanup.thread19, %cleanup.thread
%retval.118 = phi %struct.percpu_list_node* [ %9, %cleanup.thread ],
[ null, %cleanup.thread19 ]
ret %struct.percpu_list_node* %retval.118
into:
return: ; preds =
%cleanup.thread19, %cleanup.thread
ret %struct.percpu_list_node* null
Which drops the
%9 = load %struct.percpu_list_node*, %struct.percpu_list_node**
%head, align 8, !tbaa !9
I think GVN may be eliminating a phi that it should not be. Specifically, it seems that GVN thinks the default/fallthrough successor is unreachable, which seems incorrect to me. The predecessor lists all look correct to me, so GVN may be stumbling on callbr itself.
$ clang -O2 test.c -emit-llvm -S -o - -Xclang -disable-llvm-passes > test.ll $ opt -O2 -print-before=gvn -print-after=gvn -filter-print-funcs=this_cpu_list_pop test.ll -o /dev/null 2> test.gvn.ll
Then using this reproducer, I was able to narrow it down a little further:
$ opt -O2 -print-before=gvn -print-module-scope --filter-print-funcs=this_cpu_list_pop test.ll -S 2>test.pre-gvn.ll $ llvm-extract --func=this_cpu_list_pop test.pre-gvn.ll -o test.pre-gvn.this_cpu_list_pop.bc --recursive $ llvm-dis test.pre-gvn.this_cpu_list_pop.bc $ cat repro.sh #!/usr/bin/env sh set -e grep -q "%retval.118 = phi %struct.percpu_list_node\* \[ %9, %cleanup.thread \], \[ null, %cleanup.thread19 \]" $1 grep -q "ret %struct.percpu_list_node\* %retval.118" $1 OUT=$(opt -passes=gvn $1 -S -o -) echo $OUT | grep -q "ret %struct.percpu_list_node\* null" $ llvm-reduce --test=./repro.sh test.pre-gvn.this_cpu_list_pop.ll
Then pared down further manually, see reduced.ll
I've filed https://github.com/llvm/llvm-project/issues/52735, let's discuss more there?
Quoted 17 lines
on x86-64:
Buggy code:
clang-13 -O2 -o test.otest.cOK code (added a dummy
asm("")):
clang-13 -O2 -o test-asm.otest-asm.cThanks,
Mathieu
--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
Signature · 2 lines
Thanks, ~Nick Desaulniers
Quoted 25 lines · Nick Desaulniers
Quoted 6 lines · Mathieu Desnoyers
Hi Nick,
I have a small reproducer for the asm goto issue we discussed over irc. I have axed away all code that is not the problematic function by hand. Hopefully this won't change the overall optimization context too much.
If I cut the compilation pipeline off at after "middle-end" optimizations, I can see that
this_cpu_list_popreturnsNULLfortest.cwhile intest-asm.c, it's a phi node of either NULL or head (which looks correct; for thetest-asm.ccase)."GVNPass" seems to be converting
return: ; preds =
%cleanup.thread19, %cleanup.thread
%retval.118 = phi %struct.percpu_list_node* [ %9, %cleanup.thread ],
[ null, %cleanup.thread19 ]
ret %struct.percpu_list_node* %retval.118into:
Quoted 7 lines
return: ; preds =
%cleanup.thread19, %cleanup.thread
ret %struct.percpu_list_node* nullWhich drops the
%9 = load %struct.percpu_list_node*, %struct.percpu_list_node**
%head, align 8, !tbaa !9
Ah, Eli pointed out that I'm chasing the wrong squirrel. From test.c we have:
struct percpu_list_node *head;
load = (intptr_t *)&head;
ret = rseq_cmpnev_storeoffp_load(targetptr, expectnot,
offset, load, cpu);
then in rseq_cmpnev_storeoffp_load():__asm__ __volatile__ goto (
:
:
[load] "m" (*load));
Did we ever initialize head? Isn't this equivalent to:
int* foo (void) {
int *head;
int **load;
load = &head;
asm(""::"m"(*load):"memory");
return head;
}
Isn't that equivalent to:
int* foo (void) {
int *head;
asm(""::"m"(head):"memory");
return head;
}
For which neither compiler warns for -Wuninitialized? They would if head was an argument to a function call...
Quoted 54 lines
I think GVN may be eliminating a phi that it should not be. Specifically, it seems that GVN thinks the default/fallthrough successor is unreachable, which seems incorrect to me. The predecessor lists all look correct to me, so GVN may be stumbling on callbr itself.
$ clang -O2 test.c -emit-llvm -S -o - -Xclang -disable-llvm-passes > test.ll $ opt -O2 -print-before=gvn -print-after=gvn -filter-print-funcs=this_cpu_list_pop test.ll -o /dev/null 2> test.gvn.llThen using this reproducer, I was able to narrow it down a little further:
$ opt -O2 -print-before=gvn -print-module-scope --filter-print-funcs=this_cpu_list_pop test.ll -S 2>test.pre-gvn.ll $ llvm-extract --func=this_cpu_list_pop test.pre-gvn.ll -o test.pre-gvn.this_cpu_list_pop.bc --recursive $ llvm-dis test.pre-gvn.this_cpu_list_pop.bc $ cat repro.sh #!/usr/bin/env sh set -e grep -q "%retval.118 = phi %struct.percpu_list_node\* \[ %9, %cleanup.thread \], \[ null, %cleanup.thread19 \]" $1 grep -q "ret %struct.percpu_list_node\* %retval.118" $1 OUT=$(opt -passes=gvn $1 -S -o -) echo $OUT | grep -q "ret %struct.percpu_list_node\* null" $ llvm-reduce --test=./repro.sh test.pre-gvn.this_cpu_list_pop.llThen pared down further manually, see reduced.ll
I've filed https://github.com/llvm/llvm-project/issues/52735, let's discuss more there?
Quoted 9 lines
on x86-64:
Buggy code:
clang-13 -O2 -o test.otest.cOK code (added a dummy
asm("")):
clang-13 -O2 -o test-asm.otest-asm.cQuoted 8 lines
Thanks,
Mathieu
--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
Quoted 3 lines
--
Thanks,
~Nick Desaulniers
--
Thanks,
~Nick Desaulniers
Quoted 61 lines · Nick Desaulniers
Quoted 25 lines · Nick Desaulniers
Quoted 6 lines · Mathieu Desnoyers
Hi Nick,
I have a small reproducer for the asm goto issue we discussed over irc. I have axed away all code that is not the problematic function by hand. Hopefully this won't change the overall optimization context too much.
If I cut the compilation pipeline off at after "middle-end" optimizations, I can see that
this_cpu_list_popreturnsNULLfortest.cwhile intest-asm.c, it's a phi node of either NULL or head (which looks correct; for thetest-asm.ccase)."GVNPass" seems to be converting
return: ; preds =
%cleanup.thread19, %cleanup.thread
%retval.118 = phi %struct.percpu_list_node* [ %9, %cleanup.thread ],
[ null, %cleanup.thread19 ]
ret %struct.percpu_list_node* %retval.118into:
Quoted 7 lines
return: ; preds =
%cleanup.thread19, %cleanup.thread
ret %struct.percpu_list_node* nullWhich drops the
%9 = load %struct.percpu_list_node*, %struct.percpu_list_node**
%head, align 8, !tbaa !9Ah, Eli pointed out that I'm chasing the wrong squirrel. From
test.cwe have:struct
percpu_list_node*head;
load = (intptr_t*)&head;
ret = rseq_cmpnev_storeoffp_load(targetptr, expectnot,offset, load, cpu);then in
rseq_cmpnev_storeoffp_load():__asm____volatile__goto (
:
:
[load] "m" (*load));Did we ever initialize head? Isn't this equivalent to:
int* foo (void) {
int *head;
int **load;
load = &head;
asm(""::"m"(*load):"memory");
return head;
}Isn't that equivalent to:
Quoted 8 lines
int* foo (void) {
int *head;
asm(""::"m"(head):"memory");
return head;
}For which neither compiler warns for -Wuninitialized? They would if head was an argument to a function call...
What's going on here...
$ cat y.c
void x (void) {
int y;
asm(""::"r"(y));
}
void z (void) {
int y;
asm(""::"m"(y));
}
$ clang -Wuninitialized y.c -c
y.c:3:15: warning: variable 'y' is uninitialized when used here
[-Wuninitialized]
asm(""::"r"(y));
^
y.c:2:8: note: initialize the variable 'y' to silence this warning
int y;
^
= 0
1 warning generated.
$ gcc -Wuninitialized y.c -c
y.c: In function ‘x’:
y.c:3:3: warning: ‘y’ is used uninitialized [-Wuninitialized]
3 | asm(""::"r"(y));
| ^~~Only 1 warning each? I expected 2.
Is there something special about "m" constraints and -Wuninitialized?
Signature · 2 lines
Thanks, ~Nick Desaulniers
Quoted 99 lines · ----- On Dec 15, 2021, at 7:25 PM, ndesaulniers [email protected] wrote:
Quoted 61 lines · Nick Desaulniers
Quoted 25 lines · Nick Desaulniers
Quoted 6 lines · Mathieu Desnoyers
Hi Nick,
I have a small reproducer for the asm goto issue we discussed over irc. I have axed away all code that is not the problematic function by hand. Hopefully this won't change the overall optimization context too much.
If I cut the compilation pipeline off at after "middle-end" optimizations, I can see that
this_cpu_list_popreturnsNULLfortest.cwhile intest-asm.c, it's a phi node of either NULL or head (which looks correct; for thetest-asm.ccase)."GVNPass" seems to be converting
return: ; preds =
%cleanup.thread19, %cleanup.thread
%retval.118 = phi %struct.percpu_list_node* [ %9, %cleanup.thread ],
[ null, %cleanup.thread19 ]
ret %struct.percpu_list_node* %retval.118into:
Quoted 7 lines
return: ; preds =
%cleanup.thread19, %cleanup.thread
ret %struct.percpu_list_node* nullWhich drops the
%9 = load %struct.percpu_list_node*, %struct.percpu_list_node**
%head, align 8, !tbaa !9Ah, Eli pointed out that I'm chasing the wrong squirrel. From
test.cwe have:struct
percpu_list_node*head;
load = (intptr_t*)&head;
ret = rseq_cmpnev_storeoffp_load(targetptr, expectnot,offset, load, cpu);then in
rseq_cmpnev_storeoffp_load():__asm____volatile__goto (
:
:
[load] "m" (*load));Did we ever initialize head? Isn't this equivalent to:
int* foo (void) {
int *head;
int **load;
load = &head;
asm(""::"m"(*load):"memory");
return head;
}Isn't that equivalent to:
Quoted 8 lines
int* foo (void) {
int *head;
asm(""::"m"(head):"memory");
return head;
}For which neither compiler warns for -Wuninitialized? They would if head was an argument to a function call...
What's going on here...
$ cat y.c void x (void) { int y; asm(""::"r"(y)); } void z (void) { int y; asm(""::"m"(y)); } $ clang -Wuninitialized y.c -c y.c:3:15: warning: variable 'y' is uninitialized when used here [-Wuninitialized] asm(""::"r"(y)); ^ y.c:2:8: note: initialize the variable 'y' to silence this warning int y; ^ = 0 1 warning generated. $ gcc -Wuninitialized y.c -c y.c: In function ‘x’: y.c:3:3: warning: ‘y’ is used uninitialized [-Wuninitialized] 3 | asm(""::"r"(y)); | ^~~
Quoted 3 lines
Only 1 warning each? I expected 2.
Is there something special about "m" constraints and -Wuninitialized?
In my work-arounds for the lack of output operands with asm goto, I've used either of those tricks across various architectures:
- "m" (y) input operand with "memory" clobber, or
- "r" (&y) input operand with "memory" clobber
to achieve writing to a memory area from assembler without having output operands available.
It may be stretching the definitions, but those were my (perhaps ill advised) expectations.
Thanks,
Mathieu
Quoted 4 lines
--
Thanks,
~Nick Desaulniers
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
Quoted 60 lines · ----- On Dec 15, 2021, at 7:18 PM, ndesaulniers [email protected] wrote:
Quoted 25 lines · Nick Desaulniers
Quoted 6 lines · Mathieu Desnoyers
Hi Nick,
I have a small reproducer for the asm goto issue we discussed over irc. I have axed away all code that is not the problematic function by hand. Hopefully this won't change the overall optimization context too much.
If I cut the compilation pipeline off at after "middle-end" optimizations, I can see that
this_cpu_list_popreturnsNULLfortest.cwhile intest-asm.c, it's a phi node of either NULL or head (which looks correct; for thetest-asm.ccase)."GVNPass" seems to be converting
return: ; preds =
%cleanup.thread19, %cleanup.thread
%retval.118 = phi %struct.percpu_list_node* [ %9, %cleanup.thread ],
[ null, %cleanup.thread19 ]
ret %struct.percpu_list_node* %retval.118into:
Quoted 7 lines
return: ; preds =
%cleanup.thread19, %cleanup.thread
ret %struct.percpu_list_node* nullWhich drops the
%9 = load %struct.percpu_list_node*, %struct.percpu_list_node**
%head, align 8, !tbaa !9Ah, Eli pointed out that I'm chasing the wrong squirrel. From
test.cwe have:struct
percpu_list_node*head;
load = (intptr_t*)&head;
ret = rseq_cmpnev_storeoffp_load(targetptr, expectnot,offset, load, cpu);then in
rseq_cmpnev_storeoffp_load():__asm____volatile__goto (
:
:
[load] "m" (*load));Did we ever initialize head? Isn't this equivalent to:
int* foo (void) {
int *head;
int **load;
load = &head;
asm(""::"m"(*load):"memory");
return head;
}Isn't that equivalent to:
Quoted 8 lines
int* foo (void) {
int *head;
asm(""::"m"(head):"memory");
return head;
}For which neither compiler warns for -Wuninitialized? They would if head was an argument to a function call...
This is a situation where I had to work around the fact that older gcc did not support output operands. I expected that having both a "m" input operand and a "memory" clobber would allow the asm to write to the memory location pointed to by the "m" input operand, and the "memory" clobber was expected to prevent the compiler from presuming anything about memory state across the asm.
But I suspect I was expecting too much here. What would be the best course of action to fix these asm statements ? Do I have any way to do this without requiring a dependency on gcc/clang that support asm goto with output operands ?
Thanks,
Mathieu
Quoted 64 lines
Quoted 55 lines
I think GVN may be eliminating a phi that it should not be. Specifically, it seems that GVN thinks the default/fallthrough successor is unreachable, which seems incorrect to me. The predecessor lists all look correct to me, so GVN may be stumbling on callbr itself.
$ clang -O2 test.c -emit-llvm -S -o - -Xclang -disable-llvm-passes > test.ll $ opt -O2 -print-before=gvn -print-after=gvn -filter-print-funcs=this_cpu_list_pop test.ll -o /dev/null 2> test.gvn.llThen using this reproducer, I was able to narrow it down a little further:
$ opt -O2 -print-before=gvn -print-module-scope --filter-print-funcs=this_cpu_list_pop test.ll -S 2>test.pre-gvn.ll $ llvm-extract --func=this_cpu_list_pop test.pre-gvn.ll -o test.pre-gvn.this_cpu_list_pop.bc --recursive $ llvm-dis test.pre-gvn.this_cpu_list_pop.bc $ cat repro.sh #!/usr/bin/env sh set -e grep -q "%retval.118 = phi %struct.percpu_list_node\* \[ %9, %cleanup.thread \], \[ null, %cleanup.thread19 \]" $1 grep -q "ret %struct.percpu_list_node\* %retval.118" $1 OUT=$(opt -passes=gvn $1 -S -o -) echo $OUT | grep -q "ret %struct.percpu_list_node\* null" $ llvm-reduce --test=./repro.sh test.pre-gvn.this_cpu_list_pop.llThen pared down further manually, see reduced.ll
I've filed https://github.com/llvm/llvm-project/issues/52735, let's discuss more there?
Quoted 9 lines
on x86-64:
Buggy code:
clang-13 -O2 -o test.otest.cOK code (added a dummy
asm("")):
clang-13 -O2 -o test-asm.otest-asm.cQuoted 8 lines
Thanks,
Mathieu
--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.comQuoted 3 lines
--
Thanks,
~Nick Desaulniers--
Thanks,
~Nick Desaulniers
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
Hi Nick,
Here is the result of:
opt-13 -passes='gvn,gvn-hoist,gvn-sink,print<memoryssa>' -filter-print-funcs=this_cpu_list_pop test-asm.ll -o /dev/null 2> test-asm.gvn.ll opt-13 -passes='gvn,gvn-hoist,gvn-sink,print<memoryssa>' -filter-print-funcs=this_cpu_list_pop test.ll -o /dev/null 2> test.gvn.ll
I notice a missing MemoryPhi in the case without the extra asm("").
Thanks,
Mathieu
Quoted 106 lines · ----- On Dec 15, 2021, at 7:25 PM, ndesaulniers [email protected] wrote:
Quoted 61 lines · Nick Desaulniers
Quoted 25 lines · Nick Desaulniers
Quoted 6 lines · Mathieu Desnoyers
Hi Nick,
I have a small reproducer for the asm goto issue we discussed over irc. I have axed away all code that is not the problematic function by hand. Hopefully this won't change the overall optimization context too much.
If I cut the compilation pipeline off at after "middle-end" optimizations, I can see that
this_cpu_list_popreturnsNULLfortest.cwhile intest-asm.c, it's a phi node of either NULL or head (which looks correct; for thetest-asm.ccase)."GVNPass" seems to be converting
return: ; preds =
%cleanup.thread19, %cleanup.thread
%retval.118 = phi %struct.percpu_list_node* [ %9, %cleanup.thread ],
[ null, %cleanup.thread19 ]
ret %struct.percpu_list_node* %retval.118into:
Quoted 7 lines
return: ; preds =
%cleanup.thread19, %cleanup.thread
ret %struct.percpu_list_node* nullWhich drops the
%9 = load %struct.percpu_list_node*, %struct.percpu_list_node**
%head, align 8, !tbaa !9Ah, Eli pointed out that I'm chasing the wrong squirrel. From
test.cwe have:struct
percpu_list_node*head;
load = (intptr_t*)&head;
ret = rseq_cmpnev_storeoffp_load(targetptr, expectnot,offset, load, cpu);then in
rseq_cmpnev_storeoffp_load():__asm____volatile__goto (
:
:
[load] "m" (*load));Did we ever initialize head? Isn't this equivalent to:
int* foo (void) {
int *head;
int **load;
load = &head;
asm(""::"m"(*load):"memory");
return head;
}Isn't that equivalent to:
Quoted 8 lines
int* foo (void) {
int *head;
asm(""::"m"(head):"memory");
return head;
}For which neither compiler warns for -Wuninitialized? They would if head was an argument to a function call...
What's going on here...
$ cat y.c void x (void) { int y; asm(""::"r"(y)); } void z (void) { int y; asm(""::"m"(y)); } $ clang -Wuninitialized y.c -c y.c:3:15: warning: variable 'y' is uninitialized when used here [-Wuninitialized] asm(""::"r"(y)); ^ y.c:2:8: note: initialize the variable 'y' to silence this warning int y; ^ = 0 1 warning generated. $ gcc -Wuninitialized y.c -c y.c: In function ‘x’: y.c:3:3: warning: ‘y’ is used uninitialized [-Wuninitialized] 3 | asm(""::"r"(y)); | ^~~Only 1 warning each? I expected 2.
Is there something special about "m" constraints and -Wuninitialized?
--
Thanks,
~Nick Desaulniers
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com