two atomic_cmpxchg() related fixes
cover two atomic_cmpxchg() related fixes
The story short: compiler can generate code that does two distinct fetches of *ind_addr for old and _old. If that happens we can not figure out if we had the desired xchg or not.
Halil Pasic (2):
virtio-ccw: fix virtio_set_ind_atomic
s390x/pci: fix set_ind_atomic
hw/s390x/s390-pci-bus.c | 16 +++++++++------- hw/s390x/virtio-ccw.c | 18 ++++++++++-------- 2 files changed, 19 insertions(+), 15 deletions(-)
base-commit: 7d3660e79830a069f1848bb4fa1cdf8f666424fb
Signature · 1 line
2.17.1
PATCH 1/2 virtio-ccw: fix virtio_set_ind_atomic
The atomic_cmpxchg() loop is broken because we occasionally end up with old and _old having different values (a legit compiler can generate code that accessed *ind_addr again to pick up a value for _old instead of using the value of old that was already fetched according to the rules of the abstract machine). This means the underlying CS instruction may use a different old (_old) than the one we intended to use if atomic_cmpxchg() performed the xchg part.
Let us use volatile to force the rules of the abstract machine for accesses to *ind_addr. Let us also rewrite the loop so, we that the new old is used to compute the new desired value if the xchg part is not performed.
--- hw/s390x/virtio-ccw.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)
hw/s390x/virtio-ccw.c +10 -8
@@ -786,9 +786,10 @@ static inline VirtioCcwDevice *to_virtio_ccw_dev_fast(DeviceState *d)static uint8_t virtio_set_ind_atomic (SubchDev *sch,uint64_t ind_loc,uint8_t to_be_set) { -uint8_t ind_old, ind_new; +uint8_t expected, actual;hwaddr len =1 ; -uint8_t *ind_addr; +/* avoid multiple fetches */ +uint8_t volatile *ind_addr; ind_addr =cpu_physical_memory_map (ind_loc, &len, true);if (!ind_addr) {
@@ -796,14 +797,15 @@ static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, __func__, sch->cssid, sch->ssid, sch->schid);return -1 ; } + actual = *ind_addr;do { - ind_old = *ind_addr; - ind_new = ind_old | to_be_set; - }while (atomic_cmpxchg (ind_addr, ind_old, ind_new) != ind_old); -trace_virtio_ccw_set_ind (ind_loc, ind_old, ind_new); -cpu_physical_memory_unmap (ind_addr, len,1 , len); + expected = actual; + actual =atomic_cmpxchg (ind_addr, expected, expected | to_be_set); + }while (actual != expected); +trace_virtio_ccw_set_ind (ind_loc, actual, actual | to_be_set); +cpu_physical_memory_unmap ((void *)ind_addr, len,1 , len); -return ind_old; +return actual; }static void virtio_ccw_notify (DeviceState *d,uint16_t vector)
Signature · 1 line
2.17.1
Re: [PATCH 1/2] virtio-ccw: fix virtio_set_ind_atomic
Quoted 38 lines · Halil Pasic
The
atomic_cmpxchg()loop is broken because we occasionally end up with old and_oldhaving different values (a legit compiler can generate code that accessed*ind_addragain to pick up a value for_oldinstead of using the value of old that was already fetched according to the rules of the abstract machine). This means the underlying CS instruction may use a different old (_old) than the one we intended to use ifatomic_cmpxchg()performed the xchg part.Let us use volatile to force the rules of the abstract machine for accesses to
*ind_addr. Let us also rewrite the loop so, we that the new old is used to compute the new desired value if the xchg part is not performed.Signed-off-by Halil Pasic <[email protected]>Reported-by Andre Wild <[email protected]>Fixes 7e7494627f ("s390x/virtio-ccw: Adapter interrupt support.")--- hw/s390x/virtio-ccw.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)hw/s390x/virtio-ccw.c +3
-2@@ -786,9 +786,10 @@ static inline VirtioCcwDevice *to_virtio_ccw_dev_fast(DeviceState *d)static uint8_t virtio_set_ind_atomic (SubchDev *sch,uint64_t ind_loc,uint8_t to_be_set) { -uint8_t ind_old, ind_new; +uint8_t expected, actual;hwaddr len =1 ; -uint8_t *ind_addr; +/* avoid multiple fetches */ +uint8_t volatile *ind_addr; ind_addr =cpu_physical_memory_map (ind_loc, &len, true);if (!ind_addr) {
hw/s390x/virtio-ccw.c static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, · line 800
Quoted 6 lines
hw/s390x/virtio-ccw.c +1
-1@@ -796,14 +797,15 @@ static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, __func__, sch->cssid, sch->ssid, sch->schid);return -1 ; } + actual = *ind_addr; do { - ind_old = *ind_addr;
to make things easier to understand. Adding a barrier in here also fixes the issue. Reasoning follows below:
hw/s390x/virtio-ccw.c static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, · line 801
Quoted 1 line
hw/s390x/virtio-ccw.c +0
-1- ind_new = ind_old | to_be_set;
with an analysis from Andreas (cc)
#define atomic_cmpxchg__nocheck(ptr, old, new) ({ \
typeof_strip_qual(*ptr) _old = (old); \
(void)__atomic_compare_exchange_n(ptr, &_old, new, false, \
__ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST); \
_old; \ })
ind_old is copied into _old in the macro. Instead of doing the copy from the register the compiler reloads the value from memory. The result is that _old and ind_old end up having different values. _old in r1 with the bits set already and ind_old in r10 with the bits cleared. _old gets updated by CS and matches ind_old afterwards - both with the bits being 0. So the != compare is false and the loop is left without having set any bits.
Paolo (to),
I am asking myself if it would be safer to add a barrier or something like this in the macros in include/qemu/atomic.h.
Quoted 15 lines
hw/s390x/virtio-ccw.c +6
-4- }while (atomic_cmpxchg (ind_addr, ind_old, ind_new) != ind_old); -trace_virtio_ccw_set_ind (ind_loc, ind_old, ind_new); -cpu_physical_memory_unmap (ind_addr, len,1 , len); + expected = actual; + actual =atomic_cmpxchg (ind_addr, expected, expected | to_be_set); + }while (actual != expected); +trace_virtio_ccw_set_ind (ind_loc, actual, actual | to_be_set); +cpu_physical_memory_unmap ((void *)ind_addr, len,1 , len); -return ind_old; +return actual; }static void virtio_ccw_notify (DeviceState *d,uint16_t vector)
Re: [PATCH 1/2] virtio-ccw: fix virtio_set_ind_atomic
Quoted 73 lines · On Tue, 16 Jun 2020 07:58:53 +0200 Christian Borntraeger…
Quoted 38 lines · Halil Pasic
The
atomic_cmpxchg()loop is broken because we occasionally end up with old and_oldhaving different values (a legit compiler can generate code that accessed*ind_addragain to pick up a value for_oldinstead of using the value of old that was already fetched according to the rules of the abstract machine). This means the underlying CS instruction may use a different old (_old) than the one we intended to use ifatomic_cmpxchg()performed the xchg part.Let us use volatile to force the rules of the abstract machine for accesses to
*ind_addr. Let us also rewrite the loop so, we that the new old is used to compute the new desired value if the xchg part is not performed.Signed-off-by Halil Pasic <[email protected]>Reported-by Andre Wild <[email protected]>Fixes 7e7494627f ("s390x/virtio-ccw: Adapter interrupt support.")--- hw/s390x/virtio-ccw.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)hw/s390x/virtio-ccw.c +3
-2@@ -786,9 +786,10 @@ static inline VirtioCcwDevice *to_virtio_ccw_dev_fast(DeviceState *d)static uint8_t virtio_set_ind_atomic (SubchDev *sch,uint64_t ind_loc,uint8_t to_be_set) { -uint8_t ind_old, ind_new; +uint8_t expected, actual;hwaddr len =1 ; -uint8_t *ind_addr; +/* avoid multiple fetches */ +uint8_t volatile *ind_addr; ind_addr =cpu_physical_memory_map (ind_loc, &len, true);if (!ind_addr) {hw/s390x/virtio-ccw.c static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, · line 800
Quoted 6 lines
hw/s390x/virtio-ccw.c +1
-1@@ -796,14 +797,15 @@ static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, __func__, sch->cssid, sch->ssid, sch->schid);return -1 ; } + actual = *ind_addr; do { - ind_old = *ind_addr;to make things easier to understand. Adding a barrier in here also fixes the issue. Reasoning follows below:
hw/s390x/virtio-ccw.c static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, · line 801
Quoted 1 line
hw/s390x/virtio-ccw.c +0
-1- ind_new = ind_old | to_be_set;with an analysis from Andreas (cc)
#define
atomic_cmpxchg__nocheck(ptr, old, new)({ \typeof_strip_qual(*ptr) _old = (old); \ (void)__atomic_compare_exchange_n(ptr, &_old, new, false, \ __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST); \ _old; \})
ind_oldis copied into_oldin the macro. Instead of doing the copy from the register the compiler reloads the value from memory. The result is that_oldandind_oldend up having different values._oldin r1 with the bits set already andind_oldin r10 with the bits cleared._oldgets updated by CS and matchesind_oldafterwards - both with the bits being 0. So the != compare is false and the loop is left without having set any bits.
Quoted 3 lines
Paolo (to),
I am asking myself if it would be safer to add a barrier or something like this in the macros ininclude/qemu/atomic.h.
I'm also wondering whether this has been seen on other architectures as well? There are also some callers in non-s390x code, and dealing with this in common code would catch them as well.
Quoted 21 lines
Quoted 19 lines
hw/s390x/virtio-ccw.c +6
-4- }while (atomic_cmpxchg (ind_addr, ind_old, ind_new) != ind_old); -trace_virtio_ccw_set_ind (ind_loc, ind_old, ind_new); -cpu_physical_memory_unmap (ind_addr, len,1 , len); + expected = actual; + actual =atomic_cmpxchg (ind_addr, expected, expected | to_be_set); + }while (actual != expected); +trace_virtio_ccw_set_ind (ind_loc, actual, actual | to_be_set); +cpu_physical_memory_unmap ((void *)ind_addr, len,1 , len); -return ind_old; +return actual; }static void virtio_ccw_notify (DeviceState *d,uint16_t vector)
Re: [PATCH 1/2] virtio-ccw: fix virtio_set_ind_atomic
Quoted 76 lines · Cornelia Huck
Quoted 73 lines · On Tue, 16 Jun 2020 07:58:53 +0200 Christian Borntraeger…
Quoted 38 lines · Halil Pasic
The
atomic_cmpxchg()loop is broken because we occasionally end up with old and_oldhaving different values (a legit compiler can generate code that accessed*ind_addragain to pick up a value for_oldinstead of using the value of old that was already fetched according to the rules of the abstract machine). This means the underlying CS instruction may use a different old (_old) than the one we intended to use ifatomic_cmpxchg()performed the xchg part.Let us use volatile to force the rules of the abstract machine for accesses to
*ind_addr. Let us also rewrite the loop so, we that the new old is used to compute the new desired value if the xchg part is not performed.Signed-off-by Halil Pasic <[email protected]>Reported-by Andre Wild <[email protected]>Fixes 7e7494627f ("s390x/virtio-ccw: Adapter interrupt support.")--- hw/s390x/virtio-ccw.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)hw/s390x/virtio-ccw.c +3
-2@@ -786,9 +786,10 @@ static inline VirtioCcwDevice *to_virtio_ccw_dev_fast(DeviceState *d)static uint8_t virtio_set_ind_atomic (SubchDev *sch,uint64_t ind_loc,uint8_t to_be_set) { -uint8_t ind_old, ind_new; +uint8_t expected, actual;hwaddr len =1 ; -uint8_t *ind_addr; +/* avoid multiple fetches */ +uint8_t volatile *ind_addr; ind_addr =cpu_physical_memory_map (ind_loc, &len, true);if (!ind_addr) {hw/s390x/virtio-ccw.c static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, · line 800
Quoted 6 lines
hw/s390x/virtio-ccw.c +1
-1@@ -796,14 +797,15 @@ static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, __func__, sch->cssid, sch->ssid, sch->schid);return -1 ; } + actual = *ind_addr; do { - ind_old = *ind_addr;to make things easier to understand. Adding a barrier in here also fixes the issue. Reasoning follows below:
hw/s390x/virtio-ccw.c static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, · line 801
Quoted 1 line
hw/s390x/virtio-ccw.c +0
-1- ind_new = ind_old | to_be_set;with an analysis from Andreas (cc)
#define
atomic_cmpxchg__nocheck(ptr, old, new)({ \typeof_strip_qual(*ptr) _old = (old); \ (void)__atomic_compare_exchange_n(ptr, &_old, new, false, \ __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST); \ _old; \})
ind_oldis copied into_oldin the macro. Instead of doing the copy from the register the compiler reloads the value from memory. The result is that_oldandind_oldend up having different values._oldin r1 with the bits set already andind_oldin r10 with the bits cleared._oldgets updated by CS and matchesind_oldafterwards - both with the bits being 0. So the != compare is false and the loop is left without having set any bits.
Quoted 3 lines
Quoted 3 lines
Paolo (to),
I am asking myself if it would be safer to add a barrier or something like this in the macros ininclude/qemu/atomic.h.
Having said this, I think that the refactoring from Halil (to re-use actual) also makes sense independent of the fix.
Quoted 27 lines
I'm also wondering whether this has been seen on other architectures as well? There are also some callers in non-s390x code, and dealing with this in common code would catch them as well.
Quoted 21 lines
Quoted 19 lines
hw/s390x/virtio-ccw.c +6
-4- }while (atomic_cmpxchg (ind_addr, ind_old, ind_new) != ind_old); -trace_virtio_ccw_set_ind (ind_loc, ind_old, ind_new); -cpu_physical_memory_unmap (ind_addr, len,1 , len); + expected = actual; + actual =atomic_cmpxchg (ind_addr, expected, expected | to_be_set); + }while (actual != expected); +trace_virtio_ccw_set_ind (ind_loc, actual, actual | to_be_set); +cpu_physical_memory_unmap ((void *)ind_addr, len,1 , len); -return ind_old; +return actual; }static void virtio_ccw_notify (DeviceState *d,uint16_t vector)
Re: [PATCH 1/2] virtio-ccw: fix virtio_set_ind_atomic
Quoted 77 lines · On Tue, 16 Jun 2020 08:45:14 +0200 Christian Borntraeger…
Quoted 76 lines · Cornelia Huck
Quoted 73 lines · On Tue, 16 Jun 2020 07:58:53 +0200 Christian Borntraeger…
Quoted 38 lines · Halil Pasic
The
atomic_cmpxchg()loop is broken because we occasionally end up with old and_oldhaving different values (a legit compiler can generate code that accessed*ind_addragain to pick up a value for_oldinstead of using the value of old that was already fetched according to the rules of the abstract machine). This means the underlying CS instruction may use a different old (_old) than the one we intended to use ifatomic_cmpxchg()performed the xchg part.Let us use volatile to force the rules of the abstract machine for accesses to
*ind_addr. Let us also rewrite the loop so, we that the new old is used to compute the new desired value if the xchg part is not performed.Signed-off-by Halil Pasic <[email protected]>Reported-by Andre Wild <[email protected]>Fixes 7e7494627f ("s390x/virtio-ccw: Adapter interrupt support.")--- hw/s390x/virtio-ccw.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)hw/s390x/virtio-ccw.c +3
-2@@ -786,9 +786,10 @@ static inline VirtioCcwDevice *to_virtio_ccw_dev_fast(DeviceState *d)static uint8_t virtio_set_ind_atomic (SubchDev *sch,uint64_t ind_loc,uint8_t to_be_set) { -uint8_t ind_old, ind_new; +uint8_t expected, actual;hwaddr len =1 ; -uint8_t *ind_addr; +/* avoid multiple fetches */ +uint8_t volatile *ind_addr; ind_addr =cpu_physical_memory_map (ind_loc, &len, true);if (!ind_addr) {hw/s390x/virtio-ccw.c static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, · line 800
Quoted 6 lines
hw/s390x/virtio-ccw.c +1
-1@@ -796,14 +797,15 @@ static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, __func__, sch->cssid, sch->ssid, sch->schid);return -1 ; } + actual = *ind_addr; do { - ind_old = *ind_addr;to make things easier to understand. Adding a barrier in here also fixes the issue. Reasoning follows below:
hw/s390x/virtio-ccw.c static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, · line 801
Quoted 1 line
hw/s390x/virtio-ccw.c +0
-1- ind_new = ind_old | to_be_set;with an analysis from Andreas (cc)
#define
atomic_cmpxchg__nocheck(ptr, old, new)({ \typeof_strip_qual(*ptr) _old = (old); \ (void)__atomic_compare_exchange_n(ptr, &_old, new, false, \ __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST); \ _old; \})
ind_oldis copied into_oldin the macro. Instead of doing the copy from the register the compiler reloads the value from memory. The result is that_oldandind_oldend up having different values._oldin r1 with the bits set already andind_oldin r10 with the bits cleared._oldgets updated by CS and matchesind_oldafterwards - both with the bits being 0. So the != compare is false and the loop is left without having set any bits.
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines
Paolo (to),
I am asking myself if it would be safer to add a barrier or something like this in the macros ininclude/qemu/atomic.h.Having said this, I think that the refactoring from Halil (to re-use actual) also makes sense independent of the fix.
What about adding a barrier instead, as you suggested?
(Still wondering about other users of atomic_cmpxchg(), though.)
Re: [PATCH 1/2] virtio-ccw: fix virtio_set_ind_atomic
In reply to Cornelia HuckQuoted 20 lines · On Tue, 16 Jun 2020 08:33:33 +0200 Cornelia Huck <[email protected]> wrote:
Quoted 20 lines
#define
atomic_cmpxchg__nocheck(ptr, old, new)({ \typeof_strip_qual(*ptr) _old = (old); \ (void)__atomic_compare_exchange_n(ptr, &_old, new, false, \ __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST); \ _old; \})
ind_oldis copied into_oldin the macro. Instead of doing the copy from the register the compiler reloads the value from memory. The result is that_oldandind_oldend up having different values._oldin r1 with the bits set already andind_oldin r10 with the bits cleared._oldgets updated by CS and matchesind_oldafterwards - both with the bits being 0. So the != compare is false and the loop is left without having set any bits.
Quoted 7 lines
Quoted 3 lines
Paolo (to),
I am asking myself if it would be safer to add a barrier or something like this in the macros ininclude/qemu/atomic.h.I'm also wondering whether this has been seen on other architectures as well? There are also some callers in non-s390x code, and dealing with this in common code would catch them as well.
Quite a bunch of users use something like old = atomic_read(..), where atomic_read is documented as in docs/devel/atomics.rst:
- ``atomic_read()`` and ``atomic_set()``; these prevent the compiler from optimizing accesses out of existence and creating unsolicited accesses, but do not otherwise impose any ordering on loads and
stores: both the compiler and the processor are free to reorder them.
Maybe I should have used that instead of volatile, but my problem was that I didn't fully understand what atomic_read() does, and if it does more than we need. I found the documentation just now.
Another bunch uses constants as old, which is also fine. And there is a third bunch where I don't know whats up, partly because I did not dig deep enough.
Regards,
Halil
Re: [PATCH 1/2] virtio-ccw: fix virtio_set_ind_atomic
Quoted 39 lines · Halil Pasic
Quoted 20 lines · On Tue, 16 Jun 2020 08:33:33 +0200 Cornelia Huck <[email protected]> wrote:
Quoted 20 lines
#define
atomic_cmpxchg__nocheck(ptr, old, new)({ \typeof_strip_qual(*ptr) _old = (old); \ (void)__atomic_compare_exchange_n(ptr, &_old, new, false, \ __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST); \ _old; \})
ind_oldis copied into_oldin the macro. Instead of doing the copy from the register the compiler reloads the value from memory. The result is that_oldandind_oldend up having different values._oldin r1 with the bits set already andind_oldin r10 with the bits cleared._oldgets updated by CS and matchesind_oldafterwards - both with the bits being 0. So the != compare is false and the loop is left without having set any bits.Quoted 7 lines
Quoted 3 lines
Paolo (to),
I am asking myself if it would be safer to add a barrier or something like this in the macros ininclude/qemu/atomic.h.I'm also wondering whether this has been seen on other architectures as well? There are also some callers in non-s390x code, and dealing with this in common code would catch them as well.
Quite a bunch of users use something like old =
atomic_read(..), whereatomic_readis documented as indocs/devel/atomics.rst:
- ``atomic_read()`` and ``atomic_set()``; these prevent the compiler from optimizing accesses out of existence and creating unsolicited accesses, but do not otherwise impose any ordering on loads and
stores: both the compiler and the processor are free to reorder them.
Quoted 3 lines
Maybe I should have used that instead of volatile, but my problem was that I didn't fully understand what
atomic_read()does, and if it does more than we need. I found the documentation just now.
IIRC, atomic_read() is the right way of doing it, at least in the kernel. I use such a loop in QEMU in
https://lkml.kernel.org/r/[email protected]
But reading docs/devel/atomics.rst:"Comparison with Linux kernel primitives" I do wonder if that is sufficient.
Any experts around?
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH 1/2] virtio-ccw: fix virtio_set_ind_atomic
Quoted 40 lines · On Fri, 19 Jun 2020 09:33:44 +0200 David Hildenbrand <[email protected]> wrote:
Quoted 39 lines · Halil Pasic
Quoted 20 lines · On Tue, 16 Jun 2020 08:33:33 +0200 Cornelia Huck <[email protected]> wrote:
Quoted 20 lines
#define
atomic_cmpxchg__nocheck(ptr, old, new)({ \typeof_strip_qual(*ptr) _old = (old); \ (void)__atomic_compare_exchange_n(ptr, &_old, new, false, \ __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST); \ _old; \})
ind_oldis copied into_oldin the macro. Instead of doing the copy from the register the compiler reloads the value from memory. The result is that_oldandind_oldend up having different values._oldin r1 with the bits set already andind_oldin r10 with the bits cleared._oldgets updated by CS and matchesind_oldafterwards - both with the bits being 0. So the != compare is false and the loop is left without having set any bits.Quoted 7 lines
Quoted 3 lines
Paolo (to),
I am asking myself if it would be safer to add a barrier or something like this in the macros ininclude/qemu/atomic.h.I'm also wondering whether this has been seen on other architectures as well? There are also some callers in non-s390x code, and dealing with this in common code would catch them as well.
Quite a bunch of users use something like old =
atomic_read(..), whereatomic_readis documented as indocs/devel/atomics.rst:
- ``atomic_read()`` and ``atomic_set()``; these prevent the compiler from optimizing accesses out of existence and creating unsolicited accesses, but do not otherwise impose any ordering on loads and
stores: both the compiler and the processor are free to reorder them.
Quoted 6 lines
Quoted 3 lines
Maybe I should have used that instead of volatile, but my problem was that I didn't fully understand what
atomic_read()does, and if it does more than we need. I found the documentation just now.IIRC,
atomic_read()is the right way of doing it, at least in the kernel.
In kernel I would use READ_ONCE.
https://elixir.bootlin.com/linux/v5.8-rc1/source/include/asm-generic/atomic.h#L171
In this case we are not manipulating an atomic variable. For uint8_t that boils down to an access through a volatile pointer. And that is what I did :).
Quoted 9 lines
I use such a loop in QEMU in
https://lkml.kernel.org/r/[email protected]
But reading docs/devel/atomics.rst:"Comparison with Linux kernel primitives" I do wonder if that is sufficient.
Any experts around?
IMHO what we want here is READ_ONCE, i.e. volatile access, and not necessarily atomic access. But I suppose atomic access implies volatile access (the C11 standard refers to atomic_load as
C atomic_load(volatile A *object)). Because QEMU seems to use atomic_read() in such situations, and does not have READ_ONCE, for me atomic_read would also do.
Regards,
Halil
Re: [PATCH 1/2] virtio-ccw: fix virtio_set_ind_atomic
In reply to Christian BorntraegerQuoted 65 lines · On Tue, 16 Jun 2020 07:58:53 +0200 Christian Borntraeger…
Quoted 38 lines · Halil Pasic
The
atomic_cmpxchg()loop is broken because we occasionally end up with old and_oldhaving different values (a legit compiler can generate code that accessed*ind_addragain to pick up a value for_oldinstead of using the value of old that was already fetched according to the rules of the abstract machine). This means the underlying CS instruction may use a different old (_old) than the one we intended to use ifatomic_cmpxchg()performed the xchg part.Let us use volatile to force the rules of the abstract machine for accesses to
*ind_addr. Let us also rewrite the loop so, we that the new old is used to compute the new desired value if the xchg part is not performed.Signed-off-by Halil Pasic <[email protected]>Reported-by Andre Wild <[email protected]>Fixes 7e7494627f ("s390x/virtio-ccw: Adapter interrupt support.")--- hw/s390x/virtio-ccw.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)hw/s390x/virtio-ccw.c +3
-2@@ -786,9 +786,10 @@ static inline VirtioCcwDevice *to_virtio_ccw_dev_fast(DeviceState *d)static uint8_t virtio_set_ind_atomic (SubchDev *sch,uint64_t ind_loc,uint8_t to_be_set) { -uint8_t ind_old, ind_new; +uint8_t expected, actual;hwaddr len =1 ; -uint8_t *ind_addr; +/* avoid multiple fetches */ +uint8_t volatile *ind_addr; ind_addr =cpu_physical_memory_map (ind_loc, &len, true);if (!ind_addr) {hw/s390x/virtio-ccw.c static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, · line 800
Quoted 6 lines
hw/s390x/virtio-ccw.c +1
-1@@ -796,14 +797,15 @@ static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, __func__, sch->cssid, sch->ssid, sch->schid);return -1 ; } + actual = *ind_addr; do { - ind_old = *ind_addr;to make things easier to understand. Adding a barrier in here also fixes the issue. Reasoning follows below:
hw/s390x/virtio-ccw.c static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, · line 801
Quoted 1 line
hw/s390x/virtio-ccw.c +0
-1- ind_new = ind_old | to_be_set;with an analysis from Andreas (cc)
#define
atomic_cmpxchg__nocheck(ptr, old, new)({ \typeof_strip_qual(*ptr) _old = (old); \ (void)__atomic_compare_exchange_n(ptr, &_old, new, false, \ __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST); \ _old; \
Quoted 2 lines
})
There is also the
#define atomic_cmpxchg(ptr, old, new) __sync_val_compare_and_swap(ptr, old, new)
variant, I guess, when the C11 stuff is not available. I don't know if that variant is guaranteed to not have problems with multiple loads.
Quoted 8 lines
ind_oldis copied into_oldin the macro. Instead of doing the copy from the register the compiler reloads the value from memory. The result is that_oldandind_oldend up having different values._oldin r1 with the bits set already andind_oldin r10 with the bits cleared._oldgets updated by CS and matchesind_oldafterwards - both with the bits being 0. So the != compare is false and the loop is left without having set any bits.
Quoted 4 lines
Paolo (to),
I am asking myself if it would be safer to add a barrier or something like this in the macros ininclude/qemu/atomic.h.
I think accessing the initial value via a volatile pointer initially and using the value loaded by cmpxchg for subsequent iterations is cleaner.
Regards,
Halil
Quoted 19 lines
Quoted 16 lines
hw/s390x/virtio-ccw.c +6
-4- }while (atomic_cmpxchg (ind_addr, ind_old, ind_new) != ind_old); -trace_virtio_ccw_set_ind (ind_loc, ind_old, ind_new); -cpu_physical_memory_unmap (ind_addr, len,1 , len); + expected = actual; + actual =atomic_cmpxchg (ind_addr, expected, expected | to_be_set); + }while (actual != expected); +trace_virtio_ccw_set_ind (ind_loc, actual, actual | to_be_set); +cpu_physical_memory_unmap ((void *)ind_addr, len,1 , len); -return ind_old; +return actual; }static void virtio_ccw_notify (DeviceState *d,uint16_t vector)
Re: [PATCH 1/2] virtio-ccw: fix virtio_set_ind_atomic
In reply to Halil PasicQuoted 8 lines · Halil Pasic
The
atomic_cmpxchg()loop is broken because we occasionally end up with old and_oldhaving different values (a legit compiler can generate code that accessed*ind_addragain to pick up a value for_oldinstead of using the value of old that was already fetched according to the rules of the abstract machine). This means the underlying CS instruction may use a different old (_old) than the one we intended to use ifatomic_cmpxchg()performed the xchg part.
Quoted 8 lines
Let us use volatile to force the rules of the abstract machine for accesses to
*ind_addr. Let us also rewrite the loop so, we that the new old is used to compute the new desired value if the xchg part is not performed.Signed-off-by Halil Pasic <[email protected]>Reported-by Andre Wild <[email protected]>Fixes 7e7494627f ("s390x/virtio-ccw: Adapter interrupt support.")
Quoted 44 lines
--- hw/s390x/virtio-ccw.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)hw/s390x/virtio-ccw.c +10
-8@@ -786,9 +786,10 @@ static inline VirtioCcwDevice *to_virtio_ccw_dev_fast(DeviceState *d)static uint8_t virtio_set_ind_atomic (SubchDev *sch,uint64_t ind_loc,uint8_t to_be_set) { -uint8_t ind_old, ind_new; +uint8_t expected, actual;hwaddr len =1 ; -uint8_t *ind_addr; +/* avoid multiple fetches */ +uint8_t volatile *ind_addr; ind_addr =cpu_physical_memory_map (ind_loc, &len, true);if (!ind_addr) {@@ -796,14 +797,15 @@ static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, __func__, sch->cssid, sch->ssid, sch->schid);return -1 ; } + actual = *ind_addr;do { - ind_old = *ind_addr; - ind_new = ind_old | to_be_set; - }while (atomic_cmpxchg (ind_addr, ind_old, ind_new) != ind_old); -trace_virtio_ccw_set_ind (ind_loc, ind_old, ind_new); -cpu_physical_memory_unmap (ind_addr, len,1 , len); + expected = actual; + actual =atomic_cmpxchg (ind_addr, expected, expected | to_be_set); + }while (actual != expected); +trace_virtio_ccw_set_ind (ind_loc, actual, actual | to_be_set); +cpu_physical_memory_unmap ((void *)ind_addr, len,1 , len); -return ind_old; +return actual; }static void virtio_ccw_notify (DeviceState *d,uint16_t vector)
Re: [PATCH 1/2] virtio-ccw: fix virtio_set_ind_atomic
In reply to Halil PasicQuoted 7 lines · Halil Pasic
The
atomic_cmpxchg()loop is broken because we occasionally end up with old and_oldhaving different values (a legit compiler can generate code that accessed*ind_addragain to pick up a value for_oldinstead of using the value of old that was already fetched according to the rules of the abstract machine). This means the underlying CS instruction may use a different old (_old) than the one we intended to use ifatomic_cmpxchg()performed the xchg part.
And was this ever observed in the field? Or is this a theoretical issue? commit log should probably say ...
Quoted 3 lines
Let us use volatile to force the rules of the abstract machine for accesses to
*ind_addr. Let us also rewrite the loop so, we that the
we that -> we know that?
Quoted 37 lines
new old is used to compute the new desired value if the xchg part is not performed.
Signed-off-by Halil Pasic <[email protected]>Reported-by Andre Wild <[email protected]>Fixes 7e7494627f ("s390x/virtio-ccw: Adapter interrupt support.")--- hw/s390x/virtio-ccw.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)hw/s390x/virtio-ccw.c +4
-6@@ -786,9 +786,10 @@ static inline VirtioCcwDevice *to_virtio_ccw_dev_fast(DeviceState *d)static uint8_t virtio_set_ind_atomic (SubchDev *sch,uint64_t ind_loc,uint8_t to_be_set) { -uint8_t ind_old, ind_new; +uint8_t expected, actual;hwaddr len =1 ; -uint8_t *ind_addr; +/* avoid multiple fetches */ +uint8_t volatile *ind_addr; ind_addr =cpu_physical_memory_map (ind_loc, &len, true);if (!ind_addr) {@@ -796,14 +797,15 @@ static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, __func__, sch->cssid, sch->ssid, sch->schid);return -1 ; } + actual = *ind_addr; do { - ind_old = *ind_addr; - ind_new = ind_old | to_be_set; - }while (atomic_cmpxchg (ind_addr, ind_old, ind_new) != ind_old); -trace_virtio_ccw_set_ind (ind_loc, ind_old, ind_new);
hw/s390x/virtio-ccw.c static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, · line 809
Quoted 10 lines
hw/s390x/virtio-ccw.c +6
-2-cpu_physical_memory_unmap (ind_addr, len,1 , len); + expected = actual; + actual =atomic_cmpxchg (ind_addr, expected, expected | to_be_set); + }while (actual != expected); +trace_virtio_ccw_set_ind (ind_loc, actual, actual | to_be_set); +cpu_physical_memory_unmap ((void *)ind_addr, len,1 , len); -return ind_old; +return actual; }
I wonder whether cpuXX APIs should accept volatile pointers, too:
casting away volatile is always suspicious.
But that is a separate issue ...
Quoted 3 lines
hw/s390x/virtio-ccw.c +0
-0static void virtio_ccw_notify (DeviceState *d,uint16_t vector)Signature · 1 line
2.17.1
Re: [PATCH 1/2] virtio-ccw: fix virtio_set_ind_atomic
Quoted 11 lines · Michael S. Tsirkin
Quoted 7 lines · Halil Pasic
The
atomic_cmpxchg()loop is broken because we occasionally end up with old and_oldhaving different values (a legit compiler can generate code that accessed*ind_addragain to pick up a value for_oldinstead of using the value of old that was already fetched according to the rules of the abstract machine). This means the underlying CS instruction may use a different old (_old) than the one we intended to use ifatomic_cmpxchg()performed the xchg part.And was this ever observed in the field? Or is this a theoretical issue? commit log should probably say ...
It was observed in the field when the xml specified qemu instead of vhost.
Re: [PATCH 1/2] virtio-ccw: fix virtio_set_ind_atomic
In reply to Michael S. TsirkinQuoted 9 lines · On Sat, 4 Jul 2020 14:34:04 -0400 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 7 lines · Halil Pasic
The
atomic_cmpxchg()loop is broken because we occasionally end up with old and_oldhaving different values (a legit compiler can generate code that accessed*ind_addragain to pick up a value for_oldinstead of using the value of old that was already fetched according to the rules of the abstract machine). This means the underlying CS instruction may use a different old (_old) than the one we intended to use ifatomic_cmpxchg()performed the xchg part.
Quoted 3 lines
And was this ever observed in the field? Or is this a theoretical issue? commit log should probably say ...
It was observed in the field (Christian already answered). I think the message already implies this, because the only conjunctive is about the compiler behavior.
Quoted 5 lines
Quoted 3 lines
Let us use volatile to force the rules of the abstract machine for accesses to
*ind_addr. Let us also rewrite the loop so, we that thewe that -> we know that?
s/we//
It would be nice to fix this before the patch gets merged.
Quoted 45 lines
Quoted 45 lines
new old is used to compute the new desired value if the xchg part is not performed.
Signed-off-by Halil Pasic <[email protected]>Reported-by Andre Wild <[email protected]>Fixes 7e7494627f ("s390x/virtio-ccw: Adapter interrupt support.")--- hw/s390x/virtio-ccw.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)hw/s390x/virtio-ccw.c +9
-7@@ -786,9 +786,10 @@ static inline VirtioCcwDevice *to_virtio_ccw_dev_fast(DeviceState *d)static uint8_t virtio_set_ind_atomic (SubchDev *sch,uint64_t ind_loc,uint8_t to_be_set) { -uint8_t ind_old, ind_new; +uint8_t expected, actual;hwaddr len =1 ; -uint8_t *ind_addr; +/* avoid multiple fetches */ +uint8_t volatile *ind_addr; ind_addr =cpu_physical_memory_map (ind_loc, &len, true);if (!ind_addr) {@@ -796,14 +797,15 @@ static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, __func__, sch->cssid, sch->ssid, sch->schid);return -1 ; } + actual = *ind_addr;do { - ind_old = *ind_addr; - ind_new = ind_old | to_be_set; - }while (atomic_cmpxchg (ind_addr, ind_old, ind_new) != ind_old); -trace_virtio_ccw_set_ind (ind_loc, ind_old, ind_new); -cpu_physical_memory_unmap (ind_addr, len,1 , len); + expected = actual; + actual =atomic_cmpxchg (ind_addr, expected, expected | to_be_set); + }while (actual != expected); +trace_virtio_ccw_set_ind (ind_loc, actual, actual | to_be_set); +cpu_physical_memory_unmap ((void *)ind_addr, len,1 , len);
Quoted 8 lines
hw/s390x/virtio-ccw.c static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc, · line 809
Quoted 3 lines
hw/s390x/virtio-ccw.c +1
-1-return ind_old; +return actual; }I wonder whether cpuXX APIs should accept volatile pointers, too:
casting away volatile is always suspicious.
But that is a separate issue ...
Nod.
Thanks for having a look!
Quoted 4 lines
Quoted 3 lines
hw/s390x/virtio-ccw.c +0
-0static void virtio_ccw_notify (DeviceState *d,uint16_t vector)Signature · 1 line
2.17.1
PATCH 2/2 s390x/pci: fix set_ind_atomic
In reply to Halil PasicThe atomic_cmpxchg() loop is broken because we occasionally end up with old and _old having different values (a legit compiler can generate code that accessed *ind_addr again to pick up a value for _old instead of using the value of old that was already fetched according to the rules of the abstract machine). This means the underlying CS instruction may use a different old (_old) than the one we intended to use if atomic_cmpxchg() performed the xchg part.
Let us use volatile to force the rules of the abstract machine for accesses to *ind_addr. Let us also rewrite the loop so, we that the new old is used to compute the new desired value if the xchg part is not performed.
--- hw/s390x/s390-pci-bus.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-)
hw/s390x/s390-pci-bus.c +9 -7
@@ -637,22 +637,24 @@ static AddressSpace *s390_pci_dma_iommu(PCIBus *bus, void *opaque, int devfn)static uint8_t set_ind_atomic (uint64_t ind_loc,uint8_t to_be_set) { -uint8_t ind_old, ind_new; +uint8_t expected, actual;hwaddr len =1 ; -uint8_t *ind_addr; +/* avoid multiple fetches */ +uint8_t volatile *ind_addr; ind_addr =cpu_physical_memory_map (ind_loc, &len, true);if (!ind_addr) {s390_pci_generate_error_event (ERR_EVENT_AIRERR ,0 ,0 ,0 ,0 );return -1 ; } + actual = *ind_addr;do { - ind_old = *ind_addr; - ind_new = ind_old | to_be_set; - }while (atomic_cmpxchg (ind_addr, ind_old, ind_new) != ind_old); -cpu_physical_memory_unmap (ind_addr, len,1 , len); + expected = actual; + actual =atomic_cmpxchg (ind_addr, expected, expected | to_be_set); + }while (actual != expected); +cpu_physical_memory_unmap ((void *)ind_addr, len,1 , len); -return ind_old; +return actual; } static void s390_msi_ctrl_write(void *opaque, hwaddr addr, uint64_t data,
Signature · 1 line
2.17.1
Re: [PATCH 2/2] s390x/pci: fix set_ind_atomic
Quoted 8 lines · Halil Pasic
The
atomic_cmpxchg()loop is broken because we occasionally end up with old and_oldhaving different values (a legit compiler can generate code that accessed*ind_addragain to pick up a value for_oldinstead of using the value of old that was already fetched according to the rules of the abstract machine). This means the underlying CS instruction may use a different old (_old) than the one we intended to use ifatomic_cmpxchg()performed the xchg part.
Quoted 8 lines
Let us use volatile to force the rules of the abstract machine for accesses to
*ind_addr. Let us also rewrite the loop so, we that the new old is used to compute the new desired value if the xchg part is not performed.Signed-off-by Halil Pasic <[email protected]>Fixes 8cba80c3a0 ("s390: Add PCI bus support")Reported-by Christian Borntraeger <[email protected]>
Quoted 41 lines
--- hw/s390x/s390-pci-bus.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-)hw/s390x/s390-pci-bus.c +9
-7@@ -637,22 +637,24 @@ static AddressSpace *s390_pci_dma_iommu(PCIBus *bus, void *opaque, int devfn)static uint8_t set_ind_atomic (uint64_t ind_loc,uint8_t to_be_set) { -uint8_t ind_old, ind_new; +uint8_t expected, actual;hwaddr len =1 ; -uint8_t *ind_addr; +/* avoid multiple fetches */ +uint8_t volatile *ind_addr; ind_addr =cpu_physical_memory_map (ind_loc, &len, true);if (!ind_addr) {s390_pci_generate_error_event (ERR_EVENT_AIRERR ,0 ,0 ,0 ,0 );return -1 ; } + actual = *ind_addr;do { - ind_old = *ind_addr; - ind_new = ind_old | to_be_set; - }while (atomic_cmpxchg (ind_addr, ind_old, ind_new) != ind_old); -cpu_physical_memory_unmap (ind_addr, len,1 , len); + expected = actual; + actual =atomic_cmpxchg (ind_addr, expected, expected | to_be_set); + }while (actual != expected); +cpu_physical_memory_unmap ((void *)ind_addr, len,1 , len); -return ind_old; +return actual; } static void s390_msi_ctrl_write(void *opaque, hwaddr addr, uint64_t data,
Quoted 8 lines · On Tue, 16 Jun 2020 06:50:33 +0200 Halil Pasic <[email protected]> wrote:
The story short: compiler can generate code that does two distinct fetches of
*ind_addrfor old and_old. If that happens we can not figure out if we had the desired xchg or not.Halil Pasic (2):
virtio-ccw: fixvirtio_set_ind_atomic
s390x/pci: fixset_ind_atomic
Quoted 6 lines
hw/s390x/s390-pci-bus.c | 16 +++++++++------- hw/s390x/virtio-ccw.c | 18 ++++++++++-------- 2 files changed, 19 insertions(+), 15 deletions(-)base-commit:
7d3660e79830a069f1848bb4fa1cdf8f666424fb
Have we managed to reach any kind of agreement on this? (A v2?)
We can still get in fixes post-softfreeze, of course.
Quoted 16 lines · Cornelia Huck
Quoted 13 lines · On Tue, 16 Jun 2020 06:50:33 +0200 Halil Pasic <[email protected]> wrote:
The story short: compiler can generate code that does two distinct fetches of
*ind_addrfor old and_old. If that happens we can not figure out if we had the desired xchg or not.Halil Pasic (2):
virtio-ccw: fixvirtio_set_ind_atomic
s390x/pci: fixset_ind_atomichw/s390x/s390-pci-bus.c | 16 +++++++++------- hw/s390x/virtio-ccw.c | 18 ++++++++++-------- 2 files changed, 19 insertions(+), 15 deletions(-)
Quoted 5 lines
Quoted 1 line
base-commit:
7d3660e79830a069f1848bb4fa1cdf8f666424fbHave we managed to reach any kind of agreement on this? (A v2?)
We can still get in fixes post-softfreeze, of course.
Unless Halil has a v2 ready,
I think the current patch is fine as is as a fix. I would suggest to go with that and we can then do more beautification later when necessary.
Quoted 19 lines · On Wed, 1 Jul 2020 14:06:11 +0200 Christian Borntraeger…
Quoted 18 lines · Cornelia Huck
Quoted 14 lines · On Tue, 16 Jun 2020 06:50:33 +0200 Halil Pasic <[email protected]> wrote:
The story short: compiler can generate code that does two distinct fetches of
*ind_addrfor old and_old. If that happens we can not figure out if we had the desired xchg or not.Halil Pasic (2):
virtio-ccw: fixvirtio_set_ind_atomic
s390x/pci: fixset_ind_atomichw/s390x/s390-pci-bus.c | 16 +++++++++------- hw/s390x/virtio-ccw.c | 18 ++++++++++-------- 2 files changed, 19 insertions(+), 15 deletions(-)base-commit:
7d3660e79830a069f1848bb4fa1cdf8f666424fb
Quoted 8 lines
Quoted 3 lines
Have we managed to reach any kind of agreement on this? (A v2?)
We can still get in fixes post-softfreeze, of course.
Unless Halil has a v2 ready,
I think the current patch is fine as is as a fix. I would suggest to go with that and we can then do more beautification later when necessary.
Sure, no objection to the patches as they are now.
I would like to see some R-bs/A-bs, though :)
Quoted 21 lines · On Wed, 1 Jul 2020 14:06:11 +0200 Christian Borntraeger…
Quoted 18 lines · Cornelia Huck
Quoted 14 lines · On Tue, 16 Jun 2020 06:50:33 +0200 Halil Pasic <[email protected]> wrote:
The story short: compiler can generate code that does two distinct fetches of
*ind_addrfor old and_old. If that happens we can not figure out if we had the desired xchg or not.Halil Pasic (2):
virtio-ccw: fixvirtio_set_ind_atomic
s390x/pci: fixset_ind_atomichw/s390x/s390-pci-bus.c | 16 +++++++++------- hw/s390x/virtio-ccw.c | 18 ++++++++++-------- 2 files changed, 19 insertions(+), 15 deletions(-)base-commit:
7d3660e79830a069f1848bb4fa1cdf8f666424fb
Quoted 10 lines
Quoted 3 lines
Have we managed to reach any kind of agreement on this? (A v2?)
We can still get in fixes post-softfreeze, of course.
Unless Halil has a v2 ready,
I think the current patch is fine as is as a fix. I would suggest to go with that and we can then do more beautification later when necessary.
I think we were waiting for Paolo to chime in regarding the barrier(). The thing I could beautify is using atomic_read(), but frankly I'm not sure about it, especially considering multi-proccess. My understanding of volatile is better than my understanding of atomic_read(). On the other hand, same multi-process question can be asked about atomic_cmpxchg() and __atomic_compare_exchange_n()...
Regards,
Halil
Quoted 35 lines · On Fri, 3 Jul 2020 15:37:11 +0200 Halil Pasic <[email protected]> wrote:
Quoted 21 lines · On Wed, 1 Jul 2020 14:06:11 +0200 Christian Borntraeger…
Quoted 18 lines · Cornelia Huck
Quoted 14 lines · On Tue, 16 Jun 2020 06:50:33 +0200 Halil Pasic <[email protected]> wrote:
The story short: compiler can generate code that does two distinct fetches of
*ind_addrfor old and_old. If that happens we can not figure out if we had the desired xchg or not.Halil Pasic (2):
virtio-ccw: fixvirtio_set_ind_atomic
s390x/pci: fixset_ind_atomichw/s390x/s390-pci-bus.c | 16 +++++++++------- hw/s390x/virtio-ccw.c | 18 ++++++++++-------- 2 files changed, 19 insertions(+), 15 deletions(-)base-commit:
7d3660e79830a069f1848bb4fa1cdf8f666424fbQuoted 10 lines
Quoted 3 lines
Have we managed to reach any kind of agreement on this? (A v2?)
We can still get in fixes post-softfreeze, of course.
Unless Halil has a v2 ready,
I think the current patch is fine as is as a fix. I would suggest to go with that and we can then do more beautification later when necessary.
Quoted 6 lines
I think we were waiting for Paolo to chime in regarding the
barrier(). The thing I could beautify is usingatomic_read(), but frankly I'm not sure about it, especially considering multi-proccess. My understanding of volatile is better than my understanding ofatomic_read(). On the other hand, same multi-process question can be asked aboutatomic_cmpxchg()and__atomic_compare_exchange_n()...
We can just improve the code on top later. Having something that works now beats something not yet developed that might be nicer :)
(But yes, it also might be good to look at non-s390 examples of this pattern and check if something can be done in a more central place.)
Cornelia Huck
AppliedQuoted 8 lines · On Tue, 16 Jun 2020 06:50:33 +0200 Halil Pasic <[email protected]> wrote:
The story short: compiler can generate code that does two distinct fetches of
*ind_addrfor old and_old. If that happens we can not figure out if we had the desired xchg or not.Halil Pasic (2):
virtio-ccw: fixvirtio_set_ind_atomic
s390x/pci: fixset_ind_atomic
Quoted 6 lines
hw/s390x/s390-pci-bus.c | 16 +++++++++------- hw/s390x/virtio-ccw.c | 18 ++++++++++-------- 2 files changed, 19 insertions(+), 15 deletions(-)base-commit:
7d3660e79830a069f1848bb4fa1cdf8f666424fb
Thanks, applied.