Skip to contentSkip to conversation

rust: Disable 64bit atomics on crossbeam on riscv32

PATCH Superseded Khem Raj 2 participants 16 messages Oct 20, 2022 – Oct 26, 2022 1 message not in the archive

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Khem Raj

author
#0 lore raw

PATCH 1/1 rust: Disable 64bit atomics on crossbeam on riscv32

riscv32 builds fail since AtomicI64 is not available in sync::atomic for this architecture. Therefore it needs to be opted out and the way out is to pass it by RUSTFLAGS, however rust target recipe overrides RUSTFLAGS before exporting it so its better to piggy back on the variable its using which is RUST_DEBUG_REMAP

---
 meta/recipes-devtools/rust/rust_1.64.0.bb | 1 +
 1 file changed, 1 insertion(+)
meta/recipes-devtools/rust/rust_1.64.0.bb +1 -0
index 1f9dbd3cce..b05f188d9c 100644
@@ -9,6 +9,7 @@ FILES:${PN}-dev = ""
 
 # Used by crossbeam_atomic.patch
 export TARGET_VENDOR
+RUST_DEBUG_REMAP:append:riscv32 = " --cfg crossbeam_no_atomic_64"
 
 do_compile () {
     rust_runx build --stage 2
Signature · 1 line
2.38.1
#1 lore raw

We carry this patch, so that manual per-target fixups like this are not needed, and crossbeam's internal lists are used properly:

https://git.yoctoproject.org/poky/tree/meta/recipes-devtools/rust/rust/crossbeam_atomic.patch?h=master-next&id=f4fe7cdaa7fc5785c55bd507c897f4d25d43cffa

Is it not working somehow? Does the list itself need to be fixed?

Alex

Quoted 36 lines · Khem Raj

riscv32 builds fail since AtomicI64 is not available in sync::atomic for this architecture. Therefore it needs to be opted out and the way out is to pass it by RUSTFLAGS, however rust target recipe overrides RUSTFLAGS before exporting it so its better to piggy back on the variable its using which is RUST_DEBUG_REMAP

---
 meta/recipes-devtools/rust/rust_1.64.0.bb | 1 +
 1 file changed, 1 insertion(+)
meta/recipes-devtools/rust/rust_1.64.0.bb +1 -1
index 1f9dbd3cce..b05f188d9c 100644
@@ -9,6 +9,7 @@ FILES:${PN}-dev = ""
 
 # Used by crossbeam_atomic.patch
 export TARGET_VENDOR
+RUST_DEBUG_REMAP:append:riscv32 = " --cfg crossbeam_no_atomic_64"
 
 do_compile () {
     rust_runx build --stage 2
--

2.38.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#172006): https://lists.openembedded.org/g/openembedded-core/message/172006
Mute This Topic: https://lists.openembedded.org/mt/94464934/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

author
#2 lore raw
Quoted 8 lines · Alexander Kanavin

We carry this patch, so that manual per-target fixups like this are not needed, and crossbeam's internal lists are used properly:

https://git.yoctoproject.org/poky/tree/meta/recipes-devtools/rust/rust/crossbeam_atomic.patch?h=master-next&id=f4fe7cdaa7fc5785c55bd507c897f4d25d43cffa

Is it not working somehow? Does the list itself need to be fixed?

This patch is doing something else where its mapping and broadening the architecture to match crossbeam's understanding of it. Here issue is new where it does not yet realize that some architectures do not have 64bit atomics implemented.

Quoted 39 lines

Alex

Quoted 36 lines · Khem Raj

riscv32 builds fail since AtomicI64 is not available in sync::atomic for this architecture. Therefore it needs to be opted out and the way out is to pass it by RUSTFLAGS, however rust target recipe overrides RUSTFLAGS before exporting it so its better to piggy back on the variable its using which is RUST_DEBUG_REMAP

---
 meta/recipes-devtools/rust/rust_1.64.0.bb | 1 +
 1 file changed, 1 insertion(+)
meta/recipes-devtools/rust/rust_1.64.0.bb +1 -1
index 1f9dbd3cce..b05f188d9c 100644
@@ -9,6 +9,7 @@ FILES:${PN}-dev = ""
 
 # Used by crossbeam_atomic.patch
 export TARGET_VENDOR
+RUST_DEBUG_REMAP:append:riscv32 = " --cfg crossbeam_no_atomic_64"
 
 do_compile () {
     rust_runx build --stage 2
--

2.38.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#172006): https://lists.openembedded.org/g/openembedded-core/message/172006
Mute This Topic: https://lists.openembedded.org/mt/94464934/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#3 lore raw
Quoted 4 lines · Khem Raj

This patch is doing something else where its mapping and broadening the architecture to match crossbeam's understanding of it. Here issue is new where it does not yet realize that some architectures do not have 64bit atomics implemented.

Here's crossbeam's own list:

https://github.com/crossbeam-rs/crossbeam/blob/master/no_atomic.rs

You can see that riscv32 is in NO_ATOMIC_64, and crossbeam_atomic.patch ensures we use the list, so why do we need this change?

Alex

author
#4 lore raw
Quoted 7 lines · Alexander Kanavin
Quoted 4 lines · Khem Raj

This patch is doing something else where its mapping and broadening the architecture to match crossbeam's understanding of it. Here issue is new where it does not yet realize that some architectures do not have 64bit atomics implemented.

Quoted 7 lines

Here's crossbeam's own list:

https://github.com/crossbeam-rs/crossbeam/blob/master/no_atomic.rs

You can see that riscv32 is in NO_ATOMIC_64, and crossbeam_atomic.patch ensures we use the list, so why do we need this change?

Looking briefly over the code, my educated guess is that target names are still not matching
rusts tuple uses riscv32gc but we use riscv32

Quoted 2 lines

Alex

#5 lore raw
Quoted 3 lines · Khem Raj

Looking briefly over the code, my educated guess is that target names are still not matching
rusts tuple uses riscv32gc but we use riscv32

I believe we have a remapping in meta/lib/oe/rust.py, can you check if that works?

Alex

author
#6 lore raw
Quoted 8 lines · Alexander Kanavin
Quoted 3 lines · Khem Raj

Looking briefly over the code, my educated guess is that target names are still not matching
rusts tuple uses riscv32gc but we use riscv32

I believe we have a remapping in meta/lib/oe/rust.py, can you check if that works?

This is mapping host uname, it wont work.

Quoted 2 lines

Alex

#7 lore raw
Quoted 9 lines · Khem Raj
Quoted 7 lines · Khem Raj
Quoted 3 lines

Looking briefly over the code, my educated guess is that target names are still not matching
rusts tuple uses riscv32gc but we use riscv32

I believe we have a remapping in meta/lib/oe/rust.py, can you check if that works?

This is mapping host uname, it wont work.

No, it's mapping the targets too. E.g. in rust-common.bbclass:

# Responsible for taking Yocto triples and converting it to Rust triples def rust_base_triple(d, thing):
'''
Mangle bitbake's *_SYS into something that rust might support (see rust/mk/cfg/* for a list)

Note that os is assumed to be some linux form
'''

# The llvm-target for armv7 is armv7-unknown-linux-gnueabihf if d.getVar('{}_ARCH'.format(thing)) == d.getVar('TARGET_ARCH')
and target_is_armv7(d):

        arch = "armv7"
    else:
        arch = oe.rust.arch_to_rust_arch(d.getVar('{}_ARCH'.format(thing)))

Alex

author
#8 lore raw
Quoted 23 lines · Alexander Kanavin
Quoted 9 lines · Khem Raj
Quoted 7 lines · Khem Raj
Quoted 3 lines

Looking briefly over the code, my educated guess is that target names are still not matching
rusts tuple uses riscv32gc but we use riscv32

I believe we have a remapping in meta/lib/oe/rust.py, can you check if that works?

This is mapping host uname, it wont work.

No, it's mapping the targets too. E.g. in rust-common.bbclass:

# Responsible for taking Yocto triples and converting it to Rust triples def rust_base_triple(d, thing):
'''
Mangle bitbake's *_SYS into something that rust might support (see rust/mk/cfg/* for a list)

Note that os is assumed to be some linux form
'''

Quoted 7 lines

# The llvm-target for armv7 is armv7-unknown-linux-gnueabihf if d.getVar('{}_ARCH'.format(thing)) == d.getVar('TARGET_ARCH')
and target_is_armv7(d):

        arch = "armv7"
    else:
        arch = oe.rust.arch_to_rust_arch(d.getVar('{}_ARCH'.format(thing)))

anyway, the builds failed at exactly the same point so as I sensed it does not work.

Quoted 1 line

Alex

#9 lore raw
Quoted 2 lines · Khem Raj

anyway, the builds failed at exactly the same point so as I sensed it does not work.

You need to find out why though. Upstream maintains a list, and we need to use it, and not duplicate it piecemeal.

Alex

author
#10 lore raw
Quoted 8 lines · Alexander Kanavin
Quoted 2 lines · Khem Raj

anyway, the builds failed at exactly the same point so as I sensed it does not work.

You need to find out why though. Upstream maintains a list, and we need to use it, and not duplicate it piecemeal.

It needs to be mapped or added. I have sent a patch to add OE's understanding of target arch.

Quoted 2 lines

Alex

#11 lore raw
In reply to Khem Raj

Ok, I'd like to understand where the failure is exactly. I just ran 'bitbake rust' with poky master (rust 1.63) for qemuriscv32 machine, and it succeeded. How can I see the fail? Did this become broken with a 1.64.0 upgrade in your branch? (which I believe hasn't yet been through the autobuilder, and so might show similar fails for other targets without atomic_64 support)

Alex

Quoted 36 lines · Khem Raj

riscv32 builds fail since AtomicI64 is not available in sync::atomic for this architecture. Therefore it needs to be opted out and the way out is to pass it by RUSTFLAGS, however rust target recipe overrides RUSTFLAGS before exporting it so its better to piggy back on the variable its using which is RUST_DEBUG_REMAP

---
 meta/recipes-devtools/rust/rust_1.64.0.bb | 1 +
 1 file changed, 1 insertion(+)
meta/recipes-devtools/rust/rust_1.64.0.bb +1 -1
index 1f9dbd3cce..b05f188d9c 100644
@@ -9,6 +9,7 @@ FILES:${PN}-dev = ""
 
 # Used by crossbeam_atomic.patch
 export TARGET_VENDOR
+RUST_DEBUG_REMAP:append:riscv32 = " --cfg crossbeam_no_atomic_64"
 
 do_compile () {
     rust_runx build --stage 2
--

2.38.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#172006): https://lists.openembedded.org/g/openembedded-core/message/172006
Mute This Topic: https://lists.openembedded.org/mt/94464934/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

A message that is not in the archive (sent off-list or not yet received): [email protected]

#13 lore raw

Ok, I have looked into this properly now. So:

- the fail started to occur in 1.64 because crossbeam-utils is now a dependency of rust core (hard fail) and not just rustfmt (soft fail)
- the correct way to address it is indeed to map oe arch (riscv32) to rust arch (riscv32gc) via meta/lib/oe/rust.py, which works globally across all recipes where this may occur.
- however doing that exposed confusion about how we map arches, as there is actually a *third* arch involved (the internal llvm one listed in custom target definition), which too needs to be correctly mapped from oe arch so I fixed that as well.

None of the three arch sets match each other exactly, unfortunately.

Patches are coming shortly.

Alex

Quoted 50 lines · Alexander Kanavin via lists.openembedded.org

Ok, I'd like to understand where the failure is exactly. I just ran 'bitbake rust' with poky master (rust 1.63) for qemuriscv32 machine, and it succeeded. How can I see the fail? Did this become broken with a 1.64.0 upgrade in your branch? (which I believe hasn't yet been through the autobuilder, and so might show similar fails for other targets without atomic_64 support)

Alex

Quoted 25 lines · Khem Raj

riscv32 builds fail since AtomicI64 is not available in sync::atomic for this architecture. Therefore it needs to be opted out and the way out is to pass it by RUSTFLAGS, however rust target recipe overrides RUSTFLAGS before exporting it so its better to piggy back on the variable its using which is RUST_DEBUG_REMAP

---
 meta/recipes-devtools/rust/rust_1.64.0.bb | 1 +
 1 file changed, 1 insertion(+)
meta/recipes-devtools/rust/rust_1.64.0.bb +1 -1
index 1f9dbd3cce..b05f188d9c 100644
@@ -9,6 +9,7 @@ FILES:${PN}-dev = ""
 
 # Used by crossbeam_atomic.patch
 export TARGET_VENDOR
+RUST_DEBUG_REMAP:append:riscv32 = " --cfg crossbeam_no_atomic_64"
 
 do_compile () {
     rust_runx build --stage 2
--
Quoted 5 lines

2.38.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#172064): https://lists.openembedded.org/g/openembedded-core/message/172064
Mute This Topic: https://lists.openembedded.org/mt/94464934/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

author
#14 lore raw
Quoted 13 lines · Alexander Kanavin

Ok, I have looked into this properly now. So:

- the fail started to occur in 1.64 because crossbeam-utils is now a dependency of rust core (hard fail) and not just rustfmt (soft fail)
- the correct way to address it is indeed to map oe arch (riscv32) to rust arch (riscv32gc) via meta/lib/oe/rust.py, which works globally across all recipes where this may occur.
- however doing that exposed confusion about how we map arches, as there is actually a *third* arch involved (the internal llvm one listed in custom target definition), which too needs to be correctly mapped from oe arch so I fixed that as well.

Quoted 1 line

None of the three arch sets match each other exactly, unfortunately.

Right.

Quoted 3 lines

Patches are coming shortly.

Thanks, I think we can also unbolt the workaround in librsvg as well.

(quoted diff) +0 -0

b/meta/recipes-gnome/librsvg/librsvg_2.54.5.bb
index fc52ae61c5..daf60217d4 100644

meta/recipes-gnome/librsvg/librsvg_2.54.5.bb +0 -1
@@ -35,7 +35,6 @@ export RUST_TARGET = "${RUST_HOST_SYS}"
 RUSTFLAGS:append:mips = " --cfg crossbeam_no_atomic_64"
 RUSTFLAGS:append:mipsel = " --cfg crossbeam_no_atomic_64"
 RUSTFLAGS:append:powerpc = " --cfg crossbeam_no_atomic_64"
-RUSTFLAGS:append:riscv32 = " --cfg crossbeam_no_atomic_64"
 
 CARGO_DISABLE_BITBAKE_VENDORING = "1"
 do_configure[postfuncs] += "cargo_common_do_configure"
Quoted 55 lines

Alex

Quoted 50 lines · Alexander Kanavin via lists.openembedded.org

Ok, I'd like to understand where the failure is exactly. I just ran 'bitbake rust' with poky master (rust 1.63) for qemuriscv32 machine, and it succeeded. How can I see the fail? Did this become broken with a 1.64.0 upgrade in your branch? (which I believe hasn't yet been through the autobuilder, and so might show similar fails for other targets without atomic_64 support)

Alex

Quoted 25 lines · Khem Raj

riscv32 builds fail since AtomicI64 is not available in sync::atomic for this architecture. Therefore it needs to be opted out and the way out is to pass it by RUSTFLAGS, however rust target recipe overrides RUSTFLAGS before exporting it so its better to piggy back on the variable its using which is RUST_DEBUG_REMAP

---
 meta/recipes-devtools/rust/rust_1.64.0.bb | 1 +
 1 file changed, 1 insertion(+)
meta/recipes-devtools/rust/rust_1.64.0.bb +1 -1
index 1f9dbd3cce..b05f188d9c 100644
@@ -9,6 +9,7 @@ FILES:${PN}-dev = ""
 
 # Used by crossbeam_atomic.patch
 export TARGET_VENDOR
+RUST_DEBUG_REMAP:append:riscv32 = " --cfg crossbeam_no_atomic_64"
 
 do_compile () {
     rust_runx build --stage 2
--
Quoted 5 lines

2.38.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#172064): https://lists.openembedded.org/g/openembedded-core/message/172064
Mute This Topic: https://lists.openembedded.org/mt/94464934/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#15 lore raw

meta/recipes-gnome/librsvg/librsvg_2.54.5.bb export RUST_TARGET = "${RUST_HOST_SYS}" · line 38

Quoted 12 lines · Khem Raj

Thanks, I think we can also unbolt the workaround in librsvg as well.

meta/recipes-gnome/librsvg/librsvg_2.54.5.bb +0 -1
index fc52ae61c5..daf60217d4 100644
@@ -35,7 +35,6 @@ export RUST_TARGET = "${RUST_HOST_SYS}"
 RUSTFLAGS:append:mips = " --cfg crossbeam_no_atomic_64"
 RUSTFLAGS:append:mipsel = " --cfg crossbeam_no_atomic_64"
 RUSTFLAGS:append:powerpc = " --cfg crossbeam_no_atomic_64"
-RUSTFLAGS:append:riscv32 = " --cfg crossbeam_no_atomic_64"

Not yet. librsvg carries its own copy of crossbeam which is not patched to use the provided standard lists.

So we should first merge some variant of crossbeam_atomic.patch upstream. My previous attempt went nowhere:
https://github.com/crossbeam-rs/crossbeam/pull/751

Alex

author
#16 lore raw
Quoted 4 lines · Alexander Kanavin
Quoted 2 lines · Khem Raj

Thanks, I think we can also unbolt the workaround in librsvg as well.

Quoted 13 lines

meta/recipes-gnome/librsvg/librsvg_2.54.5.bb export RUST_TARGET = "${RUST_HOST_SYS}" · line 38

Quoted 10 lines
meta/recipes-gnome/librsvg/librsvg_2.54.5.bb +0 -1
index fc52ae61c5..daf60217d4 100644
@@ -35,7 +35,6 @@ export RUST_TARGET = "${RUST_HOST_SYS}"
 RUSTFLAGS:append:mips = " --cfg crossbeam_no_atomic_64"
 RUSTFLAGS:append:mipsel = " --cfg crossbeam_no_atomic_64"
 RUSTFLAGS:append:powerpc = " --cfg crossbeam_no_atomic_64"
-RUSTFLAGS:append:riscv32 = " --cfg crossbeam_no_atomic_64"

Not yet. librsvg carries its own copy of crossbeam which is not patched to use the provided standard lists.

right saw that

Quoted 6 lines

So we should first merge some variant of crossbeam_atomic.patch upstream. My previous attempt went nowhere:
https://github.com/crossbeam-rs/crossbeam/pull/751

Alex