Skip to contentSkip to conversation

rust: why is it built in do_install()

DISCUSSION Enrico Scholz 4 participants 6 messages Sep 1, 2023 – Sep 2, 2023

lore
Share & tools
lore
#0 lore raw

Hello,

rust recipe does

| do_compile () {
| }
|  
| rust_do_install() {
|     rust_runx install
| }
| 
| rust_do_install:class-nativesdk() {
|     export PSEUDO_UNLOAD=1
|     rust_runx install
|     rust_runx install clippy
|     rust_runx install rustfmt

What is the reason to run the expensive "rust_runx" in do_install() and not in do_compile().

This:

- is unexpected... do_install is usually expected to run fast

- might interfere with different ${PARALLEL_MAKE} and ${PARALLEL_MAKEINST} setup

- might slow down the build process because the 'class-native' variant seems to run with active 'pseudo'

I guess it is done in this way because of

| config.set("install", "prefix", e(d.getVar("D") + d.getVar("prefix")))

where '${D}' is cleared by do_install.

Wouldn't it be better to replace this by

| config.set("install", "prefix", e(d.getVar("B") + '/_install' + d.getVar("prefix")))

| do_compile () {
|     rust_runx install
|     rust_runx install clippy
|     rust_runx install rustfmt
| }
|
| rust_do_install() {
|     cp -a ${B}/_install ${D}

?

[adding 'rustfmt' to 'class-native' would be nice because e.g. 'bindgen'
users might expect it]

Enrico

#1 lore raw

Hello Enrico,

Quoted 55 lines · Enrico Scholz via lists.openembedded.org

Hello,

rust recipe does

| do_compile () {
| }
|
| rust_do_install() {
|     rust_runx install
| }
|
| rust_do_install:class-nativesdk() {
|     export PSEUDO_UNLOAD=1
|     rust_runx install
|     rust_runx install clippy
|     rust_runx install rustfmt

What is the reason to run the expensive "rust_runx" in do_install() and not in do_compile().

This:

- is unexpected... do_install is usually expected to run fast

- might interfere with different ${PARALLEL_MAKE} and ${PARALLEL_MAKEINST} setup

- might slow down the build process because the 'class-native' variant seems to run with active 'pseudo'

I guess it is done in this way because of

| config.set("install", "prefix", e(d.getVar("D") +
d.getVar("prefix")))

where '${D}' is cleared by do_install.

Wouldn't it be better to replace this by

| config.set("install", "prefix", e(d.getVar("B") + '/_install' +
d.getVar("prefix")))

| do_compile () {
|     rust_runx install
|     rust_runx install clippy
|     rust_runx install rustfmt
| }
|
| rust_do_install() {
|     cp -a ${B}/_install ${D}
Quoted 7 lines

?

[adding 'rustfmt' to 'class-native' would be nice because e.g. 'bindgen'
users might expect it]

I don't know why this was done that way and I lacked xp for telling you if your suggestion is correct. But I just wanna tell that the do_install:rust-native is very long (more than 45mn on my machine) so you may have a point here.

Enrico

Quoted 12 lines

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#187012):
https://lists.openembedded.org/g/openembedded-core/message/187012
Mute This Topic: https://lists.openembedded.org/mt/101098826/6213388
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#2 lore raw
In reply to Enrico Scholz
Quoted 18 lines · Enrico Scholz via lists.openembedded.org

Hello,

rust recipe does

Quoted 7 lines

do_compile () {
}

rust_do_install() {
rust_runx install
}

Quoted 5 lines

rust_do_install:class-nativesdk() {
export PSEUDO_UNLOAD=1
rust_runx install
rust_runx install clippy
rust_runx install rustfmt

Quoted 2 lines

What is the reason to run the expensive "rust_runx" in do_install() and not in do_compile().

The reason was probably that was what was found to work in the original recipe and nobody has had the time/need to improve it.

Quoted 11 lines

This:

- is unexpected... do_install is usually expected to run fast

- might interfere with different ${PARALLEL_MAKE} and ${PARALLEL_MAKEINST} setup

- might slow down the build process because the 'class-native' variant seems to run with active 'pseudo'

Quoted 5 lines

I guess it is done in this way because of

Quoted 1 line
   config.set("install", "prefix",  e(d.getVar("D") + d.getVar("prefix")))

where '${D}' is cleared by do_install.

That would seem likely to be at least one reason.

Quoted 11 lines

Wouldn't it be better to replace this by

Quoted 9 lines
   config.set("install", "prefix",  e(d.getVar("B") + '/_install' + d.getVar("prefix")))

do_compile () {
rust_runx install
rust_runx install clippy
rust_runx install rustfmt
}

Quoted 8 lines
Quoted 2 lines
rust_do_install() {
    cp -a ${B}/_install ${D}

?

[adding 'rustfmt' to 'class-native' would be nice because e.g. 'bindgen'
users might expect it]

One potential downside is that "install" would then run outside pseudo context which may cause problems with permissions. I don't really know whether that would be an issue or not though in reality.

I'd be happy to see more people working on the rust recipes. The challenge is the long testing cycles needed. Fancy working on some patches?

Cheers,

Richard

#3 lore raw
Quoted 15 lines · Richard Purdie
Quoted 6 lines
Quoted 6 lines

do_compile () {
rust_runx install
rust_runx install clippy
rust_runx install rustfmt
}

Quoted 8 lines
Quoted 2 lines
rust_do_install() {
    cp -a ${B}/_install ${D}

?

[adding 'rustfmt' to 'class-native' would be nice because e.g. 'bindgen'
users might expect it]

Quoted 7 lines

One potential downside is that "install" would then run outside pseudo context which may cause problems with permissions. I don't really know whether that would be an issue or not though in reality.

I'd be happy to see more people working on the rust recipes. The challenge is the long testing cycles needed. Fancy working on some patches?

The original recipes in meta-rust and oe-core recipes had for a while separate compile/install steps. At some point this regressed on a version update - install on top of compile became broken and errored out, but pristine install (with compile inside of it) continued to work. This is what upstream tests and documents, so that's how rust recipe does it as well nowadays. It's not any slower than separate steps, actually it's somewhat faster, as install was rebuilding supplementary things unnecessarily after compile has already done that. The only problem I see is that it's indeed visually confusing.

If somene wants to separate the steps again, they can simply find the commit in oe-core history and undo that. Any changes would have to fulfil this:
1. Rust's installation procedure should run under pseudo, e.g. in do_install. Even if it's not a problem now, it might become a problem later.
2. The changes can't lengthen build times. Rust is already one of the slowest items in core.
3. This probably requires fixing things in rust upstream, w.r.t. supporting separation of build/install, and install using artefacts from the build step properly.

Alex

#4 lore raw
Quoted 4 lines · Alexander Kanavin

Any changes would have to fulfil this:

1. Rust's installation procedure should run under pseudo, e.g. in do_install.

The current recipe does *not* run installation under pseudo

| rust_do_install:class-target() {
|     export PSEUDO_UNLOAD=1
|     rust_runx install

Only class-native runs under pseudo (or is pseudo automatically disabled for this class?).

Enrico

#5 lore raw
Quoted 7 lines · Enrico Scholz
Quoted 4 lines · Alexander Kanavin

Any changes would have to fulfil this:

1. Rust's installation procedure should run under pseudo, e.g. in do_install.

Quoted 8 lines

The current recipe does *not* run installation under pseudo

Quoted 3 lines

rust_do_install:class-target() {
export PSEUDO_UNLOAD=1
rust_runx install

Only class-native runs under pseudo (or is pseudo automatically disabled for this class?).

pseudo is disabled for native.

Cheers,

Richard