Reproducible build
Forget my previous question,
I simply edit
https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/selftest/cases/reproducible.py#n137
to replace by zavariant, and with OEQA_DEBUGGING_SAVED_OUTPUT set , I managed to see the html diff
Nevertheless, It took one hour, but it seems that it cannot be avoided since there are two builds and one
without any shared state.
Quoted 26 lines · Frederic Martinsons via lists.openembedded.org
Hello,
I plan to work on https://bugzilla.yoctoproject.org/show_bug.cgi?id=15090
in the coming days.
To show how things are tested I ran "oe-selftest -r reproducible.ReproducibleTests.test_reproducible_builds" but even on a strong machine, it took several hours (I waited for 6h and it is not finished yet).Does anyone know if it is possible to test reproducibility on a single package (here, zvariant) ?
Should I tweak the python test class or is there a simple way ?Thanks
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#179826):
https://lists.openembedded.org/g/openembedded-core/message/179826
Mute This Topic: https://lists.openembedded.org/mt/98126404/6213388
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 11 lines · Frederic Martinsons
Forget my previous question,
I simply
edit https://git.openembedded.org/openembedded-core/tree/meta/lib/oeq
a/selftest/cases/reproducible.py#n137
to replace by zavariant, and with OEQA_DEBUGGING_SAVED_OUTPUT set , I managed to see the html diffNevertheless, It took one hour, but it seems that it cannot be avoided since there are two builds and one
without any shared state.
Unfortunately that is the nature of the test, one side has to be built from scratch. Tweaking the test as you'd done to target a specific component is what I'd have done too to speed things up for a specific recipe.
To get a further speedup, you could probably keep the sstate for each of the two builds separate but retain both, then in theory on the pieces you change would rebuild. Again, it would mean tweaking the test but for debugging, I think that is fine. Rust can be a pain if you just rebuild those as it still takes a while. Thanks for taking a look!
Cheers,
Richard
Quoted 24 lines · Richard Purdie
Quoted 11 lines · Frederic Martinsons
Forget my previous question,
I simply
edit https://git.openembedded.org/openembedded-core/tree/meta/lib/oeq
a/selftest/cases/reproducible.py#n137
to replace by zavariant, and withOEQA_DEBUGGING_SAVED_OUTPUTset , I managed to see the html diffNevertheless, It took one hour, but it seems that it cannot be avoided since there are two builds and one
without any shared state.Unfortunately that is the nature of the test, one side has to be built from scratch. Tweaking the test as you'd done to target a specific component is what I'd have done too to speed things up for a specific recipe.
To get a further speedup, you could probably keep the sstate for each of the two builds separate but retain both, then in theory on the pieces you change would rebuild. Again, it would mean tweaking the test but for debugging, I think that is fine. Rust can be a pain if you just rebuild those as it still takes a while. Thanks for taking a look!
Quoted 5 lines
Cheers,
Richard
Thank you for the tip, I'll try that, everything that help to speed up is good to take of course.
Hi Frederic,
Frederic Martinsons <[email protected]> escreveu no dia domingo, 9/04/2023 à(s) 16:05:
Quoted 29 lines
Quoted 24 lines · Richard Purdie
Quoted 11 lines · Frederic Martinsons
Forget my previous question,
I simply
edit https://git.openembedded.org/openembedded-core/tree/meta/lib/oeq
a/selftest/cases/reproducible.py#n137
to replace by zavariant, and withOEQA_DEBUGGING_SAVED_OUTPUTset , I managed to see the html diffNevertheless, It took one hour, but it seems that it cannot be avoided since there are two builds and one
without any shared state.Unfortunately that is the nature of the test, one side has to be built from scratch. Tweaking the test as you'd done to target a specific component is what I'd have done too to speed things up for a specific recipe.
To get a further speedup, you could probably keep the sstate for each of the two builds separate but retain both, then in theory on the pieces you change would rebuild. Again, it would mean tweaking the test but for debugging, I think that is fine. Rust can be a pain if you just rebuild those as it still takes a while. Thanks for taking a look!
Quoted 8 lines
Quoted 5 lines
Cheers,
Richard
Thank you for the tip, I'll try that, everything that help to speed up is good to take of course.
To speedup the build you can take the rust-native from the state cache [1] and this will improve a bit bit the time needed:
sstate_target = [ "rust-native" ]
Jose
Quoted 12 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#179841):
https://lists.openembedded.org/g/openembedded-core/message/179841
Mute This Topic: https://lists.openembedded.org/mt/98126404/5052612
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
Best regards, José Quaresma
Quoted 67 lines · Jose Quaresma
Hi Frederic,
Frederic Martinsons <[email protected]> escreveu no dia domingo, 9/04/2023 à(s) 16:05:
Quoted 29 lines
Quoted 24 lines · Richard Purdie
Quoted 11 lines · Frederic Martinsons
Forget my previous question,
I simply
edit https://git.openembedded.org/openembedded-core/tree/meta/lib/oeq
a/selftest/cases/reproducible.py#n137
to replace by zavariant, and withOEQA_DEBUGGING_SAVED_OUTPUTset , I managed to see the html diffNevertheless, It took one hour, but it seems that it cannot be avoided since there are two builds and one
without any shared state.Unfortunately that is the nature of the test, one side has to be built from scratch. Tweaking the test as you'd done to target a specific component is what I'd have done too to speed things up for a specific recipe.
To get a further speedup, you could probably keep the sstate for each of the two builds separate but retain both, then in theory on the pieces you change would rebuild. Again, it would mean tweaking the test but for debugging, I think that is fine. Rust can be a pain if you just rebuild those as it still takes a while. Thanks for taking a look!
Quoted 8 lines
Quoted 5 lines
Cheers,
Richard
Thank you for the tip, I'll try that, everything that help to speed up is good to take of course.
To speedup the build you can take the rust-native from the state cache [1] and this will improve a bit bit the time needed:
sstate_target= [ "rust-native" ]Jose
Quoted 12 lines
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#179841):
https://lists.openembedded.org/g/openembedded-core/message/179841
Mute This Topic: https://lists.openembedded.org/mt/98126404/5052612
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 5 lines
--
Best regards,José Quaresma
Hey, thank you José for the advice, I'll apply that for sure.
By the way, like I commented in
https://bugzilla.yoctoproject.org/show_bug.cgi?id=15090 , I'm currently try to reproduce (ahah) the reproducibility issue of zvariant with the use of reprotest (https://salsa.debian.org/reproducible-builds/reprotest)
and outside yocto environment. For the moment, I was not able to pinpoint the issue (I must have
put aside the faketime usage since it caused me segfault of rustc).
I plan to dig further in reproducible.py class and how the package is built under yocto to try to find out
what is going on.
Hello with the patch below:
meta/lib/oeqa/selftest/cases/reproducible.py +0 -1
@@ -134,9 +134,9 @@ class ReproducibleTests(OESelftestTestCase): max_report_size = 250 * 1024 * 1024 # targets are the things we want to test the reproducibility of - targets = ['core-image-minimal', 'core-image-sato',
'core-image-full-cmdline', 'core-image-weston', 'world']
+ targets = ['zvariant']
# sstate targets are things to pull from sstate to potentially cut
build/debugging time
- sstate_targets = []
+ sstate_targets = ['rust-native', 'cargo-native']
save_results = False
if 'OEQA_DEBUGGING_SAVED_OUTPUT' in os.environ:
save_results = os.environ['OEQA_DEBUGGING_SAVED_OUTPUT']I managed to reduce the time to test to 30mn (which is reasonable). Today, I made 5 run of this
reproducibility test and if library is indeed different between build A and build B every time, this
is the same exact difference, that is to day that the 5 libraries in build A are identical and so are
the 5 libraries in build B.
I admit I'm surprised , I would have waited for some random diff. I'll continue my investigation but if this rang a bell to someone here, feel free to tell (this does
seem strange to me).
Quoted 100 lines · Frederic Martinsons via lists.openembedded.org
Quoted 59 lines · Jose Quaresma
Hi Frederic,
Frederic Martinsons <[email protected]> escreveu no dia domingo, 9/04/2023 à(s) 16:05:
Quoted 29 lines
Quoted 24 lines · Richard Purdie
Quoted 11 lines · Frederic Martinsons
Forget my previous question,
I simply
edit https://git.openembedded.org/openembedded-core/tree/meta/lib/oeq
a/selftest/cases/reproducible.py#n137
to replace by zavariant, and withOEQA_DEBUGGING_SAVED_OUTPUTset , I managed to see the html diffNevertheless, It took one hour, but it seems that it cannot be avoided since there are two builds and one
without any shared state.Unfortunately that is the nature of the test, one side has to be built from scratch. Tweaking the test as you'd done to target a specific component is what I'd have done too to speed things up for a specific recipe.
To get a further speedup, you could probably keep the sstate for each of the two builds separate but retain both, then in theory on the pieces you change would rebuild. Again, it would mean tweaking the test but for debugging, I think that is fine. Rust can be a pain if you just rebuild those as it still takes a while. Thanks for taking a look!
Quoted 8 lines
Quoted 5 lines
Cheers,
Richard
Thank you for the tip, I'll try that, everything that help to speed up is good to take of course.
To speedup the build you can take the rust-native from the state cache
[1] and this will improve a bit bit the time needed:
sstate_target= [ "rust-native" ]Jose
Quoted 4 lines
Quoted 5 lines
--
Best regards,José Quaresma
Hey, thank you José for the advice, I'll apply that for sure.
By the way, like I commented in
https://bugzilla.yoctoproject.org/show_bug.cgi?id=15090 , I'm currently try to reproduce (ahah) the reproducibility issue of zvariant with the use of reprotest (https://salsa.debian.org/reproducible-builds/reprotest)
and outside yocto environment. For the moment, I was not able to pinpoint the issue (I must have
put aside the faketime usage since it caused me segfault of rustc).I plan to dig further in
reproducible.pyclass and how the package is built under yocto to try to find out
what is going on.-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#179892):
https://lists.openembedded.org/g/openembedded-core/message/179892
Mute This Topic: https://lists.openembedded.org/mt/98126404/6213388
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 27 lines · Frédéric Martinsons
Hello with the patch below:
meta/lib/oeqa/selftest/cases/reproducible.py +0
-0@@ -134,9 +134,9 @@ class ReproducibleTests(OESelftestTestCase):max_report_size = 250 * 1024 * 1024
# targets are the things we want to test the reproducibility of - targets = ['core-image-minimal', 'core-image-sato', 'core-image- full-cmdline', 'core-image-weston', 'world'] + targets = ['zvariant'] # sstate targets are things to pull from sstate to potentially cut build/debugging time - sstate_targets = [] + sstate_targets = ['rust-native', 'cargo-native'] save_results = False if 'OEQA_DEBUGGING_SAVED_OUTPUT' in os.environ: save_results = os.environ['OEQA_DEBUGGING_SAVED_OUTPUT']I managed to reduce the time to test to 30mn (which is reasonable). Today, I made 5 run of this
reproducibility test and if library is indeed different between build A and build B every time, this
is the same exact difference, that is to day that the 5 libraries in build A are identical and so are
the 5 libraries in build B.
Quoted 4 lines
I admit I'm surprised , I would have waited for some random diff. I'll continue my investigation but if this rang a bell to someone here, feel free to tell (this does
seem strange to me).
It likely means it is the path the build is made in that is being encoded into the build somehow. The same binary is resulting for a given build path?
Cheers,
Richard
Quoted 37 lines · Richard Purdie
Quoted 27 lines · Frédéric Martinsons
Hello with the patch below:
meta/lib/oeqa/selftest/cases/reproducible.py +0
-1@@ -134,9 +134,9 @@ class ReproducibleTests(OESelftestTestCase): max_report_size = 250 * 1024 * 1024 # targets are the things we want to test the reproducibility of - targets = ['core-image-minimal', 'core-image-sato', 'core-image-full-cmdline', 'core-image-weston', 'world']
+ targets = ['zvariant']# sstate targets are things to pull from sstate to potentially cut build/debugging timemeta/lib/oeqa/selftest/cases/reproducible.py +1
-1- sstate_targets = [] + sstate_targets = ['rust-native', 'cargo-native'] save_results = False if 'OEQA_DEBUGGING_SAVED_OUTPUT' in os.environ: save_results = os.environ['OEQA_DEBUGGING_SAVED_OUTPUT']I managed to reduce the time to test to 30mn (which is reasonable). Today, I made 5 run of this
reproducibility test and if library is indeed different between build A and build B every time, this
is the same exact difference, that is to day that the 5 libraries in build A are identical and so are
the 5 libraries in build B.Quoted 4 lines
I admit I'm surprised , I would have waited for some random diff. I'll continue my investigation but if this rang a bell to someone here, feel free to tell (this does
seem strange to me).It likely means it is the path the build is made in that is being encoded into the build somehow. The same binary is resulting for a given build path?
Quoted 4 lines
Cheers,
Richard
I issued "strings" and "hexdump -C" on the libraries and didn't see any build path (the remap-path-prefix in rust-common.bbclass do it's job and I only saw /usr/src/debug with the rust source file of zvariant).
But maybe the build path is leaking anyway in some kind of binary format?