autobuild-run: initial implementation of check_reproducibility()
PATCH v3 1/3 autobuild-run: initial implementation of check_reproducibility()
For reproducible builds, we want to find out if there are any differences between two builds having the same configuration, and to find the reason behind them. The diffoscope tool looks inside different types of files to see where the differences lie.
check_reproducibility() runs diffoscope on two output directories which are expected in output/images and output/images-1. Since it uses objdump, it needs to be provided the cross-compile prefix which is derived from the TARGET_CROSS variable.
Since diffoscope may not be installed, we fall back to cmp for byte-by-byte comparison. We add diffoscope to list of optional programs to avoid repeated checking of its presence.
---
Changes v2 -> v3:
- Use file size of reproducible_results to check reproducibility status, rather than exit status of cmp or diff (suggested by arnout)
- Rename results file (diffoscope_output.txt -> reproducible_results) to avoid confusion
- Change handling of diffoscope output text file using with open()
- Changed commit message to have all necessary info (suggested by arnout)
- Removed leftover code from when I was using an exception to handle diffoscope presense (thanks to arnout)
Changes v1 -> v2:
- move diffoscope output to results dir (suggested by arnout)
- fix make printvars call
- Add diffoscope to DEFAULT_OPTIONAL_PROGS
--- scripts/autobuild-run | 39 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 38 insertions(+), 1 deletion(-)
scripts/autobuild-run +38 -1
@@ -204,7 +204,7 @@ def get_branch(): class SystemInfo: DEFAULT_NEEDED_PROGS = ["make", "git", "gcc"] - DEFAULT_OPTIONAL_PROGS = ["bzr", "java", "javac", "jar"] + DEFAULT_OPTIONAL_PROGS = ["bzr", "diffoscope", "java", "javac", "jar"] def __init__(self): self.needed_progs = list(self.__class__.DEFAULT_NEEDED_PROGS)
@@ -394,6 +394,43 @@ def stop_on_build_hang(monitor_thread_hung_build_flag, break monitor_thread_stop_flag.wait(30) +def check_reproducibility(**kwargs): + """Check reproducibility of builds + + Use diffoscope on the built images, if diffoscope is not + installed, fallback to cmp + """ + + log = kwargs['log'] + idir = "instance-%d" % kwargs['instance'] + outputdir = os.path.join(idir, "output") + srcdir = os.path.join(idir, "buildroot") + reproducible_results = os.path.join(outputdir, "results", "reproducible_results") + # Using only tar images for now + build_1_image = os.path.join(outputdir, "images-1", "rootfs.tar") + build_2_image = os.path.join(outputdir, "images", "rootfs.tar") + + with open(reproducible_results, 'w') as diff: + if kwargs['sysinfo'].has("diffoscope"): + # Prefix to point diffoscope towards cross-tools + prefix = subprocess.check_output(["make", "O=%s" % outputdir, "-C", srcdir, "printvars", "VARS=TARGET_CROSS"]) + # Remove TARGET_CROSS= and \n from the string + prefix = prefix[13:-1] + log_write(log, "INFO: running diffoscope on images") + subprocess.call(["diffoscope", build_1_image, build_2_image, + "--tool-prefix-binutils", prefix], stdout=diff, stderr=log) + else: + log_write(log, "INFO: diffoscope not installed, falling back to cmp") + subprocess.call(["cmp", "-b", "build_1_image", "build_2_image"], stdout=diff, stderr=log) + + if os.stat(reproducible_results).st_size > 0: + log_write(log, "INFO: Build is non-reproducible.") + return -1 + + # rootfs images match byte-for-byte -> reproducible image + log_write(log, "INFO: Build is reproducible!") + return 0 + def do_build(**kwargs): """Run the build itself"""
Signature · 1 line
2.20.1
PATCH v3 2/3 autobuild-run: initial implementation of do_reproducible_build()
This new function will call do_build() twice to produce two builds and then check their reproducibility.
---
Changes v2 -> v3:
- make clean outputs to logfile, similar to other make calls (suggested by y_morin)
- Output dir needs abspath (suggested by y_morin)
- Directly use make clean as arguments rather than putting it into a variable
- Added missing period at end of commit message (suggested by arnout)
Changes v1 -> v2:
- make clean outputs to devnull (suggested by arnout)
- remove unnecessary arguments to make clean (suggested by arnout)
--- scripts/autobuild-run | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+)
scripts/autobuild-run +37 -0
@@ -492,6 +492,43 @@ def do_build(**kwargs): log_write(log, "INFO: build successful") return 0 +def do_reproducible_build(**kwargs): + """Run the builds for reproducibility testing + + Build twice with the same configuration. Calls do_build() to + perform the actual build. + """ + + idir = "instance-%d" % kwargs['instance'] + outputdir = os.path.abspath(os.path.join(idir, "output")) + srcdir = os.path.join(idir, "buildroot") + log = kwargs['log'] + + # Start the first build + log_write(log, "INFO: Reproducible Build Test, starting build 1") + ret = do_build(**kwargs) + if ret != 0: + log_write(log, "INFO: build 1 failed, skipping build 2") + return ret + + # First build has been built, move files and start build 2 + os.rename(os.path.join(outputdir, "images"), os.path.join(outputdir, "images-1")) + + # Clean up build 1 + f = open(os.path.join(outputdir, "logfile"), "w+") + subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir, "clean"], stdout=f, stderr=f) + + # Start the second build + log_write(log, "INFO: Reproducible Build Test, starting build 2") + ret = do_build(**kwargs) + if ret != 0: + log_write(log, "INFO: build 2 failed") + return ret + + # Assuming both have built successfully + ret = check_reproducibility(**kwargs) + return ret + def send_results(result, **kwargs): """Prepare and store/send tarball with results
Signature · 1 line
2.20.1
PATCH v3 2/3 autobuild-run: initial implementation of do_reproducible_build()
Atharva, All,
On 2019-06-11 18:04 +0530, Atharva Lele spake thusly:
Quoted 5 lines
This new function will call
do_build()twice to produce two builds and then check their reproducibility.Signed-off-by Atharva Lele <[email protected]>
Yet, a few comments, see below...
Quoted 26 lines
---
Changes v2 -> v3:
- make clean outputs to logfile, similar to other make calls (suggested byy_morin)
- Output dir needs abspath (suggested byy_morin)
- Directly use make clean as arguments rather than putting it into a variable
- Added missing period at end of commit message (suggested by arnout)Changes v1 -> v2:
- make clean outputs to devnull (suggested by arnout)
- remove unnecessary arguments to make clean (suggested by arnout)Signed-off-by Atharva Lele <[email protected]>--- scripts/autobuild-run | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+)scripts/autobuild-run +2
-0@@ -492,6 +492,43 @@ def do_build(**kwargs): log_write(log, "INFO: build successful") return 0 +def do_reproducible_build(**kwargs): + """Run the builds for reproducibility testing
scripts/autobuild-run def do_build(**kwargs): · line 506
Quoted 10 lines
scripts/autobuild-run +10
-0+ + Build twice with the same configuration. Calls do_build() to + perform the actual build. + """ + + idir = "instance-%d" % kwargs['instance'] + outputdir = os.path.abspath(os.path.join(idir, "output")) + srcdir = os.path.join(idir, "buildroot") + log = kwargs['log'] +
Empty line made of spaces.
scripts/autobuild-run def do_build(**kwargs): · line 513
Quoted 7 lines
scripts/autobuild-run +7
-0+ # Start the first build + log_write(log, "INFO: Reproducible Build Test, starting build 1") + ret = do_build(**kwargs) + if ret != 0: + log_write(log, "INFO: build 1 failed, skipping build 2") + return ret +
Ditto.
scripts/autobuild-run def do_build(**kwargs): · line 519
Quoted 6 lines
scripts/autobuild-run +6
-0+ # First build has been built, move files and start build 2 + os.rename(os.path.join(outputdir, "images"), os.path.join(outputdir, "images-1")) + + # Clean up build 1 + f = open(os.path.join(outputdir, "logfile"), "w+") + subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir, "clean"], stdout=f, stderr=f)
Since we do create a small Makefile wrapper in $(O), you could simplify this a little with:
subprocess.call(["make", "-C", outputdir, "clean"], stdout=f, stderr=f)
Ditto in the existing do_build() function. Can be improved upon when you introduce the Builder class.
Regards,
Yann E. MORIN.
Quoted 17 lines
scripts/autobuild-run +11
-0+ # Start the second build + log_write(log, "INFO: Reproducible Build Test, starting build 2") + ret = do_build(**kwargs) + if ret != 0: + log_write(log, "INFO: build 2 failed") + return ret + + # Assuming both have built successfully + ret = check_reproducibility(**kwargs) + return ret + def send_results(result, **kwargs): """Prepare and store/send tarball with resultsSignature · 1 line
2.20.1
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH v3 2/3 autobuild-run: initial implementation of do_reproducible_build()
Quoted 3 lines · Yann E. MORIN
Atharva, All,
On 2019-06-11 18:04 +0530, Atharva Lele spake thusly:
[snip]
Quoted 11 lines
scripts/autobuild-run def do_build(**kwargs): · line 519
Quoted 3 lines
scripts/autobuild-run +3
-0+ # Clean up build 1 + f = open(os.path.join(outputdir, "logfile"), "w+") + subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir, "clean"], stdout=f, stderr=f)Since we do create a small
Makefilewrapper in $(O), you could simplify this a little with:subprocess.call(["make", "-C", outputdir, "clean"], stdout=f, stderr=f)Ditto in the existing
do_build()function. Can be improved upon when you introduce the Builder class.
This is bikeshedding, obviously, but I don't like that proposal. I think an
explicit O= -C is easier to read and understand.
Shorter is not better :-)
Regards,
Arnout
PATCH v3 2/3 autobuild-run: initial implementation of do_reproducible_build()
Quoted 1 line
I think an explicit O= -C is easier to read and understand.
I do agree with this.
Regards,
Atharva Lele
Quoted 21 lines · Arnout Vandecappelle
Quoted 3 lines · Yann E. MORIN
Atharva, All,
On 2019-06-11 18:04 +0530, Atharva Lele spake thusly:
[snip]
Quoted 3 lines
scripts/autobuild-run def do_build(**kwargs):
Quoted 3 lines
scripts/autobuild-run +3
-0+ # Clean up build 1 + f = open(os.path.join(outputdir, "logfile"), "w+") + subprocess.call(["make", "O=%s" % outputdir, "-C", srcdir,"clean"], stdout=f, stderr=f)
Quoted 5 lines
Since we do create a small
Makefilewrapper in $(O), you could simplify this a little with:subprocess.call(["make", "-C", outputdir, "clean"], stdout=f,stderr=f)
Quoted 3 lines
Ditto in the existing
do_build()function. Can be improved upon when you introduce the Builder class.
Quoted 10 lines
This is bikeshedding, obviously, but I don't like that proposal. I think
an
explicit O= -C is easier to read and understand.Shorter is not better :-)
Regards,
Arnout
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20190614/259a3597/attachment.html>
PATCH v3 3/3 autobuild-run: do reproducible builds tests if BR2_REPRODUCIBLE=y
In reply to Atharva Lele---
Changes v2 -> v3:
- use reproducible flag so that .config i snot kept open for the entire build duration. (suggested by arnout)
- remove reduntant logging (suggested by arnout)
Changes v1 -> v2:
- add trailing newline character to BR2_REPRODUCIBLE=y (suggested by arnout)
--- scripts/autobuild-run | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
scripts/autobuild-run +8 -1
@@ -727,7 +727,14 @@ def run_instance(**kwargs): log_write(kwargs['log'], "WARN: failed to generate configuration") continue - ret = do_build(**kwargs) + # Check if the build test is supposed to be a reproducible test + outputdir = os.path.abspath(os.path.join(idir, "output")) + with open(os.path.join(outputdir, ".config"), "r") as fconf: + reproducible = "BR2_REPRODUCIBLE=y\n" in fconf.read() + if reproducible: + ret = do_reproducible_build(**kwargs) + else: + ret = do_build(**kwargs) send_results(ret, **kwargs)
Signature · 1 line
2.20.1
PATCH v3 3/3 autobuild-run: do reproducible builds tests if BR2_REPRODUCIBLE=y
On 2019-06-11 18:04 +0530, Atharva Lele spake thusly:
Quoted 7 lines
Signed-off-by Atharva Lele <[email protected]>---
Changes v2 -> v3:
- use reproducible flag so that .config i snot kept open for the entire build duration. (suggested by arnout)
Sorry, but that is still the case, see below...
Quoted 17 lines
- remove reduntant logging (suggested by arnout)
Changes v1 -> v2:
- add trailing newline character to BR2_REPRODUCIBLE=y (suggested by arnout)Signed-off-by Atharva Lele <[email protected]>--- scripts/autobuild-run | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)scripts/autobuild-run +0
-0@@ -727,7 +727,14 @@ def run_instance(**kwargs): log_write(kwargs['log'], "WARN: failed to generate configuration") continue
scripts/autobuild-run def run_instance(**kwargs): · line 737
Quoted 10 lines
scripts/autobuild-run +8
-1- ret = do_build(**kwargs) + # Check if the build test is supposed to be a reproducible test + outputdir = os.path.abspath(os.path.join(idir, "output")) + with open(os.path.join(outputdir, ".config"), "r") as fconf: + reproducible = "BR2_REPRODUCIBLE=y\n" in fconf.read() + if reproducible: + ret = do_reproducible_build(**kwargs) + else: + ret = do_build(**kwargs)
This whole if reproducible block is still indented inside the with block, so you'r still having .config opened during the whole build.
You need to do (basically):
with open(os.path.join(outputdir, ".config"), "r") as fconf:
reproducible = "BR2_REPRODUCIBLE=y\n" in fconf.read()
build = do_reproducible_build if reproducible else do_build ret = build(**kwargs)
Regards,
Yann E. MORIN.
Quoted 6 lines
scripts/autobuild-run +0
-0send_results(ret, **kwargs)Signature · 1 line
2.20.1
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH v3 3/3 autobuild-run: do reproducible builds tests if BR2_REPRODUCIBLE=y
Quoted 6 lines
You need to do (basically):
with
open(os.path.join(outputdir, ".config"), "r")as fconf:
reproducible = "BR2_REPRODUCIBLE=y\n" infconf.read()
build =do_reproducible_buildif reproducible elsedo_buildret =build(**kwargs)
It would still work if I just unindent the if-else block too right? This is what I actually intended to do
It's working for me in testing..
Like this:
with open(os.path.join(outputdir, ".config"), "r") as fconf:
reproducible = "BR2_REPRODUCIBLE=y\n" in fconf.read()
if reproducible:
ret = do_reproducible_build(**kwargs)
else:
ret = do_build(**kwargs)
Regards,
Atharva Lele
Quoted 73 lines · Yann E. MORIN
On 2019-06-11 18:04 +0530, Atharva Lele spake thusly:
Quoted 7 lines
Signed-off-by Atharva Lele <[email protected]>---
Changes v2 -> v3:
- use reproducible flag so that .config i snot kept open for the entire build duration. (suggested by arnout)Sorry, but that is still the case, see below...
Quoted 4 lines
- remove reduntant logging (suggested by arnout)
Changes v1 -> v2:
- add trailing newline character to BR2_REPRODUCIBLE=y (suggested byarnout)
scripts/autobuild-run def run_instance(**kwargs): · line 727
Quoted 12 lines
Signed-off-by Atharva Lele <[email protected]>--- scripts/autobuild-run | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)scripts/autobuild-run +0
-0@@ -727,7 +727,14 @@ def run_instance(**kwargs): log_write(kwargs['log'], "WARN: failed to generateconfiguration")
scripts/autobuild-run def run_instance(**kwargs): · line 737
Quoted 11 lines
scripts/autobuild-run +8
-1continue - ret = do_build(**kwargs) + # Check if the build test is supposed to be a reproducible test + outputdir = os.path.abspath(os.path.join(idir, "output")) + with open(os.path.join(outputdir, ".config"), "r") as fconf: + reproducible = "BR2_REPRODUCIBLE=y\n" in fconf.read() + if reproducible: + ret = do_reproducible_build(**kwargs) + else: + ret = do_build(**kwargs)This whole if reproducible block is still indented inside the with block, so you'r still having .config opened during the whole build.
You need to do (basically):
with
open(os.path.join(outputdir, ".config"), "r")as fconf:
reproducible = "BR2_REPRODUCIBLE=y\n" infconf.read()
build =do_reproducible_buildif reproducible elsedo_buildret =build(**kwargs)Regards,
Yann E. MORIN.Quoted 6 lines
scripts/autobuild-run +0
-1send_results(ret, **kwargs) --2.20.1
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
Quoted 2 lines
'------------------------------^-------^------------------^--------------------'
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20190614/6362ef8b/attachment.html>
PATCH v3 3/3 autobuild-run: do reproducible builds tests if BR2_REPRODUCIBLE=y
Quoted 12 lines · Atharva Lele
Quoted 6 lines
You need to do (basically):
? ? with
open(os.path.join(outputdir, ".config"), "r")as fconf:
? ? ? ? reproducible = "BR2_REPRODUCIBLE=y\n" infconf.read()? ? build =do_reproducible_buildif reproducible elsedo_build? ? ret =build(**kwargs)It would still work if I just unindent the if-else block too right? This is what I actually intended to do
It's working for me in testing..
Like this:
Quoted 6 lines
? ? with
open(os.path.join(outputdir, ".config"), "r")as fconf:
? ? ? ? reproducible = "BR2_REPRODUCIBLE=y\n" infconf.read()? ? if reproducible:
? ? ? ? ret =do_reproducible_build(**kwargs)
? ? else:
? ? ? ? ret =do_build(**kwargs)
Yes, and I prefer that version. Yann considers the '... if ... else ...'
construct more pythonese, but I kind of like explicit conditions :-)
Regards,
Arnout
[snip]
PATCH v3 3/3 autobuild-run: do reproducible builds tests if BR2_REPRODUCIBLE=y
Quoted 2 lines
Yann considers the '... if ... else ...' construct more pythonese, but I kind of like explicit conditions :-)
Haha I do see his point of view. For me, I've only worked with C for the last 2 years at university and my last internship so it's a bit hard to 'talk' in
pythonese.
Regards,
Atharva Lele
Quoted 23 lines · Arnout Vandecappelle
Quoted 8 lines · Atharva Lele
Quoted 6 lines
You need to do (basically):
with
open(os.path.join(outputdir, ".config"), "r")as fconf:
reproducible = "BR2_REPRODUCIBLE=y\n" infconf.read()
build =do_reproducible_buildif reproducible elsedo_buildret =build(**kwargs)It would still work if I just unindent the if-else block too right? This
is what
Quoted 10 lines
I actually intended to do
It's working for me in testing..
Like this:with
open(os.path.join(outputdir, ".config"), "r")as fconf:
reproducible = "BR2_REPRODUCIBLE=y\n" infconf.read()
if reproducible:
ret =do_reproducible_build(**kwargs)
else:
ret =do_build(**kwargs)
Quoted 9 lines
Yes, and I prefer that version. Yann considers the '... if ... else ...'
construct more pythonese, but I kind of like explicit conditions :-)Regards,
Arnout[snip]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20190614/e12fa2c2/attachment.html>
PATCH v3 3/3 autobuild-run: do reproducible builds tests if BR2_REPRODUCIBLE=y
In reply to Arnout VandecappelleArnout, All,
On 2019-06-14 10:47 +0200, Arnout Vandecappelle spake thusly:
Quoted 20 lines · Atharva Lele
Quoted 12 lines
Quoted 6 lines
You need to do (basically):
? ? with
open(os.path.join(outputdir, ".config"), "r")as fconf:
? ? ? ? reproducible = "BR2_REPRODUCIBLE=y\n" infconf.read()? ? build =do_reproducible_buildif reproducible elsedo_build? ? ret =build(**kwargs)It would still work if I just unindent the if-else block too right? This is what I actually intended to do
It's working for me in testing..
Like this:Quoted 6 lines
? ? with
open(os.path.join(outputdir, ".config"), "r")as fconf:
? ? ? ? reproducible = "BR2_REPRODUCIBLE=y\n" infconf.read()? ? if reproducible:
? ? ? ? ret =do_reproducible_build(**kwargs)
? ? else:
? ? ? ? ret =do_build(**kwargs)
Quoted 2 lines
Yes, and I prefer that version. Yann considers the '... if ... else ...'
construct more pythonese, but I kind of like explicit conditions :-)
Well, if we use a language, we should write with the idioms of that language. Python is not C, so we should not write Python as we would C.
That notwithstanding, I'm no Python expert either, so I'm also OK with the more "traditional and conventional" ways. ;-)
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH v3 1/3 autobuild-run: initial implementation of check_reproducibility()
In reply to Atharva LeleAtharva, All,
On 2019-06-11 18:04 +0530, Atharva Lele spake thusly:
Quoted 10 lines
For reproducible builds, we want to find out if there are any differences between two builds having the same configuration, and to find the reason behind them. The diffoscope tool looks inside different types of files to see where the differences lie.
check_reproducibility()runs diffoscope on two output directories which are expected in output/images and output/images-1. Since it uses objdump, it needs to be provided the cross-compile prefix which is derived from theTARGET_CROSSvariable.
Quoted 5 lines
Since diffoscope may not be installed, we fall back to cmp for byte-by-byte comparison. We add diffoscope to list of optional programs to avoid repeated checking of its presence.
Signed-off-by Atharva Lele <[email protected]>
Except for two minor coding rules (see below):
However, for the future: it would be wonderfull if we could entice the diffoscope developers to make it possible to use it directly from
python: diffoscope *is* a python module, after all.
However, it is cutrrently not very amenable at being used as a python object (yet):
diffoscope.main():
https://salsa.debian.org/reproducible-builds/diffoscope/blob/master/diffoscope/main.py#L695
which is a glorified wrapper to diffoscope.run_diffoscope():
https://salsa.debian.org/reproducible-builds/diffoscope/blob/master/diffoscope/main.py#L600
which is itself a further glofied wrapper to diffoscope.compare_root_paths() and diffoscope.Difference()...
But that's for later.
Quoted 56 lines
---
Changes v2 -> v3:
- Use file size ofreproducible_resultsto check reproducibility status, rather than exit status of cmp or diff (suggested by arnout)
- Rename results file (diffoscope_output.txt ->reproducible_results) to avoid confusion
- Change handling of diffoscope output text file using withopen()
- Changed commit message to have all necessary info (suggested by arnout)
- Removed leftover code from when I was using an exception to handle diffoscope presense (thanks to arnout)Changes v1 -> v2:
- move diffoscope output to results dir (suggested by arnout)
- fix make printvars call
- Add diffoscope toDEFAULT_OPTIONAL_PROGSSigned-off-by Atharva Lele <[email protected]>--- scripts/autobuild-run | 39 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 38 insertions(+), 1 deletion(-)scripts/autobuild-run +20
-1@@ -204,7 +204,7 @@ def get_branch(): class SystemInfo: DEFAULT_NEEDED_PROGS = ["make", "git", "gcc"] - DEFAULT_OPTIONAL_PROGS = ["bzr", "java", "javac", "jar"] + DEFAULT_OPTIONAL_PROGS = ["bzr", "diffoscope", "java", "javac", "jar"] def __init__(self): self.needed_progs = list(self.__class__.DEFAULT_NEEDED_PROGS)@@ -394,6 +394,43 @@ def stop_on_build_hang(monitor_thread_hung_build_flag, break monitor_thread_stop_flag.wait(30) +def check_reproducibility(**kwargs): + """Check reproducibility of builds + + Use diffoscope on the built images, if diffoscope is not + installed, fallback to cmp + """ + + log = kwargs['log'] + idir = "instance-%d" % kwargs['instance'] + outputdir = os.path.join(idir, "output") + srcdir = os.path.join(idir, "buildroot") + reproducible_results = os.path.join(outputdir, "results", "reproducible_results") + # Using only tar images for now + build_1_image = os.path.join(outputdir, "images-1", "rootfs.tar") + build_2_image = os.path.join(outputdir, "images", "rootfs.tar") + + with open(reproducible_results, 'w') as diff: + if kwargs['sysinfo'].has("diffoscope"): + # Prefix to point diffoscope towards cross-tools
scripts/autobuild-run def stop_on_build_hang(monitor_thread_hung_build_flag, · line 425
Quoted 10 lines
scripts/autobuild-run +10
-0+ prefix = subprocess.check_output(["make", "O=%s" % outputdir, "-C", srcdir, "printvars", "VARS=TARGET_CROSS"]) + # Remove TARGET_CROSS= and \n from the string + prefix = prefix[13:-1] + log_write(log, "INFO: running diffoscope on images") + subprocess.call(["diffoscope", build_1_image, build_2_image, + "--tool-prefix-binutils", prefix], stdout=diff, stderr=log) + else: + log_write(log, "INFO: diffoscope not installed, falling back to cmp") + subprocess.call(["cmp", "-b", "build_1_image", "build_2_image"], stdout=diff, stderr=log) +
They are invisible, but this empty line has 4 spaces.
scripts/autobuild-run def stop_on_build_hang(monitor_thread_hung_build_flag, · line 429
Quoted 4 lines
scripts/autobuild-run +4
-0+ if os.stat(reproducible_results).st_size > 0: + log_write(log, "INFO: Build is non-reproducible.") + return -1 +
Ditto.
Regards,
Yann E. MORIN.
Quoted 10 lines
scripts/autobuild-run +4
-0+ # rootfs images match byte-for-byte -> reproducible image + log_write(log, "INFO: Build is reproducible!") + return 0 + def do_build(**kwargs): """Run the build itself"""Signature · 1 line
2.20.1
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'