Skip to contentSkip to conversation

sysvinit: Fix Reproducibility issue

PATCH In review Richard Purdie 2 participants 4 messages Feb 7, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/3 sysvinit: Fix Reproducibility issue

With a sequence like:

bitbake sysvinit
bitbake sysvinit -c clean
bitbake sysvinit -c package_write_ipk -f

then the resulting package has two files with group "root/70" rather than "root/shutdown". The issue is that of do_package is a setscene task, base-passwd isn't present. This patch fixes that dependency but there may be other cases of this problem around.

[YOCTO #13776]

---
 meta/recipes-core/sysvinit/sysvinit_2.96.bb | 1 +
 1 file changed, 1 insertion(+)
meta/recipes-core/sysvinit/sysvinit_2.96.bb +1 -0
index 2b146b1ef8f..349ea71837d 100644
@@ -29,6 +29,7 @@ B = "${S}/src"
 
 inherit update-alternatives features_check
 DEPENDS_append = " update-rc.d-native base-passwd virtual/crypt"
+do_package_setscene[depends] = "base-passwd:do_populate_sysroot"
 
 REQUIRED_DISTRO_FEATURES = "sysvinit"
 
Signature · 1 line
2.20.1
#1 lore raw

PATCH 2/3 libevdev: Fix determinism issue

We need to sort python dict output to be deterministic and generate consistent header files.

---
 .../libevdev/libevdev/determinism.patch       | 34 +++++++++++++++++++
 .../libevdev/libevdev_1.8.0.bb                |  3 +-
 2 files changed, 36 insertions(+), 1 deletion(-)
 create mode 100644 meta/recipes-support/libevdev/libevdev/determinism.patch
meta/recipes-support/libevdev/libevdev/determinism.patch new file +34 -0
new file mode 100644
index 00000000000..33a6076b781
@@ -0,0 +1,34 @@
+The order of dict values is not deterministic leading to differing header file generation.
+Sort to remove this inconsistency.
+
+RP 2020/2/7
+
+Signed-off-by: Richard Purdie <[email protected]>
+Upstream-Status: Pending
+
+Index: a/libevdev/make-event-names.py
+===================================================================
+--- a/libevdev/make-event-names.py
++++ b/libevdev/make-event-names.py
+@@ -67,10 +67,10 @@ def print_bits(bits, prefix):
+ 	if  not hasattr(bits, prefix):
+ 		return
+ 	print("static const char * const %s_map[%s_MAX + 1] = {" % (prefix, prefix.upper()))
+-	for val, name in list(getattr(bits, prefix).items()):
++	for val, name in sorted(list(getattr(bits, prefix).items())):
+ 		print("	[%s] = \"%s\"," % (name, name))
+ 	if prefix == "key":
+-		for val, name in list(getattr(bits, "btn").items()):
++		for val, name in sorted(list(getattr(bits, "btn").items())):
+ 			print("	[%s] = \"%s\"," % (name, name))
+ 	print("};")
+ 	print("")
+@@ -111,7 +111,7 @@ def print_lookup(bits, prefix):
+ 	if not hasattr(bits, prefix):
+ 		return
+ 
+-	names = list(getattr(bits, prefix).items())
++	names = sorted(list(getattr(bits, prefix).items()))
+ 	if prefix == "btn":
+ 		names = names + btn_additional;
+ 
meta/recipes-support/libevdev/libevdev_1.8.0.bb +2 -1
index 84274987d71..46ed5d786ae 100644
@@ -6,7 +6,8 @@ LICENSE = "MIT-X"
 LIC_FILES_CHKSUM = "file://COPYING;md5=75aae0d38feea6fda97ca381cb9132eb \
                     file://libevdev/libevdev.h;endline=21;md5=7ff4f0b5113252c2f1a828e0bbad98d1"
 
-SRC_URI = "http://www.freedesktop.org/software/libevdev/${BP}.tar.xz"
+SRC_URI = "http://www.freedesktop.org/software/libevdev/${BP}.tar.xz \
+           file://determinism.patch"
 SRC_URI[md5sum] = "879631080be18526737e33b63d848039"
 SRC_URI[sha256sum] = "20d3cae4efd277f485abdf8f2a7c46588e539998b5a08c2c4d368218379d4211"
 
Signature · 1 line
2.20.1
#2 lore raw

PATCH 3/3 perl: Fix various reproducibile build issues

In reply to Richard Purdie

Add a patch which handles the following issues:

a) Remove the \n from configure_attr.sh since it gets quoted differently depending on whether the shell is bash or dash which can cause the test result to be incorrect. Reported upstream: https://github.com/arsv/perl-cross/issues/87

b) Sort the order of the module lists from configure_mods.sh since otherwise the result isn't the same leading to makefile differences. Reported upstream: https://github.com/arsv/perl-cross/issues/88

c) Sort the Encode::Byte byte_t.fnm file output (and the makefile depends whilst there for good measure)
This needs to go to upstream perl (not done)

d) Use bash for perl-cross configure since otherwise trnl gets set to "\n" with bash and "" with dash
Reported upstream: https://github.com/arsv/perl-cross/issues/87

---
 .../perl/files/determinism.patch              | 81 +++++++++++++++++++
 meta/recipes-devtools/perl/perl_5.30.1.bb     |  1 +
 2 files changed, 82 insertions(+)
 create mode 100644 meta/recipes-devtools/perl/files/determinism.patch
meta/recipes-devtools/perl/files/determinism.patch new file +81 -0
new file mode 100644
index 00000000000..ed4d06f5ec6
@@ -0,0 +1,81 @@
+Fixes to make the perl build reproducible:
+
+a) Remove the \n from configure_attr.sh since it gets quoted differently depending on
+   whether the shell is bash or dash which can cause the test result to be incorrect.
+   Reported upstream: https://github.com/arsv/perl-cross/issues/87
+
+b) Sort the order of the module lists from configure_mods.sh since otherwise
+   the result isn't the same leading to makefile differences.
+   Reported upstream: https://github.com/arsv/perl-cross/issues/88
+
+c) Sort the Encode::Byte byte_t.fnm file output (and the makefile depends whilst 
+   there for good measure)
+   This needs to go to upstream perl (not done)
+
+d) Use bash for perl-cross configure since otherwise trnl gets set to "\n" with bash
+   and "" with dash
+   Reported upstream: https://github.com/arsv/perl-cross/issues/87
+
+RP 2020/2/7
+
+Upstream-Status: Pending [75% submitted]
+Signed-off-by: Richard Purdie <[email protected]
+
+Index: perl-5.30.1/cnf/configure_attr.sh
+===================================================================
+--- perl-5.30.1.orig/cnf/configure_attr.sh
++++ perl-5.30.1/cnf/configure_attr.sh
+@@ -131,7 +131,7 @@ if not hinted d_c99_variadic_macros 'sup
+ 	try_start
+ 	try_add '#include <stdio.h>'
+ 	try_add '#define foo(fmt, ...) printf(fmt, __VA_ARGS__)'
+-	try_add 'int main(void) { foo("%i\n", 1234); return 0; }'
++	try_add 'int main(void) { foo("%i", 1234); return 0; }'
+ 	try_compile
+ 	resdef d_c99_variadic_macros 'supported' 'missing'
+ fi
+Index: perl-5.30.1/cnf/configure_mods.sh
+===================================================================
+--- perl-5.30.1.orig/cnf/configure_mods.sh
++++ perl-5.30.1/cnf/configure_mods.sh
+@@ -82,7 +82,7 @@ extonlyif() {
+ }
+ 
+ definetrimspaces() {
+-	v=`echo "$2" | sed -r -e 's/\s+/ /g' -e 's/^\s+//' -e 's/\s+$//'`
++	v=`echo "$2" | sed -r -e 's/\s+/ /g' -e 's/^\s+//' -e 's/\s+$//' | xargs -n1 | LANG=C sort | xargs`
+ 	define $1 "$v"
+ }
+ 
+Index: perl-5.30.1/cpan/Encode/Byte/Makefile.PL
+===================================================================
+--- perl-5.30.1.orig/cpan/Encode/Byte/Makefile.PL
++++ perl-5.30.1/cpan/Encode/Byte/Makefile.PL
+@@ -171,7 +171,7 @@ sub postamble
+     my $lengthsofar = length($str);
+     my $continuator = '';
+     $str .= "$table.c : $enc2xs Makefile.PL";
+-    foreach my $file (@{$tables{$table}})
++    foreach my $file (sort (@{$tables{$table}}))
+     {
+         $str .= $continuator.' '.$self->catfile($dir,$file);
+         if ( length($str)-$lengthsofar > 128*$numlines )
+@@ -189,7 +189,7 @@ sub postamble
+         qq{\n\t\$(PERL) $plib $enc2xs $ucopts -o \$\@ -f $table.fnm\n\n};
+     open (FILELIST, ">$table.fnm")
+         || die "Could not open $table.fnm: $!";
+-    foreach my $file (@{$tables{$table}})
++    foreach my $file (sort (@{$tables{$table}}))
+     {
+         print FILELIST $self->catfile($dir,$file) . "\n";
+     }
+Index: perl-5.30.1/cnf/configure
+===================================================================
+--- perl-5.30.1.orig/cnf/configure
++++ perl-5.30.1/cnf/configure
+@@ -1,4 +1,4 @@
+-#!/bin/sh
++#!/bin/bash
+ 
+ base=${0%/*}; test -z "$base" && base=.
+ 
meta/recipes-devtools/perl/perl_5.30.1.bb +1 -0
index dcdabb692dd..5d762710df3 100644
@@ -21,6 +21,7 @@ SRC_URI = "https://www.cpan.org/src/5.0/perl-${PV}.tar.gz;name=perl \
            file://0001-enc2xs-Add-environment-variable-to-suppress-comments.patch \
            file://0002-Constant-Fix-up-shebang.patch \
            file://0001-tests-adjust-to-correctly-exclude-unbuilt-extensions.patch \
+           file://determinism.patch  \
            "
 SRC_URI_append_class-native = " \
            file://perl-configpm-switch.patch \
Signature · 1 line
2.20.1
#3 lore raw

✗ patchtest: failure for "sysvinit: Fix Reproducibility ..." and 2 more

In reply to Richard Purdie

== Series Details ==

Series: "sysvinit: Fix Reproducibility ..." and 2 more
Revision: 1
URL : https://patchwork.openembedded.org/series/22469/
State : failure

== Summary ==

Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures:

* Issue Upstream-Status is Submitted, but it is not mentioned where [test_upstream_status_presence_format]

  Suggested fix    Include where determinism.patch was submitted
  Current          Upstream-Status: Pending [75% submitted]
  Standard format  Upstream-Status: Submitted [where]

If you believe any of these test results are incorrect, please reply to the mailing list ([email protected]) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...).

---

Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe