Skip to contentSkip to conversation

Patch review

PATCH In review Armin Kuster 4 participants 34 messages Feb 9, 2020 – Feb 10, 2020

lore
Share & tools
lore
#0 lore raw

cover Patch review

These are the additional changes to help address reproducibility issues and additional fixes we would like to be included in 3.0.2

Please have comments back by Tuesday

The following changes since commit 9b1bf083129be2b849db52d4f0eda9eb6077c97e:

python2: add ntpath (2020-02-02 18:19:50 -0800)

are available in the Git repository at:

git://git.openembedded.org/openembedded-core-contrib stable/zeus-nut
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=stable/zeus-nut

Alejandro del Castillo (1):
opkg-utils: upgrade to version 0.4.2

Alexander Kanavin (1):
perl: do not install files that contain build host specific data

Anuj Mittal (3):
Revert "bzip2: Fix CVE-2019-12900"
curl: fix CVE-2019-15601
cpio: fix CVE-2019-14866

Joshua Watt (2):
classes/reproducible_build: Read SDE file later
mc: Fix build reproducibility

Lee Chee Yang (1):
rsync: whitelist CVE-2017-16548

Richard Purdie (17):
opkg-utils: Fix reproducibility issues in opkg-build oeqa/reproducible: Improve test output and ensure deb+ipk compared
sudo: Set vardir deterministically
libxshmfence: Set shm directory deterministically
mc: Set zipinfo presence determinstically
mc: Fix manpage date indeterminism
tar: Fix build determinism, disable rsh
patch: Extend to native/nativesdk and depend upon
libidn2: Fix reproducibility issue
perl: Fix various reproducibile build issues
openssl: Fix reproducibility issue
iputils: Fix build determinism
libinput: Fix determinism issue
libgcrypt: Fix determinism issue
sysvinit: Fix Reproducibility issue
libevdev: Fix determinism issue
ncurses: Fix reproducibility issue

Ross Burton (2):
gtk+3: sort resources for reproducible binaries
sudo: specify where target tools are

Taras Kondratiuk via Openembedded-core (1):
gcc-9.2: fix bug #91102 'aarch64 ICE on Linux kernel with -Os'

Tom Hochstein (1):
devtool/standard.py: Allow recipe to disable menuconfig logic

 meta/classes/patch.bbclass                    |   7 +
 meta/classes/reproducible_build.bbclass       |  40 ++-
 meta/lib/oeqa/selftest/cases/reproducible.py  |   9 +-
 .../openssl/openssl/reproducible.patch        |  32 ++
 .../openssl/openssl_1.1.1d.bb                 |   1 +
 meta/recipes-core/meta/buildtools-tarball.bb  |   1 +
 meta/recipes-core/ncurses/ncurses.inc         |   1 +
 .../recipes-core/sysvinit/sysvinit_2.88dsf.bb |   1 +
 meta/recipes-devtools/gcc/gcc-9.2.inc         |   1 +
 ...02-aarch64-ICE-on-Linux-kernel-with-.patch |  95 ++++++
 ...Switch-all-scripts-to-use-Python-3.x.patch | 113 -------
 ...ld-clamp-mtimes-to-SOURCE_DATE_EPOCH.patch |  44 ---
 .../opkg-utils/fix-reproducibility.patch      |  32 ++
 .../opkg-utils/opkg-utils/pipefail.patch      |  31 --
 ...pkg-utils_0.4.1.bb => opkg-utils_0.4.2.bb} |  13 +-
 meta/recipes-devtools/patch/patch_2.7.6.bb    |   3 +
 .../perl/files/determinism.patch              |  81 +++++
 meta/recipes-devtools/perl/perl-ptest.inc     |   3 +
 meta/recipes-devtools/perl/perl_5.30.0.bb     |   4 +
 meta/recipes-devtools/rsync/rsync_3.1.3.bb    |   3 +
 .../bzip2/bzip2-1.0.6/CVE-2019-12900.patch    |  36 --
 .../cpio/cpio-2.12/CVE-2019-14866.patch       | 316 ++++++++++++++++++
 meta/recipes-extended/cpio/cpio_2.12.bb       |   1 +
 .../iputils/iputils_s20190709.bb              |   3 +-
 meta/recipes-extended/libidn/libidn2_2.2.0.bb |   3 +-
 ...Add-option-to-control-configure-args.patch |  99 ++++++
 .../recipes-extended/mc/files/nomandate.patch |  21 ++
 meta/recipes-extended/mc/mc_4.8.23.bb         |   7 +-
 meta/recipes-extended/sudo/sudo.inc           |   2 +-
 meta/recipes-extended/sudo/sudo_1.8.27.bb     |  10 +-
 meta/recipes-extended/tar/tar_1.32.bb         |   2 +
 .../gtk+/gtk+3/sort-resources.patch           |  19 ++
 meta/recipes-gnome/gtk+/gtk+3_3.24.8.bb       |   1 +
 .../wayland/libinput/determinism.patch        |  21 ++
 .../wayland/libinput_1.14.1.bb                |   4 +-
 .../xorg-lib/libxshmfence_1.3.bb              |   2 +
 .../curl/curl/CVE-2019-15601.patch            |  46 +++
 meta/recipes-support/curl/curl_7.66.0.bb      |   1 +
 .../libevdev/libevdev/determinism.patch       |  34 ++
 .../libevdev/libevdev_1.8.0.bb                |   3 +-
 .../libgcrypt/files/determinism.patch         |  32 ++
 .../libgcrypt/libgcrypt_1.8.4.bb              |   1 +
 scripts/lib/devtool/standard.py               |   6 +-
 43 files changed, 933 insertions(+), 252 deletions(-)
 create mode 100644 meta/recipes-connectivity/openssl/openssl/reproducible.patch
 create mode 100644 meta/recipes-devtools/gcc/gcc-9.2/re-PR-target-91102-aarch64-ICE-on-Linux-kernel-with-.patch
 delete mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/0001-Switch-all-scripts-to-use-Python-3.x.patch
 delete mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/0001-opkg-build-clamp-mtimes-to-SOURCE_DATE_EPOCH.patch
 create mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/fix-reproducibility.patch
 delete mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/pipefail.patch
 rename meta/recipes-devtools/opkg-utils/{opkg-utils_0.4.1.bb => opkg-utils_0.4.2.bb} (83%)
 create mode 100644 meta/recipes-devtools/perl/files/determinism.patch
 delete mode 100644 meta/recipes-extended/bzip2/bzip2-1.0.6/CVE-2019-12900.patch
 create mode 100644 meta/recipes-extended/cpio/cpio-2.12/CVE-2019-14866.patch
 create mode 100644 meta/recipes-extended/mc/files/0001-Add-option-to-control-configure-args.patch
 create mode 100644 meta/recipes-extended/mc/files/nomandate.patch
 create mode 100644 meta/recipes-gnome/gtk+/gtk+3/sort-resources.patch
 create mode 100644 meta/recipes-graphics/wayland/libinput/determinism.patch
 create mode 100644 meta/recipes-support/curl/curl/CVE-2019-15601.patch
 create mode 100644 meta/recipes-support/libevdev/libevdev/determinism.patch
 create mode 100644 meta/recipes-support/libgcrypt/files/determinism.patch
Signature · 1 line
2.17.1
#1 lore raw

PATCH 1/29 Revert "bzip2: Fix CVE-2019-12900"

From: Anuj Mittal <[email protected]>

This reverts commit 175e6cb75ce328d51a9d4ad18c7e09d9fb92c2e1.

This change is already in bzip2 1.0.7. The change fixing a regression caused by this change is in 1.0.8 which is the current version in zeus.

This isn't resulting in failures because the patch file isn't included in SRC_URI.

---
 .../bzip2/bzip2-1.0.6/CVE-2019-12900.patch    | 36 -------------------
 1 file changed, 36 deletions(-)
 delete mode 100644 meta/recipes-extended/bzip2/bzip2-1.0.6/CVE-2019-12900.patch
meta/recipes-extended/bzip2/bzip2-1.0.6/CVE-2019-12900.patch deleted +0 -36
deleted file mode 100644
index 9859d9d1a2..0000000000
@@ -1,36 +0,0 @@
-From 74de1e2e6ffc9d51ef9824db71a8ffee5962cdbc Mon Sep 17 00:00:00 2001
-From: Albert Astals Cid <[email protected]>
-Date: Tue, 28 May 2019 19:35:18 +0200
-Subject: [PATCH] Make sure nSelectors is not out of range
-
-nSelectors is used in a loop from 0 to nSelectors to access selectorMtf
-which is
-UChar    selectorMtf[BZ_MAX_SELECTORS];
-so if nSelectors is bigger than BZ_MAX_SELECTORS it'll do an invalid memory
-access
-Fixes out of bounds access discovered while fuzzying karchive
-
-Link: https://gitlab.com/federicomenaquintero/bzip2/commit/74de1e2e6ffc9d51ef9824db71a8ffee5962cdbc.patch
-
-Upstream-Status: Backport
-CVE: CVE-2019-12900.patch
-Signed-off-by: Saloni Jain <[email protected]>
----
- decompress.c | 2 +-
- 1 file changed, 1 insertion(+), 1 deletion(-)
-
-diff --git a/decompress.c b/decompress.c
-index ab6a624..f3db91d 100644
---- a/decompress.c
-+++ b/decompress.c
-@@ -287,7 +287,7 @@ Int32 BZ2_decompress ( DState* s )
-       GET_BITS(BZ_X_SELECTOR_1, nGroups, 3);
-       if (nGroups < 2 || nGroups > 6) RETURN(BZ_DATA_ERROR);
-       GET_BITS(BZ_X_SELECTOR_2, nSelectors, 15);
--      if (nSelectors < 1) RETURN(BZ_DATA_ERROR);
-+      if (nSelectors < 1 || nSelectors > BZ_MAX_SELECTORS) RETURN(BZ_DATA_ERROR);
-       for (i = 0; i < nSelectors; i++) {
-          j = 0;
-          while (True) {
---
-2.22.0
Signature · 1 line
2.17.1
#2 lore raw

PATCH 2/29 rsync: whitelist CVE-2017-16548

In reply to Armin Kuster

From: Lee Chee Yang <[email protected]>

patch for this CVE applies to v3.1.3pre1 not for v3.1.3. patch already in v3.1.3.
see
https://git.samba.org/rsync.git/?p=rsync.git;a=commitdiff;h=47a63d90e71d3e19e0e96052bb8c6b9cb140ecc1;hp=bc112b0e7feece62ce98708092306639a8a53cce

(From OE-Core rev: 1e2739c821312527010fb0afbde5a20cd3f03d24)

---
 meta/recipes-devtools/rsync/rsync_3.1.3.bb | 3 +++
 1 file changed, 3 insertions(+)
meta/recipes-devtools/rsync/rsync_3.1.3.bb +3 -0
index ffb1d061c0..152ff02a25 100644
@@ -20,6 +20,9 @@ SRC_URI = "https://download.samba.org/pub/${BPN}/src/${BP}.tar.gz \
 SRC_URI[md5sum] = "1581a588fde9d89f6bc6201e8129afaf"
 SRC_URI[sha256sum] = "55cc554efec5fdaad70de921cd5a5eeb6c29a95524c715f3bbf849235b0800c0"
 
+# -16548 required for v3.1.3pre1. Already in v3.1.3.
+CVE_CHECK_WHITELIST += " CVE-2017-16548 "
+
 inherit autotools
 
 PACKAGECONFIG ??= "acl attr \
Signature · 1 line
2.17.1
#3 lore raw

PATCH 3/29 curl: fix CVE-2019-15601

In reply to Armin Kuster

From: Anuj Mittal <[email protected]>

---
 .../curl/curl/CVE-2019-15601.patch            | 46 +++++++++++++++++++
 meta/recipes-support/curl/curl_7.66.0.bb      |  1 +
 2 files changed, 47 insertions(+)
 create mode 100644 meta/recipes-support/curl/curl/CVE-2019-15601.patch
meta/recipes-support/curl/curl/CVE-2019-15601.patch new file +46 -0
new file mode 100644
index 0000000000..7bfaae7b21
@@ -0,0 +1,46 @@
+Upstream-Status: Backport [https://github.com/curl/curl/commit/1b71bc532bde8621fd3260843f8197182a467ff2]
+CVE: CVE-2019-15601
+Signed-off-by: Anuj Mittal <[email protected]>
+
+From 1b71bc532bde8621fd3260843f8197182a467ff2 Mon Sep 17 00:00:00 2001
+From: Daniel Stenberg <[email protected]>
+Date: Thu, 7 Nov 2019 10:13:01 +0100
+Subject: [PATCH] file: on Windows, refuse paths that start with \\
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+... as that might cause an unexpected SMB connection to a given host
+name.
+
+Reported-by: Fernando Muñoz
+CVE-2019-15601
+Bug: https://curl.haxx.se/docs/CVE-2019-15601.html
+---
+ lib/file.c | 6 ++++--
+ 1 file changed, 4 insertions(+), 2 deletions(-)
+
+diff --git a/lib/file.c b/lib/file.c
+index d349cd9241..166931d7f1 100644
+--- a/lib/file.c
++++ b/lib/file.c
+@@ -136,7 +136,7 @@ static CURLcode file_connect(struct connectdata *conn, bool *done)
+   struct Curl_easy *data = conn->data;
+   char *real_path;
+   struct FILEPROTO *file = data->req.protop;
+-  int fd;
++  int fd = -1;
+ #ifdef DOS_FILESYSTEM
+   size_t i;
+   char *actual_path;
+@@ -181,7 +181,9 @@ static CURLcode file_connect(struct connectdata *conn, bool *done)
+       return CURLE_URL_MALFORMAT;
+     }
+ 
+-  fd = open_readonly(actual_path, O_RDONLY|O_BINARY);
++  if(strncmp("\\\\", actual_path, 2))
++    /* refuse to open path that starts with two backslashes */
++    fd = open_readonly(actual_path, O_RDONLY|O_BINARY);
+   file->path = actual_path;
+ #else
+   if(memchr(real_path, 0, real_path_len)) {
meta/recipes-support/curl/curl_7.66.0.bb +1 -0
index d1975f2460..a54e0536e9 100644
@@ -7,6 +7,7 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=be5d9e1419c4363f4b32037a2d3b7ffa"
 
 SRC_URI = "http://curl.haxx.se/download/curl-${PV}.tar.bz2 \
            file://0001-replace-krb5-config-with-pkg-config.patch \
+           file://CVE-2019-15601.patch \
 "
 
 SRC_URI[md5sum] = "c238aa394e3aa47ca4fcb0491774149f"
Signature · 1 line
2.17.1
#4 lore raw

PATCH 4/29 cpio: fix CVE-2019-14866

In reply to Armin Kuster

From: Anuj Mittal <[email protected]>

---
 .../cpio/cpio-2.12/CVE-2019-14866.patch       | 316 ++++++++++++++++++
 meta/recipes-extended/cpio/cpio_2.12.bb       |   1 +
 2 files changed, 317 insertions(+)
 create mode 100644 meta/recipes-extended/cpio/cpio-2.12/CVE-2019-14866.patch
meta/recipes-extended/cpio/cpio-2.12/CVE-2019-14866.patch new file +316 -0
new file mode 100644
index 0000000000..5d587fc832
@@ -0,0 +1,316 @@
+CVE: CVE-2019-14866
+Upstream-Status: Backport [https://git.savannah.gnu.org/cgit/cpio.git/commit/?id=7554e3e42cd72f6f8304410c47fe6f8918e9bfd7]
+Signed-off-by: Anuj Mittal <[email protected]>
+
+From a052401293e45a13cded5959b258204dae6d0af5 Mon Sep 17 00:00:00 2001
+From: Sergey Poznyakoff <[email protected]>
+Date: Sun, 3 Nov 2019 23:59:39 +0200
+Subject: [PATCH] Fix CVE-2019-14866
+
+* src/copyout.c (to_ascii): Additional argument nul controls whether
+to add the terminating nul character.
+(field_width_error): Improve diagnostics: print the actual and the
+maximum allowed field value.
+* src/extern.h (to_ascii, field_width_error): New prototypes.
+* src/tar.c (to_oct): Remove.
+(to_oct_or_error): New function.
+(TO_OCT): New macro.
+(write_out_tar_header): Use TO_OCT and to_ascii. Return 0 on
+success, 1 on error.
+---
+ src/copyout.c | 49 ++++++++++++++++++++++--------------
+ src/extern.h  | 15 +++++++++--
+ src/tar.c     | 69 ++++++++++++++++++++++++---------------------------
+ 3 files changed, 75 insertions(+), 58 deletions(-)
+
+diff --git a/src/copyout.c b/src/copyout.c
+index 1f0987a..1ae5477 100644
+--- a/src/copyout.c
++++ b/src/copyout.c
+@@ -269,26 +269,32 @@ writeout_final_defers (int out_des)
+    so it should be moved to paxutils too.
+    Allowed values for logbase are: 1 (binary), 2, 3 (octal), 4 (hex) */
+ int
+-to_ascii (char *where, uintmax_t v, size_t digits, unsigned logbase)
++to_ascii (char *where, uintmax_t v, size_t digits, unsigned logbase, bool nul)
+ {
+   static char codetab[] = "0123456789ABCDEF";
+-  int i = digits;
+-  
+-  do
++
++  if (nul)
++    where[--digits] = 0;
++  while (digits > 0)
+     {
+-      where[--i] = codetab[(v & ((1 << logbase) - 1))];
++      where[--digits] = codetab[(v & ((1 << logbase) - 1))];
+       v >>= logbase;
+     }
+-  while (i);
+ 
+   return v != 0;
+ }
+ 
+-static void
+-field_width_error (const char *filename, const char *fieldname)
++void
++field_width_error (const char *filename, const char *fieldname,
++		   uintmax_t value, size_t width, bool nul)
+ {
+-  error (0, 0, _("%s: field width not sufficient for storing %s"),
+-	 filename, fieldname);
++  char valbuf[UINTMAX_STRSIZE_BOUND + 1];
++  char maxbuf[UINTMAX_STRSIZE_BOUND + 1];
++  error (0, 0, _("%s: value %s %s out of allowed range 0..%s"),
++	 filename, fieldname,
++	 STRINGIFY_BIGINT (value, valbuf),
++	 STRINGIFY_BIGINT (MAX_VAL_WITH_DIGITS (width - nul, LG_8),
++			   maxbuf));
+ }
+ 
+ static void
+@@ -303,7 +309,7 @@ to_ascii_or_warn (char *where, uintmax_t n, size_t digits,
+ 		  unsigned logbase,
+ 		  const char *filename, const char *fieldname)
+ {
+-  if (to_ascii (where, n, digits, logbase))
++  if (to_ascii (where, n, digits, logbase, false))
+     field_width_warning (filename, fieldname);
+ }    
+ 
+@@ -312,9 +318,9 @@ to_ascii_or_error (char *where, uintmax_t n, size_t digits,
+ 		   unsigned logbase,
+ 		   const char *filename, const char *fieldname)
+ {
+-  if (to_ascii (where, n, digits, logbase))
++  if (to_ascii (where, n, digits, logbase, false))
+     {
+-      field_width_error (filename, fieldname);
++      field_width_error (filename, fieldname, n, digits, false);
+       return 1;
+     }
+   return 0;
+@@ -371,7 +377,7 @@ write_out_new_ascii_header (const char *magic_string,
+ 			 _("name size")))
+     return 1;
+   p += 8;
+-  to_ascii (p, file_hdr->c_chksum & 0xffffffff, 8, LG_16);
++  to_ascii (p, file_hdr->c_chksum & 0xffffffff, 8, LG_16, false);
+ 
+   tape_buffered_write (ascii_header, out_des, sizeof ascii_header);
+ 
+@@ -388,7 +394,7 @@ write_out_old_ascii_header (dev_t dev, dev_t rdev,
+   char ascii_header[76];
+   char *p = ascii_header;
+   
+-  to_ascii (p, file_hdr->c_magic, 6, LG_8);
++  to_ascii (p, file_hdr->c_magic, 6, LG_8, false);
+   p += 6;
+   to_ascii_or_warn (p, dev, 6, LG_8, file_hdr->c_name, _("device number"));
+   p += 6;
+@@ -492,7 +498,10 @@ write_out_binary_header (dev_t rdev,
+   short_hdr.c_namesize = file_hdr->c_namesize & 0xFFFF;
+   if (short_hdr.c_namesize != file_hdr->c_namesize)
+     {
+-      field_width_error (file_hdr->c_name, _("name size"));
++      char maxbuf[UINTMAX_STRSIZE_BOUND + 1];
++      error (0, 0, _("%s: value %s %s out of allowed range 0..%u"),
++	     file_hdr->c_name, _("name size"),
++	     STRINGIFY_BIGINT (file_hdr->c_namesize, maxbuf), 0xFFFFu);
+       return 1;
+     }
+ 		      
+@@ -502,7 +511,10 @@ write_out_binary_header (dev_t rdev,
+   if (((off_t)short_hdr.c_filesizes[0] << 16) + short_hdr.c_filesizes[1]
+        != file_hdr->c_filesize)
+     {
+-      field_width_error (file_hdr->c_name, _("file size"));
++      char maxbuf[UINTMAX_STRSIZE_BOUND + 1];
++      error (0, 0, _("%s: value %s %s out of allowed range 0..%lu"),
++	     file_hdr->c_name, _("file size"),
++	     STRINGIFY_BIGINT (file_hdr->c_namesize, maxbuf), 0xFFFFFFFFlu);
+       return 1;
+     }
+ 		      
+@@ -552,8 +564,7 @@ write_out_header (struct cpio_file_stat *file_hdr, int out_des)
+ 	  error (0, 0, _("%s: file name too long"), file_hdr->c_name);
+ 	  return 1;
+ 	}
+-      write_out_tar_header (file_hdr, out_des); /* FIXME: No error checking */
+-      return 0;
++      return write_out_tar_header (file_hdr, out_des);
+ 
+     case arf_binary:
+       return write_out_binary_header (makedev (file_hdr->c_rdev_maj,
+diff --git a/src/extern.h b/src/extern.h
+index e27d662..f9ef56a 100644
+--- a/src/extern.h
++++ b/src/extern.h
+@@ -117,6 +117,10 @@ void print_name_with_quoting (char *p);
+ /* copyout.c */
+ int write_out_header (struct cpio_file_stat *file_hdr, int out_des);
+ void process_copy_out (void);
++int to_ascii (char *where, uintmax_t v, size_t digits, unsigned logbase,
++	      bool nul);
++void field_width_error (const char *filename, const char *fieldname,
++			uintmax_t value, size_t width, bool nul);
+ 
+ /* copypass.c */
+ void process_copy_pass (void);
+@@ -145,7 +149,7 @@ int make_path (char *argpath, uid_t owner, gid_t group,
+ 	       const char *verbose_fmt_string);
+ 
+ /* tar.c */
+-void write_out_tar_header (struct cpio_file_stat *file_hdr, int out_des);
++int write_out_tar_header (struct cpio_file_stat *file_hdr, int out_des);
+ int null_block (long *block, int size);
+ void read_in_tar_header (struct cpio_file_stat *file_hdr, int in_des);
+ int otoa (char *s, unsigned long *n);
+@@ -204,9 +208,16 @@ void cpio_safer_name_suffix (char *name, bool link_target,
+ int cpio_create_dir (struct cpio_file_stat *file_hdr, int existing_dir);
+ void change_dir (void);
+ 
+-/* FIXME: These two defines should be defined in paxutils */
++/* FIXME: The following three should be defined in paxutils */
+ #define LG_8  3
+ #define LG_16 4
++/* The maximum uintmax_t value that can be represented with DIGITS digits,
++   assuming that each digit is BITS_PER_DIGIT wide.  */
++#define MAX_VAL_WITH_DIGITS(digits, bits_per_digit) \
++   ((digits) * (bits_per_digit) < sizeof (uintmax_t) * CHAR_BIT \
++    ? ((uintmax_t) 1 << ((digits) * (bits_per_digit))) - 1 \
++    : (uintmax_t) -1)
++
+ 
+ uintmax_t from_ascii (char const *where, size_t digs, unsigned logbase);
+ 
+diff --git a/src/tar.c b/src/tar.c
+index a2ce171..ef58027 100644
+--- a/src/tar.c
++++ b/src/tar.c
+@@ -79,36 +79,17 @@ stash_tar_filename (char *prefix, char *filename)
+   return hold_tar_filename;
+ }
+ 
+-/* Convert a number into a string of octal digits.
+-   Convert long VALUE into a DIGITS-digit field at WHERE,
+-   including a trailing space and room for a NUL.  DIGITS==3 means
+-   1 digit, a space, and room for a NUL.
+-
+-   We assume the trailing NUL is already there and don't fill it in.
+-   This fact is used by start_header and finish_header, so don't change it!
+-
+-   This is be equivalent to:
+-   sprintf (where, "%*lo ", digits - 2, value);
+-   except that sprintf fills in the trailing NUL and we don't.  */
+-
+-static void
+-to_oct (register long value, register int digits, register char *where)
++static int
++to_oct_or_error (uintmax_t value, size_t digits, char *where, char const *field,
++		 char const *file)
+ {
+-  --digits;			/* Leave the trailing NUL slot alone.  */
+-
+-  /* Produce the digits -- at least one.  */
+-  do
++  if (to_ascii (where, value, digits, LG_8, true))
+     {
+-      where[--digits] = '0' + (char) (value & 7); /* One octal digit.  */
+-      value >>= 3;
++      field_width_error (file, field, value, digits, true);
++      return 1;
+     }
+-  while (digits > 0 && value != 0);
+-
+-  /* Add leading zeroes, if necessary.  */
+-  while (digits > 0)
+-    where[--digits] = '0';
++  return 0;
+ }
+-
+ 
+ 
+ /* Compute and return a checksum for TAR_HDR,
+@@ -134,10 +115,22 @@ tar_checksum (struct tar_header *tar_hdr)
+   return sum;
+ }
+ 
++#define TO_OCT(file_hdr, c_fld, digits, tar_hdr, tar_field) \
++  do							    \
++    {							    \
++       if (to_oct_or_error (file_hdr -> c_fld,		    \
++			    digits,			    \
++			    tar_hdr -> tar_field,	    \
++			    #tar_field,			    \
++			    file_hdr->c_name))		    \
++	 return 1;					    \
++    }							    \
++  while (0)
++
+ /* Write out header FILE_HDR, including the file name, to file
+    descriptor OUT_DES.  */
+ 
+-void
++int
+ write_out_tar_header (struct cpio_file_stat *file_hdr, int out_des)
+ {
+   int name_len;
+@@ -166,11 +159,11 @@ write_out_tar_header (struct cpio_file_stat *file_hdr, int out_des)
+ 
+   /* Ustar standard (POSIX.1-1988) requires the mode to contain only 3 octal
+      digits */
+-  to_oct (file_hdr->c_mode & MODE_ALL, 8, tar_hdr->mode);
+-  to_oct (file_hdr->c_uid, 8, tar_hdr->uid);
+-  to_oct (file_hdr->c_gid, 8, tar_hdr->gid);
+-  to_oct (file_hdr->c_filesize, 12, tar_hdr->size);
+-  to_oct (file_hdr->c_mtime, 12, tar_hdr->mtime);
++  TO_OCT (file_hdr, c_mode & MODE_ALL, 8, tar_hdr, mode);
++  TO_OCT (file_hdr, c_uid, 8, tar_hdr, uid);
++  TO_OCT (file_hdr, c_gid, 8, tar_hdr, gid);
++  TO_OCT (file_hdr, c_filesize, 12, tar_hdr, size);
++  TO_OCT (file_hdr, c_mtime, 12, tar_hdr, mtime);
+ 
+   switch (file_hdr->c_mode & CP_IFMT)
+     {
+@@ -182,7 +175,7 @@ write_out_tar_header (struct cpio_file_stat *file_hdr, int out_des)
+ 	  strncpy (tar_hdr->linkname, file_hdr->c_tar_linkname,
+ 		   TARLINKNAMESIZE);
+ 	  tar_hdr->typeflag = LNKTYPE;
+-	  to_oct (0, 12, tar_hdr->size);
++	  to_ascii (tar_hdr->size, 0, 12, LG_8, true);
+ 	}
+       else
+ 	tar_hdr->typeflag = REGTYPE;
+@@ -208,7 +201,7 @@ write_out_tar_header (struct cpio_file_stat *file_hdr, int out_des)
+ 	 than TARLINKNAMESIZE.  */
+       strncpy (tar_hdr->linkname, file_hdr->c_tar_linkname,
+ 	       TARLINKNAMESIZE);
+-      to_oct (0, 12, tar_hdr->size);
++      to_ascii (tar_hdr->size, 0, 12, LG_8, true);
+       break;
+ #endif /* CP_IFLNK */
+     }
+@@ -227,13 +220,15 @@ write_out_tar_header (struct cpio_file_stat *file_hdr, int out_des)
+       if (name)
+ 	strcpy (tar_hdr->gname, name);
+ 
+-      to_oct (file_hdr->c_rdev_maj, 8, tar_hdr->devmajor);
+-      to_oct (file_hdr->c_rdev_min, 8, tar_hdr->devminor);
++      TO_OCT (file_hdr, c_rdev_maj, 8, tar_hdr, devmajor);
++      TO_OCT (file_hdr, c_rdev_min, 8, tar_hdr, devminor);
+     }
+ 
+-  to_oct (tar_checksum (tar_hdr), 8, tar_hdr->chksum);
++  to_ascii (tar_hdr->chksum, tar_checksum (tar_hdr), 8, LG_8, true);
+ 
+   tape_buffered_write ((char *) &tar_rec, out_des, TARRECORDSIZE);
++
++  return 0;
+ }
+ 
+ /* Return nonzero iff all the bytes in BLOCK are NUL.
+-- 
+2.24.1
+
meta/recipes-extended/cpio/cpio_2.12.bb +1 -0
index 3713bf0b1f..5abe494ebc 100644
@@ -11,6 +11,7 @@ SRC_URI = "${GNU_MIRROR}/cpio/cpio-${PV}.tar.gz \
            file://0001-Fix-CVE-2015-1197.patch \
            file://0001-CVE-2016-2037-1-byte-out-of-bounds-write.patch \
            file://0001-Fix-segfault-with-append.patch \
+           file://CVE-2019-14866.patch \
            "
 
 SRC_URI[md5sum] = "fc207561a86b63862eea4b8300313e86"
Signature · 1 line
2.17.1
#5 lore raw

PATCH 5/29 devtool/standard.py: Allow recipe to disable menuconfig logic

In reply to Armin Kuster

From: Tom Hochstein <[email protected]>

u-boot.inc supports u-boot recipes with or without menuconfig [1]. However, running devtool on a u-boot recipe that does not support menuconfig results in an error:

cp: cannot stat '/home/r60874/upstream/fsl-xwayland/tmp/work/imx8mmevk-fsl-linux/u-boot-imx/2018.03-r0/u-boot-imx-2018.03//.config': No such file or directory

The problem is the devtool logic assumes that any recipe with a do_menuconfig task will generate a .config in do_configure().

Fix the problem by removing the assumption with a flag that the recipe can control, like this:

do_configure() {
if [ menuconfig-supported ]; then
...
else
DEVTOOL_DISABLE_MENUCONFIG=true
fi
}

[1] https://github.com/openembedded/openembedded-core/commit/11278e3b2c75be80645b9841763a97dbb35daadc

---
 scripts/lib/devtool/standard.py | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
scripts/lib/devtool/standard.py +4 -2
index 60c9a046f9..b43c725cf8 100644
@@ -940,8 +940,10 @@ def modify(args, config, basepath, workspace):
                         '}\n')
             if rd.getVarFlag('do_menuconfig','task'):
                 f.write('\ndo_configure_append() {\n'
-                '    cp ${B}/.config ${S}/.config.baseline\n'
-                '    ln -sfT ${B}/.config ${S}/.config.new\n'
+                '    if [ ! ${DEVTOOL_DISABLE_MENUCONFIG} ]; then\n'
+                '        cp ${B}/.config ${S}/.config.baseline\n'
+                '        ln -sfT ${B}/.config ${S}/.config.new\n'
+                '    fi\n'
                 '}\n')
             if initial_rev:
                 f.write('\n# initial_rev: %s\n' % initial_rev)
Signature · 1 line
2.17.1
#6 lore raw

PATCH 6/29 gcc-9.2: fix bug #91102 'aarch64 ICE on Linux kernel with -Os'

In reply to Armin Kuster

From: Taras Kondratiuk via Openembedded-core <[email protected]>

Linux kernel compilation for aarch64 triggers ICE if CONFIG_CC_OPTIMIZE_FOR_SIZE=y.

The rootcause is GCC bug #91102 'aarch64 ICE on Linux kernel with -Os'. Apply the fix to 9.2.

(From OE-Core rev: 14f34d32bfdaa752f5043e62750d2e7b92c4b419)

---
 meta/recipes-devtools/gcc/gcc-9.2.inc         |  1 +
 ...02-aarch64-ICE-on-Linux-kernel-with-.patch | 95 +++++++++++++++++++
 2 files changed, 96 insertions(+)
 create mode 100644 meta/recipes-devtools/gcc/gcc-9.2/re-PR-target-91102-aarch64-ICE-on-Linux-kernel-with-.patch
meta/recipes-devtools/gcc/gcc-9.2.inc +1 -0
index c6395998d5..4f068231f3 100644
@@ -68,6 +68,7 @@ SRC_URI = "\
 	   file://CVE-2019-15847_1.patch \
 	   file://CVE-2019-15847_2.patch \
 	   file://CVE-2019-15847_3.patch \
+           file://re-PR-target-91102-aarch64-ICE-on-Linux-kernel-with-.patch \
 "
 S = "${TMPDIR}/work-shared/gcc-${PV}-${PR}/gcc-${PV}"
 SRC_URI[md5sum] = "3818ad8600447f05349098232c2ddc78"
meta/recipes-devtools/gcc/gcc-9.2/re-PR-target-91102-aarch64-ICE-on-Linux-kernel-with-.patch new file +95 -0

95 lines not on this page · raw message

Signature · 1 line
2.17.1
#7 lore raw

PATCH 7/29 opkg-utils: upgrade to version 0.4.2

In reply to Armin Kuster

From: Alejandro del Castillo <[email protected]>

- Drop 00001-Switch-all-scripts-to-use-Python-3.x.patch
- Drop 00001-opkg-build-clamp-mtimes-to-SOURCE_DATE_EPOCH.patch
- Drop pipefail.patch

(From OE-Core rev: bf51a4a1312562cc9b5944b7dfccba0b3d11dc3c)

(From OE-Core rev: 1b71c28e1ca4fddc0f3c340ea4bcd76854ef620c)

---
 ...Switch-all-scripts-to-use-Python-3.x.patch | 113 ------------------
 ...ld-clamp-mtimes-to-SOURCE_DATE_EPOCH.patch |  44 -------
 .../opkg-utils/opkg-utils/pipefail.patch      |  31 -----
 ...pkg-utils_0.4.1.bb => opkg-utils_0.4.2.bb} |  12 +-
 4 files changed, 4 insertions(+), 196 deletions(-)
 delete mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/0001-Switch-all-scripts-to-use-Python-3.x.patch
 delete mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/0001-opkg-build-clamp-mtimes-to-SOURCE_DATE_EPOCH.patch
 delete mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/pipefail.patch
 rename meta/recipes-devtools/opkg-utils/{opkg-utils_0.4.1.bb => opkg-utils_0.4.2.bb} (83%)
meta/recipes-devtools/opkg-utils/opkg-utils/0001-Switch-all-scripts-to-use-Python-3.x.patch deleted +0 -113
deleted file mode 100644
index 691ed50c2b..0000000000
@@ -1,113 +0,0 @@
-From d42b23f4fb5d6bd58e92e995fe5befc76efbae0c Mon Sep 17 00:00:00 2001
-From: Alexander Kanavin <[email protected]>
-Date: Thu, 27 Apr 2017 15:47:58 +0300
-Subject: [PATCH] Switch all scripts to use Python 3.x
-
-Upstream-Status: Pending
-Signed-off-by: Alexander Kanavin <[email protected]>
----
- makePackage          | 2 +-
- opkg-compare-indexes | 2 +-
- opkg-graph-deps      | 2 +-
- opkg-list-fields     | 2 +-
- opkg-make-index      | 2 +-
- opkg-show-deps       | 2 +-
- opkg-unbuild         | 2 +-
- opkg-update-index    | 2 +-
- opkg.py              | 2 +-
- 9 files changed, 9 insertions(+), 9 deletions(-)
-
-diff --git a/makePackage b/makePackage
-index 4bdfc56..02124dd 100755
---- a/makePackage
-+++ b/makePackage
-@@ -1,4 +1,4 @@
--#!/usr/bin/python
-+#!/usr/bin/env python3
- 
- # The general algorithm this program follows goes like this:
- #   Run tar to extract control from control.tar.gz from the package.
-diff --git a/opkg-compare-indexes b/opkg-compare-indexes
-index b60d20a..80c1263 100755
---- a/opkg-compare-indexes
-+++ b/opkg-compare-indexes
-@@ -1,4 +1,4 @@
--#!/usr/bin/env python
-+#!/usr/bin/env python3
- from __future__ import absolute_import
- from __future__ import print_function
- 
-diff --git a/opkg-graph-deps b/opkg-graph-deps
-index 6653fd5..f1e376a 100755
---- a/opkg-graph-deps
-+++ b/opkg-graph-deps
-@@ -1,4 +1,4 @@
--#!/usr/bin/env python
-+#!/usr/bin/env python3
- from __future__ import absolute_import
- from __future__ import print_function
- 
-diff --git a/opkg-list-fields b/opkg-list-fields
-index c14a90f..24f7955 100755
---- a/opkg-list-fields
-+++ b/opkg-list-fields
-@@ -1,4 +1,4 @@
--#!/usr/bin/env python
-+#!/usr/bin/env python3
- from __future__ import absolute_import
- from __future__ import print_function
- 
-diff --git a/opkg-make-index b/opkg-make-index
-index 3f757f6..2988f9f 100755
---- a/opkg-make-index
-+++ b/opkg-make-index
-@@ -1,4 +1,4 @@
--#!/usr/bin/env python
-+#!/usr/bin/env python3
- """
-    Utility to create opkg compatible indexes
- """
- 
-diff --git a/opkg-show-deps b/opkg-show-deps
-index 153f21e..4e18b4f 100755
---- a/opkg-show-deps
-+++ b/opkg-show-deps
-@@ -1,4 +1,4 @@
--#!/usr/bin/env python
-+#!/usr/bin/env python3
- from __future__ import absolute_import
- from __future__ import print_function
- 
-diff --git a/opkg-unbuild b/opkg-unbuild
-index 4f36bec..57642c9 100755
---- a/opkg-unbuild
-+++ b/opkg-unbuild
-@@ -1,4 +1,4 @@
--#!/usr/bin/env python
-+#!/usr/bin/env python3
- from __future__ import absolute_import
- from __future__ import print_function
- 
-diff --git a/opkg-update-index b/opkg-update-index
-index 341c1c2..7bff8a1 100755
---- a/opkg-update-index
-+++ b/opkg-update-index
-@@ -1,4 +1,4 @@
--#!/usr/bin/env python
-+#!/usr/bin/env python3
- from __future__ import absolute_import
- 
- import sys, os
-diff --git a/opkg.py b/opkg.py
-index 2ecac8a..7e64de4 100644
---- a/opkg.py
-+++ b/opkg.py
-@@ -1,4 +1,4 @@
--#!/usr/bin/env python
-+#!/usr/bin/env python3
- #   Copyright (C) 2001 Alexander S. Guy <[email protected]>
- #                      Andern Research Labs
- #
--- 
-2.11.0
-
meta/recipes-devtools/opkg-utils/opkg-utils/0001-opkg-build-clamp-mtimes-to-SOURCE_DATE_EPOCH.patch deleted +0 -44

44 lines not on this page · raw message

meta/recipes-devtools/opkg-utils/opkg-utils/pipefail.patch deleted +0 -31
deleted file mode 100644
index 55ddcc1fd2..0000000000
@@ -1,31 +0,0 @@
-We need opkg-build to fail if for example the tar command is passed invalid 
-options. Without this, we see silently created empty packaged where data.tar
-is zero bytes in size. This creates hard to debug problems.
-
-An example is when reproducible builds are enabled and run on old hosts like
-centos7 which has tar < 1.28:
-
-Subprocess output:tar: unrecognized option '--clamp-mtime'
-Try `tar --help' or `tar --usage' for more information.
-
-Upstream-Status: Pending
-Signed-off-by: Richard Purdie <[email protected]>
-
-Index: opkg-utils-0.4.1/opkg-build
-===================================================================
---- opkg-utils-0.4.1.orig/opkg-build
-+++ opkg-utils-0.4.1/opkg-build
-@@ -1,4 +1,4 @@
--#!/bin/sh
-+#!/bin/bash
- 
- : <<=cut
- =head1 NAME
-@@ -12,6 +12,7 @@ opkg-build - construct an .opk from a di
- #   Updated to work on Familiar Pre0.7rc1, with busybox tar.
- #   Note it Requires: binutils-ar (since the busybox ar can't create)
- set -e
-+set -o pipefail
- 
- version=1.0
- 
meta/recipes-devtools/opkg-utils/opkg-utils_0.4.2.bb renamed from meta/recipes-devtools/opkg-utils/opkg-utils_0.4.1.bb +4 -8
similarity index 83%
rename from meta/recipes-devtools/opkg-utils/opkg-utils_0.4.1.bb
rename to meta/recipes-devtools/opkg-utils/opkg-utils_0.4.2.bb
index eb6c7a3a6a..6495726500 100644
@@ -4,19 +4,15 @@ SECTION = "base"
 HOMEPAGE = "http://git.yoctoproject.org/cgit/cgit.cgi/opkg-utils"
 LICENSE = "GPLv2+"
 LIC_FILES_CHKSUM = "file://COPYING;md5=94d55d512a9ba36caa9b7df079bae19f \
-                    file://opkg.py;beginline=2;endline=18;md5=63ce9e6bcc445181cd9e4baf4b4ccc35"
+                    file://opkg.py;beginline=2;endline=18;md5=ffa11ff3c15eb31c6a7ceaa00cc9f986"
 PROVIDES += "${@bb.utils.contains('PACKAGECONFIG', 'update-alternatives', 'virtual/update-alternatives', '', d)}"
 
-SRC_URI = "http://git.yoctoproject.org/cgit/cgit.cgi/${BPN}/snapshot/${BPN}-${PV}.tar.gz \
-           file://0001-Switch-all-scripts-to-use-Python-3.x.patch \
-           file://0001-opkg-build-clamp-mtimes-to-SOURCE_DATE_EPOCH.patch \
-           file://pipefail.patch \
+SRC_URI = "http://git.yoctoproject.org/cgit/cgit.cgi/${BPN}/snapshot/${BPN}-${PV}.tar.gz \ 
 "
 UPSTREAM_CHECK_URI = "http://git.yoctoproject.org/cgit/cgit.cgi/opkg-utils/refs/"
 
-
-SRC_URI[md5sum] = "8c140f835b694a0c27cfb23d2426a02b"
-SRC_URI[sha256sum] = "9ea9efdd9fe13661ad251e3a2860c1c93045adcfaa6659c3e86d9748ecda3b6e"
+SRC_URI[md5sum] = "cc210650644fcb9bba06ad5ec95a63ec"
+SRC_URI[sha256sum] = "5929ad87d541789e0b82d626db01a1201ac48df6f49f2262fcfb86cf815e5d6c"
 
 TARGET_CC_ARCH += "${LDFLAGS}"
 
Signature · 1 line
2.17.1
#8 lore raw

PATCH 8/29 opkg-utils: Fix reproducibility issues in opkg-build

In reply to Armin Kuster

From: Richard Purdie <[email protected]>

There is a sorting problem with opkg-build where the ipk generated is depending upon the order of files on disk. The reason is the --sort option to tar only influences the orders of files tar reads, not those passed by the -T option.

Add in a sort call to resolve this issue. To ensure consistent sorting we also need to force to a specific locale (C) else the results are still not deterministic.

(From OE-Core rev: a9b8287984c63420e10329a69f7ac5125f1687f8)

(From OE-Core rev: b577a6d923042cfc04e67d470e0987488ea61412)

---
 .../opkg-utils/fix-reproducibility.patch      | 32 +++++++++++++++++++
 .../opkg-utils/opkg-utils_0.4.2.bb            |  1 +
 2 files changed, 33 insertions(+)
 create mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/fix-reproducibility.patch
meta/recipes-devtools/opkg-utils/opkg-utils/fix-reproducibility.patch new file +32 -0
new file mode 100644
index 0000000000..945979bc8a
@@ -0,0 +1,32 @@
+Fix reproducibility issues in opkg-build
+
+There is a sorting problem with opkg-build where the ipk generated is depending
+upon the order of files on disk. The reason is the --sort option to tar only
+influences the orders of files tar reads, not those passed by the -T option.
+
+Add in a sort call to resolve this issue. To ensure consistent sorting we
+also need to force to a specific locale (C) else the results are still not
+deterministic.
+
+RP 2020/2/5
+
+Upstream-Status: Submitted [https://groups.google.com/forum/#!topic/opkg-devel/YttZ73NLrYQ]
+Signed-off-by: Richard Purdie <[email protected]>
+
+Index: opkg-utils-0.4.2/opkg-build
+===================================================================
+--- opkg-utils-0.4.2.orig/opkg-build
++++ opkg-utils-0.4.2/opkg-build
+@@ -305,8 +305,10 @@ if [ ! -z "$SOURCE_DATE_EPOCH"  ]; then
+     mtime_args="--mtime=@$build_date --clamp-mtime"
+ fi
+ 
+-( cd $pkg_dir/$CONTROL && find . -type f > $tmp_dir/control_list )
+-( cd $pkg_dir && find . -path ./$CONTROL -prune -o -path . -o -print  > $tmp_dir/file_list )
++export LANG=C
++export LC_ALL=C
++( cd $pkg_dir/$CONTROL && find . -type f | sort > $tmp_dir/control_list )
++( cd $pkg_dir && find . -path ./$CONTROL -prune -o -path . -o -print  | sort > $tmp_dir/file_list )
+ ( cd $pkg_dir && tar $ogargs $tsortargs --no-recursion $mtime_args -c $tarformat -T $tmp_dir/file_list | $compressor $compressorargs > $tmp_dir/data.tar.$cext )
+ ( cd $pkg_dir/$CONTROL && tar $ogargs $tsortargs --no-recursion --mtime=@$build_date -c $tarformat -T $tmp_dir/control_list | gzip $zipargs > $tmp_dir/control.tar.gz )
+ rm $tmp_dir/file_list
meta/recipes-devtools/opkg-utils/opkg-utils_0.4.2.bb +1 -0
index 6495726500..042eec7e0e 100644
@@ -8,6 +8,7 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=94d55d512a9ba36caa9b7df079bae19f \
 PROVIDES += "${@bb.utils.contains('PACKAGECONFIG', 'update-alternatives', 'virtual/update-alternatives', '', d)}"
 
 SRC_URI = "http://git.yoctoproject.org/cgit/cgit.cgi/${BPN}/snapshot/${BPN}-${PV}.tar.gz \ 
+           file://fix-reproducibility.patch \
 "
 UPSTREAM_CHECK_URI = "http://git.yoctoproject.org/cgit/cgit.cgi/opkg-utils/refs/"
 
Signature · 1 line
2.17.1
#9 lore raw

PATCH 9/29 oeqa/reproducible: Improve test output and ensure deb+ipk compared

In reply to Armin Kuster

From: Richard Purdie <[email protected]>

Adding newline characters between the packages in the failure output massively improves readability.

Also ensure to output ipk failures when there are deb failures by calling self.fail() at the end, else sometimes only partial differences are returned.

(From OE-Core rev: 6e2e0480852177db75a6108d77c99c92c4e9950f)

(From OE-Core rev: 4d470f48f7fb5e05fba1ca3a59fb4f85d910026e)

---
 meta/lib/oeqa/selftest/cases/reproducible.py | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)
meta/lib/oeqa/selftest/cases/reproducible.py +7 -2
index a9110565a9..1b0b5bae70 100644
@@ -174,6 +174,8 @@ class ReproducibleTests(OESelftestTestCase):
         # NOTE: The temp directories from the reproducible build are purposely
         # kept after the build so it can be diffed for debugging.
 
+        fails = []
+
         for c in self.package_classes:
             with self.subTest(package_class=c):
                 package_class = 'package_' + c
@@ -197,6 +199,9 @@ class ReproducibleTests(OESelftestTestCase):
                         self.copy_file(d.test, '/'.join([save_dir, d.test]))
 
                 if result.missing or result.different:
-                    self.fail("The following %s packages are missing or different: %s" %
-                            (c, ' '.join(r.test for r in (result.missing + result.different))))
+                    fails.append("The following %s packages are missing or different: %s" %
+                            (c, '\n'.join(r.test for r in (result.missing + result.different))))
+
+        if fails:
+            self.fail('\n'.join(fails))
 
Signature · 1 line
2.17.1
#10 lore raw

PATCH 10/29 classes/reproducible_build: Read SDE file later

In reply to Armin Kuster

From: Joshua Watt <[email protected]>

Defers the resolution of the SOURCE_DATE_EPOCH until the variable needs to be actually realized with a value. The previous method of loading the value in anonymous python had issues because it could occur before other anonymous python functions that affect the location of the epoch file, such as when a recipe uses AUTOINC/AUTOREV or allarch.bbclass.

Also adds more logging to help diagnose issues in the future.

[YOCTO #13763]

(From OE-Core rev: b3313a10a3eb93f0a3710a35de0404fb49cd6202)

(From OE-Core rev: 10515e5f7e38edbc4430e2599062a9ce6fdb42a8)

---
 meta/classes/reproducible_build.bbclass | 40 +++++++++++++++++++------
 1 file changed, 31 insertions(+), 9 deletions(-)
meta/classes/reproducible_build.bbclass +31 -9
index 39b6e40cac..750eb950f2 100644
@@ -44,10 +44,12 @@ SDE_DEPLOYDIR = "${WORKDIR}/deploy-source-date-epoch"
 SSTATETASKS += "do_deploy_source_date_epoch"
 
 do_deploy_source_date_epoch () {
-    echo "Deploying SDE to ${SDE_DIR}."
     mkdir -p ${SDE_DEPLOYDIR}
     if [ -e ${SDE_FILE} ]; then
+        echo "Deploying SDE from ${SDE_FILE} -> ${SDE_DEPLOYDIR}."
         cp -p ${SDE_FILE} ${SDE_DEPLOYDIR}/__source_date_epoch.txt
+    else
+        echo "${SDE_FILE} not found!"
     fi
 }
 
@@ -56,7 +58,11 @@ python do_deploy_source_date_epoch_setscene () {
     bb.utils.mkdirhier(d.getVar('SDE_DIR'))
     sde_file = os.path.join(d.getVar('SDE_DEPLOYDIR'), '__source_date_epoch.txt')
     if os.path.exists(sde_file):
-        os.rename(sde_file, d.getVar('SDE_FILE'))
+        target = d.getVar('SDE_FILE')
+        bb.debug(1, "Moving setscene SDE file %s -> %s" % (sde_file, target))
+        os.rename(sde_file, target)
+    else:
+        bb.debug(1, "%s not found!" % sde_file)
 }
 
 do_deploy_source_date_epoch[dirs] = "${SDE_DEPLOYDIR}"
@@ -164,16 +170,32 @@ python do_create_source_date_epoch_stamp() {
         f.write(str(source_date_epoch))
 }
 
+def get_source_date_epoch_value(d):
+    cached = d.getVar('__CACHED_SOURCE_DATE_EPOCH')
+    if cached:
+        return cached
+
+    epochfile = d.getVar('SDE_FILE')
+    source_date_epoch = 0
+    if os.path.isfile(epochfile):
+        with open(epochfile, 'r') as f:
+            s = f.read()
+            try:
+                source_date_epoch = int(s)
+            except ValueError:
+                bb.warn("SOURCE_DATE_EPOCH value '%s' is invalid. Reverting to 0" % s)
+                source_date_epoch = 0
+        bb.debug(1, "SOURCE_DATE_EPOCH: %d" % source_date_epoch)
+    else:
+        bb.debug(1, "Cannot find %s. SOURCE_DATE_EPOCH will default to %d" % (epochfile, source_date_epoch))
+
+    d.setVar('__CACHED_SOURCE_DATE_EPOCH', str(source_date_epoch))
+    return str(source_date_epoch)
+
+export SOURCE_DATE_EPOCH ?= "${@get_source_date_epoch_value(d)}"
 BB_HASHBASE_WHITELIST += "SOURCE_DATE_EPOCH"
 
 python () {
     if d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1':
         d.appendVarFlag("do_unpack", "postfuncs", " do_create_source_date_epoch_stamp")
-        epochfile = d.getVar('SDE_FILE')
-        source_date_epoch = "0"
-        if os.path.isfile(epochfile):
-            with open(epochfile, 'r') as f:
-                source_date_epoch = f.read()
-            bb.debug(1, "SOURCE_DATE_EPOCH: %s" % source_date_epoch)
-        d.setVar('SOURCE_DATE_EPOCH', source_date_epoch)
 }
Signature · 1 line
2.17.1
#11 lore raw

PATCH 11/29 sudo: Set vardir deterministically

In reply to Armin Kuster

From: Richard Purdie <[email protected]>

Without setting this it will vary depending on which directories are present on the host.

[YOCTO #13775]

(From OE-Core rev: 39fe849b56d70689846262c31ab7c182c8443923)

(From OE-Core rev: 51274c9b195ad00dd6362f352ad9bbf7e9a3e098)

---
 meta/recipes-extended/sudo/sudo_1.8.27.bb | 1 +
 1 file changed, 1 insertion(+)
meta/recipes-extended/sudo/sudo_1.8.27.bb +1 -0
index 0a11a1b28f..ee3be95644 100644
@@ -19,6 +19,7 @@ EXTRA_OECONF += " \
              ac_cv_type_rsize_t=no \
              ${@bb.utils.contains('DISTRO_FEATURES', 'pam', '--with-pam', '--without-pam', d)} \
              ${@bb.utils.contains('DISTRO_FEATURES', 'systemd', '--enable-tmpfiles.d=${nonarch_libdir}/tmpfiles.d', '--disable-tmpfiles.d', d)} \
+             --with-vardir=/var/lib/sudo \
              "
 
 do_install_append () {
Signature · 1 line
2.17.1
#12 lore raw

PATCH 12/29 libxshmfence: Set shm directory deterministically

In reply to Armin Kuster

From: Richard Purdie <[email protected]>

Without setting this it will vary depending on which directories are present on the host.

[YOCTO #13778]

(From OE-Core rev: 6217c3a7201b34888aa2fbae8b6e490bea545574)

(From OE-Core rev: 10357b0bf3f6b34b54c0329e7e392ecf2c7043cd)

---
 meta/recipes-graphics/xorg-lib/libxshmfence_1.3.bb | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-graphics/xorg-lib/libxshmfence_1.3.bb +2 -0
index 85a48e4c58..cc45696530 100644
@@ -11,6 +11,8 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=47e508ca280fde97906eacb77892c3ac"
 
 DEPENDS += "virtual/libx11"
 
+EXTRA_OECONF += "--with-shared-memory-dir=/dev/shm"
+
 BBCLASSEXTEND = "native nativesdk"
 
 SRC_URI[md5sum] = "42dda8016943dc12aff2c03a036e0937"
Signature · 1 line
2.17.1
#13 lore raw

PATCH 13/29 mc: Fix build reproducibility

In reply to Armin Kuster

From: Joshua Watt <[email protected]>

Fixes some issues with reproducible builds. Adds a patch to allow the configure arguments to be omitted from the build and also explicitly setting some autoconf paths that were picking up hosttools.

(From OE-Core rev: f54d60ee8f15229aa515e168b9c7d248663b48fe)

(From OE-Core rev: 164d72b7a6ab20940f9a124beaf485be9ddc07ba)

---
 ...Add-option-to-control-configure-args.patch | 99 +++++++++++++++++++
 meta/recipes-extended/mc/mc_4.8.23.bb         |  5 +-
 2 files changed, 103 insertions(+), 1 deletion(-)
 create mode 100644 meta/recipes-extended/mc/files/0001-Add-option-to-control-configure-args.patch
meta/recipes-extended/mc/files/0001-Add-option-to-control-configure-args.patch new file +99 -0
new file mode 100644
index 0000000000..e76aac8161
@@ -0,0 +1,99 @@
+From a54501d3c9541bc8600225aa2d42531f93c6def7 Mon Sep 17 00:00:00 2001
+From: Joshua Watt <[email protected]>
+Date: Sat, 9 Nov 2019 20:01:48 -0600
+Subject: [PATCH] Add option to control configure args
+
+Embedding the configure time options into the executable can lead to
+non-reproducible builds, since configure options often have embedded
+paths. Add a configure time option to control if the configure args are
+embedded so this can be disabled.
+
+Upstream-Status: Submitted [https://midnight-commander.org/ticket/4031]
+Signed-off-by: Joshua Watt <[email protected]>
+---
+ configure.ac   | 6 ++++++
+ src/args.c     | 6 ++++++
+ src/textconf.c | 2 ++
+ 3 files changed, 14 insertions(+)
+
+diff --git a/configure.ac b/configure.ac
+index 19d1a76be..a1948f6b9 100644
+--- a/configure.ac
++++ b/configure.ac
+@@ -544,6 +544,12 @@ dnl Clarify do we really need GModule
+ AM_CONDITIONAL([HAVE_GMODULE], [test -n "$g_module_supported" && \
+                                 test x"$textmode_x11_support" = x"yes" -o x"$enable_aspell" = x"yes"])
+ 
++AC_ARG_ENABLE([configure-args],
++    AS_HELP_STRING([--enable-configure-args], [Handle all compiler warnings as errors]))
++if test "x$enable_configure_args" != xno; then
++    AC_DEFINE([ENABLE_CONFIGURE_ARGS], 1, [Define to enable showing configure arguments in help])
++fi
++
+ AC_DEFINE_UNQUOTED([MC_CONFIGURE_ARGS], ["$ac_configure_args"], [MC configure arguments])
+ 
+ AC_CONFIG_FILES(
+diff --git a/src/args.c b/src/args.c
+index baef1a1c8..f8dc24020 100644
+--- a/src/args.c
++++ b/src/args.c
+@@ -95,7 +95,9 @@ static gboolean mc_args__nouse_subshell = FALSE;
+ #endif /* ENABLE_SUBSHELL */
+ static gboolean mc_args__show_datadirs = FALSE;
+ static gboolean mc_args__show_datadirs_extended = FALSE;
++#ifdef ENABLE_CONFIGURE_ARGS
+ static gboolean mc_args__show_configure_opts = FALSE;
++#endif
+ 
+ static GOptionGroup *main_group;
+ 
+@@ -125,6 +127,7 @@ static const GOptionEntry argument_main_table[] = {
+      NULL
+     },
+ 
++#ifdef ENABLE_CONFIGURE_ARGS
+     /* show configure options */
+     {
+      "configure-options", '\0', G_OPTION_FLAG_IN_MAIN, G_OPTION_ARG_NONE,
+@@ -132,6 +135,7 @@ static const GOptionEntry argument_main_table[] = {
+      N_("Print configure options"),
+      NULL
+     },
++#endif
+ 
+     {
+      "printwd", 'P', G_OPTION_FLAG_IN_MAIN, G_OPTION_ARG_STRING,
+@@ -758,11 +762,13 @@ mc_args_show_info (void)
+         return FALSE;
+     }
+ 
++#ifdef ENABLE_CONFIGURE_ARGS
+     if (mc_args__show_configure_opts)
+     {
+         show_configure_options ();
+         return FALSE;
+     }
++#endif
+ 
+     return TRUE;
+ }
+diff --git a/src/textconf.c b/src/textconf.c
+index 1e0613e58..f39b9e028 100644
+--- a/src/textconf.c
++++ b/src/textconf.c
+@@ -232,10 +232,12 @@ show_datadirs_extended (void)
+ 
+ /* --------------------------------------------------------------------------------------------- */
+ 
++#ifdef ENABLE_CONFIGURE_ARGS
+ void
+ show_configure_options (void)
+ {
+     (void) printf ("%s\n", MC_CONFIGURE_ARGS);
+ }
++#endif
+ 
+ /* --------------------------------------------------------------------------------------------- */
+-- 
+2.23.0
+
meta/recipes-extended/mc/mc_4.8.23.bb +4 -1
index 83de8dbb2c..71f61b4848 100644
@@ -8,6 +8,7 @@ RDEPENDS_${PN} = "ncurses-terminfo"
 
 SRC_URI = "http://www.midnight-commander.org/downloads/${BPN}-${PV}.tar.bz2 \
            file://0001-mc-replace-perl-w-with-use-warnings.patch \
+           file://0001-Add-option-to-control-configure-args.patch \
            "
 SRC_URI[md5sum] = "152927ac29cf0e61d7d019f261bb7d89"
 SRC_URI[sha256sum] = "238c4552545dcf3065359bd50753abbb150c1b22ec5a36eaa02c82808293267d"
@@ -21,9 +22,11 @@ PACKAGECONFIG ??= ""
 PACKAGECONFIG[smb] = "--enable-vfs-smb,--disable-vfs-smb,samba,"
 PACKAGECONFIG[sftp] = "--enable-vfs-sftp,--disable-vfs-sftp,libssh2,"
 
-EXTRA_OECONF = "--with-screen=ncurses --without-gpm-mouse --without-x"
+EXTRA_OECONF = "--with-screen=ncurses --without-gpm-mouse --without-x --disable-configure-args"
 
 CACHED_CONFIGUREVARS += "ac_cv_path_PERL='/usr/bin/env perl'"
+CACHED_CONFIGUREVARS += "ac_cv_path_PYTHON='/usr/bin/env python'"
+CACHED_CONFIGUREVARS += "ac_cv_path_GREP='/usr/bin/env grep'"
 
 do_install_append () {
 	sed -i -e '1s,#!.*perl,#!${bindir}/env perl,' ${D}${libexecdir}/mc/extfs.d/*
Signature · 1 line
2.17.1
#14 lore raw

PATCH 14/29 mc: Set zipinfo presence determinstically

In reply to Armin Kuster

From: Richard Purdie <[email protected]>

This value was floating causing differences in generated files. Set it determinstically.

(From OE-Core rev: 11d7a9e37c1d3fc21396a98fefc9d34c0b9e784b)

(From OE-Core rev: 8f77075425e2ef9c3b5adbf8e5b29e7cfd7b9b7a)

---
 meta/recipes-extended/mc/mc_4.8.23.bb | 1 +
 1 file changed, 1 insertion(+)
meta/recipes-extended/mc/mc_4.8.23.bb +1 -0
index 71f61b4848..5667038a18 100644
@@ -27,6 +27,7 @@ EXTRA_OECONF = "--with-screen=ncurses --without-gpm-mouse --without-x --disable-
 CACHED_CONFIGUREVARS += "ac_cv_path_PERL='/usr/bin/env perl'"
 CACHED_CONFIGUREVARS += "ac_cv_path_PYTHON='/usr/bin/env python'"
 CACHED_CONFIGUREVARS += "ac_cv_path_GREP='/usr/bin/env grep'"
+CACHED_CONFIGUREVARS += "mc_cv_have_zipinfo=yes"
 
 do_install_append () {
 	sed -i -e '1s,#!.*perl,#!${bindir}/env perl,' ${D}${libexecdir}/mc/extfs.d/*
Signature · 1 line
2.17.1
#15 lore raw

PATCH 15/29 mc: Fix manpage date indeterminism

In reply to Armin Kuster

From: Richard Purdie <[email protected]>

The man page date can vary depending upon the host perl, e.g. in Russian some versions print 'июня', others 'Июнь' or Polish 'czerwca' or 'czerwiec'. Rather than depend upon perl-native to fix this, just remove the date from the manpages.

(From OE-Core rev: 5553c20f9fa4f35bf711b6b9d5717dcf4bfefafa)

(From OE-Core rev: 3653fd41fbc28f70259a00bb0098ec8731526449)

---
 .../recipes-extended/mc/files/nomandate.patch | 21 +++++++++++++++++++
 meta/recipes-extended/mc/mc_4.8.23.bb         |  1 +
 2 files changed, 22 insertions(+)
 create mode 100644 meta/recipes-extended/mc/files/nomandate.patch
meta/recipes-extended/mc/files/nomandate.patch new file +21 -0
new file mode 100644
index 0000000000..48bd73b110
@@ -0,0 +1,21 @@
+The man page date can vary depending upon the host perl, e.g. in Russian
+some versions print 'июня', others 'Июнь' or Polish 'czerwca' or 'czerwiec'.
+Rather than depend upon perl-native to fix this, just remove the date from 
+the manpages.
+
+RP 2020/2/4
+
+Upstream-Status: Inappropriate [OE specficic reproducibility workaround]
+Signed-off-by: Richard Purdie <[email protected]>
+
+Index: mc-4.8.23/doc/man/date-of-man-include.am
+===================================================================
+--- mc-4.8.23.orig/doc/man/date-of-man-include.am
++++ mc-4.8.23/doc/man/date-of-man-include.am
+@@ -1,5 +1,5 @@
+ SED_PARAMETERS = \
+-	-e "s/%DATE_OF_MAN_PAGE%/$${MAN_DATE}/g" \
++	-e "s/%DATE_OF_MAN_PAGE%//g" \
+ 	-e "s/%DISTR_VERSION%/@DISTR_VERSION@/g" \
+ 	-e "s{%prefix%{@prefix@{g" \
+ 	-e "s{%sysconfdir%{@sysconfdir@{g" \
meta/recipes-extended/mc/mc_4.8.23.bb +1 -0
index 5667038a18..de76591d9b 100644
@@ -9,6 +9,7 @@ RDEPENDS_${PN} = "ncurses-terminfo"
 SRC_URI = "http://www.midnight-commander.org/downloads/${BPN}-${PV}.tar.bz2 \
            file://0001-mc-replace-perl-w-with-use-warnings.patch \
            file://0001-Add-option-to-control-configure-args.patch \
+           file://nomandate.patch \
            "
 SRC_URI[md5sum] = "152927ac29cf0e61d7d019f261bb7d89"
 SRC_URI[sha256sum] = "238c4552545dcf3065359bd50753abbb150c1b22ec5a36eaa02c82808293267d"
Signature · 1 line
2.17.1
#16 lore raw

PATCH 16/29 tar: Fix build determinism, disable rsh

In reply to Armin Kuster

From: Richard Purdie <[email protected]>

rsh is insecure and obsolete but tar will enable support if the binary is on the host system. Some systems point it at ssh. Lets explictly disable it for now unless someone actually needs/uses this at which point it could become a packageconfig.

(From OE-Core rev: d14a4b0db92a9a7d1ff72a2e0faca7f1a23a0b68)

(From OE-Core rev: 6bdc5f787af46e9c849947cad06ad40aa401b767)

---
 meta/recipes-extended/tar/tar_1.32.bb | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-extended/tar/tar_1.32.bb +2 -0
index 18f09b5711..ebe6cb0dbd 100644
@@ -22,6 +22,8 @@ PACKAGECONFIG[acl] = "--with-posix-acls,--without-posix-acls,acl"
 
 EXTRA_OECONF += "DEFAULT_RMT_DIR=${sbindir}"
 
+CACHED_CONFIGUREVARS += "tar_cv_path_RSH=no"
+
 # Let aclocal use the relative path for the m4 file rather than the
 # absolute since tar has a lot of m4 files, otherwise there might
 # be an "Argument list too long" error when it is built in a long/deep
Signature · 1 line
2.17.1
#17 lore raw

PATCH 17/29 patch: Extend to native/nativesdk and depend upon

In reply to Armin Kuster

From: Richard Purdie <[email protected]>

There is a bug in patch 2.7.3 and earlier where index lines in patches can change file modes when they shouldn't:
http://git.savannah.gnu.org/cgit/patch.git/patch/?id=82b800c9552a088a241457948219d25ce0a407a4

This leaks into debug sources in particular (e.g. tcp-wrappers where source files are read-only). Add the dependency to target recipes to avoid this problem until we can rely on 2.7.4 or later.

We could try and remove all index lines from patch files but it will be a losing battle. We could try and identify all the recipes which change modes on files in patches but again, its a losing battle.

Instead, compromise and have patch-native as a dependency for target recipes. We use patch-replacement-native since patch-native is in ASSUME_PROVIDED.

Also add nativesdk-patch to buildtools-tarball.

[YOCTO #13777]

(From OE-Core rev: 5ed0840c93804488cd1c1aba6cb382b2434714a5)

(From OE-Core rev: fd3bd61a6fe5190c575dc968f3a0be9c1cbf21ed)

---
 meta/classes/patch.bbclass                   | 7 +++++++
 meta/recipes-core/meta/buildtools-tarball.bb | 1 +
 meta/recipes-devtools/patch/patch_2.7.6.bb   | 3 +++
 3 files changed, 11 insertions(+)
meta/classes/patch.bbclass +7 -0
index cd241f1c84..25ec089ae1 100644
@@ -5,6 +5,13 @@ QUILTRCFILE ?= "${STAGING_ETCDIR_NATIVE}/quiltrc"
 
 PATCHDEPENDENCY = "${PATCHTOOL}-native:do_populate_sysroot"
 
+# There is a bug in patch 2.7.3 and earlier where index lines
+# in patches can change file modes when they shouldn't:
+# http://git.savannah.gnu.org/cgit/patch.git/patch/?id=82b800c9552a088a241457948219d25ce0a407a4
+# This leaks into debug sources in particular. Add the dependency
+# to target recipes to avoid this problem until we can rely on 2.7.4 or later.
+PATCHDEPENDENCY_append_class-target = " patch-replacement-native:do_populate_sysroot"
+
 PATCH_GIT_USER_NAME ?= "OpenEmbedded"
 PATCH_GIT_USER_EMAIL ?= "oe.patch@oe"
 
meta/recipes-core/meta/buildtools-tarball.bb +1 -0
index 91df6f1ae9..66201514d7 100644
@@ -25,6 +25,7 @@ TOOLCHAIN_HOST_TASK ?= "\
     nativesdk-texinfo \
     nativesdk-libnss-nis \
     nativesdk-rpcsvc-proto \
+    nativesdk-patch \
     "
 
 MULTIMACH_TARGET_SYS = "${SDK_ARCH}-nativesdk${SDK_VENDOR}-${SDK_OS}"
meta/recipes-devtools/patch/patch_2.7.6.bb +3 -0
index 5d7f55f8dc..b5897b357a 100644
@@ -22,3 +22,6 @@ acpaths = "-I ${S}/m4 "
 PACKAGECONFIG ?= "${@bb.utils.filter('DISTRO_FEATURES', 'xattr', d)}"
 PACKAGECONFIG[xattr] = "--enable-xattr,--disable-xattr,attr,"
 
+PROVIDES_append_class-native = " patch-replacement-native"
+
+BBCLASSEXTEND = "native nativesdk"
Signature · 1 line
2.17.1
#18 lore raw

PATCH 18/29 libidn2: Fix reproducibility issue

In reply to Armin Kuster

From: Richard Purdie <[email protected]>

The previous tweak for reproducibility didn't handle the duplicate whitepace left behind, fix this.

[YOCTO #13771]

(From OE-Core rev: 0392fcbdc85180581ce7392212808ebb822cc2e8)

(From OE-Core rev: ca213de432d8d9d715ef09e17cea2aa1e6666e91)

---
 meta/recipes-extended/libidn/libidn2_2.2.0.bb | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
meta/recipes-extended/libidn/libidn2_2.2.0.bb +2 -1
index bcbfdd85b9..71314149e1 100644
@@ -22,7 +22,8 @@ EXTRA_OECONF += "--disable-rpath \
                  "
 
 do_install_append() {
-	sed -i -e 's|-L${STAGING_LIBDIR}||' ${D}${libdir}/pkgconfig/libidn2.pc
+	# Need to remove any duplicate whitespace too for reproducibility
+	sed -i -e 's|-L${STAGING_LIBDIR}||' -e 's/  */ /g' ${D}${libdir}/pkgconfig/libidn2.pc
 }
 
 LICENSE_${PN} = "(GPLv2+ | LGPLv3)"
Signature · 1 line
2.17.1
#19 lore raw

PATCH 19/29 gtk+3: sort resources for reproducible binaries

In reply to Armin Kuster

From: Ross Burton <[email protected]>

The list of resources is gathered with $(wildcard) in Make, which isn't sorted. If this order changes then the generated libraries will differ.

(From OE-Core rev: f3675be6be29426688187a135221431a0941d007)

(From OE-Core rev: a2a6d9fa4df66a1f52c1c1fec45eedb4199e8162)

---
 .../gtk+/gtk+3/sort-resources.patch           | 19 +++++++++++++++++++
 meta/recipes-gnome/gtk+/gtk+3_3.24.8.bb       |  1 +
 2 files changed, 20 insertions(+)
 create mode 100644 meta/recipes-gnome/gtk+/gtk+3/sort-resources.patch
meta/recipes-gnome/gtk+/gtk+3/sort-resources.patch new file +19 -0
new file mode 100644
index 0000000000..7f87372c52
@@ -0,0 +1,19 @@
+If the resources file isn't sorted in some way then libgdk.so will differ
+depending on the inode order of the resource files.
+
+Upstream-Status: Pending
+Signed-off-by: Ross Burton <[email protected]>
+
+diff --git a/gdk/Makefile.am b/gdk/Makefile.am
+index e25b57ba50..26f2d57c6e 100644
+--- a/gdk/Makefile.am
++++ b/gdk/Makefile.am
+@@ -465,7 +465,7 @@ stamp-gc-h: $(top_builddir)/config.status
+ # Resources
+ #
+ 
+-glsl_sources := $(wildcard $(srcdir)/resources/glsl/*.glsl)
++glsl_sources := $(sort $(wildcard $(srcdir)/resources/glsl/*.glsl))
+ 
+ gdk.gresource.xml: Makefile.am
+ 	$(AM_V_GEN) echo "<?xml version='1.0' encoding='UTF-8'?>" > $@; \
meta/recipes-gnome/gtk+/gtk+3_3.24.8.bb +1 -0
index d79b18bee0..596dee6264 100644
@@ -7,6 +7,7 @@ SRC_URI = "http://ftp.gnome.org/pub/gnome/sources/gtk+/${MAJ_VER}/gtk+-${PV}.tar
            file://0002-Do-not-try-to-initialize-GL-without-libGL.patch \
            file://0003-Add-disable-opengl-configure-option.patch \
            file://link_fribidi.patch \
+           file://sort-resources.patch \
           "
 SRC_URI[md5sum] = "eeedde01856238114dcf4df3ebc942a5"
 SRC_URI[sha256sum] = "666962de9b9768fe9ca785b0e2f42c8b9db3868a12fa9b356b167238d70ac799"
Signature · 1 line
2.17.1
#20 lore raw

PATCH 20/29 perl: do not install files that contain build host specific data

In reply to Armin Kuster

From: Alexander Kanavin <[email protected]>

This was breaking reproducibility, and the files aren't needed on target.

[YOCTO #13772]

(From OE-Core rev: 2e0f30c4680221c693495e3a0327378d502a518b)

(From OE-Core rev: 208efc88fa3c57244b272bf7e7f7f8163f14630c)

---
 meta/recipes-devtools/perl/perl-ptest.inc | 3 +++
 meta/recipes-devtools/perl/perl_5.30.0.bb | 3 +++
 2 files changed, 6 insertions(+)
meta/recipes-devtools/perl/perl-ptest.inc +3 -0
index 7152057762..98e3361fcc 100644
@@ -42,6 +42,9 @@ do_install_ptest () {
 
 	 # Remove a useless timestamp...
 	 sed -i -e '/Autogenerated starting on/d' ${D}${PTEST_PATH}/lib/unicore/mktables.lst
+
+	 # Remove files with host-specific configuration for building native binaries
+	 rm ${D}${PTEST_PATH}/Makefile.config ${D}${PTEST_PATH}/xconfig.h ${D}${PTEST_PATH}/xconfig.sh
 }
 
 python populate_packages_prepend() {
meta/recipes-devtools/perl/perl_5.30.0.bb +3 -0
index ba2a8437d4..c567d4e7a2 100644
@@ -135,6 +135,9 @@ do_install_append_class-target() {
     # This is used to substitute target configuration when running native perl via perl-configpm-switch.patch
     ln -s Config_heavy.pl ${D}${libdir}/perl5/${PV}/${TARGET_ARCH}-linux/Config_heavy-target.pl
 
+    # This contains host-specific information used for building miniperl (a helper executable built with host compiler)
+    # and therefore isn't reproducible. I believe the file isn't actually needed on target.
+    rm ${D}${libdir}/perl5/${PV}/${TARGET_ARCH}-linux/CORE/xconfig.h
 }
 
 do_install_append_class-nativesdk() {
Signature · 1 line
2.17.1
#21 lore raw

PATCH 21/29 perl: Fix various reproducibile build issues

In reply to Armin Kuster

From: Richard Purdie <[email protected]>

Add a patch which handles the following issues:

a) Remove the \n from configure_attr.sh since it gets quoted differently depending on whether the shell is bash or dash which can cause the test result to be incorrect. Reported upstream: https://github.com/arsv/perl-cross/issues/87

b) Sort the order of the module lists from configure_mods.sh since otherwise the result isn't the same leading to makefile differences. Reported upstream: https://github.com/arsv/perl-cross/issues/88

c) Sort the Encode::Byte byte_t.fnm file output (and the makefile depends whilst there for good measure)
This needs to go to upstream perl (not done)

d) Use bash for perl-cross configure since otherwise trnl gets set to "\n" with bash and "" with dash
Reported upstream: https://github.com/arsv/perl-cross/issues/87

(From OE-Core rev: 482fd0d99f989b5a72a25bdf402fb2f219420b5d)

(From OE-Core rev: def3a9d748564883d71c506726554df622701b00)

---
 .../perl/files/determinism.patch              | 81 +++++++++++++++++++
 meta/recipes-devtools/perl/perl_5.30.0.bb     |  1 +
 2 files changed, 82 insertions(+)
 create mode 100644 meta/recipes-devtools/perl/files/determinism.patch
meta/recipes-devtools/perl/files/determinism.patch new file +81 -0
new file mode 100644
index 0000000000..ed4d06f5ec
@@ -0,0 +1,81 @@
+Fixes to make the perl build reproducible:
+
+a) Remove the \n from configure_attr.sh since it gets quoted differently depending on
+   whether the shell is bash or dash which can cause the test result to be incorrect.
+   Reported upstream: https://github.com/arsv/perl-cross/issues/87
+
+b) Sort the order of the module lists from configure_mods.sh since otherwise
+   the result isn't the same leading to makefile differences.
+   Reported upstream: https://github.com/arsv/perl-cross/issues/88
+
+c) Sort the Encode::Byte byte_t.fnm file output (and the makefile depends whilst 
+   there for good measure)
+   This needs to go to upstream perl (not done)
+
+d) Use bash for perl-cross configure since otherwise trnl gets set to "\n" with bash
+   and "" with dash
+   Reported upstream: https://github.com/arsv/perl-cross/issues/87
+
+RP 2020/2/7
+
+Upstream-Status: Pending [75% submitted]
+Signed-off-by: Richard Purdie <[email protected]
+
+Index: perl-5.30.1/cnf/configure_attr.sh
+===================================================================
+--- perl-5.30.1.orig/cnf/configure_attr.sh
++++ perl-5.30.1/cnf/configure_attr.sh
+@@ -131,7 +131,7 @@ if not hinted d_c99_variadic_macros 'sup
+ 	try_start
+ 	try_add '#include <stdio.h>'
+ 	try_add '#define foo(fmt, ...) printf(fmt, __VA_ARGS__)'
+-	try_add 'int main(void) { foo("%i\n", 1234); return 0; }'
++	try_add 'int main(void) { foo("%i", 1234); return 0; }'
+ 	try_compile
+ 	resdef d_c99_variadic_macros 'supported' 'missing'
+ fi
+Index: perl-5.30.1/cnf/configure_mods.sh
+===================================================================
+--- perl-5.30.1.orig/cnf/configure_mods.sh
++++ perl-5.30.1/cnf/configure_mods.sh
+@@ -82,7 +82,7 @@ extonlyif() {
+ }
+ 
+ definetrimspaces() {
+-	v=`echo "$2" | sed -r -e 's/\s+/ /g' -e 's/^\s+//' -e 's/\s+$//'`
++	v=`echo "$2" | sed -r -e 's/\s+/ /g' -e 's/^\s+//' -e 's/\s+$//' | xargs -n1 | LANG=C sort | xargs`
+ 	define $1 "$v"
+ }
+ 
+Index: perl-5.30.1/cpan/Encode/Byte/Makefile.PL
+===================================================================
+--- perl-5.30.1.orig/cpan/Encode/Byte/Makefile.PL
++++ perl-5.30.1/cpan/Encode/Byte/Makefile.PL
+@@ -171,7 +171,7 @@ sub postamble
+     my $lengthsofar = length($str);
+     my $continuator = '';
+     $str .= "$table.c : $enc2xs Makefile.PL";
+-    foreach my $file (@{$tables{$table}})
++    foreach my $file (sort (@{$tables{$table}}))
+     {
+         $str .= $continuator.' '.$self->catfile($dir,$file);
+         if ( length($str)-$lengthsofar > 128*$numlines )
+@@ -189,7 +189,7 @@ sub postamble
+         qq{\n\t\$(PERL) $plib $enc2xs $ucopts -o \$\@ -f $table.fnm\n\n};
+     open (FILELIST, ">$table.fnm")
+         || die "Could not open $table.fnm: $!";
+-    foreach my $file (@{$tables{$table}})
++    foreach my $file (sort (@{$tables{$table}}))
+     {
+         print FILELIST $self->catfile($dir,$file) . "\n";
+     }
+Index: perl-5.30.1/cnf/configure
+===================================================================
+--- perl-5.30.1.orig/cnf/configure
++++ perl-5.30.1/cnf/configure
+@@ -1,4 +1,4 @@
+-#!/bin/sh
++#!/bin/bash
+ 
+ base=${0%/*}; test -z "$base" && base=.
+ 
meta/recipes-devtools/perl/perl_5.30.0.bb +1 -0
index c567d4e7a2..838e52c67b 100644
@@ -21,6 +21,7 @@ SRC_URI = "https://www.cpan.org/src/5.0/perl-${PV}.tar.gz;name=perl \
            file://fix-setgroup.patch \
            file://0001-enc2xs-Add-environment-variable-to-suppress-comments.patch \
            file://0002-Constant-Fix-up-shebang.patch \
+           file://determinism.patch  \
            "
 SRC_URI_append_class-native = " \
            file://perl-configpm-switch.patch \
Signature · 1 line
2.17.1
#22 lore raw

PATCH 22/29 openssl: Fix reproducibility issue

In reply to Armin Kuster

From: Richard Purdie <[email protected]>

There was a build architecture leaking into the target ptest which could vary depending upon host. Remove it as its cosmetic.

[YOCTO #13770]

(From OE-Core rev: 37db519eedb7eb5cd4f14d05f30f5d580aa7458d)

(From OE-Core rev: c31c676319812e6fc036741db2ab8e16eccff723)

---
 .../openssl/openssl/reproducible.patch        | 32 +++++++++++++++++++
 .../openssl/openssl_1.1.1d.bb                 |  1 +
 2 files changed, 33 insertions(+)
 create mode 100644 meta/recipes-connectivity/openssl/openssl/reproducible.patch
meta/recipes-connectivity/openssl/openssl/reproducible.patch new file +32 -0
new file mode 100644
index 0000000000..a24260c95d
@@ -0,0 +1,32 @@
+The value for perl_archname can vary depending on the host, e.g. 
+x86_64-linux-gnu-thread-multi or x86_64-linux-thread-multi which
+makes the ptest package non-reproducible. Its unused other than 
+these references so drop it.
+
+RP 2020/2/6
+
+Upstream-Status: Pending
+Signed-off-by: Richard Purdie <[email protected]>
+
+Index: openssl-1.1.1d/Configure
+===================================================================
+--- openssl-1.1.1d.orig/Configure
++++ openssl-1.1.1d/Configure
+@@ -286,7 +286,7 @@ if (defined env($local_config_envname))
+ # Save away perl command information
+ $config{perl_cmd} = $^X;
+ $config{perl_version} = $Config{version};
+-$config{perl_archname} = $Config{archname};
++#$config{perl_archname} = $Config{archname};
+ 
+ $config{prefix}="";
+ $config{openssldir}="";
+@@ -2517,7 +2517,7 @@ _____
+                           @{$config{perlargv}}), "\n";
+         print "\nPerl information:\n\n";
+         print '    ',$config{perl_cmd},"\n";
+-        print '    ',$config{perl_version},' for ',$config{perl_archname},"\n";
++        print '    ',$config{perl_version},"\n";
+     }
+     if ($dump || $options) {
+         my $longest = 0;
meta/recipes-connectivity/openssl/openssl_1.1.1d.bb +1 -0
index 458ae7daf4..169824a8be 100644
@@ -17,6 +17,7 @@ SRC_URI = "http://www.openssl.org/source/openssl-${PV}.tar.gz \
            file://0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch \
            file://afalg.patch \
            file://CVE-2019-1551.patch \
+           file://reproducible.patch \
            "
 
 SRC_URI_append_class-nativesdk = " \
Signature · 1 line
2.17.1
#23 lore raw

PATCH 23/29 iputils: Fix build determinism

In reply to Armin Kuster

From: Richard Purdie <[email protected]>

The suid/setcap code depends on whether setcap is on the host system or not with suid as a fallback. Disable this functionality to be deterministic.

(From OE-Core rev: 8b00ec484fb851c301f13145e17707c0167feab1)

(From OE-Core rev: 3997c47dea49d583fd48cb03f83c007f61d2cb35)

---
 meta/recipes-extended/iputils/iputils_s20190709.bb | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
meta/recipes-extended/iputils/iputils_s20190709.bb +2 -1
index 3f9e9917f0..42260f531e 100644
@@ -32,7 +32,8 @@ PACKAGECONFIG[docs] = "-DBUILD_HTML_MANS=true -DBUILD_MANS=true,-DBUILD_HTML_MAN
 
 inherit meson update-alternatives
 
-EXTRA_OEMESON += "--prefix=${root_prefix}/"
+# Have to disable setcap/suid as its not deterministic
+EXTRA_OEMESON += "--prefix=${root_prefix}/ -DNO_SETCAP_OR_SUID=true"
 
 ALTERNATIVE_PRIORITY = "100"
 
Signature · 1 line
2.17.1
#24 lore raw

PATCH 24/29 libinput: Fix determinism issue

In reply to Armin Kuster

From: Richard Purdie <[email protected]>

The build was injection git information from the wrong git tree, stop this to allow reproducible builds.

(From OE-Core rev: c3f6a6113f562ecdb13386c3ff52adb7973980a4)

(From OE-Core rev: 22dc9a3314eef5925241a9a961bd79815d3172b6)

---
 .../wayland/libinput/determinism.patch        | 21 +++++++++++++++++++
 .../wayland/libinput_1.14.1.bb                |  4 +++-
 2 files changed, 24 insertions(+), 1 deletion(-)
 create mode 100644 meta/recipes-graphics/wayland/libinput/determinism.patch
meta/recipes-graphics/wayland/libinput/determinism.patch new file +21 -0
new file mode 100644
index 0000000000..cb554030cf
@@ -0,0 +1,21 @@
+This finds our outer git tree and that version information breaks
+determinism of this recipe. Disable it.
+
+RP 2020/2/6
+
+Upstream-Status: Pending
+Signed-off-by: Richard Purdie <[email protected]>
+
+Index: libinput-1.14.3/meson.build
+===================================================================
+--- libinput-1.14.3.orig/meson.build
++++ libinput-1.14.3/meson.build
+@@ -387,7 +387,7 @@ pkgconfig.generate(
+ 	libraries : lib_libinput
+ )
+ 
+-git_version_h = vcs_tag(command : ['git', 'describe'],
++git_version_h = vcs_tag(command : ['false'],
+ 			fallback : 'unknown',
+ 			input : 'src/libinput-git-version.h.in',
+ 			output :'libinput-git-version.h')
meta/recipes-graphics/wayland/libinput_1.14.1.bb +3 -1
index 38bc8d2c33..2c5733f33a 100644
@@ -7,7 +7,9 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=1f2ea9ebff3a2c6d458faf58492efb63"
 
 DEPENDS = "libevdev udev mtdev"
 
-SRC_URI = "http://www.freedesktop.org/software/${BPN}/${BP}.tar.xz"
+SRC_URI = "http://www.freedesktop.org/software/${BPN}/${BP}.tar.xz \
+           file://determinism.patch \
+"
 SRC_URI[md5sum] = "da29a704dc6f7ea2d5aac754db046340"
 SRC_URI[sha256sum] = "e333a3242835c019ca37d2cef8b51a87d3138eb47444119c0153dc7a8656ee70"
 
Signature · 1 line
2.17.1
#25 lore raw

PATCH 25/29 libgcrypt: Fix determinism issue

In reply to Armin Kuster

From: Richard Purdie <[email protected]>

The build was injection git information from the wrong git tree, stop this to allow reproducible builds.

(From OE-Core rev: 506b36b6d86b3454fcc3cb85f6229cbe8d14f5b5)

(From OE-Core rev: 2b68ffc6d7c45541958f1b0c0b9ca9e39064e096)

---
 .../libgcrypt/files/determinism.patch         | 32 +++++++++++++++++++
 .../libgcrypt/libgcrypt_1.8.4.bb              |  1 +
 2 files changed, 33 insertions(+)
 create mode 100644 meta/recipes-support/libgcrypt/files/determinism.patch
meta/recipes-support/libgcrypt/files/determinism.patch new file +32 -0
new file mode 100644
index 0000000000..ad0b8c7950
@@ -0,0 +1,32 @@
+gnutls detects our outer git trees and injects that revision into its objects.
+That isn't deterministic so stop it. Also ensure we're not marked as a development
+build as its git detection is faulty.
+
+RP 2020/2/6
+
+Upstream-Status: Pending
+Signed-off-by: Richard Purdie <[email protected]>
+
+
+Index: libgcrypt-1.8.5/configure.ac
+===================================================================
+--- libgcrypt-1.8.5.orig/configure.ac
++++ libgcrypt-1.8.5/configure.ac
+@@ -45,7 +45,7 @@ m4_define([mym4_revision_dec],
+ m4_define([mym4_betastring],
+           m4_esyscmd_s([git describe --match 'libgcrypt-[0-9].*[0-9]' --long|\
+                         awk -F- '$3!=0{print"-beta"$3}']))
+-m4_define([mym4_isgit],m4_if(mym4_betastring,[],[no],[yes]))
++m4_define([mym4_isgit],[no])
+ m4_define([mym4_full_version],[mym4_version[]mym4_betastring])
+ 
+ AC_INIT([libgcrypt],[mym4_full_version],[http://bugs.gnupg.org])
+@@ -2575,7 +2575,7 @@ AM_CONDITIONAL([BUILD_DOC], [test "x$bui
+ #
+ # Provide information about the build.
+ #
+-BUILD_REVISION="mym4_revision"
++BUILD_REVISION="None"
+ AC_SUBST(BUILD_REVISION)
+ AC_DEFINE_UNQUOTED(BUILD_REVISION, "$BUILD_REVISION",
+                    [GIT commit id revision used to build this package])
meta/recipes-support/libgcrypt/libgcrypt_1.8.4.bb +1 -0
index 1bd355133e..92eb2d257a 100644
@@ -26,6 +26,7 @@ SRC_URI = "${GNUPG_MIRROR}/libgcrypt/libgcrypt-${PV}.tar.bz2 \
            file://0003-GCM-move-look-up-table-to-.data-section-and-unshare-.patch \
            file://0001-ecc-Add-mitigation-against-timing-attack.patch \
            file://0001-dsa-ecdsa-Fix-use-of-nonce-use-larger-one.patch \
+           file://determinism.patch \
 "
 SRC_URI[md5sum] = "fbfdaebbbc6d7e5fbbf6ffdb3e139573"
 SRC_URI[sha256sum] = "f638143a0672628fde0cad745e9b14deb85dffb175709cacc1f4fe24b93f2227"
Signature · 1 line
2.17.1
#26 lore raw

PATCH 26/29 sudo: specify where target tools are

In reply to Armin Kuster

From: Ross Burton <[email protected]>

sudo uses AC_PATH_PROG to find target paths, which means at best potential host-contamination (and reproducible issues) and at worst it thinks sh is at /your/build/path/hosttools/sh.

Solve this by explicitly passing the correct paths to configure.

(From OE-Core rev: 61650dd8498a093f3bfa93202c9cd2e9a7fb7834)

(From OE-Core rev: 6e809474ab686fba6924d8b46fd0b9eab5c66c06)

---
 meta/recipes-extended/sudo/sudo.inc       | 2 +-
 meta/recipes-extended/sudo/sudo_1.8.27.bb | 9 ++++++++-
 2 files changed, 9 insertions(+), 2 deletions(-)
meta/recipes-extended/sudo/sudo.inc +1 -1
index 15075bcefd..4edfabe510 100644
@@ -26,7 +26,7 @@ PACKAGECONFIG[pam-wheel] = ",,,pam-plugin-wheel"
 
 CONFFILES_${PN} = "${sysconfdir}/sudoers"
 
-EXTRA_OECONF = "--with-editor=/bin/vi --with-env-editor"
+EXTRA_OECONF = "--with-editor=${base_bindir}/vi --with-env-editor"
 
 EXTRA_OECONF_append_libc-musl = " --disable-hardening "
 
meta/recipes-extended/sudo/sudo_1.8.27.bb +8 -1
index ee3be95644..6d470d0373 100644
@@ -15,8 +15,15 @@ SRC_URI[sha256sum] = "7beb68b94471ef56d8a1036dbcdc09a7b58a949a68ffce48b83f837dd3
 DEPENDS += " virtual/crypt ${@bb.utils.contains('DISTRO_FEATURES', 'pam', 'libpam', '', d)}"
 RDEPENDS_${PN} += " ${@bb.utils.contains('DISTRO_FEATURES', 'pam', 'pam-plugin-limits pam-plugin-keyinit', '', d)}"
 
+CACHED_CONFIGUREVARS = " \
+        ac_cv_type_rsize_t=no \
+        ac_cv_path_MVPROG=${base_bindir}/mv \
+        ac_cv_path_BSHELLPROG=${base_bindir}/sh \
+        ac_cv_path_SENDMAILPROG=${sbindir}/sendmail \
+        ac_cv_path_VIPROG=${base_bindir}/vi \
+        "
+
 EXTRA_OECONF += " \
-             ac_cv_type_rsize_t=no \
              ${@bb.utils.contains('DISTRO_FEATURES', 'pam', '--with-pam', '--without-pam', d)} \
              ${@bb.utils.contains('DISTRO_FEATURES', 'systemd', '--enable-tmpfiles.d=${nonarch_libdir}/tmpfiles.d', '--disable-tmpfiles.d', d)} \
              --with-vardir=/var/lib/sudo \
Signature · 1 line
2.17.1
#27 lore raw

PATCH 27/29 sysvinit: Fix Reproducibility issue

In reply to Armin Kuster

From: Richard Purdie <[email protected]>

With a sequence like:

bitbake sysvinit
bitbake sysvinit -c clean
bitbake sysvinit -c package_write_ipk -f

then the resulting package has two files with group "root/70" rather than "root/shutdown". The issue is that of do_package is a setscene task, base-passwd isn't present. This patch fixes that dependency but there may be other cases of this problem around.

[YOCTO #13776]

(From OE-Core rev: 0227e929021263c51d2e7db36224000fecb01f1c)

(From OE-Core rev: cbcba43c18d67aea0ba41f019b357fbec6570ee1)

---
 meta/recipes-core/sysvinit/sysvinit_2.88dsf.bb | 1 +
 1 file changed, 1 insertion(+)
meta/recipes-core/sysvinit/sysvinit_2.88dsf.bb +1 -0
index bfc1283f73..39f612be1f 100644
@@ -31,6 +31,7 @@ B = "${S}/src"
 
 inherit update-alternatives distro_features_check
 DEPENDS_append = " update-rc.d-native base-passwd virtual/crypt"
+do_package_setscene[depends] = "${MLPREFIX}base-passwd:do_populate_sysroot"
 
 REQUIRED_DISTRO_FEATURES = "sysvinit"
 
Signature · 1 line
2.17.1
#28 lore raw

PATCH 28/29 libevdev: Fix determinism issue

In reply to Armin Kuster

From: Richard Purdie <[email protected]>

We need to sort python dict output to be deterministic and generate consistent header files.

(From OE-Core rev: 75e4cedb986379db2e8a897df52ee1363f9a9a80)

(From OE-Core rev: fc6e8e527698a82bf8047d02e0e792c4a1a64449)

---
 .../libevdev/libevdev/determinism.patch       | 34 +++++++++++++++++++
 .../libevdev/libevdev_1.8.0.bb                |  3 +-
 2 files changed, 36 insertions(+), 1 deletion(-)
 create mode 100644 meta/recipes-support/libevdev/libevdev/determinism.patch
meta/recipes-support/libevdev/libevdev/determinism.patch new file +34 -0
new file mode 100644
index 0000000000..33a6076b78
@@ -0,0 +1,34 @@
+The order of dict values is not deterministic leading to differing header file generation.
+Sort to remove this inconsistency.
+
+RP 2020/2/7
+
+Signed-off-by: Richard Purdie <[email protected]>
+Upstream-Status: Pending
+
+Index: a/libevdev/make-event-names.py
+===================================================================
+--- a/libevdev/make-event-names.py
++++ b/libevdev/make-event-names.py
+@@ -67,10 +67,10 @@ def print_bits(bits, prefix):
+ 	if  not hasattr(bits, prefix):
+ 		return
+ 	print("static const char * const %s_map[%s_MAX + 1] = {" % (prefix, prefix.upper()))
+-	for val, name in list(getattr(bits, prefix).items()):
++	for val, name in sorted(list(getattr(bits, prefix).items())):
+ 		print("	[%s] = \"%s\"," % (name, name))
+ 	if prefix == "key":
+-		for val, name in list(getattr(bits, "btn").items()):
++		for val, name in sorted(list(getattr(bits, "btn").items())):
+ 			print("	[%s] = \"%s\"," % (name, name))
+ 	print("};")
+ 	print("")
+@@ -111,7 +111,7 @@ def print_lookup(bits, prefix):
+ 	if not hasattr(bits, prefix):
+ 		return
+ 
+-	names = list(getattr(bits, prefix).items())
++	names = sorted(list(getattr(bits, prefix).items()))
+ 	if prefix == "btn":
+ 		names = names + btn_additional;
+ 
meta/recipes-support/libevdev/libevdev_1.8.0.bb +2 -1
index 84274987d7..46ed5d786a 100644
@@ -6,7 +6,8 @@ LICENSE = "MIT-X"
 LIC_FILES_CHKSUM = "file://COPYING;md5=75aae0d38feea6fda97ca381cb9132eb \
                     file://libevdev/libevdev.h;endline=21;md5=7ff4f0b5113252c2f1a828e0bbad98d1"
 
-SRC_URI = "http://www.freedesktop.org/software/libevdev/${BP}.tar.xz"
+SRC_URI = "http://www.freedesktop.org/software/libevdev/${BP}.tar.xz \
+           file://determinism.patch"
 SRC_URI[md5sum] = "879631080be18526737e33b63d848039"
 SRC_URI[sha256sum] = "20d3cae4efd277f485abdf8f2a7c46588e539998b5a08c2c4d368218379d4211"
 
Signature · 1 line
2.17.1
#29 lore raw

PATCH 29/29 ncurses: Fix reproducibility issue

In reply to Armin Kuster

From: Richard Purdie <[email protected]>

The build was deciding whether to rename manpages based upon the presence of /etc/debian_version. Be explicit about the configuration instead and ensure determinism.

[YOCTO #13781]

(From OE-Core rev: 09c8a28893e7ca94a44232d802e1cb02a8f34b87)

(From OE-Core rev: 3d114c08c32578426646f5d97769d2c66fea06cb)

---
 meta/recipes-core/ncurses/ncurses.inc | 1 +
 1 file changed, 1 insertion(+)
meta/recipes-core/ncurses/ncurses.inc +1 -0
index 5f2cc35823..b7bf4c0d81 100644
@@ -87,6 +87,7 @@ ncurses_configure() {
 	        --disable-rpath-hack \
 		${EXCONFIG_ARGS} \
 	        --with-manpage-format=normal \
+	        --without-manpage-renames \
 	        --disable-stripping \
 	        "$@" || return 1
 	cd ..
Signature · 1 line
2.17.1
#30 lore raw
In reply to Armin Kuster

Hi Armin,

Quoted 4 lines · Armin Kuster

These are the additional changes to help address reproducibility issues and additional fixes we would like to be included in 3.0.2

Please have comments back by Tuesday

I have two questions/comments:

1. When I look at the zeus-next branch, why do I see only patches 7 to 29 from this series applied and what about patches 1 to 6? Am I missing something?

2. Patch 5 (devtool/standard.py: Allow recipe to disable menuconfig logic) goes hand in hand with a change to the u-boot recipe, that is also in master (c634b8db1a8b). This patch seems to be missing here.

Regards,
Frieder

Quoted 119 lines

The following changes since commit 9b1bf083129be2b849db52d4f0eda9eb6077c97e:

   python2: add ntpath (2020-02-02 18:19:50 -0800)

are available in the Git repository at:

git://git.openembedded.org/openembedded-core-contrib stable/zeus-nut
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=stable/zeus-nut

Alejandro del Castillo (1):
opkg-utils: upgrade to version 0.4.2

Alexander Kanavin (1):
perl: do not install files that contain build host specific data

Anuj Mittal (3):

   Revert "bzip2: Fix CVE-2019-12900"
   curl: fix CVE-2019-15601
   cpio: fix CVE-2019-14866

Joshua Watt (2):
classes/reproducible_build: Read SDE file later
mc: Fix build reproducibility

Lee Chee Yang (1):

   rsync: whitelist CVE-2017-16548

Richard Purdie (17):
opkg-utils: Fix reproducibility issues in opkg-build oeqa/reproducible: Improve test output and ensure deb+ipk compared
sudo: Set vardir deterministically
libxshmfence: Set shm directory deterministically
mc: Set zipinfo presence determinstically
mc: Fix manpage date indeterminism
tar: Fix build determinism, disable rsh
patch: Extend to native/nativesdk and depend upon
libidn2: Fix reproducibility issue
perl: Fix various reproducibile build issues
openssl: Fix reproducibility issue
iputils: Fix build determinism
libinput: Fix determinism issue
libgcrypt: Fix determinism issue
sysvinit: Fix Reproducibility issue
libevdev: Fix determinism issue
ncurses: Fix reproducibility issue

Ross Burton (2):
gtk+3: sort resources for reproducible binaries
sudo: specify where target tools are

Taras Kondratiuk via Openembedded-core (1):
gcc-9.2: fix bug #91102 'aarch64 ICE on Linux kernel with -Os'

Tom Hochstein (1):
devtool/standard.py: Allow recipe to disable menuconfig logic

  meta/classes/patch.bbclass                    |   7 +
  meta/classes/reproducible_build.bbclass       |  40 ++-
  meta/lib/oeqa/selftest/cases/reproducible.py  |   9 +-
  .../openssl/openssl/reproducible.patch        |  32 ++
  .../openssl/openssl_1.1.1d.bb                 |   1 +
  meta/recipes-core/meta/buildtools-tarball.bb  |   1 +
  meta/recipes-core/ncurses/ncurses.inc         |   1 +
  .../recipes-core/sysvinit/sysvinit_2.88dsf.bb |   1 +
  meta/recipes-devtools/gcc/gcc-9.2.inc         |   1 +
  ...02-aarch64-ICE-on-Linux-kernel-with-.patch |  95 ++++++
  ...Switch-all-scripts-to-use-Python-3.x.patch | 113 -------
  ...ld-clamp-mtimes-to-SOURCE_DATE_EPOCH.patch |  44 ---
  .../opkg-utils/fix-reproducibility.patch      |  32 ++
  .../opkg-utils/opkg-utils/pipefail.patch      |  31 --
  ...pkg-utils_0.4.1.bb => opkg-utils_0.4.2.bb} |  13 +-
  meta/recipes-devtools/patch/patch_2.7.6.bb    |   3 +
  .../perl/files/determinism.patch              |  81 +++++
  meta/recipes-devtools/perl/perl-ptest.inc     |   3 +
  meta/recipes-devtools/perl/perl_5.30.0.bb     |   4 +
  meta/recipes-devtools/rsync/rsync_3.1.3.bb    |   3 +
  .../bzip2/bzip2-1.0.6/CVE-2019-12900.patch    |  36 --
  .../cpio/cpio-2.12/CVE-2019-14866.patch       | 316 ++++++++++++++++++
  meta/recipes-extended/cpio/cpio_2.12.bb       |   1 +
  .../iputils/iputils_s20190709.bb              |   3 +-
  meta/recipes-extended/libidn/libidn2_2.2.0.bb |   3 +-
  ...Add-option-to-control-configure-args.patch |  99 ++++++
  .../recipes-extended/mc/files/nomandate.patch |  21 ++
  meta/recipes-extended/mc/mc_4.8.23.bb         |   7 +-
  meta/recipes-extended/sudo/sudo.inc           |   2 +-
  meta/recipes-extended/sudo/sudo_1.8.27.bb     |  10 +-
  meta/recipes-extended/tar/tar_1.32.bb         |   2 +
  .../gtk+/gtk+3/sort-resources.patch           |  19 ++
  meta/recipes-gnome/gtk+/gtk+3_3.24.8.bb       |   1 +
  .../wayland/libinput/determinism.patch        |  21 ++
  .../wayland/libinput_1.14.1.bb                |   4 +-
  .../xorg-lib/libxshmfence_1.3.bb              |   2 +
  .../curl/curl/CVE-2019-15601.patch            |  46 +++
  meta/recipes-support/curl/curl_7.66.0.bb      |   1 +
  .../libevdev/libevdev/determinism.patch       |  34 ++
  .../libevdev/libevdev_1.8.0.bb                |   3 +-
  .../libgcrypt/files/determinism.patch         |  32 ++
  .../libgcrypt/libgcrypt_1.8.4.bb              |   1 +
  scripts/lib/devtool/standard.py               |   6 +-
  43 files changed, 933 insertions(+), 252 deletions(-)

create mode 100644 meta/recipes-connectivity/openssl/openssl/reproducible.patch
create mode 100644 meta/recipes-devtools/gcc/gcc-9.2/re-PR-target-91102-aarch64-ICE-on-Linux-kernel-with-.patch delete mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/0001-Switch-all-scripts-to-use-Python-3.x.patch delete mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/0001-opkg-build-clamp-mtimes-to-SOURCE_DATE_EPOCH.patch create mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/fix-reproducibility.patch
delete mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/pipefail.patch
rename meta/recipes-devtools/opkg-utils/{opkg-utils_0.4.1.bb => opkg-utils_0.4.2.bb} (83%)
create mode 100644 meta/recipes-devtools/perl/files/determinism.patch
delete mode 100644 meta/recipes-extended/bzip2/bzip2-1.0.6/CVE-2019-12900.patch
create mode 100644 meta/recipes-extended/cpio/cpio-2.12/CVE-2019-14866.patch
create mode 100644 meta/recipes-extended/mc/files/0001-Add-option-to-control-configure-args.patch create mode 100644 meta/recipes-extended/mc/files/nomandate.patch
create mode 100644 meta/recipes-gnome/gtk+/gtk+3/sort-resources.patch
create mode 100644 meta/recipes-graphics/wayland/libinput/determinism.patch
create mode 100644 meta/recipes-support/curl/curl/CVE-2019-15601.patch
create mode 100644 meta/recipes-support/libevdev/libevdev/determinism.patch
create mode 100644 meta/recipes-support/libgcrypt/files/determinism.patch

#31 lore raw
Quoted 14 lines

-----Original Message-----
From: [email protected] <openembedded-core- [email protected]> On Behalf Of Schrempf Frieder
Sent: den 10 februari 2020 09:07
To: Armin Kuster <[email protected]>; openembedded- [email protected]
Subject: Re: [OE-core] [zeus 00/29] Patch review

Hi Armin,

Quoted 3 lines · Armin Kuster

These are the additional changes to help address reproducibility issues and additional fixes we would like to be included in 3.0.2

Quoted 7 lines
Quoted 1 line

Please have comments back by Tuesday

I have two questions/comments:

1. When I look at the zeus-next branch, why do I see only patches 7 to 29 from this series applied and what about patches 1 to 6? Am I missing something?

That is a really good question, and one I too would like an answer to.

Quoted 3 lines

2. Patch 5 (devtool/standard.py: Allow recipe to disable menuconfig logic) goes hand in hand with a change to the u-boot recipe, that is also in master (c634b8db1a8b). This patch seems to be missing here.

That one is probably my fault. We have a local recipe that defines a menuconfig task and thus I needed the correction for devtool and sent a request for it to be backported. However, since we do not use u-boot, I did not think of that part, which obviously should be backported too.

Quoted 2 lines

Regards,
Frieder

//Peter

#32 lore raw
Quoted 20 lines · Peter Kjellerstedt
Quoted 20 lines

-----Original Message-----
From: [email protected] <openembedded-core-
[email protected]> On Behalf Of Schrempf Frieder
Sent: den 10 februari 2020 09:07
To: Armin Kuster <[email protected]>; openembedded- [email protected]
Subject: Re: [OE-core] [zeus 00/29] Patch review

Hi Armin,

Quoted 5 lines · Armin Kuster

These are the additional changes to help address reproducibility issues
and additional fixes we would like to be included in 3.0.2

Please have comments back by Tuesday

I have two questions/comments:

Quoted 8 lines
Quoted 5 lines

1. When I look at the zeus-next branch, why do I see only patches 7 to
29 from this series applied and what about patches 1 to 6? Am I missing
something?

That is a really good question, and one I too would like an answer to.

I tried to help Armin by staging a few patches related to reproducibility and it was easiest for me to do that via the -next branch. It seems I mainly caused confusion, sorry :(

Cheers,

Richard

author
#33 lore raw
In reply to Peter Kjellerstedt
Quoted 22 lines · Peter Kjellerstedt
Quoted 14 lines

-----Original Message-----
From: [email protected] <openembedded-core- [email protected]> On Behalf Of Schrempf Frieder
Sent: den 10 februari 2020 09:07
To: Armin Kuster <[email protected]>; openembedded- [email protected]
Subject: Re: [OE-core] [zeus 00/29] Patch review

Hi Armin,

Quoted 3 lines · Armin Kuster

These are the additional changes to help address reproducibility issues and additional fixes we would like to be included in 3.0.2

Quoted 6 lines
Quoted 1 line

Please have comments back by Tuesday

I have two questions/comments:

1. When I look at the zeus-next branch, why do I see only patches 7 to 29 from this series applied and what about patches 1 to 6? Am I missing something?

That is a really good question, and one I too would like an answer to.

Quoted 7 lines
Quoted 3 lines

2. Patch 5 (devtool/standard.py: Allow recipe to disable menuconfig logic) goes hand in hand with a change to the u-boot recipe, that is also in master (c634b8db1a8b). This patch seems to be missing here.

That one is probably my fault. We have a local recipe that defines a menuconfig task and thus I needed the correction for devtool and sent a request for it to be backported. However, since we do not use u-boot, I did not think of that part, which obviously should be backported too.

will find that one.

thanks,
Armin

Quoted 5 lines
Quoted 3 lines

Regards,
Frieder

//Peter