Skip to contentSkip to conversation

opkg-utils: backport a patch to fix a sstate timestamp issue

PATCH In review [email protected] 2 participants 2 messages Apr 5, 2019 – Apr 25, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 opkg-utils: backport a patch to fix a sstate timestamp issue

From: Ming Liu <[email protected]>

When using sstate, two parallel builds can produce two packages with the same mtime but different checksums. When later one of those two builds fetches the others ipk, the package index does not get udpated properly (since mtime matches). This ends up with messages such as:
Downloading file:/../tmp/work/../image/...ipk.
Removing corrupt package file /../sysroot/../var/cache/opkg/volatile/...ipk

However, in that case, ctime is different. Use ctime instead of mtime to prevent failures like this.

---
 ...pkg-make-index-use-ctime-instead-of-mtime.patch | 59 ++++++++++++++++++++++
 .../opkg-utils/opkg-utils_0.3.6.bb                 |  1 +
 2 files changed, 60 insertions(+)
 create mode 100644 meta/recipes-devtools/opkg-utils/opkg-utils/0001-opkg-make-index-use-ctime-instead-of-mtime.patch
meta/recipes-devtools/opkg-utils/opkg-utils/0001-opkg-make-index-use-ctime-instead-of-mtime.patch new file +59 -0
new file mode 100644
index 0000000..19778ac
@@ -0,0 +1,59 @@
+From 0cd38bb1bdcdbfc091014a1f39d015a1586a33e6 Mon Sep 17 00:00:00 2001
+From: Stefan Agner <[email protected]>
+Date: Fri, 19 Oct 2018 17:38:21 +0200
+Subject: [PATCH] opkg-make-index: use ctime instead of mtime
+
+Upstream-Status: Backport
+
+When using sstate, two parallel builds can produce two packages
+with the same mtime but different checksums. When later one of
+those two builds fetches the others ipk, the package index does
+not get udpated properly (since mtime matches). This ends up with
+messages such as:
+  Downloading file:/../tmp/work/../image/...ipk.
+  Removing corrupt package file /../sysroot/../var/cache/opkg/volatile/...ipk
+
+However, in that case, ctime is different. Use ctime instead of
+mtime to prevent failures like this.
+
+Suggested-by: Khem Raj <[email protected]>
+Signed-off-by: Stefan Agner <[email protected]>
+Acked-by: Richard Purdie <[email protected]>
+Acked-by: Khem Raj <[email protected]>
+Signed-off-by: Alejandro del Castillo <[email protected]>
+Signed-off-by: Ming Liu <[email protected]>
+---
+ opkg-make-index | 6 +++---
+ 1 file changed, 3 insertions(+), 3 deletions(-)
+
+diff --git a/opkg-make-index b/opkg-make-index
+index 3227fc0..db7bf64 100755
+--- a/opkg-make-index
++++ b/opkg-make-index
+@@ -115,12 +115,12 @@ for abspath in files:
+      pkg = None
+      fnameStat = os.stat(abspath)
+      if filename in old_pkg_hash:
+-          if filename in pkgsStamps and int(fnameStat.st_mtime) == pkgsStamps[filename]:
++          if filename in pkgsStamps and int(fnameStat.st_ctime) == pkgsStamps[filename]:
+             if (verbose):
+                sys.stderr.write("Found %s in Packages\n" % (filename,))
+             pkg = old_pkg_hash[filename]
+           else:
+-               sys.stderr.write("Found %s in Packages, but mtime differs - re-reading\n" % (filename,))
++               sys.stderr.write("Found %s in Packages, but ctime differs - re-reading\n" % (filename,))
+ 
+      if not pkg:
+           if (verbose):
+@@ -137,7 +137,7 @@ for abspath in files:
+      else:
+           old_filename = ""
+      s = packages.add_package(pkg, opt_a)
+-     pkgsStamps[filename] = fnameStat.st_mtime
++     pkgsStamps[filename] = fnameStat.st_ctime
+      if s == 0:
+           if old_filename:
+                # old package was displaced by newer
+-- 
+2.7.4
+
meta/recipes-devtools/opkg-utils/opkg-utils_0.3.6.bb +1 -0
index 4c41774..41cf11c 100644
@@ -14,6 +14,7 @@ SRC_URI = "http://git.yoctoproject.org/cgit/cgit.cgi/${BPN}/snapshot/${BPN}-${PV
            file://threaded-xz.patch \
            file://pigz.patch \
            file://0001-update-alternatives-Fix-link-relocation-support.patch \
+           file://0001-opkg-make-index-use-ctime-instead-of-mtime.patch \
 "
 SRC_URI_append_class-native = " file://tar_ignore_error.patch"
 UPSTREAM_CHECK_URI = "http://git.yoctoproject.org/cgit/cgit.cgi/opkg-utils/refs/"
Signature · 1 line
2.7.4
#1 lore raw
Quoted 10 lines · On 05.04.2019 16:22, [email protected] wrote:

From: Ming Liu <[email protected]>

When using sstate, two parallel builds can produce two packages with the same mtime but different checksums. When later one of those two builds fetches the others ipk, the package index does not get udpated properly (since mtime matches). This ends up with messages such as:
Downloading file:/../tmp/work/../image/...ipk.
Removing corrupt package file /../sysroot/../var/cache/opkg/volatile/...ipk

Quoted 2 lines

However, in that case, ctime is different. Use ctime instead of mtime to prevent failures like this.

FWIW,
Acked-by Stefan Agner <[email protected]>

I fixed this in master, and it actually helps us resolving issues we see on CI on thud branch.

--
Stefan

Quoted 98 lines
---
 ...pkg-make-index-use-ctime-instead-of-mtime.patch | 59 ++++++++++++++++++++++
 .../opkg-utils/opkg-utils_0.3.6.bb                 |  1 +
 2 files changed, 60 insertions(+)
 create mode 100644

meta/recipes-devtools/opkg-utils/opkg-utils/0001-opkg-make-index-use-ctime-instead-of-mtime.patch

diff --git a/meta/recipes-devtools/opkg-utils/opkg-utils/0001-opkg-make-index-use-ctime-instead-of-mtime.patch b/meta/recipes-devtools/opkg-utils/opkg-utils/0001-opkg-make-index-use-ctime-instead-of-mtime.patch new file mode 100644
index 0000000..19778ac
--- /dev/null
+++
b/meta/recipes-devtools/opkg-utils/opkg-utils/0001-opkg-make-index-use-ctime-instead-of-mtime.patch

meta/recipes-devtools/opkg-utils/opkg-utils/0001-opkg-make-index-use-ctime-instead-of-mtime.patch new file +37 -0
@@ -0,0 +1,59 @@
+From 0cd38bb1bdcdbfc091014a1f39d015a1586a33e6 Mon Sep 17 00:00:00 2001
+From: Stefan Agner <[email protected]>
+Date: Fri, 19 Oct 2018 17:38:21 +0200
+Subject: [PATCH] opkg-make-index: use ctime instead of mtime
+
+Upstream-Status: Backport
+
+When using sstate, two parallel builds can produce two packages
+with the same mtime but different checksums. When later one of
+those two builds fetches the others ipk, the package index does
+not get udpated properly (since mtime matches). This ends up with
+messages such as:
+  Downloading file:/../tmp/work/../image/...ipk.
+  Removing corrupt package file /../sysroot/../var/cache/opkg/volatile/...ipk
+
+However, in that case, ctime is different. Use ctime instead of
+mtime to prevent failures like this.
+
+Suggested-by: Khem Raj <[email protected]>
+Signed-off-by: Stefan Agner <[email protected]>
+Acked-by: Richard Purdie <[email protected]>
+Acked-by: Khem Raj <[email protected]>
+Signed-off-by: Alejandro del Castillo <[email protected]>
+Signed-off-by: Ming Liu <[email protected]>
+---
+ opkg-make-index | 6 +++---
+ 1 file changed, 3 insertions(+), 3 deletions(-)
+
+diff --git a/opkg-make-index b/opkg-make-index
+index 3227fc0..db7bf64 100755
+--- a/opkg-make-index
++++ b/opkg-make-index
+@@ -115,12 +115,12 @@ for abspath in files:
+      pkg = None
+      fnameStat = os.stat(abspath)
+      if filename in old_pkg_hash:
+-          if filename in pkgsStamps and int(fnameStat.st_mtime) ==

pkgsStamps[filename]:
++ if filename in pkgsStamps and int(fnameStat.st_ctime) ==
pkgsStamps[filename]:

meta/recipes-devtools/opkg-utils/opkg-utils/0001-opkg-make-index-use-ctime-instead-of-mtime.patch new file +5 -0
+             if (verbose):
+                sys.stderr.write("Found %s in Packages\n" % (filename,))
+             pkg = old_pkg_hash[filename]
+           else:
+-               sys.stderr.write("Found %s in Packages, but mtime

differs - re-reading\n" % (filename,))
++ sys.stderr.write("Found %s in Packages, but ctime
differs - re-reading\n" % (filename,))
+

meta/recipes-devtools/opkg-utils/opkg-utils/0001-opkg-make-index-use-ctime-instead-of-mtime.patch new file +14 -0
+      if not pkg:
+           if (verbose):
+@@ -137,7 +137,7 @@ for abspath in files:
+      else:
+           old_filename = ""
+      s = packages.add_package(pkg, opt_a)
+-     pkgsStamps[filename] = fnameStat.st_mtime
++     pkgsStamps[filename] = fnameStat.st_ctime
+      if s == 0:
+           if old_filename:
+                # old package was displaced by newer
+-- 
+2.7.4
+
meta/recipes-devtools/opkg-utils/opkg-utils_0.3.6.bb +0 -0
index 4c41774..41cf11c 100644
@@ -14,6 +14,7 @@ SRC_URI =

"http://git.yoctoproject.org/cgit/cgit.cgi/${BPN}/snapshot/${BPN}-${PV

            file://threaded-xz.patch \
            file://pigz.patch \
            file://0001-update-alternatives-Fix-link-relocation-support.patch \

+ file://0001-opkg-make-index-use-ctime-instead-of-mtime.patch \
"
SRC_URI_append_class-native = " file://tar_ignore_error.patch" UPSTREAM_CHECK_URI =
"http://git.yoctoproject.org/cgit/cgit.cgi/opkg-utils/refs/"

Signature · 1 line
2.7.4