fetcher2: retry mirror if upstream checksum mismatch V2
cover fetcher2: retry mirror if upstream checksum mismatch V2
This is the V2 patch. the main change is introduce new function update_stamp per Richard's suggestion.
The following changes since commit 7354fc9213f27aa1b643dbe88070437f1ee4c063:
Paul Eggleton (1):
insane.bbclass: skip rdepends QA checks for kernel / modules
are available in the git repository at:
git://git.pokylinux.org/poky-contrib kyu3/verify-checksum
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=kyu3/verify-checksum
Yu Ke (1):
fetcher2: retry mirror if upstream checksum mismatch
bitbake/lib/bb/fetch2/__init__.py | 31 ++++++++++++++++++++----------- 1 files changed, 20 insertions(+), 11 deletions(-)
PATCH 1/1 fetcher2: retry mirror if upstream checksum mismatch
This patch is for [YOCTO #1085] fix.
If the upstream fails a checksum, retry from the MIRROR before giving up. This will add more robust fetching if an upstream serves a bad file or webpage.
fetching of distcc prior to the move from samba -> googlecode is a good example of this.
--- bitbake/lib/bb/fetch2/__init__.py | 31 ++++++++++++++++++++----------- 1 files changed, 20 insertions(+), 11 deletions(-)
bitbake/lib/bb/fetch2/__init__.py +20 -11
@@ -300,6 +300,22 @@ def verify_checksum(u, ud, d): if ud.sha256_expected != sha256data: raise SHA256SumError(ud.localpath, ud.sha256_expected, sha256data, u) +def update_stamp(u, ud, d): + """ + donestamp is file stamp indicating the whole fetching is done + this function update the stamp after verifying the checksum + """ + if os.path.exists(ud.donestamp): + # Touch the done stamp file to show active use of the download + try: + os.utime(ud.donestamp, None) + except: + # Errors aren't fatal here + pass + else: + verify_checksum(u, ud, d) + open(ud.donestamp, 'w').close() + def subprocess_setup(): import signal # Python installs a SIGPIPE handler by default. This is usually not what
@@ -932,6 +948,9 @@ class Fetch(object): if hasattr(m, "build_mirror_data"): m.build_mirror_data(u, ud, self.d) localpath = ud.localpath + # early checksum verify, so that if checksum mismatched, + # fetcher still have chance to fetch from mirror + update_stamp(u, ud, self.d) except bb.fetch2.NetworkAccess: raise
@@ -948,17 +967,7 @@ class Fetch(object): if not localpath or ((not os.path.exists(localpath)) and localpath.find("*") == -1): raise FetchError("Unable to fetch URL %s from any source." % u, u) - if os.path.exists(ud.donestamp): - # Touch the done stamp file to show active use of the download - try: - os.utime(ud.donestamp, None) - except: - # Errors aren't fatal here - pass - else: - # Only check the checksums if we've not seen this item before, then create the stamp - verify_checksum(u, ud, self.d) - open(ud.donestamp, 'w').close() + update_stamp(u, ud, self.d) finally: bb.utils.unlockfile(lf)
Signature · 1 line
1.7.0.4
Richard Purdie
AppliedRe: [PATCH 1/1] fetcher2: retry mirror if upstream checksum mismatch
Quoted 5 lines · Yu Ke
This patch is for [YOCTO #1085] fix.
If the upstream fails a checksum, retry from the MIRROR before giving up. This will add more robust fetching if an upstream serves a bad file or webpage.
Quoted 7 lines
fetching of distcc prior to the move from samba -> googlecode is a good example of this.
Signed-off-by Yu Ke <[email protected]>--- bitbake/lib/bb/fetch2/__init__.py | 31 ++++++++++++++++++++----------- 1 files changed, 20 insertions(+), 11 deletions(-)
Merged to master, thanks.
Richard