utils.bbclass: simplify checksum check, prepare for checksums.ini removal

* only RFC, because it needs more testing and bb.note for every checksum only in checksums.ini is maybe too much (but I'm ready to move all checksums in some semi-automated way).
* unify OE_STRICT_CHECKSUMS and OE_ALLOW_INSECURE_DOWNLOADS, one option for insane people should be enough, when the later is enabled, don't raise Exception even for missing md5sum/oe_sha256sum command or different checksums
* show note, when there are checksums only in checksums.ini (prepare for script for moving all to recipes)
* parse checksums.ini only when there is no checksum in recipe (could be faster, but for more checked items in SRC_URI it is parsed repeatedly)
* if one checksum doesn't match then count and show both (md5 as well as sha256) - usefull for copy&paste checksums for new recipe.

---
 classes/base.bbclass  |   20 +-----
 classes/utils.bbclass |  189 ++++++++++++++++++++++---------------------------
 2 files changed, 86 insertions(+), 123 deletions(-)

index e865738..28cd4a6 100644
@@ -112,24 +112,6 @@ python base_do_fetch() {
 		raise bb.build.FuncFailed("Unknown fetch Error: %s" % value)
 
 
-	# Verify the SHA and MD5 sums we have in OE and check what do
-	# in
-	checksum_paths = bb.data.getVar('BBPATH', d, True).split(":")
-
-	# reverse the list to give precedence to directories that
-	# appear first in BBPATH
-	checksum_paths.reverse()
-
-	checksum_files = ["%s/conf/checksums.ini" % path for path in checksum_paths]
-	try:
-		parser = base_chk_load_parser(checksum_files)
-	except ValueError:
-		bb.note("No conf/checksums.ini found, not checking checksums")
-		return
-	except:
-		bb.note("Creating the CheckSum parser failed: %s:%s" % (sys.exc_info()[0], sys.exc_info()[1]))
-		return
-
 	pv = bb.data.getVar('PV', d, True)
 	pn = bb.data.getVar('PN', d, True)
 

@@ -146,7 +128,7 @@ python base_do_fetch() {
 				if not "name" in params and first_uri:
 					first_uri = False
 					params["name"] = ""
-				if not (base_chk_file_vars(parser, localpath, params, d) or base_chk_file(parser, pn, pv,uri, localpath, d)):
+				if not base_chk_file(pn, pv, uri, localpath, params, d):
 					if not bb.data.getVar("OE_ALLOW_INSECURE_DOWNLOADS", d, True):
 						bb.fatal("%s-%s: %s has no checksum defined, cannot check archive integrity" % (pn,pv,uri))
 					else:

1.7.0.4

I find it very useful to distinguish between the missing checksum and invalid checksum cases. The latter should really never be allowed, at all, period, imo, but the missing checksum should have an option. If we aren't ready to remove the ability to allow invalid, then we need to be able to control the two cases independently, or via two different values in the variable that controls the behavior.

* show note, when there are checksums only in checksums.ini (prepare for

This sounds good, though it may be something best relegated to an explicit sanity check, depending on how much it clutters the output. May want to log it to a file like tmp/legacy-staging.log, also.

"Could be" .. sounds like this isn't ready to go in yet, need to do profiling. Changing something because it "could" be good is best done in proof of concept code, not as a part of a single patch like this one.

This sounds good.

Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

OK fair enough, I've expected this.. (that's also why it's RFC :)). Do you agree that with enabled OE_ALLOW_INSECURE_DOWNLOADS it should continue even when it's not possible to check checksum (like missing md5sum/oe_sha256sum command)?.

Both those points are assuming I'll be allowed to push "recipes: move checksums from checksums.ini to recipes", which is being prepared ATM with -c fetchall on all recipes. After this patch both bb.note output as well as parsing checksums.ini should be needed only in rare cases.

Thanks for comments,

uin:136542059                jid:[email protected]
Jansa Martin                 sip:[email protected] 
JaMa