cve-check: ensure database is closed on error
Ping. Any thoughts on this change?
Regards,
Ralph
Quoted 83 lines · Ralph Siemsen via lists.openembedded.org
In case of an error during download or parse of NVD JSON files, the previously opened sqlite3 database should be closed. Also any pending transactions should be flushed using
conn.commit(). Otherwise there can be a "hot journal" left behind, which can cause a subsequent read-only connection to fail.So instead of doing "return" to bail out early, instead "break" out of the loop. The existing
conn.commit()andconn.close()will be called.Signed-off-by Ralph Siemsen <[email protected]>---
I'm not entirely confident in this, would appreciate a review by folks more knowledgeable with python and its sqlite3 binding.
Some further backround information:
- occasionally the cve-check process fails with a traceback error of "attempt to write a readonly database" (see BZ #14110)
- the following commit was added to mitigate:
440f07d211cve-check:get_cve_infoshould open the database read-only
- but evidently the original error still occurs sometimesCurrent speculation is as follows:
- the database update fails for unrelated reason (network issue)
- this leaves the connection open, potentially with some updates queued (NVD data is fetched with a separate request for each year)
- if a subsequent read-only client tries to access the database, it can fail because there is a "hot journal" from the pending not-yet-committed changes, which cannot be processed since the connection is read-only [1][1] https://stackoverflow.com/questions/30082008/attempt-to-write-a-readonly-database-but-im-not
Therefore we should ensure to
commit()andclose()the database
connection during NVD update, even when there are errors.meta/recipes-core/meta/cve-update-db-native.bb | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)meta/recipes-core/meta/cve-update-db-native.bb +3
-4@@ -75,7 +75,7 @@ python do_fetch() { except urllib.error.URLError as e: cve_f.write('Warning: CVE db update error, Unable to fetch CVE data.\n\n') bb.warn("Failed to fetch CVE data (%s)" % e.reason) - return + break if response: for l in response.read().decode("utf-8").splitlines():@@ -85,7 +85,7 @@ python do_fetch() { break else: bb.warn("Cannot parse CVE metadata, update failed") - return + break # Compare with current db last modified date c.execute("select DATE from META where YEAR = ?", (year,))@@ -104,7 +104,7 @@ python do_fetch() { except urllib.error.URLError as e: cve_f.write('Warning: CVE db update error, CVE data is outdated.\n\n') bb.warn("Cannot parse CVE data (%s), update failed" % e.reason) - return + break else: bb.debug(2, "Already up to date (last modified %s)" % last_modified) # Update success, set the date to cve_check file. --2.25.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#163786): https://lists.openembedded.org/g/openembedded-core/message/163786
Mute This Topic: https://lists.openembedded.org/mt/90133195/3616981
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-