Skip to contentSkip to conversation

cve-check: ensure database is closed on error

RFC PATCH Ralph Siemsen 1 participant 1 message May 13, 2022 1 message not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw

Ping. Any thoughts on this change?

Regards,
Ralph

Quoted 83 lines · Ralph Siemsen via lists.openembedded.org

In case of an error during download or parse of NVD JSON files, the previously opened sqlite3 database should be closed. Also any pending transactions should be flushed using conn.commit(). Otherwise there can be a "hot journal" left behind, which can cause a subsequent read-only connection to fail.

So instead of doing "return" to bail out early, instead "break" out of the loop. The existing conn.commit() and conn.close() will be called.

---

I'm not entirely confident in this, would appreciate a review by folks more knowledgeable with python and its sqlite3 binding.

Some further backround information:
- occasionally the cve-check process fails with a traceback error of "attempt to write a readonly database" (see BZ #14110)
- the following commit was added to mitigate:
440f07d211 cve-check: get_cve_info should open the database read-only
- but evidently the original error still occurs sometimes

Current speculation is as follows:
- the database update fails for unrelated reason (network issue)
- this leaves the connection open, potentially with some updates queued (NVD data is fetched with a separate request for each year)
- if a subsequent read-only client tries to access the database, it can fail because there is a "hot journal" from the pending not-yet-committed changes, which cannot be processed since the connection is read-only [1]

[1] https://stackoverflow.com/questions/30082008/attempt-to-write-a-readonly-database-but-im-not

Therefore we should ensure to commit() and close() the database
connection during NVD update, even when there are errors.

 meta/recipes-core/meta/cve-update-db-native.bb | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
meta/recipes-core/meta/cve-update-db-native.bb +3 -4
index e5822cee58..19863c059d 100644
@@ -75,7 +75,7 @@ python do_fetch() {
             except urllib.error.URLError as e:
                 cve_f.write('Warning: CVE db update error, Unable to fetch CVE data.\n\n')
                 bb.warn("Failed to fetch CVE data (%s)" % e.reason)
-                return
+                break
 
             if response:
                 for l in response.read().decode("utf-8").splitlines():
@@ -85,7 +85,7 @@ python do_fetch() {
                         break
                 else:
                     bb.warn("Cannot parse CVE metadata, update failed")
-                    return
+                    break
 
             # Compare with current db last modified date
             c.execute("select DATE from META where YEAR = ?", (year,))
@@ -104,7 +104,7 @@ python do_fetch() {
                 except urllib.error.URLError as e:
                     cve_f.write('Warning: CVE db update error, CVE data is outdated.\n\n')
                     bb.warn("Cannot parse CVE data (%s), update failed" % e.reason)
-                    return
+                    break
             else:
                 bb.debug(2, "Already up to date (last modified %s)" % last_modified)
             # Update success, set the date to cve_check file.
--

2.25.1

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#163786): https://lists.openembedded.org/g/openembedded-core/message/163786
Mute This Topic: https://lists.openembedded.org/mt/90133195/3616981
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-