Skip to contentSkip to conversation

cve-update-db-native: update the CVE database once a day only

PATCH Superseded Marta Rybczynska 3 participants 6 messages Apr 29, 2022 – May 2, 2022

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 cve-update-db-native: update the CVE database once a day only

The update of the NVD database was expected to happen once per hour. However, the database file date changes only if the content was actually updated. In practice, the check worked for the first hour after the new download.

As the NVD database changes usually only once a day, we can just update it less frequently.

---
 meta/recipes-core/meta/cve-update-db-native.bb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
meta/recipes-core/meta/cve-update-db-native.bb +2 -2
index e5822cee58..af39480dda 100644
@@ -43,10 +43,10 @@ python do_fetch() {
         if os.path.exists(db_file):
             os.remove(db_file)
 
-    # Don't refresh the database more than once an hour
+    # The NVD database changes once a day, so no need to update more frequently
     try:
         import time
-        if time.time() - os.path.getmtime(db_file) < (60*60):
+        if time.time() - os.path.getmtime(db_file) < (24*60*60):
             bb.debug(2, "Recently updated, skipping")
             return
     except OSError:
Signature · 1 line
2.33.0
#1 lore raw

PATCH 2/2 cve-update-db-native: allow an option to force the CVE database update

Add a new variable FORCE_CVE_DB_UPDATE allowing the user to force the database update, if the default update frequency is too low.

---
 meta/recipes-core/meta/cve-update-db-native.bb | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
meta/recipes-core/meta/cve-update-db-native.bb +7 -3
index af39480dda..d89f79f310 100644
@@ -13,6 +13,7 @@ deltask do_install
 deltask do_populate_sysroot
 
 NVDCVE_URL ?= "https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-"
+FORCE_CVE_DB_UPDATE ?= "0"
 
 python () {
     if not bb.data.inherits_class("cve-check", d):
@@ -44,11 +45,14 @@ python do_fetch() {
             os.remove(db_file)
 
     # The NVD database changes once a day, so no need to update more frequently
+    # Allow the user to force-update
     try:
         import time
-        if time.time() - os.path.getmtime(db_file) < (24*60*60):
-            bb.debug(2, "Recently updated, skipping")
-            return
+        if d.getVar("FORCE_CVE_DB_UPDATE") == "0":
+            if time.time() - os.path.getmtime(db_file) < (24*60*60):
+                bb.debug(2, "Recently updated, skipping")
+                return
+
     except OSError:
         pass
 
Signature · 1 line
2.33.0
#2 lore raw

Re: [OE-core] [PATCH 2/2] cve-update-db-native: allow an option to force the CVE database update

Hi Marta,

This explains why the CVE database update seemed to happen far more frequently than it should. Thanks for digging into it.

Quoted 3 lines · Marta Rybczynska

Add a new variable FORCE_CVE_DB_UPDATE allowing the user to force the database update, if the default update frequency is too low.

Just an idea, maybe instead of a boolean, the variable could specify the interval, eg:

# Interval between CVE database updates, in seconds. # Set to "0" to to force an update of the database. CVE_DATABASE_UPDATE_INTERVAL ?= "24*60*60"

Regards,
Ralph

#3 lore raw

Re: [OE-core] [PATCH 2/2] cve-update-db-native: allow an option to force the CVE database update

Quoted 11 lines · Ralph Siemsen

Hi Marta,

This explains why the CVE database update seemed to happen far more frequently than it should. Thanks for digging into it.

Quoted 3 lines · Marta Rybczynska

Add a new variable FORCE_CVE_DB_UPDATE allowing the user to force the database update, if the default update frequency is too low.

Quoted 8 lines

Just an idea, maybe instead of a boolean, the variable could specify the interval, eg:

# Interval between CVE database updates, in seconds. # Set to "0" to to force an update of the database. CVE_DATABASE_UPDATE_INTERVAL ?= "24*60*60"

This is a good idea, thank you Ralph, I like it. I'll be sending a v2 shortly.

Regards,
Marta

#4 lore raw

Re: [OE-core] [PATCH 2/2] cve-update-db-native: allow an option to force the CVE database update

Quoted 10 lines · Marta Rybczynska
Quoted 5 lines · Ralph Siemsen

# Interval between CVE database updates, in seconds. # Set to "0" to to force an update of the database. CVE_DATABASE_UPDATE_INTERVAL ?= "24*60*60"

This is a good idea, thank you Ralph, I like it. I'll be sending a v2 shortly.

Thanks for this. I've tested it locally, on the fist run, the CVE database was fetched (it took quite a while, for some reason). On subsequent runs, no fetch occurs, so this seems to be working.

I then set CVE_DB_UPDATE_INTERVAL = "3600" in my local.conf, and ran the build again. As it had been over an hour since the first build, the database was downloaded again. The timestamp on nvdcve_1.1.db did not change (as noted in one of your commit descriptions).

So, it seems to be working correctly. I will re-test tomorrow (eg. after 24 hours) with the interval set back to default.

One minor point of confusion is that the log still shows "NOTE: recipe cve-update-db-native-1.0-r0: task do_fetch: Started" even when the download is skipped. This is of course understandable when looking at the python code, the check is within the do_fetch function. There is probably no simple way to avoid this from being displayed. And most users won't notice anyway. But it initially confused me about what was happening.

Regards,
Ralph

#5 lore raw

Re: [OE-core] [PATCH 2/2] cve-update-db-native: allow an option to force the CVE database update

Quoted 25 lines · Ralph Siemsen
Quoted 10 lines · Marta Rybczynska
Quoted 5 lines · Ralph Siemsen

# Interval between CVE database updates, in seconds. # Set to "0" to to force an update of the database. CVE_DATABASE_UPDATE_INTERVAL ?= "24*60*60"

This is a good idea, thank you Ralph, I like it. I'll be sending a v2 shortly.

Thanks for this. I've tested it locally, on the fist run, the CVE database was fetched (it took quite a while, for some reason). On subsequent runs, no fetch occurs, so this seems to be working.

I then set CVE_DB_UPDATE_INTERVAL = "3600" in my local.conf, and ran the build again. As it had been over an hour since the first build, the database was downloaded again. The timestamp on nvdcve_1.1.db did not change (as noted in one of your commit descriptions).

So, it seems to be working correctly. I will re-test tomorrow (eg. after 24 hours) with the interval set back to default.

Quoted 7 lines

One minor point of confusion is that the log still shows "NOTE: recipe cve-update-db-native-1.0-r0: task do_fetch: Started" even when the download is skipped. This is of course understandable when looking at the python code, the check is within the do_fetch function. There is probably no simple way to avoid this from being displayed. And most users won't notice anyway. But it initially confused me about what was happening.

yeah you can avoid it unless another logic outside of this function is invoked. perhaps we can add a diagnostic inside the do_fetch to spill out the state information
and informing like "no fetch needed" or "database uptodate" or somesuch

Quoted 12 lines

Regards,
Ralph

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165172): https://lists.openembedded.org/g/openembedded-core/message/165172
Mute This Topic: https://lists.openembedded.org/mt/90771095/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-