cve-update-db-native: update the CVE database once a day only
PATCH 1/2 cve-update-db-native: update the CVE database once a day only
The update of the NVD database was expected to happen once per hour. However, the database file date changes only if the content was actually updated. In practice, the check worked for the first hour after the new download.
As the NVD database changes usually only once a day, we can just update it less frequently.
--- meta/recipes-core/meta/cve-update-db-native.bb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
meta/recipes-core/meta/cve-update-db-native.bb +2 -2
@@ -43,10 +43,10 @@ python do_fetch() { if os.path.exists(db_file): os.remove(db_file) - # Don't refresh the database more than once an hour + # The NVD database changes once a day, so no need to update more frequently try: import time - if time.time() - os.path.getmtime(db_file) < (60*60): + if time.time() - os.path.getmtime(db_file) < (24*60*60): bb.debug(2, "Recently updated, skipping") return except OSError:
Signature · 1 line
2.33.0
PATCH 2/2 cve-update-db-native: allow an option to force the CVE database update
Add a new variable FORCE_CVE_DB_UPDATE allowing the user to force the database update, if the default update frequency is too low.
--- meta/recipes-core/meta/cve-update-db-native.bb | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
meta/recipes-core/meta/cve-update-db-native.bb +7 -3
@@ -13,6 +13,7 @@ deltask do_install deltask do_populate_sysroot NVDCVE_URL ?= "https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-" +FORCE_CVE_DB_UPDATE ?= "0" python () { if not bb.data.inherits_class("cve-check", d):
@@ -44,11 +45,14 @@ python do_fetch() { os.remove(db_file) # The NVD database changes once a day, so no need to update more frequently + # Allow the user to force-update try: import time - if time.time() - os.path.getmtime(db_file) < (24*60*60): - bb.debug(2, "Recently updated, skipping") - return + if d.getVar("FORCE_CVE_DB_UPDATE") == "0": + if time.time() - os.path.getmtime(db_file) < (24*60*60): + bb.debug(2, "Recently updated, skipping") + return + except OSError: pass
Signature · 1 line
2.33.0
Re: [OE-core] [PATCH 2/2] cve-update-db-native: allow an option to force the CVE database update
Hi Marta,
This explains why the CVE database update seemed to happen far more frequently than it should. Thanks for digging into it.
Quoted 3 lines · Marta Rybczynska
Add a new variable
FORCE_CVE_DB_UPDATEallowing the user to force the database update, if the default update frequency is too low.
Just an idea, maybe instead of a boolean, the variable could specify the interval, eg:
# Interval between CVE database updates, in seconds. # Set to "0" to to force an update of the database. CVE_DATABASE_UPDATE_INTERVAL ?= "24*60*60"
Regards,
Ralph
Re: [OE-core] [PATCH 2/2] cve-update-db-native: allow an option to force the CVE database update
Quoted 11 lines · Ralph Siemsen
Hi Marta,
This explains why the CVE database update seemed to happen far more frequently than it should. Thanks for digging into it.
Quoted 3 lines · Marta Rybczynska
Add a new variable
FORCE_CVE_DB_UPDATEallowing the user to force the database update, if the default update frequency is too low.
Quoted 8 lines
Just an idea, maybe instead of a boolean, the variable could specify the interval, eg:
# Interval between CVE database updates, in seconds. # Set to "0" to to force an update of the database.
CVE_DATABASE_UPDATE_INTERVAL?= "24*60*60"
This is a good idea, thank you Ralph, I like it. I'll be sending a v2 shortly.
Regards,
Marta
Re: [OE-core] [PATCH 2/2] cve-update-db-native: allow an option to force the CVE database update
Quoted 10 lines · Marta Rybczynska
Quoted 5 lines · Ralph Siemsen
# Interval between CVE database updates, in seconds. # Set to "0" to to force an update of the database.
CVE_DATABASE_UPDATE_INTERVAL?= "24*60*60"This is a good idea, thank you Ralph, I like it. I'll be sending a v2 shortly.
Thanks for this. I've tested it locally, on the fist run, the CVE database was fetched (it took quite a while, for some reason). On subsequent runs, no fetch occurs, so this seems to be working.
I then set CVE_DB_UPDATE_INTERVAL = "3600" in my local.conf, and ran the build again. As it had been over an hour since the first build, the database was downloaded again. The timestamp on nvdcve_1.1.db did not change (as noted in one of your commit descriptions).
So, it seems to be working correctly. I will re-test tomorrow (eg. after 24 hours) with the interval set back to default.
One minor point of confusion is that the log still shows "NOTE: recipe cve-update-db-native-1.0-r0: task do_fetch: Started" even when the download is skipped. This is of course understandable when looking at the python code, the check is within the do_fetch function. There is probably no simple way to avoid this from being displayed. And most users won't notice anyway. But it initially confused me about what was happening.
Regards,
Ralph
Re: [OE-core] [PATCH 2/2] cve-update-db-native: allow an option to force the CVE database update
Quoted 25 lines · Ralph Siemsen
Quoted 10 lines · Marta Rybczynska
Quoted 5 lines · Ralph Siemsen
# Interval between CVE database updates, in seconds. # Set to "0" to to force an update of the database.
CVE_DATABASE_UPDATE_INTERVAL?= "24*60*60"This is a good idea, thank you Ralph, I like it. I'll be sending a v2 shortly.
Thanks for this. I've tested it locally, on the fist run, the CVE database was fetched (it took quite a while, for some reason). On subsequent runs, no fetch occurs, so this seems to be working.
I then set
CVE_DB_UPDATE_INTERVAL= "3600" in my local.conf, and ran the build again. As it had been over an hour since the first build, the database was downloaded again. The timestamp on nvdcve_1.1.db did not change (as noted in one of your commit descriptions).So, it seems to be working correctly. I will re-test tomorrow (eg. after 24 hours) with the interval set back to default.
Quoted 7 lines
One minor point of confusion is that the log still shows "NOTE: recipe cve-update-db-native-1.0-r0: task
do_fetch: Started" even when the download is skipped. This is of course understandable when looking at the python code, the check is within thedo_fetchfunction. There is probably no simple way to avoid this from being displayed. And most users won't notice anyway. But it initially confused me about what was happening.
yeah you can avoid it unless another logic outside of this function is invoked. perhaps we can add a diagnostic inside the do_fetch to spill out the state information
and informing like "no fetch needed" or "database uptodate" or somesuch
Quoted 12 lines
Regards,
Ralph-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#165172): https://lists.openembedded.org/g/openembedded-core/message/165172
Mute This Topic: https://lists.openembedded.org/mt/90771095/1997914
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-