proposal: Append VENDOR_REVISION to PR for CVE scanners
cover proposal: Append VENDOR_REVISION to PR for CVE scanners
From: Robert Yang <[email protected]>
The VENDOR_REVISION is for cve scanners to know the CVEs have been fixed in a lower version, CVE scanners such as Trivy can know the CVEs have been fixed in a higher version, but it can't know the CVE is fixed in a lower version without a helper, we have the following ways to set the helper:
1) Use PR server
This doesn't work since the server updates PR for any changes.
2) Update PR manually when add a CVE patch
This is doesn't work either since:
- This is very trivial and people may forget to update the PR
- The PR may be updated for other reasons except CVE patches
So we need a specific part such as VENDOR_REVISION for cve scanners. The VENDOR_REVISION is designed as part of PR:
PR:append = ".vr51"
- ".vr51": The VENDOR_REVISION
- "vr": Vendor Revision, can be set to other values such as oe or poky
- "51": Convert from DISTRO_VERSION (Yocto 5.1), it can be customized with a function defined in GET_CURRENT_VENDOR_REVISION.
- The VENDOR_REVISION will only append to the recipes which have patches
There are two bbclasses to manage the VENDOR_REVISION automatically:
- gen-vendor-revision.bbclass: This is used for generating VENDOR_REVISION automatically, and save all the recipes' VENDOR_REVISION in
vendor-revision.conf, it is like:
VENDOR_REVISION[meta_recipes-support_libssh2_libssh2_1.11.0.bb] ??= '${VENDOR_REVISION_PREFIX}51 \
CVE-2023-48795:CVE-2023-48795.patch:b6c68cd1f0631180914ff112ac0c29c4 \
notcve:0001-disable-DSA-by-default.patch:61b6368d4a969d187805393d8b8fee85'
And in the second release such as Yocto 5.1.1, the bbclass will update the vr51 to vr511 when both of the following 2 conditions are met:
- The DISTRO VERSION is updated, for example, from 5.1 to 5.1.1
- The recipe's patches are changed (Patches added, removed or updated), otherwise, it will still be "51" when Yocto updated to 5.1.1, this can avoid unnecessary PR bump.
- enable-vendor-revision.bbclass: Append VENDOR_REVISION to PR After the VR is appended, the rpm package is like:
openssl-3.3.1-r0.vr51.core2_64.rpm
There is no change if the recipe doesn't have patches, for example:
base-files-3.0.14-r0.qemux86_64.rpm
Check the comments in the header of gen-vendor-revision.bbclass for more details.
// Robert
The following changes since commit 161c5b311f1aeb8f254dca96331b31d5b67fc92d:
build-appliance-image: Update to master head revision (2024-09-17 12:31:45 +0100)
are available in the Git repository at:
https://github.com/robertlinux/yocto rbt/vr
https://github.com/robertlinux/yocto/tree/rbt/vr
Robert Yang (2):
enable-vendor-revision.bbclass: Add it to append VENDOR_REVISION to PR gen-vendor-revision.bbclass: Add it to update VENDOR_REVISION
automatically
.../enable-vendor-revision.bbclass | 125 +++++++++ .../gen-vendor-revision.bbclass | 243 ++++++++++++++++++ 2 files changed, 368 insertions(+) create mode 100644 meta/classes-global/enable-vendor-revision.bbclass create mode 100644 meta/classes-global/gen-vendor-revision.bbclass
Signature · 1 line
2.25.1
PATCH 1/2 enable-vendor-revision.bbclass: Add it to append VENDOR_REVISION to PR
From: Robert Yang <[email protected]>
This bbclass is used for appending VENDOR_REVISION to PR from vendor-revision.conf for the recipes which have patches.
The vendor-revision.conf is not a must, but without it, all the recipes which patches will have a VR, there is no different in the first release such as Yocto 5.1, but all the VRs will be updated from "vr51" to "vr511" in Yocto 5.1.1 since there is no history data (vendor-revision.conf).
Check the comments in the header of gen-vendor-revision.bbclass for more details.
--- .../enable-vendor-revision.bbclass | 125 ++++++++++++++++++ 1 file changed, 125 insertions(+) create mode 100644 meta/classes-global/enable-vendor-revision.bbclass
meta/classes-global/enable-vendor-revision.bbclass new file +125 -0
@@ -0,0 +1,125 @@ +# +# Copyright (C) 2024 Wind River Systems, Inc +# +# SPDX-License-Identifier: GPL-2.0-only +# + +# This bbclass is used for appending VENDOR_REVISION to PR from +# vendor-revision.conf for the recipes which have patches. +# To enable it: +# 1. Generate vendor-revision.conf with gen-vendor-revision.bbclass +# 2. Remove 'gen-vendor-revision' from conf/local.conf if it is there +# 3. Add the following lines to conf/local.conf +# require /path/to/vendor-revision.conf +# INHERIT += "enable-vendor-revision" +# +# The vendor-revision.conf is not a must, but without it, all the recipes which +# patches will have a VR, there is no different in the first release such as +# Yocto 5.1, but all the VRs will be updated from "vr51" to "vr511" in Yocto +# 5.1.1 since there is no history data (vendor-revision.conf). +# +# The VENDOR_REVISION_PREFIX is used for: +# - Avoid confusions with PR Server +# - Customize for your own distro +VENDOR_REVISION_PREFIX ??= ".vr" + +def get_src_patches(d): + import oe.patch + local_patches = set() + bb.fetch.get_hashvalue(d) + for patch in oe.patch.src_patches(d): + _, _, local, _, _, parm = bb.fetch.decodeurl(patch) + local_patches.add(local) + return local_patches + +def is_work_shared(d): + sharedworkdir = os.path.join(d.getVar('TMPDIR'), 'work-shared') + return d.getVar('S').startswith(sharedworkdir) + +def vr_need_skip(d): + + if is_work_shared(d): + return False + + packages = d.getVar('PACKAGES') + if (not packages) or bb.data.inherits_class('nopackages', d) or \ + bb.data.inherits_class('nativesdk', d): + return True + + # Skip VR when no patches + if get_src_patches(d): + return False + else: + return True + +def get_var_short(var): + return '/'.join(var.split('/')[-4:]).replace('/', '_') + +def get_file_short(d): + return get_var_short(d.getVar('FILE')) + +# Allow users to customize the VR format since different Distro may have +# different DISTRO_VERSIONs +GET_CURRENT_VENDOR_REVISION ??= "get_current_vendor_revision(d)" + +def get_current_vendor_revision(d): + distro_version = d.getVar('DISTRO_VERSION') + vendor_revision = distro_version.replace('.', '') + + vr_prefix = d.getVar('VENDOR_REVISION_PREFIX') or '' + if vr_prefix: + vendor_revision = vr_prefix + vendor_revision + + vr_suffix = d.getVar('VENDOR_REVISION_SUFFIX') or '' + if vr_suffix: + vendor_revision += vr_suffix + + return vendor_revision + +python() { + """ + Set PR:append = "VENDOR_REVISION" for the recipes + """ + + # Skip when gen-vendor-revision is inherited + if bb.data.inherits_class('gen-vendor-revision', d): + bb.debug(1, 'Skipping enabling VR for %s since gen-vendor-revision is inherited' % d.getVar('PN')) + return + + if vr_need_skip(d): + return + + # Directly use VENDOR_REVISION if the recipe sets it. + vr = d.getVar('VENDOR_REVISION') + if not vr: + file_short = get_file_short(d) + val = d.getVarFlag('VENDOR_REVISION', file_short) + if val: + # The first item is VENDOR_REVISION + vr = val.split()[0] + # The d.getVarFlag("VENDOR_REVISION", "tmp_work-shared_qemux86-64_kernel-source") + # return None when kernel doesn't have patches, and vr_need_skip() + # can't skip work-shared recipes. The None is a string here. + if vr == 'None': + return + + if not vr: + # It's a new recipe, defult to current VR + vr = eval(d.getVar('GET_CURRENT_VENDOR_REVISION')) + + bb.debug(1, 'Appending VENDOR_REVISION %s to PR' % vr) + d.appendVar('PR', vr) +} + +GLOBAL_VENDOR_REVISION_WARN ??= '1' +addhandler warn_for_global_vr +warn_for_global_vr[eventmask] = "bb.event.ConfigParsed" +python warn_for_global_vr() { + # The VENDOR_REVISION should be set via recipe level, otherwise, there + # might be PR bombs or no effect, so warn for global VENDOR_REVISION + global_vr = d.getVar('VENDOR_REVISION') + if global_vr and bb.utils.to_boolean(d.getVar('GLOBAL_VENDOR_REVISION_WARN')): + bb.warn('The global VENDOR_REVISION (%s) may cause unneeded packages upgrading,' % global_vr) + bb.warn('or make VENDOR_REVISION invalid.') + bb.warn('The above warning can be disabled by GLOBAL_VENDOR_REVISION_WARN = "0"\n') +}
Signature · 1 line
2.25.1
PATCH 2/2 gen-vendor-revision.bbclass: Add it to update VENDOR_REVISION automatically
In reply to [email protected]From: Robert Yang <[email protected]>
The VENDOR_REVISION is for cve scanners to know the CVEs have been fixed in a lower version, CVE scanners such as Trivy can know the CVEs have been fixed in a higher version, but it can't know the CVE is fixed in a lower version without a helper, we have the following ways to set the helper:
1) Use PR server
This doesn't work since the server updates PR for any changes.
2) Update PR manually when add a CVE patch
This is doesn't work either since:
- This is very trivial and people may forget to update the PR
- The PR may be updated for other reasons except CVE patches
So we need a specific part such as VENDOR_REVISION for cve scanners. The VENDOR_REVISION is designed as part of PR:
PR:append = ".vr51"
- ".vr51": The VENDOR_REVISION
- "vr": Vendor Revision, can be set to other values such as oe or poky
- "51": Convert from DISTRO_VERSION (Yocto 5.1), it can be customized with a function defined in GET_CURRENT_VENDOR_REVISION.
- The VENDOR_REVISION will only append to the recipes which have patches
Check the comments in the header of gen-vendor-revision.bbclass for more details.
--- .../gen-vendor-revision.bbclass | 243 ++++++++++++++++++ 1 file changed, 243 insertions(+) create mode 100644 meta/classes-global/gen-vendor-revision.bbclass
meta/classes-global/gen-vendor-revision.bbclass new file +243 -0
@@ -0,0 +1,243 @@ +# +# Copyright (C) 2024 Wind River Systems, Inc +# +# SPDX-License-Identifier: GPL-2.0-only +# + +# The VENDOR_REVISION is for cve scanners to know the CVEs have been fixed in a +# lower version, CVE scanners such as Trivy can know the CVEs have been fixed +# in a higher version, but it can't know the CVE is fixed in a lower version without +# a helper, we have the following ways to set the helper: +# 1) Use PR server +# This doesn't work since the server updates PR for any changes. +# +# 2) Update PR manually when add a CVE patch +# This is doesn't work either since: +# - This is very trivial and people may forget to update the PR +# - The PR may be updated for other reasons except CVE patches +# +# So we need a specific part such as VENDOR_REVISION for cve scanners. +# The VENDOR_REVISION is designed as part of PR: +# PR:append = ".vr51" +# - ".vr51": The VENDOR_REVISION +# - "vr": Vendor Revision, can be set to other values such as oe or poky +# - "51": Convert from DISTRO_VERSION (Yocto 5.1), it can be customized with +# a function defined in GET_CURRENT_VENDOR_REVISION. +# - The VENDOR_REVISION will only append to the recipes which have patches +# +# The initial VENDOR_REVISION is '51', and will be updated when both of the +# following 2 conditions are met: +# - The DISTRO VERSION is updated, for example, from 5.1 to 5.1.1 +# - The recipe's patches are changed (Patches added, removed or updated), +# otherwise, it will still be "51" when Yocto updated to 5.1.1, this can avoid +# unnecessary PR bump. +# +# The VENDOR_REVISION can be set in each recipe, but that is very trivial, this +# bbclass is trying collect as much recipe's patches as possible and put all of +# them in one file, and simulate set VENDOR_REVISION in the recipes, it is like: +# +# VENDOR_REVISION[meta_recipes-support_libssh2_libssh2_1.11.0.bb] ??= '${VENDOR_REVISION_PREFIX}51 \ +# CVE-2023-48795:CVE-2023-48795.patch:b6c68cd1f0631180914ff112ac0c29c4 \ +# notcve:0001-disable-DSA-by-default.patch:61b6368d4a969d187805393d8b8fee85' +# +# - Use path_to_recipe.bb (Not PN or BPN) as the key is mainly because there +# might be multiple versions for a recipe, PN or BPN can't distinguish that, +# use recipe.bb as the key can simulate set VENDOR_REVISION in the recipes better. +# - Track Non-CVE patches (notcve) is because: +# The Non-CVE patch itself doesn't fix a CVE, but it may introduce CVEs, for example, +# a) There is a hello_1.0.bb which has no CVE issues in Yocto 5.1, +# b) A Non-CVE patch is applied to hello_1.0.bb and introduces a CVE issue +# in Yocto 5.1.1, then the cve scanners can't know whether hello-1.0.rpm in +# Yocto 5.1 is affected by the CVE or not, so we need track all the patches. +# - The VENDOR_REVISION can be set manually in the recipe or in +# vendor-revision-manual.inc if this bbclass can't handle it correctly. +# +# Examples for rpm packages with VR: +# - Without PR Server: openssl-3.3.1-r0.vr51.core2_64.rpm +# - With PR Server: openssl-3.3.1-r0.vr51.0.core2_64.rpm +# - No patches: base-files-3.0.14-r0.qemux86_64.rpm +# +# This bbclass is used for generating VENDOR_REVISION for each recipe, +# it can't be used for common building. +# +# Add the following line to conf/local.conf to enable it: +# INHERIT += "gen-vendor-revision" +# +# Run the following command to generate VR for all recipes +# $ bitbake -p +# +# The result will be in the file set by ${VENDOR_REVISION_ALL}, for example, +# tmp/vendor-revisions/qemux86-64/vendor-revision.conf +# +# Check enable-vendor-revision.bbclass on how to enable VENDOR_REVISION for the +# build. + +inherit enable-vendor-revision + +VENDOR_REVISION_DIR ?= "${TMPDIR}/vendor-revisions/${MACHINE}" +VENDOR_REVISION_ALL ?= "${VENDOR_REVISION_DIR}/vendor-revision.conf" + +# For extra customization on a released version, for example, users +# may have local patches on 5.0.1. +#VENDOR_REVISION_SUFFIX = ".custom1" + +# Do not skip libgfortran +FORTRAN:forcevariable = ",fortran" + +# Skip feature check +PARSE_ALL_RECIPES = "1" + +# Extra OVERRIDES +GEN_VENDOR_REVISION_OVERRIDES ??= "" + +addhandler gen_vr_fatal +gen_vr_fatal[eventmask] = "bb.event.BuildStarted" +python gen_vr_fatal() { + bb.fatal('Only bitbake -p is supported when gen-vendor-revision.bbclass is enabled!') +} + +addhandler gen_vr_prepare +gen_vr_prepare[eventmask] = "bb.event.CacheLoadStarted" +python gen_vr_prepare() { + vendor_revision_dir = d.getVar('VENDOR_REVISION_DIR') + # Need a fresh VENDOR_REVISION_DIR and CACHE dir + for k in ('VENDOR_REVISION_DIR', 'CACHE'): + value = d.getVar(k) + bb.note("Removing %s" % value) + bb.utils.remove(value, True) + bb.utils.mkdirhier(value) +} + +# Generate VENDOR_REVISION for each recipe +addhandler gen_vr_recipe_handler +gen_vr_recipe_handler[eventmask] = "bb.event.RecipeParsed" +python gen_vr_recipe_handler() { + """ + Generate VENDOR_REVISION for each recipe, the format is: + VENDOR_REVISION[recipe_file] ?= 'vr cveid1:patch1 notcve:patch2' + """ + + import hashlib + + if vr_need_skip(d): + return + + def get_cve_patch(patch): + ret = [] + patch_bn = os.path.basename(patch) + # Open in 'text' mode doesn't work for very a few patches such as: + # hddtemp_0.3-beta15-52.diff: 'utf-8' codec can't decode byte 0xe8 in + # position 3851: invalid continuation byte + with open(patch, 'rb') as f: + # Get md5 + md5 = hashlib.md5() + md5.update(f.read()) + hash = md5.hexdigest() + patch_bn += ':%s' % md5.hexdigest() + # Reset file postion + f.seek(0, 0) + for line in f: + if not 'CVE:' in str(line): + continue + line = line.decode('utf-8') + if line.startswith('CVE:') and 'CVE-' in line: + line_split = line.split('CVE:') + for cveid in line_split[1].split(): + cveid = cveid.strip() + if cveid.startswith('CVE-'): + ret.append('%s:%s' % (cveid, patch_bn)) + if not ret: + ret.append('notcve:%s' % patch_bn) + return ret + + def update_vr(patches): + """ + Check whether recipe's VENDOR_REVISION need update or not + * If old_vr == new_vf + - No check is needed since they are in the same + release, just update it. + + * If old_vr != new_vr: + - If the patches are the same: + Nothing changed, use the old_vr to avoid unneeded updates by dnf. + + - If the patches are different: + Update to the new_vr + """ + + patches.sort() + new_patches = ' '.join(patches) + + file_short = get_file_short(d) + vr = eval(d.getVar('GET_CURRENT_VENDOR_REVISION')) + old_val = d.getVarFlag('VENDOR_REVISION', file_short) + + # No checking is needed in the following cases: + # - No old_val: It's a new vr, just update it + if old_val: + old_vr = old_val.split()[0] + if old_vr: + # In different releases, check whether need update + if old_vr != vr: + old_patches = ' '.join(old_val.split()[1:]) + # The patches are the same, no update is needed + if old_patches == new_patches: + vr = old_vr + + # Replace .vr -> ${VENDOR_REVISION_PREFIX} + vr_prefix = d.getVar("VENDOR_REVISION_PREFIX") or "" + vr = vr.removeprefix(vr_prefix) + vr = '${VENDOR_REVISION_PREFIX}%s' % vr + out_lines = [] + val = "" + if new_patches: + val = '%s %s' % (vr, new_patches) + out_lines.append("VENDOR_REVISION[%s] ??= '%s'" % (file_short, val)) + + if is_work_shared(d): + s_short = get_var_short(d.getVar('S')) + if val: + out_lines.append("VENDOR_REVISION[%s] ?= '%s'" % (s_short, val)) + else: + out_lines.append("VENDOR_REVISION[%s] ??= '${@d.getVarFlag(\"VENDOR_REVISION\", \"%s\")}'" \ + % (file_short, s_short)) + if out_lines: + out_file = os.path.join(d.getVar('VENDOR_REVISION_DIR'), file_short) + with open(out_file, 'w') as f: + f.write('%s\n' % '\n'.join(out_lines)) + + patches = [] + localdata = bb.data.createCopy(d) + localdata.setVar('OVERRIDES', localdata.getVar('OVERRIDES') + localdata.getVar('GEN_VENDOR_REVISION_OVERRIDES')) + for local in get_src_patches(localdata): + patches += get_cve_patch(local) + update_vr(patches) +} + +# Generate vendor-revision.conf +addhandler gen_vr_all_handler +gen_vr_all_handler[eventmask] = "bb.event.ParseCompleted" +python gen_vr_all_handler () { + """ + Collect each recipe's vendor revision in VENDOR_REVISION_DIR and save + to VENDOR_REVISION_ALL + """ + import glob + vendor_revision_dir = d.getVar('VENDOR_REVISION_DIR') + patches = [] + output = d.getVar('VENDOR_REVISION_ALL') + output_dir = os.path.dirname(output) + for recipe in glob.glob(os.path.join(vendor_revision_dir, '*')): + with open(recipe) as f: + for line in f: + if not line in (patches): + patches.append(line) + patches.sort() + with open(output, 'w') as f: + f.write('include %s\n\n' % "vendor-revision-manual.inc") + f.write(''.join(patches)) + bb.note('The recipes with patches are saved to %s' % output) +} + +# Clear the base.bbclass magic srcrev call +fetcher_hashes_dummyfunc[vardepvalue] = ""
Signature · 1 line
2.25.1
I don't understand. If we fix a CVE with a backport, then there's metadata in the backported patch to indicate that even though the overall version doesn't change to the one that isn't vulnerable, the patch addresses the vulnerability. Why is a whole separate mechanism still needed?
Alex
Quoted 82 lines · Robert Yang via lists.openembedded.org
From: Robert Yang <[email protected]>
The
VENDOR_REVISIONis for cve scanners to know the CVEs have been fixed in a lower version, CVE scanners such as Trivy can know the CVEs have been fixed in a higher version, but it can't know the CVE is fixed in a lower version without a helper, we have the following ways to set the helper:
1) Use PR server
This doesn't work since the server updates PR for any changes.2) Update PR manually when add a CVE patch
This is doesn't work either since:
- This is very trivial and people may forget to update the PR
- The PR may be updated for other reasons except CVE patchesSo we need a specific part such as
VENDOR_REVISIONfor cve scanners. TheVENDOR_REVISIONis designed as part of PR:
PR:append = ".vr51"
- ".vr51": TheVENDOR_REVISION
- "vr": Vendor Revision, can be set to other values such as oe or poky
- "51": Convert fromDISTRO_VERSION(Yocto 5.1), it can be customized with a function defined inGET_CURRENT_VENDOR_REVISION.
- TheVENDOR_REVISIONwill only append to the recipes which have patchesThere are two bbclasses to manage the
VENDOR_REVISIONautomatically:
- gen-vendor-revision.bbclass: This is used for generatingVENDOR_REVISIONautomatically, and save all the recipes'VENDOR_REVISIONin vendor-revision.conf, it is like:VENDOR_REVISION[meta_recipes-support_libssh2_libssh2_1.11.0.bb] ??= '${VENDOR_REVISION_PREFIX}51 \ CVE-2023-48795:CVE-2023-48795.patch:b6c68cd1f0631180914ff112ac0c29c4 \ notcve:0001-disable-DSA-by-default.patch:61b6368d4a969d187805393d8b8fee85'And in the second release such as Yocto 5.1.1, the bbclass will update the vr51 to vr511 when both of the following 2 conditions are met:
- The DISTRO VERSION is updated, for example, from 5.1 to 5.1.1
- The recipe's patches are changed (Patches added, removed or updated), otherwise, it will still be "51" when Yocto updated to 5.1.1, this can avoid unnecessary PR bump.- enable-vendor-revision.bbclass: Append
VENDOR_REVISIONto PR After the VR is appended, the rpm package is like:
openssl-3.3.1-r0.vr51.core2_64.rpmThere is no change if the recipe doesn't have patches, for example:
base-files-3.0.14-r0.qemux86_64.rpmCheck the comments in the header of gen-vendor-revision.bbclass for more details.
// Robert
The following changes since commit
161c5b311f1aeb8f254dca96331b31d5b67fc92d:build-appliance-image: Update to master head revision (2024-09-17 12:31:45 +0100)
are available in the Git repository at:
https://github.com/robertlinux/yocto rbt/vr
https://github.com/robertlinux/yocto/tree/rbt/vrRobert Yang (2):
enable-vendor-revision.bbclass: Add it to appendVENDOR_REVISIONto PR gen-vendor-revision.bbclass: Add it to updateVENDOR_REVISION
automatically.../enable-vendor-revision.bbclass | 125 +++++++++ .../gen-vendor-revision.bbclass | 243 ++++++++++++++++++ 2 files changed, 368 insertions(+) create mode 100644 meta/classes-global/enable-vendor-revision.bbclass create mode 100644 meta/classes-global/gen-vendor-revision.bbclass--
2.25.1-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#204726): https://lists.openembedded.org/g/openembedded-core/message/204726
Mute This Topic: https://lists.openembedded.org/mt/108555445/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Hi Alexander,
Quoted 5 lines · Alexander Kanavin
I don't understand. If we fix a CVE with a backport, then there's metadata in the backported patch to indicate that even though the overall version doesn't change to the one that isn't vulnerable, the patch addresses the vulnerability. Why is a whole separate mechanism still needed?
Thanks for looking into this, the problem is that the metadata (CVE patch info) is not in the binary packages such as RPMs, so the cve scanners such as Trivy doesn't know that. For example, CentOS and Ubuntu also has the similar issues, they use the vendor revisions such as 29.el6.centos and 0.4ubuntu3.3 to help Trivy know that the CVE is fixed in a lower version package.
// Robert
Quoted 87 lines
Alex
Quoted 82 lines · Robert Yang via lists.openembedded.org
From: Robert Yang <[email protected]>
The
VENDOR_REVISIONis for cve scanners to know the CVEs have been fixed in a lower version, CVE scanners such as Trivy can know the CVEs have been fixed in a higher version, but it can't know the CVE is fixed in a lower version without a helper, we have the following ways to set the helper:
1) Use PR server
This doesn't work since the server updates PR for any changes.2) Update PR manually when add a CVE patch
This is doesn't work either since:
- This is very trivial and people may forget to update the PR
- The PR may be updated for other reasons except CVE patchesSo we need a specific part such as
VENDOR_REVISIONfor cve scanners. TheVENDOR_REVISIONis designed as part of PR:PR:append = ".vr51"- ".vr51": The
VENDOR_REVISION
- "vr": Vendor Revision, can be set to other values such as oe or poky
- "51": Convert fromDISTRO_VERSION(Yocto 5.1), it can be customized with a function defined inGET_CURRENT_VENDOR_REVISION.
- TheVENDOR_REVISIONwill only append to the recipes which have patchesThere are two bbclasses to manage the
VENDOR_REVISIONautomatically:
- gen-vendor-revision.bbclass: This is used for generatingVENDOR_REVISIONautomatically, and save all the recipes'VENDOR_REVISIONin
vendor-revision.conf, it is like:
VENDOR_REVISION[meta_recipes-support_libssh2_libssh2_1.11.0.bb] ??= '${VENDOR_REVISION_PREFIX}51 \CVE-2023-48795:CVE-2023-48795.patch:b6c68cd1f0631180914ff112ac0c29c4 \ notcve:0001-disable-DSA-by-default.patch:61b6368d4a969d187805393d8b8fee85'And in the second release such as Yocto 5.1.1, the bbclass will update the vr51 to vr511 when both of the following 2 conditions are met:
- The DISTRO VERSION is updated, for example, from 5.1 to 5.1.1
- The recipe's patches are changed (Patches added, removed or updated), otherwise, it will still be "51" when Yocto updated to 5.1.1, this can avoid unnecessary PR bump.- enable-vendor-revision.bbclass: Append
VENDOR_REVISIONto PR After the VR is appended, the rpm package is like:openssl-3.3.1-r0.vr51.core2_64.rpm There is no change if the recipe doesn't have patches, for example: base-files-3.0.14-r0.qemux86_64.rpmCheck the comments in the header of gen-vendor-revision.bbclass for more details.
// Robert
The following changes since commit
161c5b311f1aeb8f254dca96331b31d5b67fc92d:build-appliance-image: Update to master head revision (2024-09-17 12:31:45 +0100)are available in the Git repository at:
https://github.com/robertlinux/yocto rbt/vr
https://github.com/robertlinux/yocto/tree/rbt/vrRobert Yang (2):
enable-vendor-revision.bbclass: Add it to appendVENDOR_REVISIONto PR gen-vendor-revision.bbclass: Add it to updateVENDOR_REVISION
automatically.../enable-vendor-revision.bbclass | 125 +++++++++ .../gen-vendor-revision.bbclass | 243 ++++++++++++++++++ 2 files changed, 368 insertions(+)create mode 100644 meta/classes-global/enable-vendor-revision.bbclass create mode 100644 meta/classes-global/gen-vendor-revision.bbclass
--
2.25.1-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#204726): https://lists.openembedded.org/g/openembedded-core/message/204726
Mute This Topic: https://lists.openembedded.org/mt/108555445/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 5 lines · Robert Yang
Thanks for looking into this, the problem is that the metadata (CVE patch info) is not in the binary packages such as RPMs, so the cve scanners such as Trivy doesn't know that. For example, CentOS and Ubuntu also has the similar issues, they use the vendor revisions such as 29.el6.centos and 0.4ubuntu3.3 to help Trivy know that the CVE is fixed in a lower version package.
But how is trivy's database updated to include such custom vendor revisions? Is it done centrally by the vendor (and how?), or is it done by users in local installations? This does appear hack-ish to me, the better thing to do would be to actually include the list of fixed CVEs into package metadata.
I'm not sure this should be carried in core, if there's only a single known tool that needs it, and core has no support or tests for it. Maybe something like meta-trivy?
Alex
Hi,
Quoted 38 lines · Robert Yang via lists.openembedded.org
From: Robert Yang <[email protected]>
The
VENDOR_REVISIONis for cve scanners to know the CVEs have been fixed in a lower version, CVE scanners such as Trivy can know the CVEs have been fixed in a higher version, but it can't know the CVE is fixed in a lower version without a helper, we have the following ways to set the helper:
1) Use PR server
This doesn't work since the server updates PR for any changes.2) Update PR manually when add a CVE patch
This is doesn't work either since:
- This is very trivial and people may forget to update the PR
- The PR may be updated for other reasons except CVE patchesSo we need a specific part such as
VENDOR_REVISIONfor cve scanners. TheVENDOR_REVISIONis designed as part of PR:
PR:append = ".vr51"
- ".vr51": TheVENDOR_REVISION
- "vr": Vendor Revision, can be set to other values such as oe or poky
- "51": Convert fromDISTRO_VERSION(Yocto 5.1), it can be customized with a function defined inGET_CURRENT_VENDOR_REVISION.
- TheVENDOR_REVISIONwill only append to the recipes which have patchesThere are two bbclasses to manage the
VENDOR_REVISIONautomatically:
- gen-vendor-revision.bbclass: This is used for generatingVENDOR_REVISIONautomatically, and save all the recipes'VENDOR_REVISIONin vendor-revision.conf, it is like:VENDOR_REVISION[meta_recipes-support_libssh2_libssh2_1.11.0.bb] ??= '${VENDOR_REVISION_PREFIX}51 \ CVE-2023-48795:CVE-2023-48795.patch:b6c68cd1f0631180914ff112ac0c29c4 \ notcve:0001-disable-DSA-by-default.patch:61b6368d4a969d187805393d8b8fee85'And in the second release such as Yocto 5.1.1, the bbclass will update the vr51 to vr511 when both of the following 2 conditions are met:
- The DISTRO VERSION is updated, for example, from 5.1 to 5.1.1
- The recipe's patches are changed (Patches added, removed or updated), otherwise, it will still be "51" when Yocto updated to 5.1.1, this can avoid unnecessary PR bump.
Quoted 8 lines
- enable-vendor-revision.bbclass: Append
VENDOR_REVISIONto PR After the VR is appended, the rpm package is like:
openssl-3.3.1-r0.vr51.core2_64.rpmThere is no change if the recipe doesn't have patches, for example:
base-files-3.0.14-r0.qemux86_64.rpmCheck the comments in the header of gen-vendor-revision.bbclass for more details.
This is very much backwards, like Alex mentioned as well.
There is no need for this. If CVEs are fixed with patches, then those patches will mark the specific version and patch applied as not affected by the CVE. The classes export this data. If anyone feeds this data to external tooling, then the CVE patch status is a critical detail which must be exported and imported into the tools as well. Otherwiser the external tooling is not really up for the job.
I've seen several commercial tools not managing the patch status at all. IMO these tools are broken and can't be used to manage secure patches of real products which have to apply CVE patches and can't always update SW versions.
When managing a yocto based Linux distro, IMO, the tooling is already there to handle CVEs etc. External tools frequently cause more pain than actually improve things.
Cheers,
-Mikko
Quoted 9 lines · Alexander Kanavin
Quoted 5 lines · Robert Yang
Thanks for looking into this, the problem is that the metadata (CVE patch info) is not in the binary packages such as RPMs, so the cve scanners such as Trivy doesn't know that. For example, CentOS and Ubuntu also has the similar issues, they use the vendor revisions such as 29.el6.centos and 0.4ubuntu3.3 to help Trivy know that the CVE is fixed in a lower version package.
But how is trivy's database updated to include such custom vendor revisions? Is it done centrally by the vendor (and how?), or is it
The vendor itself will update trivy's database, David Reyna (in the To list) is working on a script which can update the database, we will provide more detailed examples later.
Quoted 3 lines
done by users in local installations? This does appear hack-ish to me, the better thing to do would be to actually include the list of fixed CVEs into package metadata.
The problem is that Trivy can't work in this way AFAIK.
Quoted 4 lines
I'm not sure this should be carried in core, if there's only a single known tool that needs it, and core has no support or tests for it. Maybe something like meta-trivy?
Trivy is just an example, we're trying to make more tools such as blackduck work.
// Robert
Quoted 2 lines
Alex
Quoted 58 lines · Mikko Rapeli
Hi,
Quoted 38 lines · Robert Yang via lists.openembedded.org
From: Robert Yang <[email protected]>
The
VENDOR_REVISIONis for cve scanners to know the CVEs have been fixed in a lower version, CVE scanners such as Trivy can know the CVEs have been fixed in a higher version, but it can't know the CVE is fixed in a lower version without a helper, we have the following ways to set the helper:
1) Use PR server
This doesn't work since the server updates PR for any changes.2) Update PR manually when add a CVE patch
This is doesn't work either since:
- This is very trivial and people may forget to update the PR
- The PR may be updated for other reasons except CVE patchesSo we need a specific part such as
VENDOR_REVISIONfor cve scanners. TheVENDOR_REVISIONis designed as part of PR:PR:append = ".vr51"- ".vr51": The
VENDOR_REVISION
- "vr": Vendor Revision, can be set to other values such as oe or poky
- "51": Convert fromDISTRO_VERSION(Yocto 5.1), it can be customized with a function defined inGET_CURRENT_VENDOR_REVISION.
- TheVENDOR_REVISIONwill only append to the recipes which have patchesThere are two bbclasses to manage the
VENDOR_REVISIONautomatically:
- gen-vendor-revision.bbclass: This is used for generatingVENDOR_REVISIONautomatically, and save all the recipes'VENDOR_REVISIONin
vendor-revision.conf, it is like:
VENDOR_REVISION[meta_recipes-support_libssh2_libssh2_1.11.0.bb] ??= '${VENDOR_REVISION_PREFIX}51 \CVE-2023-48795:CVE-2023-48795.patch:b6c68cd1f0631180914ff112ac0c29c4 \ notcve:0001-disable-DSA-by-default.patch:61b6368d4a969d187805393d8b8fee85'And in the second release such as Yocto 5.1.1, the bbclass will update the vr51 to vr511 when both of the following 2 conditions are met:
- The DISTRO VERSION is updated, for example, from 5.1 to 5.1.1
- The recipe's patches are changed (Patches added, removed or updated), otherwise, it will still be "51" when Yocto updated to 5.1.1, this can avoid unnecessary PR bump.Quoted 8 lines
- enable-vendor-revision.bbclass: Append
VENDOR_REVISIONto PR After the VR is appended, the rpm package is like:openssl-3.3.1-r0.vr51.core2_64.rpm There is no change if the recipe doesn't have patches, for example: base-files-3.0.14-r0.qemux86_64.rpmCheck the comments in the header of gen-vendor-revision.bbclass for more details.
This is very much backwards, like Alex mentioned as well.
There is no need for this. If CVEs are fixed with patches, then those patches will mark the specific version and patch applied as not affected by the CVE. The classes export this data. If anyone feeds this data to external tooling, then the CVE patch status is a critical detail which must be exported and imported into the tools as well. Otherwiser the external tooling is not really up for the job.
Quoted 3 lines
I've seen several commercial tools not managing the patch status at all. IMO these tools are broken and can't be used to manage secure patches of real products which have to apply CVE patches and can't always update SW versions.
Maybe you're right, but the mainstream distributions work with those tools such as Trivy:
https://aquasecurity.github.io/trivy/v0.17.2/vuln-detection/os/
// Robert
Quoted 8 lines
When managing a yocto based Linux distro, IMO, the tooling is already there to handle CVEs etc. External tools frequently cause more pain than actually improve things.
Cheers,
-Mikko
Quoted 5 lines · Robert Yang
Quoted 3 lines
done by users in local installations? This does appear hack-ish to me, the better thing to do would be to actually include the list of fixed CVEs into package metadata.
The problem is that Trivy can't work in this way AFAIK.
You do need to raise this with trivy. It's open source, and can be improved.
This whole CVE ecosystem is already mad enough and that's why I'm not getting involved, managing vendor revisions instead of directly looking at what has been fixed is just adding to that madness.
Or let's just not use broken external tools, and do things right ourselves.
Alex
Hi,
Quoted 59 lines · Robert Yang
Quoted 58 lines · Mikko Rapeli
Hi,
Quoted 38 lines · Robert Yang via lists.openembedded.org
From: Robert Yang <[email protected]>
The
VENDOR_REVISIONis for cve scanners to know the CVEs have been fixed in a lower version, CVE scanners such as Trivy can know the CVEs have been fixed in a higher version, but it can't know the CVE is fixed in a lower version without a helper, we have the following ways to set the helper:
1) Use PR server
This doesn't work since the server updates PR for any changes.2) Update PR manually when add a CVE patch
This is doesn't work either since:
- This is very trivial and people may forget to update the PR
- The PR may be updated for other reasons except CVE patchesSo we need a specific part such as
VENDOR_REVISIONfor cve scanners. TheVENDOR_REVISIONis designed as part of PR:PR:append = ".vr51"- ".vr51": The
VENDOR_REVISION
- "vr": Vendor Revision, can be set to other values such as oe or poky
- "51": Convert fromDISTRO_VERSION(Yocto 5.1), it can be customized with a function defined inGET_CURRENT_VENDOR_REVISION.
- TheVENDOR_REVISIONwill only append to the recipes which have patchesThere are two bbclasses to manage the
VENDOR_REVISIONautomatically:
- gen-vendor-revision.bbclass: This is used for generatingVENDOR_REVISIONautomatically, and save all the recipes'VENDOR_REVISIONin
vendor-revision.conf, it is like:
VENDOR_REVISION[meta_recipes-support_libssh2_libssh2_1.11.0.bb] ??= '${VENDOR_REVISION_PREFIX}51 \CVE-2023-48795:CVE-2023-48795.patch:b6c68cd1f0631180914ff112ac0c29c4 \ notcve:0001-disable-DSA-by-default.patch:61b6368d4a969d187805393d8b8fee85'And in the second release such as Yocto 5.1.1, the bbclass will update the vr51 to vr511 when both of the following 2 conditions are met:
- The DISTRO VERSION is updated, for example, from 5.1 to 5.1.1
- The recipe's patches are changed (Patches added, removed or updated), otherwise, it will still be "51" when Yocto updated to 5.1.1, this can avoid unnecessary PR bump.Quoted 8 lines
- enable-vendor-revision.bbclass: Append
VENDOR_REVISIONto PR After the VR is appended, the rpm package is like:openssl-3.3.1-r0.vr51.core2_64.rpm There is no change if the recipe doesn't have patches, for example: base-files-3.0.14-r0.qemux86_64.rpmCheck the comments in the header of gen-vendor-revision.bbclass for more details.
This is very much backwards, like Alex mentioned as well.
There is no need for this. If CVEs are fixed with patches, then those patches will mark the specific version and patch applied as not affected by the CVE. The classes export this data. If anyone feeds this data to external tooling, then the CVE patch status is a critical detail which must be exported and imported into the tools as well. Otherwiser the external tooling is not really up for the job.
Quoted 8 lines
Quoted 3 lines
I've seen several commercial tools not managing the patch status at all. IMO these tools are broken and can't be used to manage secure patches of real products which have to apply CVE patches and can't always update SW versions.
Maybe you're right, but the mainstream distributions work with those tools such as Trivy:
https://aquasecurity.github.io/trivy/v0.17.2/vuln-detection/os/
I think maintaining CVE and security patches of a distro using tools which are not used by the uptream maintainers of the distro is not a good idea.
I understand that organizations and managers may want to use specific, sometimes commercial tools.
But these may not help. If you only look at the RPM package data exported by yocto, then CVE patch status is lost. Same is true for other package output from yocto builds.
In the past I was asked to provide and use Black Duck tooling and have explained these problems multiple times, and eventually the requests went away. Time was better spent fixing actual issues found by upstream compatible tooling, e.g. cve-check.bbclass.
If there is a standard for RPM package metadata to include CVE patch status, then that could of course be added to the package built by yocto.
In similar way, if I were asked to maintain a Debian/Ubuntu variant for CVE security patches, then I'd start with
https://security-tracker.debian.org/tracker/ and the data embedded there.
Cheers,
-Mikko
Quoted 4 lines · Robert Yang via lists.openembedded.org
The
VENDOR_REVISIONis for cve scanners to know the CVEs have been fixed in a lower version, CVE scanners such as Trivy can know the CVEs have been fixed in a higher version, but it can't know the CVE is fixed in a lower version without a helper, we have the following ways to set the helper:
I understand what you're trying to do here, but the class doesn’t just work on its own out of the box, as there needs to be further configuration and workflow to make it actually useful, and the only users it is useful for are people who are maintaining a binary distribution and have sufficient clout to update the CVE tooling.
I suggest that you keep this class in your Wind River Linux layer.
Cheers,
Ross