support/scripts/pkg-stats: add infra_coinfig to json_output
PATCH 1/1 support/scripts/pkg-stats: add infra_coinfig to json_output
When we use the statistics output to generate a CVE/CPE customer report showing whether a product is affected by CVEs, we are primarily interested in whether they are relevant to the target system. Currently we cannot see if the package is configured for the build (infra==host) and/or the target system (infra==target).
Therefore, in case of a given configuration, this information is also stored in the JSON output.
--- support/scripts/pkg-stats | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-)
support/scripts/pkg-stats +15 -6
@@ -89,6 +89,7 @@ class Package: self.path = path self.pkg_path = os.path.dirname(path) self.infras = None + self.infra_config = None self.license = None self.has_license = False self.has_license_files = False
@@ -146,9 +147,9 @@ class Package: return False return True - def set_infra(self): + def set_infra(self, config_info): """ - Fills in the .infras field + Fills in the .infras and ..infra_config fields """ self.infras = list() with open(os.path.join(brpath, self.path), 'r') as f:
@@ -163,6 +164,13 @@ class Package: else: self.infras.append(("target", infra)) + if config_info: + for pkg_name in config_info: + if config_info[pkg_name]['name'] == self.name: + if self.infra_config == None: + self.infra_config = list() + self.infra_config.append(config_info[pkg_name]['type']) + def set_license(self): """ Fills in the .status['license'] and .status['license-files'] fields
@@ -369,8 +377,8 @@ def get_pkglist(npackages, package_list): def get_config_packages(): cmd = ["make", "--no-print-directory", "show-info"] - js = json.loads(subprocess.check_output(cmd)) - return set([v["name"] for v in js.values()]) + config_info = json.loads(subprocess.check_output(cmd)) + return (config_info, set([v["name"] for v in config_info.values()])) def package_init_make_info():
@@ -1115,10 +1123,11 @@ def __main__(): if args.nvd_path: import cve as cvecheck + config_info = None if args.packages: package_list = args.packages.split(",") elif args.configpackages: - package_list = get_config_packages() + (config_info, package_list) = get_config_packages() else: package_list = None date = datetime.datetime.utcnow()
@@ -1137,7 +1146,7 @@ def __main__(): package_init_make_info() print("Getting package details ...") for pkg in packages: - pkg.set_infra() + pkg.set_infra(config_info) pkg.set_license() pkg.set_hash_info() pkg.set_patch_count()
Signature · 1 line
2.30.0
PATCH 1/1 support/scripts/pkg-stats: add infra_coinfig to json_output
Quoted 50 lines · On Thu, 24 Jun 2021 14:57:03 +0200 Heiko Thiery <[email protected]> wrote:
support/scripts/pkg-stats +12
-4@@ -89,6 +89,7 @@ class Package: self.path = path self.pkg_path = os.path.dirname(path) self.infras = None + self.infra_config = None self.license = None self.has_license = False self.has_license_files = False@@ -146,9 +147,9 @@ class Package: return False return True - def set_infra(self): + def set_infra(self, config_info): """ - Fills in the .infras field + Fills in the .infras and ..infra_config fields """ self.infras = list() with open(os.path.join(brpath, self.path), 'r') as f:@@ -163,6 +164,13 @@ class Package: else: self.infras.append(("target", infra)) + if config_info: + for pkg_name in config_info: + if config_info[pkg_name]['name'] == self.name: + if self.infra_config == None: + self.infra_config = list() + self.infra_config.append(config_info[pkg_name]['type']) + def set_license(self): """ Fills in the .status['license'] and .status['license-files'] fields@@ -369,8 +377,8 @@ def get_pkglist(npackages, package_list): def get_config_packages(): cmd = ["make", "--no-print-directory", "show-info"] - js = json.loads(subprocess.check_output(cmd)) - return set([v["name"] for v in js.values()]) + config_info = json.loads(subprocess.check_output(cmd)) + return (config_info, set([v["name"] for v in config_info.values()])) def package_init_make_info():
support/scripts/pkg-stats def __main__(): · line 1130
Quoted 9 lines
support/scripts/pkg-stats +2
-1@@ -1115,10 +1123,11 @@ def __main__(): if args.nvd_path: import cve as cvecheck + config_info = None if args.packages: package_list = args.packages.split(",") elif args.configpackages: - package_list = get_config_packages() + (config_info, package_list) = get_config_packages()
Why is this information passed separately, and not within the package_list ? I'm a bit confused here. Why don't have you add a "type" field in the Package class that says whether it's a host, target, or both package ?
Thomas
Signature · 3 lines
Thomas Petazzoni, co-owner and CEO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 1/1 support/scripts/pkg-stats: add infra_coinfig to json_output
Hi Thomas,
Am Do., 24. Juni 2021 um 15:04 Uhr schrieb Thomas Petazzoni <[email protected]>:
Quoted 57 lines
Quoted 53 lines · On Thu, 24 Jun 2021 14:57:03 +0200 Heiko Thiery <[email protected]> wrote:
support/scripts/pkg-stats +12
-4@@ -89,6 +89,7 @@ class Package: self.path = path self.pkg_path = os.path.dirname(path) self.infras = None + self.infra_config = None self.license = None self.has_license = False self.has_license_files = False@@ -146,9 +147,9 @@ class Package: return False return True - def set_infra(self): + def set_infra(self, config_info): """ - Fills in the .infras field + Fills in the .infras and ..infra_config fields """ self.infras = list() with open(os.path.join(brpath, self.path), 'r') as f:@@ -163,6 +164,13 @@ class Package: else: self.infras.append(("target", infra)) + if config_info: + for pkg_name in config_info: + if config_info[pkg_name]['name'] == self.name: + if self.infra_config == None: + self.infra_config = list() + self.infra_config.append(config_info[pkg_name]['type']) + def set_license(self): """ Fills in the .status['license'] and .status['license-files'] fields@@ -369,8 +377,8 @@ def get_pkglist(npackages, package_list): def get_config_packages(): cmd = ["make", "--no-print-directory", "show-info"] - js = json.loads(subprocess.check_output(cmd)) - return set([v["name"] for v in js.values()]) + config_info = json.loads(subprocess.check_output(cmd)) + return (config_info, set([v["name"] for v in config_info.values()])) def package_init_make_info():@@ -1115,10 +1123,11 @@ def __main__(): if args.nvd_path: import cve as cvecheck
Quoted 11 lines
support/scripts/pkg-stats def __main__(): · line 1130
Quoted 6 lines
support/scripts/pkg-stats +2
-1+ config_info = None if args.packages: package_list = args.packages.split(",") elif args.configpackages: - package_list = get_config_packages() + (config_info, package_list) = get_config_packages()Why is this information passed separately, and not within the
package_list? I'm a bit confused here. Why don't have you add a "type" field in the Package class that says whether it's a host, target, or both package ?
Because the package_list is only a list of package names. It is not a class object. And I did not want to change the package_list variable. This is also filled by "packages" command line arguments. Or am I missing something here?
Signature · 1 line
Heiko
Hello Heiko,
Quoted 6 lines · On Thu, 24 Jun 2021 14:57:03 +0200 Heiko Thiery <[email protected]> wrote:
When we use the statistics output to generate a CVE/CPE customer report showing whether a product is affected by CVEs, we are primarily interested in whether they are relevant to the target system. Currently we cannot see if the package is configured for the build (infra==host) and/or the target system (infra==target).
Quoted 7 lines
Therefore, in case of a given configuration, this information is also stored in the JSON output.
Signed-off-by Heiko Thiery <[email protected]>--- support/scripts/pkg-stats | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-)
We finally had a look with Arnout. We didn't really like this new self.infra_config field. So instead, what we decided is to use the existing self.infras field.
When pkg-stats is run without a configuration (i.e without -c), self.infras lists all infrastructures supported by the package, as it is done today.
When pkg-stats is run with a configuration (i.e with -c), self.infras now only lists the infrastructures that are *used* by the package in the current configuration.
I reworked your code with this idea, and also changed the implementation quite a bit. You can see the result at:
https://git.buildroot.org/buildroot/commit/?id=28973f28acfcee81f2912e7a65dd3704b1d68f6f
When you have the chance, could you give it a test and let us know if it works for you?
Thanks a lot!
Thomas
Signature · 7 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot