Store package files list in results (branch yem/files-list)
cover Store package files list in results (branch yem/files-list)
Hello All!
This small series against the autobuilder code, stores the packages files lists in autobuild results, so that it is easier to do a quick analysis about the origin of files in a build.
Regards,
Yann E. MORIN.
The following changes since commit e90dc8753f9deaca9ecb41a5874d7a559ee2557c
utils/daily-mail: sort overall summary in global e-mail by branch name (2019-03-15 15:19:36 +0100)
are available in the git repository at:
/home/ymorin/cache/upstream/buildroot/buildroot-test
for you to fetch changes up to 1414cf2855df133bd02c1a32d8ebe3e632f37734
[AUTOBUILD] autobuild-run: also copy packages files lists if they exist (2019-03-31 19:30:26 +0200)
----------------------------------------------------------------
Yann E. MORIN (2):
[AUTOBUILD] autobuild-run: add helper function to copy-if-exists [AUTOBUILD] autobuild-run: also copy packages files lists if they exist
scripts/autobuild-run | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-)
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/2 autobuild-run: add helper function to copy-if-exists
--- scripts/autobuild-run | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-)
scripts/autobuild-run +8 -6
@@ -478,12 +478,14 @@ def send_results(result, **kwargs): os.path.join(resultdir, "defconfig")) shutil.copyfile(os.path.join(outputdir, "branch"), os.path.join(resultdir, "branch")) - if os.path.exists(os.path.join(outputdir, "build", "build-time.log")): - shutil.copyfile(os.path.join(outputdir, "build", "build-time.log"), - os.path.join(resultdir, "build-time.log")) - if os.path.exists(os.path.join(outputdir, "legal-info", "manifest.csv")): - shutil.copyfile(os.path.join(outputdir, "legal-info", "manifest.csv"), - os.path.join(resultdir, "licenses-manifest.csv")) + + def copy_if_exists(dir, src, dst=None): + if os.path.exists(os.path.join(outputdir, d, src)): + shutil.copyfile(os.path.join(outputdir, d, src), + os.path.join(resultdir, src if dst is None else dst)) + + copy_if_exists("build", "build-time.log") + copy_if_exists("legal-info", "manifest.csv", "licenses-manifest.csv") subprocess.call(["git log -n 1 --pretty=format:%%H > %s" % \ os.path.join(resultdir, "gitid")],
Signature · 1 line
2.14.1
PATCH 2/2 autobuild-run: also copy packages files lists if they exist
In reply to Yann E. MORIN--- scripts/autobuild-run | 3 +++ 1 file changed, 3 insertions(+)
scripts/autobuild-run +3 -0
@@ -485,6 +485,9 @@ def send_results(result, **kwargs): os.path.join(resultdir, src if dst is None else dst)) copy_if_exists("build", "build-time.log") + copy_if_exists("build", "packages-file-list.txt") + copy_if_exists("build", "packages-file-list-host.txt") + copy_if_exists("build", "packages-file-list-staging.txt") copy_if_exists("legal-info", "manifest.csv", "licenses-manifest.csv") subprocess.call(["git log -n 1 --pretty=format:%%H > %s" % \
Signature · 1 line
2.14.1
cover Store package files list in results (branch yem/files-list)
In reply to Yann E. MORINHello Yann,
Quoted 3 lines · On Sun, 31 Mar 2019 19:48:20 +0200 "Yann E. MORIN" <[email protected]>…
Yann E. MORIN (2):
[AUTOBUILD] autobuild-run: add helper function to copy-if-exists [AUTOBUILD] autobuild-run: also copy packages files lists if they exist
Both applied. I hope those additional files will not be too big: we keep all those files for all build results, so it can add up quite a bit over time. The build results currently take 64 GB of disk space, and require 1303600 files.
Thanks,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
cover Store package files list in results (branch yem/files-list)
Thomas, All,
On 2019-04-03 23:13 +0200, Thomas Petazzoni spake thusly:
Quoted 11 lines
Quoted 3 lines · On Sun, 31 Mar 2019 19:48:20 +0200 "Yann E. MORIN" <[email protected]>…
Yann E. MORIN (2):
[AUTOBUILD] autobuild-run: add helper function to copy-if-exists [AUTOBUILD] autobuild-run: also copy packages files lists if they existBoth applied. I hope those additional files will not be too big: we keep all those files for all build results, so it can add up quite a bit over time. The build results currently take 64 GB of disk space, and require 1303600 files.
I would expect those files to be of about the same size as the build.log snippet we keep.
If it turns out to be a problem, then we can restrict that to only failed builds.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
cover Store package files list in results (branch yem/files-list)
Quoted 5 lines · On Wed, 3 Apr 2019 23:30:12 +0200 "Yann E. MORIN" <[email protected]>…
I would expect those files to be of about the same size as the build.log snippet we keep.
If it turns out to be a problem, then we can restrict that to only failed builds.
It is true that for failed builds, not much of the information we keep is interesting. Perhaps just the .config/defconfig are interesting. The build.log and other files are mainly useful to investigate issues.
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com