Top-level parallel build support
Get series with b4
b4 am 20190301171803.GD2721@scaerVersions newest first
- v7 Latest8 patches 0 repliesNew
Patches in v7 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/8 | support/scripts/check-host-rpath: document existing functions | +0 | 0 replies | |
| 2/8 | Makefile: move definition of TARGET_DIR inside .config condition | +0 | 0 replies | |
| 3/8 | core: implement per-package SDK and target | +0 | 1 reply | |
| 3/8 | core: implement per-package SDK and target | +0 | 0 replies | |
| 4/8 | Makefile: allow top-level parallel build with BR2_PER_PACKAGE_DIRECTORIES=y | +0 | 0 replies | |
| 5/8 | package/pkg-generic: make libtool .la files compatible with per-package directories | +0 | 0 replies | |
| 7/8 | docs/manual: add details about top-level parallel build support | +0 | 1 reply | |
| 7/8 | docs/manual: add details about top-level parallel build support | +0 | 0 replies |
Reviews trailers per patch
3 of 8 patches have a Reviewed-by or Acked-by.
| Patch | Yann E. MORIN |
|---|---|
| 1/8 | R-b |
| 2/8 | R-b |
| 3/8 | |
| 3/8 | |
| 4/8 | A-b |
| 5/8 | |
| 7/8 | |
| 7/8 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [Buildroot] [PATCH v7 0/8] Top-level parallel build support
Vadym, All,
On 2019-03-01 16:50 +0200, Vadym Kochan spake thusly:
Quoted 28 lines · Thomas Petazzoni
Quoted 16 lines
Quoted 7 lines · On Mon, 25 Feb 2019 03:10:03 +0200 Vadim Kochan <[email protected]> wrote:
I think that in case of per-packages parallel build the logs became messy and meaningless so I send a patch which allows to do per-package logging too:
https://patchwork.ozlabs.org/patch/1047549/this is a POC so it just shows the idea.
The problem of logging was discussed quite a lot back when I started to work on per-package directories. I think we investigated overriding the SHELL variable, but couldn't find an approach that worked properly, but maybe the approach was different than yours, I need to get back to the original discussion.
Quoted 8 lines
Back then, what we concluded is that people could use the "make --output-sync=target" option, or "make -Otarget" in short. This option buffers the output of make on a per-target basis, and displays this output at once when the target has completed. The advantage is that the output is no longer garbled, but the drawback is that you don't see "live" what is happening: if a package takes 20 minutes to build, during 20 minutes you see nothing, and at the end of the 20 minutes, you get in one go the full build output of that package.
I updated the patch:
Quoted 7 lines
https://patchwork.ozlabs.org/patch/1047620/where I added per-package-and-per-step logging, I think it is possible to merge them info one log file, but per-step approach allows to easy update logs for perticular step.
But I 'd like to clarify if actually such approach make sense ?
As Thomas said, we already have had some discussion about the logging, and we also investigated using a shell wrapper, and a PoC similar to yours was even attempted.
However, we eventually concluded that this was not going to work reliably, so we decided against at that time. I could not find that conclusion in the devdays meeting reports, so I guess it's burried somewhere in the mailing list. I'll try to unearth those later in the evening or the week-end...
The first issue I can readily remember, was that the SHELL variable is passed down to all children, and some of them may re-use it to run commands, and the wrapper would make those fail. I don't have the details, though, as I said I'll try to dig them from the archives...
But overall, I agree with Thomas, that people that still want nice logs call 'make -Otarget'.
Quoted 6 lines
Also how do you think about to add (using the approach from the patch) prepending logs with a package name prefix (in bold) (not necessary for parellel building but enabled by config option):
[${pkg_name}] $log_line
Quoted 8 lines
after it may be possible to print the percentage of installed packages like:
[${percent}][${pkg_name}] $log_lineofcourse all this is just for providing to user at runtime the info which package is building and how many packages (in percentages) are done. So, may be it looks useless, but I feel it would be nice to have.
Sorry, but I think this is really getting too far.
We already have a simple make wrapper, utils/brmake, which filters the build log and redirects all to a file, and just prints the '>>>' lines to stdout. I think this is as far as we should go with Buildroot.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'