core: Use of percent_defconfig seems to impact performance
Status from this message: Applied to master, thanks.
Get series with b4
b4 am [email protected]Versions newest first
Changes in v2 from the cover letter
- Testing for defconfig existence and error when failing to find it is
Patches in v2 Applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/1 | core: Use of percent_defconfig seems to impact performanceMakefile | +8 | 4 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v2 yet.
Cover letter [Buildroot] [PATCH v2 0/1] core: Use of percent_defconfig seems to impact performance
Noticing serious performance issues on make X_defconfig with growth of
number of external trees. At 7 trees it is somewhat tolerable but at 8 trees we have seen it take 80-110 seconds.
Numbers for before and after the change in commit itself.
I do not know for sure what the underlying issue is but looking at some strace and make -d output it feels like implosion of implicit rules being evaluated.
Note that after I submitted v1 of this patch it was pointed out to me that a different patch is already in the works. But since that neither that patch nor mine had any seen any movement in months I decided to take a stab at addressing the feedback I received from Yann on my patch:
Quoted 2 lines
... it defers the test that the defconfig exists into the shell rather than the
Makefile
Adding the resets as suggested here
https://lists.gnu.org/archive/html/help-make/2002-11/msg00062.html seem to help a lot but not as much as the attached patch.
--8<------------
%: %,v
%: RCS/%,v
%: RCS/%
%: s.%
%: SCCS/s.%
--8<------------
Inlined below a script to repro this issue, not sure if this functionality should be integrated anywhere or just left in the mailing list. By default the script runs the scenario from 1 tree to 8 trees. Running with param (1-10) will allow a single run with that many trees.
--8<------------
#!/usr/bin/env bash
declare treedirs=()
declare -r cfg="pc_x86_64_bios_defconfig"
function setup_dir {
for tree in tree{1..10}; do
local treedir="defconf_test/${tree}"
mkdir -p "${treedir}/configs"
touch "${treedir}/Config.in"
touch "${treedir}/external.mk"
echo -e "name: ${tree^^}\ndesc: ${tree^}" > "${treedir}/external.desc"
cp "./configs/${cfg}" "${treedir}/configs/${tree}_${cfg}"
treedirs+=( "${treedir}" )
done
}function run_one {
local count="${1}"
local br2_external=$(IFS=':'; echo "${treedirs[*]:0:count}")
echo "Defconfig with ${count} trees (BR2_EXTERNAL=\"${br2_external}\")"
/usr/bin/time -f "${count},%e" /bin/make BR2_EXTERNAL="${br2_external}" "${cfg}"
}# Mainsetup_dir
if [ -n "${1}" ]; then
run_one "${1}"
else
for i in {1..8}; do
run_one $i
donefi
--8<------------
Changes v1 -> v2:
- Testing for defconfig existence and error when failing to find it is
now done at the Makefile level rather than the shell (utilizing `$(or
...` and `$(error ...`).Nevo Hed (1):
core: Use of percent_defconfig seems to impact performance
Makefile | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-)
Signature · 6 lines
2.37.3 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot