Skip to content

core: Use of percent_defconfig seems to impact performance

Nevo Hed via buildroot buildroot 1 version Applied

Status from this message: Applied to master, thanks.

Get series with b4
b4 am [email protected]

Versions newest first

  1. v2 Latest1 patch 4 repliesApplied

Changes in v2 from the cover letter

- Testing for defconfig existence and error when failing to find it is

Patches in v2 Applied

#Subject and filesSizeRepliesApplied
1/1core: Use of percent_defconfig seems to impact performanceMakefile+8 -74 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v2 yet.

Read the v2 thread

Cover letter [Buildroot] [PATCH v2 0/1] core: Use of percent_defconfig seems to impact performance

Noticing serious performance issues on make X_defconfig with growth of

number of external trees. At 7 trees it is somewhat tolerable but at 8 trees we have seen it take 80-110 seconds.

Numbers for before and after the change in commit itself.

I do not know for sure what the underlying issue is but looking at some strace and make -d output it feels like implosion of implicit rules being evaluated.

Note that after I submitted v1 of this patch it was pointed out to me that a different patch is already in the works. But since that neither that patch nor mine had any seen any movement in months I decided to take a stab at addressing the feedback I received from Yann on my patch:

Quoted 2 lines

... it defers the test that the defconfig exists into the shell rather than the Makefile

Adding the resets as suggested here
https://lists.gnu.org/archive/html/help-make/2002-11/msg00062.html seem to help a lot but not as much as the attached patch.

--8<------------
%: %,v
%: RCS/%,v
%: RCS/%
%: s.%
%: SCCS/s.%
--8<------------

Inlined below a script to repro this issue, not sure if this functionality should be integrated anywhere or just left in the mailing list. By default the script runs the scenario from 1 tree to 8 trees. Running with param (1-10) will allow a single run with that many trees.

--8<------------
#!/usr/bin/env bash

declare treedirs=()

declare -r cfg="pc_x86_64_bios_defconfig"

function setup_dir {
    for tree in tree{1..10}; do
        local treedir="defconf_test/${tree}"

        mkdir -p "${treedir}/configs"
        touch "${treedir}/Config.in"
        touch "${treedir}/external.mk"
        echo -e "name: ${tree^^}\ndesc: ${tree^}" > "${treedir}/external.desc"

        cp "./configs/${cfg}" "${treedir}/configs/${tree}_${cfg}"

        treedirs+=( "${treedir}" )
    done
}
function run_one {
    local count="${1}"
    local br2_external=$(IFS=':'; echo "${treedirs[*]:0:count}")
    echo "Defconfig with ${count} trees (BR2_EXTERNAL=\"${br2_external}\")"
    /usr/bin/time -f "${count},%e" /bin/make BR2_EXTERNAL="${br2_external}" "${cfg}"
}

# Main
setup_dir
if [ -n "${1}" ]; then

    run_one "${1}"
else
    for i in {1..8}; do
        run_one $i
    done

fi
--8<------------

Changes v1 -> v2:
- Testing for defconfig existence and error when failing to find it is

    now done at the Makefile level rather than the shell (utilizing `$(or
    ...` and `$(error ...`).

Nevo Hed (1):
core: Use of percent_defconfig seems to impact performance

 Makefile | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)
Signature · 6 lines
2.37.3

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot