testsdk.bbclass: set PREMIRRORS for kernel to speed up test
cover testsdk.bbclass: set PREMIRRORS for kernel to speed up test
The following changes since commit b6bc5b2840246ac0f0e5d8cebcdffc1a0ca2518f:
bitbake: toaster: add 'Sumo' to release selection (2018-04-26 11:43:26 +0100)
are available in the git repository at:
git://git.pokylinux.org/poky-contrib ChenQi/testsdkext-speed
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=ChenQi/testsdkext-spped
Chen Qi (1):
testsdk.bbclass: set PREMIRRORS for kernel to speed up test
meta/classes/testsdk.bbclass | 1 + 1 file changed, 1 insertion(+)
Signature · 1 line
1.9.1
PATCH 1/1 testsdk.bbclass: set PREMIRRORS for kernel to speed up test
Currently if we do a testsdkext task for the image, it would take very long time to finish. The time is mostly spent fetching kernel source via network.
We have done some configuration in auto.conf, trying to make use of own-mirrors.bbclass to avoid fetching kernel via network.
However, the solution normally does not work. Below is some log from log.do_fetch.
DEBUG: Fetcher failure: Unable to find file \ file:///path/to/downloads/git2_git.yoctoproject.org.linux-yocto.git.tar.gz \ anywhere.
The tar.gz file is not available. It is generated only if BB_GENERATE_MIRROR_TARBALLS is set to "1". The default value of BB_GENERATE_MIRROR_TARBALLS is "0", and according to the manual, users need choose to set it to "1" only if they are trying to make a source mirror. So generally, this var's value is "0".
Anyway, we do need to avoid fetching kernel source from network when doing testsdkext. So set PREMIRRORS in auto.conf to achieve this.
After this change, the time reduces from 4209.131s to 1399.436s on my local machine.
[YOCTO #12729]
--- meta/classes/testsdk.bbclass | 1 + 1 file changed, 1 insertion(+)
meta/classes/testsdk.bbclass +1 -0
@@ -157,6 +157,7 @@ def testsdkext_main(d): f.write('SSTATE_MIRRORS += " \\n file://.* file://%s/PATH"\n' % test_data.get('SSTATE_DIR')) f.write('SOURCE_MIRROR_URL = "file://%s"\n' % test_data.get('DL_DIR')) f.write('INHERIT += "own-mirrors"\n') + f.write('PREMIRRORS_prepend = " git://git.yoctoproject.org/.* git://%s/git2/git.yoctoproject.org.BASENAME;protocal=file \\n "\n' % test_data.get('DL_DIR')) # We need to do this in case we have a minimal SDK subprocess.check_output(". %s > /dev/null; devtool sdk-install meta-extsdk-toolchain" % \
Signature · 1 line
1.9.1
Quoted 4 lines · Chen Qi
+ f.write('PREMIRRORS_prepend = "
git://git.yoctoproject.org/.*git://%s/git2/git.yoctoproject.org.BASENAME;protocal=file\\n "\n' %test_data.get('DL_DIR'))
protocal?
We could turn on BB_GENERATE_SOURCE_MIRRORS by default. What's the overhead?
Ross
Quoted 8 lines · Ross
On 3 May 2018 at 09:52, Chen Qi <[email protected] <mailto:[email protected]>> wrote:
+ f.write('PREMIRRORS_prepend = " git://git.yoctoproject.org/.* <http://git.yoctoproject.org/.*> git://%s/git2/git.yoctoproject.org.BASENAME;protocal=file \\n "\n' % test_data.get('DL_DIR'))
Quoted 5 lines
protocal?
We could turn on
BB_GENERATE_SOURCE_MIRRORSby default. What's the overhead?Ross
Hi Ross,
I didn't test the overhead.
The reference manual says that this variable is only useful when users want to generate a source mirror.
Some related info from bitbake/Changelog
"""
- Add BB_GENERATE_MIRROR_TARBALLS option, set to 0 to make git
fetches
faster at the expense of not creating mirror tarballs.
"""
Besides, testsdkext is a task to verify correctness. It does not affect the generated eSDK that the users actually use. I'm not sure if I should change some default value merely to speed it up. I'd rather do some tweaks to this task instead of doing some change which could affect the building results.
That's why I chose not to change its default value.
I've removed the 'protocol=file' and sent out V2.
Best Regards,
Chen Qi