New: NodeJS slow startup performance (~30s)
https://bugs.busybox.net/show_bug.cgi?id=12426
Bug ID: 12426
Summary: NodeJS slow startup performance (~30s)
Product: buildroot
Version: 2019.11
Hardware: All
OS: Linux
Status: NEW
Severity: normal
Priority: P5
Component: Other
Assignee: unassigned at buildroot.uclibc.org
Reporter: erichiggins at gmail.com
CC: buildroot at uclibc.org
Target Milestone: ---
The performance of the NodeJS executable, as compiled by Buildroot, has exceptionally poor startup performance, taking around 30s to run.
The prebuilt binaries from NodeJS running on the same system take less than 2s to run.
I created a Gist with a comparison using the time command:
https://gist.github.com/erichiggins/ac139ebe34b8d3c6dbb68f3b09ca9008
In both cases, the runtime performance is about equal.
The prebuilt binaries require the buildroot system to use glibc and will not run if a more space-efficient alternative (uClibc-ng) is selected.
Further, NodeJS v12.x (now the major version used by Buildroot) does not officially support ARM v6l and is considered experimental.
https://github.com/nodejs/build/issues/1677#issuecomment-486204349
Current workaround is to use glibc and overwrite the NodeJS executables with the unofficial prebuilts via rootfs overlay.
https://github.com/nodejs/unofficial-builds/
It would be quite helpful to build a performant NodeJS executable without requiring these workarounds.
Thank you!
Signature · 2 lines
You are receiving this mail because: You are on the CC list for the bug.
[Buildroot] [Bug 12426] NodeJS slow startup performance (~30s)
https://bugs.busybox.net/show_bug.cgi?id=12426
--- Comment #1 from Thomas Petazzoni <[email protected]> --- Are you building Buildroot with the default of BR2_OPTIMIZE_S=y (which implies using the -Os gcc optimization flag), or using BR2_OPTIMIZE_2=y or BR2_OPTIMIZE_3=y ?
If you're using BR2_OPTIMIZE_S=y, could you try BR2_OPTIMIZE_2=y and BR2_OPTIMIZE_3=y and see how much it changes things ?
Signature · 2 lines
You are receiving this mail because: You are on the CC list for the bug.
[Buildroot] [Bug 12426] NodeJS slow startup performance (~30s)
In reply to bugzilla at busybox.nethttps://bugs.busybox.net/show_bug.cgi?id=12426
--- Comment #2 from Eric Higgins <[email protected]> ---
Yes, I'm using BR2_OPTIMIZE_S=y but I can try BR2_OPTIMIZE_2=y and report back.
I emailed the listed developers for package/nodejs and Daniel emailed back with a few suggestions specific to that package's build flags, though they were not successful for me and it started to get a bit more complex than I could commit to (hence this ticket).
Signature · 2 lines
You are receiving this mail because: You are on the CC list for the bug.
[Buildroot] [Bug 12426] NodeJS slow startup performance (~30s)
In reply to bugzilla at busybox.nethttps://bugs.busybox.net/show_bug.cgi?id=12426
--- Comment #3 from D. Price <[email protected]> ---
Hi Thomas! Long time since I've helped out with Buildroot but happy to try to be of assistance.
Eric indicated to me that this largely a startup issue, and not one of node's actual performance.
One of node's performance tricks is to take a snapshot of its V8 interpreter "world" and store that as a snapshot. This is accomplished using the mksnapshot command during the build. This has been tricky for them to get right when cross-compiling.
Eric tried removing the --without-snapshot flag from the build but then the build failed trying to run mksnapshot (I think because it was running the target command, not the host version).
Eric Said:
Quoted 12 lines
Maybe there's another problem in my config, but having removed that flag from nodejs.mk, the build failed. I reverted my changes to .config, ran make clean and removed the contents from dl/ and ~/.buildroot-ccache before building.
touch 5e8f1fa6be42bf232d2ec48bae2a475b98d6f2f6.intermediate
LD_LIBRARY_PATH=/home/vagrant/buildroot/output/build/nodejs-12.13.0/out/Release/lib.host:/home/vagrant/buildroot/output/build/nodejs-12.13.0/out/Release/lib.target:$LD_LIBRARY_PATH; exportLD_LIBRARY_PATH; cd ../tools/v8_gypfiles; mkdir -p/home/vagrant/buildroot/output/build/nodejs-12.13.0/out/Release/obj.target/v8_snapshot/geni; "/home/vagrant/buildroot/output/build/nodejs-12.13.0/out/Release/mksnapshot" --turbo_instruction_scheduling "--target_os=linux" "--target_arch=arm" --startup_src "/home/vagrant/buildroot/output/build/nodejs-12.13.0/out/Release/obj.target/v8_snapshot/geni/snapshot.cc" --embedded_variant Default --embedded_src "/home/vagrant/buildroot/output/build/nodejs-12.13.0/out/Release/obj.target/v8_snapshot/geni/embedded.S" --no-native-code-counters/lib/ld-uClibc.so.0: No such file or directorytools/v8_gypfiles/v8_snapshot.target.mk:16: recipe for target
'5e8f1fa6be42bf232d2ec48bae2a475b98d6f2f6.intermediate' failed
make[3]: *** [5e8f1fa6be42bf232d2ec48bae2a475b98d6f2f6.intermediate] Error 255
...
Makefile:101: recipe for target 'node' failed
make[2]: *** [node] Error 2
Quoted 16 lines · I said:
I haven't worked with this, but to me this looks like the mksnapshot command (which has been cross-compiled for you) is now failing to run (
/home/vagrant/buildroot/output/build/nodejs-12.13.0/out/Release/mksnapshot).I found some github bug reports related to this but nothing which really is a definitive guide. I also found this, which might help:
https://chrislea.com/2018/08/20/cross-compiling-node-js-for-arm-on-ubuntu/ . Several of the sources I found talk about settingCC_hostandCXX_host, which to the best of my recollection the buildroot recipe does not do. IIRC buildroot uses HOSTCC and HOSTCXX instead
(see https://buildroot.org/downloads/manual/manual.html, 8.6). So maybe that is important? I don't know. Somewhere I also found a thing which said that you have to be careful exactly how you setCC_hostandCXX_host, to make sure -m32 was passed. I dunno.
Buried in https://github.com/nodejs/build/issues/266 is the statement
Quoted 6 lines
"I believe I have this working now and will be able to re-enable snapshots when nodejs/node#4117 lands. V8 can handle cross compiling of snapshots if the {CC,CXX}_host variables are defined, by compiling the mksnapshot executable with the host compiler."
Which seems useful.
So that is my recommendation:
- Double check that it's strictly a startup issue
- Try to enable the snapshot feature
Signature · 2 lines
You are receiving this mail because: You are on the CC list for the bug.
[Buildroot] [Bug 12426] NodeJS slow startup performance (~30s)
In reply to bugzilla at busybox.nethttps://bugs.busybox.net/show_bug.cgi?id=12426
--- Comment #4 from Eric Higgins <[email protected]> --- Using BR2_OPTIMIZE_2=y didn't make any discernible difference.
time node --help
...
real 0m 27.46s user 0m 22.92s sys 0m 1.90s
Signature · 2 lines
You are receiving this mail because: You are on the CC list for the bug.
[Buildroot] [Bug 12426] NodeJS slow startup performance (~30s)
In reply to bugzilla at busybox.nethttps://bugs.busybox.net/show_bug.cgi?id=12426
--- Comment #5 from Peter Korsgaard <[email protected]> ---
Is this only at first startup or does running nodejs multiple times each time take ~30s on the RPI? What storage are you using? The prebuilt nodejs presumably bundles a bunch of dependencies statically, whereas the Buildroot one uses dynamic linking
Signature · 2 lines
You are receiving this mail because: You are on the CC list for the bug.
[Buildroot] [Bug 12426] NodeJS slow startup performance (~30s)
In reply to bugzilla at busybox.nethttps://bugs.busybox.net/show_bug.cgi?id=12426
--- Comment #6 from Eric Higgins <[email protected]> --- All subsequent runs of nodejs are just as slow to start.
Storage is a class 10 microSD.
Actually, the prebuilt nodejs binaries are also dynamically linked. I found this out the hard way because I was using uClibc-ng and the prebuilt expects glibc.
(prebuilt)
# file /usr/local/lib/nodejs/node-v10.18.0-linux-armv6l/bin/node
/usr/local/lib/nodejs/node-v10.18.0-linux-armv6l/bin/node: ELF 32-bit LSB executable, ARM, EABI5 version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.16.42, with debug_info, not stripped
(compiled by buildroot)
# file /usr/bin/node
/usr/bin/node: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-uClibc.so.0, stripped
For what it's worth, the flags they use for armv6l are available here:
https://github.com/nodejs/unofficial-builds/tree/master/recipes/armv6l
Signature · 2 lines
You are receiving this mail because: You are on the CC list for the bug.
[Buildroot] [Bug 12426] NodeJS slow startup performance (~30s)
In reply to bugzilla at busybox.nethttps://bugs.busybox.net/show_bug.cgi?id=12426
--- Comment #7 from Peter Korsgaard <[email protected]> --- (In reply to Eric Higgins from comment #6)
Yes, both builds dynamically link to libc, but the unofficial builds probably statically link to openssl / libuv / zlib / ..
Thanks for the link to their Dockerfile, but I don't seem to find where they run ./configure?
Signature · 2 lines
You are receiving this mail because: You are on the CC list for the bug.
[Buildroot] [Bug 12426] NodeJS slow startup performance (~30s)
In reply to bugzilla at busybox.nethttps://bugs.busybox.net/show_bug.cgi?id=12426
--- Comment #8 from Eric Higgins <[email protected]> ---
I don't see them running ./configure either. My guess is that they are passing in flags and options as env variables instead?
Signature · 2 lines
You are receiving this mail because: You are on the CC list for the bug.
[Buildroot] [Bug 12426] NodeJS slow startup performance (~30s)
In reply to bugzilla at busybox.nethttps://bugs.busybox.net/show_bug.cgi?id=12426
--- Comment #9 from Eric Higgins <[email protected]> --- Looks like ./configure is called a few times within the Makefile
https://github.com/nodejs/node/blob/v12.14.0/Makefile
Signature · 2 lines
You are receiving this mail because: You are on the CC list for the bug.
[Buildroot] [Bug 12426] NodeJS slow startup performance (~30s)
In reply to bugzilla at busybox.nethttps://bugs.busybox.net/show_bug.cgi?id=12426
Yann E. MORIN <[email protected]> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |[email protected] Resolution|--- |MOVED Status|NEW |RESOLVED
--- Comment #10 from Yann E. MORIN <[email protected]> --- Thank you for your report.
The issue tracker for the Buildroot project has been moved to the Gitlab.com issue tracker:
https://gitlab.com/buildroot.org/buildroot/-/issues
We are taking this opportunity to close old issues in this old tracker. If you believe your issue is still relevant, please open one in the new issue tracker.
Thank you!
Signature · 6 lines
You are receiving this mail because: You are on the CC list for the bug. _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot