Skip to contentSkip to conversation

package: bring two host tools

PATCH In review Alexis Lothoré via buildroot 1 participant 12 messages Aug 6 – Aug 12

lore
Share & tools
lore
#0 lore raw

cover package: bring two host tools

Hello,
this small series aims to bring two new host packages to ease the integration of web apps onto embedded platforms:
- uglifyjs ([1]), used to minify frontend assets,
- sass ([2]), a css extension adding programmation features to css.

Those two tools come in the shape of binaries installed in the host directory. The use case supported by those new packages is the possibility to run mini_asset ([3]) on host during some custom package build, with uglifyjs and sass being two dependencies of it. The series does not bring a mini_asset package though, as it can be brought by the custom composer file in the final application project.

This use case kind of similar to what have been proposed in the past with BR2_PACKAGE_HOST_NODEJS_MODULES_ADDITIONAL ([4]). At this time, the proposal was rejected, and Arnout rather suggested to create dedicated packages for this kind of host tools. This series follows this suggestion, hoping that it is still the most relevant solution, 4 years later. Another important point to mention is that the second package is fetching some deps (npm install) at build step, which is not ideal, but also suggested in Arnout's initial proposal. If that's not desired anymore, I'm open to some guidance to improve this.

[1] https://www.npmjs.com/package/uglify-js
[2] https://www.npmjs.com/package/sass
[3] https://github.com/markstory/mini-asset
[4] https://lore.kernel.org/buildroot/[email protected]/

---
Alexis Lothoré (2):
package/uglifyjs: new package
package/sass: new package

 DEVELOPERS                                         |  2 ++
 package/Config.in.host                             |  2 ++
 package/sass/Config.in.host                        |  8 ++++++
 package/sass/sass.hash                             |  3 +++
 package/sass/sass.mk                               | 30 ++++++++++++++++++++++
 package/uglifyjs/Config.in.host                    |  7 +++++
 package/uglifyjs/uglifyjs.hash                     |  3 +++
 package/uglifyjs/uglifyjs.mk                       | 23 +++++++++++++++++
 support/testing/tests/package/test_sass.py         | 23 +++++++++++++++++
 .../tests/package/test_sass/sample_sass.scss       |  7 +++++
 support/testing/tests/package/test_uglifyjs.py     | 23 +++++++++++++++++
 .../tests/package/test_uglifyjs/sample_uglifyjs.js |  5 ++++
 12 files changed, 136 insertions(+)

---
base-commit: 3c5c058d156daaba2901e539395a842d2bafd1e6
change-id: 20260806-host_tools-89a53a630ff7

Best regards,

Signature · 6 lines
Alexis Lothoré <[email protected]>

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#1 lore raw

PATCH 1/2 package/uglifyjs: new package

uglifyjs is a JavaScript parser, minifier, compressor and beautifier toolkit.

https://github.com/mishoo/UglifyJS

---
 DEVELOPERS                                         |  1 +
 package/Config.in.host                             |  1 +
 package/uglifyjs/Config.in.host                    |  7 +++++++
 package/uglifyjs/uglifyjs.hash                     |  3 +++
 package/uglifyjs/uglifyjs.mk                       | 23 ++++++++++++++++++++++
 support/testing/tests/package/test_uglifyjs.py     | 23 ++++++++++++++++++++++
 .../tests/package/test_uglifyjs/sample_uglifyjs.js |  5 +++++
 7 files changed, 63 insertions(+)
DEVELOPERS +1 -0
index 56dd9f3943d1..3027d1310528 100644
@@ -158,6 +158,7 @@ N: 	Alexis Lothoré <[email protected]>
 F:	package/libxmlsec1/
 F:	package/openscap/
 F: 	package/python-scp/
+F:	package/uglifyjs/
 
 N:	Alistair Francis <[email protected]>
 F:	board/sifive/
package/Config.in.host +1 -0
index 595caf9e7fea..0e4e29cd3514 100644
@@ -127,6 +127,7 @@ menu "Host utilities"
 	source "package/tipidee/Config.in.host"
 	source "package/uboot-tools/Config.in.host"
 	source "package/ugetty/Config.in.host"
+	source "package/uglifyjs/Config.in.host"
 	source "package/util-linux/Config.in.host"
 	source "package/utp_com/Config.in.host"
 	source "package/uuu/Config.in.host"
package/uglifyjs/Config.in.host new file +7 -0
new file mode 100644
index 000000000000..51a31634c4ec
@@ -0,0 +1,7 @@
+config BR2_PACKAGE_HOST_UGLIFYJS
+	bool "host uglifyjs"
+	help
+	  uglifyjs is a javascript parser, minifier compressor and
+	  beautifier toolkit
+
+	  https://github.com/mishoo/UglifyJS/
package/uglifyjs/uglifyjs.hash new file +3 -0
new file mode 100644
index 000000000000..b403fdd960e1
@@ -0,0 +1,3 @@
+# Locally calculated
+sha256  6988b64d4169c150ce46b8ae6e2c369aa82b1d67adc755236abf972c38857737  uglifyjs-3.19.3.tar.gz
+sha256  422ee8d0b531a3e3c3c730b4dbd6f590ff786853897b9992888e34fcdacbe526  LICENSE
package/uglifyjs/uglifyjs.mk new file +23 -0
new file mode 100644
index 000000000000..921d4dffee8e
@@ -0,0 +1,23 @@
+################################################################################
+#
+# uglifyjs
+#
+################################################################################
+
+UGLIFYJS_VERSION = 3.19.3
+UGLIFYJS_SITE = $(call github,mishoo,UglifyJS,v$(UGLIFYJS_VERSION))
+UGLIFYJS_LICENSE = BSD-2-Clause
+UGLIFYJS_LICENSE_FILES = LICENSE
+HOST_UGLIFYJS_DEPENDENCIES = host-nodejs
+
+define HOST_UGLIFYJS_INSTALL_CMDS
+	$(INSTALL) -m 0755 -d $(HOST_DIR)/lib/node_modules/uglify-js
+	rsync -a $(@D)/bin $(HOST_DIR)/lib/node_modules/uglify-js/
+	rsync -a $(@D)/lib $(HOST_DIR)/lib/node_modules/uglify-js/
+	rsync -a $(@D)/tools $(HOST_DIR)/lib/node_modules/uglify-js/
+	$(INSTALL) -m 0755 $(@D)/package.json $(HOST_DIR)/lib/node_modules/uglify-js/
+	ln -sf ../lib/node_modules/uglify-js/bin/uglifyjs $(HOST_DIR)/bin/uglifyjs
+
+endef
+
+$(eval $(host-generic-package))
support/testing/tests/package/test_uglifyjs.py new file +23 -0
new file mode 100644
index 000000000000..666a5fb46ff1
@@ -0,0 +1,23 @@
+import os
+import infra
+from pathlib import Path
+
+from infra.basetest import BRTest
+
+INPUT_SAMPLE = Path(__file__).parent / "test_uglifyjs/sample_uglifyjs.js"
+
+
+class TestHostUglifyJS(BRTest):
+    config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
+        """
+        BR2_PACKAGE_HOST_UGLIFYJS=y
+        """
+
+    def test_run(self):
+        uglifyjs_bin = os.path.join(self.builddir, "host", "bin", "uglifyjs")
+
+        cmd = [uglifyjs_bin, "--help"]
+        infra.run_cmd_on_host(self.builddir, cmd)
+
+        cmd = [uglifyjs_bin, INPUT_SAMPLE, "-o", "output.js", "-c", "-m"]
+        infra.run_cmd_on_host(self.builddir, cmd)
support/testing/tests/package/test_uglifyjs/sample_uglifyjs.js new file +5 -0
new file mode 100644
index 000000000000..f393c3d35d99
@@ -0,0 +1,5 @@
+function addNumbers(firstNumber, secondNumber) {
+    var totalResult = firstNumber + secondNumber;
+    console.log("The total is: " + totalResult);
+    return totalResult;
+}
Signature · 6 lines
2.55.0

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#2 lore raw

Re: [Buildroot] [PATCH 1/2] package/uglifyjs: new package

Hello Alexis,

Quoted 6 lines · Alexis Lothoré

uglifyjs is a JavaScript parser, minifier, compressor and beautifier toolkit.

https://github.com/mishoo/UglifyJS

Thanks for the patch. Overall looks good. One or two suggestions below.

Quoted 12 lines

new file mode 100644
index 000000000000..921d4dffee8e

package/uglifyjs/uglifyjs.mk new file +7 -0
@@ -0,0 +1,23 @@
+################################################################################
+#
+# uglifyjs
+#
+################################################################################
+
+UGLIFYJS_VERSION = 3.19.3

package/uglifyjs/uglifyjs.mk line 17

Quoted 10 lines
package/uglifyjs/uglifyjs.mk new file +10 -0
+UGLIFYJS_SITE = $(call github,mishoo,UglifyJS,v$(UGLIFYJS_VERSION))
+UGLIFYJS_LICENSE = BSD-2-Clause
+UGLIFYJS_LICENSE_FILES = LICENSE
+HOST_UGLIFYJS_DEPENDENCIES = host-nodejs
+
+define HOST_UGLIFYJS_INSTALL_CMDS
+	$(INSTALL) -m 0755 -d $(HOST_DIR)/lib/node_modules/uglify-js
+	rsync -a $(@D)/bin $(HOST_DIR)/lib/node_modules/uglify-js/
+	rsync -a $(@D)/lib $(HOST_DIR)/lib/node_modules/uglify-js/
+	rsync -a $(@D)/tools $(HOST_DIR)/lib/node_modules/uglify-js/

We more commonly use "cp -dpfr" for this kind of situation, and you could also use a loop?

package/uglifyjs/uglifyjs.mk line 18

Quoted 1 line
package/uglifyjs/uglifyjs.mk new file +1 -0
+	$(INSTALL) -m 0755 $(@D)/package.json $(HOST_DIR)/lib/node_modules/uglify-js/

Something like:

	$(foreach p,bin lib tools package.json,\
		cp -dpfr $(@D)/$(p) $(HOST_DIR)/lib/node_modules/uglify-js/
	)

package/uglifyjs/uglifyjs.mk line 20

Quoted 2 lines
package/uglifyjs/uglifyjs.mk new file +2 -0
+	ln -sf ../lib/node_modules/uglify-js/bin/uglifyjs $(HOST_DIR)/bin/uglifyjs
+

Drop this empty new line.

support/testing/tests/package/test_uglifyjs.py

Quoted 8 lines
package/uglifyjs/uglifyjs.mk new file +3 -0
+endef
+
+$(eval $(host-generic-package))
support/testing/tests/package/test_uglifyjs.py new file +0 -0
new file mode 100644
index 000000000000..666a5fb46ff1

Forgot about commenting on this on the DEVELOPERS file: please add this file to the DEVELOPERS file as well, under your name. This way if this test fails in our CI, you'll get notified.

Thanks!

Thomas

Signature · 7 lines
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#3 lore raw

Re: [Buildroot] [PATCH 1/2] package/uglifyjs: new package

Hi Thomas,

Quoted 38 lines · Thomas Petazzoni

Hello Alexis,

Quoted 6 lines · Alexis Lothoré

uglifyjs is a JavaScript parser, minifier, compressor and beautifier toolkit.

https://github.com/mishoo/UglifyJS

Thanks for the patch. Overall looks good. One or two suggestions below.

Quoted 12 lines

new file mode 100644
index 000000000000..921d4dffee8e

package/uglifyjs/uglifyjs.mk new file +7 -0
@@ -0,0 +1,23 @@
+################################################################################
+#
+# uglifyjs
+#
+################################################################################
+
+UGLIFYJS_VERSION = 3.19.3

package/uglifyjs/uglifyjs.mk line 17

Quoted 10 lines
package/uglifyjs/uglifyjs.mk new file +10 -0
+UGLIFYJS_SITE = $(call github,mishoo,UglifyJS,v$(UGLIFYJS_VERSION))
+UGLIFYJS_LICENSE = BSD-2-Clause
+UGLIFYJS_LICENSE_FILES = LICENSE
+HOST_UGLIFYJS_DEPENDENCIES = host-nodejs
+
+define HOST_UGLIFYJS_INSTALL_CMDS
+	$(INSTALL) -m 0755 -d $(HOST_DIR)/lib/node_modules/uglify-js
+	rsync -a $(@D)/bin $(HOST_DIR)/lib/node_modules/uglify-js/
+	rsync -a $(@D)/lib $(HOST_DIR)/lib/node_modules/uglify-js/
+	rsync -a $(@D)/tools $(HOST_DIR)/lib/node_modules/uglify-js/

We more commonly use "cp -dpfr" for this kind of situation, and you could also use a loop?

Quoted 8 lines

package/uglifyjs/uglifyjs.mk line 18

Quoted 1 line
package/uglifyjs/uglifyjs.mk new file +1 -0
+	$(INSTALL) -m 0755 $(@D)/package.json $(HOST_DIR)/lib/node_modules/uglify-js/

Something like:

	$(foreach p,bin lib tools package.json,\
		cp -dpfr $(@D)/$(p) $(HOST_DIR)/lib/node_modules/uglify-js/
	)

Indeed, it looks better with the loop, I'll update accordingly.

Quoted 5 lines

package/uglifyjs/uglifyjs.mk line 20

Quoted 2 lines
package/uglifyjs/uglifyjs.mk new file +2 -0
+	ln -sf ../lib/node_modules/uglify-js/bin/uglifyjs $(HOST_DIR)/bin/uglifyjs
+

Drop this empty new line.

Quoted 12 lines

support/testing/tests/package/test_uglifyjs.py

Quoted 8 lines
package/uglifyjs/uglifyjs.mk new file +3 -0
+endef
+
+$(eval $(host-generic-package))
support/testing/tests/package/test_uglifyjs.py new file +0 -0
new file mode 100644
index 000000000000..666a5fb46ff1

Forgot about commenting on this on the DEVELOPERS file: please add this file to the DEVELOPERS file as well, under your name. This way if this test fails in our CI, you'll get notified.

Ah,true, I've likely missed tests from some other packages then, I'll update here and send a separate patch for existing packages.

Alexis

Quoted 3 lines

Thanks!

Thomas

Signature · 8 lines
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#4 lore raw

PATCH 2/2 package/sass: new package

In reply to Alexis Lothoré via buildroot

sass is a css extension adding programming features to css.

https://github.com/sass/dart-sass

The source configured for the package is not a github repository, as the official sources for sass are in fact in Dart: those sources are automatically built into a javascript module by the corresponding github workflow ([1]). Rather than trying to replicate the whole dart build, this new package just downloads the sources for sass from npmjs directly.

[1]
https://github.com/sass/dart-sass/blob/main/.github/workflows/release.yml

---
 DEVELOPERS                                         |  1 +
 package/Config.in.host                             |  1 +
 package/sass/Config.in.host                        |  8 ++++++
 package/sass/sass.hash                             |  3 +++
 package/sass/sass.mk                               | 30 ++++++++++++++++++++++
 support/testing/tests/package/test_sass.py         | 23 +++++++++++++++++
 .../tests/package/test_sass/sample_sass.scss       |  7 +++++
 7 files changed, 73 insertions(+)
DEVELOPERS +1 -0
index 3027d1310528..4fee9b978467 100644
@@ -158,6 +158,7 @@ N: 	Alexis Lothoré <[email protected]>
 F:	package/libxmlsec1/
 F:	package/openscap/
 F: 	package/python-scp/
+F:	package/sass/
 F:	package/uglifyjs/
 
 N:	Alistair Francis <[email protected]>
package/Config.in.host +1 -0
index 0e4e29cd3514..4c5ed3865cb3 100644
@@ -109,6 +109,7 @@ menu "Host utilities"
 	source "package/rustc/Config.in.host"
 	source "package/s6-rc/Config.in.host"
 	source "package/sam-ba/Config.in.host"
+	source "package/sass/Config.in.host"
 	source "package/sdbus-cpp/Config.in.host"
 	source "package/sdbusplus/Config.in.host"
 	source "package/sentry-cli/Config.in.host"
package/sass/Config.in.host new file +8 -0
new file mode 100644
index 000000000000..13f47efc8e84
@@ -0,0 +1,8 @@
+config BR2_PACKAGE_HOST_SASS
+	bool "host sass"
+	help
+	  sass is a pure javascript Dart Sass distribution, with Sass
+	  being a CSS preprocessor, allowing to use programming features
+	  with CSS
+
+	  https://github.com/sass/dart-sass
package/sass/sass.hash new file +3 -0
new file mode 100644
index 000000000000..d8cce0da9d4a
@@ -0,0 +1,3 @@
+# Locally calculated
+sha256  cc0a9a8f9025c60c13eff214a13f71cbd0b76051dbe6e56f24990da73b85efdb  sass-1.102.0.tgz
+sha256  1523011dc1c6136ac2c4689f93b203f11690848d9641189b7a40848502d21568  LICENSE
package/sass/sass.mk new file +30 -0
new file mode 100644
index 000000000000..c79109be44a2
@@ -0,0 +1,30 @@
+################################################################################
+#
+# sass
+#
+################################################################################
+
+SASS_VERSION = 1.102.0
+SASS_SOURCE = sass-$(SASS_VERSION).tgz
+SASS_SITE = https://registry.npmjs.org/sass/-
+SASS_LICENSE = MIT
+SASS_LICENSE_FILES = LICENSE
+HOST_SASS_DEPENDENCIES = host-nodejs
+
+define HOST_SASS_BUILD_CMDS
+	$(HOST_DIR)/bin/npm install \
+		--prefix $(@D) \
+		--production \
+		--no-audit \
+		--no-fund
+endef
+
+define HOST_SASS_INSTALL_CMDS
+	$(INSTALL) -m 0755 -d $(HOST_DIR)/lib/node_modules/sass
+	$(INSTALL) -m 0755 $(@D)/sass.js $(HOST_DIR)/lib/node_modules/sass/
+	$(INSTALL) -m 0755 $(@D)/sass.dart.js $(HOST_DIR)/lib/node_modules/sass/
+	rsync -a $(@D)/node_modules $(HOST_DIR)/lib/node_modules/sass/
+	ln -sf ../lib/node_modules/sass/sass.js $(HOST_DIR)/bin/sass
+endef
+
+$(eval $(host-generic-package))
support/testing/tests/package/test_sass.py new file +23 -0
new file mode 100644
index 000000000000..2739eff7f680
@@ -0,0 +1,23 @@
+import os
+import infra
+from pathlib import Path
+
+from infra.basetest import BRTest
+
+INPUT_SAMPLE = Path(__file__).parent / "test_sass/sample_sass.scss"
+
+
+class TestHostSass(BRTest):
+    config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
+        """
+        BR2_PACKAGE_HOST_SASS=y
+        """
+
+    def test_run(self):
+        sass_bin = os.path.join(self.builddir, "host", "bin", "sass")
+
+        cmd = [sass_bin, "--version"]
+        infra.run_cmd_on_host(self.builddir, cmd)
+
+        cmd = [sass_bin, INPUT_SAMPLE, "output.css"]
+        infra.run_cmd_on_host(self.builddir, cmd)
support/testing/tests/package/test_sass/sample_sass.scss new file +7 -0
new file mode 100644
index 000000000000..b8bf7120118c
@@ -0,0 +1,7 @@
+$font-stack: Helvetica, sans-serif;
+$primary-color: #333;
+
+body {
+  font: 100% $font-stack;
+  color: $primary-color;
+}
Signature · 6 lines
2.55.0

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#5 lore raw

Re: [Buildroot] [PATCH 2/2] package/sass: new package

Hello Alexis,

Quoted 11 lines · Alexis Lothoré

sass is a css extension adding programming features to css.

https://github.com/sass/dart-sass

The source configured for the package is not a github repository, as the official sources for sass are in fact in Dart: those sources are automatically built into a javascript module by the corresponding github workflow ([1]). Rather than trying to replicate the whole dart build, this new package just downloads the sources for sass from npmjs directly.

Quoted 4 lines

[1]
https://github.com/sass/dart-sass/blob/main/.github/workflows/release.yml

I am not super clear on what sass exactly does/is. But some time ago, we dropped all JS libraries from Buildroot:

https://lore.kernel.org/buildroot/[email protected]/

But sass doesn't seem to be one of those static JS libraries I guess because it installs some executable host tool?

Quoted 15 lines
package/sass/sass.mk new file +9 -0
new file mode 100644
index 000000000000..c79109be44a2
@@ -0,0 +1,30 @@
+################################################################################
+#
+# sass
+#
+################################################################################
+
+SASS_VERSION = 1.102.0
+SASS_SOURCE = sass-$(SASS_VERSION).tgz
+SASS_SITE = https://registry.npmjs.org/sass/-

package/sass/sass.mk line 19

Quoted 10 lines
package/sass/sass.mk new file +10 -0
+SASS_LICENSE = MIT
+SASS_LICENSE_FILES = LICENSE
+HOST_SASS_DEPENDENCIES = host-nodejs
+
+define HOST_SASS_BUILD_CMDS
+	$(HOST_DIR)/bin/npm install \
+		--prefix $(@D) \
+		--production \
+		--no-audit \
+		--no-fund

What is this "npm install" doing? Isn't it downloading random stuff from the Internet?

package/sass/sass.mk line 26

Quoted 7 lines
package/sass/sass.mk new file +7 -0
+endef
+
+define HOST_SASS_INSTALL_CMDS
+	$(INSTALL) -m 0755 -d $(HOST_DIR)/lib/node_modules/sass
+	$(INSTALL) -m 0755 $(@D)/sass.js $(HOST_DIR)/lib/node_modules/sass/
+	$(INSTALL) -m 0755 $(@D)/sass.dart.js $(HOST_DIR)/lib/node_modules/sass/
+	rsync -a $(@D)/node_modules $(HOST_DIR)/lib/node_modules/sass/

"cp -dpfr" instead of "rsync -a".

support/testing/tests/package/test_sass.py

Quoted 9 lines
package/sass/sass.mk new file +4 -0
+	ln -sf ../lib/node_modules/sass/sass.js $(HOST_DIR)/bin/sass
+endef
+
+$(eval $(host-generic-package))
support/testing/tests/package/test_sass.py new file +0 -0
new file mode 100644
index 000000000000..2739eff7f680

Please add this file to the DEVELOPERS file.

Thomas

Signature · 7 lines
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#6 lore raw

Re: [Buildroot] [PATCH 2/2] package/sass: new package

Hi Thomas, Alexis,

Quoted 7 lines

I am not super clear on what sass exactly does/is. But some time ago, we dropped all JS libraries from Buildroot:

https://lore.kernel.org/buildroot/[email protected]/

But sass doesn't seem to be one of those static JS libraries I guess because it installs some executable host tool?

If I remember correctly node-sass is build using node-gyp so I can make sense to have it in Buildroot because the node-sass version is closely related to the node version present in Buildroot.

The question is how do you plan to use 'node-sass' once you have the executable. Correct me if I'm wrong but I guess the goal here is to use it in another Buildroot package that represent a "webapp" and you run 'node-sass' in the build step of that package and then place the generate css in the correct location ?

The correct way handle this (and minification) would probably be to use a dedicated JavaScript bundler and configure it to use node-sass and setup minification. This would probably yield smaller bundles and help handle versioning correctly for your webapp.
This was also the thinking in dropping the JS libraries from Buildroot.

If you have a package.json for the webapp with the node-sass dependency. Can you npm install the webapp dependencies using the buildroot 'npm' host-package without errors ? If that's the case I would say that it is probably a better solution.

Best regards,
PERALE Thomas
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#7 lore raw

Re: [Buildroot] [PATCH 2/2] package/sass: new package

Quoted 5 lines · Thomas Perale

Hi Thomas, Alexis,

Quoted 3 lines

I am not super clear on what sass exactly does/is. But some time ago, we dropped all JS libraries from Buildroot:

Quoted 8 lines
Quoted 4 lines

https://lore.kernel.org/buildroot/[email protected]/

But sass doesn't seem to be one of those static JS libraries I guess because it installs some executable host tool?

If I remember correctly node-sass is build using node-gyp so I can make sense to have it in Buildroot because the node-sass version is closely related to the node version present in Buildroot.

Just to make sure we are talking about the same thing, this package is not introducing node-sass ([1]), which is deprecated, but sass ([2])

Quoted 5 lines

The question is how do you plan to use 'node-sass' once you have the executable. Correct me if I'm wrong but I guess the goal here is to use it in another Buildroot package that represent a "webapp" and you run 'node-sass' in the build step of that package and then place the generate css in the correct location ?

Correct. To be exact, sass is not currently called directly by the custom buildroot package for "webapp". "webapp" is a php project, using composer to install and run "mini-asset" ([3]) which is a bundler depending on uglify and sass binaries being present on host.

Quoted 10 lines

The correct way handle this (and minification) would probably be to use a dedicated JavaScript bundler and configure it to use node-sass and setup minification. This would probably yield smaller bundles and help handle versioning correctly for your webapp.
This was also the thinking in dropping the JS libraries from Buildroot.

If you have a package.json for the webapp with the node-sass dependency. Can you npm install the webapp dependencies using the buildroot 'npm' host-package without errors ? If that's the case I would say that it is probably a better solution.

So in this case I've got no package.json, I only have a composer.json requiring mini_asset, and mini_asset does not automatically pull those dependencies. Mini-asset is configured through a .ini file, in which we can use different "filters" (eg: Sass), and each filter has different requirements, but it is up to the user to bring those dependencies on host. One thing that I can check however, close to your suggestion, is whether there is a different CSS "filter" that could be brought automatically by composer and that generates something close to what "webapp" is currently outputting.

Alexis

Quoted 3 lines

Best regards,
PERALE Thomas

[1] https://www.npmjs.com/package/node-sass
[2] https://www.npmjs.com/package/sass
[3] https://github.com/markstory/mini-asset

Signature · 8 lines
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#8 lore raw

Re: [Buildroot] [PATCH 2/2] package/sass: new package

Hello,

In reply of:

Quoted 11 lines · Thomas Perale
Quoted 10 lines

Hi Thomas, Alexis,

Quoted 7 lines

I am not super clear on what sass exactly does/is. But some time ago, we dropped all JS libraries from Buildroot:

https://lore.kernel.org/buildroot/[email protected]/

But sass doesn't seem to be one of those static JS libraries I guess because it installs some executable host tool?

Quoted 6 lines
Quoted 3 lines

If I remember correctly node-sass is build using node-gyp so I can make sense to have it in Buildroot because the node-sass version is closely related to the node version present in Buildroot.

Just to make sure we are talking about the same thing, this package is not introducing node-sass ([1]), which is deprecated, but sass ([2])

Ha yes indeed I was still thinking about that old node-sass package, I didn't know it was deprecated for good. Install should be easier with the new version.

Quoted 22 lines
Quoted 5 lines

The question is how do you plan to use 'node-sass' once you have the executable. Correct me if I'm wrong but I guess the goal here is to use it in another Buildroot package that represent a "webapp" and you run 'node-sass' in the build step of that package and then place the generate css in the correct location ?

Correct. To be exact, sass is not currently called directly by the custom buildroot package for "webapp". "webapp" is a php project, using composer to install and run "mini-asset" ([3]) which is a bundler depending on uglify and sass binaries being present on host.

Quoted 10 lines

The correct way handle this (and minification) would probably be to use a dedicated JavaScript bundler and configure it to use node-sass and setup minification. This would probably yield smaller bundles and help handle versioning correctly for your webapp.
This was also the thinking in dropping the JS libraries from Buildroot.

If you have a package.json for the webapp with the node-sass dependency. Can you npm install the webapp dependencies using the buildroot 'npm' host-package without errors ? If that's the case I would say that it is probably a better solution.

Quoted 9 lines

So in this case I've got no package.json, I only have a composer.json requiring mini_asset, and mini_asset does not automatically pull those dependencies. Mini-asset is configured through a .ini file, in which we can use different "filters" (eg: Sass), and each filter has different requirements, but it is up to the user to bring those dependencies on host. One thing that I can check however, close to your suggestion, is whether there is a different CSS "filter" that could be brought automatically by composer and that generates something close to what "webapp" is currently outputting.

I see, thanks for the clarification. I didn't know about 'mini-asset'. I think your project can still define a package.json for the 'frontend' part with uglify and sass dev dependencies and make 'mini-assets' point to the 'node_modules' directory local "bin".
But if you don't want to rely on 'package.json' or want vendoring then the best choice is probably adding the package to Buildroot indeed.

Best regards,
PERALE Thomas
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#9 lore raw

Re: [Buildroot] [PATCH 2/2] package/sass: new package

Hello Thomas, Thomas,

Quoted 4 lines · Thomas Perale

Hello,

In reply of:

Quoted 1 line · Thomas Perale

[...]

Quoted 10 lines
Quoted 9 lines

So in this case I've got no package.json, I only have a composer.json requiring mini_asset, and mini_asset does not automatically pull those dependencies. Mini-asset is configured through a .ini file, in which we can use different "filters" (eg: Sass), and each filter has different requirements, but it is up to the user to bring those dependencies on host. One thing that I can check however, close to your suggestion, is whether there is a different CSS "filter" that could be brought automatically by composer and that generates something close to what "webapp" is currently outputting.

Quoted 6 lines

I see, thanks for the clarification. I didn't know about 'mini-asset'. I think your project can still define a package.json for the 'frontend' part with uglify and sass dev dependencies and make 'mini-assets' point to the 'node_modules' directory local "bin".
But if you don't want to rely on 'package.json' or want vendoring then the best choice is probably adding the package to Buildroot indeed.

A small update on this:

While exploring the option of making the webapp build system manage the dependencies installation (as Thomas Perale suggested), we found out that we may be able to handle those directly with composer, without having to add a package.json: there seems to be alternatives to sass and uglifyjs that are packaged directly as php modules (and so, that can be pulled directly with composer install) and that mini-asset supports:
- sass -> scssphp
- uglifyjs -> jshrink

Some quick PoC validated that we are able to generate correctly the frontend assets with those alternatives, so we will likely go this way and get rid of those custom npm host packages.

If we ever happen to finally need those npm modules, Thomas Petazzoni also hinted me about the npm fetcher mechanism implemented in bitbake ([1]) that could be inspired from and ported to buildroot.

This series can then be dropped; thanks anyway for taking time to review it, and for the discussions/suggestions !

Alexis

[1] https://docs.yoctoproject.org/bitbake/2.18/bitbake-user-manual/bitbake-user-manual-fetching.html#npm-fetcher-npm

Quoted 2 lines

Best regards,
PERALE Thomas

Signature · 8 lines
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#10 lore raw

Re: [Buildroot] [PATCH 2/2] package/sass: new package

In reply to Thomas Petazzoni via buildroot
Quoted 17 lines · Thomas Petazzoni

Hello Alexis,

Quoted 14 lines · Alexis Lothoré

sass is a css extension adding programming features to css.

https://github.com/sass/dart-sass

The source configured for the package is not a github repository, as the official sources for sass are in fact in Dart: those sources are automatically built into a javascript module by the corresponding github workflow ([1]). Rather than trying to replicate the whole dart build, this new package just downloads the sources for sass from npmjs directly.

[1]
https://github.com/sass/dart-sass/blob/main/.github/workflows/release.yml

Quoted 6 lines
Quoted 1 line

I am not super clear on what sass exactly does/is. But some time ago, we dropped all JS libraries from Buildroot:

https://lore.kernel.org/buildroot/[email protected]/

Thanks for the link, good to know.

Quoted 2 lines

But sass doesn't seem to be one of those static JS libraries I guess because it installs some executable host tool?

Yes, the goal here really is to install a "sass binary" in $(HOST_DIR)/bin to be able to use it at build time, but "binary" is a kind of strong work: this really is a node script.

Quoted 20 lines
Quoted 20 lines
package/sass/sass.mk new file +14 -0
new file mode 100644
index 000000000000..c79109be44a2
@@ -0,0 +1,30 @@
+################################################################################
+#
+# sass
+#
+################################################################################
+
+SASS_VERSION = 1.102.0
+SASS_SOURCE = sass-$(SASS_VERSION).tgz
+SASS_SITE = https://registry.npmjs.org/sass/-
+SASS_LICENSE = MIT
+SASS_LICENSE_FILES = LICENSE
+HOST_SASS_DEPENDENCIES = host-nodejs
+
+define HOST_SASS_BUILD_CMDS
Quoted 8 lines

package/sass/sass.mk line 19

Quoted 5 lines
package/sass/sass.mk new file +5 -0
+	$(HOST_DIR)/bin/npm install \
+		--prefix $(@D) \
+		--production \
+		--no-audit \
+		--no-fund

What is this "npm install" doing? Isn't it downloading random stuff from the Internet?

Yes, this npm install downloads 3 dependencies for this sass tool. I understand that it is not ideal (having some download in the build step), but proposed it anyway, as it was a potential "known issue" in Arnout's proposal ([1]). But if this is finally a no-go, thinking a bit more about it, I see two main options:
- find a way to vendor the dependencies (may be as simple as setting a post download hook which performs the npm install ?)
- this sass package is a node wrapper around the official project dart-sass ([2]). I did not take a look at all at how to build and run it (I am pretty ignorant in Dart), but I can take a look. I am not sure how capable Buildroot is about building Dart projects (I see some flutter-sdk-bin package, I am not sure how re-usable it could be to build some randome dart projects)

[1] https://lore.kernel.org/buildroot/[email protected]/
[2] https://sass-lang.com/dart-sass/

Alexis

Signature · 8 lines
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#11 lore raw

Re: [Buildroot] [PATCH 2/2] package/sass: new package

Hello,

Quoted 7 lines · Alexis Lothoré

Yes, this npm install downloads 3 dependencies for this sass tool. I understand that it is not ideal (having some download in the build step), but proposed it anyway, as it was a potential "known issue" in Arnout's proposal ([1]). But if this is finally a no-go, thinking a bit more about it, I see two main options:
- find a way to vendor the dependencies (may be as simple as setting a post download hook which performs the npm install ?)

This is what Arnout was proposing. However, a post download hook doesn't work because it's too late: the tarball has already been created, while we want the dependencies to be *part* of the tarball, i.e we want the vendoring (collecting the dependencies) to happen prior to the tarball being created and added to $(DL_DIR).

Instead, you should have a look at how it works for the cargo and golang infrastructures.

We have this concept of <pkg>_DOWNLOAD_POST_PROCESS (which is NOT the same as <pkg>_POST_DOWNLOAD_HOOKS!).

package/pkg-cargo.mk does:

$(2)_DOWNLOAD_POST_PROCESS = cargo

which causes support/download/cargo-post-process to be called in the middle of the download step.

What it does is basically:

1. Unpack the tarball that was downloaded
2. Run "cargo vendor" to download the dependencies
3. Repack the tarball

support/download/go-post-process works in a similar manner, but runs "go mod vendor" to download the dependencies.

If you can identify the right incantation to ask npm to download the dependencies so that we can implement something like this, then it would be great.

Quoted 6 lines

- this sass package is a node wrapper around the official project dart-sass ([2]). I did not take a look at all at how to build and run it (I am pretty ignorant in Dart), but I can take a look. I am not sure how capable Buildroot is about building Dart projects (I see some flutter-sdk-bin package, I am not sure how re-usable it could be to build some randome dart projects)

I have also no idea :-)

Thomas

Signature · 7 lines
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot