Skip to contentSkip to conversation

rework luvi test in gitlab after luajit version bump.

PATCH Applied Romain Naour 2 participants 9 messages Nov 12, 2021

lore
Share & tools
lore

Romain Naour

author
#0 lore raw

v2 cover rework luvi test in gitlab after luajit version bump.

Luajit package has been recently updated to the latest commit in the master branch [1]. But we discovered several problems:

- LUAJIT_VERSION doesn't contain the luajit version anymore but a commit hash
- With Luajit, luvi doesn't work without rng support enabled.
- Building for 64-bit target requires a 64-bit host for all target platform
- luajit and host-luajit needs to be built with the same option, so the bytecode generated by host-luajit is valid on luajit.
- iteration with pairs() does not result in the same order since luajit 2.1.

[1] 9450b53c8e06f1b5a75840b82c0f4663a5d75c45
[2] https://gitlab.com/buildroot.org/buildroot/-/jobs/1710552605

Best regards,
Romain

Francois Perrad (1):
package/luajit: disable 64 bit GC objects

Romain Naour (5):
package/luvi: don't use LUAJIT_VERSION for the luajit installation
path
support/testing: test_luvi: switch to armv5 to boot with rng support
enabled
package/luajit: rework BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS package/luajit: building for 64-bit target requires a 64-bit host for
all platform
support/testing: test_luvi: make luvi test reproducible

 package/luajit/Config.in                   | 12 +++++++-----
 package/luajit/luajit.mk                   |  2 ++
 package/luvi/luvi.mk                       |  8 ++++++--
 support/testing/tests/package/test_luvi.py | 12 ++++++------
 4 files changed, 21 insertions(+), 13 deletions(-)
Signature · 6 lines
2.31.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
author
#1 lore raw

PATCH v2 1/6 package/luvi: don't use LUAJIT_VERSION for the luajit installation path

Luajit package has been recently updated to the latest commit in the master branch [1]. Since then LUAJIT_VERSION doesn't contain the luajit version anymore but a commit hash:

LUAJIT_VERSION = 05f1984e1a862e4b3d3c3b370c773492e2edf84a

Use pkg-config --variable=version luajit in luvi package to set LUA_PATH correctly.

Fixes:
luajit: unknown luaJIT command or jit.* modules not installed

https://gitlab.com/buildroot.org/buildroot/-/jobs/1710552605

[1] 9450b53c8e06f1b5a75840b82c0f4663a5d75c45

Signed-off-by Romain Naour <[email protected]>
Cc Jörg Krause <[email protected]>

---
The TestLuvi doesn't work yet due to missing rng support on Qemu armv7 target used to run the test. luvi -v is stuck due to getrandom().

v2: use build-time shell expansion (Yann)

---
 package/luvi/luvi.mk | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
package/luvi/luvi.mk +6 -2
index 386daafc31..5daa3c1793 100644
@@ -9,7 +9,7 @@ LUVI_SOURCE = luvi-src-v$(LUVI_VERSION).tar.gz
 LUVI_SITE = https://github.com/luvit/luvi/releases/download/v$(LUVI_VERSION)
 LUVI_LICENSE = Apache-2.0
 LUVI_LICENSE_FILES = LICENSE.txt
-LUVI_DEPENDENCIES = libuv luajit luv host-luajit
+LUVI_DEPENDENCIES = libuv luajit luv host-luajit host-pkgconf
 
 # Dispatch all architectures of LuaJIT
 ifeq ($(BR2_i386),y)

@@ -30,12 +30,16 @@ else
 LUVI_TARGET_ARCH = $(BR2_ARCH)
 endif
 
+# LUAJIT_VERSION and the luajit installation path may not use the
+# same value. Use the value from luajit.pc file.
+LUVI_LUAJIT_VERSION = `$(PKG_CONFIG_HOST_BINARY) --variable=version luajit`
+
 # Bundled lua bindings have to be linked statically into the luvi executable
 LUVI_CONF_OPTS = \
 	-DBUILD_SHARED_LIBS=OFF \
 	-DWithSharedLibluv=ON \
 	-DTARGET_ARCH=$(LUVI_TARGET_ARCH) \
-	-DLUA_PATH=$(HOST_DIR)/share/luajit-$(LUAJIT_VERSION)/?.lua
+	-DLUA_PATH=$(HOST_DIR)/share/luajit-$(LUVI_LUAJIT_VERSION)/?.lua
 
 # Add "rex" module (PCRE via bundled lrexlib)
 ifeq ($(BR2_PACKAGE_PCRE),y)
Signature · 6 lines
2.31.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
author
#2 lore raw

PATCH v2 2/6 support/testing: test_luvi: switch to armv5 to boot with rng support enabled

In reply to Romain Naour

Since the commit replacing moonjit by luajit [1] luvi doesn't work without rng support enabled.

Switch to armv5 to use virtio-rng-pci on the qemu command line [2].

[1] 9450b53c8e06f1b5a75840b82c0f4663a5d75c45
[2] https://git.buildroot.net/buildroot/tree/support/testing/infra/emulator.py?h=2021.08.1

Signed-off-by Romain Naour <[email protected]>
Cc Jörg Krause <[email protected]>

---
luvi still doesn't work since the luv static linking doesn't seems to work anymore.
(see https://github.com/luvit/luvi/commit/a9dc8dae2eb69d4a754a0cf5c08622d70f6b7067)

 # luvi -v

[string "return require('init')(...)"]:1: module 'init' not found:

	no field package.preload['init']
	no file './init.lua'
	no file '/usr/share/luajit-2.1.0-beta3/init.lua'
	no file '/usr/local/share/lua/5.1/init.lua'
	no file '/usr/local/share/lua/5.1/init/init.lua'
	no file '/usr/share/lua/5.1/init.lua'
	no file '/usr/share/lua/5.1/init/init.lua'
	no file './init.so'
	no file '/usr/local/lib/lua/5.1/init.so'
	no file '/usr/lib/lua/5.1/init.so'
	no file '/usr/local/lib/lua/5.1/loadall.so'

stack traceback:

	[C]: in function 'require'
	[string "return require('init')(...)"]:1: in main chunk

luvi now try to load init.lua script:
https://github.com/luvit/luvi/blob/v2.12.0/src/main.c#L176

Reverting to moonjit fixes the issue.

---
 support/testing/tests/package/test_luvi.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
support/testing/tests/package/test_luvi.py +1 -1
index a85a6f4d27..eb15ad31b5 100644
@@ -17,7 +17,7 @@ class TestLuvi(infra.basetest.BRTest):
 
     def login(self):
         cpio_file = os.path.join(self.builddir, "images", "rootfs.cpio")
-        self.emulator.boot(arch="armv7",
+        self.emulator.boot(arch="armv5",
                            kernel="builtin",
                            options=["-initrd", cpio_file])
         self.emulator.login()
Signature · 6 lines
2.31.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
author
#3 lore raw

PATCH v2 3/6 package/luajit: rework BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS

In reply to Romain Naour

This commit reworks how BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS is defined to follow the recommended syntax.

Signed-off-by Romain Naour <[email protected]>
Cc Francois Perrad <[email protected]>
Cc Yann E. MORIN <[email protected]>
---
 package/luajit/Config.in | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
package/luajit/Config.in +5 -5
index e4e1249240..f6d01ed09a 100644
@@ -1,10 +1,10 @@
 config BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
 	bool
-	default y if BR2_i386 || \
-		(BR2_x86_64 && BR2_HOSTARCH='x86_64') || \
-		BR2_powerpc || BR2_arm || BR2_armeb || BR2_aarch64 || \
-		((BR2_mips || BR2_mipsel) && !BR2_MIPS_SOFT_FLOAT && \
-		!BR2_MIPS_CPU_MIPS32R6 && !BR2_MIPS_CPU_MIPS64R6)
+	default y if BR2_arm || BR2_armeb || BR2_aarch64
+	default y if BR2_i386 || (BR2_x86_64 && BR2_HOSTARCH='x86_64')
+	default y if BR2_powerpc
+	default y if (BR2_mips || BR2_mipsel) && !BR2_MIPS_SOFT_FLOAT \
+		&& !BR2_MIPS_CPU_MIPS32R6 && !BR2_MIPS_CPU_MIPS64R6
 	# -m32 flag is used for 32bit builds and host-luajit has
 	# limited architecture support
 	depends on BR2_HOSTARCH = "x86_64" || BR2_HOSTARCH = "x86"
Signature · 6 lines
2.31.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Applied
#4 lore raw

Re: [Buildroot] [PATCH v2 3/6] package/luajit: rework BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS

On 2021-11-12 20:06 +0100, Romain Naour spake thusly:

Quoted 16 lines

This commit reworks how BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS is defined to follow the recommended syntax.

Signed-off-by Romain Naour <[email protected]>
Cc Francois Perrad <[email protected]>
Cc Yann E. MORIN <[email protected]>
---
 package/luajit/Config.in | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
package/luajit/Config.in +0 -0
index e4e1249240..f6d01ed09a 100644
@@ -1,10 +1,10 @@
 config BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS

package/luajit/Config.in line 6

Quoted 10 lines
package/luajit/Config.in +4 -5
 	bool
-	default y if BR2_i386 || \
-		(BR2_x86_64 && BR2_HOSTARCH='x86_64') || \
-		BR2_powerpc || BR2_arm || BR2_armeb || BR2_aarch64 || \
-		((BR2_mips || BR2_mipsel) && !BR2_MIPS_SOFT_FLOAT && \
-		!BR2_MIPS_CPU_MIPS32R6 && !BR2_MIPS_CPU_MIPS64R6)
+	default y if BR2_arm || BR2_armeb || BR2_aarch64
+	default y if BR2_i386 || (BR2_x86_64 && BR2_HOSTARCH='x86_64')
+	default y if BR2_powerpc
+	default y if (BR2_mips || BR2_mipsel) && !BR2_MIPS_SOFT_FLOAT \

Alphabetical order, damit! ;-)

Applied to master, thanks.

Regards,
Yann E. MORIN.

Quoted 11 lines
package/luajit/Config.in +1 -0
+		&& !BR2_MIPS_CPU_MIPS32R6 && !BR2_MIPS_CPU_MIPS64R6
 	# -m32 flag is used for 32bit builds and host-luajit has
 	# limited architecture support
 	depends on BR2_HOSTARCH = "x86_64" || BR2_HOSTARCH = "x86"
Signature · 6 lines
2.31.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
author
#5 lore raw

PATCH v2 4/6 package/luajit: building for 64-bit target requires a 64-bit host for all platform

In reply to Romain Naour

Currently only x86_64 target requires a 64-bit host but building aarch64 target with a 32-bit host is allowed.

This constraint is for all architectures [1].

[1] http://lists.busybox.net/pipermail/buildroot/2021-November/628301.html

Signed-off-by Romain Naour <[email protected]>
Cc Francois Perrad <[email protected]>
Cc Yann E. MORIN <[email protected]>

---
v2: fix typo BR_HOST_ARCH/BR2_HOSTARCH (Yann)

---
 package/luajit/Config.in | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
package/luajit/Config.in +3 -1
index f6d01ed09a..c313cd3248 100644
@@ -1,13 +1,15 @@
 config BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS
 	bool
 	default y if BR2_arm || BR2_armeb || BR2_aarch64
-	default y if BR2_i386 || (BR2_x86_64 && BR2_HOSTARCH='x86_64')
+	default y if BR2_i386 || BR2_x86_64
 	default y if BR2_powerpc
 	default y if (BR2_mips || BR2_mipsel) && !BR2_MIPS_SOFT_FLOAT \
 		&& !BR2_MIPS_CPU_MIPS32R6 && !BR2_MIPS_CPU_MIPS64R6
 	# -m32 flag is used for 32bit builds and host-luajit has
 	# limited architecture support
 	depends on BR2_HOSTARCH = "x86_64" || BR2_HOSTARCH = "x86"
+	# Building for 64-bit target requires a 64-bit host
+	depends on !BR2_ARCH_IS_64 || BR2_HOSTARCH = "x86_64"
 
 config BR2_PACKAGE_LUAJIT
 	bool "luajit"
Signature · 6 lines
2.31.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
author
#6 lore raw

PATCH v2 5/6 package/luajit: disable 64 bit GC objects

In reply to Romain Naour

From: Francois Perrad <[email protected]>

Luvi uses the cross bytecode generation of host-luajit the bytecode format has 2 flavors, depending of GC64 option.

Since the commit https://github.com/LuaJIT/LuaJIT/commit/bd00094c3b50e193fb32aad79b7ea8ea6b78ed25
GC64 mode is enable by default on all 64bits platform.

With this patch, luajit and host-luajit are built with the same option, so the bytecode generated by host-luajit is valid on luajit.

Signed-off-by Francois Perrad <[email protected]>

[Romain:
- update commit title
- add a comment about missing LUAJIT_ENABLE_GC64]

Signed-off-by Romain Naour <[email protected]>

---
The luvi test still doesn't works all the time since the output of "luvi -v" is not reproducible

---
 package/luajit/luajit.mk | 2 ++
 1 file changed, 2 insertions(+)
package/luajit/luajit.mk +2 -0
index fbbd8ebc70..89b164c96d 100644
@@ -26,8 +26,10 @@ endif
 # libraries are installed.
 ifeq ($(BR2_ARCH_IS_64),y)
 LUAJIT_HOST_CC = $(HOSTCC)
+# There is no LUAJIT_ENABLE_GC64 option.
 else
 LUAJIT_HOST_CC = $(HOSTCC) -m32
+LUAJIT_XCFLAGS += -DLUAJIT_DISABLE_GC64
 endif
 
 # We unfortunately can't use TARGET_CONFIGURE_OPTS, because the luajit
Signature · 6 lines
2.31.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
author
#7 lore raw

PATCH v2 6/6 support/testing: test_luvi: make luvi test reproducible

In reply to Romain Naour

As explained by Jörg [1], iteration with pairs() does not result in the same order since luajit 2.1.

From [2]
"Table iteration with pairs() does not result in the same order?

The order of table iteration is explicitly undefined by the Lua language standard. Different Lua implementations or versions may use different orders for otherwise identical tables. Different ways of constructing a table may result in different orders, too. Due to improved VM security, LuaJIT 2.1 may even use a different order on separate VM invocations or when string keys are newly interned.

If your program relies on a deterministic order, it has a bug. Rewrite it, so it doesn't rely on the key order.
Or sort the table keys, if you must."

Note: The "luvi -v" return 255 even on success.

[1] http://lists.busybox.net/pipermail/buildroot/2021-November/627938.html
[2] https://luajit.org/faq.html

Signed-off-by Romain Naour <[email protected]>
Cc Francois Perrad <[email protected]>
Cc Jörg Krause <[email protected]>

---
v2 use sort in python (Yann)

---
 support/testing/tests/package/test_luvi.py | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
support/testing/tests/package/test_luvi.py +5 -4
index eb15ad31b5..4b55091762 100644
@@ -25,11 +25,12 @@ class TestLuvi(infra.basetest.BRTest):
     def version_test(self):
         cmd = "luvi -v"
         output, exit_code = self.emulator.run(cmd)
-        self.assertIn('luvi', output[0])
-        self.assertIn('zlib', output[1])
+        output = sorted(output)
+        self.assertIn('libuv', output[0])
+        self.assertIn('luvi', output[1])
         self.assertIn('rex', output[2])
-        self.assertIn('libuv', output[3])
-        self.assertIn('ssl', output[4])
+        self.assertIn('ssl', output[3])
+        self.assertIn('zlib', output[4])
 
     def test_run(self):
         self.login()
Signature · 6 lines
2.31.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#8 lore raw
In reply to Romain Naour

Romain, All,

On 2021-11-12 20:06 +0100, Romain Naour spake thusly:

Quoted 9 lines

Luajit package has been recently updated to the latest commit in the master branch [1]. But we discovered several problems:

- LUAJIT_VERSION doesn't contain the luajit version anymore but a commit hash
- With Luajit, luvi doesn't work without rng support enabled.
- Building for 64-bit target requires a 64-bit host for all target platform
- luajit and host-luajit needs to be built with the same option, so the bytecode generated by host-luajit is valid on luajit.
- iteration with pairs() does not result in the same order since luajit 2.1.

Whole series applied to master, thanks!

Regards,
Yann E. MORIN.

Quoted 32 lines

[1] 9450b53c8e06f1b5a75840b82c0f4663a5d75c45
[2] https://gitlab.com/buildroot.org/buildroot/-/jobs/1710552605

Best regards,
Romain

Francois Perrad (1):
package/luajit: disable 64 bit GC objects

Romain Naour (5):
package/luvi: don't use LUAJIT_VERSION for the luajit installation
path
support/testing: test_luvi: switch to armv5 to boot with rng support
enabled
package/luajit: rework BR2_PACKAGE_LUAJIT_ARCH_SUPPORTS package/luajit: building for 64-bit target requires a 64-bit host for
all platform
support/testing: test_luvi: make luvi test reproducible

 package/luajit/Config.in                   | 12 +++++++-----
 package/luajit/luajit.mk                   |  2 ++
 package/luvi/luvi.mk                       |  8 ++++++--
 support/testing/tests/package/test_luvi.py | 12 ++++++------
 4 files changed, 21 insertions(+), 13 deletions(-)
Signature · 6 lines
2.31.1

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot