luajit: new package (v2)

---
 package/Config.in                     |    3 ++-
 package/luajit/Config.in              |    9 +++++++
 package/luajit/luajit-root-path.patch |   16 ++++++++++++
 package/luajit/luajit.mk              |   44 +++++++++++++++++++++++++++++++++
 4 files changed, 71 insertions(+), 1 deletion(-)
 create mode 100644 package/luajit/Config.in
 create mode 100644 package/luajit/luajit-root-path.patch
 create mode 100644 package/luajit/luajit.mk

index fb1b08f..fde30b0 100644
@@ -237,7 +237,8 @@ endmenu
 menu "Interpreter languages and scripting"
 source "package/haserl/Config.in"
 source "package/lua/Config.in"
-if BR2_PACKAGE_LUA
+source "package/luajit/Config.in"
+if BR2_PACKAGE_LUA || BR2_PACKAGE_LUAJIT
 menu "LUA libraries/modules"
 source "package/cgilua/Config.in"
 source "package/copas/Config.in"

new file mode 100644
index 0000000..c665b28
@@ -0,0 +1,9 @@
+config BR2_PACKAGE_LUAJIT
+	bool "luajit"
+	help
+	  LuaJIT implements the full set of language features defined by Lua 5.1. The
+	  virtual machine (VM) is API- and ABI-compatible to the standard

Lua interpreter
+ and can be deployed as a drop-in replacement.
+
+ http://luajit.org/
+

b/package/luajit/luajit-root-path.patch
new file mode 100644
index 0000000..2eb037c

@@ -0,0 +1,16 @@
+diff -Naur LuaJIT-2.0.0-beta10.orig/src/luaconf.h

LuaJIT-2.0.0-beta10/src/luaconf.h
+--- LuaJIT-2.0.0-beta10.orig/src/luaconf.h 2012-05-30 20:33:27.989333819 +0200
++++ LuaJIT-2.0.0-beta10/src/luaconf.h 2012-05-30 20:46:30.581354777 +0200
+@@ -22,9 +22,9 @@
+ #define LUA_CPATH_DEFAULT \
+ ".\\?.dll;" LUA_CDIR"?.dll;" LUA_CDIR"loadall.dll"
+ #else

+-#define LUA_ROOT	"/usr/local/"
+-#define LUA_LDIR	LUA_ROOT "share/lua/5.1/"
+-#define LUA_CDIR	LUA_ROOT "lib/lua/5.1/"

++#define LUA_ROOT "/usr/"
++#define LUA_LDIR LUA_ROOT "share/lua/"
++#define LUA_CDIR LUA_ROOT "lib/lua/"
+ #ifdef LUA_XROOT
+ #define LUA_JDIR LUA_XROOT "share/luajit-2.0.0-beta10/"
+ #define LUA_XPATH \

new file mode 100644
index 0000000..b91ddbc
@@ -0,0 +1,44 @@
+#############################################################
+#
+# luajit
+#
+#############################################################
+
+LUAJIT_VERSION = 2.0.0-beta10
+LUAJIT_SOURCE:=LuaJIT-$(LUAJIT_VERSION).tar.gz
+LUAJIT_SITE = http://luajit.org/download
+LUAJIT_INSTALL_STAGING = YES
+
+define LUAJIT_INSTALL_BETA_FIX_MAKEFILE
+	$(SED) 's/@echo "  $$(SYMLINK) $$(INSTALL_TNAME)

$$(INSTALL_TSYM)"/$$(SYMLINK) $$(INSTALL_TNAME) $$(INSTALL_TSYM)/' $(@D)/Makefile
+endef
+
+ifeq ($(findstring beta,$(LUAJIT_VERSION)),beta) +LUAJIT_POST_PATCH_HOOKS += LUAJIT_INSTALL_BETA_FIX_MAKEFILE +endif
+
+define LUAJIT_NOLARGEFILE_FIX_MAKEFILE
+ $(SED) 's/TARGET_XCFLAGS= -D_FILE_OFFSET_BITS=64
-D_LARGEFILE_SOURCE/TARGET_XCFLAGS=/' $(@D)/src/Makefile +endef
+
+ifneq ($(BR2_LARGEFILE),y)
+LUAJIT_POST_PATCH_HOOKS += LUAJIT_NOLARGEFILE_FIX_MAKEFILE +endif
+
+define LUAJIT_BUILD_CMDS
+ $(MAKE) PREFIX="/usr" CROSS="$(TARGET_CROSS)" -C $(@D) amalg
+endef
+
+define LUAJIT_INSTALL_STAGING_CMDS
+ $(MAKE) PREFIX="/usr" DESTDIR="$(STAGING_DIR)" -C $(@D) install
+endef
+
+define LUAJIT_INSTALL_TARGET_CMDS
+ $(MAKE) PREFIX="/usr" DESTDIR="$(TARGET_DIR)" -C $(@D) install
+endef
+
+define LUAJIT_CLEAN_CMDS
+ -$(MAKE) -C $(@D) clean
+endef
+
+$(eval $(call GENTARGETS,package,luajit))

1.7.9.5

Hello,

Le Sat, 2 Jun 2012 10:49:14 +0200,
Fran?ois Perrad <[email protected]> a ?crit :

I started integrating this. I made a number of changes (link the luajit binary dynamically against its shared library, remove the symbolic thing, addition of the proper 'depends on' for the architectures that luajit supports, etc.).

However, I've hit a wall with the fact that apparently the luajit build process requires the host compiler to have the same bitness as the target architecture. As I was building for ARM on a x86_64 machine, luajit wasn't happy, and suggested to add -m32 to the host compile flags. With this, it works.

Unfortunately, it has a number of drawbacks:

* Requires the host to have multilib libraries available. This is not a big problem IMO, as most x86_64 machines will most likely have the 32 bits libraries installed for one reason or another, and we are anyway thinking of using this multilib capability to build Grub when the target architecture is x86_64.

* The major problem is that it means that if your build machine is a 32 bits machine, then you cannot build luajit for a 64 bits architecture (at the moment, the only 64 bits architecture supported by luajit is x86_64, so basically, on a i386 build machine, you can't build a x86_64 target).

Can you have a look at this and see what's possible to do? I will send as a reply the current state of the luajit patch that includes my modifications.

Thanks!

Thomas

Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

From: Fran?ois Perrad <[email protected]>

[thomas.petazzoni at free-electrons.com:
- Added shared library patch to link luajit dynamically against the luajit library
- Added no strip patch.
- Added no symlink patch.
- Removed symlink creation logic from luajit.mk.
- Fixed compilation on x86_64 hosts by adding -m32 to the compile flags.
- Added 'depends on' for the architectures supported by luajit.
]

---
 package/Config.in                             |    3 +-
 package/luajit/Config.in                      |   10 ++++
 package/luajit/luajit-01-root-path.patch      |   22 +++++++++
 package/luajit/luajit-02-shared-lib.patch     |   17 +++++++
 package/luajit/luajit-03-dont-strip.patch     |   23 ++++++++++
 package/luajit/luajit-04-no-bin-symlink.patch |   40 ++++++++++++++++
 package/luajit/luajit.mk                      |   61 +++++++++++++++++++++++++
 7 files changed, 175 insertions(+), 1 deletion(-)
 create mode 100644 package/luajit/Config.in
 create mode 100644 package/luajit/luajit-01-root-path.patch
 create mode 100644 package/luajit/luajit-02-shared-lib.patch
 create mode 100644 package/luajit/luajit-03-dont-strip.patch
 create mode 100644 package/luajit/luajit-04-no-bin-symlink.patch
 create mode 100644 package/luajit/luajit.mk

index 99257b4..a117fc4 100644
@@ -245,7 +245,8 @@ endmenu
 menu "Interpreter languages and scripting"
 source "package/haserl/Config.in"
 source "package/lua/Config.in"
-if BR2_PACKAGE_LUA
+source "package/luajit/Config.in"
+if BR2_PACKAGE_LUA || BR2_PACKAGE_LUAJIT
 menu "LUA libraries/modules"
 source "package/cgilua/Config.in"
 source "package/copas/Config.in"

new file mode 100644
index 0000000..3ddd3c0
@@ -0,0 +1,10 @@
+config BR2_PACKAGE_LUAJIT
+	bool "luajit"
+	depends on BR2_i386 || BR2_x86_64 || BR2_powerpc || BR2_arm || BR2_armeb
+	help
+	  LuaJIT implements the full set of language features defined
+	  by Lua 5.1. The virtual machine (VM) is API- and
+	  ABI-compatible to the standard Lua interpreter and can be
+	  deployed as a drop-in replacement.
+
+	  http://luajit.org/

new file mode 100644
index 0000000..3b0a585
@@ -0,0 +1,22 @@
+Adjust installation location to /usr
+
+Signed-off-by: Francois Perrad <[email protected]>
+Signed-off-by: Thomas Petazzoni <[email protected]>
+
+Index: b/src/luaconf.h
+===================================================================
+--- a/src/luaconf.h
++++ b/src/luaconf.h
+@@ -22,9 +22,9 @@
+ #define LUA_CPATH_DEFAULT \
+   ".\\?.dll;" LUA_CDIR"?.dll;" LUA_CDIR"loadall.dll"
+ #else
+-#define LUA_ROOT	"/usr/local/"
+-#define LUA_LDIR	LUA_ROOT "share/lua/5.1/"
+-#define LUA_CDIR	LUA_ROOT "lib/lua/5.1/"
++#define LUA_ROOT	"/usr/"
++#define LUA_LDIR	LUA_ROOT "share/lua/"
++#define LUA_CDIR	LUA_ROOT "lib/lua/"
+ #ifdef LUA_XROOT
+ #define LUA_JDIR	LUA_XROOT "share/luajit-2.0.0-beta10/"
+ #define LUA_XPATH \

new file mode 100644
index 0000000..d5122ba
@@ -0,0 +1,17 @@
+Compile the luajit binary dynamically against the luajit library
+
+Signed-off-by: Thomas Petazzoni <[email protected]>
+
+Index: b/src/Makefile
+===================================================================
+--- a/src/Makefile
++++ b/src/Makefile
+@@ -629,7 +629,7 @@
+ 
+ $(LUAJIT_T): $(TARGET_O) $(LUAJIT_O) $(TARGET_DEP)
+ 	$(E) "LINK      $@"
+-	$(Q)$(TARGET_LD) $(TARGET_ALDFLAGS) -o $@ $(LUAJIT_O) $(TARGET_O) $(TARGET_ALIBS)
++	$(Q)$(TARGET_LD) $(TARGET_ALDFLAGS) -o $@ $(LUAJIT_O) -lluajit -L. $(TARGET_ALIBS)
+ 	$(Q)$(TARGET_STRIP) $@
+ 	$(E) "OK        Successfully built LuaJIT"
+ 

new file mode 100644
index 0000000..9651cdf
@@ -0,0 +1,23 @@
+Don't strip the library and binary
+
+This allows Buildroot to do it if needed, but only if needed.
+
+Signed-off-by: Thomas Petazzoni <[email protected]>
+
+Index: b/src/Makefile
+===================================================================
+--- a/src/Makefile
++++ b/src/Makefile
+@@ -625,12 +625,10 @@
+ $(LUAJIT_SO): $(LJVMCORE_O)
+ 	$(E) "DYNLINK   $@"
+ 	$(Q)$(TARGET_LD) $(TARGET_ASHLDFLAGS) -o $@ $(LJVMCORE_DYNO) $(TARGET_ALIBS)
+-	$(Q)$(TARGET_STRIP) $@
+ 
+ $(LUAJIT_T): $(TARGET_O) $(LUAJIT_O) $(TARGET_DEP)
+ 	$(E) "LINK      $@"
+ 	$(Q)$(TARGET_LD) $(TARGET_ALDFLAGS) -o $@ $(LUAJIT_O) -lluajit -L. $(TARGET_ALIBS)
+-	$(Q)$(TARGET_STRIP) $@
+ 	$(E) "OK        Successfully built LuaJIT"
+ 
+ ##############################################################################

new file mode 100644
index 0000000..3eed7de
@@ -0,0 +1,40 @@
+Do not use a symlink for the binary, simply name if luajit
+
+Signed-off-by: Thomas Petazzoni <[email protected]>
+
+Index: b/Makefile
+===================================================================
+--- a/Makefile
++++ b/Makefile
+@@ -41,8 +41,7 @@
+ INSTALL_MAN= $(INSTALL_SHARE)/man/man1
+ INSTALL_PKGCONFIG= $(INSTALL_LIB)/pkgconfig
+ 
+-INSTALL_TNAME= luajit-$(VERSION)
+-INSTALL_TSYMNAME= luajit
++INSTALL_TNAME= luajit
+ INSTALL_ANAME= libluajit-$(ABIVER).a
+ INSTALL_SONAME= libluajit-$(ABIVER).so.$(MAJVER).$(MINVER).$(RELVER)
+ INSTALL_SOSHORT= libluajit-$(ABIVER).so
+@@ -56,7 +55,6 @@
+ INSTALL_SHORT1= $(INSTALL_LIB)/$(INSTALL_SOSHORT)
+ INSTALL_SHORT2= $(INSTALL_LIB)/$(INSTALL_SOSHORT)
+ INSTALL_T= $(INSTALL_BIN)/$(INSTALL_TNAME)
+-INSTALL_TSYM= $(INSTALL_BIN)/$(INSTALL_TSYMNAME)
+ INSTALL_PC= $(INSTALL_PKGCONFIG)/$(INSTALL_PCNAME)
+ 
+ INSTALL_DIRS= $(INSTALL_BIN) $(INSTALL_LIB) $(INSTALL_INC) $(INSTALL_MAN) \
+@@ -114,13 +112,6 @@
+ 	  $(RM) $(FILE_PC).tmp
+ 	cd src && $(INSTALL_F) $(FILES_INC) $(INSTALL_INC)
+ 	cd lib && $(INSTALL_F) $(FILES_JITLIB) $(INSTALL_JITLIB)
+-	@echo "==== Successfully installed LuaJIT $(VERSION) to $(PREFIX) ===="
+-	@echo ""
+-	@echo "Note: the beta releases deliberately do NOT install a symlink for luajit"
+-	@echo "You can do this now by running this command (with sudo):"
+-	@echo ""
+-	@echo "  $(SYMLINK) $(INSTALL_TNAME) $(INSTALL_TSYM)"
+-	@echo ""
+ 
+ ##############################################################################
+ 

1.7.9.5

With a x86 host and an ARM target, I obtain the following error :

/usr/bin/make -j2 PREFIX="/usr" STATIC_CC="/home/user/build/qarm/host/usr/bin/arm-unknown-linux-uclibcgnueabi-gcc" DYNAMIC_CC="/home/user/build/qarm/host/usr/bin/arm-unknown-linux-uclibcgnueabi-gcc -fPIC" TARGET_LD="/home/user/build/qarm/host/usr/bin/arm-unknown-linux-uclibcgnueabi-gcc" TARGET_AR="/home/user/build/qarm/host/usr/bin/arm-unknown-linux-uclibcgnueabi-ar rcus" TARGET_STRIP="/home/user/build/qarm/host/usr/bin/arm-unknown-linux-uclibcgnueabi-strip" CFLAGS=" -pipe -Os " LDFLAGS="" HOST_CC=""/usr/bin/gcc -m32""
HOST_CFLAGS="-O2 -I/home/user/build/qarm/host/include -I/home/user/build/qarm/host/usr/include" HOST_LDFLAGS="-L/home/user/build/qarm/host/lib -L/home/user/build/qarm/host/usr/lib -Wl,-rpath,/home/user/build/qarm/host/usr/lib" -C /home/user/build/qarm/build/luajit-2.0.0-beta10 amalg
/usr/bin/make: invalid option -- '3'
/usr/bin/make: invalid option -- '2'

so, I remove extra quote :
ifeq ($(BR2_ARCH_IS_64),y)
LUAJIT_HOST_CC=$(HOSTCC)
else
LUAJIT_HOST_CC=$(HOSTCC) -m32
endif

and the build command could be rewrite with CROSS :
define LUAJIT_BUILD_CMDS

	$(MAKE) PREFIX="/usr" \
		HOST_CC="$(LUAJIT_HOST_CC)" \
		CROSS="$(TARGET_CROSS)" \
		-C $(@D) amalg

endef

Fran?ois

Note : please, don't forget http://patchwork.ozlabs.org/patch/162294/

2012/7/15 Thomas Petazzoni <[email protected]>:

Le Sun, 15 Jul 2012 19:19:03 +0200,
Fran?ois Perrad <[email protected]> a ?crit :

Hum, yes, this part was a last-minute addition, which was wrong apparently. But this doesn't answer my initial question: it is not possible to build luajit for a 64 bits architecture on a 32 bits host. If this isn't solved, we can't include luajit as is.

No, using TARGET_CROSS directly doesn't work when ccache is enabled. ccache is only used in TARGET_CC and TARGET_CXX, that's why I've done this this way.

Can you look at the bitness problem?

Thanks!

Thomas

Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

2012/7/15 Thomas Petazzoni <[email protected]>:

see http://www.freelists.org/post/luajit/crosscompiling-with-different-bitness

Fran?ois.

Le Tue, 17 Jul 2012 10:07:21 +0200,
Fran?ois Perrad <[email protected]> a ?crit :

Ok, the reply is:
http://www.freelists.org/post/luajit/crosscompiling-with-different-bitness,1

Essentially the first sentence shows that Mike Pall has not seen that we were talking about cross-compilation. It's not because you're running under a x86 kernel that you cannot generate binaries for x86-64.

The second sentence shows that the cross-compilation support in luajit is not really nice.

So the only solution that I see is to integrate the proposal of Jean-Christophe Plagnol-Villard that allows to have a kconfig option defining the host architecture, and then do some depends on magic to only show the package when we have the right architecture combination. Can you look into this?

Best regards,

Thomas

Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

2012/7/17 Thomas Petazzoni <[email protected]>:

I applied this patch http://patchwork.ozlabs.org/patch/153235/

now, we could write :

    config BR2_PACKAGE_LUAJIT
	    bool "luajit"
	    depends on BR2_i386 || (BR2_x86_64 && BR2_HOSTARCH='x86_64') ||

BR2_powerpc || BR2_arm || BR2_armeb

Fran?ois