Skip to contentSkip to conversation

RFC: Support building Java recipes with openjdk-8-native

PATCH In review Kyle Russell 2 participants 5 messages Feb 5, 2019 – Feb 7, 2019

lore
Share & tools
lore
#0 lore raw

cover RFC: Support building Java recipes with openjdk-8-native

This series spun out of a request on the mailing list last October on configuring meta-java for using openjdk-8-native to compile other Java recipes that might require modern API features not provided by the bootstrap VM classes.

http://lists.openembedded.org/pipermail/openembedded-devel/2018-October/197186.html
[oe] meta-java configuration

I've been running with this series on sumo, but I wanted to send it out to get other thoughts on the approach.

It seems like there's already precedent for this format with perlnative and pythonnative. That was more or less how I expected java-native to behave initially until I understood more about the bootstrap process.

If I've missed a valid use case of java-native and this change would cause breakage to someone else's layer, I'd be fine with introducing javanative.bbclass as a new class that provides the same functionality as the following patch to java-native.bbclass. (Its name would be more closely aligned to perlnative and pythonnative without the dash.) However, it appeared that java-native was essentially unused, and I thought having both a java-native and javanative class would be confusing.

Kyle Russell (2):
Add support for building Java recipes using java-native
ant-native: only use classpath tools if no JDK

 README                                          |  7 +++++++
 classes/java-native.bbclass                     | 17 ++++++++++-------
 .../ant-contrib/ant-contrib-native_1.0b3.bb     |  2 +-
 recipes-core/ant/ant-native_1.8.1.bb            |  4 ++--
 recipes-core/ant/files/ant                      |  9 +++++++++
 recipes-core/openjdk/openjdk-8-native.inc       |  2 ++
 6 files changed, 31 insertions(+), 10 deletions(-)
Signature · 1 line
2.20.1
#1 lore raw

PATCH 1/2 Add support for building Java recipes using java-native

Repurpose the old java-native class to setup a native JDK for building Java code in other layers' recipes. This provides a modern JDK, like openjdk-8-native, for building Java recipes where some of the bootstrap VMs are not sufficient.

Since there don't appear to be any usages of NATIVE_INSTALL_WORKS, java-native.bbclass appears to only provide an 'inherit native', which would be more straightforward to declare in the individual recipes.

Analogous to pythonnative and perlnative classes provided by oe-core.

---
 README                                          |  7 +++++++
 classes/java-native.bbclass                     | 17 ++++++++++-------
 .../ant-contrib/ant-contrib-native_1.0b3.bb     |  2 +-
 recipes-core/ant/ant-native_1.8.1.bb            |  2 +-
 recipes-core/openjdk/openjdk-8-native.inc       |  2 ++
 5 files changed, 21 insertions(+), 9 deletions(-)
README +7 -0
index 031709e..96132e8 100644
@@ -126,3 +126,10 @@ Please note that libstdc++ static is needed on your host to compile icedtea7-nat
 (install libstdc++-static on a Fedora).
 
 For debian-based distributions you need the libstdc++6-$compilerversion-dev package.
+
+To build a native JDK for using java tools like javac to build your own recipes,
+define the following variable in a distro include or local.conf and inherit java-native
+from your recipe.
+
+# Possible provider: openjdk-8-native
+PREFERRED_PROVIDER_virtual/jdk-native = "openjdk-8-native"
classes/java-native.bbclass +10 -7
index cade7b1..58d7266 100644
@@ -1,9 +1,12 @@
-# This is to be used by recipes which rely on java-library.bbclass
-# infrastructure and are a *-native recipe which needs to install
-# jar files into staging.
-# 
-# This class has nothing to do with Java's JNI.
+# Configures EXTRANATIVEPATH for recipes that need a modern,
+# native JDK to use java tools.
 
-inherit native
+inherit java
 
-NATIVE_INSTALL_WORKS = "1"
+JAVA_BUILD_VERSION ?= "${PREFERRED_PROVIDER_virtual/jdk-native}"
+
+export JAVA_HOME="${STAGING_LIBDIR_JVM_NATIVE}/${JAVA_BUILD_VERSION}"
+export CLASSPATH += "${STAGING_LIBDIR_JVM_NATIVE}/${JAVA_BUILD_VERSION}/lib/tools.jar"
+EXTRANATIVEPATH += "../${baselib}/jvm/${JAVA_BUILD_VERSION}/bin"
+
+DEPENDS += "virtual/jdk-native"
recipes-core/ant-contrib/ant-contrib-native_1.0b3.bb +1 -1
index 2a44ea2..db79626 100644
@@ -12,7 +12,7 @@ DEPENDS = "ant-native commons-cli-native commons-codec-native \
            oro-native bcel-native xerces-j-native \
           "
 
-inherit java-library java-native
+inherit java-library native
 
 SRC_URI = "${SOURCEFORGE_MIRROR}/ant-contrib/ant-contrib-${PV}-src.zip"
 
recipes-core/ant/ant-native_1.8.1.bb +1 -1
index 5a39f42..7839abd 100644
@@ -11,7 +11,7 @@ SRC_URI = "http://archive.apache.org/dist/ant/source/apache-ant-${PV}-src.tar.gz
 
 S = "${WORKDIR}/apache-ant-${PV}"
 
-inherit java-library java-native
+inherit java-library native
 
 DEPENDS = " \
 	jsch-native bsf-native xalan-j-native xerces-j-native \
recipes-core/openjdk/openjdk-8-native.inc +2 -0
index bae5d09..3909096 100644
@@ -7,6 +7,8 @@ DEPENDS = "\
     ca-certificates-native openssl-native coreutils-native \
 "
 
+PROVIDES += "virtual/jdk-native"
+
 PACKAGECONFIG ??= "gif jpeg png zlib"
 PACKAGECONFIG[x11] = "--with-x,,libx11-native xorgproto-native libxt-native libxext-native libxrender-native"
 PACKAGECONFIG[cups] = "--with-cups,,cups"
Signature · 1 line
2.20.1
#2 lore raw

PATCH 2/2 ant-native: only use classpath tools if no JDK

In reply to Kyle Russell

If ant gets invoked with a $JAVA_HOME that points to a JDK (for example, from an 'inherit java-native'), use the tools.jar from the JDK instead of the classpath-initial tools.

Once a native JDK has been bootstrapped, other Java recipes should be able to use a more featureful set of classes if needed.

---
 recipes-core/ant/ant-native_1.8.1.bb | 2 +-
 recipes-core/ant/files/ant           | 9 +++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)
recipes-core/ant/ant-native_1.8.1.bb +1 -1
index 7839abd..67cb5fc 100644
@@ -53,7 +53,7 @@ do_compile() {
   fastjar cf ${JARFILENAME} -C build .
 
   oe_makeclasspath cp -s ecj-bootstrap jsch bsf xalan2 xercesImpl resolver gnumail gnujaf bcel regexp log4j1.2 antlr oro junit jdepend commons-net commons-logging
-  cp=${STAGING_DATADIR_JAVA_NATIVE}/ant.jar:${STAGING_DATADIR}/classpath/tools.zip:$cp
+  sed -i -e"s|@ANT_JAR@|${STAGING_DATADIR_JAVA_NATIVE}/ant.jar|" ${WORKDIR}/ant
   sed -i -e"s|@JAR_FILE@|$cp|" ${WORKDIR}/ant
 }
 
recipes-core/ant/files/ant +9 -0
index bb282a9..939f3ec 100755
@@ -1,6 +1,15 @@
 #!/bin/sh
 
 export CLASSPATH
+
+CLASSPATH=$CLASSPATH:@ANT_JAR@
+
+if [ -f "$JAVA_HOME/lib/tools.jar" ] ; then
+        CLASSPATH="$CLASSPATH:$JAVA_HOME/lib/tools.jar"
+else
+        CLASSPATH="$CLASSPATH:${STAGING_DATADIR}/classpath/tools.zip"
+fi
+
 CLASSPATH=$CLASSPATH:@JAR_FILE@
 
 if [ ! $JAVA ];then
Signature · 1 line
2.20.1
#3 lore raw
In reply to Kyle Russell

Hi,

First, I agree the current state isn't great at all.

It does appear to me though that Java world seems to be a bit different, in that even today there are big Java projects out there that still only compile using Java 8, which others do compile fine using more recent versions.

With your suggested approach it becomes impossible to properly support that, you still have to pick the lowest denominator globally.

Other distributions, Debian based ones in particular are taking a different approach:
Each Java compiler/runtime provides virtual properties, and recipes can depend on the required version easily, something like e.g. virtual/java8-sdk virtual/java9-sdk virtual/java10-sdk etc.

I'had posted a patch series trying to mimic that before, but not really had time to work on that any more:
http://lists.openembedded.org/pipermail/openembedded-devel/2018-July/195641.html
http://lists.openembedded.org/pipermail/openembedded-devel/2018-July/195642.html
http://lists.openembedded.org/pipermail/openembedded-devel/2018-July/195643.html
http://lists.openembedded.org/pipermail/openembedded-devel/2018-July/195644.html
http://lists.openembedded.org/pipermail/openembedded-devel/2018-July/195645.html
http://lists.openembedded.org/pipermail/openembedded-devel/2018-July/195646.html
http://lists.openembedded.org/pipermail/openembedded-devel/2018-July/195647.html

What do you think?

Cheers,
Andre'

Quoted 39 lines · Kyle Russell

This series spun out of a request on the mailing list last October on configuring meta-java for using openjdk-8-native to compile other Java recipes that might require modern API features not provided by the bootstrap VM classes.

http://lists.openembedded.org/pipermail/openembedded-devel/2018-October/197186.html
[oe] meta-java configuration

I've been running with this series on sumo, but I wanted to send it out to get other thoughts on the approach.

It seems like there's already precedent for this format with perlnative and pythonnative. That was more or less how I expected java-native to behave initially until I understood more about the bootstrap process.

If I've missed a valid use case of java-native and this change would cause breakage to someone else's layer, I'd be fine with introducing javanative.bbclass as a new class that provides the same functionality as the following patch to java-native.bbclass. (Its name would be more closely aligned to perlnative and pythonnative without the dash.) However, it appeared that java-native was essentially unused, and I thought having both a java-native and javanative class would be confusing.

Kyle Russell (2):
Add support for building Java recipes using java-native
ant-native: only use classpath tools if no JDK

 README                                          |  7 +++++++
 classes/java-native.bbclass                     | 17 ++++++++++-------
 .../ant-contrib/ant-contrib-native_1.0b3.bb     |  2 +-
 recipes-core/ant/ant-native_1.8.1.bb            |  4 ++--
 recipes-core/ant/files/ant                      |  9 +++++++++
 recipes-core/openjdk/openjdk-8-native.inc       |  2 ++
 6 files changed, 31 insertions(+), 10 deletions(-)
Signature · 1 line
2.20.1
#4 lore raw

It seems like a question of how many JDK versions meta-java plans on providing, and for how long. To me the number of different per-version providers feels like a maintenance point that could quickly get out of hand. I guess my assumption has been that all of the embedded Java code for the distribution would compile with the same JDK (either the latest JDK provided by meta-java, or perhaps one behind the latest version; similar to how oe-core handles gcc upgrades, for example), and that compile would pass in the appropriate -target or -bootclasspath parameters for the preferred JVM.

I hope I understand what you're saying, because I'd like to work together toward a solution.

Does your distribution require specifying different providers per runtime version?

Quoted 84 lines · André Draszik

Hi,

First, I agree the current state isn't great at all.

It does appear to me though that Java world seems to be a bit different, in that even today there are big Java projects out there that still only compile using Java 8, which others do compile fine using more recent versions.

With your suggested approach it becomes impossible to properly support that,
you still have to pick the lowest denominator globally.

Other distributions, Debian based ones in particular are taking a different approach:
Each Java compiler/runtime provides virtual properties, and recipes can depend on the required version easily, something like e.g. virtual/java8-sdk virtual/java9-sdk virtual/java10-sdk etc.

I'had posted a patch series trying to mimic that before, but not really had time to work on that any more:

http://lists.openembedded.org/pipermail/openembedded-devel/2018-July/195641.html

http://lists.openembedded.org/pipermail/openembedded-devel/2018-July/195642.html

http://lists.openembedded.org/pipermail/openembedded-devel/2018-July/195643.html

http://lists.openembedded.org/pipermail/openembedded-devel/2018-July/195644.html

http://lists.openembedded.org/pipermail/openembedded-devel/2018-July/195645.html

http://lists.openembedded.org/pipermail/openembedded-devel/2018-July/195646.html

http://lists.openembedded.org/pipermail/openembedded-devel/2018-July/195647.html

What do you think?

Cheers,
Andre'

Quoted 6 lines · Kyle Russell

This series spun out of a request on the mailing list last October on configuring meta-java for using openjdk-8-native to compile other Java recipes that might require modern API features not provided by the bootstrap VM classes.

http://lists.openembedded.org/pipermail/openembedded-devel/2018-October/197186.html

Quoted 33 lines

[oe] meta-java configuration

I've been running with this series on sumo, but I wanted to send it out to get other thoughts on the approach.

It seems like there's already precedent for this format with perlnative and pythonnative. That was more or less how I expected java-native to behave initially until I understood more about the bootstrap process.

If I've missed a valid use case of java-native and this change would cause breakage to someone else's layer, I'd be fine with introducing javanative.bbclass as a new class that provides the same functionality as the following patch to java-native.bbclass. (Its name would be more closely aligned to perlnative and pythonnative without the dash.) However, it appeared that java-native was essentially unused, and I thought having both a java-native and javanative class would be confusing.

Kyle Russell (2):
Add support for building Java recipes using java-native
ant-native: only use classpath tools if no JDK

 README                                          |  7 +++++++
 classes/java-native.bbclass                     | 17 ++++++++++-------
 .../ant-contrib/ant-contrib-native_1.0b3.bb     |  2 +-
 recipes-core/ant/ant-native_1.8.1.bb            |  4 ++--
 recipes-core/ant/files/ant                      |  9 +++++++++
 recipes-core/openjdk/openjdk-8-native.inc       |  2 ++
 6 files changed, 31 insertions(+), 10 deletions(-)

--
2.20.1