Skip to contentSkip to conversation

package/grpc: 1.31.0

PATCH New Michael Nosthoff 2 participants 7 messages Aug 13, 2020 – Aug 30, 2020

lore
Share & tools
lore
#0 lore raw

cover package/grpc: 1.31.0

grpc 1.31.0 introduces an new dependency on Google RE2 for Regular Expression parsing.

This series adds RE2 as a new package and bumps
grpc to 1.31.0 with a dependency on RE2

Michael Nosthoff (2):
package/re2: new package
package/grpc: bump version to 1.31.0

 package/Config.in                             |  1 +
 .../0001-target-build-using-host-plugin.patch |  4 +--
 ...the-availability-of-pthread_setname_.patch |  7 +++---
 package/grpc/Config.in                        |  3 ++-
 package/grpc/grpc.hash                        |  2 +-
 package/grpc/grpc.mk                          | 12 +++++----
 package/re2/Config.in                         | 12 +++++++++
 package/re2/re2.hash                          |  3 +++
 package/re2/re2.mk                            | 25 +++++++++++++++++++
 9 files changed, 56 insertions(+), 13 deletions(-)
 create mode 100644 package/re2/Config.in
 create mode 100644 package/re2/re2.hash
 create mode 100644 package/re2/re2.mk
Signature · 1 line
2.25.1
#1 lore raw

PATCH 1/2 package/re2: new package

RE2 is a fast, safe, thread-friendly alternative to backtracking regular expression engines like those used in PCRE, Perl, and Python. It is a C++ library.

---
 package/Config.in     |  1 +
 package/re2/Config.in | 12 ++++++++++++
 package/re2/re2.hash  |  3 +++
 package/re2/re2.mk    | 25 +++++++++++++++++++++++++
 4 files changed, 41 insertions(+)
 create mode 100644 package/re2/Config.in
 create mode 100644 package/re2/re2.hash
 create mode 100644 package/re2/re2.mk
package/Config.in +1 -0
index c637b5427f..f6e9524a0f 100644
@@ -1942,6 +1942,7 @@ menu "Text and terminal handling"
 	source "package/pcre/Config.in"
 	source "package/pcre2/Config.in"
 	source "package/popt/Config.in"
+	source "package/re2/Config.in"
 	source "package/readline/Config.in"
 	source "package/slang/Config.in"
 	source "package/tclap/Config.in"
package/re2/Config.in new file +12 -0
new file mode 100644
index 0000000000..d1dcb693ea
@@ -0,0 +1,12 @@
+config BR2_PACKAGE_RE2
+	bool "re2"
+	depends on BR2_INSTALL_LIBSTDCPP
+	depends on BR2_TOOLCHAIN_HAS_ATOMIC
+	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # needs C++11
+	help
+	  RE2 is a fast, safe, thread-friendly alternative
+	  to backtracking regular expression engines like
+	  those used in PCRE, Perl, and Python.
+	  It is a C++ library.
+
+	  https://github.com/google/re2
package/re2/re2.hash new file +3 -0
new file mode 100644
index 0000000000..b8a80513b7
@@ -0,0 +1,3 @@
+# locally calculated
+sha256  6f4c8514249cd65b9e85d3e6f4c35595809a63ad71c5d93083e4d1dcdf9e0cd6  re2-2020-08-01.tar.gz
+sha256  6040cda75d90b1738292a631d89934c411ef7ffd543c4d6a1b7edfc8edf29449  LICENSE
package/re2/re2.mk new file +25 -0
new file mode 100644
index 0000000000..d2791a460b
@@ -0,0 +1,25 @@
+################################################################################
+#
+# Google RE2
+#
+################################################################################
+
+RE2_VERSION = 2020-08-01
+RE2_SITE = $(call github,google,re2,$(RE2_VERSION))
+RE2_LICENSE = BSD
+RE2_LICENSE_FILES = LICENSE
+
+RE2_INSTALL_STAGING = YES
+
+RE2_CONF_OPTS += -DRE2_BUILD_TESTING=OFF
+HOST_RE2_CONF_OPTS += -DRE2_BUILD_TESTING=OFF
+
+ifeq ($(BR2_SHARED_LIBS)$(BR2_SHARED_STATIC_LIBS),y)
+RE2_CONF_OPTS += \
+	-DBUILD_SHARED_LIBS=ON
+HOST_RE2_CONF_OPTS += \
+	-DBUILD_SHARED_LIBS=ON
+endif
+
+$(eval $(cmake-package))
+$(eval $(host-cmake-package))
Signature · 1 line
2.25.1
#2 lore raw

PATCH 1/2 package/re2: new package

Quoted 5 lines · On Thu, 13 Aug 2020 16:45:48 +0200 Michael Nosthoff via buildroot…

RE2 is a fast, safe, thread-friendly alternative to backtracking regular expression engines like those used in PCRE, Perl, and Python. It is a C++ library.

Thanks, I've applied to master, with a few changes.

Quoted 6 lines
---
 package/Config.in     |  1 +
 package/re2/Config.in | 12 ++++++++++++
 package/re2/re2.hash  |  3 +++
 package/re2/re2.mk    | 25 +++++++++++++++++++++++++
 4 files changed, 41 insertions(+)

You forgot to add an entry in the DEVELOPERS file, so I've done so.

package/re2/Config.in line 4

Quoted 10 lines
package/re2/Config.in new file +4 -0
new file mode 100644
index 0000000000..d1dcb693ea
@@ -0,0 +1,12 @@
+config BR2_PACKAGE_RE2
+	bool "re2"
+	depends on BR2_INSTALL_LIBSTDCPP
+	depends on BR2_TOOLCHAIN_HAS_ATOMIC

Even though it uses std::atomic, I'm not sure it needs the atomic intrinsics of the compiler. Indeed, I tried building for SPARC, where the atomic built-ins are implemented in the libatomic.so library, and even without linking to this library, re2 builds fine.

So I've dropped this dependency. We'll see if we have any build failure, and we can adjust.

package/re2/Config.in line 12

Quoted 8 lines
package/re2/Config.in new file +8 -0
+	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # needs C++11
+	help
+	  RE2 is a fast, safe, thread-friendly alternative
+	  to backtracking regular expression engines like
+	  those used in PCRE, Perl, and Python.
+	  It is a C++ library.
+
+	  https://github.com/google/re2

You forgot a Config.in comment about the dependency (C++, gcc >= 4.8), so I've added that.

package/re2/Config.in

Quoted 3 lines
package/re2/Config.in new file +3 -0
+RE2_VERSION = 2020-08-01
+RE2_SITE = $(call github,google,re2,$(RE2_VERSION))
+RE2_LICENSE = BSD

BSD is not specific enough. In this case, it should have been BSD-3-Clause.

package/re2/Config.in

Quoted 5 lines
package/re2/Config.in new file +5 -0
+ifeq ($(BR2_SHARED_LIBS)$(BR2_SHARED_STATIC_LIBS),y)
+RE2_CONF_OPTS += \
+	-DBUILD_SHARED_LIBS=ON
+HOST_RE2_CONF_OPTS += \
+	-DBUILD_SHARED_LIBS=ON

This was not needed: BUILD_SHARED_LIBS is handled by the cmake-package infrastructure.

Thanks, applied with those changes!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#3 lore raw

PATCH 1/2 package/re2: new package

In reply to Michael Nosthoff
Quoted 5 lines · On Thu, 13 Aug 2020 16:45:48 +0200 Michael Nosthoff via buildroot…

RE2 is a fast, safe, thread-friendly alternative to backtracking regular expression engines like those used in PCRE, Perl, and Python. It is a C++ library.

Quoted 9 lines
---
 package/Config.in     |  1 +
 package/re2/Config.in | 12 ++++++++++++
 package/re2/re2.hash  |  3 +++
 package/re2/re2.mk    | 25 +++++++++++++++++++++++++
 4 files changed, 41 insertions(+)
 create mode 100644 package/re2/Config.in
 create mode 100644 package/re2/re2.hash
 create mode 100644 package/re2/re2.mk

It seems like there is an issue with this package:

http://autobuild.buildroot.net/results/a98/a98d3203f68f0f929c544537244e7621e80ce0a1/build-end.log

Apparently, only a static version of libre2 gets installed, which is built without -fPIC (makes sense for a static library). But then it gets linked into a shared library, which booms as all the code in a shared library must be compiled with -fPIC.

Perhaps the issue is that we're not explicitly passing -DBUILD_SHARED_LIBS=ON to host CMake packages in package/pkg-cmake.mk ?

Could you have a look into this ?

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#4 lore raw

PATCH 1/2 package/re2: new package

Hi Thomas,

Quoted 17 lines · Thomas Petazzoni
Quoted 11 lines · On Thu, 13 Aug 2020 16:45:48 +0200 Michael Nosthoff via buildroot…

RE2 is a fast, safe, thread-friendly alternative to backtracking regular expression engines like those used in PCRE, Perl, and Python. It is a C++ library.

---
  package/Config.in     |  1 +
  package/re2/Config.in | 12 ++++++++++++
  package/re2/re2.hash  |  3 +++
  package/re2/re2.mk    | 25 +++++++++++++++++++++++++
  4 files changed, 41 insertions(+)
Quoted 3 lines

create mode 100644 package/re2/Config.in
create mode 100644 package/re2/re2.hash
create mode 100644 package/re2/re2.mk

Quoted 3 lines

It seems like there is an issue with this package:

   http://autobuild.buildroot.net/results/a98/a98d3203f68f0f929c544537244e7621e80ce0a1/build-end.log

that's not good...

Quoted 10 lines

Apparently, only a static version of libre2 gets installed, which is built without -fPIC (makes sense for a static library). But then it gets linked into a shared library, which booms as all the code in a shared library must be compiled with -fPIC.

Perhaps the issue is that we're not explicitly passing -DBUILD_SHARED_LIBS=ON to host CMake packages in package/pkg-cmake.mk ?

Could you have a look into this ?

What I'm wondering: Should all host packages always be built as shared libs?

From the CMake Documentation [1] (which is a bit unclear) I assume that
as default

CMake will build libs as static. So it might be a coincidence that all existing host cmake packages

built as shared as their default.

When comparing to package/pkg-autotools.mk I see that for host packages

??? --enable-shared --disable-static

is always set. So it would make sense to also add

?? -DBUILD_SHARED_LIBS=ON

to cmake host build flags.

I'm currently on vacation with no access to a proper machine for building. So I would

be able to take care of this further earliest next week.

Regards,

Michael

[1] https://cmake.org/cmake/help/latest/variable/BUILD_SHARED_LIBS.html

#5 lore raw

PATCH 2/2 package/grpc: bump version to 1.31.0

In reply to Michael Nosthoff

Related changes:
- add dependency on Google RE2 package
- update patches to new offsets

Tested on Ubuntu 20.04

---
 .../grpc/0001-target-build-using-host-plugin.patch   |  4 ++--
 ...detect-the-availability-of-pthread_setname_.patch |  7 +++----
 package/grpc/Config.in                               |  3 ++-
 package/grpc/grpc.hash                               |  2 +-
 package/grpc/grpc.mk                                 | 12 +++++++-----
 5 files changed, 15 insertions(+), 13 deletions(-)
package/grpc/0001-target-build-using-host-plugin.patch +2 -2
index ac2bf8cc38..5b3c6a6fae 100644
@@ -20,7 +20,7 @@ diff --git a/CMakeLists.txt b/CMakeLists.txt
 index be695c2..45c2fcb 100644
 --- a/CMakeLists.txt
 +++ b/CMakeLists.txt
-@@ -10844,6 +10844,18 @@ target_link_libraries(grpc_cli
+@@ -11043,6 +11043,18 @@ target_link_libraries(grpc_cli
  endif()
  if(gRPC_BUILD_CODEGEN AND gRPC_BUILD_GRPC_CPP_PLUGIN)
  
@@ -39,7 +39,7 @@ index be695c2..45c2fcb 100644
  add_executable(grpc_cpp_plugin
    src/compiler/cpp_plugin.cc
  )
-@@ -10877,6 +10889,7 @@ if(gRPC_INSTALL)
+@@ -11077,6 +11089,7 @@ if(gRPC_INSTALL)
      ARCHIVE DESTINATION ${gRPC_INSTALL_LIBDIR}
    )
  endif()
package/grpc/0002-Properly-detect-the-availability-of-pthread_setname_.patch +3 -4
index 4b7253f7aa..c8b1dc8e73 100644
@@ -20,7 +20,7 @@ diff --git a/CMakeLists.txt b/CMakeLists.txt
 index 45c2fcb..0b2be4f 100644
 --- a/CMakeLists.txt
 +++ b/CMakeLists.txt
-@@ -177,6 +177,12 @@ if (NOT DEFINED CMAKE_POSITION_INDEPENDENT_CODE)
+@@ -180,6 +180,12 @@ if (NOT DEFINED CMAKE_POSITION_INDEPENDENT_CODE)
  endif()
  list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake/modules")
  
@@ -37,7 +37,7 @@ diff --git a/include/grpc/impl/codegen/port_platform.h b/include/grpc/impl/codeg
 index 4f213ff..55ecd9d 100644
 --- a/include/grpc/impl/codegen/port_platform.h
 +++ b/include/grpc/impl/codegen/port_platform.h
-@@ -195,7 +195,9 @@
+@@ -163,7 +163,9 @@
  #endif /* _LP64 */
  #ifdef __GLIBC__
  #define GPR_POSIX_CRASH_HANDLER 1
@@ -48,5 +48,4 @@ index 4f213ff..55ecd9d 100644
  #else /* musl libc */
  #define GPR_MUSL_LIBC_COMPAT 1
 -- 
-2.26.2
-
+2.26.63
package/grpc/Config.in +2 -1
index a2da3f4c53..2eb7c1de1c 100644
@@ -3,7 +3,7 @@ config BR2_PACKAGE_GRPC
 	depends on BR2_INSTALL_LIBSTDCPP
 	depends on BR2_TOOLCHAIN_HAS_THREADS # protobuf
 	depends on BR2_PACKAGE_PROTOBUF_ARCH_SUPPORTS # protobuf
-	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # protobuf
+	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # protobuf, re2
 	depends on !BR2_STATIC_LIBS # protobuf, libabseil-cpp
 	depends on BR2_TOOLCHAIN_HAS_SYNC_4 || BR2_TOOLCHAIN_HAS_ATOMIC
 	depends on BR2_PACKAGE_LIBABSEIL_CPP_ARCH_SUPPORTS
@@ -11,6 +11,7 @@ config BR2_PACKAGE_GRPC
 	select BR2_PACKAGE_LIBABSEIL_CPP
 	select BR2_PACKAGE_OPENSSL
 	select BR2_PACKAGE_PROTOBUF
+	select BR2_PACKAGE_RE2
 	select BR2_PACKAGE_ZLIB
 	help
 	  A language-neutral, platform-neutral, open source, remote
package/grpc/grpc.hash +1 -1
index f0586e4b18..20368068d9 100644
@@ -1,3 +1,3 @@
 # Locally computed
-sha256  ba74b97a2f1b4e22ec5fb69d639d849d2069fb58ea7d6579a31f800af6fe3b6c  grpc-1.30.2.tar.gz
+sha256  1236514199d3deb111a6dd7f6092f67617cd2b147f7eda7adbafccea95de7381  grpc-1.31.0.tar.gz
 sha256  cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30  LICENSE
package/grpc/grpc.mk +7 -5
index 087deac0c6..208abd90c8 100644
@@ -4,7 +4,7 @@
 #
 ################################################################################
 
-GRPC_VERSION = 1.30.2
+GRPC_VERSION = 1.31.0
 GRPC_SITE = $(call github,grpc,grpc,v$(GRPC_VERSION))
 GRPC_LICENSE = Apache-2.0
 GRPC_LICENSE_FILES = LICENSE
@@ -12,10 +12,10 @@ GRPC_LICENSE_FILES = LICENSE
 GRPC_INSTALL_STAGING = YES
 
 # Need to use host grpc_cpp_plugin during cross compilation.
-GRPC_DEPENDENCIES = c-ares host-grpc openssl protobuf zlib libabseil-cpp
-HOST_GRPC_DEPENDENCIES = host-c-ares host-openssl host-protobuf host-zlib \
-	host-libabseil-cpp
-
+GRPC_DEPENDENCIES = c-ares host-grpc openssl protobuf re2 zlib libabseil-cpp
+HOST_GRPC_DEPENDENCIES = host-c-ares host-libabseil-cpp host-openssl host-protobuf \
+	host-re2 host-zlib
+	
 # gRPC_CARES_PROVIDER=package won't work because it requires c-ares to have
 # installed a cmake config file, but buildroot uses c-ares' autotools build,
 # which doesn't do this.  These CARES settings trick the gRPC cmake code into
@@ -24,6 +24,7 @@ GRPC_CONF_OPTS = \
 	-D_gRPC_CARES_LIBRARIES=cares \
 	-DgRPC_CARES_PROVIDER=none \
 	-DgRPC_PROTOBUF_PROVIDER=package \
+	-DgRPC_RE2_PROVIDER=package \
 	-DgRPC_SSL_PROVIDER=package \
 	-DgRPC_ZLIB_PROVIDER=package \
 	-DgRPC_ABSL_PROVIDER=package \
@@ -66,6 +67,7 @@ HOST_GRPC_CONF_OPTS = \
 	-D_gRPC_CARES_LIBRARIES=cares \
 	-DgRPC_CARES_PROVIDER=none \
 	-DgRPC_PROTOBUF_PROVIDER=package \
+	-DgRPC_RE2_PROVIDER=package \
 	-DgRPC_SSL_PROVIDER=package \
 	-DgRPC_ZLIB_PROVIDER=package \
 	-DgRPC_ABSL_PROVIDER=package
Signature · 1 line
2.25.1
#6 lore raw

PATCH 2/2 package/grpc: bump version to 1.31.0

Quoted 6 lines · On Thu, 13 Aug 2020 16:45:49 +0200 Michael Nosthoff via buildroot…

Related changes:
- add dependency on Google RE2 package
- update patches to new offsets

Tested on Ubuntu 20.04

Quoted 8 lines
---
 .../grpc/0001-target-build-using-host-plugin.patch   |  4 ++--
 ...detect-the-availability-of-pthread_setname_.patch |  7 +++----
 package/grpc/Config.in                               |  3 ++-
 package/grpc/grpc.hash                               |  2 +-
 package/grpc/grpc.mk                                 | 12 +++++++-----
 5 files changed, 15 insertions(+), 13 deletions(-)

Applied to next, thanks. Note that you had forgotten to propagate the BR2_TOOLCHAIN_HAS_ATOMIC dependency of the re2 package, but since I dropped that dependency for now, that's OK.

Thanks,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com