Skip to contentSkip to conversation

add support for gRPC C and C++ bindings

PATCH New charles.hardin at storagecraft.com 2 participants 16 messages May 21, 2018 – Jun 28, 2018

lore
Share & tools
lore

charles.hardin at storagecraft.com

author
#0 lore raw

v2 cover add support for gRPC C and C++ bindings

From: Charles Hardin <[email protected]>

This is just a rough in patch to get gRPC in some fashion into the build. It uses collectd as an integration test since that is already available and doesn't require a custome application to be developed.

Ignore the "first" patch - it is in next, but it is needed for this series to compile. The patches are suppose to be against "origin/master" according to the docs so it is included here.

Change v1 -> v2
Add the git-format-patch blurbs from github pull requests Add the static/shared lib config to gRPC
Restack on the "next" patch for protobuf 3.5.0+ Update Config.in in grpc and collectd from Thomas

Charles Hardin (4):
package/protobuf: add a patch for mips big endian package/c-ares: enable the host variant for a c-ares install
grpc: new package
package/collectd: allow the grpc plugin to be configured

Martijn de Gouw (1):
package/protobuf: bump to version 3.5.1

 package/Config.in                                  |   1 +
 package/c-ares/c-ares.mk                           |   1 +
 package/collectd/Config.in                         |  11 +++
 package/collectd/collectd.mk                       |   4 +-
 ...onmental-overrides-of-protoc-dtrace-and-l.patch |  72 +++++++++++++++
 package/grpc/Config.in                             |  19 ++++
 package/grpc/grpc.hash                             |   2 +
 package/grpc/grpc.mk                               | 101 +++++++++++++++++++++
 ...w-a-compilation-under-mips-big-endian-wit.patch |  38 ++++++++
 package/protobuf/protobuf.hash                     |   2 +-
 package/protobuf/protobuf.mk                       |   2 +-
 package/python-protobuf/python-protobuf.hash       |   2 +-
 12 files changed, 251 insertions(+), 4 deletions(-)
 create mode 100644 package/grpc/0001-Allow-environmental-overrides-of-protoc-dtrace-and-l.patch
 create mode 100644 package/grpc/Config.in
 create mode 100644 package/grpc/grpc.hash
 create mode 100644 package/grpc/grpc.mk
 create mode 100644 package/protobuf/0001-Fix-to-allow-a-compilation-under-mips-big-endian-wit.patch
Signature · 1 line
2.15.1 (Apple Git-101)
author
#1 lore raw

PATCH v2 1/5 package/protobuf: bump to version 3.5.1

From: Martijn de Gouw <[email protected]>

GRPC requires protobuf 3.5.0+

Signed-off-by Martijn de Gouw <[email protected]>
Signed-off-by Thomas Petazzoni <[email protected]>
(cherry picked from commit 454aa2da5e79ec8627f1d66ac640ec7bb04025dc)
---
 package/protobuf/protobuf.hash               | 2 +-
 package/protobuf/protobuf.mk                 | 2 +-
 package/python-protobuf/python-protobuf.hash | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
package/protobuf/protobuf.hash +1 -1
index fa02440a40..4492487efb 100644
@@ -1,2 +1,2 @@
 # Locally calculated
-sha256 2bb34b4a8211a30d12ef29fd8660995023d119c99fbab2e5fe46f17528c9cc78 protobuf-cpp-3.4.1.tar.gz
+sha256 c28dba8782da2cfea1e11c61d335958c31a9c1bc553063546af9cbe98f204092 protobuf-cpp-3.5.1.tar.gz
package/protobuf/protobuf.mk +1 -1
index d2782ddcfd..1440589c14 100644
@@ -7,7 +7,7 @@
 # When bumping this package, make sure to also verify if the
 # python-protobuf package still works, as they share the same
 # version/site variables.
-PROTOBUF_VERSION = 3.4.1
+PROTOBUF_VERSION = 3.5.1
 PROTOBUF_SOURCE = protobuf-cpp-$(PROTOBUF_VERSION).tar.gz
 PROTOBUF_SITE = https://github.com/google/protobuf/releases/download/v$(PROTOBUF_VERSION)
 PROTOBUF_LICENSE = BSD-3-Clause
package/python-protobuf/python-protobuf.hash +1 -1
index 91cab4ecc0..9c7606ce34 100644
@@ -1,2 +1,2 @@
 # Locally calculated
-sha256 1faa722cf475c8e4c43ddb393d6f1477f1a56c93be38a1c8e367c358db476b5f protobuf-python-3.4.1.tar.gz
+sha256 13d3c15ebfad8c28bee203dd4a0f6e600d2a7d2243bac8b5d0e517466500fcae protobuf-python-3.5.1.tar.gz
Signature · 1 line
2.15.1 (Apple Git-101)
#2 lore raw

PATCH v2 1/5 package/protobuf: bump to version 3.5.1

Hello Charles,

Quoted 7 lines · charles.hardin at storagecraft.com

From: Martijn de Gouw <[email protected]>

GRPC requires protobuf 3.5.0+

Signed-off-by Martijn de Gouw <[email protected]>
Signed-off-by Thomas Petazzoni <[email protected]>
(cherry picked from commit 454aa2da5e79ec8627f1d66ac640ec7bb04025dc)

Why are you sending this patch? It's already in the next branch. Since what you're sending is intended for the next branch, you should simply base your patches on the next branch, and send only what's not yet in next.

I'll mark this patch as Not Applicable in our patch tracking system.

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
author
#3 lore raw

PATCH v2 1/5 package/protobuf: bump to version 3.5.1

Thomas,

On May 22, 2018, at 3:11 AM, Thomas Petazzoni <thomas.petazzoni at bootlin.com<mailto:[email protected]>> wrote:

Hello Charles,

On Mon, 21 May 2018 10:53:06 -0700, charles.hardin at storagecraft.com<mailto:[email protected]> wrote:
From: Martijn de Gouw <martijn.de.gouw at prodrive-technologies.com<mailto:[email protected]>>

GRPC requires protobuf 3.5.0+

Signed-off-by Martijn de Gouw <martijn.de.gouw at prodrive-technologies.com<mailto:[email protected]>>
Signed-off-by Thomas Petazzoni <thomas.petazzoni at bootlin.com<mailto:[email protected]>>
(cherry picked from commit 454aa2da5e79ec8627f1d66ac640ec7bb04025dc)

Why are you sending this patch? It's already in the next branch. Since what you're sending is intended for the next branch, you should simply base your patches on the next branch, and send only what's not yet in next.

The docs say to rebase against origin/master before submitting a patch not origin/next. So, it is unclear if we are suppose to submit a patch series that compiles as a set or a patch series that relies on something already in next.

I erred on a duplicated patch at the beginning to try and make sure it was understood this patch series requires the bump - else, it won?t compile.

If there is a better a way to do this, then it wasn?t clear from contributing guide.

https://buildroot.org/downloads/manual/manual.html#submitting-patches

I'll mark this patch as Not Applicable in our patch tracking system.

Best regards,

Thomas
--
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons) Embedded Linux and Kernel engineering
https://bootlin.com

Charles

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20180522/af1ee5fb/attachment.html>

#4 lore raw

PATCH v2 1/5 package/protobuf: bump to version 3.5.1

Hello,

Would it be possible to teach your e-mail client to quote properly in the plain text version of the e-mail ? And ideally to not send HTML e-mails ?

Quoted 10 lines · Charles Hardin

The docs say to rebase against origin/master before submitting a patch not origin/next. So, it is unclear if we are suppose to submit a patch series that compiles as a set or a patch series that relies on something already in next.

I erred on a duplicated patch at the beginning to try and make sure it was understood this patch series requires the bump - else, it won?t compile.

If there is a better a way to do this, then it wasn?t clear from contributing guide.

https://buildroot.org/downloads/manual/manual.html#submitting-patches

As often, documentation is a hint, not always an absolute truth.

In the general case patches should be based on the master branch, indeed.

However, between the publication of rc1 and the final version of a given release, the master branch is only used for bug fixes, security fixes and build fixes. We do not merge version bumps and new packages in master during this period of time, which is used to stabilize the master branch.

However, since we don't want to completely stop merging patches adding new packages and updating existing packages, we open a separate "next" branch during this period of time. So, if your contribution is submitted during this period of time, and provides new packages, package updates or anything that doesn't qualify as a "fix", it should be based on the next branch.

The calendar for the year is as follows:

- January: everything goes in master

- February:

* Beginning of the month: YYYY.02-rc1 is released

* Only fixes go in master, everything else goes in next

* End of the month: YYYY.02 is released, with the contents of the master branch

- March:

* The YYYY.05 development cycle opens. Everything in next is merged back into master.

* All patches contributed are merged in master.

- April: everything goes in master.

- May:

* Beginning of the month: YYYY.05-rc1 is released

* Only fixes go in master, everything else goes in next

* End of the month: YYYY.05 is released, with the contents of the master branch

- June:

* The YYYY.08 development cycle opens. Everything in next is merged back into master.

* All patches contributed are merged in master.

- July: everything goes in master

- August:

* Beginning of the month: YYYY.08-rc1 is released

* Only fixes go in master, everything else goes in next

* End of the month: YYYY.08 is released, with the contents of the master branch

- September:

* The YYYY.11 development cycle opens. Everything in next is merged back into master.

* All patches contributed are merged in master.

- October: everything goes in master

- November:

* Beginning of the month: YYYY.11-rc1 is released

* Only fixes go in master, everything else goes in next

* End of the month: YYYY.11 is released, with the contents of the master branch

- December:

* The (YYYY+1).02 development cycle opens. Everything in next is merged back into master.

* All patches contributed are merged in master.

Does that clarify our development process ?

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
author
#5 lore raw

PATCH v2 2/5 package/protobuf: add a patch for mips big endian

In reply to charles.hardin at storagecraft.com

From: Charles Hardin <[email protected]>

This patch got submitted as a pull upstream, but hasn't been approved or yanked in yet. So, adding it here for now until an upstream solution is determined.

Signed-off-by Charles Hardin <[email protected]>
---
 ...w-a-compilation-under-mips-big-endian-wit.patch | 38 ++++++++++++++++++++++
 1 file changed, 38 insertions(+)
 create mode 100644 package/protobuf/0001-Fix-to-allow-a-compilation-under-mips-big-endian-wit.patch
package/protobuf/0001-Fix-to-allow-a-compilation-under-mips-big-endian-wit.patch new file +38 -0
new file mode 100644
index 0000000000..b98a459be8
@@ -0,0 +1,38 @@
+From d105213bccfa4392eda889a95f808dd9cdf84494 Mon Sep 17 00:00:00 2001
+From: Charles Hardin <[email protected]>
+Date: Mon, 21 May 2018 09:19:30 -0700
+Subject: [PATCH 1/1] Fix to allow a compilation under mips big endian with gcc
+
+The __MIPSEL__ is mips little endian, specific and we needed
+to compile for a big endian mips target that also happened
+to be a mips64, so replaced the __MIPSEL__ with a __mip__
+which should preserve the build on little endian and allow a
+build for a big endian setup.
+
+	$ echo | mips64-linux-gcc -E -dM -
+	... snip snip ...
+	#define __mips__ 1
+	#define __MIPSEB__ 1
+
+Signed-off-by: Charles Hardin <[email protected]>
+Signed-off-by: Charles Hardin <[email protected]>
+---
+ src/google/protobuf/stubs/platform_macros.h | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/src/google/protobuf/stubs/platform_macros.h b/src/google/protobuf/stubs/platform_macros.h
+index c3a64dd2..f9e2bff6 100644
+--- a/src/google/protobuf/stubs/platform_macros.h
++++ b/src/google/protobuf/stubs/platform_macros.h
+@@ -56,7 +56,7 @@
+ #elif defined(__aarch64__)
+ #define GOOGLE_PROTOBUF_ARCH_AARCH64 1
+ #define GOOGLE_PROTOBUF_ARCH_64_BIT 1
+-#elif defined(__MIPSEL__)
++#elif defined(__mips__)
+ #if defined(__LP64__)
+ #define GOOGLE_PROTOBUF_ARCH_MIPS64 1
+ #define GOOGLE_PROTOBUF_ARCH_64_BIT 1
+-- 
+2.15.1 (Apple Git-101)
+
Signature · 1 line
2.15.1 (Apple Git-101)
#6 lore raw

PATCH v2 2/5 package/protobuf: add a patch for mips big endian

Hello,

Quoted 11 lines · charles.hardin at storagecraft.com

From: Charles Hardin <[email protected]>

This patch got submitted as a pull upstream, but hasn't been approved or yanked in yet. So, adding it here for now until an upstream solution is determined.

Signed-off-by Charles Hardin <[email protected]>
---
 ...w-a-compilation-under-mips-big-endian-wit.patch | 38 ++++++++++++++++++++++
 1 file changed, 38 insertions(+)
 create mode 100644 package/protobuf/0001-Fix-to-allow-a-compilation-under-mips-big-endian-wit.patch

I've applied to master. However your pull request to the upstream protobuf project has been closed, because you did not sign the CLA:
https://github.com/google/protobuf/pull/4657. Could you follow that process to make sure your patch gets merged upstream ?

Thanks,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
author
#7 lore raw

PATCH v2 2/5 package/protobuf: add a patch for mips big endian

Yeah...

Quoted 18 lines · Thomas Petazzoni

Hello,

Quoted 11 lines · charles.hardin at storagecraft.com

From: Charles Hardin <[email protected]>

This patch got submitted as a pull upstream, but hasn't been approved or yanked in yet. So, adding it here for now until an upstream solution is determined.

Signed-off-by Charles Hardin <[email protected]>

---
...w-a-compilation-under-mips-big-endian-wit.patch | 38 ++++++++++++++++++++++

1 file changed, 38 insertions(+)

create mode 100644 package/protobuf/0001-Fix-to-allow-a-compilation-under-mips-big-endian-wit.patch

Quoted 5 lines

I've applied to master. However your pull request to the upstream protobuf project has been closed, because you did not sign the CLA:
https://github.com/google/protobuf/pull/4657. Could you follow that process to make sure your patch gets merged upstream ?

Looks like email confusion - reposted and it should be able to go thru.

It is unclear if there is any intention on protobuf to support mips big endian anyway.

Quoted 8 lines

Thanks,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
#8 lore raw

PATCH v2 2/5 package/protobuf: add a patch for mips big endian

Hello,

Quoted 6 lines · Charles Hardin
Quoted 4 lines

I've applied to master. However your pull request to the upstream protobuf project has been closed, because you did not sign the CLA:
https://github.com/google/protobuf/pull/4657. Could you follow that process to make sure your patch gets merged upstream ?

Looks like email confusion - reposted and it should be able to go thru.

I don't see any e-mail confusion. There is a clear comment in the pull request indicating that you need to sign the CLA.

Or perhaps you mean that you have already signed this CLA, and just because your patch had the wrong author e-mail, the Github bot didn't recognize you as a person who has signed the CLA ?

Quoted 2 lines

It is unclear if there is any intention on protobuf to support mips big endian anyway.

Why wouldn't they ? It's just a trivial one line patch.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
author
#9 lore raw

PATCH v2 3/5 package/c-ares: enable the host variant for a c-ares install

In reply to charles.hardin at storagecraft.com

From: Charles Hardin <[email protected]>

gRPC needs a c-ares library for the plugins to compile against, and it isn't something that can be easily patched out. So, allow c-ares to be compiled as a host tool.

Signed-off-by Charles Hardin <[email protected]>
---
 package/c-ares/c-ares.mk | 1 +
 1 file changed, 1 insertion(+)
package/c-ares/c-ares.mk +1 -0
index 67254244a1..76e9687b37 100644
@@ -15,3 +15,4 @@ C_ARES_LICENSE = MIT
 C_ARES_LICENSE_FILES = ares_mkquery.c
 
 $(eval $(autotools-package))
+$(eval $(host-autotools-package))
Signature · 1 line
2.15.1 (Apple Git-101)
author
#10 lore raw

PATCH v2 4/5 grpc: new package

In reply to charles.hardin at storagecraft.com

From: Charles Hardin <[email protected]>

add the gRPC package from Google's github repo. This is currently just installing the C and C++ libraries on the target installation for now. Some effort could be made to add the python bindings as a subsequent patch.

This also adds a patch to specify the protoc to use since that is provided by buildroot and not the host in addition to overriding the ldconfig command since that doesn't apply to the buildroot compile.

NOTE: This doesn't add the python or any other bindings

Signed-off-by Charles Hardin <[email protected]>
---
 package/Config.in                                  |   1 +
 ...onmental-overrides-of-protoc-dtrace-and-l.patch |  72 +++++++++++++++
 package/grpc/Config.in                             |  19 ++++
 package/grpc/grpc.hash                             |   2 +
 package/grpc/grpc.mk                               | 101 +++++++++++++++++++++
 5 files changed, 195 insertions(+)
 create mode 100644 package/grpc/0001-Allow-environmental-overrides-of-protoc-dtrace-and-l.patch
 create mode 100644 package/grpc/Config.in
 create mode 100644 package/grpc/grpc.hash
 create mode 100644 package/grpc/grpc.mk
package/Config.in +1 -0
index ecee4938c9..f4b9a7d7cd 100644
@@ -1506,6 +1506,7 @@ menu "Other"
 	source "package/glibmm/Config.in"
 	source "package/glm/Config.in"
 	source "package/gmp/Config.in"
+	source "package/grpc/Config.in"
 	source "package/gsl/Config.in"
 	source "package/gtest/Config.in"
 	source "package/jemalloc/Config.in"
package/grpc/0001-Allow-environmental-overrides-of-protoc-dtrace-and-l.patch new file +72 -0

72 lines not on this page · raw message

package/grpc/Config.in new file +19 -0
new file mode 100644
index 0000000000..d7096d0853
@@ -0,0 +1,19 @@
+config BR2_PACKAGE_GRPC
+	bool "grpc"
+	depends on BR2_INSTALL_LIBSTDCPP
+	depends on BR2_TOOLCHAIN_HAS_THREADS
+	select BR2_PACKAGE_C_ARES
+	select BR2_PACKAGE_GFLAGS
+	select BR2_PACKAGE_GTEST
+	select BR2_PACKAGE_GTEST_GMOCK
+	select BR2_PACKAGE_OPENSSL
+	select BR2_PACKAGE_PROTOBUF
+	select BR2_PACKAGE_ZLIB
+	help
+	  gRPC is Google's protocol buffer based implementation of a
+	  remote procedure call protocol.
+
+	  http://www.grpc.io
+
+comment "grpc needs a toolchain w/ C++, threads"
+	depends on !(BR2_INSTALL_LIBSTDCPP && BR2_TOOLCHAIN_HAS_THREADS)
package/grpc/grpc.hash new file +2 -0
new file mode 100644
index 0000000000..9f8e6869a5
@@ -0,0 +1,2 @@
+# Locally computed
+sha256 0068a0b11795ac89ba8f5cc7332cd8cd2870a2ee0016126f014ce0bc3004e590  grpc-v1.11.1.tar.gz
package/grpc/grpc.mk new file +101 -0

101 lines not on this page · raw message

Signature · 1 line
2.15.1 (Apple Git-101)
#11 lore raw

PATCH v2 4/5 grpc: new package

Hello Charles,

Quoted 7 lines · charles.hardin at storagecraft.com

From: Charles Hardin <[email protected]>

add the gRPC package from Google's github repo. This is currently just installing the C and C++ libraries on the target installation for now. Some effort could be made to add the python bindings as a subsequent patch.

Quoted 8 lines

This also adds a patch to specify the protoc to use since that is provided by buildroot and not the host in addition to overriding the ldconfig command since that doesn't apply to the buildroot compile.

NOTE: This doesn't add the python or any other bindings

Signed-off-by Charles Hardin <[email protected]>

I tried this here, and unfortunately it doesn't build.

First, the CFLAGS/CXXFLAGS/LDFLAGS passed as make options were completely overriding the ones from GRPC makefile, causing a build failure. I fixed that with:

package/grpc/grpc.mk +5 -7
index 738cd7f851..da5def4ceb 100644
@@ -15,7 +15,8 @@ HOST_GRPC_DEPENDENCIES = host-c-ares host-protobuf host-openssl
 
 GRPC_INSTALL_STAGING = YES
 
-GRPC_CROSS_MAKE_OPTS_BASE = \
+GRPC_MAKE_ENV = \
+       $(TARGET_MAKE_ENV) \
        GRPC_CROSS_COMPILE="true" \
        LDCONFIG=/bin/true \
        HOST_CC="$(HOSTCC)" \

@@ -34,15 +35,12 @@ GRPC_CROSS_MAKE_OPTS_BASE = \
        STRIP=/bin/true
 
 GRPC_MAKE_OPTS = \
-       $(GRPC_CROSS_MAKE_OPTS_BASE) \
        PROTOC="$(HOST_DIR)/usr/bin/protoc"
 
 GRPC_INSTALL_TARGET_OPTS = \
-       $(GRPC_CROSS_MAKE_OPTS_BASE) \
        prefix="$(TARGET_DIR)/usr"
 
 GRPC_INSTALL_STAGING_OPTS = \
-       $(GRPC_CROSS_MAKE_OPTS_BASE) \
        prefix="$(STAGING_DIR)/usr"
 
 ifeq ($(BR2_SHARED_LIBS),y)

@@ -60,17 +58,17 @@ GRPC_INSTALL_TARGETS = install-shared_c install-shared_cxx
 endif
 
 define GRPC_BUILD_CMDS
-       $(TARGET_MAKE_ENV) $(MAKE) $(GRPC_MAKE_OPTS) -C $(@D) \
+       $(GRPC_MAKE_ENV) $(MAKE) $(GRPC_MAKE_OPTS) -C $(@D) \
                $(GRPC_BUILD_TARGETS)
 endef
 
 define GRPC_INSTALL_STAGING_CMDS
-       $(TARGET_MAKE_ENV) $(MAKE) $(GRPC_INSTALL_STAGING_OPTS) -C $(@D) \
+       $(GRPC_MAKE_ENV) $(MAKE) $(GRPC_INSTALL_STAGING_OPTS) -C $(@D) \
                $(GRPC_STAGING_TARGETS)
 endef
 
 define GRPC_INSTALL_TARGET_CMDS
-       $(TARGET_MAKE_ENV) $(MAKE) $(GRPC_INSTALL_TARGET_OPTS) -C $(@D) \
+       $(GRPC_MAKE_ENV) $(MAKE) $(GRPC_INSTALL_TARGET_OPTS) -C $(@D) \
                $(GRPC_INSTALL_TARGETS)
 endef

With that in place, the GRPC build moved on, but failed at a later point, with:

[HOSTCXX] Compiling src/compiler/cpp_generator.cc
[HOSTCXX] Compiling src/compiler/csharp_generator.cc [HOSTCXX] Compiling src/compiler/node_generator.cc
In file included from /usr/include/c++/7/ext/string_conversions.h:41:0,

                 from /usr/include/c++/7/bits/basic_string.h:6349,
                 from /usr/include/c++/7/string:52,
                 from /usr/include/c++/7/stdexcept:39,
                 from /usr/include/c++/7/array:39,
                 from /usr/include/c++/7/tuple:39,
                 from /usr/include/c++/7/bits/stl_map.h:63,
                 from /usr/include/c++/7/map:61,
                 from src/compiler/cpp_generator.cc:19:
/usr/include/c++/7/cstdlib:131:11: error: ?::at_quick_exit? has not been declared
   using ::at_quick_exit;
           ^~~~~~~~~~~~~
/usr/include/c++/7/cstdlib:154:11: error: ?::quick_exit? has not been declared
   using ::quick_exit;
           ^~~~~~~~~~
[HOSTCXX] Compiling src/compiler/objective_c_generator.cc
In file included from /usr/include/c++/7/ext/string_conversions.h:41:0,
                 from /usr/include/c++/7/bits/basic_string.h:6349,
                 from /usr/include/c++/7/string:52,
                 from /usr/include/c++/7/stdexcept:39,
                 from /usr/include/c++/7/array:39,
                 from /usr/include/c++/7/tuple:39,
                 from /usr/include/c++/7/bits/stl_map.h:63,
                 from /usr/include/c++/7/map:61,
                 from src/compiler/csharp_generator.cc:20:
/usr/include/c++/7/cstdlib:131:11: error: ?::at_quick_exit? has not been declared
   using ::at_quick_exit;
           ^~~~~~~~~~~~~
/usr/include/c++/7/cstdlib:154:11: error: ?::quick_exit? has not been declared
   using ::quick_exit;
           ^~~~~~~~~~

In file included from ./src/compiler/cpp_generator.h:26:0,
from src/compiler/cpp_generator.cc:21:
/usr/include/c++/7/memory: In function ?void* std::align(std::size_t, std::size_t, void*&, std::size_t&)?:
/usr/include/c++/7/memory:116:58: error: cast from ?void*? to ?uintptr_t {aka unsigned int}? loses precision [-fpermissive]
const auto __intptr = reinterpret_cast<uintptr_t>(__ptr);

And more of those issues. Here is the defconfig I have used:

BR2_arm=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-full-2018.02-891-g046c5e2.tar.bz2" BR2_TOOLCHAIN_EXTERNAL_GCC_4_9=y
BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_1=y
BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
BR2_TOOLCHAIN_EXTERNAL_CXX=y
BR2_INIT_NONE=y
BR2_SYSTEM_BIN_SH_NONE=y
# BR2_PACKAGE_BUSYBOX is not set
BR2_PACKAGE_GRPC=y
# BR2_TARGET_ROOTFS_TAR is not set

Could you have a look ?

I'll also do a regular review of the package in a separate e-mail, so that you can take into account other comments at the same time.

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
author
#12 lore raw

PATCH v2 4/5 grpc: new package

Hmm?.

Quoted 18 lines · Thomas Petazzoni

Hello Charles,

Quoted 12 lines · charles.hardin at storagecraft.com

From: Charles Hardin <[email protected]>

add the gRPC package from Google's github repo. This is currently just installing the C and C++ libraries on the target installation for now. Some effort could be made to add the python bindings as a subsequent patch.

This also adds a patch to specify the protoc to use since that is provided by buildroot and not the host in addition to overriding the ldconfig command since that doesn't apply to the buildroot compile.

Quoted 6 lines
Quoted 3 lines

NOTE: This doesn't add the python or any other bindings

Signed-off-by Charles Hardin <[email protected]>

I tried this here, and unfortunately it doesn't build.

Ok - never tried ?arm? builds - only mips and x86, so this is likely going to be a bit more involved. But, I think this is a HOSTCXX problem - what is that version because I barely recall that I have to build the gRPC proto gen tools on the host to generate the stubs for the cross.

It?s entirely possible I just assumed the host toolchain will work - this is what we use on the host.

Quoted 6 lines

vagrant at dev:~/buildroot$ cc --version
cc (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Quoted 6 lines

vagrant at dev:~/buildroot$ c++ --version
c++ (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

So, it looks like it is a gcc 7 build on this host side?

Quoted 127 lines

First, the CFLAGS/CXXFLAGS/LDFLAGS passed as make options were completely overriding the ones from GRPC makefile, causing a build failure. I fixed that with:

package/grpc/grpc.mk +0 -0
index 738cd7f851..da5def4ceb 100644
@@ -15,7 +15,8 @@ HOST_GRPC_DEPENDENCIES = host-c-ares host-protobuf host-openssl

GRPC_INSTALL_STAGING = YES

package/grpc/grpc.mk +2 -4
-GRPC_CROSS_MAKE_OPTS_BASE = \
+GRPC_MAKE_ENV = \
+       $(TARGET_MAKE_ENV) \
       GRPC_CROSS_COMPILE="true" \
       LDCONFIG=/bin/true \
       HOST_CC="$(HOSTCC)" \

@@ -34,15 +35,12 @@ GRPC_CROSS_MAKE_OPTS_BASE = \
       STRIP=/bin/true
 
 GRPC_MAKE_OPTS = \
-       $(GRPC_CROSS_MAKE_OPTS_BASE) \
       PROTOC="$(HOST_DIR)/usr/bin/protoc"
 
 GRPC_INSTALL_TARGET_OPTS = \
-       $(GRPC_CROSS_MAKE_OPTS_BASE) \
       prefix="$(TARGET_DIR)/usr"
 
 GRPC_INSTALL_STAGING_OPTS = \
-       $(GRPC_CROSS_MAKE_OPTS_BASE) \
       prefix="$(STAGING_DIR)/usr"
 
 ifeq ($(BR2_SHARED_LIBS),y)

@@ -60,17 +58,17 @@ GRPC_INSTALL_TARGETS = install-shared_c install-shared_cxx

endif

define GRPC_BUILD_CMDS

package/grpc/grpc.mk +1 -1
-       $(TARGET_MAKE_ENV) $(MAKE) $(GRPC_MAKE_OPTS) -C $(@D) \
+       $(GRPC_MAKE_ENV) $(MAKE) $(GRPC_MAKE_OPTS) -C $(@D) \
               $(GRPC_BUILD_TARGETS)

endef

define GRPC_INSTALL_STAGING_CMDS

package/grpc/grpc.mk +1 -1
-       $(TARGET_MAKE_ENV) $(MAKE) $(GRPC_INSTALL_STAGING_OPTS) -C $(@D) \
+       $(GRPC_MAKE_ENV) $(MAKE) $(GRPC_INSTALL_STAGING_OPTS) -C $(@D) \
               $(GRPC_STAGING_TARGETS)

endef

define GRPC_INSTALL_TARGET_CMDS

package/grpc/grpc.mk +1 -1
-       $(TARGET_MAKE_ENV) $(MAKE) $(GRPC_INSTALL_TARGET_OPTS) -C $(@D) \
+       $(GRPC_MAKE_ENV) $(MAKE) $(GRPC_INSTALL_TARGET_OPTS) -C $(@D) \
               $(GRPC_INSTALL_TARGETS)

endef

With that in place, the GRPC build moved on, but failed at a later point, with:

[HOSTCXX] Compiling src/compiler/cpp_generator.cc
[HOSTCXX] Compiling src/compiler/csharp_generator.cc [HOSTCXX] Compiling src/compiler/node_generator.cc
In file included from /usr/include/c++/7/ext/string_conversions.h:41:0,

                from /usr/include/c++/7/bits/basic_string.h:6349,
                from /usr/include/c++/7/string:52,
                from /usr/include/c++/7/stdexcept:39,
                from /usr/include/c++/7/array:39,
                from /usr/include/c++/7/tuple:39,
                from /usr/include/c++/7/bits/stl_map.h:63,
                from /usr/include/c++/7/map:61,
                from src/compiler/cpp_generator.cc:19:
/usr/include/c++/7/cstdlib:131:11: error: ?::at_quick_exit? has not been declared
  using ::at_quick_exit;
          ^~~~~~~~~~~~~
/usr/include/c++/7/cstdlib:154:11: error: ?::quick_exit? has not been declared
  using ::quick_exit;
          ^~~~~~~~~~
[HOSTCXX] Compiling src/compiler/objective_c_generator.cc
In file included from /usr/include/c++/7/ext/string_conversions.h:41:0,
                from /usr/include/c++/7/bits/basic_string.h:6349,
                from /usr/include/c++/7/string:52,
                from /usr/include/c++/7/stdexcept:39,
                from /usr/include/c++/7/array:39,
                from /usr/include/c++/7/tuple:39,
                from /usr/include/c++/7/bits/stl_map.h:63,
                from /usr/include/c++/7/map:61,
                from src/compiler/csharp_generator.cc:20:
/usr/include/c++/7/cstdlib:131:11: error: ?::at_quick_exit? has not been declared
  using ::at_quick_exit;
          ^~~~~~~~~~~~~
/usr/include/c++/7/cstdlib:154:11: error: ?::quick_exit? has not been declared
  using ::quick_exit;
          ^~~~~~~~~~

In file included from ./src/compiler/cpp_generator.h:26:0,
from src/compiler/cpp_generator.cc:21:
/usr/include/c++/7/memory: In function ?void* std::align(std::size_t, std::size_t, void*&, std::size_t&)?:
/usr/include/c++/7/memory:116:58: error: cast from ?void*? to ?uintptr_t {aka unsigned int}? loses precision [-fpermissive]
const auto __intptr = reinterpret_cast<uintptr_t>(__ptr);

And more of those issues. Here is the defconfig I have used:

BR2_arm=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-arm-full-2018.02-891-g046c5e2.tar.bz2" BR2_TOOLCHAIN_EXTERNAL_GCC_4_9=y
BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_1=y
BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
# BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
BR2_TOOLCHAIN_EXTERNAL_CXX=y
BR2_INIT_NONE=y
BR2_SYSTEM_BIN_SH_NONE=y
# BR2_PACKAGE_BUSYBOX is not set
BR2_PACKAGE_GRPC=y
# BR2_TARGET_ROOTFS_TAR is not set

Could you have a look ?

I'll also do a regular review of the package in a separate e-mail, so that you can take into account other comments at the same time.

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
#13 lore raw

PATCH v2 4/5 grpc: new package

Hello,

Quoted 4 lines · Charles Hardin

Ok - never tried ?arm? builds - only mips and x86, so this is likely going to be a bit more involved. But, I think this is a HOSTCXX problem - what is that version because I barely recall that I have to build the gRPC proto gen tools on the host to generate the stubs for the cross.

The problem is when building host tools with the host compiler, so the target architecture shouldn't matter.

Quoted 1 line

So, it looks like it is a gcc 7 build on this host side?

Yes, I'm running a Fedora system with gcc 7.3.1.

I see on https://github.com/grpc/grpc/releases that they have newer releases. Perhaps this issue has been fixed upstream.

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
#14 lore raw

PATCH v2 4/5 grpc: new package

In reply to charles.hardin at storagecraft.com

Hello,

Quoted 15 lines · charles.hardin at storagecraft.com

From: Charles Hardin <[email protected]>

add the gRPC package from Google's github repo. This is currently just installing the C and C++ libraries on the target installation for now. Some effort could be made to add the python bindings as a subsequent patch.

This also adds a patch to specify the protoc to use since that is provided by buildroot and not the host in addition to overriding the ldconfig command since that doesn't apply to the buildroot compile.

NOTE: This doesn't add the python or any other bindings

Signed-off-by Charles Hardin <[email protected]>
Quoted 11 lines
---
 package/Config.in                                  |   1 +
 ...onmental-overrides-of-protoc-dtrace-and-l.patch |  72 +++++++++++++++
 package/grpc/Config.in                             |  19 ++++
 package/grpc/grpc.hash                             |   2 +
 package/grpc/grpc.mk                               | 101 +++++++++++++++++++++
 5 files changed, 195 insertions(+)
 create mode 100644 package/grpc/0001-Allow-environmental-overrides-of-protoc-dtrace-and-l.patch
 create mode 100644 package/grpc/Config.in
 create mode 100644 package/grpc/grpc.hash
 create mode 100644 package/grpc/grpc.mk

Please add an entry to the DEVELOPERS file when adding a new package.

package/grpc/0001-Allow-environmental-overrides-of-protoc-dtrace-and-l.patch line 5

Quoted 11 lines
package/grpc/0001-Allow-environmental-overrides-of-protoc-dtrace-and-l.patch new file +5 -0
new file mode 100644
index 0000000000..79a16786fd
@@ -0,0 +1,72 @@
+From dedb8e2ba6d1ea4ea5745d2d991bc333353478ba Mon Sep 17 00:00:00 2001
+From: Charles Hardin <[email protected]>
+Date: Mon, 21 May 2018 09:39:31 -0700
+Subject: [PATCH 1/1] Allow environmental overrides of protoc, dtrace, and
+ ldconfg

Please generate patches with "git format-patch -N" to not have "PATCH 1/1" but just "PATCH".

package/grpc/0001-Allow-environmental-overrides-of-protoc-dtrace-and-l.patch line 16

Quoted 12 lines
package/grpc/0001-Allow-environmental-overrides-of-protoc-dtrace-and-l.patch new file +2 -0
+Some build systems (ie. buildroot) provide host tools as part
+of the build and those paths are local to build.

To address this

package/grpc/0001-Allow-environmental-overrides-of-protoc-dtrace-and-l.patch new file +8 -0
+just do a "which" against the passed in PROTOC=my/path/protoc
+and that can be used for the check version command.
+
+In addition, allow an ldconfig override on the install shared
+library builds since this might be cross compiling or being
+dumped into a chroot environment for something else.
+
+Signed-off-by: Charles Hardin <[email protected]>

I think this should be split in two patches:

- One making sure the PROTOC and DTRACE variables get used everywhere applicable.

- One introducing LDCONFIG

Also, it would be good to submit those patches upstream.

package/grpc/Config.in line 4

Quoted 10 lines
package/grpc/Config.in new file +4 -0
new file mode 100644
index 0000000000..d7096d0853
@@ -0,0 +1,19 @@
+config BR2_PACKAGE_GRPC
+	bool "grpc"
+	depends on BR2_INSTALL_LIBSTDCPP
+	depends on BR2_TOOLCHAIN_HAS_THREADS

depends on BR2_USE_MMU
depends on BR2_USE_WCHAR

(because you select gtest)

depends on BR2_PACKAGE_PROTOBUF_ARCH_SUPPORTS depends on BR2_HOST_GCC_AT_LEAST_4_8 # C++11 depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_8 # C++11 depends on !BR2_STATIC_LIBS

(because you select protobuf)

Quoted 5 lines
package/grpc/Config.in new file +5 -0
+	select BR2_PACKAGE_C_ARES
+	select BR2_PACKAGE_GFLAGS
+	select BR2_PACKAGE_GTEST
+	select BR2_PACKAGE_GTEST_GMOCK
+	select BR2_PACKAGE_OPENSSL

package/grpc/Config.in line 19

Quoted 10 lines
package/grpc/Config.in new file +10 -0
+	select BR2_PACKAGE_PROTOBUF
+	select BR2_PACKAGE_ZLIB
+	help
+	  gRPC is Google's protocol buffer based implementation of a
+	  remote procedure call protocol.
+
+	  http://www.grpc.io
+
+comment "grpc needs a toolchain w/ C++, threads"
+	depends on !(BR2_INSTALL_LIBSTDCPP && BR2_TOOLCHAIN_HAS_THREADS)

Please update this comment according to the additional dependencies above.

package/grpc/Config.in

Quoted 3 lines
package/grpc/Config.in new file +3 -0
+GRPC_VERSION = v1.11.1
+GRPC_SITE = $(call github,grpc,grpc,$(GRPC_VERSION))
+GRPC_LICENSE = BSD-3-Clause

The LICENSE file contains the Apache-2.0 license text. Are you sure about BSD-3-Clause here ?

Quoted 15 lines
package/grpc/Config.in new file +15 -0
+GRPC_LICENSE_FILES = LICENSE
+
+# Need a host protoc grpc plugin during the compilation
+GRPC_DEPENDENCIES = host-grpc gflags gtest c-ares openssl protobuf zlib
+HOST_GRPC_DEPENDENCIES = host-c-ares host-protobuf host-openssl
+
+GRPC_INSTALL_STAGING = YES
+
+GRPC_CROSS_MAKE_OPTS_BASE = \
+	GRPC_CROSS_COMPILE="true" \
+	LDCONFIG=/bin/true \
+	HOST_CC="$(HOSTCC)" \
+	HOST_CXX="$(HOSTCXX)" \
+	HOST_LD="$(HOSTCC)" \
+	HOST_LDXX="$(HOSTCXX)" \

package/grpc/Config.in

Quoted 10 lines
package/grpc/Config.in new file +10 -0
+	HOST_CPPFLAGS="$(HOST_CPPFLAGS)" \
+	HOST_CFLAGS="$(HOST_CFLAGS)" \
+	HOST_LDFLAGS="$(HOST_LDFLAGS)" \
+	CC="$(TARGET_CC)" \
+	CXX="$(TARGET_CXX)" \
+	LD="$(TARGET_CC)" \
+	LDXX="$(TARGET_CXX)" \
+	CFLAGS="$(TARGET_CFLAGS)" \
+	LDFLAGS="$(TARGET_LDFLAGS)" \
+	STRIP=/bin/true

As explained in my other e-mail, pass this in the environment so that the Makefile can do CFLAGS += something.

package/grpc/Config.in

Quoted 13 lines
package/grpc/Config.in new file +13 -0
+ifeq ($(BR2_SHARED_LIBS),y)
+GRPC_BUILD_TARGETS = shared plugins
+GRPC_STAGING_TARGETS = install-headers install-shared_c install-shared_cxx
+GRPC_INSTALL_TARGETS = install-shared_c install-shared_cxx
+else ifeq ($(BR2_STATIC_LIBS),y)
+GRPC_BUILD_TARGETS = static plugins
+GRPC_STAGING_TARGETS = install-headers install-static_c install-static_cxx
+GRPC_INSTALL_TARGETS = install-static_c install-static_cxx
+else
+GRPC_BUILD_TARGETS = static shared plugins
+GRPC_STAGING_TARGETS = install
+GRPC_INSTALL_TARGETS = install-shared_c install-shared_cxx
+endif

Perhaps this could be:

GRPC_BUILD_TARGETS = plugins
GRPC_STAGING_INSTALL_TARGETS = install-headers

ifeq ($(BR2_SHARED_LIBS)$(BR2_SHARED_STATIC_LIBS),y) GRPC_BUILD_TARGETS += shared GRPC_STAGING_INSTALL_TARGETS += install-shared_c install-shared_cxx GRPC_TARGET_INSTALL_TARGETS += install-shared_c install-shared_cxx endif

ifeq ($(BR2_STATIC_LIBS)$(BR2_SHARED_STATIC_LIBS),y) GRPC_BUILD_TARGETS += static GRPC_STAGING_INSTALL_TARGETS += install-static_c install-static_cxx GRPC_TARGET_INSTALL_TARGETS += install-static_c install-static_cxx endif

Best regards,

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
author
#15 lore raw

PATCH v2 5/5 package/collectd: allow the grpc plugin to be configured

In reply to charles.hardin at storagecraft.com

From: Charles Hardin <[email protected]>

Allow collectd to use the gRPC package since that package can now be compiled as part of buildroot. This plugin can be used to push collectd datasets over the network to other instances as needed.

Signed-off-by Charles Hardin <[email protected]>
---
 package/collectd/Config.in   | 11 +++++++++++
 package/collectd/collectd.mk |  4 +++-
 2 files changed, 14 insertions(+), 1 deletion(-)
package/collectd/Config.in +11 -0
index ccd3fbe13f..5f25f01cb8 100644
@@ -248,6 +248,17 @@ config BR2_PACKAGE_COLLECTD_GPS
 	  Reports the number of sattelites seen by and precision
 	  of a GPS receiver.
 
+config BR2_PACKAGE_COLLECTD_GRPC
+	bool "grpc"
+	depends on BR2_INSTALL_LIBSTDCPP
+	depends on BR2_TOOLCHAIN_HAS_THREADS
+	select BR2_PACKAGE_GRPC
+	help
+	  Add support for the gRPC plugin for network communication
+
+comment "grpc needs a toolchain w/ C++, threads"
+	depends on !(BR2_INSTALL_LIBSTDCPP && BR2_TOOLCHAIN_HAS_THREADS)
+
 config BR2_PACKAGE_COLLECTD_HUGEPAGES
 	bool "hugepages"
 	help
package/collectd/collectd.mk +3 -1
index 6c987b2e36..00efee4a18 100644
@@ -15,7 +15,7 @@ COLLECTD_LICENSE_FILES = COPYING
 # These require unmet dependencies, are fringe, pointless or deprecated
 COLLECTD_PLUGINS_DISABLE = \
 	amqp apple_sensors aquaero ascent barometer dbi dpdkstat email \
-	gmond grpc hddtemp intel_rdt ipmi java libvirt lpar lvm \
+	gmond hddtemp intel_rdt ipmi java libvirt lpar lvm \
 	madwifi mbmon mic multimeter netapp notify_desktop numa \
 	nut onewire oracle perl pf pinba powerdns python redis routeros \
 	rrdcached sigrok tape target_v5upgrade teamspeak2 ted \

@@ -73,6 +73,7 @@ COLLECTD_CONF_OPTS += \
 	$(if $(BR2_PACKAGE_COLLECTD_FSCACHE),--enable-fscache,--disable-fscache) \
 	$(if $(BR2_PACKAGE_COLLECTD_GPS),--enable-gps,--disable-gps) \
 	$(if $(BR2_PACKAGE_COLLECTD_GRAPHITE),--enable-write_graphite,--disable-write_graphite) \
+	$(if $(BR2_PACKAGE_COLLECTD_GRPC),--enable-grpc,--disable-grpc) \
 	$(if $(BR2_PACKAGE_COLLECTD_HASHED),--enable-match_hashed,--disable-match_hashed) \
 	$(if $(BR2_PACKAGE_COLLECTD_HUGEPAGES),--enable-hugepages,--disable-hugepages) \
 	$(if $(BR2_PACKAGE_COLLECTD_INTERFACE),--enable-interface,--disable-interface) \

@@ -148,6 +149,7 @@ COLLECTD_DEPENDENCIES = \
 	$(if $(BR2_PACKAGE_COLLECTD_CURL_XML),libcurl libxml2) \
 	$(if $(BR2_PACKAGE_COLLECTD_DNS),libpcap) \
 	$(if $(BR2_PACKAGE_COLLECTD_GPS),gpsd) \
+	$(if $(BR2_PACKAGE_COLLECTD_GRPC),grpc) \
 	$(if $(BR2_PACKAGE_COLLECTD_IPTABLES),iptables) \
 	$(if $(BR2_PACKAGE_COLLECTD_LOGSTASH),yajl) \
 	$(if $(BR2_PACKAGE_COLLECTD_MEMCACHEC),libmemcached) \
Signature · 1 line
2.15.1 (Apple Git-101)