protobuf: static link tools when generating sdk
PATCH v3 1/5 protobuf: static link tools when generating sdk
Dynamically linked protoc.exe is failing as follows:
[libprotobuf ERROR google/protobuf/descriptor_database.cc:641] File already exists in database: google/protobuf/descriptor.proto [libprotobuf FATAL google/protobuf/descriptor.cc:1371] CHECK failed:
GeneratedDatabase()->Add(encoded_file_descriptor, size):
Switch to static linkage per upstream recommendation.
--- recipes-devtools/protobuf/protobuf_%.bbappend | 1 + 1 file changed, 1 insertion(+) create mode 100644 recipes-devtools/protobuf/protobuf_%.bbappend
recipes-devtools/protobuf/protobuf_%.bbappend new file +1 -0
@@ -0,0 +1 @@ +EXTRA_OECONF:append:sdkmingw32 = " --disable-shared"
Signature · 1 line
2.17.1
PATCH v3 2/5 protobuf-c: static link when generating sdk
[libprotobuf ERROR google/protobuf/descriptor_database.cc:641] File already exists in database: google/protobuf/descriptor.proto [libprotobuf FATAL google/protobuf/descriptor.cc:1371] CHECK failed:
GeneratedDatabase()->Add(encoded_file_descriptor, size):
Switch to static linkage per upstream recommendation.
--- recipes-devtools/protobuf-c/protobuf-c_%.bbappend | 1 + 1 file changed, 1 insertion(+) create mode 100644 recipes-devtools/protobuf-c/protobuf-c_%.bbappend
recipes-devtools/protobuf-c/protobuf-c_%.bbappend new file +1 -0
@@ -0,0 +1 @@ +EXTRA_OECONF:append:sdkmingw32 = " --disable-shared"
Signature · 1 line
2.17.1
PATCH v3 3/5 grpc: static link tools when generating SDK
In reply to Sinan Kaya[libprotobuf ERROR google/protobuf/descriptor_database.cc:641] File already exists in database: google/protobuf/descriptor.proto [libprotobuf FATAL google/protobuf/descriptor.cc:1371] CHECK failed:
GeneratedDatabase()->Add(encoded_file_descriptor, size):
Switch to static linkage per upstream recommendation.
--- recipes-devtools/grpc/grpc_%.bbappend | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 recipes-devtools/grpc/grpc_%.bbappend
recipes-devtools/grpc/grpc_%.bbappend new file +2 -0
@@ -0,0 +1,2 @@ +EXTRA_OECMAKE:remove:sdkmingw32 = "-DBUILD_SHARED_LIBS=ON" +EXTRA_OECMAKE:append:sdkmingw32 = " -DBUILD_SHARED_LIBS=OFF"
Signature · 1 line
2.17.1
PATCH v3 4/5 abseil-cpp: disable shared build as it is broken
In reply to Sinan Kaya--- recipes-devtools/abseil-cpp/abseil-cpp_%.bbappend | 1 + 1 file changed, 1 insertion(+) create mode 100644 recipes-devtools/abseil-cpp/abseil-cpp_%.bbappend
recipes-devtools/abseil-cpp/abseil-cpp_%.bbappend new file +1 -0
@@ -0,0 +1 @@ +EXTRA_OECMAKE:remove:mingw32 = "-DBUILD_SHARED_LIBS=ON"
Signature · 1 line
2.17.1
PATCH v3 5/5 conf/layer.conf: use BBFILES_DYNAMIC for dynamic layers
In reply to Sinan KayaAdd a dynamic BBFILES pattern so that patches for openembedded-layer are conditionally applied only if meta-oe is present.
--- conf/layer.conf | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
conf/layer.conf +8 -1
@@ -3,6 +3,12 @@ BBPATH := "${BBPATH}:${LAYERDIR}" # We have a packages directory, add to BBFILES BBFILES += "${LAYERDIR}/recipes*/*/*.bb ${LAYERDIR}/recipes*/*/*.bbappend" +BBFILES_DYNAMIC += "\ +openembedded-layer:${LAYERDIR}/dynamic-layers/openembedded-layer/recipes-*/*/*.bb +\ +openembedded-layer:${LAYERDIR}/dynamic-layers/openembedded-layer/recipes-*/*/*.bbappend +\ + " BBFILE_COLLECTIONS += "meta-mingw" BBFILE_PATTERN_meta-mingw := "^${LAYERDIR}/"
@@ -10,4 +16,5 @@ BBFILE_PRIORITY_meta-mingw = "8" LAYERDEPENDS_meta-mingw = "core" -LAYERSERIES_COMPAT_meta-mingw = "honister" +LAYERSERIES_COMPAT_meta-mingw = "honister" +
Signature · 1 line
2.17.1
I had to clean these up a little bit and pushed them to master-next. Please verify that they still work for you and if not used them as a starting point and send a V4 patchset.
Quoted 22 lines · Sinan Kaya
Dynamically linked protoc.exe is failing as follows:
[libprotobuf ERROR google/protobuf/descriptor_database.cc:641] File already exists in database: google/protobuf/descriptor.proto [libprotobuf FATAL google/protobuf/descriptor.cc:1371] CHECK failed:
GeneratedDatabase()->Add(encoded_file_descriptor, size):Switch to static linkage per upstream recommendation.
Signed-off-by Sinan Kaya <[email protected]>--- recipes-devtools/protobuf/protobuf_%.bbappend | 1 + 1 file changed, 1 insertion(+)create mode 100644 recipes-devtools/protobuf/protobuf_%.bbappend
recipes-devtools/protobuf/protobuf_%.bbappend new file +1
-0@@ -0,0 +1 @@ +EXTRA_OECONF:append:sdkmingw32 = " --disable-shared"
Quoted 3 lines · Joshua Watt
I had to clean these up a little bit and pushed them to master-next. Please verify that they still work for you and if not used them as a starting point and send a V4 patchset.
Sounds good.
What do you think about this?
My reading of the problem says that the problem only happens on win32 platforms due to how DLLs initialize the heap.
We have use cases of both dynamic and static linkage for the target build that we have not seen any issues with.
I'd like to keep target builds same supporting both static and dynamic build but limit the tools to static link by using sdkmingw32 instead of mingw32.
Should I send a V4?
Quoted 6 lines · Sinan Kaya
Quoted 3 lines · Joshua Watt
I had to clean these up a little bit and pushed them to master-next. Please verify that they still work for you and if not used them as a starting point and send a V4 patchset.
Sounds good.
Quoted 7 lines
What do you think about this?
My reading of the problem says that the problem only happens on win32 platforms due to how DLLs initialize the heap.
We have use cases of both dynamic and static linkage for the target build that we have not seen any issues with.
Are you building with "mingw32" as the target (not as the SDK), and it works there? I wonder why it only fails for the SDK build and not the target build? If that is truly the case, then yes I suppose it should be the "sdkmingw32" override. The strange thing about that (and why I thought it was incorrect) is that we have a few recipes that disable shared libraries and/or enable static with just the "mingw32" override, which is why I assumed it was a general limitation of MinGW, not just the SDK. I looked through the recipes, and it does seem more apparent that it is inconsistent, with a few recipe using "mingw32", a few using "class-nativesdk:mingw32", and a few using "sdkmingw32".
Anyway, please send a V4 if it needs to be changed, and I apologize for changing it unnecessarily :)
Quoted 6 lines
I'd like to keep target builds same supporting both static and dynamic build but limit the tools to static link by using sdkmingw32 instead of mingw32.
Should I send a V4?
Quoted 14 lines · Joshua Watt
Quoted 2 lines
We have use cases of both dynamic and static linkage for the target build that we have not seen any issues with.
Are you building with "mingw32" as the target (not as the SDK), and it works there? I wonder why it only fails for the SDK build and not the target build? If that is truly the case, then yes I suppose it should be the "sdkmingw32" override. The strange thing about that (and why I thought it was incorrect) is that we have a few recipes that disable shared libraries and/or enable static with just the "mingw32" override, which is why I assumed it was a general limitation of MinGW, not just the SDK. I looked through the recipes, and it does seem more apparent that it is inconsistent, with a few recipe using "mingw32", a few using "class-nativesdk:mingw32", and a few using "sdkmingw32".
I see two classes of problems based on my engagement with the mingw32. Some recipes just don't build without using the static method for the SDK and the target both. (abseil-cpp is one of them)
The nativesdk problem for grpc and protobuf is a silent failure. Even though we are able to build the binary, we get a binary that just doesn't work on windows. (crashes upon execution)
I still would like to be able to static/dynamic link grpc/protobuf for my target using the SDK.
Quoted 3 lines
Anyway, please send a V4 if it needs to be changed, and I apologize for changing it unnecessarily :)
Will do.
Quoted 18 lines · Sinan Kaya
Quoted 13 lines · Joshua Watt
Quoted 2 lines
We have use cases of both dynamic and static linkage for the target build that we have not seen any issues with.
Are you building with "mingw32" as the target (not as the SDK), and it works there? I wonder why it only fails for the SDK build and not the target build? If that is truly the case, then yes I suppose it should be the "sdkmingw32" override. The strange thing about that (and why I thought it was incorrect) is that we have a few recipes that disable shared libraries and/or enable static with just the "mingw32" override, which is why I assumed it was a general limitation of MinGW, not just the SDK. I looked through the recipes, and it does seem more apparent that it is inconsistent, with a few recipe using "mingw32", a few using "class-nativesdk:mingw32", and a few using "sdkmingw32".
I see two classes of problems based on my engagement with the mingw32. Some recipes just don't build without using the static method for the SDK and the target both. (abseil-cpp is one of them)
Quoted 6 lines
The nativesdk problem for grpc and protobuf is a silent failure. Even though we are able to build the binary, we get a binary that just doesn't work on windows. (crashes upon execution)
I still would like to be able to static/dynamic link grpc/protobuf for my target using the SDK.
RIght, the "mingw32" override shouldn't affect what you do with the *target* unless your target is MinGW; I'm not sure if anyone is actually doing that, most SDKs target Linux for ARM, Linux for AArch64, Linux for x86, etc.
What is your target?
It's important to remember that the recipe can be built differently for the SDK vs the target :)
Quoted 4 lines
Quoted 3 lines
Anyway, please send a V4 if it needs to be changed, and I apologize for changing it unnecessarily :)
Will do.
Quoted 8 lines · Joshua Watt
Quoted 2 lines
I still would like to be able to static/dynamic link grpc/protobuf for my target using the SDK.
RIght, the "mingw32" override shouldn't affect what you do with the *target* unless your target is MinGW; I'm not sure if anyone is actually doing that, most SDKs target Linux for ARM, Linux for AArch64, Linux for x86, etc.
Quoted 5 lines
What is your target?
It's important to remember that the recipe can be built differently for the SDK vs the target :)
ah, mine is both arm and arm64. No need to touch the recipe then.