docker-proxy: add recipe for docker-proxy

PATCH In review Pascal Bach 2 participants 7 messages Apr 6, 2018 – Apr 9, 2018

Share & tools

Pascal Bach

author

2018-04-06 13:42 UTC #0 lore raw

PATCH 1/2 docker-proxy: add recipe for docker-proxy

Before docker-proxy was built as part of the docker-recipe but there in some situations when built for x86_64 under x86_64 it incorrectly linked against host libraries.
This puts it into a separate recipe that makes use the go.bbclass to build the binary properly.

Signed-off-by Pascal Bach <[email protected]>

---
 recipes-containers/docker/docker-proxy_git.bb | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
 create mode 100644 recipes-containers/docker/docker-proxy_git.bb

recipes-containers/docker/docker-proxy_git.bb new file +21 -0

new file mode 100644
index 0000000..4271e58
@@ -0,0 +1,21 @@
+DESCRIPTION = "Docker proxy binary to forward traffic between host and containers"
+HOMEPAGE = "https://github.com/docker/libnetwork"
+
+LICENSE = "Apache-2.0"
+LIC_FILES_CHKSUM = "file://src/github.com/docker/libnetwork/LICENSE;md5=d2794c0df5b907fdace235a619d80314"
+
+SRC_URI = "git://${GO_IMPORT}"
+SRCREV = "4cb38c2987c236dce03c868d99b57b1e28a4b81c"
+
+GO_IMPORT = "github.com/docker/libnetwork"
+GO_INSTALL = "${GO_IMPORT}/cmd/proxy"
+
+DOCKER_VERSION = "18.03.0"
+PV = "${DOCKER_VERSION}+git${SRCREV}"
+
+inherit go
+
+# The binary is expected to be called docker-proxy
+do_install_append() {
+    mv ${D}${bindir}/proxy ${D}${bindir}/${BPN}
+}

Signature · 1 line

2.11.0

Pascal Bach

author

2018-04-06 13:42 UTC #1 lore raw

PATCH 2/2 docker: use docker-proxy instead of building it

Use the dedicated docker-proxy as RDEPENDS instead of squeezing it into the docker build.

Signed-off-by Pascal Bach <[email protected]>

---
 recipes-containers/docker/docker_git.bb | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)

recipes-containers/docker/docker_git.bb +1 -7

index 13c2731..ed7643b 100644
@@ -19,11 +19,9 @@ DESCRIPTION = "Linux container runtime \
  "
 
 SRCREV_docker = "708b068d3095c6a6be939eb2da78c921d2e945e2"
-SRCREV_libnetwork = "4cb38c2987c236dce03c868d99b57b1e28a4b81c"
 SRCREV_cli = "0f1bb353423e45e02315e985bd9ddebe6da18457"
 SRC_URI = "\
 	git://github.com/moby/moby.git;nobranch=1;name=docker \
-	git://github.com/docker/libnetwork.git;branch=master;name=libnetwork;destsuffix=libnetwork \
 	git://github.com/docker/cli;branch=master;name=cli;destsuffix=cli \
 	file://docker.init \
 	file://hi.Dockerfile \

@@ -68,7 +66,7 @@ RDEPENDS_${PN} = "util-linux iptables \
                   ${@bb.utils.contains('DISTRO_FEATURES', 'aufs', 'aufs-util', '', d)} \
                   ${@bb.utils.contains('DISTRO_FEATURES', 'systemd', '', 'cgroup-lite', d)} \
                  "
-RDEPENDS_${PN} += "virtual/containerd virtual/runc"
+RDEPENDS_${PN} += "virtual/containerd virtual/runc docker-proxy"
 
 RRECOMMENDS_${PN} = "kernel-module-dm-thin-pool kernel-module-nf-nat"
 RSUGGESTS_${PN} = "lxc rt-tests"

@@ -115,9 +113,6 @@ do_compile() {
 	# to build this:
 	VERSION="${DOCKER_VERSION}" DOCKER_GITCOMMIT="${SRCREV_docker}" ./hack/make.sh dynbinary
 
-	# build the proxy
-	go build -o ${S}/src/import/docker-proxy github.com/docker/libnetwork/cmd/proxy
-
         # build the cli
 	cd ${S}/src/import/.gopath/src/github.com/docker/cli
 	export CFLAGS=""

@@ -139,7 +134,6 @@ do_install() {
 	mkdir -p ${D}/${bindir}
 	cp ${WORKDIR}/cli/build/docker ${D}/${bindir}/docker
 	cp ${S}/src/import/bundles/latest/dynbinary-daemon/dockerd ${D}/${bindir}/dockerd
-	cp ${S}/src/import/docker-proxy ${D}/${bindir}/docker-proxy
 
 	if ${@bb.utils.contains('DISTRO_FEATURES','systemd','true','false',d)}; then
 		install -d ${D}${systemd_unitdir}/system

Signature · 1 line

2.11.0

Bruce Ashfield

2018-04-06 13:47 UTC #2 lore raw

In reply to Pascal Bach

Quoted 6 lines · Pascal Bach

Before docker-proxy was built as part of the docker-recipe but there in some situations when built for x86_64 under x86_64 it incorrectly linked against host libraries.
This puts it into a separate recipe that makes use the go.bbclass to build the binary properly.

As I hinted, I'm going to figure out how to integrate / fix this in the main docker recipe.

Having the various SRCREVs managed in multiple recipes is a maintenance pain and recipe for runtime errors.

I'll dig into what the go class is doing that is so different from what I've been trying.

Can you provide the output of your properly linked docker-proxy, so I can use it as a reference ?

Bruce

Quoted 41 lines

Signed-off-by Pascal Bach <[email protected]>

---
 recipes-containers/docker/docker-proxy_git.bb | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
 create mode 100644 recipes-containers/docker/docker-proxy_git.bb

recipes-containers/docker/docker-proxy_git.bb new file +21 -1

new file mode 100644
index 0000000..4271e58
@@ -0,0 +1,21 @@
+DESCRIPTION = "Docker proxy binary to forward traffic between host and containers"
+HOMEPAGE = "https://github.com/docker/libnetwork"
+
+LICENSE = "Apache-2.0"
+LIC_FILES_CHKSUM = "file://src/github.com/docker/libnetwork/LICENSE;md5=d2794c0df5b907fdace235a619d80314"
+
+SRC_URI = "git://${GO_IMPORT}"
+SRCREV = "4cb38c2987c236dce03c868d99b57b1e28a4b81c"
+
+GO_IMPORT = "github.com/docker/libnetwork"
+GO_INSTALL = "${GO_IMPORT}/cmd/proxy"
+
+DOCKER_VERSION = "18.03.0"
+PV = "${DOCKER_VERSION}+git${SRCREV}"
+
+inherit go
+
+# The binary is expected to be called docker-proxy
+do_install_append() {
+    mv ${D}${bindir}/proxy ${D}${bindir}/${BPN}
+}
--

2.11.0

--
_______________________________________________
meta-virtualization mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/meta-virtualization

Signature · 2 lines

"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"

Bach, Pascal

author

2018-04-06 14:23 UTC #3 lore raw

Hi Bruce

Quoted 20 lines

-----Original Message-----
From: Bruce Ashfield [mailto:[email protected]]
Sent: Freitag, 6. April 2018 15:48
To: Bach, Pascal (BT CPS R&D ZG FW CCP) <[email protected]>
Cc: [email protected]
Subject: Re: [meta-virtualization] [PATCH 1/2] docker-proxy: add recipe for docker-proxy
Quoted 6 lines · Pascal Bach
Before docker-proxy was built as part of the docker-recipe but there in some situations when built for x86_64 under x86_64 it incorrectly linked against host libraries.
This puts it into a separate recipe that makes use the go.bbclass to build the binary properly.
As I hinted, I'm going to figure out how to integrate / fix this in the main docker recipe.

Quoted 8 lines

Having the various SRCREVs managed in multiple recipes is a maintenance pain and recipe for runtime errors.
I'll dig into what the go class is doing that is so different from what I've been trying.
Can you provide the output of your properly linked docker-proxy, so I can use it as a reference ?

I'm not sure what you mean by output?

Pascal

BTW you can also ping me on IRC #yocto as bachp might be quicker to clarify

Bruce Ashfield

2018-04-09 14:25 UTC #4 lore raw

Quoted 25 lines · Pascal

Hi Bruce
Quoted 23 lines
-----Original Message-----
From: Bruce Ashfield [mailto:[email protected]]
Sent: Freitag, 6. April 2018 15:48
To: Bach, Pascal (BT CPS R&D ZG FW CCP) <[email protected]>
Cc: [email protected]
Subject: Re: [meta-virtualization] [PATCH 1/2] docker-proxy: add recipe for docker-proxy
Quoted 6 lines · Pascal Bach
Before docker-proxy was built as part of the docker-recipe but there in some situations when built for x86_64 under x86_64 it incorrectly linked against host libraries.
This puts it into a separate recipe that makes use the go.bbclass to build the binary properly.
As I hinted, I'm going to figure out how to integrate / fix this in the main docker recipe.
Having the various SRCREVs managed in multiple recipes is a maintenance pain and recipe for runtime errors.

Quoted 7 lines

Quoted 5 lines
I'll dig into what the go class is doing that is so different from what I've been trying.
Can you provide the output of your properly linked docker-proxy, so I can use it as a reference ?
I'm not sure what you mean by output?

I sorted things out using the libnetwork Make infrastructure and some spelunking into go internals.
I now have it using the sysroot infrastructure, and I can follow along with SRCREVs and any build internal changes to libnetwork:

file bin/d*
bin/dnet--amd64: ELF 64-bit LSB executable, x86-64, version 1
(SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=bc6059d72a9d6224049542214427bf3c62a41866, not stripped bin/docker-proxy--amd64: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=c52ed7c3b28284a683ade6b4c46ca4f6bad8d1fb, not stripped

Cheers,

Bruce

Quoted 5 lines

Pascal
BTW you can also ping me on IRC #yocto as bachp might be quicker to clarify

Signature · 2 lines

"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"

Bach, Pascal

author

2018-04-09 15:34 UTC #5 lore raw

Hi Bruce

Quoted 20 lines

Quoted 7 lines
Quoted 7 lines
As I hinted, I'm going to figure out how to integrate / fix this in the main docker recipe.
Having the various SRCREVs managed in multiple recipes is a maintenance pain and recipe for runtime errors.
Quoted 7 lines
Quoted 5 lines
I'll dig into what the go class is doing that is so different from what I've been trying.
Can you provide the output of your properly linked docker-proxy, so I can use it as a reference ?
I'm not sure what you mean by output?
I sorted things out using the libnetwork Make infrastructure and some spelunking into go internals.
I now have it using the sysroot infrastructure, and I can follow along with SRCREVs and any build internal changes to libnetwork:

Quoted 9 lines

file bin/d*
bin/dnet--amd64: ELF 64-bit LSB executable, x86-64, version 1
(SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=bc6059d72a9d6224049542214427bf3c62a41866, not stripped bin/docker-proxy--amd64: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=c52ed7c3b28284a683ade6b4c46ca4f6bad8d1fb, not stripped

I can confirm that it is correctly linking now.

Thanks for the fix.

Pascal

Bruce Ashfield

2018-04-09 15:39 UTC #6 lore raw

Quoted 32 lines · Pascal

Hi Bruce
Quoted 20 lines
Quoted 7 lines
Quoted 7 lines
As I hinted, I'm going to figure out how to integrate / fix this in the main docker recipe.
Having the various SRCREVs managed in multiple recipes is a maintenance pain and recipe for runtime errors.
Quoted 7 lines
Quoted 5 lines
I'll dig into what the go class is doing that is so different from what I've been trying.
Can you provide the output of your properly linked docker-proxy, so I can use it as a reference ?
I'm not sure what you mean by output?
I sorted things out using the libnetwork Make infrastructure and some spelunking into go internals.
I now have it using the sysroot infrastructure, and I can follow along with SRCREVs and any build internal changes to libnetwork:
Quoted 9 lines
file bin/d*
bin/dnet--amd64: ELF 64-bit LSB executable, x86-64, version 1
(SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=bc6059d72a9d6224049542214427bf3c62a41866, not stripped bin/docker-proxy--amd64: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=c52ed7c3b28284a683ade6b4c46ca4f6bad8d1fb, not stripped

Quoted 3 lines

I can confirm that it is correctly linking now.
Thanks for the fix.

And thanks for the report and pointers. Since it was running for me, I probably wouldn't have noticed for ages (and remembered that I forgot to remove that direct 'go' invocation!)

Cheers,

Bruce

Quoted 2 lines

Pascal

Signature · 2 lines

"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"