Skip to contentSkip to conversation

support/testing: add docker / docker-compose tests

PATCH New Peter Korsgaard 4 participants 10 messages Feb 4, 2019 – Feb 5, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 support/testing: add docker / docker-compose tests

Build for x86-64 as public containers in general are only available for x86-64. Docker needs a number of kernel options enabled, so use a custom kernel config based on the qemu one.

Docker needs entropy at startup, so enable the virtio-rng-pci device to expose entropy to the guest. The default RAM amount (128M) is not enough to run docker / docker-compose, so bump to 512MB.

---
 .gitlab-ci.yml                                     |   1 +
 support/testing/conf/docker-compose-kernel.config  | 103 +++++++++++++++++++++
 support/testing/conf/docker-compose.yml            |   4 +
 .../testing/tests/package/test_docker_compose.py   |  70 ++++++++++++++
 4 files changed, 178 insertions(+)
 create mode 100644 support/testing/conf/docker-compose-kernel.config
 create mode 100644 support/testing/conf/docker-compose.yml
 create mode 100644 support/testing/tests/package/test_docker_compose.py
.gitlab-ci.yml +1 -0
index 8f1e4ae804..69d6291039 100644
@@ -318,6 +318,7 @@ tests.init.test_systemd.TestInitSystemSystemdRwFull: *runtime_test
 tests.init.test_systemd.TestInitSystemSystemdRwIfupdown: *runtime_test
 tests.init.test_systemd.TestInitSystemSystemdRwNetworkd: *runtime_test
 tests.package.test_atop.TestAtop: *runtime_test
+tests.package.test_docker_compose.TestDockerCompose: *runtime_test
 tests.package.test_dropbear.TestDropbear: *runtime_test
 tests.package.test_ipython.TestIPythonPy2: *runtime_test
 tests.package.test_ipython.TestIPythonPy3: *runtime_test
support/testing/conf/docker-compose-kernel.config new file +103 -0

103 lines not on this page · raw message

support/testing/conf/docker-compose.yml new file +4 -0
new file mode 100644
index 0000000000..49ff2677da
@@ -0,0 +1,4 @@
+version: '3'
+services:
+  busybox:
+    image: "busybox:latest"
support/testing/tests/package/test_docker_compose.py new file +70 -0

70 lines not on this page · raw message

Signature · 1 line
2.11.0
#1 lore raw

PATCH 1/1 support/testing: add docker / docker-compose tests

Peter,

Quoted 7 lines · Peter Korsgaard

Build for x86-64 as public containers in general are only available for x86-64. Docker needs a number of kernel options enabled, so use a custom kernel config based on the qemu one.

Docker needs entropy at startup, so enable the virtio-rng-pci device to expose entropy to the guest.

Another option is enabling BR2_PACKAGE_HAVEGED to let the target compensate.

Quoted 4 lines

The default RAM amount (128M) is not enough to
run docker / docker-compose, so bump to 512MB.

Very cool, I was just setting up a similar test case. Few notes below.

Quoted 121 lines
---
 .gitlab-ci.yml                                     |   1 +
 support/testing/conf/docker-compose-kernel.config  | 103 +++++++++++++++++++++
 support/testing/conf/docker-compose.yml            |   4 +
 .../testing/tests/package/test_docker_compose.py   |  70 ++++++++++++++
 4 files changed, 178 insertions(+)
 create mode 100644 support/testing/conf/docker-compose-kernel.config
 create mode 100644 support/testing/conf/docker-compose.yml
 create mode 100644 support/testing/tests/package/test_docker_compose.py
.gitlab-ci.yml +1 -0
index 8f1e4ae804..69d6291039 100644
@@ -318,6 +318,7 @@ tests.init.test_systemd.TestInitSystemSystemdRwFull: *runtime_test
 tests.init.test_systemd.TestInitSystemSystemdRwIfupdown: *runtime_test
 tests.init.test_systemd.TestInitSystemSystemdRwNetworkd: *runtime_test
 tests.package.test_atop.TestAtop: *runtime_test
+tests.package.test_docker_compose.TestDockerCompose: *runtime_test
 tests.package.test_dropbear.TestDropbear: *runtime_test
 tests.package.test_ipython.TestIPythonPy2: *runtime_test
 tests.package.test_ipython.TestIPythonPy3: *runtime_test
support/testing/conf/docker-compose-kernel.config new file +93 -0
new file mode 100644
index 0000000000..9468ef8db8
@@ -0,0 +1,103 @@
+# CONFIG_SWAP is not set
+CONFIG_SYSVIPC=y
+CONFIG_POSIX_MQUEUE=y
+# CONFIG_CROSS_MEMORY_ATTACH is not set
+CONFIG_HIGH_RES_TIMERS=y
+CONFIG_CGROUPS=y
+CONFIG_MEMCG=y
+CONFIG_BLK_CGROUP=y
+CONFIG_CGROUP_SCHED=y
+CONFIG_CGROUP_PIDS=y
+CONFIG_CGROUP_FREEZER=y
+CONFIG_CGROUP_DEVICE=y
+CONFIG_CGROUP_CPUACCT=y
+# CONFIG_COMPAT_BRK is not set
+# CONFIG_X86_EXTENDED_PLATFORM is not set
+CONFIG_IOSF_MBI=y
+CONFIG_MCORE2=y
+# CONFIG_X86_MCE is not set
+# CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS is not set
+CONFIG_LEGACY_VSYSCALL_NONE=y
+# CONFIG_SUSPEND is not set
+# CONFIG_ACPI is not set
+CONFIG_CPU_IDLE=y
+CONFIG_PCI_MSI=y
+# CONFIG_VIRTUALIZATION is not set
+# CONFIG_IOSCHED_CFQ is not set
+# CONFIG_MQ_IOSCHED_DEADLINE is not set
+# CONFIG_MQ_IOSCHED_KYBER is not set
+# CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS is not set
+CONFIG_TRANSPARENT_HUGEPAGE=y
+CONFIG_NET=y
+CONFIG_PACKET=y
+CONFIG_UNIX=y
+CONFIG_INET=y
+CONFIG_IP_MULTICAST=y
+CONFIG_IP_ADVANCED_ROUTER=y
+# CONFIG_INET_XFRM_MODE_TRANSPORT is not set
+# CONFIG_INET_XFRM_MODE_TUNNEL is not set
+# CONFIG_INET_XFRM_MODE_BEET is not set
+# CONFIG_IPV6 is not set
+CONFIG_NETFILTER=y
+CONFIG_NF_CONNTRACK=y
+# CONFIG_NF_CONNTRACK_PROCFS is not set
+# CONFIG_NF_CT_PROTO_DCCP is not set
+# CONFIG_NF_CT_PROTO_SCTP is not set
+# CONFIG_NF_CT_PROTO_UDPLITE is not set
+CONFIG_NETFILTER_XT_MATCH_ADDRTYPE=y
+CONFIG_NETFILTER_XT_MATCH_CONNTRACK=y
+CONFIG_NETFILTER_XT_MATCH_IPVS=y
+CONFIG_IP_VS=y
+CONFIG_IP_VS_RR=y
+CONFIG_IP_VS_NFCT=y
+CONFIG_IP_NF_IPTABLES=y
+CONFIG_IP_NF_FILTER=y
+CONFIG_IP_NF_NAT=y
+CONFIG_IP_NF_TARGET_MASQUERADE=y
+CONFIG_BRIDGE=y
+# CONFIG_BRIDGE_IGMP_SNOOPING is not set
+# CONFIG_WIRELESS is not set
+CONFIG_DEVTMPFS=y
+CONFIG_DEVTMPFS_MOUNT=y
+CONFIG_VIRTIO_BLK=y
+CONFIG_NETDEVICES=y
+CONFIG_DUMMY=y
+CONFIG_MACVLAN=y
+CONFIG_VXLAN=y
+CONFIG_VETH=y
+CONFIG_VIRTIO_NET=y
+# CONFIG_ETHERNET is not set
+# CONFIG_WLAN is not set
+# CONFIG_INPUT_KEYBOARD is not set
+# CONFIG_INPUT_MOUSE is not set
+# CONFIG_SERIO is not set
+CONFIG_VT_HW_CONSOLE_BINDING=y
+# CONFIG_DEVMEM is not set
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y
+# CONFIG_SERIAL_8250_LPSS is not set
+# CONFIG_SERIAL_8250_MID is not set
+# CONFIG_HW_RANDOM_INTEL is not set
+# CONFIG_HW_RANDOM_AMD is not set
+# CONFIG_HW_RANDOM_VIA is not set
+CONFIG_HW_RANDOM_VIRTIO=y
+# CONFIG_DEVPORT is not set
+# CONFIG_HWMON is not set
+# CONFIG_USB_SUPPORT is not set
+CONFIG_RTC_CLASS=y
+CONFIG_VIRTIO_PCI=y
+# CONFIG_VIRTIO_PCI_LEGACY is not set
+# CONFIG_X86_PLATFORM_DEVICES is not set
+# CONFIG_IOMMU_SUPPORT is not set
+CONFIG_EXT4_FS=y
+CONFIG_OVERLAY_FS=y

support/testing/conf/docker-compose-kernel.config line 103

Quoted 10 lines
support/testing/conf/docker-compose-kernel.config new file +10 -0
+CONFIG_OVERLAY_FS_REDIRECT_DIR=y
+CONFIG_TMPFS=y
+# CONFIG_MISC_FILESYSTEMS is not set
+# CONFIG_NETWORK_FILESYSTEMS is not set
+CONFIG_KEYS=y
+CONFIG_SECURITYFS=y
+# CONFIG_CRYPTO_ECHAINIV is not set
+# CONFIG_CRYPTO_HW is not set
+# CONFIG_UNUSED_SYMBOLS is not set
+CONFIG_UNWINDER_FRAME_POINTER=y

There is a script "./build/docker-engine*/contrib/check-config.sh" which could be copied over to the target and used to do a docker configuration check as part of the test case (would need config.gz enabled). I had started to add this as a br2-external post script, I could send something after this merges.

Quoted 54 lines
support/testing/conf/docker-compose.yml new file +4 -0
new file mode 100644
index 0000000000..49ff2677da
@@ -0,0 +1,4 @@
+version: '3'
+services:
+  busybox:
+    image: "busybox:latest"
support/testing/tests/package/test_docker_compose.py new file +38 -0
new file mode 100644
index 0000000000..8bf3ae00b5
@@ -0,0 +1,70 @@
+import os
+
+import infra.basetest
+
+
+class TestDockerCompose(infra.basetest.BRTest):
+    config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
+        """
+        BR2_x86_64=y
+        BR2_x86_core2=y
+        BR2_TOOLCHAIN_EXTERNAL=y
+        BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+        BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+        BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-x86-64-core2-full-2018.05.tar.bz2"
+        BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+        BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_16=y
+        BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+        # BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
+        BR2_TOOLCHAIN_EXTERNAL_CXX=y
+        BR2_SYSTEM_DHCP="eth0"
+        BR2_ROOTFS_POST_BUILD_SCRIPT="{}"
+        BR2_ROOTFS_POST_SCRIPT_ARGS="{}"
+        BR2_LINUX_KERNEL=y
+        BR2_LINUX_KERNEL_CUSTOM_VERSION=y
+        BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.19"
+        BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
+        BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="{}"
+        BR2_PACKAGE_CA_CERTIFICATES=y
+        BR2_PACKAGE_CGROUPFS_MOUNT=y
+        BR2_PACKAGE_DOCKER_CLI=y
+        BR2_PACKAGE_DOCKER_COMPOSE=y
+        BR2_PACKAGE_DOCKER_ENGINE=y
+        BR2_TARGET_ROOTFS_EXT2=y
+        BR2_TARGET_ROOTFS_EXT2_SIZE="512M"
+        # BR2_TARGET_ROOTFS_TAR is not set
+        """.format(
+            infra.filepath("tests/package/copy-sample-script-to-target.sh"),
+            infra.filepath("conf/docker-compose.yml"),

support/testing/tests/package/test_docker_compose.py line 48

Quoted 10 lines
support/testing/tests/package/test_docker_compose.py new file +10 -0
+            infra.filepath("conf/docker-compose-kernel.config"))
+
+    def wait_for_dockerd(self):
+        # dockerd takes a while to start up
+        _, _ = self.emulator.run('while [ ! -e /var/run/docker.sock ]; do sleep 1; done', 120)
+
+    def docker_test(self):
+        # will download container if not available, which may take some time
+        _, exit_code = self.emulator.run('docker run --rm busybox:latest /bin/true', 120)
+        self.assertEqual(exit_code, 0)

Another way to test could be to directly grep /proc/self/cgroups for the presence of docker. Does the return code cover that the container was all the way up?

Unrelated to above, I ran into errors like the following but I was targeting my build for aarch64 in my runtime test.

Building Go toolchain3 using go_bootstrap and Go toolchain2. Building packages and commands for host, linux/amd64. Building packages and commands for target, linux/arm64.

# cmd/trace
/accts/mlweber1/wip/TestDocker/build/host-go-1.11.5/pkg/tool/linux_amd64/link:
running /accts/mlweber1/wip/TestDocker/host/bin/aarch64-linux-gnu-gcc
failed: exit status 1
aarch64-linux-gnu-gcc: error: unrecognized command line option
'-Qunused-arguments'; did you mean '-Wunused-parameter'?
make[1]: *** [package/pkg-generic.mk:233:
/accts/mlweber1/wip/TestDocker/build/host-go-1.11.5/.stamp_built]
Error 2

Matt

#2 lore raw

PATCH 1/1 support/testing: add docker / docker-compose tests

Hello,

Quoted 10 lines · Matthew Weber
Quoted 7 lines · Peter Korsgaard

Build for x86-64 as public containers in general are only available for x86-64. Docker needs a number of kernel options enabled, so use a custom kernel config based on the qemu one.

Docker needs entropy at startup, so enable the virtio-rng-pci device to expose entropy to the guest.

Another option is enabling BR2_PACKAGE_HAVEGED to let the target compensate.

This is already used for 2 python test cases (pynacl and crossbar). I don't have specific knowledge to say which is the best approach. For the python test cases I just searched for *any* way to have enough entropy.

Regards,
Ricardo

#3 lore raw

PATCH 1/1 support/testing: add docker / docker-compose tests

Quoted 11 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"ricardo" == ricardo martincoski <[email protected]> writes:

Hello,

Quoted 6 lines · Matthew Weber
Quoted 5 lines · Peter Korsgaard

Build for x86-64 as public containers in general are only available for x86-64. Docker needs a number of kernel options enabled, so use a custom kernel config based on the qemu one.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

Docker needs entropy at startup, so enable the virtio-rng-pci device to expose entropy to the guest.

Another option is enabling BR2_PACKAGE_HAVEGED to let the target compensate.

This is already used for 2 python test cases (pynacl and crossbar). I don't have specific knowledge to say which is the best approach. For the python test cases I just searched for *any* way to have enough entropy.

I don't think it is really clear cut. Haveged is probably not something you should be using on a real system, and it only starts up relatively late so there could be issues with early user space (E.G. I have had issues with running mkfs.ext4 very early), but the virtio-rng-pci device is only available for a limited set of qemu targets (E.G. the ones that emulate a PCI bus).

Signature · 1 line
Bye, Peter Korsgaard
#4 lore raw

PATCH 1/1 support/testing: add docker / docker-compose tests

In reply to Matthew Weber
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Matthew" == Matthew Weber <[email protected]> writes:

Peter,

Quoted 1 line · Peter Korsgaard
Quoted 8 lines
Quoted 6 lines

Build for x86-64 as public containers in general are only available for x86-64. Docker needs a number of kernel options enabled, so use a custom kernel config based on the qemu one.

Docker needs entropy at startup, so enable the virtio-rng-pci device to expose entropy to the guest.

Another option is enabling BR2_PACKAGE_HAVEGED to let the target compensate.

True, but given that most real systems have a hw rng, this is probably closer to real life.

Quoted 7 lines
Quoted 4 lines

The default RAM amount (128M) is not enough to
run docker / docker-compose, so bump to 512MB.

Very cool, I was just setting up a similar test case. Few notes below.

Quoted 7 lines

There is a script "./build/docker-engine*/contrib/check-config.sh" which could be copied over to the target and used to do a docker configuration check as part of the test case (would need config.gz enabled). I had started to add this as a br2-external post script, I could send something after this merges.

Yes, I had originally used this script to figure out the kernel options to use:

From the moby check-config script:
(https://github.com/moby/moby/blob/e2de2123399f494cb41a4cb62392999c80c2e99c/contrib/check-config.sh)

info: reading kernel config from /proc/config.gz ...

Generally Necessary:
- cgroup hierarchy: properly mounted [/sys/fs/cgroup]
- CONFIG_NAMESPACES: enabled
- CONFIG_NET_NS: enabled
- CONFIG_PID_NS: enabled
- CONFIG_IPC_NS: enabled
- CONFIG_UTS_NS: enabled
- CONFIG_CGROUPS: enabled
- CONFIG_CGROUP_CPUACCT: enabled
- CONFIG_CGROUP_DEVICE: enabled
- CONFIG_CGROUP_FREEZER: enabled
- CONFIG_CGROUP_SCHED: enabled
- CONFIG_CPUSETS: enabled
- CONFIG_MEMCG: enabled
- CONFIG_KEYS: enabled
- CONFIG_VETH: enabled
- CONFIG_BRIDGE: enabled
- CONFIG_BRIDGE_NETFILTER: enabled
- CONFIG_NF_NAT_IPV4: enabled
- CONFIG_IP_NF_FILTER: enabled
- CONFIG_IP_NF_TARGET_MASQUERADE: enabled
- CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: enabled
- CONFIG_NETFILTER_XT_MATCH_CONNTRACK: enabled
- CONFIG_NETFILTER_XT_MATCH_IPVS: enabled
- CONFIG_IP_NF_NAT: enabled
- CONFIG_NF_NAT: enabled
- CONFIG_NF_NAT_NEEDED: enabled
- CONFIG_POSIX_MQUEUE: enabled

Optional Features:
- CONFIG_USER_NS: missing
- CONFIG_SECCOMP: enabled
- CONFIG_CGROUP_PIDS: enabled
- CONFIG_MEMCG_SWAP: missing
- CONFIG_MEMCG_SWAP_ENABLED: missing
- CONFIG_LEGACY_VSYSCALL_NONE: enabled
(containers using eglibc <= 2.13 will not work. Switch to
"CONFIG_VSYSCALL_[NATIVE|EMULATE]" or use "vsyscall=[native|emulate]" on kernel command line. Note that this will disable ASLR for the, VDSO which may assist in exploiting security vulnerabilities.)
- CONFIG_BLK_CGROUP: enabled
- CONFIG_BLK_DEV_THROTTLING: missing
- CONFIG_IOSCHED_CFQ: missing
- CONFIG_CFQ_GROUP_IOSCHED: missing
- CONFIG_CGROUP_PERF: missing
- CONFIG_CGROUP_HUGETLB: missing
- CONFIG_NET_CLS_CGROUP: missing
- CONFIG_CGROUP_NET_PRIO: missing
- CONFIG_CFS_BANDWIDTH: missing
- CONFIG_FAIR_GROUP_SCHED: enabled
- CONFIG_RT_GROUP_SCHED: missing
- CONFIG_IP_VS: enabled
- CONFIG_IP_VS_NFCT: enabled
- CONFIG_IP_VS_RR: enabled
- CONFIG_EXT4_FS: enabled
- CONFIG_EXT4_FS_POSIX_ACL: missing
- CONFIG_EXT4_FS_SECURITY: missing
enable these ext4 configs if you are using ext3 or ext4 as backing filesystem
- Network Drivers:
- "overlay":
- CONFIG_VXLAN: enabled
Optional (for encrypted networks):
- CONFIG_CRYPTO: enabled
- CONFIG_CRYPTO_AEAD: missing
- CONFIG_CRYPTO_GCM: missing
- CONFIG_CRYPTO_SEQIV: missing
- CONFIG_CRYPTO_GHASH: missing
- CONFIG_XFRM: missing
- CONFIG_XFRM_USER: missing
- CONFIG_XFRM_ALGO: missing
- CONFIG_INET_ESP: missing
- CONFIG_INET_XFRM_MODE_TRANSPORT: missing
- "ipvlan":
- CONFIG_IPVLAN: missing
- "macvlan":
- CONFIG_MACVLAN: enabled
- CONFIG_DUMMY: enabled
- "ftp,tftp client in container":
- CONFIG_NF_NAT_FTP: missing
- CONFIG_NF_CONNTRACK_FTP: missing
- CONFIG_NF_NAT_TFTP: missing
- CONFIG_NF_CONNTRACK_TFTP: missing
- Storage Drivers:
- "aufs":
- CONFIG_AUFS_FS: missing
- "btrfs":
- CONFIG_BTRFS_FS: missing
- CONFIG_BTRFS_FS_POSIX_ACL: missing
- "devicemapper":
- CONFIG_BLK_DEV_DM: missing
- CONFIG_DM_THIN_PROVISIONING: missing
- "overlay":
- CONFIG_OVERLAY_FS: enabled
- "zfs":
- /dev/zfs: missing
- zfs command: missing
- zpool command: missing

Limits:
- /proc/sys/kernel/keys/root_maxkeys: 1000000

I did indeed afterwards clean up the kernel config a bit, and dropped CONFIG_IKCONFIG.

Quoted 62 lines
Quoted 62 lines
support/testing/conf/docker-compose.yml new file +4 -0
new file mode 100644
index 0000000000..49ff2677da
@@ -0,0 +1,4 @@
+version: '3'
+services:
+  busybox:
+    image: "busybox:latest"
support/testing/tests/package/test_docker_compose.py new file +14 -0
new file mode 100644
index 0000000000..8bf3ae00b5
@@ -0,0 +1,70 @@
+import os
+
+import infra.basetest
+
+
+class TestDockerCompose(infra.basetest.BRTest):
+    config = infra.basetest.BASIC_TOOLCHAIN_CONFIG + \
+        """
+        BR2_x86_64=y
+        BR2_x86_core2=y
+        BR2_TOOLCHAIN_EXTERNAL=y
+        BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
+        BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD=y
+

BR2_TOOLCHAIN_EXTERNAL_URL="http://autobuild.buildroot.org/toolchains/tarballs/br-x86-64-core2-full-2018.05.tar.bz2"

support/testing/tests/package/test_docker_compose.py new file +30 -0
+        BR2_TOOLCHAIN_EXTERNAL_GCC_6=y
+        BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_16=y
+        BR2_TOOLCHAIN_EXTERNAL_LOCALE=y
+        # BR2_TOOLCHAIN_EXTERNAL_HAS_THREADS_DEBUG is not set
+        BR2_TOOLCHAIN_EXTERNAL_CXX=y
+        BR2_SYSTEM_DHCP="eth0"
+        BR2_ROOTFS_POST_BUILD_SCRIPT="{}"
+        BR2_ROOTFS_POST_SCRIPT_ARGS="{}"
+        BR2_LINUX_KERNEL=y
+        BR2_LINUX_KERNEL_CUSTOM_VERSION=y
+        BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.19"
+        BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
+        BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="{}"
+        BR2_PACKAGE_CA_CERTIFICATES=y
+        BR2_PACKAGE_CGROUPFS_MOUNT=y
+        BR2_PACKAGE_DOCKER_CLI=y
+        BR2_PACKAGE_DOCKER_COMPOSE=y
+        BR2_PACKAGE_DOCKER_ENGINE=y
+        BR2_TARGET_ROOTFS_EXT2=y
+        BR2_TARGET_ROOTFS_EXT2_SIZE="512M"
+        # BR2_TARGET_ROOTFS_TAR is not set
+        """.format(
+            infra.filepath("tests/package/copy-sample-script-to-target.sh"),
+            infra.filepath("conf/docker-compose.yml"),
+            infra.filepath("conf/docker-compose-kernel.config"))
+
+    def wait_for_dockerd(self):
+        # dockerd takes a while to start up
+        _, _ = self.emulator.run('while [ ! -e /var/run/docker.sock ]; do sleep 1; done', 120)
+
Quoted 8 lines

support/testing/tests/package/test_docker_compose.py line 48

Quoted 4 lines
support/testing/tests/package/test_docker_compose.py new file +4 -0
+    def docker_test(self):
+        # will download container if not available, which may take some time
+        _, exit_code = self.emulator.run('docker run --rm busybox:latest /bin/true', 120)
+        self.assertEqual(exit_code, 0)

Another way to test could be to directly grep /proc/self/cgroups for the presence of docker. Does the return code cover that the container was all the way up?

The return code is the return code from executing /bin/true inside the container, so it should only succeed if we were really able to successfully execute it.

Quoted 6 lines

Unrelated to above, I ran into errors like the following but I was targeting my build for aarch64 in my runtime test.

Building Go toolchain3 using go_bootstrap and Go toolchain2. Building packages and commands for host, linux/amd64. Building packages and commands for target, linux/arm64.

Quoted 9 lines
# cmd/trace
/accts/mlweber1/wip/TestDocker/build/host-go-1.11.5/pkg/tool/linux_amd64/link:
running /accts/mlweber1/wip/TestDocker/host/bin/aarch64-linux-gnu-gcc
failed: exit status 1
aarch64-linux-gnu-gcc: error: unrecognized command line option
'-Qunused-arguments'; did you mean '-Wunused-parameter'?
make[1]: *** [package/pkg-generic.mk:233:
/accts/mlweber1/wip/TestDocker/build/host-go-1.11.5/.stamp_built]
Error 2

Hmm, I haven't seen such error before. Googling around, it seems to be a command line argument supported by clang:

https://github.com/Ericsson/codechecker/issues/985

Signature · 1 line
Bye, Peter Korsgaard
#5 lore raw

PATCH 1/1 support/testing: add docker / docker-compose tests

In reply to Peter Korsgaard
Quoted 3 lines · On Mon, 4 Feb 2019 17:55:57 +0100 Peter Korsgaard <[email protected]> wrote:

Build for x86-64 as public containers in general are only available for x86-64. Docker needs a number of kernel options enabled, so use a custom kernel config based on the qemu one.

Do you need to duplicate the kernel config entirely, or can you use board/qemu/x86_64/linux.config + a fragment ?

Quoted 3 lines

Docker needs entropy at startup, so enable the virtio-rng-pci device to expose entropy to the guest. The default RAM amount (128M) is not enough to run docker / docker-compose, so bump to 512MB.

Wow. Go is efficient :)

support/testing/conf/docker-compose.yml line 4

Quoted 10 lines
support/testing/conf/docker-compose.yml new file +4 -0
new file mode 100644
index 0000000000..49ff2677da
@@ -0,0 +1,4 @@
+version: '3'
+services:
+  busybox:
+    image: "busybox:latest"

Is conf/ really the right place for this ? Don't we have some other place to store the per-tests artifacts that go on the target ? I don't think we have a good organization for this at the moment, so this will do for now.

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#6 lore raw

PATCH 1/1 support/testing: add docker / docker-compose tests

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni <[email protected]> writes:

Hi,

Quoted 9 lines
Quoted 3 lines · On Mon, 4 Feb 2019 17:55:57 +0100 Peter Korsgaard <[email protected]> wrote:

Build for x86-64 as public containers in general are only available for x86-64. Docker needs a number of kernel options enabled, so use a custom kernel config based on the qemu one.

Do you need to duplicate the kernel config entirely, or can you use board/qemu/x86_64/linux.config + a fragment ?

I didn't really look closely. I recently setup docker / docker-compose at $WORK, so I based the linux config on that.

I can have a look at basing it on the qemu config, but that means that we have to be a bit careful when bumping the qemu config, so I'm not sure if it makes sense.

Quoted 5 lines
Quoted 3 lines

Docker needs entropy at startup, so enable the virtio-rng-pci device to expose entropy to the guest. The default RAM amount (128M) is not enough to run docker / docker-compose, so bump to 512MB.

Wow. Go is efficient :)

;)

Quoted 7 lines
Quoted 7 lines
support/testing/conf/docker-compose.yml new file +1 -0
new file mode 100644
index 0000000000..49ff2677da
@@ -0,0 +1,4 @@
+version: '3'
Quoted 8 lines

support/testing/conf/docker-compose.yml line 4

Quoted 3 lines
support/testing/conf/docker-compose.yml new file +3 -0
+services:
+  busybox:
+    image: "busybox:latest"

Is conf/ really the right place for this ? Don't we have some other place to store the per-tests artifacts that go on the target ? I don't think we have a good organization for this at the moment, so this will do for now.

Indeed. We have a number of more-or-less random files under conf/, and then E.G. tests/package/test_python_*.py stores files for the target also under tests/package/sample_*.py

I can move the files under tests/package if you prefer, but I'm not sure if that is really cleaner.

Signature · 1 line
Bye, Peter Korsgaard
#7 lore raw

PATCH 1/1 support/testing: add docker / docker-compose tests

Hello,

+ Yann

Quoted 12 lines · Peter Korsgaard
Quoted 11 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Thomas" == Thomas Petazzoni writes:

Quoted 3 lines · On Mon, 4 Feb 2019 17:55:57 +0100 Peter Korsgaard wrote:

Build for x86-64 as public containers in general are only available for x86-64. Docker needs a number of kernel options enabled, so use a custom kernel config based on the qemu one.

Do you need to duplicate the kernel config entirely, or can you use board/qemu/x86_64/linux.config + a fragment ?

Quoted 6 lines

I didn't really look closely. I recently setup docker / docker-compose at $WORK, so I based the linux config on that.

I can have a look at basing it on the qemu config, but that means that we have to be a bit careful when bumping the qemu config, so I'm not sure if it makes sense.

The nice thing of using a fragment is that we would have an example of the configs that one needs to enable in the kernel in order to use docker in the target.
It should be *theoretically* straight-forward created from the docker documentation. I don't know how well it is documented.
Any config that docker needs that is already in the base config (so showing in both the base config and the fragment) should be correctly set by the merge of configs, I guess.

But maybe we find some practical problem when trying to accomplish this theoretical task, perhaps when changing the major version of the kernel. Or even you think that maintaining such example config is too much out-of-scope for the Buildroot source tree.

So... up to you guys. And it can be done in a followup patch.

[snip]

Quoted 12 lines
Quoted 11 lines

support/testing/conf/docker-compose.yml line 4

Quoted 6 lines

+++ b/support/testing/conf/docker-compose.yml

support/testing/conf/docker-compose.yml new file +4 -0
@@ -0,0 +1,4 @@
+version: '3'
+services:
+  busybox:
+    image: "busybox:latest"

Is conf/ really the right place for this ? Don't we have some other place to store the per-tests artifacts that go on the target ? I don't think we have a good organization for this at the moment, so this will do for now.

Quoted 3 lines

Indeed. We have a number of more-or-less random files under conf/, and then E.G. tests/package/test_python_*.py stores files for the target also under tests/package/sample_*.py

I obviously prefer the fixtures besides the test cases, since I always send test cases in this way, AFAICR.
I just never bothered to submit a patch changing all test cases to do so. We can choose one or another and standardize.

Quoted 3 lines

I can move the files under tests/package if you prefer, but I'm not sure if that is really cleaner.

If we decide to apply that series that Yann once submitted changing the test cases to each package directory, I think it would make more sense to keep the fixtures also in the same directory.

package/pkg1/Config.in
package/pkg1/pkg1.mk
package/pkg1/test_pkg1.py
package/pkg1/sample_pkg1.anything
package/pkg1/S50_init_script

Or something like this. I am not suggesting any naming convention.

Regards,
Ricardo

#8 lore raw

PATCH 1/1 support/testing: add docker / docker-compose tests

In reply to Peter Korsgaard
Quoted 11 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Peter" == Peter Korsgaard <[email protected]> writes:

Build for x86-64 as public containers in general are only available for x86-64. Docker needs a number of kernel options enabled, so use a custom kernel config based on the qemu one.

Docker needs entropy at startup, so enable the virtio-rng-pci device to expose entropy to the guest. The default RAM amount (128M) is not enough to run docker / docker-compose, so bump to 512MB.

Committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#9 lore raw

PATCH 1/1 support/testing: add docker / docker-compose tests

Peter,

Quoted 7 lines · Peter Korsgaard
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

"Peter" == Peter Korsgaard <[email protected]> writes:

Build for x86-64 as public containers in general are only available for x86-64. Docker needs a number of kernel options enabled, so use a custom kernel config based on the qemu one.

Quoted 7 lines
Quoted 5 lines

Docker needs entropy at startup, so enable the virtio-rng-pci device to expose entropy to the guest. The default RAM amount (128M) is not enough to run docker / docker-compose, so bump to 512MB.

Committed, thanks.

It's great to have a reference test case for docker!