Skip to contentSkip to conversation

.gitlab-ci: Reduce Docker images size

PATCH In review Ramotowski, Maciej 4 participants 8 messages Mar 5, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 .gitlab-ci: Reduce Docker images size

Make Dockerfile's more universal.

Reduce size of Ubuntu based images by removing package lists.

Add support for building with build cache and without --squash flag:

- Using apt-get update alone in a RUN statement causes caching issues and subsequent apt-get install instructions fail.

- When you run an image and generate a container, you add a new writable layer (the “container layer”) on top of the underlying layers. All changes made to the running container, such as writing new files, modifying existing files, and deleting files, are written to this thin writable container layer.

Cc Arkadiusz Hiler <[email protected]>
Cc Petri Latvala <[email protected]>
---
 Dockerfile.build-debian         | 39 +++++++---------
 Dockerfile.build-debian-arm64   | 68 ++++++++++++---------------
 Dockerfile.build-debian-armhf   | 67 ++++++++++++--------------
 Dockerfile.build-debian-minimal | 44 +++++++++--------
 Dockerfile.build-debian-mips    | 70 +++++++++++++--------------
 Dockerfile.build-fedora         | 83 ++++++++++++++++-----------------
 6 files changed, 172 insertions(+), 199 deletions(-)
Dockerfile.build-debian +17 -22
index b143a532..6cf7a1f0 100644
@@ -7,25 +7,20 @@ FROM $CI_REGISTRY_IMAGE/build-debian-minimal:commit-$CI_COMMIT_SHA
 # just few extra dependencies for building IGT with all the optional components
 # enabled
 
-RUN apt-get update
-RUN apt-get install -y \
-			libunwind-dev \
-			libgsl-dev \
-			libasound2-dev \
-			libxmlrpc-core-c3-dev \
-			libjson-c-dev \
-			libcurl4-openssl-dev \
-			python-docutils \
-			valgrind \
-			peg \
-			libdrm-intel1
-
-# autotools build deps
-RUN apt-get install -y \
-			autoconf \
-			automake \
-			xutils-dev \
-			libtool \
-			make
-
-RUN apt-get clean
+RUN apt-get update && apt-get install -y \
+    autoconf \
+    automake \
+    libasound2-dev \
+    libcurl4-openssl-dev \
+    libdrm-intel1 \
+    libgsl-dev \
+    libjson-c-dev \
+    libtool \
+    libunwind-dev \
+    libxmlrpc-core-c3-dev \
+    make \
+    peg \
+    python-docutils \
+    valgrind \
+    xutils-dev \
+ && rm -rf /var/lib/apt/lists/*
Dockerfile.build-debian-arm64 +31 -37

71 lines not on this page · raw message

Dockerfile.build-debian-armhf +31 -36

70 lines not on this page · raw message

Dockerfile.build-debian-minimal +21 -23

46 lines not on this page · raw message

Dockerfile.build-debian-mips +32 -38

74 lines not on this page · raw message

Dockerfile.build-fedora +40 -43

90 lines not on this page · raw message

Signature · 5 lines
2.17.1
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#1 lore raw

[igt-dev] ✗ Fi.CI.BUILD: failure for .gitlab-ci: Reduce Docker images size (rev2)

== Series Details ==

== Series Details ==

Series: .gitlab-ci: Reduce Docker images size (rev2)
URL : https://patchwork.freedesktop.org/series/74003/
State : failure

== Summary ==

Applying: .gitlab-ci: Reduce Docker images size
Using index info to reconstruct a base tree...
Patch failed at 0001 .gitlab-ci: Reduce Docker images size When you have resolved this problem, run "git am --continue". If you prefer to skip this patch, run "git am --skip" instead. To restore the original branch and stop patching, run "git am --abort".

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#2 lore raw
In reply to Ramotowski, Maciej

In your mail header:
Content-Transfer-Encoding: base64

That's why it doesn't apply in CI, and git am here too cannot cope with the encoding. Did you use git send-email to send the patch? If not, please do.

Signature · 5 lines
Petri Latvala
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#3 lore raw

Yes I did use git send-email I did pre-check with --dry-run option it shows 8bit for encoding. Should I try to resend it?

-----Original Message-----
From: Latvala, Petri
Sent: Thursday, March 5, 2020 1:11 PM
To: Ramotowski, Maciej <[email protected]>
Cc: [email protected]; Hiler, Arkadiusz <[email protected]>
Subject: Re: [PATCH] .gitlab-ci: Reduce Docker images size

In your mail header:
Content-Transfer-Encoding: base64

That's why it doesn't apply in CI, and git am here too cannot cope with the encoding. Did you use git send-email to send the patch? If not, please do.

--
Petri Latvala
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#4 lore raw
Quoted 1 line · Maciej

Yes I did use git send-email I did pre-check with --dry-run option it shows 8bit for encoding. Should I try to resend it?

Please do, but recreate the patch after

git config --local format.subjectPrefix "PATCH i-g-t"

and send test mails to [email protected]

Signature · 5 lines
Petri Latvala
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#5 lore raw
In reply to Ramotowski, Maciej
Quoted 9 lines · Maciej

Yes I did use git send-email I did pre-check with --dry-run option it shows 8bit for encoding. Should I try to resend it?

-----Original Message-----
From: Latvala, Petri
Sent: Thursday, March 5, 2020 1:11 PM
To: Ramotowski, Maciej <[email protected]>
Cc: [email protected]; Hiler, Arkadiusz <[email protected]>
Subject: Re: [PATCH] .gitlab-ci: Reduce Docker images size

Quoted 5 lines

In your mail header:
Content-Transfer-Encoding: base64

That's why it doesn't apply in CI, and git am here too cannot cope with the encoding. Did you use git send-email to send the patch? If not, please do.

% wget https://patchwork.freedesktop.org/api/1.0/series/74003/revisions/2/mbox/ -O m.patch --2020-03-05 15:42:17-- https://patchwork.freedesktop.org/api/1.0/series/74003/revisions/2/mbox/
Resolving patchwork.freedesktop.org (patchwork.freedesktop.org)... 131.252.210.167, 2610:10:20:722:a800:ff:feee:56cf Connecting to patchwork.freedesktop.org (patchwork.freedesktop.org)|131.252.210.167|:443... connected. HTTP request sent, awaiting response... 200 OK
Length: 16478 (16K) [text/plain]
Saving to: ‘m.patch’

m.patch 100%[======================================================================================================================>] 16.09K 80.8KB/s in 0.2s

2020-03-05 15:42:19 (80.8 KB/s) - ‘m.patch’ saved [16478/16478]

% git am m.patch
Applying: .gitlab-ci: Reduce Docker images size
error: patch failed: Dockerfile.build-debian:7
error: Dockerfile.build-debian: patch does not apply
error: patch failed: Dockerfile.build-debian-arm64:1
error: Dockerfile.build-debian-arm64: patch does not apply
error: patch failed: Dockerfile.build-debian-armhf:1
error: Dockerfile.build-debian-armhf: patch does not apply
error: patch failed: Dockerfile.build-debian-minimal:1
error: Dockerfile.build-debian-minimal: patch does not apply
error: patch failed: Dockerfile.build-debian-mips:2
error: Dockerfile.build-debian-mips: patch does not apply
error: patch failed: Dockerfile.build-fedora:1
error: Dockerfile.build-fedora: patch does not apply
Patch failed at 0001 .gitlab-ci: Reduce Docker images size
hint: Use 'git am --show-current-patch' to see the failed patch When you have resolved this problem, run "git am --continue". If you prefer to skip this patch, run "git am --skip" instead. To restore the original branch and stop patching, run "git am --abort".

So it's not base64. Let me see what actually goes wrong. You can get the unencoded patch here: https://patchwork.freedesktop.org/patch/356524/raw/

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#6 lore raw
Quoted 48 lines · Arkadiusz Hiler
Quoted 9 lines · Maciej

Yes I did use git send-email I did pre-check with --dry-run option it shows 8bit for encoding. Should I try to resend it?

-----Original Message-----
From: Latvala, Petri
Sent: Thursday, March 5, 2020 1:11 PM
To: Ramotowski, Maciej <[email protected]>
Cc: [email protected]; Hiler, Arkadiusz <[email protected]>
Subject: Re: [PATCH] .gitlab-ci: Reduce Docker images size

Quoted 5 lines

In your mail header:
Content-Transfer-Encoding: base64

That's why it doesn't apply in CI, and git am here too cannot cope with the encoding. Did you use git send-email to send the patch? If not, please do.

% wget https://patchwork.freedesktop.org/api/1.0/series/74003/revisions/2/mbox/ -O m.patch --2020-03-05 15:42:17-- https://patchwork.freedesktop.org/api/1.0/series/74003/revisions/2/mbox/
Resolving patchwork.freedesktop.org (patchwork.freedesktop.org)... 131.252.210.167, 2610:10:20:722:a800:ff:feee:56cf Connecting to patchwork.freedesktop.org (patchwork.freedesktop.org)|131.252.210.167|:443... connected. HTTP request sent, awaiting response... 200 OK
Length: 16478 (16K) [text/plain]
Saving to: ‘m.patch’

m.patch 100%[======================================================================================================================>] 16.09K 80.8KB/s in 0.2s

2020-03-05 15:42:19 (80.8 KB/s) - ‘m.patch’ saved [16478/16478]

% git am m.patch
Applying: .gitlab-ci: Reduce Docker images size
error: patch failed: Dockerfile.build-debian:7
error: Dockerfile.build-debian: patch does not apply
error: patch failed: Dockerfile.build-debian-arm64:1
error: Dockerfile.build-debian-arm64: patch does not apply
error: patch failed: Dockerfile.build-debian-armhf:1
error: Dockerfile.build-debian-armhf: patch does not apply
error: patch failed: Dockerfile.build-debian-minimal:1
error: Dockerfile.build-debian-minimal: patch does not apply
error: patch failed: Dockerfile.build-debian-mips:2
error: Dockerfile.build-debian-mips: patch does not apply
error: patch failed: Dockerfile.build-fedora:1
error: Dockerfile.build-fedora: patch does not apply
Patch failed at 0001 .gitlab-ci: Reduce Docker images size
hint: Use 'git am --show-current-patch' to see the failed patch When you have resolved this problem, run "git am --continue". If you prefer to skip this patch, run "git am --skip" instead. To restore the original branch and stop patching, run "git am --abort".

Quoted 1 line

So it's not base64.

Yep, my bad.

Quoted 2 lines

Let me see what actually goes wrong. You can get the unencoded patch here: https://patchwork.freedesktop.org/patch/356524/raw/

(Stripping trailing CRs from patch; use --binary to disable.) patching file Dockerfile.build-debian
(Stripping trailing CRs from patch; use --binary to disable.) patching file Dockerfile.build-debian-arm64 (Stripping trailing CRs from patch; use --binary to disable.) patching file Dockerfile.build-debian-armhf (Stripping trailing CRs from patch; use --binary to disable.) patching file Dockerfile.build-debian-minimal (Stripping trailing CRs from patch; use --binary to disable.) patching file Dockerfile.build-debian-mips (Stripping trailing CRs from patch; use --binary to disable.) patching file Dockerfile.build-fedora

Good old \r\n, eh?

Signature · 5 lines
Petri Latvala
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#7 lore raw
In reply to Arkadiusz Hiler
Quoted 48 lines · Arkadiusz Hiler
Quoted 9 lines · Maciej

Yes I did use git send-email I did pre-check with --dry-run option it shows 8bit for encoding. Should I try to resend it?

-----Original Message-----
From: Latvala, Petri
Sent: Thursday, March 5, 2020 1:11 PM
To: Ramotowski, Maciej <[email protected]>
Cc: [email protected]; Hiler, Arkadiusz <[email protected]>
Subject: Re: [PATCH] .gitlab-ci: Reduce Docker images size

Quoted 5 lines

In your mail header:
Content-Transfer-Encoding: base64

That's why it doesn't apply in CI, and git am here too cannot cope with the encoding. Did you use git send-email to send the patch? If not, please do.

% wget https://patchwork.freedesktop.org/api/1.0/series/74003/revisions/2/mbox/ -O m.patch --2020-03-05 15:42:17-- https://patchwork.freedesktop.org/api/1.0/series/74003/revisions/2/mbox/
Resolving patchwork.freedesktop.org (patchwork.freedesktop.org)... 131.252.210.167, 2610:10:20:722:a800:ff:feee:56cf Connecting to patchwork.freedesktop.org (patchwork.freedesktop.org)|131.252.210.167|:443... connected. HTTP request sent, awaiting response... 200 OK
Length: 16478 (16K) [text/plain]
Saving to: ‘m.patch’

m.patch 100%[======================================================================================================================>] 16.09K 80.8KB/s in 0.2s

2020-03-05 15:42:19 (80.8 KB/s) - ‘m.patch’ saved [16478/16478]

% git am m.patch
Applying: .gitlab-ci: Reduce Docker images size
error: patch failed: Dockerfile.build-debian:7
error: Dockerfile.build-debian: patch does not apply
error: patch failed: Dockerfile.build-debian-arm64:1
error: Dockerfile.build-debian-arm64: patch does not apply
error: patch failed: Dockerfile.build-debian-armhf:1
error: Dockerfile.build-debian-armhf: patch does not apply
error: patch failed: Dockerfile.build-debian-minimal:1
error: Dockerfile.build-debian-minimal: patch does not apply
error: patch failed: Dockerfile.build-debian-mips:2
error: Dockerfile.build-debian-mips: patch does not apply
error: patch failed: Dockerfile.build-fedora:1
error: Dockerfile.build-fedora: patch does not apply
Patch failed at 0001 .gitlab-ci: Reduce Docker images size
hint: Use 'git am --show-current-patch' to see the failed patch When you have resolved this problem, run "git am --continue". If you prefer to skip this patch, run "git am --skip" instead. To restore the original branch and stop patching, run "git am --abort".

Quoted 2 lines

So it's not base64. Let me see what actually goes wrong. You can get the unencoded patch here: https://patchwork.freedesktop.org/patch/356524/raw/

00000030 65 2e 62 75 69 6c 64 2d 64 65 62 69 61 6e 0d 0a |e.build-debian..|

                                                     ^^^^^ CRLF

This makes the context to not match which makes this patch to not apply.

Please make sure that when you are sending emails your setup does not convert the line endings :-)

Signature · 6 lines
Cheers,
Arek
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev