Skip to contentSkip to conversation

Docker Improvements

PATCH In review Guilherme Campos Camargo 3 participants 22 messages Jan 26, 2018 – Jan 31, 2018

lore
Share & tools
lore
#0 lore raw

cover Docker Improvements

Hello, everyone

This series of patches implements a few improvements on the Docker infrastructure with the ultimate goal of allowing an image to be shared amoung different users.

One of the benefits of this approach is that it allows the build-from-scratch to be optional, providing the user much faster set-up times.

The changes involve, in short:

* Reorganization of the Dockerfile aiming to reduce the number of layers present in the resulting image (by reducing the number of RUN/COPY statements) and to reduce cache misses by moving components that change less frequently to the beginning of the Dockerfile.

* Remove persistenct of proxy and other configurations that might result in conflicts to the user of the image (that's not using it at the same host where the image was built).

We tried to provide an in-depth reasoning for each one of the proposed changes in the commit messages and we'll be glad to hear your inputs about them.

Thanks

--
Guilherme

ProFUSION Embedded Systems

Guilherme Campos Camargo (9):
Remove unused variables from docker build/create/run scripts Refactor preinst and apt-get/pip install
Use curl for downloading Jenkins
Move configuration and jenkins plugin installation to separate scripts Remove jenkins user created during build time with host's uid/gid Cleanup proxy
Move dpkg-reconfigure dash to first RUN instruction Refactor plugin installation script
Set depth to "one" and download just a single-branch on git clones

 Dockerfile                                         | 215 +++++++++++----------
 .../apt/sources/fuego-debian-jessie.list           |   2 +
 frontend-install/entrypoint.sh                     |  13 ++
 frontend-install/{ => setup/jenkins}/config.xml    |   0
 .../jenkins.model.JenkinsLocationConfiguration.xml |   0
 .../setup/jenkins/map-jenkins-uid-to-host.sh       |  12 ++
 .../plugins/flot-plotter-plugin/.gitignore         |   0
 .../jenkins}/plugins/flot-plotter-plugin/flot.hpi  | Bin
 .../jenkins}/plugins/flot-plotter-plugin/pom.xml   |   0
 .../src/main/java/flotile/ChartAction.java         |   0
 .../src/main/java/flotile/FlotPublisher.java       |   0
 .../flotile/ChartAction/floatingBox.jelly          |   0
 .../src/main/resources/index.jelly                 |   0
 .../src/main/webapp/flot/.copyarea.db              |   0
 .../src/main/webapp/flot/API.txt                   |   0
 .../src/main/webapp/flot/FAQ.txt                   |   0
 .../src/main/webapp/flot/LICENSE.txt               |   0
 .../src/main/webapp/flot/PLUGINS.txt               |   0
 .../src/main/webapp/flot/README.txt                |   0
 .../src/main/webapp/flot/base64.js                 |   0
 .../src/main/webapp/flot/canvas2image.js           |   0
 .../src/main/webapp/flot/excanvas.min.js           |   0
 .../main/webapp/flot/jquery.colorhelpers.min.js    |   0
 .../main/webapp/flot/jquery.flot.crosshair.min.js  |   0
 .../webapp/flot/jquery.flot.fillbetween.min.js     |   0
 .../src/main/webapp/flot/jquery.flot.image.min.js  |   0
 .../src/main/webapp/flot/jquery.flot.min.js        |   0
 .../main/webapp/flot/jquery.flot.navigate.min.js   |   0
 .../src/main/webapp/flot/jquery.flot.pie.min.js    |   0
 .../src/main/webapp/flot/jquery.flot.resize.min.js |   0
 .../main/webapp/flot/jquery.flot.selection.min.js  |   0
 .../src/main/webapp/flot/jquery.flot.stack.min.js  |   0
 .../src/main/webapp/flot/jquery.flot.symbol.min.js |   0
 .../main/webapp/flot/jquery.flot.threshold.min.js  |   0
 .../src/main/webapp/flot/jquery.min.js             |   0
 .../src/main/webapp/flot/mod.js                    |   0
 .../src/main/webapp/flot/my.css                    |   0
 .../src/main/webapp/help-globalConfig.html         |   0
 frontend-install/setup/jenkins/plugins/install.sh  |  43 +++++
 .../setup/jenkins/set-java-args-proxy.sh           |   7 +
 frontend-install/setup/jenkins/setup.sh            |  11 ++
 frontend-install/setup/lava/setup.sh               |  11 ++
 frontend-install/setup/serial/setup-serio.sh       |   8 +
 frontend-install/setup/serial/setup-serlogin.sh    |   6 +
 frontend-install/setup/serial/setup-ttc.sh         |   7 +
 fuego-host-scripts/docker-build-image.sh           |  21 +-
 fuego-host-scripts/docker-create-container.sh      |   2 +
 .../docker-create-usb-privileged-container.sh      |   2 +
 fuego-host-scripts/docker-remove-container.sh      |   8 -
 fuego-host-scripts/docker-remove-image.sh          |   9 +-
 fuego-host-scripts/docker-start-container.sh       |   8 -
 fuego-host-scripts/jenkins-map-uid.sh              |  18 ++
 52 files changed, 258 insertions(+), 145 deletions(-)
 create mode 100644 frontend-install/apt/sources/fuego-debian-jessie.list
 create mode 100755 frontend-install/entrypoint.sh
 rename frontend-install/{ => setup/jenkins}/config.xml (100%)
 rename frontend-install/{ => setup/jenkins}/jenkins.model.JenkinsLocationConfiguration.xml (100%)
 create mode 100755 frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/.gitignore (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/flot.hpi (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/pom.xml (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/java/flotile/ChartAction.java (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/java/flotile/FlotPublisher.java (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/resources/index.jelly (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/.copyarea.db (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/API.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/FAQ.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/LICENSE.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/PLUGINS.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/README.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/base64.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/canvas2image.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/excanvas.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.colorhelpers.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.image.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.navigate.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.pie.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.resize.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.selection.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.stack.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.symbol.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.threshold.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/mod.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/my.css (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/help-globalConfig.html (100%)
 create mode 100755 frontend-install/setup/jenkins/plugins/install.sh
 create mode 100755 frontend-install/setup/jenkins/set-java-args-proxy.sh
 create mode 100755 frontend-install/setup/jenkins/setup.sh
 create mode 100755 frontend-install/setup/lava/setup.sh
 create mode 100755 frontend-install/setup/serial/setup-serio.sh
 create mode 100755 frontend-install/setup/serial/setup-serlogin.sh
 create mode 100755 frontend-install/setup/serial/setup-ttc.sh
 create mode 100755 fuego-host-scripts/jenkins-map-uid.sh
Signature · 1 line
2.15.1
#1 lore raw

PATCH 1/9 Remove unused variables from docker build/create/run scripts

The DIR variable is only being used by the CREATE scripts. There's no reason to keep it in the BUILD/REMOVE/START scripts as well.

---
 fuego-host-scripts/docker-build-image.sh      | 9 +--------
 fuego-host-scripts/docker-remove-container.sh | 8 --------
 fuego-host-scripts/docker-remove-image.sh     | 9 +--------
 fuego-host-scripts/docker-start-container.sh  | 8 --------
 4 files changed, 2 insertions(+), 32 deletions(-)
fuego-host-scripts/docker-build-image.sh +1 -8
index a197dc5..7c11b11 100755
@@ -1,12 +1,5 @@
+#!/bin/bash
 # $1 - name for the docker image (default: fuego)
-SOURCE="${BASH_SOURCE[0]}"
-while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
-  DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-  SOURCE="$(readlink "$SOURCE")"
-  [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
-done
-DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-
 DOCKERIMAGE=${1:-fuego}
 
 if [ "$(id -u)" == "0" ]; then
fuego-host-scripts/docker-remove-container.sh +0 -8
index 2aae6a9..00c7ac9 100755
@@ -1,13 +1,5 @@
 #!/bin/bash
 # $1 - name for the docker container (default: fuego-container)
-SOURCE="${BASH_SOURCE[0]}"
-while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
-  DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-  SOURCE="$(readlink "$SOURCE")"
-  [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
-done
-DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-
 DOCKERCONTAINER=${1:-fuego-container}
 
 sudo docker rm ${DOCKERCONTAINER}
fuego-host-scripts/docker-remove-image.sh +1 -8
index 91dbda7..4c24684 100755
@@ -1,12 +1,5 @@
+#!/bin/bash
 # $1 - name for the docker image (default: fuego)
-SOURCE="${BASH_SOURCE[0]}"
-while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
-  DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-  SOURCE="$(readlink "$SOURCE")"
-  [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
-done
-DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-
 DOCKERIMAGE=${1:-fuego}
 
 sudo docker rmi ${DOCKERIMAGE}
fuego-host-scripts/docker-start-container.sh +0 -8
index d9de85a..d0cee1a 100755
@@ -1,13 +1,5 @@
 #!/bin/bash
 # $1 - name for the docker container (default: fuego-container)
-SOURCE="${BASH_SOURCE[0]}"
-while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
-  DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-  SOURCE="$(readlink "$SOURCE")"
-  [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
-done
-DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-
 DOCKERCONTAINER=${1:-fuego-container}
 
 echo "Starting Fuego container (${DOCKERCONTAINER})"
Signature · 1 line
2.15.1
#2 lore raw

Re: [Fuego] [PATCH 1/9] Remove unused variables from docker build/create/run scripts

Quoted 105 lines

-----Original Message-----
From: Guilherme Campos Camargo

The DIR variable is only being used by the CREATE scripts. There's no reason to keep it in the BUILD/REMOVE/START scripts as well.

---
 fuego-host-scripts/docker-build-image.sh      | 9 +--------
 fuego-host-scripts/docker-remove-container.sh | 8 --------
 fuego-host-scripts/docker-remove-image.sh     | 9 +--------
 fuego-host-scripts/docker-start-container.sh  | 8 --------
 4 files changed, 2 insertions(+), 32 deletions(-)
fuego-host- renamed from fuego-host-scripts/docker-build-image.sh +0 -0

scripts/docker-build-image.sh
index a197dc5..7c11b11 100755

fuego-host-scripts/docker-build-image.sh +1 -2
@@ -1,12 +1,5 @@
+#!/bin/bash
 # $1 - name for the docker image (default: fuego)
-SOURCE="${BASH_SOURCE[0]}"
-while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a

symlink

fuego-host-scripts/docker-build-image.sh +0 -3
-  DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-  SOURCE="$(readlink "$SOURCE")"
-  [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a

relative symlink, we need to resolve it relative to the path where the symlink file was located

fuego-host-scripts/docker-build-image.sh +0 -3
-done
-DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-
 DOCKERIMAGE=${1:-fuego}
 
 if [ "$(id -u)" == "0" ]; then
fuego-host- renamed from fuego-host-scripts/docker-remove-container.sh +0 -0

scripts/docker-remove-container.sh
index 2aae6a9..00c7ac9 100755

fuego-host-scripts/docker-remove-container.sh +0 -2
@@ -1,13 +1,5 @@
 #!/bin/bash
 # $1 - name for the docker container (default: fuego-container)
-SOURCE="${BASH_SOURCE[0]}"
-while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a

symlink

fuego-host-scripts/docker-remove-container.sh +0 -3
-  DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-  SOURCE="$(readlink "$SOURCE")"
-  [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a

relative symlink, we need to resolve it relative to the path where the symlink file was located

fuego-host-scripts/docker-remove-container.sh +0 -3
-done
-DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-
 DOCKERCONTAINER=${1:-fuego-container}
 
 sudo docker rm ${DOCKERCONTAINER}
fuego-host- renamed from fuego-host-scripts/docker-remove-image.sh +0 -0

scripts/docker-remove-image.sh
index 91dbda7..4c24684 100755

fuego-host-scripts/docker-remove-image.sh +1 -2
@@ -1,12 +1,5 @@
+#!/bin/bash
 # $1 - name for the docker image (default: fuego)
-SOURCE="${BASH_SOURCE[0]}"
-while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a

symlink

fuego-host-scripts/docker-remove-image.sh +0 -3
-  DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-  SOURCE="$(readlink "$SOURCE")"
-  [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a

relative symlink, we need to resolve it relative to the path where the symlink file was located

fuego-host-scripts/docker-remove-image.sh +0 -3
-done
-DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-
 DOCKERIMAGE=${1:-fuego}
 
 sudo docker rmi ${DOCKERIMAGE}
fuego-host- renamed from fuego-host-scripts/docker-start-container.sh +0 -0

scripts/docker-start-container.sh
index d9de85a..d0cee1a 100755

fuego-host-scripts/docker-start-container.sh +0 -2
@@ -1,13 +1,5 @@
 #!/bin/bash
 # $1 - name for the docker container (default: fuego-container)
-SOURCE="${BASH_SOURCE[0]}"
-while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a

symlink

fuego-host-scripts/docker-start-container.sh +0 -3
-  DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-  SOURCE="$(readlink "$SOURCE")"
-  [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a

relative symlink, we need to resolve it relative to the path where the symlink file was located

fuego-host-scripts/docker-start-container.sh +0 -4
-done
-DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-
 DOCKERCONTAINER=${1:-fuego-container}
 
 echo "Starting Fuego container (${DOCKERCONTAINER})"
--
Quoted 1 line

2.15.1

Nice cleanup. Thanks.
Applied.
-- Tim

#3 lore raw

PATCH 2/9 Refactor preinst and apt-get/pip install

In reply to Guilherme Campos Camargo

All base-image related apt-get commands (including pre-install config, as well as the package install) have been put together in the same RUN instruction.

The same has been done with the LAVA-related apt-get commands on this patch. They can't be merged because the requirements are using a specific key that needs to be downloaded with curl (that's installed in the previous step. Additionally, LAVA apt-get commands have been moved up in the Dockerfile, allowing that RUN instruction to be invalidated fewer times, because it does not depend on the files that have been copied from the host.

We have also moved apt sources to an external .list file, that's copied from the host to the container during build.

---
 Dockerfile                                         | 116 ++++++++++++++++-----
 .../apt/sources/fuego-debian-jessie.list           |   2 +
 2 files changed, 92 insertions(+), 26 deletions(-)
 create mode 100644 frontend-install/apt/sources/fuego-debian-jessie.list
Dockerfile +90 -26
index f34fe4b..b8d78e6 100644
@@ -19,25 +19,96 @@ ENV https_proxy ${HTTP_PROXY}
 # ==============================================================================
 
 WORKDIR /
-RUN echo deb http://httpredir.debian.org/debian jessie main non-free > /etc/apt/sources.list
-RUN echo deb http://httpredir.debian.org/debian jessie-updates main non-free >> /etc/apt/sources.list
-RUN echo deb http://security.debian.org/ jessie/updates main >> /etc/apt/sources.list
-RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
-RUN DEBIAN_FRONTEND=noninteractive apt-get update && apt-get -yV install \
-	apt-utils daemon gcc make cmake python-paramiko python-lxml python-simplejson \
-	python-matplotlib python-serial python-yaml python-openpyxl python-requests \
-	libtool xmlstarlet autoconf automake rsync openjdk-7-jre openjdk-7-jdk iperf \
-	netperf netpipe-tcp sshpass wget git diffstat sudo net-tools vim curl \
-	inotify-tools g++ bzip2 bc libaio-dev gettext pkg-config libglib2.0-dev \
-	time python-pip python-xmltodict at minicom lzop bsdmainutils u-boot-tools \
-	mc netcat lava-tool openssh-server python-parsedatetime \
-	libsdl1.2-dev libcairo2-dev libxmu-dev libxmuu-dev
-
-RUN pip install python-jenkins==0.4.14
-RUN pip install filelock
-RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash'
-RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
-RUN if [ -n "$HTTP_PROXY" ]; then echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; fi
+COPY frontend-install/apt/sources/fuego-debian-jessie.list \
+        /etc/apt/sources.list.d/fuego-debian-jessie.list
+RUN if [ -n "$HTTP_PROXY" ]; then \
+        echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; \
+    fi && \
+    DEBIAN_FRONTEND=noninteractive apt-get update && \
+    apt-get -yV install \
+        apt-utils \
+        at \
+        autoconf \
+        automake \
+        bc \
+        bsdmainutils \
+        bzip2 \
+        cmake \
+        curl \
+        daemon \
+        diffstat \
+        g++ \
+        gcc \
+        gettext \
+        git \
+        inotify-tools \
+        iperf \
+        lava-tool \
+        libaio-dev \
+        libcairo2-dev \
+        libglib2.0-dev \
+        libsdl1.2-dev \
+        libtool \
+        libxmu-dev \
+        libxmuu-dev \
+        lzop \
+        make \
+        mc \
+        minicom \
+        net-tools \
+        netcat \
+        netperf \
+        netpipe-tcp \
+        openjdk-7-jdk \
+        openjdk-7-jre \
+        openssh-server \
+        pkg-config \
+        python-lxml \
+        python-matplotlib \
+        python-openpyxl \
+        python-paramiko \
+        python-parsedatetime \
+        python-pip \
+        python-requests \
+        python-serial \
+        python-simplejson \
+        python-xmltodict \
+        python-yaml \
+        rsync \
+        sshpass \
+        sudo \
+        time \
+        u-boot-tools \
+        vim \
+        wget \
+        xmlstarlet && \
+    rm -rf /var/lib/apt/lists/*
+
+RUN echo dash dash/sh boolean false | debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash && \
+    if [ -n "$HTTP_PROXY" ]; then \
+        echo "use_proxy = on" >> /etc/wgetrc; \
+        echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; \
+    fi
+
+RUN pip install \
+        filelock \
+        python-jenkins==0.4.14
+
+# TODO: Move toolchain-related instalation steps to a derivate image, like fuego:${version}-arhmhf
+RUN echo deb http://emdebian.org/tools/debian/ jessie main > /etc/apt/sources.list.d/crosstools.list && \
+    curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | apt-key add - && \
+    dpkg --add-architecture armhf && \
+    DEBIAN_FRONTEND=noninteractive apt-get update && \
+    apt-get -yV install \
+        binutils-arm-linux-gnueabihf \
+        cpp-arm-linux-gnueabihf \
+        crossbuild-essential-armhf \
+        gcc-arm-linux-gnueabihf && \
+    rm -rf /var/lib/apt/lists/*
+
+RUN pip install \
+        filelock \
+        python-jenkins==0.4.14
 
 # ==============================================================================
 # Install Jenkins with the same UID/GID as the host user
@@ -125,13 +196,6 @@ RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
 #RUN echo "fuego-create-node --board raspberrypi3" >> /root/firststart.sh
 #RUN echo "fuego-create-jobs --board raspberrypi3 --testplan testplan_docker --distrib nosyslogd.dist" >> /root/firststart.sh
 
-RUN echo "deb http://emdebian.org/tools/debian/ jessie main" > /etc/apt/sources.list.d/crosstools.list
-RUN dpkg --add-architecture armhf
-RUN curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | sudo apt-key add -
-RUN DEBIAN_FRONTEND=noninteractive apt-get update
-RUN DEBIAN_FRONTEND=noninteractive apt-get -yV install crossbuild-essential-armhf cpp-arm-linux-gnueabihf gcc-arm-linux-gnueabihf binutils-arm-linux-gnueabihf
-
-
 # ==============================================================================
 # Setup startup command
 # ==============================================================================
frontend-install/apt/sources/fuego-debian-jessie.list new file +2 -0
new file mode 100644
index 0000000..d224f15
@@ -0,0 +1,2 @@
+deb http://httpredir.debian.org/debian jessie main non-free
+deb http://httpredir.debian.org/debian jessie-updates main non-free
Signature · 1 line
2.15.1
#4 lore raw

Re: [Fuego] [PATCH 2/9] Refactor preinst and apt-get/pip install

Quoted 7 lines

-----Original Message-----
From: Guilherme Campos Camargo
All base-image related apt-get commands (including pre-install config, as well as the package install) have been put together in the same RUN instruction.

The same has been done with the LAVA-related apt-get commands on this

Do you mean Fuego-related?

Quoted 3 lines

patch. They can't be merged because the requirements are using a specific key that needs to be downloaded with curl (that's installed in the previous step. Additionally, LAVA apt-get commands have been moved

Another reference to LAVA?

Quoted 79 lines

up in the Dockerfile, allowing that RUN instruction to be invalidated fewer times, because it does not depend on the files that have been copied from the host.

We have also moved apt sources to an external .list file, that's copied from the host to the container during build.

---
 Dockerfile                                         | 116 ++++++++++++++++-----
 .../apt/sources/fuego-debian-jessie.list           |   2 +
 2 files changed, 92 insertions(+), 26 deletions(-)
 create mode 100644 frontend-install/apt/sources/fuego-debian-jessie.list
Dockerfile +0 -0
index f34fe4b..b8d78e6 100644
@@ -19,25 +19,96 @@ ENV https_proxy ${HTTP_PROXY}
 #

==========================================================
====================

WORKDIR /
-RUN echo deb http://httpredir.debian.org/debian jessie main non-free > /etc/apt/sources.list
-RUN echo deb http://httpredir.debian.org/debian jessie-updates main non- free >> /etc/apt/sources.list
-RUN echo deb http://security.debian.org/ jessie/updates main >> /etc/apt/sources.list
-RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
-RUN DEBIAN_FRONTEND=noninteractive apt-get update && apt-get -yV install \

Dockerfile +0 -1
-	apt-utils daemon gcc make cmake python-paramiko python-lxml

python-simplejson \

Dockerfile +0 -1
-	python-matplotlib python-serial python-yaml python-openpyxl

python-requests \

Dockerfile +0 -1
-	libtool xmlstarlet autoconf automake rsync openjdk-7-jre openjdk-7-

jdk iperf \

Dockerfile +0 -2
-	netperf netpipe-tcp sshpass wget git diffstat sudo net-tools vim curl \
-	inotify-tools g++ bzip2 bc libaio-dev gettext pkg-config libglib2.0-dev

\

Dockerfile +0 -1
-	time python-pip python-xmltodict at minicom lzop bsdmainutils u-

boot-tools \

Dockerfile +0 -6
-	mc netcat lava-tool openssh-server python-parsedatetime \
-	libsdl1.2-dev libcairo2-dev libxmu-dev libxmuu-dev
-
-RUN pip install python-jenkins==0.4.14
-RUN pip install filelock
-RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-

selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash'

Dockerfile +0 -2
-RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
-RUN if [ -n "$HTTP_PROXY" ]; then echo -e

"http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; fi

Dockerfile +4 -0
+COPY frontend-install/apt/sources/fuego-debian-jessie.list \
+        /etc/apt/sources.list.d/fuego-debian-jessie.list
+RUN if [ -n "$HTTP_PROXY" ]; then \
+        echo 'Acquire::http::proxy "'$HTTP_PROXY'";' >

/etc/apt/apt.conf.d/80proxy; \

Dockerfile +18 -0
+    fi && \
+    DEBIAN_FRONTEND=noninteractive apt-get update && \
+    apt-get -yV install \
+        apt-utils \
+        at \
+        autoconf \
+        automake \
+        bc \
+        bsdmainutils \
+        bzip2 \
+        cmake \
+        curl \
+        daemon \
+        diffstat \
+        g++ \
+        gcc \
+        gettext \
+        git \

Dockerfile ENV https_proxy ${HTTP_PROXY} · line 53

Quoted 10 lines
Dockerfile +10 -0
+        inotify-tools \
+        iperf \
+        lava-tool \
+        libaio-dev \
+        libcairo2-dev \
+        libglib2.0-dev \
+        libsdl1.2-dev \
+        libtool \
+        libxmu-dev \
+        libxmuu-dev \

Most of the above libs are also toolchain packages, for x86_64 (the docker virtual board) and would benefit from being moved to a separate image creation step or derivative image.

Quoted 102 lines
Dockerfile +34 -0
+        lzop \
+        make \
+        mc \
+        minicom \
+        net-tools \
+        netcat \
+        netperf \
+        netpipe-tcp \
+        openjdk-7-jdk \
+        openjdk-7-jre \
+        openssh-server \
+        pkg-config \
+        python-lxml \
+        python-matplotlib \
+        python-openpyxl \
+        python-paramiko \
+        python-parsedatetime \
+        python-pip \
+        python-requests \
+        python-serial \
+        python-simplejson \
+        python-xmltodict \
+        python-yaml \
+        rsync \
+        sshpass \
+        sudo \
+        time \
+        u-boot-tools \
+        vim \
+        wget \
+        xmlstarlet && \
+    rm -rf /var/lib/apt/lists/*
+
+RUN echo dash dash/sh boolean false | debconf-set-selections ;

DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash && \

Dockerfile +3 -0
+    if [ -n "$HTTP_PROXY" ]; then \
+        echo "use_proxy = on" >> /etc/wgetrc; \
+        echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >>

/etc/environment; \

Dockerfile +7 -0
+    fi
+
+RUN pip install \
+        filelock \
+        python-jenkins==0.4.14
+
+# TODO: Move toolchain-related instalation steps to a derivate image, like

fuego:${version}-arhmhf
+RUN echo deb http://emdebian.org/tools/debian/ jessie main > /etc/apt/sources.list.d/crosstools.list && \
+ curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key
| apt-key add - && \

Dockerfile +12 -0
+    dpkg --add-architecture armhf && \
+    DEBIAN_FRONTEND=noninteractive apt-get update && \
+    apt-get -yV install \
+        binutils-arm-linux-gnueabihf \
+        cpp-arm-linux-gnueabihf \
+        crossbuild-essential-armhf \
+        gcc-arm-linux-gnueabihf && \
+    rm -rf /var/lib/apt/lists/*
+
+RUN pip install \
+        filelock \
+        python-jenkins==0.4.14
 
 #

==========================================================
====================
# Install Jenkins with the same UID/GID as the host user

Dockerfile +0 -0
@@ -125,13 +196,6 @@ RUN ln -s /fuego-ro/scripts/fuego-lava-target-

teardown /usr/local/bin
#RUN echo "fuego-create-node --board raspberrypi3" >> /root/firststart.sh #RUN echo "fuego-create-jobs --board raspberrypi3 --testplan
testplan_docker --distrib nosyslogd.dist" >> /root/firststart.sh

-RUN echo "deb http://emdebian.org/tools/debian/ jessie main" > /etc/apt/sources.list.d/crosstools.list

Dockerfile +0 -2
-RUN dpkg --add-architecture armhf
-RUN curl http://emdebian.org/tools/debian/emdebian-toolchain-

archive.key | sudo apt-key add -

Dockerfile +0 -2
-RUN DEBIAN_FRONTEND=noninteractive apt-get update
-RUN DEBIAN_FRONTEND=noninteractive apt-get -yV install crossbuild-

essential-armhf cpp-arm-linux-gnueabihf gcc-arm-linux-gnueabihf binutils- arm-linux-gnueabihf

Dockerfile +0 -2
-
-
 #

==========================================================
====================
# Setup startup command
#
==========================================================
====================

frontend- renamed from frontend-install/apt/sources/fuego-debian-jessie.list +0 -0

install/apt/sources/fuego-debian-jessie.list
new file mode 100644
index 0000000..d224f15

frontend-install/apt/sources/fuego-debian-jessie.list new file +2 -1
@@ -0,0 +1,2 @@
+deb http://httpredir.debian.org/debian jessie main non-free
+deb http://httpredir.debian.org/debian jessie-updates main non-free
--
Quoted 1 line

2.15.1

On the whole, it looks OK. I've applied it as is (with, what I believe are a few commit description errors).
-- Tim

#5 lore raw

Re: [Fuego] [PATCH 2/9] Refactor preinst and apt-get/pip install

Quoted 6 lines · On Tue, Jan 30, 2018 at 5:22 PM, <[email protected]> wrote:
Quoted 6 lines

-----Original Message-----
From: Guilherme Campos Camargo
All base-image related apt-get commands (including pre-install config, as well as the package install) have been put together in the same RUN instruction.

Quoted 7 lines
Quoted 1 line

The same has been done with the LAVA-related apt-get commands on this

Do you mean Fuego-related?

Quoted 3 lines

patch. They can't be merged because the requirements are using a specific key that needs to be downloaded with curl (that's installed in the previous step. Additionally, LAVA apt-get commands have been moved

Another reference to LAVA?

ops, and I missed those in my review as well.

...

Quoted 27 lines
Quoted 17 lines

apt-get update && \

Dockerfile +16 -0
+    apt-get -yV install \
+        apt-utils \
+        at \
+        autoconf \
+        automake \
+        bc \
+        bsdmainutils \
+        bzip2 \
+        cmake \
+        curl \
+        daemon \
+        diffstat \
+        g++ \
+        gcc \
+        gettext \
+        git \

Dockerfile ENV https_proxy ${HTTP_PROXY} · line 53

Quoted 10 lines
Dockerfile +10 -0
+        inotify-tools \
+        iperf \
+        lava-tool \
+        libaio-dev \
+        libcairo2-dev \
+        libglib2.0-dev \
+        libsdl1.2-dev \
+        libtool \
+        libxmu-dev \
+        libxmuu-dev \
Quoted 3 lines

Most of the above libs are also toolchain packages, for x86_64 (the docker virtual board) and would benefit from being moved to a separate image creation step or derivative image.

you mean those *-dev, cmake, autoconf, automake, gcc, g++, gettext and libtool? indeed makes sense to move these to separate image once we split the armhf.

Signature · 6 lines
Gustavo Sverzut Barbieri
http://profusion.mobi embedded systems
--------------------------------------
MSN, GTalk, FaceTime: [email protected]
Skype: gsbarbieri
Mobile: +55 (16) 99354-9890
#6 lore raw

Re: [Fuego] [PATCH 2/9] Refactor preinst and apt-get/pip install

Quoted 14 lines

-----Original Message-----
From: Gustavo Sverzut

Quoted 7 lines · On Tue, Jan 30, 2018 at 5:22 PM, <[email protected]> wrote:
Quoted 7 lines

-----Original Message-----
From: Guilherme Campos Camargo
All base-image related apt-get commands (including pre-install config, as well as the package install) have been put together in the same RUN instruction.

The same has been done with the LAVA-related apt-get commands on

this

Quoted 2 lines

Do you mean Fuego-related?

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines

patch. They can't be merged because the requirements are using a specific key that needs to be downloaded with curl (that's installed in the previous step. Additionally, LAVA apt-get commands have been

moved

Quoted 1 line

Another reference to LAVA?

ops, and I missed those in my review as well.

No problem. It's not like people look at commit messages a lot after the review ;-)

Quoted 30 lines

...

Quoted 27 lines
Quoted 17 lines

apt-get update && \

Dockerfile +16 -0
+    apt-get -yV install \
+        apt-utils \
+        at \
+        autoconf \
+        automake \
+        bc \
+        bsdmainutils \
+        bzip2 \
+        cmake \
+        curl \
+        daemon \
+        diffstat \
+        g++ \
+        gcc \
+        gettext \
+        git \

Dockerfile ENV https_proxy ${HTTP_PROXY} · line 53

Quoted 10 lines
Dockerfile +10 -0
+        inotify-tools \
+        iperf \
+        lava-tool \
+        libaio-dev \
+        libcairo2-dev \
+        libglib2.0-dev \
+        libsdl1.2-dev \
+        libtool \
+        libxmu-dev \
+        libxmuu-dev \
Quoted 1 line

Most of the above libs are also toolchain packages, for x86_64 (the docker

Quoted 8 lines

virtual board)

Quoted 1 line

and would benefit from being moved to a separate image creation step or

derivative

Quoted 1 line

image.

you mean those *-dev, cmake, autoconf, automake, gcc, g++, gettext and libtool? indeed makes sense to move these to separate image once we split the armhf.

Just the lib*-dev. and gcc and g++. cmake, autoconf, automake and libtool are non-architecture-specific parts of the toolchain, which would be used by any SDK used in the container, I believe. I'm not sure about gettext. I think it provides libraries or headers that would get compiled into binaries that go to the target, so likely it is x86_64-specific.

The only things that Fuego intends to compile (right now), are things for the target boards. That's not an absolute hard and fast rule. I could imagine some test program having both a host-side and target-side component that needed to be compiled. But I don't think we have any of those at the moment. ... (I just checked the Functional.commonAPI_* tests, which I thought might have both host and target compiled programs, but they apparently don't. So I'm pretty sure this is correct.)

In any event, the general rule would be that the SDK installation step should have the packages needed for compilation of programs intended to run on the target. docker is a special target, and we default to including the SDK for that included in the base image. But it would still be good to keep these in a separate list.
-- Tim

#7 lore raw

PATCH 3/9 Use curl for downloading Jenkins

In reply to Guilherme Campos Camargo

As curl had been used previously in the Dockerfile, let's make it our preferred tool for downloading resources in your scripts.

---
 Dockerfile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Dockerfile +1 -1
index b8d78e6..3dbae0f 100644
@@ -125,7 +125,7 @@ ENV JENKINS_HOME=/var/lib/jenkins
 
 RUN groupadd -g ${gid} ${group} \
 	&& useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
-RUN wget -nv ${JENKINS_URL}
+RUN curl -L -O ${JENKINS_URL}
 RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
 RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb
 RUN rm jenkins_${JENKINS_VERSION}_all.deb
Signature · 1 line
2.15.1
#8 lore raw

Re: [Fuego] [PATCH 3/9] Use curl for downloading Jenkins

OK - looks good. Applied.
-- Tim

Quoted 38 lines

-----Original Message-----
From: [email protected] [mailto:fuego- [email protected]] On Behalf Of Guilherme Campos Camargo
Sent: Friday, January 26, 2018 9:36 AM
To: [email protected]
Subject: [Fuego] [PATCH 3/9] Use curl for downloading Jenkins

As curl had been used previously in the Dockerfile, let's make it our preferred tool for downloading resources in your scripts.

---
 Dockerfile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Dockerfile +1 -1
index b8d78e6..3dbae0f 100644
@@ -125,7 +125,7 @@ ENV JENKINS_HOME=/var/lib/jenkins
 
 RUN groupadd -g ${gid} ${group} \
 	&& useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -
 s /bin/bash ${user}
-RUN wget -nv ${JENKINS_URL}
+RUN curl -L -O ${JENKINS_URL}
 RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" |

sha1sum -c -
RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb
RUN rm jenkins_${JENKINS_VERSION}_all.deb
--
2.15.1

_______________________________________________
Fuego mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/fuego

#9 lore raw

PATCH 4/9 Move configuration and jenkins plugin installation to separate scripts

In reply to Guilherme Campos Camargo

In order to simplify the structure of the Dockerfile and optimize build times.

---
 Dockerfile                                         |  96 ++++++---------------
 frontend-install/{ => setup/jenkins}/config.xml    |   0
 .../jenkins.model.JenkinsLocationConfiguration.xml |   0
 .../plugins/flot-plotter-plugin/.gitignore         |   0
 .../jenkins}/plugins/flot-plotter-plugin/flot.hpi  | Bin
 .../jenkins}/plugins/flot-plotter-plugin/pom.xml   |   0
 .../src/main/java/flotile/ChartAction.java         |   0
 .../src/main/java/flotile/FlotPublisher.java       |   0
 .../flotile/ChartAction/floatingBox.jelly          |   0
 .../src/main/resources/index.jelly                 |   0
 .../src/main/webapp/flot/.copyarea.db              |   0
 .../src/main/webapp/flot/API.txt                   |   0
 .../src/main/webapp/flot/FAQ.txt                   |   0
 .../src/main/webapp/flot/LICENSE.txt               |   0
 .../src/main/webapp/flot/PLUGINS.txt               |   0
 .../src/main/webapp/flot/README.txt                |   0
 .../src/main/webapp/flot/base64.js                 |   0
 .../src/main/webapp/flot/canvas2image.js           |   0
 .../src/main/webapp/flot/excanvas.min.js           |   0
 .../main/webapp/flot/jquery.colorhelpers.min.js    |   0
 .../main/webapp/flot/jquery.flot.crosshair.min.js  |   0
 .../webapp/flot/jquery.flot.fillbetween.min.js     |   0
 .../src/main/webapp/flot/jquery.flot.image.min.js  |   0
 .../src/main/webapp/flot/jquery.flot.min.js        |   0
 .../main/webapp/flot/jquery.flot.navigate.min.js   |   0
 .../src/main/webapp/flot/jquery.flot.pie.min.js    |   0
 .../src/main/webapp/flot/jquery.flot.resize.min.js |   0
 .../main/webapp/flot/jquery.flot.selection.min.js  |   0
 .../src/main/webapp/flot/jquery.flot.stack.min.js  |   0
 .../src/main/webapp/flot/jquery.flot.symbol.min.js |   0
 .../main/webapp/flot/jquery.flot.threshold.min.js  |   0
 .../src/main/webapp/flot/jquery.min.js             |   0
 .../src/main/webapp/flot/mod.js                    |   0
 .../src/main/webapp/flot/my.css                    |   0
 .../src/main/webapp/help-globalConfig.html         |   0
 frontend-install/setup/jenkins/plugins/install.sh  |  21 +++++
 frontend-install/setup/jenkins/setup.sh            |  26 ++++++
 frontend-install/setup/lava/setup.sh               |  11 +++
 frontend-install/setup/serial/setup-serio.sh       |   8 ++
 frontend-install/setup/serial/setup-serlogin.sh    |   6 ++
 frontend-install/setup/serial/setup-ttc.sh         |   7 ++
 41 files changed, 105 insertions(+), 70 deletions(-)
 rename frontend-install/{ => setup/jenkins}/config.xml (100%)
 rename frontend-install/{ => setup/jenkins}/jenkins.model.JenkinsLocationConfiguration.xml (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/.gitignore (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/flot.hpi (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/pom.xml (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/java/flotile/ChartAction.java (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/java/flotile/FlotPublisher.java (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/resources/index.jelly (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/.copyarea.db (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/API.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/FAQ.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/LICENSE.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/PLUGINS.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/README.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/base64.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/canvas2image.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/excanvas.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.colorhelpers.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.image.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.navigate.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.pie.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.resize.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.selection.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.stack.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.symbol.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.threshold.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/mod.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/my.css (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/help-globalConfig.html (100%)
 create mode 100755 frontend-install/setup/jenkins/plugins/install.sh
 create mode 100755 frontend-install/setup/jenkins/setup.sh
 create mode 100755 frontend-install/setup/lava/setup.sh
 create mode 100755 frontend-install/setup/serial/setup-serio.sh
 create mode 100755 frontend-install/setup/serial/setup-serlogin.sh
 create mode 100755 frontend-install/setup/serial/setup-ttc.sh
Dockerfile +26 -70
index 3dbae0f..883ac21 100644
@@ -106,95 +106,51 @@ RUN echo deb http://emdebian.org/tools/debian/ jessie main > /etc/apt/sources.li
         gcc-arm-linux-gnueabihf && \
     rm -rf /var/lib/apt/lists/*
 
-RUN pip install \
-        filelock \
-        python-jenkins==0.4.14
-
 # ==============================================================================
-# Install Jenkins with the same UID/GID as the host user
+# Download and Install Jenkins
 # ==============================================================================
 
-ARG user=jenkins
-ARG group=jenkins
-ARG uid=1000
-ARG gid=${uid}
+ENV uid=1000
+ENV gid=${uid}
 ARG JENKINS_VERSION=2.32.1
 ARG JENKINS_SHA=bfc226aabe2bb089623772950c4cc13aee613af1
 ARG JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
 ENV JENKINS_HOME=/var/lib/jenkins
 
-RUN groupadd -g ${gid} ${group} \
-	&& useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
-RUN curl -L -O ${JENKINS_URL}
-RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
-RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb
-RUN rm jenkins_${JENKINS_VERSION}_all.deb
+RUN groupadd -g ${gid} ${group} && \
+    useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user} && \
+    curl -L -O ${JENKINS_URL} && \
+    echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c - && \
+    dpkg -i jenkins_${JENKINS_VERSION}_all.deb && \
+    rm jenkins_${JENKINS_VERSION}_all.deb
 
 # ==============================================================================
-# get ttc script and helpers
+# Post installation
 # ==============================================================================
-RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
-RUN /usr/local/src/ttc/install.sh /usr/local/bin
-RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
 
-# ==============================================================================
-# Serial Config
-# ==============================================================================
+COPY frontend-install/setup/jenkins /setup/jenkins
+WORKDIR /setup/jenkins
+RUN ./setup.sh
 
-RUN /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ;  chown -R jenkins /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
+WORKDIR /setup/jenkins/plugins
+RUN ./install.sh
 
-RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ;  chown -R jenkins /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
+COPY frontend-install/setup/serial /setup/serial
+WORKDIR /setup
 
-# ==============================================================================
-# Post installation
-# ==============================================================================
+RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc && \
+    ./serial/setup-ttc.sh
 
-RUN source /etc/default/jenkins && \
-	JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
-	sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
-
-RUN source /etc/default/jenkins && \
-	JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false" && \
-	if [ -n "$HTTP_PROXY" ]; then \
-		PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
-		PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
-		JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
-	fi && \
-	sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
-
-COPY frontend-install/plugins/flot-plotter-plugin/flot.hpi /tmp
-
-RUN service jenkins start && \
-	sleep 30 && \
-	sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin description-setter && \
-	sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin pegdown-formatter && \
-    sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin /tmp/flot.hpi && \
-    sleep 10
-
-# Let Jenkins install flot before making the symlink
-RUN service jenkins restart && sleep 30 && \
-    rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
-    ln -s /fuego-core/engine/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
-
-RUN ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
-COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
+RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio && \
+    ./serial/setup-serio.sh
 
-RUN ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/
-COPY frontend-install/config.xml $JENKINS_HOME/config.xml
-COPY frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
+RUN git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin && \
+    ./serial/setup-serlogin.sh
 
-RUN chown -R jenkins:jenkins $JENKINS_HOME/
+COPY frontend-install/setup/lava /setup/lava
+RUN ./lava/setup.sh
 
-# ==============================================================================
-# Lava
-# ==============================================================================
-
-RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
-RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
-# CONVENIENCE HACKS
-# not mounted, yet
-#RUN echo "fuego-create-node --board raspberrypi3" >> /root/firststart.sh
-#RUN echo "fuego-create-jobs --board raspberrypi3 --testplan testplan_docker --distrib nosyslogd.dist" >> /root/firststart.sh
+COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
 
 # ==============================================================================
 # Setup startup command
frontend-install/setup/jenkins/config.xml renamed from frontend-install/config.xml +0 -0
similarity index 100%
rename from frontend-install/config.xml
rename to frontend-install/setup/jenkins/config.xml
frontend-install/setup/jenkins/jenkins.model.JenkinsLocationConfiguration.xml renamed from frontend-install/jenkins.model.JenkinsLocationConfiguration.xml +0 -0
similarity index 100%
rename from frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
rename to frontend-install/setup/jenkins/jenkins.model.JenkinsLocationConfiguration.xml
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/.gitignore renamed from frontend-install/plugins/flot-plotter-plugin/.gitignore +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/.gitignore
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/.gitignore
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/flot.hpi renamed from frontend-install/plugins/flot-plotter-plugin/flot.hpi +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/flot.hpi
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/flot.hpi
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/pom.xml renamed from frontend-install/plugins/flot-plotter-plugin/pom.xml +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/pom.xml
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/pom.xml
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/java/flotile/ChartAction.java renamed from frontend-install/plugins/flot-plotter-plugin/src/main/java/flotile/ChartAction.java +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/java/flotile/ChartAction.java
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/java/flotile/ChartAction.java
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/java/flotile/FlotPublisher.java renamed from frontend-install/plugins/flot-plotter-plugin/src/main/java/flotile/FlotPublisher.java +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/java/flotile/FlotPublisher.java
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/java/flotile/FlotPublisher.java
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly renamed from frontend-install/plugins/flot-plotter-plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/resources/index.jelly renamed from frontend-install/plugins/flot-plotter-plugin/src/main/resources/index.jelly +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/resources/index.jelly
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/resources/index.jelly
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/.copyarea.db renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/.copyarea.db +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/.copyarea.db
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/.copyarea.db
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/API.txt renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/API.txt +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/API.txt
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/API.txt
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/FAQ.txt renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/FAQ.txt +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/FAQ.txt
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/FAQ.txt
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/LICENSE.txt renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/LICENSE.txt +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/LICENSE.txt
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/LICENSE.txt
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/PLUGINS.txt renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/PLUGINS.txt +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/PLUGINS.txt
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/PLUGINS.txt
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/README.txt renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/README.txt +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/README.txt
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/README.txt
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/base64.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/base64.js +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/base64.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/base64.js
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/canvas2image.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/canvas2image.js +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/canvas2image.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/canvas2image.js
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/excanvas.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/excanvas.min.js +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/excanvas.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/excanvas.min.js
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.colorhelpers.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.colorhelpers.min.js +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.colorhelpers.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.colorhelpers.min.js
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.image.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.image.min.js +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.image.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.image.min.js
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.min.js +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.min.js
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.navigate.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.navigate.min.js +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.navigate.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.navigate.min.js
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.pie.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.pie.min.js +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.pie.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.pie.min.js
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.resize.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.resize.min.js +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.resize.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.resize.min.js
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.selection.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.selection.min.js +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.selection.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.selection.min.js
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.stack.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.stack.min.js +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.stack.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.stack.min.js
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.symbol.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.symbol.min.js +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.symbol.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.symbol.min.js
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.threshold.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.threshold.min.js +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.threshold.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.threshold.min.js
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.min.js +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.min.js
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/mod.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/mod.js +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/mod.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/mod.js
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/my.css renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/my.css +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/my.css
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/my.css
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/help-globalConfig.html renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/help-globalConfig.html +0 -0
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/help-globalConfig.html
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/help-globalConfig.html
frontend-install/setup/jenkins/plugins/install.sh new file +21 -0
new file mode 100755
index 0000000..0db8bf2
@@ -0,0 +1,21 @@
+#!/bin/bash
+
+set -e
+
+service jenkins start
+sleep 30
+sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin description-setter
+sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin pegdown-formatter
+sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin flot-plotter-plugin/flot.hpi
+sleep 10
+
+# Let Jenkins install flot before making the symlink
+service jenkins restart
+sleep 30
+rm $JENKINS_HOME/plugins/flot/flot/mod.js
+ln -s /fuego-core/engine/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
+ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
+ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/
+
+chown -R jenkins:jenkins $JENKINS_HOME/
+
frontend-install/setup/jenkins/setup.sh new file +26 -0
new file mode 100755
index 0000000..c550862
@@ -0,0 +1,26 @@
+#!/bin/bash
+
+set -e
+
+user=jenkins
+group=jenkins
+uid=1000
+gid=${uid}
+
+# groupadd -g ${gid} ${group}
+# useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
+
+cp config.xml jenkins.model.JenkinsLocationConfiguration.xml ${JENKINS_HOME}
+
+source /etc/default/jenkins
+JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego"
+sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
+
+JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false"
+if [ -n "$HTTP_PROXY" ]; then
+	PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/');
+	PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/');
+	JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost=${PROXYSERVER} -Dhttp.proxyPort=${PROXYPORT} -Dhttps.proxyHost=${PROXYSERVER} -Dhttps.proxyPort=${PROXYPORT}";
+fi
+sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins
+
frontend-install/setup/lava/setup.sh new file +11 -0
new file mode 100755
index 0000000..86df49f
@@ -0,0 +1,11 @@
+#!/bin/bash
+
+set -e
+
+# CONVENIENCE HACKS
+# not mounted, yet
+#RUN echo "fuego-create-node --board raspberrypi3" >> /root/firststart.sh
+#RUN echo "fuego-create-jobs --board raspberrypi3 --testplan testplan_docker --distrib nosyslogd.dist" >> /root/firststart.sh
+
+ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
+ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
frontend-install/setup/serial/setup-serio.sh new file +8 -0
new file mode 100755
index 0000000..7e25a96
@@ -0,0 +1,8 @@
+#!/bin/bash
+
+set -e
+
+chown -R jenkins:jenkins /usr/local/src/serio
+cp /usr/local/src/serio/serio /usr/local/bin/
+ln -s /usr/local/bin/serio /usr/local/bin/sercp
+ln -s /usr/local/bin/serio /usr/local/bin/sersh
frontend-install/setup/serial/setup-serlogin.sh new file +6 -0
new file mode 100755
index 0000000..0de17ea
@@ -0,0 +1,6 @@
+#!/bin/bash
+
+set -e
+
+chown -R jenkins:jenkins /usr/local/src/serlogin
+cp /usr/local/src/serlogin/serlogin /usr/local/bin
frontend-install/setup/serial/setup-ttc.sh new file +7 -0
new file mode 100755
index 0000000..d32b73e
@@ -0,0 +1,7 @@
+#!/bin/bash
+
+set -e
+
+/usr/local/src/ttc/install.sh /usr/local/bin
+perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" \
+    /usr/local/bin/ttc
Signature · 1 line
2.15.1
#10 lore raw

Re: [Fuego] [PATCH 4/9] Move configuration and jenkins plugin installation to separate scripts

Quoted 49 lines

-----Original Message-----
From: [email protected] [mailto:fuego- [email protected]] On Behalf Of Guilherme Campos Camargo
Sent: Friday, January 26, 2018 9:36 AM
To: [email protected]
Subject: [Fuego] [PATCH 4/9] Move configuration and jenkins plugin installation to separate scripts

In order to simplify the structure of the Dockerfile and optimize build times.

---
 Dockerfile                                         |  96 ++++++---------------
 frontend-install/{ => setup/jenkins}/config.xml    |   0
 .../jenkins.model.JenkinsLocationConfiguration.xml |   0
 .../plugins/flot-plotter-plugin/.gitignore         |   0
 .../jenkins}/plugins/flot-plotter-plugin/flot.hpi  | Bin
 .../jenkins}/plugins/flot-plotter-plugin/pom.xml   |   0
 .../src/main/java/flotile/ChartAction.java         |   0
 .../src/main/java/flotile/FlotPublisher.java       |   0
 .../flotile/ChartAction/floatingBox.jelly          |   0
 .../src/main/resources/index.jelly                 |   0
 .../src/main/webapp/flot/.copyarea.db              |   0
 .../src/main/webapp/flot/API.txt                   |   0
 .../src/main/webapp/flot/FAQ.txt                   |   0
 .../src/main/webapp/flot/LICENSE.txt               |   0
 .../src/main/webapp/flot/PLUGINS.txt               |   0
 .../src/main/webapp/flot/README.txt                |   0
 .../src/main/webapp/flot/base64.js                 |   0
 .../src/main/webapp/flot/canvas2image.js           |   0
 .../src/main/webapp/flot/excanvas.min.js           |   0
 .../main/webapp/flot/jquery.colorhelpers.min.js    |   0
 .../main/webapp/flot/jquery.flot.crosshair.min.js  |   0
 .../webapp/flot/jquery.flot.fillbetween.min.js     |   0
 .../src/main/webapp/flot/jquery.flot.image.min.js  |   0
 .../src/main/webapp/flot/jquery.flot.min.js        |   0
 .../main/webapp/flot/jquery.flot.navigate.min.js   |   0
 .../src/main/webapp/flot/jquery.flot.pie.min.js    |   0
 .../src/main/webapp/flot/jquery.flot.resize.min.js |   0
 .../main/webapp/flot/jquery.flot.selection.min.js  |   0
 .../src/main/webapp/flot/jquery.flot.stack.min.js  |   0
 .../src/main/webapp/flot/jquery.flot.symbol.min.js |   0
 .../main/webapp/flot/jquery.flot.threshold.min.js  |   0
 .../src/main/webapp/flot/jquery.min.js             |   0
 .../src/main/webapp/flot/mod.js                    |   0
Quoted 8 lines
 .../src/main/webapp/flot/my.css                    |   0
 .../src/main/webapp/help-globalConfig.html         |   0
 frontend-install/setup/jenkins/plugins/install.sh  |  21 +++++
 frontend-install/setup/jenkins/setup.sh            |  26 ++++++
 frontend-install/setup/lava/setup.sh               |  11 +++
 frontend-install/setup/serial/setup-serio.sh       |   8 ++
 frontend-install/setup/serial/setup-serlogin.sh    |   6 ++
 frontend-install/setup/serial/setup-ttc.sh         |   7 ++

In general, I like the idea of refactoring this, but not this specific refactoring. I'll keep your scripts,
but maybe with renames and some merges. (serio and serlogin setup don't need to be separate).
ttc is not related to serial. the lava, serial and apt dirs are not really related to 'frontend-install', which is itself
a euphemism for 'jenkins'.

I'm applying this patch, in case subsequent patches in this series depend on the Dockerfile changes here, but I plan to refactor the frontend-install directory again after I apply the rest of this series.

Quoted 97 lines
 41 files changed, 105 insertions(+), 70 deletions(-)
 rename frontend-install/{ => setup/jenkins}/config.xml (100%)
 rename frontend-install/{ =>
setup/jenkins}/jenkins.model.JenkinsLocationConfiguration.xml (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/.gitignore (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/flot.hpi (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/pom.xml (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/java/flotile/ChartAction.java (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/java/flotile/FlotPublisher.java (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/resources/index.jelly (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/.copyarea.db (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/API.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/FAQ.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/LICENSE.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/PLUGINS.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/README.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/base64.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/canvas2image.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/excanvas.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/jquery.colorhelpers.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/jquery.flot.image.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/jquery.flot.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/jquery.flot.navigate.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/jquery.flot.pie.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/jquery.flot.resize.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/jquery.flot.selection.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/jquery.flot.stack.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/jquery.flot.symbol.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/jquery.flot.threshold.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/jquery.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/mod.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/flot/my.css (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
plugin/src/main/webapp/help-globalConfig.html (100%)
 create mode 100755 frontend-install/setup/jenkins/plugins/install.sh
 create mode 100755 frontend-install/setup/jenkins/setup.sh
 create mode 100755 frontend-install/setup/lava/setup.sh
 create mode 100755 frontend-install/setup/serial/setup-serio.sh
 create mode 100755 frontend-install/setup/serial/setup-serlogin.sh
 create mode 100755 frontend-install/setup/serial/setup-ttc.sh
Dockerfile +0 -0
index 3dbae0f..883ac21 100644
@@ -106,95 +106,51 @@ RUN echo deb http://emdebian.org/tools/debian/

jessie main > /etc/apt/sources.li

         gcc-arm-linux-gnueabihf && \
     rm -rf /var/lib/apt/lists/*
Dockerfile +0 -4
-RUN pip install \
-        filelock \
-        python-jenkins==0.4.14
-
 #

==========================================================
====================

Dockerfile +1 -1
-# Install Jenkins with the same UID/GID as the host user
+# Download and Install Jenkins
 #

==========================================================
====================

Dockerfile RUN echo deb http://emdebian.org/tools/debian/ jessie main > /etc/apt/sources.li · line 114

Quoted 6 lines
Dockerfile +2 -4
-ARG user=jenkins
-ARG group=jenkins
-ARG uid=1000
-ARG gid=${uid}
+ENV uid=1000
+ENV gid=${uid}

Do these environment variables persist in the created container? If so, these names are a bit short. They don't show up in the shell environment inside the container do they?

Quoted 576 lines

ARG JENKINS_VERSION=2.32.1
ARG JENKINS_SHA=bfc226aabe2bb089623772950c4cc13aee613af1 ARG JENKINS_URL=https://pkg.jenkins.io/debian-
stable/binary/jenkins_${JENKINS_VERSION}_all.deb
ENV JENKINS_HOME=/var/lib/jenkins

Dockerfile +0 -2
-RUN groupadd -g ${gid} ${group} \
-	&& useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -

s /bin/bash ${user}

Dockerfile +0 -2
-RUN curl -L -O ${JENKINS_URL}
-RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" |

sha1sum -c -

Dockerfile +2 -2
-RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb
-RUN rm jenkins_${JENKINS_VERSION}_all.deb
+RUN groupadd -g ${gid} ${group} && \
+    useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s

/bin/bash ${user} && \

Dockerfile +4 -1
+    curl -L -O ${JENKINS_URL} && \
+    echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum
-c - && \
+    dpkg -i jenkins_${JENKINS_VERSION}_all.deb && \
+    rm jenkins_${JENKINS_VERSION}_all.deb
 
 #

==========================================================
====================

Dockerfile +1 -1
-# get ttc script and helpers
+# Post installation
 #

==========================================================
====================

Dockerfile +0 -3
-RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
-RUN /usr/local/src/ttc/install.sh /usr/local/bin
-RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#"

/usr/local/bin/ttc

-#
==========================================================
====================

Dockerfile +0 -2
-# Serial Config
-#

==========================================================
====================

Dockerfile +3 -1
+COPY frontend-install/setup/jenkins /setup/jenkins
+WORKDIR /setup/jenkins
+RUN ./setup.sh
 
-RUN /bin/bash -c 'git clone https://github.com/frowand/serio.git

/usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'

Dockerfile +2 -1
+WORKDIR /setup/jenkins/plugins
+RUN ./install.sh
 
-RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git

/usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'

Dockerfile +2 -1
+COPY frontend-install/setup/serial /setup/serial
+WORKDIR /setup
 
-#

==========================================================
====================

Dockerfile +0 -2
-# Post installation
-#

==========================================================
====================

Dockerfile +2 -3
+RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc && \
+    ./serial/setup-ttc.sh
 
-RUN source /etc/default/jenkins && \
-	JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
-	sed -i -e

"s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins

Dockerfile +0 -3
-
-RUN source /etc/default/jenkins && \
-	JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false"

&& \

Dockerfile +0 -4
-	if [ -n "$HTTP_PROXY" ]; then \
-		PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed
-E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
-		PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E

's/\///g' | sed -E 's/(.*):(.*)/\2/') && \

Dockerfile +0 -1
-		JAVA_ARGS="$JAVA_ARGS -

Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" - Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \

Dockerfile +0 -2
-	fi && \
-	sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g"

/etc/default/jenkins;

Dockerfile +0 -6
-
-COPY frontend-install/plugins/flot-plotter-plugin/flot.hpi /tmp
-
-RUN service jenkins start && \
-	sleep 30 && \
-	sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-

cli.jar -s http://localhost:8080/fuego install-plugin description-setter && \

Dockerfile +0 -1
-	sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-

cli.jar -s http://localhost:8080/fuego install-plugin pegdown-formatter && \
- sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -
s http://localhost:8080/fuego install-plugin /tmp/flot.hpi && \

Dockerfile +0 -6
-    sleep 10
-
-# Let Jenkins install flot before making the symlink
-RUN service jenkins restart && sleep 30 && \
-    rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
-    ln -s /fuego-core/engine/scripts/mod.js

$JENKINS_HOME/plugins/flot/flot/mod.js

Dockerfile +0 -3
-
-RUN ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
-COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-

docs.pdf
+RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio && \

Dockerfile +1 -3
+    ./serial/setup-serio.sh
 
-RUN ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/
-COPY frontend-install/config.xml $JENKINS_HOME/config.xml
-COPY frontend-install/jenkins.model.JenkinsLocationConfiguration.xml

$JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml +RUN git clone https://github.com/tbird20d/serlogin.git
/usr/local/src/serlogin && \

Dockerfile +3 -2
+    ./serial/setup-serlogin.sh
 
-RUN chown -R jenkins:jenkins $JENKINS_HOME/
+COPY frontend-install/setup/lava /setup/lava
+RUN ./lava/setup.sh
 
-#

==========================================================
====================

Dockerfile +0 -2
-# Lava
-#

==========================================================
====================

Dockerfile +0 -7
-
-RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
-RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
-# CONVENIENCE HACKS
-# not mounted, yet
-#RUN echo "fuego-create-node --board raspberrypi3" >> /root/firststart.sh
-#RUN echo "fuego-create-jobs --board raspberrypi3 --testplan

testplan_docker --distrib nosyslogd.dist" >> /root/firststart.sh +COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego- docs.pdf

#
==========================================================
====================
# Setup startup command

frontend- renamed from frontend-install/config.xml +0 -0

install/setup/jenkins/config.xml
similarity index 100%
rename from frontend-install/config.xml rename to frontend-install/setup/jenkins/config.xml

frontend- renamed from frontend-install/jenkins.model.JenkinsLocationConfiguration.xml +0 -0

install/setup/jenkins/jenkins.model.JenkinsLocationConfiguration.xml similarity index 100%
rename from frontend- install/jenkins.model.JenkinsLocationConfiguration.xml rename to frontend- install/setup/jenkins/jenkins.model.JenkinsLocationConfiguration.xml

frontend- renamed from frontend-install/plugins/flot-plotter-plugin/.gitignore +0 -0

install/setup/jenkins/plugins/flot-plotter-plugin/.gitignore similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/.gitignore rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/.gitignore

frontend- renamed from frontend-install/plugins/flot-plotter-plugin/flot.hpi +0 -0

install/setup/jenkins/plugins/flot-plotter-plugin/flot.hpi similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/flot.hpi rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/flot.hpi

frontend- renamed from frontend-install/plugins/flot-plotter-plugin/pom.xml +0 -0

install/setup/jenkins/plugins/flot-plotter-plugin/pom.xml similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/pom.xml rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/pom.xml

(quoted diff) +0 -0

plugin/src/main/java/flotile/ChartAction.java b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/java/flotile/ChartAction.java similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/java/flotile/ChartAction.java
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/java/flotile/ChartAction.java

(quoted diff) +0 -0

plugin/src/main/java/flotile/FlotPublisher.java b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/java/flotile/FlotPublisher.java similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/java/flotile/FlotPublisher.java rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/java/flotile/FlotPublisher.java

(quoted diff) +0 -0

plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly

(quoted diff) +0 -0

plugin/src/main/resources/index.jelly b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/resources/index.jelly
similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/resources/index.jelly
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/resources/index.jelly

(quoted diff) +0 -0

plugin/src/main/webapp/flot/.copyarea.db b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/.copyarea.db
similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/.copyarea.db
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/.copyarea.db

(quoted diff) +0 -0

plugin/src/main/webapp/flot/API.txt b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/API.txt
similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/API.txt
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/API.txt

(quoted diff) +0 -0

plugin/src/main/webapp/flot/FAQ.txt b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/FAQ.txt
similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/FAQ.txt
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/FAQ.txt

(quoted diff) +0 -0

plugin/src/main/webapp/flot/LICENSE.txt b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/LICENSE.txt
similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/LICENSE.txt
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/LICENSE.txt

(quoted diff) +0 -0

plugin/src/main/webapp/flot/PLUGINS.txt b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/PLUGINS.txt
similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/PLUGINS.txt
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/PLUGINS.txt

(quoted diff) +0 -0

plugin/src/main/webapp/flot/README.txt b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/README.txt
similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/README.txt
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/README.txt

(quoted diff) +0 -0

plugin/src/main/webapp/flot/base64.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/base64.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/base64.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/base64.js

(quoted diff) +0 -0

plugin/src/main/webapp/flot/canvas2image.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/canvas2image.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/canvas2image.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/canvas2image.js

(quoted diff) +0 -0

plugin/src/main/webapp/flot/excanvas.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/excanvas.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/excanvas.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/excanvas.min.js

(quoted diff) +0 -0

plugin/src/main/webapp/flot/jquery.colorhelpers.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.colorhelpers.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.colorhelpers.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.colorhelpers.min.js

(quoted diff) +0 -0

plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js

(quoted diff) +0 -0

plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js

(quoted diff) +0 -0

plugin/src/main/webapp/flot/jquery.flot.image.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.image.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.image.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.image.min.js

(quoted diff) +0 -0

plugin/src/main/webapp/flot/jquery.flot.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.min.js

(quoted diff) +0 -0

plugin/src/main/webapp/flot/jquery.flot.navigate.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.navigate.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.navigate.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.navigate.min.js

(quoted diff) +0 -0

plugin/src/main/webapp/flot/jquery.flot.pie.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.pie.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.pie.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.pie.min.js

(quoted diff) +0 -0

plugin/src/main/webapp/flot/jquery.flot.resize.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.resize.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.resize.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.resize.min.js

(quoted diff) +0 -0

plugin/src/main/webapp/flot/jquery.flot.selection.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.selection.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.selection.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.selection.min.js

(quoted diff) +0 -0

plugin/src/main/webapp/flot/jquery.flot.stack.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.stack.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.stack.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.stack.min.js

(quoted diff) +0 -0

plugin/src/main/webapp/flot/jquery.flot.symbol.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.symbol.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.symbol.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.symbol.min.js

(quoted diff) +0 -0

plugin/src/main/webapp/flot/jquery.flot.threshold.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.threshold.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.threshold.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.threshold.min.js

(quoted diff) +0 -0

plugin/src/main/webapp/flot/jquery.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.min.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.min.js

(quoted diff) +0 -0

plugin/src/main/webapp/flot/mod.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/mod.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/mod.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/mod.js

(quoted diff) +0 -0

plugin/src/main/webapp/flot/my.css b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/my.css
similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/my.css
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/my.css

(quoted diff) +0 -0

plugin/src/main/webapp/help-globalConfig.html b/frontend- install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/help- globalConfig.html
similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/help-globalConfig.html
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/help-globalConfig.html

frontend- renamed from frontend-install/setup/jenkins/plugins/install.sh +0 -0

install/setup/jenkins/plugins/install.sh
new file mode 100755
index 0000000..0db8bf2

frontend-install/setup/jenkins/plugins/install.sh new file +7 -0
@@ -0,0 +1,21 @@
+#!/bin/bash
+
+set -e
+
+service jenkins start
+sleep 30
+sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s

http://localhost:8080/fuego install-plugin description-setter +sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s
http://localhost:8080/fuego install-plugin pegdown-formatter +sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s
http://localhost:8080/fuego install-plugin flot-plotter-plugin/flot.hpi

frontend-install/setup/jenkins/plugins/install.sh new file +7 -0
+sleep 10
+
+# Let Jenkins install flot before making the symlink
+service jenkins restart
+sleep 30
+rm $JENKINS_HOME/plugins/flot/flot/mod.js
+ln -s /fuego-core/engine/scripts/mod.js

$JENKINS_HOME/plugins/flot/flot/mod.js

frontend-install/setup/jenkins/plugins/install.sh new file +5 -0
+ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
+ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/
+
+chown -R jenkins:jenkins $JENKINS_HOME/
+
frontend- renamed from frontend-install/setup/jenkins/setup.sh +0 -0

install/setup/jenkins/setup.sh
new file mode 100755
index 0000000..c550862

frontend-install/setup/jenkins/setup.sh new file +11 -0
@@ -0,0 +1,26 @@
+#!/bin/bash
+
+set -e
+
+user=jenkins
+group=jenkins
+uid=1000
+gid=${uid}
+
+# groupadd -g ${gid} ${group}
+# useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s

/bin/bash ${user}

frontend-install/setup/jenkins/setup.sh new file +2 -0
+
+cp config.xml jenkins.model.JenkinsLocationConfiguration.xml

${JENKINS_HOME}

frontend-install/setup/jenkins/setup.sh new file +4 -0
+
+source /etc/default/jenkins
+JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego"
+sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g"

/etc/default/jenkins

frontend-install/setup/jenkins/setup.sh new file +4 -0
+
+JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false"
+if [ -n "$HTTP_PROXY" ]; then
+	PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E

's/\///g' | sed -E 's/(.*):(.*)/\1/');

frontend-install/setup/jenkins/setup.sh new file +1 -0
+	PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E

's/\///g' | sed -E 's/(.*):(.*)/\2/');

frontend-install/setup/jenkins/setup.sh new file +1 -0
+	JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost=${PROXYSERVER} -

Dhttp.proxyPort=${PROXYPORT} -Dhttps.proxyHost=${PROXYSERVER} - Dhttps.proxyPort=${PROXYPORT}";

frontend-install/setup/jenkins/setup.sh new file +2 -0
+fi
+sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g"

/etc/default/jenkins

frontend-install/setup/jenkins/setup.sh new file +1 -0
+
frontend- renamed from frontend-install/setup/lava/setup.sh +0 -0

install/setup/lava/setup.sh
new file mode 100755
index 0000000..86df49f

frontend-install/setup/lava/setup.sh new file +8 -0
@@ -0,0 +1,11 @@
+#!/bin/bash
+
+set -e
+
+# CONVENIENCE HACKS
+# not mounted, yet
+#RUN echo "fuego-create-node --board raspberrypi3" >> /root/firststart.sh
+#RUN echo "fuego-create-jobs --board raspberrypi3 --testplan

testplan_docker --distrib nosyslogd.dist" >> /root/firststart.sh

frontend-install/setup/lava/setup.sh new file +3 -0
+
+ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
+ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
frontend- renamed from frontend-install/setup/serial/setup-serio.sh +0 -0

install/setup/serial/setup-serio.sh
new file mode 100755
index 0000000..7e25a96

frontend-install/setup/serial/setup-serio.sh new file +8 -0
@@ -0,0 +1,8 @@
+#!/bin/bash
+
+set -e
+
+chown -R jenkins:jenkins /usr/local/src/serio
+cp /usr/local/src/serio/serio /usr/local/bin/
+ln -s /usr/local/bin/serio /usr/local/bin/sercp
+ln -s /usr/local/bin/serio /usr/local/bin/sersh
frontend- renamed from frontend-install/setup/serial/setup-serlogin.sh +0 -0

install/setup/serial/setup-serlogin.sh
new file mode 100755
index 0000000..0de17ea

frontend-install/setup/serial/setup-serlogin.sh new file +6 -0
@@ -0,0 +1,6 @@
+#!/bin/bash
+
+set -e
+
+chown -R jenkins:jenkins /usr/local/src/serlogin
+cp /usr/local/src/serlogin/serlogin /usr/local/bin
frontend- renamed from frontend-install/setup/serial/setup-ttc.sh +0 -0

install/setup/serial/setup-ttc.sh
new file mode 100755
index 0000000..d32b73e

frontend-install/setup/serial/setup-ttc.sh new file +7 -1
@@ -0,0 +1,7 @@
+#!/bin/bash
+
+set -e
+
+/usr/local/src/ttc/install.sh /usr/local/bin
+perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" \
+    /usr/local/bin/ttc
--
Quoted 1 line

2.15.1

OK - applied, but with caveats mentioned above.
-- Tim

#11 lore raw

PATCH 5/9 Remove jenkins user created during build time with host's uid/gid

In reply to Guilherme Campos Camargo

Prior to this patch, the fuego-host-scripts/docker-build-image.sh script was calling docker build passing two build-args: uid/gid. According to the comments in the code, it seems that the intention was to make sure that the jenkins user (needed by Jenkins) would have the same uid/gid as the user of the host that has called the install script.

Given that the uid/gid of the jenkins user was being set at build time, the generated image would be configured only for the machine where it's built, what would prevent jenkins to change files in the /fuego-rw mountpoint in different hosts.

One approach for allowing jenkins to write to fuego-rw, is to chown /fuego-rw and it's pre-existing subdirectories during execution time, what can be done in a docker entrypoint script. That approach per se, solves the problem, by allowing jenkins to write to the mount point, but generates a side-effect: the user on the host machine will be able to inspect, but will not be able to delete/modify the files created by jenkins (unless it forces it as root).

Another approach would be to change the uid and the gid of the jenkins user in the first execution, (first call to docker-container-start.sh), also through the entrypoint. This would solve the problem of the access from both sides, but has the downside of the need of running a recursive chown in all jenkins files in /var (including cache), possibly taking a long time to be accomplished.

Since the need to edit/remove the files from fuego-rw (without sudo) depends on the user, our suggestion is to use the entrypoint for chowning only the pre-existing directories from fuego-rw and to provide the user a script for mapping the jenkins' uid/gid with user's uid/gid only if the user needs to.

On this patch we have created the entrypoint.sh script, and the uid/gid mapping script.

---
 Dockerfile                                             | 10 ++++------
 frontend-install/entrypoint.sh                         | 13 +++++++++++++
 .../setup/jenkins/map-jenkins-uid-to-host.sh           | 12 ++++++++++++
 frontend-install/setup/jenkins/setup.sh                |  8 --------
 fuego-host-scripts/docker-build-image.sh               | 10 +---------
 fuego-host-scripts/jenkins-map-uid.sh                  | 18 ++++++++++++++++++
 6 files changed, 48 insertions(+), 23 deletions(-)
 create mode 100755 frontend-install/entrypoint.sh
 create mode 100755 frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh
 create mode 100755 fuego-host-scripts/jenkins-map-uid.sh
Dockerfile +4 -6
index 883ac21..493414e 100644
@@ -110,16 +110,12 @@ RUN echo deb http://emdebian.org/tools/debian/ jessie main > /etc/apt/sources.li
 # Download and Install Jenkins
 # ==============================================================================
 
-ENV uid=1000
-ENV gid=${uid}
 ARG JENKINS_VERSION=2.32.1
 ARG JENKINS_SHA=bfc226aabe2bb089623772950c4cc13aee613af1
 ARG JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
 ENV JENKINS_HOME=/var/lib/jenkins
 
-RUN groupadd -g ${gid} ${group} && \
-    useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user} && \
-    curl -L -O ${JENKINS_URL} && \
+RUN curl -L -O ${JENKINS_URL} && \
     echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c - && \
     dpkg -i jenkins_${JENKINS_VERSION}_all.deb && \
     rm jenkins_${JENKINS_VERSION}_all.deb
@@ -156,4 +152,6 @@ COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
 # Setup startup command
 # ==============================================================================
 
-ENTRYPOINT service jenkins start && service netperf start && /bin/bash
+WORKDIR /
+COPY frontend-install/entrypoint.sh /
+ENTRYPOINT ["/entrypoint.sh"]
frontend-install/entrypoint.sh new file +13 -0
new file mode 100755
index 0000000..281b0fe
@@ -0,0 +1,13 @@
+#!/bin/bash
+set -e
+
+service jenkins stop >> /dev/null
+
+mkdir -p fuego-rw/boards /fuego-rw/logs /fuego-rw/buildzone
+chown jenkins:jenkins \
+    fuego-rw fuego-rw/boards /fuego-rw/logs /fuego-rw/buildzone
+
+service jenkins start
+service netperf start
+
+exec /bin/bash
frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh new file +12 -0
new file mode 100755
index 0000000..002f165
@@ -0,0 +1,12 @@
+#!/bin/bash
+set -e
+
+service jenkins stop
+
+usermod -u "${jenkins_uid}" jenkins
+groupmod -g "${jenkins_gid}" jenkins
+
+chown -R "${jenkins_uid}":"${jenkins_gid}" \
+    /var/lib/jenkins /var/cache/jenkins /var/log/jenkins /fuego-rw
+
+service jenkins start
frontend-install/setup/jenkins/setup.sh +0 -8
index c550862..11ad371 100755
@@ -2,14 +2,6 @@
 
 set -e
 
-user=jenkins
-group=jenkins
-uid=1000
-gid=${uid}
-
-# groupadd -g ${gid} ${group}
-# useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
-
 cp config.xml jenkins.model.JenkinsLocationConfiguration.xml ${JENKINS_HOME}
 
 source /etc/default/jenkins
fuego-host-scripts/docker-build-image.sh +1 -9
index 7c11b11..6550a98 100755
@@ -2,12 +2,4 @@
 # $1 - name for the docker image (default: fuego)
 DOCKERIMAGE=${1:-fuego}
 
-if [ "$(id -u)" == "0" ]; then
-	JENKINS_UID=$(id -u $SUDO_USER)
-	JENKINS_GID=$(id -g $SUDO_USER)
-else
-	JENKINS_UID=$(id -u $USER)
-	JENKINS_GID=$(id -g $USER)
-fi
-
-sudo docker build -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy --build-arg uid=$JENKINS_UID --build-arg gid=$JENKINS_GID .
+sudo docker build -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy .
fuego-host-scripts/jenkins-map-uid.sh new file +18 -0
new file mode 100755
index 0000000..fad619e
@@ -0,0 +1,18 @@
+#!/bin/bash
+# $1 - name for the docker container (default: fuego-container)
+DOCKERCONTAINER=${1:-fuego-container}
+
+if [ "${UID}" == "0" ]; then
+	uid=$(id -u "${SUDO_USER}")
+	gid=$(id -g "${SUDO_USER}")
+else
+	uid="${UID}"
+	gid=$(id -g "${USER}")
+fi
+
+sudo docker start "${DOCKERCONTAINER}" || \
+  echo "Please create Fuego docker container via docker-create-container.sh script"
+
+sudo docker exec -e jenkins_uid="${uid}" -e jenkins_gid="${gid}" \
+    "${DOCKERCONTAINER}" \
+    /setup/jenkins/map-jenkins-uid-to-host.sh
Signature · 1 line
2.15.1
#12 lore raw

Re: [Fuego] [PATCH 5/9] Remove jenkins user created during build time with host's uid/gid

Quoted 9 lines

-----Original Message-----
From: Guilherme Campos Camargo

Prior to this patch, the fuego-host-scripts/docker-build-image.sh script was calling docker build passing two build-args: uid/gid. According to the comments in the code, it seems that the intention was to make sure that the jenkins user (needed by Jenkins) would have the same uid/gid as the user of the host that has called the install script.

Quoted 4 lines

Given that the uid/gid of the jenkins user was being set at build time, the generated image would be configured only for the machine where it's built, what would prevent jenkins to change files in the /fuego-rw mountpoint in different hosts.

That's correct, and changing this is a nice step forward to making the container usable on other machines than the one it was built on.

Quoted 9 lines

One approach for allowing jenkins to write to fuego-rw, is to chown /fuego-rw and it's pre-existing subdirectories during execution time, what can be done in a docker entrypoint script. That approach per se, solves the problem, by allowing jenkins to write to the mount point, but generates a side-effect: the user on the host machine will be able to inspect, but will not be able to delete/modify the files created by jenkins (unless it forces it as root).

Quoted 6 lines

Another approach would be to change the uid and the gid of the jenkins user in the first execution, (first call to docker-container-start.sh), also through the entrypoint. This would solve the problem of the access from both sides, but has the downside of the need of running a recursive chown in all jenkins files in /var (including cache), possibly taking a long time to be accomplished.

I'd much rather do this. This amounts to a one-time container installation overhead on a new host, for the Fuego user.

I modify files in fuego-ro and fuego-rw all the time, from the host side. This is my normal workflow, and I'd like to see it supported.

Quoted 9 lines

Since the need to edit/remove the files from fuego-rw (without sudo) depends on the user, our suggestion is to use the entrypoint for chowning only the pre-existing directories from fuego-rw and to provide the user a script for mapping the jenkins' uid/gid with user's uid/gid only if the user needs to.

On this patch we have created the entrypoint.sh script, and the uid/gid mapping script.

At this point I'm confused. Do both approaches have an entrypoint.sh script?

Could you give an example of the steps required to install the container in both your scenarios, including the arguments required for the scripts, and the resulting permissions and ownership both inside and outside the container, for the volumes?

I'm not sure when map-jenkins-uid-to-host.sh and jenkins-map-uid.sh are run, and there seem to be no instructions for using them, or a human-readable description of what they do.

Thanks,
-- Tim

frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh

Quoted 79 lines
---
 Dockerfile                                             | 10 ++++------
 frontend-install/entrypoint.sh                         | 13 +++++++++++++
 .../setup/jenkins/map-jenkins-uid-to-host.sh           | 12 ++++++++++++
 frontend-install/setup/jenkins/setup.sh                |  8 --------
 fuego-host-scripts/docker-build-image.sh               | 10 +---------
 fuego-host-scripts/jenkins-map-uid.sh                  | 18 ++++++++++++++++++
 6 files changed, 48 insertions(+), 23 deletions(-)
 create mode 100755 frontend-install/entrypoint.sh
 create mode 100755 frontend-install/setup/jenkins/map-jenkins-uid-to-
host.sh
 create mode 100755 fuego-host-scripts/jenkins-map-uid.sh
Dockerfile +0 -0
index 883ac21..493414e 100644
@@ -110,16 +110,12 @@ RUN echo deb http://emdebian.org/tools/debian/

jessie main > /etc/apt/sources.li
# Download and Install Jenkins
#
==========================================================
====================

Dockerfile +0 -2
-ENV uid=1000
-ENV gid=${uid}
 ARG JENKINS_VERSION=2.32.1
 ARG JENKINS_SHA=bfc226aabe2bb089623772950c4cc13aee613af1
 ARG JENKINS_URL=https://pkg.jenkins.io/debian-

stable/binary/jenkins_${JENKINS_VERSION}_all.deb
ENV JENKINS_HOME=/var/lib/jenkins

Dockerfile +0 -2
-RUN groupadd -g ${gid} ${group} && \
-    useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s

/bin/bash ${user} && \

Dockerfile +1 -2
-    curl -L -O ${JENKINS_URL} && \
+RUN curl -L -O ${JENKINS_URL} && \
     echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum
-c - && \
     dpkg -i jenkins_${JENKINS_VERSION}_all.deb && \
     rm jenkins_${JENKINS_VERSION}_all.deb
@@ -156,4 +152,6 @@ COPY docs/fuego-docs.pdf

$JENKINS_HOME/userContent/docs/fuego-docs.pdf
# Setup startup command
#
==========================================================
====================

Dockerfile +3 -1
-ENTRYPOINT service jenkins start && service netperf start && /bin/bash
+WORKDIR /
+COPY frontend-install/entrypoint.sh /
+ENTRYPOINT ["/entrypoint.sh"]
frontend-install/entrypoint.sh new file +13 -0
new file mode 100755
index 0000000..281b0fe
@@ -0,0 +1,13 @@
+#!/bin/bash
+set -e
+
+service jenkins stop >> /dev/null
+
+mkdir -p fuego-rw/boards /fuego-rw/logs /fuego-rw/buildzone
+chown jenkins:jenkins \
+    fuego-rw fuego-rw/boards /fuego-rw/logs /fuego-rw/buildzone
+
+service jenkins start
+service netperf start
+
+exec /bin/bash
frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh new file +0 -0
new file mode 100755
index 0000000..002f165

Who calls this? When, and with what args?

fuego-host-scripts/jenkins-map-uid.sh

Quoted 63 lines
frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh new file +12 -0
@@ -0,0 +1,12 @@
+#!/bin/bash
+set -e
+
+service jenkins stop
+
+usermod -u "${jenkins_uid}" jenkins
+groupmod -g "${jenkins_gid}" jenkins
+
+chown -R "${jenkins_uid}":"${jenkins_gid}" \
+    /var/lib/jenkins /var/cache/jenkins /var/log/jenkins /fuego-rw
+
+service jenkins start
frontend- renamed from frontend-install/setup/jenkins/setup.sh +0 -0

install/setup/jenkins/setup.sh
index c550862..11ad371 100755

frontend-install/setup/jenkins/setup.sh +0 -8
@@ -2,14 +2,6 @@
 
 set -e
 
-user=jenkins
-group=jenkins
-uid=1000
-gid=${uid}
-
-# groupadd -g ${gid} ${group}
-# useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s
 /bin/bash ${user}
-
 cp config.xml jenkins.model.JenkinsLocationConfiguration.xml
 ${JENKINS_HOME}

source /etc/default/jenkins

fuego-host- renamed from fuego-host-scripts/docker-build-image.sh +0 -0

scripts/docker-build-image.sh
index 7c11b11..6550a98 100755

fuego-host-scripts/docker-build-image.sh +0 -9
@@ -2,12 +2,4 @@
 # $1 - name for the docker image (default: fuego)
 DOCKERIMAGE=${1:-fuego}
 
-if [ "$(id -u)" == "0" ]; then
-	JENKINS_UID=$(id -u $SUDO_USER)
-	JENKINS_GID=$(id -g $SUDO_USER)
-else
-	JENKINS_UID=$(id -u $USER)
-	JENKINS_GID=$(id -g $USER)
-fi
-
-sudo docker build -t ${DOCKERIMAGE} --build-arg

HTTP_PROXY=$http_proxy --build-arg uid=$JENKINS_UID --build-arg gid=$JENKINS_GID .
+sudo docker build -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy .

fuego-host- renamed from fuego-host-scripts/jenkins-map-uid.sh +0 -0

scripts/jenkins-map-uid.sh
new file mode 100755
index 0000000..fad619e

fuego-host-scripts/jenkins-map-uid.sh new file +0 -0

Who calls this? When, and with what args?

Quoted 21 lines
fuego-host-scripts/jenkins-map-uid.sh new file +14 -0
@@ -0,0 +1,18 @@
+#!/bin/bash
+# $1 - name for the docker container (default: fuego-container)
+DOCKERCONTAINER=${1:-fuego-container}
+
+if [ "${UID}" == "0" ]; then
+	uid=$(id -u "${SUDO_USER}")
+	gid=$(id -g "${SUDO_USER}")
+else
+	uid="${UID}"
+	gid=$(id -g "${USER}")
+fi
+
+sudo docker start "${DOCKERCONTAINER}" || \
+  echo "Please create Fuego docker container via docker-create-

container.sh script"

fuego-host-scripts/jenkins-map-uid.sh new file +4 -1
+
+sudo docker exec -e jenkins_uid="${uid}" -e jenkins_gid="${gid}" \
+    "${DOCKERCONTAINER}" \
+    /setup/jenkins/map-jenkins-uid-to-host.sh
--
Quoted 1 line

2.15.1

I need to understand this better before I apply it.
-- Tim

#13 lore raw

Re: [Fuego] [PATCH 5/9] Remove jenkins user created during build time with host's uid/gid

Hello, Tim

Thanks for your input.

Quoted 49 lines · On Tue, Jan 30, 2018 at 6:39 PM, <[email protected]> wrote:
Quoted 9 lines

-----Original Message-----
From: Guilherme Campos Camargo

Prior to this patch, the fuego-host-scripts/docker-build-image.sh script was calling docker build passing two build-args: uid/gid. According to the comments in the code, it seems that the intention was to make sure that the jenkins user (needed by Jenkins) would have the same uid/gid as the user of the host that has called the install script.

Quoted 4 lines

Given that the uid/gid of the jenkins user was being set at build time, the generated image would be configured only for the machine where it's built, what would prevent jenkins to change files in the /fuego-rw mountpoint in different hosts.

That's correct, and changing this is a nice step forward to making the container usable on other machines than the one it was built on.

Quoted 9 lines

One approach for allowing jenkins to write to fuego-rw, is to chown /fuego-rw and it's pre-existing subdirectories during execution time, what can be done in a docker entrypoint script. That approach per se, solves the problem, by allowing jenkins to write to the mount point, but generates a side-effect: the user on the host machine will be able to inspect, but will not be able to delete/modify the files created by jenkins (unless it forces it as root).

Quoted 6 lines

Another approach would be to change the uid and the gid of the jenkins user in the first execution, (first call to docker-container-start.sh), also through the entrypoint. This would solve the problem of the access from both sides, but has the downside of the need of running a recursive chown in all jenkins files in /var (including cache), possibly taking a long time to be accomplished.

I'd much rather do this. This amounts to a one-time container installation overhead on a new host, for the Fuego user.

I modify files in fuego-ro and fuego-rw all the time, from the host side. This is my normal workflow, and I'd like to see it supported.

Quoted 9 lines

Since the need to edit/remove the files from fuego-rw (without sudo) depends on the user, our suggestion is to use the entrypoint for chowning only the pre-existing directories from fuego-rw and to provide the user a script for mapping the jenkins' uid/gid with user's uid/gid only if the user needs to.

On this patch we have created the entrypoint.sh script, and the uid/gid mapping script.

Quoted 9 lines

At this point I'm confused. Do both approaches have an entrypoint.sh script?

Could you give an example of the steps required to install the container in both your scenarios, including the arguments required for the scripts, and the resulting permissions and ownership both inside and outside the container, for the volumes?

With both approaches, there would be no change compared to the instructions that
are given in Fuego's Quick Start guide.​

$ ./install.sh
$ ./fuego-host-scripts/create-container.sh
$ ./fuego-host-scripts/start-container.sh

We agree with you, and decided to move to the second approach (set the uid/gid at the first start through the entrypoint).

Explaining:

With the first approach (the one that was implemented in the first version of
my patch), the uid and the gid of the Jenkins user, inside the container, would
be different than the uid/gid of the user that's running the container from the
host. In order to allow Jenkins to write into fuego-rw, we were changing the ownership of fuego-rw, fuego-rw/boards, fuego-rw/logs and fuego-rw/buildzone to
the Jenkins user inside the container (not recursively). This would allow jenkins to write to those directories and the change would propagate to the outside, not allowing the user to modify the contents of fuego-rw anymore.

Apparently there was a misconception from our side, that most users would not use fuego-rw to write, but only to read, and that Jenkins would be the only
one (most of times) to write into that directory.

In order to allow those users to write into that directory as well, we had added a new script (fuego-host-scripts/jenkins-map-uid.sh), that mapped Jenkin's uid/gid to the host user's only on-demand.

In order to run that script the user would simply call without any arguments

$ ./fuego-host-scripts/jenkins-map-uid.sh

On docker, the only way to run a script inside the container is to have that script copied to the container. Therefore, fuego-hosts-scripts/jenkins-map-uid.sh was just calling that other script (setup/jenkins/map-jenkins-uid-to-host.sh) that's been copied during Docker build. "map-jenkins-uid-to-host.sh" is the one that would actually change jenkins uid and chown the nedeed directories.

With the second approach (the one that we have implemented in our fixup), the uid and the gid of the Jenkins user, inside the container, would be the same as the
user that's running the container from the host. What means that, if the owner
of the fuego-rw directory is the same user that's running the container, fuego-rw will be accessible from both sides seamlessly.

The downside of this approach is that the create-container.sh step would take
longer, due to the recursive chowns that are needed after replacing Jenkins' uid/gid with the user's.

However, as you said, given that most of users will actually need to write to
fuego-rw, the second approach is clearly the best, as you agree. So I'm sending
a new patch, implementing the second approach, as a replacement for this one.

I'm rebasing over the new 'next' branch as well.

Quoted 189 lines

I'm not sure when map-jenkins-uid-to-host.sh and jenkins-map-uid.sh are run, and there seem to be no instructions for using them, or a human-readable description of what they do.

Thanks,
-- Tim

Quoted 5 lines
---
 Dockerfile                                             | 10 ++++------
 frontend-install/entrypoint.sh                         | 13

+++++++++++++

Quoted 1 line
 .../setup/jenkins/map-jenkins-uid-to-host.sh           | 12

++++++++++++

Quoted 3 lines
 frontend-install/setup/jenkins/setup.sh                |  8 --------
 fuego-host-scripts/docker-build-image.sh               | 10 +---------
 fuego-host-scripts/jenkins-map-uid.sh                  | 18

++++++++++++++++++

Review

Quoted 46 lines
 6 files changed, 48 insertions(+), 23 deletions(-)
 create mode 100755 frontend-install/entrypoint.sh
 create mode 100755 frontend-install/setup/jenkins/map-jenkins-uid-to-
host.sh
 create mode 100755 fuego-host-scripts/jenkins-map-uid.sh
Dockerfile +0 -0
index 883ac21..493414e 100644
@@ -110,16 +110,12 @@ RUN echo deb http://emdebian.org/tools/debian/

jessie main > /etc/apt/sources.li
# Download and Install Jenkins
#
==========================================================
====================

Dockerfile +0 -2
-ENV uid=1000
-ENV gid=${uid}
 ARG JENKINS_VERSION=2.32.1
 ARG JENKINS_SHA=bfc226aabe2bb089623772950c4cc13aee613af1
 ARG JENKINS_URL=https://pkg.jenkins.io/debian-

stable/binary/jenkins_${JENKINS_VERSION}_all.deb
ENV JENKINS_HOME=/var/lib/jenkins

Dockerfile +0 -2
-RUN groupadd -g ${gid} ${group} && \
-    useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s

/bin/bash ${user} && \

Dockerfile +1 -2
-    curl -L -O ${JENKINS_URL} && \
+RUN curl -L -O ${JENKINS_URL} && \
     echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum
-c - && \
     dpkg -i jenkins_${JENKINS_VERSION}_all.deb && \
     rm jenkins_${JENKINS_VERSION}_all.deb
@@ -156,4 +152,6 @@ COPY docs/fuego-docs.pdf

$JENKINS_HOME/userContent/docs/fuego-docs.pdf
# Setup startup command
#
==========================================================
====================

Dockerfile +3 -1
-ENTRYPOINT service jenkins start && service netperf start && /bin/bash
+WORKDIR /
+COPY frontend-install/entrypoint.sh /
+ENTRYPOINT ["/entrypoint.sh"]
(quoted diff) +0 -0

b/frontend-install/entrypoint.sh

frontend-install/

Quoted 24 lines

new file mode 100755
index 0000000..281b0fe

frontend-install/entrypoint.sh new file +13 -0
@@ -0,0 +1,13 @@
+#!/bin/bash
+set -e
+
+service jenkins stop >> /dev/null
+
+mkdir -p fuego-rw/boards /fuego-rw/logs /fuego-rw/buildzone
+chown jenkins:jenkins \
+    fuego-rw fuego-rw/boards /fuego-rw/logs /fuego-rw/buildzone
+
+service jenkins start
+service netperf start
+
+exec /bin/bash
frontend-install/ new file +0 -0
new file mode 100755
index 0000000..002f165

​​
setup/jenkins/map-jenkins-uid-to-host.sh

Who calls this? When, and with what args?

fuego-host-scripts/jenkins-map-uid.sh

Quoted 63 lines
frontend-install/ new file +12 -0
@@ -0,0 +1,12 @@
+#!/bin/bash
+set -e
+
+service jenkins stop
+
+usermod -u "${jenkins_uid}" jenkins
+groupmod -g "${jenkins_gid}" jenkins
+
+chown -R "${jenkins_uid}":"${jenkins_gid}" \
+    /var/lib/jenkins /var/cache/jenkins /var/log/jenkins /fuego-rw
+
+service jenkins start
frontend- renamed from frontend-install/setup/jenkins/setup.sh +0 -0

install/setup/jenkins/setup.sh
index c550862..11ad371 100755

frontend-install/setup/jenkins/setup.sh +0 -8
@@ -2,14 +2,6 @@
 
 set -e
 
-user=jenkins
-group=jenkins
-uid=1000
-gid=${uid}
-
-# groupadd -g ${gid} ${group}
-# useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s
 /bin/bash ${user}
-
 cp config.xml jenkins.model.JenkinsLocationConfiguration.xml
 ${JENKINS_HOME}

source /etc/default/jenkins

fuego-host- renamed from fuego-host-scripts/docker-build-image.sh +0 -0

scripts/docker-build-image.sh
index 7c11b11..6550a98 100755

fuego-host-scripts/docker-build-image.sh +0 -9
@@ -2,12 +2,4 @@
 # $1 - name for the docker image (default: fuego)
 DOCKERIMAGE=${1:-fuego}
 
-if [ "$(id -u)" == "0" ]; then
-     JENKINS_UID=$(id -u $SUDO_USER)
-     JENKINS_GID=$(id -g $SUDO_USER)
-else
-     JENKINS_UID=$(id -u $USER)
-     JENKINS_GID=$(id -g $USER)
-fi
-
-sudo docker build -t ${DOCKERIMAGE} --build-arg

HTTP_PROXY=$http_proxy --build-arg uid=$JENKINS_UID --build-arg gid=$JENKINS_GID .
+sudo docker build -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy .

fuego-host- renamed from fuego-host-scripts/jenkins-map-uid.sh +0 -0

scripts/jenkins-map-uid.sh
new file mode 100755
index 0000000..fad619e

fuego-host-scripts/jenkins-map-uid.sh new file +0 -0

Who calls this? When, and with what args?

Quoted 21 lines
fuego-host-scripts/jenkins-map-uid.sh new file +14 -0
@@ -0,0 +1,18 @@
+#!/bin/bash
+# $1 - name for the docker container (default: fuego-container)
+DOCKERCONTAINER=${1:-fuego-container}
+
+if [ "${UID}" == "0" ]; then
+     uid=$(id -u "${SUDO_USER}")
+     gid=$(id -g "${SUDO_USER}")
+else
+     uid="${UID}"
+     gid=$(id -g "${USER}")
+fi
+
+sudo docker start "${DOCKERCONTAINER}" || \
+  echo "Please create Fuego docker container via docker-create-

container.sh script"

fuego-host-scripts/jenkins-map-uid.sh new file +4 -1
+
+sudo docker exec -e jenkins_uid="${uid}" -e jenkins_gid="${gid}" \
+    "${DOCKERCONTAINER}" \
+    /setup/jenkins/map-jenkins-uid-to-host.sh
--
Quoted 1 line

2.15.1

I need to understand this better before I apply it.
-- Tim

#14 lore raw

PATCH 6/9 Cleanup proxy

In reply to Guilherme Campos Camargo

Prior to this patch, http_proxy was being set and saved during build time according to the http_proxy values of the host.

This approach has at least two shortcomings, that are explained below:

1. Any other user that wanted to use a prebuilt image in their host would be unable to do so, because the proxy configuration is matching the host of the builder and not the host of the user.

2. A user of the image would be able to read the proxy configuration that had been set during build from /etc/default/jenkins or the apt-get config file. The proxy config may easily contain sensitive information as the proxy user/password that were set on the host that built the image.

In order to avoid the problems pointed out above, we chose to:

1. Remove the ARG/ENV instantiations of http_proxy from the Dockerfile and use only the builtin docker ARG http_proxy. This will prevent the proxy variable to be available in a running container and in the image history.

2. Instead of using the apt-get global config file, we're just passing http_proxy as a local environment variable to the RUN instruction in which apt-get is being executed.

3. Jenkins /etc/default/jenkins JAVA_ARGS (that contain the proxy information) is now being updated by a script that's been source by /etc/default/jenkins itself. This means that, whenever service jenkins is called, that script will run and append the proxy configurations to the JAVA_ARGS at runtime, not replacing the JAVA_ARGS that are stored in the file.

4. http_proxy variables are being passed as arguments to docker create as well. By doing that, we make sure that the entrypoint will be running with the http_proxy values of the user.

---
 Dockerfile                                            | 19 ++-----------------
 frontend-install/setup/jenkins/set-java-args-proxy.sh |  7 +++++++
 frontend-install/setup/jenkins/setup.sh               | 15 ++++-----------
 fuego-host-scripts/docker-build-image.sh              |  4 +++-
 fuego-host-scripts/docker-create-container.sh         |  2 ++
 .../docker-create-usb-privileged-container.sh         |  2 ++
 6 files changed, 20 insertions(+), 29 deletions(-)
 create mode 100755 frontend-install/setup/jenkins/set-java-args-proxy.sh
Dockerfile +2 -17
index 493414e..c39454d 100644
@@ -6,14 +6,6 @@
 FROM debian:jessie
 MAINTAINER [email protected]
 
-# ==============================================================================
-# Proxy variables
-# ==============================================================================
-
-ARG HTTP_PROXY
-ENV http_proxy ${HTTP_PROXY}
-ENV https_proxy ${HTTP_PROXY}
-
 # ==============================================================================
 # Prepare basic image
 # ==============================================================================
@@ -21,10 +13,7 @@ ENV https_proxy ${HTTP_PROXY}
 WORKDIR /
 COPY frontend-install/apt/sources/fuego-debian-jessie.list \
         /etc/apt/sources.list.d/fuego-debian-jessie.list
-RUN if [ -n "$HTTP_PROXY" ]; then \
-        echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; \
-    fi && \
-    DEBIAN_FRONTEND=noninteractive apt-get update && \
+RUN DEBIAN_FRONTEND=noninteractive apt-get update && \
     apt-get -yV install \
         apt-utils \
         at \
@@ -84,11 +73,7 @@ RUN if [ -n "$HTTP_PROXY" ]; then \
         xmlstarlet && \
     rm -rf /var/lib/apt/lists/*
 
-RUN echo dash dash/sh boolean false | debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash && \
-    if [ -n "$HTTP_PROXY" ]; then \
-        echo "use_proxy = on" >> /etc/wgetrc; \
-        echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; \
-    fi
+RUN echo dash dash/sh boolean false | debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
 
 RUN pip install \
         filelock \
frontend-install/setup/jenkins/set-java-args-proxy.sh new file +7 -0
new file mode 100755
index 0000000..9b04f40
@@ -0,0 +1,7 @@
+#!/bin/bash
+
+if [ -n "${http_proxy}" ]; then
+	PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/');
+	PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/');
+	JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost=${PROXYSERVER} -Dhttp.proxyPort=${PROXYPORT} -Dhttps.proxyHost=${PROXYSERVER} -Dhttps.proxyPort=${PROXYPORT}";
+fi
frontend-install/setup/jenkins/setup.sh +4 -11
index 11ad371..b95fa91 100755
@@ -2,17 +2,10 @@
 
 set -e
 
-cp config.xml jenkins.model.JenkinsLocationConfiguration.xml ${JENKINS_HOME}
+cp config.xml jenkins.model.JenkinsLocationConfiguration.xml "${JENKINS_HOME}"
 
-source /etc/default/jenkins
-JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego"
-sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
+echo 'JENKINS_ARGS="${JENKINS_ARGS} --prefix=/fuego"' >> /etc/default/jenkins
+echo 'JAVA_ARGS="${JAVA_ARGS} -Djenkins.install.runSetupWizard=false"' >> /etc/default/jenkins
 
-JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false"
-if [ -n "$HTTP_PROXY" ]; then
-	PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/');
-	PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/');
-	JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost=${PROXYSERVER} -Dhttp.proxyPort=${PROXYPORT} -Dhttps.proxyHost=${PROXYSERVER} -Dhttps.proxyPort=${PROXYPORT}";
-fi
-sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins
+echo "source /setup/jenkins/set-java-args-proxy.sh" >> /etc/default/jenkins
 
fuego-host-scripts/docker-build-image.sh +3 -1
index 6550a98..d276ee0 100755
@@ -2,4 +2,6 @@
 # $1 - name for the docker image (default: fuego)
 DOCKERIMAGE=${1:-fuego}
 
-sudo docker build -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy .
+sudo docker build -t ${DOCKERIMAGE} \
+    --build-arg http_proxy=${http_proxy} \
+    --build-arg https_proxy=${https_proxy} .
fuego-host-scripts/docker-create-container.sh +2 -0
index 20f4b59..9e3d61a 100755
@@ -22,5 +22,7 @@ sudo docker create -it --name ${DOCKERCONTAINER} \
     -v $DIR/../fuego-rw:/fuego-rw \
     -v $DIR/../fuego-ro:/fuego-ro:ro \
     -v $DIR/../../fuego-core:/fuego-core:ro \
+    -e http_proxy=${http_proxy} \
+    -e https_proxy=${https_proxy:-$http_proxy} \
     --net="host" ${DOCKERIMAGE} || \
     echo "Could not create fuego-container. See error messages."
fuego-host-scripts/docker-create-usb-privileged-container.sh +2 -0
index b3a55c4..2431214 100755
@@ -28,5 +28,7 @@ sudo docker create -it --name ${DOCKERCONTAINER} \
     -v $DIR/../fuego-rw:/fuego-rw \
     -v $DIR/../fuego-ro:/fuego-ro:ro \
     -v $DIR/../../fuego-core:/fuego-core:ro \
+    -e http_proxy=${http_proxy} \
+    -e https_proxy=${https_proxy:-$http_proxy} \
     --net="host" ${DOCKERIMAGE} || \
     echo "Could not create fuego-container. See error messages."
Signature · 1 line
2.15.1
#15 lore raw

Re: [Fuego] [PATCH 6/9] Cleanup proxy

OK - I had to modify this because I skipped patch 5/9, but I applied it.

I'm not running this from behind a proxy, so I'll have to find someone to test this.
-- Tim

Quoted 215 lines

-----Original Message-----
From: [email protected] [mailto:fuego- [email protected]] On Behalf Of Guilherme Campos Camargo
Sent: Friday, January 26, 2018 9:36 AM
To: [email protected]
Subject: [Fuego] [PATCH 6/9] Cleanup proxy

Prior to this patch, http_proxy was being set and saved during build time according to the http_proxy values of the host.

This approach has at least two shortcomings, that are explained below:

1. Any other user that wanted to use a prebuilt image in their host would be unable to do so, because the proxy configuration is matching the host of the builder and not the host of the user.

2. A user of the image would be able to read the proxy configuration that had been set during build from /etc/default/jenkins or the apt-get config file. The proxy config may easily contain sensitive information as the proxy user/password that were set on the host that built the image.

In order to avoid the problems pointed out above, we chose to:

1. Remove the ARG/ENV instantiations of http_proxy from the Dockerfile and use only the builtin docker ARG http_proxy. This will prevent the proxy variable to be available in a running container and in the image history.

2. Instead of using the apt-get global config file, we're just passing http_proxy as a local environment variable to the RUN instruction in which apt-get is being executed.

3. Jenkins /etc/default/jenkins JAVA_ARGS (that contain the proxy information) is now being updated by a script that's been source by /etc/default/jenkins itself. This means that, whenever service jenkins is called, that script will run and append the proxy configurations to the JAVA_ARGS at runtime, not replacing the JAVA_ARGS that are stored in the file.

4. http_proxy variables are being passed as arguments to docker create as well. By doing that, we make sure that the entrypoint will be running with the http_proxy values of the user.

---
 Dockerfile                                            | 19 ++-----------------
 frontend-install/setup/jenkins/set-java-args-proxy.sh |  7 +++++++
 frontend-install/setup/jenkins/setup.sh               | 15 ++++-----------
 fuego-host-scripts/docker-build-image.sh              |  4 +++-
 fuego-host-scripts/docker-create-container.sh         |  2 ++
 .../docker-create-usb-privileged-container.sh         |  2 ++
 6 files changed, 20 insertions(+), 29 deletions(-)
 create mode 100755 frontend-install/setup/jenkins/set-java-args-proxy.sh
Dockerfile +0 -1
index 493414e..c39454d 100644
@@ -6,14 +6,6 @@
 FROM debian:jessie
 MAINTAINER [email protected]
 
-#

==========================================================
====================

Dockerfile +0 -2
-# Proxy variables
-#

==========================================================
====================

Dockerfile +0 -5
-
-ARG HTTP_PROXY
-ENV http_proxy ${HTTP_PROXY}
-ENV https_proxy ${HTTP_PROXY}
-
 #

==========================================================
====================
# Prepare basic image
#
==========================================================
====================

Dockerfile +1 -4
@@ -21,10 +13,7 @@ ENV https_proxy ${HTTP_PROXY}
 WORKDIR /
 COPY frontend-install/apt/sources/fuego-debian-jessie.list \
         /etc/apt/sources.list.d/fuego-debian-jessie.list
-RUN if [ -n "$HTTP_PROXY" ]; then \
-        echo 'Acquire::http::proxy "'$HTTP_PROXY'";' >
 /etc/apt/apt.conf.d/80proxy; \
-    fi && \
-    DEBIAN_FRONTEND=noninteractive apt-get update && \
+RUN DEBIAN_FRONTEND=noninteractive apt-get update && \
     apt-get -yV install \
         apt-utils \
         at \
Dockerfile +1 -5
@@ -84,11 +73,7 @@ RUN if [ -n "$HTTP_PROXY" ]; then \
         xmlstarlet && \
     rm -rf /var/lib/apt/lists/*
 
-RUN echo dash dash/sh boolean false | debconf-set-selections ;
 DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash && \
-    if [ -n "$HTTP_PROXY" ]; then \
-        echo "use_proxy = on" >> /etc/wgetrc; \
-        echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >>
 /etc/environment; \
-    fi
+RUN echo dash dash/sh boolean false | debconf-set-selections ;

DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash

 RUN pip install \
         filelock \
frontend-install/setup/jenkins/set-java-args-proxy.sh new file +4 -0
new file mode 100755
index 0000000..9b04f40
@@ -0,0 +1,7 @@
+#!/bin/bash
+
+if [ -n "${http_proxy}" ]; then
+	PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E

's/\///g' | sed -E 's/(.*):(.*)/\1/');

frontend-install/setup/jenkins/set-java-args-proxy.sh new file +1 -0
+	PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E

's/\///g' | sed -E 's/(.*):(.*)/\2/');

frontend-install/setup/jenkins/set-java-args-proxy.sh new file +1 -0
+	JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost=${PROXYSERVER} -

Dhttp.proxyPort=${PROXYPORT} -Dhttps.proxyHost=${PROXYSERVER} - Dhttps.proxyPort=${PROXYPORT}";

frontend-install/setup/jenkins/set-java-args-proxy.sh new file +1 -0
+fi
frontend- renamed from frontend-install/setup/jenkins/setup.sh +0 -0

install/setup/jenkins/setup.sh
index 11ad371..b95fa91 100755

frontend-install/setup/jenkins/setup.sh +1 -1
@@ -2,17 +2,10 @@
 
 set -e
 
-cp config.xml jenkins.model.JenkinsLocationConfiguration.xml
 ${JENKINS_HOME}
+cp config.xml jenkins.model.JenkinsLocationConfiguration.xml

"${JENKINS_HOME}"

frontend-install/setup/jenkins/setup.sh +0 -3
-source /etc/default/jenkins
-JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego"
-sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g"

/etc/default/jenkins
+echo 'JENKINS_ARGS="${JENKINS_ARGS} --prefix=/fuego"' >> /etc/default/jenkins
+echo 'JAVA_ARGS="${JAVA_ARGS} - Djenkins.install.runSetupWizard=false"' >> /etc/default/jenkins

frontend-install/setup/jenkins/setup.sh +0 -3
-JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false"
-if [ -n "$HTTP_PROXY" ]; then
-	PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E

's/\///g' | sed -E 's/(.*):(.*)/\1/');

frontend-install/setup/jenkins/setup.sh +0 -1
-	PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E

's/\///g' | sed -E 's/(.*):(.*)/\2/');

frontend-install/setup/jenkins/setup.sh +0 -1
-	JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost=${PROXYSERVER} -

Dhttp.proxyPort=${PROXYPORT} -Dhttps.proxyHost=${PROXYSERVER} - Dhttps.proxyPort=${PROXYPORT}";

frontend-install/setup/jenkins/setup.sh +0 -2
-fi
-sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g"

/etc/default/jenkins
+echo "source /setup/jenkins/set-java-args-proxy.sh" >> /etc/default/jenkins

fuego-host- renamed from fuego-host-scripts/docker-build-image.sh +0 -0

scripts/docker-build-image.sh
index 6550a98..d276ee0 100755

fuego-host-scripts/docker-build-image.sh +0 -1
@@ -2,4 +2,6 @@
 # $1 - name for the docker image (default: fuego)
 DOCKERIMAGE=${1:-fuego}
 
-sudo docker build -t ${DOCKERIMAGE} --build-arg

HTTP_PROXY=$http_proxy .

fuego-host-scripts/docker-build-image.sh +3 -0
+sudo docker build -t ${DOCKERIMAGE} \
+    --build-arg http_proxy=${http_proxy} \
+    --build-arg https_proxy=${https_proxy} .
fuego-host- renamed from fuego-host-scripts/docker-create-container.sh +0 -0

scripts/docker-create-container.sh
index 20f4b59..9e3d61a 100755

fuego-host-scripts/docker-create-container.sh +2 -0
@@ -22,5 +22,7 @@ sudo docker create -it --name ${DOCKERCONTAINER} \
     -v $DIR/../fuego-rw:/fuego-rw \
     -v $DIR/../fuego-ro:/fuego-ro:ro \
     -v $DIR/../../fuego-core:/fuego-core:ro \
+    -e http_proxy=${http_proxy} \
+    -e https_proxy=${https_proxy:-$http_proxy} \
     --net="host" ${DOCKERIMAGE} || \
     echo "Could not create fuego-container. See error messages."
fuego-host-scripts/docker-create-usb-privileged-container.sh +2 -1
index b3a55c4..2431214 100755
@@ -28,5 +28,7 @@ sudo docker create -it --name ${DOCKERCONTAINER} \
     -v $DIR/../fuego-rw:/fuego-rw \
     -v $DIR/../fuego-ro:/fuego-ro:ro \
     -v $DIR/../../fuego-core:/fuego-core:ro \
+    -e http_proxy=${http_proxy} \
+    -e https_proxy=${https_proxy:-$http_proxy} \
     --net="host" ${DOCKERIMAGE} || \
     echo "Could not create fuego-container. See error messages."
--

2.15.1

_______________________________________________
Fuego mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/fuego

#16 lore raw

PATCH 7/9 Move dpkg-reconfigure dash to first RUN instruction

In reply to Guilherme Campos Camargo

DEBIAN_FRONTEND=noninteractive has been explicitely set as an ARG in the beginning of the Dockerfile. That environment variable is set only during build time, not being exported with the container image.

---
 Dockerfile | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)
Dockerfile +7 -6
index c39454d..499dd52 100644
@@ -10,10 +10,11 @@ MAINTAINER [email protected]
 # Prepare basic image
 # ==============================================================================
 
-WORKDIR /
+ARG DEBIAN_FRONTEND=noninteractive
+
 COPY frontend-install/apt/sources/fuego-debian-jessie.list \
         /etc/apt/sources.list.d/fuego-debian-jessie.list
-RUN DEBIAN_FRONTEND=noninteractive apt-get update && \
+RUN apt-get update && \
     apt-get -yV install \
         apt-utils \
         at \
@@ -71,9 +72,9 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get update && \
         vim \
         wget \
         xmlstarlet && \
-    rm -rf /var/lib/apt/lists/*
-
-RUN echo dash dash/sh boolean false | debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
+    rm -rf /var/lib/apt/lists/* && \
+    echo dash dash/sh boolean false | debconf-set-selections && \
+    dpkg-reconfigure dash
 
 RUN pip install \
         filelock \
@@ -83,7 +84,7 @@ RUN pip install \
 RUN echo deb http://emdebian.org/tools/debian/ jessie main > /etc/apt/sources.list.d/crosstools.list && \
     curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | apt-key add - && \
     dpkg --add-architecture armhf && \
-    DEBIAN_FRONTEND=noninteractive apt-get update && \
+    apt-get update && \
     apt-get -yV install \
         binutils-arm-linux-gnueabihf \
         cpp-arm-linux-gnueabihf \
Signature · 1 line
2.15.1
#17 lore raw

Re: [Fuego] [PATCH 7/9] Move dpkg-reconfigure dash to first RUN instruction

Looks good. Applied.
-- Tim

Quoted 71 lines

-----Original Message-----
From: [email protected] [mailto:fuego- [email protected]] On Behalf Of Guilherme Campos Camargo
Sent: Friday, January 26, 2018 9:36 AM
To: [email protected]
Subject: [Fuego] [PATCH 7/9] Move dpkg-reconfigure dash to first RUN instruction

DEBIAN_FRONTEND=noninteractive has been explicitely set as an ARG in the beginning of the Dockerfile. That environment variable is set only during build time, not being exported with the container image.

---
 Dockerfile | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)
Dockerfile +0 -0
index c39454d..499dd52 100644
@@ -10,10 +10,11 @@ MAINTAINER [email protected]
 # Prepare basic image
 #

==========================================================
====================

Dockerfile +3 -5
-WORKDIR /
+ARG DEBIAN_FRONTEND=noninteractive
+
 COPY frontend-install/apt/sources/fuego-debian-jessie.list \
         /etc/apt/sources.list.d/fuego-debian-jessie.list
-RUN DEBIAN_FRONTEND=noninteractive apt-get update && \
+RUN apt-get update && \
     apt-get -yV install \
         apt-utils \
         at \
@@ -71,9 +72,9 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get
 update && \
         vim \
         wget \
         xmlstarlet && \
-    rm -rf /var/lib/apt/lists/*
-
-RUN echo dash dash/sh boolean false | debconf-set-selections ;

DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash

Dockerfile +4 -1
+    rm -rf /var/lib/apt/lists/* && \
+    echo dash dash/sh boolean false | debconf-set-selections && \
+    dpkg-reconfigure dash
 
 RUN pip install \
         filelock \
@@ -83,7 +84,7 @@ RUN pip install \
 RUN echo deb http://emdebian.org/tools/debian/ jessie main >
 /etc/apt/sources.list.d/crosstools.list && \
     curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key |
 apt-key add - && \
     dpkg --add-architecture armhf && \
-    DEBIAN_FRONTEND=noninteractive apt-get update && \
+    apt-get update && \
     apt-get -yV install \
         binutils-arm-linux-gnueabihf \
         cpp-arm-linux-gnueabihf \

--
2.15.1

_______________________________________________
Fuego mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/fuego

#18 lore raw

PATCH 8/9 Refactor plugin installation script

In reply to Guilherme Campos Camargo

On this patch we're trying to remove the explicit sleeps from the plugin installation script, that are necessary for waiting for Jenkins start/restart and the installation of the plugins.

With this change we hope that faster hosts (or hosts with faster networks) will be able to finish Jenkins plugins installation sooner.

We also took the chance to remove the call to sudo, that are not necessary for using jenkins-cli.

---
 frontend-install/setup/jenkins/plugins/install.sh | 44 +++++++++++++++++------
 1 file changed, 33 insertions(+), 11 deletions(-)
frontend-install/setup/jenkins/plugins/install.sh +33 -11
index 0db8bf2..37e8948 100755
@@ -2,20 +2,42 @@
 
 set -e
 
+function install_jenkins_plugin {
+    echo "Waiting for Jenkins update center..."
+    while ! ret=$(java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin "$1" 2>/dev/null); do
+        sleep "$2"
+    done
+    echo "${ret}"
+}
+
+function wait_for_jenkins {
+    echo "Waiting for Jenkins..."
+    while ! curl --output /dev/null --silent --head --fail http://localhost:8080/fuego/ ; do
+        sleep "$1"
+    done
+}
+
+readonly sleep_time_s=1
+readonly required_plugins="
+   description-setter
+   pegdown-formatter
+   flot-plotter-plugin/flot.hpi
+   "
+
 service jenkins start
-sleep 30
-sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin description-setter
-sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin pegdown-formatter
-sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin flot-plotter-plugin/flot.hpi
-sleep 10
+wait_for_jenkins ${sleep_time_s}
+
+for plugin in ${required_plugins}; do
+    install_jenkins_plugin "${plugin}" "${sleep_time_s}"
+done
 
 # Let Jenkins install flot before making the symlink
 service jenkins restart
-sleep 30
-rm $JENKINS_HOME/plugins/flot/flot/mod.js
-ln -s /fuego-core/engine/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
-ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
-ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/
+wait_for_jenkins 1
 
-chown -R jenkins:jenkins $JENKINS_HOME/
+rm "$JENKINS_HOME/plugins/flot/flot/mod.js"
+ln -s /fuego-core/engine/scripts/mod.js "$JENKINS_HOME/plugins/flot/flot/mod.js"
+ln -s /fuego-rw/logs "$JENKINS_HOME/userContent/fuego.logs"
+ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/
 
+chown -R jenkins:jenkins "$JENKINS_HOME/"
Signature · 1 line
2.15.1
#19 lore raw

Re: [Fuego] [PATCH 8/9] Refactor plugin installation script

OK - applied.

Thanks.
-- Tim

Quoted 86 lines

-----Original Message-----
From: Guilherme Campos Camargo on Friday, January 26, 2018 9:36 AM On this patch we're trying to remove the explicit sleeps from the plugin installation script, that are necessary for waiting for Jenkins start/restart and the installation of the plugins.

With this change we hope that faster hosts (or hosts with faster networks) will be able to finish Jenkins plugins installation sooner.

We also took the chance to remove the call to sudo, that are not necessary for using jenkins-cli.

---
 frontend-install/setup/jenkins/plugins/install.sh | 44 +++++++++++++++++-
-----
 1 file changed, 33 insertions(+), 11 deletions(-)
frontend- renamed from frontend-install/setup/jenkins/plugins/install.sh +0 -0

install/setup/jenkins/plugins/install.sh
index 0db8bf2..37e8948 100755

frontend-install/setup/jenkins/plugins/install.sh +3 -0
@@ -2,20 +2,42 @@
 
 set -e
 
+function install_jenkins_plugin {
+    echo "Waiting for Jenkins update center..."
+    while ! ret=$(java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s

http://localhost:8080/fuego install-plugin "$1" 2>/dev/null); do

frontend-install/setup/jenkins/plugins/install.sh +8 -0
+        sleep "$2"
+    done
+    echo "${ret}"
+}
+
+function wait_for_jenkins {
+    echo "Waiting for Jenkins..."
+    while ! curl --output /dev/null --silent --head --fail

http://localhost:8080/fuego/ ; do

frontend-install/setup/jenkins/plugins/install.sh +11 -2
+        sleep "$1"
+    done
+}
+
+readonly sleep_time_s=1
+readonly required_plugins="
+   description-setter
+   pegdown-formatter
+   flot-plotter-plugin/flot.hpi
+   "
+
 service jenkins start
-sleep 30
-sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s

http://localhost:8080/fuego install-plugin description-setter -sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s
http://localhost:8080/fuego install-plugin pegdown-formatter -sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s
http://localhost:8080/fuego install-plugin flot-plotter-plugin/flot.hpi

frontend-install/setup/jenkins/plugins/install.sh +5 -4
-sleep 10
+wait_for_jenkins ${sleep_time_s}
+
+for plugin in ${required_plugins}; do
+    install_jenkins_plugin "${plugin}" "${sleep_time_s}"
+done
 
 # Let Jenkins install flot before making the symlink
 service jenkins restart
-sleep 30
-rm $JENKINS_HOME/plugins/flot/flot/mod.js
-ln -s /fuego-core/engine/scripts/mod.js

$JENKINS_HOME/plugins/flot/flot/mod.js

frontend-install/setup/jenkins/plugins/install.sh +3 -3
-ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
-ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/
+wait_for_jenkins 1
 
-chown -R jenkins:jenkins $JENKINS_HOME/
+rm "$JENKINS_HOME/plugins/flot/flot/mod.js"
+ln -s /fuego-core/engine/scripts/mod.js

"$JENKINS_HOME/plugins/flot/flot/mod.js"

frontend-install/setup/jenkins/plugins/install.sh +3 -1
+ln -s /fuego-rw/logs "$JENKINS_HOME/userContent/fuego.logs"
+ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/
 
+chown -R jenkins:jenkins "$JENKINS_HOME/"
--

2.15.1

#20 lore raw

PATCH 9/9 Set depth to "one" and download just a single-branch on git clones

In reply to Guilherme Campos Camargo

In order to improve git clone speeds

---
 Dockerfile | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
Dockerfile +6 -3
index 499dd52..0eeeca0 100644
@@ -120,13 +120,16 @@ RUN ./install.sh
 COPY frontend-install/setup/serial /setup/serial
 WORKDIR /setup
 
-RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc && \
+RUN git clone https://github.com/tbird20d/ttc.git \
+        --depth 1 --single-branch /usr/local/src/ttc && \
     ./serial/setup-ttc.sh
 
-RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio && \
+RUN git clone https://github.com/frowand/serio.git \
+        --depth 1 --single-branch /usr/local/src/serio && \
     ./serial/setup-serio.sh
 
-RUN git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin && \
+RUN git clone https://github.com/tbird20d/serlogin.git \
+        --depth 1 --single-branch /usr/local/src/serlogin && \
     ./serial/setup-serlogin.sh
 
 COPY frontend-install/setup/lava /setup/lava
Signature · 1 line
2.15.1
#21 lore raw

Re: [Fuego] [PATCH 9/9] Set depth to "one" and download just a single-branch on git clones

Quoted 5 lines

-----Original Message-----
From: Guilherme Campos Camargo on Friday, January 26, 2018 9:36 AM In order to improve git clone speeds

For repositories this small, this does not speed up the clone operation appreciably. Since it makes the clone commands a bit ugly, I don't think the tradeoff is worth it.

I tested this on my machine (with a relatively fast network), and the speed for all these clones, without the extra args, was under 1 second each. The time difference with the extra args was less than 100 milliseconds.

Do these take longer for you? These are relatively small repositories. -- Tim

Quoted 39 lines
---
 Dockerfile | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
Dockerfile +6 -3
index 499dd52..0eeeca0 100644
@@ -120,13 +120,16 @@ RUN ./install.sh
 COPY frontend-install/setup/serial /setup/serial
 WORKDIR /setup
 
-RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc && \
+RUN git clone https://github.com/tbird20d/ttc.git \
+        --depth 1 --single-branch /usr/local/src/ttc && \
     ./serial/setup-ttc.sh
 
-RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio &&
 \
+RUN git clone https://github.com/frowand/serio.git \
+        --depth 1 --single-branch /usr/local/src/serio && \
     ./serial/setup-serio.sh
 
-RUN git clone https://github.com/tbird20d/serlogin.git
 /usr/local/src/serlogin && \
+RUN git clone https://github.com/tbird20d/serlogin.git \
+        --depth 1 --single-branch /usr/local/src/serlogin && \
     ./serial/setup-serlogin.sh

COPY frontend-install/setup/lava /setup/lava
--
2.15.1

_______________________________________________
Fuego mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/fuego