Docker Improvements
cover Docker Improvements
Hello, everyone
This series of patches implements a few improvements on the Docker infrastructure with the ultimate goal of allowing an image to be shared amoung different users.
One of the benefits of this approach is that it allows the build-from-scratch to be optional, providing the user much faster set-up times.
The changes involve, in short:
* Reorganization of the Dockerfile aiming to reduce the number of layers present in the resulting image (by reducing the number of RUN/COPY statements) and to reduce cache misses by moving components that change less frequently to the beginning of the Dockerfile.
* Remove persistenct of proxy and other configurations that might result in conflicts to the user of the image (that's not using it at the same host where the image was built).
We tried to provide an in-depth reasoning for each one of the proposed changes in the commit messages and we'll be glad to hear your inputs about them.
Thanks
--
Guilherme
ProFUSION Embedded Systems
Guilherme Campos Camargo (9):
Remove unused variables from docker build/create/run scripts Refactor preinst and apt-get/pip install
Use curl for downloading Jenkins
Move configuration and jenkins plugin installation to separate scripts Remove jenkins user created during build time with host's uid/gid Cleanup proxy
Move dpkg-reconfigure dash to first RUN instruction Refactor plugin installation script
Set depth to "one" and download just a single-branch on git clones
Dockerfile | 215 +++++++++++----------
.../apt/sources/fuego-debian-jessie.list | 2 +
frontend-install/entrypoint.sh | 13 ++
frontend-install/{ => setup/jenkins}/config.xml | 0
.../jenkins.model.JenkinsLocationConfiguration.xml | 0
.../setup/jenkins/map-jenkins-uid-to-host.sh | 12 ++
.../plugins/flot-plotter-plugin/.gitignore | 0
.../jenkins}/plugins/flot-plotter-plugin/flot.hpi | Bin
.../jenkins}/plugins/flot-plotter-plugin/pom.xml | 0
.../src/main/java/flotile/ChartAction.java | 0
.../src/main/java/flotile/FlotPublisher.java | 0
.../flotile/ChartAction/floatingBox.jelly | 0
.../src/main/resources/index.jelly | 0
.../src/main/webapp/flot/.copyarea.db | 0
.../src/main/webapp/flot/API.txt | 0
.../src/main/webapp/flot/FAQ.txt | 0
.../src/main/webapp/flot/LICENSE.txt | 0
.../src/main/webapp/flot/PLUGINS.txt | 0
.../src/main/webapp/flot/README.txt | 0
.../src/main/webapp/flot/base64.js | 0
.../src/main/webapp/flot/canvas2image.js | 0
.../src/main/webapp/flot/excanvas.min.js | 0
.../main/webapp/flot/jquery.colorhelpers.min.js | 0
.../main/webapp/flot/jquery.flot.crosshair.min.js | 0
.../webapp/flot/jquery.flot.fillbetween.min.js | 0
.../src/main/webapp/flot/jquery.flot.image.min.js | 0
.../src/main/webapp/flot/jquery.flot.min.js | 0
.../main/webapp/flot/jquery.flot.navigate.min.js | 0
.../src/main/webapp/flot/jquery.flot.pie.min.js | 0
.../src/main/webapp/flot/jquery.flot.resize.min.js | 0
.../main/webapp/flot/jquery.flot.selection.min.js | 0
.../src/main/webapp/flot/jquery.flot.stack.min.js | 0
.../src/main/webapp/flot/jquery.flot.symbol.min.js | 0
.../main/webapp/flot/jquery.flot.threshold.min.js | 0
.../src/main/webapp/flot/jquery.min.js | 0
.../src/main/webapp/flot/mod.js | 0
.../src/main/webapp/flot/my.css | 0
.../src/main/webapp/help-globalConfig.html | 0
frontend-install/setup/jenkins/plugins/install.sh | 43 +++++
.../setup/jenkins/set-java-args-proxy.sh | 7 +
frontend-install/setup/jenkins/setup.sh | 11 ++
frontend-install/setup/lava/setup.sh | 11 ++
frontend-install/setup/serial/setup-serio.sh | 8 +
frontend-install/setup/serial/setup-serlogin.sh | 6 +
frontend-install/setup/serial/setup-ttc.sh | 7 +
fuego-host-scripts/docker-build-image.sh | 21 +-
fuego-host-scripts/docker-create-container.sh | 2 +
.../docker-create-usb-privileged-container.sh | 2 +
fuego-host-scripts/docker-remove-container.sh | 8 -
fuego-host-scripts/docker-remove-image.sh | 9 +-
fuego-host-scripts/docker-start-container.sh | 8 -
fuego-host-scripts/jenkins-map-uid.sh | 18 ++
52 files changed, 258 insertions(+), 145 deletions(-)
create mode 100644 frontend-install/apt/sources/fuego-debian-jessie.list
create mode 100755 frontend-install/entrypoint.sh
rename frontend-install/{ => setup/jenkins}/config.xml (100%)
rename frontend-install/{ => setup/jenkins}/jenkins.model.JenkinsLocationConfiguration.xml (100%)
create mode 100755 frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/.gitignore (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/flot.hpi (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/pom.xml (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/java/flotile/ChartAction.java (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/java/flotile/FlotPublisher.java (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/resources/index.jelly (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/.copyarea.db (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/API.txt (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/FAQ.txt (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/LICENSE.txt (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/PLUGINS.txt (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/README.txt (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/base64.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/canvas2image.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/excanvas.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.colorhelpers.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.image.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.navigate.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.pie.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.resize.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.selection.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.stack.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.symbol.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.threshold.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/mod.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/my.css (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/help-globalConfig.html (100%)
create mode 100755 frontend-install/setup/jenkins/plugins/install.sh
create mode 100755 frontend-install/setup/jenkins/set-java-args-proxy.sh
create mode 100755 frontend-install/setup/jenkins/setup.sh
create mode 100755 frontend-install/setup/lava/setup.sh
create mode 100755 frontend-install/setup/serial/setup-serio.sh
create mode 100755 frontend-install/setup/serial/setup-serlogin.sh
create mode 100755 frontend-install/setup/serial/setup-ttc.sh
create mode 100755 fuego-host-scripts/jenkins-map-uid.shSignature · 1 line
2.15.1
PATCH 1/9 Remove unused variables from docker build/create/run scripts
The DIR variable is only being used by the CREATE scripts. There's no reason to keep it in the BUILD/REMOVE/START scripts as well.
--- fuego-host-scripts/docker-build-image.sh | 9 +-------- fuego-host-scripts/docker-remove-container.sh | 8 -------- fuego-host-scripts/docker-remove-image.sh | 9 +-------- fuego-host-scripts/docker-start-container.sh | 8 -------- 4 files changed, 2 insertions(+), 32 deletions(-)
fuego-host-scripts/docker-build-image.sh +1 -8
@@ -1,12 +1,5 @@ +#!/bin/bash # $1 - name for the docker image (default: fuego) -SOURCE="${BASH_SOURCE[0]}" -while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink - DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" - SOURCE="$(readlink "$SOURCE")" - [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located -done -DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" - DOCKERIMAGE=${1:-fuego} if [ "$(id -u)" == "0" ]; then
fuego-host-scripts/docker-remove-container.sh +0 -8
@@ -1,13 +1,5 @@ #!/bin/bash # $1 - name for the docker container (default: fuego-container) -SOURCE="${BASH_SOURCE[0]}" -while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink - DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" - SOURCE="$(readlink "$SOURCE")" - [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located -done -DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" - DOCKERCONTAINER=${1:-fuego-container} sudo docker rm ${DOCKERCONTAINER}
fuego-host-scripts/docker-remove-image.sh +1 -8
@@ -1,12 +1,5 @@ +#!/bin/bash # $1 - name for the docker image (default: fuego) -SOURCE="${BASH_SOURCE[0]}" -while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink - DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" - SOURCE="$(readlink "$SOURCE")" - [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located -done -DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" - DOCKERIMAGE=${1:-fuego} sudo docker rmi ${DOCKERIMAGE}
fuego-host-scripts/docker-start-container.sh +0 -8
@@ -1,13 +1,5 @@ #!/bin/bash # $1 - name for the docker container (default: fuego-container) -SOURCE="${BASH_SOURCE[0]}" -while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink - DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" - SOURCE="$(readlink "$SOURCE")" - [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located -done -DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" - DOCKERCONTAINER=${1:-fuego-container} echo "Starting Fuego container (${DOCKERCONTAINER})"
Signature · 1 line
2.15.1
Re: [Fuego] [PATCH 1/9] Remove unused variables from docker build/create/run scripts
Quoted 105 lines
-----Original Message-----
From: Guilherme Campos CamargoThe DIR variable is only being used by the CREATE scripts. There's no reason to keep it in the BUILD/REMOVE/START scripts as well.
Signed-off-by Guilherme Campos Camargo <[email protected]>--- fuego-host-scripts/docker-build-image.sh | 9 +-------- fuego-host-scripts/docker-remove-container.sh | 8 -------- fuego-host-scripts/docker-remove-image.sh | 9 +-------- fuego-host-scripts/docker-start-container.sh | 8 -------- 4 files changed, 2 insertions(+), 32 deletions(-)fuego-host- renamed from fuego-host-scripts/docker-build-image.sh +0
-0
scripts/docker-build-image.sh
index a197dc5..7c11b11 100755fuego-host-scripts/docker-build-image.sh +1
-2@@ -1,12 +1,5 @@ +#!/bin/bash # $1 - name for the docker image (default: fuego) -SOURCE="${BASH_SOURCE[0]}" -while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer asymlink
fuego-host-scripts/docker-build-image.sh +0
-3- DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" - SOURCE="$(readlink "$SOURCE")" - [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was arelative symlink, we need to resolve it relative to the path where the symlink file was located
fuego-host-scripts/docker-build-image.sh +0
-3-done -DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" - DOCKERIMAGE=${1:-fuego} if [ "$(id -u)" == "0" ]; thenfuego-host- renamed from fuego-host-scripts/docker-remove-container.sh +0
-0
scripts/docker-remove-container.sh
index 2aae6a9..00c7ac9 100755fuego-host-scripts/docker-remove-container.sh +0
-2@@ -1,13 +1,5 @@ #!/bin/bash # $1 - name for the docker container (default: fuego-container) -SOURCE="${BASH_SOURCE[0]}" -while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer asymlink
fuego-host-scripts/docker-remove-container.sh +0
-3- DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" - SOURCE="$(readlink "$SOURCE")" - [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was arelative symlink, we need to resolve it relative to the path where the symlink file was located
fuego-host-scripts/docker-remove-container.sh +0
-3-done -DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" - DOCKERCONTAINER=${1:-fuego-container} sudo docker rm ${DOCKERCONTAINER}fuego-host- renamed from fuego-host-scripts/docker-remove-image.sh +0
-0
scripts/docker-remove-image.sh
index 91dbda7..4c24684 100755fuego-host-scripts/docker-remove-image.sh +1
-2@@ -1,12 +1,5 @@ +#!/bin/bash # $1 - name for the docker image (default: fuego) -SOURCE="${BASH_SOURCE[0]}" -while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer asymlink
fuego-host-scripts/docker-remove-image.sh +0
-3- DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" - SOURCE="$(readlink "$SOURCE")" - [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was arelative symlink, we need to resolve it relative to the path where the symlink file was located
fuego-host-scripts/docker-remove-image.sh +0
-3-done -DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" - DOCKERIMAGE=${1:-fuego} sudo docker rmi ${DOCKERIMAGE}fuego-host- renamed from fuego-host-scripts/docker-start-container.sh +0
-0
scripts/docker-start-container.sh
index d9de85a..d0cee1a 100755fuego-host-scripts/docker-start-container.sh +0
-2@@ -1,13 +1,5 @@ #!/bin/bash # $1 - name for the docker container (default: fuego-container) -SOURCE="${BASH_SOURCE[0]}" -while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer asymlink
fuego-host-scripts/docker-start-container.sh +0
-3- DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" - SOURCE="$(readlink "$SOURCE")" - [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was arelative symlink, we need to resolve it relative to the path where the symlink file was located
fuego-host-scripts/docker-start-container.sh +0
-4-done -DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" - DOCKERCONTAINER=${1:-fuego-container} echo "Starting Fuego container (${DOCKERCONTAINER})" --
Quoted 1 line
2.15.1
Nice cleanup. Thanks.
Applied.
-- Tim
PATCH 2/9 Refactor preinst and apt-get/pip install
In reply to Guilherme Campos CamargoAll base-image related apt-get commands (including pre-install config, as well as the package install) have been put together in the same RUN instruction.
The same has been done with the LAVA-related apt-get commands on this patch. They can't be merged because the requirements are using a specific key that needs to be downloaded with curl (that's installed in the previous step. Additionally, LAVA apt-get commands have been moved up in the Dockerfile, allowing that RUN instruction to be invalidated fewer times, because it does not depend on the files that have been copied from the host.
We have also moved apt sources to an external .list file, that's copied from the host to the container during build.
--- Dockerfile | 116 ++++++++++++++++----- .../apt/sources/fuego-debian-jessie.list | 2 + 2 files changed, 92 insertions(+), 26 deletions(-) create mode 100644 frontend-install/apt/sources/fuego-debian-jessie.list
Dockerfile +90 -26
@@ -19,25 +19,96 @@ ENV https_proxy ${HTTP_PROXY} # ============================================================================== WORKDIR / -RUN echo deb http://httpredir.debian.org/debian jessie main non-free > /etc/apt/sources.list -RUN echo deb http://httpredir.debian.org/debian jessie-updates main non-free >> /etc/apt/sources.list -RUN echo deb http://security.debian.org/ jessie/updates main >> /etc/apt/sources.list -RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi -RUN DEBIAN_FRONTEND=noninteractive apt-get update && apt-get -yV install \ - apt-utils daemon gcc make cmake python-paramiko python-lxml python-simplejson \ - python-matplotlib python-serial python-yaml python-openpyxl python-requests \ - libtool xmlstarlet autoconf automake rsync openjdk-7-jre openjdk-7-jdk iperf \ - netperf netpipe-tcp sshpass wget git diffstat sudo net-tools vim curl \ - inotify-tools g++ bzip2 bc libaio-dev gettext pkg-config libglib2.0-dev \ - time python-pip python-xmltodict at minicom lzop bsdmainutils u-boot-tools \ - mc netcat lava-tool openssh-server python-parsedatetime \ - libsdl1.2-dev libcairo2-dev libxmu-dev libxmuu-dev - -RUN pip install python-jenkins==0.4.14 -RUN pip install filelock -RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash' -RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi -RUN if [ -n "$HTTP_PROXY" ]; then echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; fi +COPY frontend-install/apt/sources/fuego-debian-jessie.list \ + /etc/apt/sources.list.d/fuego-debian-jessie.list +RUN if [ -n "$HTTP_PROXY" ]; then \ + echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; \ + fi && \ + DEBIAN_FRONTEND=noninteractive apt-get update && \ + apt-get -yV install \ + apt-utils \ + at \ + autoconf \ + automake \ + bc \ + bsdmainutils \ + bzip2 \ + cmake \ + curl \ + daemon \ + diffstat \ + g++ \ + gcc \ + gettext \ + git \ + inotify-tools \ + iperf \ + lava-tool \ + libaio-dev \ + libcairo2-dev \ + libglib2.0-dev \ + libsdl1.2-dev \ + libtool \ + libxmu-dev \ + libxmuu-dev \ + lzop \ + make \ + mc \ + minicom \ + net-tools \ + netcat \ + netperf \ + netpipe-tcp \ + openjdk-7-jdk \ + openjdk-7-jre \ + openssh-server \ + pkg-config \ + python-lxml \ + python-matplotlib \ + python-openpyxl \ + python-paramiko \ + python-parsedatetime \ + python-pip \ + python-requests \ + python-serial \ + python-simplejson \ + python-xmltodict \ + python-yaml \ + rsync \ + sshpass \ + sudo \ + time \ + u-boot-tools \ + vim \ + wget \ + xmlstarlet && \ + rm -rf /var/lib/apt/lists/* + +RUN echo dash dash/sh boolean false | debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash && \ + if [ -n "$HTTP_PROXY" ]; then \ + echo "use_proxy = on" >> /etc/wgetrc; \ + echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; \ + fi + +RUN pip install \ + filelock \ + python-jenkins==0.4.14 + +# TODO: Move toolchain-related instalation steps to a derivate image, like fuego:${version}-arhmhf +RUN echo deb http://emdebian.org/tools/debian/ jessie main > /etc/apt/sources.list.d/crosstools.list && \ + curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | apt-key add - && \ + dpkg --add-architecture armhf && \ + DEBIAN_FRONTEND=noninteractive apt-get update && \ + apt-get -yV install \ + binutils-arm-linux-gnueabihf \ + cpp-arm-linux-gnueabihf \ + crossbuild-essential-armhf \ + gcc-arm-linux-gnueabihf && \ + rm -rf /var/lib/apt/lists/* + +RUN pip install \ + filelock \ + python-jenkins==0.4.14 # ============================================================================== # Install Jenkins with the same UID/GID as the host user
@@ -125,13 +196,6 @@ RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin #RUN echo "fuego-create-node --board raspberrypi3" >> /root/firststart.sh #RUN echo "fuego-create-jobs --board raspberrypi3 --testplan testplan_docker --distrib nosyslogd.dist" >> /root/firststart.sh -RUN echo "deb http://emdebian.org/tools/debian/ jessie main" > /etc/apt/sources.list.d/crosstools.list -RUN dpkg --add-architecture armhf -RUN curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | sudo apt-key add - -RUN DEBIAN_FRONTEND=noninteractive apt-get update -RUN DEBIAN_FRONTEND=noninteractive apt-get -yV install crossbuild-essential-armhf cpp-arm-linux-gnueabihf gcc-arm-linux-gnueabihf binutils-arm-linux-gnueabihf - - # ============================================================================== # Setup startup command # ==============================================================================
frontend-install/apt/sources/fuego-debian-jessie.list new file +2 -0
@@ -0,0 +1,2 @@ +deb http://httpredir.debian.org/debian jessie main non-free +deb http://httpredir.debian.org/debian jessie-updates main non-free
Signature · 1 line
2.15.1
Re: [Fuego] [PATCH 2/9] Refactor preinst and apt-get/pip install
Quoted 7 lines
-----Original Message-----
From: Guilherme Campos Camargo
All base-image related apt-get commands (including pre-install config, as well as the package install) have been put together in the same RUN instruction.The same has been done with the LAVA-related apt-get commands on this
Do you mean Fuego-related?
Quoted 3 lines
patch. They can't be merged because the requirements are using a specific key that needs to be downloaded with curl (that's installed in the previous step. Additionally, LAVA apt-get commands have been moved
Another reference to LAVA?
Quoted 79 lines
up in the Dockerfile, allowing that RUN instruction to be invalidated fewer times, because it does not depend on the files that have been copied from the host.
We have also moved apt sources to an external .list file, that's copied from the host to the container during build.
Signed-off-by Guilherme Campos Camargo <[email protected]>--- Dockerfile | 116 ++++++++++++++++----- .../apt/sources/fuego-debian-jessie.list | 2 + 2 files changed, 92 insertions(+), 26 deletions(-) create mode 100644 frontend-install/apt/sources/fuego-debian-jessie.listDockerfile +0
-0@@ -19,25 +19,96 @@ ENV https_proxy ${HTTP_PROXY} #==========================================================
====================WORKDIR /
-RUN echo deb http://httpredir.debian.org/debian jessie main non-free >/etc/apt/sources.list
-RUN echo deb http://httpredir.debian.org/debian jessie-updates main non- free >>/etc/apt/sources.list
-RUN echo deb http://security.debian.org/ jessie/updates main >>/etc/apt/sources.list
-RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy"'$HTTP_PROXY'";' >/etc/apt/apt.conf.d/80proxy; fi
-RUN DEBIAN_FRONTEND=noninteractive apt-get update && apt-get -yV install \Dockerfile +0
-1- apt-utils daemon gcc make cmake python-paramiko python-lxmlpython-simplejson \
Dockerfile +0
-1- python-matplotlib python-serial python-yaml python-openpyxlpython-requests \
Dockerfile +0
-1- libtool xmlstarlet autoconf automake rsync openjdk-7-jre openjdk-7-jdk iperf \
Dockerfile +0
-2- netperf netpipe-tcp sshpass wget git diffstat sudo net-tools vim curl \ - inotify-tools g++ bzip2 bc libaio-dev gettext pkg-config libglib2.0-dev\
Dockerfile +0
-1- time python-pip python-xmltodict at minicom lzop bsdmainutils u-boot-tools \
Dockerfile +0
-6- mc netcat lava-tool openssh-server python-parsedatetime \ - libsdl1.2-dev libcairo2-dev libxmu-dev libxmuu-dev - -RUN pip install python-jenkins==0.4.14 -RUN pip install filelock -RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash'
Dockerfile +0
-2-RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi -RUN if [ -n "$HTTP_PROXY" ]; then echo -e"http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >>
/etc/environment; fiDockerfile +4
-0+COPY frontend-install/apt/sources/fuego-debian-jessie.list \ + /etc/apt/sources.list.d/fuego-debian-jessie.list +RUN if [ -n "$HTTP_PROXY" ]; then \ + echo 'Acquire::http::proxy "'$HTTP_PROXY'";' >
/etc/apt/apt.conf.d/80proxy; \Dockerfile +18
-0+ fi && \ + DEBIAN_FRONTEND=noninteractive apt-get update && \ + apt-get -yV install \ + apt-utils \ + at \ + autoconf \ + automake \ + bc \ + bsdmainutils \ + bzip2 \ + cmake \ + curl \ + daemon \ + diffstat \ + g++ \ + gcc \ + gettext \ + git \
Dockerfile ENV https_proxy ${HTTP_PROXY} · line 53
Quoted 10 lines
Dockerfile +10
-0+ inotify-tools \ + iperf \ + lava-tool \ + libaio-dev \ + libcairo2-dev \ + libglib2.0-dev \ + libsdl1.2-dev \ + libtool \ + libxmu-dev \ + libxmuu-dev \
Most of the above libs are also toolchain packages, for x86_64 (the docker virtual board) and would benefit from being moved to a separate image creation step or derivative image.
Quoted 102 lines
Dockerfile +34
-0+ lzop \ + make \ + mc \ + minicom \ + net-tools \ + netcat \ + netperf \ + netpipe-tcp \ + openjdk-7-jdk \ + openjdk-7-jre \ + openssh-server \ + pkg-config \ + python-lxml \ + python-matplotlib \ + python-openpyxl \ + python-paramiko \ + python-parsedatetime \ + python-pip \ + python-requests \ + python-serial \ + python-simplejson \ + python-xmltodict \ + python-yaml \ + rsync \ + sshpass \ + sudo \ + time \ + u-boot-tools \ + vim \ + wget \ + xmlstarlet && \ + rm -rf /var/lib/apt/lists/* + +RUN echo dash dash/sh boolean false | debconf-set-selections ;DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash && \
Dockerfile +3
-0+ if [ -n "$HTTP_PROXY" ]; then \ + echo "use_proxy = on" >> /etc/wgetrc; \ + echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >>
/etc/environment; \Dockerfile +7
-0+ fi + +RUN pip install \ + filelock \ + python-jenkins==0.4.14 + +# TODO: Move toolchain-related instalation steps to a derivate image, likefuego:${version}-arhmhf
+RUN echo deb http://emdebian.org/tools/debian/ jessie main >/etc/apt/sources.list.d/crosstools.list&& \
+ curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key
| apt-key add - && \Dockerfile +12
-0+ dpkg --add-architecture armhf && \ + DEBIAN_FRONTEND=noninteractive apt-get update && \ + apt-get -yV install \ + binutils-arm-linux-gnueabihf \ + cpp-arm-linux-gnueabihf \ + crossbuild-essential-armhf \ + gcc-arm-linux-gnueabihf && \ + rm -rf /var/lib/apt/lists/* + +RUN pip install \ + filelock \ + python-jenkins==0.4.14 #==========================================================
====================
# Install Jenkins with the same UID/GID as the host userDockerfile +0
-0@@ -125,13 +196,6 @@ RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown
/usr/local/bin
#RUN echo "fuego-create-node --board raspberrypi3" >>/root/firststart.sh#RUN echo "fuego-create-jobs --board raspberrypi3 --testplantestplan_docker--distrib nosyslogd.dist" >>/root/firststart.sh-RUN echo "deb http://emdebian.org/tools/debian/ jessie main" >
/etc/apt/sources.list.d/crosstools.listDockerfile +0
-2-RUN dpkg --add-architecture armhf -RUN curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | sudo apt-key add -
Dockerfile +0
-2-RUN DEBIAN_FRONTEND=noninteractive apt-get update -RUN DEBIAN_FRONTEND=noninteractive apt-get -yV install crossbuild-essential-armhf cpp-arm-linux-gnueabihf gcc-arm-linux-gnueabihf binutils- arm-linux-gnueabihf
Dockerfile +0
-2- - #==========================================================
====================
# Setup startup command
#
==========================================================
====================frontend- renamed from frontend-install/apt/sources/fuego-debian-jessie.list +0
-0install/apt/sources/fuego-debian-jessie.list
new file mode 100644
index 0000000..d224f15frontend-install/apt/sources/fuego-debian-jessie.list new file +2
-1@@ -0,0 +1,2 @@ +deb http://httpredir.debian.org/debian jessie main non-free +deb http://httpredir.debian.org/debian jessie-updates main non-free --
Quoted 1 line
2.15.1
On the whole, it looks OK. I've applied it as is (with, what I believe are a few commit description errors).
-- Tim
Re: [Fuego] [PATCH 2/9] Refactor preinst and apt-get/pip install
Quoted 6 lines · On Tue, Jan 30, 2018 at 5:22 PM, <[email protected]> wrote:
Quoted 6 lines
-----Original Message-----
From: Guilherme Campos Camargo
All base-image related apt-get commands (including pre-install config, as well as the package install) have been put together in the same RUN instruction.
Quoted 7 lines
Quoted 1 line
The same has been done with the LAVA-related apt-get commands on this
Do you mean Fuego-related?
Quoted 3 lines
patch. They can't be merged because the requirements are using a specific key that needs to be downloaded with curl (that's installed in the previous step. Additionally, LAVA apt-get commands have been moved
Another reference to LAVA?
ops, and I missed those in my review as well.
...
Quoted 27 lines
Quoted 17 lines
apt-get update && \
Dockerfile +16
-0+ apt-get -yV install \ + apt-utils \ + at \ + autoconf \ + automake \ + bc \ + bsdmainutils \ + bzip2 \ + cmake \ + curl \ + daemon \ + diffstat \ + g++ \ + gcc \ + gettext \ + git \Dockerfile ENV https_proxy ${HTTP_PROXY} · line 53
Quoted 10 lines
Dockerfile +10
-0+ inotify-tools \ + iperf \ + lava-tool \ + libaio-dev \ + libcairo2-dev \ + libglib2.0-dev \ + libsdl1.2-dev \ + libtool \ + libxmu-dev \ + libxmuu-dev \
Quoted 3 lines
Most of the above libs are also toolchain packages, for
x86_64(the docker virtual board) and would benefit from being moved to a separate image creation step or derivative image.
you mean those *-dev, cmake, autoconf, automake, gcc, g++, gettext and libtool? indeed makes sense to move these to separate image once we split the armhf.
Signature · 6 lines
Gustavo Sverzut Barbieri http://profusion.mobi embedded systems -------------------------------------- MSN, GTalk, FaceTime: [email protected] Skype: gsbarbieri Mobile: +55 (16) 99354-9890
Re: [Fuego] [PATCH 2/9] Refactor preinst and apt-get/pip install
Quoted 14 lines
-----Original Message-----
From: Gustavo SverzutQuoted 7 lines · On Tue, Jan 30, 2018 at 5:22 PM, <[email protected]> wrote:
Quoted 7 lines
-----Original Message-----
From: Guilherme Campos Camargo
All base-image related apt-get commands (including pre-install config, as well as the package install) have been put together in the same RUN instruction.The same has been done with the LAVA-related apt-get commands on
this
Quoted 2 lines
Do you mean Fuego-related?
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
patch. They can't be merged because the requirements are using a specific key that needs to be downloaded with curl (that's installed in the previous step. Additionally, LAVA apt-get commands have been
moved
Quoted 1 line
Another reference to LAVA?
ops, and I missed those in my review as well.
No problem. It's not like people look at commit messages a lot after the review ;-)
Quoted 30 lines
...
Quoted 27 lines
Quoted 17 lines
apt-get update && \
Dockerfile +16
-0+ apt-get -yV install \ + apt-utils \ + at \ + autoconf \ + automake \ + bc \ + bsdmainutils \ + bzip2 \ + cmake \ + curl \ + daemon \ + diffstat \ + g++ \ + gcc \ + gettext \ + git \Dockerfile ENV https_proxy ${HTTP_PROXY} · line 53
Quoted 10 lines
Dockerfile +10
-0+ inotify-tools \ + iperf \ + lava-tool \ + libaio-dev \ + libcairo2-dev \ + libglib2.0-dev \ + libsdl1.2-dev \ + libtool \ + libxmu-dev \ + libxmuu-dev \Quoted 1 line
Most of the above libs are also toolchain packages, for
x86_64(the docker
Quoted 8 lines
virtual board)
Quoted 1 line
and would benefit from being moved to a separate image creation step or
derivative
Quoted 1 line
image.
you mean those *-dev, cmake, autoconf, automake, gcc, g++, gettext and libtool? indeed makes sense to move these to separate image once we split the armhf.
Just the lib*-dev. and gcc and g++. cmake, autoconf, automake and libtool are non-architecture-specific parts of the toolchain, which would be used by any SDK used in the container, I believe. I'm not sure about gettext. I think it provides libraries or headers that would get compiled into binaries that go to the target, so likely it is x86_64-specific.
The only things that Fuego intends to compile (right now), are things for the target boards. That's not an absolute hard and fast rule. I could imagine some test program having both a host-side and target-side component that needed to be compiled. But I don't think we have any of those at the moment. ... (I just checked the Functional.commonAPI_* tests, which I thought might have both host and target compiled programs, but they apparently don't. So I'm pretty sure this is correct.)
In any event, the general rule would be that the SDK installation step should have the packages needed for compilation of programs intended to run on the target. docker is a special target, and we default to including the SDK for that included in the base image. But it would still be good to keep these in a separate list.
-- Tim
PATCH 3/9 Use curl for downloading Jenkins
In reply to Guilherme Campos CamargoAs curl had been used previously in the Dockerfile, let's make it our preferred tool for downloading resources in your scripts.
--- Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Dockerfile +1 -1
@@ -125,7 +125,7 @@ ENV JENKINS_HOME=/var/lib/jenkins RUN groupadd -g ${gid} ${group} \ && useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user} -RUN wget -nv ${JENKINS_URL} +RUN curl -L -O ${JENKINS_URL} RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c - RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb RUN rm jenkins_${JENKINS_VERSION}_all.deb
Signature · 1 line
2.15.1
Re: [Fuego] [PATCH 3/9] Use curl for downloading Jenkins
OK - looks good. Applied.
-- Tim
Quoted 38 lines
-----Original Message-----
From: [email protected] [mailto:fuego- [email protected]] On Behalf Of Guilherme Campos Camargo
Sent: Friday, January 26, 2018 9:36 AM
To: [email protected]
Subject: [Fuego] [PATCH 3/9] Use curl for downloading JenkinsAs curl had been used previously in the Dockerfile, let's make it our preferred tool for downloading resources in your scripts.
Signed-off-by Guilherme Campos Camargo <[email protected]>--- Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)Dockerfile +1
-1@@ -125,7 +125,7 @@ ENV JENKINS_HOME=/var/lib/jenkins RUN groupadd -g ${gid} ${group} \ && useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo - s /bin/bash ${user} -RUN wget -nv ${JENKINS_URL} +RUN curl -L -O ${JENKINS_URL} RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" |sha1sum -c -
RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb
RUN rm jenkins_${JENKINS_VERSION}_all.deb
--
2.15.1_______________________________________________
Fuego mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/fuego
PATCH 4/9 Move configuration and jenkins plugin installation to separate scripts
In reply to Guilherme Campos CamargoIn order to simplify the structure of the Dockerfile and optimize build times.
---
Dockerfile | 96 ++++++---------------
frontend-install/{ => setup/jenkins}/config.xml | 0
.../jenkins.model.JenkinsLocationConfiguration.xml | 0
.../plugins/flot-plotter-plugin/.gitignore | 0
.../jenkins}/plugins/flot-plotter-plugin/flot.hpi | Bin
.../jenkins}/plugins/flot-plotter-plugin/pom.xml | 0
.../src/main/java/flotile/ChartAction.java | 0
.../src/main/java/flotile/FlotPublisher.java | 0
.../flotile/ChartAction/floatingBox.jelly | 0
.../src/main/resources/index.jelly | 0
.../src/main/webapp/flot/.copyarea.db | 0
.../src/main/webapp/flot/API.txt | 0
.../src/main/webapp/flot/FAQ.txt | 0
.../src/main/webapp/flot/LICENSE.txt | 0
.../src/main/webapp/flot/PLUGINS.txt | 0
.../src/main/webapp/flot/README.txt | 0
.../src/main/webapp/flot/base64.js | 0
.../src/main/webapp/flot/canvas2image.js | 0
.../src/main/webapp/flot/excanvas.min.js | 0
.../main/webapp/flot/jquery.colorhelpers.min.js | 0
.../main/webapp/flot/jquery.flot.crosshair.min.js | 0
.../webapp/flot/jquery.flot.fillbetween.min.js | 0
.../src/main/webapp/flot/jquery.flot.image.min.js | 0
.../src/main/webapp/flot/jquery.flot.min.js | 0
.../main/webapp/flot/jquery.flot.navigate.min.js | 0
.../src/main/webapp/flot/jquery.flot.pie.min.js | 0
.../src/main/webapp/flot/jquery.flot.resize.min.js | 0
.../main/webapp/flot/jquery.flot.selection.min.js | 0
.../src/main/webapp/flot/jquery.flot.stack.min.js | 0
.../src/main/webapp/flot/jquery.flot.symbol.min.js | 0
.../main/webapp/flot/jquery.flot.threshold.min.js | 0
.../src/main/webapp/flot/jquery.min.js | 0
.../src/main/webapp/flot/mod.js | 0
.../src/main/webapp/flot/my.css | 0
.../src/main/webapp/help-globalConfig.html | 0
frontend-install/setup/jenkins/plugins/install.sh | 21 +++++
frontend-install/setup/jenkins/setup.sh | 26 ++++++
frontend-install/setup/lava/setup.sh | 11 +++
frontend-install/setup/serial/setup-serio.sh | 8 ++
frontend-install/setup/serial/setup-serlogin.sh | 6 ++
frontend-install/setup/serial/setup-ttc.sh | 7 ++
41 files changed, 105 insertions(+), 70 deletions(-)
rename frontend-install/{ => setup/jenkins}/config.xml (100%)
rename frontend-install/{ => setup/jenkins}/jenkins.model.JenkinsLocationConfiguration.xml (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/.gitignore (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/flot.hpi (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/pom.xml (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/java/flotile/ChartAction.java (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/java/flotile/FlotPublisher.java (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/resources/index.jelly (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/.copyarea.db (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/API.txt (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/FAQ.txt (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/LICENSE.txt (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/PLUGINS.txt (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/README.txt (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/base64.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/canvas2image.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/excanvas.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.colorhelpers.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.image.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.navigate.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.pie.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.resize.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.selection.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.stack.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.symbol.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.threshold.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.min.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/mod.js (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/my.css (100%)
rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/help-globalConfig.html (100%)
create mode 100755 frontend-install/setup/jenkins/plugins/install.sh
create mode 100755 frontend-install/setup/jenkins/setup.sh
create mode 100755 frontend-install/setup/lava/setup.sh
create mode 100755 frontend-install/setup/serial/setup-serio.sh
create mode 100755 frontend-install/setup/serial/setup-serlogin.sh
create mode 100755 frontend-install/setup/serial/setup-ttc.shDockerfile +26 -70
@@ -106,95 +106,51 @@ RUN echo deb http://emdebian.org/tools/debian/ jessie main > /etc/apt/sources.li gcc-arm-linux-gnueabihf && \ rm -rf /var/lib/apt/lists/* -RUN pip install \ - filelock \ - python-jenkins==0.4.14 - # ============================================================================== -# Install Jenkins with the same UID/GID as the host user +# Download and Install Jenkins # ============================================================================== -ARG user=jenkins -ARG group=jenkins -ARG uid=1000 -ARG gid=${uid} +ENV uid=1000 +ENV gid=${uid} ARG JENKINS_VERSION=2.32.1 ARG JENKINS_SHA=bfc226aabe2bb089623772950c4cc13aee613af1 ARG JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb ENV JENKINS_HOME=/var/lib/jenkins -RUN groupadd -g ${gid} ${group} \ - && useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user} -RUN curl -L -O ${JENKINS_URL} -RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c - -RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb -RUN rm jenkins_${JENKINS_VERSION}_all.deb +RUN groupadd -g ${gid} ${group} && \ + useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user} && \ + curl -L -O ${JENKINS_URL} && \ + echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c - && \ + dpkg -i jenkins_${JENKINS_VERSION}_all.deb && \ + rm jenkins_${JENKINS_VERSION}_all.deb # ============================================================================== -# get ttc script and helpers +# Post installation # ============================================================================== -RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc -RUN /usr/local/src/ttc/install.sh /usr/local/bin -RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc -# ============================================================================== -# Serial Config -# ============================================================================== +COPY frontend-install/setup/jenkins /setup/jenkins +WORKDIR /setup/jenkins +RUN ./setup.sh -RUN /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh' +WORKDIR /setup/jenkins/plugins +RUN ./install.sh -RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin' +COPY frontend-install/setup/serial /setup/serial +WORKDIR /setup -# ============================================================================== -# Post installation -# ============================================================================== +RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc && \ + ./serial/setup-ttc.sh -RUN source /etc/default/jenkins && \ - JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \ - sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins - -RUN source /etc/default/jenkins && \ - JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false" && \ - if [ -n "$HTTP_PROXY" ]; then \ - PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \ - PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \ - JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \ - fi && \ - sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins; - -COPY frontend-install/plugins/flot-plotter-plugin/flot.hpi /tmp - -RUN service jenkins start && \ - sleep 30 && \ - sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin description-setter && \ - sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin pegdown-formatter && \ - sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin /tmp/flot.hpi && \ - sleep 10 - -# Let Jenkins install flot before making the symlink -RUN service jenkins restart && sleep 30 && \ - rm $JENKINS_HOME/plugins/flot/flot/mod.js && \ - ln -s /fuego-core/engine/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js - -RUN ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs -COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf +RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio && \ + ./serial/setup-serio.sh -RUN ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/ -COPY frontend-install/config.xml $JENKINS_HOME/config.xml -COPY frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml +RUN git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin && \ + ./serial/setup-serlogin.sh -RUN chown -R jenkins:jenkins $JENKINS_HOME/ +COPY frontend-install/setup/lava /setup/lava +RUN ./lava/setup.sh -# ============================================================================== -# Lava -# ============================================================================== - -RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin -RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin -# CONVENIENCE HACKS -# not mounted, yet -#RUN echo "fuego-create-node --board raspberrypi3" >> /root/firststart.sh -#RUN echo "fuego-create-jobs --board raspberrypi3 --testplan testplan_docker --distrib nosyslogd.dist" >> /root/firststart.sh +COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf # ============================================================================== # Setup startup command
frontend-install/setup/jenkins/config.xml renamed from frontend-install/config.xml +0 -0
frontend-install/setup/jenkins/jenkins.model.JenkinsLocationConfiguration.xml renamed from frontend-install/jenkins.model.JenkinsLocationConfiguration.xml +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/.gitignore renamed from frontend-install/plugins/flot-plotter-plugin/.gitignore +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/flot.hpi renamed from frontend-install/plugins/flot-plotter-plugin/flot.hpi +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/pom.xml renamed from frontend-install/plugins/flot-plotter-plugin/pom.xml +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/java/flotile/ChartAction.java renamed from frontend-install/plugins/flot-plotter-plugin/src/main/java/flotile/ChartAction.java +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/java/flotile/FlotPublisher.java renamed from frontend-install/plugins/flot-plotter-plugin/src/main/java/flotile/FlotPublisher.java +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly renamed from frontend-install/plugins/flot-plotter-plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/resources/index.jelly renamed from frontend-install/plugins/flot-plotter-plugin/src/main/resources/index.jelly +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/.copyarea.db renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/.copyarea.db +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/API.txt renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/API.txt +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/FAQ.txt renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/FAQ.txt +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/LICENSE.txt renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/LICENSE.txt +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/PLUGINS.txt renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/PLUGINS.txt +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/README.txt renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/README.txt +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/base64.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/base64.js +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/canvas2image.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/canvas2image.js +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/excanvas.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/excanvas.min.js +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.colorhelpers.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.colorhelpers.min.js +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.image.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.image.min.js +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.min.js +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.navigate.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.navigate.min.js +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.pie.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.pie.min.js +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.resize.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.resize.min.js +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.selection.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.selection.min.js +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.stack.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.stack.min.js +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.symbol.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.symbol.min.js +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.threshold.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.threshold.min.js +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.min.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.min.js +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/mod.js renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/mod.js +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/my.css renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/my.css +0 -0
frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/help-globalConfig.html renamed from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/help-globalConfig.html +0 -0
frontend-install/setup/jenkins/plugins/install.sh new file +21 -0
@@ -0,0 +1,21 @@ +#!/bin/bash + +set -e + +service jenkins start +sleep 30 +sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin description-setter +sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin pegdown-formatter +sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin flot-plotter-plugin/flot.hpi +sleep 10 + +# Let Jenkins install flot before making the symlink +service jenkins restart +sleep 30 +rm $JENKINS_HOME/plugins/flot/flot/mod.js +ln -s /fuego-core/engine/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js +ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs +ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/ + +chown -R jenkins:jenkins $JENKINS_HOME/ +
frontend-install/setup/jenkins/setup.sh new file +26 -0
@@ -0,0 +1,26 @@ +#!/bin/bash + +set -e + +user=jenkins +group=jenkins +uid=1000 +gid=${uid} + +# groupadd -g ${gid} ${group} +# useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user} + +cp config.xml jenkins.model.JenkinsLocationConfiguration.xml ${JENKINS_HOME} + +source /etc/default/jenkins +JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" +sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins + +JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false" +if [ -n "$HTTP_PROXY" ]; then + PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/'); + PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/'); + JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost=${PROXYSERVER} -Dhttp.proxyPort=${PROXYPORT} -Dhttps.proxyHost=${PROXYSERVER} -Dhttps.proxyPort=${PROXYPORT}"; +fi +sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins +
frontend-install/setup/lava/setup.sh new file +11 -0
@@ -0,0 +1,11 @@ +#!/bin/bash + +set -e + +# CONVENIENCE HACKS +# not mounted, yet +#RUN echo "fuego-create-node --board raspberrypi3" >> /root/firststart.sh +#RUN echo "fuego-create-jobs --board raspberrypi3 --testplan testplan_docker --distrib nosyslogd.dist" >> /root/firststart.sh + +ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin +ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
frontend-install/setup/serial/setup-serio.sh new file +8 -0
@@ -0,0 +1,8 @@ +#!/bin/bash + +set -e + +chown -R jenkins:jenkins /usr/local/src/serio +cp /usr/local/src/serio/serio /usr/local/bin/ +ln -s /usr/local/bin/serio /usr/local/bin/sercp +ln -s /usr/local/bin/serio /usr/local/bin/sersh
frontend-install/setup/serial/setup-serlogin.sh new file +6 -0
@@ -0,0 +1,6 @@ +#!/bin/bash + +set -e + +chown -R jenkins:jenkins /usr/local/src/serlogin +cp /usr/local/src/serlogin/serlogin /usr/local/bin
frontend-install/setup/serial/setup-ttc.sh new file +7 -0
@@ -0,0 +1,7 @@ +#!/bin/bash + +set -e + +/usr/local/src/ttc/install.sh /usr/local/bin +perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" \ + /usr/local/bin/ttc
Signature · 1 line
2.15.1
Re: [Fuego] [PATCH 4/9] Move configuration and jenkins plugin installation to separate scripts
Quoted 49 lines
-----Original Message-----
From: [email protected] [mailto:fuego- [email protected]] On Behalf Of Guilherme Campos Camargo
Sent: Friday, January 26, 2018 9:36 AM
To: [email protected]
Subject: [Fuego] [PATCH 4/9] Move configuration and jenkins plugin installation to separate scriptsIn order to simplify the structure of the Dockerfile and optimize build times.
Signed-off-by Guilherme Campos Camargo <[email protected]>--- Dockerfile | 96 ++++++--------------- frontend-install/{ => setup/jenkins}/config.xml | 0 .../jenkins.model.JenkinsLocationConfiguration.xml | 0 .../plugins/flot-plotter-plugin/.gitignore | 0 .../jenkins}/plugins/flot-plotter-plugin/flot.hpi | Bin .../jenkins}/plugins/flot-plotter-plugin/pom.xml | 0 .../src/main/java/flotile/ChartAction.java | 0 .../src/main/java/flotile/FlotPublisher.java | 0 .../flotile/ChartAction/floatingBox.jelly | 0 .../src/main/resources/index.jelly | 0 .../src/main/webapp/flot/.copyarea.db | 0 .../src/main/webapp/flot/API.txt | 0 .../src/main/webapp/flot/FAQ.txt | 0 .../src/main/webapp/flot/LICENSE.txt | 0 .../src/main/webapp/flot/PLUGINS.txt | 0 .../src/main/webapp/flot/README.txt | 0 .../src/main/webapp/flot/base64.js | 0 .../src/main/webapp/flot/canvas2image.js | 0 .../src/main/webapp/flot/excanvas.min.js | 0 .../main/webapp/flot/jquery.colorhelpers.min.js | 0 .../main/webapp/flot/jquery.flot.crosshair.min.js | 0 .../webapp/flot/jquery.flot.fillbetween.min.js | 0 .../src/main/webapp/flot/jquery.flot.image.min.js | 0 .../src/main/webapp/flot/jquery.flot.min.js | 0 .../main/webapp/flot/jquery.flot.navigate.min.js | 0 .../src/main/webapp/flot/jquery.flot.pie.min.js | 0 .../src/main/webapp/flot/jquery.flot.resize.min.js | 0 .../main/webapp/flot/jquery.flot.selection.min.js | 0 .../src/main/webapp/flot/jquery.flot.stack.min.js | 0 .../src/main/webapp/flot/jquery.flot.symbol.min.js | 0 .../main/webapp/flot/jquery.flot.threshold.min.js | 0 .../src/main/webapp/flot/jquery.min.js | 0 .../src/main/webapp/flot/mod.js | 0
Quoted 8 lines
.../src/main/webapp/flot/my.css | 0 .../src/main/webapp/help-globalConfig.html | 0 frontend-install/setup/jenkins/plugins/install.sh | 21 +++++ frontend-install/setup/jenkins/setup.sh | 26 ++++++ frontend-install/setup/lava/setup.sh | 11 +++ frontend-install/setup/serial/setup-serio.sh | 8 ++ frontend-install/setup/serial/setup-serlogin.sh | 6 ++ frontend-install/setup/serial/setup-ttc.sh | 7 ++
In general, I like the idea of refactoring this, but not this specific refactoring. I'll keep your scripts,
but maybe with renames and some merges. (serio and serlogin setup don't need to be separate).
ttc is not related to serial. the lava, serial and apt dirs are not really related to 'frontend-install', which is itself
a euphemism for 'jenkins'.
I'm applying this patch, in case subsequent patches in this series depend on the Dockerfile changes here, but I plan to refactor the frontend-install directory again after I apply the rest of this series.
Quoted 97 lines
41 files changed, 105 insertions(+), 70 deletions(-) rename frontend-install/{ => setup/jenkins}/config.xml (100%) rename frontend-install/{ => setup/jenkins}/jenkins.model.JenkinsLocationConfiguration.xml (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/.gitignore (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/flot.hpi (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/pom.xml (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/java/flotile/ChartAction.java (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/java/flotile/FlotPublisher.java (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/resources/index.jelly (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/.copyarea.db (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/API.txt (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/FAQ.txt (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/LICENSE.txt (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/PLUGINS.txt (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/README.txt (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/base64.js (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/canvas2image.js (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/excanvas.min.js (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.colorhelpers.min.js (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.image.min.js (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.min.js (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.navigate.min.js (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.pie.min.js (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.resize.min.js (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.selection.min.js (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.stack.min.js (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.symbol.min.js (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.threshold.min.js (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.min.js (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/mod.js (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/flot/my.css (100%) rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter- plugin/src/main/webapp/help-globalConfig.html (100%) create mode 100755 frontend-install/setup/jenkins/plugins/install.sh create mode 100755 frontend-install/setup/jenkins/setup.sh create mode 100755 frontend-install/setup/lava/setup.sh create mode 100755 frontend-install/setup/serial/setup-serio.sh create mode 100755 frontend-install/setup/serial/setup-serlogin.sh create mode 100755 frontend-install/setup/serial/setup-ttc.shDockerfile +0
-0@@ -106,95 +106,51 @@ RUN echo deb http://emdebian.org/tools/debian/jessie main >
/etc/apt/sources.ligcc-arm-linux-gnueabihf && \ rm -rf /var/lib/apt/lists/*Dockerfile +0
-4-RUN pip install \ - filelock \ - python-jenkins==0.4.14 - #==========================================================
====================Dockerfile +1
-1-# Install Jenkins with the same UID/GID as the host user +# Download and Install Jenkins #==========================================================
====================
Dockerfile RUN echo deb http://emdebian.org/tools/debian/ jessie main > /etc/apt/sources.li · line 114
Quoted 6 lines
Dockerfile +2
-4-ARG user=jenkins -ARG group=jenkins -ARG uid=1000 -ARG gid=${uid} +ENV uid=1000 +ENV gid=${uid}
Do these environment variables persist in the created container? If so, these names are a bit short. They don't show up in the shell environment inside the container do they?
Quoted 576 lines
ARG JENKINS_VERSION=2.32.1
ARG JENKINS_SHA=bfc226aabe2bb089623772950c4cc13aee613af1 ARG JENKINS_URL=https://pkg.jenkins.io/debian-
stable/binary/jenkins_${JENKINS_VERSION}_all.deb
ENV JENKINS_HOME=/var/lib/jenkinsDockerfile +0
-2-RUN groupadd -g ${gid} ${group} \ - && useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s
/bin/bash${user}Dockerfile +0
-2-RUN curl -L -O ${JENKINS_URL} -RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" |sha1sum -c -
Dockerfile +2
-2-RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb -RUN rm jenkins_${JENKINS_VERSION}_all.deb +RUN groupadd -g ${gid} ${group} && \ + useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s
/bin/bash${user} && \Dockerfile +4
-1+ curl -L -O ${JENKINS_URL} && \ + echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c - && \ + dpkg -i jenkins_${JENKINS_VERSION}_all.deb && \ + rm jenkins_${JENKINS_VERSION}_all.deb #==========================================================
====================Dockerfile +1
-1-# get ttc script and helpers +# Post installation #==========================================================
====================Dockerfile +0
-3-RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc -RUN /usr/local/src/ttc/install.sh /usr/local/bin -RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#"
/usr/local/bin/ttc-#
==========================================================
====================Dockerfile +0
-2-# Serial Config -#==========================================================
====================Dockerfile +3
-1+COPY frontend-install/setup/jenkins /setup/jenkins +WORKDIR /setup/jenkins +RUN ./setup.sh -RUN /bin/bash -c 'git clone https://github.com/frowand/serio.git
/usr/local/src/serio; chown -R jenkins/usr/local/src/serio; cp/usr/local/src/serio/serio/usr/local/bin/; ln -s/usr/local/bin/serio/usr/local/bin/sercp; ln -s/usr/local/bin/serio/usr/local/bin/sersh'Dockerfile +2
-1+WORKDIR /setup/jenkins/plugins +RUN ./install.sh -RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git
/usr/local/src/serlogin; chown -R jenkins/usr/local/src/serlogin; cp/usr/local/src/serlogin/serlogin/usr/local/bin'Dockerfile +2
-1+COPY frontend-install/setup/serial /setup/serial +WORKDIR /setup -#==========================================================
====================Dockerfile +0
-2-# Post installation -#==========================================================
====================Dockerfile +2
-3+RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc && \ + ./serial/setup-ttc.sh -RUN source /etc/default/jenkins && \ - JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \ - sed -i -e"s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g"
/etc/default/jenkinsDockerfile +0
-3- -RUN source /etc/default/jenkins && \ - JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false"&& \
Dockerfile +0
-4- if [ -n "$HTTP_PROXY" ]; then \ - PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \ - PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
Dockerfile +0
-1- JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" - Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
Dockerfile +0
-2- fi && \ - sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g"
/etc/default/jenkins;Dockerfile +0
-6- -COPY frontend-install/plugins/flot-plotter-plugin/flot.hpi /tmp - -RUN service jenkins start && \ - sleep 30 && \ - sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin description-setter && \
Dockerfile +0
-1- sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin pegdown-formatter && \
- sudo -u jenkins java -jar/var/cache/jenkins/war/WEB-INF/jenkins-cli.jar-
s http://localhost:8080/fuego install-plugin/tmp/flot.hpi&& \Dockerfile +0
-6- sleep 10 - -# Let Jenkins install flot before making the symlink -RUN service jenkins restart && sleep 30 && \ - rm $JENKINS_HOME/plugins/flot/flot/mod.js && \ - ln -s /fuego-core/engine/scripts/mod.js$JENKINS_HOME/plugins/flot/flot/mod.js
Dockerfile +0
-3- -RUN ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs -COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
+RUN git clone https://github.com/frowand/serio.git/usr/local/src/serio&& \Dockerfile +1
-3+ ./serial/setup-serio.sh -RUN ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/ -COPY frontend-install/config.xml $JENKINS_HOME/config.xml -COPY frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
$JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml+RUN git clone https://github.com/tbird20d/serlogin.git/usr/local/src/serlogin&& \Dockerfile +3
-2+ ./serial/setup-serlogin.sh -RUN chown -R jenkins:jenkins $JENKINS_HOME/ +COPY frontend-install/setup/lava /setup/lava +RUN ./lava/setup.sh -#==========================================================
====================Dockerfile +0
-2-# Lava -#==========================================================
====================Dockerfile +0
-7- -RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin -RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin -# CONVENIENCE HACKS -# not mounted, yet -#RUN echo "fuego-create-node --board raspberrypi3" >> /root/firststart.sh -#RUN echo "fuego-create-jobs --board raspberrypi3 --testplan
testplan_docker--distrib nosyslogd.dist" >>/root/firststart.sh+COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego- docs.pdf#
==========================================================
====================
# Setup startup commandfrontend- renamed from frontend-install/config.xml +0
-0
install/setup/jenkins/config.xml
similarity index 100%
rename fromfrontend-install/config.xmlrename tofrontend-install/setup/jenkins/config.xmlfrontend- renamed from frontend-install/jenkins.model.JenkinsLocationConfiguration.xml +0
-0
install/setup/jenkins/jenkins.model.JenkinsLocationConfiguration.xmlsimilarity index 100%
rename from frontend-install/jenkins.model.JenkinsLocationConfiguration.xmlrename to frontend-install/setup/jenkins/jenkins.model.JenkinsLocationConfiguration.xmlfrontend- renamed from frontend-install/plugins/flot-plotter-plugin/.gitignore +0
-0install/setup/jenkins/plugins/flot-plotter-plugin/.gitignore similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/.gitignore rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/.gitignorefrontend- renamed from frontend-install/plugins/flot-plotter-plugin/flot.hpi +0
-0install/setup/jenkins/plugins/flot-plotter-plugin/flot.hpi similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/flot.hpi rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/flot.hpifrontend- renamed from frontend-install/plugins/flot-plotter-plugin/pom.xml +0
-0
install/setup/jenkins/plugins/flot-plotter-plugin/pom.xmlsimilarity index 100%
rename fromfrontend-install/plugins/flot-plotter-plugin/pom.xmlrename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/pom.xml(quoted diff) +0
-0plugin/src/main/java/flotile/ChartAction.java b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/java/flotile/ChartAction.java similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/java/flotile/ChartAction.java
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/java/flotile/ChartAction.java(quoted diff) +0
-0plugin/src/main/java/flotile/FlotPublisher.java b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/java/flotile/FlotPublisher.java similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/java/flotile/FlotPublisher.java rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/java/flotile/FlotPublisher.java(quoted diff) +0
-0plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly(quoted diff) +0
-0plugin/src/main/resources/index.jelly b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/resources/index.jelly
similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/resources/index.jelly
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/resources/index.jelly(quoted diff) +0
-0plugin/src/main/webapp/flot/.copyarea.db b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/.copyarea.db
similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/.copyarea.db
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/.copyarea.db(quoted diff) +0
-0
plugin/src/main/webapp/flot/API.txtb/frontend- install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/API.txt
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/API.txt
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/API.txt(quoted diff) +0
-0
plugin/src/main/webapp/flot/FAQ.txtb/frontend- install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/FAQ.txt
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/FAQ.txt
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/FAQ.txt(quoted diff) +0
-0
plugin/src/main/webapp/flot/LICENSE.txtb/frontend- install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/LICENSE.txt
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/LICENSE.txt
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/LICENSE.txt(quoted diff) +0
-0
plugin/src/main/webapp/flot/PLUGINS.txtb/frontend- install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/PLUGINS.txt
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/PLUGINS.txt
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/PLUGINS.txt(quoted diff) +0
-0
plugin/src/main/webapp/flot/README.txtb/frontend- install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/README.txt
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/README.txt
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/README.txt(quoted diff) +0
-0plugin/src/main/webapp/flot/base64.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/base64.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/base64.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/base64.js(quoted diff) +0
-0plugin/src/main/webapp/flot/canvas2image.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/canvas2image.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/canvas2image.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/canvas2image.js(quoted diff) +0
-0plugin/src/main/webapp/flot/excanvas.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/excanvas.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/excanvas.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/excanvas.min.js(quoted diff) +0
-0plugin/src/main/webapp/flot/jquery.colorhelpers.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.colorhelpers.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.colorhelpers.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.colorhelpers.min.js(quoted diff) +0
-0plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js(quoted diff) +0
-0plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js(quoted diff) +0
-0plugin/src/main/webapp/flot/jquery.flot.image.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.image.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.image.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.image.min.js(quoted diff) +0
-0plugin/src/main/webapp/flot/jquery.flot.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.min.js(quoted diff) +0
-0plugin/src/main/webapp/flot/jquery.flot.navigate.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.navigate.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.navigate.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.navigate.min.js(quoted diff) +0
-0plugin/src/main/webapp/flot/jquery.flot.pie.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.pie.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.pie.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.pie.min.js(quoted diff) +0
-0plugin/src/main/webapp/flot/jquery.flot.resize.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.resize.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.resize.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.resize.min.js(quoted diff) +0
-0plugin/src/main/webapp/flot/jquery.flot.selection.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.selection.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.selection.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.selection.min.js(quoted diff) +0
-0plugin/src/main/webapp/flot/jquery.flot.stack.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.stack.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.stack.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.stack.min.js(quoted diff) +0
-0plugin/src/main/webapp/flot/jquery.flot.symbol.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.symbol.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.symbol.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.symbol.min.js(quoted diff) +0
-0plugin/src/main/webapp/flot/jquery.flot.threshold.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.threshold.min.js similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.threshold.min.js rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.flot.threshold.min.js(quoted diff) +0
-0plugin/src/main/webapp/flot/jquery.min.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.min.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/jquery.min.js(quoted diff) +0
-0plugin/src/main/webapp/flot/mod.js b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/mod.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/mod.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/mod.js(quoted diff) +0
-0plugin/src/main/webapp/flot/my.css b/frontend- install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/my.css
similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/flot/my.css
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/flot/my.css(quoted diff) +0
-0plugin/src/main/webapp/help-globalConfig.html b/frontend- install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/help- globalConfig.html
similarity index 100%
rename from frontend-install/plugins/flot-plotter- plugin/src/main/webapp/help-globalConfig.html
rename to frontend-install/setup/jenkins/plugins/flot-plotter- plugin/src/main/webapp/help-globalConfig.htmlfrontend- renamed from frontend-install/setup/jenkins/plugins/install.sh +0
-0
install/setup/jenkins/plugins/install.sh
new file mode 100755
index 0000000..0db8bf2frontend-install/setup/jenkins/plugins/install.sh new file +7
-0@@ -0,0 +1,21 @@ +#!/bin/bash + +set -e + +service jenkins start +sleep 30 +sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -shttp://localhost:8080/fuego install-plugin description-setter +sudo -u jenkins java -jar
/var/cache/jenkins/war/WEB-INF/jenkins-cli.jar-s
http://localhost:8080/fuego install-plugin pegdown-formatter +sudo -u jenkins java -jar/var/cache/jenkins/war/WEB-INF/jenkins-cli.jar-s
http://localhost:8080/fuego install-plugin flot-plotter-plugin/flot.hpifrontend-install/setup/jenkins/plugins/install.sh new file +7
-0+sleep 10 + +# Let Jenkins install flot before making the symlink +service jenkins restart +sleep 30 +rm $JENKINS_HOME/plugins/flot/flot/mod.js +ln -s /fuego-core/engine/scripts/mod.js$JENKINS_HOME/plugins/flot/flot/mod.js
frontend-install/setup/jenkins/plugins/install.sh new file +5
-0+ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs +ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/ + +chown -R jenkins:jenkins $JENKINS_HOME/ +frontend- renamed from frontend-install/setup/jenkins/setup.sh +0
-0
install/setup/jenkins/setup.sh
new file mode 100755
index 0000000..c550862frontend-install/setup/jenkins/setup.sh new file +11
-0@@ -0,0 +1,26 @@ +#!/bin/bash + +set -e + +user=jenkins +group=jenkins +uid=1000 +gid=${uid} + +# groupadd -g ${gid} ${group} +# useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s
/bin/bash${user}frontend-install/setup/jenkins/setup.sh new file +2
-0+ +cp config.xml jenkins.model.JenkinsLocationConfiguration.xml${JENKINS_HOME}
frontend-install/setup/jenkins/setup.sh new file +4
-0+ +source /etc/default/jenkins +JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" +sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g"
/etc/default/jenkinsfrontend-install/setup/jenkins/setup.sh new file +4
-0+ +JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false" +if [ -n "$HTTP_PROXY" ]; then + PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E's/\///g' | sed -E 's/(.*):(.*)/\1/');
frontend-install/setup/jenkins/setup.sh new file +1
-0+ PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E's/\///g' | sed -E 's/(.*):(.*)/\2/');
frontend-install/setup/jenkins/setup.sh new file +1
-0+ JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost=${PROXYSERVER} -Dhttp.proxyPort=${PROXYPORT} -Dhttps.proxyHost=${PROXYSERVER} - Dhttps.proxyPort=${PROXYPORT}";
frontend-install/setup/jenkins/setup.sh new file +2
-0+fi +sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g"
/etc/default/jenkinsfrontend-install/setup/jenkins/setup.sh new file +1
-0+frontend- renamed from frontend-install/setup/lava/setup.sh +0
-0
install/setup/lava/setup.sh
new file mode 100755
index 0000000..86df49ffrontend-install/setup/lava/setup.sh new file +8
-0@@ -0,0 +1,11 @@ +#!/bin/bash + +set -e + +# CONVENIENCE HACKS +# not mounted, yet +#RUN echo "fuego-create-node --board raspberrypi3" >> /root/firststart.sh +#RUN echo "fuego-create-jobs --board raspberrypi3 --testplan
testplan_docker--distrib nosyslogd.dist" >>/root/firststart.shfrontend-install/setup/lava/setup.sh new file +3
-0+ +ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin +ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/binfrontend- renamed from frontend-install/setup/serial/setup-serio.sh +0
-0
install/setup/serial/setup-serio.sh
new file mode 100755
index 0000000..7e25a96frontend-install/setup/serial/setup-serio.sh new file +8
-0@@ -0,0 +1,8 @@ +#!/bin/bash + +set -e + +chown -R jenkins:jenkins /usr/local/src/serio +cp /usr/local/src/serio/serio /usr/local/bin/ +ln -s /usr/local/bin/serio /usr/local/bin/sercp +ln -s /usr/local/bin/serio /usr/local/bin/sershfrontend- renamed from frontend-install/setup/serial/setup-serlogin.sh +0
-0
install/setup/serial/setup-serlogin.sh
new file mode 100755
index 0000000..0de17eafrontend-install/setup/serial/setup-serlogin.sh new file +6
-0@@ -0,0 +1,6 @@ +#!/bin/bash + +set -e + +chown -R jenkins:jenkins /usr/local/src/serlogin +cp /usr/local/src/serlogin/serlogin /usr/local/binfrontend- renamed from frontend-install/setup/serial/setup-ttc.sh +0
-0
install/setup/serial/setup-ttc.sh
new file mode 100755
index 0000000..d32b73efrontend-install/setup/serial/setup-ttc.sh new file +7
-1@@ -0,0 +1,7 @@ +#!/bin/bash + +set -e + +/usr/local/src/ttc/install.sh /usr/local/bin +perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" \ + /usr/local/bin/ttc --
Quoted 1 line
2.15.1
OK - applied, but with caveats mentioned above.
-- Tim
PATCH 5/9 Remove jenkins user created during build time with host's uid/gid
In reply to Guilherme Campos CamargoPrior to this patch, the fuego-host-scripts/docker-build-image.sh script was calling docker build passing two build-args: uid/gid. According to the comments in the code, it seems that the intention was to make sure that the jenkins user (needed by Jenkins) would have the same uid/gid as the user of the host that has called the install script.
Given that the uid/gid of the jenkins user was being set at build time, the generated image would be configured only for the machine where it's built, what would prevent jenkins to change files in the /fuego-rw mountpoint in different hosts.
One approach for allowing jenkins to write to fuego-rw, is to chown /fuego-rw and it's pre-existing subdirectories during execution time, what can be done in a docker entrypoint script. That approach per se, solves the problem, by allowing jenkins to write to the mount point, but generates a side-effect: the user on the host machine will be able to inspect, but will not be able to delete/modify the files created by jenkins (unless it forces it as root).
Another approach would be to change the uid and the gid of the jenkins user in the first execution, (first call to docker-container-start.sh), also through the entrypoint. This would solve the problem of the access from both sides, but has the downside of the need of running a recursive chown in all jenkins files in /var (including cache), possibly taking a long time to be accomplished.
Since the need to edit/remove the files from fuego-rw (without sudo) depends on the user, our suggestion is to use the entrypoint for chowning only the pre-existing directories from fuego-rw and to provide the user a script for mapping the jenkins' uid/gid with user's uid/gid only if the user needs to.
On this patch we have created the entrypoint.sh script, and the uid/gid mapping script.
--- Dockerfile | 10 ++++------ frontend-install/entrypoint.sh | 13 +++++++++++++ .../setup/jenkins/map-jenkins-uid-to-host.sh | 12 ++++++++++++ frontend-install/setup/jenkins/setup.sh | 8 -------- fuego-host-scripts/docker-build-image.sh | 10 +--------- fuego-host-scripts/jenkins-map-uid.sh | 18 ++++++++++++++++++ 6 files changed, 48 insertions(+), 23 deletions(-) create mode 100755 frontend-install/entrypoint.sh create mode 100755 frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh create mode 100755 fuego-host-scripts/jenkins-map-uid.sh
Dockerfile +4 -6
@@ -110,16 +110,12 @@ RUN echo deb http://emdebian.org/tools/debian/ jessie main > /etc/apt/sources.li # Download and Install Jenkins # ============================================================================== -ENV uid=1000 -ENV gid=${uid} ARG JENKINS_VERSION=2.32.1 ARG JENKINS_SHA=bfc226aabe2bb089623772950c4cc13aee613af1 ARG JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb ENV JENKINS_HOME=/var/lib/jenkins -RUN groupadd -g ${gid} ${group} && \ - useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user} && \ - curl -L -O ${JENKINS_URL} && \ +RUN curl -L -O ${JENKINS_URL} && \ echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c - && \ dpkg -i jenkins_${JENKINS_VERSION}_all.deb && \ rm jenkins_${JENKINS_VERSION}_all.deb
@@ -156,4 +152,6 @@ COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf # Setup startup command # ============================================================================== -ENTRYPOINT service jenkins start && service netperf start && /bin/bash +WORKDIR / +COPY frontend-install/entrypoint.sh / +ENTRYPOINT ["/entrypoint.sh"]
frontend-install/entrypoint.sh new file +13 -0
@@ -0,0 +1,13 @@ +#!/bin/bash +set -e + +service jenkins stop >> /dev/null + +mkdir -p fuego-rw/boards /fuego-rw/logs /fuego-rw/buildzone +chown jenkins:jenkins \ + fuego-rw fuego-rw/boards /fuego-rw/logs /fuego-rw/buildzone + +service jenkins start +service netperf start + +exec /bin/bash
frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh new file +12 -0
@@ -0,0 +1,12 @@ +#!/bin/bash +set -e + +service jenkins stop + +usermod -u "${jenkins_uid}" jenkins +groupmod -g "${jenkins_gid}" jenkins + +chown -R "${jenkins_uid}":"${jenkins_gid}" \ + /var/lib/jenkins /var/cache/jenkins /var/log/jenkins /fuego-rw + +service jenkins start
frontend-install/setup/jenkins/setup.sh +0 -8
@@ -2,14 +2,6 @@ set -e -user=jenkins -group=jenkins -uid=1000 -gid=${uid} - -# groupadd -g ${gid} ${group} -# useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user} - cp config.xml jenkins.model.JenkinsLocationConfiguration.xml ${JENKINS_HOME} source /etc/default/jenkins
fuego-host-scripts/docker-build-image.sh +1 -9
@@ -2,12 +2,4 @@ # $1 - name for the docker image (default: fuego) DOCKERIMAGE=${1:-fuego} -if [ "$(id -u)" == "0" ]; then - JENKINS_UID=$(id -u $SUDO_USER) - JENKINS_GID=$(id -g $SUDO_USER) -else - JENKINS_UID=$(id -u $USER) - JENKINS_GID=$(id -g $USER) -fi - -sudo docker build -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy --build-arg uid=$JENKINS_UID --build-arg gid=$JENKINS_GID . +sudo docker build -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy .
fuego-host-scripts/jenkins-map-uid.sh new file +18 -0
@@ -0,0 +1,18 @@ +#!/bin/bash +# $1 - name for the docker container (default: fuego-container) +DOCKERCONTAINER=${1:-fuego-container} + +if [ "${UID}" == "0" ]; then + uid=$(id -u "${SUDO_USER}") + gid=$(id -g "${SUDO_USER}") +else + uid="${UID}" + gid=$(id -g "${USER}") +fi + +sudo docker start "${DOCKERCONTAINER}" || \ + echo "Please create Fuego docker container via docker-create-container.sh script" + +sudo docker exec -e jenkins_uid="${uid}" -e jenkins_gid="${gid}" \ + "${DOCKERCONTAINER}" \ + /setup/jenkins/map-jenkins-uid-to-host.sh
Signature · 1 line
2.15.1
Re: [Fuego] [PATCH 5/9] Remove jenkins user created during build time with host's uid/gid
Quoted 9 lines
-----Original Message-----
From: Guilherme Campos CamargoPrior to this patch, the
fuego-host-scripts/docker-build-image.shscript was calling docker build passing twobuild-args: uid/gid. According to the comments in the code, it seems that the intention was to make sure that the jenkins user (needed by Jenkins) would have the same uid/gid as the user of the host that has called the install script.
Quoted 4 lines
Given that the uid/gid of the jenkins user was being set at build time, the generated image would be configured only for the machine where it's built, what would prevent jenkins to change files in the /fuego-rw mountpoint in different hosts.
That's correct, and changing this is a nice step forward to making the container usable on other machines than the one it was built on.
Quoted 9 lines
One approach for allowing jenkins to write to fuego-rw, is to chown /fuego-rw and it's pre-existing subdirectories during execution time, what can be done in a docker entrypoint script. That approach per se, solves the problem, by allowing jenkins to write to the mount point, but generates a side-effect: the user on the host machine will be able to inspect, but will not be able to delete/modify the files created by jenkins (unless it forces it as root).
Quoted 6 lines
Another approach would be to change the uid and the gid of the jenkins user in the first execution, (first call to
docker-container-start.sh), also through the entrypoint. This would solve the problem of the access from both sides, but has the downside of the need of running a recursivechownin all jenkins files in/var(including cache), possibly taking a long time to be accomplished.
I'd much rather do this. This amounts to a one-time container installation overhead on a new host, for the Fuego user.
I modify files in fuego-ro and fuego-rw all the time, from the host side. This is my normal workflow, and I'd like to see it supported.
Quoted 9 lines
Since the need to edit/remove the files from fuego-rw (without sudo) depends on the user, our suggestion is to use the entrypoint for
chowningonly the pre-existing directories from fuego-rw and to provide the user a script for mapping the jenkins' uid/gid with user's uid/gid only if the user needs to.On this patch we have created the
entrypoint.shscript, and the uid/gid mapping script.
At this point I'm confused. Do both approaches have an entrypoint.sh script?
Could you give an example of the steps required to install the container in both your scenarios, including the arguments required for the scripts, and the resulting permissions and ownership both inside and outside the container, for the volumes?
I'm not sure when map-jenkins-uid-to-host.sh and jenkins-map-uid.sh are run, and there seem to be no instructions for using them, or a human-readable description of what they do.
Thanks,
-- Tim
frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh
Quoted 79 lines
Signed-off-by Guilherme Campos Camargo <[email protected]>--- Dockerfile | 10 ++++------ frontend-install/entrypoint.sh | 13 +++++++++++++ .../setup/jenkins/map-jenkins-uid-to-host.sh | 12 ++++++++++++ frontend-install/setup/jenkins/setup.sh | 8 -------- fuego-host-scripts/docker-build-image.sh | 10 +--------- fuego-host-scripts/jenkins-map-uid.sh | 18 ++++++++++++++++++ 6 files changed, 48 insertions(+), 23 deletions(-) create mode 100755 frontend-install/entrypoint.sh create mode 100755 frontend-install/setup/jenkins/map-jenkins-uid-to- host.sh create mode 100755 fuego-host-scripts/jenkins-map-uid.shDockerfile +0
-0@@ -110,16 +110,12 @@ RUN echo deb http://emdebian.org/tools/debian/jessie main >
/etc/apt/sources.li
# Download and Install Jenkins
#
==========================================================
====================Dockerfile +0
-2-ENV uid=1000 -ENV gid=${uid} ARG JENKINS_VERSION=2.32.1 ARG JENKINS_SHA=bfc226aabe2bb089623772950c4cc13aee613af1 ARG JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
ENV JENKINS_HOME=/var/lib/jenkinsDockerfile +0
-2-RUN groupadd -g ${gid} ${group} && \ - useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s
/bin/bash${user} && \Dockerfile +1
-2- curl -L -O ${JENKINS_URL} && \ +RUN curl -L -O ${JENKINS_URL} && \ echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c - && \ dpkg -i jenkins_${JENKINS_VERSION}_all.deb && \ rm jenkins_${JENKINS_VERSION}_all.deb@@ -156,4 +152,6 @@ COPY docs/fuego-docs.pdf$JENKINS_HOME/userContent/docs/fuego-docs.pdf
# Setup startup command
#
==========================================================
====================Dockerfile +3
-1-ENTRYPOINT service jenkins start && service netperf start && /bin/bash +WORKDIR / +COPY frontend-install/entrypoint.sh / +ENTRYPOINT ["/entrypoint.sh"]frontend-install/entrypoint.sh new file +13
-0@@ -0,0 +1,13 @@ +#!/bin/bash +set -e + +service jenkins stop >> /dev/null + +mkdir -p fuego-rw/boards /fuego-rw/logs /fuego-rw/buildzone +chown jenkins:jenkins \ + fuego-rw fuego-rw/boards /fuego-rw/logs /fuego-rw/buildzone + +service jenkins start +service netperf start + +exec /bin/bashfrontend-install/setup/jenkins/map-jenkins-uid-to-host.sh new file +0
-0
Who calls this? When, and with what args?
fuego-host-scripts/jenkins-map-uid.sh
Quoted 63 lines
frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh new file +12
-0@@ -0,0 +1,12 @@ +#!/bin/bash +set -e + +service jenkins stop + +usermod -u "${jenkins_uid}" jenkins +groupmod -g "${jenkins_gid}" jenkins + +chown -R "${jenkins_uid}":"${jenkins_gid}" \ + /var/lib/jenkins /var/cache/jenkins /var/log/jenkins /fuego-rw + +service jenkins startfrontend- renamed from frontend-install/setup/jenkins/setup.sh +0
-0
install/setup/jenkins/setup.sh
index c550862..11ad371 100755frontend-install/setup/jenkins/setup.sh +0
-8@@ -2,14 +2,6 @@ set -e -user=jenkins -group=jenkins -uid=1000 -gid=${uid} - -# groupadd -g ${gid} ${group} -# useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user} - cp config.xml jenkins.model.JenkinsLocationConfiguration.xml ${JENKINS_HOME}source
/etc/default/jenkinsfuego-host- renamed from fuego-host-scripts/docker-build-image.sh +0
-0
scripts/docker-build-image.sh
index 7c11b11..6550a98 100755fuego-host-scripts/docker-build-image.sh +0
-9@@ -2,12 +2,4 @@ # $1 - name for the docker image (default: fuego) DOCKERIMAGE=${1:-fuego} -if [ "$(id -u)" == "0" ]; then - JENKINS_UID=$(id -u $SUDO_USER) - JENKINS_GID=$(id -g $SUDO_USER) -else - JENKINS_UID=$(id -u $USER) - JENKINS_GID=$(id -g $USER) -fi - -sudo docker build -t ${DOCKERIMAGE} --build-argHTTP_PROXY=$http_proxy --build-arg uid=$JENKINS_UID --build-arg gid=$JENKINS_GID .
+sudo docker build -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy .fuego-host- renamed from fuego-host-scripts/jenkins-map-uid.sh +0
-0
scripts/jenkins-map-uid.sh
new file mode 100755
index 0000000..fad619efuego-host-scripts/jenkins-map-uid.sh new file +0
-0
Who calls this? When, and with what args?
Quoted 21 lines
fuego-host-scripts/jenkins-map-uid.sh new file +14
-0@@ -0,0 +1,18 @@ +#!/bin/bash +# $1 - name for the docker container (default: fuego-container) +DOCKERCONTAINER=${1:-fuego-container} + +if [ "${UID}" == "0" ]; then + uid=$(id -u "${SUDO_USER}") + gid=$(id -g "${SUDO_USER}") +else + uid="${UID}" + gid=$(id -g "${USER}") +fi + +sudo docker start "${DOCKERCONTAINER}" || \ + echo "Please create Fuego docker container via docker-create-
container.shscript"fuego-host-scripts/jenkins-map-uid.sh new file +4
-1+ +sudo docker exec -e jenkins_uid="${uid}" -e jenkins_gid="${gid}" \ + "${DOCKERCONTAINER}" \ + /setup/jenkins/map-jenkins-uid-to-host.sh --
Quoted 1 line
2.15.1
I need to understand this better before I apply it.
-- Tim
Re: [Fuego] [PATCH 5/9] Remove jenkins user created during build time with host's uid/gid
Hello, Tim
Thanks for your input.
Quoted 49 lines · On Tue, Jan 30, 2018 at 6:39 PM, <[email protected]> wrote:
Quoted 9 lines
-----Original Message-----
From: Guilherme Campos CamargoPrior to this patch, the
fuego-host-scripts/docker-build-image.shscript was calling docker build passing twobuild-args: uid/gid. According to the comments in the code, it seems that the intention was to make sure that the jenkins user (needed by Jenkins) would have the same uid/gid as the user of the host that has called the install script.Quoted 4 lines
Given that the uid/gid of the jenkins user was being set at build time, the generated image would be configured only for the machine where it's built, what would prevent jenkins to change files in the /fuego-rw mountpoint in different hosts.
That's correct, and changing this is a nice step forward to making the container usable on other machines than the one it was built on.
Quoted 9 lines
One approach for allowing jenkins to write to fuego-rw, is to chown /fuego-rw and it's pre-existing subdirectories during execution time, what can be done in a docker entrypoint script. That approach per se, solves the problem, by allowing jenkins to write to the mount point, but generates a side-effect: the user on the host machine will be able to inspect, but will not be able to delete/modify the files created by jenkins (unless it forces it as root).
Quoted 6 lines
Another approach would be to change the uid and the gid of the jenkins user in the first execution, (first call to
docker-container-start.sh), also through the entrypoint. This would solve the problem of the access from both sides, but has the downside of the need of running a recursivechownin all jenkins files in/var(including cache), possibly taking a long time to be accomplished.I'd much rather do this. This amounts to a one-time container installation overhead on a new host, for the Fuego user.
I modify files in fuego-ro and fuego-rw all the time, from the host side. This is my normal workflow, and I'd like to see it supported.
Quoted 9 lines
Since the need to edit/remove the files from fuego-rw (without sudo) depends on the user, our suggestion is to use the entrypoint for
chowningonly the pre-existing directories from fuego-rw and to provide the user a script for mapping the jenkins' uid/gid with user's uid/gid only if the user needs to.On this patch we have created the
entrypoint.shscript, and the uid/gid mapping script.
Quoted 9 lines
At this point I'm confused. Do both approaches have an
entrypoint.shscript?Could you give an example of the steps required to install the container in both your scenarios, including the arguments required for the scripts, and the resulting permissions and ownership both inside and outside the container, for the volumes?
With both approaches, there would be no change compared to the instructions that
are given in Fuego's Quick Start guide.
$ ./install.sh $ ./fuego-host-scripts/create-container.sh $ ./fuego-host-scripts/start-container.sh
We agree with you, and decided to move to the second approach (set the uid/gid at the first start through the entrypoint).
Explaining:
With the first approach (the one that was implemented in the first version of
my patch), the uid and the gid of the Jenkins user, inside the container, would
be different than the uid/gid of the user that's running the container from the
host. In order to allow Jenkins to write into fuego-rw, we were changing the ownership of fuego-rw, fuego-rw/boards, fuego-rw/logs and fuego-rw/buildzone to
the Jenkins user inside the container (not recursively). This would allow jenkins to write to those directories and the change would propagate to the outside, not allowing the user to modify the contents of fuego-rw anymore.
Apparently there was a misconception from our side, that most users would not use fuego-rw to write, but only to read, and that Jenkins would be the only
one (most of times) to write into that directory.
In order to allow those users to write into that directory as well, we had added a new script (fuego-host-scripts/jenkins-map-uid.sh), that mapped Jenkin's uid/gid to the host user's only on-demand.
In order to run that script the user would simply call without any arguments
$ ./fuego-host-scripts/jenkins-map-uid.sh
On docker, the only way to run a script inside the container is to have that script copied to the container. Therefore, fuego-hosts-scripts/jenkins-map-uid.sh was just calling that other script (setup/jenkins/map-jenkins-uid-to-host.sh) that's been copied during Docker build. "map-jenkins-uid-to-host.sh" is the one that would actually change jenkins uid and chown the nedeed directories.
With the second approach (the one that we have implemented in our fixup), the uid and the gid of the Jenkins user, inside the container, would be the same as the
user that's running the container from the host. What means that, if the owner
of the fuego-rw directory is the same user that's running the container, fuego-rw will be accessible from both sides seamlessly.
The downside of this approach is that the create-container.sh step would take
longer, due to the recursive chowns that are needed after replacing Jenkins' uid/gid with the user's.
However, as you said, given that most of users will actually need to write to
fuego-rw, the second approach is clearly the best, as you agree. So I'm sending
a new patch, implementing the second approach, as a replacement for this one.
I'm rebasing over the new 'next' branch as well.
Quoted 189 lines
I'm not sure when
map-jenkins-uid-to-host.shandjenkins-map-uid.share run, and there seem to be no instructions for using them, or a human-readable description of what they do.Thanks,
-- TimQuoted 5 lines
Signed-off-by Guilherme Campos Camargo <[email protected]>--- Dockerfile | 10 ++++------ frontend-install/entrypoint.sh | 13+++++++++++++
Quoted 1 line
.../setup/jenkins/map-jenkins-uid-to-host.sh | 12++++++++++++
Quoted 3 lines
frontend-install/setup/jenkins/setup.sh | 8 -------- fuego-host-scripts/docker-build-image.sh | 10 +--------- fuego-host-scripts/jenkins-map-uid.sh | 18++++++++++++++++++
Review
Quoted 46 lines
6 files changed, 48 insertions(+), 23 deletions(-) create mode 100755 frontend-install/entrypoint.sh create mode 100755 frontend-install/setup/jenkins/map-jenkins-uid-to- host.sh create mode 100755 fuego-host-scripts/jenkins-map-uid.shDockerfile +0
-0@@ -110,16 +110,12 @@ RUN echo deb http://emdebian.org/tools/debian/jessie main >
/etc/apt/sources.li
# Download and Install Jenkins
#
==========================================================
====================Dockerfile +0
-2-ENV uid=1000 -ENV gid=${uid} ARG JENKINS_VERSION=2.32.1 ARG JENKINS_SHA=bfc226aabe2bb089623772950c4cc13aee613af1 ARG JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
ENV JENKINS_HOME=/var/lib/jenkinsDockerfile +0
-2-RUN groupadd -g ${gid} ${group} && \ - useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s
/bin/bash${user} && \Dockerfile +1
-2- curl -L -O ${JENKINS_URL} && \ +RUN curl -L -O ${JENKINS_URL} && \ echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c - && \ dpkg -i jenkins_${JENKINS_VERSION}_all.deb && \ rm jenkins_${JENKINS_VERSION}_all.deb@@ -156,4 +152,6 @@ COPY docs/fuego-docs.pdf$JENKINS_HOME/userContent/docs/fuego-docs.pdf
# Setup startup command
#
==========================================================
====================Dockerfile +3
-1-ENTRYPOINT service jenkins start && service netperf start && /bin/bash +WORKDIR / +COPY frontend-install/entrypoint.sh / +ENTRYPOINT ["/entrypoint.sh"](quoted diff) +0
-0
b/frontend-install/entrypoint.shfrontend-install/
Quoted 24 lines
new file mode 100755
index 0000000..281b0fefrontend-install/entrypoint.sh new file +13
-0@@ -0,0 +1,13 @@ +#!/bin/bash +set -e + +service jenkins stop >> /dev/null + +mkdir -p fuego-rw/boards /fuego-rw/logs /fuego-rw/buildzone +chown jenkins:jenkins \ + fuego-rw fuego-rw/boards /fuego-rw/logs /fuego-rw/buildzone + +service jenkins start +service netperf start + +exec /bin/bashfrontend-install/ new file +0
-0
setup/jenkins/map-jenkins-uid-to-host.shWho calls this? When, and with what args?
fuego-host-scripts/jenkins-map-uid.sh
Quoted 63 lines
frontend-install/ new file +12
-0@@ -0,0 +1,12 @@ +#!/bin/bash +set -e + +service jenkins stop + +usermod -u "${jenkins_uid}" jenkins +groupmod -g "${jenkins_gid}" jenkins + +chown -R "${jenkins_uid}":"${jenkins_gid}" \ + /var/lib/jenkins /var/cache/jenkins /var/log/jenkins /fuego-rw + +service jenkins startfrontend- renamed from frontend-install/setup/jenkins/setup.sh +0
-0
install/setup/jenkins/setup.sh
index c550862..11ad371 100755frontend-install/setup/jenkins/setup.sh +0
-8@@ -2,14 +2,6 @@ set -e -user=jenkins -group=jenkins -uid=1000 -gid=${uid} - -# groupadd -g ${gid} ${group} -# useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user} - cp config.xml jenkins.model.JenkinsLocationConfiguration.xml ${JENKINS_HOME}source
/etc/default/jenkinsfuego-host- renamed from fuego-host-scripts/docker-build-image.sh +0
-0
scripts/docker-build-image.sh
index 7c11b11..6550a98 100755fuego-host-scripts/docker-build-image.sh +0
-9@@ -2,12 +2,4 @@ # $1 - name for the docker image (default: fuego) DOCKERIMAGE=${1:-fuego} -if [ "$(id -u)" == "0" ]; then - JENKINS_UID=$(id -u $SUDO_USER) - JENKINS_GID=$(id -g $SUDO_USER) -else - JENKINS_UID=$(id -u $USER) - JENKINS_GID=$(id -g $USER) -fi - -sudo docker build -t ${DOCKERIMAGE} --build-argHTTP_PROXY=$http_proxy --build-arg uid=$JENKINS_UID --build-arg gid=$JENKINS_GID .
+sudo docker build -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy .fuego-host- renamed from fuego-host-scripts/jenkins-map-uid.sh +0
-0
scripts/jenkins-map-uid.sh
new file mode 100755
index 0000000..fad619efuego-host-scripts/jenkins-map-uid.sh new file +0
-0Who calls this? When, and with what args?
Quoted 21 lines
fuego-host-scripts/jenkins-map-uid.sh new file +14
-0@@ -0,0 +1,18 @@ +#!/bin/bash +# $1 - name for the docker container (default: fuego-container) +DOCKERCONTAINER=${1:-fuego-container} + +if [ "${UID}" == "0" ]; then + uid=$(id -u "${SUDO_USER}") + gid=$(id -g "${SUDO_USER}") +else + uid="${UID}" + gid=$(id -g "${USER}") +fi + +sudo docker start "${DOCKERCONTAINER}" || \ + echo "Please create Fuego docker container via docker-create-
container.shscript"fuego-host-scripts/jenkins-map-uid.sh new file +4
-1+ +sudo docker exec -e jenkins_uid="${uid}" -e jenkins_gid="${gid}" \ + "${DOCKERCONTAINER}" \ + /setup/jenkins/map-jenkins-uid-to-host.sh --Quoted 1 line
2.15.1
I need to understand this better before I apply it.
-- Tim
PATCH 6/9 Cleanup proxy
In reply to Guilherme Campos CamargoPrior to this patch, http_proxy was being set and saved during build time according to the http_proxy values of the host.
This approach has at least two shortcomings, that are explained below:
1. Any other user that wanted to use a prebuilt image in their host would be unable to do so, because the proxy configuration is matching the host of the builder and not the host of the user.
2. A user of the image would be able to read the proxy configuration that had been set during build from /etc/default/jenkins or the apt-get config file. The proxy config may easily contain sensitive information as the proxy user/password that were set on the host that built the image.
In order to avoid the problems pointed out above, we chose to:
1. Remove the ARG/ENV instantiations of http_proxy from the Dockerfile and use only the builtin docker ARG http_proxy. This will prevent the proxy variable to be available in a running container and in the image history.
2. Instead of using the apt-get global config file, we're just passing http_proxy as a local environment variable to the RUN instruction in which apt-get is being executed.
3. Jenkins /etc/default/jenkins JAVA_ARGS (that contain the proxy information) is now being updated by a script that's been source by /etc/default/jenkins itself. This means that, whenever service jenkins is called, that script will run and append the proxy configurations to the JAVA_ARGS at runtime, not replacing the JAVA_ARGS that are stored in the file.
4. http_proxy variables are being passed as arguments to docker create as well. By doing that, we make sure that the entrypoint will be running with the http_proxy values of the user.
--- Dockerfile | 19 ++----------------- frontend-install/setup/jenkins/set-java-args-proxy.sh | 7 +++++++ frontend-install/setup/jenkins/setup.sh | 15 ++++----------- fuego-host-scripts/docker-build-image.sh | 4 +++- fuego-host-scripts/docker-create-container.sh | 2 ++ .../docker-create-usb-privileged-container.sh | 2 ++ 6 files changed, 20 insertions(+), 29 deletions(-) create mode 100755 frontend-install/setup/jenkins/set-java-args-proxy.sh
Dockerfile +2 -17
@@ -6,14 +6,6 @@ FROM debian:jessie MAINTAINER [email protected] -# ============================================================================== -# Proxy variables -# ============================================================================== - -ARG HTTP_PROXY -ENV http_proxy ${HTTP_PROXY} -ENV https_proxy ${HTTP_PROXY} - # ============================================================================== # Prepare basic image # ==============================================================================
@@ -21,10 +13,7 @@ ENV https_proxy ${HTTP_PROXY} WORKDIR / COPY frontend-install/apt/sources/fuego-debian-jessie.list \ /etc/apt/sources.list.d/fuego-debian-jessie.list -RUN if [ -n "$HTTP_PROXY" ]; then \ - echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; \ - fi && \ - DEBIAN_FRONTEND=noninteractive apt-get update && \ +RUN DEBIAN_FRONTEND=noninteractive apt-get update && \ apt-get -yV install \ apt-utils \ at \
@@ -84,11 +73,7 @@ RUN if [ -n "$HTTP_PROXY" ]; then \ xmlstarlet && \ rm -rf /var/lib/apt/lists/* -RUN echo dash dash/sh boolean false | debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash && \ - if [ -n "$HTTP_PROXY" ]; then \ - echo "use_proxy = on" >> /etc/wgetrc; \ - echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; \ - fi +RUN echo dash dash/sh boolean false | debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash RUN pip install \ filelock \
frontend-install/setup/jenkins/set-java-args-proxy.sh new file +7 -0
@@ -0,0 +1,7 @@ +#!/bin/bash + +if [ -n "${http_proxy}" ]; then + PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/'); + PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/'); + JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost=${PROXYSERVER} -Dhttp.proxyPort=${PROXYPORT} -Dhttps.proxyHost=${PROXYSERVER} -Dhttps.proxyPort=${PROXYPORT}"; +fi
frontend-install/setup/jenkins/setup.sh +4 -11
@@ -2,17 +2,10 @@ set -e -cp config.xml jenkins.model.JenkinsLocationConfiguration.xml ${JENKINS_HOME} +cp config.xml jenkins.model.JenkinsLocationConfiguration.xml "${JENKINS_HOME}" -source /etc/default/jenkins -JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" -sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins +echo 'JENKINS_ARGS="${JENKINS_ARGS} --prefix=/fuego"' >> /etc/default/jenkins +echo 'JAVA_ARGS="${JAVA_ARGS} -Djenkins.install.runSetupWizard=false"' >> /etc/default/jenkins -JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false" -if [ -n "$HTTP_PROXY" ]; then - PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/'); - PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/'); - JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost=${PROXYSERVER} -Dhttp.proxyPort=${PROXYPORT} -Dhttps.proxyHost=${PROXYSERVER} -Dhttps.proxyPort=${PROXYPORT}"; -fi -sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins +echo "source /setup/jenkins/set-java-args-proxy.sh" >> /etc/default/jenkins
fuego-host-scripts/docker-build-image.sh +3 -1
@@ -2,4 +2,6 @@ # $1 - name for the docker image (default: fuego) DOCKERIMAGE=${1:-fuego} -sudo docker build -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy . +sudo docker build -t ${DOCKERIMAGE} \ + --build-arg http_proxy=${http_proxy} \ + --build-arg https_proxy=${https_proxy} .
fuego-host-scripts/docker-create-container.sh +2 -0
@@ -22,5 +22,7 @@ sudo docker create -it --name ${DOCKERCONTAINER} \ -v $DIR/../fuego-rw:/fuego-rw \ -v $DIR/../fuego-ro:/fuego-ro:ro \ -v $DIR/../../fuego-core:/fuego-core:ro \ + -e http_proxy=${http_proxy} \ + -e https_proxy=${https_proxy:-$http_proxy} \ --net="host" ${DOCKERIMAGE} || \ echo "Could not create fuego-container. See error messages."
fuego-host-scripts/docker-create-usb-privileged-container.sh +2 -0
@@ -28,5 +28,7 @@ sudo docker create -it --name ${DOCKERCONTAINER} \ -v $DIR/../fuego-rw:/fuego-rw \ -v $DIR/../fuego-ro:/fuego-ro:ro \ -v $DIR/../../fuego-core:/fuego-core:ro \ + -e http_proxy=${http_proxy} \ + -e https_proxy=${https_proxy:-$http_proxy} \ --net="host" ${DOCKERIMAGE} || \ echo "Could not create fuego-container. See error messages."
Signature · 1 line
2.15.1
Re: [Fuego] [PATCH 6/9] Cleanup proxy
OK - I had to modify this because I skipped patch 5/9, but I applied it.
I'm not running this from behind a proxy, so I'll have to find someone to test this.
-- Tim
Quoted 215 lines
-----Original Message-----
From: [email protected] [mailto:fuego- [email protected]] On Behalf Of Guilherme Campos Camargo
Sent: Friday, January 26, 2018 9:36 AM
To: [email protected]
Subject: [Fuego] [PATCH 6/9] Cleanup proxyPrior to this patch,
http_proxywas being set and saved during build time according to thehttp_proxyvalues of the host.This approach has at least two shortcomings, that are explained below:
1. Any other user that wanted to use a prebuilt image in their host would be unable to do so, because the proxy configuration is matching the host of the builder and not the host of the user.
2. A user of the image would be able to read the proxy configuration that had been set during build from
/etc/default/jenkinsor the apt-get config file. The proxy config may easily contain sensitive information as the proxy user/password that were set on the host that built the image.In order to avoid the problems pointed out above, we chose to:
1. Remove the ARG/ENV instantiations of
http_proxyfrom the Dockerfile and use only the builtin docker ARGhttp_proxy. This will prevent the proxy variable to be available in a running container and in the image history.2. Instead of using the apt-get global config file, we're just passing
http_proxyas a local environment variable to the RUN instruction in which apt-get is being executed.3. Jenkins
/etc/default/jenkinsJAVA_ARGS(that contain the proxy information) is now being updated by a script that's been source by/etc/default/jenkinsitself. This means that, wheneverservice jenkinsis called, that script will run and append the proxy configurations to theJAVA_ARGSat runtime, not replacing theJAVA_ARGSthat are stored in the file.4.
http_proxyvariables are being passed as arguments todocker createas well. By doing that, we make sure that the entrypoint will be running with thehttp_proxyvalues of the user.Signed-off-by Guilherme Campos Camargo <[email protected]>--- Dockerfile | 19 ++----------------- frontend-install/setup/jenkins/set-java-args-proxy.sh | 7 +++++++ frontend-install/setup/jenkins/setup.sh | 15 ++++----------- fuego-host-scripts/docker-build-image.sh | 4 +++- fuego-host-scripts/docker-create-container.sh | 2 ++ .../docker-create-usb-privileged-container.sh | 2 ++ 6 files changed, 20 insertions(+), 29 deletions(-) create mode 100755 frontend-install/setup/jenkins/set-java-args-proxy.shDockerfile +0
-1@@ -6,14 +6,6 @@ FROM debian:jessie MAINTAINER [email protected] -#==========================================================
====================Dockerfile +0
-2-# Proxy variables -#==========================================================
====================Dockerfile +0
-5- -ARG HTTP_PROXY -ENV http_proxy ${HTTP_PROXY} -ENV https_proxy ${HTTP_PROXY} - #==========================================================
====================
# Prepare basic image
#
==========================================================
====================Dockerfile +1
-4@@ -21,10 +13,7 @@ ENV https_proxy ${HTTP_PROXY} WORKDIR / COPY frontend-install/apt/sources/fuego-debian-jessie.list \ /etc/apt/sources.list.d/fuego-debian-jessie.list -RUN if [ -n "$HTTP_PROXY" ]; then \ - echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; \ - fi && \ - DEBIAN_FRONTEND=noninteractive apt-get update && \ +RUN DEBIAN_FRONTEND=noninteractive apt-get update && \ apt-get -yV install \ apt-utils \at \Dockerfile +1
-5@@ -84,11 +73,7 @@ RUN if [ -n "$HTTP_PROXY" ]; then \ xmlstarlet && \ rm -rf /var/lib/apt/lists/* -RUN echo dash dash/sh boolean false | debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash && \ - if [ -n "$HTTP_PROXY" ]; then \ - echo "use_proxy = on" >> /etc/wgetrc; \ - echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; \ - fi +RUN echo dash dash/sh boolean false | debconf-set-selections ;DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
RUN pip install \ filelock \frontend-install/setup/jenkins/set-java-args-proxy.sh new file +4
-0@@ -0,0 +1,7 @@ +#!/bin/bash + +if [ -n "${http_proxy}" ]; then + PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E's/\///g' | sed -E 's/(.*):(.*)/\1/');
frontend-install/setup/jenkins/set-java-args-proxy.sh new file +1
-0+ PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E's/\///g' | sed -E 's/(.*):(.*)/\2/');
frontend-install/setup/jenkins/set-java-args-proxy.sh new file +1
-0+ JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost=${PROXYSERVER} -Dhttp.proxyPort=${PROXYPORT} -Dhttps.proxyHost=${PROXYSERVER} - Dhttps.proxyPort=${PROXYPORT}";
frontend-install/setup/jenkins/set-java-args-proxy.sh new file +1
-0+fifrontend- renamed from frontend-install/setup/jenkins/setup.sh +0
-0
install/setup/jenkins/setup.sh
index 11ad371..b95fa91 100755frontend-install/setup/jenkins/setup.sh +1
-1@@ -2,17 +2,10 @@ set -e -cp config.xml jenkins.model.JenkinsLocationConfiguration.xml ${JENKINS_HOME} +cp config.xml jenkins.model.JenkinsLocationConfiguration.xml"${JENKINS_HOME}"
frontend-install/setup/jenkins/setup.sh +0
-3-source /etc/default/jenkins -JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" -sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g"
/etc/default/jenkins
+echo 'JENKINS_ARGS="${JENKINS_ARGS} --prefix=/fuego"' >>/etc/default/jenkins
+echo 'JAVA_ARGS="${JAVA_ARGS} - Djenkins.install.runSetupWizard=false"' >>/etc/default/jenkinsfrontend-install/setup/jenkins/setup.sh +0
-3-JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false" -if [ -n "$HTTP_PROXY" ]; then - PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E's/\///g' | sed -E 's/(.*):(.*)/\1/');
frontend-install/setup/jenkins/setup.sh +0
-1- PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E's/\///g' | sed -E 's/(.*):(.*)/\2/');
frontend-install/setup/jenkins/setup.sh +0
-1- JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost=${PROXYSERVER} -Dhttp.proxyPort=${PROXYPORT} -Dhttps.proxyHost=${PROXYSERVER} - Dhttps.proxyPort=${PROXYPORT}";
frontend-install/setup/jenkins/setup.sh +0
-2-fi -sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g"
/etc/default/jenkins
+echo "source /setup/jenkins/set-java-args-proxy.sh" >>/etc/default/jenkinsfuego-host- renamed from fuego-host-scripts/docker-build-image.sh +0
-0
scripts/docker-build-image.sh
index 6550a98..d276ee0 100755fuego-host-scripts/docker-build-image.sh +0
-1@@ -2,4 +2,6 @@ # $1 - name for the docker image (default: fuego) DOCKERIMAGE=${1:-fuego} -sudo docker build -t ${DOCKERIMAGE} --build-argHTTP_PROXY=$http_proxy .
fuego-host-scripts/docker-build-image.sh +3
-0+sudo docker build -t ${DOCKERIMAGE} \ + --build-arg http_proxy=${http_proxy} \ + --build-arg https_proxy=${https_proxy} .fuego-host- renamed from fuego-host-scripts/docker-create-container.sh +0
-0
scripts/docker-create-container.sh
index 20f4b59..9e3d61a 100755fuego-host-scripts/docker-create-container.sh +2
-0@@ -22,5 +22,7 @@ sudo docker create -it --name ${DOCKERCONTAINER} \ -v $DIR/../fuego-rw:/fuego-rw \ -v $DIR/../fuego-ro:/fuego-ro:ro \ -v $DIR/../../fuego-core:/fuego-core:ro \ + -e http_proxy=${http_proxy} \ + -e https_proxy=${https_proxy:-$http_proxy} \ --net="host" ${DOCKERIMAGE} || \ echo "Could not create fuego-container. See error messages."fuego-host-scripts/docker-create-usb-privileged-container.sh +2
-1@@ -28,5 +28,7 @@ sudo docker create -it --name ${DOCKERCONTAINER} \ -v $DIR/../fuego-rw:/fuego-rw \ -v $DIR/../fuego-ro:/fuego-ro:ro \ -v $DIR/../../fuego-core:/fuego-core:ro \ + -e http_proxy=${http_proxy} \ + -e https_proxy=${https_proxy:-$http_proxy} \ --net="host" ${DOCKERIMAGE} || \ echo "Could not create fuego-container. See error messages." --2.15.1
_______________________________________________
Fuego mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/fuego
PATCH 7/9 Move dpkg-reconfigure dash to first RUN instruction
In reply to Guilherme Campos CamargoDEBIAN_FRONTEND=noninteractive has been explicitely set as an ARG in the beginning of the Dockerfile. That environment variable is set only during build time, not being exported with the container image.
--- Dockerfile | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)
Dockerfile +7 -6
@@ -10,10 +10,11 @@ MAINTAINER [email protected] # Prepare basic image # ============================================================================== -WORKDIR / +ARG DEBIAN_FRONTEND=noninteractive + COPY frontend-install/apt/sources/fuego-debian-jessie.list \ /etc/apt/sources.list.d/fuego-debian-jessie.list -RUN DEBIAN_FRONTEND=noninteractive apt-get update && \ +RUN apt-get update && \ apt-get -yV install \ apt-utils \ at \
@@ -71,9 +72,9 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get update && \ vim \ wget \ xmlstarlet && \ - rm -rf /var/lib/apt/lists/* - -RUN echo dash dash/sh boolean false | debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash + rm -rf /var/lib/apt/lists/* && \ + echo dash dash/sh boolean false | debconf-set-selections && \ + dpkg-reconfigure dash RUN pip install \ filelock \
@@ -83,7 +84,7 @@ RUN pip install \ RUN echo deb http://emdebian.org/tools/debian/ jessie main > /etc/apt/sources.list.d/crosstools.list && \ curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | apt-key add - && \ dpkg --add-architecture armhf && \ - DEBIAN_FRONTEND=noninteractive apt-get update && \ + apt-get update && \ apt-get -yV install \ binutils-arm-linux-gnueabihf \ cpp-arm-linux-gnueabihf \
Signature · 1 line
2.15.1
Re: [Fuego] [PATCH 7/9] Move dpkg-reconfigure dash to first RUN instruction
Looks good. Applied.
-- Tim
Quoted 71 lines
-----Original Message-----
From: [email protected] [mailto:fuego- [email protected]] On Behalf Of Guilherme Campos Camargo
Sent: Friday, January 26, 2018 9:36 AM
To: [email protected]
Subject: [Fuego] [PATCH 7/9] Move dpkg-reconfigure dash to first RUN instructionDEBIAN_FRONTEND=noninteractive has been explicitely set as an ARG in the beginning of the Dockerfile. That environment variable is set only during build time, not being exported with the container image.
Signed-off-by Guilherme Campos Camargo <[email protected]>--- Dockerfile | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)Dockerfile +0
-0@@ -10,10 +10,11 @@ MAINTAINER [email protected] # Prepare basic image #==========================================================
====================Dockerfile +3
-5-WORKDIR / +ARG DEBIAN_FRONTEND=noninteractive + COPY frontend-install/apt/sources/fuego-debian-jessie.list \ /etc/apt/sources.list.d/fuego-debian-jessie.list -RUN DEBIAN_FRONTEND=noninteractive apt-get update && \ +RUN apt-get update && \ apt-get -yV install \ apt-utils \ at \@@ -71,9 +72,9 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get update && \ vim \ wget \ xmlstarlet && \ - rm -rf /var/lib/apt/lists/* - -RUN echo dash dash/sh boolean false | debconf-set-selections ;DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
Dockerfile +4
-1+ rm -rf /var/lib/apt/lists/* && \ + echo dash dash/sh boolean false | debconf-set-selections && \ + dpkg-reconfigure dash RUN pip install \ filelock \@@ -83,7 +84,7 @@ RUN pip install \ RUN echo deb http://emdebian.org/tools/debian/ jessie main > /etc/apt/sources.list.d/crosstools.list && \ curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | apt-key add - && \ dpkg --add-architecture armhf && \ - DEBIAN_FRONTEND=noninteractive apt-get update && \ + apt-get update && \ apt-get -yV install \binutils-arm-linux-gnueabihf \ cpp-arm-linux-gnueabihf \--
2.15.1_______________________________________________
Fuego mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/fuego
PATCH 8/9 Refactor plugin installation script
In reply to Guilherme Campos CamargoOn this patch we're trying to remove the explicit sleeps from the plugin installation script, that are necessary for waiting for Jenkins start/restart and the installation of the plugins.
With this change we hope that faster hosts (or hosts with faster networks) will be able to finish Jenkins plugins installation sooner.
We also took the chance to remove the call to sudo, that are not necessary for using jenkins-cli.
--- frontend-install/setup/jenkins/plugins/install.sh | 44 +++++++++++++++++------ 1 file changed, 33 insertions(+), 11 deletions(-)
frontend-install/setup/jenkins/plugins/install.sh +33 -11
@@ -2,20 +2,42 @@ set -e +function install_jenkins_plugin { + echo "Waiting for Jenkins update center..." + while ! ret=$(java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin "$1" 2>/dev/null); do + sleep "$2" + done + echo "${ret}" +} + +function wait_for_jenkins { + echo "Waiting for Jenkins..." + while ! curl --output /dev/null --silent --head --fail http://localhost:8080/fuego/ ; do + sleep "$1" + done +} + +readonly sleep_time_s=1 +readonly required_plugins=" + description-setter + pegdown-formatter + flot-plotter-plugin/flot.hpi + " + service jenkins start -sleep 30 -sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin description-setter -sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin pegdown-formatter -sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin flot-plotter-plugin/flot.hpi -sleep 10 +wait_for_jenkins ${sleep_time_s} + +for plugin in ${required_plugins}; do + install_jenkins_plugin "${plugin}" "${sleep_time_s}" +done # Let Jenkins install flot before making the symlink service jenkins restart -sleep 30 -rm $JENKINS_HOME/plugins/flot/flot/mod.js -ln -s /fuego-core/engine/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js -ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs -ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/ +wait_for_jenkins 1 -chown -R jenkins:jenkins $JENKINS_HOME/ +rm "$JENKINS_HOME/plugins/flot/flot/mod.js" +ln -s /fuego-core/engine/scripts/mod.js "$JENKINS_HOME/plugins/flot/flot/mod.js" +ln -s /fuego-rw/logs "$JENKINS_HOME/userContent/fuego.logs" +ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/ +chown -R jenkins:jenkins "$JENKINS_HOME/"
Signature · 1 line
2.15.1
Re: [Fuego] [PATCH 8/9] Refactor plugin installation script
OK - applied.
Thanks.
-- Tim
Quoted 86 lines
-----Original Message-----
From: Guilherme Campos Camargo on Friday, January 26, 2018 9:36 AM On this patch we're trying to remove the explicit sleeps from the plugin installation script, that are necessary for waiting for Jenkins start/restart and the installation of the plugins.With this change we hope that faster hosts (or hosts with faster networks) will be able to finish Jenkins plugins installation sooner.
We also took the chance to remove the call to sudo, that are not necessary for using jenkins-cli.
Signed-off-by Guilherme Campos Camargo <[email protected]>--- frontend-install/setup/jenkins/plugins/install.sh | 44 +++++++++++++++++- ----- 1 file changed, 33 insertions(+), 11 deletions(-)frontend- renamed from frontend-install/setup/jenkins/plugins/install.sh +0
-0
install/setup/jenkins/plugins/install.sh
index 0db8bf2..37e8948 100755frontend-install/setup/jenkins/plugins/install.sh +3
-0@@ -2,20 +2,42 @@ set -e +function install_jenkins_plugin { + echo "Waiting for Jenkins update center..." + while ! ret=$(java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -shttp://localhost:8080/fuego install-plugin "$1" 2>/dev/null); do
frontend-install/setup/jenkins/plugins/install.sh +8
-0+ sleep "$2" + done + echo "${ret}" +} + +function wait_for_jenkins { + echo "Waiting for Jenkins..." + while ! curl --output /dev/null --silent --head --failhttp://localhost:8080/fuego/ ; do
frontend-install/setup/jenkins/plugins/install.sh +11
-2+ sleep "$1" + done +} + +readonly sleep_time_s=1 +readonly required_plugins=" + description-setter + pegdown-formatter + flot-plotter-plugin/flot.hpi + " + service jenkins start -sleep 30 -sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -shttp://localhost:8080/fuego install-plugin description-setter -sudo -u jenkins java -jar
/var/cache/jenkins/war/WEB-INF/jenkins-cli.jar-s
http://localhost:8080/fuego install-plugin pegdown-formatter -sudo -u jenkins java -jar/var/cache/jenkins/war/WEB-INF/jenkins-cli.jar-s
http://localhost:8080/fuego install-plugin flot-plotter-plugin/flot.hpifrontend-install/setup/jenkins/plugins/install.sh +5
-4-sleep 10 +wait_for_jenkins ${sleep_time_s} + +for plugin in ${required_plugins}; do + install_jenkins_plugin "${plugin}" "${sleep_time_s}" +done # Let Jenkins install flot before making the symlink service jenkins restart -sleep 30 -rm $JENKINS_HOME/plugins/flot/flot/mod.js -ln -s /fuego-core/engine/scripts/mod.js$JENKINS_HOME/plugins/flot/flot/mod.js
frontend-install/setup/jenkins/plugins/install.sh +3
-3-ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs -ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/ +wait_for_jenkins 1 -chown -R jenkins:jenkins $JENKINS_HOME/ +rm "$JENKINS_HOME/plugins/flot/flot/mod.js" +ln -s /fuego-core/engine/scripts/mod.js"$JENKINS_HOME/plugins/flot/flot/mod.js"
frontend-install/setup/jenkins/plugins/install.sh +3
-1+ln -s /fuego-rw/logs "$JENKINS_HOME/userContent/fuego.logs" +ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/ +chown -R jenkins:jenkins "$JENKINS_HOME/" --2.15.1
PATCH 9/9 Set depth to "one" and download just a single-branch on git clones
In reply to Guilherme Campos CamargoIn order to improve git clone speeds
--- Dockerfile | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
Dockerfile +6 -3
@@ -120,13 +120,16 @@ RUN ./install.sh COPY frontend-install/setup/serial /setup/serial WORKDIR /setup -RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc && \ +RUN git clone https://github.com/tbird20d/ttc.git \ + --depth 1 --single-branch /usr/local/src/ttc && \ ./serial/setup-ttc.sh -RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio && \ +RUN git clone https://github.com/frowand/serio.git \ + --depth 1 --single-branch /usr/local/src/serio && \ ./serial/setup-serio.sh -RUN git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin && \ +RUN git clone https://github.com/tbird20d/serlogin.git \ + --depth 1 --single-branch /usr/local/src/serlogin && \ ./serial/setup-serlogin.sh COPY frontend-install/setup/lava /setup/lava
Signature · 1 line
2.15.1
Re: [Fuego] [PATCH 9/9] Set depth to "one" and download just a single-branch on git clones
Quoted 5 lines
-----Original Message-----
From: Guilherme Campos Camargo on Friday, January 26, 2018 9:36 AM In order to improve git clone speeds
For repositories this small, this does not speed up the clone operation appreciably. Since it makes the clone commands a bit ugly, I don't think the tradeoff is worth it.
I tested this on my machine (with a relatively fast network), and the speed for all these clones, without the extra args, was under 1 second each. The time difference with the extra args was less than 100 milliseconds.
Do these take longer for you? These are relatively small repositories. -- Tim
Quoted 39 lines
Signed-off-by Guilherme Campos Camargo <[email protected]>--- Dockerfile | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)Dockerfile +6
-3@@ -120,13 +120,16 @@ RUN ./install.sh COPY frontend-install/setup/serial /setup/serial WORKDIR /setup -RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc && \ +RUN git clone https://github.com/tbird20d/ttc.git \ + --depth 1 --single-branch /usr/local/src/ttc && \ ./serial/setup-ttc.sh -RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio && \ +RUN git clone https://github.com/frowand/serio.git \ + --depth 1 --single-branch /usr/local/src/serio && \ ./serial/setup-serio.sh -RUN git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin && \ +RUN git clone https://github.com/tbird20d/serlogin.git \ + --depth 1 --single-branch /usr/local/src/serlogin && \ ./serial/setup-serlogin.shCOPY frontend-install/setup/lava /setup/lava
--
2.15.1_______________________________________________
Fuego mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/fuego