Skip to contentSkip to conversation

Introduce travis builds for github repositories

PATCH In review Michael Santana 6 participants 51 messages Jan 23, 2019 – Mar 26, 2019 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Introduce travis builds for github repositories

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Additionally, the travis service will send an email to the test-report list informing anyone interested in the automated build (including a result).

---
 .ci/linux-build.sh                  | 34 +++++++++++++++++++++++++
 .ci/linux-setup.sh                  |  3 +++
 .travis.yml                         | 39 +++++++++++++++++++++++++++++
 MAINTAINERS                         |  6 +++++
 doc/guides/contributing/patches.rst |  3 +++
 5 files changed, 85 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
.ci/linux-build.sh new file +34 -0
new file mode 100755
index 0000000000..2cfaa05058
@@ -0,0 +1,34 @@
+#!/bin/bash
+
+# check for whether we're clang or gcc
+# setup the right options depending on the environment variables
+# run the build
+
+# Just used for the 'classic' configuration system (ie: make)
+set_conf() {
+    c="$1/.config"
+    shift
+
+    if grep -q "$1" "$c"; then
+        sed -i "s:^$1=.*$:$1=$2:g" $c
+    else
+        echo $1=$2 >> "$c"
+    fi
+}
+
+
+if [ "${NINJABUILD}" == "1" ]; then
+    meson build
+    ninja -C build
+else
+    make config T=x86_64-native-linuxapp-${CC}
+    if [ "${SHARED}" == "1" ]; then
+        set_conf build CONFIG_RTE_BUILD_SHARED_LIB y
+    fi
+
+    if [ "${KERNEL}" == "1" ]; then
+        echo Unsupported kernel builds at the moment
+    fi
+
+    make all
+fi
.ci/linux-setup.sh new file +3 -0
new file mode 100755
index 0000000000..6f9849cb94
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+python3.5 -m pip install --upgrade meson --user
.travis.yml new file +39 -0
new file mode 100644
index 0000000000..432d6c9c6c
@@ -0,0 +1,39 @@
+language: c
+compiler:
+  - gcc
+  - clang
+
+os:
+  - linux
+
+addons:
+  apt:
+    sources:
+      - deadsnakes #source for python 3.5
+      - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+    packages:
+      - libnuma-dev
+      - linux-headers-$(uname -r)
+      - python3.5
+      - python3-pip
+      - ninja-build
+
+before_install: ./.ci/${TRAVIS_OS_NAME}-setup.sh
+
+sudo: false
+
+env:
+  - SHARED=1
+  - KERNEL=1
+  - NINJABUILD=1
+
+matrix:
+  include:
+    - compiler: clang
+
+script: ./.ci/${TRAVIS_OS_NAME}-build.sh
+
+notifications:
+  email:
+    recipients:
+      - [email protected]
MAINTAINERS +6 -0
index 66104405e5..14a7bf1284 100644
@@ -119,6 +119,12 @@ F: config/rte_config.h
 F: buildtools/gen-pmdinfo-cfile.sh
 F: buildtools/symlink-drivers-solibs.sh
 
+Public CI
+M: Aaron Conole <[email protected]>
+M: Michael Santana <[email protected]>
+F: .travis.yml
+F: .ci/
+
 ABI versioning
 M: Neil Horman <[email protected]>
 F: lib/librte_compat/
doc/guides/contributing/patches.rst +3 -0
index a64bb03683..745a11a67a 100644
@@ -32,6 +32,9 @@ The mailing list for DPDK development is `[email protected] <http://mails.dpdk.org/ar
 Contributors will need to `register for the mailing list <http://mails.dpdk.org/listinfo/dev>`_ in order to submit patches.
 It is also worth registering for the DPDK `Patchwork <http://patches.dpdk.org/project/dpdk/list/>`_
 
+If you are using the GitHub service, you can link your repository to the ``travis-ci.org`` build service.  When you push
+patches to your repository, the travis service will automatically build your changes.
+
 The development process requires some familiarity with the ``git`` version control system.
 Refer to the `Pro Git Book <http://www.git-scm.com/book/>`_ for further information.
 
Signature · 1 line
2.19.1
#1 lore raw
Quoted 45 lines · Michael Santana

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Additionally, the travis service will send an email to the test-report list informing anyone interested in the automated build (including a result).

---
 .ci/linux-build.sh                  | 34 +++++++++++++++++++++++++
 .ci/linux-setup.sh                  |  3 +++
 .travis.yml                         | 39 +++++++++++++++++++++++++++++
 MAINTAINERS                         |  6 +++++
 doc/guides/contributing/patches.rst |  3 +++
 5 files changed, 85 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
.ci/linux-build.sh new file +12 -0
new file mode 100755
index 0000000000..2cfaa05058
@@ -0,0 +1,34 @@
+#!/bin/bash
+
+# check for whether we're clang or gcc
+# setup the right options depending on the environment variables
+# run the build
+
+# Just used for the 'classic' configuration system (ie: make)
+set_conf() {
+    c="$1/.config"
+    shift
+
+    if grep -q "$1" "$c"; then

.ci/linux-build.sh line 22

Quoted 10 lines
.ci/linux-build.sh new file +10 -0
+        sed -i "s:^$1=.*$:$1=$2:g" $c
+    else
+        echo $1=$2 >> "$c"
+    fi
+}
+
+
+if [ "${NINJABUILD}" == "1" ]; then
+    meson build
+    ninja -C build

Would calling test-meson-builds.sh be a good option here? It runs multiple builds rather than a single one.

Quoted 37 lines
.ci/linux-build.sh new file +12 -0
+else
+    make config T=x86_64-native-linuxapp-${CC}
+    if [ "${SHARED}" == "1" ]; then
+        set_conf build CONFIG_RTE_BUILD_SHARED_LIB y
+    fi
+
+    if [ "${KERNEL}" == "1" ]; then
+        echo Unsupported kernel builds at the moment
+    fi
+
+    make all
+fi
.ci/linux-setup.sh new file +3 -0
new file mode 100755
index 0000000000..6f9849cb94
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+python3.5 -m pip install --upgrade meson --user
.travis.yml new file +10 -0
new file mode 100644
index 0000000000..432d6c9c6c
@@ -0,0 +1,39 @@
+language: c
+compiler:
+  - gcc
+  - clang
+
+os:
+  - linux
+
+addons:
+  apt:

.travis.yml line 20

Quoted 10 lines
.travis.yml new file +10 -0
+    sources:
+      - deadsnakes #source for python 3.5
+      - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+    packages:
+      - libnuma-dev
+      - linux-headers-$(uname -r)
+      - python3.5
+      - python3-pip
+      - ninja-build
+

Other optional packages we should consider including: libbsd, pcap, libcrypto, jansson, zlib.

Ideally, I suppose we'd have two setups - one with all the packages, the other only with the minimum.

#2 lore raw
Quoted 15 lines · Bruce Richardson
Quoted 14 lines · Michael Santana

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Additionally, the travis service will send an email to the test-report list informing anyone interested in the automated build (including a result).

Quoted 13 lines
Quoted 13 lines
---
 .ci/linux-build.sh                  | 34 +++++++++++++++++++++++++
 .ci/linux-setup.sh                  |  3 +++
 .travis.yml                         | 39 +++++++++++++++++++++++++++++
 MAINTAINERS                         |  6 +++++
 doc/guides/contributing/patches.rst |  3 +++
 5 files changed, 85 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml

And on a more general note, this is great to see! The more automated CI builds we can get, the better. Thanks.

/Bruce

#3 lore raw
In reply to Bruce Richardson
Quoted 51 lines · Bruce Richardson
Quoted 50 lines · Michael Santana

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Additionally, the travis service will send an email to the test-report list informing anyone interested in the automated build (including a result).

---
 .ci/linux-build.sh                  | 34 +++++++++++++++++++++++++
 .ci/linux-setup.sh                  |  3 +++
 .travis.yml                         | 39 +++++++++++++++++++++++++++++
 MAINTAINERS                         |  6 +++++
 doc/guides/contributing/patches.rst |  3 +++
 5 files changed, 85 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
.ci/linux-build.sh new file +17 -0
new file mode 100755
index 0000000000..2cfaa05058
@@ -0,0 +1,34 @@
+#!/bin/bash
+
+# check for whether we're clang or gcc
+# setup the right options depending on the environment variables
+# run the build
+
+# Just used for the 'classic' configuration system (ie: make)
+set_conf() {
+    c="$1/.config"
+    shift
+
+    if grep -q "$1" "$c"; then
+        sed -i "s:^$1=.*$:$1=$2:g" $c
+    else
+        echo $1=$2 >> "$c"
+    fi
+}
Quoted 8 lines

.ci/linux-build.sh line 22

Quoted 5 lines
.ci/linux-build.sh new file +5 -0
+
+
+if [ "${NINJABUILD}" == "1" ]; then
+    meson build
+    ninja -C build

Would calling test-meson-builds.sh be a good option here? It runs multiple builds rather than a single one.

Maybe it would be better to just fold in the options? For example, we already have support for SHARED vs STATIC builds using the standard config. Then all we do is change to look like:

  DEF_LIB="static"
  if [ "${SHARED}" == "1" ]; then
      DEF_LIB="shared"
  fi

  meson build --werror -Dexamples=all --default-library=${DEF_LIB}

That will build the static vs shared. I like it because it keeps the 'matrix' build from travis (see for example,
https://travis-ci.org/orgcandman/dpdk). Then if some build fails we can dive into which one failed a little bit easier. Plus, test-meson-builds.sh requires all the various compilers be installed to work right. I'd rather just flesh out the ci build.

Quoted 42 lines
Quoted 42 lines
.ci/linux-build.sh new file +12 -0
+else
+    make config T=x86_64-native-linuxapp-${CC}
+    if [ "${SHARED}" == "1" ]; then
+        set_conf build CONFIG_RTE_BUILD_SHARED_LIB y
+    fi
+
+    if [ "${KERNEL}" == "1" ]; then
+        echo Unsupported kernel builds at the moment
+    fi
+
+    make all
+fi
.ci/linux-setup.sh new file +3 -0
new file mode 100755
index 0000000000..6f9849cb94
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+python3.5 -m pip install --upgrade meson --user
.travis.yml new file +15 -0
new file mode 100644
index 0000000000..432d6c9c6c
@@ -0,0 +1,39 @@
+language: c
+compiler:
+  - gcc
+  - clang
+
+os:
+  - linux
+
+addons:
+  apt:
+    sources:
+      - deadsnakes #source for python 3.5
+      - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+    packages:
+      - libnuma-dev
Quoted 8 lines

.travis.yml line 20

Quoted 5 lines
.travis.yml new file +5 -0
+      - linux-headers-$(uname -r)
+      - python3.5
+      - python3-pip
+      - ninja-build
+

Other optional packages we should consider including: libbsd, pcap, libcrypto, jansson, zlib.

Makes sense.

Quoted 2 lines

Ideally, I suppose we'd have two setups - one with all the packages, the other only with the minimum.

One thing I'd really like to incorporate is a bunch of unit tests that I can launch...

Thanks for the review, Bruce!

#4 lore raw
Quoted 73 lines · Aaron Conole
Quoted 51 lines · Bruce Richardson
Quoted 50 lines · Michael Santana

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Additionally, the travis service will send an email to the test-report list informing anyone interested in the automated build (including a result).

---
 .ci/linux-build.sh                  | 34 +++++++++++++++++++++++++
 .ci/linux-setup.sh                  |  3 +++
 .travis.yml                         | 39 +++++++++++++++++++++++++++++
 MAINTAINERS                         |  6 +++++
 doc/guides/contributing/patches.rst |  3 +++
 5 files changed, 85 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
.ci/linux-build.sh new file +17 -0
new file mode 100755
index 0000000000..2cfaa05058
@@ -0,0 +1,34 @@
+#!/bin/bash
+
+# check for whether we're clang or gcc
+# setup the right options depending on the environment variables
+# run the build
+
+# Just used for the 'classic' configuration system (ie: make)
+set_conf() {
+    c="$1/.config"
+    shift
+
+    if grep -q "$1" "$c"; then
+        sed -i "s:^$1=.*$:$1=$2:g" $c
+    else
+        echo $1=$2 >> "$c"
+    fi
+}
Quoted 8 lines

.ci/linux-build.sh line 22

Quoted 5 lines
.ci/linux-build.sh new file +5 -0
+
+
+if [ "${NINJABUILD}" == "1" ]; then
+    meson build
+    ninja -C build

Would calling test-meson-builds.sh be a good option here? It runs multiple builds rather than a single one.

Maybe it would be better to just fold in the options? For example, we already have support for SHARED vs STATIC builds using the standard config. Then all we do is change to look like:

  DEF_LIB="static"
  if [ "${SHARED}" == "1" ]; then
      DEF_LIB="shared"
  fi

  meson build --werror -Dexamples=all --default-library=${DEF_LIB}
Quoted 6 lines

That will build the static vs shared. I like it because it keeps the 'matrix' build from travis (see for example,
https://travis-ci.org/orgcandman/dpdk). Then if some build fails we can dive into which one failed a little bit easier. Plus, test-meson-builds.sh requires all the various compilers be installed to work right. I'd rather just flesh out the ci build.

That's fine doing it that way. I don't mind what way we get the coverage. I imagine we can easily expand the coverage to check default vs native builds, and 64-bit vs 32-bit builds later.

Quoted 10 lines
Quoted 10 lines

.ci/linux-build.sh line 31

Quoted 10 lines
.ci/linux-build.sh new file +9 -0
+else
+    make config T=x86_64-native-linuxapp-${CC}
+    if [ "${SHARED}" == "1" ]; then
+        set_conf build CONFIG_RTE_BUILD_SHARED_LIB y
+    fi
+
+    if [ "${KERNEL}" == "1" ]; then
+        echo Unsupported kernel builds at the moment
+    fi

There is the --Denable_kmods parameter to meson which is equivalent to this, if you want to include it in the meson parameters.

Quoted 44 lines
Quoted 33 lines
Quoted 33 lines
.ci/linux-build.sh new file +3 -0
+
+    make all
+fi
.ci/linux-setup.sh new file +3 -0
new file mode 100755
index 0000000000..6f9849cb94
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+python3.5 -m pip install --upgrade meson --user
.travis.yml new file +15 -0
new file mode 100644
index 0000000000..432d6c9c6c
@@ -0,0 +1,39 @@
+language: c
+compiler:
+  - gcc
+  - clang
+
+os:
+  - linux
+
+addons:
+  apt:
+    sources:
+      - deadsnakes #source for python 3.5
+      - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+    packages:
+      - libnuma-dev
Quoted 8 lines

.travis.yml line 20

Quoted 5 lines
.travis.yml new file +5 -0
+      - linux-headers-$(uname -r)
+      - python3.5
+      - python3-pip
+      - ninja-build
+

Other optional packages we should consider including: libbsd, pcap, libcrypto, jansson, zlib.

Makes sense.

Quoted 8 lines
Quoted 2 lines

Ideally, I suppose we'd have two setups - one with all the packages, the other only with the minimum.

One thing I'd really like to incorporate is a bunch of unit tests that I can launch...

Thanks for the review, Bruce!

Yes, unit test cleanup is something I'd like to see too. The work done to split the unit tests into suites that can be run using meson is a start, but we need to start ensuring reliable test passing suite by suite.

However, one step at a time.

/Bruce

#5 lore raw
In reply to Michael Santana

23/01/2019 23:07, Michael Santana:

.ci/linux-build.sh line 31

Quoted 12 lines
.ci/linux-build.sh new file +12 -0
+if [ "${NINJABUILD}" == "1" ]; then
+    meson build
+    ninja -C build
+else
+    make config T=x86_64-native-linuxapp-${CC}
+    if [ "${SHARED}" == "1" ]; then
+        set_conf build CONFIG_RTE_BUILD_SHARED_LIB y
+    fi
+
+    if [ "${KERNEL}" == "1" ]; then
+        echo Unsupported kernel builds at the moment
+    fi

Do we really want to support the "make system", given that it is going to be deprecated this year?

#6 lore raw
Quoted 16 lines · Thomas Monjalon

23/01/2019 23:07, Michael Santana:

.ci/linux-build.sh line 31

Quoted 12 lines
.ci/linux-build.sh new file +12 -0
+if [ "${NINJABUILD}" == "1" ]; then
+    meson build
+    ninja -C build
+else
+    make config T=x86_64-native-linuxapp-${CC}
+    if [ "${SHARED}" == "1" ]; then
+        set_conf build CONFIG_RTE_BUILD_SHARED_LIB y
+    fi
+
+    if [ "${KERNEL}" == "1" ]; then
+        echo Unsupported kernel builds at the moment
+    fi

Do we really want to support the "make system", given that it is going to be deprecated this year?

I prefer to keep it for as long as 'make' is a supported build system. Once 'make' is deprecated (or removed, either one), it should definitely be dropped.

Currently, the meson build isn't as well documented (for example, the linux build guide doesn't even refer to meson). So, most likely the average developer who just wants to make a small contribution will run 'make' to build DPDK, and hasn't learned yet how to work with meson/ninja.

Are there plans for which release will officially mark 'make' as deprecated? If it will be 19.02, then let's drop this check in v2, since it won't be around long enough to provide any benefit. If it will be 19.05 or 19.08 then I think it probably will provide some value.

#7 lore raw
In reply to Michael Santana
Quoted 46 lines

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis- CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Additionally, the travis service will send an email to the test-report list informing anyone interested in the automated build (including a result).

---
 .ci/linux-build.sh                  | 34 +++++++++++++++++++++++++
 .ci/linux-setup.sh                  |  3 +++
 .travis.yml                         | 39 +++++++++++++++++++++++++++++
 MAINTAINERS                         |  6 +++++
 doc/guides/contributing/patches.rst |  3 +++
 5 files changed, 85 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
.ci/linux-build.sh new file mode 100755 index renamed from .ci/linux-build.sh +0 -0

0000000000..2cfaa05058

.ci/linux-build.sh new file +14 -0
@@ -0,0 +1,34 @@
+#!/bin/bash
+
+# check for whether we're clang or gcc
+# setup the right options depending on the environment variables # run
+the build
+
+# Just used for the 'classic' configuration system (ie: make)
+set_conf() {
+    c="$1/.config"
+    shift
+
+    if grep -q "$1" "$c"; then
+        sed -i "s:^$1=.*$:$1=$2:g" $c
+    else

.ci/linux-build.sh line 24

Quoted 10 lines
.ci/linux-build.sh new file +10 -0
+        echo $1=$2 >> "$c"
+    fi
+}
+
+
+if [ "${NINJABUILD}" == "1" ]; then
+    meson build
+    ninja -C build
+else
+    make config T=x86_64-native-linuxapp-${CC}

Adding Arm builds would be helpful.

Quoted 103 lines
.ci/linux-build.sh new file +10 -0
+    if [ "${SHARED}" == "1" ]; then
+        set_conf build CONFIG_RTE_BUILD_SHARED_LIB y
+    fi
+
+    if [ "${KERNEL}" == "1" ]; then
+        echo Unsupported kernel builds at the moment
+    fi
+
+    make all
+fi
.ci/linux-setup.sh new file mode 100755 index renamed from .ci/linux-setup.sh +0 -0

0000000000..6f9849cb94

.ci/linux-setup.sh new file +3 -0
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+python3.5 -m pip install --upgrade meson --user
.travis.yml new file +39 -0
new file mode 100644
index 0000000000..432d6c9c6c
@@ -0,0 +1,39 @@
+language: c
+compiler:
+  - gcc
+  - clang
+
+os:
+  - linux
+
+addons:
+  apt:
+    sources:
+      - deadsnakes #source for python 3.5
+      - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+    packages:
+      - libnuma-dev
+      - linux-headers-$(uname -r)
+      - python3.5
+      - python3-pip
+      - ninja-build
+
+before_install: ./.ci/${TRAVIS_OS_NAME}-setup.sh
+
+sudo: false
+
+env:
+  - SHARED=1
+  - KERNEL=1
+  - NINJABUILD=1
+
+matrix:
+  include:
+    - compiler: clang
+
+script: ./.ci/${TRAVIS_OS_NAME}-build.sh
+
+notifications:
+  email:
+    recipients:
+      - [email protected]
MAINTAINERS +6 -0
index 66104405e5..14a7bf1284 100644
@@ -119,6 +119,12 @@ F: config/rte_config.h
 F: buildtools/gen-pmdinfo-cfile.sh
 F: buildtools/symlink-drivers-solibs.sh
 
+Public CI
+M: Aaron Conole <[email protected]>
+M: Michael Santana <[email protected]>
+F: .travis.yml
+F: .ci/
+
 ABI versioning
 M: Neil Horman <[email protected]>
 F: lib/librte_compat/
doc/guides/contributing/patches.rst +0 -0
index a64bb03683..745a11a67a 100644
@@ -32,6 +32,9 @@ The mailing list for DPDK development is

[email protected] <http://mails.dpdk.org/ar Contributors will need to register for the mailing list <http://mails.dpdk.org/listinfo/dev>`_ in order to submit patches.
It is also worth registering for the DPDK `Patchwork
<http://patches.dpdk.org/project/dpdk/list/>`_

doc/guides/contributing/patches.rst +2 -0
+If you are using the GitHub service, you can link your repository to
+the ``travis-ci.org`` build service.  When you push patches to your

repository, the travis service will automatically build your changes. +
The development process requires some familiarity with the ``git`` version
control system.
Refer to the Pro Git Book <http://www.git-scm.com/book/>_ for further
information.

--
2.19.1

#8 lore raw

That's a good suggestion. I will look into it.

~Michael Santana

Quoted 171 lines · Honnappa Nagarahalli
Quoted 4 lines

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is

Travis-

Quoted 3 lines

CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK

project.

Quoted 1 line

It doesn't require any changes from individual developers to enable, but

will

Quoted 4 lines

allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Additionally, the travis service will send an email to the test-report

list

.ci/linux-build.sh new file mode

Quoted 16 lines

informing anyone interested in the automated build (including a result).

---
 .ci/linux-build.sh                  | 34 +++++++++++++++++++++++++
 .ci/linux-setup.sh                  |  3 +++
 .travis.yml                         | 39 +++++++++++++++++++++++++++++
 MAINTAINERS                         |  6 +++++
 doc/guides/contributing/patches.rst |  3 +++
 5 files changed, 85 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
.ci/linux-build.sh new file mode renamed from .ci/linux-build.sh +0 -0

100755 index

Quoted 18 lines

0000000000..2cfaa05058

.ci/linux-build.sh new file +14 -0
@@ -0,0 +1,34 @@
+#!/bin/bash
+
+# check for whether we're clang or gcc
+# setup the right options depending on the environment variables # run
+the build
+
+# Just used for the 'classic' configuration system (ie: make)
+set_conf() {
+    c="$1/.config"
+    shift
+
+    if grep -q "$1" "$c"; then
+        sed -i "s:^$1=.*$:$1=$2:g" $c
+    else

.ci/linux-build.sh line 24

Quoted 10 lines
.ci/linux-build.sh new file +10 -0
+        echo $1=$2 >> "$c"
+    fi
+}
+
+
+if [ "${NINJABUILD}" == "1" ]; then
+    meson build
+    ninja -C build
+else
+    make config T=x86_64-native-linuxapp-${CC}

Adding Arm builds would be helpful.

.ci/linux-setup.sh new file mode

Quoted 11 lines
.ci/linux-build.sh new file +10 -0
+    if [ "${SHARED}" == "1" ]; then
+        set_conf build CONFIG_RTE_BUILD_SHARED_LIB y
+    fi
+
+    if [ "${KERNEL}" == "1" ]; then
+        echo Unsupported kernel builds at the moment
+    fi
+
+    make all
+fi
.ci/linux-setup.sh new file mode renamed from .ci/linux-setup.sh +0 -0

100755 index

Quoted 75 lines

0000000000..6f9849cb94

.ci/linux-setup.sh new file +3 -0
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+python3.5 -m pip install --upgrade meson --user
.travis.yml new file +39 -0
new file mode 100644
index 0000000000..432d6c9c6c
@@ -0,0 +1,39 @@
+language: c
+compiler:
+  - gcc
+  - clang
+
+os:
+  - linux
+
+addons:
+  apt:
+    sources:
+      - deadsnakes #source for python 3.5
+      - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+    packages:
+      - libnuma-dev
+      - linux-headers-$(uname -r)
+      - python3.5
+      - python3-pip
+      - ninja-build
+
+before_install: ./.ci/${TRAVIS_OS_NAME}-setup.sh
+
+sudo: false
+
+env:
+  - SHARED=1
+  - KERNEL=1
+  - NINJABUILD=1
+
+matrix:
+  include:
+    - compiler: clang
+
+script: ./.ci/${TRAVIS_OS_NAME}-build.sh
+
+notifications:
+  email:
+    recipients:
+      - [email protected]
MAINTAINERS +6 -0
index 66104405e5..14a7bf1284 100644
@@ -119,6 +119,12 @@ F: config/rte_config.h
 F: buildtools/gen-pmdinfo-cfile.sh
 F: buildtools/symlink-drivers-solibs.sh
 
+Public CI
+M: Aaron Conole <[email protected]>
+M: Michael Santana <[email protected]>
+F: .travis.yml
+F: .ci/
+
 ABI versioning
 M: Neil Horman <[email protected]>
 F: lib/librte_compat/
doc/guides/contributing/patches.rst +0 -0
index a64bb03683..745a11a67a 100644
@@ -32,6 +32,9 @@ The mailing list for DPDK development is
Quoted 2 lines

[email protected] <http://mails.dpdk.org/ar Contributors will need to register for the mailing list <http://mails.dpdk.org/listinfo/dev>`_

in order

Quoted 9 lines

to submit patches.
It is also worth registering for the DPDK `Patchwork
<http://patches.dpdk.org/project/dpdk/list/>`_

doc/guides/contributing/patches.rst +2 -0
+If you are using the GitHub service, you can link your repository to
+the ``travis-ci.org`` build service.  When you push patches to your

repository, the travis service will automatically build your changes. +
The development process requires some familiarity with the ``git``

version

Quoted 6 lines

control system.
Refer to the Pro Git Book <http://www.git-scm.com/book/>_ for further
information.

--
2.19.1

#9 lore raw

v2 cover Introduce travis support

In reply to Michael Santana

This series introduces the ability for any github mirrors of the DPDK project, including developer mirrors, to kick off builds under the travis CI infrastructure. For now, this just means compilation - no other kinds of automated run exists yet. In the future, this can be expanded to execute and report results for any test-suites that might exist.

The series includes support for both the 'classic make' style builds (which are set to be deprecated at some undetermined point in the future), as well as the modern meson+ninja build system. Additionally, there is support for building the ARM64 target with both the meson and make systems.

The files added under .ci/ exist so that in the future, other CI support platforms (such as cirrus, appveyor, etc.) could have a common place to put their requisite scripts without polluting the main tree.

Some documentation is updated, making developers aware of the new travis integration.

The integration can also be included on the official DPDK mirror. Build reports can be enabled by subscribing the travis build email to the test-reports mailing list (this can be done independent of this series being applied).

v1 -> v2:
- Added patch 1/2, "examples/vhost_scsi: don't build..."
- Included arm64 builds
- Included multiple meson+ninja builds (full library, minimal library)
- Included multiple 'classic make' builds

Aaron Conole (2):
examples/vhost_scsi: Don't build without virtio_scsi.h
ci: Introduce travis builds for github repositories

 .ci/linux-build.sh                  |  88 +++++++++++++++
 .ci/linux-setup.sh                  |  31 ++++++
 .travis.yml                         | 160 ++++++++++++++++++++++++++++
 MAINTAINERS                         |   7 ++
 doc/guides/contributing/patches.rst |   4 +
 examples/vhost_scsi/meson.build     |   6 ++
 meson_cross_aarch64_gcc.txt         |  12 +++
 7 files changed, 308 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
 create mode 100644 meson_cross_aarch64_gcc.txt
Signature · 1 line
2.19.1
#10 lore raw

PATCH v2 1/2 examples/vhost_scsi: Don't build without virtio_scsi.h

From: Aaron Conole <[email protected]>

The vhost_scsi example code is set to build, even if the requisite header file virtio_scsi.h isn't available. This happens on some Ubuntu systems when some versions of the libc-dev package aren't available.

Check whether the virtio_scsi.h file exists, and if not, set the build flag to false.

---
v2:
Introduced.

 examples/vhost_scsi/meson.build | 6 ++++++
 1 file changed, 6 insertions(+)
examples/vhost_scsi/meson.build +6 -0
index 2303bcaed..3bff05c81 100644
@@ -9,6 +9,12 @@
 if host_machine.system() != 'linux'
 	build = false
 endif
+
+cc = meson.get_compiler('c')
+if not cc.has_header('virtio_scsi.h')
+	build = false
+endif
+
 deps += 'vhost'
 cflags += ['-D_FILE_OFFSET_BITS=64']
 sources = files(
Signature · 1 line
2.19.1
Acked-by on v2 1/2
#11 lore raw

Re: [PATCH v2 1/2] examples/vhost_scsi: Don't build without virtio_scsi.h

Quoted 22 lines · Michael Santana

From: Aaron Conole <[email protected]>

The vhost_scsi example code is set to build, even if the requisite header file virtio_scsi.h isn't available. This happens on some Ubuntu systems when some versions of the libc-dev package aren't available.

Check whether the virtio_scsi.h file exists, and if not, set the build flag to false.

---
v2:
Introduced.

 examples/vhost_scsi/meson.build | 6 ++++++
 1 file changed, 6 insertions(+)
examples/vhost_scsi/meson.build +0 -0
index 2303bcaed..3bff05c81 100644

examples/vhost_scsi/meson.build line 17

Quoted 9 lines
examples/vhost_scsi/meson.build +6 -0
@@ -9,6 +9,12 @@
 if host_machine.system() != 'linux'
 	build = false
 endif
+
+cc = meson.get_compiler('c')
+if not cc.has_header('virtio_scsi.h')
+	build = false
+endif
+

We already assign "cc" at the top level meson.build file, so it's not needed to be repeated here. We could merge the if conditions, but it's probably more readable this way, so with the "cc = " line removed:

Quoted 6 lines
examples/vhost_scsi/meson.build +0 -0
 deps += 'vhost'
 cflags += ['-D_FILE_OFFSET_BITS=64']
 sources = files(
Signature · 1 line
2.19.1
#12 lore raw

PATCH v2 2/2 ci: Introduce travis builds for github repositories

In reply to Michael Santana

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Additionally, the travis service will send an email to the test-report list informing anyone interested in the automated build (including a result).

---
v2:
- Added aarch64 build
- Added multiple meson options for builds
- Added multiple make/config options for builds

 .ci/linux-build.sh                  |  88 +++++++++++++++
 .ci/linux-setup.sh                  |  31 ++++++
 .travis.yml                         | 159 ++++++++++++++++++++++++++++
 MAINTAINERS                         |   7 ++
 doc/guides/contributing/patches.rst |   4 +
 meson_cross_aarch64_gcc.txt         |  12 +++
 6 files changed, 301 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
 create mode 100644 meson_cross_aarch64_gcc.txt
.ci/linux-build.sh new file +88 -0
new file mode 100755
index 000000000..66d031a21
@@ -0,0 +1,88 @@
+#!/bin/bash
+
+# check for whether we're clang or gcc
+# setup the right options depending on the environment variables
+# run the build
+
+# Just used for the 'classic' configuration system (ie: make)
+set_conf() {
+    echo "[BUILT WITH $2 SET TO $3]"
+    c="$1/.config"
+    shift
+
+    if grep -q "$1" "$c"; then
+        sed -i "s:^$1=.*$:$1=$2:g" $c
+    else
+        echo $1=$2 >> "$c"
+    fi
+}
+
+BUILD_ARCH="x86_64-native-linuxapp-"
+
+if [ "${ARM64}" == "1" ]; then
+    # convert the arch specifier
+    BUILD_ARCH="arm64-armv8a-linuxapp-"
+    ARM64_TOOL="linaro-arm-tool"
+    export PATH=$PATH:$(pwd)/${ARM64_TOOL}/bin
+fi
+
+
+if [ "${NINJABUILD}" == "1" ]; then
+    OPTS=""
+
+    DEF_LIB="static"
+    if [ "${SHARED}" == "1" ]; then
+        DEF_LIB="shared"
+    fi
+
+    if [ "${KERNEL}" == "1" ]; then
+        OPTS="-Denable_kmods=false"
+    fi
+
+    if [ "${ARM64}" == "1" ]; then
+        OPTS="${OPTS} --cross-file meson_cross_aarch64_${CC}.txt"
+    fi
+
+    OPTS="$OPTS --default-library=$DEF_LIB"
+    meson build --werror -Dexamples=all ${OPTS}
+    ninja -C build
+else
+    EXTRA_OPTS=""
+
+    make config T="${BUILD_ARCH}${CC}"
+
+    set_conf build CONFIG_RTE_KNI_KMOD n
+    set_conf build CONFIG_RTE_EAL_IGB_UIO n
+
+    if dpkg --list | grep -q zlib1g ; then
+        set_conf build CONFIG_RTE_LIBRTE_PMD_ZLIB y
+    fi
+
+    if dpkg --list | grep -q libpcap-dev ; then
+        set_conf build CONFIG_RTE_PORT_PCAP y
+    fi
+
+    if [ "${SHARED}" == "1" ]; then
+        set_conf build CONFIG_RTE_BUILD_SHARED_LIB y
+    fi
+
+    if [ "${KERNEL}" == "1" ]; then
+        echo Unsupported kernel builds at the moment
+    fi
+
+    if [ "${ARM64}" == "1" ]; then
+        EXTRA_OPTS="CROSS=aarch64-linux-gnu-"
+
+        # need to turn off these extras
+        set_conf build CONFIG_RTE_PORT_PCAP n
+        set_conf build CONFIG_RTE_LIBRTE_PMD_ZLIB n
+
+        # convert the CC/CXX variables
+        export CC=aarch64-linux-gnu-${CC}
+        export CXX=aarch64-linux-gnu-${CXX}
+        export AR=aarch64-linux-gnu-ar
+        export STRIP=aarch64-linux-gnu-strip
+    fi
+
+    make all ${EXTRA_OPTS}
+fi
.ci/linux-setup.sh new file +31 -0
new file mode 100755
index 000000000..7d6478ef9
@@ -0,0 +1,31 @@
+#!/bin/bash
+
+python3.5 -m pip install --upgrade meson --user
+
+echo "ARM64 is [ ${ARM64} ]"
+
+if [ "${ARM64}" == "1" ]; then
+    # need to build & install libnuma
+    # This will only be minimal support for now.
+    ARM64_TOOL_URL='https://releases.linaro.org/components/toolchain/binaries/latest-7/aarch64-linux-gnu/gcc-linaro-7.4.1-2019.02-x86_64_aarch64-linux-gnu.tar.xz'
+    ARM64_TOOL="linaro-arm-tool"
+    NUMA_GIT_URL="https://github.com/numactl/numactl.git"
+
+    wget -O "${ARM64_TOOL}.tar.xz" "${ARM64_TOOL_URL}"
+    tar -xf "${ARM64_TOOL}.tar.xz"
+    mv gcc-linaro* "${ARM64_TOOL}"
+    export PATH=$PATH:$(pwd)/${ARM64_TOOL}/bin
+    git clone "${NUMA_GIT_URL}"
+    cd numactl
+    git checkout v2.0.11
+    ./autogen.sh
+    autoconf -i
+    mkdir numa_bin
+    ./configure --host=aarch64-linux-gnu CC=aarch64-linux-gnu-gcc \
+                --prefix=$(pwd)/numa_bin
+    make install # install numa
+    cd ..
+    cp numactl/numa_bin/include/numa*.h "${ARM64_TOOL}/aarch64-linux-gnu/libc/usr/include/"
+    cp numactl/numa_bin/lib/libnuma.* "${ARM64_TOOL}/aarch64-linux-gnu/lib64/"
+    cp numactl/numa_bin/lib/libnuma.* "${ARM64_TOOL}/lib/"
+fi
.travis.yml new file +159 -0

159 lines not on this page · raw message

MAINTAINERS +7 -0
index 66104405e..634afc41d 100644
@@ -119,6 +119,13 @@ F: config/rte_config.h
 F: buildtools/gen-pmdinfo-cfile.sh
 F: buildtools/symlink-drivers-solibs.sh
 
+Public CI
+M: Aaron Conole <[email protected]>
+M: Michael Santana <[email protected]>
+F: .travis.yml
+F: .ci/
+F: meson_cross_aarch64_gcc.txt
+
 ABI versioning
 M: Neil Horman <[email protected]>
 F: lib/librte_compat/
doc/guides/contributing/patches.rst +4 -0
index a64bb0368..49e930cbb 100644
@@ -32,6 +32,10 @@ The mailing list for DPDK development is `[email protected] <http://mails.dpdk.org/ar
 Contributors will need to `register for the mailing list <http://mails.dpdk.org/listinfo/dev>`_ in order to submit patches.
 It is also worth registering for the DPDK `Patchwork <http://patches.dpdk.org/project/dpdk/list/>`_
 
+If you are using the GitHub service, you can link your repository to
+the ``travis-ci.org`` build service.  When you push patches to your GitHub
+repository, the travis service will automatically build your changes.
+
 The development process requires some familiarity with the ``git`` version control system.
 Refer to the `Pro Git Book <http://www.git-scm.com/book/>`_ for further information.
 
meson_cross_aarch64_gcc.txt new file +12 -0
new file mode 100644
index 000000000..aee167d13
@@ -0,0 +1,12 @@
+[binaries]
+c = 'aarch64-linux-gnu-gcc'
+cpp = 'aarch64-linux-gnu-g++'
+ar = 'aarch64-linux-gnu-gcc-ar'
+strip = 'aarch64-linux-gnu-strip'
+pkgconfig = 'aarch64-linux-gnu-pkg-config'
+
+[host_machine]
+system = 'linux'
+cpu_family = 'aarch64'
+cpu = 'aarch64'
+endian = 'little'
Signature · 1 line
2.19.1
#13 lore raw

Re: [PATCH v2 2/2] ci: Introduce travis builds for github repositories

Quoted 10 lines · Michael Santana

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Quoted 7 lines

Additionally, the travis service will send an email to the test-report list informing anyone interested in the automated build (including a result).

---

A few comments inline below.

Quoted 17 lines

v2:
- Added aarch64 build
- Added multiple meson options for builds
- Added multiple make/config options for builds

 .ci/linux-build.sh                  |  88 +++++++++++++++
 .ci/linux-setup.sh                  |  31 ++++++
 .travis.yml                         | 159 ++++++++++++++++++++++++++++
 MAINTAINERS                         |   7 ++
 doc/guides/contributing/patches.rst |   4 +
 meson_cross_aarch64_gcc.txt         |  12 +++
 6 files changed, 301 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
 create mode 100644 meson_cross_aarch64_gcc.txt

<snip>

.ci/linux-build.sh line 40

Quoted 12 lines
.ci/linux-build.sh new file +12 -0
+
+if [ "${NINJABUILD}" == "1" ]; then
+    OPTS=""
+
+    DEF_LIB="static"
+    if [ "${SHARED}" == "1" ]; then
+        DEF_LIB="shared"
+    fi
+
+    if [ "${KERNEL}" == "1" ]; then
+        OPTS="-Denable_kmods=false"
+    fi

Is this condition correct? If kernel is set to "1" you want to disable module builds? I think it should be the other way around.

Quoted 5 lines
.ci/linux-build.sh new file +5 -0
+
+    if [ "${ARM64}" == "1" ]; then
+        OPTS="${OPTS} --cross-file meson_cross_aarch64_${CC}.txt"
+    fi
+

<snip>

Quoted 8 lines
.travis.yml new file +2 -0
new file mode 100644
index 000000000..f296d6914
@@ -0,0 +1,159 @@
+language: c
+compiler:

.travis.yml line 12

Quoted 10 lines
.travis.yml new file +10 -0
+  - gcc
+  - clang
+
+os:
+  - linux
+
+addons:
+  apt:
+    sources:
+      - deadsnakes #source for python 3.5

Do we really need python sources? Are the binaries not enough?

Quoted 6 lines
.travis.yml new file +6 -0
+      - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+    packages:
+      - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip, ninja-build]
+
+before_install: ./.ci/${TRAVIS_OS_NAME}-setup.sh
+

<snip>

Quoted 9 lines
meson_cross_aarch64_gcc.txt new file +2 -0
new file mode 100644
index 000000000..aee167d13
@@ -0,0 +1,12 @@
+[binaries]
+c = 'aarch64-linux-gnu-gcc'

meson_cross_aarch64_gcc.txt line 12

Quoted 10 lines
meson_cross_aarch64_gcc.txt new file +10 -0
+cpp = 'aarch64-linux-gnu-g++'
+ar = 'aarch64-linux-gnu-gcc-ar'
+strip = 'aarch64-linux-gnu-strip'
+pkgconfig = 'aarch64-linux-gnu-pkg-config'
+
+[host_machine]
+system = 'linux'
+cpu_family = 'aarch64'
+cpu = 'aarch64'
+endian = 'little'

This looks very similar to the "config/arm/arm64_armv8_linuxapp_gcc" file. I suggest that file be used instead of adding a new one.

/Bruce

#14 lore raw

Re: [PATCH v2 2/2] ci: Introduce travis builds for github repositories

Quoted 41 lines · Bruce Richardson
Quoted 10 lines · Michael Santana

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Quoted 7 lines

Additionally, the travis service will send an email to the test-report list informing anyone interested in the automated build (including a result).

---

A few comments inline below.

Quoted 17 lines

v2:
- Added aarch64 build
- Added multiple meson options for builds
- Added multiple make/config options for builds

 .ci/linux-build.sh                  |  88 +++++++++++++++
 .ci/linux-setup.sh                  |  31 ++++++
 .travis.yml                         | 159 ++++++++++++++++++++++++++++
 MAINTAINERS                         |   7 ++
 doc/guides/contributing/patches.rst |   4 +
 meson_cross_aarch64_gcc.txt         |  12 +++
 6 files changed, 301 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
 create mode 100644 meson_cross_aarch64_gcc.txt

<snip>

Quoted 15 lines

.ci/linux-build.sh line 40

Quoted 12 lines
.ci/linux-build.sh new file +12 -0
+
+if [ "${NINJABUILD}" == "1" ]; then
+    OPTS=""
+
+    DEF_LIB="static"
+    if [ "${SHARED}" == "1" ]; then
+        DEF_LIB="shared"
+    fi
+
+    if [ "${KERNEL}" == "1" ]; then
+        OPTS="-Denable_kmods=false"
+    fi

Is this condition correct? If kernel is set to "1" you want to disable module builds? I think it should be the other way around.

For meson, it's defaulted on (IIUC):

option('enable_kmods', type: 'boolean', value: true,
description: 'build kernel modules')

So this will shut it off. Maybe the test should have been for KERNEL != "1", though. Okay, will adjust.

Quoted 20 lines
Quoted 5 lines
.ci/linux-build.sh new file +5 -0
+
+    if [ "${ARM64}" == "1" ]; then
+        OPTS="${OPTS} --cross-file meson_cross_aarch64_${CC}.txt"
+    fi
+

<snip>

Quoted 12 lines
.travis.yml new file +6 -0
new file mode 100644
index 000000000..f296d6914
@@ -0,0 +1,159 @@
+language: c
+compiler:
+  - gcc
+  - clang
+
+os:
Quoted 8 lines

.travis.yml line 12

Quoted 6 lines
.travis.yml new file +6 -0
+  - linux
+
+addons:
+  apt:
+    sources:
+      - deadsnakes #source for python 3.5

Do we really need python sources? Are the binaries not enough?

The sources section is for package repository configurations. I agree, I'm not sure why this is needed, but probably it has to do with some travis container limitation (I don't know if the right python version is available by default).

Quoted 23 lines
Quoted 6 lines
.travis.yml new file +6 -0
+      - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+    packages:
+      - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip, ninja-build]
+
+before_install: ./.ci/${TRAVIS_OS_NAME}-setup.sh
+

<snip>

Quoted 14 lines
meson_cross_aarch64_gcc.txt new file +8 -0
new file mode 100644
index 000000000..aee167d13
@@ -0,0 +1,12 @@
+[binaries]
+c = 'aarch64-linux-gnu-gcc'
+cpp = 'aarch64-linux-gnu-g++'
+ar = 'aarch64-linux-gnu-gcc-ar'
+strip = 'aarch64-linux-gnu-strip'
+pkgconfig = 'aarch64-linux-gnu-pkg-config'
+
+[host_machine]
Quoted 8 lines

meson_cross_aarch64_gcc.txt line 12

Quoted 5 lines
meson_cross_aarch64_gcc.txt new file +4 -0
+system = 'linux'
+cpu_family = 'aarch64'
+cpu = 'aarch64'
+endian = 'little'

This looks very similar to the "config/arm/arm64_armv8_linuxapp_gcc" file. I suggest that file be used instead of adding a new one.

Agreed, we can drop this. I completely missed the configurations in that directory.

Thanks again, Bruce!

Quoted 1 line

/Bruce

#15 lore raw

Re: [PATCH v2 2/2] ci: Introduce travis builds for github repositories

Quoted 2 lines · Aaron Conole

Bruce Richardson <[email protected]> writes:

<snip>

Quoted 7 lines
Quoted 3 lines

This looks very similar to the "config/arm/arm64_armv8_linuxapp_gcc" file. I suggest that file be used instead of adding a new one.

Agreed, we can drop this. I completely missed the configurations in that directory.

Yes, they probably aren't stored in a great location for discoverability. We should look to move them to a better place in the repo. Suggestions welcome!

/Bruce

#16 lore raw

Re: [PATCH v2 2/2] ci: Introduce travis builds for github repositories

Quoted 8 lines · Bruce Richardson
Quoted 2 lines · Aaron Conole

Bruce Richardson <[email protected]> writes:

<snip>

Quoted 4 lines
Quoted 3 lines

This looks very similar to the "config/arm/arm64_armv8_linuxapp_gcc" file. I suggest that file be used instead of adding a new one.

Quoted 6 lines
Quoted 3 lines

Agreed, we can drop this. I completely missed the configurations in that directory.

Yes, they probably aren't stored in a great location for discoverability. We should look to move them to a better place in the repo. Suggestions welcome!

It's probably okay to leave them there. The build guides do make reference to them (I just missed it, my fault).

Quoted 1 line

/Bruce

#17 lore raw

v3 cover Introduce travis support

In reply to Michael Santana

This series introduces the ability for any github mirrors of the DPDK project, including developer mirrors, to kick off builds under the travis CI infrastructure. For now, this just means compilation - no other kinds of automated run exists yet. In the future, this can be expanded to execute and report results for any test-suites that might exist.

The series includes support for both the 'classic make' style builds (which are set to be deprecated at some undetermined point in the future), as well as the modern meson+ninja build system. Additionally, there is support for building the ARM64 target with both the meson and make systems.

The files added under .ci/ exist so that in the future, other CI support platforms (such as cirrus, appveyor, etc.) could have a common place to put their requisite scripts without polluting the main tree.

Some documentation is updated, making developers aware of the new travis integration.

The integration can also be included on the official DPDK mirror. Build reports can be enabled by subscribing the travis build email to the test-reports mailing list (this can be done independent of this series being applied).

v2->v3:
- Removed duplicate file meson_cross_aarch64_gcc.txt. Used arm64_armv8_linuxapp_gcc file instead
- Renamed ambiguous variable names and comments, including the variable KERNEL to DISABLE_KERNEL_MODULES and comment 'source for python' to 'Repo for python'
- Removed an already-defined variable

v1 -> v2:
- Added patch 1/2, "examples/vhost_scsi: don't build..."
- Included arm64 builds
- Included multiple meson+ninja builds (full library, minimal library)
- Included multiple 'classic make' builds

Aaron Conole (1):
examples/vhost_scsi: Don't build without virtio_scsi.h

Michael Santana (1):
ci: Introduce travis builds for github repositories

 .ci/linux-build.sh                  |  88 +++++++++++++++
 .ci/linux-setup.sh                  |  31 ++++++
 .travis.yml                         | 159 ++++++++++++++++++++++++++++
 MAINTAINERS                         |   7 ++
 doc/guides/contributing/patches.rst |   4 +
 examples/vhost_scsi/meson.build     |   5 +
 6 files changed, 294 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
Signature · 1 line
2.20.1
#18 lore raw

PATCH v3 1/2 examples/vhost_scsi: Don't build without virtio_scsi.h

From: Aaron Conole <[email protected]>

The vhost_scsi example code is set to build, even if the requisite header file virtio_scsi.h isn't available. This happens on some Ubuntu systems when some versions of the libc-dev package aren't available.

Check whether the virtio_scsi.h file exists, and if not, set the build flag to false.

---
v3:
Removed an already-defined variable

 examples/vhost_scsi/meson.build | 5 +++++
 1 file changed, 5 insertions(+)
examples/vhost_scsi/meson.build +5 -0
index 2303bcaed..ca1248023 100644
@@ -9,6 +9,11 @@
 if host_machine.system() != 'linux'
 	build = false
 endif
+
+if not cc.has_header('virtio_scsi.h')
+	build = false
+endif
+
 deps += 'vhost'
 cflags += ['-D_FILE_OFFSET_BITS=64']
 sources = files(
Signature · 1 line
2.20.1
#19 lore raw

PATCH v3 2/2 ci: Introduce travis builds for github repositories

In reply to Michael Santana

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Additionally, the travis service will send an email to the test-report list informing anyone interested in the automated build (including a result).

---
v3:
- Renamed ambiguous variable names and comments, including the variable KERNEL to DISABLE_KERNEL_MODULES and comment 'source for python' to 'Repo for python'
- Removed duplicate file meson_cross_aarch64_gcc.txt. Used arm64_armv8_linuxapp_gcc file instead

 .ci/linux-build.sh                  |  88 +++++++++++++++
 .ci/linux-setup.sh                  |  31 ++++++
 .travis.yml                         | 159 ++++++++++++++++++++++++++++
 MAINTAINERS                         |   7 ++
 doc/guides/contributing/patches.rst |   4 +
 5 files changed, 289 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
.ci/linux-build.sh new file +88 -0
new file mode 100755
index 000000000..7aa90822c
@@ -0,0 +1,88 @@
+#!/bin/bash
+
+# check for whether we're clang or gcc
+# setup the right options depending on the environment variables
+# run the build
+
+# Just used for the 'classic' configuration system (ie: make)
+set_conf() {
+    echo "[BUILT WITH $2 SET TO $3]"
+    c="$1/.config"
+    shift
+
+    if grep -q "$1" "$c"; then
+        sed -i "s:^$1=.*$:$1=$2:g" $c
+    else
+        echo $1=$2 >> "$c"
+    fi
+}
+
+BUILD_ARCH="x86_64-native-linuxapp-"
+
+if [ "${ARM64}" == "1" ]; then
+    # convert the arch specifier
+    BUILD_ARCH="arm64-armv8a-linuxapp-"
+    ARM64_TOOL="linaro-arm-tool"
+    export PATH=$PATH:$(pwd)/${ARM64_TOOL}/bin
+fi
+
+
+if [ "${NINJABUILD}" == "1" ]; then
+    OPTS=""
+
+    DEF_LIB="static"
+    if [ "${SHARED}" == "1" ]; then
+        DEF_LIB="shared"
+    fi
+
+    if [ "${DISABLE_KERNEL_MODULES}" == "1" ]; then
+        OPTS="-Denable_kmods=false"
+    fi
+
+    if [ "${ARM64}" == "1" ]; then
+        OPTS="${OPTS} --cross-file config/arm/arm64_armv8_linuxapp_gcc"
+    fi
+
+    OPTS="$OPTS --default-library=$DEF_LIB"
+    meson build --werror -Dexamples=all ${OPTS}
+    ninja -C build
+else
+    EXTRA_OPTS=""
+
+    make config T="${BUILD_ARCH}${CC}"
+
+    set_conf build CONFIG_RTE_KNI_KMOD n
+    set_conf build CONFIG_RTE_EAL_IGB_UIO n
+
+    if dpkg --list | grep -q zlib1g ; then
+        set_conf build CONFIG_RTE_LIBRTE_PMD_ZLIB y
+    fi
+
+    if dpkg --list | grep -q libpcap-dev ; then
+        set_conf build CONFIG_RTE_PORT_PCAP y
+    fi
+
+    if [ "${SHARED}" == "1" ]; then
+        set_conf build CONFIG_RTE_BUILD_SHARED_LIB y
+    fi
+
+    if [ "${DISABLE_KERNEL_MODULES}" == "1" ]; then
+        echo Unsupported kernel builds at the moment
+    fi
+
+    if [ "${ARM64}" == "1" ]; then
+        EXTRA_OPTS="CROSS=aarch64-linux-gnu-"
+
+        # need to turn off these extras
+        set_conf build CONFIG_RTE_PORT_PCAP n
+        set_conf build CONFIG_RTE_LIBRTE_PMD_ZLIB n
+
+        # convert the CC/CXX variables
+        export CC=aarch64-linux-gnu-${CC}
+        export CXX=aarch64-linux-gnu-${CXX}
+        export AR=aarch64-linux-gnu-ar
+        export STRIP=aarch64-linux-gnu-strip
+    fi
+
+    make all ${EXTRA_OPTS}
+fi
.ci/linux-setup.sh new file +31 -0
new file mode 100755
index 000000000..7d6478ef9
@@ -0,0 +1,31 @@
+#!/bin/bash
+
+python3.5 -m pip install --upgrade meson --user
+
+echo "ARM64 is [ ${ARM64} ]"
+
+if [ "${ARM64}" == "1" ]; then
+    # need to build & install libnuma
+    # This will only be minimal support for now.
+    ARM64_TOOL_URL='https://releases.linaro.org/components/toolchain/binaries/latest-7/aarch64-linux-gnu/gcc-linaro-7.4.1-2019.02-x86_64_aarch64-linux-gnu.tar.xz'
+    ARM64_TOOL="linaro-arm-tool"
+    NUMA_GIT_URL="https://github.com/numactl/numactl.git"
+
+    wget -O "${ARM64_TOOL}.tar.xz" "${ARM64_TOOL_URL}"
+    tar -xf "${ARM64_TOOL}.tar.xz"
+    mv gcc-linaro* "${ARM64_TOOL}"
+    export PATH=$PATH:$(pwd)/${ARM64_TOOL}/bin
+    git clone "${NUMA_GIT_URL}"
+    cd numactl
+    git checkout v2.0.11
+    ./autogen.sh
+    autoconf -i
+    mkdir numa_bin
+    ./configure --host=aarch64-linux-gnu CC=aarch64-linux-gnu-gcc \
+                --prefix=$(pwd)/numa_bin
+    make install # install numa
+    cd ..
+    cp numactl/numa_bin/include/numa*.h "${ARM64_TOOL}/aarch64-linux-gnu/libc/usr/include/"
+    cp numactl/numa_bin/lib/libnuma.* "${ARM64_TOOL}/aarch64-linux-gnu/lib64/"
+    cp numactl/numa_bin/lib/libnuma.* "${ARM64_TOOL}/lib/"
+fi
.travis.yml new file +159 -0

159 lines not on this page · raw message

MAINTAINERS +7 -0
index 835d8a201..eed6f69d3 100644
@@ -119,6 +119,13 @@ F: config/rte_config.h
 F: buildtools/gen-pmdinfo-cfile.sh
 F: buildtools/symlink-drivers-solibs.sh
 
+Public CI
+M: Aaron Conole <[email protected]>
+M: Michael Santana <[email protected]>
+F: .travis.yml
+F: .ci/
+F: meson_cross_aarch64_gcc.txt
+
 ABI versioning
 M: Neil Horman <[email protected]>
 F: lib/librte_compat/
doc/guides/contributing/patches.rst +4 -0
index a64bb0368..49e930cbb 100644
@@ -32,6 +32,10 @@ The mailing list for DPDK development is `[email protected] <http://mails.dpdk.org/ar
 Contributors will need to `register for the mailing list <http://mails.dpdk.org/listinfo/dev>`_ in order to submit patches.
 It is also worth registering for the DPDK `Patchwork <http://patches.dpdk.org/project/dpdk/list/>`_
 
+If you are using the GitHub service, you can link your repository to
+the ``travis-ci.org`` build service.  When you push patches to your GitHub
+repository, the travis service will automatically build your changes.
+
 The development process requires some familiarity with the ``git`` version control system.
 Refer to the `Pro Git Book <http://www.git-scm.com/book/>`_ for further information.
 
Signature · 1 line
2.20.1
Acked-by on v3 2/2
#20 lore raw

Re: [PATCH v3 2/2] ci: Introduce travis builds for github repositories

Quoted 16 lines · Michael Santana

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Additionally, the travis service will send an email to the test-report list informing anyone interested in the automated build (including a result).

Quoted 8 lines

---
v3:
- Renamed ambiguous variable names and comments, including the variable KERNEL to DISABLE_KERNEL_MODULES and comment 'source for python' to 'Repo for python'
- Removed duplicate file meson_cross_aarch64_gcc.txt. Used arm64_armv8_linuxapp_gcc file instead

No further comments from at this stage.

#21 lore raw

Re: [PATCH v3 2/2] ci: Introduce travis builds for github repositories

In reply to Michael Santana

Hi Michael/Aaron,
Thanks for adding the Arm build. I have few comments inline.

Quoted 59 lines

-----Original Message-----
From: Michael Santana <[email protected]>
Sent: Friday, February 1, 2019 10:48 AM
To: [email protected]
Cc: Aaron Conole <[email protected]>; Bruce Richardson <[email protected]>; Honnappa Nagarahalli <[email protected]>; [email protected]
Subject: [PATCH v3 2/2] ci: Introduce travis builds for github repositories

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Additionally, the travis service will send an email to the test-report list informing anyone interested in the automated build (including a result).

---
v3:
- Renamed ambiguous variable names and comments, including the variable KERNEL to DISABLE_KERNEL_MODULES and comment 'source for python' to 'Repo for python'
- Removed duplicate file meson_cross_aarch64_gcc.txt. Used arm64_armv8_linuxapp_gcc file instead

 .ci/linux-build.sh                  |  88 +++++++++++++++
 .ci/linux-setup.sh                  |  31 ++++++
 .travis.yml                         | 159 ++++++++++++++++++++++++++++
 MAINTAINERS                         |   7 ++
 doc/guides/contributing/patches.rst |   4 +
 5 files changed, 289 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
.ci/linux-build.sh new file mode 100755 index renamed from .ci/linux-build.sh +0 -0

000000000..7aa90822c

.ci/linux-build.sh new file +12 -0
@@ -0,0 +1,88 @@
+#!/bin/bash
+
+# check for whether we're clang or gcc
+# setup the right options depending on the environment variables # run
+the build
+
+# Just used for the 'classic' configuration system (ie: make)
+set_conf() {
+    echo "[BUILT WITH $2 SET TO $3]"
+    c="$1/.config"
+    shift
+

.ci/linux-build.sh line 22

Quoted 10 lines
.ci/linux-build.sh new file +10 -0
+    if grep -q "$1" "$c"; then
+        sed -i "s:^$1=.*$:$1=$2:g" $c
+    else
+        echo $1=$2 >> "$c"
+    fi
+}
+
+BUILD_ARCH="x86_64-native-linuxapp-"
+
+if [ "${ARM64}" == "1" ]; then

Would be good to change the variable names from ARM64 to 'AARCH64...'

Quoted 132 lines
.ci/linux-build.sh new file +66 -0
+    # convert the arch specifier
+    BUILD_ARCH="arm64-armv8a-linuxapp-"
+    ARM64_TOOL="linaro-arm-tool"
+    export PATH=$PATH:$(pwd)/${ARM64_TOOL}/bin
+fi
+
+
+if [ "${NINJABUILD}" == "1" ]; then
+    OPTS=""
+
+    DEF_LIB="static"
+    if [ "${SHARED}" == "1" ]; then
+        DEF_LIB="shared"
+    fi
+
+    if [ "${DISABLE_KERNEL_MODULES}" == "1" ]; then
+        OPTS="-Denable_kmods=false"
+    fi
+
+    if [ "${ARM64}" == "1" ]; then
+        OPTS="${OPTS} --cross-file config/arm/arm64_armv8_linuxapp_gcc"
+    fi
+
+    OPTS="$OPTS --default-library=$DEF_LIB"
+    meson build --werror -Dexamples=all ${OPTS}
+    ninja -C build
+else
+    EXTRA_OPTS=""
+
+    make config T="${BUILD_ARCH}${CC}"
+
+    set_conf build CONFIG_RTE_KNI_KMOD n
+    set_conf build CONFIG_RTE_EAL_IGB_UIO n
+
+    if dpkg --list | grep -q zlib1g ; then
+        set_conf build CONFIG_RTE_LIBRTE_PMD_ZLIB y
+    fi
+
+    if dpkg --list | grep -q libpcap-dev ; then
+        set_conf build CONFIG_RTE_PORT_PCAP y
+    fi
+
+    if [ "${SHARED}" == "1" ]; then
+        set_conf build CONFIG_RTE_BUILD_SHARED_LIB y
+    fi
+
+    if [ "${DISABLE_KERNEL_MODULES}" == "1" ]; then
+        echo Unsupported kernel builds at the moment
+    fi
+
+    if [ "${ARM64}" == "1" ]; then
+        EXTRA_OPTS="CROSS=aarch64-linux-gnu-"
+
+        # need to turn off these extras
+        set_conf build CONFIG_RTE_PORT_PCAP n
+        set_conf build CONFIG_RTE_LIBRTE_PMD_ZLIB n
+
+        # convert the CC/CXX variables
+        export CC=aarch64-linux-gnu-${CC}
+        export CXX=aarch64-linux-gnu-${CXX}
+        export AR=aarch64-linux-gnu-ar
+        export STRIP=aarch64-linux-gnu-strip
+    fi
+
+    make all ${EXTRA_OPTS}
+fi
.ci/linux-setup.sh new file mode 100755 index renamed from .ci/linux-setup.sh +0 -0

000000000..7d6478ef9

.ci/linux-setup.sh new file +10 -0
@@ -0,0 +1,31 @@
+#!/bin/bash
+
+python3.5 -m pip install --upgrade meson --user
+
+echo "ARM64 is [ ${ARM64} ]"
+
+if [ "${ARM64}" == "1" ]; then
+    # need to build & install libnuma
+    # This will only be minimal support for now.
+

ARM64_TOOL_URL='https://releases.linaro.org/components/toolchain/binari es/latest-7/aarch64-linux-gnu/gcc-linaro-7.4.1-2019.02-x86_64_aarch64- linux-gnu.tar.xz'

.ci/linux-setup.sh new file +18 -0
+    ARM64_TOOL="linaro-arm-tool"
+    NUMA_GIT_URL="https://github.com/numactl/numactl.git"
+
+    wget -O "${ARM64_TOOL}.tar.xz" "${ARM64_TOOL_URL}"
+    tar -xf "${ARM64_TOOL}.tar.xz"
+    mv gcc-linaro* "${ARM64_TOOL}"
+    export PATH=$PATH:$(pwd)/${ARM64_TOOL}/bin
+    git clone "${NUMA_GIT_URL}"
+    cd numactl
+    git checkout v2.0.11
+    ./autogen.sh
+    autoconf -i
+    mkdir numa_bin
+    ./configure --host=aarch64-linux-gnu CC=aarch64-linux-gnu-gcc \
+                --prefix=$(pwd)/numa_bin
+    make install # install numa
+    cd ..
+    cp numactl/numa_bin/include/numa*.h "${ARM64_TOOL}/aarch64-linux-

gnu/libc/usr/include/"
+ cp numactl/numa_bin/lib/libnuma.* "${ARM64_TOOL}/aarch64-linux-
gnu/lib64/"

.ci/linux-setup.sh new file +2 -0
+    cp numactl/numa_bin/lib/libnuma.* "${ARM64_TOOL}/lib/"
+fi
.travis.yml new file +19 -0
new file mode 100644
index 000000000..afd63aa33
@@ -0,0 +1,159 @@
+language: c
+compiler:
+  - gcc
+  - clang
+
+os:
+  - linux
+
+addons:
+  apt:
+    sources:
+      - deadsnakes  #Repo for python 3.5
+      - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+    packages:
+      - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+
+before_install: ./.ci/${TRAVIS_OS_NAME}-setup.sh
+

.travis.yml line 29

Quoted 10 lines
.travis.yml new file +10 -0
+sudo: false
+
+env:
+  - SHARED=1
+  - DISABLE_KERNEL_MODULES=1
+  - SHARED=1 DISABLE_KERNEL_MODULES=1
+  - NINJABUILD=1
+  - NINJABUILD=1 SHARED=1
+  - NINJABUILD=1 DISABLE_KERNEL_MODULES=1
+  - NINJABUILD=1 SHARED=1 DISABLE_KERNEL_MODULES=1

My understanding is we need to list 'ARM64=1' and 'ARM64=1 NINJABUILD=1' here.

Quoted 148 lines
.travis.yml new file +143 -0
+
+matrix:
+  include:
+  - env: SHARED=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+  - env: DISABLE_KERNEL_MODULES=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+  - env: SHARED=1 DISABLE_KERNEL_MODULES=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+  - env: SHARED=1
+    compiler: clang
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+  - env: DISABLE_KERNEL_MODULES=1
+    compiler: clang
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+  - env: SHARED=1 DISABLE_KERNEL_MODULES=1
+    compiler: clang
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+  - env: ARM64=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), libtool,
+python3.5, python3-pip]
+  - env: ARM64=1 NINJABUILD=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [linux-headers-$(uname -r), libtool, python3.5,
+python3-pip, ninja-build]
+  - env: NINJABUILD=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+  - env: NINJABUILD=1 SHARED=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+  - env: NINJABUILD=1 DISABLE_KERNEL_MODULES=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+  - env: NINJABUILD=1 SHARED=1 DISABLE_KERNEL_MODULES=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+
+
+script: ./.ci/${TRAVIS_OS_NAME}-build.sh
+
+notifications:
+  email:
+    recipients:
+      - [email protected]
MAINTAINERS +0 -0
index 835d8a201..eed6f69d3 100644

MAINTAINERS F: config/rte_config.h · line 127

Quoted 9 lines
MAINTAINERS +6 -0
@@ -119,6 +119,13 @@ F: config/rte_config.h
 F: buildtools/gen-pmdinfo-cfile.sh
 F: buildtools/symlink-drivers-solibs.sh
 
+Public CI
+M: Aaron Conole <[email protected]>
+M: Michael Santana <[email protected]>
+F: .travis.yml
+F: .ci/
+F: meson_cross_aarch64_gcc.txt

Not required

Quoted 27 lines
MAINTAINERS +1 -0
+
 ABI versioning
 M: Neil Horman <[email protected]>
 F: lib/librte_compat/
doc/guides/contributing/patches.rst +0 -0
index a64bb0368..49e930cbb 100644
@@ -32,6 +32,10 @@ The mailing list for DPDK development is

[email protected] <http://mails.dpdk.org/ar Contributors will need to register for the mailing list <http://mails.dpdk.org/listinfo/dev>`_ in order to submit patches.
It is also worth registering for the DPDK `Patchwork
<http://patches.dpdk.org/project/dpdk/list/>`_

doc/guides/contributing/patches.rst +4 -0
+If you are using the GitHub service, you can link your repository to
+the ``travis-ci.org`` build service.  When you push patches to your
+GitHub repository, the travis service will automatically build your changes.
+
 The development process requires some familiarity with the ``git`` version

control system.
Refer to the Pro Git Book <http://www.git-scm.com/book/>_ for further
information.

--
2.20.1

#22 lore raw

Re: [PATCH v3 2/2] ci: Introduce travis builds for github repositories

Quoted 65 lines · Honnappa Nagarahalli

Hi Michael/Aaron,
Thanks for adding the Arm build. I have few comments inline.

Quoted 62 lines

-----Original Message-----
From: Michael Santana <[email protected]>
Sent: Friday, February 1, 2019 10:48 AM
To: [email protected]
Cc: Aaron Conole <[email protected]>; Bruce Richardson <[email protected]>; Honnappa Nagarahalli <[email protected]>; [email protected]
Subject: [PATCH v3 2/2] ci: Introduce travis builds for github repositories

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Additionally, the travis service will send an email to the test-report list informing anyone interested in the automated build (including a result).

---
v3:
- Renamed ambiguous variable names and comments, including the variable KERNEL to DISABLE_KERNEL_MODULES and comment 'source for python' to 'Repo for python'
- Removed duplicate file meson_cross_aarch64_gcc.txt. Used arm64_armv8_linuxapp_gcc file instead

 .ci/linux-build.sh                  |  88 +++++++++++++++
 .ci/linux-setup.sh                  |  31 ++++++
 .travis.yml                         | 159 ++++++++++++++++++++++++++++
 MAINTAINERS                         |   7 ++
 doc/guides/contributing/patches.rst |   4 +
 5 files changed, 289 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
.ci/linux-build.sh new file mode 100755 index renamed from .ci/linux-build.sh +0 -0

000000000..7aa90822c

.ci/linux-build.sh new file +15 -0
@@ -0,0 +1,88 @@
+#!/bin/bash
+
+# check for whether we're clang or gcc
+# setup the right options depending on the environment variables # run
+the build
+
+# Just used for the 'classic' configuration system (ie: make)
+set_conf() {
+    echo "[BUILT WITH $2 SET TO $3]"
+    c="$1/.config"
+    shift
+
+    if grep -q "$1" "$c"; then
+        sed -i "s:^$1=.*$:$1=$2:g" $c
+    else
Quoted 8 lines

.ci/linux-build.sh line 22

Quoted 7 lines
.ci/linux-build.sh new file +7 -0
+        echo $1=$2 >> "$c"
+    fi
+}
+
+BUILD_ARCH="x86_64-native-linuxapp-"
+
+if [ "${ARM64}" == "1" ]; then

Would be good to change the variable names from ARM64 to 'AARCH64...'

Since we need to spin a change anyway, we'll change it.

Quoted 135 lines
Quoted 135 lines
.ci/linux-build.sh new file +66 -0
+    # convert the arch specifier
+    BUILD_ARCH="arm64-armv8a-linuxapp-"
+    ARM64_TOOL="linaro-arm-tool"
+    export PATH=$PATH:$(pwd)/${ARM64_TOOL}/bin
+fi
+
+
+if [ "${NINJABUILD}" == "1" ]; then
+    OPTS=""
+
+    DEF_LIB="static"
+    if [ "${SHARED}" == "1" ]; then
+        DEF_LIB="shared"
+    fi
+
+    if [ "${DISABLE_KERNEL_MODULES}" == "1" ]; then
+        OPTS="-Denable_kmods=false"
+    fi
+
+    if [ "${ARM64}" == "1" ]; then
+        OPTS="${OPTS} --cross-file config/arm/arm64_armv8_linuxapp_gcc"
+    fi
+
+    OPTS="$OPTS --default-library=$DEF_LIB"
+    meson build --werror -Dexamples=all ${OPTS}
+    ninja -C build
+else
+    EXTRA_OPTS=""
+
+    make config T="${BUILD_ARCH}${CC}"
+
+    set_conf build CONFIG_RTE_KNI_KMOD n
+    set_conf build CONFIG_RTE_EAL_IGB_UIO n
+
+    if dpkg --list | grep -q zlib1g ; then
+        set_conf build CONFIG_RTE_LIBRTE_PMD_ZLIB y
+    fi
+
+    if dpkg --list | grep -q libpcap-dev ; then
+        set_conf build CONFIG_RTE_PORT_PCAP y
+    fi
+
+    if [ "${SHARED}" == "1" ]; then
+        set_conf build CONFIG_RTE_BUILD_SHARED_LIB y
+    fi
+
+    if [ "${DISABLE_KERNEL_MODULES}" == "1" ]; then
+        echo Unsupported kernel builds at the moment
+    fi
+
+    if [ "${ARM64}" == "1" ]; then
+        EXTRA_OPTS="CROSS=aarch64-linux-gnu-"
+
+        # need to turn off these extras
+        set_conf build CONFIG_RTE_PORT_PCAP n
+        set_conf build CONFIG_RTE_LIBRTE_PMD_ZLIB n
+
+        # convert the CC/CXX variables
+        export CC=aarch64-linux-gnu-${CC}
+        export CXX=aarch64-linux-gnu-${CXX}
+        export AR=aarch64-linux-gnu-ar
+        export STRIP=aarch64-linux-gnu-strip
+    fi
+
+    make all ${EXTRA_OPTS}
+fi
.ci/linux-setup.sh new file mode 100755 index renamed from .ci/linux-setup.sh +0 -0

000000000..7d6478ef9

.ci/linux-setup.sh new file +10 -0
@@ -0,0 +1,31 @@
+#!/bin/bash
+
+python3.5 -m pip install --upgrade meson --user
+
+echo "ARM64 is [ ${ARM64} ]"
+
+if [ "${ARM64}" == "1" ]; then
+    # need to build & install libnuma
+    # This will only be minimal support for now.
+

ARM64_TOOL_URL='https://releases.linaro.org/components/toolchain/binari es/latest-7/aarch64-linux-gnu/gcc-linaro-7.4.1-2019.02-x86_64_aarch64- linux-gnu.tar.xz'

.ci/linux-setup.sh new file +18 -0
+    ARM64_TOOL="linaro-arm-tool"
+    NUMA_GIT_URL="https://github.com/numactl/numactl.git"
+
+    wget -O "${ARM64_TOOL}.tar.xz" "${ARM64_TOOL_URL}"
+    tar -xf "${ARM64_TOOL}.tar.xz"
+    mv gcc-linaro* "${ARM64_TOOL}"
+    export PATH=$PATH:$(pwd)/${ARM64_TOOL}/bin
+    git clone "${NUMA_GIT_URL}"
+    cd numactl
+    git checkout v2.0.11
+    ./autogen.sh
+    autoconf -i
+    mkdir numa_bin
+    ./configure --host=aarch64-linux-gnu CC=aarch64-linux-gnu-gcc \
+                --prefix=$(pwd)/numa_bin
+    make install # install numa
+    cd ..
+    cp numactl/numa_bin/include/numa*.h "${ARM64_TOOL}/aarch64-linux-

gnu/libc/usr/include/"
+ cp numactl/numa_bin/lib/libnuma.* "${ARM64_TOOL}/aarch64-linux-
gnu/lib64/"

.ci/linux-setup.sh new file +2 -0
+    cp numactl/numa_bin/lib/libnuma.* "${ARM64_TOOL}/lib/"
+fi
.travis.yml new file +22 -0
new file mode 100644
index 000000000..afd63aa33
@@ -0,0 +1,159 @@
+language: c
+compiler:
+  - gcc
+  - clang
+
+os:
+  - linux
+
+addons:
+  apt:
+    sources:
+      - deadsnakes  #Repo for python 3.5
+      - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+    packages:
+      - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+
+before_install: ./.ci/${TRAVIS_OS_NAME}-setup.sh
+
+sudo: false
+
+env:
Quoted 8 lines

.travis.yml line 28

Quoted 7 lines
.travis.yml new file +7 -0
+  - SHARED=1
+  - DISABLE_KERNEL_MODULES=1
+  - SHARED=1 DISABLE_KERNEL_MODULES=1
+  - NINJABUILD=1
+  - NINJABUILD=1 SHARED=1
+  - NINJABUILD=1 DISABLE_KERNEL_MODULES=1
+  - NINJABUILD=1 SHARED=1 DISABLE_KERNEL_MODULES=1

My understanding is we need to list 'ARM64=1' and 'ARM64=1 NINJABUILD=1' here.

We don't. We have a build explicitly in the matrix section below to cover it.

Quoted 151 lines
Quoted 151 lines
.travis.yml new file +143 -0
+
+matrix:
+  include:
+  - env: SHARED=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+  - env: DISABLE_KERNEL_MODULES=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+  - env: SHARED=1 DISABLE_KERNEL_MODULES=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+  - env: SHARED=1
+    compiler: clang
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+  - env: DISABLE_KERNEL_MODULES=1
+    compiler: clang
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+  - env: SHARED=1 DISABLE_KERNEL_MODULES=1
+    compiler: clang
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+  - env: ARM64=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), libtool,
+python3.5, python3-pip]
+  - env: ARM64=1 NINJABUILD=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [linux-headers-$(uname -r), libtool, python3.5,
+python3-pip, ninja-build]
+  - env: NINJABUILD=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+  - env: NINJABUILD=1 SHARED=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+  - env: NINJABUILD=1 DISABLE_KERNEL_MODULES=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+  - env: NINJABUILD=1 SHARED=1 DISABLE_KERNEL_MODULES=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5,
+python3-pip, ninja-build]
+
+
+script: ./.ci/${TRAVIS_OS_NAME}-build.sh
+
+notifications:
+  email:
+    recipients:
+      - [email protected]
MAINTAINERS +0 -0
index 835d8a201..eed6f69d3 100644
@@ -119,6 +119,13 @@ F: config/rte_config.h
 F: buildtools/gen-pmdinfo-cfile.sh
 F: buildtools/symlink-drivers-solibs.sh
 
Quoted 7 lines

MAINTAINERS F: config/rte_config.h · line 127

Quoted 6 lines
MAINTAINERS +6 -0
+Public CI
+M: Aaron Conole <[email protected]>
+M: Michael Santana <[email protected]>
+F: .travis.yml
+F: .ci/
+F: meson_cross_aarch64_gcc.txt

Not required

Oops. Okay we will fix it.

Quoted 27 lines
Quoted 27 lines
MAINTAINERS +1 -0
+
 ABI versioning
 M: Neil Horman <[email protected]>
 F: lib/librte_compat/
doc/guides/contributing/patches.rst +0 -0
index a64bb0368..49e930cbb 100644
@@ -32,6 +32,10 @@ The mailing list for DPDK development is

[email protected] <http://mails.dpdk.org/ar Contributors will need to register for the mailing list <http://mails.dpdk.org/listinfo/dev>`_ in order to submit patches.
It is also worth registering for the DPDK `Patchwork
<http://patches.dpdk.org/project/dpdk/list/>`_

doc/guides/contributing/patches.rst +4 -0
+If you are using the GitHub service, you can link your repository to
+the ``travis-ci.org`` build service.  When you push patches to your
+GitHub repository, the travis service will automatically build your changes.
+
 The development process requires some familiarity with the ``git`` version

control system.
Refer to the Pro Git Book <http://www.git-scm.com/book/>_ for further
information.

--
2.20.1

#23 lore raw

v4 cover Introduce travis support

In reply to Michael Santana

This series introduces the ability for any github mirrors of the DPDK project, including developer mirrors, to kick off builds under the travis CI infrastructure. For now, this just means compilation - no other kinds of automated run exists yet. In the future, this can be expanded to execute and report results for any test-suites that might exist.

The series includes support for both the 'classic make' style builds (which are set to be deprecated at some undetermined point in the future), as well as the modern meson+ninja build system. Additionally, there is support for building the ARM64 target with both the meson and make systems.

The files added under .ci/ exist so that in the future, other CI support platforms (such as cirrus, appveyor, etc.) could have a common place to put their requisite scripts without polluting the main tree.

Some documentation is updated, making developers aware of the new travis integration.

The integration can also be included on the official DPDK mirror. Build reports can be enabled by subscribing the travis build email to the test-reports mailing list (this can be done independent of this series being applied).

v3->v4:
- Remove non-existing file form maintainers list:
meson_cross_aarch64_gcc.txt
- Renamed ARM64 to AARCH64 for travis environment variable

v2->v3:
- Removed duplicate file meson_cross_aarch64_gcc.txt. Used arm64_armv8_linuxapp_gcc file instead
- Renamed ambiguous variable names and comments, including the variable KERNEL to DISABLE_KERNEL_MODULES and comment 'source for python' to 'Repo for python'
- Removed an already-defined variable

v1 -> v2:
- Added patch 1/2, "examples/vhost_scsi: don't build..."
- Included arm64 builds
- Included multiple meson+ninja builds (full library, minimal library)
- Included multiple 'classic make' builds

Aaron Conole (1):
examples/vhost_scsi: Don't build without virtio_scsi.h

Michael Santana (1):
ci: Introduce travis builds for github repositories

 .ci/linux-build.sh                  |  88 +++++++++++++++
 .ci/linux-setup.sh                  |  31 ++++++
 .travis.yml                         | 159 ++++++++++++++++++++++++++++
 MAINTAINERS                         |   6 ++
 doc/guides/contributing/patches.rst |   4 +
 examples/vhost_scsi/meson.build     |   5 +
 6 files changed, 293 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
Signature · 1 line
2.20.1
#24 lore raw

PATCH v4 1/2 examples/vhost_scsi: Don't build without virtio_scsi.h

From: Aaron Conole <[email protected]>

The vhost_scsi example code is set to build, even if the requisite header file virtio_scsi.h isn't available. This happens on some Ubuntu systems when some versions of the libc-dev package aren't available.

Check whether the virtio_scsi.h file exists, and if not, set the build flag to false.

---
v4:
Nothing changed in this patch since v3.

 examples/vhost_scsi/meson.build | 5 +++++
 1 file changed, 5 insertions(+)
examples/vhost_scsi/meson.build +5 -0
index 2303bcaed..ca1248023 100644
@@ -9,6 +9,11 @@
 if host_machine.system() != 'linux'
 	build = false
 endif
+
+if not cc.has_header('virtio_scsi.h')
+	build = false
+endif
+
 deps += 'vhost'
 cflags += ['-D_FILE_OFFSET_BITS=64']
 sources = files(
Signature · 1 line
2.20.1
#25 lore raw

PATCH v4 2/2 ci: Introduce travis builds for github repositories

In reply to Michael Santana

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Additionally, the travis service will send an email to the test-report list informing anyone interested in the automated build (including a result).

---
v4:
- Remove non-existing file form maintainers list:
meson_cross_aarch64_gcc.txt
- Renamed ARCH64 to AARM64 for travis environment variable

 .ci/linux-build.sh                  |  88 +++++++++++++++
 .ci/linux-setup.sh                  |  31 ++++++
 .travis.yml                         | 159 ++++++++++++++++++++++++++++
 MAINTAINERS                         |   6 ++
 doc/guides/contributing/patches.rst |   4 +
 5 files changed, 288 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
.ci/linux-build.sh new file +88 -0
new file mode 100755
index 000000000..9c6af2ead
@@ -0,0 +1,88 @@
+#!/bin/bash
+
+# check for whether we're clang or gcc
+# setup the right options depending on the environment variables
+# run the build
+
+# Just used for the 'classic' configuration system (ie: make)
+set_conf() {
+    echo "[BUILT WITH $2 SET TO $3]"
+    c="$1/.config"
+    shift
+
+    if grep -q "$1" "$c"; then
+        sed -i "s:^$1=.*$:$1=$2:g" $c
+    else
+        echo $1=$2 >> "$c"
+    fi
+}
+
+BUILD_ARCH="x86_64-native-linuxapp-"
+
+if [ "${AARCH64}" == "1" ]; then
+    # convert the arch specifier
+    BUILD_ARCH="arm64-armv8a-linuxapp-"
+    ARM64_TOOL="linaro-arm-tool"
+    export PATH=$PATH:$(pwd)/${ARM64_TOOL}/bin
+fi
+
+
+if [ "${NINJABUILD}" == "1" ]; then
+    OPTS=""
+
+    DEF_LIB="static"
+    if [ "${SHARED}" == "1" ]; then
+        DEF_LIB="shared"
+    fi
+
+    if [ "${DISABLE_KERNEL_MODULES}" == "1" ]; then
+        OPTS="-Denable_kmods=false"
+    fi
+
+    if [ "${AARCH64}" == "1" ]; then
+        OPTS="${OPTS} --cross-file config/arm/arm64_armv8_linuxapp_gcc"
+    fi
+
+    OPTS="$OPTS --default-library=$DEF_LIB"
+    meson build --werror -Dexamples=all ${OPTS}
+    ninja -C build
+else
+    EXTRA_OPTS=""
+
+    make config T="${BUILD_ARCH}${CC}"
+
+    set_conf build CONFIG_RTE_KNI_KMOD n
+    set_conf build CONFIG_RTE_EAL_IGB_UIO n
+
+    if dpkg --list | grep -q zlib1g ; then
+        set_conf build CONFIG_RTE_LIBRTE_PMD_ZLIB y
+    fi
+
+    if dpkg --list | grep -q libpcap-dev ; then
+        set_conf build CONFIG_RTE_PORT_PCAP y
+    fi
+
+    if [ "${SHARED}" == "1" ]; then
+        set_conf build CONFIG_RTE_BUILD_SHARED_LIB y
+    fi
+
+    if [ "${DISABLE_KERNEL_MODULES}" == "1" ]; then
+        echo Unsupported kernel builds at the moment
+    fi
+
+    if [ "${AARCH64}" == "1" ]; then
+        EXTRA_OPTS="CROSS=aarch64-linux-gnu-"
+
+        # need to turn off these extras
+        set_conf build CONFIG_RTE_PORT_PCAP n
+        set_conf build CONFIG_RTE_LIBRTE_PMD_ZLIB n
+
+        # convert the CC/CXX variables
+        export CC=aarch64-linux-gnu-${CC}
+        export CXX=aarch64-linux-gnu-${CXX}
+        export AR=aarch64-linux-gnu-ar
+        export STRIP=aarch64-linux-gnu-strip
+    fi
+
+    make all ${EXTRA_OPTS}
+fi
.ci/linux-setup.sh new file +31 -0
new file mode 100755
index 000000000..15183165b
@@ -0,0 +1,31 @@
+#!/bin/bash
+
+python3.5 -m pip install --upgrade meson --user
+
+echo "AARCH64 is [ ${AARCH64} ]"
+
+if [ "${AARCH64}" == "1" ]; then
+    # need to build & install libnuma
+    # This will only be minimal support for now.
+    ARM64_TOOL_URL='https://releases.linaro.org/components/toolchain/binaries/latest-7/aarch64-linux-gnu/gcc-linaro-7.4.1-2019.02-x86_64_aarch64-linux-gnu.tar.xz'
+    ARM64_TOOL="linaro-arm-tool"
+    NUMA_GIT_URL="https://github.com/numactl/numactl.git"
+
+    wget -O "${ARM64_TOOL}.tar.xz" "${ARM64_TOOL_URL}"
+    tar -xf "${ARM64_TOOL}.tar.xz"
+    mv gcc-linaro* "${ARM64_TOOL}"
+    export PATH=$PATH:$(pwd)/${ARM64_TOOL}/bin
+    git clone "${NUMA_GIT_URL}"
+    cd numactl
+    git checkout v2.0.11
+    ./autogen.sh
+    autoconf -i
+    mkdir numa_bin
+    ./configure --host=aarch64-linux-gnu CC=aarch64-linux-gnu-gcc \
+                --prefix=$(pwd)/numa_bin
+    make install # install numa
+    cd ..
+    cp numactl/numa_bin/include/numa*.h "${ARM64_TOOL}/aarch64-linux-gnu/libc/usr/include/"
+    cp numactl/numa_bin/lib/libnuma.* "${ARM64_TOOL}/aarch64-linux-gnu/lib64/"
+    cp numactl/numa_bin/lib/libnuma.* "${ARM64_TOOL}/lib/"
+fi
.travis.yml new file +159 -0

159 lines not on this page · raw message

MAINTAINERS +6 -0
index 835d8a201..3f9de1151 100644
@@ -119,6 +119,12 @@ F: config/rte_config.h
 F: buildtools/gen-pmdinfo-cfile.sh
 F: buildtools/symlink-drivers-solibs.sh
 
+Public CI
+M: Aaron Conole <[email protected]>
+M: Michael Santana <[email protected]>
+F: .travis.yml
+F: .ci/
+
 ABI versioning
 M: Neil Horman <[email protected]>
 F: lib/librte_compat/
doc/guides/contributing/patches.rst +4 -0
index a64bb0368..49e930cbb 100644
@@ -32,6 +32,10 @@ The mailing list for DPDK development is `[email protected] <http://mails.dpdk.org/ar
 Contributors will need to `register for the mailing list <http://mails.dpdk.org/listinfo/dev>`_ in order to submit patches.
 It is also worth registering for the DPDK `Patchwork <http://patches.dpdk.org/project/dpdk/list/>`_
 
+If you are using the GitHub service, you can link your repository to
+the ``travis-ci.org`` build service.  When you push patches to your GitHub
+repository, the travis service will automatically build your changes.
+
 The development process requires some familiarity with the ``git`` version control system.
 Refer to the `Pro Git Book <http://www.git-scm.com/book/>`_ for further information.
 
Signature · 1 line
2.20.1
Reviewed-by on v4 2/2
#26 lore raw

Re: [PATCH v4 2/2] ci: Introduce travis builds for github repositories

<snip>

Quoted 20 lines
.ci/linux-build.sh new file mode 100755 index renamed from .ci/linux-build.sh +0 -0

000000000..9c6af2ead

.ci/linux-build.sh new file +15 -0
@@ -0,0 +1,88 @@
+#!/bin/bash
+
+# check for whether we're clang or gcc
+# setup the right options depending on the environment variables # run
+the build
+
+# Just used for the 'classic' configuration system (ie: make)
+set_conf() {
+    echo "[BUILT WITH $2 SET TO $3]"
+    c="$1/.config"
+    shift
+
+    if grep -q "$1" "$c"; then
+        sed -i "s:^$1=.*$:$1=$2:g" $c
+    else

.ci/linux-build.sh line 25

Quoted 10 lines
.ci/linux-build.sh new file +10 -0
+        echo $1=$2 >> "$c"
+    fi
+}
+
+BUILD_ARCH="x86_64-native-linuxapp-"
+
+if [ "${AARCH64}" == "1" ]; then
+    # convert the arch specifier
+    BUILD_ARCH="arm64-armv8a-linuxapp-"
+    ARM64_TOOL="linaro-arm-tool"

My apologies, I think I was not clear on my comments earlier. ARM64_TOOL should be changed to AARCH64_TOOL

.ci/linux-build.sh line 26

Quoted 1 line
.ci/linux-build.sh new file +1 -0
+    export PATH=$PATH:$(pwd)/${ARM64_TOOL}/bin

Change to AARCH64_TOOL

Quoted 10 lines
.ci/linux-build.sh new file +10 -0
+fi
+
+
+if [ "${NINJABUILD}" == "1" ]; then
+    OPTS=""
+
+    DEF_LIB="static"
+    if [ "${SHARED}" == "1" ]; then
+        DEF_LIB="shared"
+    fi

<snip>

Quoted 15 lines
.ci/linux-setup.sh new file mode 100755 index renamed from .ci/linux-setup.sh +0 -0

000000000..15183165b

.ci/linux-setup.sh new file +10 -0
@@ -0,0 +1,31 @@
+#!/bin/bash
+
+python3.5 -m pip install --upgrade meson --user
+
+echo "AARCH64 is [ ${AARCH64} ]"
+
+if [ "${AARCH64}" == "1" ]; then
+    # need to build & install libnuma
+    # This will only be minimal support for now.
+
Quoted 1 line

ARM64_TOOL_URL='https://releases.linaro.org/components/toolchain/binary

Change to AARCH64_TOOL_URL

Quoted 3 lines

es/latest-7/aarch64-linux-gnu/gcc-linaro-7.4.1-2019.02-x86_64_aarch64- linux-gnu.tar.xz'
+ ARM64_TOOL="linaro-arm-tool"

Change to AARCH64_TOOL

.ci/linux-setup.sh line 17

Quoted 6 lines
.ci/linux-setup.sh new file +6 -0
+    NUMA_GIT_URL="https://github.com/numactl/numactl.git"
+
+    wget -O "${ARM64_TOOL}.tar.xz" "${ARM64_TOOL_URL}"
+    tar -xf "${ARM64_TOOL}.tar.xz"
+    mv gcc-linaro* "${ARM64_TOOL}"
+    export PATH=$PATH:$(pwd)/${ARM64_TOOL}/bin

Change to AARCH64_TOOL and AARCH64_TOOL_URL

Quoted 11 lines
.ci/linux-setup.sh new file +11 -0
+    git clone "${NUMA_GIT_URL}"
+    cd numactl
+    git checkout v2.0.11
+    ./autogen.sh
+    autoconf -i
+    mkdir numa_bin
+    ./configure --host=aarch64-linux-gnu CC=aarch64-linux-gnu-gcc \
+                --prefix=$(pwd)/numa_bin
+    make install # install numa
+    cd ..
+    cp numactl/numa_bin/include/numa*.h "${ARM64_TOOL}/aarch64-linux-
Quoted 4 lines

gnu/libc/usr/include/"
+ cp numactl/numa_bin/lib/libnuma.* "${ARM64_TOOL}/aarch64-linux-
gnu/lib64/"
+ cp numactl/numa_bin/lib/libnuma.* "${ARM64_TOOL}/lib/"

Change to AARCH64_TOOL

Otherwise,

#27 lore raw

v5 cover Introduce travis support

In reply to Michael Santana

This series introduces the ability for any github mirrors of the DPDK project, including developer mirrors, to kick off builds under the travis CI infrastructure. For now, this just means compilation - no other kinds of automated run exists yet. In the future, this can be expanded to execute and report results for any test-suites that might exist.

The series includes support for both the 'classic make' style builds (which are set to be deprecated at some undetermined point in the future), as well as the modern meson+ninja build system. Additionally, there is support for building the ARM64 target with both the meson and make systems.

The files added under .ci/ exist so that in the future, other CI support platforms (such as cirrus, appveyor, etc.) could have a common place to put their requisite scripts without polluting the main tree.

Some documentation is updated, making developers aware of the new travis integration.

The integration can also be included on the official DPDK mirror. Build reports can be enabled by subscribing the travis build email to the test-reports mailing list (this can be done independent of this series being applied).

v4->v5:
- Renamed ARM64 to AARCH64.

v3->v4:
- Remove non-existing file form maintainers list:
meson_cross_aarch64_gcc.txt
- Renamed ARM64 to AARCH64 for travis environment variable

v2->v3:
- Removed duplicate file meson_cross_aarch64_gcc.txt. Used arm64_armv8_linuxapp_gcc file instead
- Renamed ambiguous variable names and comments, including the variable KERNEL to DISABLE_KERNEL_MODULES and comment 'source for python' to 'Repo for python'
- Removed an already-defined variable

v1 -> v2:
- Added patch 1/2, "examples/vhost_scsi: don't build..."
- Included arm64 builds
- Included multiple meson+ninja builds (full library, minimal library)
- Included multiple 'classic make' builds

Aaron Conole (1):
examples/vhost_scsi: Don't build without virtio_scsi.h

Michael Santana (1):
ci: Introduce travis builds for github repositories

 .ci/linux-build.sh                  |  88 +++++++++++++++
 .ci/linux-setup.sh                  |  31 ++++++
 .travis.yml                         | 159 ++++++++++++++++++++++++++++
 MAINTAINERS                         |   6 ++
 doc/guides/contributing/patches.rst |   4 +
 examples/vhost_scsi/meson.build     |   5 +
 6 files changed, 293 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
Signature · 1 line
2.20.1
#28 lore raw

PATCH v5 1/2 examples/vhost_scsi: Don't build without virtio_scsi.h

From: Aaron Conole <[email protected]>

The vhost_scsi example code is set to build, even if the requisite header file virtio_scsi.h isn't available. This happens on some Ubuntu systems when some versions of the libc-dev package aren't available.

Check whether the virtio_scsi.h file exists, and if not, set the build flag to false.

---
v5:
Nothing changed in this patch since v3.

 examples/vhost_scsi/meson.build | 5 +++++
 1 file changed, 5 insertions(+)
examples/vhost_scsi/meson.build +5 -0
index 2303bcaed..ca1248023 100644
@@ -9,6 +9,11 @@
 if host_machine.system() != 'linux'
 	build = false
 endif
+
+if not cc.has_header('virtio_scsi.h')
+	build = false
+endif
+
 deps += 'vhost'
 cflags += ['-D_FILE_OFFSET_BITS=64']
 sources = files(
Signature · 1 line
2.20.1
#29 lore raw

Re: [PATCH v5 1/2] examples/vhost_scsi: Don't build without virtio_scsi.h

07/02/2019 23:01, Michael Santana:

Quoted 6 lines

From: Aaron Conole <[email protected]>

The vhost_scsi example code is set to build, even if the requisite header file virtio_scsi.h isn't available. This happens on some Ubuntu systems when some versions of the libc-dev package aren't available.

Quoted 8 lines

Check whether the virtio_scsi.h file exists, and if not, set the build flag to false.

---
v5:
Nothing changed in this patch since v3.

Patch 1 applied alone to allow Travis refinement separately, thanks.

#30 lore raw

PATCH v5 2/2 ci: Introduce travis builds for github repositories

In reply to Michael Santana

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Additionally, the travis service will send an email to the test-report list informing anyone interested in the automated build (including a result).

---
v5:
Renamed ARM64 to AARCH64

 .ci/linux-build.sh                  |  88 +++++++++++++++
 .ci/linux-setup.sh                  |  31 ++++++
 .travis.yml                         | 159 ++++++++++++++++++++++++++++
 MAINTAINERS                         |   6 ++
 doc/guides/contributing/patches.rst |   4 +
 5 files changed, 288 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
.ci/linux-build.sh new file +88 -0
new file mode 100755
index 000000000..3535a905f
@@ -0,0 +1,88 @@
+#!/bin/bash
+
+# check for whether we're clang or gcc
+# setup the right options depending on the environment variables
+# run the build
+
+# Just used for the 'classic' configuration system (ie: make)
+set_conf() {
+    echo "[BUILT WITH $2 SET TO $3]"
+    c="$1/.config"
+    shift
+
+    if grep -q "$1" "$c"; then
+        sed -i "s:^$1=.*$:$1=$2:g" $c
+    else
+        echo $1=$2 >> "$c"
+    fi
+}
+
+BUILD_ARCH="x86_64-native-linuxapp-"
+
+if [ "${AARCH64}" == "1" ]; then
+    # convert the arch specifier
+    BUILD_ARCH="arm64-armv8a-linuxapp-"
+    AARCH64_TOOL="linaro-arm-tool"
+    export PATH=$PATH:$(pwd)/${AARCH64_TOOL}/bin
+fi
+
+
+if [ "${NINJABUILD}" == "1" ]; then
+    OPTS=""
+
+    DEF_LIB="static"
+    if [ "${SHARED}" == "1" ]; then
+        DEF_LIB="shared"
+    fi
+
+    if [ "${DISABLE_KERNEL_MODULES}" == "1" ]; then
+        OPTS="-Denable_kmods=false"
+    fi
+
+    if [ "${AARCH64}" == "1" ]; then
+        OPTS="${OPTS} --cross-file config/arm/arm64_armv8_linuxapp_gcc"
+    fi
+
+    OPTS="$OPTS --default-library=$DEF_LIB"
+    meson build --werror -Dexamples=all ${OPTS}
+    ninja -C build
+else
+    EXTRA_OPTS=""
+
+    make config T="${BUILD_ARCH}${CC}"
+
+    set_conf build CONFIG_RTE_KNI_KMOD n
+    set_conf build CONFIG_RTE_EAL_IGB_UIO n
+
+    if dpkg --list | grep -q zlib1g ; then
+        set_conf build CONFIG_RTE_LIBRTE_PMD_ZLIB y
+    fi
+
+    if dpkg --list | grep -q libpcap-dev ; then
+        set_conf build CONFIG_RTE_PORT_PCAP y
+    fi
+
+    if [ "${SHARED}" == "1" ]; then
+        set_conf build CONFIG_RTE_BUILD_SHARED_LIB y
+    fi
+
+    if [ "${DISABLE_KERNEL_MODULES}" == "1" ]; then
+        echo Unsupported kernel builds at the moment
+    fi
+
+    if [ "${AARCH64}" == "1" ]; then
+        EXTRA_OPTS="CROSS=aarch64-linux-gnu-"
+
+        # need to turn off these extras
+        set_conf build CONFIG_RTE_PORT_PCAP n
+        set_conf build CONFIG_RTE_LIBRTE_PMD_ZLIB n
+
+        # convert the CC/CXX variables
+        export CC=aarch64-linux-gnu-${CC}
+        export CXX=aarch64-linux-gnu-${CXX}
+        export AR=aarch64-linux-gnu-ar
+        export STRIP=aarch64-linux-gnu-strip
+    fi
+
+    make all ${EXTRA_OPTS}
+fi
.ci/linux-setup.sh new file +31 -0
new file mode 100755
index 000000000..8dfd5aa49
@@ -0,0 +1,31 @@
+#!/bin/bash
+
+python3.5 -m pip install --upgrade meson --user
+
+echo "AARCH64 is [ ${AARCH64} ]"
+
+if [ "${AARCH64}" == "1" ]; then
+    # need to build & install libnuma
+    # This will only be minimal support for now.
+    AARCH64_TOOL_URL='https://releases.linaro.org/components/toolchain/binaries/latest-7/aarch64-linux-gnu/gcc-linaro-7.4.1-2019.02-x86_64_aarch64-linux-gnu.tar.xz'
+    AARCH64_TOOL="linaro-arm-tool"
+    NUMA_GIT_URL="https://github.com/numactl/numactl.git"
+
+    wget -O "${AARCH64_TOOL}.tar.xz" "${AARCH64_TOOL_URL}"
+    tar -xf "${AARCH64_TOOL}.tar.xz"
+    mv gcc-linaro* "${AARCH64_TOOL}"
+    export PATH=$PATH:$(pwd)/${AARCH64_TOOL}/bin
+    git clone "${NUMA_GIT_URL}"
+    cd numactl
+    git checkout v2.0.11
+    ./autogen.sh
+    autoconf -i
+    mkdir numa_bin
+    ./configure --host=aarch64-linux-gnu CC=aarch64-linux-gnu-gcc \
+                --prefix=$(pwd)/numa_bin
+    make install # install numa
+    cd ..
+    cp numactl/numa_bin/include/numa*.h "${AARCH64_TOOL}/aarch64-linux-gnu/libc/usr/include/"
+    cp numactl/numa_bin/lib/libnuma.* "${AARCH64_TOOL}/aarch64-linux-gnu/lib64/"
+    cp numactl/numa_bin/lib/libnuma.* "${AARCH64_TOOL}/lib/"
+fi
.travis.yml new file +159 -0

159 lines not on this page · raw message

MAINTAINERS +6 -0
index 835d8a201..3f9de1151 100644
@@ -119,6 +119,12 @@ F: config/rte_config.h
 F: buildtools/gen-pmdinfo-cfile.sh
 F: buildtools/symlink-drivers-solibs.sh
 
+Public CI
+M: Aaron Conole <[email protected]>
+M: Michael Santana <[email protected]>
+F: .travis.yml
+F: .ci/
+
 ABI versioning
 M: Neil Horman <[email protected]>
 F: lib/librte_compat/
doc/guides/contributing/patches.rst +4 -0
index a64bb0368..49e930cbb 100644
@@ -32,6 +32,10 @@ The mailing list for DPDK development is `[email protected] <http://mails.dpdk.org/ar
 Contributors will need to `register for the mailing list <http://mails.dpdk.org/listinfo/dev>`_ in order to submit patches.
 It is also worth registering for the DPDK `Patchwork <http://patches.dpdk.org/project/dpdk/list/>`_
 
+If you are using the GitHub service, you can link your repository to
+the ``travis-ci.org`` build service.  When you push patches to your GitHub
+repository, the travis service will automatically build your changes.
+
 The development process requires some familiarity with the ``git`` version control system.
 Refer to the `Pro Git Book <http://www.git-scm.com/book/>`_ for further information.
 
Signature · 1 line
2.20.1
#31 lore raw

Re: [PATCH v5 2/2] ci: Introduce travis builds for github repositories

Hi,

07/02/2019 23:01, Michael Santana:

Quoted 10 lines

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Quoted 8 lines

Additionally, the travis service will send an email to the test-report list informing anyone interested in the automated build (including a result).

I think it is a good idea to provide some recipes for some well known CI.

I have some concerns about the organization of what is generic/specific and about the automation of reports.
Detailed comments below:

.ci/linux-build.sh line 7

Quoted 3 lines
.ci/linux-build.sh new file +1 -0
+# Just used for the 'classic' configuration system (ie: make)

I am not sure about supporting the legacy system in a new CI.

.ci/linux-build.sh line 9

Quoted 2 lines
.ci/linux-build.sh new file +2 -0
+set_conf() {
+    echo "[BUILT WITH $2 SET TO $3]"

In order to improve readability, I suggest to document the parameters on the first line. Example:

set_conf() { # <dir> <key> <value>

.ci/linux-build.sh line 14

Quoted 5 lines
.ci/linux-build.sh new file +5 -0
+    c="$1/.config"
+    shift
+
+    if grep -q "$1" "$c"; then
+        sed -i "s:^$1=.*$:$1=$2:g" $c

You don't need the "g" as there is only one option per line.

.ci/linux-build.sh line 16

Quoted 2 lines
.ci/linux-build.sh new file +2 -0
+    else
+        echo $1=$2 >> "$c"

The else branch is useless because .config includes all options.

.ci/linux-build.sh line 20

Quoted 4 lines
.ci/linux-build.sh new file +4 -0
+    fi
+}
+
+BUILD_ARCH="x86_64-native-linuxapp-"

We could have some native Arm compilation.

.ci/linux-build.sh line 25

Quoted 5 lines
.ci/linux-build.sh new file +5 -0
+
+if [ "${AARCH64}" == "1" ]; then
+    # convert the arch specifier
+    BUILD_ARCH="arm64-armv8a-linuxapp-"
+    AARCH64_TOOL="linaro-arm-tool"

What is this directory? It looks really specific to Travis.

Quoted 5 lines
.ci/linux-build.sh new file +5 -0
+    export PATH=$PATH:$(pwd)/${AARCH64_TOOL}/bin
+fi
+
+
+if [ "${NINJABUILD}" == "1" ]; then

.ci/linux-build.sh line 40

Quoted 10 lines
.ci/linux-build.sh new file +10 -0
+    OPTS=""
+
+    DEF_LIB="static"
+    if [ "${SHARED}" == "1" ]; then
+        DEF_LIB="shared"
+    fi
+
+    if [ "${DISABLE_KERNEL_MODULES}" == "1" ]; then
+        OPTS="-Denable_kmods=false"
+    fi

Isn't it possible to directly provide the meson options in travis.yml instead of doing a translation with new option names?

Quoted 5 lines
.ci/linux-build.sh new file +5 -0
+
+    if [ "${AARCH64}" == "1" ]; then
+        OPTS="${OPTS} --cross-file config/arm/arm64_armv8_linuxapp_gcc"
+    fi
+

.ci/linux-build.sh line 55

Quoted 10 lines
.ci/linux-build.sh new file +10 -0
+    OPTS="$OPTS --default-library=$DEF_LIB"
+    meson build --werror -Dexamples=all ${OPTS}
+    ninja -C build
+else
+    EXTRA_OPTS=""
+
+    make config T="${BUILD_ARCH}${CC}"
+
+    set_conf build CONFIG_RTE_KNI_KMOD n
+    set_conf build CONFIG_RTE_EAL_IGB_UIO n

Why these options are fixed?

.ci/linux-build.sh line 59

Quoted 4 lines
.ci/linux-build.sh new file +4 -0
+
+    if dpkg --list | grep -q zlib1g ; then
+        set_conf build CONFIG_RTE_LIBRTE_PMD_ZLIB y
+    fi

This is Debian-specific.

Quoted 35 lines
.ci/linux-build.sh new file +29 -0
+
+    if dpkg --list | grep -q libpcap-dev ; then
+        set_conf build CONFIG_RTE_PORT_PCAP y
+    fi
+
+    if [ "${SHARED}" == "1" ]; then
+        set_conf build CONFIG_RTE_BUILD_SHARED_LIB y
+    fi
+
+    if [ "${DISABLE_KERNEL_MODULES}" == "1" ]; then
+        echo Unsupported kernel builds at the moment
+    fi
+
+    if [ "${AARCH64}" == "1" ]; then
+        EXTRA_OPTS="CROSS=aarch64-linux-gnu-"
+
+        # need to turn off these extras
+        set_conf build CONFIG_RTE_PORT_PCAP n
+        set_conf build CONFIG_RTE_LIBRTE_PMD_ZLIB n
+
+        # convert the CC/CXX variables
+        export CC=aarch64-linux-gnu-${CC}
+        export CXX=aarch64-linux-gnu-${CXX}
+        export AR=aarch64-linux-gnu-ar
+        export STRIP=aarch64-linux-gnu-strip
+    fi
+
+    make all ${EXTRA_OPTS}
+fi
.ci/linux-setup.sh new file +0 -0
new file mode 100755
index 000000000..8dfd5aa49

.ci/linux-setup.sh line 3

Quoted 3 lines
.ci/linux-setup.sh new file +3 -0
@@ -0,0 +1,31 @@
+#!/bin/bash
+
+python3.5 -m pip install --upgrade meson --user

Which distributions have python3.5?
It looks very specific.

.ci/linux-setup.sh line 8

Quoted 5 lines
.ci/linux-setup.sh new file +5 -0
+
+echo "AARCH64 is [ ${AARCH64} ]"
+
+if [ "${AARCH64}" == "1" ]; then
+    # need to build & install libnuma

Why is it needed? linbnuma is optional.
I think this file can be dropped or renamed to
something like install-libnuma-for-cross-arm.sh

Quoted 178 lines
.ci/linux-setup.sh new file +23 -0
+    # This will only be minimal support for now.
+    AARCH64_TOOL_URL='https://releases.linaro.org/components/toolchain/binaries/latest-7/aarch64-linux-gnu/gcc-linaro-7.4.1-2019.02-x86_64_aarch64-linux-gnu.tar.xz'
+    AARCH64_TOOL="linaro-arm-tool"
+    NUMA_GIT_URL="https://github.com/numactl/numactl.git"
+
+    wget -O "${AARCH64_TOOL}.tar.xz" "${AARCH64_TOOL_URL}"
+    tar -xf "${AARCH64_TOOL}.tar.xz"
+    mv gcc-linaro* "${AARCH64_TOOL}"
+    export PATH=$PATH:$(pwd)/${AARCH64_TOOL}/bin
+    git clone "${NUMA_GIT_URL}"
+    cd numactl
+    git checkout v2.0.11
+    ./autogen.sh
+    autoconf -i
+    mkdir numa_bin
+    ./configure --host=aarch64-linux-gnu CC=aarch64-linux-gnu-gcc \
+                --prefix=$(pwd)/numa_bin
+    make install # install numa
+    cd ..
+    cp numactl/numa_bin/include/numa*.h "${AARCH64_TOOL}/aarch64-linux-gnu/libc/usr/include/"
+    cp numactl/numa_bin/lib/libnuma.* "${AARCH64_TOOL}/aarch64-linux-gnu/lib64/"
+    cp numactl/numa_bin/lib/libnuma.* "${AARCH64_TOOL}/lib/"
+fi
.travis.yml new file +149 -0
new file mode 100644
index 000000000..0faa35770
@@ -0,0 +1,159 @@
+language: c
+compiler:
+  - gcc
+  - clang
+
+os:
+  - linux
+
+addons:
+  apt:
+    sources:
+      - deadsnakes  #Repo for python 3.5
+      - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+    packages:
+      - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip, ninja-build]
+
+before_install: ./.ci/${TRAVIS_OS_NAME}-setup.sh
+
+sudo: false
+
+env:
+  - SHARED=1
+  - DISABLE_KERNEL_MODULES=1
+  - SHARED=1 DISABLE_KERNEL_MODULES=1
+  - NINJABUILD=1
+  - NINJABUILD=1 SHARED=1
+  - NINJABUILD=1 DISABLE_KERNEL_MODULES=1
+  - NINJABUILD=1 SHARED=1 DISABLE_KERNEL_MODULES=1
+
+matrix:
+  include:
+  - env: SHARED=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip, ninja-build]
+  - env: DISABLE_KERNEL_MODULES=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip, ninja-build]
+  - env: SHARED=1 DISABLE_KERNEL_MODULES=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip, ninja-build]
+  - env: SHARED=1
+    compiler: clang
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip, ninja-build]
+  - env: DISABLE_KERNEL_MODULES=1
+    compiler: clang
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip, ninja-build]
+  - env: SHARED=1 DISABLE_KERNEL_MODULES=1
+    compiler: clang
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip, ninja-build]
+  - env: AARCH64=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), libtool, python3.5, python3-pip]
+  - env: AARCH64=1 NINJABUILD=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [linux-headers-$(uname -r), libtool, python3.5, python3-pip, ninja-build]
+  - env: NINJABUILD=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip, ninja-build]
+  - env: NINJABUILD=1 SHARED=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip, ninja-build]
+  - env: NINJABUILD=1 DISABLE_KERNEL_MODULES=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip, ninja-build]
+  - env: NINJABUILD=1 SHARED=1 DISABLE_KERNEL_MODULES=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:

.travis.yml line 159

Quoted 10 lines
.travis.yml new file +10 -0
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip, ninja-build]
+
+
+script: ./.ci/${TRAVIS_OS_NAME}-build.sh
+
+notifications:
+  email:
+    recipients:
+      - [email protected]

The idea of this mailing list is to receive reports about the upstream development. When doing a private development, reports should not be sent. How can it be disabled?

#32 lore raw

Re: [PATCH v5 2/2] ci: Introduce travis builds for github repositories

Quoted 28 lines · Thomas Monjalon

Hi,

07/02/2019 23:01, Michael Santana:

Quoted 10 lines

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Quoted 8 lines

Additionally, the travis service will send an email to the test-report list informing anyone interested in the automated build (including a result).

I think it is a good idea to provide some recipes for some well known CI.

I have some concerns about the organization of what is generic/specific and about the automation of reports.
Detailed comments below:

Quoted 5 lines

.ci/linux-build.sh line 7

Quoted 3 lines
.ci/linux-build.sh new file +1 -0
+# Just used for the 'classic' configuration system (ie: make)

I am not sure about supporting the legacy system in a new CI.

For now, documentation all says that the legacy system is the supported system. I think it's appropriate to continue to support it until such time as it is eliminated. Otherwise, when an end user builds we don't know what to say about support - IE: if they have problems with the classic system for whatever reason, do we tell them "sorry, we cannot help"?

The patches.rst mentions that all patches must pass with the Makefile system, and the contributing/documentation.rst calls it the "standard DPDK build system." If you want to change those things to reflect something different please do, and we can drop all of the stuff related to it, but until that time we won't.

Quoted 14 lines

.ci/linux-build.sh line 9

Quoted 2 lines
.ci/linux-build.sh new file +2 -0
+set_conf() {
+    echo "[BUILT WITH $2 SET TO $3]"

In order to improve readability, I suggest to document the parameters on the first line. Example:

set_conf() { # <dir> <key> <value>

.ci/linux-build.sh line 14

Quoted 5 lines
.ci/linux-build.sh new file +5 -0
+    c="$1/.config"
+    shift
+
+    if grep -q "$1" "$c"; then
+        sed -i "s:^$1=.*$:$1=$2:g" $c
Quoted 6 lines

You don't need the "g" as there is only one option per line.

.ci/linux-build.sh line 16

Quoted 2 lines
.ci/linux-build.sh new file +2 -0
+    else
+        echo $1=$2 >> "$c"

The else branch is useless because .config includes all options.

Okay.

Quoted 6 lines

.ci/linux-build.sh line 20

Quoted 4 lines
.ci/linux-build.sh new file +4 -0
+    fi
+}
+
+BUILD_ARCH="x86_64-native-linuxapp-"

We could have some native Arm compilation.

Sure. Is this just commentary? Do you suggest a change here? This is a default, and will be adjusted later by other parameters.

Quoted 7 lines

.ci/linux-build.sh line 25

Quoted 5 lines
.ci/linux-build.sh new file +5 -0
+
+if [ "${AARCH64}" == "1" ]; then
+    # convert the arch specifier
+    BUILD_ARCH="arm64-armv8a-linuxapp-"
+    AARCH64_TOOL="linaro-arm-tool"

What is this directory? It looks really specific to Travis.

It's specific to the AARCH64 toolchain that was pulled in as part of linux-prepare.sh - do you think something should change?

Quoted 10 lines
Quoted 10 lines
.ci/linux-build.sh new file +10 -0
+    export PATH=$PATH:$(pwd)/${AARCH64_TOOL}/bin
+fi
+
+
+if [ "${NINJABUILD}" == "1" ]; then
+    OPTS=""
+
+    DEF_LIB="static"
+    if [ "${SHARED}" == "1" ]; then
+        DEF_LIB="shared"
Quoted 8 lines

.ci/linux-build.sh line 40

Quoted 5 lines
.ci/linux-build.sh new file +5 -0
+    fi
+
+    if [ "${DISABLE_KERNEL_MODULES}" == "1" ]; then
+        OPTS="-Denable_kmods=false"
+    fi

Isn't it possible to directly provide the meson options in travis.yml instead of doing a translation with new option names?

Yes and no. It would be possible, but we try to support both build systems and they have different options afaict. So we need something common. Maybe we missed something?

Quoted 9 lines
Quoted 9 lines
.ci/linux-build.sh new file +9 -0
+
+    if [ "${AARCH64}" == "1" ]; then
+        OPTS="${OPTS} --cross-file config/arm/arm64_armv8_linuxapp_gcc"
+    fi
+
+    OPTS="$OPTS --default-library=$DEF_LIB"
+    meson build --werror -Dexamples=all ${OPTS}
+    ninja -C build
+else
Quoted 8 lines

.ci/linux-build.sh line 55

Quoted 6 lines
.ci/linux-build.sh new file +6 -0
+    EXTRA_OPTS=""
+
+    make config T="${BUILD_ARCH}${CC}"
+
+    set_conf build CONFIG_RTE_KNI_KMOD n
+    set_conf build CONFIG_RTE_EAL_IGB_UIO n

Why these options are fixed?

There was a problem with the Travis system when trying to build some of the kernel modules, but I don't remember the details. Maybe Michael does.

Quoted 6 lines

.ci/linux-build.sh line 59

Quoted 4 lines
.ci/linux-build.sh new file +4 -0
+
+    if dpkg --list | grep -q zlib1g ; then
+        set_conf build CONFIG_RTE_LIBRTE_PMD_ZLIB y
+    fi

This is Debian-specific.

Will address this in v6.

Quoted 33 lines
Quoted 33 lines
.ci/linux-build.sh new file +29 -0
+
+    if dpkg --list | grep -q libpcap-dev ; then
+        set_conf build CONFIG_RTE_PORT_PCAP y
+    fi
+
+    if [ "${SHARED}" == "1" ]; then
+        set_conf build CONFIG_RTE_BUILD_SHARED_LIB y
+    fi
+
+    if [ "${DISABLE_KERNEL_MODULES}" == "1" ]; then
+        echo Unsupported kernel builds at the moment
+    fi
+
+    if [ "${AARCH64}" == "1" ]; then
+        EXTRA_OPTS="CROSS=aarch64-linux-gnu-"
+
+        # need to turn off these extras
+        set_conf build CONFIG_RTE_PORT_PCAP n
+        set_conf build CONFIG_RTE_LIBRTE_PMD_ZLIB n
+
+        # convert the CC/CXX variables
+        export CC=aarch64-linux-gnu-${CC}
+        export CXX=aarch64-linux-gnu-${CXX}
+        export AR=aarch64-linux-gnu-ar
+        export STRIP=aarch64-linux-gnu-strip
+    fi
+
+    make all ${EXTRA_OPTS}
+fi
.ci/linux-setup.sh new file +0 -1
new file mode 100755
index 000000000..8dfd5aa49
--- /dev/null
Quoted 8 lines

.ci/linux-setup.sh line 3

Quoted 5 lines

+++ b/.ci/linux-setup.sh

.ci/linux-setup.sh new file +3 -0
@@ -0,0 +1,31 @@
+#!/bin/bash
+
+python3.5 -m pip install --upgrade meson --user

Which distributions have python3.5?
It looks very specific.

I agree, could probably just be python3 we need to check and see if we can just use that.

But, we did need the upgrade.

Travis environment comes up with ubuntu 14.04, which includes python3.4, and the requisite version of meson needs python3.5 or higher.

Quoted 9 lines

.ci/linux-setup.sh line 8

Quoted 5 lines
.ci/linux-setup.sh new file +5 -0
+
+echo "AARCH64 is [ ${AARCH64} ]"
+
+if [ "${AARCH64}" == "1" ]; then
+    # need to build & install libnuma

Why is it needed? linbnuma is optional.
I think this file can be dropped or renamed to
something like install-libnuma-for-cross-arm.sh

We needed this because it broke the meson build when cross compiling, IIRC. I will investigate it further to be sure.

Quoted 196 lines
Quoted 196 lines
.ci/linux-setup.sh new file +2 -0
+    # This will only be minimal support for now.
+

AARCH64_TOOL_URL='https://releases.linaro.org/components/toolchain/binaries/latest-7/aarch64-linux-gnu/gcc-linaro-7.4.1-2019.02-x86_64_aarch64-linux-gnu.tar.xz'

.ci/linux-setup.sh new file +18 -0
+    AARCH64_TOOL="linaro-arm-tool"
+    NUMA_GIT_URL="https://github.com/numactl/numactl.git"
+
+    wget -O "${AARCH64_TOOL}.tar.xz" "${AARCH64_TOOL_URL}"
+    tar -xf "${AARCH64_TOOL}.tar.xz"
+    mv gcc-linaro* "${AARCH64_TOOL}"
+    export PATH=$PATH:$(pwd)/${AARCH64_TOOL}/bin
+    git clone "${NUMA_GIT_URL}"
+    cd numactl
+    git checkout v2.0.11
+    ./autogen.sh
+    autoconf -i
+    mkdir numa_bin
+    ./configure --host=aarch64-linux-gnu CC=aarch64-linux-gnu-gcc \
+                --prefix=$(pwd)/numa_bin
+    make install # install numa
+    cd ..
+ cp numactl/numa_bin/include/numa*.h

"${AARCH64_TOOL}/aarch64-linux-gnu/libc/usr/include/"

.ci/linux-setup.sh new file +3 -0
+    cp numactl/numa_bin/lib/libnuma.* "${AARCH64_TOOL}/aarch64-linux-gnu/lib64/"
+    cp numactl/numa_bin/lib/libnuma.* "${AARCH64_TOOL}/lib/"
+fi
.travis.yml new file +41 -0
new file mode 100644
index 000000000..0faa35770
@@ -0,0 +1,159 @@
+language: c
+compiler:
+  - gcc
+  - clang
+
+os:
+  - linux
+
+addons:
+  apt:
+    sources:
+      - deadsnakes  #Repo for python 3.5
+      - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+    packages:
+      - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip, ninja-build]
+
+before_install: ./.ci/${TRAVIS_OS_NAME}-setup.sh
+
+sudo: false
+
+env:
+  - SHARED=1
+  - DISABLE_KERNEL_MODULES=1
+  - SHARED=1 DISABLE_KERNEL_MODULES=1
+  - NINJABUILD=1
+  - NINJABUILD=1 SHARED=1
+  - NINJABUILD=1 DISABLE_KERNEL_MODULES=1
+  - NINJABUILD=1 SHARED=1 DISABLE_KERNEL_MODULES=1
+
+matrix:
+  include:
+  - env: SHARED=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+ - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip,

ninja-build]

.travis.yml new file +10 -0
+  - env: DISABLE_KERNEL_MODULES=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+ - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip,

ninja-build]

.travis.yml new file +10 -0
+  - env: SHARED=1 DISABLE_KERNEL_MODULES=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+ - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip,

ninja-build]

.travis.yml new file +10 -0
+  - env: SHARED=1
+    compiler: clang
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+ - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip,

ninja-build]

.travis.yml new file +10 -0
+  - env: DISABLE_KERNEL_MODULES=1
+    compiler: clang
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+ - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip,

ninja-build]

.travis.yml new file +10 -0
+  - env: SHARED=1 DISABLE_KERNEL_MODULES=1
+    compiler: clang
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+ - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip,

ninja-build]

.travis.yml new file +30 -0
+  - env: AARCH64=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), libtool, python3.5, python3-pip]
+  - env: AARCH64=1 NINJABUILD=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [linux-headers-$(uname -r), libtool, python3.5, python3-pip, ninja-build]
+  - env: NINJABUILD=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+ - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip,

ninja-build]

.travis.yml new file +10 -0
+  - env: NINJABUILD=1 SHARED=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+ - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip,

ninja-build]

.travis.yml new file +10 -0
+  - env: NINJABUILD=1 DISABLE_KERNEL_MODULES=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+ - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip,

ninja-build]

.travis.yml new file +10 -0
+  - env: NINJABUILD=1 SHARED=1 DISABLE_KERNEL_MODULES=1
+    compiler: gcc
+    addons:
+      apt:
+        sources:
+          - deadsnakes  #Repo for python 3.5
+          - sourceline: 'ppa:mstipicevic/ninja-build-1-7-2'
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+ - [libnuma-dev, linux-headers-$(uname -r), python3.5, python3-pip,

ninja-build]

.travis.yml new file +4 -0
+
+
+script: ./.ci/${TRAVIS_OS_NAME}-build.sh
+
Quoted 8 lines

.travis.yml line 159

Quoted 4 lines
.travis.yml new file +4 -0
+notifications:
+  email:
+    recipients:
+      - [email protected]

The idea of this mailing list is to receive reports about the upstream development. When doing a private development, reports should not be sent. How can it be disabled?

There are a few variables related to controlling this that we can investigate so that we only alert on the main repository, if you want.

We could also just remove it. Not a problem either way.

#33 lore raw

Re: [PATCH v5 2/2] ci: Introduce travis builds for github repositories

27/02/2019 15:35, Aaron Conole:

Quoted 4 lines
Quoted 3 lines · Thomas Monjalon

07/02/2019 23:01, Michael Santana:

Quoted 1 line

+# Just used for the 'classic' configuration system (ie: make)

Quoted 8 lines
Quoted 1 line

I am not sure about supporting the legacy system in a new CI.

For now, documentation all says that the legacy system is the supported system. I think it's appropriate to continue to support it until such time as it is eliminated. Otherwise, when an end user builds we don't know what to say about support - IE: if they have problems with the classic system for whatever reason, do we tell them "sorry, we cannot help"?

It is tested by others and me.

Quoted 5 lines

The patches.rst mentions that all patches must pass with the Makefile system, and the contributing/documentation.rst calls it the "standard DPDK build system." If you want to change those things to reflect something different please do, and we can drop all of the stuff related to it, but until that time we won't.

If it is forcing to have a translation of options, I think it is better to skip the legacy system in this CI.

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

+BUILD_ARCH="x86_64-native-linuxapp-"

We could have some native Arm compilation.

Sure. Is this just commentary? Do you suggest a change here? This is a default, and will be adjusted later by other parameters.

OK, I missed it is the default. Maybe a comment would help.

Quoted 9 lines
Quoted 6 lines

.ci/linux-build.sh line 25

Quoted 4 lines
.ci/linux-build.sh new file +4 -0
+if [ "${AARCH64}" == "1" ]; then
+    # convert the arch specifier
+    BUILD_ARCH="arm64-armv8a-linuxapp-"
+    AARCH64_TOOL="linaro-arm-tool"

What is this directory? It looks really specific to Travis.

It's specific to the AARCH64 toolchain that was pulled in as part of linux-prepare.sh - do you think something should change?

I think it should be a parameter somewhere else. In the .yml file?

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
.ci/linux-build.sh new file +8 -0
+    DEF_LIB="static"
+    if [ "${SHARED}" == "1" ]; then
+        DEF_LIB="shared"
+    fi
+
+    if [ "${DISABLE_KERNEL_MODULES}" == "1" ]; then
+        OPTS="-Denable_kmods=false"
+    fi
Quoted 6 lines
Quoted 2 lines

Isn't it possible to directly provide the meson options in travis.yml instead of doing a translation with new option names?

Yes and no. It would be possible, but we try to support both build systems and they have different options afaict. So we need something common. Maybe we missed something?

It was my understanding.
That's why I think we should drop the legacy support and focus on a simpler meson support.

Quoted 7 lines
Quoted 4 lines

.ci/linux-build.sh line 55

Quoted 2 lines
.ci/linux-build.sh new file +2 -0
+    set_conf build CONFIG_RTE_KNI_KMOD n
+    set_conf build CONFIG_RTE_EAL_IGB_UIO n

Why these options are fixed?

There was a problem with the Travis system when trying to build some of the kernel modules, but I don't remember the details. Maybe Michael does.

OK, please add a comment if kept.

Quoted 5 lines
Quoted 4 lines
Quoted 1 line

+python3.5 -m pip install --upgrade meson --user

Which distributions have python3.5?
It looks very specific.

Quoted 7 lines

I agree, could probably just be python3 we need to check and see if we can just use that.

But, we did need the upgrade.

Travis environment comes up with ubuntu 14.04, which includes python3.4, and the requisite version of meson needs python3.5 or higher.

It could be a python --version check then?

Quoted 9 lines
Quoted 6 lines

.ci/linux-setup.sh line 8

Quoted 2 lines
.ci/linux-setup.sh new file +2 -0
+if [ "${AARCH64}" == "1" ]; then
+    # need to build & install libnuma

Why is it needed? linbnuma is optional.
I think this file can be dropped or renamed to
something like install-libnuma-for-cross-arm.sh

We needed this because it broke the meson build when cross compiling, IIRC. I will investigate it further to be sure.

OK thanks

Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
.travis.yml new file +4 -0
+notifications:
+  email:
+    recipients:
+      - [email protected]
Quoted 8 lines
Quoted 3 lines

The idea of this mailing list is to receive reports about the upstream development. When doing a private development, reports should not be sent. How can it be disabled?

There are a few variables related to controlling this that we can investigate so that we only alert on the main repository, if you want.

We could also just remove it. Not a problem either way.

I think it should be disabled by default,
and documented how to enable it.

Then we may need a way to filter which travis report should be allowed in the mailing list, in order to avoid flooding it.
I think we cannot filter by sender. Do we need procmail?

#34 lore raw

Re: [PATCH v5 2/2] ci: Introduce travis builds for github repositories

Quoted 7 lines · Thomas Monjalon

27/02/2019 15:35, Aaron Conole:

Quoted 6 lines
Quoted 4 lines · Thomas Monjalon

07/02/2019 23:01, Michael Santana:

Quoted 1 line

+# Just used for the 'classic' configuration system (ie: make)

I am not sure about supporting the legacy system in a new CI.

Quoted 8 lines
Quoted 6 lines

For now, documentation all says that the legacy system is the supported system. I think it's appropriate to continue to support it until such time as it is eliminated. Otherwise, when an end user builds we don't know what to say about support - IE: if they have problems with the classic system for whatever reason, do we tell them "sorry, we cannot help"?

It is tested by others and me.

I test it, too. :)

Quoted 8 lines
Quoted 5 lines

The patches.rst mentions that all patches must pass with the Makefile system, and the contributing/documentation.rst calls it the "standard DPDK build system." If you want to change those things to reflect something different please do, and we can drop all of the stuff related to it, but until that time we won't.

If it is forcing to have a translation of options, I think it is better to skip the legacy system in this CI.

We can do that.

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Quoted 1 line

+BUILD_ARCH="x86_64-native-linuxapp-"

We could have some native Arm compilation.

Sure. Is this just commentary? Do you suggest a change here? This is a default, and will be adjusted later by other parameters.

OK, I missed it is the default. Maybe a comment would help.

Okay.

Quoted 24 lines
Quoted 9 lines
Quoted 6 lines

.ci/linux-build.sh line 25

Quoted 4 lines
.ci/linux-build.sh new file +4 -0
+if [ "${AARCH64}" == "1" ]; then
+    # convert the arch specifier
+    BUILD_ARCH="arm64-armv8a-linuxapp-"
+    AARCH64_TOOL="linaro-arm-tool"

What is this directory? It looks really specific to Travis.

It's specific to the AARCH64 toolchain that was pulled in as part of linux-prepare.sh - do you think something should change?

I think it should be a parameter somewhere else. In the .yml file?

Quoted 12 lines
Quoted 11 lines

.ci/linux-build.sh line 40

Quoted 8 lines
.ci/linux-build.sh new file +8 -0
+    DEF_LIB="static"
+    if [ "${SHARED}" == "1" ]; then
+        DEF_LIB="shared"
+    fi
+
+    if [ "${DISABLE_KERNEL_MODULES}" == "1" ]; then
+        OPTS="-Denable_kmods=false"
+    fi

Isn't it possible to directly provide the meson options in travis.yml instead of doing a translation with new option names?

Quoted 7 lines
Quoted 3 lines

Yes and no. It would be possible, but we try to support both build systems and they have different options afaict. So we need something common. Maybe we missed something?

It was my understanding.
That's why I think we should drop the legacy support and focus on a simpler meson support.

Okay.

Quoted 9 lines
Quoted 7 lines
Quoted 4 lines

.ci/linux-build.sh line 55

Quoted 2 lines
.ci/linux-build.sh new file +2 -0
+    set_conf build CONFIG_RTE_KNI_KMOD n
+    set_conf build CONFIG_RTE_EAL_IGB_UIO n

Why these options are fixed?

There was a problem with the Travis system when trying to build some of the kernel modules, but I don't remember the details. Maybe Michael does.

OK, please add a comment if kept.

I think the new plan is to drop it.

Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Quoted 1 line

+python3.5 -m pip install --upgrade meson --user

Which distributions have python3.5?
It looks very specific.

I agree, could probably just be python3 we need to check and see if we can just use that.

Quoted 6 lines
Quoted 4 lines

But, we did need the upgrade.

Travis environment comes up with ubuntu 14.04, which includes python3.4, and the requisite version of meson needs python3.5 or higher.

It could be a python --version check then?

Well, I don't know what the purpose of the version check would be. We can tailor the environments. Just need to pass the right information, that's all. I think we'll make it look nicer, though. We can do any environment specific fix ups in the prepare scripts, which I think makes sense. Then the build script can be a bit more generic.

Quoted 24 lines
Quoted 9 lines
Quoted 6 lines

.ci/linux-setup.sh line 8

Quoted 2 lines
.ci/linux-setup.sh new file +2 -0
+if [ "${AARCH64}" == "1" ]; then
+    # need to build & install libnuma

Why is it needed? linbnuma is optional.
I think this file can be dropped or renamed to
something like install-libnuma-for-cross-arm.sh

We needed this because it broke the meson build when cross compiling, IIRC. I will investigate it further to be sure.

OK thanks

Quoted 12 lines
Quoted 8 lines

.travis.yml line 159

Quoted 4 lines
.travis.yml new file +4 -0
+notifications:
+  email:
+    recipients:
+      - [email protected]

The idea of this mailing list is to receive reports about the upstream development. When doing a private development, reports should not be sent. How can it be disabled?

There are a few variables related to controlling this that we can investigate so that we only alert on the main repository, if you want.

Quoted 8 lines
Quoted 1 line

We could also just remove it. Not a problem either way.

I think it should be disabled by default,
and documented how to enable it.

Then we may need a way to filter which travis report should be allowed in the mailing list, in order to avoid flooding it.
I think we cannot filter by sender. Do we need procmail?

Maybe it would make sense to have a build list that we can use? Then we just let that absorb all the various build from different CI services. Just thinking out loud. That way, anyone interested in the build status can look there, and it keeps the test-report clean. Again, just a half-baked idea.

#35 lore raw

Re: [PATCH v5 2/2] ci: Introduce travis builds for github repositories

Quoted 22 lines · Aaron Conole
Quoted 12 lines · Thomas Monjalon

27/02/2019 15:35, Aaron Conole:

Quoted 11 lines
Quoted 5 lines · Thomas Monjalon

07/02/2019 23:01, Michael Santana:

Quoted 1 line

+python3.5 -m pip install --upgrade meson --user

Which distributions have python3.5?
It looks very specific.

I agree, could probably just be python3 we need to check and see if we
can just use that.

Quoted 7 lines
Quoted 5 lines

But, we did need the upgrade.

Travis environment comes up with ubuntu 14.04, which includes python3.4,
and the requisite version of meson needs python3.5 or higher.

It could be a python --version check then?

Quoted 8 lines

Well, I don't know what the purpose of the version check would be. We
can tailor the environments. Just need to pass the right information,
that's all. I think we'll make it look nicer, though. We can do any environment specific fix ups in the prepare scripts, which I think makes
sense. Then the build script can be a bit more generic.

Travis supports Ubuntu 16.04 images now, which have python 3.5, I'd suggest to just use them (dist: xenial) so you can skip all the checks and the manual installations.

Signature · 2 lines
Kind regards,
Luca Boccassi
#36 lore raw

Re: [PATCH v5 2/2] ci: Introduce travis builds for github repositories

Quoted 32 lines · Luca Boccassi
Quoted 22 lines · Aaron Conole
Quoted 12 lines · Thomas Monjalon

27/02/2019 15:35, Aaron Conole:

Quoted 11 lines
Quoted 5 lines · Thomas Monjalon

07/02/2019 23:01, Michael Santana:

Quoted 1 line

+python3.5 -m pip install --upgrade meson --user

Which distributions have python3.5?
It looks very specific.

I agree, could probably just be python3 we need to check and see if we
can just use that.

Quoted 7 lines
Quoted 5 lines

But, we did need the upgrade.

Travis environment comes up with ubuntu 14.04, which includes python3.4,
and the requisite version of meson needs python3.5 or higher.

It could be a python --version check then?

Quoted 8 lines

Well, I don't know what the purpose of the version check would be. We
can tailor the environments. Just need to pass the right information,
that's all. I think we'll make it look nicer, though. We can do any environment specific fix ups in the prepare scripts, which I think makes
sense. Then the build script can be a bit more generic.

Quoted 3 lines

Travis supports Ubuntu 16.04 images now, which have python 3.5, I'd suggest to just use them (dist: xenial) so you can skip all the checks and the manual installations.

Great news, thanks, Luca!

#37 lore raw

Re: [PATCH v5 0/2] Introduce travis support

In reply to Michael Santana

On 2/7/19 5:01 PM, Michael Santana wrote:
ping

Quoted 64 lines

This series introduces the ability for any github mirrors of the DPDK project, including developer mirrors, to kick off builds under the travis CI infrastructure. For now, this just means compilation - no other kinds of automated run exists yet. In the future, this can be expanded to execute and report results for any test-suites that might exist.

The series includes support for both the 'classic make' style builds (which are set to be deprecated at some undetermined point in the future), as well as the modern meson+ninja build system. Additionally, there is support for building the ARM64 target with both the meson and make systems.

The files added under .ci/ exist so that in the future, other CI support platforms (such as cirrus, appveyor, etc.) could have a common place to put their requisite scripts without polluting the main tree.

Some documentation is updated, making developers aware of the new travis integration.

The integration can also be included on the official DPDK mirror. Build reports can be enabled by subscribing the travis build email to the test-reports mailing list (this can be done independent of this series being applied).

v4->v5:
- Renamed ARM64 to AARCH64.

v3->v4:
- Remove non-existing file form maintainers list:
meson_cross_aarch64_gcc.txt
- Renamed ARM64 to AARCH64 for travis environment variable

v2->v3:
- Removed duplicate file meson_cross_aarch64_gcc.txt. Used arm64_armv8_linuxapp_gcc file instead
- Renamed ambiguous variable names and comments, including the variable KERNEL to DISABLE_KERNEL_MODULES and comment 'source for python' to 'Repo for python'
- Removed an already-defined variable

v1 -> v2:
- Added patch 1/2, "examples/vhost_scsi: don't build..."
- Included arm64 builds
- Included multiple meson+ninja builds (full library, minimal library)
- Included multiple 'classic make' builds

Aaron Conole (1):
examples/vhost_scsi: Don't build without virtio_scsi.h

Michael Santana (1):
ci: Introduce travis builds for github repositories

  .ci/linux-build.sh                  |  88 +++++++++++++++
  .ci/linux-setup.sh                  |  31 ++++++
  .travis.yml                         | 159 ++++++++++++++++++++++++++++
  MAINTAINERS                         |   6 ++
  doc/guides/contributing/patches.rst |   4 +
  examples/vhost_scsi/meson.build     |   5 +
  6 files changed, 293 insertions(+)

create mode 100755 .ci/linux-build.sh
create mode 100755 .ci/linux-setup.sh
create mode 100644 .travis.yml

#38 lore raw

Re: [PATCH v5 0/2] Introduce travis support

In reply to Michael Santana
Quoted 53 lines · Michael Santana

This series introduces the ability for any github mirrors of the DPDK project, including developer mirrors, to kick off builds under the travis CI infrastructure. For now, this just means compilation - no other kinds of automated run exists yet. In the future, this can be expanded to execute and report results for any test-suites that might exist.

The series includes support for both the 'classic make' style builds (which are set to be deprecated at some undetermined point in the future), as well as the modern meson+ninja build system. Additionally, there is support for building the ARM64 target with both the meson and make systems.

The files added under .ci/ exist so that in the future, other CI support platforms (such as cirrus, appveyor, etc.) could have a common place to put their requisite scripts without polluting the main tree.

Some documentation is updated, making developers aware of the new travis integration.

The integration can also be included on the official DPDK mirror. Build reports can be enabled by subscribing the travis build email to the test-reports mailing list (this can be done independent of this series being applied).

v4->v5:
- Renamed ARM64 to AARCH64.

v3->v4:
- Remove non-existing file form maintainers list:
meson_cross_aarch64_gcc.txt
- Renamed ARM64 to AARCH64 for travis environment variable

v2->v3:
- Removed duplicate file meson_cross_aarch64_gcc.txt. Used arm64_armv8_linuxapp_gcc file instead
- Renamed ambiguous variable names and comments, including the variable KERNEL to DISABLE_KERNEL_MODULES and comment 'source for python' to 'Repo for python'
- Removed an already-defined variable

v1 -> v2:
- Added patch 1/2, "examples/vhost_scsi: don't build..."
- Included arm64 builds
- Included multiple meson+ninja builds (full library, minimal library)
- Included multiple 'classic make' builds

Aaron Conole (1):
examples/vhost_scsi: Don't build without virtio_scsi.h

Michael Santana (1):
ci: Introduce travis builds for github repositories

Quoted 10 lines
 .ci/linux-build.sh                  |  88 +++++++++++++++
 .ci/linux-setup.sh                  |  31 ++++++
 .travis.yml                         | 159 ++++++++++++++++++++++++++++
 MAINTAINERS                         |   6 ++
 doc/guides/contributing/patches.rst |   4 +
 examples/vhost_scsi/meson.build     |   5 +
 6 files changed, 293 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml

Ping?

#39 lore raw

v6 cover Introduce travis support

In reply to Michael Santana

This series introduces the ability for any github mirrors of the DPDK project, including developer mirrors, to kick off builds under the travis CI infrastructure. For now, this just means compilation - no other kinds of automated run exists yet. In the future, this can be expanded to execute and report results for any test-suites that might exist.

The series includes support for both the 'classic make' style builds (which are set to be deprecated at some undetermined point in the future), as well as the modern meson+ninja build system. Additionally, there is support for building the ARM64 target with both the meson and make systems.

The files added under .ci/ exist so that in the future, other CI support platforms (such as cirrus, appveyor, etc.) could have a common place to put their requisite scripts without polluting the main tree.

Some documentation is updated, making developers aware of the new travis integration.

The integration can also be included on the official DPDK mirror. Build reports can be enabled by subscribing the travis build email to the test-reports mailing list (this can be done independent of this series being applied).

v5->v6:
- Removed all classic make builds

v4->v5:
- Renamed ARM64 to AARCH64.

v3->v4:
- Remove non-existing file form maintainers list:
meson_cross_aarch64_gcc.txt
- Renamed ARM64 to AARCH64 for travis environment variable

v2->v3:
- Removed duplicate file meson_cross_aarch64_gcc.txt. Used arm64_armv8_linuxapp_gcc file instead
- Renamed ambiguous variable names and comments, including the variable KERNEL to DISABLE_KERNEL_MODULES and comment 'source for python' to 'Repo for python'
- Removed an already-defined variable

v1 -> v2:
- Added patch 1/2, "examples/vhost_scsi: don't build..."
- Included arm64 builds
- Included multiple meson+ninja builds (full library, minimal library)
- Included multiple 'classic make' builds

Michael Santana (1):
ci: Introduce travis builds for github repositories

 .ci/linux-build.sh                  | 21 +++++++++
 .ci/linux-setup.sh                  |  3 ++
 .travis.yml                         | 73 +++++++++++++++++++++++++++++
 MAINTAINERS                         |  6 +++
 doc/guides/contributing/patches.rst |  4 ++
 5 files changed, 107 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
Signature · 1 line
2.20.1
#40 lore raw

PATCH v6 1/1 ci: Introduce travis builds for github repositories

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Additionally, the travis service will send an email to the test-report list informing anyone interested in the automated build (including a result).

---
v6:
Removed all classic make builds

 .ci/linux-build.sh                  | 21 +++++++++
 .ci/linux-setup.sh                  |  3 ++
 .travis.yml                         | 73 +++++++++++++++++++++++++++++
 MAINTAINERS                         |  6 +++
 doc/guides/contributing/patches.rst |  4 ++
 5 files changed, 107 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
.ci/linux-build.sh new file +21 -0
new file mode 100755
index 000000000..6b65ad31b
@@ -0,0 +1,21 @@
+#!/bin/bash -xe
+
+function on_error() {
+    FILES_TO_PRINT=( "build/meson-logs/testlog.txt" "build/.ninja_log" "build/meson-logs/meson-log.txt")
+
+    for pr_file in "${FILES_TO_PRINT[@]}"; do
+        if [ -e "$pr_file" ]; then
+            cat "$pr_file"
+        fi
+    done
+}
+trap on_error ERR
+
+if [ "${AARCH64}" == "1" ]; then
+    # convert the arch specifier
+    OPTS="${OPTS} -DRTE_ARCH_64=1 --cross-file config/arm/arm64_armv8_linuxapp_gcc"
+fi
+
+OPTS="$OPTS --default-library=$DEF_LIB"
+meson build --werror -Dexamples=all ${OPTS}
+ninja -C build
.ci/linux-setup.sh new file +3 -0
new file mode 100755
index 000000000..63502c90a
@@ -0,0 +1,3 @@
+ #!/bin/bash
+
+python3 -m pip install --upgrade meson --user
.travis.yml new file +73 -0

73 lines not on this page · raw message

MAINTAINERS +6 -0
index 15c53888c..e4b9a8e00 100644
@@ -119,6 +119,12 @@ F: config/rte_config.h
 F: buildtools/gen-pmdinfo-cfile.sh
 F: buildtools/symlink-drivers-solibs.sh
 
+Public CI
+M: Aaron Conole <[email protected]>
+M: Michael Santana <[email protected]>
+F: .travis.yml
+F: .ci/
+
 ABI versioning
 M: Neil Horman <[email protected]>
 F: doc/guides/rel_notes/deprecation.rst
doc/guides/contributing/patches.rst +4 -0
index 211a5cdc7..22a9039e8 100644
@@ -32,6 +32,10 @@ The mailing list for DPDK development is `[email protected] <http://mails.dpdk.org/ar
 Contributors will need to `register for the mailing list <http://mails.dpdk.org/listinfo/dev>`_ in order to submit patches.
 It is also worth registering for the DPDK `Patchwork <http://patches.dpdk.org/project/dpdk/list/>`_
 
+If you are using the GitHub service, you can link your repository to
+the ``travis-ci.org`` build service.  When you push patches to your GitHub
+repository, the travis service will automatically build your changes.
+
 The development process requires some familiarity with the ``git`` version control system.
 Refer to the `Pro Git Book <http://www.git-scm.com/book/>`_ for further information.
 
Signature · 1 line
2.20.1
Acked-by on v6 1/1
#41 lore raw

Re: [PATCH v6 1/1] ci: Introduce travis builds for github repositories

Quoted 23 lines · Michael Santana

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK
project. It doesn't require any changes from individual developers to
enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Additionally, the travis service will send an email to the test- report
list informing anyone interested in the automated build (including a result).

---
v6:
Removed all classic make builds

Quoted 10 lines
 .ci/linux-build.sh                  | 21 +++++++++
 .ci/linux-setup.sh                  |  3 ++
 .travis.yml                         | 73
+++++++++++++++++++++++++++++
 MAINTAINERS                         |  6 +++
 doc/guides/contributing/patches.rst |  4 ++
 5 files changed, 107 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
Signature · 2 lines
Kind regards,
Luca Boccassi
#42 lore raw

Re: [PATCH v6 1/1] ci: Introduce travis builds for github repositories

Quoted 34 lines · Luca Boccassi
Quoted 30 lines · Michael Santana

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This is a simple initial implementation of a travis build for the DPDK
project. It doesn't require any changes from individual developers to
enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

Additionally, the travis service will send an email to the test- report
list informing anyone interested in the automated build (including a result).

---
v6:
Removed all classic make builds

  .ci/linux-build.sh                  | 21 +++++++++
  .ci/linux-setup.sh                  |  3 ++
  .travis.yml                         | 73
+++++++++++++++++++++++++++++
  MAINTAINERS                         |  6 +++
  doc/guides/contributing/patches.rst |  4 ++
  5 files changed, 107 insertions(+)
Quoted 3 lines

create mode 100755 .ci/linux-build.sh
create mode 100755 .ci/linux-setup.sh
create mode 100644 .travis.yml

Quoted 2 lines

ping

#43 lore raw

Re: [PATCH v6 1/1] ci: Introduce travis builds for github repositories

In reply to Michael Santana

Hi,

04/03/2019 17:12, Michael Santana:

Quoted 3 lines
 .ci/linux-build.sh                  | 21 +++++++++
 .ci/linux-setup.sh                  |  3 ++
 .travis.yml                         | 73 +++++++++++++++++++++++++++++

Please, could you explain somewhere what is the relationship between these files?
What is specific to Travis?
What is specific to GitHub?

May we add "travis-" as filename prefix of the scripts? Or rename .ci to .travis?

.ci/linux-build.sh line 1

Quoted 3 lines

+++ b/.ci/linux-build.sh

.ci/linux-build.sh new file +1 -0
@@ -0,0 +1,21 @@
+#!/bin/bash -xe

If possible, I would prefer a simple /bin/sh.

.ci/linux-build.sh line 6

Quoted 4 lines
.ci/linux-build.sh new file +4 -0
+function on_error() {
+    FILES_TO_PRINT=( "build/meson-logs/testlog.txt" "build/.ninja_log" "build/meson-logs/meson-log.txt")
+
+    for pr_file in "${FILES_TO_PRINT[@]}"; do

You can make FILES_TO_PRINT as a simple word list, and so avoid bashism.

[...]

.ci/linux-build.sh line 14

Quoted 1 line
.ci/linux-build.sh new file +1 -0
+if [ "${AARCH64}" == "1" ]; then

Please explain in the comment where this variable comes from. I suggest renaming it to ARMV8 as this is what it is translated to:

.ci/linux-build.sh line 16

Quoted 2 lines
.ci/linux-build.sh new file +2 -0
+    # convert the arch specifier
+    OPTS="${OPTS} -DRTE_ARCH_64=1 --cross-file config/arm/arm64_armv8_linuxapp_gcc"

I think -DRTE_ARCH_64=1 is useless.

Quoted 5 lines
.ci/linux-build.sh new file +5 -0
+fi
+
+OPTS="$OPTS --default-library=$DEF_LIB"
+meson build --werror -Dexamples=all ${OPTS}
+ninja -C build

[...]

.travis.yml line 6

Quoted 9 lines
.travis.yml new file +6 -0
@@ -0,0 +1,73 @@
+language: c
+compiler:
+  - gcc
+  - clang
+
+dist: xenial

Are we going to update the distribution frequently? Why not adding more distros?

.travis.yml line 9

Quoted 2 lines
.travis.yml new file +2 -0
+os:
+  - linux

Is it possible to run on FreeBSD?

Quoted 5 lines
.travis.yml new file +5 -0
+addons:
+  apt:
+    update: true
+    packages:
+      - [libnuma-dev, linux-headers-$(uname -r), python3-setuptools, python3-wheel, python3-pip, ninja-build]

.travis.yml line 25

Quoted 10 lines
.travis.yml new file +10 -0
+
+before_install: ./.ci/${TRAVIS_OS_NAME}-setup.sh
+
+sudo: false
+
+env:
+  - DEF_LIB="static"
+  - DEF_LIB="shared"
+  - DEF_LIB="static" OPTS="-Denable_kmods=false"
+  - DEF_LIB="shared" OPTS="-Denable_kmods=false"

How is it different of the matrix below?
Why testing disabling kmods?

.travis.yml line 35

Quoted 10 lines
.travis.yml new file +10 -0
+
+matrix:
+  include:
+  - env: DEF_LIB="static" OPTS="-Denable_kmods=false" AARCH64=1
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - [libnuma-dev, linux-headers-$(uname -r), python3-setuptools, python3-wheel, python3-pip, ninja-build]
+          - [gcc-aarch64-linux-gnu, libc6-dev-arm64-cross]

Why packages are repeated here again?
(sorry, I don't know Travis and I want to understand)

Quoted 25 lines
.travis.yml new file +25 -0
+  - env: DEF_LIB="shared" OPTS="-Denable_kmods=false" AARCH64=1
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - [libnuma-dev, linux-headers-$(uname -r), python3-setuptools, python3-wheel, python3-pip, ninja-build]
+          - [gcc-aarch64-linux-gnu, libc6-dev-arm64-cross]
+  - env: DEF_LIB="static"
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3-setuptools, python3-wheel, python3-pip, ninja-build]
+  - env: DEF_LIB="shared"
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3-setuptools, python3-wheel, python3-pip, ninja-build]
+  - env: DEF_LIB="static" OPTS="-Denable_kmods=false"
+    compiler: gcc
+    addons:
+      apt:

.travis.yml line 70

Quoted 10 lines
.travis.yml new file +10 -0
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3-setuptools, python3-wheel, python3-pip, ninja-build]
+  - env: DEF_LIB="shared" OPTS="-Denable_kmods=false"
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3-setuptools, python3-wheel, python3-pip, ninja-build]

It seems clang is not in the matrix. Why?

Thanks for this v6.
I will be available to follow more closely in next days, so we can merge this feature soon this week.

#44 lore raw

Re: [PATCH v6 1/1] ci: Introduce travis builds for github repositories

Thank you for taking at look at this

Quoted 6 lines · Thomas Monjalon

Hi,

04/03/2019 17:12, Michael Santana:

Quoted 3 lines
  .ci/linux-build.sh                  | 21 +++++++++
  .ci/linux-setup.sh                  |  3 ++
  .travis.yml                         | 73 +++++++++++++++++++++++++++++
Quoted 7 lines

Please, could you explain somewhere what is the relationship between these files?
What is specific to Travis?
What is specific to GitHub?

May we add "travis-" as filename prefix of the scripts? Or rename .ci to .travis?

Only the .travis.yml is specific to travis. The other two files are used by travis, but are independent of travis.
This allows us for in the future to change travis as CI and use something else instead, or add another CI in addition to travis if we wanted to. Other CI's would just run these scripts just like travis does. With that said, I would not change the names, but if you really think they should then it's not a problem

Quoted 11 lines

.ci/linux-build.sh line 1

Quoted 3 lines

+++ b/.ci/linux-build.sh

.ci/linux-build.sh new file +1 -0
@@ -0,0 +1,21 @@
+#!/bin/bash -xe

If possible, I would prefer a simple /bin/sh.

.ci/linux-build.sh line 6

Quoted 4 lines
.ci/linux-build.sh new file +4 -0
+function on_error() {
+    FILES_TO_PRINT=( "build/meson-logs/testlog.txt" "build/.ninja_log" "build/meson-logs/meson-log.txt")
+
+    for pr_file in "${FILES_TO_PRINT[@]}"; do

You can make FILES_TO_PRINT as a simple word list, and so avoid bashism.

Will look into it

Quoted 5 lines

[...]

Quoted 1 line

+if [ "${AARCH64}" == "1" ]; then

Please explain in the comment where this variable comes from. I suggest renaming it to ARMV8 as this is what it is translated to:

The variable comes from travis. If you look at the matrix in the travis.yml you will see lines containing environment variables like AARCH64=1. These lines tell the travis build job to explicitly export these variables so that they can be used by the CI scripts like this one. As for ARMV8 someone had asked specifically to be named AARCH64

Quoted 11 lines

.ci/linux-build.sh line 16

Quoted 3 lines
.ci/linux-build.sh new file +2 -0
+    # convert the arch specifier
+    OPTS="${OPTS} -DRTE_ARCH_64=1 --cross-file config/arm/arm64_armv8_linuxapp_gcc"

I think -DRTE_ARCH_64=1 is useless.

Quoted 5 lines
.ci/linux-build.sh new file +5 -0
+fi
+
+OPTS="$OPTS --default-library=$DEF_LIB"
+meson build --werror -Dexamples=all ${OPTS}
+ninja -C build

[...]

Quoted 11 lines

.travis.yml line 6

Quoted 9 lines
.travis.yml new file +6 -0
@@ -0,0 +1,73 @@
+language: c
+compiler:
+  - gcc
+  - clang
+
+dist: xenial

Are we going to update the distribution frequently? Why not adding more distros?

The only Linux distribution travis supports is Ubuntu, and the latest Ubuntu version they support is xenial which is code name for Ubuntu 16.04

Quoted 4 lines

.travis.yml line 9

Quoted 3 lines
.travis.yml new file +2 -0
+os:
+  - linux

Is it possible to run on FreeBSD?

Not that I am aware. Travis only supports Ubuntu, Windows, and Mac

Quoted 10 lines
Quoted 10 lines
.travis.yml new file +9 -0
+addons:
+  apt:
+    update: true
+    packages:
+      - [libnuma-dev, linux-headers-$(uname -r), python3-setuptools, python3-wheel, python3-pip, ninja-build]
+
+before_install: ./.ci/${TRAVIS_OS_NAME}-setup.sh
+
+sudo: false
Quoted 8 lines

.travis.yml line 25

Quoted 6 lines
.travis.yml new file +6 -0
+
+env:
+  - DEF_LIB="static"
+  - DEF_LIB="shared"
+  - DEF_LIB="static" OPTS="-Denable_kmods=false"
+  - DEF_LIB="shared" OPTS="-Denable_kmods=false"

How is it different of the matrix below?
Why testing disabling kmods?

This list inherits the packages from the package list above.
The main difference is that this list does not contain extra packages such as libbsd-dev, libpcap-dev, etc, where as the matrix below does. This is so that we have a series of builds with minimal libraries installed (Minimal build) and another series of builds with many libraries installed (Full build). We want to mix and match all the possible build scenarios and see if a new changes breaks any of the build cases.
The more builds we have with the many different configurations we can have the wider the net we can cast to ensure everything is working as it should

Quoted 13 lines

.travis.yml line 35

Quoted 11 lines
.travis.yml new file +10 -0
+
+matrix:
+  include:
+  - env: DEF_LIB="static" OPTS="-Denable_kmods=false" AARCH64=1
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - [libnuma-dev, linux-headers-$(uname -r), python3-setuptools, python3-wheel, python3-pip, ninja-build]
+          - [gcc-aarch64-linux-gnu, libc6-dev-arm64-cross]

Why packages are repeated here again?
(sorry, I don't know Travis and I want to understand)

Yeah, we don't want to repeat ourselves either but we have no choice. This is due to a limitation in travis.
This matrix does not inherit any packages from the main package list way above, which means we have to list them out manually here.
In addition to the required packages we also want to install full builds with libraries like libbsd-dev, libpcap-dev, etc.
We could of just put those libraries in the main package list above and put all the builds in the env: list because then the libraries would be inherited.
The problem with that is that is that travis would not keep minimal builds and full builds separate.
We could not have minimal builds because the minimal builds will also inherit the additional libraries; Meson will then automatically detect those additional libraries and builds with them. What we would like to have is a way to tell meson which libraries we want to use and which we dont, instead of being auto-detected. This would help us to get rid of this matrix.

If someone knows a better way to do this we would greatly take in your ideas, but so far this is the best we could come up with

Quoted 36 lines
Quoted 26 lines
.travis.yml new file +25 -0
+  - env: DEF_LIB="shared" OPTS="-Denable_kmods=false" AARCH64=1
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - [libnuma-dev, linux-headers-$(uname -r), python3-setuptools, python3-wheel, python3-pip, ninja-build]
+          - [gcc-aarch64-linux-gnu, libc6-dev-arm64-cross]
+  - env: DEF_LIB="static"
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3-setuptools, python3-wheel, python3-pip, ninja-build]
+  - env: DEF_LIB="shared"
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3-setuptools, python3-wheel, python3-pip, ninja-build]
+  - env: DEF_LIB="static" OPTS="-Denable_kmods=false"
+    compiler: gcc
+    addons:
+      apt:

.travis.yml line 70

Quoted 10 lines
.travis.yml new file +10 -0
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3-setuptools, python3-wheel, python3-pip, ninja-build]
+  - env: DEF_LIB="shared" OPTS="-Denable_kmods=false"
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - [libbsd-dev, libpcap-dev, libcrypto++-dev, libjansson4]
+          - [libnuma-dev, linux-headers-$(uname -r), python3-setuptools, python3-wheel, python3-pip, ninja-build]
Quoted 1 line

It seems clang is not in the matrix. Why?

It looks like my mistake, will look into it

Quoted 6 lines

Thanks for this v6.
I will be available to follow more closely in next days, so we can merge this feature soon this week.

#45 lore raw

Re: [PATCH v6 1/1] ci: Introduce travis builds for github repositories

Quoted 37 lines · Michael Santana Francisco
Quoted 10 lines
Quoted 10 lines
.travis.yml new file +8 -0
+matrix:
+  include:
+  - env: DEF_LIB="static" OPTS="-Denable_kmods=false" AARCH64=1
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - [libnuma-dev, linux-headers-$(uname -r), python3-

setuptools, python3-wheel, python3-pip, ninja-build]
+ - [gcc-aarch64-linux-gnu, libc6-dev-arm64-cross]

Quoted 2 lines

Why packages are repeated here again?
(sorry, I don't know Travis and I want to understand)

Yeah, we don't want to repeat ourselves either but we have no choice.
This is due to a limitation in travis.
This matrix does not inherit any packages from the main package list way
above, which means we have to list them out manually here. In addition to the required packages we also want to install full builds
with libraries like libbsd-dev, libpcap-dev, etc.
We could of just put those libraries in the main package list above and
put all the builds in the env: list because then the libraries would be
inherited.
The problem with that is that is that travis would not keep minimal builds and full builds separate.
We could not have minimal builds because the minimal builds will also
inherit the additional libraries; Meson will then automatically detect
those additional libraries and builds with them.
What we would like to have is a way to tell meson which libraries we want to use and which we dont, instead of being auto-detected. This would help us to get rid of this matrix.

Quoted 3 lines

If someone knows a better way to do this we would greatly take in your
ideas, but so far this is the best we could come up with

It's yaml so you can write the list to a variable once and reference it multiple times for brevity, see for example:

https://github.com/zeromq/czmq/blob/master/.travis.yml#L128

Signature · 2 lines
Kind regards,
Luca Boccassi
#46 lore raw

Re: [PATCH v6 1/1] ci: Introduce travis builds for github repositories

Quoted 38 lines · Luca Boccassi
Quoted 37 lines · Michael Santana Francisco
Quoted 10 lines
Quoted 10 lines
.travis.yml new file +8 -0
+matrix:
+  include:
+  - env: DEF_LIB="static" OPTS="-Denable_kmods=false" AARCH64=1
+    compiler: gcc
+    addons:
+      apt:
+        packages:
+          - [libnuma-dev, linux-headers-$(uname -r), python3-

setuptools, python3-wheel, python3-pip, ninja-build]
+ - [gcc-aarch64-linux-gnu, libc6-dev-arm64-cross]

Quoted 2 lines

Why packages are repeated here again?
(sorry, I don't know Travis and I want to understand)

Yeah, we don't want to repeat ourselves either but we have no choice.
This is due to a limitation in travis.
This matrix does not inherit any packages from the main package list way
above, which means we have to list them out manually here. In addition to the required packages we also want to install full builds
with libraries like libbsd-dev, libpcap-dev, etc.
We could of just put those libraries in the main package list above and
put all the builds in the env: list because then the libraries would be
inherited.
The problem with that is that is that travis would not keep minimal builds and full builds separate.
We could not have minimal builds because the minimal builds will also
inherit the additional libraries; Meson will then automatically detect
those additional libraries and builds with them.
What we would like to have is a way to tell meson which libraries we want to use and which we dont, instead of being auto-detected. This would help us to get rid of this matrix.

Quoted 8 lines
Quoted 3 lines

If someone knows a better way to do this we would greatly take in your
ideas, but so far this is the best we could come up with

It's yaml so you can write the list to a variable once and reference it multiple times for brevity, see for example:

https://github.com/zeromq/czmq/blob/master/.travis.yml#L128

Thank you Luca!
I had previously tested this approach and then it wasn't working the way I was hoping it would. I just finished testing it again and it seems to be working correctly this time!
I got it working with the required packages in one list and use a variable to reference to that list. Next I will see about having a list for extra packages, and another for ARM packages Again, thank you! Will work on it

#47 lore raw

PATCH v7 1/1 ci: Introduce travis builds for github repositories

In reply to Michael Santana

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This series introduces the ability for any github mirrors of the DPDK project, including developer mirrors, to kick off builds under the travis CI infrastructure. For now, this just means compilation - no other kinds of automated run exists yet. In the future, this can be expanded to execute and report results for any test-suites that might exist.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

The files added under .ci/ exist so that in the future, other CI support platforms (such as cirrus, appveyor, etc.) could have a common place to put their requisite scripts without polluting the main tree.

---
 .ci/linux-build.sh                  | 24 +++++++
 .ci/linux-setup.sh                  |  3 +
 .travis.yml                         | 99 +++++++++++++++++++++++++++++
 MAINTAINERS                         |  6 ++
 doc/guides/contributing/patches.rst |  4 ++
 5 files changed, 136 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
.ci/linux-build.sh new file +24 -0
new file mode 100755
index 000000000..767840e40
@@ -0,0 +1,24 @@
+#!/bin/sh -xe
+
+on_error() {
+    if [ $? = 0 ]; then
+        exit
+    fi
+    FILES_TO_PRINT="build/meson-logs/testlog.txt build/.ninja_log build/meson-logs/meson-log.txt"
+
+    for pr_file in ${FILES_TO_PRINT}; do
+        if [ -e "$pr_file" ]; then
+            cat "$pr_file"
+        fi
+    done
+}
+trap on_error EXIT
+
+if [ "${AARCH64}" = "1" ]; then
+    # convert the arch specifier
+    OPTS="${OPTS} -DRTE_ARCH_64=1 --cross-file config/arm/arm64_armv8_linuxapp_gcc"
+fi
+
+OPTS="$OPTS --default-library=$DEF_LIB"
+meson build --werror -Dexamples=all ${OPTS}
+ninja -C build
.ci/linux-setup.sh new file +3 -0
new file mode 100755
index 000000000..acdf9f370
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+python3 -m pip install --upgrade meson --user
.travis.yml new file +99 -0

99 lines not on this page · raw message

MAINTAINERS +6 -0
index 15c53888c..e4b9a8e00 100644
@@ -119,6 +119,12 @@ F: config/rte_config.h
 F: buildtools/gen-pmdinfo-cfile.sh
 F: buildtools/symlink-drivers-solibs.sh
 
+Public CI
+M: Aaron Conole <[email protected]>
+M: Michael Santana <[email protected]>
+F: .travis.yml
+F: .ci/
+
 ABI versioning
 M: Neil Horman <[email protected]>
 F: doc/guides/rel_notes/deprecation.rst
doc/guides/contributing/patches.rst +4 -0
index 211a5cdc7..22a9039e8 100644
@@ -32,6 +32,10 @@ The mailing list for DPDK development is `[email protected] <http://mails.dpdk.org/ar
 Contributors will need to `register for the mailing list <http://mails.dpdk.org/listinfo/dev>`_ in order to submit patches.
 It is also worth registering for the DPDK `Patchwork <http://patches.dpdk.org/project/dpdk/list/>`_
 
+If you are using the GitHub service, you can link your repository to
+the ``travis-ci.org`` build service.  When you push patches to your GitHub
+repository, the travis service will automatically build your changes.
+
 The development process requires some familiarity with the ``git`` version control system.
 Refer to the `Pro Git Book <http://www.git-scm.com/book/>`_ for further information.
 
Signature · 1 line
2.20.1
#48 lore raw

PATCH v8 1/1 ci: Introduce travis builds for github repositories

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This series introduces the ability for any github mirrors of the DPDK project, including developer mirrors, to kick off builds under the travis CI infrastructure. For now, this just means compilation - no other kinds of automated run exists yet. In the future, this can be expanded to execute and report results for any test-suites that might exist.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

The files added under .ci/ exist so that in the future, other CI support platforms (such as cirrus, appveyor, etc.) could have a common place to put their requisite scripts without polluting the main tree.

---
 .ci/linux-build.sh                  | 24 +++++++
 .ci/linux-setup.sh                  |  3 +
 .travis.yml                         | 99 +++++++++++++++++++++++++++++
 MAINTAINERS                         |  6 ++
 doc/guides/contributing/patches.rst |  4 ++
 5 files changed, 136 insertions(+)
 create mode 100755 .ci/linux-build.sh
 create mode 100755 .ci/linux-setup.sh
 create mode 100644 .travis.yml
.ci/linux-build.sh new file +24 -0
new file mode 100755
index 000000000..767840e40
@@ -0,0 +1,24 @@
+#!/bin/sh -xe
+
+on_error() {
+    if [ $? = 0 ]; then
+        exit
+    fi
+    FILES_TO_PRINT="build/meson-logs/testlog.txt build/.ninja_log build/meson-logs/meson-log.txt"
+
+    for pr_file in $FILES_TO_PRINT; do
+        if [ -e "$pr_file" ]; then
+            cat "$pr_file"
+        fi
+    done
+}
+trap on_error EXIT
+
+if [ "$AARCH64" = "1" ]; then
+    # convert the arch specifier
+    OPTS="$OPTS --cross-file config/arm/arm64_armv8_linuxapp_gcc"
+fi
+
+OPTS="$OPTS --default-library=$DEF_LIB"
+meson build --werror -Dexamples=all $OPTS
+ninja -C build
.ci/linux-setup.sh new file +3 -0
new file mode 100755
index 000000000..acdf9f370
@@ -0,0 +1,3 @@
+#!/bin/sh
+
+python3 -m pip install --upgrade meson --user
.travis.yml new file +99 -0

99 lines not on this page · raw message

MAINTAINERS +6 -0
index 15c53888c..e4b9a8e00 100644
@@ -119,6 +119,12 @@ F: config/rte_config.h
 F: buildtools/gen-pmdinfo-cfile.sh
 F: buildtools/symlink-drivers-solibs.sh
 
+Public CI
+M: Aaron Conole <[email protected]>
+M: Michael Santana <[email protected]>
+F: .travis.yml
+F: .ci/
+
 ABI versioning
 M: Neil Horman <[email protected]>
 F: doc/guides/rel_notes/deprecation.rst
doc/guides/contributing/patches.rst +4 -0
index 211a5cdc7..22a9039e8 100644
@@ -32,6 +32,10 @@ The mailing list for DPDK development is `[email protected] <http://mails.dpdk.org/ar
 Contributors will need to `register for the mailing list <http://mails.dpdk.org/listinfo/dev>`_ in order to submit patches.
 It is also worth registering for the DPDK `Patchwork <http://patches.dpdk.org/project/dpdk/list/>`_
 
+If you are using the GitHub service, you can link your repository to
+the ``travis-ci.org`` build service.  When you push patches to your GitHub
+repository, the travis service will automatically build your changes.
+
 The development process requires some familiarity with the ``git`` version control system.
 Refer to the `Pro Git Book <http://www.git-scm.com/book/>`_ for further information.
 
Signature · 1 line
2.20.1
Acked-by on v8 1/1
#49 lore raw

Re: [PATCH v8] ci: Introduce travis builds for github repositories

25/03/2019 16:32, Michael Santana:

Quoted 21 lines

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This series introduces the ability for any github mirrors of the DPDK project, including developer mirrors, to kick off builds under the travis CI infrastructure. For now, this just means compilation - no other kinds of automated run exists yet. In the future, this can be expanded to execute and report results for any test-suites that might exist.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

The files added under .ci/ exist so that in the future, other CI support platforms (such as cirrus, appveyor, etc.) could have a common place to put their requisite scripts without polluting the main tree.

Quoted 5 lines
#50 lore raw

Re: [PATCH v8] ci: Introduce travis builds for github repositories

25/03/2019 17:10, Thomas Monjalon:

Quoted 22 lines

25/03/2019 16:32, Michael Santana:

Quoted 21 lines

GitHub is a service used by developers to store repositories. GitHub provides service integrations that allow 3rd party services to access developer repositories and perform actions. One of these services is Travis-CI, a simple continuous integration platform.

This series introduces the ability for any github mirrors of the DPDK project, including developer mirrors, to kick off builds under the travis CI infrastructure. For now, this just means compilation - no other kinds of automated run exists yet. In the future, this can be expanded to execute and report results for any test-suites that might exist.

This is a simple initial implementation of a travis build for the DPDK project. It doesn't require any changes from individual developers to enable, but will allow those developers who opt-in to GitHub and the travis service to get automatic builds for every push they make.

The files added under .ci/ exist so that in the future, other CI support platforms (such as cirrus, appveyor, etc.) could have a common place to put their requisite scripts without polluting the main tree.

Quoted 7 lines
Quoted 5 lines

Applied, thanks, it looks good.