Skip to contentSkip to conversation

atp: fix failing test_readme

PATCH Applied Jon Mason 2 participants 8 messages Nov 21, 2022

lore
Share & tools
lore

Jon Mason

author
#0 lore raw

PATCH 1/7 atp: fix failing test_readme

From: Adrián Herrera Arcila <[email protected]>

Test failed because there was no reference to maintainers in the meta-atp README.

Following the common structure of other layers in the meta-arm repository, the README in meta-atp now refers to the top-level README, and a documentation directory contains the guidance that was present in the original meta-atp README.

---
 README.md                                |  2 +-
 meta-atp/README.md                       | 77 +-----------------------
 meta-atp/documentation/atp-standalone.md | 13 ++++
 meta-atp/documentation/gem5-atp-arm64.md | 37 ++++++++++++
 meta-atp/documentation/summary.md        | 11 ++++
 5 files changed, 63 insertions(+), 77 deletions(-)
 create mode 100644 meta-atp/documentation/atp-standalone.md
 create mode 100644 meta-atp/documentation/gem5-atp-arm64.md
 create mode 100644 meta-atp/documentation/summary.md
README.md +1 -1
index 221633e0..e82f5532 100644
@@ -20,7 +20,7 @@ This repository contains the Arm layers for OpenEmbedded.
 
 * meta-atp
 
-  This layer contains recipes for the Adaptive Traffic Generation integration into meta-gem5.
+  This layer contains recipes for the [AMBA Adaptive Traffic Profiles (ATP)](https://developer.arm.com/documentation/ihi0082/latest) generation integration into meta-gem5.
 
 * meta-gem5
 
meta-atp/README.md +1 -76

77 lines not on this page · raw message

meta-atp/documentation/atp-standalone.md new file +13 -0
new file mode 100644
index 00000000..1ab4528a
@@ -0,0 +1,13 @@
+# Standalone ATP Engine executable
+
+Users can build the AMBA ATP Engine as a standalone native executable as follows:
+
+```bash
+bitbake atp-native
+```
+
+Users can run the executable through standard build scripts:
+
+```bash
+oe-run-native atp-native atpeng [--help | args...]
+```
meta-atp/documentation/gem5-atp-arm64.md new file +37 -0
new file mode 100644
index 00000000..39f2f3b0
@@ -0,0 +1,37 @@
+## ATP Engine integration in gem5, and the gem5-atp-arm64 machine
+
+Users should select the `gem5-atp-arm64` machine in their `conf/local.conf` file.
+
+Users can build the target image of preference, for example:
+
+```bash
+bitbake core-image-minimal
+```
+
+The resulting gem5 native executable contains the AMBA ATP Engine. The resulting target image contains the kernel modules, UAPI and test suite.
+
+Users should run the environment as follows:
+
+```bash
+./tmp/deploy/tools/start-gem5-atp.sh
+```
+
+This script launches a fast simulation to fast-forward Linux boot. Once Linux boot is completed, the fast simulation switches into a detailed simulation for the final usable environment. Users can connect and interact with the environment as follows:
+
+```bash
+oe-run-native gem5-m5term-native m5term <PORT>
+```
+
+The connection PORT is announced by the deploy script as:
+
+```bash
+system.terminal: Listening for connections on port <PORT>
+```
+
+This is usually port 3456.
+
+Users can verify access to the ATP device by running the integration test suite from within the simulated environment as follows:
+
+```bash
+test_atp.out
+```
meta-atp/documentation/summary.md new file +11 -0
new file mode 100644
index 00000000..7ac6c3f5
@@ -0,0 +1,11 @@
+# meta-atp summary
+
+The meta-atp layer supports building the following software components:
+
+- Arm's implementation of the AMBA ATP specification, namely the [AMBA ATP Engine](https://github.com/ARM-software/ATP-Engine).
+- Linux kernel modules and user API (UAPI) for programming ATP devices.
+- Integration test suite for verification of kernel modules and UAPI.
+
+It is also possible to build the AMBA ATP Engine as part of the final [gem5](https://www.gem5.org/) executable. For this, meta-atp extends the `gem5-aarch64-native` recipe to add the AMBA ATP engine code as extra sources.
+
+Users should add the meta-atp layer and layer dependencies to `conf/bblayers.conf`. See `conf/layer.conf` for dependencies.
Signature · 1 line
2.17.1
author
#1 lore raw

PATCH 2/7 atp: decouple m5readfile from m5ops

From: Adrian Herrera <[email protected]>

By making m5readfile into its own recipe, we avoid modifications to the m5ops recipe when using the meta-atp layer, which break the Yocto compatibility of the layer.

---
 meta-atp/conf/machine/gem5-atp-arm64.conf       |  4 ++--
 .../gem5/gem5-m5ops_20.bbappend                 | 14 --------------
 .../recipes-devtools/gem5/gem5-m5readfile.bb    | 17 +++++++++++++++++
 .../m5-readfile.sh                              |  2 +-
 4 files changed, 20 insertions(+), 17 deletions(-)
 delete mode 100644 meta-atp/recipes-devtools/gem5/gem5-m5ops_20.bbappend
 create mode 100644 meta-atp/recipes-devtools/gem5/gem5-m5readfile.bb
 rename meta-atp/recipes-devtools/gem5/{gem5-m5ops => gem5-m5readfile}/m5-readfile.sh (65%)
meta-atp/conf/machine/gem5-atp-arm64.conf +2 -2
index d5fe22a7..4d0adac8 100644
@@ -5,7 +5,7 @@ MACHINEOVERRIDES =. "gem5-arm64:"
 GEM5_RUN_PROFILE = "configs/baremetal_atp.py"
 # Require m5term
 EXTRA_IMAGEDEPENDS += "gem5-m5term-native"
-# Require ATP kernel modules, user API and gem5 m5ops
-MACHINE_ESSENTIAL_EXTRA_RDEPENDS += "kernel-module-atp atp-uapi gem5-m5ops"
+# Require ATP kernel modules, user API and gem5 m5readfile
+MACHINE_ESSENTIAL_EXTRA_RDEPENDS += "kernel-module-atp atp-uapi gem5-m5readfile"
 # Optionally provide ATP kernel tests
 MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "atp-test"
meta-atp/recipes-devtools/gem5/gem5-m5ops_20.bbappend deleted +0 -14
deleted file mode 100644
index 3ba0c3ca..00000000
@@ -1,14 +0,0 @@
-inherit update-rc.d
-
-FILESEXTRAPATHS:prepend := "${THISDIR}/${BPN}:"
-
-# Add startup script calling m5 readfile for automatic checkpoint and restore
-SRC_URI += "file://m5-readfile.sh"
-
-INITSCRIPT_NAME = "m5-readfile.sh"
-INITSCRIPT_PARAMS = "defaults 99"
-
-do_install:append() {
-    install -d ${D}/${INIT_D_DIR}
-    install -m 755 ${WORKDIR}/m5-readfile.sh ${D}/${INIT_D_DIR}
-}
meta-atp/recipes-devtools/gem5/gem5-m5readfile.bb new file +17 -0
new file mode 100644
index 00000000..9cddc270
@@ -0,0 +1,17 @@
+inherit update-rc.d
+
+SUMMARY = "Enables reading any script at simulation launch time"
+LICENSE = "BSD-3-Clause"
+LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/BSD-3-Clause;md5=550794465ba0ec5312d6919e203a55f9"
+
+SRC_URI = "file://m5-readfile.sh"
+
+INITSCRIPT_NAME = "m5-readfile.sh"
+INITSCRIPT_PARAMS = "defaults 99"
+
+do_install() {
+    install -d ${D}/${INIT_D_DIR}
+    install -m 755 ${WORKDIR}/m5-readfile.sh ${D}/${INIT_D_DIR}
+}
+
+RDEPENDS:${PN} = "gem5-m5ops"
meta-atp/recipes-devtools/gem5/gem5-m5readfile/m5-readfile.sh renamed from meta-atp/recipes-devtools/gem5/gem5-m5ops/m5-readfile.sh +1 -1
similarity index 65%
rename from meta-atp/recipes-devtools/gem5/gem5-m5ops/m5-readfile.sh
rename to meta-atp/recipes-devtools/gem5/gem5-m5readfile/m5-readfile.sh
index 44477e96..edf79b84 100755
@@ -3,7 +3,7 @@
 # Provides:         m5-readfile
 # Required-Start:   $all
 # Default-Start:    5
-# Description:      Enables reading any script at simulation launch time.
+# Description:      Enables reading any script at simulation launch time.
 ### END INIT INFO
 
 m5 readfile | sh
Signature · 1 line
2.17.1
author
#2 lore raw

PATCH 3/7 atp: move m5readfile to meta-gem5

In reply to Jon Mason

From: Adrian Herrera <[email protected]>

The m5 readfile recipe provides general utility for gem5 users to run any script on OS boot. We hence move it to the meta-gem5 layer.

---
 {meta-atp => meta-gem5}/recipes-devtools/gem5/gem5-m5readfile.bb  | 0
 .../recipes-devtools/gem5/gem5-m5readfile/m5-readfile.sh          | 0
 2 files changed, 0 insertions(+), 0 deletions(-)
 rename {meta-atp => meta-gem5}/recipes-devtools/gem5/gem5-m5readfile.bb (100%)
 rename {meta-atp => meta-gem5}/recipes-devtools/gem5/gem5-m5readfile/m5-readfile.sh (100%)
meta-gem5/recipes-devtools/gem5/gem5-m5readfile.bb renamed from meta-atp/recipes-devtools/gem5/gem5-m5readfile.bb +0 -0
similarity index 100%
rename from meta-atp/recipes-devtools/gem5/gem5-m5readfile.bb
rename to meta-gem5/recipes-devtools/gem5/gem5-m5readfile.bb
meta-gem5/recipes-devtools/gem5/gem5-m5readfile/m5-readfile.sh renamed from meta-atp/recipes-devtools/gem5/gem5-m5readfile/m5-readfile.sh +0 -0
similarity index 100%
rename from meta-atp/recipes-devtools/gem5/gem5-m5readfile/m5-readfile.sh
rename to meta-gem5/recipes-devtools/gem5/gem5-m5readfile/m5-readfile.sh
Signature · 1 line
2.17.1
author
#3 lore raw

PATCH 4/7 gem5: support for EXTRAS

In reply to Jon Mason

From: Adrián Herrera Arcila <[email protected]>

External models can be added to gem5 through EXTRAS:
https://www.gem5.org/documentation/general_docs/building/EXTRAS

Added GEM5_EXTRAS to the gem5-native recipe to support it.

---
 meta-gem5/recipes-devtools/gem5/gem5-native.inc | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
meta-gem5/recipes-devtools/gem5/gem5-native.inc +6 -1
index 91a554bb..0f794b3f 100644
@@ -12,10 +12,15 @@ GEM5_BUILD_VARIANT ?= "opt"
 # What gem5 binary are we building
 GEM5_BUILD_CONFIGS ?= "build/X86/gem5.{GEM5_BUILD_VARIANT}"
 
+# Extra directories with sources for gem5 build. Intended to be used from
+# machine configuration files, to add out-of-tree gem5 models of their
+# hardware components.
+GEM5_EXTRAS ?= ""
+
 # Scons build arguments
 GEM5_SCONS_ARGS ?= "CC=${BUILD_CC} CXX=${BUILD_CXX} \
     AS=${BUILD_AS} AR=${BUILD_AR} ${GEM5_BUILD_CONFIGS} \
-    PYTHON_CONFIG=python3-config"
+    PYTHON_CONFIG=python3-config EXTRAS=${GEM5_EXTRAS}"
 
 # Default profile to run
 GEM5_RUN_PROFILE ?= "configs/example/fs.py"
Signature · 1 line
2.17.1
author
#4 lore raw

PATCH 5/7 atp: separate recipe for gem5 models

In reply to Jon Mason

From: Adrián Herrera Arcila <[email protected]>

Previously, meta-atp extended the original gem5 recipe to add the ATP Engine models; the .bbappend was complex, because it pulled from two sources, and it was not possible to make machine-based overrides, as it is a native recipe.

To solve this, we use the recent EXTRAS feature to add the gem5 models from a different recipe.

---
 meta-atp/conf/machine/gem5-atp-arm64.conf     |  4 ++++
 meta-atp/documentation/gem5-atp-arm64.md      |  2 +-
 .../atp/atp-gem5-native_3.1.bb                | 24 +++++++++++++++++++
 .../atp-gem5}/start-gem5-atp.sh               |  0
 .../gem5/gem5-aarch64-native_20.bbappend      | 24 -------------------
 5 files changed, 29 insertions(+), 25 deletions(-)
 create mode 100644 meta-atp/recipes-devtools/atp/atp-gem5-native_3.1.bb
 rename meta-atp/recipes-devtools/{gem5/gem5-aarch64-native => atp/atp-gem5}/start-gem5-atp.sh (100%)
 delete mode 100644 meta-atp/recipes-devtools/gem5/gem5-aarch64-native_20.bbappend
meta-atp/conf/machine/gem5-atp-arm64.conf +4 -0
index 4d0adac8..6e6d49a9 100644
@@ -9,3 +9,7 @@ EXTRA_IMAGEDEPENDS += "gem5-m5term-native"
 MACHINE_ESSENTIAL_EXTRA_RDEPENDS += "kernel-module-atp atp-uapi gem5-m5readfile"
 # Optionally provide ATP kernel tests
 MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS += "atp-test"
+
+# Use ATP Engine gem5 models in gem5 build
+DEPENDS:append:pn-gem5-aarch64-native = " atp-gem5-native"
+GEM5_EXTRAS:pn-gem5-aarch64-native = "${STAGING_DATADIR_NATIVE}/atp"
meta-atp/documentation/gem5-atp-arm64.md +1 -1
index 39f2f3b0..018f374c 100644
@@ -13,7 +13,7 @@ The resulting gem5 native executable contains the AMBA ATP Engine. The resulting
 Users should run the environment as follows:
 
 ```bash
-./tmp/deploy/tools/start-gem5-atp.sh
+oe-run-native atp-gem5-native start-gem5-atp.sh
 ```
 
 This script launches a fast simulation to fast-forward Linux boot. Once Linux boot is completed, the fast simulation switches into a detailed simulation for the final usable environment. Users can connect and interact with the environment as follows:
meta-atp/recipes-devtools/atp/atp-gem5-native_3.1.bb new file +24 -0
new file mode 100644
index 00000000..634c9b11
@@ -0,0 +1,24 @@
+require atp-source_3.1.inc
+inherit native
+
+SUMMARY = "AMBA ATP Engine gem5 models"
+
+S = "${WORKDIR}/git"
+SRC_URI = "${ATP_SRC} file://start-gem5-atp.sh"
+
+do_configure[noexec] = "1"
+do_compile[noexec] = "1"
+
+do_install() {
+    install -d ${D}${datadir}/gem5/configs ${D}${datadir}/atp ${D}${bindir}
+
+    # baremetal_atp.py machine configuration and sample stream.atp file
+    install ${S}/gem5/baremetal_atp.py ${S}/configs/stream.atp ${D}${datadir}/gem5/configs
+    # ATP Engine sources for gem5 to use
+    install ${S}/SConscript ${S}/*.hh ${S}/*.cc ${D}${datadir}/atp
+    cp -RL ${S}/gem5 ${S}/proto ${D}${datadir}/atp
+
+    install ${WORKDIR}/start-gem5-atp.sh ${D}${bindir}
+}
+
+addtask addto_recipe_sysroot after do_populate_sysroot before do_build
meta-atp/recipes-devtools/atp/atp-gem5/start-gem5-atp.sh renamed from meta-atp/recipes-devtools/gem5/gem5-aarch64-native/start-gem5-atp.sh +0 -0
similarity index 100%
rename from meta-atp/recipes-devtools/gem5/gem5-aarch64-native/start-gem5-atp.sh
rename to meta-atp/recipes-devtools/atp/atp-gem5/start-gem5-atp.sh
meta-atp/recipes-devtools/gem5/gem5-aarch64-native_20.bbappend deleted +0 -24
deleted file mode 100644
index 6607f0f8..00000000
@@ -1,24 +0,0 @@
-require recipes-devtools/atp/atp-source_3.1.inc
-
-FILESEXTRAPATHS:prepend := "${THISDIR}/${BPN}:"
-
-SRC_URI += "${ATP_SRC};destsuffix=git/atp;name=atp \
-            file://start-gem5-atp.sh"
-SRCREV_FORMAT = "gem5_atp"
-SRCREV_atp = "${ATP_REV}"
-LICENSE += "& ${ATP_LIC}"
-LIC_FILES_CHKSUM += "file://atp/LICENSE;md5=${ATP_LIC_MD5}"
-
-EXTRA_OESCONS += "EXTRAS=${S}/atp"
-
-do_install:append() {
-    # baremetal_atp.py machine configuration and sample stream.atp file
-    install -m 644 ${B}/atp/gem5/baremetal_atp.py \
-                   ${B}/atp/configs/stream.atp \
-                   ${D}${datadir}/gem5/configs
-}
-
-do_deploy:append() {
-    # start-gem5-atp.sh launch script
-    install -m 755 ${WORKDIR}/start-gem5-atp.sh ${DEPLOYDIR}
-}
Signature · 1 line
2.17.1
author
#5 lore raw

PATCH 6/7 atp: fix machine overrides in recipes

In reply to Jon Mason

From: Adrián Herrera Arcila <[email protected]>

Ensure meta-atp recipes are only performed if a compatible machine is selected.

---
 meta-atp/recipes-devtools/gem5/gem5-aarch64-dtb.bbappend | 8 +++++---
 meta-atp/recipes-kernel/atp/atp-module_3.1.bb            | 1 +
 meta-atp/recipes-kernel/atp/atp-test_3.1.bb              | 1 +
 meta-atp/recipes-kernel/atp/atp-uapi_3.1.bb              | 1 +
 meta-atp/recipes-kernel/linux/linux-yocto_%.bbappend     | 4 ++--
 5 files changed, 10 insertions(+), 5 deletions(-)
meta-atp/recipes-devtools/gem5/gem5-aarch64-dtb.bbappend +5 -3
index 2b55b892..c96f2cdc 100644
@@ -1,3 +1,5 @@
-# Export datadir paths for baremetal_atp.py script
-export GEM5_DATADIR = "${STAGING_DATADIR_NATIVE}/gem5"
-export ATP_DATADIR = "${STAGING_DATADIR_NATIVE}/gem5"
+do_compile:prepend:gem5-atp-arm64() {
+    # Export datadir paths for baremetal_atp.py script
+    export GEM5_DATADIR="${STAGING_DATADIR_NATIVE}/gem5"
+    export ATP_DATADIR="${STAGING_DATADIR_NATIVE}/gem5"
+}
meta-atp/recipes-kernel/atp/atp-module_3.1.bb +1 -0
index 0bf4949b..9b54e1c8 100644
@@ -1,3 +1,4 @@
+COMPATIBLE_MACHINE = "gem5-atp-arm64"
 require recipes-devtools/atp/atp-source_3.1.inc
 inherit module
 
meta-atp/recipes-kernel/atp/atp-test_3.1.bb +1 -0
index e98e13cd..3c88e08d 100644
@@ -1,3 +1,4 @@
+COMPATIBLE_MACHINE = "gem5-atp-arm64"
 require recipes-devtools/atp/atp-source_3.1.inc
 
 SUMMARY = "End-to-end tests evaluating ATP kernel modules service correctness"
meta-atp/recipes-kernel/atp/atp-uapi_3.1.bb +1 -0
index 140105f8..a8b14796 100644
@@ -1,3 +1,4 @@
+COMPATIBLE_MACHINE = "gem5-atp-arm64"
 require recipes-devtools/atp/atp-source_3.1.inc
 
 SUMMARY = "User API for accessing services from ATP kernel modules"
meta-atp/recipes-kernel/linux/linux-yocto_%.bbappend +2 -2
index f59f8d44..8cb86a97 100644
@@ -1,2 +1,2 @@
-FILESEXTRAPATHS:prepend := "${THISDIR}/files:"
-SRC_URI += "file://no_ftrace.cfg file://smmuv3.cfg"
+FILESEXTRAPATHS:prepend:gem5-atp-arm64 := "${THISDIR}/files:"
+SRC_URI:append:gem5-atp-arm64 = " file://no_ftrace.cfg file://smmuv3.cfg"
Signature · 1 line
2.17.1
author
#6 lore raw

PATCH 7/7 ci: add meta-atp to check-layers

In reply to Jon Mason

From: Adrián Herrera Arcila <[email protected]>

---
 .gitlab-ci.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
.gitlab-ci.yml +1 -1
index 7d050a55..6bb42e85 100644
@@ -210,7 +210,7 @@ check-layers:
       "yocto-check-layer-wrapper $CI_PROJECT_DIR/$LAYER --dependency $CI_PROJECT_DIR/meta-* $KAS_WORK_DIR/meta-openembedded/meta-oe --no-auto-dependency"
   parallel:
     matrix:
-      - LAYER: [meta-arm, meta-arm-bsp, meta-arm-toolchain, meta-gem5]
+      - LAYER: [meta-arm, meta-arm-bsp, meta-arm-toolchain, meta-gem5, meta-atp]
 
 pending-updates:
   extends: .setup
Signature · 1 line
2.17.1
Applied
#7 lore raw
In reply to Jon Mason
Quoted 7 lines · Jon Mason

Test failed because there was no reference to maintainers in the meta-atp README.

Following the common structure of other layers in the meta-arm repository, the README in meta-atp now refers to the top-level README, and a documentation directory contains the guidance that was present in the original meta-atp README.

Applied, thanks!

[1/7] atp: fix failing test_readme

      commit: ffd0aca508deb35cb6f61a1f57b7262d9dabd307
[2/7] atp: decouple m5readfile from m5ops
      commit: 3786d6d0162f1b87e085690940dc23628cfe8ee8
[3/7] atp: move m5readfile to meta-gem5
      commit: a68a2a206eeb1427b4217b8831439cbe56341a14
[4/7] gem5: support for EXTRAS
      commit: 29fa253f002f9808871b1bd3b4696e2ff3b055b5
[5/7] atp: separate recipe for gem5 models
      commit: 8023f902ac0fd3cbcdf0cdc146a8f7e594497b83
[6/7] atp: fix machine overrides in recipes
      commit: e695d0ee252e29ba3aa129012b26bfac91d223f8
[7/7] ci: add meta-atp to check-layers
      commit: 9699e6698e1e4de2b4b76c076a8ffc585ebab100

Best regards,

Signature · 1 line
Jon Mason <[email protected]>