Skip to contentSkip to conversation

lkp: add initial opensuse support

PATCH In review Luis R. Rodriguez 3 participants 18 messages Jan 6, 2017 – Feb 21, 2017 Library code

lore
Share & tools
lore

Luis R. Rodriguez

author
#0 lore raw

cover lkp: add initial opensuse support

I received a failed test results notice from 0-day LKP test suite using trinity, however it was pretty hard to get reproduce given the LKP did not have OpenSUSE support to start off with, but the second issue I've stubmled upon is beyond me -- running either:

./bin/lkp qemu -k ~mcgrof/linux-next/arch/x86/boot/bzImage job-script

or

lkp qemu jobs/trinity.yaml

ends up creating calling a few subroutines or files which are clearly not present, then tries to create a cpio archive but clearly there are a few files which LKP tree expects you to have and are not present such as lkp and lkp/scheduled.

This is a run on OpenSUSE, first we install the dependencies for trinity.yaml deps:

 # lkp install jobs/trinity.yaml

Then we try to run it:

 # lkp qemu jobs/trinity.yaml 
jobs/trinity.yaml: line 1: suite:: command not found
jobs/trinity.yaml: line 2: testcase:: command not found
jobs/trinity.yaml: line 4: runtime:: command not found
jobs/trinity.yaml: line 6: kmsg:: command not found
jobs/trinity.yaml: line 7: rootfs:: command not found
jobs/trinity.yaml: line 9: trinity:: command not found
make: Entering directory '/home/mcgrof/devel/lkp-tests/bin/event'
klcc    -c -o wakeup.o wakeup.c
klcc  -static -o wakeup wakeup.o
rm -f wakeup.o
strip wakeup
make: Leaving directory '/home/mcgrof/devel/lkp-tests/bin/event'
cpio: lkp: Cannot stat: No such file or directory
cpio: lkp/scheduled: Cannot stat: No such file or directory
cpio: blank line ignored
cpio: File /dev/mqueue/���� shrunk by 20 bytes, padding with zeros

This never complets, it just sits there.

Either way these patches help start to add OpenSUSE support.

Luis R. Rodriguez (4):
lib/install.sh: fix shell adapt_packages()
distro: add initial opensuse support lib/install.*: make package dependency list uniq .gitignore: add bin/event/wakeup

 .gitignore                  |  1 +
 README.md                   | 22 ++++++++++++++++++++++
 distro/adaptation/archlinux |  2 +-
 distro/adaptation/fedora    |  2 +-
 distro/adaptation/opensuse  | 14 ++++++++++++++
 distro/depends/lkp-dev      |  4 ++++
 distro/installer/opensuse   |  3 +++
 lib/install.rb              |  2 +-
 lib/install.sh              | 14 +++++++++++---
 9 files changed, 58 insertions(+), 6 deletions(-)
 create mode 100644 distro/adaptation/opensuse
 create mode 100755 distro/installer/opensuse
Signature · 1 line
2.11.0
author
#1 lore raw

PATCH 1/4 lib/install.sh: fix shell adapt_packages()

There are only a few packages with architecture specific annotations right now, they all use a :i386 postfix. The current ruby adapt_packages() works well, however on shell, the distro_pkg=${mapping#$pkg:} does not work as well as Ruby's array use and find. Furthermore the bash implementation allows a final grep to end up giving you more than one package, so if you have libc-dev but a distro mapping file has a mapping for both libc-dev and libc-dev:i386 we'd end up with both mappings. Lastly the mapping for architectures simply does not work...

Fix this for now by modifying the shell implementation of adapt_packages() to check if a package we are going to query is architecture specific or not, and as a general rule for now just require non-arch packages to go first, while arch packages go last. The awk query is much better and will do what we expected.

Long term we should just modify the distro/adaptation/* files to use a delimiter which can never be part of a package name, for instance using the pipe "|". That would allow us to get rid of this small cruft on the bash implementation and enable both bash and ruby implementations to match much closely.

Since we are sticking to a rule for now of having the arch packages go last modify the few archs which do have these mappings to match this rule.

Signed-off-by Luis R. Rodriguez <[email protected]>
---
 distro/adaptation/archlinux |  2 +-
 distro/adaptation/fedora    |  2 +-
 distro/depends/lkp-dev      |  4 ++++
 lib/install.sh              | 10 ++++++++--
 4 files changed, 14 insertions(+), 4 deletions(-)
distro/adaptation/archlinux +1 -1
index dd8b4ab5967f..0a4ecd5c6d4d 100644
@@ -66,6 +66,6 @@ xfslibs-dev: xfsprogs
 libattr1-dev: attr
 libacl1-dev: acl
 libc-dev:i386:
-linux-libc-dev:i386:
 linux-libc-dev:
+linux-libc-dev:i386:
 libklibc-dev:
distro/adaptation/fedora +1 -1
index a56ed9d0b870..b3ad3fbcdf20 100644
@@ -69,6 +69,6 @@ libreadline5:
 libtool-bin:
 libnuma-dev: numactl-devel
 libc-dev:i386:
-linux-libc-dev:i386:
 linux-libc-dev:
+linux-libc-dev:i386:
 libklibc-dev:
distro/depends/lkp-dev +4 -0
index 70f3c731787e..f3f27f73a965 100644
@@ -1,3 +1,7 @@
+# Put architecture packages after the non architecture specific
+# package. We can change this requiremet if we modify
+# distro/adaptation/* files to use something uniq that is not
+# part of real packages, like: |
 make
 automake
 gcc
lib/install.sh +8 -2
index d4f8c1feab37..0957f13c9937 100755
@@ -33,9 +33,15 @@ adapt_packages()
 
 	for pkg in $generic_packages
 	do
-		local mapping="$(grep "^$pkg:" $distro_file)"
+		local mapping=""
+		local is_arch_dep="$(echo $pkg | grep ":")"
+		if [ -n "$is_arch_dep" ]; then
+			mapping="$(grep "^$pkg:" $distro_file | tail -1)"
+		else
+			mapping="$(grep "^$pkg:" $distro_file | head -1)"
+		fi
 		if [ -n "$mapping" ]; then
-			distro_pkg=${mapping#$pkg:}
+			distro_pkg=$(echo $mapping | awk -F": " '{print $2}')
 			[ -n "$distro_pkg" ] && echo $distro_pkg
 		else
 			echo $pkg
Signature · 1 line
2.11.0
Reviewed-by on v1 1/4
#2 lore raw

Re: [PATCH 1/4] lib/install.sh: fix shell adapt_packages()

Reviewed-by Ye Xiaolong <[email protected]>

Thanks,
Xiaolong

Quoted 97 lines · Luis R. Rodriguez

There are only a few packages with architecture specific annotations right now, they all use a :i386 postfix. The current ruby adapt_packages() works well, however on shell, the distro_pkg=${mapping#$pkg:} does not work as well as Ruby's array use and find. Furthermore the bash implementation allows a final grep to end up giving you more than one package, so if you have libc-dev but a distro mapping file has a mapping for both libc-dev and libc-dev:i386 we'd end up with both mappings. Lastly the mapping for architectures simply does not work...

Fix this for now by modifying the shell implementation of adapt_packages() to check if a package we are going to query is architecture specific or not, and as a general rule for now just require non-arch packages to go first, while arch packages go last. The awk query is much better and will do what we expected.

Long term we should just modify the distro/adaptation/* files to use a delimiter which can never be part of a package name, for instance using the pipe "|". That would allow us to get rid of this small cruft on the bash implementation and enable both bash and ruby implementations to match much closely.

Since we are sticking to a rule for now of having the arch packages go last modify the few archs which do have these mappings to match this rule.

Signed-off-by Luis R. Rodriguez <[email protected]>

---

distro/adaptation/archlinux |  2 +-
distro/adaptation/fedora    |  2 +-
distro/depends/lkp-dev      |  4 ++++
lib/install.sh              | 10 ++++++++--
4 files changed, 14 insertions(+), 4 deletions(-)
distro/adaptation/archlinux +0 -0
index dd8b4ab5967f..0a4ecd5c6d4d 100644
@@ -66,6 +66,6 @@ xfslibs-dev: xfsprogs

libattr1-dev: attr
libacl1-dev: acl
libc-dev:i386:
-linux-libc-dev:i386:
linux-libc-dev:
+linux-libc-dev:i386:
libklibc-dev:

distro/adaptation/fedora +0 -0
index a56ed9d0b870..b3ad3fbcdf20 100644
@@ -69,6 +69,6 @@ libreadline5:

libtool-bin:
libnuma-dev: numactl-devel
libc-dev:i386:
-linux-libc-dev:i386:
linux-libc-dev:
+linux-libc-dev:i386:
libklibc-dev:

distro/depends/lkp-dev +4 -0
index 70f3c731787e..f3f27f73a965 100644
@@ -1,3 +1,7 @@
+# Put architecture packages after the non architecture specific
+# package. We can change this requiremet if we modify
+# distro/adaptation/* files to use something uniq that is not
+# part of real packages, like: |

make
automake
gcc

lib/install.sh +8 -2
index d4f8c1feab37..0957f13c9937 100755
@@ -33,9 +33,15 @@ adapt_packages()
 
 	for pkg in $generic_packages
 	do
-		local mapping="$(grep "^$pkg:" $distro_file)"
+		local mapping=""
+		local is_arch_dep="$(echo $pkg | grep ":")"
+		if [ -n "$is_arch_dep" ]; then
+			mapping="$(grep "^$pkg:" $distro_file | tail -1)"
+		else
+			mapping="$(grep "^$pkg:" $distro_file | head -1)"
+		fi
 		if [ -n "$mapping" ]; then
-			distro_pkg=${mapping#$pkg:}
+			distro_pkg=$(echo $mapping | awk -F": " '{print $2}')
 			[ -n "$distro_pkg" ] && echo $distro_pkg
 		else
 			echo $pkg
Signature · 1 line
2.11.0
author
#3 lore raw

PATCH 2/4 distro: add initial opensuse support

In reply to Luis R. Rodriguez

Tested with:

export LKP_SRC=$PWD
export PATH=$PATH:$LKP_SRC/bin
lkp install jobs/trinity.yaml

Signed-off-by Luis R. Rodriguez <[email protected]>
---
 README.md                  | 22 ++++++++++++++++++++++
 distro/adaptation/opensuse | 14 ++++++++++++++
 distro/installer/opensuse  |  3 +++
 lib/install.sh             |  2 ++
 4 files changed, 41 insertions(+)
 create mode 100644 distro/adaptation/opensuse
 create mode 100755 distro/installer/opensuse
README.md +22 -0
index 8031c867c0af..b45a3f6bb91f 100644
@@ -50,4 +50,26 @@ Most test cases should install/run well in
 - Ubuntu 14.04
 - Archlinux
 
+There is however some initial support for:
+
+- OpenSUSE:
+	- jobs/trinity.yaml
+- Fedora
+
 As for now, lkp-tests still needs to run as root.
+
+## Adding distribution support
+
+If you want to add support for your Linux distribution you will need
+an installer file which allows us to install dependencies per job. For
+examples look at: distro/installer/* files.
+
+Since packages can have different names we provide an adaptation mapping for a
+base Ubuntu package (since development started with that) to your own
+distribution package name, for example adaptation files see:
+distro/adaptation/*. For now adaptation files must have the architecture
+dependent packages (ie, that ends with the postfix :i386) towards the end
+of the adaptation file.
+
+You will also want to add a case for your distribution on sync_distro_sources()
+on the file lib/install.sh.
distro/adaptation/opensuse new file +14 -0
new file mode 100644
index 000000000000..c4543bf92b66
@@ -0,0 +1,14 @@
+# Go get it yourself:
+# https://github.com/schacon/ruby-git
+# We have no package for it yet
+ruby-git:
+linux-libc-dev: glibc-devel
+linux-perf: perf
+build-essential:
+libc6-dev: glibc-devel
+libc6-dev:i386: glibc-devel-32bit
+libc-dev: glibc-devel
+libc-dev:i386: glibc-devel-32bit
+linux-libc-dev: glibc-devel
+linux-libc-dev:i386: glibc-devel-32bit
+libklibc-dev:
distro/installer/opensuse new file +3 -0
new file mode 100755
index 000000000000..a909f3e39c05
@@ -0,0 +1,3 @@
+#!/bin/bash
+
+sudo zypper install -y $*
lib/install.sh +2 -0
index 0957f13c9937..0b090f8b82bf 100755
@@ -17,6 +17,8 @@ sync_distro_sources()
 			yum update
 		fi ;;
 	archlinux) yaourt -Sy ;;
+	opensuse)
+		zypper update ;;
 	*) echo "Not support $distro to do update" ;;
 	esac
 }
Signature · 1 line
2.11.0
Reviewed-by on v1 2/4
#4 lore raw

Re: [PATCH 2/4] distro: add initial opensuse support

Reviewed-by Ye Xiaolong <[email protected]>

Thanks,
Xiaolong

Quoted 92 lines · Luis R. Rodriguez

Tested with:

export LKP_SRC=$PWD
export PATH=$PATH:$LKP_SRC/bin
lkp install jobs/trinity.yaml

Signed-off-by Luis R. Rodriguez <[email protected]>

---

README.md                  | 22 ++++++++++++++++++++++
distro/adaptation/opensuse | 14 ++++++++++++++
distro/installer/opensuse  |  3 +++
lib/install.sh             |  2 ++
4 files changed, 41 insertions(+)

create mode 100644 distro/adaptation/opensuse
create mode 100755 distro/installer/opensuse

README.md +6 -2
index 8031c867c0af..b45a3f6bb91f 100644
@@ -50,4 +50,26 @@ Most test cases should install/run well in
- Ubuntu 14.04
- Archlinux
 
+There is however some initial support for:
+
+- OpenSUSE:
+	- jobs/trinity.yaml
+- Fedora
+

As for now, lkp-tests still needs to run as root.

README.md +16 -0
+
+## Adding distribution support
+
+If you want to add support for your Linux distribution you will need
+an installer file which allows us to install dependencies per job. For
+examples look at: distro/installer/* files.
+
+Since packages can have different names we provide an adaptation mapping for a
+base Ubuntu package (since development started with that) to your own
+distribution package name, for example adaptation files see:
+distro/adaptation/*. For now adaptation files must have the architecture
+dependent packages (ie, that ends with the postfix :i386) towards the end
+of the adaptation file.
+
+You will also want to add a case for your distribution on sync_distro_sources()
+on the file lib/install.sh.
distro/adaptation/opensuse new file +14 -0
new file mode 100644
index 000000000000..c4543bf92b66
@@ -0,0 +1,14 @@
+# Go get it yourself:
+# https://github.com/schacon/ruby-git
+# We have no package for it yet
+ruby-git:
+linux-libc-dev: glibc-devel
+linux-perf: perf
+build-essential:
+libc6-dev: glibc-devel
+libc6-dev:i386: glibc-devel-32bit
+libc-dev: glibc-devel
+libc-dev:i386: glibc-devel-32bit
+linux-libc-dev: glibc-devel
+linux-libc-dev:i386: glibc-devel-32bit
+libklibc-dev:
distro/installer/opensuse new file +3 -0
new file mode 100755
index 000000000000..a909f3e39c05
@@ -0,0 +1,3 @@
+#!/bin/bash
+
+sudo zypper install -y $*
lib/install.sh +2 -0
index 0957f13c9937..0b090f8b82bf 100755
@@ -17,6 +17,8 @@ sync_distro_sources()
 			yum update
 		fi ;;
 	archlinux) yaourt -Sy ;;
+	opensuse)
+		zypper update ;;
 	*) echo "Not support $distro to do update" ;;
 	esac
 }
Signature · 1 line
2.11.0
author
#5 lore raw

PATCH 3/4 lib/install.*: make package dependency list uniq

In reply to Luis R. Rodriguez

This provides a small optimization to reduce the list of package dependencies we give back to the distribution installer by making the list uniq. Since bash is rather stupid we need to first sort the list prior to passing it to sort.

Signed-off-by Luis R. Rodriguez <[email protected]>
---
 lib/install.rb | 2 +-
 lib/install.sh | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
lib/install.rb +1 -1
index 63b11ad1a5b2..8f40e3e902bc 100755
@@ -31,5 +31,5 @@ def get_dependency_packages(distro, script)
 
 	packages = adapt_packages(distro, generic_packages)
 
-	return packages.flatten.compact
+	return packages.flatten.compact.uniq
 end
lib/install.sh +1 -1
index 0b090f8b82bf..cb9a8d0ea530 100755
@@ -61,5 +61,5 @@ get_dependency_packages()
 
 	local generic_packages="$(sed 's/#.*//' "$base_file")"
 
-	adapt_packages
+	adapt_packages | sort | uniq
 }
Signature · 1 line
2.11.0
Reviewed-by on v1 3/4
#6 lore raw

Re: [PATCH 3/4] lib/install.*: make package dependency list uniq

Reviewed-by Ye Xiaolong <[email protected]>

Thanks,
Xiaolong

Quoted 36 lines · Luis R. Rodriguez

This provides a small optimization to reduce the list of package dependencies we give back to the distribution installer by making the list uniq. Since bash is rather stupid we need to first sort the list prior to passing it to sort.

Signed-off-by Luis R. Rodriguez <[email protected]>

---
lib/install.rb | 2 +-
lib/install.sh | 2 +-

2 files changed, 2 insertions(+), 2 deletions(-)
lib/install.rb +1 -1
index 63b11ad1a5b2..8f40e3e902bc 100755
@@ -31,5 +31,5 @@ def get_dependency_packages(distro, script)
 
 	packages = adapt_packages(distro, generic_packages)
 
-	return packages.flatten.compact
+	return packages.flatten.compact.uniq

end

lib/install.sh +1 -1
index 0b090f8b82bf..cb9a8d0ea530 100755
@@ -61,5 +61,5 @@ get_dependency_packages()
 
 	local generic_packages="$(sed 's/#.*//' "$base_file")"
 
-	adapt_packages
+	adapt_packages | sort | uniq
 }
Signature · 1 line
2.11.0
Reviewed-by on v1 4/4
#8 lore raw

Re: [PATCH 4/4] .gitignore: add bin/event/wakeup

Reviewed-by Ye Xiaolong <[email protected]>

Thanks,
Xiaolong

Quoted 17 lines · Luis R. Rodriguez
Signed-off-by Luis R. Rodriguez <[email protected]>

---
.gitignore | 1 +

1 file changed, 1 insertion(+)
.gitignore +0 -0
index 56c20bff3d8f..d08002fb70fa 100644
@@ -2,3 +2,4 @@ update

deploy
Gemfile
Gemfile.lock
+bin/event/wakeup

Signature · 1 line
2.11.0
author
#9 lore raw
In reply to Luis R. Rodriguez
Quoted 41 lines · Luis R. Rodriguez

I received a failed test results notice from 0-day LKP test suite using trinity, however it was pretty hard to get reproduce given the LKP did not have OpenSUSE support to start off with, but the second issue I've stubmled upon is beyond me -- running either:

./bin/lkp qemu -k ~mcgrof/linux-next/arch/x86/boot/bzImage job-script

or

lkp qemu jobs/trinity.yaml

ends up creating calling a few subroutines or files which are clearly not present, then tries to create a cpio archive but clearly there are a few files which LKP tree expects you to have and are not present such as lkp and lkp/scheduled.

This is a run on OpenSUSE, first we install the dependencies for trinity.yaml deps:

 # lkp install jobs/trinity.yaml

Then we try to run it:

 # lkp qemu jobs/trinity.yaml
jobs/trinity.yaml: line 1: suite:: command not found
jobs/trinity.yaml: line 2: testcase:: command not found
jobs/trinity.yaml: line 4: runtime:: command not found
jobs/trinity.yaml: line 6: kmsg:: command not found
jobs/trinity.yaml: line 7: rootfs:: command not found
jobs/trinity.yaml: line 9: trinity:: command not found
make: Entering directory '/home/mcgrof/devel/lkp-tests/bin/event'
klcc    -c -o wakeup.o wakeup.c
klcc  -static -o wakeup wakeup.o
rm -f wakeup.o
strip wakeup
make: Leaving directory '/home/mcgrof/devel/lkp-tests/bin/event'
cpio: lkp: Cannot stat: No such file or directory
cpio: lkp/scheduled: Cannot stat: No such file or directory
cpio: blank line ignored
cpio: File /dev/mqueue/���� shrunk by 20 bytes, padding with zeros
Quoted 1 line

This never complets, it just sits there.

I just tried running all this on Debian and I run into the same exact issue so as I suspected, this is not a distribution problem but rather a bug in the code or some missing files in the repository:

# ./bin/lkp qemu -k ~mcgrof/linux-next/arch/x86/boot/bzImage job-script
or
# ./bin/lkp qemu -k ~mcgrof/linux-next/arch/x86/boot/bzImage jobs/trinity.yaml

yields:

The approx. disk space requirements are

10M             simple boot test in rootfs openwrt
50M             simple boot test in rootfs debian
1G              plan to run a number of different tests
100G or more    IO tests

Please enter a dir with enough disk space, or simply press Enter to accept the default.
You may still symlink /root/.lkp to a more suitable place in future. /root/.lkp =>
job-script: line 2: $'\r': command not found
job-script: line 3: syntax error near unexpected token $'\r'' 'ob-script: line 3: export_top_env()
make: Entering directory '/home/mcgrof/devel/lkp-tests/bin/event' klcc -static -o wakeup wakeup.o
rm -f wakeup.o
strip wakeup
make: Leaving directory '/home/mcgrof/devel/lkp-tests/bin/event'
cpio: root:lkp: invalid group
cpio: root:lkp: invalid group
gzip: /root/.lkp/cache/lkp-x86_64.cpio: No such file or directory
mv: cannot stat '/root/.lkp/cache/lkp-x86_64.cpio.gz': No such file or directory
mv: cannot stat '/root/.lkp/cache/lkp-x86_64.cgz': No such file or directory
cpio: lkp: Cannot stat: No such file or directory
cpio: lkp/scheduled: Cannot stat: No such file or directory
cpio: blank line ignored

And it just sits there without end.

Luis

#10 lore raw
In reply to Luis R. Rodriguez

Hi, Luis

Quoted 16 lines · Luis R. Rodriguez

I received a failed test results notice from 0-day LKP test suite using trinity, however it was pretty hard to get reproduce given the LKP did not have OpenSUSE support to start off with, but the second issue I've stubmled upon is beyond me -- running either:

./bin/lkp qemu -k ~mcgrof/linux-next/arch/x86/boot/bzImage job-script

or

lkp qemu jobs/trinity.yaml

ends up creating calling a few subroutines or files which are clearly not present, then tries to create a cpio archive but clearly there are a few files which LKP tree expects you to have and are not present such as lkp and lkp/scheduled.

Quoted 8 lines

This is a run on OpenSUSE, first we install the dependencies for trinity.yaml deps:

# lkp install jobs/trinity.yaml

Then we try to run it:

# lkp qemu jobs/trinity.yaml

Actually we don't use yaml file for lkp qemu, you could try "lkp run jobs/trinity.yaml" instead.

Thanks,
Xiaolong

Quoted 43 lines

jobs/trinity.yaml: line 1: suite:: command not found jobs/trinity.yaml: line 2: testcase:: command not found jobs/trinity.yaml: line 4: runtime:: command not found jobs/trinity.yaml: line 6: kmsg:: command not found jobs/trinity.yaml: line 7: rootfs:: command not found jobs/trinity.yaml: line 9: trinity:: command not found
make: Entering directory '/home/mcgrof/devel/lkp-tests/bin/event'
klcc -c -o wakeup.o wakeup.c
klcc -static -o wakeup wakeup.o
rm -f wakeup.o
strip wakeup
make: Leaving directory '/home/mcgrof/devel/lkp-tests/bin/event'
cpio: lkp: Cannot stat: No such file or directory
cpio: lkp/scheduled: Cannot stat: No such file or directory
cpio: blank line ignored
cpio: File /dev/mqueue/���� shrunk by 20 bytes, padding with zeros

This never complets, it just sits there.

Either way these patches help start to add OpenSUSE support.

Luis R. Rodriguez (4):
lib/install.sh: fix shell adapt_packages()
distro: add initial opensuse support
lib/install.*: make package dependency list uniq
.gitignore: add bin/event/wakeup

.gitignore                  |  1 +
README.md                   | 22 ++++++++++++++++++++++
distro/adaptation/archlinux |  2 +-
distro/adaptation/fedora    |  2 +-
distro/adaptation/opensuse  | 14 ++++++++++++++
distro/depends/lkp-dev      |  4 ++++
distro/installer/opensuse   |  3 +++
lib/install.rb              |  2 +-
lib/install.sh              | 14 +++++++++++---
9 files changed, 58 insertions(+), 6 deletions(-)

create mode 100644 distro/adaptation/opensuse
create mode 100755 distro/installer/opensuse

Signature · 1 line
2.11.0
author
#11 lore raw
Quoted 22 lines · Ye Xiaolong

Hi, Luis

Quoted 19 lines · Luis R. Rodriguez

I received a failed test results notice from 0-day LKP test suite using trinity, however it was pretty hard to get reproduce given the LKP did not have OpenSUSE support to start off with, but the second issue I've stubmled upon is beyond me -- running either:

./bin/lkp qemu -k ~mcgrof/linux-next/arch/x86/boot/bzImage job-script

or

lkp qemu jobs/trinity.yaml

ends up creating calling a few subroutines or files which are clearly not present, then tries to create a cpio archive but clearly there are a few files which LKP tree expects you to have and are not present such as lkp and lkp/scheduled.

This is a run on OpenSUSE, first we install the dependencies for trinity.yaml deps:

Quoted 8 lines
Quoted 5 lines
# lkp install jobs/trinity.yaml

Then we try to run it:

# lkp qemu jobs/trinity.yaml

Actually we don't use yaml file for lkp qemu, you could try "lkp run jobs/trinity.yaml" instead.

OK that runs, but that beats my actual machine with trinity, which is not desirable. I am trying to reproduce a reported error caught by lpk tests with trinity on my tree and that was with qemu. I have the job-script but that produces the same error with missing files. Can you help me with that?

Luis

#12 lore raw
Quoted 29 lines · Luis R. Rodriguez
Quoted 28 lines · Ye Xiaolong

Hi, Luis

Quoted 24 lines · Luis R. Rodriguez

I received a failed test results notice from 0-day LKP test suite using trinity, however it was pretty hard to get reproduce given the LKP did not have OpenSUSE support to start off with, but the second issue I've stubmled upon is beyond me -- running either:

./bin/lkp qemu -k ~mcgrof/linux-next/arch/x86/boot/bzImage job-script

or

lkp qemu jobs/trinity.yaml

ends up creating calling a few subroutines or files which are clearly not present, then tries to create a cpio archive but clearly there are a few files which LKP tree expects you to have and are not present such as lkp and lkp/scheduled.

This is a run on OpenSUSE, first we install the dependencies for trinity.yaml deps:

# lkp install jobs/trinity.yaml

Then we try to run it:

# lkp qemu jobs/trinity.yaml
Quoted 7 lines
Quoted 2 lines

Actually we don't use yaml file for lkp qemu, you could try "lkp run jobs/trinity.yaml" instead.

OK that runs, but that beats my actual machine with trinity, which is not desirable. I am trying to reproduce a reported error caught by lpk tests with trinity on my tree and that was with qemu. I have the job-script but that produces the same error with missing files. Can you help me with that?

Sure, could you paste the exact error log when you tried "./bin/lkp qemu -k ~mcgrof/linux-next/arch/x86/boot/bzImage job-script"?

Thanks,
Xiaolong

Quoted 2 lines

Luis

author
#13 lore raw
Quoted 11 lines · Ye Xiaolong
Quoted 10 lines · Luis R. Rodriguez
Quoted 9 lines · Ye Xiaolong

Hi, Luis

Quoted 6 lines · Luis R. Rodriguez

I received a failed test results notice from 0-day LKP test suite using trinity, however it was pretty hard to get reproduce given the LKP did not have OpenSUSE support to start off with, but the second issue I've stubmled upon is beyond me -- running either:

./bin/lkp qemu -k ~mcgrof/linux-next/arch/x86/boot/bzImage job-script

Note I had run the same command.

Quoted 22 lines
Quoted 22 lines
Quoted 21 lines
Quoted 10 lines

or

lkp qemu jobs/trinity.yaml

ends up creating calling a few subroutines or files which are clearly not present, then tries to create a cpio archive but clearly there are a few files which LKP tree expects you to have and are not present such as lkp and lkp/scheduled.

Quoted 8 lines

This is a run on OpenSUSE, first we install the dependencies for trinity.yaml deps:

# lkp install jobs/trinity.yaml

Then we try to run it:

# lkp qemu jobs/trinity.yaml

Actually we don't use yaml file for lkp qemu, you could try "lkp run jobs/trinity.yaml" instead.

Quoted 7 lines
Quoted 4 lines

OK that runs, but that beats my actual machine with trinity, which is not desirable. I am trying to reproduce a reported error caught by lpk tests with trinity on my tree and that was with qemu. I have the job-script but that produces the same error with missing files. Can you help me with that?

Sure, could you paste the exact error log when you tried "./bin/lkp qemu -k ~mcgrof/linux-next/arch/x86/boot/bzImage job-script"?

As I noted its the same exact output, but here it is again:

ergon:/home/mcgrof/devel/lkp-tests # export LKP_SRC=$PWD ergon:/home/mcgrof/devel/lkp-tests # export PATH=$PATH:$LKP_SRC/bin ergon:/home/mcgrof/devel/lkp-tests # ./bin/lkp qemu -k ~mcgrof/linux-next/arch/x86/boot/bzImage job-script
job-script: line 2: $'\r': command not found
job-script: line 3: syntax error near unexpected token `$'\r''
'ob-script: line 3: `export_top_env()
make: Entering directory '/home/mcgrof/devel/lkp-tests/bin/event'
klcc -c -o wakeup.o wakeup.c
klcc -static -o wakeup wakeup.o
rm -f wakeup.o
strip wakeup
make: Leaving directory '/home/mcgrof/devel/lkp-tests/bin/event'
cpio: lkp: Cannot stat: No such file or directory
cpio: lkp/scheduled: Cannot stat: No such file or directory
cpio: blank line ignored
cpio: File /dev/mqueue/���� shrunk by 20 bytes, padding with zeros

Luis

#14 lore raw

On 01/10, Luis R. Rodriguez wrote:

[snip]

Quoted 27 lines
Quoted 7 lines
Quoted 4 lines

OK that runs, but that beats my actual machine with trinity, which is not desirable. I am trying to reproduce a reported error caught by lpk tests with trinity on my tree and that was with qemu. I have the job-script but that produces the same error with missing files. Can you help me with that?

Sure, could you paste the exact error log when you tried "./bin/lkp qemu -k ~mcgrof/linux-next/arch/x86/boot/bzImage job-script"?

As I noted its the same exact output, but here it is again:

ergon:/home/mcgrof/devel/lkp-tests # export LKP_SRC=$PWD ergon:/home/mcgrof/devel/lkp-tests # export PATH=$PATH:$LKP_SRC/bin ergon:/home/mcgrof/devel/lkp-tests # ./bin/lkp qemu -k ~mcgrof/linux-next/arch/x86/boot/bzImage job-script
job-script: line 2: $'\r': command not found
job-script: line 3: syntax error near unexpected token `$'\r''
'ob-script: line 3: `export_top_env()
make: Entering directory '/home/mcgrof/devel/lkp-tests/bin/event'
klcc -c -o wakeup.o wakeup.c
klcc -static -o wakeup wakeup.o
rm -f wakeup.o
strip wakeup
make: Leaving directory '/home/mcgrof/devel/lkp-tests/bin/event'
cpio: lkp: Cannot stat: No such file or directory
cpio: lkp/scheduled: Cannot stat: No such file or directory
cpio: blank line ignored
cpio: File /dev/mqueue/���� shrunk by 20 bytes, padding with zeros

Quoted 1 line

Luis

Hmm, I believe it was caused by job-script saved as dos format, I can reproduce it when I downloaded it through outlook attachment.

Try dos2unix job-script before "lkp qemu" to see whether it works.

Thanks,
Xiaolong

author
#15 lore raw
Quoted 31 lines · Ye Xiaolong

On 01/10, Luis R. Rodriguez wrote:

[snip]

Quoted 27 lines
Quoted 7 lines
Quoted 4 lines

OK that runs, but that beats my actual machine with trinity, which is not desirable. I am trying to reproduce a reported error caught by lpk tests with trinity on my tree and that was with qemu. I have the job-script but that produces the same error with missing files. Can you help me with that?

Sure, could you paste the exact error log when you tried "./bin/lkp qemu -k ~mcgrof/linux-next/arch/x86/boot/bzImage job-script"?

As I noted its the same exact output, but here it is again:

ergon:/home/mcgrof/devel/lkp-tests # export LKP_SRC=$PWD ergon:/home/mcgrof/devel/lkp-tests # export PATH=$PATH:$LKP_SRC/bin ergon:/home/mcgrof/devel/lkp-tests # ./bin/lkp qemu -k ~mcgrof/linux-next/arch/x86/boot/bzImage job-script
job-script: line 2: $'\r': command not found
job-script: line 3: syntax error near unexpected token `$'\r''
'ob-script: line 3: `export_top_env()
make: Entering directory '/home/mcgrof/devel/lkp-tests/bin/event'
klcc -c -o wakeup.o wakeup.c
klcc -static -o wakeup wakeup.o
rm -f wakeup.o
strip wakeup
make: Leaving directory '/home/mcgrof/devel/lkp-tests/bin/event'
cpio: lkp: Cannot stat: No such file or directory
cpio: lkp/scheduled: Cannot stat: No such file or directory
cpio: blank line ignored
cpio: File /dev/mqueue/���� shrunk by 20 bytes, padding with zeros

Quoted 6 lines
Quoted 1 line

Luis

Hmm, I believe it was caused by job-script saved as dos format, I can reproduce it when I downloaded it through outlook attachment.

Try dos2unix job-script before "lkp qemu" to see whether it works.

Thanks, that fixed it!

I cannot reproduce the error reported. See:

root(a)10:/# dmesg| grep linkta
[    6.189785] test_linktables: linker table tests: OK!

The original report:

https://lists.01.org/pipermail/lkp/2016-December/005358.html

I also pushed out a new linker table branch out for linker table with a revert done first to force test avr32 (it worked!) a yesterday and have not received any lkp complaints about it yet.

Also if I hit CTRL-C and try to run again I get:

make: Entering directory '/home/mcgrof/devel/lkp-tests/bin/event'
gcc -m32 -static -o wakeup wakeup.o
/usr/lib64/gcc/x86_64-suse-linux/6/../../../../x86_64-suse-linux/bin/ld: skipping incompatible /usr/lib64/gcc/x86_64-suse-linux/6/libgcc.a when searching for -lgcc /usr/lib64/gcc/x86_64-suse-linux/6/../../../../x86_64-suse-linux/bin/ld: cannot find -lgcc
/usr/lib64/gcc/x86_64-suse-linux/6/../../../../x86_64-suse-linux/bin/ld: skipping incompatible /usr/lib64/gcc/x86_64-suse-linux/6/libgcc_eh.a when searching for -lgcc_eh /usr/lib64/gcc/x86_64-suse-linux/6/../../../../x86_64-suse-linux/bin/ld: cannot find -lgcc_eh
/usr/lib64/gcc/x86_64-suse-linux/6/../../../../x86_64-suse-linux/bin/ld: cannot find -lc
collect2: error: ld returned 1 exit status
make: *** [Makefile:14: main] Error 1
make: Leaving directory '/home/mcgrof/devel/lkp-tests/bin/event'
mv: cannot stat '/root/.lkp/cache/lkp-i386.cgz': No such file or directory
downloading initrds ...
/usr/bin/wget -q --local-encoding=UTF-8 --retry-connrefused --waitretry 1000 --tries 1000 https://github.com/0day-ci/lkp-qemu/raw/master/osimage/openwrt/openwrt-i386-2016-03-16.cgz -N -P /root/.lkp/cache/osimage/openwrt

It seems this tries to re-download the openwrt cpio -- why not avoid it if we already had it?

Luis

#16 lore raw

WARNING:at_lib/test_linktables/test-linktables.c:#test_linktable_init

Quoted 46 lines · Luis R. Rodriguez
Quoted 31 lines · Ye Xiaolong

On 01/10, Luis R. Rodriguez wrote:

[snip]

Quoted 27 lines
Quoted 7 lines
Quoted 4 lines

OK that runs, but that beats my actual machine with trinity, which is not desirable. I am trying to reproduce a reported error caught by lpk tests with trinity on my tree and that was with qemu. I have the job-script but that produces the same error with missing files. Can you help me with that?

Sure, could you paste the exact error log when you tried "./bin/lkp qemu -k ~mcgrof/linux-next/arch/x86/boot/bzImage job-script"?

As I noted its the same exact output, but here it is again:

ergon:/home/mcgrof/devel/lkp-tests # export LKP_SRC=$PWD ergon:/home/mcgrof/devel/lkp-tests # export PATH=$PATH:$LKP_SRC/bin ergon:/home/mcgrof/devel/lkp-tests # ./bin/lkp qemu -k ~mcgrof/linux-next/arch/x86/boot/bzImage job-script
job-script: line 2: $'\r': command not found
job-script: line 3: syntax error near unexpected token `$'\r''
'ob-script: line 3: `export_top_env()
make: Entering directory '/home/mcgrof/devel/lkp-tests/bin/event'
klcc -c -o wakeup.o wakeup.c
klcc -static -o wakeup wakeup.o
rm -f wakeup.o
strip wakeup
make: Leaving directory '/home/mcgrof/devel/lkp-tests/bin/event'
cpio: lkp: Cannot stat: No such file or directory
cpio: lkp/scheduled: Cannot stat: No such file or directory
cpio: blank line ignored
cpio: File /dev/mqueue/���� shrunk by 20 bytes, padding with zeros

Quoted 6 lines
Quoted 1 line

Luis

Hmm, I believe it was caused by job-script saved as dos format, I can reproduce it when I downloaded it through outlook attachment.

Try dos2unix job-script before "lkp qemu" to see whether it works.

Thanks, that fixed it!

I cannot reproduce the error reported. See:

root(a)10:/# dmesg| grep linkta
[    6.189785] test_linktables: linker table tests: OK!
Quoted 7 lines

The original report:

https://lists.01.org/pipermail/lkp/2016-December/005358.html

I also pushed out a new linker table branch out for linker table with a revert done first to force test avr32 (it worked!) a yesterday and have not received any lkp complaints about it yet.

Sorry for the late. Actually I just tested with commit a02e947 ("lib: add linker tables test driver") on branch mcgrof-next/20170113-linker-tables-v7-try2, and the warning log persists. See dmesg attached.

Quoted 17 lines

Also if I hit CTRL-C and try to run again I get:

make: Entering directory '/home/mcgrof/devel/lkp-tests/bin/event'
gcc -m32 -static -o wakeup wakeup.o
/usr/lib64/gcc/x86_64-suse-linux/6/../../../../x86_64-suse-linux/bin/ld: skipping incompatible /usr/lib64/gcc/x86_64-suse-linux/6/libgcc.a when searching for -lgcc /usr/lib64/gcc/x86_64-suse-linux/6/../../../../x86_64-suse-linux/bin/ld: cannot find -lgcc
/usr/lib64/gcc/x86_64-suse-linux/6/../../../../x86_64-suse-linux/bin/ld: skipping incompatible /usr/lib64/gcc/x86_64-suse-linux/6/libgcc_eh.a when searching for -lgcc_eh /usr/lib64/gcc/x86_64-suse-linux/6/../../../../x86_64-suse-linux/bin/ld: cannot find -lgcc_eh
/usr/lib64/gcc/x86_64-suse-linux/6/../../../../x86_64-suse-linux/bin/ld: cannot find -lc
collect2: error: ld returned 1 exit status
make: *** [Makefile:14: main] Error 1
make: Leaving directory '/home/mcgrof/devel/lkp-tests/bin/event'
mv: cannot stat '/root/.lkp/cache/lkp-i386.cgz': No such file or directory
downloading initrds ...
/usr/bin/wget -q --local-encoding=UTF-8 --retry-connrefused --waitretry 1000 --tries 1000 https://github.com/0day-ci/lkp-qemu/raw/master/osimage/openwrt/openwrt-i386-2016-03-16.cgz -N -P /root/.lkp/cache/osimage/openwrt

Quoted 1 line

It seems this tries to re-download the openwrt cpio -- why not avoid it if we already had it?

Nice suggestion. Actually we've added '-N' option for wget which will record file's timestamp, and it will only re-download if remote file is newer than the local one. However, github server seems refuse to send timestamp as wget would get message 'Last-modified header missing -- time-stamps turned off', thus wget will ignore timestamp and re-download every time.
One workaround is to replace '-N' with '-nc' which means skip downloads if file exists.

lib/wget.sh +1 -1
index 5524714..9db1e6f 100755
@@ -41,7 +41,7 @@ http_get_directory()
 http_get_newer()
 {
        local path="$(dirname "$2")"
-       http_do_request "$1" -N -P "$path"
+       http_do_request "$1" -nc -P "$path"
 }
 
 http_get_cgi()

The risk is that it doesn't check the timestamp, so once you have a local cached file and we update the remote one on github, lkp qemu would alway use the old one , but I figure it doesn't matter for this case.

Thanks,
Xiaolong

Quoted 2 lines

Luis

#17 lore raw
In reply to Luis R. Rodriguez
Quoted 43 lines · Luis R. Rodriguez

I received a failed test results notice from 0-day LKP test suite using trinity, however it was pretty hard to get reproduce given the LKP did not have OpenSUSE support to start off with, but the second issue I've stubmled upon is beyond me -- running either:

./bin/lkp qemu -k ~mcgrof/linux-next/arch/x86/boot/bzImage job-script

or

lkp qemu jobs/trinity.yaml

ends up creating calling a few subroutines or files which are clearly not present, then tries to create a cpio archive but clearly there are a few files which LKP tree expects you to have and are not present such as lkp and lkp/scheduled.

This is a run on OpenSUSE, first we install the dependencies for trinity.yaml deps:

 # lkp install jobs/trinity.yaml

Then we try to run it:

 # lkp qemu jobs/trinity.yaml 
jobs/trinity.yaml: line 1: suite:: command not found
jobs/trinity.yaml: line 2: testcase:: command not found
jobs/trinity.yaml: line 4: runtime:: command not found
jobs/trinity.yaml: line 6: kmsg:: command not found
jobs/trinity.yaml: line 7: rootfs:: command not found
jobs/trinity.yaml: line 9: trinity:: command not found
make: Entering directory '/home/mcgrof/devel/lkp-tests/bin/event'
klcc    -c -o wakeup.o wakeup.c
klcc  -static -o wakeup wakeup.o
rm -f wakeup.o
strip wakeup
make: Leaving directory '/home/mcgrof/devel/lkp-tests/bin/event'
cpio: lkp: Cannot stat: No such file or directory
cpio: lkp/scheduled: Cannot stat: No such file or directory
cpio: blank line ignored
cpio: File /dev/mqueue/���� shrunk by 20 bytes, padding with zeros

This never complets, it just sits there.

Quoted 7 lines

Either way these patches help start to add OpenSUSE support.

Luis R. Rodriguez (4):
lib/install.sh: fix shell adapt_packages()
distro: add initial opensuse support lib/install.*: make package dependency list uniq .gitignore: add bin/event/wakeup

thanks a lot, all 4 patches applied

Quoted 17 lines
 .gitignore                  |  1 +
 README.md                   | 22 ++++++++++++++++++++++
 distro/adaptation/archlinux |  2 +-
 distro/adaptation/fedora    |  2 +-
 distro/adaptation/opensuse  | 14 ++++++++++++++
 distro/depends/lkp-dev      |  4 ++++
 distro/installer/opensuse   |  3 +++
 lib/install.rb              |  2 +-
 lib/install.sh              | 14 +++++++++++---
 9 files changed, 58 insertions(+), 6 deletions(-)
 create mode 100644 distro/adaptation/opensuse
 create mode 100755 distro/installer/opensuse
Signature · 1 line
2.11.0