Skip to contentSkip to conversation

LTP Release

LTP Cyril Hrubis 3 participants 17 messages Apr 18, 2019 – May 15, 2019 Kbuild

lore
Share & tools
lore
#0 lore raw

Hi!
It's about the time we start to prepare for the next LTP release. Ideally I would like to go for a release at the start of the second week in the May.

So as usually please go ahead and point out anything that should go in before the release.

Signature · 2 lines
Cyril Hrubis
[email protected]
#1 lore raw

Hi Cyril,

Quoted 7 lines

Hi!
It's about the time we start to prepare for the next LTP release. Ideally I would like to go for a release at the start of the second week in the May.

So as usually please go ahead and point out anything that should go in before the release.

IMA patches (LTP reproducer on broken IMA on overlayfs) [1] are waiting for Mimi's approval and Ignaz's testing. Not sure if they'll have time till the release.

Kind regards,
Petr

[1] https://patchwork.ozlabs.org/project/ltp/list/?series=101213&state=*

#2 lore raw
In reply to Cyril Hrubis

Hi!

Quoted 2 lines

So as usually please go ahead and point out anything that should go in before the release.

Only Peter replied so far, so I suppose that we are ready to freeze git starting next week and start the pre-release testing.

Or is there anything that has to be considered to be included before we do that?

Signature · 2 lines
Cyril Hrubis
[email protected]
#3 lore raw
Quoted 10 lines · Cyril Hrubis

Hi!

Quoted 2 lines

So as usually please go ahead and point out anything that should go in before the release.

Only Peter replied so far, so I suppose that we are ready to freeze git starting next week and start the pre-release testing.

Or is there anything that has to be considered to be included before we do that?

Here is a simple fix for hugepage test broken with memory shortage. Consider merging?
http://lists.linux.it/pipermail/ltp/2019-April/011714.html

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20190428/ebcc1924/attachment.html>
#4 lore raw

Hi!

Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

So as usually please go ahead and point out anything that should go in before the release.

Only Peter replied so far, so I suppose that we are ready to freeze git starting next week and start the pre-release testing.

Quoted 7 lines
Quoted 3 lines

Or is there anything that has to be considered to be included before we do that?

Here is a simple fix for hugepage test broken with memory shortage. Consider merging?
http://lists.linux.it/pipermail/ltp/2019-April/011714.html

The basic idea looks OK however:

* the function name (reset_hugepages) is a bit confusing what about "cap_hugepages" or "limit_hugepages"

* The MemFree: may be close to zero on long running system because of cached files, AFAIK MemAvailable: would be a better choice, see:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=34e431b0ae398fc54ea69ff85ec700722c9da773

Unfortunately that one is available since 3.14 kernel, if we wanted to run on older kenrnels I would propose to fall back to dropping caches and using MemFree: on kernel that does not have MemAvailable: in meminfo.

Signature · 2 lines
Cyril Hrubis
[email protected]
#5 lore raw
Quoted 12 lines · Cyril Hrubis

Hi!

Quoted 1 line
Quoted 1 line
Quoted 1 line

So as usually please go ahead and point out anything that should go

in

Quoted 9 lines
Quoted 8 lines
Quoted 1 line

before the release.

Only Peter replied so far, so I suppose that we are ready to freeze git starting next week and start the pre-release testing.

Or is there anything that has to be considered to be included before we do that?

Quoted 9 lines
Quoted 3 lines

Here is a simple fix for hugepage test broken with memory shortage. Consider merging?
http://lists.linux.it/pipermail/ltp/2019-April/011714.html

The basic idea looks OK however:

* the function name (reset_hugepages) is a bit confusing what about "cap_hugepages" or "limit_hugepages"

"limit_hugepages" sounds good to me.

Quoted 5 lines

* The MemFree: may be close to zero on long running system because of cached files, AFAIK MemAvailable: would be a better choice, see:

Quoted 7 lines

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=34e431b0ae398fc54ea69ff85ec700722c9da773

Unfortunately that one is available since 3.14 kernel, if we wanted to run on older kenrnels I would propose to fall back to dropping caches and using MemFree: on kernel that does not have MemAvailable: in meminfo.

Good proposal. I will improve like this in patch V4. Thanks!

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20190430/8c1654cd/attachment-0001.html>
#6 lore raw
In reply to Cyril Hrubis

Hi Cyril,

Quoted 9 lines

Hi!

Quoted 2 lines

So as usually please go ahead and point out anything that should go in before the release.

Only Peter replied so far, so I suppose that we are ready to freeze git starting next week and start the pre-release testing.

Or is there anything that has to be considered to be included before we do that?

I guess it'd be great to fix sporadic out-of-tree build failures related to libs. Jan's patch [1] does not fixed it. I have no idea how to fix it.

Kind regards,
Petr

[1] https://patchwork.ozlabs.org/patch/1059002/

#7 lore raw

Hi!

Quoted 8 lines
Quoted 8 lines
Quoted 2 lines

So as usually please go ahead and point out anything that should go in before the release.

Only Peter replied so far, so I suppose that we are ready to freeze git starting next week and start the pre-release testing.

Or is there anything that has to be considered to be included before we do that?

Quoted 7 lines

I guess it'd be great to fix sporadic out-of-tree build failures related to libs. Jan's patch [1] does not fixed it. I have no idea how to fix it.

Kind regards,
Petr

[1] https://patchwork.ozlabs.org/patch/1059002/

There are two things to fix. One that the patch from Jan attempted is wrong path in the rule in the testcases.mk and the second one is that the libraries are not build from the top level Makefile first before everything else.

For the first problem fixing the path to the library Makefile should fix the problem, something as (beware untested) this should fix it:

include/mk/testcases.mk +1 -1
index 131854ec7..6d1b2d4ef 100644
@@ -49,7 +49,7 @@ LTPLIBS_FILES = $(addsuffix .a, $(addprefix $(abs_top_builddir)/libs/, $(foreach
 MAKE_DEPS += $(LTPLIBS_FILES)
 
 $(LTPLIBS_FILES): $(LTPLIBS_DIRS)
-       $(MAKE) -C "$^" -f "$^/Makefile" all
+       $(MAKE) -C "$^" -f "$(subst $(abs_top_buildir),$(abs_to_srcdir),$^)/Makefile" all
 
 LDFLAGS += $(addprefix -L$(top_builddir)/libs/lib, $(LTPLIBS))
 

I will have a look at the top level Makefile as well to see what has to be done there.

Signature · 2 lines
Cyril Hrubis
[email protected]
#8 lore raw

Hi!

Quoted 13 lines
include/mk/testcases.mk +1 -1
index 131854ec7..6d1b2d4ef 100644
@@ -49,7 +49,7 @@ LTPLIBS_FILES = $(addsuffix .a, $(addprefix $(abs_top_builddir)/libs/, $(foreach
 MAKE_DEPS += $(LTPLIBS_FILES)
 
 $(LTPLIBS_FILES): $(LTPLIBS_DIRS)
-       $(MAKE) -C "$^" -f "$^/Makefile" all
+       $(MAKE) -C "$^" -f "$(subst $(abs_top_buildir),$(abs_to_srcdir),$^)/Makefile" all
 
 LDFLAGS += $(addprefix -L$(top_builddir)/libs/lib, $(LTPLIBS))
 
Quoted 2 lines

I will have a look at the top level Makefile as well to see what has to be done there.

And I guess I found the problem.

When I tried to fix the out-of-tree build by adding a rule to create the libs directory I overlooked that there are all rules already in place and all that had to be added was the dependency of libs-all on the builddir libs path. After I added the rule that created the directory the libs-all targed was fulfilled by a redefined target that only created the directory and haven't build the libs directory before the rest of the source tree at all. We even got target redefinition warnings because of that.

Peter can you please test with this change:

Makefile +0 -3
index c46d050ce..768ca4606 100644
@@ -102,9 +102,6 @@ $(sort $(addprefix $(abs_top_builddir)/,$(BOOTSTRAP_TARGETS)) $(INSTALL_DIR) $(D
 ## Pattern based subtarget rules.
 lib-install: lib-all
 
-$(abs_top_builddir)/libs:
-       mkdir -m 00755 -p "$@"
-
 libs-all: $(abs_top_builddir)/libs
 
 $(MAKE_TARGETS) include-all lib-all libs-all:
Signature · 2 lines
Cyril Hrubis
[email protected]
#9 lore raw

Hi Cyril,

Quoted 13 lines
Quoted 2 lines

I will have a look at the top level Makefile as well to see what has to be done there.

And I guess I found the problem.

When I tried to fix the out-of-tree build by adding a rule to create the libs directory I overlooked that there are all rules already in place and all that had to be added was the dependency of libs-all on the builddir libs path. After I added the rule that created the directory the libs-all targed was fulfilled by a redefined target that only created the directory and haven't build the libs directory before the rest of the source tree at all. We even got target redefinition warnings because of that.

Quoted 1 line

Peter can you please test with this change:

Sure! While this one is ok [1], and also with previous patch [2]. But when tested on real HW with 32 cores (with merged both commits), where it took me 6 iterations to trigger this error :( (5 iterations was needed to trigger error on the current master).

Kind regards,
Petr
[1] https://travis-ci.org/pevik/ltp/builds/526011623
[2] https://travis-ci.org/pevik/ltp/builds/526017203

Quoted 14 lines
Makefile +0 -3
index c46d050ce..768ca4606 100644
@@ -102,9 +102,6 @@ $(sort $(addprefix $(abs_top_builddir)/,$(BOOTSTRAP_TARGETS)) $(INSTALL_DIR) $(D
 ## Pattern based subtarget rules.
 lib-install: lib-all
 
-$(abs_top_builddir)/libs:
-       mkdir -m 00755 -p "$@"
-
 libs-all: $(abs_top_builddir)/libs
 
 $(MAKE_TARGETS) include-all lib-all libs-all:
#10 lore raw

Hi!

Quoted 10 lines
Quoted 9 lines

And I guess I found the problem.

When I tried to fix the out-of-tree build by adding a rule to create the libs directory I overlooked that there are all rules already in place and all that had to be added was the dependency of libs-all on the builddir libs path. After I added the rule that created the directory the libs-all targed was fulfilled by a redefined target that only created the directory and haven't build the libs directory before the rest of the source tree at all. We even got target redefinition warnings because of that.

Quoted 6 lines
Quoted 1 line

Peter can you please test with this change:

Sure! While this one is ok [1], and also with previous patch [2]. But when tested on real HW with 32 cores (with merged both commits), where it took me 6 iterations to trigger this error :( (5 iterations was needed to trigger error on the current master).

That means that the dependencies are still not correct and we still fail to build the library before everything else.

Signature · 2 lines
Cyril Hrubis
[email protected]
Tested-by
#11 lore raw
In reply to Cyril Hrubis

Hi Cyril,

Quoted 21 lines

There are two things to fix. One that the patch from Jan attempted is wrong path in the rule in the testcases.mk and the second one is that the libraries are not build from the top level Makefile first before everything else.

For the first problem fixing the path to the library Makefile should fix the problem, something as (beware untested) this should fix it:

include/mk/testcases.mk +1 -1
index 131854ec7..6d1b2d4ef 100644
@@ -49,7 +49,7 @@ LTPLIBS_FILES = $(addsuffix .a, $(addprefix $(abs_top_builddir)/libs/, $(foreach
 MAKE_DEPS += $(LTPLIBS_FILES)
 
 $(LTPLIBS_FILES): $(LTPLIBS_DIRS)
-       $(MAKE) -C "$^" -f "$^/Makefile" all
+       $(MAKE) -C "$^" -f "$(subst $(abs_top_buildir),$(abs_to_srcdir),$^)/Makefile" all
 
 LDFLAGS += $(addprefix -L$(top_builddir)/libs/lib, $(LTPLIBS))
 
Quoted 2 lines

I will have a look at the top level Makefile as well to see what has to be done there.

This one looks ok. Also testing with machine with 32 cores, ok as well (but it's hard to trigger this bug even current master).

Kind regards,
Petr

[1] https://travis-ci.org/pevik/ltp/builds/526016709

#12 lore raw

Hi!

Quoted 11 lines
Quoted 10 lines
include/mk/testcases.mk +1 -1
index 131854ec7..6d1b2d4ef 100644
@@ -49,7 +49,7 @@ LTPLIBS_FILES = $(addsuffix .a, $(addprefix $(abs_top_builddir)/libs/, $(foreach
 MAKE_DEPS += $(LTPLIBS_FILES)
 
 $(LTPLIBS_FILES): $(LTPLIBS_DIRS)
-       $(MAKE) -C "$^" -f "$^/Makefile" all
+       $(MAKE) -C "$^" -f "$(subst $(abs_top_buildir),$(abs_to_srcdir),$^)/Makefile" all
Quoted 8 lines
Quoted 4 lines
include/mk/testcases.mk +0 -0
 LDFLAGS += $(addprefix -L$(top_builddir)/libs/lib, $(LTPLIBS))
 

I will have a look at the top level Makefile as well to see what has to be done there.

This one looks ok. Also testing with machine with 32 cores, ok as well (but it's hard to trigger this bug even current master).

Sigh, apparenlty this has patch has two obvious typos, both abs_top_builddir and abs_top_srcdir are spelled incorrectly, no wonder it does not work, sorry for that.

And the only reason why this works for in-tree build is because with typos we do no replace anything and the path to the Makefile is correct for in-tree build.

Signature · 2 lines
Cyril Hrubis
[email protected]
#13 lore raw

Hi Cyril,

Quoted 8 lines
Quoted 4 lines

This one looks ok. Also testing with machine with 32 cores, ok as well (but it's hard to trigger this bug even current master).

Sigh, apparenlty this has patch has two obvious typos, both abs_top_builddir and abs_top_srcdir are spelled incorrectly, no wonder it does not work, sorry for that.

With this fix (used as commits [1] [2]) looks it fixed it.
21 times or fewer was needed o trigger the build on master, now ok with 33 builds. I'll keep it running during the day just for sure, but consider it fixed.

Quoted 3 lines

And the only reason why this works for in-tree build is because with typos we do no replace anything and the path to the Makefile is correct for in-tree build.

Kind regards,
Petr

[1] https://github.com/pevik/ltp/commit/22fd15ad08166dc2ec4c8e93884e51be4a46c1b6
[2] https://github.com/pevik/ltp/commit/10a783538331df8616384f1d9d62c334c117c918

#14 lore raw
In reply to Cyril Hrubis

Hi Cyril,

I have completed the latest LTP testing on RHEL-7.6 and RHEL-8.0 across all arches(x86_64, ppc64le, ppc64, s390x, aarch64), here I post the summary FYI:

RHEL7.6
------------
1. crypto/af_alg01/02 (all arches - fail)
af_alg01 failure is a kernel bug.
af_alg02 test broken as what Christian's described in the patch. It should be fixed on the kernel without that commit ecaaab564978 too.

2. binfmt_misc01.sh (x86_64, s390x - fail, ppc64le/ppc64 - panic) It looks like a kernel bug.

RHEL8.0
-------------
1. NUMA issue (ppc64le)
Numa test failed on that machine with non-continuous NUMA nodes. Two patches for the problem solving has been sent to LTP-ML.

2. HUGETLB issue (aarch64)
hugetlb tests sometimes failed to request sufficient huge pages, even we try limiting to 80% of the max huge page count is still not being satisfied on 512MB huge page size system sometimes.

Besides these above failures, the remaining test looks pretty good.

Signature · 5 lines
Regards,
Li Wang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20190514/59eefb50/attachment.html>
#15 lore raw

Hi!

Quoted 2 lines

I have completed the latest LTP testing on RHEL-7.6 and RHEL-8.0 across all arches(x86_64, ppc64le, ppc64, s390x, aarch64), here I post the summary FYI:

Thanks a lot for the detailed writeup!

Quoted 6 lines

RHEL7.6
------------
1. crypto/af_alg01/02 (all arches - fail)
af_alg01 failure is a kernel bug.
af_alg02 test broken as what Christian's described in the patch. It should be fixed on the kernel without that commit ecaaab564978 too.

Will look into this patch.

Quoted 8 lines

2. binfmt_misc01.sh (x86_64, s390x - fail, ppc64le/ppc64 - panic) It looks like a kernel bug.

RHEL8.0
-------------
1. NUMA issue (ppc64le)
Numa test failed on that machine with non-continuous NUMA nodes. Two patches for the problem solving has been sent to LTP-ML.

And into this as well.

Quoted 4 lines

2. HUGETLB issue (aarch64)
hugetlb tests sometimes failed to request sufficient huge pages, even we try limiting to 80% of the max huge page count is still not being satisfied on 512MB huge page size system sometimes.

I guess that this will have to wait to be fixed after the release.

From my point of view the ioctl_ns tests failures with -m32 are worth fixing and that together with the afl_alg02 and numa issues should be everything that needs to be taken care of before the release.

Signature · 2 lines
Cyril Hrubis
[email protected]