LTP Release preparations

LTP Cyril Hrubis 5 participants 14 messages Apr 12, 2018 – May 11, 2018

Share & tools

Cyril Hrubis

author

2018-04-12 11:35 UTC #0 lore raw

Hi!
It's about the time we should slowly prepare for the next release, as usuall let's start with a list of patches that should go in before we enter the stabilization phase.

As for me I would like to get the two speedup patchsets in, one of them only waits for ack from Jan, the second one would probably benefit from some more review. I will try to merge everything that is good to go and just waiting for a review ASAP.

Anything else that should really go in?

Signature · 2 lines

Cyril Hrubis
[email protected]

Cyril Hrubis

author

2018-04-19 14:17 UTC #1 lore raw

Hi!

Quoted 10 lines

It's about the time we should slowly prepare for the next release, as usuall let's start with a list of patches that should go in before we enter the stabilization phase.
As for me I would like to get the two speedup patchsets in, one of them only waits for ack from Jan, the second one would probably benefit from some more review. I will try to merge everything that is good to go and just waiting for a review ASAP.
Anything else that should really go in?

Ping.

I suppose that we should get in the fix for the read_all testcase along with the exponential back off.

Also I would like to get in the needs_rofs patchset.

There is also patch for converting our python code to be python3 compatible that is worth looking into.

Anything else?

Signature · 2 lines

Cyril Hrubis
[email protected]

Li Wang

2018-04-20 03:56 UTC #2 lore raw

Quoted 10 lines · Cyril Hrubis

Hi!
Quoted 9 lines
It's about the time we should slowly prepare for the next release, as usuall let's start with a list of patches that should go in before we enter the stabilization phase.
As for me I would like to get the two speedup patchsets in, one of them only waits for ack from Jan, the second one would probably benefit from some more review. I will try to merge everything that is good to go and just waiting for a review ASAP.

Quoted 7 lines

Quoted 1 line
Anything else that should really go in?
Ping.
I suppose that we should get in the fix for the read_all testcase along with the exponential back off.

I'm hesitating if we should abandon the old patch[1] and re-rewrite a new one
(which base on the new TST_RETRY_FUNC macro) to fix the problem.

[1] http://lists.linux.it/pipermail/ltp/2018-April/007704.html

-----------------
something new like:

testcases/kernel/fs/read_all/read_all.c +3 ~~-13~~

@@ -265,23 +265,14 @@ static void spawn_workers(void)
 static void stop_workers(void)
 {
        const char stop_code[1] = { '\0' };
-       int i, stop_attempts;
+       int i;
 
        if (!workers)
                return;
 
        for (i = 0; i < worker_count; i++) {
-               stop_attempts = 0xffff;
-               if (workers[i].q) {
-               if (workers[i].q) {
-                       while (!queue_push(workers[i].q, stop_code)) {
-                               if (--stop_attempts < 0) {
-                                       tst_brk(TBROK,
-                                               "Worker %d is stalled",
-                                               workers[i].pid);
-                                       break;
-                               }
-                       }
-               }
+               if (workers[i].q)
+                       TST_RETRY_FUNC(queue_push(workers[i].q, stop_code),

1);
        }

        for (i = 0; i < worker_count; i++) {
@@ -292,33 +283,15 @@ static void stop_workers(void)
        }

}

-static void sched_work(const char *path)
-{
- static int cur;
- int push_attempts = 0, pushed;
-
- while (1) {
- pushed = queue_push(workers[cur].q, path);
-
- if (++cur >= worker_count)
- cur = 0;
-
- if (pushed)
- break;
-
- if (++push_attempts > worker_count) {
- usleep(100);
- push_attempts = 0;
- }
- }
-}
-
static void rep_sched_work(const char *path, int rep) {
- int i;
+ int i, j;

- for (i = 0; i < rep; i++)
- sched_work(path);
+ for (i = j = 0; i < rep; i++, j++) {
+ if (j >= 15)
+ j = 0;
+ TST_RETRY_FUNC(queue_push(workers[j].q, path), 1);
+ }
}

Signature · 5 lines

Li Wang
[email protected]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linux.it/pipermail/ltp/attachments/20180420/1ae8918e/attachment-0001.html>

Petr Vorel

2018-04-20 10:49 UTC #3 lore raw

In reply to Cyril Hrubis

Hi,

Quoted 13 lines

Hi!
Quoted 10 lines
It's about the time we should slowly prepare for the next release, as usuall let's start with a list of patches that should go in before we enter the stabilization phase.
As for me I would like to get the two speedup patchsets in, one of them only waits for ack from Jan, the second one would probably benefit from some more review. I will try to merge everything that is good to go and just waiting for a review ASAP.
Anything else that should really go in?
Ping.

...

Quoted 1 line

Anything else?

I hope to get my IMA patches merged to the release.

Kind regards,
Petr

Cyril Hrubis

author

2018-04-24 13:39 UTC #4 lore raw

In reply to Cyril Hrubis

Hi!

Quoted 7 lines

I suppose that we should get in the fix for the read_all testcase along with the exponential back off.
Also I would like to get in the needs_rofs patchset.
There is also patch for converting our python code to be python3 compatible that is worth looking into.

That has been done.

I've also send a patch for long-standing problem with perf_event_open02 testcase that would be a good candidate for the release.

Peter how is it with the IMA testsuite, should we merge the patchset and call it incremental improvement over the previous code?

Anything else to be merged?

I would like to go for freezing the git at the end of the week and start with a pre-release testing next week. Is that OK with everyone?

Signature · 2 lines

Cyril Hrubis
[email protected]

Jan Stancek

2018-04-24 21:08 UTC #5 lore raw

----- Original Message -----

Quoted 14 lines

Hi!
Quoted 7 lines
I suppose that we should get in the fix for the read_all testcase along with the exponential back off.
Also I would like to get in the needs_rofs patchset.
There is also patch for converting our python code to be python3 compatible that is worth looking into.
That has been done.
I've also send a patch for long-standing problem with perf_event_open02 testcase that would be a good candidate for the release.

Quoted 7 lines

Peter how is it with the IMA testsuite, should we merge the patchset and call it incremental improvement over the previous code?
Anything else to be merged?
I would like to go for freezing the git at the end of the week and start with a pre-release testing next week. Is that OK with everyone?

I started testing and so far ran into problem with building cve-2017-16939.c on RHEL7.2:

gcc -g -O2 -g -O2 -fno-strict-aliasing -pipe -Wall -W -Wold-style-definition -D_GNU_SOURCE -D_FORTIFY_SOURCE=2 -I../../include -I../../include -I../../include/old/ -L../../lib cve-2017-16939.c -lltp -o cve-2017-16939
In file included from /usr/include/linux/xfrm.h:4:0,

                 from cve-2017-16939.c:33:
/usr/include/linux/in6.h:30:8: error: redefinition of ‘struct in6_addr’
 struct in6_addr {
        ^
In file included from cve-2017-16939.c:31:0:
/usr/include/netinet/in.h:197:8: note: originally defined here
 struct in6_addr
        ^
In file included from /usr/include/linux/xfrm.h:4:0,
                 from cve-2017-16939.c:33:
/usr/include/linux/in6.h:41:8: error: redefinition of ‘struct sockaddr_in6’
 struct sockaddr_in6 {
        ^
In file included from cve-2017-16939.c:29:0:
/usr/include/sys/socket.h:91:17: note: originally defined here
 typedef union { __SOCKADDR_ALLTYPES
                 ^
In file included from /usr/include/linux/xfrm.h:4:0,
                 from cve-2017-16939.c:33:
/usr/include/linux/in6.h:49:8: error: redefinition of ‘struct ipv6_mreq’
 struct ipv6_mreq {
        ^
In file included from cve-2017-16939.c:31:0:
/usr/include/netinet/in.h:274:8: note: originally defined here
 struct ipv6_mreq
        ^

make: *** [cve-2017-16939] Error 1

Quoted 8 lines

--
Cyril Hrubis
[email protected]
--
Mailing list info: https://lists.linux.it/listinfo/ltp

Cyril Hrubis

author

2018-04-25 13:23 UTC #6 lore raw

Hi!

Quoted 31 lines

I started testing and so far ran into problem with building cve-2017-16939.c on RHEL7.2:

                 from cve-2017-16939.c:33:
/usr/include/linux/in6.h:30:8: error: redefinition of ???struct in6_addr???
 struct in6_addr {
        ^
In file included from cve-2017-16939.c:31:0:
/usr/include/netinet/in.h:197:8: note: originally defined here
 struct in6_addr
        ^
In file included from /usr/include/linux/xfrm.h:4:0,
                 from cve-2017-16939.c:33:
/usr/include/linux/in6.h:41:8: error: redefinition of ???struct sockaddr_in6???
 struct sockaddr_in6 {
        ^
In file included from cve-2017-16939.c:29:0:
/usr/include/sys/socket.h:91:17: note: originally defined here
 typedef union { __SOCKADDR_ALLTYPES
                 ^
In file included from /usr/include/linux/xfrm.h:4:0,
                 from cve-2017-16939.c:33:
/usr/include/linux/in6.h:49:8: error: redefinition of ???struct ipv6_mreq???
 struct ipv6_mreq {
        ^
In file included from cve-2017-16939.c:31:0:
/usr/include/netinet/in.h:274:8: note: originally defined here
 struct ipv6_mreq
        ^

Quoted 1 line

make: *** [cve-2017-16939] Error 1

This seems to be very strange problem with mixing linux and libc headers. I've tried to remove the netinet/in.h header from the test and build it on different distros, buy that seemed to actually cause more problems than when it's included.

And actually I haven't been able to reproduce the build problem failure the original code, but I build-test only small mumber of RHEL and Fedora builds here.

I guess that the easiest solution here may be to declare the three XFRM macros somewhere in lapi header and include it here instead fo the linux/xfrm.h.

Signature · 2 lines

Cyril Hrubis
[email protected]

Jan Stancek

2018-04-25 13:42 UTC #7 lore raw

----- Original Message -----

Quoted 45 lines

Hi!

Quoted 37 lines

I started testing and so far ran into problem with building cve-2017-16939.c
on RHEL7.2:

gcc -g -O2 -g -O2 -fno-strict-aliasing -pipe -Wall -W

(quoted diff) +0 -3

-Wold-style-definition -D_GNU_SOURCE -D_FORTIFY_SOURCE=2 -I../../include
-I../../include -I../../include/old/   -L../../lib  cve-2017-16939.c
-lltp -o cve-2017-16939

In file included from /usr/include/linux/xfrm.h:4:0,

                 from cve-2017-16939.c:33:
/usr/include/linux/in6.h:30:8: error: redefinition of ???struct in6_addr???
 struct in6_addr {
        ^
In file included from cve-2017-16939.c:31:0:
/usr/include/netinet/in.h:197:8: note: originally defined here
 struct in6_addr
        ^
In file included from /usr/include/linux/xfrm.h:4:0,
                 from cve-2017-16939.c:33:
/usr/include/linux/in6.h:41:8: error: redefinition of ???struct
sockaddr_in6???
 struct sockaddr_in6 {
        ^
In file included from cve-2017-16939.c:29:0:
/usr/include/sys/socket.h:91:17: note: originally defined here
 typedef union { __SOCKADDR_ALLTYPES
                 ^
In file included from /usr/include/linux/xfrm.h:4:0,
                 from cve-2017-16939.c:33:
/usr/include/linux/in6.h:49:8: error: redefinition of ???struct
ipv6_mreq???
 struct ipv6_mreq {
        ^
In file included from cve-2017-16939.c:31:0:
/usr/include/netinet/in.h:274:8: note: originally defined here
 struct ipv6_mreq
        ^

Quoted 1 line

make: *** [cve-2017-16939] Error 1

Quoted 7 lines

And actually I haven't been able to reproduce the build problem failure the original code, but I build-test only small mumber of RHEL and Fedora builds here.
I guess that the easiest solution here may be to declare the three XFRM macros somewhere in lapi header and include it here instead fo the linux/xfrm.h.

That worked for me, though I missed only 2 defines:
XFRMNLGRP_NONE
XFRM_MSG_GETPOLICY

Are you missing three after you drop linux/xfrm.h?

Quoted 5 lines

--
Cyril Hrubis
[email protected]

Cyril Hrubis

author

2018-04-25 13:44 UTC #8 lore raw

Hi!

Quoted 5 lines

That worked for me, though I missed only 2 defines:
XFRMNLGRP_NONE
XFRM_MSG_GETPOLICY
Are you missing three after you drop linux/xfrm.h?

I just grepped the file for XFRM_ and expect these macros to be defined in the xfrm header, looking closer the NETLINK_XFRM is indeed defined in the netlink header.

Signature · 2 lines

Cyril Hrubis
[email protected]

Petr Vorel

2018-04-25 14:11 UTC #9 lore raw

In reply to Cyril Hrubis

Hi Mimi, Cyril,

Quoted 2 lines

Peter how is it with the IMA testsuite, should we merge the patchset and call it incremental improvement over the previous code?

Yes. There ale still some issues, current patches are improvement and I'm not aware about any regression.
So I'm going to merge patches 1,3-10 on Friday afternoon. Is that ok for you or should I merge it earlier.
The only patch I'm not going to merge is the second one ("security/ima: Change order of tests") [1]

Mimi, any chance you to have a look into patches till Friday?

Kind regards,
Petr

[1] https://lists.linux.it/pipermail/ltp/2018-April/007783.html

Cyril Hrubis

author

2018-05-03 09:28 UTC #10 lore raw

In reply to Cyril Hrubis

Hi!

Quoted 2 lines

I would like to go for freezing the git at the end of the week and start with a pre-release testing next week. Is that OK with everyone?

I'm in the middle of pre-release tesing, so far it looks good.

The pending issues for the release are the compilation fix from Jan and the perf_event_open02 fix I've send.

Anything else to consider?

Signature · 2 lines

Cyril Hrubis
[email protected]

Alexey Kodanev

2018-05-03 11:41 UTC #11 lore raw

Quoted 1 line · Cyril Hrubis

Hi!

Hi Cyril,

Quoted 10 lines

Quoted 2 lines
I would like to go for freezing the git at the end of the week and start with a pre-release testing next week. Is that OK with everyone?
I'm in the middle of pre-release tesing, so far it looks good.
The pending issues for the release are the compilation fix from Jan and the perf_event_open02 fix I've send.
Anything else to consider?

Any objections against patch v3 "sctp_big_chunk: make INIT packet in the test" [1]? It should fix the test on the kernels 4.17-rc2+.

[1] http://lists.linux.it/pipermail/ltp/2018-April/007933.html

Thanks,
Alexey

Petr Vorel

2018-05-03 12:06 UTC #12 lore raw

In reply to Cyril Hrubis

Hi,

Quoted 10 lines

Hi!
Quoted 2 lines
I would like to go for freezing the git at the end of the week and start with a pre-release testing next week. Is that OK with everyone?
I'm in the middle of pre-release tesing, so far it looks good.
The pending issues for the release are the compilation fix from Jan and the perf_event_open02 fix I've send.
Anything else to consider?

I'll have a look at Alexey's patch v3 "sctp_big_chunk: make INIT packet in the test"
+ I'm going to send small fix in net in a minute.

Kind regards,
Petr

Cyril Hrubis

author

2018-05-11 12:17 UTC #13 lore raw

In reply to Cyril Hrubis

Hi!

Quoted 2 lines

The pending issues for the release are the compilation fix from Jan and the perf_event_open02 fix I've send.

As nobody objected, or commented so far I'm going to commit this one next week.

Quoted 1 line

Anything else to consider?

Li pinged the migrate_pages02 patch from Jan, anything else?

I would like to proceed with the release next week so if you have anything that should be dealt with speak up please.

Signature · 2 lines

Cyril Hrubis
[email protected]