in:imklog segfault in 4.12+git on most sparc64's

DISCUSSION Meelis Roos 5 participants 17 messages Jul 10, 2017 – Jul 12, 2017

lore

Share & tools

lore

Meelis Roos

author

2017-07-10 08:25 UTC #0 lore raw

I tested yesterdays git with 4.13 sparc merge on a big bunch of my test machines.

Good news: the timestamps starting from boot seem to work on all the machines, starting with Ultra 1 and Ultra 2.

Bad news: all working sun4u machines have the following line in dmesg
(PID varies):
in:imklog[1036]: segfault at 49 ip 0000000000035bb0 (rpc 00000000f779dfa0) sp 00000002f7a78818 error 30002 in rsyslogd[10000+88000]

This is with Debian unstable 32-bit userland as of 2015 summer. It did not happen with 4.12.

More bad news: all my sun4v machines crash on boot with DMA+IOMMU problem (will send a separate mail with details).

Signature · 1 line

Meelis Roos ([email protected])

John Paul Adrian Glaubitz

2017-07-10 08:38 UTC #1 lore raw

Quoted 2 lines · Meelis Roos

This is with Debian unstable 32-bit userland as of 2015 summer. It did not happen with 4.12.

Did you also perform tests with a current 64-bit userland?

Adrian

Signature · 4 lines

 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [email protected]
`. `'   Freie Universitaet Berlin - [email protected]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Meelis Roos

author

2017-07-10 11:46 UTC #2 lore raw

In reply to Meelis Roos

Quoted 4 lines

Quoted 2 lines
This is with Debian unstable 32-bit userland as of 2015 summer. It did not happen with 4.12.
Did you also perform tests with a current 64-bit userland?

No, for the current test this is not relevant since if the breakage happens with some userlan, it is kernel at fault.

I have tested sparc64 debian-port on my T5120 where I expected it to be fast. For some test it was faster than 32-bit userland. for some things slower, but it worked.

I never tested it on older machines so far. I might have tried installing on Blade 150 (I am not sure if it was sparc or sparc64) but the install was hanging, and I have not finished with it as I suspected the PSU that I had just replaced and I have had no time to to hardware bisecting with Blade 100.

Signature · 1 line

Meelis Roos ([email protected])

John Paul Adrian Glaubitz

2017-07-10 11:48 UTC #3 lore raw

In reply to Meelis Roos

Quoted 7 lines · Meelis Roos

Quoted 4 lines
Quoted 2 lines
This is with Debian unstable 32-bit userland as of 2015 summer. It did not happen with 4.12.
Did you also perform tests with a current 64-bit userland?
No, for the current test this is not relevant since if the breakage happens with some userlan, it is kernel at fault.

Yes, but I think it's more interesting to perform such tests with current and maintained userland software.

Quoted 5 lines

I never tested it on older machines so far. I might have tried installing on Blade 150 (I am not sure if it was sparc or sparc64) but the install was hanging, and I have not finished with it as I suspected the PSU that I had just replaced and I have had no time to to hardware bisecting with Blade 100.

Debian sparc64 runs fine on my Sun Blade 100.

Adrian

Signature · 4 lines

 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [email protected]
`. `'   Freie Universitaet Berlin - [email protected]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

David Miller

2017-07-10 18:31 UTC #4 lore raw

In reply to Meelis Roos

From: Meelis Roos <[email protected]>
Date: Mon, 10 Jul 2017 14:46:19 +0300 (EEST)

Quoted 7 lines

Quoted 4 lines
Quoted 2 lines
This is with Debian unstable 32-bit userland as of 2015 summer. It did not happen with 4.12.
Did you also perform tests with a current 64-bit userland?
No, for the current test this is not relevant since if the breakage happens with some userlan, it is kernel at fault.

David Miller

2017-07-10 18:32 UTC #5 lore raw

In reply to Meelis Roos

From: John Paul Adrian Glaubitz <[email protected]>
Date: Mon, 10 Jul 2017 13:48:46 +0200

Quoted 11 lines · Meelis Roos

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
This is with Debian unstable 32-bit userland as of 2015 summer. It did not happen with 4.12.
Did you also perform tests with a current 64-bit userland?
No, for the current test this is not relevant since if the breakage happens with some userlan, it is kernel at fault.
Yes, but I think it's more interesting to perform such tests with current and maintained userland software.

Yes, let's make sure that 32-bit userland support in the kernel gets even less testing, and gets even buggier over time.

I think people should test everything, as much as possible.

John Paul Adrian Glaubitz

2017-07-10 18:35 UTC #6 lore raw

In reply to Meelis Roos

Quoted 2 lines · David Miller

Yes, let's make sure that 32-bit userland support in the kernel gets even less testing, and gets even buggier over time.

My point was not 32-bit vs. 64-bit, my point was using an outdated userland from 2015. For 32-bit tests, I'd rather create a Gentoo a 32-bit sparc chroot or rebootstrap Debian sparc.

Adrian

Signature · 4 lines

 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [email protected]
`. `'   Freie Universitaet Berlin - [email protected]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

David Miller

2017-07-10 18:42 UTC #7 lore raw

In reply to Meelis Roos

From: John Paul Adrian Glaubitz <[email protected]>
Date: Mon, 10 Jul 2017 20:35:21 +0200

Quoted 7 lines · David Miller

Quoted 2 lines
Yes, let's make sure that 32-bit userland support in the kernel gets even less testing, and gets even buggier over time.
My point was not 32-bit vs. 64-bit, my point was using an outdated userland from 2015. For 32-bit tests, I'd rather create a Gentoo a 32-bit sparc chroot or rebootstrap Debian sparc.

My userland is from 2013 with random built-from-src pieces of updated tools and glibc, should I just throw my hands up in the air when the kernel OOPSes and not look into it at all?

None of this makes any sense at all.

If a bug shows up in the kernel between X and Y, it doesn't matter if it triggers with an old userland or a new one. It should be investigated just the same.

John Paul Adrian Glaubitz

2017-07-10 18:47 UTC #8 lore raw

In reply to Meelis Roos

Quoted 3 lines · David Miller

My userland is from 2013 with random built-from-src pieces of updated tools and glibc, should I just throw my hands up in the air when the kernel OOPSes and not look into it at all?

Well, I know that around 2-3 years ago, lots of userland packages had issues on sparc which were consequently fixed. Here's a selection [1].

Testing with a current userland just reduces the probability to stumble into any of these issues.

Adrian

Quoted 1 line

[1] https://wiki.debian.org/PortsSparc

Signature · 4 lines

 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [email protected]
`. `'   Freie Universitaet Berlin - [email protected]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Meelis Roos

author

2017-07-11 11:48 UTC #9 lore raw

In reply to Meelis Roos

Quoted 6 lines

I tested yesterdays git with 4.13 sparc merge on a big bunch of my test machines.
Good news: the timestamps starting from boot seem to work on all the machines, starting with Ultra 1 and Ultra 2.

Quoted 6 lines

Bad news: all working sun4u machines have the following line in dmesg
(PID varies):
in:imklog[1036]: segfault at 49 ip 0000000000035bb0 (rpc 00000000f779dfa0) sp 00000002f7a78818 error 30002 in rsyslogd[10000+88000]
This is with Debian unstable 32-bit userland as of 2015 summer. It did not happen with 4.12.

I bisected it to the commit below. CC Al Viro.

In following commits around this one (some bisect steps before, and the last step) the crashes were slightly different from the original crash yersterday, like

in:imuxsock[1606]: segfault at fffffff8 ip 00000000f774e154 (rpc 00000000f774dfe8) sp 00000001f7965a68 error 30002 in libc-2.19.so[f76cc000+16a000]

but this was the first commit introducing the crashes.

d9e968cb9f849770288f5fde3d8d3a5f7e339052 is the first bad commit commit d9e968cb9f849770288f5fde3d8d3a5f7e339052

Author: Al Viro <[email protected]>
Date:   Wed May 31 04:33:51 2017 -0400

getrlimit()/setrlimit(): move compat to native

Signed-off-by Al Viro <[email protected]>

:040000 040000 5e1a0c25f4554ccdc824b18dc50078d379cdbf9a
6d820451a388c5771a2a410578365d14a0194e4e M kernel

Signature · 1 line

Meelis Roos ([email protected])

David Miller

2017-07-12 03:13 UTC #10 lore raw

From: Meelis Roos <[email protected]>
Date: Tue, 11 Jul 2017 14:48:27 +0300 (EEST)

Quoted 28 lines

Quoted 6 lines
I tested yesterdays git with 4.13 sparc merge on a big bunch of my test machines.
Good news: the timestamps starting from boot seem to work on all the machines, starting with Ultra 1 and Ultra 2.
Quoted 6 lines
Bad news: all working sun4u machines have the following line in dmesg
(PID varies):
in:imklog[1036]: segfault at 49 ip 0000000000035bb0 (rpc 00000000f779dfa0) sp 00000002f7a78818 error 30002 in rsyslogd[10000+88000]
This is with Debian unstable 32-bit userland as of 2015 summer. It did not happen with 4.12.
I bisected it to the commit below. CC Al Viro.
In following commits around this one (some bisect steps before, and the last step) the crashes were slightly different from the original crash yersterday, like
in:imuxsock[1606]: segfault at fffffff8 ip 00000000f774e154 (rpc 00000000f774dfe8) sp 00000001f7965a68 error 30002 in libc-2.19.so[f76cc000+16a000]
but this was the first commit introducing the crashes.
d9e968cb9f849770288f5fde3d8d3a5f7e339052 is the first bad commit commit d9e968cb9f849770288f5fde3d8d3a5f7e339052
Author: Al Viro <[email protected]>
Date:   Wed May 31 04:33:51 2017 -0400

Quoted 6 lines

getrlimit()/setrlimit(): move compat to native
Signed-off-by Al Viro <[email protected]>
:040000 040000 5e1a0c25f4554ccdc824b18dc50078d379cdbf9a
6d820451a388c5771a2a410578365d14a0194e4e M kernel

I can't figure out what is wrong in this change, except that it does two things at once.

It moves code, and changes it at the same time. That should have been done in two changes.

Because of that, we can't tell which of the two things introduced the regression.

Al?

Al Viro

2017-07-12 03:59 UTC #11 lore raw

In reply to Meelis Roos

Quoted 4 lines

In following commits around this one (some bisect steps before, and the last step) the crashes were slightly different from the original crash yersterday, like

Quoted 9 lines

in:imuxsock[1606]: segfault at fffffff8 ip 00000000f774e154 (rpc 00000000f774dfe8) sp 00000001f7965a68 error 30002 in libc-2.19.so[f76cc000+16a000]
but this was the first commit introducing the crashes.
d9e968cb9f849770288f5fde3d8d3a5f7e339052 is the first bad commit commit d9e968cb9f849770288f5fde3d8d3a5f7e339052
Author: Al Viro <[email protected]>
Date:   Wed May 31 04:33:51 2017 -0400

w getrlimit()/setrlimit(): move compat to native

Quoted 8 lines

Signed-off-by Al Viro <[email protected]>
:040000 040000 5e1a0c25f4554ccdc824b18dc50078d379cdbf9a
6d820451a388c5771a2a410578365d14a0194e4e M kernel
Signature · 1 line
Meelis Roos ([email protected])

With apologies for ed scripts,

ed kernel/sys.c <<'EOF'
/struct rlimit r32/s/rlimit/compat_&/
w
q
EOF

AFAICS, that should fix it. If it does, commit message would be

fix a braino in compat_sys_getrlimit()

Fixes commit d9e968cb9f84 "getrlimit()/setrlimit(): move compat to native"

Signed-off-by Al Viro <[email protected]>

---

Again, my apologies for format of that all - I'm almost completely off-net and while typing an ed script is feasible, context diff is too far over the top. Usimg mutt in ssh session from android is not fun, to put it mildly...

David Miller

Acked-by

2017-07-12 04:09 UTC #12 lore raw

From: Al Viro <[email protected]>
Date: Wed, 12 Jul 2017 04:59:45 +0100

Quoted 4 lines

Again, my apologies for format of that all - I'm almost completely off-net and while typing an ed script is feasible, context diff is too far over the top. Usimg mutt in ssh session from android is not fun, to put it mildly...

Yes, that might be it.

Meelis, please test this:

==========
fix a braino in compat_sys_getrlimit()

Fixes commit d9e968cb9f84 "getrlimit()/setrlimit(): move compat to native"

Signed-off-by Al Viro <[email protected]>

Acked-by David S. Miller <[email protected]>

kernel/sys.c +1 -1

index 73fc0af..2855ee7 100644
@@ -1362,7 +1362,7 @@ COMPAT_SYSCALL_DEFINE2(getrlimit, unsigned int, resource,
 
 	ret = do_prlimit(current, resource, NULL, &r);
 	if (!ret) {
-		struct rlimit r32;
+		struct compat_rlimit r32;
 		if (r.rlim_cur > COMPAT_RLIM_INFINITY)
 			r32.rlim_cur = COMPAT_RLIM_INFINITY;
 		else

Meelis Roos

author

2017-07-12 07:09 UTC #13 lore raw

Quoted 10 lines

From: Al Viro <[email protected]>
Date: Wed, 12 Jul 2017 04:59:45 +0100
Quoted 4 lines
Again, my apologies for format of that all - I'm almost completely off-net and while typing an ed script is feasible, context diff is too far over the top. Usimg mutt in ssh session from android is not fun, to put it mildly...
Yes, that might be it.

Quoted 8 lines

Meelis, please test this:
==========
fix a braino in compat_sys_getrlimit()
Fixes commit d9e968cb9f84 "getrlimit()/setrlimit(): move compat to native"
Signed-off-by Al Viro <[email protected]>
Acked-by David S. Miller <[email protected]>

Yes, it works right on top of 4.12.0-rc1-00007-gd9e968c and on top of todays git as well.

Signature · 1 line

Meelis Roos ([email protected])

David Miller

2017-07-12 16:09 UTC #14 lore raw

From: Meelis Roos <[email protected]>
Date: Wed, 12 Jul 2017 10:09:50 +0300 (EEST)

Quoted 15 lines

Quoted 15 lines
From: Al Viro <[email protected]>
Date: Wed, 12 Jul 2017 04:59:45 +0100
Quoted 4 lines
Again, my apologies for format of that all - I'm almost completely off-net and while typing an ed script is feasible, context diff is too far over the top. Usimg mutt in ssh session from android is not fun, to put it mildly...
Yes, that might be it.
Meelis, please test this:
==========
fix a braino in compat_sys_getrlimit()

Quoted 6 lines

Quoted 3 lines
Fixes commit d9e968cb9f84 "getrlimit()/setrlimit(): move compat to native"
Signed-off-by Al Viro <[email protected]>
Acked-by David S. Miller <[email protected]>
Yes, it works right on top of 4.12.0-rc1-00007-gd9e968c and on top of todays git as well.

Al, please push this to Linus soon.

Thanks.

Linus Torvalds

2017-07-12 16:14 UTC #15 lore raw

Quoted 2 lines · David Miller

Al, please push this to Linus soon.

I'll take it directly, Al is currently in Boston without a real computer methinks.

Linus

David Miller

2017-07-11 13:56 UTC #16 lore raw

In reply to Meelis Roos

From: John Paul Adrian Glaubitz <[email protected]>
Date: Mon, 10 Jul 2017 20:47:54 +0200

Quoted 2 lines

Testing with a current userland just reduces the probability to stumble into any of these issues.

But things worked %100 properly before he changed the kernel.