x86: default to vsyscall=native

After upgrading a kernel the existing userspace should just work (assuming it did work before ;-) ), but when I upgraded my kernel from 3.0.4 to 3.1.0-rc8 a UML instance didn't come up properly.

dmesg said:
 linux-2.6.30.1[3800] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb9c498 ax:ffffffffff600000 si:0 di:606790 linux-2.6.30.1[3856] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb13168 ax:ffffffffff600000 si:0 di:606790

Looking throught the changelog I ended up at commit 3ae36655 ("x86-64: Rework vsyscall emulation and add vsyscall= parameter").

Linus suggested in https://lkml.org/lkml/2011/8/9/376 to default to vsyscall=native.

That sounds reasonable to me, and fixes the problem for me.

Signed-off-by Adrian Bunk <[email protected]>

---
 Documentation/kernel-parameters.txt |    7 ++++--- arch/x86/kernel/vsyscall_64.c       |    2 +-

 2 files changed, 5 insertions(+), 4 deletions(-)

Documentation/kernel-parameters.txt +0 -0

index 854ed5ca..d6e6724 100644
@@ -2706,10 +2706,11 @@ bytes respectively. Such letter suffixes can also be entirely omitted.

                       functions are at fixed addresses, they make nice targets for exploits that can control RIP.

-                       emulate     [default] Vsyscalls turn into traps and are
-                                   emulated reasonably safely.
+                       emulate     Vsyscalls turn into traps and are emulated
+                                   reasonably safely.

-                       native      Vsyscalls are native syscall instructions.
+                       native      [default] Vsyscalls are native syscall
+                                   instructions.
                                   This is a little bit faster than trapping and makes a few dynamic recompilers work better than they would in emulation mode.

arch/x86/kernel/vsyscall_64.c +1 -2

index 18ae83d..b56c65de 100644
@@ -56,7 +56,7 @@ DEFINE_VVAR(struct vsyscall_gtod_data, vsyscall_gtod_data) =
        .lock = __SEQLOCK_UNLOCKED(__vsyscall_gtod_data.lock),
  };
 
-static enum { EMULATE, NATIVE, NONE } vsyscall_mode = EMULATE;
+static enum { EMULATE, NATIVE, NONE } vsyscall_mode = NATIVE;
 
  static int __init vsyscall_setup(char *str)
  {
--

1.7.6.3

Quoted 14 lines · Adrian Bunk

After upgrading a kernel the existing userspace should just work (assuming it did work before ;-) ), but when I upgraded my kernel from 3.0.4 to 3.1.0-rc8 a UML instance didn't come up properly.

dmesg said:
 linux-2.6.30.1[3800] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb9c498 ax:ffffffffff600000 si:0 di:606790 linux-2.6.30.1[3856] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb13168 ax:ffffffffff600000 si:0 di:606790

Looking throught the changelog I ended up at commit 3ae36655 ("x86-64: Rework vsyscall emulation and add vsyscall= parameter").

Linus suggested in https://lkml.org/lkml/2011/8/9/376 to default to vsyscall=native.

Quoted 1 line

That sounds reasonable to me, and fixes the problem for me.

At this point in the -rc cycle, this sounds fine.

That being said, I'd like to fix it for real for 3.2. This particular failure is suspicious -- the "vsyscall fault" message means that sys_gettimeofday returned EFAULT, which means that the old (3.0 and before) vgettimeofday should *also* have segfaulted.

We do have a bit
of a bug in that the new code doesn't report si_addr properly, but that sounds unlikely as a culprit. Did you try with the offending commit reverted (i.e. fce8dc0)? I bet that it also fails there.

What's the .config for your UML binary? I'd like to see if I can reproduce this.

--Andy

Quoted 15 lines · Andrew Lutomirski

Quoted 14 lines · Adrian Bunk

After upgrading a kernel the existing userspace should just work (assuming it did work before ;-) ), but when I upgraded my kernel from 3.0.4 to 3.1.0-rc8 a UML instance didn't come up properly.

dmesg said:
 linux-2.6.30.1[3800] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb9c498 ax:ffffffffff600000 si:0 di:606790 linux-2.6.30.1[3856] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb13168 ax:ffffffffff600000 si:0 di:606790

Looking throught the changelog I ended up at commit 3ae36655 ("x86-64: Rework vsyscall emulation and add vsyscall= parameter").

Linus suggested in https://lkml.org/lkml/2011/8/9/376 to default to vsyscall=native.

Quoted 8 lines

Quoted 1 line

That sounds reasonable to me, and fixes the problem for me.

At this point in the -rc cycle, this sounds fine.

That being said, I'd like to fix it for real for 3.2.  This particular failure is suspicious -- the "vsyscall fault" message means that sys_gettimeofday returned EFAULT, which means that the old (3.0 and before) vgettimeofday should *also* have segfaulted.

This 2.6.30.1 UML kernel binary from 2009 worked for me for all host kernels from 2.6.30 to 3.0, and with 3.1.0-rc8 and vsyscall=native it also seems to run nicely.

Looking deeper into "a UML instance didn't come up properly", the problem is that it comes up in a strange (readonly) state.

There are "Using makefile-style concurrent boot in runlevel S." and "Using makefile-style concurrent boot in runlevel 2." in the logs with a Debian userspace, but no output from the init scripts in these broken bootups (normal messages are in non-broken bootups).

Perhaps the two the messages I see in dmesg on the host are from the processes running rcS and rc2 failing early?

In a working startup with a Debian userspace, I'm getting during rcS Setting the system clock.
 Cannot access the Hardware Clock via any known method.
 Use the --debug option to see the details of our search for an access method. Unable to set System Clock to: Mon Oct 3 17:01:35 UTC 2011 ... (warning).

Quoted 4 lines

We do have a bit
of a bug in that the new code doesn't report si_addr properly, but that sounds unlikely as a culprit.  Did you try with the offending commit reverted (i.e. fce8dc0)?  I bet that it also fails there.

fce8dc0 is "x86-64: Wire up getcpu syscall", is that really the one you want me to revert?

Quoted 3 lines

What's the .config for your UML binary?  I'd like to see if I can reproduce this.

It's attached.

Quoted 1 line · Adrian Bunk

On Mon, Oct 03, 2011 at 06:04:53AM -0700, Andrew Lutomirski wrote:

...

Quoted 4 lines

Quoted 3 lines

of a bug in that the new code doesn't report si_addr properly, but that sounds unlikely as a culprit.  Did you try with the offending commit reverted (i.e. fce8dc0)?  I bet that it also fails there.

Quoted 2 lines

fce8dc0 is "x86-64: Wire up getcpu syscall", is that really the one you want me to revert?

No -- I actually meant to try running that revision or to try with the vsyscall= patch reverted.

...
--Andy

Quoted 15 lines · Andrew Lutomirski

Quoted 14 lines · Adrian Bunk

After upgrading a kernel the existing userspace should just work (assuming it did work before ;-) ), but when I upgraded my kernel from 3.0.4 to 3.1.0-rc8 a UML instance didn't come up properly.

dmesg said:
 linux-2.6.30.1[3800] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb9c498 ax:ffffffffff600000 si:0 di:606790 linux-2.6.30.1[3856] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb13168 ax:ffffffffff600000 si:0 di:606790

Looking throught the changelog I ended up at commit 3ae36655 ("x86-64: Rework vsyscall emulation and add vsyscall= parameter").

Linus suggested in https://lkml.org/lkml/2011/8/9/376 to default to vsyscall=native.

Quoted 8 lines

Quoted 1 line

That sounds reasonable to me, and fixes the problem for me.

At this point in the -rc cycle, this sounds fine.

That being said, I'd like to fix it for real for 3.2.  This particular failure is suspicious -- the "vsyscall fault" message means that sys_gettimeofday returned EFAULT, which means that the old (3.0 and before) vgettimeofday should *also* have segfaulted.

This 2.6.30.1 UML kernel binary from 2009 worked for me for all host kernels from 2.6.30 to 3.0, and with 3.1.0-rc8 and vsyscall=native it also seems to run nicely.

Looking deeper into "a UML instance didn't come up properly", the problem is that it comes up in a strange (readonly) state.

There are "Using makefile-style concurrent boot in runlevel S." and "Using makefile-style concurrent boot in runlevel 2." in the logs with a Debian userspace, but no output from the init scripts in these broken bootups (normal messages are in non-broken bootups).

Perhaps the two the messages I see in dmesg on the host are from the processes running rcS and rc2 failing early?

In a working startup with a Debian userspace, I'm getting during rcS Setting the system clock.
 Cannot access the Hardware Clock via any known method.
 Use the --debug option to see the details of our search for an access method. Unable to set System Clock to: Mon Oct 3 17:01:35 UTC 2011 ... (warning).

Quoted 4 lines

We do have a bit
of a bug in that the new code doesn't report si_addr properly, but that sounds unlikely as a culprit.  Did you try with the offending commit reverted (i.e. fce8dc0)?  I bet that it also fails there.

fce8dc0 is "x86-64: Wire up getcpu syscall", is that really the one you want me to revert?

Quoted 2 lines

What's the .config for your UML binary?  I'd like to see if I can reproduce this.

It's attached.

Quoted 54 lines · Adrian Bunk

Quoted 15 lines · Andrew Lutomirski

Quoted 14 lines · Adrian Bunk

After upgrading a kernel the existing userspace should just work (assuming it did work before ;-) ), but when I upgraded my kernel from 3.0.4 to 3.1.0-rc8 a UML instance didn't come up properly.

dmesg said:
 linux-2.6.30.1[3800] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb9c498 ax:ffffffffff600000 si:0 di:606790 linux-2.6.30.1[3856] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb13168 ax:ffffffffff600000 si:0 di:606790

Looking throught the changelog I ended up at commit 3ae36655 ("x86-64: Rework vsyscall emulation and add vsyscall= parameter").

Linus suggested in https://lkml.org/lkml/2011/8/9/376 to default to vsyscall=native.

Quoted 8 lines

Quoted 1 line

That sounds reasonable to me, and fixes the problem for me.

At this point in the -rc cycle, this sounds fine.

That being said, I'd like to fix it for real for 3.2.  This particular failure is suspicious -- the "vsyscall fault" message means that sys_gettimeofday returned EFAULT, which means that the old (3.0 and before) vgettimeofday should *also* have segfaulted.

This 2.6.30.1 UML kernel binary from 2009 worked for me for all host kernels from 2.6.30 to 3.0, and with 3.1.0-rc8 and vsyscall=native it also seems to run nicely.

Looking deeper into "a UML instance didn't come up properly", the problem is that it comes up in a strange (readonly) state.

There are "Using makefile-style concurrent boot in runlevel S." and "Using makefile-style concurrent boot in runlevel 2." in the logs with a Debian userspace, but no output from the init scripts in these broken bootups (normal messages are in non-broken bootups).

Perhaps the two the messages I see in dmesg on the host are from the processes running rcS and rc2 failing early?

In a working startup with a Debian userspace, I'm getting during rcS Setting the system clock.
 Cannot access the Hardware Clock via any known method.
 Use the --debug option to see the details of our search for an access method. Unable to set System Clock to: Mon Oct 3 17:01:35 UTC 2011 ... (warning).

Quoted 4 lines

We do have a bit
of a bug in that the new code doesn't report si_addr properly, but that sounds unlikely as a culprit.  Did you try with the offending commit reverted (i.e. fce8dc0)?  I bet that it also fails there.

fce8dc0 is "x86-64: Wire up getcpu syscall", is that really the one you want me to revert?

Quoted 5 lines

Quoted 2 lines

What's the .config for your UML binary?  I'd like to see if I can reproduce this.

It's attached.

I can't reproduce it.  What distro is running inside the UML instance?

Quoted 58 lines · Andrew Lutomirski

Quoted 57 lines · Adrian Bunk

Quoted 15 lines · Andrew Lutomirski

Quoted 14 lines · Adrian Bunk

After upgrading a kernel the existing userspace should just work (assuming it did work before ;-) ), but when I upgraded my kernel from 3.0.4 to 3.1.0-rc8 a UML instance didn't come up properly.

dmesg said:
 linux-2.6.30.1[3800] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb9c498 ax:ffffffffff600000 si:0 di:606790 linux-2.6.30.1[3856] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb13168 ax:ffffffffff600000 si:0 di:606790

Looking throught the changelog I ended up at commit 3ae36655 ("x86-64: Rework vsyscall emulation and add vsyscall= parameter").

Linus suggested in https://lkml.org/lkml/2011/8/9/376 to default to vsyscall=native.

Quoted 8 lines

Quoted 1 line

That sounds reasonable to me, and fixes the problem for me.

At this point in the -rc cycle, this sounds fine.

That being said, I'd like to fix it for real for 3.2.  This particular failure is suspicious -- the "vsyscall fault" message means that sys_gettimeofday returned EFAULT, which means that the old (3.0 and before) vgettimeofday should *also* have segfaulted.

This 2.6.30.1 UML kernel binary from 2009 worked for me for all host kernels from 2.6.30 to 3.0, and with 3.1.0-rc8 and vsyscall=native it also seems to run nicely.

Looking deeper into "a UML instance didn't come up properly", the problem is that it comes up in a strange (readonly) state.

There are "Using makefile-style concurrent boot in runlevel S." and "Using makefile-style concurrent boot in runlevel 2." in the logs with a Debian userspace, but no output from the init scripts in these broken bootups (normal messages are in non-broken bootups).

Perhaps the two the messages I see in dmesg on the host are from the processes running rcS and rc2 failing early?

In a working startup with a Debian userspace, I'm getting during rcS Setting the system clock.
 Cannot access the Hardware Clock via any known method.
 Use the --debug option to see the details of our search for an access method. Unable to set System Clock to: Mon Oct 3 17:01:35 UTC 2011 ... (warning).

Quoted 4 lines

We do have a bit
of a bug in that the new code doesn't report si_addr properly, but that sounds unlikely as a culprit.  Did you try with the offending commit reverted (i.e. fce8dc0)?  I bet that it also fails there.

fce8dc0 is "x86-64: Wire up getcpu syscall", is that really the one you want me to revert?

Quoted 2 lines

What's the .config for your UML binary?  I'd like to see if I can reproduce this.

Quoted 4 lines

Quoted 2 lines

It's attached.

I can't reproduce it.  What distro is running inside the UML instance?

Same here.
Adrian, is the UML kernel crashing before executing init?

We definitely need more information...

Thanks,
//richard

Quoted 59 lines · richard -rw- weinberger

Quoted 58 lines · Andrew Lutomirski

Quoted 57 lines · Adrian Bunk

Quoted 15 lines · Andrew Lutomirski

Quoted 14 lines · Adrian Bunk

After upgrading a kernel the existing userspace should just work (assuming it did work before ;-) ), but when I upgraded my kernel from 3.0.4 to 3.1.0-rc8 a UML instance didn't come up properly.

dmesg said:
 linux-2.6.30.1[3800] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb9c498 ax:ffffffffff600000 si:0 di:606790 linux-2.6.30.1[3856] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb13168 ax:ffffffffff600000 si:0 di:606790

Looking throught the changelog I ended up at commit 3ae36655 ("x86-64: Rework vsyscall emulation and add vsyscall= parameter").

Linus suggested in https://lkml.org/lkml/2011/8/9/376 to default to vsyscall=native.

Quoted 8 lines

Quoted 1 line

That sounds reasonable to me, and fixes the problem for me.

At this point in the -rc cycle, this sounds fine.

That being said, I'd like to fix it for real for 3.2.  This particular failure is suspicious -- the "vsyscall fault" message means that sys_gettimeofday returned EFAULT, which means that the old (3.0 and before) vgettimeofday should *also* have segfaulted.

This 2.6.30.1 UML kernel binary from 2009 worked for me for all host kernels from 2.6.30 to 3.0, and with 3.1.0-rc8 and vsyscall=native it also seems to run nicely.

Looking deeper into "a UML instance didn't come up properly", the problem is that it comes up in a strange (readonly) state.

There are "Using makefile-style concurrent boot in runlevel S." and "Using makefile-style concurrent boot in runlevel 2." in the logs with a Debian userspace, but no output from the init scripts in these broken bootups (normal messages are in non-broken bootups).

Perhaps the two the messages I see in dmesg on the host are from the processes running rcS and rc2 failing early?

In a working startup with a Debian userspace, I'm getting during rcS Setting the system clock.
 Cannot access the Hardware Clock via any known method.
 Use the --debug option to see the details of our search for an access method. Unable to set System Clock to: Mon Oct 3 17:01:35 UTC 2011 ... (warning).

Quoted 4 lines

We do have a bit
of a bug in that the new code doesn't report si_addr properly, but that sounds unlikely as a culprit.  Did you try with the offending commit reverted (i.e. fce8dc0)?  I bet that it also fails there.

fce8dc0 is "x86-64: Wire up getcpu syscall", is that really the one you want me to revert?

Quoted 2 lines

What's the .config for your UML binary?  I'd like to see if I can reproduce this.

Quoted 7 lines

Quoted 4 lines

Quoted 2 lines

It's attached.

I can't reproduce it.  What distro is running inside the UML instance?

Same here.
Adrian, is the UML kernel crashing before executing init?

As I wrote:
 Looking deeper into "a UML instance didn't come up properly", the problem is that it comes up in a strange (readonly) state.

The UML kernel is running happily without crashing, and as I wrote my guess about my problems is:
 Perhaps the two the messages I see in dmesg on the host are from the processes running rcS and rc2 failing early?

Quoted 1 line

We definitely need more information...

I gave the information that was requested. plus my observations.

Quoted 59 lines · richard -rw- weinberger

Quoted 58 lines · Andrew Lutomirski

Quoted 57 lines · Adrian Bunk

Quoted 15 lines · Andrew Lutomirski

Quoted 14 lines · Adrian Bunk

After upgrading a kernel the existing userspace should just work (assuming it did work before ;-) ), but when I upgraded my kernel from 3.0.4 to 3.1.0-rc8 a UML instance didn't come up properly.

dmesg said:
 linux-2.6.30.1[3800] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb9c498 ax:ffffffffff600000 si:0 di:606790 linux-2.6.30.1[3856] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb13168 ax:ffffffffff600000 si:0 di:606790

Looking throught the changelog I ended up at commit 3ae36655 ("x86-64: Rework vsyscall emulation and add vsyscall= parameter").

Linus suggested in https://lkml.org/lkml/2011/8/9/376 to default to vsyscall=native.

Quoted 8 lines

Quoted 1 line

That sounds reasonable to me, and fixes the problem for me.

At this point in the -rc cycle, this sounds fine.

That being said, I'd like to fix it for real for 3.2.  This particular failure is suspicious -- the "vsyscall fault" message means that sys_gettimeofday returned EFAULT, which means that the old (3.0 and before) vgettimeofday should *also* have segfaulted.

This 2.6.30.1 UML kernel binary from 2009 worked for me for all host kernels from 2.6.30 to 3.0, and with 3.1.0-rc8 and vsyscall=native it also seems to run nicely.

Looking deeper into "a UML instance didn't come up properly", the problem is that it comes up in a strange (readonly) state.

There are "Using makefile-style concurrent boot in runlevel S." and "Using makefile-style concurrent boot in runlevel 2." in the logs with a Debian userspace, but no output from the init scripts in these broken bootups (normal messages are in non-broken bootups).

Perhaps the two the messages I see in dmesg on the host are from the processes running rcS and rc2 failing early?

In a working startup with a Debian userspace, I'm getting during rcS Setting the system clock.
 Cannot access the Hardware Clock via any known method.
 Use the --debug option to see the details of our search for an access method. Unable to set System Clock to: Mon Oct 3 17:01:35 UTC 2011 ... (warning).

Quoted 4 lines

We do have a bit
of a bug in that the new code doesn't report si_addr properly, but that sounds unlikely as a culprit.  Did you try with the offending commit reverted (i.e. fce8dc0)?  I bet that it also fails there.

fce8dc0 is "x86-64: Wire up getcpu syscall", is that really the one you want me to revert?

Quoted 2 lines

What's the .config for your UML binary?  I'd like to see if I can reproduce this.

Quoted 7 lines

Quoted 4 lines

Quoted 2 lines

It's attached.

I can't reproduce it.  What distro is running inside the UML instance?

Same here.
Adrian, is the UML kernel crashing before executing init?

As I wrote:
 Looking deeper into "a UML instance didn't come up properly", the problem is that it comes up in a strange (readonly) state.

The UML kernel is running happily without crashing, and as I wrote my guess about my problems is:
 Perhaps the two the messages I see in dmesg on the host are from the processes running rcS and rc2 failing early?

Quoted 1 line

We definitely need more information...

I gave the information that was requested. plus my observations.

What more information exactly do you need from me?

Quoted 77 lines · Adrian Bunk

Quoted 59 lines · richard -rw- weinberger

Quoted 58 lines · Andrew Lutomirski

Quoted 57 lines · Adrian Bunk

Quoted 15 lines · Andrew Lutomirski

Quoted 14 lines · Adrian Bunk

After upgrading a kernel the existing userspace should just work (assuming it did work before ;-) ), but when I upgraded my kernel from 3.0.4 to 3.1.0-rc8 a UML instance didn't come up properly.

dmesg said:
 linux-2.6.30.1[3800] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb9c498 ax:ffffffffff600000 si:0 di:606790 linux-2.6.30.1[3856] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb13168 ax:ffffffffff600000 si:0 di:606790

Looking throught the changelog I ended up at commit 3ae36655 ("x86-64: Rework vsyscall emulation and add vsyscall= parameter").

Linus suggested in https://lkml.org/lkml/2011/8/9/376 to default to vsyscall=native.

Quoted 8 lines

Quoted 1 line

That sounds reasonable to me, and fixes the problem for me.

At this point in the -rc cycle, this sounds fine.

That being said, I'd like to fix it for real for 3.2.  This particular failure is suspicious -- the "vsyscall fault" message means that sys_gettimeofday returned EFAULT, which means that the old (3.0 and before) vgettimeofday should *also* have segfaulted.

This 2.6.30.1 UML kernel binary from 2009 worked for me for all host kernels from 2.6.30 to 3.0, and with 3.1.0-rc8 and vsyscall=native it also seems to run nicely.

Looking deeper into "a UML instance didn't come up properly", the problem is that it comes up in a strange (readonly) state.

There are "Using makefile-style concurrent boot in runlevel S." and "Using makefile-style concurrent boot in runlevel 2." in the logs with a Debian userspace, but no output from the init scripts in these broken bootups (normal messages are in non-broken bootups).

Perhaps the two the messages I see in dmesg on the host are from the processes running rcS and rc2 failing early?

In a working startup with a Debian userspace, I'm getting during rcS Setting the system clock.
 Cannot access the Hardware Clock via any known method.
 Use the --debug option to see the details of our search for an access method. Unable to set System Clock to: Mon Oct 3 17:01:35 UTC 2011 ... (warning).

Quoted 4 lines

We do have a bit
of a bug in that the new code doesn't report si_addr properly, but that sounds unlikely as a culprit.  Did you try with the offending commit reverted (i.e. fce8dc0)?  I bet that it also fails there.

fce8dc0 is "x86-64: Wire up getcpu syscall", is that really the one you want me to revert?

Quoted 2 lines

What's the .config for your UML binary?  I'd like to see if I can reproduce this.

Quoted 7 lines

Quoted 4 lines

Quoted 2 lines

It's attached.

I can't reproduce it.  What distro is running inside the UML instance?

Same here.
Adrian, is the UML kernel crashing before executing init?

As I wrote:
 Looking deeper into "a UML instance didn't come up properly", the problem is that it comes up in a strange (readonly) state.

The UML kernel is running happily without crashing, and as I wrote my guess about my problems is:
 Perhaps the two the messages I see in dmesg on the host are from the processes running rcS and rc2 failing early?

Quoted 5 lines

Quoted 1 line

We definitely need more information...

I gave the information that was requested. plus my observations.

What more information exactly do you need from me?

None :)  I just reproduced the problem with Debian Squeeze.  Lenny works fine.

Quoted 78 lines · Andrew Lutomirski

Quoted 77 lines · Adrian Bunk

Quoted 59 lines · richard -rw- weinberger

Quoted 58 lines · Andrew Lutomirski

Quoted 57 lines · Adrian Bunk

Quoted 15 lines · Andrew Lutomirski

Quoted 14 lines · Adrian Bunk

After upgrading a kernel the existing userspace should just work (assuming it did work before ;-) ), but when I upgraded my kernel from 3.0.4 to 3.1.0-rc8 a UML instance didn't come up properly.

dmesg said:
 linux-2.6.30.1[3800] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb9c498 ax:ffffffffff600000 si:0 di:606790 linux-2.6.30.1[3856] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb13168 ax:ffffffffff600000 si:0 di:606790

Looking throught the changelog I ended up at commit 3ae36655 ("x86-64: Rework vsyscall emulation and add vsyscall= parameter").

Linus suggested in https://lkml.org/lkml/2011/8/9/376 to default to vsyscall=native.

Quoted 8 lines

Quoted 1 line

That sounds reasonable to me, and fixes the problem for me.

At this point in the -rc cycle, this sounds fine.

That being said, I'd like to fix it for real for 3.2.  This particular failure is suspicious -- the "vsyscall fault" message means that sys_gettimeofday returned EFAULT, which means that the old (3.0 and before) vgettimeofday should *also* have segfaulted.

This 2.6.30.1 UML kernel binary from 2009 worked for me for all host kernels from 2.6.30 to 3.0, and with 3.1.0-rc8 and vsyscall=native it also seems to run nicely.

Looking deeper into "a UML instance didn't come up properly", the problem is that it comes up in a strange (readonly) state.

There are "Using makefile-style concurrent boot in runlevel S." and "Using makefile-style concurrent boot in runlevel 2." in the logs with a Debian userspace, but no output from the init scripts in these broken bootups (normal messages are in non-broken bootups).

Perhaps the two the messages I see in dmesg on the host are from the processes running rcS and rc2 failing early?

In a working startup with a Debian userspace, I'm getting during rcS Setting the system clock.
 Cannot access the Hardware Clock via any known method.
 Use the --debug option to see the details of our search for an access method. Unable to set System Clock to: Mon Oct 3 17:01:35 UTC 2011 ... (warning).

Quoted 4 lines

We do have a bit
of a bug in that the new code doesn't report si_addr properly, but that sounds unlikely as a culprit.  Did you try with the offending commit reverted (i.e. fce8dc0)?  I bet that it also fails there.

fce8dc0 is "x86-64: Wire up getcpu syscall", is that really the one you want me to revert?

Quoted 2 lines

What's the .config for your UML binary?  I'd like to see if I can reproduce this.

Quoted 7 lines

Quoted 4 lines

Quoted 2 lines

It's attached.

I can't reproduce it.  What distro is running inside the UML instance?

Same here.
Adrian, is the UML kernel crashing before executing init?

As I wrote:
 Looking deeper into "a UML instance didn't come up properly", the problem is that it comes up in a strange (readonly) state.

The UML kernel is running happily without crashing, and as I wrote my guess about my problems is:
 Perhaps the two the messages I see in dmesg on the host are from the processes running rcS and rc2 failing early?

Quoted 7 lines

Quoted 5 lines

Quoted 1 line

We definitely need more information...

I gave the information that was requested. plus my observations.

What more information exactly do you need from me?

None :)  I just reproduced the problem with Debian Squeeze.  Lenny works fine.

This is strange.  The problem appears to be in startpar.  That same exact Debian image works fine on KVM running 3.1-rc8 (with vsyscall=emulate) and on 2.6.40 (i.e. Fedora 15's kernel).  If I set print-fatal-signals=1 I don't see a fatal signal in startpar.

Richard, is it possible that UML 2.6.30.1 generates a bogus vgettimeofday and recovers successfully on older kernels because the resulting SIGSEGV had a valid sigcontext?  I can try hacking the "vsyscall fault" path to generate full sigcontext and info.  This seems rather unlikely, though.

Quoted 92 lines · Andrew Lutomirski

Quoted 78 lines · Andrew Lutomirski

Quoted 77 lines · Adrian Bunk

Quoted 59 lines · richard -rw- weinberger

Quoted 58 lines · Andrew Lutomirski

Quoted 57 lines · Adrian Bunk

Quoted 15 lines · Andrew Lutomirski

Quoted 14 lines · Adrian Bunk

After upgrading a kernel the existing userspace should just work (assuming it did work before ;-) ), but when I upgraded my kernel from 3.0.4 to 3.1.0-rc8 a UML instance didn't come up properly.

dmesg said:
 linux-2.6.30.1[3800] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb9c498 ax:ffffffffff600000 si:0 di:606790 linux-2.6.30.1[3856] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb13168 ax:ffffffffff600000 si:0 di:606790

Looking throught the changelog I ended up at commit 3ae36655 ("x86-64: Rework vsyscall emulation and add vsyscall= parameter").

Linus suggested in https://lkml.org/lkml/2011/8/9/376 to default to vsyscall=native.

Quoted 8 lines

Quoted 1 line

That sounds reasonable to me, and fixes the problem for me.

At this point in the -rc cycle, this sounds fine.

That being said, I'd like to fix it for real for 3.2.  This particular failure is suspicious -- the "vsyscall fault" message means that sys_gettimeofday returned EFAULT, which means that the old (3.0 and before) vgettimeofday should *also* have segfaulted.

This 2.6.30.1 UML kernel binary from 2009 worked for me for all host kernels from 2.6.30 to 3.0, and with 3.1.0-rc8 and vsyscall=native it also seems to run nicely.

Looking deeper into "a UML instance didn't come up properly", the problem is that it comes up in a strange (readonly) state.

There are "Using makefile-style concurrent boot in runlevel S." and "Using makefile-style concurrent boot in runlevel 2." in the logs with a Debian userspace, but no output from the init scripts in these broken bootups (normal messages are in non-broken bootups).

Perhaps the two the messages I see in dmesg on the host are from the processes running rcS and rc2 failing early?

In a working startup with a Debian userspace, I'm getting during rcS Setting the system clock.
 Cannot access the Hardware Clock via any known method.
 Use the --debug option to see the details of our search for an access method. Unable to set System Clock to: Mon Oct 3 17:01:35 UTC 2011 ... (warning).

Quoted 4 lines

We do have a bit
of a bug in that the new code doesn't report si_addr properly, but that sounds unlikely as a culprit.  Did you try with the offending commit reverted (i.e. fce8dc0)?  I bet that it also fails there.

fce8dc0 is "x86-64: Wire up getcpu syscall", is that really the one you want me to revert?

Quoted 2 lines

What's the .config for your UML binary?  I'd like to see if I can reproduce this.

Quoted 7 lines

Quoted 4 lines

Quoted 2 lines

It's attached.

I can't reproduce it.  What distro is running inside the UML instance?

Same here.
Adrian, is the UML kernel crashing before executing init?

As I wrote:
 Looking deeper into "a UML instance didn't come up properly", the problem is that it comes up in a strange (readonly) state.

The UML kernel is running happily without crashing, and as I wrote my guess about my problems is:
 Perhaps the two the messages I see in dmesg on the host are from the processes running rcS and rc2 failing early?

Quoted 7 lines

Quoted 5 lines

Quoted 1 line

We definitely need more information...

I gave the information that was requested. plus my observations.

What more information exactly do you need from me?

None :)  I just reproduced the problem with Debian Squeeze.  Lenny works fine.

This is strange.  The problem appears to be in startpar.  That same exact Debian image works fine on KVM running 3.1-rc8 (with vsyscall=emulate) and on 2.6.40 (i.e. Fedora 15's kernel).  If I set print-fatal-signals=1 I don't see a fatal signal in startpar.

Quoted 5 lines

Richard, is it possible that UML 2.6.30.1 generates a bogus vgettimeofday and recovers successfully on older kernels because the resulting SIGSEGV had a valid sigcontext?  I can try hacking the "vsyscall fault" path to generate full sigcontext and info.  This seems rather unlikely, though.

I think that is the problem.  UML appears to lazily set up "page tables" just like a real machine; it does this by handling SIGSEGV and calling handle_mm_fault.  If cr2 isn't set right, though, it doesn't know where the fault was and it can't handle it, so it just sends SIGSEGV to userspace.

In 3.0 and earlier, we don't crash but we malfunction differently: UML doesn't intercept the vsyscall at all and the guest sees the hosts's time.  This should be fixed in a newer version of UML.

In vsyscall=native mode, we DTRT because UML handles the syscall itself.

I'll see how ugly the patch to get this all correct is.  It may not be all that pretty because we won't be able to use sys_gettimeofday anymore.

I'll see how ugly the patch to get this all correct is.  It may not be all that pretty because we won't be able to use sys_gettimeofday anymore.

Quoted 3 lines · Andrew Lutomirski

I'll see how ugly the patch to get this all correct is.  It may not be all that pretty because we won't be able to use sys_gettimeofday anymore.

BTW: The attached program triggers the issue.

on 3.1-rc8+:

# ./sig.dyn
faulting address: 0xdeadbeef
# ./sig.static
[   19.075106] sig.static[863] vsyscall fault (exploit attempt?)
ip:ffffffffff600000 cs:33 sp:7fff9e53d8c8 ax:ffffffffff600000 si:0
di:deadbeef
faulting address: 0x0

I guess UML is not the only user of this feature...

Fixing it will be annoying because the attached fancier version needs to work, too.  I could implement the whole mess in software, but it might be nicer to arrange for uaccess errors to stash some information somewhere (like in the thread_struct cr2 variable).

Quoted 5 lines · Andrew Lutomirski

Fixing it will be annoying because the attached fancier version needs to work, too.  I could implement the whole mess in software, but it might be nicer to arrange for uaccess errors to stash some information somewhere (like in the thread_struct cr2 variable).

That should be easy enough to do. Just add it to the "fixup_exception()" case in no_context().

Quoted 9 lines · Linus Torvalds

Quoted 5 lines · Andrew Lutomirski

Fixing it will be annoying because the attached fancier version needs to work, too.  I could implement the whole mess in software, but it might be nicer to arrange for uaccess errors to stash some information somewhere (like in the thread_struct cr2 variable).

That should be easy enough to do. Just add it to the "fixup_exception()" case in no_context().

This code is rather messy.  We stash the cr2, err, and trap fields of sigcontext in thread_struct and we *never* reset them until the next segfault.  So userspace sees stale garbage on every signal that isn't a (genuine) segfault.  I can imagine this breaking UML is remarkably bizarre ways even without vsyscall emulation because UML actually seems to rely on that stuff to determine the source of a signal.

From UML's point of view the current situation is odd.

From UML's point of view the current situation is odd. UML will no longer run on top of a default 3.1 kernel.

Why is this odd? One of the major reasons why people are still using UML is because you can run it as non-privileged user on any x86 Linux host. An user which has root privileges can setup and use KVM which is much nicer than UML...

Quoted 2 lines · richard -rw- weinberger

From UML's point of view the current situation is odd. UML will no longer run on top of a default 3.1 kernel.

This needs to be fixed (perhaps worked around in UML if that's possible and if you agree with that) - or barring a real obvious fix needs to be reverted to the last-known-working state. We are in -rc9 so nothing but really, really obvious patches can be applied.

Quoted 4 lines

Why is this odd? One of the major reasons why people are still using UML is because you can run it as non-privileged user on any x86 Linux host. An user which has root privileges can setup and use KVM which is much nicer than UML...

No, your complaint is entirely justified.

Andrew?

Quoted 1 line

Andrew?

I think I know what the root cause is and I have most of a patch to fix it. It doesn't compile (yet), it's a little less trivial than I'd like for something this late in the -rc cycle, and it adds 16 bytes to thread_struct (ugh!).

I think I can make a follow-up patch that removes 32 bytes of per-thread state to restore my karma, though, but that will definitely not be 3.1 material.

Quoted 7 lines · Andrew Lutomirski

Quoted 1 line

Andrew?

I think I know what the root cause is and I have most of a patch to fix it. It doesn't compile (yet), it's a little less trivial than I'd like for something this late in the -rc cycle, and it adds 16 bytes to thread_struct (ugh!).

Quoted 3 lines

I think I can make a follow-up patch that removes 32 bytes of per-thread state to restore my karma, though, but that will definitely not be 3.1 material.

Ok, i've queued up the vsyscall=native patch in tip:x86/urgent for now - we can re-try in v3.2 (perhaps) if a satisfactory solution is found.

Thanks,

Ingo

So here's a different approach.  It's not perfect: it always blames SEGV_MAPERR instead of SEGV_ACCERR.  I implemented it for vgettimeofday but not the other two vsyscalls.

What do you think of this approach?  If it seems good, I'll finish the patch and submit it.

Quoted 7 lines · Andrew Lutomirski

So here's a different approach.  It's not perfect: it always blames SEGV_MAPERR instead of SEGV_ACCERR.  I implemented it for vgettimeofday but not the other two vsyscalls.

What do you think of this approach?  If it seems good, I'll finish the patch and submit it.

I think the approach is valid, but you should *not* do this as some kind of crazy byte-by-byte copy_to_user() emulation.

Do the "copy tz to user mode" as individual "put_user()" calls for tv_sec/tv_usec/timezone. IOW, there are three words being written to user mode, not "two memcpy's".

Other than that, there doesn't seem to be anything wrong.

           Linus

How does that work?  The tricky case is when one of those three words spans a page boundary if the access to the first page is valid, but the access to the second page is not.  When that happens, if we report the fault as coming from the first page, then UML is likely to get think the fault was spurious and enter an infinite loop.

Quoted 6 lines · Andrew Lutomirski

How does that work?  The tricky case is when one of those three words spans a page boundary if the access to the first page is valid, but the access to the second page is not.  When that happens, if we report the fault as coming from the first page, then UML is likely to get think the fault was spurious and enter an infinite loop.

Hmm. Gaah, I just find that memcpy loop disgusting.

We already have that ugly "uaccess_error" crap in handle_exception(), we might as well do something like the attached and just say "hey, now you can catch the page fault information for a get_user/put_user fault".

Isn't that much nicer?

Another idea is to have a flag that asks the fault handlers to call force_sig_info for us.  That's just one bit of per-thread state.  Then the vsyscall emulation code could check access_ok, force a signal if access is not ok, then set the flag and do the syscall.  And maybe some processes would want to opt in to that mode anyway -- arguably EFAULT is a serious programmer error and should be dealt with more harshly than other syscall misuses.

arch/x86/kernel/vsyscall_64.c static int addr_to_vsyscall_nr(unsigned long addr) · line 145

Quoted 5 lines · Andy Lutomirski

arch/x86/kernel/vsyscall_64.c +4 -0

+static bool write_ok_or_segv(unsigned long ptr, size_t size)
+{
+       if (ptr == 0)
+               return true;

Why is ptr==0 special? That makes no sense.

Also, this whole function makes the notion of setting the "sigsegv on fault" flag much less interesting. It would be much better if access_ok() (including the cases embedded in get_user/put_user/etc) just did it right automatically for everything, rather than special-casing it for just this.

I wonder if we could just make access_ok() use a trap instead of just the regular compares (and then in the trap handler do the same logic as in the page fault handler)? Sadly, the 'bounds' instruction doesn't work for this (in 32-bit mode it does a *signed* compare, and in 64-bit mode it no longer exists), but something like that might.

That said, I think that your patch looks acceptable as a "let's fix vsyscalls without doing the bigger change". But I really don't see why ptr==0 would be special.

So I think your write_ok_or_segv() function should just be

  static bool write_ok_or_segv(unsigned long ptr, size_t size) {
     if (access_ok(ptr, size))
        return true;

      .. send signal ...

     return false;
  }

instead of that odd thing you have now.

The really nice fix (wiring up access_ok failures to be able to raise signals) won't be ready on time for 3.2, so let's try the simpler fix for now.

Quoted 3 lines · Andy Lutomirski

The really nice fix (wiring up access_ok failures to be able to raise signals) won't be ready on time for 3.2, so let's try the simpler fix for now.

I spoke to hpa about this a couple days ago, and he pointed out a problem with making access_ok send signals. Userspace expects signals that come with full context information to be restartable, and many system calls are not restartable. read() and write() are the obvious
examples: once they're processed the beginning of the buffer, unless they adjust their parameters, they can't safely be restarted. So without massive changes, I think allowing access_ok to raise a signal with full context is asking for trouble.

I can still do the patch with two modes: signals without context via arch_prctl and signals with context via vsyscall emulation, but that's probably overkill for fixing this bug. I'd say just apply these patches as is (for 3.3).

What do you think of this approach?  If it seems good, I'll finish the patch and submit it.

With this patch applied, UML appears to work, but it fills the log with exploit attempt warnings.  Any ideas on what to do about that?

Quoted 6 lines · Andrew Lutomirski

What do you think of this approach?  If it seems good, I'll finish the patch and submit it.

With this patch applied, UML appears to work, but it fills the log with exploit attempt warnings.  Any ideas on what to do about that?

I can confirm that this patch works.
And I really like vsyscall=emulate because with that UML can trap vsyscalls. :-)

Quoted 8 lines · richard -rw- weinberger

Quoted 6 lines · Andrew Lutomirski

What do you think of this approach?  If it seems good, I'll finish the patch and submit it.

With this patch applied, UML appears to work, but it fills the log with exploit attempt warnings.  Any ideas on what to do about that?

Quoted 2 lines

I can confirm that this patch works.
And I really like vsyscall=emulate because with that UML can trap vsyscalls. :-)

Are you sure you don't mean vsyscall=native?  I suspect that UML can't actually trap vsyscalls in emulate mode right now, although that ought to be fixable.

Quoted 13 lines · Andrew Lutomirski

Quoted 10 lines · richard -rw- weinberger

Quoted 6 lines · Andrew Lutomirski

What do you think of this approach?  If it seems good, I'll finish the patch and submit it.

With this patch applied, UML appears to work, but it fills the log with exploit attempt warnings.  Any ideas on what to do about that?

I can confirm that this patch works.
And I really like vsyscall=emulate because with that UML can trap vsyscalls. :-)

Quoted 4 lines

Are you sure you don't mean vsyscall=native?  I suspect that UML can't actually trap vsyscalls in emulate mode right now, although that ought to be fixable.

Doesn't vsyscall_emu_64.S transform any vsyscall into a real syscall? So UML can trap it.

Quoted 19 lines · richard -rw- weinberger

Quoted 17 lines · Andrew Lutomirski

Quoted 10 lines · richard -rw- weinberger

Quoted 6 lines · Andrew Lutomirski

What do you think of this approach?  If it seems good, I'll finish the patch and submit it.

With this patch applied, UML appears to work, but it fills the log with exploit attempt warnings.  Any ideas on what to do about that?

I can confirm that this patch works.
And I really like vsyscall=emulate because with that UML can trap vsyscalls. :-)

Are you sure you don't mean vsyscall=native?  I suspect that UML can't actually trap vsyscalls in emulate mode right now, although that ought to be fixable.

Quoted 2 lines

Doesn't vsyscall_emu_64.S transform any vsyscall into a real syscall? So UML can trap it.

Only if that code actually executes.  In vsyscall=emulate mode, the page is not executable and a trap is taken instead.  It's not entirely clear what the right thing to do is wrt ptrace users.

Quoted 8 lines · richard -rw- weinberger

Quoted 6 lines · Andrew Lutomirski

What do you think of this approach?  If it seems good, I'll finish the patch and submit it.

With this patch applied, UML appears to work, but it fills the log with exploit attempt warnings.  Any ideas on what to do about that?

Quoted 2 lines

I can confirm that this patch works.
And I really like vsyscall=emulate because with that UML can trap vsyscalls. :-)

Are you sure you don't mean vsyscall=native?  I suspect that UML can't actually trap vsyscalls in emulate mode right now, although that ought to be fixable.

Quoted 13 lines · Andrew Lutomirski

Quoted 10 lines · richard -rw- weinberger

Quoted 6 lines · Andrew Lutomirski

What do you think of this approach?  If it seems good, I'll finish the patch and submit it.

With this patch applied, UML appears to work, but it fills the log with exploit attempt warnings.  Any ideas on what to do about that?

I can confirm that this patch works.
And I really like vsyscall=emulate because with that UML can trap vsyscalls. :-)

Quoted 4 lines

Are you sure you don't mean vsyscall=native?  I suspect that UML can't actually trap vsyscalls in emulate mode right now, although that ought to be fixable.

§/%)"&!, you are so right!
I missed that vsyscall=native is the default setting now. Sorry for the confusion.

Quoted 54 lines · Adrian Bunk

Quoted 15 lines · Andrew Lutomirski

Quoted 14 lines · Adrian Bunk

After upgrading a kernel the existing userspace should just work (assuming it did work before ;-) ), but when I upgraded my kernel from 3.0.4 to 3.1.0-rc8 a UML instance didn't come up properly.

dmesg said:
 linux-2.6.30.1[3800] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb9c498 ax:ffffffffff600000 si:0 di:606790 linux-2.6.30.1[3856] vsyscall fault (exploit attempt?) ip:ffffffffff600000 cs:33 sp:7fbfb13168 ax:ffffffffff600000 si:0 di:606790

Looking throught the changelog I ended up at commit 3ae36655 ("x86-64: Rework vsyscall emulation and add vsyscall= parameter").

Linus suggested in https://lkml.org/lkml/2011/8/9/376 to default to vsyscall=native.

Quoted 8 lines

Quoted 1 line

That sounds reasonable to me, and fixes the problem for me.

At this point in the -rc cycle, this sounds fine.

That being said, I'd like to fix it for real for 3.2.  This particular failure is suspicious -- the "vsyscall fault" message means that sys_gettimeofday returned EFAULT, which means that the old (3.0 and before) vgettimeofday should *also* have segfaulted.

This 2.6.30.1 UML kernel binary from 2009 worked for me for all host kernels from 2.6.30 to 3.0, and with 3.1.0-rc8 and vsyscall=native it also seems to run nicely.

Looking deeper into "a UML instance didn't come up properly", the problem is that it comes up in a strange (readonly) state.

There are "Using makefile-style concurrent boot in runlevel S." and "Using makefile-style concurrent boot in runlevel 2." in the logs with a Debian userspace, but no output from the init scripts in these broken bootups (normal messages are in non-broken bootups).

Perhaps the two the messages I see in dmesg on the host are from the processes running rcS and rc2 failing early?

In a working startup with a Debian userspace, I'm getting during rcS Setting the system clock.
 Cannot access the Hardware Clock via any known method.
 Use the --debug option to see the details of our search for an access method. Unable to set System Clock to: Mon Oct 3 17:01:35 UTC 2011 ... (warning).

Quoted 4 lines

We do have a bit
of a bug in that the new code doesn't report si_addr properly, but that sounds unlikely as a culprit.  Did you try with the offending commit reverted (i.e. fce8dc0)?  I bet that it also fails there.

fce8dc0 is "x86-64: Wire up getcpu syscall", is that really the one you want me to revert?

Quoted 5 lines

Quoted 2 lines

What's the .config for your UML binary?  I'd like to see if I can reproduce this.

It's attached.

I can't reproduce it. What distro is running inside the UML instance?

--Andy

After upgrading a kernel the existing userspace should just work (assuming it did work before ;-) ), but when I upgraded my kernel from 3.0.4 to 3.1.0-rc8 a UML instance didn't come up properly.

Adrian,

Quoted 3 lines · Adrian Bunk

After upgrading a kernel the existing userspace should just work (assuming it did work before ;-) ), but when I upgraded my kernel from 3.0.4 to 3.1.0-rc8 a UML instance didn't come up properly.

Are only old UML kernels like 2.6.30.1 affected?

Anyway, it's time to upgrade my main machine to 3.1.0-rc8 to observe some new UML issues. ;-)

Signature · 2 lines

Thanks,
//richard