Skip to contentSkip to conversation

x86/signal/64: Fix SS handling for signals delivered to 64-bit programs breaks dosemu

regression Stas Sergeev 8 participants 121 messages Aug 12, 2015 – Sep 2, 2015 x86

lore
Share & tools
lore
#0 lore raw

Hi guys, I wonder how easily the include/uapi/* is being changed these days.
The patch:
http://lkml.kernel.org/r/405594361340a2ec32f8e2b115c142df0e180d8e.1426193719.git.luto@kernel.org
breaks dosemu (and perhaps everyone else who used to restore the segregs by hands). And the fix involves both autoconf magic and run-time magic, so it is not even trivial.
I realize this patch may be good to have in general, but breaking userspace without a single warning is a bit discouraging. Seems like the old "we don't break userspace" rule have gone.

#1 lore raw
Quoted 4 lines · Stas Sergeev

Hi guys, I wonder how easily the include/uapi/* is being changed these days.
The patch:
http://lkml.kernel.org/r/405594361340a2ec32f8e2b115c142df0e180d8e.1426193719.git.luto@kernel.org

Quoted 8 lines

breaks dosemu (and perhaps everyone else who used to restore the segregs by hands). And the fix involves both autoconf magic and run-time magic, so it is not even trivial.
I realize this patch may be good to have in general, but breaking userspace without a single warning is a bit discouraging. Seems like the old "we don't break userspace" rule have gone.

I didn't anticipate any breakage. I could have been wrong.

Do you know what the actual breakage is? I'm curious how this ever worked for DOSEMU, given that, before this patch, it appeared to be impossible to return to any nonstandard SS from a 64-bit signal handler. FWIW, DOSEMU seems to work for me on recent kernels.

We might still be able to require a new sigcontext flag to be set and to forcibly return to __USER_DS if the flag is set regardless of the ss value in sigcontext when sigreturn is called, if that is indeed the problem with DOSEMU. But I'm not actually sure that that's the problem.

In fact, DOSEMU contains this:

/* set up a frame to get back to DPMI via iret. The kernel does not save
%ss, and the SYSCALL instruction in sigreturn() destroys it.

IRET pops off everything in 64-bit mode even if the privilege does not change which is nice, but clobbers the high 48 bits of rsp if the DPMI client uses a 16-bit stack which is not so nice (see EMUfailure.txt). Setting %rsp to 0x100000000 so that bits 16-31 are zero works around this problem, as DPMI code can't see bits 32-63 anyway.
*/

So, if DOSEMU were to realize that both sigreturnissues it's complaining about are fixed in recent kernels, it could sigreturn directly back to any state. (It can't *directly* IRET back to a 16-bit state with properly controlled RSP, but it can do it via sigreturn.)

I don't actually see any code in DOSEMU that generates a sigcontext from scratch (as opposed to copying one and modifying it), so I'm not entirely sure what the problem is.

--Andy

Signature · 2 lines
Andy Lutomirski
AMA Capital Management, LLC
#2 lore raw

12.08.2015 03:38, Andy Lutomirski пишет:

Quoted 13 lines · Stas Sergeev
Quoted 4 lines

Hi guys, I wonder how easily the include/uapi/* is being changed these days.
The patch:
http://lkml.kernel.org/r/405594361340a2ec32f8e2b115c142df0e180d8e.1426193719.git.luto@kernel.org

Quoted 8 lines

breaks dosemu (and perhaps everyone else who used to restore the segregs by hands). And the fix involves both autoconf magic and run-time magic, so it is not even trivial.
I realize this patch may be good to have in general, but breaking userspace without a single warning is a bit discouraging. Seems like the old "we don't break userspace" rule have gone.

Quoted 1 line

I didn't anticipate any breakage. I could have been wrong.

You changed include/uapi/*, which is obviously an asking for problems. I applied the following changes to my local git tree to get dosemu working again:
https://github.com/stsp/dosemu2/commit/48b2a13a49a9fe1a456cd77df6b9a1feec675a01
https://github.com/stsp/dosemu2/commit/7898ac60d5e569964127d6cc48f592caecd20b81

Quoted 4 lines

Do you know what the actual breakage is? I'm curious how this ever worked for DOSEMU, given that, before this patch, it appeared to be impossible to return to any nonstandard SS from a 64-bit signal handler.

This is not the point.
What dosemu wants is to simply save the DOS SS somewhere. After your patch, it saves the Linux SS instead, then crashes.

Quoted 1 line

FWIW, DOSEMU seems to work for me on recent kernels.

Do you have any protected mode DOS program to test? I'll send you one in a private e-mail just about now.

Quoted 5 lines

We might still be able to require a new sigcontext flag to be set and to forcibly return to __USER_DS if the flag is set regardless of the ss value in sigcontext when sigreturn is called, if that is indeed the problem with DOSEMU. But I'm not actually sure that that's the problem.

Well, the flag would be an ideal solution in an ideal world, but in our world I don't know the current relevance of dosemu, and whether or not it worth a new flag to add.

Quoted 13 lines

In fact, DOSEMU contains this:

/* set up a frame to get back to DPMI via iret. The kernel does not save
%ss, and the SYSCALL instruction in sigreturn() destroys it.

IRET pops off everything in 64-bit mode even if the privilege does not change which is nice, but clobbers the high 48 bits of rsp if the DPMI client uses a 16-bit stack which is not so nice (see EMUfailure.txt). Setting %rsp to 0x100000000 so that bits 16-31 are zero works around this problem, as DPMI code can't see bits 32-63 anyway.
*/

Quoted 3 lines

So, if DOSEMU were to realize that both sigreturnissues it's complaining about are fixed in recent kernels, it could sigreturn directly back to any state.

Good, but have you added any flag for dosemu to even know it can do this? Unless I am mistaken, you didn't. So the fix you suggest, is not easy to detect and make portable with the older kernels. Any suggestions?

Quoted 1 line

I don't actually see any code in DOSEMU that generates a sigcontext

It doesn't. But it manually pops the kernel-generated sigcontext, see dpmisel.S:DPMI_direct_transfer.
This part didn't broke though, so no need to look there in fact.

Quoted 2 lines

from scratch (as opposed to copying one and modifying it), so I'm not entirely sure what the problem is.

Now you have the changes I did to get it working, and I'll also mail you a simple test-case. Let me know if you need something else.
The more important question is whether we ignore dosemu or some actions should be taken. Since you changed uapi/*, my initial guess was that you opt for ignoring it, but maybe this was not the point.

#3 lore raw
Quoted 18 lines · Stas Sergeev

12.08.2015 03:38, Andy Lutomirski пишет:

Quoted 17 lines · Stas Sergeev
Quoted 14 lines

Hi guys, I wonder how easily the include/uapi/* is being changed these days.
The patch:

http://lkml.kernel.org/r/405594361340a2ec32f8e2b115c142df0e180d8e.1426193719.git.luto@kernel.org
breaks dosemu (and perhaps everyone else who used to restore the segregs by hands). And the fix involves both autoconf magic and run-time magic, so it is not even trivial.
I realize this patch may be good to have in general, but breaking userspace without a single warning is a bit discouraging. Seems like the old "we don't break userspace" rule have gone.

Quoted 5 lines
Quoted 1 line

I didn't anticipate any breakage. I could have been wrong.

You changed include/uapi/*, which is obviously an asking for problems. I applied the following changes to my local git tree to get dosemu working again:

To be fair, I renamed a field that used to be padding. The UAPI has to change on occasion -- it's just not supposed to break things.

Quoted 1 line

https://github.com/stsp/dosemu2/commit/48b2a13a49a9fe1a456cd77df6b9a1feec675a01

Maybe I'm still missing something, but this seems like it should be unnecessary. What goes wrong without it?

The new ss field serves two purposes: it stores the old ss (dosemu needs that on new kernels and would benefit in general) and it stores the new post-sigreturn ss (dosemu doesn't currently have any use for that because of the iret trampoline trick).

But maybe you're doing this to make the next patch work.

Quoted 1 line

https://github.com/stsp/dosemu2/commit/7898ac60d5e569964127d6cc48f592caecd20b81

So the problem is that dosemu was actually hacking around the old buggy behavior and thus relying on it. Grr.

Quoted 9 lines
Quoted 5 lines

We might still be able to require a new sigcontext flag to be set and to forcibly return to __USER_DS if the flag is set regardless of the ss value in sigcontext when sigreturn is called, if that is indeed the problem with DOSEMU. But I'm not actually sure that that's the problem.

Well, the flag would be an ideal solution in an ideal world, but in our world I don't know the current relevance of dosemu, and whether or not it worth a new flag to add.

It wouldn't even help here, because the breakage isn't caused by incompatible sigcontext formats -- it's caused by dosemu's reliance on ss being preserved across signal delivery (even if it wasn't preserved on the way back).

Quoted 15 lines
Quoted 15 lines

In fact, DOSEMU contains this:

   /* set up a frame to get back to DPMI via iret. The kernel does not

save
%ss, and the SYSCALL instruction in sigreturn() destroys it.

IRET pops off everything in 64-bit mode even if the privilege does not change which is nice, but clobbers the high 48 bits of rsp if the DPMI client uses a 16-bit stack which is not so nice (see EMUfailure.txt). Setting %rsp to 0x100000000 so that bits 16-31 are zero works around this problem, as DPMI code can't see bits 32-63 anyway.
*/

Quoted 9 lines
Quoted 3 lines

So, if DOSEMU were to realize that both sigreturnissues it's complaining about are fixed in recent kernels, it could sigreturn directly back to any state.

Good, but have you added any flag for dosemu to even know it can do this? Unless I am mistaken, you didn't. So the fix you suggest, is not easy to detect and make portable with the older kernels. Any suggestions?

You could probe for it directly: raise a signal, change the saved ss and see what's in ss after sigreturn.

Let me see if I can come up with a clean kernel fix.

--Andy

#4 lore raw

12.08.2015 19:19, Andy Lutomirski пишет:

Quoted 20 lines · Stas Sergeev
Quoted 15 lines

12.08.2015 03:38, Andy Lutomirski пишет:

Quoted 14 lines · Stas Sergeev
Quoted 13 lines

Hi guys, I wonder how easily the include/uapi/* is being changed these days.
The patch:

http://lkml.kernel.org/r/405594361340a2ec32f8e2b115c142df0e180d8e.1426193719.git.luto@kernel.org
breaks dosemu (and perhaps everyone else who used to restore the segregs by hands). And the fix involves both autoconf magic and run-time magic, so it is not even trivial.
I realize this patch may be good to have in general, but breaking userspace without a single warning is a bit discouraging. Seems like the old "we don't break userspace" rule have gone.

Quoted 4 lines
Quoted 1 line

I didn't anticipate any breakage. I could have been wrong.

You changed include/uapi/*, which is obviously an asking for problems. I applied the following changes to my local git tree to get dosemu working again:

Quoted 6 lines

To be fair, I renamed a field that used to be padding. The UAPI has to change on occasion -- it's just not supposed to break things.

Quoted 1 line

https://github.com/stsp/dosemu2/commit/48b2a13a49a9fe1a456cd77df6b9a1feec675a01

Maybe I'm still missing something, but this seems like it should be unnecessary. What goes wrong without it?

Without it, dosemu stores and fetches the ss value elsewhere. It could use any place for it, be it even a global var. But with your patch, dosemu _needs_ to use the sigcontext.ss, because that's where the kernel now puts it.
As a result, dosemu had to be changed to use sigcontext.ss to load the ss from. Sounds good? Not! :)
The reality is that you'll have to work with the old headers, that still have no sigcontext.ss, and so you'd need to access ss via __pad0 in pretty much 100% of real-life setups. If there is such a need to touch uapi from time to time (and I understand this is the case), then perhaps you should invent some versioning or whatever, to save people from surprises.
Yes, you took the field that was used for padding.
But this doesn't help, because this is not a new functionality. The existing programs now _need_ to use your new field for what they did in the past without it. So it is nearly the same as renaming any of the existing widely used fields:
people will need the autoconf hacks to probe its existence.

Quoted 3 lines
Quoted 1 line

https://github.com/stsp/dosemu2/commit/7898ac60d5e569964127d6cc48f592caecd20b81

So the problem is that dosemu was actually hacking around the old buggy behavior and thus relying on it. Grr.

What else it could do? :(
Please note: I am not trying to ask you to revert the patch. I realize it is needed, I realize dosemu did bad things without it. But the way it is done is a bit annoying: sudden breakage, no run-time checks or versionings, and even the build-time hacks needed.

Quoted 11 lines
Quoted 8 lines
Quoted 5 lines

We might still be able to require a new sigcontext flag to be set and to forcibly return to __USER_DS if the flag is set regardless of the ss value in sigcontext when sigreturn is called, if that is indeed the problem with DOSEMU. But I'm not actually sure that that's the problem.

Well, the flag would be an ideal solution in an ideal world, but in our world I don't know the current relevance of dosemu, and whether or not it worth a new flag to add.

It wouldn't even help here, because the breakage isn't caused by incompatible sigcontext formats -- it's caused by dosemu's reliance on ss being preserved across signal delivery

I thought you mean some per-process flag that would preserve the backward-compatibility for the unaware apps. I probably got you wrong.

Quoted 2 lines

(even if it wasn't preserved
on the way back).

How so?
IIRC sometime fs/gs were restored, but I have no evidence the ss was. Could you clarify?

Quoted 17 lines
Quoted 17 lines
Quoted 17 lines

In fact, DOSEMU contains this:

    /* set up a frame to get back to DPMI via iret. The kernel does not

save
%ss, and the SYSCALL instruction in sigreturn() destroys it.

IRET pops off everything in 64-bit mode even if the privilege does not change which is nice, but clobbers the high 48 bits of rsp if the DPMI client uses a 16-bit stack which is not so nice (see EMUfailure.txt). Setting %rsp to 0x100000000 so that bits 16-31 are zero works around this problem, as DPMI code can't see bits 32-63 anyway.
*/

So, if DOSEMU were to realize that both sigreturnissues it's complaining about are fixed in recent kernels, it could sigreturn directly back to any state.

Quoted 7 lines
Quoted 5 lines

Good, but have you added any flag for dosemu to even know it can do this? Unless I am mistaken, you didn't. So the fix you suggest, is not easy to detect and make portable with the older kernels. Any suggestions?

You could probe for it directly: raise a signal, change the saved ss and see what's in ss after sigreturn.

Umm, nope.

Quoted 1 line

Let me see if I can come up with a clean kernel fix.

The check for proper sigreturn would be good.

#5 lore raw
Quoted 16 lines · Stas Sergeev

12.08.2015 19:19, Andy Lutomirski пишет:

Quoted 3 lines

Maybe I'm still missing something, but this seems like it should be unnecessary. What goes wrong without it?

Without it, dosemu stores and fetches the ss value elsewhere. It could use any place for it, be it even a global var. But with your patch, dosemu _needs_ to use the sigcontext.ss, because that's where the kernel now puts it.
As a result, dosemu had to be changed to use sigcontext.ss to load the ss from. Sounds good? Not! :)
The reality is that you'll have to work with the old headers, that still have no sigcontext.ss, and so you'd need to access ss via __pad0 in pretty much 100% of real-life setups. If there is such a need to touch uapi from time to time (and I understand this is the case), then perhaps you should

Quoted 9 lines

invent some versioning or whatever, to save people from surprises.
Yes, you took the field that was used for padding. But this doesn't help, because this is not a new functionality. The existing programs now _need_ to use your new field for what they did in the past without it. So it is nearly the same as renaming any of the existing widely used fields:
people will need the autoconf hacks to probe its existence.

Yeah, I see the problem here.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

https://github.com/stsp/dosemu2/commit/7898ac60d5e569964127d6cc48f592caecd20b81

So the problem is that dosemu was actually hacking around the old buggy behavior and thus relying on it. Grr.

What else it could do? :(

Going back in time? Ask the kernel to fix the issue. At this point, it's a bit late for that.

Quoted 7 lines
Quoted 3 lines

It wouldn't even help here, because the breakage isn't caused by incompatible sigcontext formats -- it's caused by dosemu's reliance on ss being preserved across signal delivery

I thought you mean some per-process flag that would preserve the backward-compatibility for the unaware apps. I probably got you wrong.

No, I meant a flag in sigcontext indicating which format was used.

Quoted 7 lines
Quoted 3 lines

(even if it wasn't preserved
on the way back).

How so?
IIRC sometime fs/gs were restored, but I have no evidence the ss was. Could you clarify?

ss was never restored by sigreturn AFAIK. I don't think that fs and gs are, but I think they might have been a long long time ago (before git).

Quoted 10 lines
Quoted 8 lines
Quoted 6 lines

Good, but have you added any flag for dosemu to even know it can do this? Unless I am mistaken, you didn't. So the fix you suggest, is not easy to detect and make portable with the older kernels. Any suggestions?

You could probe for it directly: raise a signal, change the saved ss and see what's in ss after sigreturn.

Umm, nope.

Why not? The safest general way to detect new features is to try to use them.

Quoted 4 lines
Quoted 2 lines

Let me see if I can come up with a clean kernel fix.

The check for proper sigreturn would be good.

I still don't see how sigreturn matters here. It's signal *delivery* that's the problem.

I'm thinking of having signal delivery zap ss only if the old ss looks bogus instead of zapping it unconditionally. IOW, instead of setting regs->ss = __USER_DS unconditionally, we'd do larl on the old regs->ss and keep it if it's DPL 3 RW data (exp-down or otherwise) and present.

I'll have to check the precise rules in both the SDM and APM. The idea is that we don't want IRET to fail during signal delivery, which can happen due to a bad sigreturn or a race against modify_ldt.

--Andy

#6 lore raw

12.08.2015 21:25, Andy Lutomirski пишет:

Quoted 5 lines
Quoted 4 lines
Quoted 3 lines
Quoted 1 line

https://github.com/stsp/dosemu2/commit/7898ac60d5e569964127d6cc48f592caecd20b81

So the problem is that dosemu was actually hacking around the old buggy behavior and thus relying on it. Grr.

What else it could do? :(

Going back in time? Ask the kernel to fix the issue.

Like this?
http://www.x86-64.org/pipermail/discuss/2007-May/009913.html
And this:
http://www.x86-64.org/pipermail/discuss/2007-May/009923.html

Quoted 9 lines
Quoted 8 lines
Quoted 7 lines
Quoted 5 lines

Good, but have you added any flag for dosemu to even know it can do this? Unless I am mistaken, you didn't. So the fix you suggest, is not easy to detect and make portable with the older kernels. Any suggestions?

You could probe for it directly: raise a signal, change the saved ss and see what's in ss after sigreturn.

Umm, nope.

Why not? The safest general way to detect new features is to try to use them.

But this is just too many ugly code for nothing. Since it is not very urgent to use sigreturn() instead of iret, I guess I'll better wait for an API addition that will let the check possible.

Quoted 4 lines
Quoted 2 lines
Quoted 1 line

Let me see if I can come up with a clean kernel fix.

The check for proper sigreturn would be good.

I still don't see how sigreturn matters here. It's signal *delivery* that's the problem.

But the delivery can be easily checked with "if (ss & 4)". What remains is just a sigreturn instead of iret.

Quoted 4 lines

I'm thinking of having signal delivery zap ss only if the old ss looks bogus instead of zapping it unconditionally. IOW, instead of setting regs->ss = __USER_DS unconditionally, we'd do larl on the old regs->ss and keep it if it's DPL 3 RW data (exp-down or otherwise) and present.

I am not sure how good is this.
Yes, may help for a backward-compatibility.
But OTOH the 32bit kernel saves _all_ registers, including ss, which is IMHO the right thing to do in general. So as long as the things are already "broken", I wonder if the new hacks are worth the troubles.
Please also see here:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66631#c15
Not saving fs is a pita!

Quoted 3 lines

I'll have to check the precise rules in both the SDM and APM. The idea is that we don't want IRET to fail during signal delivery, which can happen due to a bad sigreturn or a race against modify_ldt.

Well, this is a "very basic" idea, so to say. The fact that segregs are not restored, have much more consequences, and since now you already broke things, I wonder if something can be finally fixed for good...

What alternatives do we have? Can we do something really brave, introduce a new sigaction flag perhaps, that will just restore all segregs for new apps, and none - for old apps? I mean the above gcc bugzilla ticket in particular - very annoying one...

#7 lore raw
Quoted 8 lines · Stas Sergeev

12.08.2015 21:25, Andy Lutomirski пишет:

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines

https://github.com/stsp/dosemu2/commit/7898ac60d5e569964127d6cc48f592caecd20b81

So the problem is that dosemu was actually hacking around the old buggy behavior and thus relying on it. Grr.

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

What else it could do? :(

Going back in time? Ask the kernel to fix the issue.

Like this?
http://www.x86-64.org/pipermail/discuss/2007-May/009913.html
And this:
http://www.x86-64.org/pipermail/discuss/2007-May/009923.html

I apologize on behalf of the upstream kernel in 2007. :-/ I wasn't really involved at that point.

Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
Quoted 8 lines
Quoted 6 lines

Good, but have you added any flag for dosemu to even know it can do this? Unless I am mistaken, you didn't. So the fix you suggest, is not easy to detect and make portable with the older kernels. Any suggestions?

You could probe for it directly: raise a signal, change the saved ss and see what's in ss after sigreturn.

Umm, nope.

Quoted 7 lines
Quoted 2 lines

Why not? The safest general way to detect new features is to try to use them.

But this is just too many ugly code for nothing. Since it is not very urgent to use sigreturn() instead of iret, I guess I'll better wait for an API addition that will let the check possible.

I'll ponder it. You'll get a ~300 cycle speedup if you switch over (IRET is really slow), but I doubt that matters much.

Quoted 16 lines
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

Let me see if I can come up with a clean kernel fix.

The check for proper sigreturn would be good.

I still don't see how sigreturn matters here. It's signal *delivery* that's the problem.

But the delivery can be easily checked with "if (ss & 4)". What remains is just a sigreturn instead of iret.

Quoted 4 lines

I'm thinking of having signal delivery zap ss only if the old ss looks bogus instead of zapping it unconditionally. IOW, instead of setting regs->ss = __USER_DS unconditionally, we'd do larl on the old regs->ss and keep it if it's DPL 3 RW data (exp-down or otherwise) and present.

Quoted 4 lines

I am not sure how good is this.
Yes, may help for a backward-compatibility.
But OTOH the 32bit kernel saves _all_ registers, including ss, which is IMHO the right thing to do in general.

I agree. So does x32.

Are you planning on merging your patches into upstream DOSEMU?

Quoted 6 lines

So as long as the things are already "broken", I wonder if the new hacks are worth the troubles.
Please also see here:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66631#c15
Not saving fs is a pita!

That is an incredible can of worms, and it'll only get worse when WRFSBASE, etc are enabled in Linux. I don't think I like the current kernel behavior, but at least it's fully functional, unlike the old SS behavior. (Except that FS, GS, and their bases aren't context switched correctly right now if you pound on them hard enough.)

Quoted 9 lines
Quoted 3 lines

I'll have to check the precise rules in both the SDM and APM. The idea is that we don't want IRET to fail during signal delivery, which can happen due to a bad sigreturn or a race against modify_ldt.

Well, this is a "very basic" idea, so to say. The fact that segregs are not restored, have much more consequences, and since now you already broke things, I wonder if something can be finally fixed for good...

Quoted 5 lines

What alternatives do we have? Can we do something really brave, introduce a new sigaction flag perhaps, that will just restore all segregs for new apps, and none - for old apps? I mean the above gcc bugzilla ticket in particular - very annoying one...

We might need to do that.

Here's a nasty case:

void sighandler(...) {
switch_userspace_thread();
}

Suppose that switch_userspace_thread() changes fs. Now what? On current kernels, it stays switched. If we change this, it won't stay switched. Even ignoring old ABI, it's not really clear to me what the right thing to do is.

--Andy

#8 lore raw

12.08.2015 22:20, Andy Lutomirski пишет:

Quoted 6 lines · Stas Sergeev
Quoted 5 lines

12.08.2015 21:25, Andy Lutomirski пишет:

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

https://github.com/stsp/dosemu2/commit/7898ac60d5e569964127d6cc48f592caecd20b81

So the problem is that dosemu was actually hacking around the old buggy behavior and thus relying on it. Grr.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Quoted 1 line

What else it could do? :(

Going back in time? Ask the kernel to fix the issue.

Like this?
http://www.x86-64.org/pipermail/discuss/2007-May/009913.html
And this:
http://www.x86-64.org/pipermail/discuss/2007-May/009923.html

I apologize on behalf of the upstream kernel in 2007. :-/ I wasn't really involved at that point.

:)

Quoted 11 lines
Quoted 11 lines
Quoted 4 lines
Quoted 2 lines
Quoted 1 line

Let me see if I can come up with a clean kernel fix.

The check for proper sigreturn would be good.

I still don't see how sigreturn matters here. It's signal *delivery* that's the problem.

But the delivery can be easily checked with "if (ss & 4)". What remains is just a sigreturn instead of iret.

Quoted 4 lines

I'm thinking of having signal delivery zap ss only if the old ss looks bogus instead of zapping it unconditionally. IOW, instead of setting regs->ss = __USER_DS unconditionally, we'd do larl on the old regs->ss and keep it if it's DPL 3 RW data (exp-down or otherwise) and present.

Quoted 7 lines
Quoted 4 lines

I am not sure how good is this.
Yes, may help for a backward-compatibility.
But OTOH the 32bit kernel saves _all_ registers, including ss, which is IMHO the right thing to do in general.

I agree. So does x32.

Are you planning on merging your patches into upstream DOSEMU?

Unlikely.
My git tree counts ~3000 patches already, so it is an entirely separate project, which just happen to share some code with dosemu (and under different license). But we can discuss a binary-compatible fix, so that the original dosemu can work too.

Quoted 15 lines
Quoted 8 lines
Quoted 3 lines

I'll have to check the precise rules in both the SDM and APM. The idea is that we don't want IRET to fail during signal delivery, which can happen due to a bad sigreturn or a race against modify_ldt.

Well, this is a "very basic" idea, so to say. The fact that segregs are not restored, have much more consequences, and since now you already broke things, I wonder if something can be finally fixed for good...

Quoted 5 lines

What alternatives do we have? Can we do something really brave, introduce a new sigaction flag perhaps, that will just restore all segregs for new apps, and none - for old apps? I mean the above gcc bugzilla ticket in particular - very annoying one...

We might need to do that.

Quoted 7 lines

Here's a nasty case:

void sighandler(...) {
   switch_userspace_thread();
}

Suppose that switch_userspace_thread() changes fs. Now what? On

The crash - see the gcc ticket in the prev e-mail. If fs on function entry differs from fs on exit, the gcc stack protector will terminate the program.
So such code will not exist, except maybe in some asm form...

Quoted 3 lines

current kernels, it stays switched. If we change this, it won't stay switched. Even ignoring old ABI, it's not really clear to me what the right thing to do is.

There can be the following cases:
- switch_userspace_thread() switches fs to non-zero selector
- switch_userspace_thread() switches the fs base via syscall
- switch_userspace_thread() switches fs in sigcontext
- switch_userspace_thread() switches fs_base in sigcontext (???) What exactly case do you have in mind?
I'd say, the way x86_32 is doing things - is good, but the bases... perhaps, in ideal world, they should be a part of the sigcontext as well?

#9 lore raw
Quoted 5 lines · Stas Sergeev

12.08.2015 22:20, Andy Lutomirski пишет:

Quoted 3 lines

current kernels, it stays switched. If we change this, it won't stay switched. Even ignoring old ABI, it's not really clear to me what the right thing to do is.

Quoted 9 lines

There can be the following cases:
- switch_userspace_thread() switches fs to non-zero selector
- switch_userspace_thread() switches the fs base via syscall
- switch_userspace_thread() switches fs in sigcontext
- switch_userspace_thread() switches fs_base in sigcontext (???) What exactly case do you have in mind?
I'd say, the way x86_32 is doing things - is good, but the bases... perhaps, in ideal world, they should be a part of the sigcontext as well?

Any of the above. What do you want the kernel to do, and how does the kernel know you want to do that? The kernel has to pick *some* semantics here.

Signature · 2 lines
Andy Lutomirski
AMA Capital Management, LLC
#10 lore raw

12.08.2015 23:01, Andy Lutomirski пишет:

Quoted 14 lines · Stas Sergeev
Quoted 4 lines

12.08.2015 22:20, Andy Lutomirski пишет:

Quoted 3 lines

current kernels, it stays switched. If we change this, it won't stay switched. Even ignoring old ABI, it's not really clear to me what the right thing to do is.

Quoted 9 lines

There can be the following cases:
- switch_userspace_thread() switches fs to non-zero selector
- switch_userspace_thread() switches the fs base via syscall
- switch_userspace_thread() switches fs in sigcontext
- switch_userspace_thread() switches fs_base in sigcontext (???) What exactly case do you have in mind?
I'd say, the way x86_32 is doing things - is good, but the bases... perhaps, in ideal world, they should be a part of the sigcontext as well?

Quoted 3 lines

Any of the above. What do you want the kernel to do, and how does the kernel know you want to do that? The kernel has to pick *some* semantics here.

Assuming the bases are made the part of a sigcontext, I'd say there would be no ambiguities remained at all:
whatever you change in a sigcontext, will be "applied" by the sigreturn(). Whatever you put in the registers (either segregs or MSRs), is valid until sigreturn(), then forgotten forever.
The mess only comes in when some things are part of sigcontext and some are not. But if you have _all_ things accessable in sigcontext, then the user has a way of expressing his needs very clearly: he'll either touch sigcontext or direct values, depending on what he need.

Is this right?

#11 lore raw
Quoted 36 lines · Stas Sergeev

12.08.2015 23:01, Andy Lutomirski пишет:

Quoted 19 lines · Stas Sergeev
Quoted 7 lines

12.08.2015 22:20, Andy Lutomirski пишет:

Quoted 4 lines

current kernels, it stays switched. If we change this, it won't stay switched. Even ignoring old ABI, it's not really clear to me what the right thing to do is.

Quoted 9 lines

There can be the following cases:
- switch_userspace_thread() switches fs to non-zero selector
- switch_userspace_thread() switches the fs base via syscall
- switch_userspace_thread() switches fs in sigcontext
- switch_userspace_thread() switches fs_base in sigcontext (???) What exactly case do you have in mind?
I'd say, the way x86_32 is doing things - is good, but the bases... perhaps, in ideal world, they should be a part of the sigcontext as well?

Quoted 3 lines

Any of the above. What do you want the kernel to do, and how does the kernel know you want to do that? The kernel has to pick *some* semantics here.

Assuming the bases are made the part of a sigcontext, I'd say there would be no ambiguities remained at all:
whatever you change in a sigcontext, will be "applied" by the sigreturn(). Whatever you put in the registers (either segregs or MSRs), is valid until sigreturn(), then forgotten forever.
The mess only comes in when some things are part of sigcontext and some are not. But if you have _all_ things accessable in sigcontext, then the user has a way of expressing his needs very clearly: he'll either touch sigcontext or direct values, depending on what he need.

Quoted 1 line

Is this right?

Maybe, except that doing this might break existing code (Wine and Java come to mind). I'm not really sure.

Anyway, can you give this and its parent a try:

https://git.kernel.org/cgit/linux/kernel/git/luto/linux.git/commit/?h=x86/sigcontext&id=83a08d8c3f43c5524ffc0d88c0eff747716696f5

If they fix the problem for you, I'll improve the test cases and send them to -stable.

--Andy

Signature · 2 lines
Andy Lutomirski
AMA Capital Management, LLC
#12 lore raw

12.08.2015 23:28, Andy Lutomirski пишет:

Quoted 31 lines · Stas Sergeev
Quoted 30 lines

12.08.2015 23:01, Andy Lutomirski пишет:

Quoted 14 lines · Stas Sergeev
Quoted 4 lines

12.08.2015 22:20, Andy Lutomirski пишет:

Quoted 3 lines

current kernels, it stays switched. If we change this, it won't stay switched. Even ignoring old ABI, it's not really clear to me what the right thing to do is.

Quoted 9 lines

There can be the following cases:
- switch_userspace_thread() switches fs to non-zero selector
- switch_userspace_thread() switches the fs base via syscall
- switch_userspace_thread() switches fs in sigcontext
- switch_userspace_thread() switches fs_base in sigcontext (???) What exactly case do you have in mind?
I'd say, the way x86_32 is doing things - is good, but the bases... perhaps, in ideal world, they should be a part of the sigcontext as well?

Quoted 3 lines

Any of the above. What do you want the kernel to do, and how does the kernel know you want to do that? The kernel has to pick *some* semantics here.

Assuming the bases are made the part of a sigcontext, I'd say there would be no ambiguities remained at all:
whatever you change in a sigcontext, will be "applied" by the sigreturn(). Whatever you put in the registers (either segregs or MSRs), is valid until sigreturn(), then forgotten forever.
The mess only comes in when some things are part of sigcontext and some are not. But if you have _all_ things accessable in sigcontext, then the user has a way of expressing his needs very clearly: he'll either touch sigcontext or direct values, depending on what he need.

Quoted 3 lines
Quoted 1 line

Is this right?

Maybe, except that doing this might break existing code (Wine and Java come to mind). I'm not really sure.

Yes, but that's why I was talking about some new flag. Maybe a new sigaction() flag? Or something else that will allow the user to request explicitly the new handling where the things are all switched by the kernel. Then the old programs that don't use that flag, will remain unaffected. I realize this may be a lot of work... But please note that there will be no more a chance like this one, when things are already badly broken. :)

Quoted 6 lines

Anyway, can you give this and its parent a try:

https://git.kernel.org/cgit/linux/kernel/git/luto/linux.git/commit/?h=x86/sigcontext&id=83a08d8c3f43c5524ffc0d88c0eff747716696f5

If they fix the problem for you, I'll improve the test cases and send them to -stable.

:(
Doesn't look pretty at all.
Of course I'll test it if you can't think of any alternative, but do you really think explicitly requesting a new interface will not be possible, and we'll have to live with work-arounds and new problems like in the gcc tracker popping up once in a while?

#13 lore raw
Quoted 45 lines · Stas Sergeev

12.08.2015 23:28, Andy Lutomirski пишет:

Quoted 38 lines · Stas Sergeev
Quoted 37 lines

12.08.2015 23:01, Andy Lutomirski пишет:

Quoted 19 lines · Stas Sergeev
Quoted 7 lines

12.08.2015 22:20, Andy Lutomirski пишет:

Quoted 4 lines

current kernels, it stays switched. If we change this, it won't stay switched. Even ignoring old ABI, it's not really clear to me what the right thing to do is.

Quoted 9 lines

There can be the following cases:
- switch_userspace_thread() switches fs to non-zero selector
- switch_userspace_thread() switches the fs base via syscall
- switch_userspace_thread() switches fs in sigcontext
- switch_userspace_thread() switches fs_base in sigcontext (???) What exactly case do you have in mind?
I'd say, the way x86_32 is doing things - is good, but the bases... perhaps, in ideal world, they should be a part of the sigcontext as well?

Quoted 3 lines

Any of the above. What do you want the kernel to do, and how does the kernel know you want to do that? The kernel has to pick *some* semantics here.

Assuming the bases are made the part of a sigcontext, I'd say there would be no ambiguities remained at all:
whatever you change in a sigcontext, will be "applied" by the sigreturn(). Whatever you put in the registers (either segregs or MSRs), is valid until sigreturn(), then forgotten forever.
The mess only comes in when some things are part of sigcontext and some are not. But if you have _all_ things accessable in sigcontext, then the user has a way of expressing his needs very clearly: he'll either touch sigcontext or direct values, depending on what he need.

Quoted 4 lines
Quoted 1 line

Is this right?

Maybe, except that doing this might break existing code (Wine and Java come to mind). I'm not really sure.

Quoted 8 lines

Yes, but that's why I was talking about some new flag. Maybe a new sigaction() flag? Or something else that will allow the user to request explicitly the new handling where the things are all switched by the kernel. Then the old programs that don't use that flag, will remain unaffected. I realize this may be a lot of work... But please note that there will be no more a chance like this one, when things are already badly broken. :)

I think that, with my patch, we get the best of both worlds. We keep the old behavior in cases where it would work, and we switch to the new behavior in cases where the old behavior would result in killing the task.

Quoted 9 lines
Quoted 8 lines

Anyway, can you give this and its parent a try:

https://git.kernel.org/cgit/linux/kernel/git/luto/linux.git/commit/?h=x86/sigcontext&id=83a08d8c3f43c5524ffc0d88c0eff747716696f5

If they fix the problem for you, I'll improve the test cases and send them to -stable.

Quoted 7 lines

:(
Doesn't look pretty at all.
Of course I'll test it if you can't think of any alternative, but do you really think explicitly requesting a new interface will not be possible, and we'll have to live with work-arounds and new problems like in the gcc tracker popping up once in a while?

I think that explicitly asking for the new behavior would just make it uglier.

--Andy

#14 lore raw

12.08.2015 23:47, Andy Lutomirski пишет:

Quoted 45 lines · Stas Sergeev
Quoted 36 lines

12.08.2015 23:28, Andy Lutomirski пишет:

Quoted 31 lines · Stas Sergeev
Quoted 30 lines

12.08.2015 23:01, Andy Lutomirski пишет:

Quoted 14 lines · Stas Sergeev
Quoted 4 lines

12.08.2015 22:20, Andy Lutomirski пишет:

Quoted 3 lines

current kernels, it stays switched. If we change this, it won't stay switched. Even ignoring old ABI, it's not really clear to me what the right thing to do is.

Quoted 9 lines

There can be the following cases:
- switch_userspace_thread() switches fs to non-zero selector
- switch_userspace_thread() switches the fs base via syscall
- switch_userspace_thread() switches fs in sigcontext
- switch_userspace_thread() switches fs_base in sigcontext (???) What exactly case do you have in mind?
I'd say, the way x86_32 is doing things - is good, but the bases... perhaps, in ideal world, they should be a part of the sigcontext as well?

Quoted 3 lines

Any of the above. What do you want the kernel to do, and how does the kernel know you want to do that? The kernel has to pick *some* semantics here.

Assuming the bases are made the part of a sigcontext, I'd say there would be no ambiguities remained at all:
whatever you change in a sigcontext, will be "applied" by the sigreturn(). Whatever you put in the registers (either segregs or MSRs), is valid until sigreturn(), then forgotten forever.
The mess only comes in when some things are part of sigcontext and some are not. But if you have _all_ things accessable in sigcontext, then the user has a way of expressing his needs very clearly: he'll either touch sigcontext or direct values, depending on what he need.

Quoted 3 lines
Quoted 1 line

Is this right?

Maybe, except that doing this might break existing code (Wine and Java come to mind). I'm not really sure.

Quoted 8 lines

Yes, but that's why I was talking about some new flag. Maybe a new sigaction() flag? Or something else that will allow the user to request explicitly the new handling where the things are all switched by the kernel. Then the old programs that don't use that flag, will remain unaffected. I realize this may be a lot of work... But please note that there will be no more a chance like this one, when things are already badly broken. :)

Quoted 4 lines

I think that, with my patch, we get the best of both worlds. We keep the old behavior in cases where it would work, and we switch to the new behavior in cases where the old behavior would result in killing the task.

But I mean also fs/TLS.
There is a chance now to fix things for good, all at once. :) With such an ss patch applied to stable, there will be no more such a chance ever. What's your opinion on the possibility of fixing the TLS problem?
Also I am not sure about the sigreturn()'s detection: is it a subject of the subsequent patch, or you dropped an idea?

#15 lore raw
Quoted 65 lines · Stas Sergeev

12.08.2015 23:47, Andy Lutomirski пишет:

Quoted 58 lines · Stas Sergeev
Quoted 48 lines

12.08.2015 23:28, Andy Lutomirski пишет:

Quoted 40 lines · Stas Sergeev
Quoted 39 lines

12.08.2015 23:01, Andy Lutomirski пишет:

Quoted 21 lines · Stas Sergeev
Quoted 9 lines

12.08.2015 22:20, Andy Lutomirski пишет:

Quoted 6 lines

current kernels, it stays switched. If we change this, it won't stay
switched. Even ignoring old ABI, it's not really clear to me what the
right thing to do is.

Quoted 9 lines

There can be the following cases:
- switch_userspace_thread() switches fs to non-zero selector
- switch_userspace_thread() switches the fs base via syscall
- switch_userspace_thread() switches fs in sigcontext
- switch_userspace_thread() switches fs_base in sigcontext (???) What exactly case do you have in mind?
I'd say, the way x86_32 is doing things - is good, but the bases... perhaps, in ideal world, they should be a part of the sigcontext as well?

Quoted 3 lines

Any of the above. What do you want the kernel to do, and how does the kernel know you want to do that? The kernel has to pick *some* semantics here.

Assuming the bases are made the part of a sigcontext, I'd say there would be no ambiguities remained at all:
whatever you change in a sigcontext, will be "applied" by the sigreturn(). Whatever you put in the registers (either segregs or MSRs), is valid until sigreturn(), then forgotten forever.
The mess only comes in when some things are part of sigcontext and some are not. But if you have _all_ things accessable in sigcontext, then the user has a way of expressing his needs very clearly: he'll either touch sigcontext or direct values, depending on what he need.

Quoted 4 lines
Quoted 1 line

Is this right?

Maybe, except that doing this might break existing code (Wine and Java come to mind). I'm not really sure.

Quoted 8 lines

Yes, but that's why I was talking about some new flag. Maybe a new sigaction() flag? Or something else that will allow the user to request explicitly the new handling where the things are all switched by the kernel. Then the old programs that don't use that flag, will remain unaffected. I realize this may be a lot of work... But please note that there will be no more a chance like this one, when things are already badly broken. :)

Quoted 4 lines

I think that, with my patch, we get the best of both worlds. We keep the old behavior in cases where it would work, and we switch to the new behavior in cases where the old behavior would result in killing the task.

Quoted 7 lines

But I mean also fs/TLS.
There is a chance now to fix things for good, all at once. :) With such an ss patch applied to stable, there will be no more such a chance ever. What's your opinion on the possibility of fixing the TLS problem?
Also I am not sure about the sigreturn()'s detection: is it a subject of the subsequent patch, or you dropped an idea?

I think these things shouldn't be conflated. If we can fix it transparently (i.e. if my patch works), then I think we should do something like my patch.

--Andy

Signature · 2 lines
Andy Lutomirski
AMA Capital Management, LLC
#16 lore raw

13.08.2015 00:37, Andy Lutomirski пишет:

Quoted 61 lines · Stas Sergeev
Quoted 53 lines

12.08.2015 23:47, Andy Lutomirski пишет:

Quoted 47 lines · Stas Sergeev
Quoted 38 lines

12.08.2015 23:28, Andy Lutomirski пишет:

Quoted 33 lines · Stas Sergeev
Quoted 32 lines

12.08.2015 23:01, Andy Lutomirski пишет:

Quoted 16 lines · Stas Sergeev
Quoted 6 lines

12.08.2015 22:20, Andy Lutomirski пишет:

Quoted 5 lines

current kernels, it stays switched. If we change this, it won't stay
switched. Even ignoring old ABI, it's not really clear to me what the
right thing to do is.

Quoted 9 lines

There can be the following cases:
- switch_userspace_thread() switches fs to non-zero selector
- switch_userspace_thread() switches the fs base via syscall
- switch_userspace_thread() switches fs in sigcontext
- switch_userspace_thread() switches fs_base in sigcontext (???) What exactly case do you have in mind?
I'd say, the way x86_32 is doing things - is good, but the bases... perhaps, in ideal world, they should be a part of the sigcontext as well?

Quoted 3 lines

Any of the above. What do you want the kernel to do, and how does the kernel know you want to do that? The kernel has to pick *some* semantics here.

Assuming the bases are made the part of a sigcontext, I'd say there would be no ambiguities remained at all:
whatever you change in a sigcontext, will be "applied" by the sigreturn(). Whatever you put in the registers (either segregs or MSRs), is valid until sigreturn(), then forgotten forever.
The mess only comes in when some things are part of sigcontext and some are not. But if you have _all_ things accessable in sigcontext, then the user has a way of expressing his needs very clearly: he'll either touch sigcontext or direct values, depending on what he need.

Quoted 3 lines
Quoted 1 line

Is this right?

Maybe, except that doing this might break existing code (Wine and Java come to mind). I'm not really sure.

Quoted 8 lines

Yes, but that's why I was talking about some new flag. Maybe a new sigaction() flag? Or something else that will allow the user to request explicitly the new handling where the things are all switched by the kernel. Then the old programs that don't use that flag, will remain unaffected. I realize this may be a lot of work... But please note that there will be no more a chance like this one, when things are already badly broken. :)

Quoted 4 lines

I think that, with my patch, we get the best of both worlds. We keep the old behavior in cases where it would work, and we switch to the new behavior in cases where the old behavior would result in killing the task.

Quoted 7 lines

But I mean also fs/TLS.
There is a chance now to fix things for good, all at once. :) With such an ss patch applied to stable, there will be no more such a chance ever. What's your opinion on the possibility of fixing the TLS problem?
Also I am not sure about the sigreturn()'s detection: is it a subject of the subsequent patch, or you dropped an idea?

Quoted 3 lines

I think these things shouldn't be conflated. If we can fix it transparently (i.e. if my patch works), then I think we should do something like my patch.

OK.
I'll try to test the patch tomorrow, but I think the sigreturn()'s capability detection is still needed to easily replace the iret trampoline in userspace (without generating a signal and testing by hands). Can of course be done with a run-time kernel version check...

#17 lore raw
Quoted 82 lines · Stas Sergeev

13.08.2015 00:37, Andy Lutomirski пишет:

Quoted 76 lines · Stas Sergeev
Quoted 67 lines

12.08.2015 23:47, Andy Lutomirski пишет:

Quoted 59 lines · Stas Sergeev
Quoted 49 lines

12.08.2015 23:28, Andy Lutomirski пишет:

Quoted 41 lines · Stas Sergeev
Quoted 40 lines

12.08.2015 23:01, Andy Lutomirski пишет:

Quoted 21 lines · Stas Sergeev
Quoted 9 lines

12.08.2015 22:20, Andy Lutomirski пишет:

Quoted 6 lines

current kernels, it stays switched. If we change this, it won't stay
switched. Even ignoring old ABI, it's not really clear to me what the
right thing to do is.

Quoted 9 lines

There can be the following cases:
- switch_userspace_thread() switches fs to non-zero selector
- switch_userspace_thread() switches the fs base via syscall
- switch_userspace_thread() switches fs in sigcontext
- switch_userspace_thread() switches fs_base in sigcontext (???) What exactly case do you have in mind?
I'd say, the way x86_32 is doing things - is good, but the bases... perhaps, in ideal world, they should be a part of the sigcontext as well?

Quoted 4 lines

Any of the above. What do you want the kernel to do, and how does the
kernel know you want to do that? The kernel has to pick *some* semantics here.

Assuming the bases are made the part of a sigcontext, I'd say there would be no ambiguities remained at all:
whatever you change in a sigcontext, will be "applied" by the sigreturn(). Whatever you put in the registers (either segregs or MSRs), is valid until sigreturn(), then forgotten forever.
The mess only comes in when some things are part of sigcontext and some are not. But if you have _all_ things accessable in sigcontext, then the user has a way of expressing his needs very clearly: he'll either touch sigcontext or direct values, depending on what he need.

Quoted 4 lines
Quoted 1 line

Is this right?

Maybe, except that doing this might break existing code (Wine and Java come to mind). I'm not really sure.

Quoted 8 lines

Yes, but that's why I was talking about some new flag. Maybe a new sigaction() flag? Or something else that will allow the user to request explicitly the new handling where the things are all switched by the kernel. Then the old programs that don't use that flag, will remain unaffected. I realize this may be a lot of work... But please note that there will be no more a chance like this one, when things are already badly broken. :)

Quoted 4 lines

I think that, with my patch, we get the best of both worlds. We keep the old behavior in cases where it would work, and we switch to the new behavior in cases where the old behavior would result in killing the task.

Quoted 7 lines

But I mean also fs/TLS.
There is a chance now to fix things for good, all at once. :) With such an ss patch applied to stable, there will be no more such a chance ever. What's your opinion on the possibility of fixing the TLS problem?
Also I am not sure about the sigreturn()'s detection: is it a subject of the subsequent patch, or you dropped an idea?

Quoted 3 lines

I think these things shouldn't be conflated. If we can fix it transparently (i.e. if my patch works), then I think we should do something like my patch.

Quoted 5 lines

OK.
I'll try to test the patch tomorrow, but I think the sigreturn()'s capability detection is still needed to easily replace the iret trampoline in userspace (without generating a signal and testing by hands). Can of course be done with a run-time kernel version check...

That feature is so specialized that I think you should just probe it.

void foo(...) {
sigcontext->ss = 7;
}

modify_ldt(initialize descriptor 0);
sigaction(SIGUSR1, foo, SA_SIGINFO);
if (ss == 7)
yay;

Fortunately, all kernels that restore ss also have espfix64, so you don't need to worry about esp[31:16] corruption on those kernels either.

I suppose we could add a new uc_flag to indicate that ss is saved and restored, though. Ingo, hpa: any thoughts on that? There will always be some kernel versions that save and restore ss but don't set the flag, though.

--Andy

Signature · 2 lines
Andy Lutomirski
AMA Capital Management, LLC
#18 lore raw
Quoted 17 lines · Andy Lutomirski
Quoted 5 lines

OK.
I'll try to test the patch tomorrow, but I think the sigreturn()'s capability detection is still needed to easily replace the iret trampoline in userspace (without generating a signal and testing by hands). Can of course be done with a run-time kernel version check...

That feature is so specialized that I think you should just probe it.

void foo(...) {
sigcontext->ss = 7;
}

modify_ldt(initialize descriptor 0);
sigaction(SIGUSR1, foo, SA_SIGINFO);
if (ss == 7)
yay;

Quoted 7 lines

Fortunately, all kernels that restore ss also have espfix64, so you don't need to worry about esp[31:16] corruption on those kernels either.

I suppose we could add a new uc_flag to indicate that ss is saved and restored, though. Ingo, hpa: any thoughts on that? There will always be some kernel versions that save and restore ss but don't set the flag, though.

So this new flag would essentially be a 'the ss save/restore bug is fixed for sure' flag, not covering old kernels that happen to have the correct behavior, right?

Could you please map out the range of kernel versions involved - which ones:

- 'never do the right thing'
- 'do the right thing sometimes'
- 'do the right thing always, but by accident'
- 'do the right thing always and intentionally'

?

I'd hate to complicate a legacy ABI any more. My gut feeling is to let apps either assume that the kernel works right, or probe the actual behavior. Adding the flag just makes it easy to screw certain kernel versions that would still work fine if the app used actual probing. So I don't see the flag as an improvement.

If your patch fixes the regression that would be a good first step.

Please also send the fix in email instead of just pointing to a Git tree, so that people can comment on it.

Thanks,

Ingo

#19 lore raw

13.08.2015 11:39, Ingo Molnar пишет:

Quoted 22 lines · Andy Lutomirski
Quoted 21 lines
Quoted 6 lines

OK.
I'll try to test the patch tomorrow, but I think the sigreturn()'s capability detection is still needed to easily replace the iret trampoline in userspace (without generating a signal and testing by hands). Can of course be done with a run-time kernel version check...

That feature is so specialized that I think you should just probe it.

void foo(...) {
   sigcontext->ss = 7;
}

modify_ldt(initialize descriptor 0);
sigaction(SIGUSR1, foo, SA_SIGINFO);
if (ss == 7)
yay;

Fortunately, all kernels that restore ss also have espfix64, so you don't need to worry about esp[31:16] corruption on those kernels either.

Quoted 8 lines
Quoted 3 lines

I suppose we could add a new uc_flag to indicate that ss is saved and restored, though. Ingo, hpa: any thoughts on that? There will always be some kernel versions that save and restore ss but don't set the flag, though.

So this new flag would essentially be a 'the ss save/restore bug is fixed for sure' flag, not covering old kernels that happen to have the correct behavior, right?

Could you please map out the range of kernel versions involved - which ones:

"me too"
I'll just hard-code that version info into an app - testing version is the same as testing the flag.

Quoted 9 lines

- 'never do the right thing'
- 'do the right thing sometimes'
- 'do the right thing always, but by accident'
- 'do the right thing always and intentionally'

?

I'd hate to complicate a legacy ABI any more. My gut feeling is to let apps either assume that the kernel works right, or probe the actual behavior.

The problem is that current apps assume the kernel works _wrong_, here's the reference:
http://www.x86-64.org/pipermail/discuss/2007-May/009913.html
http://www.x86-64.org/pipermail/discuss/2007-May/009923.html
And as such, they break.
Details earlier in that thread.

Quoted 5 lines

Adding the flag
just makes it easy to screw certain kernel versions that would still work fine if the app used actual probing. So I don't see the flag as an improvement.

If your patch fixes the regression that would be a good first step.

Unfortunately, the currently proposed patch puts the kernel in the category 'do the right thing sometimes' from the current 'do the right thing always and intentionally'. This helps the legacy apps to work though...
I am not saying this patch should not be applied, but I'd like to discuss the alternatives a bit (although Andy already put it clear that this is likely the final decision, so I am just putting a newly CCed people into a course of events).

Possibilities:
1. leave things as they are and fix dosemu instead (this is against the kernel policy to not break userspace, but I wonder how relevant dosemu is today)
2. Apply Andy's patch that does tricky sanity tests with lar, and if they pass - do not restore the right ss value.
3. Admitting the wider problem, for example TLS is also not restored:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66631
and fix everything at once, probably adding a new sigaction() flag to allow an apps to request the behaviour they need (keeping the old behaviour by default).

Of course all of the above have pros and cons. 3 is a lot of work, both 1 and 2 have patches ready.

Quoted 2 lines

Please also send the fix in email instead of just pointing to a Git tree, so that people can comment on it.

A reference for newly CCed people:
https://git.kernel.org/cgit/linux/kernel/git/luto/linux.git/commit/?h=x86/sigcontext&id=83a08d8c3f43c5524ffc0d88c0eff747716696f5
Not the prettiest possible approach, but at least should work...

#42 lore raw
In reply to Andy Lutomirski

13.08.2015 01:00, Andy Lutomirski пишет:

Quoted 79 lines · Stas Sergeev
Quoted 64 lines

13.08.2015 00:37, Andy Lutomirski пишет:

Quoted 62 lines · Stas Sergeev
Quoted 61 lines

12.08.2015 23:47, Andy Lutomirski пишет:

Quoted 48 lines · Stas Sergeev
Quoted 39 lines

12.08.2015 23:28, Andy Lutomirski пишет:

Quoted 34 lines · Stas Sergeev
Quoted 33 lines

12.08.2015 23:01, Andy Lutomirski пишет:

Quoted 16 lines · Stas Sergeev
Quoted 6 lines

12.08.2015 22:20, Andy Lutomirski пишет:

Quoted 5 lines

current kernels, it stays switched. If we change this, it won't stay
switched. Even ignoring old ABI, it's not really clear to me what the
right thing to do is.

Quoted 9 lines

There can be the following cases:
- switch_userspace_thread() switches fs to non-zero selector
- switch_userspace_thread() switches the fs base via syscall
- switch_userspace_thread() switches fs in sigcontext
- switch_userspace_thread() switches fs_base in sigcontext (???) What exactly case do you have in mind?
I'd say, the way x86_32 is doing things - is good, but the bases... perhaps, in ideal world, they should be a part of the sigcontext as well?

Quoted 4 lines

Any of the above. What do you want the kernel to do, and how does the
kernel know you want to do that? The kernel has to pick *some* semantics here.

Assuming the bases are made the part of a sigcontext, I'd say there would be no ambiguities remained at all:
whatever you change in a sigcontext, will be "applied" by the sigreturn(). Whatever you put in the registers (either segregs or MSRs), is valid until sigreturn(), then forgotten forever.
The mess only comes in when some things are part of sigcontext and some are not. But if you have _all_ things accessable in sigcontext, then the user has a way of expressing his needs very clearly: he'll either touch sigcontext or direct values, depending on what he need.

Quoted 3 lines
Quoted 1 line

Is this right?

Maybe, except that doing this might break existing code (Wine and Java come to mind). I'm not really sure.

Quoted 8 lines

Yes, but that's why I was talking about some new flag. Maybe a new sigaction() flag? Or something else that will allow the user to request explicitly the new handling where the things are all switched by the kernel. Then the old programs that don't use that flag, will remain unaffected. I realize this may be a lot of work... But please note that there will be no more a chance like this one, when things are already badly broken. :)

Quoted 4 lines

I think that, with my patch, we get the best of both worlds. We keep the old behavior in cases where it would work, and we switch to the new behavior in cases where the old behavior would result in killing the task.

But I mean also fs/TLS.
There is a chance now to fix things for good, all at once. :) With such an ss patch applied to stable, there will be no more such a chance ever. What's your opinion on the possibility of fixing the TLS problem?
Also I am not sure about the sigreturn()'s detection: is it a subject of the subsequent patch, or you dropped an idea?

Quoted 8 lines
Quoted 3 lines

I think these things shouldn't be conflated. If we can fix it transparently (i.e. if my patch works), then I think we should do something like my patch.

OK.
I'll try to test the patch tomorrow, but I think the sigreturn()'s capability detection is still needed to easily replace the iret trampoline in userspace (without generating a signal and testing by hands). Can of course be done with a run-time kernel version check...

That feature is so specialized that I think you should just probe it.

void foo(...) {
   sigcontext->ss = 7;
}
Quoted 8 lines

modify_ldt(initialize descriptor 0);
sigaction(SIGUSR1, foo, SA_SIGINFO);
if (ss == 7)
yay;

Fortunately, all kernels that restore ss also have espfix64, so you don't need to worry about esp[31:16] corruption on those kernels either.

Unfortunately, this doesn't help.
I made a simple patch that checks the kernel version:
https://github.com/stsp/dosemu2/commit/098413ef8de98972ca795e078351ae9f3cc07ffe
but iret is still used when switching to DOS code from dosemu code, rather than from signal handler. So espfix64 doesn't help much.
I guess the only real solution to this would be to "rewrite" dosemu so that the DOS code is run in a clone(CLONE_VM) separate process...

#20 lore raw
In reply to Ingo Molnar

13.08.2015 11:39, Ingo Molnar пишет:

Quoted 36 lines · Andy Lutomirski
Quoted 17 lines
Quoted 6 lines

OK.
I'll try to test the patch tomorrow, but I think the sigreturn()'s capability detection is still needed to easily replace the iret trampoline in userspace (without generating a signal and testing by hands). Can of course be done with a run-time kernel version check...

That feature is so specialized that I think you should just probe it.

void foo(...) {
   sigcontext->ss = 7;
}

modify_ldt(initialize descriptor 0);
sigaction(SIGUSR1, foo, SA_SIGINFO);
if (ss == 7)
yay;

Quoted 7 lines

Fortunately, all kernels that restore ss also have espfix64, so you don't need to worry about esp[31:16] corruption on those kernels either.

I suppose we could add a new uc_flag to indicate that ss is saved and restored, though. Ingo, hpa: any thoughts on that? There will always be some kernel versions that save and restore ss but don't set the flag, though.

So this new flag would essentially be a 'the ss save/restore bug is fixed for sure' flag, not covering old kernels that happen to have the correct behavior, right?

Could you please map out the range of kernel versions involved - which ones:

- 'never do the right thing'
- 'do the right thing sometimes'
- 'do the right thing always, but by accident'
- 'do the right thing always and intentionally'

Quoted 8 lines

?

I'd hate to complicate a legacy ABI any more. My gut feeling is to let apps either assume that the kernel works right, or probe the actual behavior. Adding the flag just makes it easy to screw certain kernel versions that would still work fine if the app used actual probing. So I don't see the flag as an improvement.

If your patch fixes the regression that would be a good first step.

I've tested the patch.
It doesn't fix the problem.
It allows dosemu to save the ss the old way, but, because dosemu doesn't save it to the sigreturn()'s-expected place (sigcontext.__pad0), it crashes on sigreturn().

So the problem can't be fixed this way --> NACK to the patch.

I may be unavailable for further testings till next week.

#21 lore raw
Quoted 65 lines · Stas Sergeev

13.08.2015 11:39, Ingo Molnar пишет:

Quoted 56 lines · Andy Lutomirski
Quoted 24 lines
Quoted 8 lines

OK.
I'll try to test the patch tomorrow, but I think the sigreturn()'s capability detection is still needed to easily replace the iret trampoline
in userspace (without generating a signal and testing by hands). Can of course be done with a run-time kernel version check...

That feature is so specialized that I think you should just probe it.

void foo(...) {
   sigcontext->ss = 7;
}

modify_ldt(initialize descriptor 0);
sigaction(SIGUSR1, foo, SA_SIGINFO);
if (ss == 7)
yay;

Fortunately, all kernels that restore ss also have espfix64, so you don't need to worry about esp[31:16] corruption on those kernels either.

Quoted 5 lines

I suppose we could add a new uc_flag to indicate that ss is saved and restored,
though. Ingo, hpa: any thoughts on that? There will always be some kernel
versions that save and restore ss but don't set the flag, though.

So this new flag would essentially be a 'the ss save/restore bug is fixed for
sure' flag, not covering old kernels that happen to have the correct behavior,
right?

Could you please map out the range of kernel versions involved - which ones:

- 'never do the right thing'
- 'do the right thing sometimes'
- 'do the right thing always, but by accident'
- 'do the right thing always and intentionally'

?

I'd hate to complicate a legacy ABI any more. My gut feeling is to let apps either
assume that the kernel works right, or probe the actual behavior. Adding the flag
just makes it easy to screw certain kernel versions that would still work fine if
the app used actual probing. So I don't see the flag as an improvement.

Quoted 1 line

If your patch fixes the regression that would be a good first step.

I've tested the patch.
It doesn't fix the problem.
It allows dosemu to save the ss the old way, but, because dosemu doesn't save it to the sigreturn()'s-expected place (sigcontext.__pad0), it crashes on sigreturn().

Quoted 3 lines

So the problem can't be fixed this way --> NACK to the patch.

I may be unavailable for further testings till next week.

I'm still fighting with getting DOSEMU to run at all in my VM.

I must be missing something. What ends up in ss/__pad0? Wouldn't it contain whatever signal delivery put there (i.e. some valid ss value)?

--Andy

#22 lore raw

13.08.2015 17:58, Andy Lutomirski пишет:

Quoted 62 lines · Stas Sergeev
Quoted 61 lines

13.08.2015 11:39, Ingo Molnar пишет:

Quoted 53 lines · Andy Lutomirski
Quoted 23 lines
Quoted 8 lines

OK.
I'll try to test the patch tomorrow, but I think the sigreturn()'s capability detection is still needed to easily replace the iret trampoline
in userspace (without generating a signal and testing by hands). Can of course be done with a run-time kernel version check...

That feature is so specialized that I think you should just probe it.

void foo(...) {
    sigcontext->ss = 7;
}

modify_ldt(initialize descriptor 0);
sigaction(SIGUSR1, foo, SA_SIGINFO);
if (ss == 7)

    yay;

Fortunately, all kernels that restore ss also have espfix64, so you don't need to worry about esp[31:16] corruption on those kernels either.

Quoted 5 lines

I suppose we could add a new uc_flag to indicate that ss is saved and restored,
though. Ingo, hpa: any thoughts on that? There will always be some kernel
versions that save and restore ss but don't set the flag, though.

So this new flag would essentially be a 'the ss save/restore bug is fixed for
sure' flag, not covering old kernels that happen to have the correct behavior,
right?

Could you please map out the range of kernel versions involved - which ones:

- 'never do the right thing'
- 'do the right thing sometimes'
- 'do the right thing always, but by accident'
- 'do the right thing always and intentionally'

?

I'd hate to complicate a legacy ABI any more. My gut feeling is to let apps either
assume that the kernel works right, or probe the actual behavior. Adding the flag
just makes it easy to screw certain kernel versions that would still work fine if
the app used actual probing. So I don't see the flag as an improvement.

Quoted 1 line

If your patch fixes the regression that would be a good first step.

I've tested the patch.
It doesn't fix the problem.
It allows dosemu to save the ss the old way, but, because dosemu doesn't save it to the sigreturn()'s-expected place (sigcontext.__pad0), it crashes on sigreturn().

Quoted 7 lines
Quoted 3 lines

So the problem can't be fixed this way --> NACK to the patch.

I may be unavailable for further testings till next week.

I'm still fighting with getting DOSEMU to run at all in my VM.

I must be missing something. What ends up in ss/__pad0? Wouldn't it contain whatever signal delivery put there (i.e. some valid ss value)?

The crash happens when DOS program terminates.
At that point dosemu subverts the execution flow by replacing segregs and cs/ip ss/sp in sigcontext with its own. But __pad0 still has DOS SS, which crash because (presumably) the DOS LDT have been just removed.

#43 lore raw
In reply to Stas Sergeev
Quoted 5 lines · Stas Sergeev

I realize this patch may be good to have in general, but breaking userspace without a single warning is a bit discouraging. Seems like the old "we don't break userspace" rule have gone.

That rule hasn't gone anywhere.

Does a plain revert just fix everything? Because if so, that's the right thing to do, and we can just re-visit this later.

I don't understand why Andy and Ingo are even discussing this. What the f*ck, guys?

Stas, can you verify that this actually fixes it? There's two different versions here: one that reverts *just* that one commit, and one that reverts the fs/gs changes too. Can you test them both?

Linus

commit 68b72e2a41ae36de41a404e14388f73b16c4debe

Author: Linus Torvalds <[email protected]>
Date:   Thu Aug 13 08:25:20 2015 -0700

Revert x86 sigcontext cleanups

This reverts commits 9a036b93a344 ("x86/signal/64: Remove 'fs' and 'gs' from sigcontext") and c6f2062935c8 ("x86/signal/64: Fix SS handling for signals delivered to 64-bit programs").

They were cleanups, but they don't really matter,a nd they break dosemu by changing the signal stack layout.

---
 arch/x86/include/asm/sigcontext.h      |  6 +++---
 arch/x86/include/uapi/asm/sigcontext.h | 21 +++------------------
 arch/x86/kernel/signal.c               | 26 +++++++++++---------------
 3 files changed, 17 insertions(+), 36 deletions(-)
arch/x86/include/asm/sigcontext.h +3 -3
index 6fe6b182c998..9dfce4e0417d 100644
@@ -57,9 +57,9 @@ struct sigcontext {
 	unsigned long ip;
 	unsigned long flags;
 	unsigned short cs;
-	unsigned short __pad2;	/* Was called gs, but was always zero. */
-	unsigned short __pad1;	/* Was called fs, but was always zero. */
-	unsigned short ss;
+	unsigned short gs;
+	unsigned short fs;
+	unsigned short __pad0;
 	unsigned long err;
 	unsigned long trapno;
 	unsigned long oldmask;
arch/x86/include/uapi/asm/sigcontext.h +3 -18
index 0e8a973de9ee..40836a9a7250 100644
@@ -177,24 +177,9 @@ struct sigcontext {
 	__u64 rip;
 	__u64 eflags;		/* RFLAGS */
 	__u16 cs;
-
-	/*
-	 * Prior to 2.5.64 ("[PATCH] x86-64 updates for 2.5.64-bk3"),
-	 * Linux saved and restored fs and gs in these slots.  This
-	 * was counterproductive, as fsbase and gsbase were never
-	 * saved, so arch_prctl was presumably unreliable.
-	 *
-	 * If these slots are ever needed for any other purpose, there
-	 * is some risk that very old 64-bit binaries could get
-	 * confused.  I doubt that many such binaries still work,
-	 * though, since the same patch in 2.5.64 also removed the
-	 * 64-bit set_thread_area syscall, so it appears that there is
-	 * no TLS API that works in both pre- and post-2.5.64 kernels.
-	 */
-	__u16 __pad2;		/* Was gs. */
-	__u16 __pad1;		/* Was fs. */
-
-	__u16 ss;
+	__u16 gs;
+	__u16 fs;
+	__u16 __pad0;
 	__u64 err;
 	__u64 trapno;
 	__u64 oldmask;
arch/x86/kernel/signal.c +11 -15
index 206996c1669d..71820c42b6ce 100644
@@ -93,8 +93,15 @@ int restore_sigcontext(struct pt_regs *regs, struct sigcontext __user *sc)
 		COPY(r15);
 #endif /* CONFIG_X86_64 */
 
+#ifdef CONFIG_X86_32
 		COPY_SEG_CPL3(cs);
 		COPY_SEG_CPL3(ss);
+#else /* !CONFIG_X86_32 */
+		/* Kernel saves and restores only the CS segment register on signals,
+		 * which is the bare minimum needed to allow mixed 32/64-bit code.
+		 * App's signal handler can save/restore other segments if needed. */
+		COPY_SEG_CPL3(cs);
+#endif /* CONFIG_X86_32 */
 
 		get_user_ex(tmpflags, &sc->flags);
 		regs->flags = (regs->flags & ~FIX_EFLAGS) | (tmpflags & FIX_EFLAGS);
@@ -154,9 +161,8 @@ int setup_sigcontext(struct sigcontext __user *sc, void __user *fpstate,
 #else /* !CONFIG_X86_32 */
 		put_user_ex(regs->flags, &sc->flags);
 		put_user_ex(regs->cs, &sc->cs);
-		put_user_ex(0, &sc->__pad2);
-		put_user_ex(0, &sc->__pad1);
-		put_user_ex(regs->ss, &sc->ss);
+		put_user_ex(0, &sc->gs);
+		put_user_ex(0, &sc->fs);
 #endif /* CONFIG_X86_32 */
 
 		put_user_ex(fpstate, &sc->fpstate);
@@ -451,19 +457,9 @@ static int __setup_rt_frame(int sig, struct ksignal *ksig,
 
 	regs->sp = (unsigned long)frame;
 
-	/*
-	 * Set up the CS and SS registers to run signal handlers in
-	 * 64-bit mode, even if the handler happens to be interrupting
-	 * 32-bit or 16-bit code.
-	 *
-	 * SS is subtle.  In 64-bit mode, we don't need any particular
-	 * SS descriptor, but we do need SS to be valid.  It's possible
-	 * that the old SS is entirely bogus -- this can happen if the
-	 * signal we're trying to deliver is #GP or #SS caused by a bad
-	 * SS value.
-	 */
+	/* Set up the CS register to run signal handlers in 64-bit mode,
+	   even if the handler happens to be interrupting 32-bit code. */
 	regs->cs = __USER_CS;
-	regs->ss = __USER_DS;
 
 	return 0;
 }
 arch/x86/include/asm/sigcontext.h      |  2 +-
 arch/x86/include/uapi/asm/sigcontext.h |  2 +-
 arch/x86/kernel/signal.c               | 22 +++++++++-------------
 3 files changed, 11 insertions(+), 15 deletions(-)
arch/x86/include/asm/sigcontext.h +1 -1
index 6fe6b182c998..2cefce9b52bd 100644
@@ -59,7 +59,7 @@ struct sigcontext {
 	unsigned short cs;
 	unsigned short __pad2;	/* Was called gs, but was always zero. */
 	unsigned short __pad1;	/* Was called fs, but was always zero. */
-	unsigned short ss;
+	unsigned short __pad0;
 	unsigned long err;
 	unsigned long trapno;
 	unsigned long oldmask;
arch/x86/include/uapi/asm/sigcontext.h +1 -1
index 0e8a973de9ee..b659f3ee464d 100644
@@ -194,7 +194,7 @@ struct sigcontext {
 	__u16 __pad2;		/* Was gs. */
 	__u16 __pad1;		/* Was fs. */
 
-	__u16 ss;
+	__u16 __pad0;
 	__u64 err;
 	__u64 trapno;
 	__u64 oldmask;
arch/x86/kernel/signal.c +9 -13
index 206996c1669d..cecc669d397a 100644
@@ -93,8 +93,15 @@ int restore_sigcontext(struct pt_regs *regs, struct sigcontext __user *sc)
 		COPY(r15);
 #endif /* CONFIG_X86_64 */
 
+#ifdef CONFIG_X86_32
 		COPY_SEG_CPL3(cs);
 		COPY_SEG_CPL3(ss);
+#else /* !CONFIG_X86_32 */
+		/* Kernel saves and restores only the CS segment register on signals,
+		 * which is the bare minimum needed to allow mixed 32/64-bit code.
+		 * App's signal handler can save/restore other segments if needed. */
+		COPY_SEG_CPL3(cs);
+#endif /* CONFIG_X86_32 */
 
 		get_user_ex(tmpflags, &sc->flags);
 		regs->flags = (regs->flags & ~FIX_EFLAGS) | (tmpflags & FIX_EFLAGS);
@@ -156,7 +163,6 @@ int setup_sigcontext(struct sigcontext __user *sc, void __user *fpstate,
 		put_user_ex(regs->cs, &sc->cs);
 		put_user_ex(0, &sc->__pad2);
 		put_user_ex(0, &sc->__pad1);
-		put_user_ex(regs->ss, &sc->ss);
 #endif /* CONFIG_X86_32 */
 
 		put_user_ex(fpstate, &sc->fpstate);
@@ -451,19 +457,9 @@ static int __setup_rt_frame(int sig, struct ksignal *ksig,
 
 	regs->sp = (unsigned long)frame;
 
-	/*
-	 * Set up the CS and SS registers to run signal handlers in
-	 * 64-bit mode, even if the handler happens to be interrupting
-	 * 32-bit or 16-bit code.
-	 *
-	 * SS is subtle.  In 64-bit mode, we don't need any particular
-	 * SS descriptor, but we do need SS to be valid.  It's possible
-	 * that the old SS is entirely bogus -- this can happen if the
-	 * signal we're trying to deliver is #GP or #SS caused by a bad
-	 * SS value.
-	 */
+	/* Set up the CS register to run signal handlers in 64-bit mode,
+	   even if the handler happens to be interrupting 32-bit code. */
 	regs->cs = __USER_CS;
-	regs->ss = __USER_DS;
 
 	return 0;
 }
#23 lore raw
In reply to Stas Sergeev
Quoted 77 lines · Stas Sergeev

13.08.2015 17:58, Andy Lutomirski пишет:

Quoted 75 lines · Stas Sergeev
Quoted 68 lines

13.08.2015 11:39, Ingo Molnar пишет:

Quoted 58 lines · Andy Lutomirski
Quoted 24 lines
Quoted 8 lines

OK.
I'll try to test the patch tomorrow, but I think the sigreturn()'s capability detection is still needed to easily replace the iret trampoline
in userspace (without generating a signal and testing by hands). Can of course be done with a run-time kernel version check...

That feature is so specialized that I think you should just probe it.

void foo(...) {
    sigcontext->ss = 7;
}

modify_ldt(initialize descriptor 0);
sigaction(SIGUSR1, foo, SA_SIGINFO);
if (ss == 7)

    yay;

Fortunately, all kernels that restore ss also have espfix64, so you don't need to worry about esp[31:16] corruption on those kernels either.

Quoted 5 lines

I suppose we could add a new uc_flag to indicate that ss is saved and restored,
though. Ingo, hpa: any thoughts on that? There will always be some kernel
versions that save and restore ss but don't set the flag, though.

So this new flag would essentially be a 'the ss save/restore bug is fixed
for
sure' flag, not covering old kernels that happen to have the correct behavior,
right?

Could you please map out the range of kernel versions involved - which ones:

- 'never do the right thing'
- 'do the right thing sometimes'
- 'do the right thing always, but by accident'
- 'do the right thing always and intentionally'

?

I'd hate to complicate a legacy ABI any more. My gut feeling is to let apps either
assume that the kernel works right, or probe the actual behavior. Adding the flag
just makes it easy to screw certain kernel versions that would still work
fine if
the app used actual probing. So I don't see the flag as an improvement.

Quoted 1 line

If your patch fixes the regression that would be a good first step.

I've tested the patch.
It doesn't fix the problem.
It allows dosemu to save the ss the old way, but, because dosemu doesn't save it to the sigreturn()'s-expected place (sigcontext.__pad0), it crashes on sigreturn().

Quoted 3 lines

So the problem can't be fixed this way --> NACK to the patch.

I may be unavailable for further testings till next week.

I'm still fighting with getting DOSEMU to run at all in my VM.

Quoted 8 lines
Quoted 2 lines

I must be missing something. What ends up in ss/__pad0? Wouldn't it contain whatever signal delivery put there (i.e. some valid ss value)?

The crash happens when DOS program terminates.
At that point dosemu subverts the execution flow by replacing segregs and cs/ip ss/sp in sigcontext with its own. But __pad0 still has DOS SS, which crash because (presumably) the DOS LDT have been just removed.

That's unfortunate.

I don't really know what to do about this. My stupid heuristic for signal delivery seems unlikely to cause problems, but I'm not coming up with a great heuristic for detecting when a program that *modifies* sigcontext hasn't set all the fields. Even adding a flag won't really help here, since DOSEMU won't know to manipulate the flag.

Ingo, here's the situation, assuming I remember the versions right:

v4.0 and before: If we try to deliver a signal while SS is bad, we fail and the process dies. If SS is good but nonstandard, we end up in the signal handler with whatever SS value was loaded when the signal was sent. We do *not* put SS anywhere in the sigcontext, so the only way for a program to figure out what SS was is to look at the HW state before making any syscalls. We also don't even try to restore SS, so SS is unconditionally set to __USER_DS, necessitating nasty workarounds (and breaking all kinds of test cases).

v4.1 and current -linus: We always set SS to __USER_DS when delivering a signal. We save the old SS in the sigcontext and restore it, just like 32-bit signals.

My patch: We leave SS alone when delivering a signal, unless it's invalid, in which case we replace it with __USER_DS. We still save the old SS in the sigcontext and restore it on return.

Apparently the remaining regression is that DOSEMU doesn't realize that SS is saved so, when it tries to return to full 64-bit mode after a signal that hit in 16-bit mode, it fails because it's invalidated the old SS descriptor in the mean time.

So... what do we do about it? We could revert the whole mess. We could tell everyone to fix their DOSEMU, which violates policy and is especially annoying given how much effort we've put into keeping 16-bit mode fully functional lately. We could add yet more heuristics and teach sigreturn to ignore the saved SS value in sigcontext if the saved CS is 64-bit and the saved SS is unusable.

--Andy

Signature · 2 lines
Andy Lutomirski
AMA Capital Management, LLC
#44 lore raw
In reply to Linus Torvalds
Quoted 7 lines · Linus Torvalds
Quoted 5 lines · Stas Sergeev

I realize this patch may be good to have in general, but breaking userspace without a single warning is a bit discouraging. Seems like the old "we don't break userspace" rule have gone.

Quoted 8 lines

That rule hasn't gone anywhere.

Does a plain revert just fix everything? Because if so, that's the right thing to do, and we can just re-visit this later.

I don't understand why Andy and Ingo are even discussing this. What the f*ck, guys?

I'm trying to fix it without reverting. If that doesn't work, then we revert. Yesterday, I thought I had a reasonably clean fix, but it turned out that it only solved half of the problem.

If we revert, I think I need to check what will break due to the revert. I need to check at least Wine, and we'll have to do something about all the selftests that will start failing. I also need to check CRIU, and IIRC CRIU has started using the new sigcontext SS in new versions.

And, damnit, those selftests are *useful*. They've smoked out all kinds of problems. That's part of the reason I'd prefer not to revert if there's a better option.

--Andy

#24 lore raw
In reply to Andy Lutomirski

13.08.2015 18:38, Andy Lutomirski пишет:

Quoted 112 lines · Stas Sergeev
Quoted 71 lines

13.08.2015 17:58, Andy Lutomirski пишет:

Quoted 69 lines · Stas Sergeev
Quoted 63 lines

13.08.2015 11:39, Ingo Molnar пишет:

Quoted 55 lines · Andy Lutomirski
Quoted 23 lines
Quoted 8 lines

OK.
I'll try to test the patch tomorrow, but I think the sigreturn()'s capability detection is still needed to easily replace the iret trampoline
in userspace (without generating a signal and testing by hands). Can of course be done with a run-time kernel version check...

That feature is so specialized that I think you should just probe it.

void foo(...) {
     sigcontext->ss = 7;
}

modify_ldt(initialize descriptor 0);
sigaction(SIGUSR1, foo, SA_SIGINFO);
if (ss == 7)

     yay;

Fortunately, all kernels that restore ss also have espfix64, so you don't need to worry about esp[31:16] corruption on those kernels either.

Quoted 5 lines

I suppose we could add a new uc_flag to indicate that ss is saved and restored,
though. Ingo, hpa: any thoughts on that? There will always be some kernel
versions that save and restore ss but don't set the flag, though.

So this new flag would essentially be a 'the ss save/restore bug is fixed
for
sure' flag, not covering old kernels that happen to have the correct behavior,
right?

Could you please map out the range of kernel versions involved - which ones:

- 'never do the right thing'
- 'do the right thing sometimes'
- 'do the right thing always, but by accident'
- 'do the right thing always and intentionally'

?

I'd hate to complicate a legacy ABI any more. My gut feeling is to let apps either
assume that the kernel works right, or probe the actual behavior. Adding the flag
just makes it easy to screw certain kernel versions that would still work
fine if
the app used actual probing. So I don't see the flag as an improvement.

Quoted 1 line

If your patch fixes the regression that would be a good first step.

I've tested the patch.
It doesn't fix the problem.
It allows dosemu to save the ss the old way, but, because dosemu doesn't save it to the sigreturn()'s-expected place (sigcontext.__pad0), it crashes on sigreturn().

Quoted 3 lines

So the problem can't be fixed this way --> NACK to the patch.

I may be unavailable for further testings till next week.

I'm still fighting with getting DOSEMU to run at all in my VM.

Quoted 7 lines
Quoted 2 lines

I must be missing something. What ends up in ss/__pad0? Wouldn't it contain whatever signal delivery put there (i.e. some valid ss value)?

The crash happens when DOS program terminates.
At that point dosemu subverts the execution flow by replacing segregs and cs/ip ss/sp in sigcontext with its own. But __pad0 still has DOS SS, which crash because (presumably) the DOS LDT have been just removed.

That's unfortunate.

I don't really know what to do about this. My stupid heuristic for signal delivery seems unlikely to cause problems, but I'm not coming up with a great heuristic for detecting when a program that *modifies* sigcontext hasn't set all the fields. Even adding a flag won't really help here, since DOSEMU won't know to manipulate the flag.

Ingo, here's the situation, assuming I remember the versions right:

v4.0 and before: If we try to deliver a signal while SS is bad, we fail and the process dies. If SS is good but nonstandard, we end up in the signal handler with whatever SS value was loaded when the signal was sent. We do *not* put SS anywhere in the sigcontext, so the only way for a program to figure out what SS was is to look at the HW state before making any syscalls. We also don't even try to restore SS, so SS is unconditionally set to __USER_DS, necessitating nasty workarounds (and breaking all kinds of test cases).

v4.1 and current -linus: We always set SS to __USER_DS when delivering a signal. We save the old SS in the sigcontext and restore it, just like 32-bit signals.

My patch: We leave SS alone when delivering a signal, unless it's invalid, in which case we replace it with __USER_DS. We still save the old SS in the sigcontext and restore it on return.

Apparently the remaining regression is that DOSEMU doesn't realize that SS is saved so, when it tries to return to full 64-bit mode after a signal that hit in 16-bit mode, it fails because it's invalidated the old SS descriptor in the mean time.

Quoted 6 lines

So... what do we do about it? We could revert the whole mess. We could tell everyone to fix their DOSEMU, which violates policy and is especially annoying given how much effort we've put into keeping 16-bit mode fully functional lately. We could add yet more heuristics and teach sigreturn to ignore the saved SS value in sigcontext if the saved CS is 64-bit and the saved SS is unusable.

Andy, why do you constantly ignore the proposal to make new behaviour explicitly controlable? You don't have to agree with it, but you could at least comment on that possibility and/or mention it with the ones you listed above.

#25 lore raw
Quoted 10 lines · Stas Sergeev

13.08.2015 18:38, Andy Lutomirski пишет:

Quoted 8 lines

So... what do we do about it? We could revert the whole mess. We could tell everyone to fix their DOSEMU, which violates policy and is especially annoying given how much effort we've put into keeping 16-bit mode fully functional lately. We could add yet more heuristics and teach sigreturn to ignore the saved SS value in sigcontext if the saved CS is 64-bit and the saved SS is unusable.

Quoted 4 lines

Andy, why do you constantly ignore the proposal to make new behaviour explicitly controlable? You don't have to agree with it, but you could at least comment on that possibility and/or mention it with the ones you listed above.

I'm not sure what the proposal is exactly.

We could add a new uc_flags flag. If set, it means that sigcontext->ss is valid and should be used by sigreturn. If clear, then we ignore sigcontext->ss and just restore __USER_DS.

The problem is that, by itself, this won't fix old DOSEMU. We somehow need to either detect that something funny is going on or just leave the flag clear by default.

We could do this: always save SS to sigcontext->ss, but only restore sigcontext->ss if userspace explicitly sets the flag before sigreturn. If we do that, we'd need to also add my patch to preserve the actual HW SS selector if possible so that old DOSEMU knows what SS to program into its trampoline.

This at least lets *new* DOSEMU set the flag and get the improved behavior. I still don't know what effect it'll have on Wine and CRIU.

Stas, is that what you were thinking, or were you thinking of something else?

--Andy

#45 lore raw
In reply to Andy Lutomirski
Quoted 4 lines · Andy Lutomirski

I'm trying to fix it without reverting. If that doesn't work, then we revert. Yesterday, I thought I had a reasonably clean fix, but it turned out that it only solved half of the problem.

The thing is, I actually think that the current situation is crazy.

Especially given that we don't restore any of the other segment registers on x86-64 (except CS, of course)

So how about this "alternate" minimal patch instead. The difference is:

- we actually leave the

        regs->ss = __USER_DS;

in __setup_rt_frame, to guarantee that when we take a signal, we do
take it with a valid SS

- but it removes all the other games with SS (and treats it exactly
the same as FS/GS).

So now we don't play games with the actual sigcontext, and
hopefully dosemu is happier.

Hmm? That actually makes the code look better, and doesn't re-introduce that annoying CONFIG_X86_32 case (because it now does it in obviously the same place as fs/gs).

So the code is cleaner, and closer to what we used to do.

Stas, can you test this one too? I, like Luto, don't actually have a dosemu test-case.

Linus

 arch/x86/include/asm/sigcontext.h      | 2 +-
 arch/x86/include/uapi/asm/sigcontext.h | 3 +--
 arch/x86/kernel/signal.c               | 4 ++--
 3 files changed, 4 insertions(+), 5 deletions(-)
arch/x86/include/asm/sigcontext.h +1 -1
index 6fe6b182c998..2cefce9b52bd 100644
@@ -59,7 +59,7 @@ struct sigcontext {
 	unsigned short cs;
 	unsigned short __pad2;	/* Was called gs, but was always zero. */
 	unsigned short __pad1;	/* Was called fs, but was always zero. */
-	unsigned short ss;
+	unsigned short __pad0;
 	unsigned long err;
 	unsigned long trapno;
 	unsigned long oldmask;
arch/x86/include/uapi/asm/sigcontext.h +1 -2
index 0e8a973de9ee..d99bbc2d4b0b 100644
@@ -193,8 +193,7 @@ struct sigcontext {
 	 */
 	__u16 __pad2;		/* Was gs. */
 	__u16 __pad1;		/* Was fs. */
-
-	__u16 ss;
+	__u16 __pad0;		/* Not ss. */
 	__u64 err;
 	__u64 trapno;
 	__u64 oldmask;
arch/x86/kernel/signal.c +2 -2
index 206996c1669d..e955f0cd9b9d 100644
@@ -77,6 +77,7 @@ int restore_sigcontext(struct pt_regs *regs, struct sigcontext __user *sc)
 		COPY_SEG(fs);
 		COPY_SEG(es);
 		COPY_SEG(ds);
+		COPY_SEG_CPL3(ss);
 #endif /* CONFIG_X86_32 */
 
 		COPY(di); COPY(si); COPY(bp); COPY(sp); COPY(bx);
@@ -94,7 +95,6 @@ int restore_sigcontext(struct pt_regs *regs, struct sigcontext __user *sc)
 #endif /* CONFIG_X86_64 */
 
 		COPY_SEG_CPL3(cs);
-		COPY_SEG_CPL3(ss);
 
 		get_user_ex(tmpflags, &sc->flags);
 		regs->flags = (regs->flags & ~FIX_EFLAGS) | (tmpflags & FIX_EFLAGS);
@@ -156,7 +156,7 @@ int setup_sigcontext(struct sigcontext __user *sc, void __user *fpstate,
 		put_user_ex(regs->cs, &sc->cs);
 		put_user_ex(0, &sc->__pad2);
 		put_user_ex(0, &sc->__pad1);
-		put_user_ex(regs->ss, &sc->ss);
+		put_user_ex(0, &sc->__pad0);
 #endif /* CONFIG_X86_32 */
 
 		put_user_ex(fpstate, &sc->fpstate);
#26 lore raw
In reply to Andy Lutomirski

13.08.2015 19:09, Andy Lutomirski пишет:

Quoted 29 lines · Stas Sergeev
Quoted 8 lines

13.08.2015 18:38, Andy Lutomirski пишет:

Quoted 7 lines

So... what do we do about it? We could revert the whole mess. We could tell everyone to fix their DOSEMU, which violates policy and is especially annoying given how much effort we've put into keeping 16-bit mode fully functional lately. We could add yet more heuristics and teach sigreturn to ignore the saved SS value in sigcontext if the saved CS is 64-bit and the saved SS is unusable.

Quoted 4 lines

Andy, why do you constantly ignore the proposal to make new behaviour explicitly controlable? You don't have to agree with it, but you could at least comment on that possibility and/or mention it with the ones you listed above.

I'm not sure what the proposal is exactly.

We could add a new uc_flags flag. If set, it means that sigcontext->ss is valid and should be used by sigreturn. If clear, then we ignore sigcontext->ss and just restore __USER_DS.

The problem is that, by itself, this won't fix old DOSEMU. We somehow need to either detect that something funny is going on or just leave the flag clear by default.

We could do this: always save SS to sigcontext->ss, but only restore sigcontext->ss if userspace explicitly sets the flag before sigreturn. If we do that, we'd need to also add my patch to preserve the actual HW SS selector if possible so that old DOSEMU knows what SS to program into its trampoline.

Quoted 4 lines

This at least lets *new* DOSEMU set the flag and get the improved behavior. I still don't know what effect it'll have on Wine and CRIU.

Stas, is that what you were thinking, or were you thinking of something else?

Not quite.
I mean the flag that will control not only sigreturn, but the signal delivery as well. This may probably be a sigaction() flag or some other. If not set - ss is ignored by both signal delivery and sigreturn(). If set - ss is saved/restored (and in the future - also fs/gs).
Is such a flag possible?

#46 lore raw
In reply to Linus Torvalds
Quoted 11 lines · Linus Torvalds
Quoted 4 lines · Andy Lutomirski

I'm trying to fix it without reverting. If that doesn't work, then we revert. Yesterday, I thought I had a reasonably clean fix, but it turned out that it only solved half of the problem.

The thing is, I actually think that the current situation is crazy.

Especially given that we don't restore any of the other segment registers on x86-64 (except CS, of course)

Quoted 8 lines

So how about this "alternate" minimal patch instead. The difference is:

- we actually leave the

        regs->ss = __USER_DS;

in __setup_rt_frame, to guarantee that when we take a signal, we do
take it with a valid SS

That by itself is enough to break DOSEMU. I think we may be stuck with my hack to only replace regs->ss if the old one was invalid.

Quoted 6 lines

- but it removes all the other games with SS (and treats it exactly
the same as FS/GS).

So now we don't play games with the actual sigcontext, and
hopefully dosemu is happier.

You mean that we always set ss to __USER_DS on sigreturn? The problem is that user code *can't* program SS when calling sigreturn because the SYSCALL instruction zaps it.

I'll try to implement something.

If this regression were new in 4.2-rc, then I'd say revert first and ask questions later, but the regression is in 4.1 as well :(

--Andy

#27 lore raw
In reply to Stas Sergeev
Quoted 41 lines · Stas Sergeev

13.08.2015 19:09, Andy Lutomirski пишет:

Quoted 33 lines · Stas Sergeev
Quoted 11 lines

13.08.2015 18:38, Andy Lutomirski пишет:

Quoted 8 lines

So... what do we do about it? We could revert the whole mess. We could tell everyone to fix their DOSEMU, which violates policy and is especially annoying given how much effort we've put into keeping 16-bit mode fully functional lately. We could add yet more heuristics and teach sigreturn to ignore the saved SS value in sigcontext if the saved CS is 64-bit and the saved SS is unusable.

Quoted 4 lines

Andy, why do you constantly ignore the proposal to make new behaviour explicitly controlable? You don't have to agree with it, but you could at least comment on that possibility and/or mention it with the ones you listed above.

I'm not sure what the proposal is exactly.

We could add a new uc_flags flag. If set, it means that sigcontext->ss is valid and should be used by sigreturn. If clear, then we ignore sigcontext->ss and just restore __USER_DS.

The problem is that, by itself, this won't fix old DOSEMU. We somehow need to either detect that something funny is going on or just leave the flag clear by default.

We could do this: always save SS to sigcontext->ss, but only restore sigcontext->ss if userspace explicitly sets the flag before sigreturn. If we do that, we'd need to also add my patch to preserve the actual HW SS selector if possible so that old DOSEMU knows what SS to program into its trampoline.

Quoted 5 lines

This at least lets *new* DOSEMU set the flag and get the improved behavior. I still don't know what effect it'll have on Wine and CRIU.

Stas, is that what you were thinking, or were you thinking of something else?

Quoted 7 lines

Not quite.
I mean the flag that will control not only sigreturn, but the signal delivery as well. This may probably be a sigaction() flag or some other. If not set - ss is ignored by both signal delivery and sigreturn(). If set - ss is saved/restored (and in the future - also fs/gs).
Is such a flag possible?

Maybe. I think I'm more nervous about adding new flags in sigaction than I am in uc_flags.

--Andy

#47 lore raw
In reply to Andy Lutomirski
Quoted 7 lines · Andy Lutomirski
Quoted 5 lines · Linus Torvalds

So how about this "alternate" minimal patch instead. The difference is:

- we actually leave the

Quoted 7 lines
Quoted 4 lines
        regs->ss = __USER_DS;

in __setup_rt_frame, to guarantee that when we take a signal, we do
take it with a valid SS

That by itself is enough to break DOSEMU. I think we may be stuck with my hack to only replace regs->ss if the old one was invalid.

Are you sure? From the description by Stas, the problem is literally the *restoring* action of the sigcontext, and trying to restore a SS value that is no longer valid.

"The crash happens when DOS program terminates.
At that point dosemu subverts the execution flow by replacing segregs and cs/ip ss/sp in sigcontext with its own. But __pad0 still has DOS SS, which crash because (presumably) the DOS LDT have been just removed"

and my "truly-minimal" patch removes all of the sigcontext games.

Quoted 1 line

You mean that we always set ss to __USER_DS on sigreturn?

No. We never touch SS at sigreturn time at all. Only when entering the signal *handler* do we reset things to a known state. The signal handler can do anything it wants, and sigreturn won't touch it (which will obviously _leave_ it as __USER_DS, but avoids the problem with sigreturn trying to load an SS that is no longer valid)

Quoted 2 lines

If this regression were new in 4.2-rc, then I'd say revert first and ask questions later, but the regression is in 4.1 as well :(

Big deal. That's why we have the "cc stable". Distributions that ship with 4.1 are still fairly few (but it's a LTS release so it will grow) but they all pick up stable kernels.

And even if they temporarily have a broken situation, it's still better to make sure that broken situation gets fixed, rather than say "oh well, too late to do anything about it now".

Linus

#28 lore raw
In reply to Andy Lutomirski

13.08.2015 19:24, Andy Lutomirski пишет:

Quoted 44 lines · Stas Sergeev
Quoted 36 lines

13.08.2015 19:09, Andy Lutomirski пишет:

Quoted 29 lines · Stas Sergeev
Quoted 8 lines

13.08.2015 18:38, Andy Lutomirski пишет:

Quoted 7 lines

So... what do we do about it? We could revert the whole mess. We could tell everyone to fix their DOSEMU, which violates policy and is especially annoying given how much effort we've put into keeping 16-bit mode fully functional lately. We could add yet more heuristics and teach sigreturn to ignore the saved SS value in sigcontext if the saved CS is 64-bit and the saved SS is unusable.

Quoted 4 lines

Andy, why do you constantly ignore the proposal to make new behaviour explicitly controlable? You don't have to agree with it, but you could at least comment on that possibility and/or mention it with the ones you listed above.

I'm not sure what the proposal is exactly.

We could add a new uc_flags flag. If set, it means that sigcontext->ss is valid and should be used by sigreturn. If clear, then we ignore sigcontext->ss and just restore __USER_DS.

The problem is that, by itself, this won't fix old DOSEMU. We somehow need to either detect that something funny is going on or just leave the flag clear by default.

We could do this: always save SS to sigcontext->ss, but only restore sigcontext->ss if userspace explicitly sets the flag before sigreturn. If we do that, we'd need to also add my patch to preserve the actual HW SS selector if possible so that old DOSEMU knows what SS to program into its trampoline.

Quoted 5 lines

This at least lets *new* DOSEMU set the flag and get the improved behavior. I still don't know what effect it'll have on Wine and CRIU.

Stas, is that what you were thinking, or were you thinking of something else?

Quoted 7 lines

Not quite.
I mean the flag that will control not only sigreturn, but the signal delivery as well. This may probably be a sigaction() flag or some other. If not set - ss is ignored by both signal delivery and sigreturn(). If set - ss is saved/restored (and in the future - also fs/gs).
Is such a flag possible?

Quoted 2 lines

Maybe. I think I'm more nervous about adding new flags in sigaction than I am in uc_flags.

Isn't uc_flags read-only for the user?
I look into setup_rt_frame <http://lxr.free-electrons.com/ident?v=2.4.37;i=setup_rt_frame>() and see
---
/* Create the ucontext. */
err |= __put_user(0, &frame->uc.uc_flags);
---
so it doesn't look like the flag that user can use to _request_
something from the kernel. And I am talking about exactly
the flag to request the new behaviour, as only that can remove
the regression completely without patching dosemu.

#29 lore raw
Quoted 56 lines · Stas Sergeev

13.08.2015 19:24, Andy Lutomirski пишет:

Quoted 51 lines · Stas Sergeev
Quoted 42 lines

13.08.2015 19:09, Andy Lutomirski пишет:

Quoted 33 lines · Stas Sergeev
Quoted 11 lines

13.08.2015 18:38, Andy Lutomirski пишет:

Quoted 8 lines

So... what do we do about it? We could revert the whole mess. We could tell everyone to fix their DOSEMU, which violates policy and is especially annoying given how much effort we've put into keeping 16-bit mode fully functional lately. We could add yet more heuristics and teach sigreturn to ignore the saved SS value in sigcontext if the saved CS is 64-bit and the saved SS is unusable.

Quoted 4 lines

Andy, why do you constantly ignore the proposal to make new behaviour explicitly controlable? You don't have to agree with it, but you could at least comment on that possibility and/or mention it with the ones you listed above.

I'm not sure what the proposal is exactly.

We could add a new uc_flags flag. If set, it means that sigcontext->ss is valid and should be used by sigreturn. If clear, then we ignore sigcontext->ss and just restore __USER_DS.

The problem is that, by itself, this won't fix old DOSEMU. We somehow need to either detect that something funny is going on or just leave the flag clear by default.

We could do this: always save SS to sigcontext->ss, but only restore sigcontext->ss if userspace explicitly sets the flag before sigreturn. If we do that, we'd need to also add my patch to preserve the actual HW SS selector if possible so that old DOSEMU knows what SS to program into its trampoline.

Quoted 5 lines

This at least lets *new* DOSEMU set the flag and get the improved behavior. I still don't know what effect it'll have on Wine and CRIU.

Stas, is that what you were thinking, or were you thinking of something else?

Quoted 7 lines

Not quite.
I mean the flag that will control not only sigreturn, but the signal delivery as well. This may probably be a sigaction() flag or some other. If not set - ss is ignored by both signal delivery and sigreturn(). If set - ss is saved/restored (and in the future - also fs/gs).
Is such a flag possible?

Quoted 2 lines

Maybe. I think I'm more nervous about adding new flags in sigaction than I am in uc_flags.

Quoted 11 lines

Isn't uc_flags read-only for the user?
I look into setup_rt_frame
<http://lxr.free-electrons.com/ident?v=2.4.37;i=setup_rt_frame>() and see
---
/* Create the ucontext. */
err |= __put_user(0, &frame->uc.uc_flags);
---
so it doesn't look like the flag that user can use to _request_ something from the kernel. And I am talking about exactly
the flag to request the new behaviour, as only that can remove the regression completely without patching dosemu.

User code could rewrite it in the signal handler to request something.

--Andy

#48 lore raw
In reply to Linus Torvalds
Quoted 10 lines · Linus Torvalds

Are you sure? From the description by Stas, the problem is literally the *restoring* action of the sigcontext, and trying to restore a SS value that is no longer valid.

"The crash happens when DOS program terminates.
At that point dosemu subverts the execution flow by replacing segregs and cs/ip ss/sp in sigcontext with its own. But __pad0 still has DOS SS, which crash because (presumably) the DOS LDT have been just removed"

Side note: if this is the main issue, and the problem is the "iret" faulting when trying to restore SS (and causing an unexpected SIGSEGV that dosemu crashes on), then an alternate model might be to keep the save/restore SS code, but do a "VERW" on the SS descriptor in restore_sigcontext(), and silently just replacing it with __USER_DS if that fails.

Linus

#49 lore raw
Quoted 13 lines · Linus Torvalds
Quoted 10 lines · Linus Torvalds

Are you sure? From the description by Stas, the problem is literally the *restoring* action of the sigcontext, and trying to restore a SS value that is no longer valid.

"The crash happens when DOS program terminates.
At that point dosemu subverts the execution flow by replacing segregs and cs/ip ss/sp in sigcontext with its own. But __pad0 still has DOS SS, which crash because (presumably) the DOS LDT have been just removed"

Quoted 7 lines

Side note: if this is the main issue, and the problem is the "iret" faulting when trying to restore SS (and causing an unexpected SIGSEGV that dosemu crashes on), then an alternate model might be to keep the save/restore SS code, but do a "VERW" on the SS descriptor in restore_sigcontext(), and silently just replacing it with __USER_DS if that fails.

I have a patch to do approximately that (using LAR instead of VERW to rule out DPL < 3.

I'm 90% sure that the regs->ss = __USER_DS thing is a problem. Read farther up in the thread.

I'm leaning toward saying we should revert, cc: stable, and fix it better for 4.3.

--Andy

#30 lore raw
In reply to Andy Lutomirski

13.08.2015 19:42, Andy Lutomirski пишет:

Quoted 60 lines · Stas Sergeev
Quoted 48 lines

13.08.2015 19:24, Andy Lutomirski пишет:

Quoted 44 lines · Stas Sergeev
Quoted 36 lines

13.08.2015 19:09, Andy Lutomirski пишет:

Quoted 29 lines · Stas Sergeev
Quoted 8 lines

13.08.2015 18:38, Andy Lutomirski пишет:

Quoted 7 lines

So... what do we do about it? We could revert the whole mess. We could tell everyone to fix their DOSEMU, which violates policy and is especially annoying given how much effort we've put into keeping 16-bit mode fully functional lately. We could add yet more heuristics and teach sigreturn to ignore the saved SS value in sigcontext if the saved CS is 64-bit and the saved SS is unusable.

Quoted 4 lines

Andy, why do you constantly ignore the proposal to make new behaviour explicitly controlable? You don't have to agree with it, but you could at least comment on that possibility and/or mention it with the ones you listed above.

I'm not sure what the proposal is exactly.

We could add a new uc_flags flag. If set, it means that sigcontext->ss is valid and should be used by sigreturn. If clear, then we ignore sigcontext->ss and just restore __USER_DS.

The problem is that, by itself, this won't fix old DOSEMU. We somehow need to either detect that something funny is going on or just leave the flag clear by default.

We could do this: always save SS to sigcontext->ss, but only restore sigcontext->ss if userspace explicitly sets the flag before sigreturn. If we do that, we'd need to also add my patch to preserve the actual HW SS selector if possible so that old DOSEMU knows what SS to program into its trampoline.

Quoted 5 lines

This at least lets *new* DOSEMU set the flag and get the improved behavior. I still don't know what effect it'll have on Wine and CRIU.

Stas, is that what you were thinking, or were you thinking of something else?

Quoted 7 lines

Not quite.
I mean the flag that will control not only sigreturn, but the signal delivery as well. This may probably be a sigaction() flag or some other. If not set - ss is ignored by both signal delivery and sigreturn(). If set - ss is saved/restored (and in the future - also fs/gs).
Is such a flag possible?

Quoted 2 lines

Maybe. I think I'm more nervous about adding new flags in sigaction than I am in uc_flags.

Quoted 11 lines

Isn't uc_flags read-only for the user?
I look into setup_rt_frame
<http://lxr.free-electrons.com/ident?v=2.4.37;i=setup_rt_frame>() and see
---
/* Create the ucontext. */
err |= __put_user(0, &frame->uc.uc_flags);
---
so it doesn't look like the flag that user can use to _request_ something from the kernel. And I am talking about exactly
the flag to request the new behaviour, as only that can remove the regression completely without patching dosemu.

Quoted 1 line

User code could rewrite it in the signal handler to request something.

But that's too late to affect the signal _delivery_ anyhow, no? Any idea about the flag that can control both delivery and return?

#31 lore raw
Quoted 76 lines · Stas Sergeev

13.08.2015 19:42, Andy Lutomirski пишет:

Quoted 74 lines · Stas Sergeev
Quoted 72 lines

13.08.2015 19:24, Andy Lutomirski пишет:

Quoted 55 lines · Stas Sergeev
Quoted 46 lines

13.08.2015 19:09, Andy Lutomirski пишет:

Quoted 36 lines · Stas Sergeev
Quoted 14 lines

13.08.2015 18:38, Andy Lutomirski пишет:

Quoted 11 lines

So... what do we do about it? We could revert the whole mess. We could tell everyone to fix their DOSEMU, which violates policy and is
especially annoying given how much effort we've put into keeping 16-bit mode fully functional lately. We could add yet more heuristics
and teach sigreturn to ignore the saved SS value in sigcontext if the
saved CS is 64-bit and the saved SS is unusable.

Quoted 4 lines

Andy, why do you constantly ignore the proposal to make new behaviour explicitly controlable? You don't have to agree with it, but you could at least comment on that possibility and/or mention it with the ones you listed above.

I'm not sure what the proposal is exactly.

We could add a new uc_flags flag. If set, it means that sigcontext->ss is valid and should be used by sigreturn. If clear, then we ignore sigcontext->ss and just restore __USER_DS.

The problem is that, by itself, this won't fix old DOSEMU. We somehow need to either detect that something funny is going on or just leave the flag clear by default.

We could do this: always save SS to sigcontext->ss, but only restore sigcontext->ss if userspace explicitly sets the flag before sigreturn. If we do that, we'd need to also add my patch to preserve the actual HW SS selector if possible so that old DOSEMU knows what SS to program into its trampoline.

Quoted 6 lines

This at least lets *new* DOSEMU set the flag and get the improved behavior. I still don't know what effect it'll have on Wine and CRIU.

Stas, is that what you were thinking, or were you thinking of something
else?

Quoted 7 lines

Not quite.
I mean the flag that will control not only sigreturn, but the signal delivery as well. This may probably be a sigaction() flag or some other. If not set - ss is ignored by both signal delivery and sigreturn(). If set - ss is saved/restored (and in the future - also fs/gs).
Is such a flag possible?

Quoted 2 lines

Maybe. I think I'm more nervous about adding new flags in sigaction than I am in uc_flags.

Isn't uc_flags read-only for the user?
I look into setup_rt_frame
<http://lxr.free-electrons.com/ident?v=2.4.37;i=setup_rt_frame>() and see
---
/* Create the ucontext. */
err |= __put_user(0, &frame->uc.uc_flags);
---
so it doesn't look like the flag that user can use to _request_ something from the kernel. And I am talking about exactly
the flag to request the new behaviour, as only that can remove the regression completely without patching dosemu.

Quoted 4 lines
Quoted 1 line

User code could rewrite it in the signal handler to request something.

But that's too late to affect the signal _delivery_ anyhow, no? Any idea about the flag that can control both delivery and return?

I think my LAR patch should cover the signal delivery part.

--Andy

#50 lore raw
In reply to Andy Lutomirski
Quoted 22 lines · Andy Lutomirski
Quoted 7 lines · Linus Torvalds
Quoted 5 lines · Stas Sergeev

I realize this patch may be good to have in general, but breaking userspace without a single warning is a bit discouraging. Seems like the old "we don't break userspace" rule have gone.

Quoted 8 lines

That rule hasn't gone anywhere.

Does a plain revert just fix everything? Because if so, that's the right thing to do, and we can just re-visit this later.

I don't understand why Andy and Ingo are even discussing this. What the f*ck, guys?

I'm trying to fix it without reverting. If that doesn't work, then we revert. Yesterday, I thought I had a reasonably clean fix, but it turned out that it only solved half of the problem.

Quoted 5 lines

If we revert, I think I need to check what will break due to the revert. I need to check at least Wine, and we'll have to do something about all the selftests that will start failing. I also need to check CRIU, and IIRC CRIU has started using the new sigcontext SS in new versions.

I don't think Wine will be a problem, at least how it is currently set up. 16-bit support is only in the 32-bit build. The 64-bit build only supports Win64 apps, and will call the 32-bit version (installed in parallel) to run 32 and 16-bit apps.

--
Brian Gerst

#32 lore raw
In reply to Andy Lutomirski

13.08.2015 19:59, Andy Lutomirski пишет:

Quoted 67 lines · Stas Sergeev
Quoted 66 lines

13.08.2015 19:42, Andy Lutomirski пишет:

Quoted 64 lines · Stas Sergeev
Quoted 63 lines

13.08.2015 19:24, Andy Lutomirski пишет:

Quoted 48 lines · Stas Sergeev
Quoted 40 lines

13.08.2015 19:09, Andy Lutomirski пишет:

Quoted 32 lines · Stas Sergeev
Quoted 11 lines

13.08.2015 18:38, Andy Lutomirski пишет:

Quoted 10 lines

So... what do we do about it? We could revert the whole mess. We could tell everyone to fix their DOSEMU, which violates policy and is
especially annoying given how much effort we've put into keeping 16-bit mode fully functional lately. We could add yet more heuristics
and teach sigreturn to ignore the saved SS value in sigcontext if the
saved CS is 64-bit and the saved SS is unusable.

Quoted 4 lines

Andy, why do you constantly ignore the proposal to make new behaviour explicitly controlable? You don't have to agree with it, but you could at least comment on that possibility and/or mention it with the ones you listed above.

I'm not sure what the proposal is exactly.

We could add a new uc_flags flag. If set, it means that sigcontext->ss is valid and should be used by sigreturn. If clear, then we ignore sigcontext->ss and just restore __USER_DS.

The problem is that, by itself, this won't fix old DOSEMU. We somehow need to either detect that something funny is going on or just leave the flag clear by default.

We could do this: always save SS to sigcontext->ss, but only restore sigcontext->ss if userspace explicitly sets the flag before sigreturn. If we do that, we'd need to also add my patch to preserve the actual HW SS selector if possible so that old DOSEMU knows what SS to program into its trampoline.

Quoted 6 lines

This at least lets *new* DOSEMU set the flag and get the improved behavior. I still don't know what effect it'll have on Wine and CRIU.

Stas, is that what you were thinking, or were you thinking of something
else?

Quoted 7 lines

Not quite.
I mean the flag that will control not only sigreturn, but the signal delivery as well. This may probably be a sigaction() flag or some other. If not set - ss is ignored by both signal delivery and sigreturn(). If set - ss is saved/restored (and in the future - also fs/gs).
Is such a flag possible?

Quoted 2 lines

Maybe. I think I'm more nervous about adding new flags in sigaction than I am in uc_flags.

Isn't uc_flags read-only for the user?
I look into setup_rt_frame
<http://lxr.free-electrons.com/ident?v=2.4.37;i=setup_rt_frame>() and see
---
/* Create the ucontext. */
err |= __put_user(0, &frame->uc.uc_flags);
---
so it doesn't look like the flag that user can use to _request_ something from the kernel. And I am talking about exactly
the flag to request the new behaviour, as only that can remove the regression completely without patching dosemu.

Quoted 4 lines
Quoted 3 lines
Quoted 1 line

User code could rewrite it in the signal handler to request something.

But that's too late to affect the signal _delivery_ anyhow, no? Any idea about the flag that can control both delivery and return?

I think my LAR patch should cover the signal delivery part.

Ah, I see your point now.
But that's not what I mean, as it doesn't cover fs/gs, which is what Linus is looking to revert now too (I am building the testing kernels now).
So you obviously don't want the flag that will control all 3 things together without any lar heuristics, but I don't understand why... Yes, your heuristic+uc_flag may work, but IMHO far from perfection and TLS problem is not covered. I can test such
a patch but I don't understand why you don't want the flag that will just control all things together.

#33 lore raw
Quoted 93 lines · Stas Sergeev

13.08.2015 19:59, Andy Lutomirski пишет:

Quoted 84 lines · Stas Sergeev
Quoted 83 lines

13.08.2015 19:42, Andy Lutomirski пишет:

Quoted 80 lines · Stas Sergeev
Quoted 78 lines

13.08.2015 19:24, Andy Lutomirski пишет:

Quoted 60 lines · Stas Sergeev
Quoted 51 lines

13.08.2015 19:09, Andy Lutomirski пишет:

Quoted 40 lines · Stas Sergeev
Quoted 15 lines

13.08.2015 18:38, Andy Lutomirski пишет:

Quoted 12 lines

So... what do we do about it? We could revert the whole mess. We
could tell everyone to fix their DOSEMU, which violates policy and is
especially annoying given how much effort we've put into keeping 16-bit mode fully functional lately. We could add yet more heuristics
and teach sigreturn to ignore the saved SS value in sigcontext if the
saved CS is 64-bit and the saved SS is unusable.

Quoted 4 lines

Andy, why do you constantly ignore the proposal to make new behaviour explicitly controlable? You don't have to agree with it, but you could at least comment on that possibility and/or mention it with the ones you listed above.

I'm not sure what the proposal is exactly.

We could add a new uc_flags flag. If set, it means that sigcontext->ss is valid and should be used by sigreturn. If clear, then we ignore sigcontext->ss and just restore __USER_DS.

The problem is that, by itself, this won't fix old DOSEMU. We somehow
need to either detect that something funny is going on or just leave the flag clear by default.

We could do this: always save SS to sigcontext->ss, but only restore sigcontext->ss if userspace explicitly sets the flag before sigreturn.
If we do that, we'd need to also add my patch to preserve the actual HW SS selector if possible so that old DOSEMU knows what SS to program
into its trampoline.

Quoted 7 lines

This at least lets *new* DOSEMU set the flag and get the improved behavior. I still don't know what effect it'll have on Wine and CRIU.

Stas, is that what you were thinking, or were you thinking of something
else?

Quoted 7 lines

Not quite.
I mean the flag that will control not only sigreturn, but the signal delivery as well. This may probably be a sigaction() flag or some other. If not set - ss is ignored by both signal delivery and sigreturn(). If set - ss is saved/restored (and in the future - also fs/gs).
Is such a flag possible?

Quoted 2 lines

Maybe. I think I'm more nervous about adding new flags in sigaction than I am in uc_flags.

Isn't uc_flags read-only for the user?
I look into setup_rt_frame
<http://lxr.free-electrons.com/ident?v=2.4.37;i=setup_rt_frame>() and see
---
/* Create the ucontext. */
err |= __put_user(0, &frame->uc.uc_flags);
---
so it doesn't look like the flag that user can use to _request_ something from the kernel. And I am talking about exactly the flag to request the new behaviour, as only that can remove the regression completely without patching dosemu.

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

User code could rewrite it in the signal handler to request something.

But that's too late to affect the signal _delivery_ anyhow, no? Any idea about the flag that can control both delivery and return?

I think my LAR patch should cover the signal delivery part.

Quoted 10 lines

Ah, I see your point now.
But that's not what I mean, as it doesn't cover fs/gs, which is what Linus is looking to revert now too (I am building the testing kernels now).
So you obviously don't want the flag that will control all 3 things together without any lar heuristics, but I don't understand why... Yes, your heuristic+uc_flag may work, but IMHO far from perfection and TLS problem is not covered. I can test such
a patch but I don't understand why you don't want the flag that will just control all things together.

The fs/gs patch doesn't change anything, so there's nothing to control. It just renamed fields that did nothing. (It turns out they did something back before arch_prctl existed, but there's only a narrow range of kernels like that, and I'm not at all convinced that those kernels are ABI-compatible with modern kernels at all. This is all pre-git.)

Sure, it might make sense to change TLS behavior in signals at some point, but I don't think we're there yet. We need to deal with fsgsbase first, and that's a *huge* can of worms.

--Andy

Signature · 2 lines
Andy Lutomirski
AMA Capital Management, LLC
#73 lore raw
In reply to Linus Torvalds

13.08.2015 18:37, Linus Torvalds пишет:

Quoted 10 lines · Stas Sergeev
Quoted 4 lines

I realize this patch may be good to have in general, but breaking userspace without a single warning is a bit discouraging. Seems like the old "we don't break userspace" rule have gone.

That rule hasn't gone anywhere.

Does a plain revert just fix everything? Because if so, that's the right thing to do, and we can just re-visit this later.

Quoted 6 lines

I don't understand why Andy and Ingo are even discussing this. What the f*ck, guys?

Stas, can you verify that this actually fixes it? There's two different versions here: one that reverts *just* that one commit, and one that reverts the fs/gs changes too. Can you test them both?

Hello Linus, I verified that patch-minimal.diff is enough to fix the problem, BUT! dosemu is in fact using the .fs and .gs fields of sigcontext as a placeholders. Why the minimal patch alone helps is simply because the kernel headers installed in a system do not yet represent the newer kernel developments and have the .fs and .gs fields in.
So, to allow the pre-compiled dosemu binary to run, patch-minimal is enough. To allow re-compiling it, you'd need patch.diff. Although I guess compilation is not the point, and the fact that the pre-compiled binary works, is enough.

Now my PC is dying of overheat, I am struggling to even boot it. I am not sure I'll test the "really-minimal" patch, but it will unlikely to help because dosemu expects unmodified ss in a sighandler, which "really-minimal" patch doesn't seem to give.

#34 lore raw
In reply to Andy Lutomirski

13.08.2015 20:17, Andy Lutomirski пишет:

Quoted 12 lines · Stas Sergeev
Quoted 11 lines

Ah, I see your point now.
But that's not what I mean, as it doesn't cover fs/gs, which is what Linus is looking to revert now too (I am building the testing kernels now).
So you obviously don't want the flag that will control all 3 things together without any lar heuristics, but I don't understand why... Yes, your heuristic+uc_flag may work, but IMHO far from perfection and TLS problem is not covered. I can test such
a patch but I don't understand why you don't want the flag that will just control all things together.

Quoted 6 lines

The fs/gs patch doesn't change anything, so there's nothing to control. It just renamed fields that did nothing. (It turns out they did something back before arch_prctl existed, but there's only a narrow range of kernels like that, and I'm not at all convinced that those kernels are ABI-compatible with modern kernels at all. This is all pre-git.)

The problem is that dosemu existed back then too.
It still uses these fields as a place-holders. Well, this is a compile-time breakage only, so perhaps not as important as the run-time one, but still, you broke it in yet another way.

Quoted 3 lines

Sure, it might make sense to change TLS behavior in signals at some point, but I don't think we're there yet. We need to deal with fsgsbase first, and that's a *huge* can of worms.

My point is not when to fix TLS or how.
But you can get the flag ready, for now controlling only SS and fixing the regression, but it will define the course of the further developments. When the time will come, it will cover also TLS, but why not to get such a flag ready now, without yet fixing TLS?

#35 lore raw
Quoted 22 lines · Stas Sergeev

13.08.2015 20:17, Andy Lutomirski пишет:

Quoted 14 lines · Stas Sergeev
Quoted 11 lines

Ah, I see your point now.
But that's not what I mean, as it doesn't cover fs/gs, which is what Linus is looking to revert now too (I am building the testing kernels now).
So you obviously don't want the flag that will control all 3 things together without any lar heuristics, but I don't understand why... Yes, your heuristic+uc_flag may work, but IMHO far from perfection and TLS problem is not covered. I can test such
a patch but I don't understand why you don't want the flag that will just control all things together.

Quoted 6 lines

The fs/gs patch doesn't change anything, so there's nothing to control. It just renamed fields that did nothing. (It turns out they did something back before arch_prctl existed, but there's only a narrow range of kernels like that, and I'm not at all convinced that those kernels are ABI-compatible with modern kernels at all. This is all pre-git.)

Quoted 4 lines

The problem is that dosemu existed back then too.
It still uses these fields as a place-holders. Well, this is a compile-time breakage only, so perhaps not as important as the run-time one, but still, you broke it in yet another way.

Great. What exactly is DOSEMU sticking in those fields? Are we now stuck ignoring the contents in sigreturn because DOSEMU coopts them for its own purposes?

Quoted 11 lines
Quoted 4 lines

Sure, it might make sense to change TLS behavior in signals at some point, but I don't think we're there yet. We need to deal with fsgsbase first, and that's a *huge* can of worms.

My point is not when to fix TLS or how.
But you can get the flag ready, for now controlling only SS and fixing the regression, but it will define the course of the further developments. When the time will come, it will cover also TLS, but why not to get such a flag ready now, without yet fixing TLS?

I think that if we create a flag to change semantics, we shouldn't introduce the flag and make it look like it works without actually changing the semantics.

--Andy

Signature · 2 lines
Andy Lutomirski
AMA Capital Management, LLC
#36 lore raw

13.08.2015 21:05, Andy Lutomirski пишет:

Quoted 20 lines · Stas Sergeev
Quoted 19 lines

13.08.2015 20:17, Andy Lutomirski пишет:

Quoted 18 lines · Stas Sergeev
Quoted 11 lines

Ah, I see your point now.
But that's not what I mean, as it doesn't cover fs/gs, which is what Linus is looking to revert now too (I am building the testing kernels now).
So you obviously don't want the flag that will control all 3 things together without any lar heuristics, but I don't understand why... Yes, your heuristic+uc_flag may work, but IMHO far from perfection and TLS problem is not covered. I can test such
a patch but I don't understand why you don't want the flag that will just control all things together.

The fs/gs patch doesn't change anything, so there's nothing to control. It just renamed fields that did nothing. (It turns out they did something back before arch_prctl existed, but there's only a narrow range of kernels like that, and I'm not at all convinced that those kernels are ABI-compatible with modern kernels at all. This is all pre-git.)

Quoted 5 lines
Quoted 4 lines

The problem is that dosemu existed back then too.
It still uses these fields as a place-holders. Well, this is a compile-time breakage only, so perhaps not as important as the run-time one, but still, you broke it in yet another way.

Great. What exactly is DOSEMU sticking in those fields?

FS and GS of course. Saves by hands in a sighandler.

Quoted 3 lines

Are we now
stuck ignoring the contents in sigreturn because DOSEMU coopts them for its own purposes?

No, its just that these fields _were_ valid in times, and so, when kernel stopped using them, dosemu authors opted not to change their locations but just save things by hands. What else do you suppose could they do?
Well, compile-time breakage is another thing, it can safely be ignored on your side I guess.
Something like this should do:
https://github.com/stsp/dosemu2/commit/4e16d83b9c7b1f9155ec0c0c125fe6f755eb719c

Quoted 9 lines
Quoted 9 lines
Quoted 3 lines

Sure, it might make sense to change TLS behavior in signals at some point, but I don't think we're there yet. We need to deal with fsgsbase first, and that's a *huge* can of worms.

My point is not when to fix TLS or how.
But you can get the flag ready, for now controlling only SS and fixing the regression, but it will define the course of the further developments. When the time will come, it will cover also TLS, but why not to get such a flag ready now, without yet fixing TLS?

Quoted 3 lines

I think that if we create a flag to change semantics, we shouldn't introduce the flag and make it look like it works without actually changing the semantics.

It is more about selecting the right field for such a flag. You can select the right field now, and introduce some flag to it, like SIG_SAVE_SS or whatever. This will fix a regression. Then, when the TLS time will code, you'll just add SIG_SAVE_FS flag to the same field, so that they can be ORed.

#37 lore raw
Quoted 5 lines · Stas Sergeev

It is more about selecting the right field for such a flag. You can select the right field now, and introduce some flag to it, like SIG_SAVE_SS or whatever. This will fix a regression. Then, when the TLS time will code, you'll just add SIG_SAVE_FS flag to the same field, so that they can be ORed.

Oh.

I think the field is obvious: uc_flags. I also thing that all of the saving should happen automatically, since I still don't see how anything will break if the kernel starts saving more things. It's the restore part that's problematic.

--Andy

Signature · 2 lines
Andy Lutomirski
AMA Capital Management, LLC
#74 lore raw
In reply to Stas Sergeev
Quoted 7 lines · Stas Sergeev

Hello Linus, I verified that patch-minimal.diff is enough to fix the problem, BUT! dosemu is in fact using the .fs and .gs fields of sigcontext as a placeholders. Why the minimal patch alone helps is simply because the kernel headers installed in a system do not yet represent the newer kernel developments and have the .fs and .gs fields in.

Ok. So I'm inclined to do the bigger revert, just to fix the compile issue. It would be crazy to force some silly autoconf script for random header info.

Of course, we could also just let the uabi version go out of sync with the internal version, but that sounds even worse. I think we'd be better off just leaving the old names, and just having big comments about this..

Andy? I agree that we should strive to improve in this area, and this should be worked on, but that would seem to be a 4.3 issue (and mark that too for stable once it actually works). No?

Linus

#38 lore raw
In reply to Andy Lutomirski

13.08.2015 21:25, Andy Lutomirski пишет:

Quoted 10 lines · Stas Sergeev
Quoted 6 lines

It is more about selecting the right field for such a flag. You can select the right field now, and introduce some flag to it, like SIG_SAVE_SS or whatever. This will fix a regression. Then, when the TLS time will code, you'll just add SIG_SAVE_FS flag to the same field, so that they can be ORed.

Oh.

I think the field is obvious: uc_flags.

But Andy, I don't understand...
If we are talking about the field that will in the future also cover TLS, how can this be uc_flags? By using uc_flags, how will you control the restoring of FS on signal delivery?

Quoted 4 lines

I also thing that all of the
saving should happen automatically, since I still don't see how anything will break if the kernel starts saving more things. It's the restore part that's problematic.

OK, so SIG_RESTORE_SS and SIG_RESTORE_FS. How can those be the part of uc_flags, if we want to control the restoring _on a signal delivery_?

#75 lore raw
In reply to Linus Torvalds
Quoted 13 lines · Linus Torvalds
Quoted 7 lines · Stas Sergeev

Hello Linus, I verified that patch-minimal.diff is enough to fix the problem, BUT! dosemu is in fact using the .fs and .gs fields of sigcontext as a placeholders. Why the minimal patch alone helps is simply because the kernel headers installed in a system do not yet represent the newer kernel developments and have the .fs and .gs fields in.

Ok. So I'm inclined to do the bigger revert, just to fix the compile issue. It would be crazy to force some silly autoconf script for random header info.

Quoted 8 lines

Of course, we could also just let the uabi version go out of sync with the internal version, but that sounds even worse. I think we'd be better off just leaving the old names, and just having big comments about this..

Andy? I agree that we should strive to improve in this area, and this should be worked on, but that would seem to be a 4.3 issue (and mark that too for stable once it actually works). No?

Yeah, probably makes sense, but one of us should explicitly test CRIU (both new and old versions) on the result. CRIU does interesting things involving sigcontext and protocol buffers.

We can do something a bit silly wrt UABI in the long run:

union { unsigned long __pad0; unsigned long ss; };

I have half-written patches to do better, but I don't think they're 4.2 material. I'm already vastly over my quota for late-arriving 4.2 things.

Stas: I think uc_flags is okay. We don't currently read it during sigreturn, but I see no reason that we can't start reading it.

--Andy

#114 lore raw
In reply to Linus Torvalds

13.08.2015 21:35, Linus Torvalds пишет:

Quoted 10 lines · Stas Sergeev
Quoted 6 lines

Hello Linus, I verified that patch-minimal.diff is enough to fix the problem, BUT! dosemu is in fact using the .fs and .gs fields of sigcontext as a placeholders. Why the minimal patch alone helps is simply because the kernel headers installed in a system do not yet represent the newer kernel developments and have the .fs and .gs fields in.

Ok. So I'm inclined to do the bigger revert, just to fix the compile issue. It would be crazy to force some silly autoconf script for random header info.

But OTOH these fields already lost their meaning.
It may make sense to force people to stop using them, in case you ever want to re-use them again in the future. From what Andy says, it seems there are the distant plans to start restoring FS again. If people still use sigcontext.fs by that time, you'll get problems. If you force everyone to stop using them - you'll be safe.

Also, at least in the past, resolving the compile-time problems was up to the distributions: they always provided the "sanitized up" version of kernel headers. Not sure what the current policy is... In fact, here in Fedora-22, I have /usr/include/asm/sigcontext.h
that is straight from the kernel, but signal.h is instead using a "sanitized up" version in /usr/include/bits/sigcontext.h
so the userspace compiles fine.

In fact, I think the "silly autoconf script" you mentioned above, should indeed be reverted, and instead I should use sigcontext.reserved1[8] array to store FS/GS? Is this safer against ever re-using this space? Not sure...

#115 lore raw
Quoted 15 lines · Stas Sergeev

13.08.2015 21:35, Linus Torvalds пишет:

Quoted 10 lines · Stas Sergeev
Quoted 7 lines

Hello Linus, I verified that patch-minimal.diff is enough to fix the problem, BUT! dosemu is in fact using the .fs and .gs fields of sigcontext as a placeholders. Why the minimal patch alone helps is simply because the kernel headers installed in a system do not yet represent the newer kernel developments and have the .fs and .gs fields in.

Quoted 3 lines

Ok. So I'm inclined to do the bigger revert, just to fix the compile issue. It would be crazy to force some silly autoconf script for random header info.

Quoted 7 lines

But OTOH these fields already lost their meaning.
It may make sense to force people to stop using them, in case you ever want to re-use them again in the future. From what Andy says, it seems there are the distant plans to start restoring FS again. If people still use sigcontext.fs by that time, you'll get problems. If you force everyone to stop using them - you'll be safe.

There are distant plans to think about restoring them, at least. But it's not just FS -- it's FSBASE as well, and that's not going to fit in the same slot. And we still don't get to break old DOSEMU versions (knowingly, anyway).

Quoted 5 lines

In fact, I think the "silly autoconf script" you mentioned above, should indeed be reverted, and instead I should use sigcontext.reserved1[8] array to store FS/GS? Is this safer against ever re-using this space? Not sure...

Honestly, I'd just save it somewhere outside sigcontext. If it's application data, treat it as such. OTOH, if you've already been saving it in the old FS/GS slots, I see no great reason to change it.

--Andy

Signature · 2 lines
Andy Lutomirski
AMA Capital Management, LLC
#76 lore raw
In reply to Andy Lutomirski

13.08.2015 21:41, Andy Lutomirski пишет:

Quoted 2 lines

Stas: I think uc_flags is okay. We don't currently read it during sigreturn, but I see no reason that we can't start reading it.

Andy, we definitely have some communication discontinuity here. :) The point is not sigreturn. If we are talking about the flags that will in the future control also TLS, how would you limit it to sigreturn()? It should control the restoring of FS _on signal delivery_, not only on sigreturn()! So how uc_flags can be used for this at all?

#116 lore raw
In reply to Andy Lutomirski

13.08.2015 22:01, Andy Lutomirski пишет:

Quoted 24 lines · Stas Sergeev
Quoted 11 lines

13.08.2015 21:35, Linus Torvalds пишет:

Quoted 10 lines · Stas Sergeev
Quoted 6 lines

Hello Linus, I verified that patch-minimal.diff is enough to fix the problem, BUT! dosemu is in fact using the .fs and .gs fields of sigcontext as a placeholders. Why the minimal patch alone helps is simply because the kernel headers installed in a system do not yet represent the newer kernel developments and have the .fs and .gs fields in.

Ok. So I'm inclined to do the bigger revert, just to fix the compile issue. It would be crazy to force some silly autoconf script for random header info.

Quoted 7 lines

But OTOH these fields already lost their meaning.
It may make sense to force people to stop using them, in case you ever want to re-use them again in the future.
From what Andy says, it seems there are the distant plans
to start restoring FS again. If people still use sigcontext.fs by that time, you'll get problems. If you force everyone to stop using them - you'll be safe.

There are distant plans to think about restoring them, at least. But it's not just FS -- it's FSBASE as well, and that's not going to fit in the same slot. And we still don't get to break old DOSEMU versions (knowingly, anyway).

Quoted 7 lines
Quoted 4 lines

In fact, I think the "silly autoconf script" you mentioned above, should indeed be reverted, and instead I should use sigcontext.reserved1[8] array to store FS/GS? Is this safer against ever re-using this space? Not sure...

Honestly, I'd just save it somewhere outside sigcontext. If it's application data, treat it as such. OTOH, if you've already been saving it in the old FS/GS slots, I see no great reason to change it.

OK, as you promise not to re-use the same slots in the future, I won't change it. Thanks.
As for the compilation failure - I am surprised you even care. I thought the "we don't break userspace" covers only run-time, not compile-time. Oh well.

#117 lore raw
Quoted 4 lines · Stas Sergeev

As for the compilation failure - I am surprised you even care. I thought the "we don't break userspace" covers only run-time, not compile-time. Oh well.

I definitely care.

Compile issues may be slightly lower on my radar, but the basic rule should be that upgrading a kernel shouldn't cause problems.

The only exception is for stuff that gets very intimate with the kernel itself - ie things like external modules etc. Those we don't really try to cater to, or care about.

Things that actually include internal kernel headers tend to have just themselves to blame and historically we really didn't care very much (long long ago we had issues with ext3fs-tools etc until those kinds of things just ended up making their own copies).

But the point of the uabi headers was that even _that_ is supposed to actually work, at least for the limited case of those headers.. So if a uabi header change causes problems, we really *should* care, because otherwise, what was the point of that whole uabi rework, really?

(And no, we've not always succeeded in that "shouldn't cause problems" space. So I'm not claiming we have a perfect track record, but I do claim that I at least care very deeply)

Linus

#77 lore raw
In reply to Stas Sergeev
Quoted 11 lines · Stas Sergeev

13.08.2015 21:41, Andy Lutomirski пишет:

Quoted 2 lines

Stas: I think uc_flags is okay. We don't currently read it during sigreturn, but I see no reason that we can't start reading it.

Andy, we definitely have some communication discontinuity here. :) The point is not sigreturn. If we are talking about the flags that will in the future control also TLS, how would you limit it to sigreturn()? It should control the restoring of FS _on signal delivery_, not only on sigreturn()! So how uc_flags can be used for this at all?

Ah, you want it restored on signal delivery. What would it be restored to? ISTM that can be done easily enough in user code, so maybe we should leave it to user code.

--Andy

#79 lore raw
In reply to Andy Lutomirski
Quoted 10 lines · Andy Lutomirski
Quoted 4 lines · Linus Torvalds

Ok. So I'm inclined to do the bigger revert, just to fix the compile issue. It would be crazy to force some silly autoconf script for random header info.

Yeah, probably makes sense, but one of us should explicitly test CRIU (both new and old versions) on the result. CRIU does interesting things involving sigcontext and protocol buffers.

I've reverted it in the -git tree, I'm adding Pavel and Cyrill to the cc, to see if they can check the impact on CRIU..

Pavel/Cyrill? Current top-of-tree (but that will change) commit ed596cde9425 ("Revert x86 sigcontext cleanups").

Linus

#118 lore raw
In reply to Linus Torvalds

13.08.2015 22:37, Linus Torvalds пишет:

Quoted 8 lines · Stas Sergeev
Quoted 3 lines

As for the compilation failure - I am surprised you even care. I thought the "we don't break userspace" covers only run-time, not compile-time. Oh well.

I definitely care.

Compile issues may be slightly lower on my radar, but the basic rule should be that upgrading a kernel shouldn't cause problems.

It doesn't: fedora provides a "sanitized up" version of sigcontext.h in /usr/include/bits, which comes from glibc-headers-2.21-7.fc22.x86_64. So it seems the "sanitized up" headers come from glibc, which means all other distros would have that too.

Quoted 9 lines

The only exception is for stuff that gets very intimate with the kernel itself - ie things like external modules etc. Those we don't really try to cater to, or care about.

Things that actually include internal kernel headers tend to have just themselves to blame and historically we really didn't care very much (long long ago we had issues with ext3fs-tools etc until those kinds of things just ended up making their own copies).

Quoted 4 lines

But the point of the uabi headers was that even _that_ is supposed to actually work, at least for the limited case of those headers.. So if a uabi header change causes problems, we really *should* care, because otherwise, what was the point of that whole uabi rework, really?

I thought this is mainly to help glibc and distributors to provide a sanitized-up versions.
If it was for something else, then this "something else" didn't reach the user PCs yet. :) Fedora-22 still has the sanitized version in the glibc-headers package, so no compilation break because of your change was ever actually observed.

#119 lore raw
Quoted 5 lines · Stas Sergeev

It doesn't: fedora provides a "sanitized up" version of sigcontext.h in /usr/include/bits, which comes from glibc-headers-2.21-7.fc22.x86_64. So it seems the "sanitized up" headers come from glibc, which means all other distros would have that too.

Yes. Except the whole point of uapi was that in the long term the glibc people should just be able to pick up the kernel ones directly.

And while that may involve some further editing, it sure as hell shouldn't involve "oh, I know, this got renamed, so now I have to rename it back". That would be crazy.

Linus

#80 lore raw
In reply to Linus Torvalds
Quoted 12 lines · Linus Torvalds
Quoted 10 lines · Andy Lutomirski
Quoted 4 lines · Linus Torvalds

Ok. So I'm inclined to do the bigger revert, just to fix the compile issue. It would be crazy to force some silly autoconf script for random header info.

Yeah, probably makes sense, but one of us should explicitly test CRIU (both new and old versions) on the result. CRIU does interesting things involving sigcontext and protocol buffers.

Quoted 5 lines

I've reverted it in the -git tree, I'm adding Pavel and Cyrill to the cc, to see if they can check the impact on CRIU..

Pavel/Cyrill? Current top-of-tree (but that will change) commit ed596cde9425 ("Revert x86 sigcontext cleanups").

If only I'm not missin something obvious this should not hurt us. But I gonna build test kernel and check to be sure tomorrow, ok?

#81 lore raw
Quoted 3 lines · Cyrill Gorcunov

If only I'm not missin something obvious this should not hurt us. But I gonna build test kernel and check to be sure tomorrow, ok?

Thanks,

Linus

#78 lore raw
In reply to Andy Lutomirski

13.08.2015 22:49, Andy Lutomirski пишет:

Quoted 5 lines · Stas Sergeev
Quoted 4 lines

13.08.2015 21:41, Andy Lutomirski пишет:

Quoted 2 lines

Stas: I think uc_flags is okay. We don't currently read it during sigreturn, but I see no reason that we can't start reading it.

Quoted 7 lines
Quoted 5 lines

Andy, we definitely have some communication discontinuity here. :) The point is not sigreturn. If we are talking about the flags that will in the future control also TLS, how would you limit it to sigreturn()? It should control the restoring of FS _on signal delivery_, not only on sigreturn()! So how uc_flags can be used for this at all?

Ah, you want it restored on signal delivery. What would it be restored to?

Null descriptor and TLS base in MSR I guess, no?

Quoted 2 lines

ISTM that can be done easily enough in user code, so
maybe we should leave it to user code.

But it is actually not.
gcc relies of fs pointing to TLS on the function prolog, so the asm signal handlers again?
And there are just too many trickery for an asm handler. Should it do the syscall to set fs base via MSR? And to what value? Why do you think the user should mess with all this pain? It is just much easier to do on a kernel side, is it not? And IMHO this is the kernel's responsibility to adhere to the ABI constraints when entering the signal handler, and the ABI says fs should point to TLS.

#82 lore raw
In reply to Linus Torvalds
Quoted 11 lines · Linus Torvalds
Quoted 2 lines · Cyrill Gorcunov

If only I'm not missin something obvious this should not hurt us. But I gonna build test kernel and check to be sure tomorrow, ok?

Thanks,

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

I am curious about what's supposed to happen normally on signal delivery.

Is SS a register that's supposed to be preserved like EIP/RIP and CS when a signal is delivered?

#83 lore raw
Quoted 5 lines · Raymond Jennings

I am curious about what's supposed to happen normally on signal delivery.

Is SS a register that's supposed to be preserved like EIP/RIP and CS when a signal is delivered?

What exactly does "supposed" mean?

On x86-64, we traditionally haven't touched SS, because it doesn't really matter in 64-bit long mode. And apparently dosemu depended on that behavior.

So clearly, we're not "supposed" to save/restore it. Because reality matters a hell of a lot more than any theoretical arguments.

Linus

#84 lore raw
Quoted 6 lines · Linus Torvalds
Quoted 4 lines · Raymond Jennings

I am curious about what's supposed to happen normally on signal delivery.

Is SS a register that's supposed to be preserved like EIP/RIP and CS when a signal is delivered?

What exactly does "supposed" mean?

Basically, when a process/thread receives a signal, what happens to its registers?

Quoted 2 lines

So clearly, we're not "supposed" to save/restore it. Because reality matters a hell of a lot more than any theoretical arguments.

So it still counts as a regression if the kernel pulls the rug out from under someone that was relying on undocumented or buggy behavior?

#91 lore raw
In reply to Linus Torvalds

14.08.2015 00:46, Linus Torvalds пишет:

Quoted 5 lines · Raymond Jennings
Quoted 4 lines

I am curious about what's supposed to happen normally on signal delivery.

Is SS a register that's supposed to be preserved like EIP/RIP and CS when a signal is delivered?

Quoted 8 lines

What exactly does "supposed" mean?

On x86-64, we traditionally haven't touched SS, because it doesn't really matter in 64-bit long mode. And apparently dosemu depended on that behavior.

So clearly, we're not "supposed" to save/restore it. Because reality matters a hell of a lot more than any theoretical arguments.

Unless you introduce some clever flag to explicitly request its restoring. There is another problem as well which is that gcc assumes
FS base to point to TLS at function prolog. Since FS is not restored too, the only suggestion I get is to write a sighandlers in asm... I wonder if someone really should write a sighandler in asm to restore FS base manually with a syscall.
So I think the reality is asking for a new flag. :)

#85 lore raw
In reply to Raymond Jennings

14.08.2015 01:01, Raymond Jennings пишет:

Quoted 14 lines · Linus Torvalds
Quoted 6 lines · Raymond Jennings
Quoted 4 lines

I am curious about what's supposed to happen normally on signal delivery.

Is SS a register that's supposed to be preserved like EIP/RIP and CS when a signal is delivered?

What exactly does "supposed" mean?

Basically, when a process/thread receives a signal, what happens to its registers?

Quoted 2 lines

So clearly, we're not "supposed" to save/restore it. Because reality matters a hell of a lot more than any theoretical arguments.

So it still counts as a regression if the kernel pulls the rug out from under someone that was relying on undocumented or buggy behavior?

You probably want to read the whole thread...
Or start from here:
https://lkml.org/lkml/2015/8/12/800

#92 lore raw
In reply to Stas Sergeev
Quoted 20 lines · Stas Sergeev

14.08.2015 00:46, Linus Torvalds пишет:

Quoted 10 lines
Quoted 6 lines · Raymond Jennings

I am curious about what's supposed to happen normally on signal delivery.

Is SS a register that's supposed to be preserved like EIP/RIP and CS when a
signal is delivered?

Quoted 8 lines

What exactly does "supposed" mean?

On x86-64, we traditionally haven't touched SS, because it doesn't really matter in 64-bit long mode. And apparently dosemu depended on that behavior.

So clearly, we're not "supposed" to save/restore it. Because reality matters a hell of a lot more than any theoretical arguments.

Quoted 7 lines

Unless you introduce some clever flag to explicitly request its restoring. There is another problem as well which is that gcc assumes
FS base to point to TLS at function prolog. Since FS is not restored too, the only suggestion I get is to write a sighandlers in asm... I wonder if someone really should write a sighandler in asm to restore FS base manually with a syscall.
So I think the reality is asking for a new flag. :)

I still don't see how this hypothetical flag would work.

The relevant task state consists of FS and the hidden FS base register. If you build with -fstack-protector, GCC really wants the FS base register to point to the right place. So you can't change it in the middle of a C function (because the prologue and epilogue will fail to match).

Now suppose you set some magic flag and jump (via sigreturn, trampoline, whatever) into DOS code. The DOS code loads 0x7 into FS and then gets #GP. You land in a signal handler. As far as the kernel's concerned, the FS base register is whatever the base of LDT entry 0 is. What else is the kernel supposed to shove in there?

I think that making this work fully in the kernel would require a full-blown FS equivalent of sigaltstack, and that seems like overkill.

Now, if it turns out that Oracle or whoever wants to write a JVM that uses WRFSBASE but still handles signals using standard C signal handlers, they're going to have the same problem. They, too, could fix it in libc, or someone could come up with an in-kernel mechanism that gets enabled at the same time as WRFSBASE and is careful not to break existing code.

I really think that we want to have a single, coherent change whenever WRFSBASE and WRGSBASE get enabled that covers all the bases. None of the patch sets so far have done that. But this cuts both ways: I don't think we should change FS and GS behavior in signal handlers until we are confident that we know what's happening with the FSGSBASE instructions.

--Andy

#93 lore raw

14.08.2015 01:11, Andy Lutomirski пишет:

Quoted 32 lines · Stas Sergeev
Quoted 16 lines

14.08.2015 00:46, Linus Torvalds пишет:

Quoted 7 lines
Quoted 5 lines · Raymond Jennings

I am curious about what's supposed to happen normally on signal delivery.

Is SS a register that's supposed to be preserved like EIP/RIP and CS when a
signal is delivered?

Quoted 8 lines

What exactly does "supposed" mean?

On x86-64, we traditionally haven't touched SS, because it doesn't really matter in 64-bit long mode. And apparently dosemu depended on that behavior.

So clearly, we're not "supposed" to save/restore it. Because reality matters a hell of a lot more than any theoretical arguments.

Quoted 7 lines

Unless you introduce some clever flag to explicitly request its restoring. There is another problem as well which is that gcc assumes
FS base to point to TLS at function prolog. Since FS is not restored too, the only suggestion I get is to write a sighandlers in asm... I wonder if someone really should write a sighandler in asm to restore FS base manually with a syscall.
So I think the reality is asking for a new flag. :)

I still don't see how this hypothetical flag would work.

The relevant task state consists of FS and the hidden FS base register. If you build with -fstack-protector, GCC really wants the FS base register to point to the right place. So you can't change it in the middle of a C function (because the prologue and epilogue will fail to match).

Quoted 5 lines

Now suppose you set some magic flag and jump (via sigreturn, trampoline, whatever) into DOS code. The DOS code loads 0x7 into FS and then gets #GP. You land in a signal handler. As far as the kernel's concerned, the FS base register is whatever the base of LDT entry 0 is. What else is the kernel supposed to shove in there?

The same as what happens when you do in userspace:
---
asm ("mov $0,%%fs\n");
prctl(ARCH_SET_FS, my_tls_base);
---

This was the trick I did before gcc started to use FS in prolog, now I have to do this in asm.
But how simpler for the kernel is to do the same?

Quoted 2 lines

I think that making this work fully in the kernel would require a full-blown FS equivalent of sigaltstack, and that seems like overkill.

Setting selector and base is what you call an "equivalent of sigaltstack"?

#94 lore raw
Quoted 14 lines · Stas Sergeev

14.08.2015 01:11, Andy Lutomirski пишет:

Quoted 5 lines

Now suppose you set some magic flag and jump (via sigreturn, trampoline, whatever) into DOS code. The DOS code loads 0x7 into FS and then gets #GP. You land in a signal handler. As far as the kernel's concerned, the FS base register is whatever the base of LDT entry 0 is. What else is the kernel supposed to shove in there?

The same as what happens when you do in userspace:
---
asm ("mov $0,%%fs\n");
prctl(ARCH_SET_FS, my_tls_base);
---

Quoted 8 lines

This was the trick I did before gcc started to use FS in prolog, now I have to do this in asm.
But how simpler for the kernel is to do the same?

Quoted 2 lines

I think that making this work fully in the kernel would require a full-blown FS equivalent of sigaltstack, and that seems like overkill.

Setting selector and base is what you call an "equivalent of sigaltstack"?

Yes. sigaltstack says "hey, kernel! here's my SP for signal handling." I think we'd need something similar to tell the kernel what my_tls_base is. Using the most recent thing passed to ARCH_SET_FS is no good because WRFSBASE systems might not use ARCH_SET_FS, and we can't break DOSEMU on Ivy Bridge and newer as soon as we enable WRFSBASE.

--Andy

#95 lore raw

14.08.2015 01:29, Andy Lutomirski пишет:

Quoted 20 lines · Stas Sergeev
Quoted 19 lines

14.08.2015 01:11, Andy Lutomirski пишет:

Quoted 5 lines

Now suppose you set some magic flag and jump (via sigreturn, trampoline, whatever) into DOS code. The DOS code loads 0x7 into FS and then gets #GP. You land in a signal handler. As far as the kernel's concerned, the FS base register is whatever the base of LDT entry 0 is. What else is the kernel supposed to shove in there?

The same as what happens when you do in userspace:
---
asm ("mov $0,%%fs\n");
prctl(ARCH_SET_FS, my_tls_base);
---

This was the trick I did before gcc started to use FS in prolog, now I have to do this in asm.
But how simpler for the kernel is to do the same?

Quoted 2 lines

I think that making this work fully in the kernel would require a full-blown FS equivalent of sigaltstack, and that seems like overkill.

Quoted 7 lines
Quoted 1 line

Setting selector and base is what you call an "equivalent of sigaltstack"?

Yes. sigaltstack says "hey, kernel! here's my SP for signal handling." I think we'd need something similar to tell the kernel what my_tls_base is. Using the most recent thing passed to ARCH_SET_FS is no good because WRFSBASE systems might not use ARCH_SET_FS, and we can't break DOSEMU on Ivy Bridge and newer as soon as we enable WRFSBASE.

If someone uses WRFSBASE and wants things to be preserved
in a sighandler, he'll just not set the aforementioned flag. No regression. Whoever wants to use that flag properly, will not use WRFSBASE, and will use ARCH_SET_FS or set_thread_area().
What exactly breakage do you have in mind?

#96 lore raw
Quoted 35 lines · Stas Sergeev

14.08.2015 01:29, Andy Lutomirski пишет:

Quoted 27 lines · Stas Sergeev
Quoted 19 lines

14.08.2015 01:11, Andy Lutomirski пишет:

Quoted 5 lines

Now suppose you set some magic flag and jump (via sigreturn, trampoline, whatever) into DOS code. The DOS code loads 0x7 into FS and then gets #GP. You land in a signal handler. As far as the kernel's concerned, the FS base register is whatever the base of LDT entry 0 is. What else is the kernel supposed to shove in there?

The same as what happens when you do in userspace:
---
asm ("mov $0,%%fs\n");
prctl(ARCH_SET_FS, my_tls_base);
---

This was the trick I did before gcc started to use FS in prolog, now I have to do this in asm.
But how simpler for the kernel is to do the same?

Quoted 5 lines
Quoted 2 lines

I think that making this work fully in the kernel would require a full-blown FS equivalent of sigaltstack, and that seems like overkill.

Setting selector and base is what you call an "equivalent of sigaltstack"?

Quoted 6 lines

Yes. sigaltstack says "hey, kernel! here's my SP for signal handling." I think we'd need something similar to tell the kernel what my_tls_base is. Using the most recent thing passed to ARCH_SET_FS is no good because WRFSBASE systems might not use ARCH_SET_FS, and we can't break DOSEMU on Ivy Bridge and newer as soon as we enable WRFSBASE.

Quoted 5 lines

If someone uses WRFSBASE and wants things to be preserved
in a sighandler, he'll just not set the aforementioned flag. No regression. Whoever wants to use that flag properly, will not use WRFSBASE, and will use ARCH_SET_FS or set_thread_area().
What exactly breakage do you have in mind?

DOSEMU, when you set that flag, WRFSBASE gets enabled, and glibc's threading library starts using WRFSBASE instead of arch_prctl.

--Andy

#86 lore raw
In reply to Raymond Jennings
Quoted 3 lines · Raymond Jennings

So it still counts as a regression if the kernel pulls the rug out from under someone that was relying on undocumented or buggy behavior?

Absolutely. There are no excuses for regressions. If the code was badly written and left itself open to user space "misusing" it, it's our problem. Especially if the code was badly written to begin with, and user space worked around our bad code.

We don't then "fix" code and blame user space for doing bad things.

In particular, we don't cop out and say "hey, you didn't follow the docs" (whether they existed or not) or "you didn't do what we meant you to do".

The _only_ thing that matters is that something broke.

Linus

#97 lore raw
In reply to Andy Lutomirski

14.08.2015 02:00, Andy Lutomirski пишет:

Quoted 30 lines · Stas Sergeev
Quoted 29 lines

14.08.2015 01:29, Andy Lutomirski пишет:

Quoted 28 lines · Stas Sergeev
Quoted 13 lines

14.08.2015 01:11, Andy Lutomirski пишет:

Quoted 5 lines

Now suppose you set some magic flag and jump (via sigreturn, trampoline, whatever) into DOS code. The DOS code loads 0x7 into FS and then gets #GP. You land in a signal handler. As far as the kernel's concerned, the FS base register is whatever the base of LDT entry 0 is. What else is the kernel supposed to shove in there?

The same as what happens when you do in userspace:
---
asm ("mov $0,%%fs\n");
prctl(ARCH_SET_FS, my_tls_base);
---

Quoted 8 lines

This was the trick I did before gcc started to use FS in prolog, now I have to do this in asm.
But how simpler for the kernel is to do the same?

Quoted 2 lines

I think that making this work fully in the kernel would require a full-blown FS equivalent of sigaltstack, and that seems like overkill.

Setting selector and base is what you call an "equivalent of sigaltstack"?

Yes. sigaltstack says "hey, kernel! here's my SP for signal handling." I think we'd need something similar to tell the kernel what my_tls_base is. Using the most recent thing passed to ARCH_SET_FS is no good because WRFSBASE systems might not use ARCH_SET_FS, and we can't break DOSEMU on Ivy Bridge and newer as soon as we enable WRFSBASE.

Quoted 7 lines
Quoted 5 lines

If someone uses WRFSBASE and wants things to be preserved
in a sighandler, he'll just not set the aforementioned flag. No regression. Whoever wants to use that flag properly, will not use WRFSBASE, and will use ARCH_SET_FS or set_thread_area().
What exactly breakage do you have in mind?

DOSEMU, when you set that flag, WRFSBASE gets enabled, and glibc's threading library starts using WRFSBASE instead of arch_prctl.

Whoever wants to use the new flag, will need to call prctl(ARCH_SET_FS, needed_tls) or the like, before altering FS. "needed_tls" will be the current TLS in most cases. This will make kernel to know what TLS should be restored in sighandler. Yes, that's resembles sigaltstack to some degree, but is simple? Nothing special on kernel side, and not a big deal for the user to call prctl().

#87 lore raw
In reply to Linus Torvalds
Quoted 2 lines · Linus Torvalds

The _only_ thing that matters is that something broke.

To clarify: things like test programs etc don't matter. Real applications, used by real users. That's what regressions cover. If you have a workflow that isn't just some random kernel test thing, and you depend on it, and we break it, the kernel is supposed to fix it.

There are some (very few) exceptions.

If it's a security issue, we may not be able to "fix" it, because other concerns can obviously take precedence.

Also, sometimes the reports come in way too late - if you were running some stable distro kernel for several years, and updated, and notice a change that happened four years ago and modern applications now rely on the _new_ behavior, we may not be able to fix the regression any more.

But no, "it was an unintentional kernel bug and clearly just stupid crap code, and we fixed it and now the kernel is much better and cleaner" is not a valid reason for regressions. We'll go back to the stupid and crap code if necessary, however much that may annoy us.

For an example of the kind of things we may have to do, see commits

64f371bc3107 autofs: make the autofsv5 packet file descriptor use
a packetized pipe
9883035ae7ed pipes: add a "packetized pipe" mode for writing

and just wonder at the insanity. That's the kinds of things that happen when one application had actively worked around a bug in compatibility handling, and then trying to "fix" that bug just caused another application to break instead.

Linus

#88 lore raw
Quoted 30 lines · Linus Torvalds
Quoted 1 line · Linus Torvalds

The _only_ thing that matters is that something broke.

To clarify: things like test programs etc don't matter. Real applications, used by real users. That's what regressions cover. If you have a workflow that isn't just some random kernel test thing, and you depend on it, and we break it, the kernel is supposed to fix it.

There are some (very few) exceptions.

If it's a security issue, we may not be able to "fix" it, because other concerns can obviously take precedence.

Also, sometimes the reports come in way too late - if you were running some stable distro kernel for several years, and updated, and notice a change that happened four years ago and modern applications now rely on the _new_ behavior, we may not be able to fix the regression any more.

But no, "it was an unintentional kernel bug and clearly just stupid crap code, and we fixed it and now the kernel is much better and cleaner" is not a valid reason for regressions. We'll go back to the stupid and crap code if necessary, however much that may annoy us.

For an example of the kind of things we may have to do, see commits

64f371bc3107 autofs: make the autofsv5 packet file descriptor use
a packetized pipe
9883035ae7ed pipes: add a "packetized pipe" mode for writing

Quoted 6 lines

and just wonder at the insanity. That's the kinds of things that happen when one application had actively worked around a bug in compatibility handling, and then trying to "fix" that bug just caused another application to break instead.

Linus

Is there a way to temporally confine the bad crap code just to the applications that depend on it, or does a userspace app latching onto bad behavior effectively lock down the abi for the future?

I know that some features in the kernel get deprecated over a process (devfs for example) once userspace is given an alternative...would there be a process like that to deal with userspace code that is pinning a piece of crap in the kernel?

#89 lore raw
In reply to Linus Torvalds

14.08.2015 02:18, Linus Torvalds пишет:

Quoted 19 lines
Quoted 1 line · Linus Torvalds

The _only_ thing that matters is that something broke.

To clarify: things like test programs etc don't matter. Real applications, used by real users. That's what regressions cover. If you have a workflow that isn't just some random kernel test thing, and you depend on it, and we break it, the kernel is supposed to fix it.

There are some (very few) exceptions.

If it's a security issue, we may not be able to "fix" it, because other concerns can obviously take precedence.

Also, sometimes the reports come in way too late - if you were running some stable distro kernel for several years, and updated, and notice a change that happened four years ago and modern applications now rely on the _new_ behavior, we may not be able to fix the regression any more.

Quoted 4 lines

But no, "it was an unintentional kernel bug and clearly just stupid crap code, and we fixed it and now the kernel is much better and cleaner" is not a valid reason for regressions. We'll go back to the stupid and crap code if necessary, however much that may annoy us.

IMHO at least such ocasions should be listed somewhere, and the software authors should be asked to fix their code. Then you'll be able to re-visit the problem later.
It may be unreasonable to carry the compatibility hacks forever if the software that needed it, released the fix 10 years ago.

In fact, in the cases I can remember, the kernel patches were never reverted, see this for instance:
https://lkml.org/lkml/2005/3/26/21
And there were many other breakages too, for example when kernel started to use top-down memory allocations. These were because of the poor code in dosemu, and dosemu was asked to fix the code. I guess the policy to never break userspace was not existing back then. Or there is some margin below which the code quality is considered not worth the troubles. :)

#98 lore raw
In reply to Andy Lutomirski

14.08.2015 02:00, Andy Lutomirski пишет:

Quoted 30 lines · Stas Sergeev
Quoted 29 lines

14.08.2015 01:29, Andy Lutomirski пишет:

Quoted 28 lines · Stas Sergeev
Quoted 13 lines

14.08.2015 01:11, Andy Lutomirski пишет:

Quoted 5 lines

Now suppose you set some magic flag and jump (via sigreturn, trampoline, whatever) into DOS code. The DOS code loads 0x7 into FS and then gets #GP. You land in a signal handler. As far as the kernel's concerned, the FS base register is whatever the base of LDT entry 0 is. What else is the kernel supposed to shove in there?

The same as what happens when you do in userspace:
---
asm ("mov $0,%%fs\n");
prctl(ARCH_SET_FS, my_tls_base);
---

Quoted 8 lines

This was the trick I did before gcc started to use FS in prolog, now I have to do this in asm.
But how simpler for the kernel is to do the same?

Quoted 2 lines

I think that making this work fully in the kernel would require a full-blown FS equivalent of sigaltstack, and that seems like overkill.

Setting selector and base is what you call an "equivalent of sigaltstack"?

Yes. sigaltstack says "hey, kernel! here's my SP for signal handling." I think we'd need something similar to tell the kernel what my_tls_base is. Using the most recent thing passed to ARCH_SET_FS is no good because WRFSBASE systems might not use ARCH_SET_FS, and we can't break DOSEMU on Ivy Bridge and newer as soon as we enable WRFSBASE.

Quoted 7 lines
Quoted 5 lines

If someone uses WRFSBASE and wants things to be preserved
in a sighandler, he'll just not set the aforementioned flag. No regression. Whoever wants to use that flag properly, will not use WRFSBASE, and will use ARCH_SET_FS or set_thread_area().
What exactly breakage do you have in mind?

DOSEMU, when you set that flag, WRFSBASE gets enabled, and glibc's threading library starts using WRFSBASE instead of arch_prctl.

Hmm, how about the following:

prctl(ARCH_SET_SIGNAL_FS, my_tls)
If my_tls==NULL - use current fsbase (including one of WRFSBASE). If my_tls==(void)-1 - don't restore.

Can this work?

#90 lore raw
In reply to Stas Sergeev
Quoted 9 lines · Stas Sergeev

In fact, in the cases I can remember, the kernel patches were never reverted, see this for instance:
https://lkml.org/lkml/2005/3/26/21
And there were many other breakages too, for example when kernel started to use top-down memory allocations. These were because of the poor code in dosemu, and dosemu was asked to fix the code. I guess the policy to never break userspace was not existing back then. Or there is some margin below which the code quality is considered not worth the troubles. :)

Back in 2005 we may well not have been as strict about regressions as we are now, no. The strict policy of no regressions actually originally started mainly wrt suspend/resume issues, where the "fix one machine, break another" kind of back-and-forth caused endless problems, and meant that we didn't actually necessarily make any forward progress, just moving a problem around.

(That said, I'd like to think that we've _always_ tried very hard to not break stuff, it just wasn't necessarily the kind of very explicit and hard rule that it is these days).

Also, there are certainly developers who push back on fixing the regressions they caused. That is in fact the cause of some of my more memorable explosions. But if I'm not brought into the discussion, and the push-back happens on a mailing list, I may not even be aware of the regression and the fact that a developer isn't willing to fix it.

I've also seen other projects be more willing to work around problems like this than I personally would consider to be a good idea. For example, we had some Wine regressions that broke a steam game or two, and it was debugged on the steam and wine lists, and took a longish time to actually even get to the attention of kernel people, because the developers were actually willing to try to do an update and fix their application to not do the thing that caused problems. Which I certainly appreciate, but at the same time that doesn't necessarily help the user who sits there with a game that just didn't work. So even if an app is getting updated eventually, the kernel breakage should be fixed.

So if some patch causes problems, and the author of the patch doesn't acknowledge the problem or even if the application developer says "we can work around that", always feel free to escalate the issue and bring in upper maintainers.

That said, it *does* depend a bit on just how core the area is. Sometimes it just gets too painful to fix things, and while the "no regressions" rule is pretty damn close to the strictest rule we have, there are never any completely absolute rules. As mentioned, there are exceptions to the regression rule too, and sometimes the result might just end up being "very few people are actually affected, and there's a sufficiently good reason for the regression that we'll just cop out".

But that should really be a very rare occurrence. We used to have quite a lot of those kinds of issues with the GPU layer (resulting in flag-days with the X server etc), and it really causes huge problems.

So there are no absolutes. But regressions are a serious problem, and need to be treated as such. I will not _guarantee_ that we always fix them, but I would hope we do our best.

Linus

#99 lore raw
In reply to Stas Sergeev
Quoted 49 lines · Stas Sergeev

14.08.2015 02:00, Andy Lutomirski пишет:

Quoted 44 lines · Stas Sergeev
Quoted 36 lines

14.08.2015 01:29, Andy Lutomirski пишет:

Quoted 27 lines · Stas Sergeev
Quoted 19 lines

14.08.2015 01:11, Andy Lutomirski пишет:

Quoted 5 lines

Now suppose you set some magic flag and jump (via sigreturn, trampoline, whatever) into DOS code. The DOS code loads 0x7 into FS and then gets #GP. You land in a signal handler. As far as the kernel's concerned, the FS base register is whatever the base of LDT entry 0 is. What else is the kernel supposed to shove in there?

The same as what happens when you do in userspace:
---
asm ("mov $0,%%fs\n");
prctl(ARCH_SET_FS, my_tls_base);
---

This was the trick I did before gcc started to use FS in prolog, now I have to do this in asm.
But how simpler for the kernel is to do the same?

Quoted 5 lines
Quoted 2 lines

I think that making this work fully in the kernel would require a full-blown FS equivalent of sigaltstack, and that seems like overkill.

Setting selector and base is what you call an "equivalent of sigaltstack"?

Quoted 6 lines

Yes. sigaltstack says "hey, kernel! here's my SP for signal handling." I think we'd need something similar to tell the kernel what my_tls_base is. Using the most recent thing passed to ARCH_SET_FS is no good because WRFSBASE systems might not use ARCH_SET_FS, and we can't break DOSEMU on Ivy Bridge and newer as soon as we enable WRFSBASE.

Quoted 6 lines

If someone uses WRFSBASE and wants things to be preserved in a sighandler, he'll just not set the aforementioned flag. No regression.
Whoever wants to use that flag properly, will not use WRFSBASE, and will use ARCH_SET_FS or set_thread_area().
What exactly breakage do you have in mind?

Quoted 2 lines

DOSEMU, when you set that flag, WRFSBASE gets enabled, and glibc's threading library starts using WRFSBASE instead of arch_prctl.

Quoted 7 lines

Hmm, how about the following:

prctl(ARCH_SET_SIGNAL_FS, my_tls)
If my_tls==NULL - use current fsbase (including one of WRFSBASE). If my_tls==(void)-1 - don't restore.

Can this work?

Certainly, but why? ISTM user code should do this itself with a little bit of asm unless there's a good reason it wouldn't work. A good reason it wouldn't work is that high-performance applications need this and an extra syscall is too slow, but IMO that would need evidence.

--Andy

Signature · 2 lines
Andy Lutomirski
AMA Capital Management, LLC
#108 lore raw
In reply to Stas Sergeev
Quoted 5 lines · Stas Sergeev

14.08.2015 02:00, Andy Lutomirski пишет:

Quoted 3 lines

DOSEMU, when you set that flag, WRFSBASE gets enabled, and glibc's threading library starts using WRFSBASE instead of arch_prctl.

Quoted 7 lines

Hmm, how about the following:

prctl(ARCH_SET_SIGNAL_FS, my_tls)
If my_tls==NULL - use current fsbase (including one of WRFSBASE). If my_tls==(void)-1 - don't restore.

Can this work?

I'm really inclined to wonder whether we need the change and such a flag at all.

Basically, no _normal_ application will ever play with segments at all on x86-64. So our current behavior of not touching any segments at all for signal handling would seem to be the right thing to do - because it handles all the sane cases optimally.

And applications that *do* play with segments very much know they do so, and we already put the onus on *them* to save/restore segments. That's how dosemu clearly works today.

So why not just keep to that policy? It has worked fairly well so far. Only when we tried to change that policy did we hit these problems, because existing applications obviously already live with what we do (or rather, what we _don't_ do) right now...

Linus

#100 lore raw
In reply to Andy Lutomirski

14.08.2015 03:05, Andy Lutomirski пишет:

Quoted 41 lines · Stas Sergeev
Quoted 40 lines

14.08.2015 02:00, Andy Lutomirski пишет:

Quoted 38 lines · Stas Sergeev
Quoted 29 lines

14.08.2015 01:29, Andy Lutomirski пишет:

Quoted 20 lines · Stas Sergeev
Quoted 19 lines

14.08.2015 01:11, Andy Lutomirski пишет:

Quoted 5 lines

Now suppose you set some magic flag and jump (via sigreturn, trampoline, whatever) into DOS code. The DOS code loads 0x7 into FS and then gets #GP. You land in a signal handler. As far as the kernel's concerned, the FS base register is whatever the base of LDT entry 0 is. What else is the kernel supposed to shove in there?

The same as what happens when you do in userspace:
---
asm ("mov $0,%%fs\n");
prctl(ARCH_SET_FS, my_tls_base);
---

This was the trick I did before gcc started to use FS in prolog, now I have to do this in asm.
But how simpler for the kernel is to do the same?

Quoted 2 lines

I think that making this work fully in the kernel would require a full-blown FS equivalent of sigaltstack, and that seems like overkill.

Quoted 8 lines
Quoted 2 lines

Setting selector and base is what you call an "equivalent of sigaltstack"?

Yes. sigaltstack says "hey, kernel! here's my SP for signal handling." I think we'd need something similar to tell the kernel what my_tls_base is. Using the most recent thing passed to ARCH_SET_FS is no good because WRFSBASE systems might not use ARCH_SET_FS, and we can't break DOSEMU on Ivy Bridge and newer as soon as we enable WRFSBASE.

Quoted 6 lines

If someone uses WRFSBASE and wants things to be preserved in a sighandler, he'll just not set the aforementioned flag. No regression.
Whoever wants to use that flag properly, will not use WRFSBASE, and will use ARCH_SET_FS or set_thread_area().
What exactly breakage do you have in mind?

DOSEMU, when you set that flag, WRFSBASE gets enabled, and glibc's threading library starts using WRFSBASE instead of arch_prctl.

Quoted 8 lines
Quoted 7 lines

Hmm, how about the following:

prctl(ARCH_SET_SIGNAL_FS, my_tls)
If my_tls==NULL - use current fsbase (including one of WRFSBASE). If my_tls==(void)-1 - don't restore.

Can this work?

Certainly, but why?

For example because you can as well do:
prctl(ARCH_SET_SIGNAL_SS, 0)
which will mean "restore ss in sighandler to its current value", and this will fix the regression right here right now, without any lar heuristic, and will keep the correct behaviour of always restoring ss for those who need that.

Quoted 2 lines

ISTM user code should do this itself with a
little bit of asm unless there's a good reason it wouldn't work.

Any example of that asm?

I can even code up the ARCH_SET_SIGNAL_SS patch if you want, it seems absolutely trivial, much simpler than the aforementioned asm, much simpler than the patches you propose.
So my question is more like "why not", rather than "why". Just because it is simple and clean, IMHO.

#109 lore raw
In reply to Linus Torvalds
Quoted 16 lines · Linus Torvalds
Quoted 5 lines · Stas Sergeev

14.08.2015 02:00, Andy Lutomirski пишет:

Quoted 3 lines

DOSEMU, when you set that flag, WRFSBASE gets enabled, and glibc's threading library starts using WRFSBASE instead of arch_prctl.

Quoted 7 lines

Hmm, how about the following:

prctl(ARCH_SET_SIGNAL_FS, my_tls)
If my_tls==NULL - use current fsbase (including one of WRFSBASE). If my_tls==(void)-1 - don't restore.

Can this work?

I'm really inclined to wonder whether we need the change and such a flag at all.

Quoted 8 lines

Basically, no _normal_ application will ever play with segments at all on x86-64. So our current behavior of not touching any segments at all for signal handling would seem to be the right thing to do - because it handles all the sane cases optimally.

And applications that *do* play with segments very much know they do so, and we already put the onus on *them* to save/restore segments. That's how dosemu clearly works today.

I agree for all but CS and SS, which are special. CS is fine already. The way that DOSEMU works around SS it is hideous: it just gives up on sigreturn working and fixes the segments with IRET. (Also, if DOSEMU ever wants to get ESP[31:16] right, it *can't*: only the kernel can usefully do espfix64, and DOSEMU can't get the kernel to return from 64-bit code to 16-bit code, because we zap SS. DOSEMU fudges it by forcibly zeroing ESP[31:16]), but that's not a full solution and I wouldn't be surprised if something breaks as a result.

So yes, it mostly works. It also sucks, and it makes it extremely unpleasant for any other program to do this. I'd argue that keeping things like the sigreturn_64 test working is quite valuable, because it's a royal PITA to exercise this code cleanly without proper control of SS. Unfortunately, making it hard for sigreturn_64 to exercise this stuff doesn't mean that the bad guys can't do it, because they'll use malformed ELF files, or ptrace, or stupid modify_ldt races (except I hopefully fixed those), or x32, or compat, or some other hack I haven't thought of yet, and they'll hit the same bugs. I've lost count of the number of bugs of various severities that the sigreturn tests have shaken out *and* caught in patches that were emailed in.

We could say "forget sigreturn_64, use sigreturn_32 instead", but that can't exercise !IA32_EMULATION kernels, and that code is a bit different.

(BadIRET, for example, is nominally unreachable from 64-bit code without modify_ldt and the sigreturn fix, except that I think know of one really nasty way to do it. I have no intention of implementing that, so keeping selftests working nicely makes me *much* more confident.)

Obviously, if we reintroduce SS restoration, we need to do it much more carefully. My RFC patches are an attempt to do that, but it needs a lot of care to make sure all the bases are covered.

--Andy

#101 lore raw
In reply to Stas Sergeev
Quoted 4 lines · Stas Sergeev

For example because you can as well do:
prctl(ARCH_SET_SIGNAL_SS, 0)
which will mean "restore ss in sighandler to its current value",

I really think a prctl() is the wrong thing to do.

If you want a signal handler to save/restore segments, I think it should be a SA_xyz flag to sigaction() (the way we have SA_RESTART etc). And off by default because of the obvious compatibility issues.

Linus

#110 lore raw
In reply to Andy Lutomirski
Quoted 3 lines · Andy Lutomirski

So yes, it mostly works. It also sucks, and it makes it extremely unpleasant for any other program to do this.

Well, I'd argue that

(a) we don't really _want_ any other programs to do that

(b) but yeah, we might want to make it easier to do cleanly and
explicitly for dosemu (and any other possible programs that do want to do it)

so if this is for just things like dosemu and for test applications etc, let's just introduce something like SA_RESTORE_SS for those to explicitly opt in to "I want this signal handler to save and restore SS".

That's how we have handled these kinds of things in the past (ie SA_SIGINFO extends the stack frame with siginfo, sa_restorer says that user space will use its own sigrestore function, etc etc).

Yeah, the SA_xyzzy flags aren't exactly _common_, but it's how we've handled these kinds of compat issues in the past (SA_ONESHOT, SA_MASK, and SA_NOCLDWAIT are obviously about the traditional BSD/SysV differences, and SA_RESTORER is a Linux internal compatibility thing etc)

Linus

#102 lore raw
In reply to Linus Torvalds

14.08.2015 03:27, Linus Torvalds пишет:

Quoted 8 lines · Stas Sergeev
Quoted 3 lines

For example because you can as well do:
prctl(ARCH_SET_SIGNAL_SS, 0)
which will mean "restore ss in sighandler to its current value",

I really think a prctl() is the wrong thing to do.

If you want a signal handler to save/restore segments, I think it should be a SA_xyz flag to sigaction() (the way we have SA_RESTART

Yes, I was proposing the new sigaction() flag in this thread already too. But at the end, prctl() looks better to me because it allows to pass the TLS value to use when restoring FS. The thing is that I am trying to find the similar treatment for both the SS and FS problems. If you don't think they need a similar treatment, then perhaps the Andy's patch is enough.

Quoted 1 line

etc). And off by default because of the obvious compatibility issues.

Of course.

So, what we have right now (in the latest Andy's patch) is:
1. lar heuristics
2. new uc_flags flag

What it solves: dosemu's regression.

What prctl() can give:
- fix to dosemu's regression
- fix to the TLS problem in the future
- no hack and heuristics

With SA_xyz you can only solve the SS problem, so it is probably not any better than the uc_flags things coded up by Andy.

#103 lore raw
Quoted 30 lines · Stas Sergeev

14.08.2015 03:27, Linus Torvalds пишет:

Quoted 11 lines · Stas Sergeev
Quoted 4 lines

For example because you can as well do:
prctl(ARCH_SET_SIGNAL_SS, 0)
which will mean "restore ss in sighandler to its current value",

I really think a prctl() is the wrong thing to do.

If you want a signal handler to save/restore segments, I think it should be a SA_xyz flag to sigaction() (the way we have SA_RESTART

Yes, I was proposing the new sigaction() flag in this thread already too. But at the end, prctl() looks better to me because it allows to pass the TLS value to use when restoring FS. The thing is that I am trying to find the similar treatment for both the SS and FS problems. If you don't think they need a similar treatment, then perhaps the Andy's patch is enough.

Quoted 1 line

etc). And off by default because of the obvious compatibility issues.

Of course.

So, what we have right now (in the latest Andy's patch) is:
1. lar heuristics
2. new uc_flags flag

What it solves: dosemu's regression.

Quoted 8 lines

What prctl() can give:
- fix to dosemu's regression
- fix to the TLS problem in the future
- no hack and heuristics

With SA_xyz you can only solve the SS problem, so it is probably not any better than the uc_flags things coded up by Andy.

I'm leaning slightly toward LAR heuristic + SA_SAVE_SS. Using a sigaction flag is a bit less weird than using uc_flags. It's also kind of nice that it's more composable -- you can install a SIGUSR1 handler that's just normal code and set SA_SAVE_SS and it'll work, whereas with uc_flags you need to explicitly twiddle uc_flags in the handler.

Unfortunately, I don't think we were clever enough to allow this to be probed easily -- we silently ignore unrecognized sa_flags bits.

--Andy

Signature · 2 lines
Andy Lutomirski
AMA Capital Management, LLC
#104 lore raw

14.08.2015 04:21, Andy Lutomirski пишет:

Quoted 31 lines · Stas Sergeev
Quoted 30 lines

14.08.2015 03:27, Linus Torvalds пишет:

Quoted 8 lines · Stas Sergeev
Quoted 3 lines

For example because you can as well do:
prctl(ARCH_SET_SIGNAL_SS, 0)
which will mean "restore ss in sighandler to its current value",

I really think a prctl() is the wrong thing to do.

If you want a signal handler to save/restore segments, I think it should be a SA_xyz flag to sigaction() (the way we have SA_RESTART

Yes, I was proposing the new sigaction() flag in this thread already too. But at the end, prctl() looks better to me because it allows to pass the TLS value to use when restoring FS. The thing is that I am trying to find the similar treatment for both the SS and FS problems. If you don't think they need a similar treatment, then perhaps the Andy's patch is enough.

Quoted 1 line

etc). And off by default because of the obvious compatibility issues.

Of course.

So, what we have right now (in the latest Andy's patch) is:
1. lar heuristics
2. new uc_flags flag

What it solves: dosemu's regression.

What prctl() can give:
- fix to dosemu's regression
- fix to the TLS problem in the future
- no hack and heuristics

Quoted 4 lines
Quoted 3 lines

With SA_xyz you can only solve the SS problem, so it is probably not any better than the uc_flags things coded up by Andy.

I'm leaning slightly toward LAR heuristic + SA_SAVE_SS.

Stop right here, doesn't the SA_xyz allow to avoid the lar heuristic? Why would you still need the lar heuristic then? Just call it SA_RESTORE_SS instead of SA_SAVE_SS, and the lar heuristic is gone.

Quoted 2 lines

Unfortunately, I don't think we were clever enough to allow this to be probed easily -- we silently ignore unrecognized sa_flags bits.

Big deal, check the kversion. :)

Unforunately, in my eyes SA_xyz doesn't help with FS, so whether it is better than uc_flags or not, is not what I care about.

#105 lore raw
Quoted 43 lines · Stas Sergeev

14.08.2015 04:21, Andy Lutomirski пишет:

Quoted 41 lines · Stas Sergeev
Quoted 39 lines

14.08.2015 03:27, Linus Torvalds пишет:

Quoted 11 lines · Stas Sergeev
Quoted 4 lines

For example because you can as well do:
prctl(ARCH_SET_SIGNAL_SS, 0)
which will mean "restore ss in sighandler to its current value",

I really think a prctl() is the wrong thing to do.

If you want a signal handler to save/restore segments, I think it should be a SA_xyz flag to sigaction() (the way we have SA_RESTART

Yes, I was proposing the new sigaction() flag in this thread already too. But at the end, prctl() looks better to me because it allows to pass the TLS value to use when restoring FS. The thing is that I am trying to find the similar treatment for both the SS and FS problems. If you don't think they need a similar treatment, then perhaps the Andy's patch is enough.

Quoted 1 line

etc). And off by default because of the obvious compatibility issues.

Of course.

So, what we have right now (in the latest Andy's patch) is:
1. lar heuristics
2. new uc_flags flag

What it solves: dosemu's regression.

What prctl() can give:
- fix to dosemu's regression
- fix to the TLS problem in the future
- no hack and heuristics

With SA_xyz you can only solve the SS problem, so it is probably not any better than the uc_flags things coded up by Andy.

Quoted 6 lines
Quoted 1 line

I'm leaning slightly toward LAR heuristic + SA_SAVE_SS.

Stop right here, doesn't the SA_xyz allow to avoid the lar heuristic? Why would you still need the lar heuristic then? Just call it SA_RESTORE_SS instead of SA_SAVE_SS, and the lar heuristic is gone.

The LAR heuristic is about five lines of code, and it makes signal delivery more reliable. Sure, we could gate the "regs->ss = __USER_DS" line on a flag, but why?

Quoted 5 lines
Quoted 3 lines

Unfortunately, I don't think we were clever enough to allow this to be probed easily -- we silently ignore unrecognized sa_flags bits.

Big deal, check the kversion. :)

Not so good. For example, if you made your DOSEMU patch to use the saved SS check the version, then the backported revert would break you.

--Andy

#106 lore raw

14.08.2015 04:37, Andy Lutomirski пишет:

Quoted 37 lines · Stas Sergeev
Quoted 36 lines

14.08.2015 04:21, Andy Lutomirski пишет:

Quoted 34 lines · Stas Sergeev
Quoted 33 lines

14.08.2015 03:27, Linus Torvalds пишет:

Quoted 8 lines · Stas Sergeev
Quoted 3 lines

For example because you can as well do:
prctl(ARCH_SET_SIGNAL_SS, 0)
which will mean "restore ss in sighandler to its current value",

I really think a prctl() is the wrong thing to do.

If you want a signal handler to save/restore segments, I think it should be a SA_xyz flag to sigaction() (the way we have SA_RESTART

Yes, I was proposing the new sigaction() flag in this thread already too. But at the end, prctl() looks better to me because it allows to pass the TLS value to use when restoring FS. The thing is that I am trying to find the similar treatment for both the SS and FS problems. If you don't think they need a similar treatment, then perhaps the Andy's patch is enough.

Quoted 1 line

etc). And off by default because of the obvious compatibility issues.

Of course.

So, what we have right now (in the latest Andy's patch) is:
1. lar heuristics
2. new uc_flags flag

What it solves: dosemu's regression.

What prctl() can give:
- fix to dosemu's regression
- fix to the TLS problem in the future
- no hack and heuristics

With SA_xyz you can only solve the SS problem, so it is probably not any better than the uc_flags things coded up by Andy.

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

I'm leaning slightly toward LAR heuristic + SA_SAVE_SS.

Stop right here, doesn't the SA_xyz allow to avoid the lar heuristic? Why would you still need the lar heuristic then? Just call it SA_RESTORE_SS instead of SA_SAVE_SS, and the lar heuristic is gone.

The LAR heuristic is about five lines of code, and it makes signal delivery more reliable. Sure, we could gate the "regs->ss = __USER_DS" line on a flag, but why?

Speed?
I'll let others decide on that.
My vote is to no heuristic if possible, and keeping the FS problem in mind if possible (obviously, both are possible).

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

Unfortunately, I don't think we were clever enough to allow this to be probed easily -- we silently ignore unrecognized sa_flags bits.

Big deal, check the kversion. :)

Not so good. For example, if you made your DOSEMU patch to use the saved SS check the version, then the backported revert would break you.

I would be scared to imagine the program that probes for everything, adding more and more probes with years, instead of just saying "you need kernel version at least x.y.z". If some functionality is reverted, another kversion check can be added. This all is not new: dosemu's protected mode implementation doesn't work on many kernels selectively. IIRC it didn't work on 3.14 because the 16bit LDTs were disallowed, and other versions had problems too. It is easier to ban such kernels by versions instead of adding a run-time probes, because you don't know beforehand what will break in the next kernel to add a probe in advance, and when you react to an existing breakage, it already doesn't matter. The possibly of reverting a new functionality is similar to getting a breakage in an existing functionality (or even have lower probability), so I see no reason to choose different treatments for these two.
But of course every author has own habits.

#111 lore raw
In reply to Linus Torvalds
Quoted 4 lines · Linus Torvalds
Quoted 3 lines · Cyrill Gorcunov

If only I'm not missin something obvious this should not hurt us. But I gonna build test kernel and check to be sure tomorrow, ok?

Managed to test it. And criu works fine with the revert as expected. Actually it's because of commit c6f2062935c8 Oleg made us a patch:

 | commit 07dcf0dbb6ff97c255bc6b06569255a9479bccdd
 | Author: Oleg Nesterov <[email protected]>
 | Date:   Thu Mar 19 19:14:00 2015 +0300
 |
 |     restore/x86: restore_gpregs() needs to initialize ->ss as well
 |
 |     Before the recent "x86_64,signal: Fix SS handling for signals delivered
 |     to 64-bit programs" kernel patch, sigreturn paths forgot to restore ->ss
 |     after return from the signal handler.
 |
 |     Now that the kernel was fixed, restore_gpregs() has to initialize ->ss
 |     too, it is no longer ignored.
 | ...
 |
 | +++ b/arch/x86/include/asm/restorer.h
 | @@ -53,7 +53,7 @@ struct rt_sigcontext {
 |         unsigned short                  cs;
 |         unsigned short                  gs;
 |         unsigned short                  fs;
 | -       unsigned short                  __pad0;
 | +       unsigned short                  ss;
 |         unsigned long                   err;
 |         unsigned long                   trapno;
 |         unsigned long                   oldmask;

IOW we've been not setting up __pad0 which became ss inside the kernel (in result we've been passing 0 here, which caused the problem).

fwiw, we declare that new criu versions may require new kernels to work but never promised that new criu gonna be compatible with old kernels. That said if something get changed inside sigcontext structure in future we may update criu code as well.

#72 lore raw
In reply to Andy Lutomirski

On Thu, Aug 13, 2015 at 08:43:24AM -0700, Andy Lutomirski wrote:
...

Quoted 14 lines
Quoted 9 lines

That rule hasn't gone anywhere.

Does a plain revert just fix everything? Because if so, that's the right thing to do, and we can just re-visit this later.

I don't understand why Andy and Ingo are even discussing this. What the f*ck, guys?

I'm trying to fix it without reverting. If that doesn't work, then we revert. Yesterday, I thought I had a reasonably clean fix, but it turned out that it only solved half of the problem.

Quoted 5 lines

If we revert, I think I need to check what will break due to the revert. I need to check at least Wine, and we'll have to do something about all the selftests that will start failing. I also need to check CRIU, and IIRC CRIU has started using the new sigcontext SS in new versions.

Yes, we've tuned up our sigcontext structure and put ss into the place where previously __pad0 were. After the revert the kernel simply ignores this field again. But we never did any "weird" gaming over segment registers for testing purposes, neither we note any application (in containers) which does some weird things like dosemu.

Quoted 4 lines

And, damnit, those selftests are *useful*. They've smoked out all kinds of problems. That's part of the reason I'd prefer not to revert if there's a better option.

#112 lore raw
In reply to Cyrill Gorcunov
Quoted 34 lines · Cyrill Gorcunov
Quoted 4 lines · Linus Torvalds
Quoted 3 lines · Cyrill Gorcunov

If only I'm not missin something obvious this should not hurt us. But I gonna build test kernel and check to be sure tomorrow, ok?

Managed to test it. And criu works fine with the revert as expected. Actually it's because of commit c6f2062935c8 Oleg made us a patch:

 | commit 07dcf0dbb6ff97c255bc6b06569255a9479bccdd
 | Author: Oleg Nesterov <[email protected]>
 | Date:   Thu Mar 19 19:14:00 2015 +0300
 |
 |     restore/x86: restore_gpregs() needs to initialize ->ss as well
 |
 |     Before the recent "x86_64,signal: Fix SS handling for signals delivered
 |     to 64-bit programs" kernel patch, sigreturn paths forgot to restore ->ss
 |     after return from the signal handler.
 |
 |     Now that the kernel was fixed, restore_gpregs() has to initialize ->ss
 |     too, it is no longer ignored.
 | ...
 |
 | +++ b/arch/x86/include/asm/restorer.h
 | @@ -53,7 +53,7 @@ struct rt_sigcontext {
 |         unsigned short                  cs;
 |         unsigned short                  gs;
 |         unsigned short                  fs;
 | -       unsigned short                  __pad0;
 | +       unsigned short                  ss;
 |         unsigned long                   err;
 |         unsigned long                   trapno;
 |         unsigned long                   oldmask;
Quoted 7 lines

IOW we've been not setting up __pad0 which became ss inside the kernel (in result we've been passing 0 here, which caused the problem).

fwiw, we declare that new criu versions may require new kernels to work but never promised that new criu gonna be compatible with old kernels.

We did. Kernel 3.11 is still declared as supported (modulo new stuff we added after 1.0 might not work, but basic sigframe mgmt is not one of those "new" things).

Quoted 5 lines

That said if something
get changed inside sigcontext structure in future we may update criu code as well.
.

#113 lore raw

On Fri, Aug 14, 2015 at 01:02:14PM +0300, Pavel Emelyanov wrote:
...

Quoted 11 lines
Quoted 7 lines

IOW we've been not setting up __pad0 which became ss inside the kernel (in result we've been passing 0 here, which caused the problem).

fwiw, we declare that new criu versions may require new kernels to work but never promised that new criu gonna be compatible with old kernels.

We did. Kernel 3.11 is still declared as supported (modulo new stuff we added after 1.0 might not work, but basic sigframe mgmt is not one of those "new" things).

This won't last forever. And I think it's pretty normal to require a new criu version for new kernel.

Cyrill

#107 lore raw
In reply to Andy Lutomirski

14.08.2015 04:37, Andy Lutomirski пишет:

Quoted 37 lines · Stas Sergeev
Quoted 36 lines

14.08.2015 04:21, Andy Lutomirski пишет:

Quoted 34 lines · Stas Sergeev
Quoted 33 lines

14.08.2015 03:27, Linus Torvalds пишет:

Quoted 8 lines · Stas Sergeev
Quoted 3 lines

For example because you can as well do:
prctl(ARCH_SET_SIGNAL_SS, 0)
which will mean "restore ss in sighandler to its current value",

I really think a prctl() is the wrong thing to do.

If you want a signal handler to save/restore segments, I think it should be a SA_xyz flag to sigaction() (the way we have SA_RESTART

Yes, I was proposing the new sigaction() flag in this thread already too. But at the end, prctl() looks better to me because it allows to pass the TLS value to use when restoring FS. The thing is that I am trying to find the similar treatment for both the SS and FS problems. If you don't think they need a similar treatment, then perhaps the Andy's patch is enough.

Quoted 1 line

etc). And off by default because of the obvious compatibility issues.

Of course.

So, what we have right now (in the latest Andy's patch) is:
1. lar heuristics
2. new uc_flags flag

What it solves: dosemu's regression.

What prctl() can give:
- fix to dosemu's regression
- fix to the TLS problem in the future
- no hack and heuristics

With SA_xyz you can only solve the SS problem, so it is probably not any better than the uc_flags things coded up by Andy.

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

I'm leaning slightly toward LAR heuristic + SA_SAVE_SS.

Stop right here, doesn't the SA_xyz allow to avoid the lar heuristic? Why would you still need the lar heuristic then? Just call it SA_RESTORE_SS instead of SA_SAVE_SS, and the lar heuristic is gone.

The LAR heuristic is about five lines of code, and it makes signal delivery more reliable.

Why more reliable? In what case?

Quoted 2 lines

Sure, we could gate the "regs->ss =
__USER_DS" line on a flag, but why?

A few things I can think of why:
- nested signals (usual for dosemu)
- using siglongjmp() to return to dosemu (rather than to DOS code) Both cases look very scare when using SS from just freed LDT entry. How would you even justify and changelog the patch that adds a lar heuristic code that no one uses or wants? Since SA_hyz flag allows you to do without, why not to just keep things safe and simple?

#51 lore raw
In reply to Brian Gerst

13.08.2015 20:00, Brian Gerst пишет:

Quoted 25 lines · Andy Lutomirski
Quoted 19 lines
Quoted 5 lines · Linus Torvalds
Quoted 4 lines · Stas Sergeev

I realize this patch may be good to have in general, but breaking userspace without a single warning is a bit discouraging. Seems like the old "we don't break userspace" rule have gone.

Quoted 8 lines

That rule hasn't gone anywhere.

Does a plain revert just fix everything? Because if so, that's the right thing to do, and we can just re-visit this later.

I don't understand why Andy and Ingo are even discussing this. What the f*ck, guys?

I'm trying to fix it without reverting. If that doesn't work, then we revert. Yesterday, I thought I had a reasonably clean fix, but it turned out that it only solved half of the problem.

Quoted 5 lines

If we revert, I think I need to check what will break due to the revert. I need to check at least Wine, and we'll have to do something about all the selftests that will start failing. I also need to check CRIU, and IIRC CRIU has started using the new sigcontext SS in new versions.

Quoted 4 lines

I don't think Wine will be a problem, at least how it is currently set up. 16-bit support is only in the 32-bit build. The 64-bit build only supports Win64 apps, and will call the 32-bit version (installed in parallel) to run 32 and 16-bit apps.

Is this also because of the lack of the proper 32/16bit support in a 64bit kernels? If so, dosemu's work-arounds do not look like the too bad thing compared to that. :)

#120 lore raw
In reply to Linus Torvalds

13.08.2015 23:07, Linus Torvalds пишет:

Quoted 11 lines · Stas Sergeev
Quoted 4 lines

It doesn't: fedora provides a "sanitized up" version of sigcontext.h in /usr/include/bits, which comes from glibc-headers-2.21-7.fc22.x86_64. So it seems the "sanitized up" headers come from glibc, which means all other distros would have that too.

Yes. Except the whole point of uapi was that in the long term the glibc people should just be able to pick up the kernel ones directly.

And while that may involve some further editing, it sure as hell shouldn't involve "oh, I know, this got renamed, so now I have to rename it back". That would be crazy.

Of course they shouldn't rename back, but the breakage in this case will be seen only to the upstream authors and the distributors who compile the package, but never to the end-user. They'll just quickly release the update without troubling the user. This way kernel could force the uapi changes without too much troubles, and, for example, when it stopped to use .fs/.fs fields for FS/GS, it could actually reasonably do that.

#52 lore raw
In reply to Stas Sergeev
Quoted 33 lines · Stas Sergeev

13.08.2015 20:00, Brian Gerst пишет:

Quoted 31 lines
Quoted 28 lines · Andy Lutomirski
Quoted 8 lines · Linus Torvalds
Quoted 5 lines · Stas Sergeev

I realize this patch may be good to have in general, but breaking userspace without a single warning is a bit discouraging. Seems like the old "we don't break userspace" rule have gone.

Quoted 8 lines

That rule hasn't gone anywhere.

Does a plain revert just fix everything? Because if so, that's the right thing to do, and we can just re-visit this later.

I don't understand why Andy and Ingo are even discussing this. What the f*ck, guys?

I'm trying to fix it without reverting. If that doesn't work, then we revert. Yesterday, I thought I had a reasonably clean fix, but it turned out that it only solved half of the problem.

If we revert, I think I need to check what will break due to the revert. I need to check at least Wine, and we'll have to do something about all the selftests that will start failing. I also need to check CRIU, and IIRC CRIU has started using the new sigcontext SS in new versions.

Quoted 8 lines
Quoted 4 lines

I don't think Wine will be a problem, at least how it is currently set up. 16-bit support is only in the 32-bit build. The 64-bit build only supports Win64 apps, and will call the 32-bit version (installed in parallel) to run 32 and 16-bit apps.

Is this also because of the lack of the proper 32/16bit support in a 64bit kernels? If so, dosemu's work-arounds do not look like the too bad thing compared to that. :)

What do you mean lack of proper 32/16 bit support?

--Andy

#53 lore raw
Quoted 57 lines · Stas Sergeev

14.08.2015 04:37, Andy Lutomirski пишет:

Quoted 47 lines · Stas Sergeev
Quoted 46 lines

14.08.2015 04:21, Andy Lutomirski пишет:

Quoted 42 lines · Stas Sergeev
Quoted 31 lines

14.08.2015 03:27, Linus Torvalds пишет:

Quoted 11 lines · Stas Sergeev
Quoted 4 lines

For example because you can as well do:
prctl(ARCH_SET_SIGNAL_SS, 0)
which will mean "restore ss in sighandler to its current value",

I really think a prctl() is the wrong thing to do.

If you want a signal handler to save/restore segments, I think it should be a SA_xyz flag to sigaction() (the way we have SA_RESTART

Yes, I was proposing the new sigaction() flag in this thread already too. But at the end, prctl() looks better to me because it allows to pass the TLS value to use when restoring FS. The thing is that I am trying to find the similar treatment for both the SS and FS problems. If you don't think they need a similar treatment, then perhaps the Andy's patch is enough.

Quoted 1 line

etc). And off by default because of the obvious compatibility issues.

Of course.

So, what we have right now (in the latest Andy's patch) is:
1. lar heuristics
2. new uc_flags flag

What it solves: dosemu's regression.

Quoted 8 lines

What prctl() can give:
- fix to dosemu's regression
- fix to the TLS problem in the future
- no hack and heuristics

With SA_xyz you can only solve the SS problem, so it is probably not any better than the uc_flags things coded up by Andy.

I'm leaning slightly toward LAR heuristic + SA_SAVE_SS.

Quoted 7 lines
Quoted 4 lines

Stop right here, doesn't the SA_xyz allow to avoid the lar heuristic? Why would you still need the lar heuristic then? Just call it SA_RESTORE_SS instead of SA_SAVE_SS, and the lar heuristic is gone.

The LAR heuristic is about five lines of code, and it makes signal delivery more reliable.

Quoted 7 lines

Why more reliable? In what case?

Quoted 2 lines

Sure, we could gate the "regs->ss =
__USER_DS" line on a flag, but why?

A few things I can think of why:
- nested signals (usual for dosemu)

What's the issue with nested signals?

Quoted 5 lines

- using siglongjmp() to return to dosemu (rather than to DOS code) Both cases look very scare when using SS from just freed LDT entry. How would you even justify and changelog the patch that adds a lar heuristic code that no one uses or wants? Since SA_hyz flag allows you to do without, why not to just keep things safe and simple?

The LAR heuristic is just for compatibility.

ISTM what DOSEMU should want (on new kernels, anyway) is the ability to save and restore SS just like any other register, which is what my patch did. The issue is that it broke old DOSEMU. I want to find a way to keep old DOSEMU working while making things work better for new code that's aware of new behavior. That means we want some way (opt-in or magically compatible with old DOSEMU) to get SS saved and restored.

Incidentally, I tried implementing the sigaction flag approach. I think it's no good. When we return from a signal, there's no concept of sigaction -- it's just sigreturn. Sigreturn can't look up the sigaction flags -- what if the signal handler calls sigaction itself. So we either need a per-task flag, a per-sighand flag, or a sigcontext flag indicating what we should do.

(Yes, I suspect we really might want some way to get FS, GS, and their bases saved and restored, but I still think we should do that separately.)

Signature · 2 lines
Andy Lutomirski
AMA Capital Management, LLC
#54 lore raw

19.08.2015 01:47, Andy Lutomirski пишет:

Quoted 60 lines · Stas Sergeev
Quoted 59 lines

14.08.2015 04:37, Andy Lutomirski пишет:

Quoted 47 lines · Stas Sergeev
Quoted 46 lines

14.08.2015 04:21, Andy Lutomirski пишет:

Quoted 42 lines · Stas Sergeev
Quoted 31 lines

14.08.2015 03:27, Linus Torvalds пишет:

Quoted 11 lines · Stas Sergeev
Quoted 4 lines

For example because you can as well do:
prctl(ARCH_SET_SIGNAL_SS, 0)
which will mean "restore ss in sighandler to its current value",

I really think a prctl() is the wrong thing to do.

If you want a signal handler to save/restore segments, I think it should be a SA_xyz flag to sigaction() (the way we have SA_RESTART

Yes, I was proposing the new sigaction() flag in this thread already too. But at the end, prctl() looks better to me because it allows to pass the TLS value to use when restoring FS. The thing is that I am trying to find the similar treatment for both the SS and FS problems. If you don't think they need a similar treatment, then perhaps the Andy's patch is enough.

Quoted 1 line

etc). And off by default because of the obvious compatibility issues.

Of course.

So, what we have right now (in the latest Andy's patch) is:
1. lar heuristics
2. new uc_flags flag

What it solves: dosemu's regression.

Quoted 8 lines

What prctl() can give:
- fix to dosemu's regression
- fix to the TLS problem in the future
- no hack and heuristics

With SA_xyz you can only solve the SS problem, so it is probably not any better than the uc_flags things coded up by Andy.

I'm leaning slightly toward LAR heuristic + SA_SAVE_SS.

Quoted 7 lines
Quoted 4 lines

Stop right here, doesn't the SA_xyz allow to avoid the lar heuristic? Why would you still need the lar heuristic then? Just call it SA_RESTORE_SS instead of SA_SAVE_SS, and the lar heuristic is gone.

The LAR heuristic is about five lines of code, and it makes signal delivery more reliable.

Why more reliable? In what case?

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Sure, we could gate the "regs->ss =
__USER_DS" line on a flag, but why?

A few things I can think of why:
- nested signals (usual for dosemu)

What's the issue with nested signals?

If nested signal is async and SS is from LDT just freed, then the nested signal will silently change the SS value. So if you are saving it somewhere, you'll save it by luck.
Now this is unlikely to happen, as the async signals in dosemu are all blocked inside any sighandler. So the siglongjmp() case is more expressive: if dosemu jumped via siglongjmp() and as such unblocked the async signals, it will likely want to save its registers before doing a new switch. Now, since SS is invalid and will be therefore changed by a sighandler, what it will save depends on a luck.
Not that dosemu uses siglongjmp() right now, but I am just asking to please not implement the unreliable interfaces _if possible_.

Quoted 16 lines
Quoted 5 lines

- using siglongjmp() to return to dosemu (rather than to DOS code) Both cases look very scare when using SS from just freed LDT entry. How would you even justify and changelog the patch that adds a lar heuristic code that no one uses or wants? Since SA_hyz flag allows you to do without, why not to just keep things safe and simple?

The LAR heuristic is just for compatibility.

ISTM what DOSEMU should want (on new kernels, anyway) is the ability to save and restore SS just like any other register, which is what my patch did. The issue is that it broke old DOSEMU. I want to find a way to keep old DOSEMU working while making things work better for new code that's aware of new behavior. That means we want some way (opt-in or magically compatible with old DOSEMU) to get SS saved and restored.

Quoted 4 lines

Incidentally, I tried implementing the sigaction flag approach. I think it's no good. When we return from a signal, there's no concept of sigaction -- it's just sigreturn. Sigreturn can't look up the sigaction flags -- what if the signal handler calls sigaction itself.

How about the SA_hyz flag that does the following:
- Saves SS into sigcontext
- Forces SS to USER_DS on signal delivery
- Sets the uc_flags flag for sigreturn() to take care of the rest. You'll have both the control on every bit of action, and a simple detection logic: if SA_hyz didn't set the uc flag - it didn't work. You can even employ your lar heuristic here for the case when the aforementioned SA_hyz is not set. But please, please not when it is set! In fact, I wonder if you had in mind exactly that: using the lar heuristic only if the SA_hyz is not set. If so - I misunderstood. Just please don't add it when it is set.

Quoted 6 lines

So we either need a per-task flag, a per-sighand flag, or a sigcontext flag indicating what we should do.

(Yes, I suspect we really might want some way to get FS, GS, and their bases saved and restored, but I still think we should do that separately.)

In fact, I have already convinced myself that SA_hyz can take care of both cases. :) Maybe you'll just need to extend the struct sigaction to pass the TLS address, but this doesn't look absolutely impossible...

#70 lore raw
In reply to Andy Lutomirski

19.08.2015 01:42, Andy Lutomirski пишет:

Quoted 39 lines · Stas Sergeev
Quoted 38 lines

13.08.2015 20:00, Brian Gerst пишет:

Quoted 35 lines
Quoted 23 lines · Andy Lutomirski
Quoted 8 lines · Linus Torvalds
Quoted 5 lines · Stas Sergeev

I realize this patch may be good to have in general, but breaking userspace without a single warning is a bit discouraging. Seems like the old "we don't break userspace" rule have gone.

Quoted 8 lines

That rule hasn't gone anywhere.

Does a plain revert just fix everything? Because if so, that's the right thing to do, and we can just re-visit this later.

I don't understand why Andy and Ingo are even discussing this. What the f*ck, guys?

I'm trying to fix it without reverting. If that doesn't work, then we revert. Yesterday, I thought I had a reasonably clean fix, but it turned out that it only solved half of the problem.

Quoted 5 lines

If we revert, I think I need to check what will break due to the revert. I need to check at least Wine, and we'll have to do something about all the selftests that will start failing. I also need to check CRIU, and IIRC CRIU has started using the new sigcontext SS in new versions.

I don't think Wine will be a problem, at least how it is currently set up. 16-bit support is only in the 32-bit build. The 64-bit build only supports Win64 apps, and will call the 32-bit version (installed in parallel) to run 32 and 16-bit apps.

Quoted 5 lines
Quoted 3 lines

Is this also because of the lack of the proper 32/16bit support in a 64bit kernels? If so, dosemu's work-arounds do not look like the too bad thing compared to that. :)

What do you mean lack of proper 32/16 bit support?

At least the following:

1. vm86().
There was a patch:
http://v86-64.sourceforge.net/
Afaik rejected by Andi Kleen (likely for a good reason - too complex). There is some kvm-based alternative which IIRC was called by dosemu authors as "too slow", and so they started to use a jit-compiler. Wine have started to use dosbox for the DOS progs AFAIK. So both projects have a work-arounds to this limitation with which they are happy, and so it probably not worth the re-visiting.

2. espfix64.
Its there since 3.16, but dosemu have lots of work-arounds in its code. The iret trampoline, for example, uses the carefully aligned stack page, where the high word of ESP is zero.
Another part of the work-around is in a sighandler to decode the instruction to figure out what register caused a fault (corrupted esp value usually goes into ebp first, then to other regs) and zero out the high word of that, plus the high word of esp. There are also other bits of the work-around spread around the dosemu code, and I am surprised it actually even works!

3. SS problem. Was fixed in some versions of 4.1; not fixed any more. ;) dosemu did a glorious iret work-around.

4. FS problem.
Worked around by autoconf checks to ban some gcc options, plus some special care when accessing thread-local vars in a sighandler. While your suggestion is to write an asm handlers, to the date I don't think anyone did that. It is easier to work-around it by other means. Maybe if you show an example of such handler, the things will change, but it is simpler to just wait for a kernel fix IMHO.

This is what I called a 32/16bit support, and in fact, when I installed dosemu on a 64bit machine, started win31 and it just worked, I immediately wrote my regards to Bart Oldeman, so much I was impressed - I thought it is absolutely impossible to make this whole mess working reliably.
I guess wine authors just were not as brave and decided to wait for the kernel functionality in place.

#71 lore raw
Quoted 4 lines · Stas Sergeev

19.08.2015 01:42, Andy Lutomirski пишет:

Quoted 1 line

What do you mean lack of proper 32/16 bit support?

At least the following:

Quoted 10 lines

1. vm86().
There was a patch:
http://v86-64.sourceforge.net/
Afaik rejected by Andi Kleen (likely for a good reason - too complex). There is some kvm-based alternative which IIRC was called by dosemu authors as "too slow", and so they started to use a jit-compiler. Wine have started to use dosbox for the DOS progs AFAIK. So both projects have a work-arounds to this limitation with which they are happy, and so it probably not worth the re-visiting.

Wow!

Yeah, that's quite a hack. EFI mixed mode support used to play similar tricks. Unfortunately, switching in and out of long mode at runtime works very poorly and has terrible interactions with perf, so EFI stopped doing that.

Quoted 10 lines

2. espfix64.
Its there since 3.16, but dosemu have lots of work-arounds in its code. The iret trampoline, for example, uses the carefully aligned stack page, where the high word of ESP is zero.
Another part of the work-around is in a sighandler to decode the instruction to figure out what register caused a fault (corrupted esp value usually goes into ebp first, then to other regs) and zero out the high word of that, plus the high word of esp. There are also other bits of the work-around spread around the dosemu code, and I am surprised it actually even works!

Wow, no kidding! At least this is fixed now.

Quoted 4 lines

3. SS problem. Was fixed in some versions of 4.1; not fixed any more. ;) dosemu did a glorious iret work-around.

Quoted 8 lines

4. FS problem.
Worked around by autoconf checks to ban some gcc options, plus some special care when accessing thread-local vars in a sighandler. While your suggestion is to write an asm handlers, to the date I don't think anyone did that. It is easier to work-around it by other means. Maybe if you show an example of such handler, the things will change, but it is simpler to just wait for a kernel fix IMHO.

Something like:

mov $__NR_arch_prctl, %rax
mov $ARCH_GET_FS, %rdi
mov [wherever you safe fsbase], %rsi
syscall
mov $ARCH_SET_FS, %rdi
mov (wherever you stashed glibc's value), %rsi
syscall
pushq $0
call real_signal_handler
popq %rax
mov $__NR_arch_prctl, %rax
mov $ARCH_SET_FS, %rdi
mov (saved value), %rsi
syscall
ret

Yeah, it's ugly. It may very well be worth changing this once the FSBASE stuff happens.

--Andy

#55 lore raw
In reply to Stas Sergeev
Quoted 68 lines · Stas Sergeev

19.08.2015 01:47, Andy Lutomirski пишет:

Quoted 60 lines · Stas Sergeev
Quoted 59 lines

14.08.2015 04:37, Andy Lutomirski пишет:

Quoted 47 lines · Stas Sergeev
Quoted 46 lines

14.08.2015 04:21, Andy Lutomirski пишет:

Quoted 42 lines · Stas Sergeev
Quoted 31 lines

14.08.2015 03:27, Linus Torvalds пишет:

Quoted 11 lines · Stas Sergeev
Quoted 4 lines

For example because you can as well do:
prctl(ARCH_SET_SIGNAL_SS, 0)
which will mean "restore ss in sighandler to its current value",

I really think a prctl() is the wrong thing to do.

If you want a signal handler to save/restore segments, I think it should be a SA_xyz flag to sigaction() (the way we have SA_RESTART

Yes, I was proposing the new sigaction() flag in this thread already too. But at the end, prctl() looks better to me because it allows to pass the TLS value to use when restoring FS. The thing is that I am trying to find the similar treatment for both the SS and FS problems. If you don't think they need a similar treatment, then perhaps the Andy's patch is enough.

Quoted 1 line

etc). And off by default because of the obvious compatibility issues.

Of course.

So, what we have right now (in the latest Andy's patch) is:
1. lar heuristics
2. new uc_flags flag

What it solves: dosemu's regression.

Quoted 8 lines

What prctl() can give:
- fix to dosemu's regression
- fix to the TLS problem in the future
- no hack and heuristics

With SA_xyz you can only solve the SS problem, so it is probably not any better than the uc_flags things coded up by Andy.

I'm leaning slightly toward LAR heuristic + SA_SAVE_SS.

Quoted 7 lines
Quoted 4 lines

Stop right here, doesn't the SA_xyz allow to avoid the lar heuristic? Why would you still need the lar heuristic then? Just call it SA_RESTORE_SS instead of SA_SAVE_SS, and the lar heuristic is gone.

The LAR heuristic is about five lines of code, and it makes signal delivery more reliable.

Why more reliable? In what case?

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Sure, we could gate the "regs->ss =
__USER_DS" line on a flag, but why?

A few things I can think of why:
- nested signals (usual for dosemu)

What's the issue with nested signals?

Quoted 12 lines

If nested signal is async and SS is from LDT just freed, then the nested signal will silently change the SS value. So if you are saving it somewhere, you'll save it by luck.
Now this is unlikely to happen, as the async signals in dosemu are all blocked inside any sighandler. So the siglongjmp() case is more expressive: if dosemu jumped via siglongjmp() and as such unblocked the async signals, it will likely want to save its registers before doing a new switch. Now, since SS is invalid and will be therefore changed by a sighandler, what it will save depends on a luck.
Not that dosemu uses siglongjmp() right now, but I am just asking to please not implement the unreliable interfaces _if possible_.

Ok, I think I get it.

I'm not proposing the LAR thing as something that anyone should intentionally use going forward. It would only be for compatbility with old code if needed. We certainly should save SS somewhere.

Quoted 4 lines
Quoted 4 lines

Incidentally, I tried implementing the sigaction flag approach. I think it's no good. When we return from a signal, there's no concept of sigaction -- it's just sigreturn. Sigreturn can't look up the sigaction flags -- what if the signal handler calls sigaction itself.

Quoted 11 lines

How about the SA_hyz flag that does the following:
- Saves SS into sigcontext
- Forces SS to USER_DS on signal delivery
- Sets the uc_flags flag for sigreturn() to take care of the rest. You'll have both the control on every bit of action, and a simple detection logic: if SA_hyz didn't set the uc flag - it didn't work. You can even employ your lar heuristic here for the case when the aforementioned SA_hyz is not set. But please, please not when it is set! In fact, I wonder if you had in mind exactly that: using the lar heuristic only if the SA_hyz is not set. If so - I misunderstood. Just please don't add it when it is set.

Hmm, interesting. Maybe that would work for everything. How's this to make it concrete?

Add a sigaction flag SA_RESTORE_SS.

On signal delivery, always save SS into sigcontext->ss. if SA_RESTORE_SS is set, then unconditionally switch HW SS to __USER_DS and set UC_RESTORE_SS. If SA_RESTORE_SS is clear, then leave HW SS alone (i.e. preserve the old behavior).

On signal return, if UC_RESTORE_SS is set, then restore sigcontext->ss. If not, then set SS to __USER_DS (as old kernels did).

This should change nothing at all (except the initial value of sigcontext->ss / __pad0) on old kernels.

Quoted 10 lines
Quoted 7 lines

So we either need a per-task flag, a per-sighand flag, or a sigcontext flag indicating what we should do.

(Yes, I suspect we really might want some way to get FS, GS, and their bases saved and restored, but I still think we should do that separately.)

In fact, I have already convinced myself that SA_hyz can take care of both cases. :) Maybe you'll just need to extend the struct sigaction to pass the TLS address, but this doesn't look absolutely impossible...

I think that should be a separate SA_ flag down the road, if for no other reason than that the SS thing can be done more or less immediately, whereas getting FS and GS right needs to wait.

Also, it occurs to me that FS and GS could become more complicated. There are some proposals to allow tasks to opt in to having a per-cpu GS. If that happens, then figuring out how it would interact with signals could be complicated.

--Andy

#56 lore raw

19.08.2015 18:46, Andy Lutomirski пишет:

Quoted 20 lines · Stas Sergeev
Quoted 4 lines
Quoted 4 lines

Incidentally, I tried implementing the sigaction flag approach. I think it's no good. When we return from a signal, there's no concept of sigaction -- it's just sigreturn. Sigreturn can't look up the sigaction flags -- what if the signal handler calls sigaction itself.

Quoted 11 lines

How about the SA_hyz flag that does the following:
- Saves SS into sigcontext
- Forces SS to USER_DS on signal delivery
- Sets the uc_flags flag for sigreturn() to take care of the rest. You'll have both the control on every bit of action, and a simple detection logic: if SA_hyz didn't set the uc flag - it didn't work. You can even employ your lar heuristic here for the case when the aforementioned SA_hyz is not set. But please, please not when it is set! In fact, I wonder if you had in mind exactly that: using the lar heuristic only if the SA_hyz is not set. If so - I misunderstood. Just please don't add it when it is set.

Hmm, interesting. Maybe that would work for everything. How's this to make it concrete?

Quoted 6 lines

Add a sigaction flag SA_RESTORE_SS.

On signal delivery, always save SS into sigcontext->ss. if SA_RESTORE_SS is set, then unconditionally switch HW SS to __USER_DS and set UC_RESTORE_SS. If SA_RESTORE_SS is clear, then leave HW SS alone (i.e. preserve the old behavior).

Either that, or employ the lar heuristic for the "not set" case (I think its not needed).

Quoted 6 lines

On signal return, if UC_RESTORE_SS is set, then restore sigcontext->ss. If not, then set SS to __USER_DS (as old kernels did).

This should change nothing at all (except the initial value of sigcontext->ss / __pad0) on old kernels.

Agreed.

Quoted 11 lines
Quoted 9 lines
Quoted 6 lines

So we either need a per-task flag, a per-sighand flag, or a sigcontext flag indicating what we should do.

(Yes, I suspect we really might want some way to get FS, GS, and their bases saved and restored, but I still think we should do that separately.)

In fact, I have already convinced myself that SA_hyz can take care of both cases. :) Maybe you'll just need to extend the struct sigaction to pass the TLS address, but this doesn't look absolutely impossible...

I think that should be a separate SA_ flag down the road,

Of course separate.
I only wanted to get a "somewhat similar" solution, so if the TLS case can also be covered by the (different) SA_ flag (plus perhaps a struct sigaction extension), then its just excellent. What I was afraid of is to get an SS-specific fix, like your initial uc_flags+lar_heuristic solution. Such fix can't be extended to the TLS case, reducing the chances for the TLS case to be ever re-visited.

Quoted 4 lines

Also, it occurs to me that FS and GS could become more complicated. There are some proposals to allow tasks to opt in to having a per-cpu GS. If that happens, then figuring out how it would interact with signals could be complicated.

Is this a problem?
At least on 86_64, according to this:
http://wiki.osdev.org/Thread_Local_Storage#x86-64
we only need FS.
In any case, since what you say about GS is optional and should be explicitly requested by the task, it can just disable the use of the new SA_TLS flag, making the sigaction() to return error for example. I think the amount of the supported combinations should be limited by the practical needs, and I don't think someone will need per-cpu GS with sighandler re-setting it to something non-per-cpu.

#39 lore raw
In reply to Andy Lutomirski
Quoted 33 lines · Andy Lutomirski
Quoted 5 lines

The crash happens when DOS program terminates.
At that point dosemu subverts the execution flow by replacing segregs and cs/ip ss/sp in sigcontext with its own. But __pad0 still has DOS SS, which crash because (presumably) the DOS LDT have been just removed.

That's unfortunate.

I don't really know what to do about this. My stupid heuristic for signal delivery seems unlikely to cause problems, but I'm not coming up with a great heuristic for detecting when a program that *modifies* sigcontext hasn't set all the fields. Even adding a flag won't really help here, since DOSEMU won't know to manipulate the flag.

Ingo, here's the situation, assuming I remember the versions right:

v4.0 and before: If we try to deliver a signal while SS is bad, we fail and the process dies. If SS is good but nonstandard, we end up in the signal handler with whatever SS value was loaded when the signal was sent. We do *not* put SS anywhere in the sigcontext, so the only way for a program to figure out what SS was is to look at the HW state before making any syscalls. We also don't even try to restore SS, so SS is unconditionally set to __USER_DS, necessitating nasty workarounds (and breaking all kinds of test cases).

v4.1 and current -linus: We always set SS to __USER_DS when delivering a signal. We save the old SS in the sigcontext and restore it, just like 32-bit signals.

My patch: We leave SS alone when delivering a signal, unless it's invalid, in which case we replace it with __USER_DS. We still save the old SS in the sigcontext and restore it on return.

Quoted 7 lines

Apparently the remaining regression is that DOSEMU doesn't realize that SS is saved so, when it tries to return to full 64-bit mode after a signal that hit in 16-bit mode, it fails because it's invalidated the old SS descriptor in the mean time.

So... what do we do about it? We could revert the whole mess.

Yes, absolutely - we should restore the ABI behavior to where v4.0 and before stood.

Quoted 3 lines

[...] We could tell everyone to fix their DOSEMU, which violates policy and is especially annoying given how much effort we've put into keeping 16-bit mode fully functional lately. [...]

So calling DOSEMU broken annoys me: there's nothing to 'fix' in DOSEMU really.

DOSEMU tried hard to work around a kernel bug. 100% of the blame and the responsibility to keep things working is on the kernel side. No ifs and whens.

Quoted 2 lines

[...] We could add yet more heuristics and teach sigreturn to ignore the saved SS value in sigcontext if the saved CS is 64-bit and the saved SS is unusable.

We could maybe try this - assuming it doesn't break DOSEMU.

Or we could extend the ABI and allow DOSEMU to cleanly use it.

I'm not sure it's worth the complexity, given that the DOSEMU workaround already exists and will likely exist forever, so that it can support older kernels, right?

Thanks,

Ingo

#40 lore raw

22.08.2015 15:38, Ingo Molnar пишет:

Quoted 8 lines
Quoted 2 lines

[...] We could add yet more heuristics and teach sigreturn to ignore the saved SS value in sigcontext if the saved CS is 64-bit and the saved SS is unusable.

We could maybe try this - assuming it doesn't break DOSEMU.

Or we could extend the ABI and allow DOSEMU to cleanly use it.

I'm not sure it's worth the complexity, given that the DOSEMU workaround already exists and will likely exist forever, so that it can support older kernels, right?

That assumes dosemu is the only current and future user of that functionality. That's why I raised the question whether does Wine want to support a 32/16bit code in its 64bit builds too. So far it doesn't have such functionality, likely waiting for the kernel to provide it, rather than going the route of hacks and work-arounds as dosemu did.

Also, the fact that dosemu already have that functionality, doesn't mean it will not use the new API - it actually will. With a few run-time checks at first, but when all distros are upgraded, there is simply no point to support the kernels that are not in any current distro (after a few years of deprecation perhaps).

Overall, I think the existance of a few hacks in dosemu is not the reason for the linux kernel to stop any development in that area. dosemu did a great work of exploring all the pitfalls in that area, so now you know what to fix and how. Please, keep the things going.

#41 lore raw
Quoted 2 lines · Stas Sergeev

Also, the fact that dosemu already have that functionality, doesn't mean it will not use the new API - it actually will.

So if dosemu makes use of the new facility then sure, I'm not against it at all!

Thanks,

Ingo

#57 lore raw
In reply to Stas Sergeev
Quoted 30 lines · Stas Sergeev

19.08.2015 18:46, Andy Lutomirski пишет:

Quoted 20 lines · Stas Sergeev
Quoted 4 lines
Quoted 4 lines

Incidentally, I tried implementing the sigaction flag approach. I think it's no good. When we return from a signal, there's no concept of sigaction -- it's just sigreturn. Sigreturn can't look up the sigaction flags -- what if the signal handler calls sigaction itself.

Quoted 11 lines

How about the SA_hyz flag that does the following:
- Saves SS into sigcontext
- Forces SS to USER_DS on signal delivery
- Sets the uc_flags flag for sigreturn() to take care of the rest. You'll have both the control on every bit of action, and a simple detection logic: if SA_hyz didn't set the uc flag - it didn't work. You can even employ your lar heuristic here for the case when the aforementioned SA_hyz is not set. But please, please not when it is set! In fact, I wonder if you had in mind exactly that: using the lar heuristic only if the SA_hyz is not set. If so - I misunderstood. Just please don't add it when it is set.

Hmm, interesting. Maybe that would work for everything. How's this to make it concrete?

Quoted 6 lines

Add a sigaction flag SA_RESTORE_SS.

On signal delivery, always save SS into sigcontext->ss. if SA_RESTORE_SS is set, then unconditionally switch HW SS to __USER_DS and set UC_RESTORE_SS. If SA_RESTORE_SS is clear, then leave HW SS alone (i.e. preserve the old behavior).

Either that, or employ the lar heuristic for the "not set" case (I think its not needed).

Quoted 8 lines
Quoted 6 lines

On signal return, if UC_RESTORE_SS is set, then restore sigcontext->ss. If not, then set SS to __USER_DS (as old kernels did).

This should change nothing at all (except the initial value of sigcontext->ss / __pad0) on old kernels.

Agreed.

Let me throw out one more possibility, just for completeness:

We don't add any SA_xyz flags. On signal delivery, we use the LAR heuristic. We always fill in sigcontext->ss, and we set a new UC_SIGCONTEXT_SS flag to indicate that we support the new behavior.

On sigreturn, we honor the sigcontext's ss, *unless* CS is 64 bit and SS is invalid. In the latter case, we replace the saved ss with __USER_DS.

This should work for old DOSEMU. It's a bit gross, but it has the nice benefit that everyone (even things that aren't DOSEMU) gain the ability to catch signals thrown from bogus SS contexts, which probably improves debugability. It's also nice to not have the SA flag.

This is a big problematic for my sigreturn_64 test, but I can deal with that. We could optionally have another UC_RESTORE_EXACT_SS flag that you can set that means "no, really, restore the saved SS".

--Andy

#58 lore raw

02.09.2015 08:12, Andy Lutomirski пишет:

Quoted 40 lines · Stas Sergeev
Quoted 29 lines

19.08.2015 18:46, Andy Lutomirski пишет:

Quoted 19 lines · Stas Sergeev
Quoted 4 lines
Quoted 4 lines

Incidentally, I tried implementing the sigaction flag approach. I think it's no good. When we return from a signal, there's no concept of sigaction -- it's just sigreturn. Sigreturn can't look up the sigaction flags -- what if the signal handler calls sigaction itself.

Quoted 11 lines

How about the SA_hyz flag that does the following:
- Saves SS into sigcontext
- Forces SS to USER_DS on signal delivery
- Sets the uc_flags flag for sigreturn() to take care of the rest. You'll have both the control on every bit of action, and a simple detection logic: if SA_hyz didn't set the uc flag - it didn't work. You can even employ your lar heuristic here for the case when the aforementioned SA_hyz is not set. But please, please not when it is set! In fact, I wonder if you had in mind exactly that: using the lar heuristic only if the SA_hyz is not set. If so - I misunderstood. Just please don't add it when it is set.

Hmm, interesting. Maybe that would work for everything. How's this to make it concrete?

Quoted 6 lines

Add a sigaction flag SA_RESTORE_SS.

On signal delivery, always save SS into sigcontext->ss. if SA_RESTORE_SS is set, then unconditionally switch HW SS to __USER_DS and set UC_RESTORE_SS. If SA_RESTORE_SS is clear, then leave HW SS alone (i.e. preserve the old behavior).

Either that, or employ the lar heuristic for the "not set" case (I think its not needed).

Quoted 8 lines
Quoted 6 lines

On signal return, if UC_RESTORE_SS is set, then restore sigcontext->ss. If not, then set SS to __USER_DS (as old kernels did).

This should change nothing at all (except the initial value of sigcontext->ss / __pad0) on old kernels.

Agreed.

Let me throw out one more possibility, just for completeness:

Quoted 7 lines

We don't add any SA_xyz flags. On signal delivery, we use the LAR heuristic. We always fill in sigcontext->ss, and we set a new UC_SIGCONTEXT_SS flag to indicate that we support the new behavior.

On sigreturn, we honor the sigcontext's ss, *unless* CS is 64 bit and SS is invalid. In the latter case, we replace the saved ss with __USER_DS.

But this is not a new proposal, see here:
https://lkml.org/lkml/2015/8/13/436
The very last sentence says exactly the same.
I thought this is in the past. :)

Quoted 4 lines

This should work for old DOSEMU. It's a bit gross, but it has the nice benefit that everyone (even things that aren't DOSEMU) gain the ability to catch signals thrown from bogus SS contexts, which probably improves debugability. It's also nice to not have the SA flag.

Pros:
- No new SA flag
- May improve debugability in some unknown scenario where people do not want to just use the new flag to get their things improved

Cons:
- Does not allow to cleanly use siglongjmp(), as then there is a risk to jump to 64bit code with bad SS
- Async signals can silently "validate" SS behind your back
- No way to extend that solution to later fixing the TLS problem
- Many ugly checks in the code, that are not always even obvious (eg you wanted to try verw instead, and there was a gotcha with NP bit)

Is the new SA flag such a big deal here to even bother?

#59 lore raw
Quoted 53 lines · Stas Sergeev

02.09.2015 08:12, Andy Lutomirski пишет:

Quoted 51 lines · Stas Sergeev
Quoted 35 lines

19.08.2015 18:46, Andy Lutomirski пишет:

Quoted 23 lines · Stas Sergeev
Quoted 6 lines
Quoted 5 lines

Incidentally, I tried implementing the sigaction flag approach. I think it's no good. When we return from a signal, there's no concept of sigaction -- it's just sigreturn. Sigreturn can't look up the sigaction flags -- what if the signal handler calls sigaction itself.

Quoted 11 lines

How about the SA_hyz flag that does the following:
- Saves SS into sigcontext
- Forces SS to USER_DS on signal delivery
- Sets the uc_flags flag for sigreturn() to take care of the rest. You'll have both the control on every bit of action, and a simple detection logic: if SA_hyz didn't set the uc flag - it didn't work. You can even employ your lar heuristic here for the case when the aforementioned SA_hyz is not set. But please, please not when it is set! In fact, I wonder if you had in mind exactly that: using the lar heuristic only if the SA_hyz is not set. If so - I misunderstood. Just please don't add it when it is set.

Hmm, interesting. Maybe that would work for everything. How's this to make it concrete?

Quoted 6 lines

Add a sigaction flag SA_RESTORE_SS.

On signal delivery, always save SS into sigcontext->ss. if SA_RESTORE_SS is set, then unconditionally switch HW SS to __USER_DS and set UC_RESTORE_SS. If SA_RESTORE_SS is clear, then leave HW SS alone (i.e. preserve the old behavior).

Either that, or employ the lar heuristic for the "not set" case (I think its not needed).

Quoted 9 lines
Quoted 6 lines

On signal return, if UC_RESTORE_SS is set, then restore sigcontext->ss. If not, then set SS to __USER_DS (as old kernels did).

This should change nothing at all (except the initial value of sigcontext->ss / __pad0) on old kernels.

Agreed.

Let me throw out one more possibility, just for completeness:

We don't add any SA_xyz flags. On signal delivery, we use the LAR heuristic. We always fill in sigcontext->ss, and we set a new UC_SIGCONTEXT_SS flag to indicate that we support the new behavior.

Quoted 9 lines
Quoted 3 lines

On sigreturn, we honor the sigcontext's ss, *unless* CS is 64 bit and SS is invalid. In the latter case, we replace the saved ss with __USER_DS.

But this is not a new proposal, see here:
https://lkml.org/lkml/2015/8/13/436
The very last sentence says exactly the same.
I thought this is in the past. :)

True, but I still want to make sure I understand the alternatives well, and we never really considered this approach.

Quoted 5 lines
Quoted 4 lines

This should work for old DOSEMU. It's a bit gross, but it has the nice benefit that everyone (even things that aren't DOSEMU) gain the ability to catch signals thrown from bogus SS contexts, which probably improves debugability. It's also nice to not have the SA flag.

Quoted 8 lines

Pros:
- No new SA flag
- May improve debugability in some unknown scenario where people do not want to just use the new flag to get their things improved

Cons:
- Does not allow to cleanly use siglongjmp(), as then there is a risk to jump to 64bit code with bad SS

What's the issue here? I don't understand.

On musl, (sig)longjmp just restores rsp, rbx, rbp, and r12-r15, so it won't be affected. AFAIK all implementations of siglongjmp are likely to call sigprocmask or similar, and that will clobber SS. I'm not aware of an implementation of siglongjmp that uses sigreturn.

Quoted 1 line

- Async signals can silently "validate" SS behind your back

True, and that's unfortunate. But async signals without SA_SAVE_SS set with the other approach have exactly the same problem. At least with the approach it won't happen in 32-bit or 16-bit code, as we'd only do the heuristic SS fix on sigreturn when returning to 64-bit code.

Quoted 1 line

- No way to extend that solution to later fixing the TLS problem

True.

Quoted 3 lines

- Many ugly checks in the code, that are not always even obvious (eg you wanted to try verw instead, and there was a gotcha with NP bit)

Also true. OTOH, there'll be a unit test.

Quoted 2 lines

Is the new SA flag such a big deal here to even bother?

Not really, but given that the new behavior seems clearly better behaved than the old, it would be nice to be able to have the good behavior, or at least most of it, be the default.

--Andy

#60 lore raw
Quoted 104 lines · Andy Lutomirski
Quoted 53 lines · Stas Sergeev

02.09.2015 08:12, Andy Lutomirski пишет:

Quoted 51 lines · Stas Sergeev
Quoted 35 lines

19.08.2015 18:46, Andy Lutomirski пишет:

Quoted 23 lines · Stas Sergeev
Quoted 6 lines
Quoted 5 lines

Incidentally, I tried implementing the sigaction flag approach. I think it's no good. When we return from a signal, there's no concept of sigaction -- it's just sigreturn. Sigreturn can't look up the sigaction flags -- what if the signal handler calls sigaction itself.

Quoted 11 lines

How about the SA_hyz flag that does the following:
- Saves SS into sigcontext
- Forces SS to USER_DS on signal delivery
- Sets the uc_flags flag for sigreturn() to take care of the rest. You'll have both the control on every bit of action, and a simple detection logic: if SA_hyz didn't set the uc flag - it didn't work. You can even employ your lar heuristic here for the case when the aforementioned SA_hyz is not set. But please, please not when it is set! In fact, I wonder if you had in mind exactly that: using the lar heuristic only if the SA_hyz is not set. If so - I misunderstood. Just please don't add it when it is set.

Hmm, interesting. Maybe that would work for everything. How's this to make it concrete?

Quoted 6 lines

Add a sigaction flag SA_RESTORE_SS.

On signal delivery, always save SS into sigcontext->ss. if SA_RESTORE_SS is set, then unconditionally switch HW SS to __USER_DS and set UC_RESTORE_SS. If SA_RESTORE_SS is clear, then leave HW SS alone (i.e. preserve the old behavior).

Either that, or employ the lar heuristic for the "not set" case (I think its not needed).

Quoted 9 lines
Quoted 6 lines

On signal return, if UC_RESTORE_SS is set, then restore sigcontext->ss. If not, then set SS to __USER_DS (as old kernels did).

This should change nothing at all (except the initial value of sigcontext->ss / __pad0) on old kernels.

Agreed.

Let me throw out one more possibility, just for completeness:

We don't add any SA_xyz flags. On signal delivery, we use the LAR heuristic. We always fill in sigcontext->ss, and we set a new UC_SIGCONTEXT_SS flag to indicate that we support the new behavior.

Quoted 9 lines
Quoted 3 lines

On sigreturn, we honor the sigcontext's ss, *unless* CS is 64 bit and SS is invalid. In the latter case, we replace the saved ss with __USER_DS.

But this is not a new proposal, see here:
https://lkml.org/lkml/2015/8/13/436
The very last sentence says exactly the same.
I thought this is in the past. :)

True, but I still want to make sure I understand the alternatives well, and we never really considered this approach.

Quoted 5 lines
Quoted 4 lines

This should work for old DOSEMU. It's a bit gross, but it has the nice benefit that everyone (even things that aren't DOSEMU) gain the ability to catch signals thrown from bogus SS contexts, which probably improves debugability. It's also nice to not have the SA flag.

Quoted 8 lines

Pros:
- No new SA flag
- May improve debugability in some unknown scenario where people do not want to just use the new flag to get their things improved

Cons:
- Does not allow to cleanly use siglongjmp(), as then there is a risk to jump to 64bit code with bad SS

What's the issue here? I don't understand.

On musl, (sig)longjmp just restores rsp, rbx, rbp, and r12-r15, so it won't be affected. AFAIK all implementations of siglongjmp are likely to call sigprocmask or similar, and that will clobber SS. I'm not aware of an implementation of siglongjmp that uses sigreturn.

Quoted 1 line

- Async signals can silently "validate" SS behind your back

True, and that's unfortunate. But async signals without SA_SAVE_SS set with the other approach have exactly the same problem. At least with the approach it won't happen in 32-bit or 16-bit code, as we'd only do the heuristic SS fix on sigreturn when returning to 64-bit code.

Quoted 1 line

- No way to extend that solution to later fixing the TLS problem

True.

Quoted 3 lines

- Many ugly checks in the code, that are not always even obvious (eg you wanted to try verw instead, and there was a gotcha with NP bit)

Quoted 9 lines

Also true. OTOH, there'll be a unit test.

Quoted 2 lines

Is the new SA flag such a big deal here to even bother?

Not really, but given that the new behavior seems clearly better behaved than the old, it would be nice to be able to have the good behavior, or at least most of it, be the default.

In fact, I think we can do even better.

On signal delivery:
- Always set UC_SAVED_SS (for feature detection)
- Always save SS into sigcontext
- Use the LAR heuristic to fix SS
- If the old CS was 64-bit, set UC_STRICT_RESTORE_SS

On sigreturn:
- Ignore UC_SAVED_SS.
- Restore SS.
- If the restored SS is bad, the new CS is 64-bit, and
UC_STRICT_RESTORE_SS is *not* set, then replace SS with __USER_DS.

In other words, we honor the sigcontext SS *unless* the original signal context was not 64-bit (or UC_STRICT_RESTORE_SS was otherwise cleared), the new context is 64-bit, and the new SS is bad.

If I understand correctly, old dosemu will keep working: old dosemu never sigreturns back to anything other than 64-bit mode because it can't. (Unless there's a case where it does flat 32-bit mode using __USER_DS -- is there?)

CRIU should be fine, since modern CRIU already saves and restore SS and since older CRIU presumably doesn't set UC_STRICT_RESTORE_SS.

sigcontext_64 will work unmodified because it'll pick up UC_STRICT_RESTORE_SS automatically as it does its weird sigreturns directly from raise().

New DOSEMU can detect that the old workarounds aren't needed when it sees UC_SAVED_SS and, if it wants stricter behavior, it can manually mask off UC_STRICT_RESTORE_SS.

Old Wine won't be affected because I don't think Wine ever does 32-bit emulation from a 64-bit binary in the first place.

Faults due to bad SS will never be silently fixed up because the silent fixup only happens if the signal handler explicitly changes CS, and only things that are aware of segmentation will do that. (IOW some random SIGALRM handler isn't going to touch sigcontext->cs.)

The new behavior is arguably not even insane. People who reset CS to some 64-bit value probably don't care what's in SS on return since SS has no effect in 64-bit mode except possibly causing IRET to fail. The only thing that seems to care is the DOSEMU IRET trampoline, and that reprograms SS anyway (unless I'm still missing something).

Thoughts?

--Andy

#61 lore raw
In reply to Andy Lutomirski

02.09.2015 17:21, Andy Lutomirski пишет:

Quoted 14 lines
Quoted 5 lines
Quoted 4 lines

This should work for old DOSEMU. It's a bit gross, but it has the nice benefit that everyone (even things that aren't DOSEMU) gain the ability to catch signals thrown from bogus SS contexts, which probably improves debugability. It's also nice to not have the SA flag.

Quoted 8 lines

Pros:
- No new SA flag
- May improve debugability in some unknown scenario where people do not want to just use the new flag to get their things improved

Cons:
- Does not allow to cleanly use siglongjmp(), as then there is a risk to jump to 64bit code with bad SS

Quoted 6 lines

What's the issue here? I don't understand.

On musl, (sig)longjmp just restores rsp, rbx, rbp, and r12-r15, so it won't be affected. AFAIK all implementations of siglongjmp are likely to call sigprocmask or similar, and that will clobber SS. I'm not aware of an implementation of siglongjmp that uses sigreturn.

I am not saying siglongjmp() will be affected.
Quite the opposite: it won't, which is bad. :)
If you have always correct SS, you can use siglongjmp(). If you have broken SS at times, siglongjmp() will be an asking for troubles, as it exactly does not restore SS.
dosemu could do a good use of siglongjmp() to get back to 64bit code from its sighandler.

Quoted 4 lines
Quoted 1 line

- Async signals can silently "validate" SS behind your back

True, and that's unfortunate. But async signals without SA_SAVE_SS set with the other approach have exactly the same problem.

Yes, and as such, they should be blocked.
You could improve on that and on siglongjmp().
And on TLS in the future.

Quoted 5 lines
Quoted 1 line

Is the new SA flag such a big deal here to even bother?

Not really, but given that the new behavior seems clearly better behaved than the old, it would be nice to be able to have the good behavior, or at least most of it, be the default.

Surely, but how about then having the heuristics you suggest, only if the new SA_hyz is not set? And when it is set, have a properly defined and predictable behaviour. Then it seems like we'll get all the possible wishes covered.

#62 lore raw
Quoted 21 lines · Stas Sergeev

02.09.2015 17:21, Andy Lutomirski пишет:

Quoted 14 lines
Quoted 5 lines
Quoted 4 lines

This should work for old DOSEMU. It's a bit gross, but it has the nice benefit that everyone (even things that aren't DOSEMU) gain the ability to catch signals thrown from bogus SS contexts, which probably improves debugability. It's also nice to not have the SA flag.

Quoted 8 lines

Pros:
- No new SA flag
- May improve debugability in some unknown scenario where people do not want to just use the new flag to get their things improved

Cons:
- Does not allow to cleanly use siglongjmp(), as then there is a risk to jump to 64bit code with bad SS

Quoted 6 lines

What's the issue here? I don't understand.

On musl, (sig)longjmp just restores rsp, rbx, rbp, and r12-r15, so it won't be affected. AFAIK all implementations of siglongjmp are likely to call sigprocmask or similar, and that will clobber SS. I'm not aware of an implementation of siglongjmp that uses sigreturn.

Quoted 7 lines

I am not saying siglongjmp() will be affected.
Quite the opposite: it won't, which is bad. :)
If you have always correct SS, you can use siglongjmp(). If you have broken SS at times, siglongjmp() will be an asking for troubles, as it exactly does not restore SS.
dosemu could do a good use of siglongjmp() to get back to 64bit code from its sighandler.

This seems like it would be relying unpleasantly heavily on libc internals.

What *does* work is to raise a signal and stash away the entire signal context. Then raise another signal and restore the old context. Of course, this needs SS support in sigcontext, and may still need to handle DS and ES manually.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

- Async signals can silently "validate" SS behind your back

True, and that's unfortunate. But async signals without SA_SAVE_SS set with the other approach have exactly the same problem.

Yes, and as such, they should be blocked.
You could improve on that and on siglongjmp().
And on TLS in the future.

*I* can't do anything to siglongjmp because that's almost entirely outside the kernel. :-/

Quoted 10 lines
Quoted 6 lines
Quoted 2 lines

Is the new SA flag such a big deal here to even bother?

Not really, but given that the new behavior seems clearly better behaved than the old, it would be nice to be able to have the good behavior, or at least most of it, be the default.

Surely, but how about then having the heuristics you suggest, only if the new SA_hyz is not set? And when it is set, have a properly defined and predictable behaviour. Then it seems like we'll get all the possible wishes covered.

That could work. The result is quite similar to explicitly setting UC_STRICT_RESTORE_SS.

I'll draft up an implementation and we can go from there.

--Andy

Signature · 2 lines
Andy Lutomirski
AMA Capital Management, LLC
#63 lore raw

02.09.2015 21:17, Andy Lutomirski пишет:

Quoted 30 lines · Stas Sergeev
Quoted 21 lines

02.09.2015 17:21, Andy Lutomirski пишет:

Quoted 14 lines
Quoted 5 lines
Quoted 4 lines

This should work for old DOSEMU. It's a bit gross, but it has the nice benefit that everyone (even things that aren't DOSEMU) gain the ability to catch signals thrown from bogus SS contexts, which probably improves debugability. It's also nice to not have the SA flag.

Quoted 8 lines

Pros:
- No new SA flag
- May improve debugability in some unknown scenario where people do not want to just use the new flag to get their things improved

Cons:
- Does not allow to cleanly use siglongjmp(), as then there is a risk to jump to 64bit code with bad SS

Quoted 6 lines

What's the issue here? I don't understand.

On musl, (sig)longjmp just restores rsp, rbx, rbp, and r12-r15, so it won't be affected. AFAIK all implementations of siglongjmp are likely to call sigprocmask or similar, and that will clobber SS. I'm not aware of an implementation of siglongjmp that uses sigreturn.

Quoted 7 lines

I am not saying siglongjmp() will be affected.
Quite the opposite: it won't, which is bad. :)
If you have always correct SS, you can use siglongjmp(). If you have broken SS at times, siglongjmp() will be an asking for troubles, as it exactly does not restore SS.
dosemu could do a good use of siglongjmp() to get back to 64bit code from its sighandler.

Quoted 1 line

This seems like it would be relying unpleasantly heavily on libc internals.

Could you please clarify?
If kernel always passes the right SS to the sighandler, then what's the problem?

Quoted 10 lines
Quoted 7 lines
Quoted 4 lines
Quoted 1 line

- Async signals can silently "validate" SS behind your back

True, and that's unfortunate. But async signals without SA_SAVE_SS set with the other approach have exactly the same problem.

Yes, and as such, they should be blocked.
You could improve on that and on siglongjmp().
And on TLS in the future.

*I* can't do anything to siglongjmp because that's almost entirely outside the kernel. :-/

Except for passing the SS=__USER_DS to the sighandler, for which we discussed the new SA_hyz?

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line

Is the new SA flag such a big deal here to even bother?

Not really, but given that the new behavior seems clearly better behaved than the old, it would be nice to be able to have the good behavior, or at least most of it, be the default.

Quoted 7 lines
Quoted 4 lines

Surely, but how about then having the heuristics you suggest, only if the new SA_hyz is not set? And when it is set, have a properly defined and predictable behaviour. Then it seems like we'll get all the possible wishes covered.

That could work. The result is quite similar to explicitly setting UC_STRICT_RESTORE_SS.

I am much more bothered with delivering the right SS than with restoring it on sigreturn().

#64 lore raw
Quoted 31 lines · Stas Sergeev

02.09.2015 21:17, Andy Lutomirski пишет:

Quoted 30 lines · Stas Sergeev
Quoted 28 lines

02.09.2015 17:21, Andy Lutomirski пишет:

Quoted 14 lines
Quoted 5 lines
Quoted 4 lines

This should work for old DOSEMU. It's a bit gross, but it has the nice benefit that everyone (even things that aren't DOSEMU) gain the ability to catch signals thrown from bogus SS contexts, which probably improves debugability. It's also nice to not have the SA flag.

Quoted 8 lines

Pros:
- No new SA flag
- May improve debugability in some unknown scenario where people do not want to just use the new flag to get their things improved

Cons:
- Does not allow to cleanly use siglongjmp(), as then there is a risk to jump to 64bit code with bad SS

Quoted 6 lines

What's the issue here? I don't understand.

On musl, (sig)longjmp just restores rsp, rbx, rbp, and r12-r15, so it won't be affected. AFAIK all implementations of siglongjmp are likely to call sigprocmask or similar, and that will clobber SS. I'm not aware of an implementation of siglongjmp that uses sigreturn.

I am not saying siglongjmp() will be affected.
Quite the opposite: it won't, which is bad. :)
If you have always correct SS, you can use siglongjmp(). If you have broken SS at times, siglongjmp() will be an asking for troubles, as it exactly does not restore SS.
dosemu could do a good use of siglongjmp() to get back to 64bit code from its sighandler.

Quoted 4 lines
Quoted 1 line

This seems like it would be relying unpleasantly heavily on libc internals.

Could you please clarify?
If kernel always passes the right SS to the sighandler, then what's the problem?

What's the exact siglongjmp usage you have in mind? Signal context isn't normally involved AFAIK.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines

- Async signals can silently "validate" SS behind your back

True, and that's unfortunate. But async signals without SA_SAVE_SS set with the other approach have exactly the same problem.

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

Yes, and as such, they should be blocked.
You could improve on that and on siglongjmp().
And on TLS in the future.

*I* can't do anything to siglongjmp because that's almost entirely outside the kernel. :-/

Except for passing the SS=__USER_DS to the sighandler, for which we discussed the new SA_hyz?

I'm still not understanding what you're looking for. If you siglongjmp out of a signal handler, the hardware SS value is irrelevant, at least on 64-bit binaries, because siglongjmp is just going to replace it.

Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
Quoted 6 lines
Quoted 2 lines

Is the new SA flag such a big deal here to even bother?

Not really, but given that the new behavior seems clearly better behaved than the old, it would be nice to be able to have the good behavior, or at least most of it, be the default.

Surely, but how about then having the heuristics you suggest, only if the new SA_hyz is not set? And when it is set, have a properly defined and predictable behaviour. Then it seems like we'll get all the possible wishes covered.

Quoted 4 lines
Quoted 2 lines

That could work. The result is quite similar to explicitly setting UC_STRICT_RESTORE_SS.

I am much more bothered with delivering the right SS than with restoring it on sigreturn().

For 64-bit delivery, ignoring backwards compatibility, delivering signals with ss = __USER_DS would be the right solution, I think: it's trivial and it works. Because of backwards compatibility, we need to deliver signals with ss preserved when possible unless the program opts out. But I don't see why new programs would care what SS is, since it has no effect during 64-bit code execution unless you read it directly or long jmp/long ret to non-64-bit mode.

--Andy

#65 lore raw

02.09.2015 22:06, Andy Lutomirski пишет:

Quoted 29 lines · Stas Sergeev
Quoted 28 lines

02.09.2015 21:17, Andy Lutomirski пишет:

Quoted 27 lines · Stas Sergeev
Quoted 26 lines

02.09.2015 17:21, Andy Lutomirski пишет:

Quoted 12 lines
Quoted 4 lines
Quoted 4 lines

This should work for old DOSEMU. It's a bit gross, but it has the nice benefit that everyone (even things that aren't DOSEMU) gain the ability to catch signals thrown from bogus SS contexts, which probably improves debugability. It's also nice to not have the SA flag.

Quoted 8 lines

Pros:
- No new SA flag
- May improve debugability in some unknown scenario where people do not want to just use the new flag to get their things improved

Cons:
- Does not allow to cleanly use siglongjmp(), as then there is a risk to jump to 64bit code with bad SS

Quoted 6 lines

What's the issue here? I don't understand.

On musl, (sig)longjmp just restores rsp, rbx, rbp, and r12-r15, so it won't be affected. AFAIK all implementations of siglongjmp are likely to call sigprocmask or similar, and that will clobber SS. I'm not aware of an implementation of siglongjmp that uses sigreturn.

I am not saying siglongjmp() will be affected.
Quite the opposite: it won't, which is bad. :)
If you have always correct SS, you can use siglongjmp(). If you have broken SS at times, siglongjmp() will be an asking for troubles, as it exactly does not restore SS.
dosemu could do a good use of siglongjmp() to get back to 64bit code from its sighandler.

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

This seems like it would be relying unpleasantly heavily on libc internals.

Could you please clarify?
If kernel always passes the right SS to the sighandler, then what's the problem?

What's the exact siglongjmp usage you have in mind? Signal context isn't normally involved AFAIK.

dosemu needs 2 return pathes:
1. to DOS code
2. to 64bit code (dosemu is not all in a sighandler, right?)

How it is currently achieved:
dosemu1:
1. sigreturn() + iret (to DOS)
2. modify sigcontext -> sigreturn() (to 64bit asm helper)

dosemu2:
1. sigreturn() + iret (to DOS)
2. modify sigcontext -> sigreturn() -> longjmp() (to 64bit C-coded)

How dosemu2 is supposed to do this:
1. sigreturn() (to DOS)
2. siglongjmp() (to 64bit C-coded)

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
Quoted 1 line

- Async signals can silently "validate" SS behind your back

True, and that's unfortunate. But async signals without SA_SAVE_SS set with the other approach have exactly the same problem.

Yes, and as such, they should be blocked.
You could improve on that and on siglongjmp().
And on TLS in the future.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

*I* can't do anything to siglongjmp because that's almost entirely outside the kernel. :-/

Except for passing the SS=__USER_DS to the sighandler, for which we discussed the new SA_hyz?

I'm still not understanding what you're looking for. If you siglongjmp out of a signal handler, the hardware SS value is irrelevant, at least on 64-bit binaries, because siglongjmp is just going to replace it.

Hmm? IIRC you've just said this:
---
On musl, (sig)longjmp just restores rsp, rbx, rbp, and r12-r15, so it won't be affected.
---
So why would siglongjmp() replace it?

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Quoted 1 line

Is the new SA flag such a big deal here to even bother?

Not really, but given that the new behavior seems clearly better behaved than the old, it would be nice to be able to have the good behavior, or at least most of it, be the default.

Surely, but how about then having the heuristics you suggest, only if the new SA_hyz is not set? And when it is set, have a properly defined and predictable behaviour. Then it seems like we'll get all the possible wishes covered.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

That could work. The result is quite similar to explicitly setting UC_STRICT_RESTORE_SS.

I am much more bothered with delivering the right SS than with restoring it on sigreturn().

For 64-bit delivery, ignoring backwards compatibility, delivering signals with ss = __USER_DS would be the right solution, I think: it's trivial and it works. Because of backwards compatibility, we need to

... add the SA_hyz flag.
I don't understand why do you constantly ignore that part as if it was never spelled. Lets discuss the proposal as a whole, rather than with the random bits thrown away. The flag is exactly for backward compatibility, so why do you present it as a problem without the context of the new flag?

#66 lore raw
Quoted 54 lines · Stas Sergeev

02.09.2015 22:06, Andy Lutomirski пишет:

Quoted 41 lines · Stas Sergeev
Quoted 40 lines

02.09.2015 21:17, Andy Lutomirski пишет:

Quoted 37 lines · Stas Sergeev
Quoted 25 lines

02.09.2015 17:21, Andy Lutomirski пишет:

Quoted 16 lines
Quoted 7 lines
Quoted 6 lines

This should work for old DOSEMU. It's a bit gross, but it has the nice benefit that everyone (even things that aren't DOSEMU) gain the ability to catch signals thrown from bogus SS contexts, which probably
improves debugability. It's also nice to not have the SA flag.

Quoted 8 lines

Pros:
- No new SA flag
- May improve debugability in some unknown scenario where people do not want to just use the new flag to get their things improved

Cons:
- Does not allow to cleanly use siglongjmp(), as then there is a risk to jump to 64bit code with bad SS

Quoted 6 lines

What's the issue here? I don't understand.

On musl, (sig)longjmp just restores rsp, rbx, rbp, and r12-r15, so it won't be affected. AFAIK all implementations of siglongjmp are likely to call sigprocmask or similar, and that will clobber SS. I'm not aware of an implementation of siglongjmp that uses sigreturn.

Quoted 7 lines

I am not saying siglongjmp() will be affected.
Quite the opposite: it won't, which is bad. :)
If you have always correct SS, you can use siglongjmp(). If you have broken SS at times, siglongjmp() will be an asking for troubles, as it exactly does not restore SS.
dosemu could do a good use of siglongjmp() to get back to 64bit code from its sighandler.

This seems like it would be relying unpleasantly heavily on libc internals.

Quoted 6 lines
Quoted 3 lines

Could you please clarify?
If kernel always passes the right SS to the sighandler, then what's the problem?

What's the exact siglongjmp usage you have in mind? Signal context isn't normally involved AFAIK.

dosemu needs 2 return pathes:
1. to DOS code
2. to 64bit code (dosemu is not all in a sighandler, right?)

Quoted 8 lines

How it is currently achieved:
dosemu1:
1. sigreturn() + iret (to DOS)
2. modify sigcontext -> sigreturn() (to 64bit asm helper)

dosemu2:
1. sigreturn() + iret (to DOS)
2. modify sigcontext -> sigreturn() -> longjmp() (to 64bit C-coded)

So you're modifying sigcontext such that it returns to a C function that calls longjmp?

Quoted 4 lines

How dosemu2 is supposed to do this:
1. sigreturn() (to DOS)
2. siglongjmp() (to 64bit C-coded)

This should work fine on any kernel, right? The main problem will be that you presumably need to remember the old context so you can go back to DOS, I assume. So SS needs to be there somewhere.

Quoted 21 lines
Quoted 13 lines
Quoted 13 lines
Quoted 12 lines
Quoted 9 lines
Quoted 5 lines
Quoted 2 lines

- Async signals can silently "validate" SS behind your back

True, and that's unfortunate. But async signals without SA_SAVE_SS set with the other approach have exactly the same problem.

Yes, and as such, they should be blocked.
You could improve on that and on siglongjmp().
And on TLS in the future.

*I* can't do anything to siglongjmp because that's almost entirely outside the kernel. :-/

Quoted 7 lines
Quoted 2 lines

Except for passing the SS=__USER_DS to the sighandler, for which we discussed the new SA_hyz?

I'm still not understanding what you're looking for. If you siglongjmp out of a signal handler, the hardware SS value is irrelevant, at least on 64-bit binaries, because siglongjmp is just going to replace it.

Quoted 6 lines

Hmm? IIRC you've just said this:
---
On musl, (sig)longjmp just restores rsp, rbx, rbp, and r12-r15, so it won't be affected.
---
So why would siglongjmp() replace it?

Because siglongjmp calls sigprocmask, which uses SYSCALL, which clobbers SS.

Quoted 22 lines
Quoted 15 lines
Quoted 15 lines
Quoted 14 lines
Quoted 11 lines
Quoted 6 lines
Quoted 2 lines

Is the new SA flag such a big deal here to even bother?

Not really, but given that the new behavior seems clearly better behaved than the old, it would be nice to be able to have the good behavior, or at least most of it, be the default.

Surely, but how about then having the heuristics you suggest, only if the new SA_hyz is not set? And when it is set, have a properly defined and predictable behaviour. Then it seems like we'll get all the possible wishes covered.

That could work. The result is quite similar to explicitly setting UC_STRICT_RESTORE_SS.

Quoted 6 lines
Quoted 2 lines

I am much more bothered with delivering the right SS than with restoring it on sigreturn().

For 64-bit delivery, ignoring backwards compatibility, delivering signals with ss = __USER_DS would be the right solution, I think: it's trivial and it works. Because of backwards compatibility, we need to

Quoted 6 lines

... add the SA_hyz flag.
I don't understand why do you constantly ignore that part as if it was never spelled. Lets discuss the proposal as a whole, rather than with the random bits thrown away. The flag is exactly for backward compatibility, so why do you present it as a problem without the context of the new flag?

For backwards compat, we either need the default behavior to be unchanged, or we need the default behavior to be something that works with existing dosemu. For existing dosemu, the only interesting cases (I think) are signal delivery from *valid* 16-bit context, in which case we need to preserve SS so that the signal handler can read it out with mov ..., %ss, and sigreturn to 64-bit mode for the IRET trampoline. For sigreturn, IIUC old dosemu will replace the saved CS with a 64-bit code segment selector and won't touch the saved SS because it doesn't know about the saved SS. Those dosemu versions don't care what SS actually contains after sigreturn, because they're immediately going to change it again using IRET. So we just need to make sure we return without faulting.

New dosemu2 would like to sigreturn directly back to 16-bit mode, so it needs the kernel to honor the saved ss value and restore it, possibly changed by dosemu.

We obviously can't require old dosemu to set an SA flag to keep working. But, if we can get away with it, I think it's somewhat preferable not to require new DOSEMU to set an SA flag either.

This has one major benefit at least: if new dosemu loads some random library that installs some async signal handler (SIGALRM for example), everything will work with regard to CS and SS. If SIGALRM hits 16-bit code, CS and SS get saved, the signal handler gets invoked in 64-bit mode, and sigreturn restores the old state.

Of course, FS and GS still screw this up.

--Andy

#67 lore raw

03.09.2015 00:39, Andy Lutomirski пишет:

Quoted 49 lines · Stas Sergeev
Quoted 48 lines

02.09.2015 22:06, Andy Lutomirski пишет:

Quoted 30 lines · Stas Sergeev
Quoted 29 lines

02.09.2015 21:17, Andy Lutomirski пишет:

Quoted 28 lines · Stas Sergeev
Quoted 27 lines

02.09.2015 17:21, Andy Lutomirski пишет:

Quoted 13 lines
Quoted 5 lines
Quoted 5 lines

This should work for old DOSEMU. It's a bit gross, but it has the nice benefit that everyone (even things that aren't DOSEMU) gain the ability to catch signals thrown from bogus SS contexts, which probably
improves debugability. It's also nice to not have the SA flag.

Quoted 8 lines

Pros:
- No new SA flag
- May improve debugability in some unknown scenario where people do not want to just use the new flag to get their things improved

Cons:
- Does not allow to cleanly use siglongjmp(), as then there is a risk to jump to 64bit code with bad SS

Quoted 6 lines

What's the issue here? I don't understand.

On musl, (sig)longjmp just restores rsp, rbx, rbp, and r12-r15, so it won't be affected. AFAIK all implementations of siglongjmp are likely to call sigprocmask or similar, and that will clobber SS. I'm not aware of an implementation of siglongjmp that uses sigreturn.

I am not saying siglongjmp() will be affected.
Quite the opposite: it won't, which is bad. :)
If you have always correct SS, you can use siglongjmp(). If you have broken SS at times, siglongjmp() will be an asking for troubles, as it exactly does not restore SS.
dosemu could do a good use of siglongjmp() to get back to 64bit code from its sighandler.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

This seems like it would be relying unpleasantly heavily on libc internals.

Could you please clarify?
If kernel always passes the right SS to the sighandler, then what's the problem?

What's the exact siglongjmp usage you have in mind? Signal context isn't normally involved AFAIK.

dosemu needs 2 return pathes:
1. to DOS code
2. to 64bit code (dosemu is not all in a sighandler, right?)

How it is currently achieved:
dosemu1:
1. sigreturn() + iret (to DOS)
2. modify sigcontext -> sigreturn() (to 64bit asm helper)

Quoted 5 lines
Quoted 3 lines

dosemu2:
1. sigreturn() + iret (to DOS)
2. modify sigcontext -> sigreturn() -> longjmp() (to 64bit C-coded)

So you're modifying sigcontext such that it returns to a C function that calls longjmp?

Yes.

Quoted 4 lines
Quoted 3 lines

How dosemu2 is supposed to do this:
1. sigreturn() (to DOS)
2. siglongjmp() (to 64bit C-coded)

This should work fine on any kernel, right?

1 - not.
2 - maybe.
If, as you say, siglongjmp() restores SS, I need to try it out. (there is also a problem that most siglongjmp() implementations are incompatible with sigaltstack(), but this is not what you can fix).

Quoted 3 lines

The main problem will be
that you presumably need to remember the old context so you can go back to DOS, I assume. So SS needs to be there somewhere.

Its fine if you always save SS to sigcontext. This is what you proposed already and I think its fine. dosemu saves entire sigcontext before going out to 64bit.

Quoted 38 lines
Quoted 15 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 4 lines
Quoted 1 line

Is the new SA flag such a big deal here to even bother?

Not really, but given that the new behavior seems clearly better behaved than the old, it would be nice to be able to have the good behavior, or at least most of it, be the default.

Surely, but how about then having the heuristics you suggest, only if the new SA_hyz is not set? And when it is set, have a properly defined and predictable behaviour. Then it seems like we'll get all the possible wishes covered.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

That could work. The result is quite similar to explicitly setting UC_STRICT_RESTORE_SS.

I am much more bothered with delivering the right SS than with restoring it on sigreturn().

For 64-bit delivery, ignoring backwards compatibility, delivering signals with ss = __USER_DS would be the right solution, I think: it's trivial and it works. Because of backwards compatibility, we need to

Quoted 6 lines

... add the SA_hyz flag.
I don't understand why do you constantly ignore that part as if it was never spelled. Lets discuss the proposal as a whole, rather than with the random bits thrown away. The flag is exactly for backward compatibility, so why do you present it as a problem without the context of the new flag?

For backwards compat, we either need the default behavior to be unchanged, or we need the default behavior to be something that works with existing dosemu. For existing dosemu, the only interesting cases (I think) are signal delivery from *valid* 16-bit context, in which case we need to preserve SS so that the signal handler can read it out with mov ..., %ss, and sigreturn to 64-bit mode for the IRET trampoline. For sigreturn, IIUC old dosemu will replace the saved CS with a 64-bit code segment selector and won't touch the saved SS because it doesn't know about the saved SS. Those dosemu versions don't care what SS actually contains after sigreturn, because they're immediately going to change it again using IRET. So we just need to make sure we return without faulting.

New dosemu2 would like to sigreturn directly back to 16-bit mode, so it needs the kernel to honor the saved ss value and restore it, possibly changed by dosemu.

Quoted 7 lines

We obviously can't require old dosemu to set an SA flag to keep working. But, if we can get away with it, I think it's somewhat preferable not to require new DOSEMU to set an SA flag either.

This has one major benefit at least: if new dosemu loads some random library that installs some async signal handler (SIGALRM for example), everything will work with regard to CS and SS.

This case is covered if we do both things together: use your heuristic when SA_hyz is not set, and don't use it when its set. In this case dosemu2 will be able to request the proper SS delivery for its sighandlers, but the 3rd-party sighandlers will work too.
I think we have never discussed the possibility of doing both things together, even though I have proposed it many times.
After discussing this full-blown solution, we can think about reducing it, either by removing the heuristic or by removing SA_hyz, but discussing the full one would be nice too. Your opinion is likely that no one will use this SA_hyz in presence of the heuristic that "seems to work anyway". But in the light of extending it for TLS (with a new flag), I wouldn't be so sure. You can also document it as a needed flag when user code touches SS, and then it will be used. dosemu1 code that doesn't use it, will eventually be forgotten. So IMHO whether it will be used, is fully up to how will you market it. :)

#68 lore raw
Quoted 80 lines · Stas Sergeev

03.09.2015 00:39, Andy Lutomirski пишет:

Quoted 65 lines · Stas Sergeev
Quoted 64 lines

02.09.2015 22:06, Andy Lutomirski пишет:

Quoted 45 lines · Stas Sergeev
Quoted 44 lines

02.09.2015 21:17, Andy Lutomirski пишет:

Quoted 41 lines · Stas Sergeev
Quoted 29 lines

02.09.2015 17:21, Andy Lutomirski пишет:

Quoted 18 lines
Quoted 13 lines
Quoted 7 lines

This should work for old DOSEMU. It's a bit gross, but it has the nice benefit that everyone (even things that aren't DOSEMU) gain the
ability to catch signals thrown from bogus SS contexts, which probably
improves debugability. It's also nice to not have the SA flag.

Pros:
- No new SA flag
- May improve debugability in some unknown scenario where people do not want to just use the new flag to get their things improved

Quoted 4 lines

Cons:
- Does not allow to cleanly use siglongjmp(), as then there is a risk
to jump to 64bit code with bad SS

Quoted 8 lines

What's the issue here? I don't understand.

On musl, (sig)longjmp just restores rsp, rbx, rbp, and r12-r15, so it
won't be affected. AFAIK all implementations of siglongjmp are likely
to call sigprocmask or similar, and that will clobber SS. I'm not aware of an implementation of siglongjmp that uses sigreturn.

Quoted 7 lines

I am not saying siglongjmp() will be affected.
Quite the opposite: it won't, which is bad. :)
If you have always correct SS, you can use siglongjmp(). If you have broken SS at times, siglongjmp() will be an asking for troubles, as it exactly does not restore SS.
dosemu could do a good use of siglongjmp() to get back to 64bit code from its sighandler.

This seems like it would be relying unpleasantly heavily on libc internals.

Quoted 6 lines
Quoted 3 lines

Could you please clarify?
If kernel always passes the right SS to the sighandler, then what's the problem?

What's the exact siglongjmp usage you have in mind? Signal context isn't normally involved AFAIK.

dosemu needs 2 return pathes:
1. to DOS code
2. to 64bit code (dosemu is not all in a sighandler, right?)

How it is currently achieved:
dosemu1:
1. sigreturn() + iret (to DOS)
2. modify sigcontext -> sigreturn() (to 64bit asm helper)

Quoted 6 lines
Quoted 3 lines

dosemu2:
1. sigreturn() + iret (to DOS)
2. modify sigcontext -> sigreturn() -> longjmp() (to 64bit C-coded)

So you're modifying sigcontext such that it returns to a C function that calls longjmp?

Yes.

Quoted 4 lines
Quoted 3 lines

How dosemu2 is supposed to do this:
1. sigreturn() (to DOS)
2. siglongjmp() (to 64bit C-coded)

Quoted 8 lines
Quoted 1 line

This should work fine on any kernel, right?

1 - not.
2 - maybe.
If, as you say, siglongjmp() restores SS, I need to try it out. (there is also a problem that most siglongjmp() implementations are incompatible with sigaltstack(), but this is not what you can fix).

1 - definitely needs kernel changes. I was referring to #2.

2 - siglongjmp probably varies in its behavior across different libc implementations. My point is that siglongjmp isn't a kernel-provided thing.

Quoted 36 lines
Quoted 17 lines

For backwards compat, we either need the default behavior to be unchanged, or we need the default behavior to be something that works with existing dosemu. For existing dosemu, the only interesting cases (I think) are signal delivery from *valid* 16-bit context, in which case we need to preserve SS so that the signal handler can read it out with mov ..., %ss, and sigreturn to 64-bit mode for the IRET trampoline. For sigreturn, IIUC old dosemu will replace the saved CS with a 64-bit code segment selector and won't touch the saved SS because it doesn't know about the saved SS. Those dosemu versions don't care what SS actually contains after sigreturn, because they're immediately going to change it again using IRET. So we just need to make sure we return without faulting.

New dosemu2 would like to sigreturn directly back to 16-bit mode, so it needs the kernel to honor the saved ss value and restore it, possibly changed by dosemu.

Quoted 7 lines

We obviously can't require old dosemu to set an SA flag to keep working. But, if we can get away with it, I think it's somewhat preferable not to require new DOSEMU to set an SA flag either.

This has one major benefit at least: if new dosemu loads some random library that installs some async signal handler (SIGALRM for example), everything will work with regard to CS and SS.

This case is covered if we do both things together: use your heuristic when SA_hyz is not set, and don't use it when its set. In this case dosemu2 will be able to request the proper SS delivery for its sighandlers, but the 3rd-party sighandlers will work too.
I think we have never discussed the possibility of doing both things together, even though I have proposed it many times.
After discussing this full-blown solution, we can think about reducing it, either by removing the heuristic or by removing SA_hyz, but discussing the full one would be nice too.

Quoted 8 lines

Your opinion is likely that no one will use this SA_hyz in presence of the heuristic that "seems to work anyway". But in the light of extending it for TLS (with a new flag), I wouldn't be so sure. You can also document it as a needed flag when user code touches SS, and then it will be used. dosemu1 code that doesn't use it, will eventually be forgotten. So IMHO whether it will be used, is fully up to how will you market it. :)

I'll think about it. I'll think about FS and GS, too, although that's still a longer-term thing.

--Andy

Signature · 2 lines
Andy Lutomirski
AMA Capital Management, LLC
#69 lore raw

03.09.2015 01:25, Andy Lutomirski пишет:

Quoted 12 lines · Stas Sergeev
Quoted 11 lines
Quoted 5 lines
Quoted 4 lines

How dosemu2 is supposed to do this:
1. sigreturn() (to DOS)
2. siglongjmp() (to 64bit C-coded)

This should work fine on any kernel, right?

1 - not.
2 - maybe.
If, as you say, siglongjmp() restores SS, I need to try it out. (there is also a problem that most siglongjmp() implementations are incompatible with sigaltstack(), but this is not what you can fix).

Quoted 5 lines

1 - definitely needs kernel changes. I was referring to #2.

2 - siglongjmp probably varies in its behavior across different libc implementations. My point is that siglongjmp isn't a kernel-provided thing.

So if siglongjmp() restores SS by the side-effect of doing a sigprocmask() syscall, this admittedly weakens my point.
The unreliability then stays only with the async signals interrupting the main one.

Quoted 43 lines
Quoted 35 lines
Quoted 17 lines

For backwards compat, we either need the default behavior to be unchanged, or we need the default behavior to be something that works with existing dosemu. For existing dosemu, the only interesting cases (I think) are signal delivery from *valid* 16-bit context, in which case we need to preserve SS so that the signal handler can read it out with mov ..., %ss, and sigreturn to 64-bit mode for the IRET trampoline. For sigreturn, IIUC old dosemu will replace the saved CS with a 64-bit code segment selector and won't touch the saved SS because it doesn't know about the saved SS. Those dosemu versions don't care what SS actually contains after sigreturn, because they're immediately going to change it again using IRET. So we just need to make sure we return without faulting.

New dosemu2 would like to sigreturn directly back to 16-bit mode, so it needs the kernel to honor the saved ss value and restore it, possibly changed by dosemu.

Quoted 7 lines

We obviously can't require old dosemu to set an SA flag to keep working. But, if we can get away with it, I think it's somewhat preferable not to require new DOSEMU to set an SA flag either.

This has one major benefit at least: if new dosemu loads some random library that installs some async signal handler (SIGALRM for example), everything will work with regard to CS and SS.

This case is covered if we do both things together: use your heuristic when SA_hyz is not set, and don't use it when its set. In this case dosemu2 will be able to request the proper SS delivery for its sighandlers, but the 3rd-party sighandlers will work too.
I think we have never discussed the possibility of doing both things together, even though I have proposed it many times.
After discussing this full-blown solution, we can think about reducing it, either by removing the heuristic or by removing SA_hyz, but discussing the full one would be nice too.

Quoted 8 lines

Your opinion is likely that no one will use this SA_hyz in presence of the heuristic that "seems to work anyway". But in the light of extending it for TLS (with a new flag), I wouldn't be so sure. You can also document it as a needed flag when user code touches SS, and then it will be used. dosemu1 code that doesn't use it, will eventually be forgotten. So IMHO whether it will be used, is fully up to how will you market it. :)

Quoted 1 line

I'll think about it. I'll think about FS and GS, too,

OK, thanks.