Set TIF_IRET in more places

Consider SA_RESTORER - there isn't a guarantee that user space will use the same code as the kernel's trampoline. glibc happens to, but only because GDB has a hardwired idea of what a signal trampoline looks like. Of course, you could simply document that sigreturn() is another of the system calls that must be made through int 0x80.

It occurs to me that the kernel-provided signal trampoline could go in the page at 0xffff0000, instead of on the user stack, which would eliminate the need for glibc to set SA_RESTORER (it's a pure optimization).

Tangentially, I've seen people claim that the trampoline ought to be able to avoid entering the kernel, although I'm not convinced (how does the signal mask get reset, otherwise?)

zw

Glibc must use the same code as the kernel's trampoline because of MD_FALLBACK_FRAME_STATE_FOR() in GCC's exception handling... (or libgcc.so must change).

It explicitly checks for the opcode sequences 0x58b877000000cd80 and 0xb8ad000000cd80 in order to unwind exception frames around a handled signal. Ugly, isn't it?

Yup.

Welcome to a wonderful if rather unsightly optimisation:

1. libc installs its own handler function for all non-SIG_DFL signals, and sigaction() mostly updates a table in userspace.

2. The libc signal handler redirects all signals to the application through a funky trampoline in libc.

3. A signal mask is maintained in userspace. Also, a pending mask is maintained in userspace.

4. When a signal is delivered, libc's handler function checks the userspace signal mask. If the signal should be blocked, and it is possible to block it, it is marked as pending in _userspace_ pending mask, and the userspace signal mask is propagated to the kernel to prevent further signals queuing up. Any siginfo_t is also saved for tha signa. Then libc's handler returns, without calling the application handler (because that is deferred).

5. When a signal is unblocked from the userspace signal mask, if it is in the userspace pending mask, it is synthetically delivered by userspace, which creates a context _as if_ the kernel had delivered the signal.

By this mechanism, calls to unblock signals from the signal mask can be done without entering the kernel, because the unblocking can be done lazily.

Voila! sigreturn() can be written to avoid entering the kernel. Note that this is possible _now_, with no changes to the kernel. It only requires changes to libc. I think it would work on all architectures, not just i386. (It may also be possible to do it without libc help, in the vsyscall page).

-- Jamie

ps. A similar optimisation allows "spin_lock_irqsave" and "spin_unlock_irqrestore" to avoid using the cli & sti instructions. Spin locks already modify the preempt_count, so use a bit of that to hold the synthetic interrupt-disabled flag, at zero cost... :)

I'd certainly prefer playing these kinds of games _without_ any libc involvement, since making libc play games like this is fragile as hell and works really badly for threaded applications, for example (delivering signals to threads is _hard_ to get right, that's one of the things 2.5.x finally does thanks to Ingo and Uli).

But moving the signal trampoline to vsyscall space to prepare for changes like this is a good idea. I'll remove the SA_RESTORER logic too.

Linus

We're open to better ideas ...

[...]

I would want to be very sure that this was actually a performance win before implementing it, and since it requires data tables in user space I don't see how it could possibly be done in the vsyscall page.

zw

Something like having dwarf2 unwind information for the vsyscall page on the page as well?

r~

What would the unwind info look like? The current BK kernel will put the signal return into the vsyscall page, so gdb could pick up the info from there. But I have no idea what the unwind info looks like, or how to tell gdb about it.

Linus

Right now you can't, since GDB won't use the unwind info anyway; but Richard is probably talking about MD_FALLBACK_FRAME_STATE_FOR, which is used for exception handling unwinding instead of debugging unwinding.

Someday soon I hope to have GDB properly using this info, too.

Daniel Jacobowitz
MontaVista Software                         Debian GNU/Linux Developer

It would be quite nice just to have dwarf2 unwind information, with an unwind handler, for the classic non-vsyscall restorer in Glibc.

Then MD_FALLBACK_FRAME_STATE_FOR could be removed from GCC on all Linux targets, regardless of kernel version.

Once that is working it will be much clearer what exactly to send Linus for the vsyscall page.

-- Jamie

A trivial routine that just calls another routine,

extern void g(void);

void f(void) { g(); asm volatile (""); }

[the asm is to prevent sibcall optimization from kicking in] produces the assembly dump appended to this message when compiled with -O2 -fexceptions -fomit-frame-pointer. I do not know what the stuff put in .eh_frame means, and it probably isn't exactly right for __restore or __restore_rt, but it's a start.

I think we'd need to keep it around for the sake of older libcs; it shouldn't do any harm.

zw

        .text
        .p2align 2,,3

.globl f
.type f,@function
f:
.LFB1:

        subl    $12, %esp
.LCFI0:
        call    g
        addl    $12, %esp

.LCFI1:
ret
.LFE1:
.Lfe1:

        .size   f,.Lfe1-f
        .section        .eh_frame,"aw",@progbits
.Lframe1:
        .long   .LECIE1-.LSCIE1
.LSCIE1:
        .long   0x0
        .byte   0x1
        .string ""
        .uleb128 0x1
        .sleb128 -4
        .byte   0x8
        .byte   0xc
        .uleb128 0x4
        .uleb128 0x4
        .byte   0x88
        .uleb128 0x1
        .align 4

.LECIE1:
.LSFDE1:
.long .LEFDE1-.LASFDE1
.LASFDE1:

        .long   .LASFDE1-.Lframe1
        .long   .LFB1
        .long   .LFE1-.LFB1
        .byte   0x4
        .long   .LCFI0-.LFB1
        .byte   0xe
        .uleb128 0x10
        .byte   0x4
        .long   .LCFI1-.LCFI0
        .byte   0xe
        .uleb128 0x4
        .align 4
.LEFDE1:
        .ident  "GCC: (GNU) 3.2.2 20021231 (Debian prerelease)"