x86/entry/64: Fixes for syscall rework

I broke iopl(2) with my syscall rework. Fix it up. While debugging it, I found a bug in my IRQ state handling. Fix that, too.

Andy Lutomirski (3):
x86/entry/64: Fix an IRQ state error on ptregs-using syscalls x86/entry/64: Fix fast-path syscall return register state x86/syscalls/64: Mark sys_iopl as using ptregs

 arch/x86/entry/entry_64.S              | 23 ++++++++++++++++-------
 arch/x86/entry/syscalls/syscall_64.tbl |  2 +-
 2 files changed, 17 insertions(+), 8 deletions(-)

2.5.0

I messed up the IRQ state when jumping off the fast path due to invocation of a ptregs-using syscall. This bug shouldn't have had any impact yet, but it would have caused problems with subsequent context tracking cleanups.

---
 arch/x86/entry/entry_64.S | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

index 567aa522ac0a..9f7bb808035e 100644
@@ -191,8 +191,8 @@ entry_SYSCALL_64_fastpath:
 
 	/*
 	 * This call instruction is handled specially in stub_ptregs_64.
-	 * It might end up jumping to the slow path.  If it jumps, RAX is
-	 * clobbered.
+	 * It might end up jumping to the slow path.  If it jumps, RAX
+	 * and all argument registers are clobbered.
 	 */
 	call	*sys_call_table(, %rax, 8)
 .Lentry_SYSCALL_64_after_fastpath_call:

@@ -315,17 +315,24 @@ END(entry_SYSCALL_64)
 ENTRY(stub_ptregs_64)
 	/*
 	 * Syscalls marked as needing ptregs land here.
-	 * If we are on the fast path, we need to save the extra regs.
-	 * If we are on the slow path, the extra regs are already saved.
+	 * If we are on the fast path, we need to save the extra regs,
+	 * which we achieve by trying again on the slow path.  If we are on
+	 * the slow path, the extra regs are already saved.
 	 *
 	 * RAX stores a pointer to the C function implementing the syscall.
+	 * IRQs are on.
 	 */
 	cmpq	$.Lentry_SYSCALL_64_after_fastpath_call, (%rsp)
 	jne	1f
 
-	/* Called from fast path -- pop return address and jump to slow path */
+	/*
+	 * Called from fast path -- disable IRQs again, pop return address
+	 * and jump to slow path
+	 */
+	DISABLE_INTERRUPTS(CLBR_NONE)
+	TRACE_IRQS_OFF
 	popq	%rax
-	jmp	entry_SYSCALL64_slow_path	/* called from fast path */
+	jmp	entry_SYSCALL64_slow_path
 
 1:
 	/* Called from C */

2.5.0

sys_iopl both reads and writes pt_regs->flags. Mark it as using ptregs.

This isn't strictly necessary, as pt_regs->flags is available even in the fast path, but this is very lightweight now that we have syscall qualifiers and it could avoid some pain down the road.

---
 arch/x86/entry/syscalls/syscall_64.tbl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index dcf107ce2cd4..2e5b565adacc 100644
@@ -178,7 +178,7 @@
 169	common	reboot			sys_reboot
 170	common	sethostname		sys_sethostname
 171	common	setdomainname		sys_setdomainname
-172	common	iopl			sys_iopl
+172	common	iopl			sys_iopl/ptregs
 173	common	ioperm			sys_ioperm
 174	64	create_module
 175	common	init_module		sys_init_module

2.5.0

I was fishing RIP (i.e. RCX) out of pt_regs->cx and RFLAGS (i.e. R11) out of pt_regs->r11. While it usually worked (pt_regs started out with cx == ip and r11 == flags), it was very fragile. In particular, it broke iopl because iopl forgot to mark itself as using ptregs.

Undo that part of the syscall rework. There was no compelling reason to do it this way. While I'm at it, load RCX and R11 before the other regs to be a little friendlier to the CPU, as they will be the first of the reloaded registers to be used.

---
 arch/x86/entry/entry_64.S | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

index 9f7bb808035e..70eadb0ea5fa 100644
@@ -212,7 +212,9 @@ entry_SYSCALL_64_fastpath:
 
 	LOCKDEP_SYS_EXIT
 	TRACE_IRQS_ON		/* user mode is traced as IRQs on */
-	RESTORE_C_REGS
+	movq	RIP(%rsp), %rcx
+	movq	EFLAGS(%rsp), %r11
+	RESTORE_C_REGS_EXCEPT_RCX_R11
 	movq	RSP(%rsp), %rsp
 	USERGS_SYSRET64
 

2.5.0

Looks good so far:

We better hammer on those a lot more until the next merge window for more confidence.

Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.

Yes, this is why I started merging this right after -rc1.

In particular it would be interesting to get feedback from our 'special' syscall ABI users: emulators (Wine, Steam, etc.) and non-GNU toolchains (Android-x86) - at least the ones that utilize 64-bit syscalls.

Thanks,

Ingo