do_multicall_call abi violation? (x86_64)

Booting dom0 and running xend spews quite a number of:

(XEN) (file=multicall.c, line=50) Error writing result back to multicall block.

To jump right to the end of a long story, i think that do_multicall_call() is in x86_64 abi violation.

   #define do_multicall_call(_call)                         \
       do {                                                 \
           __asm__ __volatile__ (                           \
               "movq  "STR(MULTICALL_op)"(%0),%%rax; "      \
     ...
               "movq  "STR(MULTICALL_arg4)"(%0),%%r8; "     \
               "callq *(%%rax); "                           \
               "movq  %%rax,"STR(MULTICALL_result)"(%0); "  \
               : : "b" (_call)                              \
               : "rax", "rdi", "rsi", "rdx", "rcx", "r8");  \
       } while ( 0 )

The abi says the following (v0.95, page 16):

Registers %rbp, %rbx and %r12 through %r15 "belong" to the calling function and the called function is required to preserve their values. In other words, a called function must preserve these registers' values for its caller. Remaining registers "belong" to the called function. If a calling function wants to preserve such a register value across a function call, it must save the value in its local stack frame.

do_multicall_call() is doing a call, who does who knows what with registers, but doesn't advertise clobbering all the abi's volatile registers.

I tried searching for documentation, as it seems to me that gcc should have some magic flag since it has to know the abi anyway, but i couldn't find anything. Assuming that this isn't a compiler bug and there aren't any such flags, the attached patch should make things abi compliant, and fixes the problem i was seeing.

-----

If you're curious the exact connection between the abi violation and the error messages...

do_multicall() looks somewhat like (pseudo-code):

   do_multicall(...) {
      if ( asm("inline-asm-1") )
         return;

      do_multicall_call();

      if ( asm("inline-asm-2") ) {
         printk("Error");
         return;
      }

the result of inline-asm-1 is in some register X. the "if" turns into a jnz on X to a "ret". gcc realizes if we didn't jump, then X has to be 0, so it's going to use that value for initialization of the __pu_err in inline-asm-2.

the problem is that the call in do_multicall_call() stomps on X, as it isn't advertised to clobber X.

inline-asm-2 is the __put_user code, which uses the fixup/exception- table trick:

		"1:	mov"itype" %"rtype"1,%2\n"			\
		"2:\n"							\
		".section .fixup,\"ax\"\n"				\
		"3:	mov %3,%0\n"					\
		"	jmp 2b\n"					\
		".previous\n"						\
		".section __ex_table,\"a\"\n"				\
		"	"__FIXUP_ALIGN"\n"				\
		"	"__FIXUP_WORD" 1b,3b\n"				\
		".previous"						\
		: "=r"(err)						\
		: ltype (x), "m"(__m(addr)), "i"(errret), "0"(err))

so "err" (in register X) is uninitialized if we don't fault. finally depending on what the call did with X if could be a non-zero value.

sRp

Scott Parish
Signed-off-by: [email protected]

diff -rN -u -p old-xen-64-4/xen/include/asm-x86/multicall.h new-xen-64-4/xen/include/asm-x86/multicall.h
@@ -24,7 +24,8 @@
             "callq *(%%rax); "                           \
             "movq  %%rax,"STR(MULTICALL_result)"(%0); "  \
             : : "b" (_call)                              \
-            : "rax", "rdi", "rsi", "rdx", "rcx", "r8" ); \
+            : "rax", "rdi", "rsi", "rdx", "rcx", "r8", 	 \
+	      "r9", "r10", "r11");			 \
     } while ( 0 )
 
 #else

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel