Lazy FPU save/restore

My microbenchmark of choice uses FPU operations so I think the results are currently tainted. I've only tested on a 32bit SVM system.

The following patch saves the host FPU state and loads the guests FPU state if !(CR0.MP == 1 && CR0.TS == 1).

When CR0.MP == 1 && CR0.TS == 1, all FPU activity will generate exceptions. OS's use these exceptions to implement lazy FPU loading to improve context switch time. Since any FPU activity will generate traps, we don't have to worry about the guest modifying the host FPU state.

My microbenchmark of choice uses FPU operations so I think the results are currently tainted. I've only tested on a 32bit SVM system.

Signed-off-by Anthony Liguori <[email protected]>

Quoted 9 lines · Anthony Liguori

The following patch saves the host FPU state and loads the guests FPU state if !(CR0.MP == 1 && CR0.TS == 1).

When CR0.MP == 1 && CR0.TS == 1, all FPU activity will generate exceptions. OS's use these exceptions to implement lazy FPU loading to improve context switch time. Since any FPU activity will generate traps, we don't have to worry about the guest modifying the host FPU state.

Quoted 5 lines

My microbenchmark of choice uses FPU operations so I think the results are currently tainted. I've only tested on a 32bit SVM system.

Signed-off-by Anthony Liguori <[email protected]>

Applied, thanks.

#include <stdio.h>

void test_fpu()

{
    int i;
    double f = 0;

    for (i = 0; i < 10000000; ++i)
    f += 1 / (1.0 + i);
    printf("%20.16f\n", f);
}

int main(int ac, char **av)

{
    while (1)
    test_fpu();
}

Quoted 10 lines · Avi Kivity

Quoted 9 lines · Anthony Liguori

The following patch saves the host FPU state and loads the guests FPU state if !(CR0.MP == 1 && CR0.TS == 1).

When CR0.MP == 1 && CR0.TS == 1, all FPU activity will generate exceptions. OS's use these exceptions to implement lazy FPU loading to improve context switch time. Since any FPU activity will generate traps, we don't have to worry about the guest modifying the host FPU state.

Quoted 8 lines

Quoted 5 lines

My microbenchmark of choice uses FPU operations so I think the results are currently tainted. I've only tested on a 32bit SVM system.

Signed-off-by Anthony Liguori <[email protected]>

Applied, thanks.

... and reverted.

Running an fpu load on both guest and host shows corruption (both on the guest and host). The load is

Quoted 12 lines

#include <stdio.h>

void test_fpu()

{
    int i;
    double f = 0;

    for (i = 0; i < 10000000; ++i)
    f += 1 / (1.0 + i);
    printf("%20.16f\n", f);
}

Quoted 5 lines

int main(int ac, char **av)

{
    while (1)
    test_fpu();
}

Compiled statically with -O2 on 64-bit. I don't know what's wrong -- the code looks correct (well, on Intel it wants KVM_GUEST_CR0_MASK to include CR0_MP_MASK and CR0_TS_MASK, but it fails with that too, and it fails on AMD as well).

Quoted 25 lines

... and reverted.

Running an fpu load on both guest and host shows corruption (both on the guest and host). The load is

Quoted 17 lines

#include <stdio.h>

void test_fpu()

{
    int i;
    double f = 0;

    for (i = 0; i < 10000000; ++i)
    f += 1 / (1.0 + i);
    printf("%20.16f\n", f);
}

int main(int ac, char **av)

{
    while (1)
    test_fpu();
}

Quoted 4 lines

Compiled statically with -O2 on 64-bit. I don't know what's wrong -- the code looks correct (well, on Intel it wants KVM_GUEST_CR0_MASK to include CR0_MP_MASK and CR0_TS_MASK, but it fails with that too, and it fails on AMD as well).

I'll have to try and setup a 64 bit system. I cannot reproduce on my 32bit system with your test program.

Quoted 2 lines

I'll have to try and setup a 64 bit system. I cannot reproduce on my 32bit system with your test program.

It may be related to 64-bit (as that uses sse for floating point), or to the fact that I use ssh on a remote host, which causes more context switches. I'll try to reproduce on 32 bits with my setup.

Quoted 4 lines · Avi Kivity

Quoted 3 lines

I'll have to try and setup a 64 bit system. I cannot reproduce on my 32bit system with your test program.

Quoted 4 lines

It may be related to 64-bit (as that uses sse for floating point), or to the fact that I use ssh on a remote host, which causes more context switches. I'll try to reproduce on 32 bits with my setup.

It doesn't reproduce on a 32-bit host. However, it does reproduce on a 32-bit guest on a 64-bit host, and on a 32-bit user load running in a 64-bit guest on a 64-bit host.

I'm attaching an updated test program which makes it easier to spot the failures. You only need to run it on the guest, although running it on the host as well increases the failure rate significantly.

------------------------------------------------------------------------

#include <stdio.h>
#include <math.h>

double test_fpu_once()

{
    int i;
    double f = 0;

    for (i = 0; i < 10000000; ++i)
	f += 1 / (1.0 + i);
    return f;
}

void test_fpu()

{
    double a, b;
    int runs;

    runs = 0;
    a = test_fpu_once();
    while (1) {
	b = test_fpu_once();
	if (fabs(a - b) > 1e-9)
	    printf("error: %20.16f -> %20.16f\n", a, b);
	a = b;
	if (++runs % 100 == 0)
	    printf("runs: %8d\n", runs);
    }
}

int main(int ac, char **av)

{
    test_fpu();
    return 0;
}

Quoted 10 lines · Avi Kivity

Quoted 8 lines · Avi Kivity

Quoted 3 lines

I'll have to try and setup a 64 bit system. I cannot reproduce on my 32bit system with your test program.

It may be related to 64-bit (as that uses sse for floating point), or to the fact that I use ssh on a remote host, which causes more context switches. I'll try to reproduce on 32 bits with my setup.

Quoted 3 lines

It doesn't reproduce on a 32-bit host. However, it does reproduce on a 32-bit guest on a 64-bit host, and on a 32-bit user load running in a 64-bit guest on a 64-bit host.

Thanks, I'll setup a 64 bit partition and start debugging. I'm a bit confused though, I thought CR0.TS even covers SIMD instructions?