Skip to contentSkip to conversation

Emulating lwarx and stwcx instructions in PowerPc BOOKE e500

DISCUSSION Aashish Mittal 3 participants 5 messages Mar 5, 2012 – Mar 6, 2012

lore
Share & tools
lore
#0 lore raw

Hi
I'm working on powerpc booke architecture and my project requires me to remove read and write privileges on some pages. Due to this any instruction accessing these pages traps and i'm trying to emulate the behavior of these instructions.

I've emulated lwarx and stwcx instruction but i think stwcx is not working correctly. The emulation i've written is written below

case OP_31_XOP_LWARX:
{
ulong ret;
ulong addr;
int eh = inst & 0x00000001 ;
kvm_gva_to_hva(vcpu,ea,&addr);
/*lwarx RT RA RB EH*/
if(eh = 0)

      __asm__ __volatile__("lwarx %0,0,%1,0; isync":"=r" (ret) :"r" (addr));
  else
     __asm__ __volatile__("lwarx %0,0,%1,1; isync":"=r" (ret) :"r" (addr));

kvmppc_set_gpr(vcpu,rt,ret);
}

case OP_31_XOP_STWCX:
{
ulong tmp;
ulong addr;
ulong data;
kvm_gva_to_hva(vcpu,ea,&addr);
kvmppc_read_guest(vcpu,ea,&data,sizeof(data));
__asm__ __volatile__("stwcx. %1,0,%2; isync"
:"=r" (tmp):"r" (data),"r" (addr):"memory");

}

Here kvm_gva_to_hva function convrets a guest effective address to host virtual address .

void kvm_gva_to_hva(struct kvm_vcpu *vcpu, ulong ea,ulong* hva) {
gfn_t gfn;
gpa_t gpa ;
int gtlb_index;
int offset;
ulong addr;
struct kvmppc_vcpu_e500 *vcpu_e500 = to_e500(vcpu);

  gtlb_index = kvmppc_mmu_itlb_index(vcpu, ea);
  gpa = kvmppc_mmu_xlate(vcpu,gtlb_index, ea);
  gfn = gpa >> PAGE_SHIFT;
  addr = (ulong)gfn_to_hva(vcpu_e500->vcpu.kvm, gfn);
  offset = offset_in_page(gpa);

*hva = addr + offset;
return;
}

The guest just hangs once it encounters a stwcx instruction. Does anybody have any idea why this is not working and what's wrong about the emulation code.

Also i'm working on linux-3.0-rc4 kernel .

Thanks in advance

#1 lore raw
Quoted 7 lines · Aashish Mittal

Hi
I'm working on powerpc booke architecture and my project requires me to remove read and write privileges on some pages. Due to this any instruction accessing these pages traps and i'm trying to emulate the behavior of these instructions.

I've emulated lwarx and stwcx instruction but i think stwcx is not working correctly. The emulation i've written is written below

What is it you're emulating that needs lwarx/stwcx to work?

Quoted 45 lines

case OP_31_XOP_LWARX:
{
ulong ret;
ulong addr;
int eh = inst & 0x00000001 ;
kvm_gva_to_hva(vcpu,ea,&addr);
/*lwarx RT RA RB EH*/
if(eh = 0)

      __asm__ __volatile__("lwarx %0,0,%1,0; isync":"=r" (ret) :"r" (addr));
  else
     __asm__ __volatile__("lwarx %0,0,%1,1; isync":"=r" (ret) :"r" (addr));

kvmppc_set_gpr(vcpu,rt,ret);
}

case OP_31_XOP_STWCX:
{
ulong tmp;
ulong addr;
ulong data;
kvm_gva_to_hva(vcpu,ea,&addr);
kvmppc_read_guest(vcpu,ea,&data,sizeof(data));
__asm__ __volatile__("stwcx. %1,0,%2; isync"
:"=r" (tmp):"r" (data),"r" (addr):"memory");

}

Here kvm_gva_to_hva function convrets a guest effective address to host virtual address .

void kvm_gva_to_hva(struct kvm_vcpu *vcpu, ulong ea,ulong* hva) {
gfn_t gfn;
gpa_t gpa ;
int gtlb_index;
int offset;
ulong addr;
struct kvmppc_vcpu_e500 *vcpu_e500 = to_e500(vcpu);

  gtlb_index = kvmppc_mmu_itlb_index(vcpu, ea);
  gpa = kvmppc_mmu_xlate(vcpu,gtlb_index, ea);
  gfn = gpa >> PAGE_SHIFT;
  addr = (ulong)gfn_to_hva(vcpu_e500->vcpu.kvm, gfn);
  offset = offset_in_page(gpa);
Quoted 6 lines

*hva = addr + offset;
return;
}

The guest just hangs once it encounters a stwcx instruction. Does anybody have any idea why this is not working and what's wrong about the emulation code.

You're losing the reservation somewhere. Any lock or atomic operation along the emulation path will do this.

Even if this didn't happen by accident, we really don't want to leave a reservation when we return to the guest -- it could have belonged to a previously running guest operating on shared memory, for example. Perhaps we should have a dummy stwcx on KVM guest entry code, similar to the one on interrupt return?

Quoted 1 line

Also i'm working on linux-3.0-rc4 kernel .

Why are you working on something other than the current code or a stable release?

-Scott

#2 lore raw
In reply to Aashish Mittal

Hi Aashish,
Following is an example of the linux code where it uses lwarx and stcwx.

1:     lwarx   %0,0,%3         # atomic_add\n\
        add     %0,%2,%0\n"
        PPC405_ERR77(0,%3)
"       stwcx.  %0,0,%3 \n\
        bne-    1b"

As you would notice after stcwx it checks for equal bit in the cr register. So I think in your code you should also update the cr register after stcwx for the guest.

Regards
Varun

Quoted 79 lines

-----Original Message-----
From: [email protected] [mailto:kvm-ppc- [email protected]] On Behalf Of Aashish Mittal
Sent: Tuesday, March 06, 2012 2:07 AM
To: [email protected]
Subject: Emulating lwarx and stwcx instructions in PowerPc BOOKE e500

Hi
I'm working on powerpc booke architecture and my project requires me to remove read and write privileges on some pages. Due to this any instruction accessing these pages traps and i'm trying to emulate the behavior of these instructions.

I've emulated lwarx and stwcx instruction but i think stwcx is not working correctly. The emulation i've written is written below

case OP_31_XOP_LWARX:
{
ulong ret;
ulong addr;
int eh = inst & 0x00000001 ;
kvm_gva_to_hva(vcpu,ea,&addr);
/*lwarx RT RA RB EH*/
if(eh = 0)

      __asm__ __volatile__("lwarx %0,0,%1,0; isync":"=r" (ret) :"r"
(addr));
  else
     __asm__ __volatile__("lwarx %0,0,%1,1; isync":"=r" (ret) :"r"

(addr));

kvmppc_set_gpr(vcpu,rt,ret);
}

case OP_31_XOP_STWCX:
{
ulong tmp;
ulong addr;
ulong data;
kvm_gva_to_hva(vcpu,ea,&addr);
kvmppc_read_guest(vcpu,ea,&data,sizeof(data));
__asm__ __volatile__("stwcx. %1,0,%2; isync"
:"=r" (tmp):"r" (data),"r" (addr):"memory");

}

Here kvm_gva_to_hva function convrets a guest effective address to host virtual address .

void kvm_gva_to_hva(struct kvm_vcpu *vcpu, ulong ea,ulong* hva) {
gfn_t gfn;
gpa_t gpa ;
int gtlb_index;
int offset;
ulong addr;
struct kvmppc_vcpu_e500 *vcpu_e500 = to_e500(vcpu);

  gtlb_index = kvmppc_mmu_itlb_index(vcpu, ea);
  gpa = kvmppc_mmu_xlate(vcpu,gtlb_index, ea);
  gfn = gpa >> PAGE_SHIFT;
  addr = (ulong)gfn_to_hva(vcpu_e500->vcpu.kvm, gfn);
  offset = offset_in_page(gpa);

*hva = addr + offset;
return;
}

The guest just hangs once it encounters a stwcx instruction. Does anybody have any idea why this is not working and what's wrong about the emulation code.

Also i'm working on linux-3.0-rc4 kernel .

Thanks in advance

--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in the body of a message to [email protected] More majordomo info at
http://vger.kernel.org/majordomo-info.html

#3 lore raw
In reply to Aashish Mittal

I'm thinking of emulating the lwarx and stwcx instruction as follows

1) to emulate lwarx i will just do a load word instruction .
In order to emulate lwarx correctly, i will need to setup a global
reservation structure, which has a reservation entry for each CPU. Upon making the lwarx, i will set the internal reserve bit, then setup a snoop address. All writes to the memory block that the address is in will destroy the reservation in this structure.
Since i'm not calling a direct lwarx instruction upon encountering a lwarx in guest this will not leave a reservation when i'm returning to guest.

2) Upon encountering a stwcx instruction in guest i can match the reservation address
as stored in my global reservation structure. If it matches then i can fire a dummy lwarx
first in host and then fire the stwcx.

Will that work ?

Thanks in advance

Quoted 99 lines · On Tue, Mar 6, 2012 at 8:16 PM, Sethi Varun-B16395 <[email protected]> wrote:

Hi Aashish,
Following is an example of the linux code where it uses lwarx and stcwx.

1:     lwarx   %0,0,%3         # atomic_add\n\
       add     %0,%2,%0\n"
       PPC405_ERR77(0,%3)
"       stwcx.  %0,0,%3 \n\
       bne-    1b"

As you would notice after stcwx it checks for equal bit in the cr register. So I think in your code you should also update the cr register after stcwx for the guest.

Regards
Varun

Quoted 72 lines

-----Original Message-----
From: [email protected] [mailto:kvm-ppc- [email protected]] On Behalf Of Aashish Mittal
Sent: Tuesday, March 06, 2012 2:07 AM
To: [email protected]
Subject: Emulating lwarx and stwcx instructions in PowerPc BOOKE e500

Hi
I'm working on powerpc booke architecture and my project requires me to remove read and write privileges on some pages. Due to this any instruction accessing these pages traps and i'm trying to emulate the behavior of these instructions.

I've emulated lwarx and stwcx instruction but i think stwcx is not working correctly. The emulation i've written is written below

case OP_31_XOP_LWARX:
{
  ulong ret;
  ulong addr;
  int eh = inst & 0x00000001 ; kvm_gva_to_hva(vcpu,ea,&addr);
  /*lwarx RT RA RB EH*/
  if(eh = 0)
      __asm__ __volatile__("lwarx %0,0,%1,0; isync":"=r" (ret) :"r" (addr));
  else
     __asm__ __volatile__("lwarx %0,0,%1,1; isync":"=r" (ret) :"r" (addr));

  kvmppc_set_gpr(vcpu,rt,ret);
}

case OP_31_XOP_STWCX:
{
  ulong tmp;
  ulong addr;
  ulong data;
  kvm_gva_to_hva(vcpu,ea,&addr);
  kvmppc_read_guest(vcpu,ea,&data,sizeof(data));
  __asm__ __volatile__("stwcx. %1,0,%2; isync"
  :"=r" (tmp):"r" (data),"r" (addr):"memory");

}

Here kvm_gva_to_hva function convrets a guest effective address to host virtual address .

void kvm_gva_to_hva(struct kvm_vcpu *vcpu, ulong ea,ulong* hva) { gfn_t gfn;
  gpa_t gpa ;
  int gtlb_index;
  int offset;
  ulong addr;
  struct kvmppc_vcpu_e500 *vcpu_e500 = to_e500(vcpu);

  gtlb_index = kvmppc_mmu_itlb_index(vcpu, ea); gpa = kvmppc_mmu_xlate(vcpu,gtlb_index, ea); gfn = gpa >> PAGE_SHIFT;
  addr = (ulong)gfn_to_hva(vcpu_e500->vcpu.kvm, gfn); offset = offset_in_page(gpa);

  *hva = addr + offset;
  return;
}

The guest just hangs once it encounters a stwcx instruction. Does anybody have any idea why this is not working and what's wrong about the emulation code.

Also i'm working on linux-3.0-rc4 kernel .

Quoted 7 lines

Thanks in advance

--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in the body of a message to [email protected] More majordomo info at
http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe kvm-ppc" in the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Signature · 4 lines
Thanks and regards
Aashish Mittal
Final year Dual Degree
IIT delhi
#4 lore raw
In reply to Aashish Mittal
Quoted 11 lines · Aashish Mittal

I'm thinking of emulating the lwarx and stwcx instruction as follows

1) to emulate lwarx i will just do a load word instruction .
In order to emulate lwarx correctly, i will need to setup a global
reservation structure, which has a reservation entry for each CPU. Upon making the lwarx, i will set the internal reserve bit, then setup a snoop address. All writes to the memory block that the address is in will destroy the reservation in this structure.
Since i'm not calling a direct lwarx instruction upon encountering a lwarx in guest this will not leave a reservation when i'm returning to guest.

Quoted 5 lines

2) Upon encountering a stwcx instruction in guest i can match the reservation address
as stored in my global reservation structure. If it matches then i can fire a dummy lwarx
first in host and then fire the stwcx.

Why bother with the dummy lwarx and stwcx? Just do an ordinary store. You'll want to use a mutex in KVM to protect from multiple vcpus accessing the region at once.

Also check whether you're on a CPU that's supposed to clear reservations on interrupts.

Quoted 1 line

Will that work ?

It should, as long as you aren't sharing this memory with anything that can access it directly (no emulation) that will be writing to the reservation granule or using lwarx/stwcx itself.

But why do you need this?

-Scott