Skip to contentSkip to conversation

Monitor command: pfa2hva, translate guest physical address to host virtual address

PATCH In review Huang Ying 6 participants 34 messages Oct 26, 2010 – Nov 15, 2010 KVM

lore
Share & tools
lore

Huang Ying

author
#0 lore raw

PATCH 1/1 Monitor command: pfa2hva, translate guest physical address to host virtual address

Author: Max Asbock <[email protected]>

Add command pfa2hva to translate guest physical address to host virtual address.

The pfa2hva command provides one step in a chain of translations from guest virtual to guest physical to host virtual to host physical. Host physical is then used to inject a machine check error. As a consequence the HWPOISON code on the host and the MCE injection code in qemu-kvm are exercised.

v2:

- Add QMP support

---
 hmp-commands.hx |   15 +++++++++++++++
 monitor.c       |   22 ++++++++++++++++++++++
 qmp-commands.hx |   27 +++++++++++++++++++++++++++
 3 files changed, 64 insertions(+)
monitor.c +22 -0
@@ -2272,6 +2272,28 @@ static void do_inject_mce(Monitor *mon,
 }
 #endif
 
+static void do_pfa2hva_print(Monitor *mon, const QObject *data)
+{
+    QInt *qint;
+
+    qint = qobject_to_qint(data);
+    monitor_printf(mon, "0x%lx\n", (unsigned long)qint->value);
+}
+
+static int do_pfa2hva(Monitor *mon, const QDict *qdict, QObject **ret_data)
+{
+    target_phys_addr_t paddr;
+    target_phys_addr_t size = TARGET_PAGE_SIZE;
+    void *vaddr;
+
+    paddr = qdict_get_int(qdict, "addr");
+    vaddr = cpu_physical_memory_map(paddr, &size, 0);
+    cpu_physical_memory_unmap(vaddr, size, 0, 0);
+    *ret_data = qobject_from_jsonf("%ld", (unsigned long)vaddr);
+
+    return 0;
+}
+
 static int do_getfd(Monitor *mon, const QDict *qdict, QObject **ret_data)
 {
     const char *fdname = qdict_get_str(qdict, "fdname");
hmp-commands.hx +15 -0
@@ -293,6 +293,21 @@ Start gdbserver session (default @var{po
 ETEXI
 
     {
+        .name       = "pfa2hva",
+        .args_type  = "fmt:/,addr:l",
+        .params     = "/fmt addr",
+        .help       = "translate guest physical 'addr' to host virtual address",
+        .user_print = do_pfa2hva_print,
+        .mhandler.cmd_new = do_pfa2hva,
+    },
+
+STEXI
+@item pfa2hva @var{addr}
+@findex pfa2hva
+Translate guest physical @var{addr} to host virtual address.
+ETEXI
+
+    {
         .name       = "x",
         .args_type  = "fmt:/,addr:l",
         .params     = "/fmt addr",
qmp-commands.hx +27 -0
@@ -738,6 +738,33 @@ Example:
 EQMP
 
     {
+        .name       = "pfa2hva",
+        .args_type  = "addr:l",
+        .params     = "addr",
+        .help       = "translate guest physical 'addr' to host virtual address",
+        .user_print = do_pfa2hva_print,
+        .mhandler.cmd_new = do_pfa2hva,
+    },
+
+SQMP
+pfa2hva
+---
+
+Translate guest physical 'addr' to host virtual address.
+
+Arguments:
+
+- "addr": the guest physical address
+
+Example:
+
+-> { "execute": "pfa2hva",
+		"arguments": { "addr": 196608 } }
+<- { "return": 139888084717568 }
+
+EQMP
+
+    {
         .name       = "qmp_capabilities",
         .args_type  = "",
         .params     = "",
#1 lore raw
Quoted 11 lines · Huang Ying

Author: Max Asbock <[email protected]>

Add command pfa2hva to translate guest physical address to host virtual address.

The pfa2hva command provides one step in a chain of translations from guest virtual to guest physical to host virtual to host physical. Host physical is then used to inject a machine check error. As a consequence the HWPOISON code on the host and the MCE injection code in qemu-kvm are exercised.

Quoted 3 lines

v2:

- Add QMP support

Looks good to me. Anthony?

Quoted 102 lines
---
 hmp-commands.hx |   15 +++++++++++++++
 monitor.c       |   22 ++++++++++++++++++++++
 qmp-commands.hx |   27 +++++++++++++++++++++++++++
 3 files changed, 64 insertions(+)
monitor.c +22 -0
@@ -2272,6 +2272,28 @@ static void do_inject_mce(Monitor *mon,
 }
 #endif
 
+static void do_pfa2hva_print(Monitor *mon, const QObject *data)
+{
+    QInt *qint;
+
+    qint = qobject_to_qint(data);
+    monitor_printf(mon, "0x%lx\n", (unsigned long)qint->value);
+}
+
+static int do_pfa2hva(Monitor *mon, const QDict *qdict, QObject **ret_data)
+{
+    target_phys_addr_t paddr;
+    target_phys_addr_t size = TARGET_PAGE_SIZE;
+    void *vaddr;
+
+    paddr = qdict_get_int(qdict, "addr");
+    vaddr = cpu_physical_memory_map(paddr, &size, 0);
+    cpu_physical_memory_unmap(vaddr, size, 0, 0);
+    *ret_data = qobject_from_jsonf("%ld", (unsigned long)vaddr);
+
+    return 0;
+}
+
 static int do_getfd(Monitor *mon, const QDict *qdict, QObject **ret_data)
 {
     const char *fdname = qdict_get_str(qdict, "fdname");
hmp-commands.hx +15 -0
@@ -293,6 +293,21 @@ Start gdbserver session (default @var{po
 ETEXI
 
     {
+        .name       = "pfa2hva",
+        .args_type  = "fmt:/,addr:l",
+        .params     = "/fmt addr",
+        .help       = "translate guest physical 'addr' to host virtual address",
+        .user_print = do_pfa2hva_print,
+        .mhandler.cmd_new = do_pfa2hva,
+    },
+
+STEXI
+@item pfa2hva @var{addr}
+@findex pfa2hva
+Translate guest physical @var{addr} to host virtual address.
+ETEXI
+
+    {
         .name       = "x",
         .args_type  = "fmt:/,addr:l",
         .params     = "/fmt addr",
qmp-commands.hx +27 -0
@@ -738,6 +738,33 @@ Example:
 EQMP
 
     {
+        .name       = "pfa2hva",
+        .args_type  = "addr:l",
+        .params     = "addr",
+        .help       = "translate guest physical 'addr' to host virtual address",
+        .user_print = do_pfa2hva_print,
+        .mhandler.cmd_new = do_pfa2hva,
+    },
+
+SQMP
+pfa2hva
+---
+
+Translate guest physical 'addr' to host virtual address.
+
+Arguments:
+
+- "addr": the guest physical address
+
+Example:
+
+-> { "execute": "pfa2hva",
+		"arguments": { "addr": 196608 } }
+<- { "return": 139888084717568 }
+
+EQMP
+
+    {
         .name       = "qmp_capabilities",
         .args_type  = "",
         .params     = "",
#2 lore raw
Quoted 13 lines · Marcelo Tosatti
Quoted 11 lines · Huang Ying

Author: Max Asbock<[email protected]>

Add command pfa2hva to translate guest physical address to host virtual address.

The pfa2hva command provides one step in a chain of translations from guest virtual to guest physical to host virtual to host physical. Host physical is then used to inject a machine check error. As a consequence the HWPOISON code on the host and the MCE injection code in qemu-kvm are exercised.

Quoted 7 lines
Quoted 4 lines

v2:

- Add QMP support

Looks good to me. Anthony?

The translation is not stable so this is really a bad interface.

It definitely shouldn't be a QMP command and I don't think it's generally useful enough that it should be a monitor command.

It's impossible to use correctly as a general interface.

I gave this feedback when it was initially submitted.

Regards,

Anthony Liguori

Quoted 102 lines
Quoted 102 lines
---
  hmp-commands.hx |   15 +++++++++++++++
  monitor.c       |   22 ++++++++++++++++++++++
  qmp-commands.hx |   27 +++++++++++++++++++++++++++
  3 files changed, 64 insertions(+)
monitor.c +22 -0
@@ -2272,6 +2272,28 @@ static void do_inject_mce(Monitor *mon,
  }
  #endif
 
+static void do_pfa2hva_print(Monitor *mon, const QObject *data)
+{
+    QInt *qint;
+
+    qint = qobject_to_qint(data);
+    monitor_printf(mon, "0x%lx\n", (unsigned long)qint->value);
+}
+
+static int do_pfa2hva(Monitor *mon, const QDict *qdict, QObject **ret_data)
+{
+    target_phys_addr_t paddr;
+    target_phys_addr_t size = TARGET_PAGE_SIZE;
+    void *vaddr;
+
+    paddr = qdict_get_int(qdict, "addr");
+    vaddr = cpu_physical_memory_map(paddr,&size, 0);
+    cpu_physical_memory_unmap(vaddr, size, 0, 0);
+    *ret_data = qobject_from_jsonf("%ld", (unsigned long)vaddr);
+
+    return 0;
+}
+
  static int do_getfd(Monitor *mon, const QDict *qdict, QObject **ret_data)
  {
      const char *fdname = qdict_get_str(qdict, "fdname");
hmp-commands.hx +15 -0
@@ -293,6 +293,21 @@ Start gdbserver session (default @var{po
  ETEXI
 
      {
+        .name       = "pfa2hva",
+        .args_type  = "fmt:/,addr:l",
+        .params     = "/fmt addr",
+        .help       = "translate guest physical 'addr' to host virtual address",
+        .user_print = do_pfa2hva_print,
+        .mhandler.cmd_new = do_pfa2hva,
+    },
+
+STEXI
+@item pfa2hva @var{addr}
+@findex pfa2hva
+Translate guest physical @var{addr} to host virtual address.
+ETEXI
+
+    {
          .name       = "x",
          .args_type  = "fmt:/,addr:l",
          .params     = "/fmt addr",
qmp-commands.hx +27 -0
@@ -738,6 +738,33 @@ Example:
  EQMP
 
      {
+        .name       = "pfa2hva",
+        .args_type  = "addr:l",
+        .params     = "addr",
+        .help       = "translate guest physical 'addr' to host virtual address",
+        .user_print = do_pfa2hva_print,
+        .mhandler.cmd_new = do_pfa2hva,
+    },
+
+SQMP
+pfa2hva
+---
+
+Translate guest physical 'addr' to host virtual address.
+
+Arguments:
+
+- "addr": the guest physical address
+
+Example:
+
+->  { "execute": "pfa2hva",
+		"arguments": { "addr": 196608 } }
+<- { "return": 139888084717568 }
+
+EQMP
+
+    {
          .name       = "qmp_capabilities",
          .args_type  = "",
          .params     = "",
author
#3 lore raw
Quoted 22 lines · Anthony Liguori
Quoted 13 lines · Marcelo Tosatti
Quoted 11 lines · Huang Ying

Author: Max Asbock<[email protected]>

Add command pfa2hva to translate guest physical address to host virtual address.

The pfa2hva command provides one step in a chain of translations from guest virtual to guest physical to host virtual to host physical. Host physical is then used to inject a machine check error. As a consequence the HWPOISON code on the host and the MCE injection code in qemu-kvm are exercised.

Quoted 7 lines
Quoted 4 lines

v2:

- Add QMP support

Looks good to me. Anthony?

Quoted 8 lines

The translation is not stable so this is really a bad interface.

It definitely shouldn't be a QMP command and I don't think it's generally useful enough that it should be a monitor command.

It's impossible to use correctly as a general interface.

I gave this feedback when it was initially submitted.

Yes. As general interface, it may not work so well, but as test interface, it works quite well and useful.

Do we have any mechanism to add a test only interface?

Best Regards,
Huang Ying

#4 lore raw
Quoted 5 lines · Huang Ying

Yes. As general interface, it may not work so well, but as test interface, it works quite well and useful.

Do we have any mechanism to add a test only interface?

I'd like to see what Luiz/Markus think but definitely only a human monitor interface and probably prefix the command with a 'x-' prefix to indicate that it's not a supported interface.

The documentation should be very clear about the limitations of the interface and the intended use-case.

Regards,

Anthony Liguori

Quoted 5 lines

Best Regards,
Huang Ying

#5 lore raw
Quoted 5 lines · On Mon, 01 Nov 2010 14:22:21 -0500 Anthony Liguori…
Quoted 4 lines · Huang Ying

Yes. As general interface, it may not work so well, but as test interface, it works quite well and useful.

Do we have any mechanism to add a test only interface?

No.

We're likely going to introduce a new namespace for that in the near future, but only for QMP.

Is this command supposed to be useful for applications or is it primary designed for (human) debugging?

Quoted 3 lines

I'd like to see what Luiz/Markus think but definitely only a human monitor interface and probably prefix the command with a 'x-' prefix to indicate that it's not a supported interface.

Yes, I agree on having only the human interface but I'm not sure about prexing it with 'x-'. I think new HMP commands should be declared unstable by default starting with 0.14.

Quoted 13 lines

The documentation should be very clear about the limitations of the interface and the intended use-case.

Regards,

Anthony Liguori

Quoted 5 lines

Best Regards,
Huang Ying

#6 lore raw
In reply to Anthony Liguori
Quoted 8 lines · Anthony Liguori
Quoted 4 lines · Huang Ying

Yes. As general interface, it may not work so well, but as test interface, it works quite well and useful.

Do we have any mechanism to add a test only interface?

I'd like to see what Luiz/Markus think but definitely only a human monitor interface and probably prefix the command with a 'x-' prefix to indicate that it's not a supported interface.

Automated testing will want to use qmp.

Quoted 2 lines

The documentation should be very clear about the limitations of the interface and the intended use-case.

Perhaps a { execute: 'enable-version-specific-commands', arguments: ['pfa2hva'] } ?

Signature · 2 lines
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
author
#7 lore raw
Quoted 5 lines · Avi Kivity
Quoted 4 lines · Anthony Liguori
Quoted 3 lines · Huang Ying

Yes. As general interface, it may not work so well, but as test interface, it works quite well and useful.

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

Do we have any mechanism to add a test only interface?

I'd like to see what Luiz/Markus think but definitely only a human monitor interface and probably prefix the command with a 'x-' prefix to indicate that it's not a supported interface.

Automated testing will want to use qmp.

Yes. The main usage of the interface is automated testing.

Quoted 6 lines
Quoted 3 lines

The documentation should be very clear about the limitations of the interface and the intended use-case.

Perhaps a { execute: 'enable-version-specific-commands', arguments:
['pfa2hva'] } ?

Best Regards,
Huang Ying

#8 lore raw
Quoted 11 lines · Huang Ying
Quoted 9 lines · Avi Kivity
Quoted 7 lines · Anthony Liguori
Quoted 5 lines · Huang Ying

Yes. As general interface, it may not work so well, but as test interface, it works quite well and useful.

Do we have any mechanism to add a test only interface?

Quoted 9 lines
Quoted 7 lines
Quoted 5 lines

I'd like to see what Luiz/Markus think but definitely only a human monitor interface and probably prefix the command with a 'x-' prefix to indicate that it's not a supported interface.

Automated testing will want to use qmp.

Yes. The main usage of the interface is automated testing.

That's precisely what the command should not be used for.

You can't assume a gpa -> hva mapping is consistent in an external application. If you want to implement an interface for testing, you have to push more of the logic into QEMU to avoid the race.

Regards,

Anthony Liguori

Quoted 12 lines
Quoted 7 lines
Quoted 4 lines

The documentation should be very clear about the limitations of the interface and the intended use-case.

Perhaps a { execute: 'enable-version-specific-commands', arguments:
['pfa2hva'] } ?

Best Regards,
Huang Ying

author
#9 lore raw

Hi, Anthony,

Quoted 19 lines · Anthony Liguori
Quoted 18 lines · Huang Ying
Quoted 16 lines · Avi Kivity
Quoted 12 lines · Anthony Liguori
Quoted 5 lines · Huang Ying

Yes. As general interface, it may not work so well, but as test interface, it works quite well and useful.

Do we have any mechanism to add a test only interface?

I'd like to see what Luiz/Markus think but definitely only a human monitor interface and probably prefix the command with a 'x-' prefix to indicate that it's not a supported interface.

Automated testing will want to use qmp.

Quoted 8 lines
Quoted 2 lines

Yes. The main usage of the interface is automated testing.

That's precisely what the command should not be used for.

You can't assume a gpa -> hva mapping is consistent in an external application. If you want to implement an interface for testing, you have to push more of the logic into QEMU to avoid the race.

Yes. pfa2hva (should be renamed to gpa2hva) in the patch works not well in general cases, but it works well for our testing. So we do not want to make it a general interface, but a testing only interface. Maybe via putting it into a special name space like "x-". Do you agree?

Best Regards,
Huang Ying

author
#10 lore raw
In reply to Anthony Liguori
Quoted 22 lines · Anthony Liguori
Quoted 11 lines · Huang Ying
Quoted 9 lines · Avi Kivity
Quoted 7 lines · Anthony Liguori
Quoted 5 lines · Huang Ying

Yes. As general interface, it may not work so well, but as test interface, it works quite well and useful.

Do we have any mechanism to add a test only interface?

Quoted 9 lines
Quoted 7 lines
Quoted 5 lines

I'd like to see what Luiz/Markus think but definitely only a human monitor interface and probably prefix the command with a 'x-' prefix to indicate that it's not a supported interface.

Automated testing will want to use qmp.

Yes. The main usage of the interface is automated testing.

Quoted 7 lines

That's precisely what the command should not be used for.

You can't assume a gpa -> hva mapping is consistent in an external application. If you want to implement an interface for testing, you have to push more of the logic into QEMU to avoid the race.

From the code of cpu_physical_memory_map(), it seems that if the 'addr'

is physical address in RAM, the mapping should be consistent at least for x86, doesn't it?

Best Regards,
Huang Ying

#11 lore raw
In reply to Anthony Liguori
Quoted 5 lines · Anthony Liguori
Quoted 1 line

Yes. The main usage of the interface is automated testing.

That's precisely what the command should not be used for.

You can't assume a gpa -> hva mapping is consistent in an external application. If you want to implement an interface for testing, you have to push more of the logic into QEMU to avoid the race.

An alternative is to use -mempath. Does poisoning work for tmpfs?

Signature · 2 lines
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
#12 lore raw
Quoted 9 lines · Avi Kivity
Quoted 5 lines · Anthony Liguori
Quoted 1 line

Yes. The main usage of the interface is automated testing.

That's precisely what the command should not be used for.

You can't assume a gpa -> hva mapping is consistent in an external application. If you want to implement an interface for testing, you have to push more of the logic into QEMU to avoid the race.

An alternative is to use -mempath. Does poisoning work for tmpfs?

Or hugetlbfs - I think it does?

Signature · 2 lines
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
author
#13 lore raw
Quoted 6 lines · Avi Kivity
Quoted 5 lines · Avi Kivity
Quoted 4 lines · Anthony Liguori
Quoted 1 line

Yes. The main usage of the interface is automated testing.

That's precisely what the command should not be used for.

Quoted 7 lines
Quoted 6 lines
Quoted 3 lines

You can't assume a gpa -> hva mapping is consistent in an external application. If you want to implement an interface for testing, you have to push more of the logic into QEMU to avoid the race.

An alternative is to use -mempath. Does poisoning work for tmpfs?

Or hugetlbfs - I think it does?

The QEMU support for hugetlbfs has some issues now. Because it is hard for QEMU to deal with 2M poisoned page reported by host OS. Although it is possible for QEMU to relay 2M poisoned page as MCE to guest OS, the guest OS may not work properly for this kind of MCE.

Best Regards,
Huang Ying

#14 lore raw
Quoted 11 lines · Huang Ying
Quoted 10 lines · Avi Kivity
Quoted 9 lines · Avi Kivity
Quoted 7 lines · Anthony Liguori
Quoted 1 line

Yes. The main usage of the interface is automated testing.

That's precisely what the command should not be used for.

You can't assume a gpa -> hva mapping is consistent in an external application. If you want to implement an interface for testing, you have to push more of the logic into QEMU to avoid the race.

Quoted 9 lines
Quoted 3 lines
Quoted 2 lines

An alternative is to use -mempath. Does poisoning work for tmpfs?

Or hugetlbfs - I think it does?

The QEMU support for hugetlbfs has some issues now. Because it is hard for QEMU to deal with 2M poisoned page reported by host OS. Although it is possible for QEMU to relay 2M poisoned page as MCE to guest OS, the guest OS may not work properly for this kind of MCE.

If we get a full address (rather than just a frame number) then we can identify the 4k page and send an mce just for that frame?

Signature · 1 line
error compiling committee.c: too many arguments to function
author
#15 lore raw
Quoted 16 lines · Avi Kivity
Quoted 15 lines · Huang Ying
Quoted 13 lines · Avi Kivity
Quoted 11 lines · Avi Kivity
Quoted 7 lines · Anthony Liguori
Quoted 1 line

Yes. The main usage of the interface is automated testing.

That's precisely what the command should not be used for.

You can't assume a gpa -> hva mapping is consistent in an external application. If you want to implement an interface for testing, you have to push more of the logic into QEMU to avoid the race.

An alternative is to use -mempath. Does poisoning work for tmpfs?

Or hugetlbfs - I think it does?

Quoted 8 lines
Quoted 5 lines

The QEMU support for hugetlbfs has some issues now. Because it is hard for QEMU to deal with 2M poisoned page reported by host OS. Although it is possible for QEMU to relay 2M poisoned page as MCE to guest OS, the guest OS may not work properly for this kind of MCE.

If we get a full address (rather than just a frame number) then we can identify the 4k page and send an mce just for that frame?

We need host kernel to break down the 2M huge page into 4k pages. Then send SIGBUS to QEMU with the poisoned 4k page. Because host kernel will poison the whole 2M virtual address space otherwise, and other 4k pages inside the 2M page can not used accessed in guest (will trigger SIGBUS and SRAR MCE).

Best Regards,
Huang Ying

#16 lore raw
Quoted 10 lines · Huang Ying
Quoted 3 lines

If we get a full address (rather than just a frame number) then we can identify the 4k page and send an mce just for that frame?

We need host kernel to break down the 2M huge page into 4k pages. Then send SIGBUS to QEMU with the poisoned 4k page. Because host kernel will poison the whole 2M virtual address space otherwise, and other 4k pages inside the 2M page can not used accessed in guest (will trigger SIGBUS and SRAR MCE).

We could do that by using mmap(MAP_FIXED) or mremap(). It's probably not worth it just for testing, but it has a real use, no?

Note we may need to stop the guest while we copy memory.

Signature · 1 line
error compiling committee.c: too many arguments to function
#17 lore raw
In reply to Huang Ying
Quoted 5 lines

We need host kernel to break down the 2M huge page into 4k pages. Then send SIGBUS to QEMU with the poisoned 4k page. Because host kernel will poison the whole 2M virtual address space otherwise, and other 4k pages inside the 2M page can not used accessed in guest (will trigger SIGBUS and SRAR MCE).

The easiest way would be to port the respective code from Andrea's transparent hugetlb patchkit. It already does break down huge pages as needed. You just have to be careful to not touch (/copy) the corrupted subpage.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.
#18 lore raw
Quoted 10 lines · Andi Kleen
Quoted 5 lines

We need host kernel to break down the 2M huge page into 4k pages. Then send SIGBUS to QEMU with the poisoned 4k page. Because host kernel will poison the whole 2M virtual address space otherwise, and other 4k pages inside the 2M page can not used accessed in guest (will trigger SIGBUS and SRAR MCE).

The easiest way would be to port the respective code from Andrea's transparent hugetlb patchkit. It already does break down huge pages as needed. You just have to be careful to not touch (/copy) the corrupted subpage.

Doing it in userspace in easier, since we can replace the vma for that section (and avoid mixed 4k/2M pages in hugetlbfs).

With transparent hugepages it's easy of course, you just call split_hugepage() and poison the 4k page.

Signature · 1 line
error compiling committee.c: too many arguments to function
#19 lore raw
Quoted 2 lines

Doing it in userspace in easier, since we can replace the vma for that section (and avoid mixed 4k/2M pages in hugetlbfs).

You can't do that today, there's no way currently to access the non corrupted portion of the 2MB page. Once it's poisoned it's all gone.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.
#20 lore raw
Quoted 5 lines · Andi Kleen
Quoted 2 lines

Doing it in userspace in easier, since we can replace the vma for that section (and avoid mixed 4k/2M pages in hugetlbfs).

You can't do that today, there's no way currently to access the non corrupted portion of the 2MB page. Once it's poisoned it's all gone.

I see. Thanks.

Signature · 1 line
error compiling committee.c: too many arguments to function
#21 lore raw
Quoted 8 lines · Avi Kivity
Quoted 5 lines · Andi Kleen
Quoted 2 lines

Doing it in userspace in easier, since we can replace the vma for that section (and avoid mixed 4k/2M pages in hugetlbfs).

You can't do that today, there's no way currently to access the non corrupted portion of the 2MB page. Once it's poisoned it's all gone.

I see. Thanks.

BTW my understanding is that KVM will only use hugepages with transparent huge pages anyways, correct?

So it may be reasonable to simply fix this as part of the transparent hugepages work, but ignore it before that.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.
#22 lore raw
Quoted 5 lines · Andi Kleen
Quoted 4 lines · Avi Kivity
Quoted 3 lines · Andi Kleen
Quoted 2 lines

Doing it in userspace in easier, since we can replace the vma for that section (and avoid mixed 4k/2M pages in hugetlbfs).

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

You can't do that today, there's no way currently to access the non corrupted portion of the 2MB page. Once it's poisoned it's all gone.

I see. Thanks.

BTW my understanding is that KVM will only use hugepages with transparent huge pages anyways, correct?

There will always be those who want full control over every last bit; they will use hugetlbfs and numactl for manual allocation of resources for guests.

General usage will be with transparent huge pages, though.

Quoted 2 lines

So it may be reasonable to simply fix this as part of the transparent hugepages work, but ignore it before that.

Transparent hugepages work only for anonymous memory.

However, hugetlbfs is not the only option for backing -mempath memory; if poisoning works for tmpfs/shm or other file backed memory, we can use that to get named access to memory.

Signature · 2 lines
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
author
#23 lore raw
In reply to Avi Kivity
Quoted 10 lines · Avi Kivity
Quoted 7 lines · Anthony Liguori
Quoted 1 line

Yes. The main usage of the interface is automated testing.

That's precisely what the command should not be used for.

You can't assume a gpa -> hva mapping is consistent in an external application. If you want to implement an interface for testing, you have to push more of the logic into QEMU to avoid the race.

An alternative is to use -mempath. Does poisoning work for tmpfs?

Sorry for my stupid question. Why the gpa -> hva mapping is not consistent for RAM if -mempath is not used?

Best Regards,
Huang Ying

#24 lore raw
Quoted 6 lines · Huang Ying
Quoted 5 lines · Avi Kivity
Quoted 4 lines · Anthony Liguori
Quoted 1 line

Yes. The main usage of the interface is automated testing.

That's precisely what the command should not be used for.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

You can't assume a gpa -> hva mapping is consistent in an external application. If you want to implement an interface for testing, you have to push more of the logic into QEMU to avoid the race.

An alternative is to use -mempath. Does poisoning work for tmpfs?

Sorry for my stupid question.

It is not a stupid question.

Quoted 2 lines

Why the gpa -> hva mapping is not
consistent for RAM if -mempath is not used?

Video RAM in the range a0000-bffff and PCI mapped RAM can change gpas (while remaining in the same hva).

Even for ordinary RAM, which doesn't normally change gpa/hva, I'm not sure we want to guarantee that it won't.

Signature · 1 line
error compiling committee.c: too many arguments to function
#25 lore raw
Quoted 6 lines · Avi Kivity
Quoted 2 lines

Why the gpa -> hva mapping is not
consistent for RAM if -mempath is not used?

Video RAM in the range a0000-bffff and PCI mapped RAM can change gpas (while remaining in the same hva).

Even for ordinary RAM, which doesn't normally change gpa/hva, I'm not sure we want to guarantee that it won't.

We can't universally either. Memory hot remove potentially breaks the mapping and some non-x86 architectures (like ARM) can alias RAM via a guest triggered mechanism.

Regards,

Anthony Liguori

author
#26 lore raw
Quoted 7 lines · Anthony Liguori
Quoted 6 lines · Avi Kivity
Quoted 2 lines

Why the gpa -> hva mapping is not
consistent for RAM if -mempath is not used?

Video RAM in the range a0000-bffff and PCI mapped RAM can change gpas (while remaining in the same hva).

Quoted 6 lines
Quoted 2 lines

Even for ordinary RAM, which doesn't normally change gpa/hva, I'm not sure we want to guarantee that it won't.

We can't universally either. Memory hot remove potentially breaks the mapping and some non-x86 architectures (like ARM) can alias RAM via a guest triggered mechanism.

Thanks for clarification. Now I think we have two options.

1) Clearly mark gpa2hva (pfa2hva now, should renamed to gpa2hva) as a testing only interface, and should be used only on restricted environment (normal memory, without hot-remove, maybe only on x86).

2) Find some way to lock the gpa -> hva mapping during operating. Such as gpa2hva_begin and gpa2hva_end and lock the mapping in between.

I think 2) may be possible. But if there are no other users, why do that for a test case? So I think 1) may be the better option.

Best Regards,
Huang Ying

#27 lore raw
Quoted 21 lines · Huang Ying
Quoted 7 lines · Anthony Liguori
Quoted 6 lines · Avi Kivity
Quoted 2 lines

Why the gpa -> hva mapping is not
consistent for RAM if -mempath is not used?

Video RAM in the range a0000-bffff and PCI mapped RAM can change gpas (while remaining in the same hva).

Quoted 6 lines
Quoted 2 lines

Even for ordinary RAM, which doesn't normally change gpa/hva, I'm not sure we want to guarantee that it won't.

We can't universally either. Memory hot remove potentially breaks the mapping and some non-x86 architectures (like ARM) can alias RAM via a guest triggered mechanism.

Thanks for clarification. Now I think we have two options.

1) Clearly mark gpa2hva (pfa2hva now, should renamed to gpa2hva) as a testing only interface, and should be used only on restricted environment (normal memory, without hot-remove, maybe only on x86).

Quoted 6 lines

2) Find some way to lock the gpa -> hva mapping during operating. Such as gpa2hva_begin and gpa2hva_end and lock the mapping in between.

I think 2) may be possible. But if there are no other users, why do that for a test case? So I think 1) may be the better option.

3) Move the poisoning code into qemu, so the command becomes

posison-address <addr>

(though physical addresses aren't stable either)

4) Use -mempath on /dev/shm and poison a page in the backing file

(if poisoning works on tmpfs)

Signature · 1 line
error compiling committee.c: too many arguments to function
author
#28 lore raw
Quoted 29 lines · Avi Kivity
Quoted 21 lines · Huang Ying
Quoted 7 lines · Anthony Liguori
Quoted 6 lines · Avi Kivity
Quoted 2 lines

Why the gpa -> hva mapping is not
consistent for RAM if -mempath is not used?

Video RAM in the range a0000-bffff and PCI mapped RAM can change gpas (while remaining in the same hva).

Quoted 6 lines
Quoted 2 lines

Even for ordinary RAM, which doesn't normally change gpa/hva, I'm not sure we want to guarantee that it won't.

We can't universally either. Memory hot remove potentially breaks the mapping and some non-x86 architectures (like ARM) can alias RAM via a guest triggered mechanism.

Thanks for clarification. Now I think we have two options.

1) Clearly mark gpa2hva (pfa2hva now, should renamed to gpa2hva) as a testing only interface, and should be used only on restricted environment (normal memory, without hot-remove, maybe only on x86).

Quoted 6 lines

2) Find some way to lock the gpa -> hva mapping during operating. Such as gpa2hva_begin and gpa2hva_end and lock the mapping in between.

I think 2) may be possible. But if there are no other users, why do that for a test case? So I think 1) may be the better option.

Quoted 5 lines

3) Move the poisoning code into qemu, so the command becomes

posison-address <addr>

(though physical addresses aren't stable either)

Poisoning includes:

a) gva -> gpa
b) gpa -> hva
c) hva -> hpa
d) inject the MCE into host via some external tool

poison-address need to do b, c, d. Do you think it is good to do all these inside qemu?

Quoted 1 line

4) Use -mempath on /dev/shm and poison a page in the backing file

If we can poison a page in the backing file, how do we know the corresponding gpa and hpa?

Best Regards,
Huang Ying

#29 lore raw
Quoted 38 lines · Huang Ying
Quoted 29 lines · Avi Kivity
Quoted 21 lines · Huang Ying
Quoted 7 lines · Anthony Liguori
Quoted 6 lines · Avi Kivity
Quoted 2 lines

Why the gpa -> hva mapping is not
consistent for RAM if -mempath is not used?

Video RAM in the range a0000-bffff and PCI mapped RAM can change gpas (while remaining in the same hva).

Quoted 6 lines
Quoted 2 lines

Even for ordinary RAM, which doesn't normally change gpa/hva, I'm not sure we want to guarantee that it won't.

We can't universally either. Memory hot remove potentially breaks the mapping and some non-x86 architectures (like ARM) can alias RAM via a guest triggered mechanism.

Thanks for clarification. Now I think we have two options.

1) Clearly mark gpa2hva (pfa2hva now, should renamed to gpa2hva) as a testing only interface, and should be used only on restricted environment (normal memory, without hot-remove, maybe only on x86).

Quoted 6 lines

2) Find some way to lock the gpa -> hva mapping during operating. Such
as gpa2hva_begin and gpa2hva_end and lock the mapping in between.

I think 2) may be possible. But if there are no other users, why do that for a test case? So I think 1) may be the better option.

Quoted 5 lines

3) Move the poisoning code into qemu, so the command becomes

posison-address<addr>

(though physical addresses aren't stable either)

Poisoning includes:

Quoted 7 lines
a) gva ->  gpa
b) gpa ->  hva
c) hva ->  hpa

d) inject the MCE into host via some external tool

poison-address need to do b, c, d. Do you think it is good to do all these inside qemu?

If d is not too complicated (an ioctl?), we might to it in qemu.

Quoted 4 lines
Quoted 1 line

4) Use -mempath on /dev/shm and poison a page in the backing file

If we can poison a page in the backing file, how do we know the corresponding gpa and hpa?

I think you currently can't know it's gpa (why do you want to?); the upcoming NUMA enhancements should allow this (the plan is to have a file per guest NUMA node, so we can tell the host what policy to apply for that node).

gpa->hpa translation can be derived from /proc/pid/maps.

Signature · 1 line
error compiling committee.c: too many arguments to function
#30 lore raw
Quoted 10 lines · Avi Kivity
Quoted 5 lines
Quoted 2 lines

4) Use -mempath on /dev/shm and poison a page in the backing file

If we can poison a page in the backing file, how do we know the corresponding gpa and hpa?

I think you currently can't know it's gpa (why do you want to?); the upcoming NUMA enhancements should allow this (the plan is to have a file per guest NUMA node, so we can tell the host what policy to apply for that node).

gpa->hpa translation can be derived from /proc/pid/maps.

That would be gpa->hva.

Signature · 1 line
error compiling committee.c: too many arguments to function
author
#31 lore raw
In reply to Avi Kivity
Quoted 44 lines · Avi Kivity
Quoted 43 lines · Huang Ying
Quoted 29 lines · Avi Kivity
Quoted 21 lines · Huang Ying
Quoted 7 lines · Anthony Liguori
Quoted 6 lines · Avi Kivity
Quoted 2 lines

Why the gpa -> hva mapping is not
consistent for RAM if -mempath is not used?

Video RAM in the range a0000-bffff and PCI mapped RAM can change gpas (while remaining in the same hva).

Quoted 6 lines
Quoted 2 lines

Even for ordinary RAM, which doesn't normally change gpa/hva, I'm not sure we want to guarantee that it won't.

We can't universally either. Memory hot remove potentially breaks the mapping and some non-x86 architectures (like ARM) can alias RAM via a guest triggered mechanism.

Thanks for clarification. Now I think we have two options.

1) Clearly mark gpa2hva (pfa2hva now, should renamed to gpa2hva) as a testing only interface, and should be used only on restricted environment (normal memory, without hot-remove, maybe only on x86).

Quoted 6 lines

2) Find some way to lock the gpa -> hva mapping during operating. Such
as gpa2hva_begin and gpa2hva_end and lock the mapping in between.

I think 2) may be possible. But if there are no other users, why do that for a test case? So I think 1) may be the better option.

Quoted 5 lines

3) Move the poisoning code into qemu, so the command becomes

posison-address<addr>

(though physical addresses aren't stable either)

Poisoning includes:

a) gva ->  gpa
b) gpa ->  hva
c) hva ->  hpa

d) inject the MCE into host via some external tool

Quoted 4 lines
Quoted 2 lines

poison-address need to do b, c, d. Do you think it is good to do all these inside qemu?

If d is not too complicated (an ioctl?), we might to it in qemu.

The issue of d) is that there are multiple ways to inject MCE. Now one software based, one APEI based, and maybe some others in the future. They all use different interfaces. And as debug interface, there are not considered kernel ABI too (some are in debugfs). So I think it is better to use these ABI only in some test suite.

Best Regards,
Huang Ying

#32 lore raw
Quoted 5 lines

The issue of d) is that there are multiple ways to inject MCE. Now one software based, one APEI based, and maybe some others in the future. They all use different interfaces. And as debug interface, there are not considered kernel ABI too (some are in debugfs). So I think it is better to use these ABI only in some test suite.

In some cases the injection may be also through external hardware debugging mechanisms. So yes requiring that in qemu isn't a good idea.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only.
#33 lore raw
In reply to Avi Kivity
Quoted 11 lines · Avi Kivity
Quoted 8 lines · Anthony Liguori
Quoted 4 lines · Huang Ying

Yes. As general interface, it may not work so well, but as test interface, it works quite well and useful.

Do we have any mechanism to add a test only interface?

I'd like to see what Luiz/Markus think but definitely only a human monitor interface and probably prefix the command with a 'x-' prefix to indicate that it's not a supported interface.

Automated testing will want to use qmp.

The command is inherently racy so if you tried to implement an automated unit test, you'd get false

Quoted 5 lines
Quoted 2 lines

The documentation should be very clear about the limitations of the interface and the intended use-case.

Perhaps a { execute: 'enable-version-specific-commands', arguments: ['pfa2hva'] } ?