spapr/xive: fixes on EQ page addresses
cover spapr/xive: fixes on EQ page addresses
Hello
Here is a small series adding a check on the EQ page address alignment and fixing a severe issue when addresses are above 64GB.
Thanks,
C.
Cédric Le Goater (3):
spapr/xive: EQ page should be naturally aligned spapr/xive: fix EQ page addresses above 64GB spapr/xive: print out the EQ page address in the monitor
include/hw/ppc/xive_regs.h | 6 ++++++ hw/intc/spapr_xive.c | 14 ++++++++++---- hw/intc/xive.c | 9 +++------ 3 files changed, 19 insertions(+), 10 deletions(-)
Signature · 1 line
2.20.1
PATCH 1/3 spapr/xive: EQ page should be naturally aligned
When the OS configures the EQ page in which to receive event notifications from the XIVE interrupt controller, the page should be naturally aligned. Add this check.
--- hw/intc/spapr_xive.c | 6 ++++++ 1 file changed, 6 insertions(+)
hw/intc/spapr_xive.c +6 -0
@@ -993,6 +993,12 @@ static target_ulong h_int_set_queue_config(PowerPCCPU *cpu,case 16 :case 21 :case 24 : +if (!QEMU_IS_ALIGNED (qpage,1ul << qsize)) { +qemu_log_mask (LOG_GUEST_ERROR ,"XIVE: EQ @0x%" HWADDR_PRIx +" is not naturally aligned with %" HWADDR_PRIx"\n" , + qpage,1ul << qsize); +return H_P4 ; + } end.w2 =cpu_to_be32 ((qpage >>32 ) &0x0fffffff ); end.w3 =cpu_to_be32 (qpage &0xffffffff ); end.w0 |=cpu_to_be32 (END_W0_ENQUEUE );
Signature · 1 line
2.20.1
David Gibson
AppliedRe: [Qemu-devel] [PATCH 1/3] spapr/xive: EQ page should be naturally aligned
Quoted 5 lines · Cédric Le Goater
When the OS configures the EQ page in which to receive event notifications from the XIVE interrupt controller, the page should be naturally aligned. Add this check.
Signed-off-by Cédric Le Goater <[email protected]>
Applied, thanks.
Quoted 21 lines
--- hw/intc/spapr_xive.c | 6 ++++++ 1 file changed, 6 insertions(+)hw/intc/spapr_xive.c +6
-0@@ -993,6 +993,12 @@ static target_ulong h_int_set_queue_config(PowerPCCPU *cpu,case 16 :case 21 :case 24 : +if (!QEMU_IS_ALIGNED (qpage,1ul << qsize)) { +qemu_log_mask (LOG_GUEST_ERROR ,"XIVE: EQ @0x%" HWADDR_PRIx +" is not naturally aligned with %" HWADDR_PRIx"\n" , + qpage,1ul << qsize); +return H_P4 ; + } end.w2 =cpu_to_be32 ((qpage >>32 ) &0x0fffffff ); end.w3 =cpu_to_be32 (qpage &0xffffffff ); end.w0 |=cpu_to_be32 (END_W0_ENQUEUE );
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
Re: [Qemu-devel] [Qemu-ppc] [PATCH 1/3] spapr/xive: EQ page should be naturally aligned
In reply to Cédric Le GoaterQuoted 6 lines · On Wed, 8 May 2019 19:19:44 +0200 Cédric Le Goater <[email protected]> wrote:
When the OS configures the EQ page in which to receive event notifications from the XIVE interrupt controller, the page should be naturally aligned. Add this check.
Signed-off-by Cédric Le Goater <[email protected]>---
Quoted 20 lines
hw/intc/spapr_xive.c | 6 ++++++ 1 file changed, 6 insertions(+)hw/intc/spapr_xive.c +6
-0@@ -993,6 +993,12 @@ static target_ulong h_int_set_queue_config(PowerPCCPU *cpu,case 16 :case 21 :case 24 : +if (!QEMU_IS_ALIGNED (qpage,1ul << qsize)) { +qemu_log_mask (LOG_GUEST_ERROR ,"XIVE: EQ @0x%" HWADDR_PRIx +" is not naturally aligned with %" HWADDR_PRIx"\n" , + qpage,1ul << qsize); +return H_P4 ; + } end.w2 =cpu_to_be32 ((qpage >>32 ) &0x0fffffff ); end.w3 =cpu_to_be32 (qpage &0xffffffff ); end.w0 |=cpu_to_be32 (END_W0_ENQUEUE );
PATCH 2/3 spapr/xive: fix EQ page addresses above 64GB
In reply to Cédric Le GoaterThe high order bits of the address of the OS event queue is stored in bits [4-31] of word2 of the XIVE END internal structures and the low order bits in word3. This structure is using Big Endian ordering and computing the value requires some simple arithmetic which happens to be wrong. The mask removing bits [0-3] of word2 is applied to the wrong value and the resulting address is bogus when above 64GB.
Guests with more than 64GB of RAM will allocate pages for the OS event queues which will reside above the 64GB limit. In this case, the XIVE device model will wake up the CPUs in case of a notification, such as IPIs, but the update of the event queue will be written at the wrong place in memory. The result is uncertain as the guest memory is trashed and IPI are not delivered.
Introduce a helper xive_end_qaddr() to compute this value correctly in all places where it is used.
--- include/hw/ppc/xive_regs.h | 6 ++++++ hw/intc/spapr_xive.c | 3 +-- hw/intc/xive.c | 9 +++------ 3 files changed, 10 insertions(+), 8 deletions(-)
include/hw/ppc/xive_regs.h +6 -0
@@ -208,6 +208,12 @@ typedef struct XiveEND {#define xive_end_is_backlog (end) (be32_to_cpu((end)->w0) & END_W0_BACKLOG)#define xive_end_is_escalate (end) (be32_to_cpu((end)->w0) & END_W0_ESCALATE_CTL) +static inline uint64_t xive_end_qaddr (XiveEND *end) +{ +return ((uint64_t )be32_to_cpu (end->w2) &0x0fffffff ) <<32 | +be32_to_cpu (end->w3); +} +/* Notification Virtual Target (NVT) */ typedef struct XiveNVT { uint32_t w0;
hw/intc/spapr_xive.c +1 -2
@@ -1150,8 +1150,7 @@ static target_ulong h_int_get_queue_config(PowerPCCPU *cpu, }if (xive_end_is_enqueue (end)) { - args[1 ] = (uint64_t )be32_to_cpu (end->w2 &0x0fffffff ) <<32 - |be32_to_cpu (end->w3); + args[1 ] =xive_end_qaddr (end); args[2 ] =xive_get_field32 (END_W0_QSIZE , end->w0) +12 ; }else { args[1 ] =0 ;
hw/intc/xive.c +3 -6
@@ -1042,8 +1042,7 @@ static const TypeInfo xive_source_info = {void xive_end_queue_pic_print_info (XiveEND *end,uint32_t width,Monitor *mon) { -uint64_t qaddr_base = (uint64_t )be32_to_cpu (end->w2 &0x0fffffff ) <<32 - |be32_to_cpu (end->w3); +uint64_t qaddr_base =xive_end_qaddr (end);uint32_t qsize =xive_get_field32 (END_W0_QSIZE , end->w0);uint32_t qindex =xive_get_field32 (END_W1_PAGE_OFF , end->w1);uint32_t qentries =1 << (qsize +10 );
@@ -1072,8 +1071,7 @@ void xive_end_queue_pic_print_info(XiveEND *end, uint32_t width, Monitor *mon)void xive_end_pic_print_info (XiveEND *end,uint32_t end_idx,Monitor *mon) { -uint64_t qaddr_base = (uint64_t )be32_to_cpu (end->w2 &0x0fffffff ) <<32 - |be32_to_cpu (end->w3); +uint64_t qaddr_base =xive_end_qaddr (end);uint32_t qindex =xive_get_field32 (END_W1_PAGE_OFF , end->w1);uint32_t qgen =xive_get_field32 (END_W1_GENERATION , end->w1);uint32_t qsize =xive_get_field32 (END_W0_QSIZE , end->w0);
@@ -1101,8 +1099,7 @@ void xive_end_pic_print_info(XiveEND *end, uint32_t end_idx, Monitor *mon)static void xive_end_enqueue (XiveEND *end,uint32_t data) { -uint64_t qaddr_base = (uint64_t )be32_to_cpu (end->w2 &0x0fffffff ) <<32 - |be32_to_cpu (end->w3); +uint64_t qaddr_base =xive_end_qaddr (end);uint32_t qsize =xive_get_field32 (END_W0_QSIZE , end->w0);uint32_t qindex =xive_get_field32 (END_W1_PAGE_OFF , end->w1);uint32_t qgen =xive_get_field32 (END_W1_GENERATION , end->w1);
Signature · 1 line
2.20.1
David Gibson
AppliedRe: [Qemu-devel] [PATCH 2/3] spapr/xive: fix EQ page addresses above 64GB
Quoted 14 lines · Cédric Le Goater
The high order bits of the address of the OS event queue is stored in bits [4-31] of word2 of the XIVE END internal structures and the low order bits in word3. This structure is using Big Endian ordering and computing the value requires some simple arithmetic which happens to be wrong. The mask removing bits [0-3] of word2 is applied to the wrong value and the resulting address is bogus when above 64GB.
Guests with more than 64GB of RAM will allocate pages for the OS event queues which will reside above the 64GB limit. In this case, the XIVE device model will wake up the CPUs in case of a notification, such as IPIs, but the update of the event queue will be written at the wrong place in memory. The result is uncertain as the guest memory is trashed and IPI are not delivered.
Quoted 4 lines
Introduce a helper
xive_end_qaddr()to compute this value correctly in all places where it is used.Signed-off-by Cédric Le Goater <[email protected]>
Applied, thanks.
Quoted 71 lines
--- include/hw/ppc/xive_regs.h | 6 ++++++ hw/intc/spapr_xive.c | 3 +-- hw/intc/xive.c | 9 +++------ 3 files changed, 10 insertions(+), 8 deletions(-)include/hw/ppc/xive_regs.h +6
-0@@ -208,6 +208,12 @@ typedef struct XiveEND {#define xive_end_is_backlog (end) (be32_to_cpu((end)->w0) & END_W0_BACKLOG)#define xive_end_is_escalate (end) (be32_to_cpu((end)->w0) & END_W0_ESCALATE_CTL) +static inline uint64_t xive_end_qaddr (XiveEND *end) +{ +return ((uint64_t )be32_to_cpu (end->w2) &0x0fffffff ) <<32 | +be32_to_cpu (end->w3); +} +/* Notification Virtual Target (NVT) */ typedef struct XiveNVT { uint32_t w0;hw/intc/spapr_xive.c +1
-2@@ -1150,8 +1150,7 @@ static target_ulong h_int_get_queue_config(PowerPCCPU *cpu, }if (xive_end_is_enqueue (end)) { - args[1 ] = (uint64_t )be32_to_cpu (end->w2 &0x0fffffff ) <<32 - |be32_to_cpu (end->w3); + args[1 ] =xive_end_qaddr (end); args[2 ] =xive_get_field32 (END_W0_QSIZE , end->w0) +12 ; }else { args[1 ] =0 ;hw/intc/xive.c +3
-6@@ -1042,8 +1042,7 @@ static const TypeInfo xive_source_info = {void xive_end_queue_pic_print_info (XiveEND *end,uint32_t width,Monitor *mon) { -uint64_t qaddr_base = (uint64_t )be32_to_cpu (end->w2 &0x0fffffff ) <<32 - |be32_to_cpu (end->w3); +uint64_t qaddr_base =xive_end_qaddr (end);uint32_t qsize =xive_get_field32 (END_W0_QSIZE , end->w0);uint32_t qindex =xive_get_field32 (END_W1_PAGE_OFF , end->w1);uint32_t qentries =1 << (qsize +10 );@@ -1072,8 +1071,7 @@ void xive_end_queue_pic_print_info(XiveEND *end, uint32_t width, Monitor *mon)void xive_end_pic_print_info (XiveEND *end,uint32_t end_idx,Monitor *mon) { -uint64_t qaddr_base = (uint64_t )be32_to_cpu (end->w2 &0x0fffffff ) <<32 - |be32_to_cpu (end->w3); +uint64_t qaddr_base =xive_end_qaddr (end);uint32_t qindex =xive_get_field32 (END_W1_PAGE_OFF , end->w1);uint32_t qgen =xive_get_field32 (END_W1_GENERATION , end->w1);uint32_t qsize =xive_get_field32 (END_W0_QSIZE , end->w0);@@ -1101,8 +1099,7 @@ void xive_end_pic_print_info(XiveEND *end, uint32_t end_idx, Monitor *mon)static void xive_end_enqueue (XiveEND *end,uint32_t data) { -uint64_t qaddr_base = (uint64_t )be32_to_cpu (end->w2 &0x0fffffff ) <<32 - |be32_to_cpu (end->w3); +uint64_t qaddr_base =xive_end_qaddr (end);uint32_t qsize =xive_get_field32 (END_W0_QSIZE , end->w0);uint32_t qindex =xive_get_field32 (END_W1_PAGE_OFF , end->w1);uint32_t qgen =xive_get_field32 (END_W1_GENERATION , end->w1);
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
Re: [Qemu-devel] [Qemu-ppc] [PATCH 2/3] spapr/xive: fix EQ page addresses above 64GB
In reply to Cédric Le GoaterQuoted 14 lines · On Wed, 8 May 2019 19:19:45 +0200 Cédric Le Goater <[email protected]> wrote:
The high order bits of the address of the OS event queue is stored in bits [4-31] of word2 of the XIVE END internal structures and the low order bits in word3. This structure is using Big Endian ordering and computing the value requires some simple arithmetic which happens to be wrong. The mask removing bits [0-3] of word2 is applied to the wrong value and the resulting address is bogus when above 64GB.
Guests with more than 64GB of RAM will allocate pages for the OS event queues which will reside above the 64GB limit. In this case, the XIVE device model will wake up the CPUs in case of a notification, such as IPIs, but the update of the event queue will be written at the wrong place in memory. The result is uncertain as the guest memory is trashed and IPI are not delivered.
Quoted 5 lines
Introduce a helper
xive_end_qaddr()to compute this value correctly in all places where it is used.Signed-off-by Cédric Le Goater <[email protected]>---
I guess this patch should have a Fixes: tag and Cc qemu-stable as well since QEMU 4.0 has the issue.
Apart from that, LGTM.
Quoted 70 lines
include/hw/ppc/xive_regs.h | 6 ++++++ hw/intc/spapr_xive.c | 3 +-- hw/intc/xive.c | 9 +++------ 3 files changed, 10 insertions(+), 8 deletions(-)include/hw/ppc/xive_regs.h +6
-0@@ -208,6 +208,12 @@ typedef struct XiveEND {#define xive_end_is_backlog (end) (be32_to_cpu((end)->w0) & END_W0_BACKLOG)#define xive_end_is_escalate (end) (be32_to_cpu((end)->w0) & END_W0_ESCALATE_CTL) +static inline uint64_t xive_end_qaddr (XiveEND *end) +{ +return ((uint64_t )be32_to_cpu (end->w2) &0x0fffffff ) <<32 | +be32_to_cpu (end->w3); +} +/* Notification Virtual Target (NVT) */ typedef struct XiveNVT { uint32_t w0;hw/intc/spapr_xive.c +1
-2@@ -1150,8 +1150,7 @@ static target_ulong h_int_get_queue_config(PowerPCCPU *cpu, }if (xive_end_is_enqueue (end)) { - args[1 ] = (uint64_t )be32_to_cpu (end->w2 &0x0fffffff ) <<32 - |be32_to_cpu (end->w3); + args[1 ] =xive_end_qaddr (end); args[2 ] =xive_get_field32 (END_W0_QSIZE , end->w0) +12 ; }else { args[1 ] =0 ;hw/intc/xive.c +3
-6@@ -1042,8 +1042,7 @@ static const TypeInfo xive_source_info = {void xive_end_queue_pic_print_info (XiveEND *end,uint32_t width,Monitor *mon) { -uint64_t qaddr_base = (uint64_t )be32_to_cpu (end->w2 &0x0fffffff ) <<32 - |be32_to_cpu (end->w3); +uint64_t qaddr_base =xive_end_qaddr (end);uint32_t qsize =xive_get_field32 (END_W0_QSIZE , end->w0);uint32_t qindex =xive_get_field32 (END_W1_PAGE_OFF , end->w1);uint32_t qentries =1 << (qsize +10 );@@ -1072,8 +1071,7 @@ void xive_end_queue_pic_print_info(XiveEND *end, uint32_t width, Monitor *mon)void xive_end_pic_print_info (XiveEND *end,uint32_t end_idx,Monitor *mon) { -uint64_t qaddr_base = (uint64_t )be32_to_cpu (end->w2 &0x0fffffff ) <<32 - |be32_to_cpu (end->w3); +uint64_t qaddr_base =xive_end_qaddr (end);uint32_t qindex =xive_get_field32 (END_W1_PAGE_OFF , end->w1);uint32_t qgen =xive_get_field32 (END_W1_GENERATION , end->w1);uint32_t qsize =xive_get_field32 (END_W0_QSIZE , end->w0);@@ -1101,8 +1099,7 @@ void xive_end_pic_print_info(XiveEND *end, uint32_t end_idx, Monitor *mon)static void xive_end_enqueue (XiveEND *end,uint32_t data) { -uint64_t qaddr_base = (uint64_t )be32_to_cpu (end->w2 &0x0fffffff ) <<32 - |be32_to_cpu (end->w3); +uint64_t qaddr_base =xive_end_qaddr (end);uint32_t qsize =xive_get_field32 (END_W0_QSIZE , end->w0);uint32_t qindex =xive_get_field32 (END_W1_PAGE_OFF , end->w1);uint32_t qgen =xive_get_field32 (END_W1_GENERATION , end->w1);
PATCH 3/3 spapr/xive: print out the EQ page address in the monitor
In reply to Cédric Le GoaterThis proved to be a useful information when debugging issues with OS event queues allocated above 64GB.
--- hw/intc/spapr_xive.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
hw/intc/spapr_xive.c +3 -2
@@ -120,6 +120,7 @@ static int spapr_xive_target_to_end(uint32_t target, uint8_t prio,static void spapr_xive_end_pic_print_info (SpaprXive *xive,XiveEND *end,Monitor *mon) { +uint64_t qaddr_base =xive_end_qaddr (end);uint32_t qindex =xive_get_field32 (END_W1_PAGE_OFF , end->w1);uint32_t qgen =xive_get_field32 (END_W1_GENERATION , end->w1);uint32_t qsize =xive_get_field32 (END_W0_QSIZE , end->w0);
@@ -127,9 +128,9 @@ static void spapr_xive_end_pic_print_info(SpaprXive *xive, XiveEND *end,uint32_t nvt =xive_get_field32 (END_W6_NVT_INDEX , end->w6);uint8_t priority =xive_get_field32 (END_W7_F0_PRIORITY , end->w7); -monitor_printf (mon,"%3d/%d % 6d/%5d ^%d" , +monitor_printf (mon,"%3d/%d % 6d/%5d @%" PRIx64" ^%d" ,spapr_xive_nvt_to_target (0 , nvt), - priority, qindex, qentries, qgen); + priority, qindex, qentries, qaddr_base, qgen);xive_end_queue_pic_print_info (end,6 , mon);monitor_printf (mon,"]" );
Signature · 1 line
2.20.1
David Gibson
AppliedRe: [Qemu-devel] [PATCH 3/3] spapr/xive: print out the EQ page address in the monitor
Quoted 4 lines · Cédric Le Goater
This proved to be a useful information when debugging issues with OS event queues allocated above 64GB.
Signed-off-by Cédric Le Goater <[email protected]>
Applied, thanks.
Quoted 28 lines
--- hw/intc/spapr_xive.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)hw/intc/spapr_xive.c +3
-2@@ -120,6 +120,7 @@ static int spapr_xive_target_to_end(uint32_t target, uint8_t prio,static void spapr_xive_end_pic_print_info (SpaprXive *xive,XiveEND *end,Monitor *mon) { +uint64_t qaddr_base =xive_end_qaddr (end);uint32_t qindex =xive_get_field32 (END_W1_PAGE_OFF , end->w1);uint32_t qgen =xive_get_field32 (END_W1_GENERATION , end->w1);uint32_t qsize =xive_get_field32 (END_W0_QSIZE , end->w0);@@ -127,9 +128,9 @@ static void spapr_xive_end_pic_print_info(SpaprXive *xive, XiveEND *end,uint32_t nvt =xive_get_field32 (END_W6_NVT_INDEX , end->w6);uint8_t priority =xive_get_field32 (END_W7_F0_PRIORITY , end->w7); -monitor_printf (mon,"%3d/%d % 6d/%5d ^%d" , +monitor_printf (mon,"%3d/%d % 6d/%5d @%" PRIx64" ^%d" ,spapr_xive_nvt_to_target (0 , nvt), - priority, qindex, qentries, qgen); + priority, qindex, qentries, qaddr_base, qgen);xive_end_queue_pic_print_info (end,6 , mon);monitor_printf (mon,"]" );
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
Re: [Qemu-devel] [Qemu-ppc] [PATCH 3/3] spapr/xive: print out the EQ page address in the monitor
In reply to Cédric Le GoaterQuoted 5 lines · On Wed, 8 May 2019 19:19:46 +0200 Cédric Le Goater <[email protected]> wrote:
This proved to be a useful information when debugging issues with OS event queues allocated above 64GB.
Signed-off-by Cédric Le Goater <[email protected]>---
Quoted 27 lines
hw/intc/spapr_xive.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)hw/intc/spapr_xive.c +3
-2@@ -120,6 +120,7 @@ static int spapr_xive_target_to_end(uint32_t target, uint8_t prio,static void spapr_xive_end_pic_print_info (SpaprXive *xive,XiveEND *end,Monitor *mon) { +uint64_t qaddr_base =xive_end_qaddr (end);uint32_t qindex =xive_get_field32 (END_W1_PAGE_OFF , end->w1);uint32_t qgen =xive_get_field32 (END_W1_GENERATION , end->w1);uint32_t qsize =xive_get_field32 (END_W0_QSIZE , end->w0);@@ -127,9 +128,9 @@ static void spapr_xive_end_pic_print_info(SpaprXive *xive, XiveEND *end,uint32_t nvt =xive_get_field32 (END_W6_NVT_INDEX , end->w6);uint8_t priority =xive_get_field32 (END_W7_F0_PRIORITY , end->w7); -monitor_printf (mon,"%3d/%d % 6d/%5d ^%d" , +monitor_printf (mon,"%3d/%d % 6d/%5d @%" PRIx64" ^%d" ,spapr_xive_nvt_to_target (0 , nvt), - priority, qindex, qentries, qgen); + priority, qindex, qentries, qaddr_base, qgen);xive_end_queue_pic_print_info (end,6 , mon);monitor_printf (mon,"]" );