s390x/tcg: mem_helper: Fault-safe handling
v2 cover s390x/tcg: mem_helper: Fault-safe handling
This is the successor of
"[PATCH v1 0/4] s390x/tcg: MOVE (MVC): Fault-safe handling"
----
This series fixes a bunch of issues related to some mem helpers and makes sure that they are fault-safe, meaning no system state is modified in case a fault is triggered.
I can spot tons of other issues with other mem helpers that will have to be fixed later. Also, fault-safe handling for some instructions (especially TR) might be harder to implement (you don't know what will actually be accessed upfront - we might need a buffer and go over inputs twice). Focusing on the MOVE instructions for now.
----
Newer versions of glibc use memcpy() in memmove() for forward moves. The implementation makese use of MVC. The TCG implementation of MVC is currently not able to handle faults reliably when crossing pages. MVC can cross with 256 bytes at most two pages.
In case we get a fault on the second page, we already moved data. When continuing after the fault we might try to move already overwritten data, which is very bad in case we have overlapping data on a forward move.
Triggered for now only by rpmbuild (crashes when checking the spec file) and rpm (database corruptions). This fixes installing Fedora rawhide (31) under TCG.
This was horrible to debug as it barely triggers and we fail at completely different places.
v1 -> v2:
- Include many fixes
- Fix more instructions
- Use the new probe_access() function
- Include "tests/tcg: target/s390x: Test MVO"
David Hildenbrand (28):
s390x/tcg: Reset exception_index to -1 instead of 0 s390x/tcg: MVCL: Zero out unused bits of address s390x/tcg: MVCL: Detect destructive overlaps s390x/tcg: MVCL: Process max 2k bytes at a time s390x/tcg: MVC: Increment the length once s390x/tcg: MVC: Use is_destructive_overlap() s390x/tcg: MVPG: Check for specification exceptions s390x/tcg: MVPG: Properly wrap the addresses s390x/tcg: MVCLU/MVCLE: Process max 4k bytes at a time s390x/tcg: MVCS/MVCP: Check for special operation exceptions s390x/tcg: MVCS/MVCP: Properly wrap the length s390x/tcg: MVST: Check for specification exceptions s390x/tcg: MVST: Fix storing back the addresses to registers s390x/tcg: Always use MMU_USER_IDX for CONFIG_USER_ONLY s390x/tcg: Fault-safe memset
s390x/tcg: Fault-safe memmove
s390x/tcg: MVCS/MVCP: Use access_memmove_idx() s390x/tcg: MVC: Fault-safe handling on destructive overlaps s390x/tcg: MVCLU: Fault-safe handling
s390x/tcg: OC: Fault-safe handling
s390x/tcg: XC: Fault-safe handling
s390x/tcg: NC: Fault-safe handling
s390x/tcg: MVCIN: Fault-safe handling
s390x/tcg: MVN: Fault-safe handling
s390x/tcg: MVZ: Fault-safe handling
s390x/tcg: MVST: Fault-safe handling
s390x/tcg: MVO: Fault-safe handling
tests/tcg: target/s390x: Test MVO
target/s390x/cpu.h | 4 + target/s390x/helper.h | 2 +- target/s390x/insn-data.def | 2 +- target/s390x/mem_helper.c | 672 ++++++++++++++++++++++---------- target/s390x/translate.c | 12 +- tests/tcg/s390x/Makefile.target | 1 + tests/tcg/s390x/mvo.c | 25 ++ 7 files changed, 507 insertions(+), 211 deletions(-) create mode 100644 tests/tcg/s390x/mvo.c
Signature · 1 line
2.21.0
PATCH v2 1/28 s390x/tcg: Reset exception_index to -1 instead of 0
We use the marker "-1" for "no exception". s390_cpu_do_interrupt() might get confused by that.
--- target/s390x/mem_helper.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
target/s390x/mem_helper.c +3 -3
@@ -1747,7 +1747,7 @@ uint32_t HELPER(tprot)(CPUS390XState *env, uint64_t a1, uint64_t a2)if (env->int_pgm_code ==PGM_PROTECTION ) {/* retry if reading is possible */ - cs->exception_index =0 ; + cs->exception_index =-1 ;if (!s390_cpu_virt_mem_check_read (cpu, a1,0 ,1 )) {/* Fetching permitted; storing not permitted */ return 1 ;
@@ -1757,7 +1757,7 @@ uint32_t HELPER(tprot)(CPUS390XState *env, uint64_t a1, uint64_t a2)switch (env->int_pgm_code) {case PGM_PROTECTION :/* Fetching not permitted; storing not permitted */ - cs->exception_index =0 ; + cs->exception_index =-1 ;return 2 ;case PGM_ADDRESSING :case PGM_TRANS_SPEC :
@@ -1767,7 +1767,7 @@ uint32_t HELPER(tprot)(CPUS390XState *env, uint64_t a1, uint64_t a2) }/* Translation not available */ - cs->exception_index =0 ; + cs->exception_index =-1 ;return 3 ; }
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 01/28] s390x/tcg: Reset exception_index to -1 instead of 0
Quoted 7 lines · David Hildenbrand
We use the marker "-1" for "no exception".
s390_cpu_do_interrupt()might get confused by that.Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/mem_helper.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
r~
PATCH v2 2/28 s390x/tcg: MVCL: Zero out unused bits of address
In reply to David HildenbrandWe have to zero out unused bits in 24 and 31-bit addressing mode. Provide a new helper.
--- target/s390x/mem_helper.c | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-)
target/s390x/mem_helper.c +22 -2
@@ -469,6 +469,26 @@ static inline uint64_t get_address(CPUS390XState *env, int reg)return wrap_address (env, env->regs[reg]); } +/* +* Store the address to the given register, zeroing out unused leftmost +* bits in bit positions 32-63 (24-bit and 31-bit mode only). +*/ +static inline void set_address_zero (CPUS390XState *env,int reg, +uint64_t address) +{ +if (env->psw.mask &PSW_MASK_64 ) { + env->regs[reg] = address; + }else { +if (!(env->psw.mask &PSW_MASK_32 )) { + address &=0x00ffffff ; + env->regs[reg] =deposit64 (env->regs[reg],0 ,32 , address); + }else { + address &=0x7fffffff ; + env->regs[reg] =deposit64 (env->regs[reg],0 ,32 , address); + } + } +} +static inline void set_address (CPUS390XState *env,int reg,uint64_t address) {if (env->psw.mask &PSW_MASK_64 ) {
@@ -772,8 +792,8 @@ uint32_t HELPER(mvcl)(CPUS390XState *env, uint32_t r1, uint32_t r2) env->regs[r1 +1 ] =deposit64 (env->regs[r1 +1 ],0 ,24 , destlen); env->regs[r2 +1 ] =deposit64 (env->regs[r2 +1 ],0 ,24 , srclen); -set_address (env, r1, dest); -set_address (env, r2, src); +set_address_zero (env, r1, dest); +set_address_zero (env, r2, src);return cc; }
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 02/28] s390x/tcg: MVCL: Zero out unused bits of address
Quoted 7 lines · David Hildenbrand
We have to zero out unused bits in 24 and 31-bit addressing mode. Provide a new helper.
Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/mem_helper.c | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-)
Quoted 15 lines
target/s390x/mem_helper.c +7
-0@@ -469,6 +469,26 @@ static inline uint64_t get_address(CPUS390XState *env, int reg)return wrap_address (env, env->regs[reg]); } +/* +* Store the address to the given register, zeroing out unused leftmost +* bits in bit positions 32-63 (24-bit and 31-bit mode only). +*/ +static inline void set_address_zero (CPUS390XState *env,int reg, +uint64_t address) +{
target/s390x/mem_helper.c static inline uint64_t get_address(CPUS390XState *env, int reg) · line 488
Quoted 10 lines
target/s390x/mem_helper.c +10
-0+if (env->psw.mask &PSW_MASK_64 ) { + env->regs[reg] = address; + }else { +if (!(env->psw.mask &PSW_MASK_32 )) { + address &=0x00ffffff ; + env->regs[reg] =deposit64 (env->regs[reg],0 ,32 , address); + }else { + address &=0x7fffffff ; + env->regs[reg] =deposit64 (env->regs[reg],0 ,32 , address); + }
You could perhaps sink the deposit and store line into the outer else.
r~
Re: [Qemu-devel] [PATCH v2 02/28] s390x/tcg: MVCL: Zero out unused bits of address
Quoted 31 lines · Richard Henderson
Quoted 7 lines · David Hildenbrand
We have to zero out unused bits in 24 and 31-bit addressing mode. Provide a new helper.
Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/mem_helper.c | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-)Reviewed-by Richard Henderson <[email protected]>Quoted 19 lines
target/s390x/mem_helper.c +11
-0@@ -469,6 +469,26 @@ static inline uint64_t get_address(CPUS390XState *env, int reg)return wrap_address (env, env->regs[reg]); } +/* +* Store the address to the given register, zeroing out unused leftmost +* bits in bit positions 32-63 (24-bit and 31-bit mode only). +*/ +static inline void set_address_zero (CPUS390XState *env,int reg, +uint64_t address) +{ +if (env->psw.mask &PSW_MASK_64 ) { + env->regs[reg] = address; + }else { +if (!(env->psw.mask &PSW_MASK_32 )) {
Quoted 9 lines
target/s390x/mem_helper.c static inline uint64_t get_address(CPUS390XState *env, int reg) · line 488
Quoted 6 lines
target/s390x/mem_helper.c +6
-0+ address &=0x00ffffff ; + env->regs[reg] =deposit64 (env->regs[reg],0 ,32 , address); + }else { + address &=0x7fffffff ; + env->regs[reg] =deposit64 (env->regs[reg],0 ,32 , address); + }You could perhaps sink the deposit and store line into the outer else.
Thanks, will do!
Signature · 3 lines
Thanks, David / dhildenb
PATCH v2 3/28 s390x/tcg: MVCL: Detect destructive overlaps
In reply to David HildenbrandWe'll have to zero-out unused bit positions, so amke sure to write the addresses back.
--- target/s390x/mem_helper.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-)
target/s390x/mem_helper.c +18 -1
@@ -52,6 +52,19 @@ static inline bool psw_key_valid(CPUS390XState *env, uint8_t psw_key)return true; } +static bool is_destructive_overlap (CPUS390XState *env,uint64_t dest, +uint64_t src,uint32_t len) +{ +if (!len || src == dest) { +return false; + } +/* Take care of wrapping at the end of address space. */ +if (unlikely (wrap_address (env, src + len -1 ) < src)) { +return dest > src || dest <=wrap_address (env, src + len -1 ); + } +return dest > src && dest <= src + len -1 ; +} +/* Reduce the length so that addr + len doesn't cross a page boundary. */ static inline uint32_t adj_len_to_page (uint32_t len,uint64_t addr) {
@@ -788,7 +801,11 @@ uint32_t HELPER(mvcl)(CPUS390XState *env, uint32_t r1, uint32_t r2)uint8_t pad = env->regs[r2 +1 ] >>24 ;uint32_t cc; - cc =do_mvcl (env, &dest, &destlen, &src, &srclen, pad,1 , ra); +if (is_destructive_overlap (env, dest, src,MIN (srclen, destlen))) { + cc =3 ; + }else { + cc =do_mvcl (env, &dest, &destlen, &src, &srclen, pad,1 , ra); + } env->regs[r1 +1 ] =deposit64 (env->regs[r1 +1 ],0 ,24 , destlen); env->regs[r2 +1 ] =deposit64 (env->regs[r2 +1 ],0 ,24 , srclen);
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 03/28] s390x/tcg: MVCL: Detect destructive overlaps
Quoted 8 lines · David Hildenbrand
We'll have to zero-out unused bit positions, so amke sure to write the addresses back.
Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/mem_helper.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-)
r~
PATCH v2 4/28 s390x/tcg: MVCL: Process max 2k bytes at a time
In reply to David HildenbrandProcess max 2k bytes at a time, writing back registers between the accesses. The instruction is interruptible.
"For operands longer than 2K bytes, access exceptions are not recognized for locations more than 2K bytes beyond the current location being processed."
MVCL handling is quite different than MVCLE/MVCLU handling, so split up the handlers.
We'll deal with fast_memmove() and fast_memset() not probing reads/writes properly later. Also, we'll defer interrupt handling, as that will require more thought, add a TODO for that.
--- target/s390x/mem_helper.c | 44 +++++++++++++++++++++++++++++++++------ 1 file changed, 38 insertions(+), 6 deletions(-)
target/s390x/mem_helper.c +38 -6
@@ -799,19 +799,51 @@ uint32_t HELPER(mvcl)(CPUS390XState *env, uint32_t r1, uint32_t r2)uint64_t srclen = env->regs[r2 +1 ] &0xffffff ;uint64_t src =get_address (env, r2);uint8_t pad = env->regs[r2 +1 ] >>24 ; -uint32_t cc; +uint32_t cc, cur_len;if (is_destructive_overlap (env, dest, src,MIN (srclen, destlen))) { cc =3 ; + }else if (srclen == destlen) { + cc =0 ; + }else if (destlen < srclen) { + cc =1 ; }else { - cc =do_mvcl (env, &dest, &destlen, &src, &srclen, pad,1 , ra); + cc =2 ; } - env->regs[r1 +1 ] =deposit64 (env->regs[r1 +1 ],0 ,24 , destlen); - env->regs[r2 +1 ] =deposit64 (env->regs[r2 +1 ],0 ,24 , srclen); -set_address_zero (env, r1, dest); -set_address_zero (env, r2, src); +/* We might have to zero-out some bits even if there was no action. */ +if (unlikely (!destlen || cc ==3 )) { +set_address_zero (env, r2, src); +set_address_zero (env, r1, dest); +return cc; + }else if (!srclen) { +set_address_zero (env, r2, src); + } +/* +* Only perform one type of type of operation (move/pad) in one step. +* Process up to 2k bytes. +*/ +while (destlen) { + cur_len =MIN (destlen,2048 ); +if (!srclen) { +fast_memset (env, dest, pad, cur_len, ra); + }else { + cur_len =MIN (cur_len, srclen); + +fast_memmove (env, dest, src, cur_len, ra); + src =wrap_address (env, src + cur_len); + srclen -= cur_len; + env->regs[r2 +1 ] =deposit64 (env->regs[r2 +1 ],0 ,24 , srclen); +set_address_zero (env, r2, src); + } + dest =wrap_address (env, dest + cur_len); + destlen -= cur_len; + env->regs[r1 +1 ] =deposit64 (env->regs[r1 +1 ],0 ,24 , destlen); +set_address_zero (env, r1, dest); + +/* TODO: Deliver interrupts. */ + }return cc; }
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 04/28] s390x/tcg: MVCL: Process max 2k bytes at a time
Quoted 49 lines · David Hildenbrand
Process max 2k bytes at a time, writing back registers between the accesses. The instruction is interruptible.
"For operands longer than 2K bytes, access exceptions are not recognized for locations more than 2K bytes beyond the current location being processed."MVCL handling is quite different than MVCLE/MVCLU handling, so split up the handlers.
We'll deal with
fast_memmove()andfast_memset()not probing reads/writes properly later. Also, we'll defer interrupt handling, as that will require more thought, add a TODO for that.Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/mem_helper.c | 44 +++++++++++++++++++++++++++++++++------ 1 file changed, 38 insertions(+), 6 deletions(-)target/s390x/mem_helper.c +11
-6@@ -799,19 +799,51 @@ uint32_t HELPER(mvcl)(CPUS390XState *env, uint32_t r1, uint32_t r2)uint64_t srclen = env->regs[r2 +1 ] &0xffffff ;uint64_t src =get_address (env, r2);uint8_t pad = env->regs[r2 +1 ] >>24 ; -uint32_t cc; +uint32_t cc, cur_len;if (is_destructive_overlap (env, dest, src,MIN (srclen, destlen))) { cc =3 ; + }else if (srclen == destlen) { + cc =0 ; + }else if (destlen < srclen) { + cc =1 ; }else { - cc =do_mvcl (env, &dest, &destlen, &src, &srclen, pad,1 , ra); + cc =2 ; } - env->regs[r1 +1 ] =deposit64 (env->regs[r1 +1 ],0 ,24 , destlen); - env->regs[r2 +1 ] =deposit64 (env->regs[r2 +1 ],0 ,24 , srclen); -set_address_zero (env, r1, dest); -set_address_zero (env, r2, src); +/* We might have to zero-out some bits even if there was no action. */ +if (unlikely (!destlen || cc ==3 )) { +set_address_zero (env, r2, src); +set_address_zero (env, r1, dest); +return cc;
target/s390x/mem_helper.c uint32_t HELPER(mvcl)(CPUS390XState *env, uint32_t r1, uint32_t r2) · line 828
Quoted 10 lines
target/s390x/mem_helper.c +9
-0+ }else if (!srclen) { +set_address_zero (env, r2, src); + } +/* +* Only perform one type of type of operation (move/pad) in one step. +* Process up to 2k bytes. +*/ +while (destlen) { + cur_len =MIN (destlen,2048 );
The language in the PoP is horribly written, and thus confusing. I can't believe it really means what it appears to say, about access exceptions not being recognized.
The code within Hercules breaks the action at every 2k address boundary -- for both src and dest. That's the only way that actually makes sense to me, as otherwise we end up allowing userspace to read/write into a page without permission. Which is a security hole.
r~
Re: [Qemu-devel] [PATCH v2 04/28] s390x/tcg: MVCL: Process max 2k bytes at a time
Quoted 4 lines · Richard Henderson
The code within Hercules breaks the action at every 2k address boundary -- for both src and dest. That's the only way that actually makes sense to me, as otherwise we end up allowing userspace to read/write into a page without permission. Which is a security hole.
Also, doesn't "2k" come from the old esa/360 page size?
Which means that we could break at 4k pages instead of 2k now and the program wouldn't really be able to tell the difference.
r~
Re: [Qemu-devel] [PATCH v2 04/28] s390x/tcg: MVCL: Process max 2k bytes at a time
Quoted 7 lines · Richard Henderson
Quoted 4 lines · Richard Henderson
The code within Hercules breaks the action at every 2k address boundary -- for both src and dest. That's the only way that actually makes sense to me, as otherwise we end up allowing userspace to read/write into a page without permission. Which is a security hole.
Also, doesn't "2k" come from the old esa/360 page size?
I have no idea, I was very confused with that.
Quoted 3 lines
Which means that we could break at 4k pages instead of 2k now and the program wouldn't really be able to tell the difference.
What I had in a previous iteration was to simply process until the end of the page(s), to not cross pages (there is one special case with 2k vs. 4k when crossing pages when wrapping and running into a low-address-protection). So essentially what you suggest. I can add that.
Signature · 3 lines
Thanks, David / dhildenb
PATCH v2 5/28 s390x/tcg: MVC: Increment the length once
In reply to David HildenbrandLet's increment the length once.
While at it, cleanup the comment. The memset() example is given as a programming note in the PoP, so drop the description.
--- target/s390x/mem_helper.c | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-)
target/s390x/mem_helper.c +12 -8
@@ -320,16 +320,20 @@ static uint32_t do_helper_mvc(CPUS390XState *env, uint32_t l, uint64_t dest,HELPER_LOG ("%s l %d dest %" PRIx64" src %" PRIx64"\n" , __func__, l, dest, src); -/* mvc and memmove do not behave the same when areas overlap! */ -/* mvc with source pointing to the byte after the destination is the -same as memset with the first source byte */ +/* MVC always copies one more byte than specified - maximum is 256 */ + l++; + +/* +* "When the operands overlap, the result is obtained as if the operands +* were processed one byte at a time". Only non-destructive overlaps +* behave like memmove(). +*/ if (dest == src +1 ) { -fast_memset (env, dest,cpu_ldub_data_ra (env, src, ra), l +1 , ra); - }else if (dest < src || src + l < dest) { -fast_memmove (env, dest, src, l +1 , ra); +fast_memset (env, dest,cpu_ldub_data_ra (env, src, ra), l, ra); + }else if (dest < src || src + l <= dest) { +fast_memmove (env, dest, src, l, ra); }else { -/* slow version with byte accesses which always work */ -for (i =0 ; i <= l; i++) { +for (i =0 ; i < l; i++) {uint8_t x =cpu_ldub_data_ra (env, src + i, ra);cpu_stb_data_ra (env, dest + i, x, ra); }
Signature · 1 line
2.21.0
PATCH v2 6/28 s390x/tcg: MVC: Use is_destructive_overlap()
In reply to David HildenbrandLet's use the new helper, that also detects destructive overlaps when wrapping.
--- target/s390x/mem_helper.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
target/s390x/mem_helper.c +1 -1
@@ -330,7 +330,7 @@ static uint32_t do_helper_mvc(CPUS390XState *env, uint32_t l, uint64_t dest,*/ if (dest == src +1 ) {fast_memset (env, dest,cpu_ldub_data_ra (env, src, ra), l, ra); - }else if (dest < src || src + l <= dest) { + }else if (!is_destructive_overlap (env, dest, src, l)) {fast_memmove (env, dest, src, l, ra); }else {for (i =0 ; i < l; i++) {
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 06/28] s390x/tcg: MVC: Use is_destructive_overlap()
target/s390x/mem_helper.c static uint32_t do_helper_mvc(CPUS390XState *env, uint32_t l, uint64_t dest, · line 336
Quoted 7 lines · David Hildenbrand
if (dest == src + 1) { fast_memset(env, dest, cpu_ldub_data_ra(env, src, ra), l, ra);target/s390x/mem_helper.c +1
-1- }else if (dest < src || src + l <= dest) { + }else if (!is_destructive_overlap (env, dest, src, l)) {fast_memmove (env, dest, src, l, ra); }else {for (i =0 ; i < l; i++) {
I suppose, though last time I checked fast_memmove didn't support wrapping.
r~
Re: [Qemu-devel] [PATCH v2 06/28] s390x/tcg: MVC: Use is_destructive_overlap()
Quoted 10 lines · Richard Henderson
target/s390x/mem_helper.c static uint32_t do_helper_mvc(CPUS390XState *env, uint32_t l, uint64_t dest, · line 336
Quoted 7 lines · David Hildenbrand
if (dest == src + 1) { fast_memset(env, dest, cpu_ldub_data_ra(env, src, ra), l, ra);target/s390x/mem_helper.c +1
-1- }else if (dest < src || src + l <= dest) { + }else if (!is_destructive_overlap (env, dest, src, l)) {fast_memmove (env, dest, src, l, ra); }else {for (i =0 ; i < l; i++) {I suppose, though last time I checked
fast_memmovedidn't support wrapping.
Yes, that's fixed by access_prepare(), access_memmove(). Wrapping is/was broken in most mem handlers ...
Quoted 5 lines
Reviewed-by Richard Henderson <[email protected]>r~
Signature · 3 lines
Thanks, David / dhildenb
PATCH v2 7/28 s390x/tcg: MVPG: Check for specification exceptions
In reply to David HildenbrandPerform the checks documented in the PoP.
--- target/s390x/mem_helper.c | 7 +++++++ 1 file changed, 7 insertions(+)
target/s390x/mem_helper.c +7 -0
@@ -673,6 +673,13 @@ uint64_t HELPER(clst)(CPUS390XState *env, uint64_t c, uint64_t s1, uint64_t s2)/* move page */ uint32_t HELPER (mvpg )(CPUS390XState *env,uint64_t r0,uint64_t r1,uint64_t r2) { +const bool f =extract64 (r0,11 ,1 ); +const bool s =extract64 (r0,10 ,1 ); + +if ((f && s) ||extract64 (r0,12 ,4 )) { +s390_program_interrupt (env,PGM_SPECIFICATION ,ILEN_AUTO ,GETPC ()); + } +/* ??? missing r0 handling, which includes access keys, but more importantly optional suppression of the exception! */ fast_memmove (env, r1, r2,TARGET_PAGE_SIZE ,GETPC ());
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 07/28] s390x/tcg: MVPG: Check for specification exceptions
Quoted 6 lines · David Hildenbrand
Perform the checks documented in the PoP.
Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/mem_helper.c | 7 +++++++ 1 file changed, 7 insertions(+)
r~
PATCH v2 8/28 s390x/tcg: MVPG: Properly wrap the addresses
In reply to David HildenbrandWe have to mask of any unused bits. While at it, document what exactly is missing.
--- target/s390x/mem_helper.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
target/s390x/mem_helper.c +9 -2
@@ -680,8 +680,15 @@ uint32_t HELPER(mvpg)(CPUS390XState *env, uint64_t r0, uint64_t r1, uint64_t r2)s390_program_interrupt (env,PGM_SPECIFICATION ,ILEN_AUTO ,GETPC ()); } -/* ??? missing r0 handling, which includes access keys, but more -importantly optional suppression of the exception! */ + r1 =wrap_address (env, r1 &TARGET_PAGE_MASK ); + r2 =wrap_address (env, r2 &TARGET_PAGE_MASK ); + +/* +* TODO: +* - Access key handling +* - CC-option with surpression of page-translation exceptions +* - Store r1/r2 register identifiers at real location 162 +*/ fast_memmove (env, r1, r2,TARGET_PAGE_SIZE ,GETPC ());return 0 ;/* data moved */ }
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 08/28] s390x/tcg: MVPG: Properly wrap the addresses
Quoted 7 lines · David Hildenbrand
We have to mask of any unused bits. While at it, document what exactly is missing.
Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/mem_helper.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
r~
PATCH v2 9/28 s390x/tcg: MVCLU/MVCLE: Process max 4k bytes at a time
In reply to David Hildenbrand... and indicate cc=3 in case there is work remaining. Keep unicode padding simple.
While reworking, properly wrap the addresses.
--- target/s390x/mem_helper.c | 54 ++++++++++++++++++++++----------------- 1 file changed, 31 insertions(+), 23 deletions(-)
target/s390x/mem_helper.c +31 -23
@@ -769,8 +769,8 @@ static inline uint32_t do_mvcl(CPUS390XState *env, uint64_t *src, uint64_t *srclen, uint16_t pad, int wordsize, uintptr_t ra) { - uint64_t len = MIN(*srclen, *destlen); -uint32_t cc; + int len = MIN(*destlen, TARGET_PAGE_SIZE); +int i, cc;if (*destlen == *srclen) { cc =0 ;
@@ -780,32 +780,40 @@ static inline uint32_t do_mvcl(CPUS390XState *env, cc =2 ; } -/* Copy the src array */ -fast_memmove (env, *dest, *src, len, ra); - *src += len; - *srclen -= len; - *dest += len; - *destlen -= len; +if (!*destlen) { +return cc; + } -/* Pad the remaining area */ -if (wordsize ==1 ) { -fast_memset (env, *dest, pad, *destlen, ra); - *dest += *destlen; - *destlen =0 ; +/* +* Only perform one type of type of operation (move/pad) at a time. +* Process up to 4k bytes. +*/ +if (*srclen) { +/* Copy the src array */ + len =MIN (len, *srclen); + *destlen -= len; + *srclen -= len; +fast_memmove (env, *dest, *src, len, ra); + *src =wrap_address (env, *src + len); + *dest =wrap_address (env, *dest + len); + }else if (wordsize ==1 ) { +/* Pad the remaining area */ + *destlen -= len; +fast_memset (env, *dest, pad, len, ra); + *dest =wrap_address (env, *dest + len); }else { -/* If remaining length is odd, pad with odd byte first. */ -if (*destlen &1 ) { -cpu_stb_data_ra (env, *dest, pad &0xff , ra); - *dest +=1 ; - *destlen -=1 ; - } -/* The remaining length is even, pad using words. */ -for (; *destlen; *dest +=2 , *destlen -=2 ) { -cpu_stw_data_ra (env, *dest, pad, ra); +/* The remaining length selects the padding byte. */ +for (i =0 ; i < len; (*destlen)--, i++) { +if (*destlen &1 ) { +cpu_stb_data_ra (env, *dest, pad, ra); + }else { +cpu_stb_data_ra (env, *dest, pad >>8 , ra); + } + *dest =wrap_address (env, *dest +1 ); } } -return cc; +return *destlen ?3 : cc; }/* move long */
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 09/28] s390x/tcg: MVCLU/MVCLE: Process max 4k bytes at a time
target/s390x/mem_helper.c static inline uint32_t do_mvcl(CPUS390XState *env, · line 803
Quoted 13 lines · David Hildenbrand
target/s390x/mem_helper.c +13
-0+if (*srclen) { +/* Copy the src array */ + len =MIN (len, *srclen); + *destlen -= len; + *srclen -= len; +fast_memmove (env, *dest, *src, len, ra); + *src =wrap_address (env, *src + len); + *dest =wrap_address (env, *dest + len); + }else if (wordsize ==1 ) { +/* Pad the remaining area */ + *destlen -= len; +fast_memset (env, *dest, pad, len, ra); + *dest =wrap_address (env, *dest + len);
Again, I'm not sure fast_memmove actually handles wrap, yet. Would it be easier to split at page boundaries rather than a fixed 4k length?
r~
Re: [Qemu-devel] [PATCH v2 09/28] s390x/tcg: MVCLU/MVCLE: Process max 4k bytes at a time
Quoted 15 lines · Richard Henderson
target/s390x/mem_helper.c static inline uint32_t do_mvcl(CPUS390XState *env, · line 803
Quoted 13 lines · David Hildenbrand
target/s390x/mem_helper.c +13
-0+if (*srclen) { +/* Copy the src array */ + len =MIN (len, *srclen); + *destlen -= len; + *srclen -= len; +fast_memmove (env, *dest, *src, len, ra); + *src =wrap_address (env, *src + len); + *dest =wrap_address (env, *dest + len); + }else if (wordsize ==1 ) { +/* Pad the remaining area */ + *destlen -= len; +fast_memset (env, *dest, pad, len, ra); + *dest =wrap_address (env, *dest + len);
Quoted 1 line
Again, I'm not sure
fast_memmoveactually handles wrap, yet.
Similarly, fixed by access_prepare(), access_memmove().
Quoted 2 lines
Would it be easier to split at page boundaries rather than a fixed 4k length?
Also had that already, can do.
Signature · 3 lines
Thanks, David / dhildenb
PATCH v2 10/28 s390x/tcg: MVCS/MVCP: Check for special operation exceptions
In reply to David HildenbrandLet's perform the documented checks.
--- target/s390x/mem_helper.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
target/s390x/mem_helper.c +12 -0
@@ -1961,12 +1961,18 @@ uint32_t HELPER(rrbe)(CPUS390XState *env, uint64_t r2)uint32_t HELPER (mvcs )(CPUS390XState *env,uint64_t l,uint64_t a1,uint64_t a2) { +const uint8_t psw_as = (env->psw.mask &PSW_MASK_ASC ) >>PSW_SHIFT_ASC ;uintptr_t ra =GETPC ();int cc =0 , i;HELPER_LOG ("%s: %16" PRIx64" %16" PRIx64" %16" PRIx64"\n" , __func__, l, a1, a2); +if (!(env->psw.mask &PSW_MASK_DAT ) || !(env->cregs[0 ] &CR0_SECONDARY ) || + psw_as ==AS_HOME || psw_as ==AS_ACCREG ) { +s390_program_interrupt (env,PGM_SPECIAL_OP ,ILEN_AUTO , ra); + } +if (l >256 ) {/* max 256 */ l =256 ;
@@ -1984,12 +1990,18 @@ uint32_t HELPER(mvcs)(CPUS390XState *env, uint64_t l, uint64_t a1, uint64_t a2)uint32_t HELPER (mvcp )(CPUS390XState *env,uint64_t l,uint64_t a1,uint64_t a2) { +const uint8_t psw_as = (env->psw.mask &PSW_MASK_ASC ) >>PSW_SHIFT_ASC ;uintptr_t ra =GETPC ();int cc =0 , i;HELPER_LOG ("%s: %16" PRIx64" %16" PRIx64" %16" PRIx64"\n" , __func__, l, a1, a2); +if (!(env->psw.mask &PSW_MASK_DAT ) || !(env->cregs[0 ] &CR0_SECONDARY ) || + psw_as ==AS_HOME || psw_as ==AS_ACCREG ) { +s390_program_interrupt (env,PGM_SPECIAL_OP ,ILEN_AUTO , ra); + } +if (l >256 ) {/* max 256 */ l =256 ;
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 10/28] s390x/tcg: MVCS/MVCP: Check for special operation exceptions
Quoted 6 lines · David Hildenbrand
Let's perform the documented checks.
Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/mem_helper.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
r~
PATCH v2 11/28 s390x/tcg: MVCS/MVCP: Properly wrap the length
In reply to David Hildenbrand... and don't perform any move in case the length is zero.
--- target/s390x/mem_helper.c | 6 ++++++ 1 file changed, 6 insertions(+)
target/s390x/mem_helper.c +6 -0
@@ -1973,10 +1973,13 @@ uint32_t HELPER(mvcs)(CPUS390XState *env, uint64_t l, uint64_t a1, uint64_t a2)s390_program_interrupt (env,PGM_SPECIAL_OP ,ILEN_AUTO , ra); } + l =wrap_length (env, l);if (l >256 ) {/* max 256 */ l =256 ; cc =3 ; + }else if (!l) { +return cc; }/* XXX replace w/ memcpy */
@@ -2002,10 +2005,13 @@ uint32_t HELPER(mvcp)(CPUS390XState *env, uint64_t l, uint64_t a1, uint64_t a2)s390_program_interrupt (env,PGM_SPECIAL_OP ,ILEN_AUTO , ra); } + l =wrap_length (env, l);if (l >256 ) {/* max 256 */ l =256 ; cc =3 ; + }else if (!l) { +return cc; }/* XXX replace w/ memcpy */
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 11/28] s390x/tcg: MVCS/MVCP: Properly wrap the length
target/s390x/mem_helper.c uint32_t HELPER(mvcs)(CPUS390XState *env, uint64_t l, uint64_t a1, uint64_t a2) · line 1983
Quoted 8 lines · David Hildenbrand
target/s390x/mem_helper.c +3
-0+ l =wrap_length (env, l);if (l >256 ) {/* max 256 */ l =256 ; cc =3 ; + }else if (!l) { +return cc; }
Um, wrap_length only takes 31 bits.
These insns take 32 bits in 24/31-bit modes.
r~
Re: [Qemu-devel] [PATCH v2 11/28] s390x/tcg: MVCS/MVCP: Properly wrap the length
Quoted 10 lines · Richard Henderson
target/s390x/mem_helper.c uint32_t HELPER(mvcs)(CPUS390XState *env, uint64_t l, uint64_t a1, uint64_t a2) · line 1983
Quoted 8 lines · David Hildenbrand
target/s390x/mem_helper.c +3
-0+ l =wrap_length (env, l);if (l >256 ) {/* max 256 */ l =256 ; cc =3 ; + }else if (!l) { +return cc; }
Quoted 2 lines
Um,
wrap_lengthonly takes 31 bits.
These insns take 32 bits in 24/31-bit modes.
Nice observation! Thanks!
Signature · 3 lines
Thanks, David / dhildenb
PATCH v2 12/28 s390x/tcg: MVST: Check for specification exceptions
In reply to David HildenbrandBit position 32-55 of general register 0 must be zero.
--- target/s390x/mem_helper.c | 3 +++ 1 file changed, 3 insertions(+)
target/s390x/mem_helper.c +3 -0
@@ -699,6 +699,9 @@ uint64_t HELPER(mvst)(CPUS390XState *env, uint64_t c, uint64_t d, uint64_t s)uintptr_t ra =GETPC ();uint32_t len; +if (c &0xffffff00ull ) { +s390_program_interrupt (env,PGM_SPECIFICATION ,ILEN_AUTO , ra); + } c = c &0xff ; d =wrap_address (env, d); s =wrap_address (env, s);
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 12/28] s390x/tcg: MVST: Check for specification exceptions
Quoted 6 lines · David Hildenbrand
Bit position 32-55 of general register 0 must be zero.
Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/mem_helper.c | 3 +++ 1 file changed, 3 insertions(+)
r~
PATCH v2 13/28 s390x/tcg: MVST: Fix storing back the addresses to registers
In reply to David Hildenbrand24 and 31-bit address space handling is wrong when it comes to storing back the addresses to the register.
--- target/s390x/helper.h | 2 +- target/s390x/insn-data.def | 2 +- target/s390x/mem_helper.c | 20 ++++++++------------ target/s390x/translate.c | 8 ++++++-- 4 files changed, 16 insertions(+), 16 deletions(-)
target/s390x/helper.h +1 -1
@@ -20,7 +20,7 @@ DEF_HELPER_FLAGS_4(mvn, TCG_CALL_NO_WG, void, env, i32, i64, i64) DEF_HELPER_FLAGS_4(mvo, TCG_CALL_NO_WG, void, env, i32, i64, i64) DEF_HELPER_FLAGS_4(mvpg, TCG_CALL_NO_WG, i32, env, i64, i64, i64) DEF_HELPER_FLAGS_4(mvz, TCG_CALL_NO_WG, void, env, i32, i64, i64) -DEF_HELPER_4(mvst, i64, env, i64, i64, i64) +DEF_HELPER_4(mvst, i32, env, i64, i32, i32) DEF_HELPER_4(ex, void, env, i32, i64, i64) DEF_HELPER_FLAGS_4(stam, TCG_CALL_NO_WG, void, env, i32, i64, i32)DEF_HELPER_FLAGS_4 (lam,TCG_CALL_NO_WG , void, env, i32, i64, i32)
target/s390x/insn-data.def +1 -1
@@ -637,7 +637,7 @@ /* MOVE PAGE */ C(0xb254, MVPG, RRE, Z, r1_o, r2_o, 0, 0, mvpg, 0) /* MOVE STRING */ - C(0xb255, MVST, RRE, Z, r1_o, r2_o, 0, 0, mvst, 0) + C(0xb255, MVST, RRE, Z, 0, 0, 0, 0, mvst, 0) /* MOVE WITH OPTIONAL SPECIFICATION */ C(0xc800, MVCOS, SSF, MVCOS, la1, a2, 0, 0, mvcos, 0) /* MOVE WITH OFFSET */
target/s390x/mem_helper.c +8 -12
@@ -694,8 +694,10 @@ uint32_t HELPER(mvpg)(CPUS390XState *env, uint64_t r0, uint64_t r1, uint64_t r2) }/* string copy (c is string terminator) */ -uint64_t HELPER (mvst )(CPUS390XState *env,uint64_t c,uint64_t d,uint64_t s) +uint32_t HELPER (mvst )(CPUS390XState *env,uint64_t c,uint32_t r1,uint32_t r2) { +const uint64_t d =get_address (env, r1); +const uint64_t s =get_address (env, r2);uintptr_t ra =GETPC ();uint32_t len;
@@ -703,8 +705,6 @@ uint64_t HELPER(mvst)(CPUS390XState *env, uint64_t c, uint64_t d, uint64_t s)s390_program_interrupt (env,PGM_SPECIFICATION ,ILEN_AUTO , ra); } c = c &0xff ; - d =wrap_address (env, d); - s =wrap_address (env, s);/* Lest we fail to service interrupts in a timely manner, limit the amount of work we're willing to do. For now, let's cap at 8k. */
@@ -712,17 +712,13 @@ uint64_t HELPER(mvst)(CPUS390XState *env, uint64_t c, uint64_t d, uint64_t s)uint8_t v =cpu_ldub_data_ra (env, s + len, ra);cpu_stb_data_ra (env, d + len, v, ra);if (v == c) { -/* Complete. Set CC=1 and advance R1. */ - env->cc_op =1 ; - env->retxl = s; -return d + len; +set_address_zero (env, r1, d + len); +return 1 ; } } - -/* Incomplete. Set CC=3 and signal to advance R1 and R2. */ - env->cc_op =3 ; - env->retxl = s + len; -return d + len; +set_address_zero (env, r1, d + len); +set_address_zero (env, r2, s + len); +return 3 ; }/* load access registers r1 to r3 from memory at a2 */
target/s390x/translate.c +6 -2
@@ -3488,9 +3488,13 @@ static DisasJumpType op_mvpg(DisasContext *s, DisasOps *o)static DisasJumpType op_mvst (DisasContext *s,DisasOps *o) { -gen_helper_mvst (o->in1, cpu_env, regs[0 ], o->in1, o->in2); +TCGv_i32 t1 =tcg_const_i32 (get_field (s->fields, r1)); +TCGv_i32 t2 =tcg_const_i32 (get_field (s->fields, r2)); + +gen_helper_mvst (cc_op, cpu_env, regs[0 ], t1, t2); +tcg_temp_free_i32 (t1); +tcg_temp_free_i32 (t2);set_cc_static (s); -return_low128 (o->in2);return DISAS_NEXT ; }
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 13/28] s390x/tcg: MVST: Fix storing back the addresses to registers
Quoted 10 lines · David Hildenbrand
24 and 31-bit address space handling is wrong when it comes to storing back the addresses to the register.
Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/helper.h | 2 +- target/s390x/insn-data.def | 2 +- target/s390x/mem_helper.c | 20 ++++++++------------ target/s390x/translate.c | 8 ++++++-- 4 files changed, 16 insertions(+), 16 deletions(-)
target/s390x/mem_helper.c uint32_t HELPER(mvpg)(CPUS390XState *env, uint64_t r0, uint64_t r1, uint64_t r2) · line 700
Quoted 5 lines
target/s390x/mem_helper.c +3
-1-uint64_t HELPER (mvst )(CPUS390XState *env,uint64_t c,uint64_t d,uint64_t s) +uint32_t HELPER (mvst )(CPUS390XState *env,uint64_t c,uint32_t r1,uint32_t r2) { +const uint64_t d =get_address (env, r1); +const uint64_t s =get_address (env, r2);
While you're changing the signature, you could also read R0 implicitly from env instead of passing it as C.
r~
Re: [Qemu-devel] [PATCH v2 13/28] s390x/tcg: MVST: Fix storing back the addresses to registers
Quoted 15 lines · Richard Henderson
Quoted 10 lines · David Hildenbrand
24 and 31-bit address space handling is wrong when it comes to storing back the addresses to the register.
Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/helper.h | 2 +- target/s390x/insn-data.def | 2 +- target/s390x/mem_helper.c | 20 ++++++++------------ target/s390x/translate.c | 8 ++++++-- 4 files changed, 16 insertions(+), 16 deletions(-)Reviewed-by Richard Henderson <[email protected]>
Quoted 9 lines
target/s390x/mem_helper.c uint32_t HELPER(mvpg)(CPUS390XState *env, uint64_t r0, uint64_t r1, uint64_t r2) · line 700
Quoted 5 lines
target/s390x/mem_helper.c +3
-1-uint64_t HELPER (mvst )(CPUS390XState *env,uint64_t c,uint64_t d,uint64_t s) +uint32_t HELPER (mvst )(CPUS390XState *env,uint64_t c,uint32_t r1,uint32_t r2) { +const uint64_t d =get_address (env, r1); +const uint64_t s =get_address (env, r2);While you're changing the signature, you could also read R0 implicitly from env instead of passing it as C.
Had exactly the same in mind, will do!
Signature · 3 lines
Thanks, David / dhildenb
PATCH v2 14/28 s390x/tcg: Always use MMU_USER_IDX for CONFIG_USER_ONLY
In reply to David HildenbrandAlthough we basically ignore the index all the time for CONFIG_USER_ONLY, let's simply skip all the checks and always return MMU_USER_IDX in cpu_mmu_index() and get_mem_index().
--- target/s390x/cpu.h | 4 ++++ target/s390x/translate.c | 4 ++++ 2 files changed, 8 insertions(+)
target/s390x/cpu.h +4 -0
@@ -328,6 +328,9 @@ extern const VMStateDescription vmstate_s390_cpu;static inline int cpu_mmu_index (CPUS390XState *env,bool ifetch) { +#ifdef CONFIG_USER_ONLY +return MMU_USER_IDX ; +#else if (!(env->psw.mask &PSW_MASK_DAT )) {return MMU_REAL_IDX ; }
@@ -351,6 +354,7 @@ static inline int cpu_mmu_index(CPUS390XState *env, bool ifetch)default :abort (); } +#endif } static inline void cpu_get_tb_cpu_state(CPUS390XState* env, target_ulong *pc,
target/s390x/translate.c +4 -0
@@ -318,6 +318,9 @@ static inline uint64_t ld_code4(CPUS390XState *env, uint64_t pc)static int get_mem_index (DisasContext *s) { +#ifdef CONFIG_USER_ONLY +return MMU_USER_IDX ; +#else if (!(s->base.tb->flags &FLAG_MASK_DAT )) {return MMU_REAL_IDX ; }
@@ -333,6 +336,7 @@ static int get_mem_index(DisasContext *s)tcg_abort (); break; } +#endif }static void gen_exception (int excp)
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 14/28] s390x/tcg: Always use MMU_USER_IDX for CONFIG_USER_ONLY
Quoted 9 lines · David Hildenbrand
Although we basically ignore the index all the time for
CONFIG_USER_ONLY, let's simply skip all the checks and always returnMMU_USER_IDXincpu_mmu_index()andget_mem_index().Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/cpu.h | 4 ++++ target/s390x/translate.c | 4 ++++ 2 files changed, 8 insertions(+)
r~
PATCH v2 15/28 s390x/tcg: Fault-safe memset
In reply to David HildenbrandReplace fast_memset() by access_memset(), that first tries to probe access to all affected pages (maximum is two). We'll use the same mechanism for other types of accesses soon.
Only in very rare cases (especially TLB_NOTDIRTY), we'll have to fallback to ld/st helpers.
--- target/s390x/mem_helper.c | 99 ++++++++++++++++++++++++++++++--------- 1 file changed, 77 insertions(+), 22 deletions(-)
target/s390x/mem_helper.c +77 -22
@@ -117,27 +117,82 @@ static inline void cpu_stsize_data_ra(CPUS390XState *env, uint64_t addr, } } -static void fast_memset (CPUS390XState *env,uint64_t dest,uint8_t byte, -uint32_t l,uintptr_t ra) -{ -int mmu_idx =cpu_mmu_index (env, false); +/* +* An access covers at most 4096 bytes and therefore at most two pages. If +* we can't access the host page directly, we'll have to do I/O access +* via ld/st helpers. +*/ +typedef struct S390Access { +target_ulong vaddr1; +target_ulong vaddr2; +char *haddr1; +char *haddr2; +uint16_t size1; +uint16_t size2; +}S390Access ; + +static S390Access access_prepare_idx (CPUS390XState *env,vaddr vaddr,int size, +MMUAccessType access_type,int mmu_idx, +uintptr_t ra) +{ +S390Access access = { + .vaddr1 = vaddr, + .size1 =MIN (size, -(vaddr |TARGET_PAGE_MASK )), + }; + +g_assert (size >0 && size <=4096 ); + access.haddr1 =probe_access (env, access.vaddr1, access.size1, access_type, + mmu_idx, ra); + +if (unlikely (access.size1 != size)) { +/* The access crosses page boundaries. */ + access.vaddr2 =wrap_address (env, vaddr + access.size1); + access.size2 = size - access.size1; + access.haddr2 =probe_access (env, access.vaddr2, access.size2, + access_type, mmu_idx, ra); + } +return access; +} + +static void access_memset_idx (CPUS390XState *env,vaddr vaddr,uint8_t byte, +int size,int mmu_idx,uintptr_t ra) +{ +S390Access desta =access_prepare_idx (env, vaddr, size,MMU_DATA_STORE , + mmu_idx, ra); +#ifdef CONFIG_USER_ONLY +g_assert (desta.haddr1 && (desta.haddr2 || !desta.size2)); +memset (desta.haddr1, byte, desta.size1); +memset (desta.haddr2, byte, desta.size2); +#else +TCGMemOpIdx oi =make_memop_idx (MO_UB , mmu_idx); +int i; -while (l >0 ) { -void *p =tlb_vaddr_to_host (env, dest,MMU_DATA_STORE , mmu_idx); -if (p) { -/* Access to the whole page in write mode granted. */ -uint32_t l_adj =adj_len_to_page (l, dest); -memset (p, byte, l_adj); - dest += l_adj; - l -= l_adj; - }else { -/* We failed to get access to the whole page. The next write -access will likely fill the QEMU TLB for the next iteration. */ -cpu_stb_data_ra (env, dest, byte, ra); - dest++; - l--; +if (likely (desta.haddr1)) { +memset (desta.haddr1, byte, desta.size1); + }else { +for (i =0 ; i < desta.size1; i++) { +helper_ret_stb_mmu (env, desta.vaddr1 + i, byte, oi, ra); + } + } +if (likely (!desta.size2)) { +return ; + } +if (likely (desta.haddr2)) { +memset (desta.haddr2, byte, desta.size2); + }else { +for (i =0 ; i < desta.size2; i++) { +helper_ret_stb_mmu (env, desta.vaddr2 + i, byte, oi, ra); } } +#endif +} + +static void access_memset (CPUS390XState *env,vaddr vaddr,uint8_t byte, +int size,uintptr_t ra) +{ +int mmu_idx =cpu_mmu_index (env, false); + +access_memset_idx (env, vaddr, byte, size, mmu_idx, ra); }#ifndef CONFIG_USER_ONLY
@@ -267,7 +322,7 @@ static uint32_t do_helper_xc(CPUS390XState *env, uint32_t l, uint64_t dest,/* xor with itself is the same as memset(0) */ if (src == dest) { -fast_memset (env, dest,0 , l +1 , ra); +access_memset (env, dest,0 , l +1 , ra);return 0 ; }
@@ -329,7 +384,7 @@ static uint32_t do_helper_mvc(CPUS390XState *env, uint32_t l, uint64_t dest,* behave like memmove(). */ if (dest == src +1 ) { -fast_memset (env, dest,cpu_ldub_data_ra (env, src, ra), l, ra); +access_memset (env, dest,cpu_ldub_data_ra (env, src, ra), l, ra); }else if (!is_destructive_overlap (env, dest, src, l)) {fast_memmove (env, dest, src, l, ra); }else {
@@ -798,7 +853,7 @@ static inline uint32_t do_mvcl(CPUS390XState *env, }else if (wordsize ==1 ) {/* Pad the remaining area */ *destlen -= len; -fast_memset (env, *dest, pad, len, ra); +access_memset (env, *dest, pad, len, ra); *dest =wrap_address (env, *dest + len); }else {/* The remaining length selects the padding byte. */
@@ -852,7 +907,7 @@ uint32_t HELPER(mvcl)(CPUS390XState *env, uint32_t r1, uint32_t r2)while (destlen) { cur_len =MIN (destlen,2048 );if (!srclen) { -fast_memset (env, dest, pad, cur_len, ra); +access_memset (env, dest, pad, cur_len, ra); }else { cur_len =MIN (cur_len, srclen);
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 15/28] s390x/tcg: Fault-safe memset
target/s390x/mem_helper.c static inline void cpu_stsize_data_ra(CPUS390XState *env, uint64_t addr, · line 175
Quoted 6 lines · David Hildenbrand
target/s390x/mem_helper.c +6
-0+if (likely (desta.haddr1)) { +memset (desta.haddr1, byte, desta.size1); + }else { +for (i =0 ; i < desta.size1; i++) { +helper_ret_stb_mmu (env, desta.vaddr1 + i, byte, oi, ra); + }
The only thing perhaps that we could do better here is to re-check tlb_vaddr_to_host after a singe access. This would handle NOTDIRTY with a single iotlb access and then fall back to memset, without re-checking host address for real mmio after every store.
target/s390x/mem_helper.c static inline void cpu_stsize_data_ra(CPUS390XState *env, uint64_t addr, · line 181
Quoted 2 lines
target/s390x/mem_helper.c +2
-0+if (likely (desta.haddr2)) { +memset (desta.haddr2, byte, desta.size2);
Indentation.
target/s390x/mem_helper.c static inline void cpu_stsize_data_ra(CPUS390XState *env, uint64_t addr, · line 185
Quoted 4 lines
target/s390x/mem_helper.c +3
-0+ }else { +for (i =0 ; i < desta.size2; i++) { +helper_ret_stb_mmu (env, desta.vaddr2 + i, byte, oi, ra); }
Likewise wrt NOTDIRTY, which suggests a subroutine to handle a single page?
That said, what's here looks correct so
r~
Re: [Qemu-devel] [PATCH v2 15/28] s390x/tcg: Fault-safe memset
Quoted 8 lines · Richard Henderson
target/s390x/mem_helper.c static inline void cpu_stsize_data_ra(CPUS390XState *env, uint64_t addr, · line 175
Quoted 6 lines · David Hildenbrand
target/s390x/mem_helper.c +6
-0+if (likely (desta.haddr1)) { +memset (desta.haddr1, byte, desta.size1); + }else { +for (i =0 ; i < desta.size1; i++) { +helper_ret_stb_mmu (env, desta.vaddr1 + i, byte, oi, ra); + }
Quoted 4 lines
The only thing perhaps that we could do better here is to re-check
tlb_vaddr_to_hostafter a singe access. This would handle NOTDIRTY with a single iotlb access and then fall back to memset, without re-checking host address for real mmio after every store.
Yes - also thought about that :) - but I didn't find a clean way to do that without uglifying that code (and especially access_memmove() and friends).
Quoted 4 lines
target/s390x/mem_helper.c static inline void cpu_stsize_data_ra(CPUS390XState *env, uint64_t addr, · line 181
Quoted 3 lines
target/s390x/mem_helper.c +2
-0+if (likely (desta.haddr2)) { +memset (desta.haddr2, byte, desta.size2);
Quoted 8 lines
Indentation.
target/s390x/mem_helper.c static inline void cpu_stsize_data_ra(CPUS390XState *env, uint64_t addr, · line 185
Quoted 4 lines
target/s390x/mem_helper.c +3
-0+ }else { +for (i =0 ; i < desta.size2; i++) { +helper_ret_stb_mmu (env, desta.vaddr2 + i, byte, oi, ra); }Likewise wrt NOTDIRTY, which suggests a subroutine to handle a single page?
I'll try to see if this turns out okay-ish. Maybe I'll add via a separate patch on top.
Quoted 4 lines
That said, what's here looks correct so
Reviewed-by Richard Henderson <[email protected]>
Thanks!
Signature · 3 lines
Thanks, David / dhildenb
PATCH v2 16/28 s390x/tcg: Fault-safe memmove
In reply to David HildenbrandReplace fast_memmove() variants by access_memmove() variants, that first try to probe access to all affected pages (maximum is two pages).
In MVCOS, simply always call access_memmove_as() and drop the TODO about LAP. LAP is already handled in the MMU.
Get rid of adj_len_to_page(), which is now unused.
--- target/s390x/mem_helper.c | 204 +++++++++++++++++++++----------------- 1 file changed, 115 insertions(+), 89 deletions(-)
target/s390x/mem_helper.c +115 -89
@@ -65,17 +65,6 @@ static bool is_destructive_overlap(CPUS390XState *env, uint64_t dest,return dest > src && dest <= src + len -1 ; } -/* Reduce the length so that addr + len doesn't cross a page boundary. */ -static inline uint32_t adj_len_to_page (uint32_t len,uint64_t addr) -{ -#ifndef CONFIG_USER_ONLY -if ((addr & ~TARGET_PAGE_MASK ) + len -1 >=TARGET_PAGE_SIZE ) { -return -(addr |TARGET_PAGE_MASK ); - } -#endif -return len; -} -/* Trigger a SPECIFICATION exception if an address or a length is not naturally aligned. */ static inline void check_alignment(CPUS390XState *env, uint64_t v,
@@ -129,6 +118,7 @@ typedef struct S390Access {char *haddr2;uint16_t size1;uint16_t size2; +int mmu_idx; } S390Access; static S390Access access_prepare_idx(CPUS390XState *env, vaddr vaddr, int size,
@@ -138,6 +128,7 @@ static S390Access access_prepare_idx(CPUS390XState *env, vaddr vaddr, int size,S390Access access = { .vaddr1 = vaddr, .size1 =MIN (size, -(vaddr |TARGET_PAGE_MASK )), + .mmu_idx = mmu_idx, };g_assert (size >0 && size <=4096 );
@@ -195,42 +186,112 @@ static void access_memset(CPUS390XState *env, vaddr vaddr, uint8_t byte,access_memset_idx (env, vaddr, byte, size, mmu_idx, ra); } -#ifndef CONFIG_USER_ONLY -static void fast_memmove_idx (CPUS390XState *env,uint64_t dest,uint64_t src, -uint32_t len,int dest_idx,int src_idx, -uintptr_t ra) -{ -TCGMemOpIdx oi_dest =make_memop_idx (MO_UB , dest_idx); -TCGMemOpIdx oi_src =make_memop_idx (MO_UB , src_idx); -uint32_t len_adj; -void *src_p; -void *dest_p; -uint8_t x; - -while (len >0 ) { - src =wrap_address (env, src); - dest =wrap_address (env, dest); - src_p =tlb_vaddr_to_host (env, src,MMU_DATA_LOAD , src_idx); - dest_p =tlb_vaddr_to_host (env, dest,MMU_DATA_STORE , dest_idx); - -if (src_p && dest_p) { -/* Access to both whole pages granted. */ - len_adj =adj_len_to_page (adj_len_to_page (len, src), dest); -memmove (dest_p, src_p, len_adj); +static uint8_t access_get_byte (CPUS390XState *env,const S390Access *access, +int offset,uintptr_t ra) +{ +#ifdef CONFIG_USER_ONLY +if (offset < access->size1) { +return ldub_p (access->haddr1 + offset); + }else { +return ldub_p (access->haddr2 + offset - access->size1); + } +#else +TCGMemOpIdx oi =make_memop_idx (MO_UB , access->mmu_idx); + +if (offset < access->size1) { +if (likely (access->haddr1)) { +return ldub_p (access->haddr1 + offset); + } +return helper_ret_ldub_mmu (env, access->vaddr1 + offset, oi, ra); + } +if (access->haddr2) { +return ldub_p (access->haddr2 + offset - access->size1); + } +return helper_ret_ldub_mmu (env, access->vaddr2 + offset - access->size1, oi, + ra); +#endif +} + +static void access_set_byte (CPUS390XState *env,const S390Access *access, +int offset,uint8_t byte,uintptr_t ra) +{ +#ifdef CONFIG_USER_ONLY +if (offset < access->size1) { +stb_p (access->haddr1 + offset, byte); + }else { +stb_p (access->haddr2 + offset - access->size1, byte); + } +#else +TCGMemOpIdx oi =make_memop_idx (MO_UB , access->mmu_idx); + +if (offset < access->size1) { +if (likely (access->haddr1)) { +stb_p (access->haddr1 + offset, byte); }else { -/* We failed to get access to one or both whole pages. The next -read or write access will likely fill the QEMU TLB for the -next iteration. */ - len_adj =1 ; - x =helper_ret_ldub_mmu (env, src, oi_src, ra); -helper_ret_stb_mmu (env, dest, x, oi_dest, ra); +helper_ret_stb_mmu (env, access->vaddr1 + offset, byte, oi, ra); + } + }else { +if (likely (access->haddr2)) { +stb_p (access->haddr2 + offset - access->size1, byte); + }else { +helper_ret_stb_mmu (env, access->vaddr2 + offset - access->size1, + byte, oi, ra); + } + } +#endif +} + +/* +* Move data with the same semantics as memmove() in case ranges don't overlap +* or src > dest. Undefined behavior on destructive overlaps. +*/ +static void access_memmove_idx (CPUS390XState *env,vaddr dest,vaddr src, +int size,int dest_idx,int src_idx, +uintptr_t ra) +{ +S390Access srca =access_prepare_idx (env, src, size,MMU_DATA_LOAD , src_idx, + ra); +S390Access desta =access_prepare_idx (env, dest, size,MMU_DATA_STORE , + dest_idx, ra); +uint16_t diff; + +/* Fallback to slow access in case we don't have access to all host pages */ +if (unlikely (!desta.haddr1 || (desta.size2 && !desta.haddr2) || + !srca.haddr1 || (srca.size2 && !srca.haddr2))) { +uint8_t byte; +int i; + +for (i =0 ; i < desta.size1 + desta.size2; i++) { + byte =access_get_byte (env, &srca, i, ra); +access_set_byte (env, &desta, i, byte, ra); } - src += len_adj; - dest += len_adj; - len -= len_adj; +return ; + } + +if (srca.size1 == desta.size1) { +memmove (desta.haddr1, srca.haddr1, srca.size1); +memmove (desta.haddr2, srca.haddr2, srca.size2); + }else if (srca.size1 < desta.size1) { + diff = desta.size1 - srca.size1; +memmove (desta.haddr1, srca.haddr1, srca.size1); +memmove (desta.haddr1 + srca.size1, srca.haddr2, diff); +memmove (desta.haddr2, srca.haddr2 + diff, desta.size2); + }else { + diff = srca.size1 - desta.size1; +memmove (desta.haddr1, srca.haddr1, desta.size1); +memmove (desta.haddr2, srca.haddr1 + desta.size1, diff); +memmove (desta.haddr2 + diff, srca.haddr2, srca.size2); } } +static void access_memmove (CPUS390XState *env,vaddr dest,vaddr src, +int size,uintptr_t ra) +{ +int mmu_idx =cpu_mmu_index (env, false); + +access_memmove_idx (env, dest, src, size, mmu_idx, mmu_idx, ra); +} +static int mmu_idx_from_as (uint8_t as) {switch (as) {
@@ -246,43 +307,14 @@ static int mmu_idx_from_as(uint8_t as) } } -static void fast_memmove_as (CPUS390XState *env,uint64_t dest,uint64_t src, -uint32_t len,uint8_t dest_as,uint8_t src_as, -uintptr_t ra) +static void access_memmove_as (CPUS390XState *env,uint64_t dest,uint64_t src, +uint32_t len,uint8_t dest_as,uint8_t src_as, +uintptr_t ra) {int src_idx =mmu_idx_from_as (src_as);int dest_idx =mmu_idx_from_as (dest_as); -fast_memmove_idx (env, dest, src, len, dest_idx, src_idx, ra); -} -#endif - -static void fast_memmove (CPUS390XState *env,uint64_t dest,uint64_t src, -uint32_t l,uintptr_t ra) -{ -int mmu_idx =cpu_mmu_index (env, false); - -while (l >0 ) { -void *src_p =tlb_vaddr_to_host (env, src,MMU_DATA_LOAD , mmu_idx); -void *dest_p =tlb_vaddr_to_host (env, dest,MMU_DATA_STORE , mmu_idx); -if (src_p && dest_p) { -/* Access to both whole pages granted. */ -uint32_t l_adj =adj_len_to_page (l, src); - l_adj =adj_len_to_page (l_adj, dest); -memmove (dest_p, src_p, l_adj); - src += l_adj; - dest += l_adj; - l -= l_adj; - }else { -/* We failed to get access to one or both whole pages. The next -read or write access will likely fill the QEMU TLB for the -next iteration. */ -cpu_stb_data_ra (env, dest,cpu_ldub_data_ra (env, src, ra), ra); - src++; - dest++; - l--; - } - } +access_memmove_idx (env, dest, src, len, dest_idx, src_idx, ra); }/* and on array */
@@ -386,7 +418,7 @@ static uint32_t do_helper_mvc(CPUS390XState *env, uint32_t l, uint64_t dest,if (dest == src +1 ) {access_memset (env, dest,cpu_ldub_data_ra (env, src, ra), l, ra); }else if (!is_destructive_overlap (env, dest, src, l)) { -fast_memmove (env, dest, src, l, ra); +access_memmove (env, dest, src, l, ra); }else {for (i =0 ; i < l; i++) {uint8_t x =cpu_ldub_data_ra (env, src + i, ra);
@@ -744,7 +776,7 @@ uint32_t HELPER(mvpg)(CPUS390XState *env, uint64_t r0, uint64_t r1, uint64_t r2)* - CC-option with surpression of page-translation exceptions * - Store r1/r2 register identifiers at real location 162 */ -fast_memmove (env, r1, r2,TARGET_PAGE_SIZE ,GETPC ()); +access_memmove (env, r1, r2,TARGET_PAGE_SIZE ,GETPC ());return 0 ;/* data moved */ }
@@ -847,7 +879,7 @@ static inline uint32_t do_mvcl(CPUS390XState *env, len =MIN (len, *srclen); *destlen -= len; *srclen -= len; -fast_memmove (env, *dest, *src, len, ra); +access_memmove (env, *dest, *src, len, ra); *src =wrap_address (env, *src + len); *dest =wrap_address (env, *dest + len); }else if (wordsize ==1 ) {
@@ -911,7 +943,7 @@ uint32_t HELPER(mvcl)(CPUS390XState *env, uint32_t r1, uint32_t r2) }else { cur_len =MIN (cur_len, srclen); -fast_memmove (env, dest, src, cur_len, ra); +access_memmove (env, dest, src, cur_len, ra); src =wrap_address (env, src + cur_len); srclen -= cur_len; env->regs[r2 +1 ] =deposit64 (env->regs[r2 +1 ],0 ,24 , srclen);
@@ -2453,16 +2485,10 @@ uint32_t HELPER(mvcos)(CPUS390XState *env, uint64_t dest, uint64_t src,s390_program_interrupt (env,PGM_ADDRESSING ,6 , ra); } -/* FIXME: a) LAP -* b) Access using correct keys -* c) AR-mode -*/ -#ifdef CONFIG_USER_ONLY -/* psw keys are never valid in user mode, we will never reach this */ -g_assert_not_reached (); -#else -fast_memmove_as (env, dest, src, len, dest_as, src_as, ra); -#endif +/* FIXME: Access using correct keys and AR-mode */ +if (len) { +access_memmove_as (env, dest, src, len, dest_as, src_as, ra); + }return cc; }
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 16/28] s390x/tcg: Fault-safe memmove
Quoted 6 lines · David Hildenbrand
Replace
fast_memmove()variants byaccess_memmove()variants, that first try to probe access to all affected pages (maximum is two pages).In MVCOS, simply always call
access_memmove_as()and drop the TODO about LAP. LAP is already handled in the MMU.
Quoted 6 lines
Get rid of
adj_len_to_page(), which is now unused.Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/mem_helper.c | 204 +++++++++++++++++++++----------------- 1 file changed, 115 insertions(+), 89 deletions(-)
r~
Re: [Qemu-devel] [PATCH v2 16/28] s390x/tcg: Fault-safe memmove
In reply to David Hildenbrandtarget/s390x/mem_helper.c static void access_memset(CPUS390XState *env, vaddr vaddr, uint8_t byte, · line 255
Quoted 8 lines · David Hildenbrand
target/s390x/mem_helper.c +8
-0+static void access_memmove_idx (CPUS390XState *env,vaddr dest,vaddr src, +int size,int dest_idx,int src_idx, +uintptr_t ra) +{ +S390Access srca =access_prepare_idx (env, src, size,MMU_DATA_LOAD , src_idx, + ra); +S390Access desta =access_prepare_idx (env, dest, size,MMU_DATA_STORE , + dest_idx, ra);
I was just thinking that it might be worth passing in these Access structures.
It seems that usually we wind up computing them in several locations.
Hoisting it up it would make MVC look like
midx = cpu_mmu_index(env);
srca = access_prepare_idx(env, src, size, LOAD, midx, ra);
dsta = access_prepare_idx(env, dst, size, STORE, midx, ra);
if (dst == src + 1) {
uint8_t x = access_get_byte(env, &srca, 0, ra);
access_memset(env, &dsta, x, size, ra);
} else if (!is_destructive_overlap(env, dst, src, size)) {
access_memmove(env, &dsta, &srca, size, ra);
} else {
// byte by byte loop, but still need srca, dsta.
}which seems even More Correct, since the current access_memset path does not check for read watchpoints or faults on all of [src, src+size-1].
r~
Re: [Qemu-devel] [PATCH v2 16/28] s390x/tcg: Fault-safe memmove
Quoted 28 lines · Richard Henderson
target/s390x/mem_helper.c static void access_memset(CPUS390XState *env, vaddr vaddr, uint8_t byte, · line 255
Quoted 8 lines · David Hildenbrand
target/s390x/mem_helper.c +8
-0+static void access_memmove_idx (CPUS390XState *env,vaddr dest,vaddr src, +int size,int dest_idx,int src_idx, +uintptr_t ra) +{ +S390Access srca =access_prepare_idx (env, src, size,MMU_DATA_LOAD , src_idx, + ra); +S390Access desta =access_prepare_idx (env, dest, size,MMU_DATA_STORE , + dest_idx, ra);I was just thinking that it might be worth passing in these Access structures.
It seems that usually we wind up computing them in several locations.Hoisting it up it would make MVC look like
midx = cpu_mmu_index(env); srca = access_prepare_idx(env, src, size, LOAD, midx, ra); dsta = access_prepare_idx(env, dst, size, STORE, midx, ra); if (dst == src + 1) { uint8_t x = access_get_byte(env, &srca, 0, ra); access_memset(env, &dsta, x, size, ra); } else if (!is_destructive_overlap(env, dst, src, size)) { access_memmove(env, &dsta, &srca, size, ra); } else { // byte by byte loop, but still need srca, dsta. }
Quoted 3 lines
which seems even More Correct, since the current
access_memsetpath does not check for read watchpoints or faults on all of [src, src+size-1].
I had precisely that in previous versions :) Can switch to that model.
Signature · 3 lines
Thanks, David / dhildenb
PATCH v2 17/28 s390x/tcg: MVCS/MVCP: Use access_memmove_idx()
In reply to David HildenbrandAs we are moving between address spaces, we can use access_memmove_idx() without checking for destructive overlaps (especially of real storage locations):
"Each storage operand is processed left to right. The storage-operand-consistency rules are the same as
for MOVE (MVC), except that when the operands
overlap in real storage, the use of the common real- storage locations is not necessarily recognized."
--- target/s390x/mem_helper.c | 20 ++++++-------------- 1 file changed, 6 insertions(+), 14 deletions(-)
target/s390x/mem_helper.c +6 -14
@@ -2049,7 +2049,7 @@ uint32_t HELPER(mvcs)(CPUS390XState *env, uint64_t l, uint64_t a1, uint64_t a2) {const uint8_t psw_as = (env->psw.mask &PSW_MASK_ASC ) >>PSW_SHIFT_ASC ;uintptr_t ra =GETPC (); -int cc =0 , i; +int cc =0 ;HELPER_LOG ("%s: %16" PRIx64" %16" PRIx64" %16" PRIx64"\n" , __func__, l, a1, a2);
@@ -2068,12 +2068,8 @@ uint32_t HELPER(mvcs)(CPUS390XState *env, uint64_t l, uint64_t a1, uint64_t a2)return cc; } -/* XXX replace w/ memcpy */ -for (i =0 ; i < l; i++) { -uint8_t x =cpu_ldub_primary_ra (env, a2 + i, ra); -cpu_stb_secondary_ra (env, a1 + i, x, ra); - } - +/* TODO: Access key handling */ +access_memmove_idx (env, a1, a2, l,MMU_SECONDARY_IDX ,MMU_PRIMARY_IDX , ra);return cc; }
@@ -2081,7 +2077,7 @@ uint32_t HELPER(mvcp)(CPUS390XState *env, uint64_t l, uint64_t a1, uint64_t a2) {const uint8_t psw_as = (env->psw.mask &PSW_MASK_ASC ) >>PSW_SHIFT_ASC ;uintptr_t ra =GETPC (); -int cc =0 , i; +int cc =0 ;HELPER_LOG ("%s: %16" PRIx64" %16" PRIx64" %16" PRIx64"\n" , __func__, l, a1, a2);
@@ -2100,12 +2096,8 @@ uint32_t HELPER(mvcp)(CPUS390XState *env, uint64_t l, uint64_t a1, uint64_t a2)return cc; } -/* XXX replace w/ memcpy */ -for (i =0 ; i < l; i++) { -uint8_t x =cpu_ldub_secondary_ra (env, a2 + i, ra); -cpu_stb_primary_ra (env, a1 + i, x, ra); - } - +/* TODO: Access key handling */ +access_memmove_idx (env, a1, a2, l,MMU_PRIMARY_IDX ,MMU_SECONDARY_IDX , ra);return cc; }
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 17/28] s390x/tcg: MVCS/MVCP: Use access_memmove_idx()
Quoted 9 lines · David Hildenbrand
As we are moving between address spaces, we can use
access_memmove_idx()without checking for destructive overlaps (especially of real storage locations):
"Each storage operand is processed left to right. The storage-operand-consistency rules are the same as
for MOVE (MVC), except that when the operands
overlap in real storage, the use of the common real- storage locations is not necessarily recognized."
Quoted 4 lines
Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/mem_helper.c | 20 ++++++-------------- 1 file changed, 6 insertions(+), 14 deletions(-)
r~
PATCH v2 18/28 s390x/tcg: MVC: Fault-safe handling on destructive overlaps
In reply to David HildenbrandThe last remaining bit for MVC is handling destructive overlaps in a fault-safe way.
--- target/s390x/mem_helper.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-)
target/s390x/mem_helper.c +14 -2
@@ -145,6 +145,14 @@ static S390Access access_prepare_idx(CPUS390XState *env, vaddr vaddr, int size,return access; } +static S390Access access_prepare (CPUS390XState *env,vaddr vaddr,int size, +MMUAccessType access_type,uintptr_t ra) +{ +int mmu_idx =cpu_mmu_index (env, false); + +return access_prepare_idx (env, vaddr, size, access_type, mmu_idx, ra); +} +static void access_memset_idx (CPUS390XState *env,vaddr vaddr,uint8_t byte,int size,int mmu_idx,uintptr_t ra) {
@@ -420,9 +428,13 @@ static uint32_t do_helper_mvc(CPUS390XState *env, uint32_t l, uint64_t dest, }else if (!is_destructive_overlap (env, dest, src, l)) {access_memmove (env, dest, src, l, ra); }else { +S390Access srca =access_prepare (env, src, l,MMU_DATA_LOAD , ra); +S390Access desta =access_prepare (env, dest, l,MMU_DATA_STORE , ra); +for (i =0 ; i < l; i++) { -uint8_t x =cpu_ldub_data_ra (env, src + i, ra); -cpu_stb_data_ra (env, dest + i, x, ra); +uint8_t byte =access_get_byte (env, &srca, i, ra); + +access_set_byte (env, &desta, i, byte, ra); } }
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 18/28] s390x/tcg: MVC: Fault-safe handling on destructive overlaps
Quoted 12 lines · David Hildenbrand
target/s390x/mem_helper.c +8
-0+static S390Access access_prepare (CPUS390XState *env,vaddr vaddr,int size, +MMUAccessType access_type,uintptr_t ra) +{ +int mmu_idx =cpu_mmu_index (env, false); + +return access_prepare_idx (env, vaddr, size, access_type, mmu_idx, ra); +} +static void access_memset_idx (CPUS390XState *env,vaddr vaddr,uint8_t byte,int size,int mmu_idx,uintptr_t ra) {
target/s390x/mem_helper.c static uint32_t do_helper_mvc(CPUS390XState *env, uint32_t l, uint64_t dest, · line 432
Quoted 5 lines
target/s390x/mem_helper.c +2
-0@@ -420,9 +428,13 @@ static uint32_t do_helper_mvc(CPUS390XState *env, uint32_t l, uint64_t dest, }else if (!is_destructive_overlap (env, dest, src, l)) {access_memmove (env, dest, src, l, ra); }else { +S390Access srca =access_prepare (env, src, l,MMU_DATA_LOAD , ra); +S390Access desta =access_prepare (env, dest, l,MMU_DATA_STORE , ra);
I was just thinking it might be better to drop the non-idx functions:access_prepare, access_memset, access_memmove. What this is leading to is computation of cpu_mmu_index multiple times, as here.
It could just as easily be hoisted to the top of do_helper_mvc and used in all of the sub-cases.
That said, the code here is correct so
r~
PATCH v2 19/28 s390x/tcg: MVCLU: Fault-safe handling
In reply to David HildenbrandThe last remaining bit is padding with two bytes.
--- target/s390x/mem_helper.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
target/s390x/mem_helper.c +5 -3
@@ -900,15 +900,17 @@ static inline uint32_t do_mvcl(CPUS390XState *env,access_memset (env, *dest, pad, len, ra); *dest =wrap_address (env, *dest + len); }else { +S390Access desta =access_prepare (env, *dest, len,MMU_DATA_STORE , ra); +/* The remaining length selects the padding byte. */ for (i =0 ; i < len; (*destlen)--, i++) {if (*destlen &1 ) { -cpu_stb_data_ra (env, *dest, pad, ra); +access_set_byte (env, &desta, i, pad, ra); }else { -cpu_stb_data_ra (env, *dest, pad >>8 , ra); +access_set_byte (env, &desta, i, pad >>8 , ra); } - *dest =wrap_address (env, *dest +1 ); } + *dest =wrap_address (env, *dest + len); }return *destlen ?3 : cc;
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 19/28] s390x/tcg: MVCLU: Fault-safe handling
Quoted 6 lines · David Hildenbrand
The last remaining bit is padding with two bytes.
Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/mem_helper.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
r~
PATCH v2 20/28 s390x/tcg: OC: Fault-safe handling
In reply to David HildenbrandWe can process a maximum of 256 bytes, crossing two pages.
--- target/s390x/mem_helper.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
target/s390x/mem_helper.c +12 -4
@@ -385,17 +385,25 @@ uint32_t HELPER(xc)(CPUS390XState *env, uint32_t l, uint64_t dest,static uint32_t do_helper_oc (CPUS390XState *env,uint32_t l,uint64_t dest,uint64_t src,uintptr_t ra) { +S390Access srca1, srca2, desta;uint32_t i;uint8_t c =0 ;HELPER_LOG ("%s l %d dest %" PRIx64" src %" PRIx64"\n" , __func__, l, dest, src); -for (i =0 ; i <= l; i++) { -uint8_t x =cpu_ldub_data_ra (env, src + i, ra); - x |=cpu_ldub_data_ra (env, dest + i, ra); +/* OC always processes one more byte than specified - maximum is 256 */ + l++; + + srca1 =access_prepare (env, src, l,MMU_DATA_LOAD , ra); + srca2 =access_prepare (env, dest, l,MMU_DATA_LOAD , ra); + desta =access_prepare (env, dest, l,MMU_DATA_STORE , ra); +for (i =0 ; i < l; i++) { +const uint8_t x =access_get_byte (env, &srca1, i, ra) | +access_get_byte (env, &srca2, i, ra); + c |= x; -cpu_stb_data_ra (env, dest + i, x, ra); +access_set_byte (env, &desta, i, x, ra); }return c !=0 ; }
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 20/28] s390x/tcg: OC: Fault-safe handling
target/s390x/mem_helper.c uint32_t HELPER(xc)(CPUS390XState *env, uint32_t l, uint64_t dest, · line 400
Quoted 2 lines · David Hildenbrand
target/s390x/mem_helper.c +2
-0+ srca2 =access_prepare (env, dest, l,MMU_DATA_LOAD , ra); + desta =access_prepare (env, dest, l,MMU_DATA_STORE , ra);
We should find a way to perform this in one step.
RWM isn't uncommon...
That said,
r~
Re: [Qemu-devel] [PATCH v2 20/28] s390x/tcg: OC: Fault-safe handling
Quoted 6 lines · Richard Henderson
target/s390x/mem_helper.c uint32_t HELPER(xc)(CPUS390XState *env, uint32_t l, uint64_t dest, · line 400
Quoted 2 lines · David Hildenbrand
target/s390x/mem_helper.c +2
-0+ srca2 =access_prepare (env, dest, l,MMU_DATA_LOAD , ra); + desta =access_prepare (env, dest, l,MMU_DATA_STORE , ra);We should find a way to perform this in one step.
RWM isn't uncommon...
Yes, like converting MMU_* into flags we can then process in probe_access().
Quoted 6 lines
That said,
Reviewed-by Richard Henderson <[email protected]>r~
Signature · 3 lines
Thanks, David / dhildenb
PATCH v2 21/28 s390x/tcg: XC: Fault-safe handling
In reply to David HildenbrandWe can process a maximum of 256 bytes, crossing two pages. While at it, increment the length once.
--- target/s390x/mem_helper.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-)
target/s390x/mem_helper.c +13 -5
@@ -354,23 +354,31 @@ uint32_t HELPER(nc)(CPUS390XState *env, uint32_t l, uint64_t dest,static uint32_t do_helper_xc (CPUS390XState *env,uint32_t l,uint64_t dest,uint64_t src,uintptr_t ra) { +S390Access srca1, srca2, desta;uint32_t i;uint8_t c =0 ;HELPER_LOG ("%s l %d dest %" PRIx64" src %" PRIx64"\n" , __func__, l, dest, src); +/* XC always processes one more byte than specified - maximum is 256 */ + l++; +/* xor with itself is the same as memset(0) */ if (src == dest) { -access_memset (env, dest,0 , l +1 , ra); +access_memset (env, dest,0 , l, ra);return 0 ; } -for (i =0 ; i <= l; i++) { -uint8_t x =cpu_ldub_data_ra (env, src + i, ra); - x ^=cpu_ldub_data_ra (env, dest + i, ra); + srca1 =access_prepare (env, src, l,MMU_DATA_LOAD , ra); + srca2 =access_prepare (env, dest, l,MMU_DATA_LOAD , ra); + desta =access_prepare (env, dest, l,MMU_DATA_STORE , ra); +for (i =0 ; i < l; i++) { +const uint8_t x =access_get_byte (env, &srca1, i, ra) ^ +access_get_byte (env, &srca2, i, ra); + c |= x; -cpu_stb_data_ra (env, dest + i, x, ra); +access_set_byte (env, &desta, i, x, ra); }return c !=0 ; }
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 21/28] s390x/tcg: XC: Fault-safe handling
Quoted 7 lines · David Hildenbrand
We can process a maximum of 256 bytes, crossing two pages. While at it, increment the length once.
Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/mem_helper.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-)
r~
PATCH v2 22/28 s390x/tcg: NC: Fault-safe handling
In reply to David HildenbrandWe can process a maximum of 256 bytes, crossing two pages.
--- target/s390x/mem_helper.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
target/s390x/mem_helper.c +12 -4
@@ -329,17 +329,25 @@ static void access_memmove_as(CPUS390XState *env, uint64_t dest, uint64_t src,static uint32_t do_helper_nc (CPUS390XState *env,uint32_t l,uint64_t dest,uint64_t src,uintptr_t ra) { +S390Access srca1, srca2, desta;uint32_t i;uint8_t c =0 ;HELPER_LOG ("%s l %d dest %" PRIx64" src %" PRIx64"\n" , __func__, l, dest, src); -for (i =0 ; i <= l; i++) { -uint8_t x =cpu_ldub_data_ra (env, src + i, ra); - x &=cpu_ldub_data_ra (env, dest + i, ra); +/* NC always processes one more byte than specified - maximum is 256 */ + l++; + + srca1 =access_prepare (env, src, l,MMU_DATA_LOAD , ra); + srca2 =access_prepare (env, dest, l,MMU_DATA_LOAD , ra); + desta =access_prepare (env, dest, l,MMU_DATA_STORE , ra); +for (i =0 ; i < l; i++) { +const uint8_t x =access_get_byte (env, &srca1, i, ra) & +access_get_byte (env, &srca2, i, ra); + c |= x; -cpu_stb_data_ra (env, dest + i, x, ra); +access_set_byte (env, &desta, i, x, ra); }return c !=0 ; }
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 22/28] s390x/tcg: NC: Fault-safe handling
Quoted 6 lines · David Hildenbrand
We can process a maximum of 256 bytes, crossing two pages.
Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/mem_helper.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
r~
PATCH v2 23/28 s390x/tcg: MVCIN: Fault-safe handling
In reply to David HildenbrandWe can process a maximum of 256 bytes, crossing two pages. Calculate the accessed range upfront - src is accessed right-to-left.
--- target/s390x/mem_helper.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)
target/s390x/mem_helper.c +11 -3
@@ -473,12 +473,20 @@ void HELPER(mvc)(CPUS390XState *env, uint32_t l, uint64_t dest, uint64_t src)/* move inverse */ void HELPER (mvcin )(CPUS390XState *env,uint32_t l,uint64_t dest,uint64_t src) { +S390Access srca, desta;uintptr_t ra =GETPC ();int i; -for (i =0 ; i <= l; i++) { -uint8_t v =cpu_ldub_data_ra (env, src - i, ra); -cpu_stb_data_ra (env, dest + i, v, ra); +/* MVCIN always copies one more byte than specified - maximum is 256 */ + l++; + + src =wrap_address (env, src - l +1 ); + srca =access_prepare (env, src, l,MMU_DATA_LOAD , ra); + desta =access_prepare (env, dest, l,MMU_DATA_STORE , ra); +for (i =0 ; i < l; i++) { +const uint8_t x =access_get_byte (env, &srca, l - i -1 , ra); + +access_set_byte (env, &desta, i, x, ra); } }
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 23/28] s390x/tcg: MVCIN: Fault-safe handling
Quoted 7 lines · David Hildenbrand
We can process a maximum of 256 bytes, crossing two pages. Calculate the accessed range upfront - src is accessed right-to-left.
Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/mem_helper.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-)
r~
PATCH v2 24/28 s390x/tcg: MVN: Fault-safe handling
In reply to David HildenbrandWe can process a maximum of 256 bytes, crossing two pages.
--- target/s390x/mem_helper.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
target/s390x/mem_helper.c +12 -4
@@ -493,13 +493,21 @@ void HELPER(mvcin)(CPUS390XState *env, uint32_t l, uint64_t dest, uint64_t src)/* move numerics */ void HELPER (mvn )(CPUS390XState *env,uint32_t l,uint64_t dest,uint64_t src) { +S390Access srca1, srca2, desta;uintptr_t ra =GETPC ();int i; -for (i =0 ; i <= l; i++) { -uint8_t v =cpu_ldub_data_ra (env, dest + i, ra) &0xf0 ; - v |=cpu_ldub_data_ra (env, src + i, ra) &0x0f ; -cpu_stb_data_ra (env, dest + i, v, ra); +/* MVN always copies one more byte than specified - maximum is 256 */ + l++; + + srca1 =access_prepare (env, src, l,MMU_DATA_LOAD , ra); + srca2 =access_prepare (env, dest, l,MMU_DATA_LOAD , ra); + desta =access_prepare (env, dest, l,MMU_DATA_STORE , ra); +for (i =0 ; i < l; i++) { +const uint8_t x = (access_get_byte (env, &srca1, i, ra) &0x0f ) | + (access_get_byte (env, &srca2, i, ra) &0xf0 ); + +access_set_byte (env, &desta, i, x, ra); } }
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 24/28] s390x/tcg: MVN: Fault-safe handling
Quoted 6 lines · David Hildenbrand
We can process a maximum of 256 bytes, crossing two pages.
Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/mem_helper.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
r~
PATCH v2 25/28 s390x/tcg: MVZ: Fault-safe handling
In reply to David HildenbrandWe can process a maximum of 256 bytes, crossing two pages.
--- target/s390x/mem_helper.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
target/s390x/mem_helper.c +12 -4
@@ -545,13 +545,21 @@ void HELPER(mvo)(CPUS390XState *env, uint32_t l, uint64_t dest, uint64_t src)/* move zones */ void HELPER (mvz )(CPUS390XState *env,uint32_t l,uint64_t dest,uint64_t src) { +S390Access srca1, srca2, desta;uintptr_t ra =GETPC ();int i; -for (i =0 ; i <= l; i++) { -uint8_t b =cpu_ldub_data_ra (env, dest + i, ra) &0x0f ; - b |=cpu_ldub_data_ra (env, src + i, ra) &0xf0 ; -cpu_stb_data_ra (env, dest + i, b, ra); +/* MVZ always copies one more byte than specified - maximum is 256 */ + l++; + + srca1 =access_prepare (env, src, l,MMU_DATA_LOAD , ra); + srca2 =access_prepare (env, dest, l,MMU_DATA_LOAD , ra); + desta =access_prepare (env, dest, l,MMU_DATA_STORE , ra); +for (i =0 ; i < l; i++) { +const uint8_t x = (access_get_byte (env, &srca1, i, ra) &0xf0 ) | + (access_get_byte (env, &srca2, i, ra) &0x0f ); + +access_set_byte (env, &desta, i, x, ra); } }
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 25/28] s390x/tcg: MVZ: Fault-safe handling
Quoted 6 lines · David Hildenbrand
We can process a maximum of 256 bytes, crossing two pages.
Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/mem_helper.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
r~
PATCH v2 26/28 s390x/tcg: MVST: Fault-safe handling
In reply to David HildenbrandAccess at most single pages and document why. Using the access helpers might over-indicate watchpoints within the same page, I guess we can live with that.
--- target/s390x/mem_helper.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-)
target/s390x/mem_helper.c +16 -7
@@ -845,21 +845,30 @@ uint32_t HELPER(mvst)(CPUS390XState *env, uint64_t c, uint32_t r1, uint32_t r2) {const uint64_t d =get_address (env, r1);const uint64_t s =get_address (env, r2); +const int len =MIN (-(d |TARGET_PAGE_MASK ), -(s |TARGET_PAGE_MASK )); +S390Access srca, desta;uintptr_t ra =GETPC (); -uint32_t len; +int i;if (c &0xffffff00ull ) {s390_program_interrupt (env,PGM_SPECIFICATION ,ILEN_AUTO , ra); } c = c &0xff ; -/* Lest we fail to service interrupts in a timely manner, limit the -amount of work we're willing to do. For now, let's cap at 8k. */ -for (len =0 ; len <0x2000 ; ++len) { -uint8_t v =cpu_ldub_data_ra (env, s + len, ra); -cpu_stb_data_ra (env, d + len, v, ra); +/* +* Our access should not exceed single pages, as we must not report access +* exceptions exceeding the actually copied range (which we don't know at +* this point). We might over-indicate watchpoints within the pages +* (if we ever care, we have to limit processing to a single byte). +*/ + srca =access_prepare (env, s, len,MMU_DATA_LOAD , ra); + desta =access_prepare (env, d, len,MMU_DATA_STORE , ra); +for (i =0 ; i < len; i++) { +const uint8_t v =access_get_byte (env, &srca, i, ra); + +access_set_byte (env, &desta, i, v, ra);if (v == c) { -set_address_zero (env, r1, d + len); +set_address_zero (env, r1, d + i);return 1 ; } }
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 26/28] s390x/tcg: MVST: Fault-safe handling
Quoted 7 lines · David Hildenbrand
target/s390x/mem_helper.c +7
-0+/* +* Our access should not exceed single pages, as we must not report access +* exceptions exceeding the actually copied range (which we don't know at +* this point). We might over-indicate watchpoints within the pages +* (if we ever care, we have to limit processing to a single byte). +*/ + srca =access_prepare (env, s, len,MMU_DATA_LOAD , ra);
target/s390x/mem_helper.c uint32_t HELPER(mvst)(CPUS390XState *env, uint64_t c, uint32_t r1, uint32_t r2) · line 873
Quoted 10 lines
target/s390x/mem_helper.c +6
-1+ desta =access_prepare (env, d, len,MMU_DATA_STORE , ra); +for (i =0 ; i < len; i++) { +const uint8_t v =access_get_byte (env, &srca, i, ra); + +access_set_byte (env, &desta, i, v, ra);if (v == c) { -set_address_zero (env, r1, d + len); +set_address_zero (env, r1, d + i);return 1 ; }
Worth using memchr + memmove w/ nondestructive overlap?
That said,
r~
Re: [Qemu-devel] [PATCH v2 26/28] s390x/tcg: MVST: Fault-safe handling
Quoted 19 lines · Richard Henderson
Quoted 7 lines · David Hildenbrand
target/s390x/mem_helper.c +7
-0+/* +* Our access should not exceed single pages, as we must not report access +* exceptions exceeding the actually copied range (which we don't know at +* this point). We might over-indicate watchpoints within the pages +* (if we ever care, we have to limit processing to a single byte). +*/ + srca =access_prepare (env, s, len,MMU_DATA_LOAD , ra);target/s390x/mem_helper.c uint32_t HELPER(mvst)(CPUS390XState *env, uint64_t c, uint32_t r1, uint32_t r2) · line 873
Quoted 10 lines
target/s390x/mem_helper.c +6
-1+ desta =access_prepare (env, d, len,MMU_DATA_STORE , ra); +for (i =0 ; i < len; i++) { +const uint8_t v =access_get_byte (env, &srca, i, ra); + +access_set_byte (env, &desta, i, v, ra);if (v == c) { -set_address_zero (env, r1, d + len); +set_address_zero (env, r1, d + i);return 1 ; }
Quoted 1 line
Worth using memchr + memmove w/ nondestructive overlap?
In theory yes, however, the issue is that we would have multiple accesses, which is not documented for this instruction. In case the memory is modified between memchr + memmove by another CPU, we could have an inconsistent instruction result. Unlikely but possible :)
Quoted 6 lines
That said,
Reviewed-by Richard Henderson <[email protected]>r~
Signature · 3 lines
Thanks, David / dhildenb
PATCH v2 27/28 s390x/tcg: MVO: Fault-safe handling
In reply to David HildenbrandEach operand can have a maximum length of 16. Make sure to prepare all reads/writes before writing.
--- target/s390x/mem_helper.c | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-)
target/s390x/mem_helper.c +14 -12
@@ -514,31 +514,33 @@ void HELPER(mvn)(CPUS390XState *env, uint32_t l, uint64_t dest, uint64_t src)/* move with offset */ void HELPER (mvo )(CPUS390XState *env,uint32_t l,uint64_t dest,uint64_t src) { +/* MVO always processes one more byte than specified - maximum is 16 */ +const int len_dest = (l >>4 ) +1 ; +const int len_src = (l &0xf ) +1 ;uintptr_t ra =GETPC (); -int len_dest = l >>4 ; -int len_src = l &0xf ;uint8_t byte_dest, byte_src; -int i; +S390Access srca, desta; +int i, j; - src += len_src; - dest += len_dest; + srca =access_prepare (env, src, len_src,MMU_DATA_LOAD , ra); + desta =access_prepare (env, dest, len_dest,MMU_DATA_STORE , ra);/* Handle rightmost byte */ - byte_src =cpu_ldub_data_ra (env, src, ra); - byte_dest =cpu_ldub_data_ra (env, dest, ra); + byte_dest =cpu_ldub_data_ra (env, dest + len_dest -1 , ra); + byte_src =access_get_byte (env, &srca, len_src -1 , ra); byte_dest = (byte_dest &0x0f ) | (byte_src <<4 ); -cpu_stb_data_ra (env, dest, byte_dest, ra); +access_set_byte (env, &desta, len_dest -1 , byte_dest, ra);/* Process remaining bytes from right to left */ -for (i =1 ; i <= len_dest; i++) { +for (i = len_dest -2 , j = len_src -2 ; i >=0 ; i--, j--) { byte_dest = byte_src >>4 ; -if (len_src - i >=0 ) { - byte_src =cpu_ldub_data_ra (env, src - i, ra); +if (j >=0 ) { + byte_src =access_get_byte (env, &srca, j, ra); }else { byte_src =0 ; } byte_dest |= byte_src <<4 ; -cpu_stb_data_ra (env, dest - i, byte_dest, ra); +access_set_byte (env, &desta, i, byte_dest, ra); } }
Signature · 1 line
2.21.0
Re: [Qemu-devel] [PATCH v2 27/28] s390x/tcg: MVO: Fault-safe handling
Quoted 7 lines · David Hildenbrand
Each operand can have a maximum length of 16. Make sure to prepare all reads/writes before writing.
Signed-off-by David Hildenbrand <[email protected]>--- target/s390x/mem_helper.c | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-)
r~
PATCH v2 28/28 tests/tcg: target/s390x: Test MVO
In reply to David HildenbrandLet's add the simple test based on the example from the PoP.
--- tests/tcg/s390x/Makefile.target | 1 + tests/tcg/s390x/mvo.c | 25 +++++++++++++++++++++++++ 2 files changed, 26 insertions(+) create mode 100644 tests/tcg/s390x/mvo.c
tests/tcg/s390x/Makefile.target +1 -0
@@ -6,3 +6,4 @@ TESTS+=ipm TESTS+=exrl-trt TESTS+=exrl-trtr TESTS+=pack +TESTS+=mvo
tests/tcg/s390x/mvo.c new file +25 -0
@@ -0,0 +1,25 @@ +#include <stdint.h> +#include <stdio.h> + +int main (void ) +{ +uint8_t dest[6 ] = {0xff ,0x77 ,0x88 ,0x99 ,0x0c ,0xff }; +uint8_t src[5 ] = {0xee ,0x12 ,0x34 ,0x56 ,0xee }; +uint8_t expected[6 ] = {0xff ,0x01 ,0x23 ,0x45 ,0x6c ,0xff }; +int i; + + asmvolatile ( +" mvo 0(4,%[dest]),0(3,%[src])\n" + : + : [dest]"d" (dest +1 ), + [src]"d" (src +1 ) + :"memory" ); + +for (i =0 ; i <sizeof (expected); i++) { +if (dest[i] != expected[i]) { +fprintf (stderr,"bad data\n" ); +return 1 ; + } + } +return 0 ; +}
Signature · 1 line
2.21.0
Quoted 72 lines · David Hildenbrand
This is the successor of
"[PATCH v1 0/4] s390x/tcg: MOVE (MVC): Fault-safe handling"----
This series fixes a bunch of issues related to some mem helpers and makes sure that they are fault-safe, meaning no system state is modified in case a fault is triggered.
I can spot tons of other issues with other mem helpers that will have to be fixed later. Also, fault-safe handling for some instructions (especially TR) might be harder to implement (you don't know what will actually be accessed upfront - we might need a buffer and go over inputs twice). Focusing on the MOVE instructions for now.
----
Newer versions of glibc use
memcpy()inmemmove()for forward moves. The implementation makese use of MVC. The TCG implementation of MVC is currently not able to handle faults reliably when crossing pages. MVC can cross with 256 bytes at most two pages.In case we get a fault on the second page, we already moved data. When continuing after the fault we might try to move already overwritten data, which is very bad in case we have overlapping data on a forward move.
Triggered for now only by rpmbuild (crashes when checking the spec file) and rpm (database corruptions). This fixes installing Fedora rawhide (31) under TCG.
This was horrible to debug as it barely triggers and we fail at completely different places.
Cc Stefano Brivio <[email protected]>Cc Florian Weimer <[email protected]>v1 -> v2:
- Include many fixes
- Fix more instructions
- Use the newprobe_access()function
- Include "tests/tcg: target/s390x: Test MVO"David Hildenbrand (28):
s390x/tcg: Resetexception_indexto -1 instead of 0 s390x/tcg: MVCL: Zero out unused bits of address s390x/tcg: MVCL: Detect destructive overlaps s390x/tcg: MVCL: Process max 2k bytes at a time s390x/tcg: MVC: Increment the length once s390x/tcg: MVC: Useis_destructive_overlap()s390x/tcg: MVPG: Check for specification exceptions s390x/tcg: MVPG: Properly wrap the addresses s390x/tcg: MVCLU/MVCLE: Process max 4k bytes at a time s390x/tcg: MVCS/MVCP: Check for special operation exceptions s390x/tcg: MVCS/MVCP: Properly wrap the length s390x/tcg: MVST: Check for specification exceptions s390x/tcg: MVST: Fix storing back the addresses to registers s390x/tcg: Always useMMU_USER_IDXforCONFIG_USER_ONLYs390x/tcg: Fault-safe memset
s390x/tcg: Fault-safe memmove
s390x/tcg: MVCS/MVCP: Useaccess_memmove_idx()s390x/tcg: MVC: Fault-safe handling on destructive overlaps s390x/tcg: MVCLU: Fault-safe handling
s390x/tcg: OC: Fault-safe handling
s390x/tcg: XC: Fault-safe handling
s390x/tcg: NC: Fault-safe handling
s390x/tcg: MVCIN: Fault-safe handling
s390x/tcg: MVN: Fault-safe handling
s390x/tcg: MVZ: Fault-safe handling
s390x/tcg: MVST: Fault-safe handling
s390x/tcg: MVO: Fault-safe handling
tests/tcg: target/s390x: Test MVO
Quoted 10 lines
target/s390x/cpu.h | 4 + target/s390x/helper.h | 2 +- target/s390x/insn-data.def | 2 +- target/s390x/mem_helper.c | 672 ++++++++++++++++++++++---------- target/s390x/translate.c | 12 +- tests/tcg/s390x/Makefile.target | 1 + tests/tcg/s390x/mvo.c | 25 ++ 7 files changed, 507 insertions(+), 211 deletions(-) create mode 100644 tests/tcg/s390x/mvo.c
I guess, for a minimal review besides the obvious fixes, looking at
"s390x/tcg: Fault-safe memset" and "s390x/tcg: Fault-safe memmove" and "s390x/tcg: MVC: Fault-safe handling on destructive overlaps" makes sense. They contain the real magic used within remaining patches.
Signature · 3 lines
Thanks, David / dhildenb