This patch provides AVX512 support for xbzrle_encode_buffer function
cover This patch provides AVX512 support for xbzrle_encode_buffer function
This patch adds avx512 support for xbzrle_encode_buffer function to accelerate xbzrle encoding speed. The specific avx512 implementation is provided in qemu/migration/xbzrle.c.
We provide AVX512 intrinsic support in qemu/configure file to enable avx512 in complier. And added unit test called "test_encode_decode_random" is provided in qemu/tests/u nits/test-xbzrle.c.
ling xu (1):
Add AVX512 support for xbzrle_encode_buffer function
configure | 434 ++++++++++++++++++++++++++++++++++++++- migration/ram.c | 6 + migration/xbzrle.c | 177 ++++++++++++++++ migration/xbzrle.h | 4 + tests/unit/test-xbzrle.c | 307 +++++++++++++++++++++++++-- 5 files changed, 908 insertions(+), 20 deletions(-)
Signature · 1 line
2.25.1
PATCH 1/1 Add AVX512 support for xbzrle_encode_buffer function
This commit adds AVX512 implementation of xbzrle_encode_buffer function to accelerate xbzrle encoding speed. Compared with C version of xbzrle_encode_buffer function, AVX512 version can achieve almost 60%-70% performance improvement on unit test provided by qemu. In addition, we provide one more unit test called "test_encode_decode_random", in which dirty data are randomly located in 4K page, and this case can achieve almost 140% performance gain.
--- configure | 434 ++++++++++++++++++++++++++++++++++++++- migration/ram.c | 6 + migration/xbzrle.c | 177 ++++++++++++++++ migration/xbzrle.h | 4 + tests/unit/test-xbzrle.c | 307 +++++++++++++++++++++++++-- 5 files changed, 908 insertions(+), 20 deletions(-)
configure +431 -3
@@ -587,6 +587,431 @@ else cpu=$(uname -m) fi +# cpu flag for x86 +x86_cpu_flags="" + +get_x86_cpu_flags(){ + # check sse flag + cat > $TMPC << EOF +#include <immintrin.h> +int main(int argc, char *argv[]) +{ + __m128 a,b; + int res= _mm_ucomieq_ss(a,b); + return 0; +} +EOF + local_cpu_flag="-msse" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -msse" + fi + + # check sse2 flag + cat > $TMPC << EOF +#include <emmintrin.h> +int main(int argc, char *argv[]) +{ + __m128d a,b; + int res= _mm_ucomineq_sd (a, b); + return 0; +} +EOF + local_cpu_flag="-msse2" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -msse2" + fi + + # check sse3 flag + cat > $TMPC << EOF +#include <pmmintrin.h> +int main(int argc, char *argv[]) +{ + __m128 a; + __m128 res= _mm_moveldup_ps (a); + return 0; +} +EOF + local_cpu_flag="-msse3" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -msse3" + fi + + # check ssse3 flag + cat > $TMPC << EOF +#include <tmmintrin.h> +int main(int argc, char *argv[]) +{ + __m128i a; + __m12i8 res= _mm_abs_epi32 (a); + return 0; +} +EOF + local_cpu_flag="-mssse3" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -mssse3" + fi + + # check sse4.1 flag + cat > $TMPC << EOF +#include <smmintrin.h> +int main(int argc, char *argv[]) +{ + __m128 a; + __m128 res=_mm_ceil_ps (a); + return 0; +} +EOF + local_cpu_flag="-msse4.1" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -msse4.1" + fi + + # check sse4.2 flag + cat > $TMPC << EOF +#include <nmmintrin.h> +int main(int argc, char *argv[]) +{ + __m128i a,b; + __m128i res=_mm_cmpgt_epi64 (a,b); + return 0; +} +EOF + local_cpu_flag="-msse4.2" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -msse4.2" + fi + + # check avx flag + cat > $TMPC << EOF +#include <immintrin.h> +int main(int argc, char *argv[]) +{ + __m256 a; + __m256 res= _mm256_ceil_ps(a); + return 0; +} +EOF + local_cpu_flag="-mavx" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -mavx" + fi + + # check avx2 flag + cat > $TMPC << EOF +#include <immintrin.h> +int main(int argc, char *argv[]) +{ + __m256i a; + __m256i res= _mm256_abs_epi32(a); + return 0; +} +EOF + local_cpu_flag="-mavx2" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -mavx2" + fi + + # check AVX_VNNI flag + cat > $TMPC << EOF +#include <immintrin.h> +int main(int argc, char *argv[]) +{ + __m128i a,b,c; + __m128i res= _mm_dpbusd_epi32(a,b,c); + return 0; +} +EOF + local_cpu_flag="-mavxvnni" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -mavxvnni" + fi + + # check AVX512f flag + cat > $TMPC << EOF +#include <immintrin.h> +int main(int argc, char *argv[]) +{ + __m128d a; + __m512d res= _mm512_broadcastsd_pd(a); + return 0; +} +EOF + local_cpu_flag="-mavx512f" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -mavx512f" + fi + + # check AVX512bw flag + cat > $TMPC << EOF +#include <immintrin.h> +int main(int argc, char *argv[]) +{ + __m512i a; + __m512i res= _mm512_abs_epi8(a); + return 0; +} +EOF + local_cpu_flag="-mavx512bw" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -mavx512bw" + fi + + # check AVX512CD flag + cat > $TMPC << EOF +#include <immintrin.h> +int main(int argc, char *argv[]) +{ + __m512i a; + __mmask16 k; + __m512i res= _mm512_maskz_lzcnt_epi32 (k, a); + return 0; +} +EOF + local_cpu_flag="-mavx512cd" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -mavx512cd" + fi + + # check AVX512DQ flag + cat > $TMPC << EOF +#include <immintrin.h> +int main(int argc, char *argv[]) +{ + __mmask8 a,b; + __mmask8 res=_kxor_mask8(a,b); + return 0; +} +EOF + local_cpu_flag="-mavx512dq" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -mavx512dq" + fi + + # check AVX512ER flag + cat > $TMPC << EOF +#include <immintrin.h> +int main(int argc, char *argv[]) +{ + __m512d a; + __m512d res=_mm512_rsqrt28_pd(a); + return 0; +} +EOF + local_cpu_flag="-mavx512er" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -mavx512er" + fi + + # check AVX512IFMA52 flag + cat > $TMPC << EOF +#include <immintrin.h> +int main(int argc, char *argv[]) +{ + __m512i a,b,c; + __m512i res= _mm512_madd52lo_epu64 (a, b, c); + return 0; +} +EOF + local_cpu_flag="-mavx512ifma" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -mavx512ifma" + fi + + # check AVX512PF flag + cat > $TMPC << EOF +#include <immintrin.h> +int main(int argc, char *argv[]) +{ + char* base_addr; + __mmask8 k; + __m512i vindex; + _mm512_mask_prefetch_i64scatter_pd (base_addr, k, vindex, 1, 2); + return 0; +} +EOF + local_cpu_flag="-mavx512pf" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -mavx512pf" + fi + + # check AVX512VPOPCNTDQ flag + cat > $TMPC << EOF +#include <immintrin.h> +int main(int argc, char *argv[]) +{ + __m512i a; + __mmask8 k; + __m512i res= _mm512_maskz_popcnt_epi64(k,a); + return 0; +} +EOF + local_cpu_flag="-mavx512vpopcntdq" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -mavx512vpopcntdq" + fi + + # check AVX512_4VNNIW flag + cat > $TMPC << EOF +#include <immintrin.h> +int main(int argc, char *argv[]) +{ + __m512i a,b,c,d,e; + __m128 g; + __m512i res= _mm512_4dpwssd_epi32 (a, b, c, d, e, &g); + return 0; +} +EOF + local_cpu_flag="-mavx5124vnniw" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -mavx5124vnniw" + fi + + # check AVX512_BITALG flag + cat > $TMPC << EOF +#include <immintrin.h> +int main(int argc, char *argv[]) +{ + __m512i a; + __m512i res= _mm512_popcnt_epi16 (a); + return 0; +} +EOF + local_cpu_flag="-mavx512bitalg" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -mavx512bitalg" + fi + + # check AVX512_VBMI flag + cat > $TMPC << EOF +#include <immintrin.h> +int main(int argc, char *argv[]) +{ + __m512i a,b,c; + __m512i res= _mm512_permutex2var_epi8 (a, b, c); + return 0; +} +EOF + local_cpu_flag="-mavx512vbmi" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -mavx512vbmi" + fi + + # check AVX512_VBMI2 flag + cat > $TMPC << EOF +#include <immintrin.h> +int main(int argc, char *argv[]) +{ + __m512i a,b,c; + __m512i res= _mm512_shrdv_epi64 (a, b, c); + return 0; +} +EOF + local_cpu_flag="-mavx512vbmi2" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -mavx512vbmi2" + fi + + # check AVX512_VNNI flag + cat > $TMPC << EOF +#include <immintrin.h> +int main(int argc, char *argv[]) +{ + __m512i a,b,c; + __mmask16 k; + __m512i res= _mm512_maskz_dpwssds_epi32 (k,a, b, c); + return 0; +} +EOF + local_cpu_flag="-mavx512vnni" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -mavx512vnni" + fi + + # check AVX512_FP16 flag + cat > $TMPC << EOF +#include <immintrin.h> +int main(int argc, char *argv[]) +{ + __m128h a; + __m128 res= _mm_castph_ps (a); + return 0; +} +EOF + local_cpu_flag="-mavx512fp16" + rm -f $TMPE + do_cc $local_cpu_flag -o $TMPE $TMPC + if [ -e $TMPE ] + then + x86_cpu_flags="$x86_cpu_flags -mavx512fp16" + fi +} + # Normalise host CPU name, set multilib cflags # Note that this case should only have supported host CPUs, not guests. case "$cpu" in
@@ -595,16 +1020,19 @@ case "$cpu" in i386|i486|i586|i686|i86pc|BePC) cpu="i386" - CPU_CFLAGS="-m32" ;; + get_x86_cpu_flags + CPU_CFLAGS="-m32 $x86_cpu_flags" ;; x32) cpu="x86_64" - CPU_CFLAGS="-mx32" ;; + get_x86_cpu_flags + CPU_CFLAGS="-mx32 $x86_cpu_flags" ;; x86_64|amd64) cpu="x86_64" + get_x86_cpu_flags # ??? Only extremely old AMD cpus do not have cmpxchg16b. # If we truly care, we should simply detect this case at # runtime and generate the fallback to serial emulation. - CPU_CFLAGS="-m64 -mcx16" ;; + CPU_CFLAGS="-m64 -mcx16 $x86_cpu_flags" ;; mips*) cpu="mips" ;;
migration/ram.c +6 -0
@@ -747,9 +747,15 @@ static int save_xbzrle_page(RAMState *rs, uint8_t **current_data,memcpy (XBZRLE .current_buf, *current_data,TARGET_PAGE_SIZE );/* XBZRLE encoding (if there is no overflow) */ +#if defined(__x86_64__) && defined(__AVX512BW__) + encoded_len =xbzrle_encode_buffer_512 (prev_cached_page,XBZRLE .current_buf, +TARGET_PAGE_SIZE ,XBZRLE .encoded_buf, +TARGET_PAGE_SIZE ); +#else encoded_len =xbzrle_encode_buffer (prev_cached_page,XBZRLE .current_buf,TARGET_PAGE_SIZE ,XBZRLE .encoded_buf,TARGET_PAGE_SIZE ); +#endif /* * Update the cache contents, so that it corresponds to the data
migration/xbzrle.c +177 -0
180 lines not on this page · raw message
migration/xbzrle.h +4 -0
@@ -18,4 +18,8 @@ int xbzrle_encode_buffer(uint8_t *old_buf, uint8_t *new_buf, int slen, uint8_t *dst, int dlen);int xbzrle_decode_buffer (uint8_t *src,int slen,uint8_t *dst,int dlen); +#if defined(__x86_64__) && defined(__AVX512BW__) +int xbzrle_encode_buffer_512 (uint8_t *old_buf,uint8_t *new_buf,int slen, +uint8_t *dst,int dlen); +#endif #endif
tests/unit/test-xbzrle.c +290 -17
440 lines not on this page · raw message
Signature · 1 line
2.25.1
Re: [PATCH 1/1] Add AVX512 support for xbzrle_encode_buffer function
Quoted 24 lines · ling xu
This commit adds AVX512 implementation of
xbzrle_encode_bufferfunction to accelerate xbzrle encoding speed. Compared with C version ofxbzrle_encode_bufferfunction, AVX512 version can achieve almost 60%-70% performance improvement on unit test provided by qemu. In addition, we provide one more unit test called "test_encode_decode_random", in which dirty data are randomly located in 4K page, and this case can achieve almost 140% performance gain.Signed-off-by ling xu <[email protected]>Co-authored-by Zhou Zhao <[email protected]>Co-authored-by Jun Jin <[email protected]>--- configure | 434 ++++++++++++++++++++++++++++++++++++++- migration/ram.c | 6 + migration/xbzrle.c | 177 ++++++++++++++++ migration/xbzrle.h | 4 + tests/unit/test-xbzrle.c | 307 +++++++++++++++++++++++++-- 5 files changed, 908 insertions(+), 20 deletions(-)migration/ram.c +0
-0@@ -747,9 +747,15 @@ static int save_xbzrle_page(RAMState *rs, uint8_t **current_data,memcpy (XBZRLE .current_buf, *current_data,TARGET_PAGE_SIZE );
migration/ram.c static int save_xbzrle_page(RAMState *rs, uint8_t **current_data, · line 758
Quoted 10 lines
migration/ram.c +6
-0/* XBZRLE encoding (if there is no overflow) */ +#if defined(__x86_64__) && defined(__AVX512BW__) + encoded_len =xbzrle_encode_buffer_512 (prev_cached_page,XBZRLE .current_buf, +TARGET_PAGE_SIZE ,XBZRLE .encoded_buf, +TARGET_PAGE_SIZE ); +#else encoded_len =xbzrle_encode_buffer (prev_cached_page,XBZRLE .current_buf,TARGET_PAGE_SIZE ,XBZRLE .encoded_buf,TARGET_PAGE_SIZE ); +#endif
Shouldn't we be deciding which impl using a runtime check of the current CPUID, rather than a compile time check ? I'm thinking along the lines of what util/bufferiszero.c does to select different optimized versions based on CPUID. The build host CPU features can't be expected to match the runtime host CPU features.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
RE: [PATCH 1/1] Add AVX512 support for xbzrle_encode_buffer function
Hi dainel:
Cause our code depend on intel intrinsics lib implement. And this lib depend on macro like " AVX512BW ". This macro need compile time check to enable some machine options . if you only use that utility to do runtime check ,you will met compile issue. And also if we want to save cpu time , we'd better check it in compile time.
-----Original Message-----
From: Daniel P. Berrangé <[email protected]>
Sent: Thursday, July 21, 2022 11:11 PM
To: Xu, Ling1 <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; Zhao, Zhou <[email protected]>; Jin, Jun I <[email protected]>
Subject: Re: [PATCH 1/1] Add AVX512 support for xbzrle_encode_buffer function
Quoted 25 lines · ling xu
This commit adds AVX512 implementation of
xbzrle_encode_buffer
function to accelerate xbzrle encoding speed. Compared with C version
ofxbzrle_encode_bufferfunction,
AVX512 version can achieve almost 60%-70% performance improvement on unit test provided by qemu.
In addition, we provide one more unit test called
"test_encode_decode_random", in which dirty data are randomly located in 4K page, and this case can achieve almost 140% performance gain.Signed-off-by ling xu <[email protected]>Co-authored-by Zhou Zhao <[email protected]>Co-authored-by Jun Jin <[email protected]>--- configure | 434 ++++++++++++++++++++++++++++++++++++++- migration/ram.c | 6 + migration/xbzrle.c | 177 ++++++++++++++++ migration/xbzrle.h | 4 + tests/unit/test-xbzrle.c | 307 +++++++++++++++++++++++++-- 5 files changed, 908 insertions(+), 20 deletions(-)migration/ram.c index renamed from migration/ram.c +0
-0
01f9cc1d72..3b931c325f100644migration/ram.c +0
-0@@ -747,9 +747,15 @@ static int save_xbzrle_page(RAMState *rs, uint8_t **current_data,memcpy (XBZRLE .current_buf, *current_data,TARGET_PAGE_SIZE );
migration/ram.c static int save_xbzrle_page(RAMState *rs, uint8_t **current_data, · line 758
Quoted 10 lines
migration/ram.c +6
-0/* XBZRLE encoding (if there is no overflow) */ +#if defined(__x86_64__) && defined(__AVX512BW__) + encoded_len =xbzrle_encode_buffer_512 (prev_cached_page,XBZRLE .current_buf, +TARGET_PAGE_SIZE ,XBZRLE .encoded_buf, +TARGET_PAGE_SIZE ); +#else encoded_len =xbzrle_encode_buffer (prev_cached_page,XBZRLE .current_buf,TARGET_PAGE_SIZE ,XBZRLE .encoded_buf,TARGET_PAGE_SIZE ); +#endif
Shouldn't we be deciding which impl using a runtime check of the current CPUID, rather than a compile time check ? I'm thinking along the lines of what util/bufferiszero.c does to select different optimized versions based on CPUID. The build host CPU features can't be expected to match the runtime host CPU features.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH 1/1] Add AVX512 support for xbzrle_encode_buffer function
Quoted 2 lines · * Zhao, Zhou ([email protected]) wrote:
Hi dainel:
Cause our code depend on intel intrinsics lib implement. And this lib depend on macro like " AVX512BW ". This macro need compile time check to enable some machine options . if you only use that utility to do runtime check ,you will met compile issue. And also if we want to save cpu time , we'd better check it in compile time.
You need to do *both*:
a) You need to check at compile time to see if you have the
intrinsics.
b) You need to check at runtime to see if you're running on a suitable
CPU.
Other things to note (I've not checked the algorithm yet):
c) The patch needs splitting up into compile checks, the algorithm,
the tests as at least 3 patches.
d) The test includes a benchmark, we don't need to include a benchmark
program in the code, just something to check it works.
e) The benchmark is a microbenchmark on the routine; what's it's
effect on the whole migration - is it significant?
f) xbzrle isn't actually used that much these days, so I'm not sure
generally it's worth it.
Dave
Quoted 54 lines
-----Original Message-----
From: Daniel P. Berrangé <[email protected]>
Sent: Thursday, July 21, 2022 11:11 PM
To: Xu, Ling1 <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; Zhao, Zhou <[email protected]>; Jin, Jun I <[email protected]>
Subject: Re: [PATCH 1/1] Add AVX512 support forxbzrle_encode_bufferfunctionQuoted 25 lines · ling xu
This commit adds AVX512 implementation of
xbzrle_encode_buffer
function to accelerate xbzrle encoding speed. Compared with C version
ofxbzrle_encode_bufferfunction,
AVX512 version can achieve almost 60%-70% performance improvement on unit test provided by qemu.
In addition, we provide one more unit test called
"test_encode_decode_random", in which dirty data are randomly located in 4K page, and this case can achieve almost 140% performance gain.Signed-off-by ling xu <[email protected]>Co-authored-by Zhou Zhao <[email protected]>Co-authored-by Jun Jin <[email protected]>--- configure | 434 ++++++++++++++++++++++++++++++++++++++- migration/ram.c | 6 + migration/xbzrle.c | 177 ++++++++++++++++ migration/xbzrle.h | 4 + tests/unit/test-xbzrle.c | 307 +++++++++++++++++++++++++-- 5 files changed, 908 insertions(+), 20 deletions(-)migration/ram.c index renamed from migration/ram.c +0
-0
01f9cc1d72..3b931c325f100644migration/ram.c +0
-0@@ -747,9 +747,15 @@ static int save_xbzrle_page(RAMState *rs, uint8_t **current_data,memcpy (XBZRLE .current_buf, *current_data,TARGET_PAGE_SIZE );migration/ram.c static int save_xbzrle_page(RAMState *rs, uint8_t **current_data, · line 758
Quoted 10 lines
migration/ram.c +6
-0/* XBZRLE encoding (if there is no overflow) */ +#if defined(__x86_64__) && defined(__AVX512BW__) + encoded_len =xbzrle_encode_buffer_512 (prev_cached_page,XBZRLE .current_buf, +TARGET_PAGE_SIZE ,XBZRLE .encoded_buf, +TARGET_PAGE_SIZE ); +#else encoded_len =xbzrle_encode_buffer (prev_cached_page,XBZRLE .current_buf,TARGET_PAGE_SIZE ,XBZRLE .encoded_buf,TARGET_PAGE_SIZE ); +#endif Shouldn't we be deciding which impl using a runtime check of the current CPUID, rather than a compile time check ? I'm thinking along the lines of what
util/bufferiszero.cdoes to select different optimized versions based on CPUID. The build host CPU features can't be expected to match the runtime host CPU features.With regards,
DanielSignature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
Re: [PATCH 1/1] Add AVX512 support for xbzrle_encode_buffer function
In reply to Zhao, ZhouQuoted 7 lines · Zhou
Hi dainel:
Cause our code depend on intel intrinsics lib implement. And
this lib depend on macro like " AVX512BW ". This macro need compile time check to enable some machine options . if you only use that utility to do runtime check ,you will met compile issue. And also if we want to save cpu time , we'd better check it in compile time.
Can you give more info about this library ? The patch isn't linking to any new library so it was not immediately obvious that this patch even introduced a dependancy on an external library. Preferrably the library would be using 'pkg-config' so we can check for it in a single line of meson.build code instead of writing 100's of lines of configure compile tests.
Generally if the library is available in Linux dijstros, then we'll want the CI to test that too, which will mean updating the dockerfiles.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
RE: [PATCH 1/1] Add AVX512 support for xbzrle_encode_buffer function
1 we need gcc machined options to tirgger gcc enable SIMD instructions(https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html). Or it will trigger a compile error. This is the reason why we change the config file.
[cid:[email protected]]
$ gcc -mavx2 -dM -E - < /dev/null | egrep "SSE|AVX" | sort #define __AVX__ 1 #define __AVX2__ 1 #define __SSE__ 1 #define __SSE2__ 1 #define __SSE2_MATH__ 1 #define __SSE3__ 1 #define __SSE4_1__ 1 #define __SSE4_2__ 1 #define __SSE_MATH__ 1 #define __SSSE3__ 1
2 we are not introducing new intel library actually(intel already contribute our intric code to gcc https://github.com/gcc-mirror/gcc/blob/master/gcc/config/i386/immintrin.h), we just check if gcc can compile avx512 code during compiling, agree to add a runtime check will be better, let me add some code there
-----Original Message-----
From: Daniel P. Berrangé <[email protected]>
Sent: Friday, July 22, 2022 12:42 AM
To: Zhao, Zhou <[email protected]>
Cc: Xu, Ling1 <[email protected]>; [email protected]; [email protected]; [email protected]; Jin, Jun I <[email protected]>
Subject: Re: [PATCH 1/1] Add AVX512 support for xbzrle_encode_buffer function
Quoted 6 lines · Zhou
Hi dainel:
Cause our code depend on intel intrinsics lib implement. And this
lib depend on macro like " AVX512BW ". This macro need compile time
Quoted 7 lines
check to enable some machine options . if you only use that utility to
do runtime check ,you will met compile issue.
And also if we want to save cpu time , we'd better check it in compile
time.
Can you give more info about this library ? The patch isn't linking to any new library so it was not immediately obvious that this patch even introduced a dependancy on an external library. Preferrably the library would be using 'pkg-config'
so we can check for it in a single line of meson.build code instead of writing 100's of lines of configure compile tests.
Generally if the library is available in Linux dijstros, then we'll want the CI to test that too, which will mean updating the dockerfiles.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH 1/1] Add AVX512 support for xbzrle_encode_buffer function
Quoted 4 lines · Zhou
1 we need gcc machined options to tirgger gcc enable SIMD instructions(https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html). Or it will trigger a compile error. This is the reason why we change the config file.
Take a look at util/bufferiszero.c - that uses '#pragma GCC target'
to selectively change the compule target of individual functions.
This is what is njeeded here, because most distros are going to compile QEMU with a very old x86_64 baseline, never turn on avx512 for the whole QEMU build. It needs to be very selective to make it usable for typical deployments.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|