Return-Path: Received: from mail.bisel.ru (mail.bisel.ru [80.93.56.210]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.saout.de (Postfix) with ESMTPS for ; Tue, 16 Nov 2010 12:32:33 +0100 (CET) Date: Tue, 16 Nov 2010 14:32:25 +0300 From: Igor Novgorodov Message-ID: <1373448030.20101116143225@novg.net> In-Reply-To: <4CE24E02.8030608@redhat.com> References: <566573504.20101115153604@novg.net> <4CE142C3.1070408@redhat.com> <737699007.20101116103102@novg.net> <4CE24E02.8030608@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [dm-crypt] AES-XTS performance Reply-To: Igor Novgorodov List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Milan Broz Cc: dm-crypt@saout.de =D0=97=D0=B4=D1=80=D0=B0=D0=B2=D1=81=D1=82=D0=B2=D1=83=D0=B9=D1=82=D0=B5, M= ilan. =D0=92=D1=8B =D0=BF=D0=B8=D1=81=D0=B0=D0=BB=D0=B8 16 =D0=BD=D0=BE=D1=8F=D0= =B1=D1=80=D1=8F 2010 =D0=B3., 12:25:22: > On 11/16/2010 08:31 AM, Igor Novgorodov wrote: >> I've tried to apply this patch against 2.6.36 and 2.6.37-rc2 >> but without success: >>=20 >> # patch -p1 -i ./dm-crypt-patch.diff > It applies cleanly to 2.6.37-rc1 and above. It does not. I've tried 2.6.37-rc2 as i mentioned. Copied & pasted the patch from your LKML post. > For 2.6.36 small change is needed. > You can download both versions here for now: > http://mbroz.fedorapeople.org/dm-crypt/ Thanks, this patch for 2.6.36 applied fine. Setup: # echo '0 1000000000 zero' | dmsetup create zero1 # cryptsetup --cipher aes-xts-essiv:sha256 --key-size 512 --hash sha512 cre= ate c /dev/mapper/zero1 Benchmark with pure 2.6.36: # dd if=3D/dev/mapper/c of=3D/dev/null bs=3D1M count=3D30000 30000+0 records in 30000+0 records out 31457280000 bytes (31 GB) copied, 128.674 s, 244 MB/s # dd of=3D/dev/mapper/c if=3D/dev/zero bs=3D1M count=3D30000 30000+0 records in 30000+0 records out 31457280000 bytes (31 GB) copied, 129.512 s, 243 MB/s Benchmark with your patch: Read speed increased ~20% # dd if=3D/dev/mapper/c of=3D/dev/null bs=3D1M count=3D30000 30000+0 records in 30000+0 records out 31457280000 bytes (31 GB) copied, 108.363 s, 290 MB/s And according to `top` all kworker threads were runnning, but they were using only a portion CPUs time: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2691 root 20 0 0 0 0 S 36 0.0 0:37.11 kworker/1:2 2784 root 20 0 0 0 0 S 26 0.0 0:04.59 kworker/5:1 2746 root 20 0 0 0 0 S 19 0.0 0:41.36 kworker/2:0 2780 root 20 0 9304 1400 516 D 19 0.0 0:07.91 dd 2781 root 20 0 0 0 0 S 17 0.0 0:07.36 kworker/6:1 2039 root 20 0 0 0 0 R 17 0.0 1:01.45 kworker/4:2 2350 root 20 0 0 0 0 S 15 0.0 0:43.15 kworker/0:2 2783 root 20 0 0 0 0 S 2 0.0 0:00.38 kworker/1:0 2750 root 20 0 0 0 0 S 2 0.0 0:22.71 kworker/5:0 2686 root 20 0 0 0 0 S 1 0.0 0:43.79 kworker/2:2 2695 root 20 0 0 0 0 S 1 0.0 0:42.46 kworker/6:2 2782 root 20 0 0 0 0 S 1 0.0 0:02.38 kworker/4:0 2752 root 20 0 0 0 0 S 1 0.0 0:20.96 kworker/0:1 But with the write speed we get a VERY HUGE speed improvement!: # dd of=3D/dev/mapper/c if=3D/dev/zero bs=3D1M count=3D30000 30000+0 records in 30000+0 records out 31457280000 bytes (31 GB) copied, 37.2707 s, 844 MB/s This time the threads are loaded much more: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2690 root 20 0 0 0 0 R 101 0.0 0:28.17 kworker/7:2 2750 root 20 0 0 0 0 R 101 0.0 0:43.14 kworker/5:0 2752 root 20 0 0 0 0 R 99 0.0 0:43.01 kworker/0:1 2695 root 20 0 0 0 0 R 91 0.0 1:05.82 kworker/6:2 2791 root 20 0 9304 1404 516 R 46 0.0 0:17.12 dd 2748 root 20 0 0 0 0 S 34 0.0 0:19.76 kworker/3:0 2746 root 20 0 0 0 0 R 28 0.0 1:08.36 kworker/2:0 2039 root 20 0 0 0 0 S 10 0.0 1:33.78 kworker/4:2 2783 root 20 0 0 0 0 S 10 0.0 0:28.28 kworker/1:0 2686 root 20 0 0 0 0 R 2 0.0 0:47.27 kworker/2:2 Such big difference between read and write speed is by design? Or it is a problem of my setup? Maybe i should try to disable hyperthreading and check again? Anyway, it's a huge step forward from vanilla kernel's performance, thanks! I'll keep testing this, and if any new patches will come out, i'll gladly try them out too.