Skip to contentSkip to conversation

iozone write 50% regression in kernel 2.6.24-rc1

DISCUSSION Martin Knoblauch 2 participants 2 messages Nov 12, 2007 – Nov 13, 2007

lore
Share & tools
lore
#0 lore raw

----- Original Message ----

Quoted 6 lines

From: "Zhang, Yanmin" <[email protected]>
To: Martin Knoblauch <[email protected]>
Cc: [email protected]; LKML <[email protected]>
Sent: Monday, November 12, 2007 1:45:57 AM
Subject: Re: iozone write 50% regression in kernel 2.6.24-rc1

Quoted 20 lines
Quoted 18 lines · Martin Knoblauch

----- Original Message ----

Quoted 9 lines

From: "Zhang, Yanmin"
To: [email protected]
Cc: LKML
Sent: Friday, November 9, 2007 10:47:52 AM
Subject: iozone write 50% regression in kernel 2.6.24-rc1

Comparing with 2.6.23, iozone sequential write/rewrite (512M) has 50%

regression

Quoted 5 lines

in kernel 2.6.24-rc1. 2.6.24-rc2 has the same regression.

My machine has 8 processor cores and 8GB memory.

By bisect, I located patch

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h

Quoted 17 lines

=

Quoted 16 lines
Quoted 6 lines

04fbfdc14e5f48463820d6b9807daa5e9c92c51f.

Another behavior: with kernel 2.6.23, if I run iozone for many times

after rebooting machine,

Quoted 3 lines

the result looks stable. But with 2.6.24-rc1, the first run of iozone

got a very small result and

Quoted 5 lines

following run has 4Xorig_result.

What I reported is the regression of 2nd/3rd run, because first run has

Quoted 9 lines
Quoted 7 lines

bigger regression.

Quoted 4 lines

I also tried to change
/proc/sys/vm/dirty_ratio,dirty_backgroud_ratio

and didn't get improvement.
could you tell us the exact iozone command you are using?

iozone -i 0 -r 4k -s 512m

OK, I definitely do not see the reported effect. On a HP Proliant with a RAID5 on CCISS I get:

2.6.19.2: 654-738 MB/sec write, 1126-1154 MB/sec rewrite 2.6.24-rc2: 772-820 MB/sec write, 1495-1539 MB/sec rewrite

The first run is always slowest, all subsequent runs are faster and the same speed.

Quoted 4 lines
Quoted 2 lines

I would like to repeat it on my setup, because I definitely see

the

opposite behaviour in 2.6.24-rc1/rc2. The speed there is much

Quoted 2 lines

better

than in 2.6.22 and before (I skipped 2.6.23, because I was waiting

Quoted 2 lines

for

the per-bdi changes). I definitely do not see the difference between

Quoted 2 lines

1st

and subsequent runs. But then, I do my tests with 5GB file sizes like:

Quoted 4 lines
Quoted 2 lines

iozone3_283/src/current/iozone -t 5 -F /scratch/X1

/scratch/X2

/scratch/X3 /scratch/X4 /scratch/X5 -s 5000M -r 1024 -c -e -i 0 -i 1

Quoted 2 lines

My machine uses SATA (AHCI) disk.

4x72GB SCSI disks building a RAID5 on a CCISS controller with battery backed write cache. Systems are 2 CPUs (64-bit) with 8 GB memory. I could test on some IBM boxes (2x dual core, 8 GB) with RAID5 on "aacraid", but I need some time to free up one of the boxes.

Cheers
Martin

#1 lore raw
Quoted 55 lines · Martin Knoblauch

----- Original Message ----

Quoted 6 lines

From: "Zhang, Yanmin" <[email protected]>
To: Martin Knoblauch <[email protected]>
Cc: [email protected]; LKML <[email protected]>
Sent: Monday, November 12, 2007 1:45:57 AM
Subject: Re: iozone write 50% regression in kernel 2.6.24-rc1

Quoted 20 lines
Quoted 18 lines · Martin Knoblauch

----- Original Message ----

Quoted 9 lines

From: "Zhang, Yanmin"
To: [email protected]
Cc: LKML
Sent: Friday, November 9, 2007 10:47:52 AM
Subject: iozone write 50% regression in kernel 2.6.24-rc1

Comparing with 2.6.23, iozone sequential write/rewrite (512M) has 50%

regression

Quoted 5 lines

in kernel 2.6.24-rc1. 2.6.24-rc2 has the same regression.

My machine has 8 processor cores and 8GB memory.

By bisect, I located patch

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h

Quoted 17 lines

=

Quoted 16 lines
Quoted 6 lines

04fbfdc14e5f48463820d6b9807daa5e9c92c51f.

Another behavior: with kernel 2.6.23, if I run iozone for many times

after rebooting machine,

Quoted 3 lines

the result looks stable. But with 2.6.24-rc1, the first run of iozone

got a very small result and

Quoted 5 lines

following run has 4Xorig_result.

What I reported is the regression of 2nd/3rd run, because first run has

Quoted 9 lines
Quoted 7 lines

bigger regression.

Quoted 4 lines

I also tried to change
/proc/sys/vm/dirty_ratio,dirty_backgroud_ratio

and didn't get improvement.
could you tell us the exact iozone command you are using?

iozone -i 0 -r 4k -s 512m

Quoted 6 lines

OK, I definitely do not see the reported effect. On a HP Proliant with a RAID5 on CCISS I get:

2.6.19.2: 654-738 MB/sec write, 1126-1154 MB/sec rewrite 2.6.24-rc2: 772-820 MB/sec write, 1495-1539 MB/sec rewrite

The first run is always slowest, all subsequent runs are faster and the same speed.

Although the first run is always slowest, but if we compare 2.6.23 and 2.6.24-rc, we could find the first run result of 2.6.23 is 7 times of the one of 2.6.24-rc.

Originally, my test suite is just to pick up the result of first run. I might change my test suite to make it run for many times.

Now I run the the test manually for many times after machine reboots. Comparing 2.6.24-rc with 2.6.23, 3rd and following run of 2.6.24-rc has about 50% regression.