Skip to contentSkip to conversation

io performance...

DISCUSSION Max Waterman 11 participants 23 messages Jan 16, 2006 – Jan 25, 2006 Block layer

lore
Share & tools
lore
#0 lore raw

Hi,

I've been referred to this list from the linux-raid list.

I've been playing with a RAID system, trying to obtain best bandwidth from it.

I've noticed that I consistently get better (read) numbers from kernel 2.6.8 than from later kernels.

For example, I get 135MB/s on 2.6.8, but I typically get ~90MB/s on later kernels.

I'm using this :

<http://www.sharcnet.ca/~hahn/iorate.c>

to measure the iorate. I'm using the debian distribution. The h/w is a MegaRAID 320-2. The array I'm measuring is a RAID0 of 4 Fujitsu Max3073NC 15Krpm drives.

The later kernels I've been using are :

2.6.12-1-686-smp
2.6.14-2-686-smp
2.6.15-1-686-smp

The kernel which gives us the best results is :

2.6.8-2-386

(note that it's not an smp kernel)

I'm testing on an otherwise idle system.

Any ideas to why this might be? Any other advice/help?

Thanks!

Max.

#1 lore raw
Quoted 9 lines · Max Waterman

Hi,

I've been referred to this list from the linux-raid list.

I've been playing with a RAID system, trying to obtain best bandwidth from it.

I've noticed that I consistently get better (read) numbers from kernel 2.6.8 than from later kernels.

To open the bottlenecks, the following works well. Jens will shoot me for recommending this, but it works well. 2.6.9 so far has the highest numbers with this fix. You can manually putz around with these numbers, but they are an artificial constraint if you are using RAID technology that caches ad elevators requests and consolidates them.

Jeff

linux-2.6.9/./include/linux/blkdev.h renamed from include/linux/blkdev.h +4 -2
diff -Naur ./include/linux/blkdev.h ../linux-2.6.9/./include/linux/blkdev.h
@@ -23,8 +23,10 @@
 typedef struct elevator_s elevator_t;
 struct request_pm_state;
 
-#define BLKDEV_MIN_RQ	4
-#define BLKDEV_MAX_RQ	128	/* Default maximum */
+//#define BLKDEV_MIN_RQ	4
+//#define BLKDEV_MAX_RQ	128	/* Default maximum */
+#define BLKDEV_MIN_RQ	4096
+#define BLKDEV_MAX_RQ	8192	/* Default maximum */
 
#2 lore raw
Quoted 32 lines · Jeff V. Merkey
Quoted 10 lines · Max Waterman

Hi,

I've been referred to this list from the linux-raid list.

I've been playing with a RAID system, trying to obtain best bandwidth from it.

I've noticed that I consistently get better (read) numbers from kernel 2.6.8
than from later kernels.

To open the bottlenecks, the following works well. Jens will shoot me for recommending this,
but it works well. 2.6.9 so far has the highest numbers with this fix. You can manually putz
around with these numbers, but they are an artificial constraint if you are using RAID technology
that caches ad elevators requests and consolidates them.

Jeff

diff -Naur ./include/linux/blkdev.h ../linux-2.6.9/./include/linux/blkdev.h

linux-2.6.9/./include/linux/blkdev.h line 29

Quoted 9 lines
linux-2.6.9/./include/linux/blkdev.h renamed from include/linux/blkdev.h +4 -2
@@ -23,8 +23,10 @@
 typedef struct elevator_s elevator_t;
 struct request_pm_state;
 
-#define BLKDEV_MIN_RQ	4
-#define BLKDEV_MAX_RQ	128	/* Default maximum */
+//#define BLKDEV_MIN_RQ	4
+//#define BLKDEV_MAX_RQ	128	/* Default maximum */
+#define BLKDEV_MIN_RQ	4096
+#define BLKDEV_MAX_RQ	8192	/* Default maximum */

Yeah I could shoot you. However I'm more interested in why this is necessary, eg I'd like to see some numbers from you comparing:

- The stock settings
- Doing

        # echo 8192 > /sys/block/<dev>/queue/nr_requests
  for each drive you are accessing.
- The kernel with your patch.

If #2 and #3 don't provide very similar profiles/scores, then we have something to look at.

The BLKDEV_MIN_RQ increase is just silly and wastes a huge amount of memory for no good reason.

Signature · 1 line
Jens Axboe
#3 lore raw
Quoted 60 lines · Jens Axboe
Quoted 37 lines · Jeff V. Merkey
Quoted 4 lines · Max Waterman

Hi,

I've been referred to this list from the linux-raid list.

Quoted 10 lines

I've been playing with a RAID system, trying to obtain best bandwidth

from it.

I've noticed that I consistently get better (read) numbers from kernel 2.6.8 than from later kernels.

To open the bottlenecks, the following works well. Jens will shoot me for recommending this, but it works well. 2.6.9 so far has the highest numbers with this fix. You can manually putz around with these numbers, but they are an artificial constraint if you are using RAID technology that caches ad elevators requests and consolidates them.

Jeff

diff -Naur ./include/linux/blkdev.h ../linux-2.6.9/./include/linux/blkdev.h

linux-2.6.9/./include/linux/blkdev.h renamed from include/linux/blkdev.h +0 -0
@@ -23,8 +23,10 @@
 typedef struct elevator_s elevator_t;

linux-2.6.9/./include/linux/blkdev.h line 31

Quoted 10 lines
linux-2.6.9/./include/linux/blkdev.h renamed from include/linux/blkdev.h +4 -2
 struct request_pm_state;
 
-#define BLKDEV_MIN_RQ	4
-#define BLKDEV_MAX_RQ	128	/* Default maximum */
+//#define BLKDEV_MIN_RQ	4
+//#define BLKDEV_MAX_RQ	128	/* Default maximum */
+#define BLKDEV_MIN_RQ	4096
+#define BLKDEV_MAX_RQ	8192	/* Default maximum */
 
 

Yeah I could shoot you. However I'm more interested in why this is necessary, eg I'd like to see some numbers from you comparing:

- The stock settings
- Doing

       # echo 8192 > /sys/block/<dev>/queue/nr_requests
 for each drive you are accessing.
- The kernel with your patch.
Quoted 8 lines

If #2 and #3 don't provide very similar profiles/scores, then we have something to look at.

The BLKDEV_MIN_RQ increase is just silly and wastes a huge amount of memory for no good reason.

Yep. I build it into the kernel to save the trouble of sending it to proc. Jens recommendation will work just fine. It has the same affect of increasing the max requests outstanding.

Jeff

#4 lore raw
In reply to Max Waterman

Hi,

Quoted 2 lines · Max Waterman

I've noticed that I consistently get better (read) numbers from kernel 2.6.8 than from later kernels.

[snip]

Quoted 11 lines

The later kernels I've been using are :

2.6.12-1-686-smp
2.6.14-2-686-smp
2.6.15-1-686-smp

The kernel which gives us the best results is :

2.6.8-2-386

Any ideas to why this might be? Any other advice/help?

It would be helpful if you could isolate the exact changeset that introduces the regression. You can use git bisect for that. Please refer to the following URL for details:
http://www.kernel.org/pub/software/scm/git/docs/howto/isolate-bugs-with-bisect.txt

Also note that changeset for pre 2.6.11-rc2 kernels are in old-2.6-bkcvs git tree. If you are new to git, you can find a good introduction here: http://linux.yyz.us/git-howto.html. Thanks.

Pekka

#5 lore raw
In reply to Max Waterman

Did you direct the program to use O_DIRECT? If not then I believe the problem you are seeing is that the generic block layer is not performing large enough readahead to keep all the disks in the array reading at once, because the stripe width is rather large. What stripe factor did you format the array using?

I have a sata fakeraid at home of two drives using a stripe factor of 64 KB. If I don't issue O_DIRECT IO requests of at least 128 KB ( the stripe width ), then throughput drops significantly. If I issue multiple async requests of smaller size that totals at least 128 KB, throughput also remains high. If you only issue a single 32 KB request at a time, then two requests must go to one drive and be completed before the other drive gets any requests, so it remains idle a lot of the time.

Quoted 46 lines · Max Waterman

Hi,

I've been referred to this list from the linux-raid list.

I've been playing with a RAID system, trying to obtain best bandwidth from it.

I've noticed that I consistently get better (read) numbers from kernel 2.6.8 than from later kernels.

For example, I get 135MB/s on 2.6.8, but I typically get ~90MB/s on later kernels.

I'm using this :

<http://www.sharcnet.ca/~hahn/iorate.c>

to measure the iorate. I'm using the debian distribution. The h/w is a MegaRAID 320-2. The array I'm measuring is a RAID0 of 4 Fujitsu Max3073NC 15Krpm drives.

The later kernels I've been using are :

2.6.12-1-686-smp
2.6.14-2-686-smp
2.6.15-1-686-smp

The kernel which gives us the best results is :

2.6.8-2-386

(note that it's not an smp kernel)

I'm testing on an otherwise idle system.

Any ideas to why this might be? Any other advice/help?

Thanks!

Max.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#6 lore raw
Quoted 1 line · Phillip Susi

Did you direct the program to use O_DIRECT?

I'm just using the s/w (iorate/bonnie++) with default options - I'm no expert. I could try though.

Quoted 1 line

If not then I believe the problem you are seeing is that the generic block layer is not performing large enough readahead to keep all the disks in the array reading at once, because the stripe width is rather large. What stripe factor did you format the array using?

I left the stripe size at the default, which, I believe, is 64K bytes; same as your fakeraid below.

I did play with 'blockdev --setra' too.

I noticed it was 256 with a single disk, and, with s/w raid, it increased by 256 for each extra disk in the array. IE for the raid 0 array with 4 drives, it was 1024.

With h/w raid, however, it did not increase when I added disks. Should I use 'blockdev --setra 320' (ie 64 x 5 = 320, since we're now running RAID5 on 5 drives)?

Quoted 1 line

I have a sata fakeraid at home of two drives using a stripe factor of 64 KB. If I don't issue O_DIRECT IO requests of at least 128 KB ( the stripe width ), then throughput drops significantly. If I issue multiple async requests of smaller size that totals at least 128 KB, throughput also remains high. If you only issue a single 32 KB request at a time, then two requests must go to one drive and be completed before the other drive gets any requests, so it remains idle a lot of the time.

I think that makes sense (which is a change in this RAID performance business :( ).

Thanks.

Max.

Quoted 48 lines
Quoted 46 lines · Max Waterman

Hi,

I've been referred to this list from the linux-raid list.

I've been playing with a RAID system, trying to obtain best bandwidth from it.

I've noticed that I consistently get better (read) numbers from kernel 2.6.8 than from later kernels.

For example, I get 135MB/s on 2.6.8, but I typically get ~90MB/s on later kernels.

I'm using this :

<http://www.sharcnet.ca/~hahn/iorate.c>

to measure the iorate. I'm using the debian distribution. The h/w is a MegaRAID 320-2. The array I'm measuring is a RAID0 of 4 Fujitsu Max3073NC 15Krpm drives.

The later kernels I've been using are :

2.6.12-1-686-smp
2.6.14-2-686-smp
2.6.15-1-686-smp

The kernel which gives us the best results is :

2.6.8-2-386

(note that it's not an smp kernel)

I'm testing on an otherwise idle system.

Any ideas to why this might be? Any other advice/help?

Thanks!

Max.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#7 lore raw

Right, the kernel does not know how many disks are in the array, so it can't automatically increase the readahead. I'd say increasing the readahead manually should solve your throughput issues.

Quoted 9 lines · Max Waterman

I left the stripe size at the default, which, I believe, is 64K bytes; same as your fakeraid below.

I did play with 'blockdev --setra' too.

I noticed it was 256 with a single disk, and, with s/w raid, it increased by 256 for each extra disk in the array. IE for the raid 0 array with 4 drives, it was 1024.

With h/w raid, however, it did not increase when I added disks. Should I use 'blockdev --setra 320' (ie 64 x 5 = 320, since we're now running RAID5 on 5 drives)?

#8 lore raw
Quoted 1 line · Phillip Susi

Right, the kernel does not know how many disks are in the array, so it can't automatically increase the readahead. I'd say increasing the readahead manually should solve your throughput issues.

Any guesses for a good number?

We're in RAID10 (2+2) at the moment on 2.6.8-smp. These are the block numbers I'm getting using bonnie++ :

ra wr rd
256 68K 46K
512 67K 59K
640 67K 64K
1024 66K 73K
2048 67K 88K
3072 67K 91K
8192 71K 96K
9216 67K 92K
16384 67K 90K

I think we might end up going for 8192.

We're still wondering why rd performance is so low - seems to be the same as a single drive. RAID10 should be the same performance as RAID0 over two drives, shouldn't it?

Max.

Quoted 12 lines
Quoted 9 lines · Max Waterman

I left the stripe size at the default, which, I believe, is 64K bytes; same as your fakeraid below.

I did play with 'blockdev --setra' too.

I noticed it was 256 with a single disk, and, with s/w raid, it increased by 256 for each extra disk in the array. IE for the raid 0 array with 4 drives, it was 1024.

With h/w raid, however, it did not increase when I added disks. Should I use 'blockdev --setra 320' (ie 64 x 5 = 320, since we're now running RAID5 on 5 drives)?

#9 lore raw
Quoted 7 lines · Max Waterman
Quoted 3 lines · Phillip Susi

Right, the kernel does not know how many disks are in the array, so it can't automatically increase the readahead. I'd say increasing the readahead manually should solve your throughput issues.

Any guesses for a good number?

Quoted 7 lines

We're in RAID10 (2+2) at the moment on 2.6.8-smp. These are the block numbers I'm getting using bonnie++ :

[...]
We're still wondering why rd performance is so low - seems to be the same as a single drive. RAID10 should be the same performance as RAID0 over two drives, shouldn't it?

I think bonnie++ measures accesses to many small files (INN-like simulation) and database accesses. These are random accesses, which is the worst access pattern for RAID. Seek time in a RAID equals the longest of all the drives in the RAID, rather than the average. So bonnie++ is domninated by your seek time.

Ian

#10 lore raw
Quoted 16 lines · Ian Soboroff
Quoted 6 lines · Max Waterman
Quoted 3 lines · Phillip Susi

Right, the kernel does not know how many disks are in the array, so it can't automatically increase the readahead. I'd say increasing the readahead manually should solve your throughput issues.

Any guesses for a good number?

Quoted 7 lines

We're in RAID10 (2+2) at the moment on 2.6.8-smp. These are the block numbers I'm getting using bonnie++ :

[...]
We're still wondering why rd performance is so low - seems to be the same as a single drive. RAID10 should be the same performance as RAID0 over two drives, shouldn't it?

Quoted 5 lines

I think bonnie++ measures accesses to many small files (INN-like simulation) and database accesses. These are random accesses, which is the worst access pattern for RAID. Seek time in a RAID equals the longest of all the drives in the RAID, rather than the average. So bonnie++ is domninated by your seek time.

You think so? I had assumed when bonnie++'s output said 'sequential access' that it was the opposite of random, for example (raid0 on 5 drives) :

Quoted 11 lines
+---------------------------------------------------------------------------------------------------------------------------------------------------+
|                     |Sequential Output             |Sequential Input    |         |     |Sequential Create           |Random Create               |
|---------------------+------------------------------+--------------------|Random   |-----+----------------------------+----------------------------|
|          |Size:Chunk|Per Char |Block     |Rewrite  |Per Char |Block     |Seeks    |Num  |Create  |Read     |Delete   |Create  |Read     |Delete   |
|          |Size      |         |          |         |         |          |         |Files|        |         |         |        |         |         |
|---------------------+---------+----------+---------+---------+----------+---------+-----+--------+---------+---------+--------+---------+---------|
|                     |K/sec|%  |K/sec |%  |K/sec|%  |K/sec|%  |K/sec |%  |/ sec|%  |     |/   |%  |/ sec|%  |/ sec|%  |/   |%  |/ sec|%  |/ sec|%  |
|                     |     |CPU|      |CPU|     |CPU|     |CPU|      |CPU|     |CPU|     |sec |CPU|     |CPU|     |CPU|sec |CPU|     |CPU|     |CPU|
|---------------------+-----+---+------+---+-----+---+-----+---+------+---+-----+---+-----+----+---+-----+---+-----+---+----+---+-----+---+-----+---|
|hostname  |2G        |48024|96 |121412|13 |59714|10 |47844|95 |200264|21 |942.8|1  |16   |4146|99 |+++++|+++|+++++|+++|4167|99 |+++++|+++|14292|99 |
+---------------------------------------------------------------------------------------------------------------------------------------------------+

Am I wrong? If so, what exactly does 'Sequential' mean in this context?

Max.

#11 lore raw
Quoted 10 lines · Max Waterman
Quoted 9 lines
Quoted 4 lines

We're still wondering why rd performance is so low - seems to be the same as a single drive. RAID10 should be the same performance as RAID0 over two drives, shouldn't it?

I think bonnie++ measures accesses to many small files (INN-like simulation) and database accesses. These are random accesses, which is the worst access pattern for RAID. Seek time in a RAID equals the longest of all the drives in the RAID, rather than the average. So bonnie++ is domninated by your seek time.

Quoted 4 lines

You think so? I had assumed when bonnie++'s output said 'sequential access' that it was the opposite of random, for example (raid0 on 5 drives) :

I could be wrong, I was just reading the information from the bonnie++ website...

Ian

#12 lore raw
In reply to Ian Soboroff
Quoted 3 lines · Ian Soboroff

simulation) and database accesses. These are random accesses, which is the worst access pattern for RAID. Seek time in a RAID equals the longest of all the drives in the RAID, rather than the average.

Well, actually it equals to the shortest seek time and it distributes the seeks to multiple spindles (at least for raid1).

Gruss
Bernd

#13 lore raw
In reply to Max Waterman

One further question. I get these messages 'in' dmesg :

sda: asking for cache data failed
sda: assuming drive cache: write through

How can I force it to be 'write back'?

Max.

#14 lore raw
Quoted 6 lines · Max Waterman

One further question. I get these messages 'in' dmesg :

sda: asking for cache data failed
sda: assuming drive cache: write through

How can I force it to be 'write back'?

Forcing write back is a very bad idea unless you have a battery backed up RAID controller.

Jeff

Quoted 8 lines

Max.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#15 lore raw
Quoted 4 lines · Jeff V. Merkey
Quoted 2 lines · Max Waterman

One further question. I get these messages 'in' dmesg :

Quoted 8 lines
Quoted 4 lines

sda: asking for cache data failed
sda: assuming drive cache: write through

How can I force it to be 'write back'?

Forcing write back is a very bad idea unless you have a battery backed up RAID controller.

We do.

In any case, I wonder what the consequences of assuming 'write through' when the array is configured as 'write back'? Is it just different settings for different caches?

Max.

Quoted 11 lines

Jeff

Quoted 8 lines

Max.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#16 lore raw
Quoted 14 lines · Max Waterman
Quoted 12 lines · Jeff V. Merkey
Quoted 6 lines · Max Waterman

One further question. I get these messages 'in' dmesg :

sda: asking for cache data failed
sda: assuming drive cache: write through

How can I force it to be 'write back'?

Quoted 6 lines
Quoted 1 line

Forcing write back is a very bad idea unless you have a battery backed up RAID controller.

We do.

In any case, I wonder what the consequences of assuming 'write through' when the array is configured as 'write back'? Is it just different settings for different caches?

It is. This is something that should be configured in a RAID controller. OS should always be write through.

Jeff

Quoted 16 lines

Max.

Quoted 11 lines

Jeff

Quoted 8 lines

Max.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#17 lore raw
Quoted 19 lines · Jeff V. Merkey
Quoted 17 lines · Max Waterman
Quoted 13 lines · Jeff V. Merkey
Quoted 6 lines · Max Waterman

One further question. I get these messages 'in' dmesg :

sda: asking for cache data failed
sda: assuming drive cache: write through

How can I force it to be 'write back'?

Forcing write back is a very bad idea unless you have a battery backed up RAID controller.

Quoted 6 lines
Quoted 3 lines

We do.

In any case, I wonder what the consequences of assuming 'write through' when the array is configured as 'write back'? Is it just different settings for different caches?

It is. This is something that should be configured in a RAID controller. OS should always be write through.

Ok, thanks for clearing that up, though I now wonder why the message is there.

<shrug>

Max.

Quoted 20 lines

Jeff

Quoted 16 lines

Max.

Quoted 11 lines

Jeff

Quoted 8 lines

Max.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#18 lore raw
In reply to Jeff V. Merkey
Quoted 3 lines · Jeff V. Merkey
Quoted 1 line

How can I force it to be 'write back'?

Forcing write back is a very bad idea unless you have a battery backed up RAID controller.

Not always. If you have a cache flush command and the OS knows about using it, or if you don't care if the data gets lost over a power failure (eg /tmp and swap) it makes sense to force it.

The raid controller drivers that fake scsi don't always fake enough of scsi to report that they support cache flushes and the like. That doesn't mean the controller itself is neccessarily doing one thing or the other.

#19 lore raw

I was going to say, doesn't the kernel set the FUA bit on the write request to push important flushes through the disk's write-back cache? Like for filesystem journal flushes?

Quoted 17 lines · Alan Cox

Not always. If you have a cache flush command and the OS knows about using it, or if you don't care if the data gets lost over a power failure (eg /tmp and swap) it makes sense to force it.

The raid controller drivers that fake scsi don't always fake enough of scsi to report that they support cache flushes and the like. That doesn't mean the controller itself is neccessarily doing one thing or the other.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#20 lore raw
Quoted 3 lines · Phillip Susi

I was going to say, doesn't the kernel set the FUA bit on the write request to push important flushes through the disk's write-back cache? Like for filesystem journal flushes?

Yes if:
* you have a disk supporting FUA
* you have kernel >= 2.6.16-rc1
* you are using SCSI (this includes libata) driver [ support for IDE driver will be merged later when races in changing IDE settings are fixed ]

Bartlomiej

Quoted 9 lines
Quoted 8 lines · Alan Cox

Not always. If you have a cache flush command and the OS knows about using it, or if you don't care if the data gets lost over a power failure (eg /tmp and swap) it makes sense to force it.

The raid controller drivers that fake scsi don't always fake enough of scsi to report that they support cache flushes and the like. That doesn't mean the controller itself is neccessarily doing one thing or the other.

#21 lore raw
In reply to Max Waterman
Quoted 30 lines · Max Waterman

Hi,

I've been referred to this list from the linux-raid list.

I've been playing with a RAID system, trying to obtain best bandwidth from it.

I've noticed that I consistently get better (read) numbers from kernel 2.6.8 than from later kernels.

For example, I get 135MB/s on 2.6.8, but I typically get ~90MB/s on later kernels.

I'm using this :

<http://www.sharcnet.ca/~hahn/iorate.c>

to measure the iorate. I'm using the debian distribution. The h/w is a MegaRAID
320-2. The array I'm measuring is a RAID0 of 4 Fujitsu Max3073NC 15Krpm drives.

The later kernels I've been using are :

2.6.12-1-686-smp
2.6.14-2-686-smp
2.6.15-1-686-smp

The kernel which gives us the best results is :

Quoted 7 lines

2.6.8-2-386

(note that it's not an smp kernel)

I'm testing on an otherwise idle system.

Any ideas to why this might be? Any other advice/help?

You should try to narrow the problem a bit down.

Possible causes are:
- kernel regression between 2.6.8 and 2.6.12
- SMP <-> !SMP support
- patches and/or configuration changes in the Debian kernels

You should try self-compiled unmodified 2.6.8 and 2.6.12 ftp.kernel.org kernels with the same .config (modulo differences by "make oldconfig").

After this test, you know whether you are in the first case. If yes, you could do a bisect search for finding the point where the regression started.

Quoted 3 lines

Thanks!

Max.

cu
Adrian

Signature · 4 lines
       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed
#22 lore raw

Unfortunately, they don't want me to spend time doing this sort of thing, so I'm out of luck.

They're going to stick with 2.6.8-smp, which seems to give the best performance (which rules out your second case below, I suppose).

:|

Max.

Quoted 56 lines · Adrian Bunk
Quoted 28 lines · Max Waterman

Hi,

I've been referred to this list from the linux-raid list.

I've been playing with a RAID system, trying to obtain best bandwidth from it.

I've noticed that I consistently get better (read) numbers from kernel 2.6.8 than from later kernels.

For example, I get 135MB/s on 2.6.8, but I typically get ~90MB/s on later kernels.

I'm using this :

<http://www.sharcnet.ca/~hahn/iorate.c>

to measure the iorate. I'm using the debian distribution. The h/w is a MegaRAID 320-2. The array I'm measuring is a RAID0 of 4 Fujitsu Max3073NC 15Krpm drives.

The later kernels I've been using are :

2.6.12-1-686-smp
2.6.14-2-686-smp
2.6.15-1-686-smp

The kernel which gives us the best results is :

Quoted 7 lines

2.6.8-2-386

(note that it's not an smp kernel)

I'm testing on an otherwise idle system.

Any ideas to why this might be? Any other advice/help?

You should try to narrow the problem a bit down.

Possible causes are:
- kernel regression between 2.6.8 and 2.6.12
- SMP <-> !SMP support
- patches and/or configuration changes in the Debian kernels

You should try self-compiled unmodified 2.6.8 and 2.6.12 ftp.kernel.org kernels with the same .config (modulo differences by "make oldconfig").

After this test, you know whether you are in the first case.
If yes, you could do a bisect search for finding the point where the regression started.

Quoted 3 lines

Thanks!

Max.

cu
Adrian