Skip to contentSkip to conversation

Writing on raw device with software RAID 0 is slow

DISCUSSION Martin Rauh 3 participants 6 messages Feb 28, 2001 – Mar 1, 2001

lore
Share & tools
lore
#0 lore raw

Hello,

Writing to an software RAID 0 containing 4 SCSI discs is very fast. I get transfer rates of about 100 MBytes/s. The filesystem on the RAID is ext2.

Writing to the same RAID directly (that means on the raw device without a filesystem) works
but gives low transfer rates of about 31 MBytes/s.

Any explanation for that?

thanks and greetings,

Martin Rauh

#1 lore raw

Hi,

Quoted 10 lines · Martin Rauh

Writing to an software RAID 0 containing 4 SCSI discs is very fast. I get transfer rates of about 100 MBytes/s. The filesystem on the RAID is ext2.

Writing to the same RAID directly (that means on the raw device without a filesystem) works
but gives low transfer rates of about 31 MBytes/s.

Any explanation for that?

Raw IO is always synchronous: it gets flushed to disk before the write returns. You don't get any write-behind with raw IO, so the smaller the blocksize you write in, the slower things get.

--Stephen

#2 lore raw

Hello all,

Quoted 3 lines · Stephen C. Tweedie

Raw IO is always synchronous: it gets flushed to disk before the write returns. You don't get any write-behind with raw IO, so the smaller the blocksize you write in, the slower things get.

More importantly, the mainstream raw io code only writes in 64KB chunks that are unpipelined, which can lead to writes not hitting the drive before the sector passes under the rw head. You can work around this to some extent by issuing multiple writes (via threads, or the aio work I've done) at the expense of atomicity. Also, before we allow locking of arbitrary larger ios in main memory, we need bean counting to prevent the obvious DoSes.

-ben

#3 lore raw
In reply to Stephen C. Tweedie

Hi,

Quoted 7 lines · Ben LaHaise
Quoted 3 lines · Stephen C. Tweedie

Raw IO is always synchronous: it gets flushed to disk before the write returns. You don't get any write-behind with raw IO, so the smaller the blocksize you write in, the slower things get.

Quoted 7 lines

More importantly, the mainstream raw io code only writes in 64KB chunks that are unpipelined, which can lead to writes not hitting the drive before the sector passes under the rw head. You can work around this to some extent by issuing multiple writes (via threads, or the aio work I've done) at the expense of atomicity. Also, before we allow locking of arbitrary larger ios in main memory, we need bean counting to prevent the obvious DoSes.

Yep. There shouldn't be any problem increasing the 64KB size, it's only the lack of accounting for the pinned memory which stopped me increasing it by default.

--Stephen

#4 lore raw
Quoted 3 lines · Stephen C. Tweedie

Yep. There shouldn't be any problem increasing the 64KB size, it's only the lack of accounting for the pinned memory which stopped me increasing it by default.

Actually, how about making it a sysctl? That's probably the most reasonable approach for now since the optimal size depends on hardware.

-ben

#5 lore raw
In reply to Stephen C. Tweedie

Hi,

Quoted 4 lines · Ben LaHaise

On Thu, 1 Mar 2001, Stephen C. Tweedie wrote:

Actually, how about making it a sysctl? That's probably the most reasonable approach for now since the optimal size depends on hardware.

Fine with me.

--Stephen