Skip to contentSkip to conversation

dm-snapshot: poor copy-on-write performance due to I/O reordering

PATCH Bodo Eggert 2 participants 3 messages Sep 17, 2008 – Sep 24, 2008 1 message not in the archive Device mapper

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw
Quoted 5 lines · Kazuo Ito

Write throughput to LVM snapshot origin volume is an order of magnitude slower than those to LV without snapshots or snapshot target volumes, especially in the case of sequential writes with O_SYNC on.

Quoted 7 lines

The following patch originally written by Kevin Jamieson and Jan Blunck and slightly modified for the current RCs by myself tries to improve the performance by modifying the behaviour of kcopyd, so that it pushes back an I/O job to the head of the job queue instead of the tail as process_jobs() currently does when it has to wait for free pages. This way, write requests aren't shuffled to cause extra seeks.

Did you check for starvation problems, too?

#2 lore raw

Hi,

Quoted 15 lines · Bodo Eggert
Quoted 5 lines · Kazuo Ito

Write throughput to LVM snapshot origin volume is an order of magnitude slower than those to LV without snapshots or snapshot target volumes, especially in the case of sequential writes with O_SYNC on.

Quoted 7 lines

The following patch originally written by Kevin Jamieson and Jan Blunck and slightly modified for the current RCs by myself tries to improve the performance by modifying the behaviour of kcopyd, so that it pushes back an I/O job to the head of the job queue instead of the tail as process_jobs() currently does when it has to wait for free pages. This way, write requests aren't shuffled to cause extra seeks.

Quoted 1 line

Did you check for starvation problems, too?

I ran sadc along with the tests and figures like %memused, kbbuffers and kbcached didn't change much before and after the patch. And since the buffered I/O results didn't deteriorate at least in the cases of 10 and 100 megabyte sequential writes, I assumed there should be plentiful of memory for the kernel to use at least in these cases.

Anyway I can send you more detailed info upon your request.

Regards,

Kazuo Ito, NTT Open Source Software Center
Phone: +81-3-5860-5125 / FAX: +81-3-5463-5690 / E-mail: [email protected]

#3 lore raw

Hello,

Quoted 16 lines
Quoted 15 lines · Bodo Eggert
Quoted 12 lines · Kazuo Ito

Write throughput to LVM snapshot origin volume is an order of magnitude slower than those to LV without snapshots or snapshot target volumes, especially in the case of sequential writes with O_SYNC on.

The following patch originally written by Kevin Jamieson and Jan Blunck and slightly modified for the current RCs by myself tries to improve the performance by modifying the behaviour of kcopyd, so that it pushes back an I/O job to the head of the job queue instead of the tail as process_jobs() currently does when it has to wait for free pages. This way, write requests aren't shuffled to cause extra seeks.

Quoted 1 line
Quoted 1 line

Did you check for starvation problems, too?

I have twice and four times as many pages allocated
to each kcopyd client without patching the queuing behaviour and got these figures (MiB/s) -- allocating more buffers doesn't seem to help much, so I don't think it's memory shortage that matters here.

test \ # of buffer pages 256(default) 512 1024

 10M dd+fsync, create        16.00          18.60      18.95
 10M dd+fsync, update        16.14          18.39      19.70
100M dd+fsync, create        15.18          18.80      19.29
100M dd+fsync, update        15.28          19.29      19.45

--

Quoted 2 lines

Kazuo Ito, NTT Open Source Software Center
Phone: +81-3-5860-5125 / FAX: +81-3-5463-5690 / E-mail: [email protected]