Skip to contentSkip to conversation

Slow Disk I/O with QPS M3 80GB HD

DISCUSSION H . J . Lu 3 participants 10 messages Dec 11, 2001 – Dec 13, 2001 1 message not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

H . J . Lu

author
#1 lore raw
Quoted 6 lines · H . J . Lu

I have a very strange problem. The disk I/O of my QPS M3 80GB HD is very slow under 2.4.10 and above. I got like 1.77 MB/s from hdparm. But under 2.4.9, I got 14 MB/s on the same hardware. A 30GB HD has consistent I/O performance under 2.4.9 and above on the same bus. Has anyone else seen this? Does anyone have a large (>= 80GB) 1394 HD?

I did a binary search. 2.4.10-pre10 is the last good kernel. I got

# hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads: 64 MB in 4.40 seconds = 14.55 MB/sec

Even since 2.4.10-pre11 up to 2.4.16, I got about 1.77 MB/sec on the same hardware. However, I don't have problems with 80GB IDE HD. Has anyone seen I/O problems on large (>= 80GB) SCSI HD or HD with SCSI emulation?

H.J.

#2 lore raw
In reply to a message not in the archive
Quoted 8 lines · H . J . Lu
Quoted 6 lines · H . J . Lu

I have a very strange problem. The disk I/O of my QPS M3 80GB HD is very slow under 2.4.10 and above. I got like 1.77 MB/s from hdparm. But under 2.4.9, I got 14 MB/s on the same hardware. A 30GB HD has consistent I/O performance under 2.4.9 and above on the same bus. Has anyone else seen this? Does anyone have a large (>= 80GB) 1394 HD?

Quoted 6 lines

I did a binary search. 2.4.10-pre10 is the last good kernel. I got

# hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads: 64 MB in 4.40 seconds = 14.55 MB/sec

Have you checked the way that your ohci and sbp2 devices are detected under each case? Most notably the max packet size.

Ben

Signature · 4 lines
 .----------=======-=-======-=========-----------=====------------=-=-----.
/                   Ben Collins    --    Debian GNU/Linux                  \
`  [email protected]  --  [email protected]  --  [email protected]  '
 `---=========------=======-------------=-=-----=-===-======-------=--=---'
author
#3 lore raw
In reply to a message not in the archive
Quoted 11 lines · Ben Collins
Quoted 10 lines · H . J . Lu
Quoted 6 lines · H . J . Lu

I have a very strange problem. The disk I/O of my QPS M3 80GB HD is very slow under 2.4.10 and above. I got like 1.77 MB/s from hdparm. But under 2.4.9, I got 14 MB/s on the same hardware. A 30GB HD has consistent I/O performance under 2.4.9 and above on the same bus. Has anyone else seen this? Does anyone have a large (>= 80GB) 1394 HD?

I did a binary search. 2.4.10-pre10 is the last good kernel. I got

Quoted 8 lines
Quoted 4 lines
# hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads: 64 MB in 4.40 seconds = 14.55 MB/sec

Have you checked the way that your ohci and sbp2 devices are detected under each case? Most notably the max packet size.

They all say

ohci1394_0: OHCI-1394 1.0 (PCI): IRQ=[19] MMIO=[f8ffd000-f8ffe000] Max Packet=[1024]
ieee1394: sbp2: SBP-2 device max speed S200 and payload 1KB

H.J.

#4 lore raw
In reply to a message not in the archive
Quoted 17 lines · H . J . Lu
Quoted 16 lines · Ben Collins
Quoted 14 lines · H . J . Lu
Quoted 6 lines · H . J . Lu

I have a very strange problem. The disk I/O of my QPS M3 80GB HD is very slow under 2.4.10 and above. I got like 1.77 MB/s from hdparm. But under 2.4.9, I got 14 MB/s on the same hardware. A 30GB HD has consistent I/O performance under 2.4.9 and above on the same bus. Has anyone else seen this? Does anyone have a large (>= 80GB) 1394 HD?

I did a binary search. 2.4.10-pre10 is the last good kernel. I got

# hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads: 64 MB in 4.40 seconds = 14.55 MB/sec

Quoted 8 lines
Quoted 3 lines

Have you checked the way that your ohci and sbp2 devices are detected under each case? Most notably the max packet size.

They all say

ohci1394_0: OHCI-1394 1.0 (PCI): IRQ=[19] MMIO=[f8ffd000-f8ffe000] Max Packet=[1024]
ieee1394: sbp2: SBP-2 device max speed S200 and payload 1KB

Have you tried linux1394 CVS with a 2.4.10pre10 kernel to narrow down where the slowdown has occured?

Signature · 4 lines
 .----------=======-=-======-=========-----------=====------------=-=-----.
/                   Ben Collins    --    Debian GNU/Linux                  \
`  [email protected]  --  [email protected]  --  [email protected]  '
 `---=========------=======-------------=-=-----=-===-======-------=--=---'
author
#5 lore raw
In reply to a message not in the archive
Quoted 22 lines · Ben Collins
Quoted 21 lines · H . J . Lu
Quoted 19 lines · Ben Collins
Quoted 8 lines · H . J . Lu
Quoted 6 lines · H . J . Lu

I have a very strange problem. The disk I/O of my QPS M3 80GB HD is very slow under 2.4.10 and above. I got like 1.77 MB/s from hdparm. But under 2.4.9, I got 14 MB/s on the same hardware. A 30GB HD has consistent I/O performance under 2.4.9 and above on the same bus. Has anyone else seen this? Does anyone have a large (>= 80GB) 1394 HD?

Quoted 6 lines

I did a binary search. 2.4.10-pre10 is the last good kernel. I got

# hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads: 64 MB in 4.40 seconds = 14.55 MB/sec

Have you checked the way that your ohci and sbp2 devices are detected under each case? Most notably the max packet size.

Quoted 7 lines
Quoted 4 lines

They all say

ohci1394_0: OHCI-1394 1.0 (PCI): IRQ=[19] MMIO=[f8ffd000-f8ffe000] Max Packet=[1024]
ieee1394: sbp2: SBP-2 device max speed S200 and payload 1KB

Have you tried linux1394 CVS with a 2.4.10pre10 kernel to narrow down where the slowdown has occured?

I don't think the 1394 driver is the problem. I have tried the 1394 driver from a good kernel. I got the same result.

H.J.

author
#6 lore raw
In reply to a message not in the archive
Quoted 12 lines · H . J . Lu
Quoted 6 lines · H . J . Lu

I have a very strange problem. The disk I/O of my QPS M3 80GB HD is very slow under 2.4.10 and above. I got like 1.77 MB/s from hdparm. But under 2.4.9, I got 14 MB/s on the same hardware. A 30GB HD has consistent I/O performance under 2.4.9 and above on the same bus. Has anyone else seen this? Does anyone have a large (>= 80GB) 1394 HD?

I did a binary search. 2.4.10-pre10 is the last good kernel. I got

# hdparm -t /dev/sda
Quoted 7 lines

/dev/sda:
Timing buffered disk reads: 64 MB in 4.40 seconds = 14.55 MB/sec

Even since 2.4.10-pre11 up to 2.4.16, I got about 1.77 MB/sec on the same hardware. However, I don't have problems with 80GB IDE HD. Has anyone seen I/O problems on large (>= 80GB) SCSI HD or HD with SCSI emulation?

I tracked own the problem to 40_blkdev-pagecache-17 in the 2.4.10 pre10aa1 patch. When it is applied, the disk I/O on some drives become very slow. It not only happens to my 80GB 1394 HD, but also the second IDE drive. Before the patch

# hdparm -t /dev/hdd

/dev/hdd:
Timing buffered disk reads: 64 MB in 8.02 seconds = 7.98 MB/sec

After the patch

# hdparm -t /dev/hdd

/dev/hdd:
Timing buffered disk reads: 64 MB in 21.09 seconds = 3.03 MB/sec

The slow down is not as bad as 1394. But it is still very significant. I couldn't figure out why it only affects certain drives.

H.J.

#7 lore raw
In reply to a message not in the archive
Quoted 33 lines · H . J . Lu
Quoted 12 lines · H . J . Lu
Quoted 6 lines · H . J . Lu

I have a very strange problem. The disk I/O of my QPS M3 80GB HD is very slow under 2.4.10 and above. I got like 1.77 MB/s from hdparm. But under 2.4.9, I got 14 MB/s on the same hardware. A 30GB HD has consistent I/O performance under 2.4.9 and above on the same bus. Has anyone else seen this? Does anyone have a large (>= 80GB) 1394 HD?

I did a binary search. 2.4.10-pre10 is the last good kernel. I got

# hdparm -t /dev/sda
Quoted 7 lines

/dev/sda:
Timing buffered disk reads: 64 MB in 4.40 seconds = 14.55 MB/sec

Even since 2.4.10-pre11 up to 2.4.16, I got about 1.77 MB/sec on the same hardware. However, I don't have problems with 80GB IDE HD. Has anyone seen I/O problems on large (>= 80GB) SCSI HD or HD with SCSI emulation?

I tracked own the problem to 40_blkdev-pagecache-17 in the 2.4.10 pre10aa1 patch. When it is applied, the disk I/O on some drives become very slow. It not only happens to my 80GB 1394 HD, but also the second IDE drive. Before the patch

# hdparm -t /dev/hdd

/dev/hdd:
Timing buffered disk reads: 64 MB in 8.02 seconds = 7.98 MB/sec

After the patch

Quoted 7 lines
# hdparm -t /dev/hdd

/dev/hdd:
Timing buffered disk reads: 64 MB in 21.09 seconds = 3.03 MB/sec

The slow down is not as bad as 1394. But it is still very significant. I couldn't figure out why it only affects certain drives.

do you have a 4k filesystm mounted on /dev/hdd? if so then you will get the same performance with latest 2.4 (precisely after 2.4.1).

Andrea

author
#8 lore raw
In reply to a message not in the archive
Quoted 36 lines · Andrea Arcangeli
Quoted 35 lines · H . J . Lu
Quoted 12 lines · H . J . Lu
Quoted 6 lines · H . J . Lu

I have a very strange problem. The disk I/O of my QPS M3 80GB HD is very slow under 2.4.10 and above. I got like 1.77 MB/s from hdparm. But under 2.4.9, I got 14 MB/s on the same hardware. A 30GB HD has consistent I/O performance under 2.4.9 and above on the same bus. Has anyone else seen this? Does anyone have a large (>= 80GB) 1394 HD?

I did a binary search. 2.4.10-pre10 is the last good kernel. I got

# hdparm -t /dev/sda
Quoted 7 lines

/dev/sda:
Timing buffered disk reads: 64 MB in 4.40 seconds = 14.55 MB/sec

Even since 2.4.10-pre11 up to 2.4.16, I got about 1.77 MB/sec on the same hardware. However, I don't have problems with 80GB IDE HD. Has anyone seen I/O problems on large (>= 80GB) SCSI HD or HD with SCSI emulation?

I tracked own the problem to 40_blkdev-pagecache-17 in the 2.4.10 pre10aa1 patch. When it is applied, the disk I/O on some drives become very slow. It not only happens to my 80GB 1394 HD, but also the second IDE drive. Before the patch

# hdparm -t /dev/hdd

/dev/hdd:
Timing buffered disk reads: 64 MB in 8.02 seconds = 7.98 MB/sec

After the patch

# hdparm -t /dev/hdd
Quoted 9 lines
Quoted 5 lines

/dev/hdd:
Timing buffered disk reads: 64 MB in 21.09 seconds = 3.03 MB/sec

The slow down is not as bad as 1394. But it is still very significant. I couldn't figure out why it only affects certain drives.

do you have a 4k filesystm mounted on /dev/hdd? if so then you will get the same performance with latest 2.4 (precisely after 2.4.1).

Even Sony F707 digital camera is slowed because your change from 2.4.10-pre10 to 2.4.10-pre11. It is a USB memory stick. It is 10 times slower than before. So far, we have some USB and 1394 storage devices are slowed down very badly. Some IDE hds also suffer. It is very strange.

H.J.
----
3 files on USB:

# ls -l
total 5984
-rwxr-xr-x    1 root     root      1991481 Dec  8 21:44 dsc00002.jpg
-rwxr-xr-x    1 root     root      2432127 Dec  8 23:54 dsc00005.jpg
-rwxr-xr-x    1 root     root      1675563 Dec  8 23:59 dsc00009.jpg

1. 2.4.10-pre10

# time /bin/cp * /tmp/
real    0m0.623s
user    0m0.000s
sys     0m0.100s

2. 2.4.10-pre10aa1

# time /bin/cp * /tmp/
real    0m8.952s
user    0m0.000s
sys     0m0.190s

3. 2.4.10-pre11

# time /bin/cp * /tmp/
real    0m8.851s
user    0m0.000s
sys     0m0.170s
#9 lore raw
In reply to a message not in the archive
Quoted 25 lines · H . J . Lu

H.J.
----
3 files on USB:

# ls -l
total 5984
(quoted diff) +0 -3
-rwxr-xr-x    1 root     root      1991481 Dec  8 21:44 dsc00002.jpg
-rwxr-xr-x    1 root     root      2432127 Dec  8 23:54 dsc00005.jpg
-rwxr-xr-x    1 root     root      1675563 Dec  8 23:59 dsc00009.jpg

1. 2.4.10-pre10

# time /bin/cp * /tmp/
real    0m0.623s
user    0m0.000s
sys     0m0.100s

2. 2.4.10-pre10aa1

# time /bin/cp * /tmp/
real    0m8.952s
user    0m0.000s
sys     0m0.190s
Quoted 6 lines

3. 2.4.10-pre11

# time /bin/cp * /tmp/
real    0m8.851s
user    0m0.000s
sys     0m0.170s

but the above have nothing to do with the blkdev-pagecache. Blkdev-pagecache can matter only with blkdev accesses, blkdev-pagecache cannot make any difference if the I/O passes through any fs (not via the block_dev layer) like in the above case.

Can you try to take those cp under strace timestamp and see what's the syscall that blocks?

Andrea

author Applied
#10 lore raw
In reply to a message not in the archive
Quoted 11 lines · Andrea Arcangeli
Quoted 10 lines · H . J . Lu

H.J.
----
3 files on USB:

# ls -l
total 5984
(quoted diff) +0 -3
-rwxr-xr-x    1 root     root      1991481 Dec  8 21:44 dsc00002.jpg
-rwxr-xr-x    1 root     root      2432127 Dec  8 23:54 dsc00005.jpg
-rwxr-xr-x    1 root     root      1675563 Dec  8 23:59 dsc00009.jpg
Quoted 8 lines
Quoted 8 lines

1. 2.4.10-pre10

# time /bin/cp * /tmp/
real    0m0.623s
user    0m0.000s
sys     0m0.100s

2. 2.4.10-pre10aa1

I applied all the way to 40_blkdev-pagecache-17 in 2.4.10-pre10aa1.

Quoted 14 lines
Quoted 7 lines
# time /bin/cp * /tmp/
real    0m8.952s
user    0m0.000s
sys     0m0.190s
Quoted 6 lines

3. 2.4.10-pre11

# time /bin/cp * /tmp/
real    0m8.851s
user    0m0.000s
sys     0m0.170s
Quoted 4 lines

but the above have nothing to do with the blkdev-pagecache. Blkdev-pagecache can matter only with blkdev accesses, blkdev-pagecache cannot make any difference if the I/O passes through any fs (not via the block_dev layer) like in the above case.

That is why I said it was strange.

Quoted 3 lines

Can you try to take those cp under strace timestamp and see what's the syscall that blocks?

It blocks on read.

H.J.