Skip to contentSkip to conversation

What's in linux-2.6-block.git

DISCUSSION Al Boldi 3 participants 8 messages Sep 13, 2006 – Sep 14, 2006

lore
Share & tools
lore

Al Boldi

author
#0 lore raw
Quoted 24 lines · Jens Axboe

This lists the main features of the 'block' branch, which is bound for Linus when 2.6.19 opens:

- Splitting of request->flags into two parts:
- cmd type
- modified flags
Right now it's a bit of a mess, splitting this up invites a cleaner usage and also enables us to implement generic "messages" passed on the regular queue for the device.

- Abstract out the request back merging and put it into the core io scheduler layer. Cleans up all the io schedulers, and noop gets merging for "free".

- Abstract out the rbtree sorting. Gets rid of duplicated code in as/cfq/deadline.

- General shrinkage of the request structure.

- Killing dynamic rq private structures in deadline/as/cfq. This should speed up the io path somewhat, as we avoid allocating several structures (struct request + scheduler private request) for each io request.

Quoted 7 lines

- meta data io logging for blktrace.

- CFQ improvements.

- Make the block layer configurable through Kconfig (David Howells).

- Lots of cleanups.

Does it also address the strange "max_sectors_kb<>192 causes a 50%-slowdown" problem?

Thanks!

--
Al

#1 lore raw
Quoted 27 lines · Al Boldi
Quoted 26 lines · Jens Axboe

This lists the main features of the 'block' branch, which is bound for Linus when 2.6.19 opens:

- Splitting of request->flags into two parts:
- cmd type
- modified flags
Right now it's a bit of a mess, splitting this up invites a cleaner usage and also enables us to implement generic "messages" passed on the regular queue for the device.

- Abstract out the request back merging and put it into the core io scheduler layer. Cleans up all the io schedulers, and noop gets merging for "free".

- Abstract out the rbtree sorting. Gets rid of duplicated code in as/cfq/deadline.

- General shrinkage of the request structure.

- Killing dynamic rq private structures in deadline/as/cfq. This should speed up the io path somewhat, as we avoid allocating several structures (struct request + scheduler private request) for each io request.

- meta data io logging for blktrace.

Quoted 8 lines
Quoted 5 lines

- CFQ improvements.

- Make the block layer configurable through Kconfig (David Howells).

- Lots of cleanups.

Does it also address the strange "max_sectors_kb<>192 causes a 50%-slowdown" problem?

(remember to cc me/others when replying, I can easily miss lkml messages for several days otherwise).

It does not, the investigation of that is still pending I'm afraid. The data is really puzzling, I'm inclined to think it's drive related. Are you reproducing it just one box/drive, or on several?

Signature · 1 line
Jens Axboe
author
#2 lore raw
Quoted 37 lines · Jens Axboe
Quoted 27 lines · Al Boldi
Quoted 26 lines · Jens Axboe

This lists the main features of the 'block' branch, which is bound for Linus when 2.6.19 opens:

- Splitting of request->flags into two parts:
- cmd type
- modified flags
Right now it's a bit of a mess, splitting this up invites a cleaner usage and also enables us to implement generic "messages" passed on the regular queue for the device.

- Abstract out the request back merging and put it into the core io scheduler layer. Cleans up all the io schedulers, and noop gets merging for "free".

- Abstract out the rbtree sorting. Gets rid of duplicated code in as/cfq/deadline.

- General shrinkage of the request structure.

- Killing dynamic rq private structures in deadline/as/cfq. This should speed up the io path somewhat, as we avoid allocating several structures (struct request + scheduler private request) for each io request.

- meta data io logging for blktrace.

Quoted 8 lines
Quoted 5 lines

- CFQ improvements.

- Make the block layer configurable through Kconfig (David Howells).

- Lots of cleanups.

Does it also address the strange "max_sectors_kb<>192 causes a 50%-slowdown" problem?

Quoted 6 lines

(remember to cc me/others when replying, I can easily miss lkml messages for several days otherwise).

It does not, the investigation of that is still pending I'm afraid. The data is really puzzling, I'm inclined to think it's drive related. Are you reproducing it just one box/drive, or on several?

Several boxes, same drive.

/dev/hda:

ATA device, with non-removable media

	Model Number:       WDC WD1200JB-00DUA0                     
	Serial Number:      WD-WMACM1007651
	Firmware Revision:  65.13G65
Standards:
	Supported: 6 5 4 3 
	Likely used: 6
Configuration:
	Logical		max	current
	cylinders	16383	65535
	heads		16	1
	sectors/track	63	63
	--
	CHS current addressable sectors:    4128705
	LBA    user addressable sectors:  234441648
	LBA48  user addressable sectors:  234441648
	device size with M = 1024*1024:      114473 MBytes
	device size with M = 1000*1000:      120034 MBytes (120 GB)

Capabilities:
LBA, IORDY(can be disabled)
bytes avail on r/w long: 98 Queue depth: 1
Standby timer values: spec'd by Standard, with device specific minimum
R/W multiple sector transfer: Max = 16 Current = 16
Recommended acoustic management value: 128, current value: 254
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 *udma5
Cycle time: min=120ns recommended=120ns
PIO: pio0 pio1 pio2 pio3 pio4
Cycle time: no flow control=120ns IORDY flow control=120ns
Commands/features:
Enabled Supported:
* READ BUFFER cmd
* WRITE BUFFER cmd
* Host Protected Area feature set
* Look-ahead
* Write cache
* Power Management feature set
Security Mode feature set
* SMART feature set
* FLUSH CACHE EXT command
* Mandatory FLUSH CACHE command
* Device Configuration Overlay feature set
* 48-bit Address feature set
Automatic Acoustic Management feature set
SET MAX security extension
* DOWNLOAD MICROCODE cmd
* SMART self-test
* SMART error logging
Security:
supported
not enabled
not locked
frozen
not expired: security count
not supported: enhanced erase
HW reset results:
CBLID- above Vih
Device num = 0 determined by the jumper
Checksum: correct

Thanks!

--
Al

#3 lore raw
Quoted 38 lines · Al Boldi
Quoted 37 lines · Jens Axboe
Quoted 35 lines · Al Boldi
Quoted 24 lines · Jens Axboe

This lists the main features of the 'block' branch, which is bound for Linus when 2.6.19 opens:

- Splitting of request->flags into two parts:
- cmd type
- modified flags
Right now it's a bit of a mess, splitting this up invites a cleaner usage and also enables us to implement generic "messages" passed on the regular queue for the device.

- Abstract out the request back merging and put it into the core io scheduler layer. Cleans up all the io schedulers, and noop gets merging for "free".

- Abstract out the rbtree sorting. Gets rid of duplicated code in as/cfq/deadline.

- General shrinkage of the request structure.

- Killing dynamic rq private structures in deadline/as/cfq. This should speed up the io path somewhat, as we avoid allocating several structures (struct request + scheduler private request) for each io request.

Quoted 7 lines

- meta data io logging for blktrace.

- CFQ improvements.

- Make the block layer configurable through Kconfig (David Howells).

- Lots of cleanups.

Does it also address the strange "max_sectors_kb<>192 causes a 50%-slowdown" problem?

Quoted 8 lines
Quoted 6 lines

(remember to cc me/others when replying, I can easily miss lkml messages for several days otherwise).

It does not, the investigation of that is still pending I'm afraid. The data is really puzzling, I'm inclined to think it's drive related. Are you reproducing it just one box/drive, or on several?

Several boxes, same drive.

Have you / can you try and reproduce with another drive as well? It'd be an interesting data point.

Signature · 1 line
Jens Axboe
author
#4 lore raw
Quoted 43 lines · Jens Axboe
Quoted 42 lines · Al Boldi
Quoted 41 lines · Jens Axboe
Quoted 29 lines · Al Boldi
Quoted 28 lines · Jens Axboe

This lists the main features of the 'block' branch, which is bound for Linus when 2.6.19 opens:

- Splitting of request->flags into two parts:
- cmd type
- modified flags
Right now it's a bit of a mess, splitting this up invites a
cleaner usage and also enables us to implement generic "messages" passed on the regular queue for the device.

- Abstract out the request back merging and put it into the core io scheduler layer. Cleans up all the io schedulers, and noop gets merging for "free".

- Abstract out the rbtree sorting. Gets rid of duplicated code in as/cfq/deadline.

- General shrinkage of the request structure.

- Killing dynamic rq private structures in deadline/as/cfq. This should speed up the io path somewhat, as we avoid allocating several structures (struct request + scheduler private request) for each io request.

- meta data io logging for blktrace.

- CFQ improvements.

Quoted 7 lines
Quoted 4 lines

- Make the block layer configurable through Kconfig (David Howells).

- Lots of cleanups.

Does it also address the strange "max_sectors_kb<>192 causes a 50%-slowdown" problem?

(remember to cc me/others when replying, I can easily miss lkml messages for several days otherwise).

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

It does not, the investigation of that is still pending I'm afraid. The data is really puzzling, I'm inclined to think it's drive related. Are you reproducing it just one box/drive, or on several?

Several boxes, same drive.

Have you / can you try and reproduce with another drive as well? It'd be an interesting data point.

Not really, as this drive is the only one supporting max_sectors_kb>128.

Thanks!

--
Al

#5 lore raw
In reply to Al Boldi
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Al" == Al Boldi <[email protected]> writes:

Quoted 37 lines · Al> Jens Axboe wrote:
Quoted 37 lines · Al Boldi
Quoted 35 lines
Quoted 24 lines · Jens Axboe

This lists the main features of the 'block' branch, which is bound for Linus when 2.6.19 opens:

- Splitting of request->flags into two parts:
- cmd type
- modified flags
Right now it's a bit of a mess, splitting this up invites a cleaner usage and also enables us to implement generic "messages" passed on the regular queue for the device.

- Abstract out the request back merging and put it into the core io scheduler layer. Cleans up all the io schedulers, and noop gets merging for "free".

- Abstract out the rbtree sorting. Gets rid of duplicated code in as/cfq/deadline.

- General shrinkage of the request structure.

- Killing dynamic rq private structures in deadline/as/cfq. This should speed up the io path somewhat, as we avoid allocating several structures (struct request + scheduler private request) for each io request.

Quoted 7 lines

- meta data io logging for blktrace.

- CFQ improvements.

- Make the block layer configurable through Kconfig (David Howells).

- Lots of cleanups.

Does it also address the strange "max_sectors_kb<>192 causes a 50%-slowdown" problem?

Quoted 6 lines
Quoted 6 lines

(remember to cc me/others when replying, I can easily miss lkml messages for several days otherwise).

It does not, the investigation of that is still pending I'm afraid. The data is really puzzling, I'm inclined to think it's drive related. Are you reproducing it just one box/drive, or on several?

Al> Several boxes, same drive.

Al> /dev/hda:

Al> ATA device, with non-removable media

Al> 	Model Number:       WDC WD1200JB-00DUA0                     
Al> 	Serial Number:      WD-WMACM1007651
Al> 	Firmware Revision:  65.13G65

Al> Standards:
Al> Supported: 6 5 4 3
Al> Likely used: 6

I've got a pair of drives which are very close in model type, and I can run some non-destructive tests on them if you like to confirm what's going on here if you like:

/dev/hde:

ATA device, with non-removable media
Model Number: WDC WD1200JB-00CRA1
Serial Number: WD-WMA8C4365875
Firmware Revision: 17.07W17
Standards:
Supported: 5 4 3
Likely used: 6
jfsnew:~> sudo hdparm -I /dev/hdg | head

/dev/hdg:

ATA device, with non-removable media
Model Number: WDC WD1200JB-00EVA0
Serial Number: WD-WMAEK2844058
Firmware Revision: 15.05R15
Standards:
Supported: 6 5 4
Likely used: 6

The drives have different defaul max_sectors too:

Quoted 1 line

cat /sys/block/hdg/queue/max_sectors_kb

    512
    > cat /sys/block/hde/queue/max_sectors_kb
    128

Let me know your test method and I'll run it here and post the results.

John

author
#6 lore raw
Quoted 88 lines · John Stoffel
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Al" == Al Boldi <[email protected]> writes:

Quoted 38 lines · Al> Jens Axboe wrote:
Quoted 38 lines · Al Boldi
Quoted 36 lines
Quoted 24 lines · Jens Axboe

This lists the main features of the 'block' branch, which is bound for Linus when 2.6.19 opens:

- Splitting of request->flags into two parts:
- cmd type
- modified flags
Right now it's a bit of a mess, splitting this up invites a
cleaner usage and also enables us to implement generic "messages" passed on the regular queue for the device.

- Abstract out the request back merging and put it into the core io scheduler layer. Cleans up all the io schedulers, and noop gets merging for "free".

- Abstract out the rbtree sorting. Gets rid of duplicated code in as/cfq/deadline.

- General shrinkage of the request structure.

- Killing dynamic rq private structures in deadline/as/cfq. This should speed up the io path somewhat, as we avoid allocating several structures (struct request + scheduler private request) for each io request.

Quoted 8 lines

- meta data io logging for blktrace.

- CFQ improvements.

- Make the block layer configurable through Kconfig (David Howells).

- Lots of cleanups.

Does it also address the strange "max_sectors_kb<>192 causes a 50%-slowdown" problem?

Quoted 6 lines
Quoted 6 lines

(remember to cc me/others when replying, I can easily miss lkml messages for several days otherwise).

It does not, the investigation of that is still pending I'm afraid. The data is really puzzling, I'm inclined to think it's drive related. Are you reproducing it just one box/drive, or on several?

Al> Several boxes, same drive.

Al> /dev/hda:

Al> ATA device, with non-removable media

Al> 	Model Number:       WDC WD1200JB-00DUA0
Al> 	Serial Number:      WD-WMACM1007651
Al> 	Firmware Revision:  65.13G65

Al> Standards:
Al> Supported: 6 5 4 3
Al> Likely used: 6

I've got a pair of drives which are very close in model type, and I can run some non-destructive tests on them if you like to confirm what's going on here if you like:

/dev/hde:

ATA device, with non-removable media
Model Number: WDC WD1200JB-00CRA1
Serial Number: WD-WMA8C4365875
Firmware Revision: 17.07W17
Standards:
Supported: 5 4 3
Likely used: 6
jfsnew:~> sudo hdparm -I /dev/hdg | head

/dev/hdg:

ATA device, with non-removable media
Model Number: WDC WD1200JB-00EVA0
Serial Number: WD-WMAEK2844058
Firmware Revision: 15.05R15
Standards:
Supported: 6 5 4
Likely used: 6

The drives have different defaul max_sectors too:

Quoted 1 line

cat /sys/block/hdg/queue/max_sectors_kb

Quoted 8 lines
    512

    > cat /sys/block/hde/queue/max_sectors_kb

    128

Let me know your test method and I'll run it here and post the results.

Thanks for your input!

Are you running UDMA5? Can you dump the full hdparm -I

What thruput do you get with cat /dev/hd[eg] > /dev/null?

What does cat /sys/block/hde/queue/read_ahead_kb say?

Then try this for best performance:
echo 192 > /sys/block/hde/queue/max_sectors_kb
echo 192 > /sys/block/hde/queue/read_ahead_kb

and repeat the thruput test reporting vmstat results.

Thanks!

--
Al

#7 lore raw
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Al" == Al Boldi <[email protected]> writes:

Al> Thanks for your input!

You're welcome, sorry I've been delayed in getting more details to you. I'll try and run some tests tonight, but the six month old kept me up alot last niht.

Al> Are you running UDMA5? Can you dump the full hdparm -I

Here's the info for each drive. They're on an HPT302 Rev 1 controller in my system.

/dev/hde:

ATA device, with non-removable media

        Model Number:       WDC WD1200JB-00CRA1                     
        Serial Number:      WD-WMA8C4365875
        Firmware Revision:  17.07W17

Standards:
Supported: 5 4 3
Likely used: 6
Configuration:

        Logical         max     current
        cylinders       16383   16383
        heads           16      16
        sectors/track   63      63
        --
        CHS current addressable sectors:   16514064
        LBA    user addressable sectors:  234441648
        device size with M = 1024*1024:      114473 MBytes
        device size with M = 1000*1000:      120034 MBytes (120 GB)

Capabilities:
LBA, IORDY(can be disabled)
bytes avail on r/w long: 40
Standby timer values: spec'd by Standard, with device specific
minimum
R/W multiple sector transfer: Max = 16 Current = 16 Recommended acoustic management value: 128, current value: 254
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 *udma5
Cycle time: min=120ns recommended=120ns
PIO: pio0 pio1 pio2 pio3 pio4
Cycle time: no flow control=120ns IORDY flow
control=120ns
Commands/features:
Enabled Supported:
* SMART feature set
Security Mode feature set
* Power Management feature set
* Write cache
* Look-ahead
* Host Protected Area feature set
* WRITE_BUFFER command
* READ_BUFFER command
* DOWNLOAD_MICROCODE
SET_MAX security extension
Automatic Acoustic Management feature set
* Device Configuration Overlay feature set
* SMART error logging
* SMART self-test
Security:
supported
not enabled
not locked
not frozen
not expired: security count
not supported: enhanced erase
HW reset results:
CBLID- above Vih
Device num = 0 determined by the jumper
Checksum: correct

/dev/hdg:

ATA device, with non-removable media

        Model Number:       WDC WD1200JB-00EVA0                     
        Serial Number:      WD-WMAEK2844058
        Firmware Revision:  15.05R15

Standards:
Supported: 6 5 4
Likely used: 6
Configuration:

        Logical         max     current
        cylinders       16383   65535
        heads           16      1
        sectors/track   63      63
        --
        CHS current addressable sectors:    4128705
        LBA    user addressable sectors:  234441648
        LBA48  user addressable sectors:  234441648
        device size with M = 1024*1024:      114473 MBytes
        device size with M = 1000*1000:      120034 MBytes (120 GB)

Capabilities:
LBA, IORDY(can be disabled)
Standby timer values: spec'd by Standard, with device specific
minimum
R/W multiple sector transfer: Max = 16 Current = 16 Recommended acoustic management value: 128, current value: 254
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 *udma5
Cycle time: min=120ns recommended=120ns
PIO: pio0 pio1 pio2 pio3 pio4
Cycle time: no flow control=120ns IORDY flow
control=120ns
Commands/features:
Enabled Supported:
* SMART feature set
Security Mode feature set
* Power Management feature set
* Write cache
* Look-ahead
* Host Protected Area feature set
* WRITE_BUFFER command
* READ_BUFFER command
* DOWNLOAD_MICROCODE
SET_MAX security extension
Automatic Acoustic Management feature set
* 48-bit Address feature set
* Device Configuration Overlay feature set
* Mandatory FLUSH_CACHE
* FLUSH_CACHE_EXT
* SMART error logging
* SMART self-test
Security:
supported
not enabled
not locked
not frozen
not expired: security count
not supported: enhanced erase
HW reset results:
CBLID- above Vih
Device num = 0 determined by CSEL
Checksum: correct

Al> What thruput do you get with cat /dev/hd[eg] > /dev/null?

Al> What does cat /sys/block/hde/queue/read_ahead_kb say?

Quoted 1 line

cat /sys/block/hde/queue/max_hw_sectors_kb

128
> cat /sys/block/hde/queue/max_sectors_kb
128
> cat /sys/block/hde/queue/read_ahead_kb
1024
> cat /sys/block/hde/queue/scheduler
noop [anticipatory] deadline cfq
> cat /sys/block/hde/queue/nr_requests
128

> cat /sys/block/hdg/queue/max_hw_sectors_kb
1024
> cat /sys/block/hdg/queue/max_sectors_kb
512
> cat /sys/block/hdg/queue/read_ahead_kb
1024
> cat /sys/block/hdg/queue/scheduler
noop [anticipatory] deadline cfq
> cat /sys/block/hdg/queue/nr_requests
128

Al> Then try this for best performance:
Al> echo 192 > /sys/block/hde/queue/max_sectors_kb Al> echo 192 > /sys/block/hde/queue/read_ahead_kb

Al> and repeat the thruput test reporting vmstat results.

I'll try and do this all tonight if I get a chance.

Quoted 1 line

From looking at my setup, I've got two identical drives which are

slightly different in terms of BIOS levels, as well as in native capabilities, so this should be a useful test.

I've got a third drive of the same type, but it's in a USB/FireWire enclosure and I can't hdparm it directly.

John