regarding crazy head unloads

Hello, ATA/SMART fellows.

This message is regarding crazy head unloads on certain laptops. In a desperate attempt to increase battery time, some vendors configure ATA APM (advanced power management) too aggressive to the point of being fragile (can even be triggered on Windows) and the drive unloads head like crazy and kills itself quickly (in months). For more information, please take a look at the following links.

https://bugzilla.novell.com/show_bug.cgi?id=386555
https://bugs.launchpad.net/ubuntu/+source/acpi-support/+bug/59695
http://www.thinkwiki.org/wiki/Problem_with_hard_drive_clicking

This primarily is those hardware vendors' faults and updating their firmware is probably the best way to fix it; however, it can actually kill the harddrive which usually causes a lot of anxiety and stress on the user, so I think we need to take some measures.

Attached are storage-fixup script which is to be called during boot and resume and configuration file to go under /etc. The script can match dmi and hal properties and execute commands on the matching devices. The config file currently only contains three rules.

Here are two ideas to better handle this problem:

1. Describe the problem on linux-ata.org and ask people to report dmidecode and hdparm -I output on affected machines. Share storage-fixup (or any other alternative) and storage-fixup.conf on the page.

2. This is from Roland. Make smartd aware of the problem and warn user if load/unload count per powered on hours goes too high. Maybe the warning can direct the user to linux-ata.org page?

Thanks.

tejun

#! /bin/bash
#
# storage-fixup - Tejun Heo <[email protected]>
#
# Script to issue fix up commands for weird disks. This is primarily

# to adjust ATA APM setting.  Some laptop BIOSen set this value too
# aggressively causing frequent head unloads which can kill the drive
# quickly.  This script should be called during boot and resume.  It
# examines rules from /etc/stroage-fixup.conf and executes matching
# commands.
#
# In stroage-fixup.conf, empty lines and lines starting w/ # are
# ignored.  Each line starts with rule, dmi, hal or act.
#
# rule RULENAME
#	Starts a rule.  $RULENAME can't contain whitespaces.
#
# dmi KEY VALUE
#	Checks whether DMI value for KEY matches VALUE.  If not, the
#	rule is skipped.
#
# hal KEY VALUE
#	Checks whether there are devices which has KEY value matching
#	VALUE.  storage-fixup determines applies actions to devices
#	which match all hal matches, so all rules should have at least
#	one hal match.
#
# act ACTION
#	Executes ACTION on matched devices.  ACTION can contain $DEV
#	which will be substituted with device file of matching device.
#
# For example, the following (useless) rule disables APM on the first
# harddrive of my machine.
#
# rule p5w64
# dmi baseboard-product-name	P5W64 WS Pro
# dmi baseboard-manufacturer	ASUSTeK Computer INC.
# hal storage.model		WDC WD5000YS-01M
# hal storage.serial		SATA_WDC_WD5000YS-01_WD-WMANU1217262
# act hdparm -B 255 $DEV
#

conf_file=${CONF_FILE:-/etc/storage-fixup.conf} hal_find_by_property=${HAL_FIND_BY_PROPERTY:-hal-find-by-property} hal_get_property=${HAL_GET_PROPERTY:-hal-get-property} dmidecode=${DMIDECODE:-dmidecode}

verbose=0
lineno=0
skip=0
rule_name=""
declare -a dev_ids
newline=$'\n'

log() {
echo "storage-fixup: $@"
}

warn() {
log "$@" 1>&2
}

debug() {
if [ $verbose -ne 0 ]; then
warn "$@"
fi
}

#
# Match functions - do_dmi() and do_hal() - execute DMI and HAL

# matches respectively.  Return value 0 indicates match, 1 invalid
# match (triggers warning) and 2 mismatch.
#
do_dmi() {
    local val

    if [ -z "$1" -o -z "$2" ]; then
	return 1
    fi

    val=$($dmidecode --string "$1")
    if [ "$?" -ne 0 ]; then
	return 1
    fi

    if [ "$val" = "$2" ]; then
	debug "Y $lineno $rule_name dmi $1=$2"
	return 0;
    fi

    debug "N $lineno $rule_name dmi $1=$2"
    return 2
}

do_hal() {
    local i out ifs_store append=0

    if [ -z "$1" -o -z "$2" ]; then
	return 1
    fi

    if [ ${#dev_ids[@]} -eq 0 ]; then
	append=1
    fi

    #
    # bash really isn't a good programming language for this kind of
    # stuff and makes it look much more complex than it needs to be.
    # The following loop executes hal-find-by-property and ands the
    # result with the previous result.
    #
    ifs_store="$IFS"
    IFS="$newline"
    dev_ids=(
	$($hal_find_by_property --key "$1" --string "$2" \
	    | while read found; do
		if [ $append -ne 0 ]; then
		    echo "$found"
		else
		    for id in "${dev_ids[@]}"; do
			if [ "$id" = "$found" ]; then
			    echo "$found"
			    break
			fi
		    done
		fi
	    done))
    IFS="$ifs_store"

    if [ "$?" -ne 0 ]; then
	debug "N $lineno $rule_name hal $1=$2"
	return 2
    fi

    debug "Y $lineno $rule_name hal nr_devs=${#dev_ids[@]} $1=$2"

    return 0
}

do_act() {
    local id dev

    for id in "${dev_ids[@]}"; do
	if ! DEV=$($hal_get_property --udi "$id" --key block.device); then
	    warn "can't find device node for $id"
	    continue
	fi

	eval log "$rule_name: executing \"$1\""
	eval "$1"
    done

    return 0
}

while read f0 f1 f2; do

    true $((lineno++))
    if [ -z ${f0###*} ]; then
	continue
    fi

    if [ "$f0" = rule ]; then
	rule_name=$f1
	skip=0
	dev_ids=()
	continue
    fi

    if [ $skip -ne 0 ]; then
	continue
    fi

    case "$f0" in
    dmi)
	    do_dmi "$f1" "$f2"
	    ;;
    hal)
	    do_hal "$f1" "$f2"
	    ;;
    act)
	    do_act "$f1 $f2"
	    ;;
    *)
	    false
	    ;;
    esac

    ret=$?
    if [ $ret -ne 0 ]; then
	if [ $ret -eq 1 ]; then
	    warn "malformed line $lineno \"$f0 $f1 $f2\","\
	         "skipping rule $rule_name" 2>&1
	fi
	skip=1
    fi

done < $conf_file

rule tp-t60
dmi system-manufacturer LENOVO
dmi system-product-name 1952W5R
dmi system-version ThinkPad T60
hal storage.model Hitachi HTS722020K9SA00
act hdparm -B 255 $DEV

rule hp-dv6500
dmi system-manufacturer Hewlett-Packard
dmi system-product-name HP Pavilion dv6500 Notebook PC
dmi system-version Rev 1
hal storage.model SAMSUNG HM250JI
act hdparm -B 255 $DEV

rule dell-e1505
dmi system-manufacturer Dell Inc.
dmi system-product-name MM061
hal storage.model ST9100824AS
act hdparm -B 255 $DEV

Here's updated version which can do glob matching and other stuff. I've also set up a git tree.

http://git.kernel.org/?p=linux/kernel/git/tj/storage-fixup.git

tejun

#! /bin/bash
#
# storage-fixup - Tejun Heo <[email protected]>
#
# Script to issue fix up commands for weird disks. This is primarily

# to adjust ATA APM setting.  Some laptop BIOSen set this value too
# aggressively causing frequent head unloads which can kill the drive
# quickly.  This script should be called during boot and resume.  It
# examines rules from /etc/stroage-fixup.conf and executes matching
# commands.
#
# In stroage-fixup.conf, empty lines and lines starting w/ # are
# ignored.  Each line starts with rule, dmi, hal or act.
#
# rule RULENAME
#	Starts a rule.  $RULENAME can't contain whitespaces.
#
# dmi KEY PATTERN
#	Checks whether DMI value for KEY matches PATTERN.  If not, the
#	rule is skipped.
#
# hal KEY PATTERN
#	Checks whether there are devices which has KEY value matching
#	PATTERN.  storage-fixup determines applies actions to devices
#	which match all hal matches, so all rules should have at least
#	one hal match.
#
# act ACTION
#	Executes ACTION on matched devices.  ACTION can contain $DEV
#	which will be substituted with device file of matching device.
#
# PATTERN is bash glob pattern.
#
# For example, the following (useless) rule disables APM on the first
# harddrive of my machine.
#
# rule p5w64
# dmi baseboard-product-name	P5W64 WS Pro
# dmi baseboard-manufacturer	ASUSTeK Computer INC.
# hal storage.model		WDC WD5000YS-01M
# hal storage.serial		*-01_WD-WMANU1217262
# act hdparm -B 255 $DEV
#

declare usage="
Usage: storage-fixup [-h] [-V] [-v] [-b] [-c config_file]

       -h      Print this help message and exit
       -V      Print version and exit
       -v      Verbose
       -d      Dry run, don't actually execute action
       -c      Use config_file instead of /etc/storage-fixup.conf

"

declare hal_find_by_capability=${HAL_FIND_BY_CAPABILITY:-hal-find-by-capability} declare hal_get_property=${HAL_GET_PROPERTY:-hal-get-property} declare dmidecode=${DMIDECODE:-dmidecode}

declare version=0.1
declare conf_file=/etc/storage-fixup.conf

declare newline=$'\n'
declare dry_run=0 verbose=0 lineno=0 skip=0 rule_name="" reply declare -a storage_ids
declare -a hal_cache
declare -a matches

log() {
echo "storage-fixup: $@"
}

warn() {
log "$@" 1>&2
}

debug() {
if [ $verbose -ne 0 ]; then
warn "$@"
fi
}

#

# do_dmi - perform DMI match
# @key: DMI key to be passed as --string argument to dmidecode
# @pattern: glob pattern to match
#
# Returns 0 on match, 1 on mismatch, 2 on invalid match (triggers
# warning).
#
do_dmi() {
    local key="$1" pattern="$2"
    local val

    if [ -z "$key" -o -z "$pattern" ]; then
	return 1
    fi

    val=$($dmidecode --string "$key")
    if [ "$?" -ne 0 ]; then
	return 2
    fi

    if [ -z "${val##$pattern}" ]; then
	debug "Y $lineno $rule_name dmi $key=$pattern"
	return 0
    fi

    debug "N $lineno $rule_name dmi $key=$pattern"
    return 1
}

#

# search_hal_cache - search hal cache
# @id: udi of the device to search for
# @key: key of hal property to search
#
# Searches hal cache and returns 0 if found, 1 if @key properties are
# cached but matching entry is not found, 2 if @key properties are not
# cached yet.  On success, the matched property is returned in $reply.
#
search_hal_cache() {
    local id="$1" key="$2"
    local i key_found=0 cache len match

    reply=

    for ((i=0;i<${#hal_cache[@]};i++)); do
	cache=${hal_cache[i]}
	len=${#cache}

	match="${cache#$key }"
	if [ ${#match} -ne $len ]; then
	    key_found=1
	elif [ $key_found -eq 1 ]; then
	    return 1
	fi

	match="${cache#$key $id }"
	if [ ${#match} -ne $len ]; then
	    reply="$match"
	    return 0
	fi
    done

    if [ $key_found -eq 1 ]; then
	return 1
    else
	return 2
    fi
}

#

# fetch_hal_property - fetch hal property matching id and key
# @id: udi of the device to fetch property for
# @key: key of the property to fetch
#
# Fetch @key property for udi @id.  If @key properties are already
# cached, it's returned from cache.  If not, cache is populated with
# @key properties and searched again.
#
# Returns 0 if found, 1 if not found, 2 if something went wrong.  On
# success, the matched property is returned in $reply.
#
fetch_hal_property() {
    local id="$1" key="$2" property
    local i ret tid cnt=0

    # search cache
    search_hal_cache "$id" "$key"
    ret=$?
    if [ $ret -ne 2 ]; then
	return $ret
    fi

    # $key wasn't in the cache, populate the cache

    # placeholder indicating $key has been populated
    hal_cache+=("$key ")

    # run hal-get-property on each storage device and put the result in cache
    for ((i=0;i<${#storage_ids[@]};i++)); do
	tid="${storage_ids[i]}"
	property="$($hal_get_property --udi "$tid" --key "$key")"
	if [ -n "$property" ]; then
	    hal_cache+=("$key $tid $property")
	    true $((cnt++))
	fi
    done

    debug "C $cnt entries added to hal cache for $key"

    # and retry
    search_hal_cache "$id" "$key"
    return $?
}

#

# do_hal - perform HAL match
# @key: property key of interest
# @pattern: pattern to match
#
# Walk through $matches array and match each id against @key and
# @pattern.  Entries which don't match are removed from $matches.
#
# Returns 0 if $matches contain any entry after matching, 1 if it's
# empty, 2 if something went wrong.
#
do_hal() {
    local key="$1" pattern="$2" property i
    local -a old_matches=("${matches[@]}")

    if [ -z "$1" -o -z "$2" ]; then
	return 2
    fi

    matches=()

    for ((i=0;i<${#old_matches[@]};i++)); do
	fetch_hal_property "${old_matches[i]}" "$key"
	if [ $? -eq 0 -a -z "${reply##$pattern}" ]; then
	    matches+=("${old_matches[i]}")
	fi
    done

    if [ ${#matches[@]} -eq 0 ]; then
	debug "N $lineno $rule_name hal $1=$2"
	return 1
    fi

    debug "Y $lineno $rule_name hal nr_devs=${#matches[@]} $1=$2"

    return 0
}

#

# do_act - execute action
# @act: action to execute
#
# Execute @act for each device in $matches.  "$DEV" in @act is
# substituted with the /dev node of each match.  If $dry_run is set,
# the action is logged but not actually executed.
#
# Returns 0.
#
do_act() {
    local act="$1"
    local id dev

    for id in "${matches[@]}"; do
	if ! DEV=$($hal_get_property --udi "$id" --key block.device); then
	    warn "can't find device node for $id"
	    continue
	fi

	if [ $dry_run -eq 0 ]; then
	    eval log "$rule_name: executing \"$act\""
	    eval "$1"
	else
	    eval log "$rule_name: dry-run \"$act\""
	fi
    done

    return 0
}

#
# Execution starts here
#
while getopts "dvVc:h" option; do

    case $option in
	d)
	    dry_run=1;;
	v)
	    verbose=1;;
	V)
	    echo "$version"
	    exit 0;;
	c)
	    conf_file=$OPTARG;;
	*)
	    echo "$usage" 2>&1
	    exit 1;;
    esac

done

storage_ids=($($hal_find_by_capability --capability storage)) debug "I ${#storage_ids[@]} storage devices"

while read f0 f1 f2; do

    true $((lineno++))
    if [ -z ${f0###*} ]; then
	continue
    fi

    if [ "$f0" = rule ]; then
	rule_name=$f1
	skip=0
	matches=("${storage_ids[@]}")
	continue
    fi

    if [ $skip -ne 0 ]; then
	continue
    fi

    case "$f0" in
    dmi)
	    do_dmi "$f1" "$f2"
	    ;;
    hal)
	    do_hal "$f1" "$f2"
	    ;;
    act)
	    do_act "$f1 $f2"
	    ;;
    *)
	    false
	    ;;
    esac

    ret=$?
    if [ $ret -ne 0 ]; then
	if [ $ret -eq 2 ]; then
	    warn "malformed line $lineno \"$f0 $f1 $f2\","\
	         "skipping rule $rule_name" 2>&1
	fi
	skip=1
    fi

done < $conf_file

#

# /etc/storage-fixup.conf - Configuration file for storage-fixup
#
# Blank lines and lines starting with # are ignored.  Please read
# comment at the top of storage-fixup for more information.
#
# Drive model patterns are generalized to cover drives from the same
# family.  Drive manufacturers usually have datasheets or web pages
# listing all models of the same family.
#
# The DMI part is difficult to generalize as there's no such
# information.  We'll have to generalize as we collect entries.
#
# If you have a harddrive which does crazy unloading but not listed
# here, please write to [email protected] with the outputs of
# "dmidecode" and "hdparm -I DRIVE", on a laptop the DRIVE is usually
# /dev/sda.
#

# Reported drive model: Hitachi HTS722020K9SA00
rule tp-t60
dmi system-manufacturer LENOVO
dmi system-product-name 1952W5R
dmi system-version ThinkPad T60
hal storage.model Hitachi HTS7220*K9*A*
act hdparm -B 255 $DEV

# Reported drive model: SAMSUNG HM250JI
rule hp-dv6500
dmi system-manufacturer Hewlett-Packard
dmi system-product-name HP Pavilion dv6500 Notebook PC
dmi system-version Rev 1
hal storage.model SAMSUNG HM*I
act hdparm -B 255 $DEV

# Reported drive model: ST9100824AS
rule dell-e1505
dmi system-manufacturer Dell Inc.
dmi system-product-name MM061
hal storage.model ST9*AS
act hdparm -B 255 $DEV

Could this not be done better inside HAL itself?

Scott

Scott James Remnant
[email protected]

Could be. I really don't know about HAL. There were several problems w/ HAL and I had to drop HAL dependency tho. HAL depends on a lot of things and thus is started too late which makes it impossible to use in early stages of boot (e.g. single mode). Another problem was that HAL currently uses truncated ATA model ID string reported through SCSI layer. The updated version is in the following git tree.

http://git.kernel.org/?p=linux/kernel/git/tj/storage-fixup.git;a=shortlog;h=master

If there's any better way to deal with this, I'll be happy to drop the ugly script.

Thanks.

tejun

Hi Tejun,

Sorry, this is a really really slow reply. This email came at an unusually busy time and I did not reply before it got buried in the deluge.

It would be easy to implement this in smartd. But I'm concerned about three things:

-- most people don't run smartd (I think by default it is off on
the main Linux distros). Those wise enough to run smartd will probably be aware of the issue, since it's gotten lots of buzz. And even if smartd is turned on in the distro it is normally not configured to send warning emails.

-- I'm not sure if the attribute used for head load/unload is a standard
one. On my current laptop Hitachi disk it is attribute 193. Does anyone know if this is standard?

-- I'm not sure at what threshold to send warning emails. If we send
too many false alarms, they will be ignored by users.

Let me know if you have suggestions or comments about these points.

Cheers,
Bruce

Well, you were slow but I'm slower. :-)

We only have to hit one instance of each configuration where such problem is occurring. I think adding warning message out smartctl output or letting smartd warn it to users via email or log messages should be good enough for this problem.

I have no idea whatsoever. Maybe implement a regexp matching on attribute name?

To get it correct completely, I think we'll need per-device database. Not easy.... :-(

Thanks.

tejun