Skip to contentSkip to conversation

BUG: CD driver sends command during host removal

DISCUSSION Alan Stern 5 participants 21 messages Sep 27, 2004 – Oct 2, 2004 1 message not in the archive

Subsystems (3)
SCSI USB USB mass storage driver
lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): 20040926082926.GA1944@uniball

Alan Stern

author maintainer
#1 lore raw

I received the following error report, showing that something in the CD driver attempts to send a command to a USB CD-RW drive when the host is removed, leading to an oops. This was on a 2.6.9-rc2 system (Mohammed please correct me if that's wrong).

The initial part of the log shows a perfectly normal probe and scan of the drive. Here are the important bits:

Quoted 18 lines · Mohammed Sameer
Sep 26 11:20:48 localhost kernel: usb-storage: Command INQUIRY (6 bytes)
Sep 26 11:20:48 localhost kernel: usb-storage:  12 00 00 00 24 00
Sep 26 11:20:48 localhost kernel: usb-storage: scsi cmd done, result=0x0
Sep 26 11:20:48 localhost kernel:   Vendor: MSI       Model: CD-RW CR52        Rev: 3.70
Sep 26 11:20:48 localhost kernel:   Type:   CD-ROM                             ANSI SCSI revision: 02
Sep 26 11:20:48 localhost kernel: usb-storage: Command TEST_UNIT_READY (6 bytes)
Sep 26 11:20:48 localhost kernel: usb-storage:  00 00 00 00 00 00
Sep 26 11:20:48 localhost kernel: usb-storage: -- transport indicates command failure
Sep 26 11:20:48 localhost kernel: usb-storage: Issuing auto-REQUEST_SENSE
Sep 26 11:20:48 localhost kernel: usb-storage: -- code: 0x70, key: 0x6, ASC: 0x29, ASCQ: 0x0
Sep 26 11:20:48 localhost kernel: usb-storage: Unit Attention: Power on, reset, or bus device reset occurred
Sep 26 11:20:48 localhost kernel: usb-storage: scsi cmd done, result=0x2
Sep 26 11:20:48 localhost kernel: usb-storage: Command TEST_UNIT_READY (6 bytes)
Sep 26 11:20:48 localhost kernel: usb-storage:  00 00 00 00 00 00
Sep 26 11:20:48 localhost kernel: usb-storage: scsi cmd done, result=0x0
Quoted 5 lines
Sep 26 11:20:48 localhost kernel: usb-storage: Command MODE_SENSE_10 (10 bytes)
Sep 26 11:20:48 localhost kernel: usb-storage:  5a 00 2a 00 00 00 00 00 80 00
Sep 26 11:20:48 localhost kernel: usb-storage: scsi cmd done, result=0x0
Sep 26 11:20:48 localhost kernel: sr0: scsi3-mmc drive: 40x/40x writer cd/rw xa/form2 cdda tray
Sep 26 11:20:48 localhost kernel: Attached scsi CD-ROM sr0 at scsi0, channel 0, id 0, lun 0Sep 26 11:20:54 localhost kernel: usb 1-2: USB disconnect, address 2

Up to here everything looks okay. Then the user unplugged the USB cable:

Quoted 2 lines
Sep 26 11:20:54 localhost kernel: usb-storage: storage_disconnect() called
Sep 26 11:20:54 localhost kernel: usb-storage: usb_stor_stop_transport called

Next usb-storage called scsi_remove_host(). Apparently this caused some component of the CD driver to queue a command:

Quoted 4 lines
Sep 26 11:20:54 localhost kernel: usb-storage: queuecommand called
Sep 26 11:20:54 localhost kernel: usb-storage: *** thread awakened.
Sep 26 11:20:54 localhost kernel: usb-storage: No command during disconnect
Sep 26 11:20:54 localhost kernel: usb-storage: *** thread sleeping.

usb-storage accepted the command but then ignored it because the host was in process of removal. Should the queuecommand routine have rejected the command? This would involve a race, because it's possible for queuecommand to accept a command and then scsi_remove_host() to be called before the command is carried out.

After five seconds the command timed out:

Quoted 2 lines
Sep 26 11:20:59 localhost kernel: usb-storage: command_abort called
Sep 26 11:20:59 localhost kernel: usb-storage: -- nothing to abort

usb-storage ignored the request to abort the command (the command_abort routine returned FAILED because no command was running). So error recovery proceeded to try a device reset and then a bus reset. Neither one was allowed:

Quoted 4 lines
Sep 26 11:20:59 localhost kernel: usb-storage: device_reset called
Sep 26 11:20:59 localhost kernel: usb-storage: No reset during disconnect
Sep 26 11:20:59 localhost kernel: usb-storage: bus_reset called
Sep 26 11:20:59 localhost kernel: usb-storage: No reset during disconnect

usb-storage doesn't define a host reset, so error recovery gave up:

Quoted 45 lines
Sep 26 11:20:59 localhost kernel: scsi: Device offlined - not ready after error recovery: host 0 channel 0 id 0 lun 0
Sep 26 11:20:59 localhost kernel: Badness in scsi_device_set_state at drivers/scsi/scsi_lib.c:1688
Sep 26 11:20:59 localhost kernel:  [<cf92d24f>] scsi_device_set_state+0xc4/0x112 [scsi_mod]Sep 26 11:20:59 localhost kernel:  [<cf92afa0>] scsi_eh_offline_sdevs+0x64/0x80 [scsi_mod]
Sep 26 11:20:59 localhost kernel:  [<cf92b4b0>] scsi_unjam_host+0xb6/0x1eb [scsi_mod]
Sep 26 11:20:59 localhost kernel:  [<c0115777>] default_wake_function+0x0/0x12
Sep 26 11:20:59 localhost kernel:  [<cf92b6b4>] scsi_error_handler+0xcf/0x16b [scsi_mod]
Sep 26 11:20:59 localhost kernel:  [<cf92b5e5>] scsi_error_handler+0x0/0x16b [scsi_mod]
Sep 26 11:20:59 localhost kernel:  [<c010425d>] kernel_thread_helper+0x5/0xb
Sep 26 11:20:59 localhost kernel: Badness in kref_get at lib/kref.c:32
Sep 26 11:20:59 localhost kernel:  [<c01aa017>] kref_get+0x44/0x46
Sep 26 11:20:59 localhost kernel:  [<c01a9c65>] kobject_get+0x1a/0x24
Sep 26 11:20:59 localhost kernel:  [<c0218ec6>] get_device+0x18/0x21
Sep 26 11:20:59 localhost kernel:  [<cf92c9c4>] scsi_request_fn+0x25/0x367 [scsi_mod]
Sep 26 11:20:59 localhost kernel:  [<c021f1e2>] blk_insert_request+0xae/0xcc
Sep 26 11:20:59 localhost kernel:  [<c0106428>] dump_stack+0x1c/0x20
Sep 26 11:20:59 localhost kernel:  [<cf92ba11>] scsi_queue_insert+0x89/0xd0 [scsi_mod]
Sep 26 11:20:59 localhost kernel:  [<cf92b381>] scsi_eh_flush_done_q+0x6f/0xe8 [scsi_mod]
Sep 26 11:20:59 localhost kernel:  [<cf92b47c>] scsi_unjam_host+0x82/0x1eb [scsi_mod]
Sep 26 11:20:59 localhost kernel:  [<c0115777>] default_wake_function+0x0/0x12
Sep 26 11:20:59 localhost kernel:  [<cf92b6b4>] scsi_error_handler+0xcf/0x16b [scsi_mod]
Sep 26 11:20:59 localhost kernel:  [<cf92b5e5>] scsi_error_handler+0x0/0x16b [scsi_mod]
Sep 26 11:20:59 localhost kernel:  [<c010425d>] kernel_thread_helper+0x5/0xb
Sep 26 11:20:59 localhost kernel: cf92ee79
Sep 26 11:20:59 localhost kernel: Modules linked in: sr_mod usb_storage ipv6 thermal fan button processor ac battery microcode e100 yenta_socket pcmcia_core ehci_hcd usbcore ext2 dm_mod eepro100 mii toshiba_acpi psmouse pcspkr msr snd_seq_midi snd_intel8x0 snd_ac97_codec snd_pcm_oss snd_mixer_oss snd_pcm snd_page_alloc gameport snd_mpu401_uart snd_rawmidi snd_seq_oss snd_seq_midi_event snd_seq snd_timer snd_seq_device snd soundcore ide_cd cdrom sd_mod scsi_mod rtc unix
Sep 26 11:20:59 localhost kernel: CPU:    0
Sep 26 11:20:59 localhost kernel: EIP:    0060:[<cf92ee79>]    Not tainted VLI
Sep 26 11:20:59 localhost kernel: EFLAGS: 00010082   (2.6.9-rc2-Uniball-1) 
Sep 26 11:20:59 localhost kernel: EIP is at scsi_device_dev_release+0x26/0xeb [scsi_mod]
Sep 26 11:20:59 localhost kernel: eax: c974dd84   ebx: c974dc08   ecx: 00200200   edx: 00100100
Sep 26 11:20:59 localhost kernel: esi: c974dc00   edi: 00000282   ebp: cef084b4   esp: c9091e80
Sep 26 11:20:59 localhost kernel: ds: 007b   es: 007b   ss: 0068
Sep 26 11:20:59 localhost kernel: Process scsi_eh_0 (pid: 2103, threadinfo=c9090000 task=cd2b8aa0)
Sep 26 11:20:59 localhost kernel: Stack: 00000046 c974dda8 c0334488 c03344a0 cef084d8 c0218bf8 c974dd84 c974dda8 
Sep 26 11:20:59 localhost kernel:        c0334488 c03344a0 c01a9d07 c974dda8 c974ddc0 c01a9d09 cef08400 cec76128 
Sep 26 11:20:59 localhost kernel:        c01aa052 c974dda8 cef08400 cec76128 c974dd84 cef302b0 c974dc00 c01a9d31 
Sep 26 11:20:59 localhost kernel: Call Trace:
Sep 26 11:20:59 localhost kernel:  [<c0218bf8>] device_release+0x58/0x5c
Sep 26 11:20:59 localhost kernel:  [<c01a9d07>] kobject_cleanup+0x98/0x9a
Sep 26 11:20:59 localhost kernel:  [<c01a9d09>] kobject_release+0x0/0xa
Sep 26 11:20:59 localhost kernel:  [<c01aa052>] kref_put+0x39/0x93
Sep 26 11:20:59 localhost kernel:  [<c01a9d31>] kobject_put+0x1e/0x22
Sep 26 11:20:59 localhost kernel:  [<c01a9d09>] kobject_release+0x0/0xa
Sep 26 11:20:59 localhost kernel:  [<cf92cb82>] scsi_request_fn+0x1e3/0x367 [scsi_mod]
Sep 26 11:20:59 localhost kernel:  [<c021f1e2>] blk_insert_request+0xae/0xcc
Sep 26 11:20:59 localhost kernel:  [<c0106428>] dump_stack+0x1c/0x20
Quoted 8 lines
Sep 26 11:20:59 localhost kernel:  [<cf92ba11>] scsi_queue_insert+0x89/0xd0 [scsi_mod]
Sep 26 11:20:59 localhost kernel:  [<cf92b381>] scsi_eh_flush_done_q+0x6f/0xe8 [scsi_mod]
Sep 26 11:20:59 localhost kernel:  [<cf92b47c>] scsi_unjam_host+0x82/0x1eb [scsi_mod]
Sep 26 11:20:59 localhost kernel:  [<c0115777>] default_wake_function+0x0/0x12
Sep 26 11:20:59 localhost kernel:  [<cf92b6b4>] scsi_error_handler+0xcf/0x16b [scsi_mod]
Sep 26 11:20:59 localhost kernel:  [<cf92b5e5>] scsi_error_handler+0x0/0x16b [scsi_mod]
Sep 26 11:20:59 localhost kernel:  [<c010425d>] kernel_thread_helper+0x5/0xb
Sep 26 11:20:59 localhost kernel: Code: e9 7c a0 8e f0 55 57 56 53 83 ec 04 8b 44 24 18 8b 68 20 8d b0 7c fe ff ff 9c 5f fa 8d 98 84 fe ff ff 8b 90 84 fe ff ff 8b 4b 04 <89> 4a 04 89 11 c7 43 04 00 02 20 00 8d 98 8c fe ff ff 8b 90 8c

I've omitted the remaining parts of the fault cascade.

Can anyone help?

Alan Stern

#2 lore raw
Quoted 4 lines · Alan Stern

I received the following error report, showing that something in the CD driver attempts to send a command to a USB CD-RW drive when the host is removed, leading to an oops. This was on a 2.6.9-rc2 system (Mohammed please correct me if that's wrong).

thanks Alan for posting this to this mailing list. Yes i'm using a vanilla 2.6.9-rc2 on Debian testing.

I'm now subscribed to this mailing list,
And I can provide any required information.

Best refards,

Signature · 15 lines
----------------
-- Katoob Main Developer
Linux registered user #224950, ICQ #58475622
--
Don't send me any attachment in Micro$oft (.DOC, .PPT) format please
Read http://www.fsf.org/philosophy/no-word-attachments.html
Preferable attachments: .PDF, .HTML, .TXT
Thanx for adding this text to Your signature
--
-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GCM/IT d-(++)@ s+(++):->+++ a-- C+++$>++++ UL+++$>++++ P+++$>+++++
L+++(++++)$>+++++ E>+++ W++?>$ N+>+++ o? K-? !w++ !O !M !V !PS@ !PE@ Y+ PGP=+++
t? 5? !X R? tv-- b+@ DI D+ G-- e++>+++ h-->++ !r y?
------END GEEK CODE BLOCK------
#3 lore raw
In reply to Alan Stern
Quoted 59 lines · Alan Stern

I received the following error report, showing that something in the CD driver attempts to send a command to a USB CD-RW drive when the host is removed, leading to an oops. This was on a 2.6.9-rc2 system (Mohammed please correct me if that's wrong).

The initial part of the log shows a perfectly normal probe and scan of the drive. Here are the important bits:

Quoted 4 lines · Mohammed Sameer
Sep 26 11:20:48 localhost kernel: usb-storage: Command INQUIRY (6 bytes)
Sep 26 11:20:48 localhost kernel: usb-storage:  12 00 00 00 24 00
Sep 26 11:20:48 localhost kernel: usb-storage: scsi cmd done, result=0x0
Sep 26 11:20:48 localhost kernel:   Vendor: MSI       Model: CD-RW

CR52 Rev: 3.70

Quoted 1 line
Sep 26 11:20:48 localhost kernel:   Type:

CD-ROM ANSI SCSI revision: 02

Quoted 1 line
Sep 26 11:20:48 localhost kernel: usb-storage: Command

TEST_UNIT_READY (6 bytes)

Quoted 2 lines
Sep 26 11:20:48 localhost kernel: usb-storage:  00 00 00 00 00 00
Sep 26 11:20:48 localhost kernel: usb-storage: -- transport indicates

command failure

Quoted 1 line
Sep 26 11:20:48 localhost kernel: usb-storage: Issuing

auto-REQUEST_SENSE

Quoted 1 line
Sep 26 11:20:48 localhost kernel: usb-storage: -- code: 0x70, key:

0x6, ASC: 0x29, ASCQ: 0x0

Quoted 1 line
Sep 26 11:20:48 localhost kernel: usb-storage: Unit Attention: Power

on, reset, or bus device reset occurred

Quoted 3 lines
Sep 26 11:20:48 localhost kernel: usb-storage: scsi cmd done, result=0x2
Sep 26 11:20:48 localhost kernel: usb-storage: Command

TEST_UNIT_READY (6 bytes)

Quoted 4 lines
Sep 26 11:20:48 localhost kernel: usb-storage:  00 00 00 00 00 00
Sep 26 11:20:48 localhost kernel: usb-storage: scsi cmd done, result=0x0
Sep 26 11:20:48 localhost kernel: usb-storage: Command MODE_SENSE_10

(10 bytes)

Quoted 1 line
Sep 26 11:20:48 localhost kernel: usb-storage:  5a 00 2a 00 00 00 00

00 80 00

Quoted 2 lines
Sep 26 11:20:48 localhost kernel: usb-storage: scsi cmd done, result=0x0
Sep 26 11:20:48 localhost kernel: sr0: scsi3-mmc drive: 40x/40x

writer cd/rw xa/form2 cdda tray

Quoted 1 line
Sep 26 11:20:48 localhost kernel: Attached scsi CD-ROM sr0 at scsi0,

channel 0, id 0, lun 0Sep 26 11:20:54 localhost kernel: usb 1-2: USB disconnect, address 2

Up to here everything looks okay. Then the user unplugged the USB cable:

Quoted 1 line
Sep 26 11:20:54 localhost kernel: usb-storage: storage_disconnect()

called

Quoted 1 line
Sep 26 11:20:54 localhost kernel: usb-storage:

usb_stor_stop_transport called

Next usb-storage called scsi_remove_host(). Apparently this caused some component of the CD driver to queue a command:

Quoted 3 lines
Sep 26 11:20:54 localhost kernel: usb-storage: queuecommand called
Sep 26 11:20:54 localhost kernel: usb-storage: *** thread awakened.
Sep 26 11:20:54 localhost kernel: usb-storage: No command during
Quoted 6 lines

disconnect

Quoted 1 line
Sep 26 11:20:54 localhost kernel: usb-storage: *** thread sleeping.

usb-storage accepted the command but then ignored it because the host was in process of removal. Should the queuecommand routine have rejected the command?

Yes, if the service delivery subsystem (SDS) knows that the device is gone and the command wouldn't be delivered, it should *not* "ignore" the command, but return it with error.

I.e. if the LLDD has active/most recent knowledge about the device whereto the command is destined, it should act on that and return an appropriate error. After all, this is what a properly implemented SDS would do.

Quoted 3 lines

This would involve a race, because it's possible for
queuecommand to accept a command and then scsi_remove_host() to be called before the command is carried out.

If the command hasn't been carried out, then delivery would fail and SDS would return the appropriate error back to SCSI Core.

If the command has already been delivered, SCSI core would preempt it without waiting for it to timeout. This is part of proper error recovery, as it knows that the device disappeared -- notified by usb-storage.

Quoted 8 lines

After five seconds the command timed out:

Quoted 2 lines
Sep 26 11:20:59 localhost kernel: usb-storage: command_abort called
Sep 26 11:20:59 localhost kernel: usb-storage: -- nothing to abort

usb-storage ignored the request to abort the command (the command_abort routine returned FAILED because no command was running). So error

Where *was* the command? From the point of time when queuecommand() is called until scsi_done() is called, the command belongs to the LLDD. It should honor any TMF, regardless of the _state_ of the task.

Luben

Quoted 117 lines

recovery proceeded to try a device reset and then a bus reset. Neither one was allowed:

Quoted 2 lines
Sep 26 11:20:59 localhost kernel: usb-storage: device_reset called
Sep 26 11:20:59 localhost kernel: usb-storage: No reset during

disconnect

Quoted 2 lines
Sep 26 11:20:59 localhost kernel: usb-storage: bus_reset called
Sep 26 11:20:59 localhost kernel: usb-storage: No reset during

disconnect

usb-storage doesn't define a host reset, so error recovery gave up:

Quoted 1 line
Sep 26 11:20:59 localhost kernel: scsi: Device offlined - not ready

after error recovery: host 0 channel 0 id 0 lun 0

Quoted 1 line
Sep 26 11:20:59 localhost kernel: Badness in scsi_device_set_state at

drivers/scsi/scsi_lib.c:1688

Quoted 1 line
Sep 26 11:20:59 localhost kernel:  [<cf92d24f>]

scsi_device_set_state+0xc4/0x112 [scsi_mod]Sep 26 11:20:59 localhost kernel: [<cf92afa0>] scsi_eh_offline_sdevs+0x64/0x80 [scsi_mod]

Quoted 1 line
Sep 26 11:20:59 localhost kernel:  [<cf92b4b0>] 
scsi_unjam_host+0xb6/0x1eb [scsi_mod]
 > Sep 26 11:20:59 localhost kernel:  [<c0115777>] 
default_wake_function+0x0/0x12
 > Sep 26 11:20:59 localhost kernel:  [<cf92b6b4>] 
scsi_error_handler+0xcf/0x16b [scsi_mod]
 > Sep 26 11:20:59 localhost kernel:  [<cf92b5e5>] 
scsi_error_handler+0x0/0x16b [scsi_mod]
 > Sep 26 11:20:59 localhost kernel:  [<c010425d>] 
kernel_thread_helper+0x5/0xb
 > Sep 26 11:20:59 localhost kernel: Badness in kref_get at lib/kref.c:32
 > Sep 26 11:20:59 localhost kernel:  [<c01aa017>] kref_get+0x44/0x46
 > Sep 26 11:20:59 localhost kernel:  [<c01a9c65>] kobject_get+0x1a/0x24
 > Sep 26 11:20:59 localhost kernel:  [<c0218ec6>] get_device+0x18/0x21
 > Sep 26 11:20:59 localhost kernel:  [<cf92c9c4>] 
scsi_request_fn+0x25/0x367 [scsi_mod]
 > Sep 26 11:20:59 localhost kernel:  [<c021f1e2>] 
blk_insert_request+0xae/0xcc
 > Sep 26 11:20:59 localhost kernel:  [<c0106428>] dump_stack+0x1c/0x20
 > Sep 26 11:20:59 localhost kernel:  [<cf92ba11>] 
scsi_queue_insert+0x89/0xd0 [scsi_mod]
 > Sep 26 11:20:59 localhost kernel:  [<cf92b381>] 
scsi_eh_flush_done_q+0x6f/0xe8 [scsi_mod]
 > Sep 26 11:20:59 localhost kernel:  [<cf92b47c>] 
scsi_unjam_host+0x82/0x1eb [scsi_mod]
 > Sep 26 11:20:59 localhost kernel:  [<c0115777>] 
default_wake_function+0x0/0x12
 > Sep 26 11:20:59 localhost kernel:  [<cf92b6b4>] 
scsi_error_handler+0xcf/0x16b [scsi_mod]
 > Sep 26 11:20:59 localhost kernel:  [<cf92b5e5>] 
scsi_error_handler+0x0/0x16b [scsi_mod]
 > Sep 26 11:20:59 localhost kernel:  [<c010425d>] 
kernel_thread_helper+0x5/0xb
 > Sep 26 11:20:59 localhost kernel: cf92ee79
 > Sep 26 11:20:59 localhost kernel: Modules linked in: sr_mod

usb_storage ipv6 thermal fan button processor ac battery microcode e100 yenta_socket pcmcia_core ehci_hcd usbcore ext2 dm_mod eepro100 mii toshiba_acpi psmouse pcspkr msr snd_seq_midi snd_intel8x0 snd_ac97_codec snd_pcm_oss snd_mixer_oss snd_pcm snd_page_alloc gameport snd_mpu401_uart snd_rawmidi snd_seq_oss snd_seq_midi_event snd_seq snd_timer snd_seq_device snd soundcore ide_cd cdrom sd_mod scsi_mod rtc unix

Quoted 2 lines
Sep 26 11:20:59 localhost kernel: CPU:    0
Sep 26 11:20:59 localhost kernel: EIP:    0060:[<cf92ee79>]    Not

tainted VLI

Quoted 1 line
Sep 26 11:20:59 localhost kernel: EFLAGS: 00010082

(2.6.9-rc2-Uniball-1)

Quoted 1 line
Sep 26 11:20:59 localhost kernel: EIP is at 
scsi_device_dev_release+0x26/0xeb [scsi_mod]
 > Sep 26 11:20:59 localhost kernel: eax: c974dd84   ebx: c974dc08   
ecx: 00200200   edx: 00100100
 > Sep 26 11:20:59 localhost kernel: esi: c974dc00   edi: 00000282   
ebp: cef084b4   esp: c9091e80
 > Sep 26 11:20:59 localhost kernel: ds: 007b   es: 007b   ss: 0068
 > Sep 26 11:20:59 localhost kernel: Process scsi_eh_0 (pid: 2103,

threadinfo=c9090000 task=cd2b8aa0)

Quoted 1 line
Sep 26 11:20:59 localhost kernel: Stack: 00000046 c974dda8 c0334488

c03344a0 cef084d8 c0218bf8 c974dd84 c974dda8

Quoted 1 line
Sep 26 11:20:59 localhost kernel:        c0334488 c03344a0 c01a9d07

c974dda8 c974ddc0 c01a9d09 cef08400 cec76128

Quoted 1 line
Sep 26 11:20:59 localhost kernel:        c01aa052 c974dda8 cef08400

cec76128 c974dd84 cef302b0 c974dc00 c01a9d31

Quoted 3 lines
Sep 26 11:20:59 localhost kernel: Call Trace:
Sep 26 11:20:59 localhost kernel:  [<c0218bf8>] device_release+0x58/0x5c
Sep 26 11:20:59 localhost kernel:  [<c01a9d07>] 
kobject_cleanup+0x98/0x9a
 > Sep 26 11:20:59 localhost kernel:  [<c01a9d09>] kobject_release+0x0/0xa
 > Sep 26 11:20:59 localhost kernel:  [<c01aa052>] kref_put+0x39/0x93
 > Sep 26 11:20:59 localhost kernel:  [<c01a9d31>] kobject_put+0x1e/0x22
 > Sep 26 11:20:59 localhost kernel:  [<c01a9d09>] kobject_release+0x0/0xa
 > Sep 26 11:20:59 localhost kernel:  [<cf92cb82>] 
scsi_request_fn+0x1e3/0x367 [scsi_mod]
 > Sep 26 11:20:59 localhost kernel:  [<c021f1e2>] 
blk_insert_request+0xae/0xcc
 > Sep 26 11:20:59 localhost kernel:  [<c0106428>] dump_stack+0x1c/0x20
 > Sep 26 11:20:59 localhost kernel:  [<cf92ba11>] 
scsi_queue_insert+0x89/0xd0 [scsi_mod]
 > Sep 26 11:20:59 localhost kernel:  [<cf92b381>] 
scsi_eh_flush_done_q+0x6f/0xe8 [scsi_mod]
 > Sep 26 11:20:59 localhost kernel:  [<cf92b47c>] 
scsi_unjam_host+0x82/0x1eb [scsi_mod]
 > Sep 26 11:20:59 localhost kernel:  [<c0115777>] 
default_wake_function+0x0/0x12
 > Sep 26 11:20:59 localhost kernel:  [<cf92b6b4>] 
scsi_error_handler+0xcf/0x16b [scsi_mod]
 > Sep 26 11:20:59 localhost kernel:  [<cf92b5e5>] 
scsi_error_handler+0x0/0x16b [scsi_mod]
 > Sep 26 11:20:59 localhost kernel:  [<c010425d>] 
kernel_thread_helper+0x5/0xb
 > Sep 26 11:20:59 localhost kernel: Code: e9 7c a0 8e f0 55 57 56 53 83

ec 04 8b 44 24 18 8b 68 20 8d b0 7c fe ff ff 9c 5f fa 8d 98 84 fe ff ff 8b 90 84 fe ff ff 8b 4b 04 <89> 4a 04 89 11 c7 43 04 00 02 20 00 8d 98 8c fe ff ff 8b 90 8c

I've omitted the remaining parts of the fault cascade.

Can anyone help?

Alan Stern

-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

author maintainer
#4 lore raw
Quoted 2 lines · Luben Tuikov
Quoted 2 lines

Next usb-storage called scsi_remove_host(). Apparently this caused some component of the CD driver to queue a command:

This sounds like a bug, by the way. Commands shouldn't be queued because of a call to scsi_remove_host!

Quoted 10 lines
Quoted 9 lines
Quoted 3 lines
Sep 26 11:20:54 localhost kernel: usb-storage: queuecommand called
Sep 26 11:20:54 localhost kernel: usb-storage: *** thread awakened.
Sep 26 11:20:54 localhost kernel: usb-storage: No command during

disconnect

Quoted 1 line
Sep 26 11:20:54 localhost kernel: usb-storage: *** thread sleeping.

usb-storage accepted the command but then ignored it because the host was in process of removal. Should the queuecommand routine have rejected the command?

Quoted 8 lines

Yes, if the service delivery subsystem (SDS) knows that the device is gone and the command wouldn't be delivered, it should *not* "ignore" the command, but return it with error.

I.e. if the LLDD has active/most recent knowledge about the device whereto the command is destined, it should act on that and return an appropriate error. After all, this is what a properly implemented SDS would do.

According to Documentation/scsi/scsi_mid_low_api.txt, the only possible error returns are SCSI_MLQUEUE_DEVICE_BUSY and SCSI_MLQUEUE_HOST_BUSY. Neither is appropriate; should the second one be returned?

Quoted 6 lines
Quoted 3 lines

This would involve a race, because it's possible for
queuecommand to accept a command and then scsi_remove_host() to be called before the command is carried out.

If the command hasn't been carried out, then delivery would fail and SDS would return the appropriate error back to SCSI Core.

How? The SCSI core deallocates the scsi_cmnd before the SDS has a chance to return anything.

Quoted 3 lines

Where *was* the command? From the point of time when queuecommand() is called until scsi_done() is called, the command belongs to the LLDD. It should honor any TMF, regardless of the _state_ of the task.

If the command belongs to the LLDD, why does scsi_remove_host do the following:

calls scsi_host_cancel,
which calls scsi_device_cancel_cb for each device, which calls scsi_device_cancel,
which calls scsi_finish_command for each active command, which passes the command back to the upper layer

Either there's a bug in the host removal sequence, or else the LLDD doesn't own any requests once scsi_remove_host has been called.

Alan Stern

#5 lore raw
Quoted 8 lines · Alan Stern [[email protected]] wrote:
Quoted 2 lines · Luben Tuikov
Quoted 2 lines

Next usb-storage called scsi_remove_host(). Apparently this caused some component of the CD driver to queue a command:

This sounds like a bug, by the way. Commands shouldn't be queued because of a call to scsi_remove_host!

There was a reordering in the scsi_remove_host by me that may be causing your problem. Ths change was to allow devices to receive there SYNCRHONIZE_CACHE commands on rmmod.

You can see part of the thread here.

http://marc.theaimsgroup.com/?t=108701426000002&r=1&w=2

-andmike
--
Michael Anderson
[email protected]

#6 lore raw
In reply to Alan Stern
Quoted 6 lines · Alan Stern
Quoted 3 lines
Quoted 3 lines

Next usb-storage called scsi_remove_host(). Apparently this caused some component of the CD driver to queue a command:

This sounds like a bug, by the way. Commands shouldn't be queued because of a call to scsi_remove_host!

Yes.

Quoted 8 lines
Quoted 8 lines
Quoted 3 lines

usb-storage accepted the command but then ignored it because the host was in process of removal. Should the queuecommand routine have rejected the command?

Yes, if the service delivery subsystem (SDS) knows that the device is gone and the command wouldn't be delivered, it should *not* "ignore" the command, but return it with error.

Quoted 7 lines
Quoted 4 lines

I.e. if the LLDD has active/most recent knowledge about the device whereto the command is destined, it should act on that and return an appropriate error. After all, this is what a properly implemented SDS would do.

According to Documentation/scsi/scsi_mid_low_api.txt, the only possible error returns are SCSI_MLQUEUE_DEVICE_BUSY and SCSI_MLQUEUE_HOST_BUSY. Neither is appropriate; should the second one be returned?

I believe internally SCSI Core returns DID_ERROR.

Quoted 10 lines
Quoted 7 lines
Quoted 4 lines

This would involve a race, because it's possible for queuecommand to accept a command and then scsi_remove_host() to be called before the command is carried out.

If the command hasn't been carried out, then delivery would fail and SDS would return the appropriate error back to SCSI Core.

How? The SCSI core deallocates the scsi_cmnd before the SDS has a chance to return anything.

Hmm, once queuecommand() has been called, SCSI Core *should NOT* touch the struct command until the LLDD calls scsi_done() or it times out and ownership is given back indirectly via the appropriate return result of the times_out() function.

Quoted 8 lines
Quoted 3 lines

Where *was* the command? From the point of time when queuecommand() is called until scsi_done() is called, the command belongs to the LLDD. It should honor any TMF, regardless of the _state_ of the task.

If the command belongs to the LLDD, why does scsi_remove_host do the following:

Quoted 7 lines

calls scsi_host_cancel,
which calls scsi_device_cancel_cb for each device, which calls scsi_device_cancel,
which calls scsi_finish_command for each active command, which passes the command back to the upper layer

Either there's a bug in the host removal sequence, or else the LLDD doesn't own any requests once scsi_remove_host has been called.

Ah, definitely sounds like a bug -- the LLDD has not been given a chance to "return" the struct command.

One thing I wanted to point out is that in scsi_remove_host() the _very_ first thing which should be done is setting the proper shost_state, SHOST_DEL, which should imply SHOST_CANCEL (by virtue of meaning), as opposed to "doubly" setting it.

_Thought_ experiment: is it possibe to "catch" a command between a non-canceled host but canceled device (of that host)?

So, first the host state is set to "cancelled", then each device is set accordingly, then commands sent to each device are "recovered" (all this top->down); and then
the resources freed in opposite order: commands, devices, hosts. This may involve waiting for the LLDD to respond in the recovery process.

Luben

#7 lore raw
Quoted 5 lines · Luben Tuikov
Quoted 3 lines

According to Documentation/scsi/scsi_mid_low_api.txt, the only possible error returns are SCSI_MLQUEUE_DEVICE_BUSY and SCSI_MLQUEUE_HOST_BUSY. Neither is appropriate; should the second one be returned?

I believe internally SCSI Core returns DID_ERROR.

For a device that no-longer exists, DID_NO_CONNECT is probably the most appropriately descriptive.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

This would involve a race, because it's possible for queuecommand to accept a command and then scsi_remove_host() to be called before the command is carried out.

Correct, scsi_remove_host() is asynchronous ... you can get requests after calling it while the queue is halting ... you must error these.

Quoted 11 lines
Quoted 11 lines

If the command belongs to the LLDD, why does scsi_remove_host do the following:

calls scsi_host_cancel,
which calls scsi_device_cancel_cb for each device, which calls scsi_device_cancel,
which calls scsi_finish_command for each active command, which passes the command back to the upper layer

Either there's a bug in the host removal sequence, or else the LLDD doesn't own any requests once scsi_remove_host has been called.

Right. scsi_remove_host tells the mid-layer that it's OK to trash all inflight commands because you removed all their users before calling it. It also tells us that you won't accept any future commands for this host (because you'll error any attempt in queuecommand).

James

#8 lore raw
Quoted 9 lines · James Bottomley
Quoted 3 lines · Luben Tuikov
Quoted 1 line

According to Documentation/scsi/scsi_mid_low_api.txt, the only possible error returns are SCSI_MLQUEUE_DEVICE_BUSY and SCSI_MLQUEUE_HOST_BUSY. Neither is appropriate; should the second one be returned?

I believe internally SCSI Core returns DID_ERROR.

For a device that no-longer exists, DID_NO_CONNECT is probably the most appropriately descriptive.

Does this mean that scsi_device_cancel() should set the result code to DID_NO_CONNECT either?

Quoted 18 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

This would involve a race, because it's possible for queuecommand to accept a command and then scsi_remove_host() to be called before the command is carried out.

Correct, scsi_remove_host() is asynchronous ... you can get requests after calling it while the queue is halting ... you must error these.

Quoted 9 lines
Quoted 9 lines

If the command belongs to the LLDD, why does scsi_remove_host do the following:

calls scsi_host_cancel,
which calls scsi_device_cancel_cb for each device, which calls scsi_device_cancel,
which calls scsi_finish_command for each active command, which passes the command back to the upper layer

Quoted 7 lines
Quoted 1 line
Quoted 1 line

Either there's a bug in the host removal sequence, or else the LLDD doesn't own any requests once scsi_remove_host has been called.

Right. scsi_remove_host tells the mid-layer that it's OK to trash all inflight commands because you removed all their users before calling it. It also tells us that you won't accept any future commands for this host (because you'll error any attempt in queuecommand).

Do you mean to say that when scsi_remove_host() is called, the LLDD must no own any commands? This is good, it means that the LLDD plugged queuecommand(), did "recovery" of pending commands and is ready for slave_destroy().

But does this also mean that scsi_device_cancel() (cancel_pending_io) is unnecessary in the scsi_remove_host() path as there's no outstanding IO to the host?

Luben

#9 lore raw
Quoted 2 lines · Luben Tuikov

Does this mean that scsi_device_cancel() should set the result code to DID_NO_CONNECT either?

No, I think for in-flight commands, DID_ERROR is the safest. They could, after all have partially deposited their data load before being cancelled. DID_NO_CONNECT means that no processing was ever done.

Quoted 9 lines
Quoted 4 lines

Right. scsi_remove_host tells the mid-layer that it's OK to trash all inflight commands because you removed all their users before calling it. It also tells us that you won't accept any future commands for this host (because you'll error any attempt in queuecommand).

Do you mean to say that when scsi_remove_host() is called, the LLDD must no own any commands? This is good, it means that the LLDD plugged queuecommand(), did "recovery" of pending commands and is ready for slave_destroy().

No. It's up to the LLD. If it wants to complete all its commands then call scsi_remove_host(), that's fine. However it's also fine for it to call scsi_remove_host() with pending in-flight commands; if it does, ownership of these is transferred back to the mid-layer and we cancel them all.

James

author maintainer
#10 lore raw
In reply to Luben Tuikov
Quoted 9 lines · James Bottomley
Quoted 5 lines · Luben Tuikov
Quoted 3 lines

According to Documentation/scsi/scsi_mid_low_api.txt, the only possible error returns are SCSI_MLQUEUE_DEVICE_BUSY and SCSI_MLQUEUE_HOST_BUSY. Neither is appropriate; should the second one be returned?

I believe internally SCSI Core returns DID_ERROR.

For a device that no-longer exists, DID_NO_CONNECT is probably the most appropriately descriptive.

Regardless of how descriptive the value is, the code in scsi_dispatch_cmd treats anything other than SCSI_MLQUEUE_DEVICE_BUSY as SCSI_MLQUEUE_HOST_BUSY. Will this matter?

Quoted 4 lines · Luben Tuikov

Hmm, once queuecommand() has been called, SCSI Core *should NOT* touch the struct command until the LLDD calls scsi_done() or it times out and ownership is given back indirectly via the appropriate return result of the times_out() function.

Quoted 4 lines · James Bottomley

Right. scsi_remove_host tells the mid-layer that it's OK to trash all inflight commands because you removed all their users before calling it. It also tells us that you won't accept any future commands for this host (because you'll error any attempt in queuecommand).

It sounds like the two of you are in contradiction. Should the SCSI core deallocate in-flight commands without consulting the LLDD or shouldn't it?

Alan Stern

#11 lore raw
Quoted 7 lines · Alan Stern [[email protected]] wrote:
Quoted 5 lines · James Bottomley
Quoted 4 lines · Luben Tuikov
Quoted 3 lines

According to Documentation/scsi/scsi_mid_low_api.txt, the only possible error returns are SCSI_MLQUEUE_DEVICE_BUSY and SCSI_MLQUEUE_HOST_BUSY. Neither is appropriate; should the second one be returned?

Quoted 9 lines
Quoted 4 lines
Quoted 1 line

I believe internally SCSI Core returns DID_ERROR.

For a device that no-longer exists, DID_NO_CONNECT is probably the most appropriately descriptive.

Regardless of how descriptive the value is, the code in scsi_dispatch_cmd treats anything other than SCSI_MLQUEUE_DEVICE_BUSY as SCSI_MLQUEUE_HOST_BUSY. Will this matter?

You would not want to return non zero status from your queucommand, but set the result of the command to DID_NO_CONNECT and call done.

-andmike
--
Michael Anderson
[email protected]

#12 lore raw
In reply to Alan Stern
Quoted 9 lines · Alan Stern
Quoted 4 lines · Luben Tuikov

Hmm, once queuecommand() has been called, SCSI Core *should NOT* touch the struct command until the LLDD calls scsi_done() or it times out and ownership is given back indirectly via the appropriate return result of the times_out() function.

On 29 Sep 2004, James Bottomley wrote:

Quoted 7 lines
Quoted 4 lines

Right. scsi_remove_host tells the mid-layer that it's OK to trash all inflight commands because you removed all their users before calling it. It also tells us that you won't accept any future commands for this host (because you'll error any attempt in queuecommand).

It sounds like the two of you are in contradiction. Should the SCSI core deallocate in-flight commands without consulting the LLDD or shouldn't it?

If the host is really gone, then no pending commands would ever "return".

This means that the LLDD can call scsi_done() immediately, without any waiting or delay of any sort.

Basically, when your host is gone, plug queuecommand(), call scsi_done() on all your pending commands, and call scsi_remove_host(). The only problem would be if you have to sleep while "recovering" the pending commands, but I cannot imagine why you'd do that as your host is gone.

Now since LLDD shouldn't keep internal queues of pending commands, ideally, this function is left to SCSI Core.

Luben

#13 lore raw
In reply to Alan Stern
Quoted 3 lines · Alan Stern

Regardless of how descriptive the value is, the code in scsi_dispatch_cmd treats anything other than SCSI_MLQUEUE_DEVICE_BUSY as SCSI_MLQUEUE_HOST_BUSY. Will this matter?

Yes, you have to call cmd->done() having set the result field to DID_NO_CONNECT; then return zero from queuecommand() indicating that you're dealling with the command.

Quoted 2 lines

It sounds like the two of you are in contradiction. Should the SCSI core deallocate in-flight commands without consulting the LLDD or shouldn't it?

Once you've called scsi_remove_host() the mid-layer will take control of your in-flight commands (if there are any) and error them back to the user. You need to take any actions to clean up internal structures belonging to the commands *before* you do a scsi_remove_host.

James

#14 lore raw
Quoted 5 lines · James Bottomley [[email protected]] wrote:

Once you've called scsi_remove_host() the mid-layer will take control of your in-flight commands (if there are any) and error them back to the user. You need to take any actions to clean up internal structures belonging to the commands *before* you do a scsi_remove_host.

How do we address the problem we get into with the reordering in scsi_remove_host (i.e. the call to scsi_forget_host prior to scsi_host_cancel) if we possibly generate new io from sd_sync_cache that will cause the error handler to fire up if the LLDD just lets the command fall on the floor with no response.

-andmike
--
Michael Anderson
[email protected]

#15 lore raw
Quoted 7 lines · Mike Anderson
Quoted 5 lines · James Bottomley [[email protected]] wrote:

Once you've called scsi_remove_host() the mid-layer will take control of your in-flight commands (if there are any) and error them back to the user. You need to take any actions to clean up internal structures belonging to the commands *before* you do a scsi_remove_host.

Quoted 5 lines

How do we address the problem we get into with the reordering in scsi_remove_host (i.e. the call to scsi_forget_host prior to scsi_host_cancel) if we possibly generate new io from sd_sync_cache that will cause the error handler to fire up if the LLDD just lets the command fall on the floor with no response.

That's why LLD's are responsible for erroring all commands issued at this time if the removal is a surprise ejection. The commands have to be errored in a way (like DID_NO_CONNECT) that won't excite the error handler.

James

#16 lore raw
Quoted 14 lines · James Bottomley [[email protected]] wrote:
Quoted 7 lines · Mike Anderson
Quoted 5 lines · James Bottomley [[email protected]] wrote:

Once you've called scsi_remove_host() the mid-layer will take control of your in-flight commands (if there are any) and error them back to the user. You need to take any actions to clean up internal structures belonging to the commands *before* you do a scsi_remove_host.

Quoted 5 lines

How do we address the problem we get into with the reordering in scsi_remove_host (i.e. the call to scsi_forget_host prior to scsi_host_cancel) if we possibly generate new io from sd_sync_cache that will cause the error handler to fire up if the LLDD just lets the command fall on the floor with no response.

Quoted 5 lines

That's why LLD's are responsible for erroring all commands issued at this time if the removal is a surprise ejection. The commands have to be errored in a way (like DID_NO_CONNECT) that won't excite the error handler.

ok, thanks for the clarification. Your previous statement seemed to imply once scsi_remove_host was called the LLDD had no responsibility for calling done on commands sent to the LLDDs queuecommand.

So now that this thread and the other thread related to similar shutdown issues has grown long is the next step to see if we can get the usb queuecommand to return DID_NO_CONNECT in this shutdown case.

-andmike
--
Michael Anderson
[email protected]

#17 lore raw
Quoted 3 lines · Mike Anderson

ok, thanks for the clarification. Your previous statement seemed to imply once scsi_remove_host was called the LLDD had no responsibility for calling done on commands sent to the LLDDs queuecommand.

Hang on a minute, there are two cases:

Existing in-flight commands:

Here, either the LLD returns done on them or the mid-layer will cancel them during the invocation of scsi_remove_host(). In the latter case, the LLD doesn't need to call done on them.

New Commands going down into queuecommand:

These the LLD must error out (by returning done with an error status).

Quoted 3 lines

So now that this thread and the other thread related to similar shutdown issues has grown long is the next step to see if we can get the usb queuecommand to return DID_NO_CONNECT in this shutdown case.

Yes, that sounds correct.

James

#18 lore raw
Quoted 6 lines · James Bottomley
Quoted 1 line · Mike Anderson

ok, thanks for the clarification. Your previous statement seemed to

imply

Quoted 2 lines

once scsi_remove_host was called the LLDD had no responsibility for calling done on commands sent to the LLDDs queuecommand.

Quoted 7 lines

Hang on a minute, there are two cases:

Existing in-flight commands:

Here, either the LLD returns done on them or the mid-layer will cancel them during the invocation of scsi_remove_host(). In the latter case, the LLD doesn't need to call done on them.

Is it possible that there could be a slight race when the LLDD starts calling scsi_done() on pending commands, _after_ it has called scsi_remove_host()?

BTW, why have/leave two alternate paths for this behavior? This may bite us back in the future when things get more complicated, i.e. dealing with more complicated devices, transports, etc.

Quoted 3 lines

New Commands going down into queuecommand:

These the LLD must error out (by returning done with an error status).

Right at queuecommand() invocation.

Luben

Quoted 9 lines
Quoted 3 lines

So now that this thread and the other thread related to similar shutdown issues has grown long is the next step to see if we can get the usb queuecommand to return DID_NO_CONNECT in this shutdown case.

Yes, that sounds correct.

James

#19 lore raw
Quoted 3 lines · Luben Tuikov

Is it possible that there could be a slight race when the LLDD starts calling scsi_done() on pending commands, _after_ it has called scsi_remove_host()?

No, because it starts failing commmands *before* it calls scsi_remove_host().

Quoted 4 lines

BTW, why have/leave two alternate paths for this behavior? This may bite us back in the future when things get more complicated, i.e. dealing with more complicated devices, transports, etc.

Certain LLDs have no internal track of outstanding commands.

James

author maintainer
#20 lore raw
In reply to Mike Anderson
Quoted 3 lines · Mike Anderson

So now that this thread and the other thread related to similar shutdown issues has grown long is the next step to see if we can get the usb queuecommand to return DID_NO_CONNECT in this shutdown case.

Here's a patch. Mohammed, please try it out and tell us how it works. I just wrote it, so I haven't had a chance to test it myself yet.

Alan Stern

===== drivers/usb/storage/scsiglue.c 1.84 vs edited =====

drivers/usb/storage/scsiglue.c +8 -0
@@ -183,6 +183,14 @@
 		return SCSI_MLQUEUE_HOST_BUSY;
 	}
 
+	/* fail the command if we are disconnecting */
+	if (test_bit(US_FLIDX_DISCONNECTING, &us->flags)) {
+		US_DEBUGP("Command failed for disconnect\n");
+		srb->result = DID_NO_CONNECT << 16;
+		done(srb);
+		return 0;
+	}
+
 	srb->scsi_done = done;
 	us->srb = srb;
#21 lore raw
Quoted 21 lines · Alan Stern
Quoted 3 lines · Mike Anderson

So now that this thread and the other thread related to similar shutdown issues has grown long is the next step to see if we can get the usb queuecommand to return DID_NO_CONNECT in this shutdown case.

Here's a patch. Mohammed, please try it out and tell us how it works. I just wrote it, so I haven't had a chance to test it myself yet.

Alan Stern

===== drivers/usb/storage/scsiglue.c 1.84 vs edited =====

drivers/usb/storage/scsiglue.c +1 -0
@@ -183,6 +183,14 @@
 		return SCSI_MLQUEUE_HOST_BUSY;
 	}
 
+	/* fail the command if we are disconnecting */

drivers/usb/storage/scsiglue.c line 196

Quoted 10 lines
drivers/usb/storage/scsiglue.c +7 -0
+	if (test_bit(US_FLIDX_DISCONNECTING, &us->flags)) {
+		US_DEBUGP("Command failed for disconnect\n");
+		srb->result = DID_NO_CONNECT << 16;
+		done(srb);
+		return 0;
+	}
+
 	srb->scsi_done = done;
 	us->srb = srb;

though patch complained:

patching file drivers/usb/storage/scsiglue.c
Hunk #1 succeeded at 166 (offset -17 lines).

But the patch works fine, Thanks!
May god bless you all!!

Signature · 10 lines
----------------
-- Katoob Main Developer, Arabbix Maintainer.
Linux registered user #224950
Proud Egyptian GNU/Linux User Group <www.eglug.org> Admin.
Life powered by Debian, Homepage: www.foolab.org
--
Don't send me any attachment in Micro$oft (.DOC, .PPT) format please
Read http://www.fsf.org/philosophy/no-word-attachments.html
Preferable attachments: .PDF, .HTML, .TXT
Thanx for adding this text to Your signature