blocksize limitations in scsi tape driver (st) when used with DLT1 tape drives?
Hi,
I have successfully been using Legato Networker with a blocksize of 32k with my DLT here.
But the way Legato wants to do it is to decide about the blocksize itself.
This means that the driver should NOT decide on the BS, but pass on anything written through it, meaning a blocksize setting of 0 (or variable blocksize).
Perhaps Arkeia works the same way ?
Best Luck
Signature · 5 lines
Dag Nygren email: [email protected] Oy Espoon NewTech Ab phone: +358 9 8024910 Träsktorpet 3 fax: +358 9 8024916 02360 ESBO Mobile: +358 400 426312 FINLAND
Quoted 10 lines · Dag Nygren
I have successfully been using Legato Networker with a blocksize of 32k with my DLT here.
But the way Legato wants to do it is to decide about the blocksize itself.
This means that the driver should NOT decide on the BS, but pass on anything written through it, meaning a blocksize setting of 0 (or variable blocksize).Perhaps Arkeia works the same way ?
I used variable blocksize (=0) before but then after about 3.5 gigs stored on the tape I get scsi-error and arkeia interprets this as end-of-tape.
Aug 6 02:13:50 filesrv kernel: st0: Error with sense data: Info fld=0x2000, Deferred st09:00: sns = f1 3 Aug 6 02:13:50 filesrv kernel: ASC=80 ASCQ= 1 Aug 6 02:13:50 filesrv kernel: Raw sense data:0xf1 0x00 0x03 0x00 0x00 0x20 0x00 0x16 0x00 0x00 0xe6 0xc2 0x80 0x01 0x00 0x00 0x00 0x00 0x83 0x00 0x37 0x00 0x00 0x00 0x21 0x00 0x90 0x7f 0x3a 0x00 Aug 6 02:13:50 filesrv kernel: klogd 1.4.1, ---------- state change ---------- Aug 6 02:13:50 filesrv kernel: Inspecting /boot/System.map-2.4.18-64GB-SMP Aug 6 02:13:50 filesrv kernel: Loaded 13537 symbols from /boot/System.map-2.4.18-64GB-SMP. Aug 6 02:13:50 filesrv kernel: Symbols match kernel version 2.4.18. Aug 6 02:13:50 filesrv kernel: Loaded 481 symbols from 13 modules. Aug 6 02:13:50 filesrv kernel: st0: Error with sense data: Info fld=0x1, Current st09:00: sns = f0 3 Aug 6 02:13:50 filesrv kernel: ASC= c ASCQ= 0 Aug 6 02:13:50 filesrv kernel: Raw sense data:0xf0 0x00 0x03 0x00 0x00 0x00 0x01 0x16 0x00 0x00 0xe6 0xc2 0x0c 0x00 0x00 0x00 0x00 0x00 0x83 0x00 0x37 0x00 0x00 0x00 0x21 0x00 0x90 0x7f 0x3a 0x00
I had this problem with LTO drives too and here the arkeia faq tells one to set fixed block size.
And when you look a the HP document referenced in my last mail:
ISSUE: Block sizes of LESS THAN 64 KB for DLT1/DLT VS80 and 32 KB for
all other DAT and DLT drives can drastically increase the backup/restore time and severely affect the performance of the drive.
SOLUTION: Most backup applications allow viewing and adjusting the block
size used for a particular device. See below for advice on how to achieve this for CA ARCserve, Veritas Backup Exec and Tapeware.
(I capitalized "less than" to emphasis its occurance)
I you missed the link here it is again:
http://www.hp.com/cposupport/information_storage/support_doc/lpg50167.html
Did you check with mt after Legato Networker did a backup which blocksize it set?
Signature · 2 lines
MfG / Regards Friedrich Lobenstock
Quoted 10 lines · Friedrich Lobenstock
Quoted 9 lines · Dag Nygren
I have successfully been using Legato Networker with a blocksize of 32k with my DLT here.
But the way Legato wants to do it is to decide about the blocksize itself.
This means that the driver should NOT decide on the BS, but pass on anything written through it, meaning a blocksize setting of 0 (or variable blocksize).
Quoted 7 lines
Quoted 1 line
Perhaps Arkeia works the same way ?
I used variable blocksize (=0) before but then after about 3.5 gigs stored on the tape I get scsi-error and arkeia interprets this as end-of-tape.
Aug 6 02:13:50 filesrv kernel: st0: Error with sense data: Info fld=0x2000, Deferred st09:00: sns = f1 3 Aug 6 02:13:50 filesrv kernel: ASC=80 ASCQ= 1
The sense code 3 is MEDIUM ERROR. The ASC/ASCQ 80 01 is vendor specific. In other words, you have bad tape (or the drive thinks that it is bad). This is returned as an error in writing data to tape. Most applications (hopefully) interpret this as end of tape.
Kai
Quoted 5 lines
Dag Nygren wrote:
I used variable blocksize (=0) before but then after about 3.5 gigs stored on the tape I get scsi-error and arkeia interprets this as end-of-tape.
Quoted 11 lines
Aug 6 02:13:50 filesrv kernel: st0: Error with sense data: Info fld=0x2000, Deferred st09:00: sns = f1 3 Aug 6 02:13:50 filesrv kernel: ASC=80 ASCQ= 1 Aug 6 02:13:50 filesrv kernel: Raw sense data:0xf1 0x00 0x03 0x00 0x00 0x20 0x00 0x16 0x00 0x00 0xe6 0xc2 0x80 0x01 0x00 0x00 0x00 0x00 0x83 0x00 0x37 0x00 0x00 0x00 0x21 0x00 0x90 0x7f 0x3a 0x00 Aug 6 02:13:50 filesrv kernel: klogd 1.4.1, ---------- state change ---------- Aug 6 02:13:50 filesrv kernel: Inspecting /boot/System.map-2.4.18-64GB-SMP Aug 6 02:13:50 filesrv kernel: Loaded 13537 symbols from /boot/System.map-2.4.18-64GB-SMP. Aug 6 02:13:50 filesrv kernel: Symbols match kernel version 2.4.18. Aug 6 02:13:50 filesrv kernel: Loaded 481 symbols from 13 modules. Aug 6 02:13:50 filesrv kernel: st0: Error with sense data: Info fld=0x1, Current st09:00: sns = f0 3 Aug 6 02:13:50 filesrv kernel: ASC= c ASCQ= 0 Aug 6 02:13:50 filesrv kernel: Raw sense data:0xf0 0x00 0x03 0x00 0x00 0x00 0x01 0x16 0x00 0x00 0xe6 0xc2 0x0c 0x00 0x00 0x00 0x00 0x00 0x83 0x00 0x37 0x00 0x00 0x00 0x21 0x00 0x90 0x7f 0x3a 0x00
Have you checked what this sense data indicates ?
Do you have the SCSI manual for your DLT ?
Sense Key 0x03 and Additional Sense Key=0xe6
Sense key= 0x03 means MEDIUM_ERROR
ASK=0xe6 is Vendor Specific but could tell you more about the medium error
Quoted 2 lines
I had this problem with LTO drives too and here the arkeia faq tells one to set fixed block size.
This sounds very strange........
Quoted 9 lines
And when you look a the HP document referenced in my last mail:
ISSUE: Block sizes of LESS THAN 64 KB for DLT1/DLT VS80 and 32 KB for
all other DAT and DLT drives can drastically increase the backup/restore time and severely affect the performance of the drive.SOLUTION: Most backup applications allow viewing and adjusting the block
size used for a particular device. See below for advice on how to achieve this for CA ARCserve, Veritas Backup Exec and Tapeware.
And they are certainly right on that, I recently made a study on blocksizes in Linux-tapes for a customer and the performance was significantly better up to 16 k blocks, after this the perf. increase was in the regions of 2-3 %.
Quoted 8 lines
(I capitalized "less than" to emphasis its occurance)
I you missed the link here it is again:
http://www.hp.com/cposupport/information_storage/support_doc/lpg50167.htmlDid you check with mt after Legato Networker did a backup which blocksize it set?
It will not be set to anything else than 0. Using 0 just tells the driver that it shall write all the data given in ONE write() call to the driver in one block to the device. Thus the application (in this case Legato) can determine which size of blocks to use.
When using fixed blocks, the driver will split the write into that big chunks and write those onto the tape one by one.
Quoted 6 lines
Signature · 3 lines
-- MfG / Regards Friedrich Lobenstock
Quoted 49 lines · Friedrich Lobenstock
Quoted 10 lines · Dag Nygren
I have successfully been using Legato Networker with a blocksize of 32k with my DLT here.
But the way Legato wants to do it is to decide about the blocksize itself.
This means that the driver should NOT decide on the BS, but pass on anything written through it, meaning a blocksize setting of 0 (or variable blocksize).Perhaps Arkeia works the same way ?
I used variable blocksize (=0) before but then after about 3.5 gigs stored on the tape I get scsi-error and arkeia interprets this as end-of-tape.
Aug 6 02:13:50 filesrv kernel: st0: Error with sense data: Info fld=0x2000, Deferred st09:00: sns = f1 3 Aug 6 02:13:50 filesrv kernel: ASC=80 ASCQ= 1 Aug 6 02:13:50 filesrv kernel: Raw sense data:0xf1 0x00 0x03 0x00 0x00 0x20 0x00 0x16 0x00 0x00 0xe6 0xc2 0x80 0x01 0x00 0x00 0x00 0x00 0x83 0x00 0x37 0x00 0x00 0x00 0x21 0x00 0x90 0x7f 0x3a 0x00 Aug 6 02:13:50 filesrv kernel: klogd 1.4.1, ---------- state change ---------- Aug 6 02:13:50 filesrv kernel: Inspecting /boot/System.map-2.4.18-64GB-SMP Aug 6 02:13:50 filesrv kernel: Loaded 13537 symbols from /boot/System.map-2.4.18-64GB-SMP. Aug 6 02:13:50 filesrv kernel: Symbols match kernel version 2.4.18. Aug 6 02:13:50 filesrv kernel: Loaded 481 symbols from 13 modules. Aug 6 02:13:50 filesrv kernel: st0: Error with sense data: Info fld=0x1, Current st09:00: sns = f0 3 Aug 6 02:13:50 filesrv kernel: ASC= c ASCQ= 0 Aug 6 02:13:50 filesrv kernel: Raw sense data:0xf0 0x00 0x03 0x00 0x00
0x000x010x160x000x000xe60xc20x0c0x000x000x000x000x000x830x000x370x000x000x000x210x000x900x7f0x3a0x00I had this problem with LTO drives too and here the arkeia faq tells one to set fixed block size.
And when you look a the HP document referenced in my last mail:
ISSUE: Block sizes of LESS THAN 64 KB for DLT1/DLT VS80 and 32 KB for
all other DAT and DLT drives can drastically increase the backup/restore
time and severely affect the performance of the drive.SOLUTION: Most backup applications allow viewing and adjusting the block
size used for a particular device. See below for advice on how to achieve this for CA ARCserve, Veritas Backup Exec and Tapeware.
Quoted 7 lines
(I capitalized "less than" to emphasis its occurance)
I you missed the link here it is again:
http://www.hp.com/cposupport/information_storage/support_doc/lpg50167.htmlDid you check with mt after Legato Networker did a backup which blocksize it set?
Generally one sets the device block size (driver) to variable and instructs the backup software to use a largish block size. The two are somewhat independent.
HP is right about small block sizes at the app level. Writing small blocks not only ruins performance but the latencies mean the tape drive has to do lots of repositioning which stresses the tape and the drive.
I don't do tape drive firmware but the following is my best understanding of the situation. What you tell the application regarding block size determines the amount of overhead for headers and the size of the buffer used for write(2). Using a large block size at the application level usually helps performance thereby enabling it to better keep up with the tape drive.
On the device level each block is terminated by a end-of-block marker and there is a space (blank tape) between it and the next block to allow for positioning. If you set fixed size blocks it may have to reposition (stop, rewind, play looking for header) between blocks. Small blocks mean wasted tape (reduced capacity) and wear-and-tear on the drive and media. As you can imagine repositioning means reduced throughput. Having a fixed device block size that differs from the application block size can cause problems especially if their relationship is subject to hysteresis.
Using variable block size allows the drive to stream as long as you can keep up with it. If you fall behind (buffer underrun) it will put an end of block and space. This is independent of the application block size.
Signature · 5 lines
________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: [email protected] Remember Cernan and Schmitt