Abort Task ?
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 35 lines · Ming Zhang
Quoted 34 lines · Ross S. W. Walker
Quoted 21 lines · BERTRAND Joël
Quoted 4 lines · BERTRAND Joël
Quoted 3 lines · BERTRAND Joël
Hello,
When I try to create a raid1 volume over iscsi, process
aborts with :
Quoted 4 lines
Quoted 4 lines
- on target side:
iscsi_trgt:cmnd_abort(1156)29 1 0 42 57344 0 0iscsi_trgt: Abort Task (01) issued on tid:1 lun:0 bysid:630024457682948
Quoted 4 lines
Quoted 1 line
(Unknown Task)
Next run:
iscsi_trgt:cmnd_abort(1156)13 1 0 42 57344 0 0iscsi_trgt: Abort Task (01) issued on tid:1 lun:0 bysid:630058817421315
Quoted 3 lines
(Unknown Task)
You can see that both lines are very similar. I shall
try to use
Quoted 1 line
blockio instead fileio.
Quoted 8 lines
With blockio, I got the following message...
iscsi_trgt:cmnd_abort(1156)c 1 0 42 8192 0 0iscsi_trgt: Abort Task (01) issued on tid:1 lun:0 by sid:630024457682946
(Unknown Task)Command is the same. What is the signification of 1156 ?
Both outputs are from the same Abort Task management function the 1156 refers to the line in
iscsi.cwhere the debug printf was issued.
Quoted 7 lines
Quoted 2 lines
The other is the more verbose informative message that says an Abort Task command was issued, but the task was not found.
pure guess, this might because the sparc64 you are using.
could you export a NULLIO target and do some intensive io tests? sort out these platform issues first...
I can format serveral times (mkfs.ext3) a 1.5 TB volume over iSCSI
without any trouble. I can read and write on this virtual disk without any trouble.
Now, I have configured ietd with :
Lun 0 Sectors=1464725758,Type=nullio
and I run on initiator side :
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192 479482+0 records in
479482+0 records out
3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/s
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
I'm waitinfor a crash. No one when I write these lines. I suspect an
interaction between raid and iscsi.
Regards,
JKB
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Iscsitarget-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/iscsitarget-devel
Quoted 15 lines · BERTRAND Joël
I can format serveral times (mkfs.ext3) a 1.5 TB volume over iSCSI
without any trouble. I can read and write on this virtual disk without any trouble.Now, I have configured ietd with :
Lun 0 Sectors=1464725758,Type=nullio
and I run on initiator side :
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192 479482+0 records in
479482+0 records out
3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/s
Quoted 4 lines
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
I'm waitinfor a crash. No one when I write these lines. I suspect
an interaction between raid and iscsi.
I simultanely run :
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192 8397210+0 records in
8397210+0 records out
68789944320 bytes (69 GB) copied, 2732.55 seconds, 25.2 MB/s
and
Root gershwin:[~] > dd if=/dev/sdj of=/dev/null bs=8192 739200+0 records in
739199+0 records out
6055518208 bytes (6.1 GB) copied, 447.178 seconds, 13.5 MB/s
without any trouble.
Regards,
JKB
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Iscsitarget-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/iscsitarget-devel
Quoted 32 lines · BERTRAND Joël
Quoted 1 line · BERTRAND Joël
I can format serveral times (mkfs.ext3) a 1.5 TB volume
over iSCSI
Quoted 1 line
without any trouble. I can read and write on this virtual
disk without
Quoted 8 lines
any trouble.
Now, I have configured ietd with :
Lun 0 Sectors=1464725758,Type=nullio
and I run on initiator side :
Quoted 8 lines
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192 479482+0 records in
479482+0 records out
3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/sRoot gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
I'm waitinfor a crash. No one when I write these lines.
I suspect
Quoted 1 line
an interaction between raid and iscsi.
I simultanely run :
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192 8397210+0 records in
8397210+0 records out
68789944320 bytes (69 GB) copied, 2732.55 seconds, 25.2 MB/s
Quoted 8 lines
and
Root gershwin:[~] > dd if=/dev/sdj of=/dev/null bs=8192 739200+0 records in
739199+0 records out
6055518208 bytes (6.1 GB) copied, 447.178 seconds, 13.5 MB/swithout any trouble.
The speed can definitely be improved. Look at your network setup and use ping to try and get the network latency to a minimum.
# ping -A -s 8192 172.16.24.140 .... --- 172.16.24.140 ping statistics --- 14058 packets transmitted, 14057 received, 0% packet loss, time 9988ms rtt min/avg/max/mdev = 0.234/0.268/2.084/0.041 ms, ipg/ewma 0.710/0.260 ms
You want your avg ping time for 8192 byte payloads to be 300us or less.
1000/.268 = 3731 IOPS @ 8k = 30 MB/s
If you use apps that do overlapping asynchronous IO you can see better numbers.
-Ross
______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof.
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
Quoted 51 lines · Ross S. W. Walker
Quoted 32 lines · BERTRAND Joël
Quoted 1 line · BERTRAND Joël
I can format serveral times (mkfs.ext3) a 1.5 TB volume
over iSCSI
Quoted 1 line
without any trouble. I can read and write on this virtual
disk without
Quoted 8 lines
any trouble.
Now, I have configured ietd with :
Lun 0 Sectors=1464725758,Type=nullio
and I run on initiator side :
Quoted 8 lines
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192 479482+0 records in
479482+0 records out
3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/sRoot gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
I'm waitinfor a crash. No one when I write these lines.
I suspect
Quoted 1 line
an interaction between raid and iscsi.
I simultanely run :
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192 8397210+0 records in
8397210+0 records out
68789944320 bytes (69 GB) copied, 2732.55 seconds, 25.2 MB/sQuoted 8 lines
and
Root gershwin:[~] > dd if=/dev/sdj of=/dev/null bs=8192 739200+0 records in
739199+0 records out
6055518208 bytes (6.1 GB) copied, 447.178 seconds, 13.5 MB/swithout any trouble.
The speed can definitely be improved. Look at your network setup and use ping to try and get the network latency to a minimum.
# ping -A -s 8192 172.16.24.140 .... --- 172.16.24.140 ping statistics --- 14058 packets transmitted, 14057 received, 0% packet loss, time 9988ms rtt min/avg/max/mdev = 0.234/0.268/2.084/0.041 ms, ipg/ewma 0.710/0.260 ms
Quoted 3 lines
You want your avg ping time for 8192 byte payloads to be 300us or less.
1000/.268 = 3731 IOPS @ 8k = 30 MB/s
he use normal dd, which in turn use page cache and read ahead, so actual request size at scsi layer can be much larger than 8k....
Quoted 16 lines
If you use apps that do overlapping asynchronous IO you can see better numbers.
-Ross
______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof.
Signature · 12 lines
Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881 -------------------------------------------- - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 40 lines · Ross S. W. Walker
Quoted 30 lines · BERTRAND Joël
Quoted 1 line · BERTRAND Joël
I can format serveral times (mkfs.ext3) a 1.5 TB volume
over iSCSI
Quoted 1 line
without any trouble. I can read and write on this virtual
disk without
Quoted 8 lines
any trouble.
Now, I have configured ietd with :
Lun 0 Sectors=1464725758,Type=nullio
and I run on initiator side :
Quoted 8 lines
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192 479482+0 records in
479482+0 records out
3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/sRoot gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
I'm waitinfor a crash. No one when I write these lines.
I suspect
Quoted 1 line
an interaction between raid and iscsi.
I simultanely run :
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192 8397210+0 records in
8397210+0 records out
68789944320 bytes (69 GB) copied, 2732.55 seconds, 25.2 MB/sQuoted 8 lines
and
Root gershwin:[~] > dd if=/dev/sdj of=/dev/null bs=8192 739200+0 records in
739199+0 records out
6055518208 bytes (6.1 GB) copied, 447.178 seconds, 13.5 MB/swithout any trouble.
Quoted 8 lines
The speed can definitely be improved. Look at your network setup and use ping to try and get the network latency to a minimum.
# ping -A -s 8192 172.16.24.140 .... --- 172.16.24.140 ping statistics --- 14058 packets transmitted, 14057 received, 0% packet loss, time 9988ms rtt min/avg/max/mdev = 0.234/0.268/2.084/0.041 ms, ipg/ewma 0.710/0.260 ms
gershwin:[~] > ping -A -s 8192 192.168.0.2
PING 192.168.0.2 (192.168.0.2) 8192(8220) bytes of data. 8200 bytes from 192.168.0.2: icmp_seq=1 ttl=64 time=0.693 ms 8200 bytes from 192.168.0.2: icmp_seq=2 ttl=64 time=0.595 ms 8200 bytes from 192.168.0.2: icmp_seq=3 ttl=64 time=0.583 ms 8200 bytes from 192.168.0.2: icmp_seq=4 ttl=64 time=0.589 ms 8200 bytes from 192.168.0.2: icmp_seq=5 ttl=64 time=0.580 ms 8200 bytes from 192.168.0.2: icmp_seq=6 ttl=64 time=0.594 ms 8200 bytes from 192.168.0.2: icmp_seq=7 ttl=64 time=0.580 ms 8200 bytes from 192.168.0.2: icmp_seq=8 ttl=64 time=0.592 ms 8200 bytes from 192.168.0.2: icmp_seq=9 ttl=64 time=0.589 ms 8200 bytes from 192.168.0.2: icmp_seq=10 ttl=64 time=0.571 ms 8200 bytes from 192.168.0.2: icmp_seq=11 ttl=64 time=0.588 ms 8200 bytes from 192.168.0.2: icmp_seq=12 ttl=64 time=0.580 ms 8200 bytes from 192.168.0.2: icmp_seq=13 ttl=64 time=0.587 ms
--- 192.168.0.2 ping statistics ---
13 packets transmitted, 13 received, 0% packet loss, time 2400ms rtt min/avg/max/mdev = 0.571/0.593/0.693/0.044 ms, ipg/ewma 200.022/0.607 ms gershwin:[~] >
Both initiator and target are alone on a gigabit NIC (Tigon3). On
target server, istd1 takes 100% of a CPU (and only one CPU, even my T1000 can simultaneous run 32 threads). I think the limitation comes from istd1.
Quoted 6 lines
You want your avg ping time for 8192 byte payloads to be 300us or less.
1000/.268 = 3731 IOPS @ 8k = 30 MB/s
If you use apps that do overlapping asynchronous IO you can see better numbers.
Regards,
JKB
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
Quoted 71 lines · BERTRAND Joël
Quoted 40 lines · Ross S. W. Walker
Quoted 30 lines · BERTRAND Joël
Quoted 1 line · BERTRAND Joël
I can format serveral times (mkfs.ext3) a 1.5 TB volume
over iSCSI
Quoted 1 line
without any trouble. I can read and write on this virtual
disk without
Quoted 8 lines
any trouble.
Now, I have configured ietd with :
Lun 0 Sectors=1464725758,Type=nullio
and I run on initiator side :
Quoted 8 lines
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192 479482+0 records in
479482+0 records out
3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/sRoot gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
I'm waitinfor a crash. No one when I write these lines.
I suspect
Quoted 1 line
an interaction between raid and iscsi.
I simultanely run :
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192 8397210+0 records in
8397210+0 records out
68789944320 bytes (69 GB) copied, 2732.55 seconds, 25.2 MB/sQuoted 8 lines
and
Root gershwin:[~] > dd if=/dev/sdj of=/dev/null bs=8192 739200+0 records in
739199+0 records out
6055518208 bytes (6.1 GB) copied, 447.178 seconds, 13.5 MB/swithout any trouble.
Quoted 8 lines
The speed can definitely be improved. Look at your network setup and use ping to try and get the network latency to a minimum.
# ping -A -s 8192 172.16.24.140 .... --- 172.16.24.140 ping statistics --- 14058 packets transmitted, 14057 received, 0% packet loss, time 9988ms rtt min/avg/max/mdev = 0.234/0.268/2.084/0.041 ms, ipg/ewma 0.710/0.260 msgershwin:[~] > ping -A -s 8192 192.168.0.2
PING 192.168.0.2 (192.168.0.2) 8192(8220) bytes of data. 8200 bytes from 192.168.0.2: icmp_seq=1 ttl=64 time=0.693 ms 8200 bytes from 192.168.0.2: icmp_seq=2 ttl=64 time=0.595 ms 8200 bytes from 192.168.0.2: icmp_seq=3 ttl=64 time=0.583 ms 8200 bytes from 192.168.0.2: icmp_seq=4 ttl=64 time=0.589 ms 8200 bytes from 192.168.0.2: icmp_seq=5 ttl=64 time=0.580 ms 8200 bytes from 192.168.0.2: icmp_seq=6 ttl=64 time=0.594 ms 8200 bytes from 192.168.0.2: icmp_seq=7 ttl=64 time=0.580 ms 8200 bytes from 192.168.0.2: icmp_seq=8 ttl=64 time=0.592 ms 8200 bytes from 192.168.0.2: icmp_seq=9 ttl=64 time=0.589 ms 8200 bytes from 192.168.0.2: icmp_seq=10 ttl=64 time=0.571 ms 8200 bytes from 192.168.0.2: icmp_seq=11 ttl=64 time=0.588 ms 8200 bytes from 192.168.0.2: icmp_seq=12 ttl=64 time=0.580 ms 8200 bytes from 192.168.0.2: icmp_seq=13 ttl=64 time=0.587 ms--- 192.168.0.2 ping statistics ---
13 packets transmitted, 13 received, 0% packet loss, time 2400ms rtt min/avg/max/mdev = 0.571/0.593/0.693/0.044 ms, ipg/ewma 200.022/0.607 ms gershwin:[~] >
Quoted 4 lines
Both initiator and target are alone on a gigabit NIC (Tigon3). On
target server, istd1 takes 100% of a CPU (and only one CPU, even my T1000 can simultaneous run 32 threads). I think the limitation comes from istd1.
usually istdx will not take 100% cpu with 1G network, especially when using disk as back storage, some kind of profiling work might be helpful to tell what happened...
forgot to ask, your sparc64 platform cpu spec.
Quoted 11 lines
Quoted 7 lines
You want your avg ping time for 8192 byte payloads to be 300us or less.
1000/.268 = 3731 IOPS @ 8k = 30 MB/s
If you use apps that do overlapping asynchronous IO you can see better numbers.
Regards,
JKB
Signature · 18 lines
Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881 -------------------------------------------- ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Iscsitarget-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/iscsitarget-devel
Quoted 75 lines · Ming Zhang
Quoted 69 lines · BERTRAND Joël
Quoted 39 lines · Ross S. W. Walker
Quoted 30 lines · BERTRAND Joël
Quoted 1 line · BERTRAND Joël
I can format serveral times (mkfs.ext3) a 1.5 TB volume
over iSCSI
Quoted 1 line
without any trouble. I can read and write on this virtual
disk without
Quoted 8 lines
any trouble.
Now, I have configured ietd with :
Lun 0 Sectors=1464725758,Type=nullio
and I run on initiator side :
Quoted 8 lines
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192 479482+0 records in
479482+0 records out
3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/sRoot gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
I'm waitinfor a crash. No one when I write these lines.
I suspect
Quoted 1 line
an interaction between raid and iscsi.
I simultanely run :
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192 8397210+0 records in
8397210+0 records out
68789944320 bytes (69 GB) copied, 2732.55 seconds, 25.2 MB/sQuoted 8 lines
and
Root gershwin:[~] > dd if=/dev/sdj of=/dev/null bs=8192 739200+0 records in
739199+0 records out
6055518208 bytes (6.1 GB) copied, 447.178 seconds, 13.5 MB/swithout any trouble.
Quoted 8 lines
The speed can definitely be improved. Look at your network setup and use ping to try and get the network latency to a minimum.
# ping -A -s 8192 172.16.24.140 .... --- 172.16.24.140 ping statistics --- 14058 packets transmitted, 14057 received, 0% packet loss, time 9988ms rtt min/avg/max/mdev = 0.234/0.268/2.084/0.041 ms, ipg/ewma 0.710/0.260 msgershwin:[~] > ping -A -s 8192 192.168.0.2
PING 192.168.0.2 (192.168.0.2) 8192(8220) bytes of data. 8200 bytes from 192.168.0.2: icmp_seq=1 ttl=64 time=0.693 ms 8200 bytes from 192.168.0.2: icmp_seq=2 ttl=64 time=0.595 ms 8200 bytes from 192.168.0.2: icmp_seq=3 ttl=64 time=0.583 ms 8200 bytes from 192.168.0.2: icmp_seq=4 ttl=64 time=0.589 ms 8200 bytes from 192.168.0.2: icmp_seq=5 ttl=64 time=0.580 ms 8200 bytes from 192.168.0.2: icmp_seq=6 ttl=64 time=0.594 ms 8200 bytes from 192.168.0.2: icmp_seq=7 ttl=64 time=0.580 ms 8200 bytes from 192.168.0.2: icmp_seq=8 ttl=64 time=0.592 ms 8200 bytes from 192.168.0.2: icmp_seq=9 ttl=64 time=0.589 ms 8200 bytes from 192.168.0.2: icmp_seq=10 ttl=64 time=0.571 ms 8200 bytes from 192.168.0.2: icmp_seq=11 ttl=64 time=0.588 ms 8200 bytes from 192.168.0.2: icmp_seq=12 ttl=64 time=0.580 ms 8200 bytes from 192.168.0.2: icmp_seq=13 ttl=64 time=0.587 ms--- 192.168.0.2 ping statistics ---
13 packets transmitted, 13 received, 0% packet loss, time 2400ms rtt min/avg/max/mdev = 0.571/0.593/0.693/0.044 ms, ipg/ewma 200.022/0.607 ms gershwin:[~] >Quoted 4 lines
Both initiator and target are alone on a gigabit NIC (Tigon3). On
target server, istd1 takes 100% of a CPU (and only one CPU, even my T1000 can simultaneous run 32 threads). I think the limitation comes from istd1.
Quoted 5 lines
usually istdx will not take 100% cpu with 1G network, especially when using disk as back storage, some kind of profiling work might be helpful to tell what happened...
forgot to ask, your sparc64 platform cpu spec.
Root gershwin:[/mnt/solaris] > cat /proc/cpuinfo
cpu : UltraSparc T1 (Niagara)
fpu : UltraSparc T1 integrated FPU
prom : OBP 4.23.4 2006/08/04 20:45
type : sun4v
ncpus probed : 24
ncpus active : 24
D$ parity tl1 : 0
I$ parity tl1 : 0
Both servers are built with 1 GHz T1 processors (6 cores, 24 threads).
Regards,
JKB
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Iscsitarget-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/iscsitarget-devel
Quoted 92 lines · BERTRAND Joël
Quoted 75 lines · Ming Zhang
Quoted 69 lines · BERTRAND Joël
Quoted 39 lines · Ross S. W. Walker
Quoted 30 lines · BERTRAND Joël
Quoted 1 line · BERTRAND Joël
I can format serveral times (mkfs.ext3) a 1.5 TB volume
over iSCSI
Quoted 1 line
without any trouble. I can read and write on this virtual
disk without
Quoted 8 lines
any trouble.
Now, I have configured ietd with :
Lun 0 Sectors=1464725758,Type=nullio
and I run on initiator side :
Quoted 8 lines
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192 479482+0 records in
479482+0 records out
3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/sRoot gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
I'm waitinfor a crash. No one when I write these lines.
I suspect
Quoted 1 line
an interaction between raid and iscsi.
I simultanely run :
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192 8397210+0 records in
8397210+0 records out
68789944320 bytes (69 GB) copied, 2732.55 seconds, 25.2 MB/sQuoted 8 lines
and
Root gershwin:[~] > dd if=/dev/sdj of=/dev/null bs=8192 739200+0 records in
739199+0 records out
6055518208 bytes (6.1 GB) copied, 447.178 seconds, 13.5 MB/swithout any trouble.
Quoted 8 lines
The speed can definitely be improved. Look at your network setup and use ping to try and get the network latency to a minimum.
# ping -A -s 8192 172.16.24.140 .... --- 172.16.24.140 ping statistics --- 14058 packets transmitted, 14057 received, 0% packet loss, time 9988ms rtt min/avg/max/mdev = 0.234/0.268/2.084/0.041 ms, ipg/ewma 0.710/0.260 msgershwin:[~] > ping -A -s 8192 192.168.0.2
PING 192.168.0.2 (192.168.0.2) 8192(8220) bytes of data. 8200 bytes from 192.168.0.2: icmp_seq=1 ttl=64 time=0.693 ms 8200 bytes from 192.168.0.2: icmp_seq=2 ttl=64 time=0.595 ms 8200 bytes from 192.168.0.2: icmp_seq=3 ttl=64 time=0.583 ms 8200 bytes from 192.168.0.2: icmp_seq=4 ttl=64 time=0.589 ms 8200 bytes from 192.168.0.2: icmp_seq=5 ttl=64 time=0.580 ms 8200 bytes from 192.168.0.2: icmp_seq=6 ttl=64 time=0.594 ms 8200 bytes from 192.168.0.2: icmp_seq=7 ttl=64 time=0.580 ms 8200 bytes from 192.168.0.2: icmp_seq=8 ttl=64 time=0.592 ms 8200 bytes from 192.168.0.2: icmp_seq=9 ttl=64 time=0.589 ms 8200 bytes from 192.168.0.2: icmp_seq=10 ttl=64 time=0.571 ms 8200 bytes from 192.168.0.2: icmp_seq=11 ttl=64 time=0.588 ms 8200 bytes from 192.168.0.2: icmp_seq=12 ttl=64 time=0.580 ms 8200 bytes from 192.168.0.2: icmp_seq=13 ttl=64 time=0.587 ms--- 192.168.0.2 ping statistics ---
13 packets transmitted, 13 received, 0% packet loss, time 2400ms rtt min/avg/max/mdev = 0.571/0.593/0.693/0.044 ms, ipg/ewma 200.022/0.607 ms gershwin:[~] >Quoted 4 lines
Both initiator and target are alone on a gigabit NIC (Tigon3). On
target server, istd1 takes 100% of a CPU (and only one CPU, even my T1000 can simultaneous run 32 threads). I think the limitation comes from istd1.Quoted 5 lines
usually istdx will not take 100% cpu with 1G network, especially when using disk as back storage, some kind of profiling work might be helpful to tell what happened...
forgot to ask, your sparc64 platform cpu spec.
Root gershwin:[/mnt/solaris] > cat
/proc/cpuinfo
cpu : UltraSparc T1 (Niagara)
fpu : UltraSparc T1 integrated FPU
prom : OBP 4.23.4 2006/08/04 20:45
type : sun4v
ncpus probed : 24
ncpus active : 24
D$ parity tl1 : 0
I$ parity tl1 : 0
Quoted 2 lines
Both servers are built with 1 GHz T1 processors (6 cores, 24 threads).
as Ross pointed out, many io pattern only have 1 outstanding io at any time, so there is only one work thread actively to serve it. so it can not exploit the multiple core here.
you see 100% at nullio or fileio? with disk, most time should spend on iowait and cpu utilization should not high at all.
Quoted 3 lines
Regards,
JKB
Signature · 18 lines
Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881 -------------------------------------------- ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Iscsitarget-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/iscsitarget-devel
Quoted 8 lines · Ming Zhang
as Ross pointed out, many io pattern only have 1 outstanding io at any time, so there is only one work thread actively to serve it. so it can not exploit the multiple core here.
you see 100% at nullio or fileio? with disk, most time should spend on iowait and cpu utilization should not high at all.
With both nullio and fileio...
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
Quoted 11 lines · BERTRAND Joël
Quoted 8 lines · Ming Zhang
as Ross pointed out, many io pattern only have 1 outstanding io at any time, so there is only one work thread actively to serve it. so it can not exploit the multiple core here.
you see 100% at nullio or fileio? with disk, most time should spend on iowait and cpu utilization should not high at all.
With both nullio and fileio...
it is weired. with file io, run some io load, then run "vmstat 1" and post here. supposed to see some iowait instead of high sys cpu usage...
Signature · 18 lines
Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881 -------------------------------------------- ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Iscsitarget-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/iscsitarget-devel
Quoted 106 lines · Ming Zhang
Quoted 41 lines · BERTRAND Joël
Quoted 40 lines · Ming Zhang
Quoted 39 lines · BERTRAND Joël
Quoted 38 lines · Ross S. W. Walker
Quoted 37 lines · BERTRAND Joël
Quoted 1 line · BERTRAND Joël
I can format serveral times (mkfs.ext3) a 1.5 TB volume
over iSCSI
Quoted 1 line
without any trouble. I can read and write on this virtual
disk without
Quoted 8 lines
any trouble.
Now, I have configured ietd with :
Lun 0 Sectors=1464725758,Type=nullio
and I run on initiator side :
Quoted 8 lines
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192 479482+0 records in
479482+0 records out
3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/sRoot gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
I'm waitinfor a crash. No one when I write these lines.
I suspect
Quoted 1 line
an interaction between raid and iscsi.
I simultanely run :
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192 8397210+0 records in
8397210+0 records out
68789944320 bytes (69 GB) copied, 2732.55 seconds, 25.2 MB/sand
Root gershwin:[~] > dd if=/dev/sdj of=/dev/null bs=8192 739200+0 records in
739199+0 records out
6055518208 bytes (6.1 GB) copied, 447.178 seconds, 13.5 MB/sQuoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
without any trouble.
The speed can definitely be improved. Look at your network setup and use ping to try and get the network latency to a minimum.
# ping -A -s 8192 172.16.24.140 .... --- 172.16.24.140 ping statistics --- 14058 packets transmitted, 14057 received, 0% packetloss, time 9988ms
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
rtt min/avg/max/mdev = 0.234/0.268/2.084/0.041 ms,
ipg/ewma 0.710/0.260 ms
Quoted 16 lines
Quoted 16 lines
Quoted 16 lines
gershwin:[~] > ping -A -s 8192 192.168.0.2
PING 192.168.0.2 (192.168.0.2) 8192(8220) bytes of data. 8200 bytes from 192.168.0.2: icmp_seq=1 ttl=64 time=0.693 ms 8200 bytes from 192.168.0.2: icmp_seq=2 ttl=64 time=0.595 ms 8200 bytes from 192.168.0.2: icmp_seq=3 ttl=64 time=0.583 ms 8200 bytes from 192.168.0.2: icmp_seq=4 ttl=64 time=0.589 ms 8200 bytes from 192.168.0.2: icmp_seq=5 ttl=64 time=0.580 ms 8200 bytes from 192.168.0.2: icmp_seq=6 ttl=64 time=0.594 ms 8200 bytes from 192.168.0.2: icmp_seq=7 ttl=64 time=0.580 ms 8200 bytes from 192.168.0.2: icmp_seq=8 ttl=64 time=0.592 ms 8200 bytes from 192.168.0.2: icmp_seq=9 ttl=64 time=0.589 ms 8200 bytes from 192.168.0.2: icmp_seq=10 ttl=64 time=0.571 ms 8200 bytes from 192.168.0.2: icmp_seq=11 ttl=64 time=0.588 ms 8200 bytes from 192.168.0.2: icmp_seq=12 ttl=64 time=0.580 ms 8200 bytes from 192.168.0.2: icmp_seq=13 ttl=64 time=0.587 msQuoted 3 lines
Quoted 3 lines
Quoted 3 lines
--- 192.168.0.2 ping statistics ---
13 packets transmitted, 13 received, 0% packet loss, time 2400ms rtt min/avg/max/mdev = 0.571/0.593/0.693/0.044 ms,ipg/ewma 200.022/0.607 ms
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
gershwin:[~] >
Both initiator and target are alone on a gigabit NIC
(Tigon3). On
Quoted 1 line
Quoted 1 line
Quoted 1 line
target server, istd1 takes 100% of a CPU (and only one
CPU, even my
Quoted 1 line
Quoted 1 line
Quoted 1 line
T1000 can simultaneous run 32 threads). I think the
limitation comes
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
from istd1.
usually istdx will not take 100% cpu with 1G network,
especially when
Quoted 1 line
Quoted 1 line
using disk as back storage, some kind of profiling work
might be helpful
Quoted 14 lines
Quoted 3 lines
to tell what happened...
forgot to ask, your sparc64 platform cpu spec.
Root gershwin:[/mnt/solaris] > cat
/proc/cpuinfo
cpu : UltraSparc T1 (Niagara)
fpu : UltraSparc T1 integrated FPU
prom : OBP 4.23.4 2006/08/04 20:45
type : sun4v
ncpus probed : 24
ncpus active : 24
D$ parity tl1 : 0
I$ parity tl1 : 0Quoted 1 line
Both servers are built with 1 GHz T1 processors (6
cores, 24 threads).
Quoted 1 line
Quoted 7 lines
as Ross pointed out, many io pattern only have 1 outstanding io at any time, so there is only one work thread actively to serve it. so it can not exploit the multiple core here.
you see 100% at nullio or fileio? with disk, most time should spend on iowait and cpu utilization should not high at all.
Maybe it has to do with the endian-ness fix?
Look at where the fix was implemented and if there was a simpler way of implementing it? (If that is the cause)
The network is still slower then expected, I don't know what chipset the Sparcs use for their interfaces, if it is e1000 then you can set low-latency interrupt throttling with InterruptThrottleRate=1 which works well. You can explore other interface module options around Interrupt throttling or coalesence.
-Ross
______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof.
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 50 lines · BERTRAND Joël
Quoted 35 lines · Ming Zhang
Quoted 34 lines · Ross S. W. Walker
Quoted 21 lines · BERTRAND Joël
Quoted 4 lines · BERTRAND Joël
Quoted 3 lines · BERTRAND Joël
Hello,
When I try to create a raid1 volume over iscsi, process
aborts with :
Quoted 4 lines
Quoted 4 lines
- on target side:
iscsi_trgt:cmnd_abort(1156)29 1 0 42 57344 0 0iscsi_trgt: Abort Task (01) issued on tid:1 lun:0 bysid:630024457682948
Quoted 4 lines
Quoted 1 line
(Unknown Task)
Next run:
iscsi_trgt:cmnd_abort(1156)13 1 0 42 57344 0 0iscsi_trgt: Abort Task (01) issued on tid:1 lun:0 bysid:630058817421315
Quoted 3 lines
(Unknown Task)
You can see that both lines are very similar. I shall
try to use
Quoted 1 line
blockio instead fileio.
Quoted 8 lines
With blockio, I got the following message...
iscsi_trgt:cmnd_abort(1156)c 1 0 42 8192 0 0iscsi_trgt: Abort Task (01) issued on tid:1 lun:0 by sid:630024457682946
(Unknown Task)Command is the same. What is the signification of 1156 ?
Both outputs are from the same Abort Task management function the 1156 refers to the line in
iscsi.cwhere the debug printf was issued.Quoted 7 lines
Quoted 2 lines
The other is the more verbose informative message that says an Abort Task command was issued, but the task was not found.
pure guess, this might because the sparc64 you are using.
could you export a NULLIO target and do some intensive io tests? sort out these platform issues first...
I can format serveral times (mkfs.ext3) a 1.5 TB volume over iSCSI
without any trouble. I can read and write on this virtual disk without any trouble.Now, I have configured ietd with :
Quoted 8 lines
Lun 0 Sectors=1464725758,Type=nullio
and I run on initiator side :
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192 479482+0 records in
479482+0 records out
3927916544 bytes (3.9 GB) copied, 153.222 seconds, 25.6 MB/s
25MB/s over a NULLIO is a very slow result...
Quoted 9 lines
Root gershwin:[/dev] > dd if=/dev/zero of=/dev/sdj bs=8192
I'm waitinfor a crash. No one when I write these lines. I suspect an
interaction between raid and iscsi.Regards,
JKB
Signature · 12 lines
Ming Zhang @#$%^ purging memory... (*!% http://blackmagic02881.wordpress.com/ http://www.linkedin.com/in/blackmagic02881 -------------------------------------------- - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html