SCSI Userspace Target
Subsystems (3)
cover SCSI Userspace Target
The following patches begin to add the infrastructure for the SCSI Userspace target framework. We were not sure about what James meant by some of his comments, so rather than getting to far we want to post a early.
As the email subject hints, the goal of the project is to push most of the target code to userspace. The only parts we kept in the kernel were for transferring data between the LLD and userpsace and the netlink interface. We included the relevant parts of a software iscsi target as an example.
We also just based ourselves off the scsi_host. This has the benefit of being able to reuse the scsi-ml host code and for HBAs that have a initiator and target mode, they only need to allocate a single scsi_host that corresponds to the PCI device, iscsi session, or whatever your LLD allocates it for. In theory, a single scsi_host can be used by the LLD for running both the target and initiator at the same time. The drawback is it prevents us from doing a more generic target framework that works for any block device (well, someone could do a libata type of target for any transport).
Just to be clear, so people do not pee their pants :) These patches are not ready for prime time. We feel we are at a good point, where we think we can continue building and allow others to help in the development if this basic infrastructure is ok.
There is no error handling yet (since we push so much to userspace it will not be much kernel code), the userspce code is still being reworked to tweak performance (the kernel code is too), we still need build proper scatterlists for HW LLDs (although we can do this with the scsi stuff that was just merged it is not very efficient (see commented code in scsi_tgt_lib.c)). We just want to make sure we are heading in the right direction. For example, should we be using the scsi_host or should we add our own data structure, is the perfomance drop Tomo reported earlier (he has new numbers and has more tweaks up his sleeve) really that acceptable or should we go to scst.
We are also up for discussing making the interface more generic, so we can one day do the scsi scanning and async event handling through it.
Thanks
Mike and Tomo
Quoted 10 lines · Mike Christie
The following patches begin to add the infrastructure for the SCSI Userspace target framework. We were not sure about what James meant by some of his comments, so rather than getting to far we want to post a early.
As the email subject hints, the goal of the project is to push most of the target code to userspace. The only parts we kept in the kernel were for transferring data between the LLD and userpsace and the netlink interface. We included the relevant parts of a software iscsi target as an example.
I'm glad this got posted, I've been wanting something like this for a while. My own personal interest is implementing a tSCSI (my answer to iSCSI) target.
One random comment: Take a look at the mmap'd ring buffer interface provided by mmap'ing the packet socket. That's the ideal async interface, since that's fully async, very close to hardware reality.
Jeff
Quoted 14 lines · Jeff Garzik
Quoted 10 lines · Mike Christie
The following patches begin to add the infrastructure for the SCSI Userspace target framework. We were not sure about what James meant by some of his comments, so rather than getting to far we want to post a early.
As the email subject hints, the goal of the project is to push most of the target code to userspace. The only parts we kept in the kernel were for transferring data between the LLD and userpsace and the netlink interface. We included the relevant parts of a software iscsi target as an example.
Quoted 8 lines
I'm glad this got posted, I've been wanting something like this for a while. My own personal interest is implementing a tSCSI (my answer to iSCSI) target.
One random comment: Take a look at the mmap'd ring buffer interface provided by mmap'ing the packet socket. That's the ideal async interface, since that's fully async, very close to hardware reality.
Thanks will do. I had actually replaced our mapping code with calls to block layer functions like bio_map_user() and blk_rq_map_sg(). I am still looking into the packet socket interface, but if we have to change interaces I would like to somehow hook whatever is decided on into bio.c and ll_rw_blk.c so that it could be used by anyone. SG_IO through a packet socket for example. Is bsg still being worked on? Is it going to use a new interface or is it ioctl based like block/scsi_ioctl.c?
Quoted 21 lines · Mike Christie
Quoted 19 lines · Jeff Garzik
Quoted 10 lines · Mike Christie
The following patches begin to add the infrastructure for the SCSI Userspace target framework. We were not sure about what James meant by some of his comments, so rather than getting to far we want to post a early.
As the email subject hints, the goal of the project is to push most of the target code to userspace. The only parts we kept in the kernel were for transferring data between the LLD and userpsace and the netlink interface. We included the relevant parts of a software iscsi target as an example.
I'm glad this got posted, I've been wanting something like this for a while. My own personal interest is implementing a tSCSI (my answer to iSCSI) target.
Quoted 6 lines
Quoted 4 lines
One random comment: Take a look at the mmap'd ring buffer interface provided by mmap'ing the packet socket. That's the ideal async interface, since that's fully async, very close to hardware reality.
Thanks will do. I had actually replaced our mapping code with calls to block layer functions like
bio_map_user()andblk_rq_map_sg(). I am still looking into the packet socket interface, but if we have to change interaces I would like to somehow hook whatever is decided on intobio.candll_rw_blk.cso that it could be used by anyone.SG_IOthrough a packet socket for example. Is bsg still being worked on? Is it going to use a new interface or is it ioctl based likeblock/scsi_ioctl.c?
bsg is still talked about as the next iteration, but its largely not getting merged simply because there are no users yet. Were a project to pick it up, it would go in quickly.
Jeff
Quoted 2 lines · Mike Christie
packet socket for example. Is bsg still being worked on? Is it going to use a new interface or is it ioctl based like
block/scsi_ioctl.c?
bsg supports both SG_IO and read/write like the SCSI sg does. It should be pretty complete, it's been a while since I worked on it. I thought I had it in -git somewhere, I'll get it added.
Signature · 1 line
Jens Axboe
From: Jeff Garzik <[email protected]>
Subject: Re: [PATCH RFC 0/3] SCSI Userspace Target
Date: Wed, 25 Jan 2006 12:46:49 -0500
Quoted 5 lines
Quoted 5 lines
As the email subject hints, the goal of the project is to push most of the target code to userspace. The only parts we kept in the kernel were for transferring data between the LLD and userpsace and the netlink interface. We included the relevant parts of a software iscsi target as an example.
(snip)
Quoted 3 lines
One random comment: Take a look at the mmap'd ring buffer interface provided by mmap'ing the packet socket. That's the ideal async interface, since that's fully async, very close to hardware reality.
Thanks a lot. Yes, that's really nice. We've replaced some of our netlink code with the packet socket interface.
We just access the ring buffer directly with the following changes to the packet socket code. Is it OK?
One more question. Is there any interface like mmap'ing the packet socket that can be used for user -> kernel communication?
include/net/af_packet.h new file +6 -0
@@ -0,0 +1,6 @@ +#ifndef __LINUX_NET_AFPACKET_H +#define __LINUX_NET_AFPACKET_H + +extern void *packet_frame (struct sock *sk); + +#endif
net/packet/af_packet.c +18 -0
@@ -562,6 +562,24 @@ drop: }#ifdef CONFIG_PACKET_MMAP +void *packet_frame (struct sock *sk) +{ +struct packet_sock *po; +struct tpacket_hdr *h; +int err; + + po =pkt_sk (sk); +spin_lock (&sk->sk_receive_queue.lock); + h = (struct tpacket_hdr *)packet_lookup_frame (po, po->head); +if (h->tp_status) + h =ERR_PTR (-ENOBUFS ); +else + po->head = po->head != po->frame_max ? po->head+1 :0 ; +spin_unlock (&sk->sk_receive_queue.lock); +return h; +} +EXPORT_SYMBOL_GPL (packet_frame); +static int tpacket_rcv (struct sk_buff *skb,struct net_device *dev,struct packet_type *pt,struct net_device *orig_dev) {struct sock *sk;
Quoted 5 lines · Mike Christie
The following patches begin to add the infrastructure for the SCSI Userspace target framework. We were not sure about what James meant by some of his comments, so rather than getting to far we want to post a for any block device (well, someone could do a libata type of target for any transport).
Nit question...I notice those patches are from git. Do you have a git tree for stgt? I was about to start figuring out subversion based on the berlios page.
Signature · 1 line
Dave Boutcher
Quoted 9 lines · Dave C Boutcher
Quoted 5 lines · Mike Christie
The following patches begin to add the infrastructure for the SCSI Userspace target framework. We were not sure about what James meant by some of his comments, so rather than getting to far we want to post a for any block device (well, someone could do a libata type of target for any transport).
Quoted 4 lines
Nit question...I notice those patches are from git. Do you have a git tree for stgt? I was about to start figuring out subversion based on the berlios page.
No git tree. I have not uploaded it to svn yet. Tomo and I are still converting userspace. We had a tiny mishap with different versions getting merged and me basing my stuff on the wrong code. But for svn I think I might just upload the kernel parts as patches becuase I am not sure how to maintain the all code with svn when it is over different subsytems.
Quoted 12 lines · Mike Christie
Quoted 10 lines · Dave C Boutcher
Quoted 5 lines · Mike Christie
The following patches begin to add the infrastructure for the SCSI Userspace target framework. We were not sure about what James meant by some of his comments, so rather than getting to far we want to post a for any block device (well, someone could do a libata type of target for any transport).
Quoted 6 lines
Quoted 4 lines
Nit question...I notice those patches are from git. Do you have a git tree for stgt? I was about to start figuring out subversion based on the berlios page.
No git tree. I have not uploaded it to svn yet. Tomo and I are still converting userspace. We had a tiny mishap with different versions getting merged and me basing my stuff on the wrong code. But for svn I
basing my userspace stuff on the wrong code I mean