SCSITAP, Virtual SCSI HBA and user space SCSI initiators
A message that is not in the archive (sent off-list or not yet received): [email protected]
I am sending this again as the initial mail did not make it to the list.
Hi All,
I thought a lot about whether I should write to mailing list about this or not. I have requested help from the list before to complete this. May be I am too crazy to develop something like this. When I think more and more about this, it will be really useful, mainly for educational purposes and it could be useful in other environments.
I like to know everyone's opinion on this. Please let me know your thoughts and suggestions and its inclusion in the main line Linux kernel.
I have developed a Virtual HBA (Around ~600 lines of code). It is a kernel module and works very well with 2.6.9 kernel. Basically it registers a HBA (LLD) to the SCSI Mid layer. It implements a linked list with all the SCSI commands queued from Mid layer. I registered a character device driver. I ship out the SCSI commands off to user space via this character device. There is a user space application monitoring this character device looking for SCSI commands, process it and sends it back to the mid layer through this character interface. At the moment, the interface to the Virtual HBA inside the kernel is through read/write and ioctls to the character device. I am trying to get rid of reading and writing of SCSI request_buffer through character device and use memmap/splice to avoid copying kernel buffer to user space(I am facing some challenges as the request_buffer is not a linear buffer, but a scatter gather buffer).
As I mentioned above, the HBA only transports the commands out side the kernel. But the real job is done by the user space tool. Now there is end less opportunity for this user space tool. It can impliment iSCSI, NDMP, local files, shared files across the network, etc. People can even write iSCSI initiator using Java! The advantage is that the main transport is done by user space application. Currently i have implemented a small scsi engine and it presents a file as a SCSI disk to the kernel. It works great.
The Kernel driver is ready. It will need some modifications (Like implimenting memap to avoid copying, integrating with sysfs/procfs, etc). I am thinking in different directions on what I should include in the driver. May be you guys can direct me in right direction. If you think, there is not much potential for this in the Linux kernel, I will stop working on this now as I already achieved what I wanted (Presenting a file as a SCSI disk).
Please let me know your thoughts and suggestions.
Background:
-------------
I wanted to make a NDMP (A protocol used to manage tapes and perform backup/resotre on NAS heads) initiator for Linux (Just out of my interest).. I was not very good with doing stuff inside the kernel. First my idea was to modify the Cisco iSCSI initiator and make it for NDMP. But i found, it is too much of work + I already had a user space implimentation of NDMP procedures. So I thought of making a virtual HBA and expose the SCSI mid-layer to a user space application.
Thanks
Aboo
Quoted 7 lines · Aboo Valappil
I wanted to make a NDMP (A protocol used to manage tapes and perform backup/resotre on NAS heads) initiator for Linux (Just out of my interest).. I was not very good with doing stuff inside the kernel. First my idea was to modify the Cisco iSCSI initiator and make it for NDMP. But i found, it is too much of work + I already had a user space implimentation of NDMP procedures. So I thought of making a virtual HBA and expose the SCSI mid-layer to a user space application.
We actually already have an existing target infrastructure project here:
http://git.kernel.org/git/?p=linux/kernel/git/tomo/linux-2.6-target.git;a=summary
It seems to be very similar to yours ... could you see how easy it would be for you to integrate with this project?
Thanks,
James
Thanks James,
Is'nt this a target implimentation, rather than a initiator? That page does not give me more details. I have sent an email to the author of the project to get more details. Hopefully, they will reply :)
Aboo
Quoted 11 lines · James Bottomley
Quoted 7 lines · Aboo Valappil
I wanted to make a NDMP (A protocol used to manage tapes and perform backup/resotre on NAS heads) initiator for Linux (Just out of my interest).. I was not very good with doing stuff inside the kernel. First my idea was to modify the Cisco iSCSI initiator and make it for NDMP. But i found, it is too much of work + I already had a user space implimentation of NDMP procedures. So I thought of making a virtual HBA and expose the SCSI mid-layer to a user space application.
We actually already have an existing target infrastructure project here:
Quoted 8 lines
http://git.kernel.org/git/?p=linux/kernel/git/tomo/linux-2.6-target.git;a=summary
It seems to be very similar to yours ... could you see how easy it would be for you to integrate with this project?
Thanks,
James
-------------------------------------
Aboo.Org - Compliments From A & J :)
-------------------------------------
Quoted 3 lines · aboo
Is'nt this a target implimentation, rather than a initiator? That page does not give me more details. I have sent an email to the author of the project to get more details. Hopefully, they will reply :)
It's designed for use by a target driver ... but there's no reason why a virtual initator couldn't also make use of the infrastructure; it's basically just a way to pass commands up and down plus the SG elements.
James
Quoted 13 lines · Aboo Valappil
I am sending this again as the initial mail did not make it to the list.
Hi All,
I thought a lot about whether I should write to mailing list about this or not. I have requested help from the list before to complete this. May be I am too crazy to develop something like this. When I think more and more about this, it will be really useful, mainly for educational purposes and it could be useful in other environments.
I like to know everyone's opinion on this. Please let me know your thoughts and suggestions and its inclusion in the main line Linux kernel.
Quoted 14 lines
I have developed a Virtual HBA (Around ~600 lines of code). It is a kernel module and works very well with 2.6.9 kernel. Basically it registers a HBA (LLD) to the SCSI Mid layer. It implements a linked list with all the SCSI commands queued from Mid layer. I registered a character device driver. I ship out the SCSI commands off to user space via this character device. There is a user space application monitoring this character device looking for SCSI commands, process it and sends it back to the mid layer through this character interface. At the moment, the interface to the Virtual HBA inside the kernel is through read/write and ioctls to the character device. I am trying to get rid of reading and writing of SCSI
request_bufferthrough character device and use memmap/splice to avoid copying kernel buffer to user space(I am facing some challenges as therequest_bufferis not a linear buffer, but a scatter gather buffer).
Is there any reason you went with the virtual scsi hba? There is a dm-userspace module which would allow you to put any block layer driver in userspace. You should ask Dan Smith <[email protected]> and Tomo <[email protected]> to learn more.
I guess the downside would be that a lot of scsi-ml would be duplicated in userspace. I am not sure how big a deal that is. If it is a problem, you could at least look into what it takes to throw requests and buffers between the kernel and userspace and what code can be shared between your driver, dm-userspace, SG_IO, and the scsi target code.
From: Aboo Valappil <[email protected]>
Subject: SCSITAP, Virtual SCSI HBA and user space SCSI initiators
Date: Fri, 13 Oct 2006 03:25:46 +1000
Quoted 6 lines
I have developed a Virtual HBA (Around ~600 lines of code). It is a kernel module and works very well with 2.6.9 kernel. Basically it registers a HBA (LLD) to the SCSI Mid layer. It implements a linked list with all the SCSI commands queued from Mid layer. I registered a character device driver. I ship out the SCSI commands off to user space via this character device. There is a user space application monitoring
Quoted 8 lines
this character device looking for SCSI commands, process it and sends it back to the mid layer through this character interface. At the moment, the interface to the Virtual HBA inside the kernel is through read/write and ioctls to the character device. I am trying to get rid of reading and writing of SCSI
request_bufferthrough character device and use memmap/splice to avoid copying kernel buffer to user space(I am facing some challenges as therequest_bufferis not a linear buffer, but a scatter gather buffer).
As James and Mike said, tgt provides SCSI command processing in user space (target mode). I implemented the virtual HBA driver (initiator mode) for Xen that works with tgt. I guess that it is similar to your virtual HBA driver.
You can find the slides in Xen Summit site: