Skip to contentSkip to conversation

TCM Core and TCM_Loop patches for v2.6.37

RFC Superseded Nicholas A. Bellinger 4 participants 9 messages Sep 22, 2010 – Oct 11, 2010 SCSI

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover TCM Core and TCM_Loop patches for v2.6.37

From: Nicholas Bellinger <[email protected]>

Greetings Tomo-san, hch, jejb, mnc, boaz, jens, konrad, jlbec, joe, and Co,

The following is a reviewable series containing TCM Core and TCM_Loop v4.0.0-rc4 + very lastest lio-core-2.6.git/lio-4.0 updates against a fresh v2.6.36-rc5 clone who's commit log has been broken up to be reviewable rather than historical. This code is intended for mainline v2.6.37 acceptance and is currently going through the final public review.

The RFCv2 tree can be found in lio-4.0.git/rfcv2-for-37 here:

	http://git.kernel.org/?p=linux/kernel/git/nab/lio-4.0.git;a=shortlog;h=refs/heads/rfcv2-for-37

Or can be pulled via:

        git://git.kernel.org/pub/scm/linux/kernel/git/nab/lio-4.0.git rfcv2-for-37

The patches can also be found individually here:

        http://www.kernel.org/pub/linux/kernel/people/nab/TCM-v4-patches-for-v2.6.36-rc5/

The announcement for the changes between v4.0.0-rc3 -> rc4 is available here:

	http://lkml.org/lkml/2010/9/17/284

Note that this series has dropped the original ConfigFS ->check_link() patch, many thanks to jlbec for making the case that this really was unnecessary. Also note that patch #1 here is the scsi.h defs update that has been posted seperately to linux-scsi:

	http://marc.info/?l=linux-scsi&m=128513469032115&w=2

Please have a look at the code at your earliest convience. As before, I will be making any required changes into lio-core-2.6.git/lio-4.0, and respining this tree as necessary for an upstream maintainer pull for v2.6.37.

Many, many thanks again to all of the folks who contributed. It certainly would not be possible without your help!

Nicholas Bellinger (21):
scsi: Add missing SPC-4 CDB and MAINTENANCE_[IN,OUT] and
VARIABLE_LENGTH_CMD service actions
tcm: Add ConfigFS extended macro set
tcm: Add v4 base data structures and struct target_core_fabric_ops
tcm: Add SPC-4 explict and implict Asymmetric Logical Unit Access

    (ALUA)

tcm: Add ConfigFS subsystem backstore infrastructure
tcm: Add fabric and subsystem transport engine core
tcm: Add HBA core infrastructure
tcm: Add device core infrastructure
tcm: Add target portal group infrastructure
tcm: Add SPC-4 compliant Persistent Reservations (PR)
tcm: Add Task Management infrastructure
tcm: Add UNIT_ATTENTION infrastructure support
tcm: Add SCSI MIB statistics support
tcm: Add FILEIO subsystem plugin
tcm: Add IBLOCK subsystem plugin
tcm: Add PSCSI subsystem plugin
tcm: Add RAMDISK_DR and RAMDISK_MCP subsystem plugins
tcm: Add generic fabric independent ConfigFS infrastructure
tcm: Add generic ProtoID and TransportID fabric handlers for SAS, FC,
and iSCSI
tcm: Add Kbuild and Kconfig for drivers/target,
Documentation/target/tcm_mod_builder.py
tcm_loop: Add multi-fabric Linux/SCSI LLD fabric module

 Documentation/target/tcm_mod_builder.py        | 1039 +++
 Documentation/target/tcm_mod_builder.txt       |  138 +
 drivers/Kconfig                                |    2 +
 drivers/Makefile                               |    1 +
 drivers/target/Kbuild                          |   30 +
 drivers/target/Kconfig                         |   36 +
 drivers/target/target_core_alua.c              | 1993 ++++++
 drivers/target/target_core_alua.h              |  142 +
 drivers/target/target_core_configfs.c          | 3334 +++++++++
 drivers/target/target_core_device.c            | 1705 +++++
 drivers/target/target_core_fabric_configfs.c   | 1001 +++
 drivers/target/target_core_fabric_lib.c        |  451 ++
 drivers/target/target_core_file.c              | 1197 ++++
 drivers/target/target_core_file.h              |   81 +
 drivers/target/target_core_hba.c               |  211 +
 drivers/target/target_core_hba.h               |   14 +
 drivers/target/target_core_iblock.c            | 1173 ++++
 drivers/target/target_core_iblock.h            |   44 +
 drivers/target/target_core_mib.c               | 1081 +++
 drivers/target/target_core_mib.h               |   31 +
 drivers/target/target_core_pr.c                | 4243 ++++++++++++
 drivers/target/target_core_pr.h                |   67 +
 drivers/target/target_core_pscsi.c             | 1569 +++++
 drivers/target/target_core_pscsi.h             |   69 +
 drivers/target/target_core_rd.c                | 1555 +++++
 drivers/target/target_core_rd.h                |   89 +
 drivers/target/target_core_scdb.c              |  158 +
 drivers/target/target_core_scdb.h              |   15 +
 drivers/target/target_core_tmr.c               |  405 ++
 drivers/target/target_core_tpg.c               |  830 +++
 drivers/target/target_core_transport.c         | 8860 ++++++++++++++++++++++++
 drivers/target/target_core_ua.c                |  332 +
 drivers/target/target_core_ua.h                |   36 +
 drivers/target/tcm_loop/Kbuild                 |   11 +
 drivers/target/tcm_loop/Kconfig                |   13 +
 drivers/target/tcm_loop/tcm_loop_configfs.c    |  649 ++
 drivers/target/tcm_loop/tcm_loop_configfs.h    |    2 +
 drivers/target/tcm_loop/tcm_loop_core.h        |   77 +
 drivers/target/tcm_loop/tcm_loop_fabric.c      |  494 ++
 drivers/target/tcm_loop/tcm_loop_fabric.h      |   43 +
 drivers/target/tcm_loop/tcm_loop_fabric_scsi.c |  618 ++
 drivers/target/tcm_loop/tcm_loop_fabric_scsi.h |   10 +
 include/scsi/scsi.h                            |   28 +
 include/target/configfs_macros.h               |  147 +
 include/target/target_core_base.h              | 1056 +++
 include/target/target_core_configfs.h          |   57 +
 include/target/target_core_device.h            |   66 +
 include/target/target_core_fabric_configfs.h   |  106 +
 include/target/target_core_fabric_lib.h        |   28 +
 include/target/target_core_fabric_ops.h        |  101 +
 include/target/target_core_tmr.h               |   43 +
 include/target/target_core_tpg.h               |   39 +
 include/target/target_core_transport.h         |  570 ++
 53 files changed, 36090 insertions(+), 0 deletions(-)
 create mode 100755 Documentation/target/tcm_mod_builder.py
 create mode 100644 Documentation/target/tcm_mod_builder.txt
 create mode 100644 drivers/target/Kbuild
 create mode 100644 drivers/target/Kconfig
 create mode 100644 drivers/target/target_core_alua.c
 create mode 100644 drivers/target/target_core_alua.h
 create mode 100644 drivers/target/target_core_configfs.c
 create mode 100644 drivers/target/target_core_device.c
 create mode 100644 drivers/target/target_core_fabric_configfs.c
 create mode 100644 drivers/target/target_core_fabric_lib.c
 create mode 100644 drivers/target/target_core_file.c
 create mode 100644 drivers/target/target_core_file.h
 create mode 100644 drivers/target/target_core_hba.c
 create mode 100644 drivers/target/target_core_hba.h
 create mode 100644 drivers/target/target_core_iblock.c
 create mode 100644 drivers/target/target_core_iblock.h
 create mode 100644 drivers/target/target_core_mib.c
 create mode 100644 drivers/target/target_core_mib.h
 create mode 100644 drivers/target/target_core_pr.c
 create mode 100644 drivers/target/target_core_pr.h
 create mode 100644 drivers/target/target_core_pscsi.c
 create mode 100644 drivers/target/target_core_pscsi.h
 create mode 100644 drivers/target/target_core_rd.c
 create mode 100644 drivers/target/target_core_rd.h
 create mode 100644 drivers/target/target_core_scdb.c
 create mode 100644 drivers/target/target_core_scdb.h
 create mode 100644 drivers/target/target_core_tmr.c
 create mode 100644 drivers/target/target_core_tpg.c
 create mode 100644 drivers/target/target_core_transport.c
 create mode 100644 drivers/target/target_core_ua.c
 create mode 100644 drivers/target/target_core_ua.h
 create mode 100644 drivers/target/tcm_loop/Kbuild
 create mode 100644 drivers/target/tcm_loop/Kconfig
 create mode 100644 drivers/target/tcm_loop/tcm_loop_configfs.c
 create mode 100644 drivers/target/tcm_loop/tcm_loop_configfs.h
 create mode 100644 drivers/target/tcm_loop/tcm_loop_core.h
 create mode 100644 drivers/target/tcm_loop/tcm_loop_fabric.c
 create mode 100644 drivers/target/tcm_loop/tcm_loop_fabric.h
 create mode 100644 drivers/target/tcm_loop/tcm_loop_fabric_scsi.c
 create mode 100644 drivers/target/tcm_loop/tcm_loop_fabric_scsi.h
 create mode 100644 include/target/configfs_macros.h
 create mode 100644 include/target/target_core_base.h
 create mode 100644 include/target/target_core_configfs.h
 create mode 100644 include/target/target_core_device.h
 create mode 100644 include/target/target_core_fabric_configfs.h
 create mode 100644 include/target/target_core_fabric_lib.h
 create mode 100644 include/target/target_core_fabric_ops.h
 create mode 100644 include/target/target_core_tmr.h
 create mode 100644 include/target/target_core_tpg.h
 create mode 100644 include/target/target_core_transport.h
Signature · 1 line
1.7.3
#1 lore raw
Quoted 4 lines · On Wed, 22 Sep 2010 15:48:22 -0700 "Nicholas A. Bellinger"…
 drivers/Kconfig                                |    2 +
 drivers/Makefile                               |    1 +
 drivers/target/Kbuild                          |   30 +
 drivers/target/Kconfig                         |   36 +

Why do we need a new place for the target stuff? This could be used for non scsi protocl?

We had the similar discussion when I put stgt to mainline but we concluded that under drivers/scsi is the best place.

I don't like to put ibmvscsi driver under something like drivers/target/tcm_ibmvscsit because ibmvscsi needs to include some files under drivers/scsi/ibmvscsi/. It's more reasonable to put the driver there.

Can we change the name, TCM (Target Core Mod), to something more informative? I think that "Core Mod" is really pointless.

This will be the mainline scsi target feature so why can't we name the files and modules in more appropriate way?

#2 lore raw
Quoted 11 lines · FUJITA Tomonori
Quoted 4 lines · On Wed, 22 Sep 2010 15:48:22 -0700 "Nicholas A. Bellinger"…
 drivers/Kconfig                                |    2 +
 drivers/Makefile                               |    1 +
 drivers/target/Kbuild                          |   30 +
 drivers/target/Kconfig                         |   36 +

Why do we need a new place for the target stuff? This could be used for non scsi protocl?

Yes, I have envisioned the princaple pieces of TCM/ConfigFS design being very much SCSI fabric independent from the start of v3.0 development, and I think the v4.0 virtual HBA/DEV abstraction now present in target_core_configfs.c and fabric module independent control plane in target_core_fabric_configfs.c does demonstrate this design feature.

Of course doing 'SCSI-less' target mode this would still involve some work to target_core_transport.c to add ATA specific emulation/passthrough and disable others for the default SPC-3 emulation logic currently in place. However, I do believe the TCM subsystem plugin API in target_core_transport.h for pSCSI, iBLOCK, FILEIO, etc is already more or less SCSI fabric independent and adding a libata subsystem plugin (eg: with it's own set of TCM fabric modules) minus current libata-scsi.c glue code would be possible if the libata folks would like to entertain that discussion..

Quoted 8 lines

We had the similar discussion when I put stgt to mainline but we concluded that under drivers/scsi is the best place.

I don't like to put ibmvscsi driver under something like drivers/target/tcm_ibmvscsit because ibmvscsi needs to include some files under drivers/scsi/ibmvscsi/. It's more reasonable to put the driver there.

Quoted 5 lines

Can we change the name, TCM (Target Core Mod), to something more informative? I think that "Core Mod" is really pointless.

This will be the mainline scsi target feature so why can't we name the files and modules in more appropriate way?

Honestly, I tend not to care very much about naming and things, but that said I would really hate to have to rename actual TCM code at this point for .37 (other than say directory location/layout and file names) while the drivers/target/lio-target -> iscsi_proto.h conversion is still on our TODO list.

Best,

--nab

#3 lore raw
Quoted 19 lines · On Tue, 05 Oct 2010 21:46:45 -0700 "Nicholas A. Bellinger"…
Quoted 11 lines · FUJITA Tomonori
Quoted 4 lines · On Wed, 22 Sep 2010 15:48:22 -0700 "Nicholas A. Bellinger"…
 drivers/Kconfig                                |    2 +
 drivers/Makefile                               |    1 +
 drivers/target/Kbuild                          |   30 +
 drivers/target/Kconfig                         |   36 +

Why do we need a new place for the target stuff? This could be used for non scsi protocl?

Yes, I have envisioned the princaple pieces of TCM/ConfigFS design being very much SCSI fabric independent from the start of v3.0 development, and I think the v4.0 virtual HBA/DEV abstraction now present in target_core_configfs.c and fabric module independent control plane in target_core_fabric_configfs.c does demonstrate this design feature.

Quoted 9 lines

Of course doing 'SCSI-less' target mode this would still involve some work to target_core_transport.c to add ATA specific emulation/passthrough and disable others for the default SPC-3 emulation logic currently in place. However, I do believe the TCM subsystem plugin API in target_core_transport.h for pSCSI, iBLOCK, FILEIO, etc is already more or less SCSI fabric independent and adding a libata subsystem plugin (eg: with it's own set of TCM fabric modules) minus current libata-scsi.c glue code would be possible if the libata folks would like to entertain that discussion..

I like to hear the opinions of SCSI maintainer and ATA folks.

Even if the target feature is SCSI independent, the SCSI drivers should go to under driver/scsi. As I explained, at least, it's a cleaner solution for ibmvscsi target driver.

Quoted 11 lines
Quoted 11 lines

We had the similar discussion when I put stgt to mainline but we concluded that under drivers/scsi is the best place.

I don't like to put ibmvscsi driver under something like drivers/target/tcm_ibmvscsit because ibmvscsi needs to include some files under drivers/scsi/ibmvscsi/. It's more reasonable to put the driver there.

Can we change the name, TCM (Target Core Mod), to something more informative? I think that "Core Mod" is really pointless.

Quoted 8 lines
Quoted 2 lines

This will be the mainline scsi target feature so why can't we name the files and modules in more appropriate way?

Honestly, I tend not to care very much about naming and things, but that said I would really hate to have to rename actual TCM code at this point for .37 (other than say directory location/layout and file names) while the drivers/target/lio-target -> iscsi_proto.h conversion is still on our TODO list.

I'm not sure this goes for .37 (up to James) but anyway I think that we need to take care about the module names now. Once we put stuff into mainline, it's not good to change the module names. File and directory names and layout can be changed any time.

#4 lore raw
Quoted 32 lines · FUJITA Tomonori
Quoted 19 lines · On Tue, 05 Oct 2010 21:46:45 -0700 "Nicholas A. Bellinger"…
Quoted 11 lines · FUJITA Tomonori
Quoted 4 lines · On Wed, 22 Sep 2010 15:48:22 -0700 "Nicholas A. Bellinger"…
 drivers/Kconfig                                |    2 +
 drivers/Makefile                               |    1 +
 drivers/target/Kbuild                          |   30 +
 drivers/target/Kconfig                         |   36 +

Why do we need a new place for the target stuff? This could be used for non scsi protocl?

Yes, I have envisioned the princaple pieces of TCM/ConfigFS design being very much SCSI fabric independent from the start of v3.0 development, and I think the v4.0 virtual HBA/DEV abstraction now present in target_core_configfs.c and fabric module independent control plane in target_core_fabric_configfs.c does demonstrate this design feature.

Quoted 9 lines

Of course doing 'SCSI-less' target mode this would still involve some work to target_core_transport.c to add ATA specific emulation/passthrough and disable others for the default SPC-3 emulation logic currently in place. However, I do believe the TCM subsystem plugin API in target_core_transport.h for pSCSI, iBLOCK, FILEIO, etc is already more or less SCSI fabric independent and adding a libata subsystem plugin (eg: with it's own set of TCM fabric modules) minus current libata-scsi.c glue code would be possible if the libata folks would like to entertain that discussion..

Quoted 1 line

I like to hear the opinions of SCSI maintainer and ATA folks.

jejb, jgarzik, tejun and co..? Any thoughts here..?

Quoted 6 lines

Even if the target feature is SCSI independent, the SCSI drivers should go to under driver/scsi. As I explained, at least, it's a cleaner solution for ibmvscsi target driver.

Fair enough then. Then I will plan to move the upstream lio-core-2.6.git/lio-4.0 branch to live under drivers/scsi/ soon, and rebase the next .37 branch for mainline following this new layout.

Quoted 20 lines
Quoted 11 lines
Quoted 11 lines

We had the similar discussion when I put stgt to mainline but we concluded that under drivers/scsi is the best place.

I don't like to put ibmvscsi driver under something like drivers/target/tcm_ibmvscsit because ibmvscsi needs to include some files under drivers/scsi/ibmvscsi/. It's more reasonable to put the driver there.

Can we change the name, TCM (Target Core Mod), to something more informative? I think that "Core Mod" is really pointless.

Quoted 8 lines
Quoted 2 lines

This will be the mainline scsi target feature so why can't we name the files and modules in more appropriate way?

Honestly, I tend not to care very much about naming and things, but that said I would really hate to have to rename actual TCM code at this point for .37 (other than say directory location/layout and file names) while the drivers/target/lio-target -> iscsi_proto.h conversion is still on our TODO list.

Quoted 3 lines

I'm not sure this goes for .37 (up to James) but anyway I think that we need to take care about the module names now. Once we put stuff into mainline, it's not good to change the module names.

Yes, definately not..

Quoted 1 line

File and directory names and layout can be changed any time.

Thanks again for your comments Tomo!

--nab

#5 lore raw
Quoted 3 lines · Nicholas A. Bellinger
Quoted 1 line

I like to hear the opinions of SCSI maintainer and ATA folks.

jejb, jgarzik, tejun and co..? Any thoughts here..?

I frankly have no idea whatsoever. Is there something I can read to educate myself on the subject?

Thanks.

Signature · 1 line
tejun
#6 lore raw
Quoted 7 lines · Tejun Heo, on 10/06/2010 06:10 PM wrote:
Quoted 3 lines · Nicholas A. Bellinger
Quoted 1 line

I like to hear the opinions of SCSI maintainer and ATA folks.

jejb, jgarzik, tejun and co..? Any thoughts here..?

I frankly have no idea whatsoever. Is there something I can read to educate myself on the subject?

General information about a SCSI target subsystem you can find in
http://scst.sourceforge.net. More internal details about implementation in http://scst.sourceforge.net/scst_pg.html. Features which are implemented in http://scst.sourceforge.net/comparison.html.

Directly on the subject you can find some considerations in
http://lkml.org/lkml/2010/10/1/140 (second half of the message).

My opinion is that in ideal the SCSI initiator subsystem should be drivers/scsi/initiator and the target subsystem - drivers/scsi/target with shared code in drivers/scsi/. But in reality such big reorganization is unlikely possible, so it's better to leave SCSI initiator subsystem in drivers/scsi and the target subsystem - in the separate directory in drivers/ (drivers/scst for SCST).

Vlad

#7 lore raw

Hello,

Quoted 17 lines · Vladislav Bolkhovitin
Quoted 7 lines · Tejun Heo, on 10/06/2010 06:10 PM wrote:
Quoted 3 lines · Nicholas A. Bellinger
Quoted 1 line

I like to hear the opinions of SCSI maintainer and ATA folks.

jejb, jgarzik, tejun and co..? Any thoughts here..?

I frankly have no idea whatsoever. Is there something I can read to educate myself on the subject?

General information about a SCSI target subsystem you can find in
http://scst.sourceforge.net. More internal details about implementation in http://scst.sourceforge.net/scst_pg.html. Features which are implemented in http://scst.sourceforge.net/comparison.html.

Directly on the subject you can find some considerations in
http://lkml.org/lkml/2010/10/1/140 (second half of the message).

Quoted 6 lines

My opinion is that in ideal the SCSI initiator subsystem should be drivers/scsi/initiator and the target subsystem - drivers/scsi/target with shared code in drivers/scsi/. But in reality such big reorganization is unlikely possible, so it's better to leave SCSI initiator subsystem in drivers/scsi and the target subsystem - in the separate directory in drivers/ (drivers/scst for SCST).

Thanks for the pointers. Yeah, it would be pretty interesting to have ATA support there too and for ATA I don't really see why anyone would mix the initiator and target driver implementations. They may share protocol constants and some utility functions but that will be about it, so it will mostly be a completely separate subsystem and I think it probably is better to put it somewhere separate too. I think it would be best if we can keep the two subsystems clearly separated. They're targeting very different and most likely disjoint audiences after all.

Thanks.

Signature · 1 line
tejun
#8 lore raw
Quoted 27 lines · Tejun Heo

Hello,

Quoted 17 lines · Vladislav Bolkhovitin
Quoted 7 lines · Tejun Heo, on 10/06/2010 06:10 PM wrote:
Quoted 3 lines · Nicholas A. Bellinger
Quoted 1 line

I like to hear the opinions of SCSI maintainer and ATA folks.

jejb, jgarzik, tejun and co..? Any thoughts here..?

I frankly have no idea whatsoever. Is there something I can read to educate myself on the subject?

General information about a SCSI target subsystem you can find in
http://scst.sourceforge.net. More internal details about implementation in http://scst.sourceforge.net/scst_pg.html. Features which are implemented in http://scst.sourceforge.net/comparison.html.

Directly on the subject you can find some considerations in
http://lkml.org/lkml/2010/10/1/140 (second half of the message).

Quoted 6 lines

My opinion is that in ideal the SCSI initiator subsystem should be drivers/scsi/initiator and the target subsystem - drivers/scsi/target with shared code in drivers/scsi/. But in reality such big reorganization is unlikely possible, so it's better to leave SCSI initiator subsystem in drivers/scsi and the target subsystem - in the separate directory in drivers/ (drivers/scst for SCST).

Quoted 9 lines

Thanks for the pointers. Yeah, it would be pretty interesting to have ATA support there too and for ATA I don't really see why anyone would mix the initiator and target driver implementations. They may share protocol constants and some utility functions but that will be about it, so it will mostly be a completely separate subsystem and I think it probably is better to put it somewhere separate too. I think it would be best if we can keep the two subsystems clearly separated. They're targeting very different and most likely disjoint audiences after all.

Hi Tejun,

So following hch's recent input and my series for unifying the SCSI control CDB emulation handling of virtual subsystem plugins in TCM v4.0, the actual struct se_subsystem_api abstraction itself and individual IBLOCK, FILEIO and RAMDISK subsystem plugins will now be able to function indepedent of any SCSI layer code. This will obviously still require changes to TCM proper and a functioning ATA target mode driver of course :-), but depending upon the amount of community interest in this area is something we can start looking at for .38 and .39

Best,

--nab