SCSI and FC Transport: add netlink support for posting of transport events
Subsystems (2)
PATCH 1/1 SCSI and FC Transport: add netlink support for posting of transport events
All,
This patch formally adds support for the posting of FC events via netlink. It is a followup to the original RFC at:
http://marc.theaimsgroup.com/?l=linux-scsi&m=114530667923464&w=2
Per discussions at the Storage Summit and at OLS, we are to use netlink for async events from transports. Also per discussions, to avoid a netlink protocol per transport, I've create a single NETLINK_SCSITRANSPORT protocol, which can then be used by all transports.
This patch:
- Creates new files scsi_netlink.c and scsi_netlink.h, which contains the single and shared definitions for the SCSI Transport. It is tied into the base SCSI subsystem intialization.
Contains a single interface routine, scsi_send_transport_event(), for a transport to send an event (via multicast to a protocol specific group).
- Creates a new scsi_netlink_fc.h file, which contains the FC netlink event messages
- Adds 3 new routines to the fc transport, to send a simple FC event fc_get_event_number() - to get a FC event # fc_host_post_event() - to send a simple FC event (32 bits of data) fc_host_post_vendor_event() - to send a Vendor unique event, with arbitrary amounts of data.
Note: the separation of event number allows for a LLD to send a standard
event, followed by vendor-specific data for the event.
-- james s
drivers/scsi/Kconfig +1 -0
@@ -8,6 +8,7 @@ config RAID_ATTRS config SCSI tristate "SCSI device support" + select NET ---help--- If you want to use a SCSI hard disk, SCSI tape drive, SCSI CD-ROM or any other SCSI device under Linux, say Y and make sure that you know
drivers/scsi/Makefile +1 -1
@@ -155,7 +155,7 @@ obj-$(CONFIG_SCSI_DEBUG) += scsi_debug.o scsi_mod-y += scsi.o hosts.o scsi_ioctl.o constants.o \ scsicam.o scsi_error.o scsi_lib.o \ scsi_scan.o scsi_sysfs.o \ - scsi_devinfo.o + scsi_devinfo.o scsi_netlink.o scsi_mod-$(CONFIG_SYSCTL) += scsi_sysctl.o scsi_mod-$(CONFIG_SCSI_PROC_FS) += scsi_proc.o
drivers/scsi/scsi.c +3 -0
@@ -1115,6 +1115,8 @@ static int __init init_scsi(void)for_each_possible_cpu (i)INIT_LIST_HEAD (&per_cpu (scsi_done_q, i)); +scsi_netlink_init (); +printk (KERN_NOTICE "SCSI subsystem initialized\n" );return 0 ;
@@ -1135,6 +1137,7 @@ cleanup_queue: static void __exit exit_scsi(void) { +scsi_netlink_exit ();scsi_sysfs_unregister ();scsi_exit_sysctl ();scsi_exit_hosts ();
drivers/scsi/scsi_netlink.c +262 -0
262 lines not on this page · raw message
drivers/scsi/scsi_priv.h +6 -0
@@ -8,6 +8,7 @@ struct scsi_cmnd;struct scsi_device ;struct scsi_host_template ;struct Scsi_Host ; +struct scsi_nl_hdr ;/*
@@ -110,6 +111,11 @@ extern void __scsi_remove_device(structextern struct bus_type scsi_bus_type; +/* scsi_netlink.c */ +extern void scsi_netlink_init (void ); +extern void scsi_netlink_exit (void ); +extern void scsi_send_transport_event (u32 group,struct scsi_nl_hdr *hdr); +/* * internal scsi timeout functions: for use by mid-layer and transport * classes.
drivers/scsi/scsi_transport_fc.c +144 -1
166 lines not on this page · raw message
include/linux/netlink.h +2 -0
@@ -21,6 +21,8 @@#define NETLINK_DNRTMSG 14/* DECnet routing messages */ #define NETLINK_KOBJECT_UEVENT 15/* Kernel messages to userspace */ #define NETLINK_GENERIC 16 +/* leave room for NETLINK_DM (DM Events) */ +#define NETLINK_SCSITRANSPORT 18/* SCSI Transports */ #define MAX_LINKS 32
include/scsi/scsi_netlink_fc.h +68 -0
68 lines not on this page · raw message
include/scsi/scsi_netlink.h +86 -0
86 lines not on this page · raw message
include/scsi/scsi_transport_fc.h +40 -7
71 lines not on this page · raw message
Quoted 20 lines · James Smart
include/scsi/scsi_netlink_fc.h +19
-0+/* +* FC Transport Broadcast Event Message : +* FC_NL_ASYNC_EVENT +* +* Note: if Vendor Unique message, &event_data will be start of +* vendor unique payload, and the length of the payload is +* per event_datalen +*/ +struct fc_nl_event { +struct scsi_nl_hdr snlh;/* must be 1st element ! */ +uint64_t seconds; +uint32_t vendor_id; +uint16_t host_no; +uint16_t event_datalen; +uint32_t event_num; +uint32_t event_code; +uint32_t event_data; +} __attribute__((aligned (sizeof (uint64_t )))); +
include/scsi/scsi_netlink_fc.h line 68
Quoted 10 lines
include/scsi/scsi_netlink_fc.h +10
-0+ +/* SCSI_TRANSPORT_MSG event message header */ +struct scsi_nl_hdr { +uint8_t version; +uint8_t transport; +uint16_t magic; +uint16_t msgtype; +uint16_t msglen; +} __attribute__((aligned (sizeof (uint64_t )))); +
What about using netlink attributes? The attributes do have more overhead, but it would seem that using attributes instead of sending out a whole structure would make it easier on the user space side to support different kernel versions without syncing the tools with the kernel.
-andmike
--
Michael Anderson
[email protected]
Quoted 4 lines · Mike Anderson
What about using netlink attributes? The attributes do have more overhead, but it would seem that using attributes instead of sending out a whole structure would make it easier on the user space side to support different kernel versions without syncing the tools with the kernel.
Yeah, I guessed you were going to raise this.
I just didn't see any advantage to it. The macros really didn't help with anything really interesting, such as endianness. I guess they do help with some potential structure packing issues. I saw them as just ensuring that you walked on the path right - which for most intents and purposes, you will as you really are not self-discovering data. As long as you use well-defined types and watch the structure alignments, it should be fine. The overhead didn't seem worthit. Enlighten me if I'm taking them too lightly.
I'm assuming your "syncing the tools with the kernel" statement infers the use of a version element in place of the attributes above. I don't see how you lose the version. It's the key for the later structure decode regardless of whether it's attribute-ized or not. Am I missing something ? I'm also (obviously) a proponent of the version so that a tool can provide backward compatibility, or spot new un-recognized data.
-- james
Quoted 8 lines · James Smart
Quoted 4 lines · Mike Anderson
What about using netlink attributes? The attributes do have more overhead, but it would seem that using attributes instead of sending out a whole structure would make it easier on the user space side to support different kernel versions without syncing the tools with the kernel.
Yeah, I guessed you were going to raise this.
Quoted 8 lines
I just didn't see any advantage to it. The macros really didn't help with anything really interesting, such as endianness. I guess they do help with some potential structure packing issues. I saw them as just ensuring that you walked on the path right - which for most intents and purposes, you will as you really are not self-discovering data. As long as you use well-defined types and watch the structure alignments, it should be fine. The overhead didn't seem worthit. Enlighten me if I'm taking them too lightly.
Packing issue avoidance was a feature of attributes that got me to start using them.
It also allows better organization of the data through nested attributes or even just straight attributes. Allowing for a single large allocation and then multiple users can add to the payload prior to the send.
While attributes are not the only solution it appears right now in the patch that we could optimize the send sequence. Currently for an event we allocate an event only to allocate a skb shortly after to copy the event data into and then free the previous allocated event.
Quoted 7 lines
I'm assuming your "syncing the tools with the kernel" statement infers the use of a version element in place of the attributes above. I don't see how you lose the version. It's the key for the later structure decode regardless of whether it's attribute-ized or not. Am I missing something ? I'm also (obviously) a proponent of the version so that a tool can provide backward compatibility, or spot new un-recognized data.
Since an attribute is not exposing the data as a fixed structure it leads to more options in the future of reorganizing the data. An old program running on a new kernel would be able to pick up known attributes while ignoring others. One could never reorg the structure and always add to the end of the structure which would address the issue also without using attributes.
-andmike
--
Michael Anderson
[email protected]
OK... I'll recut with the attributes and optimized data structure allocations.
-- james s
Quoted 49 lines · Mike Anderson
Quoted 7 lines · James Smart
Quoted 4 lines · Mike Anderson
What about using netlink attributes? The attributes do have more overhead, but it would seem that using attributes instead of sending out a whole structure would make it easier on the user space side to support different kernel versions without syncing the tools with the kernel.
Yeah, I guessed you were going to raise this.
Quoted 7 lines
I just didn't see any advantage to it. The macros really didn't help with anything really interesting, such as endianness. I guess they do help with some potential structure packing issues. I saw them as just ensuring that you walked on the path right - which for most intents and purposes, you will as you really are not self-discovering data. As long as you use well-defined types and watch the structure alignments, it should be fine. The overhead didn't seem worthit. Enlighten me if I'm taking them too lightly.
Packing issue avoidance was a feature of attributes that got me to start using them.
It also allows better organization of the data through nested attributes or even just straight attributes. Allowing for a single large allocation and then multiple users can add to the payload prior to the send.
While attributes are not the only solution it appears right now in the patch that we could optimize the send sequence. Currently for an event we allocate an event only to allocate a skb shortly after to copy the event data into and then free the previous allocated event.
Quoted 6 lines
I'm assuming your "syncing the tools with the kernel" statement infers the use of a version element in place of the attributes above. I don't see how you lose the version. It's the key for the later structure decode regardless of whether it's attribute-ized or not. Am I missing something ? I'm also (obviously) a proponent of the version so that a tool can provide backward compatibility, or spot new un-recognized data.
Since an attribute is not exposing the data as a fixed structure it leads to more options in the future of reorganizing the data. An old program running on a new kernel would be able to pick up known attributes while ignoring others. One could never reorg the structure and always add to the end of the structure which would address the issue also without using attributes.
-andmike
--
Michael Anderson
[email protected]
Mike,
So I'm converting this over, and I really don't like using attributes.
My first major hurdle is that I have to update the netlink library to support a new "binary blob" attribute, that may be of arbitrary length, similar to the "string" attribute.
My second hurdle is dealing with arbitrary-sized messages. Due to their existence and their rarity, I punted on creating fixed size mempools. Headache I have now is trying to size the skb payload. It has to be large enough for the variable-length payload, plus large enough for all the attribute overhead for each field element, and any padding. Yech!
Coding-wise, the format and understanding of what message has what data is more confusing. It pushes for a flat "attribute" space for all message elements. Minimally, it requires a lot more formality of data structures to impose "structure" on the elements. Pure overhead and less clarity.
I started using your dm patch as a reference and came away scratching my head. It uses both a packed structure (for a header) followed by attribute-based data (actual message elements). Seemed real odd that we would use both, and if we trust it for one, why not the other ?
Also, there was one other "pro" where you dinged the implementation that wasn't fully true...
Quoted 6 lines
Since an attribute is not exposing the data as a fixed structure it leads to more options in the future of reorganizing the data. An old program running on a new kernel would be able to pick up known attributes while ignoring others. One could never reorg the structure and always add to the end of the structure which would address the issue also without using attributes.
As implemented, we could always add elements, but never reorg or subtract, and the app would continue to work fine. This is due to the msglen field being in place.
So, I currently want to update the patch only to optimize allocations on the send sequence. Would you buy into this ?
-- james
Quoted 4 lines · James Smart
Mike,
So I'm converting this over, and I really don't like using attributes.
ok, I replied to most of your comments, but you can skip to the bottom and ignore them if you want.
Quoted 4 lines
My first major hurdle is that I have to update the netlink library to support a
new "binary blob" attribute, that may be of arbitrary length, similar to the "string" attribute.
You should not need to edit the netlink library as you should be able to use NLA_PUT_TYPE. Unless I missing the question.
Quoted 10 lines
My second hurdle is dealing with arbitrary-sized messages. Due to their existence
and their rarity, I punted on creating fixed size mempools. Headache I have now
is trying to size the skb payload. It has to be large enough for the variable-length payload, plus large enough for all the attribute overhead for
each field element, and any padding. Yech!
This is hard. In dm code I over allocate, but my number of attributes are bounded. You must have to deal with correct skb payload sizing today right?
Quoted 5 lines
Coding-wise, the format and understanding of what message has what data is more confusing. It pushes for a flat "attribute" space for all message elements. Minimally, it requires a lot more formality of data structures to impose "structure" on the elements. Pure overhead and less clarity.
You can use nested attributes to help some here.
Quoted 6 lines
I started using your dm patch as a reference and came away scratching my head.
It uses both a packed structure (for a header) followed by attribute-based data (actual message elements). Seemed real odd that we would use both, and if
we trust it for one, why not the other ?
I think it is not some much trust as flexibility with what will change. The header size is expected to be more constant.
Quoted 9 lines
Also, there was one other "pro" where you dinged the implementation that wasn't fully true...
Quoted 6 lines
Since an attribute is not exposing the data as a fixed structure it leads to more options in the future of reorganizing the data. An old program running on a new kernel would be able to pick up known attributes while ignoring others. One could never reorg the structure and always add to the end of the structure which would address the issue also without using attributes.
Quoted 3 lines
As implemented, we could always add elements, but never reorg or subtract, and the app would continue to work fine. This is due to the msglen field being in place.
I think that is what I was trying to say at the end that one could increase the size of the structure without attributes and that should be ok if alignment is not an issue.
Quoted 4 lines
So, I currently want to update the patch only to optimize allocations on the send sequence. Would you buy into this ?
Yes, I do not want to hold up the overall capability on this point if others are ok with the data payload.
-andmike
--
Michael Anderson
[email protected]