InfiniBand/RDMA merge plans for 2.6.25
The 2.6.25 will open soon, so it's time to review what my plans are for the merge window opens.
A note on reviewing: the upstream push for "Reviewed-by:" tags seems to have lost some momentum, but it is still the case that if you can get someone other than me to review your work, then the chances of it being merged increase dramatically. I'm talking about a real review-- ideally, someone independent (a different domain after the '@' would be good ;) who is willing to provide a "Reviewed-by:" line that means the reviewer has really looked at and thought about the patch. There should be a mailing list thread you can point me at where the reviewer comments on the patch and a new version of that patch addressing all comments is posted (or in exceptional cases, where the patch is perfect to start with, where the reviewer says the patch is great).
Anyway, here are all the pending things that I'm aware of. As usual, if something isn't already in my tree and isn't listed below, I probably missed it or dropped it by mistake. Please remind me again in that case.
Core:
- Sean's inter-subnet CM changes. My first reaction is that I'll probably merge it, but I need to find the time to really read it over first.
ULPs:
- Rolf's IPoIB MGID scope changes. I have the core changes queued up, so at least recompiling the whole kernel should no longer be necessary. I need to look over the rest of the patches more carefully and make sure we have the correct userspace interface at least.
- Eli's IPoIB stateless offload (checksum offload, LSO, interrupt moderation, etc). It's a big series that makes quite a few core changes. I hope to merge at least part of this queue soon. Outside opinions are welcome though.
- Remove LLTX from IPoIB. I haven't posted a draft of this patch yet, so I guess it will probably wait for 2.6.26...
- I still have a few SRP changes from David Dillow to review and apply. Nothing looked to radical so they should be good to go.
HW specific:
- Neteffect "nes" driver. It's not terribly clean code but since it's a new driver that is completely self-contained, I plan on merging it and letting cleanups happen upstream.
- mlx4 WQE shrinking patch. I would like to see this as two patches:
one that vmap()s work queues, and another that does the tricky part of using smaller WQEs. If I get a chance I'll try to do the split up myself.
- ipath pending patches. Nothing looks like a problem there; it's just a matter of actually pulling the patches into my tree.
Here are a few topics that I believe will not be ready in time for the 2.6.25 window and will need to wait for 2.6.26 at least:
- Multiple CQ event vector support. I still haven't seen any discussions about how ULPs or userspace apps should decide which vector to use, and hence no progress has been made since we deferred this during the 2.6.23 merge window.
- XRC. I don't get the feeling that even the interface has converged sufficiently to merge this.
Here all the patches I already have in my for-2.6.25 branch:
Adrian Bunk (1):
IB/mthca: Remove MSI support as scheduled
Anton Blanchard (1):
IB/ehca: Use round_jiffies() for EQ polling timer
Arthur Jones (2):
IB/ipath: Better comment for rmb() in ipath_intr() IB/ipath: Add ipath_read_ireg() abstraction
Dave Olson (5):
IB/ipath: Improve interrupt handler cache footprint IB/ipath: Generalize some xxx_SHIFT macros IB/ipath: Changes for fields moving from devdata to portdata IB/ipath: Clean up some comments
IB/ipath: Drop support for the original QHT7040 board
David Dillow (3):
IB/srp: Respect target credit limit IB/srp: Enable SG list chaining
IB/srp: Add identifying information to log messages
Erez Zilber (1):
IB/iser: update URLs of iSER docs
Hoang-Nam Nguyen (1):
IB/ehca: Forward event client-reregister-required to registered clients
Jack Morgenstein (1):
mlx4_core: Fix max_eqs masking in QUERY_DEV_CAP
Joe Perches (2):
drivers/infiniband: Add missing "space"
IB: Spelling fixes in comments
John Gregor (1):
IB/ipath: Fix sendctrl locking
Matthias Kaehlcke (1):
IB/ipath: Convert ipath_eep_sem semaphore to a mutex
Nick Piggin (1):
IB/ipath: Convert from .nopage to .fault
Olaf Kirch (1):
IB/fmr_pool: Flush serial numbers can get out of sync
Oliver Pinter (1):
IB/iser: Typo fix (s/destory/destroy/)
Pradeep Satyanarayana (2):
IPoIB/cm: Add connected mode support for devices without SRQs IPoIB/CM: Enable SRQ support on HCAs that support fewer than 16 SG entries
Ralph Campbell (14):
IB/mad: Remove redundant NULL pointer check in ib_mad_recv_done_handler() IB/ipath: Enable loopback of DR SMP responses from userspace IB/ipath: Remove dead code for user process waiting for send buffer IB/ipath: Fix error returned from ib_resize_cq if new size smaller than # entries IB/ipath: Fix comments for ipath_create_srq()
IB/ipath: Add the work completion error code to the QP error debug output IB/ipath: Fix RNR NAK handling
IB/ipath: Cleanup ipath_get_egrbuf()
IB/ipath: kreceive uses portdata rather than devdata
IB/ipath: MAD performance sampling registers support
IB/ipath: Export hardware counters more consistently
IB/ipath: Allow more flexible user register alignments
IB/ipath: Port config has on-chip effects for 7220
IB/ipath: Add flag and handling for chips with swapped register bug
Roland Dreier (8):
IPoIB: Trivial formatting cleanups
IPoIB/cm: Factor out ipoib_cm_free_rx_ring() IPoIB/cm: Factor out ipoib_cm_create_srq() IPoIB/cm: Factor out ipoib_cm_free_rx_reap_list() IB/mlx4: Micro-optimize mlx4_ib_poll_one() RDMA/cxgb3: Endianness annotation for irs field IB/ipath: Fix some sparse warnings about shadowed symbols IB/umad: Simplify and fix locking
Rolf Manderscheid (1):
IPoIB: improve IPv4/IPv6 to IB mcast mapping functions
Sean Hefty (6):
IB/multicast: Report errors on multicast groups if P_key changes IB/mad: Report number of times a mad was retried IB/cm: Add basic performance counters
IB/mad: Fix incorrect access to items on local_list RDMA/cma: add support for rdma_migrate_id()
RDMA/cma: Override default responder_resources with user value
Steve Welch (1):
IB/mad: Enable loopback of DR SMP responses from userspace
Steve Wise (3):
RDMA/iwcm: Set initiator depth and responder resources to device max values RDMA/cxgb3: Hold rtnl_lock() around ethtool get_drvinfo call RDMA/cxgb3: Support version 5.0 firmware
Vladimir Sokolovsky (1):
RDMA/cma: Reenable device removal on passive side
Quoted 3 lines · Roland Dreier
Here all the patches I already have in my for-2.6.25 branch:
Sean Hefty (6):
IB/mad: Fix incorrect access to items onlocal_list
It wasn't clear to me that this issue was ever really nailed. Was the loop on this closed ?
-- Hal
Quoted 5 lines
Quoted 2 lines
Sean Hefty (6):
IB/mad: Fix incorrect access to items onlocal_listIt wasn't clear to me that this issue was ever really nailed. Was the loop on this closed ?
The error that this patches addresses is fairly obvious if you inspect the code. There's a strong chance that the patch fixes the bug that was reported, but the last I remember, they had trouble reproducing the crash to see if the patch would indeed make it go away.
- Sean
Quoted 7 lines · Sean Hefty
Quoted 5 lines
Quoted 2 lines
Sean Hefty (6):
IB/mad: Fix incorrect access to items onlocal_listIt wasn't clear to me that this issue was ever really nailed. Was the loop on this closed ?
The error that this patches addresses is fairly obvious if you inspect the code.
The bug seems obvious but I'm not sure about the fix. Just my $0.02 worth.
-- Hal
Quoted 5 lines
There's a strong chance that the patch fixes the bug that was reported, but the last I remember, they had trouble reproducing the crash to see if the patch would indeed make it go away.
- Sean
Quoted 3 lines · Roland Dreier
- Neteffect "nes" driver. It's not terribly clean code but since it's a new driver that is completely self-contained, I plan on merging it and letting cleanups happen upstream.
New code should be better quality than old code, not worse. I haven't
actually seen the driver yet, but by that statement I'd be clearly against a merge.
Quoted 7 lines
Quoted 3 lines
- Neteffect "nes" driver. It's not terribly clean code but since it's a new driver that is completely self-contained, I plan on merging it and letting cleanups happen upstream.
New code should be better quality than old code, not worse. I haven't
actually seen the driver yet, but by that statement I'd be clearly against a merge.
The driver has been posted a few times; the latest code is in the "neteffect" branch of my tree:
git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git neteffect
It's not *that* bad -- certainly there are lots of things that could be improved (sparse endianness annotation, too many lines that are way to long, strange indentation of case labeles, etc, etc) but it is a self-contained hardware driver. I agree with Linus's position (stated at the last kernel summit) that we ought to merge hardware drivers early, so that users get the drivers with as little hassle as possible. We lose a little leverage in getting cleanups done, but the number of people who see the code and are able to clean it up increases, so I think it's a good trade-off.
- R.
Quoted 37 lines
From: [email protected] [mailto:[email protected]]On Behalf Of Roland Dreier
Sent: Tuesday, January 22, 2008 3:56 PM
To: Christoph Hellwig
Cc: [email protected]; [email protected]
Subject: [ofa-general] Re: InfiniBand/RDMA merge plans for 2.6.25Quoted 1 line
Quoted 1 line
- Neteffect "nes" driver. It's not terribly clean code
but since
Quoted 4 lines
Quoted 2 lines
it's a new driver that is completely self-contained, I plan on merging it and letting cleanups happen upstream.
New code should be better quality than old code, not
worse. I haven't
Quoted 2 lines
actually seen the driver yet, but by that statement I'd be clearly against a merge.
The driver has been posted a few times; the latest code is in the "neteffect" branch of my tree:
git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniban
d.git neteffectIt's not *that* bad -- certainly there are lots of things that could be improved (sparse endianness annotation, too many lines that are way to long, strange indentation of case labeles, etc, etc) but it is a self-contained hardware driver. I agree with Linus's position (stated at the last kernel summit) that we ought to merge hardware drivers early, so that users get the drivers with as little hassle as possible. We lose a little leverage in getting cleanups done, but the number of people who see the code and are able to clean it up increases, so I think it's a good trade-off.
Quoted 2 lines
- R.
My view is the code should and will be cleaned up based upon the feedback we've gotten from the community. It is a priority for me.
Several cleanup fixes are in the queue and are being worked. Haven't slipped into complacency at the prospect of the merge.
Glenn
[email protected]
Quoted 1 line · Roland Dreier
be improved (sparse endianness annotation,
that's a blocker for sure. No new code that's not sparse clean, please.
Quoted 3 lines
Quoted 1 line
be improved (sparse endianness annotation,
that's a blocker for sure. No new code that's not sparse clean, please.
I have to disagree -- remember how strongly Linus pushed to merge hardware drivers early? the code in question will not run unless you have the hardware it drives, and that hardware is useless without the code. And *something* is better for users than nothing.
Anyway I don't think the endianness annotations are that hard so it will probably show up soon.
- R.
Rather than arguing over whether we have to have sparse clean code, I decided to annotate the code myself. Here's a patch that fixes most of the sparse warnings in the nes driver. There's still some stuff that actually looks buggy, like the way hte_index stuff is handled. You initialize hte_index_mask as:
hte_index_mask = ((u32)1 << ((u32temp & 0x001f)+1))-1; nesadapter->hte_index_mask = hte_index_mask;
but then compute hte_index stuff with:
nesqp->hte_index = cpu_to_be32( crc32c(~0, (void *)&nes_quad, sizeof(nes_quad)) ^ 0xffffffff);
and then do:
nesqp->hte_index &= nesadapter->hte_index_mask;
which seems odd to say the least (hte_index is big-endian, hte_index_mask is cpu-endian).
And also, there's code with the loc_addr/rem_addr etc that seem very confused. For example
cm_info->loc_addr = htonl(cm_info->loc_addr); cm_info->rem_addr = htonl(cm_info->rem_addr); cm_info->loc_port = htons(cm_info->loc_port); cm_info->rem_port = htons(cm_info->rem_port);
which is obviously impossible to annotate correctly, and I couldn't keep track of the endianness stuff elsewhere.
Anyway this is what I have in case the promised cleanups don't turn up in time...
drivers/infiniband/hw/nes/nes.c +2 -2
@@ -231,10 +231,10 @@ static int nes_net_event(struct notifier_block *notifier, }else {if (neigh->nud_state &NUD_VALID ) {nes_manage_arp_cache (neigh->dev, neigh->ha, -ntohl (*(u32 *)neigh->primary_key),NES_ARP_ADD ); +ntohl (*(__be32 *)neigh->primary_key),NES_ARP_ADD ); }else {nes_manage_arp_cache (neigh->dev, neigh->ha, -ntohl (*(u32 *)neigh->primary_key),NES_ARP_DELETE ); +ntohl (*(__be32 *)neigh->primary_key),NES_ARP_DELETE ); } }return NOTIFY_OK ;
drivers/infiniband/hw/nes/nes.h +1 -0
@@ -203,6 +203,7 @@ extern u32 cm_packets_retrans;extern u32 cm_listens_created;extern u32 cm_listens_destroyed;extern u32 cm_backlog_drops; +extern atomic_t cm_loopbacks;extern atomic_t cm_nodes_created;extern atomic_t cm_nodes_destroyed;extern atomic_t cm_accel_dropped_pkts;
drivers/infiniband/hw/nes/nes_cm.c +10 -10
@@ -187,7 +187,7 @@ static int parse_mpa(struct nes_cm_node *cm_node, u8 *buffer, u32 len) } mpa_frame = (struct ietf_mpa_frame *)buffer; - cm_node->mpa_frame_size = (u32 )ntohs (mpa_frame->priv_data_len); + cm_node->mpa_frame_size =ntohs (mpa_frame->priv_data_len);if (cm_node->mpa_frame_size +sizeof (struct ietf_mpa_frame ) != len) { nes_debug(NES_DBG_CM, "The received ietf buffer was not right"
@@ -271,7 +271,7 @@ struct sk_buff *form_cm_frame(struct sk_buff *skb, struct nes_cm_node *cm_node, buf +=sizeof (*tcph); skb->ip_summed =CHECKSUM_PARTIAL ; - skb->protocol =ntohs (0x800 ); + skb->protocol =htons (0x800 ); skb->data_len =0 ; skb->mac_len =ETH_HLEN ;
@@ -285,7 +285,7 @@ struct sk_buff *form_cm_frame(struct sk_buff *skb, struct nes_cm_node *cm_node, iph->tot_len =htons (packetsize); iph->id =htons (++cm_node->tcp_cntxt.loc_id); - iph->frag_off =ntohs (0x4000 ); + iph->frag_off =htons (0x4000 ); iph->ttl =0x40 ; iph->protocol=0x06 ;/* IPPROTO_TCP */
@@ -394,7 +394,7 @@ int schedule_nes_timer(struct nes_cm_node *cm_node, struct sk_buff *skb, }if (type ==NES_TIMER_TYPE_SEND ) { - new_send->seq_num =htonl (tcp_hdr (skb)->seq); + new_send->seq_num =ntohl (tcp_hdr (skb)->seq);atomic_inc (&new_send->skb->users); ret =nes_nic_cm_xmit (new_send->skb, cm_node->netdev);
@@ -419,7 +419,7 @@ int schedule_nes_timer(struct nes_cm_node *cm_node, struct sk_buff *skb,spin_unlock_irqrestore (&cm_node->retrans_list_lock, flags); }if (type ==NES_TIMER_TYPE_RECV ) { - new_send->seq_num =htonl (tcp_hdr (skb)->seq); + new_send->seq_num =ntohl (tcp_hdr (skb)->seq); new_send->timetosend = jiffies;spin_lock_irqsave (&cm_node->recv_list_lock, flags);list_add_tail (&new_send->list, &cm_node->recv_list);
@@ -1245,7 +1245,7 @@ static int process_options(struct nes_cm_node *cm_node, u8 *optionsloc, u32 optiif (all_options->as_mss.length !=4 ) {return 1 ; }else { - tmp =htons (all_options->as_mss.mss); + tmp =ntohs (all_options->as_mss.mss);if (tmp >0 && tmp < cm_node->tcp_cntxt.mss) cm_node->tcp_cntxt.mss = tmp; }
@@ -1369,7 +1369,7 @@ int process_packet(struct nes_cm_node *cm_node, struct sk_buff *skb,else if (tcph->syn) cm_node->tcp_cntxt.mss =NES_CM_DEFAULT_MSS ; - cm_node->tcp_cntxt.snd_wnd =htons (tcph->window) << + cm_node->tcp_cntxt.snd_wnd =ntohs (tcph->window) << cm_node->tcp_cntxt.snd_wscale;if (cm_node->tcp_cntxt.snd_wnd > cm_node->tcp_cntxt.max_snd_wnd) {
@@ -1621,7 +1621,7 @@ static struct nes_cm_listener *mini_cm_listen(struct nes_cm_core *cm_core,nes_debug (NES_DBG_CM ,"Api - listen(): addr=0x%08X, port=0x%04x," " listener = %p, backlog = %d, cm_id = %p.\n" , -ntohl (cm_info->loc_addr),ntohs (cm_info->loc_port), +htonl (cm_info->loc_addr),htons (cm_info->loc_port), listener, listener->backlog, listener->cm_id);return listener;
@@ -1827,7 +1827,7 @@ int mini_cm_recv_pkt(struct nes_cm_core *cm_core, struct nes_vnic *nesvnic, tcph = (struct tcphdr *)(skb->data +sizeof (struct iphdr ));skb_reset_network_header (skb);skb_set_transport_header (skb,sizeof (*tcph)); - skb->len =htons (iph->tot_len); + skb->len =ntohs (iph->tot_len); nfo.loc_addr =ntohl (iph->daddr); nfo.loc_port =ntohs (tcph->dest);
@@ -2832,7 +2832,7 @@ void cm_event_connected(struct nes_cm_event *event) nesqp->hte_index =cpu_to_be32 (crc32c (~0 , (void *)&nes_quad,sizeof (nes_quad)) ^0xffffffff );nes_debug (NES_DBG_CM ,"HTE Index = 0x%08X, After CRC = 0x%08X\n" , - nesqp->hte_index, nesqp->hte_index & nesadapter->hte_index_mask); +be32_to_cpu (nesqp->hte_index),be32_to_cpu (nesqp->hte_index) & nesadapter->hte_index_mask); nesqp->hte_index &= nesadapter->hte_index_mask; nesqp->nesqp_context->hte_index =cpu_to_le32 (nesqp->hte_index);
drivers/infiniband/hw/nes/nes_cm.h +5 -5
@@ -55,7 +55,7 @@ struct ietf_mpa_frame {u8 key[IETF_MPA_KEY_SIZE ];u8 flags;u8 rev; -u16 priv_data_len; +__be16 priv_data_len;u8 priv_data[0 ]; };
@@ -63,9 +63,9 @@ struct ietf_mpa_frame {struct nes_v4_quad {u32 rsvd0; -u32 DstIpAdrIndex;/* Only most significant 5 bits are valid */ -u32 SrcIpadr; -u16 TcpPorts[2 ];/* src is low, dest is high */ +__le32 DstIpAdrIndex;/* Only most significant 5 bits are valid */ +__be32 SrcIpadr; +__be16 TcpPorts[2 ];/* src is low, dest is high */ };struct nes_cm_node ;
@@ -101,7 +101,7 @@ enum option_numbers {struct option_mss {u8 optionnum;u8 length; -u16 mss; +__be16 mss; };struct option_windowscale {
drivers/infiniband/hw/nes/nes_context.h +37 -37
87 lines not on this page · raw message
drivers/infiniband/hw/nes/nes_hw.c +2 -2
@@ -2199,7 +2199,7 @@ void nes_nic_ce_handler(struct nes_device *nesdev, struct nes_hw_nic_cq *cq)struct nes_hw_nic_sq_wqe *nic_sqe;struct sk_buff *skb;struct sk_buff *rx_skb; -u16 *wqe_fragment_length; +__le16 *wqe_fragment_length;unsigned long flags;u32 head;u32 cq_size;
@@ -2227,7 +2227,7 @@ void nes_nic_ce_handler(struct nes_device *nesdev, struct nes_hw_nic_cq *cq) wqe_fragment_index =1 ; nic_sqe = &nesnic->sq_vbase[nesnic->sq_tail]; skb = nesnic->tx_skb[nesnic->sq_tail]; - wqe_fragment_length = (u16 *)&nic_sqe->wqe_words[NES_NIC_SQ_WQE_LENGTH_0_TAG_IDX ]; + wqe_fragment_length = (__le16 *)&nic_sqe->wqe_words[NES_NIC_SQ_WQE_LENGTH_0_TAG_IDX ];/* bump past the vlan tag */ wqe_fragment_length++;if (le16_to_cpu (wqe_fragment_length[wqe_fragment_index]) !=0 ) {
drivers/infiniband/hw/nes/nes_hw.h +1 -1
@@ -1106,7 +1106,7 @@ struct nes_adapter { };struct nes_pbl { -u64 *pbl_vbase; +__le64 *pbl_vbase;dma_addr_t pbl_pbase;struct page *page;unsigned long user_base;
drivers/infiniband/hw/nes/nes_nic.c +6 -42
89 lines not on this page · raw message
drivers/infiniband/hw/nes/nes_utils.c +2 -2
@@ -610,8 +610,8 @@ void nes_post_cqp_request(struct nes_device *nesdev,nes_debug (NES_DBG_CQP ,"CQP request %p (opcode 0x%02X), line 1 = 0x%08X" " put on the pending queue.\n" , cqp_request, - cqp_request->cqp_wqe.wqe_words[NES_CQP_WQE_OPCODE_IDX ]&0x3f , - cqp_request->cqp_wqe.wqe_words[NES_CQP_WQE_ID_IDX ]); +le32_to_cpu (cqp_request->cqp_wqe.wqe_words[NES_CQP_WQE_OPCODE_IDX ]) &0x3f , +le32_to_cpu (cqp_request->cqp_wqe.wqe_words[NES_CQP_WQE_ID_IDX ]));list_add_tail (&cqp_request->list, &nesdev->cqp_pending_reqs); }
drivers/infiniband/hw/nes/nes_verbs.c +14 -16
@@ -298,10 +298,8 @@ static int nes_bind_mw(struct ib_qp *ibqp, struct ib_mw *ibmw,cpu_to_le32 ((u32)((u64temp)>>32 )); wqe->wqe_words[NES_IWARP_SQ_WQE_COMP_CTX_HIGH_IDX ] =cpu_to_le32 ((u32)(upper_32_bits ((unsigned long )nesqp))); - wqe->wqe_words[NES_IWARP_SQ_WQE_COMP_CTX_LOW_IDX ] = (u32)((unsigned long )nesqp); - wqe->wqe_words[NES_IWARP_SQ_WQE_COMP_CTX_LOW_IDX ] |= head; - wqe->wqe_words[NES_IWARP_SQ_WQE_COMP_CTX_LOW_IDX ] = -cpu_to_le32 (wqe->wqe_words[NES_IWARP_SQ_WQE_COMP_CTX_LOW_IDX ]); + wqe->wqe_words[NES_IWARP_SQ_WQE_COMP_CTX_LOW_IDX ] = +cpu_to_le32 (head | (u32 )(unsigned long ) nesqp); wqe_misc =NES_IWARP_SQ_OP_BIND ; wqe_misc |=NES_IWARP_SQ_WQE_LOCAL_FENCE ;
@@ -1072,9 +1070,9 @@ static int nes_setup_virt_qp(struct nes_qp *nesqp, struct nes_pbl *nespbl, {unsigned long flags;void *mem; -u64 *pbl =NULL ; -u64 *tpbl; -u64 *pblbuffer; +__le64 *pbl =NULL ; +__le64 *tpbl; +__le64 *pblbuffer;struct nes_device *nesdev = nesvnic->nesdev;struct nes_adapter *nesadapter = nesdev->nesadapter;u32 pbl_entries;
@@ -1091,7 +1089,7 @@ static int nes_setup_virt_qp(struct nes_qp *nesqp, struct nes_pbl *nespbl,/* the first pbl to be fro the rq_vbase... */ rq_pbl_entries = (rq_size *sizeof (struct nes_hw_qp_wqe )) >>12 ; sq_pbl_entries = (sq_size *sizeof (struct nes_hw_qp_wqe )) >>12 ; - nesqp->hwqp.sq_pbase = (le32_to_cpu (((u32 *)pbl)[0 ]) ) | ((u64)((le32_to_cpu (((u32 *)pbl)[1 ]))) <<32 ); + nesqp->hwqp.sq_pbase = (le32_to_cpu (((__le32 *)pbl)[0 ]) ) | ((u64)((le32_to_cpu (((__le32 *)pbl)[1 ]))) <<32 );if (!nespbl->page) {nes_debug (NES_DBG_QP ,"QP nespbl->page is NULL \n" );kfree (nespbl);
@@ -1109,7 +1107,7 @@ static int nes_setup_virt_qp(struct nes_qp *nesqp, struct nes_pbl *nespbl,/* Now to get to sq.. we need to calculate how many */ /* PBL entries were used by the rq.. */ pbl += sq_pbl_entries; - nesqp->hwqp.rq_pbase = (le32_to_cpu (((u32 *)pbl)[0 ]) ) | ((u64)((le32_to_cpu (((u32 *)pbl)[1 ]))) <<32 ); + nesqp->hwqp.rq_pbase = (le32_to_cpu (((__le32 *)pbl)[0 ]) ) | ((u64)((le32_to_cpu (((__le32 *)pbl)[1 ]))) <<32 );/* nesqp->hwqp.rq_vbase = bus_to_virt(*pbl); */ /*nesqp->hwqp.rq_vbase = phys_to_virt(*pbl); */
@@ -2405,7 +2403,7 @@ static struct ib_mr *nes_reg_user_mr(struct ib_pd *pd, u64 start, u64 length, u64 virt, int acc, struct ib_udata *udata) { u64 iova_start; -u64 *pbl; +__le64 *pbl;u64 region_length;dma_addr_t last_dma_addr =0 ;dma_addr_t first_dma_addr =0 ;
@@ -2729,15 +2727,15 @@ static struct ib_mr *nes_reg_user_mr(struct ib_pd *pd, u64 start, u64 length, chunk_pages += (sg_dma_len (&chunk->page_list[nmap_index]) & (4096 -1 )) ?1 :0 ; nespbl->page =sg_page (&chunk->page_list[0 ]);for (page_index=0 ; page_index<chunk_pages; page_index++) { - ((u32 *)pbl)[0 ] =cpu_to_le32 ((u32) + ((__le32 *)pbl)[0 ] =cpu_to_le32 ((u32) (sg_dma_address (&chunk->page_list[nmap_index])+ (page_index*4096 ))); - ((u32 *)pbl)[1 ] =cpu_to_le32 (((u64) + ((__le32 *)pbl)[1 ] =cpu_to_le32 (((u64) (sg_dma_address (&chunk->page_list[nmap_index])+ (page_index*4096 )))>>32 );nes_debug (NES_DBG_MR ,"pbl=%p, *pbl=0x%016llx, 0x%08x%08x\n" , pbl, (unsigned long long )*pbl, -le32_to_cpu (((u32 *)pbl)[1 ]),le32_to_cpu (((u32 *)pbl)[0 ])); +le32_to_cpu (((__le32 *)pbl)[1 ]),le32_to_cpu (((__le32 *)pbl)[0 ])); pbl++; } }
@@ -3730,10 +3728,10 @@ static int nes_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *entry)/* Working on a SQ Completion*/ wq_tail = wqe_index; nesqp->hwqp.sq_tail = (wqe_index+1 )&(nesqp->hwqp.sq_size -1 ); - wrid = (((u64)(cpu_to_le32 ((u32 )nesqp->hwqp.sq_vbase[wq_tail]. + wrid = (((u64)(le32_to_cpu (nesqp->hwqp.sq_vbase[wq_tail]. wqe_words[NES_IWARP_SQ_WQE_COMP_SCRATCH_HIGH_IDX ]))) <<32 ) | - ((u64)(cpu_to_le32 ((u32 )nesqp->hwqp.sq_vbase[wq_tail]. - wqe_words[NES_IWARP_SQ_WQE_COMP_SCRATCH_LOW_IDX ]))); +le32_to_cpu (nesqp->hwqp.sq_vbase[wq_tail]. + wqe_words[NES_IWARP_SQ_WQE_COMP_SCRATCH_LOW_IDX ]); entry->byte_len =le32_to_cpu (nesqp->hwqp.sq_vbase[wq_tail]. wqe_words[NES_IWARP_SQ_WQE_TOTAL_PAYLOAD_IDX ]);
drivers/infiniband/hw/nes/nes_verbs.h +3 -3
@@ -77,8 +77,8 @@ struct nes_mr { };struct nes_hw_pb { -u32 pa_low; -u32 pa_high; +__le32 pa_low; +__le32 pa_high; };struct nes_vpbl {
@@ -139,7 +139,7 @@ struct nes_qp {struct work_struct ae_work;enum ib_qp_state ibqp_state;u32 iwarp_state; -u32 hte_index; +__be32 hte_index;u32 last_aeq;u32 qp_mem_size;atomic_t refcount;
Quoted 29 lines
From: Roland Dreier [mailto:[email protected]]
To: Christoph Hellwig; Glenn StreiffRather than arguing over whether we have to have sparse clean code, I decided to annotate the code myself. Here's a patch that fixes most of the sparse warnings in the nes driver. There's still some stuff that actually looks buggy, like the way
hte_indexstuff is handled. You initializehte_index_maskas:hte_index_mask = ((u32)1 << ((u32temp & 0x001f)+1))-1; nesadapter->hte_index_mask = hte_index_mask;but then compute
hte_indexstuff with:nesqp->hte_index = cpu_to_be32( crc32c(~0, (void *)&nes_quad,
sizeof(nes_quad)) ^0xffffffff);and then do:
nesqp->hte_index &= nesadapter->hte_index_mask;which seems odd to say the least (
hte_indexis big-endian,hte_index_maskis cpu-endian).And also, there's code with the loc_addr/rem_addr etc that seem very confused. For example
Quoted 7 lines
cm_info->loc_addr = htonl(cm_info->loc_addr); cm_info->rem_addr = htonl(cm_info->rem_addr); cm_info->loc_port = htons(cm_info->loc_port); cm_info->rem_port = htons(cm_info->rem_port);which is obviously impossible to annotate correctly, and I couldn't keep track of the endianness stuff elsewhere.
Thanks for the additional review and patch. I take your point.
The part is little endian and the driver is functional for little and big endian platforms. There may have been some expedience with the declarations there. I think it can be improved. Let me take it up with the person who wrote that code.
Also, I want everyone to understand that my skill set is weighted more towards build/install/config. And I guess I'll be patch wrangling as well. So I'll rely on input from my developers for issues that drill down or I'll have them post directly. I respect the work you guys do.
For now, let me get some qa cycles with your patch across x86_64 and power (and probably a couple others).
Regards,
Glenn
Quoted 7 lines
Anyway this is what I have in case the promised cleanups don't turn up in time...
Signed-off-by Roland Dreier <[email protected]>
Quoted 4 lines · Roland Dreier
Anyway, here are all the pending things that I'm aware of. As usual, if something isn't already in my tree and isn't listed below, I probably missed it or dropped it by mistake. Please remind me again in that case.
Are there any plans to merge the SDP (Sockets Direct Protocol) implementation ?
Bart.
Quoted 6 lines
Quoted 4 lines
Anyway, here are all the pending things that I'm aware of. As usual, if something isn't already in my tree and isn't listed below, I probably missed it or dropped it by mistake. Please remind me again in that case.
Are there any plans to merge the SDP (Sockets Direct Protocol) implementation ?
No. To my knowledge, it has never been proposed for merging or been posted for review.