ipath driver - for inclusion in 2.6.17
PATCH 1/1 ipath - misc infiniband code, part 2
In reply to Bryan O'SullivanManagement datagram support, queue pairs, and reliable and unreliable connections.
drivers/infiniband/hw/ipath/ipath_mad.c new file +1350 -0
@@ -0,0 +1,1350 @@ +/* +* Copyright (c) 2005, 2006 PathScale, Inc. All rights reserved. +* +* This software is available to you under a choice of one of two +* licenses. You may choose to be licensed under the terms of the GNU +* General Public License (GPL) Version 2, available from the file +* COPYING in the main directory of this source tree, or the +* OpenIB.org BSD license below: +* +* Redistribution and use in source and binary forms, with or +* without modification, are permitted provided that the following +* conditions are met: +* +* - Redistributions of source code must retain the above +* copyright notice, this list of conditions and the following +* disclaimer. +* +* - Redistributions in binary form must reproduce the above +* copyright notice, this list of conditions and the following +* disclaimer in the documentation and/or other materials +* provided with the distribution. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS +* BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN +* ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +* SOFTWARE. +*/ + +#include <rdma/ib_smi.h> + +#include "ipath_kernel.h" +#include "ipath_verbs.h" +#include "ips_common.h" + +#define IB_SMP_UNSUP_VERSION __constant_htons(0x0004) +#define IB_SMP_UNSUP_METHOD __constant_htons(0x0008) +#define IB_SMP_UNSUP_METH_ATTR __constant_htons(0x000C) +#define IB_SMP_INVALID_FIELD __constant_htons(0x001C) + +static int reply (struct ib_smp *smp) +{ +/* +* The verbs framework will handle the directed/LID route +* packet changes. +*/ + smp->method =IB_MGMT_METHOD_GET_RESP ; +if (smp->mgmt_class ==IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE ) + smp->status |=IB_SMP_DIRECTION ; +return IB_MAD_RESULT_SUCCESS |IB_MAD_RESULT_REPLY ; +} + +static int recv_subn_get_nodedescription (struct ib_smp *smp, +struct ib_device *ibdev) +{ +if (smp->attr_mod) + smp->status |=IB_SMP_INVALID_FIELD ; + +strncpy (smp->data, ibdev->node_desc,sizeof (smp->data)); + +return reply (smp); +} + +struct nodeinfo { +u8 base_version; +u8 class_version; +u8 node_type; +u8 num_ports; +__be64 sys_guid; +__be64 node_guid; +__be64 port_guid; +__be16 partition_cap; +__be16 device_id; +__be32 revision; +u8 local_port_num; +u8 vendor_id[3 ]; +} __attribute__ ((packed)); + +static int recv_subn_get_nodeinfo (struct ib_smp *smp, +struct ib_device *ibdev,u8 port) +{ +struct nodeinfo *nip = (struct nodeinfo *)&smp->data; +struct ipath_devdata *dd =to_idev (ibdev)->dd; +u32 vendor, boardid, majrev, minrev; + +if (smp->attr_mod) + smp->status |=IB_SMP_INVALID_FIELD ; + + nip->base_version =1 ; + nip->class_version =1 ; + nip->node_type =1 ;/* channel adapter */ +/* +* XXX The num_ports value will need a layer function to get +* the value if we ever have more than one IB port on a chip. +* We will also need to get the GUID for the port. +*/ + nip->num_ports = ibdev->phys_port_cnt; +/* This is already in network order */ + nip->sys_guid =to_idev (ibdev)->sys_image_guid; + nip->node_guid =ipath_layer_get_guid (dd); + nip->port_guid = nip->sys_guid; + nip->partition_cap =cpu_to_be16 (ipath_layer_get_npkeys (dd)); + nip->device_id =cpu_to_be16 (ipath_layer_get_deviceid (dd)); +ipath_layer_query_device (dd, &vendor, &boardid, &majrev, &minrev); + nip->revision =cpu_to_be32 ((majrev <<16 ) | minrev); + nip->local_port_num = port; + nip->vendor_id[0 ] =0 ; + nip->vendor_id[1 ] = vendor >>8 ; + nip->vendor_id[2 ] = vendor; + +return reply (smp); +} + +static int recv_subn_get_guidinfo (struct ib_smp *smp, +struct ib_device *ibdev) +{ +u32 startgx =8 *be32_to_cpu (smp->attr_mod); +u64 *p = (u64 *) smp->data; + +/* 32 blocks of 8 64-bit GUIDs per block */ + +memset (smp->data,0 ,sizeof (smp->data)); + +/* +* We only support one GUID for now. If this changes, the +* portinfo.guid_cap field needs to be updated too. +*/ +if (startgx ==0 ) +/* The first is a copy of the read-only HW GUID. */ + *p =ipath_layer_get_guid (to_idev (ibdev)->dd); +else + smp->status |=IB_SMP_INVALID_FIELD ; + +return reply (smp); +} + +struct port_info { +__be64 mkey; +__be64 gid_prefix; +__be16 lid; +__be16 sm_lid; +__be32 cap_mask; +__be16 diag_code; +__be16 mkey_lease_period; +u8 local_port_num; +u8 link_width_enabled; +u8 link_width_supported; +u8 link_width_active; +u8 linkspeed_portstate;/* 4 bits, 4 bits */ +u8 portphysstate_linkdown;/* 4 bits, 4 bits */ +u8 mkeyprot_resv_lmc;/* 2 bits, 3, 3 */ +u8 linkspeedactive_enabled;/* 4 bits, 4 bits */ +u8 neighbormtu_mastersmsl;/* 4 bits, 4 bits */ +u8 vlcap_inittype;/* 4 bits, 4 bits */ +u8 vl_high_limit; +u8 vl_arb_high_cap; +u8 vl_arb_low_cap; +u8 inittypereply_mtucap;/* 4 bits, 4 bits */ +u8 vlstallcnt_hoqlife;/* 3 bits, 5 bits */ +u8 operationalvl_pei_peo_fpi_fpo;/* 4 bits, 1, 1, 1, 1 */ +__be16 mkey_violations; +__be16 pkey_violations; +__be16 qkey_violations; +u8 guid_cap; +u8 clientrereg_resv_subnetto;/* 1 bit, 2 bits, 5 */ +u8 resv_resptimevalue;/* 3 bits, 5 bits */ +u8 localphyerrors_overrunerrors;/* 4 bits, 4 bits */ +__be16 max_credit_hint; +u8 resv; +u8 link_roundtrip_latency[3 ]; +} __attribute__ ((packed)); + +static int recv_subn_get_portinfo (struct ib_smp *smp, +struct ib_device *ibdev,u8 port) +{ +struct ipath_ibdev *dev; +struct port_info *pip = (struct port_info *)smp->data; +u16 lid; +u8 ibcstat; +u8 mtu; +int ret; + +if (be32_to_cpu (smp->attr_mod) > ibdev->phys_port_cnt) { + smp->status |=IB_SMP_INVALID_FIELD ; + ret =reply (smp); + goto bail; + } + + dev =to_idev (ibdev); + +/* Clear all fields. Only set the non-zero fields. */ +memset (smp->data,0 ,sizeof (smp->data)); + +/* Only return the mkey if the protection field allows it. */ +if (smp->method ==IB_MGMT_METHOD_SET || dev->mkey == smp->mkey || + (dev->mkeyprot_resv_lmc >>6 ) ==0 ) + pip->mkey = dev->mkey; + pip->gid_prefix = dev->gid_prefix; + lid =ipath_layer_get_lid (dev->dd); + pip->lid = lid ?cpu_to_be16 (lid) :IB_LID_PERMISSIVE ; + pip->sm_lid =cpu_to_be16 (dev->sm_lid); + pip->cap_mask =cpu_to_be32 (dev->port_cap_flags); +/* pip->diag_code; */ + pip->mkey_lease_period =cpu_to_be16 (dev->mkey_lease_period); + pip->local_port_num = port; + pip->link_width_enabled = dev->link_width_enabled; + pip->link_width_supported =3 ;/* 1x or 4x */ + pip->link_width_active =2 ;/* 4x */ + pip->linkspeed_portstate =0x10 ;/* 2.5Gbps */ + ibcstat =ipath_layer_get_lastibcstat (dev->dd); + pip->linkspeed_portstate |= ((ibcstat >>4 ) &0x3 ) +1 ; + pip->portphysstate_linkdown = + (ipath_cvt_physportstate[ibcstat &0xf ] <<4 ) | + (ipath_layer_get_linkdowndefaultstate (dev->dd) ?1 :2 ); + pip->mkeyprot_resv_lmc = dev->mkeyprot_resv_lmc; + pip->linkspeedactive_enabled =0x11 ;/* 2.5Gbps, 2.5Gbps */ +switch (ipath_layer_get_ibmtu (dev->dd)) { +case 4096 : + mtu =IB_MTU_4096 ; +break ; +case 2048 : + mtu =IB_MTU_2048 ; +break ; +case 1024 : + mtu =IB_MTU_1024 ; +break ; +case 512 : + mtu =IB_MTU_512 ; +break ; +case 256 : + mtu =IB_MTU_256 ; +break ; +default :/* oops, something is wrong */ + mtu =IB_MTU_2048 ; +break ; + } + pip->neighbormtu_mastersmsl = (mtu <<4 ) | dev->sm_sl; + pip->vlcap_inittype =0x10 ;/* VLCap = VL0, InitType = 0 */ + pip->vl_high_limit = dev->vl_high_limit; +/* pip->vl_arb_high_cap; // only one VL */ +/* pip->vl_arb_low_cap; // only one VL */ +/* InitTypeReply = 0 */ + pip->inittypereply_mtucap =IB_MTU_4096 ; +// HCAs ignore VLStallCount and HOQLife +/* pip->vlstallcnt_hoqlife; */ + pip->operationalvl_pei_peo_fpi_fpo =0x10 ;/* OVLs = 1 */ + pip->mkey_violations =cpu_to_be16 (dev->mkey_violations); +/* P_KeyViolations are counted by hardware. */ + pip->pkey_violations = +cpu_to_be16 ((ipath_layer_get_cr_errpkey (dev->dd) - + dev->n_pkey_violations) &0xFFFF ); + pip->qkey_violations =cpu_to_be16 (dev->qkey_violations); +/* Only the hardware GUID is supported for now */ + pip->guid_cap =1 ; + pip->clientrereg_resv_subnetto = dev->subnet_timeout; +/* 32.768 usec. response time (guessing) */ + pip->resv_resptimevalue =3 ; + pip->localphyerrors_overrunerrors = + (ipath_layer_get_phyerrthreshold (dev->dd) <<4 ) | +ipath_layer_get_overrunthreshold (dev->dd); +/* pip->max_credit_hint; */ +/* pip->link_roundtrip_latency[3]; */ + + ret =reply (smp); + +bail: +return ret; +} + +static int recv_subn_get_pkeytable (struct ib_smp *smp, +struct ib_device *ibdev) +{ +u32 startpx =32 * (be32_to_cpu (smp->attr_mod) &0xffff ); +u16 *p = (u16 *) smp->data; + +/* 64 blocks of 32 16-bit P_Key entries */ + +memset (smp->data,0 ,sizeof (smp->data)); +if (startpx ==0 ) { +struct ipath_ibdev *dev =to_idev (ibdev); +unsigned i, n =ipath_layer_get_npkeys (dev->dd); + +ipath_layer_get_pkeys (dev->dd, p); + +for (i =0 ; i < n; i++) + p[i] =cpu_to_be16 (p[i]); + }else + smp->status |=IB_SMP_INVALID_FIELD ; + +return reply (smp); +} + +static int recv_subn_set_guidinfo (struct ib_smp *smp, +struct ib_device *ibdev) +{ +/* The only GUID we support is the first read-only entry. */ +return recv_subn_get_guidinfo (smp, ibdev); +} + +/** +* recv_subn_set_portinfo - set port information +* @smp: the incoming SM packet +* @ibdev: the infiniband device +* @port: the port on the device +* +* Set Portinfo (see ch. 14.2.5.6). +*/ +static int recv_subn_set_portinfo (struct ib_smp *smp, +struct ib_device *ibdev,u8 port) +{ +struct port_info *pip = (struct port_info *)smp->data; +struct ib_event event; +struct ipath_ibdev *dev; +u32 flags; +char clientrereg =0 ; +u16 lid, smlid; +u8 lwe; +u8 lse; +u8 state; +u16 lstate; +u32 mtu; +int ret; + +if (be32_to_cpu (smp->attr_mod) > ibdev->phys_port_cnt) + goto err; + + dev =to_idev (ibdev); + event.device = ibdev; + event.element.port_num = port; + + dev->mkey = pip->mkey; + dev->gid_prefix = pip->gid_prefix; + dev->mkey_lease_period =be16_to_cpu (pip->mkey_lease_period); + + lid =be16_to_cpu (pip->lid); +if (lid !=ipath_layer_get_lid (dev->dd)) { +/* Must be a valid unicast LID address. */ +if (lid ==0 || lid >=IPS_MULTICAST_LID_BASE ) + goto err; +ipath_set_sps_lid (dev->dd, lid, pip->mkeyprot_resv_lmc &7 ); + event.event =IB_EVENT_LID_CHANGE ; +ib_dispatch_event (&event); + } + + smlid =be16_to_cpu (pip->sm_lid); +if (smlid != dev->sm_lid) { +/* Must be a valid unicast LID address. */ +if (smlid ==0 || smlid >=IPS_MULTICAST_LID_BASE ) + goto err; + dev->sm_lid = smlid; + event.event =IB_EVENT_SM_CHANGE ; +ib_dispatch_event (&event); + } + +/* Only 4x supported but allow 1x or 4x to be set (see 14.2.6.6). */ + lwe = pip->link_width_enabled; +if ((lwe >=4 && lwe <=8 ) || (lwe >=0xC && lwe <=0xFE )) + goto err; +if (lwe ==0xFF ) + dev->link_width_enabled =3 ;/* 1x or 4x */ +else if (lwe) + dev->link_width_enabled = lwe; + +/* Only 2.5 Gbs supported. */ + lse = pip->linkspeedactive_enabled &0xF ; +if (lse >=2 && lse <=0xE ) + goto err; + +/* Set link down default state. */ +switch (pip->portphysstate_linkdown &0xF ) { +case 0 :/* NOP */ +break ; +case 1 :/* SLEEP */ +if (ipath_layer_set_linkdowndefaultstate (dev->dd,1 )) + goto err; +break ; +case 2 :/* POLL */ +if (ipath_layer_set_linkdowndefaultstate (dev->dd,0 )) + goto err; +break ; +default : + goto err; + } + + dev->mkeyprot_resv_lmc = pip->mkeyprot_resv_lmc; + dev->vl_high_limit = pip->vl_high_limit; + +switch ((pip->neighbormtu_mastersmsl >>4 ) &0xF ) { +case IB_MTU_256 : + mtu =256 ; +break ; +case IB_MTU_512 : + mtu =512 ; +break ; +case IB_MTU_1024 : + mtu =1024 ; +break ; +case IB_MTU_2048 : + mtu =2048 ; +break ; +case IB_MTU_4096 : + mtu =4096 ; +break ; +default : +/* XXX We have already partially updated our state! */ + goto err; + } +ipath_layer_set_mtu (dev->dd, mtu); + + dev->sm_sl = pip->neighbormtu_mastersmsl &0xF ; + +/* We only support VL0 */ +if (((pip->operationalvl_pei_peo_fpi_fpo >>4 ) &0xF ) >1 ) + goto err; + +if (pip->mkey_violations ==0 ) + dev->mkey_violations =0 ; + +/* +* Hardware counter can't be reset so snapshot and subtract +* later. +*/ +if (pip->pkey_violations ==0 ) + dev->n_pkey_violations = +ipath_layer_get_cr_errpkey (dev->dd); + +if (pip->qkey_violations ==0 ) + dev->qkey_violations =0 ; + +if (ipath_layer_set_phyerrthreshold ( + dev->dd, + (pip->localphyerrors_overrunerrors >>4 ) &0xF )) + goto err; + +if (ipath_layer_set_overrunthreshold ( + dev->dd, + (pip->localphyerrors_overrunerrors &0xF ))) + goto err; + + dev->subnet_timeout = pip->clientrereg_resv_subnetto &0x1F ; + +if (pip->clientrereg_resv_subnetto &0x80 ) { + clientrereg =1 ; + event.event =IB_EVENT_LID_CHANGE ; +ib_dispatch_event (&event); + } + +/* +* Do the port state change now that the other link parameters +* have been set. +* Changing the port physical state only makes sense if the link +* is down or is being set to down. +*/ + state = pip->linkspeed_portstate &0xF ; + flags =ipath_layer_get_flags (dev->dd); + lstate = (pip->portphysstate_linkdown >>4 ) &0xF ; +if (lstate && !(state ==IB_PORT_DOWN || state ==IB_PORT_NOP )) + goto err; + +/* +* Only state changes of DOWN, ARM, and ACTIVE are valid +* and must be in the correct state to take effect (see 7.2.6). +*/ +switch (state) { +case IB_PORT_NOP : +if (lstate ==0 ) +break ; +/* FALLTHROUGH */ +case IB_PORT_DOWN : +if (lstate ==0 ) +if (ipath_layer_get_linkdowndefaultstate (dev->dd)) + lstate =IPATH_IB_LINKDOWN_SLEEP ; +else + lstate =IPATH_IB_LINKDOWN ; +else if (lstate ==1 ) + lstate =IPATH_IB_LINKDOWN_SLEEP ; +else if (lstate ==2 ) + lstate =IPATH_IB_LINKDOWN ; +else if (lstate ==3 ) + lstate =IPATH_IB_LINKDOWN_DISABLE ; +else + goto err; +ipath_layer_set_linkstate (dev->dd, lstate); +if (flags &IPATH_LINKACTIVE ) { + event.event =IB_EVENT_PORT_ERR ; +ib_dispatch_event (&event); + } +break ; +case IB_PORT_ARMED : +if (!(flags & (IPATH_LINKINIT |IPATH_LINKACTIVE ))) +break ; +ipath_layer_set_linkstate (dev->dd,IPATH_IB_LINKARM ); +if (flags &IPATH_LINKACTIVE ) { + event.event =IB_EVENT_PORT_ERR ; +ib_dispatch_event (&event); + } +break ; +case IB_PORT_ACTIVE : +if (!(flags &IPATH_LINKARMED )) +break ; +ipath_layer_set_linkstate (dev->dd,IPATH_IB_LINKACTIVE ); + event.event =IB_EVENT_PORT_ACTIVE ; +ib_dispatch_event (&event); +break ; +default : +/* XXX We have already partially updated our state! */ + goto err; + } + + ret =recv_subn_get_portinfo (smp, ibdev, port); + +if (clientrereg) + pip->clientrereg_resv_subnetto |=0x80 ; + + goto done; + +err: + smp->status |=IB_SMP_INVALID_FIELD ; + ret =recv_subn_get_portinfo (smp, ibdev, port); + +done: +return ret; +} + +static int recv_subn_set_pkeytable (struct ib_smp *smp, +struct ib_device *ibdev) +{ +u32 startpx =32 * (be32_to_cpu (smp->attr_mod) &0xffff ); +u16 *p = (u16 *) smp->data; +struct ipath_ibdev *dev =to_idev (ibdev); +unsigned i, n =ipath_layer_get_npkeys (dev->dd); + +for (i =0 ; i < n; i++) + p[i] =be16_to_cpu (p[i]); + +if (startpx !=0 || +ipath_layer_set_pkeys (dev->dd, p) !=0 ) + smp->status |=IB_SMP_INVALID_FIELD ; + +return recv_subn_get_pkeytable (smp, ibdev); +} + +#define IB_PMA_CLASS_PORT_INFO __constant_htons(0x0001) +#define IB_PMA_PORT_SAMPLES_CONTROL __constant_htons(0x0010) +#define IB_PMA_PORT_SAMPLES_RESULT __constant_htons(0x0011) +#define IB_PMA_PORT_COUNTERS __constant_htons(0x0012) +#define IB_PMA_PORT_COUNTERS_EXT __constant_htons(0x001D) +#define IB_PMA_PORT_SAMPLES_RESULT_EXT __constant_htons(0x001E) + +struct ib_perf { +u8 base_version; +u8 mgmt_class; +u8 class_version; +u8 method; +__be16 status; +__be16 unused; +__be64 tid; +__be16 attr_id; +__be16 resv; +__be32 attr_mod; +u8 reserved[40 ]; +u8 data[192 ]; +} __attribute__ ((packed)); + +struct ib_pma_classportinfo { +u8 base_version; +u8 class_version; +__be16 cap_mask; +u8 reserved[3 ]; +u8 resp_time_value;/* only lower 5 bits */ +union ib_gid redirect_gid; +__be32 redirect_tc_sl_fl;/* 8, 4, 20 bits respectively */ +__be16 redirect_lid; +__be16 redirect_pkey; +__be32 redirect_qp;/* only lower 24 bits */ +__be32 redirect_qkey; +union ib_gid trap_gid; +__be32 trap_tc_sl_fl;/* 8, 4, 20 bits respectively */ +__be16 trap_lid; +__be16 trap_pkey; +__be32 trap_hl_qp;/* 8, 24 bits respectively */ +__be32 trap_qkey; +} __attribute__ ((packed)); + +struct ib_pma_portsamplescontrol { +u8 opcode; +u8 port_select; +u8 tick; +u8 counter_width;/* only lower 3 bits */ +__be32 counter_mask0_9;/* 2, 10 * 3, bits */ +__be16 counter_mask10_14;/* 1, 5 * 3, bits */ +u8 sample_mechanisms; +u8 sample_status;/* only lower 2 bits */ +__be64 option_mask; +__be64 vendor_mask; +__be32 sample_start; +__be32 sample_interval; +__be16 tag; +__be16 counter_select[15 ]; +} __attribute__ ((packed)); + +struct ib_pma_portsamplesresult { +__be16 tag; +__be16 sample_status;/* only lower 2 bits */ +__be32 counter[15 ]; +} __attribute__ ((packed)); + +struct ib_pma_portsamplesresult_ext { +__be16 tag; +__be16 sample_status;/* only lower 2 bits */ +__be32 extended_width;/* only upper 2 bits */ +__be64 counter[15 ]; +} __attribute__ ((packed)); + +struct ib_pma_portcounters { +u8 reserved; +u8 port_select; +__be16 counter_select; +__be16 symbol_error_counter; +u8 link_error_recovery_counter; +u8 link_downed_counter; +__be16 port_rcv_errors; +__be16 port_rcv_remphys_errors; +__be16 port_rcv_switch_relay_errors; +__be16 port_xmit_discards; +u8 port_xmit_constraint_errors; +u8 port_rcv_constraint_errors; +u8 reserved1; +u8 lli_ebor_errors;/* 4, 4, bits */ +__be16 reserved2; +__be16 vl15_dropped; +__be32 port_xmit_data; +__be32 port_rcv_data; +__be32 port_xmit_packets; +__be32 port_rcv_packets; +} __attribute__ ((packed)); + +#define IB_PMA_SEL_SYMBOL_ERROR __constant_htons(0x0001) +#define IB_PMA_SEL_LINK_ERROR_RECOVERY __constant_htons(0x0002) +#define IB_PMA_SEL_LINK_DOWNED __constant_htons(0x0004) +#define IB_PMA_SEL_PORT_RCV_ERRORS __constant_htons(0x0008) +#define IB_PMA_SEL_PORT_RCV_REMPHYS_ERRORS __constant_htons(0x0010) +#define IB_PMA_SEL_PORT_XMIT_DISCARDS __constant_htons(0x0040) +#define IB_PMA_SEL_PORT_XMIT_DATA __constant_htons(0x1000) +#define IB_PMA_SEL_PORT_RCV_DATA __constant_htons(0x2000) +#define IB_PMA_SEL_PORT_XMIT_PACKETS __constant_htons(0x4000) +#define IB_PMA_SEL_PORT_RCV_PACKETS __constant_htons(0x8000) + +struct ib_pma_portcounters_ext { +u8 reserved; +u8 port_select; +__be16 counter_select; +__be32 reserved1; +__be64 port_xmit_data; +__be64 port_rcv_data; +__be64 port_xmit_packets; +__be64 port_rcv_packets; +__be64 port_unicast_xmit_packets; +__be64 port_unicast_rcv_packets; +__be64 port_multicast_xmit_packets; +__be64 port_multicast_rcv_packets; +} __attribute__ ((packed)); + +#define IB_PMA_SELX_PORT_XMIT_DATA __constant_htons(0x0001) +#define IB_PMA_SELX_PORT_RCV_DATA __constant_htons(0x0002) +#define IB_PMA_SELX_PORT_XMIT_PACKETS __constant_htons(0x0004) +#define IB_PMA_SELX_PORT_RCV_PACKETS __constant_htons(0x0008) +#define IB_PMA_SELX_PORT_UNI_XMIT_PACKETS __constant_htons(0x0010) +#define IB_PMA_SELX_PORT_UNI_RCV_PACKETS __constant_htons(0x0020) +#define IB_PMA_SELX_PORT_MULTI_XMIT_PACKETS __constant_htons(0x0040) +#define IB_PMA_SELX_PORT_MULTI_RCV_PACKETS __constant_htons(0x0080) + +static int recv_pma_get_classportinfo (struct ib_perf *pmp) +{ +struct ib_pma_classportinfo *p = + (struct ib_pma_classportinfo *)pmp->data; + +memset (pmp->data,0 ,sizeof (pmp->data)); + +if (pmp->attr_mod !=0 ) + pmp->status |=IB_SMP_INVALID_FIELD ; + +/* Indicate AllPortSelect is valid (only one port anyway) */ + p->cap_mask =cpu_to_be16 (1 <<8 ); + p->base_version =1 ; + p->class_version =1 ; +/* +* Expected response time is 4.096 usec. * 2^18 == 1.073741824 +* sec. +*/ + p->resp_time_value =18 ; + +return reply ((struct ib_smp *) pmp); +} + +/* +* The PortSamplesControl.CounterMasks field is an array of 3 bit fields +* which specify the N'th counter's capabilities. See ch. 16.1.3.2. +* We support 5 counters which only count the mandatory quantities. +*/ +#define COUNTER_MASK (q, n) (q << ((9 - n) * 3)) +#define COUNTER_MASK0_9 \ + __constant_cpu_to_be32(COUNTER_MASK(1, 0) | \ + COUNTER_MASK(1, 1) | \ + COUNTER_MASK(1, 2) | \ + COUNTER_MASK(1, 3) | \ + COUNTER_MASK(1, 4)) + +static int recv_pma_get_portsamplescontrol (struct ib_perf *pmp, +struct ib_device *ibdev,u8 port) +{ +struct ib_pma_portsamplescontrol *p = + (struct ib_pma_portsamplescontrol *)pmp->data; +struct ipath_ibdev *dev =to_idev (ibdev); +unsigned long flags; +u8 port_select = p->port_select; + +memset (pmp->data,0 ,sizeof (pmp->data)); + + p->port_select = port_select; +if (pmp->attr_mod !=0 || + (port_select != port && port_select !=0xFF )) + pmp->status |=IB_SMP_INVALID_FIELD ; +/* +* Ticks are 10x the link transfer period which for 2.5Gbs is 4 +* nsec. 0 == 4 nsec., 1 == 8 nsec., ..., 255 == 1020 nsec. Sample +* intervals are counted in ticks. Since we use Linux timers, that +* count in jiffies, we can't sample for less than 1000 ticks if HZ +* == 1000 (4000 ticks if HZ is 250). +*/ +/* XXX This is WRONG. */ + p->tick =250 ;/* 1 usec. */ + p->counter_width =4 ;/* 32 bit counters */ + p->counter_mask0_9 =COUNTER_MASK0_9 ; +spin_lock_irqsave (&dev->pending_lock, flags); + p->sample_status = dev->pma_sample_status; + p->sample_start =cpu_to_be32 (dev->pma_sample_start); + p->sample_interval =cpu_to_be32 (dev->pma_sample_interval); + p->tag =cpu_to_be16 (dev->pma_tag); + p->counter_select[0 ] = dev->pma_counter_select[0 ]; + p->counter_select[1 ] = dev->pma_counter_select[1 ]; + p->counter_select[2 ] = dev->pma_counter_select[2 ]; + p->counter_select[3 ] = dev->pma_counter_select[3 ]; + p->counter_select[4 ] = dev->pma_counter_select[4 ]; +spin_unlock_irqrestore (&dev->pending_lock, flags); + +return reply ((struct ib_smp *) pmp); +} + +static int recv_pma_set_portsamplescontrol (struct ib_perf *pmp, +struct ib_device *ibdev,u8 port) +{ +struct ib_pma_portsamplescontrol *p = + (struct ib_pma_portsamplescontrol *)pmp->data; +struct ipath_ibdev *dev =to_idev (ibdev); +unsigned long flags; +u32 start; +int ret; + +if (pmp->attr_mod !=0 || + (p->port_select != port && p->port_select !=0xFF )) { + pmp->status |=IB_SMP_INVALID_FIELD ; + ret =reply ((struct ib_smp *) pmp); + goto bail; + } + + start =be32_to_cpu (p->sample_start); +if (start !=0 ) { +spin_lock_irqsave (&dev->pending_lock, flags); +if (dev->pma_sample_status ==IB_PMA_SAMPLE_STATUS_DONE ) { + dev->pma_sample_status = +IB_PMA_SAMPLE_STATUS_STARTED ; + dev->pma_sample_start = start; + dev->pma_sample_interval = +be32_to_cpu (p->sample_interval); + dev->pma_tag =be16_to_cpu (p->tag); +if (p->counter_select[0 ]) + dev->pma_counter_select[0 ] = + p->counter_select[0 ]; +if (p->counter_select[1 ]) + dev->pma_counter_select[1 ] = + p->counter_select[1 ]; +if (p->counter_select[2 ]) + dev->pma_counter_select[2 ] = + p->counter_select[2 ]; +if (p->counter_select[3 ]) + dev->pma_counter_select[3 ] = + p->counter_select[3 ]; +if (p->counter_select[4 ]) + dev->pma_counter_select[4 ] = + p->counter_select[4 ]; + } +spin_unlock_irqrestore (&dev->pending_lock, flags); + } + ret =recv_pma_get_portsamplescontrol (pmp, ibdev, port); + +bail: +return ret; +} + +static u64 get_counter (struct ipath_ibdev *dev,__be16 sel) +{ +u64 ret; + +switch (sel) { +case IB_PMA_PORT_XMIT_DATA : + ret = dev->ipath_sword; +break ; +case IB_PMA_PORT_RCV_DATA : + ret = dev->ipath_rword; +break ; +case IB_PMA_PORT_XMIT_PKTS : + ret = dev->ipath_spkts; +break ; +case IB_PMA_PORT_RCV_PKTS : + ret = dev->ipath_rpkts; +break ; +case IB_PMA_PORT_XMIT_WAIT : + ret = dev->ipath_xmit_wait; +break ; +default : + ret =0 ; + } + +return ret; +} + +static int recv_pma_get_portsamplesresult (struct ib_perf *pmp, +struct ib_device *ibdev) +{ +struct ib_pma_portsamplesresult *p = + (struct ib_pma_portsamplesresult *)pmp->data; +struct ipath_ibdev *dev =to_idev (ibdev); +int i; + +memset (pmp->data,0 ,sizeof (pmp->data)); + p->tag =cpu_to_be16 (dev->pma_tag); + p->sample_status =cpu_to_be16 (dev->pma_sample_status); +for (i =0 ; i <ARRAY_SIZE (dev->pma_counter_select); i++) + p->counter[i] =cpu_to_be32 ( +get_counter (dev, dev->pma_counter_select[i])); + +return reply ((struct ib_smp *) pmp); +} + +static int recv_pma_get_portsamplesresult_ext (struct ib_perf *pmp, +struct ib_device *ibdev) +{ +struct ib_pma_portsamplesresult_ext *p = + (struct ib_pma_portsamplesresult_ext *)pmp->data; +struct ipath_ibdev *dev =to_idev (ibdev); +int i; + +memset (pmp->data,0 ,sizeof (pmp->data)); + p->tag =cpu_to_be16 (dev->pma_tag); + p->sample_status =cpu_to_be16 (dev->pma_sample_status); +/* 64 bits */ + p->extended_width =__constant_cpu_to_be32 (0x80000000 ); +for (i =0 ; i <ARRAY_SIZE (dev->pma_counter_select); i++) + p->counter[i] =cpu_to_be64 ( +get_counter (dev, dev->pma_counter_select[i])); + +return reply ((struct ib_smp *) pmp); +} + +static int recv_pma_get_portcounters (struct ib_perf *pmp, +struct ib_device *ibdev,u8 port) +{ +struct ib_pma_portcounters *p = (struct ib_pma_portcounters *) + pmp->data; +struct ipath_ibdev *dev =to_idev (ibdev); +struct ipath_layer_counters cntrs; +u8 port_select = p->port_select; + +ipath_layer_get_counters (dev->dd, &cntrs); + +/* Adjust counters for any resets done. */ + cntrs.symbol_error_counter -= dev->n_symbol_error_counter; + cntrs.link_error_recovery_counter -= + dev->n_link_error_recovery_counter; + cntrs.link_downed_counter -= dev->n_link_downed_counter; + cntrs.port_rcv_errors += dev->rcv_errors; + cntrs.port_rcv_errors -= dev->n_port_rcv_errors; + cntrs.port_rcv_remphys_errors -= dev->n_port_rcv_remphys_errors; + cntrs.port_xmit_discards -= dev->n_port_xmit_discards; + cntrs.port_xmit_data -= dev->n_port_xmit_data; + cntrs.port_rcv_data -= dev->n_port_rcv_data; + cntrs.port_xmit_packets -= dev->n_port_xmit_packets; + cntrs.port_rcv_packets -= dev->n_port_rcv_packets; + +memset (pmp->data,0 ,sizeof (pmp->data)); + + p->port_select = port_select; +if (pmp->attr_mod !=0 || + (port_select != port && port_select !=0xFF )) + pmp->status |=IB_SMP_INVALID_FIELD ; + +if (cntrs.symbol_error_counter >0xFFFFUL ) + p->symbol_error_counter =0xFFFF ; +else + p->symbol_error_counter = +cpu_to_be16 ((u16 )cntrs.symbol_error_counter); +if (cntrs.link_error_recovery_counter >0xFFUL ) + p->link_error_recovery_counter =0xFF ; +else + p->link_error_recovery_counter = + (u8 )cntrs.link_error_recovery_counter; +if (cntrs.link_downed_counter >0xFFUL ) + p->link_downed_counter =0xFF ; +else + p->link_downed_counter = (u8 )cntrs.link_downed_counter; +if (cntrs.port_rcv_errors >0xFFFFUL ) + p->port_rcv_errors =0xFFFF ; +else + p->port_rcv_errors = +cpu_to_be16 ((u16 ) cntrs.port_rcv_errors); +if (cntrs.port_rcv_remphys_errors >0xFFFFUL ) + p->port_rcv_remphys_errors =0xFFFF ; +else + p->port_rcv_remphys_errors = +cpu_to_be16 ((u16 )cntrs.port_rcv_remphys_errors); +if (cntrs.port_xmit_discards >0xFFFFUL ) + p->port_xmit_discards =0xFFFF ; +else + p->port_xmit_discards = +cpu_to_be16 ((u16 )cntrs.port_xmit_discards); +if (cntrs.port_xmit_data >0xFFFFFFFFUL ) + p->port_xmit_data =0xFFFFFFFF ; +else + p->port_xmit_data =cpu_to_be32 ((u32 )cntrs.port_xmit_data); +if (cntrs.port_rcv_data >0xFFFFFFFFUL ) + p->port_rcv_data =0xFFFFFFFF ; +else + p->port_rcv_data =cpu_to_be32 ((u32 )cntrs.port_rcv_data); +if (cntrs.port_xmit_packets >0xFFFFFFFFUL ) + p->port_xmit_packets =0xFFFFFFFF ; +else + p->port_xmit_packets = +cpu_to_be32 ((u32 )cntrs.port_xmit_packets); +if (cntrs.port_rcv_packets >0xFFFFFFFFUL ) + p->port_rcv_packets =0xFFFFFFFF ; +else + p->port_rcv_packets = +cpu_to_be32 ((u32 ) cntrs.port_rcv_packets); + +return reply ((struct ib_smp *) pmp); +} + +static int recv_pma_get_portcounters_ext (struct ib_perf *pmp, +struct ib_device *ibdev,u8 port) +{ +struct ib_pma_portcounters_ext *p = + (struct ib_pma_portcounters_ext *)pmp->data; +struct ipath_ibdev *dev =to_idev (ibdev); +u64 swords, rwords, spkts, rpkts, xwait; +u8 port_select = p->port_select; + +ipath_layer_snapshot_counters (dev->dd, &swords, &rwords, &spkts, + &rpkts, &xwait); + +/* Adjust counters for any resets done. */ + swords -= dev->n_port_xmit_data; + rwords -= dev->n_port_rcv_data; + spkts -= dev->n_port_xmit_packets; + rpkts -= dev->n_port_rcv_packets; + +memset (pmp->data,0 ,sizeof (pmp->data)); + + p->port_select = port_select; +if (pmp->attr_mod !=0 || + (port_select != port && port_select !=0xFF )) + pmp->status |=IB_SMP_INVALID_FIELD ; + + p->port_xmit_data =cpu_to_be64 (swords); + p->port_rcv_data =cpu_to_be64 (rwords); + p->port_xmit_packets =cpu_to_be64 (spkts); + p->port_rcv_packets =cpu_to_be64 (rpkts); + p->port_unicast_xmit_packets =cpu_to_be64 (dev->n_unicast_xmit); + p->port_unicast_rcv_packets =cpu_to_be64 (dev->n_unicast_rcv); + p->port_multicast_xmit_packets =cpu_to_be64 (dev->n_multicast_xmit); + p->port_multicast_rcv_packets =cpu_to_be64 (dev->n_multicast_rcv); + +return reply ((struct ib_smp *) pmp); +} + +static int recv_pma_set_portcounters (struct ib_perf *pmp, +struct ib_device *ibdev,u8 port) +{ +struct ib_pma_portcounters *p = (struct ib_pma_portcounters *) + pmp->data; +struct ipath_ibdev *dev =to_idev (ibdev); +struct ipath_layer_counters cntrs; + +/* +* Since the HW doesn't support clearing counters, we save the +* current count and subtract it from future responses. +*/ +ipath_layer_get_counters (dev->dd, &cntrs); + +if (p->counter_select &IB_PMA_SEL_SYMBOL_ERROR ) + dev->n_symbol_error_counter = cntrs.symbol_error_counter; + +if (p->counter_select &IB_PMA_SEL_LINK_ERROR_RECOVERY ) + dev->n_link_error_recovery_counter = + cntrs.link_error_recovery_counter; + +if (p->counter_select &IB_PMA_SEL_LINK_DOWNED ) + dev->n_link_downed_counter = cntrs.link_downed_counter; + +if (p->counter_select &IB_PMA_SEL_PORT_RCV_ERRORS ) + dev->n_port_rcv_errors = + cntrs.port_rcv_errors + dev->rcv_errors; + +if (p->counter_select &IB_PMA_SEL_PORT_RCV_REMPHYS_ERRORS ) + dev->n_port_rcv_remphys_errors = + cntrs.port_rcv_remphys_errors; + +if (p->counter_select &IB_PMA_SEL_PORT_XMIT_DISCARDS ) + dev->n_port_xmit_discards = cntrs.port_xmit_discards; + +if (p->counter_select &IB_PMA_SEL_PORT_XMIT_DATA ) + dev->n_port_xmit_data = cntrs.port_xmit_data; + +if (p->counter_select &IB_PMA_SEL_PORT_RCV_DATA ) + dev->n_port_rcv_data = cntrs.port_rcv_data; + +if (p->counter_select &IB_PMA_SEL_PORT_XMIT_PACKETS ) + dev->n_port_xmit_packets = cntrs.port_xmit_packets; + +if (p->counter_select &IB_PMA_SEL_PORT_RCV_PACKETS ) + dev->n_port_rcv_packets = cntrs.port_rcv_packets; + +return recv_pma_get_portcounters (pmp, ibdev, port); +} + +static int recv_pma_set_portcounters_ext (struct ib_perf *pmp, +struct ib_device *ibdev,u8 port) +{ +struct ib_pma_portcounters *p = (struct ib_pma_portcounters *) + pmp->data; +struct ipath_ibdev *dev =to_idev (ibdev); +u64 swords, rwords, spkts, rpkts, xwait; + +ipath_layer_snapshot_counters (dev->dd, &swords, &rwords, &spkts, + &rpkts, &xwait); + +if (p->counter_select &IB_PMA_SELX_PORT_XMIT_DATA ) + dev->n_port_xmit_data = swords; + +if (p->counter_select &IB_PMA_SELX_PORT_RCV_DATA ) + dev->n_port_rcv_data = rwords; + +if (p->counter_select &IB_PMA_SELX_PORT_XMIT_PACKETS ) + dev->n_port_xmit_packets = spkts; + +if (p->counter_select &IB_PMA_SELX_PORT_RCV_PACKETS ) + dev->n_port_rcv_packets = rpkts; + +if (p->counter_select &IB_PMA_SELX_PORT_UNI_XMIT_PACKETS ) + dev->n_unicast_xmit =0 ; + +if (p->counter_select &IB_PMA_SELX_PORT_UNI_RCV_PACKETS ) + dev->n_unicast_rcv =0 ; + +if (p->counter_select &IB_PMA_SELX_PORT_MULTI_XMIT_PACKETS ) + dev->n_multicast_xmit =0 ; + +if (p->counter_select &IB_PMA_SELX_PORT_MULTI_RCV_PACKETS ) + dev->n_multicast_rcv =0 ; + +return recv_pma_get_portcounters_ext (pmp, ibdev, port); +} + +static int process_subn (struct ib_device *ibdev,int mad_flags, +u8 port_num,struct ib_mad *in_mad, +struct ib_mad *out_mad) +{ +struct ib_smp *smp = (struct ib_smp *)out_mad; +struct ipath_ibdev *dev =to_idev (ibdev); +int ret; + + *out_mad = *in_mad; +if (smp->class_version !=1 ) { + smp->status |=IB_SMP_UNSUP_VERSION ; + ret =reply (smp); + goto bail; + } + +/* Is the mkey in the process of expiring? */ +if (dev->mkey_lease_timeout && jiffies >= dev->mkey_lease_timeout) { +/* Clear timeout and mkey protection field. */ + dev->mkey_lease_timeout =0 ; + dev->mkeyprot_resv_lmc &=0x3F ; + } + +/* +* M_Key checking depends on +* Portinfo:M_Key_protect_bits +*/ +if ((mad_flags &IB_MAD_IGNORE_MKEY ) ==0 && dev->mkey !=0 && + dev->mkey != smp->mkey && + (smp->method ==IB_MGMT_METHOD_SET || + (smp->method ==IB_MGMT_METHOD_GET && + (dev->mkeyprot_resv_lmc >>7 ) !=0 ))) { +if (dev->mkey_violations !=0xFFFF ) + ++dev->mkey_violations; +if (dev->mkey_lease_timeout || + dev->mkey_lease_period ==0 ) { + ret =IB_MAD_RESULT_SUCCESS | +IB_MAD_RESULT_CONSUMED ; + goto bail; + } + dev->mkey_lease_timeout = jiffies + + dev->mkey_lease_period *HZ ; +/* Future: Generate a trap notice. */ + ret =IB_MAD_RESULT_SUCCESS |IB_MAD_RESULT_CONSUMED ; + goto bail; + }else if (dev->mkey_lease_timeout) + dev->mkey_lease_timeout =0 ; + +switch (smp->method) { +case IB_MGMT_METHOD_GET : +switch (smp->attr_id) { +case IB_SMP_ATTR_NODE_DESC : + ret =recv_subn_get_nodedescription (smp, ibdev); + goto bail; +case IB_SMP_ATTR_NODE_INFO : + ret =recv_subn_get_nodeinfo (smp, ibdev, port_num); + goto bail; +case IB_SMP_ATTR_GUID_INFO : + ret =recv_subn_get_guidinfo (smp, ibdev); + goto bail; +case IB_SMP_ATTR_PORT_INFO : + ret =recv_subn_get_portinfo (smp, ibdev, port_num); + goto bail; +case IB_SMP_ATTR_PKEY_TABLE : + ret =recv_subn_get_pkeytable (smp, ibdev); + goto bail; +case IB_SMP_ATTR_SM_INFO : +if (dev->port_cap_flags &IB_PORT_SM_DISABLED ) { + ret =IB_MAD_RESULT_SUCCESS | +IB_MAD_RESULT_CONSUMED ; + goto bail; + } +if (dev->port_cap_flags &IB_PORT_SM ) { + ret =IB_MAD_RESULT_SUCCESS ; + goto bail; + } +/* FALLTHROUGH */ +default : + smp->status |=IB_SMP_UNSUP_METH_ATTR ; + ret =reply (smp); + goto bail; + } + +case IB_MGMT_METHOD_SET : +switch (smp->attr_id) { +case IB_SMP_ATTR_GUID_INFO : + ret =recv_subn_set_guidinfo (smp, ibdev); + goto bail; +case IB_SMP_ATTR_PORT_INFO : + ret =recv_subn_set_portinfo (smp, ibdev, port_num); + goto bail; +case IB_SMP_ATTR_PKEY_TABLE : + ret =recv_subn_set_pkeytable (smp, ibdev); + goto bail; +case IB_SMP_ATTR_SM_INFO : +if (dev->port_cap_flags &IB_PORT_SM_DISABLED ) { + ret =IB_MAD_RESULT_SUCCESS | +IB_MAD_RESULT_CONSUMED ; + goto bail; + } +if (dev->port_cap_flags &IB_PORT_SM ) { + ret =IB_MAD_RESULT_SUCCESS ; + goto bail; + } +/* FALLTHROUGH */ +default : + smp->status |=IB_SMP_UNSUP_METH_ATTR ; + ret =reply (smp); + goto bail; + } + +case IB_MGMT_METHOD_GET_RESP : +/* +* The ib_mad module will call us to process responses +* before checking for other consumers. +* Just tell the caller to process it normally. +*/ + ret =IB_MAD_RESULT_FAILURE ; + goto bail; +default : + smp->status |=IB_SMP_UNSUP_METHOD ; + ret =reply (smp); + } + +bail: +return ret; +} + +static int process_perf (struct ib_device *ibdev,u8 port_num, +struct ib_mad *in_mad, +struct ib_mad *out_mad) +{ +struct ib_perf *pmp = (struct ib_perf *)out_mad; +int ret; + + *out_mad = *in_mad; +if (pmp->class_version !=1 ) { + pmp->status |=IB_SMP_UNSUP_VERSION ; + ret =reply ((struct ib_smp *) pmp); + goto bail; + } + +switch (pmp->method) { +case IB_MGMT_METHOD_GET : +switch (pmp->attr_id) { +case IB_PMA_CLASS_PORT_INFO : + ret =recv_pma_get_classportinfo (pmp); + goto bail; +case IB_PMA_PORT_SAMPLES_CONTROL : + ret =recv_pma_get_portsamplescontrol (pmp, ibdev, + port_num); + goto bail; +case IB_PMA_PORT_SAMPLES_RESULT : + ret =recv_pma_get_portsamplesresult (pmp, ibdev); + goto bail; +case IB_PMA_PORT_SAMPLES_RESULT_EXT : + ret =recv_pma_get_portsamplesresult_ext (pmp, + ibdev); + goto bail; +case IB_PMA_PORT_COUNTERS : + ret =recv_pma_get_portcounters (pmp, ibdev, + port_num); + goto bail; +case IB_PMA_PORT_COUNTERS_EXT : + ret =recv_pma_get_portcounters_ext (pmp, ibdev, + port_num); + goto bail; +default : + pmp->status |=IB_SMP_UNSUP_METH_ATTR ; + ret =reply ((struct ib_smp *) pmp); + goto bail; + } + +case IB_MGMT_METHOD_SET : +switch (pmp->attr_id) { +case IB_PMA_PORT_SAMPLES_CONTROL : + ret =recv_pma_set_portsamplescontrol (pmp, ibdev, + port_num); + goto bail; +case IB_PMA_PORT_COUNTERS : + ret =recv_pma_set_portcounters (pmp, ibdev, + port_num); + goto bail; +case IB_PMA_PORT_COUNTERS_EXT : + ret =recv_pma_set_portcounters_ext (pmp, ibdev, + port_num); + goto bail; +default : + pmp->status |=IB_SMP_UNSUP_METH_ATTR ; + ret =reply ((struct ib_smp *) pmp); + goto bail; + } + +case IB_MGMT_METHOD_GET_RESP : +/* +* The ib_mad module will call us to process responses +* before checking for other consumers. +* Just tell the caller to process it normally. +*/ + ret =IB_MAD_RESULT_FAILURE ; + goto bail; +default : + pmp->status |=IB_SMP_UNSUP_METHOD ; + ret =reply ((struct ib_smp *) pmp); + } + +bail: +return ret; +} + +/** +* ipath_process_mad - process an incoming MAD packet +* @ibdev: the infiniband device this packet came in on +* @mad_flags: MAD flags +* @port_num: the port number this packet came in on +* @in_wc: the work completion entry for this packet +* @in_grh: the global route header for this packet +* @in_mad: the incoming MAD +* @out_mad: any outgoing MAD reply +* +* Returns IB_MAD_RESULT_SUCCESS if this is a MAD that we are not +* interested in processing. +* +* Note that the verbs framework has already done the MAD sanity checks, +* and hop count/pointer updating for IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE +* MADs. +* +* This is called by the ib_mad module. +*/ +int ipath_process_mad (struct ib_device *ibdev,int mad_flags,u8 port_num, +struct ib_wc *in_wc,struct ib_grh *in_grh, +struct ib_mad *in_mad,struct ib_mad *out_mad) +{ +struct ipath_ibdev *dev =to_idev (ibdev); +int ret; + +/* +* Snapshot current HW counters to "clear" them. +* This should be done when the driver is loaded except that for +* some reason we get a zillion errors when brining up the link. +*/ +if (dev->rcv_errors ==0 ) { +struct ipath_layer_counters cntrs; + +ipath_layer_get_counters (to_idev (ibdev)->dd, &cntrs); + dev->rcv_errors++; + dev->n_symbol_error_counter = cntrs.symbol_error_counter; + dev->n_link_error_recovery_counter = + cntrs.link_error_recovery_counter; + dev->n_link_downed_counter = cntrs.link_downed_counter; + dev->n_port_rcv_errors = cntrs.port_rcv_errors +1 ; + dev->n_port_rcv_remphys_errors = + cntrs.port_rcv_remphys_errors; + dev->n_port_xmit_discards = cntrs.port_xmit_discards; + dev->n_port_xmit_data = cntrs.port_xmit_data; + dev->n_port_rcv_data = cntrs.port_rcv_data; + dev->n_port_xmit_packets = cntrs.port_xmit_packets; + dev->n_port_rcv_packets = cntrs.port_rcv_packets; + } +switch (in_mad->mad_hdr.mgmt_class) { +case IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE : +case IB_MGMT_CLASS_SUBN_LID_ROUTED : + ret =process_subn (ibdev, mad_flags, port_num, + in_mad, out_mad); + goto bail; +case IB_MGMT_CLASS_PERF_MGMT : + ret =process_perf (ibdev, port_num, in_mad, out_mad); + goto bail; +default : + ret =IB_MAD_RESULT_SUCCESS ; + } + +bail: +return ret; +}
drivers/infiniband/hw/ipath/ipath_qp.c new file +913 -0
913 lines not on this page · raw message
drivers/infiniband/hw/ipath/ipath_ruc.c new file +551 -0
551 lines not on this page · raw message
PATCH 1/1 ipath - infiniband UC and UD protocol support
In reply to Bryan O'SullivanThese files implement the Infiniband UC ("unreliable connection") and UD ("unreliable datagram") protocols.
drivers/infiniband/hw/ipath/ipath_uc.c new file +645 -0
@@ -0,0 +1,645 @@ +/* +* Copyright (c) 2005, 2006 PathScale, Inc. All rights reserved. +* +* This software is available to you under a choice of one of two +* licenses. You may choose to be licensed under the terms of the GNU +* General Public License (GPL) Version 2, available from the file +* COPYING in the main directory of this source tree, or the +* OpenIB.org BSD license below: +* +* Redistribution and use in source and binary forms, with or +* without modification, are permitted provided that the following +* conditions are met: +* +* - Redistributions of source code must retain the above +* copyright notice, this list of conditions and the following +* disclaimer. +* +* - Redistributions in binary form must reproduce the above +* copyright notice, this list of conditions and the following +* disclaimer in the documentation and/or other materials +* provided with the distribution. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS +* BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN +* ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +* SOFTWARE. +*/ + +#include "ipath_verbs.h" +#include "ips_common.h" + +/* cut down ridiculously long IB macro names */ +#define OP (x) IB_OPCODE_UC_##x + +static void complete_last_send (struct ipath_qp *qp,struct ipath_swqe *wqe, +struct ib_wc *wc) +{ +if (++qp->s_last == qp->s_size) + qp->s_last =0 ; +if (!test_bit (IPATH_S_SIGNAL_REQ_WR , &qp->s_flags) || + (wqe->wr.send_flags &IB_SEND_SIGNALED )) { + wc->wr_id = wqe->wr.wr_id; + wc->status =IB_WC_SUCCESS ; + wc->opcode = ib_ipath_wc_opcode[wqe->wr.opcode]; + wc->vendor_err =0 ; + wc->byte_len = wqe->length; + wc->qp_num = qp->ibqp.qp_num; + wc->src_qp = qp->remote_qpn; + wc->pkey_index =0 ; + wc->slid = qp->remote_ah_attr.dlid; + wc->sl = qp->remote_ah_attr.sl; + wc->dlid_path_bits =0 ; + wc->port_num =0 ; +ipath_cq_enter (to_icq (qp->ibqp.send_cq), wc,0 ); + } + wqe =get_swqe_ptr (qp, qp->s_last); +} + +/** +* ipath_do_uc_send - do a send on a UC queue +* @data: contains a pointer to the QP to send on +* +* Process entries in the send work queue until the queue is exhausted. +* Only allow one CPU to send a packet per QP (tasklet). +* Otherwise, after we drop the QP lock, two threads could send +* packets out of order. +* This is similar to ipath_do_rc_send() below except we don't have +* timeouts or resends. +*/ +void ipath_do_uc_send (unsigned long data) +{ +struct ipath_qp *qp = (struct ipath_qp *)data; +struct ipath_ibdev *dev =to_idev (qp->ibqp.device); +struct ipath_swqe *wqe; +unsigned long flags; +u16 lrh0; +u32 hwords; +u32 nwords; +u32 extra_bytes; +u32 bth0; +u32 bth2; +u32 pmtu =ib_mtu_enum_to_int (qp->path_mtu); +u32 len; +struct ipath_other_headers *ohdr; +struct ib_wc wc; + +if (test_and_set_bit (IPATH_S_BUSY , &qp->s_flags)) + goto bail; + +if (unlikely (qp->remote_ah_attr.dlid == +ipath_layer_get_lid (dev->dd))) { +/* Pass in an uninitialized ib_wc to save stack space. */ +ipath_ruc_loopback (qp, &wc); +clear_bit (IPATH_S_BUSY , &qp->s_flags); + goto bail; + } + + ohdr = &qp->s_hdr.u.oth; +if (qp->remote_ah_attr.ah_flags &IB_AH_GRH ) + ohdr = &qp->s_hdr.u.l.oth; + +again: +/* Check for a constructed packet to be sent. */ +if (qp->s_hdrwords !=0 ) { +/* +* If no PIO bufs are available, return. +* An interrupt will call ipath_ib_piobufavail() +* when one is available. +*/ +if (ipath_verbs_send (dev->dd, qp->s_hdrwords, + (u32 *) &qp->s_hdr, + qp->s_cur_size, + qp->s_cur_sge)) { +ipath_no_bufs_available (qp, dev); + goto bail; + } + dev->n_unicast_xmit++; +/* Record that we sent the packet and s_hdr is empty. */ + qp->s_hdrwords =0 ; + } + + lrh0 =IPS_LRH_BTH ; +/* header size in 32-bit words LRH+BTH = (8+12)/4. */ + hwords =5 ; + +/* +* The lock is needed to synchronize between +* setting qp->s_ack_state and post_send(). +*/ +spin_lock_irqsave (&qp->s_lock, flags); + +if (!(ib_ipath_state_ops[qp->state] &IPATH_PROCESS_SEND_OK )) + goto done; + + bth0 =ipath_layer_get_pkey (dev->dd, qp->s_pkey_index); + +/* Send a request. */ + wqe =get_swqe_ptr (qp, qp->s_last); +switch (qp->s_state) { +default : +/* +* Signal the completion of the last send (if there is +* one). +*/ +if (qp->s_last != qp->s_tail) +complete_last_send (qp, wqe, &wc); + +/* Check if send work queue is empty. */ +if (qp->s_tail == qp->s_head) + goto done; +/* +* Start a new request. +*/ + qp->s_psn = wqe->psn = qp->s_next_psn; + qp->s_sge.sge = wqe->sg_list[0 ]; + qp->s_sge.sg_list = wqe->sg_list +1 ; + qp->s_sge.num_sge = wqe->wr.num_sge; + qp->s_len = len = wqe->length; +switch (wqe->wr.opcode) { +case IB_WR_SEND : +case IB_WR_SEND_WITH_IMM : +if (len > pmtu) { + qp->s_state =OP (SEND_FIRST ); + len = pmtu; +break ; + } +if (wqe->wr.opcode ==IB_WR_SEND ) + qp->s_state =OP (SEND_ONLY ); +else { + qp->s_state = +OP (SEND_ONLY_WITH_IMMEDIATE ); +/* Immediate data comes after the BTH */ + ohdr->u.imm_data = wqe->wr.imm_data; + hwords +=1 ; + } +if (wqe->wr.send_flags &IB_SEND_SOLICITED ) + bth0 |=1 <<23 ; +break ; + +case IB_WR_RDMA_WRITE : +case IB_WR_RDMA_WRITE_WITH_IMM : + ohdr->u.rc.reth.vaddr = +cpu_to_be64 (wqe->wr.wr.rdma.remote_addr); + ohdr->u.rc.reth.rkey = +cpu_to_be32 (wqe->wr.wr.rdma.rkey); + ohdr->u.rc.reth.length =cpu_to_be32 (len); + hwords +=sizeof (struct ib_reth ) /4 ; +if (len > pmtu) { + qp->s_state =OP (RDMA_WRITE_FIRST ); + len = pmtu; +break ; + } +if (wqe->wr.opcode ==IB_WR_RDMA_WRITE ) + qp->s_state =OP (RDMA_WRITE_ONLY ); +else { + qp->s_state = +OP (RDMA_WRITE_ONLY_WITH_IMMEDIATE ); +/* Immediate data comes after the RETH */ + ohdr->u.rc.imm_data = wqe->wr.imm_data; + hwords +=1 ; +if (wqe->wr.send_flags &IB_SEND_SOLICITED ) + bth0 |=1 <<23 ; + } +break ; + +default : + goto done; + } +if (++qp->s_tail >= qp->s_size) + qp->s_tail =0 ; +break ; + +case OP (SEND_FIRST ): + qp->s_state =OP (SEND_MIDDLE ); +/* FALLTHROUGH */ +case OP (SEND_MIDDLE ): + len = qp->s_len; +if (len > pmtu) { + len = pmtu; +break ; + } +if (wqe->wr.opcode ==IB_WR_SEND ) + qp->s_state =OP (SEND_LAST ); +else { + qp->s_state =OP (SEND_LAST_WITH_IMMEDIATE ); +/* Immediate data comes after the BTH */ + ohdr->u.imm_data = wqe->wr.imm_data; + hwords +=1 ; + } +if (wqe->wr.send_flags &IB_SEND_SOLICITED ) + bth0 |=1 <<23 ; +break ; + +case OP (RDMA_WRITE_FIRST ): + qp->s_state =OP (RDMA_WRITE_MIDDLE ); +/* FALLTHROUGH */ +case OP (RDMA_WRITE_MIDDLE ): + len = qp->s_len; +if (len > pmtu) { + len = pmtu; +break ; + } +if (wqe->wr.opcode ==IB_WR_RDMA_WRITE ) + qp->s_state =OP (RDMA_WRITE_LAST ); +else { + qp->s_state = +OP (RDMA_WRITE_LAST_WITH_IMMEDIATE ); +/* Immediate data comes after the BTH */ + ohdr->u.imm_data = wqe->wr.imm_data; + hwords +=1 ; +if (wqe->wr.send_flags &IB_SEND_SOLICITED ) + bth0 |=1 <<23 ; + } +break ; + } + bth2 = qp->s_next_psn++ &IPS_PSN_MASK ; + qp->s_len -= len; + bth0 |= qp->s_state <<24 ; + +spin_unlock_irqrestore (&qp->s_lock, flags); + +/* Construct the header. */ + extra_bytes = (4 - len) &3 ; + nwords = (len + extra_bytes) >>2 ; +if (unlikely (qp->remote_ah_attr.ah_flags &IB_AH_GRH )) { +/* Header size in 32-bit words. */ + hwords +=10 ; + lrh0 =IPS_LRH_GRH ; + qp->s_hdr.u.l.grh.version_tclass_flow = +cpu_to_be32 ((6 <<28 ) | + (qp->remote_ah_attr.grh.traffic_class + <<20 ) | + qp->remote_ah_attr.grh.flow_label); + qp->s_hdr.u.l.grh.paylen = +cpu_to_be16 (((hwords -12 ) + nwords + +SIZE_OF_CRC ) <<2 ); +/* next_hdr is defined by C8-7 in ch. 8.4.1 */ + qp->s_hdr.u.l.grh.next_hdr =0x1B ; + qp->s_hdr.u.l.grh.hop_limit = + qp->remote_ah_attr.grh.hop_limit; +/* The SGID is 32-bit aligned. */ + qp->s_hdr.u.l.grh.sgid.global.subnet_prefix = + dev->gid_prefix; + qp->s_hdr.u.l.grh.sgid.global.interface_id = +ipath_layer_get_guid (dev->dd); + qp->s_hdr.u.l.grh.dgid = qp->remote_ah_attr.grh.dgid; + } + qp->s_hdrwords = hwords; + qp->s_cur_sge = &qp->s_sge; + qp->s_cur_size = len; + lrh0 |= qp->remote_ah_attr.sl <<4 ; + qp->s_hdr.lrh[0 ] =cpu_to_be16 (lrh0); +/* DEST LID */ + qp->s_hdr.lrh[1 ] =cpu_to_be16 (qp->remote_ah_attr.dlid); + qp->s_hdr.lrh[2 ] =cpu_to_be16 (hwords + nwords +SIZE_OF_CRC ); + qp->s_hdr.lrh[3 ] =cpu_to_be16 (ipath_layer_get_lid (dev->dd)); + bth0 |= extra_bytes <<20 ; + ohdr->bth[0 ] =cpu_to_be32 (bth0); + ohdr->bth[1 ] =cpu_to_be32 (qp->remote_qpn); + ohdr->bth[2 ] =cpu_to_be32 (bth2); + +/* Check for more work to do. */ + goto again; + +done: +spin_unlock_irqrestore (&qp->s_lock, flags); +clear_bit (IPATH_S_BUSY , &qp->s_flags); + +bail: +return ; +} + +/** +* ipath_uc_rcv - handle an incoming UC packet +* @dev: the device the packet came in on +* @hdr: the header of the packet +* @has_grh: true if the packet has a GRH +* @data: the packet data +* @tlen: the length of the packet +* @qp: the QP for this packet. +* +* This is called from ipath_qp_rcv() to process an incoming UC packet +* for the given QP. +* Called at interrupt level. +*/ +void ipath_uc_rcv (struct ipath_ibdev *dev,struct ipath_ib_header *hdr, +int has_grh,void *data,u32 tlen,struct ipath_qp *qp) +{ +struct ipath_other_headers *ohdr; +int opcode; +u32 hdrsize; +u32 psn; +u32 pad; +unsigned long flags; +struct ib_wc wc; +u32 pmtu =ib_mtu_enum_to_int (qp->path_mtu); +struct ib_reth *reth; +int header_in_data; + +/* Check for GRH */ +if (!has_grh) { + ohdr = &hdr->u.oth; + hdrsize =8 +12 ;/* LRH + BTH */ + psn =be32_to_cpu (ohdr->bth[2 ]); + header_in_data =0 ; + }else { + ohdr = &hdr->u.l.oth; + hdrsize =8 +40 +12 ;/* LRH + GRH + BTH */ +/* +* The header with GRH is 60 bytes and the +* core driver sets the eager header buffer +* size to 56 bytes so the last 4 bytes of +* the BTH header (PSN) is in the data buffer. +*/ + header_in_data = +ipath_layer_get_rcvhdrentsize (dev->dd) ==16 ; +if (header_in_data) { + psn =be32_to_cpu (((u32 *) data)[0 ]); + data +=sizeof (u32); + }else + psn =be32_to_cpu (ohdr->bth[2 ]); + } +/* +* The opcode is in the low byte when its in network order +* (top byte when in host order). +*/ + opcode =be32_to_cpu (ohdr->bth[0 ]) >>24 ; + + wc.imm_data =0 ; + wc.wc_flags =0 ; + +spin_lock_irqsave (&qp->r_rq.lock, flags); + +/* Compare the PSN verses the expected PSN. */ +if (unlikely (ipath_cmp24 (psn, qp->r_psn) !=0 )) { +/* +* Handle a sequence error. +* Silently drop any current message. +*/ + qp->r_psn = psn; + inv: + qp->r_state =OP (SEND_LAST ); +switch (opcode) { +case OP (SEND_FIRST ): +case OP (SEND_ONLY ): +case OP (SEND_ONLY_WITH_IMMEDIATE ): + goto send_first; + +case OP (RDMA_WRITE_FIRST ): +case OP (RDMA_WRITE_ONLY ): +case OP (RDMA_WRITE_ONLY_WITH_IMMEDIATE ): + goto rdma_first; + +default : + dev->n_pkt_drops++; + goto done; + } + } + +/* Check for opcode sequence errors. */ +switch (qp->r_state) { +case OP (SEND_FIRST ): +case OP (SEND_MIDDLE ): +if (opcode ==OP (SEND_MIDDLE ) || + opcode ==OP (SEND_LAST ) || + opcode ==OP (SEND_LAST_WITH_IMMEDIATE )) +break ; + goto inv; + +case OP (RDMA_WRITE_FIRST ): +case OP (RDMA_WRITE_MIDDLE ): +if (opcode ==OP (RDMA_WRITE_MIDDLE ) || + opcode ==OP (RDMA_WRITE_LAST ) || + opcode ==OP (RDMA_WRITE_LAST_WITH_IMMEDIATE )) +break ; + goto inv; + +default : +if (opcode ==OP (SEND_FIRST ) || + opcode ==OP (SEND_ONLY ) || + opcode ==OP (SEND_ONLY_WITH_IMMEDIATE ) || + opcode ==OP (RDMA_WRITE_FIRST ) || + opcode ==OP (RDMA_WRITE_ONLY ) || + opcode ==OP (RDMA_WRITE_ONLY_WITH_IMMEDIATE )) +break ; + goto inv; + } + +/* OK, process the packet. */ +switch (opcode) { +case OP (SEND_FIRST ): +case OP (SEND_ONLY ): +case OP (SEND_ONLY_WITH_IMMEDIATE ): + send_first: +if (qp->r_reuse_sge) { + qp->r_reuse_sge =0 ; + qp->r_sge = qp->s_rdma_sge; + }else if (!ipath_get_rwqe (qp,0 )) { + dev->n_pkt_drops++; + goto done; + } +/* Save the WQE so we can reuse it in case of an error. */ + qp->s_rdma_sge = qp->r_sge; + qp->r_rcv_len =0 ; +if (opcode ==OP (SEND_ONLY )) + goto send_last; +else if (opcode ==OP (SEND_ONLY_WITH_IMMEDIATE )) + goto send_last_imm; +/* FALLTHROUGH */ +case OP (SEND_MIDDLE ): +/* Check for invalid length PMTU or posted rwqe len. */ +if (unlikely (tlen != (hdrsize + pmtu +4 ))) { + qp->r_reuse_sge =1 ; + dev->n_pkt_drops++; + goto done; + } + qp->r_rcv_len += pmtu; +if (unlikely (qp->r_rcv_len > qp->r_len)) { + qp->r_reuse_sge =1 ; + dev->n_pkt_drops++; + goto done; + } +ipath_copy_sge (&qp->r_sge, data, pmtu); +break ; + +case OP (SEND_LAST_WITH_IMMEDIATE ): + send_last_imm: +if (header_in_data) { + wc.imm_data = *(u32 *) data; + data +=sizeof (u32); + }else { +/* Immediate data comes after BTH */ + wc.imm_data = ohdr->u.imm_data; + } + hdrsize +=4 ; + wc.wc_flags =IB_WC_WITH_IMM ; +/* FALLTHROUGH */ +case OP (SEND_LAST ): + send_last: +/* Get the number of bytes the message was padded by. */ + pad = (be32_to_cpu (ohdr->bth[0 ]) >>20 ) &3 ; +/* Check for invalid length. */ +/* XXX LAST len should be >= 1 */ +if (unlikely (tlen < (hdrsize + pad +4 ))) { + qp->r_reuse_sge =1 ; + dev->n_pkt_drops++; + goto done; + } +/* Don't count the CRC. */ + tlen -= (hdrsize + pad +4 ); + wc.byte_len = tlen + qp->r_rcv_len; +if (unlikely (wc.byte_len > qp->r_len)) { + qp->r_reuse_sge =1 ; + dev->n_pkt_drops++; + goto done; + } +/* XXX Need to free SGEs */ + last_imm: +ipath_copy_sge (&qp->r_sge, data, tlen); + wc.wr_id = qp->r_wr_id; + wc.status =IB_WC_SUCCESS ; + wc.opcode =IB_WC_RECV ; + wc.vendor_err =0 ; + wc.qp_num = qp->ibqp.qp_num; + wc.src_qp = qp->remote_qpn; + wc.pkey_index =0 ; + wc.slid = qp->remote_ah_attr.dlid; + wc.sl = qp->remote_ah_attr.sl; + wc.dlid_path_bits =0 ; + wc.port_num =0 ; +/* Signal completion event if the solicited bit is set. */ +ipath_cq_enter (to_icq (qp->ibqp.recv_cq), &wc, + ohdr->bth[0 ] & +__constant_cpu_to_be32 (1 <<23 )); +break ; + +case OP (RDMA_WRITE_FIRST ): +case OP (RDMA_WRITE_ONLY ): +case OP (RDMA_WRITE_ONLY_WITH_IMMEDIATE ):/* consume RWQE */ + rdma_first: +/* RETH comes after BTH */ +if (!header_in_data) + reth = &ohdr->u.rc.reth; +else { + reth = (struct ib_reth *)data; + data +=sizeof (*reth); + } + hdrsize +=sizeof (*reth); + qp->r_len =be32_to_cpu (reth->length); + qp->r_rcv_len =0 ; +if (qp->r_len !=0 ) { +u32 rkey =be32_to_cpu (reth->rkey); +u64 vaddr =be64_to_cpu (reth->vaddr); + +/* Check rkey */ +if (unlikely (!ipath_rkey_ok ( + dev, &qp->r_sge, qp->r_len, + vaddr, rkey, +IB_ACCESS_REMOTE_WRITE ))) { + dev->n_pkt_drops++; + goto done; + } + }else { + qp->r_sge.sg_list =NULL ; + qp->r_sge.sge.mr =NULL ; + qp->r_sge.sge.vaddr =NULL ; + qp->r_sge.sge.length =0 ; + qp->r_sge.sge.sge_length =0 ; + } +if (unlikely (!(qp->qp_access_flags & +IB_ACCESS_REMOTE_WRITE ))) { + dev->n_pkt_drops++; + goto done; + } +if (opcode ==OP (RDMA_WRITE_ONLY )) + goto rdma_last; +else if (opcode == +OP (RDMA_WRITE_ONLY_WITH_IMMEDIATE )) + goto rdma_last_imm; +/* FALLTHROUGH */ +case OP (RDMA_WRITE_MIDDLE ): +/* Check for invalid length PMTU or posted rwqe len. */ +if (unlikely (tlen != (hdrsize + pmtu +4 ))) { + dev->n_pkt_drops++; + goto done; + } + qp->r_rcv_len += pmtu; +if (unlikely (qp->r_rcv_len > qp->r_len)) { + dev->n_pkt_drops++; + goto done; + } +ipath_copy_sge (&qp->r_sge, data, pmtu); +break ; + +case OP (RDMA_WRITE_LAST_WITH_IMMEDIATE ): + rdma_last_imm: +/* Get the number of bytes the message was padded by. */ + pad = (be32_to_cpu (ohdr->bth[0 ]) >>20 ) &3 ; +/* Check for invalid length. */ +/* XXX LAST len should be >= 1 */ +if (unlikely (tlen < (hdrsize + pad +4 ))) { + dev->n_pkt_drops++; + goto done; + } +/* Don't count the CRC. */ + tlen -= (hdrsize + pad +4 ); +if (unlikely (tlen + qp->r_rcv_len != qp->r_len)) { + dev->n_pkt_drops++; + goto done; + } +if (qp->r_reuse_sge) { + qp->r_reuse_sge =0 ; + }else if (!ipath_get_rwqe (qp,1 )) { + dev->n_pkt_drops++; + goto done; + } +if (header_in_data) { + wc.imm_data = *(u32 *) data; + data +=sizeof (u32); + }else { +/* Immediate data comes after BTH */ + wc.imm_data = ohdr->u.imm_data; + } + hdrsize +=4 ; + wc.wc_flags =IB_WC_WITH_IMM ; + wc.byte_len =0 ; + goto last_imm; + +case OP (RDMA_WRITE_LAST ): + rdma_last: +/* Get the number of bytes the message was padded by. */ + pad = (be32_to_cpu (ohdr->bth[0 ]) >>20 ) &3 ; +/* Check for invalid length. */ +/* XXX LAST len should be >= 1 */ +if (unlikely (tlen < (hdrsize + pad +4 ))) { + dev->n_pkt_drops++; + goto done; + } +/* Don't count the CRC. */ + tlen -= (hdrsize + pad +4 ); +if (unlikely (tlen + qp->r_rcv_len != qp->r_len)) { + dev->n_pkt_drops++; + goto done; + } +ipath_copy_sge (&qp->r_sge, data, tlen); +break ; + +default : +/* Drop packet for unknown opcodes. */ +spin_unlock_irqrestore (&qp->r_rq.lock, flags); + dev->n_pkt_drops++; + goto bail; + } + qp->r_psn++; + qp->r_state = opcode; +done: +spin_unlock_irqrestore (&qp->r_rq.lock, flags); + +bail: +return ; +}
drivers/infiniband/hw/ipath/ipath_ud.c new file +620 -0
620 lines not on this page · raw message
PATCH 1/1 ipath - misc driver support code
In reply to Bryan O'SullivanEEPROM support, interrupt handling, statistics gathering, and write combining management for x86_64.
A note regarding i2c: The Atmel EEPROM hardware we use looks like an i2c device electrically, but is not i2c compliant at all from a functional perspective. We tried using the kernel's i2c support to talk to it, but failed.
Normal i2c devices have a single 7-bit or 10-bit i2c address that they respond to. Valid 7-bit addresses range from 0x03 to 0x77. Addresses 0x00 to 0x02 and 0x78 to 0x7F are special reserved addresses (e.g. 0x00 is the "general call" address.) The Atmel device, on the other hand, responds to ALL addresses. It's designed to be the only device on a given i2c bus. A given i2c device address corresponds to the memory address within the i2c device itself.
At least one reason why the linux core i2c stuff won't work for this is that it prohibits access to reserved addresses like 0x00, which are really valid addresses on the Atmel devices.
drivers/infiniband/hw/ipath/ipath_eeprom.c new file +613 -0
@@ -0,0 +1,613 @@ +/* +* Copyright (c) 2003, 2004, 2005, 2006 PathScale, Inc. All rights reserved. +* +* This software is available to you under a choice of one of two +* licenses. You may choose to be licensed under the terms of the GNU +* General Public License (GPL) Version 2, available from the file +* COPYING in the main directory of this source tree, or the +* OpenIB.org BSD license below: +* +* Redistribution and use in source and binary forms, with or +* without modification, are permitted provided that the following +* conditions are met: +* +* - Redistributions of source code must retain the above +* copyright notice, this list of conditions and the following +* disclaimer. +* +* - Redistributions in binary form must reproduce the above +* copyright notice, this list of conditions and the following +* disclaimer in the documentation and/or other materials +* provided with the distribution. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS +* BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN +* ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +* SOFTWARE. +*/ + +#include <linux/delay.h> +#include <linux/pci.h> +#include <linux/vmalloc.h> + +#include "ipath_kernel.h" + +/* +* InfiniPath I2C driver for a serial eeprom. This is not a generic +* I2C interface. For a start, the device we're using (Atmel AT24C11) +* doesn't work like a regular I2C device. It looks like one +* electrically, but not logically. Normal I2C devices have a single +* 7-bit or 10-bit I2C address that they respond to. Valid 7-bit +* addresses range from 0x03 to 0x77. Addresses 0x00 to 0x02 and 0x78 +* to 0x7F are special reserved addresses (e.g. 0x00 is the "general +* call" address.) The Atmel device, on the other hand, responds to ALL +* 7-bit addresses. It's designed to be the only device on a given I2C +* bus. A 7-bit address corresponds to the memory address within the +* Atmel device itself. +* +* Also, the timing requirements mean more than simple software +* bitbanging, with readbacks from chip to ensure timing (simple udelay +* is not enough). +* +* This all means that accessing the device is specialized enough +* that using the standard kernel I2C bitbanging interface would be +* impossible. For example, the core I2C eeprom driver expects to find +* a device at one or more of a limited set of addresses only. It doesn't +* allow writing to an eeprom. It also doesn't provide any means of +* accessing eeprom contents from within the kernel, only via sysfs. +*/ + +enum i2c_type { + i2c_line_scl =0 , + i2c_line_sda +}; + +enum i2c_state { + i2c_line_low =0 , + i2c_line_high +}; + +#define READ_CMD 1 +#define WRITE_CMD 0 + +static int eeprom_init; + +/* +* The gpioval manipulation really should be protected by spinlocks +* or be converted to use atomic operations. +*/ + +/** +* i2c_gpio_set - set a GPIO line +* @dd: the infinipath device +* @line: the line to set +* @new_line_state: the state to set +* +* Returns 0 if the line was set to the new state successfully, non-zero +* on error. +*/ +static int i2c_gpio_set (struct ipath_devdata *dd, +enum i2c_type line, +enum i2c_state new_line_state) +{ +u64 read_val, write_val, mask, *gpioval; + + gpioval = &dd->ipath_gpio_out; + read_val =ipath_read_kreg64 (dd, dd->ipath_kregs->kr_extctrl); +if (line == i2c_line_scl) + mask = ipath_gpio_scl; +else + mask = ipath_gpio_sda; + +if (new_line_state == i2c_line_high) +/* tri-state the output rather than force high */ + write_val = read_val & ~mask; +else +/* config line to be an output */ + write_val = read_val | mask; +ipath_write_kreg (dd, dd->ipath_kregs->kr_extctrl, write_val); + +/* set high and verify */ +if (new_line_state == i2c_line_high) + write_val =0x1UL ; +else + write_val =0x0UL ; + +if (line == i2c_line_scl) { + write_val <<= ipath_gpio_scl_num; + *gpioval = *gpioval & ~(1UL << ipath_gpio_scl_num); + *gpioval |= write_val; + }else { + write_val <<= ipath_gpio_sda_num; + *gpioval = *gpioval & ~(1UL << ipath_gpio_sda_num); + *gpioval |= write_val; + } +ipath_write_kreg (dd, dd->ipath_kregs->kr_gpio_out, *gpioval); + +return 0 ; +} + +/** +* i2c_gpio_get - get a GPIO line state +* @dd: the infinipath device +* @line: the line to get +* @curr_statep: where to put the line state +* +* Returns 0 if the line was set to the new state successfully, non-zero +* on error. curr_state is not set on error. +*/ +static int i2c_gpio_get (struct ipath_devdata *dd, +enum i2c_type line, +enum i2c_state *curr_statep) +{ +u64 read_val, write_val, mask; +int ret; + +/* check args */ +if (curr_statep ==NULL ) { + ret =1 ; + goto bail; + } + + read_val =ipath_read_kreg64 (dd, dd->ipath_kregs->kr_extctrl); +/* config line to be an input */ +if (line == i2c_line_scl) + mask = ipath_gpio_scl; +else + mask = ipath_gpio_sda; + write_val = read_val & ~mask; +ipath_write_kreg (dd, dd->ipath_kregs->kr_extctrl, write_val); + read_val =ipath_read_kreg64 (dd, dd->ipath_kregs->kr_extstatus); + +if (read_val & mask) + *curr_statep = i2c_line_high; +else + *curr_statep = i2c_line_low; + + ret =0 ; + +bail: +return ret; +} + +/** +* i2c_wait_for_writes - wait for a write +* @dd: the infinipath device +* +* We use this instead of udelay directly, so we can make sure +* that previous register writes have been flushed all the way +* to the chip. Since we are delaying anyway, the cost doesn't +* hurt, and makes the bit twiddling more regular +*/ +static void i2c_wait_for_writes (struct ipath_devdata *dd) +{ + (void )ipath_read_kreg32 (dd, dd->ipath_kregs->kr_scratch); +} + +static void scl_out (struct ipath_devdata *dd,u8 bit) +{ +i2c_gpio_set (dd, i2c_line_scl, bit ? i2c_line_high : i2c_line_low); + +i2c_wait_for_writes (dd); +} + +static void sda_out (struct ipath_devdata *dd,u8 bit) +{ +i2c_gpio_set (dd, i2c_line_sda, bit ? i2c_line_high : i2c_line_low); + +i2c_wait_for_writes (dd); +} + +static u8 sda_in (struct ipath_devdata *dd,int wait) +{ +enum i2c_state bit; + +if (i2c_gpio_get (dd, i2c_line_sda, &bit)) +ipath_dbg ("get bit failed!\n" ); + +if (wait) +i2c_wait_for_writes (dd); + +return bit == i2c_line_high ?1U :0 ; +} + +/** +* i2c_ackrcv - see if ack following write is true +* @dd: the infinipath device +*/ +static int i2c_ackrcv (struct ipath_devdata *dd) +{ +u8 ack_received; + +/* AT ENTRY SCL = LOW */ +/* change direction, ignore data */ + ack_received =sda_in (dd,1 ); +scl_out (dd, i2c_line_high); + ack_received =sda_in (dd,1 ) ==0 ; +scl_out (dd, i2c_line_low); +return ack_received; +} + +/** +* wr_byte - write a byte, one bit at a time +* @dd: the infinipath device +* @data: the byte to write +* +* Returns 0 if we got the following ack, otherwise 1 +*/ +static int wr_byte (struct ipath_devdata *dd,u8 data) +{ +int bit_cntr; +u8 bit; + +for (bit_cntr =7 ; bit_cntr >=0 ; bit_cntr--) { + bit = (data >> bit_cntr) &1 ; +sda_out (dd, bit); +scl_out (dd, i2c_line_high); +scl_out (dd, i2c_line_low); + } +return (!i2c_ackrcv (dd)) ?1 :0 ; +} + +static void send_ack (struct ipath_devdata *dd) +{ +sda_out (dd, i2c_line_low); +scl_out (dd, i2c_line_high); +scl_out (dd, i2c_line_low); +sda_out (dd, i2c_line_high); +} + +/** +* i2c_startcmd - transmit the start condition, followed by address/cmd +* @dd: the infinipath device +* @offset_dir: direction byte +* +* (both clock/data high, clock high, data low while clock is high) +*/ +static int i2c_startcmd (struct ipath_devdata *dd,u8 offset_dir) +{ +int res; + +/* issue start sequence */ +sda_out (dd, i2c_line_high); +scl_out (dd, i2c_line_high); +sda_out (dd, i2c_line_low); +scl_out (dd, i2c_line_low); + +/* issue length and direction byte */ + res =wr_byte (dd, offset_dir); + +if (res) +ipath_cdbg (VERBOSE ,"No ack to complete start\n" ); + +return res; +} + +/** +* stop_cmd - transmit the stop condition +* @dd: the infinipath device +* +* (both clock/data low, clock high, data high while clock is high) +*/ +static void stop_cmd (struct ipath_devdata *dd) +{ +scl_out (dd, i2c_line_low); +sda_out (dd, i2c_line_low); +scl_out (dd, i2c_line_high); +sda_out (dd, i2c_line_high); +udelay (2 ); +} + +/** +* eeprom_reset - reset I2C communication +* @dd: the infinipath device +*/ + +static int eeprom_reset (struct ipath_devdata *dd) +{ +int clock_cycles_left =9 ; +u64 *gpioval = &dd->ipath_gpio_out; +int ret; + + eeprom_init =1 ; + *gpioval =ipath_read_kreg64 (dd, dd->ipath_kregs->kr_gpio_out); +ipath_cdbg (VERBOSE ,"Resetting i2c eeprom; initial gpioout reg " +"is %llx\n" , (unsigned long long ) *gpioval); + +/* +* This is to get the i2c into a known state, by first going low, +* then tristate sda (and then tristate scl as first thing +* in loop) +*/ +scl_out (dd, i2c_line_low); +sda_out (dd, i2c_line_high); + +while (clock_cycles_left--) { +scl_out (dd, i2c_line_high); + +if (sda_in (dd,0 )) { +sda_out (dd, i2c_line_low); +scl_out (dd, i2c_line_low); + ret =0 ; + goto bail; + } + +scl_out (dd, i2c_line_low); + } + + ret =1 ; + +bail: +return ret; +} + +/** +* ipath_eeprom_read - receives bytes from the eeprom via I2C +* @dd: the infinipath device +* @eeprom_offset: address to read from +* @buffer: where to store result +* @len: number of bytes to receive +*/ + +int ipath_eeprom_read (struct ipath_devdata *dd,u8 eeprom_offset, +void *buffer,int len) +{ +/* compiler complains unless initialized */ +u8 single_byte =0 ; +int bit_cntr; +int ret; + +if (!eeprom_init) +eeprom_reset (dd); + + eeprom_offset = (eeprom_offset <<1 ) |READ_CMD ; + +if (i2c_startcmd (dd, eeprom_offset)) { +ipath_dbg ("Failed startcmd\n" ); +stop_cmd (dd); + ret =1 ; + goto bail; + } + +/* +* eeprom keeps clocking data out as long as we ack, automatically +* incrementing the address. +*/ +while (len-- >0 ) { +/* get data */ + single_byte =0 ; +for (bit_cntr =8 ; bit_cntr; bit_cntr--) { +u8 bit; +scl_out (dd, i2c_line_high); + bit =sda_in (dd,0 ); + single_byte |= bit << (bit_cntr -1 ); +scl_out (dd, i2c_line_low); + } + +/* send ack if not the last byte */ +if (len) +send_ack (dd); + + *((u8 *) buffer) = single_byte; + buffer++; + } + +stop_cmd (dd); + + ret =0 ; + +bail: +return ret; +} + +/** +* ipath_eeprom_write - writes data to the eeprom via I2C +* @dd: the infinipath device +* @eeprom_offset: where to place data +* @buffer: data to write +* @len: number of bytes to write +*/ +int ipath_eeprom_write (struct ipath_devdata *dd,u8 eeprom_offset, +const void *buffer,int len) +{ +u8 single_byte; +int sub_len; +const u8 *bp = buffer; +int max_wait_time, i; +int ret; + +if (!eeprom_init) +eeprom_reset (dd); + +while (len >0 ) { +if (i2c_startcmd (dd, (eeprom_offset <<1 ) |WRITE_CMD )) { +ipath_dbg ("Failed to start cmd offset %u\n" , + eeprom_offset); + goto failed_write; + } + + sub_len =min (len,4 ); + eeprom_offset += sub_len; + len -= sub_len; + +for (i =0 ; i < sub_len; i++) { +if (wr_byte (dd, *bp++)) { +ipath_dbg ("no ack after byte %u/%u (%u " +"total remain)\n" , i, sub_len, + len + sub_len - i); + goto failed_write; + } + } + +stop_cmd (dd); + +/* +* wait for write complete by waiting for a successful +* read (the chip replies with a zero after the write +* cmd completes, and before it writes to the eeprom. +* The startcmd for the read will fail the ack until +* the writes have completed. We do this inline to avoid +* the debug prints that are in the real read routine +* if the startcmd fails. +*/ + max_wait_time =100 ; +while (i2c_startcmd (dd,READ_CMD )) { +stop_cmd (dd); +if (!--max_wait_time) { +ipath_dbg ("Did not get successful read to " +"complete write\n" ); + goto failed_write; + } + } +/* now read the zero byte */ +for (i = single_byte =0 ; i <8 ; i++) { +u8 bit; +scl_out (dd, i2c_line_high); + bit =sda_in (dd,0 ); +scl_out (dd, i2c_line_low); + single_byte <<=1 ; + single_byte |= bit; + } +stop_cmd (dd); + } + + ret =0 ; + goto bail; + +failed_write: +stop_cmd (dd); + ret =1 ; + +bail: +return ret; +} + +static u8 flash_csum (struct ipath_flash *ifp,int adjust) +{ +u8 *ip = (u8 *) ifp; +u8 csum =0 , len; + +for (len =0 ; len < ifp->if_length; len++) + csum += *ip++; + csum -= ifp->if_csum; + csum = ~csum; +if (adjust) + ifp->if_csum = csum; + +return csum; +} + +/** +* ipath_get_guid - get the GUID from the i2c device +* @dd: the infinipath device +* +* When we add the multi-chip support, we will probably have to add +* the ability to use the number of guids field, and get the guid from +* the first chip's flash, to use for all of them. +*/ +void ipath_get_guid (struct ipath_devdata *dd) +{ +void *buf; +struct ipath_flash *ifp; +u64 guid; +int len; +u8 csum, *bguid; +int t = dd->ipath_unit; +struct ipath_devdata *dd0 =ipath_lookup (0 ); + +if (t && dd0->ipath_nguid >1 && t <= dd0->ipath_nguid) { +u8 *bguid, oguid; + dd->ipath_guid = dd0->ipath_guid; + bguid = (u8 *) & dd->ipath_guid; + + oguid = bguid[7 ]; + bguid[7 ] += t; +if (oguid > bguid[7 ]) { +if (bguid[6 ] ==0xff ) { +if (bguid[5 ] ==0xff ) { +ipath_dev_err ( + dd, +"Can't set %s GUID from " +"base, wraps to OUI!\n" , +ipath_get_unit_name (t)); + dd->ipath_guid =0 ; + goto bail; + } + bguid[5 ]++; + } + bguid[6 ]++; + } + dd->ipath_nguid =1 ; + +ipath_dbg ("nguid %u, so adding %u to device 0 guid, " +"for %llx (big-endian)\n" , + dd0->ipath_nguid, t, + (unsigned long long ) dd->ipath_guid); + goto bail; + } + + len = offsetof(struct ipath_flash , if_future); + buf =vmalloc (len); +if (!buf) { +ipath_dev_err (dd,"Couldn't allocate memory to read %u " +"bytes from eeprom for GUID\n" , len); + goto bail; + } + +if (ipath_eeprom_read (dd,0 , buf, len)) { +ipath_dev_err (dd,"Failed reading GUID from eeprom\n" ); + goto done; + } + ifp = (struct ipath_flash *)buf; + + csum =flash_csum (ifp,0 ); +if (csum != ifp->if_csum) { +dev_info (&dd->pcidev->dev,"Bad I2C flash checksum: " +"0x%x, not 0x%x\n" , csum, ifp->if_csum); + goto done; + } +if (*(u64 *) ifp->if_guid ==0ULL || + *(u64 *) ifp->if_guid ==-1LL ) { +ipath_dev_err (dd,"Invalid GUID %llx from flash; " +"ignoring\n" , + *(unsigned long long *) ifp->if_guid); +/* don't allow GUID if all 0 or all 1's */ + goto done; + } + +/* complain, but allow it */ +if (*(u64 *) ifp->if_guid ==0x100007511000000ULL ) +dev_info (&dd->pcidev->dev,"Warning, GUID %llx is " +"default, probably not correct!\n" , + *(unsigned long long *) ifp->if_guid); + + bguid = ifp->if_guid; +if (!bguid[0 ] && !bguid[1 ] && !bguid[2 ]) { +/* original incorrect GUID format in flash; fix in +* core copy, by shifting up 2 octets; don't need to +* change top octet, since both it and shifted are +* 0.. */ + bguid[1 ] = bguid[3 ]; + bguid[2 ] = bguid[4 ]; + bguid[3 ] = bguid[4 ] =0 ; + guid = *(u64 *) ifp->if_guid; +ipath_cdbg (VERBOSE ,"Old GUID format in flash, top 3 zero, " +"shifting 2 octets\n" ); + }else + guid = *(u64 *) ifp->if_guid; + dd->ipath_guid = guid; + dd->ipath_nguid = ifp->if_numguid; +memcpy (dd->ipath_serial, ifp->if_serial, +sizeof (ifp->if_serial)); +ipath_cdbg (VERBOSE ,"Initted GUID to %llx (big-endian) from i2c " +"flash\n" , (unsigned long long ) dd->ipath_guid); + +done: +vfree (buf); + +bail:; +}
drivers/infiniband/hw/ipath/ipath_intr.c new file +841 -0
841 lines not on this page · raw message
drivers/infiniband/hw/ipath/ipath_stats.c new file +295 -0
295 lines not on this page · raw message
drivers/infiniband/hw/ipath/ipath_wc_x86_64.c new file +157 -0
157 lines not on this page · raw message
PATCH 1/1 ipath - misc infiniband code, part 1
In reply to Bryan O'SullivanCompletion queues, local and remote memory keys, and memory region support.
drivers/infiniband/hw/ipath/ipath_cq.c new file +295 -0
@@ -0,0 +1,295 @@ +/* +* Copyright (c) 2005, 2006 PathScale, Inc. All rights reserved. +* +* This software is available to you under a choice of one of two +* licenses. You may choose to be licensed under the terms of the GNU +* General Public License (GPL) Version 2, available from the file +* COPYING in the main directory of this source tree, or the +* OpenIB.org BSD license below: +* +* Redistribution and use in source and binary forms, with or +* without modification, are permitted provided that the following +* conditions are met: +* +* - Redistributions of source code must retain the above +* copyright notice, this list of conditions and the following +* disclaimer. +* +* - Redistributions in binary form must reproduce the above +* copyright notice, this list of conditions and the following +* disclaimer in the documentation and/or other materials +* provided with the distribution. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS +* BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN +* ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +* SOFTWARE. +*/ + +#include <linux/err.h> +#include <linux/vmalloc.h> + +#include "ipath_verbs.h" + +/** +* ipath_cq_enter - add a new entry to the completion queue +* @cq: completion queue +* @entry: work completion entry to add +* @sig: true if @entry is a solicitated entry +* +* This may be called with one of the qp->s_lock or qp->r_rq.lock held. +*/ +void ipath_cq_enter (struct ipath_cq *cq,struct ib_wc *entry,int solicited) +{ +unsigned long flags; +u32 next; + +spin_lock_irqsave (&cq->lock, flags); + +if (cq->head == cq->ibcq.cqe) + next =0 ; +else + next = cq->head +1 ; +if (unlikely (next == cq->tail)) { +spin_unlock_irqrestore (&cq->lock, flags); +if (cq->ibcq.event_handler) { +struct ib_event ev; + + ev.device = cq->ibcq.device; + ev.element.cq = &cq->ibcq; + ev.event =IB_EVENT_CQ_ERR ; + cq->ibcq.event_handler (&ev, cq->ibcq.cq_context); + } +return ; + } + cq->queue[cq->head] = *entry; + cq->head = next; + +if (cq->notify ==IB_CQ_NEXT_COMP || + (cq->notify ==IB_CQ_SOLICITED && solicited)) { + cq->notify =IB_CQ_NONE ; + cq->triggered++; +/* +* This will cause send_complete() to be called in +* another thread. +*/ +tasklet_hi_schedule (&cq->comptask); + } + +spin_unlock_irqrestore (&cq->lock, flags); + +if (entry->status !=IB_WC_SUCCESS ) +to_idev (cq->ibcq.device)->n_wqe_errs++; +} + +/** +* ipath_poll_cq - poll for work completion entries +* @ibcq: the completion queue to poll +* @num_entries: the maximum number of entries to return +* @entry: pointer to array where work completions are placed +* +* Returns the number of completion entries polled. +* +* This may be called from interrupt context. Also called by ib_poll_cq() +* in the generic verbs code. +*/ +int ipath_poll_cq (struct ib_cq *ibcq,int num_entries,struct ib_wc *entry) +{ +struct ipath_cq *cq =to_icq (ibcq); +unsigned long flags; +int npolled; + +spin_lock_irqsave (&cq->lock, flags); + +for (npolled =0 ; npolled < num_entries; ++npolled, ++entry) { +if (cq->tail == cq->head) +break ; + *entry = cq->queue[cq->tail]; +if (cq->tail == cq->ibcq.cqe) + cq->tail =0 ; +else + cq->tail++; + } + +spin_unlock_irqrestore (&cq->lock, flags); + +return npolled; +} + +static void send_complete (unsigned long data) +{ +struct ipath_cq *cq = (struct ipath_cq *)data; + +/* +* The completion handler will most likely rearm the notification +* and poll for all pending entries. If a new completion entry +* is added while we are in this routine, tasklet_hi_schedule() +* won't call us again until we return so we check triggered to +* see if we need to call the handler again. +*/ +for (;;) { +u8 triggered = cq->triggered; + + cq->ibcq.comp_handler (&cq->ibcq, cq->ibcq.cq_context); + +if (cq->triggered == triggered) +return ; + } +} + +/** +* ipath_create_cq - create a completion queue +* @ibdev: the device this completion queue is attached to +* @entries: the minimum size of the completion queue +* @context: unused by the InfiniPath driver +* @udata: unused by the InfiniPath driver +* +* Returns a pointer to the completion queue or negative errno values +* for failure. +* +* Called by ib_create_cq() in the generic verbs code. +*/ +struct ib_cq *ipath_create_cq (struct ib_device *ibdev,int entries, +struct ib_ucontext *context, +struct ib_udata *udata) +{ +struct ipath_cq *cq; +struct ib_wc *wc; +struct ib_cq *ret; + +/* +* Need to use vmalloc() if we want to support large #s of +* entries. +*/ + cq =kmalloc (sizeof (*cq),GFP_KERNEL ); +if (!cq) { + ret =ERR_PTR (-ENOMEM ); + goto bail; + } + +/* +* Need to use vmalloc() if we want to support large #s of entries. +*/ + wc =vmalloc (sizeof (*wc) * (entries +1 )); +if (!wc) { +kfree (cq); + ret =ERR_PTR (-ENOMEM ); + goto bail; + } +/* +* ib_create_cq() will initialize cq->ibcq except for cq->ibcq.cqe. +* The number of entries should be >= the number requested or return +* an error. +*/ + cq->ibcq.cqe = entries; + cq->notify =IB_CQ_NONE ; + cq->triggered =0 ; +spin_lock_init (&cq->lock); +tasklet_init (&cq->comptask, send_complete, (unsigned long )cq); + cq->head =0 ; + cq->tail =0 ; + cq->queue = wc; + + ret = &cq->ibcq; + +bail: +return ret; +} + +/** +* ipath_destroy_cq - destroy a completion queue +* @ibcq: the completion queue to destroy. +* +* Returns 0 for success. +* +* Called by ib_destroy_cq() in the generic verbs code. +*/ +int ipath_destroy_cq (struct ib_cq *ibcq) +{ +struct ipath_cq *cq =to_icq (ibcq); + +tasklet_kill (&cq->comptask); +vfree (cq->queue); +kfree (cq); + +return 0 ; +} + +/** +* ipath_req_notify_cq - change the notification type for a completion queue +* @ibcq: the completion queue +* @notify: the type of notification to request +* +* Returns 0 for success. +* +* This may be called from interrupt context. Also called by +* ib_req_notify_cq() in the generic verbs code. +*/ +int ipath_req_notify_cq (struct ib_cq *ibcq,enum ib_cq_notify notify) +{ +struct ipath_cq *cq =to_icq (ibcq); +unsigned long flags; + +spin_lock_irqsave (&cq->lock, flags); +/* +* Don't change IB_CQ_NEXT_COMP to IB_CQ_SOLICITED but allow +* any other transitions. +*/ +if (cq->notify !=IB_CQ_NEXT_COMP ) + cq->notify = notify; +spin_unlock_irqrestore (&cq->lock, flags); +return 0 ; +} + +int ipath_resize_cq (struct ib_cq *ibcq,int cqe,struct ib_udata *udata) +{ +struct ipath_cq *cq =to_icq (ibcq); +struct ib_wc *wc, *old_wc; +u32 n; +int ret; + +/* +* Need to use vmalloc() if we want to support large #s of entries. +*/ + wc =vmalloc (sizeof (*wc) * (cqe +1 )); +if (!wc) { + ret = -ENOMEM ; + goto bail; + } + +spin_lock_irq (&cq->lock); +if (cq->head < cq->tail) + n = cq->ibcq.cqe +1 + cq->head - cq->tail; +else + n = cq->head - cq->tail; +if (unlikely ((u32 )cqe < n)) { +spin_unlock_irq (&cq->lock); +vfree (wc); + ret = -EOVERFLOW ; + goto bail; + } +for (n =0 ; cq->tail != cq->head; n++) { + wc[n] = cq->queue[cq->tail]; +if (cq->tail == cq->ibcq.cqe) + cq->tail =0 ; +else + cq->tail++; + } + cq->ibcq.cqe = cqe; + cq->head = n; + cq->tail =0 ; + old_wc = cq->queue; + cq->queue = wc; +spin_unlock_irq (&cq->lock); + +vfree (old_wc); + + ret =0 ; + +bail: +return ret; +}
drivers/infiniband/hw/ipath/ipath_keys.c new file +236 -0
236 lines not on this page · raw message
drivers/infiniband/hw/ipath/ipath_mr.c new file +383 -0
383 lines not on this page · raw message
drivers/infiniband/hw/ipath/ipath_srq.c new file +273 -0
273 lines not on this page · raw message
PATCH 1/1 ipath - char devices for diagnostics and lightweight subnet management
In reply to Bryan O'SullivanThe ipath_diag.c file permits userspace diagnostic tools to read and write a chip's registers. It is different in purpose from the mmap interfaces to the /sys/bus/pci resource files.
The ipath_sma.c file supports a lightweight userspace subnet management agent (SMA). This is used in deployments (such as HPC clusters) where a full Infiniband protocol stack is not needed. The facilities provided by ipath_sma.c are also used by userspace hardware diagnostic tools.
drivers/infiniband/hw/ipath/ipath_diag.c new file +379 -0
@@ -0,0 +1,379 @@ +/* +* Copyright (c) 2003, 2004, 2005, 2006 PathScale, Inc. All rights reserved. +* +* This software is available to you under a choice of one of two +* licenses. You may choose to be licensed under the terms of the GNU +* General Public License (GPL) Version 2, available from the file +* COPYING in the main directory of this source tree, or the +* OpenIB.org BSD license below: +* +* Redistribution and use in source and binary forms, with or +* without modification, are permitted provided that the following +* conditions are met: +* +* - Redistributions of source code must retain the above +* copyright notice, this list of conditions and the following +* disclaimer. +* +* - Redistributions in binary form must reproduce the above +* copyright notice, this list of conditions and the following +* disclaimer in the documentation and/or other materials +* provided with the distribution. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS +* BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN +* ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +* SOFTWARE. +*/ + +/* +* This file contains support for diagnostic functions. It is accessed by +* opening the ipath_diag device, normally minor number 129. Diagnostic use +* of the InfiniPath chip may render the chip or board unusable until the +* driver is unloaded, or in some cases, until the system is rebooted. +* +* Accesses to the chip through this interface are not similar to going +* through the /sys/bus/pci resource mmap interface. +*/ + +#include <linux/pci.h> +#include <asm/uaccess.h> + +#include "ipath_common.h" +#include "ipath_kernel.h" +#include "ips_common.h" +#include "ipath_layer.h" + +int ipath_diag_inuse; +static int diag_set_link; + +static int ipath_diag_open (struct inode *in,struct file *fp); +static int ipath_diag_release (struct inode *in,struct file *fp); +static ssize_t ipath_diag_read (struct file *fp,char __user *data, +size_t count,loff_t *off); +static ssize_t ipath_diag_write (struct file *fp,const char __user *data, +size_t count,loff_t *off); + +static struct file_operations diag_file_ops = { + .owner =THIS_MODULE , + .write = ipath_diag_write, + .read = ipath_diag_read, + .open = ipath_diag_open, + .release = ipath_diag_release +}; + +static struct cdev *diag_cdev; +static struct class_device *diag_class_dev; + +int ipath_diag_init (void ) +{ +return ipath_cdev_init (IPATH_DIAG_MINOR ,"ipath_diag" , + &diag_file_ops, &diag_cdev, &diag_class_dev); +} + +void ipath_diag_cleanup (void ) +{ +ipath_cdev_cleanup (&diag_cdev, &diag_class_dev); +} + +/** +* ipath_read_umem64 - read a 64-bit quantity from the chip into user space +* @dd: the infinipath device +* @uaddr: the location to store the data in user memory +* @caddr: the source chip address (full pointer, not offset) +* @count: number of bytes to copy (multiple of 32 bits) +* +* This function also localizes all chip memory accesses. +* The copy should be written such that we read full cacheline packets +* from the chip. This is usually used for a single qword +* +* NOTE: This assumes the chip address is 64-bit aligned. +*/ +static int ipath_read_umem64 (struct ipath_devdata *dd,void __user *uaddr, +const void __iomem *caddr,size_t count) +{ +const u64 __iomem *reg_addr = caddr; +const u64 __iomem *reg_end = reg_addr + (count /sizeof (u64)); +int ret; + +/* not very efficient, but it works for now */ +if (reg_addr < dd->ipath_kregbase || + reg_end > dd->ipath_kregend) { + ret = -EINVAL ; + goto bail; + } +while (reg_addr < reg_end) { +u64 data =readq (reg_addr); +if (copy_to_user (uaddr, &data,sizeof (u64))) { + ret = -EFAULT ; + goto bail; + } + reg_addr++; + uaddr++; + } + ret =0 ; +bail: +return ret; +} + +/** +* ipath_write_umem64 - write a 64-bit quantity to the chip from user space +* @dd: the infinipath device +* @caddr: the destination chip address (full pointer, not offset) +* @uaddr: the source of the data in user memory +* @count: the number of bytes to copy (multiple of 32 bits) +* +* This is usually used for a single qword +* NOTE: This assumes the chip address is 64-bit aligned. +*/ + +static int ipath_write_umem64 (struct ipath_devdata *dd,void __iomem *caddr, +const void __user *uaddr,size_t count) +{ +u64 __iomem *reg_addr = caddr; +const u64 __iomem *reg_end = reg_addr + (count /sizeof (u64)); +int ret; + +/* not very efficient, but it works for now */ +if (reg_addr < dd->ipath_kregbase || + reg_end > dd->ipath_kregend) { + ret = -EINVAL ; + goto bail; + } +while (reg_addr < reg_end) { +u64 data; +if (copy_from_user (&data, uaddr,sizeof (data))) { + ret = -EFAULT ; + goto bail; + } +writeq (data, reg_addr); + + reg_addr++; + uaddr++; + } + ret =0 ; +bail: +return ret; +} + +/** +* ipath_read_umem32 - read a 32-bit quantity from the chip into user space +* @dd: the infinipath device +* @uaddr: the location to store the data in user memory +* @caddr: the source chip address (full pointer, not offset) +* @count: number of bytes to copy +* +* read 32 bit values, not 64 bit; for memories that only +* support 32 bit reads; usually a single dword. +*/ +static int ipath_read_umem32 (struct ipath_devdata *dd,void __user *uaddr, +const void __iomem *caddr,size_t count) +{ +const u32 __iomem *reg_addr = caddr; +const u32 __iomem *reg_end = reg_addr + (count /sizeof (u32)); +int ret; + +if (reg_addr < (u32 __iomem *) dd->ipath_kregbase || + reg_end > (u32 __iomem *) dd->ipath_kregend) { + ret = -EINVAL ; + goto bail; + } +/* not very efficient, but it works for now */ +while (reg_addr < reg_end) { +u32 data =readl (reg_addr); +if (copy_to_user (uaddr, &data,sizeof (data))) { + ret = -EFAULT ; + goto bail; + } + + reg_addr++; + uaddr++; + } + ret =0 ; +bail: +return ret; +} + +/** +* ipath_write_umem32 - write a 32-bit quantity to the chip from user space +* @dd: the infinipath device +* @caddr: the destination chip address (full pointer, not offset) +* @uaddr: the source of the data in user memory +* @count: number of bytes to copy +* +* write 32 bit values, not 64 bit; for memories that only +* support 32 bit write; usually a single dword. +*/ + +static int ipath_write_umem32 (struct ipath_devdata *dd,void __iomem *caddr, +const void __user *uaddr,size_t count) +{ +u32 __iomem *reg_addr = caddr; +const u32 __iomem *reg_end = reg_addr + (count /sizeof (u32)); +int ret; + +if (reg_addr < (u32 __iomem *) dd->ipath_kregbase || + reg_end > (u32 __iomem *) dd->ipath_kregend) { + ret = -EINVAL ; + goto bail; + } +while (reg_addr < reg_end) { +u32 data; +if (copy_from_user (&data, uaddr,sizeof (data))) { + ret = -EFAULT ; + goto bail; + } +writel (data, reg_addr); + + reg_addr++; + uaddr++; + } + ret =0 ; +bail: +return ret; +} + +static int ipath_diag_open (struct inode *in,struct file *fp) +{ +struct ipath_devdata *dd; +int unit =0 ;/* XXX this is bogus */ +unsigned long flags; +int ret; + + dd =ipath_lookup (unit); + +mutex_lock (&ipath_mutex); +spin_lock_irqsave (&ipath_devs_lock, flags); + +if (ipath_diag_inuse) { + ret = -EBUSY ; + goto bail; + } + +list_for_each_entry (dd, &ipath_dev_list, ipath_list) { +/* +* we need at least one infinipath device to be present +* (don't use INITTED, because we want to be able to open +* even if device is in freeze mode, which cleared INITTED). +* There is a small amount of risk to this, which is why we +* also verify kregbase is set. +*/ + +if (!(dd->ipath_flags &IPATH_PRESENT ) || + !dd->ipath_kregbase) +continue ; + + ipath_diag_inuse =1 ; + diag_set_link =0 ; + ret =0 ; + goto bail; + } + + ret = -ENODEV ; + +bail: +spin_unlock_irqrestore (&ipath_devs_lock, flags); +mutex_unlock (&ipath_mutex); + +/* Only expose a way to reset the device if we +make it into diag mode. */ +if (ret ==0 ) +ipath_expose_reset (&dd->pcidev->dev); + +return ret; +} + +static int ipath_diag_release (struct inode *i,struct file *f) +{ +mutex_lock (&ipath_mutex); + ipath_diag_inuse =0 ; +mutex_unlock (&ipath_mutex); +return 0 ; +} + +static ssize_t ipath_diag_read (struct file *fp,char __user *data, +size_t count,loff_t *off) +{ +int unit =0 ;/* XXX provide for reads on other units some day */ +struct ipath_devdata *dd; +void __iomem *kreg_base; +ssize_t ret; + + dd =ipath_lookup (unit); +if (!dd) { + ret = -ENODEV ; + goto bail; + } + + kreg_base = dd->ipath_kregbase; + +if (count ==0 ) + ret =0 ; +else if ((count %4 ) || (*off %4 )) +/* address or length is not 32-bit aligned, hence invalid */ + ret = -EINVAL ; +else if ((count %8 ) || (*off %8 )) +/* address or length not 64-bit aligned; do 32-bit reads */ + ret =ipath_read_umem32 (dd, data, kreg_base + *off, count); +else + ret =ipath_read_umem64 (dd, data, kreg_base + *off, count); + +if (ret >=0 ) { + *off += count; + ret = count; + } + +bail: +return ret; +} + +static ssize_t ipath_diag_write (struct file *fp,const char __user *data, +size_t count,loff_t *off) +{ +int unit =0 ;/* XXX this is bogus */ +struct ipath_devdata *dd; +void __iomem *kreg_base; +ssize_t ret; + + dd =ipath_lookup (unit); +if (!dd) { + ret = -ENODEV ; + goto bail; + } + kreg_base = dd->ipath_kregbase; + +if (count ==0 ) + ret =0 ; +else if ((count %4 ) || (*off %4 )) +/* address or length is not 32-bit aligned, hence invalid */ + ret = -EINVAL ; +else if ((count %8 ) || (*off %8 )) +/* address or length not 64-bit aligned; do 32-bit writes */ + ret =ipath_write_umem32 (dd, kreg_base + *off, data, count); +else + ret =ipath_write_umem64 (dd, kreg_base + *off, data, count); + +if (ret >=0 ) { + *off += count; + ret = count; + } + +bail: +return ret; +} + +void ipath_diag_bringup_link (struct ipath_devdata *dd) +{ +if (diag_set_link || (dd->ipath_flags &IPATH_LINKACTIVE )) +return ; + + diag_set_link =1 ; +ipath_cdbg (VERBOSE ,"Trying to set to set link active for " +"diag pkt\n" ); +ipath_layer_set_linkstate (dd,IPATH_IB_LINKARM ); +ipath_layer_set_linkstate (dd,IPATH_IB_LINKACTIVE ); +}
drivers/infiniband/hw/ipath/ipath_sma.c new file +390 -0
390 lines not on this page · raw message
Roland Dreier
AppliedBryan> Hi - This is a submission of the ipath driver for inclusion Bryan> in 2.6.17. Andrew, if this looks good to you, please Bryan> apply.
It's customary to work through subsystem maintainers to merge new drivers...
Bryan> We have addressed all earlier rounds of feedback; the Bryan> driver is stable; it compiles with no compiler or sparse Bryan> warnings against current -git (it's comprehensively Bryan> annotated for sparse); and I think it's in good shape. We Bryan> have gone to great lengths over the past several months to Bryan> make it an exemplary kernel citizen.
I merged this series into my git tree at
git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git ipath
I fixed a couple of minor whitespace problems and made it actually build (hint: the constant RDMA_NODE_IB_CA does not exist in the upstream kernel).
It seems you need to fix up your Kconfig dependencies somewhat. On x86_64 allnoconfig + CONFIG_PCI=y, CONFIG_PCI_MSI=y, CONFIG_INFINIBAND=y, CONFIG_IPATH_CORE=y, CONFIG_INFINIBAND_IPATH=y, I get:
drivers/built-in.o: In function ipath_free_pddata': undefined reference to kfree_skb' drivers/built-in.o: In function ipath_alloc_skb': undefined reference to __alloc_skb' drivers/built-in.o: In function ipath_kreceive': undefined reference to skb_over_panic' drivers/built-in.o: In function ipath_init_chip': undefined reference to kfree_skb'
It also looks like there are a few problems on ia64:
drivers/infiniband/hw/ipath/ipath_verbs.c:733: warning: implicit declaration of function atomic_set_mask' drivers/infiniband/hw/ipath/ipath_verbs.c:734: warning: implicit declaration of function atomic_clear_mask' drivers/infiniband/hw/ipath/ipath_verbs.c: In function `show_stats':
drivers/infiniband/hw/ipath/ipath_verbs.c:1184: warning: long long unsigned int format, u64 arg (arg 4) drivers/infiniband/hw/ipath/ipath_verbs.c:1184: warning: long long unsigned int format, u64 arg (arg 5) drivers/infiniband/hw/ipath/ipath_pe800.c: In function `ipath_pe_handle_hwerrors':
drivers/infiniband/hw/ipath/ipath_pe800.c:353: warning: long long unsigned int format, long unsigned int arg (arg 5) drivers/infiniband/hw/ipath/ipath_pe800.c:353: warning: long long unsigned int format, long unsigned int arg (arg 3)
and then the build fails with:
drivers/built-in.o(.text+0x133aa2): In function `ipath_modify_port':
: undefined reference to atomic_set_mask' drivers/built-in.o(.text+0x133ac2): In function ipath_modify_port':
: undefined reference to `atomic_clear_mask'
I also wouldn't say the driver is sparse clean. I get 270 lines of warnings when I try it:
drivers/infiniband/hw/ipath/ipath_mad.c:103:17: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_mad.c:103:17: expected restricted unsigned long long [usertype] node_guid
drivers/infiniband/hw/ipath/ipath_mad.c:103:17: got unsigned long long
drivers/infiniband/hw/ipath/ipath_mad.c:289:9: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_mad.c:289:9: expected unsigned short [unsigned] [short] [usertype] <noident>
drivers/infiniband/hw/ipath/ipath_mad.c:289:9: got restricted unsigned short [usertype] [force] <noident>
drivers/infiniband/hw/ipath/ipath_mad.c:537:10: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_mad.c:537:10: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_mad.c:537:10: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_mad.c:902:27: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_mad.c:902:27: expected restricted unsigned short [usertype] symbol_error_counter
drivers/infiniband/hw/ipath/ipath_mad.c:902:27: got int
drivers/infiniband/hw/ipath/ipath_mad.c:916:22: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_mad.c:916:22: expected restricted unsigned short [usertype] port_rcv_errors
drivers/infiniband/hw/ipath/ipath_mad.c:916:22: got int
drivers/infiniband/hw/ipath/ipath_mad.c:921:30: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_mad.c:921:30: expected restricted unsigned short [usertype] port_rcv_remphys_errors
drivers/infiniband/hw/ipath/ipath_mad.c:921:30: got int
drivers/infiniband/hw/ipath/ipath_mad.c:926:25: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_mad.c:926:25: expected restricted unsigned short [usertype] port_xmit_discards
drivers/infiniband/hw/ipath/ipath_mad.c:926:25: got int
drivers/infiniband/hw/ipath/ipath_mad.c:931:21: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_mad.c:931:21: expected restricted unsigned int [usertype] port_xmit_data
drivers/infiniband/hw/ipath/ipath_mad.c:931:21: got unsigned int
drivers/infiniband/hw/ipath/ipath_mad.c:935:20: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_mad.c:935:20: expected restricted unsigned int [usertype] port_rcv_data
drivers/infiniband/hw/ipath/ipath_mad.c:935:20: got unsigned int
drivers/infiniband/hw/ipath/ipath_mad.c:939:24: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_mad.c:939:24: expected restricted unsigned int [usertype] port_xmit_packets
drivers/infiniband/hw/ipath/ipath_mad.c:939:24: got unsigned int
drivers/infiniband/hw/ipath/ipath_mad.c:944:23: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_mad.c:944:23: expected restricted unsigned int [usertype] port_rcv_packets
drivers/infiniband/hw/ipath/ipath_mad.c:944:23: got unsigned int
drivers/infiniband/hw/ipath/ipath_qp.c:581:9: warning: incorrect type in return expression (different base types)
drivers/infiniband/hw/ipath/ipath_qp.c:581:9: expected unsigned int
drivers/infiniband/hw/ipath/ipath_qp.c:581:9: got restricted unsigned int [usertype] [force] <noident>
drivers/infiniband/hw/ipath/ipath_rc.c:103:16: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_rc.c:103:16: expected restricted unsigned int [usertype] aeth
drivers/infiniband/hw/ipath/ipath_rc.c:103:16: got unsigned int
drivers/infiniband/hw/ipath/ipath_rc.c:116:17: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_rc.c:116:17: expected restricted unsigned int [usertype] aeth
drivers/infiniband/hw/ipath/ipath_rc.c:116:17: got unsigned int
drivers/infiniband/hw/ipath/ipath_rc.c:147:19: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_rc.c:147:19: expected restricted unsigned int [usertype] aeth
drivers/infiniband/hw/ipath/ipath_rc.c:147:19: got unsigned int
drivers/infiniband/hw/ipath/ipath_rc.c:158:16: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_rc.c:158:16: expected restricted unsigned int [usertype] aeth
drivers/infiniband/hw/ipath/ipath_rc.c:158:16: got unsigned int
drivers/infiniband/hw/ipath/ipath_rc.c:519:45: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_rc.c:519:45: expected restricted unsigned long long [usertype] interface_id
drivers/infiniband/hw/ipath/ipath_rc.c:519:45: got unsigned long long
drivers/infiniband/hw/ipath/ipath_rc.c:519:45: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_rc.c:519:45: expected restricted unsigned long long [usertype] interface_id
drivers/infiniband/hw/ipath/ipath_rc.c:519:45: got unsigned long long
drivers/infiniband/hw/ipath/ipath_rc.c:656:15: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_rc.c:656:15: expected restricted unsigned int [usertype] aeth
drivers/infiniband/hw/ipath/ipath_rc.c:656:15: got unsigned int
drivers/infiniband/hw/ipath/ipath_rc.c:1456:10: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_rc.c:1456:10: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_rc.c:1456:10: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_rc.c:1147:12: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_rc.c:1147:12: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_rc.c:1147:12: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_rc.c:1163:11: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_rc.c:1163:11: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_rc.c:1163:11: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_rc.c:1253:11: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_rc.c:1253:11: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_rc.c:1253:11: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_rc.c:1609:16: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_rc.c:1609:16: expected restricted unsigned int [assigned] [usertype] imm_data
drivers/infiniband/hw/ipath/ipath_rc.c:1609:16: got unsigned int [unsigned] [usertype] <noident>
drivers/infiniband/hw/ipath/ipath_rc.c:1650:24: warning: incorrect type in argument 3 (different base types)
drivers/infiniband/hw/ipath/ipath_rc.c:1650:24: expected int [signed] sig
drivers/infiniband/hw/ipath/ipath_rc.c:1650:24: got restricted unsigned int
drivers/infiniband/hw/ipath/ipath_uc.c:288:46: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_uc.c:288:46: expected restricted unsigned long long [usertype] interface_id
drivers/infiniband/hw/ipath/ipath_uc.c:288:46: got unsigned long long
drivers/infiniband/hw/ipath/ipath_uc.c:362:10: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_uc.c:362:10: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_uc.c:362:10: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_uc.c:473:16: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_uc.c:473:16: expected restricted unsigned int [assigned] [usertype] imm_data
drivers/infiniband/hw/ipath/ipath_uc.c:473:16: got unsigned int [unsigned] [usertype] <noident>
drivers/infiniband/hw/ipath/ipath_uc.c:517:24: warning: incorrect type in argument 3 (different base types)
drivers/infiniband/hw/ipath/ipath_uc.c:517:24: expected int [signed] sig
drivers/infiniband/hw/ipath/ipath_uc.c:517:24: got restricted unsigned int
drivers/infiniband/hw/ipath/ipath_uc.c:602:16: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_uc.c:602:16: expected restricted unsigned int [addressable] [assigned] [usertype] imm_data
drivers/infiniband/hw/ipath/ipath_uc.c:602:16: got unsigned int [unsigned] [usertype] <noident>
drivers/infiniband/hw/ipath/ipath_ud.c:321:46: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_ud.c:321:46: expected restricted unsigned long long [usertype] interface_id
drivers/infiniband/hw/ipath/ipath_ud.c:321:46: got unsigned long long
drivers/infiniband/hw/ipath/ipath_ud.c:456:11: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_ud.c:456:11: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_ud.c:456:11: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_ud.c:457:13: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_ud.c:457:13: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_ud.c:457:13: warning: cast to restricted type drivers/infiniband/hw/ipath/ipath_ud.c:526:16: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_ud.c:526:16: expected restricted unsigned int [assigned] [usertype] imm_data
drivers/infiniband/hw/ipath/ipath_ud.c:526:16: got unsigned int [unsigned] [usertype] <noident>
drivers/infiniband/hw/ipath/ipath_ud.c:617:23: warning: incorrect type in argument 3 (different base types)
drivers/infiniband/hw/ipath/ipath_ud.c:617:23: expected int [signed] sig
drivers/infiniband/hw/ipath/ipath_ud.c:617:23: got restricted unsigned int
drivers/infiniband/hw/ipath/ipath_verbs.c:654:19: error: incompatible types in conditional expression (different base types) drivers/infiniband/hw/ipath/ipath_copy.c:104:2: warning: incorrect type in argument 4 (different base types)
drivers/infiniband/hw/ipath/ipath_copy.c:104:2: expected int
drivers/infiniband/hw/ipath/ipath_copy.c:104:2: got restricted unsigned short <noident>
drivers/infiniband/hw/ipath/ipath_copy.c:104:2: warning: incorrect type in argument 5 (different base types)
drivers/infiniband/hw/ipath/ipath_copy.c:104:2: expected int
drivers/infiniband/hw/ipath/ipath_copy.c:104:2: got restricted unsigned short <noident>
drivers/infiniband/hw/ipath/ipath_copy.c:104:2: warning: incorrect type in argument 6 (different base types)
drivers/infiniband/hw/ipath/ipath_copy.c:104:2: expected int
drivers/infiniband/hw/ipath/ipath_copy.c:104:2: got restricted unsigned short <noident>
drivers/infiniband/hw/ipath/ipath_copy.c:104:2: warning: incorrect type in argument 7 (different base types)
drivers/infiniband/hw/ipath/ipath_copy.c:104:2: expected int
drivers/infiniband/hw/ipath/ipath_copy.c:104:2: got restricted unsigned short <noident>
drivers/infiniband/hw/ipath/ipath_copy.c:432:14: error: incompatible types for operation (!=) drivers/infiniband/hw/ipath/ipath_driver.c:669:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:671:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:673:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:675:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:677:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:679:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:681:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:683:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:685:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:687:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:689:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:691:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:693:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:695:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:697:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:699:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:701:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:703:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:705:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:707:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:709:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:711:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:713:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:715:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:717:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:719:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:721:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:723:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:725:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:727:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:729:10: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_driver.c:1075:4: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:290:5: warning: incorrect type in initializer (different base types)
drivers/infiniband/hw/ipath/ipath_ht400.c:290:5: expected restricted unsigned long long static const [toplevel] [usertype] infinipath_hwe_htcmemparityerr_mask
drivers/infiniband/hw/ipath/ipath_ht400.c:290:5: got unsigned long long
drivers/infiniband/hw/ipath/ipath_ht400.c:295:5: warning: incorrect type in initializer (different base types)
drivers/infiniband/hw/ipath/ipath_ht400.c:295:5: expected restricted unsigned long long static const [toplevel] [usertype] infinipath_hwe_htclnkabyte0crcerr
drivers/infiniband/hw/ipath/ipath_ht400.c:295:5: got unsigned long long
drivers/infiniband/hw/ipath/ipath_ht400.c:297:5: warning: incorrect type in initializer (different base types)
drivers/infiniband/hw/ipath/ipath_ht400.c:297:5: expected restricted unsigned long long static const [toplevel] [usertype] infinipath_hwe_htclnkabyte1crcerr
drivers/infiniband/hw/ipath/ipath_ht400.c:297:5: got unsigned long long
drivers/infiniband/hw/ipath/ipath_ht400.c:299:5: warning: incorrect type in initializer (different base types)
drivers/infiniband/hw/ipath/ipath_ht400.c:299:5: expected restricted unsigned long long static const [toplevel] [usertype] infinipath_hwe_htclnkbbyte0crcerr
drivers/infiniband/hw/ipath/ipath_ht400.c:299:5: got unsigned long long
drivers/infiniband/hw/ipath/ipath_ht400.c:301:5: warning: incorrect type in initializer (different base types)
drivers/infiniband/hw/ipath/ipath_ht400.c:301:5: expected restricted unsigned long long static const [toplevel] [usertype] infinipath_hwe_htclnkbbyte1crcerr
drivers/infiniband/hw/ipath/ipath_ht400.c:301:5: got unsigned long long
drivers/infiniband/hw/ipath/ipath_ht400.c:346:17: warning: cast from restricted type drivers/infiniband/hw/ipath/ipath_ht400.c:382:8: warning: incorrect type in argument 3 (different base types)
drivers/infiniband/hw/ipath/ipath_ht400.c:382:8: expected unsigned long long [unsigned] [usertype] value
drivers/infiniband/hw/ipath/ipath_ht400.c:382:8: got restricted unsigned long long [usertype] ipath_hwerrmask
drivers/infiniband/hw/ipath/ipath_ht400.c:385:3: warning: cast from restricted type drivers/infiniband/hw/ipath/ipath_ht400.c:411:9: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_ht400.c:411:9: expected restricted unsigned long long [usertype] hwerrs
drivers/infiniband/hw/ipath/ipath_ht400.c:411:9: got unsigned long long
drivers/infiniband/hw/ipath/ipath_ht400.c:421:20: error: incompatible types for operation (==) drivers/infiniband/hw/ipath/ipath_ht400.c:434:11: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:448:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:449:3: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:449:3: error: cast from unknown type drivers/infiniband/hw/ipath/ipath_ht400.c:449:3: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:449:3: error: cast from unknown type drivers/infiniband/hw/ipath/ipath_ht400.c:492:9: error: incompatible types for operation (<<) drivers/infiniband/hw/ipath/ipath_ht400.c:493:25: error: incompatible types for operation (>>) drivers/infiniband/hw/ipath/ipath_ht400.c:493:11: error: cast from unknown type drivers/infiniband/hw/ipath/ipath_ht400.c:500:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:502:25: error: incompatible types for operation (>>) drivers/infiniband/hw/ipath/ipath_ht400.c:502:11: error: cast from unknown type drivers/infiniband/hw/ipath/ipath_ht400.c:509:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:511:25: error: incompatible types for operation (>>) drivers/infiniband/hw/ipath/ipath_ht400.c:511:11: error: cast from unknown type drivers/infiniband/hw/ipath/ipath_ht400.c:518:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:520:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:522:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:524:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:526:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:532:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:534:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:536:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:538:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:553:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:556:34: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:556:6: error: cast from unknown type drivers/infiniband/hw/ipath/ipath_ht400.c:559:35: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:564:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:574:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_ht400.c:576:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_init_chip.c:415:24: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_init_chip.c:415:24: expected restricted unsigned long long volatile [usertype] *ipath_hdrqtailptr
drivers/infiniband/hw/ipath/ipath_init_chip.c:415:24: got unsigned long long [usertype] *extern [addressable] [toplevel] ipath_port0_rcvhdrtail
drivers/infiniband/hw/ipath/ipath_init_chip.c:822:7: warning: incorrect type in argument 3 (different base types)
drivers/infiniband/hw/ipath/ipath_init_chip.c:822:7: expected unsigned long long [unsigned] [usertype] value
drivers/infiniband/hw/ipath/ipath_init_chip.c:822:7: got restricted unsigned long long [usertype] ipath_hwerrmask
drivers/infiniband/hw/ipath/ipath_init_chip.c:829:5: warning: incorrect type in argument 3 (different base types)
drivers/infiniband/hw/ipath/ipath_init_chip.c:829:5: expected unsigned long long [unsigned] [usertype] value
drivers/infiniband/hw/ipath/ipath_init_chip.c:829:5: got restricted unsigned long long [usertype] ipath_maskederrs
drivers/infiniband/hw/ipath/ipath_intr.c:103:12: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_intr.c:297:26: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_intr.c:298:57: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_intr.c:300:27: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_intr.c:370:11: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_intr.c:375:24: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_intr.c:376:3: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_intr.c:376:3: error: cast from unknown type drivers/infiniband/hw/ipath/ipath_intr.c:376:3: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_intr.c:376:3: error: cast from unknown type drivers/infiniband/hw/ipath/ipath_intr.c:380:11: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_intr.c:397:55: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_intr.c:432:31: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_intr.c:444:11: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_intr.c:452:11: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_intr.c:456:11: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_intr.c:459:11: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_intr.c:471:11: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_intr.c:503:11: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_intr.c:521:11: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_intr.c:539:11: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_intr.c:542:11: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_intr.c:770:18: error: incompatible types for operation (==) drivers/infiniband/hw/ipath/ipath_layer.c:1083:14: error: incompatible types for operation (!=) drivers/infiniband/hw/ipath/ipath_pe800.c:316:9: warning: incorrect type in assignment (different base types)
drivers/infiniband/hw/ipath/ipath_pe800.c:316:9: expected restricted unsigned long long [usertype] hwerrs
drivers/infiniband/hw/ipath/ipath_pe800.c:316:9: got unsigned long long
drivers/infiniband/hw/ipath/ipath_pe800.c:325:20: error: incompatible types for operation (==) drivers/infiniband/hw/ipath/ipath_pe800.c:338:11: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_pe800.c:352:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_pe800.c:389:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_pe800.c:398:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_pe800.c:400:25: error: incompatible types for operation (>>) drivers/infiniband/hw/ipath/ipath_pe800.c:400:11: error: cast from unknown type drivers/infiniband/hw/ipath/ipath_pe800.c:407:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_pe800.c:409:25: error: incompatible types for operation (>>) drivers/infiniband/hw/ipath/ipath_pe800.c:409:11: error: cast from unknown type drivers/infiniband/hw/ipath/ipath_pe800.c:416:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_pe800.c:418:25: error: incompatible types for operation (>>) drivers/infiniband/hw/ipath/ipath_pe800.c:418:11: error: cast from unknown type drivers/infiniband/hw/ipath/ipath_pe800.c:425:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_pe800.c:427:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_pe800.c:433:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_pe800.c:439:35: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_pe800.c:444:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_pe800.c:454:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_pe800.c:456:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_pe800.c:466:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_pe800.c:468:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_pe800.c:470:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_pe800.c:472:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_pe800.c:474:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_pe800.c:477:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_pe800.c:479:13: error: incompatible types for operation (&) drivers/infiniband/hw/ipath/ipath_stats.c:243:55: error: incompatible types for operation (&)
Quoted 1 line · Roland Dreier
It's customary to work through subsystem maintainers to merge new drivers...
OK, I'll feed patches to you, then. No inconvenience intended.
Thanks for applying the patches.
Quoted 3 lines
I fixed a couple of minor whitespace problems and made it actually build (hint: the constant
RDMA_NODE_IB_CAdoes not exist in the upstream kernel).
Oops.
Quoted 1 line
It seems you need to fix up your
Kconfigdependencies somewhat.
Will do.
Quoted 8 lines
On
x86_64allnoconfig +CONFIG_PCI=y,CONFIG_PCI_MSI=y,CONFIG_INFINIBAND=y,CONFIG_IPATH_CORE=y,CONFIG_INFINIBAND_IPATH=y, I get:
drivers/built-in.o: In functionipath_free_pddata': undefined reference tokfree_skb'drivers/built-in.o: In functionipath_alloc_skb': undefined reference to__alloc_skb'drivers/built-in.o: In functionipath_kreceive': undefined reference toskb_over_panic'drivers/built-in.o: In functionipath_init_chip': undefined reference tokfree_skb'
Would your preference be to slap #ifdefs around those, or to just require CONFIG_NET in Kconfig? The core driver should work fine without any kernel-level networking support, so I suppose the former makes more sense.
Quoted 10 lines
It also looks like there are a few problems on ia64:
drivers/infiniband/hw/ipath/ipath_verbs.c:733: warning: implicit declaration of functionatomic_set_mask' drivers/infiniband/hw/ipath/ipath_verbs.c:734: warning: implicit declaration of functionatomic_clear_mask'drivers/infiniband/hw/ipath/ipath_verbs.c: In function `show_stats':
drivers/infiniband/hw/ipath/ipath_verbs.c:1184: warning: long long unsigned int format, u64 arg (arg 4)drivers/infiniband/hw/ipath/ipath_verbs.c:1184: warning: long long unsigned int format, u64 arg (arg 5)drivers/infiniband/hw/ipath/ipath_pe800.c: In function `ipath_pe_handle_hwerrors':
drivers/infiniband/hw/ipath/ipath_pe800.c:353: warning: long long unsigned int format, long unsigned int arg (arg 5)drivers/infiniband/hw/ipath/ipath_pe800.c:353: warning: long long unsigned int format, long unsigned int arg (arg 3)
That's going to be interesting to test, because I don't have any ia64 hardware to even compile on. I have tested on x86_64 and powerpc, so this seems like an arch-level header deficiency. Any idea what to do about it?
Quoted 1 line
I also wouldn't say the driver is sparse clean.
I've been building with C=1 for months. I'll see if I can figure out why you're getting such different results.
<b
Quoted 4 lines · Bryan O'Sullivan
That's going to be interesting to test, because I don't have any ia64 hardware to even compile on. I have tested on
x86_64and powerpc, so this seems like an arch-level header deficiency. Any idea what to do about it?
You can build an ia64 (or almost any other arch) toolchain using crosstool and compile test with
it. http://www.kegel.com/crosstool/#download
Cheers,
Muli
Signature · 2 lines
Muli Ben-Yehuda http://www.mulix.org | http://mulix.livejournal.com/
Bryan> Would your preference be to slap #ifdefs around those, or Bryan> to just require CONFIG_NET in Kconfig? The core driver Bryan> should work fine without any kernel-level networking Bryan> support, so I suppose the former makes more sense.
Having #ifdef CONFIG_NET all over is definitely suboptimal. Unfortunately it looks kind of hard to untangle your skb use from the rest of the driver, so putting a dependency on NET might be the best bet.
Bryan> That's going to be interesting to test, because I don't Bryan> have any ia64 hardware to even compile on. I have tested Bryan> on x86_64 and powerpc, so this seems like an arch-level Bryan> header deficiency. Any idea what to do about it?
How are you building on powerpc? I don't see any way to turn on CONFIG_PCI_MSI except on i386/x86_64 and ia64.
Anyway building an ia64 cross toolchain is easy with http://kegel.com/crosstool
I would just get rid of your atomic_clear_mask() and atomic_set_mask() calls. They're bogus because you're not even operating on an atomic_t, and not many architectures implement them. Just take a lock if you need to modify the bitmap atomically. A spinlock is cheaper than two atomic operations (although I guess for a slow path, it hurts in .text size).
Bryan> I've been building with C=1 for months. I'll see if I can Bryan> figure out why you're getting such different results.
It's probably because I use CF=-D__CHECK_ENDIAN__ too.
There are a few other things I don't think we've really closed on:
- The whole duplicated SMA / ipath_verbs doesn't work without ib_mad loaded.
- Andrew raised some questions about the special "pick a device for me" that I'm not sure we satisfied him on. I don't find the /dev/ptmx argument that convincing, since I don't think /dev/ptmx is considered the best example of interface design.
- It looks like ipath_copy.c is completely unused now that you're not including the ipath_ether driver.
- R.
Quoted 3 lines · Roland Dreier
Having #ifdef
CONFIG_NETall over is definitely suboptimal. Unfortunately it looks kind of hard to untangle your skb use from the rest of the driver, so putting a dependency on NET might be the best bet.
I don't think it will be that bad, actually. I'll see how it works to just move skb-based code into a single source file and build it iff CONFIG_NET.
Quoted 1 line
How are you building on powerpc?
I have a hack that lets me compile-test, which I used to make sure I was getting sparse annotations and header inclusions into decent shape.
Quoted 1 line
Anyway building an ia64 cross toolchain is easy with http://kegel.com/crosstool
I'll take a look.
Quoted 3 lines
I would just get rid of your
atomic_clear_mask()andatomic_set_mask()calls. They're bogus because you're not even operating on anatomic_t, and not many architectures implement them.
They're not obviously defined to operate on atomic_t objects, but what you say makes sense. I guess that's a peril of using macros for that stuff.
Quoted 4 lines
Bryan> I've been building with C=1 for months. I'll see if I can Bryan> figure out why you're getting such different results.
It's probably because I use CF=-D__CHECK_ENDIAN__ too.
Ah.
Quoted 1 line
The whole duplicated SMA /
ipath_verbsdoesn't work withoutib_madloaded.
There is no duplicated SMA code. There are two routines in ipathfs that handle nodeinfo and portinfo structures, but they're for passing them to userspace; they don't even really resemble the code in ipath_mad.c.
Regarding "ipath_verbs doesn't work without ib_mad loaded", I don't know that there's a problem there any more. We took out the use of ib_register_mad_agent to create an automatic dependency that depmod would find, and we're just recommending mangling modprobe.conf instead for now.
Quoted 2 lines
- Andrew raised some questions about the special "pick a device for me" that I'm not sure we satisfied him on.
That's possible. Andrew, what's your opinion on that?
Quoted 2 lines
It looks like
ipath_copy.cis completely unused now that you're not including theipath_etherdriver.
That's true; sorry about that. Do you want me to send you a patch that drops it and pulls it out of headers and kbuild stuff?
<b
Roland> I would just get rid of your atomic_clear_mask() and Roland> atomic_set_mask() calls. They're bogus because you're not Roland> even operating on an atomic_t, and not many architectures Roland> implement them.
Bryan> They're not obviously defined to operate on atomic_t Bryan> objects, but what you say makes sense. I guess that's a Bryan> peril of using macros for that stuff.
Not on x86_64, but:
./asm-s390/atomic.h:static __inline__ void atomic_clear_mask(unsigned long mask, atomic_t * v) ./asm-sh/atomic.h:static __inline__ void atomic_clear_mask(unsigned int mask, atomic_t *v) ./asm-sh64/atomic.h:static __inline__ void atomic_clear_mask(unsigned int mask, atomic_t *v)
etc. Some other architectures do use unsigned long instead. It all looks rather non-portable, and the only drivers that use these functions are s390-specific. Clearly the simplest solution for your situation is just to kill it.
Bryan> There is no duplicated SMA code. There are two routines in Bryan> ipathfs that handle nodeinfo and portinfo structures, but Bryan> they're for passing them to userspace; they don't even Bryan> really resemble the code in ipath_mad.c.
We seem to be going around and around on this. There definitely is duplicated code; you just hide some of it in userspace. You clearly have two copies of the function to generate a reply to a GET of NodeInfo, for example.
What if you moved the MAD query handling code into your core driver? You could use your current method of sending and receiving replies directly when ib_ipath isn't loaded, but just process the queries in the kernel without proxying to userspace. Then if/when QP0 is created, switch to letting the MAD layer handle sending and receiving queries and let it call the same query handling code via your process_mad method.
But it would (I think) solve all the issues of needing ib_mad loaded for things to work. Users could even load ib_ipath without ib_mad and have IB verbs work -- anything that actually needed MADs would pull in ib_mad as a dependency, and everything else should work fine with the SMA stuff handled by your driver.
Of course this would expose the duplication of the SMI directed route handling logic between your driver and the IB midlayer. Perhaps it makes sense to move this directed route logic into its own module to fix this.
I just really don't like the fact that you apparently have two different MAD handling methods, with different features. Diluting your development by a factor of 2 isn't good for the code. Also, the way your are working around the IB midlayer just smells wrong -- either the midlayer or your driver should be fixed so that they work well together.
Bryan> That's true; sorry about that. Do you want me to send you Bryan> a patch that drops it and pulls it out of headers and Bryan> kbuild stuff?
Let's just try to get closure on everything and then you can send a real final patchset for merging.
I understand that you really, really want your driver in 2.6.17. But let's do things right. Also, in my opinion, we can still merge ipath even after 2.6.17-rc1, since it doesn't touch anything (except trivial kbuild stuff) outside of drivers/infiniband/hw/ipath.
- R.
Quoted 2 lines · Roland Dreier
Clearly the simplest solution for your
situation is just to kill it.
Yes, I'll do that.
Quoted 4 lines
We seem to be going around and around on this. There definitely is duplicated code; you just hide some of it in userspace. You clearly have two copies of the function to generate a reply to a GET of NodeInfo, for example.
That's true. What I'm not so clear on is why you care that we have a similar facility in userland. The userland SMA is so simple, we haven't had to touch it in a long time, except to use the new ioctl-free ABI. There's negligible duplicated coding effort going on.
Quoted 8 lines
What if you moved the MAD query handling code into your core driver? You could use your current method of sending and receiving replies directly when
ib_ipathisn't loaded, but just process the queries in the kernel without proxying to userspace. Then if/when QP0 is created, switch to letting the MAD layer handle sending and receiving queries and let it call the same query handling code via yourprocess_madmethod.
Quoted 5 lines
But it would (I think) solve all the issues of needing
ib_madloaded for things to work. Users could even loadib_ipathwithoutib_madand have IB verbs work -- anything that actually needed MADs would pull inib_madas a dependency, and everything else should work fine with the SMA stuff handled by your driver.
I'll have to chew over this for a bit with a few other people. I'm not actually trying to be difficult; it's just that changing the SMA at this point is quite disruptive. we have schedules to muck with, plans to accelerate, people to reallocate, and the like.
It would be a huge relief to me to be able to simply merge what we have with the understanding that we'll resolve the SMA issue to your liking as soon as possible, but you're the gatekeeper.
Quoted 1 line
I understand that you really, really want your driver in 2.6.17.
Very much.
Quoted 3 lines
Also, in my opinion, we can still merge ipath
even after 2.6.17-rc1, since it doesn't touch anything (except trivial kbuild stuff) outside ofdrivers/infiniband/hw/ipath.
I hope you're right.
<b