Please pull rdma.git

Hi Linus,

The following changes since commit 4ef7675344d687a0ef5b0d7c0cee12da005870c0:

Linux 4.4-rc6 (2015-12-20 16:06:09 -0800)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/dledford/rdma.git
tags/for-linus

for you to fetch changes up to 34356f64ac0df2326fa50e2d4bca6f7c03ed16c1:

IB/mlx5: Unify CQ create flags check (2016-01-21 12:05:37 -0500)

This is a rather large update, and it's not entirely complete. I have some changes to merge for the staging/rdma tree too, but I wanted to get this set in first for dependency reasons, then I'll make a new branch against Greg's staging tree, then send the changes relative to that. Expect that pull request early next week. Thanks.

----------------------------------------------------------------
Initial roundup of 4.5 merge window patches

- Remove usage of ib_query_device and instead store attributes in ib_device struct
- Move iopoll out of block and into lib, rename to irqpoll, and use in several places in the rdma stack as our new completion queue polling library mechanism. Update the other block drivers that already used iopoll to use the new mechanism too.
- Replace the per-entry GID table locks with a single GID table lock
- IPoIB multicast cleanup
- Cleanups to the IB MR facility
- Add support for 64bit extended IB counters
- Fix for netlink oops while parsing RDMA nl messages
- RoCEv2 support for the core IB code
- mlx4 RoCEv2 support
- mlx5 RoCEv2 support
- Cross Channel support for mlx5
- Timestamp support for mlx5
- Atomic support for mlx5
- Raw QP support for mlx5
- MAINTAINERS update for mlx4/mlx5
- Misc ocrdma, qib, nes, usNIC, cxgb3, cxgb4, mlx4, mlx5 updates
- Add support for remote invalidate to the iSER driver (pushed through the RDMA tree due to dependencies, acknowledged by nab)
- Update to NFSoRDMA (pushed through the RDMA tree due to dependencies, acknowledged by Bruce)

----------------------------------------------------------------
Achiad Shochat (10):
IB/mlx5: Support IB device's callback for getting the link layer IB/mlx5: Support IB device's callback for getting its netdev net/mlx5_core: Break down the vport mac address query function net/mlx5_core: Introduce access functions to enable/disable RoCE net/mlx5_core: Introduce access functions to query vport RoCE fields IB/mlx5: Extend query_device/port to support RoCE IB/mlx5: Set network_hdr_type upon RoCE responder completion IB/mlx5: Support IB device's callbacks for adding/deleting GIDs IB/mlx5: Add RoCE fields to Address Vector
IB/mlx5: Support RoCE

Bart Van Assche (4):
IB/core: Remove set-but-not-used variable from ib_sg_to_pages()
irq_poll: Fix irq_poll_sched()
IB/srpt: Fix the RDMA completion handlers
IB/cm: Fix a recently introduced deadlock

Bodong Wang (1):
IB/mlx5: report tx/rx checksum cap in query results

Christoph Hellwig (23):
irq_poll: make blk-iopoll available outside the block layer
irq_poll: don't disable new irq_poll instances
irq_poll: fold irq_poll_sched_prep into irq_poll_sched
irq_poll: fold irq_poll_disable_pending into irq_poll_softirq
irq_poll: mark __irq_poll_complete static
irq_poll: remove unused data and max fields
IB: add a proper completion queue abstraction IB/srpt: chain RDMA READ/WRITE requests
IB/srp: use the new CQ API
IB/iser: Convert to CQ abstraction
IB: start documenting device capabilities
IB: remove ib_query_mr
IB: remove support for phys MRs
IB: remove in-kernel support for memory windows
cxgb3: simplify iwch_get_dma_wr
nes: simplify nes_reg_phys_mr calling conventions
amso1100: fold c2_reg_phys_mr into c2_get_dma_mr
ehca: stop using struct ib_phys_buf
IB: remove the struct ib_phys_buf definition
IB: remove the write-only usecnt field from struct ib_mr IB/mad: pass ib_mad_send_buf explicitly to the recv_handler IB/mad: use CQ abstraction
svc_rdma: use local_dma_lkey

Christoph Lameter (5):
IB/IPoIB: factor out common multicast list removal code IB/IPoIB: Move multicast specific code out of ipoib_main.c IB/core: Create get_perf_mad function in sysfs.c IB/core: Specify attribute_id in port_table_attribute IB/core: Display extended counter set if available

Chuck Lever (10):
svcrdma: Clean up rdma_create_xprt()
svcrdma: Clean up process_context()
svcrdma: Improve allocation of struct svc_rdma_op_ctxt
svcrdma: Improve allocation of struct svc_rdma_req_map
svcrdma: Remove unused req_map and ctxt kmem_caches
svcrdma: Add gfp flags to svc_rdma_post_recv()
svcrdma: Remove last two __GFP_NOFAIL call sites
svcrdma: Make map_xdr non-static
svcrdma: Define maximum number of backchannel requests
svcrdma: Add class for RDMA backwards direction transport

Dan Carpenter (2):
RDMA/nes: checking for NULL instead of IS_ERR IB/cma: allocating too much memory in make_cma_ports()

Dean Luick (1):
IB/mad: Ensure fairness in ib_mad_completion_handler

Devesh Sharma (4):
RDMA/ocrdma: Fix vlan-id assignment in qp parameters RDMA/ocrdma: Dispatch only port event when port state changes RDMA/ocrdma: Depend on async link events from CNA RDMA/be2net: Remove open and close entry points

Doug Ledford (2):
Merge branch 'rdma-cq.2' of git://git.infradead.org/users/hch/rdma
into 4.5/rdma-cq

      Merge branches '4.5/Or-cleanup' and '4.5/rdma-cq' into k.o/for-4.5

Eran Ben Elisha (2):
net/mlx5_core: Add setting ATOMIC endian mode IB/mlx5: Advertise atomic capabilities in query device

Erez Shitrit (1):
IB/IPoIB: Fix kernel panic on multicast flow

Haggai Abramovsky (3):
IB/mlx5: Fix data validation in mlx5_ib_alloc_ucontext IB/mlx5: Add CQE version 1 support to user QPs and SRQs IB/mlx5: Expose CQE version to user-space

Hal Rosenstock (1):
IB/core: sysfs.c: Fix PerfMgt ClassPortInfo handling

Hariprasad S (5):
iw_cxgb4: Pass qid range to user space driver
iw_cxgb3: Fix incorrectly returning error on success
iw_cxgb4: Fixes static checker warning in c4iw_rdev_open()
iw_cxgb4: Fixes GW-Basic labels to meaningful error names
iw_cxgb4: Take clip reference before starting IPv6 listen

Ira Weiny (2):
IB/core: Save the device attributes on the device structure IB/sysfs: Fix sparse warning on attr_id

Jenny Derzhavetz (5):
IB/iser: Don't register memory for all immediate data writes IB/iser: set intuitive values for mr_valid IB/isert: Declare correct flags when accepting a connection IB/isert: Support the remote invalidation exception IB/iser: Support the remote invalidation exception

Julia Lawall (4):
IB/usnic: delete unneeded IS_ERR test RDMA/nes: constify nes_cm_ops structure IB/iser: constify iser_reg_ops structure IB/core: constify mmu_notifier_ops structures

Kaike Wan (1):
IB/sa: Fix netlink local service GFP crash

Leon Romanovsky (9):
IB/core: Align coding style of ib_device_cap_flags structure IB/core: Add cross-channel support
IB/mlx5: Add driver cross-channel support IB/mlx4: Suppress non-fatal memory allocations IB/mlx4: Convert kmalloc to kmalloc_array for checkpatch IB/mlx5: Expose correct maximum number of CQE capacity IB/mlx5: Delete locally redefined variable IB/mlx5: Fix passing casted pointer in mlx5_query_port_roce IB/mlx5: Unify CQ create flags check

Lucas Tanure (1):
infiniband: Replace memset with eth_zero_addr

Matan Barak (23):
IB/core: Refactor GID cache's ib_dispatch_event IB/core: Change per-entry lock in RoCE GID table to one lock IB/core: don't search the GID table twice IB/core: Add gid_type to gid attribute
IB/cm: Use the source GID index type
IB/core: Add gid attributes to sysfs
IB/core: Add ROCE_UDP_ENCAP (RoCE V2) type IB/core: Move rdma_is_upper_dev_rcu to header file IB/core: Validate route when we init ah IB/rdma_cm: Add wrapper for cma reference count IB/cma: Add configfs for rdma_cm
IB/mlx5: Add create_cq extended command
IB/core: Add ib_is_udata_cleared
IB/mlx5: Add support for hca_core_clock and timestamp_mask IB/mlx5: Add hca_core_clock_offset to udata in init_ucontext IB/mlx5: Mmap the HCA's core clock register to user-space IB/cma: Fix RDMA port validation for iWarp IB/mlx4: Initialize hop_limit when creating address handle IB/core: Eliminate sparse false context imbalance warning IB/core: Fix dereference before check
IB/core: Rename rdma_addr_find_dmac_by_grh IB/core: Use hop-limit from IP stack for RoCE IB/mlx4: Advertise RoCE v2 support

Mike Marciniszyn (2):
IB/qib: fix mcast detach when qp not attached IB/qib: Improve ipoib UD performance

Moni Shoua (15):
IB/core: Initialize UD header structure with IP and UDP headers IB/cma: Join and leave multicast groups with IGMP IB/mlx4: Take source mac from AH instead from the port net/mlx4: Remove unused macro
net/mlx4: Query RoCE support
IB/mlx4: Add gid_type to GID properties
net/mlx4_core: Configure mlx4 hardware for mixed RoCE v1/v2 modes IB/mlx4: Add support for setting RoCEv2 gids in hardware net/mlx4_core: Add support for configuring RoCE v2 UDP port net/mlx4_core: Add support for RoCE v2 entropy IB/core: Add definition for the standard RoCE V2 UDP port IB/mlx4: Support modify_qp for RoCE v2
IB/mlx4: Enable RoCE v2 when the IB device is added IB/mlx4: Enable send of RoCE QP1 packets with IP/UDP headers IB/mlx4: Create and use another QP1 for RoCEv2

Nelson Escobar (7):
IB/usnic: Remove unused prototype
IB/usnic: Improve a failure message
IB/usnic: Fix incorrect cast in usnic_ib_fw_string_to_u64 IB/usnic: Fix message typo
IB/usnic: Support more QP state transitions IB/usnic: Fix resource leak in error case IB/usnic: Handle 0 counts in resource allocation

Or Gerlitz (8):
IB/core: Avoid calling ib_query_device
IB/ulps: Avoid calling ib_query_device
net/rds: Avoid calling ib_query_device
xprtrdma: Avoid calling ib_query_device staging/o2iblnd: Avoid calling ib_query_device
IB/core: Remove ib_query_device
MAINTAINERS: Assign new maintainers to Mellanox mlx5 core and IB
drivers
MAINTAINERS: Assign maintainer to Mellanox mlx4 core and IB drivers

Roi Dayan (1):
IB/iser: Fix module init not cleaning up on error flow

Sagi Grimberg (7):
IB/iser: Use a dedicated descriptor for login IB/iser: Use helper for container_of IB/iser: Reuse ib_sg_to_pages IB/iser,isert: Create and use new shared header IB/isert: Remove unused file iser_proto.h IB/iser: Change the increment rkey flow logic IB/srpt: Remove redundant wc array

Somnath Kotur (1):
IB/core: Add rdma_network_type to wc

Vinit Agnihotri (1):
IB/qib: Support creating qps with GFP_NOIO flag

majd-VPRAkNaXOzVWk0Htik3J/[email protected] (10):
net/mlx5_core: Export transport objects net/mlx5_core: Add RQ and SQ event handling net/mlx5_core: Warn on unsupported events of QP/RQ/SQ IB/mlx5: Allocate a Transport Domain for each ucontext IB/mlx5: Refactor mlx5_ib_qp to accommodate other QP types IB/mlx5: Add create and destroy functionality for Raw Packet QP IB/mlx5: Add Raw Packet QP query functionality IB/mlx5: Support setting Ethernet priority for Raw Packet QPs {IB, net}/mlx5: Move the modify QP operation table to mlx5_ib IB/mlx5: Expose Raw Packet QP to user space consumers

 Documentation/ABI/testing/configfs-rdma_cm         |   22 +
 Documentation/ABI/testing/sysfs-class-infiniband   |   16 +
 Documentation/infiniband/core_locking.txt          |    2 -
 Documentation/kernel-per-CPU-kthreads.txt          |    2 +-
 MAINTAINERS                                        |   32 +-
 block/Makefile                                     |    2 +-
 drivers/infiniband/Kconfig                         |   10 +
 drivers/infiniband/core/Makefile                   |    4 +-
 drivers/infiniband/core/addr.c                     |  194 +++-
 drivers/infiniband/core/cache.c                    |  345 +++++--
 drivers/infiniband/core/cm.c                       |   49 +-
 drivers/infiniband/core/cma.c                      |  265 ++++-
 drivers/infiniband/core/cma_configfs.c             |  321 +++++++
 drivers/infiniband/core/core_priv.h                |   45 +
 drivers/infiniband/core/cq.c                       |  209 ++++
 drivers/infiniband/core/device.c                   |   51 +-
 drivers/infiniband/core/fmr_pool.c                 |   20 +-
 drivers/infiniband/core/mad.c                      |  162 ++--
 drivers/infiniband/core/mad_priv.h                 |    2 +-
 drivers/infiniband/core/multicast.c                |   17 +-
 drivers/infiniband/core/roce_gid_mgmt.c            |   81 +-
 drivers/infiniband/core/sa_query.c                 |   91 +-
 drivers/infiniband/core/sysfs.c                    |  377 +++++++-
 drivers/infiniband/core/ud_header.c                |  155 ++-
 drivers/infiniband/core/umem_odp.c                 |    2 +-
 drivers/infiniband/core/user_mad.c                 |    1 +
 drivers/infiniband/core/uverbs.h                   |    2 +
 drivers/infiniband/core/uverbs_cmd.c               |   38 +-
 drivers/infiniband/core/uverbs_main.c              |   13 +-
 drivers/infiniband/core/uverbs_marshall.c          |    1 +
 drivers/infiniband/core/verbs.c                    |  238 +++--
 drivers/infiniband/hw/cxgb3/iwch_cm.c              |    4 +-
 drivers/infiniband/hw/cxgb3/iwch_cq.c              |    4 -
 drivers/infiniband/hw/cxgb3/iwch_mem.c             |  102 --
 drivers/infiniband/hw/cxgb3/iwch_provider.c        |  146 +--
 drivers/infiniband/hw/cxgb3/iwch_provider.h        |   15 -
 drivers/infiniband/hw/cxgb3/iwch_qp.c              |   82 --
 drivers/infiniband/hw/cxgb4/cm.c                   |   14 +-
 drivers/infiniband/hw/cxgb4/cq.c                   |    3 -
 drivers/infiniband/hw/cxgb4/device.c               |   57 +-
 drivers/infiniband/hw/cxgb4/iw_cxgb4.h             |   13 -
 drivers/infiniband/hw/cxgb4/mem.c                  |  251 -----
 drivers/infiniband/hw/cxgb4/provider.c             |    3 -
 drivers/infiniband/hw/cxgb4/qp.c                   |    5 -
 drivers/infiniband/hw/cxgb4/t4.h                   |    7 +
 drivers/infiniband/hw/cxgb4/user.h                 |    2 +-
 drivers/infiniband/hw/mlx4/ah.c                    |    3 +-
 drivers/infiniband/hw/mlx4/cq.c                    |    3 -
 drivers/infiniband/hw/mlx4/main.c                  |  102 +-
 drivers/infiniband/hw/mlx4/mlx4_ib.h               |   10 +-
 drivers/infiniband/hw/mlx4/mr.c                    |   22 -
 drivers/infiniband/hw/mlx4/qp.c                    |  318 ++++--
 drivers/infiniband/hw/mlx4/srq.c                   |    3 +-
 drivers/infiniband/hw/mlx5/ah.c                    |   32 +-
 drivers/infiniband/hw/mlx5/cq.c                    |   31 +-
 drivers/infiniband/hw/mlx5/main.c                  |  445 ++++++++-
 drivers/infiniband/hw/mlx5/mlx5_ib.h               |  129 ++-
 drivers/infiniband/hw/mlx5/odp.c                   |   29 +-
 drivers/infiniband/hw/mlx5/qp.c                    | 1014
+++++++++++++++++---
 drivers/infiniband/hw/mlx5/srq.c                   |   41 +-
 drivers/infiniband/hw/mlx5/user.h                  |   63 +-
 drivers/infiniband/hw/mthca/mthca_cq.c             |    3 -
 drivers/infiniband/hw/mthca/mthca_provider.c       |   84 --
 drivers/infiniband/hw/mthca/mthca_qp.c             |    2 +-
 drivers/infiniband/hw/nes/nes_cm.c                 |   17 +-
 drivers/infiniband/hw/nes/nes_cm.h                 |    2 +-
 drivers/infiniband/hw/nes/nes_utils.c              |    2 +-
 drivers/infiniband/hw/nes/nes_verbs.c              |  216 +----
 drivers/infiniband/hw/nes/nes_verbs.h              |    4 +
 drivers/infiniband/hw/ocrdma/ocrdma.h              |   10 +
 drivers/infiniband/hw/ocrdma/ocrdma_ah.c           |    7 +-
 drivers/infiniband/hw/ocrdma/ocrdma_hw.c           |   49 +-
 drivers/infiniband/hw/ocrdma/ocrdma_hw.h           |    4 +-
 drivers/infiniband/hw/ocrdma/ocrdma_main.c         |   58 +-
 drivers/infiniband/hw/ocrdma/ocrdma_sli.h          |   49 +-
 drivers/infiniband/hw/ocrdma/ocrdma_verbs.c        |  165 +---
 drivers/infiniband/hw/ocrdma/ocrdma_verbs.h        |    3 -
 drivers/infiniband/hw/qib/qib_mr.c                 |   51 +-
 drivers/infiniband/hw/qib/qib_qp.c                 |   46 +-
 drivers/infiniband/hw/qib/qib_verbs.c              |   12 +-
 drivers/infiniband/hw/qib/qib_verbs.h              |    4 -
 drivers/infiniband/hw/qib/qib_verbs_mcast.c        |   35 +-
 drivers/infiniband/hw/usnic/usnic_debugfs.c        |    5 +-
 drivers/infiniband/hw/usnic/usnic_ib_qp_grp.c      |    4 +-
 drivers/infiniband/hw/usnic/usnic_ib_verbs.c       |   24 +-
 drivers/infiniband/hw/usnic/usnic_ib_verbs.h       |    2 -
 drivers/infiniband/hw/usnic/usnic_vnic.c           |   54 +-
 drivers/infiniband/ulp/ipoib/ipoib.h               |    6 +-
 drivers/infiniband/ulp/ipoib/ipoib_cm.c            |   21 +-
 drivers/infiniband/ulp/ipoib/ipoib_ethtool.c       |   14 +-
 drivers/infiniband/ulp/ipoib/ipoib_main.c          |   40 +-
 drivers/infiniband/ulp/ipoib/ipoib_multicast.c     |   45 +-
 drivers/infiniband/ulp/iser/iscsi_iser.c           |   13 +-
 drivers/infiniband/ulp/iser/iscsi_iser.h           |  156 +--
 drivers/infiniband/ulp/iser/iser_initiator.c       |  323 ++++---
 drivers/infiniband/ulp/iser/iser_memory.c          |  179 ++--
 drivers/infiniband/ulp/iser/iser_verbs.c           |  337 ++-----
 drivers/infiniband/ulp/isert/ib_isert.c            |  118 ++-
 drivers/infiniband/ulp/isert/ib_isert.h            |   41 +-
 drivers/infiniband/ulp/isert/isert_proto.h         |   47 -
 drivers/infiniband/ulp/srp/ib_srp.c                |  205 ++--
 drivers/infiniband/ulp/srp/ib_srp.h                |    7 +-
 drivers/infiniband/ulp/srpt/ib_srpt.c              |  443 +++------
 drivers/infiniband/ulp/srpt/ib_srpt.h              |   53 +-
 drivers/net/ethernet/emulex/benet/be.h             |    2 -
 drivers/net/ethernet/emulex/benet/be_main.c        |    4 -
 drivers/net/ethernet/emulex/benet/be_roce.c        |   36 -
 drivers/net/ethernet/emulex/benet/be_roce.h        |    4 +-
 drivers/net/ethernet/mellanox/mlx4/fw.c            |   39 +-
 drivers/net/ethernet/mellanox/mlx4/mlx4.h          |    5 +-
 drivers/net/ethernet/mellanox/mlx4/port.c          |    7 +
 drivers/net/ethernet/mellanox/mlx4/qp.c            |   26 +
 drivers/net/ethernet/mellanox/mlx5/core/en.h       |    2 +-
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c  |    6 +-
 drivers/net/ethernet/mellanox/mlx5/core/eq.c       |    1 +
 drivers/net/ethernet/mellanox/mlx5/core/main.c     |   61 +-
 drivers/net/ethernet/mellanox/mlx5/core/qp.c       |  233 +++--
 drivers/net/ethernet/mellanox/mlx5/core/srq.c      |    4 +-
 drivers/net/ethernet/mellanox/mlx5/core/transobj.c |   50 +-
 drivers/net/ethernet/mellanox/mlx5/core/vport.c    |  139 ++-
 drivers/scsi/Kconfig                               |    1 +
 drivers/scsi/be2iscsi/Kconfig                      |    1 +
 drivers/scsi/be2iscsi/be.h                         |    4 +-
 drivers/scsi/be2iscsi/be_iscsi.c                   |    4 +-
 drivers/scsi/be2iscsi/be_main.c                    |   20 +-
 drivers/scsi/ipr.c                                 |   25 +-
 drivers/scsi/ipr.h                                 |    4 +-
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd.c    |   21 +-
 drivers/staging/rdma/amso1100/c2_cq.c              |    3 -
 drivers/staging/rdma/amso1100/c2_provider.c        |   72 +-
 drivers/staging/rdma/ehca/ehca_classes.h           |    5 +-
 drivers/staging/rdma/ehca/ehca_iverbs.h            |   16 -
 drivers/staging/rdma/ehca/ehca_main.c              |    4 -
 drivers/staging/rdma/ehca/ehca_mrmw.c              |  477 +--------
 drivers/staging/rdma/ehca/ehca_mrmw.h              |    5 -
 drivers/staging/rdma/ehca/ehca_reqs.c              |    1 -
 drivers/staging/rdma/hfi1/mr.c                     |   51 +-
 drivers/staging/rdma/hfi1/verbs.c                  |    1 -
 drivers/staging/rdma/hfi1/verbs.h                  |    4 -
 drivers/staging/rdma/ipath/ipath_mr.c              |   55 --
 drivers/staging/rdma/ipath/ipath_verbs.c           |    1 -
 drivers/staging/rdma/ipath/ipath_verbs.h           |    4 -
 include/linux/blk-iopoll.h                         |   46 -
 include/linux/interrupt.h                          |    2 +-
 include/linux/irq_poll.h                           |   25 +
 include/linux/mlx4/cmd.h                           |    3 +-
 include/linux/mlx4/device.h                        |   15 +-
 include/linux/mlx4/qp.h                            |   15 +-
 include/linux/mlx5/device.h                        |   45 +-
 include/linux/mlx5/driver.h                        |   20 +-
 include/linux/mlx5/mlx5_ifc.h                      |   50 +-
 include/linux/mlx5/qp.h                            |   46 +-
 .../mlx5/core => include/linux/mlx5}/transobj.h    |   10 +-
 include/linux/mlx5/vport.h                         |    8 +
 include/linux/sunrpc/svc_rdma.h                    |   39 +-
 include/rdma/ib_addr.h                             |   16 +-
 include/rdma/ib_cache.h                            |    4 +
 include/rdma/ib_mad.h                              |    2 +
 include/rdma/ib_pack.h                             |   45 +-
 include/rdma/ib_pma.h                              |    1 +
 include/rdma/ib_sa.h                               |    3 +
 include/rdma/ib_verbs.h                            |  356 ++++---
 include/scsi/iser.h                                |   78 ++
 include/trace/events/irq.h                         |    2 +-
 lib/Kconfig                                        |    5 +
 lib/Makefile                                       |    1 +
 block/blk-iopoll.c => lib/irq_poll.c               |  108 +--
 net/rds/ib.c                                       |   34 +-
 net/rds/iw.c                                       |   23 +-
 net/sunrpc/xprt.c                                  |    1 +
 net/sunrpc/xprtrdma/Makefile                       |    2 +-
 net/sunrpc/xprtrdma/frwr_ops.c                     |    7 +-
 net/sunrpc/xprtrdma/svc_rdma.c                     |   41 +-
 net/sunrpc/xprtrdma/svc_rdma_backchannel.c         |  371 +++++++
 net/sunrpc/xprtrdma/svc_rdma_recvfrom.c            |   56 +-
 net/sunrpc/xprtrdma/svc_rdma_sendto.c              |   33 +-
 net/sunrpc/xprtrdma/svc_rdma_transport.c           |  360 ++++---
 net/sunrpc/xprtrdma/transport.c                    |   30 +-
 net/sunrpc/xprtrdma/verbs.c                        |   24 +-
 net/sunrpc/xprtrdma/xprt_rdma.h                    |   21 +-
 tools/lib/traceevent/event-parse.c                 |    2 +-
 tools/perf/util/trace-event-parse.c                |    2 +-
 182 files changed, 7272 insertions(+), 4760 deletions(-)
 create mode 100644 Documentation/ABI/testing/configfs-rdma_cm
 create mode 100644 Documentation/ABI/testing/sysfs-class-infiniband
 create mode 100644 drivers/infiniband/core/cma_configfs.c
 create mode 100644 drivers/infiniband/core/cq.c
 delete mode 100644 drivers/infiniband/ulp/isert/isert_proto.h
 delete mode 100644 include/linux/blk-iopoll.h
 create mode 100644 include/linux/irq_poll.h
 rename {drivers/net/ethernet/mellanox/mlx5/core =>
include/linux/mlx5}/transobj.h (88%)
 create mode 100644 include/scsi/iser.h
 rename block/blk-iopoll.c => lib/irq_poll.c (55%)
 create mode 100644 net/sunrpc/xprtrdma/svc_rdma_backchannel.c

Doug Ledford <[email protected]>
              GPG KeyID: 0E572FDD

No Doug.

The merge window closes this Sunday.

It's already damn late to send me anything at all, it is *much* too late to say "this is just part of it".

This can all wait for 4.6, and maybe you'll have the pull requests lined up at the *beginning* of the merge window rather than trying to push them in whan I'm already trying to calm things down for the end of it.

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

I obviously have my understanding of the development cycle here wrong. I had thought that the merge window was 4 weeks with a following 8 weeks of RC candidates resulting in roughly quarterly pace. But you tagged v4.4 on 1/10/16, so this Sunday is only 2 weeks. That certainly changes things. I did not believe that the merge window was so short when I sent this request.

See above about the 2 vs. 4 week misconception.

But also, really? Even ignoring that misconception, it's the merge window. And it has a deadline. And if you give me a deadline, then even though I don't try to cut it too close, I, generally speaking, *will* use my available time if needed. But from your description here, there is an additional deadline in the merge window that is an unwritten, unspoken, visceral thing that you feel.

Ok, so be it. A little guidance on that would be appreciated. Is the deadline really 1 week after the merge window opens? 10 days? 4 days?
I ask because this stuff is actually important in terms of planning. I
had PTO and was off in Hawaii when the merge window opened. We all have personal lives, and schedules don't always line up with the merge window. Knowing how small the merge window *really* is is important for me to be able to plan and prioritize actions when I return from something like this. So, for my future planning purpses, some elucidation here would be appreciated.

There were some issues of coordination between work being committed by GKH and myself. I needed to make the last things relative to what he submitted already. My branch with the code on it that I sent the pull request for was based on 4.4-rc6 and therefore did no have GKH's work in it. I didn't like the idea of just blindly merging his branch into mine so that his commits were there and I could make my follow on commits, so it seemed I needed a new branch based upon either GKH's branch that he submitted to you, or on your current master, to plop the last commits into. Given that I thought it was still a couple weeks to go to the close of the window, Monday seemed reasonable. I'll probably still submit the deletes to you sometime tomorrow.

Sure. I can just merge it up to a 4.5-rc sometime then resubmit next window. However, I'm going to have to pull at least a few patches out and send them separate as they are needed fixes for the code in tree already. But those are all -rc appropriate patches anyway so it doesn't matter if they land by Sunday.

Doug Ledford <[email protected]>
              GPG KeyID: 0E572FDD

We've had this model now for ten years by now. Yes, it's really been that long.

Two weeks of merge window, followed by weekly release candidates until it's "ready" (where the most common last one tends to be rc7, but that has certainly fluctuated - we had 3.1 go to rc10 before release, and we've had a few that only got to rc6).

This all is not new, and it's not undocumented. I have absolutely no idea why you suddenly started multiplying all the times by two.

It's actually been very stable over the years. We've had exceptions where travel schedules (or just holidays etc) have moved things around by a week here and there, but on the whole it's actually not varied a lot.

It's a *MERGE* window.

Not a *DEVELOPMENT* window.

See the difference?

This is not new either. The rule has been that things should basically be ready by the time the merge window *opens*. Things should have been in linux-next so that they get some testing, and so that people get a heads-up not only about what is coming up, but how it interacts with other things coming up.

And no, not everything ends up being in linux-next. But I think we've been hitting almost 90% most recent releases, so that whole "things should be in linux-next beforehand" is not just some theoretical goal, it's something we actually hit in practice pretty well.

And yes, I do pull requests until the last minute (although I have been known to hurry up the last minute and release -rc1 a day early just because I don't want people to game my timing).

But generally at the last minute I want to feel that there was some _reason_ for the last minute behavior.

And no, the reason is not "I am hurrying up to get everything done by 5pm on a Friday afternoon, and I didn't even get everything done, so I'll send the rest on Monday".

Because that just makes me go "ok, this is clearly not ready".

[ And as to why the merge window is two weeks and not just "one single day" if everything is supposed to be ready before-hand..

That's actually a valid question, but it's at least partially about
_me_ (but also about flexibility for others, so that we don't have to make the 90% be 100%).

In particular, just the merging process itself would take me one
week. I can do about 20-30 merges a day for the first few days, and I don't feel I could do more while still actually feeling like I had some kind of overview of what I'm merging (the merge itself is often quick, it's looking at it and doing a basic build test etc that takes time).

So one week would be the absolute minimum with the amount of
different trees I merge - and that assumes that every single tree is ready and done at the beginning.

In practice, things slow down after a few days as I catch up with
the queue of pull requests. So in the end the average over the two week period is about 10-15 merges per day for me.

By the end of the two weeks, it really should have turned into a
slow trickle of merges of things that had dependencies or some other valid reason to be late.. Or rather, I _wish_ it turned into a trickle, but sadly the last Friday afternoon is seldom anything but slow.

So two weeks "works" - and has worked for the last ten years. It may
not be perfect, but I think it has worked exceptionally well in the end - certainly better than people thought initially when we started trying this whole no-longer-very-new-thing. ]

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Well Doug is a relatively new maintainer. And the problem we have in the rdma subsystem is a pretty high degree of change with various infrastructure changes (new protocols, new drivers, new features, 100G support, code cleanup ongoing etc). He has trouble keeping up in particular since he has just been back from maternity leave.

But ongoing development in this area is depending on these changes being merged in order to move forward with further development and in order to stabilize what work has been done over the last few month.

So please be generous and lets go through with this merge. We will ensure that things work smoother in the future.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

I never said it was new.

There is a schedule document somewhere?

Only the merge window, not all of it. Probably too many years working internally at Red Hat and dealing with schedules that fall into alignment with quarters without seeming to have any effort to make it that way.

I know the difference, there's no need to resort to yelling.

Certainly. And most of it was there prior to Dec 23rd when I went on PTO. It had linux-next and 0day and other testing. The things I added since then were either A) local to the rdma subsystem or B) had been in linux-next via someone else's tree prior to my going on PTO (the NFSoRDMA changes qualify here) and were moved to my tree to resolve dependency issues after I returned. I made a specific point of preserving this testing, that's part of why my changes were sent as late as they were.

Indeed. I strove for that myself. The only problem here is that there is no positive result from linux-next, only negative. So, I just had to wait and see if I got a notification email about failure or not.

Fair enough. What I'm used to is that a deadline is a deadline and as long as you meet it, you've done your job, no special explanation necessary. As previously mentioned, I *thought* the deadline was later, that was a mistake. But regardless of that, I did meet the final deadline anyway, hence I didn't provide a detailed explanation of my reasons for being on the latter half of the merge window. But from what you just wrote, I get the impression that I should have. The is a difference between what you are looking for and what I dealt with in my previous responsibilities. Difference noted.

I sent the request when I was reasonably sure I wasn't going to get a linux-next failure notification, not when I was rushing to get out the door. You should have gotten it about 9am your time, so I wouldn't have expected you to think this characterization would have been accurate.

I don't think it's as "clearly not ready" as you think, but that's just my opinion ;-)

Thanks for the explanation. It all makes sense to me.

Doug Ledford <[email protected]>
              GPG KeyID: 0E572FDD

For just the schedule? No. But there's this, for example:

Documentation/development-process/2.Process

which talks about it and says "The merge window lasts for approximately two weeks".

The easiest way to find that piece of documentation is actually to just google "merge window". It's the first hit.

So how ready and stable is this? IOW, if I do this pull, can I rely on that being "it", and really just get fixes.

One of the reasons I absolutely detest the "last Friday" pull requests is that it really tends to smell like "this pull request was hurried to hit the merge window". I much prefer seeing early pull requests because that just shows that the subsystem is on top of the timing. And that in turn means that I also get that warm and fuzzy feeling that if the subsystem has its act together, I'll be getting just fixes after the merge window is over.

(The _other_ reason I don't like the last-Friday pull requests and prefer the early ones is that then I'm not hurried, and I can take my time and do the pull requests in sensible groups - filesystems together etc)

So if I end up pulling this, can I rely on getting just fixes for the next ~6-7 weeks?

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

I still want to send the staging changes. But those are actually very
small: there are three drivers in staging/rdma that we put there as a deprecation step and that need to be deleted, and I need to update the maintainers file to exclude staging/rdma from the directory list for the primary staging area so people quit sending patches for it to the normal staging mailing list. The problem we had with staging/rdma is that it's waffled back and forth as to whether Greg or I would handle the patches, but the hfi1 driver is now at a point where it needs to have its patches coordinated with patches in the main rdma tree, and so it was agreed that I would take over staging/rdma and Greg would ignore it, but Greg needed to go ahead and push through all of his staged changes before we put that in place and I'll take over starting with the next for-next cycle. Otherwise, yes, the pull request I sent you is "it" for this release. I have no other queued series or misc items, everything else is relegated to for-next status (of which I have quite a few of those and probably need to open my for-next quickly anyway).

I tried to carefully select what I felt was ready versus should wait when I processed patches upon my return from PTO. There are 171 patches total in the pull request. Of those, 99 have been there since before Dec. 24th. There were 57 that I processed on Monday and Tuesday when I got back from PTO. That broke down to 34 one off patches that include a smattering of things like sparse and other checker fixes, actual oops fixes, and cleanups; 12 patches for the NFSoRDMA stack that were previously in Bruce's tree; and a series specific to the mlx4 driver. I let this sit specifically for linux-next automated runs. On Thursday I decided to take a series I had previously skipped (support for raw QP types in the mlx5 driver) and I took one fix I had previously missed and was brought to my attention, and I took one fix that didn't get authored until that morning and needs to go into your tree even if the rest of the pull request doesn't. I then let that sit overnight again just to get another round of linux-next testing. It goes without saying that I got positive 0day results on all of these, and generally much faster than waiting overnight and checking to see if any failure messages came back.

Yes, that much I can assure you. And if you don't take it, I can also attest that there will be roughly 20 fixes coming your way shortly ;-)

Doug Ledford <[email protected]>
              GPG KeyID: 0E572FDD

I'm not at all convinced I will pull them.

And I'm now looking at pulling the stuff you sent yesterday, and I'm close to just saying "screw this". Again.

This is now the *second* time that two different teams inside Mellanox decided to play games with the kernel maintainers, and I'm not at all sure the end result is not worth my time to sort out.

Doug, you need to stop taking patches from the Mellanox people until the get their shit together. Really. I'll spend some time looking at this mess, but next time I see two Mellanox groups fighting inside their own driver, I will just not pull. It's that simple. If you take shit from them, I'll not take the end result.

I don't know what problem the Mellanox people have, but one group sends their changes through the networking tree, and another group sends it through you. They do similar things, but different enough to not be the same.

You tell them to stop sending that stuff to you, because I'm getting it through Davem. And I'm not interested in cleaning up after their mess.

Adding David and Mellanox people to the cc.

This kind of idiocy where one company has two different groups, and they are fighting over the same driver, and then expecting upstream to sort out their mental problems for them is not acceptable. It's not the job of either me or the subsystem maintainers to sort out your differences for you.

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Understood.

Can I get some specifics of what you are talking about here? The reason I ask is that I know in this merge cycle there was a function modified in the mlx4 driver (I think, or maybe mlx5) where it was first modified by a patch series that went through Dave's tree, then further modified by a series in my tree. In both cases, the modification was part of a larger series that was specific either to the network tree or the rdma tree. If that's the sort of thing you are talking about, I'm not sure how we can resolve that. The fact that the mlx4_core and mlx5_core modules essentially straddle the networking and rdma trees presents a unique situation with its own difficulties. If the problem I listed is what you are referring to, the only way I know to fix that is to have Mellanox send all of their patches through one tree, and that means either Dave or myself is going to have to deal with stuff they wouldn't normally deal with. Of course, if I'm wrong about what you are referring to, then enlightenment would be appreciated ;-)

Doug Ledford <[email protected]>
              GPG KeyID: 0E572FDD

So these are in your tree:

mlx5_core: Break down the vport mac address query function net/mlx5_core: Introduce access functions to enable/disable RoCE net/mlx5_core: Introduce access functions to query vport RoCE fields

and the networking tree has

net/mlx5: Update access functions to Query/Modify vport MAC address net/mlx5: Introduce access functions to modify/query vport mac lists net/mlx5: Introduce access functions to modify/query vport state net/mlx5: Introduce access functions to modify/query vport promisc mode net/mlx5: Introduce access functions to modify/query vport vlans

which are _similar_ but different. The differences are small annoying details, like slightly different error handling ("goto out" vs just conditional), slightly different arguments (vport or not), and a mix of EXPORT_SYMBOL and EXPORT_SYMBOL_GPL.

The differences are just *stupid*. They are doing very similar things in very similar areas, but the two groups didn't try to match things up or apparently talk to each other, or try to actively make it easier for maintainers to merge their annoying small differences to the exact same area of the file.

So they just changed their driver as if they were the only people working on it. Two separate groups. In the same company.

What is even more annoying to merge, though, is how the two groups again separately changed the structures that apparently describe the hardware layout of things.

That "struct mlx5_ifc_cmd_hca_cap_bits" is a major pain in the arse, for example. It doesn't help that the *undocumented* parts of that array are described with things like

    u8 reserved_66[0x8];
   ...
    u8 reserved_67[0x40];

and then when one group documents something in the middle of that reserved region, it gets split up and the reserved_NNN[xyz] numbers change.

In fact, just looking at that particular "struct mlx5_ifc_cmd_hca_cap_bits", I think one of the two branches got the padding wrong at some point, because they no longer seem to agree about the offsets in the structure.

Imagine what a joy it is to merge crap like that.

If it was two different independent groups, I'd go "ok, this is my life now", take an extra dose of happy pills, and merge things. Because I'd be the person who has to sort that shit out. It's my job.

But what annoys me about this situation is that the two groups that crap on each other are in the same company. They should talk to each other.

For example, to make those hardware descriptor structures easier to keep track of (not just for merging, since apparently people got the offsets wrong even outside of the merges), maybe instead of using a random number for the reserved field, the code could use the expected _offset_. So instead of

    u8 reserved_66[0x8];

it could be something like

    u8 reserved_at_0240[0x8];

to not cause the stupid things to change names just because some _earlier_ reserved field was split. Add a few lines of offsets-in-comments in the parts that don't have a lot of "reserved_xyz[]" fields, and suddenly those hardware description structures would be a hell of a lot easier to see what they do.

Because right now they literally have random noise in them. This is a random sampling:

   ...
    u8 reserved_60[0x1b];
    u8 reserved_61[0xa];
    u8 reserved_62[0x3];
    u8 reserved_63[0x3];
    u8 reserved_64[0x80];
    u8 reserved_65[0x3];
    u8 reserved_66[0x8];
    u8 reserved_67[0x20];
    u8 reserved_68[0x5f];
    u8 reserved_69[0x220];
    u8 reserved_0[0x20];
    u8 reserved_0[0xa00];
   ...

and that's a _small_ random sampling: there are about 1500 lines of those kinds of "reserved_nn[]" fields in that one file.

Just imagine how fun it is to see one of those random numbers (one of the bigger ones), split - completely differently - in the two different versions, and everything else get renumbered.

I wish I was kidding. I'm not.

[ I'm actually planning on re-doing my merge with something like that in both branches before the merge, just because I want to make sure I got the offsets right - maybe the reason I think somebody screwed up their offsets earlier was just because I screwed up when trying to merge this mess ]

But even more than things like that, I'd expect that two groups inside of Mellanox would talk to each other, and NOT MERGE DIFFERENT CHANGES TO THE SAME DRIVER THROUGH TWO DIFFERENT MAINTAINERS!

Or maybe they could use a shared git branch to at least handle the common changes so that they don't conflict. Or maybe they should split up their driver so that the two different groups don't step on each others toes. Or maybe the could use tricks like that "reserved_at_offset[]" thing to at least make the step-on-each-other not be such a pain.

In other words, there are lots of different alternatives to improve on what's going on. But this "throw crap against the wall and hope it sticks" approach that the two groups are clearly using now is *not* acceptable.

If this was the first time it happened, I'd just be annoyed. As it is, something similar happened not that long ago, witht he same people. At this point I'm past "pissed", and firmly in "if this happens again, I'll just stop pulling the crap" camp.

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

So here is the end of this from the two branches before the merge:

- from the networking tree:

        u8         reserved_60[0x1b];
        u8         log_max_wq_sz[0x5];

        u8         nic_vport_change_event[0x1];
        u8         reserved_61[0xa];
        u8         log_max_vlan_list[0x5];
        u8         reserved_62[0x3];
        u8         log_max_current_mc_list[0x5];
        u8         reserved_63[0x3];
        u8         log_max_current_uc_list[0x5];

        u8         reserved_64[0x80];

- from your tree:

        u8         reserved_60[0x1b];
        u8         log_max_wq_sz[0x5];

        u8         reserved_61[0xa0];

(I've thrown out the common parts at the beginning of that structure).

Now, that's a split that makes sense: the "reserved_61[0xa0]" in the second case has been split right. Good. It matches, and they both add up to the same size (0xa0 bytes).

Then we have a common part (except the "reserved_xyz" numbers now don't match due to the split:

- networking tree again:

        u8         reserved_65[0x3];
        u8         log_max_l2_table[0x5];
        u8         reserved_66[0x8];
        u8         log_uar_page_sz[0x10];

- your branch:

        u8         reserved_62[0x3];
        u8         log_max_l2_table[0x5];
        u8         reserved_63[0x8];
        u8         log_uar_page_sz[0x10];

Fine, still matching, just annoying to merge due to the different names for the reserved areas.

Then, we have another split:

- networking tree:

        u8         reserved_67[0x40];
        u8         device_frequency_khz[0x20];
        u8         reserved_68[0x5f];
        u8         cqe_zip[0x1];

        u8         cqe_zip_timeout[0x10];
        u8         cqe_zip_max_num[0x10];

        u8         reserved_69[0x220];

- your branch:

        u8         reserved_64[0x20];
        u8         device_frequency_mhz[0x20];
        u8         device_frequency_khz[0x20];
        u8         reserved_65[0xa0];

        u8         reserved_66[0x1f];
        u8         cqe_zip[0x1];

        u8         cqe_zip_timeout[0x10];
        u8         cqe_zip_max_num[0x10];

        u8         reserved_67[0x220];

Ok, so your branch inserted that 32-byte "device_frequency_mhz" thing, which is why the reserved area 67/64 changed in size.

But look at what happens *after* that. The networking tree has

        u8         reserved_68[0x5f];

but your branch has

        u8         reserved_65[0xa0];

        u8         reserved_66[0x1f];

before the cqe_zip[]. They no longer match up. Something bad happened in one of the branches, and "cqe_zip[]" and the subsequent fields are at different offsets.

This is impossible to merge. One of the branches is wrong.

I picked one at random, not having the energy to figure out where those offsets went wrong.

But it annoys me enormously how much time and effort I had to put into this merge.

Because it was all a complete waste of my time. I could have done more productive, or at least more pleasant things. Like digging out my eyes with a rusty spoon rather than look at this crap and trying to count byte offsets.

If those "reserved" fields were named for the expected offset they are at, then not only wouldn't the names have changed (which would have made things much easier to merge - the parts that weren't split would have auto-merged cleanly and correctly), but it would be much more obvious which one was right and which one was wrong.

As it is, it's impossible to tell without knowing the hardware details or going back in history and trying to see where the offsets magically changed.

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

My merge is out there now. Hopefully mellanox rdma works. And I cannot for the life of me find it in myself to care.

rdma had better not be this kind of pain in the arse in the future, because I really will just stop pulling if it is.

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Side note: I'd suggest you start using branches for rdma. Because I'm going to be very picky about my rdma pulls going forward, and if you put everything in one big pile, the likelihood that it just gets rejected is now very very high.

Using separate branches means that I can decide to merge the sane and easy stuff. The stuff that doesn't contain Mellanox shit, in other words. The stuff that isn't controversial.

At least that way you get *something* merged. Because right now I'm actually so fed up with this that if I hadn't already pushed it out, I would have decided to just undo the pull after all.

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Understood. This release cycle was a bit of a flag day. There were early cleanups that the rest of the code depended on, hence the way it was put together. I tend to use branches if they can be merged easily and don't have these sorts of early, intrusive dependencies.

Ok.

Or, can you please make sure that the right people inside Mellanox review Linus tree for any merge issues ASAP. Then let's have a separate conversation about preventing this in the future?

Doug Ledford <[email protected]>
              GPG KeyID: 0E572FDD

Yes, people will be reviewing/testing the bits they authored/submitted in their merged form and if/where needed rc fixes will be sent.

Or.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

[ snip detailed explanation of horrible merge mess ]

Thanks, that makes absolute sense. Digging your eyeballs out with a rusty spoon seems preferable in this case.

Agreed.

Mellanox has been working to get one official Mellanox git repo through which all of their patch submissions flow. I'll simply stop taking anything from them that doesn't come from this (yet to be announced where it will live) git repo. They can then merge their disparate efforts prior to ever submitting them upstream to either Dave or myself.
That should address the merge issue. It won't, however, address
consistency issues in coding style. That's something Mellanox will need to address internally. I know they have tapped Saeed to handle that merge tree, and he's on the Cc: list here, so Saeed, you will need to get your tree up sooner rather than later and I'll expect patch submissions to start coming from you.

Doug Ledford <[email protected]>
              GPG KeyID: 0E572FDD

Linus,

Re different styling (branching, exporting), in the same driver, point taken.
We will work to improve on that through tighter internal codes reviews and briefing of people on such practices.

Re the header file that described the layout for the driver / firmware layouts, point taken too. We should be changing how the currently reserved fields are named, such that when one person opens feature X and a 2nd person opens feature Y, both on structure Z, the two people would have to change only distinct fields, and hence less noise which creates unneeded conflicts.

The Mellanox NIC HW is used for both plain Eth networking and RDMA. The upstream RDMA stack support IB, RoCE (RDMA over Eth) and user-space Eth offloading.As such, forboth mlx4 and mlx5 there's a core driver plus on top of Eth and RDMA driver.

It's possible that for a given merge cycle, we have networking changes that require core changes and RDMA changes which require different core changes.

We aim to minimize that, but this happens (and is also too sensitive now, as you pointed out re the mlx5_ifc header, which we need to fix), and... we counted on linux-next merge tests, as the process to know things are on the righttrack.

So I'd like to clarify with you a point re linux-next.

From my experience working under few maintainers (Dave, Roland, Doug) over the years...what the maintainer has to do is (A) register their for-next tree/branch to linux-next merge tests and (B) respond to the linux-next maintainer when he identifies conflicts and resolves them.

AFAIK, when the conflict is resolved, Stephen uses git rerere to produce corrective action and the result is applied from the rr-cache when you do the merge.

Can you confirm this is how things work?

A2nd question here, AFAIK, git rerere assumes some ordering... is it okay we'll assume that you will always pull the networking changes and only later the rdma changes?

The case you mentioned of conflict during one of the 2015 merge windows was
I think with a patch that either landed too late in the rdma-next branch, or that the branch wasn't registered for linux-next tests.

For this merge window there were 1-2 cases where Stephen reported and solved the conflict,and another one were we did that and sent him the rere files. We thought we're all Okay.

As Doug noted, the majority of the code for the merge window was in his tree since Dec 24. It's possible that some code landed there later or even really close to the point where the pull request sent to you, so one conflict went unnoticed, this we (RDMA maintainer + Mellanox) need to make sure will not happen.

And this brings me to the last point, merge tests should be done before sending you pull requests. I know Dave is doing that... we will be discussing this with Doug to agree on the details.

Or.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Yes.

However, you should synchronize it some way at least _internally_. Maybe by using a git tree inside Mellanox, so that the pain of your two development groups doesn't then hit people who don't know the hardware and can't test.

Yes. And as far as I can tell, this conflict _was_ in linux-next.

However, the fact that it got resolved in linux-next is purely informational. It doesn't "fix" the conflict - it just means that both sides should have gotten informed about it. That doesn't mean that the conflict goes away or becomes better.

So what linux-next conflicts should have resulted in was that people were aware of it before it even hit my tree, and there might be mitigating efforts taken (where "mitigation" might be just informing me about it in the pull request, which didn't even happen).

And Stephen is actually really really good at handling merge conflicts these days (he's probably the _one_ person who does more merges than I do), but at the same time, he tends to have a "merge and forget" model
- not only does he use "git rerere" so that he never has to worry about the merge later, he doesn't tend to care very much about the end result and think about it.

For example, I doubt Stephen spent a lot of time worrying about the byte offsets in that header file - he cares about getting things to compile. Or did he notice and notify people that the offsets were crap?

In contrast, when I do a merge, I end up trying to see what the background for the conflict is. And that's not just about the code, it's explicitly about things like "why did these two trees conflict in the first place".

Because _I_ care about not just trying to get things to compile, and not just about getting the right merge result, but in fact things like "is there some problem in code maintenance that causes this conflict - should we maybe split development some way?" is very much a primary concern for me.

In 99% of all conflicts, the answer is "it's a trivial conflict, both sides did the rigth thing, I'm just fixing things up".

But sometimes the conflicts are about fundamental problems. The ARM SoC updates were one such area a few years ago, where the insane devicetree people continually stepped on each others toes and it got annoying conflicts many times.

I told the ARM people that their model wasn't working, and the ARM situation has improved tremendously.

Now I'm telling you guys. There's something rotten in the state of Mellanox. It needs to be fixed.

No.

Also note that I do not use "rerere" myself. I do not import the state from linux-next, although I _use_ linux-next to see what I still have to merge, and sometimes to look at whether things had been resolved there (which is why I know at least _some_ of the Mellanox issues already showed up in linux-next).

So linux-next merging is very different. The merge I do is independent of linux-next, and it's often done in a different order.

The thing is, merge tests are all god, and I appreciate them as a heads-up. Some groups do pre-merges to verify that nothing bad happened, and also to give pointers about what is going on when they do happen. The ARM team does that, for example, probable exactly because of the historical issues they used to have.

But at the same time I do *not* appreciate submaintainer pre-merging the end result in order to hide problems in the development model. I don't want to see "I did a merge for you because it was nasty". I'm not trying to push the pain of merging onto Doug, just so that I don't need to know about it. Because I _do_ need to know about it, because problems during the merge are important information for me. They are a sign that something is not right, and two groups are stomping on each others feet.

Problematic merges often cause problems later. Sometimes they cause problems for immediate reasons - the merge resulted in bugs, and it's *really* nasty when mis-merges cause issues, because there is often not a very legible patch to blame. But at other times, the merge is fine, but it hides the conflict of people working on the same code, which will just cause problems later when it happens again.

So no, "merge things right" is not the answer. Well, not the whole answer. I really want your two groups to be aware of each other. Talk before-hand. When you work on the same thing, TALK TO EACH OTHER - so that you share the same end result, so that it gets testing in both your groups.

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Yes, sounds like that's the right thing to do.

That's news to me. When such things happen and caught by Stephen, we are getting an email saying something like

"Today's linux-next merge of the infiniband tree got a conflict between commit X from net-next tree and commit Y from the infiniband tree. I fixed it up (see below) and can carry the fix as necessary (no action is required)."

Also asked around a bit and got to learn on Stephen using git rerere, so all (no action needed note + seeing git rerere in action...) that leaded me to think that indeed no action is required from our side, but after reading your email (twice, so far), I realized that this was wrong conclusion.

nope, the byte offsets party was not reported.

Would it be correct to phrase the points you are trying to make in this email as:

If the conflict doesn't fall into a category of being a trivial one, were both sides did the right thing and someone only has to fix things up to co-exist -- something is wrong, and the conflict should be resolved by a per-merge fix to at least one of the patches

?

[...]

Agree... we'll also share the internal practices for that rdma-next/net-next shared-git tree with Doug to see he's happy with how we act to improve things.

Or.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

So that's purely about linux-next, and Stephen carrying the merge around there. He's informing you that he has resolved it in linux-next, and you now should know about the issue, and that he won't inform you again, because it's resolved in linux-next (sometimes - very seldom - the clashes get so bad that he just throws out one tree as unresolvable, and then that tree needs to actually work on getting it resolved in linux-next).

So it's not entirely about the "trivial" part.

Sometimes we have complicated merges, simply because two different people did very different things, and they clashed.

But of those two people had a good _reason_ to do different things, and they were fundamentally independent things to do, then even a non-trivial merge migth still be about everybody doing the right thing, and it just happened to clash. Unfortunate, but it can happen. For example, maybe Al Viro did some VFS re-organization that impacted individual filesystems, and then one of the filesystem maintainers did some independent changes for one of the filesystems that just happened to clash. That would still be "everything is fine, we were just slightly unlucky".

But when there are two groups in the same company, working on the _same_ driver, then those two groups need to talk. When they clash, it's not "we were just slightly unlucky, working on different things, and normally these things don't interact that way".

This is not a matter of two independent groups working on independent things that just have overlap. You're working on the same f*cking driver, doing variations of the same f*cking thing, for chrissake.

So talk to each other.

We had the same thing happen with the NETDEV_CHANGEUPPER thing from Mellanox. Again, it wasn't two different groups working on two different things that just happened to get a clash. No, it was two different groups inside of Mellanox working on the exact same thing, inside the same company, and they just did the same thing (slightly differently) without ever talking to each other, and sent it out to two different maintainers.

See the difference? A _good_ merge conflict is when two groups work on entirely different things and they just happen to have some unfortunate overlap. A bad conflict is when two groups work on the same exact thing and didn't talk it over.

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

From: Doug Ledford <[email protected]>
Date: Sat, 23 Jan 2016 20:26:01 -0500

Nothing prevents you from pulling from my tree Doug.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

From: Linus Torvalds <torvalds-de/[email protected]>
Date: Sat, 23 Jan 2016 22:51:34 -0800

Yes that's a large part of the problem.

I agree.

But if they put all of their mlx{4,5}_core changes into my tree, then they have to coordinate.

Because obviously all the screaming and complaining we've done the past two merge windows is not working. They still suck at this.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html