Please pull rdma.git

Hi Linus,

This is the first of two pull requests. It is the smaller request, but touches for more different things (this is everything but what is in or going into staging). The pull request for the code in staging/rdma is on hold until after we decide what to do on the write/writev API issue and may be partially deferred until 4.7 as a result.

There is the one merge issue between the net tree and this tree that was simple and obvious to fix up. I've had a phone call with Mellanox about how to avoid this in the future (the result of that conversation was that when they have a pre-req patch that is going through Dave's tree, but the dependent patches are going through my tree, then they should submit the pre-req patch to me as well, we can carry identical patches in both trees, and when you merge the duplicate patches will just drop one and then there will be no merge issue).

Here's the boilerplate of the request:

The following changes since commit fc77dbd34c5c99bce46d40a2491937c3bcbd10af:

Linux 4.5-rc6 (2016-02-28 08:41:20 -0800)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/dledford/rdma.git
tags/for-linus

for you to fetch changes up to 082eaa50838c6b70a8244f8b01d7ed7d686f84db:

Merge branches 'nes', 'cxgb4' and 'iwpm' into k.o/for-4.6 (2016-03-16
13:57:43 -0400)

----------------------------------------------------------------
Initial roundup of 4.6 merge window patches

- cxgb4 updates
- nes updates
- unification of iwarp portmapper code to core
- add drain_cq API
- various ib_core updates
- minor ipoib updates
- minor mlx4 updates
- more significant mlx5 updates (including a minor merge conflict with net-next tree...merge is simple to resolve and Stephen's resolution was confirmed by Mellanox)
- trivial net/9p rdma conversion
- ocrdma RoCEv2 update
- srpt updates

----------------------------------------------------------------
Abhilash Jindal (1):
IB/mlx4: Use boottime

Amitoj Kaur Chawla (1):
IB/core: Replace memset with eth_zero_addr

Arnd Bergmann (1):
infiniband: cxgb4: use %pR format string for printing resources

Bart Van Assche (21):
IB/srpt: Simplify srpt_handle_tsk_mgmt() IB/srpt: Add parentheses around sizeof argument IB/srpt: Remove struct srpt_node_acl IB/srpt: Inline srpt_sdev_name()
IB/srpt: Inline srpt_get_ch_state()
IB/srpt: Introduce target_reverse_dma_direction() IB/srpt: Use scsilun_to_int()
IB/srpt: Simplify channel state management IB/srpt: Simplify srpt_shutdown_session() IB/srpt: Fix srpt_close_session()
IB/srpt: Fix srpt_handle_cmd() error paths IB/srpt: Fix how aborted commands are processed IB/srpt: Inline trivial CM callback functions IB/srpt: Eliminate srpt_find_channel() IB/srpt: Log private data associated with REJ IB/srpt: Use a mutex to protect the channel list IB/srpt: Detect session shutdown reliably IB/srpt: Fix srpt_write_pending()
IB/srpt: Log out all initiators if a port is disabled IB/srpt: Introduce srpt_process_wait_list() IB/srpt: Fix wait list processing

Ben Hutchings (1):
RDMA/nes: Replace LRO with GRO

Christoph Hellwig (2):
IB/mlx5: Convert UMR CQ to new CQ API
net/9p: convert to new CQ API

Devesh Sharma (3):
RDMA/ocrdma: Support RoCE-v2 in the UD path RDMA/ocrdma: Support RoCE-v2 in the RC path RDMA/ocrdma: Support user AH creation for RoCE-v2

Doug Ledford (5):
Merge branch 'k.o/for-4.5-rc' into HEAD
IB/mlx5: Make coding style more consistent
Merge branches 'ib_core', 'ib_ipoib', 'srpt', 'drain-cq-v4' and
'net/9p' into k.o/for-4.6
Merge branches 'mlx4', 'mlx5' and 'ocrdma' into k.o/for-4.6 Merge branches 'nes', 'cxgb4' and 'iwpm' into k.o/for-4.6

Eli Cohen (3):
IB/core: Avoid duplicate code
IB/core: IB/core: Allow legacy verbs through extended interfaces IB/core: Modify conditional on ucontext existence

Erez Shitrit (3):
IB/mlx5: Define interface bits for IPoIB offloads IB/mlx5: Implement UD QP offloads for IPoIB in the TX flow IB/mlx5: Add support for CSUM in RX flow

Faisal Latif (3):
iwcm: common code for port mapper
iw_nes: remove port mapper related code
iwpm: crash fix for large connections test

Haggai Eran (10):
IB/mlx5: Add support for setting source QP number IB/mlx5: Modify QP debugging prints
IB/mlx5: Add GSI QP wrapper
IB/mlx5: Create multiple transmission GSI QPs IB/mlx5: Create GSI transmission QPs when P_Key table is changed IB/mlx5: Generate completions in software
IB/mlx5: Reorder GSI completions
IB/mlx5: Pick the right GSI transmission QP for sending IB/mlx5: Eliminate GSI RX QP's send buffers
IB/cma: Print warning on different inner and header P_Keys

Hal Rosenstock (1):
IB/core: Documentation fix in the MAD header file

Hans Westgaard Ry (1):
IB/ipoib: Add handling for sending of skb with many frags

Hariprasad S (4):
iw_cxgb4: make queue allocation code more readable
iw_cxgb4: remove false error log entry cxgb4/iw_cxgb4: TOS support
iw_cxgb4: Max fastreg depth depends on DSGL support

Insu Yun (1):
nes: handling failed allocation when creating workqueue

Leon Romanovsky (1):
IB/core: Fix missed clean call in registration path

Majd Dibbiny (2):
IB/mlx5: Avoid using user-index for SRQs
IB/{core, mlx5}: Fix input len in vendor part of create_qp/srq

Maor Gottlieb (3):
net/mlx5_core: Create anchor of last flow table net/mlx5_core: Introduce forward to next priority action IB/mlx5: Add support for don't trap rules

Marina Varshaver (2):
IB/core: Add don't trap flag to flow creation IB/mlx4: Add support for the don't trap rule

Markus Elfring (3):
IB/ocrdma: Delete unnecessary variable initialisations in 11 functions IB/ocrdma: Skip using unneeded intermediate variable IB/ocrdma: Skip using unneeded intermediate variable

Matan Barak (3):
net/mlx5: Refactor mlx5_core_mr to mkey IB/core: Add vendor's specific data to alloc mw IB/mlx5: Add memory windows allocation support

Meny Yossefi (3):
net/mlx5_core: Add helper function to read virtual port counters net/mlx5_core: Add helper function to read IB error counters IB/mlx5: Modify MAD reading counters method to use counter registers

Noa Osherovich (2):
IB/mlx5: Refactoring register MR code IB/mlx5: Added support for re-registration of MRs

Parav Pandit (1):
IB/core: trivial prink cleanup.

Sagi Grimberg (4):
IB/mlx5: Expose correct max_fast_reg_page_list_len IB/core: Add arbitrary sg_list support
mlx5: Add arbitrary sg list support
iser: Accept arbitrary sg lists mapping if the device supports it

Somnath Kotur (1):
RDMA/ocrdma: Export udp encapsulation capability

Steve Wise (6):
IB: new common API for draining queues
iw_cxgb4: add queue drain functions
IB/srp: Use ib_drain_rq()
IB/iser: Use ib_drain_sq()
iw_cxgb4: remove port mapper related code
iw_cxgb3: support for iWARP port mapping

 drivers/infiniband/core/cache.c                   |  15 +-
 drivers/infiniband/core/cma.c                     |  22 +-
 drivers/infiniband/core/device.c                  |  29 +-
 drivers/infiniband/core/fmr_pool.c                |  37 +-
 drivers/infiniband/core/iwcm.c                    | 190 ++++-
 drivers/infiniband/core/iwpm_msg.c                |  12 +-
 drivers/infiniband/core/iwpm_util.c               |  14 +-
 drivers/infiniband/core/iwpm_util.h               |   2 +-
 drivers/infiniband/core/packer.c                  |  14 +-
 drivers/infiniband/core/sa_query.c                |  13 +-
 drivers/infiniband/core/ucm.c                     |   8 +-
 drivers/infiniband/core/ucma.c                    |   6 +-
 drivers/infiniband/core/ud_header.c               |  23 +-
 drivers/infiniband/core/uverbs_cmd.c              |  25 +-
 drivers/infiniband/core/uverbs_main.c             |  80 +-
 drivers/infiniband/core/verbs.c                   | 166 ++++
 drivers/infiniband/hw/cxgb3/iwch_cm.c             |  16 +-
 drivers/infiniband/hw/cxgb3/iwch_provider.c       |   3 +-
 drivers/infiniband/hw/cxgb4/cm.c                  | 274 ++-----
 drivers/infiniband/hw/cxgb4/cq.c                  |   9 +-
 drivers/infiniband/hw/cxgb4/device.c              |  72 +-
 drivers/infiniband/hw/cxgb4/iw_cxgb4.h            |  49 +-
 drivers/infiniband/hw/cxgb4/mem.c                 |  12 +-
 drivers/infiniband/hw/cxgb4/provider.c            |   5 +-
 drivers/infiniband/hw/cxgb4/qp.c                  | 107 +--
 drivers/infiniband/hw/mlx4/alias_GUID.c           |   6 +-
 drivers/infiniband/hw/mlx4/main.c                 |  72 +-
 drivers/infiniband/hw/mlx4/mlx4_ib.h              |   3 +-
 drivers/infiniband/hw/mlx4/mr.c                   |   4 +-
 drivers/infiniband/hw/mlx5/Makefile               |   2 +-
 drivers/infiniband/hw/mlx5/cq.c                   | 104 ++-
 drivers/infiniband/hw/mlx5/gsi.c                  | 548 +++++++++++++
 drivers/infiniband/hw/mlx5/mad.c                  | 166 +++-
 drivers/infiniband/hw/mlx5/main.c                 | 119 ++-
 drivers/infiniband/hw/mlx5/mlx5_ib.h              | 108 ++-
 drivers/infiniband/hw/mlx5/mr.c                   | 601 +++++++++++---
 drivers/infiniband/hw/mlx5/odp.c                  |  10 +-
 drivers/infiniband/hw/mlx5/qp.c                   | 271 ++++++-
 drivers/infiniband/hw/mlx5/srq.c                  |  41 +-
 drivers/infiniband/hw/mlx5/user.h                 |   7 +
 drivers/infiniband/hw/nes/Kconfig                 |   1 -
 drivers/infiniband/hw/nes/nes.c                   |  25 -
 drivers/infiniband/hw/nes/nes_cm.c                | 361 ++-------
 drivers/infiniband/hw/nes/nes_cm.h                |  11 +-
 drivers/infiniband/hw/nes/nes_hw.c                |  44 +-
 drivers/infiniband/hw/nes/nes_hw.h                |   7 -
 drivers/infiniband/hw/nes/nes_nic.c               |   7 -
 drivers/infiniband/hw/nes/nes_verbs.c             |   5 +-
 drivers/infiniband/hw/ocrdma/ocrdma.h             |   8 +
 drivers/infiniband/hw/ocrdma/ocrdma_ah.c          |  77 +-
 drivers/infiniband/hw/ocrdma/ocrdma_ah.h          |   5 +-
 drivers/infiniband/hw/ocrdma/ocrdma_hw.c          |  33 +-
 drivers/infiniband/hw/ocrdma/ocrdma_main.c        |   4 +
 drivers/infiniband/hw/ocrdma/ocrdma_sli.h         |  16 +-
 drivers/infiniband/hw/ocrdma/ocrdma_stats.c       |   4 +-
 drivers/infiniband/hw/ocrdma/ocrdma_verbs.c       |  38 +-
 drivers/infiniband/ulp/ipoib/ipoib.h              |   2 +
 drivers/infiniband/ulp/ipoib/ipoib_cm.c           |  23 +-
 drivers/infiniband/ulp/ipoib/ipoib_ib.c           |  18 +
 drivers/infiniband/ulp/ipoib/ipoib_verbs.c        |   5 +-
 drivers/infiniband/ulp/iser/iscsi_iser.c          |  11 +-
 drivers/infiniband/ulp/iser/iscsi_iser.h          |   7 -
 drivers/infiniband/ulp/iser/iser_initiator.c      |   7 -
 drivers/infiniband/ulp/iser/iser_verbs.c          |  38 +-
 drivers/infiniband/ulp/srp/ib_srp.c               |  40 +-
 drivers/infiniband/ulp/srpt/ib_srpt.c             | 912
++++++++--------------
 drivers/infiniband/ulp/srpt/ib_srpt.h             |  31 +-
 drivers/net/ethernet/chelsio/cxgb4/t4_msg.h       |   2 +
 drivers/net/ethernet/chelsio/cxgb4/t4fw_api.h     |   1 +
 drivers/net/ethernet/mellanox/mlx4/fw.c           |   5 +-
 drivers/net/ethernet/mellanox/mlx4/mcg.c          |   6 +-
 drivers/net/ethernet/mellanox/mlx5/core/en.h      |   2 +-
 drivers/net/ethernet/mellanox/mlx5/core/en_main.c |  12 +-
 drivers/net/ethernet/mellanox/mlx5/core/fs_core.c | 225 +++++-
 drivers/net/ethernet/mellanox/mlx5/core/fs_core.h |  15 +
 drivers/net/ethernet/mellanox/mlx5/core/main.c    |   6 +-
 drivers/net/ethernet/mellanox/mlx5/core/mr.c      |  54 +-
 drivers/net/ethernet/mellanox/mlx5/core/port.c    |  23 +
 drivers/net/ethernet/mellanox/mlx5/core/vport.c   |  40 +
 include/linux/mlx4/device.h                       |   3 +
 include/linux/mlx5/device.h                       |  33 +-
 include/linux/mlx5/driver.h                       |  26 +-
 include/linux/mlx5/fs.h                           |   5 +
 include/linux/mlx5/mlx5_ifc.h                     |  51 +-
 include/linux/mlx5/qp.h                           |   7 +-
 include/linux/mlx5/vport.h                        |   2 +
 include/rdma/ib_mad.h                             |   4 +-
 include/rdma/ib_verbs.h                           |  19 +-
 include/rdma/iw_cm.h                              |   6 +-
 include/uapi/rdma/rdma_netlink.h                  |   4 +-
 net/9p/trans_rdma.c                               |  86 +-
 91 files changed, 3647 insertions(+), 1995 deletions(-)
 create mode 100644 drivers/infiniband/hw/mlx5/gsi.c

Doug Ledford <[email protected]>
              GPG KeyID: 0E572FDD

That will end up causing just more problems if there are then other patches on top in either tree.

It would be much better to have the shared pre-requisite not as a _patch_, but as a separate branch. So that you actually share the commits, not just the diff.

Other subsystems have been able to do that.

..

My diffstat doesn't match yours, and it's that "srq.c" file in particular that I have no changes to, and see no changes in your pull request.

Other than that it looks normal, but it makes me wonder how that diff was generated.

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Linus,

I'm not sure that my understanding is correct and will be glad if you can point us to the examples of such branches, so we will use it as soon as possible.

Thanks.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Ok, let me try to explain with a very rough example what I'm looking for.

What Mellanox has been doing is to have two different teams that touch their "own" areas (networking vs rdma), but with a lot of overlap in the same files.

So, for example, you might have twenty patches from one group, and thirty from another, and most of them are pretty independent, but some of them really touch not just the same file, but the same function - either right next to each other or even the same lines.

Then the two groups send their code to two different maintainers, and the end result is that the upper level maintainers need to sort out the fact that two independent groups touched the same things.

That won't fly any more. I have a very simple solution for it if it ever happens again: I'll just stop pulling anything from the rdma tree that has any Mellanox patches in it. Why rdma? It's simply much less important than the networking tree. It's not personal, that's just how it is.

So that's what used to happen, and it's not an option any more.

What you are suggesting you do going forward is to have the two trees at least talk to each other, and instead of having 20/30 separate patches, maybe you'll have 15+5+25 patches, where the five patches in the middle are stuff that both groups need and use as a base, and you still send twenty patches (15+5) to one maintainer and thirty (25+5) to another, but you guarantee that those five patches are exactly the same.

So that definitely *improves* on things. It improves on things in three very real ways:

- we won't see the really *crazy* case, which is that you implemented
the exact same thing in two different ways, and now it's not just the code that clashes, it's actually core functionality.

- yes, there may still be 50 commits total, but now ten of those
fifty will be identical patches, which can make it more likely that things merge cleanly.

- when things clash, at least when I look at the conflicts, I can see
"ok, both branches did _this_ part the same, so I can ignore those two commits that are the same from both sides".

So that's three real improvements, but note how actual conflicts are still going to happen - they just might be easier to resolve (because no core semantic clashes, and because the top-level maintainers can see the "same thing" parts more easily).

The reason they will still happen is that when you send the same patch to two different maintainers, and one of the maintainers then updates his file in other ways *afterwards*, what git will see is that both sides changed the same code *differently*. One did A+B, and one did just A', and it will all conflict anyway. So it's better, but it's not great.

So the *best* situation would be:

- your two groups talk it over, and figure out what the common commits are

- you put those common commits as a "base" branch in git

- you ask the two upper-level maintainers to both pull that base branch

- you then make sure that you send the later patches (whether as
emailed patches or as pull requests) based on top of that base branch.

In the above scenario, you actually don't have any duplicate commits, so now there is only 45 commits total. But the big improvement is not in "fewer commits", it's in what happens when you have that situation of "A+B" vs "just A". Because "A" is now a single shared commit, git really sees that as shared state, and the fact that one side then did B on top of that really doesn't matter, and doesn't cause conflicts.

Of course, if one side does "A+B", and another does "A+C", and B and C conflict, then you'll still see conflicts, but if that happens, you guys screwed up on the whole "what is our common base" question. But the conflict will still be smaller than it would have been with two different copies of A.

NOTE! There is one downside to this model, and that's is that you guys really have to be more careful about that shared base branch. Because it's a shared branch that both upstream maintainers will pull from, that branch really has to be very high quality, and it cannot be rebased etc after you've asked upstream to pull, because that would negate the whole point.

And it has to be higher quality, because now you really need to convince *two* maintainers to take it, and they both need to feel comfortable doing a "git pull" from you. So you'd need to make sure that branch really looks good, and that it's based on a good base that work for both your upstream maintainers. But if you can do that, everybody will be happier.

So it requires a bit more care, and a bit more control. You need to be very confident about those shared commits that both the networking and rdma people take. But quite frankly, after the mess that has been Mellanox, that kind of care and effort just sounds like a good idea.

I hope that clarified.

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Thank you for your such detailed explanation. It gave me a clear picture on how to handle our future submissions.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi David and Doug,

Are you OK with the approach suggested by Linus? We are eager to know it, so we will adopt it as soon as possible in our development flow.

The original thread [1].

[1] http://thread.gmane.org/gmane.linux.drivers.rdma/34907

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

I'm fine with it. Since I happen to use topic branches to build my for-next from anyway, I might need to be the one that Dave pulls from versus the other way around.

Doug Ledford <[email protected]>
              GPG KeyID: 0E572FDD

Doug,
Thank you,

requests from us
with Mellanox core patches for common code. Our specific IB patches/rfc will be based on
that "common" branch, so you won't need to deal with net parts of code at all.

Let's wait for response from David before moving forward.

Thanks
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Resending to linux-netdev.

David,
Can you please express your opinion about Linus's suggestion to eliminate merge conflicts in Mellanox related products?

Thanks

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

From: Leon Romanovsky <leon-2ukJVAZIZ/[email protected]>
Date: Fri, 1 Apr 2016 02:38:28 +0300

Sure, sounds fine.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Thank you, I appreciate a lot Doug's and your openness and willingness to help us eliminate the future merge obstacles.

Fair enough, I'll "encourage" Mellanox to identify which patches will have merge conflicts between the net/rdma trees, and when identified, submit the patch series that contains the pre-req patch with a request for a specific topic branch that I can pull from Dave, they can then be on the hook for letting me know about the topic branch.

Sorry, that's my fault. I have a little scriptlet that I feed whatever kernel version I based my for-next work on and it spits out the pull request using git request-pull and several static items and that kernel version. However, even though I had based my for-next work on v4.5-rc6 (I had been on v4.5-rc4 until I saw that Mellanox put the change to their offset header file in via Dave's tree between rc5 and rc6, so I rebased), I had also pulled in my final -rc pull request via this line in my k.o/for-4.6-topics/ib_core:
bbdfcf18c3b5 Merge branch 'k.o/for-4.5-rc' into HEAD

That merge meant that the diffstat my pull request spit out is showing the diffs from v4.5-rc6 to my last 4.5-rc pull request + the queued for-next work. So your diffstat should rightfully drop the changes to that file since you have them already (and to three other core IB files).