infiniband: fix ulp/srpt/ib_srpt.c kernel-doc notation

Use correct parameter names in kernel-doc notation to eliminate warnings from scripts/kernel-doc.

../drivers/infiniband/ulp/srpt/ib_srpt.c:1146: warning: Excess function parameter 'context' description in 'srpt_abort_cmd' ../drivers/infiniband/ulp/srpt/ib_srpt.c:1482: warning: Excess function parameter 'ioctx' description in 'srpt_handle_new_iu'

---
 drivers/infiniband/ulp/srpt/ib_srpt.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

@@ -1139,7 +1139,6 @@ static struct srpt_send_ioctx *srpt_get_
 /**
  * srpt_abort_cmd() - Abort a SCSI command.
  * @ioctx:   I/O context associated with the SCSI command.
- * @context: Preferred execution context.
  */
 static int srpt_abort_cmd(struct srpt_send_ioctx *ioctx)
 {

@@ -1473,7 +1472,8 @@ fail:
 /**
  * srpt_handle_new_iu() - Process a newly received information unit.
  * @ch:    RDMA channel through which the information unit has been received.
- * @ioctx: SRPT I/O context associated with the information unit.
+ * @recv_ioctx: SRPT I/O context associated with the receive information unit.
+ * @send_ioctx: SRPT I/O context associated with the send information unit.
  */
 static void srpt_handle_new_iu(struct srpt_rdma_ch *ch,
 			       struct srpt_recv_ioctx *recv_ioctx,

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Sure, no problem. Do you have this kernel-doc notation fixed, then?

~Randy
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

T24gRnJpLCAyMDE4LTAxLTA1IGF0IDIxOjU1IC0wODAwLCBSYW5keSBEdW5sYXAgd3JvdGU6DQo+ IE9uIDAxLzA1LzE4IDE2OjM2LCBCYXJ0IFZhbiBBc3NjaGUgd3JvdGU6DQo+ID4gUGxlYXNlIGRy b3AgdGhpcyBwYXRjaC4gSXQgY29uZmxpY3RzIHdpdGggYSBwYXRjaCBzZXJpZXMgSSdtIHdvcmtp bmcgb24uDQo+IA0KPiBTdXJlLCBubyBwcm9ibGVtLiAgRG8geW91IGhhdmUgdGhpcyBrZXJuZWwt ZG9jIG5vdGF0aW9uIGZpeGVkLCB0aGVuPw0KDQpZZXMuIFBsZWFzZSBoYXZlIGEgbG9vayBhdCB0 aGUgc2VyaWVzIEkgcG9zdGVkIHRocmVlIGRheXMgYWdvOg0KaHR0cHM6Ly93d3cuc3Bpbmljcy5u ZXQvbGlzdHMvbGludXgtcmRtYS9tc2c1ODg5Ni5odG1sLg0KDQpUaGFua3MsDQoNCkJhcnQu --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Hello Bart

As agreed, I pulled your tree and checked out block-scsi-for-next branch
I built a kernel to test on mlx5 and booted into that kernel and mapped my SRP devices.

My first test I always run is a reboot after mapping the LUNS, my server is not yet running your kernel only the client.

Anyway, I panicked on the client due to a list corruption and have the capture below.

I am thinking you may not have seen this because you don't have mlx5, only mlx4 in your test bed.

[  202.449161] sd 1:0:0:1: [sdbk] Synchronizing SCSI cache
[  202.478733] sd 1:0:0:2: [sdbj] Synchronizing SCSI cache
[  202.508986] sd 1:0:0:3: [sdbi] Synchronizing SCSI cache
[  202.538082] sd 1:0:0:4: [sdbh] Synchronizing SCSI cache
[  202.568329] sd 1:0:0:5: [sdbg] Synchronizing SCSI cache
[  202.598275] sd 1:0:0:6: [sdbf] Synchronizing SCSI cache
[  202.627607] sd 1:0:0:7: [sdbe] Synchronizing SCSI cache
[  202.657557] sd 1:0:0:8: [sdbd] Synchronizing SCSI cache
[  202.686773] sd 1:0:0:9: [sdbc] Synchronizing SCSI cache
[  202.716227] sd 1:0:0:10: [sdbb] Synchronizing SCSI cache [  202.746555] sd 1:0:0:11: [sdba] Synchronizing SCSI cache [  202.777826] sd 1:0:0:12: [sdaz] Synchronizing SCSI cache [  202.808770] sd 1:0:0:13: [sday] Synchronizing SCSI cache [  202.839954] sd 1:0:0:14: [sdax] Synchronizing SCSI cache [  202.870355] sd 1:0:0:15: [sdaw] Synchronizing SCSI cache [  202.900917] sd 1:0:0:16: [sdav] Synchronizing SCSI cache [  202.930718] sd 1:0:0:17: [sdau] Synchronizing SCSI cache [  202.960734] sd 1:0:0:18: [sdat] Synchronizing SCSI cache [  202.990976] sd 1:0:0:19: [sdas] Synchronizing SCSI cache [  203.020733] sd 1:0:0:20: [sdar] Synchronizing SCSI cache [  203.050828] sd 1:0:0:21: [sdaq] Synchronizing SCSI cache [  203.081566] sd 1:0:0:22: [sdap] Synchronizing SCSI cache [  203.112472] sd 1:0:0:23: [sdao] Synchronizing SCSI cache [  203.143305] sd 1:0:0:24: [sdan] Synchronizing SCSI cache [  203.174065] sd 1:0:0:25: [sdam] Synchronizing SCSI cache [  203.205173] sd 1:0:0:26: [sdal] Synchronizing SCSI cache [  203.236178] sd 1:0:0:27: [sdak] Synchronizing SCSI cache [  203.266446] sd 1:0:0:28: [sdaj] Synchronizing SCSI cache [  203.297050] sd 1:0:0:29: [sdai] Synchronizing SCSI cache [  203.327570] sd 1:0:0:0: [sdah] Synchronizing SCSI cache
[  203.357475] sd 2:0:0:1: [sdag] Synchronizing SCSI cache
[  203.387259] sd 2:0:0:2: [sdaf] Synchronizing SCSI cache
[  203.416950] sd 2:0:0:3: [sdae] Synchronizing SCSI cache
[  203.447112] sd 2:0:0:4: [sdad] Synchronizing SCSI cache
[  203.477650] sd 2:0:0:5: [sdac] Synchronizing SCSI cache
[  203.508438] sd 2:0:0:6: [sdab] Synchronizing SCSI cache
[  203.539018] sd 2:0:0:7: [sdaa] Synchronizing SCSI cache
[  203.568806] sd 2:0:0:8: [sdz] Synchronizing SCSI cache
[  203.598575] sd 2:0:0:9: [sdy] Synchronizing SCSI cache
[  203.628063] sd 2:0:0:10: [sdx] Synchronizing SCSI cache
[  203.658096] sd 2:0:0:11: [sdw] Synchronizing SCSI cache
[  203.687453] sd 2:0:0:12: [sdv] Synchronizing SCSI cache
[  203.718127] sd 2:0:0:13: [sdu] Synchronizing SCSI cache
[  203.747953] sd 2:0:0:14: [sdt] Synchronizing SCSI cache
[  203.777593] sd 2:0:0:15: [sds] Synchronizing SCSI cache
[  203.808214] sd 2:0:0:16: [sdr] Synchronizing SCSI cache
[  203.837516] sd 2:0:0:17: [sdq] Synchronizing SCSI cache
[  203.866690] sd 2:0:0:18: [sdp] Synchronizing SCSI cache
[  203.896013] sd 2:0:0:19: [sdo] Synchronizing SCSI cache
[  203.925029] sd 2:0:0:20: [sdn] Synchronizing SCSI cache
[  203.953954] sd 2:0:0:21: [sdm] Synchronizing SCSI cache
[  203.982830] sd 2:0:0:22: [sdl] Synchronizing SCSI cache
[  204.012713] sd 2:0:0:23: [sdk] Synchronizing SCSI cache
[  204.043456] sd 2:0:0:24: [sdj] Synchronizing SCSI cache
[  204.073671] sd 2:0:0:25: [sdi] Synchronizing SCSI cache
[  204.104050] sd 2:0:0:26: [sdh] Synchronizing SCSI cache
[  204.134239] sd 2:0:0:27: [sdg] Synchronizing SCSI cache
[  204.164603] sd 2:0:0:28: [sdf] Synchronizing SCSI cache
[  204.195387] sd 2:0:0:29: [sde] Synchronizing SCSI cache
[  204.225894] sd 2:0:0:0: [sdd] Synchronizing SCSI cache
[  204.256062] mlx5_core 0000:08:00.1: Shutdown was called
[  204.286882] mlx5_core 0000:08:00.1:
mlx5_cmd_force_teardown_hca:245:(pid 15875): teardown with force mode failed
[  204.296810] mlx5_core 0000:08:00.1: mlx5_cmd_comp_handler:1445:(pid 1028): Command completion arrived after timeout (entry idx = 0). [  207.477515] mlx5_1:wait_for_async_commands:735:(pid 15875): done with all pending requests
[  207.529305] sd 1:0:0:0: [sdah] Synchronizing SCSI cache
[  207.563161] scsi 1:0:0:0: alua: Detached
[  207.586589] sd 1:0:0:29: [sdai] Synchronizing SCSI cache [  207.623036] scsi 1:0:0:29: alua: Detached
[  207.646005] sd 1:0:0:28: [sdaj] Synchronizing SCSI cache [  207.690180] scsi 1:0:0:28: alua: Detached
[  207.713360] sd 1:0:0:27: [sdak] Synchronizing SCSI cache [  207.749020] scsi 1:0:0:27: alua: Detached
[  207.771957] sd 1:0:0:26: [sdal] Synchronizing SCSI cache [  207.808036] scsi 1:0:0:26: alua: Detached
[  207.831913] sd 1:0:0:25: [sdam] Synchronizing SCSI cache [  207.872192] scsi 1:0:0:25: alua: Detached
[  207.895678] sd 1:0:0:24: [sdan] Synchronizing SCSI cache [  207.931020] scsi 1:0:0:24: alua: Detached
[  207.954279] sd 1:0:0:23: [sdao] Synchronizing SCSI cache [  207.990180] scsi 1:0:0:23: alua: Detached
[  208.013315] sd 1:0:0:22: [sdap] Synchronizing SCSI cache [  208.049012] scsi 1:0:0:22: alua: Detached
[  208.072381] sd 1:0:0:21: [sdaq] Synchronizing SCSI cache [  208.112041] scsi 1:0:0:21: alua: Detached
[  208.135881] sd 1:0:0:20: [sdar] Synchronizing SCSI cache [  208.176006] scsi 1:0:0:20: alua: Detached
[  208.199316] sd 1:0:0:19: [sdas] Synchronizing SCSI cache [  208.235018] scsi 1:0:0:19: alua: Detached
[  208.257835] sd 1:0:0:18: [sdat] Synchronizing SCSI cache [  208.294019] scsi 1:0:0:18: alua: Detached
[  208.317725] sd 1:0:0:17: [sdau] Synchronizing SCSI cache [  208.357016] scsi 1:0:0:17: alua: Detached
[  208.380742] sd 1:0:0:16: [sdav] Synchronizing SCSI cache [  208.417015] scsi 1:0:0:16: alua: Detached
[  208.440017] sd 1:0:0:15: [sdaw] Synchronizing SCSI cache [  208.479001] scsi 1:0:0:15: alua: Detached
[  208.501658] sd 1:0:0:14: [sdax] Synchronizing SCSI cache [  208.536039] scsi 1:0:0:14: alua: Detached
[  208.559162] sd 1:0:0:13: [sday] Synchronizing SCSI cache [  208.595027] scsi 1:0:0:13: alua: Detached
[  208.618418] sd 1:0:0:12: [sdaz] Synchronizing SCSI cache [  208.662175] scsi 1:0:0:12: alua: Detached
[  208.685158] sd 1:0:0:11: [sdba] Synchronizing SCSI cache [  208.723993] scsi 1:0:0:11: alua: Detached
[  208.747988] sd 1:0:0:10: [sdbb] Synchronizing SCSI cache [  208.787003] scsi 1:0:0:10: alua: Detached
[  208.810841] sd 1:0:0:9: [sdbc] Synchronizing SCSI cache
[  208.850000] scsi 1:0:0:9: alua: Detached
[  208.873249] sd 1:0:0:8: [sdbd] Synchronizing SCSI cache
[  208.913186] scsi 1:0:0:8: alua: Detached
[  208.936783] sd 1:0:0:7: [sdbe] Synchronizing SCSI cache
[  208.973192] scsi 1:0:0:7: alua: Detached
[  208.995709] sd 1:0:0:6: [sdbf] Synchronizing SCSI cache
[  209.031179] scsi 1:0:0:6: alua: Detached
[  209.053746] sd 1:0:0:5: [sdbg] Synchronizing SCSI cache
[  209.089022] scsi 1:0:0:5: alua: Detached
[  209.112261] sd 1:0:0:4: [sdbh] Synchronizing SCSI cache
[  209.148024] scsi 1:0:0:4: alua: Detached
[  209.171354] sd 1:0:0:3: [sdbi] Synchronizing SCSI cache
[  209.207014] scsi 1:0:0:3: alua: Detached
[  209.229514] sd 1:0:0:2: [sdbj] Synchronizing SCSI cache
[  209.267005] scsi 1:0:0:2: alua: Detached
[  209.290867] sd 1:0:0:1: [sdbk] Synchronizing SCSI cache
[  209.326303] scsi 1:0:0:1: alua: Detached
[  211.376056] ib0: multicast join failed for ff12:601b:ffff:0000:0000:0000:0000:0001, status -22
[  211.439940] scsi host1: ib_srp: connection closed
[  211.466771] scsi host1: ib_srp: connection closed
[  211.493623] scsi host1: ib_srp: connection closed
[  213.425511] ib0: multicast join failed for ff12:601b:ffff:0000:0000:0000:0000:0001, status -22
[  217.521341] ib0: multicast join failed for ff12:601b:ffff:0000:0000:0000:0000:0001, status -22
[  220.843344] ------------[ cut here ]------------
[  220.869309] list_add corruption. prev->next should be next (000000002a07d255), but was           (null). (prev=000000000edf5e8c). [  220.935392] WARNING: CPU: 1 PID: 694 at lib/list_debug.c:28 __list_add_valid+0x6a/0x70
[  220.979462] Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter rpcrdma ib_isert iscsi_target_mod target_core_mod ib_iser libiscsi scsi_transport_iscsi ib_srp scsi_transport_srp ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx5_ib ib_core intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul ipmi_ssif ghash_clmulni_intel pcbc aesni_intel joydev ipmi_si crypto_simd dm_service_time iTCO_wdt hpwdt iTCO_vendor_support glue_helper cryptd ipmi_devintf sg gpio_ich pcspkr hpilo ipmi_msghandler lpc_ich acpi_power_meter i7core_edac shpchp
[  221.385270]  pcc_cpufreq nfsd auth_rpcgss nfs_acl lockd grace sunrpc dm_multipath ip_tables xfs libcrc32c radeon i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mlx5_core mlxfw sd_mod drm ptp hpsa pps_core crc32c_intel i2c_core serio_raw bnx2 devlink scsi_transport_sas dm_mirror dm_region_hash dm_log dm_mod [  221.554496] CPU: 1 PID: 694 Comm: kworker/1:1H Tainted:
G          I      4.15.0-rc7+ #1
[  221.606907] Hardware name: HP ProLiant DL380 G7, BIOS P67 08/16/2015 [  221.642980] Workqueue: ib-comp-wq ib_cq_poll_work [ib_core] [  221.674616] RIP: 0010:__list_add_valid+0x6a/0x70
[  221.700561] RSP: 0018:ffffb2bdc75c7cf0 EFLAGS: 00010086
[  221.730608] RAX: 0000000000000000 RBX: ffff94342d610880 RCX:
ffffffff8ba62928
[  221.771490] RDX: 0000000000000001 RSI: 0000000000000082 RDI:
0000000000000046
[  221.812721] RBP: ffff94342d6108b8 R08: 0000000000000000 R09:
0000000000000722
[  221.853073] R10: 0000000000000000 R11: ffffb2bdc75c7a58 R12:
0000000000000200
[  221.894156] R13: 0000000000000246 R14: ffff943fb7fd5000 R15:
ffff943fb7fd5000
[  221.935233] FS:  0000000000000000(0000) GS:ffff944033200000(0000) knlGS:0000000000000000
[  221.980521] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [  222.013062] CR2: 00007f1bdc0ee910 CR3: 00000017e7e0a002 CR4:
00000000000206e0
[  222.052302] Call Trace:
[  222.065971]  ib_mad_post_receive_mads+0x177/0x310 [ib_core] [  222.097349]  ib_mad_recv_done+0x471/0x9c0 [ib_core]
[  222.124387]  __ib_process_cq+0x55/0xa0 [ib_core]
[  222.150827]  ib_cq_poll_work+0x1b/0x60 [ib_core]
[  222.177751]  process_one_work+0x141/0x340
[  222.200383]  worker_thread+0x47/0x3e0
[  222.220641]  kthread+0xf5/0x130
[  222.238951]  ? rescuer_thread+0x380/0x380
[  222.262034]  ? kthread_associate_blkcg+0x90/0x90
[  222.288514]  ? do_group_exit+0x39/0xa0
[  222.309492]  ret_from_fork+0x1f/0x30
[  222.330073] Code: fe 31 c0 48 c7 c7 98 36 89 8b e8 02 9c cf ff 0f ff 31 c0 c3 48 89 d1 48 c7 c7 48 36 89 8b 48 89 f2 48 89 c6 31 c0 e8 e6 9b cf ff <0f> ff 31 c0 c3 90 48 8b 07 48 b9 00 01 00 00 00 00 ad de 48 8b [  222.438058] ---[ end trace 5d41544bf17ab73b ]---
[  222.465993] BUG: unable to handle kernel NULL pointer dereference at 0000000000000028
[  222.510316] IP: ib_mad_post_receive_mads+0x3c/0x310 [ib_core] [  222.543188] PGD 0 P4D 0
[  222.557625] Oops: 0000 [#1] SMP PTI
[  222.576674] Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter rpcrdma ib_isert iscsi_target_mod target_core_mod ib_iser libiscsi scsi_transport_iscsi ib_srp scsi_transport_srp ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx5_ib ib_core intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul ipmi_ssif ghash_clmulni_intel pcbc aesni_intel joydev ipmi_si crypto_simd dm_service_time iTCO_wdt hpwdt iTCO_vendor_support glue_helper cryptd ipmi_devintf sg gpio_ich pcspkr hpilo ipmi_msghandler lpc_ich acpi_power_meter i7core_edac shpchp
[  222.981443]  pcc_cpufreq nfsd auth_rpcgss nfs_acl lockd grace sunrpc dm_multipath ip_tables xfs libcrc32c radeon i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mlx5_core mlxfw sd_mod drm ptp hpsa pps_core crc32c_intel i2c_core serio_raw bnx2 devlink scsi_transport_sas dm_mirror dm_region_hash dm_log dm_mod [  223.152359] CPU: 1 PID: 694 Comm: kworker/1:1H Tainted: G        W I      4.15.0-rc7+ #1
[  223.198577] Hardware name: HP ProLiant DL380 G7, BIOS P67 08/16/2015 [  223.235101] Workqueue: ib-comp-wq ib_cq_poll_work [ib_core] [  223.266750] RIP: 0010:ib_mad_post_receive_mads+0x3c/0x310 [ib_core] [  223.303012] RSP: 0018:ffffb2bdc75c7cf8 EFLAGS: 00010286
[  223.333022] RAX: 0000000000000000 RBX: ffff94342d610908 RCX:
ffff94342d610948
[  223.373307] RDX: 0000000000000001 RSI: ffff94342d6108c0 RDI:
ffff94342d610908
[  223.414451] RBP: ffff94342d610940 R08: ffff94342a8e64c0 R09:
ffff94342a8e64e8
[  223.454789] R10: ffff94342a8e64e8 R11: ffff94342d6109a8 R12:
ffff944029c2e048
[  223.496554] R13: 0000000000000000 R14: ffff94342a8e64c0 R15:
ffff94342d6108c0
[  223.537489] FS:  0000000000000000(0000) GS:ffff944033200000(0000) knlGS:0000000000000000
[  223.583538] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [  223.616545] CR2: 0000000000000028 CR3: 00000017e7e0a002 CR4:
00000000000206e0
[  223.657337] Call Trace:
[  223.671022]  ? find_mad_agent+0x77/0x1b0 [ib_core]
[  223.698581]  ? __kmalloc+0x1be/0x1f0
[  223.719074]  ib_mad_recv_done+0x471/0x9c0 [ib_core]
[  223.747190]  __ib_process_cq+0x55/0xa0 [ib_core]
[  223.774140]  ib_cq_poll_work+0x1b/0x60 [ib_core]
[  223.800719]  process_one_work+0x141/0x340
[  223.824120]  worker_thread+0x47/0x3e0
[  223.845133]  kthread+0xf5/0x130
[  223.863116]  ? rescuer_thread+0x380/0x380
[  223.886173]  ? kthread_associate_blkcg+0x90/0x90
[  223.912207]  ? do_group_exit+0x39/0xa0
[  223.933198]  ret_from_fork+0x1f/0x30
[  223.953218] Code: 55 41 54 55 48 8d 6f 38 53 48 89 fb 48 83 ec 50 65 48 8b 04 25 28 00 00 00 48 89 44 24 48 31 c0 48 8b 07 48 85 f6 48 89 4c 24 08 <48> 8b 50 28 8b 12 48 c7 44 24 28 00 00 00 00 c7 44 24 40 01 00 [  224.059985] RIP: ib_mad_post_receive_mads+0x3c/0x310 [ib_core] RSP:
ffffb2bdc75c7cf8
[  224.103994] CR2: 0000000000000028

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi Bart

Just wanted to add that the panic is consistent, rebooted into only a single path to my SRP LUNS and on reboot had the same panic.

Regards
Laurence
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

T24gVHVlLCAyMDE4LTAxLTA5IGF0IDE1OjMxIC0wNTAwLCBMYXVyZW5jZSBPYmVybWFuIHdyb3Rl Og0KPiBPbiBUdWUsIDIwMTgtMDEtMDkgYXQgMTU6MTUgLTA1MDAsIExhdXJlbmNlIE9iZXJtYW4g d3JvdGU6DQo+ID4gWyAgMjIwLjg0MzM0NF0gLS0tLS0tLS0tLS0tWyBjdXQgaGVyZSBdLS0tLS0t LS0tLS0tDQo+ID4gWyAgMjIwLjg2OTMwOV0gbGlzdF9hZGQgY29ycnVwdGlvbi4gcHJldi0+bmV4 dCBzaG91bGQgYmUgbmV4dA0KPiA+ICgwMDAwMDAwMDJhMDdkMjU1KSwgYnV0IHdhcyAgICAgICAg ICAgKG51bGwpLg0KPiA+IChwcmV2PTAwMDAwMDAwMGVkZjVlOGMpLg0KPiA+IFsgIDIyMC45MzUz OTJdIFdBUk5JTkc6IENQVTogMSBQSUQ6IDY5NCBhdCBsaWIvbGlzdF9kZWJ1Zy5jOjI4DQo+ID4g X19saXN0X2FkZF92YWxpZCsweDZhLzB4NzANCj4gPiBbICAyMjAuOTc5NDYyXSBNb2R1bGVzIGxp bmtlZCBpbjogeHRfQ0hFQ0tTVU0gaXB0YWJsZV9tYW5nbGUNCj4gPiBpcHRfTUFTUVVFUkFERSBu Zl9uYXRfbWFzcXVlcmFkZV9pcHY0IGlwdGFibGVfbmF0IG5mX25hdF9pcHY0IG5mX25hdA0KPiA+ IG5mX2Nvbm50cmFja19pcHY0IG5mX2RlZnJhZ19pcHY0IHh0X2Nvbm50cmFjayBuZl9jb25udHJh Y2sgaXB0X1JFSkVDVA0KPiA+IG5mX3JlamVjdF9pcHY0IHR1biBicmlkZ2Ugc3RwIGxsYyBlYnRh YmxlX2ZpbHRlciBlYnRhYmxlcw0KPiA+IGlwNnRhYmxlX2ZpbHRlciBpcDZfdGFibGVzIGlwdGFi bGVfZmlsdGVyIHJwY3JkbWEgaWJfaXNlcnQNCj4gPiBpc2NzaV90YXJnZXRfbW9kIHRhcmdldF9j b3JlX21vZCBpYl9pc2VyIGxpYmlzY3NpDQo+ID4gc2NzaV90cmFuc3BvcnRfaXNjc2kNCj4gPiBp Yl9zcnAgc2NzaV90cmFuc3BvcnRfc3JwIGliX2lwb2liIHJkbWFfdWNtIGliX3VjbSBpYl91dmVy YnMgaWJfdW1hZA0KPiA+IHJkbWFfY20gaWJfY20gaXdfY20gbWx4NV9pYiBpYl9jb3JlIGludGVs X3Bvd2VyY2xhbXAgY29yZXRlbXANCj4gPiBrdm1faW50ZWwNCj4gPiBrdm0gaXJxYnlwYXNzIGNy Y3QxMGRpZl9wY2xtdWwgY3JjMzJfcGNsbXVsIGlwbWlfc3NpZg0KPiA+IGdoYXNoX2NsbXVsbmlf aW50ZWwgcGNiYyBhZXNuaV9pbnRlbCBqb3lkZXYgaXBtaV9zaSBjcnlwdG9fc2ltZA0KPiA+IGRt X3NlcnZpY2VfdGltZSBpVENPX3dkdCBocHdkdCBpVENPX3ZlbmRvcl9zdXBwb3J0IGdsdWVfaGVs cGVyIGNyeXB0ZA0KPiA+IGlwbWlfZGV2aW50ZiBzZyBncGlvX2ljaCBwY3Nwa3IgaHBpbG8gaXBt aV9tc2doYW5kbGVyIGxwY19pY2gNCj4gPiBhY3BpX3Bvd2VyX21ldGVyIGk3Y29yZV9lZGFjIHNo cGNocA0KPiA+IFsgIDIyMS4zODUyNzBdICBwY2NfY3B1ZnJlcSBuZnNkIGF1dGhfcnBjZ3NzIG5m c19hY2wgbG9ja2QgZ3JhY2UNCj4gPiBzdW5ycGMNCj4gPiBkbV9tdWx0aXBhdGggaXBfdGFibGVz IHhmcyBsaWJjcmMzMmMgcmFkZW9uIGkyY19hbGdvX2JpdA0KPiA+IGRybV9rbXNfaGVscGVyDQo+ ID4gc3lzY29weWFyZWEgc3lzZmlsbHJlY3Qgc3lzaW1nYmx0IGZiX3N5c19mb3BzIHR0bSBtbHg1 X2NvcmUgbWx4ZncNCj4gPiBzZF9tb2QgZHJtIHB0cCBocHNhIHBwc19jb3JlIGNyYzMyY19pbnRl bCBpMmNfY29yZSBzZXJpb19yYXcgYm54Mg0KPiA+IGRldmxpbmsgc2NzaV90cmFuc3BvcnRfc2Fz IGRtX21pcnJvciBkbV9yZWdpb25faGFzaCBkbV9sb2cgZG1fbW9kDQo+ID4gWyAgMjIxLjU1NDQ5 Nl0gQ1BVOiAxIFBJRDogNjk0IENvbW06IGt3b3JrZXIvMToxSCBUYWludGVkOg0KPiA+IEcgICAg ICAgICAgSSAgICAgIDQuMTUuMC1yYzcrICMxDQo+ID4gWyAgMjIxLjYwNjkwN10gSGFyZHdhcmUg bmFtZTogSFAgUHJvTGlhbnQgREwzODAgRzcsIEJJT1MgUDY3DQo+ID4gMDgvMTYvMjAxNQ0KPiA+ IFsgIDIyMS42NDI5ODBdIFdvcmtxdWV1ZTogaWItY29tcC13cSBpYl9jcV9wb2xsX3dvcmsgW2li X2NvcmVdDQo+ID4gWyAgMjIxLjY3NDYxNl0gUklQOiAwMDEwOl9fbGlzdF9hZGRfdmFsaWQrMHg2 YS8weDcwDQo+ID4gWyAgMjIxLjcwMDU2MV0gUlNQOiAwMDE4OmZmZmZiMmJkYzc1YzdjZjAgRUZM QUdTOiAwMDAxMDA4Ng0KPiA+IFsgIDIyMS43MzA2MDhdIFJBWDogMDAwMDAwMDAwMDAwMDAwMCBS Qlg6IGZmZmY5NDM0MmQ2MTA4ODAgUkNYOg0KPiA+IGZmZmZmZmZmOGJhNjI5MjgNCj4gPiBbICAy MjEuNzcxNDkwXSBSRFg6IDAwMDAwMDAwMDAwMDAwMDEgUlNJOiAwMDAwMDAwMDAwMDAwMDgyIFJE SToNCj4gPiAwMDAwMDAwMDAwMDAwMDQ2DQo+ID4gWyAgMjIxLjgxMjcyMV0gUkJQOiBmZmZmOTQz NDJkNjEwOGI4IFIwODogMDAwMDAwMDAwMDAwMDAwMCBSMDk6DQo+ID4gMDAwMDAwMDAwMDAwMDcy Mg0KPiA+IFsgIDIyMS44NTMwNzNdIFIxMDogMDAwMDAwMDAwMDAwMDAwMCBSMTE6IGZmZmZiMmJk Yzc1YzdhNTggUjEyOg0KPiA+IDAwMDAwMDAwMDAwMDAyMDANCj4gPiBbICAyMjEuODk0MTU2XSBS MTM6IDAwMDAwMDAwMDAwMDAyNDYgUjE0OiBmZmZmOTQzZmI3ZmQ1MDAwIFIxNToNCj4gPiBmZmZm OTQzZmI3ZmQ1MDAwDQo+ID4gWyAgMjIxLjkzNTIzM10gRlM6ICAwMDAwMDAwMDAwMDAwMDAwKDAw MDApIEdTOmZmZmY5NDQwMzMyMDAwMDAoMDAwMCkNCj4gPiBrbmxHUzowMDAwMDAwMDAwMDAwMDAw DQo+ID4gWyAgMjIxLjk4MDUyMV0gQ1M6ICAwMDEwIERTOiAwMDAwIEVTOiAwMDAwIENSMDogMDAw MDAwMDA4MDA1MDAzMw0KPiA+IFsgIDIyMi4wMTMwNjJdIENSMjogMDAwMDdmMWJkYzBlZTkxMCBD UjM6IDAwMDAwMDE3ZTdlMGEwMDIgQ1I0Og0KPiA+IDAwMDAwMDAwMDAwMjA2ZTANCj4gPiBbICAy MjIuMDUyMzAyXSBDYWxsIFRyYWNlOg0KPiA+IFsgIDIyMi4wNjU5NzFdICBpYl9tYWRfcG9zdF9y ZWNlaXZlX21hZHMrMHgxNzcvMHgzMTAgW2liX2NvcmVdDQo+ID4gWyAgMjIyLjA5NzM0OV0gIGli X21hZF9yZWN2X2RvbmUrMHg0NzEvMHg5YzAgW2liX2NvcmVdDQo+ID4gWyAgMjIyLjEyNDM4N10g IF9faWJfcHJvY2Vzc19jcSsweDU1LzB4YTAgW2liX2NvcmVdDQo+ID4gWyAgMjIyLjE1MDgyN10g IGliX2NxX3BvbGxfd29yaysweDFiLzB4NjAgW2liX2NvcmVdDQo+ID4gWyAgMjIyLjE3Nzc1MV0g IHByb2Nlc3Nfb25lX3dvcmsrMHgxNDEvMHgzNDANCj4gPiBbICAyMjIuMjAwMzgzXSAgd29ya2Vy X3RocmVhZCsweDQ3LzB4M2UwDQo+ID4gWyAgMjIyLjIyMDY0MV0gIGt0aHJlYWQrMHhmNS8weDEz MA0KPiA+IFsgIDIyMi4yMzg5NTFdICA/IHJlc2N1ZXJfdGhyZWFkKzB4MzgwLzB4MzgwDQo+ID4g WyAgMjIyLjI2MjAzNF0gID8ga3RocmVhZF9hc3NvY2lhdGVfYmxrY2crMHg5MC8weDkwDQo+ID4g WyAgMjIyLjI4ODUxNF0gID8gZG9fZ3JvdXBfZXhpdCsweDM5LzB4YTANCj4gPiBbICAyMjIuMzA5 NDkyXSAgcmV0X2Zyb21fZm9yaysweDFmLzB4MzANCj4gPiBbICAyMjIuMzMwMDczXSBDb2RlOiBm ZSAzMSBjMCA0OCBjNyBjNyA5OCAzNiA4OSA4YiBlOCAwMiA5YyBjZiBmZiAwZg0KPiA+IGZmDQo+ ID4gMzEgYzAgYzMgNDggODkgZDEgNDggYzcgYzcgNDggMzYgODkgOGIgNDggODkgZjIgNDggODkg YzYgMzEgYzAgZTggZTYNCj4gPiA5Yg0KPiA+IGNmIGZmIDwwZj4gZmYgMzEgYzAgYzMgOTAgNDgg OGIgMDcgNDggYjkgMDAgMDEgMDAgMDAgMDAgMDAgYWQgZGUgNDgNCj4gPiA4YiANCj4gPiBbICAy MjIuNDM4MDU4XSAtLS1bIGVuZCB0cmFjZSA1ZDQxNTQ0YmYxN2FiNzNiIF0tLS0NCj4gPiBbICAy MjIuNDY1OTkzXSBCVUc6IHVuYWJsZSB0byBoYW5kbGUga2VybmVsIE5VTEwgcG9pbnRlciBkZXJl ZmVyZW5jZQ0KPiA+IGF0DQo+ID4gMDAwMDAwMDAwMDAwMDAyOA0KPiA+IFsgIDIyMi41MTAzMTZd IElQOiBpYl9tYWRfcG9zdF9yZWNlaXZlX21hZHMrMHgzYy8weDMxMCBbaWJfY29yZV0NCj4gPiBb ICAyMjIuNTQzMTg4XSBQR0QgMCBQNEQgMCANCj4gPiBbICAyMjIuNTU3NjI1XSBPb3BzOiAwMDAw IFsjMV0gU01QIFBUSQ0KPiA+IFsgIDIyMi41NzY2NzRdIE1vZHVsZXMgbGlua2VkIGluOiB4dF9D SEVDS1NVTSBpcHRhYmxlX21hbmdsZQ0KPiA+IGlwdF9NQVNRVUVSQURFIG5mX25hdF9tYXNxdWVy YWRlX2lwdjQgaXB0YWJsZV9uYXQgbmZfbmF0X2lwdjQgbmZfbmF0DQo+ID4gbmZfY29ubnRyYWNr X2lwdjQgbmZfZGVmcmFnX2lwdjQgeHRfY29ubnRyYWNrIG5mX2Nvbm50cmFjayBpcHRfUkVKRUNU DQo+ID4gbmZfcmVqZWN0X2lwdjQgdHVuIGJyaWRnZSBzdHAgbGxjIGVidGFibGVfZmlsdGVyIGVi dGFibGVzDQo+ID4gaXA2dGFibGVfZmlsdGVyIGlwNl90YWJsZXMgaXB0YWJsZV9maWx0ZXIgcnBj cmRtYSBpYl9pc2VydA0KPiA+IGlzY3NpX3RhcmdldF9tb2QgdGFyZ2V0X2NvcmVfbW9kIGliX2lz ZXIgbGliaXNjc2kNCj4gPiBzY3NpX3RyYW5zcG9ydF9pc2NzaQ0KPiA+IGliX3NycCBzY3NpX3Ry YW5zcG9ydF9zcnAgaWJfaXBvaWIgcmRtYV91Y20gaWJfdWNtIGliX3V2ZXJicyBpYl91bWFkDQo+ ID4gcmRtYV9jbSBpYl9jbSBpd19jbSBtbHg1X2liIGliX2NvcmUgaW50ZWxfcG93ZXJjbGFtcCBj b3JldGVtcA0KPiA+IGt2bV9pbnRlbA0KPiA+IGt2bSBpcnFieXBhc3MgY3JjdDEwZGlmX3BjbG11 bCBjcmMzMl9wY2xtdWwgaXBtaV9zc2lmDQo+ID4gZ2hhc2hfY2xtdWxuaV9pbnRlbCBwY2JjIGFl c25pX2ludGVsIGpveWRldiBpcG1pX3NpIGNyeXB0b19zaW1kDQo+ID4gZG1fc2VydmljZV90aW1l IGlUQ09fd2R0IGhwd2R0IGlUQ09fdmVuZG9yX3N1cHBvcnQgZ2x1ZV9oZWxwZXIgY3J5cHRkDQo+ ID4gaXBtaV9kZXZpbnRmIHNnIGdwaW9faWNoIHBjc3BrciBocGlsbyBpcG1pX21zZ2hhbmRsZXIg bHBjX2ljaA0KPiA+IGFjcGlfcG93ZXJfbWV0ZXIgaTdjb3JlX2VkYWMgc2hwY2hwDQo+ID4gWyAg MjIyLjk4MTQ0M10gIHBjY19jcHVmcmVxIG5mc2QgYXV0aF9ycGNnc3MgbmZzX2FjbCBsb2NrZCBn cmFjZQ0KPiA+IHN1bnJwYw0KPiA+IGRtX211bHRpcGF0aCBpcF90YWJsZXMgeGZzIGxpYmNyYzMy YyByYWRlb24gaTJjX2FsZ29fYml0DQo+ID4gZHJtX2ttc19oZWxwZXINCj4gPiBzeXNjb3B5YXJl YSBzeXNmaWxscmVjdCBzeXNpbWdibHQgZmJfc3lzX2ZvcHMgdHRtIG1seDVfY29yZSBtbHhmdw0K PiA+IHNkX21vZCBkcm0gcHRwIGhwc2EgcHBzX2NvcmUgY3JjMzJjX2ludGVsIGkyY19jb3JlIHNl cmlvX3JhdyBibngyDQo+ID4gZGV2bGluayBzY3NpX3RyYW5zcG9ydF9zYXMgZG1fbWlycm9yIGRt X3JlZ2lvbl9oYXNoIGRtX2xvZyBkbV9tb2QNCj4gPiBbICAyMjMuMTUyMzU5XSBDUFU6IDEgUElE OiA2OTQgQ29tbToga3dvcmtlci8xOjFIIFRhaW50ZWQ6IEcgICAgICAgIFcNCj4gPiBJICAgICAg NC4xNS4wLXJjNysgIzENCj4gPiBbICAyMjMuMTk4NTc3XSBIYXJkd2FyZSBuYW1lOiBIUCBQcm9M aWFudCBETDM4MCBHNywgQklPUyBQNjcNCj4gPiAwOC8xNi8yMDE1DQo+ID4gWyAgMjIzLjIzNTEw MV0gV29ya3F1ZXVlOiBpYi1jb21wLXdxIGliX2NxX3BvbGxfd29yayBbaWJfY29yZV0NCj4gPiBb ICAyMjMuMjY2NzUwXSBSSVA6IDAwMTA6aWJfbWFkX3Bvc3RfcmVjZWl2ZV9tYWRzKzB4M2MvMHgz MTANCj4gPiBbaWJfY29yZV0NCj4gPiBbICAyMjMuMzAzMDEyXSBSU1A6IDAwMTg6ZmZmZmIyYmRj NzVjN2NmOCBFRkxBR1M6IDAwMDEwMjg2DQo+ID4gWyAgMjIzLjMzMzAyMl0gUkFYOiAwMDAwMDAw MDAwMDAwMDAwIFJCWDogZmZmZjk0MzQyZDYxMDkwOCBSQ1g6DQo+ID4gZmZmZjk0MzQyZDYxMDk0 OA0KPiA+IFsgIDIyMy4zNzMzMDddIFJEWDogMDAwMDAwMDAwMDAwMDAwMSBSU0k6IGZmZmY5NDM0 MmQ2MTA4YzAgUkRJOg0KPiA+IGZmZmY5NDM0MmQ2MTA5MDgNCj4gPiBbICAyMjMuNDE0NDUxXSBS QlA6IGZmZmY5NDM0MmQ2MTA5NDAgUjA4OiBmZmZmOTQzNDJhOGU2NGMwIFIwOToNCj4gPiBmZmZm OTQzNDJhOGU2NGU4DQo+ID4gWyAgMjIzLjQ1NDc4OV0gUjEwOiBmZmZmOTQzNDJhOGU2NGU4IFIx MTogZmZmZjk0MzQyZDYxMDlhOCBSMTI6DQo+ID4gZmZmZjk0NDAyOWMyZTA0OA0KPiA+IFsgIDIy My40OTY1NTRdIFIxMzogMDAwMDAwMDAwMDAwMDAwMCBSMTQ6IGZmZmY5NDM0MmE4ZTY0YzAgUjE1 Og0KPiA+IGZmZmY5NDM0MmQ2MTA4YzANCj4gPiBbICAyMjMuNTM3NDg5XSBGUzogIDAwMDAwMDAw MDAwMDAwMDAoMDAwMCkgR1M6ZmZmZjk0NDAzMzIwMDAwMCgwMDAwKQ0KPiA+IGtubEdTOjAwMDAw MDAwMDAwMDAwMDANCj4gPiBbICAyMjMuNTgzNTM4XSBDUzogIDAwMTAgRFM6IDAwMDAgRVM6IDAw MDAgQ1IwOiAwMDAwMDAwMDgwMDUwMDMzDQo+ID4gWyAgMjIzLjYxNjU0NV0gQ1IyOiAwMDAwMDAw MDAwMDAwMDI4IENSMzogMDAwMDAwMTdlN2UwYTAwMiBDUjQ6DQo+ID4gMDAwMDAwMDAwMDAyMDZl MA0KPiA+IFsgIDIyMy42NTczMzddIENhbGwgVHJhY2U6DQo+ID4gWyAgMjIzLjY3MTAyMl0gID8g ZmluZF9tYWRfYWdlbnQrMHg3Ny8weDFiMCBbaWJfY29yZV0NCj4gPiBbICAyMjMuNjk4NTgxXSAg PyBfX2ttYWxsb2MrMHgxYmUvMHgxZjANCj4gPiBbICAyMjMuNzE5MDc0XSAgaWJfbWFkX3JlY3Zf ZG9uZSsweDQ3MS8weDljMCBbaWJfY29yZV0NCj4gPiBbICAyMjMuNzQ3MTkwXSAgX19pYl9wcm9j ZXNzX2NxKzB4NTUvMHhhMCBbaWJfY29yZV0NCj4gPiBbICAyMjMuNzc0MTQwXSAgaWJfY3FfcG9s bF93b3JrKzB4MWIvMHg2MCBbaWJfY29yZV0NCj4gPiBbICAyMjMuODAwNzE5XSAgcHJvY2Vzc19v bmVfd29yaysweDE0MS8weDM0MA0KPiA+IFsgIDIyMy44MjQxMjBdICB3b3JrZXJfdGhyZWFkKzB4 NDcvMHgzZTANCj4gPiBbICAyMjMuODQ1MTMzXSAga3RocmVhZCsweGY1LzB4MTMwDQo+ID4gWyAg MjIzLjg2MzExNl0gID8gcmVzY3Vlcl90aHJlYWQrMHgzODAvMHgzODANCj4gPiBbICAyMjMuODg2 MTczXSAgPyBrdGhyZWFkX2Fzc29jaWF0ZV9ibGtjZysweDkwLzB4OTANCj4gPiBbICAyMjMuOTEy MjA3XSAgPyBkb19ncm91cF9leGl0KzB4MzkvMHhhMA0KPiA+IFsgIDIyMy45MzMxOThdICByZXRf ZnJvbV9mb3JrKzB4MWYvMHgzMA0KPiA+IFsgIDIyMy45NTMyMThdIENvZGU6IDU1IDQxIDU0IDU1 IDQ4IDhkIDZmIDM4IDUzIDQ4IDg5IGZiIDQ4IDgzIGVjIDUwDQo+ID4gNjUNCj4gPiA0OCA4YiAw NCAyNSAyOCAwMCAwMCAwMCA0OCA4OSA0NCAyNCA0OCAzMSBjMCA0OCA4YiAwNyA0OCA4NSBmNiA0 OCA4OQ0KPiA+IDRjDQo+ID4gMjQgMDggPDQ4PiA4YiA1MCAyOCA4YiAxMiA0OCBjNyA0NCAyNCAy OCAwMCAwMCAwMCAwMCBjNyA0NCAyNCA0MCAwMQ0KPiA+IDAwIA0KPiA+IFsgIDIyNC4wNTk5ODVd IFJJUDogaWJfbWFkX3Bvc3RfcmVjZWl2ZV9tYWRzKzB4M2MvMHgzMTAgW2liX2NvcmVdDQo+ID4g UlNQOg0KPiA+IGZmZmZiMmJkYzc1YzdjZjgNCj4gPiBbICAyMjQuMTAzOTk0XSBDUjI6IDAwMDAw MDAwMDAwMDAwMjgNCj4gDQo+IEp1c3Qgd2FudGVkIHRvIGFkZCB0aGF0IHRoZSBwYW5pYyBpcyBj b25zaXN0ZW50LCByZWJvb3RlZCBpbnRvIG9ubHkgYQ0KPiBzaW5nbGUgcGF0aCB0byBteSBTUlAg TFVOUyBhbmQgb24gcmVib290IGhhZCB0aGUgc2FtZSBwYW5pYy4NCg0KSGVsbG8gTGF1cmVuY2Us DQoNCkNhbiB5b3UgcmVwZWF0IHlvdXIgdGVzdCB3aXRoIHRoZSBmb2xsb3dpbmcgdHdvIGtlcm5l bHM6DQoqIHY0LjE1LXJjNyAoTGludXMnIGxhdGVzdCkuDQoqIFRoZSBmb3ItbmV4dCBicmFuY2gg b2YgZ2l0Oi8vZ2l0Lmtlcm5lbC5vcmcvcHViL3NjbS9saW51eC9rZXJuZWwvZ2l0L3JkbWEvcmRt YS5naXQuDQoNCkknbSBhc2tpbmcgdGhpcyBiZWNhdXNlIHRoZSBjcmFzaCBvY2N1cnJlZCBpbiBh IGNvZGUgcGF0aCB0aGF0IGlzIG5vdCBtb2RpZmllZCBieQ0KYW55IG9mIG15IHBhdGNoZXMuDQoN ClRoYW5rcywNCg0KQmFydC4=
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

ʇڙ,jfhz
w
j:+vwjmzZ+ݢj"!

Bart, Yep, I saw it was not in code you touched specific to your patches.

Doing that now, although I had already tested 4.15.0-rc4 from Mike Snitzers tree that only had NVME changes in it and did not see it. So maybe it crept in in the kernels you mentioned.

Its clearly in the ib_mad_xxxx code.

I will baseline again on the ones you asked me to test with v4.15-rc7 (Linus' latest).
The for-next branch of

Back later
Regards
Laurence
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

ʇڙ,jfhz
w
j:+vwjmzZ+ݢj"!

Interesting

On Linus's kernel we don't panic, but we see this below
I will reboot one more time, validate same behavior and then try the rdma tree
I am pretty sure there are changes in that RDMA tree that piggy back on the below scenario to trigger the panic.
And I know Bart you have the RDMA stuff pulled in to yours.

If needed I can capture a vmcore to fully triage the panic.

Rebooting.

[ 1358.714127] sd 1:0:0:1: [sdbk] Synchronizing SCSI cache
[ 1358.744171] sd 1:0:0:2: [sdbj] Synchronizing SCSI cache
[ 1358.773412] sd 1:0:0:3: [sdbi] Synchronizing SCSI cache
[ 1358.803791] sd 1:0:0:4: [sdbh] Synchronizing SCSI cache
[ 1358.833925] sd 1:0:0:5: [sdbg] Synchronizing SCSI cache
[ 1358.864175] sd 1:0:0:6: [sdbf] Synchronizing SCSI cache
[ 1358.893766] sd 1:0:0:7: [sdbe] Synchronizing SCSI cache
[ 1358.924356] sd 1:0:0:8: [sdbd] Synchronizing SCSI cache
[ 1358.954940] sd 1:0:0:9: [sdbc] Synchronizing SCSI cache
[ 1358.985734] sd 1:0:0:10: [sdbb] Synchronizing SCSI cache
[ 1359.015816] sd 1:0:0:11: [sdba] Synchronizing SCSI cache
[ 1359.046156] sd 1:0:0:12: [sdaz] Synchronizing SCSI cache
[ 1359.076851] sd 1:0:0:13: [sday] Synchronizing SCSI cache
[ 1359.106872] sd 1:0:0:14: [sdax] Synchronizing SCSI cache
[ 1359.137053] sd 1:0:0:15: [sdaw] Synchronizing SCSI cache
[ 1359.167544] sd 1:0:0:16: [sdav] Synchronizing SCSI cache
[ 1359.197517] sd 1:0:0:17: [sdau] Synchronizing SCSI cache
[ 1359.229360] sd 1:0:0:18: [sdat] Synchronizing SCSI cache
[ 1359.258470] sd 1:0:0:19: [sdas] Synchronizing SCSI cache
[ 1359.286950] sd 1:0:0:20: [sdar] Synchronizing SCSI cache
[ 1359.317636] sd 1:0:0:21: [sdaq] Synchronizing SCSI cache
[ 1359.348601] sd 1:0:0:22: [sdap] Synchronizing SCSI cache
[ 1359.379196] sd 1:0:0:23: [sdao] Synchronizing SCSI cache
[ 1359.409689] sd 1:0:0:24: [sdan] Synchronizing SCSI cache
[ 1359.440632] sd 1:0:0:25: [sdam] Synchronizing SCSI cache
[ 1359.470780] sd 1:0:0:26: [sdal] Synchronizing SCSI cache
[ 1359.501198] sd 1:0:0:27: [sdak] Synchronizing SCSI cache
[ 1359.531820] sd 1:0:0:28: [sdaj] Synchronizing SCSI cache
[ 1359.561622] sd 1:0:0:29: [sdai] Synchronizing SCSI cache
[ 1359.591658] sd 1:0:0:0: [sdah] Synchronizing SCSI cache
[ 1359.621801] sd 2:0:0:1: [sdag] Synchronizing SCSI cache
[ 1359.651696] sd 2:0:0:2: [sdaf] Synchronizing SCSI cache
[ 1359.681975] sd 2:0:0:3: [sdae] Synchronizing SCSI cache
[ 1359.712012] sd 2:0:0:4: [sdad] Synchronizing SCSI cache
[ 1359.741984] sd 2:0:0:5: [sdac] Synchronizing SCSI cache
[ 1359.771704] sd 2:0:0:6: [sdab] Synchronizing SCSI cache
[ 1359.801829] sd 2:0:0:7: [sdaa] Synchronizing SCSI cache
[ 1359.832076] sd 2:0:0:8: [sdz] Synchronizing SCSI cache
[ 1359.861697] sd 2:0:0:9: [sdy] Synchronizing SCSI cache
[ 1359.890470] sd 2:0:0:10: [sdx] Synchronizing SCSI cache
[ 1359.920747] sd 2:0:0:11: [sdw] Synchronizing SCSI cache
[ 1359.950125] sd 2:0:0:12: [sdv] Synchronizing SCSI cache
[ 1359.978736] sd 2:0:0:13: [sdu] Synchronizing SCSI cache
[ 1360.008490] sd 2:0:0:14: [sdt] Synchronizing SCSI cache
[ 1360.037894] sd 2:0:0:15: [sds] Synchronizing SCSI cache
[ 1360.067282] sd 2:0:0:16: [sdr] Synchronizing SCSI cache
[ 1360.095579] sd 2:0:0:17: [sdq] Synchronizing SCSI cache
[ 1360.125297] sd 2:0:0:18: [sdp] Synchronizing SCSI cache
[ 1360.154539] sd 2:0:0:19: [sdo] Synchronizing SCSI cache
[ 1360.184087] sd 2:0:0:20: [sdn] Synchronizing SCSI cache
[ 1360.213859] sd 2:0:0:21: [sdm] Synchronizing SCSI cache
[ 1360.243405] sd 2:0:0:22: [sdl] Synchronizing SCSI cache
[ 1360.272676] sd 2:0:0:23: [sdk] Synchronizing SCSI cache
[ 1360.303088] sd 2:0:0:24: [sdj] Synchronizing SCSI cache
[ 1360.332838] sd 2:0:0:25: [sdi] Synchronizing SCSI cache
[ 1360.362778] sd 2:0:0:26: [sdh] Synchronizing SCSI cache
[ 1360.392887] sd 2:0:0:27: [sdg] Synchronizing SCSI cache
[ 1360.422989] sd 2:0:0:28: [sdf] Synchronizing SCSI cache
[ 1360.452909] sd 2:0:0:29: [sde] Synchronizing SCSI cache
[ 1360.482103] sd 2:0:0:0: [sdd] Synchronizing SCSI cache
[ 1360.511682] mlx5_core 0000:08:00.1: Shutdown was called
[ 1360.550531] mlx5_core 0000:08:00.1: mlx5_enter_error_state:121:(pid
15149): start
[ 1360.593520] ------------[ cut here ]------------
[ 1360.619930] got unsolicited completion for CQ 0x0000000068694acd
[ 1360.654434] WARNING: CPU: 15 PID: 15149 at
drivers/infiniband/core/cq.c:80 ib_cq_completion_direct+0x28/0x30
[ib_core]
[ 1360.716099] Modules linked in: xt_CHECKSUM iptable_mangle

ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter rpcrdma ib_isert iscsi_target_mod target_core_mod ib_iser libiscsi scsi_transport_iscsi ib_srp scsi_transport_srp ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx5_ib ib_core intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul ipmi_ssif ghash_clmulni_intel pcbc joydev aesni_intel dm_service_time ipmi_si crypto_simd glue_helper sg hpilo cryptd hpwdt ipmi_devintf iTCO_wdt gpio_ich acpi_power_meter iTCO_vendor_support ipmi_msghandler shpchp pcspkr i7core_edac lpc_ich

[ 1361.120851]  pcc_cpufreq nfsd auth_rpcgss nfs_acl lockd grace

dm_multipath sunrpc ip_tables xfs libcrc32c radeon i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm sd_mod drm mlx5_core mlxfw ptp serio_raw crc32c_intel i2c_core hpsa pps_core bnx2 devlink scsi_transport_sas dm_mirror dm_region_hash dm_log dm_mod

[ 1361.288913] CPU: 15 PID: 15149 Comm: reboot Tainted:
G          I      4.15.0-rc7 #1
[ 1361.333577] Hardware name: HP ProLiant DL380 G7, BIOS P67 08/16/2015
[ 1361.369976] RIP: 0010:ib_cq_completion_direct+0x28/0x30 [ib_core]
[ 1361.404971] RSP: 0018:ffffa08c8747fc60 EFLAGS: 00010086
[ 1361.435007] RAX: 0000000000000000 RBX: ffff8d37a6f8b468 RCX:
ffffffffae662928
[ 1361.474397] RDX: 0000000000000001 RSI: 0000000000000082 RDI:
0000000000000046
[ 1361.515097] RBP: ffff8d2bb07e0000 R08: 0000000000000000 R09:
0000000000000717
[ 1361.555054] R10: 0000000000000000 R11: ffffa08c8747f9c8 R12:
ffff8d2ed1edc264
[ 1361.595593] R13: ffff8d37a6f8b400 R14: ffffa08c8747fca8 R15:
0000000000000083
[ 1361.635133] FS:  00007fc09956a880(0000) GS:ffff8d37b33c0000(0000)
knlGS:0000000000000000
[ 1361.681800] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1361.714217] CR2: 0000000001034f80 CR3: 0000000ba0f9e005 CR4:
00000000000206e0
[ 1361.754794] Call Trace:
[ 1361.768980]  mlx5_ib_event+0x335/0x410 [mlx5_ib]
[ 1361.795303]  mlx5_core_event+0x7b/0x1a0 [mlx5_core]
[ 1361.823438]  ? synchronize_irq+0x35/0xa0
[ 1361.845962]  mlx5_enter_error_state+0xe4/0x1c0 [mlx5_core]
[ 1361.877382]  shutdown+0x127/0x170 [mlx5_core]
[ 1361.902688]  pci_device_shutdown+0x31/0x60
[ 1361.925924]  device_shutdown+0x101/0x1d0
[ 1361.948642]  kernel_restart+0xe/0x60
[ 1361.968517]  SYSC_reboot+0x1e8/0x210
[ 1361.988062]  ? __audit_syscall_entry+0xaf/0x100
[ 1362.013500]  ? syscall_trace_enter+0x1cc/0x2b0
[ 1362.038483]  ? __audit_syscall_exit+0x1ff/0x280
[ 1362.064598]  do_syscall_64+0x61/0x1a0
[ 1362.084635]  entry_SYSCALL64_slow_path+0x25/0x25
[ 1362.111113] RIP: 0033:0x7fc098377a56
[ 1362.131668] RSP: 002b:00007ffd4b3377e8 EFLAGS: 00000206 ORIG_RAX:
00000000000000a9
[ 1362.174578] RAX: ffffffffffffffda RBX: 0000000000000004 RCX:
00007fc098377a56
[ 1362.213620] RDX: 0000000001234567 RSI: 0000000028121969 RDI:
fffffffffee1dead
[ 1362.255259] RBP: 0000000000000000 R08: 000056141a7642a0 R09:
00007ffd4b336eb0
[ 1362.296293] R10: 0000000000000024 R11: 0000000000000206 R12:
0000000000000000
[ 1362.338341] R13: 00007ffd4b337ab0 R14: 0000000000000000 R15:
0000000000000000
[ 1362.378518] Code: 00 00 00 66 66 66 66 90 80 3d 65 e1 02 00 00 74 02
f3 c3 48 89 fe 31 c0 48 c7 c7 68 58 92 c0 c6 05 4e e1 02 00 01 e8 a8 23
d8 ec <0f> ff c3 0f 1f 44 00 00 66 66 66 66 90 41 55 45 89 c5 41 54 49 
[ 1362.483962] ---[ end trace 528ee06930a5763f ]---
[ 1362.509435] mlx5_1:mlx5_ib_event:2992:(pid 15149): warning: event on
port 0
[ 1362.548716] scsi host2: ib_srp: failed RECV status WR flushed (5)
for CQE 0000000023e53497
[ 1362.595980] mlx5_core 0000:08:00.1: mlx5_enter_error_state:128:(pid
15149): end
[ 1362.637630] mlx5_core 0000:08:00.0: Shutdown was called
[ 1362.677523] mlx5_core 0000:08:00.0: mlx5_enter_error_state:121:(pid
15149): start
[ 1362.720734] mlx5_0:mlx5_ib_event:2992:(pid 15149): warning: event on
port 0
[ 1362.760795] scsi host1: ib_srp: failed RECV status WR flushed (5)
for CQE 000000009ad07e27
[ 1362.806977] mlx5_core 0000:08:00.0: mlx5_enter_error_state:128:(pid
15149): end
[ 1363.331808] reboot: Restarting system
[ 1363.349889] reboot: machine restart

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

ʇڙ,jfhz
w
j:+vwjmzZ+ݢj"!

Hello Bart

Confirmed the panic in 4.15.0-rc2.rdma+

This kernel is built off the for-next branch of
git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git.

Leon and RDMA folks can you look into this so I can avoid a bisect please

Snippet from below seems to be important:

[  938.938946] mlx5_core 0000:08:00.1: Shutdown was called [  938.968423] mlx5_core 0000:08:00.1:
mlx5_cmd_force_teardown_hca:245:(pid 14752): teardown with force mode failed
[  938.978359] mlx5_core 0000:08:00.1: mlx5_cmd_comp_handler:1445:(pid 13186): Command completion arrived after timeout (entry idx = 0). [  942.209464] mlx5_1:wait_for_async_commands:735:(pid 14752): done with all pending requests

Rebooting.
[  937.142748] sd 1:0:0:1: [sdbk] Synchronizing SCSI cache
[  937.173076] sd 1:0:0:2: [sdbj] Synchronizing SCSI cache
[  937.203855] sd 1:0:0:3: [sdbi] Synchronizing SCSI cache
[  937.234117] sd 1:0:0:4: [sdbh] Synchronizing SCSI cache
[  937.264894] sd 1:0:0:5: [sdbg] Synchronizing SCSI cache
[  937.295257] sd 1:0:0:6: [sdbf] Synchronizing SCSI cache
[  937.325107] sd 1:0:0:7: [sdbe] Synchronizing SCSI cache
[  937.354969] sd 1:0:0:8: [sdbd] Synchronizing SCSI cache
[  937.385332] sd 1:0:0:9: [sdbc] Synchronizing SCSI cache
[  937.414118] sd 1:0:0:10: [sdbb] Synchronizing SCSI cache [  937.444397] sd 1:0:0:11: [sdba] Synchronizing SCSI cache [  937.473847] sd 1:0:0:12: [sdaz] Synchronizing SCSI cache [  937.504494] sd 1:0:0:13: [sday] Synchronizing SCSI cache [  937.535032] sd 1:0:0:14: [sdax] Synchronizing SCSI cache [  937.565076] sd 1:0:0:15: [sdaw] Synchronizing SCSI cache [  937.596843] sd 1:0:0:16: [sdav] Synchronizing SCSI cache [  937.627773] sd 1:0:0:17: [sdau] Synchronizing SCSI cache [  937.657933] sd 1:0:0:18: [sdat] Synchronizing SCSI cache [  937.688828] sd 1:0:0:19: [sdas] Synchronizing SCSI cache [  937.720082] sd 1:0:0:20: [sdar] Synchronizing SCSI cache [  937.750818] sd 1:0:0:21: [sdaq] Synchronizing SCSI cache [  937.781396] sd 1:0:0:22: [sdap] Synchronizing SCSI cache [  937.811480] sd 1:0:0:23: [sdao] Synchronizing SCSI cache [  937.841955] sd 1:0:0:24: [sdan] Synchronizing SCSI cache [  937.872095] sd 1:0:0:25: [sdam] Synchronizing SCSI cache [  937.902674] sd 1:0:0:26: [sdal] Synchronizing SCSI cache [  937.932741] sd 1:0:0:27: [sdak] Synchronizing SCSI cache [  937.962074] sd 1:0:0:28: [sdaj] Synchronizing SCSI cache [  937.991450] sd 1:0:0:29: [sdai] Synchronizing SCSI cache [  938.020907] sd 2:0:0:1: [sdah] Synchronizing SCSI cache
[  938.050879] sd 2:0:0:2: [sdag] Synchronizing SCSI cache
[  938.080505] sd 2:0:0:3: [sdaf] Synchronizing SCSI cache
[  938.110263] sd 2:0:0:4: [sdae] Synchronizing SCSI cache
[  938.139903] sd 2:0:0:5: [sdad] Synchronizing SCSI cache
[  938.169782] sd 2:0:0:6: [sdac] Synchronizing SCSI cache
[  938.199754] sd 2:0:0:7: [sdab] Synchronizing SCSI cache
[  938.230013] sd 2:0:0:8: [sdaa] Synchronizing SCSI cache
[  938.259811] sd 2:0:0:9: [sdz] Synchronizing SCSI cache
[  938.289633] sd 2:0:0:10: [sdy] Synchronizing SCSI cache
[  938.318870] sd 2:0:0:11: [sdx] Synchronizing SCSI cache
[  938.348961] sd 2:0:0:12: [sdw] Synchronizing SCSI cache
[  938.378774] sd 2:0:0:13: [sdv] Synchronizing SCSI cache
[  938.408485] sd 2:0:0:14: [sdu] Synchronizing SCSI cache
[  938.438075] sd 2:0:0:15: [sdt] Synchronizing SCSI cache
[  938.466951] sd 2:0:0:16: [sds] Synchronizing SCSI cache
[  938.496511] sd 2:0:0:17: [sdr] Synchronizing SCSI cache
[  938.526718] sd 2:0:0:18: [sdq] Synchronizing SCSI cache
[  938.556687] sd 2:0:0:19: [sdp] Synchronizing SCSI cache
[  938.585200] sd 2:0:0:20: [sdo] Synchronizing SCSI cache
[  938.614874] sd 2:0:0:21: [sdn] Synchronizing SCSI cache
[  938.644621] sd 2:0:0:22: [sdm] Synchronizing SCSI cache
[  938.674399] sd 2:0:0:23: [sdl] Synchronizing SCSI cache
[  938.704763] sd 2:0:0:24: [sdk] Synchronizing SCSI cache
[  938.734895] sd 2:0:0:25: [sdj] Synchronizing SCSI cache
[  938.765294] sd 2:0:0:26: [sdi] Synchronizing SCSI cache
[  938.794818] sd 2:0:0:27: [sdh] Synchronizing SCSI cache
[  938.824447] sd 2:0:0:28: [sdg] Synchronizing SCSI cache
[  938.853212] sd 2:0:0:29: [sdf] Synchronizing SCSI cache
[  938.881769] sd 2:0:0:0: [sde] Synchronizing SCSI cache
[  938.910640] sd 1:0:0:0: [sdd] Synchronizing SCSI cache
[  938.938946] mlx5_core 0000:08:00.1: Shutdown was called
[  938.968423] mlx5_core 0000:08:00.1:
mlx5_cmd_force_teardown_hca:245:(pid 14752): teardown with force mode failed
[  938.978359] mlx5_core 0000:08:00.1: mlx5_cmd_comp_handler:1445:(pid 13186): Command completion arrived after timeout (entry idx = 0). [  942.209464] mlx5_1:wait_for_async_commands:735:(pid 14752): done with all pending requests
[  942.259812] sd 1:0:0:0: [sdd] Synchronizing SCSI cache
[  942.294448] scsi 1:0:0:0: alua: Detached
[  942.317433] sd 1:0:0:29: [sdai] Synchronizing SCSI cache [  942.355461] scsi 1:0:0:29: alua: Detached
[  942.379602] sd 1:0:0:28: [sdaj] Synchronizing SCSI cache [  942.418441] scsi 1:0:0:28: alua: Detached
[  942.440965] sd 1:0:0:27: [sdak] Synchronizing SCSI cache [  942.479447] scsi 1:0:0:27: alua: Detached
[  942.502351] sd 1:0:0:26: [sdal] Synchronizing SCSI cache [  942.537745] scsi 1:0:0:26: alua: Detached
[  942.561479] sd 1:0:0:25: [sdam] Synchronizing SCSI cache [  942.599444] scsi 1:0:0:25: alua: Detached
[  942.623153] sd 1:0:0:24: [sdan] Synchronizing SCSI cache [  942.659633] scsi 1:0:0:24: alua: Detached
[  942.682904] sd 1:0:0:23: [sdao] Synchronizing SCSI cache [  942.722444] scsi 1:0:0:23: alua: Detached
[  942.745058] sd 1:0:0:22: [sdap] Synchronizing SCSI cache [  942.780644] scsi 1:0:0:22: alua: Detached
[  942.803690] sd 1:0:0:21: [sdaq] Synchronizing SCSI cache [  942.839647] scsi 1:0:0:21: alua: Detached
[  942.863364] sd 1:0:0:20: [sdar] Synchronizing SCSI cache [  942.899617] scsi 1:0:0:20: alua: Detached
[  942.922661] sd 1:0:0:19: [sdas] Synchronizing SCSI cache [  942.957640] scsi 1:0:0:19: alua: Detached
[  942.981039] sd 1:0:0:18: [sdat] Synchronizing SCSI cache [  943.016637] scsi 1:0:0:18: alua: Detached
[  943.040163] sd 1:0:0:17: [sdau] Synchronizing SCSI cache [  943.075648] scsi 1:0:0:17: alua: Detached
[  943.099057] sd 1:0:0:16: [sdav] Synchronizing SCSI cache [  943.135627] scsi 1:0:0:16: alua: Detached
[  943.159647] sd 1:0:0:15: [sdaw] Synchronizing SCSI cache [  943.199447] scsi 1:0:0:15: alua: Detached
[  943.222318] sd 1:0:0:14: [sdax] Synchronizing SCSI cache [  943.256648] scsi 1:0:0:14: alua: Detached
[  943.279739] sd 1:0:0:13: [sday] Synchronizing SCSI cache [  943.319442] scsi 1:0:0:13: alua: Detached
[  943.341975] sd 1:0:0:12: [sdaz] Synchronizing SCSI cache [  943.377454] scsi 1:0:0:12: alua: Detached
[  943.400574] sd 1:0:0:11: [sdba] Synchronizing SCSI cache [  943.436438] scsi 1:0:0:11: alua: Detached
[  943.459168] sd 1:0:0:10: [sdbb] Synchronizing SCSI cache [  943.495649] scsi 1:0:0:10: alua: Detached
[  943.518395] sd 1:0:0:9: [sdbc] Synchronizing SCSI cache
[  943.554455] scsi 1:0:0:9: alua: Detached
[  943.577524] sd 1:0:0:8: [sdbd] Synchronizing SCSI cache
[  943.617643] scsi 1:0:0:8: alua: Detached
[  943.640599] sd 1:0:0:7: [sdbe] Synchronizing SCSI cache
[  943.676596] scsi 1:0:0:7: alua: Detached
[  943.699790] sd 1:0:0:6: [sdbf] Synchronizing SCSI cache
[  943.737440] scsi 1:0:0:6: alua: Detached
[  943.760309] sd 1:0:0:5: [sdbg] Synchronizing SCSI cache
[  943.796634] scsi 1:0:0:5: alua: Detached
[  943.819456] sd 1:0:0:4: [sdbh] Synchronizing SCSI cache
[  943.854634] scsi 1:0:0:4: alua: Detached
[  943.877433] sd 1:0:0:3: [sdbi] Synchronizing SCSI cache
[  943.914621] scsi 1:0:0:3: alua: Detached
[  943.938146] sd 1:0:0:2: [sdbj] Synchronizing SCSI cache
[  943.973712] scsi 1:0:0:2: alua: Detached
[  943.995848] sd 1:0:0:1: [sdbk] Synchronizing SCSI cache
[  944.029648] scsi 1:0:0:1: alua: Detached
[  946.135367] scsi host1: ib_srp: connection closed
[  946.159601] scsi host1: ib_srp: connection closed
[  946.185789] scsi host1: ib_srp: connection closed
[  946.647514] kernel tried to execute NX-protected page - exploit attempt? (uid: 0)
[  946.691954] BUG: unable to handle kernel paging request at 00000000a2129b93
[  946.731023] IP: 0xffff9dcd6684dfc0
[  946.749587] PGD 1346a3b067 P4D 1346a3b067 PUD 8000000b000001e3 [  946.783502] Oops: 0011 [#1] SMP
[  946.800543] Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter rpcrdma ib_isert iscsi_target_mod target_core_mod ib_iser libiscsi scsi_transport_iscsi ib_srp scsi_transport_srp ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx5_ib ib_core intel_powerclamp coretemp kvm_intel kvm irqbypass ipmi_ssif crct10dif_pclmul crc32_pclmul ghash_clmulni_intel dm_service_time joydev hpilo pcbc sg ipmi_si aesni_intel ipmi_devintf crypto_simd glue_helper gpio_ich cryptd hpwdt ipmi_msghandler iTCO_wdt acpi_power_meter iTCO_vendor_support i7core_edac shpchp pcspkr lpc_ich
[  947.201595]  pcc_cpufreq nfsd auth_rpcgss nfs_acl lockd dm_multipath grace sunrpc ip_tables xfs libcrc32c radeon i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mlx5_core mlxfw sd_mod drm ptp pps_core i2c_core crc32c_intel hpsa serio_raw devlink bnx2 scsi_transport_sas dm_mirror dm_region_hash dm_log dm_mod [  947.368091] CPU: 0 PID: 832 Comm: kworker/0:1H Tainted:
G          I      4.15.0-rc2.rdma+ #1
[  947.416086] Hardware name: HP ProLiant DL380 G7, BIOS P67 08/16/2015 [  947.452642] Workqueue: ib-comp-wq ib_cq_poll_work [ib_core] [  947.484966] task: 00000000f16afaf6 task.stack: 000000000a5a26e0 [  947.519275] RIP: 0010:0xffff9dcd6684dfc0
[  947.541610] RSP: 0018:ffffbf04072d3e28 EFLAGS: 00010282
[  947.571795] RAX: ffff9dce20c57a10 RBX: 0000000000000048 RCX:
0000000000000000
[  947.612745] RDX: ffff9dce2facd500 RSI: ffff9dda2a4d6848 RDI:
ffff9dce2e07a800
[  947.652788] RBP: 0000000000000090 R08: ffff9dce2facd4c8 R09:
ffff9dce2facd4c8
[  947.693568] R10: 0000000000000000 R11: ffff9dce2e07a9d0 R12:
0000000000000002
[  947.733332] R13: 0000000000000000 R14: 0000000000010000 R15:
ffff9dce2e07a800
[  947.772737] FS:  0000000000000000(0000) GS:ffff9dce37a00000(0000) knlGS:0000000000000000
[  947.817595] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [  947.850061] CR2: ffff9dcd6684dfc0 CR3: 0000001346009002 CR4:
00000000000206f0
[  947.889552] Call Trace:
[  947.903724]  ? __ib_process_cq+0x55/0xa0 [ib_core]
[  947.931179]  ? ib_cq_poll_work+0x1b/0x60 [ib_core]
[  947.958153]  ? process_one_work+0x141/0x340
[  947.981362]  ? worker_thread+0x47/0x3e0
[  948.002102]  ? kthread+0xf5/0x130
[  948.020538]  ? rescuer_thread+0x380/0x380
[  948.043180]  ? kthread_associate_blkcg+0x90/0x90
[  948.070184]  ? ret_from_fork+0x1f/0x30
[  948.091250] Code: 00 00 00 10 40 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 40 4c cb bc cd 9d ff ff 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [  948.199700] RIP: 0xffff9dcd6684dfc0 RSP: ffffbf04072d3e28 [  948.229734] CR2: ffff9dcd6684dfc0

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

These oops's you have are very suggestive that ib_wc->wr_cqe is garbage..

Did SRP free its wr_cqe data before completion somehow?

Turn on slab poisoning to confirm?

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

T24gV2VkLCAyMDE4LTAxLTEwIGF0IDExOjI2IC0wNzAwLCBKYXNvbiBHdW50aG9ycGUgd3JvdGU6 DQo+IE9uIFdlZCwgSmFuIDEwLCAyMDE4IGF0IDA4OjQyOjAzQU0gLTA1MDAsIExhdXJlbmNlIE9i ZXJtYW4gd3JvdGU6DQo+IA0KPiA+IFsgIDk0Ni42NDc1MTRdIGtlcm5lbCB0cmllZCB0byBleGVj dXRlIE5YLXByb3RlY3RlZCBwYWdlIC0gZXhwbG9pdA0KPiA+IGF0dGVtcHQ/ICh1aWQ6IDApDQo+ ID4gWyAgOTQ2LjY5MTk1NF0gQlVHOiB1bmFibGUgdG8gaGFuZGxlIGtlcm5lbCBwYWdpbmcgcmVx dWVzdCBhdA0KPiA+IDAwMDAwMDAwYTIxMjliOTMNCj4gPiBbICA5NDcuODg5NTUyXSBDYWxsIFRy YWNlOg0KPiA+IFsgIDk0Ny45MDM3MjRdICA/IF9faWJfcHJvY2Vzc19jcSsweDU1LzB4YTAgW2li X2NvcmVdDQo+ID4gWyAgOTQ3LjkzMTE3OV0gID8gaWJfY3FfcG9sbF93b3JrKzB4MWIvMHg2MCBb aWJfY29yZV0NCj4gPiBbICA5NDcuOTU4MTUzXSAgPyBwcm9jZXNzX29uZV93b3JrKzB4MTQxLzB4 MzQwDQo+ID4gWyAgOTQ3Ljk4MTM2Ml0gID8gd29ya2VyX3RocmVhZCsweDQ3LzB4M2UwDQo+ID4g WyAgOTQ4LjAwMjEwMl0gID8ga3RocmVhZCsweGY1LzB4MTMwDQo+ID4gWyAgOTQ4LjAyMDUzOF0g ID8gcmVzY3Vlcl90aHJlYWQrMHgzODAvMHgzODANCj4gPiBbICA5NDguMDQzMTgwXSAgPyBrdGhy ZWFkX2Fzc29jaWF0ZV9ibGtjZysweDkwLzB4OTANCj4gPiBbICA5NDguMDcwMTg0XSAgPyByZXRf ZnJvbV9mb3JrKzB4MWYvMHgzMA0KPiANCj4gVGhlc2Ugb29wcydzIHlvdSBoYXZlIGFyZSB2ZXJ5 IHN1Z2dlc3RpdmUgdGhhdCBpYl93Yy0+d3JfY3FlDQo+IGlzIGdhcmJhZ2UuLg0KPiANCj4gRGlk IFNSUCBmcmVlIGl0cyB3cl9jcWUgZGF0YSBiZWZvcmUgY29tcGxldGlvbiBzb21laG93Pw0KPiAN Cj4gVHVybiBvbiBzbGFiIHBvaXNvbmluZyB0byBjb25maXJtPw0KDQpIZWxsbyBKYXNvbiwNCg0K SXQncyBlYXN5IHRvIHNlZSBpbiBkcml2ZXJzL2luZmluaWJhbmQvY29yZS9jcS5jIHRoYXQgcG9s bGluZyBpcyBzdG9wcGVkDQpiZWZvcmUgYSBjb21wbGV0aW9uIHF1ZXVlIGlzIGRlc3Ryb3llZCAo c2VlIGFsc28gdGhlIGNhbmNlbF93b3JrX3N5bmMoJmNxLT53b3JrKQ0KYW5kIHRoZSBjcS0+ZGV2 aWNlLT5kZXN0cm95X2NxKGNxKSBjYWxscyBpbiBpYl9mcmVlX2NxKCkpLg0KDQpCVFcsIEkgcnVu IGFsbCBteSB0ZXN0cyB3aXRoIFNMQUIgcG9pc29uaW5nIGVuYWJsZWQuIE15IFNSUCB0ZXN0cyBw YXNzIGlmIEkgcnVuDQp0aGUgU1JQIGluaXRpYXRvciBhbmQgdGFyZ2V0IGRyaXZlcnMgb24gdG9w IG9mIHRoZSBtbHg0IGFuZCByZG1hX3J4ZSBkcml2ZXJzLg0KDQpCYXJ0Lg==
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi Jason

Yep, this seems specific to the mlx5 and IB.
The problem though is Linus's tree 4.15-rc-7 already has enough of the part of the RDMA updates to see issues.

With his tree I don't panic but I see this

[ 1360.511682] mlx5_core 0000:08:00.1: Shutdown was called
[ 1360.550531] mlx5_core 0000:08:00.1: mlx5_enter_error_state:121:(pid
15149): start
[ 1360.593520] ------------[ cut here ]------------
[ 1360.619930] got unsolicited completion for CQ 0x0000000068694acd
[ 1360.654434] WARNING: CPU: 15 PID: 15149 at
drivers/infiniband/core/cq.c:80 ib_cq_completion_direct+0x28/0x30
[ib_core]
[ 1360.716099] Modules linked in: xt_CHECKSUM iptable_mangle

ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter rpcrdma ib_isert iscsi_target_mod target_core_mod ib_iser libiscsi scsi_transport_iscsi ib_srp scsi_transport_srp ib_ipoib rdma_ucm ib_ucm ib_uverbs ib_umad rdma_cm ib_cm iw_cm mlx5_ib ib_core intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul ipmi_ssif ghash_clmulni_intel pcbc joydev aesni_intel dm_service_time ipmi_si crypto_simd glue_helper sg hpilo cryptd hpwdt ipmi_devintf iTCO_wdt gpio_ich acpi_power_meter iTCO_vendor_support ipmi_msghandler shpchp pcspkr i7core_edac lpc_ich

[ 1361.120851]  pcc_cpufreq nfsd auth_rpcgss nfs_acl lockd grace

dm_multipath sunrpc ip_tables xfs libcrc32c radeon i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm sd_mod drm mlx5_core mlxfw ptp serio_raw crc32c_intel i2c_core hpsa pps_core bnx2 devlink scsi_transport_sas dm_mirror dm_region_hash dm_log dm_mod

[ 1361.288913] CPU: 15 PID: 15149 Comm: reboot Tainted:
G          I      4.15.0-rc7 #1
[ 1361.333577] Hardware name: HP ProLiant DL380 G7, BIOS P67 08/16/2015
[ 1361.369976] RIP: 0010:ib_cq_completion_direct+0x28/0x30 [ib_core]
[ 1361.404971] RSP: 0018:ffffa08c8747fc60 EFLAGS: 00010086
[ 1361.435007] RAX: 0000000000000000 RBX: ffff8d37a6f8b468 RCX:
ffffffffae662928
[ 1361.474397] RDX: 0000000000000001 RSI: 0000000000000082 RDI:
0000000000000046
[ 1361.515097] RBP: ffff8d2bb07e0000 R08: 0000000000000000 R09:
0000000000000717
[ 1361.555054] R10: 0000000000000000 R11: ffffa08c8747f9c8 R12:
ffff8d2ed1edc264
[ 1361.595593] R13: ffff8d37a6f8b400 R14: ffffa08c8747fca8 R15:
0000000000000083
[ 1361.635133] FS:  00007fc09956a880(0000) GS:ffff8d37b33c0000(0000)
knlGS:0000000000000000
[ 1361.681800] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1361.714217] CR2: 0000000001034f80 CR3: 0000000ba0f9e005 CR4:
00000000000206e0
[ 1361.754794] Call Trace:
[ 1361.768980]  mlx5_ib_event+0x335/0x410 [mlx5_ib]
[ 1361.795303]  mlx5_core_event+0x7b/0x1a0 [mlx5_core]
[ 1361.823438]  ? synchronize_irq+0x35/0xa0
[ 1361.845962]  mlx5_enter_error_state+0xe4/0x1c0 [mlx5_core]
[ 1361.877382]  shutdown+0x127/0x170 [mlx5_core]
[ 1361.902688]  pci_device_shutdown+0x31/0x60
[ 1361.925924]  device_shutdown+0x101/0x1d0
[ 1361.948642]  kernel_restart+0xe/0x60
[ 1361.968517]  SYSC_reboot+0x1e8/0x210
[ 1361.988062]  ? __audit_syscall_entry+0xaf/0x100
[ 1362.013500]  ? syscall_trace_enter+0x1cc/0x2b0
[ 1362.038483]  ? __audit_syscall_exit+0x1ff/0x280
[ 1362.064598]  do_syscall_64+0x61/0x1a0
[ 1362.084635]  entry_SYSCALL64_slow_path+0x25/0x25
[ 1362.111113] RIP: 0033:0x7fc098377a56
[ 1362.131668] RSP: 002b:00007ffd4b3377e8 EFLAGS: 00000206 ORIG_RAX:
00000000000000a9
[ 1362.174578] RAX: ffffffffffffffda RBX: 0000000000000004 RCX:
00007fc098377a56
[ 1362.213620] RDX: 0000000001234567 RSI: 0000000028121969 RDI:
fffffffffee1dead
[ 1362.255259] RBP: 0000000000000000 R08: 000056141a7642a0 R09:
00007ffd4b336eb0
[ 1362.296293] R10: 0000000000000024 R11: 0000000000000206 R12:
0000000000000000
[ 1362.338341] R13: 00007ffd4b337ab0 R14: 0000000000000000 R15:
0000000000000000
[ 1362.378518] Code: 00 00 00 66 66 66 66 90 80 3d 65 e1 02 00 00 74 02
f3 c3 48 89 fe 31 c0 48 c7 c7 68 58 92 c0 c6 05 4e e1 02 00 01 e8 a8 23
d8 ec <0f> ff c3 0f 1f 44 00 00 66 66 66 66 90 41 55 45 89 c5 41 54 49 
[ 1362.483962] ---[ end trace 528ee06930a5763f ]---
[ 1362.509435] mlx5_1:mlx5_ib_event:2992:(pid 15149): warning: event on
port 0
[ 1362.548716] scsi host2: ib_srp: failed RECV status WR flushed (5)
for CQE 0000000023e53497
[ 1362.595980] mlx5_core 0000:08:00.1: mlx5_enter_error_state:128:(pid
15149): end
[ 1362.637630] mlx5_core 0000:08:00.0: Shutdown was called
[ 1362.677523] mlx5_core 0000:08:00.0: mlx5_enter_error_state:121:(pid
15149): start
[ 1362.720734] mlx5_0:mlx5_ib_event:2992:(pid 15149): warning: event on
port 0
[ 1362.760795] scsi host1: ib_srp: failed RECV status WR flushed (5)
for CQE 000000009ad07e27
[ 1362.806977] mlx5_core 0000:08:00.0: mlx5_enter_error_state:128:(pid

15149): end

With the latest RDMA tree additions I panic every time on shutdown. This is built against 4.15.0-rc2 with whatever other patches are in the RDMA tree.

I was testing Bart's tree when I panicked and we know now we hve an issue in mlx5/ib

I am waiting to see what Leon and the RDMA folks want to do so I can avoid another bisect, but if I have to instrument and/or bisect I will do it.

Regards
Laurence

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Every time you post a backtrace it is different.. The only commonality seems to be that the CQ completion core appears to be processing garbage, accompanied by these sorts of sketch kernel messages from mlx5:

My other guess is a mlx5 issue where it is returning CQ wrids it should not return?

Leon?

I don't see anything changing in this area in rdma.git for-rc, so I can't give you a guess on a patch, sorry.

Do you think this test ever worked for you? You said bisect, so I assume so?

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi Jason

Just to be clear, I have posted two types of stack traces, one where I panic the other here above where I am not panicking.

This is not any special type of test. I booted the kernel, mapped the SRP devices from the target server and proceeded to shutdown the client with shutdown -r now.
This is part of my holistic test I always do against new patches in Bart's tree.
I start with reboots, them rmmod's etc. before I go on to perform I/O against the LUNS from the target.

The panic was the first issue I came across after building a kernel with Bart's tree.
I have not even started testing anything else yet.

The trace above was provided because Bart asked me to test two kernels,

1. Linus's tree 4.15-rc7
2. The RDMA tree.

Bart's Tree panics the same as the RDMA tree I cloned.

I will look at prior release candidates in Linus's tree and see where this maybe crept in. I am of course puzzled why I am the only one to see it, other folks must have MLX5 (CX4) like I do.

Would be good to know what test was last performed on the current RDMA tree by Leon and team.

Regards
Laurence

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Guessing it is just luck which you hit.. Random corrupted memory and all..

Well, your shtudown is triggering the mlx driver shutdown code, then it looks like the SRP stuff gets cleaned up? That certainly is getting a bit exciting code wise

I see there have been some changes in the mlx5 shutdown handling recently..

As an experiment comment out the '.shutdown = shutdown' in drivers/net/ethernet/mellanox/mlx5/core/main.c?

And it would be interesting to know if your past success kernels were printing the mlx5 shutdown message too? Perhaps something core kernel changed to enable this path for your test?

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Its a solid issue each time, the shutdown.

Here is rc6, I am building rc1 now and will then go to 4.14 to peel this onion

4.15.0-rc6

[  150.600416] ---[ end trace fc9e16dc996e3246 ]---
[  150.626405] mlx5_1:mlx5_ib_event:2992:(pid 14203): warning: event on port 0
[  150.666308] scsi host1: ib_srp: failed RECV status WR flushed (5) for CQE 00000000ecb7c551
[  150.712873] mlx5_core 0000:08:00.1: mlx5_enter_error_state:128:(pid 14203): end
[  150.753463] mlx5_core 0000:08:00.0: Shutdown was called
[  150.793126] mlx5_core 0000:08:00.0: mlx5_enter_error_state:121:(pid 14203): start
[  150.835047] mlx5_0:mlx5_ib_event:2992:(pid 14203): warning: event on port 0
[  150.874155] scsi host2: ib_srp: failed RECV status WR flushed (5) for CQE 00000000f7f26a7b
[  150.919317] mlx5_core 0000:08:00.0: mlx5_enter_error_state:128:(pid 14203): end
[  151.449010] reboot: Restarting system
[  151.467644] reboot: machine restart

Almost looks like changes made may require new Firmware maybe for my CX4 card because its coming from here and I dont like to see pci_err** called.

static pci_ers_result_t mlx5_pci_err_detected(struct pci_dev *pdev, pci_channel_state_t state)
{
        struct mlx5_core_dev *dev = pci_get_drvdata(pdev); struct mlx5_priv *priv = &dev->priv;

        dev_info(&pdev->dev, "%s was called\n", __func__);

        mlx5_enter_error_state(dev, false); mlx5_unload_one(dev, priv, false);
        /* In case of kernel call drain the health wq */ if (state) {
                mlx5_drain_health_wq(dev); mlx5_pci_disable_device(dev);
        }

        return state == pci_channel_io_perm_failure ? PCI_ERS_RESULT_DISCONNECT : PCI_ERS_RESULT_NEED_RESET; }

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

git tells me the shutdown error feature is newish, and surely interacts with CQ shutdown, so guessing it is broken is a pretty good starting point.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

So as expected removing the shutdown call stops this happening.

"
As an experiment comment out the '.shutdown = shutdown' in drivers/net/ethernet/mellanox/mlx5/core/main.c?
"

Leon, can you look into this please becaus ethe issue is already in Linus' tree and with the changes in the RDMA tree its not just messaging, we panic on shutdown.

Thanks
Laurence
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Moving this to a new thread as I have a patch I am submitting for it. I will then continue with all the rest of the tests on Bart's Latest SRP/SRPT patches

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Changes to the mlx5_core call mlx5_try_fast_unload in the shutdown. This is causing error messages on shutdown and with the latest rdma tree panics due to list corruption.
Remove the mlx5_try_fast_unload call so we go back to calling the original mlx5_unload_one call only.

This patch was tested against the latest RDMA for-next tree

b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index d4a471a..1c66df6 100644

@@ -1522,9 +1522,7 @@ static void shutdown(struct pci_dev *pdev)

  int err;

  dev_info(&pdev->dev, "Shutdown was called\n");
- err = mlx5_try_fast_unload(dev);
- if (err)
- mlx5_unload_one(dev, priv, false);
+ mlx5_unload_one(dev, priv, false);
  mlx5_pci_disable_device(dev);
 }

--
1.8.3.1

Now on shutdown we are clean
Rebooting.
[  203.281646] kvm: exiting hardware virtualization
[  203.309916] sd 2:0:0:1: [sdbk] Synchronizing SCSI cache
..
..
[  204.240158] sd 1:0:0:2: [sdaf] Synchronizing SCSI cache
[  204.269623] sd 1:0:0:3: [sdae] Synchronizing SCSI cache
[  204.298736] sd 1:0:0:4: [sdad] Synchronizing SCSI cache
..
..
[  205.074525] sd 1:0:0:0: [sdd] Synchronizing SCSI cache
[  205.103639] mlx5_core 0000:08:00.1: Shutdown was called
[  208.244242] mlx5_1:wait_for_async_commands:735:(pid 14464): done with all pending requests
..
..
[  208.294459] sd 1:0:0:0: [sdd] Synchronizing SCSI cache
[  208.329616] scsi 1:0:0:0: alua: Detached
[  208.352899] sd 1:0:0:29: [sde] Synchronizing SCSI cache
[  208.388955] scsi 1:0:0:29: alua: Detached
..
..
[  212.230718] scsi host1: ib_srp: connection closed
[  226.697119] mlx5_core 0000:08:00.0: Shutdown was called
[  229.899254] mlx5_0:wait_for_async_commands:735:(pid 14464): done with all pending requests
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi Bart

So after sorting out the panic issue I started full testing of your patch sets for SRPT and SRP.

This is the fisrt time i am using your kernel for both the SRPT side (server) and SRP side (Client)

Server and Client both running kernel off your tree

[root@fedstorage ~]# uname -a
Linux fedstorage.bos.redhat.com 4.15.0-rc7.bart+ #1 SMP Thu Jan 11 14:17:33 EST 2018 x86_64 x86_64 x86_64 GNU/Linux

Prior to booting the Server on your tree I could find my LUNS, I was using 4.15 from Mike Snitzers tree for the server.

Now I see this on probe for SRP devices.

I have not changed any configuration entries for the srpt driver.

Still at
[root@fedstorage modprobe.d]# cat ib_srpt.conf options ib_srpt srp_max_req_size=8296 srpt_srq_size=32768

Server log
----------
Linux fedstorage.bos.redhat.com 4.15.0-rc7.bart+ #1 SMP Thu Jan 11 14:17:33 EST 2018 x86_64 x86_64 x86_64 GNU/Linux [root@fedstorage ~]# [  533.405291] IPv6: ADDRCONF(NETDEV_CHANGE): ib0:
link becomes ready
[  533.442884] IPv6: ADDRCONF(NETDEV_CHANGE): ib1: link becomes ready [  540.024438] ib_srpt Received SRP_LOGIN_REQ with i_port_id 7cfe:9003:0072:6e4e:7cfe:9003:0072:6ed2, t_port_id 7cfe:9003:0072:6e4e:7cfe:9003:0072:6e4e and it_iu_len 2116 on port 1 (guid=fe80:0000:0000:0000:7cfe:9003:0072:6e4e); pkey 0xffff [  540.141464] ib_srpt failed to create queue pair with sq_size = 16384 (-12) - retrying
[  540.186159] ib_srpt failed to create queue pair with sq_size = 8192 (-12) - retrying
[  540.231794] ib_srpt Rejected login for initiator 7cfe:9003:0072:6ed2: r[root@fedstorage modprobe.d]# cat ib_srpt.conf options ib_srpt srp_max_req_size=8296 srpt_srq_size=32768
et = -13.
[  540.274077] ib_srpt Rejecting login with reason 0x10006
[  540.304117] ib_srpt Received SRP_LOGIN_REQ with i_port_id 7cfe:9003:0072:6e4f:7cfe:9003:0072:6ed3, t_port_id 7cfe:9003:0072:6e4e:7cfe:9003:0072:6e4e and it_iu_len 2116 on port 1 (guid=fe80:0000:0000:0000:7cfe:9003:0072:6e4f); pkey 0xffff [  540.426295] ib_srpt failed to create queue pair with sq_size = 16384 (-12) - retrying
[  540.470829] ib_srpt failed to create queue pair with sq_size = 8192 (-12) - retrying
[  540.515817] ib_srpt Rejected login for initiator 7cfe:9003:0072:6ed3: ret = -13.
[  540.556485] ib_srpt Rejecting login with reason 0x10006

Client log
----------
[  554.414466] IPv6: ADDRCONF(NETDEV_CHANGE): ib0: link becomes ready [  559.264828] scsi host2: ib_srp: SRP LOGIN from fe80:0000:0000:0000:7cfe:9003:0072:6ed2 to fe80:0000:0000:0000:7cfe:9003:0072:6e4e REJECTED, reason 0x00010006 [  559.342571] scsi host2: ib_srp: Connection 0/8 to fe80:0000:0000:0000:7cfe:9003:0072:6e4e failed
[  559.546713] scsi host1: ib_srp: SRP LOGIN from fe80:0000:0000:0000:7cfe:9003:0072:6ed3 to fe80:0000:0000:0000:7cfe:9003:0072:6e4f REJECTED, reason 0x00010006 [  559.626139] scsi host1: ib_srp: Connection 0/8 to fe80:0000:0000:0000:7cfe:9003:0072:6e4f failed
[  591.325685] scsi host1: ib_srp: SRP LOGIN from fe80:0000:0000:0000:7cfe:9003:0072:6ed2 to fe80:0000:0000:0000:7cfe:9003:0072:6e4e REJECTED, reason 0x00010006 [  591.404376] scsi host1: ib_srp: Connection 0/8 to fe80:0000:0000:0000:7cfe:9003:0072:6e4e failed
[  591.605330] scsi host2: ib_srp: SRP LOGIN from fe80:0000:0000:0000:7cfe:9003:0072:6ed3 to fe80:0000:0000:0000:7cfe:9003:0072:6e4f REJECTED, reason 0x00010006 [  591.681975] scsi host2: ib_srp: Connection 0/8 to fe80:0000:0000:0000:7cfe:9003:0072:6e4f failed
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

T24gVGh1LCAyMDE4LTAxLTExIGF0IDE1OjQzIC0wNTAwLCBMYXVyZW5jZSBPYmVybWFuIHdyb3Rl Og0KPiBTZXJ2ZXIgbG9nDQo+IC0tLS0tLS0tLS0NCj4gTGludXggZmVkc3RvcmFnZS5ib3MucmVk aGF0LmNvbSA0LjE1LjAtcmM3LmJhcnQrICMxIFNNUCBUaHUgSmFuIDExDQo+IDE0OjE3OjMzIEVT VCAyMDE4IHg4Nl82NCB4ODZfNjQgeDg2XzY0IEdOVS9MaW51eA0KPiBbcm9vdEBmZWRzdG9yYWdl IH5dIyBbICA1MzMuNDA1MjkxXSBJUHY2OiBBRERSQ09ORihORVRERVZfQ0hBTkdFKTogaWIwOiBs aW5rIGJlY29tZXMgcmVhZHkNCj4gWyAgNTMzLjQ0Mjg4NF0gSVB2NjogQUREUkNPTkYoTkVUREVW X0NIQU5HRSk6IGliMTogbGluayBiZWNvbWVzIHJlYWR5DQo+IFsgIDU0MC4wMjQ0MzhdIGliX3Ny cHQgUmVjZWl2ZWQgU1JQX0xPR0lOX1JFUSB3aXRoIGlfcG9ydF9pZCA3Y2ZlOjkwMDM6MDA3Mjo2 ZTRlOjdjZmU6OTAwMzowMDcyOjZlZDIsIHRfcG9ydF9pZCA3Y2ZlOjkwMDM6MDA3Mjo2ZTRlOjdj ZmU6OTAwMzowMDcyOjZlNGUgYW5kIGl0X2l1X2xlbiAyMTE2IG9uIHBvcnQgMQ0KPiAoZ3VpZD1m ZTgwOjAwMDA6MDAwMDowMDAwOjdjZmU6OTAwMzowMDcyOjZlNGUpOyBwa2V5IDB4ZmZmZg0KPiBb ICA1NDAuMTQxNDY0XSBpYl9zcnB0IGZhaWxlZCB0byBjcmVhdGUgcXVldWUgcGFpciB3aXRoIHNx X3NpemUgPSAxNjM4NCAoLTEyKSAtIHJldHJ5aW5nDQo+IFsgIDU0MC4xODYxNTldIGliX3NycHQg ZmFpbGVkIHRvIGNyZWF0ZSBxdWV1ZSBwYWlyIHdpdGggc3Ffc2l6ZSA9IDgxOTIgKC0xMikgLSBy ZXRyeWluZw0KPiBbICA1NDAuMjMxNzk0XSBpYl9zcnB0IFJlamVjdGVkIGxvZ2luIGZvciBpbml0 aWF0b3IgN2NmZTo5MDAzOjAwNzI6NmVkMjogcmV0ID0gLTEzLg0KPiBbICA1NDAuMjc0MDc3XSBp Yl9zcnB0IFJlamVjdGluZyBsb2dpbiB3aXRoIHJlYXNvbiAweDEwMDA2DQoNCkhlbGxvIExhdXJl bmNlLA0KDQpJIHRoaW5rIHRoYXQgbWVhbnMgdGhhdCBubyBBQ0wgd2FzIGNvbmZpZ3VyZWQgb24g dGhlIHRhcmdldCBzaWRlIGZvcg0KaW5pdGlhdG9yIHBvcnQgN2NmZTo5MDAzOjAwNzI6NmVkMi4g Q2FuIHlvdSBwcm92aWRlIHRoZSBvdXRwdXQgb2YNCihjZCAvc3lzL2tlcm5lbC9jb25maWcvdGFy Z2V0L3NycHQgJiYgbHMgLWQgKi8qL2FjbHMvKik/DQoNClRoYW5rcywNCg0KQmFydC4NCg== --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

ʇڙ,jfhz
w
j:+vwjmzZ+ݢj"!

Hi Bart

I have not changed the configuration in LIO target though. The /etc/target json file is the same as always and Its been like this for years

I just rebooted the server into 4.13 and its fine again and found all the targets with the same kernel on the client.

So its specific to your new tree with srpt

I will reboot again and re-load LIO and show you but here is my ACL list that has been this way for some time.

o- srpt
.......................................................................
...................................... [Targets: 2] | o- ib.fe800000000000007cfe900300726e4e
.......................................................................
.... [no-gen-acls]
  | | o- acls
.......................................................................
..................................... [ACLs: 8] | | | o- ib.4e6e72000390fe7c7cfe900300726ed2
...................................................................
[Mapped LUNs: 30]
  | | | o- ib.4e6e72000390fe7c7cfe900300726ed3
...................................................................
[Mapped LUNs: 30]
  | | | o- ib.4f6e72000390fe7c7cfe900300726ed2
...................................................................
[Mapped LUNs: 30]
  | | | o- ib.4f6e72000390fe7c7cfe900300726ed3
...................................................................
[Mapped LUNs: 30]
  | | | o- ib.7cfe900300726e4e7cfe900300726ed2
...................................................................
[Mapped LUNs: 30]
  | | | o- ib.7cfe900300726e4e7cfe900300726ed3
...................................................................
[Mapped LUNs: 30]
  | | | o- ib.7cfe900300726e4f7cfe900300726ed2
...................................................................
[Mapped LUNs: 30]
  | | | o- ib.fe800000000000007cfe900300726e4e
...................................................................
[Mapped LUNs: 30]
  | o- ib.fe800000000000007cfe900300726e4f
.......................................................................
.... [no-gen-acls]
  |   o- acls
.......................................................................
..................................... [ACLs: 8] |   | o- ib.4e6e72000390fe7c7cfe900300726ed2
...................................................................
[Mapped LUNs: 30]
  |   | o- ib.4e6e72000390fe7c7cfe900300726ed3
...................................................................
[Mapped LUNs: 30]
  |   | o- ib.4f6e72000390fe7c7cfe900300726ed2
...................................................................
[Mapped LUNs: 30]
  |   | o- ib.4f6e72000390fe7c7cfe900300726ed3
...................................................................
[Mapped LUNs: 30]
  |   | o- ib.7cfe900300726e4e7cfe900300726ed2
...................................................................
[Mapped LUNs: 30]
  |   | o- ib.7cfe900300726e4e7cfe900300726ed3
...................................................................
[Mapped LUNs: 30]
  |   | o- ib.7cfe900300726e4f7cfe900300726ed2
...................................................................
[Mapped LUNs: 30]
  |   | o- ib.7cfe900300726e4f7cfe900300726ed3
...................................................................
[Mapped LUNs: 30]

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

T24gVGh1LCAyMDE4LTAxLTExIGF0IDE2OjMzIC0wNTAwLCBMYXVyZW5jZSBPYmVybWFuIHdyb3Rl Og0KPiBJIGhhdmUgbm90IGNoYW5nZWQgdGhlIGNvbmZpZ3VyYXRpb24gaW4gTElPIHRhcmdldCB0 aG91Z2guDQoNClllcywgSSB0cnVzdCB0aGF0IHlvdSBoYXZlbid0IGNoYW5nZWQgdGhlIGNvbmZp Z3VyYXRpb24gYnV0IEkgZGlkbid0IHRydXN0DQp3aGV0aGVyIG15IG93biBpYl9zcnB0IGNoYW5n ZXMgaGFkIHByZXNlcnZlZCBiYWNrd2FyZHMgY29tcGF0aWJpbGl0eSA6LSkNCg0KPiAgIHwgfCB8 IG8tIGliLjRlNmU3MjAwMDM5MGZlN2M3Y2ZlOTAwMzAwNzI2ZWQyDQo+IC4uLi4uLi4uLi4uLi4u Li4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4NCj4g DQo+IFsgLi4uIF0NCj4gW01hcHBlZCBMVU5zOiAzMF0NCj4gICB8IHwgfCBvLSBpYi40ZjZlNzIw MDAzOTBmZTdjN2NmZTkwMDMwMDcyNmVkMg0KDQpUaGFua3MsIHRoaXMgY29uZmlybXMgbXkgc3Vz cGljaW9uLiBJIHdpbGwgc2VlIHdoYXQgbmVlZHMgdG8gYmUgY2hhbmdlZCBpbg0KdGhlIGliX3Ny cHQgZHJpdmVyIHRvIHJlc3RvcmUgYmFja3dhcmRzIGNvbXBhdGliaWxpdHkuDQoNCkJhcnQu --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

T24gVGh1LCAyMDE4LTAxLTExIGF0IDE2OjMzIC0wNTAwLCBMYXVyZW5jZSBPYmVybWFuIHdyb3Rl Og0KPiBJIGp1c3QgcmVib290ZWQgdGhlIHNlcnZlciBpbnRvIDQuMTMgYW5kIGl0cyBmaW5lIGFn YWluIGFuZCBmb3VuZCBhbGwNCj4gdGhlIHRhcmdldHMgd2l0aCB0aGUgc2FtZSBrZXJuZWwgb24g dGhlIGNsaWVudC4NCj4gDQo+IFNvIGl0cyBzcGVjaWZpYyB0byB5b3VyIG5ldyB0cmVlIHdpdGgg c3JwdA0KPiANCj4gSSB3aWxsIHJlYm9vdCBhZ2FpbiBhbmQgcmUtbG9hZCBMSU8gYW5kIHNob3cg eW91IGJ1dCBoZXJlIGlzIG15IEFDTA0KPiBsaXN0IHRoYXQgaGFzIGJlZW4gdGhpcyB3YXkgZm9y IHNvbWUgdGltZS4NCj4gDQo+IA0KPiBvLSBzcnB0DQo+IC4uLi4uLi4uLi4uLi4uLi4uLi4uLi4u Li4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uDQo+IC4uLi4u Li4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uIFtUYXJnZXRzOiAyXQ0KPiAgIHwgby0g aWIuZmU4MDAwMDAwMDAwMDAwMDdjZmU5MDAzMDA3MjZlNGUNCj4gLi4uLi4uLi4uLi4uLi4uLi4u Li4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4NCj4g Li4uLiBbbm8tZ2VuLWFjbHNdDQo+ICAgfCB8IG8tIGFjbHMNCj4gLi4uLi4uLi4uLi4uLi4uLi4u Li4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4NCj4g Li4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLiBbQUNMczogOF0NCj4gICB8IHwg fCBvLSBpYi40ZTZlNzIwMDAzOTBmZTdjN2NmZTkwMDMwMDcyNmVkMg0KPiANCj4gWyAuLi4gXQ0K DQpIZWxsbyBMYXVyZW5jZSwNCg0KQWx0aG91Z2ggSSdtIG5vdCBzdXJlIEkgdGhpbmsgSSBmb3Vu ZCB0aGUgcm9vdCBjYXVzZSBvZiB0aGlzIGZhaWx1cmUuIFRoZQ0KZm9sbG93aW5nIHBhdGNoIHNo b3VsZCBmaXggdGhlIGZhaWx1cmU6DQoNCmRpZmYgLS1naXQgYS9kcml2ZXJzL2luZmluaWJhbmQv dWxwL3NycHQvaWJfc3JwdC5jIGIvZHJpdmVycy9pbmZpbmliYW5kL3VscC9zcnB0L2liX3NycHQu Yw0KaW5kZXggOTYxNDIxMTBhMTU1Li41Mjk3OTYzYzgzNGQgMTAwNjQ0DQotLS0gYS9kcml2ZXJz L2luZmluaWJhbmQvdWxwL3NycHQvaWJfc3JwdC5jDQorKysgYi9kcml2ZXJzL2luZmluaWJhbmQv dWxwL3NycHQvaWJfc3JwdC5jDQpAQCAtMjA4Myw3ICsyMDgzLDcgQEAgc3RhdGljIGludCBzcnB0 X2NtX3JlcV9yZWN2KHN0cnVjdCBzcnB0X2RldmljZSAqY29uc3Qgc2RldiwNCiAJCXN0cnVjdCBp Yl9jbV9yZXBfcGFyYW0gaWJfY207DQogCX0gKnJlcF9wYXJhbSA9IE5VTEw7DQogCXN0cnVjdCBz cnB0X3JkbWFfY2ggKmNoOw0KLQljaGFyIGlfcG9ydF9pZFsyNF07DQorCWNoYXIgaV9wb3J0X2lk WzM2XTsNCiAJdTMyIGl0X2l1X2xlbjsNCiAJaW50IGksIHJldDsNCiANCmRpZmYgLS1naXQgYS9k cml2ZXJzL2luZmluaWJhbmQvdWxwL3NycHQvaWJfc3JwdC5oIGIvZHJpdmVycy9pbmZpbmliYW5k L3VscC9zcnB0L2liX3NycHQuaA0KaW5kZXggYmY0NTI1YjI0ZDk4Li4wMjg4M2Y4ZTljNzEgMTAw NjQ0DQotLS0gYS9kcml2ZXJzL2luZmluaWJhbmQvdWxwL3NycHQvaWJfc3JwdC5oDQorKysgYi9k cml2ZXJzL2luZmluaWJhbmQvdWxwL3NycHQvaWJfc3JwdC5oDQpAQCAtMzA4LDcgKzMwOCw3IEBA IHN0cnVjdCBzcnB0X3JkbWFfY2ggew0KIAlib29sCQkJdXNpbmdfcmRtYV9jbTsNCiAJYm9vbAkJ CXByb2Nlc3Npbmdfd2FpdF9saXN0Ow0KIAlzdHJ1Y3Qgc2Vfc2Vzc2lvbgkqc2VzczsNCi0JdTgJ CQlzZXNzX25hbWVbMzZdOw0KKwl1OAkJCXNlc3NfbmFtZVsyNF07DQogCXN0cnVjdCB3b3JrX3N0 cnVjdAlyZWxlYXNlX3dvcms7DQogfTsNCiANCg0KSSB3cm90ZSAic2hvdWxkIiBiZWNhdXNlIHRh cmdldGNsaSBpcyBub3QgaW5zdGFsbGVkIG9uIG15IHRlc3Qgc2V0dXAgYW5kDQpiZWNhdXNlIEkg aGF2ZSBub3QgeWV0IHZlcmlmaWVkIHRoaXMgY2hhbmdlIHdpdGggdGFyZ2V0Y2xpLiBJZiB5b3Ug aGF2ZSB0aGUNCnRpbWUgdG8gdmVyaWZ5IHRoaXMgY2hhbmdlIHRoYXQgd291bGQgYmUgZ3JlYXQu IElmIG5vdCB0aGVuIEkgd2lsbCBpbnN0YWxsDQp0YXJnZXRjbGkgbXlzZWxmIGFuZCB2ZXJpZnkg dGhpcyBjaGFuZ2UuDQoNClRoYW5rcywNCg0KQmFydC4=
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

ʇڙ,jfhz
w
j:+vwjmzZ+ݢj"!

Hi Bart

I will get this tested tonight and report back.

Fix makes sesne.

Regards
Laurence
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

ʇڙ,jfhz
w
j:+vwjmzZ+ݢj"!

Hello Bart
For the patch above:

This corrects the connectivity issue with LIO targets and I will continue now testing your patches from your tree.

Thank you for your quick response Sir.

Laurence

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

ʇڙ,jfhz
w
j:+vwjmzZ+ݢj"!

Hello Bart

I missed some logs when I tested last night.
Its working fine as mentioned with the the above patch and I see all the targets (That's what I checked for).

However I still see these in the srpt server, but I get access to all the targets now on the client.

[  239.502025] ib_srpt Received SRP_LOGIN_REQ with i_port_id 7cfe:9003:0072:6e4f:7cfe:9003:0072:6ed3, t_port_id 7cfe:9003:0072:6e4e:7cfe:9003:0072:6e4e and it_iu_len 2116 on port 1 (guid=fe80:0000:0000:0000:7cfe:9003:0072:6e4f); pkey 0xffff [  239.623881] ib_srpt failed to create queue pair with sq_size = 16384 (-12) - retrying
[  239.669381] ib_srpt failed to create queue pair with sq_size = 8192 (-12) - retrying
[  239.715366] ib_srpt Received SRP_LOGIN_REQ with i_port_id 7cfe:9003:0072:6e4e:7cfe:9003:0072:6ed2, t_port_id 7cfe:9003:0072:6e4e:7cfe:9003:0072:6e4e and it_iu_len 2116 on port 1 (guid=fe80:0000:0000:0000:7cfe:9003:0072:6e4e); pkey 0xffff [  239.831661] ib_srpt failed to create queue pair with sq_size = 16384 (-12) - retrying
[  239.877193] ib_srpt failed to create queue pair with sq_size = 8192 (-12) - retrying
[  239.967259] ib_srpt Received SRP_LOGIN_REQ with i_port_id 7cfe:9003:0072:6e4f:7cfe:9003:0072:6ed3, t_port_id 7cfe:9003:0072:6e4e:7cfe:9003:0072:6e4e and it_iu_len 2116 on port 1 (guid=fe80:0000:0000:0000:7cfe:9003:0072:6e4f); pkey 0xffff [  240.087362] ib_srpt failed to create queue pair with sq_size = 16384 (-12) - retrying
[  240.130981] ib_srpt failed to create queue pair with sq_size = 8192 (-12) - retrying
..
..

So the functional report was valid but we need to see why we are still getting the messages above.

Apologies, should have checked all the logs last night before my first reply.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

T24gU2F0LCAyMDE4LTAxLTEzIGF0IDA5OjUzIC0wNTAwLCBMYXVyZW5jZSBPYmVybWFuIHdyb3Rl Og0KPiBbICAyMzkuNTAyMDI1XSBpYl9zcnB0IFJlY2VpdmVkIFNSUF9MT0dJTl9SRVEgd2l0aCBp X3BvcnRfaWQNCj4gN2NmZTo5MDAzOjAwNzI6NmU0Zjo3Y2ZlOjkwMDM6MDA3Mjo2ZWQzLCB0X3Bv cnRfaWQNCj4gN2NmZTo5MDAzOjAwNzI6NmU0ZTo3Y2ZlOjkwMDM6MDA3Mjo2ZTRlIGFuZCBpdF9p dV9sZW4gMjExNiBvbiBwb3J0IDENCj4gKGd1aWQ9ZmU4MDowMDAwOjAwMDA6MDAwMDo3Y2ZlOjkw MDM6MDA3Mjo2ZTRmKTsgcGtleSAweGZmZmYNCj4gWyAgMjM5LjYyMzg4MV0gaWJfc3JwdCBmYWls ZWQgdG8gY3JlYXRlIHF1ZXVlIHBhaXIgd2l0aCBzcV9zaXplID0gMTYzODQNCj4gKC0xMikgLSBy ZXRyeWluZw0KPiBbICAyMzkuNjY5MzgxXSBpYl9zcnB0IGZhaWxlZCB0byBjcmVhdGUgcXVldWUg cGFpciB3aXRoIHNxX3NpemUgPSA4MTkyDQo+ICgtMTIpIC0gcmV0cnlpbmcNCj4gWyAgMjM5Ljcx NTM2Nl0gaWJfc3JwdCBSZWNlaXZlZCBTUlBfTE9HSU5fUkVRIHdpdGggaV9wb3J0X2lkDQo+IDdj ZmU6OTAwMzowMDcyOjZlNGU6N2NmZTo5MDAzOjAwNzI6NmVkMiwgdF9wb3J0X2lkDQo+IDdjZmU6 OTAwMzowMDcyOjZlNGU6N2NmZTo5MDAzOjAwNzI6NmU0ZSBhbmQgaXRfaXVfbGVuIDIxMTYgb24g cG9ydCAxDQo+IChndWlkPWZlODA6MDAwMDowMDAwOjAwMDA6N2NmZTo5MDAzOjAwNzI6NmU0ZSk7 IHBrZXkgMHhmZmZmDQo+IFsgIDIzOS44MzE2NjFdIGliX3NycHQgZmFpbGVkIHRvIGNyZWF0ZSBx dWV1ZSBwYWlyIHdpdGggc3Ffc2l6ZSA9IDE2Mzg0DQo+ICgtMTIpIC0gcmV0cnlpbmcNCj4gWyAg MjM5Ljg3NzE5M10gaWJfc3JwdCBmYWlsZWQgdG8gY3JlYXRlIHF1ZXVlIHBhaXIgd2l0aCBzcV9z aXplID0gODE5Mg0KPiAoLTEyKSAtIHJldHJ5aW5nDQoNCkhlbGxvIExhdXJlbmNlLA0KDQpUaGVz ZSBtZXNzYWdlcyBhcmUgZXhwZWN0ZWQgYW5kIGRvIG5vdCBpbmRpY2F0ZSBhIGZhaWx1cmUuIFRo ZSByZXRyeSBsb29wDQp0aGUgYWJvdmUgbWVzc2FnZXMgcmVmZXIgdG8gZ290IGludHJvZHVjZWQg YSBsb25nIHRpbWUgYWdvOg0KDQpjb21taXQgYWI0NzdjMWZmNWUwYTc0NGMwNzI0MDRiZjdkYjUx YmZlMWYwNWI2ZQ0KQXV0aG9yOiBCYXJ0IFZhbiBBc3NjaGUgPGJ2YW5hc3NjaGVAYWNtLm9yZz4N CkRhdGU6ICAgU3VuIE9jdCAxOSAxODowNTozMyAyMDE0ICswMzAwDQoNCiAgICBzcnAtdGFyZ2V0 OiBSZXRyeSB3aGVuIFFQIGNyZWF0aW9uIGZhaWxzIHdpdGggRU5PTUVNDQogICAgDQogICAgSXQg aXMgbm90IGd1YXJhbnRlZWQgdG8gdGhhdCBzcnBfc3Ffc2l6ZSBpcyBzdXBwb3J0ZWQNCiAgICBi eSB0aGUgSENBLiBTbyBpZiB3ZSBmYWlsZWQgdG8gY3JlYXRlIHRoZSBRUCB3aXRoIEVOT01FTSwN CiAgICB0cnkgd2l0aCBhIHNtYWxsZXIgc3JwX3NxX3NpemUuIEtlZXAgaXQgdXAgdW50aWwgd2Ug aGl0DQogICAgTUlOX1NSUFRfU1FfU0laRSwgdGhlbiBmYWlsIHRoZSBjb25uZWN0aW9uLg0KICAg IA0KWyAuLi4gXQ0KDQpUaGUgb25seSByZWNlbnQgY2hhbmdlIGluIHRoYXQgY29kZSBpcyB0aGF0 IHJldHJ5IGF0dGVtcHRzIGFyZSBub3cgbG9nZ2VkLg0KRnJvbSBjb21taXQgMGU5OTQ5ZjFkYjZj ICJJQi9zcnB0OiBBZGQgUkRNQS9DTSBzdXBwb3J0IjoNCg0KKyAgICAgICBpZiAocmV0KSB7DQor ICAgICAgICAgICAgICAgYm9vbCByZXRyeSA9IHNxX3NpemUgPiBNSU5fU1JQVF9TUV9TSVpFOw0K Kw0KKyAgICAgICAgICAgICAgIHByX2VycigiZmFpbGVkIHRvIGNyZWF0ZSBxdWV1ZSBwYWlyIHdp dGggc3Ffc2l6ZSA9ICVkICglZCklc1xuIiwNCisgICAgICAgICAgICAgICAgICAgICAgc3Ffc2l6 ZSwgcmV0LCByZXRyeSA/ICIgLSByZXRyeWluZyIgOiAiIik7DQorICAgICAgICAgICAgICAgaWYg KHJldHJ5KSB7DQorICAgICAgICAgICAgICAgICAgICAgICBpYl9mcmVlX2NxKGNoLT5jcSk7DQor ICAgICAgICAgICAgICAgICAgICAgICBzcV9zaXplID0gbWF4KHNxX3NpemUgLyAyLCBNSU5fU1JQ VF9TUV9TSVpFKTsNCisgICAgICAgICAgICAgICAgICAgICAgIGdvdG8gcmV0cnk7DQorICAgICAg ICAgICAgICAgfSBlbHNlIHsNCisgICAgICAgICAgICAgICAgICAgICAgIGdvdG8gZXJyX2Rlc3Ry b3lfY3E7DQogICAgICAgICAgICAgICAgfQ0KLSAgICAgICAgICAgICAgIHByX2VycigiZmFpbGVk IHRvIGNyZWF0ZV9xcCByZXQ9ICVkXG4iLCByZXQpOw0KLSAgICAgICAgICAgICAgIGdvdG8gZXJy X2Rlc3Ryb3lfY3E7DQogICAgICAgIH0NCg0KRG8geW91IHBlcmhhcHMgd2FudCB0aGF0IHByX2Vy cigpIHRvIGJlIGNoYW5nZWQgaW50byBhIHByX2RlYnVnKCkgZm9yIHJldHJ5DQphdHRlbXB0cz8N Cg0KVGhhbmtzLA0KDQpCYXJ0Lg==
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi Bart,

I recognized those as maybe just reporting messages so I thought we were were good with the recent patch to fix the connection issue. However when I attempted to actually use the targets with your latest SRPT I had failures on the client.

It was a tough weekend for me, and maybe I made mistakes. Let me complete the irq/cpu test Ming is waiting for and I will revisit this fully with a clean build and your most recent patch.

I will answer off list while we figure it out

Many Thanks
Laurence
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

I will do this next, its possible its been there for a while and missed as with no panics the messages would not have been a focus.

However keep in mind that other change sin the RDMA tree are more sordid to see this shutdown then leas to list corruptions and panics.

Starting to make sense now based on what you said about the new shutdown code.

Just going to try rc1 and then will do below as a test.

"As an experiment comment out the '.shutdown = shutdown' in

"

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

But that has nothing directly to do with the lifetime of, say, struct srp_request which contains ib_wc->wr_cqe?

eg freeing struct srp_request before the wrid has passed through the CQ poll would produce these sorts of symptoms...

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

T24gV2VkLCAyMDE4LTAxLTEwIGF0IDEyOjE3IC0wNzAwLCBKYXNvbiBHdW50aG9ycGUgd3JvdGU6 DQo+IEJ1dCB0aGF0IGhhcyBub3RoaW5nIGRpcmVjdGx5IHRvIGRvIHdpdGggdGhlIGxpZmV0aW1l IG9mLCBzYXksIHN0cnVjdA0KPiBzcnBfcmVxdWVzdCB3aGljaCBjb250YWlucyBpYl93Yy0+d3Jf Y3FlPw0KPiANCj4gZWcgZnJlZWluZyBzdHJ1Y3Qgc3JwX3JlcXVlc3QgYmVmb3JlIHRoZSB3cmlk IGhhcyBwYXNzZWQgdGhyb3VnaCB0aGUNCj4gQ1EgcG9sbCB3b3VsZCBwcm9kdWNlIHRoZXNlIHNv cnRzIG9mIHN5bXB0b21zLi4uDQoNCkhlbGxvIEphc29uLA0KDQpUaGUgU1JQIGluaXRpYXRvciBk cml2ZXIgUkRNQSBjaGFubmVsIHNodXRkb3duIHNlcXVlbmNlIGlzIGFzIGZvbGxvd3M6DQoqIHNy cF9yZW1vdmVfdGFyZ2V0KCkgY2FsbHMgc3JwX2ZyZWVfY2hfaWIoKS4gVGhhdCBsYXN0IGZ1bmN0 aW9uIGNhbGxzDQogIHNycF9kZXN0cm95X3FwKCkgd2hpY2ggaW4gdHVybiBjYWxscyBpYl9kcmFp bl9xcCgpLiBpYl9kcmFpbl9xcCgpIHdhaXRzIHVudGlsDQogIGFsbCBDUUVzIGhhdmUgYmVlbiBk ZXF1ZXVlZCBieSBjaGFuZ2luZyB0aGUgUVAgc3RhdGUgaW50byBJQl9RUFNfRVJSIGFuZCBieQ0K ICB3YWl0aW5nIHVudGlsIHRoZSBjb21wbGV0aW9uIGZvciBhIG5ld2x5IHBvc3RlZCByZXF1ZXN0 IGhhcyBiZWVuIHJlY2VpdmVkLg0KKiBzcnBfcmVtb3ZlX3RhcmdldCgpIGNhbGxzIHNycF9mcmVl X3JlcV9kYXRhKCkuIFRoYXQgbGFzdCBmdW5jdGlvbg0KICBjYWxscyBrZnJlZShjaC0+cmVxX3Jp bmcpLCB0aGF0IGlzIHRoZSBkYXRhIHN0cnVjdHVyZSB0aGF0IGNvbnRhaW5zIHRoZQ0KICBTUlAg cmVxdWVzdCBzdHJ1Y3R1cmVzLg0KDQpCYXJ0Lg==
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

Thanks Bart, informative..

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html