Skip to contentSkip to conversation

iscsi_iser: Re-enable 'iser_pi_guard' module parameter

PATCH In review Hannes Reinecke 7 participants 13 messages Jan 10, 2018 – Jan 18, 2018 10 messages not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 iscsi_iser: Re-enable 'iser_pi_guard' module parameter

The module parameter 'iser_pi_guard' has been disabled by commit 5bb6e543d2a7d58 ("IB/iser: DIX update"), but the functionality to select the guard algorithm is still required.

Fixes 5bb6e543d2a7d58 ("IB/iser: DIX update")
Cc Steve Schremmer <[email protected]>
---
 drivers/infiniband/ulp/iser/iscsi_iser.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/infiniband/ulp/iser/iscsi_iser.c +4 -2
index 19624e0..34c4d0a 100644
@@ -649,8 +649,10 @@ static void iscsi_iser_cleanup_task(struct iscsi_task *task)
 			u32 sig_caps = ib_conn->device->ib_device->attrs.sig_prot_cap;
 
 			scsi_host_set_prot(shost, iser_dif_prot_caps(sig_caps));
-			scsi_host_set_guard(shost, SHOST_DIX_GUARD_IP |
-						   SHOST_DIX_GUARD_CRC);
+			if (iser_pi_guard)
+				scsi_host_set_guard(shost, SHOST_DIX_GUARD_IP);
+			else
+				scsi_host_set_guard(shost, SHOST_DIX_GUARD_CRC);
 		}
 
 		if (iscsi_host_add(shost,
Signature · 6 lines
1.8.5.6

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#2 lore raw
In reply to a message not in the archive
Quoted 3 lines · Hannes Reinecke

The module parameter 'iser_pi_guard' has been disabled by commit 5bb6e543d2a7d58 ("IB/iser: DIX update"), but the functionality to select the guard algorithm is still required.

Commit should explain why it is still required? What is the actual bug here?

Someone who understands this is going to have to Ack it for it to go through the rdma tree..

Quoted 12 lines
Fixes 5bb6e543d2a7d58 ("IB/iser: DIX update")
Cc Steve Schremmer <[email protected]>
 drivers/infiniband/ulp/iser/iscsi_iser.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/infiniband/ulp/iser/iscsi_iser.c +1 -0
index 19624e0..34c4d0a 100644
+++ b/drivers/infiniband/ulp/iser/iscsi_iser.c
@@ -649,8 +649,10 @@ static void iscsi_iser_cleanup_task(struct iscsi_task *task)
 			u32 sig_caps = ib_conn->device->ib_device->attrs.sig_prot_cap;
 

drivers/infiniband/ulp/iser/iscsi_iser.c static void iscsi_iser_cleanup_task(struct iscsi_task *task) · line 658

Quoted 10 lines
drivers/infiniband/ulp/iser/iscsi_iser.c +4 -2
 			scsi_host_set_prot(shost, iser_dif_prot_caps(sig_caps));
-			scsi_host_set_guard(shost, SHOST_DIX_GUARD_IP |
-						   SHOST_DIX_GUARD_CRC);
+			if (iser_pi_guard)
+				scsi_host_set_guard(shost, SHOST_DIX_GUARD_IP);
+			else
+				scsi_host_set_guard(shost, SHOST_DIX_GUARD_CRC);
 		}
 
 		if (iscsi_host_add(shost,

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw
In reply to a message not in the archive
Quoted 4 lines

From: Jason Gunthorpe [mailto:jgg-uk2M96/[email protected]]
Sent: Wednesday, January 10, 2018 5:03 PM

On Wed, Jan 10, 2018 at 10:07:36AM +0100, Hannes Reinecke wrote:

Quoted 9 lines
Quoted 3 lines

The module parameter 'iser_pi_guard' has been disabled by commit 5bb6e543d2a7d58 ("IB/iser: DIX update"), but the functionality to select the guard algorithm is still required.

Commit should explain why it is still required? What is the actual bug here?

Someone who understands this is going to have to Ack it for it to go through the rdma tree..

Without the module parameter, there is no way to actually use the CRC guard format. Currently, iscsi_iser_session_create() indicates support for both IP and CRC formats, but sd_dif_config_host() always checks for IP support first, so CRC guard won't be used. --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#6 lore raw
In reply to a message not in the archive
Quoted 8 lines · Steven
Quoted 8 lines

From: Jason Gunthorpe [mailto:jgg-uk2M96/[email protected]]
Sent: Wednesday, January 10, 2018 5:03 PM

Quoted 3 lines · Hannes Reinecke

The module parameter 'iser_pi_guard' has been disabled by commit 5bb6e543d2a7d58 ("IB/iser: DIX update"), but the functionality to select the guard algorithm is still required.

Quoted 8 lines
Quoted 5 lines

Commit should explain why it is still required? What is the actual bug here?

Someone who understands this is going to have to Ack it for it to go through the rdma tree..

Without the module parameter, there is no way to actually use the CRC guard format. Currently, iscsi_iser_session_create() indicates support for both IP and CRC formats, but sd_dif_config_host() always checks for IP support first, so CRC guard won't be used.

The above paragraph would be a great addition to the commit message.

Still need an ack :)

BTW - not knowing anything, why isn't this knob in the core sd code? Other drivers need it too from whatI could see?

We really don't like module parameters in the kernel.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#8 lore raw
In reply to a message not in the archive

T24gVGh1LCAyMDE4LTAxLTExIGF0IDA5OjU4IC0wNzAwLCBKYXNvbiBHdW50aG9ycGUgd3JvdGU6 DQo+IE9uIFRodSwgSmFuIDExLCAyMDE4IGF0IDEyOjA0OjI3QU0gKzAwMDAsIFNjaHJlbW1lciwg U3RldmVuIHdyb3RlOg0KPiA+IFdpdGhvdXQgdGhlIG1vZHVsZSBwYXJhbWV0ZXIsIHRoZXJlIGlz IG5vIHdheSB0byBhY3R1YWxseSB1c2UgdGhlIENSQyBndWFyZCBmb3JtYXQuDQo+ID4gQ3VycmVu dGx5LCBpc2NzaV9pc2VyX3Nlc3Npb25fY3JlYXRlKCkgaW5kaWNhdGVzIHN1cHBvcnQgZm9yIGJv dGggSVAgYW5kIENSQyBmb3JtYXRzLCBidXQgDQo+ID4gc2RfZGlmX2NvbmZpZ19ob3N0KCkgYWx3 YXlzIGNoZWNrcyBmb3IgSVAgc3VwcG9ydCBmaXJzdCwgc28gQ1JDIGd1YXJkIHdvbid0IGJlIHVz ZWQuDQo+IA0KPiBUaGUgYWJvdmUgcGFyYWdyYXBoIHdvdWxkIGJlIGEgZ3JlYXQgYWRkaXRpb24g dG8gdGhlIGNvbW1pdCBtZXNzYWdlLg0KPiANCj4gU3RpbGwgbmVlZCBhbiBhY2sgOikNCj4gDQo+ IEJUVyAtIG5vdCBrbm93aW5nIGFueXRoaW5nLCB3aHkgaXNuJ3QgdGhpcyBrbm9iIGluIHRoZSBj b3JlIHNkIGNvZGU/DQo+IE90aGVyIGRyaXZlcnMgbmVlZCBpdCB0b28gZnJvbSB3aGF0SSBjb3Vs ZCBzZWU/DQo+IA0KPiBXZSByZWFsbHkgZG9uJ3QgbGlrZSBtb2R1bGUgcGFyYW1ldGVycyBpbiB0 aGUga2VybmVsLg0KDQpIYXMgaXQgYmVlbiBjb25zaWRlcmVkIHRvIHNwZWNpZnkgd2hpY2ggY2hl Y2tzdW0gdHlwZSB0byB1c2UgdGhyb3VnaCBpc2NzaWFkbQ0KdG8gdGhlIGlTRVIgaW5pdGlhdG9y IGRyaXZlcj8gSSB0aGluayB0aGF0IHdvdWxkIGF2b2lkIHRoYXQgd2UgaGF2ZSB0byByZXN1cnJl Y3QNCnRoZSBrZXJuZWwgbW9kdWxlIHBhcmFtZXRlci4NCg0KVGhhbmtzLA0KDQpCYXJ0Lg== --
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#9 lore raw
In reply to a message not in the archive

Hi Steven and Hannes,

Quoted 5 lines
Quoted 5 lines · Hannes Reinecke
Quoted 3 lines

The module parameter 'iser_pi_guard' has been disabled by commit 5bb6e543d2a7d58 ("IB/iser: DIX update"), but the functionality to select the guard algorithm is still required.

Quoted 8 lines
Quoted 5 lines

Commit should explain why it is still required? What is the actual bug here?

Someone who understands this is going to have to Ack it for it to go through the rdma tree..

Without the module parameter, there is no way to actually use the CRC guard format. Currently, iscsi_iser_session_create() indicates support for both IP and CRC formats, but sd_dif_config_host() always checks for IP support first, so CRC guard won't be used.

Isn't a bit backwards that each individual driver needs this knob to modify the block layer behavior? I think a better approach would be to get rid of the drivers modparams and simply add a block sysfs knob that would take the knob guard if supported...

Thoughts?

CCing Martin...
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#11 lore raw
In reply to a message not in the archive

Sagi,

Quoted 4 lines

Isn't a bit backwards that each individual driver needs this knob to modify the block layer behavior? I think a better approach would be to get rid of the drivers modparams and simply add a block sysfs knob that would take the knob guard if supported...

Originally the IP checksum thing was an optimization for a single device. But others adopted it as well so it grew from being a driver tweak to a common feature.

I don't have a problem adding a way to toggle it at the block layer. But it would have to be an additional knob. We can't nuke the module parameters without breaking a ton of stuff...

Signature · 5 lines
Martin K. Petersen	Oracle Linux Engineering
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#13 lore raw
In reply to a message not in the archive
Quoted 8 lines · Martin K. Petersen

Sagi,

Quoted 4 lines

Isn't a bit backwards that each individual driver needs this knob to modify the block layer behavior? I think a better approach would be to get rid of the drivers modparams and simply add a block sysfs knob that would take the knob guard if supported...

Quoted 8 lines

Originally the IP checksum thing was an optimization for a single device. But others adopted it as well so it grew from being a driver tweak to a common feature.

I don't have a problem adding a way to toggle it at the block layer. But it would have to be an additional knob. We can't nuke the module parameters without breaking a ton of stuff...

?

So what now?
Do we need to keep the parameter?
If so we really should be re-enable it; ATM it's just a no-op leading the user to believe something has actually happened.

I'm fine with adding a knob in sysfs to enable things, but I'm a bit puzzled now what will happen with this parameter...

Cheers,

Hannes

Signature · 9 lines
Dr. Hannes Reinecke		               zSeries & Storage
hare-IBi9RG/[email protected]			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#15 lore raw
In reply to a message not in the archive

Hannes,

Quoted 3 lines

Do we need to keep the parameter?
If so we really should be re-enable it; ATM it's just a no-op leading the user to believe something has actually happened.

The driver module parameters existed mainly for the purpose of hw testing. One could disable IP checksum and revert to CRC to ensure both code paths were working properly. But it wasn't really meant to be something ordinary users would ever muck with. Why would anybody use the much slower CRC when IP checksum was supported by the hardware? (*)

The reason the kernel interface changed from a per-driver to a per-I/O flag for checksum selection was that there are a few things you can't express when checksum conversion is taking place. For instance, say you're writing a RAID parity disk. The 8 additional bytes on the parity disk are not valid T10 PI but rather the XOR of the PI across the stripe. So you need to be able to disable checking and conversion on a per-I/O basis when accessing the parity disk. Deliberately writing "bad" PI for test purposes is another example where conversion must be disabled.

Anyway. I think Sagi's iser_pi_guard patch was a result of that transition from driver global checksum setting to per-I/O ditto.

I don't have a problem with having a block-level preference knob (slightly convoluted to implement since they are different integrity profiles). But I do question whether it is worth the hassle. What exactly is your use case that requires twiddling this?

(*) The modern PCLMULQDQ-optimized CRC calculation blurs that picture slightly.

Signature · 5 lines
Martin K. Petersen	Oracle Linux Engineering
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#22 lore raw
In reply to a message not in the archive
Quoted 7 lines

Sagi,

Quoted 4 lines

Isn't a bit backwards that each individual driver needs this knob to modify the block layer behavior? I think a better approach would be to get rid of the drivers modparams and simply add a block sysfs knob that would take the knob guard if supported...

Quoted 6 lines

Originally the IP checksum thing was an optimization for a single device. But others adopted it as well so it grew from being a driver tweak to a common feature.

I don't have a problem adding a way to toggle it at the block layer. But it would have to be an additional knob.

Personally I think that the clean way to have it is that the driver exposes capabilities and the core selects the method instead of having each driver expose what it supports based on the desired method.

But I really don't care that much, if its not something we consider worth doing then I can easily ack this patch.

Quoted 1 line

We can't nuke the module parameters without breaking a ton of stuff...

Correct, but we can gradually deprecate them..
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#17 lore raw
In reply to a message not in the archive
Quoted 11 lines

Hannes,

Quoted 3 lines

Do we need to keep the parameter?
If so we really should be re-enable it; ATM it's just a no-op leading the user to believe something has actually happened.

The driver module parameters existed mainly for the purpose of hw testing. One could disable IP checksum and revert to CRC to ensure both code paths were working properly. But it wasn't really meant to be something ordinary users would ever muck with. Why would anybody use the much slower CRC when IP checksum was supported by the hardware? (*)

Agree, but I definitely might not see the full picture here.

Quoted 2 lines

Anyway. I think Sagi's iser_pi_guard patch was a result of that transition from driver global checksum setting to per-I/O ditto.

Correct, and due to your comment above.

Quoted 4 lines

I don't have a problem with having a block-level preference knob (slightly convoluted to implement since they are different integrity profiles). But I do question whether it is worth the hassle. What exactly is your use case that requires twiddling this?

I'm not sure its worth it either, I can easily ack this one if that's the case.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#19 lore raw
In reply to a message not in the archive

PiBGcm9tOiBTYWdpIEdyaW1iZXJnIFttYWlsdG86c2FnaUBncmltYmVyZy5tZV0NCj4gU2VudDog V2VkbmVzZGF5LCBKYW51YXJ5IDE3LCAyMDE4IDM6NTkgQU0NCj4gPiBIYW5uZXMsDQo+ID4NCj4g Pj4gRG8gd2UgbmVlZCB0byBrZWVwIHRoZSBwYXJhbWV0ZXI/DQo+ID4+IElmIHNvIHdlIHJlYWxs eSBzaG91bGQgYmUgcmUtZW5hYmxlIGl0OyBBVE0gaXQncyBqdXN0IGEgbm8tb3AgbGVhZGluZw0K PiA+PiB0aGUgdXNlciB0byBiZWxpZXZlIHNvbWV0aGluZyBoYXMgYWN0dWFsbHkgaGFwcGVuZWQu DQo+ID4NCj4gPiBUaGUgZHJpdmVyIG1vZHVsZSBwYXJhbWV0ZXJzIGV4aXN0ZWQgbWFpbmx5IGZv ciB0aGUgcHVycG9zZSBvZiBodw0KPiA+IHRlc3RpbmcuIE9uZSBjb3VsZCBkaXNhYmxlIElQIGNo ZWNrc3VtIGFuZCByZXZlcnQgdG8gQ1JDIHRvIGVuc3VyZSBib3RoDQo+ID4gY29kZSBwYXRocyB3 ZXJlIHdvcmtpbmcgcHJvcGVybHkuIEJ1dCBpdCB3YXNuJ3QgcmVhbGx5IG1lYW50IHRvIGJlDQo+ ID4gc29tZXRoaW5nIG9yZGluYXJ5IHVzZXJzIHdvdWxkIGV2ZXIgbXVjayB3aXRoLiBXaHkgd291 bGQgYW55Ym9keSB1c2UgdGhlDQo+ID4gbXVjaCBzbG93ZXIgQ1JDIHdoZW4gSVAgY2hlY2tzdW0g d2FzIHN1cHBvcnRlZCBieSB0aGUgaGFyZHdhcmU/ICgqKQ0KPiANCj4gQWdyZWUsIGJ1dCBJIGRl ZmluaXRlbHkgbWlnaHQgbm90IHNlZSB0aGUgZnVsbCBwaWN0dXJlIGhlcmUuDQpJZiB0aGUgdGFy Z2V0IHVzZXMgQ1JDLCB0aGUgaG9zdCB3aWxsIG5lZWQgYSB3YXkgdG8gbWF0Y2guDQpBRkFJSywg dGhlcmUgaXMgbm8gd2F5IGluIHRoZSBTQ1NJIHByb3RvY29sIHRvIGRldGVybWluZSB0aGlzLCBz byBhZG1pbg0KaGFzIHRvIHNldCBpdCB1cC4NCg0KPiANCj4gPiBBbnl3YXkuIEkgdGhpbmsgU2Fn aSdzIGlzZXJfcGlfZ3VhcmQgcGF0Y2ggd2FzIGEgcmVzdWx0IG9mIHRoYXQNCj4gPiB0cmFuc2l0 aW9uIGZyb20gZHJpdmVyIGdsb2JhbCBjaGVja3N1bSBzZXR0aW5nIHRvIHBlci1JL08gZGl0dG8u DQo+IA0KPiBDb3JyZWN0LCBhbmQgZHVlIHRvIHlvdXIgY29tbWVudCBhYm92ZS4NCj4gDQo+ID4g SSBkb24ndCBoYXZlIGEgcHJvYmxlbSB3aXRoIGhhdmluZyBhIGJsb2NrLWxldmVsIHByZWZlcmVu Y2Uga25vYg0KPiA+IChzbGlnaHRseSBjb252b2x1dGVkIHRvIGltcGxlbWVudCBzaW5jZSB0aGV5 IGFyZSBkaWZmZXJlbnQgaW50ZWdyaXR5DQo+ID4gcHJvZmlsZXMpLiBCdXQgSSBkbyBxdWVzdGlv biB3aGV0aGVyIGl0IGlzIHdvcnRoIHRoZSBoYXNzbGUuIFdoYXQNCj4gPiBleGFjdGx5IGlzIHlv dXIgdXNlIGNhc2UgdGhhdCByZXF1aXJlcyB0d2lkZGxpbmcgdGhpcz8NCkEgdGFyZ2V0IHRoYXQg Y2FuIGRvIENSQyBHdWFyZCBjaGVja3MgaW50ZXJuYWxseSwgYnV0IGRvZXNuJ3QgdXNlIElQDQpj aGVja3N1bS4NCg0KPiANCj4gSSdtIG5vdCBzdXJlIGl0cyB3b3J0aCBpdCBlaXRoZXIsIEkgY2Fu IGVhc2lseSBhY2sgdGhpcyBvbmUgaWYgdGhhdCdzDQo+IHRoZSBjYXNlLg0KVGhpcyBwYXRjaCB3 b3VsZCByZXN0b3JlIHRoZSBwcmV2aW91cyBrbm9iIHdoaWNoIG1lZXRzIHRoZSBuZWVkDQpmb3Ig bm93LiBJZiBhIGJldHRlciBtZXRob2QgaXMgZGV2ZWxvcGVkIGluIHRoZSBmdXR1cmUsIHRoZW4g Z3JlYXQuDQoNClRoYW5rcywNClN0ZXZlDQo=
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#21 lore raw
In reply to a message not in the archive

Steven,

Quoted 2 lines

If the target uses CRC, the host will need a way to match. AFAIK, there is no way in the SCSI protocol to determine this,

The IP checksum feature is entirely DIX, so no.

There is no guard format feature discovery taking place since only the initiator supports IP checksum. It is always T10 CRC on the wire between initiator and target on SAS/FC.

Quoted 2 lines

This patch would restore the previous knob which meets the need for now. If a better method is developed in the future, then great.

I'm not against moving the checksum selection up the stack in principle. But it's a lot of churn for little gain, IMHO. So I think I'm in favor of reviving the iser_pi_guard parameter. At least in the short term.

Signature · 5 lines
Martin K. Petersen	Oracle Linux Engineering
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html