Skip to contentSkip to conversation

libibmad: Distinguish timed out from other errors

PATCH Applied Hal Rosenstock 3 participants 9 messages Apr 9, 2011 – Apr 11, 2011 8 messages not in the archive InfiniBand

v1 of 2 Applied Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/4 libibmad: Distinguish timed out from other errors

Add error field like errno to ib_rpc_t structure
In mad_rpc code, make sure error is cleared and
in _do_madrpc, set ETIMEDOUT.

error field could also be used to distinguish other errors if needed.

---

include/infiniband/mad.h +1 -0
index 5d18ec3..0ccdd36 100644
@@ -233,6 +233,7 @@ typedef struct {
 	unsigned recsz;		/* for sa mads (attribute offset) */
 	int timeout;
 	uint32_t oui;		/* for vendor range 2 mads */
+	int error;		/* errno */
 } ib_rpc_t;
 
 typedef struct portid {
src/rpc.c +7 -3
index a702046..557a33b 100644
@@ -127,7 +127,7 @@ int mad_rpc_class_agent(struct ibmad_port *port, int class)
 
 static int
 _do_madrpc(int port_id, void *sndbuf, void *rcvbuf, int agentid, int len,
-	   int timeout, int max_retries)
+	   int timeout, int max_retries, int *p_error)
 {
 	uint32_t trid;		/* only low 32 bits */
 	int retries;
@@ -182,6 +182,8 @@ _do_madrpc(int port_id, void *sndbuf, void *rcvbuf, int agentid, int len,
 			return length;
 	}
 
+	if (p_error)
+		*p_error = ETIMEDOUT;
 	ERRS("timeout after %d retries, %d ms", retries, timeout * retries);
 	return -1;
 }
@@ -214,6 +216,7 @@ void *mad_rpc(const struct ibmad_port *port, ib_rpc_t * rpc,
 	uint8_t sndbuf[1024], rcvbuf[1024], *mad;
 	int redirect = 1;
 
+	rpc->error = 0;
 	while (redirect) {
 		len = 0;
 		memset(sndbuf, 0, umad_size() + IB_MAD_SIZE);
@@ -224,7 +227,7 @@ void *mad_rpc(const struct ibmad_port *port, ib_rpc_t * rpc,
 		if ((len = _do_madrpc(port->port_id, sndbuf, rcvbuf,
 				      port->class_agents[rpc->mgtclass],
 				      len, mad_get_timeout(port, rpc->timeout),
-				      mad_get_retries(port))) < 0) {
+				      mad_get_retries(port), &rpc->error)) < 0) {
 			IBWARN("_do_madrpc failed; dport (%s)",
 			       portid2str(dport));
 			return NULL;
@@ -273,13 +276,14 @@ void *mad_rpc_rmpp(const struct ibmad_port *port, ib_rpc_t * rpc,
 
 	DEBUG("rmpp %p data %p", rmpp, data);
 
+	rpc->error = 0;
 	if ((len = mad_build_pkt(sndbuf, rpc, dport, rmpp, data)) < 0)
 		return NULL;
 
 	if ((len = _do_madrpc(port->port_id, sndbuf, rcvbuf,
 			      port->class_agents[rpc->mgtclass],
 			      len, mad_get_timeout(port, rpc->timeout),
-			      mad_get_retries(port))) < 0) {
+			      mad_get_retries(port), &rpc->error)) < 0) {
 		IBWARN("_do_madrpc failed; dport (%s)", portid2str(dport));
 		return NULL;
 	}

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 4DA07983.8060308-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org

#2 lore raw
Quoted 6 lines · Hal Rosenstock

Add error field like errno to ib_rpc_t structure
In mad_rpc code, make sure error is cleared and
in _do_madrpc, set ETIMEDOUT.

error field could also be used to distinguish other errors if needed.

This looks like it breaks the ABI since callers using functions that accept ib_rpc_t are expected to allocate the ib_rpc_t struct themselves.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20110409171239.GA4372-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org

#4 lore raw
Quoted 11 lines · Jason Gunthorpe
Quoted 6 lines · Hal Rosenstock

Add error field like errno to ib_rpc_t structure
In mad_rpc code, make sure error is cleared and
in _do_madrpc, set ETIMEDOUT.

error field could also be used to distinguish other errors if needed.

This looks like it breaks the ABI since callers using functions that accept ib_rpc_t are expected to allocate the ib_rpc_t struct themselves.

Yes, this changes the ABI. I didn't see a good way not to do this. Do you have any ideas ? The only possible approach I saw was to steal the high order bit from mgtclass (class is really only a byte) to indicate which version of ib_rpc_t is being passed as to whether this field is present.

-- Hal

Quoted 2 lines

Jason

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 4DA1875F.2050601-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org

#6 lore raw
Quoted 13 lines · Hal Rosenstock
Quoted 11 lines · Jason Gunthorpe
Quoted 6 lines · Hal Rosenstock

Add error field like errno to ib_rpc_t structure
In mad_rpc code, make sure error is cleared and
in _do_madrpc, set ETIMEDOUT.

error field could also be used to distinguish other errors if needed.

This looks like it breaks the ABI since callers using functions that accept ib_rpc_t are expected to allocate the ib_rpc_t struct themselves.

Quoted 5 lines

Yes, this changes the ABI. I didn't see a good way not to do this. Do you have any ideas ? The only possible approach I saw was to steal the high order bit from mgtclass (class is really only a byte) to indicate which version of ib_rpc_t is being passed as to whether this field is present.

If you are going to change the ABI like this then you have to bump the SONAME.

Setting a flag would work.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20110410140956.GB4372-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org

#8 lore raw
Quoted 14 lines · Jason Gunthorpe
Quoted 13 lines · Hal Rosenstock
Quoted 11 lines · Jason Gunthorpe
Quoted 6 lines · Hal Rosenstock

Add error field like errno to ib_rpc_t structure
In mad_rpc code, make sure error is cleared and
in _do_madrpc, set ETIMEDOUT.

error field could also be used to distinguish other errors if needed.

This looks like it breaks the ABI since callers using functions that accept ib_rpc_t are expected to allocate the ib_rpc_t struct themselves.

Quoted 8 lines
Quoted 5 lines

Yes, this changes the ABI. I didn't see a good way not to do this. Do you have any ideas ? The only possible approach I saw was to steal the high order bit from mgtclass (class is really only a byte) to indicate which version of ib_rpc_t is being passed as to whether this field is present.

If you are going to change the ABI like this then you have to bump the SONAME.

Typically it's been the libibmad maintainer who's bumped the versions when appropriate as a release is readied.

Quoted 1 line

Setting a flag would work.

Not sure what you're referring to. Setting a flag where ?

-- Hal

Quoted 1 line

Jason

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 4DA1D405.4040305-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org

#10 lore raw
Quoted 5 lines · Hal Rosenstock
Quoted 2 lines

If you are going to change the ABI like this then you have to bump the SONAME.

Typically it's been the libibmad maintainer who's bumped the versions when appropriate as a release is readied.

The version should be bumped in the same commit that makes the incompatible change so that people testing the library don't get messed up by it.

Quoted 3 lines
Quoted 1 line

Setting a flag would work.

Not sure what you're referring to. Setting a flag where ?

Your idea to place a new ABI flag in mgmClass.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20110410164459.GC4372-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org

#12 lore raw
Quoted 10 lines · Jason Gunthorpe
Quoted 5 lines · Hal Rosenstock
Quoted 2 lines

If you are going to change the ABI like this then you have to bump the SONAME.

Typically it's been the libibmad maintainer who's bumped the versions when appropriate as a release is readied.

The version should be bumped in the same commit that makes the incompatible change so that people testing the library don't get messed up by it.

That's how it used to be done but with the previous maintainer version bumps were left until release time.

I'll add this to an updated/reworked patch but I hope we'll avoid an extra last minute/release version bump.

-- Hal

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Setting a flag would work.

Not sure what you're referring to. Setting a flag where ?

Your idea to place a new ABI flag in mgmClass.

Jason

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 4DA2E511.2030204-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org

#14 lore raw
Quoted 2 lines · Hal Rosenstock

I'll add this to an updated/reworked patch but I hope we'll avoid an extra last minute/release version bump.

IMHO you only bump it once during devel, even if two ABI bumping changes are done. The point is to not mix it up with the already installed library.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20110411155853.GA30452-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org

#16 lore raw
Quoted 4 lines · On Mon, 11 Apr 2011 08:58:53 -0700 Jason Gunthorpe…
Quoted 2 lines · Hal Rosenstock

I'll add this to an updated/reworked patch but I hope we'll avoid an extra last minute/release version bump.

In the end when an official version comes out it is on me to make sure we don't rev twice.

Quoted 4 lines

IMHO you only bump it once during devel, even if two ABI bumping changes are done. The point is to not mix it up with the already installed library.

This makes sense to me, and is how I have done things in the past. Bump the version and run with that until the official branch/tag.

Ira

Quoted 2 lines

Jason

Signature · 9 lines
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
[email protected]
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html