Skip to contentSkip to conversation

QoS SL=>VL issue in user mode librdmacm‏

DISCUSSION Prasanna Vv 1 participant 2 messages Nov 25, 2011 1 message not in the archive InfiniBand

lore
Share & tools
lore
#0 lore raw

Hi
I configure QoS in user mode using librdmacm call:
rdma_set_option(cm_id, RDMA_OPTION_ID,RDMA_OPTION_ID_TOS, &tos, sizeof _tos)

After the connection is established, i query ibv_query_qp (and also cm_id->route.path_rec->sl) to check SL, which is correct as per the qos choice from qos_policy.config settings.

But the data transfer is not happening in correct VL as per SL=>VL.

For example if SL is 3 the data transfer VL should be 3 (based on SL2VLmaptable). Instead data is always transfered on VL:0, irrepective of SL value.

Interestingly, the same thing is working properly in kernel mode when i set tos using the call:
rdma_set_service_type(cm_id, conn->c_tos);
If i use kernel mode rdma cm, SL=>VL is correct, data transfer post send takes place on VL based on SL.

Do you have clue why the user mode libibverb is not tranfering on VL based on SL.

Thanks,
Prasanna

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#2 lore raw
In reply to a message not in the archive

Hi
I configure QoS in user mode using librdmacm call:
rdma_set_option(cm_id, RDMA_OPTION_ID,RDMA_OPTION_ID_TOS, &tos, sizeof _tos)

After the connection is established, i query ibv_query_qp (and also cm_id->route.path_rec->sl) to check SL, which is correct as per the qos choice from qos_policy.config settings.

But the data transfer is not happening in correct VL as per SL=>VL.

For example if SL is 3 the data transfer VL should be 3 (based on SL2VLmaptable). Instead data is always transfered on VL:0, irrepective of SL value.

Interestingly, the same thing is working properly in kernel mode when i set tos using the call:
rdma_set_service_type(cm_id, conn->c_tos);
If i use kernel mode rdma cm, SL=>VL is correct, data transfer post send takes place on VL based on SL.

Do you have clue why the user mode libibverb is not tranfering on VL based on SL.

Thanks,
Prasanna

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html