How to monitor traffic with IBoE/RoCEE?
With IBoE/RoCEE, the traditional SM in IB clusters is not needed. Most of the current IB tools rely on the SM and PM to get packet and error statistics and so on. These won't be applicable with IBoE/RoCEE. netstat will have no value since the kernel has been bypassed. So, how does one monitor traffic in such a cluster?
The possibilities that I can think of are to get information from the switches or if there are some special tools to get information from the adapter ports themselves. Are such tools available with ConnectX adapters? How would one get cluster-wide traffic information when such a cluster is deployed?
Thanks
Pradeep
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Pradeep wrote,
Quoted 4 lines
With IBoE/RoCEE, the traditional SM in IB clusters is not needed. Most of the current IB tools rely on the SM and PM to get packet and error statistics and so on. These won't be applicable with IBoE/RoCEE. netstat will have no value since the kernel has been bypassed. So, how does one monitor traffic in such a cluster?
Some managed Ethernet switches have the ability to log into them and get information on packets transmited/received on each port and errors that are detected. However, these are typically proprietary. I am not sure about the Mellanox card.
So I guess the answer is that you manage RoCCE and iWarp clusters just like you would any Ethernet cluster, with existing management tools that are available for managing Ethernet.
woody
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
A widely-supported method for collecting statistics from ethernet switching devices is SNMP. As Woody said, you wouldn't manage your ethernet network any differently than you do now. In addition many many tools have been written to make use of SNMP counters, so you won't have a hard time finding tools, both commercial and open source.
Tom
Quoted 24 lines · Robert J
Pradeep wrote,
Quoted 5 lines
With IBoE/RoCEE, the traditional SM in IB clusters is not needed. Most of the current IB tools rely on the SM and PM to get packet and error statistics and so on. These won't be applicable with IBoE/RoCEE. netstat will have no value since the kernel has been bypassed. So, how does one monitor traffic in such a cluster?
Some managed Ethernet switches have the ability to log into them and get information on packets transmited/received on each port and errors that are detected. However, these are typically proprietary. I am not sure about the Mellanox card.
So I guess the answer is that you manage RoCCE and iWarp clusters just like you would any Ethernet cluster, with existing management tools that are available for managing Ethernet.
woody
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 14 lines
-------------------------------------------------------------------- Tom Ammon Network Engineer Office: 801.587.0976 Mobile: 801.674.9273 Center for High Performance Computing University of Utah http://www.chpc.utah.edu -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 6 lines · Tom Ammon
A widely-supported method for collecting statistics from ethernet switching devices is SNMP. As Woody said, you wouldn't manage your ethernet network any differently than you do now. In addition many many tools have been written to make use of SNMP counters, so you won't have a hard time finding tools, both commercial and open source.
Quoted 27 lines
Tom
Quoted 24 lines · Robert J
Pradeep wrote,
Quoted 8 lines
With IBoE/RoCEE, the traditional SM in IB clusters is not needed. Most of the current
IB tools rely on the SM and PM to get packet and error statistics and so on. These
won't be applicable with IBoE/RoCEE. netstat will have no value since the kernel
has been bypassed. So, how does one monitor traffic in such a cluster?Some managed Ethernet switches have the ability to log into them and get information
on packets transmited/received on each port and errors that are detected. However,
these are typically proprietary. I am not sure about the Mellanox card.So I guess the answer is that you manage RoCCE and iWarp clusters just like
you would any Ethernet cluster, with existing management tools that are available
for managing Ethernet.woody
True, I suspected that would be the case. However, that means we are using a different management model than with IB clusters. Hence, I was curious if Mellanox had any tools to get say some hardware counters without going to a different management model.
Thanks
Pradeep