napi layer and packet throttling
Subsystems (2)
Hi all,
I have a general question to the napi layer. I have a board with a MPC8360 processor which uses the ucc_geth.c driver for it's ethernet device. The kernel version is 3.9. We see in lab tests that the board gets unresponsive if we send a high packet rate to this ethernet device via an packet generator. In the end we would like to limit the amount of incoming packets at a very early stage, to prevent the system of a to high softirq load.
The NAPI layer description I read on:
http://www.linuxfoundation.org/collaborate/workgroups/networking/napi
pointed two main functions out for the napi layer.
a) Interrupt mitigation
Which works perfectly on the ucc_geth.c driver under high load the hard irqs for the RX packets are disabled and we switch to a polling mode. If the load is more relaxed we switch back.
b) Packet-Throttling
Here the description says "NAPI-compliant drivers can often cause packets to be dropped in the network adaptor itself, before the kernel sees them at all."
This is exactly what I need for my usecase. But I don't see any hints how this can be implemented with the napi layer. So what I need would be something like a possibility to accept on the eth0 device in timeframe x only packetcount y. If this packetcount is reached disable hard irq and napi_poll until the timeframe is expired.
So are there any pointers to other ethernet drivers doing something like this? Do I misunderstand the packet throttling purpose of the NAPI layer completely?
Any help is appreciated.
Regards
Holger
Quoted 17 lines · Holger Brunck
Hi all,
I have a general question to the napi layer. I have a board with a MPC8360 processor which uses theucc_geth.cdriver for it's ethernet device. The kernel version is 3.9. We see in lab tests that the board gets unresponsive if we send a high packet rate to this ethernet device via an packet generator. In the end we would like to limit the amount of incoming packets at a very early stage, to prevent the system of a to high softirq load.The NAPI layer description I read on:
http://www.linuxfoundation.org/collaborate/workgroups/networking/napi
pointed two main functions out for the napi layer.a) Interrupt mitigation
Which works perfectly on theucc_geth.cdriver under high load the hard irqs for the RX packets are disabled and we switch to a polling mode. If the load is more relaxed we switch back.
Quoted 6 lines
b) Packet-Throttling
Here the description says "NAPI-compliant drivers can often cause packets to be dropped in the network adaptor itself, before the kernel sees them at all."This is exactly what I need for my usecase. But I don't see any hints how this can be implemented with the napi layer.
[...]
If the RX ring is not cleaned and refilled quickly enough, the network controller will naturally start to drop packets. It's not something you should do explicitly in the driver.
Ben.
Signature · 3 lines
Ben Hutchings, Staff Engineer, Solarflare Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
Hi Ben,
Quoted 5 lines · Ben Hutchings
Quoted 4 lines · Holger Brunck
b) Packet-Throttling
Here the description says "NAPI-compliant drivers can often cause packets to be dropped in the network adaptor itself, before the kernel sees them at all."
Quoted 8 lines
Quoted 2 lines
This is exactly what I need for my usecase. But I don't see any hints how this can be implemented with the napi layer.
[...]
If the RX ring is not cleaned and refilled quickly enough, the network controller will naturally start to drop packets. It's not something you should do explicitly in the driver.
yes. But what if the remaining amount of packets which are getting through the napi_poll function into the linux system are still to many and generate therefore a to high softirq load on the system which leads to the problems I see. Ok I could use a smaller amount of RX ring buffers, but then the system would get more intolerant for RX bursts what I don't want. I would like to protect the system if someone sends continuously a high packet rate to the interface, similar to DoS attacks.
Regards
Holger
Quoted 17 lines · Holger Brunck
Hi Ben,
Quoted 5 lines · Ben Hutchings
Quoted 4 lines · Holger Brunck
b) Packet-Throttling
Here the description says "NAPI-compliant drivers can often cause packets to be dropped in the network adaptor itself, before the kernel sees them at all."Quoted 8 lines
Quoted 2 lines
This is exactly what I need for my usecase. But I don't see any hints how this can be implemented with the napi layer.
[...]
If the RX ring is not cleaned and refilled quickly enough, the network controller will naturally start to drop packets. It's not something you should do explicitly in the driver.
Quoted 7 lines
yes. But what if the remaining amount of packets which are getting through the
napi_pollfunction into the linux system are still to many and generate therefore a to high softirq load on the system which leads to the problems I see. Ok I could use a smaller amount of RX ring buffers, but then the system would get more intolerant for RX bursts what I don't want. I would like to protect the system if someone sends continuously a high packet rate to the interface, similar to DoS attacks.
what if i measure no. of packets over a period of time and find the rate of arrival (packets/msec) and if that rate is greater than the choosen rate, allow only packets that are in the window that you can accept, This window can be time window or the number of packets (like 1 packet in 10 usec or 1 packet out of 100 packets). Like wise this measurement can be repeated after every full period.
Quoted 8 lines
Regards
Holger--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 26 lines · devendra.aaru
On Fri, May 24, 2013 at 12:58 PM, Holger Brunck
Quoted 16 lines · Ben Hutchings
Quoted 5 lines · Holger Brunck
Quoted 4 lines
b) Packet-Throttling
Here the description says "NAPI-compliant drivers can often cause packets to be dropped in the network adaptor itself, before the kernel sees them at all."Quoted 8 lines
Quoted 2 lines
This is exactly what I need for my usecase. But I don't see any hints how this can be implemented with the napi layer.
[...]
If the RX ring is not cleaned and refilled quickly enough, the network controller will naturally start to drop packets. It's not something you should do explicitly in the driver.
Quoted 7 lines
yes. But what if the remaining amount of packets which are getting through the
napi_pollfunction into the linux system are still to many and generate therefore a to high softirq load on the system which leads to the problems I see. Ok I could use a smaller amount of RX ring buffers, but then the system would get more intolerant for RX bursts what I don't want. I would like to protect the system if someone sends continuously a high packet rate to the interface, similar to DoS attacks.
Quoted 7 lines
what if i measure no. of packets over a period of time and find the rate of arrival (packets/msec) and if that rate is greater than the choosen rate, allow only packets that are in the window that you can accept, This window can be time window or the number of packets (like 1 packet in 10 usec or 1 packet out of 100 packets). Like wise this measurement can be repeated after every full period.
thats similar to what I have implemented in ucc_geth.c for my workspace and yes whith such a mechanism my problems seems to be fixed. But I still ask myself if there is a common approach for such a problem already implemented in similar drivers...
Regards
Holger
Quoted 17 lines · Holger Brunck
Hi Ben,
Quoted 5 lines · Ben Hutchings
Quoted 4 lines · Holger Brunck
b) Packet-Throttling
Here the description says "NAPI-compliant drivers can often cause packets to be dropped in the network adaptor itself, before the kernel sees them at all."Quoted 8 lines
Quoted 2 lines
This is exactly what I need for my usecase. But I don't see any hints how this can be implemented with the napi layer.
[...]
If the RX ring is not cleaned and refilled quickly enough, the network controller will naturally start to drop packets. It's not something you should do explicitly in the driver.
Quoted 4 lines
yes. But what if the remaining amount of packets which are getting through the
napi_pollfunction into the linux system are still to many and generate therefore a to high softirq load on the system which leads to the problems I see.
Soft IRQs are then handled in a ksoftirqd thread, which is scheduled like any other task (though it has high priority).
Quoted 4 lines
Ok I could use a smaller amount of RX ring buffers, but then the system would get more intolerant for RX bursts what I don't want. I would like to protect the system if someone sends continuously a high packet rate to the interface, similar to DoS attacks.
This is not the job of the driver (beyond using NAPI in the first place).
Ben.
Signature · 3 lines
Ben Hutchings, Staff Engineer, Solarflare Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.