pktgen terminating condition
Subsystems (2)
A message that is not in the archive (sent off-list or not yet received): 1188347327.4305.27.camel@localhost
A message that is not in the archive (sent off-list or not yet received): 20070829044352.GA4133@ludhiana
Quoted 3 lines · Mandeep Singh Baines
I interpret this to mean that the interrupt gets generates after a packet is transferred to the TFIFO on the NIC and the next packet in the ring is NULL.
iow, when tx transits to idle ..
Quoted 4 lines
This interrupt gets generates less often then TxOK which gets generated after every completed packet transmit. So I'm thinking that maybe the driver was written to use this interrupt instead of TxOK for this reason. Really just my speculation.
It does make sense if you are trying to cut down on tx interupts. It's a little extreme - and if you dont have much memory it may not be the best scheme. OTOH, you have moved to the other extreme imo ;-> You are generating an interupt per tx packet. This may not matter on modern hardware because that NIC seems to be Fast Ethernet.
You may wanna heed Dave's advice and just always set the idle on a per-descriptor basis as well as txcomplete on every Xth packet (4 was suggested).
Looking at sis900_start_xmit() at the spot where OWN is set on the descriptor, theres possibly another bit in tx_ring[entry].cmdsts you may could set that will ask for tx complete; which makes me wonder:
is that "outl(TxENA | inl(ioaddr + cr), ioaddr + cr)" really needed on a per-packet basis? Should it not be sufficient to turn it on once at open()?
Quoted 2 lines
Anyway, if I rewrite the driver to use TxOK instead of TxIdle, pktgen works fine.
I think its a good thing pktgen caught this; i am unsure however if it is doing the right thing. Hoping Robert would respond.
One thing pktgen could do is restrict the amount of outstanding buffers by using accounting the way sockets do - this at least wont stop progress as it did in your case.
Quoted 1 line
I'm attaching the patch.
Looks good to me given the desire. I would bounce it by whoever the maintainer is - they may have some insights on the lazy tx prune habit.
cheers,
jamal
Quoted 7 lines · jamal
On Tue, 2007-28-08 at 21:43 -0700, Mandeep Singh Baines wrote:
I think its a good thing pktgen caught this; i am unsure however if it is doing the right thing. Hoping Robert would respond.
One thing pktgen could do is restrict the amount of outstanding buffers by using accounting the way sockets do - this at least wont stop progress as it did in your case.
Hello,
Yes it's synchronization issue... the test is over and we have sent all pkts to the device but pktgen cannot free the skb for it still has refcounts.
IMO must drivers have provisions to handle situation like. I'll guess we can discuss last-resort timer if it should be us, ms or possibly seconds but shouldn't need ping to make this happen. If so we probably have a ICMP packet sitting there waiting instead.
Cheers
--ro
Quoted 3 lines · Robert Olsson
Yes it's synchronization issue... the test is over and we have sent all pkts to the device but pktgen cannot free the skb for it still has refcounts.
Ok, right.
I was confusing it with another issue where pktgen could send a lot of packets without waiting for them to be freed; there are some drivers (10G) which may hold onto 8K skbs. A gazillion ooms start spewing ;-> My thinking in resolving that was to do something like waht sockets do and charge pktgen so it doesnt have too many outstanding packets in flight.
Quoted 1 line
IMO must drivers have provisions to handle situation like.
Mandeep was saying he found less than a handful that didnt conform.
Quoted 4 lines
I'll
guess we can discuss last-resort timer if it should be us, ms or possibly seconds but shouldn't need ping to make this happen. If so we probably have a ICMP packet sitting there waiting instead.
I think as long as it doesnt affect throughput calculation (it just adds to idle time) its fine.
cheers,
jamal
David Miller
maintainerFrom: jamal <[email protected]>
Date: Thu, 30 Aug 2007 08:08:40 -0400
Quoted 5 lines
I was confusing it with another issue where pktgen could send a lot of packets without waiting for them to be freed; there are some drivers (10G) which may hold onto 8K skbs. A gazillion ooms start spewing ;-> My thinking in resolving that was to do something like waht sockets do and charge pktgen so it doesnt have too many outstanding packets in flight.
You could implement this quite simply using skb->destructor.
It will add some atomics, so on weaker pktgen source systems it might decrease the generators rate.
Quoted 1 line · David Miller
You could implement this quite simply using
skb->destructor.
Thats what i was thinking ..
Quoted 2 lines
It will add some atomics, so on weaker pktgen source systems it might decrease the generators rate.
Indeed. So maybe a config option instead; it has value afaics in the batching case only. Need to experiment.
cheers,
jamal
Quoted 13 lines · jamal
Quoted 3 lines · Mandeep Singh Baines
I interpret this to mean that the interrupt gets generates after a packet is transferred to the TFIFO on the NIC and the next packet in the ring is NULL.
iow, when tx transits to idle ..
Quoted 4 lines
This interrupt gets generates less often then TxOK which gets generated after every completed packet transmit. So I'm thinking that maybe the driver was written to use this interrupt instead of TxOK for this reason. Really just my speculation.
Quoted 8 lines
It does make sense if you are trying to cut down on tx interupts. It's a little extreme - and if you dont have much memory it may not be the best scheme. OTOH, you have moved to the other extreme imo ;-> You are generating an interupt per tx packet. This may not matter on modern hardware because that NIC seems to be Fast Ethernet.
You may wanna heed Dave's advice and just always set the idle on a per-descriptor basis as well as txcomplete on every Xth packet (4 was suggested).
My hardware at home is not so modern:( I'm running a AMD Geode NX1750. I like to punish myself:) Actually, it does the job and is very power efficient.
Agreed, interrupting every packet is inefficient. A better solution is probably NAPI. I'll work on a new patch.
Quoted 7 lines
Looking at
sis900_start_xmit()at the spot where OWN is set on the descriptor, theres possibly another bit in tx_ring[entry].cmdsts you may could set that will ask for tx complete; which makes me wonder:
is that "outl(TxENA | inl(ioaddr + cr), ioaddr + cr)" really needed on a per-packet basis? Should it not be sufficient to turn it on once atopen()?
You have to set this bit to restart the Tx State Machine. You could optimize this if you use the TxIdle interrupt to tell you when you really need to set this bit.
Quoted 9 lines
Quoted 2 lines
Anyway, if I rewrite the driver to use TxOK instead of TxIdle, pktgen works fine.
I think its a good thing pktgen caught this; i am unsure however if it is doing the right thing. Hoping Robert would respond.
One thing pktgen could do is restrict the amount of outstanding buffers by using accounting the way sockets do - this at least wont stop progress as it did in your case.
Quoted 5 lines
Quoted 1 line
I'm attaching the patch.
Looks good to me given the desire. I would bounce it by whoever the maintainer is - they may have some insights on the lazy tx prune habit.
+ [email protected]
+ [email protected]
I'll work on a NAPI patch.
Regards,
Mandeep
Quoted 1 line · Mandeep Baines
I'll work on a NAPI patch.
It's a GoodThing - go for it.
cheers,
jamal
----- Message d'origine -----
De: "Mandeep Baines" <[email protected]>
Date: Wed, 29 Aug 2007 09:59:42 -0700
Sujet: Re: pktgen terminating condition
Quoted 8 lines
Quoted 3 lines
Looks good to me given the desire. I would bounce it by whoever the maintainer is - they may have some insights on the lazy tx prune habit.
+ [email protected]
+ [email protected]I'll work on a NAPI patch.
Here I'm (the maintainer). Sorry for the delay replying.
I don't regard the TxOK solution as something usable for mainline, but it has its use for the users of pktgen.
NAPI is the way to go.
Thanks, bye.
--
Daniele Venzano
[email protected]
Quoted 2 lines · Daniele Venzano
I don't regard the TxOK solution as something usable for mainline, but it has its use for the users of pktgen.
I dont know if you followed the discussion - by defering the freeing of skbs, you will be slowing down socket apps sending from the local machine. It may be ok if the socket buffers were huge, but that comes at the cost of system memory (which may not be a big deal)
Do you by any chance recall why you used the idle interupt instead of txok to kick the prunning of tx descriptors?
cheers,
jamal
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
On 8/29/07, Mandeep Baines <[email protected]> wrote:
...
Quoted 3 lines
Unfortunately, the TxIdle interrupt would not free the last odd packet. However, if we want to quiet interrupts couldn't the driver just be converted NAPI. If this seems reasonable I could work on a patch.
It seems reasonable and that's what I was thinking when you submitted the patch to use TXOk.
I think NAPI could be used with either TxOK or TxIdle interrupts. For TxIdle to work, the driver would have to recognize one more packet is still in flight and poll for completion of that packet - then re-enable interrupts and switch back to interrupt mode of operation. I just don't know if netperf TCP_RR perf numbers will suffer (assuming TCP_RR is more important than CPU utilization/efficiency on heavy TX loads like pktgen.)
grant