Skip to contentSkip to conversation

ip_queue & QUEUE target

DISCUSSION Nigel Kukard 3 participants 8 messages Jun 15, 2003 netfilter

lore
Share & tools
lore
#0 lore raw

Hi all,

I have a very interesting problem, I have all packets passing through my firewall being thrown to userspace with -j QUEUE, I them pick certain packets and generate statistics.

My problem comes when I do a ping -f, at random times will my small program just not receive any packets. I'm using ipq_read with a timeout of 250000ms. I also get "Timeout" error messages when using ping -f.

I've tried to re-open the IPQ handle each time it fails, to no avail.

does anyone know how i can solve this problem?

Regars

Nigel Kukard

#1 lore raw

Hi Nigel,

Quoted 6 lines · Nigel Kukard

Hi all,

I have a very interesting problem, I have all packets passing through my firewall being thrown to userspace with -j QUEUE, I them pick certain packets and generate statistics.

Quoted 10 lines

My problem comes when I do a ping -f, at random times will my small program just not receive any packets. I'm using ipq_read with a timeout of 250000ms. I also get "Timeout" error messages when using ping -f.

I've tried to re-open the IPQ handle each time it fails, to no avail.

does anyone know how i can solve this problem?

netlink sockets provide unreliable communication. your problem is probably the sockets receive queue overflowing. ip_queue then returns a negative error code to nf_queue which drops the packet.

Hope that helps,
Patrick

Quoted 6 lines

Regars

Nigel Kukard

#2 lore raw
Quoted 5 lines

netlink sockets provide unreliable communication. your problem is probably the sockets receive queue overflowing. ip_queue then returns a negative error code to nf_queue which drops the packet.

well i'm pumping through about 5,000 pkt/s, doing a few Mb/s. What would the best thing be to use? or how could i remedy this problem?

i set the rcv buffer to a few Mb, and also the ipq_maxlen to 8196... is there anything else i can do?

Regards
Nigel

#3 lore raw
Quoted 6 lines · Nigel Kukard
Quoted 5 lines

netlink sockets provide unreliable communication. your problem is probably the sockets receive queue overflowing. ip_queue then returns a negative error code to nf_queue which drops the packet.

Quoted 8 lines

well i'm pumping through about 5,000 pkt/s, doing a few Mb/s. What would the best thing be to use? or how could i remedy this problem?

i set the rcv buffer to a few Mb, and also the ipq_maxlen to 8196... is there anything else i can do?

Probably not much, but did you make sure you also increased /proc/sys/net/core/rmem_max ? If you don't mind your statistics beeing wrong (better than high drop rates imho) you could modify ip_queue so it nf_reinjects packets which could not be sent instead of returning a error to nf_queue.

Bye
Patrick

Quoted 5 lines

Regards
Nigel

#4 lore raw
Quoted 5 lines

Probably not much, but did you make sure you also increased /proc/sys/net/core/rmem_max ? If you don't mind your statistics beeing wrong (better than high drop rates imho) you could modify ip_queue so it nf_reinjects packets which could not be sent instead of returning a error to nf_queue.

This is very strange, i'm sure i'm doing something wrong.

I have rmem_max set to 272144, i use setsockopt to set the options for the socket, namely SO_RCVBUF, this i set to 1048576.

The socket i create with socket(PF_NETLINK,SOCK_RAW,NETLINK_FIREWALL).

(this is basicallly all in libipq, i've not changed more than the rcvbuf).

now the strange thing is, when receiving data, i get IPQ_ERR_TIMEOUT's, this being basically impossible imho because of the high packet rate.

after these timeouts, i get no data at all... data hits the queue, but my small proggie doesn't get any (using a 1/4 timeout), funny enough i don't get IPQ_ERR_TIMEOUTS... *shrug*... any help would be very much appreciated!

-Nigel

#5 lore raw
In reply to Patrick McHardy
Quoted 6 lines

Probably not much, but did you make sure you also increased /proc/sys/net/core/rmem_max ? If you don't mind your statistics beeing wrong (better than high drop rates imho) you could modify ip_queue so it nf_reinjects packets which could not be sent instead of returning a error to nf_queue.

What about DIVERT sockets, would these be reliable?

-Nigel

#6 lore raw
Quoted 1 line · Nigel Kukard

What about DIVERT sockets, would these be reliable?

If you are only using the packets to generate statistics I'd say ULOG is what you want. It's quite cpu efficient. It batches packets together before sending them to userspace, and copies only as much of the packets as you specify.

Signature · 1 line
/Martin
#7 lore raw
In reply to Nigel Kukard
Quoted 4 lines · Nigel Kukard

What about DIVERT sockets, would these be reliable?

Sorry, i'm not familiar with divert sockets nor libipq. One thing i forgot
before: do you send whole packets to userspace or only the metadata ?

Bye,
Patrick