libnetfilter_queue issues

PATCH New dorian 4 participants 12 messages Jan 12, 2013 – Jan 14, 2013 netfilter

Share & tools

dorian

author

2013-01-12 12:43 UTC #0 lore raw

Hi all,
Sorry if the queries has been answered yet but I can't find the reliable informations about issues concerned with libnetfilter_queue.

I need to run a program which would firstly fork let's say 10 times (or creates 10 threads) and next each child-process (or thread) will include

   while ((rv = recv(fd, buf, sizeof(buf), 0)) && rv >= 0) {
      nfq_handle_packet(h, buf, rv);
   }

sequence.

The fork/multiple threads reason usage is that for some packets the processing time will be relatively long.
So I would like to handle next packets in "the meantime" using another process (or thread)

But I have 2 doubts which I would like to clarify.

1. - - - - - -
Is it possible to handle concurrently several packets with several processes ?
i.e. if the process1 is handling a packet will the process2 receive the next existing packet?

The essence of my doubt is:
if process1 receives the packet1 but the verdict is not issued will next recv() (in process2) be successful and not hold process2 until verdict for packet1 is made?

Please confirm.

2. - - - - - -
Let's assume that we have process1 which handles packet 1, process 2 which handles packet 2, etc
Let's say verdict for packet2 is issued before verdict for packet1

Will packet2 leave the queue or has it to "wait" for verdict for packet1 ?

The essence of my doubt is:
can packet2 leave the queue before packet1 ?

In other words can we change the packets order leaving system ?

Or, maybe, although verdict for packet2 is done it has to wait for verdict for packet1 and next both leave the queue in original order?

I would be obliged if you clarify above two matter

Best Regards,
Dorian

Felix

2013-01-12 13:23 UTC #1 lore raw

excellent question !
I too am interested in this.

Another similar, related question is do packets that are not filtered wait on a verdict for a filtered packet?
Again, if not, it means we can change packet order?
If so,it means we can degrade system performance unrelated to our filtered packets.

Thanks,
Credzba

Quoted 58 lines · dorian

Hi all,
Sorry if the queries has been answered yet but I can't find the reliable informations about issues concerned with libnetfilter_queue.
I need to run a program which would firstly fork let's say 10 times (or creates 10 threads) and next each child-process (or thread) will include
   while ((rv = recv(fd, buf, sizeof(buf), 0)) && rv >= 0) {
      nfq_handle_packet(h, buf, rv);
   }
sequence.
The fork/multiple threads reason usage is that for some packets the processing time will be relatively long.
So I would like to handle next packets in "the meantime" using another process (or thread)
But I have 2 doubts which I would like to clarify.
1. - - - - - -
Is it possible to handle concurrently several packets with several processes ?
i.e. if the process1 is handling a packet will the process2 receive the next existing packet?
The essence of my doubt is:
if process1 receives the packet1 but the verdict is not issued will next recv() (in process2) be successful and not hold process2 until verdict for packet1 is made?
Please confirm.
2. - - - - - -
Let's assume that we have process1 which handles packet 1, process 2 which handles packet 2, etc
Let's say verdict for packet2 is issued before verdict for packet1
Will packet2 leave the queue or has it to "wait" for verdict for packet1 ?
The essence of my doubt is:
can packet2 leave the queue before packet1 ?
In other words can we change the packets order leaving system ?
Or, maybe, although verdict for packet2 is done it has to wait for verdict for packet1 and next both leave the queue in original order?
I would be obliged if you clarify above two matter
Best Regards,
Dorian
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Eric Leblond

2013-01-12 19:04 UTC #2 lore raw

Hello,

Quoted 8 lines · Felix

excellent question !
I too am interested in this.
Another similar, related question is do packets that are not filtered wait on a verdict for a filtered packet?
Again, if not, it means we can change packet order?
If so,it means we can degrade system performance unrelated to our filtered packets.

I've just wrote an article describing libnetfilter_queue and NFQUEUE. It has been written to answer your questions:
https://home.regit.org/netfilter-en/using-nfqueue-and-libnetfilter_queue/

Let me know if it is the case and don't hesitate if you have any suggestions.

BR,
--
Eric

Quoted 66 lines

Thanks,
Credzba
Quoted 51 lines · dorian
Hi all,
Sorry if the queries has been answered yet but I can't find the reliable informations about issues concerned with libnetfilter_queue.
I need to run a program which would firstly fork let's say 10 times (or creates 10 threads) and next each child-process (or thread) will include
   while ((rv = recv(fd, buf, sizeof(buf), 0)) && rv >= 0) {
      nfq_handle_packet(h, buf, rv);
   }
sequence.
The fork/multiple threads reason usage is that for some packets the processing time will be relatively long.
So I would like to handle next packets in "the meantime" using another process (or thread)
But I have 2 doubts which I would like to clarify.
1. - - - - - -
Is it possible to handle concurrently several packets with several processes ?
i.e. if the process1 is handling a packet will the process2 receive the next existing packet?
The essence of my doubt is:
if process1 receives the packet1 but the verdict is not issued will next recv() (in process2) be successful and not hold process2 until verdict for packet1 is made?
Please confirm.
2. - - - - - -
Let's assume that we have process1 which handles packet 1, process 2 which handles packet 2, etc
Let's say verdict for packet2 is issued before verdict for packet1
Will packet2 leave the queue or has it to "wait" for verdict for packet1 ?
The essence of my doubt is:
can packet2 leave the queue before packet1 ?
In other words can we change the packets order leaving system ?
Or, maybe, although verdict for packet2 is done it has to wait for verdict for packet1 and next both leave the queue in original order?
I would be obliged if you clarify above two matter
Quoted 7 lines
Best Regards,
Dorian
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Signature · 2 lines

Eric Leblond <[email protected]>
Blog: https://home.regit.org/

Felix

2013-01-12 19:00 UTC #3 lore raw

Good job.
I and everyone who needs this information will appreciate your work.

I posted some questions on the site, but they are ipfilter/nfqueue performance questions.
You answered the original question well.

Thank you.

Quoted 94 lines · Eric Leblond

Hello,
Quoted 8 lines · Felix
excellent question !
I too am interested in this.
Another similar, related question is do packets that are not filtered wait on a verdict for a filtered packet?
Again, if not, it means we can change packet order?
If so,it means we can degrade system performance unrelated to our filtered packets.
I've just wrote an article describing libnetfilter_queue and NFQUEUE. It has been written to answer your questions:
https://home.regit.org/netfilter-en/using-nfqueue-and-libnetfilter_queue/
Let me know if it is the case and don't hesitate if you have any suggestions.
BR,
--
Eric
Quoted 58 lines
Thanks,
Credzba
Quoted 54 lines · dorian
Hi all,
Sorry if the queries has been answered yet but I can't find the reliable informations about issues concerned with libnetfilter_queue.
I need to run a program which would firstly fork let's say 10 times (or creates 10 threads) and next each child-process (or thread) will include
   while ((rv = recv(fd, buf, sizeof(buf), 0)) && rv >= 0) {
      nfq_handle_packet(h, buf, rv);
   }
sequence.
The fork/multiple threads reason usage is that for some packets the processing time will be relatively long.
So I would like to handle next packets in "the meantime" using another process (or thread)
But I have 2 doubts which I would like to clarify.
1. - - - - - -
Is it possible to handle concurrently several packets with several processes ?
i.e. if the process1 is handling a packet will the process2 receive the next existing packet?
The essence of my doubt is:
if process1 receives the packet1 but the verdict is not issued will next recv() (in process2) be successful and not hold process2 until verdict for packet1 is made?
Please confirm.
2. - - - - - -
Let's assume that we have process1 which handles packet 1, process 2 which handles packet 2, etc
Let's say verdict for packet2 is issued before verdict for packet1
Will packet2 leave the queue or has it to "wait" for verdict for packet1 ?
The essence of my doubt is:
can packet2 leave the queue before packet1 ?
In other words can we change the packets order leaving system ?
Or, maybe, although verdict for packet2 is done it has to wait for verdict for packet1 and next both leave the queue in original order?
I would be obliged if you clarify above two matter
Best Regards,
Dorian
Quoted 8 lines
Quoted 4 lines
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Eric Leblond <[email protected]>
Blog: https://home.regit.org/

dorian

author

2013-01-12 21:16 UTC #4 lore raw

In reply to Eric Leblond

Hi Eric,
Thanks a lot for reply.

I read your article but apart from clarifying my doubts it created the new ones :)

Below are the issues....

1.- - - - -
You wrote: "packet can be verdict without order".
I am very sorry but my English is not perfect so I understand above as "having packets in the order 1,2,3,4 the verdict can be set in any order (f.ex. in the order 4,1,3,2)"
Am I right?

If my above understanding is correct it also contains implied information that I can call successfully several times recv() function and none of the calls block despite the fact that the verdicts have not been set for the recv'ed packets.

Am I right till now?

2. - - - - - -
Next you wrote
a) "The packet queue is a real queue"
and next
b) "A packet is released when userspace issue a verdict"

Well, I have some doubts here.

According to my primitive understanding (but which is the same as at
http://en.wikipedia.org/wiki/Queue_%28abstract_data_type%29) the queue is First-In-First-Out (FIFO) data structure.
So statement (b) is not true or not complete.

Because having several packets in queue (let's say 1,2,3,4) and setting verdict for packet 2 doesn't mean that it will be released (=sent) since it is _in the middle_ of the queue.

Please explain above.

Anyway at the bottom you also mentioned about 'packet reordering' which suggests that queue you are writing about in the article is not a queue or my understanding of therm 'queue' is too primitive.

3. - - - - - -
And "new doubt" appeared after reading your article.
You wrote: "The send/recv operation need to be protected by lock to avoid concurrent writing"

Well, I wrote a number of multi-processed programs (run also on multi-core CPUs) which used the same file descriptor by each child-process and I never found the situation that any locking has been required.
Each read or written messages (=sequence of bytes) were consistent. Of course without locking I do not have control on the order of the messages read/written from/to file descriptor but none of the read/written data were corrupted (i.e. messages were not mixed)

So is the lock really required? Why?
Is it a matter of the fact that 'fd' is the netlink socket?

4. - - - - - -
Another "new doubt" requires longer explanation.

Using the file 'source-nfq.c' as reference we can found there

   while ((rv = recv(fd, buf, sizeof(buf), 0)) && rv >= 0) {
      nfq_handle_packet(h, buf, rv);
   }

At the article you wrote: "the kernel sends a message containing packet data and related information to a socket and userpace reads this message" Does it means that 'buf' contains packet data?

I am asking about since 'fd' is the only socket I can find in the source-nfq.c.
On the other hand the packet data are accessible via struct nfq_data *nfa in callback function and 'nfa' has nothing in common (at least documentation does not state it) with the 'buf'.

And if the 'buf' really contains packet data what for the callback function is required?
Wasn't be simpler to access packet directly from 'buf'?

Till now I assumed that 'buf' contained only some kind of "libnetfilter_queue internal metadata" which was not interesting for me.

Any comment will be appreciated.

- - - - - - - -
And that's all.
I would be be grateful to you for helping me to understand how to use the libnetfilter_queue and what can be done using it.

With best regards,
Dorian

Quoted 88 lines

Hello,
Quoted 8 lines · Felix
excellent question !
I too am interested in this.
Another similar, related question is do packets that are not filtered wait on a verdict for a filtered packet?
Again, if not, it means we can change packet order?
If so,it means we can degrade system performance unrelated to our filtered packets.
I've just wrote an article describing libnetfilter_queue and NFQUEUE. It has been written to answer your questions:
https://home.regit.org/netfilter-en/using-nfqueue-and-libnetfilter_queue/
Let me know if it is the case and don't hesitate if you have any suggestions.
BR,
--
Eric
Quoted 66 lines
Thanks,
Credzba
Quoted 51 lines · dorian
Hi all,
Sorry if the queries has been answered yet but I can't find the reliable informations about issues concerned with libnetfilter_queue.
I need to run a program which would firstly fork let's say 10 times (or creates 10 threads) and next each child-process (or thread) will include
   while ((rv = recv(fd, buf, sizeof(buf), 0)) && rv >= 0) {
      nfq_handle_packet(h, buf, rv);
   }
sequence.
The fork/multiple threads reason usage is that for some packets the processing time will be relatively long.
So I would like to handle next packets in "the meantime" using another process (or thread)
But I have 2 doubts which I would like to clarify.
1. - - - - - -
Is it possible to handle concurrently several packets with several processes ?
i.e. if the process1 is handling a packet will the process2 receive the next existing packet?
The essence of my doubt is:
if process1 receives the packet1 but the verdict is not issued will next recv() (in process2) be successful and not hold process2 until verdict for packet1 is made?
Please confirm.
2. - - - - - -
Let's assume that we have process1 which handles packet 1, process 2 which handles packet 2, etc
Let's say verdict for packet2 is issued before verdict for packet1
Will packet2 leave the queue or has it to "wait" for verdict for packet1 ?
The essence of my doubt is:
can packet2 leave the queue before packet1 ?
In other words can we change the packets order leaving system ?
Or, maybe, although verdict for packet2 is done it has to wait for verdict for packet1 and next both leave the queue in original order?
I would be obliged if you clarify above two matter
Quoted 7 lines
Best Regards,
Dorian
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Eric Leblond

2013-01-12 22:12 UTC #5 lore raw

Hello,

Thanks for all these questions, I will improve/update the article thanks to that.

Quoted 6 lines · dorian

Hi Eric,
Thanks a lot for reply.
I read your article but apart from clarifying my doubts it created the new ones :)

Quoted 8 lines

Below are the issues....
1.- - - - -
You wrote: "packet can be verdict without order".
I am very sorry but my English is not perfect so I understand above as "having packets in the order 1,2,3,4 the verdict can be set in any order (f.ex. in the order 4,1,3,2)"
Am I right?

Exactly.

Quoted 5 lines

If my above understanding is correct it also contains implied information that I can call successfully several times recv() function and none of the calls block despite the fact that the verdicts have not been set for the recv'ed packets.
Am I right till now?

Yes. You can read packets, take a coffee and came back later to verdict them all at once.

Quoted 9 lines

2. - - - - - -
Next you wrote
a) "The packet queue is a real queue"
and next
b) "A packet is released when userspace issue a verdict"
Well, I have some doubts here.

Quoted 8 lines

According to my primitive understanding (but which is the same as at
http://en.wikipedia.org/wiki/Queue_%28abstract_data_type%29) the queue is First-In-First-Out (FIFO) data structure.
So statement (b) is not true or not complete.
Because having several packets in queue (let's say 1,2,3,4) and setting verdict for packet 2 doesn't mean that it will be released (=sent) since it is _in the middle_ of the queue.

OK real queue was not a correct term. See explanation below.

Quoted 5 lines

Please explain above.
Anyway at the bottom you also mentioned about 'packet reordering' which suggests that queue you are writing about in the article is not a queue or my understanding of therm 'queue' is too primitive.

It is a chained list with the following operation:
* Enqueue: element is added at the end
* Dequeue (verdict): element is removed from the chained list.

Quoted 14 lines

3. - - - - - -
And "new doubt" appeared after reading your article.
You wrote: "The send/recv operation need to be protected by lock to avoid concurrent writing"
Well, I wrote a number of multi-processed programs (run also on multi-core CPUs) which used the same file descriptor by each child-process and I never found the situation that any locking has been required.
Each read or written messages (=sequence of bytes) were consistent. Of course without locking I do not have control on the order of the messages read/written from/to file descriptor but none of the read/written data were corrupted (i.e. messages were not mixed)

Quoted 2 lines

So is the lock really required? Why?
Is it a matter of the fact that 'fd' is the netlink socket?

Humm, you may have a point here. I'm doing this by habit and a check could be needed.

Quoted 19 lines

4. - - - - - -
Another "new doubt" requires longer explanation.
Using the file 'source-nfq.c' as reference we can found there
   while ((rv = recv(fd, buf, sizeof(buf), 0)) && rv >= 0) {
      nfq_handle_packet(h, buf, rv);
   }
At the article you wrote: "the kernel sends a message containing packet data and related information to a socket and userpace reads this message" Does it means that 'buf' contains packet data?
I am asking about since 'fd' is the only socket I can find in the source-nfq.c.
On the other hand the packet data are accessible via struct nfq_data *nfa in callback function and 'nfa' has nothing in common (at least documentation does not state it) with the 'buf'.

Quoted 6 lines

And if the 'buf' really contains packet data what for the callback function is required?
Wasn't be simpler to access packet directly from 'buf'?
Till now I assumed that 'buf' contained only some kind of "libnetfilter_queue internal metadata" which was not interesting for me.

buf is containing a nfnetlink message not the packet by itself. The protocol messages are nfnetlink formatted.

Quoted 10 lines

Any comment will be appreciated.
- - - - - - - -
And that's all.
I would be be grateful to you for helping me to understand how to use the libnetfilter_queue and what can be done using it.
With best regards,
Dorian

BR,
--
Eric

Quoted 96 lines

Quoted 18 lines
Hello,
Quoted 8 lines · Felix
excellent question !
I too am interested in this.
Another similar, related question is do packets that are not filtered wait on a verdict for a filtered packet?
Again, if not, it means we can change packet order?
If so,it means we can degrade system performance unrelated to our filtered packets.
I've just wrote an article describing libnetfilter_queue and NFQUEUE. It has been written to answer your questions:
https://home.regit.org/netfilter-en/using-nfqueue-and-libnetfilter_queue/
Quoted 73 lines
Let me know if it is the case and don't hesitate if you have any suggestions.
BR,
--
Eric
Quoted 66 lines
Thanks,
Credzba
Quoted 51 lines · dorian
Hi all,
Sorry if the queries has been answered yet but I can't find the reliable informations about issues concerned with libnetfilter_queue.
I need to run a program which would firstly fork let's say 10 times (or creates 10 threads) and next each child-process (or thread) will include
   while ((rv = recv(fd, buf, sizeof(buf), 0)) && rv >= 0) {
      nfq_handle_packet(h, buf, rv);
   }
sequence.
The fork/multiple threads reason usage is that for some packets the processing time will be relatively long.
So I would like to handle next packets in "the meantime" using another process (or thread)
But I have 2 doubts which I would like to clarify.
1. - - - - - -
Is it possible to handle concurrently several packets with several processes ?
i.e. if the process1 is handling a packet will the process2 receive the next existing packet?
The essence of my doubt is:
if process1 receives the packet1 but the verdict is not issued will next recv() (in process2) be successful and not hold process2 until verdict for packet1 is made?
Please confirm.
2. - - - - - -
Let's assume that we have process1 which handles packet 1, process 2 which handles packet 2, etc
Let's say verdict for packet2 is issued before verdict for packet1
Will packet2 leave the queue or has it to "wait" for verdict for packet1 ?
The essence of my doubt is:
can packet2 leave the queue before packet1 ?
In other words can we change the packets order leaving system ?
Or, maybe, although verdict for packet2 is done it has to wait for verdict for packet1 and next both leave the queue in original order?
I would be obliged if you clarify above two matter
Quoted 7 lines
Best Regards,
Dorian
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Signature · 2 lines

Eric Leblond <[email protected]>
Blog: https://home.regit.org/

dorian

author

2013-01-12 22:34 UTC #6 lore raw

Many thanks for explanations & updates.

Seems now I have a proper (although probably not full) image how libnetfilter_queue works

Thanks once again.

Dorian

Quoted 212 lines

Hello,
Thanks for all these questions, I will improve/update the article thanks to that.
Quoted 6 lines · dorian
Hi Eric,
Thanks a lot for reply.
I read your article but apart from clarifying my doubts it created the new ones :)
Quoted 8 lines
Below are the issues....
1.- - - - -
You wrote: "packet can be verdict without order".
I am very sorry but my English is not perfect so I understand above as "having packets in the order 1,2,3,4 the verdict can be set in any order (f.ex. in the order 4,1,3,2)"
Am I right?
Exactly.
Quoted 5 lines
If my above understanding is correct it also contains implied information that I can call successfully several times recv() function and none of the calls block despite the fact that the verdicts have not been set for the recv'ed packets.
Am I right till now?
Yes. You can read packets, take a coffee and came back later to verdict them all at once.
Quoted 8 lines
2. - - - - - -
Next you wrote
a) "The packet queue is a real queue"
and next
b) "A packet is released when userspace issue a verdict"
Well, I have some doubts here.
Quoted 8 lines
According to my primitive understanding (but which is the same as at
http://en.wikipedia.org/wiki/Queue_%28abstract_data_type%29) the queue is First-In-First-Out (FIFO) data structure.
So statement (b) is not true or not complete.
Because having several packets in queue (let's say 1,2,3,4) and setting verdict for packet 2 doesn't mean that it will be released (=sent) since it is _in the middle_ of the queue.
OK real queue was not a correct term. See explanation below.
Quoted 5 lines
Please explain above.
Anyway at the bottom you also mentioned about 'packet reordering' which suggests that queue you are writing about in the article is not a queue or my understanding of therm 'queue' is too primitive.
It is a chained list with the following operation:
* Enqueue: element is added at the end
* Dequeue (verdict): element is removed from the chained list.
Quoted 14 lines
3. - - - - - -
And "new doubt" appeared after reading your article.
You wrote: "The send/recv operation need to be protected by lock to avoid concurrent writing"
Well, I wrote a number of multi-processed programs (run also on multi-core CPUs) which used the same file descriptor by each child-process and I never found the situation that any locking has been required.
Each read or written messages (=sequence of bytes) were consistent. Of course without locking I do not have control on the order of the messages read/written from/to file descriptor but none of the read/written data were corrupted (i.e. messages were not mixed)
Quoted 2 lines
So is the lock really required? Why?
Is it a matter of the fact that 'fd' is the netlink socket?
Humm, you may have a point here. I'm doing this by habit and a check could be needed.
Quoted 18 lines
4. - - - - - -
Another "new doubt" requires longer explanation.
Using the file 'source-nfq.c' as reference we can found there
   while ((rv = recv(fd, buf, sizeof(buf), 0)) && rv >= 0) {
      nfq_handle_packet(h, buf, rv);
   }
At the article you wrote: "the kernel sends a message containing packet data and related information to a socket and userpace reads this message" Does it means that 'buf' contains packet data?
I am asking about since 'fd' is the only socket I can find in the source-nfq.c.
On the other hand the packet data are accessible via struct nfq_data *nfa in callback function and 'nfa' has nothing in common (at least documentation does not state it) with the 'buf'.
Quoted 6 lines
And if the 'buf' really contains packet data what for the callback function is required?
Wasn't be simpler to access packet directly from 'buf'?
Till now I assumed that 'buf' contained only some kind of "libnetfilter_queue internal metadata" which was not interesting for me.
buf is containing a nfnetlink message not the packet by itself. The protocol messages are nfnetlink formatted.
Quoted 10 lines
Any comment will be appreciated.
- - - - - - - -
And that's all.
I would be be grateful to you for helping me to understand how to use the libnetfilter_queue and what can be done using it.
With best regards,
Dorian
BR,
--
Eric
Quoted 94 lines
Quoted 17 lines
Hello,
Quoted 8 lines · Felix
excellent question !
I too am interested in this.
Another similar, related question is do packets that are not filtered wait on a verdict for a filtered packet?
Again, if not, it means we can change packet order?
If so,it means we can degrade system performance unrelated to our filtered packets.
I've just wrote an article describing libnetfilter_queue and NFQUEUE. It has been written to answer your questions:
https://home.regit.org/netfilter-en/using-nfqueue-and-libnetfilter_queue/
Quoted 73 lines
Let me know if it is the case and don't hesitate if you have any suggestions.
BR,
--
Eric
Quoted 66 lines
Thanks,
Credzba
Quoted 51 lines · dorian
Hi all,
Sorry if the queries has been answered yet but I can't find the reliable informations about issues concerned with libnetfilter_queue.
I need to run a program which would firstly fork let's say 10 times (or creates 10 threads) and next each child-process (or thread) will include
   while ((rv = recv(fd, buf, sizeof(buf), 0)) && rv >= 0) {
      nfq_handle_packet(h, buf, rv);
   }
sequence.
The fork/multiple threads reason usage is that for some packets the processing time will be relatively long.
So I would like to handle next packets in "the meantime" using another process (or thread)
But I have 2 doubts which I would like to clarify.
1. - - - - - -
Is it possible to handle concurrently several packets with several processes ?
i.e. if the process1 is handling a packet will the process2 receive the next existing packet?
The essence of my doubt is:
if process1 receives the packet1 but the verdict is not issued will next recv() (in process2) be successful and not hold process2 until verdict for packet1 is made?
Please confirm.
2. - - - - - -
Let's assume that we have process1 which handles packet 1, process 2 which handles packet 2, etc
Let's say verdict for packet2 is issued before verdict for packet1
Will packet2 leave the queue or has it to "wait" for verdict for packet1 ?
The essence of my doubt is:
can packet2 leave the queue before packet1 ?
In other words can we change the packets order leaving system ?
Or, maybe, although verdict for packet2 is done it has to wait for verdict for packet1 and next both leave the queue in original order?
I would be obliged if you clarify above two matter
Quoted 7 lines
Best Regards,
Dorian
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Pablo Neira Ayuso

maintainer

2013-01-13 11:29 UTC #7 lore raw

In reply to Eric Leblond

Quoted 12 lines · Eric Leblond

Hello,
Quoted 8 lines · Felix
excellent question !
I too am interested in this.
Another similar, related question is do packets that are not filtered wait on a verdict for a filtered packet?
Again, if not, it means we can change packet order?
If so,it means we can degrade system performance unrelated to our filtered packets.

Quoted 3 lines

I've just wrote an article describing libnetfilter_queue and NFQUEUE. It has been written to answer your questions:
https://home.regit.org/netfilter-en/using-nfqueue-and-libnetfilter_queue/

It would be good to integrate part of this information to the doxygen library documentation:

http://www.netfilter.org/projects/libnetfilter_queue/

Or simply add reference to Eric's page in the main page of the doxygen.

Regards.

Eric Leblond

2013-01-13 22:46 UTC #8 lore raw

Hello,

Quoted 17 lines · Pablo Neira Ayuso

Quoted 12 lines · Eric Leblond
Hello,
Quoted 8 lines · Felix
excellent question !
I too am interested in this.
Another similar, related question is do packets that are not filtered wait on a verdict for a filtered packet?
Again, if not, it means we can change packet order?
If so,it means we can degrade system performance unrelated to our filtered packets.
Quoted 3 lines
I've just wrote an article describing libnetfilter_queue and NFQUEUE. It has been written to answer your questions:
https://home.regit.org/netfilter-en/using-nfqueue-and-libnetfilter_queue/

Quoted 6 lines

It would be good to integrate part of this information to the doxygen library documentation:
http://www.netfilter.org/projects/libnetfilter_queue/
Or simply add reference to Eric's page in the main page of the doxygen.

To follow a small patchset trying to improve libnetfilter_queue doxygen documentation.

BR,

Signature · 2 lines

Eric Leblond <[email protected]>
Blog: https://home.regit.org/

Eric Leblond

2013-01-13 22:49 UTC #9 lore raw

PATCH 1/2 doxygen: improve fail-open documentation.

---
 src/libnetfilter_queue.c |    8 ++++++++
 1 file changed, 8 insertions(+)

src/libnetfilter_queue.c +8 -0

index 9fe9dfa..f93ac66 100644
@@ -87,6 +87,8 @@
  * (requires Linux kernel >= 2.6.30).
  * - see --queue-balance option in NFQUEUE target for multi-threaded apps
  * (it requires Linux kernel >= 2.6.31).
+ * - consider using fail-open option see nfq_set_queue_flags() (it requires
+ *  Linux kernel >= 3.6)
  */
 
 struct nfq_handle

@@ -620,6 +622,12 @@ int nfq_set_mode(struct nfq_q_handle *qh,
 	flags &= ~NFQA_CFG_F_FAIL_OPEN;
 	err = nfq_set_queue_flags(qh, mask, flags);
 \endverbatim
+ *
+ * If NFQA_CFG_F_FAIL_OPEN is used, the kernel will accept instead of
+ * drop packets that should have been enqueued to a full queue. This
+ * results in the system being able to handle high network load but at
+ * the depend of the control on the packets.
+ *
  * \return -1 on error with errno set appropriately; =0 otherwise.
  */
 int nfq_set_queue_flags(struct nfq_q_handle *qh,

Signature · 1 line

1.7.10.4

Eric Leblond

2013-01-13 22:49 UTC #10 lore raw

PATCH 2/2 doxygen: improve documentation

This patch improves the doxygen documentation and adds a reference to an external article.

---
 src/libnetfilter_queue.c |   22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

src/libnetfilter_queue.c +21 -1

index f93ac66..be25217 100644
@@ -51,7 +51,21 @@
  *  - receiving queued packets from the kernel nfnetlink_queue subsystem
  *  - issuing verdicts and/or reinjecting altered packets to the kernel
  *  nfnetlink_queue subsystem
- * 
+ *
+ * The cinematic is the following: When an iptables rules with target NFQUEUE
+ * matches, the kernel en-queued the packet in a chained list. It then format
+ * a nfnetlink message and sends the information (packet data , packet id and
+ * metadata) via a socket to the software connected to the queue. The software
+ * can then read the message.
+ *
+ * To remove the packet from the queue, the userspace software must issue a
+ * verdict asking kernel to accept or drop the packet. Userspace can also alter
+ * the packet. Verdict can be done in asynchronous manner, as the only needed
+ * information is the packet id.
+ *
+ * When a queue is full, packets that should have been en-queued are dropped by
+ * kernel instead of being en-queued.
+ *
  * \section Git Tree
  * The current development version of libnetfilter_queue can be accessed
  * at https://git.netfilter.org/cgi-bin/gitweb.cgi?p=libnetfilter_queue.git;a=summary.

@@ -66,6 +80,10 @@
  * the doxygen documentation (start by \link LibrarySetup \endlink page) and
  * nf-queue.c source file.
  *
+ * Another source of information on libnetfilter_queue usage is the following
+ * article:
+ *  https://home.regit.org/netfilter-en/using-nfqueue-and-libnetfilter_queue/
+ *
  * \section errors ENOBUFS errors in recv()
  *
  * recv() may return -1 and errno is set to ENOBUFS in case that your

@@ -89,6 +107,8 @@
  * (it requires Linux kernel >= 2.6.31).
  * - consider using fail-open option see nfq_set_queue_flags() (it requires
  *  Linux kernel >= 3.6)
+ * - increase queue max length with nfq_set_queue_maxlen() to resist to packets
+ * burst
  */
 
 struct nfq_handle

Signature · 1 line

1.7.10.4

dorian

author

2013-01-14 14:24 UTC #11 lore raw

In reply to Eric Leblond

Hi Eric,
Thanks a lot for explanations.

I would like ask you about another matter.

Previously, without deeper knowledge about dequeuing rule based on verdict rather then the order of the packets I've tried to implement another solution.

I assumed that for queued packets 1,2,3,4 the verdict should be set in the (same) order 1,2,3,4.

But since for packet 2 I needed to make some processing (which took a while) and not wanting to block the packets 3 & 4 I used the following strategy in my program.

packet1 -> NF_ACCEPT
packet2 -> need to be processed -> copy packet to program buffer -> NF_DROP packet3 -> NF_ACCEPT
packet4 -> NF_ACCEPT

And the copy of the packet was processed "off-line" and finally the packet should be "accepted"

For this I needed to find out how to re-insert the copy of the packet 2 into the network and raw udp socket seemed to be right choice.

And I found the problem - using sendto() function I observed that the packet incoming to the client was altered (somewhere in the system).

To be more specific: all above concerned http packets transmitted from a server to the client and the problem was that the tcp source (server's) port has been altered.

Additionally tcp checksum was changed for new but correct value (but probably this was recalculated by interface)

Finally the packet arrived to the destination host but was refused by the host as it was not from port 80.

I detected also that above "change" concerns only packets without 'fin' flag.

Packet2 with 'fin' flag has been delivered properly.

I understand that above does not directly concern libnetfilter_queue matters but it is an issue which explanation maybe interesting.

Maybe you have some knowledge why above could happened?

Why system modifies the packet (and additionally in specific packet
place: always source port) sent with raw socket?

And how to send packet without this unexpected system interventions ?

Any advice will be appreciated.

Best regards,
Dorian

Quoted 28 lines

Hello,
Quoted 15 lines · Pablo Neira Ayuso
Quoted 6 lines · Eric Leblond
Hello,
Quoted 3 lines · Felix
excellent question !
I too am interested in this.
Quoted 8 lines
Quoted 5 lines
Another similar, related question is do packets that are not filtered wait on a verdict for a filtered packet?
Again, if not, it means we can change packet order?
If so,it means we can degrade system performance unrelated to our filtered packets.
I've just wrote an article describing libnetfilter_queue and NFQUEUE. It has been written to answer your questions:
https://home.regit.org/netfilter-en/using-nfqueue-and-libnetfilter_queue/
Quoted 6 lines
It would be good to integrate part of this information to the doxygen library documentation:
http://www.netfilter.org/projects/libnetfilter_queue/
Or simply add reference to Eric's page in the main page of the doxygen.
To follow a small patchset trying to improve libnetfilter_queue doxygen documentation.
BR,