nftables: bridge filter with queue to userspace
I'm trying to build a transparent filter with application level filtering. First experiment with ip and output hook and queue to userspace was successful. Then I changed to bridge filtering with forward hook. With counter action I see that the packets match the rule, but the queue to the usersapce doesn't work.
Am I right, that this fuction should work?
I'm using Fedora 22 with nftables 0.4.
Quoted 7 lines · Martin Gröger
I'm trying to build a transparent filter with application level filtering. First experiment with ip and output hook and queue to userspace was successful. Then I changed to bridge filtering with forward hook. With counter action I see that the packets match the rule, but the queue to the usersapce doesn't work.
Am I right, that this fuction should work?
I guess you're using the 'bridge_netfilter' module?
Florian told me he will come up sooner or later with native queue support for nft (ie. no bridge_netfilter required anymore).
Quoted 1 line
I'm using Fedora 22 with nftables 0.4.
Not related to this problem, but it's a good idea to stick to latest. Lots of fixes and updates have happened between 0.4 and 0.5.
Pablo Neira Ayuso <[email protected]> wrote:
[ CC Bernhard ]
Quoted 8 lines · Martin Gröger
Quoted 7 lines
I'm trying to build a transparent filter with application level filtering. First experiment with ip and output hook and queue to userspace was successful. Then I changed to bridge filtering with forward hook. With counter action I see that the packets match the rule, but the queue to the usersapce doesn't work.
Am I right, that this fuction should work?
nfqueue backend only works with NFPROTO_IPV4 and _IPV6 at the moment, i.e. nft ip and nft ip6, or via bridge_netfilter hack (which 'pushes' packets though ipv4/ipv6 netfilter hooks).
Quoted 2 lines
Florian told me he will come up sooner or later with native queue support for nft (ie. no
bridge_netfilterrequired anymore).
Argh. I'm a moron and forgot about this.
I still have the q&d hack that makes it work but no reroute (re-bridge, cough) support, just dump-to-userspace.
Bernhard, did you have time to work on this?
If not, I think I can make time available soon since the other work (nft bridge conntrack, nf netns hook stuff) is delayed at the moment anyways.
Am 29.10.2015 23:23, schrieb Florian Westphal:
Quoted 11 lines
Pablo Neira Ayuso <[email protected]> wrote:
[ CC Bernhard ]
Quoted 7 lines · Martin Gröger
Quoted 6 lines
I'm trying to build a transparent filter with application level filtering. First experiment with ip and output hook and queue to userspace was successful. Then I changed to bridge filtering with forward hook. With counter action I see that the packets match the rule, but the queue to the usersapce doesn't work.
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Am I right, that this fuction should work?
nfqueue backend only works with
NFPROTO_IPV4and_IPV6at the moment, i.e. nft ip and nft ip6, or viabridge_netfilterhack (which 'pushes' packets though ipv4/ipv6 netfilter hooks).
Ok, this explains why my experiment with ip work, but not the one with bridge family.
Quoted 4 lines
Quoted 3 lines
Florian told me he will come up sooner or later with native queue support for nft (ie. no
bridge_netfilterrequired anymore).Argh. I'm a moron and forgot about this.
How does a native (indepent on bridge) queue support should work ?
Quoted 3 lines
I still have the q&d hack that makes it work but no reroute (re-bridge, cough) support, just dump-to-userspace.
As far as I understand this would be sufficient for my usecase, since I want simply to inspect the packets and then decide to accept or drop them.
Quoted 7 lines
Bernhard, did you have time to work on this?
If not, I think I can make time available soon since the other work (nft bridge conntrack, nf netns hook stuff) is delayed at the moment anyways.
Would be great, if you can help me!
Quoted 4 lines · Martin Gröger
Quoted 3 lines
Quoted 2 lines
Florian told me he will come up sooner or later with native queue support for nft (ie. no
bridge_netfilterrequired anymore).Argh. I'm a moron and forgot about this.
How does a native (indepent on bridge) queue support should work ?
Highlevel?
nft add bridge filter forward queue
But on the backend/userspace side its a good question.
For instance, where should NFQA_PAYLOAD point to?
Start of ethernet header?
Start of network header?
If the former, what should NFQA_HWADDR contain?
Should it even be present?
What about ->hw_protocol in nfqnl_msg_packet_hdr ?
We also should definitely present VLAN headers to userspace in some way. Standard question is if that should be extra attribute or if we should inject the vlan header into the packet data.
I'm leaning towards:
- NFQA_PAYLOAD starts at beginning of ethernet header
- nfqnl_msg_packet_hdr->hw_protocol is set to skb->protocol OR skb->vlan_proto in offload case (even though this information is redundant since its also present in the ethernet header ...)
- NFQA_PAYLOAD maintains illusion of disabled/non-existant VLAN hw offload, i.e. we insert it into NFQA_PAYLOAD between mac and network header.
- NFQA_HWADDR attribute is not present (redundant, we have this in NFQA_PAYLOAD).
Quoted 5 lines
Quoted 2 lines
I still have the q&d hack that makes it work but no reroute (re-bridge, cough) support, just dump-to-userspace.
As far as I understand this would be sufficient for my usecase, since I want simply to inspect the packets and then decide to accept or drop them.
Yes, in fact I think we should just ignore reroute (bad idea) or rebridge (what would be the use case of this...?)
If someone really needs to be able to resend/relay packet they could do this in userspace or just use PRE_ROUTING since thats before bridge asks the FDB for the output port.
We could add bridge_me_harder to pick another output device for queueing in BRIDGE OUTPUT but I have no idea why one would want such feature.
Am 30.10.2015 14:38, schrieb Florian Westphal:
Quoted 7 lines
Quoted 4 lines · Martin Gröger
Quoted 3 lines
Quoted 2 lines
Florian told me he will come up sooner or later with native queue support for nft (ie. no
bridge_netfilterrequired anymore).Argh. I'm a moron and forgot about this.
How does a native (indepent on bridge) queue support should work ?
Highlevel?
nft add bridge filter forward queue
That's exactly what I expected.
Quoted 6 lines
But on the backend/userspace side its a good question.
For instance, where should
NFQA_PAYLOADpoint to?Start of ethernet header?
Start of network header?
However, the full information contained in the paket (incl. VLAN) should be given to the user. So it wouold make sense to point to the ethernet header with NFQA_PAYLOAD.
Quoted 4 lines
If the former, what should
NFQA_HWADDRcontain?
Should it even be present?
Quoted 8 lines
What about ->hw_protocol in
nfqnl_msg_packet_hdr?We also should definitely present VLAN headers to userspace in some way. Standard question is if that should be extra attribute or if we should inject the vlan header into the packet data.
I'm leaning towards:
-NFQA_PAYLOADstarts at beginning of ethernet header
ok
Quoted 6 lines
-
nfqnl_msg_packet_hdr->hw_protocolis set toskb->protocolORskb->vlan_protoin offload case (even though this information is redundant since its also present in the ethernet header ...)
-NFQA_PAYLOADmaintains illusion of disabled/non-existant VLAN hw offload, i.e. we insert it intoNFQA_PAYLOADbetween mac and network header.
Sorry, I don't understand this. The VLAN header is (if exists) after the source MAC. If a paket is bridged, I would expect, that the VLAN header is kept unchanged. I would expect to find the VLAN header exactly there! Is this wrong?
Quoted 10 lines
-
NFQA_HWADDRattribute is not present (redundant, we have this inNFQA_PAYLOAD).Quoted 5 lines
Quoted 2 lines
I still have the q&d hack that makes it work but no reroute (re-bridge, cough) support, just dump-to-userspace.
As far as I understand this would be sufficient for my usecase, since I want simply to inspect the packets and then decide to accept or drop them.
Yes, in fact I think we should just ignore reroute (bad idea) or rebridge (what would be the use case of this...?)
Quoted 7 lines
If someone really needs to be able to resend/relay packet they could do this in userspace or just use
PRE_ROUTINGsince thats before bridge asks the FDB for the output port.We could add
bridge_me_harderto pick another output device for queueing in BRIDGE OUTPUT but I have no idea why one would want such feature.
I'm currently trying to understand the structure. So to add the missing parts:
- there is no change in the nftable kernel modules necessary?
- there are changes in the nftnl library necessary?
- there are changes in nft necessary?
Is this correct?
I didn't find much documentation conecrning the architecture. Only Pablo_nftables-osd-userday-2013.pdf and Nftables-osd-2013-developer.pdf. Is there somethimg more to read to get an better understanding of nftables?
Thanks,
Martin
Quoted 7 lines · Martin Gröger
Quoted 3 lines
-
NFQA_PAYLOADmaintains illusion of disabled/non-existant VLAN hw offload, i.e. we insert it intoNFQA_PAYLOADbetween mac and network header.Sorry, I don't understand this. The VLAN header is (if exists) after the source MAC. If a paket is bridged, I would expect, that the VLAN header is kept unchanged. I would expect to find the VLAN header exactly there! Is this wrong?
Yes, there is no VLAN header, its removed (usually by hardware offloads) and stored in meta data only.
Thats why I think we should transparently re-insert when sending the copy to userspace.
(I.e. undo what hardware offloads did).
Quoted 14 lines
Quoted 14 lines
-
NFQA_HWADDRattribute is not present (redundant, we have this inNFQA_PAYLOAD).Quoted 5 lines
Quoted 2 lines
I still have the q&d hack that makes it work but no reroute (re-bridge, cough) support, just dump-to-userspace.
As far as I understand this would be sufficient for my usecase, since I want simply to inspect the packets and then decide to accept or drop them.
Yes, in fact I think we should just ignore reroute (bad idea) or rebridge (what would be the use case of this...?)
If someone really needs to be able to resend/relay packet they could do this in userspace or just use
PRE_ROUTINGsince thats before bridge asks the FDB for the output port.
Quoted 6 lines
Quoted 3 lines
We could add
bridge_me_harderto pick another output device for queueing in BRIDGE OUTPUT but I have no idea why one would want such feature.I'm currently trying to understand the structure. So to add the missing parts:
- there is no change in the nftable kernel modules necessary?
Not to nftables but to nfnetlink_queue module (and to bridge netfilter kernel part).
Quoted 1 line
- there are changes in the nftnl library necessary?
No, unless we add extra attribute e.g. for vlan header but I'd like to avoid it.
Quoted 1 line
- there are changes in nft necessary?
No, nft side should already work just fine.
Quoted 3 lines
Only Pablo_nftables-osd-userday-2013.pdf and Nftables-osd-2013-developer.pdf.
Is there somethimg more to read to get an better understanding of nftables?
Sorry, not that I know of. Perhaps Patrick or Pablo have more information available somewhere.
Am 30.10.2015 22:27, schrieb Florian Westphal:
Quoted 8 lines
Quoted 7 lines · Martin Gröger
Quoted 3 lines
-
NFQA_PAYLOADmaintains illusion of disabled/non-existant VLAN hw offload, i.e. we insert it intoNFQA_PAYLOADbetween mac and network header.Sorry, I don't understand this. The VLAN header is (if exists) after the source MAC. If a paket is bridged, I would expect, that the VLAN header is kept unchanged. I would expect to find the VLAN header exactly there! Is this wrong?
Quoted 7 lines
Yes, there is no VLAN header, its removed (usually by hardware offloads) and stored in meta data only.
Thats why I think we should transparently re-insert when sending the copy to userspace.
(I.e. undo what hardware offloads did).
Ok, now I understand.
So there are basically 2 options:
1.) reinsert the VLAN header
pro: looks like the wire
con: Is the insert measurable in terms of performance?
2.) pass der VLAN header also as meta data to the user.
pro: network header is always on the same offset, so it might be
easier for higher layer inspection
con: not like the wire
So, although I would have expected option 1, I think option 2 has advantages too. So I think both shoud be ok.
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
-
NFQA_HWADDRattribute is not present (redundant, we have this inNFQA_PAYLOAD).Quoted 5 lines
Quoted 2 lines
I still have the q&d hack that makes it work but no reroute (re-bridge, cough) support, just dump-to-userspace.
As far as I understand this would be sufficient for my usecase, since I want simply to inspect the packets and then decide to accept or drop them.
Yes, in fact I think we should just ignore reroute (bad idea) or rebridge (what would be the use case of this...?)
If someone really needs to be able to resend/relay packet they could do this in userspace or just use
PRE_ROUTINGsince thats before bridge asks the FDB for the output port.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
We could add
bridge_me_harderto pick another output device for queueing in BRIDGE OUTPUT but I have no idea why one would want such feature.I'm currently trying to understand the structure. So to add the missing parts:
- there is no change in the nftable kernel modules necessary?Not to nftables but to
nfnetlink_queuemodule (and to bridge netfilter kernel part).
From the presentation Nftables-osd-2013-developer.pdf I understood, that there is a generic instruction set part of the kernel. New functionality will be added by new combination of this instruction set. Additionally there must be some code for each hook, but I would expext, that this code provides the packet and some metadata to the engine, which process the instructions. Also for the interface from the main engine to the nfnetlink_queue module I would expect the packet and a metadata structure. Since the queueing for the IP hook works, I expected that all generic instructions for this purpose are available and just have to be used for the bridge hook.
So obviously somehow my expectation is wrong - but what's wrong?
Quoted 4 lines
Quoted 2 lines
- there are changes in the nftnl library necessary?
No, unless we add extra attribute e.g. for vlan header but I'd like
avoid it.
Ok, is this again an argument for Option 1 above?
Is this, because there is currently in the generic instructions no option to add VLAN metadata somehow to the nfnetlink_queue?
Quoted 10 lines
Quoted 2 lines
- there are changes in nft necessary?
No, nft side should already work just fine.
Quoted 3 lines
Only Pablo_nftables-osd-userday-2013.pdf and Nftables-osd-2013-developer.pdf.
Is there somethimg more to read to get an better understanding of nftables?Sorry, not that I know of. Perhaps Patrick or Pablo have more information available somewhere.