Masquerade based on skb->mark ?
I'm now trying to masquerade packets that have been marked a certain way. I'm using these commands:
# I'm not sure this is doing the right thing, but it is not giving errors. iptables -A POSTROUTING -t nat -j MASQUERADE -m mark --mark 10001
# This appears to work as planned.
iptables -t mangle -A PREROUTING -i eth1 -j MARK --set-mark 10001 iptables -t mangle -A PREROUTING -i eth2 -j MARK --set-mark 10001
I added a u32 'mark' field to the conn-track tuple, and I now get different conn-tracking objects for the same source/dest but with different 'mark'. However, the ct->status bit does not have the IPS_SRC_NAT flag set.
I think I need to figure out what code creates the initial conn-track and make sure it is setting the status bits correctly based on the skb->mark, but I am not sure where this code exists.
Any ideas on where to start looking? I've been trying to follow the code path in the netfilter/nat logic, but it's proving slow going!
Thanks,
Ben
PS. If anyone does this sort of work for hire, please contact me off-list.
Signature · 2 lines
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com
Quoted 6 lines · Ben Greear
I'm now trying to masquerade packets that have been marked a certain way. I'm using these commands:
# I'm not sure this is doing the right thing, but it is not giving errors. iptables -A POSTROUTING -t nat -j MASQUERADE -m mark --mark 10001
It does what the programmer said:
Masquerade only packets with a mark of 10001.
Quoted 3 lines
# This appears to work as planned.
iptables -t mangle -A PREROUTING -i eth1 -j MARK --set-mark 10001 iptables -t mangle -A PREROUTING -i eth2 -j MARK --set-mark 10001
And this says:
mark all packets that come from eth1 and eth2.
So in essence, you have "masquerade everything that came from eth1 and eth2". A slight bug, but ok. (In most cases, you only want to masquerade on some interfaces, not all, so the use of -o is usually wanted.)
Quoted 1 line
I added a u32 'mark' field to the conn-track tuple,
Just why?
Quoted 9 lines
and I now get
different conn-tracking objects for the same source/dest but with different 'mark'. However, thect->statusbit does not have theIPS_SRC_NATflag set.I think I need to figure out what code creates the initial conn-track and make sure it is setting the status bits correctly based on the
skb->mark, but I am not sure where this code exists.
Quoted 8 lines
Any ideas on where to start looking? I've been trying to follow the code path in the netfilter/nat logic, but it's proving slow going!
Thanks,
BenPS. If anyone does this sort of work for hire, please contact me off-list.
Jan
Quoted 17 lines · Jan Engelhardt
Quoted 5 lines · Ben Greear
I'm now trying to masquerade packets that have been marked a certain way. I'm using these commands:
# I'm not sure this is doing the right thing, but it is not giving errors. iptables -A POSTROUTING -t nat -j MASQUERADE -m mark --mark 10001
It does what the programmer said:
Masquerade only packets with a mark of 10001.Quoted 3 lines
# This appears to work as planned.
iptables -t mangle -A PREROUTING -i eth1 -j MARK --set-mark 10001 iptables -t mangle -A PREROUTING -i eth2 -j MARK --set-mark 10001And this says:
mark all packets that come from eth1 and eth2.
Quoted 7 lines
So in essence, you have "masquerade everything that came from eth1 and eth2". A slight bug, but ok. (In most cases, you only want to masquerade on some interfaces, not all, so the use of -o is usually wanted.)
Quoted 1 line
I added a u32 'mark' field to the conn-track tuple,
Just why?
Because otherwise it seems to me that there is only a single conn-tracking tuple for src -- dest, and it also seems to me that the conn-track entity has the should-we-NAT flags (in the 'status' bitfield).
My scenario involves
virtual routers (ie, routing tables with rules so that pkts hit certain routing tables) and sending packets through (virtual) looped-back ethernet ports, so the same source-dest tuple will be seen on multiple interfaces. I need a different tuple for the flow that should be NATed (so only that flow is NATed), so that is why I added the MARK rules and the mark field to the conn-track tuple.
As you noticed, my attempt to MASQ based on skb->mark was not really the right thing to do, so I changed it back to mask with -o [outgoing-device]. This still does not seem to trigger NAT to happen, and I had no luck figuring out why yesterday...
I found someone who is interested in considering doing this for hire, so I will be sending details and such to them. If that works out, hopefully there will be a patch from him for review sometime soon.
There's also the possibility I have totally mis-understood what is currently supported in the kernel and maybe I just need to adjust my rules or something like that....
Thanks,
Ben
Signature · 2 lines
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com
Quoted 8 lines · Ben Greear
Quoted 4 lines
Quoted 3 lines
iptables -A POSTROUTING -t nat -j MASQUERADE -m mark --mark 10001 iptables -t mangle -A PREROUTING -i eth1 -j MARK --set-mark 10001 iptables -t mangle -A PREROUTING -i eth2 -j MARK --set-mark 10001
Because otherwise it seems to me that there is only a single conn-tracking tuple for src -- dest, and it also seems to me that the conn-track entity has the should-we-NAT flags (in the 'status' bitfield).
A ct tuple, to my knowledge, constitutes (srcip, srcport, dstip, dstport). Whether a connection is actually NATed or nat, is for you to decide (MASQUERADE, SNAT, SAME, you name it.)
Quoted 2 lines
My scenario involves virtual routers (ie, routing tables with rules so that pkts hit certain routing tables)
ip rule add src 192.168.123.0/24 table 7
or ip rule add fwmark 999 table 666
for example would do (I assume you do that)
Quoted 5 lines
and sending packets through (virtual) looped-back ethernet ports, so the same source-dest tuple will be seen on multiple interfaces. I need a different tuple for the flow that should be NATed (so only that flow is NATed), so that is why I added the MARK rules and the mark field to the conn-track tuple.
Why is a different tuple needed?
Regards,
Jan
Quoted 5 lines · Jan Engelhardt
Quoted 3 lines · Ben Greear
Quoted 3 lines
Quoted 3 lines
iptables -A POSTROUTING -t nat -j MASQUERADE -m mark --mark 10001 iptables -t mangle -A PREROUTING -i eth1 -j MARK --set-mark 10001 iptables -t mangle -A PREROUTING -i eth2 -j MARK --set-mark 10001
Quoted 8 lines
Quoted 4 lines
Because otherwise it seems to me that there is only a single conn-tracking tuple for src -- dest, and it also seems to me that the conn-track entity has the should-we-NAT flags (in the 'status' bitfield).
A ct tuple, to my knowledge, constitutes (srcip, srcport, dstip, dstport). Whether a connection is actually NATed or nat, is for you to decide (MASQUERADE, SNAT, SAME, you name it.)
From looking at this method in net/ipv4/netfilter/nf_nat_core.c, I assume it stores NAT decision as well:
/* Do packet manipulations according to nf_nat_setup_info. */ unsigned int nf_nat_packet(struct nf_conn *ct,
enum ip_conntrack_info ctinfo, unsigned int hooknum, struct sk_buff **pskb)
{
...
/* Non-atomic: these bits don't change. */
if (ct->status & statusbit) {
struct nf_conntrack_tuple target;
/* We are aiming to look like inverse of other direction. */
nf_ct_invert_tuplepr(&target, &ct->tuplehash[!dir].tuple);Quoted 5 lines
Quoted 2 lines
My scenario involves virtual routers (ie, routing tables with rules so that pkts hit certain routing tables)
ip rule add src 192.168.123.0/24 table 7or ip rule add fwmark 999 table 666
Yes, I'm using commands similar to the first line. I have not tried using fwmark.
Quoted 8 lines
for example would do (I assume you do that)
Quoted 4 lines
and sending packets through (virtual) looped-back ethernet ports,
so the same source-dest tuple will be seen on multiple interfaces. I need a different tuple for the flow that should be NATed (so only that flow is NATed), so that is why I added the MARK rules and the
mark field to the conn-track tuple.Why is a different tuple needed?
Isn't the decision to NAT or not stored in the ct->status bitfield?
If so, then if I want to NAT some packets and not others, they must belong to different tuples.
If virtual router 1 is routing pkts from 1.1.1.1 to 2.2.2.2, and virtual router 2 is routing pkts from 1.1.1.1 to 2.2.2.2, and I only want to NAT pkts leaving virtual router 1, then I think I have to somehow force different ct tuples based on which virtual router the pkts are flowing through. I was trying to do this by MARKing packets entering a device in a particular virtual router and using the mark as part of the tuple....
Thanks,
Ben
Quoted 4 lines
Regards,
Jan
Signature · 2 lines
Ben Greear <[email protected]> Candela Technologies Inc http://www.candelatech.com
Quoted 7 lines · Ben Greear
Quoted 2 lines
Why is a different tuple needed?
Isn't the decision to NAT or not stored in the
ct->statusbitfield?If so, then if I want to NAT some packets and not others, they must belong to different tuples.
Why do you want to NAT some, and don't NAT others?
Quoted 7 lines
If virtual router 1 is routing pkts from 1.1.1.1 to 2.2.2.2, and virtual router 2 is routing pkts from 1.1.1.1 to 2.2.2.2, and I only want to NAT pkts leaving virtual router 1, then I think I have to somehow force different ct tuples based on which virtual router the pkts are flowing through. I was trying to do this by MARKing packets entering a device in a particular virtual router and using the mark as part of the tuple....
Ah in that case it might be easiest to write a netfilter target that does a tupleless NAT.
Jan
Hello,
Quoted 4 lines · Ben Greear
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
iptables -A POSTROUTING -t nat -j MASQUERADE -m mark --mark 10001 iptables -t mangle -A PREROUTING -i eth1 -j MARK --set-mark 10001 iptables -t mangle -A PREROUTING -i eth2 -j MARK --set-mark 10001
[...]
Quoted 7 lines
If virtual router 1 is routing pkts from 1.1.1.1 to 2.2.2.2, and virtual router 2 is routing pkts from 1.1.1.1 to 2.2.2.2, and I only want to NAT pkts leaving virtual router 1, then I think I have to somehow force different ct tuples based on which virtual router the pkts are flowing through. I was trying to do this by MARKing packets entering a device in a particular virtual router and using the mark as part of the tuple....
From what you wrote I understand you need a sort of separate connection tracking for each "virtual router". But I am afraid it cannot be done with MARK, because packet classification by the connection tracking takes place before the packet reaches the mangle PREROUTING (or OUTPUT) chain.