Packet marking for ingress shapping and NET
Hi,
I have typical situation, local LAN with private addresses, translated via NAT to internet. I need to shape ingress traffic (from internet to local LAN) in several HTB queues accorting to destination (private not public) IP. So I need mark packets to divide them to corresponding queue. According to http://www.docum.org/stef.coene/qos/kptd/ I thing I have only one way how to do it, because MARK in PREROUTING is before (de)NAT
PREROUTING (de)NAT
V
FORWARD marking
V
FORWARD put to IMQ
V
HTB shapping
V
routing decision
V
output interface
It has a small problem. After PREROUTING some packets are routed to INPUT (packets intended for this machine for local processes)
Does exists solution how to NAT and MARK in PREROUTING, but in this order?
Patrick
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Am Monday 10 May 2004 12:10 schrieb Patrick Spousta:
Quoted 1 line
So I need mark packets to divide them to corresponding queue.
That's all right so far. But the qdisc that shapes incoming traffic usually sits on your LAN device.
Quoted 5 lines
It has a small problem. After PREROUTING some packets are routed to INPUT (packets intended for this machine for local processes)
Does exists solution how to NAT and MARK in PREROUTING, but in this order?
I'm not sure if I understand what you want to do. Why do you want to mark INPUT packets? There is no qdisc/class to put them in. As for shaping incoming traffic that doesn't get forwarded to the LAN, I haven't found a proper solution to do that yet. So all I can do is make sure that the router doesn't produce any traffic (e.g. don't put a Webserver or similar services on it).
Andreas
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quoted 21 lines · Patrick Spousta
Hi,
I have typical situation, local LAN with private addresses, translated via NAT to internet. I need to shape ingress traffic (from internet to local LAN) in several HTB queues accorting to destination (private not public) IP. So I need mark packets to divide them to corresponding queue. According to http://www.docum.org/stef.coene/qos/kptd/ I thing I have only one way how to do it, because MARK in PREROUTING is before (de)NATPREROUTING (de)NAT
V
FORWARD marking
V
FORWARD put to IMQ
V
HTB shapping
V
routing decision
V
output interface
Quoted 5 lines
It has a small problem. After PREROUTING some packets are routed to INPUT (packets intended for this machine for local processes)
Does exists solution how to NAT and MARK in PREROUTING, but in this order?
If you really need to shape for local and forwarded on ingress then you use IMQ + the IMQ NAT patch and use u32 to filter on dst IP (if you are masquerading a dynamic IP mark LAN traffic and use default for local).
If the traffic to local is not "bulk" ie just dns or ntp etc. then it would be less trouble to ignore it and just shape on your LAN facing interface marking on dst in postrouting or using u32 on dst - both should work, you may want to exclude traffic from server to LAN.
Andy.
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] Packet marking for ingress shapping and NAT
In reply to Patrick SpoustaQuoted 7 lines · Andreas Klauer
Am Monday 10 May 2004 12:10 schrieb Patrick Spousta:
Quoted 1 line
So I need mark packets to divide them to corresponding queue.
That's all right so far. But the qdisc that shapes incoming traffic usually sits on your LAN device.
I think you are wrong. Shapping can sits on all interfaces, physical and logical. IMQ is logical interface.
Quoted 10 lines
Quoted 7 lines
It has a small problem. After PREROUTING some packets are routed to INPUT (packets intended for this machine for local processes)
Does exists solution how to NAT and MARK in PREROUTING, but in this order?
I'm not sure if I understand what you want to do. Why do you want to mark
My linux box has 1 WAN interface (to ISP with public IP address) and 3 LAN interfaces (with private IP addresses). Only way how to shape incoming traffic is use IMG device because shapping is provided on egress. I understood that packet 'path' looks like this
eth0 -> kernel -> IMQ -> kernel -> ethX
^^^ here is 'egress' where I can do shapping.
But I need divide traffic to the corresponding queues according to real destination IP. Maybe I don't need marking, I can only use tc filter, but it must be done in place where packet has real destination IP, ie. behind (de)NAT.
To IMQ 'interface' I put packets via iptables. Ideal in PREROUTING chain, but I think I can use only 'mange' table and that is before 'nat' :-( So now I'm using FORWARD chain but local traffic is going outside of shapping path
Quoted 5 lines
INPUT packets? There is no qdisc/class to put them in. As for shaping incoming traffic that doesn't get forwarded to the LAN, I haven't found a proper solution to do that yet. So all I can do is make sure that the router doesn't produce any traffic (e.g. don't put a Webserver or similar services on it).
But it isn't goor solution :-(
Patrick
Quoted 7 lines
Andreas
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hi
Quoted 7 lines · Andy Furniss
Quoted 3 lines
Does exists solution how to NAT and MARK in PREROUTING, but in this order?
If you really need to shape for local and forwarded on ingress then you use IMQ + the IMQ NAT patch and use u32 to filter on dst IP (if you are masquerading a dynamic IP mark LAN traffic and use default for local).
It sounds good, but can you be more conrete?
Quoted 2 lines
If the traffic to local is not "bulk" ie just dns or ntp etc. then it
Sometimes it is 'bulk' - FTP etc.
Quoted 1 line
would be less trouble to ignore it and just shape on your LAN facing
Sorry, I forgot write that I have 3 LAN interfaces, so IMQ is only way how to do it.
Thanks
Patrick
Quoted 5 lines
interface marking on dst in postrouting or using u32 on dst - both should work, you may want to exclude traffic from server to LAN.
Andy.
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quoted 8 lines · Patrick Spousta
Hi
Quoted 4 lines · Andy Furniss
Quoted 3 lines
Does exists solution how to NAT and MARK in PREROUTING, but in this order?
Quoted 7 lines
Quoted 4 lines
If you really need to shape for local and forwarded on ingress then you use IMQ + the IMQ NAT patch and use u32 to filter on dst IP (if you are masquerading a dynamic IP mark LAN traffic and use default for local).
It sounds good, but can you be more conrete?
Using IMQ generally or a script? - mine is pretty lame, unfinished and needs netfilter patches, though I suppose it could give an indication of what to do - I am still learning HTB myself, but have got sidetracked at the moment playing with esfq.
There is a new imq website www.linuximq.net from which you should be able to get imq working for whatever kernel you use. I don't know if they include the NAT patch yet - but it's only a couple of lines and should apply OK. If you happen to use 2.4.24 I can give urls for the patches I use.
Andy.
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quoted 5 lines · Patrick Spousta
It looks working fine :-) I never found any details about IMQ and NAT patch, it looks that packet processing in kernel has path
| PREROUTING chain |
input interface -> contrack -> mangle -> nat -> imq
So which IMQ did you use - did you need to patch for NAT (there are different versions about)
Quoted 7 lines
ESFQ works fine, but only for ingress shapping over imq and NAT with destination hash (= download on private IPs). I'm trying to setup ESFQ on egress shaping for traffic from private to public IPs with source hash (upload from private) but qdisc sits after NAT, ie. packets source addresses are always the same public IP of external (wan) interface :-(
I try to use imq for egress shapping (on POSTROUTING chain), may it helps
I don't think IMQ will help - but you can mark local src in postrouting mangle OK. If you really want to use esfq, someone posted a patch on here a while back which made esfq hash on fwmark.
Andy.
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] Packet marking for ingress shapping and NAT
In reply to Patrick SpoustaQuoted 11 lines · Andy Furniss
Quoted 5 lines · Patrick Spousta
It looks working fine :-) I never found any details about IMQ and NAT patch, it looks that packet processing in kernel has path
| PREROUTING chain |
input interface -> contrack -> mangle -> nat -> imqSo which IMQ did you use - did you need to patch for NAT (there are different versions about)
Now I'm using patches from http://www.digriz.org.uk/jdg-qos-script/
(latest version which contains patches for IMQ, IMQ+NAT, ESFQ, IPP2P, CONNMARK, also recompiledtc andlibrarie for iptables, nice package) on kernel 2.4.25. It works good.
Quoted 8 lines
Quoted 8 lines
ESFQ works fine, but only for ingress shapping over imq and NAT with destination hash (= download on private IPs). I'm trying to setup ESFQ on egress shaping for traffic from private to public IPs with source hash (upload from private) but qdisc sits after NAT, ie. packets source addresses are always the same public IP of external (wan) interface :-(
Quoted 4 lines
Quoted 1 line
I try to use imq for egress shapping (on POSTROUTING chain), may it helps
I don't think IMQ will help - but you can mark local src in postrouting
Do you mean manualy configured marking for many, many IP addresses? I think it isn't right way :-( I like ESFQ for it's source or destination hash because I don't need to setup any filters/markers for those IPs, ESFQ creats it's own queues for each IP.
In POSTROUTING chain it normaly look like this
... -> mangle -> nat -> imq -> (output interface)
I don't understand C language so I don't understand IMQ+NAT patch, but I'll try to use imq for egress shapping. Maybe the patch is working identically on PRE i POST chains.
Patrick
Quoted 8 lines
mangle OK. If you really want to use esfq, someone posted a patch on here a while back which made esfq hash on fwmark.
Andy.
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] Packet marking for ingress shapping and NAT
In reply to Patrick SpoustaPatrick Spousta wrote:
<snip>
Quoted 7 lines
Quoted 1 line
I don't think IMQ will help - but you can mark local src in postrouting
Do you mean manualy configured marking for many, many IP addresses? I think it isn't right way :-( I like ESFQ for it's source or destination hash because I don't need to setup any filters/markers for those IPs, ESFQ creats it's own queues for each IP.
Yes I agree - not nice for your setup, though personally the thing I don't like about using esfq on src/dst is you loose per tcp fairness - it was less than a year ago that I was on 56K and anyone with high latency downloading from you will get their already small bandwidth squeezed out by the low latency downloaders.
Maybe it's less noticable/of an issue for your big setup anyway.
Quoted 8 lines
In POSTROUTING chain it normaly look like this
... -> mangle -> nat -> imq -> (output interface)
I don't understand C language so I don't understand IMQ+NAT patch, but I'll try to use imq for egress shapping. Maybe the patch is working identically on PRE i POST chains.
I only just started getting into C myself - (used motorolla 68000 assembly years ago on an atari ST - these "high level" languages are much trickier :-) ).
I can see that the patch is trivial and needs an understanding of netfilter hooks more than C.
I just tested with u32 to double confirm what I knew really - the patch only affects prerouting hooks.
Then knowing nothing about netfilter decided to have a go at changing the egress hook - it appears to be working as expected.
One caveat - some people have reported stability problems using postrouting IMQ, probably to do with dropping locally generated traffic. I and others don't, but then I don't leave my gateway PC up that long. Grepping my logs :
Sent 3744702472 bytes 5539814 pkts (dropped 354902, overlimits 11722774)
is the most I can see (most of the drops are locally generated packets ie. bittorrent running on the shaping PC).
If you wan't to give my blind and possibly stupid hack a go you just need to change near the top of drivers/net/imq.c so it looks like -
static struct nf_hook_ops imq_egress_ipv4 = {
{ NULL, NULL},
imq_nf_hook,
PF_INET,
NF_IP_POST_ROUTING,
NF_IP_PRI_NAT_SRC - 1
};Rather than -
static struct nf_hook_ops imq_egress_ipv4 = {
{ NULL, NULL},
imq_nf_hook,
PF_INET,
NF_IP_POST_ROUTING,
NF_IP_PRI_LAST
};You could do the same for the egress ipv6 bits below it aswell.
If you still have your source tree intact and use modules cd to top dir in kernel tree do
make SUBDIRS=drivers/net modules
which should build a new imq.o in drivers/net
Backup /lib/modules/[your version]/kernel/drivers/net/imq.o
and replace with new one.
Take down shaping and modprobe -r imq (check it's gone with lsmod) and restart shaper.
It's a bit of a pain that imq is unstable for some anyway - you won't know whoose fault it is if/when it crashes :-)
Andy.
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/