tracking usage by mac address
Hello,
I have been reading this list for several months, and I've really enjoyed learning all that I have, thank you everybody for the opportunity to listen:)
Ultimately I am hoping to track the bandwidth usage of about 50 client computers through my router based on their MAC address. I understand that by simply writing a rule that does nothing to the packet, such as 'iptables -A FORWARD -m <mac address>' I can parse the netfilter log and find out what I need. This seems rather convoluted though - getting netfilter to create a basically human readable log file, and then parsing it.
All of the network traffic is passing through unmanaged switches until finally hitting the interface on the router.
I'm sure this must have been done by many others before, so could anybody give me some idea of what the most common way to handle this situation would be?
I appreciate any input.
Henry Baxter
Quoted 6 lines · Henry Baxter
Ultimately I am hoping to track the bandwidth usage of about 50 client computers through my router based on their MAC address. I understand that by simply writing a rule that does nothing to the packet, such as 'iptables -A FORWARD -m <mac address>' I can parse the netfilter log and find out what I need.
How about:
iptables -A FORWARD -m mac --mac-source <mac address 1> -j LOG " CLIENT_1 "
iptables -A FORWARD -m mac --mac-source <mac address 2> -j LOG " CLIENT_2 "
Then when you need a report, just run:
iptables -L -nvx
Now the caveat is this will only show you outbound traffic, not inbound. In other words, there is no "--mac-destination" option so you can't log reply packets based on MAC address. Kind of a bummer in your case because if these clients are mostly downloading data, that's the direction that's going to see the most bandwidth usage.
Two options:
1) Log by IP instead of MAC (maybe hand the clients fixed IP's via DHCP)
2) Use ebtables instead of iptables (90% certain it will work but have not tried it)
HTH,
C
Quoted 7 lines · Henry Baxter
Ultimately I am hoping to track the bandwidth usage of about 50 client computers through my router based on their MAC address. I understand that by simply writing a rule that does nothing to the packet, such as 'iptables -A FORWARD -m <mac address>' I can parse the netfilter log and find out what I need. This seems rather convoluted though - getting netfilter to create a basically human readable log file, and then parsing it.
You could also use ULOG and the ulog-acctd from
http://alioth.debian.org/projects/pkg-ulog-acctd/
(if you want to use this on RedHat/Fedora, I could send you my RPM I made from it)
This also generates a somewhat user-readable log file which you'd need to parse, but it can aggregate several packets (thus reducing the size of the log file) and generate a Cisco-compatible traffic log file. Parsers for that should not be hard to find.
Greetings,
Torsten
You can use ip_conntrack this way
iptables -t nat -A PREROUTING -i $LANIF -s $LANIPCLIENT1 -m mac --mac-source $CLIENT1_MAC_ADDRESS -j ACCEPT
iptables -t nat -A PREROUTING -i $LANIF -s $LANIPCLIENT2 -m mac --mac-source $CLIENT2_MAC_ADDRESS -j ACCEPT
......
iptables -t nat -A PREROUTING -i $LANIF -j DROP
Then
iptables -A FORWARD -s $LANIPCLIENT1 -i $LANIF -o $INETIF #upload iptables -A FORWARD -d $LANIPCLIENT1 -d $LANIF -o $INETIF #download
and to see the traffic, use
iptables -L FORWARD -nv
and look for those 2 rules. You can add a -j LOG target, or whatever. This way you will see the client's download based on his mac, because you allow beginning streams only with those macs
Quoted 24 lines · Torsten Luettgert
Quoted 7 lines · Henry Baxter
Ultimately I am hoping to track the bandwidth usage of about 50 client computers through my router based on their MAC address. I understand that by simply writing a rule that does nothing to the packet, such as 'iptables -A FORWARD -m <mac address>' I can parse the netfilter log and find out what I need. This seems rather convoluted though - getting netfilter to create a basically human readable log file, and then parsing it.
You could also use ULOG and the ulog-acctd from
http://alioth.debian.org/projects/pkg-ulog-acctd/(if you want to use this on RedHat/Fedora, I could send you my RPM I made from it)
This also generates a somewhat user-readable log file which you'd need to parse, but it can aggregate several packets (thus reducing the size of the log file) and generate a Cisco-compatible traffic log file. Parsers for that should not be hard to find.
Greetings,
Torsten
Signature · 1 line
Bla bla
Quoted 17 lines · Henry Baxter
Hello,
I have been reading this list for several months, and I've really enjoyed learning all that I have, thank you everybody for the opportunity to listen:)
Ultimately I am hoping to track the bandwidth usage of about 50 client computers through my router based on their MAC address. I understand that by simply writing a rule that does nothing to the packet, such as 'iptables -A FORWARD -m <mac address>' I can parse the netfilter log and find out what I need. This seems rather convoluted though - getting netfilter to create a basically human readable log file, and then parsing it.
All of the network traffic is passing through unmanaged switches until finally hitting the interface on the router.
Quoted 7 lines
I'm sure this must have been done by many others before, so could anybody give me some idea of what the most common way to handle this situation would be?
I appreciate any input.
Henry Baxter
If you don't have a big number of users you can do something like this:
iptables -N MACSTATS
iptables -A INPUT -j MACSTATS
iptables -A OUTPUT -j MACSTATS
iptables -A FORWARD -j MACSTATS
iptables -A MACSTATS -m mac --mac-source $CLIENT1_MAC_ADDRESS -j RETURN iptables -A MACSTATS -m mac --mac-source $CLIENT2_MAC_ADDRESS -j RETURN
...
So you can read the data transfered by each client with the command:
iptables -L MACSTATS -nv
More or less this is what we do in our bastion-firewall-stats module from our bastion-firewall GPL firewall, but we extract the counters with C code to put it in a rrdtool database and then create graphs with the data. If need code you can look at the source code of this addon from our firewall.
Signature · 12 lines
Jose Maria Lopez Hernandez Director Tecnico de bgSEC [email protected] bgSEC Seguridad y Consultoria de Sistemas Informaticos http://www.bgsec.com ESPAÑA The only people for me are the mad ones -- the ones who are mad to live, mad to talk, mad to be saved, desirous of everything at the same time, the ones who never yawn or say a commonplace thing, but burn, burn, burn like fabulous yellow Roman candles. -- Jack Kerouac, "On the Road"
Well, i don't know if you want to log EVERYTHING.
Remember ip_conntrackworkson streams, so you can log only NEW packets. I have like 90 rules with -m mac like those i said before + several port forwarding, on a P2 450Mhz, 100mbit internet connections, used a lot, almoust all the time at 11MB/s at upload (exactly where those rules aremostly hitted), and top says the sys load is arround 40% at most when i have full bandwith in use, but i think it is not because of the netfilter, but the PCI usage. Traffic at 50% usually needs much less CPU, like 5-10%. I also have many other rules for SYN scan limiting, bandwith counting, and so on.
Quoted 62 lines · Jose Maria Lopez
Quoted 17 lines · Henry Baxter
Hello,
I have been reading this list for several months, and I've really enjoyed learning all that I have, thank you everybody for the opportunity to listen:)
Ultimately I am hoping to track the bandwidth usage of about 50 client computers through my router based on their MAC address. I understand that by simply writing a rule that does nothing to the packet, such as 'iptables -A FORWARD -m <mac address>' I can parse the netfilter log and find out what I need. This seems rather convoluted though - getting netfilter to create a basically human readable log file, and then parsing it.
All of the network traffic is passing through unmanaged switches until finally hitting the interface on the router.
Quoted 7 lines
I'm sure this must have been done by many others before, so could anybody give me some idea of what the most common way to handle this situation would be?
I appreciate any input.
Henry Baxter
If you don't have a big number of users you can do something like this:
iptables -N MACSTATS
iptables -A INPUT -j MACSTATS
iptables -A OUTPUT -j MACSTATS
iptables -A FORWARD -j MACSTATS
iptables -A MACSTATS -m mac --mac-source $CLIENT1_MAC_ADDRESS -j RETURN iptables -A MACSTATS -m mac --mac-source $CLIENT2_MAC_ADDRESS -j RETURN
...So you can read the data transfered by each client with the command:
iptables -L MACSTATS -nvMore or less this is what we do in our bastion-firewall-stats module from our bastion-firewall GPL firewall, but we extract the counters with C code to put it in a rrdtool database and then create graphs with the data. If need code you can look at the source code of this addon from our firewall.
--
Jose Maria Lopez Hernandez
Director Tecnico de bgSEC
[email protected]
bgSEC Seguridad y Consultoria de Sistemas Informaticos
http://www.bgsec.com
ESPAÑAThe only people for me are the mad ones -- the ones who are mad to live, mad to talk, mad to be saved, desirous of everything at the same time, the ones who never yawn or say a commonplace thing, but burn, burn, burn like fabulous yellow Roman candles.
-- Jack Kerouac, "On the Road"
Signature · 1 line
Bla bla
Thank you Jose, I'm going to go with parsing the log with C code which I wouldn't mind writing - but if you could point me to your source, that would be very helpful. From the sounds of your setup George it should work great for us here (a tenth of your bandwidth usage!).
This mailing list rocks
Henry Baxter
Quoted 85 lines · George Alexandru Dragoi
Well, i don't know if you want to log EVERYTHING.
Rememberip_conntrackworksonstreams, so you can log only NEW packets. I have like 90 rules with -m mac like those i said before + several port forwarding, on a P2 450Mhz, 100mbit internet connections, used a lot, almoust all the time at 11MB/s at upload (exactly where those rules aremostly hitted), and top says the sys load is arround 40% at most when i have full bandwith in use, but i think it is not because of the netfilter, but the PCI usage. Traffic at 50% usually needs much less CPU, like 5-10%. I also have many other rules for SYN scan limiting, bandwith counting, and so on.Quoted 67 lines · Jose Maria Lopez
Quoted 17 lines · Henry Baxter
Hello,
I have been reading this list for several months, and I've really enjoyed learning all that I have, thank you everybody for the opportunity to listen:)
Ultimately I am hoping to track the bandwidth usage of about 50 client computers through my router based on their MAC address. I understand that by simply writing a rule that does nothing to the packet, such as 'iptables -A FORWARD -m <mac address>' I can parse the netfilter log and find out what I need. This seems rather convoluted though - getting netfilter to create a basically human readable log file, and then parsing it.
All of the network traffic is passing through unmanaged switches until finally hitting the interface on the router.
Quoted 9 lines
I'm sure this must have been done by many others before, so could anybody give me some idea of what the most common way to handle this situation would be?
I appreciate any input.
Henry Baxter
If you don't have a big number of users you can do something like this:
iptables -N MACSTATS
iptables -A INPUT -j MACSTATS
iptables -A OUTPUT -j MACSTATS
iptables -A FORWARD -j MACSTATS
iptables -A MACSTATS -m mac --mac-source $CLIENT1_MAC_ADDRESS -j RETURN iptables -A MACSTATS -m mac --mac-source $CLIENT2_MAC_ADDRESS -j RETURN
...So you can read the data transfered by each client with the command:
iptables -L MACSTATS -nvMore or less this is what we do in our bastion-firewall-stats module from our bastion-firewall GPL firewall, but we extract the counters with C code to put it in a rrdtool database and then create graphs with the data. If need code you can look at the source code of this addon from our firewall.
--
Jose Maria Lopez Hernandez
Director Tecnico de bgSEC
[email protected]
bgSEC Seguridad y Consultoria de Sistemas Informaticos
http://www.bgsec.com
ESPAÑAThe only people for me are the mad ones -- the ones who are mad to live, mad to talk, mad to be saved, desirous of everything at the same time, the ones who never yawn or say a commonplace thing, but burn, burn, burn like fabulous yellow Roman candles.
-- Jack Kerouac, "On the Road"
Quoted 6 lines · Henry Baxter
Thank you Jose, I'm going to go with parsing the log with C code which I wouldn't mind writing - but if you could point me to your source, that would be very helpful. From the sounds of your setup George it should work great for us here (a tenth of your bandwidth usage!).
This mailing list rocks
You can find the source code at our downloads web page:
http://www.bgsec.com/downloads.html
or at the sourceforge web site:
http://bastionfirewall.sourceforge.net
the module you could use it's named
bastion-firewall-stats-1.0.src.tar.bz2
But it's you can also look at google for the Querying Libiptc HOWTO, that was the document we used to write our code. Just have in mind that it has a big bug, because it allocates memory when it open the chain to read the counters but it doesn't free the memory. If you use the code in the examples you must do this after you read the counters:
iptc_free(&h);
If you don't do it your program starts to eat memory and grows and grows and grows until it uses all the memory in the system.
We have sent an email to the author of the Howto, but we have not received any response yet. Hope he reads it's mail and the new version of the HOWTO adds this code.
Signature · 12 lines
Jose Maria Lopez Hernandez Director Tecnico de bgSEC [email protected] bgSEC Seguridad y Consultoria de Sistemas Informaticos http://www.bgsec.com ESPAÑA The only people for me are the mad ones -- the ones who are mad to live, mad to talk, mad to be saved, desirous of everything at the same time, the ones who never yawn or say a commonplace thing, but burn, burn, burn like fabulous yellow Roman candles. -- Jack Kerouac, "On the Road"
Quoted 11 lines · George Alexandru Dragoi
Well, i don't know if you want to log EVERYTHING.
Rememberip_conntrackworksonstreams, so you can log only NEW packets. I have like 90 rules with -m mac like those i said before + several port forwarding, on a P2 450Mhz, 100mbit internet connections, used a lot, almoust all the time at 11MB/s at upload (exactly where those rules aremostly hitted), and top says the sys load is arround 40% at most when i have full bandwith in use, but i think it is not because of the netfilter, but the PCI usage. Traffic at 50% usually needs much less CPU, like 5-10%. I also have many other rules for SYN scan limiting, bandwith counting, and so on.
Obviously our system it's useful for a not huge set of rules, we use it for a per service basis, not per IP or MAC. We have been using it with a big number of rules (services) and it works like a charm, without slowing the system, but if you have a lot of MACs our system can be surely a bad idea.
Signature · 12 lines
Jose Maria Lopez Hernandez Director Tecnico de bgSEC [email protected] bgSEC Seguridad y Consultoria de Sistemas Informaticos http://www.bgsec.com ESPAÑA The only people for me are the mad ones -- the ones who are mad to live, mad to talk, mad to be saved, desirous of everything at the same time, the ones who never yawn or say a commonplace thing, but burn, burn, burn like fabulous yellow Roman candles. -- Jack Kerouac, "On the Road"