Skip to contentSkip to conversation

Performanace fo the Iptables Server

LARTC hare ram 4 participants 11 messages Aug 27, 2003 – Sep 2, 2003

lore
Share & tools
lore

hare ram

author
#0 lore raw

Hi all

I have RH 9.0
PIII with 550Mhz
512MB RAM

working fine with Iptables+TC+HTB

Now i want o make accounting system using ULOG

so i introduced to send all logs to Mysql
So i can make my own Graphs

iptables -A INPUT -j ULOG
iptables -A FORWARD -j ULOG
iptables -A OUTPUT -j ULOG

the Server is connected to Several Clients

does this server will have any Impact of performance or this PC can handle Easily

how can caluculate the PC configuration,

any help
hare

With

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#1 lore raw
Quoted 18 lines · hare ram

Hi all

I have RH 9.0
PIII with 550Mhz
512MB RAM

working fine with Iptables+TC+HTB

Now i want o make accounting system using ULOG

so i introduced to send all logs to Mysql
So i can make my own Graphs

iptables -A INPUT -j ULOG
iptables -A FORWARD -j ULOG
iptables -A OUTPUT -j ULOG

Quoted 6 lines

the Server is connected to Several Clients

does this server will have any Impact of performance or this PC can handle Easily

how can caluculate the PC configuration,

The needed configutation also depends on how many logs you will have and how you are going to proces them.
And do you really want to log everything? If you are interested in what's going thru your firewall, you can also log the connections. Or you can install ntop on the firewall.

Stef

Signature · 8 lines
[email protected]
 "Using Linux as bandwidth manager"
     http://www.docum.org/
     #lartc @ irc.oftc.net

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
author
#2 lore raw
In reply to hare ram

Hi Stef

thanks for the reply

yes i would like to do accounting the source IP so i need to log them all the traffic,
so i will know each IP how much transfering DATA

with my configuration, is there any performance issue
i may have more than thousands of passing through this firewall or qos server

so my PC will have any performance issue or, this configuration need any modification

thanks
hare
----- Original Message -----
From: "Stef Coene" <[email protected]>
To: "hare ram" <[email protected]>; <[email protected]>
Cc: <[email protected]>
Sent: Wednesday, August 27, 2003 9:00 PM
Subject: Re: [LARTC] Performanace fo the Iptables Server

Quoted 19 lines · hare ram
Quoted 18 lines

Hi all

I have RH 9.0
PIII with 550Mhz
512MB RAM

working fine with Iptables+TC+HTB

Now i want o make accounting system using ULOG

so i introduced to send all logs to Mysql
So i can make my own Graphs

iptables -A INPUT -j ULOG
iptables -A FORWARD -j ULOG
iptables -A OUTPUT -j ULOG

Quoted 7 lines
Quoted 6 lines

the Server is connected to Several Clients

does this server will have any Impact of performance or this PC can handle Easily

how can caluculate the PC configuration,

The needed configutation also depends on how many logs you will have and

how

Quoted 5 lines

you are going to proces them.
And do you really want to log everything? If you are interested in what's going thru your firewall, you can also log the connections. Or you can install ntop on the firewall.

Quoted 10 lines

Stef

--

[email protected]
"Using Linux as bandwidth manager"
http://www.docum.org/
#lartc @ irc.oftc.net

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#3 lore raw
In reply to hare ram
Quoted 8 lines · hare ram

Hi Stef

thanks for the reply

yes i would like to do accounting the source IP so i need to log them all the traffic,
so i will know each IP how much transfering DATA

Quoted 6 lines

with my configuration, is there any performance issue
i may have more than thousands of passing through this firewall or qos server

so my PC will have any performance issue or, this configuration need any modification

You don't have to log each packet to do accounting. If you create 1 iptables rule for each src address, you know how many packets and bytes that that src address transmitted.

iptables -A INPUT -s 192.168.1.0
iptables -A INPUT -s 192.168.1.1
iptables -A INPUT -s 192.168.1.2

iptables -L -v -n
Chain INPUT (policy DROP 47 packets, 5842 bytes)

 pkts bytes target     prot opt in     out     source   destination
    0     0            all  --  *      *       192.168.1.0          0.0.0.0/0
    0     0            all  --  *      *       192.168.1.1          0.0.0.0/0
    0     0            all  --  *      *       192.168.1.2          0.0.0.0/0

Stef

Signature · 8 lines
[email protected]
 "Using Linux as bandwidth manager"
     http://www.docum.org/
     #lartc @ irc.oftc.net

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#4 lore raw
In reply to hare ram
Quoted 7 lines · hare ram

Hi Stef

How about Source and Desitination port, and protocol wise bytes transfer ( like http, ftp, icmp, udp), if i want to track.

next limitation, if the Server goes down, the traffic will be zero, that will be another Limitation

Not true. If you record the counters each 5 minutes, you loose max 5 minutes of counters. You can record the difference between the 2 reads and discard negative values.

Quoted 5 lines

If i send the all the traffic to Mysql, so i can get the old data to.

may be you can correct me if, i may be wrong thinking

correct me with the best methods to do

If you really want it very detailed (src/dst - address/port), you indeed have to log it to mysql or so.
You can calulcate the number of updates you have to do mysql and simulate this on a test box.

Stef

Signature · 8 lines
[email protected]
 "Using Linux as bandwidth manager"
     http://www.docum.org/
     #lartc @ irc.oftc.net

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
author
#5 lore raw
In reply to hare ram

Hi Stef

How about Source and Desitination port, and protocol wise bytes transfer ( like http, ftp, icmp, udp), if i want to track.

next limitation, if the Server goes down, the traffic will be zero, that will be another Limitation

If i send the all the traffic to Mysql, so i can get the old data to.

may be you can correct me if, i may be wrong thinking

correct me with the best methods to do

hare

----- Original Message -----
From: "Stef Coene" <[email protected]>
To: "hare ram" <[email protected]>; <[email protected]>
Cc: <[email protected]>
Sent: Wednesday, August 27, 2003 10:47 PM
Subject: Re: [LARTC] Performanace fo the Iptables Server

Quoted 9 lines · hare ram
Quoted 8 lines

Hi Stef

thanks for the reply

yes i would like to do accounting the source IP so i need to log them all the traffic,
so i will know each IP how much transfering DATA

Quoted 7 lines
Quoted 6 lines

with my configuration, is there any performance issue
i may have more than thousands of passing through this firewall or qos server

so my PC will have any performance issue or, this configuration need any modification

You don't have to log each packet to do accounting. If you create 1

iptables

Quoted 1 line

rule for each src address, you know how many packets and bytes that that

src

Quoted 10 lines

address transmitted.

iptables -A INPUT -s 192.168.1.0 iptables -A INPUT -s 192.168.1.1 iptables -A INPUT -s 192.168.1.2

iptables -L -v -n
Chain INPUT (policy DROP 47 packets, 5842 bytes)

 pkts bytes target     prot opt in     out     source   destination
    0     0            all  --  *      *       192.168.1.0

0.0.0.0/0

Quoted 1 line
    0     0            all  --  *      *       192.168.1.1

0.0.0.0/0

Quoted 1 line
    0     0            all  --  *      *       192.168.1.2

0.0.0.0/0

Quoted 11 lines

Stef

--

[email protected]
"Using Linux as bandwidth manager"
http://www.docum.org/
#lartc @ irc.oftc.net

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

author
#6 lore raw
In reply to hare ram

Hi Stef

Thats right, if i make cron job run every 5min, so i will loose only 5min of data.

i was not understand
" You can record the difference between the 2 reads and discard negative values. "

you mean to say first 5min and next 5min bytes increasing ok if not i will discard and take new values you mean.

If i make the log to mysql, so it will be 24hours right
is there anything i can make every 5min dump the data to Mysql using ULOG

hare
----- Original Message -----
From: "Stef Coene" <[email protected]>
To: "hare ram" <[email protected]>; <[email protected]>
Cc: <[email protected]>
Sent: Wednesday, August 27, 2003 11:11 PM
Subject: Re: [LARTC] Performanace fo the Iptables Server

Quoted 4 lines · hare ram
Quoted 3 lines

Hi Stef

How about Source and Desitination port, and protocol wise bytes transfer

(

Quoted 5 lines
Quoted 4 lines

like http, ftp, icmp, udp), if i want to track.

next limitation, if the Server goes down, the traffic will be zero, that will be another Limitation

Not true. If you record the counters each 5 minutes, you loose max 5

minutes

Quoted 1 line

of counters. You can record the difference between the 2 reads and

discard

Quoted 8 lines

negative values.

Quoted 5 lines

If i send the all the traffic to Mysql, so i can get the old data to.

may be you can correct me if, i may be wrong thinking

correct me with the best methods to do

If you really want it very detailed (src/dst - address/port), you indeed

have

Quoted 2 lines

to log it to mysql or so.
You can calulcate the number of updates you have to do mysql and simulate

this

Quoted 12 lines

on a test box.

Stef

--

[email protected]
"Using Linux as bandwidth manager"
http://www.docum.org/
#lartc @ irc.oftc.net

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

#7 lore raw
In reply to hare ram
Quoted 8 lines · Stef Coene
Quoted 2 lines · hare ram

Hi Stef

If you really want it very detailed (src/dst - address/port), you indeed have to log it to mysql or so.
You can calulcate the number of updates you have to do mysql and simulate this on a test box.

As the mark you can put on the packet is quiet long you can use a mask system : [IP user][proto].
Next the script has only to split the information contained in the mark.

That's add a multiplicative factor to the number of rules but that's all. So no need to use mysql.
By the way you will need use connmark to track non linear protocol like ftp.

BR,

Signature · 2 lines
Eric Leblond
Nufw : http://www.nufw.org
#8 lore raw
In reply to hare ram
Quoted 11 lines · hare ram

Hi Stef

Thats right, if i make cron job run every 5min, so i will loose only 5min of data.

i was not understand
" You can record the difference between the 2 reads and discard negative values. "

you mean to say first 5min and next 5min bytes increasing ok if not i will discard and take new values you mean.

You can also store the difference between 2 readings. Otherwise your counters will increase forever.

Quoted 2 lines

If i make the log to mysql, so it will be 24hours right
is there anything i can make every 5min dump the data to Mysql using ULOG

You can try to redirect the ULOG to a named pipe or a shell script.

Named pipe example :
mkfifo test
echo 1 > test &
echo 2 > test 1
echo 3 > test &
cat test
3
2
1

[1]   Done                    echo 1 >test
[2]-  Done                    echo 2 >test
[3]+  Done                    echo 3 >test

So the numbers 1, 2 and 3 were cached in the named pipe untill the cat command was executed.

Stef

Signature · 8 lines
[email protected]
 "Using Linux as bandwidth manager"
     http://www.docum.org/
     #lartc @ irc.oftc.net

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
#9 lore raw
In reply to hare ram
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Hare" = hare ram <[email protected]> writes:

Hare> Hi Stef Thats right, if i make cron job run every 5min, so i Hare> will loose only 5min of data.

Hare> i was not understand " You can record the difference between Hare> the 2 reads and discard negative values. "

Hare> you mean to say first 5min and next 5min bytes increasing ok Hare> if not i will discard and take new values you mean.

Hare> If i make the log to mysql, so it will be 24hours right is Hare> there anything i can make every 5min dump the data to Mysql Hare> using ULOG

...or just use iptables -L -v -z to read the counters afresh each 5 minutes. -z will zero them out after displaying the current values.

-- Raju

Signature · 6 lines
Raj Mathur                [email protected]      http://kandalaya.org/
       GPG: 78D4 FC67 367F 40E2 0DD5  0FEF C968 D0EF CC68 D17F
                      It is the mind that moves
_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
author
#10 lore raw
In reply to hare ram

Hi Stef

regarding the same topic
if i use

iptables -A INPUT -s 192.168.2.1

iam able to capture packets going fro 192.168.2.1 to any address

how about any address coming to 192.168.2.1 this will be added in to same rule

or i need to add another Rule

iptables -A INPUT -d 192.168.2.1

hare
----- Original Message -----
From: "Stef Coene" <[email protected]>
To: "hare ram" <[email protected]>; <[email protected]>
Cc: <[email protected]>
Sent: Wednesday, August 27, 2003 10:47 PM
Subject: Re: [LARTC] Performanace fo the Iptables Server

Quoted 9 lines · hare ram
Quoted 8 lines

Hi Stef

thanks for the reply

yes i would like to do accounting the source IP so i need to log them all the traffic,
so i will know each IP how much transfering DATA

Quoted 7 lines
Quoted 6 lines

with my configuration, is there any performance issue
i may have more than thousands of passing through this firewall or qos server

so my PC will have any performance issue or, this configuration need any modification

You don't have to log each packet to do accounting. If you create 1

iptables

Quoted 1 line

rule for each src address, you know how many packets and bytes that that

src

Quoted 10 lines

address transmitted.

iptables -A INPUT -s 192.168.1.0 iptables -A INPUT -s 192.168.1.1 iptables -A INPUT -s 192.168.1.2

iptables -L -v -n
Chain INPUT (policy DROP 47 packets, 5842 bytes)

 pkts bytes target     prot opt in     out     source   destination
    0     0            all  --  *      *       192.168.1.0

0.0.0.0/0

Quoted 1 line
    0     0            all  --  *      *       192.168.1.1

0.0.0.0/0

Quoted 1 line
    0     0            all  --  *      *       192.168.1.2

0.0.0.0/0

Quoted 12 lines

Stef

--

[email protected]
"Using Linux as bandwidth manager"
http://www.docum.org/
#lartc @ irc.oftc.net

_______________________________________________
LARTC mailing list / [email protected]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/