ipporthash, ipportiphash, ipportnethash problems

I am not able to get *any* matches when I have the following combinations:

1. ipporthash: x.x.x.x,y [src,dst] or [dst,src] (i.e. source IP address and destination port and vice versa);
2. ipportiphash: x.x.x.x,y,z.z.z.z [src,dst,dst] or [dst,src,src] (i.e. source IP address, destination port and destination IP address and vice versa);
3. ipportnethash x.x.x.x,y,z.z.z.z/c [src,dst,dst] or [dst,src,src] (i.e. source IP address, destination port and destination subnet and vice versa);

When I split up the above sets like:

For case 1: match-set single-set src match-set single-port dst (and vice versa with src and dst reversed) - all in one line - i.e. creating two separate sets containing the appropriate src IP address and and dst ports respectively;
For case 2: match-set single-set src match-set double-set dst,dst (and vice versa with src and dst reversed) - all in one line - i.e. creating two separate sets containing the appropriate src IP address and another set containing the destination IP addresses *and* ports respectively; For case 3: match-set single-set src match-set double-net-set dst,dst (and vice versa with src and dst reversed) - all in one line - i.e. creating two separate sets containing the appropriate src IP address and another set containing the destination IP subnet addresses *and* ports respectively;

I was able to get a match! This leads me to believe that either xtables has a bug and can't handle mixed src,dst designations in the same set, or, I am doing something wrong. Which is it?

You should provide at least the following: kernel, iptables, ipset version numbers, the iptables rules and the (non)matching sets with the elements. Without exact details I cannot help.

Best regards,
Jozsef
-
E-mail : [email protected], [email protected] PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics

          H-1525 Budapest 114, POB. 49, Hungary

kernel-2.6.34.7-56.fc13.x86_64
iptables-1.4.7-2.fc13.x86_64
ipset 4.3
xtables-addons 1.29

For the purpose of this small test, I am initiating a connection from the local machine (10.1.2.7, network 10.1.2.0/24) to my other machine (10.1.1.3, network 10.1.1.0/24) over ssh (port 22) and ask iptables to match that and log it to the syslog.

1. IP,port hash (non)match (trying to match source IP 10.1.2.7 with destination port 22)

ipset -N double-set ipporthash --network 10.1.2.0/24
ipset -A double-set 10.1.2.7,22
ipset -L double-set
==================
Name: double-set
Type: ipporthash
References: 0
Header: from: 10.1.2.0 to: 10.1.2.255 hashsize: 1024 probes: 8 resize: 50 Members:
10.1.2.7,22
==================

iptables -A fw2net -p 6 -m set --match-set double-set src,dst -d 10.1.1.3 -g log0

==================
Chain fw2net (1 references)

pkts bytes target     prot opt in     out     source               destination        
   0     0 blackout   all  --  *      *       0.0.0.0/0            

0.0.0.0/0 ctstate INVALID,NEW
0 0 ACCEPT all -- * * 0.0.0.0/0
0.0.0.0/0 ctstate RELATED,ESTABLISHED
0 0 log0 tcp -- * * 0.0.0.0/0
10.1.1.3 [goto] match-set double-set src,dst
0 0 ACCEPT all -- * * 0.0.0.0/0
0.0.0.0/0 ==================

After initiating a connection to 10.1.1.3:22 (check for matches):

==================
Chain fw2net (1 references)

pkts bytes target     prot opt in     out     source               destination        
   7   520 blackout   all  --  *      *       0.0.0.0/0            

0.0.0.0/0 ctstate INVALID,NEW
11 1473 ACCEPT all -- * * 0.0.0.0/0
0.0.0.0/0 ctstate RELATED,ESTABLISHED
0 0 log0 tcp -- * * 0.0.0.0/0
10.1.1.3 [goto] match-set double-set src,dst
7 520 ACCEPT all -- * * 0.0.0.0/0
0.0.0.0/0 ==================

As you can see - there are NO matches!

2. IP,port,IP hash (non)match (trying to match source IP 10.1.2.7 with destination port 22 and destination IP 10.1.1.3)

ipset -N tripple-set ipportiphash --network 10.1.2.0/24
ipset -A tripple-set 10.1.2.7,22,10.1.1.3
ipset -L tripple-set
==================
Name: tripple-set
Type: ipportiphash
References: 0
Header: from: 10.1.2.0 to: 10.1.2.255 hashsize: 1024 probes: 8 resize: 50 Members:
10.1.2.7,22,10.1.1.3
==================

iptables -A fw2net -p 6 -m set --match-set tripple-set src,dst,dst -g log0

==================
Chain fw2net (1 references)

pkts bytes target     prot opt in     out     source               destination         
   0     0 blackout   all  --  *      *       0.0.0.0/0            

0.0.0.0/0 ctstate INVALID,NEW
0 0 ACCEPT all -- * * 0.0.0.0/0
0.0.0.0/0 ctstate RELATED,ESTABLISHED
0 0 log0 tcp -- * * 0.0.0.0/0
0.0.0.0/0 [goto] match-set tripple-set src,dst,dst
0 0 ACCEPT all -- * * 0.0.0.0/0
0.0.0.0/0 ==================

After initiating a connection to 10.1.1.3:22 (check for matches):

==================
Chain fw2net (1 references)

pkts bytes target     prot opt in     out     source               destination         
   6   412 blackout   all  --  *      *       0.0.0.0/0            

0.0.0.0/0 ctstate INVALID,NEW
10 1397 ACCEPT all -- * * 0.0.0.0/0
0.0.0.0/0 ctstate RELATED,ESTABLISHED
0 0 log0 tcp -- * * 0.0.0.0/0
0.0.0.0/0 [goto] match-set tripple-set src,dst,dst
6 412 ACCEPT all -- * * 0.0.0.0/0
0.0.0.0/0 ==================

As you can see - there are NO matches!

3. IP,port,IP/cidr (non)match (trying to match source IP 10.1.2.7 with destination port 22 and destination subnet 10.1.1.0/24)

ipset -N tripple-set ipportnethash --network 10.1.2.0/24 ipset -A tripple-set 10.1.2.7,22,10.1.1.0/24
ipset -L tripple-set
==================
Name: tripple-set
Type: ipportnethash
References: 0
Header: from: 10.1.2.0 to: 10.1.2.255 hashsize: 1024 probes: 8 resize: 50 Members:
10.1.2.7,22,10.1.1.0/24
==================

iptables -A fw2net -p 6 -m set --match-set tripple-set src,dst,dst -g log0

==================
Chain fw2net (1 references)

pkts bytes target     prot opt in     out     source               destination        
   0     0 blackout   all  --  *      *       0.0.0.0/0            

0.0.0.0/0 ctstate INVALID,NEW
0 0 ACCEPT all -- * * 0.0.0.0/0
0.0.0.0/0 ctstate RELATED,ESTABLISHED
0 0 log0 tcp -- * * 0.0.0.0/0
0.0.0.0/0 [goto] match-set tripple-set src,dst,dst
0 0 ACCEPT all -- * * 0.0.0.0/0
0.0.0.0/0 ==================

After initiating a connection to 10.1.1.3:22 (check for matches):

==================
Chain fw2net (1 references)

pkts bytes target     prot opt in     out     source               destination        
   6   412 blackout   all  --  *      *       0.0.0.0/0            

0.0.0.0/0 ctstate INVALID,NEW
11 1473 ACCEPT all -- * * 0.0.0.0/0
0.0.0.0/0 ctstate RELATED,ESTABLISHED
0 0 log0 tcp -- * * 0.0.0.0/0
0.0.0.0/0 [goto] match-set tripple-set src,dst,dst
6 412 ACCEPT all -- * * 0.0.0.0/0
0.0.0.0/0 ==================

As you can see - there are NO matches!

So, in all three scenarios above there are NO matches! As I already indicated - 1) I assume when I reverse the src and dst combinations in all three scenarios above the result would be the same; and 2) When I split up the above sets in the way I already indicated in my initial post on this thread I am getting the correct matches, which leads me to believe that either xtables has a bug and can't handle mixed src,dst designations in the same set, or, I am doing something wrong. Which is it?

[...]

Thanks for the complete description - I have just released ipset 4.4 which fixes this nasty bug.

Best regards,
Jozsef
-
E-mail : [email protected], [email protected] PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics

          H-1525 Budapest 114, POB. 49, Hungary

No worries - I was surprised to find it as ipporthash in particular should have been widely used in practice. As for ipportiphash and ipportnethash - there were recently included in the distribution due to another bug I found when compiling xtables (even though ipportiphash and ipportnethash were compiled the .ko files were never included in the final release making it impossible to use these two construct types).

A couple of questions and ideas for future releases:

1. Normally I used to get ipset as part of the xtables source. Is that still the case or do I have to download the ipset source and compile it separately to get the latest changes you made?
2. Related to that question - I used to get the xtables .src.rpm from which to compile the code and make an installation rpm. I don't seem to be able to do that any longer as all fedora repos are too slow to update the latest changes. Do you have a link from which I could get the latest changes in ipset/xtables in src.rpm which allows me to build an installation rpm. The reason I am asking this is because I am building an update images on all my machines (using kickstart) and need the installation rpms with the latest changes you made. Compiling this from source won't be possible as there is now way I could install this as part of the image-building process.

I think you mentioned that in future releases the restrictions on all *hash constructs (with regards to using /16 - B class - IP addresses) will be removed. Could you confirm that is still the case?

Would it be possible to include a new construct - IP,port,IP,port (without IP address restriction) - and how difficult is this to implement?

Similar question - would it be possible to include protocol (either by name or its number) as part of all available constructs for matching? The reason for this is that, as it stands, in order for me to match, say, "IP,port" hosts form my personal VPN I need to use two separate sets - one for UDP and one for TCP and I also to create two separate statements for iptables (one for tcp and one for udp), which is a bit of a waste and not very easy to administer. It would be nice if I could use "IP,protocol", "port,protocol", "IP,port,protocol", "IP,port,IP,protocol", "IP,port,IP/cidr,protocol" and so forth.

I'm baffled too for not discovering the bug much much earlier, by myself or by somebody else.

It's absolutely up to you: currently ipset can be installed alone, or as a part of xtables-addons. Jan updates xtables-addons quite reqularly.

No, I'm not aware of the source rpm for ipset. That's bad if there's one out there without refreshed.

Yes, definitely.

It's not difficult, but I see little value in such a type. The client source port (except for a few protocols in some cases) is random, so how could it be useful to mach for that? Or do you know a case where it can still be useful?

The next branch includes exactly that, i.e. the possibility to specify proto:port for the ipport, ipportip and ipportnet types. And IPv6 support too.

Best regards,
Jozsef
-
E-mail : [email protected], [email protected] PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics

          H-1525 Budapest 114, POB. 49, Hungary

I just taken the latest git from xtables-addons and it is not there.

This is a major headache for me for 2 reasons:

1. I use 'rpmbuild' to compile from source as it allows me to build rpms for different architectures (I compile on x86_64 for i686 with 'rpmbuild --target=xxx'). If I use 'ordinary' make there is no way (at least I don't know of any) I could compile for different architecture.

2. rpmbuild also allow me to pack the necessary binaries (executables & modules/.ko files) for the target architecture to be transferred to the target machine - conveniently in rpm which could be installed (together with the necessary dependencies) as part of the target image building process.

Even if I solve problem No 2 (i.e. by tar-ing the necessary modules and executables) I can't see a way to get past compiling for a different architecture (and resolve all the dependencies)! Any ideas?

On a side note, I just managed to built v4.4. in the last hour or so for x86_64 (haven't tested it yet) and had to readjust the prefix & lib directories in the Makefile (I use prefix=/ and libdir=/lib64) and installed the new version, but I can't do anything for my other machines (which are mostly i686-based and their image is built using kickstart).

I can give you of at least 2 uses based on my experience:

1. voip - normally I can restrict the range of source ports and ip address when a voip connection is initialed (usually ports 8000-8050 and I have to also define the right ip address to point to the outside network), but the only way I can do this at present is if I base this on UID in iptables (i.e. the id of the process this is run under) - I cannot use sets at all because I do not have the appropriate construct. If I had "IP,port,IP,port (and protocol!)" as a construct to work with I can just built the sets (as I use 4-5 different servers outside the network and they work on different ports too) and that would be it!

2. VPN - for the same reason: to increase security I normally bind the source port to a particular range and then if I had a "IP,port,IP,port (and protocol!)" construct to work with I will not be using UID in iptables to match, but will use this construct directly without any further need to do anything!

That's brilliant news! I take it you will be introducing protocol support for all the constructs, is that right? How long would it take before you release this?

Well give me a little leeway. It hasn't even been 24 hours since 4.4.

What is sad too is that Fedora has had so far no interest (or perhaps just no maintainer) to take it up into the distro.

You could use the openSUSE Build Service to produce packages for various distributions (including Fedora) and the two archs. It should be possible to include Fedora-specific blocks in the preexisting xtables-addons.spec too.

Yeah but rpmbuild is often run in a jailed environment (for various definitions of "jail"; ranging from simple chroot to Xen or KVM instances), and network operations don't exactly contribute to the idempotency of the build process that is sought.

But that should not really matter, because after a repo is done building, you can use the regular system tools (zypper, yum) to update, so that rpmbuild itself needs no network activity.

Well, I know you are busy working on the latest nf_conntrack patches so... (-;

I won't miss it that as much if I could prepare the rpms myself from source (via make with dist-rpm target for example) - I can live with that.

I have a policy that I built all system-sensitive packages from source on one or two of my machines locally, so doing that externally isn't really an option for me.

If I could compile and, at least, build the binaries in one single package (be it rpm, tar.gz or similar) so that they can be installed during my kickstart I would be happy with that.

As a last option I could compile the source on i686 environment (for i686 machine to be installed on), but I definitely need a complete package (in rpm, tar.gz etc - as I already pointed out) so that I could transfer it for my kickstart to unpack and install when building the update image. That's my dilemma at present.

Am I?

Talk Fedora into it and become a maintainer :p

You can put up a local BS instance. Of course, the same prerequisite as for raw building applies - you need some x86_64 box to begin building x86_64.

[...]

Sorry, what I provide is a generic, distribution-independent package. I'm aware that this can create a maintenance problem in a distribution-dependent environment, but I cannot help at that.

The present 4.x branch is in "maintenance" mode for me. I'll think on adding such a type to 5.x.

I'm going to release ipset 5.0 around the netfilter developer workshop this month.

Best regards,
Jozsef
-
E-mail : [email protected], [email protected] PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics

          H-1525 Budapest 114, POB. 49, Hungary

I have managed to find a solution, but it is pretty ugly! I can now package the compiled files (from BUILDROOT) into rpm, though what I will work on when I next have the time for it is to get the compilation process to execute in arch-independent environment. I will also fine-tune the rpm spec file and post it here so that whoever is interested in packaging xtables+ipset into rpm can use this file to prepare rpms instead of relying on the people from fedora who 'maintain' the repos to do it (I am still waiting for the 1.29 rpms to show up on fedora updates which is a disgrace really)!

If I can help you out with some testing I would gladly do it.

Another feature you may add to your list is support for port ranges in a single set element, like "IP,port-port" for example. You already have similar support for multiple IP addresses (when subnets are used) - port ranges is another useful feature to have. One example where I can use this is when defining 'high-' (or unprivileged) ports - currently I 'solve' this particular problem with enrolling a set consisting of 1024 elements containing ports 0-1023 and then specifying a negative match (i.e. not privileged) on that set, which is not very convenient.

Superb news!

OK, I quickly tested the new version with my previous test on the ssh port and it all works now, though I found some other (minor) issues:

When I do 'make IP_NF_SET_MAX=xx IP_NF_SET_HASHSIZE=xx' and then do 'make install' (as root) both values I have given during my first 'make' were completely disregarded.

If I do 'make IP_NF_SET_MAX=xx IP_NF_SET_HASHSIZE=xx' and 'make IP_NF_SET_MAX=xx IP_NF_SET_HASHSIZE=xx install' I can see that at least the default hash size has also been disregarded (built a set without specifying --hashsize and when I executed 'ipset -L' the hashsize was 1024 and not the number I have given during make). I cannot check whether IP_NF_SET_MAX has also been disregarded (don't know how to do it!).

Is this a known issue or have I done something wrong?

IP_NF_SET_MAX is honored. Run "make ... V=1" and you see that the value is passed to CONFIG_IP_NF_SET_MAX which is used up directly in kernel/ip_set.c. However, IP_NF_SET_HASHSIZE is forgotten and not referenced in the userspace code, so setting the default hashsize at compiling time does not work yet.

Best regards,
Jozsef
-
E-mail : [email protected], [email protected] PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics

          H-1525 Budapest 114, POB. 49, Hungary

So I should not use IP_NF_SET_HASHSIZE for the time being until it is fixed, right?

Yes, because currently it's ignored. In this week I'm going to fix it in the git repository but won't release a new version just for this.

Best regards,
Jozsef
-
E-mail : [email protected], [email protected] PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics

          H-1525 Budapest 114, POB. 49, Hungary

There is another issue I found when using ipmap sets: when I execute, for example, "ipset -N port-map ipmap --from 10 --to 30000" and then add an element "ipset -A port-map 20" the two statements are accepted without any error given (they shouldn't be as the map defined is an IP map, not a port map and the values submitted are numbers, not IP addresses)!

When I issue "ipset -L port-map" I get:

Header: from: 0.0.0.10 to: 0.0.117.48
members:
0.0.0.20

Is this deliberate or a bug?

It's deliberate.

The numbers are treated as modulo 256.

30,000 / 256 = 117.1875
.1875 * 256 = 48

Thus the whole number parts become segments of the quad: 117.48

Right, thanks for clearing that out - I now know how to convert my ip-to-country numbers to IP addresses ;-)

Whatever name has got the set, the type what counts. So if it is defined as an ipmap type, then the system tries to interpret the input parameters as IP addresses. "10" and "20", etc are strange-looking, but valid IP addresses :-)

Best regards,
Jozsef
-
E-mail : [email protected], [email protected] PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics

          H-1525 Budapest 114, POB. 49, Hungary