nftables rule optimization - evaluating efficiency

Hi,

Since it is possible to do the same thing using different rules, I am looking for the most optimal (low resource usage, high speed) way to write my rules.

Here is just a very simple test to compare the different approaches:

#!/usr/sbin/nft -f

flush ruleset

table ip6 t {
	# Goal: fast processing through early "exit"
	chain A {
		ip6 hoplimit != 255 return
		icmpv6 type  != 133 return
		icmpv6 code  != 0   return
		accept
	}

	# Goal: compact syntax
	chain B {
		icmpv6 type . icmpv6 code . ip6 hoplimit {
				133 . 0 . 255
			} \
			accept
		return
	}

	# Goal: no specific, using "general" syntax
	chain C {
		icmpv6 type 133 icmpv6 code 0 ip6 hoplimit 255 \
			accept
		return
	}
}

Looking at the output of 'nft -c --debug=netlink -f <this file>', it seems:

- chain A would work best (least instructions to verdict) if there is no match (e.g. if hoplimit is indeed not 255) but in all other cases the total number of instructions to be processed is greater

- chain B and C seem to have the same number of instructions but perhaps B would outperform C in case of multiple elements in the set (e.g. more types or codes to check)

Also, it is not clear what is the actual "load" of different instructions in terms of CPU cycles and memory, i.e. one instruction may look as "one" but may actually cost more than another 2, right?

What is the proper way to evaluate and optimize rule efficiency?

Am 02.07.24 um 21:03 schrieb William N.:

understanding what is your primary load and make final decisions as soon as possible

"ctstate RELATED,ESTABLISHED" hits 99% of all packages and after that you only handle new connections

when you have 99% of your load on port 443 and before the ACCEPT rule are 50 others rules for whatever services they are all evaluated

the same for drop/reject rules - on top the ones which hit most of teh time

you have rule counters how much packets every rule triggered

That particular problem was discussed in another thread:

https://marc.info/?t=171360284600001&r=1&w=2

A little side note: The capitalized words imply iptables syntax. In case I may somehow been misunderstood, please let me note just for the sake of clarity that the actual question is about nftables.

Sure. That is clear. The question is not how to order rules but how to write a rule in the most optimal way and to evaluate its performance, i.e. I would like to go beyond ordering and into the rule itself.

Counters don't tell how much system resources a rule consumes.

Are those difficult or somehow inappropriate questions? Or is there a better place to ask?

Indeed. It cannot be presumed that all instructions are equal in expense.

You are enquiring as to how to assess the relative efficiency of the bytecode instructions through the direct understanding of how they are processed by the nftables VM. Said VM is unique and - as far as I know - wholly undocumented. As regards difficulty, the intersection of people who follow the list and possess the necessary expertise can probably be conveyed quite comfortably by a single digit. It is a pity; I would also have been interested to see an informed reply but the absence of one wasn't altogether surprising to me.

Kerin Millar

Yes, but then again nft will try to make reasonable choices.

tcp dport { 22, 80 } will not allocate a huge hash table for two values. Is it faster than
tcp dport 22
tcp dport 80
?

No idea. And given the volume of bugs I don't really care too much.

The additional issue that its hard to give a useful answer.

Use 'perf record -a ... and pktgen' is likely a rather useless reply. And might even incorrect depending on what one wants to measure.

In general, its best to compact the ruleset as much as possible and use sets/maps/vmaps wherever possible.

I suppose it may actually be even more complicated if different CPU architectures (all capable of running nft) are considered in a comparison. So, for simplicity, I was hoping to be able to evaluate things at least on a single architecture.

Well, I guess I have touched a subject not interesting for many (hence not documented).

That's what I suspected too.