Skip to contentSkip to conversation

[5.0] ip route show table default: "Error: ipv4: FIB table does not exist."

BUG [email protected] 5 participants 21 messages Mar 23, 2019 – May 17, 2019

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

After upgrading iproute2 from 4.20 to 5.0 the following error occurs:

$ ip route show table default
Error: ipv4: FIB table does not exist.
Dump terminated

The command works for all tables other than 'default' one. It seems related to this commit[1]

I also saw "Error: ipv4: MR table does not exist." message in logs related to this commit[2] but don't know exact command to reproduce it. I've seen some fixups[3] for mentioned commits and wonder it they weren't complete.

I reproduced it on Linux 4.20.17 and 5.0.3.

[1] https://github.com/torvalds/linux/commit/18a8021a7be3207686851208f91a2f105b2d4703#diff-04a14e4f51765994f87e7e5e7681d0e1R861 <https://github.com/torvalds/linux/commit/18a8021a7be3207686851208f91a2f105b2d4703#diff-04a14e4f51765994f87e7e5e7681d0e1R861>

[2] https://github.com/torvalds/linux/commit/cb167893f41e21e6bd283d78e53489289dc0592d#diff-9900db808ce5e5dd24a7341cd8ed1609R2545 <https://github.com/torvalds/linux/commit/cb167893f41e21e6bd283d78e53489289dc0592d#diff-9900db808ce5e5dd24a7341cd8ed1609R2545>

[3] https://github.com/torvalds/linux/commit/73155879b3c1ac3ace35208a54a3a160ec520bef

#1 lore raw
Quoted 7 lines · On 3/23/19 7:00 AM, [email protected] wrote:

After upgrading iproute2 from 4.20 to 5.0 the following error occurs:

$ ip route show table default
Error: ipv4: FIB table does not exist.
Dump terminated

The command works for all tables other than 'default' one. It seems related to this commit[1]

'default' maps to table 253, and the table has not been created in the kernel. Local (255) and main (254) do exist.

#2 lore raw

Mar 24, 2019, 2:17 PM by [email protected]:

Quoted 8 lines · On 3/23/19 7:00 AM, > [email protected]…
Quoted 7 lines

After upgrading iproute2 from 4.20 to 5.0 the following error occurs:

$ ip route show table default
Error: ipv4: FIB table does not exist.
Dump terminated
Quoted 7 lines
Quoted 2 lines

The command works for all tables other than 'default' one. It seems related to this commit[1]

'default' maps to table 253, and the table has not been created in the kernel. Local (255) and main (254) do exist.

Ok but previous versions of iproute2 didn't treat this as error and didn't exited with non-zero status. Is non existing default route a system error which needs fixing?

#3 lore raw
Quoted 1 line · On 3/24/19 11:02 AM, [email protected] wrote:

Ok but previous versions of iproute2 didn't treat this as error and didn't exited with non-zero status. Is non existing default route a system error which needs fixing?

The kernel is returning that error, not iproute2.

It is the default *table*, not a default route.

#4 lore raw

Mar 24, 2019, 5:20 PM by [email protected]:

Quoted 7 lines · On 3/24/19 11:02 AM, > [email protected]…
Quoted 3 lines

Ok but previous versions of iproute2 didn't treat this as error and didn't exited with non-zero status. Is non existing default route a system error which needs fixing?

The kernel is returning that error, not iproute2.

I mean that with older versions of iproute2 this error isn't shown for userspace.

Quoted 2 lines

It is the default *table*, not a default route.

True. Still the question remains: is non existing default table a system error which needs fixing?

#5 lore raw
Quoted 7 lines · On 3/24/19 11:34 AM, [email protected] wrote:

Mar 24, 2019, 5:20 PM by [email protected]:

Quoted 5 lines · On 3/24/19 11:02 AM, > [email protected]…
Quoted 3 lines

Ok but previous versions of iproute2 didn't treat this as error and didn't exited with non-zero status. Is non existing default route a system error which needs fixing?

Quoted 7 lines
Quoted 2 lines

The kernel is returning that error, not iproute2.

I mean that with older versions of iproute2 this error isn't shown for userspace.

Quoted 2 lines

It is the default *table*, not a default route.

True. Still the question remains: is non existing default table a system error which needs fixing?

no, there is no system error. The kernel is telling iproute2 the requested table does not exist. Arguably iproute2 can catch a request to dump the 'default' table and not throw an error.

#6 lore raw
In reply to David Ahern
Quoted 6 lines · David Ahern
Quoted 1 line · On 3/24/19 11:02 AM, [email protected] wrote:

Ok but previous versions of iproute2 didn't treat this as error and didn't exited with non-zero status. Is non existing default route a system error which needs fixing?

The kernel is returning that error, not iproute2.

It is the default *table*, not a default route.

Something did change on iproute2 side between 4.20 and 5.0, though:

lion:~ # rpm -q iproute2
iproute2-4.20-0.x86_64
lion:~ # ip route show table default ; echo $?
0
lion:~ # ip route show table 123 ; echo $?
0
...
lion:~ # rpm -q iproute2
iproute2-5.0.0-0.x86_64
lion:~ # ip route show table default ; echo $?
Error: ipv4: FIB table does not exist.
Dump terminated
2
lion:~ # ip route show table 123 ; echo $?
Error: ipv4: FIB table does not exist.
Dump terminated
2

All I did was updating iproute2 package, the same kernel was running for both (I tried 5.0.3 and 5.1-rc1).

Michal

#7 lore raw
Quoted 28 lines · Michal Kubecek
Quoted 6 lines · David Ahern
Quoted 1 line · On 3/24/19 11:02 AM, [email protected] wrote:

Ok but previous versions of iproute2 didn't treat this as error and didn't exited with non-zero status. Is non existing default route a system error which needs fixing?

The kernel is returning that error, not iproute2.

It is the default *table*, not a default route.

Something did change on iproute2 side between 4.20 and 5.0, though:

lion:~ # rpm -q iproute2
iproute2-4.20-0.x86_64
lion:~ # ip route show table default ; echo $?
0
lion:~ # ip route show table 123 ; echo $?
0
...
lion:~ # rpm -q iproute2
iproute2-5.0.0-0.x86_64
lion:~ # ip route show table default ; echo $?
Error: ipv4: FIB table does not exist.
Dump terminated
2
lion:~ # ip route show table 123 ; echo $?
Error: ipv4: FIB table does not exist.
Dump terminated
2

Quoted 2 lines

All I did was updating iproute2 package, the same kernel was running for both (I tried 5.0.3 and 5.1-rc1).

Commit c7e6371bc4af ("ip route: Add protocol, table id and device to dump request") seems to be an obvious candidate. Before it, no matching rules in the dump used to be presented as an empty table but now ip gets a kernel error which it displays to the user.

Michal

#8 lore raw
Quoted 29 lines · Michal Kubecek
Quoted 28 lines · Michal Kubecek
Quoted 6 lines · David Ahern
Quoted 1 line · On 3/24/19 11:02 AM, [email protected] wrote:

Ok but previous versions of iproute2 didn't treat this as error and didn't exited with non-zero status. Is non existing default route a system error which needs fixing?

The kernel is returning that error, not iproute2.

It is the default *table*, not a default route.

Something did change on iproute2 side between 4.20 and 5.0, though:

lion:~ # rpm -q iproute2
iproute2-4.20-0.x86_64
lion:~ # ip route show table default ; echo $?
0
lion:~ # ip route show table 123 ; echo $?
0
...
lion:~ # rpm -q iproute2
iproute2-5.0.0-0.x86_64
lion:~ # ip route show table default ; echo $?
Error: ipv4: FIB table does not exist.
Dump terminated
2
lion:~ # ip route show table 123 ; echo $?
Error: ipv4: FIB table does not exist.
Dump terminated
2

Quoted 8 lines
Quoted 2 lines

All I did was updating iproute2 package, the same kernel was running for both (I tried 5.0.3 and 5.1-rc1).

Commit c7e6371bc4af ("ip route: Add protocol, table id and device to dump request") seems to be an obvious candidate. Before it, no matching rules in the dump used to be presented as an empty table but now ip gets a kernel error which it displays to the user.

It's the commit that enables strict checking. Kernel side has been changed to better inform the user of what happens on a request when strict checking is enabled. iproute2 has been updated to use this.

Essentially, ip asks for a dump of table 253. In the past there is no data, so nothing to return. The strict checking tells you explicitly "that table does not exist" versus the old method where nothing is returned and the user has to guess "the table exists but is empty or there is a bug dumping the table?"

Given the legacy of table 253/default, iproute2 can easily catch this error and just do nothing for backwards compatibility.

#9 lore raw

Mar 24, 2019, 10:09 PM by [email protected]:

Quoted 50 lines · Michal Kubecek
Quoted 35 lines · Michal Kubecek
Quoted 33 lines · David Ahern
Quoted 10 lines · On 3/24/19 11:02 AM, >>>> [email protected]…
Quoted 3 lines

Ok but previous versions of iproute2 didn't treat this as error and didn't exited with non-zero status. Is non existing default route a system error which needs fixing?

The kernel is returning that error, not iproute2.

It is the default *table*, not a default route.

Something did change on iproute2 side between 4.20 and 5.0, though:

lion:~ # rpm -q iproute2
iproute2-4.20-0.x86_64
lion:~ # ip route show table default ; echo $?
0
lion:~ # ip route show table 123 ; echo $?
0
...
lion:~ # rpm -q iproute2
iproute2-5.0.0-0.x86_64
lion:~ # ip route show table default ; echo $?
Error: ipv4: FIB table does not exist.
Dump terminated
2
lion:~ # ip route show table 123 ; echo $?
Error: ipv4: FIB table does not exist.
Dump terminated
2

Quoted 9 lines
Quoted 3 lines

All I did was updating iproute2 package, the same kernel was running for both (I tried 5.0.3 and 5.1-rc1).

Commit c7e6371bc4af ("ip route: Add protocol, table id and device to dump request") seems to be an obvious candidate. Before it, no matching rules in the dump used to be presented as an empty table but now ip gets a kernel error which it displays to the user.

It's the commit that enables strict checking. Kernel side has been changed to better inform the user of what happens on a request when strict checking is enabled. iproute2 has been updated to use this.

Quoted 9 lines

Essentially, ip asks for a dump of table 253. In the past there is no data, so nothing to return. The strict checking tells you explicitly "that table does not exist" versus the old method where nothing is returned and the user has to guess "the table exists but is empty or there is a bug dumping the table?"

Given the legacy of table 253/default, iproute2 can easily catch this error and just do nothing for backwards compatibility.

Hi,

are those changes planned then? I don't see anything in repo
https://git.kernel.org/pub/scm/network/iproute2/iproute2.git/log/

#11 lore raw

5.1 `ip route get addr/cidr` regression

In reply to [email protected]

Hi,

I'm back now and catching up with a lot of things. A few people have mentioned to me that wg-quick(8), a bash script that makes a bunch of iproute2 invocations, appears to be broken on 5.1. I've distilled the behavior change down to the following.

Behavior on 5.0:

+ ip link add wg0 type dummy
+ ip address add 192.168.50.2/24 dev wg0
+ ip link set mtu 1420 up dev wg0
+ ip route get 192.168.50.0/24
broadcast 192.168.50.0 dev wg0 src 192.168.50.2 uid 0
cache <local,brd>

Behavior on 5.1:

+ ip link add wg0 type dummy
+ ip address add 192.168.50.2/24 dev wg0
+ ip link set mtu 1420 up dev wg0
+ ip route get 192.168.50.0/24
RTNETLINK answers: Invalid argument

Upon investigating, I'm not sure that ip route get was ever suitable for getting details on a particular route. So I'll adjust the wg-quick(8) code accordingly. But FYI, this is unexpected userspace breakage.

Jason

Quoted 18 lines · On Sat, Mar 23, 2019 at 2:00 PM <[email protected]> wrote:

After upgrading iproute2 from 4.20 to 5.0 the following error occurs:

$ ip route show table default
Error: ipv4: FIB table does not exist.
Dump terminated

The command works for all tables other than 'default' one. It seems related to this commit[1]

I also saw "Error: ipv4: MR table does not exist." message in logs related to this commit[2] but don't know exact command to reproduce it. I've seen some fixups[3] for mentioned commits and wonder it they weren't complete.

I reproduced it on Linux 4.20.17 and 5.0.3.

[1] https://github.com/torvalds/linux/commit/18a8021a7be3207686851208f91a2f105b2d4703#diff-04a14e4f51765994f87e7e5e7681d0e1R861 <https://github.com/torvalds/linux/commit/18a8021a7be3207686851208f91a2f105b2d4703#diff-04a14e4f51765994f87e7e5e7681d0e1R861>

[2] https://github.com/torvalds/linux/commit/cb167893f41e21e6bd283d78e53489289dc0592d#diff-9900db808ce5e5dd24a7341cd8ed1609R2545 <https://github.com/torvalds/linux/commit/cb167893f41e21e6bd283d78e53489289dc0592d#diff-9900db808ce5e5dd24a7341cd8ed1609R2545>

[3] https://github.com/torvalds/linux/commit/73155879b3c1ac3ace35208a54a3a160ec520bef

#12 lore raw

Re: 5.1 `ip route get addr/cidr` regression

Quoted 16 lines · Jason A. Donenfeld

Hi,

I'm back now and catching up with a lot of things. A few people have mentioned to me that wg-quick(8), a bash script that makes a bunch of iproute2 invocations, appears to be broken on 5.1. I've distilled the behavior change down to the following.

Behavior on 5.0:

+ ip link add wg0 type dummy
+ ip address add 192.168.50.2/24 dev wg0
+ ip link set mtu 1420 up dev wg0
+ ip route get 192.168.50.0/24
broadcast 192.168.50.0 dev wg0 src 192.168.50.2 uid 0
cache <local,brd>

Quoted 7 lines

Behavior on 5.1:

+ ip link add wg0 type dummy
+ ip address add 192.168.50.2/24 dev wg0
+ ip link set mtu 1420 up dev wg0
+ ip route get 192.168.50.0/24
RTNETLINK answers: Invalid argument

With recent kernel and iproute2 5.1, I get

alaris:~ # ip route get 172.17.1.2/24
Error: ipv4: Invalid values in header for route get request.

This message comes from kernel commit a00302b60777 ("net: ipv4: route:
perform strict checks also for doit handlers") which only considers the range valid if the prefix is /32 (a single address).

But these checks are only performed when userspace requests strict validation which iproute2 does since (iproute2) commit aea41afcfd6d ("ip
bridge: Set NETLINK_GET_STRICT_CHK on socket"). So I would say the change is a result of the combination of kernel (5.1) commit a00302b60777 and iproute2 (5.0) commit aea41afcfd6d.

Quoted 4 lines

Upon investigating, I'm not sure that ip route get was ever suitable for getting details on a particular route. So I'll adjust the wg-quick(8) code accordingly. But FYI, this is unexpected userspace breakage.

AFAIK the purpose of 'ip route get' always was to let the user check the result of a route lookup, i.e. "what route would be used if I sent a packet to an address". To be honest I would have to check how exactly was "ip route get <addr>/<prefixlen>" implemented before.

Michal Kubecek

#13 lore raw

Re: 5.1 `ip route get addr/cidr` regression

Quoted 5 lines · Michal Kubecek

AFAIK the purpose of 'ip route get' always was to let the user check the result of a route lookup, i.e. "what route would be used if I sent a packet to an address". To be honest I would have to check how exactly was "ip route get <addr>/<prefixlen>" implemented before.

The prefixlen was always silently ignored. We are trying to clean up this 'silent ignoring' just hitting a few speed bumps.

#14 lore raw

Re: 5.1 `ip route get addr/cidr` regression

Quoted 10 lines · David Ahern
Quoted 5 lines · Michal Kubecek

AFAIK the purpose of 'ip route get' always was to let the user check the result of a route lookup, i.e. "what route would be used if I sent a packet to an address". To be honest I would have to check how exactly was "ip route get <addr>/<prefixlen>" implemented before.

The prefixlen was always silently ignored. We are trying to clean up this 'silent ignoring' just hitting a few speed bumps.

Indeed what we were after has always been, ip route show dev <dev> match <addr>/<prefixlen>, and the old positive return value from ip route get wasn't always correct for what we were using it for. So mostly the breakage exposed another bug here.

#15 lore raw

Re: 5.1 `ip route get addr/cidr` regression

In reply to Jason A. Donenfeld
Quoted 16 lines · Jason A. Donenfeld

Hi,

I'm back now and catching up with a lot of things. A few people have mentioned to me that wg-quick(8), a bash script that makes a bunch of iproute2 invocations, appears to be broken on 5.1. I've distilled the behavior change down to the following.

Behavior on 5.0:

+ ip link add wg0 type dummy
+ ip address add 192.168.50.2/24 dev wg0
+ ip link set mtu 1420 up dev wg0
+ ip route get 192.168.50.0/24
broadcast 192.168.50.0 dev wg0 src 192.168.50.2 uid 0
cache <local,brd>

Quoted 7 lines

Behavior on 5.1:

+ ip link add wg0 type dummy
+ ip address add 192.168.50.2/24 dev wg0
+ ip link set mtu 1420 up dev wg0
+ ip route get 192.168.50.0/24
RTNETLINK answers: Invalid argument

This is a 5.1 change.
a00302b607770 ("net: ipv4: route: perform strict checks also for doit handlers")

Basically, the /24 is unexpected. I'll send a patch.

Quoted 3 lines

Upon investigating, I'm not sure that ip route get was ever suitable for getting details on a particular route. So I'll adjust the

'ip route get <prefix> fibmatch' will show the fib entry.

#16 lore raw

Re: 5.1 `ip route get addr/cidr` regression

Quoted 29 lines · On Fri, 17 May 2019 09:17:51 -0600 David Ahern <[email protected]> wrote:
Quoted 16 lines · Jason A. Donenfeld

Hi,

I'm back now and catching up with a lot of things. A few people have mentioned to me that wg-quick(8), a bash script that makes a bunch of iproute2 invocations, appears to be broken on 5.1. I've distilled the behavior change down to the following.

Behavior on 5.0:

+ ip link add wg0 type dummy
+ ip address add 192.168.50.2/24 dev wg0
+ ip link set mtu 1420 up dev wg0
+ ip route get 192.168.50.0/24
broadcast 192.168.50.0 dev wg0 src 192.168.50.2 uid 0
cache <local,brd>

Quoted 7 lines

Behavior on 5.1:

+ ip link add wg0 type dummy
+ ip address add 192.168.50.2/24 dev wg0
+ ip link set mtu 1420 up dev wg0
+ ip route get 192.168.50.0/24
RTNETLINK answers: Invalid argument

This is a 5.1 change.
a00302b607770 ("net: ipv4: route: perform strict checks also for doit handlers")

Quoted 8 lines

Basically, the /24 is unexpected. I'll send a patch.

Quoted 3 lines

Upon investigating, I'm not sure that ip route get was ever suitable for getting details on a particular route. So I'll adjust the

'ip route get <prefix> fibmatch' will show the fib entry.

If you want to keep the error, the kernel should send additional extack as to reason. EINVAL is not user friendly...

#17 lore raw

Re: 5.1 `ip route get addr/cidr` regression

Quoted 35 lines · Stephen Hemminger
Quoted 32 lines · On Fri, 17 May 2019 09:17:51 -0600 David Ahern <[email protected]> wrote:
Quoted 16 lines · Jason A. Donenfeld

Hi,

I'm back now and catching up with a lot of things. A few people have mentioned to me that wg-quick(8), a bash script that makes a bunch of iproute2 invocations, appears to be broken on 5.1. I've distilled the behavior change down to the following.

Behavior on 5.0:

+ ip link add wg0 type dummy
+ ip address add 192.168.50.2/24 dev wg0
+ ip link set mtu 1420 up dev wg0
+ ip route get 192.168.50.0/24
broadcast 192.168.50.0 dev wg0 src 192.168.50.2 uid 0
cache <local,brd>

Quoted 7 lines

Behavior on 5.1:

+ ip link add wg0 type dummy
+ ip address add 192.168.50.2/24 dev wg0
+ ip link set mtu 1420 up dev wg0
+ ip route get 192.168.50.0/24
RTNETLINK answers: Invalid argument

This is a 5.1 change.
a00302b607770 ("net: ipv4: route: perform strict checks also for doit handlers")

Basically, the /24 is unexpected. I'll send a patch.

Quoted 1 line
Quoted 9 lines
Quoted 5 lines
Quoted 2 lines

Upon investigating, I'm not sure that ip route get was ever suitable for getting details on a particular route. So I'll adjust the

'ip route get <prefix> fibmatch' will show the fib entry.

If you want to keep the error, the kernel should send additional extack as to reason. EINVAL is not user friendly...

The kernel does set an extack for all EINVAL in this code path.

Not sure why Jason is not seeing that. Really odd that he hits the error AND does not get a message back since it requires an updated ip command to set the strict checking flag and that command understands extack. Perhaps no libmnl?

#18 lore raw

Re: 5.1 `ip route get addr/cidr` regression

Quoted 4 lines · David Ahern

Not sure why Jason is not seeing that. Really odd that he hits the error AND does not get a message back since it requires an updated ip command to set the strict checking flag and that command understands extack. Perhaps no libmnl?

Right, no libmnl. This is coming out of the iproute2 compiled for the tests at https://www.wireguard.com/build-status/ which are pretty minimal. Extact support would be kind of useful for diagnostics, and wg(8) already uses it, so I can probably put that in my build system.

#19 lore raw

Re: 5.1 `ip route get addr/cidr` regression

Quoted 11 lines · Jason A. Donenfeld
Quoted 4 lines · David Ahern

Not sure why Jason is not seeing that. Really odd that he hits the error AND does not get a message back since it requires an updated ip command to set the strict checking flag and that command understands extack. Perhaps no libmnl?

Right, no libmnl. This is coming out of the iproute2 compiled for the tests at https://www.wireguard.com/build-status/ which are pretty minimal. Extact support would be kind of useful for diagnostics, and wg(8) already uses it, so I can probably put that in my build system.

Voila, extack:

+ ip link add wg0 type dummy
+ ip addr add 192.168.4.2/24 dev wg0
+ ip link set wg0 up
+ ip route get 192.168.4.0/24
Error: ipv4: Invalid values in header for route

#20 lore raw

Re: 5.1 `ip route get addr/cidr` regression

In reply to Stephen Hemminger
Quoted 28 lines · Stephen Hemminger
Quoted 25 lines · On Fri, 17 May 2019 09:17:51 -0600 David Ahern <[email protected]> wrote:
Quoted 23 lines · Jason A. Donenfeld

Hi,

I'm back now and catching up with a lot of things. A few people have mentioned to me that wg-quick(8), a bash script that makes a bunch of iproute2 invocations, appears to be broken on 5.1. I've distilled the behavior change down to the following.

Behavior on 5.0:

+ ip link add wg0 type dummy
+ ip address add 192.168.50.2/24 dev wg0
+ ip link set mtu 1420 up dev wg0
+ ip route get 192.168.50.0/24
broadcast 192.168.50.0 dev wg0 src 192.168.50.2 uid 0
cache <local,brd>

Behavior on 5.1:

+ ip link add wg0 type dummy
+ ip address add 192.168.50.2/24 dev wg0
+ ip link set mtu 1420 up dev wg0
+ ip route get 192.168.50.0/24
RTNETLINK answers: Invalid argument

Quoted 5 lines
Quoted 5 lines

This is a 5.1 change.
a00302b607770 ("net: ipv4: route: perform strict checks also for doit handlers")

Basically, the /24 is unexpected. I'll send a patch.

oh, and I think changing iproute2 to ignore the /24 and always set to 32 is better than changing the kernel to allow a prefix length that is ignored.