ip link vf info truncating with many VFs
Subsystems (2)
Hi,
I recently noticed an issue in the rtnetlink API for obtaining VF information.
If a device creates 222 or more VF devices, the rtnl_fill_vf function will incorrectly label the size of the IFLA_VFINFO_LIST attribute. This occurs because rtnl_fill_vfinfo will have added more than 65k (maximum size of a single attribute since nla_len is a __u16).
This causes the calculation in nla_nest_end to overflow and report a significantly shorter length value. Worse case, with 222 VFs, the "ip link show <device>" reports no VF info at all.
For some reason, the nla_put calls do not trigger an EMSGSIZE error, because the skb itself is capable of holding the data.
I think the right thing is probably to do some sort of overflow-protected calculation and print a warning... or find a way to fix nla_put to error with -EMSGSIZE if we would exceed the nested attribute size limit... I am not sure how to do that at a glance.
Thanks,
Jake
Jakub Kicinski
maintainerQuoted 14 lines · On Fri, 28 Feb 2020 16:33:40 -0800 Jacob Keller wrote:
Hi,
I recently noticed an issue in the rtnetlink API for obtaining VF information.
If a device creates 222 or more VF devices, the
rtnl_fill_vffunction will incorrectly label the size of theIFLA_VFINFO_LISTattribute. This occurs becausertnl_fill_vfinfowill have added more than 65k (maximum size of a single attribute sincenla_lenis a__u16).This causes the calculation in
nla_nest_endto overflow and report a significantly shorter length value. Worse case, with 222 VFs, the "ip link show <device>" reports no VF info at all.
Quoted 7 lines
For some reason, the
nla_putcalls do not trigger an EMSGSIZE error, because the skb itself is capable of holding the data.I think the right thing is probably to do some sort of overflow-protected calculation and print a warning... or find a way to fix
nla_putto error with-EMSGSIZEif we would exceed the nested attribute size limit... I am not sure how to do that at a glance.
Making nla_nest_end() return an error on overflow seems like the most reasonable way forward to me, FWIW. Simply compare the result to U16_MAX, I don't think anything more clever is needed.
Some of the callers actually already check for errors of nla_nest_end() (qdiscs' dump methods use the result which is later checked for less that zero).
Then rtnetlink code should be made aware that nla_nest_end() may fail.
(When you post it's probably a good idea to widen the CC list to Johannes Berg, Pablo, DaveA, Jiri..)
Quoted 23 lines · Jakub Kicinski
Quoted 14 lines · On Fri, 28 Feb 2020 16:33:40 -0800 Jacob Keller wrote:
Hi,
I recently noticed an issue in the rtnetlink API for obtaining VF information.
If a device creates 222 or more VF devices, the
rtnl_fill_vffunction will incorrectly label the size of theIFLA_VFINFO_LISTattribute. This occurs becausertnl_fill_vfinfowill have added more than 65k (maximum size of a single attribute sincenla_lenis a__u16).This causes the calculation in
nla_nest_endto overflow and report a significantly shorter length value. Worse case, with 222 VFs, the "ip link show <device>" reports no VF info at all.Quoted 7 lines
For some reason, the
nla_putcalls do not trigger an EMSGSIZE error, because the skb itself is capable of holding the data.I think the right thing is probably to do some sort of overflow-protected calculation and print a warning... or find a way to fix
nla_putto error with-EMSGSIZEif we would exceed the nested attribute size limit... I am not sure how to do that at a glance.
Quoted 5 lines
Making
nla_nest_end()return an error on overflow seems like the most reasonable way forward to me, FWIW. Simply compare the result toU16_MAX, I don't think anything more clever is needed.
Sure, I alto think that's the right approach to fix this.
As long we calculate the value using something larger than a u16 first, that should work.
Quoted 3 lines
Some of the callers actually already check for errors of
nla_nest_end()(qdiscs' dump methods use the result which is later checked for less that zero).
I'll take a look at the qdisc code.
Quoted 4 lines
Then rtnetlink code should be made aware that
nla_nest_end()may fail.
Right.
Quoted 3 lines
(When you post it's probably a good idea to widen the CC list to Johannes Berg, Pablo, DaveA, Jiri..)
Yep. I wasn't sure who all to add here, so I tried looking at the MAINTAINERS file.
Thanks,
Jake
Quoted 30 lines · Jacob Keller
Quoted 23 lines · Jakub Kicinski
Quoted 14 lines · On Fri, 28 Feb 2020 16:33:40 -0800 Jacob Keller wrote:
Hi,
I recently noticed an issue in the rtnetlink API for obtaining VF information.
If a device creates 222 or more VF devices, the
rtnl_fill_vffunction will incorrectly label the size of theIFLA_VFINFO_LISTattribute. This occurs becausertnl_fill_vfinfowill have added more than 65k (maximum size of a single attribute sincenla_lenis a__u16).This causes the calculation in
nla_nest_endto overflow and report a significantly shorter length value. Worse case, with 222 VFs, the "ip link show <device>" reports no VF info at all.Quoted 7 lines
For some reason, the
nla_putcalls do not trigger an EMSGSIZE error, because the skb itself is capable of holding the data.I think the right thing is probably to do some sort of overflow-protected calculation and print a warning... or find a way to fix
nla_putto error with-EMSGSIZEif we would exceed the nested attribute size limit... I am not sure how to do that at a glance.Quoted 5 lines
Making
nla_nest_end()return an error on overflow seems like the most reasonable way forward to me, FWIW. Simply compare the result toU16_MAX, I don't think anything more clever is needed.
Quoted 5 lines
Sure, I alto think that's the right approach to fix this.
As long we calculate the value using something larger than a u16 first, that should work.
Another pandora's box.
Seems like there are a few other places that set nla_len that should be checked as well - like __nla_reserve and that bleeds into __nla_put.
Quoted 33 lines · David Ahern
Quoted 32 lines · Jacob Keller
Quoted 23 lines · Jakub Kicinski
Quoted 14 lines · On Fri, 28 Feb 2020 16:33:40 -0800 Jacob Keller wrote:
Hi,
I recently noticed an issue in the rtnetlink API for obtaining VF information.
If a device creates 222 or more VF devices, the
rtnl_fill_vffunction will incorrectly label the size of theIFLA_VFINFO_LISTattribute. This occurs becausertnl_fill_vfinfowill have added more than 65k (maximum size of a single attribute sincenla_lenis a__u16).This causes the calculation in
nla_nest_endto overflow and report a significantly shorter length value. Worse case, with 222 VFs, the "ip link show <device>" reports no VF info at all.Quoted 7 lines
For some reason, the
nla_putcalls do not trigger an EMSGSIZE error, because the skb itself is capable of holding the data.I think the right thing is probably to do some sort of overflow-protected calculation and print a warning... or find a way to fix
nla_putto error with-EMSGSIZEif we would exceed the nested attribute size limit... I am not sure how to do that at a glance.Quoted 5 lines
Making
nla_nest_end()return an error on overflow seems like the most reasonable way forward to me, FWIW. Simply compare the result toU16_MAX, I don't think anything more clever is needed.Sure, I alto think that's the right approach to fix this.
Quoted 9 lines
Quoted 3 lines
As long we calculate the value using something larger than a u16 first, that should work.
Another pandora's box.
Seems like there are a few other places that set
nla_lenthat should be checked as well - like__nla_reserveand that bleeds into__nla_put.
Yea, I bet. I'm working on a patch for this nla_nest_end first.
Thanks,
Jake