esp: downgrade zerocopy managed frags before mutating skb frags
Subsystems (3)
PATCH v2 1/1 esp: downgrade zerocopy managed frags before mutating skb frags
On the out-of-place output path (esp->inplace == false) ESP rewrites the skb frag array: esp_output_head() appends a trailer frag and esp_output_tail() replaces the frags with a destination page, both referenced with get_page().
When the skb carries zerocopy managed frags (SKBFL_MANAGED_FRAG_REFS) the payload frags are owned by the ubuf and must not be referenced or unreferenced individually, but ESP mutates the frag array without ever downgrading the skb. This breaks the managed-frag invariant two ways:
- esp_ssg_unref() walks the source scatterlist and drops a page reference for every frag, including the ubuf-owned payload frags, pushing their refcount below the GUP pin bias while the pages are still pinned, i.e. a use-after-free of the zerocopy pages;
- esp_output_tail() installs its destination page as frag 0 with get_page() but leaves SKBFL_MANAGED_FRAG_REFS set, so skb_release_data() takes the skip_unref branch and never drops that reference, leaking the x->xfrag page at packet rate.
Fix this the way every other frag-mutating site does (__ip_append_data(), __ip6_append_data(), tcp_sendmsg_locked()) and call skb_zcopy_downgrade_managed() before ESP touches the frag array: it takes a real reference on each existing frag and clears SKBFL_MANAGED_FRAG_REFS, so the per-frag unref in esp_ssg_unref() and the frag release in skb_release_data() are both balanced and no mixed-ownership frag array is left behind.
---
Changes in v2:
- Use skb_zcopy_downgrade_managed() before ESP mutates the frag array, instead of the early return in esp_ssg_unref(). As pointed out in review, that early return also suppressed release of the ESP-owned trailer and destination page references (a leak); downgrading the skb the way __ip_append_data() does fixes both the underflow and the leak.
- Fix the Fixes: tag - the bug needs managed frags (753f1ca4e1e5), not the 2017 skb_cow_data avoidance commits.
v1: https://lore.kernel.org/netdev/[email protected]/
net/ipv4/esp4.c | 6 ++++++ net/ipv6/esp6.c | 6 ++++++ 2 files changed, 12 insertions(+)
net/ipv4/esp4.c +6 -0
@@ -441,6 +441,12 @@ int esp_output_head(struct xfrm_state *x, struct sk_buff *skb, struct esp_info * esp->inplace = false; +/* Take real page refs and clear SKBFL_MANAGED_FRAG_REFS before +* we mutate the frag array, so the per-frag unref stays balanced +* for zerocopy managed frags (see __ip_append_data()). +*/ +skb_zcopy_downgrade_managed (skb); + allocsize =ALIGN (tailen,L1_CACHE_BYTES );spin_lock_bh (&x->lock);
net/ipv6/esp6.c +6 -0
@@ -470,6 +470,12 @@ int esp6_output_head(struct xfrm_state *x, struct sk_buff *skb, struct esp_info esp->inplace = false; +/* Take real page refs and clear SKBFL_MANAGED_FRAG_REFS before +* we mutate the frag array, so the per-frag unref stays balanced +* for zerocopy managed frags (see __ip_append_data()). +*/ +skb_zcopy_downgrade_managed (skb); + allocsize =ALIGN (tailen,L1_CACHE_BYTES );spin_lock_bh (&x->lock);
Signature · 1 line
2.34.1
Steffen Klassert
maintainerQuoted 28 lines · Maher Azzouzi
On the out-of-place output path (
esp->inplace== false) ESP rewrites the skb frag array:esp_output_head()appends a trailer frag andesp_output_tail()replaces the frags with a destination page, both referenced withget_page().When the skb carries zerocopy managed frags (
SKBFL_MANAGED_FRAG_REFS) the payload frags are owned by the ubuf and must not be referenced or unreferenced individually, but ESP mutates the frag array without ever downgrading the skb. This breaks the managed-frag invariant two ways:-
esp_ssg_unref()walks the source scatterlist and drops a page reference for every frag, including the ubuf-owned payload frags, pushing their refcount below the GUP pin bias while the pages are still pinned, i.e. a use-after-free of the zerocopy pages;-
esp_output_tail()installs its destination page as frag 0 withget_page()but leavesSKBFL_MANAGED_FRAG_REFSset, soskb_release_data()takes theskip_unrefbranch and never drops that reference, leaking thex->xfragpage at packet rate.Fix this the way every other frag-mutating site does (
__ip_append_data(),__ip6_append_data(),tcp_sendmsg_locked()) and callskb_zcopy_downgrade_managed()before ESP touches the frag array: it takes a real reference on each existing frag and clearsSKBFL_MANAGED_FRAG_REFS, so the per-frag unref inesp_ssg_unref()and the frag release inskb_release_data()are both balanced and no mixed-ownership frag array is left behind.
Quoted 2 lines
Fixes 753f1ca4e1e5 ("net: introduce managed frags infrastructure")Signed-off-by Maher Azzouzi <[email protected]>
Now applied to the ipsec tree, thanks a lot!