tipc: avoid unnecessary copying of bundled messages
Subsystems (3)
PATCH v2 1/1 tipc: avoid unnecessary copying of bundled messages
A received sk buffer may contain dozens of smaller 'bundled' messages which after extraction go each in their own direction.
Unfortunately, when we extract those messages using skb_clone() each of the extracted buffers inherit the truesize value of the original buffer. Apart from causing massive overaccounting of the base buffer's memory, this often causes tipc_msg_validate() to come to the false conclusion that the ratio truesize/datasize > 4, and perform an unnecessary copying of the extracted buffer.
We now fix this problem by explicitly correcting the truesize value of the buffer clones to be the truesize of the clone itself plus a calculated fraction of the base buffer's overhead. This change eliminates the overaccounting and at least mitigates the occurrence of unnecessary buffer copying.
--- net/tipc/msg.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
net/tipc/msg.c +6 -1
@@ -416,8 +416,8 @@ bool tipc_msg_bundle(struct sk_buff *skb, struct tipc_msg *msg, u32 mtu)*/ bool tipc_msg_extract (struct sk_buff *skb,struct sk_buff **iskb,int *pos) { +int imsz, offset, clone_cnt, skb_overhead;struct tipc_msg *msg; -int imsz, offset; *iskb =NULL ;if (unlikely (skb_linearize (skb)))
@@ -434,6 +434,11 @@ bool tipc_msg_extract(struct sk_buff *skb, struct sk_buff **iskb, int *pos)skb_pull (*iskb, offset); imsz =msg_size (buf_msg (*iskb));skb_trim (*iskb, imsz); + +/* Scale extracted buffer's truesize to avoid double accounting */ + clone_cnt =max_t (u32,1 ,msg_msgcnt (msg)); + skb_overhead = skb->truesize - skb->len; + (*iskb)->truesize =SKB_TRUESIZE (imsz) + skb_overhead / clone_cnt;if (unlikely (!tipc_msg_validate (iskb))) goto none; *pos +=align (imsz);
Signature · 1 line
2.1.4
David Miller
maintainerFrom: Jon Maloy <[email protected]>
Date: Thu, 15 Feb 2018 14:14:37 +0100
Quoted 16 lines
A received sk buffer may contain dozens of smaller 'bundled' messages which after extraction go each in their own direction.
Unfortunately, when we extract those messages using
skb_clone()each of the extracted buffers inherit the truesize value of the original buffer. Apart from causing massive overaccounting of the base buffer's memory, this often causestipc_msg_validate()to come to the false conclusion that the ratio truesize/datasize > 4, and perform an unnecessary copying of the extracted buffer.We now fix this problem by explicitly correcting the truesize value of the buffer clones to be the truesize of the clone itself plus a calculated fraction of the base buffer's overhead. This change eliminates the overaccounting and at least mitigates the occurrence of unnecessary buffer copying.
Quoted 3 lines
Reported-by Hoang Le <[email protected]>Acked-by Ying Xue <[email protected]>Signed-off-by Jon Maloy <[email protected]>
As I explained in my previous two emails, I don't think this method of accounting is appropriate.
All of your clones must use the same skb->truesize as the original SKB because each and every one of them keeps the full buffer from being liberated until they are released.
Quoted 38 lines
-----Original Message-----
From: David Miller [mailto:[email protected]]
Sent: Friday, February 16, 2018 21:33
To: Jon Maloy <[email protected]>
Cc: [email protected]; Mohan Krishna Ghanta Krishnamurthy <[email protected]>; Tung Quang Nguyen <[email protected]>; Hoang Huu Le <[email protected]>; Canh Duc Luu <[email protected]>; Ying Xue <[email protected]>; tipc- [email protected]
Subject: Re: [net-next v2 1/1] tipc: avoid unnecessary copying of bundled messagesFrom: Jon Maloy <[email protected]>
Date: Thu, 15 Feb 2018 14:14:37 +0100Quoted 16 lines
A received sk buffer may contain dozens of smaller 'bundled' messages which after extraction go each in their own direction.
Unfortunately, when we extract those messages using
skb_clone()each of the extracted buffers inherit the truesize value of the original buffer. Apart from causing massive overaccounting of the base buffer's memory, this often causestipc_msg_validate()to come to the false conclusion that the ratio truesize/datasize > 4, and perform an unnecessary copying of the extracted buffer.We now fix this problem by explicitly correcting the truesize value of the buffer clones to be the truesize of the clone itself plus a calculated fraction of the base buffer's overhead. This change eliminates the overaccounting and at least mitigates the occurrence of unnecessary buffer copying.
Quoted 3 lines
Reported-by Hoang Le <[email protected]>Acked-by Ying Xue <[email protected]>Signed-off-by Jon Maloy <[email protected]>
Quoted 6 lines
As I explained in my previous two emails, I don't think this method of accounting is appropriate.
All of your clones must use the same
skb->truesizeas the original SKB because each and every one of them keeps the full buffer from being liberated until they are released.
I understand what you are saying, although I am not happy with its consequences in this case. I guess I will just leave it the way it is until I can come up with something smarter. ///jon
From: Jon Maloy <[email protected]>
Date: Thu, 15 Feb 2018 14:14:37 +0100
Quoted 10 lines
A received sk buffer may contain dozens of smaller 'bundled' messages which after extraction go each in their own direction.
Unfortunately, when we extract those messages using
skb_clone()each of the extracted buffers inherit the truesize value of the original buffer. Apart from causing massive overaccounting of the base buffer's memory, this often causestipc_msg_validate()to come to the false conclusion that the ratio truesize/datasize > 4, and perform an unnecessary copying of the extracted buffer.
Quoted 5 lines
We now fix this problem by explicitly correcting the truesize value of the buffer clones to be the truesize of the clone itself plus a calculated fraction of the base buffer's overhead. This change eliminates the overaccounting and at least mitigates the occurrence of unnecessary buffer copying.
Have you actually checked that copying the data when you extract the messages isn't faster than cloning and trying to avoid the copy? Copying at the point is probably cheaper because it leads to a simpler message structure.
David
Quoted 34 lines
-----Original Message-----
From: David Laight [mailto:[email protected]]
Sent: Monday, February 19, 2018 14:30
To: Jon Maloy <[email protected]>
Cc: [email protected]; Mohan Krishna Ghanta Krishnamurthy <[email protected]>; Tung Quang Nguyen <[email protected]>; Hoang Huu Le <[email protected]>; Canh Duc Luu <[email protected]>; Ying Xue <[email protected]>; tipc- [email protected]
Subject: RE: [net-next v2 1/1] tipc: avoid unnecessary copying of bundled messagesFrom: Jon Maloy <[email protected]>
Date: Thu, 15 Feb 2018 14:14:37 +0100Quoted 10 lines
A received sk buffer may contain dozens of smaller 'bundled' messages which after extraction go each in their own direction.
Unfortunately, when we extract those messages using
skb_clone()each of the extracted buffers inherit the truesize value of the original buffer. Apart from causing massive overaccounting of the base buffer's memory, this often causestipc_msg_validate()to come to the false conclusion that the ratio truesize/datasize > 4, and perform an unnecessary copying of the extracted buffer.Quoted 5 lines
We now fix this problem by explicitly correcting the truesize value of the buffer clones to be the truesize of the clone itself plus a calculated fraction of the base buffer's overhead. This change eliminates the overaccounting and at least mitigates the occurrence of unnecessary buffer copying.
Quoted 4 lines
Have you actually checked that copying the data when you extract the messages isn't faster than cloning and trying to avoid the copy? Copying at the point is probably cheaper because it leads to a simpler message structure.
Yes, that is probably what I'll end up doing, if copying is unavoidable anyway.
///jon
Quoted 2 lines
David
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot