Skip to contentSkip to conversation

tipc: avoid unnecessary copying of bundled messages

PATCH In review Jon Maloy 3 participants 5 messages Feb 15, 2018 – Feb 19, 2018

Subsystems (3)
lore
Share & tools
lore

Jon Maloy

author
#0 lore raw

PATCH v2 1/1 tipc: avoid unnecessary copying of bundled messages

A received sk buffer may contain dozens of smaller 'bundled' messages which after extraction go each in their own direction.

Unfortunately, when we extract those messages using skb_clone() each of the extracted buffers inherit the truesize value of the original buffer. Apart from causing massive overaccounting of the base buffer's memory, this often causes tipc_msg_validate() to come to the false conclusion that the ratio truesize/datasize > 4, and perform an unnecessary copying of the extracted buffer.

We now fix this problem by explicitly correcting the truesize value of the buffer clones to be the truesize of the clone itself plus a calculated fraction of the base buffer's overhead. This change eliminates the overaccounting and at least mitigates the occurrence of unnecessary buffer copying.

---
 net/tipc/msg.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
net/tipc/msg.c +6 -1
index 4e1c6f6..ce0bfc4 100644
@@ -416,8 +416,8 @@ bool tipc_msg_bundle(struct sk_buff *skb, struct tipc_msg *msg, u32 mtu)
  */
 bool tipc_msg_extract(struct sk_buff *skb, struct sk_buff **iskb, int *pos)
 {
+	int imsz, offset, clone_cnt, skb_overhead;
 	struct tipc_msg *msg;
-	int imsz, offset;
 
 	*iskb = NULL;
 	if (unlikely(skb_linearize(skb)))
@@ -434,6 +434,11 @@ bool tipc_msg_extract(struct sk_buff *skb, struct sk_buff **iskb, int *pos)
 	skb_pull(*iskb, offset);
 	imsz = msg_size(buf_msg(*iskb));
 	skb_trim(*iskb, imsz);
+
+	/* Scale extracted buffer's truesize to avoid double accounting */
+	clone_cnt = max_t(u32, 1, msg_msgcnt(msg));
+	skb_overhead = skb->truesize - skb->len;
+	(*iskb)->truesize = SKB_TRUESIZE(imsz) + skb_overhead / clone_cnt;
 	if (unlikely(!tipc_msg_validate(iskb)))
 		goto none;
 	*pos += align(imsz);
Signature · 1 line
2.1.4
maintainer
#1 lore raw

From: Jon Maloy <[email protected]>
Date: Thu, 15 Feb 2018 14:14:37 +0100

Quoted 16 lines

A received sk buffer may contain dozens of smaller 'bundled' messages which after extraction go each in their own direction.

Unfortunately, when we extract those messages using skb_clone() each of the extracted buffers inherit the truesize value of the original buffer. Apart from causing massive overaccounting of the base buffer's memory, this often causes tipc_msg_validate() to come to the false conclusion that the ratio truesize/datasize > 4, and perform an unnecessary copying of the extracted buffer.

We now fix this problem by explicitly correcting the truesize value of the buffer clones to be the truesize of the clone itself plus a calculated fraction of the base buffer's overhead. This change eliminates the overaccounting and at least mitigates the occurrence of unnecessary buffer copying.

Quoted 3 lines

As I explained in my previous two emails, I don't think this method of accounting is appropriate.

All of your clones must use the same skb->truesize as the original SKB because each and every one of them keeps the full buffer from being liberated until they are released.

author
#2 lore raw
Quoted 38 lines

-----Original Message-----
From: David Miller [mailto:[email protected]]
Sent: Friday, February 16, 2018 21:33
To: Jon Maloy <[email protected]>
Cc: [email protected]; Mohan Krishna Ghanta Krishnamurthy <[email protected]>; Tung Quang Nguyen <[email protected]>; Hoang Huu Le <[email protected]>; Canh Duc Luu <[email protected]>; Ying Xue <[email protected]>; tipc- [email protected]
Subject: Re: [net-next v2 1/1] tipc: avoid unnecessary copying of bundled messages

From: Jon Maloy <[email protected]>
Date: Thu, 15 Feb 2018 14:14:37 +0100

Quoted 16 lines

A received sk buffer may contain dozens of smaller 'bundled' messages which after extraction go each in their own direction.

Unfortunately, when we extract those messages using skb_clone() each of the extracted buffers inherit the truesize value of the original buffer. Apart from causing massive overaccounting of the base buffer's memory, this often causes tipc_msg_validate() to come to the false conclusion that the ratio truesize/datasize > 4, and perform an unnecessary copying of the extracted buffer.

We now fix this problem by explicitly correcting the truesize value of the buffer clones to be the truesize of the clone itself plus a calculated fraction of the base buffer's overhead. This change eliminates the overaccounting and at least mitigates the occurrence of unnecessary buffer copying.

Quoted 3 lines
Quoted 6 lines

As I explained in my previous two emails, I don't think this method of accounting is appropriate.

All of your clones must use the same skb->truesize as the original SKB because each and every one of them keeps the full buffer from being liberated until they are released.

I understand what you are saying, although I am not happy with its consequences in this case. I guess I will just leave it the way it is until I can come up with something smarter. ///jon

#3 lore raw
In reply to David Miller

From: Jon Maloy <[email protected]>
Date: Thu, 15 Feb 2018 14:14:37 +0100

Quoted 10 lines

A received sk buffer may contain dozens of smaller 'bundled' messages which after extraction go each in their own direction.

Unfortunately, when we extract those messages using skb_clone() each of the extracted buffers inherit the truesize value of the original buffer. Apart from causing massive overaccounting of the base buffer's memory, this often causes tipc_msg_validate() to come to the false conclusion that the ratio truesize/datasize > 4, and perform an unnecessary copying of the extracted buffer.

Quoted 5 lines

We now fix this problem by explicitly correcting the truesize value of the buffer clones to be the truesize of the clone itself plus a calculated fraction of the base buffer's overhead. This change eliminates the overaccounting and at least mitigates the occurrence of unnecessary buffer copying.

Have you actually checked that copying the data when you extract the messages isn't faster than cloning and trying to avoid the copy? Copying at the point is probably cheaper because it leads to a simpler message structure.

David

author
#4 lore raw
Quoted 34 lines

-----Original Message-----
From: David Laight [mailto:[email protected]]
Sent: Monday, February 19, 2018 14:30
To: Jon Maloy <[email protected]>
Cc: [email protected]; Mohan Krishna Ghanta Krishnamurthy <[email protected]>; Tung Quang Nguyen <[email protected]>; Hoang Huu Le <[email protected]>; Canh Duc Luu <[email protected]>; Ying Xue <[email protected]>; tipc- [email protected]
Subject: RE: [net-next v2 1/1] tipc: avoid unnecessary copying of bundled messages

From: Jon Maloy <[email protected]>
Date: Thu, 15 Feb 2018 14:14:37 +0100

Quoted 10 lines

A received sk buffer may contain dozens of smaller 'bundled' messages which after extraction go each in their own direction.

Unfortunately, when we extract those messages using skb_clone() each of the extracted buffers inherit the truesize value of the original buffer. Apart from causing massive overaccounting of the base buffer's memory, this often causes tipc_msg_validate() to come to the false conclusion that the ratio truesize/datasize > 4, and perform an unnecessary copying of the extracted buffer.

Quoted 5 lines

We now fix this problem by explicitly correcting the truesize value of the buffer clones to be the truesize of the clone itself plus a calculated fraction of the base buffer's overhead. This change eliminates the overaccounting and at least mitigates the occurrence of unnecessary buffer copying.

Quoted 4 lines

Have you actually checked that copying the data when you extract the messages isn't faster than cloning and trying to avoid the copy? Copying at the point is probably cheaper because it leads to a simpler message structure.

Yes, that is probably what I'll end up doing, if copying is unavoidable anyway.

///jon

Quoted 2 lines

David

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot