Revert "virtio_net: replace netdev_alloc_skb_ip_align() with napi_alloc_skb()"
Subsystems (3)
PATCH 1/1 Revert "virtio_net: replace netdev_alloc_skb_ip_align() with napi_alloc_skb()"
From: Eric Dumazet <[email protected]>
This reverts commit c67f5db82027ba6d2ea4ac9176bc45996a03ae6a.
While using page fragments instead of a kmalloc backed skb->head might give a small performance improvement in some cases, there is a huge risk of memory use under estimation.
GOOD_COPY_LEN is 128 bytes. This means that we need a small amount of memory to hold the headers and struct skb_shared_info
Yet, napi_alloc_skb() might use a whole 32KB page (or 64KB on PowerPc) for long lived incoming TCP packets.
We have been tracking OOM issues on GKE hosts hitting tcp_mem limits but consuming far more memory for TCP buffers than instructed in tcp_mem[2]
Even if we force napi_alloc_skb() to only use order-0 pages, the issue would still be there on arches with PAGE_SIZE >= 32768
Using alloc_skb() and thus standard kmallloc() for skb->head allocations will get the benefit of letting other objects in each page being independently used by other skbs, regardless of the lifetime.
Note that a similar problem exists for skbs allocated from napi_get_frags(), this is handled in a separate patch.
I would like to thank Greg Thelen for his precious help on this matter, analysing crash dumps is always a time consuming task.
--- drivers/net/virtio_net.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/net/virtio_net.c +1 -1
@@ -386,7 +386,7 @@ static struct sk_buff *page_to_skb(struct virtnet_info *vi, p =page_address (page) + offset;/* copy small packet so we can reuse these pages for small data */ - skb =napi_alloc_skb (&rq->napi,GOOD_COPY_LEN ); + skb =netdev_alloc_skb_ip_align (vi->dev,GOOD_COPY_LEN );if (unlikely (!skb))return NULL ;
Signature · 1 line
2.30.0.284.gd98b1dd5eaa7-goog
Quoted 53 lines · Eric Dumazet
From: Eric Dumazet <[email protected]>
This reverts commit
c67f5db82027ba6d2ea4ac9176bc45996a03ae6a.While using page fragments instead of a kmalloc backed
skb->headmight give a small performance improvement in some cases, there is a huge risk of memory use under estimation.
GOOD_COPY_LENis 128 bytes. This means that we need a small amount of memory to hold the headers and structskb_shared_infoYet,
napi_alloc_skb()might use a whole 32KB page (or 64KB on PowerPc) for long lived incoming TCP packets.We have been tracking OOM issues on GKE hosts hitting
tcp_memlimits but consuming far more memory for TCP buffers than instructed in tcp_mem[2]Even if we force
napi_alloc_skb()to only use order-0 pages, the issue would still be there on arches withPAGE_SIZE>= 32768Using
alloc_skb()and thus standardkmallloc()forskb->headallocations will get the benefit of letting other objects in each page being independently used by other skbs, regardless of the lifetime.Note that a similar problem exists for skbs allocated from
napi_get_frags(), this is handled in a separate patch.I would like to thank Greg Thelen for his precious help on this matter, analysing crash dumps is always a time consuming task.
Signed-off-by Eric Dumazet <[email protected]>Cc Paolo Abeni <[email protected]>Cc Michael S. Tsirkin <[email protected]>Cc Greg Thelen <[email protected]>--- drivers/net/virtio_net.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/net/virtio_net.c +1
-2@@ -386,7 +386,7 @@ static struct sk_buff *page_to_skb(struct virtnet_info *vi, p =page_address (page) + offset;/* copy small packet so we can reuse these pages for small data */ - skb =napi_alloc_skb (&rq->napi,GOOD_COPY_LEN ); + skb =netdev_alloc_skb_ip_align (vi->dev,GOOD_COPY_LEN );if (unlikely (!skb))return NULL ; --
Quoted 2 lines
2.30.0.284.gd98b1dd5eaa7-goog
Note that __netdev_alloc_skb() will also need to be changed.
Fitting 32 skb->head in a page without SLAB/SLUB help is simply too dangerous.
Michael S. Tsirkin
maintainerQuoted 8 lines · Eric Dumazet
From: Eric Dumazet <[email protected]>
This reverts commit
c67f5db82027ba6d2ea4ac9176bc45996a03ae6a.While using page fragments instead of a kmalloc backed
skb->headmight give a small performance improvement in some cases, there is a huge risk of memory use under estimation.
Quoted 8 lines
GOOD_COPY_LENis 128 bytes. This means that we need a small amount of memory to hold the headers and structskb_shared_infoYet,
napi_alloc_skb()might use a whole 32KB page (or 64KB on PowerPc) for long lived incoming TCP packets.We have been tracking OOM issues on GKE hosts hitting
tcp_memlimits but consuming far more memory for TCP buffers than instructed in tcp_mem[2]
Are you using virtio on the host then? Is this with a hardware virtio device? These do exist, guest is just more common, so I wanted to make sure this is not a mistake.
Quoted 10 lines
Even if we force
napi_alloc_skb()to only use order-0 pages, the issue would still be there on arches withPAGE_SIZE>= 32768Using
alloc_skb()and thus standardkmallloc()forskb->headallocations will get the benefit of letting other objects in each page being independently used by other skbs, regardless of the lifetime.Note that a similar problem exists for skbs allocated from
napi_get_frags(), this is handled in a separate patch.
Quoted 7 lines
I would like to thank Greg Thelen for his precious help on this matter, analysing crash dumps is always a time consuming task.
Signed-off-by Eric Dumazet <[email protected]>Cc Paolo Abeni <[email protected]>Cc Michael S. Tsirkin <[email protected]>Cc Greg Thelen <[email protected]>
Just curious - is the way virtio used napi_alloc_skb wrong somehow?
The idea was to benefit from better batching and less play with irq save
...
It would be helpful to improve the comments for napi_alloc_skb to make it clearer how to use it correctly.
Are other uses of napi_alloc_skb ok?
Quoted 19 lines
--- drivers/net/virtio_net.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/net/virtio_net.c +1
-1@@ -386,7 +386,7 @@ static struct sk_buff *page_to_skb(struct virtnet_info *vi, p =page_address (page) + offset;/* copy small packet so we can reuse these pages for small data */ - skb =napi_alloc_skb (&rq->napi,GOOD_COPY_LEN ); + skb =netdev_alloc_skb_ip_align (vi->dev,GOOD_COPY_LEN );if (unlikely (!skb))return NULL ;Signature · 1 line
2.30.0.284.gd98b1dd5eaa7-goog
Quoted 43 lines · Michael S. Tsirkin
Quoted 8 lines · Eric Dumazet
From: Eric Dumazet <[email protected]>
This reverts commit
c67f5db82027ba6d2ea4ac9176bc45996a03ae6a.While using page fragments instead of a kmalloc backed
skb->headmight give a small performance improvement in some cases, there is a huge risk of memory use under estimation.Quoted 8 lines
GOOD_COPY_LENis 128 bytes. This means that we need a small amount of memory to hold the headers and structskb_shared_infoYet,
napi_alloc_skb()might use a whole 32KB page (or 64KB on PowerPc) for long lived incoming TCP packets.We have been tracking OOM issues on GKE hosts hitting
tcp_memlimits but consuming far more memory for TCP buffers than instructed in tcp_mem[2]Are you using virtio on the host then? Is this with a hardware virtio device? These do exist, guest is just more common, so I wanted to make sure this is not a mistake.
Quoted 10 lines
Even if we force
napi_alloc_skb()to only use order-0 pages, the issue would still be there on arches withPAGE_SIZE>= 32768Using
alloc_skb()and thus standardkmallloc()forskb->headallocations will get the benefit of letting other objects in each page being independently used by other skbs, regardless of the lifetime.Note that a similar problem exists for skbs allocated from
napi_get_frags(), this is handled in a separate patch.Quoted 7 lines
I would like to thank Greg Thelen for his precious help on this matter, analysing crash dumps is always a time consuming task.
Signed-off-by Eric Dumazet <[email protected]>Cc Paolo Abeni <[email protected]>Cc Michael S. Tsirkin <[email protected]>Cc Greg Thelen <[email protected]>Just curious - is the way virtio used
napi_alloc_skbwrong somehow?
Quoted 7 lines
The idea was to benefit from better batching and less play with irq save
...It would be helpful to improve the comments for
napi_alloc_skbto make it clearer how to use it correctly.Are other uses of
napi_alloc_skbok?
Yeah, as I mentioned already, the real problem lies in use of fragments for skb->head in general,
especially for small buffers.
It is too dangerous to use 32KB pages to allocate ~1KB buffers.
I am working on it.
Quoted 43 lines · Eric Dumazet
From: Eric Dumazet <[email protected]>
This reverts commit
c67f5db82027ba6d2ea4ac9176bc45996a03ae6a.While using page fragments instead of a kmalloc backed
skb->headmight give a small performance improvement in some cases, there is a huge risk of memory use under estimation.
GOOD_COPY_LENis 128 bytes. This means that we need a small amount of memory to hold the headers and structskb_shared_infoYet,
napi_alloc_skb()might use a whole 32KB page (or 64KB on PowerPc) for long lived incoming TCP packets.We have been tracking OOM issues on GKE hosts hitting
tcp_memlimits but consuming far more memory for TCP buffers than instructed in tcp_mem[2]Even if we force
napi_alloc_skb()to only use order-0 pages, the issue would still be there on arches withPAGE_SIZE>= 32768Using
alloc_skb()and thus standardkmallloc()forskb->headallocations will get the benefit of letting other objects in each page being independently used by other skbs, regardless of the lifetime.Note that a similar problem exists for skbs allocated from
napi_get_frags(), this is handled in a separate patch.I would like to thank Greg Thelen for his precious help on this matter, analysing crash dumps is always a time consuming task.
Signed-off-by Eric Dumazet <[email protected]>Cc Paolo Abeni <[email protected]>Cc Michael S. Tsirkin <[email protected]>Cc Greg Thelen <[email protected]>--- drivers/net/virtio_net.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/net/virtio_net.c +0
-0
drivers/net/virtio_net.c static struct sk_buff *page_to_skb(struct virtnet_info *vi, · line 391
Quoted 7 lines
drivers/net/virtio_net.c +1
-1@@ -386,7 +386,7 @@ static struct sk_buff *page_to_skb(struct virtnet_info *vi, p =page_address (page) + offset;/* copy small packet so we can reuse these pages for small data */ - skb =napi_alloc_skb (&rq->napi,GOOD_COPY_LEN ); + skb =netdev_alloc_skb_ip_align (vi->dev,GOOD_COPY_LEN );if (unlikely (!skb))return NULL ;
I'm ok with the revert. The gain given by the original change in my tests was measurable, but small.
Quoted 56 lines · Paolo Abeni
Quoted 43 lines · Eric Dumazet
From: Eric Dumazet <[email protected]>
This reverts commit
c67f5db82027ba6d2ea4ac9176bc45996a03ae6a.While using page fragments instead of a kmalloc backed
skb->headmight give a small performance improvement in some cases, there is a huge risk of memory use under estimation.
GOOD_COPY_LENis 128 bytes. This means that we need a small amount of memory to hold the headers and structskb_shared_infoYet,
napi_alloc_skb()might use a whole 32KB page (or 64KB on PowerPc) for long lived incoming TCP packets.We have been tracking OOM issues on GKE hosts hitting
tcp_memlimits but consuming far more memory for TCP buffers than instructed in tcp_mem[2]Even if we force
napi_alloc_skb()to only use order-0 pages, the issue would still be there on arches withPAGE_SIZE>= 32768Using
alloc_skb()and thus standardkmallloc()forskb->headallocations will get the benefit of letting other objects in each page being independently used by other skbs, regardless of the lifetime.Note that a similar problem exists for skbs allocated from
napi_get_frags(), this is handled in a separate patch.I would like to thank Greg Thelen for his precious help on this matter, analysing crash dumps is always a time consuming task.
Signed-off-by Eric Dumazet <[email protected]>Cc Paolo Abeni <[email protected]>Cc Michael S. Tsirkin <[email protected]>Cc Greg Thelen <[email protected]>--- drivers/net/virtio_net.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/net/virtio_net.c +0
-0drivers/net/virtio_net.c static struct sk_buff *page_to_skb(struct virtnet_info *vi, · line 391
Quoted 7 lines
drivers/net/virtio_net.c +1
-1@@ -386,7 +386,7 @@ static struct sk_buff *page_to_skb(struct virtnet_info *vi, p =page_address (page) + offset;/* copy small packet so we can reuse these pages for small data */ - skb =napi_alloc_skb (&rq->napi,GOOD_COPY_LEN ); + skb =netdev_alloc_skb_ip_align (vi->dev,GOOD_COPY_LEN );if (unlikely (!skb))return NULL ;I'm ok with the revert. The gain given by the original change in my tests was measurable, but small.
Quoted 1 line
Acked-by Paolo Abeni <[email protected]>
To be clear, I now think the revert is not needed.
I will post instead a patch that should take care of the problem both for virtio and napi_get_frags() tiny skbs
Something like :
net/core/skbuff.c +0 -0
100644
net/core/skbuff.c +6 -2
@@ -501,13 +501,17 @@ EXPORT_SYMBOL(__netdev_alloc_skb);struct sk_buff *__napi_alloc_skb (struct napi_struct *napi,unsigned int len,gfp_t gfp_mask) { -struct napi_alloc_cache *nc =this_cpu_ptr (&napi_alloc_cache); +struct napi_alloc_cache *nc;struct sk_buff *skb;void *data; len +=NET_SKB_PAD +NET_IP_ALIGN ; -if ((len >SKB_WITH_OVERHEAD (PAGE_SIZE )) || +/* If requested length is either too small or too big, +* we use kmalloc() for skb->head allocation. +*/ +if (len <=SKB_WITH_OVERHEAD (1024 ) || + len >SKB_WITH_OVERHEAD (PAGE_SIZE ) || (gfp_mask & (__GFP_DIRECT_RECLAIM |GFP_DMA ))) { skb =__alloc_skb (len, gfp_mask,SKB_ALLOC_RX ,NUMA_NO_NODE );if (!skb)
@@ -515,6 +519,7 @@ struct sk_buff *__napi_alloc_skb(struct
napi_struct *napi, unsigned int len,
goto skb_success;
}+ nc = this_cpu_ptr(&napi_alloc_cache);
len += SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
len = SKB_DATA_ALIGN(len);Quoted 6 lines · Eric Dumazet
To be clear, I now think the revert is not needed.
I will post instead a patch that should take care of the problem both for virtio and
napi_get_frags()tiny skbs
Then later, we might change things so that the sk_buff cache being changed by Alexander Lobakin
can also be used for these allocations.
Michael S. Tsirkin
maintainerQuoted 11 lines · Eric Dumazet
Quoted 6 lines · Eric Dumazet
To be clear, I now think the revert is not needed.
I will post instead a patch that should take care of the problem both for virtio and
napi_get_frags()tiny skbsThen later, we might change things so that the
sk_buffcache being changed by Alexander Lobakin
can also be used for these allocations.
That sounds like a great idea.
Signature · 1 line
MST