swap over nfs -v21
Subsystems (22)
cover swap over nfs -v21
Hi,
Here's the latest version of swap over NFS series since -v20 last October. We decide to push this feature as it is useful for NAS or virt environment.
The patches are against the mmotm-2010-07-01. We can split the patchset into following parts:
Patch 1 - 12: provides a generic reserve framework. This framework could also be used to get rid of some of the __GFP_NOFAIL users.
Patch 13 - 15: Provide some generic network infrastructure needed later on.
Patch 16 - 21: reserve a little pool to act as a receive buffer, this allows us to inspect packets before tossing them.
Patch 22 - 23: Generic vm infrastructure to handle swapping to a filesystem instead of a block device.
Patch 24 - 27: convert NFS to make use of the new network and vm infrastructure to provide swap over NFS.
Patch 28 - 30: minor bug fixing with latest -mmotm.
[some history]
v19: http://lwn.net/Articles/301915/
v20: http://lwn.net/Articles/355350/
Changes since v20:
- rebased to mmotm-2010-07-01
- dropped the null pointer deref patch for the root cause is wrong SWP_FILE enum
- some minor build fixes
- fix a null pointer deref with mmotm-2010-07-01
- fix a bug when swap with multi files on the same nfs server
Regards
Xiaotian
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 1/30 mm: serialize access to min_free_kbytes
Quoted 1 line
From
bc55bacd6bcc0f8a69c0d7e0d554c78237233e07Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Mon, 12 Jul 2010 17:58:34 +0800
Subject: [PATCH 01/30] mm: serialize access to min_free_kbytes
There is a small race between the procfs caller and the memory hotplug caller of setup_per_zone_wmarks(). Not a big deal, but the next patch will add yet another caller. Time to close the gap.
--- mm/page_alloc.c | 16 +++++++++++++--- 1 files changed, 13 insertions(+), 3 deletions(-)
mm/page_alloc.c +13 -3
@@ -165,6 +165,7 @@ static char * const zone_names[MAX_NR_ZONES] = { "Movable", }; +static DEFINE_SPINLOCK (min_free_lock);int min_free_kbytes =1024 ;static unsigned long __meminitdata nr_kernel_pages;
@@ -4839,13 +4840,13 @@ static void setup_per_zone_lowmem_reserve(void) }/** -* setup_per_zone_wmarks - called when min_free_kbytes changes +* __setup_per_zone_wmarks - called when min_free_kbytes changes * or when memory is hot-{added|removed} * * Ensures that the watermark[min,low,high] values for each zone are set * correctly with respect to min_free_kbytes. */ -void setup_per_zone_wmarks (void ) +static void __setup_per_zone_wmarks (void ) {unsigned long pages_min = min_free_kbytes >> (PAGE_SHIFT -10 );unsigned long lowmem_pages =0 ;
@@ -4943,6 +4944,15 @@ static void __init setup_per_zone_inactive_ratio(void)calculate_zone_inactive_ratio (zone); } +void setup_per_zone_wmarks (void ) +{ +unsigned long flags; + +spin_lock_irqsave (&min_free_lock, flags); +__setup_per_zone_wmarks (); +spin_unlock_irqrestore (&min_free_lock, flags); +} +/* * Initialise min_free_kbytes. *
@@ -4978,7 +4988,7 @@ static int __init init_per_zone_wmark_min(void) min_free_kbytes =128 ;if (min_free_kbytes >65536 ) min_free_kbytes =65536 ; -setup_per_zone_wmarks (); +__setup_per_zone_wmarks ();setup_per_zone_lowmem_reserve ();setup_per_zone_inactive_ratio ();return 0 ;
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 2/30 Swap over network documentation
In reply to Xiaotian FengQuoted 1 line
From
8c68e4dc644be32cd82ba9711ba3ef89cb687cdfMon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Mon, 12 Jul 2010 17:59:16 +0800
Subject: [PATCH 02/30] Swap over network documentation
Document describing the problem and proposed solution
--- Documentation/network-swap.txt | 268 ++++++++++++++++++++++++++++++++++++++++ 1 files changed, 268 insertions(+), 0 deletions(-) create mode 100644 Documentation/network-swap.txt
Documentation/network-swap.txt new file +268 -0
@@ -0,0 +1,268 @@ + +Problem: + When Linux needs to allocate memory it may find that there is + insufficient free memory so it needs to reclaim space that is in + use but not needed at the moment. There are several options: + + 1/ Shrink a kernel cache such as the inode or dentry cache. This + is fairly easy but provides limited returns. + 2/ Discard 'clean' pages from the page cache. This is easy, and + works well as long as there are clean pages in the page cache. + Similarly clean 'anonymous' pages can be discarded - if there + are any. + 3/ Write out some dirty page-cache pages so that they become clean. + The VM limits the number of dirty page-cache pages to e.g. 40% + of available memory so that (among other reasons) a "sync" will + not take excessively long. So there should never be excessive + amounts of dirty pagecache. + Writing out dirty page-cache pages involves work by the + filesystem which may need to allocate memory itself. To avoid + deadlock, filesystems use GFP_NOFS when allocating memory on the + write-out path. When this is used, cleaning dirty page-cache + pages is not an option so if the filesystem finds that memory + is tight, another option must be found. + 4/ Write out dirty anonymous pages to the "Swap" partition/file. + This is the most interesting for a couple of reasons. + a/ Unlike dirty page-cache pages, there is no need to write anon + pages out unless we are actually short of memory. Thus they + tend to be left to last. + b/ Anon pages tend to be updated randomly and unpredictably, and + flushing them out of memory can have a very significant + performance impact on the process using them. This contrasts + with page-cache pages which are often written sequentially + and often treated as "write-once, read-many". + So anon pages tend to be left until last to be cleaned, and may + be the only cleanable pages while there are still some dirty + page-cache pages (which are waiting on a GFP_NOFS allocation). + +[I don't find the above wholly satisfying. There seems to be too much + hand-waving. If someone can provide better text explaining why + swapout is a special case, that would be great.] + +So we need to be able to write to the swap file/partition without +needing to allocate any memory ... or only a small well controlled +amount. + +The VM reserves a small amount of memory that can only be allocated +for use as part of the swap-out procedure. It is only available to +processes with the PF_MEMALLOC flag set, which is typically just the +memory cleaner. + +Traditionally swap-out is performed directly to block devices (swap +files on block-device filesystems are supported by examining the +mapping from file offset to device offset in advance, and then using +the device offsets to write directly to the device). Block devices +are (required to be) written to pre-allocate any memory that might be +needed during write-out, and to block when the pre-allocated memory is +exhausted and no other memory is available. They can be sure not to +block forever as the pre-allocated memory will be returned as soon as +the data it is being used for has been written out. The primary +mechanism for pre-allocating memory is called "mempools". + +This approach does not work for writing anonymous pages +(i.e. swapping) over a network, using e.g NFS or NBD or iSCSI. + + +The main reason that it does not work is that when data from an anon +page is written to the network, we must wait for a reply to confirm +the data is safe. Receiving that reply will consume memory and, +significantly, we need to allocate memory to an incoming packet before +we can tell if it is the reply we are waiting for or not. + +The secondary reason is that the network code is not written to use +mempools and in most cases does not need to use them. Changing all +allocations in the networking layer to use mempools would be quite +intrusive, and would waste memory, and probably cause a slow-down in +the common case of not swapping over the network. + +These problems are addressed by enhancing the system of memory +reserves used by PF_MEMALLOC and requiring any in-kernel networking +client that is used for swap-out to indicate which sockets are used +for swapout so they can be handled specially in low memory situations. + +There are several major parts to this enhancement: + +1/ page->reserve, GFP_MEMALLOC + + To handle low memory conditions we need to know when those + conditions exist. Having a global "low on memory" flag seems easy, + but its implementation is problematic. Instead we make it possible + to tell if a recent memory allocation required use of the emergency + memory pool. + For pages returned by alloc_page, the new page->reserve flag + can be tested. If this is set, then a low memory condition was + current when the page was allocated, so the memory should be used + carefully. (Because low memory conditions are transient, this + state is kept in an overloaded member instead of in page flags, which + would suggest a more permanent state.) + + For memory allocated using slab/slub: If a page that is added to a + kmem_cache is found to have page->reserve set, then a s->reserve + flag is set for the whole kmem_cache. Further allocations will only + be returned from that page (or any other page in the cache) if they + are emergency allocation (i.e. PF_MEMALLOC or GFP_MEMALLOC is set). + Non-emergency allocations will block in alloc_page until a + non-reserve page is available. Once a non-reserve page has been + added to the cache, the s->reserve flag on the cache is removed. + + Because slab objects have no individual state its hard to pass + reserve state along, the current code relies on a regular alloc + failing. There are various allocation wrappers help here. + + This allows us to + a/ request use of the emergency pool when allocating memory + (GFP_MEMALLOC), and + b/ to find out if the emergency pool was used. + +2/ SK_MEMALLOC, sk_buff->emergency. + + When memory from the reserve is used to store incoming network + packets, the memory must be freed (and the packet dropped) as soon + as we find out that the packet is not for a socket that is used for + swap-out. + To achieve this we have an ->emergency flag for skbs, and an + SK_MEMALLOC flag for sockets. + When memory is allocated for an skb, it is allocated with + GFP_MEMALLOC (if we are currently swapping over the network at + all). If a subsequent test shows that the emergency pool was used, + ->emergency is set. + When the skb is finally attached to its destination socket, the + SK_MEMALLOC flag on the socket is tested. If the skb has + ->emergency set, but the socket does not have SK_MEMALLOC set, then + the skb is immediately freed and the packet is dropped. + This ensures that reserve memory is never queued on a socket that is + not used for swapout. + + Similarly, if an skb is ever queued for delivery to user-space for + example by netfilter, the ->emergency flag is tested and the skb is + released if ->emergency is set. (so obviously the storage route may + not pass through a userspace helper, otherwise the packets will never + arrive and we'll deadlock) + + This ensures that memory from the emergency reserve can be used to + allow swapout to proceed, but will not get caught up in any other + network queue. + + +3/ pages_emergency + + The above would be sufficient if the total memory below the lowest + memory watermark (i.e the size of the emergency reserve) were known + to be enough to hold all transient allocations needed for writeout. + I'm a little blurry on how big the current emergency pool is, but it + isn't big and certainly hasn't been sized to allow network traffic + to consume any. + + We could simply make the size of the reserve bigger. However in the + common case that we are not swapping over the network, that would be + a waste of memory. + + So a new "watermark" is defined: pages_emergency. This is + effectively added to the current low water marks, so that pages from + this emergency pool can only be allocated if one of PF_MEMALLOC or + GFP_MEMALLOC are set. + + pages_emergency can be changed dynamically based on need. When + swapout over the network is required, pages_emergency is increased + to cover the maximum expected load. When network swapout is + disabled, pages_emergency is decreased. + + To determine how much to increase it by, we introduce reservation + groups.... + +3a/ reservation groups + + The memory used transiently for swapout can be in a number of + different places. e.g. the network route cache, the network + fragment cache, in transit between network card and socket, or (in + the case of NFS) in sunrpc data structures awaiting a reply. + We need to ensure each of these is limited in the amount of memory + they use, and that the maximum is included in the reserve. + + The memory required by the network layer only needs to be reserved + once, even if there are multiple swapout paths using the network + (e.g. NFS and NDB and iSCSI, though using all three for swapout at + the same time would be unusual). + + So we create a tree of reservation groups. The network might + register a collection of reservations, but not mark them as being in + use. NFS and sunrpc might similarly register a collection of + reservations, and attach it to the network reservations as it + depends on them. + When swapout over NFS is requested, the NFS/sunrpc reservations are + activated which implicitly activates the network reservations. + + The total new reservation is added to pages_emergency. + + Provided each memory usage stays beneath the registered limit (at + least when allocating memory from reserves), the system will never + run out of emergency memory, and swapout will not deadlock. + + It is worth noting here that it is not critical that each usage + stays beneath the limit 100% of the time. Occasional excess is + acceptable provided that the memory will be freed again within a + short amount of time that does *not* require waiting for any event + that itself might require memory. + This is because, at all stages of transmit and receive, it is + acceptable to discard all transient memory associated with a + particular writeout and try again later. On transmit, the page can + be re-queued for later transmission. On receive, the packet can be + dropped assuming that the peer will resend after a timeout. + + Thus allocations that are truly transient and will be freed without + blocking do not strictly need to be reserved for. Doing so might + still be a good idea to ensure forward progress doesn't take too + long. + +4/ low-mem accounting + + Most places that might hold on to emergency memory (e.g. route + cache, fragment cache etc) already place a limit on the amount of + memory that they can use. This limit can simply be reserved using + the above mechanism and no more needs to be done. + + However some memory usage might not be accounted with sufficient + firmness to allow an appropriate emergency reservation. The + in-flight skbs for incoming packets is one such example. + + To support this, a low-overhead mechanism for accounting memory + usage against the reserves is provided. This mechanism uses the + same data structure that is used to store the emergency memory + reservations through the addition of a 'usage' field. + + Before we attempt allocation from the memory reserves, we much check + if the resulting 'usage' is below the reservation. If so, we increase + the usage and attempt the allocation (which should succeed). If + the projected 'usage' exceeds the reservation we'll either fail the + allocation, or wait for 'usage' to decrease enough so that it would + succeed, depending on __GFP_WAIT. + + When memory that was allocated for that purpose is freed, the + 'usage' field is checked again. If it is non-zero, then the size of + the freed memory is subtracted from the usage, making sure the usage + never becomes less than zero. + + This provides adequate accounting with minimal overheads when not in + a low memory condition. When a low memory condition is encountered + it does add the cost of a spin lock necessary to serialise updates + to 'usage'. + + + +5/ swapon/swapoff/swap_out/swap_in + + So that a filesystem (e.g. NFS) can know when to set SK_MEMALLOC on + any network socket that it uses, and can know when to account + reserve memory carefully, new address_space_operations are + available. + "swapon" requests that an address space (i.e a file) be make ready + for swapout. swap_out and swap_in request the actual IO. They + together must ensure that each swap_out request can succeed without + allocating more emergency memory that was reserved by swapon. swapoff + is used to reverse the state changes caused by swapon when we disable + the swap file. + + +Thanks for reading this far. I hope it made sense :-) + +Neil Brown (with updates from Peter Zijlstra)
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 3/30 mm: expose gfp_to_alloc_flags()
In reply to Xiaotian FengQuoted 1 line
From
8a554f86ab2fd4d8a1daecaef4cc5bb3901c4423Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Mon, 12 Jul 2010 17:59:52 +0800
Subject: [PATCH 03/30] mm: expose gfp_to_alloc_flags()
Expose the gfp to alloc_flags mapping, so we can use it in other parts of the vm.
--- mm/internal.h | 15 +++++++++++++++ mm/page_alloc.c | 16 +--------------- 2 files changed, 16 insertions(+), 15 deletions(-)
mm/internal.h +15 -0
@@ -185,6 +185,21 @@ static inline struct page *mem_map_next(struct page *iter,#define __paginginit __init#endif +/* The ALLOC_WMARK bits are used as an index to zone->watermark */ +#define ALLOC_WMARK_MIN WMARK_MIN +#define ALLOC_WMARK_LOW WMARK_LOW +#define ALLOC_WMARK_HIGH WMARK_HIGH +#define ALLOC_NO_WATERMARKS 0x04/* don't check watermarks at all */ + +/* Mask to get the watermark bits */ +#define ALLOC_WMARK_MASK (ALLOC_NO_WATERMARKS-1) + +#define ALLOC_HARDER 0x10/* try to alloc harder */ +#define ALLOC_HIGH 0x20/* __GFP_HIGH set */ +#define ALLOC_CPUSET 0x40/* check for correct cpuset */ + +int gfp_to_alloc_flags (gfp_t gfp_mask); +/* Memory initialisation debug and verification */ enum mminit_level {MMINIT_WARNING ,
mm/page_alloc.c +1 -15
@@ -1345,19 +1345,6 @@ failed:return NULL ; } -/* The ALLOC_WMARK bits are used as an index to zone->watermark */ -#define ALLOC_WMARK_MIN WMARK_MIN -#define ALLOC_WMARK_LOW WMARK_LOW -#define ALLOC_WMARK_HIGH WMARK_HIGH -#define ALLOC_NO_WATERMARKS 0x04/* don't check watermarks at all */ - -/* Mask to get the watermark bits */ -#define ALLOC_WMARK_MASK (ALLOC_NO_WATERMARKS-1) - -#define ALLOC_HARDER 0x10/* try to alloc harder */ -#define ALLOC_HIGH 0x20/* __GFP_HIGH set */ -#define ALLOC_CPUSET 0x40/* check for correct cpuset */ - #ifdef CONFIG_FAIL_PAGE_ALLOC static struct fail_page_alloc_attr {
@@ -1911,8 +1898,7 @@ void wake_all_kswapd(unsigned int order, struct zonelist *zonelist,wakeup_kswapd (zone, order); } -static inline int -gfp_to_alloc_flags (gfp_t gfp_mask) +int gfp_to_alloc_flags (gfp_t gfp_mask) {struct task_struct *p = current;int alloc_flags =ALLOC_WMARK_MIN |ALLOC_CPUSET ;
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 4/30 mm: tag reseve pages
In reply to Xiaotian FengQuoted 1 line
From
14f7823986429b8374d18e3f648cd84575296e03Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Mon, 12 Jul 2010 18:00:37 +0800
Subject: [PATCH 04/30] mm: tag reseve pages
Tag pages allocated from the reserves with a non-zero page->reserve. This allows us to distinguish and account reserve pages.
Since low-memory situations are transient, and unrelated the the actual page (any page can be on the freelist when we run low), don't mark the page in any permanent way - just pass along the information to the allocatee.
--- include/linux/mm_types.h | 1 + mm/page_alloc.c | 4 +++- 2 files changed, 4 insertions(+), 1 deletions(-)
include/linux/mm_types.h +1 -0
@@ -71,6 +71,7 @@ struct page {union {pgoff_t index;/* Our offset within mapping. */ void *freelist;/* SLUB: freelist req. slab lock */ +int reserve;/* page_alloc: page is a reserve page */ };struct list_head lru;/* Pageout list, eg. active_list * protected by zone->lru_lock !
mm/page_alloc.c +3 -1
@@ -1656,8 +1656,10 @@ zonelist_scan: try_this_zone: page =buffered_rmqueue (preferred_zone, zone, order, gfp_mask, migratetype); -if (page) +if (page) { + page->reserve = !!(alloc_flags &ALLOC_NO_WATERMARKS );break ; + } this_zone_full:if (NUMA_BUILD )zlc_mark_zone_full (zonelist, z);
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 5/30 mm: sl[au]b: add knowledge of reserve pages
In reply to Xiaotian FengQuoted 1 line
From
fba0bdebc34d3db41a2c975eb38e9548ea5c2ed1Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 10:40:05 +0800
Subject: [PATCH 05/30] mm: sl[au]b: add knowledge of reserve pages
Restrict objects from reserve slabs (ALLOC_NO_WATERMARKS) to allocation contexts that are entitled to it. This is done to ensure reserve pages don't leak out and get consumed.
The basic pattern used for all # allocators is the following, for each active slab page we store if it came from an emergency allocation. When we find it did, make sure the current allocation context would have been able to allocate page from the emergency reserves as well. In that case allow the allocation. If not, force a new slab allocation. When that works the memory pressure has lifted enough to allow this context to get an object, otherwise fail the allocation.
--- include/linux/slub_def.h | 1 + mm/slab.c | 62 +++++++++++++++++++++++++++++++++++++++------ mm/slob.c | 16 +++++++++++- mm/slub.c | 42 ++++++++++++++++++++++++++----- 4 files changed, 104 insertions(+), 17 deletions(-)
include/linux/slub_def.h +1 -0
@@ -39,6 +39,7 @@ struct kmem_cache_cpu {void **freelist;/* Pointer to first free per cpu object */ struct page *page;/* The slab from which we are allocating */ int node;/* The node of the page (or -1 for debug) */ +int reserve;/* Did the current page come from the reserve */ #ifdef CONFIG_SLUB_STATS unsigned stat[NR_SLUB_STAT_ITEMS ];#endif
mm/slab.c +53 -9
186 lines not on this page · raw message
mm/slob.c +15 -1
@@ -71,6 +71,7 @@#include <trace/events/kmem.h> #include <asm/atomic.h> +#include "internal.h" /* * slob_block has a field 'units', which indicates size of block if +ve,
@@ -193,6 +194,11 @@ struct slob_rcu {static DEFINE_SPINLOCK (slob_lock);/* +* tracks the reserve state for the allocator. +*/ +static int slob_reserve; + +/* * Encode the given size and next info into a free slob block s. */ static void set_slob (slob_t *s,slobidx_t size,slob_t *next)
@@ -242,7 +248,7 @@ static int slob_last(slob_t *s)static void *slob_new_pages (gfp_t gfp,int order,int node) { -void *page; +struct page *page;#ifdef CONFIG_NUMA if (node !=-1 )
@@ -254,6 +260,8 @@ static void *slob_new_pages(gfp_t gfp, int order, int node)if (!page)return NULL ; + slob_reserve = page->reserve; +return page_address (page); }
@@ -326,6 +334,11 @@ static void *slob_alloc(size_t size, gfp_t gfp, int align, int node)slob_t *b =NULL ;unsigned long flags; +if (unlikely (slob_reserve)) { +if (!(gfp_to_alloc_flags (gfp) &ALLOC_NO_WATERMARKS )) + goto grow; + } +if (size <SLOB_BREAK1 ) slob_list = &free_slob_small;else if (size <SLOB_BREAK2 )
@@ -364,6 +377,7 @@ static void *slob_alloc(size_t size, gfp_t gfp, int align, int node) }spin_unlock_irqrestore (&slob_lock, flags); +grow:/* Not enough space: must allocate a new page */ if (!b) { b =slob_new_pages (gfp & ~__GFP_ZERO,0 , node);
mm/slub.c +35 -7
108 lines not on this page · raw message
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH -mmotm 05/30] mm: sl[au]b: add knowledge of reserve pages
Hi Xiaotian!
I would actually prefer that the SLAB, SLOB, and SLUB changes were in separate patches to make reviewing easier.
Looking at SLUB:
Quoted 39 lines · Xiaotian Feng
mm/slub.c +7
-1@@ -27,6 +27,8 @@ #include <linux/memory.h> #include <linux/math64.h> #include <linux/fault-inject.h> +#include "internal.h" + /** Lock order: @@ -1139,7 +1141,8 @@ static void setup_object(struct kmem_cache *s, struct page *page, s->ctor(object); } -static struct page *new_slab (struct kmem_cache *s,gfp_t flags,int node) +static +struct page *new_slab (struct kmem_cache *s,gfp_t flags,int node,int *reserve) { struct page *page; void *start;@@ -1153,6 +1156,8 @@ static struct page *new_slab(struct kmem_cache *s, gfp_t flags, int node) if (!page) goto out; + *reserve = page->reserve; + inc_slabs_node(s, page_to_nid(page), page->objects); page->slab = s; page->flags |= 1 << PG_slab;@@ -1606,10 +1611,20 @@ static void *__slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, { void **object; struct page *new; + int reserve; /* We handle __GFP_ZERO in the caller */
mm/slub.c static void *__slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, · line 1626
Quoted 10 lines
mm/slub.c +8
-0gfpflags &= ~__GFP_ZERO; + if (unlikely(c->reserve)) { + /* + * If the current slab is a reserve slab and the current + * allocation context does not allow access to the reserves we + * must force an allocation to test the current levels. + */ + if (!(gfp_to_alloc_flags(gfpflags) & ALLOC_NO_WATERMARKS)) + goto grow_slab;
OK, so assume that:
(1) c->reserve is set to one
(2) GFP flags don't allow dipping into the reserves
(3) we've managed to free enough pages so normal allocations are fine
(4) the page from reserves is not yet empty
we will call flush_slab() and put the "emergency page" on partial list and clear c->reserve. This effectively means that now some other allocation can fetch the partial page and start to use it. Is this OK? Who makes sure the emergency reserves are large enough for the next out-of-memory condition where we swap over NFS?
Quoted 74 lines
mm/slub.c +20
-7+ } if (!c->page) goto new_slab;@@ -1623,8 +1638,8 @@ load_freelist: object = c->page->freelist; if (unlikely(!object)) goto another_slab; - if (unlikely(SLABDEBUG && PageSlubDebug(c->page))) - goto debug; + if (unlikely(SLABDEBUG && PageSlubDebug(c->page) || c->reserve)) + goto slow_path; c->freelist = get_freepointer(s, object); c->page->inuse = c->page->objects;@@ -1646,16 +1661,18 @@ new_slab: goto load_freelist; } +grow_slab: if (gfpflags & __GFP_WAIT) local_irq_enable(); - new = new_slab(s, gfpflags, node); + new = new_slab(s, gfpflags, node, &reserve); if (gfpflags & __GFP_WAIT) local_irq_disable(); if (new) { c = __this_cpu_ptr(s->cpu_slab); + c->reserve = reserve; stat(s, ALLOC_SLAB); if (c->page) flush_slab(s, c);@@ -1667,10 +1684,20 @@ new_slab: if (!(gfpflags & __GFP_NOWARN) && printk_ratelimit()) slab_out_of_memory(s, gfpflags, node); return NULL; -debug: - if (!alloc_debug_processing(s, c->page, object, addr)) + +slow_path: + if (!c->reserve && !alloc_debug_processing(s, c->page, object, addr)) goto another_slab; + /* + * Avoid the slub fast path in slab_alloc() by not setting + * c->freelist and the fast path in slab_free() by making + * node_match() fail by setting c->node to -1. + * + * We use this for for debug and reserve checks which need + * to be done for each allocation. + */ + c->page->inuse++; c->page->freelist = get_freepointer(s, object); c->node = -1;@@ -2095,10 +2122,11 @@ static void early_kmem_cache_node_alloc(gfp_t gfpflags, int node) struct page *page; struct kmem_cache_node *n; unsigned long flags; + int reserve; BUG_ON(kmalloc_caches->size < sizeof(struct kmem_cache_node)); - page = new_slab(kmalloc_caches, gfpflags, node); + page = new_slab(kmalloc_caches, gfpflags, node, &reserve); BUG_ON(!page); if (page_to_nid(page) != node) { --1.7.1.1
Quoted 6 lines
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected] For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH -mmotm 05/30] mm: sl[au]b: add knowledge of reserve pages
Quoted 67 lines · Pekka Enberg
Hi Xiaotian!
I would actually prefer that the SLAB, SLOB, and SLUB changes were in separate patches to make reviewing easier.
Looking at SLUB:
Quoted 39 lines · Xiaotian Feng
mm/slub.c +7
-1@@ -27,6 +27,8 @@#include <linux/memory.h> #include <linux/math64.h> #include <linux/fault-inject.h> +#include "internal.h" +/* * Lock order: @@ -1139,7 +1141,8 @@ static void setup_object(struct kmem_cache *s, struct page *page, s->ctor (object); } -static struct page *new_slab (struct kmem_cache *s,gfp_t flags,int node) +static +struct page *new_slab (struct kmem_cache *s,gfp_t flags,int node,int *reserve) {struct page *page;void *start;@@ -1153,6 +1156,8 @@ static struct page *new_slab(struct kmem_cache *s, gfp_t flags, int node)if (!page) goto out; + *reserve = page->reserve; +inc_slabs_node (s,page_to_nid (page), page->objects); page->slab = s; page->flags |=1 << PG_slab;@@ -1606,10 +1611,20 @@ static void *__slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, {void **object;struct page *new; +int reserve;/* We handle __GFP_ZERO in the caller */ mm/slub.c static void *__slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, · line 1626
Quoted 10 lines
mm/slub.c +8
-0gfpflags&= ~__GFP_ZERO; +if (unlikely (c->reserve)) { +/* +* If the current slab is a reserve slab and the current +* allocation context does not allow access to the reserves we +* must force an allocation to test the current levels. +*/ +if (!(gfp_to_alloc_flags (gfpflags)&ALLOC_NO_WATERMARKS )) + goto grow_slab;OK, so assume that:
(1)
c->reserveis set to one(2) GFP flags don't allow dipping into the reserves
(3) we've managed to free enough pages so normal allocations are fine
Quoted 8 lines
(4) the page from reserves is not yet empty
we will call
flush_slab()and put the "emergency page" on partial list and clearc->reserve. This effectively means that now some other allocation can fetch the partial page and start to use it. Is this OK? Who makes sure the emergency reserves are large enough for the next out-of-memory condition where we swap over NFS?
Good catch. I'm just wondering if above check is necessary. For "emergency page", we don't set c->freelist. How can we get a reserved slab, if GPF flags don't allow dipping into reserves?
Quoted 74 lines
Quoted 74 lines
mm/slub.c +20
-7+ }if (!c->page) goto new_slab;@@ -1623,8 +1638,8 @@ load_freelist: object = c->page->freelist;if (unlikely (!object)) goto another_slab; -if (unlikely (SLABDEBUG &&PageSlubDebug (c->page))) - goto debug; +if (unlikely (SLABDEBUG &&PageSlubDebug (c->page) || c->reserve)) + goto slow_path; c->freelist =get_freepointer (s, object); c->page->inuse = c->page->objects;@@ -1646,16 +1661,18 @@ new_slab: goto load_freelist; } +grow_slab:if (gfpflags& __GFP_WAIT)local_irq_enable (); - new =new_slab (s, gfpflags, node); + new =new_slab (s, gfpflags, node,&reserve);if (gfpflags& __GFP_WAIT)local_irq_disable ();if (new) { c =__this_cpu_ptr (s->cpu_slab); + c->reserve = reserve;stat (s,ALLOC_SLAB );if (c->page)flush_slab (s, c);@@ -1667,10 +1684,20 @@ new_slab:if (!(gfpflags& __GFP_NOWARN)&&printk_ratelimit ())slab_out_of_memory (s, gfpflags, node);return NULL ; -debug: -if (!alloc_debug_processing (s, c->page, object, addr)) + +slow_path: +if (!c->reserve&& !alloc_debug_processing (s, c->page, object, addr)) goto another_slab; +/* +* Avoid the slub fast path in slab_alloc() by not setting +* c->freelist and the fast path in slab_free() by making +* node_match() fail by setting c->node to -1. +* +* We use this for for debug and reserve checks which need +* to be done for each allocation. +*/ + c->page->inuse++; c->page->freelist =get_freepointer (s, object); c->node =-1 ;@@ -2095,10 +2122,11 @@ static void early_kmem_cache_node_alloc(gfp_t gfpflags, int node)struct page *page;struct kmem_cache_node *n;unsigned long flags; +int reserve;BUG_ON (kmalloc_caches->size<sizeof (struct kmem_cache_node )); - page =new_slab (kmalloc_caches, gfpflags, node); + page =new_slab (kmalloc_caches, gfpflags, node,&reserve);BUG_ON (!page);if (page_to_nid (page) != node) { --1.7.1.1
Quoted 7 lines
Quoted 6 lines
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email:<a href=mailto:"[email protected]"> [email protected]</a>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH -mmotm 05/30] mm: sl[au]b: add knowledge of reserve pages
In reply to Pekka EnbergQuoted 67 lines · On Tue, 13 Jul 2010 23:33:14 +0300 Pekka Enberg <[email protected]> wrote:
Hi Xiaotian!
I would actually prefer that the SLAB, SLOB, and SLUB changes were in separate patches to make reviewing easier.
Looking at SLUB:
Quoted 39 lines · Xiaotian Feng
mm/slub.c +7
-1@@ -27,6 +27,8 @@ #include <linux/memory.h> #include <linux/math64.h> #include <linux/fault-inject.h> +#include "internal.h" + /** Lock order: @@ -1139,7 +1141,8 @@ static void setup_object(struct kmem_cache *s, struct page *page, s->ctor(object); } -static struct page *new_slab (struct kmem_cache *s,gfp_t flags,int node) +static +struct page *new_slab (struct kmem_cache *s,gfp_t flags,int node,int *reserve) { struct page *page; void *start;@@ -1153,6 +1156,8 @@ static struct page *new_slab(struct kmem_cache *s, gfp_t flags, int node) if (!page) goto out; + *reserve = page->reserve; + inc_slabs_node(s, page_to_nid(page), page->objects); page->slab = s; page->flags |= 1 << PG_slab;@@ -1606,10 +1611,20 @@ static void *__slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, { void **object; struct page *new; + int reserve; /* We handle __GFP_ZERO in the caller */mm/slub.c static void *__slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, · line 1626
Quoted 10 lines
mm/slub.c +8
-0gfpflags &= ~__GFP_ZERO; + if (unlikely(c->reserve)) { + /* + * If the current slab is a reserve slab and the current + * allocation context does not allow access to the reserves we + * must force an allocation to test the current levels. + */ + if (!(gfp_to_alloc_flags(gfpflags) & ALLOC_NO_WATERMARKS)) + goto grow_slab;OK, so assume that:
(1)
c->reserveis set to one(2) GFP flags don't allow dipping into the reserves
(3) we've managed to free enough pages so normal allocations are fine
Quoted 7 lines
(4) the page from reserves is not yet empty
we will call
flush_slab()and put the "emergency page" on partial list and clearc->reserve. This effectively means that now some other allocation can fetch the partial page and start to use it. Is this OK? Who makes sure the emergency reserves are large enough for the next out-of-memory condition where we swap over NFS?
Yes, this is OK. The emergency reserves are maintained at a lower level - within alloc_page.
The fact that (3) normal allocations are fine means that there are enough free pages to satisfy any swap-out allocation - so any pages that were previously allocated as 'emergency' pages can have their emergency status forgotten (the emergency has passed).
This is a subtle but important aspect of the emergency reservation scheme in swap-over-NFS. It is the act-of-allocating that is emergency-or-not. The memory itself, once allocated, is not special.
c->reserve means "the last page allocated required an emergency allocation". This means that parts of that page, or any other page, can only be given as emergency allocations. Once the slab succeeds at a non-emergency allocation, the flag should obviously be cleared.
Similarly the page->reserve flag does not mean "this is a reserve page", but simply "when this page was allocated, it was an emergency allocation". The flag is often soon lost as it is in a union with e.g. freelist. But that doesn't matter as it is only really meaningful at the moment of allocation.
I hope that clarifies the situation,
NeilBrown
Quoted 75 lines
Quoted 75 lines
mm/slub.c +20
-7+ } if (!c->page) goto new_slab;@@ -1623,8 +1638,8 @@ load_freelist: object = c->page->freelist; if (unlikely(!object)) goto another_slab; - if (unlikely(SLABDEBUG && PageSlubDebug(c->page))) - goto debug; + if (unlikely(SLABDEBUG && PageSlubDebug(c->page) || c->reserve)) + goto slow_path; c->freelist = get_freepointer(s, object); c->page->inuse = c->page->objects;@@ -1646,16 +1661,18 @@ new_slab: goto load_freelist; } +grow_slab: if (gfpflags & __GFP_WAIT) local_irq_enable(); - new = new_slab(s, gfpflags, node); + new = new_slab(s, gfpflags, node, &reserve); if (gfpflags & __GFP_WAIT) local_irq_disable(); if (new) { c = __this_cpu_ptr(s->cpu_slab); + c->reserve = reserve; stat(s, ALLOC_SLAB); if (c->page) flush_slab(s, c);@@ -1667,10 +1684,20 @@ new_slab: if (!(gfpflags & __GFP_NOWARN) && printk_ratelimit()) slab_out_of_memory(s, gfpflags, node); return NULL; -debug: - if (!alloc_debug_processing(s, c->page, object, addr)) + +slow_path: + if (!c->reserve && !alloc_debug_processing(s, c->page, object, addr)) goto another_slab; + /* + * Avoid the slub fast path in slab_alloc() by not setting + * c->freelist and the fast path in slab_free() by making + * node_match() fail by setting c->node to -1. + * + * We use this for for debug and reserve checks which need + * to be done for each allocation. + */ + c->page->inuse++; c->page->freelist = get_freepointer(s, object); c->node = -1;@@ -2095,10 +2122,11 @@ static void early_kmem_cache_node_alloc(gfp_t gfpflags, int node) struct page *page; struct kmem_cache_node *n; unsigned long flags; + int reserve; BUG_ON(kmalloc_caches->size < sizeof(struct kmem_cache_node)); - page = new_slab(kmalloc_caches, gfpflags, node); + page = new_slab(kmalloc_caches, gfpflags, node, &reserve); BUG_ON(!page); if (page_to_nid(page) != node) { --1.7.1.1
Quoted 10 lines
Quoted 6 lines
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 6/30 mm: kmem_alloc_estimate()
In reply to Xiaotian FengQuoted 1 line
From
4a2dff5cf02e9d7f6ee9345c337697c4ab66c6dcMon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 10:41:22 +0800
Subject: [PATCH 06/30] mm: kmem_alloc_estimate()
Provide a method to get the upper bound on the pages needed to allocate a given number of objects from a given kmem_cache.
This lays the foundation for a generic reserve framework as presented in a later patch in this series. This framework needs to convert object demand (kmalloc() bytes, kmem_cache_alloc() objects) to pages.
--- include/linux/slab.h | 4 ++ mm/slab.c | 75 +++++++++++++++++++++++++++++++++++++++++++ mm/slob.c | 67 ++++++++++++++++++++++++++++++++++++++ mm/slub.c | 87 ++++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 233 insertions(+), 0 deletions(-)
include/linux/slab.h +4 -0
@@ -108,6 +108,8 @@ unsigned int kmem_cache_size(struct kmem_cache *);const char *kmem_cache_name (struct kmem_cache *);int kern_ptr_validate (const void *ptr,unsigned long size);int kmem_ptr_validate (struct kmem_cache *cachep,const void *ptr); +unsigned kmem_alloc_estimate (struct kmem_cache *cachep, +gfp_t flags,int objects);/* * Please use this macro to create slab caches. Simply specify the
@@ -144,6 +146,8 @@ void * __must_check krealloc(const void *, size_t, gfp_t);void kfree (const void *);void kzfree (const void *);size_t ksize (const void *); +unsigned kmalloc_estimate_objs (size_t ,gfp_t ,int ); +unsigned kmalloc_estimate_bytes (gfp_t ,size_t );/* * Allocator specific definitions. These are mainly used to establish optimized
mm/slab.c +75 -0
81 lines not on this page · raw message
mm/slob.c +67 -0
73 lines not on this page · raw message
mm/slub.c +87 -0
99 lines not on this page · raw message
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 7/30 mm: allow PF_MEMALLOC from softirq context
In reply to Xiaotian FengQuoted 1 line
From
32f474cffc76551ddb792454845bd473634219b5Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 10:41:53 +0800
Subject: [PATCH 07/30] mm: allow PF_MEMALLOC from softirq context
This is needed to allow network softirq packet processing to make use of PF_MEMALLOC.
Currently softirq context cannot use PF_MEMALLOC due to it not being associated with a task, and therefore not having task flags to fiddle with - thus the gfp to alloc flag mapping ignores the task flags when in interrupts (hard or soft) context.
Allowing softirqs to make use of PF_MEMALLOC therefore requires some trickery. We basically borrow the task flags from whatever process happens to be preempted by the softirq.
So we modify the gfp to alloc flags mapping to not exclude task flags in softirq context, and modify the softirq code to save, clear and restore the PF_MEMALLOC flag.
The save and clear, ensures the preempted task's PF_MEMALLOC flag doesn't leak into the softirq. The restore ensures a softirq's PF_MEMALLOC flag cannot leak back into the preempted process.
--- include/linux/sched.h | 7 +++++++ kernel/softirq.c | 3 +++ mm/page_alloc.c | 7 ++++--- 3 files changed, 14 insertions(+), 3 deletions(-)
include/linux/sched.h +7 -0
@@ -1765,6 +1765,13 @@ static inline void rcu_copy_process(struct task_struct *p)#endif +static inline void tsk_restore_flags (struct task_struct *p, +unsigned long pflags,unsigned long mask) +{ + p->flags &= ~mask; + p->flags |= pflags & mask; +} +#ifdef CONFIG_SMP extern int set_cpus_allowed_ptr (struct task_struct *p,const struct cpumask *new_mask);
kernel/softirq.c +3 -0
@@ -194,6 +194,8 @@ asmlinkage void __do_softirq(void)__u32 pending;int max_restart =MAX_SOFTIRQ_RESTART ;int cpu; +unsigned long pflags = current->flags; + current->flags &= ~PF_MEMALLOC ; pending =local_softirq_pending ();account_system_vtime (current);
@@ -246,6 +248,7 @@ restart:account_system_vtime (current);_local_bh_enable (); +tsk_restore_flags (current, pflags,PF_MEMALLOC ); }#ifndef __ARCH_HAS_DO_SOFTIRQ
mm/page_alloc.c +4 -3
@@ -1928,9 +1928,10 @@ int gfp_to_alloc_flags(gfp_t gfp_mask) alloc_flags |=ALLOC_HARDER ;if (likely (!(gfp_mask & __GFP_NOMEMALLOC))) { -if (!in_interrupt () && - ((p->flags &PF_MEMALLOC ) || -unlikely (test_thread_flag (TIF_MEMDIE )))) +if (!in_irq () && (p->flags &PF_MEMALLOC )) + alloc_flags |=ALLOC_NO_WATERMARKS ; +else if (!in_interrupt () && +unlikely (test_thread_flag (TIF_MEMDIE ))) alloc_flags |=ALLOC_NO_WATERMARKS ; }
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 8/30 mm: emergency pool
In reply to Xiaotian FengQuoted 1 line
From
973ce2ee797025f055ccb61359180e53c3ecc681Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 10:45:48 +0800
Subject: [PATCH 08/30] mm: emergency pool
Provide means to reserve a specific amount of pages.
The emergency pool is separated from the min watermark because ALLOC_HARDER and ALLOC_HIGH modify the watermark in a relative way and thus do not ensure a strict minimum.
--- include/linux/mmzone.h | 3 ++ mm/page_alloc.c | 84 ++++++++++++++++++++++++++++++++++++++++++------ mm/vmstat.c | 6 ++-- 3 files changed, 80 insertions(+), 13 deletions(-)
include/linux/mmzone.h +3 -0
@@ -279,6 +279,7 @@ struct zone_reclaim_stat {struct zone {/* Fields commonly accessed by the page allocator */ +unsigned long pages_emerg;/* emergency pool */ /* zone watermarks, access with *_wmark_pages(zone) macros */ unsigned long watermark[NR_WMARK ];
@@ -770,6 +771,8 @@ int sysctl_min_unmapped_ratio_sysctl_handler(struct ctl_table *, int,int sysctl_min_slab_ratio_sysctl_handler (struct ctl_table *,int ,void __user *,size_t *,loff_t *); +int adjust_memalloc_reserve (int pages); +extern int numa_zonelist_order_handler (struct ctl_table *,int ,void __user *,size_t *,loff_t *);extern char numa_zonelist_order[];
mm/page_alloc.c +74 -10
154 lines not on this page · raw message
mm/vmstat.c +3 -3
@@ -814,9 +814,9 @@ static void zoneinfo_show_print(struct seq_file *m, pg_data_t *pgdat,"\n spanned %lu" "\n present %lu" ,zone_page_state (zone,NR_FREE_PAGES ), -min_wmark_pages (zone), -low_wmark_pages (zone), -high_wmark_pages (zone), + zone->pages_emerg +min_wmark_pages (zone), + zone->pages_emerg +min_wmark_pages (zone), + zone->pages_emerg +high_wmark_pages (zone), zone->pages_scanned, zone->spanned_pages, zone->present_pages);
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 9/30 mm: system wide ALLOC_NO_WATERMARK
In reply to Xiaotian FengQuoted 1 line
From
6d9ae36dfe34cdbc6e1fc52e6db0b27286eb4b58Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 10:46:50 +0800
Subject: [PATCH 09/30] mm: system wide ALLOC_NO_WATERMARK
The reserve is proportionally distributed over all (!highmem) zones in the system. So we need to allow an emergency allocation access to all zones. In order to do that we need to break out of any mempolicy boundaries we might have.
In my opinion that does not break mempolicies as those are user oriented and not system oriented. That is, system allocations are not guaranteed to be within mempolicy boundaries. For instance IRQs don't even have a mempolicy.
So breaking out of mempolicy boundaries for 'rare' emergency allocations, which are always system allocations (as opposed to user) is ok.
--- mm/page_alloc.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-)
mm/page_alloc.c +5 -0
@@ -1995,6 +1995,11 @@ restart: rebalance:/* Allocate without watermarks if the context allows */ if (alloc_flags &ALLOC_NO_WATERMARKS ) { +/* +* break out mempolicy boundaries +*/ + zonelist =node_zonelist (numa_node_id (), gfp_mask); + page =__alloc_pages_high_priority (gfp_mask, order, zonelist, high_zoneidx, nodemask, preferred_zone, migratetype);
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 10/30 mm: __GFP_MEMALLOC
In reply to Xiaotian FengQuoted 1 line
From
78618cc83e6a21b876c30a8fb3940ccc1f5b99e1Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 10:49:16 +0800
Subject: [PATCH 10/30] mm: __GFP_MEMALLOC
__GFP_MEMALLOC will allow the allocation to disregard the watermarks, much like PF_MEMALLOC.
It allows one to pass along the memalloc state in object related allocation flags as opposed to task related flags, such as sk->sk_allocation.
--- include/linux/gfp.h | 3 ++- mm/page_alloc.c | 4 +++- 2 files changed, 5 insertions(+), 2 deletions(-)
include/linux/gfp.h +2 -1
@@ -47,6 +47,7 @@ struct vm_area_struct;#define __GFP_REPEAT ((__force gfp_t)0x400u)/* See above */ #define __GFP_NOFAIL ((__force gfp_t)0x800u)/* See above */ #define __GFP_NORETRY ((__force gfp_t)0x1000u)/* See above */ +#define __GFP_MEMALLOC ((__force gfp_t)0x2000u)/* Use emergency reserves */ #define __GFP_COMP ((__force gfp_t)0x4000u)/* Add compound page metadata */ #define __GFP_ZERO ((__force gfp_t)0x8000u)/* Return zeroed page on success */ #define __GFP_NOMEMALLOC ((__force gfp_t)0x10000u)/* Don't use emergency reserves */
@@ -98,7 +99,7 @@ struct vm_area_struct;/* Control page allocator reclaim behavior */ #define GFP_RECLAIM_MASK (__GFP_WAIT|__GFP_HIGH|__GFP_IO|__GFP_FS|\ __GFP_NOWARN|__GFP_REPEAT|__GFP_NOFAIL|\ - __GFP_NORETRY|__GFP_NOMEMALLOC) + __GFP_NORETRY|__GFP_MEMALLOC|__GFP_NOMEMALLOC)/* Control slab gfp mask during early boot */ #define GFP_BOOT_MASK (__GFP_BITS_MASK & ~(__GFP_WAIT|__GFP_IO|__GFP_FS))
mm/page_alloc.c +3 -1
@@ -1930,7 +1930,9 @@ int gfp_to_alloc_flags(gfp_t gfp_mask) alloc_flags |=ALLOC_HARDER ;if (likely (!(gfp_mask & __GFP_NOMEMALLOC))) { -if (!in_irq () && (p->flags &PF_MEMALLOC )) +if (gfp_mask & __GFP_MEMALLOC) + alloc_flags |=ALLOC_NO_WATERMARKS ; +else if (!in_irq () && (p->flags &PF_MEMALLOC )) alloc_flags |=ALLOC_NO_WATERMARKS ;else if (!in_interrupt () &&unlikely (test_thread_flag (TIF_MEMDIE )))
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 11/30 mm: memory reserve management
In reply to Xiaotian FengQuoted 1 line
From
c598d3bbb8b5e900ad0f8397b1acbe922003ac5dMon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 11:01:48 +0800
Subject: [PATCH 11/30] mm: memory reserve management
Generic reserve management code.
It provides methods to reserve and charge. Upon this, generic alloc/free style reserve pools could be build, which could fully replace mempool_t functionality.
It should also allow for a Banker's algorithm replacement of __GFP_NOFAIL.
--- include/linux/reserve.h | 198 +++++++++++++++ include/linux/slab.h | 25 +- mm/Makefile | 2 +- mm/reserve.c | 637 +++++++++++++++++++++++++++++++++++++++++++++++ mm/slub.c | 2 +- 5 files changed, 851 insertions(+), 13 deletions(-) create mode 100644 include/linux/reserve.h create mode 100644 mm/reserve.c
include/linux/reserve.h new file +198 -0
@@ -0,0 +1,198 @@ +/* +* Memory reserve management. +* +* Copyright (C) 2007-2008 Red Hat, Inc., +* Peter Zijlstra <[email protected]> +* +* This file contains the public data structure and API definitions. +*/ + +#ifndef _LINUX_RESERVE_H +#define _LINUX_RESERVE_H + +#include <linux/list.h> +#include <linux/spinlock.h> +#include <linux/wait.h> +#include <linux/slab.h> + +struct mem_reserve { +struct mem_reserve *parent; +struct list_head children; +struct list_head siblings; + +const char *name; + +long pages; +long limit; +long usage; +spinlock_t lock;/* protects limit and usage */ + +wait_queue_head_t waitqueue; +}; + +extern struct mem_reserve mem_reserve_root; + +void mem_reserve_init (struct mem_reserve *res,const char *name, +struct mem_reserve *parent); +int mem_reserve_connect (struct mem_reserve *new_child, +struct mem_reserve *node); +void mem_reserve_disconnect (struct mem_reserve *node); + +int mem_reserve_pages_set (struct mem_reserve *res,long pages); +int mem_reserve_pages_add (struct mem_reserve *res,long pages); +int mem_reserve_pages_charge (struct mem_reserve *res,long pages); + +int mem_reserve_kmalloc_set (struct mem_reserve *res,long bytes); +int mem_reserve_kmalloc_charge (struct mem_reserve *res,long bytes); + +struct kmem_cache ; + +int mem_reserve_kmem_cache_set (struct mem_reserve *res, +struct kmem_cache *s, +int objects); +int mem_reserve_kmem_cache_charge (struct mem_reserve *res, +struct kmem_cache *s,long objs); + +void *___kmalloc_reserve (size_t size,gfp_t flags,int node,unsigned long ip, +struct mem_reserve *res,int *emerg); + +static inline +void *__kmalloc_reserve (size_t size,gfp_t flags,int node,unsigned long ip, +struct mem_reserve *res,int *emerg) +{ +void *obj; + + obj =__kmalloc_node_track_caller (size, + flags | __GFP_NOMEMALLOC | __GFP_NOWARN, node, ip); +if (!obj) + obj =___kmalloc_reserve (size, flags, node, ip, res, emerg); + +return obj; +} + +/** +* kmalloc_reserve() - kmalloc() and charge against @res for @emerg allocations +* @size - size of the requested memory region +* @gfp - allocation flags to use for this allocation +* @node - preferred memory node for this allocation +* @res - reserve to charge emergency allocations against +* @emerg - bit 0 is set when the allocation was an emergency allocation +* +* Returns NULL on failure +*/ +#define kmalloc_reserve (size, gfp, node, res, emerg) \ + __kmalloc_reserve(size, gfp, node, \ + _RET_IP_, res, emerg) + +void __kfree_reserve (void *obj,struct mem_reserve *res,int emerg); + +/** +* kfree_reserve() - kfree() and uncharge against @res for @emerg allocations +* @obj - memory to free +* @res - reserve to uncharge emergency allocations from +* @emerg - was this an emergency allocation +*/ +static inline +void kfree_reserve (void *obj,struct mem_reserve *res,int emerg) +{ +if (unlikely (obj && res && emerg)) +__kfree_reserve (obj, res, emerg); +else +kfree (obj); +} + +void *__kmem_cache_alloc_reserve (struct kmem_cache *s,gfp_t flags,int node, +struct mem_reserve *res,int *emerg); + +/** +* kmem_cache_alloc_reserve() - kmem_cache_alloc() and charge against @res +* @s - kmem_cache to allocate from +* @gfp - allocation flags to use for this allocation +* @node - preferred memory node for this allocation +* @res - reserve to charge emergency allocations against +* @emerg - bit 0 is set when the allocation was an emergency allocation +* +* Returns NULL on failure +*/ +static inline +void *kmem_cache_alloc_reserve (struct kmem_cache *s,gfp_t flags,int node, +struct mem_reserve *res,int *emerg) +{ +void *obj; + + obj =kmem_cache_alloc_node (s, + flags | __GFP_NOMEMALLOC | __GFP_NOWARN, node); +if (!obj) + obj =__kmem_cache_alloc_reserve (s, flags, node, res, emerg); + +return obj; +} + +void __kmem_cache_free_reserve (struct kmem_cache *s,void *obj, +struct mem_reserve *res,int emerg); + +/** +* kmem_cache_free_reserve() - kmem_cache_free() and uncharge against @res +* @s - kmem_cache to free to +* @obj - memory to free +* @res - reserve to uncharge emergency allocations from +* @emerg - was this an emergency allocation +*/ +static inline +void kmem_cache_free_reserve (struct kmem_cache *s,void *obj, +struct mem_reserve *res,int emerg) +{ +if (unlikely (obj && res && emerg)) +__kmem_cache_free_reserve (s, obj, res, emerg); +else +kmem_cache_free (s, obj); +} + +struct page *__alloc_pages_reserve (int node,gfp_t flags,int order, +struct mem_reserve *res,int *emerg); + +/** +* alloc_pages_reserve() - alloc_pages() and charge against @res +* @node - preferred memory node for this allocation +* @gfp - allocation flags to use for this allocation +* @order - page order +* @res - reserve to charge emergency allocations against +* @emerg - bit 0 is set when the allocation was an emergency allocation +* +* Returns NULL on failure +*/ +static inline +struct page *alloc_pages_reserve (int node,gfp_t flags,int order, +struct mem_reserve *res,int *emerg) +{ +struct page *page; + + page =alloc_pages_node (node, + flags | __GFP_NOMEMALLOC | __GFP_NOWARN, order); +if (!page) + page =__alloc_pages_reserve (node, flags, order, res, emerg); + +return page; +} + +void __free_pages_reserve (struct page *page,int order, +struct mem_reserve *res,int emerg); + +/** +* free_pages_reserve() - __free_pages() and uncharge against @res +* @page - page to free +* @order - page order +* @res - reserve to uncharge emergency allocations from +* @emerg - was this an emergency allocation +*/ +static inline +void free_pages_reserve (struct page *page,int order, +struct mem_reserve *res,int emerg) +{ +if (unlikely (page && res && emerg)) +__free_pages_reserve (page, order, res, emerg); +else +__free_pages (page, order); +} + +#endif /* _LINUX_RESERVE_H */
include/linux/slab.h +14 -11
@@ -272,15 +272,17 @@ static inline void *kmem_cache_alloc_node(struct kmem_cache *cachep,* allocator where we care about the real place the memory allocation * request comes from. */ -#if defined(CONFIG_DEBUG_SLAB ) || defined(CONFIG_SLUB ) +#if defined(CONFIG_DEBUG_SLAB ) || defined(CONFIG_SLUB ) || \ + (defined(CONFIG_SLAB ) && defined(CONFIG_TRACING ))extern void *__kmalloc_track_caller (size_t ,gfp_t ,unsigned long ); -#define kmalloc_track_caller (size, flags) \ - __kmalloc_track_caller(size, flags, _RET_IP_)#else -#define kmalloc_track_caller (size, flags) \ +#define __kmalloc_track_caller (size, flags, ip) \ __kmalloc(size, flags)#endif /* DEBUG_SLAB */ +#define kmalloc_track_caller (size, flags) \ + __kmalloc_track_caller(size, flags, _RET_IP_) +#ifdef CONFIG_NUMA /* * kmalloc_node_track_caller is a special version of kmalloc_node that
@@ -290,23 +292,24 @@ extern void *__kmalloc_track_caller(size_t, gfp_t, unsigned long);* standard allocator where we care about the real place the memory * allocation request comes from. */ -#if defined(CONFIG_DEBUG_SLAB ) || defined(CONFIG_SLUB ) +#if defined(CONFIG_DEBUG_SLAB ) || defined(CONFIG_SLUB ) || \ + (defined(CONFIG_SLAB ) && defined(CONFIG_TRACING ))extern void *__kmalloc_node_track_caller (size_t ,gfp_t ,int ,unsigned long ); -#define kmalloc_node_track_caller (size, flags, node) \ - __kmalloc_node_track_caller(size, flags, node, \ - _RET_IP_)#else -#define kmalloc_node_track_caller (size, flags, node) \ +#define __kmalloc_node_track_caller (size, flags, node, ip) \ __kmalloc_node(size, flags, node)#endif #else /* CONFIG_NUMA */ -#define kmalloc_node_track_caller (size, flags, node) \ - kmalloc_track_caller(size, flags) +#define __kmalloc_node_track_caller (size, flags, node, ip) \ + __kmalloc_track_caller(size, flags, ip)#endif /* CONFIG_NUMA */ +#define kmalloc_node_track_caller (size, flags, node) \ + __kmalloc_node_track_caller(size, flags, node, \ + _RET_IP_)/* * Shortcuts */
mm/Makefile +1 -1
@@ -11,7 +11,7 @@ obj-y := bootmem.o filemap.o mempool.o oom_kill.o fadvise.o \ maccess.o page_alloc.o page-writeback.o \ readahead.o swap.o truncate.o vmscan.o shmem.o \ prio_tree.o util.o mmzone.o vmstat.o backing-dev.o \ - page_isolation.o mm_init.o mmu_context.o \ + page_isolation.o mm_init.o mmu_context.o reserve.o \ $(mmu-y) obj-y += init-mm.o
mm/reserve.c new file +637 -0
637 lines not on this page · raw message
mm/slub.c +1 -1
@@ -2799,6 +2799,7 @@ void *__kmalloc(size_t size, gfp_t flags) }EXPORT_SYMBOL (__kmalloc); +#ifdef CONFIG_NUMA static void *kmalloc_large_node (size_t size,gfp_t flags,int node) {struct page *page;
@@ -2813,7 +2814,6 @@ static void *kmalloc_large_node(size_t size, gfp_t flags, int node)return ptr; } -#ifdef CONFIG_NUMA void *__kmalloc_node (size_t size,gfp_t flags,int node) {struct kmem_cache *s;
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 12/30 selinux: tag avc cache alloc as non-critical
In reply to Xiaotian FengQuoted 1 line
From
6c3a91091b2910c23908a9f9953efcf3df14e522Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 11:02:41 +0800
Subject: [PATCH 12/30] selinux: tag avc cache alloc as non-critical
Failing to allocate a cache entry will only harm performance not correctness. Do not consume valuable reserve pages for something like that.
--- security/selinux/avc.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
security/selinux/avc.c +1 -1
@@ -284,7 +284,7 @@ static struct avc_node *avc_alloc_node(void) {struct avc_node *node; - node =kmem_cache_zalloc (avc_node_cachep,GFP_ATOMIC ); + node =kmem_cache_zalloc (avc_node_cachep,GFP_ATOMIC |__GFP_NOMEMALLOC);if (!node) goto out;
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH -mmotm 12/30] selinux: tag avc cache alloc as non-critical
Quoted 1 line · Xiaotian Feng
=46rom
6c3a91091b2910c23908a9f9953efcf3df14e522Mon Sep 17 00:00:00 =
2001
Quoted 5 lines
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 11:02:41 +0800
Subject: [PATCH 12/30] selinux: tag avc cache alloc as non-critical =20
Failing to allocate a cache entry will only harm performance not =
correctness.
Quoted 17 lines
Do not consume valuable reserve pages for something like that. =20
Signed-off-by Peter Zijlstra <[email protected]>Signed-off-by Suresh Jayaraman <[email protected]>Signed-off-by Xiaotian Feng <[email protected]>---
security/selinux/avc.c| 2 +-1 files changed, 1 insertions(+), 1 deletions(-)=20
security/selinux/avc.c +0
-0@@ -284,7 +284,7 @@ static struct avc_node *avc_alloc_node(void) {struct avc_node *node;=20
security/selinux/avc.c
Quoted 2 lines
security/selinux/avc.c +1
-1- node =3D kmem_cache_zalloc(avc_node_cachep, GFP_ATOMIC); + node =3D kmem_cache_zalloc(avc_node_cachep, =
GFP_ATOMIC|__GFP_NOMEMALLOC);
Quoted 6 lines
if (!node) goto out;=20
--=20
1.7.1.1
=20
Why not just replace GFP_ATOMIC with GFP_NOWAIT?
This would NOT consume the valuable last pages.
Mitchell Erblich
Quoted 4 lines
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH -mmotm 12/30] selinux: tag avc cache alloc as non-critical
Quoted 31 lines · Mitchell Erblich
Quoted 28 lines · Xiaotian Feng
From
6c3a91091b2910c23908a9f9953efcf3df14e522Mon Sep 17 00:00:00 2001
From: Xiaotian Feng<[email protected]>
Date: Tue, 13 Jul 2010 11:02:41 +0800
Subject: [PATCH 12/30] selinux: tag avc cache alloc as non-criticalFailing to allocate a cache entry will only harm performance not correctness. Do not consume valuable reserve pages for something like that.
Signed-off-by Peter Zijlstra<[email protected]>Signed-off-by Suresh Jayaraman<[email protected]>Signed-off-by Xiaotian Feng<[email protected]>---
security/selinux/avc.c| 2 +-1 files changed, 1 insertions(+), 1 deletions(-)security/selinux/avc.c +1
-1@@ -284,7 +284,7 @@ static struct avc_node *avc_alloc_node(void) {struct avc_node *node; - node =kmem_cache_zalloc (avc_node_cachep,GFP_ATOMIC ); + node =kmem_cache_zalloc (avc_node_cachep,GFP_ATOMIC |__GFP_NOMEMALLOC);if (!node) goto out;
Quoted 7 lines
Quoted 3 lines
Signature · 1 line
1.7.1.1Why not just replace
GFP_ATOMICwithGFP_NOWAIT?This would NOT consume the valuable last pages.
But replace GFP_ATOMIC with GFP_NOWAIT can not prevent avc_alloc_node consume reserved pages.
Quoted 8 lines
Mitchell Erblich
Quoted 4 lines
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 13/30 net: packet split receive api
In reply to Xiaotian FengQuoted 1 line
From
8d908090b5314bed0c3318d82891b8c3bbf27815Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 11:03:55 +0800
Subject: [PATCH 13/30] net: packet split receive api
Add some packet-split receive hooks.
For one this allows to do NUMA node affine page allocs. Later on these hooks will be extended to do emergency reserve allocations for fragments.
Thanks to Jiri Bohac for fixing a bug in bnx2.
--- drivers/net/bnx2.c | 9 +++------ drivers/net/e1000e/netdev.c | 7 ++----- drivers/net/igb/igb_main.c | 6 +----- drivers/net/ixgbe/ixgbe_main.c | 14 ++++++-------- drivers/net/sky2.c | 16 ++++++---------- 5 files changed, 18 insertions(+), 34 deletions(-)
drivers/net/bnx2.c +3 -6
@@ -2670,7 +2670,7 @@ bnx2_alloc_rx_page(struct bnx2 *bp, struct bnx2_rx_ring_info *rxr, u16 index)struct sw_pg *rx_pg = &rxr->rx_pg_ring[index];struct rx_bd *rxbd = &rxr->rx_pg_desc_ring[RX_RING (index)][RX_IDX (index)]; -struct page *page =alloc_page (GFP_ATOMIC ); +struct page *page =netdev_alloc_page (bp->dev);if (!page)return -ENOMEM ;
@@ -2700,7 +2700,7 @@ bnx2_free_rx_page(struct bnx2 *bp, struct bnx2_rx_ring_info *rxr, u16 index)pci_unmap_page (bp->pdev,dma_unmap_addr (rx_pg, mapping),PAGE_SIZE ,PCI_DMA_FROMDEVICE ); -__free_page (page); +netdev_free_page (bp->dev, page); rx_pg->page =NULL ; }
@@ -3035,7 +3035,7 @@ bnx2_rx_skb(struct bnx2 *bp, struct bnx2_rx_ring_info *rxr, struct sk_buff *skb,if (i == pages -1 ) frag_len -=4 ; -skb_fill_page_desc (skb, i, rx_pg->page,0 , frag_len); +skb_add_rx_frag (skb, i, rx_pg->page,0 , frag_len); rx_pg->page =NULL ; err = bnx2_alloc_rx_page(bp, rxr,
@@ -3052,9 +3052,6 @@ bnx2_rx_skb(struct bnx2 *bp, struct bnx2_rx_ring_info *rxr, struct sk_buff *skb, PAGE_SIZE, PCI_DMA_FROMDEVICE); frag_size -= frag_len; - skb->data_len += frag_len; - skb->truesize += frag_len; - skb->len += frag_len; pg_prod =NEXT_RX_BD (pg_prod); pg_cons =RX_PG_RING_IDX (NEXT_RX_BD (pg_cons));
drivers/net/e1000e/netdev.c +2 -5
@@ -604,7 +604,7 @@ static void e1000_alloc_rx_buffers_ps(struct e1000_adapter *adapter,continue ; }if (!ps_page->page) { - ps_page->page =alloc_page (GFP_ATOMIC ); + ps_page->page =netdev_alloc_page (netdev);if (!ps_page->page) { adapter->alloc_rx_buff_failed++; goto no_buffers;
@@ -1185,11 +1185,8 @@ static bool e1000_clean_rx_irq_ps(struct e1000_adapter *adapter,dma_unmap_page (&pdev->dev, ps_page->dma,PAGE_SIZE ,DMA_FROM_DEVICE ); ps_page->dma =0 ; -skb_fill_page_desc (skb, j, ps_page->page,0 , length); +skb_add_rx_frag (skb, j, ps_page->page,0 , length); ps_page->page =NULL ; - skb->len += length; - skb->data_len += length; - skb->truesize += length; }/* strip the ethernet crc, problem is we're using pages now so
drivers/net/igb/igb_main.c +1 -5
@@ -5574,7 +5574,7 @@ static bool igb_clean_rx_irq_adv(struct igb_q_vector *q_vector, PAGE_SIZE / 2, DMA_FROM_DEVICE); buffer_info->page_dma =0 ; -skb_fill_page_desc (skb,skb_shinfo (skb)->nr_frags, +skb_add_rx_frag (skb,skb_shinfo (skb)->nr_frags, buffer_info->page, buffer_info->page_offset, length);
@@ -5584,10 +5584,6 @@ static bool igb_clean_rx_irq_adv(struct igb_q_vector *q_vector, buffer_info->page =NULL ; else get_page(buffer_info->page); - - skb->len += length; - skb->data_len += length; - skb->truesize += length; }if (!(staterr &E1000_RXD_STAT_EOP )) {
drivers/net/ixgbe/ixgbe_main.c +6 -8
@@ -1037,6 +1037,7 @@ static void ixgbe_alloc_rx_buffers(struct ixgbe_adapter *adapter, int cleaned_count) {struct pci_dev *pdev = adapter->pdev; +struct net_device *netdev = adapter->netdev;union ixgbe_adv_rx_desc *rx_desc;struct ixgbe_rx_buffer *bi;unsigned int i;
@@ -1050,7 +1051,7 @@ static void ixgbe_alloc_rx_buffers(struct ixgbe_adapter *adapter,if (!bi->page_dma && (rx_ring->flags &IXGBE_RING_RX_PS_ENABLED )) {if (!bi->page) { - bi->page =alloc_page (GFP_ATOMIC ); + bi->page =netdev_alloc_page (netdev);if (!bi->page) { adapter->alloc_rx_page_failed++; goto no_buffers;
@@ -1242,10 +1243,10 @@ static bool ixgbe_clean_rx_irq(struct ixgbe_q_vector *q_vector,dma_unmap_page (&pdev->dev, rx_buffer_info->page_dma,PAGE_SIZE /2 ,DMA_FROM_DEVICE ); rx_buffer_info->page_dma =0 ; -skb_fill_page_desc (skb,skb_shinfo (skb)->nr_frags, - rx_buffer_info->page, - rx_buffer_info->page_offset, - upper_len); +skb_add_rx_frag (skb,skb_shinfo (skb)->nr_frags, + rx_buffer_info->page, + rx_buffer_info->page_offset, + upper_len);if ((rx_ring->rx_buf_len > (PAGE_SIZE /2 )) || (page_count (rx_buffer_info->page) !=1 ))
@@ -1253,9 +1254,6 @@ static bool ixgbe_clean_rx_irq(struct ixgbe_q_vector *q_vector,else get_page (rx_buffer_info->page); - skb->len += upper_len; - skb->data_len += upper_len; - skb->truesize += upper_len; } i++;
drivers/net/sky2.c +6 -10
@@ -1394,7 +1394,7 @@ static struct sk_buff *sky2_rx_alloc(struct sky2_port *sky2)skb_reserve (skb,NET_IP_ALIGN );for (i =0 ; i < sky2->rx_nfrags; i++) { -struct page *page =alloc_page (GFP_ATOMIC ); +struct page *page =netdev_alloc_page (sky2->netdev);if (!page) goto free_partial;
@@ -2353,8 +2353,8 @@ static struct sk_buff *receive_copy(struct sky2_port *sky2, }/* Adjust length of skb with fragments to match received data */ -static void skb_put_frags (struct sk_buff *skb,unsigned int hdr_space, -unsigned int length) +static void skb_put_frags (struct sky2_port *sky2,struct sk_buff *skb, +unsigned int hdr_space,unsigned int length) {int i, num_frags;unsigned int size;
@@ -2371,15 +2371,11 @@ static void skb_put_frags(struct sk_buff *skb, unsigned int hdr_space,if (length ==0 ) {/* don't need this page */ -__free_page (frag->page); +netdev_free_page (sky2->netdev, frag->page); --skb_shinfo (skb)->nr_frags; }else { size =min (length, (unsigned )PAGE_SIZE ); - - frag->size = size; - skb->data_len += size; - skb->truesize += size; - skb->len += size; +skb_add_rx_frag (skb, i, frag->page,0 , size); length -= size; } }
@@ -2407,7 +2403,7 @@ static struct sk_buff *receive_new(struct sky2_port *sky2, *re = nre;if (skb_shinfo (skb)->nr_frags) -skb_put_frags (skb, hdr_space, length); +skb_put_frags (sky2, skb, hdr_space, length);else skb_put (skb, length);return skb;
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 14/30 net: sk_allocation() - concentrate socket related allocations
In reply to Xiaotian FengQuoted 1 line
From
3bc4f5211d8716267891ff85385177f181e418eaMon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 11:04:45 +0800
Subject: [PATCH 14/30] net: sk_allocation() - concentrate socket related allocations
Introduce sk_allocation(), this function allows to inject sock specific flags to each sock related allocation.
--- include/linux/skbuff.h | 3 +++ include/net/sock.h | 5 +++++ net/ipv4/tcp.c | 3 ++- net/ipv4/tcp_output.c | 11 ++++++----- net/ipv6/tcp_ipv6.c | 15 +++++++++++---- 5 files changed, 27 insertions(+), 10 deletions(-)
include/linux/skbuff.h +3 -0
@@ -1119,6 +1119,9 @@ static inline void skb_fill_page_desc(struct sk_buff *skb, int i,extern void skb_add_rx_frag (struct sk_buff *skb,int i,struct page *page,int off,int size); +extern void skb_add_rx_frag (struct sk_buff *skb,int i,struct page *page, +int off,int size); +#define SKB_PAGE_ASSERT (skb) BUG_ON(skb_shinfo(skb)->nr_frags)#define SKB_FRAG_ASSERT (skb) BUG_ON(skb_has_frags(skb))#define SKB_LINEAR_ASSERT (skb) BUG_ON(skb_is_nonlinear(skb))
include/net/sock.h +5 -0
@@ -563,6 +563,11 @@ static inline int sock_flag(struct sock *sk, enum sock_flags flag)return test_bit (flag, &sk->sk_flags); } +static inline gfp_t sk_allocation (struct sock *sk,gfp_t gfp_mask) +{ +return gfp_mask; +} +static inline void sk_acceptq_removed (struct sock *sk) { sk->sk_ack_backlog--;
net/ipv4/tcp.c +2 -1
@@ -683,7 +683,8 @@ struct sk_buff *sk_stream_alloc_skb(struct sock *sk, int size, gfp_t gfp)/* The TCP header must be at least 32-bit aligned. */ size =ALIGN (size,4 ); - skb =alloc_skb_fclone (size + sk->sk_prot->max_header, gfp); + skb =alloc_skb_fclone (size + sk->sk_prot->max_header, +sk_allocation (sk, gfp));if (skb) {if (sk_wmem_schedule (sk, skb->truesize)) {/*
net/ipv4/tcp_output.c +6 -5
@@ -2307,7 +2307,7 @@ void tcp_send_fin(struct sock *sk)/* Socket is locked, keep trying until memory is available. */ for (;;) { skb =alloc_skb_fclone (MAX_TCP_HEADER , - sk->sk_allocation); +sk_allocation (sk, sk->sk_allocation));if (skb)break ;yield ();
@@ -2333,7 +2333,7 @@ void tcp_send_active_reset(struct sock *sk, gfp_t priority)struct sk_buff *skb;/* NOTE: No TCP options attached and we never retransmit this. */ - skb =alloc_skb (MAX_TCP_HEADER , priority); + skb =alloc_skb (MAX_TCP_HEADER ,sk_allocation (sk, priority));if (!skb) {NET_INC_STATS (sock_net (sk),LINUX_MIB_TCPABORTFAILED );return ;
@@ -2406,7 +2406,8 @@ struct sk_buff *tcp_make_synack(struct sock *sk, struct dst_entry *dst,if (cvp !=NULL && cvp->s_data_constant && cvp->s_data_desired) s_data_desired = cvp->s_data_desired; - skb =sock_wmalloc (sk,MAX_TCP_HEADER +15 + s_data_desired,1 ,GFP_ATOMIC ); + skb =sock_wmalloc (sk,MAX_TCP_HEADER +15 + s_data_desired,1 , +sk_allocation (sk,GFP_ATOMIC ));if (skb ==NULL )return NULL ;
@@ -2686,7 +2687,7 @@ void tcp_send_ack(struct sock *sk)* tcp_transmit_skb() will set the ownership to this * sock. */ - buff =alloc_skb (MAX_TCP_HEADER ,GFP_ATOMIC ); + buff =alloc_skb (MAX_TCP_HEADER ,sk_allocation (sk,GFP_ATOMIC ));if (buff ==NULL ) {inet_csk_schedule_ack (sk);inet_csk (sk)->icsk_ack.ato =TCP_ATO_MIN ;
@@ -2721,7 +2722,7 @@ static int tcp_xmit_probe_skb(struct sock *sk, int urgent)struct sk_buff *skb;/* We don't queue it, tcp_transmit_skb() sets ownership. */ - skb =alloc_skb (MAX_TCP_HEADER ,GFP_ATOMIC ); + skb =alloc_skb (MAX_TCP_HEADER ,sk_allocation (sk,GFP_ATOMIC ));if (skb ==NULL )return -1 ;
net/ipv6/tcp_ipv6.c +11 -4
@@ -589,7 +589,8 @@ static int tcp_v6_md5_do_add(struct sock *sk, struct in6_addr *peer, }else {/* reallocate new list if current one is full. */ if (!tp->md5sig_info) { - tp->md5sig_info =kzalloc (sizeof (*tp->md5sig_info),GFP_ATOMIC ); + tp->md5sig_info =kzalloc (sizeof (*tp->md5sig_info), +sk_allocation (sk,GFP_ATOMIC ));if (!tp->md5sig_info) {kfree (newkey);return -ENOMEM ;
@@ -602,7 +603,8 @@ static int tcp_v6_md5_do_add(struct sock *sk, struct in6_addr *peer, }if (tp->md5sig_info->alloced6 == tp->md5sig_info->entries6) { keys =kmalloc ((sizeof (tp->md5sig_info->keys6[0 ]) * - (tp->md5sig_info->entries6 +1 )),GFP_ATOMIC ); + (tp->md5sig_info->entries6 +1 )), +sk_allocation (sk,GFP_ATOMIC ));if (!keys) {tcp_free_md5sig_pool ();
@@ -726,7 +728,8 @@ static int tcp_v6_parse_md5_keys (struct sock *sk, char __user *optval,struct tcp_sock *tp =tcp_sk (sk);struct tcp_md5sig_info *p; - p =kzalloc (sizeof (struct tcp_md5sig_info ),GFP_KERNEL ); + p =kzalloc (sizeof (struct tcp_md5sig_info ), +sk_allocation (sk,GFP_KERNEL ));if (!p)return -ENOMEM ;
@@ -997,6 +1000,7 @@ static void tcp_v6_send_response(struct sk_buff *skb, u32 seq, u32 ack, u32 win,unsigned int tot_len =sizeof (struct tcphdr );struct dst_entry *dst;__be32 *topt; +gfp_t gfp_mask =GFP_ATOMIC ;if (ts) tot_len +=TCPOLEN_TSTAMP_ALIGNED ;
@@ -1006,7 +1010,7 @@ static void tcp_v6_send_response(struct sk_buff *skb, u32 seq, u32 ack, u32 win,#endif buff =alloc_skb (MAX_HEADER +sizeof (struct ipv6hdr ) + tot_len, -GFP_ATOMIC ); + gfp_mask);if (buff ==NULL )return ;
@@ -1083,6 +1087,7 @@ static void tcp_v6_send_reset(struct sock *sk, struct sk_buff *skb)struct tcphdr *th =tcp_hdr (skb);u32 seq =0 , ack_seq =0 ;struct tcp_md5sig_key *key =NULL ; +gfp_t gfp_mask =GFP_ATOMIC ;if (th->rst)return ;
@@ -1094,6 +1099,8 @@ static void tcp_v6_send_reset(struct sock *sk, struct sk_buff *skb)if (sk) key =tcp_v6_md5_do_lookup (sk, &ipv6_hdr (skb)->daddr);#endif +if (sk) + gfp_mask =sk_allocation (sk, gfp_mask);if (th->ack) seq =ntohl (th->ack_seq);
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 15/30 netvm: network reserve infrastructure
In reply to Xiaotian FengQuoted 1 line
From
e8a09c013cf8416ece804ddcbc0d016d2f936e6dMon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 11:06:54 +0800
Subject: [PATCH 15/30] netvm: network reserve infrastructure
Provide the basic infrastructure to reserve and charge/account network memory.
We provide the following reserve tree:
1) total network reserve
2) network TX reserve
3) protocol TX pages
4) network RX reserve
5) SKB data reserve
[1] is used to make all the network reserves a single subtree, for easy manipulation.
[2] and [4] are merely for eastetic reasons.
The TX pages reserve [3] is assumed bounded by it being the upper bound of memory that can be used for sending pages (not quite true, but good enough)
The SKB reserve [5] is an aggregate reserve, which is used to charge SKB data against in the fallback path.
The consumers for these reserves are sockets marked with:
SOCK_MEMALLOC
Such sockets are to be used to service the VM (iow. to swap over). They must be handled kernel side, exposing such a socket to user-space is a BUG.
--- include/net/sock.h | 43 ++++++++++++++++++++- net/Kconfig | 3 + net/core/sock.c | 107 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 152 insertions(+), 1 deletions(-)
include/net/sock.h +42 -1
@@ -52,6 +52,7 @@#include <linux/mm.h> #include <linux/security.h> #include <linux/slab.h> +#include <linux/reserve.h> #include <linux/filter.h> #include <linux/rculist_nulls.h>
@@ -532,6 +533,7 @@ enum sock_flags {SOCK_RCVTSTAMPNS ,/* %SO_TIMESTAMPNS setting */ SOCK_LOCALROUTE ,/* route locally only, %SO_DONTROUTE setting */ SOCK_QUEUE_SHRUNK ,/* write queue has been shrunk recently */ +SOCK_MEMALLOC ,/* the VM depends on us - make sure we're serviced */ SOCK_TIMESTAMPING_TX_HARDWARE ,/* %SOF_TIMESTAMPING_TX_HARDWARE */ SOCK_TIMESTAMPING_TX_SOFTWARE ,/* %SOF_TIMESTAMPING_TX_SOFTWARE */ SOCK_TIMESTAMPING_RX_HARDWARE, /* %SOF_TIMESTAMPING_RX_HARDWARE */
@@ -563,9 +565,48 @@ static inline int sock_flag(struct sock *sk, enum sock_flags flag)return test_bit (flag, &sk->sk_flags); } +static inline int sk_has_memalloc (struct sock *sk) +{ +return sock_flag (sk,SOCK_MEMALLOC ); +} + +extern struct mem_reserve net_rx_reserve; +extern struct mem_reserve net_skb_reserve; + +#ifdef CONFIG_NETVM +/* +* Guestimate the per request queue TX upper bound. +* +* Max packet size is 64k, and we need to reserve that much since the data +* might need to bounce it. Double it to be on the safe side. +*/ +#define TX_RESERVE_PAGES DIV_ROUND_UP(2*65536, PAGE_SIZE) + +extern int memalloc_socks; + +static inline int sk_memalloc_socks (void ) +{ +return memalloc_socks; +} + +extern int sk_adjust_memalloc (int socks,long tx_reserve_pages); +extern int sk_set_memalloc (struct sock *sk); +extern int sk_clear_memalloc (struct sock *sk); +#else +static inline int sk_memalloc_socks (void ) +{ +return 0 ; +} + +static inline int sk_clear_memalloc (struct sock *sk) +{ +return 0 ; +} +#endif +static inline gfp_t sk_allocation (struct sock *sk,gfp_t gfp_mask) { -return gfp_mask; +return gfp_mask | (sk->sk_allocation & __GFP_MEMALLOC); }static inline void sk_acceptq_removed (struct sock *sk)
net/Kconfig +3 -0
@@ -284,4 +284,7 @@ source "net/9p/Kconfig" source "net/caif/Kconfig" +config NETVM + def_bool n + endif # if NET
net/core/sock.c +107 -0
131 lines not on this page · raw message
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 16/30 netvm: INET reserves
In reply to Xiaotian FengQuoted 1 line
From
e1a39da88a7b093474c48bb5f22f3b715e5ec205Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 11:17:23 +0800
Subject: [PATCH 16/30] netvm: INET reserves.
Add reserves for INET.
The two big users seem to be the route cache and ip-fragment cache.
Reserve the route cache under generic RX reserve, its usage is bounded by the high reclaim watermark, and thus does not need further accounting.
Reserve the ip-fragement caches under SKB data reserve, these add to the SKB RX limit. By ensuring we can at least receive as much data as fits in the reassmbly line we avoid fragment attack deadlocks.
Adds to the reserve tree:
total network reserve
network TX reserve
protocol TX pages
network RX reserve
+ IPv6 route cache
+ IPv4 route cache
SKB data reserve
+ IPv6 fragment cache
+ IPv4 fragment cache
--- include/net/inet_frag.h | 7 ++++++ include/net/netns/ipv6.h | 4 +++ net/ipv4/inet_fragment.c | 3 ++ net/ipv4/ip_fragment.c | 34 +++++++++++++++++++++++++++- net/ipv4/route.c | 42 ++++++++++++++++++++++++++++++++++- net/ipv6/reassembly.c | 55 ++++++++++++++++++++++++++++++++++++++++++++- net/ipv6/route.c | 47 +++++++++++++++++++++++++++++++++++++- 7 files changed, 186 insertions(+), 6 deletions(-)
include/net/inet_frag.h +7 -0
@@ -1,6 +1,9 @@ #ifndef __NET_FRAG_H__ #define __NET_FRAG_H__ +#include <linux/reserve.h> +#include <linux/mutex.h> + struct netns_frags { int nqueues; atomic_t mem;
@@ -10,6 +13,10 @@ struct netns_frags {int timeout;int high_thresh;int low_thresh; + +/* reserves */ +struct mutex lock; +struct mem_reserve reserve; };struct inet_frag_queue {
include/net/netns/ipv6.h +4 -0
@@ -25,6 +25,8 @@ struct netns_sysctl_ipv6 {int ip6_rt_mtu_expires;int ip6_rt_min_advmss;int icmpv6_time; + +struct mutex ip6_rt_lock; };struct netns_ipv6 {
@@ -58,6 +60,8 @@ struct netns_ipv6 {struct sock *ndisc_sk;struct sock *tcp_sk;struct sock *igmp_sk; + +struct mem_reserve ip6_rt_reserve;#ifdef CONFIG_IPV6_MROUTE #ifndef CONFIG_IPV6_MROUTE_MULTIPLE_TABLES struct mr6_table *mrt6;
net/ipv4/inet_fragment.c +3 -0
@@ -20,6 +20,7 @@#include <linux/skbuff.h> #include <linux/rtnetlink.h> #include <linux/slab.h> +#include <linux/reserve.h> #include <net/inet_frag.h>
@@ -75,6 +76,8 @@ void inet_frags_init_net(struct netns_frags *nf) nf->nqueues =0 ;atomic_set (&nf->mem,0 );INIT_LIST_HEAD (&nf->lru_list); +mutex_init (&nf->lock); +mem_reserve_init (&nf->reserve,"IP fragement cache" ,NULL ); }EXPORT_SYMBOL (inet_frags_init_net);
net/ipv4/ip_fragment.c +33 -1
@@ -45,6 +45,8 @@#include <linux/udp.h> #include <linux/inet.h> #include <linux/netfilter_ipv4.h> +#include <linux/reserve.h> +#include <linux/nsproxy.h> /* NOTE. Logic of IP defragmentation is parallel to corresponding IPv6 * code now. If you change something here, _PLEASE_ update ipv6/reassembly.c
@@ -634,6 +636,34 @@ int ip_defrag(struct sk_buff *skb, u32 user) }#ifdef CONFIG_SYSCTL +static int +proc_dointvec_fragment (struct ctl_table *table,int write, +void __user *buffer,size_t *lenp,loff_t *ppos) +{ + struct net *net = container_of(table->data, struct net, + ipv4.frags.high_thresh); +ctl_table tmp = *table; +int new_bytes, ret; + +mutex_lock (&net->ipv4.frags.lock); +if (write) { + tmp.data = &new_bytes; + table = &tmp; + } + + ret =proc_dointvec (table, write, buffer, lenp, ppos); + +if (!ret && write) { + ret =mem_reserve_kmalloc_set (&net->ipv4.frags.reserve, + new_bytes); +if (!ret) + net->ipv4.frags.high_thresh = new_bytes; + } +mutex_unlock (&net->ipv4.frags.lock); + +return ret; +} +static int zero; static struct ctl_table ip4_frags_ns_ctl_table[] = {
@@ -642,7 +672,7 @@ static struct ctl_table ip4_frags_ns_ctl_table[] = { .data = &init_net.ipv4.frags.high_thresh, .maxlen = sizeof(int), .mode = 0644, - .proc_handler = proc_dointvec + .proc_handler = &proc_dointvec_fragment, }, { .procname = "ipfrag_low_thresh",
@@ -740,6 +770,8 @@ static inline void ip4_frags_ctl_register(void)static int __net_init ipv4_frags_init_net (struct net *net) { +int ret; +/* * Fragment cache limits. We will commit 256K at one time. Should we * cross that limit we will prune down to 192K. This should cope with
net/ipv4/route.c +41 -1
72 lines not on this page · raw message
net/ipv6/reassembly.c +53 -2
89 lines not on this page · raw message
net/ipv6/route.c +45 -2
94 lines not on this page · raw message
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 17/30 netvm: hook skb allocation to reserves
In reply to Xiaotian FengQuoted 1 line
From
3e824860934af5aa0150608314693c5e0e3608b6Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 11:30:27 +0800
Subject: [PATCH 17/30] netvm: hook skb allocation to reserves
Change the skb allocation api to indicate RX usage and use this to fall back to the reserve when needed. SKBs allocated from the reserve are tagged in skb->emergency.
Teach all other skb ops about emergency skbs and the reserve accounting.
Use the (new) packet split API to allocate and track fragment pages from the emergency reserve. Do this using an atomic counter in page->index. This is needed because the fragments have a different sharing semantic than that indicated by skb_shinfo()->dataref.
Note that the decision to distinguish between regular and emergency SKBs allows the accounting overhead to be limited to the later kind.
--- include/linux/mm_types.h | 1 + include/linux/skbuff.h | 27 +++++++-- net/core/skbuff.c | 137 +++++++++++++++++++++++++++++++++++++--------- 3 files changed, 133 insertions(+), 32 deletions(-)
include/linux/mm_types.h +1 -0
@@ -72,6 +72,7 @@ struct page {pgoff_t index;/* Our offset within mapping. */ void *freelist;/* SLUB: freelist req. slab lock */ int reserve;/* page_alloc: page is a reserve page */ +atomic_t frag_count;/* skb fragment use count */ };struct list_head lru;/* Pageout list, eg. active_list * protected by zone->lru_lock !
include/linux/skbuff.h +22 -5
@@ -385,9 +385,12 @@ struct sk_buff {#else __u8 deliver_no_wcard:1;#endif +#ifdef CONFIG_NETVM + __u8 emergency:1; +#endif kmemcheck_bitfield_end (flags2); -/* 0/14 bit hole */ +/* 0/13/14 bit hole */ #ifdef CONFIG_NET_DMA dma_cookie_t dma_cookie;
@@ -429,6 +432,18 @@ struct sk_buff {#define SKB_DST_NOREF 1UL#define SKB_DST_PTRMASK ~(SKB_DST_NOREF) +#define SKB_ALLOC_FCLONE 0x01 +#define SKB_ALLOC_RX 0x02 + +static inline bool skb_emergency (const struct sk_buff *skb) +{ +#ifdef CONFIG_NETVM +return unlikely (skb->emergency); +#else +return false; +#endif +} +/** * skb_dst - returns skb dst_entry * @skb: buffer
@@ -491,7 +506,7 @@ extern void kfree_skb(struct sk_buff *skb);extern void consume_skb (struct sk_buff *skb);extern void __kfree_skb (struct sk_buff *skb);extern struct sk_buff *__alloc_skb (unsigned int size, -gfp_t priority,int fclone,int node); +gfp_t priority,int flags,int node);static inline struct sk_buff *alloc_skb (unsigned int size,gfp_t priority) {
@@ -501,7 +516,7 @@ static inline struct sk_buff *alloc_skb(unsigned int size,static inline struct sk_buff *alloc_skb_fclone (unsigned int size,gfp_t priority) { -return __alloc_skb (size, priority,1 ,-1 ); +return __alloc_skb (size, priority,SKB_ALLOC_FCLONE ,-1 ); }extern bool skb_recycle_check (struct sk_buff *skb,int skb_size);
@@ -1516,7 +1531,8 @@ static inline void __skb_queue_purge(struct sk_buff_head *list)static inline struct sk_buff *__dev_alloc_skb (unsigned int length,gfp_t gfp_mask) { -struct sk_buff *skb =alloc_skb (length +NET_SKB_PAD , gfp_mask); +struct sk_buff *skb = +__alloc_skb (length +NET_SKB_PAD , gfp_mask,SKB_ALLOC_RX ,-1 );if (likely (skb))skb_reserve (skb,NET_SKB_PAD );return skb;
@@ -1557,6 +1573,7 @@ static inline struct sk_buff *netdev_alloc_skb_ip_align(struct net_device *dev, }extern struct page *__netdev_alloc_page (struct net_device *dev,gfp_t gfp_mask); +extern void __netdev_free_page (struct net_device *dev,struct page *page);/** * netdev_alloc_page - allocate a page for ps-rx on a specific device
@@ -1573,7 +1590,7 @@ static inline struct page *netdev_alloc_page(struct net_device *dev)static inline void netdev_free_page (struct net_device *dev,struct page *page) { -__free_page (page); +__netdev_free_page (dev, page); }/**
net/core/skbuff.c +110 -27
314 lines not on this page · raw message
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 18/30 netvm: filter emergency skbs
In reply to Xiaotian FengQuoted 1 line
From
b0240dd1e2ee0b4dc30f98c67cfe35e8c1833753Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 11:36:53 +0800
Subject: [PATCH 18/30] netvm: filter emergency skbs.
Toss all emergency packets not for a SOCK_MEMALLOC socket. This ensures our precious memory reserve doesn't get stuck waiting for user-space.
The correctness of this approach relies on the fact that networks must be assumed lossy.
--- net/core/filter.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-)
net/core/filter.c +3 -0
@@ -82,6 +82,9 @@ int sk_filter(struct sock *sk, struct sk_buff *skb)int err;struct sk_filter *filter; +if (skb_emergency (skb) && !sk_has_memalloc (sk)) +return -ENOMEM ; + err =security_sock_rcv_skb (sk, skb);if (err)return err;
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 19/30 netvm: prevent a stream specific deadlock
In reply to Xiaotian FengQuoted 1 line
From
97cab9c6b5964ba48bee576214d71edbef74d0a6Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 11:37:56 +0800
Subject: [PATCH 19/30] netvm: prevent a stream specific deadlock
It could happen that all !SOCK_MEMALLOC sockets have buffered so much data that we're over the global rmem limit. This will prevent SOCK_MEMALLOC buffers from receiving data, which will prevent userspace from running, which is needed to reduce the buffered data.
Fix this by exempting the SOCK_MEMALLOC sockets from the rmem limit.
--- include/net/sock.h | 7 ++++--- net/core/sock.c | 2 +- net/ipv4/tcp_input.c | 12 ++++++------ net/sctp/ulpevent.c | 2 +- 4 files changed, 12 insertions(+), 11 deletions(-)
include/net/sock.h +4 -3
@@ -977,12 +977,13 @@ static inline int sk_wmem_schedule(struct sock *sk, int size)__sk_mem_schedule (sk, size,SK_MEM_SEND ); } -static inline int sk_rmem_schedule (struct sock *sk,int size) +static inline int sk_rmem_schedule (struct sock *sk,struct sk_buff *skb) {if (!sk_has_account (sk))return 1 ; -return size <= sk->sk_forward_alloc || -__sk_mem_schedule (sk, size,SK_MEM_RECV ); +return skb->truesize <= sk->sk_forward_alloc || +__sk_mem_schedule (sk, skb->truesize,SK_MEM_RECV ) || +skb_emergency (skb); }static inline void sk_mem_reclaim (struct sock *sk)
net/core/sock.c +1 -1
@@ -399,7 +399,7 @@ int sock_queue_rcv_skb(struct sock *sk, struct sk_buff *skb)if (err)return err; -if (!sk_rmem_schedule (sk, skb->truesize)) { +if (!sk_rmem_schedule (sk, skb)) {atomic_inc (&sk->sk_drops);return -ENOBUFS ; }
net/ipv4/tcp_input.c +6 -6
@@ -4340,19 +4340,19 @@ static void tcp_ofo_queue(struct sock *sk)static int tcp_prune_ofo_queue (struct sock *sk);static int tcp_prune_queue (struct sock *sk); -static inline int tcp_try_rmem_schedule (struct sock *sk,unsigned int size) +static inline int tcp_try_rmem_schedule (struct sock *sk,struct sk_buff *skb) {if (atomic_read (&sk->sk_rmem_alloc) > sk->sk_rcvbuf || - !sk_rmem_schedule (sk, size)) { + !sk_rmem_schedule (sk, skb)) {if (tcp_prune_queue (sk) <0 )return -1 ; -if (!sk_rmem_schedule (sk, size)) { +if (!sk_rmem_schedule (sk, skb)) {if (!tcp_prune_ofo_queue (sk))return -1 ; -if (!sk_rmem_schedule (sk, size)) +if (!sk_rmem_schedule (sk, skb))return -1 ; } }
@@ -4405,7 +4405,7 @@ static void tcp_data_queue(struct sock *sk, struct sk_buff *skb)if (eaten <=0 ) { queue_and_out:if (eaten <0 && -tcp_try_rmem_schedule (sk, skb->truesize)) +tcp_try_rmem_schedule (sk, skb)) goto drop;skb_set_owner_r (skb, sk);
@@ -4476,7 +4476,7 @@ drop:TCP_ECN_check_ce (tp, skb); -if (tcp_try_rmem_schedule (sk, skb->truesize)) +if (tcp_try_rmem_schedule (sk, skb)) goto drop;/* Disable header prediction. */
net/sctp/ulpevent.c +1 -1
@@ -702,7 +702,7 @@ struct sctp_ulpevent *sctp_ulpevent_make_rcvmsg(struct sctp_association *asoc,if (rx_count >= asoc->base.sk->sk_rcvbuf) {if ((asoc->base.sk->sk_userlocks &SOCK_RCVBUF_LOCK ) || - (!sk_rmem_schedule (asoc->base.sk, chunk->skb->truesize))) + (!sk_rmem_schedule (asoc->base.sk, chunk->skb))) goto fail; }
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 20/30 netfilter: NF_QUEUE vs emergency skbs
In reply to Xiaotian FengQuoted 1 line
From
6c5ccad4c45a73a6d9ebecbfcb1bce8ff3ca462fMon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 11:38:29 +0800
Subject: [PATCH 20/30] netfilter: NF_QUEUE vs emergency skbs
Avoid memory getting stuck waiting for userspace, drop all emergency packets. This of course requires the regular storage route to not include an NF_QUEUE target ;-)
--- net/netfilter/core.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-)
net/netfilter/core.c +3 -0
@@ -176,9 +176,12 @@ next_hook:if (verdict ==NF_ACCEPT || verdict ==NF_STOP ) { ret =1 ; }else if (verdict ==NF_DROP ) { +drop:kfree_skb (skb); ret = -EPERM ; }else if ((verdict &NF_VERDICT_MASK ) ==NF_QUEUE ) { +if (skb_emergency (skb)) + goto drop;if (!nf_queue (skb, elem, pf, hook, indev, outdev, okfn, verdict >>NF_VERDICT_BITS )) goto next_hook;
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 21/30 netvm: skb processing
In reply to Xiaotian FengQuoted 1 line
From
15437174f171e197ecdfa5fe71ae89334bb58fd2Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 13:07:28 +0800
Subject: [PATCH 21/30] netvm: skb processing
In order to make sure emergency packets receive all memory needed to proceed ensure processing of emergency SKBs happens under PF_MEMALLOC.
Use the (new) sk_backlog_rcv() wrapper to ensure this for backlog processing.
Skip taps, since those are user-space again.
--- include/net/sock.h | 5 ++++ net/core/dev.c | 55 ++++++++++++++++++++++++++++++++++++++++++++++++--- net/core/sock.c | 16 +++++++++++++++ 3 files changed, 72 insertions(+), 4 deletions(-)
include/net/sock.h +5 -0
@@ -680,8 +680,13 @@ static inline __must_check int sk_add_backlog(struct sock *sk, struct sk_buff *sreturn 0 ; } +extern int __sk_backlog_rcv (struct sock *sk,struct sk_buff *skb); +static inline int sk_backlog_rcv (struct sock *sk,struct sk_buff *skb) { +if (skb_emergency (skb)) +return __sk_backlog_rcv (sk, skb); +return sk->sk_backlog_rcv (sk, skb); }
net/core/dev.c +51 -4
109 lines not on this page · raw message
net/core/sock.c +16 -0
@@ -322,6 +322,22 @@ int sk_clear_memalloc(struct sock *sk) return set; }EXPORT_SYMBOL_GPL (sk_clear_memalloc); + +int __sk_backlog_rcv (struct sock *sk,struct sk_buff *skb) +{ +int ret; +unsigned long pflags = current->flags; + +/* these should have been dropped before queueing */ +BUG_ON (!sk_has_memalloc (sk)); + + current->flags |=PF_MEMALLOC ; + ret = sk->sk_backlog_rcv (sk, skb); +tsk_restore_flags (current, pflags,PF_MEMALLOC ); + +return ret; +} +EXPORT_SYMBOL (__sk_backlog_rcv);#endif static int sock_set_timeout (long *timeo_p,char __user *optval,int optlen)
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 22/30 mm: add support for non block device backed swap files
In reply to Xiaotian FengQuoted 1 line
From
0c3f6a5db5c61a222135286e8a6ada7411b3ac3bMon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 13:08:45 +0800
Subject: [PATCH 22/30] mm: add support for non block device backed swap files
New addres_space_operations methods are added:
int swapon(struct file *);
int swapoff(struct file *);
int swap_out(struct file *, struct page *, struct writeback_control *); int swap_in(struct file *, struct page *);
When during sys_swapon() the ->swapon() method is found and returns no error the swapper_space.a_ops will proxy to sis->swap_file->f_mapping->a_ops, and make use of ->swap_{out,in}() to write/read swapcache pages.
The ->swapon() method will be used to communicate to the file that the VM relies on it, and the address_space should take adequate measures (like reserving memory for mempools or the like). The ->swapoff() method will be called on sys_swapoff() when ->swapon() was found and returned no error.
This new interface can be used to obviate the need for ->bmap in the swapfile code. A filesystem would need to load (and maybe even allocate) the full block map for a file into memory and pin it there on ->swapon() so that ->swap_{out,in}() have instant access to it. It can be released on ->swapoff().
The reason to provide ->swap_{out,in}() over using {write,read}page() is to
1) make a distinction between swapcache and pagecache pages, and
2) to provide a struct file * for credential context (normally not needed in the context of writepage, as the page content is normally dirtied using either of the following interfaces:
write_{begin,end}()
{prepare,commit}_write()
page_mkwrite()
which do have the file context.
--- Documentation/filesystems/Locking | 22 +++++++++++++++ Documentation/filesystems/vfs.txt | 18 ++++++++++++ include/linux/buffer_head.h | 1 + include/linux/fs.h | 9 ++++++ include/linux/swap.h | 5 +++- mm/page_io.c | 54 +++++++++++++++++++++++++++++++++++++ mm/swap_state.c | 4 +- mm/swapfile.c | 36 +++++++++++++++++++++++- 8 files changed, 144 insertions(+), 5 deletions(-)
Documentation/filesystems/Locking +22 -0
@@ -174,6 +174,10 @@ prototypes: int (*direct_IO)(int, struct kiocb *, const struct iovec *iov, loff_t offset, unsigned long nr_segs); int (*launder_page) (struct page *); + int (*swapon) (struct file *); + int (*swapoff) (struct file *); + int (*swap_out) (struct file *, struct page *, struct writeback_control *); + int (*swap_in) (struct file *, struct page *); locking rules: All except set_page_dirty may block
@@ -193,6 +197,10 @@ invalidatepage: no yes releasepage: no yes direct_IO: no launder_page: no yes +swapon no +swapoff no +swap_out no yes, unlocks +swap_in no yes, unlocks ->write_begin(), ->write_end(), ->sync_page() and ->readpage() may be called from the request handler (/dev/loop).
@@ -292,6 +300,20 @@ cleaned, or an error value if not. Note that in order to prevent the page getting mapped back in and redirtied, it needs to be kept locked across the entire operation. + ->swapon() will be called with a non-zero argument on files backing +(non block device backed) swapfiles. A return value of zero indicates success, +in which case this file can be used for backing swapspace. The swapspace +operations will be proxied to the address space operations. + + ->swapoff() will be called in the sys_swapoff() path when ->swapon() +returned success. + + ->swap_out() when swapon() returned success, this method is used to +write the swap page. + + ->swap_in() when swapon() returned success, this method is used to +read the swap page. + Note: currently almost all instances of address_space methods are using BKL for internal serialization and that's one of the worst sources of contention. Normally they are calling library functions (in fs/buffer.c)
Documentation/filesystems/vfs.txt +18 -0
@@ -542,6 +542,11 @@ struct address_space_operations { int (*migratepage) (struct page *, struct page *); int (*launder_page) (struct page *); int (*error_remove_page) (struct mapping *mapping, struct page *page); + int (*swapon)(struct file *); + int (*swapoff)(struct file *); + int (*swap_out)(struct file *file, struct page *page, + struct writeback_control *wbc); + int (*swap_in)(struct file *file, struct page *page); }; writepage: called by the VM to write a dirty page to backing store.
@@ -706,6 +711,19 @@ struct address_space_operations { unless you have them locked or reference counts increased. + swapon: Called when swapon is used on a file. A + return value of zero indicates success, in which case this + file can be used to back swapspace. The swapspace operations + will be proxied to this address space's ->swap_{out,in} methods. + + swapoff: Called during swapoff on files where swapon was successfull. + + swap_out: Called to write a swapcache page to a backing store, similar to + writepage. + + swap_in: Called to read a swapcache page from a backing store, similar to + readpage. + The File Object ===============
include/linux/buffer_head.h +1 -0
@@ -329,6 +329,7 @@ static inline int inode_has_buffers(struct inode *inode) { return 0; }static inline void invalidate_inode_buffers (struct inode *inode) {}static inline int remove_inode_buffers (struct inode *inode) {return 1 ; }static inline int sync_mapping_buffers (struct address_space *mapping) {return 0 ; } +static inline void block_sync_page (struct page *) { }#endif /* CONFIG_BLOCK */ #endif /* _LINUX_BUFFER_HEAD_H */
include/linux/fs.h +9 -0
@@ -612,6 +612,15 @@ struct address_space_operations {int (*is_partially_uptodate) (struct page *,read_descriptor_t *,unsigned long );int (*error_remove_page)(struct address_space *,struct page *); + +/* +* swapfile support +*/ +int (*swapon)(struct file *file); +int (*swapoff)(struct file *file); +int (*swap_out)(struct file *file,struct page *page, +struct writeback_control *wbc); +int (*swap_in)(struct file *file,struct page *page); };/*
include/linux/swap.h +4 -1
@@ -147,7 +147,7 @@ enum {SWP_SOLIDSTATE = (1 <<4 ),/* blkdev seeks are cheap */ SWP_CONTINUED = (1 <<5 ),/* swap_map has count continuation */ SWP_BLKDEV = (1 <<6 ),/* its a block device */ -/* add others here before... */ +SWP_FILE = (1 <<7 ),/* file swap area */ SWP_SCANNING = (1 << 8), /* refcount in scan_swap_map */ };
@@ -293,6 +293,8 @@ extern void swap_unplug_io_fn(struct backing_dev_info *, struct page *);/* linux/mm/page_io.c */ extern int swap_readpage (struct page *);extern int swap_writepage (struct page *page,struct writeback_control *wbc); +extern void swap_sync_page (struct page *page); +extern int swap_set_page_dirty (struct page *page);extern void end_swap_bio_read (struct bio *bio,int err);/* linux/mm/swap_state.c */
@@ -329,6 +331,7 @@ extern int swap_type_of(dev_t, sector_t, struct block_device **);extern unsigned int count_swap_pages (int ,int );extern sector_t map_swap_page (struct page *,struct block_device **);extern sector_t swapdev_block (int ,pgoff_t ); +extern struct swap_info_struct *page_swap_info (struct page *);extern int reuse_swap_page (struct page *);extern int try_to_free_swap (struct page *);struct backing_dev_info ;
mm/page_io.c +54 -0
79 lines not on this page · raw message
mm/swap_state.c +2 -2
@@ -28,8 +28,8 @@*/ static const struct address_space_operations swap_aops = { .writepage = swap_writepage, - .sync_page = block_sync_page, - .set_page_dirty = __set_page_dirty_nobuffers, + .sync_page = swap_sync_page, + .set_page_dirty = swap_set_page_dirty, .migratepage = migrate_page, };
mm/swapfile.c +34 -2
@@ -1353,6 +1353,14 @@ static void destroy_swap_extents(struct swap_info_struct *sis)list_del (&se->list);kfree (se); } + +if (sis->flags &SWP_FILE ) { +struct file *swap_file = sis->swap_file; +struct address_space *mapping = swap_file->f_mapping; + + sis->flags &= ~SWP_FILE ; + mapping->a_ops->swapoff (swap_file); + } }/*
@@ -1434,7 +1442,9 @@ add_swap_extent(struct swap_info_struct *sis, unsigned long start_page,*/ static int setup_swap_extents (struct swap_info_struct *sis,sector_t *span) { -struct inode *inode; +struct file *swap_file = sis->swap_file; +struct address_space *mapping = swap_file->f_mapping; +struct inode *inode = mapping->host;unsigned blocks_per_page;unsigned long page_no;unsigned blkbits;
@@ -1445,13 +1455,22 @@ static int setup_swap_extents(struct swap_info_struct *sis, sector_t *span)int nr_extents =0 ;int ret; - inode = sis->swap_file->f_mapping->host;if (S_ISBLK (inode->i_mode)) { ret =add_swap_extent (sis,0 , sis->max,0 ); *span = sis->pages; goto out; } +if (mapping->a_ops->swapon) { + ret = mapping->a_ops->swapon (swap_file); +if (!ret) { + sis->flags |=SWP_FILE ; + ret =add_swap_extent (sis,0 , sis->max,0 ); + *span = sis->pages; + } + goto out; + } + blkbits = inode->i_blkbits; blocks_per_page =PAGE_SIZE >> blkbits;
@@ -2228,6 +2247,19 @@ int swapcache_prepare(swp_entry_t entry)return __swap_duplicate (entry,SWAP_HAS_CACHE ); } +struct swap_info_struct *page_swap_info (struct page *page) +{ +swp_entry_t swap = { .val =page_private (page) }; +if (!PageSwapCache (page) || !swap.val) { +/* This should only happen from sync_page. +* In other cases the page should be locked and +* should be in a SwapCache +*/ +return NULL ; + } +return swap_info[swp_type (swap)]; +} +/* * swap_lock prevents swap_map being freed. Don't grab an extra * reference on the swaphandle, it doesn't matter if it becomes unused.
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 23/30 mm: methods for teaching filesystems about PG_swapcache pages
In reply to Xiaotian FengQuoted 1 line
From
50e813068c51de733bbbdd04eb4af9c43919cd57Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 13:09:50 +0800
Subject: [PATCH 23/30] mm: methods for teaching filesystems about PG_swapcache pages
In order to teach filesystems to handle swap cache pages, three new page functions are introduced:
pgoff_t page_file_index(struct page *); loff_t page_file_offset(struct page *); struct address_space *page_file_mapping(struct page *);
page_file_index() - gives the offset of this page in the file in PAGE_CACHE_SIZE blocks. Like page->index is for mapped pages, this function also gives the correct index for PG_swapcache pages.
page_file_offset() - uses page_file_index(), so that it will give the expected result, even for PG_swapcache pages.
page_file_mapping() - gives the mapping backing the actual page; that is for swap cache pages it will give swap_file->f_mapping.
--- include/linux/mm.h | 25 +++++++++++++++++++++++++ include/linux/pagemap.h | 5 +++++ mm/swapfile.c | 19 +++++++++++++++++++ 3 files changed, 49 insertions(+), 0 deletions(-)
include/linux/mm.h +25 -0
@@ -663,6 +663,17 @@ static inline void *page_rmapping(struct page *page)return (void *)((unsigned long )page->mapping & ~PAGE_MAPPING_FLAGS ); } +extern struct address_space *__page_file_mapping (struct page *); + +static inline +struct address_space *page_file_mapping (struct page *page) +{ +if (unlikely (PageSwapCache (page))) +return __page_file_mapping (page); + +return page->mapping; +} +static inline int PageAnon (struct page *page) {return ((unsigned long )page->mapping &PAGE_MAPPING_ANON ) !=0 ;
@@ -679,6 +690,20 @@ static inline pgoff_t page_index(struct page *page)return page->index; } +extern pgoff_t __page_file_index (struct page *page); + +/* +* Return the file index of the page. Regular pagecache pages use ->index +* whereas swapcache pages use swp_offset(->private) +*/ +static inline pgoff_t page_file_index (struct page *page) +{ +if (unlikely (PageSwapCache (page))) +return __page_file_index (page); + +return page->index; +} +/* * The atomic page->_mapcount, like _count, starts from -1: * so that transitions both from it and to it can be tracked,
include/linux/pagemap.h +5 -0
@@ -285,6 +285,11 @@ static inline loff_t page_offset(struct page *page)extern pgoff_t linear_hugepage_index (struct vm_area_struct *vma,unsigned long address); +static inline loff_t page_file_offset (struct page *page) +{ +return ((loff_t )page_file_index (page)) <<PAGE_CACHE_SHIFT ; +} +static inline pgoff_t linear_page_index (struct vm_area_struct *vma,unsigned long address) {
mm/swapfile.c +19 -0
@@ -2261,6 +2261,25 @@ struct swap_info_struct *page_swap_info(struct page *page) }/* +* out-of-line __page_file_ methods to avoid include hell. +*/ + +struct address_space *__page_file_mapping (struct page *page) +{ +VM_BUG_ON (!PageSwapCache (page)); +return page_swap_info (page)->swap_file->f_mapping; +} +EXPORT_SYMBOL_GPL (__page_file_mapping); + +pgoff_t __page_file_index (struct page *page) +{ +swp_entry_t swap = { .val =page_private (page) }; +VM_BUG_ON (!PageSwapCache (page)); +return swp_offset (swap); +} +EXPORT_SYMBOL_GPL (__page_file_index); + +/* * swap_lock prevents swap_map being freed. Don't grab an extra * reference on the swaphandle, it doesn't matter if it becomes unused. */
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 24/30 nfs: teach the NFS client how to treat PG_swapcache pages
In reply to Xiaotian FengQuoted 1 line
From
743090cf0c129f3c83506260866f525a9f181f99Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 13:10:26 +0800
Subject: [PATCH 24/30] nfs: teach the NFS client how to treat PG_swapcache pages
Replace all relevant occurences of page->index and page->mapping in the NFS client with the new page_file_index() and page_file_mapping() functions.
--- fs/nfs/dir.c | 4 ++-- fs/nfs/file.c | 14 +++++++------- fs/nfs/fscache-index.c | 2 +- fs/nfs/fscache.c | 14 +++++++------- fs/nfs/internal.h | 7 ++++--- fs/nfs/pagelist.c | 6 +++--- fs/nfs/read.c | 6 +++--- fs/nfs/write.c | 45 +++++++++++++++++++++++---------------------- 8 files changed, 50 insertions(+), 48 deletions(-)
fs/nfs/dir.c +2 -2
@@ -182,7 +182,7 @@ int nfs_readdir_filler(nfs_readdir_descriptor_t *desc, struct page *page)dfprintk (DIRCACHE ,"NFS: %s: reading cookie %Lu into page %lu\n" , __func__, (long long )desc->entry->cookie, - page->index); +page_file_index (page)); again: timestamp = jiffies;
@@ -207,7 +207,7 @@ int nfs_readdir_filler(nfs_readdir_descriptor_t *desc, struct page *page)* Note: assumes we have exclusive access to this mapping either * through inode->i_mutex or some other mechanism. */ -if (invalidate_inode_pages2_range (inode->i_mapping, page->index +1 ,-1 ) <0 ) { +if (invalidate_inode_pages2_range (inode->i_mapping,page_file_index (page) +1 ,-1 ) <0 ) {/* Should never happen */ nfs_zap_mapping (inode, inode->i_mapping); }
fs/nfs/file.c +7 -7
@@ -480,9 +480,9 @@ static void nfs_invalidate_page(struct page *page, unsigned long offset)if (offset !=0 )return ;/* Cancel any unstarted writes on this page */ -nfs_wb_page_cancel (page->mapping->host, page); +nfs_wb_page_cancel (page_file_mapping (page)->host, page); -nfs_fscache_invalidate_page (page, page->mapping->host); +nfs_fscache_invalidate_page (page,page_file_mapping (page)->host); }/*
@@ -497,7 +497,7 @@ static int nfs_release_page(struct page *page, gfp_t gfp)/* Only do I/O if gfp is a superset of GFP_KERNEL */ if ((gfp &GFP_KERNEL ) ==GFP_KERNEL ) -nfs_wb_page (page->mapping->host, page); +nfs_wb_page (page_file_mapping (page)->host, page);/* If PagePrivate() is set, then the page is not freeable */ if (PagePrivate (page))return 0 ;
@@ -514,11 +514,11 @@ static int nfs_release_page(struct page *page, gfp_t gfp)*/ static int nfs_launder_page (struct page *page) { -struct inode *inode = page->mapping->host; +struct inode *inode =page_file_mapping (page)->host;struct nfs_inode *nfsi =NFS_I (inode);dfprintk (PAGECACHE ,"NFS: launder_page(%ld, %llu)\n" , - inode->i_ino, (long long )page_offset (page)); + inode->i_ino, (long long )page_file_offset (page));nfs_fscache_wait_on_page_write (nfsi, page);return nfs_wb_page (inode, page);
@@ -557,13 +557,13 @@ static int nfs_vm_page_mkwrite(struct vm_area_struct *vma, struct vm_fault *vmf)dfprintk (PAGECACHE ,"NFS: vm_page_mkwrite(%s/%s(%ld), offset %lld)\n" , dentry->d_parent->d_name.name, dentry->d_name.name, filp->f_mapping->host->i_ino, - (long long )page_offset (page)); + (long long )page_file_offset (page));/* make sure the cache has finished storing the page */ nfs_fscache_wait_on_page_write (NFS_I (dentry->d_inode), page);lock_page (page); - mapping = page->mapping; + mapping =page_file_mapping (page);if (mapping != dentry->d_inode->i_mapping) goto out_unlock;
fs/nfs/fscache-index.c +1 -1
@@ -283,7 +283,7 @@ static void nfs_fscache_inode_now_uncached(void *cookie_netfs_data)for (loop =0 ; loop < nr_pages; loop++)ClearPageFsCache (pvec.pages[loop]); - first = pvec.pages[nr_pages -1 ]->index +1 ; + first =page_file_index (pvec.pages[nr_pages -1 ]) +1 ; pvec.nr = nr_pages;pagevec_release (&pvec);
fs/nfs/fscache.c +7 -7
@@ -356,7 +356,7 @@ void nfs_fscache_reset_inode_cookie(struct inode *inode)int nfs_fscache_release_page (struct page *page,gfp_t gfp) {if (PageFsCache (page)) { -struct nfs_inode *nfsi =NFS_I (page->mapping->host); +struct nfs_inode *nfsi =NFS_I (page_file_mapping (page)->host);struct fscache_cookie *cookie = nfsi->fscache;BUG_ON (!cookie);
@@ -366,7 +366,7 @@ int nfs_fscache_release_page(struct page *page, gfp_t gfp)if (!fscache_maybe_release_page (cookie, page, gfp))return 0 ; -nfs_add_fscache_stats (page->mapping->host, +nfs_add_fscache_stats (page_file_mapping (page)->host,NFSIOS_FSCACHE_PAGES_UNCACHED ,1 ); }
@@ -391,7 +391,7 @@ void __nfs_fscache_invalidate_page(struct page *page, struct inode *inode)BUG_ON (!PageLocked (page));fscache_uncache_page (cookie, page); -nfs_add_fscache_stats (page->mapping->host, +nfs_add_fscache_stats (page_file_mapping (page)->host,NFSIOS_FSCACHE_PAGES_UNCACHED ,1 ); }
@@ -413,7 +413,7 @@ static void nfs_readpage_from_fscache_complete(struct page *page,SetPageUptodate (page);unlock_page (page); }else { - error =nfs_readpage_async (context, page->mapping->host, page); + error =nfs_readpage_async (context,page_file_mapping (page)->host, page);if (error)unlock_page (page); }
@@ -429,7 +429,7 @@ int __nfs_readpage_from_fscache(struct nfs_open_context *ctx,dfprintk (FSCACHE ,"NFS: readpage_from_fscache(fsc:%p/p:%p(i:%lx f:%lx)/0x%p)\n" , -NFS_I (inode)->fscache, page, page->index, page->flags, inode); +NFS_I (inode)->fscache, page,page_file_index (page), page->flags, inode); ret = fscache_read_or_alloc_page(NFS_I(inode)->fscache, page,
@@ -518,12 +518,12 @@ void __nfs_readpage_to_fscache(struct inode *inode, struct page *page, int sync)dfprintk (FSCACHE ,"NFS: readpage_to_fscache(fsc:%p/p:%p(i:%lx f:%lx)/%d)\n" , -NFS_I (inode)->fscache, page, page->index, page->flags, sync); +NFS_I (inode)->fscache, page,page_file_index (page), page->flags, sync); ret =fscache_write_page (NFS_I (inode)->fscache, page,GFP_KERNEL );dfprintk (FSCACHE ,"NFS: readpage_to_fscache: p:%p(i:%lu f:%lx) ret %d\n" , - page, page->index, page->flags, ret); + page,page_file_index (page), page->flags, ret);if (ret !=0 ) {fscache_uncache_page (NFS_I (inode)->fscache, page);
fs/nfs/internal.h +4 -3
@@ -342,13 +342,14 @@ void nfs_super_set_maxbytes(struct super_block *sb, __u64 maxfilesize)static inline unsigned int nfs_page_length (struct page *page) { -loff_t i_size =i_size_read (page->mapping->host); +loff_t i_size =i_size_read (page_file_mapping (page)->host);if (i_size >0 ) { +pgoff_t page_index =page_file_index (page);pgoff_t end_index = (i_size -1 ) >>PAGE_CACHE_SHIFT ; -if (page->index < end_index) +if (page_index < end_index)return PAGE_CACHE_SIZE ; -if (page->index == end_index) +if (page_index == end_index)return ((i_size -1 ) & ~PAGE_CACHE_MASK ) +1 ; }return 0 ;
fs/nfs/pagelist.c +3 -3
@@ -70,11 +70,11 @@ nfs_create_request(struct nfs_open_context *ctx, struct inode *inode,* update_nfs_request below if the region is not locked. */ req->wb_page = page;atomic_set (&req->wb_complete,0 ); - req->wb_index = page->index; + req->wb_index =page_file_index (page);page_cache_get (page);BUG_ON (PagePrivate (page));BUG_ON (!PageLocked (page)); -BUG_ON (page->mapping->host != inode); +BUG_ON (page_file_mapping (page)->host != inode); req->wb_offset = offset; req->wb_pgbase = offset; req->wb_bytes = count;
@@ -363,7 +363,7 @@ void nfs_pageio_cond_complete(struct nfs_pageio_descriptor *desc, pgoff_t index)* nfs_scan_list - Scan a list for matching requests * @nfsi: NFS inode * @dst: Destination list -* @idx_start: lower bound of page->index to scan +* @idx_start: lower bound of page_file_index(page) to scan * @npages: idx_start + npages sets the upper bound to scan. * @tag: tag to scan for *
fs/nfs/read.c +3 -3
@@ -501,11 +501,11 @@ static const struct rpc_call_ops nfs_read_full_ops = {int nfs_readpage (struct file *file,struct page *page) {struct nfs_open_context *ctx; -struct inode *inode = page->mapping->host; +struct inode *inode =page_file_mapping (page)->host;int error;dprintk ("NFS: nfs_readpage (%p %ld@%lu)\n" , - page,PAGE_CACHE_SIZE , page->index); + page,PAGE_CACHE_SIZE ,page_file_index (page));nfs_inc_stats (inode,NFSIOS_VFSREADPAGE );nfs_add_stats (inode,NFSIOS_READPAGES ,1 );
@@ -559,7 +559,7 @@ static int readpage_async_filler(void *data, struct page *page) {struct nfs_readdesc *desc = (struct nfs_readdesc *)data; -struct inode *inode = page->mapping->host; +struct inode *inode =page_file_mapping (page)->host;struct nfs_page *new;unsigned int len;int error;
fs/nfs/write.c +23 -22
170 lines not on this page · raw message
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 25/30 nfs: disable data cache revalidation for swapfiles
In reply to Xiaotian FengQuoted 1 line
From
ee72952409a0b811d61f435682e6d161e3b5189bMon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 13:10:49 +0800
Subject: [PATCH 25/30] nfs: disable data cache revalidation for swapfiles
Do as Trond suggested:
http://lkml.org/lkml/2006/8/25/348
Disable NFS data cache revalidation on swap files since it doesn't really make sense to have other clients change the file while you are using it.
Thereby we can stop setting PG_private on swap pages, since there ought to be no further races with invalidate_inode_pages2() to deal with.
And since we cannot set PG_private we cannot use page->private (which is already used by PG_swapcache pages anyway) to store the nfs_page. Thus augment the new nfs_page_find_request logic.
--- fs/nfs/inode.c | 6 +++++ fs/nfs/write.c | 69 ++++++++++++++++++++++++++++++++++++++++++++++--------- 2 files changed, 63 insertions(+), 12 deletions(-)
fs/nfs/inode.c +6 -0
@@ -798,6 +798,12 @@ int nfs_revalidate_mapping(struct inode *inode, struct address_space *mapping)struct nfs_inode *nfsi =NFS_I (inode);int ret =0 ; +/* +* swapfiles are not supposed to be shared. +*/ +if (IS_SWAPFILE (inode)) + goto out; +if ((nfsi->cache_validity &NFS_INO_REVAL_PAGECACHE ) ||nfs_attribute_cache_expired (inode) ||NFS_STALE (inode)) {
fs/nfs/write.c +57 -12
112 lines not on this page · raw message
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 26/30 nfs: enable swap on NFS
In reply to Xiaotian FengQuoted 1 line
From
61388a8872071bb5b0015b9f5e3183410a98d949Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 13:11:13 +0800
Subject: [PATCH 26/30] nfs: enable swap on NFS
Implement all the new swapfile a_ops for NFS. This will set the NFS socket to SOCK_MEMALLOC and run socket reconnect under PF_MEMALLOC as well as reset SOCK_MEMALLOC before engaging the protocol ->connect() method.
PF_MEMALLOC should allow the allocation of struct socket and related objects and the early (re)setting of SOCK_MEMALLOC should allow us to receive the packets required for the TCP connection buildup.
(swapping continues over a server reset during heavy network traffic)
--- fs/nfs/Kconfig | 10 ++++++ fs/nfs/file.c | 18 +++++++++++ fs/nfs/write.c | 22 ++++++++++++++ include/linux/nfs_fs.h | 2 + include/linux/sunrpc/xprt.h | 5 ++- mm/page_io.c | 1 + net/sunrpc/Kconfig | 5 +++ net/sunrpc/sched.c | 9 ++++- net/sunrpc/xprtsock.c | 68 +++++++++++++++++++++++++++++++++++++++++++ 9 files changed, 137 insertions(+), 3 deletions(-)
fs/nfs/Kconfig +10 -0
@@ -74,6 +74,16 @@ config NFS_V4 If unsure, say N. +config NFS_SWAP + bool "Provide swap over NFS support" + default n + depends on NFS_FS + select SUNRPC_SWAP + help + This option enables swapon to work on files located on NFS mounts. + + For more details, see Documentation/network-swap.txt + config NFS_V4_1 bool "NFS client support for NFSv4.1 (DEVELOPER ONLY)" depends on NFS_V4 && EXPERIMENTAL
fs/nfs/file.c +18 -0
@@ -524,6 +524,18 @@ static int nfs_launder_page(struct page *page)return nfs_wb_page (inode, page); } +#ifdef CONFIG_NFS_SWAP +static int nfs_swapon (struct file *file) +{ +return xs_swapper (NFS_CLIENT (file->f_mapping->host)->cl_xprt,1 ); +} + +static int nfs_swapoff (struct file *file) +{ +return xs_swapper (NFS_CLIENT (file->f_mapping->host)->cl_xprt,0 ); +} +#endif +const struct address_space_operations nfs_file_aops = { .readpage = nfs_readpage, .readpages = nfs_readpages,
@@ -538,6 +550,12 @@ const struct address_space_operations nfs_file_aops = { .migratepage = nfs_migrate_page, .launder_page = nfs_launder_page, .error_remove_page = generic_error_remove_page, +#ifdef CONFIG_NFS_SWAP + .swapon = nfs_swapon, + .swapoff = nfs_swapoff, + .swap_out = nfs_swap_out, + .swap_in = nfs_readpage, +#endif };/*
fs/nfs/write.c +22 -0
@@ -355,6 +355,28 @@ int nfs_writepage(struct page *page, struct writeback_control *wbc)return ret; } +static int nfs_writepage_setup (struct nfs_open_context *ctx,struct page *page, +unsigned int offset,unsigned int count); + +int nfs_swap_out (struct file *file,struct page *page, +struct writeback_control *wbc) +{ +struct nfs_open_context *ctx =nfs_file_open_context (file); +int status; + + status =nfs_writepage_setup (ctx, page,0 ,nfs_page_length (page)); +if (status <0 ) { +nfs_set_pageerror (page); + goto out; + } + + status =nfs_writepage_locked (page, wbc); + +out: +unlock_page (page); +return status; +} +static int nfs_writepages_callback (struct page *page,struct writeback_control *wbc,void *data) {int ret;
include/linux/nfs_fs.h +2 -0
@@ -484,6 +484,8 @@ extern int nfs_writepages(struct address_space *, struct writeback_control *);extern int nfs_flush_incompatible (struct file *file,struct page *page);extern int nfs_updatepage (struct file *,struct page *,unsigned int ,unsigned int );extern int nfs_writeback_done (struct rpc_task *,struct nfs_write_data *); +extern int nfs_swap_out (struct file *file,struct page *page, +struct writeback_control *wbc);/* * Try to write back everything synchronously (but check the
include/linux/sunrpc/xprt.h +4 -1
@@ -171,7 +171,9 @@ struct rpc_xprt {unsigned int max_reqs;/* total slots */ unsigned long state;/* transport state */ unsigned char shutdown : 1, /* being shut down */ - resvport : 1; /* use a reserved port */ + resvport : 1, /* use a reserved port */ + swapper : 1; /* we're swapping over this +transport */ unsigned int bind_index;/* bind function index */ /*
@@ -303,6 +305,7 @@ void xprt_release_rqst_cong(struct rpc_task *task);void xprt_disconnect_done (struct rpc_xprt *xprt);void xprt_force_disconnect (struct rpc_xprt *xprt);void xprt_conditional_disconnect (struct rpc_xprt *xprt,unsigned int cookie); +int xs_swapper (struct rpc_xprt *xprt,int enable);/* * Reserved bit positions in xprt->state
mm/page_io.c +1 -0
@@ -94,6 +94,7 @@ int swap_writepage(struct page *page, struct writeback_control *wbc) {struct bio *bio;int ret =0 , rw =WRITE ; +struct swap_info_struct *sis =page_swap_info (page);if (try_to_free_swap (page)) {unlock_page (page);
net/sunrpc/Kconfig +5 -0
@@ -17,6 +17,11 @@ config SUNRPC_XPRT_RDMA If unsure, say N. +config SUNRPC_SWAP + def_bool n + depends on SUNRPC + select NETVM + config RPCSEC_GSS_KRB5 tristate "Secure RPC: Kerberos V mechanism (EXPERIMENTAL)" depends on SUNRPC && EXPERIMENTAL
net/sunrpc/sched.c +7 -2
@@ -742,7 +742,10 @@ static void rpc_async_schedule(struct work_struct *work)void *rpc_malloc (struct rpc_task *task,size_t size) {struct rpc_buffer *buf; -gfp_t gfp =RPC_IS_SWAPPER (task) ?GFP_ATOMIC :GFP_NOWAIT ; +gfp_t gfp =GFP_NOWAIT ; + +if (RPC_IS_SWAPPER (task)) + gfp |= __GFP_MEMALLOC; size +=sizeof (struct rpc_buffer );if (size <=RPC_BUFFER_MAXSIZE )
@@ -813,6 +816,8 @@ static void rpc_init_task(struct rpc_task *task, const struct rpc_task_setup *takref_get (&task->tk_client->cl_kref);if (task->tk_client->cl_softrtry) task->tk_flags |=RPC_TASK_SOFT ; +if (task->tk_client->cl_xprt->swapper) + task->tk_flags |=RPC_TASK_SWAPPER ; }if (task->tk_ops->rpc_call_prepare !=NULL )
@@ -838,7 +843,7 @@ static void rpc_init_task(struct rpc_task *task, const struct rpc_task_setup *tastatic struct rpc_task *rpc_alloc_task (void ) { -return (struct rpc_task *)mempool_alloc (rpc_task_mempool,GFP_NOFS ); +return (struct rpc_task *)mempool_alloc (rpc_task_mempool,GFP_NOIO ); }/*
net/sunrpc/xprtsock.c +68 -0
137 lines not on this page · raw message
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 27/30 nfs: fix various memory recursions possible with swap over NFS
In reply to Xiaotian FengQuoted 1 line
From
df0106f58d7ac2337f74efb1d8caaf27f635e050Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 13:11:32 +0800
Subject: [PATCH 27/30] nfs: fix various memory recursions possible with swap over NFS.
GFP_NOFS is _more_ permissive than GFP_NOIO in that it will initiate IO, just not of any filesystem data.
The problem is that previuosly NOFS was correct because that avoids recursion into the NFS code, it now is not, because also IO (swap) can lead to this recursion.
--- fs/nfs/pagelist.c | 2 +- fs/nfs/write.c | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-)
fs/nfs/pagelist.c +1 -1
@@ -27,7 +27,7 @@ static inline struct nfs_page *nfs_page_alloc (void) {struct nfs_page *p; - p =kmem_cache_alloc (nfs_page_cachep,GFP_KERNEL ); + p =kmem_cache_alloc (nfs_page_cachep,GFP_NOIO );if (p) {memset (p,0 ,sizeof (*p));INIT_LIST_HEAD (&p->wb_list);
fs/nfs/write.c +3 -3
@@ -50,7 +50,7 @@ static mempool_t *nfs_commit_mempool;struct nfs_write_data *nfs_commitdata_alloc (void ) { -struct nfs_write_data *p =mempool_alloc (nfs_commit_mempool,GFP_NOFS ); +struct nfs_write_data *p =mempool_alloc (nfs_commit_mempool,GFP_NOIO );if (p) {memset (p,0 ,sizeof (*p));
@@ -69,7 +69,7 @@ void nfs_commit_free(struct nfs_write_data *p)struct nfs_write_data *nfs_writedata_alloc (unsigned int pagecount) { -struct nfs_write_data *p =mempool_alloc (nfs_wdata_mempool,GFP_NOFS ); +struct nfs_write_data *p =mempool_alloc (nfs_wdata_mempool,GFP_NOIO );if (p) {memset (p,0 ,sizeof (*p));
@@ -79,7 +79,7 @@ struct nfs_write_data *nfs_writedata_alloc(unsigned int pagecount)if (pagecount <=ARRAY_SIZE (p->page_array)) p->pagevec = p->page_array;else { - p->pagevec =kcalloc (pagecount,sizeof (struct page *),GFP_NOFS ); + p->pagevec =kcalloc (pagecount,sizeof (struct page *),GFP_NOIO );if (!p->pagevec) {mempool_free (p, nfs_wdata_mempool); p =NULL ;
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 28/30 build fix for skb_emergency_protocol
In reply to Xiaotian FengQuoted 1 line
From
50d2e72527b3e821544cc97c4dd5b1e5a44b6659Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 13:21:10 +0800
Subject: [PATCH 28/30] build fix for skb_emergency_protocol
--- net/core/dev.c | 48 ++++++++++++++++++++++++------------------------ 1 files changed, 24 insertions(+), 24 deletions(-)
net/core/dev.c +24 -24
@@ -2791,6 +2791,30 @@ int __skb_bond_should_drop(struct sk_buff *skb, struct net_device *master) }EXPORT_SYMBOL (__skb_bond_should_drop); +/* +* Filter the protocols for which the reserves are adequate. +* +* Before adding a protocol make sure that it is either covered by the existing +* reserves, or add reserves covering the memory need of the new protocol's +* packet processing. +*/ +static int skb_emergency_protocol (struct sk_buff *skb) +{ +if (skb_emergency (skb)) +switch (skb->protocol) { +case __constant_htons (ETH_P_ARP ): +case __constant_htons (ETH_P_IP ): +case __constant_htons (ETH_P_IPV6 ): +case __constant_htons (ETH_P_8021Q ): +break ; + +default : +return 0 ; + } + +return 1 ; +} +static int __netif_receive_skb (struct sk_buff *skb) {struct packet_type *ptype, *pt_prev;
@@ -2942,30 +2966,6 @@ out:return ret; } -/* -* Filter the protocols for which the reserves are adequate. -* -* Before adding a protocol make sure that it is either covered by the existing -* reserves, or add reserves covering the memory need of the new protocol's -* packet processing. -*/ -static int skb_emergency_protocol (struct sk_buff *skb) -{ -if (skb_emergency (skb)) -switch (skb->protocol) { -case __constant_htons (ETH_P_ARP ): -case __constant_htons (ETH_P_IP ): -case __constant_htons (ETH_P_IPV6 ): -case __constant_htons (ETH_P_8021Q ): -break ; - -default : -return 0 ; - } - -return 1 ; -} -/** * netif_receive_skb - process receive buffer from network * @skb: buffer to process
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 29/30 fix null pointer deref in swap_entry_free
In reply to Xiaotian FengQuoted 1 line
From
ea7b13006f42f7dcadd1bfb874d5e525b4c259e3Mon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 13:44:08 +0800
Subject: [PATCH 29/30] fix null pointer deref in swap_entry_free
Commit b3a27d uses p->bdev->bd_disk, this will lead a null pointer deref with swap over nfs.
--- mm/swapfile.c | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-)
mm/swapfile.c +5 -4
@@ -577,7 +577,6 @@ static unsigned char swap_entry_free(struct swap_info_struct *p,/* free if no reference */ if (!usage) { -struct gendisk *disk = p->bdev->bd_disk;if (offset < p->lowest_bit) p->lowest_bit = offset;if (offset > p->highest_bit)
@@ -587,9 +586,11 @@ static unsigned char swap_entry_free(struct swap_info_struct *p, swap_list.next = p->type; nr_swap_pages++; p->inuse_pages--; -if ((p->flags &SWP_BLKDEV ) && - disk->fops->swap_slot_free_notify) - disk->fops->swap_slot_free_notify (p->bdev, offset); +if (p->flags &SWP_BLKDEV ) { +struct gendisk *disk = p->bdev->bd_disk; +if (disk->fops->swap_slot_free_notify) + disk->fops->swap_slot_free_notify (p->bdev, offset); + } }return usage;
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 30/30 fix mess up on swap with multi files from same nfs server
In reply to Xiaotian FengQuoted 1 line
From
fd03848cadf5719228f617b72039cc8302d892efMon Sep 17 00:00:00 2001
From: Xiaotian Feng <[email protected]>
Date: Tue, 13 Jul 2010 14:00:02 +0800
Subject: [PATCH 30/30] fix mess up on swap with multi files from same nfs server
xs_swapper() will set xprt->swapper when swapon nfs files, unset xprt->swapper when swapoff nfs files. This will lead a bug if we swapon multi files from the same nfs server, they had the same xprt, then the reserved memory could not be disconnected when we swapoff all files.
--- include/linux/sunrpc/xprt.h | 4 ++-- net/sunrpc/xprtsock.c | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-)
include/linux/sunrpc/xprt.h +2 -2
@@ -171,8 +171,8 @@ struct rpc_xprt {unsigned int max_reqs;/* total slots */ unsigned long state;/* transport state */ unsigned char shutdown : 1, /* being shut down */ - resvport : 1, /* use a reserved port */ - swapper : 1; /* we're swapping over this + resvport : 1; /* use a reserved port */ +unsigned int swapper;/* we're swapping over this transport */ unsigned int bind_index;/* bind function index */
net/sunrpc/xprtsock.c +2 -2
@@ -1662,11 +1662,11 @@ int xs_swapper(struct rpc_xprt *xprt, int enable)*/ err =sk_adjust_memalloc (1 ,RPC_RESERVE_PAGES );if (!err) { - xprt->swapper =1 ; + xprt->swapper++;xs_set_memalloc (xprt); } }else if (xprt->swapper) { - xprt->swapper =0 ; + xprt->swapper--;sk_clear_memalloc (transport->inet);sk_adjust_memalloc (-1 , -RPC_RESERVE_PAGES ); }
Signature · 7 lines
1.7.1.1 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 27 lines · Xiaotian Feng
Hi,
Here's the latest version of swap over NFS series since -v20 last October. We decide to push this feature as it is useful for NAS or virt environment.
The patches are against the mmotm-2010-07-01. We can split the patchset into following parts:
Patch 1 - 12: provides a generic reserve framework. This framework could also be used to get rid of some of the
__GFP_NOFAILusers.Patch 13 - 15: Provide some generic network infrastructure needed later on.
Patch 16 - 21: reserve a little pool to act as a receive buffer, this allows us to inspect packets before tossing them.
Patch 22 - 23: Generic vm infrastructure to handle swapping to a filesystem instead of a block device.
Patch 24 - 27: convert NFS to make use of the new network and vm infrastructure to provide swap over NFS.
Patch 28 - 30: minor bug fixing with latest -mmotm.
[some history]
v19: http://lwn.net/Articles/301915/
v20: http://lwn.net/Articles/355350/
Quoted 6 lines
Changes since v20:
- rebased to mmotm-2010-07-01
- dropped the null pointer deref patch for the root cause is wrongSWP_FILEenum
- some minor build fixes
- fix a null pointer deref with mmotm-2010-07-01
- fix a bug when swap with multi files on the same nfs server
Please use the "From:" line correctly, as stated in Documentation/SubmittingPatches:
The "from" line must be the very first line in the message body, and has the form:
From: Original Author <[email protected]>
The "from" line specifies who will be credited as the author of the patch in the permanent changelog. If the "from" line is missing, then the "From:" line from the email header will be used to determine the patch author in the changelog.
I think you are using git format-patch to generate those patches, please supply --author=<author> to git commit when you commit them to your local tree. (or git am
if the patches you received already had the correct From: line.)
Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html