kasan: resched in quarantine_remove_cache()

PATCH Applied Dmitry Vyukov 3 participants 3 messages Mar 8, 2017 – Mar 9, 2017

Subsystems (2)

Memory management KASAN

lore

Share & tools

lore

Dmitry Vyukov

author reviewer

2017-03-08 15:42 UTC #0 lore raw

PATCH 1/1 kasan: resched in quarantine_remove_cache()

We see reported stalls/lockups in quarantine_remove_cache() on machines with large amounts of RAM. quarantine_remove_cache() needs to scan whole quarantine in order to take out all objects belonging to the cache. Quarantine is currently 1/32-th of RAM, e.g. on a machine with 256GB of memory that will be 8GB. Moreover quarantine scanning is a walk over uncached linked list, which is slow.

Add cond_resched() after scanning of each non-empty batch of objects. Batches are specifically kept of reasonable size for quarantine_put(). On a machine with 256GB of RAM we should have ~512 non-empty batches, each with 16MB of objects.

Signed-off-by Dmitry Vyukov <[email protected]>

Cc [email protected]

Cc Andrey Ryabinin <[email protected]>

Cc Greg Thelen <[email protected]>

---
 mm/kasan/quarantine.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

mm/kasan/quarantine.c +8 -1

index 075422c3cee3..3021d2976dd6 100644
@@ -311,8 +311,15 @@ void quarantine_remove_cache(struct kmem_cache *cache)
 	on_each_cpu(per_cpu_remove_cache, cache, 1);
 
 	spin_lock_irqsave(&quarantine_lock, flags);
-	for (i = 0; i < QUARANTINE_BATCHES; i++)
+	for (i = 0; i < QUARANTINE_BATCHES; i++) {
+		if (qlist_empty(&global_quarantine[i]))
+			continue;
 		qlist_move_cache(&global_quarantine[i], &to_free, cache);
+		/* Scanning whole quarantine can take a while. */
+		spin_unlock_irqrestore(&quarantine_lock, flags);
+		cond_resched();
+		spin_lock_irqsave(&quarantine_lock, flags);
+	}
 	spin_unlock_irqrestore(&quarantine_lock, flags);
 
 	qlist_free_all(&to_free, cache);

Signature · 7 lines

2.12.0.246.ga2ecc84866-goog

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Andrew Morton

maintainer

2017-03-08 23:06 UTC #1 lore raw

Quoted 11 lines · On Wed, 8 Mar 2017 16:42:39 +0100 Dmitry Vyukov <[email protected]> wrote:

We see reported stalls/lockups in quarantine_remove_cache() on machines with large amounts of RAM. quarantine_remove_cache() needs to scan whole quarantine in order to take out all objects belonging to the cache. Quarantine is currently 1/32-th of RAM, e.g. on a machine with 256GB of memory that will be 8GB. Moreover quarantine scanning is a walk over uncached linked list, which is slow.
Add cond_resched() after scanning of each non-empty batch of objects. Batches are specifically kept of reasonable size for quarantine_put(). On a machine with 256GB of RAM we should have ~512 non-empty batches, each with 16MB of objects.

I'll add cc:stable to this one - softlockup reports on large machines is a pretty significant issue.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Andrey Ryabinin

Acked-by on v1 1/1

2017-03-09 08:36 UTC #2 lore raw

In reply to Dmitry Vyukov

Quoted 12 lines · Dmitry Vyukov

We see reported stalls/lockups in quarantine_remove_cache() on machines with large amounts of RAM. quarantine_remove_cache() needs to scan whole quarantine in order to take out all objects belonging to the cache. Quarantine is currently 1/32-th of RAM, e.g. on a machine with 256GB of memory that will be 8GB. Moreover quarantine scanning is a walk over uncached linked list, which is slow.
Add cond_resched() after scanning of each non-empty batch of objects. Batches are specifically kept of reasonable size for quarantine_put(). On a machine with 256GB of RAM we should have ~512 non-empty batches, each with 16MB of objects.

Quoted 5 lines

Signed-off-by Dmitry Vyukov <[email protected]>
Cc [email protected]
Cc [email protected]
Cc Andrey Ryabinin <[email protected]>
Cc Greg Thelen <[email protected]>

Acked-by Andrey Ryabinin <[email protected]>