Skip to contentSkip to conversation

xen-blkback: prevent premature module unload

PATCH Applied Paul Durrant 3 participants 7 messages Dec 10, 2019 – Dec 13, 2019

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xen-blkback: prevent premature module unload

Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem cache. This cache is destoyed when xen-blkif is unloaded so it is necessary to wait for the deferred free routine used for such objects to complete. This necessity was missed in commit 14855954f636 "xen-blkback:
allow module to be cleanly unloaded". This patch fixes the problem by taking/releasing extra module references in xen_blkif_alloc/free() respectively.

---
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
Cc: Jens Axboe <[email protected]>

---
 drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
drivers/block/xen-blkback/xenbus.c +10 -0
index e8c5c54e1d26..59d576d27ca7 100644
@@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t domid)
 	blkif->domid = domid;
 	atomic_set(&blkif->refcnt, 1);
 	init_completion(&blkif->drain_complete);
+
+	/*
+	 * Because freeing back to the cache may be deferred, it is not
+	 * safe to unload the module (and hence destroy the cache) until
+	 * this has completed. To prevent premature unloading, take an
+	 * extra module reference here and release only when the object
+	 * has been free back to the cache.
+	 */
+	__module_get(THIS_MODULE);
 	INIT_WORK(&blkif->free_work, xen_blkif_deferred_free);
 
 	return blkif;
@@ -320,6 +329,7 @@ static void xen_blkif_free(struct xen_blkif *blkif)
 
 	/* Make sure everything is drained before shutting down */
 	kmem_cache_free(xen_blkif_cachep, blkif);
+	module_put(THIS_MODULE);
 }
 
 int __init xen_blkif_interface_init(void)
Signature · 1 line
2.20.1
Reviewed-by on v1 1/1
#1 lore raw
Quoted 9 lines · Paul Durrant

Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem cache. This cache is destoyed when xen-blkif is unloaded so it is necessary to wait for the deferred free routine used for such objects to complete. This necessity was missed in commit 14855954f636 "xen-blkback:
allow module to be cleanly unloaded". This patch fixes the problem by taking/releasing extra module references in xen_blkif_alloc/free() respectively.

One nit below.

Quoted 13 lines

---
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
Cc: Jens Axboe <[email protected]>

---
 drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
drivers/block/xen-blkback/xenbus.c +0 -0
index e8c5c54e1d26..59d576d27ca7 100644

drivers/block/xen-blkback/xenbus.c static struct xen_blkif *xen_blkif_alloc(domid_t domid) · line 180

Quoted 10 lines
drivers/block/xen-blkback/xenbus.c +7 -0
@@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t domid)
 	blkif->domid = domid;
 	atomic_set(&blkif->refcnt, 1);
 	init_completion(&blkif->drain_complete);
+
+	/*
+	 * Because freeing back to the cache may be deferred, it is not
+	 * safe to unload the module (and hence destroy the cache) until
+	 * this has completed. To prevent premature unloading, take an
+	 * extra module reference here and release only when the object
+	 * has been free back to the cache.

^ freed

Quoted 16 lines
drivers/block/xen-blkback/xenbus.c +3 -0
+	 */
+	__module_get(THIS_MODULE);
 	INIT_WORK(&blkif->free_work, xen_blkif_deferred_free);
 
 	return blkif;
@@ -320,6 +329,7 @@ static void xen_blkif_free(struct xen_blkif *blkif)
 
 	/* Make sure everything is drained before shutting down */
 	kmem_cache_free(xen_blkif_cachep, blkif);
+	module_put(THIS_MODULE);
 }
 
 int __init xen_blkif_interface_init(void)
Signature · 1 line
2.20.1
#2 lore raw
Quoted 39 lines

-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 11 December 2019 11:29
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; linux- [email protected]; Konrad Rzeszutek Wilk <[email protected]>; Jens Axboe <[email protected]>
Subject: Re: [PATCH] xen-blkback: prevent premature module unload

Quoted 9 lines · Paul Durrant

Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem cache. This cache is destoyed when xen-blkif is unloaded so it is necessary to wait for the deferred free routine used for such objects to complete. This necessity was missed in commit 14855954f636 "xen-blkback:
allow module to be cleanly unloaded". This patch fixes the problem by taking/releasing extra module references in xen_blkif_alloc/free() respectively.

One nit below.

drivers/block/xen-

Quoted 9 lines

---
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
Cc: Jens Axboe <[email protected]>

---
 drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
drivers/block/xen- renamed from drivers/block/xen-blkback/xenbus.c +0 -0

blkback/xenbus.c

drivers/block/xen-blkback/xenbus.c static struct xen_blkif *xen_blkif_alloc(domid_t

Quoted 4 lines

index e8c5c54e1d26..59d576d27ca7 100644

drivers/block/xen-blkback/xenbus.c +0 -0
@@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t

domid)

Quoted 11 lines

drivers/block/xen-blkback/xenbus.c static struct xen_blkif *xen_blkif_alloc(domid_t domid) · line 180

Quoted 10 lines
drivers/block/xen-blkback/xenbus.c +7 -0
 	blkif->domid = domid;
 	atomic_set(&blkif->refcnt, 1);
 	init_completion(&blkif->drain_complete);
+
+	/*
+	 * Because freeing back to the cache may be deferred, it is not
+	 * safe to unload the module (and hence destroy the cache) until
+	 * this has completed. To prevent premature unloading, take an
+	 * extra module reference here and release only when the object
+	 * has been free back to the cache.

^ freed

Oh yes. Can this be done on commit, or would you like me to send a v2?

Paul

Quoted 16 lines
Quoted 16 lines
drivers/block/xen-blkback/xenbus.c +3 -1
+	 */
+	__module_get(THIS_MODULE);
 	INIT_WORK(&blkif->free_work, xen_blkif_deferred_free);
 
 	return blkif;
@@ -320,6 +329,7 @@ static void xen_blkif_free(struct xen_blkif *blkif)
 
 	/* Make sure everything is drained before shutting down */
 	kmem_cache_free(xen_blkif_cachep, blkif);
+	module_put(THIS_MODULE);
 }
 
 int __init xen_blkif_interface_init(void)
--

2.20.1

#3 lore raw
Quoted 51 lines · Paul
Quoted 39 lines

-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 11 December 2019 11:29
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; linux- [email protected]; Konrad Rzeszutek Wilk <[email protected]>; Jens Axboe <[email protected]>
Subject: Re: [PATCH] xen-blkback: prevent premature module unload

Quoted 9 lines · Paul Durrant

Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem cache. This cache is destoyed when xen-blkif is unloaded so it is necessary to wait for the deferred free routine used for such objects to complete. This necessity was missed in commit 14855954f636 "xen-blkback:
allow module to be cleanly unloaded". This patch fixes the problem by taking/releasing extra module references in xen_blkif_alloc/free() respectively.

One nit below.

drivers/block/xen-

Quoted 9 lines

---
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
Cc: Jens Axboe <[email protected]>

---
 drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
drivers/block/xen- renamed from drivers/block/xen-blkback/xenbus.c +0 -0

blkback/xenbus.c

drivers/block/xen-blkback/xenbus.c static struct xen_blkif *xen_blkif_alloc(domid_t

Quoted 4 lines

index e8c5c54e1d26..59d576d27ca7 100644

drivers/block/xen-blkback/xenbus.c +0 -0
@@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t

domid)

Quoted 11 lines

drivers/block/xen-blkback/xenbus.c static struct xen_blkif *xen_blkif_alloc(domid_t domid) · line 180

Quoted 10 lines
drivers/block/xen-blkback/xenbus.c +7 -0
 	blkif->domid = domid;
 	atomic_set(&blkif->refcnt, 1);
 	init_completion(&blkif->drain_complete);
+
+	/*
+	 * Because freeing back to the cache may be deferred, it is not
+	 * safe to unload the module (and hence destroy the cache) until
+	 * this has completed. To prevent premature unloading, take an
+	 * extra module reference here and release only when the object
+	 * has been free back to the cache.

^ freed

Quoted 1 line

Oh yes. Can this be done on commit, or would you like me to send a v2?

Adjusting on commit would be fine for me, but it's up to Juergen since he is the one that will pick this up. IIRC the module unload patches didn't go through the block subsystem.

Thanks, Roger.

#4 lore raw
Quoted 52 lines · Roger Pau Monné
Quoted 51 lines · Paul
Quoted 50 lines

-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 11 December 2019 11:29
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; linux- [email protected]; Konrad Rzeszutek Wilk <[email protected]>; Jens Axboe <[email protected]>
Subject: Re: [PATCH] xen-blkback: prevent premature module unload

Quoted 9 lines · Paul Durrant

Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem cache. This cache is destoyed when xen-blkif is unloaded so it is necessary to wait for the deferred free routine used for such objects to complete. This necessity was missed in commit 14855954f636 "xen-blkback:
allow module to be cleanly unloaded". This patch fixes the problem by taking/releasing extra module references in xen_blkif_alloc/free() respectively.

One nit below.

drivers/block/xen-

Quoted 9 lines

---
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
Cc: Jens Axboe <[email protected]>

---
  drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
  1 file changed, 10 insertions(+)
drivers/block/xen- renamed from drivers/block/xen-blkback/xenbus.c +0 -0

blkback/xenbus.c

drivers/block/xen-blkback/xenbus.c static struct xen_blkif *xen_blkif_alloc(domid_t

Quoted 4 lines

index e8c5c54e1d26..59d576d27ca7 100644

drivers/block/xen-blkback/xenbus.c +0 -0
@@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t

domid)

drivers/block/xen-blkback/xenbus.c static struct xen_blkif *xen_blkif_alloc(domid_t domid) · line 180

Quoted 10 lines
drivers/block/xen-blkback/xenbus.c +7 -0
  	blkif->domid = domid;
  	atomic_set(&blkif->refcnt, 1);
  	init_completion(&blkif->drain_complete);
+
+	/*
+	 * Because freeing back to the cache may be deferred, it is not
+	 * safe to unload the module (and hence destroy the cache) until
+	 * this has completed. To prevent premature unloading, take an
+	 * extra module reference here and release only when the object
+	 * has been free back to the cache.

^ freed

Quoted 5 lines
Quoted 1 line

Oh yes. Can this be done on commit, or would you like me to send a v2?

Adjusting on commit would be fine for me, but it's up to Juergen since he is the one that will pick this up. IIRC the module unload patches didn't go through the block subsystem.

Oh, right. Yes, will fix this when committing.

Juergen

#5 lore raw
In reply to Roger Pau Monné
Quoted 68 lines

-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 11 December 2019 13:55
To: Durrant, Paul <[email protected]>; Juergen Gross <[email protected]>
Cc: [email protected]; [email protected]; linux- [email protected]; Konrad Rzeszutek Wilk <[email protected]>; Jens Axboe <[email protected]>
Subject: Re: [PATCH] xen-blkback: prevent premature module unload

Quoted 5 lines · Paul
Quoted 5 lines

-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 11 December 2019 11:29
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];

linux-

Quoted 1 line
Quoted 1 line

[email protected]; Konrad Rzeszutek Wilk

<[email protected]>;

Quoted 5 lines
Quoted 5 lines

Jens Axboe <[email protected]>
Subject: Re: [PATCH] xen-blkback: prevent premature module unload

Quoted 1 line · Paul Durrant

Objects allocated by xen_blkif_alloc come from the 'blkif_cache'

kmem

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

cache. This cache is destoyed when xen-blkif is unloaded so it is necessary to wait for the deferred free routine used for such

objects to

Quoted 1 line
Quoted 1 line
Quoted 1 line

complete. This necessity was missed in commit 14855954f636 "xen-

blkback:

Quoted 1 line
Quoted 1 line
Quoted 1 line

allow module to be cleanly unloaded". This patch fixes the problem

by

Quoted 17 lines
Quoted 17 lines
Quoted 4 lines

taking/releasing extra module references in xen_blkif_alloc/free() respectively.

One nit below.

Quoted 8 lines

---
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
Cc: Jens Axboe <[email protected]>

---
 drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
Quoted 6 lines
Quoted 6 lines

drivers/block/xen-

Quoted 1 line
drivers/block/xen- renamed from drivers/block/xen-blkback/xenbus.c +0 -0

blkback/xenbus.c

drivers/block/xen-blkback/xenbus.c static struct xen_blkif

Quoted 4 lines

index e8c5c54e1d26..59d576d27ca7 100644

drivers/block/xen-blkback/xenbus.c +0 -0
@@ -171,6 +171,15 @@ static struct xen_blkif

*xen_blkif_alloc(domid_t

Quoted 7 lines
Quoted 7 lines

domid)

drivers/block/xen-blkback/xenbus.c static struct xen_blkif *xen_blkif_alloc(domid_t domid)

Quoted 6 lines
drivers/block/xen-blkback/xenbus.c +3 -0
 	blkif->domid = domid;
 	atomic_set(&blkif->refcnt, 1);
 	init_completion(&blkif->drain_complete);
+
+	/*
+	 * Because freeing back to the cache may be deferred, it is

not

Quoted 1 line
Quoted 1 line

drivers/block/xen-blkback/xenbus.c static struct xen_blkif · line 177

Quoted 1 line
drivers/block/xen-blkback/xenbus.c +1 -0
+	 * safe to unload the module (and hence destroy the cache)

until

Quoted 2 lines
Quoted 2 lines

drivers/block/xen-blkback/xenbus.c static struct xen_blkif · line 179

Quoted 2 lines
drivers/block/xen-blkback/xenbus.c +2 -0
+	 * this has completed. To prevent premature unloading, take an
+	 * extra module reference here and release only when the

object

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

drivers/block/xen-blkback/xenbus.c static struct xen_blkif *xen_blkif_alloc(domid_t domid) · line 180

Quoted 1 line
drivers/block/xen-blkback/xenbus.c +1 -0
+	 * has been free back to the cache.

^ freed

Oh yes. Can this be done on commit, or would you like me to send a v2?

Adjusting on commit would be fine for me, but it's up to Juergen since he is the one that will pick this up. IIRC the module unload patches didn't go through the block subsystem.

True. I forgot manually add Juergen cc list.

Paul

Quoted 2 lines

Thanks, Roger.

#6 lore raw
In reply to Paul Durrant
Quoted 9 lines · Paul Durrant

Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem cache. This cache is destoyed when xen-blkif is unloaded so it is necessary to wait for the deferred free routine used for such objects to complete. This necessity was missed in commit 14855954f636 "xen-blkback:
allow module to be cleanly unloaded". This patch fixes the problem by taking/releasing extra module references in xen_blkif_alloc/free() respectively.

Pushed to xen/tip.git for-linus-5.5b

Juergen