Skip to contentSkip to conversation

xen: fixes for gnt and block

PATCH In review Roger Pau Monne 4 participants 23 messages Jul 31, 2013 – Aug 12, 2013

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover xen: fixes for gnt and block

This series contain a small bugfix for the grant table code (patch 1) and a couple of improvements to blkfront (patches 2 and 3) to make it work better if there's a shortage on available free grants.

#1 lore raw

PATCH 1/3 xen-gnt: prevent adding duplicate gnt callbacks

With the current implementation, the callback in the tail of the list can be added twice, because the check done in gnttab_request_free_callback is bogus, callback->next can be NULL if it is the last callback in the list. If we add the same callback twice we end up with an infinite loop, were callback == callback->next.

Replace this check with a proper one that iterates over the list to see if the callback has already been added.

Cc Konrad Rzeszutek Wilk <[email protected]>
Cc David Vrabel <[email protected]>

---
This patch should be backported to stable trees

---
 drivers/xen/grant-table.c |   13 +++++++++++--
 1 files changed, 11 insertions(+), 2 deletions(-)
drivers/xen/grant-table.c +11 -2
index 04c1b2d..d5418c1 100644
@@ -729,9 +729,18 @@ void gnttab_request_free_callback(struct gnttab_free_callback *callback,
 				  void (*fn)(void *), void *arg, u16 count)
 {
 	unsigned long flags;
+	struct gnttab_free_callback *cb;
+
 	spin_lock_irqsave(&gnttab_list_lock, flags);
-	if (callback->next)
-		goto out;
+
+	/* Check if the callback is already on the list */
+	cb = gnttab_free_callback_list;
+	while (cb) {
+		if (cb == callback)
+			goto out;
+		cb = cb->next;
+	}
+
 	callback->fn = fn;
 	callback->arg = arg;
 	callback->count = count;
Signature · 7 lines
1.7.7.5 (Apple Git-26)


_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel
#2 lore raw

PATCH 1/3 xen-gnt: prevent adding duplicate gnt callbacks

In reply to Roger Pau Monne

With the current implementation, the callback in the tail of the list can be added twice, because the check done in gnttab_request_free_callback is bogus, callback->next can be NULL if it is the last callback in the list. If we add the same callback twice we end up with an infinite loop, were callback == callback->next.

Replace this check with a proper one that iterates over the list to see if the callback has already been added.

Cc Konrad Rzeszutek Wilk <[email protected]>
Cc David Vrabel <[email protected]>

---
This patch should be backported to stable trees

---
 drivers/xen/grant-table.c |   13 +++++++++++--
 1 files changed, 11 insertions(+), 2 deletions(-)
drivers/xen/grant-table.c +11 -2
index 04c1b2d..d5418c1 100644
@@ -729,9 +729,18 @@ void gnttab_request_free_callback(struct gnttab_free_callback *callback,
 				  void (*fn)(void *), void *arg, u16 count)
 {
 	unsigned long flags;
+	struct gnttab_free_callback *cb;
+
 	spin_lock_irqsave(&gnttab_list_lock, flags);
-	if (callback->next)
-		goto out;
+
+	/* Check if the callback is already on the list */
+	cb = gnttab_free_callback_list;
+	while (cb) {
+		if (cb == callback)
+			goto out;
+		cb = cb->next;
+	}
+
 	callback->fn = fn;
 	callback->arg = arg;
 	callback->count = count;
Signature · 1 line
1.7.7.5 (Apple Git-26)
Acked-by on v1 1/3
#3 lore raw

Re: [PATCH 1/3] xen-gnt: prevent adding duplicate gnt callbacks

Quoted 8 lines · Roger Pau Monne

With the current implementation, the callback in the tail of the list can be added twice, because the check done in gnttab_request_free_callback is bogus, callback->next can be NULL if it is the last callback in the list. If we add the same callback twice we end up with an infinite loop, were callback == callback->next.

Replace this check with a proper one that iterates over the list to see if the callback has already been added.

Quoted 34 lines
Cc Konrad Rzeszutek Wilk <[email protected]>
Cc David Vrabel <[email protected]>

---
This patch should be backported to stable trees

---
 drivers/xen/grant-table.c |   13 +++++++++++--
 1 files changed, 11 insertions(+), 2 deletions(-)
drivers/xen/grant-table.c +11 -2
index 04c1b2d..d5418c1 100644
@@ -729,9 +729,18 @@ void gnttab_request_free_callback(struct gnttab_free_callback *callback,
 				  void (*fn)(void *), void *arg, u16 count)
 {
 	unsigned long flags;
+	struct gnttab_free_callback *cb;
+
 	spin_lock_irqsave(&gnttab_list_lock, flags);
-	if (callback->next)
-		goto out;
+
+	/* Check if the callback is already on the list */
+	cb = gnttab_free_callback_list;
+	while (cb) {
+		if (cb == callback)
+			goto out;
+		cb = cb->next;
+	}
+
 	callback->fn = fn;
 	callback->arg = arg;
 	callback->count = count;
Acked-by on v1 1/3
#4 lore raw

Re: [PATCH 1/3] xen-gnt: prevent adding duplicate gnt callbacks

In reply to Roger Pau Monne
Quoted 8 lines · Roger Pau Monne

With the current implementation, the callback in the tail of the list can be added twice, because the check done in gnttab_request_free_callback is bogus, callback->next can be NULL if it is the last callback in the list. If we add the same callback twice we end up with an infinite loop, were callback == callback->next.

Replace this check with a proper one that iterates over the list to see if the callback has already been added.

Quoted 34 lines
Cc Konrad Rzeszutek Wilk <[email protected]>
Cc David Vrabel <[email protected]>

---
This patch should be backported to stable trees

---
 drivers/xen/grant-table.c |   13 +++++++++++--
 1 files changed, 11 insertions(+), 2 deletions(-)
drivers/xen/grant-table.c +11 -2
index 04c1b2d..d5418c1 100644
@@ -729,9 +729,18 @@ void gnttab_request_free_callback(struct gnttab_free_callback *callback,
 				  void (*fn)(void *), void *arg, u16 count)
 {
 	unsigned long flags;
+	struct gnttab_free_callback *cb;
+
 	spin_lock_irqsave(&gnttab_list_lock, flags);
-	if (callback->next)
-		goto out;
+
+	/* Check if the callback is already on the list */
+	cb = gnttab_free_callback_list;
+	while (cb) {
+		if (cb == callback)
+			goto out;
+		cb = cb->next;
+	}
+
 	callback->fn = fn;
 	callback->arg = arg;
 	callback->count = count;
Reviewed-by on v1 1/3
#5 lore raw

Re: [PATCH 1/3] xen-gnt: prevent adding duplicate gnt callbacks

In reply to Roger Pau Monne
Quoted 8 lines · Roger Pau Monne

With the current implementation, the callback in the tail of the list can be added twice, because the check done in gnttab_request_free_callback is bogus, callback->next can be NULL if it is the last callback in the list. If we add the same callback twice we end up with an infinite loop, were callback == callback->next.

Replace this check with a proper one that iterates over the list to see if the callback has already been added.

As a minimal fix suitable for stable,

David

Reviewed-by on v1 1/3
#6 lore raw

Re: [PATCH 1/3] xen-gnt: prevent adding duplicate gnt callbacks

In reply to Roger Pau Monne
Quoted 8 lines · Roger Pau Monne

With the current implementation, the callback in the tail of the list can be added twice, because the check done in gnttab_request_free_callback is bogus, callback->next can be NULL if it is the last callback in the list. If we add the same callback twice we end up with an infinite loop, were callback == callback->next.

Replace this check with a proper one that iterates over the list to see if the callback has already been added.

As a minimal fix suitable for stable,

David

#7 lore raw

PATCH 2/3 xen-blkfront: improve aproximation of required grants per request

In reply to Roger Pau Monne

Improve the calculation of required grants to process a request by using nr_phys_segments instead of always assuming a request is going to use all posible segments.

nr_phys_segments contains the number of scatter-gather DMA addr+len pairs, which is basically what we put at every granted page. for_each_sg iterates over the DMA addr+len pairs and uses a grant page for each of them.

Cc Konrad Rzeszutek Wilk <[email protected]>
Cc David Vrabel <[email protected]>
---
 drivers/block/xen-blkfront.c |   11 +++++++----
 1 files changed, 7 insertions(+), 4 deletions(-)
drivers/block/xen-blkfront.c +7 -4
index 2e1ee34..187a437 100644
@@ -400,10 +400,13 @@ static int blkif_queue_request(struct request *req)
 	if (unlikely(info->connected != BLKIF_STATE_CONNECTED))
 		return 1;
 
-	max_grefs = info->max_indirect_segments ?
-		    info->max_indirect_segments +
-		    INDIRECT_GREFS(info->max_indirect_segments) :
-		    BLKIF_MAX_SEGMENTS_PER_REQUEST;
+	max_grefs = req->nr_phys_segments;
+	if (max_grefs > BLKIF_MAX_SEGMENTS_PER_REQUEST)
+		/*
+		 * If we are using indirect segments we need to account
+		 * for the indirect grefs used in the request.
+		 */
+		max_grefs += INDIRECT_GREFS(req->nr_phys_segments);
 
 	/* Check if we have enough grants to allocate a requests */
 	if (info->persistent_gnts_c < max_grefs) {
Signature · 7 lines
1.7.7.5 (Apple Git-26)


_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel
#8 lore raw

PATCH 2/3 xen-blkfront: improve aproximation of required grants per request

In reply to Roger Pau Monne

Improve the calculation of required grants to process a request by using nr_phys_segments instead of always assuming a request is going to use all posible segments.

nr_phys_segments contains the number of scatter-gather DMA addr+len pairs, which is basically what we put at every granted page. for_each_sg iterates over the DMA addr+len pairs and uses a grant page for each of them.

Cc Konrad Rzeszutek Wilk <[email protected]>
Cc David Vrabel <[email protected]>
---
 drivers/block/xen-blkfront.c |   11 +++++++----
 1 files changed, 7 insertions(+), 4 deletions(-)
drivers/block/xen-blkfront.c +7 -4
index 2e1ee34..187a437 100644
@@ -400,10 +400,13 @@ static int blkif_queue_request(struct request *req)
 	if (unlikely(info->connected != BLKIF_STATE_CONNECTED))
 		return 1;
 
-	max_grefs = info->max_indirect_segments ?
-		    info->max_indirect_segments +
-		    INDIRECT_GREFS(info->max_indirect_segments) :
-		    BLKIF_MAX_SEGMENTS_PER_REQUEST;
+	max_grefs = req->nr_phys_segments;
+	if (max_grefs > BLKIF_MAX_SEGMENTS_PER_REQUEST)
+		/*
+		 * If we are using indirect segments we need to account
+		 * for the indirect grefs used in the request.
+		 */
+		max_grefs += INDIRECT_GREFS(req->nr_phys_segments);
 
 	/* Check if we have enough grants to allocate a requests */
 	if (info->persistent_gnts_c < max_grefs) {
Signature · 1 line
1.7.7.5 (Apple Git-26)
Reviewed-by on v1 2/3
#9 lore raw

Re: [PATCH 2/3] xen-blkfront: improve aproximation of required grants per request

Quoted 8 lines · Roger Pau Monne

Improve the calculation of required grants to process a request by using nr_phys_segments instead of always assuming a request is going to use all posible segments.

nr_phys_segments contains the number of scatter-gather DMA addr+len pairs, which is basically what we put at every granted page. for_each_sg iterates over the DMA addr+len pairs and uses a grant page for each of them.

David

Reviewed-by on v1 2/3
#10 lore raw

Re: [Xen-devel] [PATCH 2/3] xen-blkfront: improve aproximation of required grants per request

In reply to Roger Pau Monne
Quoted 8 lines · Roger Pau Monne

Improve the calculation of required grants to process a request by using nr_phys_segments instead of always assuming a request is going to use all posible segments.

nr_phys_segments contains the number of scatter-gather DMA addr+len pairs, which is basically what we put at every granted page. for_each_sg iterates over the DMA addr+len pairs and uses a grant page for each of them.

David

#11 lore raw

PATCH 3/3 xen-blkfront: revoke foreign access for grants not mapped by the backend

In reply to Roger Pau Monne

There's no need to keep the foreign access in a grant if it is not persistently mapped by the backend. This allows us to free grants that are not mapped by the backend, thus preventing blkfront from hoarding all grants.

The main effect of this is that blkfront will only persistently map the same grants as the backend, and it will always try to use grants that are already mapped by the backend. Also the number of persistent grants in blkfront is the same as in blkback (and is controlled by the value in blkback).

Cc Konrad Rzeszutek Wilk <[email protected]>
Cc David Vrabel <[email protected]>
---
 drivers/block/xen-blkfront.c |   33 +++++++++++++++++++++++++++++----
 1 files changed, 29 insertions(+), 4 deletions(-)
drivers/block/xen-blkfront.c +29 -4
index 187a437..bc9fc7d 100644
@@ -1016,13 +1016,38 @@ static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info,
 	}
 	/* Add the persistent grant into the list of free grants */
 	for (i = 0; i < nseg; i++) {
-		list_add(&s->grants_used[i]->node, &info->persistent_gnts);
-		info->persistent_gnts_c++;
+		if (gnttab_query_foreign_access(s->grants_used[i]->gref)) {
+			/*
+			 * If the grant is still mapped by the backend (the
+			 * backend has chosen to make this grant persistent)
+			 * we add it at the head of the list, so it will be
+			 * reused first.
+			 */
+			list_add(&s->grants_used[i]->node, &info->persistent_gnts);
+			info->persistent_gnts_c++;
+		} else {
+			/*
+			 * If the grant is not mapped by the backend we end the
+			 * foreign access and add it to the tail of the list,
+			 * so it will not be picked again unless we run out of
+			 * persistent grants.
+			 */
+			gnttab_end_foreign_access(s->grants_used[i]->gref, 0, 0UL);
+			s->grants_used[i]->gref = GRANT_INVALID_REF;
+			list_add_tail(&s->grants_used[i]->node, &info->persistent_gnts);
+		}
 	}
 	if (s->req.operation == BLKIF_OP_INDIRECT) {
 		for (i = 0; i < INDIRECT_GREFS(nseg); i++) {
-			list_add(&s->indirect_grants[i]->node, &info->persistent_gnts);
-			info->persistent_gnts_c++;
+			if (gnttab_query_foreign_access(s->indirect_grants[i]->gref)) {
+				list_add(&s->indirect_grants[i]->node, &info->persistent_gnts);
+				info->persistent_gnts_c++;
+			} else {
+				gnttab_end_foreign_access(s->indirect_grants[i]->gref, 0, 0UL);
+				s->indirect_grants[i]->gref = GRANT_INVALID_REF;
+				list_add_tail(&s->indirect_grants[i]->node,
+				              &info->persistent_gnts);
+			}
 		}
 	}
 }
Signature · 7 lines
1.7.7.5 (Apple Git-26)


_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel
#12 lore raw

PATCH 3/3 xen-blkfront: revoke foreign access for grants not mapped by the backend

In reply to Roger Pau Monne

There's no need to keep the foreign access in a grant if it is not persistently mapped by the backend. This allows us to free grants that are not mapped by the backend, thus preventing blkfront from hoarding all grants.

The main effect of this is that blkfront will only persistently map the same grants as the backend, and it will always try to use grants that are already mapped by the backend. Also the number of persistent grants in blkfront is the same as in blkback (and is controlled by the value in blkback).

Cc Konrad Rzeszutek Wilk <[email protected]>
Cc David Vrabel <[email protected]>
---
 drivers/block/xen-blkfront.c |   33 +++++++++++++++++++++++++++++----
 1 files changed, 29 insertions(+), 4 deletions(-)
drivers/block/xen-blkfront.c +29 -4
index 187a437..bc9fc7d 100644
@@ -1016,13 +1016,38 @@ static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info,
 	}
 	/* Add the persistent grant into the list of free grants */
 	for (i = 0; i < nseg; i++) {
-		list_add(&s->grants_used[i]->node, &info->persistent_gnts);
-		info->persistent_gnts_c++;
+		if (gnttab_query_foreign_access(s->grants_used[i]->gref)) {
+			/*
+			 * If the grant is still mapped by the backend (the
+			 * backend has chosen to make this grant persistent)
+			 * we add it at the head of the list, so it will be
+			 * reused first.
+			 */
+			list_add(&s->grants_used[i]->node, &info->persistent_gnts);
+			info->persistent_gnts_c++;
+		} else {
+			/*
+			 * If the grant is not mapped by the backend we end the
+			 * foreign access and add it to the tail of the list,
+			 * so it will not be picked again unless we run out of
+			 * persistent grants.
+			 */
+			gnttab_end_foreign_access(s->grants_used[i]->gref, 0, 0UL);
+			s->grants_used[i]->gref = GRANT_INVALID_REF;
+			list_add_tail(&s->grants_used[i]->node, &info->persistent_gnts);
+		}
 	}
 	if (s->req.operation == BLKIF_OP_INDIRECT) {
 		for (i = 0; i < INDIRECT_GREFS(nseg); i++) {
-			list_add(&s->indirect_grants[i]->node, &info->persistent_gnts);
-			info->persistent_gnts_c++;
+			if (gnttab_query_foreign_access(s->indirect_grants[i]->gref)) {
+				list_add(&s->indirect_grants[i]->node, &info->persistent_gnts);
+				info->persistent_gnts_c++;
+			} else {
+				gnttab_end_foreign_access(s->indirect_grants[i]->gref, 0, 0UL);
+				s->indirect_grants[i]->gref = GRANT_INVALID_REF;
+				list_add_tail(&s->indirect_grants[i]->node,
+				              &info->persistent_gnts);
+			}
 		}
 	}
 }
Signature · 1 line
1.7.7.5 (Apple Git-26)
Acked-by on v1 3/3
#13 lore raw

Re: [PATCH 3/3] xen-blkfront: revoke foreign access for grants not mapped by the backend

Quoted 5 lines · Roger Pau Monne

There's no need to keep the foreign access in a grant if it is not persistently mapped by the backend. This allows us to free grants that are not mapped by the backend, thus preventing blkfront from hoarding all grants.

Quoted 7 lines

The main effect of this is that blkfront will only persistently map the same grants as the backend, and it will always try to use grants that are already mapped by the backend. Also the number of persistent grants in blkfront is the same as in blkback (and is controlled by the value in blkback).

Quoted 53 lines
Cc Konrad Rzeszutek Wilk <[email protected]>
Cc David Vrabel <[email protected]>
---
 drivers/block/xen-blkfront.c |   33 +++++++++++++++++++++++++++++----
 1 files changed, 29 insertions(+), 4 deletions(-)
drivers/block/xen-blkfront.c +29 -4
index 187a437..bc9fc7d 100644
@@ -1016,13 +1016,38 @@ static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info,
 	}
 	/* Add the persistent grant into the list of free grants */
 	for (i = 0; i < nseg; i++) {
-		list_add(&s->grants_used[i]->node, &info->persistent_gnts);
-		info->persistent_gnts_c++;
+		if (gnttab_query_foreign_access(s->grants_used[i]->gref)) {
+			/*
+			 * If the grant is still mapped by the backend (the
+			 * backend has chosen to make this grant persistent)
+			 * we add it at the head of the list, so it will be
+			 * reused first.
+			 */
+			list_add(&s->grants_used[i]->node, &info->persistent_gnts);
+			info->persistent_gnts_c++;
+		} else {
+			/*
+			 * If the grant is not mapped by the backend we end the
+			 * foreign access and add it to the tail of the list,
+			 * so it will not be picked again unless we run out of
+			 * persistent grants.
+			 */
+			gnttab_end_foreign_access(s->grants_used[i]->gref, 0, 0UL);
+			s->grants_used[i]->gref = GRANT_INVALID_REF;
+			list_add_tail(&s->grants_used[i]->node, &info->persistent_gnts);
+		}
 	}
 	if (s->req.operation == BLKIF_OP_INDIRECT) {
 		for (i = 0; i < INDIRECT_GREFS(nseg); i++) {
-			list_add(&s->indirect_grants[i]->node, &info->persistent_gnts);
-			info->persistent_gnts_c++;
+			if (gnttab_query_foreign_access(s->indirect_grants[i]->gref)) {
+				list_add(&s->indirect_grants[i]->node, &info->persistent_gnts);
+				info->persistent_gnts_c++;
+			} else {
+				gnttab_end_foreign_access(s->indirect_grants[i]->gref, 0, 0UL);
+				s->indirect_grants[i]->gref = GRANT_INVALID_REF;
+				list_add_tail(&s->indirect_grants[i]->node,
+				              &info->persistent_gnts);
+			}
 		}
 	}
 }
Acked-by on v1 3/3
#14 lore raw

Re: [PATCH 3/3] xen-blkfront: revoke foreign access for grants not mapped by the backend

In reply to Roger Pau Monne
Quoted 5 lines · Roger Pau Monne

There's no need to keep the foreign access in a grant if it is not persistently mapped by the backend. This allows us to free grants that are not mapped by the backend, thus preventing blkfront from hoarding all grants.

Quoted 7 lines

The main effect of this is that blkfront will only persistently map the same grants as the backend, and it will always try to use grants that are already mapped by the backend. Also the number of persistent grants in blkfront is the same as in blkback (and is controlled by the value in blkback).

Quoted 53 lines
Cc Konrad Rzeszutek Wilk <[email protected]>
Cc David Vrabel <[email protected]>
---
 drivers/block/xen-blkfront.c |   33 +++++++++++++++++++++++++++++----
 1 files changed, 29 insertions(+), 4 deletions(-)
drivers/block/xen-blkfront.c +29 -4
index 187a437..bc9fc7d 100644
@@ -1016,13 +1016,38 @@ static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info,
 	}
 	/* Add the persistent grant into the list of free grants */
 	for (i = 0; i < nseg; i++) {
-		list_add(&s->grants_used[i]->node, &info->persistent_gnts);
-		info->persistent_gnts_c++;
+		if (gnttab_query_foreign_access(s->grants_used[i]->gref)) {
+			/*
+			 * If the grant is still mapped by the backend (the
+			 * backend has chosen to make this grant persistent)
+			 * we add it at the head of the list, so it will be
+			 * reused first.
+			 */
+			list_add(&s->grants_used[i]->node, &info->persistent_gnts);
+			info->persistent_gnts_c++;
+		} else {
+			/*
+			 * If the grant is not mapped by the backend we end the
+			 * foreign access and add it to the tail of the list,
+			 * so it will not be picked again unless we run out of
+			 * persistent grants.
+			 */
+			gnttab_end_foreign_access(s->grants_used[i]->gref, 0, 0UL);
+			s->grants_used[i]->gref = GRANT_INVALID_REF;
+			list_add_tail(&s->grants_used[i]->node, &info->persistent_gnts);
+		}
 	}
 	if (s->req.operation == BLKIF_OP_INDIRECT) {
 		for (i = 0; i < INDIRECT_GREFS(nseg); i++) {
-			list_add(&s->indirect_grants[i]->node, &info->persistent_gnts);
-			info->persistent_gnts_c++;
+			if (gnttab_query_foreign_access(s->indirect_grants[i]->gref)) {
+				list_add(&s->indirect_grants[i]->node, &info->persistent_gnts);
+				info->persistent_gnts_c++;
+			} else {
+				gnttab_end_foreign_access(s->indirect_grants[i]->gref, 0, 0UL);
+				s->indirect_grants[i]->gref = GRANT_INVALID_REF;
+				list_add_tail(&s->indirect_grants[i]->node,
+				              &info->persistent_gnts);
+			}
 		}
 	}
 }
Reviewed-by on v1 3/3
#15 lore raw

Re: [PATCH 3/3] xen-blkfront: revoke foreign access for grants not mapped by the backend

In reply to Roger Pau Monne
Quoted 5 lines · Roger Pau Monne

There's no need to keep the foreign access in a grant if it is not persistently mapped by the backend. This allows us to free grants that are not mapped by the backend, thus preventing blkfront from hoarding all grants.

Quoted 7 lines

The main effect of this is that blkfront will only persistently map the same grants as the backend, and it will always try to use grants that are already mapped by the backend. Also the number of persistent grants in blkfront is the same as in blkback (and is controlled by the value in blkback).

But please see the documentation updates needed below.

Quoted 18 lines
drivers/block/xen-blkfront.c +10 -2
@@ -1016,13 +1016,38 @@ static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info,
 	}
 	/* Add the persistent grant into the list of free grants */
 	for (i = 0; i < nseg; i++) {
-		list_add(&s->grants_used[i]->node, &info->persistent_gnts);
-		info->persistent_gnts_c++;
+		if (gnttab_query_foreign_access(s->grants_used[i]->gref)) {
+			/*
+			 * If the grant is still mapped by the backend (the
+			 * backend has chosen to make this grant persistent)
+			 * we add it at the head of the list, so it will be
+			 * reused first.
+			 */
+			list_add(&s->grants_used[i]->node, &info->persistent_gnts);
+			info->persistent_gnts_c++;
+		} else {

drivers/block/xen-blkfront.c static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info, · line 1038

Quoted 10 lines
drivers/block/xen-blkfront.c +10 -0
+			/*
+			 * If the grant is not mapped by the backend we end the
+			 * foreign access and add it to the tail of the list,
+			 * so it will not be picked again unless we run out of
+			 * persistent grants.
+			 */
+			gnttab_end_foreign_access(s->grants_used[i]->gref, 0, 0UL);
+			s->grants_used[i]->gref = GRANT_INVALID_REF;
+			list_add_tail(&s->grants_used[i]->node, &info->persistent_gnts);
+		}

Because you only reclaim grants when a request is completed, can you add text similar to the following to xen/include/public/io/blkif.h (and the Linux copy).

feature-persistent:
...
When the backend driver needs to unmap a persistent grant it should do so prior to completing a request that used that grant reference. If a persistent grant is unmapped at any other time, the frontend driver may not notice and may be unable to reclaim the grant reference.

Thanks.

David

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel

Reviewed-by on v1 3/3
#16 lore raw

Re: [Xen-devel] [PATCH 3/3] xen-blkfront: revoke foreign access for grants not mapped by the backend

In reply to Roger Pau Monne
Quoted 5 lines · Roger Pau Monne

There's no need to keep the foreign access in a grant if it is not persistently mapped by the backend. This allows us to free grants that are not mapped by the backend, thus preventing blkfront from hoarding all grants.

Quoted 7 lines

The main effect of this is that blkfront will only persistently map the same grants as the backend, and it will always try to use grants that are already mapped by the backend. Also the number of persistent grants in blkfront is the same as in blkback (and is controlled by the value in blkback).

But please see the documentation updates needed below.

Quoted 18 lines
drivers/block/xen-blkfront.c +10 -2
@@ -1016,13 +1016,38 @@ static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info,
 	}
 	/* Add the persistent grant into the list of free grants */
 	for (i = 0; i < nseg; i++) {
-		list_add(&s->grants_used[i]->node, &info->persistent_gnts);
-		info->persistent_gnts_c++;
+		if (gnttab_query_foreign_access(s->grants_used[i]->gref)) {
+			/*
+			 * If the grant is still mapped by the backend (the
+			 * backend has chosen to make this grant persistent)
+			 * we add it at the head of the list, so it will be
+			 * reused first.
+			 */
+			list_add(&s->grants_used[i]->node, &info->persistent_gnts);
+			info->persistent_gnts_c++;
+		} else {

drivers/block/xen-blkfront.c static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info, · line 1038

Quoted 10 lines
drivers/block/xen-blkfront.c +10 -0
+			/*
+			 * If the grant is not mapped by the backend we end the
+			 * foreign access and add it to the tail of the list,
+			 * so it will not be picked again unless we run out of
+			 * persistent grants.
+			 */
+			gnttab_end_foreign_access(s->grants_used[i]->gref, 0, 0UL);
+			s->grants_used[i]->gref = GRANT_INVALID_REF;
+			list_add_tail(&s->grants_used[i]->node, &info->persistent_gnts);
+		}

Because you only reclaim grants when a request is completed, can you add text similar to the following to xen/include/public/io/blkif.h (and the Linux copy).

feature-persistent:
...
When the backend driver needs to unmap a persistent grant it should do so prior to completing a request that used that grant reference. If a persistent grant is unmapped at any other time, the frontend driver may not notice and may be unable to reclaim the grant reference.

Thanks.

David

#17 lore raw

Re: [PATCH 3/3] xen-blkfront: revoke foreign access for grants not mapped by the backend

Quoted 12 lines · David Vrabel
Quoted 11 lines · Roger Pau Monne

There's no need to keep the foreign access in a grant if it is not persistently mapped by the backend. This allows us to free grants that are not mapped by the backend, thus preventing blkfront from hoarding all grants.

The main effect of this is that blkfront will only persistently map the same grants as the backend, and it will always try to use grants that are already mapped by the backend. Also the number of persistent grants in blkfront is the same as in blkback (and is controlled by the value in blkback).

Quoted 3 lines
Quoted 1 line

Roger,

Could you repost patch #2 and #3 (as #1 is in v3.11-rc4) with the comments and the Ack from Matt and Roger's Review-by tag addressed?

Thanks.

Quoted 52 lines

But please see the documentation updates needed below.

Quoted 18 lines
drivers/block/xen-blkfront.c +10 -2
@@ -1016,13 +1016,38 @@ static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info,
 	}
 	/* Add the persistent grant into the list of free grants */
 	for (i = 0; i < nseg; i++) {
-		list_add(&s->grants_used[i]->node, &info->persistent_gnts);
-		info->persistent_gnts_c++;
+		if (gnttab_query_foreign_access(s->grants_used[i]->gref)) {
+			/*
+			 * If the grant is still mapped by the backend (the
+			 * backend has chosen to make this grant persistent)
+			 * we add it at the head of the list, so it will be
+			 * reused first.
+			 */
+			list_add(&s->grants_used[i]->node, &info->persistent_gnts);
+			info->persistent_gnts_c++;
+		} else {

drivers/block/xen-blkfront.c static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info, · line 1038

Quoted 10 lines
drivers/block/xen-blkfront.c +10 -0
+			/*
+			 * If the grant is not mapped by the backend we end the
+			 * foreign access and add it to the tail of the list,
+			 * so it will not be picked again unless we run out of
+			 * persistent grants.
+			 */
+			gnttab_end_foreign_access(s->grants_used[i]->gref, 0, 0UL);
+			s->grants_used[i]->gref = GRANT_INVALID_REF;
+			list_add_tail(&s->grants_used[i]->node, &info->persistent_gnts);
+		}

Because you only reclaim grants when a request is completed, can you add text similar to the following to xen/include/public/io/blkif.h (and the Linux copy).

feature-persistent:
...
When the backend driver needs to unmap a persistent grant it should do so prior to completing a request that used that grant reference. If a persistent grant is unmapped at any other time, the frontend driver may not notice and may be unable to reclaim the grant reference.

Thanks.

David

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel

#18 lore raw

Re: [Xen-devel] [PATCH 3/3] xen-blkfront: revoke foreign access for grants not mapped by the backend

In reply to David Vrabel
Quoted 12 lines · David Vrabel
Quoted 11 lines · Roger Pau Monne

There's no need to keep the foreign access in a grant if it is not persistently mapped by the backend. This allows us to free grants that are not mapped by the backend, thus preventing blkfront from hoarding all grants.

The main effect of this is that blkfront will only persistently map the same grants as the backend, and it will always try to use grants that are already mapped by the backend. Also the number of persistent grants in blkfront is the same as in blkback (and is controlled by the value in blkback).

Quoted 3 lines
Quoted 1 line

Roger,

Could you repost patch #2 and #3 (as #1 is in v3.11-rc4) with the comments and the Ack from Matt and Roger's Review-by tag addressed?

Thanks.

Quoted 52 lines

But please see the documentation updates needed below.

Quoted 18 lines
drivers/block/xen-blkfront.c +10 -2
@@ -1016,13 +1016,38 @@ static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info,
 	}
 	/* Add the persistent grant into the list of free grants */
 	for (i = 0; i < nseg; i++) {
-		list_add(&s->grants_used[i]->node, &info->persistent_gnts);
-		info->persistent_gnts_c++;
+		if (gnttab_query_foreign_access(s->grants_used[i]->gref)) {
+			/*
+			 * If the grant is still mapped by the backend (the
+			 * backend has chosen to make this grant persistent)
+			 * we add it at the head of the list, so it will be
+			 * reused first.
+			 */
+			list_add(&s->grants_used[i]->node, &info->persistent_gnts);
+			info->persistent_gnts_c++;
+		} else {

drivers/block/xen-blkfront.c static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info, · line 1038

Quoted 10 lines
drivers/block/xen-blkfront.c +10 -0
+			/*
+			 * If the grant is not mapped by the backend we end the
+			 * foreign access and add it to the tail of the list,
+			 * so it will not be picked again unless we run out of
+			 * persistent grants.
+			 */
+			gnttab_end_foreign_access(s->grants_used[i]->gref, 0, 0UL);
+			s->grants_used[i]->gref = GRANT_INVALID_REF;
+			list_add_tail(&s->grants_used[i]->node, &info->persistent_gnts);
+		}

Because you only reclaim grants when a request is completed, can you add text similar to the following to xen/include/public/io/blkif.h (and the Linux copy).

feature-persistent:
...
When the backend driver needs to unmap a persistent grant it should do so prior to completing a request that used that grant reference. If a persistent grant is unmapped at any other time, the frontend driver may not notice and may be unable to reclaim the grant reference.

Thanks.

David

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel

#19 lore raw

Re: [PATCH 3/3] xen-blkfront: revoke foreign access for grants not mapped by the backend

Quoted 13 lines · Konrad Rzeszutek Wilk
Quoted 12 lines · David Vrabel
Quoted 11 lines · Roger Pau Monne

There's no need to keep the foreign access in a grant if it is not persistently mapped by the backend. This allows us to free grants that are not mapped by the backend, thus preventing blkfront from hoarding all grants.

The main effect of this is that blkfront will only persistently map the same grants as the backend, and it will always try to use grants that are already mapped by the backend. Also the number of persistent grants in blkfront is the same as in blkback (and is controlled by the value in blkback).

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Roger,

Could you repost patch #2 and #3 (as #1 is in v3.11-rc4) with the comments and the Ack from Matt and Roger's Review-by tag addressed?

Done (I guess you meant the Reviewed-by tag from David), see below however.

Quoted 45 lines

Thanks.

Quoted 44 lines

But please see the documentation updates needed below.

Quoted 18 lines
drivers/block/xen-blkfront.c +10 -2
@@ -1016,13 +1016,38 @@ static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info,
 	}
 	/* Add the persistent grant into the list of free grants */
 	for (i = 0; i < nseg; i++) {
-		list_add(&s->grants_used[i]->node, &info->persistent_gnts);
-		info->persistent_gnts_c++;
+		if (gnttab_query_foreign_access(s->grants_used[i]->gref)) {
+			/*
+			 * If the grant is still mapped by the backend (the
+			 * backend has chosen to make this grant persistent)
+			 * we add it at the head of the list, so it will be
+			 * reused first.
+			 */
+			list_add(&s->grants_used[i]->node, &info->persistent_gnts);
+			info->persistent_gnts_c++;
+		} else {

drivers/block/xen-blkfront.c static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info, · line 1038

Quoted 10 lines
drivers/block/xen-blkfront.c +10 -0
+			/*
+			 * If the grant is not mapped by the backend we end the
+			 * foreign access and add it to the tail of the list,
+			 * so it will not be picked again unless we run out of
+			 * persistent grants.
+			 */
+			gnttab_end_foreign_access(s->grants_used[i]->gref, 0, 0UL);
+			s->grants_used[i]->gref = GRANT_INVALID_REF;
+			list_add_tail(&s->grants_used[i]->node, &info->persistent_gnts);
+		}

Because you only reclaim grants when a request is completed, can you add text similar to the following to xen/include/public/io/blkif.h (and the Linux copy).

feature-persistent:
...
When the backend driver needs to unmap a persistent grant it should do so prior to completing a request that used that grant reference. If a persistent grant is unmapped at any other time, the frontend driver may not notice and may be unable to reclaim the grant reference.

I've sent a patch to update the documentation about persistent grants in Xen source tree, but the Linux copy of blkif.h lacks any information about the blkif protocol extensions, so I have not added it there. If nobody else does it first (before I come back from vacation), I will send a patch to add the documentation about all the blkif extensions to the Linux copy of blkif.h (and of course add your comment about persistent grants unmap).

Roger.

#20 lore raw

Re: [Xen-devel] [PATCH 3/3] xen-blkfront: revoke foreign access for grants not mapped by the backend

In reply to Konrad Rzeszutek Wilk
Quoted 13 lines · Konrad Rzeszutek Wilk
Quoted 12 lines · David Vrabel
Quoted 11 lines · Roger Pau Monne

There's no need to keep the foreign access in a grant if it is not persistently mapped by the backend. This allows us to free grants that are not mapped by the backend, thus preventing blkfront from hoarding all grants.

The main effect of this is that blkfront will only persistently map the same grants as the backend, and it will always try to use grants that are already mapped by the backend. Also the number of persistent grants in blkfront is the same as in blkback (and is controlled by the value in blkback).

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Roger,

Could you repost patch #2 and #3 (as #1 is in v3.11-rc4) with the comments and the Ack from Matt and Roger's Review-by tag addressed?

Done (I guess you meant the Reviewed-by tag from David), see below however.

Quoted 45 lines

Thanks.

Quoted 44 lines

But please see the documentation updates needed below.

Quoted 18 lines
drivers/block/xen-blkfront.c +10 -2
@@ -1016,13 +1016,38 @@ static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info,
 	}
 	/* Add the persistent grant into the list of free grants */
 	for (i = 0; i < nseg; i++) {
-		list_add(&s->grants_used[i]->node, &info->persistent_gnts);
-		info->persistent_gnts_c++;
+		if (gnttab_query_foreign_access(s->grants_used[i]->gref)) {
+			/*
+			 * If the grant is still mapped by the backend (the
+			 * backend has chosen to make this grant persistent)
+			 * we add it at the head of the list, so it will be
+			 * reused first.
+			 */
+			list_add(&s->grants_used[i]->node, &info->persistent_gnts);
+			info->persistent_gnts_c++;
+		} else {

drivers/block/xen-blkfront.c static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info, · line 1038

Quoted 10 lines
drivers/block/xen-blkfront.c +10 -0
+			/*
+			 * If the grant is not mapped by the backend we end the
+			 * foreign access and add it to the tail of the list,
+			 * so it will not be picked again unless we run out of
+			 * persistent grants.
+			 */
+			gnttab_end_foreign_access(s->grants_used[i]->gref, 0, 0UL);
+			s->grants_used[i]->gref = GRANT_INVALID_REF;
+			list_add_tail(&s->grants_used[i]->node, &info->persistent_gnts);
+		}

Because you only reclaim grants when a request is completed, can you add text similar to the following to xen/include/public/io/blkif.h (and the Linux copy).

feature-persistent:
...
When the backend driver needs to unmap a persistent grant it should do so prior to completing a request that used that grant reference. If a persistent grant is unmapped at any other time, the frontend driver may not notice and may be unable to reclaim the grant reference.

I've sent a patch to update the documentation about persistent grants in Xen source tree, but the Linux copy of blkif.h lacks any information about the blkif protocol extensions, so I have not added it there. If nobody else does it first (before I come back from vacation), I will send a patch to add the documentation about all the blkif extensions to the Linux copy of blkif.h (and of course add your comment about persistent grants unmap).

Roger.

#21 lore raw

Re: [PATCH 3/3] xen-blkfront: revoke foreign access for grants not mapped by the backend

Quoted 71 lines · Roger Pau Monné
Quoted 13 lines · Konrad Rzeszutek Wilk
Quoted 12 lines · David Vrabel
Quoted 11 lines · Roger Pau Monne

There's no need to keep the foreign access in a grant if it is not persistently mapped by the backend. This allows us to free grants that are not mapped by the backend, thus preventing blkfront from hoarding all grants.

The main effect of this is that blkfront will only persistently map the same grants as the backend, and it will always try to use grants that are already mapped by the backend. Also the number of persistent grants in blkfront is the same as in blkback (and is controlled by the value in blkback).

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Roger,

Could you repost patch #2 and #3 (as #1 is in v3.11-rc4) with the comments and the Ack from Matt and Roger's Review-by tag addressed?

Done (I guess you meant the Reviewed-by tag from David), see below however.

Quoted 45 lines

Thanks.

Quoted 44 lines

But please see the documentation updates needed below.

Quoted 18 lines
drivers/block/xen-blkfront.c +10 -2
@@ -1016,13 +1016,38 @@ static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info,
 	}
 	/* Add the persistent grant into the list of free grants */
 	for (i = 0; i < nseg; i++) {
-		list_add(&s->grants_used[i]->node, &info->persistent_gnts);
-		info->persistent_gnts_c++;
+		if (gnttab_query_foreign_access(s->grants_used[i]->gref)) {
+			/*
+			 * If the grant is still mapped by the backend (the
+			 * backend has chosen to make this grant persistent)
+			 * we add it at the head of the list, so it will be
+			 * reused first.
+			 */
+			list_add(&s->grants_used[i]->node, &info->persistent_gnts);
+			info->persistent_gnts_c++;
+		} else {

drivers/block/xen-blkfront.c static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info, · line 1038

Quoted 10 lines
drivers/block/xen-blkfront.c +10 -0
+			/*
+			 * If the grant is not mapped by the backend we end the
+			 * foreign access and add it to the tail of the list,
+			 * so it will not be picked again unless we run out of
+			 * persistent grants.
+			 */
+			gnttab_end_foreign_access(s->grants_used[i]->gref, 0, 0UL);
+			s->grants_used[i]->gref = GRANT_INVALID_REF;
+			list_add_tail(&s->grants_used[i]->node, &info->persistent_gnts);
+		}

Because you only reclaim grants when a request is completed, can you add text similar to the following to xen/include/public/io/blkif.h (and the Linux copy).

feature-persistent:
...
When the backend driver needs to unmap a persistent grant it should do so prior to completing a request that used that grant reference. If a persistent grant is unmapped at any other time, the frontend driver may not notice and may be unable to reclaim the grant reference.

Quoted 7 lines

I've sent a patch to update the documentation about persistent grants in Xen source tree, but the Linux copy of blkif.h lacks any information about the blkif protocol extensions, so I have not added it there. If nobody else does it first (before I come back from vacation), I will send a patch to add the documentation about all the blkif extensions to the Linux copy of blkif.h (and of course add your comment about persistent grants unmap).

Please do. It would be good to resync that header file. Thanks.

#22 lore raw

Re: [Xen-devel] [PATCH 3/3] xen-blkfront: revoke foreign access for grants not mapped by the backend

In reply to Roger Pau Monné
Quoted 71 lines · Roger Pau Monné
Quoted 13 lines · Konrad Rzeszutek Wilk
Quoted 12 lines · David Vrabel
Quoted 11 lines · Roger Pau Monne

There's no need to keep the foreign access in a grant if it is not persistently mapped by the backend. This allows us to free grants that are not mapped by the backend, thus preventing blkfront from hoarding all grants.

The main effect of this is that blkfront will only persistently map the same grants as the backend, and it will always try to use grants that are already mapped by the backend. Also the number of persistent grants in blkfront is the same as in blkback (and is controlled by the value in blkback).

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Roger,

Could you repost patch #2 and #3 (as #1 is in v3.11-rc4) with the comments and the Ack from Matt and Roger's Review-by tag addressed?

Done (I guess you meant the Reviewed-by tag from David), see below however.

Quoted 45 lines

Thanks.

Quoted 44 lines

But please see the documentation updates needed below.

Quoted 18 lines
drivers/block/xen-blkfront.c +10 -2
@@ -1016,13 +1016,38 @@ static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info,
 	}
 	/* Add the persistent grant into the list of free grants */
 	for (i = 0; i < nseg; i++) {
-		list_add(&s->grants_used[i]->node, &info->persistent_gnts);
-		info->persistent_gnts_c++;
+		if (gnttab_query_foreign_access(s->grants_used[i]->gref)) {
+			/*
+			 * If the grant is still mapped by the backend (the
+			 * backend has chosen to make this grant persistent)
+			 * we add it at the head of the list, so it will be
+			 * reused first.
+			 */
+			list_add(&s->grants_used[i]->node, &info->persistent_gnts);
+			info->persistent_gnts_c++;
+		} else {

drivers/block/xen-blkfront.c static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info, · line 1038

Quoted 10 lines
drivers/block/xen-blkfront.c +10 -0
+			/*
+			 * If the grant is not mapped by the backend we end the
+			 * foreign access and add it to the tail of the list,
+			 * so it will not be picked again unless we run out of
+			 * persistent grants.
+			 */
+			gnttab_end_foreign_access(s->grants_used[i]->gref, 0, 0UL);
+			s->grants_used[i]->gref = GRANT_INVALID_REF;
+			list_add_tail(&s->grants_used[i]->node, &info->persistent_gnts);
+		}

Because you only reclaim grants when a request is completed, can you add text similar to the following to xen/include/public/io/blkif.h (and the Linux copy).

feature-persistent:
...
When the backend driver needs to unmap a persistent grant it should do so prior to completing a request that used that grant reference. If a persistent grant is unmapped at any other time, the frontend driver may not notice and may be unable to reclaim the grant reference.

Quoted 7 lines

I've sent a patch to update the documentation about persistent grants in Xen source tree, but the Linux copy of blkif.h lacks any information about the blkif protocol extensions, so I have not added it there. If nobody else does it first (before I come back from vacation), I will send a patch to add the documentation about all the blkif extensions to the Linux copy of blkif.h (and of course add your comment about persistent grants unmap).

Please do. It would be good to resync that header file. Thanks.