Skip to contentSkip to conversation

Third version patches for upstreaming grant table version 2

PATCH In review annie li 3 participants 8 messages Nov 21, 2011 – Nov 22, 2011

Subsystems (3)
lore
Share & tools
lore

annie li

author
#0 lore raw

Hi

The following patches introduce and implement grant table version 2. This is the third version patches, and based on v3.2.0-rc1+. Changes from the previous patches are following:

Patch 1:
* removing gnttab_v1_ops declaration
* renaming update_grant_entry_v1 to gnttab_update_entry_v1

Patch3:
* changing following code

xen_raw_printk();
panic();

back to

panic();
* re-arrange "else if" format in gnttab_request_version function.

Descriptions for those patches:

1. In those patches, the grant table code supports both grant table v1 and v2 version, v2 is an extension from v1. Grant table of guest domain can be either v1 or v2 version, and every grant table entry on one guest should be the same version.

2. Full page structure of grant table v2 play the same role as grant table v1. Although full page structure is different from v1, grant table 2 is totally backwards compatible with v1. Grant table is shared between guest and Xen, domu and dom0 all have their own grant table shared with Xen, and their grant table version should be set before any grants are activated. When domu grants an entry to dom0 to map a frame,following are steps: * domu introduces a grant entry by reference * domu informs dom0 the gref * dom0 sends hypercall to map frame through this reference, Xen copy shared entry to active entry and update frame * dom0 does its work and release the frame, Xen releases the entry. * domu redo those steps for a new cycle. Xen mapping process can be found in function __gnttab_map_grant_ref in Xen code: xen/common/grant_table.c

3. If dom0 supports grant table v2, guests run on it can either supports v1 or v2. Xen is responsible to judge what version the guests are using. This is implemented in Xen code: xen/common/grant_table.c. Key word is: rd->grant_table->gt_version.

4. Grant table v2 has been supported by Xen for a long time, and receiver-side copying mechanism bases on this implementation. Netback and netfront driver can pick up this new feature to get better network performance and better CPU accounting.

Diff:

arch/x86/xen/grant-table.c          |   42 ++++-
drivers/xen/grant-table.c           |  354 ++++++++++++++++++++++++++++++-----
include/xen/grant_table.h           |   10 +-
include/xen/interface/grant_table.h |  167 ++++++++++++++++-
include/xen/interface/xen.h         |    2 +
5 files changed, 519 insertions(+), 56 deletions(-)

Shortlog:
Annie Li (4):
xen/granttable: Introducing grant table V2 stucture xen/granttable: Refactor some code xen/granttable: Grant tables V2 implementation xen/granttable: Keep code format clean

Thanks
Annie

#1 lore raw

PATCH 1/4 xen/granttable: Introducing grant table V2 stucture

From: Annie Li <[email protected]>

This patch introduces new structures of grant table V2, grant table V2 is an extension from V1. Grant table is shared between guest and Xen, and Xen is responsible to do corresponding work for grant operations, such as: figure out guest's grant table version, perform different actions based on different grant table version, etc. Although full-page structure of V2 is different from V1, it play the same role as V1.

---
 arch/x86/xen/grant-table.c          |    7 +-
 drivers/xen/grant-table.c           |  181 +++++++++++++++++++++++++++--------
 include/xen/grant_table.h           |    4 +-
 include/xen/interface/grant_table.h |  167 +++++++++++++++++++++++++++++++-
 include/xen/interface/xen.h         |    2 +
 5 files changed, 310 insertions(+), 51 deletions(-)
arch/x86/xen/grant-table.c +3 -4
index 6bbfd7a..c6ab2e7 100644
@@ -64,10 +64,10 @@ static int unmap_pte_fn(pte_t *pte, struct page *pmd_page,
 
 int arch_gnttab_map_shared(unsigned long *frames, unsigned long nr_gframes,
 			   unsigned long max_nr_gframes,
-			   struct grant_entry **__shared)
+			   void **__shared)
 {
 	int rc;
-	struct grant_entry *shared = *__shared;
+	void *shared = *__shared;
 
 	if (shared == NULL) {
 		struct vm_struct *area =
@@ -83,8 +83,7 @@ int arch_gnttab_map_shared(unsigned long *frames, unsigned long nr_gframes,
 	return rc;
 }
 
-void arch_gnttab_unmap_shared(struct grant_entry *shared,
-			      unsigned long nr_gframes)
+void arch_gnttab_unmap_shared(void *shared, unsigned long nr_gframes)
 {
 	apply_to_page_range(&init_mm, (unsigned long)shared,
 			    PAGE_SIZE * nr_gframes, unmap_pte_fn, NULL);
drivers/xen/grant-table.c +141 -40

292 lines not on this page · raw message

include/xen/grant_table.h +2 -2
index 11e2dfc..c7a40f8 100644
@@ -145,8 +145,8 @@ gnttab_set_unmap_op(struct gnttab_unmap_grant_ref *unmap, phys_addr_t addr,
 
 int arch_gnttab_map_shared(unsigned long *frames, unsigned long nr_gframes,
 			   unsigned long max_nr_gframes,
-			   struct grant_entry **__shared);
-void arch_gnttab_unmap_shared(struct grant_entry *shared,
+			   void **__shared);
+void arch_gnttab_unmap_shared(void *shared,
 			      unsigned long nr_gframes);
 
 extern unsigned long xen_hvm_resume_frames;
include/xen/interface/grant_table.h +162 -5

217 lines not on this page · raw message

include/xen/interface/xen.h +2 -0
index 6a6e914..a890804 100644
@@ -523,6 +523,8 @@ struct tmem_op {
 	} u;
 };
 
+DEFINE_GUEST_HANDLE(u64);
+
 #else /* __ASSEMBLY__ */
 
 /* In assembly code we cannot use C numeric constant suffixes. */
Signature · 1 line
1.7.6.4
#2 lore raw

PATCH 2/4 xen/granttable: Refactor some code

In reply to annie li

From: Annie Li <[email protected]>

---
 drivers/xen/grant-table.c |   15 ++++++++++-----
 1 files changed, 10 insertions(+), 5 deletions(-)
drivers/xen/grant-table.c +10 -5
index 18355a5..0518d04 100644
@@ -257,15 +257,17 @@ EXPORT_SYMBOL_GPL(gnttab_query_foreign_access);
 static int gnttab_end_foreign_access_ref_v1(grant_ref_t ref, int readonly)
 {
 	u16 flags, nflags;
+	u16 *pflags;
 
-	nflags = gnttab_shared.v1[ref].flags;
+	pflags = &gnttab_shared.v1[ref].flags;
+	nflags = *pflags;
 	do {
 		flags = nflags;
 		if (flags & (GTF_reading|GTF_writing)) {
 			printk(KERN_ALERT "WARNING: g.e. still in use!\n");
 			return 0;
 		}
-	} while ((nflags = sync_cmpxchg(&gnttab_shared.v1[ref].flags, flags, 0)) != flags);
+	} while ((nflags = sync_cmpxchg(pflags, flags, 0)) != flags);
 
 	return 1;
 }
@@ -316,20 +318,23 @@ static unsigned long gnttab_end_foreign_transfer_ref_v1(grant_ref_t ref)
 {
 	unsigned long frame;
 	u16           flags;
+	u16          *pflags;
+
+	pflags = &gnttab_shared.v1[ref].flags;
 
 	/*
 	 * If a transfer is not even yet started, try to reclaim the grant
 	 * reference and return failure (== 0).
 	 */
-	while (!((flags = gnttab_shared.v1[ref].flags) & GTF_transfer_committed)) {
-		if (sync_cmpxchg(&gnttab_shared.v1[ref].flags, flags, 0) == flags)
+	while (!((flags = *pflags) & GTF_transfer_committed)) {
+		if (sync_cmpxchg(pflags, flags, 0) == flags)
 			return 0;
 		cpu_relax();
 	}
 
 	/* If a transfer is in progress then wait until it is completed. */
 	while (!(flags & GTF_transfer_completed)) {
-		flags = gnttab_shared.v1[ref].flags;
+		flags = *pflags;
 		cpu_relax();
 	}
 
Signature · 1 line
1.7.6.4
#3 lore raw

PATCH 3/4 xen/granttable: Grant tables V2 implementation

In reply to annie li

From: Annie Li <[email protected]>

Receiver-side copying of packets is based on this implementation, it gives better performance and better CPU accounting. It totally supports three types:
full-page, sub-page and transitive grants.

However this patch does not cover sub-page and transitive grants, it mainly focus on Full-page part and implements grant table V2 interfaces corresponding to what already exists in grant table V1, such as: grant table V2 initialization, mapping, releasing and exported interfaces.

Each guest can only supports one type of grant table type, every entry in grant table should be the same version. It is necessary to set V1 or V2 version before initializing the grant table.

Grant table exported interfaces of V2 are same with those of V1, Xen is responsible to judge what grant table version guests are using in every grant operation.

V2 fulfills the same role of V1, and it is totally backwards compitable with V1. If dom0 support grant table V2, the guests runing on it can run with either V1 or V2.

---
 arch/x86/xen/grant-table.c |   37 +++++++++-
 drivers/xen/grant-table.c  |  171 ++++++++++++++++++++++++++++++++++++++++++-
 include/xen/grant_table.h  |    6 +-
 3 files changed, 207 insertions(+), 7 deletions(-)
arch/x86/xen/grant-table.c +36 -1
index c6ab2e7..510fca0 100644
@@ -54,6 +54,20 @@ static int map_pte_fn(pte_t *pte, struct page *pmd_page,
 	return 0;
 }
 
+/*
+ * This function is used to map shared frames to store grant status. It is
+ * different from map_pte_fn above, the frames type here is uint64_t.
+ */
+static int map_pte_fn_status(pte_t *pte, struct page *pmd_page,
+			     unsigned long addr, void *data)
+{
+	uint64_t **frames = (uint64_t **)data;
+
+	set_pte_at(&init_mm, addr, pte, mfn_pte((*frames)[0], PAGE_KERNEL));
+	(*frames)++;
+	return 0;
+}
+
 static int unmap_pte_fn(pte_t *pte, struct page *pmd_page,
 			unsigned long addr, void *data)
 {
@@ -83,7 +97,28 @@ int arch_gnttab_map_shared(unsigned long *frames, unsigned long nr_gframes,
 	return rc;
 }
 
-void arch_gnttab_unmap_shared(void *shared, unsigned long nr_gframes)
+int arch_gnttab_map_status(uint64_t *frames, unsigned long nr_gframes,
+			   unsigned long max_nr_gframes,
+			   grant_status_t **__shared)
+{
+	int rc;
+	grant_status_t *shared = *__shared;
+
+	if (shared == NULL) {
+		struct vm_struct *area =
+			alloc_vm_area(PAGE_SIZE * max_nr_gframes);
+		BUG_ON(area == NULL);
+		shared = area->addr;
+		*__shared = shared;
+	}
+
+	rc = apply_to_page_range(&init_mm, (unsigned long)shared,
+				 PAGE_SIZE * nr_gframes,
+				 map_pte_fn_status, &frames);
+	return rc;
+}
+
+void arch_gnttab_unmap(void *shared, unsigned long nr_gframes)
 {
 	apply_to_page_range(&init_mm, (unsigned long)shared,
 			    PAGE_SIZE * nr_gframes, unmap_pte_fn, NULL);
drivers/xen/grant-table.c +167 -4

257 lines not on this page · raw message

include/xen/grant_table.h +4 -2
index c7a40f8..5494c40 100644
@@ -146,8 +146,10 @@ gnttab_set_unmap_op(struct gnttab_unmap_grant_ref *unmap, phys_addr_t addr,
 int arch_gnttab_map_shared(unsigned long *frames, unsigned long nr_gframes,
 			   unsigned long max_nr_gframes,
 			   void **__shared);
-void arch_gnttab_unmap_shared(void *shared,
-			      unsigned long nr_gframes);
+int arch_gnttab_map_status(uint64_t *frames, unsigned long nr_gframes,
+			   unsigned long max_nr_gframes,
+			   grant_status_t **__shared);
+void arch_gnttab_unmap(void *shared, unsigned long nr_gframes);
 
 extern unsigned long xen_hvm_resume_frames;
 unsigned int gnttab_max_grant_frames(void);
Signature · 1 line
1.7.6.4
#7 lore raw

PATCH 4/4 xen/granttable: Keep code format clean

In reply to annie li

From: Annie Li <[email protected]>

---
 drivers/xen/grant-table.c |    7 +++----
 include/xen/grant_table.h |    2 +-
 2 files changed, 4 insertions(+), 5 deletions(-)
drivers/xen/grant-table.c +3 -4
index 77a060c..b76f419 100644
@@ -50,7 +50,6 @@
 #include <asm/pgtable.h>
 #include <asm/sync_bitops.h>
 
-
 /* External tools reserve first few grant table entries. */
 #define NR_RESERVED_ENTRIES 8
 #define GNTTAB_LIST_END 0xffffffff
@@ -598,8 +597,8 @@ unsigned int gnttab_max_grant_frames(void)
 EXPORT_SYMBOL_GPL(gnttab_max_grant_frames);
 
 int gnttab_map_refs(struct gnttab_map_grant_ref *map_ops,
-			struct gnttab_map_grant_ref *kmap_ops,
-			struct page **pages, unsigned int count)
+		    struct gnttab_map_grant_ref *kmap_ops,
+		    struct page **pages, unsigned int count)
 {
 	int i, ret;
 	pte_t *pte;
@@ -649,7 +648,7 @@ int gnttab_map_refs(struct gnttab_map_grant_ref *map_ops,
 EXPORT_SYMBOL_GPL(gnttab_map_refs);
 
 int gnttab_unmap_refs(struct gnttab_unmap_grant_ref *unmap_ops,
-		struct page **pages, unsigned int count)
+		      struct page **pages, unsigned int count)
 {
 	int i, ret;
 
include/xen/grant_table.h +1 -1
index 5494c40..fea4954 100644
@@ -157,7 +157,7 @@ unsigned int gnttab_max_grant_frames(void);
 #define gnttab_map_vaddr(map) ((void *)(map.host_virt_addr))
 
 int gnttab_map_refs(struct gnttab_map_grant_ref *map_ops,
-			struct gnttab_map_grant_ref *kmap_ops,
+		    struct gnttab_map_grant_ref *kmap_ops,
 		    struct page **pages, unsigned int count);
 int gnttab_unmap_refs(struct gnttab_unmap_grant_ref *unmap_ops,
 		      struct page **pages, unsigned int count);
Signature · 1 line
1.7.6.4
Acked-by on v1 3/4
#4 lore raw

Re: [PATCH 3/4] xen/granttable: Grant tables V2 implementation

In reply to [email protected]

On Tue, 2011-11-22 at 01:59 +0000, [email protected] wrote:
[...]

drivers/xen/grant-table.c static int gnttab_end_foreign_access_ref_v1(grant_ref_t ref, int readonly) · line 317

Quoted 6 lines
drivers/xen/grant-table.c +6 -0
+#ifdef CONFIG_X86
+		barrier();
+#else
+		mb();
+#endif
+		}

Niggle: Indentation of this final } is off.

Otherwise all 4 patches:

[...]

author
#5 lore raw

Re: [PATCH 3/4] xen/granttable: Grant tables V2 implementation

Quoted 9 lines · Ian Campbell

On Tue, 2011-11-22 at 01:59 +0000, [email protected] wrote:
[...]

drivers/xen/grant-table.c static int gnttab_end_foreign_access_ref_v1(grant_ref_t ref, int readonly) · line 317

Quoted 6 lines
drivers/xen/grant-table.c +6 -0
+#ifdef CONFIG_X86
+		barrier();
+#else
+		mb();
+#endif
+		}

Niggle: Indentation of this final } is off.

Sorry, I did not notice that.

Quoted 4 lines

Otherwise all 4 patches:

[...]

Thanks, I will resend those patches after above indentation issue is fixed.

Thanks.

Thanks
Annie

#6 lore raw

Re: [PATCH 3/4] xen/granttable: Grant tables V2 implementation

Quoted 12 lines · ANNIE LI
Quoted 9 lines · Ian Campbell

On Tue, 2011-11-22 at 01:59 +0000, [email protected] wrote:
[...]

drivers/xen/grant-table.c static int gnttab_end_foreign_access_ref_v1(grant_ref_t ref, int readonly) · line 317

Quoted 6 lines
drivers/xen/grant-table.c +6 -0
+#ifdef CONFIG_X86
+		barrier();
+#else
+		mb();
+#endif
+		}

Niggle: Indentation of this final } is off.

Quoted 6 lines

Sorry, I did not notice that.

Quoted 4 lines

Otherwise all 4 patches:

[...]

Thanks, I will resend those patches after above indentation issue is fixed.

That is OK. I can modify them.

Quoted 5 lines

Thanks.

Thanks
Annie