Skip to contentSkip to conversation

convert more Xen page table code to the new API

PATCH In review Hongyan Xia 4 participants 33 messages Apr 17, 2020 – Apr 29, 2020

lore
Share & tools
lore
#0 lore raw

cover convert more Xen page table code to the new API

From: Hongyan Xia <[email protected]>

Basically just rewriting functions using the new API to map and unmap PTEs. Each patch is independent.

Apart from mapping and unmapping page tables, no other functional change intended.

Wei Liu (6):
x86_64/mm: map and unmap page tables in cleanup_frame_table x86_64/mm: map and unmap page tables in subarch_init_memory x86_64/mm: map and unmap page tables in subarch_memory_op x86/smpboot: map and unmap page tables in cleanup_cpu_root_pgt x86/pv: map and unmap page tables in mark_pv_pt_pages_rdonly x86/pv: map and unmap page table in dom0_construct_pv

 xen/arch/x86/pv/dom0_build.c | 38 ++++++++++++++++++++++++------------
 xen/arch/x86/smpboot.c       | 25 ++++++++++++++++--------
 xen/arch/x86/x86_64/mm.c     | 32 +++++++++++++++---------------
 3 files changed, 58 insertions(+), 37 deletions(-)
Signature · 1 line
2.24.1.AMZN
#1 lore raw

PATCH 1/6 x86_64/mm: map and unmap page tables in cleanup_frame_table

From: Wei Liu <[email protected]>

Also fix a weird indentation.

---
 xen/arch/x86/x86_64/mm.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
xen/arch/x86/x86_64/mm.c +7 -7
index e85ef449f3..18210405f4 100644
@@ -737,8 +737,8 @@ static void cleanup_frame_table(struct mem_hotadd_info *info)
 
     while (sva < eva)
     {
-        l3e = l4e_to_l3e(idle_pg_table[l4_table_offset(sva)])[
-          l3_table_offset(sva)];
+        l3e = l3e_from_l4e(idle_pg_table[l4_table_offset(sva)],
+                           l3_table_offset(sva));
         if ( !(l3e_get_flags(l3e) & _PAGE_PRESENT) ||
              (l3e_get_flags(l3e) & _PAGE_PSE) )
         {
@@ -747,7 +747,7 @@ static void cleanup_frame_table(struct mem_hotadd_info *info)
             continue;
         }
 
-        l2e = l3e_to_l2e(l3e)[l2_table_offset(sva)];
+        l2e = l2e_from_l3e(l3e, l2_table_offset(sva));
         ASSERT(l2e_get_flags(l2e) & _PAGE_PRESENT);
 
         if ( (l2e_get_flags(l2e) & (_PAGE_PRESENT | _PAGE_PSE)) ==
@@ -763,10 +763,10 @@ static void cleanup_frame_table(struct mem_hotadd_info *info)
             continue;
         }
 
-        ASSERT(l1e_get_flags(l2e_to_l1e(l2e)[l1_table_offset(sva)]) &
-                _PAGE_PRESENT);
-         sva = (sva & ~((1UL << PAGE_SHIFT) - 1)) +
-                    (1UL << PAGE_SHIFT);
+        ASSERT(l1e_get_flags(l1e_from_l2e(l2e, l1_table_offset(sva))) &
+               _PAGE_PRESENT);
+
+        sva = (sva & ~((1UL << PAGE_SHIFT) - 1)) + (1UL << PAGE_SHIFT);
     }
 
     /* Brute-Force flush all TLB */
Signature · 1 line
2.24.1.AMZN
Reviewed-by on v1 1/6
#2 lore raw

Re: [PATCH 1/6] x86_64/mm: map and unmap page tables in cleanup_frame_table

Hi,

On 17/04/2020 10:52, Hongyan Xia wrote:> diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c> index e85ef449f3..18210405f4 100644> --- a/xen/arch/x86/x86_64/mm.c> +++ b/xen/arch/x86/x86_64/mm.c> @@ -737,8 +737,8 @@ static void cleanup_frame_table(struct mem_hotadd_info *info)> > while (sva < eva)> {> - l3e = l4e_to_l3e(idle_pg_table[l4_table_offset(sva)])[> - l3_table_offset(sva)];> + l3e = l3e_from_l4e(idle_pg_table[l4_table_offset(sva)],> +

      l3_table_offset(sva));

This macro doesn't exist yet in the tree. It would be good to spell out the dependencies in the cover letter so this doesn't get merged before the dependency is merged.

Cheers,

Signature · 1 line
Julien Grall
#3 lore raw

Re: [PATCH 1/6] x86_64/mm: map and unmap page tables in cleanup_frame_table

Quoted 6 lines · Julien Grall

Hi,

On 17/04/2020 10:52, Hongyan Xia wrote:> diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c> index e85ef449f3..18210405f4 100644> --- a/xen/arch/x86/x86_64/mm.c> +++ b/xen/arch/x86/x86_64/mm.c> @@ -737,8 +737,8 @@ static void cleanup_frame_table(struct mem_hotadd_info *info)>   >       while (sva < eva)>       {> -        l3e = l4e_to_l3e(idle_pg_table[l4_table_offset(sva)])[> - l3_table_offset(sva)];> +        l3e = l3e_from_l4e(idle_pg_table[l4_table_offset(sva)],> + l3_table_offset(sva));
This macro doesn't exist yet in the tree. It would be good to spell out the dependencies in the cover letter so this doesn't get merged before the dependency is merged.

Argh, I screwed the reply. Sorry for that. I will resend it.

Signature · 1 line
Julien Grall
Reviewed-by on v1 1/6
#4 lore raw

Re: [PATCH 1/6] x86_64/mm: map and unmap page tables in cleanup_frame_table

In reply to Hongyan Xia

(resending)

Quoted 15 lines · Hongyan Xia

From: Wei Liu <[email protected]>

Also fix a weird indentation.

---
  xen/arch/x86/x86_64/mm.c | 14 +++++++-------
  1 file changed, 7 insertions(+), 7 deletions(-)
xen/arch/x86/x86_64/mm.c +0 -0
index e85ef449f3..18210405f4 100644

xen/arch/x86/x86_64/mm.c static void cleanup_frame_table(struct mem_hotadd_info *info) · line 741

Quoted 7 lines
xen/arch/x86/x86_64/mm.c +2 -2
@@ -737,8 +737,8 @@ static void cleanup_frame_table(struct mem_hotadd_info *info)
 
      while (sva < eva)
      {
-        l3e = l4e_to_l3e(idle_pg_table[l4_table_offset(sva)])[
-          l3_table_offset(sva)];
+        l3e = l3e_from_l4e(idle_pg_table[l4_table_offset(sva)],
+                           l3_table_offset(sva));

This macro doesn't exist yet in the tree. It would be good to spell out the dependencies in the cover letter so this doesn't get merged before the dependency is merged.

Cheers,

Signature · 1 line
Julien Grall
#5 lore raw

Re: [PATCH 1/6] x86_64/mm: map and unmap page tables in cleanup_frame_table

Hi Julien,

Quoted 20 lines · Julien Grall

(resending)

Quoted 17 lines · Hongyan Xia

From: Wei Liu <[email protected]>

Also fix a weird indentation.

---
  xen/arch/x86/x86_64/mm.c | 14 +++++++-------
  1 file changed, 7 insertions(+), 7 deletions(-)
xen/arch/x86/x86_64/mm.c +0 -0
index e85ef449f3..18210405f4 100644
@@ -737,8 +737,8 @@ static void cleanup_frame_table(struct
 mem_hotadd_info *info)
 
Quoted 12 lines

xen/arch/x86/x86_64/mm.c static void cleanup_frame_table(struct mem_hotadd_info *info) · line 741

Quoted 6 lines
xen/arch/x86/x86_64/mm.c +2 -2
      while (sva < eva)
      {
-        l3e = l4e_to_l3e(idle_pg_table[l4_table_offset(sva)])[
-          l3_table_offset(sva)];
+        l3e = l3e_from_l4e(idle_pg_table[l4_table_offset(sva)],
+                           l3_table_offset(sva));

This macro doesn't exist yet in the tree. It would be good to spell out
the dependencies in the cover letter so this doesn't get merged before
the dependency is merged.

I believe the introduction of the new macros has been merged in staging as 6c8afe5aadb33761431b24157d99b25eac15fc7e.

Quoted 1 line

Thanks!

Hongyan

#6 lore raw

Re: [PATCH 1/6] x86_64/mm: map and unmap page tables in cleanup_frame_table

Quoted 36 lines · Hongyan Xia

Hi Julien,

Quoted 20 lines · Julien Grall

(resending)

Quoted 17 lines · Hongyan Xia

From: Wei Liu <[email protected]>

Also fix a weird indentation.

---
   xen/arch/x86/x86_64/mm.c | 14 +++++++-------
   1 file changed, 7 insertions(+), 7 deletions(-)
xen/arch/x86/x86_64/mm.c +0 -0
index e85ef449f3..18210405f4 100644
@@ -737,8 +737,8 @@ static void cleanup_frame_table(struct
 mem_hotadd_info *info)
 
Quoted 12 lines

xen/arch/x86/x86_64/mm.c static void cleanup_frame_table(struct mem_hotadd_info *info) · line 741

Quoted 6 lines
xen/arch/x86/x86_64/mm.c +2 -2
       while (sva < eva)
       {
-        l3e = l4e_to_l3e(idle_pg_table[l4_table_offset(sva)])[
-          l3_table_offset(sva)];
+        l3e = l3e_from_l4e(idle_pg_table[l4_table_offset(sva)],
+                           l3_table_offset(sva));

This macro doesn't exist yet in the tree. It would be good to spell out
the dependencies in the cover letter so this doesn't get merged before
the dependency is merged.

Quoted 2 lines

I believe the introduction of the new macros has been merged in staging as 6c8afe5aadb33761431b24157d99b25eac15fc7e.

Hmmmm you are right. I must have been blind. Sorry for the noise.

Cheers,

Signature · 1 line
Julien Grall
#7 lore raw

Re: [PATCH 1/6] x86_64/mm: map and unmap page tables in cleanup_frame_table

In reply to Hongyan Xia
Quoted 36 lines · Hongyan Xia

From: Wei Liu <[email protected]>

Also fix a weird indentation.

---
  xen/arch/x86/x86_64/mm.c | 14 +++++++-------
  1 file changed, 7 insertions(+), 7 deletions(-)
xen/arch/x86/x86_64/mm.c +3 -3
index e85ef449f3..18210405f4 100644
@@ -737,8 +737,8 @@ static void cleanup_frame_table(struct mem_hotadd_info *info)
 
      while (sva < eva)
      {
-        l3e = l4e_to_l3e(idle_pg_table[l4_table_offset(sva)])[
-          l3_table_offset(sva)];
+        l3e = l3e_from_l4e(idle_pg_table[l4_table_offset(sva)],
+                           l3_table_offset(sva));
          if ( !(l3e_get_flags(l3e) & _PAGE_PRESENT) ||
               (l3e_get_flags(l3e) & _PAGE_PSE) )
          {
@@ -747,7 +747,7 @@ static void cleanup_frame_table(struct mem_hotadd_info *info)
              continue;
          }
 
-        l2e = l3e_to_l2e(l3e)[l2_table_offset(sva)];
+        l2e = l2e_from_l3e(l3e, l2_table_offset(sva));
          ASSERT(l2e_get_flags(l2e) & _PAGE_PRESENT);
 
          if ( (l2e_get_flags(l2e) & (_PAGE_PRESENT | _PAGE_PSE)) ==
@@ -763,10 +763,10 @@ static void cleanup_frame_table(struct mem_hotadd_info *info)
              continue;

xen/arch/x86/x86_64/mm.c static void cleanup_frame_table(struct mem_hotadd_info *info) · line 769

Quoted 10 lines
xen/arch/x86/x86_64/mm.c +4 -4
          }
 
-        ASSERT(l1e_get_flags(l2e_to_l1e(l2e)[l1_table_offset(sva)]) &
-                _PAGE_PRESENT);
-         sva = (sva & ~((1UL << PAGE_SHIFT) - 1)) +
-                    (1UL << PAGE_SHIFT);
+        ASSERT(l1e_get_flags(l1e_from_l2e(l2e, l1_table_offset(sva))) &
+               _PAGE_PRESENT);
+
+        sva = (sva & ~((1UL << PAGE_SHIFT) - 1)) + (1UL << PAGE_SHIFT);

NIT: While you are modifying the indentation. Couldn't we use PAGE_MASK and PAGE_SIZE here?

Cheers,

Signature · 1 line
Julien Grall
#8 lore raw

Re: [PATCH 1/6] x86_64/mm: map and unmap page tables in cleanup_frame_table

Quoted 14 lines · Julien Grall
Quoted 11 lines · Hongyan Xia
(quoted diff) +0 -0
@@ -763,10 +763,10 @@ static void cleanup_frame_table(struct mem_hotadd_info *info)
               continue;
           }

  -        ASSERT(l1e_get_flags(l2e_to_l1e(l2e)[l1_table_offset(sva)]) & -                _PAGE_PRESENT);
-         sva = (sva & ~((1UL << PAGE_SHIFT) - 1)) +
-                    (1UL << PAGE_SHIFT);
+        ASSERT(l1e_get_flags(l1e_from_l2e(l2e, l1_table_offset(sva))) & +               _PAGE_PRESENT);
+
+        sva = (sva & ~((1UL << PAGE_SHIFT) - 1)) + (1UL << PAGE_SHIFT);

NIT: While you are modifying the indentation. Couldn't we use PAGE_MASK and PAGE_SIZE here?

Oh, yes, this would be nice.

Jan

Reviewed-by
#9 lore raw

Re: [PATCH 1/6] x86_64/mm: map and unmap page tables in cleanup_frame_table

In reply to Julien Grall
Quoted 14 lines · Julien Grall
Quoted 11 lines · Hongyan Xia
(quoted diff) +0 -0
@@ -763,10 +763,10 @@ static void cleanup_frame_table(struct mem_hotadd_info *info)
               continue;
           }

  -        ASSERT(l1e_get_flags(l2e_to_l1e(l2e)[l1_table_offset(sva)]) & -                _PAGE_PRESENT);
-         sva = (sva & ~((1UL << PAGE_SHIFT) - 1)) +
-                    (1UL << PAGE_SHIFT);
+        ASSERT(l1e_get_flags(l1e_from_l2e(l2e, l1_table_offset(sva))) & +               _PAGE_PRESENT);
+
+        sva = (sva & ~((1UL << PAGE_SHIFT) - 1)) + (1UL << PAGE_SHIFT);

NIT: While you are modifying the indentation. Couldn't we use PAGE_MASK and PAGE_SIZE here?

And with this (which I think can be done while committing)

Jan

#10 lore raw

PATCH 2/6 x86_64/mm: map and unmap page tables in subarch_init_memory

In reply to Hongyan Xia

From: Wei Liu <[email protected]>

---
 xen/arch/x86/x86_64/mm.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
xen/arch/x86/x86_64/mm.c +6 -6
index 18210405f4..5714e5ba62 100644
@@ -852,14 +852,14 @@ void __init subarch_init_memory(void)
           v += n << PAGE_SHIFT )
     {
         n = L2_PAGETABLE_ENTRIES * L1_PAGETABLE_ENTRIES;
-        l3e = l4e_to_l3e(idle_pg_table[l4_table_offset(v)])[
-            l3_table_offset(v)];
+        l3e = l3e_from_l4e(idle_pg_table[l4_table_offset(v)],
+                           l3_table_offset(v));
         if ( !(l3e_get_flags(l3e) & _PAGE_PRESENT) )
             continue;
         if ( !(l3e_get_flags(l3e) & _PAGE_PSE) )
         {
             n = L1_PAGETABLE_ENTRIES;
-            l2e = l3e_to_l2e(l3e)[l2_table_offset(v)];
+            l2e = l2e_from_l3e(l3e, l2_table_offset(v));
             if ( !(l2e_get_flags(l2e) & _PAGE_PRESENT) )
                 continue;
             m2p_start_mfn = l2e_get_pfn(l2e);
@@ -878,11 +878,11 @@ void __init subarch_init_memory(void)
           v != RDWR_COMPAT_MPT_VIRT_END;
           v += 1 << L2_PAGETABLE_SHIFT )
     {
-        l3e = l4e_to_l3e(idle_pg_table[l4_table_offset(v)])[
-            l3_table_offset(v)];
+        l3e = l3e_from_l4e(idle_pg_table[l4_table_offset(v)],
+                           l3_table_offset(v));
         if ( !(l3e_get_flags(l3e) & _PAGE_PRESENT) )
             continue;
-        l2e = l3e_to_l2e(l3e)[l2_table_offset(v)];
+        l2e = l2e_from_l3e(l3e, l2_table_offset(v));
         if ( !(l2e_get_flags(l2e) & _PAGE_PRESENT) )
             continue;
         m2p_start_mfn = l2e_get_pfn(l2e);
Signature · 1 line
2.24.1.AMZN
#13 lore raw

PATCH 3/6 x86_64/mm: map and unmap page tables in subarch_memory_op

In reply to Hongyan Xia

From: Wei Liu <[email protected]>

---
 xen/arch/x86/x86_64/mm.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
xen/arch/x86/x86_64/mm.c +3 -3
index 5714e5ba62..6d52183559 100644
@@ -932,13 +932,13 @@ long subarch_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
               (v < (unsigned long)(machine_to_phys_mapping + max_page));
               i++, v += 1UL << L2_PAGETABLE_SHIFT )
         {
-            l3e = l4e_to_l3e(idle_pg_table[l4_table_offset(v)])[
-                l3_table_offset(v)];
+            l3e = l3e_from_l4e(idle_pg_table[l4_table_offset(v)],
+                               l3_table_offset(v));
             if ( !(l3e_get_flags(l3e) & _PAGE_PRESENT) )
                 mfn = last_mfn;
             else if ( !(l3e_get_flags(l3e) & _PAGE_PSE) )
             {
-                l2e = l3e_to_l2e(l3e)[l2_table_offset(v)];
+                l2e = l2e_from_l3e(l3e, l2_table_offset(v));
                 if ( l2e_get_flags(l2e) & _PAGE_PRESENT )
                     mfn = l2e_get_pfn(l2e);
                 else
Signature · 1 line
2.24.1.AMZN
#16 lore raw

PATCH 4/6 x86/smpboot: map and unmap page tables in cleanup_cpu_root_pgt

In reply to Hongyan Xia

From: Wei Liu <[email protected]>

---
 xen/arch/x86/smpboot.c | 25 +++++++++++++++++--------
 1 file changed, 17 insertions(+), 8 deletions(-)
xen/arch/x86/smpboot.c +17 -8
index 09264b02d1..275ce7661d 100644
@@ -858,23 +858,27 @@ static void cleanup_cpu_root_pgt(unsigned int cpu)
           r < root_table_offset(HYPERVISOR_VIRT_END); ++r )
     {
         l3_pgentry_t *l3t;
+        mfn_t l3mfn;
         unsigned int i3;
 
         if ( !(root_get_flags(rpt[r]) & _PAGE_PRESENT) )
             continue;
 
-        l3t = l4e_to_l3e(rpt[r]);
+        l3mfn = l4e_get_mfn(rpt[r]);
+        l3t = map_domain_page(l3mfn);
 
         for ( i3 = 0; i3 < L3_PAGETABLE_ENTRIES; ++i3 )
         {
             l2_pgentry_t *l2t;
+            mfn_t l2mfn;
             unsigned int i2;
 
             if ( !(l3e_get_flags(l3t[i3]) & _PAGE_PRESENT) )
                 continue;
 
             ASSERT(!(l3e_get_flags(l3t[i3]) & _PAGE_PSE));
-            l2t = l3e_to_l2e(l3t[i3]);
+            l2mfn = l3e_get_mfn(l3t[i3]);
+            l2t = map_domain_page(l2mfn);
 
             for ( i2 = 0; i2 < L2_PAGETABLE_ENTRIES; ++i2 )
             {
@@ -882,13 +886,15 @@ static void cleanup_cpu_root_pgt(unsigned int cpu)
                     continue;
 
                 ASSERT(!(l2e_get_flags(l2t[i2]) & _PAGE_PSE));
-                free_xen_pagetable(l2e_to_l1e(l2t[i2]));
+                free_xen_pagetable_new(l2e_get_mfn(l2t[i2]));
             }
 
-            free_xen_pagetable(l2t);
+            unmap_domain_page(l2t);
+            free_xen_pagetable_new(l2mfn);
         }
 
-        free_xen_pagetable(l3t);
+        unmap_domain_page(l3t);
+        free_xen_pagetable_new(l3mfn);
     }
 
     free_xen_pagetable(rpt);
@@ -896,11 +902,14 @@ static void cleanup_cpu_root_pgt(unsigned int cpu)
     /* Also zap the stub mapping for this CPU. */
     if ( stub_linear )
     {
-        l3_pgentry_t *l3t = l4e_to_l3e(common_pgt);
-        l2_pgentry_t *l2t = l3e_to_l2e(l3t[l3_table_offset(stub_linear)]);
-        l1_pgentry_t *l1t = l2e_to_l1e(l2t[l2_table_offset(stub_linear)]);
+        l3_pgentry_t l3e = l3e_from_l4e(common_pgt,
+                                        l3_table_offset(stub_linear));
+        l2_pgentry_t l2e = l2e_from_l3e(l3e, l2_table_offset(stub_linear));
+        l1_pgentry_t *l1t = map_l1t_from_l2e(l2e);
 
         l1t[l1_table_offset(stub_linear)] = l1e_empty();
+
+        unmap_domain_page(l1t);
     }
 }
 
Signature · 1 line
2.24.1.AMZN
Reviewed-by on v1 4/6
#17 lore raw

Re: [PATCH 4/6] x86/smpboot: map and unmap page tables in cleanup_cpu_root_pgt

Hi,

Quoted 4 lines · Hongyan Xia

From: Wei Liu <[email protected]>

Quoted 66 lines
---
  xen/arch/x86/smpboot.c | 25 +++++++++++++++++--------
  1 file changed, 17 insertions(+), 8 deletions(-)
xen/arch/x86/smpboot.c +12 -8
index 09264b02d1..275ce7661d 100644
@@ -858,23 +858,27 @@ static void cleanup_cpu_root_pgt(unsigned int cpu)
            r < root_table_offset(HYPERVISOR_VIRT_END); ++r )
      {
          l3_pgentry_t *l3t;
+        mfn_t l3mfn;
          unsigned int i3;
 
          if ( !(root_get_flags(rpt[r]) & _PAGE_PRESENT) )
              continue;
 
-        l3t = l4e_to_l3e(rpt[r]);
+        l3mfn = l4e_get_mfn(rpt[r]);
+        l3t = map_domain_page(l3mfn);
 
          for ( i3 = 0; i3 < L3_PAGETABLE_ENTRIES; ++i3 )
          {
              l2_pgentry_t *l2t;
+            mfn_t l2mfn;
              unsigned int i2;
 
              if ( !(l3e_get_flags(l3t[i3]) & _PAGE_PRESENT) )
                  continue;
 
              ASSERT(!(l3e_get_flags(l3t[i3]) & _PAGE_PSE));
-            l2t = l3e_to_l2e(l3t[i3]);
+            l2mfn = l3e_get_mfn(l3t[i3]);
+            l2t = map_domain_page(l2mfn);
 
              for ( i2 = 0; i2 < L2_PAGETABLE_ENTRIES; ++i2 )
              {
@@ -882,13 +886,15 @@ static void cleanup_cpu_root_pgt(unsigned int cpu)
                      continue;
 
                  ASSERT(!(l2e_get_flags(l2t[i2]) & _PAGE_PSE));
-                free_xen_pagetable(l2e_to_l1e(l2t[i2]));
+                free_xen_pagetable_new(l2e_get_mfn(l2t[i2]));
              }
 
-            free_xen_pagetable(l2t);
+            unmap_domain_page(l2t);
+            free_xen_pagetable_new(l2mfn);
          }
 
-        free_xen_pagetable(l3t);
+        unmap_domain_page(l3t);
+        free_xen_pagetable_new(l3mfn);
      }
 
      free_xen_pagetable(rpt);
@@ -896,11 +902,14 @@ static void cleanup_cpu_root_pgt(unsigned int cpu)
      /* Also zap the stub mapping for this CPU. */
      if ( stub_linear )
      {
-        l3_pgentry_t *l3t = l4e_to_l3e(common_pgt);
-        l2_pgentry_t *l2t = l3e_to_l2e(l3t[l3_table_offset(stub_linear)]);
-        l1_pgentry_t *l1t = l2e_to_l1e(l2t[l2_table_offset(stub_linear)]);
+        l3_pgentry_t l3e = l3e_from_l4e(common_pgt,

xen/arch/x86/smpboot.c static void cleanup_cpu_root_pgt(unsigned int cpu) · line 915

Quoted 10 lines
xen/arch/x86/smpboot.c +5 -0
+                                        l3_table_offset(stub_linear));
+        l2_pgentry_t l2e = l2e_from_l3e(l3e, l2_table_offset(stub_linear));
+        l1_pgentry_t *l1t = map_l1t_from_l2e(l2e);
 
          l1t[l1_table_offset(stub_linear)] = l1e_empty();
+
+        unmap_domain_page(l1t);
      }
  }
 

Cheers,

Signature · 1 line
Julien Grall
#19 lore raw

PATCH 5/6 x86/pv: map and unmap page tables in mark_pv_pt_pages_rdonly

In reply to Hongyan Xia

From: Wei Liu <[email protected]>

---
 xen/arch/x86/pv/dom0_build.c | 32 ++++++++++++++++++++------------
 1 file changed, 20 insertions(+), 12 deletions(-)
xen/arch/x86/pv/dom0_build.c +20 -12
index 5678da782d..28a939b68a 100644
@@ -50,17 +50,17 @@ static __init void mark_pv_pt_pages_rdonly(struct domain *d,
     unsigned long count;
     struct page_info *page;
     l4_pgentry_t *pl4e;
-    l3_pgentry_t *pl3e;
-    l2_pgentry_t *pl2e;
-    l1_pgentry_t *pl1e;
+    l3_pgentry_t *pl3e, *l3t;
+    l2_pgentry_t *pl2e, *l2t;
+    l1_pgentry_t *pl1e, *l1t;
 
     pl4e = l4start + l4_table_offset(vpt_start);
-    pl3e = l4e_to_l3e(*pl4e);
-    pl3e += l3_table_offset(vpt_start);
-    pl2e = l3e_to_l2e(*pl3e);
-    pl2e += l2_table_offset(vpt_start);
-    pl1e = l2e_to_l1e(*pl2e);
-    pl1e += l1_table_offset(vpt_start);
+    l3t = map_l3t_from_l4e(*pl4e);
+    pl3e = l3t + l3_table_offset(vpt_start);
+    l2t = map_l2t_from_l3e(*pl3e);
+    pl2e = l2t + l2_table_offset(vpt_start);
+    l1t = map_l1t_from_l2e(*pl2e);
+    pl1e = l1t + l1_table_offset(vpt_start);
     for ( count = 0; count < nr_pt_pages; count++ )
     {
         l1e_remove_flags(*pl1e, _PAGE_RW);
@@ -85,12 +85,20 @@ static __init void mark_pv_pt_pages_rdonly(struct domain *d,
             if ( !((unsigned long)++pl2e & (PAGE_SIZE - 1)) )
             {
                 if ( !((unsigned long)++pl3e & (PAGE_SIZE - 1)) )
-                    pl3e = l4e_to_l3e(*++pl4e);
-                pl2e = l3e_to_l2e(*pl3e);
+                {
+                    unmap_domain_page(l3t);
+                    pl3e = l3t = map_l3t_from_l4e(*++pl4e);
+                }
+                unmap_domain_page(l2t);
+                pl2e = l2t = map_l2t_from_l3e(*pl3e);
             }
-            pl1e = l2e_to_l1e(*pl2e);
+            unmap_domain_page(l1t);
+            pl1e = l1t = map_l1t_from_l2e(*pl2e);
         }
     }
+    unmap_domain_page(l1t);
+    unmap_domain_page(l2t);
+    unmap_domain_page(l3t);
 }
 
 static __init void setup_pv_physmap(struct domain *d, unsigned long pgtbl_pfn,
Signature · 1 line
2.24.1.AMZN
#21 lore raw

Re: [PATCH 5/6] x86/pv: map and unmap page tables in mark_pv_pt_pages_rdonly

In reply to Hongyan Xia

xen/arch/x86/pv/dom0_build.c static __init void mark_pv_pt_pages_rdonly(struct domain *d, · line 55

Quoted 12 lines · Hongyan Xia
xen/arch/x86/pv/dom0_build.c +3 -3
@@ -50,17 +50,17 @@ static __init void mark_pv_pt_pages_rdonly(struct domain *d,
     unsigned long count;
     struct page_info *page;
     l4_pgentry_t *pl4e;
-    l3_pgentry_t *pl3e;
-    l2_pgentry_t *pl2e;
-    l1_pgentry_t *pl1e;
+    l3_pgentry_t *pl3e, *l3t;
+    l2_pgentry_t *pl2e, *l2t;
+    l1_pgentry_t *pl1e, *l1t;

I don't quite see why the new local variables get introduced:
unmap_domain_page(), iirc, is quite fine with a non-page- aligned argument.

Jan

#22 lore raw

Re: [PATCH 5/6] x86/pv: map and unmap page tables in mark_pv_pt_pages_rdonly

Quoted 17 lines · Jan Beulich

xen/arch/x86/pv/dom0_build.c static __init void mark_pv_pt_pages_rdonly(struct domain *d, · line 55

Quoted 12 lines · Hongyan Xia
xen/arch/x86/pv/dom0_build.c +3 -3
@@ -50,17 +50,17 @@ static __init void mark_pv_pt_pages_rdonly(struct domain *d,
     unsigned long count;
     struct page_info *page;
     l4_pgentry_t *pl4e;
-    l3_pgentry_t *pl3e;
-    l2_pgentry_t *pl2e;
-    l1_pgentry_t *pl1e;
+    l3_pgentry_t *pl3e, *l3t;
+    l2_pgentry_t *pl2e, *l2t;
+    l1_pgentry_t *pl1e, *l1t;

I don't quite see why the new local variables get introduced:
unmap_domain_page(), iirc, is quite fine with a non-page- aligned argument.

(Assuming this is actually written by me)

I wanted to make things abundantly clear: plXe points to an entry while lXt points to the start of a page table.

In a long function the distinction could be helpful; in a short function (like this one?) not so much.

Wei.

Quoted 2 lines

Jan

#23 lore raw

Re: [PATCH 5/6] x86/pv: map and unmap page tables in mark_pv_pt_pages_rdonly

In reply to Jan Beulich
Quoted 10 lines · Jan Beulich
Quoted 9 lines · Hongyan Xia
xen/arch/x86/pv/dom0_build.c +0 -2
@@ -50,17 +50,17 @@ static __init void
 mark_pv_pt_pages_rdonly(struct domain *d,
     unsigned long count;
     struct page_info *page;
     l4_pgentry_t *pl4e;
-    l3_pgentry_t *pl3e;
-    l2_pgentry_t *pl2e;
Quoted 8 lines

xen/arch/x86/pv/dom0_build.c static __init void mark_pv_pt_pages_rdonly(struct domain *d, · line 55

Quoted 4 lines
xen/arch/x86/pv/dom0_build.c +3 -1
-    l1_pgentry_t *pl1e;
+    l3_pgentry_t *pl3e, *l3t;
+    l2_pgentry_t *pl2e, *l2t;
+    l1_pgentry_t *pl1e, *l1t;

I don't quite see why the new local variables get introduced:
unmap_domain_page(), iirc, is quite fine with a non-page- aligned argument.

You are right, although in this function, where plXe points to may not be the page we want to unmap. When plXe becomes aligned and points to a new page, we actually want to unmap the page before it increments to an aligned value.

Hongyan

#24 lore raw

Re: [PATCH 5/6] x86/pv: map and unmap page tables in mark_pv_pt_pages_rdonly

Quoted 20 lines · Hongyan Xia
Quoted 10 lines · Jan Beulich
Quoted 9 lines · Hongyan Xia
xen/arch/x86/pv/dom0_build.c +0 -2
@@ -50,17 +50,17 @@ static __init void
 mark_pv_pt_pages_rdonly(struct domain *d,
     unsigned long count;
     struct page_info *page;
     l4_pgentry_t *pl4e;
-    l3_pgentry_t *pl3e;
-    l2_pgentry_t *pl2e;
Quoted 8 lines

xen/arch/x86/pv/dom0_build.c static __init void mark_pv_pt_pages_rdonly(struct domain *d, · line 55

Quoted 4 lines
xen/arch/x86/pv/dom0_build.c +3 -1
-    l1_pgentry_t *pl1e;
+    l3_pgentry_t *pl3e, *l3t;
+    l2_pgentry_t *pl2e, *l2t;
+    l1_pgentry_t *pl1e, *l1t;

I don't quite see why the new local variables get introduced:
unmap_domain_page(), iirc, is quite fine with a non-page- aligned argument.

Quoted 7 lines

You are right, although in this function, where plXe points to may not
be the page we want to unmap. When plXe becomes aligned and points to a
new page, we actually want to unmap the page before it increments to an
aligned value.

Hmm, we can just unmap(plXe - 1) if my logic is correct, and save 3 local variables.

Hongyan

#25 lore raw

Re: [PATCH 5/6] x86/pv: map and unmap page tables in mark_pv_pt_pages_rdonly

Quoted 31 lines · Hongyan Xia
Quoted 20 lines · Hongyan Xia
Quoted 10 lines · Jan Beulich
Quoted 9 lines · Hongyan Xia
xen/arch/x86/pv/dom0_build.c +0 -2
@@ -50,17 +50,17 @@ static __init void
 mark_pv_pt_pages_rdonly(struct domain *d,
     unsigned long count;
     struct page_info *page;
     l4_pgentry_t *pl4e;
-    l3_pgentry_t *pl3e;
-    l2_pgentry_t *pl2e;
Quoted 8 lines

xen/arch/x86/pv/dom0_build.c static __init void mark_pv_pt_pages_rdonly(struct domain *d, · line 55

Quoted 4 lines
xen/arch/x86/pv/dom0_build.c +3 -1
-    l1_pgentry_t *pl1e;
+    l3_pgentry_t *pl3e, *l3t;
+    l2_pgentry_t *pl2e, *l2t;
+    l1_pgentry_t *pl1e, *l1t;

I don't quite see why the new local variables get introduced:
unmap_domain_page(), iirc, is quite fine with a non-page- aligned argument.

Quoted 9 lines

You are right, although in this function, where plXe points to may not
be the page we want to unmap. When plXe becomes aligned and points to
a
new page, we actually want to unmap the page before it increments to
an
aligned value.

Quoted 2 lines

Hmm, we can just unmap(plXe - 1) if my logic is correct, and save 3 local variables.

Sorry for monologuing, but I still prefer separating plXe and lXt because it makes it clear what we are unmapping. Unmapping plXe - 1 is a bit hackish.

But if people do not have a problem with plXe - 1, I will post a new revision for that.

Hongyan

#26 lore raw

Re: [PATCH 5/6] x86/pv: map and unmap page tables in mark_pv_pt_pages_rdonly

Quoted 2 lines · Hongyan Xia

But if people do not have a problem with plXe - 1, I will post a new revision for that.

I'd prefer that; I could live with the current version, but I'm not in favor of it.

Jan

author Reviewed-by Julien Grall
#27 lore raw

Re: [PATCH 5/6] x86/pv: map and unmap page tables in mark_pv_pt_pages_rdonly

(Looks like other patches in this series have been merged. Replying to this one only.)

From: Wei Liu <[email protected]>
Date: Tue, 5 Feb 2019 16:32:54 +0000
Subject: [PATCH] x86/pv: map and unmap page tables in mark_pv_pt_pages_rdonly

Also, clean up the initialisation of plXe.

---
 xen/arch/x86/pv/dom0_build.c | 32 +++++++++++++++++---------------
 1 file changed, 17 insertions(+), 15 deletions(-)
(quoted diff) +0 -0

b/xen/arch/x86/pv/dom0_build.c
index abfbe5f436..3522eb0114 100644

xen/arch/x86/pv/dom0_build.c +0 -0
@@ -49,18 +49,11 @@ static __init void mark_pv_pt_pages_rdonly(struct

domain *d,

 {
     unsigned long count;
     struct page_info *page;
-    l4_pgentry_t *pl4e;
-    l3_pgentry_t *pl3e;
-    l2_pgentry_t *pl2e;
-    l1_pgentry_t *pl1e;
-
-    pl4e = l4start + l4_table_offset(vpt_start);
-    pl3e = l4e_to_l3e(*pl4e);
-    pl3e += l3_table_offset(vpt_start);
-    pl2e = l3e_to_l2e(*pl3e);
-    pl2e += l2_table_offset(vpt_start);
-    pl1e = l2e_to_l1e(*pl2e);
-    pl1e += l1_table_offset(vpt_start);
+    l4_pgentry_t *pl4e = l4start + l4_table_offset(vpt_start);
+    l3_pgentry_t *pl3e = map_l3t_from_l4e(*pl4e) +

l3_table_offset(vpt_start);
+ l2_pgentry_t *pl2e = map_l2t_from_l3e(*pl3e) +
l2_table_offset(vpt_start);
+ l1_pgentry_t *pl1e = map_l1t_from_l2e(*pl2e) +
l1_table_offset(vpt_start);
+

     for ( count = 0; count < nr_pt_pages; count++ )
     {
         l1e_remove_flags(*pl1e, _PAGE_RW);
@@ -85,12 +78,21 @@ static __init void mark_pv_pt_pages_rdonly(struct
domain *d,
             if ( !((unsigned long)++pl2e & (PAGE_SIZE - 1)) )
             {
                 if ( !((unsigned long)++pl3e & (PAGE_SIZE - 1)) )
-                    pl3e = l4e_to_l3e(*++pl4e);
-                pl2e = l3e_to_l2e(*pl3e);
+                {
+                    /* Need to unmap the page before the increment. */
+                    unmap_domain_page(pl3e - 1);
+                    pl3e = map_l3t_from_l4e(*++pl4e);
+                }
+                unmap_domain_page(pl2e - 1);
+                pl2e = map_l2t_from_l3e(*pl3e);
             }
-            pl1e = l2e_to_l1e(*pl2e);
+            unmap_domain_page(pl1e - 1);
+            pl1e = map_l1t_from_l2e(*pl2e);
         }
     }
+    unmap_domain_page(pl1e);
+    unmap_domain_page(pl2e);
+    unmap_domain_page(pl3e);

}

static __init void setup_pv_physmap(struct domain *d, unsigned long
pgtbl_pfn,

Signature · 1 line
2.24.1.AMZN
#28 lore raw

Re: [PATCH 5/6] x86/pv: map and unmap page tables in mark_pv_pt_pages_rdonly

Quoted 2 lines · Hongyan Xia

(Looks like other patches in this series have been merged. Replying to this one only.)

Please send as a proper patch, this one came through ...

Quoted 42 lines

From: Wei Liu <[email protected]>
Date: Tue, 5 Feb 2019 16:32:54 +0000
Subject: [PATCH] x86/pv: map and unmap page tables in mark_pv_pt_pages_rdonly

Also, clean up the initialisation of plXe.

---
 xen/arch/x86/pv/dom0_build.c | 32 +++++++++++++++++---------------
 1 file changed, 17 insertions(+), 15 deletions(-)
xen/arch/x86/pv/dom0_build.c +4 -12
index abfbe5f436..3522eb0114 100644
@@ -49,18 +49,11 @@ static __init void mark_pv_pt_pages_rdonly(struct
 domain *d,
 {
     unsigned long count;
     struct page_info *page;
-    l4_pgentry_t *pl4e;
-    l3_pgentry_t *pl3e;
-    l2_pgentry_t *pl2e;
-    l1_pgentry_t *pl1e;
-
-    pl4e = l4start + l4_table_offset(vpt_start);
-    pl3e = l4e_to_l3e(*pl4e);
-    pl3e += l3_table_offset(vpt_start);
-    pl2e = l3e_to_l2e(*pl3e);
-    pl2e += l2_table_offset(vpt_start);
-    pl1e = l2e_to_l1e(*pl2e);
-    pl1e += l1_table_offset(vpt_start);
+    l4_pgentry_t *pl4e = l4start + l4_table_offset(vpt_start);
+    l3_pgentry_t *pl3e = map_l3t_from_l4e(*pl4e) +
 l3_table_offset(vpt_start);
+    l2_pgentry_t *pl2e = map_l2t_from_l3e(*pl3e) +
 l2_table_offset(vpt_start);
+    l1_pgentry_t *pl1e = map_l1t_from_l2e(*pl2e) +
Quoted 1 line

l1_table_offset(vpt_start);

... mangled anyway. I also think with the change made you need to drop the R-b.

Jan

#29 lore raw

PATCH 6/6 x86/pv: map and unmap page table in dom0_construct_pv

In reply to Hongyan Xia

From: Wei Liu <[email protected]>

---
 xen/arch/x86/pv/dom0_build.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
xen/arch/x86/pv/dom0_build.c +5 -1
index 28a939b68a..a03f0501ab 100644
@@ -677,6 +677,8 @@ int __init dom0_construct_pv(struct domain *d,
 
     if ( is_pv_32bit_domain(d) )
     {
+        l2_pgentry_t *l2t;
+
         /* Ensure the first four L3 entries are all populated. */
         for ( i = 0, l3tab = l3start; i < 4; ++i, ++l3tab )
         {
@@ -691,7 +693,9 @@ int __init dom0_construct_pv(struct domain *d,
                 l3e_get_page(*l3tab)->u.inuse.type_info |= PGT_pae_xen_l2;
         }
 
-        init_xen_pae_l2_slots(l3e_to_l2e(l3start[3]), d);
+        l2t = map_l2t_from_l3e(l3start[3]);
+        init_xen_pae_l2_slots(l2t, d);
+        unmap_domain_page(l2t);
     }
 
     /* Pages that are part of page tables must be read only. */
Signature · 1 line
2.24.1.AMZN
#32 lore raw
In reply to Hongyan Xia

A gentle ping.

Quoted 23 lines · Hongyan Xia

From: Hongyan Xia <[email protected]>

Basically just rewriting functions using the new API to map and unmap PTEs. Each patch is independent.

Apart from mapping and unmapping page tables, no other functional change
intended.

Wei Liu (6):
x86_64/mm: map and unmap page tables in cleanup_frame_table x86_64/mm: map and unmap page tables in subarch_init_memory x86_64/mm: map and unmap page tables in subarch_memory_op x86/smpboot: map and unmap page tables in cleanup_cpu_root_pgt x86/pv: map and unmap page tables in mark_pv_pt_pages_rdonly x86/pv: map and unmap page table in dom0_construct_pv

 xen/arch/x86/pv/dom0_build.c | 38 ++++++++++++++++++++++++--------
----
 xen/arch/x86/smpboot.c       | 25 ++++++++++++++++--------
 xen/arch/x86/x86_64/mm.c     | 32 +++++++++++++++---------------
 3 files changed, 58 insertions(+), 37 deletions(-)