Skip to contentSkip to conversation

prospective 13/12 s390 pgtable_list patch

PATCH Hugh Dickins 3 participants 6 messages Jun 23, 2023 – Jun 28, 2023

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Hi Gerald,

It's that moment you've been dreading: I'm hoping that you can, please, take a look at the patch below, and try building and running with it, on top of the v2 series of 12 I sent out on Tuesday.

If this seems okay to you, I'll send it properly as 13/12 of that series, to the full Cc list; but of course you may find I've missed typos and worse
- please don't waste your time on it if it's rubbish, but so far as I can tell, it is complete and ready now.

Thanks!
Hugh

s390: SLAB_TYPESAFE_BY_RCU mm->context.pgtable_list

Move the s390 pgtable_list (with dedicated spinlock) out of mm->context into a separate kmem_cache object specified with SLAB_TYPESAFE_BY_RCU:
which then allows __tlb_remove_table() to add a page back to the list, safely, when the other half is still in use.

Update the commentary on pgtable_list above page_table_alloc().

Stop using pt_mm in struct page: this needs a pt_list in that union.

---
 arch/s390/include/asm/mmu.h         |  10 +-
 arch/s390/include/asm/mmu_context.h |   8 +-
 arch/s390/mm/pgalloc.c              | 151 ++++++++++++++++++----------
 include/linux/mm_types.h            |   4 +-
 4 files changed, 117 insertions(+), 56 deletions(-)
arch/s390/include/asm/mmu.h +8 -2
index 829d68e2c685..069cdd612d86 100644
@@ -6,12 +6,18 @@
 #include <linux/errno.h>
 #include <asm/asm-extable.h>
 
+struct pgtable_list {
+	struct list_head list;
+	spinlock_t lock;
+};
+extern struct pgtable_list init_pgtable_list;
+
 typedef struct {
 	spinlock_t lock;
 	cpumask_t cpu_attach_mask;
 	atomic_t flush_count;
 	unsigned int flush_mm;
-	struct list_head pgtable_list;
+	struct pgtable_list *pgtable_list;
 	struct list_head gmap_list;
 	unsigned long gmap_asce;
 	unsigned long asce;
@@ -39,7 +45,7 @@ typedef struct {
 
 #define INIT_MM_CONTEXT(name)						   \
 	.context.lock =	__SPIN_LOCK_UNLOCKED(name.context.lock),	   \
-	.context.pgtable_list = LIST_HEAD_INIT(name.context.pgtable_list), \
+	.context.pgtable_list = &init_pgtable_list,			   \
 	.context.gmap_list = LIST_HEAD_INIT(name.context.gmap_list),
 
 #endif
arch/s390/include/asm/mmu_context.h +7 -1
index 2a38af5a00c2..725f82c55036 100644
@@ -15,14 +15,20 @@
 #include <asm/ctl_reg.h>
 #include <asm-generic/mm_hooks.h>
 
+bool alloc_pgtable_list(struct mm_struct *mm);
+void destroy_context(struct mm_struct *mm);
+#define destroy_context destroy_context
+
 #define init_new_context init_new_context
 static inline int init_new_context(struct task_struct *tsk,
 				   struct mm_struct *mm)
 {
 	unsigned long asce_type, init_entry;
 
+	if (!alloc_pgtable_list(mm))
+		return -ENOMEM;
+
 	spin_lock_init(&mm->context.lock);
-	INIT_LIST_HEAD(&mm->context.pgtable_list);
 	INIT_LIST_HEAD(&mm->context.gmap_list);
 	cpumask_clear(&mm->context.cpu_attach_mask);
 	atomic_set(&mm->context.flush_count, 0);
arch/s390/mm/pgalloc.c +99 -52

287 lines not on this page · raw message

include/linux/mm_types.h +3 -1
index 09335fa28c41..a5b131abd82f 100644
@@ -31,6 +31,7 @@
 
 struct address_space;
 struct mem_cgroup;
+struct pgtable_list;
 
 /*
  * Each physical page in the system has a struct page associated with
@@ -150,7 +151,8 @@ struct page {
 			 */
 			unsigned long _pt_pad_2;	/* mapping */
 			union {
-				struct mm_struct *pt_mm; /* x86 pgd, s390 */
+				struct mm_struct *pt_mm; /* x86 pgds only */
+				struct pgtable_list *pt_list; /* s390 */
 				atomic_t pt_frag_refcount; /* powerpc */
 			};
 #if ALLOC_SPLIT_PTLOCKS
Signature · 1 line
2.35.3
#1 lore raw
Quoted 6 lines · Hugh Dickins

Hi Gerald,

It's that moment you've been dreading: I'm hoping that you can, please, take a look at the patch below, and try building and running with it, on top of the v2 series of 12 I sent out on Tuesday.

Quoted 7 lines

If this seems okay to you, I'll send it properly as 13/12 of that series, to the full Cc list; but of course you may find I've missed typos and worse
- please don't waste your time on it if it's rubbish, but so far as I can tell, it is complete and ready now.

Thanks!
Hugh

Hi Hugh,

Gerald is off until Monday and I think is not able to answer right now.

We had discussions with regard to how to better approach your series and did not come to a conclusion unfortunatelly.

Gerald had several concerns - one of them is global mm_pgtable_list_lock, wich is luckily avoided with this follow-up patch. But there were others, which I am not able to articulate in detail.

While you are doing an outstanding job in trying to adjust our fragmented page tables reuse scheme, one of the ideas emerged was to actually give it
up: partially or may be even fully. That is - not to reuse page tables returned via pte_free_defer() or not to reuse them at all. To assess this possible new approaches some statistics is needed and am working on a prototype that would allow collecting it.

Note, our existing code is extremly complicated already and we had hard time fixing (at least one) ugly race related to that. With the changes you suggest that complexity (to me personally) multiplies. But that well could be the other way around and I am just not smart enough to grasp it. At least the claim that page_table_free() no longer needs the two-step release indicates that.

I am sorry that it is probably not the status you would like to hear, but I still wonder what is your opinion on that do-not-reuse-fragments approach? Would it simplify pte_free_defer() or had no effect?

Anyway, that is just another option and I will try your patch.

Thanks!

#2 lore raw
Quoted 12 lines · Alexander Gordeev
Quoted 11 lines · Hugh Dickins

Hi Gerald,

It's that moment you've been dreading: I'm hoping that you can, please, take a look at the patch below, and try building and running with it, on top of the v2 series of 12 I sent out on Tuesday.

If this seems okay to you, I'll send it properly as 13/12 of that series, to the full Cc list; but of course you may find I've missed typos and worse
- please don't waste your time on it if it's rubbish, but so far as I can tell, it is complete and ready now.

Quoted 6 lines
Quoted 2 lines

Thanks!
Hugh

Hi Hugh,

Gerald is off until Monday and I think is not able to answer right now.

Thanks a lot for stepping in, Alexander.

Quoted 15 lines

We had discussions with regard to how to better approach your series and did not come to a conclusion unfortunatelly.

Gerald had several concerns - one of them is global mm_pgtable_list_lock, wich is luckily avoided with this follow-up patch. But there were others, which I am not able to articulate in detail.

While you are doing an outstanding job in trying to adjust our fragmented page tables reuse scheme, one of the ideas emerged was to actually give it
up: partially or may be even fully. That is - not to reuse page tables returned via pte_free_defer() or not to reuse them at all. To assess this possible new approaches some statistics is needed and am working on a prototype that would allow collecting it.

Quoted 6 lines

Note, our existing code is extremly complicated already and we had hard time fixing (at least one) ugly race related to that. With the changes you suggest that complexity (to me personally) multiplies. But that well could be the other way around and I am just not smart enough to grasp it. At least the claim that page_table_free() no longer needs the two-step release indicates that.

Yes, I had to cool myself down to a low temperature, and think about it for several hours before arriving at that conclusion. It would be nice if I could point to one fact which explains it convincingly (something along the lines of "look, we already took it off the list in that case"); but didn't manage to put it into words, and ended up deciding that we each have to do our own thinking to convince ourselves on that issue.

Quoted 2 lines

I am sorry that it is probably not the status you would like to hear,

Not a problem for me at all: you're absolutely right to question whether the existing, and the added, complexity is worth it - all of us who have looked into that code have wondered the same.

Initially I refused to even try to get into it; but in the end was quite proud that I had, given time, managed to work with it. But no problem if that work is quickly discarded in favour of simplicity.

Quoted 1 line

but I still wonder what is your opinion on that do-not-reuse-fragments

I don't want to sway you one way or the other on that: I just want to work with whatever you decide. I know Matthew Wilcox would prefer if powerpc and s390 went about things in the same way (but they do have different constraints, so I don't necessarily agree); but I did not feel able to persuade powerpc to adopt s390's more complex approach, nor to persuade s390 to downgrade to powerpc's simpler approach.

Quoted 1 line

approach? Would it simplify pte_free_defer() or had no effect?

It would certainly simplify it a lot. Whether it would just be a matter of deleting my attempt to list_add() from __tlb_remove_table(), so restoring the per-mm lock, and forgetting the SLAB_TYPESAFE_BY_RCU; or whether it would go further, and most of the PP-AA bit tracking fall away, I cannot predict - depends rather on who does the job, and how far they choose to take it.

Two notes I want to throw into the mix:

One, FWIW my guess is that for most mms, the s390 PP-AA tracking adds very little value; but without it, perhaps there is some mm, some workload, which degrades to the point of using only half of the pgtable memory it allocates.

Two, maybe you'll see this as a further complication, to avoid getting into, rather than a simplification: but it occurred to me while working here that s390 has no need to keep the pgtable_trans_huge_deposit/withdraw() code in mm/pgtable.c separate from the mm/pgalloc.c code. They can use just the one list, and deposit/withdraw simply keep a count of how many immediately usable free pgtables must always be kept in reserve on that list. (But this would not be true at all for powerpc.)

Quoted 2 lines

Anyway, that is just another option and I will try your patch.

Thank you, please do, if you have the time: my series does need some kind of s390 solution, depending on whatever s390 has in its tree at the time: for now I should at least be showing the 13/12 (preferably known to build and appearing to work!), even if all it does is help people to say "no, that's too much".

Hugh

#3 lore raw
Quoted 7 lines · Hugh Dickins
Quoted 1 line

Anyway, that is just another option and I will try your patch.

Thank you, please do, if you have the time: my series does need some kind of s390 solution, depending on whatever s390 has in its tree at the time: for now I should at least be showing the 13/12 (preferably known to build and appearing to work!), even if all it does is help people to say "no, that's too much".

Hi Hugh,

I took a bit different approach to run it: with and withough HH bits in play.

IOW (if I am not mistaken) without the three patch serieses these two patches on top of v6.4-rc5 alone still should manage page allocation, as if pte_free_defer() does not exist/never get called:

s390: add pte_free_defer() for pgtables sharing page prospective 13/12 s390 pgtable_list patch

These patches appear to work (aka survive LTP MM tests).

The next step would be to try the three serieses on top of v6.4-rc5 plus patch 13/12, as you intended:

https://lore.kernel.org/linux-mm/[email protected]/
https://lore.kernel.org/linux-mm/[email protected]/
https://lore.kernel.org/linux-mm/[email protected]
https://lore.kernel.org/r/[email protected]

I had to manually tweak this one with a fuzz in order to circumvent 'git am' error:
[PATCH v2 28/32] mm/memory: allow pte_offset_map[_lock]() to fail: fix

But in the end it still does not compile for me:

mm/memory.c: In function ‘zap_pmd_range’:
mm/memory.c:1561:21: error: implicit declaration of function ‘pmd_none_or_trans_huge_or_clear_bad’ [-Werror=implicit-function-declaration]
1561 | if (pmd_none_or_trans_huge_or_clear_bad(pmd))

      |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  CC      fs/btrfs/dev-replace.o
In file included from ./include/linux/build_bug.h:5,
                 from ./include/linux/container_of.h:5,
                 from ./include/linux/list.h:5,
                 from ./include/linux/smp.h:12,
                 from ./include/linux/kernel_stat.h:5,
                 from mm/memory.c:42:
mm/memory.c: In function ‘do_anonymous_page’:
mm/memory.c:4058:22: error: implicit declaration of function ‘pmd_trans_unstable’; did you mean ‘pud_trans_unstable’? [-Werror=implicit-function-declaration]
 4058 |         if (unlikely(pmd_trans_unstable(vmf->pmd)))
      |                      ^~~~~~~~~~~~~~~~~~

./include/linux/compiler.h:77:45: note: in definition of macro ‘unlikely’
77 | # define unlikely(x) __builtin_expect(!!(x), 0)
| ^
CC fs/xfs/xfs_iops.o
mm/memory.c: In function ‘finish_fault’:
mm/memory.c:4388:13: error: implicit declaration of function ‘pmd_devmap_trans_unstable’; did you mean ‘pud_trans_unstable’? [-Werror=implicit-function-declaration]

 4388 |         if (pmd_devmap_trans_unstable(vmf->pmd))
      |             ^~~~~~~~~~~~~~~~~~~~~~~~~
      |             pud_trans_unstable

Could you please let me know if I am missing something?

Thanks!

Quoted 1 line

Hugh

#4 lore raw
Quoted 14 lines · Alexander Gordeev
Quoted 7 lines · Hugh Dickins
Quoted 1 line

Anyway, that is just another option and I will try your patch.

Thank you, please do, if you have the time: my series does need some kind of s390 solution, depending on whatever s390 has in its tree at the time: for now I should at least be showing the 13/12 (preferably known to build and appearing to work!), even if all it does is help people to say "no, that's too much".

Hi Hugh,

I took a bit different approach to run it: with and withough HH bits in play.

Quoted 6 lines

IOW (if I am not mistaken) without the three patch serieses these two patches on top of v6.4-rc5 alone still should manage page allocation, as if pte_free_defer() does not exist/never get called:

s390: add pte_free_defer() for pgtables sharing page prospective 13/12 s390 pgtable_list patch

That was a good idea, to try the "bare bones" first. Yes, I think that should work okay: with no actual dependence on the preceding patches.

Quoted 2 lines

These patches appear to work (aka survive LTP MM tests).

And that is very good news, thank you (I was expecting I'd at least have done some silly typo like last time).

Quoted 5 lines

The next step would be to try the three serieses on top of v6.4-rc5 plus patch 13/12, as you intended:

https://lore.kernel.org/linux-mm/[email protected]/

There's maybe no difference that matters, but that was the v1 series: v2 was
https://lore.kernel.org/linux-mm/[email protected]/
series of 23 posted on 2023-06-08.

Quoted 1 line

https://lore.kernel.org/linux-mm/[email protected]/

There's maybe no difference that matters, but that was the v1 series: v2 was
https://lore.kernel.org/linux-mm/[email protected]/
series of 32 posted on 2023-06-08.

Quoted 1 line

https://lore.kernel.org/linux-mm/[email protected]

Right.

Quoted 1 line

https://lore.kernel.org/r/[email protected]

Right.

Quoted 5 lines

I had to manually tweak this one with a fuzz in order to circumvent 'git am' error:
[PATCH v2 28/32] mm/memory: allow pte_offset_map[_lock]() to fail: fix

But in the end it still does not compile for me:

I think perhaps you missed out the actual
[PATCH v2 28/32] mm/memory: allow pte_offset_map[_lock]() to fail (oh, not exactly, you were working with v1 there) and tried to apply [PATCH v2 28/32 fix] mm/memory: allow pte_offset_map[_lock]() to fail: fix instead of it; whereas the "fix" patch is a cumulative patch on top of the missed one (and the "fix" patch is not at all important for your testing, since update_mmu_tlb() is a no-op on s390).

(There was other pfaffing around on PATCH v2 28/32, but that was about applying it to akpm's tree at the time: v6.4-rc5 no problem.)

Quoted 22 lines

mm/memory.c: In function ‘zap_pmd_range’:
mm/memory.c:1561:21: error: implicit declaration of function ‘pmd_none_or_trans_huge_or_clear_bad’ [-Werror=implicit-function-declaration]
1561 | if (pmd_none_or_trans_huge_or_clear_bad(pmd))

      |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  CC      fs/btrfs/dev-replace.o
In file included from ./include/linux/build_bug.h:5,
                 from ./include/linux/container_of.h:5,
                 from ./include/linux/list.h:5,
                 from ./include/linux/smp.h:12,
                 from ./include/linux/kernel_stat.h:5,
                 from mm/memory.c:42:
mm/memory.c: In function ‘do_anonymous_page’:
mm/memory.c:4058:22: error: implicit declaration of function ‘pmd_trans_unstable’; did you mean ‘pud_trans_unstable’? [-Werror=implicit-function-declaration]
 4058 |         if (unlikely(pmd_trans_unstable(vmf->pmd)))
      |                      ^~~~~~~~~~~~~~~~~~

./include/linux/compiler.h:77:45: note: in definition of macro ‘unlikely’
77 | # define unlikely(x) __builtin_expect(!!(x), 0)
| ^
CC fs/xfs/xfs_iops.o
mm/memory.c: In function ‘finish_fault’:
mm/memory.c:4388:13: error: implicit declaration of function ‘pmd_devmap_trans_unstable’; did you mean ‘pud_trans_unstable’? [-Werror=implicit-function-declaration]

Quoted 6 lines
 4388 |         if (pmd_devmap_trans_unstable(vmf->pmd))
      |             ^~~~~~~~~~~~~~~~~~~~~~~~~
      |             pud_trans_unstable

Could you please let me know if I am missing something?

I think, start again with the v2 of each of the three series on v6.4-rc5, including the v2 28/32, and I expect that to work out fine.

What you already tested above has achieved a lot: since my changes left pte_free_defer() and page_table_free_rcu() sharing __tlb_remove_table(), so page_table_free_rcu() would already be exercising that list_add().

But you may have difficulty actually exercising the pte_free_defer(). Maybe hack in a counter (I tend to hack on BALLOON events in vmstat, since normally those would be 0 for me) to check whether running LTP is actually exercising those paths - if not, we'll devise something more.

Thanks,
Hugh

maintainer
#5 lore raw
In reply to Hugh Dickins
Quoted 5 lines · On Thu, 22 Jun 2023 22:49:43 -0700 (PDT) Hugh Dickins <[email protected]> wrote:

Hi Gerald,

It's that moment you've been dreading: I'm hoping that you can, please, take a look at the patch below, and try building and running with it, on top of the v2 series of 12 I sent out on Tuesday.

Wow, not sure if this is now dreadful or awesome, it could be the latter since this might actually work. But as Alexander already said, we would rather not add more complexity, for an already questionable benefit with regard to saving some page table memory.

I have revisited my last approach with not adding back fragments in the pte_free_defer() path, and I am rather confident now that there is no "list_del() without list_add()" flaw any more. I had to use the page->pt_frag_refcount for this, to track list status. So even though we do not need page->pt_mm any more, that union is still not free for other things. But I guess that is acceptable, and I especially like that the patch will almost not change existing code at all, apart from the list status tracking.

See my reply to your patch 07/12 in the other thread, for the patch. It is tested with LTP and patches from all three series applied, and it would allow going further with your work by having some s390 solution, that is neither obviously flawed, nor too complex to handle or understand also in the future.

I also left a TODO comment in the patch, with regard to the still open question if we need the gmap_unlink(mm, pgtable, addr) also in pte_free_defer(), similar to page_table_free_rcu(). If yes, we would need some way to pass along the addr, which should be possible. However, IIUC, retract_page_tables() is currently ending up in a normal page_table_free(), w/o your work. And there we also have no addr available, and do no gmap_unlink(), so there might be a good chance that this is not needed here.