Documentation: hugetlb: Update hugetlb options

PATCH In review Peter Xu 3 participants 5 messages Apr 9, 2020 – Apr 10, 2020

Subsystems (2)

Memory management Documentation

lore

Share & tools

lore

Peter Xu

author

2020-04-09 21:58 UTC #0 lore raw

PATCH 1/1 Documentation: hugetlb: Update hugetlb options

The hugepage options are not documented clearly.

Firstly, default_hugepagesz= should always be specified after the declaration of the same type of huge page using hugepagesz=. For example, if we boot a x86_64 system with kernel cmdline "default_hugepagesz=2M", we'll get a very funny error message:

"HugeTLB: unsupported default_hugepagesz 2097152. Reverting to 2097152"

It's understandable from code-wise because when hugetlb_init() we didn't have the 2M page hstate registered, so it's unsupported. However 2M is actually the default huge page size on x86_64, so we'll register it right after the error message. However it's very confusing if without these knowledges.

Secondly, hugepages= option must be used _after_ another hugepagesz=. The word "interleave" is fine but it didn't declare the fact that each of the hugepages= option will be applied to the previous parsed hugepagesz= option.

State all these clear.

Signed-off-by Peter Xu <[email protected]>

---
 .../admin-guide/kernel-parameters.txt         | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

Documentation/admin-guide/kernel-parameters.txt +12 -7

index 86aae1fa099a..2a77b2b01e5e 100644
@@ -839,7 +839,8 @@
 			the legacy /proc/ hugepages APIs, used for SHM, and
 			default size when mounting hugetlbfs filesystems.
 			Defaults to the default architecture's huge page size
-			if not specified.
+			if not specified.  This option can only be used with a
+			pre-defined "hugepagesz=" of the same size.
 
 	deferred_probe_timeout=
 			[KNL] Debugging option to set a timeout in seconds for

@@ -1475,13 +1476,17 @@
 	hpet_mmap=	[X86, HPET_MMAP] Allow userspace to mmap HPET
 			registers.  Default set by CONFIG_HPET_MMAP_DEFAULT.
 
+	hugepagesz=	[HW,IA-64,PPC,X86-64] Declare one type of HugeTLB pages
+			with the size specified.  Valid pages sizes on x86-64
+			are 2M (when the CPU supports "pse") and 1G (when the
+			CPU supports the "pdpe1gb" cpuinfo flag).
+
 	hugepages=	[HW,X86-32,IA-64] HugeTLB pages to allocate at boot.
-	hugepagesz=	[HW,IA-64,PPC,X86-64] The size of the HugeTLB pages.
-			On x86-64 and powerpc, this option can be specified
-			multiple times interleaved with hugepages= to reserve
-			huge pages of different sizes. Valid pages sizes on
-			x86-64 are 2M (when the CPU supports "pse") and 1G
-			(when the CPU supports the "pdpe1gb" cpuinfo flag).
+			On x86-64 and powerpc, this option must be used after
+			one hugepagesz= to allocate huge pages of that specific
+			size.  This option can also be used multiple times
+			pairing with hugepagesz= to allocate huge pages for
+			different sizes.
 
 	hung_task_panic=
 			[KNL] Should the hung task detector generate panics.

Signature · 1 line

2.24.1

Mike Kravetz

2020-04-09 22:08 UTC #1 lore raw

Quoted 15 lines · Peter Xu

The hugepage options are not documented clearly.
Firstly, default_hugepagesz= should always be specified after the declaration of the same type of huge page using hugepagesz=. For example, if we boot a x86_64 system with kernel cmdline "default_hugepagesz=2M", we'll get a very funny error message:
"HugeTLB: unsupported default_hugepagesz 2097152. Reverting to 2097152"
It's understandable from code-wise because when hugetlb_init() we didn't have the 2M page hstate registered, so it's unsupported. However 2M is actually the default huge page size on x86_64, so we'll register it right after the error message. However it's very confusing if without these knowledges.

Quoted 8 lines

Secondly, hugepages= option must be used _after_ another hugepagesz=. The word "interleave" is fine but it didn't declare the fact that each of the hugepages= option will be applied to the previous parsed hugepagesz= option.
State all these clear.
Signed-off-by Peter Xu <[email protected]>

Hi Peter,

Did you happen to see this patch series?

https://lore.kernel.org/linux-mm/[email protected]/

That should address the documentation issue and more.

Signature · 1 line

Mike Kravetz

Peter Xu

author

2020-04-09 22:18 UTC #2 lore raw

Quoted 25 lines · Mike Kravetz

Quoted 15 lines · Peter Xu
The hugepage options are not documented clearly.
Firstly, default_hugepagesz= should always be specified after the declaration of the same type of huge page using hugepagesz=. For example, if we boot a x86_64 system with kernel cmdline "default_hugepagesz=2M", we'll get a very funny error message:
"HugeTLB: unsupported default_hugepagesz 2097152. Reverting to 2097152"
It's understandable from code-wise because when hugetlb_init() we didn't have the 2M page hstate registered, so it's unsupported. However 2M is actually the default huge page size on x86_64, so we'll register it right after the error message. However it's very confusing if without these knowledges.
Quoted 8 lines
Secondly, hugepages= option must be used _after_ another hugepagesz=. The word "interleave" is fine but it didn't declare the fact that each of the hugepages= option will be applied to the previous parsed hugepagesz= option.
State all these clear.
Signed-off-by Peter Xu <[email protected]>

Quoted 7 lines

Hi Peter,
Did you happen to see this patch series?
https://lore.kernel.org/linux-mm/[email protected]/
That should address the documentation issue and more.

Great!

I missed that, but I'll definitely read it (probably tomorrow). Let's ignore this patch then.

Thanks,

Signature · 1 line

Peter Xu

Nitesh Narayan Lal

2020-04-10 12:02 UTC #3 lore raw

Quoted 29 lines · Peter Xu

Quoted 28 lines · Mike Kravetz
Quoted 15 lines · Peter Xu
The hugepage options are not documented clearly.
Firstly, default_hugepagesz= should always be specified after the declaration of the same type of huge page using hugepagesz=. For example, if we boot a x86_64 system with kernel cmdline "default_hugepagesz=2M", we'll get a very funny error message:
"HugeTLB: unsupported default_hugepagesz 2097152. Reverting to 2097152"
It's understandable from code-wise because when hugetlb_init() we didn't have the 2M page hstate registered, so it's unsupported. However 2M is actually the default huge page size on x86_64, so we'll register it right after the error message. However it's very confusing if without these knowledges.
Quoted 8 lines
Secondly, hugepages= option must be used _after_ another hugepagesz=. The word "interleave" is fine but it didn't declare the fact that each of the hugepages= option will be applied to the previous parsed hugepagesz= option.
State all these clear.
Signed-off-by Peter Xu <[email protected]>
Hi Peter,
Did you happen to see this patch series?

Quoted 7 lines

Quoted 3 lines
https://lore.kernel.org/linux-mm/[email protected]/
That should address the documentation issue and more.
Great!
I missed that, but I'll definitely read it (probably tomorrow). Let's ignore this patch then.

My bad, I didn't realize that Mike has already pointed you in the right direction. Please ignore my earlier email. :)

Quoted 3 lines

Thanks,

Signature · 1 line

Nitesh

Nitesh Narayan Lal

2020-04-10 11:57 UTC #4 lore raw

In reply to Peter Xu

Quoted 8 lines · Peter Xu

The hugepage options are not documented clearly.
Firstly, default_hugepagesz= should always be specified after the declaration of the same type of huge page using hugepagesz=. For example, if we boot a x86_64 system with kernel cmdline "default_hugepagesz=2M", we'll get a very funny error message:
"HugeTLB: unsupported default_hugepagesz 2097152. Reverting to 2097152"

There is an ongoing discussion to fix this issue.
AFAIK it has not been settled yet and Mike was going to post a patch-set.
https://lkml.org/lkml/2020/3/9/96

Quoted 57 lines

State all these clear.

Signed-off-by Peter Xu <[email protected]>

---
 .../admin-guide/kernel-parameters.txt         | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

Documentation/admin-guide/kernel-parameters.txt +12 -7

index 86aae1fa099a..2a77b2b01e5e 100644
@@ -839,7 +839,8 @@
 			the legacy /proc/ hugepages APIs, used for SHM, and
 			default size when mounting hugetlbfs filesystems.
 			Defaults to the default architecture's huge page size
-			if not specified.
+			if not specified.  This option can only be used with a
+			pre-defined "hugepagesz=" of the same size.
 
 	deferred_probe_timeout=
 			[KNL] Debugging option to set a timeout in seconds for

@@ -1475,13 +1476,17 @@
 	hpet_mmap=	[X86, HPET_MMAP] Allow userspace to mmap HPET
 			registers.  Default set by CONFIG_HPET_MMAP_DEFAULT.
 
+	hugepagesz=	[HW,IA-64,PPC,X86-64] Declare one type of HugeTLB pages
+			with the size specified.  Valid pages sizes on x86-64
+			are 2M (when the CPU supports "pse") and 1G (when the
+			CPU supports the "pdpe1gb" cpuinfo flag).
+
 	hugepages=	[HW,X86-32,IA-64] HugeTLB pages to allocate at boot.
-	hugepagesz=	[HW,IA-64,PPC,X86-64] The size of the HugeTLB pages.
-			On x86-64 and powerpc, this option can be specified
-			multiple times interleaved with hugepages= to reserve
-			huge pages of different sizes. Valid pages sizes on
-			x86-64 are 2M (when the CPU supports "pse") and 1G
-			(when the CPU supports the "pdpe1gb" cpuinfo flag).
+			On x86-64 and powerpc, this option must be used after
+			one hugepagesz= to allocate huge pages of that specific
+			size.  This option can also be used multiple times
+			pairing with hugepagesz= to allocate huge pages for
+			different sizes.
 
 	hung_task_panic=
 			[KNL] Should the hung task detector generate panics.

Signature · 1 line

Nitesh