Skip to contentSkip to conversation

IA64 Linux Kernel Page Sizes

Linux-ia64 Donny Cooper 5 participants 6 messages Sep 6, 2001 – Sep 11, 2001

lore
Share & tools
lore
#0 lore raw

(1) Below is a previous mail, from David M., responding to page size questions for IA64 kernels.

Quoted 5 lines

As for the future: I'd like to start playing with superpage support fairly soon (once I get some spare time on my hands... ;-). With superpages, physically contiguous regions of memory could be mapped with a single TLB entry, giving you most of the benefit of (very)
large page sizes. --david

Are there any updates on these plans?

(2) Below are current available kernel options (from config).

4KB For best IA-32 compatibility
8KB For best IA-64 performance
16KB For best IA-64 performance
64KB Not for Itanium.

What is "IA64 best performance" based on?

We have a large memory system (32GB), and are not concerned with wasting memory during certain benchmarks. Are 64KB pages truly not functional on Itanium, or just not recommended because of memory waste. To my knowledge, HP-UX seems to benefit from 64KB pages on ITANIUM.

Any input or recommendation on these items is appreciated.

Thanks.
-----------------------------------
Donny Cooper
NEC Systems, Inc.
Advanced Technical Computing Center
[email protected]
www.necservers.com
-----------------------------------

#1 lore raw
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Donny Cooper

Donny> (1) Below is a previous mail, from David M., responding to Donny> page size questions for IA64 kernels.

Quoted 5 lines
Quoted 5 lines

As for the future: I'd like to start playing with superpage support fairly soon (once I get some spare time on my hands... ;-). With superpages, physically contiguous regions of memory could be mapped with a single TLB entry, giving you most of the benefit of (very) large page sizes. --david

Donny> Are there any updates on these plans?

Nope, not yet.

Donny> (2) Below are current available kernel options (from config).

Donny> 4KB For best IA-32 compatibility 8KB For best IA-64 Donny> performance 16KB For best IA-64 performance 64KB Not for Donny> Itanium.

Donny> What is "IA64 best performance" based on?

Don made a bunch of measurement. I tried both local and NFS-mounted kernel compiles. Both benefited from 16KB page size. I never tried 64KB page size. I'm not sure it's worth bothering with it on Itanium (but I'm always willing to consider evidence to the contrary... ;-).

Donny> We have a large memory system (32GB), and are not concerned Donny> with wasting memory during certain benchmarks. Are 64KB Donny> pages truly not functional on Itanium, or just not Donny> recommended because of memory waste. To my knowledge, HP-UX Donny> seems to benefit from 64KB pages on ITANIUM.

Donny> Any input or recommendation on these items is appreciated.

No, 64KB page size functions fine on Itanium, it's just that we don't support it in Linux (due to some silly limits that you'll run into). I think Jack Steiner from SGI posted some patches to work around those issues. I'd play with those first if you're interested in this.

	--david
#2 lore raw
In reply to Donny Cooper

I have an early prototype of super large page(256MB) implementation. This is only a proof of concept for large apps (and configurations) to use super large pages. The use is for apps that need large data regions (Instructions and regular data still use regular page size). Current implementation is through shared memory calls (shmat) with either SHM_LPAGE (a new flag) or size of requested segment >=LPAGE_SIZE(256MB). In either case the size has to be multiple of LPAGE_SIZE to get a large shared memory segment. Further the whole physical memory is divided in two parts, one that is seen by kernel and the other that is only used to fill in large page requests. And there is no moving of memory between two parts (in case of shortages). This implementation is based on 2.4.5 kernel. Currently IA-32 apps don't work in this environment, but that can be fixed.

Let me know if this fits in your requirements and you want to give this a try. I can send you a cut of (modified)tar files out of my source tree early next week.

rohit

-----Original Message-----
From: Donny Cooper [mailto:[email protected]]
Sent: Thursday, September 06, 2001 9:23 AM
To: [email protected]
Cc: Cooper (E-mail)
Subject: [Linux-ia64] IA64 Linux Kernel Page Sizes

(1) Below is a previous mail, from David M., responding to page size questions for IA64 kernels.

Quoted 5 lines

As for the future: I'd like to start playing with superpage support fairly soon (once I get some spare time on my hands... ;-). With superpages, physically contiguous regions of memory could be mapped with a single TLB entry, giving you most of the benefit of (very)
large page sizes. --david

Are there any updates on these plans?

(2) Below are current available kernel options (from config).

4KB For best IA-32 compatibility
8KB For best IA-64 performance
16KB For best IA-64 performance
64KB Not for Itanium.

What is "IA64 best performance" based on?

We have a large memory system (32GB), and are not concerned with wasting memory during certain benchmarks. Are 64KB
pages truly not functional on Itanium, or just not recommended because of memory waste. To my knowledge, HP-UX seems to
benefit from 64KB pages on ITANIUM.

Any input or recommendation on these items is appreciated.

Thanks.
-----------------------------------
Donny Cooper
NEC Systems, Inc.
Advanced Technical Computing Center
[email protected]
www.necservers.com
-----------------------------------

_______________________________________________
Linux-IA64 mailing list
[email protected]
http://lists.linuxia64.org/lists/listinfo/linux-ia64

#3 lore raw
In reply to Donny Cooper

Dear Rohit,

I'd like to try the super large pages which you mentioned in your mail. I tried the 64k pages with IA64 Linux but the performance advantage was less than expected (at least with the applications I used)...

Thanks a lot in advance!

Best regards,
Erich

---
Erich Focht <[email protected]>
NEC European Supercomputer Systems, European HPC Technology Center

Quoted 65 lines · Rohit

I have an early prototype of super large page(256MB) implementation. This is only a proof of concept for large apps (and configurations) to use super large pages. The use is for apps that need large data regions (Instructions and regular data still use regular page size). Current implementation is through shared memory calls (shmat) with either SHM_LPAGE (a new flag) or size of requested segment >=LPAGE_SIZE(256MB). In either case the size has to be multiple of LPAGE_SIZE to get a large shared memory segment. Further the whole physical memory is divided in two parts, one that is seen by kernel and the other that is only used to fill in large page requests. And there is no moving of memory between two parts (in case of shortages). This implementation is based on 2.4.5 kernel. Currently IA-32 apps don't work in this environment, but that can be fixed.

Let me know if this fits in your requirements and you want to give this a try. I can send you a cut of (modified)tar files out of my source tree early next week.

rohit

-----Original Message-----
From: Donny Cooper [mailto:[email protected]]
Sent: Thursday, September 06, 2001 9:23 AM
To: [email protected]
Cc: Cooper (E-mail)
Subject: [Linux-ia64] IA64 Linux Kernel Page Sizes

(1) Below is a previous mail, from David M., responding to page size questions for IA64 kernels.

Quoted 5 lines

As for the future: I'd like to start playing with superpage support fairly soon (once I get some spare time on my hands... ;-). With superpages, physically contiguous regions of memory could be mapped with a single TLB entry, giving you most of the benefit of (very)
large page sizes. --david

Are there any updates on these plans?

(2) Below are current available kernel options (from config).

4KB For best IA-32 compatibility
8KB For best IA-64 performance
16KB For best IA-64 performance
64KB Not for Itanium.

What is "IA64 best performance" based on?

We have a large memory system (32GB), and are not concerned with wasting memory during certain benchmarks. Are 64KB
pages truly not functional on Itanium, or just not recommended because of memory waste. To my knowledge, HP-UX seems to
benefit from 64KB pages on ITANIUM.

Any input or recommendation on these items is appreciated.

Thanks.
-----------------------------------
Donny Cooper
NEC Systems, Inc.
Advanced Technical Computing Center
[email protected]
www.necservers.com
-----------------------------------

#4 lore raw
In reply to Donny Cooper

Erich,
I will put the kernel binary on ftp.intel.com (I'm currently having problems ftping to that site) for you to download. If you want I can put the tar of modified files (no patch) on that site also. And will let you know. Meanwhile if you and Donny are in same place (both of you have ....nec... in your addresses) then you can get the kernel binary and other info (for apps to use this feature) from him.

I'll let you know once I have downloaded the files to ftp site.

rohit

-----Original Message-----
From: Erich Focht [mailto:[email protected]]
Sent: Monday, September 10, 2001 1:32 PM
To: Seth, Rohit
Cc: Donny Cooper; [email protected]
Subject: Re: [Linux-ia64] IA64 Linux Kernel Page Sizes

Dear Rohit,

I'd like to try the super large pages which you mentioned in your mail. I tried the 64k pages with IA64 Linux but the performance advantage was less than expected (at least with the applications I used)...

Thanks a lot in advance!

Best regards,
Erich

---
Erich Focht <[email protected]>
NEC European Supercomputer Systems, European HPC Technology Center

Quoted 2 lines · Rohit

I have an early prototype of super large page(256MB) implementation. This is only a proof of concept for large apps (and configurations) to use

super

Quoted 1 line

large pages. The use is for apps that need large data regions

(Instructions

Quoted 3 lines

and regular data still use regular page size). Current implementation is through shared memory calls (shmat) with either SHM_LPAGE (a new flag) or size of requested segment >=LPAGE_SIZE(256MB). In either case the size

has

Quoted 1 line

to be multiple of LPAGE_SIZE to get a large shared memory segment.

Further

Quoted 2 lines

the whole physical memory is divided in two parts, one that is seen by kernel and the other that is only used to fill in large page requests.

And

Quoted 1 line

there is no moving of memory between two parts (in case of shortages).

This

Quoted 1 line

implementation is based on 2.4.5 kernel. Currently IA-32 apps don't work

in

Quoted 54 lines

this environment, but that can be fixed.

Let me know if this fits in your requirements and you want to give this a try. I can send you a cut of (modified)tar files out of my source tree early next week.

rohit

-----Original Message-----
From: Donny Cooper [mailto:[email protected]]
Sent: Thursday, September 06, 2001 9:23 AM
To: [email protected]
Cc: Cooper (E-mail)
Subject: [Linux-ia64] IA64 Linux Kernel Page Sizes

(1) Below is a previous mail, from David M., responding to page size questions for IA64 kernels.

Quoted 5 lines

As for the future: I'd like to start playing with superpage support fairly soon (once I get some spare time on my hands... ;-). With superpages, physically contiguous regions of memory could be mapped with a single TLB entry, giving you most of the benefit of (very)
large page sizes. --david

Are there any updates on these plans?

(2) Below are current available kernel options (from config).

4KB For best IA-32 compatibility
8KB For best IA-64 performance
16KB For best IA-64 performance
64KB Not for Itanium.

What is "IA64 best performance" based on?

We have a large memory system (32GB), and are not concerned with wasting memory during certain benchmarks. Are 64KB
pages truly not functional on Itanium, or just not recommended because of memory waste. To my knowledge, HP-UX seems to
benefit from 64KB pages on ITANIUM.

Any input or recommendation on these items is appreciated.

Thanks.
-----------------------------------
Donny Cooper
NEC Systems, Inc.
Advanced Technical Computing Center
[email protected]
www.necservers.com
-----------------------------------

#5 lore raw
In reply to Donny Cooper
Quoted 7 lines · Rohit

Erich,
I will put the kernel binary on ftp.intel.com (I'm currently having problems ftping to that site) for you to download. If you want I can put the tar of modified files (no patch) on that site also. And will let you know. Meanwhile if you and Donny are in same place (both of you have ....nec... in your addresses) then you can get the kernel binary and other info (for apps to use this feature) from him.

I you put the binary there you have to upload the sources as well. (And yes, I _am_ interested in the source).

Christoph

Signature · 1 line
Of course it doesn't work. We've performed a software upgrade.