Skip to contentSkip to conversation

Extended Addressing Mode

DISCUSSION Régis Odeyé 5 participants 19 messages Oct 22, 2008

lore
Share & tools
lore
#0 lore raw

Hello Everyone,
I'm looking for some information about the Extended Addressing Mode (XAEN bit of HID0 register) of PPC32 support in Linux. I do not see anything in the main kernel tree but there may be some patches available ?
Any information will be appreciate.
Regards.

Signature · 6 lines
Régis ODEYE

Kontron Modular Computers SA
150, rue M. Berthelot / ZI Toulon Est / BP 244 / Fr 83078 TOULON Cedex 9
Phone: (33) 4 98 16 34 86           Fax: (33) 4 98 16 34 01
E-mail: [email protected]  Web : www.kontron.com
#1 lore raw
Quoted 9 lines · On Oct 22, 2008, at 3:59 AM, R=E9gis Odey=E9 wrote:

Hello Everyone,
I'm looking for some information about the Extended Addressing Mode =20=

(XAEN bit of HID0 register) of PPC32 support in Linux.
I do not see anything in the main kernel tree but there may be some =20=

patches available ?
Any information will be appreciate.
Regards.

There are patches from Becky Bruce that are going into 2.6.28. What =20 are you needing >32-bit for? There are some SW IO MMU changes that =20 are still pending to complete this work.

- k=

#2 lore raw
Quoted 9 lines · Kumar Gala
Quoted 5 lines · Régis Odeyé

Hello Everyone,
I'm looking for some information about the Extended Addressing Mode (XAEN bit of HID0 register) of PPC32 support in Linux. I do not see anything in the main kernel tree but there may be some patches available ?
Any information will be appreciate.
Regards.

Quoted 3 lines

There are patches from Becky Bruce that are going into 2.6.28. What are you needing >32-bit for? There are some SW IO MMU changes that are still pending to complete this work.

- k

We are developing a board based on Freescale 8641D which can get 4GB of ram. So I need 4GB+IOs (~1GB) of physical addressing space.
My plan is to put a part of this ram above of 4GB to keep accesses to the IOs below the 4GB limit. It means non-contiguous ram addressing and XAEN features to be working. Where can I glance through Becky patches ?

Signature · 6 lines
Régis ODEYE

Kontron Modular Computers SA
150, rue M. Berthelot / ZI Toulon Est / BP 244 / Fr 83078 TOULON Cedex 9
Phone: (33) 4 98 16 34 86           Fax: (33) 4 98 16 34 01
E-mail: [email protected]  Web : www.kontron.com
#3 lore raw
Quoted 17 lines · On Oct 22, 2008, at 7:59 AM, R=E9gis Odey=E9 wrote:
Quoted 16 lines · Kumar Gala
Quoted 7 lines · On Oct 22, 2008, at 3:59 AM, R=E9gis Odey=E9 wrote:

Hello Everyone,
I'm looking for some information about the Extended Addressing =20 Mode (XAEN bit of HID0 register) of PPC32 support in Linux. I do not see anything in the main kernel tree but there may be =20 some patches available ?
Any information will be appreciate.
Regards.

There are patches from Becky Bruce that are going into 2.6.28. =20 What are you needing >32-bit for? There are some SW IO MMU changes =20=

that are still pending to complete this work.

Quoted 7 lines
Quoted 1 line

- k

We are developing a board based on Freescale 8641D which can get 4GB =20=

of ram. So I need 4GB+IOs (~1GB) of physical addressing space. My plan is to put a part of this ram above of 4GB to keep accesses =20 to the IOs below the 4GB limit. It means non-contiguous ram =20 addressing and XAEN features to be working.

So we have XAEN support in the tree.. however non-contiguous is =20 something you'll have to work on yourself. Patches are welcome for this

Quoted 2 lines

Where can I glance through Becky patches ?

This is the bulk:

=
http://git.kernel.org/?p=3Dlinux/kernel/git/torvalds/linux-2.6.git;a=3Dcom=
mitdiff;h=3D4ee7084eb11e00eb02dc8435fd18273a61ffa9bf

- k=

#4 lore raw
Quoted 1 line · Kumar Gala

So we have XAEN support in the tree.. however non-contiguous is something you'll have to work on yourself. Patches are welcome for this

OK. I will let you know about this.

Quoted 7 lines
Quoted 1 line

Where can I glance through Becky patches ?

This is the bulk:

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=4ee7084eb11e00eb02dc8435fd18273a61ffa9bf

Thank you very much, Sir.

Signature · 6 lines
Régis ODEYE

Kontron Modular Computers SA
150, rue M. Berthelot / ZI Toulon Est / BP 244 / Fr 83078 TOULON Cedex 9
Phone: (33) 4 98 16 34 86           Fax: (33) 4 98 16 34 01
E-mail: [email protected]  Web : www.kontron.com
#5 lore raw
In reply to Kumar Gala
Quoted 7 lines · Kumar Gala
Quoted 2 lines · Régis Odeyé

of ram. So I need 4GB+IOs (~1GB) of physical addressing space.
My plan is to put a part of this ram above of 4GB to keep accesses to the IOs below the 4GB limit. It means non-contiguous ram addressing and XAEN features to be working.

So we have XAEN support in the tree.. however non-contiguous is something you'll have to work on yourself. Patches are welcome for this

So to confirm, XAEN support through Becky's patches does support the MPC8641D/e600 cores?

Quoted 5 lines
Quoted 1 line

Where can I glance through Becky patches ?

This is the bulk:

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=4ee7084eb11e00eb02dc8435fd18273a61ffa9bf

I'd also be interested in any work done to enable non-contiguous memory areas. Reading the docs for the MPC8641D though I am not sure you can set up LAWs for it?

One thing I wanted to try was installing 4GB in a system and "overlapping" IO (since there is very little of it on a stock MPC8641DHPCN) in the top ~256MB-512MB, but I am fairly sure this is NOT supported because of the way the LAWs work, and also the alignment of the LAWs means it is not fine enough granularity to map between 2GB and 4GB into a window (you can have 2GB or 4GB but not some more arbitrary value?)

Signature · 2 lines
Matt Sealey <[email protected]>
Genesi, Manager, Developer Relations
#6 lore raw
Quoted 9 lines · Matt Sealey
Quoted 6 lines · Kumar Gala
Quoted 5 lines · On Oct 22, 2008, at 7:59 AM, R=E9gis Odey=E9 wrote:

of ram. So I need 4GB+IOs (~1GB) of physical addressing space. My plan is to put a part of this ram above of 4GB to keep accesses =20=

to the IOs below the 4GB limit. It means non-contiguous ram =20 addressing and XAEN features to be working.

Quoted 8 lines
Quoted 4 lines

So we have XAEN support in the tree.. however non-contiguous is =20 something you'll have to work on yourself. Patches are welcome for =20=

this

So to confirm, XAEN support through Becky's patches does support the =20=

MPC8641D/e600 cores?

Yes, its the only part that has XAEN.

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

Where can I glance through Becky patches ?

This is the bulk:
=

http://git.kernel.org/?p=3Dlinux/kernel/git/torvalds/linux-2.6.git;a=3Dcom=
mitdiff;h=3D4ee7084eb11e00eb02dc8435fd18273a61ffa9bf

Quoted 5 lines

I'd also be interested in any work done to enable non-contiguous =20 memory areas. Reading the docs for the MPC8641D though I am not sure =20=

you can set up LAWs for it?

You can if you can configure your DDR to support it.

Quoted 9 lines

One thing I wanted to try was installing 4GB in a system and =20 "overlapping" IO (since there is very little of it on a stock =20 MPC8641DHPCN) in the top ~256MB-512MB, but I am fairly sure this is =20=

NOT supported because of the way the LAWs work, and also the =20 alignment of the LAWs means it is not fine enough granularity to map =20=

between 2GB and 4GB into a window (you can have 2GB or 4GB but not =20 some more arbitrary value?)

You can overlap LAWs because they have priority encoding. The lower =20 the LAW # the higher the priority.

Your bigger issue is if you can setup the DDR controller for the hole =20=

you want.

- k=

#7 lore raw
Quoted 5 lines · Kumar Gala
Quoted 1 line · Matt Sealey

So to confirm, XAEN support through Becky's patches does support the MPC8641D/e600 cores?

Yes, its the only part that has XAEN.

Okay I saw a lot of e500/BookE support go past but nothing specific :)

Quoted 3 lines
Quoted 1 line

NOT supported because of the way the LAWs work, and also the alignment of the LAWs means it is not fine enough granularity to map between 2GB and 4GB into a window (you can have 2GB or 4GB but not some more arbitrary value?)

You can overlap LAWs because they have priority encoding. The lower the LAW # the higher the priority.

Hmm :)

Quoted 1 line

Your bigger issue is if you can setup the DDR controller for the hole you want.

Hmmmmm :)

Okay good to know. I'll go back to reading the docs.

Signature · 2 lines
Matt Sealey <[email protected]>
Genesi, Manager, Developer Relations
#8 lore raw
Quoted 8 lines · Matt Sealey
Quoted 3 lines · Kumar Gala
Quoted 1 line · Matt Sealey

So to confirm, XAEN support through Becky's patches does support the MPC8641D/e600 cores?

Yes, its the only part that has XAEN.

Okay I saw a lot of e500/BookE support go past but nothing specific :)

The "XAEN" is specific to e600. e500 has its own ability to do 36-bit physical (not called XAEN).

- k

#9 lore raw
Quoted 5 lines · Kumar Gala
Quoted 2 lines · Matt Sealey
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines · Kumar Gala
Quoted 1 line · Matt Sealey

So to confirm, XAEN support through Becky's patches does support the MPC8641D/e600 cores?

Yes, its the only part that has XAEN.

Okay I saw a lot of e500/BookE support go past but nothing specific :)

The "XAEN" is specific to e600. e500 has its own ability to do 36-bit physical (not called XAEN).

You have to forgive me I was only half awake when I started :D

Yeah so I saw BookE and e500 stuff go past but nothing specific for e600/XAEN. I mailed Becky and got no response. I'm glad to know it's in there though, somewhere.

Just so it has been asked, do you know in a broad sense what it would take to add the non-contiguous memory mapping support? Doesn't ppc64 already have this?

Signature · 2 lines
Matt Sealey <[email protected]>
Genesi, Manager, Developer Relations
#10 lore raw
Quoted 9 lines · Matt Sealey
Quoted 6 lines · Kumar Gala
Quoted 5 lines · Matt Sealey
Quoted 2 lines · Kumar Gala
Quoted 1 line · Matt Sealey

So to confirm, XAEN support through Becky's patches does support the MPC8641D/e600 cores?

Quoted 8 lines
Quoted 4 lines
Quoted 3 lines
Quoted 1 line

Yes, its the only part that has XAEN.

Okay I saw a lot of e500/BookE support go past but nothing specific :)

The "XAEN" is specific to e600. e500 has its own ability to do 36-bit physical (not called XAEN).

You have to forgive me I was only half awake when I started :D

Yeah so I saw BookE and e500 stuff go past but nothing specific for e600/XAEN. I mailed Becky and got no response. I'm glad to know it's in there though, somewhere.

Huh? I have one mail from you, on Sep 1, to which I responded on Sep 2. Was there another mail I missed, or did you not see my response? Freescale's mail server is unreliable, and since I'm not in the habit of ignoring emails, feel free to nag me anytime if you send me something and don't get a response.

Quoted 3 lines

Just so it has been asked, do you know in a broad sense what it would take to add the non-contiguous memory mapping support? Doesn't ppc64 already have this?

PPC64 has SPARSEMEM support, IIRC. It's non-trivial to add for 32-bit, although I haven't scoped it out in any detail.

Cheers,
B

#11 lore raw
Quoted 6 lines · Becky Bruce
Quoted 1 line · Matt Sealey

Yeah so I saw BookE and e500 stuff go past but nothing specific for e600/XAEN. I mailed Becky and got no response. I'm glad to know it's in there though, somewhere.

Huh? I have one mail from you, on Sep 1, to which I responded on Sep 2. Was there another mail I missed, or did you not see my response? Freescale's mail server is unreliable, and since I'm not in the habit of ignoring emails, feel free to nag me anytime if you send me something and don't get a response.

Between Freescale's mail server and the trouble we're having with Google right now (along with thousands of others), I am NOT surprised it got lost somewhere along the way.

I wasn't bitching :)

Quoted 3 lines
Quoted 1 line

Just so it has been asked, do you know in a broad sense what it would take to add the non-contiguous memory mapping support? Doesn't ppc64 already have this?

PPC64 has SPARSEMEM support, IIRC. It's non-trivial to add for 32-bit, although I haven't scoped it out in any detail.

Okay, non-trivial was pretty much what I was looking for, I was just trying to weigh up how much effort is required..

If you actually scope it out in any detail or someone has some 10% time and decides this would be an awesome project, give me a nudge? My MPC8641D is itching to actually do something besides build packages. Actually having 8GB of memory would REALLY help run SUSE Build Service :]

Signature · 2 lines
Matt Sealey <[email protected]>
Genesi, Manager, Developer Relations
#12 lore raw
In reply to Kumar Gala
Quoted 3 lines · Kumar Gala

Your bigger issue is if you can setup the DDR controller for the hole you want.

I just remembered;;

~~
The CCSR window always takes precedence over all local access windows. However, the CCSR window must not overlap an LAW that maps to the DDR controller. Otherwise, undefined behavior occurs. ~~

So, it's not really possible to map 4GB of RAM in the lower 32-bit area, without interacting badly with the CCSR. This means you're forced to select a 2GB LAW for DDR, then leave 2GB free, then map the rest above.. using more than 2Gb therefore absolutely requires non-contiguous memory..?

Signature · 2 lines
Matt Sealey <[email protected]>
Genesi, Manager, Developer Relations
#13 lore raw
Quoted 5 lines · Matt Sealey
Quoted 1 line · Kumar Gala

Your bigger issue is if you can setup the DDR controller for the hole you want.

Quoted 7 lines

I just remembered;;

~~
The CCSR window always takes precedence over all local access windows. However, the CCSR window must not overlap an LAW that maps to the DDR controller. Otherwise, undefined behavior occurs. ~~

So, it's not really possible to map 4GB of RAM in the lower 32-bit area, without interacting badly with the CCSR. This means you're forced to select a 2GB LAW for DDR, then leave 2GB free, then map the rest above.. using more than 2Gb therefore absolutely requires non-contiguous memory..?

As I said, its all about your physical DDR layout. If you have two DDR dimms each 2Gb you can do:

0..2G - DDR DIMM A
2G..4G - IO
4G..6G - DDR DIMM B

- k

#14 lore raw
Quoted 7 lines · Kumar Gala
Quoted 4 lines · Matt Sealey

~~
The CCSR window always takes precedence over all local access windows. However, the CCSR window must not overlap an LAW that maps to the DDR controller. Otherwise, undefined behavior occurs. ~~

Quoted 7 lines
Quoted 1 line

So, it's not really possible to map 4GB of RAM in the lower 32-bit area, without interacting badly with the CCSR. This means you're forced to select a 2GB LAW for DDR, then leave 2GB free, then map the rest above.. using more than 2Gb therefore absolutely requires non-contiguous memory..?

As I said, its all about your physical DDR layout. If you have two DDR dimms each 2Gb you can do:

0..2G - DDR DIMM A
2G..4G - IO
4G..6G - DDR DIMM B

I assume on the HPCN "DDR DIMM A" would be one or both of one set of DDR slots, and DDR DIMM B would be one or both o the other set (since there are 4 slots, two for each controller)?

Or are we talking about actual, physical DIMMs?

If we're talking about controllers, could you not do;

0..2GB DDR Controller 1 (partial)
2G..4GB IO
4GB..NGB DDR Controller 1 (the rest)
NGB-64GB DDR Controller 2 (or whatever)

Or do LAWs not cooperate when for the same target? I would assume if you set up the CSn_BNDS registers right you could get a real fine grained mapping of DDR controller to memory space in combination with the LAWs? It would then be actually possible (however disgusting this config would be) to have a 2GB DIMM, 1GB DIMM on the first controller with two LAWs, and the appropriate chip selects, then 1GB IO space, then up to 32GB (since 16GB DIMMs are about as high as it goes for DDR2) memory space mapped after that, with a single LAW?

Or more comfortably, pair up 2x 2GB DIMMs and simply ignore the last 1GB, and pair up 2x 16GB DIMMs?

Signature · 2 lines
Matt Sealey <[email protected]>
Genesi, Manager, Developer Relations
#15 lore raw
In reply to Matt Sealey
Quoted 5 lines · Matt Sealey

I'd also be interested in any work done to enable non-contiguous memory areas. Reading the docs for the MPC8641D though I am not sure you can set up LAWs for it?

One thing I wanted to try was installing 4GB in a system and "overlapping" IO (since there is very little of it on a stock MPC8641DHPCN) in the top ~256MB-512MB, but I am fairly sure this is NOT supported because of the way the LAWs work, and also the alignment of the LAWs means it is not fine enough granularity to map between 2GB and 4GB into a window (you can have 2GB or 4GB but not some more arbitrary value?)

On my point of view, LAWs sizes are power of 2 and should be aligned to the window size. But there is 10 LAWs you can combine to achieve quite a lot of different mappings. Regards.

Signature · 6 lines
Régis ODEYE

Kontron Modular Computers SA
150, rue M. Berthelot / ZI Toulon Est / BP 244 / Fr 83078 TOULON Cedex 9
Phone: (33) 4 98 16 34 86           Fax: (33) 4 98 16 34 01
E-mail: [email protected]  Web : www.kontron.com
#16 lore raw
In reply to Kumar Gala
Quoted 6 lines · Kumar Gala
Quoted 1 line

We are developing a board based on Freescale 8641D which can get

4GB

Quoted 4 lines

of ram. So I need 4GB+IOs (~1GB) of physical addressing space. My plan is to put a part of this ram above of 4GB to keep accesses to the IOs below the 4GB limit. It means non-contiguous ram addressing and XAEN features to be working.

Why would you put the IOs below the 4G point ? It's actually easier to have the IOs up above, like a lot of 4xx do.

Cheers,
Ben.

#17 lore raw
Quoted 8 lines · Benjamin Herrenschmidt
Quoted 4 lines · Kumar Gala
Quoted 1 line

We are developing a board based on Freescale 8641D which can get

4GB

Quoted 2 lines

of ram. So I need 4GB+IOs (~1GB) of physical addressing space.
My plan is to put a part of this ram above of 4GB to keep accesses to the IOs below the 4GB limit. It means non-contiguous ram addressing and XAEN features to be working.

Why would you put the IOs below the 4G point ? It's actually easier to have the IOs up above, like a lot of 4xx do.

Because we're Genesi! And we have a firmware solution that kind of has to keep 32-bit pointers in the unlikely event that someone actually uses the client interface (besides yaboot!).

Having I/O in the 36-bit range could cause all sorts of explosions, and we're running real-mode so trapping and faking I/O accesses is REALLy difficult :}

Signature · 2 lines
Matt Sealey <[email protected]>
Genesi, Manager, Developer Relations
#18 lore raw
Quoted 8 lines · Matt Sealey

Because we're Genesi! And we have a firmware solution that kind of has to keep 32-bit pointers in the unlikely event that someone actually uses the client interface (besides yaboot!).

Having I/O in the 36-bit range could cause all sorts of explosions, and we're running real-mode so trapping and faking I/O accesses is REALLy difficult :}

how so ? we have plethora of 32-bit firmwares including OF implementations that can perfectly address 36 bit physical address space. Nothing to do with having 32-bit pointers.

Cheers,
Ben.