Skip to contentSkip to conversation

Get ride of CPHYSADDR() in setup.c

PATCH In review Franck Bui-Huu 3 participants 20 messages Oct 11, 2006 – Oct 13, 2006 MIPS

lore
Share & tools
lore
#0 lore raw

cover Get ride of CPHYSADDR() in setup.c

Indeed there's no point to have 2 macros __pa()/CPHYSADDR() that do exactly the same thing that is convert a virtual address into physical one at early boot.

So this patch simply removes CPHYSADDR() usages and make __pa() aware of 64 bit kernels with CONFIG_BUILD_ELF64=n constraint.

At the same time, it also removes all hacks related to the previous constraint and initrd.

Unfortunately, no tests have been done for 64 bit kernels because of lacks of hardware. If anybody could give a try that would be great.

Please consider,

Franck
---

 arch/mips/kernel/setup.c |   53 ++++++++++++++++++++++++++--------------------
 arch/mips/mm/init.c      |    5 ----
 include/asm-mips/page.h  |    7 +++++-
 3 files changed, 36 insertions(+), 29 deletions(-)
#1 lore raw

PATCH 1/5 Make __pa() uses CPHYSADDR() if really needed

During early boot mem init, some configs can't use __pa() to convert virtual into physical addresses. This patch make __pa() uses CPHYSADDR() for these configs only, others don't need this hack. This will allow to not use anymore CPHYSADDR() in generic code.

---
 include/asm-mips/page.h |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)
include/asm-mips/page.h +5 -1
index 32e5625..b4851ac 100644
@@ -131,7 +131,11 @@ #endif /* !__ASSEMBLY__ */
 /* to align the pointer to the (next) page boundary */
 #define PAGE_ALIGN(addr)	(((addr) + PAGE_SIZE - 1) & PAGE_MASK)
 
-#define __pa(x)			((unsigned long) (x) - PAGE_OFFSET)
+#if defined(CONFIG_64BITS) && !defined(CONFIG_BUILD_ELF64)
+#define __pa(x)			CPHYSADDR(x)
+#else
+#define __pa(x)			((unsigned long)(x) - PAGE_OFFSET)
+#endif
 #define __va(x)			((void *)((unsigned long) (x) + PAGE_OFFSET))
 
 #define pfn_to_kaddr(pfn)	__va((pfn) << PAGE_SHIFT)
Signature · 1 line
1.4.2.3
#2 lore raw

Re: [PATCH 1/5] Make __pa() uses CPHYSADDR() if really needed

include/asm-mips/page.h #endif /* !__ASSEMBLY__ */ · line 139

Quoted 7 lines · Franck Bui-Huu
include/asm-mips/page.h +5 -1
-#define __pa(x)			((unsigned long) (x) - PAGE_OFFSET)
+#if defined(CONFIG_64BITS) && !defined(CONFIG_BUILD_ELF64)
+#define __pa(x)			CPHYSADDR(x)
+#else
+#define __pa(x)			((unsigned long)(x) - PAGE_OFFSET)
+#endif
 #define __va(x)			((void *)((unsigned long) (x) + PAGE_OFFSET))

Please do not do this. CONFIG_BUILD_ELF64=n does not mean we only have less then 512MB memory. We can have large flat area at PAGE_OFFSET (0x9800000000000000) in 64-bit kernel, so __pa() should accepct a value such as 0x9800000020000000.

---
Atsushi Nemoto

#3 lore raw

Re: [PATCH 1/5] Make __pa() uses CPHYSADDR() if really needed

Hi Atsushi,

Quoted 14 lines · Atsushi Nemoto

include/asm-mips/page.h #endif /* !__ASSEMBLY__ */ · line 139

Quoted 7 lines · Franck Bui-Huu
include/asm-mips/page.h +5 -1
-#define __pa(x)			((unsigned long) (x) - PAGE_OFFSET)
+#if defined(CONFIG_64BITS) && !defined(CONFIG_BUILD_ELF64)
+#define __pa(x)			CPHYSADDR(x)
+#else
+#define __pa(x)			((unsigned long)(x) - PAGE_OFFSET)
+#endif
 #define __va(x)			((void *)((unsigned long) (x) + PAGE_OFFSET))

Please do not do this. CONFIG_BUILD_ELF64=n does not mean we only have less then 512MB memory. We can have large flat area at PAGE_OFFSET (0x9800000000000000) in 64-bit kernel, so __pa() should accepct a value such as 0x9800000020000000.

To deal with such address there's virt_to_phys() routine. __pa() is normally used during early bootmem init (as CPHYSADDR(), BTW). At early bootmem, we normally deal with address in KSEG0. Once we have switched to XKPHYS address, I agree we should use virt_to_phys().

If we look at how to convert a virtual address into a physical one, we have:

	CPHYSADDR()
	__pa()
	virt_to_phys()

What definition would you give to each of them ?

BTW, if you grep for __pa() you'll notice that it's almost not used by the kernel. I suspect that's because of CPHYSADDR() existence which is really confusing.

Franck

#4 lore raw

Re: [PATCH 1/5] Make __pa() uses CPHYSADDR() if really needed

Quoted 8 lines · Franck Bui-Huu

If we look at how to convert a virtual address into a physical one, we have:

	CPHYSADDR()
	__pa()
	virt_to_phys()

What definition would you give to each of them ?

Here's my understanding.

CPHYSADDR --- used to convert a CKSEG0 or CKSEG1 to a physical address. Be carefull when use it in 64-bit kernel.

__pa() --- inverse of __va(), of course :-) used to convert a kernel linear logical address to a physycal address.

virt_to_phys() --- synonym of __pa() ?

Now I think passing kernel symbol address to __pa() is wrong. This is implied by DMA-mapping.txt:

This rule also means that you may use neither kernel image addresses (items in data/text/bss segments), nor module image addresses, nor stack addresses for DMA. These could all be mapped somewhere entirely different than the rest of physical memory. Even if those classes of
...

Maybe introducing __pa_symbol() is better as your patch tried.

Quoted 3 lines

BTW, if you grep for __pa() you'll notice that it's almost not used by the kernel. I suspect that's because of CPHYSADDR() existence which is really confusing.

__pa() is used in many place indirectly via virt_to_page().

---
Atsushi Nemoto

#5 lore raw

Re: [PATCH 1/5] Make __pa() uses CPHYSADDR() if really needed

Quoted 9 lines · Atsushi Nemoto

Here's my understanding.

CPHYSADDR --- used to convert a CKSEG0 or CKSEG1 to a physical address. Be carefull when use it in 64-bit kernel.

__pa() --- inverse of __va(), of course :-) used to convert a kernel linear logical address to a physycal address.

virt_to_phys() --- synonym of __pa() ?

Well in my understanding __pa() is used during bootmem init. It deals with initialisation issues and ugliness and that's what CPHYSADDR() actually does. virt_to_phys() is used once everything is correctly setup.

Quoted 2 lines

__pa() is used in many place indirectly via virt_to_page().

what about make virt_to_page() use virt_to_phys() instead ?

Franck

#6 lore raw

Re: [PATCH 1/5] Make __pa() uses CPHYSADDR() if really needed

Quoted 11 lines · Franck Bui-Huu
Quoted 9 lines · Atsushi Nemoto

Here's my understanding.

CPHYSADDR --- used to convert a CKSEG0 or CKSEG1 to a physical address. Be carefull when use it in 64-bit kernel.

__pa() --- inverse of __va(), of course :-) used to convert a kernel linear logical address to a physycal address.

virt_to_phys() --- synonym of __pa() ?

Quoted 9 lines

Well in my understanding __pa() is used during bootmem init. It deals with initialisation issues and ugliness and that's what CPHYSADDR() actually does. virt_to_phys() is used once everything is correctly setup.

Quoted 2 lines

__pa() is used in many place indirectly via virt_to_page().

what about make virt_to_page() use virt_to_phys() instead ?

Actually that would make sense. virt_to_page() returns a page which means that all kernel allocation machinery is setup. It should be safe to use phys_to_virt() at that time.

Signature · 1 line
              Franck
#7 lore raw

Re: [PATCH 1/5] Make __pa() uses CPHYSADDR() if really needed

In reply to Franck Bui-Huu
Quoted 3 lines · Franck Bui-Huu
Quoted 1 line

__pa() is used in many place indirectly via virt_to_page().

what about make virt_to_page() use virt_to_phys() instead ?

No objection for virt_to_phys(), but I found other __pa() usages in kernel.

drivers/char/mem.c:       && addr >= __pa(high_memory);
drivers/char/mem.c:     return addr >= __pa(high_memory);
drivers/char/mem.c:     if (addr + count > __pa(high_memory))
drivers/char/mem.c:     pfn = __pa((u64)vma->vm_pgoff << PAGE_SHIFT) >> PAGE_SHIFT;

So __pa() should handle >512MB address unless we change all __pa() usages.

How about something like this ?

#if defined(CONFIG_64BITS) && !defined(CONFIG_BUILD_ELF64)
#define __pa(x) ((unsigned long)(x) - ((unsigned long)(x) >= CKSEG0 ? CKSEG0 : PAGE_OFFSET))
#else
#define __pa(x) ((unsigned long)(x) - PAGE_OFFSET)
#endif

In any case, I think virt_to_page() should use simpler virt_to_phys() for better performance.

---
Atsushi Nemoto

#8 lore raw

Re: [PATCH 1/5] Make __pa() uses CPHYSADDR() if really needed

Quoted 7 lines · Atsushi Nemoto
Quoted 2 lines · Franck Bui-Huu
Quoted 1 line

__pa() is used in many place indirectly via virt_to_page().

what about make virt_to_page() use virt_to_phys() instead ?

No objection for virt_to_phys(), but I found other __pa() usages in kernel.

Quoted 6 lines
drivers/char/mem.c:       && addr >= __pa(high_memory);
drivers/char/mem.c:     return addr >= __pa(high_memory);
drivers/char/mem.c:     if (addr + count > __pa(high_memory))
drivers/char/mem.c:     pfn = __pa((u64)vma->vm_pgoff << PAGE_SHIFT) >> PAGE_SHIFT;

So __pa() should handle >512MB address unless we change all __pa() usages.

weird, I really thought that __pa() shouldn't be used by drivers... Let see what people on lkml think about this usage, you'll be CC'ed if you want to.

Quoted 9 lines

How about something like this ?

#if defined(CONFIG_64BITS) && !defined(CONFIG_BUILD_ELF64)
#define __pa(x) ((unsigned long)(x) - ((unsigned long)(x) >= CKSEG0 ? CKSEG0 : PAGE_OFFSET))
#else
#define __pa(x) ((unsigned long)(x) - PAGE_OFFSET)
#endif

It would be safer indeed, specially because __pa() definition is not really well defined and seems to be confusing for most people.

Quoted 2 lines

In any case, I think virt_to_page() should use simpler virt_to_phys() for better performance.

I agree.

Franck

#9 lore raw

Re: [PATCH 1/5] Make __pa() uses CPHYSADDR() if really needed

Quoted 13 lines · Franck Bui-Huu
Quoted 12 lines · Atsushi Nemoto
Quoted 2 lines · Franck Bui-Huu
Quoted 1 line

__pa() is used in many place indirectly via virt_to_page().

what about make virt_to_page() use virt_to_phys() instead ?

No objection for virt_to_phys(), but I found other __pa() usages in kernel.

drivers/char/mem.c:       && addr >= __pa(high_memory);
drivers/char/mem.c:     return addr >= __pa(high_memory);
drivers/char/mem.c:     if (addr + count > __pa(high_memory))
drivers/char/mem.c:     pfn = __pa((u64)vma->vm_pgoff << PAGE_SHIFT) >> PAGE_SHIFT;
Quoted 5 lines
Quoted 1 line

So __pa() should handle >512MB address unless we change all __pa() usages.

weird, I really thought that __pa() shouldn't be used by drivers... Let see what people on lkml think about this usage, you'll be CC'ed if you want to.

I guess /dev/mem is allowed to as an exception.

Thiemo

#10 lore raw

PATCH 2/5 setup.c: get ride of CPHYSADDR()

In reply to Franck Bui-Huu

and use __pa() instead. Indeed this macro can be used even by the 64 bit kernels with CONFIG_BUILD_ELF64=n config.

---
 arch/mips/kernel/setup.c |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)
arch/mips/kernel/setup.c +3 -3
index fdbb508..00d62bd 100644
@@ -204,12 +204,12 @@ static void __init finalize_initrd(void)
 		printk(KERN_INFO "Initrd not found or empty");
 		goto disable;
 	}
-	if (CPHYSADDR(initrd_end) > PFN_PHYS(max_low_pfn)) {
+	if (__pa(initrd_end) > PFN_PHYS(max_low_pfn)) {
 		printk("Initrd extends beyond end of memory");
 		goto disable;
 	}
 
-	reserve_bootmem(CPHYSADDR(initrd_start), size);
+	reserve_bootmem(__pa(initrd_start), size);
 	initrd_below_start_ok = 1;
 
 	printk(KERN_INFO "Initial ramdisk at: 0x%lx (%lu bytes)\n",
@@ -256,7 +256,7 @@ static void __init bootmem_init(void)
 	 * of usable memory.
 	 */
 	reserved_end = init_initrd();
-	reserved_end = PFN_UP(CPHYSADDR(max(reserved_end, (unsigned long)&_end)));
+	reserved_end = PFN_UP(__pa(max(reserved_end, (unsigned long)&_end)));
 
 	/*
 	 * Find the highest page frame number we have available.
Signature · 1 line
1.4.2.3
#11 lore raw

PATCH 3/5 setup.c: clean up initrd related code

In reply to Franck Bui-Huu
---
 arch/mips/kernel/setup.c |   41 ++++++++++++++++++++++++-----------------
 arch/mips/mm/init.c      |    5 -----
 2 files changed, 24 insertions(+), 22 deletions(-)
arch/mips/kernel/setup.c +24 -17
index 00d62bd..0e92b9b 100644
@@ -144,14 +144,12 @@ static int __init rd_start_early(char *p
 {
 	unsigned long start = memparse(p, &p);
 
-#ifdef CONFIG_64BIT
-	/* HACK: Guess if the sign extension was forgotten */
-	if (start > 0x0000000080000000 && start < 0x00000000ffffffff)
-		start |= 0xffffffff00000000UL;
-#endif
+	/*
+	 * No sanity checkings are needed here, they're going to be
+	 * done by init_initrd() soon.
+	 */
 	initrd_start = start;
 	initrd_end += start;
-
 	return 0;
 }
 early_param("rd_start", rd_start_early);
@@ -159,14 +157,14 @@ early_param("rd_start", rd_start_early);
 static int __init rd_size_early(char *p)
 {
 	initrd_end += memparse(p, &p);
-
 	return 0;
 }
 early_param("rd_size", rd_size_early);
 
+/* it returns the next free pfn after initrd */
 static unsigned long __init init_initrd(void)
 {
-	unsigned long tmp, end, size;
+	unsigned long tmp, end;
 	u32 *initrd_header;
 
 	ROOT_DEV = Root_RAM0;
@@ -176,24 +174,34 @@ static unsigned long __init init_initrd(
 	 * already set up initrd_start and initrd_end. In these cases
 	 * perfom sanity checks and use them if all looks good.
 	 */
-	size = initrd_end - initrd_start;
-	if (initrd_end == 0 || size == 0) {
-		initrd_start = 0;
-		initrd_end = 0;
-	} else
-		return initrd_end;
+	if (initrd_end > initrd_start)
+		goto sanitize;
 
 	end = (unsigned long)&_end;
 	tmp = PAGE_ALIGN(end) - sizeof(u32) * 2;
 	if (tmp < end)
 		tmp += PAGE_SIZE;
 
+	initrd_start = 0;
+	initrd_end = 0;
+	end = 0;
 	initrd_header = (u32 *)tmp;
 	if (initrd_header[0] == 0x494E5244) {
 		initrd_start = (unsigned long)&initrd_header[2];
 		initrd_end = initrd_start + initrd_header[1];
+sanitize:
+		/*
+		 * Sanitize initrd addresses. For example firmware
+		 * can't guess if they need to pass them through
+		 * 64-bits values if the kernel has been built in pure
+		 * 32-bit. We need also to switch from KSEG0 to XKPHYS
+		 * addresses now, so the code can now safely use __pa().
+		 */
+		end = __pa(initrd_end);
+		initrd_end = (unsigned long)__va(end);
+		initrd_start = (unsigned long)__va(__pa(initrd_start));
 	}
-	return initrd_end;
+	return PFN_UP(end);
 }
 
 static void __init finalize_initrd(void)
@@ -255,8 +263,7 @@ static void __init bootmem_init(void)
 	 * not selected. Once that done we can determine the low bound
 	 * of usable memory.
 	 */
-	reserved_end = init_initrd();
-	reserved_end = PFN_UP(__pa(max(reserved_end, (unsigned long)&_end)));
+	reserved_end = max(init_initrd(), PFN_UP(__pa(&_end)));
 
 	/*
 	 * Find the highest page frame number we have available.
arch/mips/mm/init.c +0 -5
index 2f346d1..1ec0f22 100644
@@ -502,11 +502,6 @@ void free_init_pages(char *what, unsigne
 #ifdef CONFIG_BLK_DEV_INITRD
 void free_initrd_mem(unsigned long start, unsigned long end)
 {
-#ifdef CONFIG_64BIT
-	/* Switch from KSEG0 to XKPHYS addresses */
-	start = (unsigned long)phys_to_virt(CPHYSADDR(start));
-	end = (unsigned long)phys_to_virt(CPHYSADDR(end));
-#endif
 	free_init_pages("initrd memory", start, end);
 }
 #endif
Signature · 1 line
1.4.2.3
#12 lore raw

Re: [PATCH 3/5] setup.c: clean up initrd related code

Quoted 2 lines · Franck Bui-Huu

---

[snip]

Quoted 1 line
(quoted diff) +0 -0
@@ -176,24 +174,34 @@ static unsigned long __init init_initrd(

[snip]

arch/mips/kernel/setup.c static unsigned long __init init_initrd( · line 183

Quoted 5 lines
arch/mips/kernel/setup.c +0 -0
 	end = (unsigned long)&_end;
 	tmp = PAGE_ALIGN(end) - sizeof(u32) * 2;
 	if (tmp < end)
 		tmp += PAGE_SIZE;

BTW, can anybody tell me what that bit of code is for ? It seems that a minumum gap is needed betweend the end of kernel code and initrd but I don't see why ?

Thanks
Franck

#13 lore raw

PATCH 4/5 Introduce __pa_symbol()

In reply to Franck Bui-Huu

This patch introduces __pa_symbol() macro which should be used to calculate the physical address of kernel symbols. It also relies on RELOC_HIDE() to avoid any compiler's oddities when doing arithmetics on symbols.

---
 include/asm-mips/page.h |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
include/asm-mips/page.h +1 -0
index b4851ac..2bf101f 100644
@@ -136,6 +136,7 @@ #define __pa(x)			CPHYSADDR(x)
 #else
 #define __pa(x)			((unsigned long)(x) - PAGE_OFFSET)
 #endif
+#define __pa_symbol(x)		__pa(RELOC_HIDE((unsigned long)(x),0))
 #define __va(x)			((void *)((unsigned long) (x) + PAGE_OFFSET))
 
 #define pfn_to_kaddr(pfn)	__va((pfn) << PAGE_SHIFT)
Signature · 1 line
1.4.2.3
#14 lore raw

Re: [PATCH 4/5] Introduce __pa_symbol()

Quoted 4 lines · Franck Bui-Huu

This patch introduces __pa_symbol() macro which should be used to calculate the physical address of kernel symbols. It also relies on RELOC_HIDE() to avoid any compiler's oddities when doing arithmetics on symbols.

I agree with you that we need __pa_symbol(), but what is a purpose of using RELOC_HIDE() here? Frankly I do not understand what RELOC_HIDE() does...

---
Atsushi Nemoto

#15 lore raw

Re: [PATCH 4/5] Introduce __pa_symbol()

Quoted 10 lines · Atsushi Nemoto
Quoted 4 lines · Franck Bui-Huu

This patch introduces __pa_symbol() macro which should be used to calculate the physical address of kernel symbols. It also relies on RELOC_HIDE() to avoid any compiler's oddities when doing arithmetics on symbols.

I agree with you that we need __pa_symbol(), but what is a purpose of using RELOC_HIDE() here? Frankly I do not understand what RELOC_HIDE() does...

RELOC_HIDE(x) is used because arithmetic on symbol addresses is undefined in C language. It avoid gcc to know that:

RELOC_HIDE(&_end) + OFFSET

is an operation on a symbol address and thus avoid an undefined operation.

Franck

#16 lore raw

Re: [PATCH 4/5] Introduce __pa_symbol()

Quoted 7 lines · Franck Bui-Huu

RELOC_HIDE(x) is used because arithmetic on symbol addresses is undefined in C language. It avoid gcc to know that:

RELOC_HIDE(&_end) + OFFSET

is an operation on a symbol address and thus avoid an undefined operation.

Thanks, I see, but can not imagine the case the RELOC_HIDE() is _really_ needed. Do you have any example?

---
Atsushi Nemoto

#17 lore raw

Re: [PATCH 4/5] Introduce __pa_symbol()

Quoted 4 lines · Atsushi Nemoto
Quoted 3 lines · Franck Bui-Huu

RELOC_HIDE(x) is used because arithmetic on symbol addresses is undefined in C language. It avoid gcc to know that:

Quoted 8 lines
Quoted 4 lines

RELOC_HIDE(&_end) + OFFSET

is an operation on a symbol address and thus avoid an undefined operation.

Thanks, I see, but can not imagine the case the RELOC_HIDE() is _really_ needed. Do you have any example?

No I don't have any example but you can take a look to

http://lkml.org/lkml/2006/8/23/175

Franck

#18 lore raw

Re: [PATCH 4/5] Introduce __pa_symbol()

Quoted 3 lines · Franck Bui-Huu

No I don't have any example but you can take a look to

http://lkml.org/lkml/2006/8/23/175

Thanks! Reading the thread, I remember "near" vs. "far" pointer issue in DOS age ...

---
Atsushi Nemoto

#19 lore raw

PATCH 5/5 setup.c: use __pa_symbol() where needed

In reply to Franck Bui-Huu

It should fix the broken code in resource_init() too.

---
 arch/mips/kernel/setup.c |   10 +++++-----
 1 files changed, 5 insertions(+), 5 deletions(-)
arch/mips/kernel/setup.c +5 -5
index 0e92b9b..0316e04 100644
@@ -263,7 +263,7 @@ static void __init bootmem_init(void)
 	 * not selected. Once that done we can determine the low bound
 	 * of usable memory.
 	 */
-	reserved_end = max(init_initrd(), PFN_UP(__pa(&_end)));
+	reserved_end = max(init_initrd(), PFN_UP(__pa_symbol(&_end)));
 
 	/*
 	 * Find the highest page frame number we have available.
@@ -435,10 +435,10 @@ static void __init resource_init(void)
 	if (UNCAC_BASE != IO_BASE)
 		return;
 
-	code_resource.start = virt_to_phys(&_text);
-	code_resource.end = virt_to_phys(&_etext) - 1;
-	data_resource.start = virt_to_phys(&_etext);
-	data_resource.end = virt_to_phys(&_edata) - 1;
+	code_resource.start = __pa_symbol(&_text);
+	code_resource.end = __pa_symbol(&_etext) - 1;
+	data_resource.start = __pa_symbol(&_etext);
+	data_resource.end = __pa_symbol(&_edata) - 1;
 
 	/*
 	 * Request address space for all standard RAM.
Signature · 1 line
1.4.2.3