Status of 64K pagesize support
What is the status of support for 64K pagesizes in IA64. Is anyone actively working this??
I built a 2.4.2 kernel with 64K pages & immediately hit 2 problems:
- gcc doesnt support 64K alignment (perhaps I am using an old gcc??)
- drivers/scsi/scsi_dma.c has a bitmap
that overflows with 64K pages.
I dont mind working on these issues but first wanted to check if anyone else is working on them.
What other issues are people aware of that are lurking.....
Signature · 3 lines
Thanks Jack Steiner (651-683-5302) (vnet 233-5302) [email protected]
Jack-
I was working on this and discovered the same two problems you ran into. The alignment issue isn't really a problem, turns out that where that directive is give the code is appropriately aligned already so just remove the line should be OK.
The bitmap in `scsi_dma.c' is the real issue and needs to be re-written in a more general fashion. I don't really have time to work on this right now so if you're interested go for it. I'd be interested in hearing about anything you find out.
Quoted 30 lines · Jack Steiner
What is the status of support for 64K pagesizes in IA64. Is anyone actively working this??
I built a 2.4.2 kernel with 64K pages & immediately hit 2 problems:
- gcc doesnt support 64K alignment (perhaps I am using an old gcc??)
-
drivers/scsi/scsi_dma.chas a bitmap
that overflows with 64K pages.I dont mind working on these issues but first wanted to check if anyone else is working on them.
What other issues are people aware of that are lurking.....
Signature · 9 lines
Thanks Jack Steiner (651-683-5302) (vnet 233-5302) [email protected] _______________________________________________ Linux-IA64 mailing list [email protected] http://lists.linuxia64.org/lists/listinfo/linux-ia64
Signature · 4 lines
Don Dugger "Censeo Toto nos in Kansa esse decisse." - D. Gale [email protected] Ph: 303/938-9838
Don Dugger of VA has looked at 64K page support.
- gcc doesnt support 64K alignment (perhaps I am using an old gcc??)
Don reported an assembler problem. Is that perhaps what you meant? In gas/read.c, function s_align, there is a check
if (align > 15)
{
align = 15;
as_bad (_("Alignment too large: %u assumed"), align);
}I don't know why it is there. I would hypothesize limitations of old object file formats like aout/coff. This problem has been reported on the [email protected] mailing list, but no one there volunteered to look into the problem. This is still on my todo list, along with a lot of more important stuff.
Jim
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Jack Steiner
Jack> What is the status of support for 64K pagesizes in IA64. Is Jack> anyone actively working this??
Jack> I built a 2.4.2 kernel with 64K pages & immediately hit 2 Jack> problems:
Jack> - gcc doesnt support 64K alignment (perhaps I am using
Jack> an old gcc??)
Jack> - drivers/scsi/scsi_dma.c has a bitmap that overflows
Jack> with 64K pages.
Jack> I dont mind working on these issues but first wanted to check Jack> if anyone else is working on them.
Jack> What other issues are people aware of that are lurking.....
There shouldn't be any. The VM system should handle it just fine.
--david
Quoted 8 lines
Jack-
I was working on this and discovered the same two problems you ran into. The alignment issue isn't really a problem, turns out that where that directive is give the code is appropriately aligned already so just remove the line should be OK.
Quoted 5 lines
The bitmap in `scsi_dma.c' is the real issue and needs to be re-written in a more general fashion. I don't really have time to work on this right now so if you're interested go for it. I'd be interested in hearing about anything you find out.
I took a look at scsi_dma.c. As you pointed out, changing it to work with 64K pages overflows the bitmap/page (128 bits).
Fixing this is a little more than I wanted to bite off right now so I took an interim approach that may be "good enough" for a while - at least until someone has more time to invest here.
It is relatively simple to change scsi_dma.c so that
1) it uses longs to represent the free bit map for a page
2) each bit represents 1024 bytes instead of 512 bytes.
A request to allocate 512 bytes will actually get a 1024 byte chunk.
With this change (& a bunch more), I have been able to boot & run numerous tests with 64K pages. The only thing not working yet is swapping (next on my list).
Do you feel that this is an acceptible interim solution.
The scsi_dma.c. patch is appended:
Signature · 124 lines
Thanks Jack Steiner (651-683-5302) (vnet 233-5302) [email protected] diff -Naur linux_base/drivers/scsi/scsi_dma.c linux/drivers/scsi/scsi_dma.c --- linux_base/drivers/scsi/scsi_dma.c Mon Mar 12 13:17:38 2001 +++ linux/drivers/scsi/scsi_dma.c Tue Mar 20 16:35:27 2001 @@ -23,13 +23,25 @@ * PAGE_SIZE must be a multiple of the sector size (512). True * for all reasonably recent architectures (even the VAX...). */ -#define SECTOR_SIZE 512 +#ifdef CONFIG_IA64_PAGE_SIZE_64KB +#define BIG_SECTORS +#endif + +#ifdef BIG_SECTORS +#define SECTOR_SHIFT 10 +#else +#define SECTOR_SHIFT 9 +#endif + +#define SECTOR_SIZE (1<<SECTOR_SHIFT) #define SECTORS_PER_PAGE (PAGE_SIZE/SECTOR_SIZE) #if SECTORS_PER_PAGE <= 8 typedef unsigned char FreeSectorBitmap; #elif SECTORS_PER_PAGE <= 32 typedef unsigned int FreeSectorBitmap; +#elif SECTORS_PER_PAGE <= 64 +typedef unsigned long FreeSectorBitmap; #else #error You lose. #endif @@ -75,10 +87,14 @@ unsigned long flags; int i, j; +#ifdef BIG_SECTORS + if (len % SECTOR_SIZE) + len += 512; +#endif if (len % SECTOR_SIZE != 0 || len > PAGE_SIZE) return NULL; - nbits = len >> 9; + nbits = len >> SECTOR_SHIFT; mask = (1 << nbits) - 1; spin_lock_irqsave(&allocator_request_lock, flags); @@ -89,11 +105,11 @@ dma_malloc_freelist[i] |= (mask << j); scsi_dma_free_sectors -= nbits; #ifdef DEBUG - SCSI_LOG_MLQUEUE(3, printk("SMalloc: %d %p [From:%p]\n", len, dma_malloc_pages[i] + (j << 9))); - printk("SMalloc: %d %p [From:%p]\n", len, dma_malloc_pages[i] + (j << 9)); + SCSI_LOG_MLQUEUE(3, printk("SMalloc: %d %p [From:%p]\n", len, dma_malloc_pages[i] + (j << SECTOR_SHIFT))); + printk("SMalloc: %d %p [From:%p]\n", len, dma_malloc_pages[i] + (j << SECTOR_SHIFT)); #endif spin_unlock_irqrestore(&allocator_request_lock, flags); - return (void *) ((unsigned long) dma_malloc_pages[i] + (j << 9)); + return (void *) ((unsigned long) dma_malloc_pages[i] + (j << SECTOR_SHIFT)); } } spin_unlock_irqrestore(&allocator_request_lock, flags); @@ -136,15 +152,19 @@ SCSI_LOG_MLQUEUE(3, printk("SFree: %p %d\n", obj, len)); #endif +#ifdef BIG_SECTORS + if (len % SECTOR_SIZE) + len += 512; +#endif spin_lock_irqsave(&allocator_request_lock, flags); for (page = 0; page < dma_sectors / SECTORS_PER_PAGE; page++) { unsigned long page_addr = (unsigned long) dma_malloc_pages[page]; if ((unsigned long) obj >= page_addr && (unsigned long) obj < page_addr + PAGE_SIZE) { - sector = (((unsigned long) obj) - page_addr) >> 9; + sector = (((unsigned long) obj) - page_addr) >> SECTOR_SHIFT; - nbits = len >> 9; + nbits = len >> SECTOR_SHIFT; mask = (1 << nbits) - 1; if (sector + nbits > SECTORS_PER_PAGE) @@ -254,27 +274,27 @@ if (nents < 64) nents = 64; #endif new_dma_sectors += ((nents * - sizeof(struct scatterlist) + 511) >> 9) * + sizeof(struct scatterlist) + SECTOR_SIZE-1) >> SECTOR_SHIFT) * SDpnt->queue_depth; if (SDpnt->type = TYPE_WORM || SDpnt->type = TYPE_ROM) - new_dma_sectors += (2048 >> 9) * SDpnt->queue_depth; + new_dma_sectors += (2048 >> SECTOR_SHIFT) * SDpnt->queue_depth; } else if (SDpnt->type = TYPE_SCANNER || SDpnt->type = TYPE_PROCESSOR || SDpnt->type = TYPE_COMM || SDpnt->type = TYPE_MEDIUM_CHANGER || SDpnt->type = TYPE_ENCLOSURE) { - new_dma_sectors += (4096 >> 9) * SDpnt->queue_depth; + new_dma_sectors += (4096 >> SECTOR_SHIFT) * SDpnt->queue_depth; } else { if (SDpnt->type != TYPE_TAPE) { printk("resize_dma_pool: unknown device type %d\n", SDpnt->type); - new_dma_sectors += (4096 >> 9) * SDpnt->queue_depth; + new_dma_sectors += (4096 >> SECTOR_SHIFT) * SDpnt->queue_depth; } } if (host->unchecked_isa_dma && need_isa_bounce_buffers && SDpnt->type != TYPE_TAPE) { - new_dma_sectors += (PAGE_SIZE >> 9) * host->sg_tablesize * + new_dma_sectors += (PAGE_SIZE >> SECTOR_SHIFT) * host->sg_tablesize * SDpnt->queue_depth; new_need_isa_buffer++; }
I have built a system with 64K pages.
As long as you do not try mount swap devices, everything seems to be running ok. (Swapping to files works OK).
I ran fairly heavy stress for a couple hours
and did not see any failures. More testing is still needed, however.
There are still a few issues with swapping to devices. The buffer_head field "b_size" is a short, the kernel tries to do 64K IO & overflows the field. (I'm assuming that changing b_size to an "int" is NOT acceptible).
I'm working on fixing this but thought I'd send these fixes out now since they enable others to use 64K pages.
Here is a summary of the changes I've made so far:
include/asm-ia64/system.h
add BOOT_PARAM_ADDR. If pagesize is 64K, BOOT_PARAM_ADDR. is not the same as ZERO_PAGE_ADDR.
arch/ia64/boot/bootloader.c
change name of ZERO_PAGE_ADDR to BOOT_PARAM_ADDR
arch/ia64/kernel/fw-emu.c
change name of ZERO_PAGE_ADDR to BOOT_PARAM_ADDR
arch/ia64/kernel/gate.S
delete ".align PAGE_SIZE". Code is already correctly aligned & gcc doesnt support alignment > 16k
arch/ia64/kernel/ivt.S
add alignment pragmas to the end to make the size of the IVT an integral number of pages.
arch/ia64/kernel/setup.c
change name of ZERO_PAGE_ADDR to BOOT_PARAM_ADDR
arch/ia64/sn/fprom/fw-emu.c
change name of ZERO_PAGE_ADDR to BOOT_PARAM_ADDR
drivers/scsi/scsi_dma.c
change the FreeSectorBitmap so that on 64k page systems, a bit in the map represents 1K instead of 512 bytes. Otherwise, you overflow the long used to manage free space in a page. I view this as an interim fix..... (I swapped mail with dugger about this patch).
include/asm-ia64/a.out.h
dont make STACK_TOP bigger than the max virtual address supported on itanium.
arch/ia64/boot/bootloader.c +3 -3
@@ -224,11 +224,11 @@ssc (0 , (long ) kpath,0 ,0 ,SSC_LOAD_SYMBOLS );/* -* Install the kernel's command line argument on ZERO_PAGE -* just after the botoparam structure. +* Install the kernel's command line argument at BOOT_PARAM_ADDR +* just after the bootparam structure. * In case we don't have any argument just put \0 */ -memcpy (((struct ia64_boot_param *)ZERO_PAGE_ADDR ) +1 , args, arglen); +memcpy (((struct ia64_boot_param *)BOOT_PARAM_ADDR ) +1 , args, arglen); sp =__pa (&stack); asmvolatile ("br.sptk.few %0" ::"b" (e_entry));
arch/ia64/kernel/fw-emu.c +1 -1
@@ -469,7 +469,7 @@ md->attribute =EFI_MEMORY_WB ;#endif - bp =id (ZERO_PAGE_ADDR ); + bp =id (BOOT_PARAM_ADDR ); bp->efi_systab =__pa (&fw_mem); bp->efi_memmap =__pa (efi_memmap); bp->efi_memmap_size =NUM_MEM_DESCS *sizeof (efi_memory_desc_t);
arch/ia64/kernel/gate.S +0 -2
@@ -19,8 +19,6 @@ .section .text.gate,"ax" - .align PAGE_SIZE - # define SIGINFO_OFF 16 # define SIGCONTEXT_OFF (SIGINFO_OFF + ((IA64_SIGINFO_SIZE + 15) & ~15)) # define FLAGS_OFF IA64_SIGCONTEXT_FLAGS_OFFSET
arch/ia64/kernel/ivt.S +17 -0
@@ -1357,3 +1357,20 @@ ///////////////////////////////////////////////////////////////////////////////////////// // 0x7f00 Entry 67 (size 16 bundles) Reserved FAULT(67) + + +// +// The end of the IVT must be aligned at a page boundary so that the structures +// that follow it are also aligned on page boundaries. We would like to use the +// gcc .align pragma but that doesnt work for sizes greater than 16K alignment +// and we may need 64K alignment. For now, the following works. If/when gcc is +// fixed, this should be revisited..... +// +#ifdef CONFIG_IA64_PAGE_SIZE_64KB + .align 16384 + nop 0;; + .align 16384 + nop 0;; + .align 16384 +#endif +
arch/ia64/kernel/setup.c +3 -2
@@ -143,10 +143,11 @@/* * The secondary bootstrap loader passes us the boot -* parameters at the beginning of the ZERO_PAGE, so let's +* parameters at the end of the 32K IVT. Depending on pagesize, +* this may be the location of the ZERO_PAGE, so let's * stash away those values before ZERO_PAGE gets cleared out. */ -memcpy (&ia64_boot_param, (void *)ZERO_PAGE_ADDR ,sizeof (ia64_boot_param)); +memcpy (&ia64_boot_param, (void *)BOOT_PARAM_ADDR ,sizeof (ia64_boot_param)); *cmdline_p =__va (ia64_boot_param.command_line);strncpy (saved_command_line, *cmdline_p,sizeof (saved_command_line));
arch/ia64/sn/fprom/fw-emu.c +1 -1
@@ -494,7 +494,7 @@ md = &efi_memmap[0 ]; num_memmd =build_efi_memmap ((void *)md, mdsize) ; - bp =id (ZERO_PAGE_ADDR + (((long )base_nasid)<<33 )); + bp =id (BOOT_PARAM_ADDR + (((long )base_nasid)<<33 )); bp->efi_systab =__fwtab_pa (base_nasid, &fw_mem); bp->efi_memmap =__fwtab_pa (base_nasid, efi_memmap); bp->efi_memmap_size = num_memmd*mdsize;
drivers/scsi/scsi_dma.c +32 -12
108 lines not on this page · raw message
include/asm-ia64/a.out.h +4 -0
@@ -30,7 +30,11 @@#define N_TXTOFF (x) 0#ifdef __KERNEL__ +#ifdef CONFIG_IA64_PAGE_SIZE_64KB +# define STACK_TOP (0x8000000000000000UL + (1UL << 50)) +#else # define STACK_TOP (0x8000000000000000UL + (1UL << (4*PAGE_SHIFT - 12))) +#endif # define IA64_RBS_BOT (STACK_TOP - 0x80000000L)/* bottom of register backing store */ #endif
include/asm-ia64/system.h +9 -3
@@ -17,13 +17,19 @@#include <asm/page.h> #define KERNEL_START (PAGE_OFFSET + 0x500000) +#define BOOT_PARAM_ADDR (KERNEL_START + 0x8000)/* * The following #defines must match with vmlinux.lds.S: */ -#define IVT_END_ADDR (KERNEL_START + 0x8000) -#define ZERO_PAGE_ADDR (IVT_END_ADDR + 0*PAGE_SIZE) -#define SWAPPER_PGD_ADDR (IVT_END_ADDR + 1*PAGE_SIZE) +#ifdef CONFIG_IA64_PAGE_SIZE_64KB +#define IVT_PAGE_END_ADDR (KERNEL_START + 0x10000) +#else +#define IVT_PAGE_END_ADDR (KERNEL_START + 0x8000) +#endif + +#define ZERO_PAGE_ADDR (IVT_PAGE_END_ADDR + 0*PAGE_SIZE) +#define SWAPPER_PGD_ADDR (IVT_PAGE_END_ADDR + 1*PAGE_SIZE)#define GATE_ADDR (0xa000000000000000 + PAGE_SIZE)#define PERCPU_ADDR (0xa000000000000000 + 2*PAGE_SIZE)
Signature · 3 lines
Thanks Jack Steiner (651-683-5302) (vnet 233-5302) [email protected]
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Jack Steiner
Jack> Here is a summary of the changes I've made so far:
Jack> include/asm-ia64/system.h add BOOT_PARAM_ADDR. If pagesize
Jack> is 64K, BOOT_PARAM_ADDR. is not the same as ZERO_PAGE_ADDR.
OK, sounds reasonable. I made that change.
Jack> arch/ia64/boot/bootloader.c change name of ZERO_PAGE_ADDR to
Jack> BOOT_PARAM_ADDR
OK.
Jack> arch/ia64/kernel/fw-emu.c change name of ZERO_PAGE_ADDR to
Jack> BOOT_PARAM_ADDR
OK.
Jack> arch/ia64/kernel/gate.S delete ".align PAGE_SIZE". Code is
Jack> already correctly aligned & gcc doesnt support alignment > 16k
I don't want to do that. The .align is there to make it clear that the code must be page aligned. We should fix gas instead.
Jack> arch/ia64/kernel/ivt.S add alignment pragmas to the end to
Jack> make the size of the IVT an integral number of pages.
I don't really like that. The ivt shouldn't have to care what the page size is. I put an alignment directive in the linker script instead.
Jack> arch/ia64/kernel/setup.c change name of ZERO_PAGE_ADDR to
Jack> BOOT_PARAM_ADDR
OK.
Jack> arch/ia64/sn/fprom/fw-emu.c change name of ZERO_PAGE_ADDR to
Jack> BOOT_PARAM_ADDR
OK.
Jack> drivers/scsi/scsi_dma.c change the FreeSectorBitmap so that
Jack> on 64k page systems, a bit in the map represents 1K instead of Jack> 512 bytes. Otherwise, you overflow the long used to manage Jack> free space in a page. I view this as an interim fix..... (I Jack> swapped mail with dugger about this patch).
I'll wait with this one until there is a permanent fix.
Jack> include/asm-ia64/a.out.h dont make STACK_TOP bigger than the
Jack> max virtual address supported on itanium.
Ditto here.
--david