small sym-2 fix
PATCH 1/1 small sym-2 fix
Hi,
I tested the sym-2 driver on ppc64 and found that hcb_p can be > 1 page but __sym_malloc fails for allocations over 1 page. This means we die in sym_attach.
With this patch the sym-2 works on ppc64. BTW so far it looks solid :)
Anton
drivers/scsi/sym53c8xx_2/sym_glue.h +1 -1
@@ -526,7 +526,7 @@* couple of things related to the memory allocator. */ typedef u_long m_addr_t ;/* Enough bits to represent any address */ -#define SYM_MEM_PAGE_ORDER 0/* 1 PAGE maximum */ +#define SYM_MEM_PAGE_ORDER 1/* 2 PAGE maximum */ #define SYM_MEM_CLUSTER_SHIFT (PAGE_SHIFT+SYM_MEM_PAGE_ORDER)#ifdef MODULE #define SYM_MEM_FREE_UNUSED /* Free unused pages immediately */
From: Anton Blanchard <[email protected]>
Date: Thu, 15 Nov 2001 15:36:54 +1100
I tested the sym-2 driver on ppc64 and found that hcb_p can be > 1 page but __sym_malloc fails for allocations over 1 page. This means we die in sym_attach.
Are you using 4K pages on ppc64? :-(
Franks a lot,
David S. Miller
[email protected]
Quoted 2 lines
Are you using 4K pages on ppc64? :-(
Unfortunately so. We will definitely be looking to decouple hardware and software page sizes (like sparc64 is doing) once things stabilise, a 4KB page size is pretty small for a 64 bit arch.
Anton
From: Anton Blanchard <[email protected]>
Date: Thu, 15 Nov 2001 22:35:26 +1100
Quoted 1 line
Are you using 4K pages on ppc64? :-(
Unfortunately so. We will definitely be looking to decouple hardware and software page sizes (like sparc64 is doing) once things stabilise, a 4KB page size is pretty small for a 64 bit arch.
Using an 8K page size should really be transparent to any sane ELF userland, why not just do it? Is there some hardcoded dependency in the ppc ELF stuff or is it just a "some of our kernel code still assumes PAGE_SIZE = 4K"?
Quoted 6 lines
Using an 8K page size should really be transparent to any sane ELF userland, why not just do it? Is there some hardcoded dependency in the ppc ELF stuff or is it just a "some of our kernel code still assumes
PAGE_SIZE= 4K"?
Its just that the kernel code assumes 4k PAGE_SIZE. Fixing this is high on my list but there are still more basic things to finish first :)
Anton
Quoted 6 lines · Anton Blanchard
Hi,
I tested the sym-2 driver on ppc64 and found that
hcb_pcan be > 1 page but__sym_mallocfails for allocations over 1 page. This means we die insym_attach.
The driver should not need more than 4096 bytes for a single allocation. If the ppc64 page size is smaller, your patch is ok, otherwise something may have to be fixed, likely in the driver. I cannot access to kernel source immediately but I will check what kind of page size ppc64 is using asap.
Quoted 1 line
With this patch the sym-2 works on ppc64. BTW so far it looks solid :)
Great!
Thanks for your report.
Regards,
Gérard.
Quoted 17 lines
Anton
diff -urN
2.4.15-pre4/drivers/scsi/sym53c8xx_2/sym_glue.hlinuxppc_2_4_devel_work/drivers/scsi/sym53c8xx_2/sym_glue.hdrivers/scsi/sym53c8xx_2/sym_glue.h +1
-1@@ -526,7 +526,7 @@* couple of things related to the memory allocator. */ typedef u_long m_addr_t ;/* Enough bits to represent any address */ -#define SYM_MEM_PAGE_ORDER 0/* 1 PAGE maximum */ +#define SYM_MEM_PAGE_ORDER 1/* 2 PAGE maximum */ #define SYM_MEM_CLUSTER_SHIFT (PAGE_SHIFT+SYM_MEM_PAGE_ORDER)#ifdef MODULE #define SYM_MEM_FREE_UNUSED /* Free unused pages immediately */
From: G廨ard Roudier <[email protected]>
Date: Thu, 15 Nov 2001 17:27:38 +0100 (CET)
The driver should not need more than 4096 bytes for a single allocation.
If platform is 64-bit and PAGE_SIZE < 8K, yes it will.
And ppc64 fits this criteria. �:.𣳾𤎖忷m賨▎a𪘁b習鮓wm�b嘅��捩滵嶵𪐴�𡟯蔰_�傂n嫑←屹��h膥霅�&�輈蹂z嫡𣚭+��+zfㄑ搬�坍��i����𡟯蔰_霈熝:+v��)舝鷰锭憃�剜𧙕��菈��蟳簏霈�i
Quoted 7 lines · David S. Miller
From: Gérard Roudier <[email protected]>
Date: Thu, 15 Nov 2001 17:27:38 +0100 (CET)The driver should not need more than 4096 bytes for a single allocation.
If platform is 64-bit and
PAGE_SIZE< 8K, yes it will. And ppc64 fits this criteria.
Btw, I didn't see any ppc64 stuff in linux kernel. As a result this platform is still in the unsupported status. :-)
Indeed a 4K page on a 64 bit machine looks very suboptimal. Note that it is also way too small for 32 bits machines with hundreds of megabytes of memory. I am wondering about the reasons that made us keep with so small a page size. As you know earlier BSDs used 2K on Vaxen that as you also know had (have?) a physical PAGE_SIZE of 512 bytes.
About the sym-2 driver, it may well be some pointers that make it need more than 4K allocation on 64 bit machines.
I will try to make it fit a page size max allocation even on such weird configuration, since I do consider as high^H^H^H^Hslightly broken any piece of software that requires more that 1 PAGE of physically contiguous allocation. :-) :-) :-)
To be serious, the right fix is to have some logical page be some power of two of the physical page when the physical page is too small. Can we hope Linux-2.5 to allow this?
Gérard.
Quoted 4 lines · Gérard Roudier
To be serious, the right fix is to have some logical page be some power of two of the physical page when the physical page is too small. Can we hope Linux-2.5 to allow this?
It's certainly doable. I have an i386 patch against 2.4.7 which did that, MMUPAGE_SIZE 4kB distinguished from PAGE_SIZE 4kB, 8kB, 16kB or 32kB (but 64kB truncates to 0 in unsigned short b_size, doesn't work so well!); while still presenting the 4kB EXEC_PAGESIZE to userspace.
It's a bit tedious working through each kernel update, to decide which PAGEs should be MMUPAGEs etc, and I didn't see an immediate reward of a huge leap in performance, so I haven't kept it up to date since then.
Hugh
Quoted 6 lines · Hugh Dickins
Quoted 4 lines · Gérard Roudier
To be serious, the right fix is to have some logical page be some power of two of the physical page when the physical page is too small. Can we hope Linux-2.5 to allow this?
Quoted 8 lines
It's certainly doable. I have an i386 patch against 2.4.7 which did that,
MMUPAGE_SIZE4kB distinguished fromPAGE_SIZE4kB, 8kB, 16kB or 32kB (but 64kB truncates to 0 in unsigned shortb_size, doesn't work so well!); while still presenting the 4kBEXEC_PAGESIZEto userspace.It's a bit tedious working through each kernel update, to decide which PAGEs should be MMUPAGEs etc, and I didn't see an immediate reward of a huge leap in performance, so I haven't kept it up to date since then.
Think about portable _software_ as large parts of the kernel that have to cope with very different _hardware_ allocation granularities depending on targetted platforms. This led and still leads to useless complexity in many places. Apart this that will require to modify the kernel code in order to gain full advantage of a new larger page size, you will get some not negligible immediate reward on IA32, as for example fork() to be really unlikely to fail on loaded machines, etc..., etc...
Gérard.
Quoted 9 lines · Gérard Roudier
Quoted 6 lines · Anton Blanchard
Hi,
I tested the sym-2 driver on ppc64 and found that
hcb_pcan be > 1 page but__sym_mallocfails for allocations over 1 page. This means we die insym_attach.
Quoted 5 lines
The driver should not need more than 4096 bytes for a single allocation. If the ppc64 page size is smaller, your patch is ok, otherwise something may have to be fixed, likely in the driver. I cannot access to kernel source immediately but I will check what kind of page size ppc64 is using asap.
Could you revert your change and give my patch below a try. Btw, you will be in sync with my current sources. Booting with sym53c8xx=debug:1 will let the driver print all memory allocations to the syslog. You may send me the drivers messages related to these allocations for information.
Gérard.
PS: I do have tried the patch on a IA32 machine under linux-2.4.13.
Quoted 11 lines
Quoted 1 line
With this patch the sym-2 works on ppc64. BTW so far it looks solid :)
Great!
Thanks for your report.
Regards,
Gérard.Quoted 3 lines
Anton
Quoted 12 lines
drivers/scsi/sym53c8xx_2/sym_glue.h line 532
Quoted 12 lines
diff -urN
2.4.15-pre4/drivers/scsi/sym53c8xx_2/sym_glue.hlinuxppc_2_4_devel_work/drivers/scsi/sym53c8xx_2/sym_glue.hdrivers/scsi/sym53c8xx_2/sym_glue.h +1
-1@@ -526,7 +526,7 @@* couple of things related to the memory allocator. */ typedef u_long m_addr_t ;/* Enough bits to represent any address */ -#define SYM_MEM_PAGE_ORDER 0/* 1 PAGE maximum */ +#define SYM_MEM_PAGE_ORDER 1/* 2 PAGE maximum */ #define SYM_MEM_CLUSTER_SHIFT (PAGE_SHIFT+SYM_MEM_PAGE_ORDER)#ifdef MODULE #define SYM_MEM_FREE_UNUSED /* Free unused pages immediately */
sym_glue.h renamed from sym-2-orig/sym_glue.h +0 -1
@@ -77,7 +77,6 @@#include <linux/errno.h> #include <linux/pci.h> #include <linux/string.h> -#include <linux/malloc.h> #include <linux/mm.h> #include <linux/ioport.h> #include <linux/time.h>
Only in .: sym_glue.o
sym_hipd.c renamed from sym-2-orig/sym_hipd.c +11 -0
@@ -4691,6 +4691,7 @@OUTL_DSP (SCRIPTA_BA (np, clrack));return ; out_stuck: +return ; }/*
@@ -5226,6 +5227,7 @@return ; fail: +return ; }/*
@@ -5788,6 +5790,13 @@ goto attach_failed;/* +* Allocate the array of lists of CCBs hashed by DSA. +*/ + np->ccbh =sym_calloc (sizeof (ccb_p *)*CCB_HASH_SIZE ,"CCBH" ); +if (!np->ccbh) + goto attach_failed; + +/* * Initialyze the CCB free and busy queues. */ sym_que_init (&np->free_ccbq);
@@ -5978,6 +5987,8 @@sym_mfree_dma (cp,sizeof (*cp),"CCB" ); } } +if (np->ccbh) +sym_mfree (np->ccbh,sizeof (ccb_p *)*CCB_HASH_SIZE ,"CCBH" );if (np->badluntbl)sym_mfree_dma (np->badluntbl,256 ,"BADLUNTBL" );
sym_hipd.h renamed from sym-2-orig/sym_hipd.h +2 -1
@@ -1068,7 +1068,8 @@/* * CCB lists and queue. */ -ccb_p ccbh[CCB_HASH_SIZE ];/* CCB hashed by DSA value */ +ccb_p *ccbh;/* CCBs hashed by DSA value */ +/* CCB_HASH_SIZE lists of CCBs */ SYM_QUEHEAD free_ccbq;/* Queue of available CCBs */ SYM_QUEHEAD busy_ccbq;/* Queue of busy CCBs */
sym_nvram.c renamed from sym-2-orig/sym_nvram.c +4 -4
@@ -505,10 +505,10 @@return retv; } -#undef SET_BIT 0 -#undef CLR_BIT 1 -#undef SET_CLK 2 -#undef CLR_CLK 3 +#undef SET_BIT +#undef CLR_BIT +#undef SET_CLK +#undef CLR_CLK/* * Try reading Symbios NVRAM.
From: G廨ard Roudier <[email protected]>
Date: Thu, 15 Nov 2001 20:46:34 +0100 (CET)
diff -u ../sym-2-orig/sym_glue.h ./sym_glue.h
--- ../sym-2-orig/sym_glue.h Thu Nov 15 22:53:34 2001
+++ ./sym_glue.h Thu Nov 15 23:18:58 2001
@@ -77,7 +77,6 @@
#include <linux/errno.h>
#include <linux/pci.h>
#include <linux/string.h>
-#include <linux/malloc.h>
#include <linux/mm.h>
#include <linux/ioport.h>
#include <linux/time.h>Hmmm, why not add linux/slab.h? It exists in every Linux kernel tree
your driver would ever be compiled under. �:.𣳾𤎖忷m賨▎a𪘁b習鮓wm�b嘅��捩滵嶵𪐴�𡟯蔰_�傂n嫑←屹��h膥霅�&�輈蹂z嫡𣚭+��+zfㄑ搬�坍��i����𡟯蔰_霈熝:+v��)舝鷰锭憃�剜𧙕��菈��蟳簏霈�i
I promise to fix my mailer when responding to non-ascii postings, sigh :(
Quoted 15 lines · David S. Miller
From: Gérard Roudier <[email protected]> Date: Thu, 15 Nov 2001 20:46:34 +0100 (CET) diff -u ../sym-2-orig/sym_glue.h ./sym_glue.h --- ../sym-2-orig/sym_glue.h Thu Nov 15 22:53:34 2001 +++ ./sym_glue.h Thu Nov 15 23:18:58 2001 @@ -77,7 +77,6 @@ #include <linux/errno.h> #include <linux/pci.h> #include <linux/string.h> -#include <linux/malloc.h> #include <linux/mm.h> #include <linux/ioport.h> #include <linux/time.h>
Quoted 2 lines
Hmmm, why not add
linux/slab.h? It exists in every Linux kernel tree your driver would ever be compiled under.
Just because the driver has its own allocator as you know and thus does not need this header (it didn't need malloc.h too).
Gérard.
Hi,
Quoted 4 lines
Could you revert your change and give my patch below a try. Btw, you will be in sync with my current sources. Booting with sym53c8xx=debug:1 will let the driver print all memory allocations to the syslog. You may send me the drivers messages related to these allocations for information.
Thanks, it boots OK now. Do you still want a debug log?
BTW on ppc64 we can have io port addresses > 32 bits so this change is required.
Anton
drivers/scsi/sym53c8xx_2/sym_glue.h +1 -1
@@ -463,7 +462,7 @@vm_offset_t mmio_va;/* MMIO kernel virtual address */ vm_offset_t ram_va;/* RAM kernel virtual address */ -u32 io_port;/* IO port address */ +u_long io_port;/* IO port address */ u_short io_ws;/* IO window size */ int irq;/* IRQ number */
Hello,
Quoted 8 lines · Anton Blanchard
Hi,
Quoted 4 lines
Could you revert your change and give my patch below a try. Btw, you will be in sync with my current sources. Booting with sym53c8xx=debug:1 will let the driver print all memory allocations to the syslog. You may send me the drivers messages related to these allocations for information.
Thanks, it boots OK now. Do you still want a debug log?
I can guess the result.
Quoted 2 lines
BTW on ppc64 we can have io port addresses > 32 bits so this change is required.
Linux/ppc64 looks strange invention to me. As you know IO base addresses are limited to 32 bit in PCI. And, btw, 32 bits seems to work just fine here as PPC is defined from the driver as using normal IO. But, IIRC, the strange Linux/PPC invention only supports MMIO. :-)
If you want to play with _explicit_ MMIO, you just have to remove a couple of line from sym53c8xx.h. Here they are:
/*
* Use normal IO if configured. Forced for alpha and powerpc. * Powerpc fails copying to on-chip RAM using memcpy_toio(). * Forced to MMIO for sparc.
*/
#if defined(__alpha__)
#define SYM_CONF_IOMAPPED
#elif defined(__powerpc__)
- #define SYM_CONF_IOMAPPED
- #define SYM_OPT_NO_BUS_MEMORY_MAPPING
#elif defined(__sparc__)
#undef SYM_CONF_IOMAPPED
#elif defined(CONFIG_SCSI_SYM53C8XX_IOMAPPED)
#define SYM_CONF_IOMAPPED
#endif
Btw, I cannot guess the result here. You may want to really let me know this time. :)
I cannot apply your patch as it is, since I want the driver to distinguish between kernel fake cookies associated with base addresses and actual values of those registers. This is needed, since some of these values must be known from SCSI SCRIPTS and thus must fit the _reality_ and not any kernel developpers' dream, could be the greatest ones:).
Thanks, anyway, for reporting this problem.
Regards,
Gérard.
Quoted 11 lines
diff -urN
linuxppc_2_4_devel/drivers/scsi/sym53c8xx_2/sym_glue.hlinuxppc_2_4_devel_work/drivers/scsi/sym53c8xx_2/sym_glue.hdrivers/scsi/sym53c8xx_2/sym_glue.h +1
-1@@ -463,7 +462,7 @@vm_offset_t mmio_va;/* MMIO kernel virtual address */ vm_offset_t ram_va;/* RAM kernel virtual address */ -u32 io_port;/* IO port address */ +u_long io_port;/* IO port address */ u_short io_ws;/* IO window size */ int irq;/* IRQ number */
Hi,
Quoted 4 lines
Linux/ppc64 looks strange invention to me. As you know IO base addresses are limited to 32 bit in PCI. And, btw, 32 bits seems to work just fine here as PPC is defined from the driver as using normal IO. But, IIRC, the strange Linux/PPC invention only supports MMIO. :-)
Since all PCI IO is memory mapped on ppc64, IO addresses end up > 32 bits. Until recently we used to have an IO offset that we added to all accesses which kept the driver visible IO addresses < 32 bits. (This is still the case with ppc32)
The change was made to support error handling, the 64 bit token has the pci bus,dev,fn embedded in it so that the low level IO routines can do error recovery if in{b,w,l} fails.
I didnt make these changes and it would seem we can link IO address <-> pci bus,dev,fn in other ways, if it turns out many drivers cannot use u64 for IO ports then we will have to investigate them.
Quoted 2 lines
If you want to play with _explicit_ MMIO, you just have to remove a couple of line from
sym53c8xx.h. Here they are:
Yes MMIO works fine. Is there a reason we force PCI IO on __powerpc__?
Anton
From: Anton Blanchard <[email protected]>
Date: Wed, 21 Nov 2001 13:19:01 +1100
I didnt make these changes and it would seem we can link IO address <-> pci bus,dev,fn in other ways, if it turns out many drivers cannot use u64 for IO ports then we will have to investigate them.
Any driver which cannot handle PCI I/O resources being an unsigned long is completely broken.
They are 64-bits on Sparc64 already, and this is the type of PCI resource values.
Quoted 7 lines · Anton Blanchard
Hi,
Quoted 4 lines
Linux/ppc64 looks strange invention to me. As you know IO base addresses are limited to 32 bit in PCI. And, btw, 32 bits seems to work just fine here as PPC is defined from the driver as using normal IO. But, IIRC, the strange Linux/PPC invention only supports MMIO. :-)
Quoted 8 lines
Since all PCI IO is memory mapped on ppc64, IO addresses end up > 32 bits. Until recently we used to have an IO offset that we added to all accesses which kept the driver visible IO addresses < 32 bits. (This is still the case with ppc32)
The change was made to support error handling, the 64 bit token has the pci bus,dev,fn embedded in it so that the low level IO routines can do error recovery if in{b,w,l} fails.
When it happens to implement a driver that involves both C code and some firmware that works on the PCI-chip, using opaque handles as for C is not enough. If you think your driver only with those abstractions, you will get a bag of bits that does not work.
In such situations C abstractions are just dreams of some ideal world that does not exists and that will never do. Having in mind and in some places of the code the _reality_ is required if you want your stuff to have chance to work.
Quoted 3 lines
I didnt make these changes and it would seem we can link IO address <-> pci bus,dev,fn in other ways, if it turns out many drivers cannot use u64 for IO ports then we will have to investigate them.
The sym driver can do all what I wrote about regarding DMA addressing, given expected kernel interfaces and appropriate hardware.
Quoted 4 lines
Quoted 2 lines
If you want to play with _explicit_ MMIO, you just have to remove a couple of line from
sym53c8xx.h. Here they are:Yes MMIO works fine. Is there a reason we force PCI IO on
__powerpc__?
My opinion, may-be just 0.02 euros:
At the time of early PPC port, only a few courageous PCI device drivers wanted to default to PCI-recommended MMIO. There were a least 1 driver that wanted so, but not may more :). At this early time, the MMIO kernel interface in ppc port was not behaving as expected and MMIO wasn't usable. This lasted years. I would be very glad if MMIO related kernel interface is now fine, but allow me to still doubt a bit about.
If Linux/ppc team can ensure that Linux/ppc is now ok for MMIO, then you can send the corresponding patch, that default PPC to MMIO as seen by the driver, to kernel maintainer(s) with my _full_ approbation.
Regards,
Gérard.
Quoted 3 lines · Gerard Roudier
If Linux/ppc team can ensure that Linux/ppc is now ok for MMIO, then you can send the corresponding patch, that default PPC to MMIO as seen by the driver, to kernel maintainer(s) with my _full_ approbation.
Yes, Linux/PPC now does MMIO just fine.
Paul.