Issues with vmalloc and ebtables on ColdFire V4e

Hi,

We are having some stability issues on our ColdFire V4e based board and hope this forum can provide some advice on how to track the issue down.

The CPU is the 54415 running a 2.6.38 kernel with BSP patches and bug fixes provided by Freescale. 64MiB SRAM and 64MiB NAND Flash. The compiler is GCC 4.4.1-cs4.4-54.

The problem is that when modifying the ebtables structures by adding/flushing the rules, sooner or later a segfault occurs in a (seemingly random) userspace process, with the faulting address within a region of memory vmalloc'ed by ebtables (for our BSP, the vmalloc region is 0xc0000000-0xcfffffff). The fault does not occur right after modifying the rules, it occurs some time (sometimes up to hours) later.

I was able to avoid the segfaults by replacing all vmalloc/vfree calls with kmalloc/kfree in net/bridge/netfilter/ebtables.c.

Although other kernel subsystems make use of vmalloc, these regions seem to be allocated once and never free'd during runtime.

For example:

cat /proc/vmallocinfo

0xc0026000-0xc0048000 139264 ubi_attach_mtd_dev+0x436/0x9d0 pages=16 pfn=41891 vmalloc
0xc004a000-0xc006c000 139264 ubi_attach_mtd_dev+0x44a/0x9d0 pages=16 pfn=4188e vmalloc
0xc006e000-0xc0076000 32768 ubi_read_volume+0x612/0x9cc pages=3 pfn=418b4
vmalloc
0xc009a000-0xc00ac000 73728 lzo_init+0x10/0x28 pages=8 pfn=418bf vmalloc
0xc00ae000-0xc00f2000 278528 deflate_init+0x1e/0xea pages=33 pfn=418d7 vmalloc 0xc00f4000-0xc0116000 139264 ubifs_mount+0x808/0x145a pages=16 pfn=418c5 vmalloc
0xc0118000-0xc013a000 139264 ubifs_mount+0x9a0/0x145a pages=16 pfn=41911 vmalloc
0xc013c000-0xc0140000 16384 ubifs_lpt_init+0x6a/0x446 pages=1 pfn=41922
vmalloc
0xc0148000-0xc014c000 16384 ubifs_lpt_init+0x3c/0x446 pages=1 pfn=41924
vmalloc
0xc014e000-0xc0170000 139264 ubifs_lpt_init+0x142/0x446 pages=16 pfn=41925 vmalloc
0xc0172000-0xc0194000 139264 ubifs_mount_orphans+0x360/0x434 pages=16 pfn=41935 vmalloc
0xc05f2000-0xc05f6000 16384 T.727+0x9a/0x17a pages=1 pfn=41ac9 vmalloc
0xc05f8000-0xc05fc000 16384 T.727+0xae/0x17a pages=1 pfn=41a40 vmalloc
<--- ebtables
0xc064c000-0xc0650000 16384 T.727+0x9a/0x17a pages=1 pfn=419b6 vmalloc
0xc0652000-0xc0656000 16384 T.727+0xae/0x17a pages=1 pfn=41a46 vmalloc

Just prior to the userspace segfault I captured the following in the do_page_fault handler:

pc 0x402cc286, mmusr 0x0, complainingAddress 0xc05f8024
pc 0x60012b7e, mmusr 0x0, complainingAddress 0xc05f8024
do_page_fault: if (!vma)
regs->sr=0x18, regs->pc=0x60012b7e, address=0xc05f8024, signo=SIGSEGV, code=SEGV_MAPERR

I appreciate any comments or advice on narrowing this issue down. Let me know if more information is needed.

Thanks and regards
Reid Lindsay

Should those patches be merged in the kernel if they fix bugs?

Adrian

 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - [email protected]
`. `'   Freie Universitaet Berlin - [email protected]
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Yes, absolutely. We are considering starting a project to upgrade our kernel, and I would be happy to contribute any patches and bug fixes which result from this effort.

The Freescale (now NXP) BSP for ColdFire was last updated in Oct 2011, and as far as I can tell, there has been no effort to merge patches into the mainline kernel. The BSP can be found here: http://www.nxp.com/products/software-and-tools/run-time-software/linux-software-and-development-tools/bsps-for-coldfire-architectures:CW_BSP_COLDFIRE. In response to bugs we discovered in the DMA and Ethernet drivers, Freescale directly provided us with some additional patches on top of this BSP. Judging by the website, these patches have not yet been released publicly. Some effort will be required to port them to mainline (if indeed they are still relevant) given the m68k code has diverged somewhat.

Reid

The mainline support for v4e is not directly based on Freescales changes - though it does contain some parts of code they developed. And mainline doesn't currently support the 54415. (There are some patches that are close but not quite there yet).

Have you tried asking Freescale?
After all you are using code and patches they supplied.

Regards
Greg

I have examined the code in mainline and it is, as you say, close but not quite there yet. I have borrowed some parts of mainline code to improve our platform over the years.

Asking Freescale is the next step. Arm twisting may be required as they no longer provide ongoing support for this BSP. And it gets us no closer to moving towards a more recent kernel.

Incidentally, our board has been stable since 2012, this issue only came to light once ebtables was enabled.

Reid

If the issue were to be reproduced using a 680x0 emulator like Aranym (and maybe the Debian compiler) one might suspect some code in ebtables is assuming natural alignment of integers...

$ grep -i align net/bridge/netfilter/ebt*