Skip to contentSkip to conversation

Virtual device and ARP table

DISCUSSION Christophe Jelger 2 participants 5 messages Jun 7, 2010

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hello,

I am currently "resurrecting" a Linux module (called LUNAR) which I co-developed in 2007 and I'm having a weird kernel crash. This code basically used to work fine up to 2.6.18 which was the latest version before we stopped our development. I quickly ported it to 2.6.{31,32}: it compiles fine and loads fine, but it crashes/hangs the kernel when it's really being used.

The module is a virtual device used for MANET routing: with the current version, it basically "captures" DNS requests sent to the virtual interface --> this triggers the sending of a fake DNS reply (see below) and the creation of an ARP table entry for the destination (the MANET route is built at the same time). Packets can then be sent to the destination.

The problem I'm having is that the kernel quickly hangs after I create a new ARP entry (actually only if it's being used). If the entry I create is set to NUD_PERMANENT, then everything works fine! I use __neigh_lookup_errno to lookup/create the entry and neigh_lookup to set/update the MAC address. Note that the ARP entry is created without problem, but typically even just doing a userspace "arp -a" command can crash the kernel (it also hangs the userspace command!). Doing "arp -na" usually does NOT crash the kernel!

I guess the problem comes from a combination of ARP + DNS lookups/replies. Note that my kernel module has its own internal fake DNS server which captures lookups and sends replies directly back to the stack. What is amazing: if the ARP entry I create is set to NUD_PERMANENT, then I don't get any crash (however I cannot develop my module with permanent ARP entries).

I'm wondering if there were any major changes to the neighbor and arp code (between 2.6.18 and 2.6.31) that are somehow causing this problem ?...

Any hint is very welcome.

thanks in advance,
Christophe

PS: I can easily reproduce the problem, and was trying to debug with qemu and gdb server but so fra no success to clearly identify the problem. Last point: it seems the kernel does not really "crash" but rather ends up in some unstable state and maybe in a loop.

#1 lore raw
Quoted 37 lines · Christophe Jelger

Hello,

I am currently "resurrecting" a Linux module (called LUNAR) which I co-developed in 2007 and I'm having a weird kernel crash. This code basically used to work fine up to 2.6.18 which was the latest version before we stopped our development. I quickly ported it to 2.6.{31,32}:
it compiles fine and loads fine, but it crashes/hangs the kernel when it's really being used.

The module is a virtual device used for MANET routing: with the current version, it basically "captures" DNS requests sent to the virtual interface --> this triggers the sending of a fake DNS reply (see below) and the creation of an ARP table entry for the destination (the MANET route is built at the same time). Packets can then be sent to the destination.

The problem I'm having is that the kernel quickly hangs after I create a new ARP entry (actually only if it's being used). If the entry I create is set to NUD_PERMANENT, then everything works fine! I use __neigh_lookup_errno to lookup/create the entry and neigh_lookup to set/update the MAC address. Note that the ARP entry is created without problem, but typically even just doing a userspace "arp -a" command can crash the kernel (it also hangs the userspace command!). Doing "arp -na" usually does NOT crash the kernel!

I guess the problem comes from a combination of ARP + DNS lookups/replies. Note that my kernel module has its own internal fake DNS server which captures lookups and sends replies directly back to the stack. What is amazing: if the ARP entry I create is set to NUD_PERMANENT, then I don't get any crash (however I cannot develop my module with permanent ARP entries).

I'm wondering if there were any major changes to the neighbor and arp code (between 2.6.18 and 2.6.31) that are somehow causing this problem ?...

Any hint is very welcome.

Quoted 8 lines

thanks in advance,
Christophe

PS: I can easily reproduce the problem, and was trying to debug with qemu and gdb server but so fra no success to clearly identify the problem. Last point: it seems the kernel does not really "crash" but rather ends up in some unstable state and maybe in a loop. --

Hi Christophe

You should ask these kind of questions on netdev instead of lkml.

And of course, post your patch, or send us a crystal ball ;)

Yes, many things changed between 2.6.18 and 2.6.34

#2 lore raw
Quoted 21 lines · Eric Dumazet
Quoted 12 lines · Christophe Jelger

Hello,

I am currently "resurrecting" a Linux module (called LUNAR) which I co-developed in 2007 and I'm having a weird kernel crash. This code basically used to work fine up to 2.6.18 which was the latest version before we stopped our development. I quickly ported it to 2.6.{31,32}: it compiles fine and loads fine, but it crashes/hangs the kernel when it's really being used.

The module is a virtual device used for MANET routing: with the current version, it basically "captures" DNS requests sent to the virtual interface --> this triggers the sending of a fake DNS reply (see below) and the creation of an ARP table entry for the destination (the MANET route is built at the same time). Packets can then be sent to the destination.

The problem I'm having is that the kernel quickly hangs after I create a new ARP entry (actually only if it's being used). If the entry I create is set to NUD_PERMANENT, then everything works fine! I use __neigh_lookup_errno to lookup/create the entry and neigh_lookup to set/update the MAC address. Note that the ARP entry is created without problem, but typically even just doing a userspace "arp -a" command can crash the kernel (it also hangs the userspace command!). Doing "arp -na" usually does NOT crash the kernel!

I guess the problem comes from a combination of ARP + DNS lookups/replies. Note that my kernel module has its own internal fake DNS server which captures lookups and sends replies directly back to the stack. What is amazing: if the ARP entry I create is set to NUD_PERMANENT, then I don't get any crash (however I cannot develop my module with permanent ARP entries).

I'm wondering if there were any major changes to the neighbor and arp code (between 2.6.18 and 2.6.31) that are somehow causing this problem ?...

Quoted 7 lines

Any hint is very welcome.

thanks in advance,
Christophe

PS: I can easily reproduce the problem, and was trying to debug with qemu and gdb server but so fra no success to clearly identify the problem. Last point: it seems the kernel does not really "crash" but rather ends up in some unstable state and maybe in a loop. --

Quoted 8 lines

Hi Christophe

You should ask these kind of questions on netdev instead of lkml.

And of course, post your patch, or send us a crystal ball ;)

Yes, many things changed between 2.6.18 and 2.6.34

Eric: thanks for the forward to the netdev list. Regarding the code, I of course welcome any help but didn't want to pollute the list with unsollicited code: I can of course of course send it directly to anyone who is willing to help (I can easily reproduce the problem on different machines).

Christophe

#3 lore raw
Quoted 7 lines · Christophe Jelger

Eric: thanks for the forward to the netdev list. Regarding the code, I of course welcome any help but didn't want to pollute the list with unsollicited code: I can of course of course send it directly to anyone who is willing to help (I can easily reproduce the problem on different machines).

Christophe,

Unless patch is really huge, its ok to send it on netdev, with a RFC label, so that only people with free time take a look, eventually.

[RFC] lunar: ....

#4 lore raw

[RFC] lunar manet routing module

Quoted 7 lines · Eric Dumazet

Christophe,

Unless patch is really huge, its ok to send it on netdev, with a RFC label, so that only people with free time take a look, eventually.

[RFC] lunar: ....

[not sure what 'huge' means, I'm sending 60k -- sorry for the pollution]

Eric: thanks for the advice. Instead of a patch I attach a .tgz (hope it's ok) of the module code with a README explaining everything: it compiles for 2.6.31 and 2.6.32, didn't try more recent kernels because I actually want to deploy the whole thing on OpenWRT 2.6.32 on Linksys devices.

To all: the lunar module crashes the kernel, so be careful. I tried debugging with qemu and gdb server but could not find the bug(s) -- my experience for kernel debugging is in fact limited.

thanks in advance for any help,
Christophe