kernel panic

Hi,

I used netconsole to document a kernel panic related to AX25 on one of my systems.
This occurs systematically as soon as I close an AX25 session established through a level 3 ROSE network (FPAC) via AXUDP link (ax25ipd). It is not triggered when connecting locally or via AX25 call through LAN.

This is observed with kernel-2.6.24.2 and -3 with most recent AX25 patches applied.

Bernard Pidoux

BUG: unable to handle kernel paging request at virtual address 6b6b6c2b
printing eip: c0139585 *pde = 00000000
Oops: 0002 [#1]
Modules linked in: netconsole configfs rose netrom mkiss crc16 ax25 mga drm nfsd exportfs nfs lockd nfs_acl sunrpc af_packet ipv6 snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_pcm_oss snd_mixer_oss binfmt_misc loop floppy usbhid snd_ens1371 snd_rawmidi snd_seq_device snd_ac97_codec snd_pcm uhci_hcd snd_timer snd soundcore snd_page_alloc usbcore ne2k_pci intel_agp ac97_bus ide_cd 8390 agpgart ext3 jbd ide_disk piix ide_core

Pid: 3527, comm: ax25ipd Not tainted (2.6.24.2 #2)
EIP: 0060:[<c0139585>] EFLAGS: 00010012 CPU: 0
EIP is at __lock_acquire+0x65/0x11d0
EAX: 00000012 EBX: 00000001 ECX: c7670068 EDX: 00000002
ESI: 00000000 EDI: 6b6b6b6b EBP: c6bf5bc0 ESP: c6bf5b38
DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068
Process ax25ipd (pid: 3527, ti=c6bf4000 task=c60251d0 task.ti=c6bf4000)

Stack: 0000580b 00000000 02c059e8 00000000 c04f73d8 02c059e8 00000000 c6bf5be4
       c01397d8 00000001 c017ecc0 00000000 00000002 00000000 c7670068 
c60251d0
       0000580b 00000000 02c059d1 00000000 c04f3ce0 00000002 00000000 
c75290f8
Call Trace:
 [<c010513a>] show_trace_log_lvl+0x1a/0x30
 [<c01051f9>] show_stack_log_lvl+0xa9/0xd0
 [<c01052e6>] show_registers+0xc6/0x1b0
 [<c01054b5>] die+0xe5/0x220
 [<c011244f>] do_page_fault+0x15f/0x610
 [<c02c2cfa>] error_code+0x6a/0x70
 [<c013a773>] lock_acquire+0x83/0xa0
 [<c02c2abe>] _spin_lock_irqsave+0x3e/0x70
 [<c011458b>] __wake_up+0x1b/0x50
 [<c024d56c>] sock_def_write_space+0x9c/0xc0
 [<c024d96a>] sock_wfree+0x3a/0x40
 [<c024fb61>] skb_release_all+0x51/0x120
 [<c024f30b>] __kfree_skb+0xb/0x90
 [<c024f3a9>] kfree_skb+0x19/0x30
 [<c8c19e03>] ax25_frames_acked+0x43/0x70 [ax25]
 [<c8c17a10>] ax25_check_iframes_acked+0x40/0x60 [ax25]
 [<c8c1946f>] ax25_std_frame_in+0x6af/0x890 [ax25]
 [<c8c16dba>] ax25_kiss_rcv+0x2ba/0x7b0 [ax25]
 [<c0254e89>] netif_receive_skb+0x379/0x470
 [<c0257a85>] process_backlog+0x75/0xe0
 [<c025759b>] net_rx_action+0x12b/0x1d0
 [<c011d595>] __do_softirq+0x55/0xc0
 [<c011d647>] do_softirq+0x47/0x50
 [<c011d9f5>] local_bh_enable_ip+0x95/0xe0
 [<c02c24b5>] _spin_unlock_bh+0x25/0x30
 [<c8c0c6b0>] mkiss_receive_buf+0x360/0x420 [mkiss]
 [<c020e077>] pty_write+0x47/0x60
 [<c020b642>] write_chan+0x222/0x330
 [<c0208d06>] tty_write+0x126/0x1b0
 [<c01727ff>] vfs_write+0xaf/0x120
 [<c0172d4d>] sys_write+0x3d/0x70
 [<c010412a>] sysenter_past_esp+0x5f/0xa5
 =======================
Code: 00 9c 58 f6 c4 02 0f 85 52 06 00 00 83 fe 07 0f 87 7b 06 00 00 85 f6 0f 85 fe 03 00 00 8b 4d b0 8b 79 04 85 ff 0f 84 f0 03 00 00 <ff> 87 c0 00 00 00 89 f8 e8 ce c9 ff ff 85 c0 0f 85 d6 02 00 00
EIP: [<c0139585>] __lock_acquire+0x65/0x11d0 SS:ESP 0068:c6bf5b38
Kernel panic - not syncing: Fatal exception in interrupt

Bernard Pidoux F6BVP <[email protected]> :
[...]

May be replacing netif_rx() in drivers/net/hamradio/mkiss.c::ax_bump() by netif_receive_skb() ?

Ueimor

Sorry, this does not made it.
I replaced netif_rx() by netif_receive_skb(skb) but this gives a worst situation : system is frozen. Thank you for the suggestion.

=======================

BUG: soft lockup - CPU#0 stuck for 11s! [ax25ipd:3584]

Pid: 3584, comm: ax25ipd Not tainted (2.6.24.3 #7)
EIP: 0060:[<c01ed892>] EFLAGS: 00000246 CPU: 0 EIP is at delay_tsc+0x12/0x20
EAX: 7752234b EBX: 00000001 ECX: 7752234b EDX: 00000016
ESI: 087022c8 EDI: 00000000 EBP: c6bb5bc0 ESP: c6bb5bbc
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
CR0: 8005003b CR2: b7dd86f8 CR3: 07960000 CR4: 000000d0
DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
DR6: ffff0ff0 DR7: 00000400

 [<c010535a>] show_trace_log_lvl+0x1a/0x30
 [<c0105d82>] show_trace+0x12/0x20
 [<c01024cc>] show_regs+0x1c/0x20
 [<c015555b>] softlockup_tick+0x11b/0x150
 [<c0126c12>] run_local_timers+0x12/0x20
 [<c012708f>] update_process_times+0x2f/0x60
 [<c013a355>] tick_periodic+0x25/0x80
 [<c013a3c9>] tick_handle_periodic+0x19/0x80
 [<c0107aca>] timer_interrupt+0x4a/0x60
 [<c0155928>] handle_IRQ_event+0x28/0x60
 [<c015725f>] handle_level_irq+0x7f/0xf0
 [<c0106e68>] do_IRQ+0x48/0xa0
 [<c0104d16>] common_interrupt+0x2e/0x34
 [<c01ed839>] __delay+0x9/0x10
 [<c01f15f2>] _raw_spin_lock+0xb2/0x140
 [<c02cd350>] _spin_lock_bh+0x50/0x60
 [<c8c59e72>] ax_xmit+0x112/0x370 [mkiss]
 [<c025e947>] dev_hard_start_xmit+0x207/0x270
 [<c026f454>] __qdisc_run+0x54/0x1b0
 [<c0261226>] dev_queue_xmit+0x266/0x340
 [<c8c27a87>] ax25_queue_xmit+0x47/0x70 [ax25]
 [<c8c27b77>] ax25_transmit_buffer+0xc7/0x110 [ax25]
 [<c8c27c3d>] ax25_send_iframe+0x7d/0x110 [ax25]
 [<c8c27d80>] ax25_kick+0xb0/0x1b0 [ax25]
 [<c8c28e49>] ax25_std_frame_in+0x69/0x890 [ax25]
 [<c8c26dbb>] ax25_kiss_rcv+0x2bb/0x7c0 [ax25]
 [<c025e24a>] netif_receive_skb+0x38a/0x480
 [<c8c596ab>] mkiss_receive_buf+0x34b/0x420 [mkiss]
 [<c0219457>] pty_write+0x47/0x60
 [<c0216a19>] write_chan+0x229/0x330
 [<c021407b>] tty_write+0x12b/0x1c0
 [<c017b66f>] vfs_write+0xaf/0x120
 [<c017bcbd>] sys_write+0x3d/0x70
 [<c010427e>] sysenter_past_esp+0x5f/0xa5
 =======================

Regards,

Bernard Pidoux

I also tried to revert Jarek's mkiss ax_bump() patch, but this did not prevent kernel lockup, although this time swapper is involved and not ax25ipd.

Bernard Pidoux

BUG: unable to handle kernel paging request at virtual address 6b6b6c2b
printing eip: c013fc67 *pde = 00000000
Oops: 0002 [#1] SMP
Modules linked in: rose netrom mkiss crc16 ax25 mga netconsole drm configfs nfsd exportfs nfs lockd nfs_acl sunrpc af_packet ipv6 snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_pcm_oss snd_mixer_oss binfmt_misc loop usb_storage scsi_mod floppy usbhid snd_ens1371 snd_rawmidi snd_seq_device snd_ac97_codec snd_pcm snd_timer snd uhci_hcd soundcore usbcore snd_page_alloc ne2k_pci ide_cd ac97_bus intel_agp 8390 agpgart genrtc ext3 jbd ide_disk piix ide_core

Pid: 0, comm: swapper Not tainted (2.6.24.3 #7)
EIP: 0060:[<c013fc67>] EFLAGS: 00010012 CPU: 0
EIP is at __lock_acquire+0x67/0x1170
EAX: 00000002 EBX: 00000001 ECX: c7657030 EDX: 00000002
ESI: 6b6b6b6b EDI: 00000000 EBP: c0395c84 ESP: c0395c00
DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
Process swapper (pid: 0, ti=c0394000 task=c036a3c0 task.ti=c0394000)

Stack: 0000600c 00000000 030061f9 00000000 c0514298 030061f9 00000000 c0395ca8
       c013fe71 c0395cb0 c013fe71 00000001 000001e2 00000000 00000002 
00000000
       c7657030 c036a3c0 030061e2 00000000 c0510ba8 030061e2 000001f9 
00000002
Call Trace:
 [<c010535a>] show_trace_log_lvl+0x1a/0x30
 [<c0105419>] show_stack_log_lvl+0xa9/0xd0
 [<c0105507>] show_registers+0xc7/0x1b0
 [<c01056fa>] die+0x10a/0x230
 [<c01146f0>] do_page_fault+0x160/0x630
 [<c02cdaea>] error_code+0x72/0x78
 [<c0140df5>] lock_acquire+0x85/0xa0
 [<c02cd79e>] _spin_lock_irqsave+0x3e/0x70
 [<c0117c4b>] __wake_up+0x1b/0x50
 [<c025689c>] sock_def_write_space+0x9c/0xc0
 [<c0256c9c>] sock_wfree+0x3c/0x50
 [<c0258ef7>] skb_release_all+0x57/0x130
 [<c025868b>] __kfree_skb+0xb/0x90
 [<c0258729>] kfree_skb+0x19/0x40
 [<c8cc1e23>] ax25_frames_acked+0x43/0x70 [ax25]
 [<c8cc11db>] ax25_std_frame_in+0x3fb/0x890 [ax25]
 [<c8cbedbb>] ax25_kiss_rcv+0x2bb/0x7c0 [ax25]
 [<c025e24a>] netif_receive_skb+0x38a/0x480
 [<c0260e9b>] process_backlog+0x7b/0xf0
 [<c02609a7>] net_rx_action+0x167/0x210
 [<c0122da3>] __do_softirq+0x93/0x120
 [<c0122e87>] do_softirq+0x57/0x60
 [<c0123235>] irq_exit+0x85/0x90
 [<c0106e6d>] do_IRQ+0x4d/0xa0
 [<c0104d16>] common_interrupt+0x2e/0x34
 [<c0102606>] cpu_idle+0x76/0xa0
 [<c02ca6a9>] rest_init+0x49/0x50
 [<c0399d75>] start_kernel+0x2d5/0x360
 [<00000000>] 0x0
 =======================
Code: 9c 58 f6 c4 02 0f 85 ff 05 00 00 83 ff 07 0f 87 28 06 00 00 85 ff 0f 85 ac 03 00 00 8b 4d bc 8b 71 04 85 f6 0f 84 9e 03 00 00 90 <ff> 86 c0 00 00 00 89 f0 e8 bc c8 ff ff 85 c0 0f 85 84 02 00 00
EIP: [<c013fc67>] __lock_acquire+0x67/0x1170 SS:ESP 0068:c0395c00
Kernel panic - not syncing: Fatal exception in interrupt

On Wed, Mar 19, 2008 at 09:57:18PM +0100, Bernard Pidoux wrote:
...

...

Well, the rest looks very similarly and not trivial... Probably a socket is destructed too soon (or something...). Could you try to reproduce this with some debugging turned off, e.g.:
CONFIG_DEBUG_LOCK_ALLOC, CONFIG_DEBUG_PAGEALLOC, (CONFIG_SLUB_DEBUG?).

Regards,
Jarek P.

I recompiled the kernel after turning off the above options. However there is still a kernel panic documented below. Does it help ?

Bernard Pidoux

BUG: spinlock bad magic on CPU#0, swapper/0
BUG: unable to handle kernel paging request at virtual address 6b6b6c2f
printing eip: c01e730b *pde = 00000000
Oops: 0000 [#1] SMP
Modules linked in: rose netrom mkiss crc16 ax25 mga drm netconsole configfs nfsd exportfs nfs lockd nfs_acl sunrpc af_packet ipv6 snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_pcm_oss snd_mixer_oss binfmt_misc loop usb_storage scsi_mod floppy usbhid snd_ens1371 snd_rawmidi snd_seq_device snd_ac97_codec snd_pcm snd_timer snd uhci_hcd intel_agp ne2k_pci ide_cd usbcore soundcore snd_page_alloc ac97_bus 8390 agpgart genrtc ext3 jbd ide_disk piix ide_core

Pid: 0, comm: swapper Not tainted (2.6.24.3 #8)
EIP: 0060:[<c01e730b>] EFLAGS: 00010002 CPU: 0
EIP is at spin_bug+0xdb/0xf0
EAX: 0000002f EBX: 6b6b6b6b ECX: 6b6b6b6b EDX: 00000082
ESI: c07e5a38 EDI: c0333633 EBP: c0379c78 ESP: c0379c54
DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
Process swapper (pid: 0, ti=c0378000 task=c0353340 task.ti=c0378000)

Stack: c0333724 c0333633 00000000 c0353509 00000000 00000010 c07e5a38 00000001
       c7b8abb8 c0379cb0 c01e7409 c11448c8 c1147740 00000000 c11448c0 
c0379cb8
       c114775c c11448c8 c1147740 00000000 00000282 00000001 c7b8abb8 
c0379cbc
Call Trace:
 [<c010527a>] show_trace_log_lvl+0x1a/0x30
 [<c0105339>] show_stack_log_lvl+0xa9/0xd0
 [<c0105427>] show_registers+0xc7/0x1b0
 [<c010561a>] die+0x10a/0x230
 [<c0114855>] do_page_fault+0x2f5/0x600
 [<c02c039a>] error_code+0x72/0x78
 [<c01e7409>] _raw_spin_lock+0x69/0x140
 [<c02c01cc>] _spin_lock_irqsave+0xc/0x20
 [<c0117c9b>] __wake_up+0x1b/0x50
 [<c024badc>] sock_def_write_space+0x9c/0xc0
 [<c024bdcc>] sock_wfree+0x3c/0x50
 [<c024dfc7>] skb_release_all+0x57/0x130
 [<c024d75b>] __kfree_skb+0xb/0x90
 [<c024d7f9>] kfree_skb+0x19/0x40
 [<c8c49e33>] ax25_frames_acked+0x43/0x70 [ax25]
 [<c8c47a30>] ax25_check_iframes_acked+0x40/0x60 [ax25]
 [<c8c4949f>] ax25_std_frame_in+0x6af/0x890 [ax25]
 [<c8c46dbb>] ax25_kiss_rcv+0x2bb/0x7c0 [ax25]
 [<c02531eb>] netif_receive_skb+0x33b/0x420
 [<c0255c1b>] process_backlog+0x6b/0xd0
 [<c025573f>] net_rx_action+0xff/0x1a0
 [<c0122864>] __do_softirq+0x84/0x100
 [<c012291b>] do_softirq+0x3b/0x50
 [<c0122b69>] irq_exit+0x79/0x90
 [<c0106d46>] do_IRQ+0x46/0x80
 [<c0104c67>] common_interrupt+0x23/0x28
 [<c01025f6>] cpu_idle+0x76/0xa0
 [<c02bd8e9>] rest_init+0x49/0x50
 [<c037dd55>] start_kernel+0x2b5/0x330
 [<00000000>] 0x0
 =======================
Code: 00 64 8b 0d 08 f0 3a c0 89 44 24 0c 89 4c 24 08 89 54 24 10 89 7c 24 04 c7 04 24 24 37 33 c0 e8 5c 6f f3 ff 85 db 8b 4e 08 74 83 <8b> 93 c4 00 00 00 8d 83 c9 01 00 00 e9 7c ff ff ff 8d 74 26 00
EIP: [<c01e730b>] spin_bug+0xdb/0xf0 SS:ESP 0068:c0379c54
Kernel panic - not syncing: Fatal exception in interrupt

On Thu, Mar 20, 2008 at 11:58:35PM +0100, Bernard Pidoux wrote:
...

I wanted to see something more on the stack than lockdep's code only. Will see... It seems some memory debugging is still on and doing this poisoning with '6b', but I can't check now what it is. (Sometimes this could omit some warnings).

Thanks,
Jarek P.

The only difference I can see compared to the original spinlock report is this line :

[<c01e7409>] _raw_spin_lock+0x69/0x140

Regards,

Bernard P.

BUG: spinlock bad magic on CPU#0, ax25ipd/3648
BUG: unable to handle kernel paging request at virtual address 6b6b6c2f
printing eip: c01e730b *pde = 00000000
Oops: 0000 [#1] SMP
Modules linked in: rose netrom mkiss crc16 ax25 mga drm netconsole configfs nfsd exportfs nfs lockd nfs_acl sunrpc af_packet ipv6 snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_pcm_oss snd_mixer_oss binfmt_misc loop usb_storage scsi_mod floppy usbhid snd_ens1371 snd_rawmidi snd_seq_device snd_ac97_codec snd_pcm snd_timer snd uhci_hcd soundcore snd_page_alloc usbcore ne2k_pci ide_cd 8390 ac97_bus intel_agp agpgart genrtc ext3 jbd ide_disk piix ide_core

Pid: 3648, comm: ax25ipd Not tainted (2.6.24.3 #8)
EIP: 0060:[<c01e730b>] EFLAGS: 00010002 CPU: 0
EIP is at spin_bug+0xdb/0xf0
EAX: 00000032 EBX: 6b6b6b6b ECX: 6b6b6b6b EDX: 00000096
ESI: c0e4bc30 EDI: c0333633 EBP: c5c7bbc4 ESP: c5c7bba0
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process ax25ipd (pid: 3648, ti=c5c7a000 task=c6abec90 task.ti=c5c7a000)

Stack: c0333724 c0333633 00000000 c6abee59 00000e40 00000007 c0e4bc30 00000001
       c697ebf8 c5c7bbfc c01e7409 00000000 00000001 c0180640 00000000 
00000000
       00000003 c0a320b8 00000000 c6abec90 00000282 00000001 c697ebf8 
c5c7bc08
Call Trace:
 [<c010527a>] show_trace_log_lvl+0x1a/0x30
 [<c0105339>] show_stack_log_lvl+0xa9/0xd0
 [<c0105427>] show_registers+0xc7/0x1b0
 [<c010561a>] die+0x10a/0x230
 [<c0114855>] do_page_fault+0x2f5/0x600
 [<c02c039a>] error_code+0x72/0x78
 [<c01e7409>] _raw_spin_lock+0x69/0x140
 [<c02c01cc>] _spin_lock_irqsave+0xc/0x20
 [<c0117c9b>] __wake_up+0x1b/0x50
 [<c024badc>] sock_def_write_space+0x9c/0xc0
 [<c024bdcc>] sock_wfree+0x3c/0x50
 [<c024dfc7>] skb_release_all+0x57/0x130
 [<c024d75b>] __kfree_skb+0xb/0x90
 [<c024d7f9>] kfree_skb+0x19/0x40
 [<c8c49e33>] ax25_frames_acked+0x43/0x70 [ax25]
 [<c8c47a30>] ax25_check_iframes_acked+0x40/0x60 [ax25]
 [<c8c4949f>] ax25_std_frame_in+0x6af/0x890 [ax25]
 [<c8c46dbb>] ax25_kiss_rcv+0x2bb/0x7c0 [ax25]
 [<c02531eb>] netif_receive_skb+0x33b/0x420
 [<c0255c1b>] process_backlog+0x6b/0xd0
 [<c025573f>] net_rx_action+0xff/0x1a0
 [<c0122864>] __do_softirq+0x84/0x100
 [<c012291b>] do_softirq+0x3b/0x50
 [<c0122ae5>] local_bh_enable_ip+0x45/0x50
 [<c02c0050>] _spin_unlock_bh+0x10/0x20
 [<c8c396ca>] mkiss_receive_buf+0x36a/0x420 [mkiss]
 [<c020f037>] pty_write+0x47/0x60
 [<c020c5f9>] write_chan+0x229/0x330
 [<c0209c98>] tty_write+0x128/0x1c0
 [<c017346f>] vfs_write+0xaf/0x120
 [<c0173aad>] sys_write+0x3d/0x70
 [<c0104256>] sysenter_past_esp+0x5f/0x85
 =======================
Code: 00 64 8b 0d 08 f0 3a c0 89 44 24 0c 89 4c 24 08 89 54 24 10 89 7c 24 04 c7 04 24 24 37 33 c0 e8 5c 6f f3 ff 85 db 8b 4e 08 74 83 <8b> 93 c4 00 00 00 8d 83 c9 01 00 00 e9 7c ff ff ff 8d 74 26 00
EIP: [<c01e730b>] spin_bug+0xdb/0xf0 SS:ESP 0068:c5c7bba0
Kernel panic - not syncing: Fatal exception in interrupt

On Fri, Mar 21, 2008 at 10:52:27AM +0100, Bernard Pidoux wrote:
...

Bernard, thanks for these reports. I simply need more time to find various places in ax25 which could matter here. Of course any additional information could help, but these reports mostly repeat the same. BTW, I wonder if it's possible to run this command for closing the session which causes the bug with strace?:
strace -o strace.log a_command

Regards,
Jarek P.

Hi Jarek,

Please ignore the spinlock report I sent in this previous message. I simply forgot to apply back your mkiss ax_bump() patch when doing some trials.

After applying again your patch and turning off the debug you asked for, I observe exactly the same spinlock I first reported i.e. involving ax25ipd.

Sorry for the confusion.

Regards,

Bernard

On Thu, Mar 20, 2008 at 11:58:35PM +0100, Bernard Pidoux wrote:
...

...

...

Hi,

Bernard, could you try this debugging patch?

Thanks,
Jarek P.

---

diff -Nurp 2.6.24.3-/net/core/sock.c 2.6.24.3+/net/core/sock.c
@@ -1517,14 +1517,21 @@ static void sock_def_write_space(struct 
 	 * progress.  --DaveM
 	 */
 	if ((atomic_read(&sk->sk_wmem_alloc) << 1) <= sk->sk_sndbuf) {
-		if (sk->sk_sleep && waitqueue_active(sk->sk_sleep))
+		if (sk->sk_sleep && waitqueue_active(sk->sk_sleep)) {
+			if (!sk->sk_socket || sk->sk_socket->sk != sk) {
+				WARN_ON_ONCE(1);
+				sk->sk_socket = NULL;
+				sk->sk_sleep = NULL;
+				goto out;
+			}
 			wake_up_interruptible(sk->sk_sleep);
+		}
 
 		/* Should agree with poll, otherwise some programs break */
 		if (sock_writeable(sk))
 			sk_wake_async(sk, 2, POLL_OUT);
 	}
-
+out:
 	read_unlock(&sk->sk_callback_lock);
 }
 

On Sun, Mar 23, 2008 at 06:00:17PM +0100, Bernard Pidoux wrote:
...

...

Thanks. Here is one more (to apply after the previous one).

Happy Holidays...
Jarek P.

---

diff -Nurp 2.6.24.3-with1/include/net/sock.h 2.6.24.3-with2/include/net/sock.h
@@ -263,6 +263,7 @@ struct sock {
   	int			(*sk_backlog_rcv)(struct sock *sk,
 						  struct sk_buff *skb);  
 	void                    (*sk_destruct)(struct sock *sk);
+	int			sk_ax25_debug;
 };
 
 /*

diff -Nurp 2.6.24.3-with1/net/ax25/af_ax25.c 2.6.24.3-with2/net/ax25/af_ax25.c
@@ -936,6 +936,7 @@ struct sock *ax25_make_new(struct sock *
 	}
 
 	sk->sk_protinfo = ax25;
+	sk->sk_ax25_debug = 1;
 	ax25->sk    = sk;
 
 	return sk;

diff -Nurp 2.6.24.3-with1/net/core/sock.c 2.6.24.3-with2/net/core/sock.c
@@ -1519,6 +1519,9 @@ static void sock_def_write_space(struct 
 	if ((atomic_read(&sk->sk_wmem_alloc) << 1) <= sk->sk_sndbuf) {
 		if (sk->sk_sleep && waitqueue_active(sk->sk_sleep)) {
 			if (!sk->sk_socket || sk->sk_socket->sk != sk) {
+				if (net_ratelimit())
+					printk(KERN_INFO "sk_ax25_debug = %d, %p\n",
+						 sk->sk_ax25_debug, sk->sk_socket);
 				WARN_ON_ONCE(1);
 				sk->sk_socket = NULL;
 				sk->sk_sleep = NULL;

On Tue, Mar 25, 2008 at 02:23:46PM +0100, Bernard Pidoux wrote:
...

I need some time yet; it looks like 2.6.24.4 doesn't collide with these patches, but upgrade could be reasonable for other reasons.

Best regards,
Jarek P.

On Tue, Mar 25, 2008 at 02:23:46PM +0100, Bernard Pidoux wrote:
...

...

OK, here is the next try. (Apply after all previous patches to any 2.6.24.x; there would be more noise, so revert it after catching some data.)

Thanks,
Jarek P.

(debugging patch #3)
---

diff -Nurp 2.6.24.3-with2/include/net/sock.h 2.6.24.3-with3/include/net/sock.h
@@ -1414,4 +1414,20 @@ extern int sysctl_optmem_max;
 extern __u32 sysctl_wmem_default;
 extern __u32 sysctl_rmem_default;
 
+static inline int sock_debug_ax25(struct sk_buff *skb, struct sock *ax25_sk)
+{
+	struct sock *sk = skb->sk;
+
+	if (sk && sk->sk_socket && sk->sk_socket->sk != sk) {
+		if (net_ratelimit())
+			printk(KERN_INFO "AX25: %s, %d, %p, %p, %p, %p, %p\n",
+				__FUNCTION__, __LINE__, sk, sk->sk_socket,
+				sk->sk_socket->sk, ax25_sk,
+				ax25_sk == NULL ? NULL : ax25_sk->sk_socket);
+		skb->destructor = NULL;
+		return 1;
+	}
+	return 0;
+}
+
 #endif	/* _SOCK_H */

diff -Nurp 2.6.24.3-with2/net/ax25/af_ax25.c 2.6.24.3-with3/net/ax25/af_ax25.c
@@ -952,6 +952,8 @@ static int ax25_release(struct socket *s
 
 	sock_hold(sk);
 	sock_orphan(sk);
+	if (net_ratelimit())
+		printk(KERN_INFO "AX25- %s, %d, %p, %p\n", __FUNCTION__, __LINE__, sk, sock);
 	lock_sock(sk);
 	ax25 = ax25_sk(sk);
 

@@ -1372,12 +1374,16 @@ static int ax25_accept(struct socket *so
 	newsk		 = skb->sk;
 	newsk->sk_socket = newsock;
 	newsk->sk_sleep	 = &newsock->wait;
+	newsk->sk_ax25_debug = 2;
 
 	/* Now attach up the new socket */
 	kfree_skb(skb);
 	sk->sk_ack_backlog--;
 	newsock->sk    = newsk;
 	newsock->state = SS_CONNECTED;
+	if (net_ratelimit())
+		printk(KERN_INFO "AX25= %s, %d, %p, %p, %p\n",
+			 __FUNCTION__, __LINE__, newsk, sock, newsock);
 
 out:
 	release_sock(sk);

diff -Nurp 2.6.24.3-with2/net/ax25/ax25_out.c 2.6.24.3-with3/net/ax25/ax25_out.c
@@ -98,6 +98,7 @@ ax25_cb *ax25_send_frame(struct sk_buff 
 
 	ax25_start_heartbeat(ax25);
 
+	sock_debug_ax25(skb, ax25->sk);
 	ax25_output(ax25, paclen, skb);
 
 	return ax25;			/* We had to create it */

@@ -176,11 +177,13 @@ void ax25_output(ax25_cb *ax25, int pacl
 			}
 
 			skb_pull(skb, len);
+			sock_debug_ax25(skbn, ax25->sk);
 			skb_queue_tail(&ax25->write_queue, skbn); /* Throw it on the queue */
 		}
 
 		kfree_skb(skb);
 	} else {
+		sock_debug_ax25(skb, ax25->sk);
 		skb_queue_tail(&ax25->write_queue, skb);	  /* Throw it on the queue */
 	}
 

@@ -274,6 +277,8 @@ void ax25_kick(ax25_cb *ax25)
 	ax25->vs = start;
 
 	do {
+		sock_debug_ax25(skb, ax25->sk);
+
 		if ((skbn = skb_clone(skb, GFP_ATOMIC)) == NULL) {
 			skb_queue_head(&ax25->write_queue, skb);
 			break;

@@ -308,6 +313,8 @@ void ax25_kick(ax25_cb *ax25)
 		/*
 		 * Requeue the original data frame.
 		 */
+
+		sock_debug_ax25(skb, ax25->sk);
 		skb_queue_tail(&ax25->ack_queue, skb);
 
 	} while (!last && (skb = skb_dequeue(&ax25->write_queue)) != NULL);

diff -Nurp 2.6.24.3-with2/net/ax25/ax25_subr.c 2.6.24.3-with3/net/ax25/ax25_subr.c
@@ -56,6 +56,7 @@ void ax25_frames_acked(ax25_cb *ax25, un
 	if (ax25->va != nr) {
 		while (skb_peek(&ax25->ack_queue) != NULL && ax25->va != nr) {
 			skb = skb_dequeue(&ax25->ack_queue);
+			sock_debug_ax25(skb, ax25->sk);
 			kfree_skb(skb);
 			ax25->va = (ax25->va + 1) % ax25->modulus;
 		}

diff -Nurp 2.6.24.3-with2/net/core/sock.c 2.6.24.3-with3/net/core/sock.c
@@ -1520,8 +1520,10 @@ static void sock_def_write_space(struct 
 		if (sk->sk_sleep && waitqueue_active(sk->sk_sleep)) {
 			if (!sk->sk_socket || sk->sk_socket->sk != sk) {
 				if (net_ratelimit())
-					printk(KERN_INFO "sk_ax25_debug = %d, %p\n",
-						 sk->sk_ax25_debug, sk->sk_socket);
+					printk(KERN_INFO "sk_ax25_debug = %d, %d, %d, %d, %p, %p, %p\n",
+						 sk->sk_ax25_debug, sk->sk_family,
+						 (int)sk->sk_protocol, sk->sk_type, sk->sk_socket,
+						 sk->sk_socket == NULL ? NULL : sk->sk_socket->sk, sk);
 				WARN_ON_ONCE(1);
 				sk->sk_socket = NULL;
 				sk->sk_sleep = NULL;

Hi Jarek,

Thank you for the new debugging patch.
I think I have collected enough data for you to make some interesting deductions.

Here are the data collected and filtered with

cat /var/log/kernel/info.log | grep AX25
I have noticed that there is no AX25= result, although the system ran a few hours.

Mar 27 15:50:49 f6bvp-9 kernel: AX25- ax25_release, 952, c340fad0, c513a820
Mar 27 15:50:54 f6bvp-9 kernel: AX25- ax25_release, 952, c0f07b10, c0cc0820
Mar 27 15:50:55 f6bvp-9 kernel: AX25- ax25_release, 952, c34676d8, c0cc0d60
Mar 27 15:50:55 f6bvp-9 kernel: AX25- ax25_release, 952, c340e240, c0cc0820
Mar 27 15:50:55 f6bvp-9 kernel: AX25- ax25_release, 952, c0f07b10, c0cc02e0
Mar 27 15:50:55 f6bvp-9 kernel: AX25- ax25_release, 952, c0f07b10, c0cc0d60
Mar 27 15:50:55 f6bvp-9 kernel: AX25- ax25_release, 952, c0f07b10, c0cc02e0
Mar 27 15:51:17 f6bvp-9 kernel: AX25- ax25_release, 952, c0fc4ad0, c148aac0
Mar 27 15:51:22 f6bvp-9 kernel: AX25- ax25_release, 952, c0d482c0, c148a2e0
Mar 27 15:51:22 f6bvp-9 kernel: AX25- ax25_release, 952, c0805758, c148a580
Mar 27 15:51:22 f6bvp-9 kernel: AX25- ax25_release, 952, c0804f28, c148a820
Mar 27 15:51:22 f6bvp-9 kernel: AX25- ax25_release, 952, c08046f8, c0cfdd60
Mar 27 15:51:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0804f28, c148a820
Mar 27 15:51:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0d48af0, c148a820
Mar 27 15:51:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0fc4ad0, c148aac0
Mar 27 15:51:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0804f28, c148a820
Mar 27 15:51:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0804f28, c148a040
Mar 27 15:51:37 f6bvp-9 kernel: AX25- ax25_release, 952, c0fc4ad0, c148aac0
Mar 27 15:51:47 f6bvp-9 kernel: AX25- ax25_release, 952, c0fc4ad0, c148aac0
Mar 27 15:51:47 f6bvp-9 kernel: AX25- ax25_release, 952, c0d486d8, c0cfdd60
Mar 27 15:52:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0d486d8, c0cfdd60
Mar 27 15:52:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0d486d8, c0cfdd60
Mar 27 15:53:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0fc4ad0, c0cfd820
Mar 27 15:53:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0fc4ad0, c0cfd820
Mar 27 15:54:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0fc4ad0, c0cfdd60
Mar 27 15:54:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0fc4ad0, c0cfdd60
Mar 27 15:55:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0d49738, c08922e0
Mar 27 15:55:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0d49738, c08922e0
Mar 27 15:56:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0d49738, c0892580
Mar 27 15:57:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0d49738, c0892580
Mar 27 15:58:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0d49738, c08922e0
Mar 27 15:58:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0d49738, c08922e0
Mar 27 15:58:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0d49738, c08922e0
Mar 27 15:58:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0d49738, c08922e0
Mar 27 15:59:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0d49738, c0892580
Mar 27 16:02:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0d49738, c08922e0
Mar 27 16:02:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0d49738, c08922e0
Mar 27 16:03:07 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c7bb7640,
c0892580, 6b6b6b6b, 00000000, 00000000
Mar 27 16:03:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0d28718, c08922e0
Mar 27 16:03:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0d28718, c08922e0
Mar 27 16:04:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0d49738, c0892040
Mar 27 16:04:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0d49738, c0892040
Mar 27 16:05:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0d49738, c08922e0
Mar 27 16:06:27 f6bvp-9 kernel: AX25- ax25_release, 952, c0d49738, c08922e0
Mar 27 16:07:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c08922e0
Mar 27 16:07:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c08922e0
Mar 27 16:08:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892580
Mar 27 16:08:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892580
Mar 27 16:09:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8d380, c0892580
Mar 27 16:09:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8d380, c0892580
Mar 27 16:10:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2758, c0892580
Mar 27 16:11:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2758, c0892580
Mar 27 16:12:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892580
Mar 27 16:13:21 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c7bb6610,
c0892580, 6b6b6b6b, 00000000, 00000000
Mar 27 16:13:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c08922e0
Mar 27 16:13:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c08922e0
Mar 27 16:14:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892580
Mar 27 16:14:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892580
Mar 27 16:15:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 16:16:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 16:16:57 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c7bb6e28,
c08922e0, 6b6b6b6b, 00000000, 00000000
Mar 27 16:17:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 16:17:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 16:18:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892580
Mar 27 16:18:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892580
Mar 27 16:19:04 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c56c8f68,
c0892040, 6b6b6b6b, 00000000, 00000000
Mar 27 16:19:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892580
Mar 27 16:20:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892580
Mar 27 16:21:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892040
Mar 27 16:21:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892040
Mar 27 16:22:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892580
Mar 27 16:22:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892580
Mar 27 16:23:03 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c56c9780,
c0892040, 6b6b6b6b, 00000000, 00000000
Mar 27 16:23:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 16:23:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 16:24:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892040
Mar 27 16:24:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892040
Mar 27 16:25:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892580
Mar 27 16:28:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c08922e0
Mar 27 16:28:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c08922e0
Mar 27 16:29:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892580
Mar 27 16:29:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892580
Mar 27 16:30:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c08922e0
Mar 27 16:30:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c08922e0
Mar 27 16:31:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892580
Mar 27 16:32:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892580
Mar 27 16:33:05 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c7790190,
c08922e0, 6b6b6b6b, 00000000, 00000000
Mar 27 16:33:07 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c77989c8,
c0892580, 6b6b6b6b, 00000000, 00000000
Mar 27 16:33:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 16:34:06 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c77991e0,
c0892040, 6b6b6b6b, 00000000, 00000000
Mar 27 16:34:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c08922e0
Mar 27 16:35:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c08922e0
Mar 27 16:36:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892040
Mar 27 16:37:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892040
Mar 27 16:38:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892040
Mar 27 16:39:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892040
Mar 27 16:40:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c08922e0
Mar 27 16:41:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c08922e0
Mar 27 16:41:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c08922e0
Mar 27 16:42:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892580
Mar 27 16:42:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892580
Mar 27 16:43:04 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c2180510,
c08922e0, 6b6b6b6b, 00000000, 00000000
Mar 27 16:43:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892580
Mar 27 16:43:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892580
Mar 27 16:44:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c08922e0
Mar 27 16:44:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c08922e0
Mar 27 16:45:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 16:46:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 16:47:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c0892580
Mar 27 16:48:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892040
Mar 27 16:48:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892040
Mar 27 16:48:56 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c77981b0,
c08922e0, 6b6b6b6b, 00000000, 00000000
Mar 27 16:49:04 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c2181540,
c0892040, 6b6b6b6b, 00000000, 00000000
Mar 27 16:49:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c08922e0
Mar 27 16:49:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c08922e0
Mar 27 16:50:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892040
Mar 27 16:50:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892040
Mar 27 16:51:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8d798, c08922e0
Mar 27 16:51:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8d798, c08922e0
Mar 27 16:51:33 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892040
Mar 27 16:52:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8d798, c08922e0
Mar 27 16:52:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8d798, c08922e0
Mar 27 16:52:48 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892040
Mar 27 16:53:10 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c77909a8,
c0892580, 6b6b6b6b, 00000000, 00000000
Mar 27 16:53:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892040
Mar 27 16:53:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892040
Mar 27 16:54:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c0892040
Mar 27 16:55:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892040
Mar 27 16:55:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892040
Mar 27 16:56:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 16:57:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 16:58:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892040
Mar 27 16:58:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892040
Mar 27 16:59:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892580
Mar 27 16:59:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8d380, c0892580
Mar 27 17:00:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892040
Mar 27 17:00:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892040
Mar 27 17:00:56 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c77911c0,
c50c2d60, 6b6b6b6b, 00000000, 00000000
Mar 27 17:01:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c50c2d60
Mar 27 17:02:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c50c2d60
Mar 27 17:03:04 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c3432c88,
c50c2d60, 6b6b6b6b, 00000000, 00000000
Mar 27 17:03:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c08922e0
Mar 27 17:03:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c08922e0
Mar 27 17:04:04 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c3432470,
c0892580, 6b6b6b6b, 00000000, 00000000
Mar 27 17:04:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c08922e0
Mar 27 17:04:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c08922e0
Mar 27 17:05:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c0892040
Mar 27 17:05:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c0892040
Mar 27 17:06:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c08922e0
Mar 27 17:06:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c08922e0
Mar 27 17:07:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c0892580
Mar 27 17:10:19 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c2180d28,
c08922e0, 6b6b6b6b, 00000000, 00000000
Mar 27 17:10:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c0892580
Mar 27 17:10:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c0892580
Mar 27 17:11:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c08922e0
Mar 27 17:11:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c08922e0
Mar 27 17:12:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c0892580
Mar 27 17:12:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c0892580
Mar 27 17:13:08 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c0894770,
c08922e0, 6b6b6b6b, 00000000, 00000000
Mar 27 17:13:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c0892580
Mar 27 17:13:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c0892580
Mar 27 17:14:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c08922e0
Mar 27 17:14:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2b70, c08922e0
Mar 27 17:15:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 17:16:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892580
Mar 27 17:16:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892580
Mar 27 17:16:56 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c56c8750,
c08922e0, 6b6b6b6b, 00000000, 00000000
Mar 27 17:17:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892580
Mar 27 17:17:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892580
Mar 27 17:18:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:18:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:19:03 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c695b440,
c0892040, 6b6b6b6b, 00000000, 00000000
Mar 27 17:19:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:19:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:20:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892040
Mar 27 17:20:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892040
Mar 27 17:21:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:21:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:22:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892040
Mar 27 17:23:01 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c695a410,
c08922e0, 6b6b6b6b, 00000000, 00000000
Mar 27 17:23:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:23:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:24:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892040
Mar 27 17:24:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892040
Mar 27 17:25:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 17:25:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 17:26:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892040
Mar 27 17:26:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892040
Mar 27 17:27:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 17:28:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 17:29:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892040
Mar 27 17:30:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 17:30:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 17:31:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892040
Mar 27 17:31:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892040
Mar 27 17:32:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 17:32:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 17:32:57 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c67b34c0,
c08922e0, 6b6b6b6b, 00000000, 00000000
Mar 27 17:33:05 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c695ac28,
c0892040, 6b6b6b6b, 00000000, 00000000
Mar 27 17:33:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:33:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:34:04 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c67b2490,
c0892580, 6b6b6b6b, 00000000, 00000000
Mar 27 17:34:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:34:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:35:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892580
Mar 27 17:37:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:37:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:38:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892580
Mar 27 17:38:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892580
Mar 27 17:39:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 17:39:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 17:40:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892580
Mar 27 17:40:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892580
Mar 27 17:41:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 17:41:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 17:41:46 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c67b2ca8,
c0892580, 6b6b6b6b, 00000000, 00000000
Mar 27 17:42:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 17:42:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 17:43:08 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c3bcb520,
c0892580, 6b6b6b6b, 00000000, 00000000
Mar 27 17:43:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 17:44:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 17:45:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892580
Mar 27 17:45:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892580
Mar 27 17:46:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 17:46:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 17:47:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892580
Mar 27 17:47:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892580
Mar 27 17:48:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:48:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:48:56 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c3bcad08,
c0892040, 6b6b6b6b, 00000000, 00000000
Mar 27 17:49:04 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c3bca4f0,
c08922e0, 6b6b6b6b, 00000000, 00000000
Mar 27 17:49:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:49:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:50:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892040
Mar 27 17:51:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3466ea8, c08922e0
Mar 27 17:51:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3466ea8, c08922e0
Mar 27 17:51:34 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892040
Mar 27 17:52:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3466ea8, c0892580
Mar 27 17:52:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3466ea8, c0892580
Mar 27 17:52:49 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892040
Mar 27 17:53:02 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c7bc35a0,
c08922e0, 6b6b6b6b, 00000000, 00000000
Mar 27 17:53:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892040
Mar 27 17:54:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892040
Mar 27 17:55:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:55:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:56:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892040
Mar 27 17:58:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892040
Mar 27 17:59:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 17:59:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 18:00:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 18:00:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 18:00:57 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c7bc2570,
c0892040, 6b6b6b6b, 00000000, 00000000
Mar 27 18:01:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 18:01:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 18:02:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892040
Mar 27 18:02:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892040
Mar 27 18:03:04 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c7bc2d88,
c08922e0, 6b6b6b6b, 00000000, 00000000
Mar 27 18:03:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892040
Mar 27 18:03:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892040
Mar 27 18:04:04 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c6a3d5e0,
c0892580, 6b6b6b6b, 00000000, 00000000
Mar 27 18:04:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892040
Mar 27 18:05:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892040
Mar 27 18:06:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892580
Mar 27 18:06:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892580
Mar 27 18:07:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 18:07:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 18:08:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892580
Mar 27 18:11:27 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 18:12:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892580
Mar 27 18:12:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892580
Mar 27 18:13:07 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c6a3c5b0,
c08922e0, 6b6b6b6b, 00000000, 00000000
Mar 27 18:13:11 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c6a3cdc8,
c08922e0, 6b6b6b6b, 00000000, 00000000
Mar 27 18:13:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892580
Mar 27 18:13:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c0892580
Mar 27 18:14:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 18:14:28 f6bvp-9 kernel: AX25- ax25_release, 952, c340ea70, c08922e0
Mar 27 18:15:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c0892580
Mar 27 18:15:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c0892580
Mar 27 18:16:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c08922e0
Mar 27 18:16:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c08922e0
Mar 27 18:16:56 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c7bc5660,
c0892580, 6b6b6b6b, 00000000, 00000000
Mar 27 18:17:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c08922e0
Mar 27 18:17:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c08922e0
Mar 27 18:18:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 18:19:04 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c7bc4630,
c0892040, 6b6b6b6b, 00000000, 00000000
Mar 27 18:19:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 18:20:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 18:21:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892040
Mar 27 18:21:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892040
Mar 27 18:22:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 18:22:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 18:23:14 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c7bc4e48,
c0892040, 6b6b6b6b, 00000000, 00000000
Mar 27 18:23:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 18:23:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 18:24:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c0892040
Mar 27 18:26:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c08922e0
Mar 27 18:26:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c08922e0
Mar 27 18:27:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c0892040
Mar 27 18:28:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c0892040
Mar 27 18:29:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c0892580
Mar 27 18:30:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c0892580
Mar 27 18:31:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c0892040
Mar 27 18:31:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c0892040
Mar 27 18:32:27 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c0892580
Mar 27 18:32:59 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c3bd0ee8,
c0892580, 6b6b6b6b, 00000000, 00000000
Mar 27 18:33:04 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c3bd1700,
c0892040, 6b6b6b6b, 00000000, 00000000
Mar 27 18:33:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 18:36:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 18:37:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 18:38:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 18:40:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 18:40:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 18:41:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 18:41:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 18:42:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 18:42:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 18:43:12 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c3bd06d0,
c0892580, 6b6b6b6b, 00000000, 00000000
Mar 27 18:43:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 18:43:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 18:44:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 18:45:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 18:46:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 18:47:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 18:47:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 18:48:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c08922e0
Mar 27 18:48:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c08922e0
Mar 27 18:48:57 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c67bb7a0,
c0892040, 6b6b6b6b, 00000000, 00000000
Mar 27 18:49:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c08922e0
Mar 27 18:50:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c08922e0
Mar 27 18:51:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3466ea8, c08922e0
Mar 27 18:51:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3466ea8, c08922e0
Mar 27 18:51:36 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c0892040
Mar 27 18:52:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3466ea8, c0892580
Mar 27 18:52:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3466ea8, c0892580
Mar 27 18:52:51 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c0892040
Mar 27 18:53:02 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c67ba770,
c08922e0, 6b6b6b6b, 00000000, 00000000
Mar 27 18:53:27 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c0892040
Mar 27 18:54:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c08922e0
Mar 27 18:54:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c08922e0
Mar 27 18:55:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c0892040
Mar 27 18:56:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c0892040
Mar 27 18:57:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c08922e0
Mar 27 18:58:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c08922e0
Mar 27 18:59:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c0892040
Mar 27 18:59:28 f6bvp-9 kernel: AX25- ax25_release, 952, c34672c0, c0892040
Mar 27 19:00:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c50c2d60
Mar 27 19:00:58 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c0a43080,
c50c2d60, 6b6b6b6b, 00000000, 00000000
Mar 27 19:01:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c50c2d60
Mar 27 19:02:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c50c2d60
Mar 27 19:03:05 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c0a42868,
c50c2d60, 6b6b6b6b, 00000000, 00000000
Mar 27 19:03:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892040
Mar 27 19:03:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892040
Mar 27 19:04:05 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c0a42050,
c0892580, 6b6b6b6b, 00000000, 00000000
Mar 27 19:04:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892040
Mar 27 19:04:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892040
Mar 27 19:05:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 19:05:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 19:06:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 19:06:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 19:07:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 19:08:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 19:09:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 19:10:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 19:13:08 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c67baf88,
c08922e0, 6b6b6b6b, 00000000, 00000000
Mar 27 19:13:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 19:13:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892580
Mar 27 19:14:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 19:15:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892580
Mar 27 19:16:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892580
Mar 27 19:16:57 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c7bb3140,
c08922e0, 6b6b6b6b, 00000000, 00000000
Mar 27 19:17:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892580
Mar 27 19:17:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892580
Mar 27 19:18:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c08922e0
Mar 27 19:18:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c08922e0
Mar 27 19:19:04 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c7bb2110,
c0892040, 6b6b6b6b, 00000000, 00000000
Mar 27 19:19:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c08922e0
Mar 27 19:19:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c08922e0
Mar 27 19:20:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892040
Mar 27 19:22:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892040
Mar 27 19:23:20 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c7bb2928,
c08922e0, 6b6b6b6b, 00000000, 00000000
Mar 27 19:23:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892040
Mar 27 19:23:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892040
Mar 27 19:24:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892040
Mar 27 19:24:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892040
Mar 27 19:25:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c08922e0
Mar 27 19:28:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892040
Mar 27 19:29:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3467af0, c0892040
Mar 27 19:30:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892040
Mar 27 19:30:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c0892040
Mar 27 19:31:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 19:31:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 19:32:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892040
Mar 27 19:32:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892040
Mar 27 19:32:57 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c6a9a9c8,
c509c040, 6b6b6b6b, 00000000, 00000000
Mar 27 19:33:01 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c6a9b1e0,
c0892040, 6b6b6b6b, 00000000, 00000000
Mar 27 19:33:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892040
Mar 27 19:33:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892040
Mar 27 19:34:04 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c6a9a1b0,
c0892580, 6b6b6b6b, 00000000, 00000000
Mar 27 19:34:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892040
Mar 27 19:34:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892040
Mar 27 19:35:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892580
Mar 27 19:36:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892040
Mar 27 19:37:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892040
Mar 27 19:38:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892580
Mar 27 19:38:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892580
Mar 27 19:39:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892580
Mar 27 19:39:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892580
Mar 27 19:40:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892040
Mar 27 19:40:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892040
Mar 27 19:41:23 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c3bd3240,
c0892580, 6b6b6b6b, 00000000, 00000000
Mar 27 19:41:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892040
Mar 27 19:43:13 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c3bd2a28,
c0892040, 6b6b6b6b, 00000000, 00000000
Mar 27 19:43:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892040
Mar 27 19:44:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892040
Mar 27 19:45:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3466ea8, c0892040
Mar 27 19:45:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3466ea8, c0892040
Mar 27 19:46:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3466ea8, c509c2e0
Mar 27 19:46:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3466ea8, c509c2e0
Mar 27 19:47:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3466ea8, c0892580
Mar 27 19:47:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3466ea8, c0892580
Mar 27 19:48:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c2e0
Mar 27 19:48:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c2e0
Mar 27 19:48:57 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c3bd2210,
c509c040, 6b6b6b6b, 00000000, 00000000
Mar 27 19:49:04 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c1a032a0,
c509c040, 6b6b6b6b, 00000000, 00000000
Mar 27 19:49:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c2e0
Mar 27 19:50:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c040
Mar 27 19:50:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c040
Mar 27 19:51:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8d380, c509c2e0
Mar 27 19:51:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8d380, c509c2e0
Mar 27 19:51:37 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c040
Mar 27 19:52:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8d380, c509c2e0
Mar 27 19:52:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8d380, c509c2e0
Mar 27 19:52:52 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c040
Mar 27 19:53:11 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c1a02a88,
c509c2e0, 6b6b6b6b, 00000000, 00000000
Mar 27 19:53:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c040
Mar 27 19:53:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c040
Mar 27 19:54:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c2e0
Mar 27 19:54:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c2e0
Mar 27 19:55:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c040
Mar 27 19:55:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c040
Mar 27 19:56:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c2e0
Mar 27 19:57:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c040
Mar 27 20:00:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892040
Mar 27 20:00:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892040
Mar 27 20:00:57 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c1a02270,
c0892580, 6b6b6b6b, 00000000, 00000000
Mar 27 20:01:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892040
Mar 27 20:01:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892040
Mar 27 20:02:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892580
Mar 27 20:02:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892580
Mar 27 20:03:03 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c1a05320,
c0892580, 6b6b6b6b, 00000000, 00000000
Mar 27 20:03:27 f6bvp-9 kernel: AX25- ax25_release, 952, c3466ea8, c509c2e0
Mar 27 20:04:04 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c1a042f0,
c509c040, 6b6b6b6b, 00000000, 00000000
Mar 27 20:04:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3466ea8, c509c2e0
Mar 27 20:04:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3466ea8, c509c2e0
Mar 27 20:05:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3466ea8, c509c040
Mar 27 20:08:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3466ea8, c509c2e0
Mar 27 20:08:28 f6bvp-9 kernel: AX25- ax25_release, 952, c3466ea8, c509c2e0
Mar 27 20:09:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c2e0
Mar 27 20:09:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c2e0
Mar 27 20:10:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c040
Mar 27 20:11:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c2e0
Mar 27 20:11:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c2e0
Mar 27 20:11:42 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c1a04b08,
c509c040, 6b6b6b6b, 00000000, 00000000
Mar 27 20:12:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c2e0
Mar 27 20:12:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c509c2e0
Mar 27 20:13:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cf68, c0892580
Mar 27 20:13:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cf68, c0892580
Mar 27 20:14:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cf68, c0892040
Mar 27 20:14:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cf68, c0892040
Mar 27 20:15:16 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c1a18370,
c509c040, 6b6b6b6b, 00000000, 00000000
Mar 27 20:15:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2758, c509c040
Mar 27 20:16:28 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa2758, c509c040
Mar 27 20:17:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa37b8, c0892580
Mar 27 20:17:46 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c1a18b88,
c509c040, 6b6b6b6b, 00000000, 00000000
Mar 27 20:18:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7b28d48, c0892580
Mar 27 20:18:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7b28d48, c0892580
Mar 27 20:19:04 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c7b9d400,
c0892040, 6b6b6b6b, 00000000, 00000000
Mar 27 20:19:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892580
Mar 27 20:19:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892580
Mar 27 20:20:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7b28d48, c0892040
Mar 27 20:20:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7b28d48, c0892040
Mar 27 20:21:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892580
Mar 27 20:21:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892580
Mar 27 20:22:28 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8c738, c0892040

Regards,

Bernard P.

Hi Jarek,

Here is the result of cat /var/log/messages | grep -B1 -A50 core/sock No event was detected before 16:33.

--

Mar 27 16:33:05 f6bvp-9 kernel: sk_ax25_debug = 0, 11, 0, 5, c08922e0,
6b6b6b6b, c7790190
Mar 27 16:33:05 f6bvp-9 kernel: WARNING: at net/core/sock.c:1561
sock_def_write_space()
Mar 27 16:33:05 f6bvp-9 kernel: Pid: 3454, comm: mkiss Not tainted
2.6.24.4 #4
Mar 27 16:33:05 f6bvp-9 kernel:  [<c010513a>] show_trace_log_lvl+0x1a/0x30
Mar 27 16:33:05 f6bvp-9 kernel:  [<c0105b52>] show_trace+0x12/0x20
Mar 27 16:33:05 f6bvp-9 kernel:  [<c01064ac>] dump_stack+0x6c/0x80
Mar 27 16:33:05 f6bvp-9 kernel:  [<c024d68b>]
sock_def_write_space+0x17b/0x190
Mar 27 16:33:05 f6bvp-9 kernel:  [<c024da7a>] sock_wfree+0x3a/0x40
Mar 27 16:33:05 f6bvp-9 kernel:  [<c024fc3d>] skb_release_all+0x4d/0xf0
Mar 27 16:33:05 f6bvp-9 kernel:  [<c024f3eb>] __kfree_skb+0xb/0x90
Mar 27 16:33:05 f6bvp-9 kernel:  [<c024f489>] kfree_skb+0x19/0x30
Mar 27 16:33:05 f6bvp-9 kernel:  [<c8cbfe37>] ax_xmit+0xf7/0x360 [mkiss]
Mar 27 16:33:05 f6bvp-9 kernel:  [<c02554a7>]
dev_hard_start_xmit+0x207/0x270
Mar 27 16:33:05 f6bvp-9 kernel:  [<c0265934>] __qdisc_run+0x54/0x1a0
Mar 27 16:33:05 f6bvp-9 kernel:  [<c0257cec>] dev_queue_xmit+0x25c/0x330
Mar 27 16:33:05 f6bvp-9 kernel:  [<c8befa77>] ax25_queue_xmit+0x47/0x70
[ax25]
Mar 27 16:33:05 f6bvp-9 kernel:  [<c8befb65>]
ax25_transmit_buffer+0xc5/0x110 [ax25]
Mar 27 16:33:05 f6bvp-9 kernel:  [<c8befc2d>]
ax25_send_iframe+0x7d/0x110 [ax25]
Mar 27 16:33:06 f6bvp-9 kernel:  [<c8befd70>] ax25_kick+0xb0/0x2b0 [ax25]
Mar 27 16:33:06 f6bvp-9 kernel:  [<c8bf10c9>]
ax25_std_frame_in+0x69/0x890 [ax25]
Mar 27 16:33:06 f6bvp-9 kernel:  [<c8beedba>] ax25_kiss_rcv+0x2ba/0x7b0
[ax25]
Mar 27 16:33:06 f6bvp-9 kernel:  [<c0254e04>] netif_receive_skb+0x244/0x2e0
Mar 27 16:33:06 f6bvp-9 kernel:  [<c0257975>] process_backlog+0x75/0xe0
Mar 27 16:33:06 f6bvp-9 kernel:  [<c025748b>] net_rx_action+0x12b/0x1d0
Mar 27 16:33:06 f6bvp-9 kernel:  [<c011d5c5>] __do_softirq+0x55/0xc0
Mar 27 16:33:06 f6bvp-9 kernel:  [<c011d677>] do_softirq+0x47/0x50
Mar 27 16:33:06 f6bvp-9 kernel:  [<c011da25>] local_bh_enable_ip+0x95/0xe0
Mar 27 16:33:06 f6bvp-9 kernel:  [<c02bad05>] _spin_unlock_bh+0x25/0x30
Mar 27 16:33:06 f6bvp-9 kernel:  [<c8cbf6b0>]
mkiss_receive_buf+0x360/0x420 [mkiss]
Mar 27 16:33:06 f6bvp-9 kernel:  [<c020e097>] pty_write+0x47/0x60
Mar 27 16:33:06 f6bvp-9 kernel:  [<c020b662>] write_chan+0x222/0x330
Mar 27 16:33:06 f6bvp-9 kernel:  [<c0208d26>] tty_write+0x126/0x1b0
Mar 27 16:33:06 f6bvp-9 kernel:  [<c017298f>] vfs_write+0xaf/0x120
Mar 27 16:33:06 f6bvp-9 kernel:  [<c0172edd>] sys_write+0x3d/0x70
Mar 27 16:33:06 f6bvp-9 kernel:  [<c010412a>] sysenter_past_esp+0x5f/0xa5
Mar 27 16:33:06 f6bvp-9 kernel:  =======================
Mar 27 16:33:07 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c77989c8,
c0892580, 6b6b6b6b, 00000000, 00000000
Mar 27 16:33:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8cb50, c08922e0
Mar 27 16:33:27 f6bvp-9 last message repeated 3 times
Mar 27 16:34:06 f6bvp-9 kernel: AX25: sock_debug_ax25, 1424, c77991e0,
c0892040, 6b6b6b6b, 00000000, 00000000
Mar 27 16:34:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c08922e0
Mar 27 16:35:27 f6bvp-9 last message repeated 2 times
Mar 27 16:35:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c08922e0
Mar 27 16:36:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892040
Mar 27 16:37:27 f6bvp-9 last message repeated 2 times
Mar 27 16:37:27 f6bvp-9 kernel: AX25- ax25_release, 952, c6aa33a0, c0892040
Mar 27 16:38:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892040
Mar 27 16:38:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892040
Mar 27 16:39:01 f6bvp-9 crond[4300]: (root) CMD ([ -d /var/lib/php ] &&
find /var/lib/php/ -type f -mmin +$(/usr/lib/php/maxlifetime) -print0 |
xargs -r -0 rm)
Mar 27 16:39:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892040
Mar 27 16:39:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c0892040
Mar 27 16:40:27 f6bvp-9 kernel: AX25- ax25_release, 952, c7d8dbb0, c08922e0

Regards,

Bernard P.

On Wed, Mar 26, 2008 at 07:35:38PM +0100, Jarek Poplawski wrote:
...

Bernard, I fucked it up with this #3: (changed macro to function at the last minute). I'm currently reading this, and maybe I'll figure out something, but if it's not a big problem for you to repeat this after adding this #4 (after all previous) it should be 'a bit' easier. (But no need to hurry.)

Very sorry,
Jarek P.

(debugging patch #4)
---

diff -Nurp 2.6.24.3-with3/include/net/sock.h 2.6.24.3-with4/include/net/sock.h
@@ -1414,20 +1414,18 @@ extern int sysctl_optmem_max;
 extern __u32 sysctl_wmem_default;
 extern __u32 sysctl_rmem_default;
 
-static inline int sock_debug_ax25(struct sk_buff *skb, struct sock *ax25_sk)
-{
-	struct sock *sk = skb->sk;
-
-	if (sk && sk->sk_socket && sk->sk_socket->sk != sk) {
-		if (net_ratelimit())
-			printk(KERN_INFO "AX25: %s, %d, %p, %p, %p, %p, %p\n",
-				__FUNCTION__, __LINE__, sk, sk->sk_socket,
-				sk->sk_socket->sk, ax25_sk,
-				ax25_sk == NULL ? NULL : ax25_sk->sk_socket);
-		skb->destructor = NULL;
-		return 1;
-	}
-	return 0;
-}
+#define sock_debug_ax25(__skb, __ax25_sk) 					\
+({										\
+	struct sock *__sk = __skb->sk;						\
+										\
+	if (__sk && __sk->sk_socket && __sk->sk_socket->sk != __sk) {		\
+		if (net_ratelimit())						\
+			printk(KERN_INFO "AX25: %s, %d, %p, %p, %p, %p, %p\n",	\
+				__FUNCTION__, __LINE__, __sk, __sk->sk_socket,	\
+				__sk->sk_socket->sk, __ax25_sk,			\
+				__ax25_sk == NULL ? NULL : __ax25_sk->sk_socket);	\
+		__skb->destructor = NULL;					\
+	}									\
+})
 
 #endif	/* _SOCK_H */

diff -Nurp 2.6.24.3-with3/net/ax25/af_ax25.c 2.6.24.3-with4/net/ax25/af_ax25.c
@@ -866,6 +866,9 @@ static int ax25_create(struct net *net, 
 
 	ax25->sk    = sk;
 
+	if (net_ratelimit())
+		printk(KERN_INFO "AX25+ %s, %d, %p, %p\n", __FUNCTION__, __LINE__, sk, sock);
+
 	return 0;
 }
 

@@ -939,6 +942,10 @@ struct sock *ax25_make_new(struct sock *
 	sk->sk_ax25_debug = 1;
 	ax25->sk    = sk;
 
+	if (net_ratelimit())
+		printk(KERN_INFO "AX25; %s, %d, %p, %p\n",
+			 __FUNCTION__, __LINE__, sk, sk->sk_socket);
+
 	return sk;
 }
 

Please find below an extract of /var/log/messages
The time of interest is around 12:03

 Mar 29 11:52:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0f0a820
 Mar 29 11:52:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0f0a820
 Mar 29 11:52:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0f0a820
 Mar 29 11:52:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0f0a820
 Mar 29 11:52:59 f6bvp-9 kernel: AX25: ax25_frames_acked, 59, c6ae2bc8, 
c0f0a040, 6b6b6b6b, 00000000, 00000000
 Mar 29 11:53:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0f0a580
 Mar 29 11:53:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0f0a580
 Mar 29 11:53:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0f0a580
 Mar 29 11:53:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0f0a580
 Mar 29 11:54:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0f0a580
 Mar 29 11:54:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0f0a580
 Mar 29 11:54:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0f0a580
 Mar 29 11:54:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0f0a580
 Mar 29 11:55:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0f60820
 Mar 29 11:55:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0f60820
 Mar 29 11:55:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0f60820
 Mar 29 11:55:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0f60820
 Mar 29 11:56:52 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0f60820
 Mar 29 11:56:52 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0f60820
 Mar 29 11:56:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0f60820
 Mar 29 11:56:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0f60820
 Mar 29 11:56:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0f60820
 Mar 29 11:56:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0f60820
 Mar 29 11:56:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0f60820
 Mar 29 11:56:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0f60820
 Mar 29 11:57:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0f60820
 Mar 29 11:57:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0f60820
 Mar 29 11:57:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0f60820
 Mar 29 11:57:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0f60820
 Mar 29 11:58:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0f60820
 Mar 29 11:58:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0f60820
 Mar 29 11:58:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0f60820
 Mar 29 11:58:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0f60820
 Mar 29 11:59:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0f60820
 Mar 29 11:59:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0f60820
 Mar 29 11:59:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0f60820
 Mar 29 11:59:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0f60820
 Mar 29 12:00:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0d58040
 Mar 29 12:00:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0d58040
 Mar 29 12:00:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0d58040
 Mar 29 12:00:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0d58040
 Mar 29 12:01:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0d58040
 Mar 29 12:01:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0d58040
 Mar 29 12:01:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0d58040
 Mar 29 12:01:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0d58040
 Mar 29 12:02:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0f60820
 Mar 29 12:02:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0f60820
 Mar 29 12:02:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0f60820
 Mar 29 12:02:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0f60820
 Mar 29 12:03:07 f6bvp-9 kernel: AX25: ax25_kick, 279, c65e43f0, 
c0d58040, 6b6b6b6b, 00000000, 00000000
 Mar 29 12:03:07 f6bvp-9 kernel: sk_ax25_debug = 0, 11, 0, 5, c0d58040, 
6b6b6b6b, c65e43f0
 Mar 29 12:03:07 f6bvp-9 kernel: WARNING: at net/core/sock.c:1561 
sock_def_write_space()
 Mar 29 12:03:07 f6bvp-9 kernel: Pid: 3447, comm: mkiss Not tainted 
2.6.24.4 #6
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c010513a>] show_trace_log_lvl+0x1a/0x30
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c0105b52>] show_trace+0x12/0x20
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c01064ac>] dump_stack+0x6c/0x80
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c024d68b>] 
sock_def_write_space+0x17b/0x190
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c024da7a>] sock_wfree+0x3a/0x40
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c024fc3d>] skb_release_all+0x4d/0xf0
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c024f3eb>] __kfree_skb+0xb/0x90
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c024f489>] kfree_skb+0x19/0x30
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c8c6ae37>] ax_xmit+0xf7/0x360 [mkiss]
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c0255497>] 
dev_hard_start_xmit+0x207/0x270
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c0265924>] __qdisc_run+0x54/0x1a0
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c0257cdc>] dev_queue_xmit+0x25c/0x330
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c8c77a77>] 
ax25_queue_xmit+0x47/0x70 [ax25]
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c8c77b65>] 
ax25_transmit_buffer+0xc5/0x110 [ax25]
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c8c77c2d>] 
ax25_send_iframe+0x7d/0x110 [ax25]
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c8c77d70>] ax25_kick+0xb0/0x2a0 [ax25]
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c8c790a9>] 
ax25_std_frame_in+0x69/0x890 [ax25]
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c8c76dba>] 
ax25_kiss_rcv+0x2ba/0x7b0 [ax25]
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c0254df4>] 
netif_receive_skb+0x244/0x2e0
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c0257965>] process_backlog+0x75/0xe0
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c025747b>] net_rx_action+0x12b/0x1d0
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c011d5c5>] __do_softirq+0x55/0xc0
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c011d677>] do_softirq+0x47/0x50
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c011da25>] local_bh_enable_ip+0x95/0xe0
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c02bace5>] _spin_unlock_bh+0x25/0x30
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c8c6a6b0>] 
mkiss_receive_buf+0x360/0x420 [mkiss]
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c020e097>] pty_write+0x47/0x60
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c020b662>] write_chan+0x222/0x330
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c0208d26>] tty_write+0x126/0x1b0
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c017298f>] vfs_write+0xaf/0x120
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c0172edd>] sys_write+0x3d/0x70
 Mar 29 12:03:07 f6bvp-9 kernel:  [<c010412a>] sysenter_past_esp+0x5f/0xa5
 Mar 29 12:03:07 f6bvp-9 kernel:  =======================
 Mar 29 12:03:52 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d58040
 Mar 29 12:03:52 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d58040
 Mar 29 12:03:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d58040
 Mar 29 12:03:53 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d58040
 Mar 29 12:03:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d58040
 Mar 29 12:03:53 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d58040
 Mar 29 12:03:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d58040
 Mar 29 12:03:53 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d58040
 Mar 29 12:04:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d582e0
 Mar 29 12:04:53 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d582e0
 Mar 29 12:04:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d582e0
 Mar 29 12:04:53 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d582e0
 Mar 29 12:05:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d582e0
 Mar 29 12:05:53 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d582e0
 Mar 29 12:05:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d582e0
 Mar 29 12:05:53 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d582e0
 Mar 29 12:06:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d582e0
 Mar 29 12:06:53 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d582e0
 Mar 29 12:06:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d582e0
 Mar 29 12:06:53 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d582e0
 Mar 29 12:07:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d58040
 Mar 29 12:07:53 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d58040
 Mar 29 12:07:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d58040
 Mar 29 12:07:53 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d58040
 Mar 29 12:08:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d582e0
 Mar 29 12:08:53 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d582e0
 Mar 29 12:08:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d582e0
 Mar 29 12:08:53 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d582e0
 Mar 29 12:09:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d58040
 Mar 29 12:09:53 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d58040
 Mar 29 12:09:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d58040
 Mar 29 12:09:53 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d58040
 Mar 29 12:10:52 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d582e0
 Mar 29 12:10:52 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d582e0
 Mar 29 12:10:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d582e0
 Mar 29 12:10:53 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d582e0
 Mar 29 12:10:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d582e0
 Mar 29 12:10:53 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d582e0
 Mar 29 12:10:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d582e0
 Mar 29 12:10:53 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d582e0
 Mar 29 12:11:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d582e0
 Mar 29 12:11:53 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d582e0
 Mar 29 12:11:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c64f5a90, c0d582e0
 Mar 29 12:11:53 f6bvp-9 kernel: AX25- ax25_release, 959, c64f5a90, 
c0d582e0
 Mar 29 12:11:59 f6bvp-9 netromd[3557]: netromr: SIOCADDRT: Invalid 
argument
 Mar 29 12:12:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0d582e0
 Mar 29 12:12:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0d582e0
 Mar 29 12:12:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0d582e0
 Mar 29 12:12:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 
c0d582e0
 Mar 29 12:12:55 f6bvp-9 kernel: AX25: ax25_frames_acked, 59, c65e4c08, 
c0d58040, 6b6b6b6b, 00000000, 00000000
 Mar 29 12:12:57 f6bvp-9 kernel: AX25: ax25_frames_acked, 59, c7ba74a0, 
c0d582e0, 6b6b6b6b, 00000000, 00000000
 Mar 29 12:13:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c15602e0, c0d582e0
 Mar 29 12:13:53 f6bvp-9 kernel: AX25- ax25_release, 959, c15602e0, 

c0d582e0

Regards,

Bernard P.

On Sat, Mar 29, 2008 at 01:04:16PM +0100, Bernard Pidoux wrote:
...

...

Thanks Bernard! ...On the other hand, if you could hold these patches a while yet and maybe find more of these "AX25:" it should be very helpful (of course with other AX25* lines).

Regards,
Jarek P.

Hi Jarek,

please find attached another collection of data from /var/log/kernel/info.log that I bziped for it is rather large.

Bernard

Hi Bernard,

Thanks again. It looks like ROSE sockets aren't released properly. Here is a try to fix this. Could you test it with all previous patches present?

Regards,
Jarek P.

(patch #5)
---

diff -Nurp 2.6.24.4-/net/rose/af_rose.c 2.6.24.4+/net/rose/af_rose.c
@@ -599,17 +599,24 @@ static int rose_release(struct socket *s
 
 	if (sk == NULL) return 0;
 
+	sock_hold(sk);
+	sock_orphan(sk);
+	lock_sock(sk);
 	rose = rose_sk(sk);
 
 	switch (rose->state) {
 	case ROSE_STATE_0:
+		release_sock(sk);
 		rose_disconnect(sk, 0, -1, -1);
+		lock_sock(sk);
 		rose_destroy_socket(sk);
 		break;
 
 	case ROSE_STATE_2:
 		rose->neighbour->use--;
+		release_sock(sk);
 		rose_disconnect(sk, 0, -1, -1);
+		lock_sock(sk);
 		rose_destroy_socket(sk);
 		break;
 

@@ -634,6 +641,8 @@ static int rose_release(struct socket *s
 	}
 
 	sock->sk = NULL;
+	release_sock(sk);
+	sock_put(sk);
 
 	return 0;
 }

On Sun, Mar 30, 2008 at 07:38:51PM +0200, Bernard Pidoux wrote:
...

Yes, if it's only like this, it's OK.

No problem! Patch #3 added to "sk_ax25_debug =" sk->sk_family as the 2nd number, and 11 according to include/linux/socket.h means AF_ROSE.

sock_def_write() function in net/core/sock.c where the OOPs triggered runs some cleaning on sockets when their skbs are kfreed, but this shouldn't be done after socket had been closed (sock_orphan() removes some pointers). When ROSE sends it's skbs with ax25_send_frame(), they are added to AX25 queues, and kfreed after transmitting. If it's after ROSE socket had been released, but not properly updated, the cleaning is done on kfreed memory not used by the socket anymore.

Cheers,
Jarek P.

Jarek,

I enlarged grep filter in order to grab AX.25 data that might be interesting too.

And this is the result :

# cat /var/log/kernel/info.log | grep AX...
Mar 30 10:19:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e6e28, c74322e0
Mar 30 10:19:00 f6bvp-9 kernel: AX25- ax25_release, 959, c14e6e28, c74322e0
Mar 30 10:19:13 f6bvp-9 kernel: AX25: ax25_frames_acked, 59, c2aa4130, c74322e0, 6b6b6b6b, 00000000, 00000000
Mar 30 10:19:59 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e7a70, c74322e0
Mar 30 10:19:59 f6bvp-9 kernel: AX25- ax25_release, 959, c14e7a70, c74322e0
Mar 30 10:20:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e7a70, c74322e0
Mar 30 10:20:00 f6bvp-9 kernel: AX25- ax25_release, 959, c14e7a70, c74322e0
Mar 30 10:20:59 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e6e28, c74322e0
Mar 30 10:20:59 f6bvp-9 kernel: AX25- ax25_release, 959, c14e6e28, c74322e0
Mar 30 10:21:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e6e28, c74322e0
Mar 30 10:21:00 f6bvp-9 kernel: AX25- ax25_release, 959, c14e6e28, c74322e0
Mar 30 10:21:59 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e6e28, c74322e0
Mar 30 10:21:59 f6bvp-9 kernel: AX25- ax25_release, 959, c14e6e28, c74322e0
Mar 30 10:22:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e6e28, c74322e0
Mar 30 10:22:00 f6bvp-9 kernel: AX25- ax25_release, 959, c14e6e28, c74322e0
Mar 30 10:22:37 f6bvp-9 kernel: AX25; ax25_make_new, 943, c14e7a70, c51ad820
Mar 30 10:22:37 f6bvp-9 kernel: AX25= ax25_accept, 1386, c14e7a70, c51ad820, c74322e0
Mar 30 10:22:38 f6bvp-9 kernel: AX.25: sendto: Addresses built.
Mar 30 10:22:38 f6bvp-9 kernel: AX.25: sendto: building packet.
Mar 30 10:22:38 f6bvp-9 kernel: AX.25: Appending user data
Mar 30 10:22:38 f6bvp-9 kernel: AX.25: Transmitting buffer
Mar 30 10:22:39 f6bvp-9 kernel: AX.25: sendto: Addresses built.
Mar 30 10:22:39 f6bvp-9 kernel: AX.25: sendto: building packet.
Mar 30 10:22:39 f6bvp-9 kernel: AX.25: Appending user data
Mar 30 10:22:39 f6bvp-9 kernel: AX.25: Transmitting buffer
Mar 30 10:22:39 f6bvp-9 kernel: AX.25: sendto: Addresses built.
Mar 30 10:22:39 f6bvp-9 kernel: AX.25: sendto: building packet.
Mar 30 10:22:39 f6bvp-9 kernel: AX.25: Appending user data
Mar 30 10:22:39 f6bvp-9 kernel: AX.25: Transmitting buffer
Mar 30 10:22:48 f6bvp-9 kernel: AX25: ax25_frames_acked, 59, c0f72290, c744a580, 6b6b6b6b, 00000000, 00000000
Mar 30 10:22:59 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e7658, c744a580
Mar 30 10:22:59 f6bvp-9 kernel: AX25- ax25_release, 959, c14e7658, c744a580
Mar 30 10:23:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e7658, c744a580
Mar 30 10:23:00 f6bvp-9 kernel: AX25- ax25_release, 959, c14e7658, c744a580
Mar 30 10:23:59 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e6a10, c744a580
Mar 30 10:24:00 f6bvp-9 kernel: AX25- ax25_release, 959, c14e6a10, c744a580
Mar 30 10:24:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e6a10, c744a580
Mar 30 10:24:00 f6bvp-9 kernel: AX25- ax25_release, 959, c14e6a10, c744a580
Mar 30 10:24:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e7658, c744a580
Mar 30 10:24:53 f6bvp-9 kernel: AX25- ax25_release, 959, c14e7658, c744a580
Mar 30 10:24:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e7658, c744a580
Mar 30 10:24:53 f6bvp-9 kernel: AX25- ax25_release, 959, c14e7658, c744a580
Mar 30 10:25:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e7658, c744a580
Mar 30 10:25:00 f6bvp-9 kernel: AX25- ax25_release, 959, c14e7658, c744a580
Mar 30 10:25:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e7658, c744a580
Mar 30 10:25:00 f6bvp-9 kernel: AX25- ax25_release, 959, c14e7658, c744a580
Mar 30 10:26:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c5afc8b0, c744a580
Mar 30 10:26:00 f6bvp-9 kernel: AX25- ax25_release, 959, c5afc8b0, c744a580
Mar 30 10:26:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c5afc8b0, c744a580
Mar 30 10:26:00 f6bvp-9 kernel: AX25- ax25_release, 959, c5afc8b0, c744a580
Mar 30 10:27:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e6e28, c744a580
Mar 30 10:27:00 f6bvp-9 kernel: AX25- ax25_release, 959, c14e6e28, c744a580
Mar 30 10:27:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e6e28, c744a580
Mar 30 10:27:00 f6bvp-9 kernel: AX25- ax25_release, 959, c14e6e28, c744a580
Mar 30 10:28:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e6a10, c744a580
Mar 30 10:28:00 f6bvp-9 kernel: AX25- ax25_release, 959, c14e6a10, c744a580
Mar 30 10:28:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e6a10, c744a580
Mar 30 10:28:00 f6bvp-9 kernel: AX25- ax25_release, 959, c14e6a10, c744a580
Mar 30 10:29:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e6e28, c744a580
Mar 30 10:29:00 f6bvp-9 kernel: AX25- ax25_release, 959, c14e6e28, c744a580
Mar 30 10:29:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e6e28, c744a580
Mar 30 10:29:00 f6bvp-9 kernel: AX25- ax25_release, 959, c14e6e28, c744a580
Mar 30 10:30:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e6e28, c77bd040
Mar 30 10:30:00 f6bvp-9 kernel: AX25- ax25_release, 959, c14e6e28, c77bd040
Mar 30 10:30:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c5afc8b0, c7527040
Mar 30 10:30:00 f6bvp-9 kernel: AX25- ax25_release, 959, c5afc8b0, c7527040
Mar 30 10:31:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e6a10, c744a580
Mar 30 10:31:00 f6bvp-9 kernel: AX25- ax25_release, 959, c14e6a10, c744a580
Mar 30 10:31:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c14e6a10, c744a580
Mar 30 10:31:00 f6bvp-9 kernel: AX25- ax25_release, 959, c14e6a10, c744a580
Mar 30 10:31:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c5afc8b0, c744a580
Mar 30 10:31:53 f6bvp-9 kernel: AX25- ax25_release, 959, c5afc8b0, c744a580
Mar 30 10:31:53 f6bvp-9 kernel: AX25+ ax25_create, 866, c5afc8b0, c744a580
Mar 30 10:31:53 f6bvp-9 kernel: AX25- ax25_release, 959, c5afc8b0, c744a580
Mar 30 10:32:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c5afc8b0, c744a580
Mar 30 10:32:00 f6bvp-9 kernel: AX25- ax25_release, 959, c5afc8b0, c744a580
Mar 30 10:32:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c5afc8b0, c744a580
Mar 30 10:32:00 f6bvp-9 kernel: AX25- ax25_release, 959, c5afc8b0, c744a580
Mar 30 10:32:38 f6bvp-9 kernel: AX.25: sendto: Addresses built.
Mar 30 10:32:38 f6bvp-9 kernel: AX.25: sendto: building packet.
Mar 30 10:32:38 f6bvp-9 kernel: AX.25: Appending user data
Mar 30 10:32:38 f6bvp-9 kernel: AX.25: Transmitting buffer
Mar 30 10:32:39 f6bvp-9 kernel: AX25- ax25_release, 959, c14e7a70, c74322e0
Mar 30 10:32:52 f6bvp-9 kernel: AX25: ax25_frames_acked, 59, c0f72aa8, c74322e0, 6b6b6b6b, 00000000, 00000000
Mar 30 10:32:52 f6bvp-9 kernel: AX25: ax25_frames_acked, 59, c0f732c0, c779cd60, 6b6b6b6b, 00000000, 00000000
Mar 30 10:33:00 f6bvp-9 kernel: AX25+ ax25_create, 866, c5afc8b0, c779cd60
Mar 30 10:33:00 f6bvp-9 kernel: AX25- ax25_release, 959, c5afc8b0, c779cd60

#

Regards,

Bernard

I did what you suggested and reversed your patches #2,#3 and #4 to verify that there was no system incompatibilities with the new locks you introduced into rose_release().

Patch #1 has also been removed on another machine running a 64 bit Core2 duo CPU.

Three Linux boxes have been are running for more than 24 hours now with only patch #5. They handled a lot of AX25 connections with no problems.

Thus, we can reasonably expect that things are going to stay stable.

If you want to commit patch#5 I will sustain it very much.

I hope that it will be accepted quickly in order to make a coherent system with AX25 patches already applied or not yet applied (if any).

Thank you very much Jarek for this very nice job.

Best regards,

Bernard Pidoux,
F6BVP

On Tue, Apr 01, 2008 at 10:37:02PM +0200, Bernard Pidoux wrote:
...

The same to you Bernard!

David & Ralf:

Here I resend the testing patch #5 from this thread: (the only change
- it's against current netdev tree, so 1 line offset added).

Thanks,
Jarek P.

-------------------->

Subject: [ROSE/AX25] af_rose: rose_release() fix

rose_release() doesn't release sockets properly, e.g. it skips sock_orphan(), so OOPSes are triggered in sock_def_write_space(), which was observed especially while ROSE skbs were kfreed from ax25_frames_acked(). There is also sock_hold() and lock_sock() added - similarly to ax25_release(). Thanks to Bernard Pidoux for substantial help in debugging this problem.

---

 net/rose/af_rose.c |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)

index 92d85c3..d1ff3f8 100644
@@ -598,17 +598,24 @@ static int rose_release(struct socket *sock)
 
 	if (sk == NULL) return 0;
 
+	sock_hold(sk);
+	sock_orphan(sk);
+	lock_sock(sk);
 	rose = rose_sk(sk);
 
 	switch (rose->state) {
 	case ROSE_STATE_0:
+		release_sock(sk);
 		rose_disconnect(sk, 0, -1, -1);
+		lock_sock(sk);
 		rose_destroy_socket(sk);
 		break;
 
 	case ROSE_STATE_2:
 		rose->neighbour->use--;
+		release_sock(sk);
 		rose_disconnect(sk, 0, -1, -1);
+		lock_sock(sk);
 		rose_destroy_socket(sk);
 		break;
 

@@ -633,6 +640,8 @@ static int rose_release(struct socket *sock)
 	}
 
 	sock->sk = NULL;
+	release_sock(sk);
+	sock_put(sk);
 
 	return 0;
 }

From: Jarek Poplawski <[email protected]>
Date: Wed, 2 Apr 2008 06:41:07 +0000

Thanks so much for fixing this bug, applied to net-2.6