PROBLEM: Conntrack/NAT bug

DISCUSSION David S. Miller 3 participants 4 messages Oct 10, 2002 – Oct 14, 2002

Subsystems (2)

Networking netfilter

lore

Share & tools

lore

David S. Miller

author

2002-10-10 13:20 UTC #0 lore raw

[1.] One line summary of the problem: Some sort of coding problem in
conntrack.
[2.] Full description of the problem/report:
The following error pops up quite often in my console:
ASSERT ip_conntrack_core.c:94 &ip_conntrack_lock readlocked
ASSERT: ip_nat_core.c:841 &ip_conntrack_lock not readlocked [3.]
[4.] Kernel version - 2.4.20-pre8-ac3
[5.]
[6.]
[7.] Environment
[7.1.] Software:

Gnu C                  2.95.4
Gnu make               3.79.1
util-linux             2.11n
mount                  2.11n
modutils               2.4.15
e2fsprogs              1.27
PPP                    2.4.1
Linux C Library        2.2.5
Dynamic linker (ldd)   2.2.5
Procps                 2.0.7
Net-tools              1.60
Console-tools          0.2.3
Sh-utils               2.0.11
Modules Loaded         e100

[7.2.] Processor information:
processor : 0
vendor_id : GenuineIntel
cpu family : 15
model : 2
model name : Intel(R) Pentium(R) 4 CPU 2.00GHz
stepping : 4
cpu MHz : 2017.996
cache size : 512 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
bogomips : 4023.91

[7.3.] Module information:
e100 51576 1

[7.4.] Loaded driver and hardware information:
00000000-0009ffff : System RAM
000a0000-000bffff : Video RAM area
000c0000-000c7fff : Video ROM
000f0000-000fffff : System ROM
00100000-0fef9fff : System RAM
00100000-0023c353 : Kernel code
0023c354-002a0817 : Kernel data
0fefa000-0fefcfff : ACPI Tables
0fefd000-0fefdfff : ACPI Non-volatile Storage
0fefe000-0fefffff : reserved
10000000-100003ff : Intel Corp. 82801DB ICH4 IDE e6800000-e6800fff : Intel Corp. 82801BD PRO/100 VE (CNR) Ethernet Controller
e6800000-e6800fff : e100
e7000000-e70003ff : Intel Corp. 82801DB USB EHCI Controller e7800000-e787ffff : Intel Corp. 82845G/GL [Brookdale-G] Chipset Integrated Graphics Device
e8000000-efffffff : Intel Corp. 82845G/GL [Brookdale-G] Chipset Integrated Graphics Device
f0000000-f7ffffff : Intel Corp. 82845G/GL [Brookdale-G] Chipset Host Bridge fec00000-fec00fff : reserved
fee00000-fee00fff : reserved
ffff0000-ffffffff : reserved

[7.5.] PCI information ('lspci -vvv' as root)
00:00.0 Host bridge: Intel Corp.: Unknown device 2560 (rev 01)
Subsystem: Asustek Computer, Inc.: Unknown device 8093
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort+ >SERR- <PERR-

        Latency: 0
        Region 0: Memory at f0000000 (32-bit, prefetchable) [size=128M]
        Capabilities: [e4] #09 [0105]

00:02.0 VGA compatible controller: Intel Corp.: Unknown device 2562 (rev 01) (prog-if 00 [VGA])
Subsystem: Asustek Computer, Inc.: Unknown device 2562
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
Latency: 0
Interrupt: pin A routed to IRQ 11
Region 0: Memory at e8000000 (32-bit, prefetchable) [size=128M] Region 1: Memory at e7800000 (32-bit, non-prefetchable) [size=512K]
Capabilities: [d0] Power Management version 1
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-

00:1d.0 USB Controller: Intel Corp.: Unknown device 24c2 (rev 01) (prog-if 00 [UHCI])
Subsystem: Asustek Computer, Inc.: Unknown device 8089
Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
Latency: 0
Interrupt: pin A routed to IRQ 11
Region 4: I/O ports at d800 [size=32]

00:1d.1 USB Controller: Intel Corp.: Unknown device 24c4 (rev 01) (prog-if 00 [UHCI])
Subsystem: Asustek Computer, Inc.: Unknown device 8089
Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
Latency: 0
Interrupt: pin B routed to IRQ 12
Region 4: I/O ports at d400 [size=32]

00:1d.2 USB Controller: Intel Corp.: Unknown device 24c7 (rev 01) (prog-if 00 [UHCI])
Subsystem: Asustek Computer, Inc.: Unknown device 8089
Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
Latency: 0
Interrupt: pin C routed to IRQ 10
Region 4: I/O ports at d000 [size=32]

00:1d.7 USB Controller: Intel Corp.: Unknown device 24cd (rev 01) (prog-if
20)
Subsystem: Asustek Computer, Inc.: Unknown device 8089
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
Latency: 0
Interrupt: pin D routed to IRQ 3
Region 0: Memory at e7000000 (32-bit, non-prefetchable) [size=1K]
Capabilities: [50] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [58] #0a [2080]

00:1e.0 PCI bridge: Intel Corp. 82820 820 (Camino 2) Chipset PCI (rev 81) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR+

        Latency: 0
        Bus: primary=00, secondary=01, subordinate=01, sec-latency=32
        I/O behind bridge: 0000b000-0000bfff
        Memory behind bridge: e6800000-e6ffffff
        BridgeCtl: Parity- SERR+ NoISA+ VGA- MAbort- >Reset- FastB2B-

00:1f.0 ISA bridge: Intel Corp.: Unknown device 24c0 (rev 01)
Control: I/O+ Mem+ BusMaster+ SpecCycle+ MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
Latency: 0

00:1f.1 IDE interface: Intel Corp.: Unknown device 24cb (rev 01) (prog-if 8a [Master SecP PriP])
Subsystem: Asustek Computer, Inc.: Unknown device 8089
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B-
Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
Latency: 0
Interrupt: pin A routed to IRQ 10
Region 0: I/O ports at <unassigned> [size=8]
Region 1: I/O ports at <unassigned> [size=4]
Region 2: I/O ports at <unassigned> [size=8]
Region 3: I/O ports at <unassigned> [size=4]
Region 4: I/O ports at f000 [size=16]
Region 5: Memory at 10000000 (32-bit, non-prefetchable) [size=1K]

01:08.0 Ethernet controller: Intel Corp.: Unknown device 103a (rev 81)
Subsystem: Asustek Computer, Inc.: Unknown device 8071
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr-
Stepping- SERR- FastB2B-
Status: Cap+ 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
Latency: 64 (2000ns min, 14000ns max), cache line size 08
Interrupt: pin A routed to IRQ 4
Region 0: Memory at e6800000 (32-bit, non-prefetchable) [size=4K] Region 1: I/O ports at b800 [size=64]
Capabilities: [dc] Power Management version 2
Flags: PMEClk- DSI+ D1+ D2+ AuxCurrent=0mA
PME(D0+,D1+,D2+,D3hot+,D3cold+)
Status: D0 PME-Enable- DSel=0 DScale=2 PME-

[7.6.] SCSI information : None
[7.7.] Other information that might be relevant to the problem [X.] Other notes, patches, fixes, workarounds:

Thanks for your help
Tom Lanyon
[email protected]

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Martin Josefsson

2002-10-10 14:18 UTC #1 lore raw

Quoted 7 lines

[1.] One line summary of the problem: Some sort of coding problem in
conntrack.
[2.] Full description of the problem/report:
The following error pops up quite often in my console:
ASSERT ip_conntrack_core.c:94 &ip_conntrack_lock readlocked
ASSERT: ip_nat_core.c:841 &ip_conntrack_lock not readlocked

This is not a real bug, it's harmless. The ASSERT's are because the lock-debugging in netfilter can't handle "recursive" read-locks on the same lock and cpu.

Attached is a patch that will eliminate these printouts, it's a bit unclean and was rejected by Harald (it's perfectly safe) but it will shut them up (at least the ASSERT's you reported in your mail).

Signature · 4 lines

/Martin

Never argue with an idiot. They drag you down to their level, then beat
you with experience.

include/linux/netfilter_ipv4/ip_conntrack_core.h +1 -1

diff -x '*.orig' -urN linux-2.4.20-pre8-ac3.orig/include/linux/netfilter_ipv4/ip_conntrack_core.h linux-2.4.20-pre8-ac3/include/linux/netfilter_ipv4/ip_conntrack_core.h
@@ -17,7 +17,7 @@
 struct ip_conntrack_protocol;
 extern struct ip_conntrack_protocol *ip_ct_find_proto(u_int8_t protocol);
 /* Like above, but you already have conntrack read lock. */
-extern struct ip_conntrack_protocol *__find_proto(u_int8_t protocol);
+extern struct ip_conntrack_protocol *__ip_ct_find_proto(u_int8_t protocol);
 extern struct list_head protocol_list;
 
 /* Returns conntrack if it dealt with ICMP, and filled in skb->nfct */

net/ipv4/netfilter/ip_conntrack_core.c +2 -2

diff -x '*.orig' -urN linux-2.4.20-pre8-ac3.orig/net/ipv4/netfilter/ip_conntrack_core.c linux-2.4.20-pre8-ac3/net/ipv4/netfilter/ip_conntrack_core.c
@@ -74,7 +74,7 @@
 	return protocol == curr->proto;
 }
 
-struct ip_conntrack_protocol *__find_proto(u_int8_t protocol)
+struct ip_conntrack_protocol *__ip_ct_find_proto(u_int8_t protocol)
 {
 	struct ip_conntrack_protocol *p;

@@ -92,7 +92,7 @@
 	struct ip_conntrack_protocol *p;
 
 	READ_LOCK(&ip_conntrack_lock);
-	p = __find_proto(protocol);
+	p = __ip_ct_find_proto(protocol);
 	READ_UNLOCK(&ip_conntrack_lock);
 	return p;
 }

net/ipv4/netfilter/ip_conntrack_standalone.c +3 -2

diff -x '*.orig' -urN linux-2.4.20-pre8-ac3.orig/net/ipv4/netfilter/ip_conntrack_standalone.c linux-2.4.20-pre8-ac3/net/ipv4/netfilter/ip_conntrack_standalone.c
@@ -71,7 +71,7 @@
 	len += sprintf(buffer + len, "use=%u proto=%u ",
 		      atomic_read(&expect->use), expect->tuple.dst.protonum);
 	len += print_tuple(buffer + len, &expect->tuple,
-			   __find_proto(expect->tuple.dst.protonum));
+			   __ip_ct_find_proto(expect->tuple.dst.protonum));
 	len += sprintf(buffer + len, "\n");
 	return len;
 }

@@ -81,7 +81,7 @@
 {
 	unsigned int len;
 	struct ip_conntrack_protocol *proto
-		= __find_proto(conntrack->tuplehash[IP_CT_DIR_ORIGINAL]
+		= __ip_ct_find_proto(conntrack->tuplehash[IP_CT_DIR_ORIGINAL]
 			       .tuple.dst.protonum);
 
 	len = sprintf(buffer, "%-8s %u %lu ",

@@ -361,6 +361,7 @@
 EXPORT_SYMBOL(ip_ct_selective_cleanup);
 EXPORT_SYMBOL(ip_ct_refresh);
 EXPORT_SYMBOL(ip_ct_find_proto);
+EXPORT_SYMBOL(__ip_ct_find_proto);
 EXPORT_SYMBOL(ip_ct_find_helper);
 EXPORT_SYMBOL(ip_conntrack_expect_related);
 EXPORT_SYMBOL(ip_conntrack_change_expect);

net/ipv4/netfilter/ip_nat_core.c +1 -1

diff -x '*.orig' -urN linux-2.4.20-pre8-ac3.orig/net/ipv4/netfilter/ip_nat_core.c linux-2.4.20-pre8-ac3/net/ipv4/netfilter/ip_nat_core.c
@@ -739,7 +739,7 @@
 	int ret = 1;
 
 	MUST_BE_READ_LOCKED(&ip_conntrack_lock);
-	proto = ip_ct_find_proto((*pskb)->nh.iph->protocol);
+	proto = __ip_ct_find_proto((*pskb)->nh.iph->protocol);
 	if (proto->exp_matches_pkt)
 		ret = proto->exp_matches_pkt(exp, pskb);

Harald Welte

2002-10-14 11:43 UTC #2 lore raw

In reply to David S. Miller

Quoted 2 lines · Tom Lanyon

[1.] One line summary of the problem: Some sort of coding problem in
conntrack.

It's not a bug in the code, it's a bug in the debugging ASSERT macros, as correctly pointed out by Martin Josefsson.

The issue is that we are nesting readlocks, because ip_ct_find_proto() grabs itself a readlock on ip_conntrack_lock.

The patch below now exports the non-locking __ip_ct_find_proto() function, thus no more grabbing a nested readlock.

I wanted to go for a different solution, since exporting __ functions is not exactly clean coding style - but due to the circumstances other solutions look even more ugly.

Dave, please apply the following fix (again by Martin Josefsson), Thanks.

include/linux/netfilter_ipv4/ip_conntrack_core.h +1 -1

diff -x '*.orig' -urN linux-2.4.20-pre8-ac3.orig/include/linux/netfilter_ipv4/ip_conntrack_core.h linux-2.4.20-pre8-ac3/include/linux/netfilter_ipv4/ip_conntrack_core.h
@@ -17,7 +17,7 @@
 struct ip_conntrack_protocol;
 extern struct ip_conntrack_protocol *ip_ct_find_proto(u_int8_t protocol);
 /* Like above, but you already have conntrack read lock. */
-extern struct ip_conntrack_protocol *__find_proto(u_int8_t protocol);
+extern struct ip_conntrack_protocol *__ip_ct_find_proto(u_int8_t protocol);
 extern struct list_head protocol_list;
 
 /* Returns conntrack if it dealt with ICMP, and filled in skb->nfct */

net/ipv4/netfilter/ip_conntrack_core.c +2 -2

diff -x '*.orig' -urN linux-2.4.20-pre8-ac3.orig/net/ipv4/netfilter/ip_conntrack_core.c linux-2.4.20-pre8-ac3/net/ipv4/netfilter/ip_conntrack_core.c
@@ -74,7 +74,7 @@
 	return protocol == curr->proto;
 }
 
-struct ip_conntrack_protocol *__find_proto(u_int8_t protocol)
+struct ip_conntrack_protocol *__ip_ct_find_proto(u_int8_t protocol)
 {
 	struct ip_conntrack_protocol *p;

@@ -92,7 +92,7 @@
 	struct ip_conntrack_protocol *p;
 
 	READ_LOCK(&ip_conntrack_lock);
-	p = __find_proto(protocol);
+	p = __ip_ct_find_proto(protocol);
 	READ_UNLOCK(&ip_conntrack_lock);
 	return p;
 }

net/ipv4/netfilter/ip_conntrack_standalone.c +3 -2

diff -x '*.orig' -urN linux-2.4.20-pre8-ac3.orig/net/ipv4/netfilter/ip_conntrack_standalone.c linux-2.4.20-pre8-ac3/net/ipv4/netfilter/ip_conntrack_standalone.c
@@ -71,7 +71,7 @@
 	len += sprintf(buffer + len, "use=%u proto=%u ",
 		      atomic_read(&expect->use), expect->tuple.dst.protonum);
 	len += print_tuple(buffer + len, &expect->tuple,
-			   __find_proto(expect->tuple.dst.protonum));
+			   __ip_ct_find_proto(expect->tuple.dst.protonum));
 	len += sprintf(buffer + len, "\n");
 	return len;
 }

@@ -81,7 +81,7 @@
 {
 	unsigned int len;
 	struct ip_conntrack_protocol *proto
-		= __find_proto(conntrack->tuplehash[IP_CT_DIR_ORIGINAL]
+		= __ip_ct_find_proto(conntrack->tuplehash[IP_CT_DIR_ORIGINAL]
 			       .tuple.dst.protonum);
 
 	len = sprintf(buffer, "%-8s %u %lu ",

@@ -361,6 +361,7 @@
 EXPORT_SYMBOL(ip_ct_selective_cleanup);
 EXPORT_SYMBOL(ip_ct_refresh);
 EXPORT_SYMBOL(ip_ct_find_proto);
+EXPORT_SYMBOL(__ip_ct_find_proto);
 EXPORT_SYMBOL(ip_ct_find_helper);
 EXPORT_SYMBOL(ip_conntrack_expect_related);
 EXPORT_SYMBOL(ip_conntrack_change_expect);

net/ipv4/netfilter/ip_nat_core.c +1 -1

diff -x '*.orig' -urN linux-2.4.20-pre8-ac3.orig/net/ipv4/netfilter/ip_nat_core.c linux-2.4.20-pre8-ac3/net/ipv4/netfilter/ip_nat_core.c
@@ -739,7 +739,7 @@
 	int ret = 1;
 
 	MUST_BE_READ_LOCKED(&ip_conntrack_lock);
-	proto = ip_ct_find_proto((*pskb)->nh.iph->protocol);
+	proto = __ip_ct_find_proto((*pskb)->nh.iph->protocol);
 	if (proto->exp_matches_pkt)
 		ret = proto->exp_matches_pkt(exp, pskb);

Quoted 1 line

Thanks for your help

Thanks for your help (i mean it!).

Quoted 2 lines

Tom Lanyon
[email protected]

Signature · 5 lines

Live long and prosper
- Harald Welte / [email protected]               http://www.gnumonks.org/
============================================================================
"If this were a dictatorship, it'd be a heck of a lot easier, just so long
 as I'm the dictator."  --  George W. Bush Dec 18, 2000

David S. Miller

author

2002-10-14 11:48 UTC #3 lore raw

From: Harald Welte <[email protected]>
Date: Mon, 14 Oct 2002 13:47:35 +0200

Dave, please apply the following fix (again by Martin Josefsson), Thanks.

Patch applied, thanks.