Skip to contentSkip to conversation

linux-2.6.12-rc1-mm4-RT-V0.7.42-08

DISCUSSION Peter Zijlstra 2 participants 3 messages Apr 3, 2005 – Apr 4, 2005 1 message not in the archive

Subsystems (3)
lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): 1112433149.13131.8.camel@twins

#1 lore raw

Hi Ingo,

I need the following two patches to keep my system alive and avoid the BUGs in the log send to you earlier (private mail).

------------------- example BUG ------------------------

BUG: using smp_processor_id() in preemptible [00000001] code: java/16460
caller is icmp_send+0x9b/0x3d0
 [<c020b1db>] smp_processor_id+0x9b/0xb0 (8)
 [<c032246b>] icmp_send+0x9b/0x3d0 (28)
 [<c013ae67>] check_wakeup_timing+0x47/0x160 (8)
 [<c034269d>] _raw_spin_unlock+0xd/0x30 (40)
 [<c014d159>] kmem_cache_free+0x39/0x120 (8)
 [<c034269d>] _raw_spin_unlock+0xd/0x30 (12)
 [<c014d159>] kmem_cache_free+0x39/0x120 (8)
 [<c034269d>] _raw_spin_unlock+0xd/0x30 (24)
 [<c034269d>] _raw_spin_unlock+0xd/0x30 (28)
 [<c0276066>] rtl8139_start_xmit+0x86/0x1d0 (8)
 [<c034269d>] _raw_spin_unlock+0xd/0x30 (40)
 [<c02f3259>] qdisc_restart+0x1b9/0x230 (8)
 [<c02f3259>] qdisc_restart+0x1b9/0x230 (36)
 [<c034269d>] _raw_spin_unlock+0xd/0x30 (8)
 [<c02760ae>] rtl8139_start_xmit+0xce/0x1d0 (16)
 [<c034269d>] _raw_spin_unlock+0xd/0x30 (32)
 [<c02ffe33>] ip_fragment+0x663/0x7f0 (60)
 [<c013138f>] rcu_read_unlock+0x5f/0x70 (32)
 [<c02f9d2a>] __ip_route_output_key+0xba/0xe0 (8)
 [<c02fe950>] ip_finish_output+0x0/0x2f0 (16)
 [<c02ff383>] ip_queue_xmit+0x263/0x590 (24)
 [<c02ff383>] ip_queue_xmit+0x263/0x590 (20)
 [<c03102eb>] tcp_transmit_skb+0x38b/0x750 (196)
 [<c031113e>] tcp_write_xmit+0x10e/0x2f0 (56)
 [<c03054ab>] tcp_sendmsg+0xffb/0x1050 (52)
 [<c034269d>] _raw_spin_unlock+0xd/0x30 (12)
 [<c013b9c5>] unqueue_me+0x25/0x130 (8)
 [<c0325f4a>] inet_sendmsg+0x4a/0x70 (92)
 [<c02dd795>] sock_aio_write+0x135/0x150 (24)
 [<c034269d>] _raw_spin_unlock+0xd/0x30 (88)
 [<c0163c69>] do_sync_write+0xb9/0x110 (20)
 [<c034269d>] _raw_spin_unlock+0xd/0x30 (44)
 [<c0164d52>] fget_light+0x62/0x90 (8)
 [<c034269d>] _raw_spin_unlock+0xd/0x30 (44)
 [<c0134c50>] autoremove_wake_function+0x0/0x50 (8)
 [<c0163d9b>] vfs_write+0xdb/0x170 (52)
 [<c0163ee1>] sys_write+0x41/0x70 (24)
 [<c0102f0b>] sysenter_past_esp+0x54/0x75 (28)
---------------------------

| preempt count: 00000002 ]
| 2-level deep critical section nesting:
----------------------------------------
.. [<c03422e6>] .... _raw_spin_lock+0x16/0x90
.....[<00000000>] .. ( <= 0x0)
.. [<c03422e6>] .... _raw_spin_lock+0x16/0x90
.....[<00000000>] .. ( <= 0x0)

Signature · 1 line
Peter Zijlstra <[email protected]>

Since there is no lock held yet we are still preemptable; and since icmp_socket is a per cpu variable switching cpus gives weird results.

usr/src/linux-2.6.12-rc1-mm4-RT-V0.7.42-08/net/ipv4/icmp.c renamed from usr/src/linux-2.6.12-rc1-mm4-RT-V0.7.42-08/net/ipv4/icmp.c~ +5 -2
@@ -376,8 +376,8 @@
 
 static void icmp_reply(struct icmp_bxm *icmp_param, struct sk_buff *skb)
 {
-	struct sock *sk = icmp_socket->sk;
-	struct inet_sock *inet = inet_sk(sk);
+	struct sock *sk;
+	struct inet_sock *inet;
 	struct ipcm_cookie ipc;
 	struct rtable *rt = (struct rtable *)skb->dst;
 	u32 daddr;
@@ -388,6 +388,9 @@
 	if (icmp_xmit_lock())
 		return;
 
+	sk = icmp_socket->sk;
+	inet = inet_sk(sk);
+
 	icmp_param->data.icmph.checksum = 0;
 	icmp_out_count(icmp_param->data.icmph.type);
 

Since on PREEMPT_RT spinlocks allow preemption, it's possible to change cpu at (almost) any point in time. Make sure we stick to the per cpu variable we started with.

usr/src/linux-2.6.12-rc1-mm4-RT-V0.7.42-08/net/ipv4/icmp.c renamed from usr/src/linux-2.6.12-rc1-mm4-RT-V0.7.42-08/net/ipv4/icmp_orig.c +28 -17

141 lines not on this page · raw message

#2 lore raw
Quoted 4 lines · Peter Zijlstra

Hi Ingo,

I need the following two patches to keep my system alive and avoid the BUGs in the log send to you earlier (private mail).

hm, the second patch does not apply (and the merge didnt look trivial) - maybe it depends on some patch in -mm that is not yet upstream?

Ingo

#3 lore raw
Quoted 10 lines · Ingo Molnar
Quoted 4 lines · Peter Zijlstra

Hi Ingo,

I need the following two patches to keep my system alive and avoid the BUGs in the log send to you earlier (private mail).

hm, the second patch does not apply (and the merge didnt look trivial) - maybe it depends on some patch in -mm that is not yet upstream?

I messed it up; apparently I type :wa to often and forgot to keep a decent base file (yes I'm a vim user). I also seem to have gotten the patch base dir wrong.

The two patches again; now hopefully with correct patch level and proper base. They apply to my tree in order 2.6.12-rc1-icmp{,-rt}.patch.

Not ran nor compile tested on vanilla; just made then apply cleanly.

Could you please comment on the patches since I'm learning this stuff as I go; and validation of my ideas would be most welcome. For instance; was I right to convert the per cpu var __icmp_socket to a _LOCKED? I think so since without the extra lock one could have race conditions within the spin_trylock arguments double dereference.

Kind regards,

Peter Zijlstra

Since there is no lock held yet we are still preemptable; and since icmp_socket is a per cpu variable switching cpus gives weird results.

net/ipv4/icmp.c renamed from net/ipv4/icmp_base.c +5 -2
@@ -376,8 +376,8 @@
 
 static void icmp_reply(struct icmp_bxm *icmp_param, struct sk_buff *skb)
 {
-	struct sock *sk = icmp_socket->sk;
-	struct inet_sock *inet = inet_sk(sk);
+	struct sock *sk;
+	struct inet_sock *inet;
 	struct ipcm_cookie ipc;
 	struct rtable *rt = (struct rtable *)skb->dst;
 	u32 daddr;
@@ -388,6 +388,9 @@
 	if (icmp_xmit_lock())
 		return;
 
+	sk = icmp_socket->sk;
+	inet = inet_sk(sk);
+
 	icmp_param->data.icmph.checksum = 0;
 	icmp_out_count(icmp_param->data.icmph.type);
 

Since on PREEMPT_RT spinlocks allow preemption, it's possible to change cpu at (almost) any point in time. Make sure we stick to the per cpu variable we started with.

net/ipv4/icmp.c renamed from net/ipv4/icmp_p1.c +31 -19

143 lines not on this page · raw message