Skip to contentSkip to conversation

futex warning tracback on 2.6.29.1-rt6

DISCUSSION Clark Williams 2 participants 3 messages Apr 10, 2009

lore
Share & tools
lore
#0 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Thomas,

Got the below warning/traceback on 2.6.29.1-rt6. This is running on my T60, 64-bit kernel. Haven't been able to figure out how to trigger it yet.

$ uname -a
Linux torg 2.6.29.1-rt6 #20 SMP PREEMPT RT Thu Apr 9 15:30:06 CDT 2009
x86_64 x86_64 x86_64 GNU/Linux

Clark

- ------------[ cut here ]------------

WARNING: at kernel/futex.c:184 drop_futex_key_refs+0x7e/0x111()
Hardware name:        
Modules linked in: tun fuse i915 drm i2c_algo_bit sunrpc ip6t_REJECT nf_conntrack_ipv6 ip6table_filter ip6_tables ipv6 cpufreq_ondemand acpi_cpufreq freq_table loop dm_multipath scsi_dh uinput snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq arc4 ecb snd_seq_device iwl3945 snd_pcm_oss mac80211 btusb snd_mixer_oss snd_pcm snd_timer sr_mod iTCO_wdt snd cdrom lib80211 pl2303 iTCO_vendor_support video bluetooth yenta_socket ata_generic joydev pcspkr usbserial ata_piix soundcore i2c_i801 i2c_core cfg80211 rsrc_nonstatic e1000e sg ati_remote thinkpad_acpi rfkill hwmon output snd_page_alloc dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod ahci libata sd_mod scsi_mod uhci_hcd ohci_hcd ssb [last unloaded: microcode]
Pid: 12898, comm: sbcl Not tainted 2.6.29.1-rt6 #20
Call Trace:
 [<ffffffff81065411>] warn_slowpath+0xe2/0x116
 [<ffffffff8109e00a>] ? __lock_release+0x10c/0x125
 [<ffffffff8100ce96>] ? ftrace_call+0x5/0x2b
 [<ffffffff8152078b>] ? rt_mutex_unlock+0x47/0x67
 [<ffffffff810a7414>] ? rt_up_read+0x2b/0x2f
 [<ffffffff81044bbb>] ? get_user_pages_fast+0x1e4/0x235
 [<ffffffff8100ce96>] ? ftrace_call+0x5/0x2b
 [<ffffffff810a023d>] drop_futex_key_refs+0x7e/0x111
 [<ffffffff810a04b9>] futex_wait_setup+0x1e9/0x1fb
 [<ffffffff8100ce96>] ? ftrace_call+0x5/0x2b
 [<ffffffff810a08ee>] futex_wait+0x13f/0x2ce
 [<ffffffff8101573b>] ? sched_clock+0x42/0x48
 [<ffffffff8109b621>] ? lock_release_holdtime+0x18d/0x192
 [<ffffffff8109e00a>] ? __lock_release+0x10c/0x125
 [<ffffffff8100ce96>] ? ftrace_call+0x5/0x2b
 [<ffffffff81086896>] ? __wake_up_bit+0xd/0x51
 [<ffffffff811052c1>] ? page_waitqueue+0x9/0x68
 [<ffffffff811072f2>] ? unlock_page+0x27/0x2b
 [<ffffffff8109c92e>] ? __lock_acquire+0x659/0x744
 [<ffffffff81527701>] ? do_page_fault+0x7e2/0x80d
 [<ffffffff810157cd>] ? native_sched_clock+0x8c/0xba
 [<ffffffff8101573b>] ? sched_clock+0x42/0x48
 [<ffffffff8109b621>] ? lock_release_holdtime+0x18d/0x192
 [<ffffffff8105b130>] ? default_wake_function+0x0/0x16
 [<ffffffff8100ce96>] ? ftrace_call+0x5/0x2b
 [<ffffffff810a4937>] do_futex+0xaa/0x1d1
 [<ffffffff810a4c2b>] sys_futex+0x1cd/0x1dc
 [<ffffffff8100d1b2>] system_call_fastpath+0x16/0x1b

- ---[ end trace 384450cb13d7d800 ]---

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.10 (GNU/Linux)

iEYEARECAAYFAknetR0ACgkQHyuj/+TTEp2XkACgqc+Ko+v6cmDsbaVOg9tWaNUF pO0AnRHVIqZYUo9PrxeKc4ws+BXZMyvN
=mJTq
-----END PGP SIGNATURE-----

#1 lore raw
Quoted 9 lines · Clark Williams

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Thomas,

Got the below warning/traceback on 2.6.29.1-rt6. This is running on my T60, 64-bit kernel. Haven't been able to figure out how to trigger it yet.

Quoted 8 lines
$ uname -a
Linux torg 2.6.29.1-rt6 #20 SMP PREEMPT RT Thu Apr 9 15:30:06 CDT 2009
x86_64 x86_64 x86_64 GNU/Linux

Clark

- ------------[ cut here ]------------

WARNING: at kernel/futex.c:184 drop_futex_key_refs+0x7e/0x111()

static void drop_futex_key_refs(union futex_key *key)

{
       if (!key->both.ptr) {
               /* If we're here then we tried to put a key we failed to get */
               WARN_ON_ONCE(1);
               return;
       }

OK, looks like we have some key management issues to sort out. Clark, I presume you weren't running the glibc patch to enable requeue_pi? From the trace below this appears to have happened during a normal futex_wait() call. Looking at futex_wait_setup() I see a rather obvious bug:

       ret = get_futex_key(uaddr, fshared, &q->key);
       if (unlikely(ret != 0))
               goto out;

...

out:

       if (ret)
               put_futex_key(fshared, &q->key);

So if we fail to get the key, we goto out... and then put the key. Oops. I'll prepare a patch and send out after some testing.

Quoted 41 lines
Hardware name:        Modules linked in: tun fuse i915 drm i2c_algo_bit sunrpc ip6t_REJECT nf_conntrack_ipv6 ip6table_filter ip6_tables ipv6 cpufreq_ondemand acpi_cpufreq freq_table loop dm_multipath scsi_dh uinput snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq arc4 ecb snd_seq_device iwl3945 snd_pcm_oss mac80211 btusb snd_mixer_oss snd_pcm snd_timer sr_mod iTCO_wdt snd cdrom lib80211 pl2303 iTCO_vendor_support video bluetooth yenta_socket ata_generic joydev pcspkr usbserial ata_piix soundcore i2c_i801 i2c_core cfg80211 rsrc_nonstatic e1000e sg ati_remote thinkpad_acpi rfkill hwmon output snd_page_alloc dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod ahci libata sd_mod scsi_mod uhci_hcd ohci_hcd ssb [last unloaded: microcode]
Pid: 12898, comm: sbcl Not tainted 2.6.29.1-rt6 #20
Call Trace:
 [<ffffffff81065411>] warn_slowpath+0xe2/0x116
 [<ffffffff8109e00a>] ? __lock_release+0x10c/0x125
 [<ffffffff8100ce96>] ? ftrace_call+0x5/0x2b
 [<ffffffff8152078b>] ? rt_mutex_unlock+0x47/0x67
 [<ffffffff810a7414>] ? rt_up_read+0x2b/0x2f
 [<ffffffff81044bbb>] ? get_user_pages_fast+0x1e4/0x235
 [<ffffffff8100ce96>] ? ftrace_call+0x5/0x2b
 [<ffffffff810a023d>] drop_futex_key_refs+0x7e/0x111
 [<ffffffff810a04b9>] futex_wait_setup+0x1e9/0x1fb
 [<ffffffff8100ce96>] ? ftrace_call+0x5/0x2b
 [<ffffffff810a08ee>] futex_wait+0x13f/0x2ce
 [<ffffffff8101573b>] ? sched_clock+0x42/0x48
 [<ffffffff8109b621>] ? lock_release_holdtime+0x18d/0x192
 [<ffffffff8109e00a>] ? __lock_release+0x10c/0x125
 [<ffffffff8100ce96>] ? ftrace_call+0x5/0x2b
 [<ffffffff81086896>] ? __wake_up_bit+0xd/0x51
 [<ffffffff811052c1>] ? page_waitqueue+0x9/0x68
 [<ffffffff811072f2>] ? unlock_page+0x27/0x2b
 [<ffffffff8109c92e>] ? __lock_acquire+0x659/0x744
 [<ffffffff81527701>] ? do_page_fault+0x7e2/0x80d
 [<ffffffff810157cd>] ? native_sched_clock+0x8c/0xba
 [<ffffffff8101573b>] ? sched_clock+0x42/0x48
 [<ffffffff8109b621>] ? lock_release_holdtime+0x18d/0x192
 [<ffffffff8105b130>] ? default_wake_function+0x0/0x16
 [<ffffffff8100ce96>] ? ftrace_call+0x5/0x2b
 [<ffffffff810a4937>] do_futex+0xaa/0x1d1
 [<ffffffff810a4c2b>] sys_futex+0x1cd/0x1dc
 [<ffffffff8100d1b2>] system_call_fastpath+0x16/0x1b

- ---[ end trace 384450cb13d7d800 ]---

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.10 (GNU/Linux)

iEYEARECAAYFAknetR0ACgkQHyuj/+TTEp2XkACgqc+Ko+v6cmDsbaVOg9tWaNUF
pO0AnRHVIqZYUo9PrxeKc4ws+BXZMyvN
=mJTq
-----END PGP SIGNATURE----- N‹§²æìr¸›yúèšØb²X¬¶Ç§vØ^–)Þº{.nÇ+‰·¥Š{±®Û¬z»"žØ^n‡r¡ö¦zË�ëh™¨è­Ú&¢ø®G«�éh®(­éšŽŠÝ¢j"�ú¶m§ÿï�êäz¹Þ–Šàþf£¢·hšˆ§~ˆmml==

Signature · 7 lines
Darren Hart
IBM Linux Technology Center
Real-Time Linux Team
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#2 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 5 lines · On Fri, 10 Apr 2009 08:50:26 -0700 Darren Hart <[email protected]> wrote:

OK, looks like we have some key management issues to sort out. Clark, I presume you weren't running the glibc patch to enable requeue_pi? From the trace below this appears to have happened during a normal futex_wait() call. Looking at futex_wait_setup() I see a rather obvious bug:

That's correct, I was just running the stock F10 glibc. -----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.10 (GNU/Linux)

iEYEARECAAYFAknffZ0ACgkQHyuj/+TTEp2yXACeJB0p/GlgzjuMXU5MHGdOniNT J2oAmwREpNfq83JOP+OuxTqPTaXkzLDQ
=tZu+
-----END PGP SIGNATURE-----