2.6.27-rc4-git1: Reported regressions from 2.6.26
Subsystems (74)
This message contains a list of some regressions from 2.6.26, for which there are no fixes in the mainline I know of. If any of them have been fixed already, please let me know.
If you know of any other unresolved regressions from 2.6.26, please let me know either and I'll add them to the list. Also, please let me know if any of the entries below are invalid.
Each entry from the list will be sent additionally in an automatic reply to this message with CCs to the people involved in reporting and handling the issue.
Listed regressions statistics:
Date Total Pending Unresolved
----------------------------------------
2008-08-23 122 48 40 2008-08-16 103 47 37 2008-08-10 80 52 31 2008-08-02 47 31 20
Unresolved regressions
----------------------
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11414
Subject : Random crashes with 2.6.27-rc3 on PPC
Submitter : Michael Buesch <[email protected]>
Date : 2008-08-23 14:10 (1 days old)
References : http://marc.info/?l=linux-kernel&m=121950076812616&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11410
Subject : SLUB list_lock vs obj_hash.lock...
Submitter : Daniel J Blueman <[email protected]>
Date : 2008-08-22 21:48 (2 days old)
References : http://marc.info/?l=linux-kernel&m=121944176609042&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11407
Subject : suspend: unable to handle kernel paging request
Submitter : Vegard Nossum <[email protected]>
Date : 2008-08-21 17:28 (3 days old)
References : http://marc.info/?l=linux-kernel&m=121933974928881&w=4
Pekka Enberg <penberg-bbCR+/[email protected]> Pavel Machek <[email protected]>
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11406
Subject : patch "x86: MOVE PCI IO ECS code to x86/pci" breaks CPU hotplug
Submitter : Jan Beulich <[email protected]>
Date : 2008-08-21 12:59 (3 days old)
References : http://marc.info/?l=linux-kernel&m=121932366326572&w=4
Robert Richter <[email protected]>
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11405
Subject : 2.6.27-rc3 segfault on cold boot; not on warm boot.
Submitter : David Greaves <david-FQ/kcb21CSxWk0Htik3J/[email protected]>
Date : 2008-08-21 9:45 (3 days old)
References : http://marc.info/?l=linux-kernel&m=121931198904777&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11404
Subject : BUG: in 2.6.23-rc3-git7 in do_cciss_intr
Submitter : rdunlap <[email protected]>
Date : 2008-08-21 5:52 (3 days old)
References : http://marc.info/?l=linux-kernel&m=121929819616273&w=4
http://marc.info/?l=linux-kernel&m=121932889105368&w=4
James Bottomley <James.Bottomley-JuX6DAaQMKPCXq6kfMZ53/[email protected]>
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11403
Subject : 2.6.27-rc2 USB suspend regression
Submitter : Jeremy Fitzhardinge <[email protected]>
Date : 2008-08-20 20:48 (4 days old)
References : http://marc.info/?l=linux-kernel&m=121926536103630&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11402
Subject : skbuff bug?
Submitter : Yinghai Lu <[email protected]>
Date : 2008-08-21 3:56 (3 days old)
References : http://marc.info/?l=linux-kernel&m=121929102707658&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11401
Subject : pktcdvd: BUG, NULL pointer dereference in pkt_ioctl, bisected
Submitter : Laurent Riffard <[email protected]>
Date : 2008-08-22 08:16 (2 days old)
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11398
Subject : hda_intel: IRQ timing workaround is activated for card #0. Suggest a bigger bdl_pos_adj.
Submitter : Frans Pop <elendil-EIBgga6/0yRmR6Xm/[email protected]>
Date : 2008-08-21 17:17 (3 days old)
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11388
Subject : 2.6.27-rc3 warns about MTRR range; only 3 of 16gb of memory is usable
Submitter : Joshua Hoblitt <[email protected]>
Date : 2008-08-20 17:38 (4 days old)
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11382
Subject : e1000e: 2.6.27-rc1 corrupts EEPROM/NVM
Submitter : David Vrabel <[email protected]>
Date : 2008-08-08 10:47 (16 days old)
References : http://marc.info/?l=linux-kernel&m=121819267211679&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11380
Subject : lockdep warning: cpu_add_remove_lock at:cpu_maps_update_begin+0x14/0x16
Submitter : Ingo Molnar <[email protected]>
Date : 2008-08-20 6:44 (4 days old)
References : http://marc.info/?l=linux-kernel&m=121921480931970&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11379
Subject : char/tpm: tpm_infineon no longer loaded for HP 2510p laptop
Submitter : Frans Pop <elendil-EIBgga6/0yRmR6Xm/[email protected]>
Date : 2008-08-18 13:40 (6 days old)
References : http://marc.info/?l=linux-kernel&m=121906698213329&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11357
Subject : Can not boot up with zd1211rw USB-Wlan Stick
Submitter : uwe <[email protected]>
Date : 2008-08-16 14:17 (8 days old)
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11356
Subject : Linux 2.6.27-rc3 - build failure: undefined reference to `.lockdep_count_forward_deps'
Submitter : Frans Pop <elendil-EIBgga6/0yRmR6Xm/[email protected]>
Date : 2008-08-16 19:11 (8 days old)
References : http://marc.info/?l=linux-kernel&m=121891396320127&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11355
Subject : Regression in 2.6.27-rc2 when cross-building the kernel
Submitter : Larry Finger <[email protected]>
Date : 2008-08-16 2:38 (8 days old)
References : http://marc.info/?l=linux-kernel&m=121885432118368&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11354
Subject : AMD Elan regression with 2.6.27-rc3
Submitter : Sean Young <[email protected]>
Date : 2008-08-15 18:37 (9 days old)
References : http://marc.info/?l=linux-kernel&m=121882578430056&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11343
Subject : SATA Cold Boot Problems with 2.6.27-rc[23] on nVidia 680i
Submitter : Manny Maxwell <[email protected]>
Date : 2008-08-14 4:16 (10 days old)
References : http://marc.info/?l=linux-kernel&m=121868782917600&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11342
Subject : Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Submitter : Alan D. Brunelle <[email protected]>
Date : 2008-08-13 23:03 (11 days old)
References : http://marc.info/?l=linux-kernel&m=121866876027629&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11340
Subject : LTP overnight run resulted in unusable box
Submitter : Alexey Dobriyan <[email protected]>
Date : 2008-08-13 9:24 (11 days old)
References : http://marc.info/?l=linux-kernel&m=121861951902949&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11336
Subject : 2.6.27-rc2:stall while mounting root fs
Submitter : Torsten Kaiser <just.for.lkml-gM/[email protected]>
Date : 2008-08-12 12:37 (12 days old)
References : http://marc.info/?l=linux-kernel&m=121854484015909&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11335
Subject : 2.6.27-rc2-git5 BUG: unable to handle kernel paging request
Submitter : Randy Dunlap <[email protected]>
Date : 2008-08-12 4:18 (12 days old)
References : http://marc.info/?l=linux-kernel&m=121851477201960&w=4
http://lkml.org/lkml/2008/8/16/274
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11334
Subject : myri10ge: use ioremap_wc: compilation failure on ARM
Submitter : Martin Michlmayr <[email protected]>
Date : 2008-08-10 11:25 (14 days old)
References : http://marc.info/?l=linux-netdev&m=121836771727632&w=2
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11308
Subject : tbench regression on each kernel release from 2.6.22 -> 2.6.28
Submitter : Christoph Lameter <cl-de/[email protected]>
Date : 2008-08-11 18:36 (13 days old)
References : http://marc.info/?l=linux-kernel&m=121847986119495&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11282
Subject : Please fix x86 defconfig regression
Submitter : Andi Kleen <andi-Vw/[email protected]>
Date : 2008-08-07 20:46 (17 days old)
References : http://marc.info/?l=linux-kernel&m=121814188805666&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11279
Subject : 2.6.27-rc0 Power Bugs with HP/Compaq Laptops
Submitter : Matt Parnell <[email protected]>
Date : 2008-08-07 14:57 (17 days old)
References : http://marc.info/?l=linux-kernel&m=121812108031685&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11272
Subject : BUG: parport_serial in 2.6.27-rc1 for NetMos Technology PCI 9835
Submitter : Jaswinder Singh <[email protected]>
Date : 2008-08-05 15:12 (19 days old)
References : http://marc.info/?l=linux-kernel&m=121794900319776&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11271
Subject : BUG: fealnx in 2.6.27-rc1
Submitter : Jaswinder Singh <[email protected]>
Date : 2008-08-05 14:58 (19 days old)
References : http://marc.info/?l=linux-netdev&m=121794762016830&w=4
http://lkml.org/lkml/2008/8/10/98
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11264
Subject : Invalid op opcode in kernel/workqueue
Submitter : Jean-Luc Coulon <[email protected]>
Date : 2008-08-07 04:18 (17 days old)
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11237
Subject : corrupt PMD after resume
Submitter : Alan Jenkins <[email protected]>
Date : 2008-08-02 9:51 (22 days old)
References : http://marc.info/?l=linux-kernel&m=121767073424952&w=4
Jeremy Fitzhardinge <[email protected]>
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11230
Subject : Kconfig no longer outputs a .config with freshly updated defconfigs
Submitter : Josh Boyer <[email protected]>
Date : 2008-08-02 16:03 (22 days old)
References : http://marc.info/?l=linux-kernel&m=121769306319391&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11224
Subject : Only three cores found on quad-core machine.
Submitter : Dave Jones <[email protected]>
Date : 2008-08-01 18:15 (23 days old)
References : http://marc.info/?l=linux-kernel&m=121761475224719&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11220
Subject : Screen stays black after resume
Submitter : Nico Schottelius <[email protected]>
Date : 2008-07-31 21:05 (24 days old)
References : http://marc.info/?l=linux-kernel&m=121753882422899&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11219
Subject : KVM modules break emergency reboot
Submitter : Zdenek Kabelac <[email protected]>
Date : 2008-08-01 20:25 (23 days old)
References : http://marc.info/?l=linux-kernel&m=121762241105336&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11215
Subject : INFO: possible recursive locking detected ps2_command
Submitter : Zdenek Kabelac <[email protected]>
Date : 2008-07-31 9:41 (24 days old)
References : http://marc.info/?l=linux-kernel&m=121749737011637&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11210
Subject : libata badness
Submitter : Kumar Gala <[email protected]>
Date : 2008-07-31 18:53 (24 days old)
References : http://marc.info/?l=linux-ide&m=121753059307310&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11209
Subject : 2.6.27-rc1 process time accounting
Submitter : Lukas Hejtmanek <xhejtman-8qz54MUs51PtwjQa/[email protected]>
Date : 2008-07-31 10:43 (24 days old)
References : http://marc.info/?l=linux-kernel&m=121750102917490&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11191
Subject : 2.6.26-git8: spinlock lockup in c1e_idle()
Submitter : Mikhail Kshevetskiy <[email protected]>
Date : 2008-07-24 03:22 (31 days old)
References : http://lkml.org/lkml/2008/7/23/317
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11141
Subject : no battery or DC status - Dell i1501
Submitter : Gu Rui <[email protected]>
Date : 2008-07-21 19:43 (34 days old)
Regressions with patches
------------------------
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11413
Subject : get_rtc_time() triggers NMI watchdog in hpet_rtc_interrupt()
Submitter : Mikael Pettersson <[email protected]>
Date : 2008-08-23 9:48 (1 days old)
References : http://marc.info/?l=linux-kernel&m=121948503224161&w=4
Patch : http://marc.info/?l=linux-kernel&m=121950734922457&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11409
Subject : build issue #564 for v2.6.27-rc4 : undefined reference to `NS8390p_init'
Submitter : Toralf Förster <[email protected]>
Date : 2008-08-22 8:33 (2 days old)
References : http://marc.info/?l=linux-kernel&m=121939410214677&w=4
Patch : http://marc.info/?l=linux-kernel&m=121943097320451&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11361
Subject : my servers with nvidia mcp55 nic don't work with msi in second kernel by kexec
Submitter : Yinghai Lu <[email protected]>
Date : 2008-08-17 6:25 (7 days old)
References : http://marc.info/?l=linux-kernel&m=121895439927053&w=4
Patch : http://marc.info/?l=linux-kernel&m=121917167232014&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11360
Subject : mpc8xxx_wdt.c doesn't build modular
Submitter : Dave Jones <[email protected]>
Date : 2008-08-17 08:07 (7 days old)
References : http://lkml.org/lkml/2008/8/12/465
Patch : http://lkml.org/lkml/2008/8/13/344
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11358
Subject : net: forcedeth call restore mac addr in nv_shutdown path
Submitter : Yinghai Lu <[email protected]>
Date : 2008-08-17 3:30 (7 days old)
References : http://marc.info/?l=linux-kernel&m=121894389018584&w=4
Patch : http://marc.info/?l=linux-kernel&m=121894389018584&w=4
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11276
Subject : build error: CONFIG_OPTIMIZE_INLINING=y causes gcc 4.2 to do stupid things
Submitter : Randy Dunlap <[email protected]>
Date : 2008-08-06 17:18 (18 days old)
References : http://marc.info/?l=linux-kernel&m=121804329014332&w=4
http://lkml.org/lkml/2008/7/22/353
Patch : http://lkml.org/lkml/2008/7/22/364
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11254
Subject : KVM: fix userspace ABI breakage
Submitter : Adrian Bunk <[email protected]>
Date : 21 Jul 2008 17:58:26 (0 days old)
References : http://lkml.org/lkml/2008/7/21/197
Patch : http://lkml.org/lkml/2008/7/21/197
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11207
Subject : VolanoMark regression with 2.6.27-rc1
Submitter : Zhang, Yanmin <yanmin_zhang-VuQAYsv1563Yd54FQh9/[email protected]>
Date : 2008-07-31 3:20 (24 days old)
References : http://marc.info/?l=linux-kernel&m=121747464114335&w=4
Peter Zijlstra <a.p.zijlstra-/NLkJaSkS4VmR6Xm/[email protected]> Dhaval Giani <[email protected]> Miao Xie <[email protected]>
Patch : http://marc.info/?l=linux-kernel&m=121922991027344&w=4
For details, please visit the bug entries and follow the links given in references.
As you can see, there is a Bugzilla entry for each of the listed regressions. There also is a Bugzilla entry used for tracking the regressions from 2.6.26, unresolved as well as resolved, at:
http://bugzilla.kernel.org/show_bug.cgi?id=11167
Please let me know if there are any Bugzilla entries that should be added to the list in there.
Thanks,
Rafael
[Bug #11141] no battery or DC status - Dell i1501
This message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11141
Subject : no battery or DC status - Dell i1501
Submitter : Gu Rui <[email protected]>
Date : 2008-07-21 19:43 (34 days old)
[Bug #11207] VolanoMark regression with 2.6.27-rc1
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11207
Subject : VolanoMark regression with 2.6.27-rc1
Submitter : Zhang, Yanmin <yanmin_zhang-VuQAYsv1563Yd54FQh9/[email protected]>
Date : 2008-07-31 3:20 (24 days old)
References : http://marc.info/?l=linux-kernel&m=121747464114335&w=4
Peter Zijlstra <a.p.zijlstra-/NLkJaSkS4VmR6Xm/[email protected]> Dhaval Giani <[email protected]> Miao Xie <[email protected]>
Patch : http://marc.info/?l=linux-kernel&m=121922991027344&w=4
[Bug #11191] 2.6.26-git8: spinlock lockup in c1e_idle()
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11191
Subject : 2.6.26-git8: spinlock lockup in c1e_idle()
Submitter : Mikhail Kshevetskiy <[email protected]>
Date : 2008-07-24 03:22 (31 days old)
References : http://lkml.org/lkml/2008/7/23/317
[Bug #11209] 2.6.27-rc1 process time accounting
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11209
Subject : 2.6.27-rc1 process time accounting
Submitter : Lukas Hejtmanek <xhejtman-8qz54MUs51PtwjQa/[email protected]>
Date : 2008-07-31 10:43 (24 days old)
References : http://marc.info/?l=linux-kernel&m=121750102917490&w=4
[Bug #11210] libata badness
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11210
Subject : libata badness
Submitter : Kumar Gala <[email protected]>
Date : 2008-07-31 18:53 (24 days old)
References : http://marc.info/?l=linux-ide&m=121753059307310&w=4
Re: [Bug #11210] libata badness
Quoted 8 lines · Rafael J. Wysocki
This message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Quoted 6 lines
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11210
Subject : libata badness
Submitter : Kumar Gala <[email protected]>
Date : 2008-07-31 18:53 (24 days old)
References : http://marc.info/?l=linux-ide&m=121753059307310&w=4Handled-By Ben Dooks <[email protected]>
FWIW,
http://marc.info/?l=linux-kernel&m=121754161727539&w=4
So IMO handled-by is Kumar?
Re: [Bug #11210] libata badness
Quoted 17 lines · Jeff Garzik
Quoted 8 lines · Rafael J. Wysocki
This message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Quoted 6 lines
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11210
Subject : libata badness
Submitter : Kumar Gala <[email protected]>
Date : 2008-07-31 18:53 (24 days old)
References : http://marc.info/?l=linux-ide&m=121753059307310&w=4Handled-By Ben Dooks <[email protected]>
Quoted 5 lines
FWIW,
http://marc.info/?l=linux-kernel&m=121754161727539&w=4
So IMO handled-by is Kumar?
OK
[Bug #11219] KVM modules break emergency reboot
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11219
Subject : KVM modules break emergency reboot
Submitter : Zdenek Kabelac <[email protected]>
Date : 2008-08-01 20:25 (23 days old)
References : http://marc.info/?l=linux-kernel&m=121762241105336&w=4
[Bug #11220] Screen stays black after resume
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11220
Subject : Screen stays black after resume
Submitter : Nico Schottelius <[email protected]>
Date : 2008-07-31 21:05 (24 days old)
References : http://marc.info/?l=linux-kernel&m=121753882422899&w=4
[Bug #11215] INFO: possible recursive locking detected ps2_command
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11215
Subject : INFO: possible recursive locking detected ps2_command
Submitter : Zdenek Kabelac <[email protected]>
Date : 2008-07-31 9:41 (24 days old)
References : http://marc.info/?l=linux-kernel&m=121749737011637&w=4
[Bug #11237] corrupt PMD after resume
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11237
Subject : corrupt PMD after resume
Submitter : Alan Jenkins <[email protected]>
Date : 2008-08-02 9:51 (22 days old)
References : http://marc.info/?l=linux-kernel&m=121767073424952&w=4
Jeremy Fitzhardinge <[email protected]>
[Bug #11230] Kconfig no longer outputs a .config with freshly updated defconfigs
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11230
Subject : Kconfig no longer outputs a .config with freshly updated defconfigs
Submitter : Josh Boyer <[email protected]>
Date : 2008-08-02 16:03 (22 days old)
References : http://marc.info/?l=linux-kernel&m=121769306319391&w=4
[Bug #11224] Only three cores found on quad-core machine.
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11224
Subject : Only three cores found on quad-core machine.
Submitter : Dave Jones <[email protected]>
Date : 2008-08-01 18:15 (23 days old)
References : http://marc.info/?l=linux-kernel&m=121761475224719&w=4
[Bug #11271] BUG: fealnx in 2.6.27-rc1
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11271
Subject : BUG: fealnx in 2.6.27-rc1
Submitter : Jaswinder Singh <[email protected]>
Date : 2008-08-05 14:58 (19 days old)
References : http://marc.info/?l=linux-netdev&m=121794762016830&w=4
http://lkml.org/lkml/2008/8/10/98
Re: [Bug #11271] BUG: fealnx in 2.6.27-rc1
Quoted 8 lines · Rafael J. Wysocki
This message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Quoted 7 lines
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11271
Subject : BUG: fealnx in 2.6.27-rc1
Submitter : Jaswinder Singh <[email protected]>
Date : 2008-08-05 14:58 (19 days old)
References : http://marc.info/?l=linux-netdev&m=121794762016830&w=4http://lkml.org/lkml/2008/8/10/98Handled-By Francois Romieu <[email protected]>
Jaswinder, does reverting 28cd4289abc2c8db90344ee4ff064a9bdf086fdf help?
That's the only material change to fealnx itself in years.
If not, any chance you could bisect this problem, and add more info to the bug?
[Bug #11264] Invalid op opcode in kernel/workqueue
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11264
Subject : Invalid op opcode in kernel/workqueue
Submitter : Jean-Luc Coulon <[email protected]>
Date : 2008-08-07 04:18 (17 days old)
[Bug #11254] KVM: fix userspace ABI breakage
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11254
Subject : KVM: fix userspace ABI breakage
Submitter : Adrian Bunk <[email protected]>
Date : 21 Jul 2008 17:58:26 (0 days old)
References : http://lkml.org/lkml/2008/7/21/197
Re: [Bug #11254] KVM: fix userspace ABI breakage
Quoted 8 lines · Rafael J. Wysocki
This message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Quoted 7 lines
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11254
Subject : KVM: fix userspace ABI breakage
Submitter : Adrian Bunk <[email protected]>
Date : 21 Jul 2008 17:58:26 (0 days old)
References : http://lkml.org/lkml/2008/7/21/197Handled-By Adrian Bunk <[email protected]>
The discussion in Bugzilla whether it is a regression at all can be condensed to the following question:
Can a struct that is part of the 2.6.26 userspace headers be defined to be part of an "experimental ABI" and therefore be changed?
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedRe: [Bug #11254] KVM: fix userspace ABI breakage
Quoted 4 lines · Adrian Bunk
The discussion in Bugzilla whether it is a regression at all can be condensed to the following question:
Can a struct that is part of the 2.6.26 userspace headers be defined to be part of an "experimental ABI" and therefore be changed?
It is part of the experimental ABI.
However, as I'm going to apply your patch (as being the simplest fix, and as there is no measurable performance impact), the question is moot.
Signature · 1 line
error compiling committee.c: too many arguments to function
[Bug #11279] 2.6.27-rc0 Power Bugs with HP/Compaq Laptops
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11279
Subject : 2.6.27-rc0 Power Bugs with HP/Compaq Laptops
Submitter : Matt Parnell <[email protected]>
Date : 2008-08-07 14:57 (17 days old)
References : http://marc.info/?l=linux-kernel&m=121812108031685&w=4
[Bug #11276] build error: CONFIG_OPTIMIZE_INLINING=y causes gcc 4.2 to do stupid things
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11276
Subject : build error: CONFIG_OPTIMIZE_INLINING=y causes gcc 4.2 to do stupid things
Submitter : Randy Dunlap <[email protected]>
Date : 2008-08-06 17:18 (18 days old)
References : http://marc.info/?l=linux-kernel&m=121804329014332&w=4
http://lkml.org/lkml/2008/7/22/353
[Bug #11272] BUG: parport_serial in 2.6.27-rc1 for NetMos Technology PCI 9835
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11272
Subject : BUG: parport_serial in 2.6.27-rc1 for NetMos Technology PCI 9835
Submitter : Jaswinder Singh <[email protected]>
Date : 2008-08-05 15:12 (19 days old)
References : http://marc.info/?l=linux-kernel&m=121794900319776&w=4
[Bug #11282] Please fix x86 defconfig regression
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11282
Subject : Please fix x86 defconfig regression
Submitter : Andi Kleen <andi-Vw/[email protected]>
Date : 2008-08-07 20:46 (17 days old)
References : http://marc.info/?l=linux-kernel&m=121814188805666&w=4
[Bug #11334] myri10ge: use ioremap_wc: compilation failure on ARM
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11334
Subject : myri10ge: use ioremap_wc: compilation failure on ARM
Submitter : Martin Michlmayr <[email protected]>
Date : 2008-08-10 11:25 (14 days old)
References : http://marc.info/?l=linux-netdev&m=121836771727632&w=2
Re: [Bug #11334] myri10ge: use ioremap_wc: compilation failure on ARM
Quoted 6 lines · Rafael J. Wysocki
This message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Yes, this is still there.
Signature · 2 lines
Martin Michlmayr http://www.cyrius.com/
Re: [Bug #11334] myri10ge: use ioremap_wc: compilation failure on ARM
Quoted 9 lines · Martin Michlmayr
Quoted 6 lines · Rafael J. Wysocki
This message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Yes, this is still there.
Thanks for the update.
Rafael
[Bug #11335] 2.6.27-rc2-git5 BUG: unable to handle kernel paging request
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11335
Subject : 2.6.27-rc2-git5 BUG: unable to handle kernel paging request
Submitter : Randy Dunlap <[email protected]>
Date : 2008-08-12 4:18 (12 days old)
References : http://marc.info/?l=linux-kernel&m=121851477201960&w=4
http://lkml.org/lkml/2008/8/16/274
[Bug #11336] 2.6.27-rc2:stall while mounting root fs
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11336
Subject : 2.6.27-rc2:stall while mounting root fs
Submitter : Torsten Kaiser <just.for.lkml-gM/[email protected]>
Date : 2008-08-12 12:37 (12 days old)
References : http://marc.info/?l=linux-kernel&m=121854484015909&w=4
[Bug #11308] tbench regression on each kernel release from 2.6.22 -> 2.6.28
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11308
Subject : tbench regression on each kernel release from 2.6.22 -> 2.6.28
Submitter : Christoph Lameter <cl-de/[email protected]>
Date : 2008-08-11 18:36 (13 days old)
References : http://marc.info/?l=linux-kernel&m=121847986119495&w=4
[Bug #11340] LTP overnight run resulted in unusable box
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11340
Subject : LTP overnight run resulted in unusable box
Submitter : Alexey Dobriyan <[email protected]>
Date : 2008-08-13 9:24 (11 days old)
References : http://marc.info/?l=linux-kernel&m=121861951902949&w=4
[Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11342
Subject : Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Submitter : Alan D. Brunelle <[email protected]>
Date : 2008-08-13 23:03 (11 days old)
References : http://marc.info/?l=linux-kernel&m=121866876027629&w=4
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 6 lines · Rafael J. Wysocki
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Quoted 6 lines
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11342
Subject : Linux 2.6.27-rc3: kernel BUG atmm/vmalloc.c- bisected
Submitter : Alan D. Brunelle <[email protected]>
Date : 2008-08-13 23:03 (11 days old)
References : http://marc.info/?l=linux-kernel&m=121866876027629&w=4Handled-By Andrew Morton <akpm-de/[email protected]>
This one makes no sense. It's triggering a BUG_ON(in_interrupt()), but then the call chain shows that there is no interrupt going on.
Also, the bisection is senseless - there's a trivial change wrt "do_one_initcall()" that got merged, but everything else is trivial about lguest and has nothing to do with the whole CPU-init thing. But if it was that initcall one, then "git bisect" woul have pointed to it, not the merge. And the merge itself had no conflicts or anything else going on..
The fact that it came and went later also implies that it's probably just some timing-dependent thing or some subtle memory corruption, making the bisection result even less likely to be exact.
But I'm adding Arjan and Rusty to the Cc, because that merge was takign Rusty's branch, and the "do_one_initcall()" is Arjan's commit. Since undoing that merge apparently does fix it, I'm wondering if something there just does end up triggering the problem.
The do_one_commit() thing _is_ in the path of sys_init_module(), so it _is_ at least somewhat relevant from an oops standpoint.
One thing the "do_one_commit()" thing does is to put more pressure on the stack due to that whole buffer for the printk's going on.
Alan, can you try
- seeing how consistent it is with one kernel (ie boot a known-bad kernel a few times just to see if it really is 100% consistent)
- try enabling 'initcall_debug' on the kernel command line, to (a) see the new code actually do something and (b) see what it is actually calling just before.
Hmm..
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 18 lines · Linus Torvalds
Quoted 11 lines · Rafael J. Wysocki
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11342
Subject : Linux 2.6.27-rc3: kernel BUG atmm/vmalloc.c- bisected
Submitter : Alan D. Brunelle <[email protected]>
Date : 2008-08-13 23:03 (11 days old)
References : http://marc.info/?l=linux-kernel&m=121866876027629&w=4Handled-By Andrew Morton <akpm-de/[email protected]>This one makes no sense. It's triggering a BUG_ON(in_interrupt()), but then the call chain shows that there is no interrupt going on.
Also, the bisection is senseless - there's a trivial change wrt "
do_one_initcall()" that got merged, but everything else is trivial about lguest and has nothing to do with the whole CPU-init thing. But if it was that initcall one, then "git bisect" woul have pointed to it, not the merge. And the merge itself had no conflicts or anything else going on..
Quoted 7 lines
The fact that it came and went later also implies that it's probably just some timing-dependent thing or some subtle memory corruption, making the bisection result even less likely to be exact.
But I'm adding Arjan and Rusty to the Cc, because that merge was takign Rusty's branch, and the "
do_one_initcall()" is Arjan's commit. Since undoing that merge apparently does fix it, I'm wondering if something there just does end up triggering the problem.The
do_one_commit()thing _is_ in the path ofsys_init_module(), so it _is_ at least somewhat relevant from an oops standpoint.One thing the "
do_one_commit()" thing does is to put more pressure on the stack due to that whole buffer for the printk's going on.
but it's 64 bit.. with 8Kb stack and separate irq stacks. I'd be surprised if we blow that this easily. the trace is a tad long with a long ACPI call chain.
Wonder what gcc is in use?
(newer ones tend to be a ton better... but maybe Alex is using a really old one)
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 4 lines · Arjan van de Ven
Wonder what gcc is in use?
(newer ones tend to be a ton better... but maybe Alex is using a really old one)
I'm running Ubuntu 8.04 w/ gcc:
gcc (GCC) 4.2.3 (Ubuntu 4.2.3-2ubuntu7)
Alan
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Linus TorvaldsQuoted 3 lines · Linus Torvalds
This one makes no sense. It's triggering a BUG_ON(in_interrupt()), but then the call chain shows that there is no interrupt going on.
Ahh, later in that thread there's another totally unrelated oops in debug_mutex_add_waiter().
I'd guess that it is really wild pointer corrupting memory, quite possibly due to a double free or something like that. Alan - it would be good to run with DEBUG_PAGE_ALLOC and SLUB debugging etc if you don't already do that?
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 8 lines · Linus Torvalds
Quoted 2 lines · Linus Torvalds
This one makes no sense. It's triggering a BUG_ON(in_interrupt()), but then the call chain shows that there is no interrupt going on.
Ahh, later in that thread there's another totally unrelated oops in
debug_mutex_add_waiter().
Quoted 7 lines
I'd guess that it is really wild pointer corrupting memory, quite possibly due to a double free or something like that. Alan - it would be good to run with
DEBUG_PAGE_ALLOCand SLUB debugging etc if you don't already do that?Linus
I'll add those in - as to the repeatability: The "bad" kernels seem to repeat quite reliably - not only in terms of counts (5 or 6 times in a row before trying something else), but also in terms of the "what" - either the original issue () or the other kernel with the later issue (debug_mutex_add_waiter). That's /goodness/ in that it should help narrow it down.
I'll make sure the kernel is still failing this morning, and then add in DEBUG_PAGE_ALLOC and if that doesn't help, SLUB debugging...
Alan
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Before adding any more debugging, this is the status of my kernel boots:
3 times in a row w/ this same error. (Primary problem is the same, secondary stacks differ of course.)
Alan
Loading, please [ 6.482953] busybox used greatest stack depth: 4840 bytes left
wait...
[ 6.521876] all_generic_ide used greatest stack depth: 4784 bytes left Begin: Loading essential drivers... ... [ 6.625509] fuse init (API version 7.9) [ 6.625509] modprobe used greatest stack depth: 1720 bytes left [ 6.644854] ACPI: SSDT CFFD0D0A, 08C4 (r1 HPQOEM CPU_TM2 1 MSFT 100000E) [ 6.651489] BUG: unable to handle kernel NULL pointer dereference at 0000000000000858 [ 6.655631] IP: [<ffffffff8025e302>] debug_mutex_add_waiter+0x32/0x80 [ 6.655631] PGD 21a0a4067 PUD 21a4bd067 PMD 0 [ 6.655631] Oops: 0002 [1] SMP [ 6.655631] CPU 1 [ 6.655631] Modules linked in: processor(+) fan thermal_sys fuse [ 6.655631] Pid: 1259, comm: modprobe Not tainted 2.6.27-rc3 #29 [ 6.655631] RIP: 0010:[<ffffffff8025e302>] [<ffffffff8025e302>] debug_mutex_add_waiter+0x32/0x80 [ 6.655631] RSP: 0018:ffff88021a4e7998 EFLAGS: 00010002 [ 6.655631] RAX: 0000000000000000 RBX: ffff88021a4e79d8 RCX: 0000000000000000 [ 6.655631] RDX: 0000000000000001 RSI: ffff88021a4e79d8 RDI: ffffffffa0091a60 [ 6.655631] RBP: ffff88021a4e79b8 R08: ffffffff811deff0 R09: ffff8800a6fdb000 [ 6.655631] R10: ffffffffa008f524 R11: 0000000000000000 R12: ffffffffa0091a60 [ 6.655631] R13: ffff88021a4e6000 R14: ffff88021a9c40a0 R15: ffffffffa0091a98 [ 6.655631] FS: 00007f233f11d6e0(0000) GS:ffff88022fc02a00(0000) knlGS:0000000000000000 [ 6.655631] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 6.655631] CR2: 0000000000000858 CR3: 000000021a07e000 CR4: 00000000000006e0 [ 6.655631] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 6.655631] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 6.655631] Process modprobe (pid: 1259, threadinfo ffff88021a4e6000, task ffff88021a9c40a0) [ 6.655631] Stack: 0000000000000000 ffffffffa0091a60 0000000000000246 ffffffffa008f524 [ 6.655631] ffff88021a4e7a38 ffffffff8049f596 ffffffffa008f524 ffffffffa0091a18 [ 6.655631] ffff88021a4e79d8 ffff88021a4e79d8 1111111111111111 1111111111111111 [ 6.655631] Call Trace: [ 6.655631] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 6.655631] [<ffffffff8049f596>] mutex_lock_nested+0xa6/0x250 [ 6.655631] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 6.655631] [<ffffffff803635c4>] ? idr_pre_get+0x44/0x90 [ 6.655631] [<ffffffffa008f524>] get_idr+0x44/0xa0 [thermal_sys] [ 6.655631] [<ffffffffa008fe43>] thermal_cooling_device_register+0x83/0x250 [thermal_sys] [ 6.655631] [<ffffffffa019b2a3>] acpi_processor_start+0x64b/0x774 [processor] [ 6.655631] [<ffffffff8031a94b>] ? __sysfs_add_one+0x6b/0xa0 [ 6.655631] [<ffffffff8031ba3c>] ? sysfs_do_create_link+0xbc/0x150 [ 6.655631] [<ffffffff803a7f5e>] acpi_start_single_object+0x2d/0x52 [ 6.655631] [<ffffffff803a9556>] acpi_device_probe+0x7e/0x92 [ 6.655631] [<ffffffff803dd3eb>] driver_probe_device+0x9b/0x1a0 [ 6.655631] [<ffffffff803dd576>] __driver_attach+0x86/0x90 [ 6.655631] [<ffffffff803dd4f0>] ? __driver_attach+0x0/0x90 [ 6.655631] [<ffffffff803dc93d>] bus_for_each_dev+0x5d/0x90 [ 6.655631] [<ffffffff803dd22c>] driver_attach+0x1c/0x20 [ 6.655631] [<ffffffff803dcf79>] bus_add_driver+0x1e9/0x260 [ 6.655631] [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor] [ 6.655631] [<ffffffff803dd74f>] driver_register+0x5f/0x140 [ 6.655631] [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor] [ 6.655631] [<ffffffff803a9866>] acpi_bus_register_driver+0x3e/0x40 [ 6.655631] [<ffffffffa0222094>] acpi_processor_init+0x94/0x107 [processor] [ 6.655631] [<ffffffff80209040>] _stext+0x40/0x180 [ 6.655631] [<ffffffff802a8911>] ? __vunmap+0xa1/0x110 [ 6.655631] [<ffffffff802676c2>] sys_init_module+0x142/0x1dc0 [ 6.655631] [<ffffffff80367b16>] ? __up_read+0x46/0xb0 [ 6.655631] [<ffffffff8048e570>] ? cpu_down+0x0/0x70 [ 6.655631] [<ffffffff8020c34b>] system_call_fastpath+0x16/0x1b [ 6.655631] [ 6.655631] [ 6.655631] Code: 20 48 89 5d e8 4c 89 65 f0 48 89 f3 4c 89 6d f8 8b 47 08 49 89 d5 49 89 fc 89 c2 25 ff ff 00 00 c1 ea 10 39 c2 74 1d 49 8b 4 [ 6.655631] RIP [<ffffffff8025e302>] debug_mutex_add_waiter+0x32/0x80 [ 6.655631] RSP <ffff88021a4e7998> [ 6.655631] CR2: 0000000000000858 [ 6.655631] ---[ end trace 8bbd31df1403e48e ]--- [ 7.024992] modprobe used greatest stack depth: 408 bytes left [ 7.030988] BUG: unable to handle kernel NULL pointer dereference at 0000000000000048 [ 7.031053] IP: [<ffffffff8023f39c>] do_exit+0x28c/0xa10 [ 7.031053] PGD 0 [ 7.031053] Oops: 0000 [2] SMP [ 7.031053] CPU 1 [ 7.031053] Modules linked in: processor(+) fan thermal_sys fuse [ 7.031053] Pid: 1259, comm: modprobe Tainted: G D 2.6.27-rc3 #29 [ 7.031053] RIP: 0010:[<ffffffff8023f39c>] [<ffffffff8023f39c>] do_exit+0x28c/0xa10 [ 7.031053] RSP: 0018:ffff88021a4e77e8 EFLAGS: 00010246 [ 7.031053] RAX: 0000000000000000 RBX: 0000000000000198 RCX: 0000000000000000 [ 7.031053] RDX: 0000000000000000 RSI: ffffffff802740d0 RDI: 0000000000000000 [ 7.031053] RBP: ffff88021a4e7848 R08: 0000000000000001 R09: 0000000000000000 [ 7.031053] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88021a9c40a0 [ 7.031053] R13: 0000000000000009 R14: ffff88021a4e78e8 R15: ffff88021a18b8a0 [ 7.031053] FS: 0000000000000000(0000) GS:ffff88022fc02a00(0000) knlGS:0000000000000000 [ 7.031053] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 7.031053] CR2: 0000000000000048 CR3: 0000000000201000 CR4: 00000000000006e0 [ 7.031053] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 7.031053] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 7.031053] Process modprobe (pid: 1259, threadinfo ffff88021a4e6000, task ffff88021a9c40a0) [ 7.031053] Stack: 0000000000000001 0000000000000001 0000000000000040 0000000000000001 [ 7.031053] ffff88021a4e7828 ffffffff803c7d59 0000000000000092 0000000000000092 [ 7.031053] ffff88021a4e78e8 0000000000000009 ffff88021a4e78e8 ffff88021a18b8a0 [ 7.031053] Call Trace: [ 7.031053] [<ffffffff803c7d59>] ? do_unblank_screen+0x19/0x130 [ 7.031053] [<ffffffff804a1a57>] oops_end+0x87/0x90 [ 7.031053] [<ffffffff804a3d13>] do_page_fault+0x663/0x800 [ 7.031053] [<ffffffff804a162d>] error_exit+0x0/0x9a [ 7.031053] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.031053] [<ffffffff8025e302>] ? debug_mutex_add_waiter+0x32/0x80 [ 7.031053] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.031053] [<ffffffff8049f596>] mutex_lock_nested+0xa6/0x250 [ 7.031053] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.031053] [<ffffffff803635c4>] ? idr_pre_get+0x44/0x90 [ 7.031053] [<ffffffffa008f524>] get_idr+0x44/0xa0 [thermal_sys] [ 7.031053] [<ffffffffa008fe43>] thermal_cooling_device_register+0x83/0x250 [thermal_sys] [ 7.031053] [<ffffffffa019b2a3>] acpi_processor_start+0x64b/0x774 [processor] [ 7.031053] [<ffffffff8031a94b>] ? __sysfs_add_one+0x6b/0xa0 [ 7.031053] [<ffffffff8031ba3c>] ? sysfs_do_create_link+0xbc/0x150 [ 7.031053] [<ffffffff803a7f5e>] acpi_start_single_object+0x2d/0x52 [ 7.031053] [<ffffffff803a9556>] acpi_device_probe+0x7e/0x92 [ 7.031053] [<ffffffff803dd3eb>] driver_probe_device+0x9b/0x1a0 [ 7.031053] [<ffffffff803dd576>] __driver_attach+0x86/0x90 [ 7.031053] [<ffffffff803dd4f0>] ? __driver_attach+0x0/0x90 [ 7.031053] [<ffffffff803dc93d>] bus_for_each_dev+0x5d/0x90 [ 7.031053] [<ffffffff803dd22c>] driver_attach+0x1c/0x20 [ 7.031053] [<ffffffff803dcf79>] bus_add_driver+0x1e9/0x260 [ 7.031053] [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor] [ 7.031053] [<ffffffff803dd74f>] driver_register+0x5f/0x140 [ 7.031053] [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor] [ 7.031053] [<ffffffff803a9866>] acpi_bus_register_driver+0x3e/0x40 [ 7.031053] [<ffffffffa0222094>] acpi_processor_init+0x94/0x107 [processor] [ 7.031053] [<ffffffff80209040>] _stext+0x40/0x180 [ 7.031053] [<ffffffff802a8911>] ? __vunmap+0xa1/0x110 [ 7.031053] [<ffffffff802676c2>] sys_init_module+0x142/0x1dc0 [ 7.031053] [<ffffffff80367b16>] ? __up_read+0x46/0xb0 [ 7.031053] [<ffffffff8048e570>] ? cpu_down+0x0/0x70 [ 7.031053] [<ffffffff8020c34b>] system_call_fastpath+0x16/0x1b [ 7.031053] [ 7.031053] [ 7.031053] Code: e8 8a e3 0e 00 8b 45 b8 85 c0 74 16 49 8b 84 24 40 07 00 00 8b 80 4c 01 00 00 85 c0 0f 85 77 07 00 00 49 8b 44 24 08 48 8b 4 [ 7.031053] RIP [<ffffffff8023f39c>] do_exit+0x28c/0xa10 [ 7.031053] RSP <ffff88021a4e77e8> [ 7.031053] CR2: 0000000000000048 [ 7.421063] ------------[ cut here ]------------ [ 7.424883] WARNING: at kernel/sched_fair.c:884 hrtick_start_fair+0x187/0x190() [ 7.424883] Modules linked in: processor(+) fan thermal_sys fuse [ 7.424883] Pid: 1259, comm: modprobe Tainted: G D 2.6.27-rc3 #29 [ 7.424883] [ 7.424883] Call Trace: [ 7.424883] <IRQ> [<ffffffff8023baef>] warn_on_slowpath+0x5f/0x80 [ 7.424883] [<ffffffff8022d927>] hrtick_start_fair+0x187/0x190 [ 7.424883] [<ffffffff8022ec79>] enqueue_task_fair+0x49/0x250 [ 7.424883] [<ffffffff8022c290>] enqueue_task+0x50/0x60 [ 7.424883] [<ffffffff8022c2c3>] activate_task+0x23/0x40 [ 7.424883] [<ffffffff80231653>] try_to_wake_up+0x253/0x280 [ 7.424883] [<ffffffff8023168d>] default_wake_function+0xd/0x10 [ 7.424883] [<ffffffff802521d1>] autoremove_wake_function+0x11/0x40 [ 7.424883] [<ffffffff8022bd6a>] __wake_up_common+0x5a/0x90 [ 7.424883] [<ffffffff8022d373>] __wake_up+0x43/0x70 [ 7.424883] [<ffffffff8024e910>] ? delayed_work_timer_fn+0x0/0x40 [ 7.424883] [<ffffffff8024df68>] insert_work+0x48/0x50 [ 7.424883] [<ffffffff8024e8f1>] __queue_work+0x31/0x50 [ 7.424883] [<ffffffff8024e942>] delayed_work_timer_fn+0x32/0x40 [ 7.424883] [<ffffffff80245e5b>] run_timer_softirq+0x1bb/0x230 [ 7.424883] [<ffffffff80255afa>] ? ktime_get_ts+0x4a/0x60 [ 7.424883] [<ffffffff8024157a>] __do_softirq+0x7a/0xf0 [ 7.424883] [<ffffffff8025cd8e>] ? tick_program_event+0x3e/0x70 [ 7.424883] [<ffffffff8020d69c>] call_softirq+0x1c/0x30 [ 7.424883] [<ffffffff8020f28d>] do_softirq+0x3d/0x80 [ 7.424883] [<ffffffff802414f5>] irq_exit+0x85/0x90 [ 7.424883] [<ffffffff8021d648>] smp_apic_timer_interrupt+0x88/0xc0 [ 7.424883] [<ffffffff8020d0e6>] apic_timer_interrupt+0x66/0x70 [ 7.424883] <EOI> [<ffffffff804a1a1a>] ? oops_end+0x4a/0x90 [ 7.424883] [<ffffffff804a3d13>] ? do_page_fault+0x663/0x800 [ 7.424883] [<ffffffff804a162d>] ? error_exit+0x0/0x9a [ 7.424883] [<ffffffff802740d0>] ? release_css_set_taskexit+0x0/0x10 [ 7.424883] [<ffffffff8023f39c>] ? do_exit+0x28c/0xa10 [ 7.424883] [<ffffffff8023f376>] ? do_exit+0x266/0xa10 [ 7.424883] [<ffffffff803c7d59>] ? do_unblank_screen+0x19/0x130 [ 7.424883] [<ffffffff804a1a57>] ? oops_end+0x87/0x90 [ 7.424883] [<ffffffff804a3d13>] ? do_page_fault+0x663/0x800 [ 7.424883] [<ffffffff804a162d>] ? error_exit+0x0/0x9a [ 7.424883] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.424883] [<ffffffff8025e302>] ? debug_mutex_add_waiter+0x32/0x80 [ 7.424883] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.424883] [<ffffffff8049f596>] ? mutex_lock_nested+0xa6/0x250 [ 7.424883] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.424883] [<ffffffff803635c4>] ? idr_pre_get+0x44/0x90 [ 7.424883] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.424883] [<ffffffffa008fe43>] ? thermal_cooling_device_register+0x83/0x250 [thermal_sys] [ 7.424883] [<ffffffffa019b2a3>] ? acpi_processor_start+0x64b/0x774 [processor] [ 7.424883] [<ffffffff8031a94b>] ? __sysfs_add_one+0x6b/0xa0 [ 7.424883] [<ffffffff8031ba3c>] ? sysfs_do_create_link+0xbc/0x150 [ 7.424883] [<ffffffff803a7f5e>] ? acpi_start_single_object+0x2d/0x52 [ 7.424883] [<ffffffff803a9556>] ? acpi_device_probe+0x7e/0x92 [ 7.424883] [<ffffffff803dd3eb>] ? driver_probe_device+0x9b/0x1a0 [ 7.424883] [<ffffffff803dd576>] ? __driver_attach+0x86/0x90 [ 7.424883] [<ffffffff803dd4f0>] ? __driver_attach+0x0/0x90 [ 7.424883] [<ffffffff803dc93d>] ? bus_for_each_dev+0x5d/0x90 [ 7.424883] [<ffffffff803dd22c>] ? driver_attach+0x1c/0x20 [ 7.424883] [<ffffffff803dcf79>] ? bus_add_driver+0x1e9/0x260 [ 7.424883] [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor] [ 7.424883] [<ffffffff803dd74f>] ? driver_register+0x5f/0x140 [ 7.424883] [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor] [ 7.424883] [<ffffffff803a9866>] ? acpi_bus_register_driver+0x3e/0x40 [ 7.424883] [<ffffffffa0222094>] ? acpi_processor_init+0x94/0x107 [processor] [ 7.424883] [<ffffffff80209040>] ? _stext+0x40/0x180 [ 7.424883] [<ffffffff802a8911>] ? __vunmap+0xa1/0x110 [ 7.424883] [<ffffffff802676c2>] ? sys_init_module+0x142/0x1dc0 [ 7.424883] [<ffffffff80367b16>] ? __up_read+0x46/0xb0 [ 7.424883] [<ffffffff8048e570>] ? cpu_down+0x0/0x70 [ 7.424883] [<ffffffff8020c34b>] ? system_call_fastpath+0x16/0x1b [ 7.424883] [ 7.424883] ---[ end trace 8bbd31df1403e48e ]--- [ 7.424883] ------------[ cut here ]------------ [ 7.424883] kernel BUG at kernel/sched.c:1155! [ 7.424883] invalid opcode: 0000 [3] SMP [ 7.424883] CPU 1 [ 7.424883] Modules linked in: processor(+) fan thermal_sys fuse [ 7.424883] Pid: 1259, comm: modprobe Tainted: G D W 2.6.27-rc3 #29 [ 7.424883] RIP: 0010:[<ffffffff8022cc2b>] [<ffffffff8022cc2b>] resched_task+0x6b/0x70 [ 7.424883] RSP: 0018:ffff88022f0abce0 EFLAGS: 00010046 [ 7.424883] RAX: 0000000000000709 RBX: 0000000004bfe971 RCX: ffff88021a4e6000 [ 7.424883] RDX: 0000000000000709 RSI: 0000000000000000 RDI: ffff88021a9c40a0 [ 7.424883] RBP: ffff88022f0abce0 R08: ffff88022f180038 R09: ffff88021a9c40d8 [ 7.424883] R10: ffffffff810c9e00 R11: 0000000000000000 R12: ffff8800a6fc9000 [ 7.424883] R13: ffffffff810c9e00 R14: ffff88021a9c40a0 R15: 0000000000000001 [ 7.424883] FS: 0000000000000000(0000) GS:ffff88022fc02a00(0000) knlGS:0000000000000000 [ 7.424883] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 7.424883] CR2: 0000000000000048 CR3: 0000000000201000 CR4: 00000000000006e0 [ 7.424883] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 7.424883] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 7.424883] Process modprobe (pid: 1259, threadinfo ffff88021a4e6000, task ffff88021a9c40a0) [ 7.424883] Stack: ffff88022f0abd20 ffffffff802387b3 0000000000000400 0000000000400000 [ 7.424883] ffff88022f180000 ffff8800280a4e00 0000000000000001 0000000000000003 [ 7.424883] ffff88022f0abd70 ffffffff802314bf 0000000100000001 0000000000000000 [ 7.424883] Call Trace: [ 7.424883] <IRQ> [<ffffffff802387b3>] check_preempt_wakeup+0x133/0x1c0 [ 7.424883] [<ffffffff802314bf>] try_to_wake_up+0xbf/0x280 [ 7.424883] [<ffffffff8023168d>] default_wake_function+0xd/0x10 [ 7.424883] [<ffffffff802521d1>] autoremove_wake_function+0x11/0x40 [ 7.424883] [<ffffffff8022bd6a>] __wake_up_common+0x5a/0x90 [ 7.424883] [<ffffffff8022d373>] __wake_up+0x43/0x70 [ 7.424883] [<ffffffff8024e910>] ? delayed_work_timer_fn+0x0/0x40 [ 7.424883] [<ffffffff8024df68>] insert_work+0x48/0x50 [ 7.424883] [<ffffffff8024e8f1>] __queue_work+0x31/0x50 [ 7.424883] [<ffffffff8024e942>] delayed_work_timer_fn+0x32/0x40 [ 7.424883] [<ffffffff80245e5b>] run_timer_softirq+0x1bb/0x230 [ 7.424883] [<ffffffff80255afa>] ? ktime_get_ts+0x4a/0x60 [ 7.424883] [<ffffffff8024157a>] __do_softirq+0x7a/0xf0 [ 7.424883] [<ffffffff8025cd8e>] ? tick_program_event+0x3e/0x70 [ 7.424883] [<ffffffff8020d69c>] call_softirq+0x1c/0x30 [ 7.424883] [<ffffffff8020f28d>] do_softirq+0x3d/0x80 [ 7.424883] [<ffffffff802414f5>] irq_exit+0x85/0x90 [ 7.424883] [<ffffffff8021d648>] smp_apic_timer_interrupt+0x88/0xc0 [ 7.424883] [<ffffffff8020d0e6>] apic_timer_interrupt+0x66/0x70 [ 7.424883] <EOI> [<ffffffff804a1a1a>] ? oops_end+0x4a/0x90 [ 7.424883] [<ffffffff804a3d13>] ? do_page_fault+0x663/0x800 [ 7.424883] [<ffffffff804a162d>] ? error_exit+0x0/0x9a [ 7.424883] [<ffffffff802740d0>] ? release_css_set_taskexit+0x0/0x10 [ 7.424883] [<ffffffff8023f39c>] ? do_exit+0x28c/0xa10 [ 7.424883] [<ffffffff8023f376>] ? do_exit+0x266/0xa10 [ 7.424883] [<ffffffff803c7d59>] ? do_unblank_screen+0x19/0x130 [ 7.424883] [<ffffffff804a1a57>] ? oops_end+0x87/0x90 [ 7.424883] [<ffffffff804a3d13>] ? do_page_fault+0x663/0x800 [ 7.424883] [<ffffffff804a162d>] ? error_exit+0x0/0x9a [ 7.424883] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.424883] [<ffffffff8025e302>] ? debug_mutex_add_waiter+0x32/0x80 [ 7.424883] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.424883] [<ffffffff8049f596>] ? mutex_lock_nested+0xa6/0x250 [ 7.424883] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.424883] [<ffffffff803635c4>] ? idr_pre_get+0x44/0x90 [ 7.424883] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.424883] [<ffffffffa008fe43>] ? thermal_cooling_device_register+0x83/0x250 [thermal_sys] [ 7.424883] [<ffffffffa019b2a3>] ? acpi_processor_start+0x64b/0x774 [processor] [ 7.424883] [<ffffffff8031a94b>] ? __sysfs_add_one+0x6b/0xa0 [ 7.424883] [<ffffffff8031ba3c>] ? sysfs_do_create_link+0xbc/0x150 [ 7.424883] [<ffffffff803a7f5e>] ? acpi_start_single_object+0x2d/0x52 [ 7.424883] [<ffffffff803a9556>] ? acpi_device_probe+0x7e/0x92 [ 7.424883] [<ffffffff803dd3eb>] ? driver_probe_device+0x9b/0x1a0 [ 7.424883] [<ffffffff803dd576>] ? __driver_attach+0x86/0x90 [ 7.424883] [<ffffffff803dd4f0>] ? __driver_attach+0x0/0x90 [ 7.424883] [<ffffffff803dc93d>] ? bus_for_each_dev+0x5d/0x90 [ 7.424883] [<ffffffff803dd22c>] ? driver_attach+0x1c/0x20 [ 7.424883] [<ffffffff803dcf79>] ? bus_add_driver+0x1e9/0x260 [ 7.424883] [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor] [ 7.424883] [<ffffffff803dd74f>] ? driver_register+0x5f/0x140 [ 7.424883] [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor] [ 7.424883] [<ffffffff803a9866>] ? acpi_bus_register_driver+0x3e/0x40 [ 7.424883] [<ffffffffa0222094>] ? acpi_processor_init+0x94/0x107 [processor] [ 7.424883] [<ffffffff80209040>] ? _stext+0x40/0x180 [ 7.424883] [<ffffffff802a8911>] ? __vunmap+0xa1/0x110 [ 7.424883] [<ffffffff802676c2>] ? sys_init_module+0x142/0x1dc0 [ 7.424883] [<ffffffff80367b16>] ? __up_read+0x46/0xb0 [ 7.424883] [<ffffffff8048e570>] ? cpu_down+0x0/0x70 [ 7.424883] [<ffffffff8020c34b>] ? system_call_fastpath+0x16/0x1b [ 7.424883] [ 7.424883] [ 7.424883] Code: 8b 47 08 8b 50 1c 65 8b 04 25 24 00 00 00 39 c2 74 0d 0f ae f0 48 8b 47 08 f6 40 18 04 74 02 c9 c3 89 d7 ff 15 1f 7b 3c 00 c [ 7.424883] RIP [<ffffffff8022cc2b>] resched_task+0x6b/0x70 [ 7.424883] RSP <ffff88022f0abce0> [ 7.424883] ---[ end trace 8bbd31df1403e48e ]--- [ 7.424883] Kernel panic - not syncing: Aiee, killing interrupt handler! [ 7.424883] ------------[ cut here ]------------ [ 7.424883] WARNING: at kernel/smp.c:328 smp_call_function_mask+0x25a/0x260() [ 7.424883] Modules linked in: processor(+) fan thermal_sys fuse [ 7.424883] Pid: 1259, comm: modprobe Tainted: G D W 2.6.27-rc3 #29 [ 7.424883] [ 7.424883] Call Trace: [ 7.424883] <IRQ> [<ffffffff8023baef>] warn_on_slowpath+0x5f/0x80 [ 7.424883] [<ffffffff8026514a>] smp_call_function_mask+0x25a/0x260 [ 7.424883] [<ffffffff803695bd>] ? string+0x3d/0xd0 [ 7.424883] [<ffffffff80369a8b>] ? vsnprintf+0x43b/0x720 [ 7.424883] [<ffffffff803695bd>] ? string+0x3d/0xd0 [ 7.424883] [<ffffffff80369a8b>] ? vsnprintf+0x43b/0x720 [ 7.424883] [<ffffffff803695bd>] ? string+0x3d/0xd0 [ 7.424883] [<ffffffff803695bd>] ? string+0x3d/0xd0 [ 7.424883] [<ffffffff80369a8b>] ? vsnprintf+0x43b/0x720 [ 7.424883] [<ffffffff80368d5e>] ? number+0x2ae/0x2d0 [ 7.424883] [<ffffffff80368d5e>] ? number+0x2ae/0x2d0 [ 7.424883] [<ffffffff80269ecd>] ? kallsyms_lookup+0x5d/0xa0 [ 7.424883] [<ffffffff80368d5e>] ? number+0x2ae/0x2d0 [ 7.424883] [<ffffffff80369a8b>] ? vsnprintf+0x43b/0x720 [ 7.424883] [<ffffffff80369dd8>] ? sprintf+0x68/0x70 [ 7.424883] [<ffffffff803695bd>] ? string+0x3d/0xd0 [ 7.424883] [<ffffffff804a3fa3>] ? __atomic_notifier_call_chain+0x83/0xa0 [ 7.424883] [<ffffffff804a3f20>] ? __atomic_notifier_call_chain+0x0/0xa0 [ 7.424883] [<ffffffff804a0ef6>] ? _spin_unlock+0x26/0x30 [ 7.424883] [<ffffffff8021c470>] ? stop_this_cpu+0x0/0x30 [ 7.424883] [<ffffffff80265190>] smp_call_function+0x40/0x50 [ 7.424883] [<ffffffff8021c4f3>] native_smp_send_stop+0x23/0x40 [ 7.424883] [<ffffffff8023be3f>] panic+0xaf/0x190 [ 7.424883] [<ffffffff8023cc97>] ? printk+0x67/0x70 [ 7.424883] [<ffffffff8049f4e9>] ? mutex_unlock+0x9/0x10 [ 7.424883] [<ffffffff80256d11>] ? blocking_notifier_call_chain+0x11/0x20 [ 7.424883] [<ffffffff8023f979>] do_exit+0x869/0xa10 [ 7.424883] [<ffffffff803c7d59>] ? do_unblank_screen+0x19/0x130 [ 7.424883] [<ffffffff804a1a57>] oops_end+0x87/0x90 [ 7.424883] [<ffffffff8020e08e>] die+0x5e/0x90 [ 7.424883] [<ffffffff804a1f60>] do_trap+0x130/0x150 [ 7.424883] [<ffffffff8020e662>] do_invalid_op+0x92/0xb0 [ 7.424883] [<ffffffff8022cc2b>] ? resched_task+0x6b/0x70 [ 7.424883] [<ffffffff804a162d>] error_exit+0x0/0x9a [ 7.424883] [<ffffffff8022cc2b>] ? resched_task+0x6b/0x70 [ 7.424883] [<ffffffff802387b3>] check_preempt_wakeup+0x133/0x1c0 [ 7.424883] [<ffffffff802314bf>] try_to_wake_up+0xbf/0x280 [ 7.424883] [<ffffffff8023168d>] default_wake_function+0xd/0x10 [ 7.424883] [<ffffffff802521d1>] autoremove_wake_function+0x11/0x40 [ 7.424883] [<ffffffff8022bd6a>] __wake_up_common+0x5a/0x90 [ 7.424883] [<ffffffff8022d373>] __wake_up+0x43/0x70 [ 7.424883] [<ffffffff8024e910>] ? delayed_work_timer_fn+0x0/0x40 [ 7.424883] [<ffffffff8024df68>] insert_work+0x48/0x50 [ 7.424883] [<ffffffff8024e8f1>] __queue_work+0x31/0x50 [ 7.424883] [<ffffffff8024e942>] delayed_work_timer_fn+0x32/0x40 [ 7.424883] [<ffffffff80245e5b>] run_timer_softirq+0x1bb/0x230 [ 7.424883] [<ffffffff80255afa>] ? ktime_get_ts+0x4a/0x60 [ 7.424883] [<ffffffff8024157a>] __do_softirq+0x7a/0xf0 [ 7.424883] [<ffffffff8025cd8e>] ? tick_program_event+0x3e/0x70 [ 7.424883] [<ffffffff8020d69c>] call_softirq+0x1c/0x30 [ 7.424883] [<ffffffff8020f28d>] do_softirq+0x3d/0x80 [ 7.424883] [<ffffffff802414f5>] irq_exit+0x85/0x90 [ 7.424883] [<ffffffff8021d648>] smp_apic_timer_interrupt+0x88/0xc0 [ 7.424883] [<ffffffff8020d0e6>] apic_timer_interrupt+0x66/0x70 [ 7.424883] <EOI> [<ffffffff804a1a1a>] ? oops_end+0x4a/0x90 [ 7.424883] [<ffffffff804a3d13>] ? do_page_fault+0x663/0x800 [ 7.424883] [<ffffffff804a162d>] ? error_exit+0x0/0x9a [ 7.424883] [<ffffffff802740d0>] ? release_css_set_taskexit+0x0/0x10 [ 7.424883] [<ffffffff8023f39c>] ? do_exit+0x28c/0xa10 [ 7.424883] [<ffffffff8023f376>] ? do_exit+0x266/0xa10 [ 7.424883] [<ffffffff803c7d59>] ? do_unblank_screen+0x19/0x130 [ 7.424883] [<ffffffff804a1a57>] ? oops_end+0x87/0x90 [ 7.424883] [<ffffffff804a3d13>] ? do_page_fault+0x663/0x800 [ 7.424883] [<ffffffff804a162d>] ? error_exit+0x0/0x9a [ 7.424883] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.424883] [<ffffffff8025e302>] ? debug_mutex_add_waiter+0x32/0x80 [ 7.424883] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.424883] [<ffffffff8049f596>] ? mutex_lock_nested+0xa6/0x250 [ 7.424883] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.424883] [<ffffffff803635c4>] ? idr_pre_get+0x44/0x90 [ 7.424883] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.424883] [<ffffffffa008fe43>] ? thermal_cooling_device_register+0x83/0x250 [thermal_sys] [ 7.424883] [<ffffffffa019b2a3>] ? acpi_processor_start+0x64b/0x774 [processor] [ 7.424883] [<ffffffff8031a94b>] ? __sysfs_add_one+0x6b/0xa0 [ 7.424883] [<ffffffff8031ba3c>] ? sysfs_do_create_link+0xbc/0x150 [ 7.424883] [<ffffffff803a7f5e>] ? acpi_start_single_object+0x2d/0x52 [ 7.424883] [<ffffffff803a9556>] ? acpi_device_probe+0x7e/0x92 [ 7.424883] [<ffffffff803dd3eb>] ? driver_probe_device+0x9b/0x1a0 [ 7.424883] [<ffffffff803dd576>] ? __driver_attach+0x86/0x90 [ 7.424883] [<ffffffff803dd4f0>] ? __driver_attach+0x0/0x90 [ 7.424883] [<ffffffff803dc93d>] ? bus_for_each_dev+0x5d/0x90 [ 7.424883] [<ffffffff803dd22c>] ? driver_attach+0x1c/0x20 [ 7.424883] [<ffffffff803dcf79>] ? bus_add_driver+0x1e9/0x260 [ 7.424883] [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor] [ 7.424883] [<ffffffff803dd74f>] ? driver_register+0x5f/0x140 [ 7.424883] [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor] [ 7.424883] [<ffffffff803a9866>] ? acpi_bus_register_driver+0x3e/0x40 [ 7.424883] [<ffffffffa0222094>] ? acpi_processor_init+0x94/0x107 [processor] [ 7.424883] [<ffffffff80209040>] ? _stext+0x40/0x180 [ 7.424883] [<ffffffff802a8911>] ? __vunmap+0xa1/0x110 [ 7.424883] [<ffffffff802676c2>] ? sys_init_module+0x142/0x1dc0 [ 7.424883] [<ffffffff80367b16>] ? __up_read+0x46/0xb0 [ 7.424883] [<ffffffff8048e570>] ? cpu_down+0x0/0x70 [ 7.424883] [<ffffffff8020c34b>] ? system_call_fastpath+0x16/0x1b [ 7.424883] [ 7.424883] ---[ end trace 8bbd31df1403e48e ]---
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 4 lines · Alan D. Brunelle
Before adding any more debugging, this is the status of my kernel boots:
3 times in a row w/ this same error. (Primary problem is the same, secondary stacks differ of course.)
Ok, so I took a closer look, and the oops really is suggestive..
Quoted 1 line
[ 6.482953] busybox used greatest stack depth: 4840 bytes left
Ok, 4840 bytes left out of 8kB.
Quoted 1 line
[ 6.521876] all_generic_ide used greatest stack depth: 4784 bytes left
.. and this one is 4784 bytes left..
Quoted 3 lines
Begin: Loading essential drivers... ...
[ 6.625509] fuse init (API version 7.9) [ 6.625509] modprobe used greatest stack depth: 1720 bytes left
Uhhuh! The previous "modprobe" uses stack like mad. It could be "fuse_init()" that has done it, but looking at fuse, I seriously doubt it. It doesn't seem to do anything particularly bad.
So something has used over 6kB of stack, and it may well be the module loading code itself.
The next stage is the actual oops itself:
Quoted 2 lines
[ 6.644854] ACPI: SSDT CFFD0D0A, 08C4 (r1 HPQOEM CPU_TM2 1 MSFT 100000E) [ 6.651489] BUG: unable to handle kernel NULL pointer dereference at 0000000000000858
This really looks like
ti->task->blocked_on = waiter;
where "ti->task" is NULL. You probably have almost everything enabled in order to turn "struct task_struct" that big, but judging by your register state it's really an offset off a NULL pointer, not some small integer.
Now, there is no way "ti->task" can _possibly_ be NULL. No way.
Well, except that "ti" is just below the stack, and if you had a stack overflow that overwrote it.
So I seriously do believe that you have run out of stack. If that is true, then it's quite likely that with DEBUG_PAGE_ALLOC you'll actually get a double fault, which in turn is fairly hard to debug (you look at it wrong and it turns into a triple fault which is going to just reboot your machine immediately).
Now, the stack oveflow probably happened a few calls earlier (and just left your thread_info corrupted), but there is more reason to believe you have stack overflow and thread_info corruption later in your output:
Quoted 3 lines
[ 7.024992] modprobe used greatest stack depth: 408 bytes left [ 7.030988] BUG: unable to handle kernel NULL pointer dereference at 0000000000000048 [ 7.031053] IP: [<ffffffff8023f39c>] do_exit+0x28c/0xa10
Here there is only 408 bytes left, which is _way_ too little, but it's also an optimistic measure. What the stack code usage code does is to just see how many zeroes it can find on the stack. If you have a big stack frame somewhere, it's quite possible that it actually used all your stack and then some, but left a bunch of zeroes around.
And the do_exit() oops is simply because once the thread_info is corrupted, all the basic thread data structures are crap, and yes, you're almost guaranteed to oops at that point.
Could you make your kernel image available somewhere, and we can take a look at it? Some versions of gcc are total pigs when it comes to stack usage, and your exact configuration matters too. But yes, module loading is a bad case, for me "sys_init_module()" contains
subq $392, %rsp #,
which is probably mostly because of the insane inlining gcc does (ie it will likely have inlined every single function in that file that is only called once, and then it will make all local variables of all those functions alive over the whole function and allocate stack-space for them ALL AT THE SAME TIME).
Gcc sometimes drives me mad. It's inlining decisions are almost always pure and utter sh*t. But clearly something changed for you to start triggering this, and I think that also explains why you bisected things to the merge commit rather than to any individual change - because it was probably not any individual change that pushed it over the limit, but two different changes that made for bigger stack pressure, and _together_ they pushed you over the limit.
So it also explains why the merge you found had no possible merge errors on a source level - there were no actual clashes anywhere. Just a slow growth of stack that combined to something that overflowed.
And yes, I bet the change by Arjan to use do_one_initcall() was _part_ of it. It adds roughly 112 bytes of stack pressure to that module loading path, because of the 64-byte array and the extra function call (8 bytes for return address) with at least 5 quad-words saved (40 bytes) for register spills.
But there were probably other things happening too that made things worse.
So if there is some place where you can upload your 'vmlinux' binary, it would be good.
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 6 lines · Linus Torvalds
Could you make your kernel image available somewhere, and we can take a look at it? Some versions of gcc are total pigs when it comes to stack usage, and your exact configuration matters too. But yes, module loading is a bad case, for me "
sys_init_module()" contains
Quoted 7 lines
subq $392, %rsp #,which is probably mostly because of the insane inlining gcc does (ie it will likely have inlined every single function in that file that is only called once, and then it will make all local variables of all those functions alive over the whole function and allocate stack-space for them ALL AT THE SAME TIME).
I bet this one-liner will probably make your kernel work. It's not a full solution, but it will make the module-loading path lose _all_ of the above stack slots by just not inlining "load_module()" - the stack slots will still be used when the module is _loaded_, but by the time we actually callt he ->init function they will have been released since it's not all in the same crazy function any more.
I _seriously_ believe that we were better off back when gcc only inlined what we told it to inline, and never inlined on its own. The gcc inlining logic is pure and utter sh*t in an environment like the kernel where stack space is a valuable resource.
Anyway, Alan, even if this solves your particular problem, I'd still like to see your kernel image, so that I can hunt for other problems like this..
Linus
--- kernel/module.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/module.c +1 -1
@@ -1799,7 +1799,7 @@ static void *module_alloc_update_bounds(unsigned long size)/* Allocate and load the module: note that size of section 0 is always zero, and we rely on this for optional sections. */ -static struct module *load_module (void __user *umod, +static noinline struct module *load_module(void __user *umod,unsigned long len,const char __user *uargs) {
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 7 lines · Linus Torvalds
Quoted 5 lines · Linus Torvalds
Could you make your kernel image available somewhere, and we can take a look at it? Some versions of gcc are total pigs when it comes to stack usage, and your exact configuration matters too. But yes, module loading is a bad case, for me "
sys_init_module()" contains
Quoted 7 lines
Quoted 7 lines
subq $392, %rsp #,which is probably mostly because of the insane inlining gcc does (ie it will likely have inlined every single function in that file that is only called once, and then it will make all local variables of all those functions alive over the whole function and allocate stack-space for them ALL AT THE SAME TIME).
Mine has:
Dump of assembler code for function sys_init_module:
0xffffffff802688c0 <sys_init_module+0>: push %rbp 0xffffffff802688c1 <sys_init_module+1>: mov %rsp,%rbp 0xffffffff802688c4 <sys_init_module+4>: sub $0x1c0,%rsp
0xffffffff802688cb <sys_init_module+11>: mov %r12,-0x20(%rbp)0xffffffff802688cf <sys_init_module+15>: mov %rdi,%r12
so 448 bytes.
The kernel is up at: http://free.linux.hp.com/~adb/bug.11342/vmlinux (if you would let me know when you are through with it so I can free up some space there I'd appreciate it...)
By doing the patch you provided, sys_init_module now looks like:
Dump of assembler code for function sys_init_module:
0xffffffff8026aa20 <sys_init_module+0>: push %rbp 0xffffffff8026aa21 <sys_init_module+1>: mov %rsp,%rbp 0xffffffff8026aa24 <sys_init_module+4>: sub $0x20,%rsp 0xffffffff8026aa28 <sys_init_module+8>: mov %r14,0x18(%rsp)
0xffffffff8026aa2d <sys_init_module+13>: mov %rdi,%r14
So only 32 bytes. (But of course, load_module() exists, and now has 0x1d0 (464) bytes...)
With the patch you provide, I /was/ able to repeatedly boot OK (latest tree, and I also ran the patch against the 26.27.rc3-based kernel I was having problems with initially, and that booted OK as well).
Alan
Quoted 41 lines
I bet this one-liner will probably make your kernel work. It's not a full solution, but it will make the module-loading path lose _all_ of the above stack slots by just not inlining "
load_module()" - the stack slots will still be used when the module is _loaded_, but by the time we actually callt he ->init function they will have been released since it's not all in the same crazy function any more.I _seriously_ believe that we were better off back when gcc only inlined what we told it to inline, and never inlined on its own. The gcc inlining logic is pure and utter sh*t in an environment like the kernel where stack space is a valuable resource.
Anyway, Alan, even if this solves your particular problem, I'd still like to see your kernel image, so that I can hunt for other problems like this..
Linus
--- kernel/module.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)kernel/module.c +1
-2@@ -1799,7 +1799,7 @@ static void *module_alloc_update_bounds(unsigned long size)/* Allocate and load the module: note that size of section 0 is always zero, and we rely on this for optional sections. */ -static struct module *load_module (void __user *umod, +static noinline struct module *load_module(void __user *umod,unsigned long len,const char __user *uargs) { --To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 7 lines · Alan D. Brunelle
Mine has:
Dump of assembler code for function
sys_init_module:0xffffffff802688c4<sys_init_module+4>: sub $0x1c0,%rspso 448 bytes.
Yeah, your build seems to have consistently bigger stack usage, and that may be due to some config option, but most likely it's a compiler version issue.
But I think part of the reason is that you have frame pointers enabled:
that makes the stack frames bigger not only because of the frame pointer save/restore, but also because you have more register pressure and thus spills.
Quoted 3 lines
The kernel is up at: http://free.linux.hp.com/~adb/bug.11342/vmlinux (if you would let me know when you are through with it so I can free up some space there I'd appreciate it...)
I'm downloading it now, I'll probably be done by the time you get this email.
[ Update. Done. You can remove it ]
Quoted 7 lines
By doing the patch you provided,
sys_init_modulenow looks like:Dump of assembler code for function
sys_init_module:0xffffffff8026aa24<sys_init_module+4>: sub $0x20,%rspSo only 32 bytes. (But of course,
load_module()exists, and now has0x1d0(464) bytes...)
Right - the stack usage didn't go away, but the _lifetimes_ changed.
So now load_module() will still use almost 500 bytes of stack, and it will call other routines that use stack too, but the lifetime of that stack usage is no longer over the whole module loading and initialization part, it's purely over just the loading thing.
And since the deep callchain came much later (in the actual ->init routines), by the time we do that, we no longer now have the load_module stack usage active any more.
Quoted 3 lines
With the patch you provide, I /was/ able to repeatedly boot OK (latest tree, and I also ran the patch against the 26.27.rc3-based kernel I was having problems with initially, and that booted OK as well).
I had actually already committed it, because it was correct regardless (and gcc really is a total ass for doing that inlining to begin with), but it's good to have verification that the behaviour you saw was literally about this thing.
I'll look at your vmlinux binary to see what else sucks from a stack depth standpoint, but one of the problems in this whole thing is that the stack usage is obviously both a static thing (with some functions using _way_ too much stack!) _and_ a dynamic thing (with the total stack use being not about any individual function, but the whole chain).
My patch obviously doesn't change the static stack usage, it just moves it around a bit so that it's no longer on that same deep path, so the dynamic stack usage is much less.
But I'll look at your vmlinux, see what stands out.
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 10 lines · Linus Torvalds
Quoted 6 lines · Alan D. Brunelle
Mine has:
Dump of assembler code for function
sys_init_module:0xffffffff802688c4<sys_init_module+4>: sub $0x1c0,%rspso 448 bytes.
Yeah, your build seems to have consistently bigger stack usage, and that may be due to some config option, but most likely it's a compiler version issue.
I wonder if we ought to have a light version of "make checkstack" always run, but in such a way that we make a file with "limits" on the stack usage for key functions (and we can grow this list over time when we learn about critical ones).. and either warn very loudly or even fail the build if we're way over what could work.
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Linus TorvaldsQuoted 2 lines · Linus Torvalds
But I'll look at your vmlinux, see what stands out.
Oops. I already see the problem.
Your .config has soem _huge_ CPU count, doesn't it?
checkstack.pl shows these things as the top problems:
0xffffffff80266234 smp_call_function_mask [vmlinux]: 2736 0xffffffff80234747 __build_sched_domains [vmlinux]: 2232 0xffffffff8023523f __build_sched_domains [vmlinux]: 2232 0xffffffff8021e884 setup_IO_APIC_irq [vmlinux]: 1616 0xffffffff8021ee24 arch_setup_ht_irq [vmlinux]: 1600 0xffffffff8021f144 arch_setup_msi_irq [vmlinux]: 1600 0xffffffff8021e3b0 __assign_irq_vector [vmlinux]: 1592 0xffffffff8021e626 __assign_irq_vector [vmlinux]: 1592 0xffffffff8023257e move_task_off_dead_cpu [vmlinux]: 1592 0xffffffff802326e8 move_task_off_dead_cpu [vmlinux]: 1592 0xffffffff8025dbc5 tick_handle_oneshot_broadcast [vmlinux]:1544 0xffffffff8025dcb4 tick_handle_oneshot_broadcast [vmlinux]:1544 0xffffffff803f3dc4 store_scaling_governor [vmlinux]: 1376 0xffffffff80279ef4 cpuset_write_resmask [vmlinux]: 1360 0xffffffff803f465d cpufreq_add_dev [vmlinux]: 1352 0xffffffff803f495b cpufreq_add_dev [vmlinux]: 1352 0xffffffff803f3fc4 store_scaling_max_freq [vmlinux]: 1328 0xffffffff803f4064 store_scaling_min_freq [vmlinux]: 1328 0xffffffff803f44c4 cpufreq_update_policy [vmlinux]: 1328 ..
and sys_init_module is actually way way down the list. I bet the only reason it showed up at all was because dynamically it was such a deep callchain, and part of that callchain probably called some of those really nasty things.
Anyway, the reason smp_call_function_mask and friends have such _huge_ stack usages for you is that they contain a 'cpumask_t' on the stack.
For example, for me, usign a sane NR_CPU, the size of the stack frame for smp_call_function_mask is under 200 bytes. For you, it's 2736 bytes.
How about you make CONFIG_NR_CPU's something _sane_? Like 16? Or do you really have four thousand CPU's in that system?
Oh, I guess you have the MAXSMP config enabled? I really think that was a bit too aggressive.
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 9 lines · Linus Torvalds
checkstack.pl shows these things as the top problems:
0xffffffff80266234 smp_call_function_mask [vmlinux]: 2736 0xffffffff80234747 __build_sched_domains [vmlinux]: 2232 0xffffffff8023523f __build_sched_domains [vmlinux]: 2232Anyway, the reason
smp_call_function_maskand friends have such _huge_ stack usages for you is that they contain a 'cpumask_t' on the stack.
In fact, they contain multiple CPU-masks, each 4k-bits - 512 bytes - in size. And they tend to call each other.
Quite frankly, I don't think we were really ready for 4k CPU's. I'm going to commit this patch to make sure others don't do that many CPU's by mistake. It marks MAXCPU's as being 'broken' so you cannot select it, and also limits the number of CPU's that you _can_ select to "just" 512.
Right now, 4k cpu's is known broken because of the stack usage. I'm not willing to debug more of these kinds of stack smashers, they're really nasty to work with. I wonder how many other random failures these have been involved with?
This patch also makes the ifdef mess in Kconfig much cleaner and avoids duplicate definitions by just conditionally suppressing the question and giving higher defaults.
We can enable MAXSMP and raise the CPU limits some time in the future. But that future is not going to be before 2.6.27 - the code simply isn't ready for it.
The reason I picked 512 CPU's as the limit is that we _used_ to limit things to 255. So it's higher than it used to be, but low enough to still feel safe. Considering that a 4k-bit CPU mask (512 bytes) _almost_ worked, the 512-bit (64 bytes) masks are almost certainly fine.
Still, sane people should limit their NR_CPUS to 8 or 16 or something like that. Very very few people really need the pain of big NR_CPUS. Not even "just" 512 CPU's.
Travis, Ingo and Thomas cc'd, since they were involved in the original commit (1184dc2ffe2c8fb9afb766d870850f2c3165ef25) that raised the limit.
Linus
--- arch/x86/Kconfig | 30 ++++++++---------------------- 1 files changed, 8 insertions(+), 22 deletions(-)
arch/x86/Kconfig +8 -22
@@ -577,35 +577,29 @@ config SWIOTLB config IOMMU_HELPER def_bool (CALGARY_IOMMU || GART_IOMMU || SWIOTLB || AMD_IOMMU) + config MAXSMP bool "Configure Maximum number of SMP Processors and NUMA Nodes" - depends on X86_64 && SMP + depends on X86_64 && SMP && BROKEN default n help Configure maximum number of CPUS and NUMA Nodes for this architecture. If unsure, say N. -if MAXSMP -config NR_CPUS - int - default "4096" -endif - -if !MAXSMP config NR_CPUS - int "Maximum number of CPUs (2-4096)" - range 2 4096 + int "Maximum number of CPUs (2-512)" if !MAXSMP + range 2 512 depends on SMP + default "4096" if MAXSMP default "32" if X86_NUMAQ || X86_SUMMIT || X86_BIGSMP || X86_ES7000 default "8" help This allows you to specify the maximum number of CPUs which this - kernel will support. The maximum supported value is 4096 and the + kernel will support. The maximum supported value is 512 and the minimum value which makes sense is 2. This is purely to save memory - each supported CPU adds approximately eight kilobytes to the kernel image. -endif config SCHED_SMT bool "SMT (Hyperthreading) scheduler support"
@@ -996,17 +990,10 @@ config NUMA_EMU into virtual nodes when booted with "numa=fake=N", where N is the number of nodes. This is only useful for debugging. -if MAXSMP - config NODES_SHIFT - int - default "9" -endif - -if !MAXSMP -config NODES_SHIFT - int "Maximum NUMA Nodes (as a power of 2)" + int "Maximum NUMA Nodes (as a power of 2)" if !MAXSMP range 1 9 if X86_64 + default "9" if MAXSMP default "6" if X86_64 default "4" if X86_NUMAQ default "3"
@@ -1014,7 +1001,6 @@ config NODES_SHIFT help Specify the maximum number of NUMA Nodes available on the target system. Increases memory reserved to accomodate various tables. -endif config HAVE_ARCH_BOOTMEM_NODE def_bool y
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 11 lines · Linus Torvalds
Quoted 9 lines · Linus Torvalds
checkstack.pl shows these things as the top problems:
0xffffffff80266234 smp_call_function_mask [vmlinux]: 2736 0xffffffff80234747 __build_sched_domains [vmlinux]: 2232 0xffffffff8023523f __build_sched_domains [vmlinux]: 2232Anyway, the reason
smp_call_function_maskand friends have such _huge_ stack usages for you is that they contain a 'cpumask_t' on the stack.
Quoted 8 lines
In fact, they contain multiple CPU-masks, each 4k-bits - 512 bytes - in size. And they tend to call each other.
Quite frankly, I don't think we were really ready for 4k CPU's. I'm going to commit this patch to make sure others don't do that many CPU's by mistake. It marks MAXCPU's as being 'broken' so you cannot select it, and also limits the number of CPU's that you _can_ select to "just" 512.
yeah, that's OK i guess - distros can still enable 4K support if they wish to. Someone interested in improving the stack footprint situation should dust off the max-stack-footprint tracer so that we can catch these things in a more structured way.
And i guess the next generation of 4K CPUs support should just get away from cpumask_t-on-kernel-stack model altogether, as the current model is not maintainable. We tried the on-kernel-stack variant, and it really does not work reliably. We can fix this in v2.6.28.
Ingo
David Miller
maintainerRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
From: Ingo Molnar <[email protected]>
Date: Tue, 26 Aug 2008 09:22:20 +0200
Quoted 4 lines
And i guess the next generation of 4K CPUs support should just get away from cpumask_t-on-kernel-stack model altogether, as the current model is not maintainable. We tried the on-kernel-stack variant, and it really does not work reliably. We can fix this in v2.6.28.
I recenetly did some work on sparc64 to use cpumask pointers as much as possible.
The only case that didn't work was due to a limitation in arch interfaces for the new generic smp_call_function() code. It passes a cpumask_t instead of a pointer to one via arch_send_call_function_ipi().
But other than that, the whole sparc64 SMP stuff uses cpumask_t pointers only.
What it comes down to is that you have to do the "self cpu" and other tests in the cross-call dispatch routines themselves, instead of at the top-level working on cpumask_t objects.
Otherwise you have to modify cpumask_t objects and thus pluck them onto the stack where they take up silly amounts of space.
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 11 lines · David Miller
From: Ingo Molnar <[email protected]>
Date: Tue, 26 Aug 2008 09:22:20 +0200Quoted 4 lines
And i guess the next generation of 4K CPUs support should just get away from cpumask_t-on-kernel-stack model altogether, as the current model is not maintainable. We tried the on-kernel-stack variant, and it really does not work reliably. We can fix this in v2.6.28.
I recenetly did some work on sparc64 to use cpumask pointers as much as possible.
Quoted 7 lines
The only case that didn't work was due to a limitation in arch interfaces for the new generic
smp_call_function()code. It passes acpumask_tinstead of a pointer to one viaarch_send_call_function_ipi().But other than that, the whole sparc64 SMP stuff uses
cpumask_tpointers only.
nice!
Quoted 6 lines
What it comes down to is that you have to do the "self cpu" and other tests in the cross-call dispatch routines themselves, instead of at the top-level working on
cpumask_tobjects.Otherwise you have to modify
cpumask_tobjects and thus pluck them onto the stack where they take up silly amounts of space.
What we did was this: we added MAXSMP which just revs up all the SMP tunables to the maximum, so that we can see any problems early in testing.
And we triggered problems, and we fixed a couple of regressions all around stack footprint. But we didnt catch all of them - some were gcc version dependent and configuration dependent. So i think it's safe to say that the whole concept of allowing such a large cpumask_t to be on the stack is fragile.
Hence, i think the best way forward is to change the whole cpumask_t concept and disallow explicit masks altogether. It's so easy to smack a cpumask_t variable on the stack and nothing really warns about it, and any function can become part of a nested call sequence.
So i think the dynamics of it has to be changed: we need a get/put API and we need to make on-stack cpumask illegal on the build level (in generic code at least). This has been Rusty's main argument early on i think, and i now concur.
Ingo
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 14 lines · Ingo Molnar
Quoted 11 lines · David Miller
From: Ingo Molnar <[email protected]>
Date: Tue, 26 Aug 2008 09:22:20 +0200Quoted 4 lines
And i guess the next generation of 4K CPUs support should just get away from cpumask_t-on-kernel-stack model altogether, as the current model is not maintainable. We tried the on-kernel-stack variant, and it really does not work reliably. We can fix this in v2.6.28.
I recenetly did some work on sparc64 to use cpumask pointers as much as possible.
Quoted 7 lines
Quoted 7 lines
The only case that didn't work was due to a limitation in arch interfaces for the new generic
smp_call_function()code. It passes acpumask_tinstead of a pointer to one viaarch_send_call_function_ipi().But other than that, the whole sparc64 SMP stuff uses
cpumask_tpointers only.
wonder if could use "unsigned long *" directly.
so could dyn_array directly like
int cpumask_size;
unsigned long *online_cpu_map; DEFINE_DYN_ARRAY(online_cpu_map, sizeof(unsigned long), cpumask_size, PAGE_SIZE, NULL);
and after nr_cpu_ids is assigned, havecpumask_size = (nr_cpu_ids + sizeof(unsigned long) - 1)/sizeof(unsigned long);
then we could NR_CPUS=4096 kernel to small system. ...
YH
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 2 lines · Yinghai Lu
wonder if could use "unsigned long *" directly.
I would actually suggest something like this:
- we continue to have a magic "cpumask_t".
- we do different cases for big and small NR_CPUS:
#if NR_CPUS <= BITS_PER_LONG
/*
* Make it an array - that way passing it as an argument will
* always pass it as a pointer!
*/
typedef unsigned long cpumask_t[1];
static inline void create_cpumask(cpumask_t *p)
{
*p = 0;
}
static inline void free_cpumask(cpumask_t *p)
{
}
#else
typedef unsigned long *cpumask_t;
static inline void create_cpumask(cpumask_t *p)
{
*p = kcalloc(..);
}
static inline void free_cpumask(cpumask_t *p)
{
kfree(*p);
}
#endifand now after you do this, you can just do something like
cpumask_t mycpu; create_cpumask(&mycpu); .. free_cpumask(&mycpu);
and in between, you can use 'cpumask' as a pointer, because even when it is an array directly allocated on the stack, the array can always degenerate into a pointer by C type rules!
And for the small-NR_CPUS case there is zero overhead.
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 52 lines · Linus Torvalds
Quoted 2 lines · Yinghai Lu
wonder if could use "unsigned long *" directly.
I would actually suggest something like this:
- we continue to have a magic "
cpumask_t".- we do different cases for big and small
NR_CPUS:#if NR_CPUS <= BITS_PER_LONG /* * Make it an array - that way passing it as an argument will * always pass it as a pointer! */ typedef unsigned long cpumask_t[1]; static inline void create_cpumask(cpumask_t *p) { *p = 0; } static inline void free_cpumask(cpumask_t *p) { } #else typedef unsigned long *cpumask_t; static inline void create_cpumask(cpumask_t *p) { *p = kcalloc(..); } static inline void free_cpumask(cpumask_t *p) { kfree(*p); } #endifand now after you do this, you can just do something like
cpumask_t mycpu; create_cpumask(&mycpu); .. free_cpumask(&mycpu);
Quoted 4 lines
and in between, you can use 'cpumask' as a pointer, because even when it is an array directly allocated on the stack, the array can always degenerate into a pointer by C type rules!
that is good for local variables.
for global variables, need to allocate them in some point. may need one int cpumask_size;
cpumask_t online_cpu_map;DEFINE_DYN_ARRAY(online_cpu_map, sizeof(unsigned long), cpumask_size, PAGE_SIZE, NULL);
or something like that.
YH
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Linus TorvaldsQuoted 49 lines · On Wednesday 27 August 2008 02:51:46 Linus Torvalds wrote:
Quoted 1 line · Yinghai Lu
wonder if could use "unsigned long *" directly.
I would actually suggest something like this:
- we continue to have a magic "
cpumask_t".- we do different cases for big and small
NR_CPUS:#if NR_CPUS <= BITS_PER_LONG /* * Make it an array - that way passing it as an argument will * always pass it as a pointer! */ typedef unsigned long cpumask_t[1]; static inline void create_cpumask(cpumask_t *p) { *p = 0; } static inline void free_cpumask(cpumask_t *p) { } #else typedef unsigned long *cpumask_t; static inline void create_cpumask(cpumask_t *p) { *p = kcalloc(..); } static inline void free_cpumask(cpumask_t *p) { kfree(*p); } #endifand now after you do this, you can just do something like
cpumask_t mycpu; create_cpumask(&mycpu); .. free_cpumask(&mycpu);
Quoted 3 lines
and in between, you can use 'cpumask' as a pointer, because even when it is an array directly allocated on the stack, the array can always degenerate into a pointer by C type rules!
Hi Linus,
This turns out to be awful in practice, mainly due to const. Consider: #ifdef CONFIG_CPUMASK_OFFSTACK typedef unsigned long *cpumask_t; #else typedef unsigned long cpumask_t[1]; #endif cpumask_t returns_cpumask(void);
That's obviously illegal if cpumask_t is an array. So we need a typedef which says "really always a pointer".
typedef unsigned long *cpumask_return_t; cpumask_return_t returns_cpumask(void);
But we usually want it to return a const ptr, and this doesn't work:
const cpumask_return_t returns_cpumask(void); foo.c:12: warning: type qualifiers ignored on function return type
So now we need:
typedef const unsigned long *cpumask_const_return_t; cpumask_const_return_t returns_cpumask(void);
OK, now consider a function which wants to take a const cpu bitmap:
void cpus_copy(cpumask_t dst, const cpumask_t src); ... cpus_copy(cpus, returns_cpumask()); foo.c:34: warning: passing argument 2 of ‘cpus_copy’ discards qualifiers from
pointer target type
Oops, that didn't work with the pointer version. So we need another typedef:
#ifdef CONFIG_CPUMASK_OFFSTACK typedef const unsigned long *cpumask_const_t; #else typedef const unsigned long cpumask_const_t[1]; #endif void cpus_copy(cpumask_t dst, cpumask_const_t src);
We end up with this:
#ifdef CONFIG_CPUMASK_OFFSTACK typedef unsigned long *cpumask_t; typedef const unsigned long *cpumask_const_t; #else typedef unsigned long cpumask_t[1]; typedef const unsigned long cpumask_const_t[1]; #endif typedef unsigned long *cpumask_return_t; typedef const unsigned long *cpumask_const_return_t; typedef unsigned long cpumask_data_t[1];
I can't see a neater way down this path, and I don't want to lose const.
I can see three alternatives:
1) An ONSTACK_CPUMASK(name) macro which declares "struct cpumask name[1]" or "struct cpumask *name". Same idea as yours, without the typedef.
2) Use a normal struct for cpumask, make everyone use pointers, but have an struct cpumask *alloc_stack_cpumask() which uses alloca() for small NR_CPUS.
3) Same, but just use kmalloc everywhere. Optimize important cases by hand.
Anyone see a better way?
Rusty.
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 11 lines · Rusty Russell
I can't see a neater way down this path, and I don't want to lose const.
I can see three alternatives:
1) AnONSTACK_CPUMASK(name)macro which declares "struct cpumask name[1]" or "struct cpumask *name". Same idea as yours, without the typedef.
2) Use a normal struct for cpumask, make everyone use pointers, but have an struct cpumask*alloc_stack_cpumask()which usesalloca()for smallNR_CPUS.
3) Same, but just use kmalloc everywhere. Optimize important cases by hand.
Quoted 1 line
Anyone see a better way?
since most of the important cpumasks in high-perf codepaths are already pre-constructed and embedded in some existing object (say task_struct), i think a variant of #3 is the best approach:
- get rid of cpumask_t and use 'struct cpumask' everywhere.
- do not expose normal kernel code to struct cpumask's definition, only declare the type via 'struct cpumask;' in sched.h - a'la kmem_cache_t.
- even hide the structure from sched.h - use an extra indirection for struct cpumask *cpus_allowed in task_struct and be done with it.
- have normal cpumask object alloc/free codepaths.
- optimize any remaining important cases by hand, if needed. (the scheduler mostly)
(wrt. #2, alloca() is a nightmare i think.)
Ingo
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Rusty RussellQuoted 10 lines · Rusty Russell
This turns out to be awful in practice, mainly due to const. Consider:
#ifdef CONFIG_CPUMASK_OFFSTACK
typedef unsigned long *cpumask_t;
#else
typedef unsigned long cpumask_t[1];
#endif
cpumask_t returns_cpumask(void);No. That's already broken. You cannot return a cpumask_t, regardless of interface. We must not do it regardless of how we pass those things around, since it generates _yet_ another temporary on the stack for the return slot for any kind of structure.
So all cpumask functions should always return pointers and/or take pointers to be filled in. That's true *regardless* of how we actually are to then allocate them.
So forget returning cpumasks. It's irrelevant.
What _is_ relevant is how we allocate them when we need temporary CPU masks. And _that_ is where my suggestion comes in. For small NR_CPUS, we really do want to allocate them on the stack, because calling kmalloc for a 4- or 8-byte allocation is just _stupid_.
So all your arguments are invalid, because you're looking at the wrong thing. The thing that I was talking about is converting current code that has
random_function(..)
{
cpumask_t mask;
.. do something with mask ...}
which has to be converted some way. And I think it needs to be converted in a way that does *not* force us to call kmalloc() for idiotically small values.
Linus
Subject: [RFC 1/1] cpumask: Provide new cpumask API
Quoted 39 lines · Linus Torvalds
Quoted 9 lines · Rusty Russell
This turns out to be awful in practice, mainly due to const. Consider: #ifdef CONFIG_CPUMASK_OFFSTACK typedef unsigned long *cpumask_t; #else typedef unsigned long cpumask_t[1]; #endif cpumask_t returns_cpumask(void);No. That's already broken. You cannot return a
cpumask_t, regardless of interface. We must not do it regardless of how we pass those things around, since it generates _yet_ another temporary on the stack for the return slot for any kind of structure.So all cpumask functions should always return pointers and/or take pointers to be filled in. That's true *regardless* of how we actually are to then allocate them.
So forget returning cpumasks. It's irrelevant.
What _is_ relevant is how we allocate them when we need temporary CPU masks. And _that_ is where my suggestion comes in. For small
NR_CPUS, we really do want to allocate them on the stack, because calling kmalloc for a 4- or 8-byte allocation is just _stupid_.So all your arguments are invalid, because you're looking at the wrong thing. The thing that I was talking about is converting current code that has
random_function(..) { cpumask_t mask; .. do something with mask ... }
Quoted 5 lines
which has to be converted some way. And I think it needs to be converted in a way that does *not* force us to call
kmalloc()for idiotically small values.Linus
Subject: [RFC 1/1] cpumask: Provide new cpumask API
Provide new cpumask interface API. The relevant change is basically cpumask becomes an opaque object. I believe this results in the minimum amount of editing while still allowing the inline cpumask functions, and the ability to declare static cpumask objects.
/* raw declaration */
struct __cpumask_data_s { DECLARE_BITMAP(bits, NR_CPUS); };
/* cpumask_map_t used for declaring static cpumask maps */
typedef struct __cpumask_data_s cpumask_map_t[1];
/* cpumask_t used for function args and return pointers */
typedef struct __cpumask_data_s *cpumask_t;
/* cpumask_var_t used for local variable */
typedef struct __cpumask_data_s cpumask_var_t[1]; /* SMALL NR_CPUS */
typedef struct __cpumask_data_s *cpumask_var_t; /* LARGE NR_CPUS */
/* replaces cpumask_t dst = (cpumask_t)src */
void cpus_copy(cpumask_t dst, const cpumask_t src);Remove the '*' indirection in all references to cpumask_t objects. You can change the reference to the cpumask object but not the cpumask object itself without using the functions that operate on cpumask objects (f.e. the cpu_* operators). Functions can return a cpumask_t (which is a pointer to the cpumask object) and only be passed a cpumask_t.
All uses of cpumask_t on the stack are changed to be cpumask_var_t. Allocation of local cpumask objects will follow...
All cpumask operators now operate using nr_cpu_ids instead of NR_CPUS. All variants of the cpumask operators which used nr_cpu_ids instead of NR_CPUS are deleted.
All variants of functions which use the (old cpumask_t *) pointer are deleted (f.e. set_cpus_allowed_ptr()).
Based on code from Rusty Russell <rusty-8n+1lVoiYb80n/[email protected]> (THANKS!!)
--- include/linux/cpumask.h | 340 ++++++++++++++++++++++++------------------------ 1 file changed, 174 insertions(+), 166 deletions(-)
include/linux/cpumask.h +174 -166
@@ -3,7 +3,8 @@/* * Cpumasks provide a bitmap suitable for representing the -* set of CPU's in a system, one bit position per CPU number. +* set of CPU's in a system, one bit position per CPU number up to +* nr_cpu_ids (<= NR_CPUS). * * See detailed comments in the file linux/bitmap.h describing the * data type on which these cpumasks are based.
@@ -18,18 +19,6 @@* For details of cpus_fold(), see bitmap_fold in lib/bitmap.c. * * . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -* Note: The alternate operations with the suffix "_nr" are used -* to limit the range of the loop to nr_cpu_ids instead of -* NR_CPUS when NR_CPUS > 64 for performance reasons. -* If NR_CPUS is <= 64 then most assembler bitmask -* operators execute faster with a constant range, so -* the operator will continue to use NR_CPUS. -* -* Another consideration is that nr_cpu_ids is initialized -* to NR_CPUS and isn't lowered until the possible cpus are -* discovered (including any disabled cpus). So early uses -* will span the entire range of NR_CPUS. -* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . * * The available cpumask operations are: *
@@ -37,6 +26,7 @@* void cpu_clear(cpu, mask) turn off bit 'cpu' in mask * void cpus_setall(mask) set all bits * void cpus_clear(mask) clear all bits +* void cpus_copy(dst, src) copies cpumask bits from src to dst * int cpu_isset(cpu, mask) true iff bit 'cpu' set in mask * int cpu_test_and_set(cpu, mask) test and set bit 'cpu' in mask *
@@ -52,17 +42,17 @@* int cpus_empty(mask) Is mask empty (no bits sets)? * int cpus_full(mask) Is mask full (all bits sets)? * int cpus_weight(mask) Hamming weigh - number of set bits -* int cpus_weight_nr(mask) Same using nr_cpu_ids instead of NR_CPUS * * void cpus_shift_right(dst, src, n) Shift right * void cpus_shift_left(dst, src, n) Shift left * -* int first_cpu(mask) Number lowest set bit, or NR_CPUS -* int next_cpu(cpu, mask) Next cpu past 'cpu', or NR_CPUS -* int next_cpu_nr(cpu, mask) Next cpu past 'cpu', or nr_cpu_ids +* int first_cpu(mask) Number lowest set bit, or nr_cpu_ids +* int next_cpu(cpu, mask) Next cpu past 'cpu', or nr_cpu_ids +* +* cpumask_t cpumask_of_cpu(cpu) Return pointer to cpumask with bit +* 'cpu' set * -* cpumask_t cpumask_of_cpu(cpu) Return cpumask with bit 'cpu' set -* (can be used as an lvalue) +* cpu_mask_all cpumask_map_t of all bits set * CPU_MASK_ALL Initializer - all bits set * CPU_MASK_NONE Initializer - no bits set * unsigned long *cpus_addr(mask) Array of unsigned long's in mask
@@ -76,8 +66,7 @@* void cpus_onto(dst, orig, relmap) *dst = orig relative to relmap * void cpus_fold(dst, orig, sz) dst bits = orig bits mod sz * -* for_each_cpu_mask(cpu, mask) for-loop cpu over mask using NR_CPUS -* for_each_cpu_mask_nr(cpu, mask) for-loop cpu over mask using nr_cpu_ids +* for_each_cpu_mask(cpu, mask) for-loop cpu over mask * * int num_online_cpus() Number of online CPUs * int num_possible_cpus() Number of all possible CPUs
@@ -107,129 +96,209 @@#include <linux/threads.h> #include <linux/bitmap.h> -typedef struct { DECLARE_BITMAP(bits, NR_CPUS); } cpumask_t; -extern cpumask_t _unused_cpumask_arg_; +/* raw declaration */ +struct __cpumask_data_s { DECLARE_BITMAP(bits, NR_CPUS); }; + +/* cpumask_map_t used for declaring static cpumask maps */ +typedef struct __cpumask_data_s cpumask_map_t [1 ]; + +/* cpumask_t used for function args and return pointers */ +typedef struct __cpumask_data_s *cpumask_t ; + +/* cpumask_var_t used for local variable, definition follows */ + +#if NR_CPUS ==1 + +/* cpumask_var_t used for local variable */ +typedef struct __cpumask_data_s cpumask_var_t [1 ]; + +#define nr_cpu_ids 1 +#define first_cpu (src) ({ (void)(src); 0; }) +#define next_cpu (n, src) ({ (void)(src); 1; }) +#define any_online_cpu (mask) 0 +#define for_each_cpu_mask (cpu, mask) \ + for ((cpu) = 0; (cpu) < 1; (cpu)++, (void)mask) + +#define num_online_cpus () 1 +#define num_possible_cpus () 1 +#define num_present_cpus () 1 +#define cpu_online (cpu) ((cpu) == 0) +#define cpu_possible (cpu) ((cpu) == 0) +#define cpu_present (cpu) ((cpu) == 0) +#define cpu_active (cpu) ((cpu) == 0) + +#else /* ... NR_CPUS > 1 */ + +#ifdef CONFIG_CPUMASKS_ONSTACK + +/* Constant is usually more efficient than a variable for small NR_CPUS */ +#define nr_cpu_ids NR_CPUS +typedef struct __cpumask_data_s cpumask_var_t [1 ]; +static inline int cpumask_size (void ) +{ +return sizeof (struct __cpumask_data_s ); +} + +#else + +/* Starts at NR_CPUS until acpi code discovers actual number. */ +extern int nr_cpu_ids; +typedef struct __cpumask_data_s *cpumask_var_t ; +static inline int cpumask_size (void ) +{ +return sizeof BITS_TO_LONGS (nr_cpu_ids) *sizeof (long ); +} + +#endif /* NR_CPUS > BITS_PER_LONG */ + +int __first_cpu (const cpumask_t srcp); +int __next_cpu (int n,const cpumask_t srcp); +int __any_online_cpu (const cpumask_t mask); + +#define first_cpu (src) __first_cpu((src)) +#define next_cpu (n, src) __next_cpu((n), (src)) +#define any_online_cpu (mask) __any_online_cpu((mask)) + +#define for_each_cpu_mask (cpu, mask) \ + for ((cpu) = -1; \ + (cpu) = next_cpu((cpu), (mask)), \ + (cpu) < nr_cpu_ids; ) + +#define num_online_cpus () cpus_weight(cpu_online_map) +#define num_possible_cpus () cpus_weight(cpu_possible_map) +#define num_present_cpus () cpus_weight(cpu_present_map) +#define cpu_online (cpu) cpu_isset((cpu), cpu_online_map) +#define cpu_possible (cpu) cpu_isset((cpu), cpu_possible_map) +#define cpu_present (cpu) cpu_isset((cpu), cpu_present_map) +#define cpu_active (cpu) cpu_isset((cpu), cpu_active_map) +#endif /* NR_CPUS > 1 */ -#define cpu_set (cpu, dst) __cpu_set((cpu), &(dst)) -static inline void __cpu_set (int cpu,volatile cpumask_t *dstp) +#define cpu_set (cpu, dst) __cpu_set((cpu), (dst)) +static inline void __cpu_set (int cpu,volatile cpumask_t dstp) {set_bit (cpu, dstp->bits); } -#define cpu_clear (cpu, dst) __cpu_clear((cpu), &(dst)) -static inline void __cpu_clear (int cpu,volatile cpumask_t *dstp) +#define cpu_clear (cpu, dst) __cpu_clear((cpu), (dst)) +static inline void __cpu_clear (int cpu,volatile cpumask_t dstp) {clear_bit (cpu, dstp->bits); } -#define cpus_setall (dst) __cpus_setall(&(dst), NR_CPUS) -static inline void __cpus_setall (cpumask_t *dstp,int nbits) +#define cpus_setall (dst) __cpus_setall((dst), nr_cpu_ids) +static inline void __cpus_setall (cpumask_t dstp,int nbits) {bitmap_fill (dstp->bits, nbits); } -#define cpus_clear (dst) __cpus_clear(&(dst), NR_CPUS) -static inline void __cpus_clear (cpumask_t *dstp,int nbits) +#define cpus_clear (dst) __cpus_clear((dst), nr_cpu_ids) +static inline void __cpus_clear (cpumask_t dstp,int nbits) {bitmap_zero (dstp->bits, nbits); } +#define cpus_copy (dst, src) __cpus_copy((dst), (src), nr_cpu_ids) +static inline void __cpus_copy (cpumask_t dstp,const cpumask_t srcp,int nbits) +{ +bitmap_copy (dstp->bits, srcp->bits, nbits); +} +/* No static inline type checking - see Subtlety (1) above. */ -#define cpu_isset (cpu, cpumask) test_bit((cpu), (cpumask).bits) +#define cpu_isset (cpu, cpumask) test_bit((cpu), (cpumask)->bits) -#define cpu_test_and_set (cpu, cpumask) __cpu_test_and_set((cpu), &(cpumask)) -static inline int __cpu_test_and_set (int cpu,cpumask_t *addr) +#define cpu_test_and_set (cpu, cpumask) __cpu_test_and_set((cpu), (cpumask)) +static inline int __cpu_test_and_set (int cpu,cpumask_t addr) {return test_and_set_bit (cpu, addr->bits); } -#define cpus_and (dst, src1, src2) __cpus_and(&(dst), &(src1), &(src2), NR_CPUS) -static inline void __cpus_and (cpumask_t *dstp,const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) +#define cpus_and (dst, src1, src2) __cpus_and((dst), (src1), (src2), nr_cpu_ids) +static inline void __cpus_and (cpumask_t dstp,const cpumask_t src1p, +const cpumask_t src2p,int nbits) {bitmap_and (dstp->bits, src1p->bits, src2p->bits, nbits); } -#define cpus_or (dst, src1, src2) __cpus_or(&(dst), &(src1), &(src2), NR_CPUS) -static inline void __cpus_or (cpumask_t *dstp,const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) +#define cpus_or (dst, src1, src2) __cpus_or((dst), (src1), (src2), nr_cpu_ids) +static inline void __cpus_or (cpumask_t dstp,const cpumask_t src1p, +const cpumask_t src2p,int nbits) {bitmap_or (dstp->bits, src1p->bits, src2p->bits, nbits); } -#define cpus_xor (dst, src1, src2) __cpus_xor(&(dst), &(src1), &(src2), NR_CPUS) -static inline void __cpus_xor (cpumask_t *dstp,const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) +#define cpus_xor (dst, src1, src2) __cpus_xor((dst), (src1), (src2), nr_cpu_ids) +static inline void __cpus_xor (cpumask_t dstp,const cpumask_t src1p, +const cpumask_t src2p,int nbits) {bitmap_xor (dstp->bits, src1p->bits, src2p->bits, nbits); }#define cpus_andnot (dst, src1, src2) \ - __cpus_andnot(&(dst), &(src1), &(src2), NR_CPUS) -static inline void __cpus_andnot (cpumask_t *dstp,const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) + __cpus_andnot((dst), (src1), (src2), nr_cpu_ids) +static inline void __cpus_andnot (cpumask_t dstp,const cpumask_t src1p, +const cpumask_t src2p,int nbits) {bitmap_andnot (dstp->bits, src1p->bits, src2p->bits, nbits); } -#define cpus_complement (dst, src) __cpus_complement(&(dst), &(src), NR_CPUS) -static inline void __cpus_complement (cpumask_t *dstp, -const cpumask_t *srcp,int nbits) +#define cpus_complement (dst, src) __cpus_complement((dst), (src), nr_cpu_ids) +static inline void __cpus_complement (cpumask_t dstp, +const cpumask_t srcp,int nbits) {bitmap_complement (dstp->bits, srcp->bits, nbits); } -#define cpus_equal (src1, src2) __cpus_equal(&(src1), &(src2), NR_CPUS) -static inline int __cpus_equal (const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) +#define cpus_equal (src1, src2) __cpus_equal((src1), (src2), nr_cpu_ids) +static inline int __cpus_equal (const cpumask_t src1p, +const cpumask_t src2p,int nbits) {return bitmap_equal (src1p->bits, src2p->bits, nbits); } -#define cpus_intersects (src1, src2) __cpus_intersects(&(src1), &(src2), NR_CPUS) -static inline int __cpus_intersects (const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) +#define cpus_intersects (src1, src2) __cpus_intersects((src1), (src2), nr_cpu_ids) +static inline int __cpus_intersects (const cpumask_t src1p, +const cpumask_t src2p,int nbits) {return bitmap_intersects (src1p->bits, src2p->bits, nbits); } -#define cpus_subset (src1, src2) __cpus_subset(&(src1), &(src2), NR_CPUS) -static inline int __cpus_subset (const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) +#define cpus_subset (src1, src2) __cpus_subset((src1), (src2), nr_cpu_ids) +static inline int __cpus_subset (const cpumask_t src1p, +const cpumask_t src2p,int nbits) {return bitmap_subset (src1p->bits, src2p->bits, nbits); } -#define cpus_empty (src) __cpus_empty(&(src), NR_CPUS) -static inline int __cpus_empty (const cpumask_t *srcp,int nbits) +#define cpus_empty (src) __cpus_empty((src), nr_cpu_ids) +static inline int __cpus_empty (const cpumask_t srcp,int nbits) {return bitmap_empty (srcp->bits, nbits); } -#define cpus_full (cpumask) __cpus_full(&(cpumask), NR_CPUS) -static inline int __cpus_full (const cpumask_t *srcp,int nbits) +#define cpus_full (cpumask) __cpus_full((cpumask), nr_cpu_ids) +static inline int __cpus_full (const cpumask_t srcp,int nbits) {return bitmap_full (srcp->bits, nbits); } -#define cpus_weight (cpumask) __cpus_weight(&(cpumask), NR_CPUS) -static inline int __cpus_weight (const cpumask_t *srcp,int nbits) +#define cpus_weight (cpumask) __cpus_weight((cpumask), nr_cpu_ids) +static inline int __cpus_weight (const cpumask_t srcp,int nbits) {return bitmap_weight (srcp->bits, nbits); }#define cpus_shift_right (dst, src, n) \ - __cpus_shift_right(&(dst), &(src), (n), NR_CPUS) -static inline void __cpus_shift_right (cpumask_t *dstp, -const cpumask_t *srcp,int n,int nbits) + __cpus_shift_right((dst), (src), (n), nr_cpu_ids) +static inline void __cpus_shift_right (cpumask_t dstp, +const cpumask_t srcp,int n,int nbits) {bitmap_shift_right (dstp->bits, srcp->bits, n, nbits); }#define cpus_shift_left (dst, src, n) \ - __cpus_shift_left(&(dst), &(src), (n), NR_CPUS) -static inline void __cpus_shift_left (cpumask_t *dstp, -const cpumask_t *srcp,int n,int nbits) + __cpus_shift_left((dst), (src), (n), nr_cpu_ids) +static inline void __cpus_shift_left (cpumask_t dstp, +const cpumask_t srcp,int n,int nbits) {bitmap_shift_left (dstp->bits, srcp->bits, n, nbits); }
@@ -244,11 +313,11 @@ static inline void __cpus_shift_left(cpuextern const unsigned long cpu_bit_bitmap[BITS_PER_LONG +1 ][BITS_TO_LONGS (NR_CPUS )]; -static inline const cpumask_t *get_cpu_mask (unsigned int cpu) +static inline const cpumask_t get_cpu_mask (unsigned int cpu) {const unsigned long *p = cpu_bit_bitmap[1 + cpu %BITS_PER_LONG ]; p -= cpu /BITS_PER_LONG ; -return (const cpumask_t *)p; +return (const cpumask_t )p; }/*
@@ -256,7 +325,7 @@ static inline const cpumask_t *get_cpu_m* gcc optimizes it out (it's a constant) and there's no huge stack * variable created: */ -#define cpumask_of_cpu (cpu) (*get_cpu_mask(cpu)) +#define cpumask_of_cpu (cpu) (get_cpu_mask(cpu))#define CPU_MASK_LAST_WORD BITMAP_LAST_WORD_MASK(NR_CPUS)
@@ -264,143 +333,100 @@ static inline const cpumask_t *get_cpu_m#if NR_CPUS <=BITS_PER_LONG #define CPU_MASK_ALL \ -(cpumask_t) { { \ +(cpumask_map_t) { { \ [BITS_TO_LONGS(NR_CPUS)-1] = CPU_MASK_LAST_WORD \ } } -#define CPU_MASK_ALL_PTR (&CPU_MASK_ALL) +#define CPU_MASK_ALL_PTR ((cpumask_t)CPU_MASK_ALL)#else #define CPU_MASK_ALL \ -(cpumask_t) { { \ +(cpumask_map_t) { { \ [0 ... BITS_TO_LONGS(NR_CPUS)-2] = ~0UL, \ [BITS_TO_LONGS(NR_CPUS)-1] = CPU_MASK_LAST_WORD \ } }/* cpu_mask_all is in init/main.c */ -extern cpumask_t cpu_mask_all; -#define CPU_MASK_ALL_PTR (&cpu_mask_all) +extern cpumask_map_t cpu_mask_all; +#define CPU_MASK_ALL_PTR (cpu_mask_all)#endif #define CPU_MASK_NONE \ -(cpumask_t) { { \ +(cpumask_map_t) { { \ [0 ... BITS_TO_LONGS(NR_CPUS)-1] = 0UL \ } }#define CPU_MASK_CPU0 \ -(cpumask_t) { { \ +(cpumask_map_t) { { \ [0] = 1UL \ } }#define cpus_addr (src) ((src).bits)#define cpumask_scnprintf (buf, len, src) \ - __cpumask_scnprintf((buf), (len), &(src), NR_CPUS) + __cpumask_scnprintf((buf), (len), (src), nr_cpu_ids)static inline int __cpumask_scnprintf (char *buf,int len, -const cpumask_t *srcp,int nbits) +const cpumask_t srcp,int nbits) {return bitmap_scnprintf (buf, len, srcp->bits, nbits); }#define cpumask_parse_user (ubuf, ulen, dst) \ - __cpumask_parse_user((ubuf), (ulen), &(dst), NR_CPUS) + __cpumask_parse_user((ubuf), (ulen), (dst), nr_cpu_ids)static inline int __cpumask_parse_user (const char __user *buf,int len, -cpumask_t *dstp,int nbits) +cpumask_t dstp,int nbits) {return bitmap_parse_user (buf, len, dstp->bits, nbits); }#define cpulist_scnprintf (buf, len, src) \ - __cpulist_scnprintf((buf), (len), &(src), NR_CPUS) + __cpulist_scnprintf((buf), (len), (src), nr_cpu_ids)static inline int __cpulist_scnprintf (char *buf,int len, -const cpumask_t *srcp,int nbits) +const cpumask_t srcp,int nbits) {return bitmap_scnlistprintf (buf, len, srcp->bits, nbits); } -#define cpulist_parse (buf, dst) __cpulist_parse((buf), &(dst), NR_CPUS) -static inline int __cpulist_parse (const char *buf,cpumask_t *dstp,int nbits) +#define cpulist_parse (buf, dst) __cpulist_parse((buf), (dst), nr_cpu_ids) +static inline int __cpulist_parse (const char *buf,cpumask_t dstp,int nbits) {return bitmap_parselist (buf, dstp->bits, nbits); }#define cpu_remap (oldbit, old, new) \ - __cpu_remap((oldbit), &(old), &(new), NR_CPUS) + __cpu_remap((oldbit), (old), (new), nr_cpu_ids)static inline int __cpu_remap (int oldbit, -const cpumask_t *oldp,const cpumask_t *newp,int nbits) +const cpumask_t oldp,const cpumask_t newp,int nbits) {return bitmap_bitremap (oldbit, oldp->bits, newp->bits, nbits); }#define cpus_remap (dst, src, old, new) \ - __cpus_remap(&(dst), &(src), &(old), &(new), NR_CPUS) -static inline void __cpus_remap (cpumask_t *dstp,const cpumask_t *srcp, -const cpumask_t *oldp,const cpumask_t *newp,int nbits) + __cpus_remap((dst), (src), (old), (new), nr_cpu_ids) +static inline void __cpus_remap (cpumask_t dstp,const cpumask_t srcp, +const cpumask_t oldp,const cpumask_t newp,int nbits) {bitmap_remap (dstp->bits, srcp->bits, oldp->bits, newp->bits, nbits); }#define cpus_onto (dst, orig, relmap) \ - __cpus_onto(&(dst), &(orig), &(relmap), NR_CPUS) -static inline void __cpus_onto (cpumask_t *dstp,const cpumask_t *origp, -const cpumask_t *relmapp,int nbits) + __cpus_onto((dst), (orig), (relmap), nr_cpu_ids) +static inline void __cpus_onto (cpumask_t dstp,const cpumask_t origp, +const cpumask_t relmapp,int nbits) {bitmap_onto (dstp->bits, origp->bits, relmapp->bits, nbits); }#define cpus_fold (dst, orig, sz) \ - __cpus_fold(&(dst), &(orig), sz, NR_CPUS) -static inline void __cpus_fold (cpumask_t *dstp,const cpumask_t *origp, + __cpus_fold((dst), (orig), sz, nr_cpu_ids) +static inline void __cpus_fold (cpumask_t dstp,const cpumask_t origp,int sz,int nbits) {bitmap_fold (dstp->bits, origp->bits, sz, nbits); } -#if NR_CPUS ==1 - -#define nr_cpu_ids 1 -#define first_cpu (src) ({ (void)(src); 0; }) -#define next_cpu (n, src) ({ (void)(src); 1; }) -#define any_online_cpu (mask) 0 -#define for_each_cpu_mask (cpu, mask) \ - for ((cpu) = 0; (cpu) < 1; (cpu)++, (void)mask) - -#else /* NR_CPUS > 1 */ - -extern int nr_cpu_ids; -int __first_cpu (const cpumask_t *srcp); -int __next_cpu (int n,const cpumask_t *srcp); -int __any_online_cpu (const cpumask_t *mask); - -#define first_cpu (src) __first_cpu(&(src)) -#define next_cpu (n, src) __next_cpu((n), &(src)) -#define any_online_cpu (mask) __any_online_cpu(&(mask)) -#define for_each_cpu_mask (cpu, mask) \ - for ((cpu) = -1; \ - (cpu) = next_cpu((cpu), (mask)), \ - (cpu) < NR_CPUS; ) -#endif - -#if NR_CPUS <=64 - -#define next_cpu_nr (n, src) next_cpu(n, src) -#define cpus_weight_nr (cpumask) cpus_weight(cpumask) -#define for_each_cpu_mask_nr (cpu, mask) for_each_cpu_mask(cpu, mask) - -#else /* NR_CPUS > 64 */ - -int __next_cpu_nr (int n,const cpumask_t *srcp); -#define next_cpu_nr (n, src) __next_cpu_nr((n), &(src)) -#define cpus_weight_nr (cpumask) __cpus_weight(&(cpumask), nr_cpu_ids) -#define for_each_cpu_mask_nr (cpu, mask) \ - for ((cpu) = -1; \ - (cpu) = next_cpu_nr((cpu), (mask)), \ - (cpu) < nr_cpu_ids; ) - -#endif /* NR_CPUS > 64 */ -/* * The following particular system cpumasks and operations manage * possible, present, active and online cpus. Each of them is a fixed size
@@ -458,33 +484,15 @@ int __next_cpu_nr(int n, const cpumask_t* main(){ set1(3); set2(5); } */ -extern cpumask_t cpu_possible_map; -extern cpumask_t cpu_online_map; -extern cpumask_t cpu_present_map; -extern cpumask_t cpu_active_map; - -#if NR_CPUS >1 -#define num_online_cpus () cpus_weight_nr(cpu_online_map) -#define num_possible_cpus () cpus_weight_nr(cpu_possible_map) -#define num_present_cpus () cpus_weight_nr(cpu_present_map) -#define cpu_online (cpu) cpu_isset((cpu), cpu_online_map) -#define cpu_possible (cpu) cpu_isset((cpu), cpu_possible_map) -#define cpu_present (cpu) cpu_isset((cpu), cpu_present_map) -#define cpu_active (cpu) cpu_isset((cpu), cpu_active_map) -#else -#define num_online_cpus () 1 -#define num_possible_cpus () 1 -#define num_present_cpus () 1 -#define cpu_online (cpu) ((cpu) == 0) -#define cpu_possible (cpu) ((cpu) == 0) -#define cpu_present (cpu) ((cpu) == 0) -#define cpu_active (cpu) ((cpu) == 0) -#endif +extern cpumask_map_t cpu_possible_map; +extern cpumask_map_t cpu_online_map; +extern cpumask_map_t cpu_present_map; +extern cpumask_map_t cpu_active_map;#define cpu_is_offline (cpu) unlikely(!cpu_online(cpu)) -#define for_each_possible_cpu (cpu) for_each_cpu_mask_nr((cpu), cpu_possible_map) -#define for_each_online_cpu (cpu) for_each_cpu_mask_nr((cpu), cpu_online_map) -#define for_each_present_cpu (cpu) for_each_cpu_mask_nr((cpu), cpu_present_map) +#define for_each_possible_cpu (cpu) for_each_cpu_mask((cpu), cpu_possible_map) +#define for_each_online_cpu (cpu) for_each_cpu_mask((cpu), cpu_online_map) +#define for_each_present_cpu (cpu) for_each_cpu_mask((cpu), cpu_present_map)#endif /* __LINUX_CPUMASK_H */
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Linus TorvaldsQuoted 10 lines · On Friday 26 September 2008 01:42:13 Linus Torvalds wrote:
Quoted 9 lines · Rusty Russell
This turns out to be awful in practice, mainly due to const.
Consider:#ifdef CONFIG_CPUMASK_OFFSTACK typedef unsigned long *cpumask_t; #else typedef unsigned long cpumask_t[1]; #endif
Quoted 6 lines
Quoted 1 line
cpumask_t returns_cpumask(void);No. That's already broken. You cannot return a
cpumask_t, regardless of interface. We must not do it regardless of how we pass those things around, since it generates _yet_ another temporary on the stack for the return slot for any kind of structure.
No, for large NR_CPUS, cpumask_t is a pointer as shown. And we have numerous basic functions which return a cpumask_t. Yes, this is part of the problem.
Quoted 4 lines
What _is_ relevant is how we allocate them when we need temporary CPU masks. And _that_ is where my suggestion comes in. For small
NR_CPUS, we really do want to allocate them on the stack, because calling kmalloc for a 4- or 8-byte allocation is just _stupid_.
Right, but cpumask_t is used for far more than stack decls, thus the problems.
I can make a separate "cpumask_stack_t" and use your method tho. I think that might even reduce churn and allow us to do this in parts.
Quoted 3 lines
which has to be converted some way. And I think it needs to be converted in a way that does *not* force us to call
kmalloc()for idiotically small values.
Yeah, got that. But your suggestion to change cpumask_t turned out horribly ugly.
Cheers,
Rusty.
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 34 lines · Rusty Russell
Quoted 10 lines · On Friday 26 September 2008 01:42:13 Linus Torvalds wrote:
Quoted 9 lines · Rusty Russell
This turns out to be awful in practice, mainly due to const.
Consider:#ifdef CONFIG_CPUMASK_OFFSTACK typedef unsigned long *cpumask_t; #else typedef unsigned long cpumask_t[1]; #endifQuoted 5 lines
Quoted 1 line
cpumask_t returns_cpumask(void);No. That's already broken. You cannot return a
cpumask_t, regardless of interface. We must not do it regardless of how we pass those things around, since it generates _yet_ another temporary on the stack for the return slot for any kind of structure.No, for large
NR_CPUS,cpumask_tis a pointer as shown. And we have numerous basic functions which return acpumask_t. Yes, this is part of the problem.Quoted 4 lines
What _is_ relevant is how we allocate them when we need temporary CPU masks. And _that_ is where my suggestion comes in. For small
NR_CPUS, we really do want to allocate them on the stack, because calling kmalloc for a 4- or 8-byte allocation is just _stupid_.Right, but
cpumask_tis used for far more than stack decls, thus the problems.I can make a separate "
cpumask_stack_t" and use your method tho. I think that might even reduce churn and allow us to do this in parts.Quoted 3 lines
which has to be converted some way. And I think it needs to be converted in a way that does *not* force us to call
kmalloc()for idiotically small values.
Quoted 5 lines
Yeah, got that. But your suggestion to change
cpumask_tturned out horribly ugly.Cheers,
Rusty.
Hi Rusty,
I've gotten some good traction on the changes in the following patch. About 30% of the kernel is compiling right now and I'm picking up errors and warnings as I'm going along. I think it's doing most of what we need. Attempting to hide the cpumask struct definition caused all kinds of problems with the inline functions and statically declaring cpumask's.
(The following patch is a combination of all the changes to cpumask.h with the header from the first patch. I'll send you a complete copy in separate email.)
Thanks,
Mike
--
Subject: [RFC 1/1] cpumask: Provide new cpumask API
Provide new cpumask interface API. The relevant change is basically cpumask_t becomes an opaque object. I believe this results in the minimum amount of editing while still allowing the inline cpumask functions, and the ability to declare static cpumask objects.
/* raw declaration */
struct __cpumask_data_s { DECLARE_BITMAP(bits, NR_CPUS); };
/* cpumask_map_t used for declaring static cpumask maps */
typedef struct __cpumask_data_s cpumask_map_t[1];
/* cpumask_t used for function args and return pointers */
typedef struct __cpumask_data_s *cpumask_t;
/* cpumask_var_t used for local variable, definition follows */
typedef struct __cpumask_data_s cpumask_var_t[1]; /* SMALL NR_CPUS */
typedef struct __cpumask_data_s *cpumask_var_t; /* LARGE NR_CPUS */
/* replaces cpumask_t dst = (cpumask_t)src */
void cpus_copy(cpumask_t dst, const cpumask_t src);Remove the '*' indirection in all references to cpumask_t objects. You can change the reference to the cpumask object but not the cpumask object itself without using the functions that operate on cpumask objects (f.e. the cpu_* operators). Functions can return a cpumask_t (which is a pointer to the cpumask object) and only be passed a cpumask_t.
All uses of cpumask_t on the stack are changed to be cpumask_var_t except for pointers to static cpumask objects. Allocation of local (temp) cpumask objects will follow...
All cpumask operators now operate using nr_cpu_ids instead of NR_CPUS. All variants of the cpumask operators which used nr_cpu_ids instead of NR_CPUS are deleted.
All variants of functions which use the (old cpumask_t *) pointer are deleted (f.e. set_cpus_allowed_ptr()).
Based on code from Rusty Russell <rusty-8n+1lVoiYb80n/[email protected]> (THANKS!!)
include/linux/cpumask.h +201 -214
@@ -3,7 +3,8 @@/* * Cpumasks provide a bitmap suitable for representing the -* set of CPU's in a system, one bit position per CPU number. +* set of CPU's in a system, one bit position per CPU number up to +* nr_cpu_ids (<= NR_CPUS). * * See detailed comments in the file linux/bitmap.h describing the * data type on which these cpumasks are based.
@@ -18,18 +19,6 @@* For details of cpus_fold(), see bitmap_fold in lib/bitmap.c. * * . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -* Note: The alternate operations with the suffix "_nr" are used -* to limit the range of the loop to nr_cpu_ids instead of -* NR_CPUS when NR_CPUS > 64 for performance reasons. -* If NR_CPUS is <= 64 then most assembler bitmask -* operators execute faster with a constant range, so -* the operator will continue to use NR_CPUS. -* -* Another consideration is that nr_cpu_ids is initialized -* to NR_CPUS and isn't lowered until the possible cpus are -* discovered (including any disabled cpus). So early uses -* will span the entire range of NR_CPUS. -* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . * * The available cpumask operations are: *
@@ -37,6 +26,7 @@* void cpu_clear(cpu, mask) turn off bit 'cpu' in mask * void cpus_setall(mask) set all bits * void cpus_clear(mask) clear all bits +* void cpus_copy(dst, src) copies cpumask bits from src to dst * int cpu_isset(cpu, mask) true iff bit 'cpu' set in mask * int cpu_test_and_set(cpu, mask) test and set bit 'cpu' in mask *
@@ -52,52 +42,22 @@* int cpus_empty(mask) Is mask empty (no bits sets)? * int cpus_full(mask) Is mask full (all bits sets)? * int cpus_weight(mask) Hamming weigh - number of set bits -* int cpus_weight_nr(mask) Same using nr_cpu_ids instead of NR_CPUS * * void cpus_shift_right(dst, src, n) Shift right * void cpus_shift_left(dst, src, n) Shift left * -* int first_cpu(mask) Number lowest set bit, or NR_CPUS -* int next_cpu(cpu, mask) Next cpu past 'cpu', or NR_CPUS -* int next_cpu_nr(cpu, mask) Next cpu past 'cpu', or nr_cpu_ids +* int cpus_first(mask) Number lowest set bit, or nr_cpu_ids +* int cpus_next(cpu, mask) Next cpu past 'cpu', or nr_cpu_ids +* int cpus_next_in(cpu, mask, andmask) Next cpu in mask & andmask or nr_cpu_ids +* +* cpumask_t cpumask_of_cpu(cpu) Return pointer to cpumask with bit +* 'cpu' set * -* cpumask_t cpumask_of_cpu(cpu) Return cpumask with bit 'cpu' set -* (can be used as an lvalue) +* cpu_mask_all cpumask_map_t of all bits set * CPU_MASK_ALL Initializer - all bits set * CPU_MASK_NONE Initializer - no bits set * unsigned long *cpus_addr(mask) Array of unsigned long's in mask * -* CPUMASK_ALLOC kmalloc's a structure that is a composite of many cpumask_t -* variables, and CPUMASK_PTR provides pointers to each field. -* -* The structure should be defined something like this: -* struct my_cpumasks { -* cpumask_t mask1; -* cpumask_t mask2; -* }; -* -* Usage is then: -* CPUMASK_ALLOC(my_cpumasks); -* CPUMASK_PTR(mask1, my_cpumasks); -* CPUMASK_PTR(mask2, my_cpumasks); -* -* --- DO NOT reference cpumask_t pointers until this check --- -* if (my_cpumasks == NULL) -* "kmalloc failed"... -* -* References are now pointers to the cpumask_t variables (*mask1, ...) -* -*if NR_CPUS > BITS_PER_LONG -* CPUMASK_ALLOC(m) Declares and allocates struct m *m = -* kmalloc(sizeof(*m), GFP_KERNEL) -* CPUMASK_FREE(m) Macro for kfree(m) -*else -* CPUMASK_ALLOC(m) Declares struct m _m, *m = &_m -* CPUMASK_FREE(m) Nop -*endif -* CPUMASK_PTR(v, m) Declares cpumask_t *v = &(m->v) -* ------------------------------------------------------------------------ -* * int cpumask_scnprintf(buf, len, mask) Format cpumask for printing * int cpumask_parse_user(ubuf, ulen, mask) Parse ascii string as cpumask * int cpulist_scnprintf(buf, len, mask) Format cpumask as list for printing
@@ -107,8 +67,8 @@* void cpus_onto(dst, orig, relmap) *dst = orig relative to relmap * void cpus_fold(dst, orig, sz) dst bits = orig bits mod sz * -* for_each_cpu_mask(cpu, mask) for-loop cpu over mask using NR_CPUS -* for_each_cpu_mask_nr(cpu, mask) for-loop cpu over mask using nr_cpu_ids +* for_each_cpu(cpu, mask) for-loop cpu over mask +* for_each_cpu_in(cpu, mask, andmask) for-loop cpu over mask & andmask * * int num_online_cpus() Number of online CPUs * int num_possible_cpus() Number of all possible CPUs
@@ -118,6 +78,7 @@* int cpu_possible(cpu) Is some cpu possible? * int cpu_present(cpu) Is some cpu present (can schedule)? * +* int any_cpu_in(mask, andmask) First cpu in mask & andmask * int any_online_cpu(mask) First online cpu in mask * * for_each_possible_cpu(cpu) for-loop cpu over cpu_possible_map
@@ -138,129 +99,229 @@#include <linux/threads.h> #include <linux/bitmap.h> -typedef struct { DECLARE_BITMAP(bits, NR_CPUS); } cpumask_t; -extern cpumask_t _unused_cpumask_arg_; +/* raw declaration */ +struct __cpumask_data_s { DECLARE_BITMAP(bits, NR_CPUS); }; + +/* cpumask_map_t used for declaring static cpumask maps */ +typedef struct __cpumask_data_s cpumask_map_t [1 ]; + +/* cpumask_t used for function args and return pointers */ +typedef struct __cpumask_data_s *cpumask_t ; + +/* cpumask_var_t used for local variable, definition follows */ + +#if NR_CPUS ==1 + +/* cpumask_var_t used for local variable */ +typedef struct __cpumask_data_s cpumask_var_t [1 ]; + +#define nr_cpu_ids 1 +#define cpus_first (src) ({ (void)(src); 0; }) +#define cpus_next (n, src) ({ (void)(src); 1; }) +#define cpus_next_in (n, src, andsrc) ({ (void)(src); 1; }) +#define any_online_cpu (mask) 0 +#define for_each_cpu (cpu, mask) \ + for ((cpu) = 0; (cpu) < 1; (cpu)++, (void)mask) +#define for_each_cpu_in (cpu, mask, andmask) \ + for ((cpu) = 0; (cpu) < 1; (cpu)++, (void)mask, (void)andmask) + +#define num_online_cpus () 1 +#define num_possible_cpus () 1 +#define num_present_cpus () 1 +#define cpu_online (cpu) ((cpu) == 0) +#define cpu_possible (cpu) ((cpu) == 0) +#define cpu_present (cpu) ((cpu) == 0) +#define cpu_active (cpu) ((cpu) == 0) + +#else /* ... NR_CPUS > 1 */ + +#ifdef CONFIG_CPUMASKS_ONSTACK + +/* Constant is usually more efficient than a variable for small NR_CPUS */ +#define nr_cpu_ids NR_CPUS + +/* cpumask_var_t used for local variable */ +typedef struct __cpumask_data_s cpumask_var_t [1 ]; +static inline int cpumask_size (void ) +{ +return sizeof (struct __cpumask_data_s ); +} + +#else + +/* Starts at NR_CPUS until acpi code discovers actual number. */ +extern int nr_cpu_ids; + +/* cpumask_var_t used for local variable */ +typedef struct __cpumask_data_s *cpumask_var_t ; +static inline int cpumask_size (void ) +{ +return BITS_TO_LONGS (nr_cpu_ids) *sizeof (long ); +} + +#endif /* NR_CPUS > BITS_PER_LONG */ + +/* Deprecated: use for_each_cpu() */ +#define for_each_cpu_mask (cpu, mask) for_each_cpu(cpu, mask) + +/* Deprecated: use cpus_first()/cpus_next() */ +#define first_cpu (src) cpus_first((src)) +#define next_cpu (n, src) cpus_next((n), (src)) + +extern int cpus_first (const cpumask_t srcp); +extern int cpus_next (int n,const cpumask_t srcp); +extern int cpus_next_in (int n,const cpumask_t srcp,const cpumask_t andsrc); +extern int any_cpu_in (const cpumask_t mask); + +#define any_online_cpu (mask) any_cpu_in((const cpumask_t)(mask), \ + (const cpumask_t)cpu_online_map) + +#define for_each_cpu (cpu, mask) \ + for ((cpu) = -1; \ + (cpu) = cpus_next((cpu), (mask)), \ + (cpu) < nr_cpu_ids; ) + +#define for_each_cpu_in (cpu, mask, andmask) \ + for ((cpu) = -1; \ + (cpu) = cpus_next_in((cpu), (mask), (andmask)), \ + (cpu) < nr_cpu_ids; ) + -#define cpu_set (cpu, dst) __cpu_set((cpu), &(dst)) -static inline void __cpu_set (int cpu,volatile cpumask_t *dstp) +#define num_online_cpus () cpus_weight(cpu_online_map) +#define num_possible_cpus () cpus_weight(cpu_possible_map) +#define num_present_cpus () cpus_weight(cpu_present_map) +#define cpu_online (cpu) cpu_isset((cpu), cpu_online_map) +#define cpu_possible (cpu) cpu_isset((cpu), cpu_possible_map) +#define cpu_present (cpu) cpu_isset((cpu), cpu_present_map) +#define cpu_active (cpu) cpu_isset((cpu), cpu_active_map) +#endif /* NR_CPUS > 1 */ + +#define cpu_set (cpu, dst) __cpu_set((cpu), (dst)) +static inline void __cpu_set (int cpu,volatile cpumask_t dstp) {set_bit (cpu, dstp->bits); } -#define cpu_clear (cpu, dst) __cpu_clear((cpu), &(dst)) -static inline void __cpu_clear (int cpu,volatile cpumask_t *dstp) +#define cpu_clear (cpu, dst) __cpu_clear((cpu), (dst)) +static inline void __cpu_clear (int cpu,volatile cpumask_t dstp) {clear_bit (cpu, dstp->bits); } -#define cpus_setall (dst) __cpus_setall(&(dst), NR_CPUS) -static inline void __cpus_setall (cpumask_t *dstp,int nbits) +#define cpus_setall (dst) __cpus_setall((dst), nr_cpu_ids) +static inline void __cpus_setall (cpumask_t dstp,int nbits) {bitmap_fill (dstp->bits, nbits); } -#define cpus_clear (dst) __cpus_clear(&(dst), NR_CPUS) -static inline void __cpus_clear (cpumask_t *dstp,int nbits) +#define cpus_clear (dst) __cpus_clear((dst), nr_cpu_ids) +static inline void __cpus_clear (cpumask_t dstp,int nbits) {bitmap_zero (dstp->bits, nbits); } +#define cpus_copy (dst, src) __cpus_copy((dst), (src), nr_cpu_ids) +static inline void __cpus_copy (cpumask_t dstp,const cpumask_t srcp,int nbits) +{ +bitmap_copy (dstp->bits, srcp->bits, nbits); +} +/* No static inline type checking - see Subtlety (1) above. */ -#define cpu_isset (cpu, cpumask) test_bit((cpu), (cpumask).bits) +#define cpu_isset (cpu, cpumask) test_bit((cpu), (cpumask)->bits) -#define cpu_test_and_set (cpu, cpumask) __cpu_test_and_set((cpu), &(cpumask)) -static inline int __cpu_test_and_set (int cpu,cpumask_t *addr) +#define cpu_test_and_set (cpu, cpumask) __cpu_test_and_set((cpu), (cpumask)) +static inline int __cpu_test_and_set (int cpu,cpumask_t addr) {return test_and_set_bit (cpu, addr->bits); } -#define cpus_and (dst, src1, src2) __cpus_and(&(dst), &(src1), &(src2), NR_CPUS) -static inline void __cpus_and (cpumask_t *dstp,const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) +#define cpus_and (dst, src1, src2) __cpus_and((dst), (src1), (src2), nr_cpu_ids) +static inline void __cpus_and (cpumask_t dstp,const cpumask_t src1p, +const cpumask_t src2p,int nbits) {bitmap_and (dstp->bits, src1p->bits, src2p->bits, nbits); } -#define cpus_or (dst, src1, src2) __cpus_or(&(dst), &(src1), &(src2), NR_CPUS) -static inline void __cpus_or (cpumask_t *dstp,const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) +#define cpus_or (dst, src1, src2) __cpus_or((dst), (src1), (src2), nr_cpu_ids) +static inline void __cpus_or (cpumask_t dstp,const cpumask_t src1p, +const cpumask_t src2p,int nbits) {bitmap_or (dstp->bits, src1p->bits, src2p->bits, nbits); } -#define cpus_xor (dst, src1, src2) __cpus_xor(&(dst), &(src1), &(src2), NR_CPUS) -static inline void __cpus_xor (cpumask_t *dstp,const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) +#define cpus_xor (dst, src1, src2) __cpus_xor((dst), (src1), (src2), nr_cpu_ids) +static inline void __cpus_xor (cpumask_t dstp,const cpumask_t src1p, +const cpumask_t src2p,int nbits) {bitmap_xor (dstp->bits, src1p->bits, src2p->bits, nbits); }#define cpus_andnot (dst, src1, src2) \ - __cpus_andnot(&(dst), &(src1), &(src2), NR_CPUS) -static inline void __cpus_andnot (cpumask_t *dstp,const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) + __cpus_andnot((dst), (src1), (src2), nr_cpu_ids) +static inline void __cpus_andnot (cpumask_t dstp,const cpumask_t src1p, +const cpumask_t src2p,int nbits) {bitmap_andnot (dstp->bits, src1p->bits, src2p->bits, nbits); } -#define cpus_complement (dst, src) __cpus_complement(&(dst), &(src), NR_CPUS) -static inline void __cpus_complement (cpumask_t *dstp, -const cpumask_t *srcp,int nbits) +#define cpus_complement (dst, src) __cpus_complement((dst), (src), nr_cpu_ids) +static inline void __cpus_complement (cpumask_t dstp, +const cpumask_t srcp,int nbits) {bitmap_complement (dstp->bits, srcp->bits, nbits); } -#define cpus_equal (src1, src2) __cpus_equal(&(src1), &(src2), NR_CPUS) -static inline int __cpus_equal (const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) +#define cpus_equal (src1, src2) __cpus_equal((src1), (src2), nr_cpu_ids) +static inline int __cpus_equal (const cpumask_t src1p, +const cpumask_t src2p,int nbits) {return bitmap_equal (src1p->bits, src2p->bits, nbits); } -#define cpus_intersects (src1, src2) __cpus_intersects(&(src1), &(src2), NR_CPUS) -static inline int __cpus_intersects (const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) +#define cpus_intersects (src1, src2) __cpus_intersects((src1), (src2), nr_cpu_ids) +static inline int __cpus_intersects (const cpumask_t src1p, +const cpumask_t src2p,int nbits) {return bitmap_intersects (src1p->bits, src2p->bits, nbits); } -#define cpus_subset (src1, src2) __cpus_subset(&(src1), &(src2), NR_CPUS) -static inline int __cpus_subset (const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) +#define cpus_subset (src1, src2) __cpus_subset((src1), (src2), nr_cpu_ids) +static inline int __cpus_subset (const cpumask_t src1p, +const cpumask_t src2p,int nbits) {return bitmap_subset (src1p->bits, src2p->bits, nbits); } -#define cpus_empty (src) __cpus_empty(&(src), NR_CPUS) -static inline int __cpus_empty (const cpumask_t *srcp,int nbits) +#define cpus_empty (src) __cpus_empty((src), nr_cpu_ids) +static inline int __cpus_empty (const cpumask_t srcp,int nbits) {return bitmap_empty (srcp->bits, nbits); } -#define cpus_full (cpumask) __cpus_full(&(cpumask), NR_CPUS) -static inline int __cpus_full (const cpumask_t *srcp,int nbits) +#define cpus_full (cpumask) __cpus_full((cpumask), nr_cpu_ids) +static inline int __cpus_full (const cpumask_t srcp,int nbits) {return bitmap_full (srcp->bits, nbits); } -#define cpus_weight (cpumask) __cpus_weight(&(cpumask), NR_CPUS) -static inline int __cpus_weight (const cpumask_t *srcp,int nbits) +#define cpus_weight (cpumask) __cpus_weight((cpumask), nr_cpu_ids) +static inline int __cpus_weight (const cpumask_t srcp,int nbits) {return bitmap_weight (srcp->bits, nbits); }#define cpus_shift_right (dst, src, n) \ - __cpus_shift_right(&(dst), &(src), (n), NR_CPUS) -static inline void __cpus_shift_right (cpumask_t *dstp, -const cpumask_t *srcp,int n,int nbits) + __cpus_shift_right((dst), (src), (n), nr_cpu_ids) +static inline void __cpus_shift_right (cpumask_t dstp, +const cpumask_t srcp,int n,int nbits) {bitmap_shift_right (dstp->bits, srcp->bits, n, nbits); }#define cpus_shift_left (dst, src, n) \ - __cpus_shift_left(&(dst), &(src), (n), NR_CPUS) -static inline void __cpus_shift_left (cpumask_t *dstp, -const cpumask_t *srcp,int n,int nbits) + __cpus_shift_left((dst), (src), (n), nr_cpu_ids) +static inline void __cpus_shift_left (cpumask_t dstp, +const cpumask_t srcp,int n,int nbits) {bitmap_shift_left (dstp->bits, srcp->bits, n, nbits); }
@@ -275,11 +336,12 @@ static inline void __cpus_shift_left(cpuextern const unsigned long cpu_bit_bitmap[BITS_PER_LONG +1 ][BITS_TO_LONGS (NR_CPUS )]; -static inline const cpumask_t *get_cpu_mask (unsigned int cpu) +/* XXX - "const" causes: "warning: type qualifiers ignored on function return type" */ +static inline /*const*/ cpumask_t get_cpu_mask (unsigned int cpu) {const unsigned long *p = cpu_bit_bitmap[1 + cpu %BITS_PER_LONG ]; p -= cpu /BITS_PER_LONG ; -return (const cpumask_t *)p; +return (const cpumask_t )p; }/*
@@ -287,7 +349,7 @@ static inline const cpumask_t *get_cpu_m* gcc optimizes it out (it's a constant) and there's no huge stack * variable created: */ -#define cpumask_of_cpu (cpu) (*get_cpu_mask(cpu)) +#define cpumask_of_cpu (cpu) (get_cpu_mask(cpu))#define CPU_MASK_LAST_WORD BITMAP_LAST_WORD_MASK(NR_CPUS)
@@ -295,152 +357,94 @@ static inline const cpumask_t *get_cpu_m#if NR_CPUS <=BITS_PER_LONG #define CPU_MASK_ALL \ -(cpumask_t) { { \ +(cpumask_map_t) { [0] = { { \ [BITS_TO_LONGS(NR_CPUS)-1] = CPU_MASK_LAST_WORD \ -} } - -#define CPU_MASK_ALL_PTR (&CPU_MASK_ALL) +} } }#else #define CPU_MASK_ALL \ -(cpumask_t) { { \ +(struct __cpumask_data_s) { [0] = { { \ [0 ... BITS_TO_LONGS(NR_CPUS)-2] = ~0UL, \ [BITS_TO_LONGS(NR_CPUS)-1] = CPU_MASK_LAST_WORD \ -} } - -/* cpu_mask_all is in init/main.c */ -extern cpumask_t cpu_mask_all; -#define CPU_MASK_ALL_PTR (&cpu_mask_all) +} } }#endif #define CPU_MASK_NONE \ -(cpumask_t) { { \ +(cpumask_map_t) { [0] = { { \ [0 ... BITS_TO_LONGS(NR_CPUS)-1] = 0UL \ -} } +} } }#define CPU_MASK_CPU0 \ -(cpumask_t) { { \ +(cpumask_map_t) { [0] = { { \ [0] = 1UL \ -} } +} } } -#define cpus_addr (src) ((src).bits) - -#if NR_CPUS >BITS_PER_LONG -#define CPUMASK_ALLOC (m) struct m *m = kmalloc(sizeof(*m), GFP_KERNEL) -#define CPUMASK_FREE (m) kfree(m) -#else -#define CPUMASK_ALLOC (m) struct m _m, *m = &_m -#define CPUMASK_FREE (m) -#endif -#define CPUMASK_PTR (v, m) cpumask_t *v = &(m->v) +#define cpus_addr (src) ((src)->bits)#define cpumask_scnprintf (buf, len, src) \ - __cpumask_scnprintf((buf), (len), &(src), NR_CPUS) + __cpumask_scnprintf((buf), (len), (src), nr_cpu_ids)static inline int __cpumask_scnprintf (char *buf,int len, -const cpumask_t *srcp,int nbits) +const cpumask_t srcp,int nbits) {return bitmap_scnprintf (buf, len, srcp->bits, nbits); }#define cpumask_parse_user (ubuf, ulen, dst) \ - __cpumask_parse_user((ubuf), (ulen), &(dst), NR_CPUS) + __cpumask_parse_user((ubuf), (ulen), (dst), nr_cpu_ids)static inline int __cpumask_parse_user (const char __user *buf,int len, -cpumask_t *dstp,int nbits) +cpumask_t dstp,int nbits) {return bitmap_parse_user (buf, len, dstp->bits, nbits); }#define cpulist_scnprintf (buf, len, src) \ - __cpulist_scnprintf((buf), (len), &(src), NR_CPUS) + __cpulist_scnprintf((buf), (len), (src), nr_cpu_ids)static inline int __cpulist_scnprintf (char *buf,int len, -const cpumask_t *srcp,int nbits) +const cpumask_t srcp,int nbits) {return bitmap_scnlistprintf (buf, len, srcp->bits, nbits); } -#define cpulist_parse (buf, dst) __cpulist_parse((buf), &(dst), NR_CPUS) -static inline int __cpulist_parse (const char *buf,cpumask_t *dstp,int nbits) +#define cpulist_parse (buf, dst) __cpulist_parse((buf), (dst), nr_cpu_ids) +static inline int __cpulist_parse (const char *buf,cpumask_t dstp,int nbits) {return bitmap_parselist (buf, dstp->bits, nbits); }#define cpu_remap (oldbit, old, new) \ - __cpu_remap((oldbit), &(old), &(new), NR_CPUS) + __cpu_remap((oldbit), (old), (new), nr_cpu_ids)static inline int __cpu_remap (int oldbit, -const cpumask_t *oldp,const cpumask_t *newp,int nbits) +const cpumask_t oldp,const cpumask_t newp,int nbits) {return bitmap_bitremap (oldbit, oldp->bits, newp->bits, nbits); }#define cpus_remap (dst, src, old, new) \ - __cpus_remap(&(dst), &(src), &(old), &(new), NR_CPUS) -static inline void __cpus_remap (cpumask_t *dstp,const cpumask_t *srcp, -const cpumask_t *oldp,const cpumask_t *newp,int nbits) + __cpus_remap((dst), (src), (old), (new), nr_cpu_ids) +static inline void __cpus_remap (cpumask_t dstp,const cpumask_t srcp, +const cpumask_t oldp,const cpumask_t newp,int nbits) {bitmap_remap (dstp->bits, srcp->bits, oldp->bits, newp->bits, nbits); }#define cpus_onto (dst, orig, relmap) \ - __cpus_onto(&(dst), &(orig), &(relmap), NR_CPUS) -static inline void __cpus_onto (cpumask_t *dstp,const cpumask_t *origp, -const cpumask_t *relmapp,int nbits) + __cpus_onto((dst), (orig), (relmap), nr_cpu_ids) +static inline void __cpus_onto (cpumask_t dstp,const cpumask_t origp, +const cpumask_t relmapp,int nbits) {bitmap_onto (dstp->bits, origp->bits, relmapp->bits, nbits); }#define cpus_fold (dst, orig, sz) \ - __cpus_fold(&(dst), &(orig), sz, NR_CPUS) -static inline void __cpus_fold (cpumask_t *dstp,const cpumask_t *origp, + __cpus_fold((dst), (orig), sz, nr_cpu_ids) +static inline void __cpus_fold (cpumask_t dstp,const cpumask_t origp,int sz,int nbits) {bitmap_fold (dstp->bits, origp->bits, sz, nbits); } -#if NR_CPUS ==1 - -#define nr_cpu_ids 1 -#define first_cpu (src) ({ (void)(src); 0; }) -#define next_cpu (n, src) ({ (void)(src); 1; }) -#define any_online_cpu (mask) 0 -#define for_each_cpu_mask (cpu, mask) \ - for ((cpu) = 0; (cpu) < 1; (cpu)++, (void)mask) - -#else /* NR_CPUS > 1 */ - -extern int nr_cpu_ids; -int __first_cpu (const cpumask_t *srcp); -int __next_cpu (int n,const cpumask_t *srcp); -int __any_online_cpu (const cpumask_t *mask); - -#define first_cpu (src) __first_cpu(&(src)) -#define next_cpu (n, src) __next_cpu((n), &(src)) -#define any_online_cpu (mask) __any_online_cpu(&(mask)) -#define for_each_cpu_mask (cpu, mask) \ - for ((cpu) = -1; \ - (cpu) = next_cpu((cpu), (mask)), \ - (cpu) < NR_CPUS; ) -#endif - -#if NR_CPUS <=64 - -#define next_cpu_nr (n, src) next_cpu(n, src) -#define cpus_weight_nr (cpumask) cpus_weight(cpumask) -#define for_each_cpu_mask_nr (cpu, mask) for_each_cpu_mask(cpu, mask) - -#else /* NR_CPUS > 64 */ - -int __next_cpu_nr (int n,const cpumask_t *srcp); -#define next_cpu_nr (n, src) __next_cpu_nr((n), &(src)) -#define cpus_weight_nr (cpumask) __cpus_weight(&(cpumask), nr_cpu_ids) -#define for_each_cpu_mask_nr (cpu, mask) \ - for ((cpu) = -1; \ - (cpu) = next_cpu_nr((cpu), (mask)), \ - (cpu) < nr_cpu_ids; ) - -#endif /* NR_CPUS > 64 */ -/* * The following particular system cpumasks and operations manage * possible, present, active and online cpus. Each of them is a fixed size
@@ -498,33 +502,16 @@ int __next_cpu_nr(int n, const cpumask_t* main(){ set1(3); set2(5); } */ -extern cpumask_t cpu_possible_map; -extern cpumask_t cpu_online_map; -extern cpumask_t cpu_present_map; -extern cpumask_t cpu_active_map; - -#if NR_CPUS >1 -#define num_online_cpus () cpus_weight_nr(cpu_online_map) -#define num_possible_cpus () cpus_weight_nr(cpu_possible_map) -#define num_present_cpus () cpus_weight_nr(cpu_present_map) -#define cpu_online (cpu) cpu_isset((cpu), cpu_online_map) -#define cpu_possible (cpu) cpu_isset((cpu), cpu_possible_map) -#define cpu_present (cpu) cpu_isset((cpu), cpu_present_map) -#define cpu_active (cpu) cpu_isset((cpu), cpu_active_map) -#else -#define num_online_cpus () 1 -#define num_possible_cpus () 1 -#define num_present_cpus () 1 -#define cpu_online (cpu) ((cpu) == 0) -#define cpu_possible (cpu) ((cpu) == 0) -#define cpu_present (cpu) ((cpu) == 0) -#define cpu_active (cpu) ((cpu) == 0) -#endif +extern cpumask_map_t cpu_possible_map; +extern cpumask_map_t cpu_online_map; +extern cpumask_map_t cpu_present_map; +extern cpumask_map_t cpu_active_map; +extern cpumask_map_t cpu_mask_all;#define cpu_is_offline (cpu) unlikely(!cpu_online(cpu)) -#define for_each_possible_cpu (cpu) for_each_cpu_mask_nr((cpu), cpu_possible_map) -#define for_each_online_cpu (cpu) for_each_cpu_mask_nr((cpu), cpu_online_map) -#define for_each_present_cpu (cpu) for_each_cpu_mask_nr((cpu), cpu_present_map) +#define for_each_possible_cpu (cpu) for_each_cpu((cpu), cpu_possible_map) +#define for_each_online_cpu (cpu) for_each_cpu((cpu), cpu_online_map) +#define for_each_present_cpu (cpu) for_each_cpu((cpu), cpu_present_map)#endif /* __LINUX_CPUMASK_H */
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 9 lines · Mike Travis
Hi Rusty,
I've gotten some good traction on the changes in the following patch. About 30% of the kernel is compiling right now and I'm picking up errors and warnings as I'm going along. I think it's doing most of what we need. Attempting to hide the cpumask struct definition caused all kinds of problems with the inline functions and statically declaring cpumask's.
Quoted 3 lines
(The following patch is a combination of all the changes to
cpumask.hwith the header from the first patch. I'll send you a complete copy in separate email.)
could you please send whatever .c changes you have already, so that we can have a look at how the end result will look like? Doesnt have to build, i'm just curious about how it looks like in practice, semantically.
Ingo
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 15 lines · Ingo Molnar
Quoted 9 lines · Mike Travis
Hi Rusty,
I've gotten some good traction on the changes in the following patch. About 30% of the kernel is compiling right now and I'm picking up errors and warnings as I'm going along. I think it's doing most of what we need. Attempting to hide the cpumask struct definition caused all kinds of problems with the inline functions and statically declaring cpumask's.
Quoted 3 lines
(The following patch is a combination of all the changes to
cpumask.hwith the header from the first patch. I'll send you a complete copy in separate email.)
Quoted 6 lines
could you please send whatever .c changes you have already, so that we can have a look at how the end result will look like? Doesnt have to build, i'm just curious about how it looks like in practice, semantically.
Ingo
I will, and the full "allyesconfig" does compile. And it's basically a benign change in that the functionality is still the same. I'm currently reordering it a bit to clean it up.
Thanks,
Mike
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 9 lines · Mike Travis
Quoted 4 lines
could you please send whatever .c changes you have already, so that we can have a look at how the end result will look like? Doesnt have to build, i'm just curious about how it looks like in practice, semantically.
I will, and the full "allyesconfig" does compile. And it's basically a benign change in that the functionality is still the same. I'm currently reordering it a bit to clean it up.
btw., are the resulting instructions also expected to be the same? If yes then you might want to verify it all by making sure the md5's of the .o's do not change.
(If that's not possible (gcc decides to compile it a bit differently) then no big deal, just wanted to mention the possibility.)
Ingo
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 11 lines · Ingo Molnar
Quoted 8 lines · Mike Travis
Quoted 4 lines
could you please send whatever .c changes you have already, so that we can have a look at how the end result will look like? Doesnt have to build, i'm just curious about how it looks like in practice, semantically.
I will, and the full "allyesconfig" does compile. And it's basically a benign change in that the functionality is still the same. I'm currently reordering it a bit to clean it up.
Quoted 8 lines
btw., are the resulting instructions also expected to be the same? If yes then you might want to verify it all by making sure the md5's of the .o's do not change.
(If that's not possible (gcc decides to compile it a bit differently) then no big deal, just wanted to mention the possibility.)
Ingo
Well, not exactly... ;-) It does institute the new API change that specifies only pointers to cpumask's can be passed to functions and returned from functions. I really wanted the default cpumask_t to be a constant so those instances where the passed in cpumask is used as a read/write temp variable would be caught. But it started getting messy.
One pain is:
typedef struct __cpumask_s *cpumask_t; const cpumask_t xxx;
is not the same as:
typedef const struct __cpumask_s *const_cpumask_t; const_cpumask_t xxx;
and I'm not exactly sure why. It came up when I tried to declare functions that returned a constant cpumask_t pointer (node_to_cpumask, cpumask_of_cpu, etc.)
The other major change I'm contemplating is to remove "cpumask_t" completely (maybe cpumask_ptr_t?). This would force every instance of cpumask_t to be examined. (I found quite a few I had missed in my original edits when I added the task struct temp cpumask's.)
Oh yeah, one question ... is "current" always valid?
Thanks,
Mike
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 6 lines · Mike Travis
One pain is:
typedef struct __cpumask_s *cpumask_t; const cpumask_t xxx;
Quoted 6 lines
is not the same as:
typedef const struct __cpumask_s *const_cpumask_t; const_cpumask_t xxx;and I'm not exactly sure why.
Umm. The const has different
One is
typedef const struct __cpumask_s *const_cpumask_t;
which becomes
(const struct __cpumask_s) *
while the other is
const cpumask_t xxx
which is
const (struct __cpumask_s *)
and if you look a bit more closely, you'll see that they are _obviously_ not the same thing at all.
Quite frankly, I personally do hate typedefs that end up being pointers, and used as pointers, without showing that in the source code.
When you do
type_t a; fn(a);
I expect the code to essentially do a pass-by-value. But when the type_t is a pointer, that doesn't really work.
Your issue with 'const' is just another version of the same. You don't want the _pointer_ to be const, you want what it points _to_ to be const. But because you hid the pointerness inside the typedef, you simply cannot do that.
The problem with cpumask's, of course, is that for the "small mask" case, we really don't want it to be a pointer. So now it's sometimes a pointer and sometimes not. The typedef hides that, and I understand why it's a good idea, but I'm surprised you didn't understand what the implications were for 'const', and I'm now a bit more leery about this whole thing just because the typedef ends up hiding so much - it doesn't just hide the basic type, it hides a very basic *code* issue.
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 26 lines · Linus Torvalds
Quoted 11 lines · Mike Travis
One pain is:
typedef struct __cpumask_s *cpumask_t; const cpumask_t xxx;is not the same as:
typedef const struct __cpumask_s *const_cpumask_t; const_cpumask_t xxx;and I'm not exactly sure why.
Umm. The const has different
One is
typedef const struct __cpumask_s *const_cpumask_t;which becomes
(const struct
__cpumask_s) *while the other is
Quoted 8 lines
const
cpumask_txxxwhich is
const (struct
__cpumask_s*)and if you look a bit more closely, you'll see that they are _obviously_ not the same thing at all.
Thanks, yes that explains what I should have figured out. (Nice way to explain it btw... ;-)
Quoted 4 lines
Quite frankly, I personally do hate typedefs that end up being pointers, and used as pointers, without showing that in the source code.
Quoted 8 lines
When you do
type_t a; fn(a);I expect the code to essentially do a pass-by-value. But when the
type_tis a pointer, that doesn't really work.
I agree, and as I mentioned, Rusty was working towards an alternative method of declaring cpumask's which does not hide this.
My goal was to create an (apparent) opaque handle for cpumask_t and modify the code so all changes to the contents of the cpumask are via functions.
Quoted 6 lines
Your issue with 'const' is just another version of the same. You don't want the _pointer_ to be const, you want what it points _to_ to be const. But because you hid the pointerness inside the typedef, you simply cannot do that.
Quoted 7 lines
The problem with cpumask's, of course, is that for the "small mask" case, we really don't want it to be a pointer. So now it's sometimes a pointer and sometimes not. The typedef hides that, and I understand why it's a good idea, but I'm surprised you didn't understand what the implications were for 'const', and I'm now a bit more leery about this whole thing just because the typedef ends up hiding so much - it doesn't just hide the basic type, it hides a very basic *code* issue.
Perhaps then defining the cpumask as a list of unsigned longs (remove the outer struct) would play more "naturally". Lists by definition are always referenced by pointers. I have another version of the patchset that shows this and I'll post just the cpumask.h and doc files.
Quoted 2 lines
Linus
Thanks!
Mike
RFC 1/1 cpumask: New cpumask API - take 2 - use unsigned longs
Quoted 1 line · Mike Travis
Linus Torvalds wrote:
...
Quoted 13 lines
Quoted 5 lines
Your issue with 'const' is just another version of the same. You don't want the _pointer_ to be const, you want what it points _to_ to be const. But because you hid the pointerness inside the typedef, you simply cannot do that.
Quoted 7 lines
The problem with cpumask's, of course, is that for the "small mask" case, we really don't want it to be a pointer. So now it's sometimes a pointer and sometimes not. The typedef hides that, and I understand why it's a good idea, but I'm surprised you didn't understand what the implications were for 'const', and I'm now a bit more leery about this whole thing just because the typedef ends up hiding so much - it doesn't just hide the basic type, it hides a very basic *code* issue.
Quoted 4 lines
Perhaps then defining the cpumask as a list of unsigned longs (remove the outer struct) would play more "naturally". Lists by definition are always referenced by pointers. I have another version of the patchset that shows this and I'll post just the
cpumask.hand doc files.
...
Here's an alternate proposal for the new cpumask API. I have not yet begun the tedious work of making it compile so there may still be some syntax errors. (Init/main.c compiles.) But I believe it more closely adheres to proper 'C' coding styles.
Thanks,
Mike
--
Subject: cpumask: Provide new cpumask API
[Copied from Documentation/cpumask.txt]
Introduction
Cpumask variables are used to represent (with a single bit) all the CPU's in a system. With the prolific increase in the number of CPU's in a single system image (SSI), managing this large number of cpus has become an ordeal. When the limit of CPU's in a system was small, the cpumask could fit within an integer. Even with the increase to 128, a bit pattern was well within a manageable size. Now with systems available that have 2048 cores, with hyper-threading there are 4096 cpu threads. And the number of cpus in an SSI is growing, 16,384 is right around the corner. Even desktop systems with only 2 or 4 sockets, a new generation of Intel processors that have 128 cores per socket will increase the count of CPU threads tremendously.
Thus the handling of the cpumask that represents this 4096 limit needs 512 bytes, putting pressure on the amount of stack space needed to accommodate temporary cpumask variables.
The primary goal of the cpumasks API is to accommodate these large number of cpus without effecting the compactness of Linux for small, compact systems.
The Changes
Provide new cpumask interface API. The relevant change is basically cpumask_t becomes an pointer to a constant list of unsigned longs and two new typedef's are used for declaring cpumask_t variables, cpumask_var_t and cpumask_map_t. This should result in the minimum amount of modifications while still allowing the inline cpumask functions, and the ability to declare static cpumask objects.
/* cpumask_t is used when pointing to a constant cpumask */
typedef const unsigned long *cpumask_t;
/* cpumask_map_t used for declaring static cpumask maps */
typedef unsigned long cpumask_map_t[BITS_TO_LONGS(nr_cpu_ids)];
/* cpumask_var_t used for read/write cpumask variables */
typedef unsigned long cpumask_var_t[BITS_TO_LONGS(nr_cpu_ids)];
/* SMALL NR_CPUS */
typedef unsigned long *cpumask_var_t; /* LARGE NR_CPUS */
/* replaces cpumask_t dst = (cpumask_t)src */
void cpus_copy(cpumask_var_t dst, cpumask_t src);You can change the reference to the cpumask object but to change the contents of the cpumask object itself you must use the functions that operate on cpumask objects (f.e. the cpu_* operators). Functions can return a cpumask_t (cpumask pointer) and can only be passed a cpumask_t.
All uses of a cpumask_t variable on the stack are changed to be cpumask_var_t except for pointers to static (read only) cpumask objects. Allocation of local (temp) cpumask objects use the functions available in cpumask_alloc.h. (descriptions to be supplied.)
All cpumask operators now operate using nr_cpu_ids instead of NR_CPUS. All variants of the cpumask operators which used nr_cpu_ids instead of NR_CPUS have been deleted.
All variants of functions which use the (old cpumask_t *) pointer have been deleted (f.e. set_cpus_allowed_ptr()).
Based on code from Rusty Russell <rusty-8n+1lVoiYb80n/[email protected]> (THANKS!!)
--- include/linux/cpumask.h | 432 +++++++++++++++++++----------------------- include/linux/cpumask_alloc.h | 58 +++-- lib/cpumask.c | 36 ++- 3 files changed, 258 insertions(+), 268 deletions(-)
include/linux/cpumask.h +203 -229
@@ -3,7 +3,8 @@/* * Cpumasks provide a bitmap suitable for representing the -* set of CPU's in a system, one bit position per CPU number. +* set of CPU's in a system, one bit position per CPU number up to +* nr_cpu_ids (<= NR_CPUS). * * See detailed comments in the file linux/bitmap.h describing the * data type on which these cpumasks are based.
@@ -18,18 +19,6 @@* For details of cpus_fold(), see bitmap_fold in lib/bitmap.c. * * . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . -* Note: The alternate operations with the suffix "_nr" are used -* to limit the range of the loop to nr_cpu_ids instead of -* NR_CPUS when NR_CPUS > 64 for performance reasons. -* If NR_CPUS is <= 64 then most assembler bitmask -* operators execute faster with a constant range, so -* the operator will continue to use NR_CPUS. -* -* Another consideration is that nr_cpu_ids is initialized -* to NR_CPUS and isn't lowered until the possible cpus are -* discovered (including any disabled cpus). So early uses -* will span the entire range of NR_CPUS. -* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . * * The available cpumask operations are: *
@@ -37,6 +26,7 @@* void cpu_clear(cpu, mask) turn off bit 'cpu' in mask * void cpus_setall(mask) set all bits * void cpus_clear(mask) clear all bits +* void cpus_copy(dst, src) copies cpumask bits from src to dst * int cpu_isset(cpu, mask) true iff bit 'cpu' set in mask * int cpu_test_and_set(cpu, mask) test and set bit 'cpu' in mask *
@@ -52,19 +42,21 @@* int cpus_empty(mask) Is mask empty (no bits sets)? * int cpus_full(mask) Is mask full (all bits sets)? * int cpus_weight(mask) Hamming weigh - number of set bits -* int cpus_weight_nr(mask) Same using nr_cpu_ids instead of NR_CPUS * * void cpus_shift_right(dst, src, n) Shift right * void cpus_shift_left(dst, src, n) Shift left * -* int first_cpu(mask) Number lowest set bit, or NR_CPUS -* int next_cpu(cpu, mask) Next cpu past 'cpu', or NR_CPUS -* int next_cpu_nr(cpu, mask) Next cpu past 'cpu', or nr_cpu_ids -* -* cpumask_t cpumask_of_cpu(cpu) Return cpumask with bit 'cpu' set -* (can be used as an lvalue) -* CPU_MASK_ALL Initializer - all bits set -* CPU_MASK_NONE Initializer - no bits set +* int cpus_first(mask) Number lowest set bit, or nr_cpu_ids +* int cpus_next(cpu, mask) Next cpu past 'cpu', or nr_cpu_ids +* int cpus_next_in(cpu, mask, andmask) Next cpu in mask & andmask or nr_cpu_ids +* +* const cpumask_t cpumask_of_cpu(cpu) Return pointer to cpumask with bit +* 'cpu' set +* +* cpu_mask_all cpumask_map_t of all bits set +* CPU_MASK_ALL Initializer only - all bits set +* CPU_MASK_NONE Initializer only - no bits set +* CPU_MASK_CPU0 Initializer only - cpu 0 bit set * unsigned long *cpus_addr(mask) Array of unsigned long's in mask * * int cpumask_scnprintf(buf, len, mask) Format cpumask for printing
@@ -76,8 +68,8 @@* void cpus_onto(dst, orig, relmap) *dst = orig relative to relmap * void cpus_fold(dst, orig, sz) dst bits = orig bits mod sz * -* for_each_cpu_mask(cpu, mask) for-loop cpu over mask using NR_CPUS -* for_each_cpu_mask_nr(cpu, mask) for-loop cpu over mask using nr_cpu_ids +* for_each_cpu(cpu, mask) for-loop cpu over mask +* for_each_cpu_in(cpu, mask, andmask) for-loop cpu over mask & andmask * * int num_online_cpus() Number of online CPUs * int num_possible_cpus() Number of all possible CPUs
@@ -87,6 +79,7 @@* int cpu_possible(cpu) Is some cpu possible? * int cpu_present(cpu) Is some cpu present (can schedule)? * +* int any_cpu_in(mask, andmask) First cpu in mask & andmask * int any_online_cpu(mask) First online cpu in mask * * for_each_possible_cpu(cpu) for-loop cpu over cpu_possible_map
@@ -107,131 +100,200 @@#include <linux/threads.h> #include <linux/bitmap.h> -typedef struct { DECLARE_BITMAP(bits, NR_CPUS); } cpumask_t; -extern cpumask_t _unused_cpumask_arg_; +#if NR_CPUS ==1 + +typedef (const unsigned long) *cpumask_t; +typedef unsigned long cpumask_map_t [1 ]; +typedef unsigned long cpumask_var_t [1 ]; + +#define nr_cpu_ids 1 +#define cpus_first (src) ({ (void)(src); 0; }) +#define cpus_next (n, src) ({ (void)(src); 1; }) +#define cpus_next_in (n, src, andsrc) ({ (void)(src); 1; }) +#define any_online_cpu (mask) 0 +#define for_each_cpu (cpu, mask) \ + for ((cpu) = 0; (cpu) < 1; (cpu)++, (void)mask) +#define for_each_cpu_in (cpu, mask, andmask) \ + for ((cpu) = 0; (cpu) < 1; (cpu)++, (void)mask, (void)andmask) + +#define num_online_cpus () 1 +#define num_possible_cpus () 1 +#define num_present_cpus () 1 +#define cpu_online (cpu) ((cpu) == 0) +#define cpu_possible (cpu) ((cpu) == 0) +#define cpu_present (cpu) ((cpu) == 0) +#define cpu_active (cpu) ((cpu) == 0) + +#else /* ... NR_CPUS > 1 */ + +#ifdef CONFIG_CPUMASKS_OFFSTACK + +/* Starts at NR_CPUS until acpi code discovers actual number. */ +extern int nr_cpu_ids; +typedef unsigned long *cpumask_var_t ; + +#else /* CPUMASKS "ONSTACK" */ + +/* Constant is usually more efficient than a variable for small NR_CPUS */ +#define nr_cpu_ids NR_CPUS +typedef unsigned long cpumask_var_t [BITS_TO_LONGS (nr_cpu_ids)]; + +#endif + +/* cpumask_t defaults to pointer to constant bit map */ +typedef const unsigned long *cpumask_t ; +typedef unsigned long cpumask_map_t [BITS_TO_LONGS (nr_cpu_ids)]; + +extern int cpus_first (cpumask_t src); +extern int cpus_next (int n,cpumask_t src); +extern int cpus_next_in (int n,cpumask_t src,cpumask_t andsrc); +extern int any_cpu_in (cpumask_t mask,cpumask_t andmask); + +#define any_online_cpu (mask) any_cpu_in((cpumask_t)(mask), \ + (cpumask_t)cpu_online_map) + +#define for_each_cpu (cpu, mask) \ + for ((cpu) = -1; \ + (cpu) = cpus_next((cpu), (mask)), \ + (cpu) < nr_cpu_ids; ) + +#define for_each_cpu_in (cpu, mask, andmask) \ + for ((cpu) = -1; \ + (cpu) = cpus_next_in((cpu), (mask), (andmask)), \ + (cpu) < nr_cpu_ids; ) + +#define num_online_cpus () cpus_weight(cpu_online_map) +#define num_possible_cpus () cpus_weight(cpu_possible_map) +#define num_present_cpus () cpus_weight(cpu_present_map) +#define cpu_online (cpu) cpu_isset((cpu), cpu_online_map) +#define cpu_possible (cpu) cpu_isset((cpu), cpu_possible_map) +#define cpu_present (cpu) cpu_isset((cpu), cpu_present_map) +#define cpu_active (cpu) cpu_isset((cpu), cpu_active_map) + +#endif /* NR_CPUS > 1 */ + +/* Deprecated: use for_each_cpu() */ +#define for_each_cpu_mask (cpu, mask) \ + for_each_cpu(cpu, mask) +/* +* XXX - how to add this to the above macro? +* #warning "for_each_cpu_mask is deprecated, use for_each_cpu" +*/ + +/* Deprecated: use cpus_first() */ +static inline int __deprecated first_cpu (cpumask_t src) +{ +return cpus_first (src); +} + +/* Deprecated: use cpus_next() */ +static inline int __deprecated next_cpu (int n,cpumask_t src) +{ +return cpus_next (n, src); +} + +static inline int cpumask_size (void ) +{ +return BITS_TO_LONGS (nr_cpu_ids) *sizeof (unsigned long ); +} -#define cpu_set (cpu, dst) __cpu_set((cpu), &(dst)) -static inline void __cpu_set (int cpu,volatile cpumask_t *dstp) +static inline void cpu_set (int cpu,volatile cpumask_var_t dst) { -set_bit (cpu, dstp->bits); +set_bit (cpu, dst); } -#define cpu_clear (cpu, dst) __cpu_clear((cpu), &(dst)) -static inline void __cpu_clear (int cpu,volatile cpumask_t *dstp) +static inline void cpu_clear (int cpu,volatile cpumask_var_t dst) { -clear_bit (cpu, dstp->bits); +clear_bit (cpu, dst); } -#define cpus_setall (dst) __cpus_setall(&(dst), NR_CPUS) -static inline void __cpus_setall (cpumask_t *dstp,int nbits) +static inline void cpus_setall (cpumask_var_t dst) { -bitmap_fill (dstp->bits, nbits); +bitmap_fill (dst, nr_cpu_ids); } -#define cpus_clear (dst) __cpus_clear(&(dst), NR_CPUS) -static inline void __cpus_clear (cpumask_t *dstp,int nbits) +static inline void cpus_clear (cpumask_var_t dst) { -bitmap_zero (dstp->bits, nbits); +bitmap_zero (dst, nr_cpu_ids); +} + +static inline void cpus_copy (cpumask_var_t dst,cpumask_t src) +{ +bitmap_copy (dst, src, nr_cpu_ids); }/* No static inline type checking - see Subtlety (1) above. */ -#define cpu_isset (cpu, cpumask) test_bit((cpu), (cpumask).bits) +#define cpu_isset (cpu, cpumask) test_bit((cpu), (cpumask)) -#define cpu_test_and_set (cpu, cpumask) __cpu_test_and_set((cpu), &(cpumask)) -static inline int __cpu_test_and_set (int cpu,cpumask_t *addr) +static inline int cpu_test_and_set (int cpu,cpumask_var_t addr) { -return test_and_set_bit (cpu, addr->bits); +return test_and_set_bit (cpu, addr); } -#define cpus_and (dst, src1, src2) __cpus_and(&(dst), &(src1), &(src2), NR_CPUS) -static inline void __cpus_and (cpumask_t *dstp,const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) +static inline void cpus_and (cpumask_var_t dst,cpumask_t src1,cpumask_t src2) { -bitmap_and (dstp->bits, src1p->bits, src2p->bits, nbits); +bitmap_and (dst, src1, src2, nr_cpu_ids); } -#define cpus_or (dst, src1, src2) __cpus_or(&(dst), &(src1), &(src2), NR_CPUS) -static inline void __cpus_or (cpumask_t *dstp,const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) +static inline void cpus_or (cpumask_var_t dst,cpumask_t src1,cpumask_t src2) { -bitmap_or (dstp->bits, src1p->bits, src2p->bits, nbits); +bitmap_or (dst, src1, src2, nr_cpu_ids); } -#define cpus_xor (dst, src1, src2) __cpus_xor(&(dst), &(src1), &(src2), NR_CPUS) -static inline void __cpus_xor (cpumask_t *dstp,const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) +static inline void cpus_xor (cpumask_var_t dst,cpumask_t src1,cpumask_t src2) { -bitmap_xor (dstp->bits, src1p->bits, src2p->bits, nbits); +bitmap_xor (dst, src1, src2, nr_cpu_ids); } -#define cpus_andnot (dst, src1, src2) \ - __cpus_andnot(&(dst), &(src1), &(src2), NR_CPUS) -static inline void __cpus_andnot (cpumask_t *dstp,const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) +static inline void cpus_andnot (cpumask_var_t dst,cpumask_t src1, +cpumask_t src2) { -bitmap_andnot (dstp->bits, src1p->bits, src2p->bits, nbits); +bitmap_andnot (dst, src1, src2, nr_cpu_ids); } -#define cpus_complement (dst, src) __cpus_complement(&(dst), &(src), NR_CPUS) -static inline void __cpus_complement (cpumask_t *dstp, -const cpumask_t *srcp,int nbits) +static inline void cpus_complement (cpumask_var_t dst,cpumask_t src) { -bitmap_complement (dstp->bits, srcp->bits, nbits); +bitmap_complement (dst, src, nr_cpu_ids); } -#define cpus_equal (src1, src2) __cpus_equal(&(src1), &(src2), NR_CPUS) -static inline int __cpus_equal (const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) +static inline int cpus_equal (cpumask_t src1,cpumask_t src2) { -return bitmap_equal (src1p->bits, src2p->bits, nbits); +return bitmap_equal (src1, src2, nr_cpu_ids); } -#define cpus_intersects (src1, src2) __cpus_intersects(&(src1), &(src2), NR_CPUS) -static inline int __cpus_intersects (const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) +static inline int cpus_intersects (cpumask_t src1,cpumask_t src2) { -return bitmap_intersects (src1p->bits, src2p->bits, nbits); +return bitmap_intersects (src1, src2, nr_cpu_ids); } -#define cpus_subset (src1, src2) __cpus_subset(&(src1), &(src2), NR_CPUS) -static inline int __cpus_subset (const cpumask_t *src1p, -const cpumask_t *src2p,int nbits) +static inline int cpus_subset (cpumask_t src1,cpumask_t src2) { -return bitmap_subset (src1p->bits, src2p->bits, nbits); +return bitmap_subset (src1, src2, nr_cpu_ids); } -#define cpus_empty (src) __cpus_empty(&(src), NR_CPUS) -static inline int __cpus_empty (const cpumask_t *srcp,int nbits) +static inline int cpus_empty (cpumask_t src) { -return bitmap_empty (srcp->bits, nbits); +return bitmap_empty (src, nr_cpu_ids); } -#define cpus_full (cpumask) __cpus_full(&(cpumask), NR_CPUS) -static inline int __cpus_full (const cpumask_t *srcp,int nbits) +static inline int cpus_full (cpumask_t src) { -return bitmap_full (srcp->bits, nbits); +return bitmap_full (src, nr_cpu_ids); } -#define cpus_weight (cpumask) __cpus_weight(&(cpumask), NR_CPUS) -static inline int __cpus_weight (const cpumask_t *srcp,int nbits) +static inline int cpus_weight (cpumask_t src) { -return bitmap_weight (srcp->bits, nbits); +return bitmap_weight (src, nr_cpu_ids); } -#define cpus_shift_right (dst, src, n) \ - __cpus_shift_right(&(dst), &(src), (n), NR_CPUS) -static inline void __cpus_shift_right (cpumask_t *dstp, -const cpumask_t *srcp,int n,int nbits) +static inline void cpus_shift_right (cpumask_var_t dst,cpumask_t src,int n) { -bitmap_shift_right (dstp->bits, srcp->bits, n, nbits); +bitmap_shift_right (dst, src, n, nr_cpu_ids); } -#define cpus_shift_left (dst, src, n) \ - __cpus_shift_left(&(dst), &(src), (n), NR_CPUS) -static inline void __cpus_shift_left (cpumask_t *dstp, -const cpumask_t *srcp,int n,int nbits) +static inline void cpus_shift_left (cpumask_var_t dst,cpumask_t src,int n) { -bitmap_shift_left (dstp->bits, srcp->bits, n, nbits); +bitmap_shift_left (dst, src, n, nr_cpu_ids); }/*
@@ -244,11 +306,11 @@ static inline void __cpus_shift_left(cpuextern const unsigned long cpu_bit_bitmap[BITS_PER_LONG +1 ][BITS_TO_LONGS (NR_CPUS )]; -static inline const cpumask_t *get_cpu_mask (unsigned int cpu) +static inline cpumask_t get_cpu_mask (unsigned int cpu) {const unsigned long *p = cpu_bit_bitmap[1 + cpu %BITS_PER_LONG ]; p -= cpu /BITS_PER_LONG ; -return (const cpumask_t *)p; +return (cpumask_t )p; }/*
@@ -256,150 +318,79 @@ static inline const cpumask_t *get_cpu_m* gcc optimizes it out (it's a constant) and there's no huge stack * variable created: */ -#define cpumask_of_cpu (cpu) (*get_cpu_mask(cpu)) - +#define cpumask_of_cpu (cpu) ((cpumask_t)get_cpu_mask(cpu))#define CPU_MASK_LAST_WORD BITMAP_LAST_WORD_MASK(NR_CPUS) +#define CPU_MASK_INIT (value) { \ + [0 ... BITS_TO_LONGS(NR_CPUS)-1] = value \ +}#if NR_CPUS <=BITS_PER_LONG -#define CPU_MASK_ALL \ -(cpumask_t) { { \ - [BITS_TO_LONGS(NR_CPUS)-1] = CPU_MASK_LAST_WORD \ -} } - -#define CPU_MASK_ALL_PTR (&CPU_MASK_ALL) +/* Initializer only, use cpu_mask_all in function arguments */ +#define CPU_MASK_ALL CPU_MASK_INIT(CPU_MASK_LAST_WORD)#else -#define CPU_MASK_ALL \ -(cpumask_t) { { \ - [0 ... BITS_TO_LONGS(NR_CPUS)-2] = ~0UL, \ - [BITS_TO_LONGS(NR_CPUS)-1] = CPU_MASK_LAST_WORD \ -} } - -/* cpu_mask_all is in init/main.c */ -extern cpumask_t cpu_mask_all; -#define CPU_MASK_ALL_PTR (&cpu_mask_all) +/* Initializer only, use cpu_mask_all in function arguments */ +#define CPU_MASK_ALL { \ + [0 ... BITS_TO_LONGS(NR_CPUS)-2] = ~0UL, \ + [BITS_TO_LONGS(NR_CPUS)-1] = CPU_MASK_LAST_WORD \ +}#endif -#define CPU_MASK_NONE \ -(cpumask_t) { { \ - [0 ... BITS_TO_LONGS(NR_CPUS)-1] = 0UL \ -} } - -#define CPU_MASK_CPU0 \ -(cpumask_t) { { \ - [0] = 1UL \ -} } +/* Initializers only */ +#define CPU_MASK_NONE CPU_MASK_INIT(0UL) +#define CPU_MASK_CPU0 CPU_MASK_INIT(1UL) -#define cpus_addr (src) ((src).bits) - -#define cpumask_scnprintf (buf, len, src) \ - __cpumask_scnprintf((buf), (len), &(src), NR_CPUS) -static inline int __cpumask_scnprintf (char *buf,int len, -const cpumask_t *srcp,int nbits) +static inline unsigned long *cpus_addr (cpumask_t src) { -return bitmap_scnprintf (buf, len, srcp->bits, nbits); +return (unsigned long *)src; } -#define cpumask_parse_user (ubuf, ulen, dst) \ - __cpumask_parse_user((ubuf), (ulen), &(dst), NR_CPUS) -static inline int __cpumask_parse_user (const char __user *buf,int len, -cpumask_t *dstp,int nbits) +static inline int cpumask_scnprintf (char *buf,int len,cpumask_t src) { -return bitmap_parse_user (buf, len, dstp->bits, nbits); +return bitmap_scnprintf (buf, len, src, nr_cpu_ids); } -#define cpulist_scnprintf (buf, len, src) \ - __cpulist_scnprintf((buf), (len), &(src), NR_CPUS) -static inline int __cpulist_scnprintf (char *buf,int len, -const cpumask_t *srcp,int nbits) +static inline int cpumask_parse_user (const char __user *buf,int len, +cpumask_var_t dst) { -return bitmap_scnlistprintf (buf, len, srcp->bits, nbits); +return bitmap_parse_user (buf, len, dst, nr_cpu_ids); } -#define cpulist_parse (buf, dst) __cpulist_parse((buf), &(dst), NR_CPUS) -static inline int __cpulist_parse (const char *buf,cpumask_t *dstp,int nbits) +static inline int cpulist_scnprintf (char *buf,int len,cpumask_t src) { -return bitmap_parselist (buf, dstp->bits, nbits); +return bitmap_scnlistprintf (buf, len, src, nr_cpu_ids); } -#define cpu_remap (oldbit, old, new) \ - __cpu_remap((oldbit), &(old), &(new), NR_CPUS) -static inline int __cpu_remap (int oldbit, -const cpumask_t *oldp,const cpumask_t *newp,int nbits) +static inline int cpulist_parse (const char *buf,cpumask_var_t dst) { -return bitmap_bitremap (oldbit, oldp->bits, newp->bits, nbits); +return bitmap_parselist (buf, dst, nr_cpu_ids); } -#define cpus_remap (dst, src, old, new) \ - __cpus_remap(&(dst), &(src), &(old), &(new), NR_CPUS) -static inline void __cpus_remap (cpumask_t *dstp,const cpumask_t *srcp, -const cpumask_t *oldp,const cpumask_t *newp,int nbits) +static inline int cpu_remap (int oldbit,cpumask_t old,cpumask_t new) { -bitmap_remap (dstp->bits, srcp->bits, oldp->bits, newp->bits, nbits); +return bitmap_bitremap (oldbit, old, new, nr_cpu_ids); } -#define cpus_onto (dst, orig, relmap) \ - __cpus_onto(&(dst), &(orig), &(relmap), NR_CPUS) -static inline void __cpus_onto (cpumask_t *dstp,const cpumask_t *origp, -const cpumask_t *relmapp,int nbits) +static inline void cpus_remap (cpumask_var_t dst,cpumask_t src,cpumask_t old, +cpumask_t new) { -bitmap_onto (dstp->bits, origp->bits, relmapp->bits, nbits); +bitmap_remap (dst, src, old, new, nr_cpu_ids); } -#define cpus_fold (dst, orig, sz) \ - __cpus_fold(&(dst), &(orig), sz, NR_CPUS) -static inline void __cpus_fold (cpumask_t *dstp,const cpumask_t *origp, -int sz,int nbits) +static inline void cpus_onto (cpumask_var_t dst,cpumask_t orig, +cpumask_t relmap) { -bitmap_fold (dstp->bits, origp->bits, sz, nbits); +bitmap_onto (dst, orig, relmap, nr_cpu_ids); } -#if NR_CPUS ==1 - -#define nr_cpu_ids 1 -#define first_cpu (src) ({ (void)(src); 0; }) -#define next_cpu (n, src) ({ (void)(src); 1; }) -#define any_online_cpu (mask) 0 -#define for_each_cpu_mask (cpu, mask) \ - for ((cpu) = 0; (cpu) < 1; (cpu)++, (void)mask) - -#else /* NR_CPUS > 1 */ - -extern int nr_cpu_ids; -int __first_cpu (const cpumask_t *srcp); -int __next_cpu (int n,const cpumask_t *srcp); -int __any_online_cpu (const cpumask_t *mask); - -#define first_cpu (src) __first_cpu(&(src)) -#define next_cpu (n, src) __next_cpu((n), &(src)) -#define any_online_cpu (mask) __any_online_cpu(&(mask)) -#define for_each_cpu_mask (cpu, mask) \ - for ((cpu) = -1; \ - (cpu) = next_cpu((cpu), (mask)), \ - (cpu) < NR_CPUS; ) -#endif - -#if NR_CPUS <=64 - -#define next_cpu_nr (n, src) next_cpu(n, src) -#define cpus_weight_nr (cpumask) cpus_weight(cpumask) -#define for_each_cpu_mask_nr (cpu, mask) for_each_cpu_mask(cpu, mask) - -#else /* NR_CPUS > 64 */ - -int __next_cpu_nr (int n,const cpumask_t *srcp); -#define next_cpu_nr (n, src) __next_cpu_nr((n), &(src)) -#define cpus_weight_nr (cpumask) __cpus_weight(&(cpumask), nr_cpu_ids) -#define for_each_cpu_mask_nr (cpu, mask) \ - for ((cpu) = -1; \ - (cpu) = next_cpu_nr((cpu), (mask)), \ - (cpu) < nr_cpu_ids; ) - -#endif /* NR_CPUS > 64 */ +static inline void cpus_fold (cpumask_var_t dst,cpumask_t orig,int sz) +{ +bitmap_fold (dst, orig, sz, nr_cpu_ids); +}/* * The following particular system cpumasks and operations manage
@@ -458,33 +449,16 @@ int __next_cpu_nr(int n, const cpumask_t* main(){ set1(3); set2(5); } */ -extern cpumask_t cpu_possible_map; -extern cpumask_t cpu_online_map; -extern cpumask_t cpu_present_map; -extern cpumask_t cpu_active_map; - -#if NR_CPUS >1 -#define num_online_cpus () cpus_weight_nr(cpu_online_map) -#define num_possible_cpus () cpus_weight_nr(cpu_possible_map) -#define num_present_cpus () cpus_weight_nr(cpu_present_map) -#define cpu_online (cpu) cpu_isset((cpu), cpu_online_map) -#define cpu_possible (cpu) cpu_isset((cpu), cpu_possible_map) -#define cpu_present (cpu) cpu_isset((cpu), cpu_present_map) -#define cpu_active (cpu) cpu_isset((cpu), cpu_active_map) -#else -#define num_online_cpus () 1 -#define num_possible_cpus () 1 -#define num_present_cpus () 1 -#define cpu_online (cpu) ((cpu) == 0) -#define cpu_possible (cpu) ((cpu) == 0) -#define cpu_present (cpu) ((cpu) == 0) -#define cpu_active (cpu) ((cpu) == 0) -#endif +extern cpumask_map_t cpu_possible_map; +extern cpumask_map_t cpu_online_map; +extern cpumask_map_t cpu_present_map; +extern cpumask_map_t cpu_active_map; +extern cpumask_map_t cpu_mask_all;#define cpu_is_offline (cpu) unlikely(!cpu_online(cpu)) -#define for_each_possible_cpu (cpu) for_each_cpu_mask_nr((cpu), cpu_possible_map) -#define for_each_online_cpu (cpu) for_each_cpu_mask_nr((cpu), cpu_online_map) -#define for_each_present_cpu (cpu) for_each_cpu_mask_nr((cpu), cpu_present_map) +#define for_each_possible_cpu (cpu) for_each_cpu((cpu), cpu_possible_map) +#define for_each_online_cpu (cpu) for_each_cpu((cpu), cpu_online_map) +#define for_each_present_cpu (cpu) for_each_cpu((cpu), cpu_present_map)#endif /* __LINUX_CPUMASK_H */
lib/cpumask.c +21 -15
@@ -3,34 +3,40 @@#include <linux/cpumask.h> #include <linux/module.h> -int __first_cpu (const cpumask_t *srcp) +int cpus_first (cpumask_t srcp) { -return find_first_bit (srcp->bits,NR_CPUS ); +return find_first_bit (srcp, nr_cpu_ids); } -EXPORT_SYMBOL (__first_cpu); +EXPORT_SYMBOL (cpus_first); -int __next_cpu (int n,const cpumask_t *srcp) +int cpus_next (int n,cpumask_t srcp) { -return find_next_bit (srcp->bits,NR_CPUS , n+1 ); +return find_next_bit (srcp, nr_cpu_ids, n+1 ); } -EXPORT_SYMBOL (__next_cpu); +EXPORT_SYMBOL (cpus_next); -#if NR_CPUS >64 -int __next_cpu_nr (int n,const cpumask_t *srcp) +int cpus_next_in (int n,cpumask_t srcp,cpumask_t andp) { -return find_next_bit (srcp->bits, nr_cpu_ids, n+1 ); +int cpu; + +for (cpu = n +1 ; cpu < nr_cpu_ids; cpu++) { + cpu =find_next_bit (srcp, nr_cpu_ids, cpu); + +if (cpu < nr_cpu_ids &&cpu_isset (cpu, andp)) +return cpu; + } +return nr_cpu_ids; } -EXPORT_SYMBOL (__next_cpu_nr); -#endif +EXPORT_SYMBOL (cpus_next_in); -int __any_online_cpu (const cpumask_t *mask) +int any_cpu_in (cpumask_t mask,cpumask_t andmask) {int cpu; -for_each_cpu_mask (cpu, *mask) { -if (cpu_online (cpu)) +for_each_cpu (cpu, mask) { +if (cpu_isset (cpu, andmask))break ; }return cpu; } -EXPORT_SYMBOL (__any_online_cpu); +EXPORT_SYMBOL (any_cpu_in);
Re: [RFC 1/1] cpumask: New cpumask API - take 2 - use unsigned longs
Quoted 1 line · On Wednesday 01 October 2008 08:22:09 Mike Travis wrote:
Here's an alternate proposal for the new cpumask API. I have not yet begun
...
include/linux/cpumask.h line 144
Quoted 2 lines
include/linux/cpumask.h +2
-0+/* cpumask_t defaults to pointer to constant bit map */ +typedef const unsigned long *cpumask_t ;
Hiding a const here is not a good idea either, I think :(
Rusty.
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Linus TorvaldsQuoted 2 lines · On Wednesday 01 October 2008 02:46:59 Linus Torvalds wrote:
Quite frankly, I personally do hate typedefs that end up being pointers, and used as pointers, without showing that in the source code.
...
Quoted 3 lines
I'm now a bit more leery about this whole thing just because the typedef ends up hiding so much - it doesn't just hide the basic type, it hides a very basic *code* issue.
Yes, this is why my version of the rework moved away from typedefs, except for the special case of "cpumask_var_t" for stack vars where this trick is really desired.
Everywhere else, the code becomes nice and clear: struct cpumask *.
Cheers,
Rusty.
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Ingo MolnarQuoted 33 lines · Ingo Molnar
Quoted 10 lines · David Miller
From: Ingo Molnar <[email protected]>
Date: Tue, 26 Aug 2008 09:22:20 +0200Quoted 4 lines
And i guess the next generation of 4K CPUs support should just get away from cpumask_t-on-kernel-stack model altogether, as the current model is not maintainable. We tried the on-kernel-stack variant, and it really does not work reliably. We can fix this in v2.6.28.
I recenetly did some work on sparc64 to use cpumask pointers as much as possible.
Quoted 7 lines
The only case that didn't work was due to a limitation in arch interfaces for the new generic
smp_call_function()code. It passes acpumask_tinstead of a pointer to one viaarch_send_call_function_ipi().But other than that, the whole sparc64 SMP stuff uses
cpumask_tpointers only.nice!
Quoted 6 lines
What it comes down to is that you have to do the "self cpu" and other tests in the cross-call dispatch routines themselves, instead of at the top-level working on
cpumask_tobjects.Otherwise you have to modify
cpumask_tobjects and thus pluck them onto the stack where they take up silly amounts of space.What we did was this: we added MAXSMP which just revs up all the SMP tunables to the maximum, so that we can see any problems early in testing.
Quoted 5 lines
And we triggered problems, and we fixed a couple of regressions all around stack footprint. But we didnt catch all of them - some were gcc version dependent and configuration dependent. So i think it's safe to say that the whole concept of allowing such a large
cpumask_tto be on the stack is fragile.
Iirc, it was the problem of basing percpu variables at zero that hit problems with various gcc toolset versions. I don't remember any version problems with cpumask's on the stack, they all failed the same way... :-)
Quoted 5 lines
Hence, i think the best way forward is to change the whole
cpumask_tconcept and disallow explicit masks altogether. It's so easy to smack acpumask_tvariable on the stack and nothing really warns about it, and any function can become part of a nested call sequence.
This is a great idea!
Quoted 7 lines
So i think the dynamics of it has to be changed: we need a get/put API and we need to make on-stack cpumask illegal on the build level (in generic code at least). This has been Rusty's main argument early on i think, and i now concur.
Ingo
Removing cpumask_t's from the stack is fairly straight forward. The problem of changing all functions to expect a cpumask pointer via a global change is much more problematic. And of course all those functions that return a cpumask value would need to be addressed.
Thanks,
Mike
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to David MillerQuoted 19 lines · David Miller
From: Ingo Molnar <[email protected]>
Date: Tue, 26 Aug 2008 09:22:20 +0200Quoted 4 lines
And i guess the next generation of 4K CPUs support should just get away from cpumask_t-on-kernel-stack model altogether, as the current model is not maintainable. We tried the on-kernel-stack variant, and it really does not work reliably. We can fix this in v2.6.28.
I recently did some work on sparc64 to use cpumask pointers as much as possible.
The only case that didn't work was due to a limitation in arch interfaces for the new generic
smp_call_function()code. It passes acpumask_tinstead of a pointer to one viaarch_send_call_function_ipi().But other than that, the whole sparc64 SMP stuff uses
cpumask_tpointers only.
Quoted 6 lines
What it comes down to is that you have to do the "self cpu" and other tests in the cross-call dispatch routines themselves, instead of at the top-level working on
cpumask_tobjects.Otherwise you have to modify
cpumask_tobjects and thus pluck them onto the stack where they take up silly amounts of space.
Yes, I had proposed either modifying, or supplementing a new smp_call function to pass the cpumask_t as a pointer (similar to set_cpus_allowed_ptr.) But an ABI change such as this was not well received at the time.
Thanks,
Mike
David Miller
maintainerRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
From: Mike Travis <travis-sJ/[email protected]>
Date: Tue, 26 Aug 2008 12:06:18 -0700
Quoted 13 lines
Quoted 12 lines · David Miller
The only case that didn't work was due to a limitation in arch interfaces for the new generic
smp_call_function()code. It passes acpumask_tinstead of a pointer to one viaarch_send_call_function_ipi().But other than that, the whole sparc64 SMP stuff uses
cpumask_tpointers only.What it comes down to is that you have to do the "self cpu" and other tests in the cross-call dispatch routines themselves, instead of at the top-level working on
cpumask_tobjects.
Quoted 7 lines
Quoted 2 lines
Otherwise you have to modify
cpumask_tobjects and thus pluck them onto the stack where they take up silly amounts of space.Yes, I had proposed either modifying, or supplementing a new
smp_callfunction to pass thecpumask_tas a pointer (similar toset_cpus_allowed_ptr.) But an ABI change such as this was not well received at the time.
What it seems to come down to is that any cpumask_t not inside of a dynamically allocated object should be marked const.
And that is something we can enforce at compile time.
Linus has just suggested dynamically allocating cpumask_t's for such cases but I don't see that as the fix either.
Just mark them const and enforce that cpumask_t objects can only be modified when they appear in dynamically allocated objects.
You really don't need to modify the ones that passed around functions anyways. The only code that wants to change bits in these things is the cpu cross-call dispatch stuff, and that cpu choice logic can just live where it belongs down in the cross-call dispatch code.
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 10 lines · David Miller
Quoted 6 lines
Quoted 2 lines
Otherwise you have to modify
cpumask_tobjects and thus pluck them onto the stack where they take up silly amounts of space.Yes, I had proposed either modifying, or supplementing a new
smp_callfunction to pass thecpumask_tas a pointer (similar toset_cpus_allowed_ptr.) But an ABI change such as this was not well received at the time.What it seems to come down to is that any
cpumask_tnot inside of a dynamically allocated object should be marked const.
Quoted 7 lines
And that is something we can enforce at compile time.
Linus has just suggested dynamically allocating cpumask_t's for such cases but I don't see that as the fix either.
Just mark them const and enforce that
cpumask_tobjects can only be modified when they appear in dynamically allocated objects.
Dave and others,
Sorry if I jump into the middle of the thread. Stopped subscribing to lkml for a while, so this is through the archives.
Ran into some of these issues with KVM too, and noticed just how much we pass cpumask_t around in the smp_call functions :-( In fact, the only arch that did pretty well on this front was sparc64.
I totally agree, that marking them const makes a ton of sense, but at the same time I suggest we convert smp_call_function_mask() to take a pointer to the cpumask_t. I whipped up the following patch, which cuts down the amont of memcpy calls emitted quite a fair bit.
I have only tested this on ia64, but it boots, so it's obviously perfect<tm> :-)
Comments, suggestions welcome.
I have a followup patch that makes virt/kvm/kvm_main.c use the new interface.
Cheers,
Jes
Change smp_call_function_mask() to take a pointer to the cpumask_t rather than passing it by value. This avoids recursive copies of the cpumask_t on the stack in the IPI call. For large NR_CPUS, this is particularly bad, and the cost of doing this forNR_CPUS < bits_per_long is negligeble.
--- arch/alpha/include/asm/smp.h | 2 +- arch/alpha/kernel/smp.c | 4 ++-- arch/arm/include/asm/smp.h | 2 +- arch/arm/kernel/smp.c | 4 ++-- arch/ia64/include/asm/smp.h | 2 +- arch/ia64/kernel/smp.c | 6 +++--- arch/m32r/kernel/smp.c | 4 ++-- arch/mips/kernel/smp.c | 4 ++-- arch/parisc/kernel/smp.c | 6 +++--- arch/powerpc/include/asm/smp.h | 2 +- arch/powerpc/kernel/smp.c | 4 ++-- arch/sh/include/asm/smp.h | 2 +- arch/sh/kernel/smp.c | 4 ++-- arch/sparc/include/asm/smp_64.h | 2 +- arch/sparc64/kernel/smp.c | 4 ++-- include/asm-m32r/smp.h | 2 +- include/asm-mips/smp.h | 2 +- include/asm-parisc/smp.h | 2 +- include/asm-x86/smp.h | 4 ++-- include/linux/smp.h | 2 +- kernel/smp.c | 15 ++++++++------- virt/kvm/kvm_main.c | 4 ++-- 22 files changed, 42 insertions(+), 41 deletions(-)
arch/alpha/include/asm/smp.h +1 -1
@@ -48,7 +48,7 @@#define cpu_possible_map cpu_present_mapextern void arch_send_call_function_single_ipi (int cpu); -extern void arch_send_call_function_ipi (cpumask_t mask); +extern void arch_send_call_function_ipi (cpumask_t *mask);#else /* CONFIG_SMP */
arch/alpha/kernel/smp.c +2 -2
@@ -637,9 +637,9 @@send_ipi_message (to_whom,IPI_CPU_STOP ); } -void arch_send_call_function_ipi (cpumask_t mask) +void arch_send_call_function_ipi (cpumask_t *mask) { -send_ipi_message (mask,IPI_CALL_FUNC ); +send_ipi_message (*mask,IPI_CALL_FUNC ); }void arch_send_call_function_single_ipi (int cpu)
arch/arm/include/asm/smp.h +1 -1
@@ -102,7 +102,7 @@extern void platform_cpu_enable (unsigned int cpu);extern void arch_send_call_function_single_ipi (int cpu); -extern void arch_send_call_function_ipi (cpumask_t mask); +extern void arch_send_call_function_ipi (cpumask_t *mask);/* * Local timer interrupt handling function (can be IPI'ed).
arch/arm/kernel/smp.c +2 -2
@@ -356,9 +356,9 @@local_irq_restore (flags); } -void arch_send_call_function_ipi (cpumask_t mask) +void arch_send_call_function_ipi (cpumask_t *mask) { -send_ipi_message (mask,IPI_CALL_FUNC ); +send_ipi_message (*mask,IPI_CALL_FUNC ); }void arch_send_call_function_single_ipi (int cpu)
arch/ia64/include/asm/smp.h +1 -1
@@ -127,7 +127,7 @@extern int is_multithreading_enabled (void );extern void arch_send_call_function_single_ipi (int cpu); -extern void arch_send_call_function_ipi (cpumask_t mask); +extern void arch_send_call_function_ipi (cpumask_t *mask);#else /* CONFIG_SMP */
arch/ia64/kernel/smp.c +3 -3
@@ -166,11 +166,11 @@* Called with preemption disabled. */ static inline void -send_IPI_mask (cpumask_t mask,int op) +send_IPI_mask (cpumask_t *mask,int op) {unsigned int cpu; -for_each_cpu_mask (cpu, mask) { +for_each_cpu_mask (cpu, *mask) {send_IPI_single (cpu, op); } }
@@ -316,7 +316,7 @@send_IPI_single (cpu,IPI_CALL_FUNC_SINGLE ); } -void arch_send_call_function_ipi (cpumask_t mask) +void arch_send_call_function_ipi (cpumask_t *mask) {send_IPI_mask (mask,IPI_CALL_FUNC ); }
arch/m32r/kernel/smp.c +2 -2
@@ -546,9 +546,9 @@for ( ; ; ); } -void arch_send_call_function_ipi (cpumask_t mask) +void arch_send_call_function_ipi (cpumask_t *mask) { -send_IPI_mask (mask,CALL_FUNCTION_IPI ,0 ); +send_IPI_mask (*mask,CALL_FUNCTION_IPI ,0 ); }void arch_send_call_function_single_ipi (int cpu)
arch/mips/kernel/smp.c +2 -2
@@ -131,9 +131,9 @@cpu_idle (); } -void arch_send_call_function_ipi (cpumask_t mask) +void arch_send_call_function_ipi (cpumask_t *mask) { - mp_ops->send_ipi_mask (mask,SMP_CALL_FUNCTION ); + mp_ops->send_ipi_mask (*mask,SMP_CALL_FUNCTION ); }/*
arch/parisc/kernel/smp.c +3 -3
@@ -228,11 +228,11 @@ }static void -send_IPI_mask (cpumask_t mask,enum ipi_message_type op) +send_IPI_mask (cpumask_t *mask,enum ipi_message_type op) {int cpu; -for_each_cpu_mask (cpu, mask) +for_each_cpu_mask (cpu, *mask)ipi_send (cpu, op); }
@@ -274,7 +274,7 @@send_IPI_allbutself (IPI_NOP ); } -void arch_send_call_function_ipi (cpumask_t mask) +void arch_send_call_function_ipi (cpumask_t *mask) {send_IPI_mask (mask,IPI_CALL_FUNC ); }
arch/powerpc/include/asm/smp.h +1 -1
@@ -119,7 +119,7 @@extern struct smp_ops_t *smp_ops;extern void arch_send_call_function_single_ipi (int cpu); -extern void arch_send_call_function_ipi (cpumask_t mask); +extern void arch_send_call_function_ipi (cpumask_t *mask);#endif /* __ASSEMBLY__ */
arch/powerpc/kernel/smp.c +2 -2
@@ -135,11 +135,11 @@ smp_ops->message_pass (cpu,PPC_MSG_CALL_FUNC_SINGLE ); } -void arch_send_call_function_ipi (cpumask_t mask) +void arch_send_call_function_ipi (cpumask_t *mask) {unsigned int cpu; -for_each_cpu_mask (cpu, mask) +for_each_cpu_mask (cpu, *mask) smp_ops->message_pass (cpu,PPC_MSG_CALL_FUNCTION ); }
arch/sh/include/asm/smp.h +1 -1
@@ -39,7 +39,7 @@int plat_register_ipi_handler (unsigned int message,void (*handler)(void *),void *arg);extern void arch_send_call_function_single_ipi (int cpu); -extern void arch_send_call_function_ipi (cpumask_t mask); +extern void arch_send_call_function_ipi (cpumask_t *mask);#else
arch/sh/kernel/smp.c +2 -2
@@ -171,11 +171,11 @@smp_call_function (stop_this_cpu,0 ,0 ); } -void arch_send_call_function_ipi (cpumask_t mask) +void arch_send_call_function_ipi (cpumask_t *mask) {int cpu; -for_each_cpu_mask (cpu, mask) +for_each_cpu_mask (cpu, *mask)plat_send_ipi (cpu,SMP_MSG_FUNCTION ); }
arch/sparc/include/asm/smp_64.h +1 -1
@@ -35,7 +35,7 @@extern int sparc64_multi_core;extern void arch_send_call_function_single_ipi (int cpu); -extern void arch_send_call_function_ipi (cpumask_t mask); +extern void arch_send_call_function_ipi (cpumask_t *mask);/* * General functions that each host system must provide.
arch/sparc64/kernel/smp.c +2 -2
@@ -810,9 +810,9 @@extern unsigned long xcall_call_function; -void arch_send_call_function_ipi (cpumask_t mask) +void arch_send_call_function_ipi (cpumask_t *mask) { -xcall_deliver ((u64) &xcall_call_function,0 ,0 , &mask); +xcall_deliver ((u64) &xcall_call_function,0 ,0 , mask); }extern unsigned long xcall_call_function_single;
include/asm-m32r/smp.h +1 -1
@@ -90,7 +90,7 @@extern unsigned long send_IPI_mask_phys (cpumask_t ,int ,int );extern void arch_send_call_function_single_ipi (int cpu); -extern void arch_send_call_function_ipi (cpumask_t mask); +extern void arch_send_call_function_ipi (cpumask_t *mask);#endif /* not __ASSEMBLY__ */
include/asm-mips/smp.h +1 -1
@@ -58,6 +58,6 @@ extern asmlinkage void smp_call_function_interrupt(void);extern void arch_send_call_function_single_ipi (int cpu); -extern void arch_send_call_function_ipi (cpumask_t mask); +extern void arch_send_call_function_ipi (cpumask_t *mask);#endif /* __ASM_SMP_H */
include/asm-parisc/smp.h +1 -1
@@ -31,7 +31,7 @@extern void smp_send_all_nop (void );extern void arch_send_call_function_single_ipi (int cpu); -extern void arch_send_call_function_ipi (cpumask_t mask); +extern void arch_send_call_function_ipi (cpumask_t *mask);#endif /* !ASSEMBLY */
include/asm-x86/smp.h +2 -2
@@ -101,9 +101,9 @@ smp_ops.send_call_func_single_ipi (cpu); } -static inline void arch_send_call_function_ipi (cpumask_t mask) +static inline void arch_send_call_function_ipi (cpumask_t *mask) { - smp_ops.send_call_func_ipi (mask); + smp_ops.send_call_func_ipi (*mask); }void native_smp_prepare_boot_cpu (void );
include/linux/smp.h +1 -1
@@ -62,7 +62,7 @@* Call a function on all other processors */ int smp_call_function (void (*func)(void *info),void *info,int wait); -int smp_call_function_mask (cpumask_t mask,void (*func)(void *info),void *info, +int smp_call_function_mask (cpumask_t *mask,void (*func)(void *info),void *info,int wait);int smp_call_function_single (int cpuid,void (*func) (void *info),void *info,int wait);
kernel/smp.c +8 -7
@@ -318,7 +318,7 @@* hardware interrupt handler or from a bottom half handler. Preemption * must be disabled when calling this function. */ -int smp_call_function_mask (cpumask_t mask,void (*func)(void *),void *info, +int smp_call_function_mask (cpumask_t *mask,void (*func)(void *),void *info,int wait) {struct call_function_data d;
@@ -334,8 +334,8 @@ cpu =smp_processor_id (); allbutself = cpu_online_map;cpu_clear (cpu, allbutself); -cpus_and (mask, mask, allbutself); - num_cpus =cpus_weight (mask); +cpus_and (*mask, *mask, allbutself); + num_cpus =cpus_weight (*mask);/* * If zero CPUs, return. If just a single CPU, turn this request
@@ -344,7 +344,7 @@if (!num_cpus)return 0 ;else if (num_cpus ==1 ) { - cpu =first_cpu (mask); + cpu =first_cpu (*mask);return smp_call_function_single (cpu, func, info, wait); }
@@ -364,7 +364,7 @@ data->csd.func = func; data->csd.info = info; data->refs = num_cpus; - data->cpumask = mask; + data->cpumask = *mask;spin_lock_irqsave (&call_function_lock, flags);list_add_tail_rcu (&data->csd.list, &call_function_queue);
@@ -377,7 +377,7 @@if (wait) {csd_flag_wait (&data->csd);if (unlikely (slowpath)) -smp_call_function_mask_quiesce_stack (mask); +smp_call_function_mask_quiesce_stack (*mask); }return 0 ;
@@ -402,9 +402,10 @@int smp_call_function (void (*func)(void *),void *info,int wait) {int ret; +cpumask_t tmp_online_map = cpu_online_map;preempt_disable (); - ret =smp_call_function_mask (cpu_online_map, func, info, wait); + ret =smp_call_function_mask (&tmp_online_map, func, info, wait);preempt_enable ();return ret; }
virt/kvm/kvm_main.c +2 -2
@@ -124,7 +124,7 @@if (cpus_empty (cpus)) goto out; ++kvm->stat.remote_tlb_flush; -smp_call_function_mask (cpus, ack_flush,NULL ,1 ); +smp_call_function_mask (&cpus, ack_flush,NULL ,1 ); out:put_cpu (); }
@@ -149,7 +149,7 @@ }if (cpus_empty (cpus)) goto out; -smp_call_function_mask (cpus, ack_flush,NULL ,1 ); +smp_call_function_mask (&cpus, ack_flush,NULL ,1 ); out:put_cpu (); }
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 3 lines · Jes Sorensen
I have only tested this on ia64, but it boots, so it's obviously perfect<tm> :-)
Well, it probably boots because it doesn't really seem to _change_ much of anything.
Things like this:
-static inline void arch_send_call_function_ipi(cpumask_t mask) +static inline void arch_send_call_function_ipi(cpumask_t *mask)
{
- smp_ops.send_call_func_ipi(mask);
+ smp_ops.send_call_func_ipi(*mask);
}
will still do that stack allocation at the time of the call. You'd have to pass the thing all the way down as a pointer..
Linus
David Miller
maintainerRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
From: Linus Torvalds <torvalds-de/[email protected]>
Date: Fri, 29 Aug 2008 09:14:44 -0700 (PDT)
Quoted 12 lines
Well, it probably boots because it doesn't really seem to _change_ much of anything.
Things like this:
-static inline void
arch_send_call_function_ipi(cpumask_t mask)+static inline voidarch_send_call_function_ipi(cpumask_t *mask)
{
-smp_ops.send_call_func_ipi(mask);
+smp_ops.send_call_func_ipi(*mask);
}
Quoted 2 lines
will still do that stack allocation at the time of the call. You'd have to pass the thing all the way down as a pointer..
True, but we have to get there one step at a time.
BTW, sparc64 already wants a pointer here, so it's completely ready for this:
void arch_send_call_function_ipi(cpumask_t mask)
{
xcall_deliver((u64) &xcall_call_function, 0, 0, &mask);
}Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Linus TorvaldsQuoted 6 lines · Linus Torvalds
Quoted 2 lines · Jes Sorensen
I have only tested this on ia64, but it boots, so it's obviously perfect<tm> :-)
Well, it probably boots because it doesn't really seem to _change_ much of anything.
Hi Linus,
I realize that, but as I have been doing this work on ia64, I didn't want to mess too much with the x86 code. The ia64 part of the patch does change things :-)
If someone with more x86 knowledge would want to try and make that part of the patch better, and more importantly test it, I'd be quite keen on helping out.
Cheers,
Jes
Quoted 13 lines
Things like this:
-static inline void
arch_send_call_function_ipi(cpumask_t mask)+static inline voidarch_send_call_function_ipi(cpumask_t *mask)
{
-smp_ops.send_call_func_ipi(mask);
+smp_ops.send_call_func_ipi(*mask);
}will still do that stack allocation at the time of the call. You'd have to pass the thing all the way down as a pointer..
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Linus TorvaldsQuoted 9 lines · Linus Torvalds
Quoted 2 lines · Jes Sorensen
I have only tested this on ia64, but it boots, so it's obviously perfect<tm> :-)
Well, it probably boots because it doesn't really seem to _change_ much of anything.
Things like this:
Quoted 8 lines
-static inline void
arch_send_call_function_ipi(cpumask_t mask)+static inline voidarch_send_call_function_ipi(cpumask_t *mask)
{
-smp_ops.send_call_func_ipi(mask);
+smp_ops.send_call_func_ipi(*mask);
}will still do that stack allocation at the time of the call. You'd have to pass the thing all the way down as a pointer..
Linus,
Ok, so here's a version which tries to do the right thing on x86 as well. Build tested on x86_64, but don't have an easy way to test it right now. It's booting on ia64.
Cheers,
Jes
Change smp_call_function_mask() to take a pointer to the cpumask_t rather than passing it by value. This avoids recursive copies of the cpumask_t on the stack in the IPI call. For large NR_CPUS, this is particularly bad, and the cost of doing this forNR_CPUS < bits_per_long is negligeble.
--- arch/alpha/include/asm/smp.h | 2 +- arch/alpha/kernel/smp.c | 4 ++-- arch/arm/include/asm/smp.h | 2 +- arch/arm/kernel/smp.c | 4 ++-- arch/ia64/include/asm/smp.h | 2 +- arch/ia64/kernel/smp.c | 6 +++--- arch/m32r/kernel/smp.c | 4 ++-- arch/mips/kernel/smp.c | 4 ++-- arch/parisc/kernel/smp.c | 6 +++--- arch/powerpc/include/asm/smp.h | 2 +- arch/powerpc/kernel/smp.c | 4 ++-- arch/sh/include/asm/smp.h | 2 +- arch/sh/kernel/smp.c | 4 ++-- arch/sparc/include/asm/smp_64.h | 2 +- arch/sparc64/kernel/smp.c | 4 ++-- arch/x86/kernel/apic_32.c | 2 +- arch/x86/kernel/apic_64.c | 2 +- arch/x86/kernel/crash.c | 2 +- arch/x86/kernel/genapic_flat_64.c | 20 ++++++++++++-------- arch/x86/kernel/genx2apic_uv_x.c | 10 ++++++---- arch/x86/kernel/io_apic_64.c | 6 ++++-- arch/x86/kernel/smp.c | 12 ++++++++---- arch/x86/kernel/tlb_32.c | 2 +- arch/x86/kernel/tlb_64.c | 2 +- arch/x86/xen/smp.c | 13 +++++++------ include/asm-m32r/smp.h | 2 +- include/asm-mips/smp.h | 2 +- include/asm-parisc/smp.h | 2 +- include/asm-x86/genapic_32.h | 2 +- include/asm-x86/genapic_64.h | 2 +- include/asm-x86/mach-default/mach_ipi.h | 10 ++++++---- include/asm-x86/smp.h | 6 +++--- include/linux/smp.h | 2 +- kernel/smp.c | 15 ++++++++------- virt/kvm/kvm_main.c | 4 ++-- 35 files changed, 93 insertions(+), 77 deletions(-)
arch/alpha/include/asm/smp.h +1 -1
@@ -48,7 +48,7 @@#define cpu_possible_map cpu_present_mapextern void arch_send_call_function_single_ipi (int cpu); -extern void arch_send_call_function_ipi (cpumask_t mask); +extern void arch_send_call_function_ipi (cpumask_t *mask);#else /* CONFIG_SMP */
arch/alpha/kernel/smp.c +2 -2
@@ -637,9 +637,9 @@send_ipi_message (to_whom,IPI_CPU_STOP ); } -void arch_send_call_function_ipi (cpumask_t mask) +void arch_send_call_function_ipi (cpumask_t *mask) { -send_ipi_message (mask,IPI_CALL_FUNC ); +send_ipi_message (*mask,IPI_CALL_FUNC ); }void arch_send_call_function_single_ipi (int cpu)
arch/arm/include/asm/smp.h +1 -1
@@ -102,7 +102,7 @@extern void platform_cpu_enable (unsigned int cpu);extern void arch_send_call_function_single_ipi (int cpu); -extern void arch_send_call_function_ipi (cpumask_t mask); +extern void arch_send_call_function_ipi (cpumask_t *mask);/* * Local timer interrupt handling function (can be IPI'ed).
arch/arm/kernel/smp.c +2 -2
@@ -356,9 +356,9 @@local_irq_restore (flags); } -void arch_send_call_function_ipi (cpumask_t mask) +void arch_send_call_function_ipi (cpumask_t *mask) { -send_ipi_message (mask,IPI_CALL_FUNC ); +send_ipi_message (*mask,IPI_CALL_FUNC ); }void arch_send_call_function_single_ipi (int cpu)
arch/ia64/include/asm/smp.h +1 -1
@@ -127,7 +127,7 @@extern int is_multithreading_enabled (void );extern void arch_send_call_function_single_ipi (int cpu); -extern void arch_send_call_function_ipi (cpumask_t mask); +extern void arch_send_call_function_ipi (cpumask_t *mask);#else /* CONFIG_SMP */
arch/ia64/kernel/smp.c +3 -3
@@ -166,11 +166,11 @@* Called with preemption disabled. */ static inline void -send_IPI_mask (cpumask_t mask,int op) +send_IPI_mask (cpumask_t *mask,int op) {unsigned int cpu; -for_each_cpu_mask (cpu, mask) { +for_each_cpu_mask (cpu, *mask) {send_IPI_single (cpu, op); } }
@@ -316,7 +316,7 @@send_IPI_single (cpu,IPI_CALL_FUNC_SINGLE ); } -void arch_send_call_function_ipi (cpumask_t mask) +void arch_send_call_function_ipi (cpumask_t *mask) {send_IPI_mask (mask,IPI_CALL_FUNC ); }
arch/m32r/kernel/smp.c +2 -2
@@ -546,9 +546,9 @@for ( ; ; ); } -void arch_send_call_function_ipi (cpumask_t mask) +void arch_send_call_function_ipi (cpumask_t *mask) { -send_IPI_mask (mask,CALL_FUNCTION_IPI ,0 ); +send_IPI_mask (*mask,CALL_FUNCTION_IPI ,0 ); }void arch_send_call_function_single_ipi (int cpu)
arch/mips/kernel/smp.c +2 -2
@@ -131,9 +131,9 @@cpu_idle (); } -void arch_send_call_function_ipi (cpumask_t mask) +void arch_send_call_function_ipi (cpumask_t *mask) { - mp_ops->send_ipi_mask (mask,SMP_CALL_FUNCTION ); + mp_ops->send_ipi_mask (*mask,SMP_CALL_FUNCTION ); }/*
arch/parisc/kernel/smp.c +3 -3
@@ -228,11 +228,11 @@ }static void -send_IPI_mask (cpumask_t mask,enum ipi_message_type op) +send_IPI_mask (cpumask_t *mask,enum ipi_message_type op) {int cpu; -for_each_cpu_mask (cpu, mask) +for_each_cpu_mask (cpu, *mask)ipi_send (cpu, op); }
@@ -274,7 +274,7 @@send_IPI_allbutself (IPI_NOP ); } -void arch_send_call_function_ipi (cpumask_t mask) +void arch_send_call_function_ipi (cpumask_t *mask) {send_IPI_mask (mask,IPI_CALL_FUNC ); }
arch/powerpc/include/asm/smp.h +1 -1
@@ -119,7 +119,7 @@extern struct smp_ops_t *smp_ops;extern void arch_send_call_function_single_ipi (int cpu); -extern void arch_send_call_function_ipi (cpumask_t mask); +extern void arch_send_call_function_ipi (cpumask_t *mask);#endif /* __ASSEMBLY__ */
arch/powerpc/kernel/smp.c +2 -2
@@ -135,11 +135,11 @@ smp_ops->message_pass (cpu,PPC_MSG_CALL_FUNC_SINGLE ); } -void arch_send_call_function_ipi (cpumask_t mask) +void arch_send_call_function_ipi (cpumask_t *mask) {unsigned int cpu; -for_each_cpu_mask (cpu, mask) +for_each_cpu_mask (cpu, *mask) smp_ops->message_pass (cpu,PPC_MSG_CALL_FUNCTION ); }
arch/sh/include/asm/smp.h +1 -1
@@ -39,7 +39,7 @@int plat_register_ipi_handler (unsigned int message,void (*handler)(void *),void *arg);extern void arch_send_call_function_single_ipi (int cpu); -extern void arch_send_call_function_ipi (cpumask_t mask); +extern void arch_send_call_function_ipi (cpumask_t *mask);#else
arch/sh/kernel/smp.c +2 -2
@@ -171,11 +171,11 @@smp_call_function (stop_this_cpu,0 ,0 ); } -void arch_send_call_function_ipi (cpumask_t mask) +void arch_send_call_function_ipi (cpumask_t *mask) {int cpu; -for_each_cpu_mask (cpu, mask) +for_each_cpu_mask (cpu, *mask)plat_send_ipi (cpu,SMP_MSG_FUNCTION ); }
arch/sparc/include/asm/smp_64.h +1 -1
@@ -35,7 +35,7 @@extern int sparc64_multi_core;extern void arch_send_call_function_single_ipi (int cpu); -extern void arch_send_call_function_ipi (cpumask_t mask); +extern void arch_send_call_function_ipi (cpumask_t *mask);/* * General functions that each host system must provide.
arch/sparc64/kernel/smp.c +2 -2
@@ -810,9 +810,9 @@extern unsigned long xcall_call_function; -void arch_send_call_function_ipi (cpumask_t mask) +void arch_send_call_function_ipi (cpumask_t *mask) { -xcall_deliver ((u64) &xcall_call_function,0 ,0 , &mask); +xcall_deliver ((u64) &xcall_call_function,0 ,0 , mask); }extern unsigned long xcall_call_function_single;
arch/x86/kernel/apic_32.c +1 -1
@@ -291,7 +291,7 @@static void lapic_timer_broadcast (cpumask_t mask) {#ifdef CONFIG_SMP -send_IPI_mask (mask,LOCAL_TIMER_VECTOR ); +send_IPI_mask (&mask,LOCAL_TIMER_VECTOR );#endif }
arch/x86/kernel/apic_64.c +1 -1
@@ -280,7 +280,7 @@static void lapic_timer_broadcast (cpumask_t mask) {#ifdef CONFIG_SMP -send_IPI_mask (mask,LOCAL_TIMER_VECTOR ); +send_IPI_mask (&mask,LOCAL_TIMER_VECTOR );#endif }
arch/x86/kernel/crash.c +1 -1
@@ -80,7 +80,7 @@cpumask_t mask = cpu_online_map;cpu_clear (safe_smp_processor_id (), mask);if (!cpus_empty (mask)) -send_IPI_mask (mask,NMI_VECTOR ); +send_IPI_mask (&mask,NMI_VECTOR ); }static struct notifier_block crash_nmi_nb = {
arch/x86/kernel/genapic_flat_64.c +12 -8
@@ -58,9 +58,9 @@apic_write (APIC_LDR , val); } -static void flat_send_IPI_mask (cpumask_t cpumask,int vector) +static void flat_send_IPI_mask (cpumask_t *cpumask,int vector) { -unsigned long mask =cpus_addr (cpumask)[0 ]; +unsigned long mask =cpus_addr (*cpumask)[0 ];unsigned long flags;local_irq_save (flags);
@@ -81,7 +81,7 @@cpu_clear (smp_processor_id (), allbutme);if (!cpus_empty (allbutme)) -flat_send_IPI_mask (allbutme, vector); +flat_send_IPI_mask (&allbutme, vector); }else if (num_online_cpus () >1 ) {__send_IPI_shortcut (APIC_DEST_ALLBUT , vector,APIC_DEST_LOGICAL ); }
@@ -89,8 +89,10 @@static void flat_send_IPI_all (int vector) { +cpumask_t tmp_online_map = cpu_online_map; +if (vector ==NMI_VECTOR ) -flat_send_IPI_mask (cpu_online_map, vector); +flat_send_IPI_mask (&tmp_online_map, vector);else __send_IPI_shortcut (APIC_DEST_ALLINC , vector,APIC_DEST_LOGICAL ); }
@@ -141,9 +143,9 @@return cpumask_of_cpu (cpu); } -static void physflat_send_IPI_mask (cpumask_t cpumask,int vector) +static void physflat_send_IPI_mask (cpumask_t *cpumask,int vector) { -send_IPI_mask_sequence (cpumask, vector); +send_IPI_mask_sequence (*cpumask, vector); }static void physflat_send_IPI_allbutself (int vector)
@@ -151,12 +153,14 @@cpumask_t allbutme = cpu_online_map;cpu_clear (smp_processor_id (), allbutme); -physflat_send_IPI_mask (allbutme, vector); +physflat_send_IPI_mask (&allbutme, vector); }static void physflat_send_IPI_all (int vector) { -physflat_send_IPI_mask (cpu_online_map, vector); +cpumask_t tmp_online_map = cpu_online_map; + +physflat_send_IPI_mask (&tmp_online_map, vector); }static unsigned int physflat_cpu_mask_to_apicid (cpumask_t cpumask)
arch/x86/kernel/genx2apic_uv_x.c +6 -4
@@ -94,12 +94,12 @@uv_write_global_mmr64 (pnode,UVH_IPI_INT , val); } -static void uv_send_IPI_mask (cpumask_t mask,int vector) +static void uv_send_IPI_mask (cpumask_t *mask,int vector) {unsigned int cpu;for_each_possible_cpu (cpu) -if (cpu_isset (cpu, mask)) +if (cpu_isset (cpu, *mask))uv_send_IPI_one (cpu, vector); }
@@ -110,12 +110,14 @@cpu_clear (smp_processor_id (), mask);if (!cpus_empty (mask)) -uv_send_IPI_mask (mask, vector); +uv_send_IPI_mask (&mask, vector); }static void uv_send_IPI_all (int vector) { -uv_send_IPI_mask (cpu_online_map, vector); +cpumask_t mask = cpu_online_map; + +uv_send_IPI_mask (&mask, vector); }static int uv_apic_id_registered (void )
arch/x86/kernel/io_apic_64.c +4 -2
@@ -1379,9 +1379,11 @@ {struct irq_cfg *cfg = &irq_cfg[irq];unsigned long flags; +cpumask_t mask;spin_lock_irqsave (&vector_lock, flags); -send_IPI_mask (cpumask_of_cpu (first_cpu (cfg->domain)), cfg->vector); + mask =cpumask_of_cpu (first_cpu (cfg->domain)); +send_IPI_mask (&mask, cfg->vector);spin_unlock_irqrestore (&vector_lock, flags);return 1 ;
@@ -1446,7 +1448,7 @@cpus_and (cleanup_mask, cfg->old_domain, cpu_online_map); cfg->move_cleanup_count =cpus_weight (cleanup_mask); -send_IPI_mask (cleanup_mask,IRQ_MOVE_CLEANUP_VECTOR ); +send_IPI_mask (&cleanup_mask,IRQ_MOVE_CLEANUP_VECTOR ); cfg->move_in_progress =0 ; } }
arch/x86/kernel/smp.c +8 -4
@@ -114,26 +114,30 @@*/ static void native_smp_send_reschedule (int cpu) { +cpumask_t mask; +if (unlikely (cpu_is_offline (cpu))) {WARN_ON (1 );return ; } -send_IPI_mask (cpumask_of_cpu (cpu),RESCHEDULE_VECTOR ); + mask =cpumask_of_cpu (cpu); +send_IPI_mask (&mask,RESCHEDULE_VECTOR ); }void native_send_call_func_single_ipi (int cpu) { -send_IPI_mask (cpumask_of_cpu (cpu),CALL_FUNCTION_SINGLE_VECTOR ); +cpumask_t mask =cpumask_of_cpu (cpu); +send_IPI_mask (&mask,CALL_FUNCTION_SINGLE_VECTOR ); } -void native_send_call_func_ipi (cpumask_t mask) +void native_send_call_func_ipi (cpumask_t *mask) {cpumask_t allbutself; allbutself = cpu_online_map;cpu_clear (smp_processor_id (), allbutself); -if (cpus_equal (mask, allbutself) && +if (cpus_equal (*mask, allbutself) &&cpus_equal (cpu_online_map, cpu_callout_map))send_IPI_allbutself (CALL_FUNCTION_VECTOR );else
arch/x86/kernel/tlb_32.c +1 -1
@@ -158,7 +158,7 @@* We have to send the IPI only to * CPUs affected. */ -send_IPI_mask (cpumask,INVALIDATE_TLB_VECTOR ); +send_IPI_mask (&cpumask,INVALIDATE_TLB_VECTOR );while (!cpus_empty (flush_cpumask))/* nothing. lockup detection does not belong here */
arch/x86/kernel/tlb_64.c +1 -1
@@ -186,7 +186,7 @@* We have to send the IPI only to * CPUs affected. */ -send_IPI_mask (cpumask,INVALIDATE_TLB_VECTOR_START + sender); +send_IPI_mask (&cpumask,INVALIDATE_TLB_VECTOR_START + sender);while (!cpus_empty (f->flush_cpumask))cpu_relax ();
arch/x86/xen/smp.c +7 -6
@@ -361,24 +361,25 @@xen_send_IPI_one (cpu,XEN_RESCHEDULE_VECTOR ); } -static void xen_send_IPI_mask (cpumask_t mask,enum ipi_vector vector) +static void xen_send_IPI_mask (cpumask_t *mask,enum ipi_vector vector) {unsigned cpu; +cpumask_t newmask; -cpus_and (mask, mask, cpu_online_map); +cpus_and (newmask, *mask, cpu_online_map); -for_each_cpu_mask_nr (cpu, mask) +for_each_cpu_mask_nr (cpu, newmask)xen_send_IPI_one (cpu, vector); } -static void xen_smp_send_call_function_ipi (cpumask_t mask) +static void xen_smp_send_call_function_ipi (cpumask_t *mask) {int cpu;xen_send_IPI_mask (mask,XEN_CALL_FUNCTION_VECTOR );/* Make sure other vcpus get a chance to run if they need to. */ -for_each_cpu_mask_nr (cpu, mask) { +for_each_cpu_mask_nr (cpu, *mask) {if (xen_vcpu_stolen (cpu)) {HYPERVISOR_sched_op (SCHEDOP_yield,0 );break ;
@@ -388,7 +389,7 @@static void xen_smp_send_call_function_single_ipi (int cpu) { -xen_send_IPI_mask (cpumask_of_cpu (cpu),XEN_CALL_FUNCTION_SINGLE_VECTOR ); +xen_send_IPI_mask (&cpumask_of_cpu (cpu),XEN_CALL_FUNCTION_SINGLE_VECTOR ); }static irqreturn_t xen_call_function_interrupt (int irq,void *dev_id)
include/asm-m32r/smp.h +1 -1
@@ -90,7 +90,7 @@extern unsigned long send_IPI_mask_phys (cpumask_t ,int ,int );extern void arch_send_call_function_single_ipi (int cpu); -extern void arch_send_call_function_ipi (cpumask_t mask); +extern void arch_send_call_function_ipi (cpumask_t *mask);#endif /* not __ASSEMBLY__ */
include/asm-mips/smp.h +1 -1
@@ -58,6 +58,6 @@ extern asmlinkage void smp_call_function_interrupt(void);extern void arch_send_call_function_single_ipi (int cpu); -extern void arch_send_call_function_ipi (cpumask_t mask); +extern void arch_send_call_function_ipi (cpumask_t *mask);#endif /* __ASM_SMP_H */
include/asm-parisc/smp.h +1 -1
@@ -31,7 +31,7 @@extern void smp_send_all_nop (void );extern void arch_send_call_function_single_ipi (int cpu); -extern void arch_send_call_function_ipi (cpumask_t mask); +extern void arch_send_call_function_ipi (cpumask_t *mask);#endif /* !ASSEMBLY */
include/asm-x86/genapic_32.h +1 -1
@@ -60,7 +60,7 @@#ifdef CONFIG_SMP /* ipi */ -void (*send_IPI_mask)(cpumask_t mask,int vector); +void (*send_IPI_mask)(cpumask_t *mask,int vector);void (*send_IPI_allbutself)(int vector);void (*send_IPI_all)(int vector);#endif
include/asm-x86/genapic_64.h +1 -1
@@ -21,7 +21,7 @@cpumask_t (*vector_allocation_domain)(int cpu);void (*init_apic_ldr)(void );/* ipi */ -void (*send_IPI_mask)(cpumask_t mask,int vector); +void (*send_IPI_mask)(cpumask_t *mask,int vector);void (*send_IPI_allbutself)(int vector);void (*send_IPI_all)(int vector);/* */
include/asm-x86/mach-default/mach_ipi.h +6 -4
@@ -13,9 +13,9 @@#include <asm/genapic.h> #define send_IPI_mask (genapic->send_IPI_mask)#else -static inline void send_IPI_mask (cpumask_t mask,int vector) +static inline void send_IPI_mask (cpumask_t *mask,int vector) { -send_IPI_mask_bitmask (mask, vector); +send_IPI_mask_bitmask (*mask, vector); }#endif
@@ -25,15 +25,17 @@cpumask_t mask = cpu_online_map;cpu_clear (smp_processor_id (), mask); -send_IPI_mask (mask, vector); +send_IPI_mask (&mask, vector); }else __send_IPI_shortcut (APIC_DEST_ALLBUT , vector); }static inline void __local_send_IPI_all (int vector) { +cpumask_t mask = cpu_online_map; +if (no_broadcast || vector ==NMI_VECTOR ) -send_IPI_mask (cpu_online_map, vector); +send_IPI_mask (&mask, vector);else __send_IPI_shortcut (APIC_DEST_ALLINC , vector); }
include/asm-x86/smp.h +3 -3
@@ -53,7 +53,7 @@void (*smp_send_stop)(void );void (*smp_send_reschedule)(int cpu); -void (*send_call_func_ipi)(cpumask_t mask); +void (*send_call_func_ipi)(cpumask_t *mask);void (*send_call_func_single_ipi)(int cpu); };
@@ -101,7 +101,7 @@ smp_ops.send_call_func_single_ipi (cpu); } -static inline void arch_send_call_function_ipi (cpumask_t mask) +static inline void arch_send_call_function_ipi (cpumask_t *mask) { smp_ops.send_call_func_ipi (mask); }
@@ -110,7 +110,7 @@void native_smp_prepare_cpus (unsigned int max_cpus);void native_smp_cpus_done (unsigned int max_cpus);int native_cpu_up (unsigned int cpunum); -void native_send_call_func_ipi (cpumask_t mask); +void native_send_call_func_ipi (cpumask_t *mask);void native_send_call_func_single_ipi (int cpu);extern int __cpu_disable (void );
include/linux/smp.h +1 -1
@@ -62,7 +62,7 @@* Call a function on all other processors */ int smp_call_function (void (*func)(void *info),void *info,int wait); -int smp_call_function_mask (cpumask_t mask,void (*func)(void *info),void *info, +int smp_call_function_mask (cpumask_t *mask,void (*func)(void *info),void *info,int wait);int smp_call_function_single (int cpuid,void (*func) (void *info),void *info,int wait);
kernel/smp.c +8 -7
@@ -318,7 +318,7 @@* hardware interrupt handler or from a bottom half handler. Preemption * must be disabled when calling this function. */ -int smp_call_function_mask (cpumask_t mask,void (*func)(void *),void *info, +int smp_call_function_mask (cpumask_t *mask,void (*func)(void *),void *info,int wait) {struct call_function_data d;
@@ -334,8 +334,8 @@ cpu =smp_processor_id (); allbutself = cpu_online_map;cpu_clear (cpu, allbutself); -cpus_and (mask, mask, allbutself); - num_cpus =cpus_weight (mask); +cpus_and (*mask, *mask, allbutself); + num_cpus =cpus_weight (*mask);/* * If zero CPUs, return. If just a single CPU, turn this request
@@ -344,7 +344,7 @@if (!num_cpus)return 0 ;else if (num_cpus ==1 ) { - cpu =first_cpu (mask); + cpu =first_cpu (*mask);return smp_call_function_single (cpu, func, info, wait); }
@@ -364,7 +364,7 @@ data->csd.func = func; data->csd.info = info; data->refs = num_cpus; - data->cpumask = mask; + data->cpumask = *mask;spin_lock_irqsave (&call_function_lock, flags);list_add_tail_rcu (&data->csd.list, &call_function_queue);
@@ -377,7 +377,7 @@if (wait) {csd_flag_wait (&data->csd);if (unlikely (slowpath)) -smp_call_function_mask_quiesce_stack (mask); +smp_call_function_mask_quiesce_stack (*mask); }return 0 ;
@@ -402,9 +402,10 @@int smp_call_function (void (*func)(void *),void *info,int wait) {int ret; +cpumask_t tmp_online_map = cpu_online_map;preempt_disable (); - ret =smp_call_function_mask (cpu_online_map, func, info, wait); + ret =smp_call_function_mask (&tmp_online_map, func, info, wait);preempt_enable ();return ret; }
virt/kvm/kvm_main.c +2 -2
@@ -124,7 +124,7 @@if (cpus_empty (cpus)) goto out; ++kvm->stat.remote_tlb_flush; -smp_call_function_mask (cpus, ack_flush,NULL ,1 ); +smp_call_function_mask (&cpus, ack_flush,NULL ,1 ); out:put_cpu (); }
@@ -149,7 +149,7 @@ }if (cpus_empty (cpus)) goto out; -smp_call_function_mask (cpus, ack_flush,NULL ,1 ); +smp_call_function_mask (&cpus, ack_flush,NULL ,1 ); out:put_cpu (); }
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Ingo MolnarQuoted 25 lines · Ingo Molnar
Quoted 9 lines · Linus Torvalds
Quoted 8 lines · Linus Torvalds
checkstack.pl shows these things as the top problems:
0xffffffff80266234 smp_call_function_mask [vmlinux]: 2736 0xffffffff80234747 __build_sched_domains [vmlinux]: 2232 0xffffffff8023523f __build_sched_domains [vmlinux]: 2232Anyway, the reason
smp_call_function_maskand friends have such _huge_ stack usages for you is that they contain a 'cpumask_t' on the stack.Quoted 8 lines
In fact, they contain multiple CPU-masks, each 4k-bits - 512 bytes - in size. And they tend to call each other.
Quite frankly, I don't think we were really ready for 4k CPU's. I'm going to commit this patch to make sure others don't do that many CPU's by mistake. It marks MAXCPU's as being 'broken' so you cannot select it, and also limits the number of CPU's that you _can_ select to "just" 512.
yeah, that's OK i guess - distros can still enable 4K support if they wish to. Someone interested in improving the stack footprint situation should dust off the max-stack-footprint tracer so that we can catch these things in a more structured way.
Quoted 6 lines
And i guess the next generation of 4K CPUs support should just get away from cpumask_t-on-kernel-stack model altogether, as the current model is not maintainable. We tried the on-kernel-stack variant, and it really does not work reliably. We can fix this in v2.6.28.
Ingo
I would be most interested in any tools to analyze call-trees and accumulated stack usages. My current method of using kdb is really time consuming.
Thanks!
Mike
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 4 lines · Mike Travis
I would be most interested in any tools to analyze call-trees and accumulated stack usages. My current method of using kdb is really time consuming.
Well, even just scripts/checkstack.pl is quite relevant.
The fact is, anything with a stack footprint of more than a hundred bytes is suspect. We _do_ have a lot of cases of several hundred bytes, and some of them are even very intentional.
For an example of _intentional_ and valid large stacks, look at do_sys_poll and do_select. They both have a big stack footprint in a normal kernel, and that's on purpose - it's not pretty, but they are very common and performance-sensitive functions, and using a big stack allows some basic allocations to be much cheaper by default.
Same goes for early_printk(), although I don't think the reasons are really very strong in that case.
Sadly, while those functions are _fairly_ high up, they aren't at the top, and we do have a lot of other functions that have huge stack footprints for totally bogus reasons. But the intentional ones are at least in the top ten.
But the kernel that Alan had problems with was different. The _intentional_ ones were way down in the noise. do_sys_poll wasn't in the top ten, it was barely even in the top 50! (It was in fact #49, to be exact).
So look at the top ten in my kernel:
1 ide_generic_init [vmlinux]: 1384
2 idefloppy_ioctl [vmlinux]: 1208
3 e1000_check_options [vmlinux]: 1152
4 do_sys_poll [vmlinux]: 904
5 ide_floppy_get_capacity [vmlinux]: 872
6 do_select [vmlinux]: 744
7 early_printk [vmlinux]: 720
8 do_task_stat [vmlinux]: 680
9 mmc_ioctl [vmlinux]: 648
10 elf_kcore_store_hdr [vmlinux]: 576.. and in Alan's kernel:
1 smp_call_function_mask [vmlinux]: 2736
2 __build_sched_domains [vmlinux]: 2232
3 setup_IO_APIC_irq [vmlinux]: 1616
4 arch_setup_ht_irq [vmlinux]: 1600
5 arch_setup_msi_irq [vmlinux]: 1600
6 __assign_irq_vector [vmlinux]: 1592
7 move_task_off_dead_cpu [vmlinux]: 1592
8 tick_handle_oneshot_broadcast [vmlinux]:1544
9 store_scaling_governor [vmlinux]: 1376
10 cpuset_write_resmask [vmlinux]: 1360That's a big difference. The top #1 in my kernel would just _barely_ be in the top 10 in Alan's kernel (he doesn't have it at all, because he didn't compile the drives I did into the kernel).
And the top three in my kernel are just because of crap code. That "e1000_check_options" thing is there just because it creates multiple "struct e1000_option" structures. I wrote an ugly but totally trivial patch to get it down to ~600 bytes, and it would be less if I had bothered to waste any more time on it.
The others are similar issues of "people just didn't think".
But look at the top ones in Alan's kernel. Not only are they _much_ bigger than the top ones in a sane kernel, they are _all_ due to cpumask_t, I think.
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Linus TorvaldsQuoted 37 lines · Linus Torvalds
Quoted 8 lines · Linus Torvalds
checkstack.pl shows these things as the top problems:
0xffffffff80266234 smp_call_function_mask [vmlinux]: 2736 0xffffffff80234747 __build_sched_domains [vmlinux]: 2232 0xffffffff8023523f __build_sched_domains [vmlinux]: 2232Anyway, the reason
smp_call_function_maskand friends have such _huge_ stack usages for you is that they contain a 'cpumask_t' on the stack.In fact, they contain multiple CPU-masks, each 4k-bits - 512 bytes - in size. And they tend to call each other.
Quite frankly, I don't think we were really ready for 4k CPU's. I'm going to commit this patch to make sure others don't do that many CPU's by mistake. It marks MAXCPU's as being 'broken' so you cannot select it, and also limits the number of CPU's that you _can_ select to "just" 512.
Right now, 4k cpu's is known broken because of the stack usage. I'm not willing to debug more of these kinds of stack smashers, they're really nasty to work with. I wonder how many other random failures these have been involved with?
This patch also makes the ifdef mess in
Kconfigmuch cleaner and avoids duplicate definitions by just conditionally suppressing the question and giving higher defaults.We can enable MAXSMP and raise the CPU limits some time in the future. But that future is not going to be before 2.6.27 - the code simply isn't ready for it.
The reason I picked 512 CPU's as the limit is that we _used_ to limit things to 255. So it's higher than it used to be, but low enough to still feel safe. Considering that a 4k-bit CPU mask (512 bytes) _almost_ worked, the 512-bit (64 bytes) masks are almost certainly fine.
Quoted 8 lines
Still, sane people should limit their
NR_CPUSto 8 or 16 or something like that. Very very few people really need the pain of bigNR_CPUS. Not even "just" 512 CPU's.Travis, Ingo and Thomas cc'd, since they were involved in the original commit (
1184dc2ffe2c8fb9afb766d870850f2c3165ef25) that raised the limit.Linus
Hi Linus,
[Sorry for the long winded response, but I felt that sufficient background is needed to address this issue.... YOMV :-)]
The need to allow distros to set NR_CPUS=4096 (and NODES_SHIFT=9) is critical to our upcoming SGI systems using what we have been calling "UV". This system will be capable of supporting 4096 cpu threads in a single system image (and 16k cpus/2k nodes is right around the corner). While obviously I cannot divulge too many details, it's sufficient to say there are customers who not only require this extended capability, but are extremely excited about it.
But the nature of some of these system environments is that they will not accept a specially built kernel, but only a kernel that has been built and certified (both from the application standpoint as well as the security standpoint) by standard distributions. And you probably know how extensively these distributions test and certify for many known defects and absolutely require that incoming source changes come from the community supported source bases, primarily yours.
Due to the lead time required to accomplish these certifications, the version of the distributions that will be available when this system releases will be based on 2.6.27. (They will allow patches "post-2.6.27-rc.final" as long as those are committed in the source base.) The two distributions that SGI supports for our customers is SLES (SUSE Linux Enterprise Server) and RHEL (Red Hat Enterprise Linux). [They, of course, are free to run any OS of their choosing, but SGI only provides front line support for those two.]
I started last August to begin analyzing how to accomplish the above goals and where exactly are the hot spots in the kernel that would require attention. It quickly became clear that cpumask_t and nodemask_t are two variables that are very casually used (along with NR_CPUS), because the assumption was that 64 "was more than sufficient" for an upper limit and even extending it to 128 or 255 (254 was the maximum IPI broadcast ID until x2apic), only added a few more bytes here and there.
I chose not to introduce too many dramatic changes and instead analyzed every instance where cpumask_t and NR_CPUS was being used (along with the node counterparts.) An initial proposal was to allow the default stack size to be increased, but this was met with a lot of objections because of the extensive work that was done to bring it to it's current size.
So in summary, the goals of the changes that I have been making since last October are:
1. Allow a "typical distro" configured kernel with NR_CPUS=4096 and NODES_SHIFT=9 to be booted on an x86_64 system with 2GB of memory. (Some thought was given to use a 512Mb laptop as the base, but because of other memory bloat from using a 64-bit kernel, that was considered not very useful.)
[Note I frequently use an allyes and allmod config for testing.]
2. To lessen as much as possible, the impact on memory usage for that same kernel on that same system.
3. To lessen as much as possible, the impact on system performance for that same kernel on that same system. [Which mostly depended on #2.]
I booted the first 4096 cpu kernel last February, and since around March or April, Ingo has been (build and boot) testing the x86 branch using "MAXSMP" to trigger the increased defaults quite extensively (IIRC, it was somewhere between 75% and 90% of all kernels built.) We here at SGI nightly build 4 trees (linux-2.6, linux-next, linux-mm, linux-x86) to insure new checkins don't conflict with changes we've made in the past. Unfortunately, our run testing wasn't sufficient to catch this latest error (and I will be quickly fixing that.)
I will also revisit all the past areas to analyze if there have been other abuses of stack and memory space added since the 4k cpu limit was "certified" as usable and releasable. (See below for an initial survey of size increases between a 512cpu/64node configuration and a 4096cpu/512node configuration.)
So perhaps "MAXSMP" is not needed (or perhaps should be more hidden to reduce accidental uses), but allowing the defaults listed above to be in the standard x86/Kconfig insures that the distros can at least attempt certification with the maximally configured kernels for their enterprise editions of Linux.
There are many more changes that will be proposed for the 2.6.28 window. Most certainly your concerns, as well as others, about how to change the current "cpumask paradigm" to be more easily manageable for systems with huge cpu counts, will be visited. (And surely be well discussed. :-)
Thanks,
Mike
---
linux-2.6: v2.6.27-rc4-176-gb8e6c91
====== Data (-l 500)
... files 2 vars 1421 all 0 lim 500 unch 0
1 - 512-64-allmodconfig
2 - 4096-512-allmodconfig
.1. .2. ..final..
1671168 +3899392 5570560 +233% irq_desc(.data.cacheline_aligned)
591872 +3899392 4491264 +658% irq_cfg(.data.read_mostly)
76800 +537600 614400 +700% early_node_map(.init.data)
66176 +462336 528512 +698% init_mem_cgroup(.bss)
65536 +458752 524288 +700% boot_pageset(.bss)
63648 +419328 482976 +658% kmalloc_caches(.data.cacheline_aligned)
15328 +61376 76704 +400% def_root_domain(.bss)
10240 +43008 53248 +420% change_point_list(.init.data)
8760 +504 9264 +5% init_task(.data)
8192 +57344 65536 +700% kgdb_info(.bss)
6404 +26880 33284 +419% e820_saved(.bss)
6404 +26880 33284 +419% e820(.bss)
6400 +26880 33280 +420% new_bios(.init.data)
5120 +35840 40960 +700% node_devices(.bss)
5120 +21504 26624 +420% change_point(.init.data)
4160 +29120 33280 +700% cpu_bit_bitmap(.rodata)
4096 +28672 32768 +700% __cpu_pda(.init.data)
3776 +25088 28864 +664% hstates(.bss)
3584 +25088 28672 +700% bootmem_node_data(.init.data)
2560 +10752 13312 +420% overlap_list(.init.data)
2048 +14336 16384 +700% x86_cpu_to_node_map_early_map(.init.data)
2048 +14336 16384 +700% was_in_debug_nmi(.bss)
2048 +14336 16384 +700% rio_devs(.init.data)
2048 +14336 16384 +700% passive_cpu_wait(.bss)
2048 +14336 16384 +700% node_memblk_range(.init.data)
2048 +14336 16384 +700% ints(.init.data)
2048 +14336 16384 +700% cpu_in_kgdb(.bss)
1024 +7168 8192 +700% x86_cpu_to_apicid_early_map(.init.data)
1024 +7168 8192 +700% x86_bios_cpu_apicid_early_map(.init.data)
1024 +1024 2048 +100% pxm_to_node_map(.data)
1024 +7168 8192 +700% nodes_add(.bss)
1024 +7168 8192 +700% nodes(.init.data)
512 +3584 4096 +700% zone_movable_pfn(.init.data)
512 +3584 4096 +700% tvec_base_done(.data)
512 +3584 4096 +700% scal_devs(.init.data)
512 +3584 4096 +700% node_data(.data.read_mostly)
512 +3584 4096 +700% memblk_nodeid(.init.data)
0 +2048 2048 . node_to_pxm_map(.data)
0 +2048 2048 . node_order(.bss)
0 +2048 2048 . node_load(.bss)
0 +2048 2048 . fake_node_to_pxm_map(.init.data)
0 +768 768 . rcu_ctrlblk(.data)
0 +768 768 . rcu_bh_ctrlblk(.data)
0 +768 768 . per_cpu__cpu_info(.data.percpu)
0 +768 768 . boot_cpu_data(.data.read_mostly)
0 +760 760 . per_cpu__phys_domains(.data.percpu)
0 +760 760 . per_cpu__node_domains(.data.percpu)
0 +760 760 . per_cpu__cpu_domains(.data.percpu)
0 +760 760 . per_cpu__core_domains(.data.percpu)
0 +760 760 . per_cpu__allnodes_domains(.data.percpu)
0 +720 720 . top_cpuset(.data)
0 +640 640 . per_cpu__flush_state(.data.percpu)
0 +632 632 . pit_clockevent(.data)
0 +632 632 . per_cpu__lapic_events(.data.percpu)
0 +632 632 . lapic_clockevent(.data)
0 +632 632 . hpet_clockevent(.data)
0 +616 616 . net_dma(.data)
0 +579 579 . do_migrate_pages(.text)
0 +568 568 . irq2(.data)
0 +568 568 . irq0(.data)
0 +528 528 . per_cpu__sched_group_phys(.data.percpu)
0 +528 528 . per_cpu__sched_group_cpus(.data.percpu)
0 +528 528 . per_cpu__sched_group_core(.data.percpu)
0 +528 528 . per_cpu__sched_group_allnodes(.data.percpu)
0 +520 520 . out_of_memory(.text)
0 +520 520 . nohz(.data)
0 +512 512 . tick_broadcast_oneshot_mask(.bss)
0 +512 512 . tick_broadcast_mask(.bss)
0 +512 512 . prof_cpu_mask(.data)
0 +512 512 . per_cpu__local_cpu_mask(.data.percpu)
0 +512 512 . per_cpu__cpu_sibling_map(.data.percpu)
0 +512 512 . per_cpu__cpu_core_map(.data.percpu)
0 +512 512 . nohz_cpu_mask(.bss)
0 +512 512 . mce_device_initialized(.bss)
0 +512 512 . mce_cpus(.bss)
0 +512 512 . marked_cpus(.bss)
0 +512 512 . kmem_cach_cpu_free_init_once(.bss)
0 +512 512 . irq_default_affinity(.data)
0 +512 512 . frozen_cpus(.bss)
0 +512 512 . fallback_doms(.bss)
0 +512 512 . cpu_singlethread_map(.data.read_mostly)
0 +512 512 . cpu_sibling_setup_map(.bss)
0 +512 512 . cpu_present_map(.data.read_mostly)
0 +512 512 . cpu_possible_map(.bss)
0 +512 512 . cpu_populated_map(.data.read_mostly)
0 +512 512 . cpu_online_map(.data.read_mostly)
0 +512 512 . cpu_mask_none(.bss)
0 +512 512 . cpu_mask_all(.data.read_mostly)
0 +512 512 . cpu_isolated_map(.bss)
0 +512 512 . cpu_initialized(.data)
0 +512 512 . cpu_callout_map(.bss)
0 +512 512 . cpu_callin_map(.bss)
0 +512 512 . cpu_active_map(.bss)
0 +512 512 . cache_dev_map(.bss)
0 +512 512 . c1e_mask(.bss)
0 +512 512 . backtrace_mask(.bss)2647360 +10283499 12930859 +388% Totals
====== Sections (-l 500)
... files 2 vars 36 all 0 lim 500 unch 0
1 - 512-64-allmodconfig
2 - 4096-512-allmodconfig
.1. .2. ..final..
66688274 +10345296 77033570 +15% Total
38237848 +44031 38281879 <1% .debug_info
8441752 +1215872 9657624 +14% .bss
2551715 +3136 2554851 <1% .text
1737600 +4318720 6056320 +248% .data.cacheline_aligned
1640096 +6784 1646880 <1% .data.percpu
1175061 +29104 1204165 +2% .rodata
1073400 +13712 1087112 +1% .debug_abbrev
901760 +1392 903152 <1% .debug_ranges
608192 +3906016 4514208 +642% .data.read_mostly
302704 +13504 316208 +4% .data
244896 +792112 1037008 +323% .init.data123603298 +20689679 144292977 +16% Totals
====== Text/Data ()
... files 2 vars 6 all 0 lim 0 unch 0
1 - 512-64-allmodconfig
2 - 4096-512-allmodconfig
.1. .2. ..final..
2551808 +2048 2553856 <1% TextSize
1679360 +43008 1722368 +2% DataSize
8441856 +1216512 9658368 +14% BssSize
2138112 +798720 2936832 +37% InitSize
1640448 +6144 1646592 <1% PerCPU
2383872 +8228864 10612736 +345% OtherSize18835456 +10295296 29130752 +54% Totals
====== PerCPU ()
... files 2 vars 22 all 0 lim 0 unch 0
1 - 512-64-allmodconfig
2 - 4096-512-allmodconfig .1. .2. ..final..
2048 -2048 . -100% vm_event_states
2048 -2048 . -100% softnet_data
2048 -2048 . -100% init_sched_rt_entity
2048 -2048 . -100% core_domains
0 +2048 2048 . sched_group_core
0 +2048 2048 . node_domains
0 +2048 2048 . lru_add_active_pvecs
0 +2048 2048 . init_rt_rq
0 +2048 2048 . cpu_domains
0 +2048 2048 . cpu_core_map
0 +2048 2048 . cpu_buffer8192 +6144 14336 +75% Totals
====== Stack (-l 500)
... files 2 vars 126 all 0 lim 500 unch 0
1 - 512-64-allmodconfig
2 - 4096-512-allmodconfig.1. .2. ..final..
0 +2712 2712 . smp_call_function_mask
0 +1576 1576 . setup_IO_APIC_irq
0 +1576 1576 . move_task_off_dead_cpu
0 +1560 1560 . arch_setup_ht_irq
0 +1560 1560 . __assign_irq_vector
0 +1544 1544 . tick_handle_oneshot_broadcast
0 +1544 1544 . msi_compose_msg
0 +1440 1440 . cpuset_write_resmask
0 +1352 1352 . store_scaling_governor
0 +1352 1352 . cpufreq_add_dev
0 +1320 1320 . cpufreq_update_policy
0 +1312 1312 . store_scaling_min_freq
0 +1312 1312 . store_scaling_max_freq
0 +1176 1176 . threshold_create_device
0 +1128 1128 . setup_IO_APIC
0 +1096 1096 . sched_balance_self
0 +1080 1080 . sched_rt_period_timer
0 +1080 1080 . _cpu_down
0 +1064 1064 . set_ioapic_affinity_irq
0 +1048 1048 . store_interrupt_enable
0 +1048 1048 . setup_timer_IRQ0_pin
0 +1048 1048 . setup_ioapic_dest
0 +1048 1048 . set_msi_irq_affinity
0 +1048 1048 . set_ht_irq_affinity
0 +1048 1048 . native_machine_crash_shutdown
0 +1048 1048 . native_flush_tlb_others
0 +1048 1048 . dmar_msi_set_affinity
0 +1040 1040 . store_threshold_limit
0 +1040 1040 . show_error_count
0 +1040 1040 . acpi_map_lsapic
0 +1032 1032 . tick_do_periodic_broadcast
0 +1032 1032 . sched_setaffinity
0 +1032 1032 . native_send_call_func_ipi
0 +1032 1032 . local_cpus_show
0 +1032 1032 . local_cpulist_show
0 +1032 1032 . irq_select_affinity
0 +1032 1032 . irq_complete_move
0 +1032 1032 . irq_affinity_proc_write
0 +1032 1032 . flush_tlb_mm
0 +1032 1032 . flush_tlb_current_task
0 +1032 1032 . fixup_irqs
0 +1032 1032 . create_irq
0 +1024 1024 . uv_vector_allocation_domain
0 +1024 1024 . uv_send_IPI_allbutself
0 +1024 1024 . store_error_count
0 +1024 1024 . physflat_send_IPI_allbutself
0 +1024 1024 . pci_bus_show_cpuaffinity
0 +1024 1024 . move_masked_irq
0 +1024 1024 . flush_tlb_page
0 +1024 1024 . flat_send_IPI_allbutself
0 +784 784 . sd_init_ALLNODES
0 +776 776 . sd_init_SIBLING
0 +776 776 . sd_init_NODE
0 +768 768 . sd_init_MC
0 +768 768 . sd_init_CPU
0 +728 728 . update_flag
0 +696 696 . init_intel_cacheinfo
0 +680 680 . __build_sched_domains
0 +648 648 . thread_return
0 +648 648 . schedule
0 +640 640 . cpuset_attach
0 +616 616 . rebalance_domains
0 +600 600 . select_task_rq_fair
0 +600 600 . cache_add_dev
0 +584 584 . shmem_getpage
0 +568 568 . pdflush
0 +552 552 . tick_notify
0 +552 552 . partition_sched_domains
0 +552 552 . free_sched_groups
0 +552 552 . __percpu_alloc_mask
0 +544 544 . taskstats_user_cmd
0 +536 536 . sched_init_smp
0 +536 536 . pci_device_probe
0 +536 536 . cpuset_common_file_read
0 +536 536 . cpupri_find
0 +536 536 . acpi_processor_ffh_cstate_probe
0 +536 536 . __cpu_disable
0 +520 520 . uv_send_IPI_all
0 +520 520 . tick_do_broadcast
0 +520 520 . smp_call_function
0 +520 520 . show_related_cpus
0 +520 520 . show_affected_cpus
0 +520 520 . prof_cpu_mask_write_proc
0 +520 520 . physflat_send_IPI_mask
0 +520 520 . physflat_send_IPI_all
0 +520 520 . native_smp_send_reschedule
0 +520 520 . native_send_call_func_single_ipi
0 +520 520 . lapic_timer_broadcast
0 +520 520 . irq_set_affinity
0 +520 520 . flat_vector_allocation_domain
0 +520 520 . flat_send_IPI_all
0 +520 520 . find_lowest_rq
0 +520 520 . cpuset_can_attach
0 +520 520 . cpu_callback
0 +520 520 . compat_sys_sched_setaffinity
0 +520 520 . add_del_listener
0 +512 512 . sys_sched_setaffinity
0 +512 512 . sys_sched_getaffinity
0 +512 512 . run_rebalance_domains
0 +512 512 . ioapic_retrigger_irq
0 +512 512 . generic_processor_info
0 +512 512 . force_quiescent_state
0 +512 512 . destroy_irq
0 +512 512 . default_affinity_write
0 +512 512 . cpu_to_phys_group
0 +512 512 . cpu_to_allnodes_group
0 +512 512 . compat_sys_sched_getaffinity
0 +512 512 . check_preempt_curr_rt
0 +512 512 . assign_irq_vector0 +92248 92248 +0% Totals
====== MemInfo ()
... files 0 vars 0 all 0 lim 0 unch 0
(runtime meminfo not collected.)
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 4 lines · Mike Travis
The need to allow distros to set NR_CPUS=4096 (and NODES_SHIFT=9) is critical to our upcoming SGI systems using what we have been calling "UV".
That's fine. You can do it. The default kernel will not, because it's clearly not safe.
I really don't care what you do to _your_ images. But I will not distribute a known-broken kernel, and I will not debug random stack overflows that happen in it.
If you want the default kernel to support 4k cores, we'll need to fix the stack usage. I don't think that is impossible, but IT IS NOT GOING TO HAPPEN for 2.6.27.
And quite frankly, if some vendor like RedHat enables NR_CPUS=4096 by default, they are totally and utterly crazy.
But some SGI-specific binary that is meant for SGI machines only, and has been extensively tested with the setup used on SGI machines is a different thing.
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 6 lines · Linus Torvalds
If you want the default kernel to support 4k cores, we'll need to fix the stack usage. I don't think that is impossible, but IT IS NOT GOING TO HAPPEN for 2.6.27.
And quite frankly, if some vendor like RedHat enables NR_CPUS=4096 by default, they are totally and utterly crazy.
heh. *picks through Fedora changelog*
* Thu Aug 14 2008 Dave Jones <[email protected]>
- Bump max cpus supported on x86-64 to 4096. Just to see what happens.
I never did get to find out unfortunatly, because of the security fiasco in Fedora infrastructure the last week or two.
Quoted 3 lines
But some SGI-specific binary that is meant for SGI machines only, and has been extensively tested with the setup used on SGI machines is a different thing.
Every extra kernel image a distro vendor ends up shipping has an associated cost.
* build time: It currently takes about 2 hours for a set of Fedora RPMs. For RHEL it'll be even worse due to the extra archs). Killing off -smp specific builds was a big win for us in this regard. Adding extra flavours is always painful.
* diskspace (distro kernels aren't small. With the associated debugging symbols, they take up a shitload of disk space really fast).
* Having everyone running the same kernel makes it much easier to test/debug. Our QA guys hate adding extra columns to their test matrix.
But yes, for this to be even remotely feasible, there has to be a negligable performance cost associated with it, which right now, we clearly don't have. Given that the number of people running 4096 CPU boxes even in a few years time will still be tiny, punishing the common case is obviously absurd.
Dave
Signature · 1 line
http://www.codemonkey.org.uk
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Dave Jones wrote:
...
Quoted 8 lines
But yes, for this to be even remotely feasible, there has to be a negligable performance cost associated with it, which right now, we clearly don't have. Given that the number of people running 4096 CPU boxes even in a few years time will still be tiny, punishing the common case is obviously absurd.
Dave
I did do some fairly extensive benchmarking between configs of NR_CPUS = 128 and 4096 and most performance hits were in the neighborhood of < 5% on systems with 8 cpus and 4GB of memory (our most common test system). [But changing cpumask_t's to be pointers instead of values will likely increase this.] I've tried to be very sensitive to this issue with all my previous changes, so convincing the distros to set NR_CPUS=4096 would be as painless for them as possible. ;-)
Btw, huge count cpu systems I don't think are that far away. I believe the nextgen Larabbee chips will be geared towards HPC applications [instead of just GFX apps], and putting 4 of these chips on a motherboard would add up to 512 cpu threads (1024 if they support hyperthreading.)
Thanks,
Mike
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 9 lines · Mike Travis
Dave Jones wrote:
...Quoted 6 lines
But yes, for this to be even remotely feasible, there has to be a negligable performance cost associated with it, which right now, we clearly don't have. Given that the number of people running 4096 CPU boxes even in a few years time will still be tiny, punishing the common case is obviously absurd.
Quoted 5 lines
Quoted 1 line
Dave
I did do some fairly extensive benchmarking between configs of
NR_CPUS= 128 and 4096 and most performance hits were in the neighborhood of < 5% on systems with 8 cpus and 4GB of memory (our most common test system).
5% is a pretty nasty performance hit... what sort of benchmarks are we talking about here?
I just made some pretty crazy changes to the VM to get "only" around 5 or so % performance improvement in some workloads.
What places are making heavy use of cpumasks that causes such a slowdown? Hopefully callers can mostly be improved so they don't need to use cpumasks for common cases.
Until then, it would be kind of sad for a distro to ship a generic x86 kernel and lose 5% performance because it is set to 4096 CPUs...
But if I misunderstand and you're talking about specific microbenchmarks to find the worst case for huge cpumasks, then I take that back.
Quoted 10 lines
[But
changing cpumask_t's to be pointers instead of values will likely increase this.] I've tried to be very sensitive to this issue with all my previous changes, so convincing the distros to set NR_CPUS=4096 would be as painless for them as possible. ;-)Btw, huge count cpu systems I don't think are that far away. I believe the nextgen Larabbee chips will be geared towards HPC applications [instead of just GFX apps], and putting 4 of these chips on a motherboard would add up to 512 cpu threads (1024 if they support hyperthreading.)
It would be quite interesting if they make them cache coherent / MP capable. Will they be?
David Miller
maintainerRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
From: Nick Piggin <[email protected]>
Date: Wed, 27 Aug 2008 16:54:32 +1000
Quoted 9 lines
5% is a pretty nasty performance hit... what sort of benchmarks are we talking about here?
I just made some pretty crazy changes to the VM to get "only" around 5 or so % performance improvement in some workloads.
What places are making heavy use of cpumasks that causes such a slowdown? Hopefully callers can mostly be improved so they don't need to use cpumasks for common cases.
It's almost certainly from the cross-call dispatch call chain.
As just one example, just to do a TLB flush mm->cpu_vm_mask probably gets passed around as an aggregate two or three times on the way down to the APIC programming code on x86. That's two or three 512 byte copies on the stack :)
Look at the sparc64 SMP code for how I solved the problem there.
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 13 lines · David Miller
From: Nick Piggin <nickpiggin-/[email protected]>
Date: Wed, 27 Aug 2008 16:54:32 +1000Quoted 9 lines
5% is a pretty nasty performance hit... what sort of benchmarks are we talking about here?
I just made some pretty crazy changes to the VM to get "only" around 5 or so % performance improvement in some workloads.
What places are making heavy use of cpumasks that causes such a slowdown? Hopefully callers can mostly be improved so they don't need to use cpumasks for common cases.
Quoted 6 lines
It's almost certainly from the cross-call dispatch call chain.
As just one example, just to do a TLB flush
mm->cpu_vm_maskprobably gets passed around as an aggregate two or three times on the way down to the APIC programming code on x86. That's two or three 512 byte copies on the stack :)
Yeah, I see. That's stupid isn't it? (Well, I guess it was completely sane when cpumasks were word sized ;))
Hopefully that accounts for a significant chunk...
David Miller
maintainerRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
From: Nick Piggin <nickpiggin-/[email protected]>
Date: Wed, 27 Aug 2008 17:47:14 +1000
Quoted 4 lines
Yeah, I see. That's stupid isn't it? (Well, I guess it was completely sane when cpumasks were word sized ;))
Hopefully that accounts for a significant chunk...
There is a lot of indirect costs that are hard to see as well.
Two things a lot of these cross-call dispatch paths do is:
1) Clear self-cpu
2) AND with cpus_online
#1 can normally be a simple bit clear, but some places can also implement this with something like "cpus_andn(X, cpumask_of_cpu(cpu))"
It's simply easier to move those two things down to the bottom of the APIC programming code, they just loop over the cpumask doing an expensive APIC I/O operation anyways, might as well overlap it with these "skip self-cpu" and "skip not-online cpus" checks.
And oh yeah we get the stack wastage fixed too, isn't what what we were talking about? :-)
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 19 lines · David Miller
From: Nick Piggin <nickpiggin-/[email protected]>
Date: Wed, 27 Aug 2008 17:47:14 +1000Quoted 4 lines
Yeah, I see. That's stupid isn't it? (Well, I guess it was completely sane when cpumasks were word sized ;))
Hopefully that accounts for a significant chunk...
There is a lot of indirect costs that are hard to see as well.
Two things a lot of these cross-call dispatch paths do is:
1) Clear self-cpu
2) AND with
cpus_online#1 can normally be a simple bit clear, but some places can also implement this with something like "
cpus_andn(X, cpumask_of_cpu(cpu))"
Quoted 7 lines
It's simply easier to move those two things down to the bottom of the APIC programming code, they just loop over the cpumask doing an expensive APIC I/O operation anyways, might as well overlap it with these "skip self-cpu" and "skip not-online cpus" checks.
And oh yeah we get the stack wastage fixed too, isn't what what we were talking about? :-)
Yes, the most time consuming part was determining whether a kmalloc could safely be used in the context of the function, and what to do about the out-of-memory problem. Pushing that down to something
like: for_each_cpu_thats_online(cpu, *maskptr) would remove the need for many of the temp masks. A simple if (cpu != me) would take care of excluding self. It might have better interaction with cpu hotplug as well, since the online map would be checked just before the call to that cpu is made.
Thanks,
Mike
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to David MillerQuoted 13 lines · David Miller
From: Nick Piggin <nickpiggin-/[email protected]>
Date: Wed, 27 Aug 2008 16:54:32 +1000Quoted 9 lines
5% is a pretty nasty performance hit... what sort of benchmarks are we talking about here?
I just made some pretty crazy changes to the VM to get "only" around 5 or so % performance improvement in some workloads.
What places are making heavy use of cpumasks that causes such a slowdown? Hopefully callers can mostly be improved so they don't need to use cpumasks for common cases.
Quoted 8 lines
It's almost certainly from the cross-call dispatch call chain.
As just one example, just to do a TLB flush
mm->cpu_vm_maskprobably gets passed around as an aggregate two or three times on the way down to the APIC programming code on x86. That's two or three 512 byte copies on the stack :)Look at the sparc64 SMP code for how I solved the problem there.
I will, thanks!
Mike
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Nick PigginQuoted 10 lines · Nick Piggin
Quoted 9 lines · Mike Travis
Dave Jones wrote:
...Quoted 6 lines
But yes, for this to be even remotely feasible, there has to be a negligable performance cost associated with it, which right now, we clearly don't have. Given that the number of people running 4096 CPU boxes even in a few years time will still be tiny, punishing the common case is obviously absurd.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Dave
I did do some fairly extensive benchmarking between configs of
NR_CPUS= 128 and 4096 and most performance hits were in the neighborhood of < 5% on systems with 8 cpus and 4GB of memory (our most common test system).5% is a pretty nasty performance hit... what sort of benchmarks are we talking about here?
It's been a while now, I should go back and check my notes. Many of the BM's did not have any changes. I believe the ones that were right on the edge of paging were affected by the fact that less memory was available.
Quoted 7 lines
I just made some pretty crazy changes to the VM to get "only" around 5 or so % performance improvement in some workloads.
What places are making heavy use of cpumasks that causes such a slowdown? Hopefully callers can mostly be improved so they don't need to use cpumasks for common cases.
That's another study I did, and it seemed that maybe 95% of the functions would not be affected by passing pointers to cpumasks instead of the cpumasks themselves, because the data was processed by a cpu_xxx function that uses a pointer. Most commonly was to create a temp cpumask, using cpus_and(temp_mask, callers_mask, cpu_online_map); The speedup to use nr_cpu_ids instead of NR_CPUS in the traversal functions helped quite a bit. Using this same method in the cpus_xxx functions would further speed up things. (As well as only allocating the cpumask sized by nr_cpu_ids instead of NR_CPUS as the current cpumask_t definition specifies.)
Quoted 6 lines
Until then, it would be kind of sad for a distro to ship a generic x86 kernel and lose 5% performance because it is set to 4096 CPUs...
But if I misunderstand and you're talking about specific microbenchmarks to find the worst case for huge cpumasks, then I take that back.
Yes, I was (at the time) trying to determine how many of the cpumask functions were actually in play by user tasks, so I was zeroing in on those (cpusets, rescheds, etc.)
Quoted 8 lines
Quoted 8 lines
[But
changing cpumask_t's to be pointers instead of values will likely increase this.] I've tried to be very sensitive to this issue with all my previous changes, so convincing the distros to set NR_CPUS=4096 would be as painless for them as possible. ;-)
Quoted 7 lines
Quoted 4 lines
Btw, huge count cpu systems I don't think are that far away. I believe the nextgen Larabbee chips will be geared towards HPC applications [instead of just GFX apps], and putting 4 of these chips on a motherboard would add up to 512 cpu threads (1024 if they support hyperthreading.)
It would be quite interesting if they make them cache coherent / MP capable. Will they be?
There's not been a lot of info available yet, but I think the 128 cores will share at least an L2 cache + memory controller. How the APIC's interact is also another big question. And most likely some standard system controller CPU will be needed, but that could be a tiny VIA processor... ;-)
Thanks,
Mike
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Linus TorvaldsQuoted 17 lines · Linus Torvalds
Quoted 3 lines · Mike Travis
The need to allow distros to set NR_CPUS=4096 (and NODES_SHIFT=9) is critical to our upcoming SGI systems using what we have been calling "UV".
That's fine. You can do it. The default kernel will not, because it's clearly not safe.
I really don't care what you do to _your_ images. But I will not distribute a known-broken kernel, and I will not debug random stack overflows that happen in it.
If you want the default kernel to support 4k cores, we'll need to fix the stack usage. I don't think that is impossible, but IT IS NOT GOING TO HAPPEN for 2.6.27.
Quoted 8 lines
And quite frankly, if some vendor like RedHat enables NR_CPUS=4096 by default, they are totally and utterly crazy.
But some SGI-specific binary that is meant for SGI machines only, and has been extensively tested with the setup used on SGI machines is a different thing.
Linus
Ok, thanks for the reply, and looking into this issue. We will "strongly encourage" our distros to base the relevant releases on 2.6.28. :-)
[Supplying an SGI-specific kernel would not be acceptable to many of our customers because of the certification issues I mentioned.]
Mike
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Linus TorvaldsQuoted 42 lines · Linus Torvalds
Quoted 1 line · Linus Torvalds
But I'll look at your vmlinux, see what stands out.
Oops. I already see the problem.
Your .config has soem _huge_ CPU count, doesn't it?
checkstack.pl shows these things as the top problems:
0xffffffff80266234 smp_call_function_mask [vmlinux]: 2736 0xffffffff80234747 __build_sched_domains [vmlinux]: 2232 0xffffffff8023523f __build_sched_domains [vmlinux]: 2232 0xffffffff8021e884 setup_IO_APIC_irq [vmlinux]: 1616 0xffffffff8021ee24 arch_setup_ht_irq [vmlinux]: 1600 0xffffffff8021f144 arch_setup_msi_irq [vmlinux]: 1600 0xffffffff8021e3b0 __assign_irq_vector [vmlinux]: 1592 0xffffffff8021e626 __assign_irq_vector [vmlinux]: 1592 0xffffffff8023257e move_task_off_dead_cpu [vmlinux]: 1592 0xffffffff802326e8 move_task_off_dead_cpu [vmlinux]: 1592 0xffffffff8025dbc5 tick_handle_oneshot_broadcast [vmlinux]:1544 0xffffffff8025dcb4 tick_handle_oneshot_broadcast [vmlinux]:1544 0xffffffff803f3dc4 store_scaling_governor [vmlinux]: 1376 0xffffffff80279ef4 cpuset_write_resmask [vmlinux]: 1360 0xffffffff803f465d cpufreq_add_dev [vmlinux]: 1352 0xffffffff803f495b cpufreq_add_dev [vmlinux]: 1352 0xffffffff803f3fc4 store_scaling_max_freq [vmlinux]: 1328 0xffffffff803f4064 store_scaling_min_freq [vmlinux]: 1328 0xffffffff803f44c4 cpufreq_update_policy [vmlinux]: 1328 ..and
sys_init_moduleis actually way way down the list. I bet the only reason it showed up at all was because dynamically it was such a deep callchain, and part of that callchain probably called some of those really nasty things.Anyway, the reason
smp_call_function_maskand friends have such _huge_ stack usages for you is that they contain a 'cpumask_t' on the stack.For example, for me, usign a sane
NR_CPU, the size of the stack frame forsmp_call_function_maskis under 200 bytes. For you, it's 2736 bytes.
Quoted 7 lines
How about you make CONFIG_NR_CPU's something _sane_? Like 16? Or do you really have four thousand CPU's in that system?
Oh, I guess you have the MAXSMP config enabled? I really think that was a bit too aggressive.
Linus
This probably all started when I was working on a software tool (aiod) that was failing because somebody ELSE had 4,096 CPUs configured. [[Seems that gcc had/has? it's MAX CPU value set to 1,024 (bits/sched.h __CPU_SETSIZE), so when you issue system calls like sched_getaffinity, it will "fail" for systems configured w/ 4,096 CPUs. I worked around it by simply forgetting about the gcc values, and kept allocating larger CPU masks until it worked.]]
I think you're right: the kernel as a whole may not be ready for 4,096 CPUs apparently...
Thanks for taking the time to look into this...
Alan
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 2 lines · Alan D. Brunelle
I think you're right: the kernel as a whole may not be ready for 4,096 CPUs apparently...
Mike has been working diligently on getting all these cpumasks off the stack for the last months and has created an infrastructure to do this. So I think we are close. It might just be a matter of merging some more patches that are still left in Ingo's tree.
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 9 lines · Christoph Lameter
Quoted 2 lines · Alan D. Brunelle
I think you're right: the kernel as a whole may not be ready for 4,096 CPUs apparently...
Mike has been working diligently on getting all these cpumasks off the stack for the last months and has created an infrastructure to do this. So I think we are close. It might just be a matter of merging some more patches that are still left in Ingo's tree.
hm, there are no such patches left that i know of - the only bits in -tip are the zero-based percpu, which was found to be a bit fragile in testing:
earth4:~/tip> git-log-line --author=Travis linus..
d379497: Zero based percpu: infrastructure to rebase the per cpu area to zero
b3a0cb4: x86: extend percpu ops to 64 bit
[and it has no relevance to stack footprint.]
So i dont think the current cpumask_t approach will work. We simply should not get into an endless fight against the windmills that introduce on-stack cpumask_t again and again. We should just take the plunge once and do a clean alloc/free cpumask model. Most of the hotpath cpumasks are constant or pre-constructed, so they are not a real issue.
Plus, on the general question of stack footprint problems and the difficulty of debugging them, the worst-case stack footprint tracer i wrote for -rt some time ago should be dusted off as well and put into ftrace. David has something quite close to that for Sparc64 already.
Ingo
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 6 lines · Ingo Molnar
Quoted 4 lines · Christoph Lameter
Quoted 2 lines · Alan D. Brunelle
I think you're right: the kernel as a whole may not be ready for 4,096 CPUs apparently...
Quoted 8 lines
Quoted 4 lines
Mike has been working diligently on getting all these cpumasks off the stack for the last months and has created an infrastructure to do this. So I think we are close. It might just be a matter of merging some more patches that are still left in Ingo's tree.
hm, there are no such patches left that i know of - the only bits in -tip are the zero-based percpu, which was found to be a bit fragile in testing:
Yes, it's just a case of new changes abusing the stack.
Quoted 5 lines
earth4:~/tip> git-log-line --author=Travis linus..
d379497: Zero based percpu: infrastructure to rebase the per cpu area to zero
b3a0cb4: x86: extend percpu ops to 64 bit
Quoted 7 lines
[and it has no relevance to stack footprint.]
So i dont think the current
cpumask_tapproach will work. We simply should not get into an endless fight against the windmills that introduce on-stackcpumask_tagain and again. We should just take the plunge once and do a clean alloc/free cpumask model. Most of the hotpath cpumasks are constant or pre-constructed, so they are not a real issue.
It would have been nice to know this 9 months ago... ;-)
Quoted 7 lines
Plus, on the general question of stack footprint problems and the difficulty of debugging them, the worst-case stack footprint tracer i wrote for -rt some time ago should be dusted off as well and put into ftrace. David has something quite close to that for Sparc64 already.
Ingo
I'll start experimenting with globally changing cpumask_t to be a pointer, and see what falls out.
Thanks,
Mike
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Linus TorvaldsQuoted 1 line · On Tuesday 26 August 2008 06:43:03 Linus Torvalds wrote:
So now
load_module()will still use almost 500 bytes of stack
Hmm, wants neatening anyway; I'll see if I can reduce stack usage side effect.
Your workaround is very random, and that scares me. I think a huge number of CPUs needs a real solution (an actual cpumask allocator, then do something clever if we come across an actual fastpath).
Thanks,
Rusty.
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 4 lines · Rusty Russell
Your workaround is very random, and that scares me. I think a huge number of CPUs needs a real solution (an actual cpumask allocator, then do something clever if we come across an actual fastpath).
The thing is, the inlining thing is a separate issue.
Yes, the cpumasks were what made stack pressure so critical to begin with, but no, a 400-byte stack frame in a deep callchain isn't acceptable _regardless_ of any cpumask_t issues.
Gcc inlining is a total and utter pile of shit. And _that_ is the problem. I seriously think we shouldn't allow gcc to inline anything at all unless we tell it to. That's how it used to work, and quite frankly, that's how it _should_ work.
The downsides of inlining are big enough from both a debugging and a real code generation angle (eg stack usage like this), that the upsides (_somesimes_ smaller kernel, possibly slightly faster code) simply aren't relevant.
So the "noinline" was random, yes, but this is a real issue. Looking at checkstack output for a saner config (NR_CPUS=16), the top entries for me are things like
ide_generic_init [vmlinux]: 1384 idefloppy_ioctl [vmlinux]: 1208 e1000_check_options [vmlinux]: 1152 ...
which are "leaf" functions. They are broken as hell (the e1000 is apparently because it builds structs on the stack that should all be "static const", for example), but they are different from something like the module init sequence in that they are not going to affect anything else.
It would be interesting to see what "-fno-default-inline" does to the kernel. It really would get rid of a _lot_ of gcc version issues too. Inlining behavior of gcc has long been a problem for us.
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 19 lines · Linus Torvalds
Quoted 4 lines · Rusty Russell
Your workaround is very random, and that scares me. I think a huge number of CPUs needs a real solution (an actual cpumask allocator, then do something clever if we come across an actual fastpath).
The thing is, the inlining thing is a separate issue.
Yes, the cpumasks were what made stack pressure so critical to begin with, but no, a 400-byte stack frame in a deep callchain isn't acceptable _regardless_ of any
cpumask_tissues.Gcc inlining is a total and utter pile of shit. And _that_ is the problem. I seriously think we shouldn't allow gcc to inline anything at all unless we tell it to. That's how it used to work, and quite frankly, that's how it _should_ work.
Quoted 8 lines
The downsides of inlining are big enough from both a debugging and a real code generation angle (eg stack usage like this), that the upsides (_somesimes_ smaller kernel, possibly slightly faster code) simply aren't relevant.
...
It would be interesting to see what "-fno-default-inline" does to the kernel. It really would get rid of a _lot_ of gcc version issues too. Inlining behavior of gcc has long been a problem for us.
I added "-fno-inline-functions-called-once -fno-early-inlining" to KBUILD_CFLAGS, and (with gcc 4.3) that increased the size of my kernel image by 2%.
And when David's "-fwhole-program --combine" will become ready the cost of disallowing gcc to inline functions will most likely increase.
A debugging option (for better traces) to disallow gcc some inlining might make sense (and might even make sense for distributions to enable in their kernels), but when you go to use cases that require really small kernels the cost is too high.
But if you don't trust gcc's inlining you should revert commit 3f9b5cc018566ad9562df0648395649aebdbc5e0 that increases gcc's freedom regarding what to inline in 2.6.27 - what gcc 4.2 does in the case of the regression tracked as Bugzilla #11276 is really not funny (two callers -> function not inlined; gcc seems to emit the function although both callers later get removed (or at least should be removed) by dead code elimination).
Quoted 1 line
Linus
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 5 lines · Adrian Bunk
A debugging option (for better traces) to disallow gcc some inlining might make sense (and might even make sense for distributions to enable in their kernels), but when you go to use cases that require really small kernels the cost is too high.
You ignore the fact that it's really not just about debugging.
Inlining really isn't the great tool some people think it is. Especially not since gcc stack allocation is so horrid that it won't re-use stack slots etc (which I don't disagree with per se - it's _hard_ to re-use stack slots while still allowing code scheduling).
NOTE! I also would never claim that _our_ choices of "inline" are all that great, and we've often inlined too much or not inlined things that really could be inlined. But at least when a developer says "inline" (or forgets to say it), we have somebody to blame. When the compiler does insane things that doesn't suit us, we're just screwed.
Quoted 3 lines
But if you don't trust gcc's inlining you should revert commit
3f9b5cc018566ad9562df0648395649aebdbc5e0that increases gcc's freedom regarding what to inline in 2.6.27
Actually, that just allows gcc to _not_ inline. Which is probably ok.
(Well, it would be ok if gcc did it well enough, it obviously has some problems at times).
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 10 lines · Linus Torvalds
Quoted 5 lines · Adrian Bunk
A debugging option (for better traces) to disallow gcc some inlining might make sense (and might even make sense for distributions to enable in their kernels), but when you go to use cases that require really small kernels the cost is too high.
You ignore the fact that it's really not just about debugging.
I had in mind that we anyway have to support it for tiny kernels.
I simply don't see that we add kconfig options for 5kB of code for tiny kernels but remove something like this that can cause size increases > 1%.
Quoted 4 lines
Inlining really isn't the great tool some people think it is. Especially not since gcc stack allocation is so horrid that it won't re-use stack slots etc (which I don't disagree with per se - it's _hard_ to re-use stack slots while still allowing code scheduling).
gcc's stack allocation has become better
(that's why we disable unit-at-a-time only for gcc 3.4 on i386).
Quoted 5 lines
NOTE! I also would never claim that _our_ choices of "inline" are all that great, and we've often inlined too much or not inlined things that really could be inlined. But at least when a developer says "inline" (or forgets to say it), we have somebody to blame. When the compiler does insane things that doesn't suit us, we're just screwed.
Most LOCs of the kernel are not written by people like you or Al Viro or David Miller, and the average kernel developer is unlikely to do it as good as gcc.
For the average driver the choice is realistically between "inline's randomly sprinkled across the driver" and "no inline's, leave it to gcc".
And code evolves during the years from tiny with 1 caller to huge with many callers.
BTW:
I just ran checkstack on a (roughly) allyesconfig kernel, and we have a new driver that allocates "unsigned char recvbuf[1500];" on the stack...
Quoted 8 lines
Quoted 3 lines
But if you don't trust gcc's inlining you should revert commit
3f9b5cc018566ad9562df0648395649aebdbc5e0that increases gcc's freedom regarding what to inline in 2.6.27Actually, that just allows gcc to _not_ inline. Which is probably ok.
(Well, it would be ok if gcc did it well enough, it obviously has some problems at times).
With the "gcc inline's static functions" you complain about we have 4-5 years of experience.
Suddenly allowing 4 release series of gcc to ignore any inline's is a completely new area for us. I'd generally agree with giving gcc more freedom here, but I'd rather do it right by removing tons of wrong inline's than doing one global change hoping that it will make things better.
And whether the "optimized inlining" actually makes the kernel bigger or smaller depends in my experience on the .config and the gcc version.
Quoted 1 line
Linus
cu
Adrian
[1] there are some rare exceptions
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 2 lines · Adrian Bunk
I had in mind that we anyway have to support it for tiny kernels.
I actually don't think that is true.
If we really were to decide to be stricter about it, and it makes a big size difference, we can probably also add a tool to warn about functions that really should be inline.
Quoted 7 lines
Quoted 4 lines
Inlining really isn't the great tool some people think it is. Especially not since gcc stack allocation is so horrid that it won't re-use stack slots etc (which I don't disagree with per se - it's _hard_ to re-use stack slots while still allowing code scheduling).
gcc's stack allocation has become better
(that's why we disable unit-at-a-time only for gcc 3.4 on i386).
I agree that it has become better. But it still absolutely *sucks*.
For example, see the patch I just posted about e1000 stack usage. Even though the variables were all in completely separate scopes, they all got individual space on the stack over the whole lifetime of the function, causing an explosion of stack-space. As such, gcc used 500 bytes too much of stack, just because it didn't re-use the stackspace.
That was with gcc-4.3.0, and no, there were hardly any inlining issues involevd, although it is true that inlining actually did make it slightly worse in that case too (but since it was essentially a leaf function, that had little real life impact, since there were no deep callchains below it to care).
So the fact is, "better" simply is not "good enough". We still need to do a lot of optimizations _manually_, because gcc cannot see that it can re-use the stack-slots.
And sometimes those "optimizations" are actually performance pessimizations, because in order to make gcc not use all the stack at the same time, you simply have to break things out and force-disable inlining.
Quoted 3 lines
Most LOCs of the kernel are not written by people like you or Al Viro or David Miller, and the average kernel developer is unlikely to do it as good as gcc.
Sure. But we do have tools. We do have checkstack.pl, it's just that it hasn't been an issue in a long time, so I suspect many people didn't even _realize_ we have it, and I certainly can attest to the fact that even people who remember it - like me - don't actually tend to run it all that often.
Quoted 3 lines
For the average driver the choice is realistically between "inline's randomly sprinkled across the driver" and "no inline's, leave it to gcc".
And neither is likely to be a big problem.
Quoted 3 lines
BTW:
I just ran checkstack on a (roughly) allyesconfig kernel, and we have a new driver that allocates "unsigned char recvbuf[1500];" on the stack...
Yeah, it's _way_ too easy to do bad things.
Quoted 2 lines
With the "gcc inline's static functions" you complain about we have 4-5 years of experience.
Sure. And most of it isn't all that great.
But I do agree that lettign gcc make more decisions is _dangerous_. However, in this case, at least, the decisions it makes would at least make for less inlining, and thus less stack space explosion.
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 9 lines · Linus Torvalds
Quoted 3 lines
Most LOCs of the kernel are not written by people like you or Al Viro or David Miller, and the average kernel developer is unlikely to do it as good as gcc.
Sure. But we do have tools. We do have checkstack.pl, it's just that it hasn't been an issue in a long time, so I suspect many people didn't even _realize_ we have it, and I certainly can attest to the fact that even people who remember it - like me - don't actually tend to run it all that often.
Sounds like what's really desired here isn't more worry and unpredictability, but for GCC+Binutils to gain the ability to calculate the stack depth over all callchains (doesn't have to be exact, just an upper bound; annotate recursions) in a way that's good enough to do on every compile, complain if a depth is exceeded statically (or it can't be proven), and to gain the architecture-independent option "optimise to reduce stack usage".
Quoted 5 lines
Quoted 3 lines
BTW:
I just ran checkstack on a (roughly) allyesconfig kernel, and we have a new driver that allocates "unsigned char recvbuf[1500];" on the stack...Yeah, it's _way_ too easy to do bad things.
In my userspace code, I have macros tmp_alloc and tmp_free. They must be matched in the same function:
unsigned char * recvbuf = tmp_alloc(1500);
....
tmp_free(recvbuf);When stack is plentiful, it maps to alloca() which is roughly equivalent to using a stack variable.
When stack is constrained (as it is on my little devices), that maps to xmalloc/free. The kernel equivalent would be kmalloc GFP_ATOMIC (perhaps).
With different macros to mine, it may be possible to map small fixed-size requests exactly onto local variables, and large ones to kmalloc(). A stab at it (not tested):
#define LOCAL_ALLOC_THRESHOLD 128
#define LOCAL_ALLOC(type, ptr) \
__typeof__(type) __attribute__((__unused__)) ptr##_local_struct; \
__typeof__(type) * ptr = \
((__builtin_constant_p(sizeof(type)) \
&& sizeof(type) <= LOCAL_ALLOC_THRESHOLD) \
? &ptr##_local_struct : kmalloc(sizeof(type), GFP_ATOMIC))
#define LOCAL_FREE(ptr) \
((__builtin_constant_p(sizeof (*(ptr))) \
&& sizeof(*(ptr)) <= LOCAL_ALLOC_THRESHOLD) \
? (void) 0 : kfree(ptr))Would that be useful in the kernel?
I'm thinking if it were a commonly used pattern for temporary buffers, unknown structures and arrays of macro-determined size, the "new driver" author would be less likely to accidentally drop a big object on the stack.
Obviously it would be nicer for GCC to code such a thing automatically, but that really is wishful thinking.
-- Jamie
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Adrian BunkQuoted 4 lines · Adrian Bunk
I added "-fno-inline-functions-called-once -fno-early-inlining" to
KBUILD_CFLAGS, and (with gcc 4.3) that increased the size of my kernel image by 2%.
Btw, did you check with just "-fno-inline-functions-called-once"?
The -fearly-inlining decisions _should_ be mostly right. If gcc sees early that a function is so small (even without any constant propagation etc) that it can be inlined, it's probably right.
The inline-functions-called-once thing is what causes even big functions to be inlined, and that's where you find the big downsides too (eg the stack usage).
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 3 lines · Linus Torvalds
The inline-functions-called-once thing is what causes even big functions to be inlined, and that's where you find the big downsides too (eg the stack usage).
That's a bit bizarre, though, isn't it?
A function which is only called from one place should, if everything made sense, _never_ use more stack through being inlined. Inlining should just increase the opportunities that the called function's local variables can share the same stack slots are the caller's dead locals.
Whereas not inlining guarantees they occupy separate, immediately adjacent regions of the stack, and shouldn't be increasing the total numbers of local variables.
-- Jamie
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 3 lines · Jamie Lokier
A function which is only called from one place should, if everything made sense, _never_ use more stack through being inlined.
But that's simply not true.
See the whole discussion.
The problem is that if you inline that function, the stack usage of the newly inlined function is now added to ALL THE OTHER paths too!
So the case we had in module loading was that yes, we had a function with a big stack footprint, but it was NOT in the deep path.
But by inlining it, it now moved the stack footprint "up" one level to another function, and now the big stack footprint really _was_ in the deep path, because the caller was involved in a much deeper chain.
So inlining moves the code up the callchain, and that is a problem for the backtrace, but that's "just" a debugging issue. But it also moves the stack footprint up the callchain, and that can actually be a correctness issue.
Of course, a compiler doesn't _have_ to do that. A compiler _could_ have multiple different stack footprints for a single function, and do liveness analysis etc. But no sane compiler probably does that, because it's very painful indeed, and it's not even an issue if you aren't stack-limited (and being stack-limited is really just a kernel thing).
(Yeah, it can be an issue even if you have a big stack, in that you get worse cache behaviour, so a dense stack footprint _would_ help. But the complexity of stack liveness analysis is almost certainly not worth the relatively small gains it would get on some odd cases).
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Linus TorvaldsQuoted 10 lines · Linus Torvalds
Quoted 4 lines · Adrian Bunk
I added "-fno-inline-functions-called-once -fno-early-inlining" to
KBUILD_CFLAGS, and (with gcc 4.3) that increased the size of my kernel image by 2%.Btw, did you check with just "-fno-inline-functions-called-once"?
Quoted 7 lines
The -fearly-inlining decisions _should_ be mostly right. If gcc sees early that a function is so small (even without any constant propagation etc) that it can be inlined, it's probably right.
The inline-functions-called-once thing is what causes even big functions to be inlined, and that's where you find the big downsides too (eg the stack usage).
-fno-inline-functions-called-once alone costs me nearly 1% in code size.
And I'd expect it to become more with "-fwhole-program --combine".
If you think we have too many stacksize problems I'd suggest to consider removing the choice of 4k stacks on i386, sh and m68knommu instead of using -fno-inline-functions-called-once:
Now that 32bit x86 is no longer used for extreme highend configurations the only serious usecase for 4k stacks are AFAIK space savings on embedded archs.
4k stacks have caused us much pain [1], and the cases where gcc inlined too much were the easy ones.
I'm not saying that I'd like removing the choice of 4k stacks, but if we want to reduce the number of stack related problems that's IMHO the better alternative.
Quoted 1 line
Linus
cu
Adrian
[1] AFAIR some callpaths in the kernel are still too big
BTW: In case anyone wonders about why I suggest removing 4k stacks:
My position is that 4k stacks should either be enabled unconditionally or no longer offered at all.
And if we remove 4k stacks from 32bit x86 it's no longer realistically maintainable for other architectures.
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 4 lines · Adrian Bunk
If you think we have too many stacksize problems I'd suggest to consider removing the choice of 4k stacks on i386, sh and m68knommu instead of using -fno-inline-functions-called-once:
Don't be silly. That makes the problem _worse_.
We're much better off with a 1% code-size reduction than forcing big stacks on people. The 4kB stack option is also a good way of saying "if it works with this, then 8kB is certainly safe".
And embedded people (the ones that might care about 1% code size) are the ones that would also want smaller stacks even more!
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 2 lines · Linus Torvalds
And embedded people (the ones that might care about 1% code size) are the ones that would also want smaller stacks even more!
This is something I never understood - embedded devices are not going to run more than a few processes and 4K*(Few Processes)
IMHO is not worth a saving now a days even in embedded world given
falling memory prices. Or do I misunderstand?
Parag
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 10 lines · Parag Warudkar
Quoted 2 lines · Linus Torvalds
And embedded people (the ones that might care about 1% code size) are the ones that would also want smaller stacks even more!
This is something I never understood - embedded devices are not going to run more than a few processes and 4K*(Few Processes)
IMHO is not worth a saving now a days even in embedded world given
falling memory prices. Or do I misunderstand?
Embedded applications span a huge range of sizes, from the very small devices to which you refer, to quite complex devices. The cable settop boxes we develop have over a hundred interrupt sources, typically run 250-300 threads, and have 192+ MiB of memory. For all that, we are very cost sensitive and are under constant pressure to come up with reliable ways to save memory.
Quoted 1 line
Parag
--
David VomLehn
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 12 lines · David VomLehn
Quoted 10 lines · Parag Warudkar
Quoted 2 lines · Linus Torvalds
And embedded people (the ones that might care about 1% code size) are the ones that would also want smaller stacks even more!
This is something I never understood - embedded devices are not going to run more than a few processes and 4K*(Few Processes)
IMHO is not worth a saving now a days even in embedded world given
falling memory prices. Or do I misunderstand?
Quoted 6 lines
Embedded applications span a huge range of sizes, from the very small devices to which you refer, to quite complex devices. The cable settop boxes we develop have over a hundred interrupt sources, typically run 250-300 threads, and have 192+ MiB of memory. For all that, we are very cost sensitive and are under constant pressure to come up with reliable ways to save memory.
As you say correctly the term "embedded" gets used for many different devices.
And if you have 192+ MiB of memory you have so much that all these kernel size discussions don't really matter.
Quoted 1 line
David VomLehn
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Parag WarudkarQuoted 5 lines · Parag Warudkar
This is something I never understood - embedded devices are not going to run more than a few processes and 4K*(Few Processes)
IMHO is not worth a saving now a days even in embedded world given
falling memory prices. Or do I misunderstand?
Well, by that argument, 1% of kernel size doesn't matter either..
1% of a kernel for an embedded device is roughly 10-30kB or so depending on how small you make the configuration.
If that matters, then so should the difference of 3-8 processes' kernel stack usage when you have a 4k/8k stack choice.
And they _all_ will have at least 3-8 processes on them. Even the simplest ones will tend to have many more.
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 6 lines · Linus Torvalds
Quoted 4 lines · Parag Warudkar
This is something I never understood - embedded devices are not going to run more than a few processes and 4K*(Few Processes)
IMHO is not worth a saving now a days even in embedded world given
falling memory prices. Or do I misunderstand?
Quoted 7 lines
Well, by that argument, 1% of kernel size doesn't matter either..
1% of a kernel for an embedded device is roughly 10-30kB or so depending on how small you make the configuration.
If that matters, then so should the difference of 3-8 processes' kernel stack usage when you have a 4k/8k stack choice.
And they _all_ will have at least 3-8 processes on them. Even the simplest ones will tend to have many more.
I have some simple devices (network access/routers) with 8MB of RAM, at power up not really being configured to do anything running 25 processes. (Heck there is over 10 kernel processes running!). Configure some interfaces and services and that will easily push past 40. I'd be happy with a 160k saving :-)
The init memory being freed at the end of the kernel boot is 88k, 4k stacks could save more than that.
Regards
Greg
------------------------------------------------------------------------
Greg Ungerer -- Chief Software Dude EMAIL: gerg-XXXsiaCtIV5Wk0Htik3J/[email protected] Secure Computing Corporation PHONE: +61 7 3435 2888 825 Stanley St, FAX: +61 7 3891 3630 Woolloongabba, QLD, 4102, Australia WEB: http://www.SnapGear.com
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 6 lines · Greg Ungerer
I have some simple devices (network access/routers) with 8MB of RAM, at power up not really being configured to do anything running 25 processes. (Heck there is over 10 kernel processes running!). Configure some interfaces and services and that will easily push past 40. I'd be happy with a 160k saving :-)
So you really need to run all 25 processes on that 8Mb box? (For reference even the NGW100 development board comes with 16Mb RAM).
Even if you do need those all 25 processes on the 8Mb box, fixing the memory usage of those user space hogs is lot better than trying to save 160Kb in kernel stacks.
Last I looked, user space wasn't particularly frugal with memory usage.
Parag
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 10 lines · Parag Warudkar
Quoted 6 lines · Greg Ungerer
I have some simple devices (network access/routers) with 8MB of RAM, at power up not really being configured to do anything running 25 processes. (Heck there is over 10 kernel processes running!). Configure some interfaces and services and that will easily push past 40. I'd be happy with a 160k saving :-)
So you really need to run all 25 processes on that 8Mb box?
Yes, of course. Considerable effort has been put into running a minimal set of processes (that still for fills the required function set of this device).
Quoted 1 line
(For reference even the NGW100 development board comes with 16Mb RAM).
Lots of development boards are fitted with lots of RAM.
And the pressure will still be on in _real_ products to reduce the RAM footprint as much as possible. There are exceptions but generally less is cheaper. Simple economics really.
Quoted 3 lines
Even if you do need those all 25 processes on the 8Mb box, fixing the memory usage of those user space hogs is lot better than trying to save 160Kb in kernel stacks.
Yep, been done too. You don't squeeze a lot into these smaller devices without looking at everything in it.
Quoted 1 line
Last I looked, user space wasn't particularly frugal with memory usage.
Then you haven't looked in the right places :-)
There are plenty of choices for making things small in user space. Simple stuff like using uClibc, busybox, etc.
In this specific example things like /bin/init is 10k, /bin/inetd is 10k, /bin/crond is 11k, etc. (Ofcourse it is a shared uClibc setup, uClibc is ~300k). And XIP can help out here too.
Regards
Greg
------------------------------------------------------------------------
Greg Ungerer -- Chief Software Dude EMAIL: [email protected] Secure Computing Corporation PHONE: +61 7 3435 2888 825 Stanley St, FAX: +61 7 3891 3630 Woolloongabba, QLD, 4102, Australia WEB: http://www.SnapGear.com
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 4 lines · Greg Ungerer
And the pressure will still be on in _real_ products to reduce the RAM footprint as much as possible. There are exceptions but generally less is cheaper. Simple economics really.
Well, sure - but the industry as a whole seems to have gone the other way - do more with more at the similar or lower price points! By that definition of less is better we should try and make the kernel memory pageable (or has someone already done that?) - Windows does it, by default ;)
Parag
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
On Tue, 2008-08-26 at 22:16 -0400, Parag Warudkar wrote:
[...]
Quoted 1 line
Well, sure - but the industry as a whole seems to have gone the other
"The industry as a whole" doesn't exist on that low level. You can't compare the laptop and/or desktop computer market (where one may buy today hardware that runs in 3 years with the next generation/release of the OS and applications) with the e.g. "WLAN router" market where - from the commercial point of view - every Euro counts (and where the requirements for the lifetime of the device are long frozen before the thing gets in a shop).
Quoted 3 lines
way - do more with more at the similar or lower price points! By that definition of less is better we should try and make the kernel memory pageable (or has someone already done that?) - Windows does it,
That doesn't help as in really small devices (like WLAN routers, cable modems, etc.) you run without any means of paging/swapping. And even binaries/read-only files are not necessarily executable in place (but must be loaded into RAM). So you can't flush these pages.
And pageable kernel memory doesn't come for free - even if one only counts the increased code and it's complexity.
Quoted 1 line
by default ;)
Which is more a sign that it is probably a very bad idea.
Bernd
Signature · 3 lines
Firmix Software GmbH http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
Embedded Linux Development and ServicesRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Linus TorvaldsQuoted 2 lines · Linus Torvalds
If that matters, then so should the difference of 3-8 processes' kernel stack usage when you have a 4k/8k stack choice.
The savings part -financial ones- are not always realizable with the way memory is priced/sized/fitted.
Savings in few Mb of Kernel stack are not necessarily going to allow getting rid of a single memory chip of 64M or so.
Either that or embedded manufacturing/configurations are different than the desktop world.
(If my device has 2 memory slots and my user space requires 100Mb including kernel memory - I anyways have to put in 64Mx2 there to take advantage of mass manufactured, general purpose memory - so no big deal if I saved 1.2Mb in Kernel stack or not. And savings of 64Mb Kernel memory are not feasible anyways to allow user space to work with 64Mb.)
On the other hand reducing user space memory usage on those devices (not counting savings from kernel stack size) is a way more attractive option.
And although you said in your later reply that Linux x86 with 4K stacks should be more than usable - my experiences running a untainted desktop/file server with 4K stack have been always disastrous XFS or not. It _might_ work for some well defined workloads but you would not want to risk 4K stacks otherwise.
I understand the having 4K stack option as a non-default for very specific workloads is a good idea but apart from that I think no one else seems to bother with reducing stack sizes (by no one I mean other OSes.)
Parag
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 6 lines · Parag Warudkar
And although you said in your later reply that Linux x86 with 4K stacks should be more than usable - my experiences running a untainted desktop/file server with 4K stack have been always disastrous XFS or not. It _might_ work for some well defined workloads but you would not want to risk 4K stacks otherwise.
Umm. How long?
4kB used to be the _only_ choice. And no, there weren't even irq stacks. So that 4kB was not just the whole kernel call-chain, it was also all the irq nesting above it.
And yes, we've gotten much worse over time, and no, I can't really suggest going back to that in general. The code bloat has certainly been accompanied by a stack bloat too.
But part of it is definitely gcc. Some versions of gcc used to be absolutely _horrid_ when it came to stack usage, especially with some flags, and especially with the crazy inlining that module-at-a-time caused.
But I'd be really happy if some embedded people tried to take some of that bloat back, and aim for 4kB stacks. Because it's definitely not unrealistic. At least it _shouldn't_ be. And a lot of the cases of us having structures on the stack is actually not worth it, and tends to be about being lazy rather than anything else.
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 4 lines · Linus Torvalds
Quoted 1 line · Parag Warudkar
Quoted 8 lines
Quoted 5 lines
And although you said in your later reply that Linux x86 with 4K stacks should be more than usable - my experiences running a untainted desktop/file server with 4K stack have been always disastrous XFS or not. It _might_ work for some well defined workloads but you would not want to risk 4K stacks otherwise.
Umm. How long?
IIRC the last I tried 4K stacks with x86 was on 2.6.21 - Fedora 7 kernel, around June 07 time frame.
The oops included a ugly and long call trace that I still remember.
Quoted 3 lines
And a lot of the cases of us
having structures on the stack is actually not worth it, and tends to be about being lazy rather than anything else.
What about deep call chains? The problem with the uptake of 4K stacks seems to be that is not reliably provable that it will work under all circumstances.
Parag
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 4 lines · Parag Warudkar
What about deep call chains? The problem with the uptake of 4K stacks seems to be that is not reliably provable that it will work under all circumstances.
Umm. Neither is 8k stacks. Nobody "proved" anything.
But yes, some subsystems have insanely deep call chains. And yes, things like the VFS recursion (for symlinks) makes that deeper yet for filesystems, although only on the lookup path. And that is exactly the kind of thing that can exacerbate the problem of the compiler artificially making for a bigger stack footprint of a function (*).
For things like the VFS layer, right now we allow a nesting level of 8, I think. If I remember correctly, it was 5 historically. Part of raising that depth, though, was that we actually moved the recursive part into fs/namei.c, and the nesting stack-depth was something pretty damn small when the filesystem used "follow_link" properly and let the VFS do it for it (ie the callchain to actually look up the link could be deep, but it would not recurse back, and instead just return a pointer, so that the actual _recursive_ part was just __do_follow_link() and is just a few words on the stack).
So yes, we do have some deep callchains, but they tend to be pretty well managed for _good_ code. The problems tend to be the areas with lots of indirection layers, and yeah, XFS, MD and ACPI all have those kinds of things.
In an embdedded world, many of those should be a non-issue, though.
Linus
(*) ie the function that _is_ on the deep chain doesn't actually need much of a stack footprint at all itself, but it may call a helper function that is _not_ in the deep chain, and if it gets inlined it may give its excessive stack footprint to the deep chain - and this is _exactly_ the problem that happened with inlining "load_module()".
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Parag WarudkarQuoted 3 lines
What about deep call chains? The problem with the uptake of 4K stacks seems to be that is not reliably provable that it will work under all circumstances.
On x86-32 with 8K stacks your IRQ paths share them so that is even harder to prove (not that you can prove any of them) and the bugs are more obscure and random.
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Linus TorvaldsQuoted 3 lines · Linus Torvalds
4kB used to be the _only_ choice. And no, there weren't even irq stacks. So that 4kB was not just the whole kernel call-chain, it was also all the irq nesting above it.
I think your memory is failing you. In 2.4 and earlier, the kernel stack was 8kB minus the size of the task_struct, which sat at the start of the 8kB. For instance, from include/asm-i386/processor.h for 2.4.29:
#define THREAD_SIZE (2*PAGE_SIZE)
#define alloc_task_struct() ((struct task_struct *) __get_free_pages(GFP_KERNEL,1)) #define free_task_struct(p) free_pages((unsigned long) (p), 1)
Paul.
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 8 lines · Paul Mackerras
Quoted 1 line · Linus Torvalds
4kB used to be the _only_ choice. And no, there weren't even irq stacks. So that 4kB was not just the whole kernel call-chain, it was also all the irq nesting above it.
I think your memory is failing you. In 2.4 and earlier, the kernel stack was 8kB minus the size of the
task_struct, which sat at the start of the 8kB. For instance, frominclude/asm-i386/processor.hfor 2.4.29:
but was shared with interrupts; so out of the 6Kb left, you had still really only 4Kb for user context stack
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Paul MackerrasQuoted 4 lines · Paul Mackerras
I think your memory is failing you. In 2.4 and earlier, the kernel stack was 8kB minus the size of the
task_struct, which sat at the start of the 8kB.
Yup, you're right.
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Linus TorvaldsQuoted 6 lines · Linus Torvalds
...
But part of it is definitely gcc. Some versions of gcc used to be absolutely _horrid_ when it came to stack usage, especially with some flags, and especially with the crazy inlining that module-at-a-time caused.
...
That was gcc 3.4.
And due to that we disable unit-at-a-time for gcc 3.4 on 32bit x86.
Quoted 1 line
Linus
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Parag WarudkarOn Tue, 2008-08-26 at 20:58 -0400, Parag Warudkar wrote:
[...]
Quoted 4 lines
The savings part -financial ones- are not always realizable with the way memory is priced/sized/fitted.
Savings in few Mb of Kernel stack are not necessarily going to allow getting rid of a single memory chip of 64M or so.
No, but you can put an additional service(s) on it and sales people have one (or two or ....) line more for their sales brochures.
Quoted 2 lines
Either that or embedded manufacturing/configurations are different than the desktop world.
They are different. Think of running the complete system acting as a bridge, router and/or firewall (Kernel early 2.4 though) from 4MB flash in 32MB RAM and - listing the outside visible services - having a command-line interface, web-GUI (implying a http server) and and a (net-)SNMP agent on it.
Running a glibc without thread support is win there (implying that there is no thread support available on that device).
Quoted 6 lines
(If my device has 2 memory slots and my user space requires 100Mb including kernel memory - I anyways have to put in 64Mx2 there to take advantage of mass manufactured, general purpose memory - so no big deal if I saved 1.2Mb in Kernel stack or not. And savings of 64Mb Kernel memory are not feasible anyways to allow user space to work with 64Mb.)
As soon as product management realizes that there is space left on the device, they get new ideas and/or customer requirements to run more services on that device.
Quoted 3 lines
On the other hand reducing user space memory usage on those devices (not counting savings from kernel stack size) is a way more attractive option.
There is no question if save space here or there. You save it - sooner or later - on all fronts. Period.
Quoted 5 lines
And although you said in your later reply that Linux x86 with 4K stacks should be more than usable - my experiences running a untainted desktop/file server with 4K stack have been always disastrous XFS or not. It _might_ work for some well defined workloads but you would not want to risk 4K stacks otherwise.
The embedded world of really small devices usually doesn't run XFS (or ext? or reiser* of jfs or NFS or ...) or stacks block devices on files or .....
Quoted 4 lines
I understand the having 4K stack option as a non-default for very specific workloads is a good idea but apart from that I think no one else seems to bother with reducing stack sizes (by no one I mean other OSes.)
They probably gave the idea pretty soon because you need to rework/improve large parts of the kernel + drivers (and that has two major problems - it consumes a lot of man power for "no new features and everything must be completely tested again"[0] and it adds new risks). And that is practically impossible if one sells "stable driver APIs" for 3rd party (commercial) drivers because these must be changed too.
Bernd
[0]: Let alone if you (or your customers) need certificates from some
governmental agencys.
Signature · 3 lines
Firmix Software GmbH http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
Embedded Linux Development and ServicesRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 8 lines · Bernd Petrovitsch
They probably gave the idea pretty soon because you need to rework/improve large parts of the kernel + drivers (and that has two major problems - it consumes a lot of man power for "no new features and everything must be completely tested again"[0] and it adds new risks). And that is practically impossible if one sells "stable driver APIs" for 3rd party (commercial) drivers because these must be changed too.
But not many embedded Linux arches support 4K stacks like Adrian pointed out earlier.
So the same (lot of man power requirement) would apply to Linux.
Sure it will be good - but how reasonable it is to attempt it and how reliably it will work under all conceived loads - those are the questions.
Thanks
Parag
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 9 lines · Parag Warudkar
Quoted 6 lines · Bernd Petrovitsch
They probably gave the idea pretty soon because you need to rework/improve large parts of the kernel + drivers (and that has two major problems - it consumes a lot of man power for "no new features and everything must be completely tested again"[0] and it adds new risks). And that is practically impossible if one sells "stable driver APIs" for 3rd party (commercial) drivers because these must be changed too.
But not many embedded Linux arches support 4K stacks like Adrian
What is an "embedded Linux arch"?
Personally I encountered i386, ARM, MIPS and PPC in the embedded world.
Quoted 2 lines
pointed out earlier.
So the same (lot of man power requirement) would apply to Linux.
Of course. Look at the amount of work done by lots of people in that area (including stack frame size reductions) and on-going discussions.
Quoted 3 lines
Sure it will be good - but how reasonable it is to attempt it and how reliably it will work under all conceived loads - those are the questions.
If you "develop" an embedded system (which is partly system integration of existing apps) to be installed in the field, you don't have that many conceivable work loads compared to a desktop/server system. And you have a fixed list of drivers and applications.
A usual approach is to run stress tests on several (or all) subsystems/services/... in parallel and if the device survives it functioning correctly, it is at least good enough.
Bernd
Signature · 3 lines
Firmix Software GmbH http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
Embedded Linux Development and ServicesRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 4 lines · Bernd Petrovitsch
If you "develop" an embedded system (which is partly system integration of existing apps) to be installed in the field, you don't have that many conceivable work loads compared to a desktop/server system. And you have a fixed list of drivers and applications.
Hah! Not in my line of embedded device.
32MB no-MMU ARM boards which people run new things and attach new devices to rather often - without making new hardware. Volume's too low per individual application to get new hardware designed and made.
I'm seriously thinking of forwarding porting the 4 year old firmware from 2.4.26 to 2.6.current, just to get new drivers and capabilities. Backporting is tedious, so's feeling wretchedly far from the mainline world.
Quoted 3 lines
A usual approach is to run stress tests on several (or all) subsystems/services/... in parallel and if the device survives it functioning correctly, it is at least good enough.
Per application.
Some little devices run hundreds of different applications and customers expect to customise, script themselves, and attach different devices (over USB). The next customer in the chain expects the bits you supplied to work in a variety of unexpected situations, even when you advise that it probably won't do that.
Much like desktop/server Linux, but on a small device where silly little things like 'create a process' are a stress for the dear little thing.
(My biggest lesson: insist on an MMU next time!)
-- Jamie
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 11 lines · Jamie Lokier
Quoted 4 lines · Bernd Petrovitsch
If you "develop" an embedded system (which is partly system integration of existing apps) to be installed in the field, you don't have that many conceivable work loads compared to a desktop/server system. And you have a fixed list of drivers and applications.
Hah! Not in my line of embedded device.
32MB no-MMU ARM boards which people run new things and attach new devices to rather often - without making new hardware. Volume's too low per individual application to get new hardware designed and made.
Yes, you may have several products on the same hardware with somewhat differing requirements (or not). But that is much less than a general purpose system IMHO.
Quoted 2 lines
I'm seriously thinking of forwarding porting the 4 year old firmware from 2.4.26 to 2.6.current, just to get new drivers and capabilities.
That sounds reasonable (and I never meant maintaining the old system infinitely. Actually once the thing is shipped it usually enters deep maintenance mode and the next is more a fork from the old).
Quoted 2 lines
Backporting is tedious, so's feeling wretchedly far from the mainline world.
ACK. But that also depends on amount local changes (and sorry, but not all locally necessary patches would be accepted in mainline in any way).
Quoted 11 lines
Quoted 3 lines
A usual approach is to run stress tests on several (or all) subsystems/services/... in parallel and if the device survives it functioning correctly, it is at least good enough.
Per application.
Some little devices run hundreds of different applications and customers expect to customise, script themselves, and attach different devices (over USB). The next customer in the chain expects the bits you supplied to work in a variety of unexpected situations, even when you advise that it probably won't do that.
Basically their problem. Yes, "they" actually think they get a Linux system where they can do everything and it simply works.
Oh, that's obviously not a usual "WLAN-router style" of product (where you are not expected to actually login on a console or per ssh).
Quoted 5 lines
Much like desktop/server Linux, but on a small device where silly little things like 'create a process' are a stress for the dear little thing.
(My biggest lesson: insist on an MMU next time!)
ACK. We avoid MMU-less hardware too - especially since there is enough hardware with a MMU around.
Bernd
Signature · 3 lines
Firmix Software GmbH http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
Embedded Linux Development and ServicesRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 7 lines · Bernd Petrovitsch
Quoted 3 lines
32MB no-MMU ARM boards which people run new things and attach new devices to rather often - without making new hardware. Volume's too low per individual application to get new hardware designed and made.
Yes, you may have several products on the same hardware with somewhat differing requirements (or not). But that is much less than a general purpose system IMHO.
It is, but the idea that small embedded systems go through a 'all components are known, drivers are known, test and if it passes it's shippable' does not always apply.
Quoted 5 lines
Quoted 2 lines
I'm seriously thinking of forwarding porting the 4 year old firmware from 2.4.26 to 2.6.current, just to get new drivers and capabilities.
That sounds reasonable (and I never meant maintaining the old system infinitely.
Sounds reasonable, but it's vetoed for anticipated time and cost, compared with backporting on demand. Fair enough, since 2.6.current doesn't support ARM no-MMU last I heard ('soon'?).
On the other hand, the 2.6 anti-fragmentation patches, including latest SLUB stuff, ironically meant to help big machines, sound really appealing for my current problem and totally unrealistic to backport...
Quoted 2 lines
ACK. We avoid MMU-less hardware too - especially since there is enough hardware with a MMU around.
I can't emphasise enough how much difference MMU makes to Linux userspace.
It's practically: MMU = standard Linux (with less RAM), have everything. No-MMU = lots of familiar 'Linux' things not available or break.
-- Jamie
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 1 line · Jamie Lokier
Bernd Petrovitsch wrote:
[...]
Quoted 3 lines
It is, but the idea that small embedded systems go through a 'all components are known, drivers are known, test and if it passes it's shippable' does not always apply.
Not always but often enough. And yes, there is ARM-based embedded hardware with 1GB Flash-RAM and 128MB RAM.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
I'm seriously thinking of forwarding porting the 4 year old firmware from 2.4.26 to 2.6.current, just to get new drivers and capabilities.
That sounds reasonable (and I never meant maintaining the old system infinitely.
Sounds reasonable, but it's vetoed for anticipated time and cost,
That is to be expected;-)
[....]
Quoted 7 lines
Quoted 2 lines
ACK. We avoid MMU-less hardware too - especially since there is enough hardware with a MMU around.
I can't emphasise enough how much difference MMU makes to Linux userspace.
It's practically: MMU = standard Linux (with less RAM), have everything. No-MMU = lots of familiar 'Linux' things not available or break.
ACK. And tell that a customer that everything is more effort and more risk and not just "simply cross-compile it as it runs on my desktop too".
Bernd
Signature · 3 lines
Firmix Software GmbH http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
Embedded Linux Development and ServicesRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Jamie LokierQuoted 26 lines · Jamie Lokier
Quoted 6 lines · Bernd Petrovitsch
Quoted 3 lines
32MB no-MMU ARM boards which people run new things and attach new devices to rather often - without making new hardware. Volume's too low per individual application to get new hardware designed and made.
Yes, you may have several products on the same hardware with somewhat differing requirements (or not). But that is much less than a general purpose system IMHO.
It is, but the idea that small embedded systems go through a 'all components are known, drivers are known, test and if it passes it's shippable' does not always apply.
Quoted 4 lines
Quoted 2 lines
I'm seriously thinking of forwarding porting the 4 year old firmware from 2.4.26 to 2.6.current, just to get new drivers and capabilities.
That sounds reasonable (and I never meant maintaining the old system infinitely.
Sounds reasonable, but it's vetoed for anticipated time and cost, compared with backporting on demand. Fair enough, since 2.6.current doesn't support ARM no-MMU last I heard ('soon'?).
On the other hand, the 2.6 anti-fragmentation patches, including latest SLUB stuff, ironically meant to help big machines, sound really appealing for my current problem and totally unrealistic to backport...
Quoted 7 lines
Quoted 2 lines
ACK. We avoid MMU-less hardware too - especially since there is enough hardware with a MMU around.
I can't emphasise enough how much difference MMU makes to Linux userspace.
It's practically: MMU = standard Linux (with less RAM), have everything. No-MMU = lots of familiar 'Linux' things not available or break.
And lots of things work in the usual way...
Of course the flip side is that for people who have platforms without MMU they can run something more than the mostly "toy" like operating systems typically available. There are plenty of problem domains that the non-MMU limitations are not a problem for. (Yours doesn't sound like one of them :-)
Regards
Greg
------------------------------------------------------------------------
Greg Ungerer -- Chief Software Dude EMAIL: [email protected] Secure Computing Corporation PHONE: +61 7 3435 2888 825 Stanley St, FAX: +61 7 3891 3630 Woolloongabba, QLD, 4102, Australia WEB: http://www.SnapGear.com
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Jamie LokierQuoted 29 lines · Jamie Lokier
Quoted 4 lines · Bernd Petrovitsch
If you "develop" an embedded system (which is partly system integration of existing apps) to be installed in the field, you don't have that many conceivable work loads compared to a desktop/server system. And you have a fixed list of drivers and applications.
Hah! Not in my line of embedded device.
32MB no-MMU ARM boards which people run new things and attach new devices to rather often - without making new hardware. Volume's too low per individual application to get new hardware designed and made.
I'm seriously thinking of forwarding porting the 4 year old firmware from 2.4.26 to 2.6.current, just to get new drivers and capabilities. Backporting is tedious, so's feeling wretchedly far from the mainline world.
Quoted 3 lines
A usual approach is to run stress tests on several (or all) subsystems/services/... in parallel and if the device survives it functioning correctly, it is at least good enough.
Per application.
Some little devices run hundreds of different applications and customers expect to customise, script themselves, and attach different devices (over USB). The next customer in the chain expects the bits you supplied to work in a variety of unexpected situations, even when you advise that it probably won't do that.
Quoted 5 lines
Much like desktop/server Linux, but on a small device where silly little things like 'create a process' are a stress for the dear little thing.
(My biggest lesson: insist on an MMU next time!)
But given you have hardware you can't change would you choose to not run Linux, even with the limitations of non-MMU?
Hell no :-)
Regards
Greg
------------------------------------------------------------------------
Greg Ungerer -- Chief Software Dude EMAIL: gerg-XXXsiaCtIV5Wk0Htik3J/[email protected] Secure Computing Corporation PHONE: +61 7 3435 2888 825 Stanley St, FAX: +61 7 3891 3630 Woolloongabba, QLD, 4102, Australia WEB: http://www.SnapGear.com
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Parag WarudkarQuoted 10 lines · Parag Warudkar
Quoted 2 lines · Linus Torvalds
And embedded people (the ones that might care about 1% code size) are the ones that would also want smaller stacks even more!
This is something I never understood - embedded devices are not going to run more than a few processes and 4K*(Few Processes)
IMHO is not worth a saving now a days even in embedded world given
falling memory prices. Or do I misunderstand?
Falling prices are no reason to increase the amount of available RAM (or other hardware).
Especially if you (intend to) build >1E5 devices - where every Euro counts.
Bernd
Signature · 3 lines
Firmix Software GmbH http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
Embedded Linux Development and ServicesRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Linus TorvaldsQuoted 8 lines · Linus Torvalds
Quoted 4 lines · Adrian Bunk
If you think we have too many stacksize problems I'd suggest to consider removing the choice of 4k stacks on i386, sh and m68knommu instead of using -fno-inline-functions-called-once:
Quoted 6 lines
Don't be silly. That makes the problem _worse_.
We're much better off with a 1% code-size reduction than forcing big stacks on people. The 4kB stack option is also a good way of saying "if it works with this, then 8kB is certainly safe".
...
You implicitely assume both would solve the same problem.
While 4kB stacks are something we anyway never got 100% working, the cases where gcc inlining functions causes a critical increase in stack usage are usually not that hard to find, and once found the fix is trivial.
We should anyway monitor stack usages better since we have frequent programming errors in this area, and problems caused by gcc can this way be detected en passant.
You have a good point that aiming at 4kB makes 8kB a very safe choice.
But I do not think the problem you'd solve with -fno-inline-functions-called-once is big enough to warrant the size increase it causes.
Quoted 1 line
Linus
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 6 lines · Adrian Bunk
Quoted 4 lines
We're much better off with a 1% code-size reduction than forcing big stacks on people. The 4kB stack option is also a good way of saying "if it works with this, then 8kB is certainly safe".
You implicitely assume both would solve the same problem.
I'm just saying that your logic doesn't hold water.
If we can save kernel stack usage, then a 1% increase in kernel size is more than worth it.
Quoted 1 line
While 4kB stacks are something we anyway never got 100% working
What? Don't be silly.
Linux _historically_ always used 4kB stacks.
No, they are likely not usable on x86-64, but dammit, they should be more than usable on x86-32 still.
Quoted 3 lines
But I do not think the problem you'd solve with -fno-inline-functions-called-once is big enough to warrant the size increase it causes.
You continually try to see the inlining as a single solution to one problem (debuggability, stack, whatever).
The biggest problem with gcc inlining has always been that it has been _unpredictable_. It causes problems in many different ways. It has caused stability issues due to gcc versions doing random things. It causes the stack expansion. It makes stack traces harder for debugging, etc.
If it was any one thing, I wouldn't care. But it's exactly the fact that it causes all these problems in different areas.
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 8 lines · Linus Torvalds
Quoted 5 lines · Adrian Bunk
Quoted 4 lines
We're much better off with a 1% code-size reduction than forcing big stacks on people. The 4kB stack option is also a good way of saying "if it works with this, then 8kB is certainly safe".
Quoted 6 lines
Quoted 1 line
You implicitely assume both would solve the same problem.
I'm just saying that your logic doesn't hold water.
If we can save kernel stack usage, then a 1% increase in kernel size is more than worth it.
From some tests the size increase seems to become bigger for smaller kernels, but I don't have any really good data.
An interesting question is why most of our architectures for embedded devices only offer bigger stacks:
The only architectures offering a 4kB stacks option are:
- m68knommu
- sh
- 32bit x86
The following architectures that are used in embedded devices always use 8kB stacks (or bigger) in your tree:
- arm
- avr32
- blackfin
- cris
- frv
- h8300
- m32r
- m68k
- mips
- mn10300 (has an #ifdef CONFIG_4KSTACKS but no kconfig option)
- powerpc
- xtensa
Quoted 8 lines
Quoted 1 line
While 4kB stacks are something we anyway never got 100% working
What? Don't be silly.
Linux _historically_ always used 4kB stacks.
No, they are likely not usable on x86-64, but dammit, they should be more than usable on x86-32 still.
When did we get callpaths like like nfs+xfs+md+scsi reliably working with 4kB stacks on x86-32?
Quoted 2 lines
...
Linus
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 3 lines · Adrian Bunk
When did we get callpaths like like nfs+xfs+md+scsi reliably working with 4kB stacks on x86-32?
XFS may never have been usable, but the rest, sure.
And you seem to be making this whole argument an excuse to SUCK, adn an excuse to let gcc crap even more on our stack space.
Why?
Why aren't you saying that we should be able to do better? Instead, you seem to asking us to do even worse than we do now?
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 9 lines · Linus Torvalds
Quoted 3 lines · Adrian Bunk
When did we get callpaths like like nfs+xfs+md+scsi reliably working with 4kB stacks on x86-32?
XFS may never have been usable, but the rest, sure.
Quoted 7 lines
And you seem to be making this whole argument an excuse to SUCK, adn an excuse to let gcc crap even more on our stack space.
Why?
Why aren't you saying that we should be able to do better? Instead, you seem to asking us to do even worse than we do now?
My main point is:
- getting 4kB stacks working reliably is a hard task
- having an eye on gcc increasing the stack usage, and fixing it if required, is relatively easy
If we should be able to do better at getting (and keeping) 4kB stacks working, then coping with possible inlining problems caused by gcc should not be a big problem for us.
Quoted 1 line
Linus
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 16 lines · Adrian Bunk
Quoted 7 lines · Linus Torvalds
Quoted 3 lines · Adrian Bunk
When did we get callpaths like like nfs+xfs+md+scsi reliably working with 4kB stacks on x86-32?
XFS may never have been usable, but the rest, sure.
Quoted 7 lines
And you seem to be making this whole argument an excuse to SUCK, adn an excuse to let gcc crap even more on our stack space.
Why?
Why aren't you saying that we should be able to do better? Instead, you seem to asking us to do even worse than we do now?
Quoted 9 lines
My main point is:
- getting 4kB stacks working reliably is a hard task
- having an eye on gcc increasing the stack usage, and fixing it if required, is relatively easyIf we should be able to do better at getting (and keeping) 4kB stacks working, then coping with possible inlining problems caused by gcc should not be a big problem for us.
Out of the architectures you've mentioned for 4k stacks, they also tend to do IRQ stacks, which is something you seem to have overlooked.
In addition to that, debugging the runaway stack users on 4k tends to be easier anyways since you end up blowing the stack a lot sooner. On sh we've had pretty good luck with it, though most of our users are using fairly deterministic workloads and continually profiling the footprint. Anything that runs away or uses an insane amount of stack space needs to be fixed well before that anyways, so catching it sooner is always preferable. I imagine the same case is true for m68knommu (even sans IRQ stacks).
Things might be more sensitive on x86, but it's certainly not something that's a huge problem for the various embedded platforms to wire up, whether they want to go the IRQ stack route or not.
In any event, lack of support for something on embedded architectures in the kernel is more often due to apathy/utter indifference on the part of the architecture maintainer rather than being indicative of any intrinsic difficulty in supporting the thing in question. Most new "features" on the lesser maintained architectures tend to end up there either out of peer pressure or copying-and-pasting accidents rather than any sort of design. ;-)
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 22 lines · Paul Mundt
Quoted 21 lines · Adrian Bunk
Quoted 7 lines · Linus Torvalds
Quoted 3 lines · Adrian Bunk
When did we get callpaths like like nfs+xfs+md+scsi reliably working with 4kB stacks on x86-32?
XFS may never have been usable, but the rest, sure.
Quoted 7 lines
And you seem to be making this whole argument an excuse to SUCK, adn an excuse to let gcc crap even more on our stack space.
Why?
Why aren't you saying that we should be able to do better? Instead, you seem to asking us to do even worse than we do now?
My main point is:
- getting 4kB stacks working reliably is a hard task
- having an eye on gcc increasing the stack usage, and fixing it if required, is relatively easy
Quoted 6 lines
Quoted 4 lines
If we should be able to do better at getting (and keeping) 4kB stacks working, then coping with possible inlining problems caused by gcc should not be a big problem for us.
Out of the architectures you've mentioned for 4k stacks, they also tend to do IRQ stacks, which is something you seem to have overlooked.
No, I am aware of that, and on i386 IRQ stacks are only used with 4kB stacks.
On i386 it is effectively a step from 6kB to 4kB.
Quoted 8 lines
In addition to that, debugging the runaway stack users on 4k tends to be easier anyways since you end up blowing the stack a lot sooner. On sh we've had pretty good luck with it, though most of our users are using fairly deterministic workloads and continually profiling the footprint. Anything that runs away or uses an insane amount of stack space needs to be fixed well before that anyways, so catching it sooner is always preferable. I imagine the same case is true for m68knommu (even sans IRQ stacks).
CONFIG_DEBUG_STACKOVERFLOW should give you the same information, and if wanted with an arbitrary limit.
Quoted 3 lines
Things might be more sensitive on x86, but it's certainly not something that's a huge problem for the various embedded platforms to wire up, whether they want to go the IRQ stack route or not.
How many platforms use 4kB stacks on sh?
Only 1 out of 34 defconfigs uses it.
Are there any numbers for real life usage.
Quoted 7 lines
In any event, lack of support for something on embedded architectures in the kernel is more often due to apathy/utter indifference on the part of the architecture maintainer rather than being indicative of any intrinsic difficulty in supporting the thing in question. Most new "features" on the lesser maintained architectures tend to end up there either out of peer pressure or copying-and-pasting accidents rather than any sort of design. ;-)
arm or powerpc aren't exactly lesser maintained architectures.
4kB has shown to be a hard to achieve limit. After more than 4 years in mainline being available on i386 there are still cases where 4kB are not enough.
IMHO there seems to currently be a mismatch between it's maintainance cost and the actual number of users. That's in my opinion the main problem with it, no matter in which direction it gets resolved.
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 11 lines · Adrian Bunk
Quoted 9 lines · Paul Mundt
On Wed, Aug 27, 2008 at 02:58:30PM +0300, Adrian Bunk wrote:
In addition to that, debugging the runaway stack users on 4k tends to be easier anyways since you end up blowing the stack a lot sooner. On sh we've had pretty good luck with it, though most of our users are using fairly deterministic workloads and continually profiling the footprint. Anything that runs away or uses an insane amount of stack space needs to be fixed well before that anyways, so catching it sooner is always preferable. I imagine the same case is true for m68knommu (even sans IRQ stacks).
Quoted 3 lines
CONFIG_DEBUG_STACKOVERFLOWshould give you the same information, and if wanted with an arbitrary limit.
In some cases, yes. In the CONFIG_DEBUG_STACKOVERFLOW case the check is only performed from do_IRQ(), which is sporadic at best, especially on tickless. While it catches some things, it's not a complete solution in and of iteslf.
In addition to this, there are even fewer platforms that support it than there are platforms that do 4k stacks. At first glance, it looks like it's only m32r, powerpc, sh, x86, and xtensa. Others support the Kconfig option, but don't seem to realize that it's not an option that the kernel does anything with by itself, and so don't actually do anything (ie, FRV).
Quoted 8 lines
Quoted 3 lines
Things might be more sensitive on x86, but it's certainly not something that's a huge problem for the various embedded platforms to wire up, whether they want to go the IRQ stack route or not.
How many platforms use 4kB stacks on sh?
Only 1 out of 34 defconfigs uses it.
The defconfigs tend to enable as much random stuff as people are interested in for development and testing purposes. Most of these end up being reference boards and are the basis for products, rather than shipping products themselves. In the latter case, everything is gradually tightened down, and 4k stack utilization in that case is the norm, rather than the exception.
Quoted 10 lines
Quoted 7 lines
In any event, lack of support for something on embedded architectures in the kernel is more often due to apathy/utter indifference on the part of the architecture maintainer rather than being indicative of any intrinsic difficulty in supporting the thing in question. Most new "features" on the lesser maintained architectures tend to end up there either out of peer pressure or copying-and-pasting accidents rather than any sort of design. ;-)
arm or powerpc aren't exactly lesser maintained architectures.
Indeed, which is why I find it bizarre that you would even bother applying what was said to those platforms. Specifically I was referring to the embedded platforms that don't do 4k stacks today. The fact they don't support them today has much less to do with 4k being an unattainable limit as it does with people simply not bothering to implement it on their platform.
Quoted 4 lines
IMHO there seems to currently be a mismatch between it's maintainance cost and the actual number of users. That's in my opinion the main problem with it, no matter in which direction it gets resolved.
Perhaps that's true on x86, but in general I take issue with that. On sh we've had to do very little maintenance for it and most shipping products are using it today (at least on MMU-Linux, we don't bother with it on nommu). Most of the problems we ran in to with 4k stacks tended to be stuff that we wanted to fix for 8k anyways. I suspect that this case is true for the other embedded platforms also.
Note that on sh we also conditionalize IRQ stacks separately, so while they are often used together, it's possible to use 4k stacks without resorting to IRQ stacks (as m68knommu also seems to do).
David Miller
maintainerRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
From: Paul Mundt <[email protected]>
Date: Thu, 28 Aug 2008 09:32:13 +0900
Quoted 8 lines · Adrian Bunk
Quoted 2 lines
CONFIG_DEBUG_STACKOVERFLOWshould give you the same information, and if wanted with an arbitrary limit.In some cases, yes. In the
CONFIG_DEBUG_STACKOVERFLOWcase the check is only performed fromdo_IRQ(), which is sporadic at best, especially on tickless. While it catches some things, it's not a complete solution in and of iteslf.
BTW, on sparc64 we have a stack overflow checker that runs via the profiling _mcount hook. So every function call we check if the stack is getting overused.
If so, we jump onto a special static debugging stack and print the stack overflow message.
And yes it works with IRQ stacks which is all that sparc64 uses nowadays.
Perhaps this is useful enough to make generic.
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 16 lines · David Miller
From: Paul Mundt <[email protected]>
Date: Thu, 28 Aug 2008 09:32:13 +0900Quoted 8 lines · Adrian Bunk
Quoted 2 lines
CONFIG_DEBUG_STACKOVERFLOWshould give you the same information, and if wanted with an arbitrary limit.In some cases, yes. In the
CONFIG_DEBUG_STACKOVERFLOWcase the check is only performed fromdo_IRQ(), which is sporadic at best, especially on tickless. While it catches some things, it's not a complete solution in and of iteslf.BTW, on sparc64 we have a stack overflow checker that runs via the profiling
_mcounthook. So every function call we check if the stack is getting overused.
Quoted 7 lines
If so, we jump onto a special static debugging stack and print the stack overflow message.
And yes it works with IRQ stacks which is all that sparc64 uses nowadays.
Perhaps this is useful enough to make generic.
Thanks for the pointer, I'll take a look at it!
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Paul MundtQuoted 20 lines · Paul Mundt
Quoted 11 lines · Adrian Bunk
Quoted 9 lines · Paul Mundt
On Wed, Aug 27, 2008 at 02:58:30PM +0300, Adrian Bunk wrote:
In addition to that, debugging the runaway stack users on 4k tends to be easier anyways since you end up blowing the stack a lot sooner. On sh we've had pretty good luck with it, though most of our users are using fairly deterministic workloads and continually profiling the footprint. Anything that runs away or uses an insane amount of stack space needs to be fixed well before that anyways, so catching it sooner is always preferable. I imagine the same case is true for m68knommu (even sans IRQ stacks).Quoted 3 lines
CONFIG_DEBUG_STACKOVERFLOWshould give you the same information, and if wanted with an arbitrary limit.In some cases, yes. In the
CONFIG_DEBUG_STACKOVERFLOWcase the check is only performed fromdo_IRQ(), which is sporadic at best, especially on tickless. While it catches some things, it's not a complete solution in and of iteslf.
Quoted 4 lines
In addition to this, there are even fewer platforms that support it than there are platforms that do 4k stacks. At first glance, it looks like it's only m32r, powerpc, sh, x86, and xtensa.
...
As far as I can see the only architectures that optionally offer 4kB stacks today are m68knommu, s390, sh and x86.
Did I miss some architectures or is 5 < 4 ;) ?
Quoted 4 lines
Others support the
Kconfig
option, but don't seem to realize that it's not an option that the kernel does anything with by itself, and so don't actually do anything (ie, FRV).
Unless I miss anything these "others" include only FRV.
Quoted 11 lines
Quoted 4 lines
IMHO there seems to currently be a mismatch between it's maintainance cost and the actual number of users. That's in my opinion the main problem with it, no matter in which direction it gets resolved.
Perhaps that's true on x86, but in general I take issue with that. On sh we've had to do very little maintenance for it and most shipping products are using it today (at least on MMU-Linux, we don't bother with it on nommu). Most of the problems we ran in to with 4k stacks tended to be stuff that we wanted to fix for 8k anyways. I suspect that this case is true for the other embedded platforms also.
...
Most stack issues are not platform or architecture specific.
The maintainance effort therefore mostly depends on whether a non-zero number of architectures uses 4kB stacks.
And if something is considered to be important for small embedded systems, but not supported on ARM, MIPS or PowerPC, then that's a bit strange.
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Paul MundtQuoted 20 lines · Paul Mundt
Quoted 11 lines · Adrian Bunk
Quoted 9 lines · Paul Mundt
On Wed, Aug 27, 2008 at 02:58:30PM +0300, Adrian Bunk wrote:
In addition to that, debugging the runaway stack users on 4k tends to be easier anyways since you end up blowing the stack a lot sooner. On sh we've had pretty good luck with it, though most of our users are using fairly deterministic workloads and continually profiling the footprint. Anything that runs away or uses an insane amount of stack space needs to be fixed well before that anyways, so catching it sooner is always preferable. I imagine the same case is true for m68knommu (even sans IRQ stacks).Quoted 3 lines
CONFIG_DEBUG_STACKOVERFLOWshould give you the same information, and if wanted with an arbitrary limit.In some cases, yes. In the
CONFIG_DEBUG_STACKOVERFLOWcase the check is only performed fromdo_IRQ(), which is sporadic at best, especially on tickless. While it catches some things, it's not a complete solution in and of iteslf.
Quoted 4 lines
In addition to this, there are even fewer platforms that support it than there are platforms that do 4k stacks. At first glance, it looks like it's only m32r, powerpc, sh, x86, and xtensa.
...
As far as I can see the only architectures that optionally offer 4kB stacks today are m68knommu, s390, sh and x86.
Did I miss some architectures or is 5 < 4 ;) ?
Quoted 4 lines
Others support the
Kconfig
option, but don't seem to realize that it's not an option that the kernel does anything with by itself, and so don't actually do anything (ie, FRV).
Unless I miss anything these "others" include only FRV.
Quoted 11 lines
Quoted 4 lines
IMHO there seems to currently be a mismatch between it's maintainance cost and the actual number of users. That's in my opinion the main problem with it, no matter in which direction it gets resolved.
Perhaps that's true on x86, but in general I take issue with that. On sh we've had to do very little maintenance for it and most shipping products are using it today (at least on MMU-Linux, we don't bother with it on nommu). Most of the problems we ran in to with 4k stacks tended to be stuff that we wanted to fix for 8k anyways. I suspect that this case is true for the other embedded platforms also.
...
Most stack issues are not platform or architecture specific.
The maintainance effort therefore mostly depends on whether a non-zero number of architectures uses 4kB stacks.
And if something is considered to be important for small embedded systems, but not supported on ARM, MIPS or PowerPC, then that's a bit strange.
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedA message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 22 lines · Paul Mundt
Quoted 21 lines · Adrian Bunk
Quoted 7 lines · Linus Torvalds
Quoted 3 lines · Adrian Bunk
When did we get callpaths like like nfs+xfs+md+scsi reliably working with 4kB stacks on x86-32?
XFS may never have been usable, but the rest, sure.
Quoted 7 lines
And you seem to be making this whole argument an excuse to SUCK, adn an excuse to let gcc crap even more on our stack space.
Why?
Why aren't you saying that we should be able to do better? Instead, you seem to asking us to do even worse than we do now?
My main point is:
- getting 4kB stacks working reliably is a hard task
- having an eye on gcc increasing the stack usage, and fixing it if required, is relatively easy
Quoted 6 lines
Quoted 4 lines
If we should be able to do better at getting (and keeping) 4kB stacks working, then coping with possible inlining problems caused by gcc should not be a big problem for us.
Out of the architectures you've mentioned for 4k stacks, they also tend to do IRQ stacks, which is something you seem to have overlooked.
No, I am aware of that, and on i386 IRQ stacks are only used with 4kB stacks.
On i386 it is effectively a step from 6kB to 4kB.
Quoted 8 lines
In addition to that, debugging the runaway stack users on 4k tends to be easier anyways since you end up blowing the stack a lot sooner. On sh we've had pretty good luck with it, though most of our users are using fairly deterministic workloads and continually profiling the footprint. Anything that runs away or uses an insane amount of stack space needs to be fixed well before that anyways, so catching it sooner is always preferable. I imagine the same case is true for m68knommu (even sans IRQ stacks).
CONFIG_DEBUG_STACKOVERFLOW should give you the same information, and if wanted with an arbitrary limit.
Quoted 3 lines
Things might be more sensitive on x86, but it's certainly not something that's a huge problem for the various embedded platforms to wire up, whether they want to go the IRQ stack route or not.
How many platforms use 4kB stacks on sh?
Only 1 out of 34 defconfigs uses it.
Are there any numbers for real life usage.
Quoted 7 lines
In any event, lack of support for something on embedded architectures in the kernel is more often due to apathy/utter indifference on the part of the architecture maintainer rather than being indicative of any intrinsic difficulty in supporting the thing in question. Most new "features" on the lesser maintained architectures tend to end up there either out of peer pressure or copying-and-pasting accidents rather than any sort of design. ;-)
arm or powerpc aren't exactly lesser maintained architectures.
4kB has shown to be a hard to achieve limit. After more than 4 years in mainline being available on i386 there are still cases where 4kB are not enough.
IMHO there seems to currently be a mismatch between it's maintainance cost and the actual number of users. That's in my opinion the main problem with it, no matter in which direction it gets resolved.
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedRe: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Paul MundtQuoted 22 lines · Paul Mundt
Quoted 9 lines · Adrian Bunk
Quoted 8 lines · Linus Torvalds
Quoted 1 line · Adrian Bunk
When did we get callpaths like like nfs+xfs+md+scsi reliably working with 4kB stacks on x86-32?
XFS may never have been usable, but the rest, sure.
And you seem to be making this whole argument an excuse to SUCK, adn an excuse to let gcc crap even more on our stack space.
Why?
Quoted 9 lines
Quoted 1 line
Why aren't you saying that we should be able to do better? Instead, you seem to asking us to do even worse than we do now?
My main point is:
- getting 4kB stacks working reliably is a hard task
- having an eye on gcc increasing the stack usage, and fixing it if required, is relatively easyIf we should be able to do better at getting (and keeping) 4kB stacks working, then coping with possible inlining problems caused by gcc should not be a big problem for us.
Out of the architectures you've mentioned for 4k stacks, they also tend to do IRQ stacks, which is something you seem to have overlooked.
Quoted 8 lines
In addition to that, debugging the runaway stack users on 4k tends to be easier anyways since you end up blowing the stack a lot sooner. On sh we've had pretty good luck with it, though most of our users are using fairly deterministic workloads and continually profiling the footprint. Anything that runs away or uses an insane amount of stack space needs to be fixed well before that anyways, so catching it sooner is always preferable. I imagine the same case is true for m68knommu (even sans IRQ stacks).
Yep, definitely true for m68knommu in my experience. I haven't had any problems with 4k stacks recently. But yes the workloads do tend to be constrained - and almost never use any of the more exotic filesystems or drivers.
Quoted 11 lines
Things might be more sensitive on x86, but it's certainly not something that's a huge problem for the various embedded platforms to wire up, whether they want to go the IRQ stack route or not.
In any event, lack of support for something on embedded architectures in the kernel is more often due to apathy/utter indifference on the part of the architecture maintainer rather than being indicative of any intrinsic difficulty in supporting the thing in question. Most new "features" on the lesser maintained architectures tend to end up there either out of peer pressure or copying-and-pasting accidents rather than any sort of design. ;-)
Indeed :-)
Regards
Greg
------------------------------------------------------------------------
Greg Ungerer -- Chief Software Dude EMAIL: gerg-XXXsiaCtIV5Wk0Htik3J/[email protected] Secure Computing Corporation PHONE: +61 7 3435 2888 825 Stanley St, FAX: +61 7 3891 3630 Woolloongabba, QLD, 4102, Australia WEB: http://www.SnapGear.com
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Paul MundtQuoted 22 lines · Paul Mundt
Quoted 9 lines · Adrian Bunk
Quoted 8 lines · Linus Torvalds
Quoted 1 line · Adrian Bunk
When did we get callpaths like like nfs+xfs+md+scsi reliably working with 4kB stacks on x86-32?
XFS may never have been usable, but the rest, sure.
And you seem to be making this whole argument an excuse to SUCK, adn an excuse to let gcc crap even more on our stack space.
Why?
Quoted 9 lines
Quoted 1 line
Why aren't you saying that we should be able to do better? Instead, you seem to asking us to do even worse than we do now?
My main point is:
- getting 4kB stacks working reliably is a hard task
- having an eye on gcc increasing the stack usage, and fixing it if required, is relatively easyIf we should be able to do better at getting (and keeping) 4kB stacks working, then coping with possible inlining problems caused by gcc should not be a big problem for us.
Out of the architectures you've mentioned for 4k stacks, they also tend to do IRQ stacks, which is something you seem to have overlooked.
Quoted 8 lines
In addition to that, debugging the runaway stack users on 4k tends to be easier anyways since you end up blowing the stack a lot sooner. On sh we've had pretty good luck with it, though most of our users are using fairly deterministic workloads and continually profiling the footprint. Anything that runs away or uses an insane amount of stack space needs to be fixed well before that anyways, so catching it sooner is always preferable. I imagine the same case is true for m68knommu (even sans IRQ stacks).
Yep, definitely true for m68knommu in my experience. I haven't had any problems with 4k stacks recently. But yes the workloads do tend to be constrained - and almost never use any of the more exotic filesystems or drivers.
Quoted 11 lines
Things might be more sensitive on x86, but it's certainly not something that's a huge problem for the various embedded platforms to wire up, whether they want to go the IRQ stack route or not.
In any event, lack of support for something on embedded architectures in the kernel is more often due to apathy/utter indifference on the part of the architecture maintainer rather than being indicative of any intrinsic difficulty in supporting the thing in question. Most new "features" on the lesser maintained architectures tend to end up there either out of peer pressure or copying-and-pasting accidents rather than any sort of design. ;-)
Indeed :-)
Regards
Greg
------------------------------------------------------------------------
Greg Ungerer -- Chief Software Dude EMAIL: [email protected] Secure Computing Corporation PHONE: +61 7 3435 2888 825 Stanley St, FAX: +61 7 3891 3630 Woolloongabba, QLD, 4102, Australia WEB: http://www.SnapGear.com
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Adrian BunkQuoted 1 line
You have a good point that aiming at 4kB makes 8kB a very safe choice.
Not really no - we use separate IRQ stacks in 4K but not 8K mode on x86-32. That means you've actually got no more space if you are unlucky with the timing of events. The 8K mode is merely harder to debug.
If 4K stacks really are not safe then x86-32 really really needs to switch to using IRQ stacks in 8K stack mode as well.
Alan
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 6 lines · Alan Cox
Quoted 1 line
You have a good point that aiming at 4kB makes 8kB a very safe choice.
Not really no - we use separate IRQ stacks in 4K but not 8K mode on x86-32. That means you've actually got no more space if you are unlucky with the timing of events. The 8K mode is merely harder to debug.
By your logic though, XFS on x86 should work fine with 4K stacks - many will attest that it does not and blows up due to stack issues.
I have first hand experiences of things blowing up with deep call chains when using 4K stacks where 8K worked just fine on same workload.
So there is definitely some other problem with 4K stacks.
Thanks
Parag
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 8 lines
By your logic though, XFS on x86 should work fine with 4K stacks - many will attest that it does not and blows up due to stack issues.
I have first hand experiences of things blowing up with deep call chains when using 4K stacks where 8K worked just fine on same workload.
So there is definitely some other problem with 4K stacks.
Nothing of the sort. If it blows up with a 4K stack it will almost certainly blow up with an 8K stack *eventually* - when a heavy stack usage coincides with a heavy stack using IRQ handler.
You won't catch it in simple testing, you won't catch it in trivial simulation and it'll be incredibly hard to reproduce. Not the kind of bug you want in a production system really. IRQ stacks make things much more predictable.
Alan
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Quoted 9 lines · Alan Cox
Quoted 8 lines
By your logic though, XFS on x86 should work fine with 4K stacks - many will attest that it does not and blows up due to stack issues.
I have first hand experiences of things blowing up with deep call chains when using 4K stacks where 8K worked just fine on same workload.
So there is definitely some other problem with 4K stacks.
Quoted 8 lines
Nothing of the sort. If it blows up with a 4K stack it will almost certainly blow up with an 8K stack *eventually* - when a heavy stack usage coincides with a heavy stack using IRQ handler.
You won't catch it in simple testing, you won't catch it in trivial simulation and it'll be incredibly hard to reproduce. Not the kind of bug you want in a production system really. IRQ stacks make things much more predictable.
I see - so if I end up having a workload on 8k where heavy stack using IRQs and deep kernel call chains come at the same time - even 8K will blow up.
So 4K will blow too except that it doesn't require IRQs also to use heavy stack, just XFS is good enough :)
It then seems like the IRQs using lot of stack is not so much of a problem in the current kernel as much as deeper call chains and stack usage of normal non-irq path code is.
So 8k makes it possible for the deeper call chains of non-irq path to survive since they get better part of the 8K to themselves and IRQs can do with less almost always.
At least that's what I can derive from the fact that we do not have lots of reports of 8K stack blowing up.
Thanks
Parag
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Linus TorvaldsQuoted 10 lines · Linus Torvalds
The downsides of inlining are big enough from both a debugging and a real code generation angle (eg stack usage like this), that the upsides (_somesimes_ smaller kernel, possibly slightly faster code) simply aren't relevant.
So the "noinline" was random, yes, but this is a real issue. Looking at checkstack output for a saner config (NR_CPUS=16), the top entries for me are things like
ide_generic_init [vmlinux]: 1384 idefloppy_ioctl [vmlinux]: 1208 e1000_check_options [vmlinux]: 1152 ...which are "leaf" functions. They are broken as hell (the e1000 is apparently because it builds structs on the stack that should all be "static const", for example), but they are different from something like the module init sequence in that they are not going to affect anything else.
e1000_check_options builds a struct (singular) on the stack, really... struct e1000_option is reasonably small.
The problem, which has also shown itself in large ioctl-style switch{} statements, is that gcc will generate code such that the stack usage from independent code branches
if {cond1} {
char buster1[1000];
foo(buster1);
} else if (cond2) {
char buster2[1000];
foo(buster2);
}are added together, not noticed as mutually exclusive.
Of course, adding 'static const' as you noted is a reasonable workaround, but gcc is really annoying WRT stack allocation in this manner.
I had problems in the past, before struct ethtool_ops, with like ethtool ioctl switch statements using gobs of stack. In fact, that was a big motivation for struct ethtool_ops.
Jeff
e1000 horridness (was Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected)
Quoted 3 lines · Jeff Garzik
e1000_check_optionsbuilds a struct (singular) on the stack, really... structe1000_optionis reasonably small.
No it doesn't.
Look a bit more closely.
It builds a struct (singular) MANY MANY times. It also then builds up a huge e1000_opt_list[] array, even though it is const and should be static (and const).
I know. I wrote a patch to FIX it.
Here's the patch. It shrinks the stack from 1152 bytes to 192 bytes (the first version, that only did the e1000_option part, got it down to 600 bytes). About half comes from not using multiple "e1000_option" structures, the other half comes from turning the "e1000_opt_list[]" arrays into "static const" instead, so that gcc doesn't copy them onto the stack.
Most of the patch is actually doing things like turning
struct struct e1000_option opt = {(which declares a _new_ e1000_option variable each time) into
opt = (struct e1000_option) {which just re-uses the single variable.
It becomes slightly larger than that, because some places the "opt = .." had to be moved around, since it's no longer a variable declaration, but a regular assignment.
The rest is just adding "const" to the right places, and turning
struct e1000_opt_list speed_list[] = ..
into
static const struct e1000_opt_list speed_list[] = ..
instead, and fixing the indentation to be more straightforward.
I have not tested the dang thing, but I think it's correct. And it turns stack usage from "totally horrible and broken" into "pretty reasonable".
Linus
--- drivers/net/e1000/e1000_param.c | 81 +++++++++++++++++++++----------------- 1 files changed, 45 insertions(+), 36 deletions(-)
drivers/net/e1000/e1000_param.c +45 -36
@@ -208,7 +208,7 @@ struct e1000_option { } r;struct {/* list_option info */ int nr; -struct e1000_opt_list {int i;char *str; } *p; +const struct e1000_opt_list {int i;char *str; } *p; } l; } arg; };
@@ -242,7 +242,7 @@ static int __devinit e1000_validate_option(unsigned int *value,break ;case list_option: {int i; -struct e1000_opt_list *ent; +const struct e1000_opt_list *ent;for (i =0 ; i < opt->arg.l.nr; i++) { ent = &opt->arg.l.p[i];
@@ -279,7 +279,9 @@ static void e1000_check_copper_options(struct e1000_adapter *adapter);void __devinit e1000_check_options (struct e1000_adapter *adapter) { +struct e1000_option opt;int bd = adapter->bd_number; +if (bd >=E1000_MAX_NIC ) {DPRINTK (PROBE ,NOTICE ,"Warning: no configuration for board #%i\n" , bd);
@@ -287,19 +289,21 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) } {/* Transmit Descriptor Count */ -struct e1000_option opt = { +struct e1000_tx_ring *tx_ring = adapter->tx_ring; +int i; +e1000_mac_type mac_type = adapter->hw.mac_type; + + opt = (struct e1000_option ) { .type = range_option, .name ="Transmit Descriptors" , .err ="using default of " __MODULE_STRING(E1000_DEFAULT_TXD ), .def =E1000_DEFAULT_TXD , - .arg = { .r = { .min =E1000_MIN_TXD }} + .arg = { .r = { + .min =E1000_MIN_TXD , + .max = mac_type < e1000_82544 ?E1000_MAX_TXD :E1000_MAX_82544_TXD + }} }; -struct e1000_tx_ring *tx_ring = adapter->tx_ring; -int i; -e1000_mac_type mac_type = adapter->hw.mac_type; - opt.arg.r.max = mac_type < e1000_82544 ? -E1000_MAX_TXD :E1000_MAX_82544_TXD ;if (num_TxDescriptors > bd) { tx_ring->count = TxDescriptors[bd];
@@ -313,19 +317,21 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) tx_ring[i].count = tx_ring->count; } {/* Receive Descriptor Count */ -struct e1000_option opt = { +struct e1000_rx_ring *rx_ring = adapter->rx_ring; +int i; +e1000_mac_type mac_type = adapter->hw.mac_type; + + opt = (struct e1000_option ) { .type = range_option, .name ="Receive Descriptors" , .err ="using default of " __MODULE_STRING(E1000_DEFAULT_RXD ), .def =E1000_DEFAULT_RXD , - .arg = { .r = { .min =E1000_MIN_RXD }} + .arg = { .r = { + .min =E1000_MIN_RXD , + .max = mac_type < e1000_82544 ?E1000_MAX_RXD :E1000_MAX_82544_RXD + }} }; -struct e1000_rx_ring *rx_ring = adapter->rx_ring; -int i; -e1000_mac_type mac_type = adapter->hw.mac_type; - opt.arg.r.max = mac_type < e1000_82544 ?E1000_MAX_RXD : -E1000_MAX_82544_RXD ;if (num_RxDescriptors > bd) { rx_ring->count = RxDescriptors[bd];
@@ -339,7 +345,7 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) rx_ring[i].count = rx_ring->count; } {/* Checksum Offload Enable/Disable */ -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = enable_option, .name ="Checksum Offload" , .err ="defaulting to Enabled" ,
@@ -363,7 +369,7 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) { E1000_FC_FULL, "Flow Control Enabled" }, { E1000_FC_DEFAULT, "Flow Control Hardware Default" }}; -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = list_option, .name ="Flow Control" , .err ="reading default settings from EEPROM" ,
@@ -381,7 +387,7 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) } } {/* Transmit Interrupt Delay */ -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = range_option, .name ="Transmit Interrupt Delay" , .err ="using default of " __MODULE_STRING(DEFAULT_TIDV ),
@@ -399,7 +405,7 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) } } {/* Transmit Absolute Interrupt Delay */ -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = range_option, .name ="Transmit Absolute Interrupt Delay" , .err ="using default of " __MODULE_STRING(DEFAULT_TADV ),
@@ -417,7 +423,7 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) } } {/* Receive Interrupt Delay */ -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = range_option, .name ="Receive Interrupt Delay" , .err ="using default of " __MODULE_STRING(DEFAULT_RDTR ),
@@ -435,7 +441,7 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) } } {/* Receive Absolute Interrupt Delay */ -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = range_option, .name ="Receive Absolute Interrupt Delay" , .err ="using default of " __MODULE_STRING(DEFAULT_RADV ),
@@ -453,7 +459,7 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) } } {/* Interrupt Throttling Rate */ -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = range_option, .name ="Interrupt Throttling Rate (ints/sec)" , .err ="using default of " __MODULE_STRING(DEFAULT_ITR ),
@@ -497,7 +503,7 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) } } {/* Smart Power Down */ -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = enable_option, .name ="PHY Smart Power Down" , .err ="defaulting to Disabled" ,
@@ -513,7 +519,7 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) } } {/* Kumeran Lock Loss Workaround */ -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = enable_option, .name ="Kumeran Lock Loss Workaround" , .err ="defaulting to Enabled" ,
@@ -578,16 +584,18 @@ static void __devinit e1000_check_fiber_options(struct e1000_adapter *adapter)static void __devinit e1000_check_copper_options (struct e1000_adapter *adapter) { +struct e1000_option opt;unsigned int speed, dplx, an;int bd = adapter->bd_number; {/* Speed */ -struct e1000_opt_list speed_list[] = {{0 ,"" }, - {SPEED_10 ,"" }, - {SPEED_100 ,"" }, - {SPEED_1000 ,"" }}; +static const struct e1000_opt_list speed_list[] = { + {0 ,"" }, + {SPEED_10 ,"" }, + {SPEED_100 ,"" }, + {SPEED_1000 ,"" }}; -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = list_option, .name ="Speed" , .err = "parameter ignored",
@@ -604,11 +612,12 @@ static void __devinit e1000_check_copper_options(struct e1000_adapter *adapter) } } {/* Duplex */ -struct e1000_opt_list dplx_list[] = {{0 ,"" }, - {HALF_DUPLEX ,"" }, - {FULL_DUPLEX ,"" }}; +static const struct e1000_opt_list dplx_list[] = { + {0 ,"" }, + {HALF_DUPLEX ,"" }, + {FULL_DUPLEX ,"" }}; -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = list_option, .name ="Duplex" , .err ="parameter ignored" ,
@@ -637,7 +646,7 @@ static void __devinit e1000_check_copper_options(struct e1000_adapter *adapter) "parameter ignored\n"); adapter->hw.autoneg_advertised =AUTONEG_ADV_DEFAULT ; }else {/* Autoneg */ - struct e1000_opt_list an_list[] = +static const struct e1000_opt_list an_list[] = #define AA "AutoNeg advertising " {{0x01 ,AA "10/HD" }, { 0x02, AA "10/FD" },
@@ -671,7 +680,7 @@ static void __devinit e1000_check_copper_options(struct e1000_adapter *adapter) { 0x2e, AA "1000/FD, 100/FD, 100/HD, 10/FD" }, { 0x2f, AA "1000/FD, 100/FD, 100/HD, 10/FD, 10/HD" }}; -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = list_option, .name ="AutoNeg" , .err ="parameter ignored" ,
Re: e1000 horridness (was Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected)
Quoted 7 lines · Linus Torvalds
Quoted 2 lines · Jeff Garzik
e1000_check_optionsbuilds a struct (singular) on the stack, really... structe1000_optionis reasonably small.No it doesn't.
Quoted 7 lines
Look a bit more closely.
It builds a struct (singular) MANY MANY times. It also then builds up a huge e1000_opt_list[] array, even though it is const and should be static (and const).
I know. I wrote a patch to FIX it.
totally cool patch afaics - if I still maintained this driver I'd have this tested and merged right away :)
I suppose Jeff Kirsher is already doing so right now.
I suppose that he'll have to look at the other Intel ethernet drivers as well :)
Jeff, please add my:
Cheers,
Auke
Quoted 271 lines
Here's the patch. It shrinks the stack from 1152 bytes to 192 bytes (the first version, that only did the
e1000_optionpart, got it down to 600 bytes). About half comes from not using multiple "e1000_option" structures, the other half comes from turning the "e1000_opt_list[]" arrays into "static const" instead, so that gcc doesn't copy them onto the stack.Most of the patch is actually doing things like turning
struct struct e1000_option opt = {(which declares a _new_
e1000_optionvariable each time) intoopt = (struct e1000_option) {which just re-uses the single variable.
It becomes slightly larger than that, because some places the "opt = .." had to be moved around, since it's no longer a variable declaration, but a regular assignment.
The rest is just adding "const" to the right places, and turning
struct e1000_opt_list speed_list[] = ..into
static const struct e1000_opt_list speed_list[] = ..instead, and fixing the indentation to be more straightforward.
I have not tested the dang thing, but I think it's correct. And it turns stack usage from "totally horrible and broken" into "pretty reasonable".
Linus
--- drivers/net/e1000/e1000_param.c | 81 +++++++++++++++++++++----------------- 1 files changed, 45 insertions(+), 36 deletions(-)drivers/net/e1000/e1000_param.c +45
-36@@ -208,7 +208,7 @@ struct e1000_option { } r;struct {/* list_option info */ int nr; -struct e1000_opt_list {int i;char *str; } *p; +const struct e1000_opt_list {int i;char *str; } *p; } l; } arg; };@@ -242,7 +242,7 @@ static int __devinit e1000_validate_option(unsigned int *value,break ;case list_option: {int i; -struct e1000_opt_list *ent; +const struct e1000_opt_list *ent;for (i =0 ; i < opt->arg.l.nr; i++) { ent = &opt->arg.l.p[i];@@ -279,7 +279,9 @@ static void e1000_check_copper_options(struct e1000_adapter *adapter);void __devinit e1000_check_options (struct e1000_adapter *adapter) { +struct e1000_option opt;int bd = adapter->bd_number; +if (bd >=E1000_MAX_NIC ) {DPRINTK (PROBE ,NOTICE ,"Warning: no configuration for board #%i\n" , bd);@@ -287,19 +289,21 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) } {/* Transmit Descriptor Count */ -struct e1000_option opt = { +struct e1000_tx_ring *tx_ring = adapter->tx_ring; +int i; +e1000_mac_type mac_type = adapter->hw.mac_type; + + opt = (struct e1000_option ) { .type = range_option, .name ="Transmit Descriptors" , .err ="using default of " __MODULE_STRING(E1000_DEFAULT_TXD ), .def =E1000_DEFAULT_TXD , - .arg = { .r = { .min =E1000_MIN_TXD }} + .arg = { .r = { + .min =E1000_MIN_TXD , + .max = mac_type < e1000_82544 ?E1000_MAX_TXD :E1000_MAX_82544_TXD + }} }; -struct e1000_tx_ring *tx_ring = adapter->tx_ring; -int i; -e1000_mac_type mac_type = adapter->hw.mac_type; - opt.arg.r.max = mac_type < e1000_82544 ? -E1000_MAX_TXD :E1000_MAX_82544_TXD ;if (num_TxDescriptors > bd) { tx_ring->count = TxDescriptors[bd];@@ -313,19 +317,21 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) tx_ring[i].count = tx_ring->count; } {/* Receive Descriptor Count */ -struct e1000_option opt = { +struct e1000_rx_ring *rx_ring = adapter->rx_ring; +int i; +e1000_mac_type mac_type = adapter->hw.mac_type; + + opt = (struct e1000_option ) { .type = range_option, .name ="Receive Descriptors" , .err ="using default of " __MODULE_STRING(E1000_DEFAULT_RXD ), .def =E1000_DEFAULT_RXD , - .arg = { .r = { .min =E1000_MIN_RXD }} + .arg = { .r = { + .min =E1000_MIN_RXD , + .max = mac_type < e1000_82544 ?E1000_MAX_RXD :E1000_MAX_82544_RXD + }} }; -struct e1000_rx_ring *rx_ring = adapter->rx_ring; -int i; -e1000_mac_type mac_type = adapter->hw.mac_type; - opt.arg.r.max = mac_type < e1000_82544 ?E1000_MAX_RXD : -E1000_MAX_82544_RXD ;if (num_RxDescriptors > bd) { rx_ring->count = RxDescriptors[bd];@@ -339,7 +345,7 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) rx_ring[i].count = rx_ring->count; } {/* Checksum Offload Enable/Disable */ -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = enable_option, .name ="Checksum Offload" , .err ="defaulting to Enabled" ,@@ -363,7 +369,7 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) { E1000_FC_FULL, "Flow Control Enabled" }, { E1000_FC_DEFAULT, "Flow Control Hardware Default" }}; -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = list_option, .name ="Flow Control" , .err ="reading default settings from EEPROM" ,@@ -381,7 +387,7 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) } } {/* Transmit Interrupt Delay */ -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = range_option, .name ="Transmit Interrupt Delay" , .err ="using default of " __MODULE_STRING(DEFAULT_TIDV ),@@ -399,7 +405,7 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) } } {/* Transmit Absolute Interrupt Delay */ -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = range_option, .name ="Transmit Absolute Interrupt Delay" , .err ="using default of " __MODULE_STRING(DEFAULT_TADV ),@@ -417,7 +423,7 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) } } {/* Receive Interrupt Delay */ -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = range_option, .name ="Receive Interrupt Delay" , .err ="using default of " __MODULE_STRING(DEFAULT_RDTR ),@@ -435,7 +441,7 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) } } {/* Receive Absolute Interrupt Delay */ -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = range_option, .name ="Receive Absolute Interrupt Delay" , .err ="using default of " __MODULE_STRING(DEFAULT_RADV ),@@ -453,7 +459,7 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) } } {/* Interrupt Throttling Rate */ -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = range_option, .name ="Interrupt Throttling Rate (ints/sec)" , .err ="using default of " __MODULE_STRING(DEFAULT_ITR ),@@ -497,7 +503,7 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) } } {/* Smart Power Down */ -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = enable_option, .name ="PHY Smart Power Down" , .err ="defaulting to Disabled" ,@@ -513,7 +519,7 @@ void __devinit e1000_check_options(struct e1000_adapter *adapter) } } {/* Kumeran Lock Loss Workaround */ -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = enable_option, .name ="Kumeran Lock Loss Workaround" , .err ="defaulting to Enabled" ,@@ -578,16 +584,18 @@ static void __devinit e1000_check_fiber_options(struct e1000_adapter *adapter)static void __devinit e1000_check_copper_options (struct e1000_adapter *adapter) { +struct e1000_option opt;unsigned int speed, dplx, an;int bd = adapter->bd_number; {/* Speed */ -struct e1000_opt_list speed_list[] = {{0 ,"" }, - {SPEED_10 ,"" }, - {SPEED_100 ,"" }, - {SPEED_1000 ,"" }}; +static const struct e1000_opt_list speed_list[] = { + {0 ,"" }, + {SPEED_10 ,"" }, + {SPEED_100 ,"" }, + {SPEED_1000 ,"" }}; -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = list_option, .name ="Speed" , .err = "parameter ignored",@@ -604,11 +612,12 @@ static void __devinit e1000_check_copper_options(struct e1000_adapter *adapter) } } {/* Duplex */ -struct e1000_opt_list dplx_list[] = {{0 ,"" }, - {HALF_DUPLEX ,"" }, - {FULL_DUPLEX ,"" }}; +static const struct e1000_opt_list dplx_list[] = { + {0 ,"" }, + {HALF_DUPLEX ,"" }, + {FULL_DUPLEX ,"" }}; -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = list_option, .name ="Duplex" , .err ="parameter ignored" ,@@ -637,7 +646,7 @@ static void __devinit e1000_check_copper_options(struct e1000_adapter *adapter) "parameter ignored\n"); adapter->hw.autoneg_advertised =AUTONEG_ADV_DEFAULT ; }else {/* Autoneg */ - struct e1000_opt_list an_list[] = +static const struct e1000_opt_list an_list[] = #define AA "AutoNeg advertising " {{0x01 ,AA "10/HD" }, { 0x02, AA "10/FD" },@@ -671,7 +680,7 @@ static void __devinit e1000_check_copper_options(struct e1000_adapter *adapter) { 0x2e, AA "1000/FD, 100/FD, 100/HD, 10/FD" }, { 0x2f, AA "1000/FD, 100/FD, 100/HD, 10/FD, 10/HD" }}; -struct e1000_option opt = { + opt = (struct e1000_option ) { .type = list_option, .name ="AutoNeg" , .err ="parameter ignored" ,
Re: e1000 horridness (was Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected)
Quoted 14 lines · Auke
Quoted 13 lines · Linus Torvalds
Quoted 2 lines · Jeff Garzik
e1000_check_optionsbuilds a struct (singular) on the stack, really... structe1000_optionis reasonably small.No it doesn't.
Look a bit more closely.
It builds a struct (singular) MANY MANY times. It also then builds up a huge e1000_opt_list[] array, even though it is const and should be static (and const).
Quoted 6 lines
Quoted 1 line
I know. I wrote a patch to FIX it.
totally cool patch afaics - if I still maintained this driver I'd have this tested and merged right away :)
I suppose Jeff Kirsher is already doing so right now
You suppose correctly.
.
Quoted 7 lines
I suppose that he'll have to look at the other Intel ethernet drivers as well :)
Jeff, please add my:
Reveiewed-by Auke Kok <[email protected]>
Will do.
Signature · 2 lines
Cheers, Jeff
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Linus TorvaldsQuoted 5 lines · Linus Torvalds
Quoted 2 lines · Linus Torvalds
This one makes no sense. It's triggering a BUG_ON(in_interrupt()), but then the call chain shows that there is no interrupt going on.
Quoted 7 lines
Ahh, later in that thread there's another totally unrelated oops in
debug_mutex_add_waiter().I'd guess that it is really wild pointer corrupting memory, quite possibly due to a double free or something like that. Alan - it would be good to run with
DEBUG_PAGE_ALLOCand SLUB debugging etc if you don't already do that?
With /just/ DEBUG_PAGE_ALLOC defined, I have seen two general panic types:
o A new double fault w/ SMP_DEBUG_PAGEALLOC problem (prob4.txt)
o The NULL pointer dereference @ 0x858 (prob4a.txt)
Enabling SLUB debugging to see what that shows
Alan
Begin: Loading essential drivers... ...
[ 6.680626] fuse init (API version 7.9) [ 6.680626] modprobe used greatest stack depth: 1720 bytes left [ 6.704224] double fault: 0000 [1] SMP DEBUG_PAGEALLOC [ 6.704224] CPU 1 [ 6.704224] Modules linked in: processor(+) fan thermal_sys fuse [ 6.710629] Pid: 1259, comm: modprobe Not tainted 2.6.27-rc3 #30 [ 6.710629] RIP: 0010:[<ffffffff802214dc>] [<ffffffff802214dc>] flat_send_IPI_allbutself+0x2c/0x80 [ 6.710629] RSP: 0018:ffff88021a513ff8 EFLAGS: 00010282 [ 6.710629] RAX: ffffffff805f5520 RBX: ffff88021a5141f8 RCX: 000000000000003f [ 6.710629] RDX: 0000000000000200 RSI: ffffffff80bf7920 RDI: ffff88021a5141f8 [ 6.710629] RBP: ffff88021a514408 R08: 0000000000000040 R09: 0000000000000040 [ 6.710629] R10: 0000000000001000 R11: 0000000000000000 R12: 00000000000000fc [ 6.710629] R13: ffff88021a514eb8 R14: 0000000000000001 R15: ffffffff8021cba0 [ 6.710629] FS: 00007f39acc136e0(0000) GS:ffff88022fc81a00(0000) knlGS:0000000000000000 [ 6.710629] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 6.710629] CR2: ffff88021a513fe8 CR3: 000000021a469000 CR4: 00000000000006e0 [ 6.710629] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 6.710629] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 6.710629] Process modprobe (pid: 1259, threadinfo ffff88021a514000, task ffff88021a9da050) [ 6.710629] Stack: <1>BUG: unable to handle kernel paging request at ffff88021a513ff8 [ 6.710629] IP: [<ffffffff8020dd22>] show_stack_log_lvl+0x82/0x130 [ 6.710629] PGD 202063 PUD 10067 PMD 22f176163 PTE 800000021a513160 [ 6.710629] Oops: 0000 [2] SMP DEBUG_PAGEALLOC [ 6.710629] CPU 1 [ 6.710629] Modules linked in: processor(+) fan thermal_sys fuse [ 6.710629] Pid: 1259, comm: modprobe Not tainted 2.6.27-rc3 #30 [ 6.710629] RIP: 0010:[<ffffffff8020dd22>] [<ffffffff8020dd22>] show_stack_log_lvl+0x82/0x130 [ 6.710629] RSP: 0018:ffff88022f12ce28 EFLAGS: 00010046 [ 6.710629] RAX: ffff88022fc81a00 RBX: ffff88021a513ff8 RCX: 000000000000000c [ 6.710629] RDX: ffff88021a513ff8 RSI: ffff88022f12cf58 RDI: 0000000000000000 [ 6.710629] RBP: ffff88022f12ce78 R08: ffffffff8059aea9 R09: 0000000000000001 [ 6.710629] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 [ 6.710629] R13: ffff88022f127fc0 R14: ffff88022f12bfc0 R15: 0000000000000000 [ 6.710629] FS: 00007f39acc136e0(0000) GS:ffff88022fc81a00(0000) knlGS:0000000000000000 [ 6.710629] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 6.710629] CR2: ffff88021a513ff8 CR3: 000000021a469000 CR4: 00000000000006e0 [ 6.710629] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 6.710629] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 6.710629] Process modprobe (pid: 1259, threadinfo ffff88021a514000, task ffff88021a9da050) [ 6.710629] Stack: ffff88022f12ce78 ffffffff8059aea9 ffff88021a514408 ffff88022f12cf58 [ 6.710629] ffff88021a513ff8 ffff88021a9da050 0000000000000000 ffff88022f12cf58 [ 6.710629] 0000000000000040 000000000000002b ffff88022f12ceb8 ffffffff8020dea8 [ 6.710629] Call Trace: [ 6.710629] <#DF> [<ffffffff8020dea8>] show_registers+0xd8/0x260 [ 6.710629] [<ffffffff8021cba0>] ? do_flush_tlb_all+0x0/0x40 [ 6.710629] [<ffffffff804a2083>] __die+0xa3/0x120 [ 6.710629] [<ffffffff8020e073>] die+0x43/0x90 [ 6.710629] [<ffffffff8020e1e3>] do_double_fault+0x63/0x70 [ 6.710629] [<ffffffff8020d4fd>] double_fault+0x7d/0x90 [ 6.710629] [<ffffffff8021cba0>] ? do_flush_tlb_all+0x0/0x40 [ 6.710629] [<ffffffff802214dc>] ? flat_send_IPI_allbutself+0x2c/0x80 [ 6.710629] <<EOE>> [ 6.710629] [ 6.710629] Code: 55 d0 85 c9 48 89 d3 7e 5a 45 31 e4 eb 44 4c 39 f3 77 44 66 0f 1f 44 00 00 74 7e 45 85 e4 74 0b 41 f6 c4 03 0f 1f 44 00 00 7 [ 6.720629] RIP [<ffffffff8020dd22>] show_stack_log_lvl+0x82/0x130 [ 6.720629] RSP <ffff88022f12ce28> [ 6.720629] CR2: ffff88021a513ff8 [ 6.720629] ---[ end trace d4fdf12ff3e07cc3 ]--- [ 7.052961] modprobe used greatest stack depth: 920 bytes left Killed [ 6.551876] all_generic_ide used greatest stack depth: 4784 bytes left Begin: Loading essential drivers... ... [ 6.658003] fuse init (API version 7.9) [ 6.661876] modprobe used greatest stack depth: 1720 bytes left [ 6.683510] ACPI: SSDT CFFD0D0A, 08C4 (r1 HPQOEM CPU_TM2 1 MSFT 100000E) [ 6.690632] BUG: unable to handle kernel NULL pointer dereference at 0000000000000858 [ 6.690632] IP: [<ffffffff8025e512>] debug_mutex_add_waiter+0x32/0x80 [ 6.690632] PGD 21a145067 PUD 22f13a067 PMD 0 [ 6.690632] Oops: 0002 [1] SMP DEBUG_PAGEALLOC [ 6.690632] CPU 1 [ 6.690632] Modules linked in: processor(+) fan thermal_sys fuse [ 6.690632] Pid: 1259, comm: modprobe Not tainted 2.6.27-rc3 #30 [ 6.690632] RIP: 0010:[<ffffffff8025e512>] [<ffffffff8025e512>] debug_mutex_add_waiter+0x32/0x80 [ 6.690632] RSP: 0018:ffff88021a959998 EFLAGS: 00010002 [ 6.690632] RAX: 0000000000000000 RBX: ffff88021a9599d8 RCX: 0000000000000000 [ 6.690632] RDX: 0000000000000001 RSI: ffff88021a9599d8 RDI: ffffffffa0091a60 [ 6.690632] RBP: ffff88021a9599b8 R08: ffffffff811deff0 R09: ffff8800a6fdb000 [ 6.690632] R10: ffffffffa008f524 R11: 0000000000000000 R12: ffffffffa0091a60 [ 6.690632] R13: ffff88021a958000 R14: ffff88021a1c2050 R15: ffffffffa0091a98 [ 6.690632] FS: 00007f28063c16e0(0000) GS:ffff88022fc81a00(0000) knlGS:0000000000000000 [ 6.690632] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 6.690632] CR2: 0000000000000858 CR3: 0000000219c64000 CR4: 00000000000006e0 [ 6.690632] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 6.690632] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 6.690632] Process modprobe (pid: 1259, threadinfo ffff88021a958000, task ffff88021a1c2050) [ 6.690632] Stack: 0000000000000000 ffffffffa0091a60 0000000000000246 ffffffffa008f524 [ 6.690632] ffff88021a959a38 ffffffff8049f856 ffffffffa008f524 ffffffffa0091a18 [ 6.690632] ffff88021a9599d8 ffff88021a9599d8 1111111111111111 1111111111111111 [ 6.690632] Call Trace: [ 6.690632] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 6.690632] [<ffffffff8049f856>] mutex_lock_nested+0xa6/0x250 [ 6.690632] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 6.690632] [<ffffffff80363884>] ? idr_pre_get+0x44/0x90 [ 6.690632] [<ffffffffa008f524>] get_idr+0x44/0xa0 [thermal_sys] [ 6.690632] [<ffffffffa008fe43>] thermal_cooling_device_register+0x83/0x250 [thermal_sys] [ 6.690632] [<ffffffffa019b2a3>] acpi_processor_start+0x64b/0x774 [processor] [ 6.690632] [<ffffffff8031ac0b>] ? __sysfs_add_one+0x6b/0xa0 [ 6.690632] [<ffffffff8031bcfc>] ? sysfs_do_create_link+0xbc/0x150 [ 6.690632] [<ffffffff803a821e>] acpi_start_single_object+0x2d/0x52 [ 6.690632] [<ffffffff803a9816>] acpi_device_probe+0x7e/0x92 [ 6.690632] [<ffffffff803dd6ab>] driver_probe_device+0x9b/0x1a0 [ 6.690632] [<ffffffff803dd836>] __driver_attach+0x86/0x90 [ 6.690632] [<ffffffff803dd7b0>] ? __driver_attach+0x0/0x90 [ 6.690632] [<ffffffff803dcbfd>] bus_for_each_dev+0x5d/0x90 [ 6.690632] [<ffffffff803dd4ec>] driver_attach+0x1c/0x20 [ 6.690632] [<ffffffff803dd239>] bus_add_driver+0x1e9/0x260 [ 6.690632] [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor] [ 6.690632] [<ffffffff803dda0f>] driver_register+0x5f/0x140 [ 6.690632] [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor] [ 6.690632] [<ffffffff803a9b26>] acpi_bus_register_driver+0x3e/0x40 [ 6.690632] [<ffffffffa0222094>] acpi_processor_init+0x94/0x107 [processor] [ 6.690632] [<ffffffff80209040>] _stext+0x40/0x180 [ 6.690632] [<ffffffff802a8bd1>] ? __vunmap+0xa1/0x110 [ 6.690632] [<ffffffff802678d2>] sys_init_module+0x142/0x1dc0 [ 6.690632] [<ffffffff80367dd6>] ? __up_read+0x46/0xb0 [ 6.690632] [<ffffffff8048e830>] ? cpu_down+0x0/0x70 [ 6.690632] [<ffffffff8020c34b>] system_call_fastpath+0x16/0x1b [ 6.690632] [ 6.690632] [ 6.690632] Code: 20 48 89 5d e8 4c 89 65 f0 48 89 f3 4c 89 6d f8 8b 47 08 49 89 d5 49 89 fc 89 c2 25 ff ff 00 00 c1 ea 10 39 c2 74 1d 49 8b 4 [ 6.690632] RIP [<ffffffff8025e512>] debug_mutex_add_waiter+0x32/0x80 [ 6.690632] RSP <ffff88021a959998> [ 6.690632] CR2: 0000000000000858 [ 6.690632] ---[ end trace 62c38812ae35bad0 ]--- [ 7.060556] ------------[ cut here ]------------ [ 7.060741] WARNING: at kernel/sched_fair.c:884 hrtick_start_fair+0x187/0x190() [ 7.060741] Modules linked in: processor(+) fan thermal_sys fuse [ 7.060741] Pid: 1259, comm: modprobe Tainted: G D 2.6.27-rc3 #30 [ 7.060741] [ 7.060741] Call Trace: [ 7.060741] <IRQ> [<ffffffff8023bcff>] warn_on_slowpath+0x5f/0x80 [ 7.060741] [<ffffffff8022db37>] hrtick_start_fair+0x187/0x190 [ 7.060741] [<ffffffff8022ee89>] enqueue_task_fair+0x49/0x250 [ 7.060741] [<ffffffff8022c4a0>] enqueue_task+0x50/0x60 [ 7.060741] [<ffffffff8022c4d3>] activate_task+0x23/0x40 [ 7.060741] [<ffffffff80231863>] try_to_wake_up+0x253/0x280 [ 7.060741] [<ffffffff8023189d>] default_wake_function+0xd/0x10 [ 7.060741] [<ffffffff802523e1>] autoremove_wake_function+0x11/0x40 [ 7.060741] [<ffffffff8022bf7a>] __wake_up_common+0x5a/0x90 [ 7.060741] [<ffffffff8022d583>] __wake_up+0x43/0x70 [ 7.060741] [<ffffffff8024eb20>] ? delayed_work_timer_fn+0x0/0x40 [ 7.060741] [<ffffffff8024e178>] insert_work+0x48/0x50 [ 7.060741] [<ffffffff8024eb01>] __queue_work+0x31/0x50 [ 7.060741] [<ffffffff8024eb52>] delayed_work_timer_fn+0x32/0x40 [ 7.060741] [<ffffffff8024606b>] run_timer_softirq+0x1bb/0x230 [ 7.060741] [<ffffffff80255d0a>] ? ktime_get_ts+0x4a/0x60 [ 7.060741] [<ffffffff8024178a>] __do_softirq+0x7a/0xf0 [ 7.060741] [<ffffffff8025cf9e>] ? tick_program_event+0x3e/0x70 [ 7.060741] [<ffffffff8020d69c>] call_softirq+0x1c/0x30 [ 7.060741] [<ffffffff8020f28d>] do_softirq+0x3d/0x80 [ 7.060741] [<ffffffff80241705>] irq_exit+0x85/0x90 [ 7.060741] [<ffffffff8021d648>] smp_apic_timer_interrupt+0x88/0xc0 [ 7.060741] [<ffffffff8020d0e6>] apic_timer_interrupt+0x66/0x70 [ 7.060741] <EOI> [<ffffffff804a15eb>] ? _spin_unlock_irq+0x2b/0x30 [ 7.060741] [<ffffffff804a0e75>] ? __down_read+0xa5/0xb7 [ 7.060741] [<ffffffff8026fdf5>] ? acct_collect+0x45/0x1d0 [ 7.060741] [<ffffffff8049fe57>] ? down_read+0x37/0x40 [ 7.060741] [<ffffffff8026fdf5>] ? acct_collect+0x45/0x1d0 [ 7.060741] [<ffffffff8026fdf5>] ? acct_collect+0x45/0x1d0 [ 7.060741] [<ffffffff8023f4ad>] ? do_exit+0x18d/0xa10 [ 7.060741] [<ffffffff803c8019>] ? do_unblank_screen+0x19/0x130 [ 7.060741] [<ffffffff804a1d17>] ? oops_end+0x87/0x90 [ 7.060741] [<ffffffff804a3fe3>] ? do_page_fault+0x663/0x800 [ 7.060741] [<ffffffff804a18ed>] ? error_exit+0x0/0x9a [ 7.060741] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.060741] [<ffffffff8025e512>] ? debug_mutex_add_waiter+0x32/0x80 [ 7.060741] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.060741] [<ffffffff8049f856>] ? mutex_lock_nested+0xa6/0x250 [ 7.060741] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.060741] [<ffffffff80363884>] ? idr_pre_get+0x44/0x90 [ 7.060741] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.060741] [<ffffffffa008fe43>] ? thermal_cooling_device_register+0x83/0x250 [thermal_sys] [ 7.060741] [<ffffffffa019b2a3>] ? acpi_processor_start+0x64b/0x774 [processor] [ 7.060741] [<ffffffff8031ac0b>] ? __sysfs_add_one+0x6b/0xa0 [ 7.060741] [<ffffffff8031bcfc>] ? sysfs_do_create_link+0xbc/0x150 [ 7.060741] [<ffffffff803a821e>] ? acpi_start_single_object+0x2d/0x52 [ 7.060741] [<ffffffff803a9816>] ? acpi_device_probe+0x7e/0x92 [ 7.060741] [<ffffffff803dd6ab>] ? driver_probe_device+0x9b/0x1a0 [ 7.060741] [<ffffffff803dd836>] ? __driver_attach+0x86/0x90 [ 7.060741] [<ffffffff803dd7b0>] ? __driver_attach+0x0/0x90 [ 7.060741] [<ffffffff803dcbfd>] ? bus_for_each_dev+0x5d/0x90 [ 7.060741] [<ffffffff803dd4ec>] ? driver_attach+0x1c/0x20 [ 7.060741] [<ffffffff803dd239>] ? bus_add_driver+0x1e9/0x260 [ 7.060741] [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor] [ 7.060741] [<ffffffff803dda0f>] ? driver_register+0x5f/0x140 [ 7.060741] [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor] [ 7.060741] [<ffffffff803a9b26>] ? acpi_bus_register_driver+0x3e/0x40 [ 7.060741] [<ffffffffa0222094>] ? acpi_processor_init+0x94/0x107 [processor] [ 7.060741] [<ffffffff80209040>] ? _stext+0x40/0x180 [ 7.060741] [<ffffffff802a8bd1>] ? __vunmap+0xa1/0x110 [ 7.060741] [<ffffffff802678d2>] ? sys_init_module+0x142/0x1dc0 [ 7.060741] [<ffffffff80367dd6>] ? __up_read+0x46/0xb0 [ 7.060741] [<ffffffff8048e830>] ? cpu_down+0x0/0x70 [ 7.060741] [<ffffffff8020c34b>] ? system_call_fastpath+0x16/0x1b [ 7.060741] [ 7.060741] ---[ end trace 62c38812ae35bad0 ]--- [ 7.060741] ------------[ cut here ]------------ [ 7.060741] kernel BUG at kernel/sched.c:1155! [ 7.060741] invalid opcode: 0000 [2] SMP DEBUG_PAGEALLOC [ 7.060741] CPU 1 [ 7.060741] Modules linked in: processor(+) fan thermal_sys fuse [ 7.060741] Pid: 1259, comm: modprobe Tainted: G D W 2.6.27-rc3 #30 [ 7.060741] RIP: 0010:[<ffffffff8022ce3b>] [<ffffffff8022ce3b>] resched_task+0x6b/0x70 [ 7.060741] RSP: 0018:ffff88022f12bce0 EFLAGS: 00010046 [ 7.060741] RAX: 00000000000006e5 RBX: 00000000012c627a RCX: ffff88021a958000 [ 7.060741] RDX: 00000000000006e5 RSI: 0000000000000000 RDI: ffff88021a1c2050 [ 7.060741] RBP: ffff88022f12bce0 R08: ffff88022f1d8038 R09: ffff88021a1c2088 [ 7.060741] R10: ffffffff810c9e00 R11: 0000000000000000 R12: ffff8800a6fc9000 [ 7.060741] R13: ffffffff810c9e00 R14: ffff88021a1c2050 R15: 0000000000000001 [ 7.060741] FS: 00007f28063c16e0(0000) GS:ffff88022fc81a00(0000) knlGS:0000000000000000 [ 7.060741] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [ 7.060741] CR2: 0000000000000858 CR3: 0000000219c64000 CR4: 00000000000006e0 [ 7.060741] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 7.060741] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 7.060741] Process modprobe (pid: 1259, threadinfo ffff88021a958000, task ffff88021a1c2050) [ 7.060741] Stack: ffff88022f12bd20 ffffffff802389c3 0000000000000400 0000000000400000 [ 7.060741] ffff88022f1d8000 ffff8800280a4e00 0000000000000001 0000000000000003 [ 7.060741] ffff88022f12bd70 ffffffff802316cf 0000000100000001 0000000000000000 [ 7.060741] Call Trace: [ 7.060741] <IRQ> [<ffffffff802389c3>] check_preempt_wakeup+0x133/0x1c0 [ 7.060741] [<ffffffff802316cf>] try_to_wake_up+0xbf/0x280 [ 7.060741] [<ffffffff8023189d>] default_wake_function+0xd/0x10 [ 7.060741] [<ffffffff802523e1>] autoremove_wake_function+0x11/0x40 [ 7.060741] [<ffffffff8022bf7a>] __wake_up_common+0x5a/0x90 [ 7.060741] [<ffffffff8022d583>] __wake_up+0x43/0x70 [ 7.060741] [<ffffffff8024eb20>] ? delayed_work_timer_fn+0x0/0x40 [ 7.060741] [<ffffffff8024e178>] insert_work+0x48/0x50 [ 7.060741] [<ffffffff8024eb01>] __queue_work+0x31/0x50 [ 7.060741] [<ffffffff8024eb52>] delayed_work_timer_fn+0x32/0x40 [ 7.060741] [<ffffffff8024606b>] run_timer_softirq+0x1bb/0x230 [ 7.060741] [<ffffffff80255d0a>] ? ktime_get_ts+0x4a/0x60 [ 7.060741] [<ffffffff8024178a>] __do_softirq+0x7a/0xf0 [ 7.060741] [<ffffffff8025cf9e>] ? tick_program_event+0x3e/0x70 [ 7.060741] [<ffffffff8020d69c>] call_softirq+0x1c/0x30 [ 7.060741] [<ffffffff8020f28d>] do_softirq+0x3d/0x80 [ 7.060741] [<ffffffff80241705>] irq_exit+0x85/0x90 [ 7.060741] [<ffffffff8021d648>] smp_apic_timer_interrupt+0x88/0xc0 [ 7.060741] [<ffffffff8020d0e6>] apic_timer_interrupt+0x66/0x70 [ 7.060741] <EOI> [<ffffffff804a15eb>] ? _spin_unlock_irq+0x2b/0x30 [ 7.060741] [<ffffffff804a0e75>] ? __down_read+0xa5/0xb7 [ 7.060741] [<ffffffff8026fdf5>] ? acct_collect+0x45/0x1d0 [ 7.060741] [<ffffffff8049fe57>] ? down_read+0x37/0x40 [ 7.060741] [<ffffffff8026fdf5>] ? acct_collect+0x45/0x1d0 [ 7.060741] [<ffffffff8026fdf5>] ? acct_collect+0x45/0x1d0 [ 7.060741] [<ffffffff8023f4ad>] ? do_exit+0x18d/0xa10 [ 7.060741] [<ffffffff803c8019>] ? do_unblank_screen+0x19/0x130 [ 7.060741] [<ffffffff804a1d17>] ? oops_end+0x87/0x90 [ 7.060741] [<ffffffff804a3fe3>] ? do_page_fault+0x663/0x800 [ 7.060741] [<ffffffff804a18ed>] ? error_exit+0x0/0x9a [ 7.060741] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.060741] [<ffffffff8025e512>] ? debug_mutex_add_waiter+0x32/0x80 [ 7.060741] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.060741] [<ffffffff8049f856>] ? mutex_lock_nested+0xa6/0x250 [ 7.060741] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.060741] [<ffffffff80363884>] ? idr_pre_get+0x44/0x90 [ 7.060741] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.060741] [<ffffffffa008fe43>] ? thermal_cooling_device_register+0x83/0x250 [thermal_sys] [ 7.060741] [<ffffffffa019b2a3>] ? acpi_processor_start+0x64b/0x774 [processor] [ 7.060741] [<ffffffff8031ac0b>] ? __sysfs_add_one+0x6b/0xa0 [ 7.060741] [<ffffffff8031bcfc>] ? sysfs_do_create_link+0xbc/0x150 [ 7.060741] [<ffffffff803a821e>] ? acpi_start_single_object+0x2d/0x52 [ 7.060741] [<ffffffff803a9816>] ? acpi_device_probe+0x7e/0x92 [ 7.060741] [<ffffffff803dd6ab>] ? driver_probe_device+0x9b/0x1a0 [ 7.060741] [<ffffffff803dd836>] ? __driver_attach+0x86/0x90 [ 7.060741] [<ffffffff803dd7b0>] ? __driver_attach+0x0/0x90 [ 7.060741] [<ffffffff803dcbfd>] ? bus_for_each_dev+0x5d/0x90 [ 7.060741] [<ffffffff803dd4ec>] ? driver_attach+0x1c/0x20 [ 7.060741] [<ffffffff803dd239>] ? bus_add_driver+0x1e9/0x260 [ 7.060741] [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor] [ 7.060741] [<ffffffff803dda0f>] ? driver_register+0x5f/0x140 [ 7.060741] [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor] [ 7.060741] [<ffffffff803a9b26>] ? acpi_bus_register_driver+0x3e/0x40 [ 7.060741] [<ffffffffa0222094>] ? acpi_processor_init+0x94/0x107 [processor] [ 7.060741] [<ffffffff80209040>] ? _stext+0x40/0x180 [ 7.060741] [<ffffffff802a8bd1>] ? __vunmap+0xa1/0x110 [ 7.060741] [<ffffffff802678d2>] ? sys_init_module+0x142/0x1dc0 [ 7.060741] [<ffffffff80367dd6>] ? __up_read+0x46/0xb0 [ 7.060741] [<ffffffff8048e830>] ? cpu_down+0x0/0x70 [ 7.060741] [<ffffffff8020c34b>] ? system_call_fastpath+0x16/0x1b [ 7.060741] [ 7.060741] [ 7.060741] Code: 8b 47 08 8b 50 1c 65 8b 04 25 24 00 00 00 39 c2 74 0d 0f ae f0 48 8b 47 08 f6 40 18 04 74 02 c9 c3 89 d7 ff 15 0f 79 3c 00 c [ 7.060741] RIP [<ffffffff8022ce3b>] resched_task+0x6b/0x70 [ 7.060741] RSP <ffff88022f12bce0> [ 7.060741] ---[ end trace 62c38812ae35bad0 ]--- [ 7.060741] Kernel panic - not syncing: Aiee, killing interrupt handler! [ 7.060741] ------------[ cut here ]------------ [ 7.060741] WARNING: at kernel/smp.c:328 smp_call_function_mask+0x25a/0x260() [ 7.060741] Modules linked in: processor(+) fan thermal_sys fuse [ 7.060741] Pid: 1259, comm: modprobe Tainted: G D W 2.6.27-rc3 #30 [ 7.060741] [ 7.060741] Call Trace: [ 7.060741] <IRQ> [<ffffffff8023bcff>] warn_on_slowpath+0x5f/0x80 [ 7.060741] [<ffffffff8026535a>] smp_call_function_mask+0x25a/0x260 [ 7.060741] [<ffffffff8036987d>] ? string+0x3d/0xd0 [ 7.060741] [<ffffffff80369d4b>] ? vsnprintf+0x43b/0x720 [ 7.060741] [<ffffffff8036987d>] ? string+0x3d/0xd0 [ 7.060741] [<ffffffff80369d4b>] ? vsnprintf+0x43b/0x720 [ 7.060741] [<ffffffff8036987d>] ? string+0x3d/0xd0 [ 7.060741] [<ffffffff8036987d>] ? string+0x3d/0xd0 [ 7.060741] [<ffffffff80369d4b>] ? vsnprintf+0x43b/0x720 [ 7.060741] [<ffffffff8036901e>] ? number+0x2ae/0x2d0 [ 7.060741] [<ffffffff8036901e>] ? number+0x2ae/0x2d0 [ 7.060741] [<ffffffff8026a0dd>] ? kallsyms_lookup+0x5d/0xa0 [ 7.060741] [<ffffffff8036901e>] ? number+0x2ae/0x2d0 [ 7.060741] [<ffffffff80369d4b>] ? vsnprintf+0x43b/0x720 [ 7.060741] [<ffffffff8036a098>] ? sprintf+0x68/0x70 [ 7.060741] [<ffffffff8036987d>] ? string+0x3d/0xd0 [ 7.060741] [<ffffffff804a4273>] ? __atomic_notifier_call_chain+0x83/0xa0 [ 7.060741] [<ffffffff804a41f0>] ? __atomic_notifier_call_chain+0x0/0xa0 [ 7.060741] [<ffffffff804a11b6>] ? _spin_unlock+0x26/0x30 [ 7.060741] [<ffffffff8021c470>] ? stop_this_cpu+0x0/0x30 [ 7.060741] [<ffffffff802653a0>] smp_call_function+0x40/0x50 [ 7.060741] [<ffffffff8021c4f3>] native_smp_send_stop+0x23/0x40 [ 7.060741] [<ffffffff8023c04f>] panic+0xaf/0x190 [ 7.060741] [<ffffffff8023cea7>] ? printk+0x67/0x70 [ 7.060741] [<ffffffff8049f7a9>] ? mutex_unlock+0x9/0x10 [ 7.060741] [<ffffffff80256f21>] ? blocking_notifier_call_chain+0x11/0x20 [ 7.060741] [<ffffffff8023fb89>] do_exit+0x869/0xa10 [ 7.060741] [<ffffffff803c8019>] ? do_unblank_screen+0x19/0x130 [ 7.060741] [<ffffffff804a1d17>] oops_end+0x87/0x90 [ 7.060741] [<ffffffff8020e08e>] die+0x5e/0x90 [ 7.060741] [<ffffffff804a2230>] do_trap+0x130/0x150 [ 7.060741] [<ffffffff8020e662>] do_invalid_op+0x92/0xb0 [ 7.060741] [<ffffffff8022ce3b>] ? resched_task+0x6b/0x70 [ 7.060741] [<ffffffff804a18ed>] error_exit+0x0/0x9a [ 7.060741] [<ffffffff8022ce3b>] ? resched_task+0x6b/0x70 [ 7.060741] [<ffffffff802389c3>] check_preempt_wakeup+0x133/0x1c0 [ 7.060741] [<ffffffff802316cf>] try_to_wake_up+0xbf/0x280 [ 7.060741] [<ffffffff8023189d>] default_wake_function+0xd/0x10 [ 7.060741] [<ffffffff802523e1>] autoremove_wake_function+0x11/0x40 [ 7.060741] [<ffffffff8022bf7a>] __wake_up_common+0x5a/0x90 [ 7.060741] [<ffffffff8022d583>] __wake_up+0x43/0x70 [ 7.060741] [<ffffffff8024eb20>] ? delayed_work_timer_fn+0x0/0x40 [ 7.060741] [<ffffffff8024e178>] insert_work+0x48/0x50 [ 7.060741] [<ffffffff8024eb01>] __queue_work+0x31/0x50 [ 7.060741] [<ffffffff8024eb52>] delayed_work_timer_fn+0x32/0x40 [ 7.060741] [<ffffffff8024606b>] run_timer_softirq+0x1bb/0x230 [ 7.060741] [<ffffffff80255d0a>] ? ktime_get_ts+0x4a/0x60 [ 7.060741] [<ffffffff8024178a>] __do_softirq+0x7a/0xf0 [ 7.060741] [<ffffffff8025cf9e>] ? tick_program_event+0x3e/0x70 [ 7.060741] [<ffffffff8020d69c>] call_softirq+0x1c/0x30 [ 7.060741] [<ffffffff8020f28d>] do_softirq+0x3d/0x80 [ 7.060741] [<ffffffff80241705>] irq_exit+0x85/0x90 [ 7.060741] [<ffffffff8021d648>] smp_apic_timer_interrupt+0x88/0xc0 [ 7.060741] [<ffffffff8020d0e6>] apic_timer_interrupt+0x66/0x70 [ 7.060741] <EOI> [<ffffffff804a15eb>] ? _spin_unlock_irq+0x2b/0x30 [ 7.060741] [<ffffffff804a0e75>] ? __down_read+0xa5/0xb7 [ 7.060741] [<ffffffff8026fdf5>] ? acct_collect+0x45/0x1d0 [ 7.060742] [<ffffffff8049fe57>] ? down_read+0x37/0x40 [ 7.060742] [<ffffffff8026fdf5>] ? acct_collect+0x45/0x1d0 [ 7.060742] [<ffffffff8026fdf5>] ? acct_collect+0x45/0x1d0 [ 7.060742] [<ffffffff8023f4ad>] ? do_exit+0x18d/0xa10 [ 7.060742] [<ffffffff803c8019>] ? do_unblank_screen+0x19/0x130 [ 7.060742] [<ffffffff804a1d17>] ? oops_end+0x87/0x90 [ 7.060742] [<ffffffff804a3fe3>] ? do_page_fault+0x663/0x800 [ 7.060742] [<ffffffff804a18ed>] ? error_exit+0x0/0x9a [ 7.060742] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.060742] [<ffffffff8025e512>] ? debug_mutex_add_waiter+0x32/0x80 [ 7.060742] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.060742] [<ffffffff8049f856>] ? mutex_lock_nested+0xa6/0x250 [ 7.060742] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.060742] [<ffffffff80363884>] ? idr_pre_get+0x44/0x90 [ 7.060742] [<ffffffffa008f524>] ? get_idr+0x44/0xa0 [thermal_sys] [ 7.060742] [<ffffffffa008fe43>] ? thermal_cooling_device_register+0x83/0x250 [thermal_sys] [ 7.060742] [<ffffffffa019b2a3>] ? acpi_processor_start+0x64b/0x774 [processor] [ 7.060742] [<ffffffff8031ac0b>] ? __sysfs_add_one+0x6b/0xa0 [ 7.060742] [<ffffffff8031bcfc>] ? sysfs_do_create_link+0xbc/0x150 [ 7.060742] [<ffffffff803a821e>] ? acpi_start_single_object+0x2d/0x52 [ 7.060742] [<ffffffff803a9816>] ? acpi_device_probe+0x7e/0x92 [ 7.060742] [<ffffffff803dd6ab>] ? driver_probe_device+0x9b/0x1a0 [ 7.060742] [<ffffffff803dd836>] ? __driver_attach+0x86/0x90 [ 7.060742] [<ffffffff803dd7b0>] ? __driver_attach+0x0/0x90 [ 7.060742] [<ffffffff803dcbfd>] ? bus_for_each_dev+0x5d/0x90 [ 7.060742] [<ffffffff803dd4ec>] ? driver_attach+0x1c/0x20 [ 7.060742] [<ffffffff803dd239>] ? bus_add_driver+0x1e9/0x260 [ 7.060742] [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor] [ 7.060742] [<ffffffff803dda0f>] ? driver_register+0x5f/0x140 [ 7.060742] [<ffffffffa0222000>] ? acpi_processor_init+0x0/0x107 [processor] [ 7.060742] [<ffffffff803a9b26>] ? acpi_bus_register_driver+0x3e/0x40 [ 7.060742] [<ffffffffa0222094>] ? acpi_processor_init+0x94/0x107 [processor] [ 7.060742] [<ffffffff80209040>] ? _stext+0x40/0x180 [ 7.060742] [<ffffffff802a8bd1>] ? __vunmap+0xa1/0x110 [ 7.060742] [<ffffffff802678d2>] ? sys_init_module+0x142/0x1dc0 [ 7.060742] [<ffffffff80367dd6>] ? __up_read+0x46/0xb0 [ 7.060742] [<ffffffff8048e830>] ? cpu_down+0x0/0x70 [ 7.060742] [<ffffffff8020c34b>] ? system_call_fastpath+0x16/0x1b [ 7.060742] [ 7.060742] ---[ end trace 62c38812ae35bad0 ]---
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
Adding in SLUB debugging doesn't show anything new (I think). Example boot log (w/ initcall_debug enabled) is at:
http://free.linux.hp.com/~adb/bug.11342/prob5.txt
This has happened 3 times in a row as well. Whilst this is being looked at, I'm going to fast-forward ahead to the latest in Linus' tree, and see if the problem is still occurring (I think Linus' point earlier about some sort of rogue timing and/or corruption bug is spot on, but it's probably better to see how close to "today's tree" I can reproduce this). I'll also try kernels w/ the problematic merge patch backed out to see if that still "fixes" (or more likely(?) just patches over the real problem).
Alan
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Alan D. BrunelleQuoted 4 lines · Alan D. Brunelle
With /just/
DEBUG_PAGE_ALLOCdefined, I have seen two general panic types:o A new double fault w/
SMP_DEBUG_PAGEALLOCproblem (prob4.txt)
Yeah, that's a stack overflow.
Confirmed.
Linus
Re: [Bug #11342] Linux 2.6.27-rc3: kernel BUG at mm/vmalloc.c - bisected
In reply to Linus TorvaldsI built a kernel @
commit 83097aca8567a0bd593534853b71fe0fa9a75d69
Author: Arjan van de Ven <arjan-VuQAYsv1563Yd54FQh9/[email protected]> Date: Sat Aug 23 21:45:21 2008 -0700
And it fails like the others do
o http://free.linux.hp.com/~adb/bug.11342/prob6.txt
SMP_DEBUG_PAGEALLOC
o http://free.linux.hp.com/~adb/bug.11342/prob6a.txt
[ 7.591198] BUG: unable to handle kernel NULL pointer dereference at
0000000000000858
I then backed out /just/ the merge for
commit 1c89ac55017f982355c7761e1c912c88c941483d
Merge: 88fa08f... b1b135c...
Author: Linus Torvalds <torvalds-de/[email protected]>
Date: Tue Aug 12 08:40:19 2008 -0700
And the machine has booted fine 5 times in a row.
I've put the latest .config up at
http://free.linux.hp.com/~adb/bug.11342/config.txt
Is there /some/ way to break down the patches within the merged patch, and I could by-hand bisect through those?
Here's what I did to take the latest tree, and back out that merge (to get booting kernels):
git-diff 88fa08f67bee1a0c765237bdac106a32872f57d2..1c89ac55017f982355c7761e1c912c88c941483d | patch -p1 -R
patching file Documentation/lguest/lguest.c
patching file arch/powerpc/Kconfig
patching file arch/x86/Kconfig
patching file arch/x86/mm/Makefile
patching file drivers/char/hvc_console.c
patching file drivers/lguest/page_tables.c
patching file include/linux/Kbuild
Hunk #1 succeeded at 358 (offset 2 lines).
patching file include/linux/init.h
patching file include/linux/mm.h
patching file init/main.c
patching file kernel/module.c
patching file kernel/stop_machine.c
patching file mm/Kconfig
patching file mm/util.c
Alan
[Bug #11343] SATA Cold Boot Problems with 2.6.27-rc[23] on nVidia 680i
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11343
Subject : SATA Cold Boot Problems with 2.6.27-rc[23] on nVidia 680i
Submitter : Manny Maxwell <[email protected]>
Date : 2008-08-14 4:16 (10 days old)
References : http://marc.info/?l=linux-kernel&m=121868782917600&w=4
Re: [Bug #11343] SATA Cold Boot Problems with 2.6.27-rc[23] on nVidia 680i
Quoted 8 lines · Rafael J. Wysocki
This message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Quoted 5 lines
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11343
Subject : SATA Cold Boot Problems with 2.6.27-rc[23] on nVidia 680i
Submitter : Manny Maxwell <[email protected]>
Date : 2008-08-14 4:16 (10 days old)
References : http://marc.info/?l=linux-kernel&m=121868782917600&w=4
hmmmm. Looking at changes between the two csets listed in the email (623fa57..8f616cd), all of them are driver-specific and unrelated to Manny's hardware except for
commit 2486fa561a3192bbbec39c7feef87a1e07bd6342 Author: Tejun Heo <[email protected]> Date: Thu Jul 31 07:52:40 2008 +0900 libata: update atapi disable handling
So you could try to revert that and see what happens. But given that small range of changes, it really seems like something else, maybe in the PCI subsystem (random guess).
Looking at the entire kernel, nothing jumps out, either. Its mostly fs updates (ext4, xfs), a networking update, an ARM update, and a libata update.
Also, some reset-related fixes just went in, so re-testing the latest -git would be helpful as well.
Jeff
[Bug #11358] net: forcedeth call restore mac addr in nv_shutdown path
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11358
Subject : net: forcedeth call restore mac addr in nv_shutdown path
Submitter : Yinghai Lu <[email protected]>
Date : 2008-08-17 3:30 (7 days old)
References : http://marc.info/?l=linux-kernel&m=121894389018584&w=4
Patch : http://marc.info/?l=linux-kernel&m=121894389018584&w=4
[Bug #11354] AMD Elan regression with 2.6.27-rc3
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11354
Subject : AMD Elan regression with 2.6.27-rc3
Submitter : Sean Young <[email protected]>
Date : 2008-08-15 18:37 (9 days old)
References : http://marc.info/?l=linux-kernel&m=121882578430056&w=4
[Bug #11355] Regression in 2.6.27-rc2 when cross-building the kernel
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11355
Subject : Regression in 2.6.27-rc2 when cross-building the kernel
Submitter : Larry Finger <[email protected]>
Date : 2008-08-16 2:38 (8 days old)
References : http://marc.info/?l=linux-kernel&m=121885432118368&w=4
Re: [Bug #11355] Regression in 2.6.27-rc2 when cross-building the kernel
Quoted 8 lines · Rafael J. Wysocki
This message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Quoted 5 lines
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11355
Subject : Regression in 2.6.27-rc2 when cross-building the kernel
Submitter : Larry Finger <[email protected]>
Date : 2008-08-16 2:38 (8 days old)
References : http://marc.info/?l=linux-kernel&m=121885432118368&w=4
As I wrote in the Bugzilla, I'm seeing a related problem.
Namely, I build kernels on one box, with 'make O=<target>', then I mount <target> on another one over NFS, 'cd' to it and try to install the kernel modules with 'make modules_install'. This results in 'HOSTCC firmware/ihex2fw' and 'fatal error: ...: Read-only file system'. It's readily reproducible.
Commenting out line 1130 of Makefile
("$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.fwinst obj=firmware __fw_modinst") obviously helps, so it looks like Makefile.fwinst needs fixing.
Thanks,
Rafael
David Woodhouse
maintainerRe: [Bug #11355] Regression in 2.6.27-rc2 when cross-building the kernel
Quoted 17 lines · Rafael J. Wysocki
Quoted 8 lines · Rafael J. Wysocki
This message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Quoted 5 lines
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11355
Subject : Regression in 2.6.27-rc2 when cross-building the kernel
Submitter : Larry Finger <[email protected]>
Date : 2008-08-16 2:38 (8 days old)
References : http://marc.info/?l=linux-kernel&m=121885432118368&w=4As I wrote in the Bugzilla, I'm seeing a related problem.
Quoted 8 lines
Namely, I build kernels on one box, with 'make O=<target>', then I mount <target> on another one over NFS, 'cd' to it and try to install the kernel modules with 'make
modules_install'. This results in 'HOSTCCfirmware/ihex2fw' and 'fatal error: ...: Read-only file system'. It's readily reproducible.Commenting out line 1130 of
Makefile
("$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.fwinst obj=firmware__fw_modinst") obviously helps, so it looks like Makefile.fwinst needs fixing.
I don't like this much, but it should do the trick... please confirm.
firmware/Makefile +15 -2
@@ -166,15 +166,28 @@ $(patsubst %,$(obj)/%.gen.o, $(fw-external-y)): $(obj)/%.gen.o: $(fwdir)/% $(obj)/%: $(obj)/%.ihex | $(objtree)/$(obj)/$$(dir %) $(call cmd,ihex) +# Don't depend on ihex2fw if we're installing and it already exists. +# Putting it after | in the dependencies doesn't seem sufficient when +# we're installing after a cross-compile, because ihex2fw has dependencies +# on stuff like /usr/lib/gcc/ppc64-redhat-linux/4.3.0/include/stddef.h and +# thus wants to be rebuilt. Which it can't be, if the prebuilt kernel tree +# is exported read-only for someone to run 'make install'. +ifeq ($(INSTALL):$(wildcard $(obj)/ihex2fw),install:$(obj)/ihex2fw) +ihex2fw_dep := +else +ihex2fw_dep := $(obj)/ihex2fw +endif + + # .HEX is also Intel HEX, but where the offset and length in each record # is actually meaningful, because the firmware has to be loaded in a certain # order rather than as a single binary blob. Thus, we convert them into our # more compact binary representation of ihex records (<linux/ihex.h>) -$(obj)/%.fw: $(obj)/%.HEX $(obj)/ihex2fw | $(objtree)/$(obj)/$$(dir %) +$(obj)/%.fw: $(obj)/%.HEX $(ihex2fw_dep) | $(objtree)/$(obj)/$$(dir %) $(call cmd,ihex2fw) # .H16 is our own modified form of Intel HEX, with 16-bit length for records. -$(obj)/%.fw: $(obj)/%.H16 $(obj)/ihex2fw | $(objtree)/$(obj)/$$(dir %) +$(obj)/%.fw: $(obj)/%.H16 $(ihex2fw_dep) | $(objtree)/$(obj)/$$(dir %) $(call cmd,h16tofw) $(firmware-dirs):
Signature · 1 line
dwmw2
Re: [Bug #11355] Regression in 2.6.27-rc2 when cross-building the kernel
Quoted 22 lines · David Woodhouse
Quoted 21 lines · Rafael J. Wysocki
Quoted 8 lines · Rafael J. Wysocki
This message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Quoted 5 lines
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11355
Subject : Regression in 2.6.27-rc2 when cross-building the kernel
Submitter : Larry Finger <[email protected]>
Date : 2008-08-16 2:38 (8 days old)
References : http://marc.info/?l=linux-kernel&m=121885432118368&w=4As I wrote in the Bugzilla, I'm seeing a related problem.
Namely, I build kernels on one box, with 'make O=<target>', then I mount <target> on another one over NFS, 'cd' to it and try to install the kernel modules with 'make
modules_install'. This results in 'HOSTCCfirmware/ihex2fw' and 'fatal error: ...: Read-only file system'. It's readily reproducible.
Quoted 5 lines
Quoted 3 lines
Commenting out line 1130 of
Makefile
("$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.fwinst obj=firmware__fw_modinst") obviously helps, so it looks like Makefile.fwinst needs fixing.I don't like this much, but it should do the trick... please confirm.
Yes, the patch fixes the problem. Thanks.
Larry
David Woodhouse
maintainerRe: [Bug #11355] Regression in 2.6.27-rc2 when cross-building the kernel
Quoted 23 lines · Larry Finger
Quoted 22 lines · David Woodhouse
Quoted 21 lines · Rafael J. Wysocki
Quoted 8 lines · Rafael J. Wysocki
This message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Quoted 5 lines
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11355
Subject : Regression in 2.6.27-rc2 when cross-building the kernel
Submitter : Larry Finger <[email protected]>
Date : 2008-08-16 2:38 (8 days old)
References : http://marc.info/?l=linux-kernel&m=121885432118368&w=4As I wrote in the Bugzilla, I'm seeing a related problem.
Namely, I build kernels on one box, with 'make O=<target>', then I mount <target> on another one over NFS, 'cd' to it and try to install the kernel modules with 'make
modules_install'. This results in 'HOSTCCfirmware/ihex2fw' and 'fatal error: ...: Read-only file system'. It's readily reproducible.
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Commenting out line 1130 of
Makefile
("$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.fwinst obj=firmware__fw_modinst") obviously helps, so it looks like Makefile.fwinst needs fixing.I don't like this much, but it should do the trick... please confirm.
Yes, the patch fixes the problem. Thanks.
Ok, I have a small handful of fixes to send to Linus for 2.6.27; Unless Sam (or someone else) comes up with a better answer, I'll make sure that goes with them.
Thanks.
Signature · 1 line
dwmw2
[Bug #11356] Linux 2.6.27-rc3 - build failure: undefined reference to `.lockdep_count_forward_deps'
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11356
Subject : Linux 2.6.27-rc3 - build failure: undefined reference to `.lockdep_count_forward_deps'
Submitter : Frans Pop <elendil-EIBgga6/0yRmR6Xm/[email protected]>
Date : 2008-08-16 19:11 (8 days old)
References : http://marc.info/?l=linux-kernel&m=121891396320127&w=4
Re: [Bug #11356] Linux 2.6.27-rc3 - build failure: undefined reference to `.lockdep_count_forward_deps'
Quoted 6 lines · Rafael J. Wysocki
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11356
Subject : Linux 2.6.27-rc3 - build failure: undefined reference to
`.lockdep_count_forward_deps'
Submitter : Frans Pop <elendil-EIBgga6/0yRmR6Xm/[email protected]>
Date : 2008-08-16 19:11 (8 days old)
References : http://marc.info/?l=linux-kernel&m=121891396320127&w=4
Fixed as per: http://marc.info/?l=linux-kernel&m=121898767530602&w=4
Adrian mentioned that he'd closed the bug, but apparently not.
Re: [Bug #11356] Linux 2.6.27-rc3 - build failure: undefined reference to `.lockdep_count_forward_deps'
Quoted 10 lines · Frans Pop
Quoted 6 lines · Rafael J. Wysocki
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11356
Subject : Linux 2.6.27-rc3 - build failure: undefined reference to
`.lockdep_count_forward_deps'
Submitter : Frans Pop <[email protected]>
Date : 2008-08-16 19:11 (8 days old)
References : http://marc.info/?l=linux-kernel&m=121891396320127&w=4Fixed as per: http://marc.info/?l=linux-kernel&m=121898767530602&w=4
Adrian mentioned that he'd closed the bug, but apparently not.
The bug is closed now.
Thanks,
Rafael
Re: [Bug #11356] Linux 2.6.27-rc3 - build failure: undefined reference to `.lockdep_count_forward_deps'
In reply to Frans PopQuoted 10 lines · Frans Pop
Quoted 6 lines · Rafael J. Wysocki
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11356
Subject : Linux 2.6.27-rc3 - build failure: undefined reference to
`.lockdep_count_forward_deps'
Submitter : Frans Pop <elendil-EIBgga6/0yRmR6Xm/[email protected]>
Date : 2008-08-16 19:11 (8 days old)
References : http://marc.info/?l=linux-kernel&m=121891396320127&w=4Fixed as per: http://marc.info/?l=linux-kernel&m=121898767530602&w=4
Adrian mentioned that he'd closed the bug, but apparently not.
Sorry, I missed that Rafael had opened two bugs for two people reporting the same issue, and only closed the other one.
cu
Adrian
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed[Bug #11357] Can not boot up with zd1211rw USB-Wlan Stick
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11357
Subject : Can not boot up with zd1211rw USB-Wlan Stick
Submitter : uwe <[email protected]>
Date : 2008-08-16 14:17 (8 days old)
[Bug #11361] my servers with nvidia mcp55 nic don't work with msi in second kernel by kexec
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11361
Subject : my servers with nvidia mcp55 nic don't work with msi in second kernel by kexec
Submitter : Yinghai Lu <[email protected]>
Date : 2008-08-17 6:25 (7 days old)
References : http://marc.info/?l=linux-kernel&m=121895439927053&w=4
Patch : http://marc.info/?l=linux-kernel&m=121917167232014&w=4
[Bug #11379] char/tpm: tpm_infineon no longer loaded for HP 2510p laptop
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11379
Subject : char/tpm: tpm_infineon no longer loaded for HP 2510p laptop
Submitter : Frans Pop <elendil-EIBgga6/0yRmR6Xm/[email protected]>
Date : 2008-08-18 13:40 (6 days old)
References : http://marc.info/?l=linux-kernel&m=121906698213329&w=4
Re: [Bug #11379] char/tpm: tpm_infineon no longer loaded for HP 2510p laptop
Quoted 6 lines · Rafael J. Wysocki
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11379
Subject : char/tpm:tpm_infineonno longer loaded for HP 2510p laptop
Submitter : Frans Pop <elendil-EIBgga6/0yRmR6Xm/[email protected]>
Date : 2008-08-18 13:40 (6 days old)
References : http://marc.info/?l=linux-kernel&m=121906698213329&w=4Handled-By Bjorn Helgaas <[email protected]>
Fixed with:
commit 5e4c6564c95ce127beeefe75e15cd11c93487436
Author: Kay Sievers <[email protected]> Date: Thu Aug 21 15:28:56 2008 +0200
pnp: fix "add acpi:* modalias entries"
Re: [Bug #11379] char/tpm: tpm_infineon no longer loaded for HP 2510p laptop
Quoted 8 lines · Frans Pop
Quoted 6 lines · Rafael J. Wysocki
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11379
Subject : char/tpm:tpm_infineonno longer loaded for HP 2510p laptop
Submitter : Frans Pop <[email protected]>
Date : 2008-08-18 13:40 (6 days old)
References : http://marc.info/?l=linux-kernel&m=121906698213329&w=4Handled-By Bjorn Helgaas <[email protected]>
Quoted 6 lines
Fixed with:
commit5e4c6564c95ce127beeefe75e15cd11c93487436Author: Kay Sievers <[email protected]> Date: Thu Aug 21 15:28:56 2008 +0200pnp: fix "add acpi:* modalias entries"
Thanks, closed.
Rafael
[Bug #11380] lockdep warning: cpu_add_remove_lock at:cpu_maps_update_begin+0x14/0x16
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11380
Subject : lockdep warning: cpu_add_remove_lock at:cpu_maps_update_begin+0x14/0x16
Submitter : Ingo Molnar <[email protected]>
Date : 2008-08-20 6:44 (4 days old)
References : http://marc.info/?l=linux-kernel&m=121921480931970&w=4
[Bug #11360] mpc8xxx_wdt.c doesn't build modular
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11360
Subject : mpc8xxx_wdt.c doesn't build modular
Submitter : Dave Jones <[email protected]>
Date : 2008-08-17 08:07 (7 days old)
References : http://lkml.org/lkml/2008/8/12/465
[Bug #11401] pktcdvd: BUG, NULL pointer dereference in pkt_ioctl, bisected
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11401
Subject : pktcdvd: BUG, NULL pointer dereference in pkt_ioctl, bisected
Submitter : Laurent Riffard <[email protected]>
Date : 2008-08-22 08:16 (2 days old)
[Bug #11388] 2.6.27-rc3 warns about MTRR range; only 3 of 16gb of memory is usable
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11388
Subject : 2.6.27-rc3 warns about MTRR range; only 3 of 16gb of memory is usable
Submitter : Joshua Hoblitt <[email protected]>
Date : 2008-08-20 17:38 (4 days old)
[Bug #11382] e1000e: 2.6.27-rc1 corrupts EEPROM/NVM
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11382
Subject : e1000e: 2.6.27-rc1 corrupts EEPROM/NVM
Submitter : David Vrabel <[email protected]>
Date : 2008-08-08 10:47 (16 days old)
References : http://marc.info/?l=linux-kernel&m=121819267211679&w=4
[Bug #11398] hda_intel: IRQ timing workaround is activated for card #0. Suggest a bigger bdl_pos_adj.
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11398
Subject : hda_intel: IRQ timing workaround is activated for card #0. Suggest a bigger bdl_pos_adj.
Submitter : Frans Pop <elendil-EIBgga6/0yRmR6Xm/[email protected]>
Date : 2008-08-21 17:17 (3 days old)
[Bug #11402] skbuff bug?
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11402
Subject : skbuff bug?
Submitter : Yinghai Lu <[email protected]>
Date : 2008-08-21 3:56 (3 days old)
References : http://marc.info/?l=linux-kernel&m=121929102707658&w=4
[Bug #11403] 2.6.27-rc2 USB suspend regression
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11403
Subject : 2.6.27-rc2 USB suspend regression
Submitter : Jeremy Fitzhardinge <[email protected]>
Date : 2008-08-20 20:48 (4 days old)
References : http://marc.info/?l=linux-kernel&m=121926536103630&w=4
[Bug #11404] BUG: in 2.6.23-rc3-git7 in do_cciss_intr
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11404
Subject : BUG: in 2.6.23-rc3-git7 in do_cciss_intr
Submitter : rdunlap <[email protected]>
Date : 2008-08-21 5:52 (3 days old)
References : http://marc.info/?l=linux-kernel&m=121929819616273&w=4
http://marc.info/?l=linux-kernel&m=121932889105368&w=4
James Bottomley <James.Bottomley-JuX6DAaQMKPCXq6kfMZ53/[email protected]>
[Bug #11405] 2.6.27-rc3 segfault on cold boot; not on warm boot.
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11405
Subject : 2.6.27-rc3 segfault on cold boot; not on warm boot.
Submitter : David Greaves <david-FQ/kcb21CSxWk0Htik3J/[email protected]>
Date : 2008-08-21 9:45 (3 days old)
References : http://marc.info/?l=linux-kernel&m=121931198904777&w=4
[Bug #11406] patch "x86: MOVE PCI IO ECS code to x86/pci" breaks CPU hotplug
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11406
Subject : patch "x86: MOVE PCI IO ECS code to x86/pci" breaks CPU hotplug
Submitter : Jan Beulich <[email protected]>
Date : 2008-08-21 12:59 (3 days old)
References : http://marc.info/?l=linux-kernel&m=121932366326572&w=4
Robert Richter <[email protected]>
[Bug #11409] build issue #564 for v2.6.27-rc4 : undefined reference to `NS8390p_init'
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11409
Subject : build issue #564 for v2.6.27-rc4 : undefined reference to `NS8390p_init'
Submitter : Toralf Förster <[email protected]>
Date : 2008-08-22 8:33 (2 days old)
References : http://marc.info/?l=linux-kernel&m=121939410214677&w=4
Patch : http://marc.info/?l=linux-kernel&m=121943097320451&w=4
[Bug #11410] SLUB list_lock vs obj_hash.lock...
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11410
Subject : SLUB list_lock vs obj_hash.lock...
Submitter : Daniel J Blueman <[email protected]>
Date : 2008-08-22 21:48 (2 days old)
References : http://marc.info/?l=linux-kernel&m=121944176609042&w=4
[Bug #11413] get_rtc_time() triggers NMI watchdog in hpet_rtc_interrupt()
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11413
Subject : get_rtc_time() triggers NMI watchdog in hpet_rtc_interrupt()
Submitter : Mikael Pettersson <[email protected]>
Date : 2008-08-23 9:48 (1 days old)
References : http://marc.info/?l=linux-kernel&m=121948503224161&w=4
Patch : http://marc.info/?l=linux-kernel&m=121950734922457&w=4
[Bug #11407] suspend: unable to handle kernel paging request
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11407
Subject : suspend: unable to handle kernel paging request
Submitter : Vegard Nossum <[email protected]>
Date : 2008-08-21 17:28 (3 days old)
References : http://marc.info/?l=linux-kernel&m=121933974928881&w=4
Pekka Enberg <penberg-bbCR+/[email protected]> Pavel Machek <[email protected]>
[Bug #11414] Random crashes with 2.6.27-rc3 on PPC
In reply to Rafael J. WysockiThis message has been generated automatically as a part of a report of recent regressions.
The following bug entry is on the current list of known regressions from 2.6.26. Please verify if it still should be listed and let me know (either way).
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11414
Subject : Random crashes with 2.6.27-rc3 on PPC
Submitter : Michael Buesch <[email protected]>
Date : 2008-08-23 14:10 (1 days old)
References : http://marc.info/?l=linux-kernel&m=121950076812616&w=4
Quoted 6 lines · Rafael J. Wysocki
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11405
Subject : 2.6.27-rc3 segfault on cold boot; not on warm boot.
Submitter : David Greaves <david-FQ/kcb21CSxWk0Htik3J/[email protected]>
Date : 2008-08-21 9:45 (3 days old)
References : http://marc.info/?l=linux-kernel&m=121931198904777&w=4
It would be good to have some kind of bisection of this one, because it looks pretty odd. Also, google doesn't find anybody else seeing that "segfault at ffffffbf", even though it seems to be very consistent for David. So I don't think we'll be able to even _guess_ where it is without some more information about exactly when it started happening.
Since it's present in 2.6.26 too, it's clearly not a regression from that one, but perhaps more importantly, since it's apparently an old one I'd have expected more reports like this if it was some common problem. And the warm-vs-cold-boot thing makes me think it's some hardware setup issue.
Possibly the disk controller, possibly the CPU (eg some MTRR/PAT setup issue or TLB thing). But the dmesg's are all from late enough at boot that I can't even tell what disk controller it is (except that it is SATA), nor can I tell what CPU it is.
But again, if it was some MTRR/PAT issue, I'd expect a _lot_ more reports of this.
MD/XFS sounds unlikely, since they should have absolutely nothing that could possibly matter for cold/hot boot.
Linus
Quoted 19 lines · Linus Torvalds
Quoted 5 lines · Rafael J. Wysocki
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11405
Subject : 2.6.27-rc3 segfault on cold boot; not on warm boot.
Submitter : David Greaves <david-FQ/kcb21CSxWk0Htik3J/[email protected]>
Date : 2008-08-21 9:45 (3 days old)
References : http://marc.info/?l=linux-kernel&m=121931198904777&w=4It would be good to have some kind of bisection of this one, because it looks pretty odd. Also, google doesn't find anybody else seeing that "segfault at ffffffbf", even though it seems to be very consistent for David. So I don't think we'll be able to even _guess_ where it is without some more information about exactly when it started happening.
Since it's present in 2.6.26 too, it's clearly not a regression from that one, but perhaps more importantly, since it's apparently an old one I'd have expected more reports like this if it was some common problem. And the warm-vs-cold-boot thing makes me think it's some hardware setup issue.
Quoted 7 lines
Possibly the disk controller, possibly the CPU (eg some MTRR/PAT setup issue or TLB thing). But the dmesg's are all from late enough at boot that I can't even tell what disk controller it is (except that it is SATA), nor can I tell what CPU it is.
But again, if it was some MTRR/PAT issue, I'd expect a _lot_ more reports of this.
OK, that all makes sense.
Given that I'll manage at best 1 bisect/day with a reasonable chance of data corruption and hardware intermittency screwing it all up I thought it best to ask first in case there was another debug approach that could work. However since it does indeed sounds somewhat hardware related and it's an isolated problem for my wife (as opposed to a problem that others are having too) then I think she deserves a new machine...
Thanks for the impetus to cheer her up ;)
David
PS if anyone really is interested then I am happy to try the bisection once I've moved her to a new box; otherwise I'm happy to close this.
Quoted 4 lines · David Greaves
Given that I'll manage at best 1 bisect/day with a reasonable chance of data corruption and hardware intermittency screwing it all up I thought it best to ask first in case there was another debug approach that could work.
Well, regardless, I think it would be good to fill in the hardware info, especially wrt CPU data and the exact SATA controller you have.
There's another regression for SATA cold/hot boot issues, and while that one looks very different, and is probably not really related, it's still a good idea to try to see if we can match them up. See
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11343 Subject : SATA Cold Boot Problems with 2.6.27-rc[23] on nVidia 680i Submitter : Manny Maxwell <[email protected]> Date : 2008-08-14 4:16 (10 days old) References : http://marc.info/?l=linux-kernel&m=121868782917600&w=4
which actually has a patch, and which seems to work fine in 2.6.26 (so not only is failure pattern different, the point were it starts is different).
But regardless of the big differences, it does seem to point to some weakness in SATA initialization. But is it limited to _that_ particular SATA controller, or just a few ones? Or a generic issue? Without more reports to really find a pattern, I don't think we have a clue, and the two may be _totally_ unrelated in all ways, but it would be good to at least report and log the information you have..
Oh, I just noticed that your dmesg _does_ mention sata_sil and sata_via, so we know which of two drivers it would be, at least. Not the nVidia one.
However, there's been tons of changes in soem core functions: both the reset handling and the wait-for-ready has changed and caused lots of churn across most drivers in between 2.6.25 and 2.6.26.
Quoted 2 lines
PS if anyone really is interested then I am happy to try the bisection once I've moved her to a new box; otherwise I'm happy to close this.
I think it would be good to try to bisect. It could be something that is really just limited to that particular machine (maybe it really is some flaky hardware that just triggers some timing changes), but more likely it isn't. So the more information, the better. So keep the thing open as long as somebody is willing to try to gather more info, by all means.
Linus
Quoted 6 lines · Rafael J. Wysocki
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11410
Subject : SLUBlist_lockvs obj_hash.lock...
Submitter : Daniel J Blueman <[email protected]>
Date : 2008-08-22 21:48 (2 days old)
References : http://marc.info/?l=linux-kernel&m=121944176609042&w=4
This one now has a suggested patch for Daniel to try from Vegard, but no reply yet:
http://marc.info/?l=linux-kernel&m=121946972307110&w=4
Vegard, I think your patch is a bit odd, though. The result of your patch is
- first loop:
hlist_for_each_entry_safe(obj, node, tmp, &db->list, node) {
hlist_del(&obj->node);
hlist_add_head(&obj->node, &freelist);
}and quite frankly, I don't see what the difference between that and a something like a simple
struct hlist_node *first = bd->list.first;
if (first) {
bd->list.first = NULL;
first->pprev = &first;
}really is?
I dunno. We don't have list splicing ops for the hlist things.
Linus
Quoted 10 lines · Linus Torvalds
Quoted 6 lines · Rafael J. Wysocki
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11410
Subject : SLUBlist_lockvs obj_hash.lock...
Submitter : Daniel J Blueman <[email protected]>
Date : 2008-08-22 21:48 (2 days old)
References : http://marc.info/?l=linux-kernel&m=121944176609042&w=4
Quoted 5 lines
This one now has a suggested patch for Daniel to try from Vegard, but no reply yet:
http://marc.info/?l=linux-kernel&m=121946972307110&w=4
Hi!
Quoted 19 lines
Vegard, I think your patch is a bit odd, though. The result of your patch is
- first loop:
hlist_for_each_entry_safe(obj, node, tmp, &db->list, node) { hlist_del(&obj->node); hlist_add_head(&obj->node, &freelist); }and quite frankly, I don't see what the difference between that and a something like a simple
struct hlist_node *first = bd->list.first; if (first) { bd->list.first = NULL; first->pprev = &first; }
Quoted 3 lines
really is?
I dunno. We don't have list splicing ops for the hlist things.
Hm.
I haven't really used the hlists before, so my first instinct was to do what is obvious. That's also why I put the XXX comment. Other than that, I guess open-coding list ops is also not very good programming practice? :-)
But... feel free to submit your own patch. Oh, what am I saying.
Vegard
Signature · 4 lines
"The animistic metaphor of the bug that maliciously sneaked in while the programmer was not looking is intellectually dishonest as it disguises that the error is the programmer's own creation." -- E. W. Dijkstra, EWD1036
Quoted 3 lines · Vegard Nossum
I haven't really used the hlists before, so my first instinct was to do what is obvious.
I do agree that the hlist versions aren't very nice in this regard. The regular lists are much better at moving lists around.
Quoted 2 lines
Other than that, I guess open-coding list ops is also not very good programming practice? :-)
Agreed. It would be better if the people who use hlists most (I think that would be networking) would think about this.
Quoted 1 line
But... feel free to submit your own patch. Oh, what am I saying.
Silly boy. Next you'll ask me to _test_ any patches I send out.
Anyway, I think your patch is likely fine, I just thought it looked a bit odd to have a loop to move a list from one head pointer to another.
But regardless, it would need some testing. Daniel?
Linus
Hi Linus, Vegard,
Quoted 1 line · Linus Torvalds
On Sun, 24 Aug 2008, Vegard Nossum wrote:
[snip]
Quoted 4 lines
Anyway, I think your patch is likely fine, I just thought it looked a bit odd to have a loop to move a list from one head pointer to another.
But regardless, it would need some testing. Daniel?
This opens another lockdep report at boot-time [1] - promoting pool_lock may not be the best fix?
We then see a new deadlock condition (on the pool_lock spinlock) [2], which seemingly was avoided by taking the debug-bucket lock first.
We reproduce this by booting with debug_objects=1 and causing a lot of activity.
Daniel
--- [1]
[ INFO: possible irq lock inversion dependency detected ] 2.6.27-rc4-225c-debug #3
---------------------------------------------------------
rcu_sched_grace/9 just changed the state of lock:
(pool_lock){-...}, at: [<ffffffff80466c2c>] free_object+0x7c/0xc0
but this lock was taken by another, hard-irq-safe lock in the past:
(xtime_lock){++..}
and interrupts could create inverse lock ordering between them.
other info that might help us debug this:
no locks held by rcu_sched_grace/9.
the first lock's dependencies:
-> (pool_lock){-...} ops: 59 {
initial-use at:
[<ffffffff8026f795>]
__lock_acquire+0x1a5/0x1160
[<ffffffff802707e1>]
lock_acquire+0x91/0xc0
[<ffffffff806a2bb1>] _spin_lock+0x41/0x80
[<ffffffff804675fa>]
__debug_object_init+0x14a/0x3e0
[<ffffffff804678df>]
debug_object_init+0x1f/0x30
[<ffffffff80260eee>]
hrtimer_init+0x2e/0x50
[<ffffffff80237571>]
init_rt_bandwidth+0x41/0x60
[<ffffffff812f3810>]
sched_init+0x72/0x63d
[<ffffffff812e17e2>]
start_kernel+0x19c/0x456
[<ffffffff812e10a9>]
x86_64_start_reservations+0x99/0xb6
[<ffffffff812e11d0>]
x86_64_start_kernel+0xe2/0xe9
[<ffffffffffffffff>] 0xffffffffffffffff
hardirq-on-W at:
[<ffffffff8026fae9>]
__lock_acquire+0x4f9/0x1160
[<ffffffff802707e1>]
lock_acquire+0x91/0xc0
[<ffffffff806a2bb1>] _spin_lock+0x41/0x80
[<ffffffff80466c2c>]
free_object+0x7c/0xc0
[<ffffffff8046707e>]
debug_object_free+0xbe/0x130
[<ffffffff806a070e>]
schedule_timeout+0x7e/0xe0
[<ffffffff806a07ce>]
schedule_timeout_interruptible+0x1e/0x20
[<ffffffff8029cfb2>]
rcu_sched_grace_period+0xa2/0x3a0
[<ffffffff8025d50e>] kthread+0x4e/0x90
[<ffffffff8020d919>] child_rip+0xa/0x11
[<ffffffffffffffff>] 0xffffffffffffffff
}
... key at: [<ffffffff8088f5d8>] pool_lock+0x18/0x40the second lock's dependencies:
-> (xtime_lock){++..} ops: 211 {
initial-use at:
[<ffffffff8026f795>]
__lock_acquire+0x1a5/0x1160
[<ffffffff802707e1>]
lock_acquire+0x91/0xc0
[<ffffffff806a2bb1>] _spin_lock+0x41/0x80
[<ffffffff812f5628>]
timekeeping_init+0x2f/0x144
[<ffffffff812e189d>]
start_kernel+0x257/0x456
[<ffffffff812e10a9>]
x86_64_start_reservations+0x99/0xb6
[<ffffffff812e11d0>]
x86_64_start_kernel+0xe2/0xe9
[<ffffffffffffffff>] 0xffffffffffffffff
in-hardirq-W at:
[<ffffffffffffffff>] 0xffffffffffffffff
in-softirq-W at:
[<ffffffffffffffff>] 0xffffffffffffffff
}
... key at: [<ffffffff80907220>] xtime_lock+0x20/0x40
-> (&obj_hash[i].lock){.+..} ops: 1003901 {
initial-use at:
[<ffffffff8026f795>]
__lock_acquire+0x1a5/0x1160
[<ffffffff802707e1>]
lock_acquire+0x91/0xc0
[<ffffffff806a2d43>]
_spin_lock_irqsave+0x53/0x90
[<ffffffff80467567>]
__debug_object_init+0xb7/0x3e0
[<ffffffff804678df>]
debug_object_init+0x1f/0x30
[<ffffffff80260eee>]
hrtimer_init+0x2e/0x50
[<ffffffff80237571>]
init_rt_bandwidth+0x41/0x60
[<ffffffff812f3810>]
sched_init+0x72/0x63d
[<ffffffff812e17e2>]
start_kernel+0x19c/0x456
[<ffffffff812e10a9>]
x86_64_start_reservations+0x99/0xb6
[<ffffffff812e11d0>]
x86_64_start_kernel+0xe2/0xe9
[<ffffffffffffffff>] 0xffffffffffffffff
in-softirq-W at:
[<ffffffffffffffff>] 0xffffffffffffffff
}
... key at: [<ffffffff81cd2eb0>] __key.16550+0x0/0x8
-> (pool_lock){-...} ops: 59 {
initial-use at:
[<ffffffff8026f795>]
__lock_acquire+0x1a5/0x1160
[<ffffffff802707e1>]
lock_acquire+0x91/0xc0
[<ffffffff806a2bb1>]
_spin_lock+0x41/0x80
[<ffffffff804675fa>]
__debug_object_init+0x14a/0x3e0
[<ffffffff804678df>]
debug_object_init+0x1f/0x30
[<ffffffff80260eee>]
hrtimer_init+0x2e/0x50
[<ffffffff80237571>]
init_rt_bandwidth+0x41/0x60
[<ffffffff812f3810>]
sched_init+0x72/0x63d
[<ffffffff812e17e2>]
start_kernel+0x19c/0x456
[<ffffffff812e10a9>]
x86_64_start_reservations+0x99/0xb6
[<ffffffff812e11d0>]
x86_64_start_kernel+0xe2/0xe9
[<ffffffffffffffff>]
0xffffffffffffffff
hardirq-on-W at:
[<ffffffff8026fae9>]
__lock_acquire+0x4f9/0x1160
[<ffffffff802707e1>]
lock_acquire+0x91/0xc0
[<ffffffff806a2bb1>]
_spin_lock+0x41/0x80
[<ffffffff80466c2c>]
free_object+0x7c/0xc0
[<ffffffff8046707e>]
debug_object_free+0xbe/0x130
[<ffffffff806a070e>]
schedule_timeout+0x7e/0xe0
[<ffffffff806a07ce>]
schedule_timeout_interruptible+0x1e/0x20
[<ffffffff8029cfb2>]
rcu_sched_grace_period+0xa2/0x3a0
[<ffffffff8025d50e>]
kthread+0x4e/0x90
[<ffffffff8020d919>]
child_rip+0xa/0x11
[<ffffffffffffffff>]0xffffffffffffffff
}
... key at: [<ffffffff8088f5d8>] pool_lock+0x18/0x40
... acquired at:
[<ffffffff802703b1>] __lock_acquire+0xdc1/0x1160 [<ffffffff802707e1>] lock_acquire+0x91/0xc0 [<ffffffff806a2bb1>] _spin_lock+0x41/0x80 [<ffffffff804675fa>] __debug_object_init+0x14a/0x3e0 [<ffffffff804678df>] debug_object_init+0x1f/0x30 [<ffffffff80260eee>] hrtimer_init+0x2e/0x50 [<ffffffff80237571>] init_rt_bandwidth+0x41/0x60 [<ffffffff812f3810>] sched_init+0x72/0x63d [<ffffffff812e17e2>] start_kernel+0x19c/0x456 [<ffffffff812e10a9>] x86_64_start_reservations+0x99/0xb6 [<ffffffff812e11d0>] x86_64_start_kernel+0xe2/0xe9 [<ffffffffffffffff>] 0xffffffffffffffff
... acquired at:
[<ffffffff802703b1>] __lock_acquire+0xdc1/0x1160 [<ffffffff802707e1>] lock_acquire+0x91/0xc0 [<ffffffff806a2d43>] _spin_lock_irqsave+0x53/0x90 [<ffffffff80467567>] __debug_object_init+0xb7/0x3e0 [<ffffffff804678df>] debug_object_init+0x1f/0x30 [<ffffffff80260eee>] hrtimer_init+0x2e/0x50 [<ffffffff812f577b>] ntp_init+0x1e/0x2b [<ffffffff812f5633>] timekeeping_init+0x3a/0x144 [<ffffffff812e189d>] start_kernel+0x257/0x456 [<ffffffff812e10a9>] x86_64_start_reservations+0x99/0xb6 [<ffffffff812e11d0>] x86_64_start_kernel+0xe2/0xe9 [<ffffffffffffffff>] 0xffffffffffffffff
-> (clocksource_lock){++..} ops: 214 {
initial-use at:
[<ffffffff8026f795>]
__lock_acquire+0x1a5/0x1160
[<ffffffff802707e1>]
lock_acquire+0x91/0xc0
[<ffffffff806a2d43>]
_spin_lock_irqsave+0x53/0x90
[<ffffffff80266245>]
clocksource_get_next+0x15/0x60
[<ffffffff812f5638>]
timekeeping_init+0x3f/0x144
[<ffffffff812e189d>]
start_kernel+0x257/0x456
[<ffffffff812e10a9>]
x86_64_start_reservations+0x99/0xb6
[<ffffffff812e11d0>]
x86_64_start_kernel+0xe2/0xe9
[<ffffffffffffffff>] 0xffffffffffffffff
in-hardirq-W at:
[<ffffffffffffffff>] 0xffffffffffffffff
in-softirq-W at:
[<ffffffffffffffff>] 0xffffffffffffffff
}
... key at: [<ffffffff80877bb8>] clocksource_lock+0x18/0x40
... acquired at:
[<ffffffff802703b1>] __lock_acquire+0xdc1/0x1160
[<ffffffff802707e1>] lock_acquire+0x91/0xc0
[<ffffffff806a2d43>] _spin_lock_irqsave+0x53/0x90
[<ffffffff80266245>] clocksource_get_next+0x15/0x60
[<ffffffff812f5638>] timekeeping_init+0x3f/0x144
[<ffffffff812e189d>] start_kernel+0x257/0x456
[<ffffffff812e10a9>] x86_64_start_reservations+0x99/0xb6
[<ffffffff812e11d0>] x86_64_start_kernel+0xe2/0xe9
[<ffffffffffffffff>] 0xffffffffffffffff -> (old_style_seqlock_init){++..} ops: 210 {
initial-use at:
[<ffffffffffffffff>] 0xffffffffffffffff
in-hardirq-W at:
[<ffffffffffffffff>] 0xffffffffffffffff
in-softirq-W at:
[<ffffffffffffffff>] 0xffffffffffffffff }
... key at: [<ffffffff812d61a0>] nl80211_policy+0xda0/0x2c00
... acquired at:
[<ffffffffffffffff>] 0xffffffffffffffff
-> (ftrace_shutdown_lock){++..} ops: 480 {
initial-use at:
[<ffffffff8026f795>]
__lock_acquire+0x1a5/0x1160
[<ffffffff802707e1>]
lock_acquire+0x91/0xc0
[<ffffffff806a2d43>]
_spin_lock_irqsave+0x53/0x90
[<ffffffff802a4216>]
ftrace_record_ip+0x196/0x2f0
[<ffffffff8020c6b4>]
mcount_call+0x5/0x31
[<ffffffff812e1c9b>]
kernel_init+0x14d/0x1b2
[<ffffffff8020d919>] child_rip+0xa/0x11
[<ffffffffffffffff>] 0xffffffffffffffff
in-hardirq-W at:
[<ffffffffffffffff>] 0xffffffffffffffff
in-softirq-W at:
[<ffffffffffffffff>] 0xffffffffffffffff
}
... key at: [<ffffffff8087c1b8>] ftrace_shutdown_lock+0x18/0x40
... acquired at:
[<ffffffffffffffff>] 0xffffffffffffffff
stack backtrace:Pid: 9, comm: rcu_sched_grace Not tainted 2.6.27-rc4-225c-debug #3
Call Trace: [<ffffffff8026d8b2>] print_irq_inversion_bug+0x142/0x160 [<ffffffff8026dd27>] check_usage_backwards+0x67/0xb0 [<ffffffff8026ebd3>] mark_lock+0x363/0x7f0 [<ffffffff8026fae9>] __lock_acquire+0x4f9/0x1160 [<ffffffff802707e1>] lock_acquire+0x91/0xc0 [<ffffffff80466c2c>] ? free_object+0x7c/0xc0 [<ffffffff806a2bb1>] _spin_lock+0x41/0x80 [<ffffffff80466c2c>] ? free_object+0x7c/0xc0 [<ffffffff806a6bc9>] ? sub_preempt_count+0x69/0xd0 [<ffffffff80466c2c>] free_object+0x7c/0xc0 [<ffffffff8046707e>] debug_object_free+0xbe/0x130 [<ffffffff806a070e>] schedule_timeout+0x7e/0xe0 [<ffffffff802503a0>] ? process_timeout+0x0/0x10 [<ffffffff806a06f2>] ? schedule_timeout+0x62/0xe0 [<ffffffff8029cf10>] ? rcu_sched_grace_period+0x0/0x3a0 [<ffffffff806a07ce>] schedule_timeout_interruptible+0x1e/0x20 [<ffffffff8029cfb2>] rcu_sched_grace_period+0xa2/0x3a0 [<ffffffff8029cf10>] ? rcu_sched_grace_period+0x0/0x3a0 [<ffffffff8025d50e>] kthread+0x4e/0x90 [<ffffffff8020d919>] child_rip+0xa/0x11 [<ffffffff80239b8f>] ? finish_task_switch+0x5f/0x120 [<ffffffff806a356b>] ? _spin_unlock_irq+0x3b/0x70 [<ffffffff8020cf23>] ? restore_args+0x0/0x30 [<ffffffff8025d4c0>] ? kthread+0x0/0x90 [<ffffffff8020d90f>] ? child_rip+0x0/0x11
--- [2]
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa 58 0 63972 17316 16 145172 128 21176 128 21176 6054 8662 79 21 0 0 47 2 81020 14380 16 139472 1120 17052 1240 17052 6106 9532 77 23 0 0 52 1 86276 33656 16 137140 604 5304 796 5304 5349 7954 81 19 0 0 94 0 86276 32192 16 137484 480 0 772 0 5418 7618 84 16 0 0 88 0 86264 22416 16 137800 96 0 396 0 4746 5937 87 13 0 0 63 1 86200 54636 16 137932 1380 0 1408 0 5472 8007 82 18 0 0 47 0 86020 22848 16 138132 256 0 312 0 6126 12227 72 28 0 0 75 2 103828 20252 16 135500 528 17836 592 17836 6655 12862 69 31 0 0 21 0 128568 17536 16 128888 2336 24732 2336 24732 6762 12891 66 34 0 0 159 0 154996 16888 16 124808 480 26236 504 26236 5930 7689 80 20 0 0 45 0 165616 40108 16 120544 192 10696 248 10696 6136 9163 77 23 0 0 95 0 165616 27296 16 120632 924 0 940 0 5293 7468 82 18 0 0 BUG: NMI Watchdog detected LOCKUP on CPU0, ip ffffffff80214407, registers: CPU 0 Modules linked in: rfcomm l2cap bluetooth kvm_intel kvm microcode dvb_usb_dtt200u dvb_usb uvcvideo dvb_core compat_ioctl32 i2c_core videodev v4l1_compat shpchp pcig Pid: 6948, comm: spiral Not tainted 2.6.27-rc4-225c-debug #3 RIP: 0010:[<ffffffff80214407>] [<ffffffff80214407>] native_read_tsc+0x7/0x30 RSP: 0018:ffffffff8153ab70 EFLAGS: 00000002 RAX: 00000467c85cb001 RBX: ffffffff8088f5c0 RCX: 00000000c85cb001 RDX: 00000000c85cb001 RSI: 0000000000000103 RDI: 0000000000000001 RBP: ffffffff8153ab70 R08: 0000000000000000 R09: 0000000000000001 R10: 0000000000000000 R11: ffff8800cf978000 R12: 00000000c85cb001 R13: 0000000000000001 R14: 0000000000000000 R15: ffff88012a61c8c0 FS: 00007fb48cb796e0(0000) GS:ffffffff80e82dc0(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00000000075212e6 CR3: 00000000cc1e6000 CR4: 00000000000026e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff4ff0 DR7: 0000000000000400 Process spiral (pid: 6948, threadinfo ffff8800cf950000, task ffff8800cf978000) Stack: ffffffff8153aba0 ffffffff804560c7 ffffffff8088f5c0 0000000005b61b94 00000000d6915628 0000000000000001 ffffffff8153abb0 ffffffff80455fff ffffffff8153abe0 ffffffff80466302 ffffffff8088f5d8 ffffffff8088f5c0 Call Trace: <IRQ> [<ffffffff804560c7>] delay_tsc+0x67/0xd0 [<ffffffff80455fff>] __delay+0xf/0x20 [<ffffffff80466302>] _raw_spin_lock+0x122/0x170 [<ffffffff806a2bd1>] _spin_lock+0x61/0x80 [<ffffffff80466c2c>] ? free_object+0x7c/0xc0 [<ffffffff80466c2c>] free_object+0x7c/0xc0 [<ffffffff80466e0f>] __debug_check_no_obj_freed+0x19f/0x1e0 [<ffffffff80466e65>] debug_check_no_obj_freed+0x15/0x20 [<ffffffff802dff0c>] kmem_cache_free+0xec/0x110 [<ffffffff80517fa1>] ? scsi_pool_free_command+0x51/0x60 [<ffffffff80517fa1>] scsi_pool_free_command+0x51/0x60 [<ffffffff805184bf>] __scsi_put_command+0x5f/0xa0 [<ffffffff80518561>] scsi_put_command+0x61/0x70 [<ffffffff8051e47a>] scsi_next_command+0x3a/0x60 [<ffffffff8051e544>] scsi_end_request+0xa4/0xc0 [<ffffffff8051e68f>] scsi_io_completion+0x12f/0x440 [<ffffffff80517bf5>] scsi_finish_command+0x95/0xd0 [<ffffffff8051eb36>] scsi_softirq_done+0x86/0x110 [<ffffffff8043bbed>] blk_done_softirq+0x8d/0xa0 [<ffffffff8024ba04>] __do_softirq+0x74/0xf0 [<ffffffff8020dc7c>] call_softirq+0x1c/0x30 [<ffffffff8020f485>] do_softirq+0x75/0xb0 [<ffffffff8024b155>] irq_exit+0xa5/0xb0 [<ffffffff8020f7d3>] do_IRQ+0xe3/0x1d0 [<ffffffff80466c66>] ? free_object+0xb6/0xc0 [<ffffffff8020ce76>] ret_from_intr+0x0/0xf <EOI> [<ffffffff80270ba0>] ? lock_release+0xe0/0x210 [<ffffffff806a3253>] ? _spin_unlock+0x23/0x60 [<ffffffff80466c66>] ? free_object+0xb6/0xc0 [<ffffffff8046707e>] ? debug_object_free+0xbe/0x130 [<ffffffff806a070e>] ? schedule_timeout+0x7e/0xe0 [<ffffffff802503a0>] ? process_timeout+0x0/0x10 [<ffffffff806a06f2>] ? schedule_timeout+0x62/0xe0 [<ffffffff802f8b3e>] ? do_select+0x4be/0x610 [<ffffffff802f8c90>] ? __pollwait+0x0/0x120 [<ffffffff8026f1d9>] ? trace_hardirqs_on_caller+0x29/0x1b0 [<ffffffff8026f1d9>] ? trace_hardirqs_on_caller+0x29/0x1b0 [<ffffffff8026f36d>] ? trace_hardirqs_on+0xd/0x10 [<ffffffff806a0b9e>] ? mutex_unlock+0xe/0x10 [<ffffffff8065919e>] ? unix_stream_recvmsg+0x32e/0x6d0 [<ffffffff8026f1d9>] ? trace_hardirqs_on_caller+0x29/0x1b0 [<ffffffff8026f36d>] ? trace_hardirqs_on+0xd/0x10 [<ffffffff802f8f4b>] ? core_sys_select+0x19b/0x2e0 [<ffffffff802e89e9>] ? do_sync_read+0xf9/0x140 [<ffffffff8025d8e0>] ? autoremove_wake_function+0x0/0x40 [<ffffffff806a6bc9>] ? sub_preempt_count+0x69/0xd0 [<ffffffff802f94b0>] ? sys_select+0xd0/0x1c0 [<ffffffff8020c86b>] ? system_call_fastpath+0x16/0x1b Code: 90 90 90 90 55 89 f8 48 89 e5 e6 70 e4 71 c9 c3 0f 1f 40 00 55 89 f0 48 89 e5 e6 70 89 f8 e6 71 c9 c3 66 90 55 48 89 e5 0f 1f 00 <0f> ae e8 0f 31 89 c1 0f 1f BUG: NMI Watchdog detected LOCKUP<4>---[ end trace fd851c3db62e5044 ]--- Kernel panic - not syncing: Aiee, killing interrupt handler! on CPU1, ip ffffffff80214407, registers: CPU 1 Modules linked in: rfcomm l2cap bluetooth kvm_intel kvm microcode dvb_usb_dtt200u dvb_usb uvcvideo dvb_core compat_ioctl32 i2c_core videodev v4l1_compat shpchp pcig Pid: 10150, comm: gcc Not tainted 2.6.27-rc4-225c-debug #3 RIP: 0010:[<ffffffff80214407>] [<ffffffff80214407>] native_read_tsc+0x7/0x30 RSP: 0018:ffff8800b6269c28 EFLAGS: 00000092 RAX: 0000000000000001 RBX: ffffffff8088f5c0 RCX: 00000000c85cafc2 RDX: 000000008b468b00 RSI: 0000000000000002 RDI: 0000000000000001 RBP: ffff8800b6269c28 R08: 0000000000000000 R09: 0000000000000001 R10: 0000000000000000 R11: ffff8800a65047e0 R12: 0000000005d22695 R13: 0000000000000001 R14: 0000000000000001 R15: ffff88009e9be940 FS: 00002b124e4fc6e0(0000) GS:ffff88012fa644b0(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 00002b8a265f5960 CR3: 00000000b61e9000 CR4: 00000000000026e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process gcc (pid: 10150, threadinfo ffff8800b6268000, task ffff8800a65047e0) Stack: ffff8800b6269c58 ffffffff8045608a ffffffff8088f5c0 0000000005d22695 00000000d6915628 0000000000000001 ffff8800b6269c68 ffffffff80455fff ffff8800b6269c98 ffffffff80466302 ffffffff8088f5d8 ffffffff8088f5c0 Call Trace: [<ffffffff8045608a>] delay_tsc+0x2a/0xd0 [<ffffffff80455fff>] __delay+0xf/0x20 [<ffffffff80466302>] _raw_spin_lock+0x122/0x170 [<ffffffff806a2bd1>] _spin_lock+0x61/0x80 [<ffffffff80466c2c>] ? free_object+0x7c/0xc0 [<ffffffff80466c2c>] free_object+0x7c/0xc0 [<ffffffff80466e0f>] __debug_check_no_obj_freed+0x19f/0x1e0 [<ffffffff80466e65>] debug_check_no_obj_freed+0x15/0x20 [<ffffffff802dff0c>] kmem_cache_free+0xec/0x110 [<ffffffff8024264b>] ? __cleanup_signal+0x1b/0x20 [<ffffffff8024264b>] __cleanup_signal+0x1b/0x20 [<ffffffff80248273>] release_task+0x233/0x3d0 [<ffffffff80248960>] wait_consider_task+0x550/0x8b0 [<ffffffff80248e16>] do_wait+0x156/0x350 [<ffffffff8023b6f0>] ? default_wake_function+0x0/0x10 [<ffffffff802490a6>] sys_wait4+0x96/0xf0 [<ffffffff8020c86b>] system_call_fastpath+0x16/0x1b Code: 90 90 90 90 55 89 f8 48 89 e5 e6 70 e4 71 c9 c3 0f 1f 40 00 55 89 f0 48 89 e5 e6 70 89 f8 e6 71 c9 c3 66 90 55 48 89 e5 0f 1f 00 <0f> ae e8 0f 31 89 c1 0f 1f ---[ end trace fd851c3db62e5044 ]---
Signature · 1 line
Daniel J Blueman
Quoted 5 lines · Rafael J. Wysocki
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11401
Subject : pktcdvd: BUG, NULL pointer dereference inpkt_ioctl, bisected
Submitter : Laurent Riffard <[email protected]>
Date : 2008-08-22 08:16 (2 days old)
This one looks irritating.
It's bisected to 5b6155ee70e9c4d2ad7e6f514c8eee06e2711c3a ("pktcdvd: push BKL down into driver"), but the problem goes deeper than that.
The "unlocked" ioctl's do not get a "struct inode *" pointer, they _only_ get the "struct file *". And this is very much historical usage, where some internal functions only passed in the inode (good or not, whatever).
And ioctl_by_bdev() doesn't have a "struct file *" and has depended on passing in a NUMM "struct file *" and its own "struct inode *", and expects the ioctl's to just use that instead. But the unlocked ioctl just drops it on the floor, and uses just the (unusable) file pointer.
Grr.
And some other cases (like pkt_ioctl() itself) that simply pass in a _different_ inode than the file itself is attached to. It does
blkdev_ioctl(pd->bdev->bd_inode, file, cmd, arg);
where "file" points to the pkt_ioctl thing, but "inode" points to the inode "behind" the pkt interface.
Double grr.
I really think the only sane model is to literally make "unlocked_ioctl()" have the same calling convention as the old "ioctl()" thing had, and pass in both file * and inode *. It was a stupid "cleanup" to try to have a simpler interface for the unlocked version. Having two different models, where we have actually _depended_ on the old model and then are trying to convert to a (weaker) new model, is not a good idea.
The alternative is to do this _only_ for the blkdev_ioctl's, and have those only take the "inode *", and then create a new fake "struct file *" to go with it, regardless of what "struct file" was passed in (exactly because the blockdev ones really think that the inode is the important part).
Hmm?
We need to fix this.
Linus
Quoted 11 lines · Linus Torvalds
Quoted 5 lines · Rafael J. Wysocki
Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=11401
Subject : pktcdvd: BUG, NULL pointer dereference inpkt_ioctl, bisected
Submitter : Laurent Riffard <[email protected]>
Date : 2008-08-22 08:16 (2 days old)This one looks irritating.
It's bisected to
5b6155ee70e9c4d2ad7e6f514c8eee06e2711c3a("pktcdvd: push BKL down into driver"), but the problem goes deeper than that.
...
Quoted 1 line
Grr.
...
Quoted 1 line
Double grr.
...
Quoted 3 lines
Hmm?
We need to fix this.
Why not just revert the offending change and try again during the next merge window, assuming someone has figured out an acceptable way to handle this mess by then?
Signature · 2 lines
Peter Osterlund - [email protected] http://web.telia.com/~u89404340
Quoted 4 lines · Peter Osterlund
Why not just revert the offending change and try again during the next merge window, assuming someone has figured out an acceptable way to handle this mess by then?
Well,, for 2.6.27 that's what we'll have to do. But there's actually a real problem here - the unlocked ioctl's (which we _should_ prefer) have a strictly weaker and worse interface. I also wonder if any other block_ioctl users were converted..
Anyway, I'll take your email as an ack for the revert.
Linus
Quoted 2 lines · Linus Torvalds
I also wonder if any other
block_ioctlusers were converted..
Well, doing
git log -p v2.6.26.. -Sunlocked_ioctl
and looking for blkdev_ioctl, that does seem to be the only one. So hopefully no other case like this is lurking, although it is possible that non-block areas have similar issues.
Linus
Quoted 4 lines · Linus Torvalds
Quoted 1 line · Peter Osterlund
Quoted 8 lines
Quoted 3 lines
Why not just revert the offending change and try again during the next merge window, assuming someone has figured out an acceptable way to handle this mess by then?
Well,, for 2.6.27 that's what we'll have to do. But there's actually a real problem here - the unlocked ioctl's (which we _should_ prefer) have a strictly weaker and worse interface. I also wonder if any other
block_ioctlusers were converted..
Actually both interfaces are a fscking disaster. The right things to pass is neither and inode nor a file but a struct block_device. Al had all this work done a while and it just needs rebasing to a current tree:
http://git.kernel.org/?p=linux/kernel/git/viro/bdev.git;a=summary
Quoted 14 lines · Christoph Hellwig
Quoted 4 lines · Linus Torvalds
Quoted 1 line · Peter Osterlund
Quoted 8 lines
Quoted 3 lines
Why not just revert the offending change and try again during the next merge window, assuming someone has figured out an acceptable way to handle this mess by then?
Well,, for 2.6.27 that's what we'll have to do. But there's actually a real problem here - the unlocked ioctl's (which we _should_ prefer) have a strictly weaker and worse interface. I also wonder if any other
block_ioctlusers were converted..
Quoted 5 lines
Actually both interfaces are a fscking disaster. The right things to pass is neither and inode nor a file but a struct
block_device. Al had all this work done a while and it just needs rebasing to a current tree:http://git.kernel.org/?p=linux/kernel/git/viro/bdev.git;a=summary
Completely agreed. Al, I remember talking to you about this at the storage summit back in february. What are your current plans wrt moving this forward?
Signature · 1 line
Jens Axboe
Al Viro
maintainerQuoted 9 lines · Jens Axboe
Quoted 5 lines
Actually both interfaces are a fscking disaster. The right things to pass is neither and inode nor a file but a struct
block_device. Al had all this work done a while and it just needs rebasing to a current tree:http://git.kernel.org/?p=linux/kernel/git/viro/bdev.git;a=summary
Completely agreed. Al, I remember talking to you about this at the storage summit back in february. What are your current plans wrt moving this forward?
Rebased, with nfs parts of fmode_t patch taken out (irrelevant for bdev anyway and really better off in intent-killing queue). Other than that, it's a straight port... Same place, same branch.
Quoted 7 lines · Al Viro
Quoted 6 lines · Jens Axboe
Quoted 5 lines
Actually both interfaces are a fscking disaster. The right things to pass is neither and inode nor a file but a struct
block_device. Al had all this work done a while and it just needs rebasing to a current tree:http://git.kernel.org/?p=linux/kernel/git/viro/bdev.git;a=summary
Quoted 7 lines
Quoted 3 lines
Completely agreed. Al, I remember talking to you about this at the storage summit back in february. What are your current plans wrt moving this forward?
Rebased, with nfs parts of
fmode_tpatch taken out (irrelevant for bdev anyway and really better off in intent-killing queue). Other than that, it's a straight port... Same place, same branch.
So what's your plan with this - 2.6.28?
Signature · 1 line
Jens Axboe
Al Viro
maintainerQuoted 12 lines · Jens Axboe
Quoted 11 lines · Al Viro
Quoted 9 lines · Jens Axboe
Quoted 5 lines
Actually both interfaces are a fscking disaster. The right things to pass is neither and inode nor a file but a struct
block_device. Al had all this work done a while and it just needs rebasing to a current tree:http://git.kernel.org/?p=linux/kernel/git/viro/bdev.git;a=summary
Completely agreed. Al, I remember talking to you about this at the storage summit back in february. What are your current plans wrt moving this forward?
Quoted 5 lines
Quoted 3 lines
Rebased, with nfs parts of
fmode_tpatch taken out (irrelevant for bdev anyway and really better off in intent-killing queue). Other than that, it's a straight port... Same place, same branch.So what's your plan with this - 2.6.28?
Yes. The only nastiness is around drivers/ide - there it gets a bunch of annoying conflicts from the ide-{disk,floppy}_ioctl.c splitoff. Other than that, it's trivially ported on top of current linux-next. Merge order is going to be interesting - depending on whether block merge happens before or after ide one.
I'm going to put linux-next-based series on kernel.org tonight, before going to Portland...
Quoted 6 lines
Quoted 2 lines
We need to fix this.
Why not just revert the offending change and try again during the next merge window, assuming someone has figured out an acceptable way to handle this mess by then?
Easier just to fix it. Its a case of building everything until it compiles with the prototype change. Almost all stuff will just take the argument initially and not use it.
Anyone else plan to do it or shall I hit all the x86 cases and post a patch ?
Quoted 7 lines · Alan Cox
Easier just to fix it. Its a case of building everything until it compiles with the prototype change. Almost all stuff will just take the argument initially and not use it.
Anyone else plan to do it or shall I hit all the x86 cases and post a patch ?
Well, I alrady reverted it, but if you actually fix unlocked_ioctl() to have the same calling convention as regular ioctl() then a lot of the noise from ioctl conversion goes away, and all that remains is literally just the BKL part.
Btw, why is unlocked_ioctl returning "long"? Does anybody depend on that too? That's another difference between the "unlocked" and the traditional version..
As to the "x86 cases", I think you should try to hit them all. Doing a "git grep unlocked_ioctl" gets 185 entries, and it looks like only something like 8 of them are non-x86 (3 in the arch/ directory, five in s390 drivers).
Of course, some of them may be drivers that aren't available on x86 for other reasons (ie the ARM embedded stuff), but regardless..
Anyway, the pure size of that patch makes me suspect that we might as well leave it until the next merge window, but if you do it and it's obviously totally mechanical, I'd be likely to just let it slip in early.
Linus
Quoted 3 lines
Btw, why is
unlocked_ioctlreturning "long"? Does anybody depend on that too? That's another difference between the "unlocked" and the traditional version..
I don't know - a lot of syscall returns got defined as long and I guess someone thought propogating the right type was a good diea ?
Quoted 6 lines
As to the "x86 cases", I think you should try to hit them all. Doing a "git grep
unlocked_ioctl" gets 185 entries, and it looks like only something like 8 of them are non-x86 (3 in thearch/directory, five in s390 drivers).
Quoted 6 lines
Of course, some of them may be drivers that aren't available on x86 for other reasons (ie the ARM embedded stuff), but regardless..
Anyway, the pure size of that patch makes me suspect that we might as well leave it until the next merge window, but if you do it and it's obviously totally mechanical, I'd be likely to just let it slip in early.
I'll take a crack at it tomorrow - but if its 185 entries then it probably wants to go into -next instead.
Alan