Skip to contentSkip to conversation

Regression - locking (all from 2.6.28)

DISCUSSION Andrew Morton 8 participants 18 messages Mar 2, 2009 – Mar 18, 2009 1 message not in the archive

Subsystems (2)
lore
Share & tools
lore

Andrew Morton

author maintainer
#1 lore raw
In reply to a message not in the archive
Quoted 10 lines · On Mon, 02 Mar 2009 20:28:10 +0100 jan sonnek <[email protected]> wrote:

Later I have reported regression, now I have better debug info in the attachements.

Later reporter - still actual for (2.6.29-rc6-mm1):
From 2.6.28 and other 2.6.29-rc3-mm1 I have problem with starting
system with GDM (gdm-2.24.0-12). Without login screen system stop and generate error (all kernel soon then 2.6.27 are ok).

It is bug: http://bugzilla.kernel.org/show_bug.cgi?id=12619

The linux-kernel mailing list probably won't accept a 662 kbyte email. Please trim these reports down to some sane size.

Quoted 4 lines
Mar  1 00:07:03 localhost kernel: [   86.440261] =========================================================
Mar  1 00:07:03 localhost kernel: [   86.440266] [ INFO: possible irq lock inversion dependency detected ]
Mar  1 00:07:03 localhost kernel: [   86.440271] 2.6.29-rc6-mm1-hanny #17
Mar  1 00:07:03 localhost kernel: [   86.440273] ---------------------------------------------------------

I stared at this for a while, but my brain broke trying to work out what lockdep is trying to tell us.

Quoted 4 lines
Mar  1 00:07:03 localhost kernel: [   86.440277] Xorg/2733 just changed the state of lock:
Mar  1 00:07:03 localhost kernel: [   86.440280]  (fasync_lock){.-....}, at: [<c01952bb>] kill_fasync+0x20/0x3a
Mar  1 00:07:03 localhost kernel: [   86.440292] but this lock took another, HARDIRQ-READ-irq-unsafe lock in the past:
Mar  1 00:07:03 localhost kernel: [   86.440296]  (&f->f_lock){+.+...}

This message needs help. A lock cannot "take" another lock. And why is f_lock described as "HARDIRQ-READ-irq-unsafe"? It's a spinlock and the "READ" part is not relevant.

Quoted 164 lines
Mar  1 00:07:03 localhost kernel: [   86.440299] 
Mar  1 00:07:03 localhost kernel: [   86.440300] and interrupts could create inverse lock ordering between them.
Mar  1 00:07:03 localhost kernel: [   86.440302] 
Mar  1 00:07:03 localhost kernel: [   86.440305] 
Mar  1 00:07:03 localhost kernel: [   86.440305] other info that might help us debug this:
Mar  1 00:07:03 localhost kernel: [   86.440309] 3 locks held by Xorg/2733:
Mar  1 00:07:03 localhost kernel: [   86.440312]  #0:  (&dev->event_lock){-.-...}, at: [<c02d5e8c>] input_event+0x35/0x69
Mar  1 00:07:03 localhost kernel: [   86.440322]  #1:  (rcu_read_lock){.+.+..}, at: [<c02d4af3>] __rcu_read_lock+0x0/0x30
Mar  1 00:07:03 localhost kernel: [   86.440331]  #2:  (rcu_read_lock){.+.+..}, at: [<c02d872a>] evdev_event+0x0/0xe2
Mar  1 00:07:03 localhost kernel: [   86.440340] 
Mar  1 00:07:03 localhost kernel: [   86.440341] the first lock's dependencies:
Mar  1 00:07:03 localhost kernel: [   86.440344] -> (fasync_lock){.-....} ops: 190 {
Mar  1 00:07:03 localhost kernel: [   86.440351]    IN-HARDIRQ-R at:
Mar  1 00:07:03 localhost kernel: [   86.440355]                                        [<c01471a2>] __lock_acquire+0x204/0xb4a
Mar  1 00:07:03 localhost kernel: [   86.440362]                                        [<c0147b45>] lock_acquire+0x5d/0x7a
Mar  1 00:07:03 localhost kernel: [   86.440368]                                        [<c0398bc2>] _read_lock+0x2d/0x5d
Mar  1 00:07:03 localhost kernel: [   86.440375]                                        [<c01952bb>] kill_fasync+0x20/0x3a
Mar  1 00:07:03 localhost kernel: [   86.440381]                                        [<c02d84a7>] evdev_pass_event+0x60/0x66
Mar  1 00:07:03 localhost kernel: [   86.440387]                                        [<c02d879d>] evdev_event+0x73/0xe2
Mar  1 00:07:03 localhost kernel: [   86.440393]                                        [<c02d4bb9>] input_pass_event+0x5c/0x7f
Mar  1 00:07:03 localhost kernel: [   86.440399]                                        [<c02d5dd6>] input_handle_event+0x366/0x36f
Mar  1 00:07:03 localhost kernel: [   86.440405]                                        [<c02d5eab>] input_event+0x54/0x69
Mar  1 00:07:03 localhost kernel: [   86.440410]                                        [<c02f1e03>] hidinput_hid_event+0x24c/0x279
Mar  1 00:07:03 localhost kernel: [   86.440418]                                        [<c02ef1ed>] hid_process_event+0x8d/0xbc
Mar  1 00:07:03 localhost kernel: [   86.440424]                                        [<c02ef558>] hid_report_raw_event+0x33c/0x348
Mar  1 00:07:03 localhost kernel: [   86.440431]                                        [<c02ef60f>] hid_input_report+0xab/0xbc
Mar  1 00:07:03 localhost kernel: [   86.440437]                                        [<c02f576c>] hid_irq_in+0x86/0x182
Mar  1 00:07:03 localhost kernel: [   86.440443]                                        [<c02af7fa>] usb_hcd_giveback_urb+0x68/0x9c
Mar  1 00:07:03 localhost kernel: [   86.440451]                                        [<c02cec0f>] uhci_giveback_urb+0xf6/0x1f1
Mar  1 00:07:03 localhost kernel: [   86.440458]                                        [<c02cf404>] uhci_scan_schedule+0x5f8/0x85f
Mar  1 00:07:03 localhost kernel: [   86.440464]                                        [<c02d10e7>] uhci_irq+0x12b/0x13f
Mar  1 00:07:03 localhost kernel: [   86.440470]                                        [<c02af3cc>] usb_hcd_irq+0x32/0x81
Mar  1 00:07:03 localhost kernel: [   86.440475]                                        [<c0156b6b>] handle_IRQ_event+0x1f/0x4b
Mar  1 00:07:03 localhost kernel: [   86.440483]                                        [<c0157d09>] handle_fasteoi_irq+0x77/0xb0
Mar  1 00:07:03 localhost kernel: [   86.440489]                                        [<ffffffff>] 0xffffffff
Mar  1 00:07:03 localhost kernel: [   86.440494]    INITIAL USE at:
Mar  1 00:07:03 localhost kernel: [   86.440498]                                       [<c01472e2>] __lock_acquire+0x344/0xb4a
Mar  1 00:07:03 localhost kernel: [   86.440504]                                       [<c0147b45>] lock_acquire+0x5d/0x7a
Mar  1 00:07:03 localhost kernel: [   86.440510]                                       [<c039895d>] _write_lock_irq+0x33/0x63
Mar  1 00:07:03 localhost kernel: [   86.440515]                                       [<c0194ef6>] fasync_helper+0x44/0xe4
Mar  1 00:07:03 localhost kernel: [   86.440521]                                       [<c0247242>] tty_fasync+0x50/0xea
Mar  1 00:07:03 localhost kernel: [   86.440528]                                       [<c0249322>] tty_release_dev+0x57/0x409
Mar  1 00:07:03 localhost kernel: [   86.440534]                                       [<c02496eb>] tty_release+0x17/0x21
Mar  1 00:07:03 localhost kernel: [   86.440539]                                       [<c018c369>] __fput+0xcf/0x158
Mar  1 00:07:03 localhost kernel: [   86.440546]                                       [<c018c410>] fput+0x1e/0x20
Mar  1 00:07:03 localhost kernel: [   86.440551]                                       [<c0189b12>] filp_close+0x56/0x60
Mar  1 00:07:03 localhost kernel: [   86.440557]                                       [<c0189b8b>] sys_close+0x6f/0xa9
Mar  1 00:07:03 localhost kernel: [   86.440562]                                       [<c0102f47>] sysenter_do_call+0x12/0x35
Mar  1 00:07:03 localhost kernel: [   86.440569]                                       [<ffffffff>] 0xffffffff
Mar  1 00:07:03 localhost kernel: [   86.440574]  }
Mar  1 00:07:03 localhost kernel: [   86.440576]  ... key      at: [<c04fea24>] fasync_lock+0x10/0x24
Mar  1 00:07:03 localhost kernel: [   86.440583]  -> (&f->f_lock){+.+...} ops: 493 {
Mar  1 00:07:03 localhost kernel: [   86.440590]     HARDIRQ-ON-W at:
Mar  1 00:07:03 localhost kernel: [   86.440594]                                          [<c0147260>] __lock_acquire+0x2c2/0xb4a
Mar  1 00:07:03 localhost kernel: [   86.440600]                                          [<c0147b45>] lock_acquire+0x5d/0x7a
Mar  1 00:07:03 localhost kernel: [   86.440606]                                          [<c039869c>] _spin_lock+0x2d/0x5d
Mar  1 00:07:03 localhost kernel: [   86.440612]                                          [<c019561e>] do_fcntl+0x222/0x2bc
Mar  1 00:07:03 localhost kernel: [   86.440617]                                          [<c0195712>] sys_fcntl64+0x5a/0x6e
Mar  1 00:07:03 localhost kernel: [   86.440623]                                          [<c0102f47>] sysenter_do_call+0x12/0x35
Mar  1 00:07:03 localhost kernel: [   86.440629]                                          [<ffffffff>] 0xffffffff
Mar  1 00:07:03 localhost kernel: [   86.440633]     SOFTIRQ-ON-W at:
Mar  1 00:07:03 localhost kernel: [   86.440637]                                          [<c0147283>] __lock_acquire+0x2e5/0xb4a
Mar  1 00:07:03 localhost kernel: [   86.440643]                                          [<c0147b45>] lock_acquire+0x5d/0x7a
Mar  1 00:07:03 localhost kernel: [   86.440649]                                          [<c039869c>] _spin_lock+0x2d/0x5d
Mar  1 00:07:03 localhost kernel: [   86.440654]                                          [<c019561e>] do_fcntl+0x222/0x2bc
Mar  1 00:07:03 localhost kernel: [   86.440660]                                          [<c0195712>] sys_fcntl64+0x5a/0x6e
Mar  1 00:07:03 localhost kernel: [   86.440666]                                          [<c0102f47>] sysenter_do_call+0x12/0x35
Mar  1 00:07:03 localhost kernel: [   86.440672]                                          [<ffffffff>] 0xffffffff
Mar  1 00:07:03 localhost kernel: [   86.440676]     INITIAL USE at:
Mar  1 00:07:03 localhost kernel: [   86.440680]                                         [<c01472e2>] __lock_acquire+0x344/0xb4a
Mar  1 00:07:03 localhost kernel: [   86.440686]                                         [<c0147b45>] lock_acquire+0x5d/0x7a
Mar  1 00:07:03 localhost kernel: [   86.440691]                                         [<c039869c>] _spin_lock+0x2d/0x5d
Mar  1 00:07:03 localhost kernel: [   86.440697]                                         [<c0194f66>] fasync_helper+0xb4/0xe4
Mar  1 00:07:03 localhost kernel: [   86.440703]                                         [<c0247242>] tty_fasync+0x50/0xea
Mar  1 00:07:03 localhost kernel: [   86.440708]                                         [<c0249322>] tty_release_dev+0x57/0x409
Mar  1 00:07:03 localhost kernel: [   86.440714]                                         [<c02496eb>] tty_release+0x17/0x21
Mar  1 00:07:03 localhost kernel: [   86.440720]                                         [<c018c369>] __fput+0xcf/0x158
Mar  1 00:07:03 localhost kernel: [   86.440725]                                         [<c018c410>] fput+0x1e/0x20
Mar  1 00:07:03 localhost kernel: [   86.440731]                                         [<c0189b12>] filp_close+0x56/0x60
Mar  1 00:07:03 localhost kernel: [   86.440736]                                         [<c0189b8b>] sys_close+0x6f/0xa9
Mar  1 00:07:03 localhost kernel: [   86.440741]                                         [<c0102f47>] sysenter_do_call+0x12/0x35
Mar  1 00:07:03 localhost kernel: [   86.440747]                                         [<ffffffff>] 0xffffffff
Mar  1 00:07:03 localhost kernel: [   86.440752]   }
Mar  1 00:07:03 localhost kernel: [   86.440754]   ... key      at: [<c0b839d0>] __key.20190+0x0/0x8
Mar  1 00:07:03 localhost kernel: [   86.440760]  ... acquired at:
Mar  1 00:07:03 localhost kernel: [   86.440763]    [<c0147965>] __lock_acquire+0x9c7/0xb4a
Mar  1 00:07:03 localhost kernel: [   86.440768]    [<c0147b45>] lock_acquire+0x5d/0x7a
Mar  1 00:07:03 localhost kernel: [   86.440773]    [<c039869c>] _spin_lock+0x2d/0x5d
Mar  1 00:07:03 localhost kernel: [   86.440778]    [<c0194f66>] fasync_helper+0xb4/0xe4
Mar  1 00:07:03 localhost kernel: [   86.440783]    [<c0247242>] tty_fasync+0x50/0xea
Mar  1 00:07:03 localhost kernel: [   86.440788]    [<c0249322>] tty_release_dev+0x57/0x409
Mar  1 00:07:03 localhost kernel: [   86.440793]    [<c02496eb>] tty_release+0x17/0x21
Mar  1 00:07:03 localhost kernel: [   86.440798]    [<c018c369>] __fput+0xcf/0x158
Mar  1 00:07:03 localhost kernel: [   86.440803]    [<c018c410>] fput+0x1e/0x20
Mar  1 00:07:03 localhost kernel: [   86.440807]    [<c0189b12>] filp_close+0x56/0x60
Mar  1 00:07:03 localhost kernel: [   86.440812]    [<c0189b8b>] sys_close+0x6f/0xa9
Mar  1 00:07:03 localhost kernel: [   86.440817]    [<c0102f47>] sysenter_do_call+0x12/0x35
Mar  1 00:07:03 localhost kernel: [   86.440822]    [<ffffffff>] 0xffffffff
Mar  1 00:07:03 localhost kernel: [   86.440826] 
Mar  1 00:07:03 localhost kernel: [   86.440828] 
Mar  1 00:07:03 localhost kernel: [   86.440829] the second lock's dependencies:
Mar  1 00:07:03 localhost kernel: [   86.440832] -> (&f->f_lock){+.+...} ops: 493 {
Mar  1 00:07:03 localhost kernel: [   86.440839]    HARDIRQ-ON-W at:
Mar  1 00:07:03 localhost kernel: [   86.440842]                                        [<c0147260>] __lock_acquire+0x2c2/0xb4a
Mar  1 00:07:03 localhost kernel: [   86.440848]                                        [<c0147b45>] lock_acquire+0x5d/0x7a
Mar  1 00:07:03 localhost kernel: [   86.440854]                                        [<c039869c>] _spin_lock+0x2d/0x5d
Mar  1 00:07:03 localhost kernel: [   86.440859]                                        [<c019561e>] do_fcntl+0x222/0x2bc
Mar  1 00:07:03 localhost kernel: [   86.440865]                                        [<c0195712>] sys_fcntl64+0x5a/0x6e
Mar  1 00:07:03 localhost kernel: [   86.440871]                                        [<c0102f47>] sysenter_do_call+0x12/0x35
Mar  1 00:07:03 localhost kernel: [   86.440876]                                        [<ffffffff>] 0xffffffff
Mar  1 00:07:03 localhost kernel: [   86.440881]    SOFTIRQ-ON-W at:
Mar  1 00:07:03 localhost kernel: [   86.440884]                                        [<c0147283>] __lock_acquire+0x2e5/0xb4a
Mar  1 00:07:03 localhost kernel: [   86.440890]                                        [<c0147b45>] lock_acquire+0x5d/0x7a
Mar  1 00:07:03 localhost kernel: [   86.440896]                                        [<c039869c>] _spin_lock+0x2d/0x5d
Mar  1 00:07:03 localhost kernel: [   86.440901]                                        [<c019561e>] do_fcntl+0x222/0x2bc
Mar  1 00:07:03 localhost kernel: [   86.440907]                                        [<c0195712>] sys_fcntl64+0x5a/0x6e
Mar  1 00:07:03 localhost kernel: [   86.440913]                                        [<c0102f47>] sysenter_do_call+0x12/0x35
Mar  1 00:07:03 localhost kernel: [   86.440918]                                        [<ffffffff>] 0xffffffff
Mar  1 00:07:03 localhost kernel: [   86.440923]    INITIAL USE at:
Mar  1 00:07:03 localhost kernel: [   86.440926]                                       [<c01472e2>] __lock_acquire+0x344/0xb4a
Mar  1 00:07:03 localhost kernel: [   86.440932]                                       [<c0147b45>] lock_acquire+0x5d/0x7a
Mar  1 00:07:03 localhost kernel: [   86.440938]                                       [<c039869c>] _spin_lock+0x2d/0x5d
Mar  1 00:07:03 localhost kernel: [   86.440943]                                       [<c0194f66>] fasync_helper+0xb4/0xe4
Mar  1 00:07:03 localhost kernel: [   86.440949]                                       [<c0247242>] tty_fasync+0x50/0xea
Mar  1 00:07:03 localhost kernel: [   86.440955]                                       [<c0249322>] tty_release_dev+0x57/0x409
Mar  1 00:07:03 localhost kernel: [   86.440960]                                       [<c02496eb>] tty_release+0x17/0x21
Mar  1 00:07:03 localhost kernel: [   86.440966]                                       [<c018c369>] __fput+0xcf/0x158
Mar  1 00:07:03 localhost kernel: [   86.440971]                                       [<c018c410>] fput+0x1e/0x20
Mar  1 00:07:03 localhost kernel: [   86.440977]                                       [<c0189b12>] filp_close+0x56/0x60
Mar  1 00:07:03 localhost kernel: [   86.440982]                                       [<c0189b8b>] sys_close+0x6f/0xa9
Mar  1 00:07:03 localhost kernel: [   86.440988]                                       [<c0102f47>] sysenter_do_call+0x12/0x35
Mar  1 00:07:03 localhost kernel: [   86.440993]                                       [<ffffffff>] 0xffffffff
Mar  1 00:07:03 localhost kernel: [   86.440998]  }
Mar  1 00:07:03 localhost kernel: [   86.441000]  ... key      at: [<c0b839d0>] __key.20190+0x0/0x8
Mar  1 00:07:03 localhost kernel: [   86.441005] 
Mar  1 00:07:03 localhost kernel: [   86.441006] stack backtrace:
Mar  1 00:07:03 localhost kernel: [   86.441010] Pid: 2733, comm: Xorg Not tainted 2.6.29-rc6-mm1-hanny #17
Mar  1 00:07:03 localhost kernel: [   86.441013] Call Trace:
Mar  1 00:07:03 localhost kernel: [   86.441018]  [<c0396150>] ? printk+0x14/0x1c
Mar  1 00:07:03 localhost kernel: [   86.441023]  [<c0146a58>] print_irq_inversion_bug+0xea/0xf7
Mar  1 00:07:03 localhost kernel: [   86.441029]  [<c0146a9b>] check_usage_forwards+0x36/0x3f
Mar  1 00:07:03 localhost kernel: [   86.441034]  [<c0146425>] mark_lock+0x129/0x20b
Mar  1 00:07:03 localhost kernel: [   86.441038]  [<c0146a65>] ? check_usage_forwards+0x0/0x3f
Mar  1 00:07:03 localhost kernel: [   86.441043]  [<c01471a2>] __lock_acquire+0x204/0xb4a
Mar  1 00:07:03 localhost kernel: [   86.441048]  [<c02d8467>] ? evdev_pass_event+0x20/0x66
Mar  1 00:07:03 localhost kernel: [   86.441054]  [<c0147b45>] lock_acquire+0x5d/0x7a
Mar  1 00:07:03 localhost kernel: [   86.441059]  [<c01952bb>] ? kill_fasync+0x20/0x3a
Mar  1 00:07:03 localhost kernel: [   86.441063]  [<c0398bc2>] _read_lock+0x2d/0x5d
Mar  1 00:07:03 localhost kernel: [   86.441068]  [<c01952bb>] ? kill_fasync+0x20/0x3a
Mar  1 00:07:03 localhost kernel: [   86.441073]  [<c01952bb>] kill_fasync+0x20/0x3a
Mar  1 00:07:03 localhost kernel: [   86.441078]  [<c02d84a7>] evdev_pass_event+0x60/0x66
Mar  1 00:07:03 localhost kernel: [   86.441083]  [<c02d879d>] evdev_event+0x73/0xe2
Mar  1 00:07:03 localhost kernel: [   86.441087]  [<c02d4bb9>] input_pass_event+0x5c/0x7f
Mar  1 00:07:03 localhost kernel: [   86.441092]  [<c02d5dd6>] input_handle_event+0x366/0x36f
Mar  1 00:07:03 localhost kernel: [   86.441098]  [<c0246eac>] ? add_timer_randomness+0xee/0x108
Mar  1 00:07:03 localhost kernel: [   86.441103]  [<c02d5eab>] input_event+0x54/0x69
Mar  1 00:07:03 localhost kernel: [   86.441108]  [<c02f1e03>] hidinput_hid_event+0x24c/0x279
Mar  1 00:07:03 localhost kernel: [   86.441114]  [<c02ef1ed>] hid_process_event+0x8d/0xbc
Mar  1 00:07:03 localhost kernel: [   86.441119]  [<c02ef558>] hid_report_raw_event+0x33c/0x348
Mar  1 00:07:03 localhost kernel: [   86.441125]  [<c02ef60f>] hid_input_report+0xab/0xbc
Mar  1 00:07:03 localhost kernel: [   86.441130]  [<c02f576c>] hid_irq_in+0x86/0x182
Mar  1 00:07:03 localhost kernel: [   86.441135]  [<c02af7fa>] usb_hcd_giveback_urb+0x68/0x9c
Mar  1 00:07:03 localhost kernel: [   86.441140]  [<c02cec0f>] uhci_giveback_urb+0xf6/0x1f1
Mar  1 00:07:03 localhost kernel: [   86.441145]  [<c0147ad9>] ? __lock_acquire+0xb3b/0xb4a
Quoted 9 lines
Mar  1 00:07:03 localhost kernel: [   86.441151]  [<c02cf404>] uhci_scan_schedule+0x5f8/0x85f
Mar  1 00:07:03 localhost kernel: [   86.441156]  [<c014548a>] ? put_lock_stats+0xd/0x21
Mar  1 00:07:03 localhost kernel: [   86.441161]  [<c02d10e7>] uhci_irq+0x12b/0x13f
Mar  1 00:07:03 localhost kernel: [   86.441166]  [<c02af3cc>] usb_hcd_irq+0x32/0x81
Mar  1 00:07:03 localhost kernel: [   86.441172]  [<c0156b6b>] handle_IRQ_event+0x1f/0x4b
Mar  1 00:07:03 localhost kernel: [   86.441176]  [<c0157d09>] handle_fasteoi_irq+0x77/0xb0
Mar  1 00:07:03 localhost kernel: [   86.441181]  [<c0157c92>] ? handle_fasteoi_irq+0x0/0xb0
Mar  1 00:07:03 localhost kernel: [   86.441184]  <IRQ>  [<c0398faa>] ? __irqentry_text_start+0x4a/0x8c

And bazillions of these:

Quoted 49 lines
Mar  1 00:06:51 localhost kernel: [   74.007988] unreferenced object 0xf6c4da80 (size 52):
Mar  1 00:06:51 localhost kernel: [   74.007991]   comm "swapper", pid 1, jiffies 4294893427
Mar  1 00:06:51 localhost kernel: [   74.007994]   backtrace:
Mar  1 00:06:51 localhost kernel: [   74.007997]     [<c018978c>] kmemleak_alloc+0x17e/0x28e
Mar  1 00:06:51 localhost kernel: [   74.008002]     [<c0186b86>] kmem_cache_alloc+0xdc/0xe7
Mar  1 00:06:51 localhost kernel: [   74.008006]     [<c01a53bd>] alloc_buffer_head+0x16/0x71
Mar  1 00:06:51 localhost kernel: [   74.008011]     [<c01a5b91>] alloc_page_buffers+0x23/0xad
Mar  1 00:06:51 localhost kernel: [   74.008015]     [<c01a5fd4>] __getblk+0x192/0x26b
Mar  1 00:06:51 localhost kernel: [   74.008020]     [<c01d91f4>] jread+0x105/0x1de
Mar  1 00:06:51 localhost kernel: [   74.008026]     [<c01d932b>] do_one_pass+0x5e/0x38c
Mar  1 00:06:51 localhost kernel: [   74.008031]     [<c01d96f8>] journal_recover+0x41/0x9d
Mar  1 00:06:51 localhost kernel: [   74.008037]     [<c01db8d4>] journal_load+0x47/0x7b
Mar  1 00:06:51 localhost kernel: [   74.008042]     [<c01d43d1>] ext3_fill_super+0xe9d/0x144c
Mar  1 00:06:51 localhost kernel: [   74.008047]     [<c018d721>] get_sb_bdev+0xfa/0x140
Mar  1 00:06:51 localhost kernel: [   74.008052]     [<c01d2070>] ext3_get_sb+0x18/0x1a
Mar  1 00:06:51 localhost kernel: [   74.008057]     [<c018c71f>] vfs_kern_mount+0x41/0x7c
Mar  1 00:06:51 localhost kernel: [   74.008062]     [<c018c7a8>] do_kern_mount+0x37/0xbe
Mar  1 00:06:51 localhost kernel: [   74.008067]     [<c019f0bf>] do_mount+0x5f7/0x630
Mar  1 00:06:51 localhost kernel: [   74.008070]     [<c019f167>] sys_mount+0x6f/0xac
Mar  1 00:06:51 localhost kernel: [   74.008075] unreferenced object 0xf6c4dab8 (size 52):
Mar  1 00:06:51 localhost kernel: [   74.008078]   comm "swapper", pid 1, jiffies 4294893427
Mar  1 00:06:51 localhost kernel: [   74.008081]   backtrace:
Mar  1 00:06:51 localhost kernel: [   74.008085]     [<c018978c>] kmemleak_alloc+0x17e/0x28e
Mar  1 00:06:51 localhost kernel: [   74.008091]     [<c0186b86>] kmem_cache_alloc+0xdc/0xe7
Mar  1 00:06:51 localhost kernel: [   74.008097]     [<c01a53bd>] alloc_buffer_head+0x16/0x71
Mar  1 00:06:51 localhost kernel: [   74.008103]     [<c01a5b91>] alloc_page_buffers+0x23/0xad
Mar  1 00:06:51 localhost kernel: [   74.008109]     [<c01a5fd4>] __getblk+0x192/0x26b
Mar  1 00:06:51 localhost kernel: [   74.008114]     [<c01d91f4>] jread+0x105/0x1de
Mar  1 00:06:51 localhost kernel: [   74.008118]     [<c01d932b>] do_one_pass+0x5e/0x38c
Mar  1 00:06:51 localhost kernel: [   74.008122]     [<c01d96f8>] journal_recover+0x41/0x9d
Mar  1 00:06:51 localhost kernel: [   74.008127]     [<c01db8d4>] journal_load+0x47/0x7b
Mar  1 00:06:51 localhost kernel: [   74.008132]     [<c01d43d1>] ext3_fill_super+0xe9d/0x144c
Mar  1 00:06:51 localhost kernel: [   74.008136]     [<c018d721>] get_sb_bdev+0xfa/0x140
Mar  1 00:06:51 localhost kernel: [   74.008141]     [<c01d2070>] ext3_get_sb+0x18/0x1a
Mar  1 00:06:51 localhost kernel: [   74.008145]     [<c018c71f>] vfs_kern_mount+0x41/0x7c
Mar  1 00:06:51 localhost kernel: [   74.008149]     [<c018c7a8>] do_kern_mount+0x37/0xbe
Mar  1 00:06:51 localhost kernel: [   74.008154]     [<c019f0bf>] do_mount+0x5f7/0x630
Mar  1 00:06:51 localhost kernel: [   74.008159]     [<c019f167>] sys_mount+0x6f/0xac
Mar  1 00:06:51 localhost kernel: [   74.008165] unreferenced object 0xf6c4daf0 (size 52):
Mar  1 00:06:51 localhost kernel: [   74.008170]   comm "swapper", pid 1, jiffies 4294893427
Mar  1 00:06:51 localhost kernel: [   74.008175]   backtrace:
Mar  1 00:06:51 localhost kernel: [   74.008179]     [<c018978c>] kmemleak_alloc+0x17e/0x28e
Mar  1 00:06:51 localhost kernel: [   74.008185]     [<c0186b86>] kmem_cache_alloc+0xdc/0xe7
Mar  1 00:06:51 localhost kernel: [   74.008190]     [<c01a53bd>] alloc_buffer_head+0x16/0x71
Mar  1 00:06:51 localhost kernel: [   74.008196]     [<c01a5b91>] alloc_page_buffers+0x23/0xad
Mar  1 00:06:51 localhost kernel: [   74.008200]     [<c01a5fd4>] __getblk+0x192/0x26b
Mar  1 00:06:51 localhost kernel: [   74.008205]     [<c01d91f4>] jread+0x105/0x1de
Mar  1 00:06:51 localhost kernel: [   74.008209]     [<c01d932b>] do_one_pass+0x5e/0x38c
Mar  1 00:06:51 localhost kernel: [   74.008213]     [<c01d96f8>] journal_recover+0x41/0x9d
Quoted 8 lines
Mar  1 00:06:51 localhost kernel: [   74.008218]     [<c01db8d4>] journal_load+0x47/0x7b
Mar  1 00:06:51 localhost kernel: [   74.008221]     [<c01d43d1>] ext3_fill_super+0xe9d/0x144c
Mar  1 00:06:51 localhost kernel: [   74.008225]     [<c018d721>] get_sb_bdev+0xfa/0x140
Mar  1 00:06:51 localhost kernel: [   74.008231]     [<c01d2070>] ext3_get_sb+0x18/0x1a
Mar  1 00:06:51 localhost kernel: [   74.008235]     [<c018c71f>] vfs_kern_mount+0x41/0x7c
Mar  1 00:06:51 localhost kernel: [   74.008241]     [<c018c7a8>] do_kern_mount+0x37/0xbe
Mar  1 00:06:51 localhost kernel: [   74.008247]     [<c019f0bf>] do_mount+0x5f7/0x630
Mar  1 00:06:51 localhost kernel: [   74.008253]     [<c019f167>] sys_mount+0x6f/0xac

I suspect kmemleak has gone nuts here.

kmemleak has no MAINTAINERS entry, btw.

#2 lore raw
Quoted 20 lines · Andrew Morton
Quoted 11 lines
Mar  1 00:06:51 localhost kernel: [   74.008165] unreferenced object 0xf6c4daf0 (size 52):
Mar  1 00:06:51 localhost kernel: [   74.008170]   comm "swapper", pid 1, jiffies 4294893427
Mar  1 00:06:51 localhost kernel: [   74.008175]   backtrace:
Mar  1 00:06:51 localhost kernel: [   74.008179]     [<c018978c>] kmemleak_alloc+0x17e/0x28e
Mar  1 00:06:51 localhost kernel: [   74.008185]     [<c0186b86>] kmem_cache_alloc+0xdc/0xe7
Mar  1 00:06:51 localhost kernel: [   74.008190]     [<c01a53bd>] alloc_buffer_head+0x16/0x71
Mar  1 00:06:51 localhost kernel: [   74.008196]     [<c01a5b91>] alloc_page_buffers+0x23/0xad
Mar  1 00:06:51 localhost kernel: [   74.008200]     [<c01a5fd4>] __getblk+0x192/0x26b
Mar  1 00:06:51 localhost kernel: [   74.008205]     [<c01d91f4>] jread+0x105/0x1de
Mar  1 00:06:51 localhost kernel: [   74.008209]     [<c01d932b>] do_one_pass+0x5e/0x38c
Mar  1 00:06:51 localhost kernel: [   74.008213]     [<c01d96f8>] journal_recover+0x41/0x9d
Quoted 8 lines
Mar  1 00:06:51 localhost kernel: [   74.008218]     [<c01db8d4>] journal_load+0x47/0x7b
Mar  1 00:06:51 localhost kernel: [   74.008221]     [<c01d43d1>] ext3_fill_super+0xe9d/0x144c
Mar  1 00:06:51 localhost kernel: [   74.008225]     [<c018d721>] get_sb_bdev+0xfa/0x140
Mar  1 00:06:51 localhost kernel: [   74.008231]     [<c01d2070>] ext3_get_sb+0x18/0x1a
Mar  1 00:06:51 localhost kernel: [   74.008235]     [<c018c71f>] vfs_kern_mount+0x41/0x7c
Mar  1 00:06:51 localhost kernel: [   74.008241]     [<c018c7a8>] do_kern_mount+0x37/0xbe
Mar  1 00:06:51 localhost kernel: [   74.008247]     [<c019f0bf>] do_mount+0x5f7/0x630
Mar  1 00:06:51 localhost kernel: [   74.008253]     [<c019f167>] sys_mount+0x6f/0xac
Quoted 1 line

I suspect kmemleak has gone nuts here.

It seems that the buffer_head structure allocated above is stored in page->private. However, the page structures are no longer scanned in newer versions of kmemleak. That's the hunk that was removed after comments about the contiguity of a node's memory:

+ /* mem_map scanning */
+ for_each_online_node(i) {
+ struct page *page, *end;
+
+ page = NODE_MEM_MAP(i);
+ end = page + NODE_DATA(i)->node_spanned_pages;
+
+ scan_block(page, end, NULL);
+ }

The alternative is to inform kmemleak about the page structures returned from __alloc_pages_internal() but there would be problems with recursive calls into kmemleak when it allocates its own data structures.

I'll look at re-adding the hunk above, maybe with some extra checks like pfn_valid().

Quoted 1 line

kmemleak has no MAINTAINERS entry, btw.

The entry is called "KERNEL MEMORY LEAK DETECTOR" but it makes sense to change it to KMEMLEAK.

Thanks.

Signature · 1 line
Catalin
#3 lore raw
Quoted 38 lines · Catalin Marinas
Quoted 20 lines · Andrew Morton
Quoted 11 lines
Mar  1 00:06:51 localhost kernel: [   74.008165] unreferenced object 0xf6c4daf0 (size 52):
Mar  1 00:06:51 localhost kernel: [   74.008170]   comm "swapper", pid 1, jiffies 4294893427
Mar  1 00:06:51 localhost kernel: [   74.008175]   backtrace:
Mar  1 00:06:51 localhost kernel: [   74.008179]     [<c018978c>] kmemleak_alloc+0x17e/0x28e
Mar  1 00:06:51 localhost kernel: [   74.008185]     [<c0186b86>] kmem_cache_alloc+0xdc/0xe7
Mar  1 00:06:51 localhost kernel: [   74.008190]     [<c01a53bd>] alloc_buffer_head+0x16/0x71
Mar  1 00:06:51 localhost kernel: [   74.008196]     [<c01a5b91>] alloc_page_buffers+0x23/0xad
Mar  1 00:06:51 localhost kernel: [   74.008200]     [<c01a5fd4>] __getblk+0x192/0x26b
Mar  1 00:06:51 localhost kernel: [   74.008205]     [<c01d91f4>] jread+0x105/0x1de
Mar  1 00:06:51 localhost kernel: [   74.008209]     [<c01d932b>] do_one_pass+0x5e/0x38c
Mar  1 00:06:51 localhost kernel: [   74.008213]     [<c01d96f8>] journal_recover+0x41/0x9d
Quoted 8 lines
Mar  1 00:06:51 localhost kernel: [   74.008218]     [<c01db8d4>] journal_load+0x47/0x7b
Mar  1 00:06:51 localhost kernel: [   74.008221]     [<c01d43d1>] ext3_fill_super+0xe9d/0x144c
Mar  1 00:06:51 localhost kernel: [   74.008225]     [<c018d721>] get_sb_bdev+0xfa/0x140
Mar  1 00:06:51 localhost kernel: [   74.008231]     [<c01d2070>] ext3_get_sb+0x18/0x1a
Mar  1 00:06:51 localhost kernel: [   74.008235]     [<c018c71f>] vfs_kern_mount+0x41/0x7c
Mar  1 00:06:51 localhost kernel: [   74.008241]     [<c018c7a8>] do_kern_mount+0x37/0xbe
Mar  1 00:06:51 localhost kernel: [   74.008247]     [<c019f0bf>] do_mount+0x5f7/0x630
Mar  1 00:06:51 localhost kernel: [   74.008253]     [<c019f167>] sys_mount+0x6f/0xac
Quoted 1 line

I suspect kmemleak has gone nuts here.

It seems that the buffer_head structure allocated above is stored in page->private. However, the page structures are no longer scanned in newer versions of kmemleak. That's the hunk that was removed after comments about the contiguity of a node's memory:

(quoted diff) +9 -0
+     /* mem_map scanning */
+     for_each_online_node(i) {
+             struct page *page, *end;
+
+             page = NODE_MEM_MAP(i);
+             end  = page + NODE_DATA(i)->node_spanned_pages;
+
+             scan_block(page, end, NULL);
+     }
Quoted 6 lines

The alternative is to inform kmemleak about the page structures returned from __alloc_pages_internal() but there would be problems with recursive calls into kmemleak when it allocates its own data structures.

I'll look at re-adding the hunk above, maybe with some extra checks like pfn_valid().

Looking again at this, the node_mem_map is always contiguous and the code above only scans the node_mem_map, not the memory represented by the node (which may not be contiguous). So I think it is a valid code sequence.

If the above gets too deep into the nodes structure, an alternative would be:

mm/page_alloc.c +5 -0
index 09b6fd7..0f17e62 100644
@@ -3552,6 +3552,11 @@ static void __init_refok alloc_node_mem_map(struct pglist_data *pgdat)
 		map = alloc_remap(pgdat->node_id, size);
 		if (!map)
 			map = alloc_bootmem_node(pgdat, size);
+		/*
+		 * Inform kmemleak to scan the node_mem_map arrays as the page
+		 * structure may contain pointers to other objects.
+		 */
+		kmemleak_alloc(map, size, 1, GFP_ATOMIC);
 		pgdat->node_mem_map = map + (pgdat->node_start_pfn - start);
 	}
 #ifndef CONFIG_NEED_MULTIPLE_NODES
Signature · 1 line
Catalin
#18 lore raw
In reply to Andrew Morton
Quoted 8 lines · Andrew Morton
Quoted 4 lines
Mar  1 00:07:03 localhost kernel: [   86.440261] =========================================================
Mar  1 00:07:03 localhost kernel: [   86.440266] [ INFO: possible irq lock inversion dependency detected ]
Mar  1 00:07:03 localhost kernel: [   86.440271] 2.6.29-rc6-mm1-hanny #17
Mar  1 00:07:03 localhost kernel: [   86.440273] ---------------------------------------------------------

I stared at this for a while, but my brain broke trying to work out what lockdep is trying to tell us.

Quoted 6 lines
Quoted 4 lines
Mar  1 00:07:03 localhost kernel: [   86.440277] Xorg/2733 just changed the state of lock:
Mar  1 00:07:03 localhost kernel: [   86.440280]  (fasync_lock){.-....}, at: [<c01952bb>] kill_fasync+0x20/0x3a
Mar  1 00:07:03 localhost kernel: [   86.440292] but this lock took another, HARDIRQ-READ-irq-unsafe lock in the past:
Mar  1 00:07:03 localhost kernel: [   86.440296]  (&f->f_lock){+.+...}

This message needs help. A lock cannot "take" another lock.

It seemed a simple enough way to tell that the latter lock nests inside the former lock.

So what its saying is that we have:

fasync_lock

    f->f_lock

nesting, and fasync_lock got used in hardirq context, but the lock that was previously found to nest inside, was an IRQ-unsafe lock.

So $CODE code take f->f_lock, then IRQ could happen and fasync_lock, f->f_lock could happen and we'd be stuck.

Would something like:

"but this lock had a %s-irq-unsafe nestee in the past:" read better?

Quoted 3 lines

And why
is f_lock described as "HARDIRQ-READ-irq-unsafe"? It's a spinlock and the "READ" part is not relevant.

I think that's a bug due to the recent irq state tracking generalization patches, will hunt.

Quoted 3 lines
Quoted 3 lines
Mar  1 00:07:03 localhost kernel: [   86.440299] 
Mar  1 00:07:03 localhost kernel: [   86.440300] and interrupts could create inverse lock ordering between them.
Mar  1 00:07:03 localhost kernel: [   86.440302] 
#4 lore raw
In reply to Catalin Marinas
Quoted 14 lines · Catalin Marinas
Quoted 14 lines
(quoted diff) +9 -0
+     /* mem_map scanning */
+     for_each_online_node(i) {
+             struct page *page, *end;
+
+             page = NODE_MEM_MAP(i);
+             end  = page + NODE_DATA(i)->node_spanned_pages;
+
+             scan_block(page, end, NULL);
+     }

The alternative is to inform kmemleak about the page structures returned from __alloc_pages_internal() but there would be problems with recursive calls into kmemleak when it allocates its own data structures.

Quoted 7 lines
Quoted 2 lines

I'll look at re-adding the hunk above, maybe with some extra checks like pfn_valid().

Looking again at this, the node_mem_map is always contiguous and the code above only scans the node_mem_map, not the memory represented by the node (which may not be contiguous). So I think it is a valid code sequence.

The above is *not* a valid code sequence.

It is valid with discontig, but isn't valid for sparsemem. You simply can't expect to do math on 'struct page' pointers for any granularity larger than MAX_ORDER_NR_PAGES.

Also, we don't even define NODE_MEM_MAP() for all configurations so that code snippet won't even compile. We would be smart to kill that macro.

One completely unoptimized thing you can do which will scan a 'struct page' at a time is this:

	for_each_online_node(i) {
		unsigned long pfn;
		for (pfn = node_start_pfn(i); pfn < node_end_pfn(i); pfn++) {
			struct page *page;
			if (!pfn_valid(pfn))
				continue;
			page = pfn_to_page(pfn);
			scan_block(page, page+1, NULL);
		}
	}

The way to optimize it would be to call scan_block() only once for each MAX_ORDER_NR_PAGES that you encounter. The other option would be to use the active_regions functions to walk the memory.

Is there a requirement to reduce the number of calls to scan_block() here?

-- Dave

#5 lore raw
Quoted 10 lines · Dave Hansen
Quoted 9 lines · Catalin Marinas
Quoted 9 lines
(quoted diff) +9 -0
+     /* mem_map scanning */
+     for_each_online_node(i) {
+             struct page *page, *end;
+
+             page = NODE_MEM_MAP(i);
+             end  = page + NODE_DATA(i)->node_spanned_pages;
+
+             scan_block(page, end, NULL);
+     }

[...]

Quoted 23 lines

The above is *not* a valid code sequence.

It is valid with discontig, but isn't valid for sparsemem. You simply can't expect to do math on 'struct page' pointers for any granularity larger than MAX_ORDER_NR_PAGES.

Also, we don't even define NODE_MEM_MAP() for all configurations so that code snippet won't even compile. We would be smart to kill that macro.

One completely unoptimized thing you can do which will scan a 'struct page' at a time is this:

	for_each_online_node(i) {
		unsigned long pfn;
		for (pfn = node_start_pfn(i); pfn < node_end_pfn(i); pfn++) {
			struct page *page;
			if (!pfn_valid(pfn))
				continue;
			page = pfn_to_page(pfn);
			scan_block(page, page+1, NULL);
		}
	}
Quoted 6 lines

The way to optimize it would be to call scan_block() only once for each MAX_ORDER_NR_PAGES that you encounter. The other option would be to use the active_regions functions to walk the memory.

Is there a requirement to reduce the number of calls to scan_block() here?

I think the improvement wouldn't be that big since scan_block() is pretty time consuming as it checks every value that looks like a pointer against a prio_tree.

BTW, is there a way to know whether the page is in use or on the free list? Is page_count(page) feasible?

Thanks.

Signature · 1 line
Catalin
#6 lore raw
Quoted 3 lines · Catalin Marinas

BTW, is there a way to know whether the page is in use or on the free list? Is page_count(page) feasible?

page_count() == 0, means that the page is not used. PageBuddy() means its on the freelist.

#7 lore raw
In reply to Dave Hansen
Quoted 10 lines · Dave Hansen
Quoted 9 lines · Catalin Marinas
Quoted 9 lines
(quoted diff) +9 -0
+     /* mem_map scanning */
+     for_each_online_node(i) {
+             struct page *page, *end;
+
+             page = NODE_MEM_MAP(i);
+             end  = page + NODE_DATA(i)->node_spanned_pages;
+
+             scan_block(page, end, NULL);
+     }

[...]

Quoted 13 lines

One completely unoptimized thing you can do which will scan a 'struct page' at a time is this:

	for_each_online_node(i) {
		unsigned long pfn;
		for (pfn = node_start_pfn(i); pfn < node_end_pfn(i); pfn++) {
			struct page *page;
			if (!pfn_valid(pfn))
				continue;
			page = pfn_to_page(pfn);
			scan_block(page, page+1, NULL);
		}
	}

It seems that node_start_pfn() isn't present on all the architectures. I ended up with something like below:

+ /* struct page scanning for each node */
+ for_each_online_node(i) {
+ pg_data_t *pgdat = NODE_DATA(i);
+ unsigned long start_pfn = pgdat->node_start_pfn;
+ unsigned long end_pfn = start_pfn +
+ pgdat->node_spanned_pages - 1;
+ unsigned long pfn;
+
+ for (pfn = start_pfn; pfn < end_pfn; pfn++) {
+ struct page *page;
+
+ if (!pfn_valid(pfn))
+ continue;
+ page = pfn_to_page(pfn);
+ /* only scan if page is in use */
+ if (page_count(page) == 0)
+ continue;
+ scan_block(page, page + 1, NULL);
+ }
+ }

Are the pgdat->node_start_pfn and pgdat->node_spanned_pages always valid? Thanks.

Signature · 1 line
Catalin
#8 lore raw
Quoted 13 lines · Catalin Marinas

It seems that node_start_pfn() isn't present on all the architectures. I ended up with something like below:

(quoted diff) +10 -0
+	/* struct page scanning for each node */
+	for_each_online_node(i) {
+		pg_data_t *pgdat = NODE_DATA(i);
+		unsigned long start_pfn = pgdat->node_start_pfn;
+		unsigned long end_pfn = start_pfn +
+			pgdat->node_spanned_pages - 1;
+		unsigned long pfn;
+
+		for (pfn = start_pfn; pfn < end_pfn; pfn++) {
+			struct page *page;

Review

Quoted 10 lines
(quoted diff) +10 -0
+
+			if (!pfn_valid(pfn))
+				continue;
+			page = pfn_to_page(pfn);
+			/* only scan if page is in use */
+			if (page_count(page) == 0)
+				continue;
+			scan_block(page, page + 1, NULL);
+		}
+	}

What does scan_block() actually scan? Is that second argument inclusive?

I think you will miss scanning the contents of the last 'struct page' if you do it this way because of the -1 you do to end_pfn.

Quoted 2 lines

Are the pgdat->node_start_pfn and pgdat->node_spanned_pages always valid? Thanks.

The variables themselves? I'm sure there's a window in early boot where they aren't valid, but other than that they should be OK unless you're int the middle of a hotplug operation.

See pgdat_resize_(un)lock() in include/linux/memory_hotplug.h.

-- Dave

#9 lore raw
Quoted 19 lines · Dave Hansen
Quoted 18 lines · Catalin Marinas

It seems that node_start_pfn() isn't present on all the architectures. I ended up with something like below:

(quoted diff) +15 -0
+	/* struct page scanning for each node */
+	for_each_online_node(i) {
+		pg_data_t *pgdat = NODE_DATA(i);
+		unsigned long start_pfn = pgdat->node_start_pfn;
+		unsigned long end_pfn = start_pfn +
+			pgdat->node_spanned_pages - 1;
+		unsigned long pfn;
+
+		for (pfn = start_pfn; pfn < end_pfn; pfn++) {
+			struct page *page;
+
+			if (!pfn_valid(pfn))
+				continue;
+			page = pfn_to_page(pfn);
+			/* only scan if page is in use */
Quoted 8 lines

Review

Quoted 5 lines
(quoted diff) +5 -0
+			if (page_count(page) == 0)
+				continue;
+			scan_block(page, page + 1, NULL);
+		}
+	}

What does scan_block() actually scan? Is that second argument inclusive?

No, it's exclusive. In the above case, it just scans a struct page at the given pointer.

Quoted 2 lines

I think you will miss scanning the contents of the last 'struct page' if you do it this way because of the -1 you do to end_pfn.

Fixed.

Quoted 8 lines
Quoted 2 lines

Are the pgdat->node_start_pfn and pgdat->node_spanned_pages always valid? Thanks.

The variables themselves? I'm sure there's a window in early boot where they aren't valid, but other than that they should be OK unless you're int the middle of a hotplug operation.

See pgdat_resize_(un)lock() in include/linux/memory_hotplug.h.

I wouldn't hold a lock for that long. It's not really critical to scan all the page structures at a time as there are subsequent scans as well, so some can be missed.

Signature · 1 line
Catalin
#10 lore raw
Quoted 7 lines · Catalin Marinas
Quoted 7 lines
Quoted 2 lines

Are the pgdat->node_start_pfn and pgdat->node_spanned_pages always valid? Thanks.

The variables themselves? I'm sure there's a window in early boot where they aren't valid, but other than that they should be OK unless you're int the middle of a hotplug operation.

Quoted 5 lines
Quoted 1 line

See pgdat_resize_(un)lock() in include/linux/memory_hotplug.h.

I wouldn't hold a lock for that long. It's not really critical to scan all the page structures at a time as there are subsequent scans as well, so some can be missed.

I think you should be more worried about consistency rather than missing entries. Take these two lines of code:

	start_pfn = node->node_start_pfn;
	/* hotplug occurs here */
	end_pfn = start_pfn + node->node_spanned_pages;

What if someone comes in and adds memory to the node, at the beginning of the node, after you have calculated start_pfn? Try to think of what value you'll get for end_pfn and whether it is consistent and was *ever* valid at all. Would that oops the kernel?

-- Dave

#11 lore raw
Quoted 17 lines · Dave Hansen
Quoted 7 lines · Catalin Marinas
Quoted 7 lines
Quoted 2 lines

Are the pgdat->node_start_pfn and pgdat->node_spanned_pages always valid? Thanks.

The variables themselves? I'm sure there's a window in early boot where they aren't valid, but other than that they should be OK unless you're int the middle of a hotplug operation.

Quoted 5 lines
Quoted 1 line

See pgdat_resize_(un)lock() in include/linux/memory_hotplug.h.

I wouldn't hold a lock for that long. It's not really critical to scan all the page structures at a time as there are subsequent scans as well, so some can be missed.

I think you should be more worried about consistency rather than missing entries. Take these two lines of code:

Quoted 8 lines
	start_pfn = node->node_start_pfn;
	/* hotplug occurs here */
	end_pfn = start_pfn + node->node_spanned_pages;

What if someone comes in and adds memory to the node, at the beginning of the node, after you have calculated start_pfn? Try to think of what value you'll get for end_pfn and whether it is consistent and was *ever* valid at all. Would that oops the kernel?

I assume pfn_valid() should handle this and kmemleak wouldn't scan the page, unless we need locks around pfn_valid as well but I haven't seen any used in the kernel.

Signature · 1 line
Catalin
#12 lore raw
Quoted 7 lines · Catalin Marinas
Quoted 7 lines

I think you should be more worried about consistency rather than missing entries. Take these two lines of code:

      start_pfn = node->node_start_pfn;
      /* hotplug occurs here */
      end_pfn = start_pfn + node->node_spanned_pages;
Quoted 8 lines
Quoted 4 lines

What if someone comes in and adds memory to the node, at the beginning of the node, after you have calculated start_pfn? Try to think of what value you'll get for end_pfn and whether it is consistent and was *ever* valid at all. Would that oops the kernel?

I assume pfn_valid() should handle this and kmemleak wouldn't scan the page, unless we need locks around pfn_valid as well but I haven't seen any used in the kernel.

You assume incorrectly. :(

Take my above example, and assume that you have two nodes which are right next to each other. You might run over the end of one node and into the next one. Your pages will be pfn_valid() but you will be on the wrong node.

Please take a look at those locks that I mentioned. Notice that they are lock the pgdat *span*, not the validity of pages inside the pgdat. Your code deals with what pages the pgdats *span* and thus needs that lock. Notice that my example also had to do with those two lines of code incorrectly guessing the pgdat's *span*.

We recently went to some pain to make sure that the software suspend code (which walks pgdat ranges as well) worked with memory hotplug. There really isn't that much code around that actually cares at runtime about which physical areas a particular node or zone spans. Yours is a rarity and will require some caution.

You could probably also use the memory hotplug mutex found here:

https://lists.linux-foundation.org/pipermail/linux-pm/2008-November/018884.html

But I'm not sure where those patches have gone. Hmmm. Pavel?

-- Dave

#13 lore raw

Hi!

Quoted 36 lines · Dave Hansen
Quoted 7 lines · Catalin Marinas
Quoted 7 lines

I think you should be more worried about consistency rather than missing entries. Take these two lines of code:

      start_pfn = node->node_start_pfn;
      /* hotplug occurs here */
      end_pfn = start_pfn + node->node_spanned_pages;
Quoted 8 lines
Quoted 4 lines

What if someone comes in and adds memory to the node, at the beginning of the node, after you have calculated start_pfn? Try to think of what value you'll get for end_pfn and whether it is consistent and was *ever* valid at all. Would that oops the kernel?

I assume pfn_valid() should handle this and kmemleak wouldn't scan the page, unless we need locks around pfn_valid as well but I haven't seen any used in the kernel.

You assume incorrectly. :(

Take my above example, and assume that you have two nodes which are right next to each other. You might run over the end of one node and into the next one. Your pages will be pfn_valid() but you will be on the wrong node.

Please take a look at those locks that I mentioned. Notice that they are lock the pgdat *span*, not the validity of pages inside the pgdat. Your code deals with what pages the pgdats *span* and thus needs that lock. Notice that my example also had to do with those two lines of code incorrectly guessing the pgdat's *span*.

We recently went to some pain to make sure that the software suspend code (which walks pgdat ranges as well) worked with memory hotplug. There really isn't that much code around that actually cares at runtime about which physical areas a particular node or zone spans. Yours is a rarity and will require some caution.

Quoted 5 lines

You could probably also use the memory hotplug mutex found here:

https://lists.linux-foundation.org/pipermail/linux-pm/2008-November/018884.html

But I'm not sure where those patches have gone. Hmmm. Pavel?

I don't think they were applied. They probably should... Rafael was about to look into that, but he lost the patch pointer.
Pavel

Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
#17 lore raw
In reply to Dave Hansen

Hi Dave,

Quoted 17 lines · Catalin Marinas
Quoted 7 lines
Quoted 7 lines

I think you should be more worried about consistency rather than missing entries. Take these two lines of code:

      start_pfn = node->node_start_pfn;
      /* hotplug occurs here */
      end_pfn = start_pfn + node->node_spanned_pages;
Quoted 8 lines
Quoted 4 lines

What if someone comes in and adds memory to the node, at the beginning of the node, after you have calculated start_pfn? Try to think of what value you'll get for end_pfn and whether it is consistent and was *ever* valid at all. Would that oops the kernel?

I assume pfn_valid() should handle this and kmemleak wouldn't scan the page, unless we need locks around pfn_valid as well but I haven't seen any used in the kernel.

Quoted 6 lines

You assume incorrectly. :(

Take my above example, and assume that you have two nodes which are right next to each other. You might run over the end of one node and into the next one. Your pages will be pfn_valid() but you will be on the wrong node.

OK, thanks for taking the time to explain this. I currently added a dependency on !MEMORY_HOTPLUG for kmemleak since holding the lock while traversing the page structures is not really feasible.

Quoted 3 lines

You could probably also use the memory hotplug mutex found here:

https://lists.linux-foundation.org/pipermail/linux-pm/2008-November/018884.html

That would be a better option for kmemleak as well.

Signature · 1 line
Catalin
#14 lore raw
In reply to Pavel Machek
Quoted 39 lines · Pavel Machek

Hi!

Quoted 36 lines · Dave Hansen
Quoted 7 lines · Catalin Marinas
Quoted 7 lines

I think you should be more worried about consistency rather than missing entries. Take these two lines of code:

      start_pfn = node->node_start_pfn;
      /* hotplug occurs here */
      end_pfn = start_pfn + node->node_spanned_pages;
Quoted 8 lines
Quoted 4 lines

What if someone comes in and adds memory to the node, at the beginning of the node, after you have calculated start_pfn? Try to think of what value you'll get for end_pfn and whether it is consistent and was *ever* valid at all. Would that oops the kernel?

I assume pfn_valid() should handle this and kmemleak wouldn't scan the page, unless we need locks around pfn_valid as well but I haven't seen any used in the kernel.

You assume incorrectly. :(

Take my above example, and assume that you have two nodes which are right next to each other. You might run over the end of one node and into the next one. Your pages will be pfn_valid() but you will be on the wrong node.

Please take a look at those locks that I mentioned. Notice that they are lock the pgdat *span*, not the validity of pages inside the pgdat. Your code deals with what pages the pgdats *span* and thus needs that lock. Notice that my example also had to do with those two lines of code incorrectly guessing the pgdat's *span*.

We recently went to some pain to make sure that the software suspend code (which walks pgdat ranges as well) worked with memory hotplug. There really isn't that much code around that actually cares at runtime about which physical areas a particular node or zone spans. Yours is a rarity and will require some caution.

Quoted 8 lines
Quoted 5 lines

You could probably also use the memory hotplug mutex found here:

https://lists.linux-foundation.org/pipermail/linux-pm/2008-November/018884.html

But I'm not sure where those patches have gone. Hmmm. Pavel?

I don't think they were applied. They probably should... Rafael was about to look into that, but he lost the patch pointer.

Yes. In fact, sending the patch again to me would be appreciated.

Thanks,
Rafael

#15 lore raw
Quoted 42 lines · On Mon, 16 Mar 2009 23:04:27 +0100 "Rafael J. Wysocki" <[email protected]> wrote:
Quoted 41 lines · Pavel Machek

Hi!

Quoted 38 lines · Dave Hansen
Quoted 7 lines · Catalin Marinas
Quoted 7 lines

I think you should be more worried about consistency rather than missing entries. Take these two lines of code:

      start_pfn = node->node_start_pfn;
      /* hotplug occurs here */
      end_pfn = start_pfn + node->node_spanned_pages;
Quoted 8 lines
Quoted 4 lines

What if someone comes in and adds memory to the node, at the beginning of the node, after you have calculated start_pfn? Try to think of what value you'll get for end_pfn and whether it is consistent and was *ever* valid at all. Would that oops the kernel?

I assume pfn_valid() should handle this and kmemleak wouldn't scan the page, unless we need locks around pfn_valid as well but I haven't seen any used in the kernel.

You assume incorrectly. :(

Take my above example, and assume that you have two nodes which are right next to each other. You might run over the end of one node and into the next one. Your pages will be pfn_valid() but you will be on the wrong node.

Please take a look at those locks that I mentioned. Notice that they are lock the pgdat *span*, not the validity of pages inside the pgdat. Your code deals with what pages the pgdats *span* and thus needs that lock. Notice that my example also had to do with those two lines of code incorrectly guessing the pgdat's *span*.

We recently went to some pain to make sure that the software suspend code (which walks pgdat ranges as well) worked with memory hotplug. There really isn't that much code around that actually cares at runtime about which physical areas a particular node or zone spans. Yours is a rarity and will require some caution.

You could probably also use the memory hotplug mutex found here:

Quoted 9 lines
Quoted 6 lines
Quoted 3 lines

https://lists.linux-foundation.org/pipermail/linux-pm/2008-November/018884.html

But I'm not sure where those patches have gone. Hmmm. Pavel?

I don't think they were applied. They probably should... Rafael was about to look into that, but he lost the patch pointer.

Yes. In fact, sending the patch again to me would be appreciated.

Ah...but as I wrote, for me, testing this is difficult.
Could some expert of hibernation update and post this again ? For memoyr hotplug. it seems enough that sending the patch witch CC: to linux-mm and including "Memory Hotplug" in the subject.

Thanks,
-Kame

#16 lore raw
Quoted 52 lines · KAMEZAWA Hiroyuki
Quoted 49 lines · On Mon, 16 Mar 2009 23:04:27 +0100 "Rafael J. Wysocki" <[email protected]> wrote:
Quoted 47 lines · Pavel Machek

Hi!

Quoted 36 lines · Dave Hansen
Quoted 7 lines · Catalin Marinas
Quoted 7 lines

I think you should be more worried about consistency rather than missing entries. Take these two lines of code:

      start_pfn = node->node_start_pfn;
      /* hotplug occurs here */
      end_pfn = start_pfn + node->node_spanned_pages;
Quoted 8 lines
Quoted 4 lines

What if someone comes in and adds memory to the node, at the beginning of the node, after you have calculated start_pfn? Try to think of what value you'll get for end_pfn and whether it is consistent and was *ever* valid at all. Would that oops the kernel?

I assume pfn_valid() should handle this and kmemleak wouldn't scan the page, unless we need locks around pfn_valid as well but I haven't seen any used in the kernel.

You assume incorrectly. :(

Take my above example, and assume that you have two nodes which are right next to each other. You might run over the end of one node and into the next one. Your pages will be pfn_valid() but you will be on the wrong node.

Please take a look at those locks that I mentioned. Notice that they are lock the pgdat *span*, not the validity of pages inside the pgdat. Your code deals with what pages the pgdats *span* and thus needs that lock. Notice that my example also had to do with those two lines of code incorrectly guessing the pgdat's *span*.

We recently went to some pain to make sure that the software suspend code (which walks pgdat ranges as well) worked with memory hotplug. There really isn't that much code around that actually cares at runtime about which physical areas a particular node or zone spans. Yours is a rarity and will require some caution.

Quoted 5 lines

You could probably also use the memory hotplug mutex found here:

https://lists.linux-foundation.org/pipermail/linux-pm/2008-November/018884.html

But I'm not sure where those patches have gone. Hmmm. Pavel?

I don't think they were applied. They probably should... Rafael was about to look into that, but he lost the patch pointer.

Quoted 8 lines
Quoted 2 lines

Yes. In fact, sending the patch again to me would be appreciated.

Ah...but as I wrote, for me, testing this is difficult.
Could some expert of hibernation update and post this again ? For memoyr hotplug. it seems enough that sending the patch witch CC: to linux-mm and including "Memory Hotplug" in the subject.

Well, testing this is probably difficult for everyone, as hotplug-capable machines are rare/expensive.

Can youjust retransmit the patch to rafael?

Quoted 1 line
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html