Skip to contentSkip to conversation

resume from s2ram regression (bisected to ftrace...)

DISCUSSION Marcin Slusarz 3 participants 11 messages Aug 10, 2008 – Aug 12, 2008

lore
Share & tools
lore
#0 lore raw

Hi

Few days ago I discovered that resume from suspend to ram does not work anymore on my 2.6.27-rc2 (actually 796aadeb1b2db9b5d463946766c5bbfd7717158c) gentoo box. My computer just boots on resume. Today I had enough time to bisect it and found (after 1 panic and 1 build breakage) out "the reason":

$ git bisect good
d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b is first bad commit
commit d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b
Author: Steven Rostedt <[email protected]>
Date:   Mon May 12 21:20:54 2008 +0200

ftrace: fix dynamic ftrace selftest

With the adding of the configuration changes in the Makefile to prevent tracing of functions in the ftrace code, all tracing of all the ftrace code has been removed. Unfortunately, one of the selftests, relied on a function to be traced. With the new change, the function was no longer traced and the test failed.

This patch separates out the test function into its own file so that we can add the "-pg" flag to the compilation of that function and the adding of the mcount call to that function.

:040000 040000 0fe901f75979b8a110e58133e1d31e304d4214d5 6a63e96d5afe318a2f6f541cc48a8cf224088611 M kernel

$ git bisect log
git-bisect start
# bad: [0967d61ea0d8e8a7826bd8949cd93dd1e829ac55] Linux 2.6.27-rc2
git-bisect bad 0967d61ea0d8e8a7826bd8949cd93dd1e829ac55
# good: [bce7f793daec3e65ec5c5705d2457b81fe7b5725] Linux 2.6.26
git-bisect good bce7f793daec3e65ec5c5705d2457b81fe7b5725
# bad: [49997d75152b3d23c53b0fa730599f2f74c92c65] Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/torvalds/linux-2.6
git-bisect bad 49997d75152b3d23c53b0fa730599f2f74c92c65
# bad: [43d2548bb2ef7e6d753f91468a746784041e522d] Merge commit '85082fd7cbe3173198aac0eb5e85ab1edcc6352c' into test-build
git-bisect bad 43d2548bb2ef7e6d753f91468a746784041e522d
# bad: [5806b81ac1c0c52665b91723fd4146a4f86e386b] Merge branch 'auto-ftrace-next' into tracing/for-linus
git-bisect bad 5806b81ac1c0c52665b91723fd4146a4f86e386b
# good: [6236af82d8a989e150a02800c210eb61cb1e17be] Merge branch 'x86/fixmap' into x86/devel
git-bisect good 6236af82d8a989e150a02800c210eb61cb1e17be
# good: [32b23e9a7331fce57eb0af52e19e8409fdef831b] x86: max_low_pfn_mapped fix #4
git-bisect good 32b23e9a7331fce57eb0af52e19e8409fdef831b
# bad: [e0fd5c2fa188311667267c02a702ae699a9fc2bd] mmiotrace: do not print bogus pid for maps either
git-bisect bad e0fd5c2fa188311667267c02a702ae699a9fc2bd
# good: [5429db2d26a59903c81a4f6c6dae7eb9daaea5fc] ftrace: fix wakeup callback
git-bisect good 5429db2d26a59903c81a4f6c6dae7eb9daaea5fc
# bad: [2bb6f8d6389cbfadd657e7dc069f6986abf35e4f] ftrace: use raw_smp_processor_id for mcount functions
git-bisect bad 2bb6f8d6389cbfadd657e7dc069f6986abf35e4f
# bad: [8487c23765b6e0444ec6b5f1530766d63fe68e35] ftrace: allow trace_pipe to block on all reads
git-bisect bad 8487c23765b6e0444ec6b5f1530766d63fe68e35
# bad: [1d09daa55d2e9bab7e7d30f0d05e5a7bc60b2a4a] ftrace: use Makefile to remove tracing from lockdep
git-bisect bad 1d09daa55d2e9bab7e7d30f0d05e5a7bc60b2a4a
# bad: [d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b] ftrace: fix dynamic ftrace selftest
git-bisect bad d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b
# good: [8f96da02c14d722ad9a3713cd7273ce28c9036ad] ftrace: remove wakeup from function trace
git-bisect good 8f96da02c14d722ad9a3713cd7273ce28c9036ad
# good: [06fa75ab566c50e01bfd7b055bde85cf9b1bc98a] ftrace: add TRACE_STACK and TRACE_SPECIAL to selftest validation
git-bisect good 06fa75ab566c50e01bfd7b055bde85cf9b1bc98a

This commit looks completely unrelated to s2ram, but funny thing happened - when I reverted it on top of 796aadeb1b2db9b5d463946766c5bbfd7717158c (2.6.27-rc2+), suspend to ram started to work...

You can find my .config, successful s2ram dmesg and lspci at
http://www.kadu.net/~joi/kernel/2008.08.10/

Marcin

#1 lore raw
Quoted 7 lines · Marcin Slusarz

Hi

Few days ago I discovered that resume from suspend to ram does not work anymore on my 2.6.27-rc2 (actually 796aadeb1b2db9b5d463946766c5bbfd7717158c) gentoo box. My computer just boots on resume. Today I had enough time to bisect it and found (after 1 panic and 1 build breakage) out "the reason":

Quoted 7 lines
$ git bisect good
d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b is first bad commit
commit d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b
Author: Steven Rostedt <[email protected]>
Date:   Mon May 12 21:20:54 2008 +0200

ftrace: fix dynamic ftrace selftest

Does it still happen if:
- CONFIG_FTRACE is unset
- CONFIG_FTRACE is set, but CONFIG_FTRACE_STARTUP_TEST is unset?

Rafael

#2 lore raw
Quoted 14 lines · Rafael J. Wysocki
Quoted 13 lines · Marcin Slusarz

Hi

Few days ago I discovered that resume from suspend to ram does not work anymore on my 2.6.27-rc2 (actually 796aadeb1b2db9b5d463946766c5bbfd7717158c) gentoo box. My computer just boots on resume. Today I had enough time to bisect it and found (after 1 panic and 1 build breakage) out "the reason":

$ git bisect good
d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b is first bad commit
commit d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b
Author: Steven Rostedt <[email protected]>
Date:   Mon May 12 21:20:54 2008 +0200
Quoted 4 lines
Quoted 1 line

ftrace: fix dynamic ftrace selftest

Does it still happen if:
- CONFIG_FTRACE is unset

no

Quoted 1 line

- CONFIG_FTRACE is set, but CONFIG_FTRACE_STARTUP_TEST is unset?

yes

Marcin

#3 lore raw
In reply to Rafael J. Wysocki
Quoted 14 lines · Rafael J. Wysocki
Quoted 13 lines · Marcin Slusarz

Hi

Few days ago I discovered that resume from suspend to ram does not work anymore on my 2.6.27-rc2 (actually 796aadeb1b2db9b5d463946766c5bbfd7717158c) gentoo box. My computer just boots on resume. Today I had enough time to bisect it and found (after 1 panic and 1 build breakage) out "the reason":

$ git bisect good
d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b is first bad commit
commit d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b
Author: Steven Rostedt <[email protected]>
Date:   Mon May 12 21:20:54 2008 +0200
Quoted 4 lines
Quoted 1 line

ftrace: fix dynamic ftrace selftest

Does it still happen if:
- CONFIG_FTRACE is unset

no

Quoted 1 line

- CONFIG_FTRACE is set, but CONFIG_FTRACE_STARTUP_TEST is unset?

yes

Marcin

#4 lore raw
Quoted 17 lines · Marcin Slusarz
Quoted 16 lines · Rafael J. Wysocki
Quoted 14 lines · Marcin Slusarz

Hi

Few days ago I discovered that resume from suspend to ram does not work anymore on my 2.6.27-rc2 (actually 796aadeb1b2db9b5d463946766c5bbfd7717158c) gentoo box. My computer just boots on resume. Today I had enough time to bisect it and found (after 1 panic and 1 build breakage) out "the reason":

$ git bisect good
d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b is first bad commit
commit d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b
Author: Steven Rostedt <[email protected]>
Date:   Mon May 12 21:20:54 2008 +0200

ftrace: fix dynamic ftrace selftest

Quoted 8 lines
Quoted 2 lines

Does it still happen if:
- CONFIG_FTRACE is unset

no

Quoted 1 line

- CONFIG_FTRACE is set, but CONFIG_FTRACE_STARTUP_TEST is unset?

yes

Thanks for testing.

Well, ftrace is a new feature in 2.6.27, IIRC, so this is not a regression in the strict sense.

Still, of couse, there is a bug in ftrace that needs to be fixed. Steven?

#5 lore raw
Quoted 32 lines · Rafael J. Wysocki
Quoted 22 lines · Marcin Slusarz
Quoted 15 lines · Rafael J. Wysocki
Quoted 13 lines · Marcin Slusarz

Hi

Few days ago I discovered that resume from suspend to ram does not work anymore on my 2.6.27-rc2 (actually 796aadeb1b2db9b5d463946766c5bbfd7717158c) gentoo box. My computer just boots on resume. Today I had enough time to bisect it and found (after 1 panic and 1 build breakage) out "the reason":

$ git bisect good
d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b is first bad commit
commit d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b
Author: Steven Rostedt <[email protected]>
Date:   Mon May 12 21:20:54 2008 +0200
Quoted 5 lines
Quoted 2 lines

ftrace: fix dynamic ftrace selftest

Does it still happen if:
- CONFIG_FTRACE is unset

Quoted 7 lines

no

Quoted 2 lines

- CONFIG_FTRACE is set, but CONFIG_FTRACE_STARTUP_TEST is unset?

yes

Quoted 7 lines

Thanks for testing.

Well, ftrace is a new feature in 2.6.27, IIRC, so this is not a regression in the strict sense.

Still, of couse, there is a bug in ftrace that needs to be fixed. Steven?

Sorry for the late response.

Red Hat changed its mail servers, and my mail was in limbo for a while.

I'm not doubting that FTRACE enabled breaks suspend to ram, but I find it highly unlikely that the commit you posted was the culprit. Especially since it still breaks when STARTUP_TEST is disabled. That change was a STARTUP_TEST change only. Perhaps it just moved things around enough to cause your issues.

But you say that if you take Linus's latest git tree, and revert only this change, it works??

Hmm, this is very strange.

-- Steve

#6 lore raw
In reply to Rafael J. Wysocki
Quoted 32 lines · Rafael J. Wysocki
Quoted 22 lines · Marcin Slusarz
Quoted 15 lines · Rafael J. Wysocki
Quoted 13 lines · Marcin Slusarz

Hi

Few days ago I discovered that resume from suspend to ram does not work anymore on my 2.6.27-rc2 (actually 796aadeb1b2db9b5d463946766c5bbfd7717158c) gentoo box. My computer just boots on resume. Today I had enough time to bisect it and found (after 1 panic and 1 build breakage) out "the reason":

$ git bisect good
d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b is first bad commit
commit d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b
Author: Steven Rostedt <[email protected]>
Date:   Mon May 12 21:20:54 2008 +0200
Quoted 5 lines
Quoted 2 lines

ftrace: fix dynamic ftrace selftest

Does it still happen if:
- CONFIG_FTRACE is unset

Quoted 7 lines

no

Quoted 2 lines

- CONFIG_FTRACE is set, but CONFIG_FTRACE_STARTUP_TEST is unset?

yes

Quoted 7 lines

Thanks for testing.

Well, ftrace is a new feature in 2.6.27, IIRC, so this is not a regression in the strict sense.

Still, of couse, there is a bug in ftrace that needs to be fixed. Steven?

Sorry for the late response.

Red Hat changed its mail servers, and my mail was in limbo for a while.

I'm not doubting that FTRACE enabled breaks suspend to ram, but I find it highly unlikely that the commit you posted was the culprit. Especially since it still breaks when STARTUP_TEST is disabled. That change was a STARTUP_TEST change only. Perhaps it just moved things around enough to cause your issues.

But you say that if you take Linus's latest git tree, and revert only this change, it works??

Hmm, this is very strange.

-- Steve

#7 lore raw
Quoted 48 lines · Steven Rostedt
Quoted 35 lines · Rafael J. Wysocki
Quoted 25 lines · Marcin Slusarz
Quoted 18 lines · Rafael J. Wysocki
Quoted 16 lines · Marcin Slusarz

Hi

Few days ago I discovered that resume from suspend to ram does not work anymore
on my 2.6.27-rc2 (actually 796aadeb1b2db9b5d463946766c5bbfd7717158c) gentoo box.
My computer just boots on resume. Today I had enough time to bisect it and found
(after 1 panic and 1 build breakage) out "the reason":

$ git bisect good
d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b is first bad commit
commit d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b
Author: Steven Rostedt <[email protected]>
Date:   Mon May 12 21:20:54 2008 +0200
Quoted 5 lines
Quoted 2 lines

ftrace: fix dynamic ftrace selftest

Does it still happen if:
- CONFIG_FTRACE is unset

Quoted 7 lines

no

Quoted 2 lines

- CONFIG_FTRACE is set, but CONFIG_FTRACE_STARTUP_TEST is unset?

yes

Quoted 8 lines

Thanks for testing.

Well, ftrace is a new feature in 2.6.27, IIRC, so this is not a regression in
the strict sense.

Still, of couse, there is a bug in ftrace that needs to be fixed. Steven?

Sorry for the late response.

Red Hat changed its mail servers, and my mail was in limbo for a while.

Quoted 8 lines

I'm not doubting that FTRACE enabled breaks suspend to ram, but I find it highly unlikely that the commit you posted was the culprit. Especially since it still breaks when STARTUP_TEST is disabled. That change was a STARTUP_TEST change only. Perhaps it just moved things around enough to cause your issues.

But you say that if you take Linus's latest git tree, and revert only this change, it works??

Yes.

Quoted 1 line

Hmm, this is very strange.

I agree. Any idea how to debug it?

Marcin

#8 lore raw
In reply to Steven Rostedt
Quoted 48 lines · Steven Rostedt
Quoted 35 lines · Rafael J. Wysocki
Quoted 25 lines · Marcin Slusarz
Quoted 18 lines · Rafael J. Wysocki
Quoted 16 lines · Marcin Slusarz

Hi

Few days ago I discovered that resume from suspend to ram does not work anymore
on my 2.6.27-rc2 (actually 796aadeb1b2db9b5d463946766c5bbfd7717158c) gentoo box.
My computer just boots on resume. Today I had enough time to bisect it and found
(after 1 panic and 1 build breakage) out "the reason":

$ git bisect good
d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b is first bad commit
commit d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b
Author: Steven Rostedt <[email protected]>
Date:   Mon May 12 21:20:54 2008 +0200
Quoted 5 lines
Quoted 2 lines

ftrace: fix dynamic ftrace selftest

Does it still happen if:
- CONFIG_FTRACE is unset

Quoted 7 lines

no

Quoted 2 lines

- CONFIG_FTRACE is set, but CONFIG_FTRACE_STARTUP_TEST is unset?

yes

Quoted 8 lines

Thanks for testing.

Well, ftrace is a new feature in 2.6.27, IIRC, so this is not a regression in
the strict sense.

Still, of couse, there is a bug in ftrace that needs to be fixed. Steven?

Sorry for the late response.

Red Hat changed its mail servers, and my mail was in limbo for a while.

Quoted 8 lines

I'm not doubting that FTRACE enabled breaks suspend to ram, but I find it highly unlikely that the commit you posted was the culprit. Especially since it still breaks when STARTUP_TEST is disabled. That change was a STARTUP_TEST change only. Perhaps it just moved things around enough to cause your issues.

But you say that if you take Linus's latest git tree, and revert only this change, it works??

Yes.

Quoted 1 line

Hmm, this is very strange.

I agree. Any idea how to debug it?

Marcin

#9 lore raw
In reply to Marcin Slusarz
Quoted 17 lines · Marcin Slusarz
Quoted 16 lines · Rafael J. Wysocki
Quoted 14 lines · Marcin Slusarz

Hi

Few days ago I discovered that resume from suspend to ram does not work anymore on my 2.6.27-rc2 (actually 796aadeb1b2db9b5d463946766c5bbfd7717158c) gentoo box. My computer just boots on resume. Today I had enough time to bisect it and found (after 1 panic and 1 build breakage) out "the reason":

$ git bisect good
d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b is first bad commit
commit d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b
Author: Steven Rostedt <[email protected]>
Date:   Mon May 12 21:20:54 2008 +0200

ftrace: fix dynamic ftrace selftest

Quoted 8 lines
Quoted 2 lines

Does it still happen if:
- CONFIG_FTRACE is unset

no

Quoted 1 line

- CONFIG_FTRACE is set, but CONFIG_FTRACE_STARTUP_TEST is unset?

yes

Thanks for testing.

Well, ftrace is a new feature in 2.6.27, IIRC, so this is not a regression in the strict sense.

Still, of couse, there is a bug in ftrace that needs to be fixed. Steven?

#10 lore raw
In reply to Marcin Slusarz
Quoted 7 lines · Marcin Slusarz

Hi

Few days ago I discovered that resume from suspend to ram does not work anymore on my 2.6.27-rc2 (actually 796aadeb1b2db9b5d463946766c5bbfd7717158c) gentoo box. My computer just boots on resume. Today I had enough time to bisect it and found (after 1 panic and 1 build breakage) out "the reason":

Quoted 7 lines
$ git bisect good
d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b is first bad commit
commit d05cdb25d80f06f77aa6bddb53cd1390d4d91a0b
Author: Steven Rostedt <[email protected]>
Date:   Mon May 12 21:20:54 2008 +0200

ftrace: fix dynamic ftrace selftest

Does it still happen if:
- CONFIG_FTRACE is unset
- CONFIG_FTRACE is set, but CONFIG_FTRACE_STARTUP_TEST is unset?

Rafael