help to bisect

DISCUSSION James 2 participants 7 messages May 18, 2011 – May 19, 2011

Share & tools

Borislav Petkov

2011-05-18 07:22 UTC #1 lore raw

Quoted 10 lines · James

How do I build v2.6.36 so I can test it mark it good so I can bisect?
Here is what I want to do:
$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.37.y.git
$ cd linux-2.6.37.y
$ git bisect start
$ git bisect good v2.6.36
$ git bisect bad v2.6.37.6
Do I do "git checkout v2.6.36"?

Yes, but you should do it first, before starting bisection. After you've verified .36 is fine, then you do the last three commands:

Quoted 3 lines

$ git bisect start
$ git bisect good v2.6.36
$ git bisect bad v2.6.37.6

assuming you have verified also that .37.6 is bad, but I guess so, otherwise you wouldn't be bisecting in the first place :)

Also, it might make sense to check whether .37 is fine too before bisecting. If so, you save yourself quite a bit of bisection runs by marking .37 as good instead of .36.

HTH.

Signature · 2 lines

Regards/Gruss,
    Boris.

James

author

2011-05-18 20:41 UTC #2 lore raw

Quoted 17 lines · Borislav Petkov

Quoted 10 lines · James
How do I build v2.6.36 so I can test it mark it good so I can bisect?
Here is what I want to do:
$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.37.y.git
$ cd linux-2.6.37.y
$ git bisect start
$ git bisect good v2.6.36
$ git bisect bad v2.6.37.6
Do I do "git checkout v2.6.36"?
Yes, but you should do it first, before starting bisection. After you've verified .36 is fine, then you do the last three commands:
Quoted 3 lines
$ git bisect start
$ git bisect good v2.6.36
$ git bisect bad v2.6.37.6

Quoted 9 lines

assuming you have verified also that .37.6 is bad, but I guess so, otherwise you wouldn't be bisecting in the first place :)
Also, it might make sense to check whether .37 is fine too before bisecting. If so, you save yourself quite a bit of bisection runs by marking .37 as good instead of .36.
HTH.

Here is what I did:

$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.37.y.git
$ cd linux-2.6.37.y
$ git checkout v2.6.36
build and boot kernel
$ cd linux-2.6.37.y
$ git bisect start
$ git bisect good v2.6.36
$ git bisect bad v2.6.37.6

build and boot kernel
git bisect good|bad
repeat

3044100e58c84e133791c8b60a2f5bef69d732e4 is the first bad commit.

I did:

$ git show 3044100e58c84e133791c8b60a2f5bef69d732e4

commit 3044100e58c84e133791c8b60a2f5bef69d732e4
Merge: b515316 67e87f0
Author: Linus Torvalds <[email protected]>
Date: Thu Oct 21 18:52:11 2010 -0700

Merge branch 'core-memblock-for-linus' of git://git.kernel.org/pub/scm/linux

* 'core-memblock-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git
x86-64: Only set max_pfn_mapped to 512 MiB if we enter via head_64.S
xen: Cope with unmapped pages when initializing kernel pagetable memblock, bootmem: Round pfn properly for memory and reserved regions
memblock: Annotate memblock functions with __init_memblock
memblock: Allow memblock_init to be called early memblock/arm: Fix memblock_region_is_memory() typo
x86, memblock: Remove __memblock_x86_find_in_range_size()
memblock: Fix wraparound in find_region()
x86-32, memblock: Make add_highpages honor early reserved ranges x86, memblock: Fix crashkernel allocation
arm, memblock: Fix the sparsemem build
memblock: Fix section mismatch warnings
powerpc, memblock: Fix memblock API change fallout
memblock, microblaze: Fix memblock API change fallout
x86: Remove old bootmem code
x86, memblock: Use memblock_memory_size()/memblock_free_memory_size() to g
x86: Remove not used early_res code
x86, memblock: Replace e820_/_early string with memblock_
x86: Use memblock to replace early_res
x86, memblock: Use memblock_debug to control debug message print out
...

Fix up trivial conflicts in arch/x86/kernel/setup.c and kernel/Makefile

How do I find out what change causes the problem?
The problem causes my wireless card not to work.
Other people have similar cards that do work so the problem seems isolated to my hardware but kernel-2.6.36 works so I don't think my hardware is faulty.

Borislav Petkov

2011-05-18 21:17 UTC #3 lore raw

First of all, please hit "reply-to-all" next time so that all recipients can receive your mail and not find it by chance when looking thru the new lkml messages.

Quoted 6 lines · James

Here is what I did:

$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.37.y.git
$ cd linux-2.6.37.y
$ git checkout v2.6.36
build and boot kernel

what does that mean? Is .36 ok?

Quoted 4 lines

$ cd linux-2.6.37.y
$ git bisect start
$ git bisect good v2.6.36
$ git bisect bad v2.6.37.6

Also, is .37.6 bad?

Quoted 6 lines

build and boot kernel
git bisect good|bad
repeat
3044100e58c84e133791c8b60a2f5bef69d732e4 is the first bad commit.

Which is a merge commit and it means that you most likely made a mistake during bisection.

What is actually the problem you're experiencing with .37.6?

Quoted 2 lines

How do I find out what change causes the problem?
The problem causes my wireless card not to work.

Ah, here it is.

Quoted 3 lines

Other people have similar cards that do work so the problem seems isolated to my hardware but kernel-2.6.36 works so I don't think my hardware is faulty.

Which card is that? (adding linux-wireless)

Please, describe in a very detailed way how your problem incarnates
itself: dmesg, error messages, what exactly do you do to trigger it?

Also, can you test whether 38.6 works for ya - it could've been fixed in the meantime. You can also test .39 which will be released any minute now.

I think that should be enough for now.

Signature · 2 lines

Regards/Gruss,
    Boris.

James

author

2011-05-18 23:58 UTC #4 lore raw

Quoted 4 lines · Borislav Petkov

First of all, please hit "reply-to-all" next time so that all recipients can receive your mail and not find it by chance when looking thru the new lkml messages.

Quoted 8 lines

Quoted 6 lines · James
Here is what I did:
$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.37.y.git
$ cd linux-2.6.37.y
$ git checkout v2.6.36
build and boot kernel
what does that mean? Is .36 ok?

Yes, no problems.

Quoted 5 lines

Quoted 4 lines

$ cd linux-2.6.37.y
$ git bisect start
$ git bisect good v2.6.36
$ git bisect bad v2.6.37.6

Also, is .37.6 bad?

Yes, it is bad.

Quoted 7 lines

Quoted 5 lines
build and boot kernel
git bisect good|bad
repeat
3044100e58c84e133791c8b60a2f5bef69d732e4 is the first bad commit.
Which is a merge commit and it means that you most likely made a mistake during bisection.

Am I doing it right?
boot kernel, cd to git source, git bisect good or bad, compile new kernel and copy it to /boot, repeat

Quoted 4 lines

What is actually the problem you're experiencing with .37.6?
..

Quoted 8 lines

Quoted 2 lines
How do I find out what change causes the problem?
The problem causes my wireless card not to work.
Ah, here it is.
Quoted 3 lines
Other people have similar cards that do work so the problem seems isolated to my hardware but kernel-2.6.36 works so I don't think my hardware is faulty.
Which card is that? (adding linux-wireless)

D-Link dwa552
ieee80211 phy0: Atheros AR5416 MAC/BB Rev:2 AR2133 RF Rev:81 mem=0xffffc90000140
000, irq=16
Other people have this card working so it is probably my system configuration that is causing an obscure bug.

Quoted 2 lines

Please, describe in a very detailed way how your problem incarnates
itself: dmesg, error messages, what exactly do you do to trigger it?

It seems to be 100% that if "ath: Failed to stop TX DMA in 100 msec after killing last frame
ath: Failed to stop TX DMA!" shows up in dmesg that scans don't return any APs or 1-3 (iwlist wlan0 scan | grep SSID).
If there is a different way to scan, I'll try that.

Quoted 3 lines

Also, can you test whether 38.6 works for ya - it could've been fixed in the meantime. You can also test .39 which will be released any minute now.

38.6 does NOT work and compat-wireless-2011-05-11 works on kernel-2.6.36 which is why I think it is the kernel and not the wireless code.

Quoted 2 lines

I think that should be enough for now.

Borislav Petkov

2011-05-19 07:47 UTC #5 lore raw

Quoted 24 lines · James

Quoted 4 lines · Borislav Petkov
First of all, please hit "reply-to-all" next time so that all recipients can receive your mail and not find it by chance when looking thru the new lkml messages.
Quoted 8 lines
Quoted 6 lines · James
Here is what I did:
$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.37.y.git
$ cd linux-2.6.37.y
$ git checkout v2.6.36
build and boot kernel
what does that mean? Is .36 ok?
Yes, no problems.
Quoted 5 lines
Quoted 4 lines
$ cd linux-2.6.37.y
$ git bisect start
$ git bisect good v2.6.36
$ git bisect bad v2.6.37.6
Also, is .37.6 bad?
Yes, it is bad.
Quoted 4 lines
Quoted 4 lines
build and boot kernel
git bisect good|bad
repeat

Quoted 6 lines

Quoted 3 lines
Quoted 1 line
3044100e58c84e133791c8b60a2f5bef69d732e4 is the first bad commit.
Which is a merge commit and it means that you most likely made a mistake during bisection.
Am I doing it right?
boot kernel, cd to git source, git bisect good or bad, compile new kernel and copy it to /boot,

Dumb question: do you boot into that new kernel and test it each time before tagging it as good or bad?

Quoted 30 lines

repeat
Quoted 4 lines
What is actually the problem you're experiencing with .37.6?
..
Quoted 8 lines
Quoted 2 lines
How do I find out what change causes the problem?
The problem causes my wireless card not to work.
Ah, here it is.
Quoted 3 lines
Other people have similar cards that do work so the problem seems isolated to my hardware but kernel-2.6.36 works so I don't think my hardware is faulty.
Which card is that? (adding linux-wireless)
D-Link dwa552
ieee80211 phy0: Atheros AR5416 MAC/BB Rev:2 AR2133 RF Rev:81 mem=0xffffc90000140
000, irq=16
Other people have this card working so it is probably my system configuration that is causing an obscure bug.
Quoted 2 lines
Please, describe in a very detailed way how your problem incarnates
itself: dmesg, error messages, what exactly do you do to trigger it?
It seems to be 100% that if "ath: Failed to stop TX DMA in 100 msec after killing last frame
ath: Failed to stop TX DMA!" shows up in dmesg that scans don't return any APs or 1-3 (iwlist wlan0 scan | grep SSID).
If there is a different way to scan, I'll try that.

Quoted 6 lines

Quoted 3 lines
Also, can you test whether 38.6 works for ya - it could've been fixed in the meantime. You can also test .39 which will be released any minute now.
38.6 does NOT work and compat-wireless-2011-05-11 works on kernel-2.6.36 which is why I think it is the kernel and not the wireless code.
Quoted 1 line
I think that should be enough for now.

Aha, here's the deal. You're using compat-wireless which is a bunch of patches ontop of the kernel so you want to add them to the repository you're bisecting in so that they can be considered too. Anyway, I've added Luis to Cc for comment.

@Luis: bug descrption is above.

Btw, there's a newer version against 2.6.39-rc6 which you might want to test too: http://marc.info/?l=linux-wireless&m=130455450126421&w=2

HTH.

Signature · 2 lines

Regards/Gruss,
    Boris.

James

author

2011-05-19 15:57 UTC #6 lore raw

Quoted 25 lines · Borislav Petkov

Quoted 24 lines · James
Quoted 4 lines · Borislav Petkov
First of all, please hit "reply-to-all" next time so that all recipients can receive your mail and not find it by chance when looking thru the new lkml messages.
Quoted 8 lines
Quoted 6 lines · James
Here is what I did:
$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.37.y.git
$ cd linux-2.6.37.y
$ git checkout v2.6.36
build and boot kernel
what does that mean? Is .36 ok?
Yes, no problems.
Quoted 5 lines
Quoted 4 lines
$ cd linux-2.6.37.y
$ git bisect start
$ git bisect good v2.6.36
$ git bisect bad v2.6.37.6
Also, is .37.6 bad?
Yes, it is bad.
Quoted 4 lines
Quoted 4 lines
build and boot kernel
git bisect good|bad
repeat

Quoted 8 lines

Quoted 6 lines
Quoted 3 lines
Quoted 1 line
3044100e58c84e133791c8b60a2f5bef69d732e4 is the first bad commit.
Which is a merge commit and it means that you most likely made a mistake during bisection.
Am I doing it right?
boot kernel, cd to git source, git bisect good or bad, compile new kernel and copy it to /boot,
Dumb question: do you boot into that new kernel and test it each time before tagging it as good or bad?

Yes, I boot the new kernel.
I also did 'make clean' and 'make oldconfig' before I built each kernel.

Quoted 33 lines

Quoted 33 lines
repeat
Quoted 4 lines
What is actually the problem you're experiencing with .37.6?
..
Quoted 8 lines
Quoted 2 lines
How do I find out what change causes the problem?
The problem causes my wireless card not to work.
Ah, here it is.
Quoted 3 lines
Other people have similar cards that do work so the problem seems isolated to my hardware but kernel-2.6.36 works so I don't think my hardware is faulty.
Which card is that? (adding linux-wireless)
D-Link dwa552
ieee80211 phy0: Atheros AR5416 MAC/BB Rev:2 AR2133 RF Rev:81 mem=0xffffc90000140
000, irq=16
Other people have this card working so it is probably my system configuration that is causing an obscure bug.
Quoted 2 lines
Please, describe in a very detailed way how your problem incarnates
itself: dmesg, error messages, what exactly do you do to trigger it?
It seems to be 100% that if "ath: Failed to stop TX DMA in 100 msec after killing last frame
ath: Failed to stop TX DMA!" shows up in dmesg that scans don't return any APs or 1-3 (iwlist wlan0 scan | grep SSID).
If there is a different way to scan, I'll try that.
Quoted 3 lines
Also, can you test whether 38.6 works for ya - it could've been fixed in the meantime. You can also test .39 which will be released any minute now.

Quoted 7 lines

Quoted 3 lines
38.6 does NOT work and compat-wireless-2011-05-11 works on kernel-2.6.36 which is why I think it is the kernel and not the wireless code.
Quoted 1 line
I think that should be enough for now.
Aha, here's the deal. You're using compat-wireless which is a bunch of patches ontop of the kernel so you want to add them to the repository you're bisecting in so that they can be considered too. Anyway, I've added Luis to Cc for comment.

I made a typo, I tried compatwireless to kernel-2.6.36.4 to make sure. None of the kernels I bisected had compatwireless.

Quoted 6 lines

@Luis: bug descrption is above.
Btw, there's a newer version against 2.6.39-rc6 which you might want to test too: http://marc.info/?l=linux-wireless&m=130455450126421&w=2
HTH.

I built http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.39.tar.bz2

I tried to compile
http://www.orbit-lab.org/kernel/compat-wireless-2.6-stable/v2.6.39/compat-wireless-2.6.39-rc6-1-sp.tar.bz2
./scripts/gen-compat-autoconf.sh config.mk >
include/linux/compat_autoconf.h
make -C /lib/modules/2.6.38.6/build
M=/usr/src/compat-wireless-2.6.39-rc6-1-sp
modules

make[1]: Entering directory
`/usr/src/linux-2.6.38.6'
CC [M]
/usr/src/compat-wireless-2.6.39-rc6-1-sp/compat/main.o
CC [M]
/usr/src/compat-wireless-2.6.39-rc6-1-sp/compat/compat-2.6.39.o /usr/src/compat-wireless-2.6.39-rc6-1-sp/compat/compat-2.6.39.c: In function
'tty_set_termios':

/usr/src/compat-wireless-2.6.39-rc6-1-sp/compat/compat-2.6.39.c:93:
error: 'TASK_INTERRUPTIBLE' undeclared (first use in this function) /usr/src/compat-wireless-2.6.39-rc6-1-sp/compat/compat-2.6.39.c:93:
error: (Each undeclared identifier is reported only
once /usr/src/compat-wireless-2.6.39-rc6-1-sp/compat/compat-2.6.39.c:93:
error: for each function it appears
in.)
make[3]: *** [/usr/src/compat-wireless-2.6.39-rc6-1-sp/compat/compat-2.6.39.o] Error 1
make[2]: *** [/usr/src/compat-wireless-2.6.39-rc6-1-sp/compat] Error 2
make[1]: *** [_module_/usr/src/compat-wireless-2.6.39-rc6-1-sp] Error 2
make[1]: Leaving directory
`/usr/src/linux-2.6.38.6'
make: *** [modules] Error
2

I wonder if it could be simply something is missing in my .config I put it on http://lockie.ca/test/config.bz2

This is the first kernel that doesn't work that displays the dmesg errors after I try a scan.
Usually the messages show up immediately.

I can't reproduce this but this was in dmesg the first time I booted:

ath: DMA failed to stop in 10 ms AR_CR=0x00000024 AR_DIAG_SW=0x42000020 DMADBG_7=0x000062c0
ath: Could not stop RX, we could be confusing the DMA engine when we start RX up

------------[ cut here ]------------
WARNING: at drivers/net/wireless/ath/ath9k/recv.c:507
ath_stoprecv+0xcf/0xf1 [ath9k]()
Hardware name: To Be Filled By O.E.M.
Modules linked in: ath9k mac80211 ath9k_common ath9k_hw ath cfg80211
Pid: 699, comm: kworker/u:4 Not tainted 2.6.39 #1
Call Trace:
 [<ffffffff81029eb8>] ? warn_slowpath_common+0x78/0x8c
 [<ffffffffa00bfd90>] ? ath_stoprecv+0xcf/0xf1 [ath9k]
 [<ffffffffa00bd6fb>] ? ath_set_channel+0xce/0x273 [ath9k]
 [<ffffffffa002485f>] ? ath_hw_cycle_counters_update+0xdf/0x123 [ath]
 [<ffffffffa00bdbc0>] ? ath9k_config+0x320/0x435 [ath9k]
 [<ffffffffa008b796>] ? ieee80211_scan_work+0x2e7/0x456 [mac80211]
 [<ffffffffa008b4af>] ? ieee80211_scan_completed+0x29/0x29 [mac80211]
 [<ffffffff8103b7fa>] ? process_one_work+0x20e/0x34e
 [<ffffffff8103bd39>] ? worker_thread+0x1c9/0x340
 [<ffffffff81020af5>] ? __wake_up_common+0x41/0x78
 [<ffffffff8103bb70>] ? rescuer_thread+0x236/0x236
 [<ffffffff8103bb70>] ? rescuer_thread+0x236/0x236
 [<ffffffff8103eb4a>] ? kthread+0x7a/0x82
 [<ffffffff8133a9d4>] ? kernel_thread_helper+0x4/0x10
 [<ffffffff8103ead0>] ? kthread_worker_fn+0x107/0x107
 [<ffffffff8133a9d0>] ? gs_change+0xb/0xb
---[ end trace 7d1c5ea6cf770ada ]---

I put the full dmesg output on I put it on
http://lockie.ca/test/dmesg.txt.bz2