Problems with commit 'kallsyms: add support for relative offsets in kallsyms address table' (in mmotm)

Hi,

I see runtime problems with the current mmotm branch. All qemu mips targets (32 and 64 bit, big and little endian) are stuck in boot after this commit.

Bisect points to commit d13682e4d9d2 ("kallsyms: add support for relative offsets in kallsyms address table". Disabling CONFIG_KALLSYMS_BASE_RELATIVE fixes the problem, ie I can boot the image with qemu.

Bisect log is attached.

Playing with the problem, I found the following:

1) The problem is only seen with a toolchain using binutils 2.22, but not with a toolchain using binutils 2.25. The compiler configuration may be different for both toolchains.
2) Message "kallsyms failure: absolute symbol value 0xffffffff807afd14 out of range in relative mode" (twice) when using the toolchain with binutils 2.22. This does not cause the build to fail, though.
3) kallsyms_sym_address() parameter variable type is "int". In the calling code, the variable type used is "unsigned long". That has no impact on the problem, though.

Please let me know if there is anything I can do to help tracking down the problem.

Thanks,
Guenter

---
# bad: [437b7ef6d6484bb36afb2c75897e14bf03113dbd] pci: test for unexpectedly disabled bridges
# good: [afd2ff9b7e1b367172f18ba7f693dfb62bdcb2dc] Linux 4.4
git bisect start 'HEAD' 'v4.4'
# bad: [10078c1820d81131bc338c7c919feb2638a912ad] ocfs2: o2hb: add NEGOTIATE_APPROVE message
git bisect bad 10078c1820d81131bc338c7c919feb2638a912ad
# bad: [1b06048b37e888edb609e1509070ec0c6580eedd] kallsyms-add-support-for-relative-offsets-in-kallsyms-address-table-fix git bisect bad 1b06048b37e888edb609e1509070ec0c6580eedd
# good: [ed312d94332968c86f53295795a3d122fd32cea2] arch/alpha/kernel/systbls.S: remove debug check git bisect good ed312d94332968c86f53295795a3d122fd32cea2
# good: [1cbe7000217b0126821ecd8fba0ba5f9315a0560] thp: limit number of object to scan on deferred_split_scan() git bisect good 1cbe7000217b0126821ecd8fba0ba5f9315a0560
# good: [474324b13e8184aa088c722b771f0f31ab773476] vmstat: remove BUG_ON from vmstat_update
git bisect good 474324b13e8184aa088c722b771f0f31ab773476
# good: [d94a9840c248207040d62ca0f380943db9423418] proc: fix missing reference of mm
git bisect good d94a9840c248207040d62ca0f380943db9423418
# bad: [d13682e4d9d2ad8220354b5e1a2bd6623e1f2e38] kallsyms: add support for relative offsets in kallsyms address table git bisect bad d13682e4d9d2ad8220354b5e1a2bd6623e1f2e38

# first bad commit: [d13682e4d9d2ad8220354b5e1a2bd6623e1f2e38] kallsyms: add support for relative offsets in kallsyms address table

Quoted 15 lines · Guenter Roeck

Hi,

I see runtime problems with the current mmotm branch. All qemu mips targets (32 and 64 bit, big and little endian) are stuck in boot after this commit.

Bisect points to commit d13682e4d9d2 ("kallsyms: add support for relative offsets in kallsyms address table". Disabling CONFIG_KALLSYMS_BASE_RELATIVE fixes the problem, ie I can boot the image with qemu.

Bisect log is attached.

Playing with the problem, I found the following:

Quoted 9 lines

1) The problem is only seen with a toolchain using binutils 2.22, but not with a toolchain using binutils 2.25. The compiler configuration may be different for both toolchains.
2) Message "kallsyms failure: absolute symbol value 0xffffffff807afd14 out of range in relative mode" (twice) when using the toolchain with binutils 2.22. This does not cause the build to fail, though.
3) kallsyms_sym_address() parameter variable type is "int". In the calling code, the variable type used is "unsigned long". That has no impact on the problem, though.

An additional data point: When using the older toolchain, many symbols in System.map are marked "A".
ffffffff80100000 A _text
With the more recent toolchain, the same symbols are marked "T".
ffffffff80100000 T _text

Quoted 25 lines

Please let me know if there is anything I can do to help tracking down the problem.

Thanks,
Guenter

---
# bad: [437b7ef6d6484bb36afb2c75897e14bf03113dbd] pci: test for unexpectedly disabled bridges
# good: [afd2ff9b7e1b367172f18ba7f693dfb62bdcb2dc] Linux 4.4
git bisect start 'HEAD' 'v4.4'
# bad: [10078c1820d81131bc338c7c919feb2638a912ad] ocfs2: o2hb: add NEGOTIATE_APPROVE message
git bisect bad 10078c1820d81131bc338c7c919feb2638a912ad
# bad: [1b06048b37e888edb609e1509070ec0c6580eedd] kallsyms-add-support-for-relative-offsets-in-kallsyms-address-table-fix git bisect bad 1b06048b37e888edb609e1509070ec0c6580eedd
# good: [ed312d94332968c86f53295795a3d122fd32cea2] arch/alpha/kernel/systbls.S: remove debug check git bisect good ed312d94332968c86f53295795a3d122fd32cea2
# good: [1cbe7000217b0126821ecd8fba0ba5f9315a0560] thp: limit number of object to scan on deferred_split_scan() git bisect good 1cbe7000217b0126821ecd8fba0ba5f9315a0560
# good: [474324b13e8184aa088c722b771f0f31ab773476] vmstat: remove BUG_ON from vmstat_update
git bisect good 474324b13e8184aa088c722b771f0f31ab773476
# good: [d94a9840c248207040d62ca0f380943db9423418] proc: fix missing reference of mm
git bisect good d94a9840c248207040d62ca0f380943db9423418
# bad: [d13682e4d9d2ad8220354b5e1a2bd6623e1f2e38] kallsyms: add support for relative offsets in kallsyms address table git bisect bad d13682e4d9d2ad8220354b5e1a2bd6623e1f2e38

# first bad commit: [d13682e4d9d2ad8220354b5e1a2bd6623e1f2e38] kallsyms: add support for relative offsets in kallsyms address table

Quoted 28 lines · Guenter Roeck

Quoted 24 lines · Guenter Roeck

Hi,

I see runtime problems with the current mmotm branch. All qemu mips targets (32 and 64 bit, big and little endian) are stuck in boot after this commit.

Bisect points to commit d13682e4d9d2 ("kallsyms: add support for relative offsets in kallsyms address table". Disabling CONFIG_KALLSYMS_BASE_RELATIVE fixes the problem, ie I can boot the image with qemu.

Bisect log is attached.

Playing with the problem, I found the following:

1) The problem is only seen with a toolchain using binutils 2.22, but not with a toolchain using binutils 2.25. The compiler configuration may be different for both toolchains.
2) Message "kallsyms failure: absolute symbol value 0xffffffff807afd14 out of range in relative mode" (twice) when using the toolchain with binutils 2.22. This does not cause the build to fail, though.
3) kallsyms_sym_address() parameter variable type is "int". In the calling code, the variable type used is "unsigned long". That has no impact on the problem, though.

Quoted 6 lines

An additional data point: When using the older toolchain, many symbols in System.map are marked "A".

    ffffffff80100000 A _text
With the more recent toolchain, the same symbols are marked "T".
    ffffffff80100000 T _text

Thanks for the analysis. It is surprising that the build does not fail when this occurs, and the subsequent hangs themselves are probably caused by missing kallsyms data.

scripts/kallsyms.c ignores all A symbols except _text, which is actually a relative symbol by nature so we can simply assume it is relative (i.e., override it as T)

Re x86_64 !SMP, any build time errors there as well? Likewise for sparc32?

Thanks again, and sorry for the trouble,

Quoted 48 lines · Ard Biesheuvel

Quoted 40 lines · Guenter Roeck

Quoted 19 lines · Guenter Roeck

Hi,

I see runtime problems with the current mmotm branch. All qemu mips targets
(32 and 64 bit, big and little endian) are stuck in boot after this commit.

Bisect points to commit d13682e4d9d2 ("kallsyms: add support for relative offsets
in kallsyms address table". Disabling CONFIG_KALLSYMS_BASE_RELATIVE fixes the problem,
ie I can boot the image with qemu.

Bisect log is attached.

Playing with the problem, I found the following:

Quoted 15 lines

1) The problem is only seen with a toolchain using binutils 2.22, but not
with a toolchain using binutils 2.25. The compiler configuration may
be
different for both toolchains.
2) Message "kallsyms failure: absolute symbol value 0xffffffff807afd14 out of range
in relative mode" (twice) when using the toolchain with binutils
2.22.
This does not cause the build to fail, though.
3) kallsyms_sym_address() parameter variable type is "int". In the calling code,
the variable type used is "unsigned long". That has no impact on the
problem,
though.

Quoted 7 lines

An additional data point: When using the older toolchain, many symbols in System.map
are marked "A".

    ffffffff80100000 A _text
With the more recent toolchain, the same symbols are marked "T".
    ffffffff80100000 T _text

Quoted 4 lines

Thanks for the analysis. It is surprising that the build does not fail when this occurs, and the subsequent hangs themselves are probably caused by missing kallsyms data.

Yes, I wondered why the build doesn't fail. Seems odd.

Quoted 6 lines

scripts/kallsyms.c ignores all A symbols except _text, which is actually a relative symbol by nature so we can simply assume it is relative (i.e., override it as T)

Re x86_64 !SMP, any build time errors there as well? Likewise for sparc32?

Yes, same kind of errors for both. For x86_64/nosmp I also get the error message
when using the Ubuntu native toolchain, so it doesn't seem to be (directly) related to binutils 2.22 vs. 2.25 for that architecture.

Runtime behavior is a bit different for the different architectures. x86_64 dies silently without any console output, mips just hangs, and sparc32 gets a panic with NULL pointer access.
Of course, with missing kallsyms data all bets are off.

Quoted 1 line

Quoted 1 line

Thanks again, and sorry for the trouble,

No worries. Hope you'll get this sorted out.

Quoted 73 lines · Guenter Roeck

Quoted 48 lines · Ard Biesheuvel

Quoted 40 lines · Guenter Roeck

Quoted 19 lines · Guenter Roeck

Hi,

I see runtime problems with the current mmotm branch. All qemu mips targets
(32 and 64 bit, big and little endian) are stuck in boot after this commit.

Bisect points to commit d13682e4d9d2 ("kallsyms: add support for relative offsets
in kallsyms address table". Disabling CONFIG_KALLSYMS_BASE_RELATIVE fixes the problem,
ie I can boot the image with qemu.

Bisect log is attached.

Playing with the problem, I found the following:

Quoted 15 lines

1) The problem is only seen with a toolchain using binutils 2.22, but not
with a toolchain using binutils 2.25. The compiler configuration may
be
different for both toolchains.
2) Message "kallsyms failure: absolute symbol value 0xffffffff807afd14 out of range
in relative mode" (twice) when using the toolchain with binutils
2.22.
This does not cause the build to fail, though.
3) kallsyms_sym_address() parameter variable type is "int". In the calling code,
the variable type used is "unsigned long". That has no impact on the
problem,
though.

Quoted 7 lines

An additional data point: When using the older toolchain, many symbols in System.map
are marked "A".

     ffffffff80100000 A _text
With the more recent toolchain, the same symbols are marked "T".
     ffffffff80100000 T _text

Quoted 4 lines

Thanks for the analysis. It is surprising that the build does not fail when this occurs, and the subsequent hangs themselves are probably caused by missing kallsyms data.

Yes, I wondered why the build doesn't fail. Seems odd.

Quoted 6 lines

scripts/kallsyms.c ignores all A symbols except _text, which is actually a relative symbol by nature so we can simply assume it is relative (i.e., override it as T)

Re x86_64 !SMP, any build time errors there as well? Likewise for sparc32?

Yes, same kind of errors for both. For x86_64/nosmp I also get the error message
when using the Ubuntu native toolchain, so it doesn't seem to be (directly) related to binutils 2.22 vs. 2.25 for that architecture.

Runtime behavior is a bit different for the different architectures. x86_64 dies silently without any console output, mips just hangs, and sparc32 gets a panic with NULL pointer access.
Of course, with missing kallsyms data all bets are off.

Quoted 1 line

Quoted 5 lines

Quoted 1 line

Thanks again, and sorry for the trouble,

No worries. Hope you'll get this sorted out.

OK, there's an additional issue in my latest version: the kallsyms_relative_base value itself is not relocated.

If you have more time to burn on this, could you try the following on top? (If not, that is also fine, I will look into it myself on Monday)

scripts/kallsyms.c +3 -1

index 5ab13394dfd9..0f43f0751d47 100644
@@ -137,8 +137,10 @@ static int read_symbol(FILE *in, struct sym_entry *s)
                 sym++;
 
         /* Ignore most absolute/undefined (?) symbols. */
-       if (strcmp(sym, "_text") == 0)
+       if (strcmp(sym, "_text") == 0) {
                 _text = s->addr;
+               stype = 'T';
+       }
         else if (check_symbol_range(sym, s->addr, text_ranges,
                                     ARRAY_SIZE(text_ranges)) == 0)
                 /* nothing to do */;

Quoted 102 lines · Ard Biesheuvel

Quoted 71 lines · Guenter Roeck

Quoted 48 lines · Ard Biesheuvel

Quoted 40 lines

Quoted 23 lines · Guenter Roeck

On 01/23/2016 06:06 PM, Guenter Roeck wrote:
Hi,

I see runtime problems with the current mmotm branch. All qemu mips targets
(32 and 64 bit, big and little endian) are stuck in boot after this commit.

Bisect points to commit d13682e4d9d2 ("kallsyms: add support for relative offsets
in kallsyms address table". Disabling CONFIG_KALLSYMS_BASE_RELATIVE fixes the problem,
ie I can boot the image with qemu.

Bisect log is attached.

Playing with the problem, I found the following:

Quoted 15 lines

1) The problem is only seen with a toolchain using binutils 2.22, but not
with a toolchain using binutils 2.25. The compiler configuration may
be
different for both toolchains.
2) Message "kallsyms failure: absolute symbol value 0xffffffff807afd14 out of range
in relative mode" (twice) when using the toolchain with binutils
2.22.
This does not cause the build to fail, though.
3) kallsyms_sym_address() parameter variable type is "int". In the calling code,
the variable type used is "unsigned long". That has no impact on the
problem,
though.

Quoted 6 lines

An additional data point: When using the older toolchain, many symbols in System.map
are marked "A".

    ffffffff80100000 A _text
With the more recent toolchain, the same symbols are marked "T".
    ffffffff80100000 T _text

Quoted 3 lines

Thanks for the analysis. It is surprising that the build does not fail when this occurs, and the subsequent hangs themselves are probably caused by missing kallsyms data.

Yes, I wondered why the build doesn't fail. Seems odd.

Quoted 5 lines

scripts/kallsyms.c ignores all A symbols except _text, which is actually a relative symbol by nature so we can simply assume it is relative (i.e., override it as T)

Re x86_64 !SMP, any build time errors there as well? Likewise for sparc32?

Yes, same kind of errors for both. For x86_64/nosmp I also get the error message
when using the Ubuntu native toolchain, so it doesn't seem to be (directly) related to binutils 2.22 vs. 2.25 for that architecture.

Runtime behavior is a bit different for the different architectures. x86_64 dies silently without any console output, mips just hangs, and sparc32 gets a panic with NULL pointer access.
Of course, with missing kallsyms data all bets are off.

Quoted 1 line

Quoted 4 lines

Quoted 1 line

Thanks again, and sorry for the trouble,

No worries. Hope you'll get this sorted out.

OK, there's an additional issue in my latest version: the kallsyms_relative_base value itself is not relocated.

If you have more time to burn on this, could you try the following on top? (If not, that is also fine, I will look into it myself on Monday)

scripts/kallsyms.c +3 -1

index 5ab13394dfd9..0f43f0751d47 100644
@@ -137,8 +137,10 @@ static int read_symbol(FILE *in, struct sym_entry *s)
                sym++;
 
        /* Ignore most absolute/undefined (?) symbols. */
-       if (strcmp(sym, "_text") == 0)
+       if (strcmp(sym, "_text") == 0) {
                _text = s->addr;
+               stype = 'T';
+       }
        else if (check_symbol_range(sym, s->addr, text_ranges,
                                    ARRAY_SIZE(text_ranges)) == 0)
                /* nothing to do */;

@@ -406,7 +408,7 @@ static void write_src(void)
 

scripts/kallsyms.c static void write_src(void) · line 413

Quoted 6 lines

scripts/kallsyms.c +1 -1

        if (base_relative) {
                output_label("kallsyms_relative_base");
-               printf("\tPTR\t%#llx\n", relative_base);
+               printf("\tPTR\t_text - %#llx\n", _text - relative_base);
                printf("\n");
        }

Does not help.

Here is part of the problem. This is from a log message added to make_percpus_absolute().

Marking symbol 'B__bss_start' as absolute
Marking symbol '?__init_end' as absolute
Marking symbol 'D__nosave_begin' as absolute
Marking symbol 'D__nosave_end' as absolute
Marking symbol 'D__per_cpu_end' as absolute
Marking symbol 'D__per_cpu_load' as absolute
Marking symbol 'D__per_cpu_start' as absolute
Marking symbol '?__smp_locks' as absolute
Marking symbol '?__smp_locks_end' as absolute
Marking symbol 'Bempty_zero_page' as absolute

This is with x86_64/nosmp. At least some of those symbols don't really reflect 'percpu' values. Maybe the distinction between percpu and non-percpu variables gets lost if SMP is not configured.

On top of that, older versions of binutils mark additional symbols as absolute, even with x86_64.

ffffffff81a00000 A __end_rodata_hpage_align
ffffffff81b19000 A __vvar_page
ffffffff81d3d000 A _end

Hope this helps,

Quoted 24 lines

Quoted 24 lines

OK, there's an additional issue in my latest version: the kallsyms_relative_base value itself is not relocated.

If you have more time to burn on this, could you try the following on top? (If not, that is also fine, I will look into it myself on Monday)

scripts/kallsyms.c +3 -1

index 5ab13394dfd9..0f43f0751d47 100644
@@ -137,8 +137,10 @@ static int read_symbol(FILE *in, struct sym_entry *s)
                 sym++;
 
         /* Ignore most absolute/undefined (?) symbols. */
-       if (strcmp(sym, "_text") == 0)
+       if (strcmp(sym, "_text") == 0) {
                 _text = s->addr;
+               stype = 'T';
+       }
         else if (check_symbol_range(sym, s->addr, text_ranges,
                                     ARRAY_SIZE(text_ranges)) == 0)
                 /* nothing to do */;

@@ -406,7 +408,7 @@ static void write_src(void)
 

Quoted 9 lines

scripts/kallsyms.c static void write_src(void) · line 413

Quoted 6 lines

scripts/kallsyms.c +1 -1

         if (base_relative) {
                 output_label("kallsyms_relative_base");
-               printf("\tPTR\t%#llx\n", relative_base);
+               printf("\tPTR\t_text - %#llx\n", _text - relative_base);
                 printf("\n");
         }

Does not help.

For x86? Or none of them?

On 01/24/2016 09:20 AM, Ard Biesheuvel wrote:
[ ... ]

Quoted 34 lines

Quoted 24 lines

Quoted 24 lines

OK, there's an additional issue in my latest version: the kallsyms_relative_base value itself is not relocated.

If you have more time to burn on this, could you try the following on top? (If not, that is also fine, I will look into it myself on Monday)

scripts/kallsyms.c +3 -1

index 5ab13394dfd9..0f43f0751d47 100644
@@ -137,8 +137,10 @@ static int read_symbol(FILE *in, struct sym_entry *s)
                sym++;
 
        /* Ignore most absolute/undefined (?) symbols. */
-       if (strcmp(sym, "_text") == 0)
+       if (strcmp(sym, "_text") == 0) {
                _text = s->addr;
+               stype = 'T';
+       }
        else if (check_symbol_range(sym, s->addr, text_ranges,
                                    ARRAY_SIZE(text_ranges)) == 0)
                /* nothing to do */;

@@ -406,7 +408,7 @@ static void write_src(void)
 

Quoted 9 lines

scripts/kallsyms.c static void write_src(void) · line 413

Quoted 6 lines

scripts/kallsyms.c +1 -1

        if (base_relative) {
                output_label("kallsyms_relative_base");
-               printf("\tPTR\t%#llx\n", relative_base);
+               printf("\tPTR\t_text - %#llx\n", _text - relative_base);
                printf("\n");
        }

Does not help.

Quoted 2 lines

For x86? Or none of them?

I tested sparc32 and x86_64/nosmp. Doesn't help for any of them. sparc32 has the following absolute symbols.

f035a420 A _etext
f03d9000 A _sdata
f03de8c4 A jiffies
f03f8860 A _edata
f03fc000 A __init_begin
f041bdc8 A __init_text_end
f0423000 A __bss_start
f0423000 A __init_end
f044457d A __bss_stop
f044457d A _end

This results in:

kallsyms failure: absolute symbol value 0xf035a420 out of range in relative mode

This is with binutils 2.22. I didn't test with binutils 2.25 for sparc, or re-test mips.

Looks like I'll need to add more test cases with binutils 2.22 vs. 2.25 for various architectures, as well as more SMP vs. !SMP builds.

Quoted 45 lines · Guenter Roeck

On 01/24/2016 09:20 AM, Ard Biesheuvel wrote:
[ ... ]

Quoted 33 lines

Quoted 24 lines

Quoted 24 lines

OK, there's an additional issue in my latest version: the kallsyms_relative_base value itself is not relocated.

If you have more time to burn on this, could you try the following on top? (If not, that is also fine, I will look into it myself on Monday)

scripts/kallsyms.c +3 -1

index 5ab13394dfd9..0f43f0751d47 100644
@@ -137,8 +137,10 @@ static int read_symbol(FILE *in, struct sym_entry *s)
               sym++;
 
       /* Ignore most absolute/undefined (?) symbols. */
-       if (strcmp(sym, "_text") == 0)
+       if (strcmp(sym, "_text") == 0) {
               _text = s->addr;
+               stype = 'T';
+       }
       else if (check_symbol_range(sym, s->addr, text_ranges,
                                   ARRAY_SIZE(text_ranges)) == 0)
               /* nothing to do */;

@@ -406,7 +408,7 @@ static void write_src(void)
 

Quoted 8 lines

scripts/kallsyms.c static void write_src(void) · line 413

Quoted 6 lines

scripts/kallsyms.c +1 -1

       if (base_relative) {
               output_label("kallsyms_relative_base");
-               printf("\tPTR\t%#llx\n", relative_base);
+               printf("\tPTR\t_text - %#llx\n", _text - relative_base);
               printf("\n");
       }

Does not help.

Quoted 1 line

For x86? Or none of them?

I tested sparc32 and x86_64/nosmp. Doesn't help for any of them. sparc32 has the following absolute symbols.

Quoted 10 lines

f035a420 A _etext
f03d9000 A _sdata
f03de8c4 A jiffies
f03f8860 A _edata
f03fc000 A __init_begin
f041bdc8 A __init_text_end
f0423000 A __bss_start
f0423000 A __init_end
f044457d A __bss_stop
f044457d A _end

Any clue why these don't get dropped? Am I missing something? Afaict A symbols get dropped unless they are whitelisted (i.e., the few ia64 ones)

Quoted 9 lines

This results in:

kallsyms failure: absolute symbol value 0xf035a420 out of range in relative mode

This is with binutils 2.22. I didn't test with binutils 2.25 for sparc, or re-test mips.

Looks like I'll need to add more test cases with binutils 2.22 vs. 2.25 for various architectures, as well as more SMP vs. !SMP builds.

Thanks once again

Hi,

I see runtime problems with the current mmotm branch. All qemu mips targets (32 and 64 bit, big and little endian) are stuck in boot after this commit.

Bisect points to commit d13682e4d9d2 ("kallsyms: add support for relative offsets in kallsyms address table". Disabling CONFIG_KALLSYMS_BASE_RELATIVE fixes the problem, ie I can boot the image with qemu.

Bisect log is attached.

Playing with the problem, I found the following:

1) The problem is only seen with a toolchain using binutils 2.22, but not with a toolchain using binutils 2.25. The compiler configuration may be different for both toolchains.
2) Message "kallsyms failure: absolute symbol value 0xffffffff807afd14 out of range in relative mode" (twice) when using the toolchain with binutils 2.22. This does not cause the build to fail, though.
3) kallsyms_sym_address() parameter variable type is "int". In the calling code, the variable type used is "unsigned long". That has no impact on the problem, though.

Please let me know if there is anything I can do to help tracking down the problem.

Thanks,
Guenter

---
# bad: [437b7ef6d6484bb36afb2c75897e14bf03113dbd] pci: test for unexpectedly disabled bridges
# good: [afd2ff9b7e1b367172f18ba7f693dfb62bdcb2dc] Linux 4.4
git bisect start 'HEAD' 'v4.4'
# bad: [10078c1820d81131bc338c7c919feb2638a912ad] ocfs2: o2hb: add NEGOTIATE_APPROVE message
git bisect bad 10078c1820d81131bc338c7c919feb2638a912ad
# bad: [1b06048b37e888edb609e1509070ec0c6580eedd] kallsyms-add-support-for-relative-offsets-in-kallsyms-address-table-fix git bisect bad 1b06048b37e888edb609e1509070ec0c6580eedd
# good: [ed312d94332968c86f53295795a3d122fd32cea2] arch/alpha/kernel/systbls.S: remove debug check git bisect good ed312d94332968c86f53295795a3d122fd32cea2
# good: [1cbe7000217b0126821ecd8fba0ba5f9315a0560] thp: limit number of object to scan on deferred_split_scan() git bisect good 1cbe7000217b0126821ecd8fba0ba5f9315a0560
# good: [474324b13e8184aa088c722b771f0f31ab773476] vmstat: remove BUG_ON from vmstat_update
git bisect good 474324b13e8184aa088c722b771f0f31ab773476
# good: [d94a9840c248207040d62ca0f380943db9423418] proc: fix missing reference of mm
git bisect good d94a9840c248207040d62ca0f380943db9423418
# bad: [d13682e4d9d2ad8220354b5e1a2bd6623e1f2e38] kallsyms: add support for relative offsets in kallsyms address table git bisect bad d13682e4d9d2ad8220354b5e1a2bd6623e1f2e38

# first bad commit: [d13682e4d9d2ad8220354b5e1a2bd6623e1f2e38] kallsyms: add support for relative offsets in kallsyms address table