Skip to contentSkip to conversation

New: x86_32: bpf: multiple test_bpf failures using eBPF JIT

Bug 205469 Stephen Hemminger 4 participants 7 messages Nov 8, 2019 – Nov 21, 2019 BPF

lore
Share & tools
lore
#0 lore raw

Begin forwarded message:

Date: Fri, 08 Nov 2019 07:35:59 +0000
From: [email protected]
To: [email protected]
Subject: [Bug 205469] New: x86_32: bpf: multiple test_bpf failures using eBPF JIT

https://bugzilla.kernel.org/show_bug.cgi?id=205469

            Bug ID: 205469
           Summary: x86_32: bpf: multiple test_bpf failures using eBPF JIT
           Product: Networking
           Version: 2.5
    Kernel Version: 4.19.81 LTS
          Hardware: i386
                OS: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: Other
          Assignee: [email protected]
          Reporter: [email protected]
                CC: [email protected]
        Regression: No

Created attachment 285829
--> https://bugzilla.kernel.org/attachment.cgi?id=285829&action=edit
test_bpf failures: kernel 4.19.81/x86_32 (OpenWrt)

Summary:
========

Running the 4.19.81 LTS kernel on QEMU/x86_32, the standard test_bpf.ko testsuite generates multiple errors with the eBPF JIT enabled:

...
test_bpf: #32 JSET jited:1 40 ret 0 != 20 46 FAIL
test_bpf: #321 LD_IND word positive offset jited:1 ret 0 != -291897430 FAIL
test_bpf: #322 LD_IND word negative offset jited:1 ret 0 != -1437222042 FAIL
test_bpf: #323 LD_IND word unaligned (addr & 3 == 2) jited:1 ret 0 !=
-1150890889 FAIL
test_bpf: #326 LD_IND word positive offset, all ff jited:1 ret 0 != -1 FAIL
...
test_bpf: Summary: 373 PASSED, 5 FAILED, [344/366 JIT'ed]

However, with eBPF JIT disabled (net.core.bpf_jit_enable=0) all tests pass.

Steps to Reproduce:
===================

  # sysctl net.core.bpf_jit_enable=1
  # modprobe test_bpf
  <Kernel log with failures and test summary>

Affected Systems Tested:
========================

OpenWrt master on QEMU/pc-q35(x86_32) [LTS kernel 4.19.81]

Kernel Logs:
============

Boot log with test results is attached.

Signature · 2 lines
You are receiving this mail because:
You are the assignee for the bug.
maintainer
#1 lore raw

[ Cc Wang (x86_32 BPF JIT maintainer) ]

Quoted 70 lines · Stephen Hemminger

Begin forwarded message:

Date: Fri, 08 Nov 2019 07:35:59 +0000
From: [email protected]
To: [email protected]
Subject: [Bug 205469] New: x86_32: bpf: multiple test_bpf failures using eBPF JIT

https://bugzilla.kernel.org/show_bug.cgi?id=205469

             Bug ID: 205469
            Summary: x86_32: bpf: multiple test_bpf failures using eBPF JIT
            Product: Networking
            Version: 2.5
     Kernel Version: 4.19.81 LTS
           Hardware: i386
                 OS: Linux
               Tree: Mainline
             Status: NEW
           Severity: normal
           Priority: P1
          Component: Other
           Assignee: [email protected]
           Reporter: [email protected]
                 CC: [email protected]
         Regression: No

Created attachment 285829
--> https://bugzilla.kernel.org/attachment.cgi?id=285829&action=edit
test_bpf failures: kernel 4.19.81/x86_32 (OpenWrt)

Summary:
========

Running the 4.19.81 LTS kernel on QEMU/x86_32, the standard test_bpf.ko testsuite generates multiple errors with the eBPF JIT enabled:

   ...
   test_bpf: #32 JSET jited:1 40 ret 0 != 20 46 FAIL
   test_bpf: #321 LD_IND word positive offset jited:1 ret 0 != -291897430 FAIL
   test_bpf: #322 LD_IND word negative offset jited:1 ret 0 != -1437222042 FAIL
   test_bpf: #323 LD_IND word unaligned (addr & 3 == 2) jited:1 ret 0 !=

-1150890889 FAIL
test_bpf: #326 LD_IND word positive offset, all ff jited:1 ret 0 != -1 FAIL
...
test_bpf: Summary: 373 PASSED, 5 FAILED, [344/366 JIT'ed]

However, with eBPF JIT disabled (net.core.bpf_jit_enable=0) all tests pass.

Steps to Reproduce:
===================

   # sysctl net.core.bpf_jit_enable=1
   # modprobe test_bpf
   <Kernel log with failures and test summary>

Affected Systems Tested:
========================

   OpenWrt master on QEMU/pc-q35(x86_32) [LTS kernel 4.19.81]

Kernel Logs:
============

Boot log with test results is attached.

#2 lore raw
Quoted 73 lines · Daniel Borkmann

[ Cc Wang (x86_32 BPF JIT maintainer) ]

Quoted 70 lines · Stephen Hemminger

Begin forwarded message:

Date: Fri, 08 Nov 2019 07:35:59 +0000
From: [email protected]
To: [email protected]
Subject: [Bug 205469] New: x86_32: bpf: multiple test_bpf failures using eBPF JIT

https://bugzilla.kernel.org/show_bug.cgi?id=205469

             Bug ID: 205469
            Summary: x86_32: bpf: multiple test_bpf failures using eBPF JIT
            Product: Networking
            Version: 2.5
     Kernel Version: 4.19.81 LTS
           Hardware: i386
                 OS: Linux
               Tree: Mainline
             Status: NEW
           Severity: normal
           Priority: P1
          Component: Other
           Assignee: [email protected]
           Reporter: [email protected]
                 CC: [email protected]
         Regression: No

Created attachment 285829
--> https://bugzilla.kernel.org/attachment.cgi?id=285829&action=edit
test_bpf failures: kernel 4.19.81/x86_32 (OpenWrt)

Summary:
========

Running the 4.19.81 LTS kernel on QEMU/x86_32, the standard test_bpf.ko testsuite generates multiple errors with the eBPF JIT enabled:

   ...
   test_bpf: #32 JSET jited:1 40 ret 0 != 20 46 FAIL
   test_bpf: #321 LD_IND word positive offset jited:1 ret 0 != -291897430 FAIL
   test_bpf: #322 LD_IND word negative offset jited:1 ret 0 != -1437222042 FAIL
   test_bpf: #323 LD_IND word unaligned (addr & 3 == 2) jited:1 ret 0 !=

-1150890889 FAIL
test_bpf: #326 LD_IND word positive offset, all ff jited:1 ret 0 != -1 FAIL
...
test_bpf: Summary: 373 PASSED, 5 FAILED, [344/366 JIT'ed]

However, with eBPF JIT disabled (net.core.bpf_jit_enable=0) all tests pass.

Steps to Reproduce:
===================

   # sysctl net.core.bpf_jit_enable=1
   # modprobe test_bpf
   <Kernel log with failures and test summary>

Affected Systems Tested:
========================

   OpenWrt master on QEMU/pc-q35(x86_32) [LTS kernel 4.19.81]

Kernel Logs:
============

Boot log with test results is attached.

Hi Daniel Borkmann!

I have tested and verified that the report bug has been fixed by commit 711aef1bbf88 ("bpf, x32: Fix bug for BPF_JMP | {BPF_JSGT, BPF_JSLE, BPF_JSLT, BPF_JSGE}")

But that fix hasn't been backport to stable trees, so maybe we should do it:)

maintainer
#3 lore raw
Quoted 75 lines · Wang YanQing
Quoted 73 lines · Daniel Borkmann

[ Cc Wang (x86_32 BPF JIT maintainer) ]

Quoted 70 lines · Stephen Hemminger

Begin forwarded message:

Date: Fri, 08 Nov 2019 07:35:59 +0000
From: [email protected]
To: [email protected]
Subject: [Bug 205469] New: x86_32: bpf: multiple test_bpf failures using eBPF JIT

https://bugzilla.kernel.org/show_bug.cgi?id=205469

              Bug ID: 205469
             Summary: x86_32: bpf: multiple test_bpf failures using eBPF JIT
             Product: Networking
             Version: 2.5
      Kernel Version: 4.19.81 LTS
            Hardware: i386
                  OS: Linux
                Tree: Mainline
              Status: NEW
            Severity: normal
            Priority: P1
           Component: Other
            Assignee: [email protected]
            Reporter: [email protected]
                  CC: [email protected]
          Regression: No

Created attachment 285829

    --> https://bugzilla.kernel.org/attachment.cgi?id=285829&action=edit

test_bpf failures: kernel 4.19.81/x86_32 (OpenWrt)

Summary:
========

Running the 4.19.81 LTS kernel on QEMU/x86_32, the standard test_bpf.ko testsuite generates multiple errors with the eBPF JIT enabled:

    ...
    test_bpf: #32 JSET jited:1 40 ret 0 != 20 46 FAIL
    test_bpf: #321 LD_IND word positive offset jited:1 ret 0 != -291897430 FAIL
    test_bpf: #322 LD_IND word negative offset jited:1 ret 0 != -1437222042 FAIL
    test_bpf: #323 LD_IND word unaligned (addr & 3 == 2) jited:1 ret 0 !=
-1150890889 FAIL
    test_bpf: #326 LD_IND word positive offset, all ff jited:1 ret 0 != -1 FAIL
    ...
    test_bpf: Summary: 373 PASSED, 5 FAILED, [344/366 JIT'ed]

However, with eBPF JIT disabled (net.core.bpf_jit_enable=0) all tests pass.

Steps to Reproduce:
===================

    # sysctl net.core.bpf_jit_enable=1
    # modprobe test_bpf
    <Kernel log with failures and test summary>

Affected Systems Tested:
========================

    OpenWrt master on QEMU/pc-q35(x86_32) [LTS kernel 4.19.81]

Kernel Logs:
============

Boot log with test results is attached.

Quoted 6 lines

Hi Daniel Borkmann!

I have tested and verified that the report bug has been fixed by commit 711aef1bbf88 ("bpf, x32: Fix bug for BPF_JMP | {BPF_JSGT, BPF_JSLE, BPF_JSLT, BPF_JSGE}")

But that fix hasn't been backport to stable trees, so maybe we should do it:)

Yes, given you have access to a x32 setup and are also able to runtime test the backported JIT changes, please submit it to stable with us in Cc. Thanks Wang!

Best,
Daniel

#4 lore raw
Quoted 78 lines · Daniel Borkmann
Quoted 77 lines · Wang YanQing
Quoted 73 lines · Daniel Borkmann

[ Cc Wang (x86_32 BPF JIT maintainer) ]

Quoted 70 lines · Stephen Hemminger

Begin forwarded message:

Date: Fri, 08 Nov 2019 07:35:59 +0000
From: [email protected]
To: [email protected]
Subject: [Bug 205469] New: x86_32: bpf: multiple test_bpf failures using eBPF JIT

https://bugzilla.kernel.org/show_bug.cgi?id=205469

              Bug ID: 205469
             Summary: x86_32: bpf: multiple test_bpf failures using eBPF JIT
             Product: Networking
             Version: 2.5
      Kernel Version: 4.19.81 LTS
            Hardware: i386
                  OS: Linux
                Tree: Mainline
              Status: NEW
            Severity: normal
            Priority: P1
           Component: Other
            Assignee: [email protected]
            Reporter: [email protected]
                  CC: [email protected]
          Regression: No

Created attachment 285829

    --> https://bugzilla.kernel.org/attachment.cgi?id=285829&action=edit

test_bpf failures: kernel 4.19.81/x86_32 (OpenWrt)

Summary:
========

Running the 4.19.81 LTS kernel on QEMU/x86_32, the standard test_bpf.ko testsuite generates multiple errors with the eBPF JIT enabled:

    ...
    test_bpf: #32 JSET jited:1 40 ret 0 != 20 46 FAIL
    test_bpf: #321 LD_IND word positive offset jited:1 ret 0 != -291897430 FAIL
    test_bpf: #322 LD_IND word negative offset jited:1 ret 0 != -1437222042 FAIL
    test_bpf: #323 LD_IND word unaligned (addr & 3 == 2) jited:1 ret 0 !=
-1150890889 FAIL
    test_bpf: #326 LD_IND word positive offset, all ff jited:1 ret 0 != -1 FAIL
    ...
    test_bpf: Summary: 373 PASSED, 5 FAILED, [344/366 JIT'ed]

However, with eBPF JIT disabled (net.core.bpf_jit_enable=0) all tests pass.

Steps to Reproduce:
===================

    # sysctl net.core.bpf_jit_enable=1
    # modprobe test_bpf
    <Kernel log with failures and test summary>

Affected Systems Tested:
========================

    OpenWrt master on QEMU/pc-q35(x86_32) [LTS kernel 4.19.81]

Kernel Logs:
============

Boot log with test results is attached.

Hi Daniel Borkmann!

Quoted 7 lines
Quoted 4 lines

I have tested and verified that the report bug has been fixed by commit 711aef1bbf88 ("bpf, x32: Fix bug for BPF_JMP | {BPF_JSGT, BPF_JSLE, BPF_JSLT, BPF_JSGE}")

But that fix hasn't been backport to stable trees, so maybe we should do it:)

Yes, given you have access to a x32 setup and are also able to runtime test the backported JIT changes, please submit it to stable with us in Cc. Thanks Wang!

Backporting this would be nice if someone could do it :)

thanks,

greg k-h

maintainer
#5 lore raw
Quoted 82 lines · Greg KH
Quoted 81 lines · Daniel Borkmann
Quoted 80 lines · Wang YanQing
Quoted 73 lines · Daniel Borkmann

[ Cc Wang (x86_32 BPF JIT maintainer) ]

Quoted 70 lines · Stephen Hemminger

Begin forwarded message:

Date: Fri, 08 Nov 2019 07:35:59 +0000
From: [email protected]
To: [email protected]
Subject: [Bug 205469] New: x86_32: bpf: multiple test_bpf failures using eBPF JIT

https://bugzilla.kernel.org/show_bug.cgi?id=205469

               Bug ID: 205469
              Summary: x86_32: bpf: multiple test_bpf failures using eBPF JIT
              Product: Networking
              Version: 2.5
       Kernel Version: 4.19.81 LTS
             Hardware: i386
                   OS: Linux
                 Tree: Mainline
               Status: NEW
             Severity: normal
             Priority: P1
            Component: Other
             Assignee: [email protected]
             Reporter: [email protected]
                   CC: [email protected]
           Regression: No

Created attachment 285829

     --> https://bugzilla.kernel.org/attachment.cgi?id=285829&action=edit

test_bpf failures: kernel 4.19.81/x86_32 (OpenWrt)

Summary:
========

Running the 4.19.81 LTS kernel on QEMU/x86_32, the standard test_bpf.ko testsuite generates multiple errors with the eBPF JIT enabled:

     ...
     test_bpf: #32 JSET jited:1 40 ret 0 != 20 46 FAIL
     test_bpf: #321 LD_IND word positive offset jited:1 ret 0 != -291897430 FAIL
     test_bpf: #322 LD_IND word negative offset jited:1 ret 0 != -1437222042 FAIL
     test_bpf: #323 LD_IND word unaligned (addr & 3 == 2) jited:1 ret 0 !=
-1150890889 FAIL
     test_bpf: #326 LD_IND word positive offset, all ff jited:1 ret 0 != -1 FAIL
     ...
     test_bpf: Summary: 373 PASSED, 5 FAILED, [344/366 JIT'ed]

However, with eBPF JIT disabled (net.core.bpf_jit_enable=0) all tests pass.

Steps to Reproduce:
===================

     # sysctl net.core.bpf_jit_enable=1
     # modprobe test_bpf
     <Kernel log with failures and test summary>

Affected Systems Tested:
========================

     OpenWrt master on QEMU/pc-q35(x86_32) [LTS kernel 4.19.81]

Kernel Logs:
============

Boot log with test results is attached.

Hi Daniel Borkmann!

I have tested and verified that the report bug has been fixed by commit 711aef1bbf88 ("bpf, x32: Fix bug for BPF_JMP | {BPF_JSGT, BPF_JSLE, BPF_JSLT, BPF_JSGE}")

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

But that fix hasn't been backport to stable trees, so maybe we should do it:)

Yes, given you have access to a x32 setup and are also able to runtime test the backported JIT changes, please submit it to stable with us in Cc. Thanks Wang!

Backporting this would be nice if someone could do it :)

It landed on the stable list today:

https://lore.kernel.org/stable/20191121074336.GA15326@udknight/
https://lore.kernel.org/stable/20191121074725.GA15476@udknight/
https://lore.kernel.org/stable/20191121074511.GC15326@udknight/
https://lore.kernel.org/stable/20191121074452.GB15326@udknight/

Thanks,
Daniel

#6 lore raw
Quoted 88 lines · Daniel Borkmann
Quoted 87 lines · Greg KH
Quoted 85 lines · Daniel Borkmann
Quoted 75 lines · Wang YanQing
Quoted 73 lines · Daniel Borkmann

[ Cc Wang (x86_32 BPF JIT maintainer) ]

Quoted 70 lines · Stephen Hemminger

Begin forwarded message:

Date: Fri, 08 Nov 2019 07:35:59 +0000
From: [email protected]
To: [email protected]
Subject: [Bug 205469] New: x86_32: bpf: multiple test_bpf failures using eBPF JIT

https://bugzilla.kernel.org/show_bug.cgi?id=205469

               Bug ID: 205469
              Summary: x86_32: bpf: multiple test_bpf failures using eBPF JIT
              Product: Networking
              Version: 2.5
       Kernel Version: 4.19.81 LTS
             Hardware: i386
                   OS: Linux
                 Tree: Mainline
               Status: NEW
             Severity: normal
             Priority: P1
            Component: Other
             Assignee: [email protected]
             Reporter: [email protected]
                   CC: [email protected]
           Regression: No

Created attachment 285829

     --> https://bugzilla.kernel.org/attachment.cgi?id=285829&action=edit

test_bpf failures: kernel 4.19.81/x86_32 (OpenWrt)

Summary:
========

Running the 4.19.81 LTS kernel on QEMU/x86_32, the standard test_bpf.ko testsuite generates multiple errors with the eBPF JIT enabled:

     ...
     test_bpf: #32 JSET jited:1 40 ret 0 != 20 46 FAIL
     test_bpf: #321 LD_IND word positive offset jited:1 ret 0 != -291897430 FAIL
     test_bpf: #322 LD_IND word negative offset jited:1 ret 0 != -1437222042 FAIL
     test_bpf: #323 LD_IND word unaligned (addr & 3 == 2) jited:1 ret 0 !=
-1150890889 FAIL
     test_bpf: #326 LD_IND word positive offset, all ff jited:1 ret 0 != -1 FAIL
     ...
     test_bpf: Summary: 373 PASSED, 5 FAILED, [344/366 JIT'ed]

However, with eBPF JIT disabled (net.core.bpf_jit_enable=0) all tests pass.

Steps to Reproduce:
===================

     # sysctl net.core.bpf_jit_enable=1
     # modprobe test_bpf
     <Kernel log with failures and test summary>

Affected Systems Tested:
========================

     OpenWrt master on QEMU/pc-q35(x86_32) [LTS kernel 4.19.81]

Kernel Logs:
============

Boot log with test results is attached.

Quoted 6 lines

Hi Daniel Borkmann!

I have tested and verified that the report bug has been fixed by commit 711aef1bbf88 ("bpf, x32: Fix bug for BPF_JMP | {BPF_JSGT, BPF_JSLE, BPF_JSLT, BPF_JSGE}")

But that fix hasn't been backport to stable trees, so maybe we should do it:)

Yes, given you have access to a x32 setup and are also able to runtime test the backported JIT changes, please submit it to stable with us in Cc. Thanks Wang!

Quoted 8 lines
Quoted 1 line

Backporting this would be nice if someone could do it :)

It landed on the stable list today:

https://lore.kernel.org/stable/20191121074336.GA15326@udknight/
https://lore.kernel.org/stable/20191121074725.GA15476@udknight/
https://lore.kernel.org/stable/20191121074511.GC15326@udknight/
https://lore.kernel.org/stable/20191121074452.GB15326@udknight/

Ugh, I totally missed them as they got filed into my bpf@ mailing list mbox. Thanks, I'll go dig them up now!

greg k-h